Random Matrix Theory-Based Reduced-Dimension Space-Time Adaptive Processing under Finite Training Samples

Abstract

Space-time adaptive processing (STAP) is a fundamental topic in airborne radar applications due to its clutter suppression ability. Reduced-dimension (RD)-STAP can release the requirement of the number of training samples and reduce the computational load from traditional STAP, which attracts much attention. However, under the situation that training samples are severely deficient, RD-STAP will become poor like the traditional STAP. To enhance RD-STAP performance in such cases, this paper develops a novel RD-STAP algorithm using random matrix theory (RMT), RMT-RD-STAP. By minimizing the output clutter-plus-noise power, the estimate of the inversion of clutter plus noise covariance matrix (CNCM) can be obtained through optimally manipulating its eigenvalues, thus producing the optimal STAP weight vector. Specifically, the clutter-related eigenvalues are estimated according to the clutter-related sample eigenvalues via RMT, and the noise-related eigenvalue is optimally selected from the noise-related sample eigenvalues. It is found that RMT-RD-STAP significantly outperforms the RD-STAP algorithm when the RMB rule cannot be satisfied. Theoretical analyses and numerical results demonstrate the effectiveness and the performance advantages of the proposed RMT-RD-STAP algorithm.

1. Introduction

Space-time adaptive processing (STAP) was first proposed almost 50 years ago [1], and since then it has been actively investigated by the radar community due to its strong capability in suppressing clutter and enhancing target [2,3,4,5,6]. The optimal STAP can maximize the output signal-to-interference plus noise ratio (SINR) and provide reliable target detection. However, the optimal STAP involves an ideal clutter plus noise covariance matrix (CNCM), which generally cannot be attained and must be estimated with independent identical distributed (IID) training samples, called sample covariance matrix (SCM). Once the number of training samples for SCM is insufficient, STAP would severely suffer from performance deterioration. Particularly, the famous Reed, Mallet, and Brennan (RMB) rule [4] provides an explicit theoretical guide for the requirement on the number of training samples, i.e., to limit the SINR loss to 3 dB, the number of training samples must be not less than two times the system degrees of freedom (DoFs). However, in practice, the RMB rule is commonly hard to satisfy. In addition, the computational load in optimal STAP is extremely huge owing to high-dimensional matrix inversion. Therefore, the traditional STAP needs to be improved.

With the development of STAP, a great number of methods have been developed to confront the performance deterioration stemming from finite training samples. Sparsity-recovery (SR)-STAP [7,8,9,10] can greatly improve the SCM with only a few or even one training sample, however, the computational load involved in SR is unbearable. Knowledge-aided (KA)-STAP [11,12,13,14] uses the prior knowledge to improve SCM, i.e., covariance matrix and terrain data, but its performance will severely deteriorate once the prior knowledge deviates from the actual one. Recently, deep convolutional neural networks (CNNs) [15,16,17] have been used for STAP also with few training samples, but it usually needs a lot of sampling data as the training set for network learning, which is unavailable in many radar scenarios. Moreover, it is hard to obtain the ground truth in actual clutter scenarios, which is necessarily required as a label for supervised learning.

Reduced-dimension (RD)-STAP [18,19,20,21,22,23] is one of the most popular methods due to its fast convergence, simple structure, and a low requirement for training samples, which has been widely used in engineering applications. RD-STAP algorithms reduce the system DoFs through linear transformation on training samples to satisfy the RMB rule. Meanwhile, the lower system DoFs cause lower computational load, which significantly boosts real-time processing for airborne radar. In terms of different principles, several linear transformations have been constructed, such as the extended factored approach (EFA) [18] and joint domain localized (JDL) [23]. However, in a radar system, the linear transformation is often fixed in a predesigned clutter scenario. When the scenarios deviate from the predesigned ones, the RMB rule may not be satisfied and the RD-STAP performance becomes poor like the traditional STAP. Therefore, it is greatly meaningful to resort to other techniques to enhance RD-STAP performance.

It could be noticed that CNCM is composed of eigenvalues and eigenvectors, and a lot of works have demonstrated that the SCM can be improved by adjusting the sample eigenvalues while maintaining the sample eigenvectors [24,25,26,27]. Currently, random matrix theory (RMT), originating from quantum mechanics [28] and mainly used to study the asymptotic behavior of the empirical spectral distribution of different random matrix models as their dimensions go into infinity [29,30,31,32], has been applied to estimate the eigenvalues and eigenvectors of the SCM constructed from the finite observations [33,34,35,36]. It is found that the estimation is consistent, not only when the sample size increases without bound for a fixed observation dimension but also when the observation dimension increases to infinity at the same rate as the sample size increases [37,38,39]. Particularly, the spiked covariance model, defined as a low-rank perturbation of an identity matrix in RMT [40], attracts much attention since it is similar to the classical signal-plus-noise model in signal processing. It is shown in [40,41,42,43] that for such a model, consistent estimation of its eigenvalues can be obtained with small-size samples. Recently, ref. [27] applies the theory to directly estimate the inverse covariance matrix for spatial beamforming and shows superior performance under high dimension and finite training samples. Aiming at the STAP problem, some works, like [44,45], use RMT to enhance the performance under finite training samples. However, these methods are implemented in full-dimension STAP, which leads to a huge computational load and is not applicable to RD-STAP. Based on this consideration, we try to improve the performance of RD-STAP by using RMT.

Inspired by [27], this paper proposes a new RD-STAP algorithm using RMT, called RMT-RD-STAP. Like the CNCM’s eigenvalues, the RD CNCM’s eigenvalues consist of two parts, the noise-related part, and the clutter-related part. The noise-related part contains the large portion of the RD CNCM’s eigenvalues and their values are determined by the noise power, whereas the clutter-related part takes up the small portion of the RD CNCM’s eigenvalues and their values depend on the clutter power. The eigenvalues belonging to the clutter-related part are commonly much larger than those belonging to the noise-related part. Such eigenvalue distribution is similar to that of a spiked covariance model. The main difference is that the noise covariance matrix in RD CNCM is not strictly a scaled identity matrix which is required in the spiked covariance model. If the noise covariance matrix in RD CNCM can be considered a scaled identity matrix, the RMT-based technique in [27] is applicable to RD STAP with finite training samples and the performance of clutter suppression will be enhanced. In this paper, we deal with the noise covariance matrix in RD CNCM as a scaled identity matrix, and fortunately, we find that the spiked covariance model can effectively work for RD-STAP with finite training samples. We first construct a new estimate on the inverse RD CNCM by employing the eigenvectors of the RD SCM and then estimate the clutter-related eigenvalues by using the RMT. Meanwhile, we provide the estimate on the scaled factor of the scaled identity matrix according to the eigenvalues of the RD SCM. With the proposed estimate on the inverse RD CNCM, we can directly obtain the space-time adaptive weight vector of RMT-RD-STAP in terms of the linear constraint minimum variance (LCMV) criterion. We conduct extensive numerical experiments to verify the effectiveness of the RMT-RD-STAP, and we find that the proposed RMT-RD-STAP greatly outperforms the conventional RD-STAP in the case of finite training samples.

The rest of this paper is organized as follows. Section 2 introduces the echo model for airborne radar and the fundamentals of STAP and RD-STAP. Section 3 presents the proposed RMT-RD-STAP algorithm in detail. Numerical experiments are performed in Section 4 and the conclusions of this work are drawn in Section 5.

Notation: ${\mathbf{I}}_N$ denotes the $N\times N$ identity matrix. Boldface uppercase letters denote the matrices, and boldface lowercase letters denote the vectors. $j=\sqrt{-1}$. The $\otimes$ represents the Kronecker product. The ${(\cdot )}^{\mathrm{T}}$ and ${(\cdot )}^{\mathrm{H}}$ represent the transpose and Hermitian transpose, respectively. The $ℂ$ represents the sets of complex values. The $\mathbb{E}\left\{\cdot \right\}$ denotes the statistical expectation.

2. Background

2.1. Echo Model of Airborne Radar and Optimal STAP

As shown in Figure 1, consider a side-looking airborne pulse-Doppler radar system equipped with a uniform linear array (ULA) of $N$ array elements with the inter-element spacing $\Delta =\lambda /2$ and the wavelength $\lambda$. Let $\mathrm{H}$ and $V$ denote the height and velocity of the platform, respectively, and $\psi$, $\theta$, and $\phi$ denote the cone, azimuth, and elevation angle, respectively. Assume that the radar transmits $K$ pulses at a constant pulse repetition frequency (PRF) $f_r$ during a coherent processing interval (CPI). Then under the assumption of no range ambiguity and no internal clutter motion, the received signal ${\mathit{x}}_l\in {ℂ}^{NK}$ of the $l$-th range cell or the range cell under test (CUT) can be represented as

$${\mathit{x}}_l=\mathit{s}+{\mathit{c}}_l+\mathit{n},$$

(1)

where $\mathit{s}\in {ℂ}^{NK}$ denotes target echo, $\mathit{n}\sim \mathcal{C}\mathcal{N}\left(0,{\sigma }_n^2{\mathbf{I}}_{NK}\right)$ denotes the complex white Gaussian noise with the noise power ${\sigma }_n^2$, and ${\mathit{c}}_l\in {ℂ}^{NK}$ denotes the clutter echo,

$${\mathit{c}}_l={\displaystyle \sum _{i=1}^{N_c}{\Gamma }_i\mathit{a}\left(f_{ci}^t,f_{ci}^s\right)}.$$

(2)

In (2), $N_c$ is the number of clutter patches evenly divided in azimuth, ${\Gamma }_i$ denotes the complex amplitude of the $i$-th clutter patch, and $\mathit{a}\left(f_{ci}^t,f_{ci}^s\right)={\mathit{a}}_t\left(f_{ci}^t\right)\otimes {\mathit{a}}_s\left(f_{ci}^s\right)\in {ℂ}^{NK}$ denotes the normalized spatial-temporal steering vector of the $i$-th clutter patch where ${\mathit{a}}_t\left(f_{ci}^{\mathit{t}}\right)\in {ℂ}^K$, and ${\mathit{a}}_s\left(f_{ci}^s\right)\in {ℂ}^N$ respectively denote the temporal and spatial steering vectors as

$$\begin{array}{l}{\mathit{a}}_t\left(f_{ci}^t\right)={\left[1,e^{j2\pi f_{ci}^t},\cdots ,e^{j2\pi f_{ci}^t(K-1)}\right]}^{\mathrm{T}}\\ {\mathit{a}}_s\left(f_{ci}^s\right)={\left[1,e^{j2\pi f_{ci}^s},\cdots ,e^{j2\pi f_{ci}^s(N-1)}\right]}^{\mathrm{T}}\end{array}$$

(3)

In (3), $f_{ci}^t$ and $f_{ci}^s$ denote the normalized temporal and spatial frequency, respectively,

$$\begin{array}{c}{f}_{ci}^t=\frac{2V}{\lambda f_r}\cos {\psi }_i=\frac{2V}{\lambda f_r}\cos {\theta }_i\cos {\phi }_l\\ f_{ci}^s=\frac{\Delta }{\lambda }\cos {\psi }_i=\frac{\Delta }{\lambda }\cos {\theta }_i\cos {\phi }_l\end{array}$$

(4)

Then the CNCM $\mathit{R}$ of the $l$-th range cell can be expressed from (1) as

$$\mathit{R}=\mathbb{E}\left({\mathit{x}}_l{\mathit{x}}_l^{\mathrm{H}}\right)={\displaystyle \sum _{i=1}^{N_c}{\left|{\Gamma }_i\right|}^2\mathit{a}\left(f_{ci}^t,f_{ci}^s\right){\mathit{a}}^{\mathrm{H}}\left(f_{ci}^t,f_{ci}^s\right)}+{\sigma }_n^2{\mathbf{I}}_{NK}.$$

(5)

Under the LCMV criterion [2], the optimal STAP is defined as

$$\left\{\begin{array}{l}\min P\left(\mathit{w}\right)={\mathit{w}}^{\mathrm{H}}\mathit{R}\mathit{w}\\ \mathrm{s}.\mathrm{t}. {\mathit{w}}^{\mathrm{H}}\mathit{a}\left(f_0^t,f_0^s\right)=1\end{array}\right.,$$

(6)

where $P\left(\mathit{w}\right)$ denotes the output clutter-plus-noise power, $\mathit{a}\left(f_0^t,f_0^s\right)\in {ℂ}^{NK}$ denotes the spatial-temporal steering vector of the target with $f_0^t$ and $f_0^s$ as the normalized temporal and spatial frequency. The optimal STAP weight vector $\mathit{w}\in {ℂ}^{NK}$ can be derived as

$${\mathit{w}}^{opt}=\frac{{\mathit{R}}^{-1}\mathit{a}\left(f_0^t,f_0^s\right)}{{\mathit{a}}^{\mathrm{H}}\left(f_0^t,f_0^s\right){\mathit{R}}^{-1}\mathit{a}\left(f_0^t,f_0^s\right)},$$

(7)

from which, the minimal clutter-plus-noise power is given by

$$P\left({\mathit{w}}^{opt}\right)=\frac{1}{{\mathit{a}}^{\mathrm{H}}\left(f_0^t,f_0^s\right){\mathit{R}}^{-1}\mathit{a}\left(f_0^t,f_0^s\right)}.$$

(8)

2.2. Sample-Based STAP and RD-STAP

For the implementation of (7), it is fundamental to know the CNCM $\mathit{R}$ and its inversion. However, it is not practical to obtain the matrix $\mathit{R}$. In practice, the matrix $\mathit{R}$ is estimated from finite echo samples or training samples,

$${\mathit{R}}_L={\displaystyle \sum _{l=1}^L{\mathit{x}}_l{\mathit{x}}_l^{\mathrm{H}}},$$

(9)

where ${\mathit{R}}_L$ is called the SCM with $L$ training samples. In such case, the optimal weight vector (7) is approximated as

$${\mathit{w}}_L^{opt}=\frac{{\mathit{R}}_L^{-1}\mathit{a}\left(f_0^t,f_0^s\right)}{{\mathit{a}}^{\mathrm{H}}\left(f_0^t,f_0^s\right){\mathit{R}}_L^{-1}\mathit{a}\left(f_0^t,f_0^s\right)},$$

(10)

where ${\mathit{w}}_L^{opt}$ is called the adaptive weight vector of the traditional STAP. The output clutter-plus-noise power by the weight vector (10) is given by

$$P\left({\mathit{w}}_L^{opt}\right)=\frac{{\mathit{a}}^{\mathrm{H}}\left(f_0^t,f_0^s\right){\mathit{R}}_L^{-1}\mathit{R}{\mathit{R}}_L^{-1}\mathit{a}\left(f_0^t,f_0^s\right)}{{\left[{\mathit{a}}^{\mathrm{H}}\left(f_0^t,f_0^s\right){\mathit{R}}_L^{-1}\mathit{a}\left(f_0^t,f_0^s\right)\right]}^2}.$$

(11)

It is seen from (11) that the traditional STAP will have good performance if ${\mathit{R}}_L^{-1}$ well approximates ${\mathit{R}}^{-1}$. However, ${\mathit{R}}_L^{-1}$ is estimated from finite training samples and deviates from ${\mathit{R}}^{-1}$ if the number of the training samples decreases. This deviation will result in a higher output clutter-plus-noise power by (11) than the theoretical one by (8). Define a constant $c\in (0,1)$. Then from [46,47], it is known that $P\left({\mathit{w}}_L^{opt}\right)/P\left({\mathit{w}}^{opt}\right)\to 1/\left(1-c\right)$ as $NK,L\to \infty$ with $c_N=NK/L\to c$, which coincides with the RMB rule [4], i.e., the sample-based STAP performance by (10) has less than 3 dB loss when the number of training samples is not less than two times the system DoFs, $L\ge 2NK$. However, in real radar scenarios, it is a common case that $L<NK$ [2]. The sample-based STAP by (10) will have much poorer performance than the optimal STAP by (7).

To reduce the requirement of (10) on the number of training samples, RD-STAP [18,19,20,21,22,23] is developed, which produces a secondary data by projecting the $NK$-dimensional signal vector ${\mathit{x}}_l$ into a $M$-dimensional one via linear transformation matrix $\mathit{T}\in {ℂ}^{NK\times M}$, namely

$${\tilde{\mathit{x}}}_l={\mathit{T}}^{\mathrm{H}}{\mathit{x}}_l\in {ℂ}^M,$$

(12)

where $M$ denotes the reduced-dimensional system DoFs and is much smaller than the original system DoFs, $M\ll NK$. Then, it can be predicted that the STAP with the data in (12) will have good performance when $L\ge 2M$ in terms of the RMB rule.

The RD CNCM and the corresponding target spatial-temporal steering vector are given as

$${\mathit{R}}_{rd}=\mathbb{E}\left({\tilde{\mathit{x}}}_l{\tilde{\mathit{x}}}_l^{\mathrm{H}}\right)={\mathit{T}}^{\mathrm{H}}\mathit{R}\mathit{T}\in {ℂ}^{M\times M},$$

(13)

and

$${\mathit{a}}_{rd}\left(f_0^t,f_0^s\right)={\mathit{T}}^{\mathrm{H}}\mathit{a}\left(f_0^t,f_0^s\right)\in {ℂ}^M.$$

(14)

Under the LCMV criterion, the RD-STAP is defined as

$$\left\{\begin{array}{l}\min P_{rd}\left({\mathit{w}}_{rd}\right)={\mathit{w}}_{rd}^{\mathrm{H}}{\mathit{R}}_{rd}{\mathit{w}}_{rd}\\ \mathrm{s}.\mathrm{t}. {\mathit{w}}_{rd}^{\mathrm{H}}{\mathit{a}}_{rd}\left(f_0^t,f_0^s\right)=1\end{array}\right.,$$

(15)

where ${\mathit{w}}_{rd}\in {ℂ}^M$ denotes the RD-STAP weight vector. Then the optimal RD-STAP weight vector is given as

$${\mathit{w}}_{rd}^{op\mathit{t}}=\frac{{\mathit{R}}_{rd}^{-1}{\mathit{a}}_{rd}\left(f_0^{\mathit{t}},f_0^s\right)}{{\mathit{a}}_{rd}^{\mathrm{H}}\left(f_0^{\mathit{t}},f_0^s\right){\mathit{R}}_{rd}^{-1}{\mathit{a}}_{rd}\left(f_0^{\mathit{t}},f_0^s\right)}.$$

(16)

When the ${\mathit{R}}_{rd}$ is generated from the finite samples, the adaptive weight vector of the sample-based RD-STAP is

$${\mathit{w}}_{Lrd}^{op\mathit{t}}=\frac{{\mathit{R}}_{Lrd}^{-1}{\mathit{a}}_{rd}\left(f_0^{\mathit{t}},f_0^s\right)}{{\mathit{a}}_{rd}^{\mathrm{H}}\left(f_0^{\mathit{t}},f_0^s\right){\mathit{R}}_L{}_{rd}^{-1}{\mathit{a}}_{rd}\left(f_0^{\mathit{t}},f_0^s\right)},$$

(17)

where ${\mathit{R}}_{Lrd}={\mathit{T}}^{\mathrm{H}}{\mathit{R}}_L\mathit{T}\in {ℂ}^{M\times M}$ is the RD SCM. Similar to (11), the output clutter-plus-noise power associated with the weight vector (17) is given as

$$P_{rd}\left({\mathit{w}}_{Lrd}^{op\mathit{t}}\right)=\frac{{\mathit{a}}_{rd}^{\mathrm{H}}\left(f_0^{\mathit{t}},f_0^s\right){\mathit{R}}_{Lrd}^{-1}{\mathit{R}}_{rd}{\mathit{R}}_{Lrd}^{-1}{\mathit{a}}_{rd}\left(f_0^{\mathit{t}},f_0^s\right)}{{\left[{\mathit{a}}_{rd}^{\mathrm{H}}\left(f_0^{\mathit{t}},f_0^s\right){\mathit{R}}_{Lrd}^{-1}{\mathit{a}}_{rd}\left(f_0^{\mathit{t}},f_0^s\right)\right]}^2}.$$

(18)

2.3. Motivation by Introducing RMT in RD-STAP

Although RD-STAP can release the requirement on the number of training samples, the RD processing structure is commonly fixed for a radar system, then the RD-STAP performance becomes poor like the traditional STAP once the number of training samples is small. Therefore, it is greatly necessary to resort to other techniques to improve the RD-STAP performance. In this paper, we study the consistent estimate of ${\mathit{R}}_{rd}^{-1}$ from the finite training samples, especially in the case of $L\le 2M$.

For the inverse matrix ${\mathit{R}}_{rd}^{-1}$, let us consider the eigen-decomposition of the matrix ${\mathit{R}}_{rd}$ as

$${\mathit{R}}_{rd}={\tilde{\sigma }}_n^2{\displaystyle \sum _{i=1}^M{\gamma }_i{\mathbf{e}}_i{\mathbf{e}}_i^{\mathrm{H}}}={\tilde{\sigma }}_n^2{\displaystyle \sum _{i=1}^{Q_{rd}}{\gamma }_i{\mathbf{e}}_i{\mathbf{e}}_i^{\mathrm{H}}}+{\tilde{\sigma }}_n^2{\displaystyle \sum _{i=Q_{rd}+1}^M{\gamma }_i{\mathbf{e}}_i{\mathbf{e}}_i^{\mathrm{H}}},$$

(19)

where $Q_{rd}$ denotes the local clutter DoFs, which can be calculated by the method in [48], ${\mathbf{e}}_i$ is the $i$-th eigenvector with the corresponding eigenvalue ${\tilde{\sigma }}_n^2{\gamma }_i$ and ${\gamma }_1\ge {\gamma }_2\ge \cdots \ge {\gamma }_{Q_{rd}}\gg {\gamma }_{Q_{rd}+1}\ge \cdots \ge {\gamma }_M$. The leading eigenvalues ${\gamma }_i$ $\left(i=1,\cdots ,Q_{rd}\right)$ are related to the clutter, whereas the others are related to the noise.

Notice that $Q_{rd}$ is generally far less than the RD system DoFs $M$ in practice [48], and the clutter-related eigenvalues are much larger than the noise-related ones. Such eigenvalue distribution is like that of a spiked covariance model although it is not strict. In the area of RMT, this model has acquired extensive study on its statistical behaviors and the consistent estimation theory has been established on estimating the isolated eigenvalues of the spiked covariance model from finite training samples [38,39,40,41]. In particular, the spike covariance matrix estimation can be improved by managing the sample eigenvalues while keeping the sample eigenvectors [24,25,26,27]. Then, this theory can be leveraged to improve the estimation of ${\mathit{R}}_{rd}^{-1}$ and in turn enhance the RD-STAP performance.

3. RD-STAP Using RMT

In this section, we propose a novel RD-STAP algorithm by using the RMT, called RMT-RD-STAP. Firstly, the RMT-RD-STAP problem with the spiked covariance model is defined. Then, its asymptotic deterministic equivalence is given and the optimal solution is derived; ultimately an estimated adaptive weight vector is optimized.

3.1. RD-STAP Problem under the Spiked Covariance Model

It is seen from (18) that the ${\mathit{R}}_{rd}$ does not have the spiked structure as $\mathit{R}$ because the noise covariance matrix is not a scaled identity one [48]. Then the RMT-based technique in [27] cannot be directly applicable to the estimate of the inverse CNCM ${\mathit{R}}_{rd}^{-1}$. Note that the optimal RD-STAP is to minimize the output clutter-plus-noise power. Then the essence of the sample-based RD-STAP is to find the estimate of ${\mathit{R}}_{rd}^{-1}$ which can reduce the output clutter-plus-noise power (17). We will find that the output clutter-plus-noise power in (17) can be reduced if the noise-related covariance matrix is replaced by a scaled identity matrix. Although there are no theoretical guarantees, a number of numerical experiments in Section 4 have confirmed the assertion. Then we make Assumption 1 as follows.

Assumption 1.

The noise component in (12) is taken to be the complex white Gaussian noise with zero mean and covariance $\sigma {\mathbf{I}}_M$.

Define ${\beta }_i={\gamma }_i-1$. Then under Assumption 1, the RD CNCM ${\mathit{R}}_{rd}$ in (19) can be approximated as

$${\mathit{R}}_{rd}\approx \sigma \left({\displaystyle \sum _{i=1}^{Q_{rd}}{\beta }_i{\mathbf{e}}_i{\mathbf{e}}_i^{\mathrm{H}}}+{\mathbf{I}}_M\right),$$

(20)

where the noise power $\sigma$ can be cosidered as the scaled factor of the identity matrix in the spiked covariance model. With the eigen-decomposition of the RD SCM ${\mathit{R}}_{Lrd}$ as

$${\mathit{R}}_{Lrd}=\sigma {\displaystyle \sum _{i=1}^M{\alpha }_i{\mathbf{d}}_i{\mathbf{d}}_i^{\mathrm{H}}},$$

(21)

in which ${\mathbf{d}}_i$ is the $i$-th eigenvector with the corresponding eigenvalue $\sigma {\alpha }_i$ and ${\alpha }_1\ge \cdots \ge {\alpha }_M$, according to the spike covariance matrix estimation principle [25], an inverse RD CNCM ${\overline{\mathit{R}}}_{rd}^{-1}$ can be constructed as

$${\overline{\mathit{R}}}_{rd}^{-1}=\frac{1}{\sigma }{\displaystyle \sum _{i=1}^M{\varsigma }_i{\mathbf{d}}_i{\mathbf{d}}_i^{\mathrm{H}}},$$

(22)

where $1/\sigma {\varsigma }_i$ are the eigenvalues of ${\overline{\mathit{R}}}_{rd}^{-1}$ and the parameters ${\varsigma }_i$ need to be determined. By using Assumption 1, setting ${\varsigma }_{Q_{rd}+1}=\cdots ={\varsigma }_M=1$, and letting $g_i={\varsigma }_i-1$, (22) can be re-expressed as

$${\overline{\mathit{R}}}_{rd}^{-1}(\mathbf{g})=\frac{1}{\sigma }\left({\displaystyle \sum _{i=1}^{Q_{rd}}{g}_i{\mathbf{d}}_i{\mathbf{d}}_i^{\mathrm{H}}}+{\mathbf{I}}_M\right),$$

(23)

where the inverse matrix ${\overline{\mathit{R}}}_{rd}^{-1}$ is implicitly expressed as a function of the vector $\mathbf{g}={[g_1\cdots g_{Q_{rd}}]}^{\mathrm{T}}$. Correspondingly, the adaptive weight vector ${\overline{\mathit{w}}}_{rd}^{op\mathit{t}}$ is given as

$${\overline{\mathit{w}}}_{rd}^{op\mathit{t}}\left(\mathbf{g}\right)=\frac{{\overline{\mathit{R}}}_{rd}^{-1}\left(\mathbf{g}\right){\mathit{a}}_{rd}\left(f_0^{\mathit{t}},f_0^s\right)}{{\mathit{a}}_{rd}^{\mathrm{H}}\left(f_0^{\mathit{t}},f_0^s\right){\overline{\mathit{R}}}_{rd}^{-1}\left(\mathbf{g}\right){\mathit{a}}_{rd}\left(f_0^{\mathit{t}},f_0^s\right)}.$$

(24)

Then the RD-STAP problem under the spiked covariance model can be defined as

$${\mathbf{g}}^{\ast }=\underset{\mathbf{g}}{\mathrm{argmin}}{P}_{rd}\left({\overline{\mathit{w}}}_{rd}^{opt}\left(\mathbf{g}\right)\right),$$

(25)

where

$$\begin{array}{cc}{P}_{rd}\left({\overline{\mathit{w}}}_{rd}^{opt}\left(\mathbf{g}\right)\right)\hfill & =\frac{{\mathit{a}}_{rd}^{\mathrm{H}}\left(f_0^{\mathit{t}},f_0^s\right){\overline{\mathit{R}}}_{rd}^{-1}\left(\mathbf{g}\right){\mathit{R}}_{rd}{\overline{\mathit{R}}}_{rd}^{-1}\left(\mathbf{g}\right){\mathit{a}}_{rd}\left(f_0^{\mathit{t}},f_0^s\right)}{{\left[{\mathit{a}}_{rd}^{\mathrm{H}}\left(f_0^{\mathit{t}},f_0^s\right){\overline{\mathit{R}}}_{rd}^{-1}\left(\mathbf{g}\right){\mathit{a}}_{rd}\left(f_0^{\mathit{t}},f_0^s\right)\right]}^2}\hfill \\ \hfill & =\sigma \frac{{\mathit{a}}_{rd}^{\mathrm{H}}\left(f_0^t,f_0^s\right)\left({\mathbf{I}}_M+{\displaystyle \sum _{q=1}^{Q_{rd}}{g}_q{\mathbf{d}}_q{\mathbf{d}}_q^{\mathrm{H}}}\right)\left({\mathbf{I}}_M+{\displaystyle \sum _{j=1}^{Q_{rd}}{\beta }_j{\mathbf{e}}_j{\mathbf{e}}_j^{\mathrm{H}}}\right)\left({\mathbf{I}}_M+{\displaystyle \sum _{i=1}^{Q_{rd}}{g}_i{\mathbf{d}}_i{\mathbf{d}}_i^{\mathrm{H}}}\right){\mathit{a}}_{rd}\left(f_0^t,f_0^s\right)}{{\left[{\mathit{a}}_{rd}^{\mathrm{H}}\left(f_0^t,f_0^s\right)\left({\mathbf{I}}_M+{\displaystyle \sum _{i=1}^{Q_{rd}}{g}_i{\mathbf{d}}_i{\mathbf{d}}_i^{\mathrm{H}}}\right){\mathit{a}}_{rd}\left(f_0^t,f_0^s\right)\right]}^2}\hfill \end{array}$$

(26)

The optimal RMT-RD-STAP is equivalent to finding the optimal ${\mathbf{g}}^{\ast }$.

3.2. Asymptotic Deterministic Equivalent ${\tilde{P}}_{rd}\left({\overline{\mathit{w}}}_{rd}^{opt}\left(\mathbf{g}\right)\right)$ and the Optimal ${\tilde{\mathbf{g}}}^{\ast }$

Note that it is difficult to give a closed solution of (25). To address the issue, the asymptotic properties of (26) under $M,L\to \infty$ are considered. To conduct such an analysis, three assumptions are made as follows:

Assumption 2.

As $M,L\to \infty$, $M/L=c_{Nrd}\to c$for a certain $c>0$.

Assumption 3.

The number of$Q_{rd}$is fixed and smaller than$M$.

Assumption 4.

${\beta }_1\ge \cdots \ge {\beta }_{Q_{rd}}>\sqrt{c}$and ${\alpha }_1\ge \cdots \ge {\alpha }_{Q_{\mathit{e}d}}>{\left(1+\sqrt{c}\right)}^2$.

For Assumption 2, both $M$ and $L$ are assumed to be reasonably large. When $c>1/2$, the number of training samples is less than the system DoFs, i.e., $L\le 2M$, the case considered in this paper. Assumption 3 is often taken in the airborne phased-array radar system and $Q_{rd}<M$ [2,4]. The assumption ${\beta }_1\ge \cdots \ge {\beta }_{Q_{rd}}>\sqrt{c}$ in Assumption 4 is easily satisfied in practice because the clutter power is far higher than the noise power [2,4]. For ${\alpha }_1\ge \cdots \ge {\alpha }_{Q_{rd}}>{\left(1+\sqrt{c}\right)}^2$, the number $L$ of the training samples is not less than the local clutter DoFs, $L\ge Q_{rd}$. Otherwise ${\alpha }_1=\cdots ={\alpha }_{Q_{rd}}=0<{\left(1+\sqrt{c}\right)}^2$, and ${\beta }_i$ cannot be uniquely determined from ${\alpha }_i$ [40,41,42,43], leading to the impossibility of recovering the matrix ${\mathit{R}}_{rd}$.

The three assumptions follow [49,50], which are fundamental to deriving the asymptotic properties of (26), as discussed in [27,51]. Then it can be shown that

$$\begin{array}{l}\left|{\mathbf{e}}_i^{\mathrm{H}}{\mathbf{d}}_j{\mathbf{d}}_j^{\mathrm{H}}{\mathbf{e}}_i-s_{rdi}{\delta }_{ij}\right|\stackrel{a.s.}{\to }0\hspace{1em}i,j=1,\dots ,Q_{rd}\\ \left|{\mathit{a}}_{rd}^{\mathrm{H}}\left(f_0^{\mathit{t}},f_0^s\right){\mathbf{d}}_i{\mathbf{d}}_i^{\mathrm{H}}{\mathit{a}}_{rd}\left(f_0^{\mathit{t}},f_0^s\right)-s_{rdi}{k}_{rdi}\right|\stackrel{a.s.}{\to }0\\ \left|{\mathit{a}}_{rd}^{\mathrm{H}}\left(f_0^{\mathit{t}},f_0^s\right){\mathbf{d}}_i{\mathbf{d}}_i^{\mathrm{H}}{\mathbf{e}}_i{\mathbf{e}}_i^{\mathrm{H}}{\mathit{a}}_{rd}\left(f_0^{\mathit{t}},f_0^s\right)-s_{rdi}{k}_{rdi}\right|\stackrel{a.s.}{\to }0\\ \left|{\mathit{a}}_{rd}^{\mathrm{H}}\left(f_0^{\mathit{t}},f_0^s\right){\mathbf{d}}_i{\mathbf{d}}_i^{\mathrm{H}}{\mathbf{e}}_i{\mathbf{e}}_i^{\mathrm{H}}{\mathbf{d}}_i{\mathbf{d}}_i^{\mathrm{H}}{\mathit{a}}_{rd}\left(f_0^{\mathit{t}},f_0^s\right)-s_{rdi}^2{k}_{rdi}\right|\stackrel{a.s.}{\to }0\end{array}$$

(27)

where ${\delta }_{ij}$ is the Kronecker delta function, $s_{rdi}=\left(1-c_{Nrd}/{\beta }_i^2\right)/\left(1+c_{Nrd}/{\beta }_i\right)$, and $k_{rdi}={\mathit{a}}_{rd}^{\mathrm{H}}\left(f_0^{\mathit{t}},f_0^s\right){\mathbf{e}}_i{\mathbf{e}}_i^{\mathrm{H}}{\mathit{a}}_{rd}\left(f_0^{\mathit{t}},f_0^s\right)$. In terms of (27), $P_{rd}\left({\overline{\mathit{w}}}_{rd}^{opt}\left(\mathbf{g}\right)\right)$ has its asymptotic deterministic equivalent ${\tilde{P}}_{rd}\left({\overline{\mathit{w}}}_{rd}^{opt}\left(\mathbf{g}\right)\right)$ as

$${\tilde{P}}_{rd}\left({\overline{\mathit{w}}}_{rd}^{op\mathit{t}}\left(\mathbf{g}\right)\right)=\frac{\sigma \left({\mathit{a}}_{rd}^{\mathrm{H}}\left(f_0^{\mathit{t}},f_0^s\right){\mathit{a}}_{rd}\left(f_0^{\mathit{t}},f_0^s\right)+2{\displaystyle \sum _{i=1}^{Q_{rd}}{g}_i{s}_{rdi}{k}_{rdi}}+{\displaystyle \sum _{i=1}^{Q_{rd}}{k}_{rdi}{\beta }_i}+2{\displaystyle \sum _{i=1}^{Q_{rd}}{g}_i{\beta }_i{s}_{rdi}{k}_{rdi}}+{\displaystyle \sum _{i=1}^{Q_{rd}}{g}_i^2{s}_{rdi}{k}_{rdi}}+{\displaystyle \sum _{i=1}^{Q_{rd}}{g}_i^2{\beta }_i{s}_{rdi}^2{k}_{rdi}}\right)}{{\left[{\mathit{a}}_{rd}^{\mathrm{H}}\left(f_0^{\mathit{t}},f_0^s\right){\mathit{a}}_{rd}\left(f_0^{\mathit{t}},f_0^s\right)+{\displaystyle \sum _{i=1}^{Q_{rd}}{g}_i{s}_{rdi}{k}_{rdi}}\right]}^2},$$

(28)

and

$$\left|P_{rd}\left({\overline{\mathit{w}}}_{rd}^{op\mathit{t}}\left(\mathbf{g}\right)\right)-{\tilde{P}}_{rd}\left({\overline{\mathit{w}}}_{rd}^{op\mathit{t}}\left(\mathbf{g}\right)\right)\right|\stackrel{\mathit{a}.s.}{\to }0.$$

(29)

By replacing $P_{rd}\left({\overline{\mathit{w}}}_{rd}^{op\mathit{t}}\left(\mathbf{g}\right)\right)$ with ${\tilde{P}}_{rd}\left({\overline{\mathit{w}}}_{rd}^{op\mathit{t}}\left(\mathbf{g}\right)\right)$ in (25), we have the optimal ${\tilde{\mathbf{g}}}^{\ast }={\left[{\tilde{g}}_1^{\ast },\cdots ,{\tilde{g}}_{Q_{rd}}^{\ast }\right]}^{\mathrm{T}}$ by minimizing (28) as

$${\tilde{g}}_i^{\ast }=\frac{{\beta }_i+c_{Nrd}}{{\beta }_i^2+{\beta }_i}\left[\frac{{\displaystyle \sum _{q=1}^{Q_{rd}}\frac{c_{Nrd}{k}_{rdq}}{{\widehat{\beta }}_q}}}{{\mathit{a}}_{rd}^{\mathrm{H}}\left(f_0^{\mathit{t}},f_0^s\right){\mathit{a}}_{rd}\left(f_0^{\mathit{t}},f_0^s\right)-{\displaystyle \sum _{q=1}^{Q_{rd}}{k}_{rdq}}+{\displaystyle \sum _{q=1}^{Q_{rd}}\frac{c_{Nrd}{k}_{rdq}}{{\beta }_q^2}}}-{\beta }_i\right].$$

(30)

3.3. Obtaining Optimal Adaptive Weight Vector ${\widehat{\mathit{w}}}_{rd}^{op\mathit{t}}$ by Estimating ${\tilde{g}}_i^{\ast }$

It is seen from (30) that the computation of the optimal values ${\tilde{g}}_i^{\ast }$ involves the unknown quantities ${\beta }_i$ and $k_{rdi}$. Note that under Assumption 4, there exist one-to-one maps between ${\alpha }_i$ and ${\beta }_i$, $i=1,\dots ,Q_{rd}$. With the maps, the unknown quantities ${\beta }_i$ and $k_{rdi}$ satisfy [39,40,41,42],

$$\begin{array}{l}\left|{\alpha }_i-1-{\beta }_i-\frac{c_N\left(1+{\beta }_i\right)}{{\beta }_i}\right|\stackrel{a.s.}{\to }0\\ \left|k_{rdi}-\frac{1+c_{Nrd}/{\widehat{\beta }}_i}{1-c_{Nrd}/{\widehat{\beta }}_i^2}{\mathit{a}}_{rd}^{\mathrm{H}}\left(f_0^{\mathit{t}},f_0^s\right){\mathbf{d}}_i{\mathbf{d}}_i^{\mathrm{H}}{\mathit{a}}_{rd}\left(f_0^{\mathit{t}},f_0^s\right)\right|\stackrel{a.s.}{\to }0\end{array}$$

(31)

Then the consistent estimate of the two unknown quantities ${\beta }_i$ and $k_{rdi}$ can be given as

$$\begin{array}{l}{\widehat{\beta }}_i=\frac{{\alpha }_i-1-c_{Nrd}+\sqrt{{\left({\alpha }_i-1-c_{Nrd}\right)}^2-4c_{Nrd}}}{2}\\ {\widehat{k}}_{rdi}=\frac{1+c_{Nrd}/{\widehat{\beta }}_i}{1-c_{Nrd}/{\widehat{\beta }}_i^2}{\mathit{a}}_{rd}^{\mathrm{H}}\left(f_0^{\mathit{t}},f_0^s\right){\mathbf{d}}_i{\mathbf{d}}_i^{\mathrm{H}}{\mathit{a}}_{rd}\left(f_0^{\mathit{t}},f_0^s\right)\end{array}$$

(32)

With (32), we have the optimal estimate ${\widehat{g}}_i^{\ast }$ as

$${\widehat{g}}_i^{\ast }=\frac{{\widehat{\beta }}_i+c_{Nrd}}{{\widehat{\beta }}_i^2+{\widehat{\beta }}_i}\left[\frac{{\displaystyle \sum _{q=1}^{Q_{rd}}\frac{c_{Nrd}{\widehat{k}}_{rdq}}{{\widehat{\beta }}_q}}}{{\mathit{a}}_{rd}^{\mathrm{H}}\left(f_0^{\mathit{t}},f_0^s\right){\mathit{a}}_{rd}\left(f_0^{\mathit{t}},f_0^s\right)-{\displaystyle \sum _{q=1}^{Q_{rd}}{\widehat{k}}_{rdq}}+{\displaystyle \sum _{q=1}^{Q_{rd}}\frac{c_{Nrd}{\widehat{k}}_{rdq}}{{\widehat{\beta }}_q^2}}}-{\widehat{\beta }}_i\right],$$

(33)

and

$$\left|{\tilde{g}}_i^{\ast }-{\widehat{g}}_i^{\ast }\right|\stackrel{a.s.}{\to }0.$$

(34)

Then the estimated optimal inverse RD CNCM of ${\overline{\mathit{R}}}_{rd}^{-1}$ in the asymptotic case can be expressed as

$${\widehat{\mathit{R}}}_{rd}^{-1}\left({\widehat{\mathbf{g}}}^{\ast }\right)=\frac{1}{\sigma }\left({\displaystyle \sum _{i=1}^{Q_{rd}}{\widehat{g}}_i^{\ast }{\mathbf{d}}_i{\mathbf{d}}_i^{\mathrm{H}}}+{\mathbf{I}}_M\right),$$

(35)

and the corresponding RD-STAP weight vector is given as

$${\widehat{\mathit{w}}}_{rd}^{op\mathit{t}}\left({\widehat{\mathbf{g}}}^{\ast }\right)=\frac{{\overline{\mathit{R}}}_{rd}^{-1}\left({\widehat{\mathbf{g}}}^{\ast }\right){\mathit{a}}_{rd}\left(f_0^{\mathit{t}},f_0^s\right)}{{\mathit{a}}_{rd}^{\mathrm{H}}\left(f_0^{\mathit{t}},f_0^s\right){\overline{\mathit{R}}}_{rd}^{-1}\left({\widehat{\mathbf{g}}}^{\ast }\right){\mathit{a}}_{rd}\left(f_0^{\mathit{t}},f_0^s\right)}.$$

(36)

3.4. Determining the Noise Power $\sigma$

To obtain the optimal estimate of ${\widehat{\mathit{R}}}_{rd}^{-1}$ from (33) and (35), we need to know the noise power $\sigma$ in advance. If the noise power is not determined, we would have no access to the optimal RD-STAP weight vector. In this subsection, we provide an estimation method for the noise power $\sigma$.

In terms of [39], under finite training samples, the eigenvalues distribution of the covariance matrix will be expanded, i.e., the famous M-P law. However, among all the sample eigenvalues, there must be a sample eigenvalue that can approach the actual one. For RD CNCM, we assume that the noise-related eigenvalues are all the same. Similarly, under finite training samples, there must exist a noise-related sample eigenvalue, which can minimize the output clutter-plus-noise power in (18). Therefore, we can select the optimal noise-related sample eigenvalue as the noise power $\sigma$ by minimizing the output clutter-plus-noise power in (18).

With the estimated ${\widehat{\mathit{R}}}_{rd}^{-1}\left({\widehat{\mathbf{g}}}^{\ast }\right)$, the output clutter-plus-noise power in (18) can be re-expressed as

$$P_{rd}\left({\widehat{\mathit{w}}}_{rd}^{opt}\left({\widehat{\mathbf{g}}}^{\ast }\right)\right)=\frac{{\mathit{a}}_{rd}^{\mathrm{H}}\left(f_0^t,f_0^s\right){\overline{\mathit{R}}}_{rd}^{-1}\left({\widehat{\mathbf{g}}}^{\ast }\right){\mathit{R}}_{rd}{\overline{\mathit{R}}}_{rd}^{-1}\left({\widehat{\mathbf{g}}}^{\ast }\right){\mathit{a}}_{rd}\left(f_0^t,f_0^s\right)}{{\left[{\mathit{a}}_{rd}^{\mathrm{H}}\left(f_0^t,f_0^s\right){\overline{\mathit{R}}}_{rd}^{-1}\left({\widehat{\mathbf{g}}}^{\ast }\right){\mathit{a}}_{rd}\left(f_0^t,f_0^s\right)\right]}^2},$$

(37)

To obtain the optimal noise power $\sigma$ among all the noise-related sample eigenvalues, we need to solve the following optimization problem

$$\widehat{\sigma }=\underset{\sigma }{\min }{P}_{rd}\left({\widehat{\mathit{w}}}_{rd}^{opt}\left({\widehat{\mathbf{g}}}^{\ast }\left(\sigma \right)\right)\right).$$

(38)

By solving this problem, we have derived the following result.

Proposition 1.

If$c_{Nrd}>1$, the output clutter-plus-noise power $P_{rd}$in (38) decreases as$\sigma$decreases. If$c_{Nrd}<1$, the output clutter-plus-noise power$P_{rd}$in (38) decreases as$\sigma$increases.

Proof. 

See Appendix A. □

According to Proposition 1, we can determine the minimum noise-related sample eigenvalue as the estimated noise power $\widehat{\sigma }$ when $c_{Nrd}>1$, and we can determine the maximum noise-related sample eigenvalue as the estimated noise power $\widehat{\sigma }$ when $c_{Nrd}<1$.

4. Numerical Results

In this section, we conduct several numerical experiments to demonstrate the reasonability and accuracy of Assumption 1 and Proposition 1 and evaluate the performance of the proposed RMT-RD-STAP algorithm. The parameters used in the experiment scenarios are listed in

Table 1. Parameters used in the experiment scenarios.

Parameter	Value	Unit
Height	6000	m
Wavelength	0.3	m
Array number	8	/
Pulse number	8	/
PRF	2000	Hz
CNR	30	dB

. In terms of these parameters, the clutter data used in experiments are generated via the Ward model by Lincoln Laboratory at MIT [2]. For comparison, the classical EFA with three Doppler channels is taken as an example of the RD-STAP algorithm [16] with $M=3N$. In the experiments, the noise power is ${\sigma }_n^2=1$, and the platform velocity $V$ is set to be different values to reveal the performance of the proposed algorithms under different local clutter DoFs. STAP is taken in the main lobe of antenna arrays.

4.1. Numerical Validation on Assumption 1 and Proposition 1

Firstly, we make a comparison of the output clutter-plus-noise powers generated by different STAP weight vectors (7), (17), and (36) for two platform velocities, V = 150 m/s and V = 300 m/s, corresponding to two local clutter DoFs, $Q_{Rd}=10$ and $Q_{Rd}=12$, respectively. Figure 2 shows the output clutter-plus-noise power versus the number of training samples with $f_0^{\mathit{t}}=0.3$. It is obvious that for large $L$, both RD-STAP and RMT-RD-STAP have close output power. However, when the number $L$ of training samples becomes small, the output clutter-plus-noise power by the RD-STAP rapidly increases, whereas the power by the RMT-RD-STAP is almost kept intact. These results illustrate that the assumption of complex white Gaussian noise in reduced dimension is reasonable from the viewpoint of reducing the output clutter-plus-noise power. Comparing Figure 2a with Figure 2b, we find that the local clutter DoFs have negative effects on the output clutter-plus-noise powers. The larger the local clutter DoFs are, the larger the optimal output powers are. But the gap between the output powers produced by RMT-RD-STAP and the optimal STAP changes slightly for these two local clutter DoFs.

Next, we make a comparison of the output clutter-plus-noise powers under different platform velocities for RMT-RD-STAP, V = 150 m/s and V = 300 m/s, respectively. Figure 3 shows the output clutter-plus-noise power versus the noise eigenvalues with $f_0^{\mathit{t}}=0.3$, where $L=20$ and $L=30$ which correspond to two situations, $c_{Nrd}=3N/20>1$ and $c_{Nrd}=3N/30<1$. The noise-related eigenvalues are sorted in ascending order in the figures. It is seen that when $c_{Nrd}>1$, the output clutter-plus-noise power decreases as the noise power $\sigma$ decreases, and when $c_{Nrd}<1$, the output clutter-plus-noise power decreases as the noise power $\sigma$ increases. These results verify the correctness of Proposition 1. By comparing the four figures, it is obvious that the above conclusion can still be maintained when the local clutter DoFs vary.

4.2. Performance Comparisons

In this subsection, output SINR loss, the ratio of the output SINR to the SNR achieved by a matched filter in a clutter-free environment [2], is taken as a performance metric to measure different STAP algorithms. For RD-STAP, the SINR loss is given as

$$SIN{R}_{Loss}^{RD-STAP}=\frac{{\sigma }_n^2}{NK}\frac{{\left|{\mathit{w}}_{Lrd}^{opt}{}^{\mathrm{H}}{\mathit{a}}_{rd}\left(f_0^t,f_0^s\right)\right|}^2}{{\mathit{w}}_{Lrd}^{opt}{}^{\mathrm{H}}\mathit{R}{\mathit{w}}_{Lrd}^{opt}},$$

(39)

Similar loss functions are given by optimal STAP and RMT-RD-STAP. In the following experiments, the output SINR losses are presented by averaging 1000 independent runs.

Firstly, we present the variations of the output SINR losses as the normalized Doppler frequencies for different numbers of training samples. The results are shown in Figure 4 with $L=M$ and $L=2M$. In such cases, we choose the maximum noise-related eigenvalue of RD SCM as the noise power. The airborne platform is assumed to move at the velocity V = 150 m/s. It is seen that the RMT-RD-STAP algorithm is superior to the RD-STAP algorithm for small training samples. In Figure 4a, $L=M$, it is seen that RMT-RD-STAP is much superior to RD-STAP, and The SINR loss by RMT-RD-STAP decreases about 8 dB in comparison with RD-STAP. In Figure 4b, $L=2M$, which means that the number of the training samples is sufficient, and the SINR losses of both two RD-STAP algorithms decrease in comparison with that in Figure 4a. Although RMT-RD-STAP and RD-STAP have almost similar performance, RMT-RD-STAP is still slightly superior to RD-STAP.

Figure 5 shows the output SINR losses versus the number of training samples $L$ with $f_0^{\mathit{t}}=0.3$. It is seen that for large $L$, RMT-RD-STAP and RD-STAP achieve performance close to each other. However, as the number of training samples decreases, RMT-RD-STAP has distinct performance advantages over RD-STAP. Along with Figure 4, Figure 5 further demonstrates the advantages of the proposed RMT-RD-STAP algorithm.

Next, we present the variations of the output SINR losses as the function of normalized Doppler frequencies for different numbers of training samples. The results are shown in Figure 6 with $L=M$ and $L=2M$, where the maximum noise-related sample eigenvalue is selected as the noise power. In the experiments, the platform is assumed to move at the velocity V = 300 m/s. Comparing Figure 6a,b, it is seen that RMT-RD-STAP is superior to RD-STAP for small-size training samples. From ref. [48], we know that the local clutter DoFs increase as the velocity increases. In comparison with Figure 4, we find that the performance of all the algorithms decreases. However, RMT-RD-STAP is still superior to RD-STAP for small-size training samples. The SINR losses of RMT-RD-STAP in $L=M$ and $L=2M$ decrease by about 8 dB and 1 dB in comparison with RD-STAP, respectively.

In the implementation of the proposed RMT-RD-STAP algorithm, the local clutter DoFs need to be calculated according to radar parameters and platform parameters [48]. In a real environment, these parameters often deviate from the designed ones and then the calculated local clutter DoFs are different from the real local clutter DoFs. To demonstrate the robustness of the proposed algorithm, we use Figure 7 to show the output SINR losses versus the clutter of DoF errors with $f_0^{\mathit{t}}=0.3$. In the experiments, the platform velocities are assumed to be V = 150 m/s and V = 300 m/s, and the calculated local clutter of DoFs are 10 and 12, respectively. The clutter DoF errors vary from −3 to 3. The numbers of training samples are set to be $L=M$ and $L=2M$, respectively. It is well-known that the conventional RD-STAP is independent of the local clutter DoFs estimation, whereas the proposed RMT-RD-STAP is related to the local clutter of DoFs estimation. As observed from Figure 7, under the predesigned clutter DoF errors scope, the performance of RD-STAP is stable for all four cases. Surprisingly, we also find that the proposed RMT-RD-STAP is not sensitive to the local clutter DoFs errors and it always provides stable output SINR losses. Furthermore, the proposed RMT-RD-STAP has superior performance to RD-STAP. Specifically, the SINR losses of RMT-RD-STAP in $L=M$ and $L=2M$, respectively, decrease by about 8 dB and 1 dB in comparison with RD-STAP.

Under real clutter circumstance, the internal clutter motion (ICM) unavoidably exists due to wind and other factors [52], which is reflected by that the increase in clutter bandwidth in temporal domain. ICM will result in the RD-STAP performance deterioration. To illustrate the robustness of the proposed RMT-RD-STAP, we consider 2% ICM, which is a general situation. Figure 8 presents the output SINR losses versus normalized Doppler frequencies with 2% ICM. In the experiments, the platform velocities are assumed to be V = 150 m/s and V = 150 m/s, and the numbers of training samples are set to be $L=M$ and $L=2M$, respectively. We find that, under small training samples size $L=M$, the proposed RMT-RD-STAP still has great performance advantage in comparison with RD-STAP. On the other hand, for large training samples size $L=2M$, we find that RMT-RD-STAP and RD-STAP have almost the same performance, where RD-STAP achieves almost the optimal performance in this case according to the RMB rule.

To demonstrate the effectiveness of the proposed RMT-RD-STAP, we finally apply it to the available real clutter data, Mountain Top data, i.e., t38pre01v1CPI6 [53]. In this clutter data, the number $K$ of transmitted pulses and the number $N$ of array elements are 16 and 14, respectively. The PRF is 625 Hz and the distance resolution is 150 m. In the radar work time, there exist 403 range cells. We artificially add a target located in the 150-th range cell with a normalized Doppler frequency of 0.25 and the normalized spatial frequency 0.43 into the clutter data. In this case, the RD system DoFs is 42, and the local clutter DoFs is 16 in terms of [48]. Figure 9 presents the STAP output power verse the range cells with $L=45$ and $L=95$. In these two cases, four range cells adjacent to the CUT are regarded as the guard cells, and 45 and 95 range cells symmetrically located around the CUT are selected as training samples, respectively. It is seen that when $L=95$, which satisfies the RMB rule, the proposed RMT-RD-STAP and RD-STAP have similar performance. However, when $L=45$, which cannot satisfy the RMB rule, the RD-STAP will show a severe false alarm in the 144-th range cell, whereas the proposed RMT-RD-STAP has a distinct performance advantage and can accurately detect the target.

5. Conclusions

In this paper, we have proposed the RMT-based RD-STAP algorithm to enhance performance in the case of finite training samples. In terms of the spiked covariance model, the proposed algorithm estimates the inverse RD CNCM by optimally manipulating its sample eigenvalues while maintaining its sample eigenvectors. By minimizing the output clutter-plus-noise power of RD-STAP, the clutter-related eigenvalues are estimated according to the clutter-related sample eigenvalues via RMT, and the noise-related eigenvalue is optimally selected from the noise-related sample eigenvalues. Benefitted from RMT, the proposed RMT-RD-STAP greatly outperforms the conventional RD-STAP when the RMB rule is not satisfied and has superior performance when the number of training samples satisfies the RMB rule.