An Adaptive Radar Target Detection Method Based on Alternate Estimation in Power Heterogeneous Clutter

Abstract

Multichannel radars generally need to utilize a certain amount of training samples to estimate the covariance matrix of clutter for target detection. Due to factors such as severe terrain fluctuations and complex electromagnetic environments, the training samples usually have different statistical characteristics from the data to be detected. One of the most common scenarios is that all data have the same clutter covariance matrix structure, while different data have different power mismatches, called power heterogeneous characteristics. For detection problems in the power heterogeneous clutter environments, we propose detectors based on alternate estimation, using the generalized likelihood ratio test (GLRT) criterion, Rao criterion, Wald criterion, Gradient criterion, and Durbin criterion. Monte Carlo simulation experiments and real data indicate that the detector based on the Rao criterion has the highest probability of detection (PD). Furthermore, when signal mismatch occurs, the detector based on the GLRT criterion has the best selectivity, while the detector based on the Durbin criterion has the most robust detection performance.

1. Introduction

A radar system is an electronic device that employs electromagnetic waves for target detection, which is extensively used in various aspects of life, such as military early warning, wireless communication, navigation, meteorology, and so on. One of the principal roles of radar is to detect targets [1,2,3,4]. Traditional radar target detection can be categorized into two steps: the first step is the filtering process, which involves suppressing clutter to enhance the signal-to-clutter ratio (SCR); the second step is to process the filtered data with a constant false alarm rate (CFAR). This method is complex and costly. In the past few years, adaptive detection has gradually come into people’s sight. Its core idea is to jointly use the data to be detected and training samples to construct an adaptive detector, which integrates filtering and CFAR processing [5]. Compared with the step-by-step detection method, adaptive detection technology not only simplifies the process but also improves detection performance.

It is important to mention that the clutter covariance matrix is typically unknown and requires estimation using training samples [6]. The traditional assumption model holds that the training samples and the data to be detected have the same statistical characteristics, following a complex Gaussian distribution [7], but this assumption often does not hold in complex electromagnetic environments. Due to factors such as clutter fluctuations in real scenarios, environments often show non-homogeneous characteristics, especially non-Gaussian environments. However, training samples should have some similarities to the data to be detected [8]; otherwise, they cannot provide useful information. In 1999, Kraut et al. put forward the generalized likelihood ratio test (GLRT) detector in a partially homogeneous environment, known as the adaptive coherence estimator [9]. In a partially homogeneous environment, the training samples share the same covariance matrix, but an unknown power mismatch exists between the training samples and the data to be detected. In a complicated and changeable real environment, this assumption is also too idealistic.

As the resolution of radar enhances, the quantity of scatterers within each resolution cell diminishes, leading to different levels of local roughness of clutter within different range resolution cells and causing larger amplitude fluctuations. In such cases, clutters exhibit non-Gaussian characteristics [10,11,12]. The distribution of amplitudes for non-Gaussian clutter exhibits a heavier tail, rendering the Gaussian model insufficient in accurately describing such clutter. The compound-Gaussian model [13,14,15] is an extensively accepted model for modeling non-Gaussian clutter, which can be characterized as the product of mutually independent slow-changing texture components and fast-changing speckle components, where the former is a non-negative random variable and the latter is a complex Gaussian random vector. Non-Gaussian clutter can be expressed as various compound-Gaussian models, depending on the specific texture component models. When the texture component follows a gamma distribution, the compound-Gaussian clutter is K distribution clutter. It can describe non-homogeneous clutter well at high resolution and small grazing angles. When the texture component follows an inverse gamma distribution, compound-Gaussian clutter is Pareto distribution clutter. It is more suitable for describing clutter with heavy tails. When the texture component follows an inverse Gaussian distribution, compound-Gaussian clutter is CG-IG distribution clutter. It has a better fitting capability for high-resolution sea clutter. Among them, the most common clutter is K distribution clutter [16,17,18]. However, not all types of real clutter can be modeled as a compound-Gaussian model. Additionally, some models that fit real clutter well do not conform to the compound-Gaussian model, such as Weibull distribution [19] and Log-Normal distribution [20]. Under the compound-Gaussian model, it is often necessary to assign a certain prior statistical distribution to the texture component [21], but real clutter environments may not necessarily correspond to the given distribution. In this case, the detector, designed according to the predetermined statistical distribution of the texture component, is bound to result in performance loss.

To solve the aforementioned problem, it is assumed in [22] that there also exists unknown power mismatches between the training samples, and the amount of mismatches are unknown non-random and non-negative quantities. This model is an important complement to the compound-Gaussian model, which can also be transformed from the compound-Gaussian model. Based on the GLRT criterion, together with the alternate estimation algorithm, Coluccia et al. proposed an adaptive detector in [22], which has the CFAR property and offers better detection performance compared with the well-known adaptive normalized matched filter (ANMF).

It is important to highlight that the detection problem for only rank-one point targets is considered in [22], and the detector design is solely based on the GLRT criterion. However, in practice, the signal might be categorized as a subspace signal, which is an extension of rank-one signals and refers to the signal located in the known subspace with unknown coordinates. Meanwhile, for target detection problems in real scenarios, there is often no optimal detector due to the presence of too many unknown parameters. In addition to the GLRT criterion, other useful detector design criteria also include the Rao criterion, Wald criterion, Gradient criterion, and Durbin criterion, among others. Therefore, in this paper, we extend the signal model in [22], extending the rank-one model to the subspace signal model, and design detectors based on the GLRT criterion, Rao criterion, Wald criterion, Gradient criterion, and Durbin criterion.

In addition, signal mismatch was not considered in [22]. In practical applications, factors such as array element errors, multipath effects, and rapid maneuvers of targets often lead to signal mismatch [23,24,25], meaning that the actual steering vector of the target does not match the value assumed by the radar system. There is a type of detector whose detection performance sharply degrades with signal mismatch increases, known as mismatch-sensitive detectors, which are usually applied in fields such as target tracking [26]. Conversely, another type of detector can still detect targets with a high probability when signal mismatch occurs, known as robust detectors, which are usually applied in fields such as searching models [27]. Therefore, in the performance evaluation, we also consider the performance of proposed detectors when signal mismatch occurs.

In summary, this paper investigates the detection problem for subspace signals in a power heterogeneous clutter environment. Utilizing the GLRT criterion, Rao criterion, Wald criterion, Gradient criterion, and Durbin criterion, in conjunction with the iterative concept of alternating estimation, we propose a variety of detectors, named the alternating estimation GLRT (AE-GLRT) detector, alternating estimation Rao (AE-Rao) detector, alternating estimation Wald (AE-Wald) detector, alternating estimation Gradient (AE-Gradient) detector, and alternating estimation Durbin (AE-Durbin) detector, respectively. Monte Carlo simulation experiments and IPIX radar data both demonstrate that the AE-Rao detector has the highest probability of detection (PD) in the absence of signal mismatch. When signal mismatch occurs, the AE-GLRT exhibits the best selectivity, while the AE-Durbin offers the most robust performance.

Notation: Vectors are represented by bold lowercase letters, whereas matrices are represented by bold uppercase letters. The symbols ${(·)}^{\mathrm{T}}$, ${(·)}^{*}$, ${(·)}^{\mathrm{H}}$, ${(·)}^{-1}$, ${(·)}^{-}$, $\partial (·)$, $\mathrm{Re}\{·\}$, and $|·|$ express transpose, conjugate, conjugate transpose, inverse, generalized inverse, partial derivative, the real part of a parameter, and the matrix determinant, respectively. $\widehat{\mathit{\xi }}$ denotes the estimation of $\mathit{\xi }$, $\mathrm{E}\left[·\right]$ indicates statistical expectation, and ${\mathbb{C}}^{N\times M}$ represents the $N\times M$ dimensional complex field.

2. Problem Formulation

According to the compound-Gaussian model, vector $\mathbf{c}\in {\mathbb{C}}^{N\times 1}$ represents the clutter in the cell under test (CUT), which can be denoted as

$$\mathbf{c}=\sqrt{\tau }\mathbf{x}$$

(1)

where $\tau$ represents the slowly changing texture component, which is generally a non-negative random variable and $\mathbf{x}$ represents the rapidly changing speckle component, following a circular symmetric complex Gaussian distribution with conditional covariance matrix $\mathrm{E}\left[\mathbf{x}{\mathbf{x}}^{\mathrm{H}}|\tau \right]=\tau \mathbf{R}$, where $\mathbf{R}\in {\mathbb{C}}^{N\times N}$ represents the unknown clutter covariance matrix. The training samples can be expressed as ${\mathbf{c}}_l=\sqrt{{\tau }_l}{\mathbf{x}}_l,l=1,2,\cdots ,L$, whose conditional covariance matrix is $\mathrm{E}\left[{\mathbf{x}}_l{\mathbf{x}}_l^{\mathrm{H}}|{\tau }_l\right]={\tau }_l\mathbf{R},l=1,2,\cdots ,L$.

Assume that the radar system possesses N channels. If a signal exists, it can be expressed as $N\times 1$ vector $\mathbf{s}$. For subspace signals, assuming $\mathbf{s}$ is located in the subspace $\mathbf{H}$, it can be represented as $\mathbf{s}=\mathbf{H}\mathit{\xi }$. Therefore, in the power heterogeneous clutter environment, the detection problem can be represented by the following binary hypothesis test

$$\begin{array}{c}{\mathrm{H}}_0:\left\{\begin{array}{c}\mathbf{z}\sim \mathcal{C}{\mathcal{N}}_N(0,\mathbf{R})\hfill \\ {\mathbf{z}}_l\sim \mathcal{C}{\mathcal{N}}_N(0,{\gamma }_l\mathbf{R}),l=1,2,\cdots ,L\hfill \end{array}\right.\\ {\mathrm{H}}_1:\left\{\begin{array}{c}\mathbf{z}\sim \mathcal{C}{\mathcal{N}}_N(\mathbf{H}\mathit{\xi },\mathbf{R})\hfill \\ {\mathbf{z}}_l\sim \mathcal{C}{\mathcal{N}}_N(0,{\gamma }_l\mathbf{R}),l=1,2,\cdots ,L\hfill \end{array}\right.\end{array}$$

(2)

where $\mathbf{z}\in {\mathbb{C}}^{N\times 1}$ is the data to be detected, ${\mathbf{z}}_l\in {\mathbb{C}}^{N\times 1}$ represents the training samples, $\mathbf{H}\in {\mathbb{C}}^{N\times p}$ is a known column-full-rank signal matrix, $\mathit{\xi }\in {\mathbb{C}}^{p\times 1}$ denotes an unknown signal coordinate vector, $\mathbf{R}\in {\mathbb{C}}^{N\times N}$ represents the unknown clutter covariance matrix, and ${\gamma }_1,{\gamma }_2,\cdots ,{\gamma }_l>0$ are unknown parameters that denote the different power mismatch amounts between each training sample and the data to be detected.

As mentioned above, the detection model in (2) can be transformed from the compound-Gaussian model. Specifically, dividing all the clutter data by $\sqrt{\tau }$ yields the clutter in the CUT as ${\mathbf{c}}^{\prime }=\mathbf{x}$ with conditional covariance matrix $\mathbf{R}$, where the value of $\tau$ can be estimated by [24]. Meanwhile, the training samples can be rewritten as ${\mathbf{c}}_l^{\prime }={\mathbf{c}}_l/\sqrt{\tau }$. According to the definition of ${\mathbf{c}}_l$ above, we set ${\gamma }_l={\sqrt{\tau }}_l/\sqrt{\tau },l=1,2,\cdots ,L$, so the training samples can be written as ${\mathbf{c}}_l^{\prime }={\gamma }_l{\mathbf{x}}_l,l=1,2,\cdots ,L$ with conditional covariance matrix being ${\gamma }_l\mathbf{R},l=1,2,\cdots ,L$. In addition, this paper regards ${\gamma }_l,l=1,2,\cdots ,L$ as definite unknown parameters, which can increase the applicability of this model; that is, it is not limited to a particular compound-Gaussian clutter with a predetermined texture distribution.

3. Detector Design

The joint probability density functions (PDFs) of the data to be detected and the training samples under hypotheses ${\mathrm{H}}_1$ and ${\mathrm{H}}_0$ can be expressed as

$$\begin{array}{c}\hfill f_1(\mathbf{z},\mathbf{Z};\mathit{\xi },\mathbf{R},\mathbf{\gamma })=\frac{1}{{\pi }^{N(L+1)}{\displaystyle \prod _{l=1}^L}{\gamma }_l^N{\left|\mathbf{R}\right|}^{L+1}}exp\left\{-\left({\left(\mathbf{z}-\mathbf{H}\mathit{\xi }\right)}^{\mathrm{H}}{\mathbf{R}}^{-1}\left(\mathbf{z}-\mathbf{H}\mathit{\xi }\right)+{\displaystyle \sum _{l=1}^L}\frac{{\mathbf{z}}_l^{\mathrm{H}}{\mathbf{R}}^{-1}{\mathbf{z}}_l}{{\gamma }_l}\right)\right\}\end{array}$$

(3)

and

$$\begin{array}{c}\hfill f_0(\mathbf{z},\mathbf{Z};\mathbf{R},\mathbf{\gamma })=\frac{1}{{\pi }^{N(L+1)}{\displaystyle \prod _{l=1}^L}{\gamma }_l^N{\left|\mathbf{R}\right|}^{L+1}}exp\left\{-\left({\mathbf{z}}^{\mathrm{H}}{\mathbf{R}}^{-1}\mathbf{z}+{\displaystyle \sum _{l=1}^L}\frac{{\mathbf{z}}_l^{\mathrm{H}}{\mathbf{R}}^{-1}{\mathbf{z}}_l}{{\gamma }_l}\right)\right\}\end{array}$$

(4)

respectively, where $\mathbf{Z}=\left[{\mathbf{z}}_1,{\mathbf{z}}_2,\cdots {\mathbf{z}}_L\right]$ denotes the matrix of training samples and $\mathbf{\gamma }={\left[{\gamma }_1,{\gamma }_2,\cdots ,{\gamma }_L\right]}^{\mathrm{T}}$ represents the column vector of power mismatches between the training samples and the data to be detected. The following subsections will design detectors according to the GLRT criterion, Rao criterion, Wald criterion, Gradient criterion, and Durbin criterion.

3.1. AE-GLRT Detector

The GLRT design criterion can be formulated as [28]

$$\frac{{max}_{\mathit{\xi },\mathbf{R},\mathbf{\gamma }}{f}_1(\mathbf{z},\mathbf{Z};\mathit{\xi },\mathbf{R},\mathbf{\gamma })}{{max}_{\mathbf{R},\mathbf{\gamma }}{f}_0(\mathbf{z},\mathbf{Z};\mathbf{R},\mathbf{\gamma })}\underset{{\mathrm{H}}_0}{\stackrel{{\mathrm{H}}_1}{\gtrless }}\sigma$$

(5)

where $\sigma$ denotes the detection threshold set based on the predesigned PFA. Under hypothesis ${\mathrm{H}}_1$, taking the partial derivative of (3) with respect to $\mathbf{R}$ to be zero, we can obtain the maximum likelihood estimation (MLE) of $\mathbf{R}$ for a given ${\gamma }_l$ as

$${\widehat{\mathbf{R}}}_1=\frac{1}{L+1}\left[\left(\mathbf{z}-\mathbf{H}\mathit{\xi }\right){\left(\mathbf{z}-\mathbf{H}\mathit{\xi }\right)}^{\mathrm{H}}+{\displaystyle \sum _{l=1}^L}\frac{{\mathbf{z}}_l{\mathbf{z}}_l^{\mathrm{H}}}{{\gamma }_l}\right]$$

(6)

Utilizing the same method, we can derive the MLE of $\mathbf{R}$ under hypothesis ${\mathrm{H}}_0$ as

$${\widehat{\mathbf{R}}}_0=\frac{1}{L+1}\left(\mathbf{z}{\mathbf{z}}^{\mathrm{H}}+{\displaystyle \sum _{l=1}^L}\frac{{\mathbf{z}}_l{\mathbf{z}}_l^{\mathrm{H}}}{{\gamma }_l}\right)$$

(7)

Substituting (6) and (7) into (3) and (4), respectively, we can obtain

$$l_1(\mathbf{z},\mathbf{Z};\mathit{\xi },{\widehat{\mathbf{R}}}_1,\mathbf{\gamma })=\frac{{\left(\frac{L+1}{e\pi }\right)}^{N(L+1)}}{{\displaystyle \prod _{l=1}^L}{\gamma }_l^N{\left.\left|(\mathbf{z}-\mathbf{H}\mathit{\xi }){(\mathbf{z}-\mathbf{H}\mathit{\xi })}^{\mathrm{H}}+{\displaystyle \sum _{l=1}^L}\frac{{\mathbf{z}}_l{\mathbf{z}}_l^{\mathrm{H}}}{{\gamma }_l}\right.\right|}^{(L+1)}}$$

(8)

and

$$l_0(\mathbf{z},\mathbf{Z};{\widehat{\mathbf{R}}}_0,\mathbf{\gamma })=\frac{{\left(\frac{L+1}{e\pi }\right)}^{N(L+1)}}{{\displaystyle \prod _{l=1}^L}{\gamma }_l^N{\left.\left|\mathbf{z}{\mathbf{z}}^{\mathrm{H}}+{\displaystyle \sum _{l=1}^L}\frac{{\mathbf{z}}_l{\mathbf{z}}_l^N}{{\gamma }_l}\right.\right|}^{(L+1)}}$$

(9)

Directly solving for the MLE of $\mathit{\xi }$ and ${\gamma }_l,l=1,2,\cdots ,L$ does not yield a closed-form solution. Therefore, we use the alternate estimation algorithm to address this problem. For ease of expression, we specify the following symbols: ${\widehat{\gamma }}_l^{1,\left(0\right)}$ and ${\widehat{\gamma }}_l^{1,\left(t_{max}\right)}$ represent the initial value and final estimation of ${\gamma }_l$ under hypothesis ${\mathrm{H}}_1$, respectively. ${\widehat{\mathit{\xi }}}^{\left(t_{max}\right)}$ denotes the final estimation of $\mathit{\xi }$ under hypothesis ${\mathrm{H}}_1$; ${\widehat{\gamma }}_l^{0,\left(0\right)}$ and ${\widehat{\gamma }}_l^{0,\left(t_{max}\right)}$ represent the initial value and final estimation of ${\gamma }_l$ under hypothesis ${\mathrm{H}}_0$, respectively.

The estimation process is as follows: First, we let the initial value of ${\gamma }_l$ under hypothesis ${\mathrm{H}}_1$ be ${\widehat{\gamma }}_l^{1,\left(0\right)},l=1,2,\cdots ,L$ [22], and ${\widehat{\mathit{\xi }}}^{\left(1\right)}$ can be obtained by (A4); then, we obtain ${\widehat{\gamma }}_l^{1,\left(1\right)},l=1,2,\cdots ,L$ based on (A8). The process is repeated in a loop until the maximum number of iterations $t_{max}$ is reached. In this paper, we set it to 15, whose validity will be shown in the fourth section. Algorithm 1 details the steps of the alternate estimation iterative algorithm.

Algorithm 1 The algorithm of alternate estimation.

Input: ${\widehat{\gamma }}_l^{0,\left(0\right)},l=1,2,\cdots ,L$, ${\widehat{\gamma }}_l^{1,\left(0\right)},l=1,2,\cdots ,L$ and $t_{max}$ Output: ${\widehat{\gamma }}_l^{0,\left(t_{max}\right)},l=1,2,\cdots ,L$, ${\widehat{\gamma }}_l^{1,\left(t_{max}\right)},l=1,2,\cdots ,L$ and ${\widehat{\mathit{\xi }}}^{\left(t_{max}\right)}$  1: Let the initial value of ${\gamma }_l$ under hypothesis ${\mathrm{H}}_1$ be ${\widehat{\gamma }}_l^{1,\left(0\right)}={\mathbf{z}}_l^{\mathrm{H}}{\left(\mathbf{z}{\mathbf{z}}^{\mathrm{H}}\right)}^{-}{\mathbf{z}}_l,l=1,2,\cdots ,L$;  2: Substituting ${\widehat{\gamma }}_l^{1,\left(0\right)},l=1,2,\cdots ,L$ into (A4) yields the value of ${\widehat{\mathit{\xi }}}^{\left(1\right)}$;  3: Substituting ${\widehat{\mathit{\xi }}}^{\left(1\right)}$ into (A8) results in ${\widehat{\gamma }}_1^{1,\left(1\right)},{\widehat{\gamma }}_2^{1,\left(1\right)},\cdots ,{\widehat{\gamma }}_L^{1,\left(1\right)}$;  4: Substituting ${\widehat{\gamma }}_1^{1,\left(1\right)},{\widehat{\gamma }}_2^{1,\left(1\right)},\cdots ,{\widehat{\gamma }}_L^{1,\left(1\right)}$ into (A4) results in ${\widehat{\mathit{\xi }}}^{\left(2\right)}$;  5: Substituting ${\widehat{\mathit{\xi }}}^{\left(2\right)}$ into (A8) results in ${\widehat{\gamma }}_1^{1,\left(2\right)},{\widehat{\gamma }}_2^{1,\left(2\right)},\cdots ,{\widehat{\gamma }}_L^{1,\left(2\right)}$. Terminating the process when the number of iterations reaches $t_{max}$. Finally, we can obtain ${\widehat{\gamma }}_1^{1,\left(t_{max}\right)},{\widehat{\gamma }}_2^{1,\left(t_{max}\right)},\cdots ,{\widehat{\gamma }}_L^{1,\left(t_{max}\right)}$, and ${\widehat{\mathit{\xi }}}^{\left(t_{max}\right)}$;  6: Setting $\widehat{\mathit{\xi }}=\mathbf{0}$ under hypothesis ${\mathrm{H}}_0$, and then repeating the above steps to obtain ${\widehat{\gamma }}_1^{0,\left(t_{max}\right)},{\widehat{\gamma }}_2^{0,\left(t_{max}\right)},\cdots ,{\widehat{\gamma }}_L^{0,\left(t_{max}\right)}$.

When obtaining the estimations of $\mathit{\xi }$ and ${\gamma }_l,l=1,2,\cdots ,L$ from Appendix A, we can substitute (6), (7), (A4), (A8), and (A10) into (5), resulting in the AE-GLRT as

$$t_{\mathrm{AE}-\mathrm{GLRT}}=\frac{{\left[\frac{{\prod }_{l=1}^L{\widehat{\gamma }}_l^{0,\left(t_{max}\right)}}{{\prod }_{l=1}^L{\widehat{\gamma }}_l^{1,\left(t_{max}\right)}}\right]}^{\frac{N}{L+1}}\left|\left.\mathbf{z}{\mathbf{z}}^{\mathrm{H}}+{\displaystyle \sum _{l=1}^L}\frac{{\mathbf{z}}_l{\mathbf{z}}_l^{\mathrm{H}}}{{\widehat{\gamma }}_l^{0,\left(t_{max}\right)}}\right|\right.}{\left|\left.\left(\mathbf{z}-\mathbf{H}{\widehat{\mathit{\xi }}}^{\left(t_{max}\right)}\right){\left(\mathbf{z}-\mathbf{H}{\widehat{\mathit{\xi }}}^{\left(t_{max}\right)}\right)}^{\mathrm{H}}+{\displaystyle \sum _{l=1}^L}\frac{{\mathbf{z}}_l{\mathbf{z}}_l^{\mathrm{H}}}{{\widehat{\gamma }}_l^{1,\left(t_{max}\right)}}\right|\right.}$$

(10)

3.2. AE-Rao Detector

The Rao design criterion can be expressed as [29]

$$t_{\mathrm{Rao}}={\left.\frac{\partial ln{f}_1(\mathbf{z},\mathbf{Z})}{\partial {\Theta }_r}\right|}_{\Theta ={\widehat{\Theta }}_0}^{\mathrm{T}}{\left[{\mathbf{I}}^{-1}\left({\widehat{\Theta }}_0\right)\right]}_{{\Theta }_r{\Theta }_r}{\left.\frac{\partial ln{f}_1(\mathbf{z},\mathbf{Z})}{\partial {\Theta }_r^{*}}\right|}_{\Theta ={\widehat{\Theta }}_0}$$

(11)

where ${\left[{\mathbf{I}}^{-1}\left({\widehat{\Theta }}_0\right)\right]}_{{\Theta }_r{\Theta }_r}$ is the value at $\Theta ={\widehat{\Theta }}_0$ of the $\left({\Theta }_r,{\Theta }_r\right)$ block of the inverse of the fisher information matrix (FIM) $\mathbf{I}\left(\Theta \right)$. It is known by the matrix partitioning lemma

$${\left[{\mathbf{I}}^{-1}(\Theta )\right]}_{{\Theta }_r,{\Theta }_r}={\left[{\mathbf{I}}_{{\Theta }_r,{\Theta }_r}(\Theta )-{\mathbf{I}}_{{\Theta }_r,{\Theta }_s}(\Theta ){\mathbf{I}}_{{\Theta }_s,{\Theta }_s}^{-1}(\Theta ){\mathbf{I}}_{{\Theta }_s,{\Theta }_r}(\Theta )\right]}^{-1}$$

(12)

For the complex parameter $\Theta$, its FIM is defined as

$$\mathbf{I}\left(\Theta \right)=\mathrm{E}\left[\frac{\partial lnf(\mathbf{z};\Theta )}{\partial {\Theta }^{*}}\frac{\partial lnf(\mathbf{z};\Theta )}{\partial {\Theta }^{\mathrm{T}}}\right]$$

(13)

which can be divided into

$$\mathbf{I}(\Theta )=\left[\begin{array}{cc}{\mathbf{I}}_{{\Theta }_r,{\Theta }_r}(\Theta )& {\mathbf{I}}_{{\Theta }_r,{\Theta }_s}(\Theta )\\ {\mathbf{I}}_{{\Theta }_s,{\Theta }_r}(\Theta )& {\mathbf{I}}_{{\Theta }_s,{\Theta }_s}(\Theta )\end{array}\right]$$

(14)

with

$$\Theta ={\left[{\Theta }_r^{\mathrm{T}},{\Theta }_s^{\mathrm{T}}\right]}^{\mathrm{T}}$$

(15)

being a $(N^2+L+p)\times 1$ dimensional parameter vector. ${\Theta }_r=\mathit{\xi }$ denotes the relevant paramater and ${\Theta }_s={\left[{\mathbf{\gamma }}^{\mathrm{T}},\mathrm{ve}{\mathrm{c}}^{\mathrm{T}}\left(\mathbf{R}\right)\right]}^{\mathrm{T}}$ represents the irrelevant parameter. ${\widehat{\Theta }}_0={\left[{\widehat{\Theta }}_{r0}^{\mathrm{T}},{\widehat{\Theta }}_{s0}^{\mathrm{T}}\right]}^{\mathrm{T}}$ indicates the MLE of parameter $\Theta$ under hypothesis ${\mathrm{H}}_0$ with ${\widehat{\Theta }}_{r0}={\mathbf{0}}_{p\times 1}$ and ${\widehat{\Theta }}_{s0}={\left[{{\widehat{\mathbf{\gamma }}}_0}^{\mathrm{T}},\mathrm{ve}{\mathrm{c}}^{\mathrm{T}}\left(\widehat{\mathbf{R}}\right)\right]}^{\mathrm{T}}$. ${\widehat{\Theta }}_1={\left[{\widehat{\Theta }}_{r1}^{\mathrm{T}},{\widehat{\Theta }}_{s1}^{\mathrm{T}}\right]}^{\mathrm{T}}$ indicates the MLE of parameter $\Theta$ under hypothesis ${\mathrm{H}}_1$ with ${\widehat{\Theta }}_{r1}=\widehat{\mathit{\xi }}$ and ${\widehat{\Theta }}_{s1}={\left[{{\widehat{\mathbf{\gamma }}}_1}^{\mathrm{T}},\mathrm{ve}{\mathrm{c}}^{\mathrm{T}}\left(\widehat{\mathbf{R}}\right)\right]}^{\mathrm{T}}$. According to (3), the partial derivatives of $ln{f}_1\left(\mathbf{z},\mathbf{Z}\right)$ with respect to $\mathit{\xi }$ and ${\mathit{\xi }}^{*}$ are

$$\frac{\partial ln{f}_1\left(\mathbf{z},\mathbf{Z}\right)}{\partial {\Theta }_r}=\frac{\partial ln{f}_1\left(\mathbf{z},\mathbf{Z}\right)}{\partial \mathit{\xi }}={\left[{\left(\mathbf{z}-\mathbf{H}\mathit{\xi }\right)}^{\mathrm{H}}{\mathbf{R}}^{-\mathbf{1}}\mathbf{H}\right]}^{\mathrm{T}}$$

(16)

and

$$\frac{\partial ln{f}_1\left(\mathbf{z},\mathbf{Z}\right)}{\partial {\Theta }_r^{*}}=\frac{\partial ln{f}_1\left(\mathbf{z},\mathbf{Z}\right)}{\partial {\mathit{\xi }}^{*}}={\mathbf{H}}^{\mathrm{H}}{\mathbf{R}}^{-1}\left(\mathbf{z}-\mathbf{H}\mathit{\xi }\right)$$

(17)

respectively. According to the definition of (14), substituting (16) and (17) into (13) results in

$$\begin{array}{cc}\hfill {\mathbf{I}}_{{\Theta }_r,{\Theta }_r}(\Theta )=& \mathrm{E}\left[\frac{\partial lnf(\mathbf{z},\mathbf{Z})}{\partial {\Theta }_r^{*}}\frac{\partial lnf(\mathbf{z},\mathbf{Z})}{\partial {\Theta }_r^{\mathrm{T}}}\right]\hfill \\ \hfill =& {\mathbf{H}}^{\mathrm{H}}{\mathbf{R}}^{-1}\mathrm{E}\left[\left(\mathbf{z}-\mathbf{H}\mathit{\xi }\right){\left(\mathbf{z}-\mathbf{H}\mathit{\xi }\right)}^{\mathrm{H}}\right]{\mathbf{R}}^{-1}\mathbf{H}={\mathbf{H}}^{\mathrm{H}}{\mathbf{R}}^{-1}\mathbf{H}\hfill \end{array}$$

(18)

${\mathbf{I}}_{{\Theta }_r,{\Theta }_s}(\Theta )$ can be written as

$${\mathbf{I}}_{{\Theta }_r,{\Theta }_s}(\Theta )=\left[{\mathbf{I}}_{\mathit{\xi },\mathbf{\gamma }}(\Theta ){\mathbf{I}}_{\mathit{\xi },\mathrm{vec}\left(\mathbf{R}\right)}(\Theta )\right]$$

(19)

We can derive the following two equations according to (17)

$$\begin{array}{c}\hfill {\mathbf{I}}_{\mathit{\xi },\mathbf{\gamma }}(\Theta )=-\mathrm{E}\left[\frac{{\partial }^{2}ln{f}_1(\mathbf{z},\mathbf{Z})}{\partial {\mathit{\xi }}^{*}\partial {\mathbf{\gamma }}^{\mathrm{T}}}\right]=-\mathrm{E}\left[\frac{\partial }{\partial {\mathbf{\gamma }}^{\mathrm{T}}}\left({\mathbf{H}}^{\mathrm{H}}{\mathbf{R}}^{-1}\left(\mathbf{z}-\mathbf{H}\mathit{\xi }\right)\right)\right]={\mathbf{0}}_{p\times L}\end{array}$$

(20)

and

$$\begin{array}{c}\hfill {\mathbf{I}}_{\mathit{\xi },\mathrm{vec}\left(\mathbf{R}\right)}(\Theta )=-\mathrm{E}\left[\frac{{\partial }^{2}ln{f}_1(\mathbf{z},\mathbf{Z})}{\partial {\mathit{\xi }}^{*}\partial \mathrm{ve}{\mathrm{c}}^{\mathrm{T}}\left(\mathbf{R}\right)}\right]=-\mathrm{E}\left[\frac{\partial }{\partial \mathrm{ve}{\mathrm{c}}^{\mathrm{T}}\left(\mathbf{R}\right)}\left({\mathbf{H}}^{\mathrm{H}}{\mathbf{R}}^{-1}\left(\mathbf{z}-\mathbf{H}\mathit{\xi }\right)\right)\right]={\mathbf{0}}_{p\times N^2}\end{array}$$

(21)

Substituting (18), (19), (20), and (21) into (12) results in

$${\left[{\mathbf{I}}^{-1}(\Theta )\right]}_{{\Theta }_r{\Theta }_r}={\left({\mathbf{H}}^{\mathrm{H}}{\mathbf{R}}^{-1}\mathbf{H}\right)}^{-1}$$

(22)

Substituting (16), (17) and (22) into (11) yields the AE-Rao for the given $\mathbf{R}$ as

$$t_{\mathrm{AE}-{\mathrm{Rao}}_{\mathbf{R}}}={\mathbf{z}}^{\mathrm{H}}{\mathbf{R}}^{-1}\mathbf{H}{\left({\mathbf{H}}^{\mathrm{H}}{\mathbf{R}}^{-1}\mathbf{H}\right)}^{-1}{\mathbf{H}}^{\mathrm{H}}{\mathbf{R}}^{-1}\mathbf{z}$$

(23)

Substituting the final alternate estimation of ${\mathbf{\gamma }}_0$ into (7) results in the MLE of $\mathbf{R}$ under hypothesis ${\mathrm{H}}_0$ as

$${\widehat{\mathbf{R}}}_0=\frac{1}{L+1}\left({\displaystyle \sum _{l=1}^L}\frac{{\mathbf{z}}_l{\mathbf{z}}_l^{\mathrm{H}}}{{\widehat{\gamma }}_l^{0,\left(t_{max}\right)}}+\mathbf{z}{\mathbf{z}}^{\mathrm{H}}\right)$$

(24)

where the value of ${\widehat{\gamma }}_l^{0,\left(t_{max}\right)}$ can be obtained by (A9) and (A10). For convenience, we define ${\Sigma }_0={\displaystyle \sum _{l=1}^L}\frac{{\mathbf{z}}_l{\mathbf{z}}_l^{\mathrm{H}}}{{\widehat{\gamma }}_l^{0,\left(t_{max}\right)}}+\mathbf{z}{\mathbf{z}}^{\mathrm{H}}$. Substituting (24) into (23) and ignoring the constant terms results in the AE-Rao as

$$t_{\mathrm{AE}-\mathrm{Rao}}={\mathbf{z}}^{\mathrm{H}}{\Sigma }_0^{-1}\mathbf{H}{\left({\mathbf{H}}^{\mathrm{H}}{\Sigma }_0^{-1}\mathbf{H}\right)}^{-1}{\mathbf{H}}^{\mathrm{H}}{\Sigma }_0^{-1}\mathbf{z}$$

(25)

3.3. AE-Wald Detector

The Wald design criterion can be expressed as [29]

$$t_{\mathrm{Wald}}={\left({\widehat{\Theta }}_{r1}-{\Theta }_{r0}\right)}^{\mathrm{H}}{\left\{{\left[{\mathbf{I}}^{-1}\left({\widehat{\Theta }}_1\right)\right]}_{{\Theta }_r,{\Theta }_r}\right\}}^{-1}\left({\widehat{\Theta }}_{r1}-{\Theta }_{r0}\right)$$

(26)

Nulling the derivative of (3) with respect to $\mathit{\xi }$ yields the MLE of $\mathit{\xi }$ for the given $\mathbf{R}$ as

$$\widehat{\mathit{\xi }}={\left({\mathbf{H}}^{\mathrm{H}}{\mathbf{R}}^{-1}\mathbf{H}\right)}^{-1}{\mathbf{H}}^{\mathrm{H}}{\mathbf{R}}^{-1}\mathbf{z}$$

(27)

Plugging (22) and (27) into (26) results in the AE-Wald for the given $\mathbf{R}$ as

$$t_{\mathrm{AE}-\mathrm{Wal}{\mathrm{d}}_{\mathbf{R}}}={\mathbf{z}}^{\mathrm{H}}{\mathbf{R}}^{-1}\mathbf{H}{\left({\mathbf{H}}^{\mathrm{H}}{\mathbf{R}}^{-1}\mathbf{H}\right)}^{-1}{\mathbf{H}}^{\mathrm{H}}{\mathbf{R}}^{-1}\mathbf{z}$$

(28)

Substituting the final alternate estimations of ${\mathbf{\gamma }}_1$ and $\mathit{\xi }$ into (6) results in the MLE of $\mathbf{R}$ under hypothesis ${\mathrm{H}}_1$ as

$${\widehat{\mathbf{R}}}_1=\frac{1}{L+1}\left[{\displaystyle \sum _{l=1}^L}\frac{{\mathbf{z}}_l{\mathbf{z}}_l^{\mathrm{H}}}{{\widehat{\gamma }}_l^{1,\left(t_{max}\right)}}+(\mathbf{z}-\mathbf{H}{\widehat{\mathit{\xi }}}^{\left(t_{max}\right)}){(\mathbf{z}-\mathbf{H}{\widehat{\mathit{\xi }}}^{\left(t_{max}\right)})}^{\mathrm{H}}\right]$$

(29)

where the values of ${\widehat{\gamma }}_l^{1,\left(t_{max}\right)}$ and ${\widehat{\mathit{\xi }}}^{\left(t_{max}\right)}$ can be obtained by iterating (A4) and (A8) to the $t_{max}$. For convenience, we define ${\Sigma }_1={\displaystyle \sum _{l=1}^L}\frac{{\mathbf{z}}_l{\mathbf{z}}_l^{\mathrm{H}}}{{\widehat{\gamma }}_l^{1,\left(t_{max}\right)}}+(\mathbf{z}-\mathbf{H}{\widehat{\mathit{\xi }}}^{\left(t_{max}\right)}){(\mathbf{z}-\mathbf{H}{\widehat{\mathit{\xi }}}^{\left(t_{max}\right)})}^{\mathrm{H}}$. Substituting (29) into (28) and ignoring the constant terms lead to the AE-Wald as

$$t_{\mathrm{AE}-\mathrm{Wald}}={\mathbf{z}}^{\mathrm{H}}{\Sigma }_1^{-1}\mathbf{H}{\left({\mathbf{H}}^{\mathrm{H}}{\Sigma }_1^{-1}\mathbf{H}\right)}^{-1}{\mathbf{H}}^{\mathrm{H}}{\Sigma }_1^{-1}\mathbf{z}$$

(30)

3.4. AE-Gradient Detector

The Gradient design criterion can be written as [30]

$$t_{\mathrm{Gradient}}=Re\left\{{{\left.\frac{\partial ln{f}_1(\mathbf{z},\mathbf{Z})}{\partial {\Theta }_r}\right|}^{\mathrm{T}}}_{\Theta ={\widehat{\Theta }}_0}\left({\widehat{\Theta }}_{r1}-{\Theta }_{r0}\right)\right\}$$

(31)

Substituting (16) and (27) into (31) results in the AE-Gradient for the given $\mathbf{R}$ as

$$t_{\mathrm{AE}-{\mathrm{Gradient}}_{\mathbf{R}}}={\mathbf{z}}^{\mathrm{H}}{\mathbf{R}}^{-1}\mathbf{H}{\left({\mathbf{H}}^{\mathrm{H}}{\mathbf{R}}^{-1}\mathbf{H}\right)}^{-1}{\mathbf{H}}^{\mathrm{H}}{\mathbf{R}}^{-1}\mathbf{z}$$

(32)

The MLE of $\mathbf{R}$ under hypotheses ${\mathrm{H}}_1$ and ${\mathrm{H}}_0$ are (29) and (24), respectively. Substituting (30) and (24) into (32), simplifying with the definition of ${\Sigma }_0$ and ${\Sigma }_1$, and ignoring the constant terms yield the AE-Gradient as

$$t_{\mathrm{AE}-\mathrm{Gradient}}={\mathbf{z}}^{\mathrm{H}}{\Sigma }_0^{-1}\mathbf{H}{\left({\mathbf{H}}^{\mathrm{H}}{\Sigma }_1^{-1}\mathbf{H}\right)}^{-1}{\mathbf{H}}^{\mathrm{H}}{\Sigma }_1^{-1}\mathbf{z}$$

(33)

3.5. AE-Durbin Detector

The Durbin design criterion can be formulated as [30]

$$t_{\mathrm{Durbin}}={\left({\widehat{\Theta }}_{r,01}-{\Theta }_{r0}\right)}^{\mathrm{H}}\left\{{\left[\mathbf{I}\left({\widehat{\Theta }}_0\right)\right]}_{{\Theta }_r,{\Theta }_r}{\left[{\mathbf{I}}^{-1}\left({\widehat{\Theta }}_0\right)\right]}_{{\Theta }_{r,}{\Theta }_r}{\left[\mathbf{I}\left({\widehat{\Theta }}_0\right)\right]}_{{\Theta }_r,{\Theta }_r}\right\}\left({\widehat{\Theta }}_{r,01}-{\Theta }_{r0}\right)$$

(34)

where ${\widehat{\Theta }}_{r,01}$ is the MLE of ${\Theta }_r$ under hypothesis ${\mathrm{H}}_1$ when ${\Theta }_s$ is set to its MLE under hypothesis ${\mathrm{H}}_0$. Substituting (18), (22) and (27) into (34) results in the Durbin detector as

$$\begin{array}{c}\hfill t_{\mathrm{AE}-\mathrm{Durbin}}={\mathbf{z}}^{\mathrm{H}}{\widehat{\mathbf{R}}}_1^{-1}\mathbf{H}{\left({\mathbf{H}}^{\mathrm{H}}{\widehat{\mathbf{R}}}_1^{-1}\mathbf{H}\right)}^{-1}{\mathbf{H}}^{\mathrm{H}}{\widehat{\mathbf{R}}}_0^{-1}\mathbf{H}{\left({\mathbf{H}}^{\mathrm{H}}{\widehat{\mathbf{R}}}_1^{-1}\mathbf{H}\right)}^{-1}{\mathbf{H}}^{\mathrm{H}}{\widehat{\mathbf{R}}}_1^{-1}\mathbf{z}\end{array}$$

(35)

In the same way, substituting (29) and (24) into (35), simplifying with the definition of ${\Sigma }_0$ and ${\Sigma }_1$, and ignoring the constant terms yield the AE-Durbin as

$$t_{\mathrm{AE}-\mathrm{Durbin}}={\mathbf{z}}^{\mathrm{H}}{\Sigma }_1^{-1}\mathbf{H}{\left({\mathbf{H}}^{\mathrm{H}}{\Sigma }_1^{-1}\mathbf{H}\right)}^{-1}{\mathbf{z}}^{\mathrm{H}}{\Sigma }_0^{-1}\mathbf{H}{\left({\mathbf{H}}^{\mathrm{H}}{\Sigma }_1^{-1}\mathbf{H}\right)}^{-1}{\mathbf{H}}^{\mathrm{H}}{\Sigma }_1^{-1}\mathbf{z}$$

(36)

4. Performance Evaluation

We evaluate the proposed detectors with simulated and measured data in this section. To measure the relative variation of the clutter covariance matrix $\mathbf{R}$ across consecutive iterations, we define the relative variation function

$$\Delta R=\frac{\left|{∥{\widehat{\mathbf{R}}}^{\left(t\right)}∥}_{\mathrm{F}}-{∥{\widehat{\mathbf{R}}}^{(t-1)}∥}_{\mathrm{F}}\right|}{{∥{\widehat{\mathbf{R}}}^{(t-1)}∥}_{\mathrm{F}}}$$

(37)

where ${∥·∥}_{\mathrm{F}}$ represents the Frobenius norm of a matrix and ${\widehat{\mathbf{R}}}^{\left(t\right)}$ denotes the estimation of $\mathbf{R}$ after t iterations.

Figure 1 illustrates the relative change in the estimation of the covariance matrix $\mathbf{R}$ over successive iterations under hypotheses ${\mathrm{H}}_0$ and ${\mathrm{H}}_1$. Obviously, as the number of iterations increases, the degree of relative change in $\mathbf{R}$ gradually decreases. When the number of iterations exceeds 15, the degree of relative change gradually slows down. Considering the time complexity of the algorithm, the number of iterations for the power mismatches ${\gamma }_l$ and the coordinate vector $\mathit{\xi }$ is set to be 15.

Figure 1. The relative variation in the estimation of the covariance matrix across successive iterations.

4.1. Simulation Experiment

First, we set the number of system channels to 8, the number of training samples to 16, the dimension of signal subspace to 3, and the expected PFA to ${10}^{-3}$. A total of $100/\mathrm{PFA}$ Monte Carlo simulations are run to derive the detection threshold, and ${10}^4$ independent repeated tests are used to calculate the PD.

In the simulation experiment, the speckle component follows a zero-mean complex Gaussian random process, and its covariance matrix is [31]

$${\left[\mathbf{R}\right]}_{j,k}={\delta }^{\left|\left.j-k\right|\right.},1\le j,k\le N$$

(38)

where $\delta$ denotes the one-lag correlation coefficient, set to 0.95.

In this paper, the SCR is defined as [32]

$$\mathrm{SC}{\mathrm{R}}_{\mathrm{out}}={\mathit{\xi }}^{\mathrm{H}}{\mathbf{H}}^{\mathrm{H}}{\mathbf{R}}^{-1}\mathbf{H}\mathit{\xi }$$

(39)

In the experiment, we set $\mathit{\xi }=\alpha {\mathit{\xi }}_0$, where $\mathbf{H}$ and ${\mathit{\xi }}_0$ are randomly generated, but fixed after generation. $\alpha$ is the set regulatory factors to meet the given SCR as

$$\alpha =\sqrt{\frac{{\mathrm{SCR}}_{\mathrm{out}}}{{\mathit{\xi }}_0^{\mathrm{H}}{\mathbf{H}}^{\mathrm{H}}{\mathbf{R}}^{-1}\mathbf{H}{\mathit{\xi }}_0}}$$

(40)

In addition, the detection performance when the signal mismatch occurs is also discussed in this paper. For subspace signals, signal mismatch refers to the fact that the actual signal ${\mathbf{s}}_0$ is not entirely located in the assumed subspace $\mathbf{H}$. For the purpose of quantifying the degree of signal mismatch, we employ the generalized cosine square [33]

$$\mathrm{co}{\mathrm{s}}^2\varphi =\frac{{\mathbf{s}}_0^{\mathrm{H}}{\mathbf{R}}^{-1}\mathbf{H}{\left({\mathbf{H}}^{\mathrm{H}}{\mathbf{R}}^{-1}\mathbf{H}\right)}^{-1}{\mathbf{H}}^{\mathrm{H}}{\mathbf{R}}^{-1}{\mathbf{s}}_0}{{\mathbf{s}}_0^{\mathrm{H}}{\mathbf{R}}^{-1}{\mathbf{s}}_0}$$

(41)

We consider K distribution clutter, whose texture component follows a gamma distribution with the PDF as [18]

$$p_{\tau }\left(\tau \right)=\frac{1}{\Gamma \left(v\right)}{\left(\frac{v}{u}\right)}^v{\tau }^{v-1}exp\left(-\frac{v}{u}\tau \right)$$

(42)

where v is the shape parameter, representing the degree of the clutter trailing, and u is the scale parameter, representing the size of clutter power. Note that the smaller the shape parameter, the heavier the clutter trailing degree and the stronger the non-Gaussian characteristics. In the experiment, we set v to 1. Under the condition of a given texture component $\tau$, the condition PDF of the clutter amplitude is

$$f\left(r\right|\tau )=\frac{2r}{\tau }exp\left(-\frac{r^2}{\tau }\right)$$

(43)

From the full probability formula, the PDF of the clutter amplitude is

$$f\left(r\right)={\int }_0^{\infty }p\left(\tau \right)f\left(r|\tau \right)d\tau$$

(44)

Substituting (42) and (43) into (44) results in

$$\begin{array}{cc}\hfill f\left(r\right)=& {\int }_0^{\infty }p\left(\tau \right)f\left(r|\tau \right)d\tau ={\int }_0^{\infty }\frac{v^v}{\Gamma \left(v\right)u^v}{\tau }^{v-1}\frac{2r}{\tau }exp\left(-\frac{v}{u}\tau -\frac{r^2}{\tau }\right)d\tau \hfill \\ \hfill =& \frac{2}{2^{v-1}\Gamma \left(v\right)\sqrt{u}}{\left(\frac{2r}{\sqrt{u}}\right)}^v{K}_{v-1}\left(\frac{2r}{\sqrt{u}}\right)\hfill \end{array}$$

(45)

where $K_v\left(·\right)$ represents the modified Bessel function of the second kind and v-th order. The cumulative distribution function (CDF) of the K distribution can be obtained by integrating (45) as

$$F\left(r\right)=1-\frac{1}{2^{v-1}\Gamma \left(v\right)}{\left(\frac{2r}{\sqrt{u}}\right)}^v{K}_v\left(\frac{2r}{\sqrt{u}}\right)$$

(46)

In Figure 2, we use simulated data to compare the variation in PDs of the proposed detectors with respect to SCR; we can find that AE-GLRT, AE-Rao, AE-Wald, and AE-Gradient have similar PDs, but the performance of AE-Rao is slightly better than the other three proposed detectors. Specifically, when the $\mathrm{SCR}=27\mathrm{dB}$, AE-Rao can detect almost exactly for the given PFA. However, the PD of AE-Durbin is relatively low, when the SCR = 27 dB, PD is close to 0.91; when the SCR = 35 dB, PD is close to 1.

Figure 2. Probability of detection (PD) versus signal-to-clutter ratio (SCR) without signal mismatch with simulated data.

Figure 3 depicts the variation in PDs when signal mismatch occurs at SCR = 35 dB. Signal mismatch refers to the case that the actual signal ${\mathbf{s}}_0$ is not entirely located within the nominal signal subspace spanned by the columns of $\mathbf{H}$. A quantity to measure the amount of signal mismatch is given in (41). Note that the case ${cos}^2\varphi =1$ indicates that the actual signal completely lies in the signal subspace assumed by the radar system. In contrast, ${cos}^2\varphi =0$ means that the actual signal is orthogonal to the signal subspace assumed by the radar system. We use Figure 3 to measure the robustness or selectivity of the proposed detectors to mismatched signals. The robustness of a detector refers to the ability of the detector to detect targets with a high probability when signal mismatch occurs. The selectivity of a detector refers to the ability to decrease the PD of a detector when signal mismatch occurs. Figure 3 indicates that AE-Rao and AE-GLRT have better selectivity, while AE-Durbin is the most robust.

Figure 3. PD versus ${cos}^2\varphi$ with signal mismatch with simulated data. $\mathrm{SCR}=35\mathrm{dB}$.

Figure 4 demonstrates the PDs at different SNRs with v as a parameter. It is evident that as the shape parameter v decreases, indicating stronger non-Gaussian characteristics of the clutter, there is an enhancement in the PDs of all detectors. Moreover, as indicated in Figure 4a,b, the PD of AE-Rao remains the highest.

Figure 4. PD versus SCR without signal mismatch with simulated data for different v.

4.2. Real Data Experiment

We utilize a set of sea clutter measured data with a range resolution of 60 m from the IPIX database in 1998, named “19980204_223753_ANTSTEP.mat”, including 60,000 pulses and 28 range cells. We selected the 13th range cell as the CUT, divided the data within it into 200 equally spaced segments, and used the histogram density estimation method to plot the empirical PDF and CDF curves of the real data. To verify the rationality of the selected measured data, we used the fractional moment estimation method (MoFM) [34,35,36] for the real data to calculate the parameters of three common compound-Gaussian distributions, namely, the shape parameters and scale parameters of K distribution, Pareto and CG−IG distributions, and plotted the amplitude PDF and CDF curves of the three compound-Gaussian distributions. Specifically, to reduce the error caused by abnormal samples, for K-distributed clutter, we use the first moment and the half-order moment of the clutter samples for parameter estimation, which are denoted by ${\widehat{m}}_1$ and ${\widehat{m}}_{0.5}$, with the expressions as follows

$$\left\{\begin{array}{c}{\widehat{m}}_{0.5}=\frac{\Gamma \left(1.25\right)\Gamma \left(v+0.25\right)}{\Gamma \left(v\right)}{\left(2u\right)}^{\frac{1}{2}}\hfill \\ {\widehat{m}}_1=\frac{\Gamma \left(1.5\right)\Gamma \left(v+0.5\right)}{\Gamma \left(v\right)}\left(2u\right)\hfill \end{array}\right.$$

(47)

where (47) serves as an implicit function of v. It is worth mentioning that the gamma function is defined as $\Gamma \left(z\right)={\int }_0^{+\infty }{t}^{z-1}{e}^{-t}dt$. So we can eliminate u, unfold, and simplify (47), learning that v has a unique solution. Next, we use the lookup table method or employ the $fsolve\left(·\right)$ function in MATLAB to find the solution. As shown in Figure 5 and Figure 6, the K distribution provides a better fit to the real data.

Figure 5. Compound-Gaussian distribution PDF fitting to empirical PDF of real data.

Figure 6. Compound-Gaussian distribution CDF fitting to empirical CDF of real data.

To further measure the fitting degree of the three compound-Gaussian distributions mentioned above to the real data, Kolmogorov-Smirnov (KS) distance is commonly used as a metric. In addition, we plot the fitting results of the clutter amplitude distribution of real data in Figure 7. Here, the vertical axis represents the $1-\mathrm{CDF}$ domain, which is equivalent to the meaning of the PFA [37]. As the PFA we set is ${10}^{-2}$, we narrow the scope of the KS test to around ${10}^{-2}$, and the corresponding expression of the KS distance can be written as

$$KS(v,u;\widehat{v},\widehat{u})=\underset{r\in (2.5,4.5)}{max}\left\{\left|\left.F(r;v,u)-F(r;\widehat{v},\widehat{u})\right|\right.\right\}$$

(48)

where $F(r;v,u)$ represents the empirical CDF obtained from the real data and $F(r;\widehat{v},\widehat{u})$ represents the fitting CDF obtained through parameter estimation based on the selected distribution model. Table 1 indicates that the KS distance for the K distribution fit is 0.0011 for the PFA we set. As we know, a smaller KS distance indicates a better fit between the empirical CDF of clutter amplitude and the selected clutter model. So, we can draw a conclusion that the K distribution still offers a good fit. It is worth noting that using real data with good fitting results from the K distribution for experiments aims to correspond better with simulation experiments. In this paper, the power mismatch of the training samples obtained through iterative estimation does not need to consider the specific distribution of clutter.

Figure 7. Compound-Gaussian distribution modeling results of real data.

Table 1. Kolmogorov-Smirnov (KS) distances for each distribution fit.

Distribution	KS Distance
K distribution	0.0011
Pareto distribution	0.0066
CG-IG distribution	0.0013

We set $\mathrm{PFA}={10}^{-2}$ due to the limited availability of pulses in the real data, and selected the $\left(13-L/2,12\right)$ and $\left(14,13+L/2\right)$ range cells near the CUT as training samples. The other parameters are consistent with the simulation experiment.

In Figure 8, we graph the PDs of the proposed detectors across various SCRs in the absence of signal mismatch. The observation reveals that the results from the real data are basically consistent with the simulation experiment. The detection performances of AE-GLRT, AE-Rao, AE-Wald, and AE-Gradient are comparable and significantly better than AE-Durbin. Specifically, AE-Rao still has the highest PD, and the performances of AE-Wald and AE-Gradient slightly surpass that of AE-GLRT.

Figure 8. PD versus SCR without signal mismatch with IPIX radar data.

In Figure 9, utilizing the measured data from the IPIX radar, we compare and analyze the variation in the PDs of the proposed detectors in relation to the degree of signal mismatch. It can be observed that the result of the measured data is also basically consistent with the simulation experiment, indicating that AE-Durbin has the strongest robustness; when the signal mismatch angle is approximately 53 degrees, AE-Durbin can still detect the target with a PD of 0.9, while both AE-GLRT and AE-Rao have good selectivity, but AE-GLRT has stronger selectivity, indicating a better ability to suppress mismatched signals.

Figure 9. PD versus ${cos}^2\varphi$ with signal mismatch with IPIX radar data. $\mathrm{SCR}=25\mathrm{dB}$.

Meanwhile, we also conduct experiments with “19980204_224024_ANTSTEP.mat” real data, which is well fitted by the Pareto distribution, and the results are basically consistent with the above. We can reasonably infer that the detectors proposed in this paper have a certain universality in power heterogeneous clutter environments.

5. Conclusions

In this article, we investigated the adaptive detection problem of subspace signals within power heterogeneous clutter environments. Based on the GLRT, Rao, Wald, Gradient, and Durbin criteria, we used an alternate estimation iterative algorithm to design five adaptive detectors: AE-GLRT, AE-Rao, AE-Wald, AE-Gradient, and AE-Durbin. The simulation experiments and measured data illustrated that AE-Rao had a relatively higher PD when there was no signal mismatch. However, in the case of signal mismatch, AE-GLRT had the best selectivity, whereas AE-Durbin had the most robust detection performance.