Design of Selective Detector for Distributed Targets Through Stochastic Characteristic of the Fictitious Signal

Abstract

We investigate the problem of detecting the distributed targets buried in the Gaussian noise whose covariance matrix is unknown when signal mismatch occurs. The idea is to add a fictitious signal under the null hypothesis of the origin detection problem so that when signal mismatch occurs, the fictitious signal captures the mismatched signals, thus making the null hypothesis more plausible. More precisely, the fictitious signal is modeled as a Gaussian component with a covariance matrix of a stochastic factor multiplied by a rank-one matrix. The generalized likelihood ratio test (GLRT) is employed to address the modification detection problem. We present an exhaustive derivation of the detector and prove that it possesses the constant false alarm rate (CFAR) property. The performance analysis demonstrates the effectiveness of the proposed detector. When the SNR is 23 dB, as generalized cosine squared decreases from 1 to 0.83, the detection probability of the proposed GLRT-SL drops to 0.65, exhibiting the fastest decline compared to the G-ABORT-HE, which falls to 0.98, and the GW-ABORT-HE, which decreases to 0.85.

1. Introduction

The primary mission of a radar system is to detect and track targets of interest within an intricate environment rife with noise, clutter, and potential interference [1,2,3,4]. In essence, radar target detection is based on a detector. One would like a detector to deliver as high a probability of detection as possible under matched conditions, i.e., when the actual steering vector coincides with the nominal one, while minimizing errors. Also, it is usually desirable to have a detector with the constant false alarm (CFAR) property that allows the desired threshold to be set without considering the statistics of the noise. Naturally, a criterion called the generalized likelihood ratio test (GLRT) that meets the above two objectives was proposed for solving the problem of point-like target detection in Gaussian noise [5]. Kelly also derived the statistical distribution of the GLRT statistic under the signal-present hypothesis and the null hypothesis and conducted a performance analysis. It is worth nothing that any detector based on Kelly’s GLRT statistic naturally inherits the CFAR property and is more convenient for performance evaluation [6]. In addition, there are some other well-known detectors for the classical detection problem, such as adaptive match filter (AMF) [7] and De Maio’s Rao (DMRao) test [8]. The Wald test proposed in [9] has identical detection statistics to the AMF.

In practice, when the target is large or the radar resolution is high, the echoes reflected from the target could occupy multiple range cells, in which case the point-like target model becomes insufficient [10]. The target of this nature is commonly referred to as a distributed target. The problem of distributed target detection has been extensively studied. In [11], the GLRT and two-step GLRT for distributed targets embedded in Gaussian noise with an unknown covariance matrix were proposed. In [12], two-step variants of GLRT, Rao, and Wald criteria were employed to address the distributed target detection in compound-Gaussian sea clutter. In this work, for the purpose of reducing the amount of training data, the covariance matrix is assumed to possess a persymmetric structure, which is estimated utilizing the fixed-point estimator. The GLRT-based detectors were derived in [13] for distributed target detection in the presence of partially homogeneous noise plus subspace interference. Related studies on distributed MIMO radar can be found in [14,15,16]. It was mentioned in [17] that as radar resolution increases, distributed targets exhibit weaker perturbations compared to point-like targets, which is one of the factors that brings about performance improvement. It naturally follows that the number of unknowns in the classical problem model grows with the range cells. In contrast to the usual assumption of deterministic target amplitude in the classical problem, a target model with a constant number of unknowns is proposed in [18], which treats the target amplitude as a stochastic variable that follows a zero-mean Gaussian distribution with unknown variance. Specifically, the variance of the observation matrix under signal-present hypothesis is given by $\Sigma +Pv{v}^H$, where $v$ is the target steering vector, $\Sigma$ represents the noise covariance matrix and $P$ is a positive stochastic variable. The stochastic variable $P$ eliminates the drawback of having a large number of unknown amplitudes in classical distributed target models. The statistical distribution form of binary hypothesis test can be expressed as

$$\left\{\begin{array}{l}{\mathrm{H}}_0:X~\mathcal{C}{\mathcal{N}}_{N,K}\left(0,\Sigma ,I_K\right)\hfill \\ {\mathrm{H}}_1:X~\mathcal{C}{\mathcal{N}}_{N,K}\left(0,\Sigma +Pv{v}^H,I_K\right)\hfill \end{array}\right.$$

(1)

This newly derived GLRT performs better than the natural competitors under the usual target model for a small number of training data. Further research on this stochastic rank-one model can be found in [19,20,21,22].

The majority of the aforementioned research has been carried out with the underlying assumption of the absence of signal mismatch. However, signal mismatch inevitably occurs due to uncalibrated arrays, uncertainties about the direction of arrival of the target and the possible sidelobe interference, etc., [23,24,25]. Detectors can typically be categorized into two types based on their sensitivity against mismatched signals: selective detectors and robust detectors. The characteristic of a selective detector is that the detection probability decreases rapidly with the increase in mismatch. In contrast, a robust detector maintains a high probability of detection. In some practical scenarios, such as target orientation, it is more appropriate for the detector to possess some selectivity, whereas when the radar is operating in search mode, a robust detector is more desirable. Hence, many studies have investigated how to improve the selectivity or robustness of detectors against signal mismatch.

A common and dominant selective detector design technique is to add a fictitious signal under the null hypothesis and assume that this fictitious signal is orthogonal to the nominal steering vector in truly whitened space or quasi-whitened space [26,27]. When signal mismatch occurs, the whitened or quasi-whitened component of the mismatched signals in the direction of the fictitious signal causes the detector to prefer the null hypothesis and is less likely to declare a detection. Many selective detectors have been designed based on these ABORT-like idea [28,29,30,31]. In [28], the idea of W-ABORT was relaxed by introducing a Bernoulli random variable multiplied by a fictitious signal; thereby, an adjustable detector with an improved detection vs. rejection tradeoff was proposed. The promotion form of W-ABORT in a partially homogeneous environment was reported in [29]. In [30], the generalized ABORT was derived for the distributed targets, but its selectivity is not strong enough. To address the shortage of training samples, probabilistic modeling was adopted for fictitious signals in [31], and a Bayesian framework was introduced in the design phase. In [32], a selective detector called the double-normalized adaptive matched filter (DN-AMF) was proposed with the idea of assuming that both the signal-present hypothesis and null hypothesis contain noise-like interference and a suitable design criterion is used. It should be noted that this approach seems to lack a reasonable explanation and largely depends on a specific design criterion. For approaches to enhance robustness, see [33,34,35,36].

Notice that the ABORT-like idea is to model the fictitious signal injected under the null hypothesis as a deterministic unknown quantity. To the best of our knowledge, the approach to treating it as a stochastic quantity has not been considered. Inspired by the rank-one stochastic signal model in [18] and the ABORT-like idea, we investigate a possible approach to selective detector design for distributed targets. The proposed idea is to introduce a fictitious signal under the null hypothesis that obeys a complex Gaussian distribution with zero mean and variance of an unknown stochastic variable multiplied by a known rank-one matrix. The rank-one matrix is composed of the mismatched steering vector around the target steering vector. In fact, this model is a possible generalization of the fictitious signal introduced in [26,27] and the stochastic target model $Pv{v}^H$ in [18]. Furthermore, this fictitious signal does not exhibit any form of orthogonality with the target steering vector. When signal mismatch occurs, the leaked signal is precisely captured by the stochastic fictitious signal, which makes the null hypothesis more plausible. We derive the detector using the GLRT and show that it possesses the CFAR property. The proposed GLRT demonstrates the highest selectivity compared to its natural competitors when the fictitious steering vector is a slightly mismatched target steering vector.

2. Problem Formulation

Assume that the radar system receives $N$ pulses in a pulse processing interval (CPI). The echoes reflected from the target occupy consecutive $K$ range cells, which are usually referred to as the test data. The test data of the kth range cell are recorded as $x_k\in {ℂ}^{N\times 1}$, $k=1,\cdots ,K$. For a binary hypothesis test of an injected fictitious signal, under the null hypothesis, $x_k$ contains noise $n_k$ and the fictitious signal ${\xi }_k$. Conversely, under the hypothesis of the signal being present, $x_k$ is composed of the target signal and noise. Specifically, the structure of the target signal is expressed as $a_k^{*}h$, where $a_k^{*}$ stands for the unknown target amplitude and $h$ denotes the known normalized target steering vector. Hence, the binary hypothesis test can be written as

$$\left\{\begin{array}{l}{\mathrm{H}}_0:x_k={\xi }_k+n_k;k=1,\dots ,K\hfill \\ {\mathrm{H}}_1:x_k=a_k^{*}h+n_k;k=1,\dots ,K\hfill \end{array}\right.$$

(2)

Further, we assume that the injected fictitious signal ${\xi }_k$, $k=1,\cdots ,K$ is independently and identically distributed (IID) and distributed as ${\xi }_k~\mathcal{C}{\mathcal{N}}_N\left(0,Pd{d}^H\right)$, where $P$ is an unknown positive factor, $d\in {ℂ}^{N\times 1}$ is a known column vector indicating the mismatched steering vector around the target steering vector $h$. The rank-one matrix $d{d}^H$ reflects our knowledge of the stochastic variations around $h$. When signal mismatch occurs, $d{d}^H$ collects some, if not the whole, of the leaked steering vector that is different from $h$, and the factor $P$ captures the corresponding target amplitude. As a result, a detector is more inclined to decide the null hypothesis. In addition, the noise $n_k$ is usually assumed to have an IID complex Gaussian distribution with zero mean and variance of a positive definite matrix $\Sigma$, i.e., $n_k~\mathcal{C}{\mathcal{N}}_N\left(0,\Sigma \right)$. Here, we consider the case where $\Sigma$ is unknown. To estimate the noise covariance matrix $\Sigma$, the training data collected in the vicinity of the test data is in used, which is usually assumed to contain only noise and share the statistical characteristics of the noise with the test data. Suppose there are $L$ training data, which is denoted as $y_{e,l}$, $l=1,\cdots ,L$, then accordingly its distribution is written as $y_{e,l}~\mathcal{C}{\mathcal{N}}_N\left(0,\Sigma \right)$. Finally, assume that $L>N$. Based on the above assumptions, we summarize as

$$\left\{\begin{array}{l}{\mathrm{H}}_0:\left\{\begin{array}{l}X~\mathcal{C}{\mathcal{N}}_{N,K}\left(0,\Sigma +Pd{d}^H,I_K\right)\hfill \\ Y~\mathcal{C}{\mathcal{N}}_{N,L}\left(0,\Sigma ,I_L\right)\hfill \end{array}\right.\hfill \\ {\mathrm{H}}_1:\left\{\begin{array}{l}X~\mathcal{C}{\mathcal{N}}_{N,K}\left(h{a}^H,\Sigma ,I_K\right)\hfill \\ Y~\mathcal{C}{\mathcal{N}}_{N,L}\left(0,\Sigma ,I_L\right)\hfill \end{array}\right.\hfill \end{array}\right.$$

(3)

where $X=\left[x_1,\cdots ,x_K\right]\in {ℂ}^{N\times K}$, $Y=\left[y_{e,1},y_{e,2},\cdots ,y_{e,L}\right]\in {ℂ}^{N\times L}$, and $a={\left[a_1,a_2,\cdots ,a_K\right]}^T\in {ℂ}^{K\times 1}$.

Notations: Matrices are represented by bold uppercase letters, vectors by bold lowercase letters, and scalars by lowercase letters. ${ℂ}^{p\times q}$ stands for the complex matrix space with dimension $p\times q$. For a matrix $\Sigma \in {ℂ}^{N\times N}$, $\mathrm{tr}\left\{\Sigma \right\}$, $\mathrm{etr}\left\{\Sigma \right\}$, and $\left|\Sigma \right|$ are the trace of $\Sigma$, the exponential of the trace of $\Sigma$, the determinant of $\Sigma$, respectively. ${\left(\cdot \right)}^{*}$, ${\left(\cdot \right)}^T$ and ${\left(\cdot \right)}^H$ represent conjugate, transpose and conjugate transpose, respectively. If $\Sigma$ is a nonsingular matrix, ${\Sigma }^{-1}$ denotes its inverse. If $\Sigma$ is a positive definite matrix, ${\Sigma }^{1/2}$ stands for its Hermitian square root. $I_N$ is an $N\times N$ identity matrix. $P_h=h{h}^H{\left(h^{H}h\right)}^{-1}$ is the orthogonal projection matrix onto the subspace spanned by $N\times 1$ column vector $h$, and $P_h^{\perp }=I_N-P_h$. $Z~\mathcal{C}{\mathcal{N}}_{N,K}\left(M,{\Sigma }_A,{\Sigma }_B\right)$ denotes that the matrix follows the complex matrix variate Gaussian distribution with probability density function (PDF) $f\left(Z\right)={\pi }^{-NK}{\left|{\Sigma }_A\right|}^{-K}{\left|{\Sigma }_B\right|}^{-N}\mathrm{etr}\left\{-{\left(Z-M\right)}^H{\Sigma }_A^{-1}\left(Z-M\right){\Sigma }_B^{-1}\right\}$. As for the complex (vector) Gaussian distribution, we note as $z~\mathcal{C}{\mathcal{N}}_N\left(a,{\Sigma }_a\right)$, where $z$ and $a$ are the $N\times 1$ column vectors. $\mathcal{C}{\mathcal{W}}_N\left(K,{\Sigma }_t\right)$ stands for the $N\times N$ complex Wishart distribution with $K$ degrees of freedom and scale matrix ${\Sigma }_t$.

3. Detector Design

The design criteria of GLRT for the detection problem in (2) can be expressed as

$$t_{\mathrm{GLRT}}={\displaystyle \frac{\underset{\Sigma ,a}{\max }{f}_1\left(X,Y\right)}{\underset{\Sigma ,P}{\max }{f}_0\left(X,Y\right)}}$$

(4)

where

$$f_1\left(X,Y\right)={\pi }^{-N\left(L+K\right)}{\left|\Sigma \right|}^{-\left(L+K\right)}\mathrm{etr}\left\{-{\Sigma }^{-1}\left[S+\left(X-h{a}^H\right){\left(X-h{a}^H\right)}^H\right]\right\}$$

(5)

and

$$f_0\left(X,Y\right)={\pi }^{-N\left(L+K\right)}{\left|\Sigma \right|}^{-L}{\left|\Sigma +Pd{d}^H\right|}^{-K}\mathrm{etr}\left\{-{\Sigma }^{-1}S-{\left(\Sigma +Pd{d}^H\right)}^{-1}X{X}^H\right\}$$

(6)

represent the joint PDF of $X$ and $Y$ under ${\mathrm{H}}_1$ and ${\mathrm{H}}_0$, respectively. $S$ is defined as $L$ times sample covariance matrix (SCM), i.e., $S={\displaystyle {\sum }_{l=1}^L{y}_{e,l}{y}_{e,l}^H}=Y{Y}^H$.

The detailed derivation procedure can be found in Appendix A. Here, we directly present the final GLRT as

$$t_{\mathrm{GLRT}}=t_{\mathrm{GLRT}\text{-}\mathrm{HE}}\times \underset{0<\mu <1}{\min }{\mu }^{-{\displaystyle \frac{K}{L+K}}}\left[{\displaystyle \frac{d^H{\left(S+X{X}^H\right)}^{-1}d}{d^H{\left(S+\mu X{X}^H\right)}^{-1}d}}\right]$$

(7)

where $t_{\mathrm{GLRT}\text{-}\mathrm{HE}}$ is the one-step GLRT for homogeneous environment, which is equivalent to Kelly’s GLRT when $K=1$. The statistic of the GLRT-HE is denoted as [11]

$$t_{\mathrm{GLRT}\text{-}\mathrm{HE}}={\displaystyle \frac{\left|I_K+{\tilde{X}}^H\tilde{X}\right|}{\left|I_K+{\tilde{X}}^H{P}_{\tilde{h}}^{\perp }\tilde{X}\right|}}$$

(8)

where $\tilde{X}=S^{-1/2}X$, $\tilde{h}=S^{-1/2}h$, $P_{\tilde{h}}=\tilde{h}{\tilde{h}}^H/\left({\tilde{h}}^H\tilde{h}\right)$, and $P_{\tilde{h}}^{\perp }=I_N-P_{\tilde{h}}$.

Remark 1. 

The GLRT statistic is constituted by the product of two terms. The first term is the GLRT-HE statistic, and the second term is a correction term arising from the stochastic nature of the injected fictitious signal. When the correction term approaches 1, it can be deduced that the estimate of $\mu$ is approximately equal to 1. In such a scenario, $d$ exerts minimal influence on the GLRT, and the overall performance of the GLRT remains comparable to that of the GLRT-HE. In contrast, when $\mu$ tends to 0, under the assumption that $X$ contains a strong target, the denominator of the correction term is more likely to eliminate it, and $d$ will have a stronger effect on the selectivity of the GLRT. Note that $d\ne h$ and $d$ are not related to $h$ in any orthogonal form.

Remark 2. 

The presence of the minimum optimization problem in the correction term introduces additional computational overhead for the GLRT compared to the GLRT-HE. However, this overhead is not considered to be significant. The correction term in (7) can be further expanded as 

$${\widehat{\mu }}_0=\underset{0<\mu <1}{\min }{\mu }^{-{\displaystyle \frac{K}{L+K}}}\left[{\displaystyle \frac{d^H{S}^{-1}d-d^H{S}^{-1}X{\left(I_K+X^H{S}^{-1}X\right)}^{-1}{X}^H{S}^{-1}d}{d^H{S}^{-1}d-\mu d^H{S}^{-1}X{\left(I_K+\mu X^H{S}^{-1}X\right)}^{-1}{X}^H{S}^{-1}d}}\right]$$

(9)

which can be efficiently computed by performing a grid search on $\mu$. Another equivalent form of the GLRT is as follows 

$$t_{\mathrm{GLRT}}={\displaystyle \frac{\left|I_K+{\tilde{X}}^H{P}_{\tilde{d}}^{\perp }\tilde{X}\right|}{\left|I_K+{\tilde{X}}^H{P}_{\tilde{h}}^{\perp }\tilde{X}\right|}}\times \underset{0<\mu <1}{\min }{\mu }^{-{\displaystyle \frac{K}{L+K}}}{\displaystyle \frac{\left|I_K+\mu {\tilde{X}}^H\tilde{X}\right|}{\left|I_K+\mu {\tilde{X}}^H{P}_{\tilde{d}}^{\perp }\tilde{X}\right|}}$$

(10)

For the optimization problem in determinant-based GLRT, an efficient computation approach is to use the fact that 

$$\left|I_K+\mu M\right|={\displaystyle {\prod }_{j=1}^K\left[1+\mu {\lambda }_j\left(M\right)\right]}$$

(11)

where ${\lambda }_j\left(M\right)$ denotes the eigenvalues of $M$.

Remark 3. 

The distribution of the GLRT statistic is independent of $\Sigma$ under the null hypothesis, i.e., the GLRT is CFAR against $\Sigma$. For proof, see Appendix B.

4. Performance Analysis

We employ Monte Carlo simulation to assess the behavior of the designed detector. The probability of false alarm (PFA) is fixed at $\mathrm{PFA}={10}^3$. In order to determine the detection threshold and the probability of detection (PD), $100/\mathrm{PFA}$ and ${10}^4$ independent trials are performed, respectively. For the covariance matrix, the standard exponentially correlated model is considered, where the $\left(i,j\right)$th element of $\Sigma$ is denoted as ${\epsilon }^{\left|i-j\right|}$, $i,j=1,2,\cdots ,N$, where $\epsilon$ is the one-lag correlation coefficient. For the training data $Y$, we construct it as $Y={\Sigma }^{1/2}{W}_L$, where $W_L$ is a complex Gaussian white noise with $N\times L$ dimensions and variance of 1. Similarly, the noise term is generated by ${\Sigma }^{1/2}{W}_K$, where $W_K$ is a complex Gaussian white noise with $N\times L$ dimensions and variance of 1. The structure of the target steering vector is

$$h={\displaystyle \frac{1}{\sqrt{N}}}{\left[1,{\mathrm{e}}^{-j2\pi \overline{f}},\cdots ,{\mathrm{e}}^{-j2\pi \left(N-1\right)\overline{f}}\right]}^T$$

(12)

where $j$ represents the imaginary unit and $\overline{f}$ is the normalized Doppler frequency. Under mismatched conditions, the normalized frequency of the actual steering vector $h_0$ is $\overline{f}+\delta /N$. To better quantify the degree of mismatch between $h$ and $h_0$, the widely used generalized cosine squared (GCS) is introduced below with [26]

$${\cos }^2\theta ={\displaystyle \frac{{\left|h_0^H{\Sigma }^{-1}h\right|}^2}{h_0^H{\Sigma }^{-1}{h}_0{h}^H{\Sigma }^{-1}h}}$$

(13)

where $\theta$ represents the angle between the whitened versions of $h$ and $h_0$. It is straightforward to conclude that $\delta =0$ indicates no mismatch, corresponding to ${\cos }^2\theta =1$. Similarly, the mismatched steering vector $d$ can be configured through $\overline{f}+{\delta }_{\mathrm{d}}/N$, with its GCS defined as

$${\cos }^2\varphi ={\displaystyle \frac{{\left|d^H{\Sigma }^{-1}h\right|}^2}{d^H{\Sigma }^{-1}d{h}^H{\Sigma }^{-1}h}}$$

(14)

where $\varphi$ represents the angle between the whitened versions of $h$ and $d$. The signal to noise ratio (SNR) is defined as

$$\mathrm{SNR}=a^{H}a{h}_0^H{\Sigma }^{-1}{h}_0$$

(15)

For comparison, the generalized ABORT and its whitened version in homogeneous environment are considered due to their ability to reject mismatched signals. In the subsequent simulations, these two detectors are recorded as G-ABORT-HE [30] and GW-ABORT-HE [28], respectively. The detection statistics of these two selective detectors are given by

$$t_{\mathrm{G}\text{-}\mathrm{ABORT}\text{-}\mathrm{HE}}={\displaystyle \frac{\left|I_K+{\tilde{X}}^H{P}_{\tilde{h}}\tilde{X}\right|}{\left|I_K+{\tilde{X}}^H{P}_{\tilde{h}}^{\perp }\tilde{X}\right|}}$$

(16)

and

$$t_{\mathrm{GW}\text{-}\mathrm{ABORT}\text{-}\mathrm{HE}}={\displaystyle \frac{\left|I_K+{\tilde{X}}^H\tilde{X}\right|}{{\left|I_K+{\tilde{X}}^H{P}_{\tilde{h}}^{\perp }\tilde{X}\right|}^2}}$$

(17)

respectively. Additionally, since the proposed GLRT includes the GLRT-HE statistic, the GLRT-HE is also introduced [11], primarily for comparison of matched detection performance. Unless otherwise specified, the settings $N=16$, $\overline{f}=0.08$, and $\epsilon =0.9$ are consistently applied.

Figure 1 depicts the PD of the proposed GLRT under different $d$. The result indicates that as the degree of mismatch of $d$ decreases (i.e., ${\cos }^2\varphi$ increases), the PD of the proposed GLRT initially exhibits a gradual decline, followed by a rapid drop. Compared to the GLRT-HE, the proposed GLRT exhibits a certain performance loss when ${\cos }^2\varphi$ is large, whereas it has a similar PD as the GLRT-HE or even a slight performance gain when ${\cos }^2\varphi$ is small. Regardless of what degree $d$ is the mismatched steering vector around $h$, the proposed GLRT consistently outperforms the G-ABORT-HE and GW-ABORT-HE. In view of the above results, in subsequent simulations, we then consider three scenarios for $d$: slight mismatch, moderate mismatch, and severe mismatch, labeled as GLRT-SL, GLRT-MO, and GLRT-SE, respectively. More specifically, we select ${\delta }_{\mathrm{d}}=0.2$, 0.4 and 0.6, corresponding to ${\cos }^2\varphi =0.83$, 0.46, and 0.14.

Figure 2 reports the PD versus SNR under matched conditions. It is clear that the GLRT-SE has the best behavior at $L=1.5N$, with a slight performance overrun compared to the GLRT-HE. As $L$ increases to $2.5N$, the PD of the GLRT-SE and GLRT-HE becomes nearly identical. The PD of GLRT-MO is between that of the GLRT-SL and GLRT-SE, and the GLRT-SE is more powerful than the G-ABORT-HE and GW-ABORT-HE. The findings align with those depicted in Figure 1.

The PD versus ${\cos }^2\theta$ for $\mathrm{SNR}=23\mathrm{dB}$ is reported in Figure 3. As observed, as the degree of signal mismatch intensifies, the GLRT-SL exhibits the fastest decline, indicating that it has the strongest selectivity compared to its natural competitors. The GLRT-MO demonstrates greater selectivity than the G-ABORT-HE but is less selective than the GW-ABORT-HE. For the GLRT-SE, the decrease in PD is slower than that of the G-ABORT-HE when ${\cos }^2\theta >0.65$ (corresponding to $\delta =0.3$), and the GLRT-SE no longer provides a PD above 0.5 when ${\cos }^2\theta \le 0.65$. Therefore, the selectivity of GLRT-SE is inferior to that of the G-ABORT-HE. Notably, compared to the other two scenarios, considering $d$ as a slightly mismatched steering vector around $h$ enhances the ability to capture mismatched signals. As a result, when signal mismatch occurs, the detector is more likely to favor the null hypothesis, leading to the fastest rate of decline observed in the figure.

Figure 4 further presents the selectivity of the proposed GLRT against mismatched signals. The mesa plot was first depicted in [26] to describe the PD of a detector under different SNR and ${\cos }^2\theta$. Along a fixed SNR, the faster the PD decreases as ${\cos }^2\theta$ diminishes, the stronger the selectivity of a detector against mismatched signals. Overall, as the mismatch increases, the rate of PD decline, from fastest to slowest, follows the order of the GLRT-SL, GW-ABORT-HE, GLRT-MO, G-ABORT-HE, GLRT-SE, and GLRT-HE. This result is fully consistent with that observed in Figure 3. Note that among the detectors corresponding to the three cases of $d$, the GLRT-SL exhibits the strongest selectivity but has the lowest PD for matched targets, while the GLRT-SE is the most powerful but less selective compared to the other two. This indicates that setting $d$ as a smaller mismatched steering vector around $h$ enhances selectivity at the cost of sacrificing matched detection performance. Moreover, the GLRT-SL outperforms the GW-ABORT-HE in both selectivity and matched detection performance, and the GLRT-MO similarly outperforms the G-ABORT-HE. These results highlight the advantage of the approach outlined in model (2) over the ABORT-like idea in designing selective detectors for distributed targets.

In order to gain further insight into the behavior of the proposed GLRT, we consider a lower value of $K$. The PD versus SNR for $K=2$ under matched conditions is reported in Figure 5, and the comparison mesa plots under different values of $K$ are depicted in Figure 6. Compared to Figure 2b, the GLRT-SL experiences a more significant performance loss, with its PD falling below that of G-ABORT-HE. The gap between the detectors is also noticeably smaller. The results in Figure 6 reveal that as $K$ decreases, the selectivity of the proposed GLRT slightly improves.

Figure 7 shows the CFAR property of the proposed GLRT against the noise covariance matrix. No matter how the correlation coefficient $\epsilon$ changes between 0.1 and 0.9, the PFA hardly changes, which indicates that the proposed GLRT is CFAR.

Table 1. Computation complexity of the proposed and existing detectors.

Detectors	Running Time(s)
GLRT-HE	23.9290
G-ABORT-HE	24.4529
GW-ABORT-HE	24.1048
Proposed GLRT	32.9585

reports the average running time of 20 experiments for both existing and proposed detectors. Specifically, we set ${\cos }^2\theta =1$, $\mathrm{SNR}=15\mathrm{dB}$, and $K=2$, with the same parameters as above. It is obvious that the GLRT-HE has the shortest running time, while the proposed GLRT has the highest running overhead. This is due to the need to perform a grid search on $\mu$ to find the minimum value of (9). Moreover, observing (8), (16), and (17), it is evident that they solely consist of components ${\tilde{X}}^H\tilde{X}$ and ${\tilde{X}}^H{P}_{\tilde{h}}^{\perp }\tilde{X}$, and they are all designed based on GLRT. Therefore, the computation complexity among the G-ABORT-HE, GW-ABORT-HE and GLRT-HE is nearly indistinguishable.

5. Conclusions

In this paper, we investigate the design of selective detectors for distributed targets embedded in Gaussian noise. The idea is to introduce a fictitious signal under the null hypothesis and model it as a stochastic quantity that follows a complex Gaussian distribution with zero mean and variance of an unknown stochastic factor multiplied by a known rank-one matrix. When signal mismatch occurs, the stochastic fictitious signal captures leaked signals, making the null hypothesis more plausible. Using the GLRT criterion, we derive the corresponding detector and prove its CFAR property. The simulation results reveal that, for example, at $\mathrm{SNR}=23\mathrm{dB}$, the GLRT-SL exhibits the strongest selectivity, even surpassing the GW-ABORT-HE, while at $K=2$, its detection probability is significantly weaker than G-ABORT-HE in the SNR range of 10 dB to 20 dB. The selectivity of GLRT-SE is weaker than G-ABORT-HE under $\mathrm{SNR}=23\mathrm{dB}$, while it achieves the best performance in matched target detection; in particular, when $L=1.5N$, the GLRT-SE has the highest detection probability in the range of 13 dB to 20 dB. The behavior of the GLRT-MO lies between these two detectors. Notably, reducing the mismatch between $d$ and the target steering vector enhances selectivity, but at the expense of matched detection performance. Furthermore, the GLRT-SL is superior to the GW-ABORT-HE in both selectivity and matched detection performance, and the GLRT-MO similarly outperforms the G-ABORT-HE. This result demonstrates the benefits of the proposed approach compared to the ABORT-like strategy.