Target Detection Adapting to Spectral Variability in Multi-Temporal Hyperspectral Images Using Implicit Contrastive Learning

Abstract

Hyperspectral target detection (HTD) is a crucial aspect of remote sensing applications, aiming to identify targets in hyperspectral images (HSIs) based on their known prior spectral signatures. However, the spectral variability resulting from various imaging conditions in multi-temporal hyperspectral images poses a challenge to both classical and deep learning (DL) methods. To overcome the limitations imposed by spectral variability, an implicit contrastive learning-based target detector (ICLTD) is proposed to exploit in-scene spectra in an unsupervised way. First, only prior spectra are utilized for explicit supervision, while an implicit contrastive learning module (ICLM) is designed to normalize the feature distributions of prior and in-scene spectra. This paper theoretically demonstrates that the ICLM can transfer the gradients from prior spectral features to those of in-scene spectra based on their feature similarities and differences. Because of transferred gradient signals, the ICLTD is regularized to extract similar representations for the prior and in-scene target spectra, while augmenting feature differences between the target and background spectra. Additionally, a local spectral similarity constraint (LSSC) is proposed to enhance the capability of scene adaptation by leveraging the spectral similarities among in-scene targets. To validate the performance of the ICLTD under spectral variability, multi-temporal HSIs captured under various imaging conditions are collected to generate prior spectra and in-scene spectra. Comparative evaluations against several DL detectors and classical methods reveal the superior performance of the ICLTD in achieving a balance between target detectability and background suppressibility under spectral variability.

1. Introduction

With the advancement of hyperspectral imaging techniques, the spectral and spatial characteristics of hyperspectral images (HSIs) have found applications in remote sensing observations, including military defense [1], mineral exploration [2], and agricultural monitoring [3]. Hyperspectral target detection (HTD), which stands out as a critical technique for interpreting remote sensing HSIs, aims to identify the target of interest within the test HSIs given their known waveform signatures. This paper focuses on HTD in the radiance domain where the prior spectra and test HSIs are collected from multi-temporal images. For clarity of description, “in-scene HSIs” refers to the test images in this paper. An example shown is in Figure 1, where the diverse imaging conditions of the multi-temporal HSIs lead to variations between the prior and in-scene target spectra, known as spectral variability [4]. This variability presents a significant obstacle to achieving robust detection performance. Although the in-scene radiative parameters could be potentially estimated to address radiance variations [5], such estimation necessitates the presence of typical materials in the in-scene HSIs and prior knowledge of their spectra. This work aims to mitigate the limitations posed by spectral variability and realize robust detection performance without prior knowledge of radiative parameters.

In the last few decades, numerous classical methods have been developed for HTD. Spectral angle mapper [6] and spectral information divergence [7] are two well-known distance-based metrics used to assess the similarities between prior and in-scene spectra. Signal detection-based approaches, such as the generalized likelihood ratio test [8] and the adaptive coherence/cosine estimator (ACE) [9,10], employ hypothesis tests to model HTD. Constrained energy minimization (CEM) [11] can be used to construct a detector that minimizes the output energy while preserving prior spectra. To overcome the limitations of the distribution assumption, Chang et al. [12] proposed more general theories based on the signal-to-noise ratio and spectral angle, including the spectral angle in the correlated space (R-SA). In addition, innovative approaches have been integrated into classical methods to address limitations arising from model complexity. These include kernel methods [13], hierarchical structures [14,15,16], and fractional Fourier transforms [17]. To utilize in-scene spectra better, spectral unmixing techniques [18,19] and sparsity assumptions [20,21,22,23,24] are introduced to construct hyperspectral target detectors, which require certain assumptions.

In the past decade, deep learning (DL) models have achieved great success in HSI interpretation based on their complex model representation capabilities, such as classification [25], mapping [26], and unmixing [27]. Meanwhile, DL-based HTD approaches have also received considerable attention. Supervised learning-based detectors were first proposed to optimize the network in a contrastive learning method [28]. Two-stream [29,30] and Siamese structures [31,32,33] are commonly used to realize contrastive learning. With the advancement of neural networks, there have been significant improvements in feature extraction techniques employed by supervised learning approaches. These techniques have evolved from fully connected networks [28] and convolutional networks [29] to more recent transformer models [31,34]. The optimization of numerous network parameters requires abundant labeled data, while only prior spectra are available with annotations. Therefore, supervised HTD methods commonly generate pseudo-data (e.g., pre-detect spectra from in-scene HSIs or simulate spectra) to supplement the training data. Classical detectors [30,31], end-member extraction [31], and clustering methods [29] are employed to pre-detect background samples from in-scene HSIs. Background samples that are easily misidentified as targets are valuable for training; however, they are relatively difficult to detect. Spectral mixture models [29,32,35,36] and generative adversarial networks [37,38] are mainstream simulation approaches. However, simulating high-quality spectra is hard under spectral variability. For supervised learning-based detectors, the reliability of generated pseudo-data limits detection performance.

To overcome the few-shot samples problem, unsupervised and self-supervised learning-based detectors were proposed to optimize networks with pretext tasks [39], which are free of manual annotations. Typical pretext tasks for HTD include spectra reconstruction [40,41,42], denoising [43], and band selection [44]. Ref. [38] proposed a novel pretext task that constructs contrastive learning between odd and even bands of unlabeled spectra. These pretext tasks allow the detectors to learn the intrinsic properties of the HSIs and extract representations of the in-scene spectra. Because pretext tasks do not introduce supervision from manual annotations, supervised fine-tuning [45] or regularization [46,47,48] is embedded in the optimization of pretext tasks, which commonly also requires labeled data. Alternatively, additional classifiers or post-processing are necessary for the final results [41,49,50], which complicates the detection process.

To summarize, exploiting in-scene spectra is crucial to alleviate the challenges posed by spectral variability. Non-learning methods model in-scene spectra under certain assumptions, such as statistical assumptions and spectral mixing models. For DL methods, utilizing in-scene spectra to generate pseudo-data is commonly adopted for supervised learning or fine-tuning. However, ensuring the quality and reliability of the generated spectra-annotation pairs requires significant effort. Additionally, most current approaches focus on HTD using single-epoch HSIs, where prior spectra and in-scene spectra are captured during the same time frame. Dealing with spectral variations of targets in multi-temporal images is more challenging.

Unlike pseudo-data-based detectors, which distinguish background and target spectral samples under the guidance of numerous pseudo annotations, this paper proposes a pseudo-data-free detector that exploits in-scene spectra with implicit contrastive learning [51], which realizes contrastive learning without manual supervision. The proposed detector is explicitly optimized to classify prior spectra while no other annotations are needed. To prevent model collapse resulting from few-shot training samples, an implicit contrastive learning module (ICLM) is proposed to regularize the detector to minimize the loss function through learning differentiated representations of in-scene spectra instead of over-fitting the prior spectra. Specifically, the ICLM normalizes the latent features of prior and in-scene spectra and establishes gradient propagation paths between them. Later in this paper, we theoretically analyze how the ICLM allocates differentiated gradient signals from prior spectral features to the latent features of unlabeled in-scene spectra based on their inherent feature differences. Based on the various allocated gradients, feature differences between targets and backgrounds in the in-scene HSIs are augmented during optimization. In addition to the ICLM, a local spectral similarity constraint (LSSC), which utilizes the spectral similarities of in-scene targets within local neighborhoods, is proposed to improve in-scene adaptability. Specifically, the LSSC assumes that the in-scene target spectra of multi-pixel targets are similar and spatially connected. To better study the detection performance under spectral variability, three multi-temporal HSI pairs collected from Airborne Visible InfraRed Imaging Spectrometer (AVIRIS) data were used to conduct experiments. Each pair consisted of two HSIs captured under different imagining conditions, one for prior spectra generation and the other for target rediscovery experiments. The same category of targets appeared in each multi-temporal HSI pair. Four classical methods and three DL detectors were used for performance comparison under spectral variability.

The main contributions of this paper are as follows:

(1)

This paper proposes a DL-based detector that adapts to spectral variability in multi-temporal hyperspectral images using implicit contrastive learning with prior and in-scene spectra.

(2)

The ICLM is designed to regularize the optimization process of classifying prior spectra. This regularization helps in learning distinct representations of in-scene spectra based on their inherent differences from the prior spectra.

(3)

The LSSC is proposed to enhance scene adaptability by leveraging spectral similarities among in-scene targets within local neighborhoods.

(4)

Three datasets of multi-temporal HSI pairs were collected from AVIRIS data to validate the detection performance under spectral variability. Comparison experiments with classical and DL detectors validate that the proposed detector realizes a superior balance between target detectability and background suppression under spectral variability.

The remainder of this article is organized as follows. Section 2 introduces the proposed implicit contrastive learning detector (ICLTD). Section 3 presents extensive experimental results from the multi-temporal HTD datasets. Conclusions are presented in Section 4.

2. Method

2.1. Spectral Variability of Target Spectra in Multi-Temporal Images

Due to varying atmospheric, illumination, and environmental conditions, the spectral signatures of a target captured at different times may exhibit variability, which is commonly referred to as spectral variability [52]. In addition, intra-class differences in spectral characteristics may exist within the same category of targets, also resulting in variations [4]. Therefore, the prior and in-scene target spectra collected from multi-temporal HSIs may vary.

An urban-scene multi-temporal HSI pair was collected from AVIRIS data to illustrate spectral variability, as exhibited in Figure 2. These two images were captured at different periods within Jefferson County, Washington, United States. Image 1 was acquired under conditions of minimal cloud coverage, whereas Image 2 was acquired under conditions with significant cloud presence. Storage tanks were selected as the targets of interest for this example. Manual target annotations are shown in Figure 1, which were obtained with the help of ENVI software (Version 5.1). Based on these manual annotations, we computed the average target spectra from one of the images to obtain prior spectra. A comparison was then made with the target spectra in the other HSI. It is important to note that these spectra are in the radiance domain and have been normalized for better comparison.

Based on the visualization depicted in Figure 2, notable differences can be observed between the prior spectra and in-scene target spectra. Some conventional HTD methods, such as ACE, generally assume that the prior spectrum represents the mean of the target spectral distribution. However, in real-world scenarios, this assumption may not always hold true, thereby affecting the effectiveness of these methods. Due to the availability of only prior and in-scene spectra (without labels), certain DL-based methods utilize non-learning detectors to pre-detect background spectra. These methods generate pseudo-target spectra by combining prior spectra with the detected background spectra. The augmented data are considered pseudo-data because their labels are not manually annotated and they are used to optimize the parameters of neural networks. Simulating high-quality variational target spectra for training DL detectors is challenging because the variations in target spectra are not solely caused by spectral mixture.

2.2. Exploiting In-Scene Spectra with Implicit Contrastive Learning

The motivation of this paper is to make the detector adapt to spectral variability by exploiting in-scene spectra. Based on the exhibition in Section 2.1, it is more beneficial to explore in-scene target spectra than the pseudo-data. Because the labels of in-scene spectra are unknown, supervised contrastive learning, which teaches the detector to distinguish spectra based on annotations, cannot be used for optimization. Instead, we introduce implicit contrastive learning to exploit the in-scene spectra. Unlike original contrastive learning, which provides loss functions that need to be minimized, implicit contrastive learning does not provide any loss function and is more like a regularization process. A comparison schematic diagram is shown in Figure 3 to exhibit their differences.

For the proposed detector, an objective function is used to detect prior spectra. However, such optimization may easily collapse without regularization, i.e., when considering all the in-scene spectra as targets. One interpretation of model collapse is that the detector only learns a simple and non-discriminative representation of the prior spectra. In contrast, the introduced implicit contrastive learning aims to regularize the optimization by learning representations of the prior and in-scene spectra. This regularization helps prevent the model from over-fitting to the prior spectrum and allows it to recognize in-scene target spectra under spectral variability. To realize this, the implicit contrastive learning module (ICLM) is designed to create a gradient back-propagation channel through which the gradients of the prior spectral features (computed based on the loss function) are allocated to the in-scene spectral features. The allocation of gradients is based on the feature differences between prior spectra and in-scene spectra. As the in-scene target and background spectra inherently differ, the gradients allocated to the different in-scene spectral features vary. A specific description of this will be provided in the following subsections.

2.3. Structure and Pipeline of the Proposed Detector

This subsection introduces the structure and pipeline of the proposed ICLTD. An in-scene HSI is denoted $\mathit{S}\in {\mathbb{R}}^{wh\times l}$, where w, h, and l are the width, height, and band number of the HSI, respectively. The ith in-scene spectrum of $\mathit{S}$ is denoted ${\mathit{s}}_i$. A prior spectrum captured from another HSI is denoted ${\mathit{s}}_p\in {\mathbb{R}}^l$. Different from box-level target detection in the remote sensing field, existing HTD detectors commonly access in-scene spectra (without annotations) for optimization. After training, the detector outputs the target confidence scores of the in-scene spectra as detection results, which are denoted $\mathit{C}\in {\mathbb{R}}^{wh\times 1}$. In the training phase, all of the in-scene spectra and the prior spectrum make up the data batch. For convenience, the number of in-scene spectra is denoted as $n_1=w\times h$. In this paper, only one prior spectrum is generated from the HSI captured at different times. Hence, the batch size is $n_1+1$.

The network structure is shown in Figure 4. As the illumination intensity varies over time and space, significant differences in the radiance amplitude may occur. To mitigate the impact of varying amplitudes, a linear transformation process is used to normalize the Euclidean norm (L2 norm) of the in-scene spectra and prior spectra to 1. The normalized spectra are then inputted into multiple fully connected blocks (FCBs) for feature extraction. Each FCB consists of a fully connected layer, the ICLM, and a nonlinear activation layer. The FC is related to the normalized vector ${\mathit{s}}_i$ by:

$${\mathit{x}}_i=\mathit{W}\times {\mathit{s}}_i+\mathit{b},$$

(1)

where × represents matrix multiplication. $\mathit{W}\in {\mathbb{R}}^{l\times d}$ and $\mathit{b}\in {\mathbb{R}}^d$ are the weight and bias of the fully connected layer, respectively, where d is the dimension number of feature representations. The FC transforms each input vector independently; that is, there is no gradient path from ${\mathit{x}}_i$ to ${\mathit{s}}_j$ when $i\ne j$.

The realization of implicit contrastive learning is based on the designed ICLM, which is a refinement of the original batch normalization (BN) [53]. The original BN was proposed to accelerate the training process while the ICLM is designed to regularize the optimization. The motivation for creating the ICLM is to force the detector to learn a representation of the in-scene spectra during the optimization. Without the ICLM, the detector only learns to transform the prior spectra for classification, and it cannot distinguish other spectra. The original BN could actually mitigate the above model collapse by leaking gradients from the features of the prior spectra to the in-scene spectra. However, the original BN was not specially designed for HTD and cannot provide adequate regularization because BN preserves most gradients for the learning of prior spectra rather than delegating to the in-scene spectra. Further theoretical analysis of this process is presented in Section 2.4.

To solve this problem, the ICLM duplicates the prior spectra representations for normalization to provide adequate regularization. The pipeline of the ICLM is exhibited in Figure 5. Specifically, the ICLM augments the prior spectra representation, ${\mathit{x}}_p$, $n_2$ times and combines them with in-scene spectral features to compute the mean ($\mathsf{\mu }\in {\mathbb{R}}^d$) and variance (${\mathsf{\sigma }}^2\in {\mathbb{R}}^d$), where d is the dimension number of input spectral feature $\mathit{x}$. ICLM is computed using $\mathsf{\mu }$ and ${\mathsf{\sigma }}^2$ as:

$$\left\{\begin{array}{cc}\hfill \mathsf{\mu }& =\frac{1}{m}\left(n_2{\mathit{x}}_p+\sum _{i=1}^{n_1}{\mathit{x}}_i\right)\hfill \\ \hfill {\mathsf{\sigma }}^2& =\frac{1}{m}\left(n_2{({\mathit{x}}_p-\mathsf{\mu })}^2+\sum _{i=1}^{n_1}{({\mathit{x}}_i-\mathsf{\mu })}^2\right),\hfill \end{array}\right.$$

(2)

where $n_2$ is the duplication number of prior spectra representation and m is the sum of $n_1$ and $n_2$.

The computed mean and variance vectors are used to normalize the features of in-scene spectra and the prior spectra:

$$\widehat{\mathit{x}}=\frac{\mathit{x}-\mathsf{\mu }}{\sqrt{{\mathsf{\sigma }}^2}},$$

(3)

where $\widehat{\mathit{x}}$ is the normalized feature vector.

Learnable vectors of the mean and variance (${\mathit{\alpha }}_b\in {\mathbb{R}}^d$ and ${\mathit{\beta }}_b\in {\mathbb{R}}^d$, respectively) are used to fine-tune the distributions:

$$\mathit{y}={\mathit{\alpha }}_b\odot \widehat{\mathit{x}}+{\mathit{\beta }}_b,$$

(4)

where ⊙ represents the element-wise product (Hadamard product). The complete algorithm flow of ICLM is exhibited in Algorithm 1.

Algorithm 1 Pipeline of ICLM

Input: Features of the prior spectra and in-scene HSI: ${\mathit{x}}_p$ and ${\mathit{x}}_1,\dots ,{\mathit{x}}_i,\dots ,{\mathit{x}}_{wh}$; and learnable parameters of the ICLM: ${\mathit{\alpha }}_b$ and ${\mathit{\beta }}_b$.   Output: Normalized spectral features: ${\mathit{y}}_p$ and ${\mathit{y}}_1,\dots ,{\mathit{y}}_{wh}$. 1: Duplicate ${\mathit{x}}_p$$n_2$ times and construct a feature batch. 2: Compute $\mathsf{\mu }$ and ${\mathsf{\sigma }}^2$ of the feature batch following Equation (2). 3: Normalize ${\mathit{x}}_p$ and ${\mathit{x}}_1,\dots ,{\mathit{x}}_{wh}$ with $\mathsf{\mu }$ and ${\mathsf{\sigma }}^2$ following Equation (3). 4: Fine-tune the features with ${\mathit{\alpha }}_b$ and ${\mathit{\beta }}_b$ following Equation (4). 5: return ${\mathit{y}}_p$ and ${\mathit{y}}_1,\dots ,{\mathit{y}}_{wh}$.

After using the ICLM, a sigmoid function is applied to features to increase the nonlinear feature extraction ability of ICLTD. Multiple FCBs are used to extract features of the input spectra in a cascading manner. It is worth noting that the last FCB is free of the sigmoid layer, which brings a larger numerical range to the following classifier.

The classifier is composed of a fully connected layer and a softmax layer. The fully connected layer outputs predicted confidence vectors. The softmax layer normalizes the output confidence vectors to represent probabilities. The dimensions of the predicted probability vectors are the same as those of the latent features. The values of different dimensions represent the confidence of an in-scene spectrum belonging to different categories (targets of interest or various backgrounds). Because only the target prior is known and target spectra are our interest, the probabilities belonging to the target are selected from the output. The first dimension is assumed to be the target class by default, which is used for computing the loss function. The other dimensions of the output vectors represent the probabilities of class-agnostic backgrounds. For the visualization shown in Figure 4, the predicted target probabilities of the input images are denoted $\mathit{C}\in {\mathbb{R}}^{wh\times 1}$. The predicted confidence of the prior spectrum and the ith in-scene spectrum of $\mathit{S}$ are denoted $c_p$ and $c_i$, respectively.

Unlike existing contrastive learning-based detectors, which require both positive and negative sample pairs for their loss functions, the proposed detector only utilizes the manual annotations of the prior spectrum for explicit supervised learning. The specific loss function for classifying positive samples is as follows:

$$\ell =-log\left(c_p\right).$$

(5)

When the ICLM is not present in the detector, the feature extraction and classification of a batch of spectra become independent of each other. This implies that the learning of detecting prior spectra does not help the detector to distinguish in-scene spectra. If optimizing the detector with Equation (5) in the absence of the ICLM, the detector will over-fit when detecting prior spectra. When the ICLM is present, the detector will learn to minimize Equation (5) by changing the representation of in-scene spectra, which regularizes the optimization.

2.4. Analysis of the ICLM

Under the HTD task, this subsection illustrates how the ICLM realizes implicit contrastive learning and brings discriminative feature extraction capability to the detector.

From the data distribution perspective, the ICLM normalizes features to restrict extreme output aggregation or divergence [51]. Therefore, the detector will not output identical results (feature aggregation) or will only be able to detect prior spectra (feature divergence). The ICLM, to some extent, mitigates the differences between the prior and target spectra by adjusting the data distributions. However, adjusting the feature distribution cannot help the optimization much. To validate the effectiveness of distribution adjustment, an ablation study is conducted in Section 3.5 where the gradient propagation component of the ICLM is disabled.

Compared to feature distribution adjustment, the gradient propagation path established by the ICLM between the in-scene spectra and prior spectra is more important. The last ICLM layer is chosen to analyze the implicit contrastive learning because the gradient calculation in this layer is simpler than those in the previous layers. Note that all of the vector multiplication presented in this subsection is the Hadamard product, which is omitted for brevity.

For convenience, the hidden features before and after the last ICLM layer are denoted $\mathit{x}$ and $\widehat{\mathit{x}}$, respectively. We define a spectral feature sequence to exhibit the gradient propagation path from ${\mathit{x}}_p$ to unlabeled $\mathit{x}$ as:

$${\mathit{x}}_i={\mathit{x}}_1,{\mathit{x}}_2,\dots ,{\mathit{x}}_{n_1+n_2},$$

(6)

where ${\mathit{x}}_1,\dots ,{\mathit{x}}_{n_1}$ are in-scene spectral features and ${\mathit{x}}_{n_1+1},\dots ,{\mathit{x}}_{n_2}$ are duplicated prior features. ${\mathit{x}}_i$ equals ${\mathit{x}}_p$ when $i>n_1$. The normalized spectral features outputted by the last ICLM are computed by substituting $\mathsf{\mu }$ and ${\mathsf{\sigma }}^2$ in Equation (3) according to Equation (2):

$${\widehat{\mathit{x}}}_p=\frac{m{\sum }_{i=1}^m{\mathit{x}}_p-{\mathit{x}}_i}{\sqrt{{\sum }_{i=1}^m({\sum }_{j=1}^m({\mathit{x}}_i-{\mathit{x}}_j){)}^2}},$$

(7)

where only the normalized prior spectral features are exhibited because the in-scene spectral features are not used in the loss function.

According to the numerator in Equation (7), the prior spectral features are subtracted from each input feature when the ICLM normalizes the distribution means. Similarly, the denominator constructs computational relationships between any two spectral samples. Because of the established gradient propagation paths, representations of in-scene spectra also receive supervised signals during optimization.

The gradient of features before the last ICLM ($\mathit{x}$) received from the ICLM is calculated to analyze the regularization of the implicit contrastive learning. There are three paths that could pass the gradients to $\mathit{x}$: $\widehat{\mathit{x}}$, $\mathsf{\mu }$, and ${\mathsf{\sigma }}^2$. Because the gradients $\frac{\partial \ell }{\partial \mathsf{\mu }}$ and $\frac{\partial \ell }{\partial {\mathsf{\sigma }}^2}$ rely on $\frac{\partial \ell }{\partial \widehat{\mathit{x}}}$, the latter is calculated first.

Since in-scene spectral features are not used in classification loss, their derivatives are zero:

$$\frac{\partial \ell }{\partial {\widehat{\mathit{x}}}_i}={\mathit{g}}_i=\left\{\begin{array}{c}\begin{array}{ccc}\hfill & 0,\hfill & \hfill {\widehat{\mathit{x}}}_i\mathrm{is}\mathrm{unlabeled}\\ \hfill & {\mathit{g}}_p,\hfill & \hfill {\widehat{\mathit{x}}}_i\mathrm{is}\mathrm{prior},\end{array}\hfill \end{array}\right.$$

(8)

where $\frac{\partial \ell }{\partial {\widehat{\mathit{x}}}_i}$ is denoted ${\mathit{g}}_i$ for convenience. When $i>n_1$, $\frac{\partial \ell }{\partial {\widehat{\mathit{x}}}_i}$ are the same because of duplication and are denoted ${\mathit{g}}_p$.

Next, the derivative of ℓ with respect to $\mathsf{\mu }$ and ${\mathsf{\sigma }}^2$ is calculated. According to Equation (3), the gradient of the loss function ℓ with respect to ${\mathsf{\sigma }}^2$ is:

$$\begin{array}{c}\hfill \frac{\partial \ell }{\partial {\mathsf{\sigma }}^2}=\frac{-n_2{\mathit{g}}_p{\widehat{\mathit{x}}}_{\mathit{p}}}{2{\mathsf{\sigma }}^2}.\end{array}$$

(9)

The gradient of the loss function ℓ with respect to $\mathsf{\mu }$ can be simplified as:

$$\begin{array}{c}\hfill \frac{\partial \ell }{\partial \mathsf{\mu }}=\frac{-n_2{\mathit{g}}_p}{\sqrt{{\mathsf{\sigma }}^2}}.\end{array}$$

(10)

According to Equation (3) and Equation (2), the gradients of $\widehat{\mathit{x}}$, $\mathsf{\mu }$, and ${\mathsf{\sigma }}^2$ with respect to $\mathit{x}$ are, respectively:

$$\left\{\begin{array}{c}\begin{array}{cc}\hfill \frac{\partial \widehat{\mathit{x}}}{\partial \mathit{x}}& =\frac{1}{\sqrt[ ]{{\mathsf{\sigma }}^2}}\hfill \\ \hfill \frac{\partial \mathsf{\mu }}{\partial \mathit{x}}& =\frac{1}{m}\hfill \\ \hfill \frac{\partial {\mathsf{\sigma }}^2}{\partial \mathit{x}}& =\frac{2(\mathit{x}-\mathsf{\mu })}{m}.\hfill \end{array}\hfill \end{array}\right.$$

(11)

According to the above gradients, $\frac{\partial \ell }{\partial \mathit{x}}$ can be expressed as:

$$\begin{array}{c}\hfill \frac{\partial \ell }{\partial \mathit{x}}=\frac{1}{m\sqrt{{\mathsf{\sigma }}^2}}(\frac{m\mathit{g}}{➀}-\frac{n_2{\mathit{g}}_p}{➁}-\frac{n_2{\widehat{\mathit{x}}}_p\widehat{\mathit{x}}{\mathit{g}}_p}{➂}),\end{array}$$

(12)

where the gradients labeled as ①, ②, and ③ come from $\widehat{\mathit{x}}$, $\mathsf{\mu }$, and ${\mathsf{\sigma }}^2$, respectively.

Based on the loss function (Equation 5), ${\mathit{g}}_i=0$ for in-scene spectral features ($i<n_1$). According to Equation (12), the gradient of ℓ with respect to ${\mathit{x}}_{\mathit{i}}$ is summarized as:

$$\frac{\partial \ell }{\partial {\mathit{x}}_i}=\left\{\begin{array}{c}\begin{array}{cc}\hfill \frac{1}{m\sqrt{{\mathsf{\sigma }}^2}}(-n_2{\mathit{g}}_p-n_2{\widehat{\mathit{x}}}_p{\widehat{\mathit{x}}}_i{\mathit{g}}_p)& ,i\le n_1\hfill \\ \hfill \frac{1}{m\sqrt{{\mathsf{\sigma }}^2}}(n_1{\mathit{g}}_p-n_2{\widehat{\mathit{x}}}_p^2{\mathit{g}}_p)& ,i>n_1\hfill \end{array}\hfill \end{array}\right.$$

(13)

Based on Equation (13), the gradients computed from ℓ are passed to in-scene spectra through $\mathsf{\mu }$ and ${\mathsf{\sigma }}^2$ in the ICLM. In other words, the features of in-scene spectra are also optimized to realize the classification of prior spectra. Therefore, implicit contrastive learning of the in-scene spectra regularizes the optimization and prevents model collapse.

To analyze how the ICLM enables the model to learn the differentiated representation of in-scene spectra, we calculate the difference in gradients obtained through the ICLM for two in-scene spectra:

$$\frac{\partial \ell }{\partial {\mathit{x}}_i}-\frac{\partial \ell }{\partial {\mathit{x}}_j}=\frac{n_2{\widehat{\mathit{x}}}_p{\mathit{g}}_p({\widehat{\mathit{x}}}_i-{\widehat{\mathit{x}}}_j)}{m\sqrt{{\mathsf{\sigma }}^2}}.$$

(14)

Assuming the two in-scene spectra in Equation (14) are the target and background spectra, their inherent differences are automatically utilized by the ICLM to allocate differentiated gradients. We also calculate the difference in gradients obtained through ${\mathsf{\sigma }}^2$ for the in-scene spectra and the prior spectra:

$$\begin{array}{c}\hfill \frac{\partial \ell }{\partial {\mathsf{\sigma }}^2}\frac{\partial {\mathsf{\sigma }}^2}{\partial {\mathit{x}}_i}-\frac{\partial \ell }{\partial {\mathsf{\sigma }}^2}\frac{\partial {\mathsf{\sigma }}^2}{\partial {\mathit{x}}_p}=\frac{n_2{\widehat{\mathit{x}}}_p{\mathit{g}}_p({\widehat{\mathit{x}}}_i-{\widehat{\mathit{x}}}_p)}{m\sqrt{{\mathsf{\sigma }}^2}}.\end{array}$$

(15)

Based on Equation (15), the gradients received by in-scene target spectra (through ${\mathsf{\sigma }}^2$) are more similar to those of the prior spectra than the in-scene background spectra, because of their inherent data similarities. In contrast, the gradients received by the in-scene background spectra are different from those of the prior spectra because of inherent data differences.

$$\begin{array}{c}\hfill \sum \frac{\partial \ell }{\partial \mathit{x}}=\frac{n_2{\mathit{g}}_p{\widehat{\mathit{x}}}_p{\sum }_{i=1}^m{\widehat{\mathit{x}}}_i}{m\sqrt{{\mathsf{\sigma }}^2}}=0.\end{array}$$

(16)

According to Equation (16), the total amount of gradients passed through the ICLM from ℓ to $\mathit{x}$ equals 0, which means that the signs of the transmitted gradients through the ICLM to the prior and in-scene spectral features are opposite. Therefore, the more prior spectra in the ICLM are duplicated, the stronger the supervised signal transferred to the unlabeled spectra will be. Therefore, $n_2$ determines the strength of regularization. The larger $n_2$ is, the more the model relies on learning the representations of in-scene spectra to minimize the loss function. When $n_2=1$, the ICLM becomes the original BN. Because $n_1\gg n_2$, the gradients received by in-scene spectral features are close to 0 according to Equation (15). Hence, the original BN could not provide adequate regularization.

To summarize, in this section, we analyzed the ICLM from three perspectives. From the perspective of data distribution, the ICLM avoids excessive aggregation or divergence of the extracted features. Regarding forward propagation, the ICLM establishes gradient propagation paths between prior and unlabeled spectra. From the perspective of gradient back-propagation, the ICLM transfers the gradient of the loss function to in-scene spectra, and the feature differences are used to determine the transferred gradients. As the number of prior spectra increases in the ICLM (i.e., as $n_2$ increases), the regularization brought by implicit contrastive learning is enhanced.

2.5. Local Spectral Similarity Constraint

With the development of imaging spectroscopy technology, the spatial and spectral resolution of HSIs has improved. Targets in HSIs may occupy multiple pixels, as shown by the example in Figure 6. For a multiple-pixel target, its 3D geometric structure and its relationship with the background vary from different locations, which cause differences in its spectral characteristics. Our aim is to minimize the feature differences of these locally connected spectra, thereby improving the performance of in-scene adaptability. During training, the predicted confidence scores of in-scene spectra can be used to select candidate targets. For each candidate sample, if there are samples with higher confidence within their 3 × 3 neighborhoods, we increase the similarity between the feature representations of these two samples. Note that the LSSC can be employed if a target occupies two or more connected pixels.

According to the above motivation, the LSSC was proposed and applied to representations of in-scene spectra outputted by each fully connected layer of the FCBs. First, we find the target neighborhoods. A threshold, t, is set to select high-confidence in-scene spectra. Given the detection results of in-scene HSI, $\mathit{C}$, the mask that reflects the distribution of target candidates is:

$$\mathit{M}=\mathit{C}>t.$$

(17)

Latent features of the in-scene spectra extracted by the kth fully connected layer are denoted $\{{\mathit{x}}_1^k,\dots ,{\mathit{x}}_{wh}^k\}$. Their corresponding confidence scores are denoted $\{c_1,\dots ,c_{wh}\}$. According to $\mathit{M}$, the representations of target candidates are collected into the set: ${\mathbf{T}}^k=\left\{{\mathit{x}}_i^k\right|{\mathit{M}}_i>0\}$. For each candidate, its 3 × 3 neighborhood is constructed, encompassing multi-level representations and confidence scores. The neighborhood of features is denoted $\{{\mathit{x}}_i^{k,1},{\mathit{x}}_i^{k,2},\dots ,{\mathit{x}}_i^{k,8}\}$, where ${\mathit{x}}_i^{k,1}$ represents the first neighboring feature of ${\mathit{x}}_i^k$. The corresponding confidence scores are $\{c_i^1,c_i^2,\dots ,c_i^8\}$. For each ${\mathit{x}}_i^k$, the set consisting of its neighboring target spectral features is denoted $\mathsf{\eta }\left({\mathit{x}}_i^k\right)=\left\{{\mathit{x}}_i^{k,j}\right|c_i<c_i^j\}$, where $c_i^j$ represents the target confidence score of ${\mathit{x}}_i^{k,j}$. With the above features, the proposed LSSC is defined as:

$${\ell }_{LSS\mathrm{C}=}\frac{-1}{n_s}{\sum }_{k=1}^4{\sum }_{\mathit{x}\in {\mathbf{T}}^k}^{ }{\sum }_{{\mathit{x}}^{\prime }\in \mathsf{\eta }\left(\mathit{x}\right)}^{ }log\left(\frac{sg\left(f\left({\mathit{x}}^{\prime }\right)\right)·f\left(\mathit{x}\right)}{\left|\right|sg\left(f\left({\mathit{x}}^{\prime }\right)\right){\left|\right|}_2\left|\right|f\left(\mathit{x}\right){\left|\right|}_2}\right),$$

(18)

where $sg\left({\mathit{x}}^{\prime }\right)$ represents stopping the gradient propagation of ${\mathit{x}}^{\prime }$, f is the softmax operation, and $n_s$ is the number of candidate targets. Stopping the gradients of neighboring features is to ensure that the LSSC enhances the detectability of target candidates rather than diminishing that of the neighboring targets. The complete calculation process of LSSC is described in Algorithm 2. The total optimization function, L, is the combination of positive sample classification loss and the LSSC:

$$L=\ell +{\ell }_{LSS\mathrm{C}}.$$

(19)

Algorithm 2 Pipeline for computing LSSC

Input: Spectral features extracted by the kth fully connected layer: $\{{\mathit{x}}_1^k,\dots ,{\mathit{x}}_{wh}^k\},k\in \{1,2,3,4\}$; confidence scores of in-scene spectra: $c_1,\dots ,c_{wh}$; and the threshold of the LSSC: t.   Output: The loss of the LSSC, ${\ell }_{LSS\mathrm{C}}$. 1: Generate the target candidate mask $\mathit{M}$ according to predicted confidence scores following Equation (17). 2: Collect features of candidate in-scene targets: ${\mathbf{T}}^k=\left\{{\mathit{x}}_i^k\right|{\mathit{M}}_i>0\}$. 3: Collect confidence scores of the candidates $\left\{c_i\right|{\mathit{M}}_i>0\}$. 4: Collect neighboring spectral features of each ${\mathit{x}}_i^k$ in ${\mathbf{T}}^k$: $\{{\mathit{x}}_i^{k,1},{\mathit{x}}_i^{k,2},\dots ,{\mathit{x}}_i^{k,8}\}$. 5: Collect confidence scores of neighboring features: $\{c_i^1,c_i^2,\dots ,c_i^8\}$. 6: Collect desired neighboring features of ${\mathit{x}}_i^k$: $\mathsf{\eta }\left({\mathit{x}}_i^k\right)=\left\{{\mathit{x}}_i^{k,j}\right|c_i<c_i^j\}$. 7: Get ${\ell }_{LSS\mathrm{C}}$ following Equation (18). 8: return ${\ell }_{LSS\mathrm{C}}$.

3. Experiments

3.1. Experimental Setup

3.1.1. Datasets

We conducted experiments using three pairs of multi-temporal HSIs collected from AVIRIS data (https://aviris.jpl.nasa.gov/, accessed on 23 December 2023). AVIRIS is an airborne imaging spectrometer instrument capable of capturing 224 contiguous spectral radiance images in the wavelength range of 400–2500 nm. The AVIRIS data used in this study consist of the hyperspectral data collected during ground observation experiments when AVIRIS was mounted on an aircraft. The AVIRIS data have been ortho-corrected and underwent radiometric calibration.

Specifically, we collected three sets of multi-temporal HSIs from airport, beach, and urban scenes. These datasets are referred to as MT-ABU for convenience. The groundtruth maps of targets were annotated using ENVI software. The pseudo-color images and corresponding annotations are shown in Figure 7. Further details of the MT-ABU dataset can be found in

Table 1. Detailed features of the collected multi-temporal datasets.

Dataset	Place	Time	Spatial Resolution	Target Pixel	Image Size	Source of Prior Spectra
MT-ABU-A1	Riverside	4 August 2013	3.8 m	185	150 × 150	MT-ABU-A2
MT-ABU-A2	San Francisco Bay Area	23 June 2008	13.7 m	8	50 × 50	MT-ABU-A1
MT-ABU-B1	Long Beach	19 April 2014	16.7 m	14	150 × 150	MT-ABU-B2
MT-ABU-B2	Bay Area	22 November 2013	16.7 m	16	100 × 100	MT-ABU-B1
MT-ABU-U1	Jefferson County	29 August 2010	7.1 m	51	100 × 100	MT-ABU-U2
MT-ABU-U2	Jefferson County	28 August 2010	7.1 m	118	100 × 100	MT-ABU-U1

.

Each scene consists of two images captured at different times to form an image pair. Although the targets in each pair of images are not identical, they belong to the same category. The spatial resolution of each image was estimated based on the flying altitude. In cases where the AVIRIS data did not provide fly altitudes, the spatial resolution was estimated using information from Google Maps.

We conducted HTD experiments in a target-rediscovery way. Specifically, two HSIs of each scene provided prior spectra for each other. For example, the prior spectrum used for detecting targets in MT-ABU-A1 was generated by averaging the target spectral radiances from MT-ABU-A2 based on the ground-truth annotations. As a result, six images were used in the experiments, each with its corresponding prior spectrum. The in-scene HSIs and their corresponding prior spectra sources are presented in Table 1. For convenience, we will refer to each experimental dataset by the name of its respective in-scene HSI.

3.1.2. Comparison Methods

In order to validate the performance of the proposed method under the spectral variability condition, both learning and non-learning methods were employed for comparison. Three DL-based methods that required pseudo-data generation were compared: Siamese fully connected target detector (SFCTD) [32], two stream convolutional target detector (TSCNTD) [29], and Siamese transformer target detector (STTD) [31]. Non-deep learning methods included a classical method (ACE), a spectral distance-based method (R-SA²) [12]), and the improved classical methods hierarchical CEM (HCEM) [14] and ensemble-based CEM (ECEM) [15]. Note that all of the methods utilized the same prior spectral information and did not employ any additional data; they used just the prior spectra and in-scene HSIs. Therefore, the experiments provide a fair comparison.

3.1.3. Evaluation Criteria

Three-dimensional (3D)-receiver operator characteristic (ROC) curves [54], area under the curve (AUC), and detection map visualization were used to evaluate the performances comprehensively. The training and inference times of DL methods and the processing times of the non-learning methods were used to evaluate their efficiency. The 3D-ROC curve reflects the relationship between detection probability, $P_D$, false alarm probability, $P_F$, and confidence threshold, $\tau$. The desired curve has a lower $P_F$ and a higher $\tau$ for the same $P_D$.

The AUC values of 3D-ROC curves quantitatively reflect the quality of the curves, which are denoted $\mathrm{A}\mathrm{U}{\mathrm{C}}_{(D,F)}$, $\mathrm{A}\mathrm{U}{\mathrm{C}}_{(D,\tau )}$, and $\mathrm{A}\mathrm{U}{\mathrm{C}}_{(F,\tau )}$. $\mathrm{A}\mathrm{U}{\mathrm{C}}_{(D,F)}$ measures the ability of the detector to discriminate between targets and backgrounds. $\mathrm{A}\mathrm{U}{\mathrm{C}}_{(D,\tau )}$ and $\mathrm{A}\mathrm{U}{\mathrm{C}}_{(F,\tau )}$ reflect the average confidence values of targets and backgrounds, respectively. Because $\mathrm{A}\mathrm{U}{\mathrm{C}}_{(D,\tau )}$ and $\mathrm{A}\mathrm{U}{\mathrm{C}}_{(F,\tau )}$ are unable to evaluate target–background separation ability, two derived AUC values that are proposed in this work [12] are utilized to replace them:

$$\left\{\begin{array}{cc}\mathrm{A}\mathrm{U}{\mathrm{C}}_{TDBS}=\hfill & \mathrm{A}\mathrm{U}{\mathrm{C}}_{(D,\tau )}-\mathrm{A}\mathrm{U}{\mathrm{C}}_{(F,\tau )},\hfill \\ \mathrm{A}\mathrm{U}{\mathrm{C}}_{SNP\mathrm{R}}=\hfill & \mathrm{A}\mathrm{U}{\mathrm{C}}_{(D,\tau )}/\mathrm{A}\mathrm{U}{\mathrm{C}}_{(F,\tau )}.\hfill \end{array}\right.$$

(20)

$\mathrm{A}\mathrm{U}{\mathrm{C}}_{TDBS}$ reflects the difference in confidence scores between targets and backgrounds, while $\mathrm{A}\mathrm{U}{\mathrm{C}}_{SNP\mathrm{R}}$ measures the ratio between them. Since the number of background samples in an HSI is much larger than the number of target samples, changes in confidence scores for a subset of background samples may not significantly affect the overall amplitude but can still impact background suppression performance. In such cases, $\mathrm{A}\mathrm{U}{\mathrm{C}}_{SNP\mathrm{R}}$, which measures the differences in magnitude, provides a better reflection of performance changes. However, it is worth noting that even though some results may have high $\mathrm{A}\mathrm{U}{\mathrm{C}}_{SNP\mathrm{R}}$ values, both the confidence scores for the target and the background may still be low. In such situations, $\mathrm{A}\mathrm{U}{\mathrm{C}}_{TDBS}$ is a more suitable metric for evaluation.

3.1.4. Implementation Details

The output dimension of each fully connected layer, d, was 50, including feature extraction and the final classifier. For convenience, the ratio of prior spectral features to in-scene spectral features in the ICLM is denoted $r=n_2/n_1$. Unless otherwise specified, r was set to 0.5 and the confidence threshold t in the LSSC was set to 0.3. These two parameters were fixed for all the comparison experiments. Parameter sensitivity experiments of r and t were conducted. During training, all of the in-scene spectra were put into a data batch. The Adam optimizer was used to optimize the model parameters, with a learning rate of $1\times {10}^{-4}$ and a weight decay of $5\times {10}^{-4}$. The number of training epochs was set to 500.

The configuration of the server used for the experiment consisted of an Intel i9-9900x CPU, NVIDIA TITAN Xp, and 32 GB of RAM. The operating system was Windows 11. The Pytorch package was used for running the DL-based approach codes. When comparing the detection performance, all methods used the same prior spectra. Non-learning methods were run using the NumPy package in the Python environment. For all the methods, the L2-norm of in-scene and prior spectra were normalized to 1 for better performance.

3.2. Parameter Sensitivity Experiments

The proposed HTD method consists of two main components: the ICLM and LSSC. Both modules have parameters that are manually set, r and t. r is the ratio of the number of prior spectra to in-scene spectral features in the ICLM, and t is a confidence threshold for collecting target candidates. In order to analyze the influence of these two parameters on performance, we conducted experiments with different parameter settings. The tested values for r were {0.1, 0.25, 0.5, 0.75, 1} and for t were {0.1, 0.2, 0.3, 0.4, 0.5}. In total, 25 experiments were performed on each dataset, and the AUC results of each experiment are displayed in a 3D bar graph in Figure 8.

The $\mathrm{A}\mathrm{U}{\mathrm{C}}_{(D,F)}$ results under different t are stable and excellent for the six datasets. Although there is a slight fluctuation in the $\mathrm{A}\mathrm{U}{\mathrm{C}}_{(D,F)}$ results for the MT-ABU-A1 data, they remain at a high level. The differences in the $\mathrm{A}\mathrm{U}{\mathrm{C}}_{TDBS}$ and $\mathrm{A}\mathrm{U}{\mathrm{C}}_{SNP\mathrm{R}}$ results under the different settings of t are not significant. According to the above results, the ICLTD has relatively weak sensitivity to the variable t. In practical applications, t has the flexibility to assume values across a wide range without significantly influencing the performance.

The $\mathrm{A}\mathrm{U}{\mathrm{C}}_{(D,F)}$ results with different r are also stable and great. However, the $\mathrm{A}\mathrm{U}{\mathrm{C}}_{TDBS}$ and $\mathrm{A}\mathrm{U}{\mathrm{C}}_{SNP\mathrm{R}}$ results significantly changes for different values of r. As r increases, $\mathrm{A}\mathrm{U}{\mathrm{C}}_{TDBS}$ decreases while $\mathrm{A}\mathrm{U}{\mathrm{C}}_{SNP\mathrm{R}}$ increases. Although the changes are significant, the $\mathrm{A}\mathrm{U}{\mathrm{C}}_{TDBS}$ and $\mathrm{A}\mathrm{U}{\mathrm{C}}_{SNP\mathrm{R}}$ results are promising. According to the illustration given in Section 2.4, r determines the regularization strength brought by implicit comparison learning. The above experimental results reflect that an increase in regularization strength improves the background suppression ability while impacting the detector’s ability to detect variational in-scene target spectra. In practical applications, if spectral variability is strong, a small r could ensure target detectability. On the other hand, when spectral variability is not obvious, a larger r could be set to achieve better background suppression performance.

3.3. Visualization of Feature Distance during Training

According to the theoretical analysis illustrated in Section 2.4, implicit contrastive learning of in-scene spectra regularizes the optimization and allocates various gradients for discriminative representations of in-scene spectra. Moreover, the LSSC was designed to reduce variance among the in-scene target spectra representations. To study how the ICLM and LSSC influence feature extraction, we computed the feature difference between in-scene spectra and prior spectra during the training process. The L1 distance was applied to features extracted by the last FCB. For convenience, the distance between in-scene targets and priors is denoted $d_{(t,p)-\mathrm{LSSC}}$, and the distance between in-scene backgrounds and priors is denoted $d_{(b,p)-\mathrm{LSSC}}$. The above two distances are denoted $d_{(t,p)}$ and $d_{(b,p)}$, respectively, if the LSSC was not applied to the optimization. The distances of all the targets and backgrounds were averaged for visualization, as exhibited in Figure 9.

Because the parameters of the ICLTD were randomly initialized, $d_{(t,p)}$ and $d_{(b,p)}$ were small at the beginning. During the optimization process, $d_{(b,p)}$ consistently increased, and $d_{(t,p)}$ also increased during the first few epochs. Then, $d_{(t,p)}$ started to decline and eventually stabilized or increased slowly. Because of the different growth paths of $d_{(b,p)}$ and $d_{(t,p)}$, the optimized detector learns differentiated representations of in-scene spectra. These results validate that the introduced implicit contrastive learning of in-scene spectra prevents model collapse and can help the ICLTD distinguish in-scene target spectra from background spectra. Because the regularization of implicit contrastive learning is based on inherent data differences, some variational in-scene targets are also suppressed due to spectral variability, which is the reason why $d_{(b,p)}$ may increase slowly near the end of the process.

When LSSC was applied to the optimization, $d_{(b,p)-\mathrm{LSSC}}$ also consistently increased, eventually equaling $d_{(b,p)}$, which validates why the LSSC is automatically applied to the in-scene targets and does not have a significant impact on background suppression. The trends of $d_{(t,p)}$ and $d_{(t,p)-\mathrm{LSSC}}$ exhibited similarities in the initial epochs. However, as training progressed, $d_{(t,p)-\mathrm{LSSC}}$ consistently showed a notably lower value compared to $d_{(t,p)}$. Furthermore, $d_{(t,p)-\mathrm{LSSC}}$ either stabilized or continued to decrease in the later stages of training. These findings provide evidence that the introduction of the LSSC effectively reduces feature variations between the prior and in-scene spectra. Consequently, it mitigates the impact caused by spectral variability.

3.4. Comparison with Other Approaches

3.4.1. 3D-ROC Curves

We first analyzed the results of $\mathrm{R}\mathrm{O}{\mathrm{C}}_{(P_D,P_F)}$, shown in the third left column of Figure 10. It is desirable for $P_D$ to be higher for a given $P_F$. For all datasets except MT-ABU-B2, the proposed ICLTD has the best $\mathrm{R}\mathrm{O}{\mathrm{C}}_{(P_D,P_F)}$ results. For the MT-ABU-B2 dataset, the $\mathrm{R}\mathrm{O}{\mathrm{C}}_{(P_D,P_F)}$ of the ICLTD is competitive. Among the three DL methods that were compared, SFCTD has the best $\mathrm{R}\mathrm{O}{\mathrm{C}}_{(P_D,P_F)}$ results, with STTD coming in second place. The $\mathrm{R}\mathrm{O}{\mathrm{C}}_{(P_D,P_F)}$ results of HCEM are better and more stable than for ECEM. For all of the HSIs except MT-ABU-B2, the $\mathrm{R}\mathrm{O}{\mathrm{C}}_{(P_D,P_F)}$ results of ACE and R-SA² are inferior compared to those of STTD, STTD, HCEM, and ECEM. The $\mathrm{R}\mathrm{O}{\mathrm{C}}_{(P_D,P_F)}$ results of TSCNTD are inferior to the other methods for nearly all datasets.

The experimental results of $\mathrm{R}\mathrm{O}{\mathrm{C}}_{(P_D,\tau )}$ and $\mathrm{R}\mathrm{O}{\mathrm{C}}_{(P_F,\tau )}$ were analyzed together. These two curves reflect the target detectability and background suppression of the detector, respectively, and the difference between them reflects target–background separability. The desired $\mathrm{R}\mathrm{O}{\mathrm{C}}_{(P_D,\tau )}$ has a higher $P_D$ at given $\tau$, while the desired $\mathrm{R}\mathrm{O}{\mathrm{C}}_{(P_F,\tau )}$ has the opposite behavior. The proposed ICLTD, STTD, and HCEM have desirable $\mathrm{R}\mathrm{O}{\mathrm{C}}_{(P_D,\tau )}$ and $\mathrm{R}\mathrm{O}{\mathrm{C}}_{(P_F,\tau )}$ results, while the ICLTD makes the best balance between these two curves. For ECEM and SFCTD, they have great $\mathrm{R}\mathrm{O}{\mathrm{C}}_{(P_D,\tau )}$ results but unsatisfactory $\mathrm{R}\mathrm{O}{\mathrm{C}}_{(P_F,\tau )}$ results. For ACE and R-SA², their $\mathrm{R}\mathrm{O}{\mathrm{C}}_{(P_F,\tau )}$ results are desirable but their $\mathrm{R}\mathrm{O}{\mathrm{C}}_{(P_D,\tau )}$ results are not ideal.

The 3D-ROC combines the above three 2D-ROC curves together. For convenience, we defined a coordinate order ($\tau$, $P_F$, $P_D$) and positioned two planes in 3D coordinate space, where the green one contains samples with $\tau =1$ while the gray one contains samples with $\tau =0$. As shown in the visualization in Figure 10, each 3D-ROC is from (1, 0, 0) to (0, 1, 1). The ideal 3D-ROC should first increase $P_D$ and then $P_F$. The curve should be close to the green plane when ascending, which reflects great target detectability. Meanwhile, the desired curve should be close to the gray plane when the value of $P_F$ increases, proving its background suppression ability. For instance, the best curve will connect the following coordinates with a straight line: (1, 0, 0), (1, 0, 1), (0, 0, 1), and (0, 1, 1). Based on the results, the curves of the ICLTD are superior to those of the other methods in terms of balancing target detectability and background suppression. The curves of the ICLTD first ascend then maintain high confidence scores. When the $P_F$ increases, the curve of the ICLTD is close to the gray plane. Although the curves of ECEM are closer to the green plane when ascending, they are far from the gray plane when $P_F$ increases. The curves of ACE and R-SA² are far from the green plane when ascending.

In summary, the proposed ICLTD has the best overall detection performance, which balances the performance of distinguishing targets and backgrounds ($\mathrm{R}\mathrm{O}{\mathrm{C}}_{(P_D,P_F)}$), target detectability ($\mathrm{R}\mathrm{O}{\mathrm{C}}_{(P_D,\tau )}$), and background suppression ($\mathrm{R}\mathrm{O}{\mathrm{C}}_{(P_F,\tau )}$). HCEM has the second-best detection performance, followed by SFCTD and STTD. SFCTD is better than STTD in terms of distinguishing targets and backgrounds while STTD has better target–background separability. ECEM is not as good as the above methods but is still better than TSCNTD, ACE, and R-SA². TSCNTD, ACE, and R-SA² do not perform well on multi-temporal HTD datasets. The unsatisfactory performance of TSCNTD may come from optimization difficulties resulting from the absence of normalization layers. The inferior performances of ACE and R-SA² may result from the variations in the prior spectra.

3.4.2. AUC Values

AUC values can quantitatively reflect the quality of ROC curves. The $\mathrm{A}\mathrm{U}{\mathrm{C}}_{(D,F)}$ results are shown in

Table 2. $\mathrm{A}\mathrm{U}{\mathrm{C}}_{(D,F)}$ results for the MT-ABU datasets. The best results are indicated in bold.

	Dataset	A1	A2	B1	B2	U1	U2
Method
ACE		0.6785	0.8974	0.9909	0.9661	0.8690	0.7125
R-SA2		0.6805	0.8996	0.9912	0.9663	0.8701	0.7147
HCEM		0.9934	1.0000	0.9997	1.0000	0.9980	0.8612
ECEM		0.9048	0.9501	0.9958	1.0000	0.9950	0.8880
SFCTD		0.9697	0.9975	0.9999	1.0000	0.9987	0.9754
TSCNTD		0.2921	0.8768	0.9647	1.0000	0.3980	0.7060
STTD		0.9939	0.9743	0.9998	1.0000	0.9974	0.9263
ICLTD		0.9956	1.0000	0.9999	1.0000	0.9997	0.9993

. The proposed method has the highest $\mathrm{A}\mathrm{U}{\mathrm{C}}_{(D,F)}$ results for all datasets. HCEM, SFCTD, and STTD also achieved promising $\mathrm{A}\mathrm{U}{\mathrm{C}}_{(D,F)}$ results, followed by ECEM, ACE, and R-SA². TSCNTD did not perform well in terms of $\mathrm{A}\mathrm{U}{\mathrm{C}}_{(D,F)}$.

The $\mathrm{A}\mathrm{U}{\mathrm{C}}_{TDBS}$ results are shown in

Table 3. $\mathrm{A}\mathrm{U}{\mathrm{C}}_{TDBS}$ results for the MT-ABU datasets. The best results are indicated in bold.

	Dataset	A1	A2	B1	B2	U1	U2
Method
ACE		0.0064	0.0247	0.0391	0.1023	0.0413	0.0178
R-SA2		0.0065	0.0248	0.0393	0.1024	0.0417	0.0179
HCEM		0.5844	0.6767	0.7185	0.6808	0.6391	0.3264
ECEM		0.1701	0.5862	0.4510	0.1973	0.9081	0.4017
SFCTD		0.0695	0.2473	0.2767	0.2299	0.3570	0.2013
TSCNTD		−0.0371	0.0665	0.5022	0.6141	0.0000	0.1172
STTD		0.5906	0.2736	0.9070	0.4578	0.5235	0.1543
ICLTD		0.4417	0.4125	0.8994	0.7838	0.7698	0.5748

. For all datasets, the proposed ICLTD maintained a competitive $\mathrm{A}\mathrm{U}{\mathrm{C}}_{TDBS}$ performance. HCEM also showed a stable and great $\mathrm{A}\mathrm{U}{\mathrm{C}}_{TDBS}$ performance. The $\mathrm{A}\mathrm{U}{\mathrm{C}}_{TDBS}$ results of STTD are the best for the MT-ABU-A1 and MT-ABU-B1 datasets, but are not ideal for the MT-ABU-A2 and MT-ABU-U2 datasets. Similarly, the $\mathrm{A}\mathrm{U}{\mathrm{C}}_{TDBS}$ results of ECEM are competitive but not promising for the MT-ABU-A1 and MT-ABU-B2 datasets. SFCTD is inferior to ICLTD, HCEM, STTD, and ECEM in terms of $\mathrm{A}\mathrm{U}{\mathrm{C}}_{TDBS}$, but superior to ACE, R-SA², and TSCNTD.

The $\mathrm{A}\mathrm{U}{\mathrm{C}}_{SNP\mathrm{R}}$ results are shown in

Table 4. $\mathrm{A}\mathrm{U}{\mathrm{C}}_{SNP\mathrm{R}}$ results for the MT-ABU datasets. The best results are indicated in bold.

	Dataset	A1	A2	B1	B2	U1	U2
Method
ACE		2.449	6.319	10.400	29.094	10.561	5.062
R-SA2		2.474	6.376	10.498	29.267	10.724	5.127
HCEM		21.997	61.046	149.261	591.155	112.825	55.556
ECEM		1.213	2.420	1.852	1.254	10.222	1.731
SFCTD		2.036	3.723	9.439	19.885	7.662	4.285
TSCNTD		0.197	1.715	9.674	20.905	0.606	2.420
STTD		231.164	10.794	538.713	13,189.051	320.844	343.675
ICLTD		409.669	789.488	717.131	2158.172	348.794	471.063

. The proposed ICLTD shows the highest overall $\mathrm{A}\mathrm{U}{\mathrm{C}}_{SNP\mathrm{R}}$ results, followed by STTD. HCEM is the third-best method in terms of $\mathrm{A}\mathrm{U}{\mathrm{C}}_{SNP\mathrm{R}}$ results, followed by ACE, R-SA², SFCTD, and ECEM. TSCNTD performed the worst among all methods.

According to the above results, the confidence differences between targets and backgrounds of the proposed method were significant for all the datasets from both the amplitude and ratio perspectives. HCEM also performed well from the amplitude perspective but not as well as the ICLTD in terms of amplitude ratio. The target–background separation performances of the other approaches were not as stable as those of HCEM and the ICLTD.

3.4.3. Detection Map Visualization

Detection map visualization can intuitively reflect the saliency of targets and the contrast between targets and backgrounds. Here, to show real contrast, the original detection results of HTD methods (except HCEM and ECEM) were visualized without normalization. When the maximum values outputted by HCEM and ECEM were larger than 1, their detection maps were normalized to 0–1 for visualization.

The visualization results are shown in Figure 11. The ICLTD is highly effective in providing accurate target distribution while suppressing the background spectra, resulting in optimal target–background contrast. The visualization results of HCEM are the second-best, whose background suppression performance was slightly worse than that of the ICLTD. However, its achieved target saliency for the MT-ABU-A2 dataset was even better than that achieved with the ICLTD. STTD and SFCTD both provided fairly comprehensive target distributions. STTD showed excellent target–background contrast but more false alarms. SFCTD presented a slightly lower target–background contrast compared to STTD. ECEM, ACE, and R-SA showed reasonable performance for certain datasets. However, they performed poorly for specific datasets, such as the MT-ABU-A1 dataset. According to the detection map visualization, TSCNTD only performed well for the MT-ABU-B2 dataset.

3.4.4. Inference Time Comparison

The training times of the DL methods were used for comparison because these methods need to retrain their respective parameters for new in-scene HSIs. However, when there is a large amount of data, DL-based methods can train models before detecting in-scene HSIs, just like object detection in RGB images [55]. In that case, these methods only require a small amount of time (inference time) to complete the detection task.

The consumed times of the ICLTD and the other methods are exhibited in

Table 5. Time consumption (s) of traditional approaches and DL methods on six datasets. ${ }^{*}$ represents the inference time of the DL methods. The best results are shown in bold.

	Dataset	A1	A2	B1	B2	U1	U2
Method
ACE		0.784	0.551	0.574	0.752	0.758	0.542
R-SA2		0.872	0.723	0.694	0.475	0.392	0.650
HCEM		14.942	1.374	2.022	5.916	2.734	3.445
ECEM		13.532	3.666	7.418	8.186	14.156	7.778
SFCTD		50.964	4.215	15.513	16.247	47.035	15.475
TSCNTD		90.925	10.716	35.577	37.047	89.884	36.882
STTD		133.413	14.486	53.971	54.563	133.266	54.133
ICLTD		37.193	14.245	21.611	21.457	35.574	22.234
SFCTD ${ }^{*}$		0.019	0.003	0.007	0.007	0.014	0.004
TSCNTD ${ }^{*}$		0.169	0.052	0.066	0.063	0.145	0.062
STTD ${ }^{*}$		3.061	0.281	1.376	1.188	2.883	1.357
ICLTD ${ }^{*}$		0.018	0.015	0.007	0.008	0.030	0.015

. We have also exhibited the inference times of the DL methods for reference. Of the compared methods, ACE and R-SA² are the most efficient. HCEM and ECEM take less time than the DL methods. In terms of DL methods, SFCTD and the ICLTD are more efficient due to their composition of fully connected layers. On the other hand, TSCNTD consists of multiple 1D convolution layers, and STTD includes several self-attention modules, resulting in longer training times compared to SFCTD and the ICLTD. Although the training times of the DL methods are longer than the time consumed by the classical approaches, their inference times are very fast. If the trained models can be directly applied to other similar in-scene HSIs, the efficiency of DL methods will be much higher than non-learning methods.

3.5. Ablation Study

The results in this section are used to validate and discuss the effectiveness of the ICLM and LSSC. The ICLM realizes implicit contrastive learning using in-scene spectra. It aims to regularize the learning process of classifying few-shot prior spectra by enabling the model to learn differentiated representations of in-scene spectra, rather than over-fitting the few-shot prior spectra. According to the analysis in Section 2.4, although the ICLM changes the input feature distributions, the work of the ICLM relies on sufficient gradient signals that propagate between the prior spectra and in-scene spectra.

In order to validate the aforementioned analysis and the effectiveness of the LSSC, the following four experiments were conducted:

• ICLM removal.
• Zeroing gradient signals in the ICLM.
• Replacing the ICLM with the original batch normalization (BN).
• LSSC removal.

Experiments 1–3 were conducted with the LSSC removed. The first experiment aimed to validate the effectiveness of the ICLM; the second aimed to validate that the effectiveness of the ICLM lies in signal transmission rather than changes in feature distribution; and the third experiment was designed to study the consequences of insufficient gradient signal transmission. Experiment 4 was conducted to validate the effectiveness of the LSSC. The results of these four experiments are denoted ICLTD-w/o-ICLM, ICLTD-ZG, ICLTD-BN, and ICLTD-w/o-LSSC, respectively.

The visualization results are exhibited in Figure 12. The AUC results are presented in

Table 6. $\mathrm{A}\mathrm{U}{\mathrm{C}}_{(D,F)}$ results for the ablation studies using the MT-ABU datasets. The best results are indicated in bold.

	Dataset	A1	A2	B1	B2	U1	U2
Exp
ICLTD-w/o-ICLM		0.8983	0.7621	0.9895	1.0000	0.9713	0.4907
ICLTD-ZG		0.7931	0.9891	0.9993	0.9994	0.8453	0.6297
ICLTD-BN		0.9949	1.0000	0.9995	0.9985	0.9867	0.9235
ICLTD-w/o-LSSC		0.9950	1.0000	0.9999	1.0000	0.9989	0.9991
ICLTD		0.9956	1.0000	0.9999	1.0000	0.9997	0.9993

,

Table 7. $\mathrm{A}\mathrm{U}{\mathrm{C}}_{TDBS}$ results for the ablation studies using the MT-ABU datasets. The best results are indicated in bold.

	Dataset	A1	A2	B1	B2	U1	U2
Exp
ICLTD-w/o-ICLM		0.0000	0.0053	0.0003	0.0011	0.0002	0.0001
ICLTD-ZG		0.2407	0.8372	0.9084	0.9026	0.5237	0.1482
ICLTD-BN		0.6028	0.9214	0.9151	0.6077	0.7296	0.3461
ICLTD-w/o-LSSC		0.1187	0.4267	0.7361	0.4559	0.5215	0.2742
ICLTD		0.4417	0.4125	0.8994	0.7838	0.7698	0.5748

and

Table 8. $\mathrm{A}\mathrm{U}{\mathrm{C}}_{SNP\mathrm{R}}$ results for the ablation studies using the MT-ABU datasets. The best results are indicated in bold.

	Dataset	A1	A2	B1	B2	U1	U2
Exp
ICLTD-w/o-ICLM		1.000	1.005	1.000	1.001	1.002	1.000
ICLTD-ZG		2.524	7.482	12.554	15.076	3.042	1.724
ICLTD-BN		40.127	29.447	47.522	38.923	23.485	30.188
ICLTD-w/o-LSSC		340.796	1000.888	999.948	1605.910	326.152	421.725
ICLTD		408.216	753.550	710.780	1749.182	342.586	465.132

. After removing the ICLM, optimization of the ICLTD collapsed and all the in-scene spectra were considered as targets with higher confidence. The AUC results and visualization of the ICLTD-w/o-ICLM are poor. The results of Experiment 1 prove the effectiveness of the ICLM in regularizing the optimization.

After zeroing the gradient signal transmission in the ICLM, the performance of target–background separation is better than that of ICLTD-w/o-ICLM, which is reflected by the visualization and $\mathrm{A}\mathrm{U}{\mathrm{C}}_{TDBS}$ results. However, the $\mathrm{A}\mathrm{U}{\mathrm{C}}_{(D,F)}$ and $\mathrm{A}\mathrm{U}{\mathrm{C}}_{SNP\mathrm{R}}$ results of ICLTD-ZG are unsatisfactory, which means that ICLTD-ZG mistakenly detects many background samples. The comparison of ICLTD-w/o-ICLTD and ICLTD-ZG validates that changing the feature distribution with the ICLM could improve the target–background separation. The comparison of ICLTD-ZG and ICLTD-w/o-LSSC proves that the gradient transmission by the ICLM is the key to success.

The $\mathrm{A}\mathrm{U}{\mathrm{C}}_{(D,F)}$ results after replacing the ICLM with the original BN are promising for all datasets. The target–background separation performance in terms of amplitude difference was excellent. However, the $\mathrm{A}\mathrm{U}{\mathrm{C}}_{SNP\mathrm{R}}$ results and background suppression performance of the ICLTD-BN are not as good as those of the ICLTD-w/o-LSSC. Especially for the MT-ABU-A2, MT-ABU-B1, and MT-ABU-U1 datasets, the ICLTD-BN mistakenly detected a few background spectra. These results validate that insufficient prior spectral features for normalization reduce the regularization strength, resulting in unpromising background suppression performance.

After removing the LSSC, the completeness of targets in the visualization results decreased. The $\mathrm{A}\mathrm{U}{\mathrm{C}}_{TDBS}$ results of the ICLTD-w/o-LSSC also degraded. The performance comparison between the ICLTD-w/o-LSSC and ICLTD proves the effectiveness of the LSSC in improving target detectability. It is worth noting that the background suppression ability of the ICLTD does not decrease significantly with the addition of the LSSC. Therefore, the LSSC can utilize local spectral similarities to improve target–background separation performance.

4. Conclusions

This paper introduced a DL-based method for HTD that effectively mitigates the challenge of spectral variability in multi-temporal HSIs. The proposed approach utilizes only prior spectra for supervised learning and does not rely on pseudo-data generation for optimization. By introducing implicit contrastive learning between the prior and in-scene spectra, the optimization process is regularized through a specifically designed ICLM. A theoretical analysis demonstrated the efficacy of the ICLM, showing that it successfully transfers differentiated gradient signals from prior spectral features to the representation of in-scene spectra, leveraging their inherent differences. This allows the ICLTD to learn discriminative representations of the in-scene spectra, avoiding over-fitting the prior spectra. Consequently, it is capable of effectively distinguishing variational in-scene target spectra from background spectra and adapting to the spectral variability in multi-temporal HSIs. Additionally, the proposed LSSC leverages the spectral similarity of multi-pixel targets within local neighborhoods to enhance target detectability. The performance of the proposed method was evaluated using three multi-temporal image sets obtained from AVIRIS data, which demonstrated its robustness under spectral variability. A comparison with classical detectors and DL detectors confirmed the superior performance of the proposed method in achieving a balance between target detectability and background suppression. Based on the experimental results, practical recommendations for application of the ICLTD emphasize the consideration of adjusting the strength of implicit contrast learning based on the intensity of spectral variations. In future research, exploring how to better combine implicit contrastive learning with annotation-based supervised learning from both theoretical and experimental perspectives can enhance the performance of hyperspectral target detection and expand its application scenarios.