Radar HRRP Open Set Target Recognition Based on Closed Classification Boundary

Abstract

Radar automatic target recognition based on high-resolution range profile (HRRP) has become a research hotspot in recent years. The current works mainly focus on closed set recognition, where all the test samples are assigned to the training classes. However, radar may capture many unknown targets in practical applications, and most current methods are incapable of identifying the unknown targets as the ’unknown’. Therefore, open set recognition is proposed to solve this kind of recognition task. This paper analyzes the basic classification principle of both recognitions and makes sure that determining the closed classification boundary is the key to addressing open set recognition. To achieve this goal, this paper proposes a novel boundary detection algorithm based on the distribution balance property of k-nearest neighbor objects, which can be used to realize the identification of the known and unknown targets simultaneously by detecting the boundary of the known classes. Finally, extensive experiments based on measured HRRP data have demonstrated that the proposed algorithm is indeed helpful to greatly improve the open set performance by determining the closed classification boundary of the known classes.

1. Introduction

A high-resolution range profile (HRRP) is a one-dimensional distribution of a target’s scattering centers along the radar line of sight, which has abundant geometrical structure information that can be used to identify the target. Thus, radar automatic target recognition (RATR) based on HRRP has attracted much attention in recent years.

The current methods mainly perform closed set recognition (CSR), such as [1,2,3,4,5], where all the test samples are assigned to the training classes. However, radar may encounter many unknown targets in the practical applications of RATR. When the unknown targets are captured, the current CSR models are incapable of identifying these targets as the ‘unknown’. Instead, these models will misjudge them as the known classes, which can be shown in Figure 1a. Obviously, this kind of classification could seriously affect the accuracy of RATR. Therefore, open set recognition (OSR, [6]) is proposed to solve this kind of recognition task that needs to identify the ‘unknown’ targets at the same time as classification.

As shown in Figure 1a, it is the half-open classification boundary that causes the classification error in an open set environment. Naturally, if we can determine a closed classification boundary for every known class, as shown in Figure 1b, then the sample inside the boundary will be classified as the corresponding known class, and the sample outside all the boundaries will be identified as an unknown class. Therefore, how to determine the specific closed classification boundary is the key to addressing the OSR problems.

There are some HRRP OSR works. According to the different ways of feature extraction, the existing methods can be mainly classified into two categories, i.e., traditional machine learning methods and deep learning methods.

The first category of methods is mainly based on support vector machine (SVM), support vector data description (SVDD), and random forest method. The authors of Chai et al. [7] proposed multi-kernel SVDD to describe the multi-mode distribution of HRRP; Zhang et al. [8] proposed a multi-classifier combination algorithm based on maximum correlation classifier, SVM, and relevance vector machine; Wang et al. [9] incorporated the extreme value theory into the random forest classifier for HRRP OSR. In addition, there are also some SVM-based methods focusing on public image datasets, such as 1-vs-set machine [6], $P_I$-SVM [10], one class SVM (OCSVM, [11]), and W-SVM [12] methods. Among them, [13] evaluated the performance of $P_I$-SVM on HRRP, and [14] evaluated the performance of two other SVM-based methods on MSTAR datasets. Among these methods, only SVDD can determine the closed classification boundary. Apart from the lack of determining the closed classification boundary, using the specific kernel function to extract distinguishable features may also limit the performance of these methods. With the development of deep learning, the methods based on the deep neural network have replaced these traditional machine learning methods.

The second category of methods mainly includes two kinds of models: discriminative models and generative models. The former focuses on proposing novel network architecture [15,16] and discriminative strategies [17,18,19,20,21,22] to extract more distinguishable features. Such as [16], Ma et al. [16] added another full-connection network branch to the end of the discriminator in the generative adversarial network (GAN, [23]), so the network structure has the ability to accomplish both classifying the known and identifying the unknown in synthetic aperture radar OSR. Zhang et al. [17] proposed a novel algorithm for open and imbalanced HRPP recognition tasks by learning from realistic data distribution and optimizing the accuracy over the majority, minority, and open classes.

In addition, there are some prototype learning-based methods that have achieved good performance on public image datasets, such as generalized convolutional prototype learning (GCPL, [24,25]), reciprocal points learning (RPL, [26]), and adversarial reciprocal points learning (ARPL, [27]). In prototype learning-based methods, the compactness of known features is usually improved to obtain more distinguishable features. Moreover, the distance between the sample and the prototype is used to evaluate the probability of belonging to the corresponding category, so this distance can be used to determine the closed classification boundary.

Different from discriminative models, new samples are generated in the training phase of generative models, and these samples are usually added into the training phase for better performance. Generally, the reconstruction error of the test sample is an important principle to identify the unknown classes in generative models, whose idea is similar to auto-encoder (AE, [28]). There are some representatives of generative models, such as [29,30,31,32]. In general, except for prototype learning-based methods, these deep learning methods almost ignore the importance of the closed classification boundary, which may limit their performance.

To fill the gap of the current methods, this paper proposes a novel boundary detection algorithm based on the distribution balance property of k-nearest neighbor objects (k-NNs). According to the theory of pattern recognition, a well-trained network can distinguish the known classes well and form stable clusters with clear boundaries in the feature space. Generally, the center points of these clusters are more densely distributed, while the boundary points are more sparsely distributed. Therefore, the k-NNs of a central point are more evenly distributed than that of a boundary point, which can be used to determine whether a point is a center point or a boundary point. Specifically, the proposed method calculates the distribution equilibrium coefficient and distance correction coefficient of a point based on its k-NNs to judge where it belongs, which are used to address the OSR problem in this paper. Finally, extensive experiments based on measured HRRP data are performed to evaluate the OSR performance of the proposed method, and the results verify the effectiveness of the proposed method.

The rest of this paper is organized as follows. Section 2 gives the general definition of OSR. In Section 3, the proposed method is described in detail. The experimental results based on measured data, as well as the comparison to some other methods, are discussed in Section 4. Finally, Section 5 concludes this paper.

2. Materials

To help our readers to further understand the OSR problems, this section gives the general definition of OSR.

Given a training set $T_{train}=\left\{({\mathit{t}}_i,y_i)\right|i=1,2,\cdots \}$, the label of training sample $y_i$ belongs to $\{1,\cdots ,N\}$. As for the test set $T_{test}$, it contains two parts: the known classes $T_k$ and the unknown class $T_u$. Specifically, the sample of $T_k$ and training set $T_{train}$ have the same category $\{1,\cdots ,N\}$. All samples in $T_u$ belong to $N+1$ category, which represents ‘unknown’. All raw samples come from m-dimensional full space ${\mathbb{R}}^m$, so OSR aims to determine a mapping function $f:{\mathbb{R}}^m\mapsto {\mathbb{N}}^{+}$ based on $T_{train}$, which can be expressed as

$$\left\{\begin{array}{c}{\mathit{t}}^j\in T_k\stackrel{f}{⟼}j,j\in \{1,\cdots ,N\}\hfill \\ \mathit{t}\in T_u\stackrel{f}{⟼}N+1\hfill \end{array}\right.$$

(1)

where ${\mathit{t}}^j$ represents the known samples of class j. As introduced in Section 1, OSR needs to not only classify the known classes, but also identify the unknown classes, which correspond to the 1st and 2nd line of Equation (1), respectively.

3. Methods

The flow diagram of the proposed method for HRRP OSR is shown in Figure 2. First, all raw HRRP samples need to be preprocessed to obtain the standardized training and test set. In the training phase, the preprocessed training data are used to train the network, which is a CSR training process. The trained network extracts the high-dimensional features from all training data, which are also called the known features or training features. After the training phase, this paper detects the closed classification boundary of these known features, which is the core of the proposed method. In the test phase, the specific category of the test features will be identified according to the closed classification boundary detected by the proposed method. Additionally, this paper hardly restricts the specific form of the loss function. In other words, the proposed method can be used in combination with a variety of CSR or OSR loss functions, which demonstrates the wide applicability of the proposed method.

3.1. Data Preprocessing

The way the radar acquires the HRRP data of the target determines the intensity sensitivity and translation sensitivity of the HRRP data. Therefore, the data preprocessing mainly aims at extinguishing both sensitivities.

3.1.1. Intensity Sensitivity

According to the working process of the radar, radar transmit power, target distance, radar antenna gain, radar receiver gain, and some other factors can affect the intensity of HRRP data, which indicates that the intensity may vary from target to target. Therefore, to avoid the recognition model taking the intensity information as the basis for judging the category of the target, normalization is usually used to eliminate the intensity sensitivity of the HRRP data. Specifically, $l_2$ normalization is used to process the raw HRRP data $\mathit{z}=(z_1,\cdots ,z_m)$, which can be expressed as

$$\tilde{\mathit{z}}=\frac{\mathit{z}}{{\parallel \mathit{z}\parallel }_2},$$

(2)

where ${\parallel \mathit{z}\parallel }_2$ represents the $l_2$ norm of the raw data $\mathit{z}$.

3.1.2. Translation Sensitivity

HRRP data are obtained from the echo signal intercepted by the sliding range window. The sliding of this distance window will affect the support area of HRRP data. Therefore, to avoid the influence of the sliding support area on the recognition result, it is necessary to carry out the corresponding translation operation on HRRP to obtain the standardized data. Specifically, this paper chooses the center of gravity self-alignment to eliminate translation sensitivity. The center of gravity of HRRP data $\tilde{\mathit{z}}=({\tilde{z}}_1,\cdots ,{\tilde{z}}_m)$ can be expressed as

$$G_{\tilde{\mathit{z}}}=round(\frac{{\sum }_{i=1}^{m}i{\tilde{z}}_i}{{\sum }_{i=1}^m{\tilde{z}}_i}).$$

(3)

After several cyclic shifts, the center of gravity $G_{\tilde{\mathit{z}}}$ of HRRP can be moved to the center position $round(m/2)$. As shown in Figure 3, the support area of HRRP moves to the center position after preprocessing.

3.2. Boundary Detection Algorithm

The boundary detection algorithm is the core of the proposed method, and this section gives a detailed introduction.

As shown in Figure 4a, the center of a cluster tends to have a denser distribution of points than the boundary of a cluster. In other words, the distribution of the center point’s k-NNs is more symmetric and balanced than that of the boundary point. Thus, a novel indicator, boundary detection coefficient (BDC), is proposed to describe the k-NNs’ distribution difference between the center point and the boundary point, which can be used to detect the boundary point of a cluster. Figure 4a visually shows the boundary detection results of the proposed method, whose distance distribution and BDC distribution can be seen in Figure 4b,c, respectively. Among them, a point with a larger BDC value is a boundary point.

Given a set of points $X=\left\{x_i\right|i=1,2,\cdots \}$, the k-NNs of $x_i$ are denoted as

$$x_i^{\mathbb{k}}=\{x_i^j|j=1,\cdots ,k\},$$

(4)

where $x_i^j$ is the j-th k-NNs of $x_i$. In this paper, the balance of k-NN distribution is described by the distribution equilibrium coefficient ($dec$) of $x_i$, which can be expressed as the following formula:

$$dec(x_i)=tanh(\parallel \sum _{j=1}^k(x_i^j-x_i){\parallel }_1),$$

(5)

where ${\parallel ·\parallel }_1$ represents $l_1$ norm operation, and the tanh function is used to compress the range of $dec$ into $(0,1)$. Obviously, the better the balance of k-NN distribution, the smaller $dec$.

In fact, it is insufficient to use $dec$ alone to distinguish the center and boundary point since it is possible that the k-NNs of a boundary point are distributed evenly. Nevertheless, the point distribution density of the center and boundary regions are still different, which can be used to further distinguish the center and boundary points. Specifically, this paper chooses the distance between k-NNs and the test point to describe this kind of density difference. Thus, a distance correction coefficient ($dcc$) is proposed to improve $dec$, which can be expressed as

$$dcc\left(x_i\right)=tanh(\sum _{j=1}^k\parallel x_i^j-x_i{\parallel }_2),$$

(6)

where ${\parallel ·\parallel }_2$ represents $l_2$ norm operation. Likewise, the better the balance of k-NNs distribution, the smaller $dcc$.

In this paper, both $dec$ and $dcc$ are used to describe the distribution balance of k-NNs, which can be expressed as

$$\mathbb{D}\left(x_i\right)=dec\left(x_i\right)\times dcc\left(x_i\right),$$

(7)

where $\mathbb{D}(·)$ is the proposed BDC. Obviously, the largest terms in $\mathbb{D}\left(X\right)$ are the boundary points of the cluster.

Next, we explain how to use BDC to detect the boundary points in the proposed OSR problem. Assuming that there are n samples in the training set $T_{train}$, the training features can be denoted as

$$S=\mathsf{\Theta }\left(T_{train}\right)=\left\{\mathsf{\Theta }\left({\mathit{t}}_i\right)\right|i=1,\cdots ,n\},$$

(8)

where $\mathsf{\Theta }(·)$ is the embedding function of the trained network, and ${\mathit{t}}_i$ is the i-th training sample of the training set $T_{train}$. Then, $\mathbb{D}\left(S\right)$ should be calculated to detect the boundary points of S, and the corresponding algorithm is shown in Algorithm 1. The calculation complexity of every procedure is also provided in Algorithm 1. It shows that the calculation complexity mainly depends on the detection procedure of the k-NNs, whose complexity is $O\left(n^2\right)$. Apart from the hyper-parameter k, there is another hyper-parameter $\varphi$ that will affect the boundary detection results of the algorithm. This parameter determines the ratio of boundary points to all points, so its value is usually low, such as $0.1$ or $0.05$. After all, there are only a few boundary points in the cluster.

Algorithm 1 Boundary detection algorithm.

Require:  Training features $S=\mathsf{\Theta }\left(T_{train}\right)=\left\{\mathsf{\Theta }\left({\mathit{t}}_i\right)\right|i=1,\cdots ,n\}$, embedding function $\mathsf{\Theta }(·)$, parameter k, and threshold $\varphi \in (0,1)$. 1: Calculate the k-NNs of every feature in S. $▹ O\left(n^2\right)$ 2: Calculate the BDC of S: $\alpha =\mathbb{D}\left(S\right)$. $▹O\left(n\right)$ 3: Sort $\alpha$ inversely, and the sorted vector is denoted as $\beta$. $▹O(nlogn)$ 4: Record the $\alpha$ index of the first $round\left(n\varphi \right)$ element in $\beta$, and this index vector is denoted as $\xi$. $▹O\left(n\right)$ 5: return Boundary points of S: $S_{\xi }$ (the points in S whose index is in $\xi$).

3.3. Identification Procedure

This section discusses how to use the proposed boundary detection algorithm to identify the unknown classes.

After the training phase, the training features form N clusters with definite boundaries in the feature space, and the boundary points have been detected by the proposed method. At this time, for any test sample ${\mathit{t}}^{\prime }$, what we need to do is to investigate the relationship between its embedding feature $\mathsf{\Theta }\left({\mathit{t}}^{\prime }\right)$ and the detected boundary in the feature space. Obviously, if the feature $\mathsf{\Theta }\left({\mathit{t}}^{\prime }\right)$ is outside the detected boundary, then ${\mathit{t}}^{\prime }$ should be classified as the unknown class. Conversely, if the feature $\mathsf{\Theta }\left({\mathit{t}}^{\prime }\right)$ is within the detected boundary, then ${\mathit{t}}^{\prime }$ should be recognized as a known class. As for how to judge whether feature $\mathsf{\Theta }\left({\mathit{t}}^{\prime }\right)$ is inside or outside the detected boundary, it is accomplished by the value of BDC of the proposed method. Specifically, if the BDC of feature $\mathsf{\Theta }\left({\mathit{t}}^{\prime }\right)$, $\mathbb{D}\left(\mathsf{\Theta }\left({\mathit{t}}^{\prime }\right)\right)$ is greater than the BDC of the detected boundary, it is judged to be outside the boundary. The specific algorithm is shown in Algorithm 2.

Algorithm 2 Identification algorithm.

Require:  Training features $S=\mathsf{\Theta }\left(T_{train}\right)$, embedding function $\mathsf{\Theta }(·)$, parameter k, threshold $\varphi \in (0,1)$, the sorted vector $\beta$, the center of i-th known cluster $C^i$ and the test sample ${\mathit{t}}^{\prime }$. 1: Calculate the feature of the test sample ${\mathit{t}}^{\prime }$: $\mathsf{\Theta }\left({\mathit{t}}^{\prime }\right)$. 2: Calculate the k-NNs of $\mathsf{\Theta }\left({\mathit{t}}^{\prime }\right)$ in S. 3: Calculate the BDC of $\mathsf{\Theta }\left({\mathit{t}}^{\prime }\right)$: $\mathbb{D}\left(\mathsf{\Theta }\left({\mathit{t}}^{\prime }\right)\right)$. 4: The predicted category $\widehat{y}$ of ${\mathit{t}}^{\prime }$ can be identified by the following formula: $$\widehat{y}=f\left({\mathit{t}}^{\prime }\right)=\left\{\begin{array}{cc}\underset{i}{\mathrm{arg\; min}}{\parallel \mathsf{\Theta }\left({\mathit{t}}^{\prime }\right)-C^i\parallel }_2,\hfill & \mathrm{if}\mathbb{D}\left(\mathsf{\Theta }\left({\mathit{t}}^{\prime }\right)\right)\le {\beta }_{round\left(n\varphi \right)},\hfill \\ N+1,\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$ (9) where ${\beta }_{round\left(n\varphi \right)}$ represents the element in $\beta$ that is indexed as $round\left(n\varphi \right)$. 5: return the predicted category $\widehat{y}$ of the test sample ${\mathit{t}}^{\prime }$.

As shown in Algorithm 2, it gives all calculation processes of OSR with the proposed method and establishes a complete mapping function f from the raw data to the predicted label. In fact, the proposed method mainly focuses on how to identify the unknown classes while classifying the known classes requires additional operations. Specifically, when the test sample ${\mathit{t}}^{\prime }$ is identified as the known class, the 1st line of Equation (9) gives the specific predicted known class of ${\mathit{t}}^{\prime }$, which is in $\{1,\cdots ,N\}$. Although the core of the proposed method is based on the k-NN classification algorithm, the proposed method does not use the k-NNs of the test sample to determine its specific category. Instead, the proposed method determines the category of the sample feature by its distance from the feature center of the known cluster $C=\left\{C^i\right|i=1,\cdots ,N\}$, which can be expressed as

$$C^i=\frac{1}{n^i}\sum _{j=1}^{n^i}\mathsf{\Theta }\left({\mathit{t}}_j^i\right),$$

(10)

where $C^i$ represents the center of the i-th known cluster, $n^i$ represents the number of i-th classes training samples, and ${\mathit{t}}_j^i$ represents the j-th training sample of i-th classes. It is believed that using the feature information of the whole category is more beneficial for classification than using the local k-NNs information.

4. Results and Discussion

4.1. Experimental Settings

4.1.1. Data

This paper conducts the OSR experiments on 13 kinds of measured airplane HRRP data, which are transmitted vertically and received vertically in linear polarization mode. Among them, An-26, Cessna, and Yak-42 are selected as the known classes, whose flight path and preprocessed HRRP are shown in Figure 5. The 1st column of Figure 5 shows the flight path of the known classes, which are cut into several segments. Specifically, the 5th and 6th segments of An-26, 6th and 7th segments of Cessna, and 2nd and 5th segments of Yak-42 are selected as the training samples, while the remaining segments are used as test samples. The reasons for such data division are as follows: first, the selected path can cover a relatively comprehensive flight attitude, which is conducive to avoiding the impact of target attitude on the recognition task; Second, the training data and the test data are not completely consistent, which is conducive to testing the generalization ability of the model. The 2nd column of Figure 5 shows that the support area width of three aircraft HRRP is different, which also reflects the size of the aircraft size from one side.

Table 1. HRRP data of known classes overview.

Class	Training Samples	Test Samples
An-26	17,334	10,400
Cessna	17,333	13,000
Yak-42	14,650	7800

shows the number information on these three known classes. In addition, the remaining 10 kinds of aircraft each have 3000 samples, and they are used as the unknown classes for open set evaluating, whose specific category information can be seen in Section 4.4.

4.1.2. Implementation Settings

This paper uses a simple one-dimensional convolutional neural network as the classifier, which has three convolutional layers and two fully-connected layers. In addition, the stochastic gradient descent-momentum optimizer is used to train the classifier for 100 epochs. The learning rate is set to $0.1$ and drops to one-tenth of the original after 30 epochs. The training batch size is set to 128.

4.2. Identification Evaluation

This experiment focuses on the ability of the model to identify the unknown classes. Thus, the area under the receiver operating characteristic curve (AUROC) is used to evaluate different methods. The specific experimental settings are shown in Section 4.1, and the results are shown in

Table 2. This table shows the experimental AUROC(%) results of various methods. Specifically, the 3rd, 4th, and 5th columns of this table are the experimental results of the proposed method combined with the corresponding loss function under parameter k. The best experimental results are shown in bold.

Method	AUROC	$\mathit{k}=5$	$\mathit{k}=7$	$\mathit{k}=9$	$\mathit{k}=11$
OCSVM [11]	66.7	-	-	-	-
AE [28]	70.5	-	-	-	-
AAE [33]	76.4	-	-	-	-
EVM [34]	79.8	-	-	-	-
SoftMax [22]	67.8	90.7	85.9	79.6	72.0
OpenMax [21]	69.7	-	-	-	-
OSRCI [22]	79.6	-	-	-	-
GCPL [24]	77.4	94.5	94.6	94.6	94.4
RPL [26]	78.7	93.6	93.9	93.9	93.8
ARPL [27]	71.1	96.2	96.2	96.2	95.0
ARPL + CS [27]	73.5	93.3	93.4	93.3	91.9
Ring [35]	75.1	96.4	96.4	96.4	96.4

.

Obviously, when the proposed method is combined with different loss functions, the identification performance is greatly improved. In fact, the proposed method and other methods use the same feature extraction network. Thus, the performance improvement is entirely due to the boundary detection function of the proposed method, which strongly verifies the effectiveness of the proposed method. Moreover, the effect of parameter k on the performance of the proposed method is also investigated in this experiment. As shown in Table 2, except for SoftMax, the performance of the proposed method combined with other methods has slightly changed with the increases in parameter k. We speculate that this result may be related to the compactness of the feature clusters because all other methods have the ability to improve the compactness of the inter-class features. Moreover, receiver operating characteristic (ROC) curves of the proposed method combined with several loss functions are also presented, as shown in Figure 6, such as boundary detection SoftMax (BD_SoftMax), BD_ARPL, and BD_Ring. Finally, it is not certain which loss function combined with the proposed method will obtain the best performance, which is a limitation of the proposed method.

4.3. Category Ablation Experiment

This experiment intends to investigate the performance change in the proposed method as the number in the unknown class increases gradually. Thus, 10 unknown classes are successively added to the test set for the category ablation experiment. In this experiment, macro average F1-score (F1) is used to evaluate the comprehensive performance of different methods, including the ability to classify the known classes and identify the unknown classes. Its expression is as follows:

$$\mathrm{F}1=\frac{1}{N+1}\sum _{i=1}^N\frac{2\times P_i\times R_i}{P_i+R_i},$$

(11)

where $P_i$ and $R_i$ represent the Precision and Recall of i-th category, respectively. The experimental results are shown in Figure 7. The horizontal axis of Figure 7 is openness [6], which can be expressed as

$$\mathbb{O}=1-\sqrt{\frac{2\times N_{train}}{N_{test}+N_{target}}},$$

(12)

where $N_{train}$ is the number of known classes, $N_{test}$ is the number of test classes that includes the known and unknown classes, and $N_{target}$ is the number of known classes that need to be correctly recognized during testing. Obviously, the difficulty of OSR increases with increasing $\mathbb{O}$, so Figure 7 shows the performance of all methods decreases with increasing $\mathbb{O}$. Nevertheless, the proposed method still achieves the best performance when combined with the Ring loss function, as shown in Figure 7d. Moreover, the performance degradation rate of the proposed method is much lower than that of other methods, which further demonstrates that the proposed boundary detection method can effectively improve the OSR performance. Finally, this experiment shows that the proposed method can still obtain performance advantages in application scenarios with many unknown classes, which also verifies the effectiveness of the proposed method.

4.4. Threshold Discussion

As is known, radar target detection need to set a specific detection threshold to judge whether there is a target in the echo. Specifically, when the echo intensity is higher than the threshold, it is considered that there is a target in the range window. When the echo intensity of the target is lower than the threshold, the radar will miss this target. Therefore, the setting of the detection threshold directly affects the results of target detection, which is very important in real applications. Similarly, the setting of thresholds also plays a significant role in OSR problems, which needs to determine a specific threshold for recognizing a test sample as a specific category in practical applications. Therefore, this section discusses how threshold $\varphi$ affects the performance of the proposed method (Section 4.2 does not discuss the threshold because the evaluation of the metric AUROC is independent of the threshold $\varphi$ in the proposed method).

Section 3.2 has introduced that $\varphi$ represents the ratio of boundary points to all points. In other words, the identification process of the proposed method considers that only samples in the center $1-\varphi$ ratio of known classes can be identified as ‘known’. Thus, it can be speculated that as $1-\varphi$ increases, the ability of the model to recognize known classes increases. Nevertheless, when the value of $\varphi$ is too low, the boundary range may be so large that it reduces the ability of the model to identify the unknown samples. Therefore, there may be an equilibrium point for the value of $\varphi$, which makes the comprehensive performance of the model reach its best. This phenomenon is the same as in radar target detection. If the detection threshold is set too high, fewer targets are detected and the higher the missed detection rate is. If the detection threshold is set too low, more targets are detected and the higher the false alarm rate will be. Therefore, there is an optimal detection threshold that can balance the missed detection rate and false alarm rate.

To explore the best threshold, this experiment examines the performance of different methods as a function of $\tau =1-\varphi$. Apart from the experimental settings in Section 4.1, parameter k is set to 5, and the experimental results are shown in Figure 8. As shown in Figure 8a–c, when the proposed method is combined with SoftMax and Ring, such as BD_SoftMax, BD_GCPL, and BD_Ring, the performance is greatly improved, which verifies the effectiveness of the proposed method. As expected, the performance of the model increases first and then decreases with the increase in the threshold $\tau$. In addition, when the threshold $\tau$ increases to a certain extent, the performance of the proposed method suddenly decreases significantly, indicating that the threshold is close to the boundary of the feature cluster. Figure 8d shows that the proposed method BD_Ring achieves astonished performance among many methods.

Moreover, Figure 9 shows several confusion matrices of BD_Ring with different thresholds $\tau$. Obviously, with the increase in $\tau$, fewer boundary points are detected by the proposed method, and the recognition ability of the proposed method for known classes is becoming stronger and stronger. At the same time, the ability of the proposed method to identify the unknown classes is slightly decreased with increasing threshold $\tau$. Nevertheless, the proposed method can still effectively identify the unknown classes. Although the threshold $\tau$ is close to 1, there is still a gap between the recognition probability of the model for the known classes and 1, which is caused by the fact that the samples of the test set and the training set are incapable of completely meeting the same distribution. Taking ‘Cessna’ as an example, as shown in Figure 5c, the flight path data of the 6th and 7th segments are selected as the training set because it is believed that the circular flight path is considered to cover enough flight attitude. However, the attitude of the target is also related to the pitch angle of the target captured by the radar. Therefore, the target attitude of the flight path’s 6th and 7th segments is incapable of representing the target attitude of the other five flight paths, which results in the low classification rate of known classes in Figure 9. Additionally, these confusion matrices also show us that some unknown targets are easier to identify than others. Such as the class ‘CRJ-900’, even though the threshold $\tau$ has been selected as $0.999$, the proposed model can still identify it with $100\%$ probability. By contrast, class ‘A320’ is more likely to be misjudged as ‘Yak-42’, which may result from the nature of the targets themselves.

4.5. Feature Visualization Analysis

This section gives more feature visualization results to better demonstrate the application of the proposed method in HRRP OSR.

The proposed method can identify not only the boundary points of two-dimensional clusters, as shown in Figure 4a, but also the boundary points of high-dimensional clusters. In our experiments, the features extracted from the network structure are all 128 dimensional features, which are difficult to be visualized directly. Therefore, this section visualizes the high-dimensional feature distribution by using the t-SNE [36] method to reduce the dimension. As shown in Figure 10a, the three known classes features can be well distinguished, while most of the unknown features can be distinguished except that a few of them overlap with the known features. In fact, the shape of known clusters can be extremely complex, especially when the dimension of the feature space is very high. Under this circumstance, it is naive and inappropriate to use the distance from the sample feature to the cluster center or to the prototype to determine the boundary points of a known cluster, which is why those prototype-based methods have poor performance, such as GCPL, RPL, and ARPL. For example, as shown in Figure 10b–d, although the proposed method can detect the boundary points of the known clusters, it is obviously difficult to identify the unknown samples based on the distance between the features of these boundary samples and the center of known clusters. In contrast, as shown in Figure 10e, the proposed method is much more reliable in identifying the unknown samples based on the BDC of the sample features. Compared with the known samples, the BDC distribution of the unknown samples is significantly higher. Therefore, it is easy to determine a BDC boundary to better distinguish between the known and unknown samples.

4.6. Computational Complexity

This section only discusses the computational complexity of the different methods in the test phase because the training phase of the model is not involved in practical applications. As for the training phase, Algorithm 1 has given more details.

First, the computational complexity of a deep learning algorithm is highly related to the network structure it uses. The larger and more complex the neural network structure is, the higher the computational complexity is. Therefore, this section does not discuss the computational complexity of using the network structure to extract the features, which corresponds to the 1st step in Algorithm 2. In fact, the proposed method in this step is the same as the other methods.

After extracting the test features, the decision stage involving f function will be directly entered for many OSR methods, such as SoftMax, GCPL, and RPL. Obviously, the computational complexity of this decision stage is $O\left(q\right)$ (where q is the number of test samples). Therefore, most methods have a computational complexity of $O\left(q\right)$ in the test phase if the computational amount of feature extraction is not taken into account. In terms of the proposed method, it is easy to find out that the computational complexity of the 2nd, 3rd, and 4th steps in Algorithm 2 are $O\left(nq\right)$, $O\left(q\right)$, and $O\left(q\right)$, respectively. Therefore, the computational complexity of the proposed method in the test phase is $O\left(nq\right)$. Specifically, when the number of training samples n is much larger than the number of test samples q, the computational complexity of the proposed method will be much larger than that of other methods. When n and q are roughly equivalent, the computational complexity of the proposed method is on the same order of magnitude as other methods.

5. Conclusions

This paper aims to solve the OSR problems on radar HRRP data. First, in view of the shortcomings of current OSR works on HRRP, this paper proposes that the key of OSR is to determine the closed classification boundary of the known clusters in the feature space. Then, this paper proposes a novel boundary detection algorithm for OSR, which is based on the distribution balance property of k-NNs. Generally, the k-NN distribution of the center points is more balanced and symmetrical than that of the boundary points. Thus, the proposed method designs the BDC of a sample according to the k-NNs of this sample, which can be used to evaluate whether this sample is more likely to be a center point or a boundary point. Finally, the proposed boundary detection algorithm is applied to HRRP OSR in this paper, and extensive experiments based on measured HRRP data are performed to verify the effectiveness of the proposed method.

In fact, the proposed method also has some limitations, such as high computational complexity in the boundary detection stage. Moreover, the proposed method is incapable of limiting the specific form of the loss function used to train the neural network. On the one hand, it shows that the proposed method has wide applicability. However, on the other hand, it also shows that it is difficult to determine which loss function can be used to achieve better performance in the proposed method. Empirically, loss functions that make inter-class features more compact seem to improve the performance of the proposed method. Therefore, future work will focus on loss function, which can further improve the compactness of the inter-class features.