Classification of Personnel Targets with Baggage Using Dual-band Radar

Abstract

In this paper, we aim to identify passengers with different baggage by analyzing the micro-Doppler radar signatures corresponding to different kinds of gaits, which is helpful to improve the efficiency of security check in airports. After performing time-frequency analysis on the X-band and K-band radar data, three kinds of micro-Doppler features, i.e., the period, the Doppler offset, and the bandwidth, are extracted from the time-frequency domain. By combining the features extracted by dual-band radar with the one-versus-one support vector machine (SVM) classifier, three kinds of gaits, i.e., walking with no bag, walking with only one carry-on baggage by one hand, and walking with one carry-on baggage by one hand and one handbag by another hand, can be accurately classified. The experimental results based on the measured data demonstrate that the classification accuracy using dual-band radar is higher than that using only a single-band radar sensor.

1. Introduction

The classification of personnel targets with different sizes of baggage has the potential to improve the efficiency of security checks in airports [1,2,3]. For example, individuals with no bag, with a small bag, and with large luggage can be categorized and arranged into different places for security checks. In this paper, we investigate how to automatically achieve this goal with continuous wave (CW) radar sensors because of its numerous benefits, including privacy, the robustness against illumination, accurate persistent monitoring capabilities, no physical discomfort caused by wearing motion sensor devices or modifying human normal behavior, and its low cost compared to other sensors such as high-resolution video camera systems [4,5,6]. When radar observes a moving target, the main Doppler shift of the received signal corresponds to the bulk motion of the target, while the micro-Doppler effect represents the motions of the parts of the target relative to its main body [7,8]. Based on this fact, the micro-Doppler effect can be used as a powerful tool for inferring the motion status of the target [9].

There have been many studies published on classification systems based on micro-Doppler information. The classification among different kinds of targets was shown in References [10,11,12]. The authors extracted very basic information to recognize walking humans using a spectral analysis with a simple classifier in Reference [13]. In Reference [14], the authors used a continuous wave radar to distinguish between different persons or other moving objects. In Reference [15], the authors classified seven kinds of human activities by selecting six features from the time-frequency spectrogram. In Reference [16], specific components of micro-Doppler gait signatures related to parts of the body at a long range for identification purposes was shown. In Reference [17], armed/unarmed personnel targets were distinguished based on multi-static micro-Doppler signatures. The authors of Reference [18] classified eight types of specific finer-grained human activities using through-wall stepped frequency continuous wave (SFCW) radar. An automatic procedure to detect, in real-time, the presence of one or several human subjects behind walls is discussed in Reference [19]. The method in Reference [20] is capable of recognizing men and women. All of the above methods are based on the empirical selection of features in the time-frequency domain with single band radar. Other approaches of feature selection include the linear predictive coding (LPC) [21], the principal component analysis (PCA) [22], the singular value decomposition (SVD) [23], the empirical mode decomposition (EMD) [24], the hierarchical image classification architecture (HICA) [25], and the deep convolutional neural networks (DCNN) [26,27], etc. The existing literature indicates that, (1) the extraction of proper features is of great importance for accurate classification and (2) the appropriate features may vary with different applications.

In this paper, we focus on the classification of three kinds of human walking postures in a realistic manner, i.e., walking with no bag, walking with only one carry-on baggage by one hand, and walking with one carry-on baggage by one hand and one handbag by another hand, with both X-band and K-band radar sensors. Different from most of the existing methods that only use single-band radar, the proposed method uses two micro-Doppler radar sensors operating simultaneously in different bands for the classification of personnel targets with baggage. The time-frequency analysis is firstly performed and three features, i.e., the period, the Doppler offset, and the bandwidth, are extracted from the time-frequency spectrogram at each radar sensor. Since the human object is observed with dual-band radar sensors at the same time, the dimension of the extracted features is increased by feature fusion, which is helpful to improve the classification accuracy compared to using only single radar sensor [28,29]. The fused features are input into the one-versus-one support vector machine (SVM) to achieve the classification. The experimental results based on measured data demonstrate that the proposed method works well to classify the three gaits of interest and the classification accuracy using dual-band radar is higher than that of single radar operating alone, whether the single band radar is X-band radar alone or K-band radar alone.

The remainder of this paper is organized as follows. Section 2 describes the field experiment setup with X-band and K-band CW radars. Section 3 presents the detailed steps of the proposed method. Experimental results on the measured data are presented in Section 4. Conclusion is given in Section 5.

2. Experimental Setup

Two CW mode radars, which adopt the I/Q demodulation because it is possible to discriminate between positive and negative frequencies corresponding to front and back micro-motions, are used in our experiments. The carrier frequency and the base-band sampling frequency are 25 GHz and 2 kHz for the K-band radar system, respectively, and the carrier frequency and the base-band sampling frequency are 9.8 GHz and 2 kHz for the X-band radar system, respectively. The power output of the X-band radar and K-band radar are 18 dBm and 16 dBm, respectively. The gain of horn antenna used in the X-band radar and K-band radar are 10 dB and 18 dB, respectively. Figure 1a shows the experimental scene, where two radars are located on a platform suspended one meter above the ground in an indoor environment under the line-of-sight condition. The distance between two radars is two meters, and one person walks directly toward the radars with a constant speed. The initial range between the person and the radar sensors is 10 m. Three different human gaits are considered, as shown in Figure 1b, i.e., gait A: walking with no bag; gait B: walking with only one carry-on baggage in one hand; and gait C: walking with one carry-on baggage in one hand and one handbag in another hand.

For each gait, 20 recordings per personnel target were collected and each 4-s time interval was recorded as a signal segment. Five personnel targets took part in the experiment and only a single subject was observed by both radar sensors for each trial. Three men and two women, all of them students of the Tsinghua University, served as volunteers to acquire the experiment data. Five volunteers were denoted as individual 1, 2, 3, 4, and 5, respectively. The information on the volunteers (age, sex, height, weight, etc.) are described as follows. Individual 1: female, 25 years old, height 1.6 m, weight 45 kg; individual 2: female, 25 years old, height 1.68 m, weight 60 kg; individual 3: male, 20 years old, height 1.7 m, weight 55 kg; individual 4: male, 25 years old, height 1.8 m, weight 75 kg; and individual 5: male, 20 years old, height 1.75 m, weight 60 kg. The amplitudes of arms swinging when they walking at a normal speed were about forward 0.3 m and backward 0.2 m distance from the torso for all of the subjects. The total number of recordings was (2 radars) × (5 persons) × (3 gaits) × (20 measurements) = 600. The size and the weight of the handbag used in the experiment were approximately 0.2 m × 0.4 m × 0.1 m and 3 kg, respectively, while the size and the weight of the carry-on baggage were approximately 0.4 m × 0.5 m × 0.2 m and 10 kg, respectively.

3. The Proposed Method

The proposed method consisted of three major steps, which will be described in details in this section.

3.1. Time-Frequency Analysis

The first step was to obtain the micro-Doppler information from the measured data, which was carried out by the short-time Fourier transform (STFT). In the time-frequency spectrogram, the strongest components, which mainly spread in the low Doppler frequency area, corresponded to the movement of the torso. The value of the torso’s Doppler frequency indicated the walking speed of the person. The periodic micro-Doppler components stood for the motions of limbs, and the peak value of micro-Doppler component corresponded to the largest speed of swinging limbs. Time-frequency spectrograms of three kinds of gaits obtained by the X-band and K-band radar sensors are shown in Figure 2.

3.2. Feature Extraction

Here, three features, i.e., the period, the Doppler offset, and the bandwidth, were extracted from the time-frequency spectrogram [15,23]. In order to obtain the exact feature values, two envelopes of the time-frequency signal needed to be extracted firstly, i.e., the positive-frequency envelope and the negative-frequency envelope. There have been many approaches on envelope extraction [30,31]. After extracting two envelopes, the three features can be extracted from the time-frequency domain, which will be described in details as follows.

3.2.1. Period

Period is the time interval between the adjacent two highest positive peaks. The period can be obtained through auto-correlation [32]. The mathematical expression of auto-correlation function can be written as:

$$R_{SS}(\tau )={\displaystyle \sum _{t=0}^{L_S}s(t)s^{*}(t-\tau )},$$

(1)

where $s(t)$ is the time-frequency signal, which is obtained by the STFT of the raw data, * is the conjugate, $\tau$ is the time delay, and $L_S$ is the length of the time-frequency signal. If the time delay $\tau$ is zero, the value of the auto-correlation function reaches the maximum. By calculating the time interval between the peaks of the absolute value of the autocorrelation function, the period of the time-frequency signal can be estimated.

3.2.2. Doppler Offset

In order to better describe the following two features, i.e., Doppler offset and bandwidth, the torso’s Doppler frequency, the positive peak value, and the negative peak value are denoted as $f_0$, $f_{+}$ and $f_{-}$, respectively, as shown in Figure 3a. We found that there were many peak frequency values during the observation time. The strongest component in the time frequency domain corresponded to the torso’s Doppler frequency $f_0$, which was estimated by accumulating the time-frequency signal over the time axes and finding the peak of frequency spectrum, as shown in Figure 3b. We also extracted the positive-frequency envelope and the negative-frequency envelope by using the method in Reference [31], as shown in Figure 3c. After extracting two envelopes from the time-frequency spectrogram, several peak values, i.e., $f_{+}$ and $f_{-}$ were estimated by peak finding.

The Doppler offset indicates the symmetry of the frequency components and would be zero if the swinging limbs forward and backward are exactly symmetric. Here, the length of each step along the time axes means the distance from the location of the middle position of $f_{+}^{(n-1)}$ and $f_{+}^{(n)}$ to the location of the middle position of $f_{+}^{(n)}$ and $f_{+}^{(n+1)}$. The definition of the Doppler offset can be expressed as:

$$F_O=\frac{1}{N}{\displaystyle \sum _{n=1}^N\left[\left(f_{+}^{(n)}-f_0\right)-\left(f_0-f_{-}^{(n)}\right)\right]}=\frac{1}{N}{\displaystyle \sum _{n=1}^N\left(f_{+}^{(n)}+f_{-}^{(n)}-2f_0\right)},$$

(2)

where $N$ is the total number of steps during the observation time, and $f_{+}^{(n)}$ and $f_{-}^{(n)}$ denote the positive and negative values of the micro-Doppler peaks in the n-th step, respectively.

3.2.3. Bandwidth

Bandwidth is the frequency range that the received signal occupies. Employing the same method as the Doppler offset, several peak values, i.e., $f_{+}$ and $f_{-}$, can be estimated from the positive-frequency and negative-frequency envelopes. The bandwidth can be described as:

$$F_B=\frac{1}{N}{\displaystyle \sum _{n=1}^N\left[\left(f_{+}^{(n)}-f_{-}^{(n)}\right)\right]},$$

(3)

From Figure 2 and the definition of the three features, we can find that the period of gait B is almost two times as that of gait A and gait C due to the asymmetry of the motions of the left arm and the right arm in gait B. The bandwidths of gait A and gait B are at the same level and much larger than that of gait C, because the motions of arms are limited in gait C. The Doppler offset of gait B is obviously different from that of gait A and C, because only one arm is swinging and another arm keeps almost static during the observation time in the case of gait B. The combination of bandwidth and Doppler offset or the combination of bandwidth and period can classify gait A from gait B and gait C. Thus, the period, the Doppler offset, and the bandwidth are capable of classifying the three gaits of interest.

Figure 4 shows the feature distribution of the experimental data using K-band radar sensor alone and using X-band radar sensor alone, in which there are one hundred data recordings and three selected features for each gait. The three axes denote the three features used for classification, namely the period, the Doppler offset, and the bandwidth. From Figure 4, we can see that the feature vectors of the three gaits are clustered in different space regions, which gives an intuitional interpretation of the effectiveness of classification of the three gaits based on the selected three features. After fusing the features extracted by dual-band radar, the dimension of the features will be six.

3.3. Classification

The final step of the proposed method is to input the features extracted by both the X-band radar and K-band radar into a proper classifier and execute the process of classification. Since dual-band radar is used to observe the same human object, the dimension of features used as the input of the classifier is two times as that of one radar operating alone. In this paper, we adopted the one-versus-one SVM method to deal with the multi-class classification problem [33,34]. The one-versus-one SVM strategy operates by constructing one SVM classifier for each pair of classes. Therefore, for a problem with k classes, k (k − 1)/2 SVMs are trained to distinguish all possible pairs of classes. Then, the majority voting is performed to obtain the final classification result. The actual steps were as follows: Firstly, we built three double-class SVMs named SVM1, SVM2, and SVM3, respectively. SVM1 was obtained with the training data of gait A and gait B. Similarly, SVM2 was obtained with the training data of gait B and gait C, and SMV3 was obtained with the training data of gait C and gait A. Note that the training data of each class were selected randomly from the entire data set, and the rest were used as the test data. The test data were input into the three classifiers separately. With the outputs of the three classifiers, a majority vote operation was applied; if more than one classification result labels were the same, then this label was chosen as the final classification result for the test data. If all the class labels put out by the three classifiers were different from each other, the randomly selected result of any classifier was used as the final result, since one cannot exactly determine which classification result is more reliable in this case.

4. Classification Results

The total number of recordings obtained from the field experiment was 600 for the dual-band radar. To evaluate the classification results, two scenarios were considered. In the first scenario, the data of all subjects were mixed together, and then a certain proportion of data was selected as training samples and the rest were employed as test samples. In the second scenario, the data reflected from two (one male and one female) of the five subjects were used as the training set, and the data reflected from the other three subjects (two males and one female) were used as the test set. That is to say, in the second scenario, the training and testing data were from different volunteers. Generally speaking, the second scenario is more realistic because it classifies the gaits of unknown individuals based on the data from known human subjects. For the first scenario, 20% of the total recordings were used in the training phase and the remaining 80% of recordings were used as test data to obtain the confusion matrices, which indicated the classification accuracy/error. This process was repeated one hundred times with a randomly chosen set of samples for training to validate the classifier performance. The accuracy/error rates of classification using different types of radars are shown in

Table 1. The first scenario: confusion matrix using only the X-band radar.

	Gait A	Gait B	Gait C
Gait A	92.46%	0.63%	6.91%
Gait B	15.41%	81.77%	2.82%
Gait C	10.30%	0.50%	89.20%

,

Table 2. The first scenario: confusion matrix using only the K-band radar.

	Gait A	Gait B	Gait C
Gait A	98.51%	0.23%	1.26%
Gait B	1.01%	98.99%	0.00%
Gait C	1.68%	0.91%	97.41%

and

Table 3. The first scenario: confusion matrix using the dual-band radar data fusion.

	Gait A	Gait B	Gait C
Gait A	99.41%	0.30%	0.42%
Gait B	1.00%	99.00%	0.00%
Gait C	1.55%	1.01%	97.44%

, respectively. For the second scenario, the data measured from individual 1 and 3 were used for training, and the data measured from individual 2, 4, and 5 were used for testing. This experiment aims to validate the proposed method in the condition of recognizing the human gaits of unknown personnel targets. The accuracy/error rates of classification using different types of radars are shown in

Table 4. The second scenario: confusion matrix using only the X-band radar.

	Gait A	Gait B	Gait C
Gait A	97.00%	0.00%	0.00%
Gait B	17.00%	82.00%	1.00%
Gait C	12.00%	0.00%	88.00%

,

Table 5. The second scenario: confusion matrix using only the K-band radar.

	Gait A	Gait B	Gait C
Gait A	99.00%	0.00%	1.00%
Gait B	1.00%	99.00%	0.00%
Gait C	0.00%	0.00%	100%

and

Table 6. The second scenario: confusion matrix using the dual-band radar data fusion.

	Gait A	Gait B	Gait C
Gait A	100%	0.00%	0.00%
Gait B	1.00%	99.00%	0.00%
Gait C	0.00%	0.00%	100%

, respectively.

The values in the tables stand for the classification rate, where columns represent the real groups, and rows represent the classified groups. Thus, the diagonal entries are the accuracy rates and others are the error rates. From the tables, it can be seen that the classification accuracy of using only the K-band radar sensor is higher than that of the X-band radar sensor. This is because the K-band radar is more sensitive to the torso motion and the swings of arms and legs. As the target is observed with the dual-band radar at the same time, the dimension of features extracted from the dual-band radar sensors is two times that obtained by only one radar sensor, which increases the classification accuracy as indicated in Table 1, Table 2, Table 3, Table 4, Table 5 and Table 6. Comparing the classification results of the two scenarios, we can find that the classification accuracy in the second scenario is slightly higher than that in the first scenario. This is because the percentage of training data in the second scenario is larger than that in the first scenario.

5. Conclusions

In this paper, we investigate how to improve the classification accuracy of personnel targets with different baggage by using dual-band radar. The human activities we are interested in include walking with no bag, walking with only one carry-on baggage held by one hand, and walking with one carry-on baggage held by one hand and one handbag held by another hand. Three features are extracted from the time-frequency spectrogram, namely the period, the Doppler offset, and the bandwidth. Furthermore, classification performances of using only X-band radar, only K-band radar and dual-band radar are compared. The experimental results on the measured data demonstrate that higher classification accuracy can be obtained by using dual-band radar compared to only using a single radar sensor. The potential application of this study is to improve the efficiency of security checks in airports.