A Robust Target Tracking Method for Crowded Indoor Environments Using mmWave Radar

Abstract

Millimeter-wave-based extended target tracking has attracted extensive interest recently because of its privacy, high precision, and low cost. This paper concentrated on crowded indoor situations and presents a novel method for group tracking. First, the proposed alpha-extended Kalman filter and the group association were carried out, which can constantly estimate the target expansion and the number of reflection points, consequently modifying the measurement noise and covariance estimation. Then, to initialize the actual targets, we employed a density-based spatial clustering approach that includes false target suppression. After the targets have been updated, the track re-association and estimation procedure is conducted to account for the unanticipated break of moving and near-static targets. Finally, various experiments involving fewer than 11 participants were designed to assess the robustness of the method. As a result, continuous and steady tracking results, as well as high counting accuracy were obtained.

1. Introduction

Millimeter-wave radar has a high research value in civilian applications and can be used in new industries such as autonomous driving, nursing care monitoring, and smart homes [1]. The factors to be taken into account for sensor-based target tracking systems were presented in [2]: accuracy, user privacy, coverage, user-side equipment, cost (infrastructure, installation, and maintenance costs), continuity, complexity, etc. These factors influence the market acceptability of technology and serve as a major criterion for our choice of sensors.

Traditional active sensors are made to be smaller and lighter than other types of sensors while still offering targets a selection of label formats and wearing options, including smart pendants, wristbands, belts, and other wearable gadgets [3]. These devices must often be worn repeatedly, incurring high maintenance and replacement costs. As a result, passive-sensor-based systems have won more academic favor over the past few years.

Numerous approaches for non-intrusive target detection and tracking based on various sensors already exist [4,5]. Millimeter-wave radar-based target tracking technology can locate and track targets by evaluating the reflected signal in the scene, and its high-bandwidth and high-frequency band features enable the system to achieve high precision and high coverage. Millimeter-wave devices are simple to deploy and require neither “fingerprinting” using pre-mapped signals, nor multiple Bluetooth gateways, unlike WiFi and Bluetooth technologies [6]. Moreover, home consumers are more responsive to millimeter-wave tracking systems due to their outstanding privacy level compared to vision-based tracking systems. Therefore, millimeter-wave radar is the optimal sensor choice for target tracking systems.

However, expansibility is a characteristic property of mmWave detecting and tracking. Traditional multi-target tracking (MTT) is often based on the assumption that the target is distant, i.e., a single target occupies only one resolution unit [7]. This tracking problem is referred to as point target tracking (PTT), and each target will be modeled as a single independent point, producing no more than one measurement point every frame. Due to the high range resolution of millimeter-wave radar and the proximity of the target, a target creates many reflection points, generating a series of point clouds at the measurement level. This problem is called the extended target tracking (ETT) problem.

In comparison to typical PTT problems, mmWave-based ETT issues are more challenging [8]. Target expansion can exponentially boost the number of association hypotheses, and the typical global nearest neighbor (GNN) [9], joint probability data association (JPDA) [10,11], and multi-hypothesis tracking (MHT) [12] will result in a tremendous computing burden. Numerous studies have been carried out on ETT-related topics. These studies are divided into two distinct categories based on how the objectives are modeled. The first is the PTT-like type, which transforms the point cloud into multiple measurement points using clustering techniques such as density-based clustering methods (DBSCAN) [13] and k-means clustering algorithm (k-means) [14], so that ETT is remodeled as a PTT problem and tracked using conventional frameworks. This scheme has proven useful in simple circumstances. By using an updated clustering technique, the counting accuracy can achieve 68% in the case of five people [15]. However, in dense situations, this approach performs significantly worse. When the previous positions of two objects are near together, for instance, their point clouds may be merged into a single point measure. Likewise, as the number of people increases, the quality of the target point cloud and the clustering effect will decrease dramatically. The second type is group tracking (GT), which keeps the initial sparse point cloud and tracks the point cloud, making this method more conducive to track maintenance. Typical Texas Instruments (TI) research uses a group association strategy to ensure that targets are continuously monitored [16]. Tests showed that the TI approach is 96% accurate for one person and 45% accurate for five people [17]. Nonetheless, this method may be more susceptible to noise and form false tracks when false targets repeatedly appear in comparable areas.

Moreover, research is also being conducted on target identification to aid in target tracking to address issues such as target breakage. Human gait is now categorized as a soft biometric trait [18]. Hence, it is possible to identify targets by gait. There has been some research exploring merging target tracking and target recognition, which involves applying deep learning classifiers to raw micro-Doppler spectrograms [19] or accumulation point clouds [20,21]. When tracking fails, these identification results can be fed back to the trajectory tracking module to re-establish the correct target labels. However, the employment of deep learning methods diminishes the real-time character of tracking, and it is challenging to transfer to embedded devices.

There is, to the best of our knowledge, a need for literature focused on complex indoor scenes with more than five individuals present. The proposed solution focuses on dense indoor conditions and tries to increase the precision and consistency of target tracking, which provides the following contributions:

• An alpha-extended Kalman (AEKF) filter and the corresponding group-target correlation method are proposed, which can continually estimate the target expansion and number of points, as well as modify the covariance size and gating parameters adaptively. This method is superior to the standard PTT-like Kalman filtering [22] and correlation for continuous and reliable tracking of extended targets.
• A ghost and split target suppression method appropriate for mmWave tracking is presented. During the initiation (clustering) of new subjects, this method is applied to suppress the false targets by considering the features of these two kinds of subjects.
• A method for track re-association and completion is proposed, which can handle the unexpected fragmentation of both the moving and near-static trajectories. The conventional tracking method does not utilize continuous multi-frame track information, which can accurately describe the movement characteristics of a human target. By considering the state transitions of numerous frames of short tracks, we determined the attribution of trajectories and estimated the missing target states, thereby reducing the probability of ID switches or failures in continuous tracking.

The remainder of the paper is organized as follows. Section 2 introduces the system model and problems. Section 3 gives the methods for millimeter-wave radar signal processing. The proposed multi-target tracking methods are described in detail in Section 4. Multiple experimental results are displayed to demonstrate the reliability and robustness of our strategy, which are outlined in Section 5.

2. System Model

Consider a room with numerous human targets and a ceiling-mounted millimeter-wave radar. The radar transmits a frequency modulation continuous wave (FMCW) signal throughout the room, and by continually producing signals known as chirps, the radar senses and monitors environmental information. The process is shown in Figure 1.

The transmitted signal $s_t\left(t\right)$ can be expressed as

$$s_t\left(t\right)=A\exp \left[j\left(2\pi f_{c}t+\pi S{t}^2\right)+{\phi }_0\right]\begin{array}{cc},& 0\le t\le T\end{array},$$

(1)

where $f_c$ denotes the carrier frequency, ${\phi }_0$ represents the initial phase, and A is the amplitude. The chirp frequency changes uniformly throughout a single period T with a change rate S and a bandwidth B.

After the signal is reflected, the receiver mixes the received signal $s_r\left(t\right)$ with $s_t\left(t\right)$ and uses a low-pass filter to acquire an intermediate frequency (IF) signal, which is given by

$$s_{\mathrm{IF}}\left(t\right)=\mathrm{LPF}\left\{s_t\left(t\right)s_r\left(t\right)\right\}=A_{\mathrm{IF}}\exp \left[j\left(2\pi f_{\mathrm{IF}}t+{\varphi }_{\mathrm{IF}}\right)\right],$$

(2)

where $f_{\mathrm{IF}}$, $A_{\mathrm{IF}}$, and ${\varphi }_{\mathrm{IF}}$ denote the IF, the amplitude, and the phase of the IF signal, respectively. By preprocessing the IF signal, the point cloud information can be obtained (Section 3).

These measurement points will be entered into the tracking module (Section 4), which aims to track each indoor target continuously and robustly with as few false targets as possible. The number of people tracked varies from one to ten, and in order to imitate real-life events, the states of these persons usually change. Some states and issues create considerable obstacles to the continuity and resilience of the tracking system. These complex problems are shown in Figure 2.

They are characterized as follows:

• Track split:
  Human targets perform actions such as arm swinging, which extend the measurements significantly. Some measurements from a single target are considered to belong to a new target.
• False target:
  Targets resulting from measurements that are not human targets are referred to as false targets. Measurement sources can be clutter, multi-path effects, and direct current component.
• Targets’ crossover:
  Several human targets are simultaneously approaching. This is characterized by the merging of point clouds, which results in association mistakes or the loss of targets.
• Near-static target:
  The human target hardly moves while seated or lying. It is characterized by the fact that there is less information in the target point cloud and that there is frequently no measurement of the target in multiple frames, leading to the loss of the target.

The traditional PTT method is less effective in complex multi-target scenarios because it ignores the expansion characteristics, neglects the overall track information, does not consider the multi-path effect, and is highly dependent on the clustering effect. In order to cope with the complex indoor situation in daily life, a novel tracking method is proposed in this paper to solve the above tracking problems.

3. mmWave Radar Signal Preprocessing

The IF signal described in Section 2 must be preprocessed to obtain the necessary target parameters before tracking. The chain of signal preprocessing implements the range-Doppler (RD) information acquisition, static clutter removal, measurements’ detection, signal-to-noise ratio (SNR) estimation, and elevation–azimuth (EA) information acquisition and, finally outputs the point cloud including the range, Doppler, azimuth, elevation, and SNR. The flowchart of signal preprocessing is shown in Figure 3.

3.1. RD Information Acquisition

Specifically, the IF signal can be stated as follows when the target is located at a distance of d meters from the radar:

$$s_{\mathrm{IF}}\left(t\right)=A\sin \left(2\pi f_{0}t+{\varphi }_0\right)$$

(3)

where $f_0=\frac{S2d}{c}$, ${\varphi }_0=\frac{4\pi d}{\lambda }$, c is the speed of light, and $\lambda$ represents the wavelength. It is demonstrated that objects at different distances can generate IF signals with specific frequencies, which enables the Fourier transform [23] in the fast time dimension to separate various instantaneous frequency values to obtain the range information of multiple objects and create a range map. It is worth noting that the range profile can show significant scalability with numerous isolated peaks, even in the single-target scenarios shown in Figure 4, due to the height of the human body.

Additionally, the signal phase is susceptible to minor object displacements. Considering that successive chirps separated by $T_c$ have a phase difference $\Delta \mathsf{\Phi }$, the Doppler can be calculated by

$$v=\frac{\lambda \Delta \mathsf{\Phi }}{4\pi T_c}.$$

(4)

3.2. Static Clutter Filtering

There is no interest in stationary items since we focused on locating moving targets. RD maps contain noise at zero frequency, which can severely affect the detection and tracking of moving objects. These clutter signals are created mainly by the surroundings indoors, including floors, walls, and tables and other furniture, which have velocities near zero. Therefore, a moving target indication (MTI) method [24] is used to detect and suppress static clutter from the Doppler dimension.

First, by averaging all chirps in a single frame, the reference receiving chirp can be calculated as

$$C_{re}\left(r\right)=\frac{1}{N}{\sum }_{d=1}^{N}F\left(d,r\right),$$

(5)

where $F\left(d,r\right)$ represents the r-th fast-time DFT point in the d-th slow time and N is the number of slow-time DFT points. For moving targets, the signal phase changes as the target moves; thus, the phase of each chirp is different, resulting in phase cancellation, which reduces the amplitude of the reference chirp. After subtracting the reference receiving chirp from all chirps, the amplitude of the moving target rarely changes, whereas the amplitude of the stationary target is decreased.

Compared with the commonly used two-pulse cancellation [25], the background noise of this method is cleaner and more suitable for multi-target detection scenarios with multiple peaks.

3.3. Measurements Detection and SNR Estimation

After static clutter filtering, the RD map shows numerous distinct peaks. The ordered statistics constant false alarm rate (OS-CFAR) [26] detector is a method for estimating interference level thresholds adaptively and ensuring a constant false alarm probability. The clutter properties of the current unit to be detected (CUT) can be approximated using data from adjacent units.

First, the N reference cells are sorted from smallest to biggest, and the k-th sample is selected as the background clutter power level ${\widehat{\sigma }}_w^2$. Then, cells exceeding the adaptive threshold $T=\alpha {\widehat{\sigma }}_w^2$ are referred to as detected measurement points, where $\alpha$ is a constant value. Next, the SNR for a CUT of power ${\widehat{\sigma }}_w^2$ is estimated to be $SN{R}_n={\widehat{\sigma }}_n^2/\phantom{{\widehat{\sigma }}_n^2{\widehat{\sigma }}_w^2}{\widehat{\sigma }}_w^2$. This method is more sensitive to continuous multi-peak scenarios and more appropriate for multi-object detection than the widely applied cell averaging constant false alarm rate (CA-CFAR) detector [27].

3.4. EA Information Acquisition

Conventional angle of arrival (AOA) estimate methods [25] are constrained by the limits of physical angle resolution and cannot precisely measure angles. Since DFT in the antenna dimension is far from meeting the requirements, the minimum variance distortionless response (MVDR) algorithm [24] is required to obtain the target angle, which is a super-resolution beamforming technique.

First, the experimental scene can be modeled as shown in Figure 5, where a Cartesian coordinate is formed with the radar as the origin of the coordinate, $\varphi$ indicates the elevation, whereas $\theta$ represents the azimuth.

Assume that $d_x$ and $d_y$ are horizontal and vertical equivalent array element intervals, respectively, and the phase shift of the incident wave to the array element ${\mathbf{p}}_{(n,m)}$ relative to the reference signal point ${\mathbf{p}}_{(0,0)}$ is

$$\begin{array}{c}{\psi }_{n,m}\left(\theta ,\varphi \right)=\frac{2\pi }{\lambda }\left(n{d}_x\sin \varphi \cos \theta +m{d}_y\sin \varphi \sin \theta \right)=n{\psi }_x+m{\psi }_y.\end{array}$$

(6)

Its power spectrum can be expressed as:

$$P(\theta ,\varphi )={\mathbf{w}}^{\mathbf{H}}\mathbf{Rw}=\frac{1}{{\mathbf{w}}^{\mathbf{H}}(\theta ,\varphi ){\mathbf{R}}^{-1}\mathbf{w}(\theta ,\varphi )},$$

(7)

where $\mathbf{w}=\mathbf{v}\left({\psi }_{\mathrm{y}}\right)\otimes \mathbf{v}\left({\psi }_z\right)$, $\mathbf{R}=E\left\{x\left(n\right)x^H\left(n\right)\right\}$, $\mathbf{v}\left(·\right)$ denotes the steering vector, ⊗ is the Kronecker product, and $x\left(n\right)$ represents the receiving signal.

The EA profile can be generated by analyzing all elevation and azimuth angles in the power spectral space, followed by CA-CFAR and connected domain analysis to create single or multiple connected domains. A reference point is chosen in each connected domain to serve as the angle information for the reflected point. When $N_{RD}$ detection points are obtained from the RD profile and $N_{EA}$ detection points are obtained from the EA profile in the frame k, a total of $N_P=N_{RD}\times N_{EA}$ reflection points are obtained in a single frame.

3.5. Point Cloud Generation

Because the range and angle information of each measurement point has been obtained, the position information can be obtained through three-dimensional mapping:

$$\left\{\begin{array}{c}x=R\sin \left(\varphi \right)\cos \left(\theta \right)\hfill \\ y=R\sin \left(\varphi \right)\sin \left(\theta \right)\hfill \\ z=R\cos \left(\varphi \right)\hfill \end{array}\right..$$

(8)

Assume that the i-th reflection point vector contains information about its location in a polar coordinate system and can be expressed as ${\mathbf{M}}_k^i={\left[R_k^i,D_k^i,{\theta }_k^i,{\varphi }_k^i,SN{R}_k^i\right]}^T$, and the elements within the vector are the range, Doppler, azimuth, elevation, and SNR. The final point cloud set ${\mathbf{M}}_k=\left\{{\mathbf{M}}_k^i\right\},0\le i\le N_P$ will be entered into the tracking module.

4. The Proposed Tracking Method

The signal preprocessing described in Section 3 generates a series of point cloud collections at instant k. For each frame of the constantly transmitted point cloud, our method carries out the following tracking processes depicted in Figure 6.

The proposed method performs the following process sequentially:

• Prediction:
  For the prediction of multi-dimensional information on targets, an AEKF method suited for extended targets is employed. The number of reflected points corresponding to a single human target is estimated using alpha filtering, and the extension information is characterized by the multivariate Gaussian distribution covariance. In this step, we can obtain prior information, including the position, velocity, covariance, number of points, and extension.
• Points-to-prior association:
  Given the prior expectations and multiple reflection points obtained, the number of data association assumptions is extremely high, and GNN is a practical association estimation approach. It prunes posterior density estimates with the exception of the best estimate. Since the processing is not point target tracking, the classic GNN “one-to-one” distribution strategy cannot deal with human target point clouds. Therefore, a “many-to-one” point cloud association technique is paired with gating to associate all reflection points belonging to the same human target with the relevant prior expectation. When the quality of the point cloud on the field is insufficient, this approach can nonetheless correlate and preserve the target track across numerous frames.
• Track initialization:
  A density-based spatial clustering of applications with noise (DBSCAN) with false target suppression is utilized to acquire several emerging targets throughout the tracking procedure, which is exclusively employed for unassigned point clouds. Simultaneously, the method estimates all present subjects during the first frame of the complete tracking process. In addition, we improved the initialization scheme to make it more suitable for mmWave indoor applications in terms of target split and false targets. According to the number of people, the clustering parameters can vary adaptively to fit the changing quality of the point cloud for each individual.
• Update:
  The reference centroid of the related group is determined using a weighting approach based on the SNR of various reflection points. According to the number estimation and extension estimation produced through filtering, the AEKF is used to estimate the posterior information by adjusting the measurement noise estimation adaptively to ensure updating the extended target accurately. The mean of the posterior multivariate Gaussian distribution is then used as the target state estimation. Additionally, a person counting process is performed during this phase.
• Track re-association:
  To address the issue of track break, a track re-association approach is presented for quasi-static targets and track fracture, which may be brought on by occlusion or crossover. To be specific, people can interact more frequently as the number of people grows. As the target velocity drops, the target information may be filtered out as static clutter. This may cause several fractures in the track, as well as the loss of point clouds of specific targets in subsequent multiple frames. With this method, the attribution of the new trajectory is determined by comparing how similar the old and new tracks are.
• Track management:
  Track management includes track status such as temporary, active, reserved, leaving, and released tracks. This process transfers the status of trails between multiple pre-set states through specific judgment standards, ensuring the initialization of new tracks, the update of continuous tracks, the retention of unassociated tracks, and the release of free and leaving tracks, which can be seen in Figure 7.

4.1. Alpha-Extended Kalman Filter

In order to not only filter the target position, Doppler, and covariance, but also continually evaluate the number of reflection points and extension information, we propose an AEKF in the prediction and updating process. The C-K equation, which is computationally intensive, is used in Bayesian filtering to determine the prior information [28]. To lower the computing expenses, the following assumptions were adopted: (a) prior density follows a Gaussian distribution with a mean of $\mathbf{\mu }$ and a covariance of $\mathbf{P}$; (b) the detection probability is constant; (c) measurement likelihood follows a Gaussian distribution with a mean of $\mathbf{H}\mathbf{\mu }$ and a covariance of $\mathbf{R}$; (d) clutter intensity ${\lambda }_c\left(c\right)\ge 0$. The specific calculation process of the above parameters will be given later in this subsection.

Suppose that there are $N_t$ active targets in the previous frame and the posterior estimate of the j-th target state is ${\mathbf{\mu }}_{k-1|k-1}^j={\left[x_{k-1|k-1}^j,y_{k-1|k-1}^j,{\dot{x}}_{k-1|k-1}^j,{\dot{y}}_{k-1|k-1}^j\right]}^T$, including the target position ($x_{k-1|k-1}^j$ and $y_{k-1|k-1}^j$) and the target velocity (${\dot{x}}_{k-1|k-1}^j$ and ${\dot{y}}_{k-1|k-1}^j$). The target extension is estimated to be ${\mathbf{E}}_{k-1|k-1}^j=\left[N_{k-1|k-1}^j,C_{k-1|k-1}^j\right]$, containing the number of reflection points $N_{k-1|k-1}^j$ and extension ${\mathbf{C}}_{k-1|k-1}^j$. The target posterior state covariance matrix is ${\mathbf{P}}_{k-1|k-1}^j$. In addition, we define the m-th reflection point taken from the sparse point cloud generation module in this frame to be ${\mathbf{Z}}_k^i={\left[R_k^i,D_k^i,{\theta }_k^i,{\varphi }_k^i\right]}^T$ with $SN{R}_k^i$.

For the target moving model, a constant-velocity motion (CV) model [29] was selected. However, even if the CV model is employed, the target is not required to move at a constant speed. The velocity change in one frame can be fully characterized by process noise since the acceleration of the human target is low. The target state transition matrix is defined as follows:

$$\mathbf{F}=\left[\begin{array}{cc}{I}_2& \Delta t·I_2\\ 0·I_2& I_2\end{array}\right],$$

(9)

which describes the expected state change between two adjacent frames, $\Delta t$ is the time of one frame, and $I_2$ is the identity matrix.

The prior information of the target state and covariance matrix for target j is calculated as

$${\mathbf{\mu }}_{k|k-1}^j=\mathbf{F}{\mathbf{\mu }}_{k-1|k-1}^j$$

(10)

$${\mathbf{P}}_{k|k-1}^j={\mathbf{FP}}_{k-1|k-1}^j{\mathbf{F}}^{\mathrm{T}}+\mathbf{Q},$$

(11)

where $j=1,\dots ,N_t$, and $\mathbf{Q}$ represents the covariance matrix of process noise, which follows a Gaussian distribution with zero mean. Moreover, in the prediction step, assume that the prior information of the number of points and expansion remains the same, i.e., $N_{k|k-1}^j=N_{k-1|k-1}^j$, ${\mathbf{C}}_{k|k-1}^j={\mathbf{C}}_{k-1|k-1}^j$.

The collected point cloud is in Cartesian coordinates, and the target state must be mapped from the Cartesian space to the polar coordinate space using an observation equation with a nonlinear function. The Jacobian matrix [30] is used to linearize the nonlinear function by Taylor expanding and retaining the first-order component of the expansion terms to obtain the observation matrix $\mathbf{H}$.

During the update phase, a series of reflection points associated with the target is gathered. The sequence numbers of reflection points related to the j-th prior expectation are defined by ${\mathbf{\xi }}^j=\left\{{\mathbf{c}}^j\right\}$ with a length of $N_k^j$. The association process will be elaborated upon in Section 4.2.

First, we updated the group expansion and the estimated amount of points. The extension is defined as the covariance matrix of the SNR-weighted mixture Gaussian distribution consisting of all correlation points, which can be obtained using the EM algorithm [31]:

$${\mathbf{C}}_k^j=\frac{1}{{\displaystyle \sum _{c=1}^{N_k^j}}SN{R}_k^c}\sum _{c=1}^{N_k^j}SN{R}_k^c\left({\mathbf{Z}}_k^c-{\overline{\mathbf{Z}}}_k^c\right){\left({\mathbf{Z}}_k^c-{\overline{\mathbf{Z}}}_k^c\right)}^T.$$

(12)

where ${\overline{\mathbf{Z}}}_k^c$ represents the mean value of all associated points.

The values of $\alpha$ and $\gamma$ were set as constants, and the number of the target reflection points was updated to

$$N_{k|k}^j=N_{k|k-1}^j+\gamma \left(N_k^j-N_{k|k-1}^j\right).$$

(13)

The extension is calculated as

$${\mathbf{C}}_{k|k}^j=\frac{N_{k|k-1}^j}{N_{k|k}^j}\left[{\mathbf{C}}_{k|k-1}^j+\alpha \left({\mathbf{C}}_k^j-{\mathbf{C}}_{k|k-1}^j\right)\right].$$

(14)

Meanwhile, the target posterior state is expressed as

$${\mathbf{\mu }}_{k|k}^j=\left\{\begin{array}{cc}{\mathbf{\mu }}_{k|k-1}^j+K^j\left({\tilde{\mathbf{Z}}}^j-\mathbf{H}{\mathbf{\mu }}_{k|k-1}^j\right)\hfill & \mathrm{if}{\mathbf{\xi }}^j\ne \varnothing \\ {\mathbf{\mu }}_{k|k-1}^j\hfill & \mathrm{if}{\mathbf{\xi }}^j=\varnothing \end{array}\right..$$

(15)

The target covariance is

$${\mathbf{P}}_{k|k}^j=\left\{\begin{array}{cc}{\mathbf{P}}_{k|k-1}^j-K^j{\mathbf{HP}}_{k|k-1}^j\hfill & \mathrm{if}{\mathbf{\xi }}^j\ne \varnothing \\ {\mathbf{P}}_{k|k-1}^j\hfill & \mathrm{if}{\mathbf{\xi }}^j=\varnothing \end{array}\right..$$

(16)

In the above equations, the Kalman gain is written as $K_k^j={\mathbf{P}}_{k|k-1}^j{\mathbf{H}}^T{\left({\mathbf{HP}}_{k|k-1}^j{\mathbf{H}}^T+\widehat{\mathbf{R}}\right)}^{-1}$, and ${\tilde{\mathbf{Z}}}^j$ stands for the reference point of the group, which is the SNR-weighted mean of related points.

Moreover, conventional PTT often assigns a constant value for $\widehat{\mathbf{R}}$ that is of interest. By considering the time-varying character of ETT, $\widehat{\mathbf{R}}$ is defined as an equation involving the number of expansion and reflection points:

$$\widehat{\mathbf{R}}=\frac{\mathrm{diag}\left({\mathbf{R}}_m^2\right)}{N_{k\mid k}^j}+{\mathbf{C}}_{k\mid k}^j.$$

(17)

The first part of (17) represents the observed noise covariance of the single reflection point, where ${\mathbf{R}}_m$ is the difference between the maximum and minimum values of each dimension in the associated sequence; the second part of (17) is the estimation of the expansion, whose physical significance is that, as the expansion ${\mathbf{P}}_{k|k}^j$ becomes larger, the uncertainty of the measurement increases, which leads directly to the target posterior covariance becoming large. Furthermore, a constraint is placed on the value of ${\mathbf{R}}_m$ to avoid situations where convergence is impossible. In practice, it should be less than the approximate length of the open arms of the human body and larger than the width of the human shoulders.

The complete process of the algorithm is demonstrated in Algorithm 1.

Algorithm 1: AEKF.

Input:  ${\mathbf{\mu }}_{k-1|k-1}^j$, ${\mathbf{P}}_{k-1|k-1}^j$, $N_{k-1|k-1}^j$, ${\mathbf{C}}_{k-1|k-1}^j$ Output:  ${\mathbf{\mu }}_{k|k}^j$, ${\mathbf{P}}_{k|k}^j$, $N_{k|k}^j$, ${\mathbf{C}}_{k|k}^j$   1: for each $j\in [1,N_t]$ do   2:    Compute ${\mathbf{\mu }}_{k|k-1}^j$, ${\mathbf{P}}_{k|k-1}^j$, $N_{k|k-1}^j$, ${\mathbf{C}}_{k|k-1}^j$, and ${\overline{\mathbf{Z}}}_{k|k-1}^j$   3: end for   4: Get the association result ${\mathbf{\xi }}_k^j$ from group association module   5: Get $N_I$ new tracks from the track initialization module   6: for each $j\in [1,N_t+N_I]$ do   7:    Compute $N_{k|k}^j$ and ${\mathbf{C}}_{k|k}^j$   8:    Compute $\widehat{\mathbf{R}}$ and $K_k^j$   9:    Compute ${\mathbf{\mu }}_{k|k}^j$ and ${\mathbf{P}}_{k|k}^j$ 10: end for

Compared to the standard extended Kalman method, the proposed method benefits from considering the “many-to-one” association in the case of ETT, which ensures that the posterior density covariance can accurately model the uncertainty and enhances the accuracy of tracking associations.

4.2. Points-to-Prior Association

Target association is a crucial step that substantially decreases the number of posterior mixing density components. In other words, only the major components of the Gaussian mixture density are maintained, while the minor parts are eliminated. It can significantly reduce the algorithm’s complexity while maintaining its efficacy. In PTT-like association processes, the Hungarian algorithm [32], Murty algorithm [33], and Gibbs sampling method [34] are frequently employed to determine single or many optimum allocation strategies. They are characterized by their “one-to-one” relationship, which is supplemented by a clustering technique before tracking in each frame to accomplish a priori cluster matching. This approach relies primarily on clustering, which might result in many association failures as the population grows and the quality of the point cloud deteriorates.

The non-normalized weight of the posterior mixture-density component resulting from the j-th prior expectation associated with the reflection point ${\mathbf{Z}}_k^c$ is

$${\tilde{W}}_k^c=\frac{P^{\mathrm{D}}\mathcal{N}\left({\mathbf{Z}}_k^c;{\overline{\mathbf{Z}}}_{k|k-1}^j,{\mathbf{S}}_k^j\right)}{{\lambda }_c\left({\mathbf{Z}}_k^c\right)}.$$

(18)

Under the assumption that the detection probability and clutter intensity are constants, we have

$${\tilde{W}}_k^c=\frac{\exp \left(-\frac{1}{2}{\left({\mathbf{Z}}_k^c-{\overline{\mathbf{Z}}}_{k|k-1}^j\right)}^{T}inv\left({\mathbf{S}}_k^j\right)\left({\mathbf{Z}}_k^c-{\overline{\mathbf{Z}}}_{k|k-1}^j\right)\right)}{{\left|2\pi {\mathbf{S}}_k^j\right|}^{1/2}},$$

(19)

where ${\overline{\mathbf{Z}}}_{k|k-1}^j=\mathbf{H}{\mathbf{\mu }}_{k|k-1}^j$, ${\mathbf{S}}_k^j={\mathbf{HP}}_{k|k-1}^j{\mathbf{H}}^T+\widehat{\mathbf{R}}$, and the calculation processes was described in detail in Section 4.1.

The entire general pseudocode to the algorithm is shown in Algorithm 2. After the procedure, a series of correlation sequences are obtained. In contrast to the standard PTT-like approach, we associated and matched each reflection point that satisfies the gating requirements to choose the best-matching track, so the number of reflection points associated with each track might be zero or larger than one. The gating parameter G changes in each frame based on the expected number of people on the field. Furthermore, ${\mathbf{\theta }}_k=\left\{{\theta }_k^1,\cdots ,{\theta }_k^c,\cdots ,{\theta }_k^{N_k}\right\}$ is the association result, where ${\theta }_k^c=j$ stands for that for the c-th measurement whose optimal pairing is the j-th prior; ${\theta }_k^c=0$ means that there is no matching target for this reflection point. Finally, we obtain the associated sequence for each subject:

$${\mathbf{\xi }}_k^j=\left\{\begin{array}{cc}0\hfill & \mathrm{if}{\theta }_k^c=j\mathrm{does}\mathrm{not}\mathrm{exist},c\in 1\cdots N_k\hfill \\ \left\{{\mathbf{c}}^j\right\}\hfill & \mathrm{if}{\theta }_k^c=j\mathrm{exists},c\in 1\cdots N_k\hfill \end{array}\right..$$

(20)

When it is not equal to 0, the sequence length is $N_k^j$, which represents the number of reflected points matched by the track j, and ${\mathbf{c}}^j=\left[c_1^j,c_2^j,\cdots ,c_{N_k^j}^j\right]$ is the index vector of these points.

Algorithm 2: Group association.

Input:  ${\mathbf{Z}}_k$, ${\overline{\mathbf{Z}}}_{k|k-1}$, gating parameter G Output:  ${\mathbf{\theta }}_k=\left\{{\mathbf{\theta }}_k^1,{\mathbf{\theta }}_k^2,\cdots ,{\mathbf{\theta }}_k^{N_k}\right\}$   1: for each $c\in [1,N_k]$ do   2:    for each $j\in [1,N_t]$ do   3:      Compute ${M^2}_{cj}={\left({\mathbf{Z}}_k^c-{\overline{\mathbf{Z}}}_{k|k-1}^j\right)}^{T}inv\left({\mathbf{S}}_k^j\right)\left({\mathbf{Z}}_k^c-{\overline{\mathbf{Z}}}_{k|k-1}^j\right)$   4:      if ${M^2}_{cj}<G$ then   5:         Compute $W_k^c$   6:      end if   7:    end for   8:    Find ${\theta }_k^c=\arg {\max }_j{W}_k^c$   9: end for

4.3. Track Initialization

For points that ${\theta }_k=0$, the probability of a new target entering the field of view is considered. In our method, clustering is used to determine the possible new targets, while a false target removal method is applied to distinguish unreal targets within them.

DBSCAN can divide sufficiently dense areas into clusters and form arbitrary clusters in a noisy spatial database, where clusters are defined as the greatest group of densely linked points. This has benefits over k-means of not requiring the number of clusters to be specified in advance, not initializing the cluster center, and not being sensitive to noise. Thus, in crowded indoor situations, DBSCAN is better suited for human target initialization than k-means. Traditional DBSCAN requires just the parameters $\epsilon$ and $m_{pt}$, which describe the neighborhood radius and the minimum number of points, to classify and cluster a set of points.

However, on the one hand, there are cases of target splitting in mmWave tracking situations due to target expansion or inadequate angular resolution, which is shown in Figure 8a. A single human target may generate two distinct high-density clusters in the X-Y plane. Increasing the value of $\epsilon$ can mitigate this problem, but it will result in a single group when numerous new targets approach, which is shown in Figure 8b. On the other hand, multipath effects may generate ghost targets. These fake targets are related to the actual target movement for numerous frames in succession, as shown in Figure 9, finally generating a ghost trace.

To address these two issues, we examined a vast quantity of measured data and investigated reducing the appearance of false targets from three perspectives: cluster location, cluster centroid height, and cluster SNR.

Suppose that the unassociated points can be written as ${{\mathbf{Z}}^{\prime }}_k=\left\{{{\mathbf{Z}}^{\prime }}_k^1,{{\mathbf{Z}}^{\prime }}_k^2,\cdots ,{{\mathbf{Z}}^{\prime }}_k^{N_i}\right\}$, whose length is equal to $N_i=N_k-{\displaystyle \sum _{j=1}^{N_t}}{N}_k^j$. They are entered into the new target initialization module, and DBSCAN and false target removal algorithm are executed. The methods are shown in Algorithm 3. After the algorithm has completed, the track management module initializes temporary tracks for the new targets since $\mathit{Cluster}{\mathit{s}}_{\mathit{k}}$ are considered new human targets in this frame.

Algorithm 3: DBSCAN-based false target suppression.

Input:  ${{\mathbf{Z}}^{\prime }}_k$, ${\mathbf{\mu }}_{k|k-1}$, $A_{max}$, $SN{R}_g$ Output:  $\mathit{Cluster}{\mathit{s}}_{\mathit{k}}$   1: Perform the DBSCAN algorithm on ${{\mathbf{Z}}^{\prime }}_k$, and $N_c$$\mathit{Cluster}{\mathit{s}}_{\mathit{k}}$ are obtained   2: Initialize $\mathit{Cluster}{\mathit{s}}_{\mathit{s}}=\varnothing$, $\mathit{Cluster}{\mathit{s}}_{\mathit{f}}=\varnothing$   3: for each $r\in [1,N_c]$ do   4:    for each $j\in [1,N_t]$ do   5:      Compute reference centroid ${\mathbf{Z}}_r$   6:      if $\mathrm{Height}\left({\mathbf{Z}}_r\right)\le 22/33\mathrm{Height}\left({\mathbf{\mu }}_{k|k-1}^j\right)$ then   7:         if $\mathrm{distance}\left({\mathbf{Z}}_r,{\mathbf{\mu }}_{k|k-1}^j\right)\le R_{min}$ then   8:           ${\mathit{Cluster}}_s\leftarrow {\mathit{Cluster}}_s\cup {\mathit{Cluster}}_r$   9:         end if 10:         Compute group $SN{R}_r$, azimuth difference A 11:         if $SN{R}_r<SN{R}_g$, $A<A_{max}$, and $\mathrm{Range}\left({\mathbf{Z}}_r\right)>\mathrm{Range}\left({\mathbf{\mu }}_{k|k-1}^j\right)$ then 12:           ${\mathit{Cluster}}_f\leftarrow {\mathit{Cluster}}_f\cup {\mathit{Cluster}}_r$ 13:         end if 14:      end if 15:      ${\mathit{Cluster}\mathit{s}}_{\mathit{k}}\leftarrow \left\{{\mathit{Cluster}\mathit{s}}_{\mathit{k}}\backslash \left\{{\mathit{Cluster}\mathit{s}}_{\mathit{s}}\cup {\mathit{Cluster}\mathit{s}}_{\mathit{f}}\right\}\right\}$ 16:    end for 17: end for

4.4. Track Re-Association and Missing Track Estimation

Given a track variable ${\mathbf{X}}_k=\left(\beta ,\tau ,{\mathbf{T}}_{\beta :\tau }\right)$, where $\beta$ represents the short track start time, $\tau$ denotes the short track stop time, and ${\mathbf{T}}_{\beta :\tau }$ is the posterior estimate of successive target locations, the possibility of track matching was considered. Duplicate tracks reserved by track management were not considered, as these reserved tracks will change the natural features of the trajectory. Moreover, considering the complexity of human target movement, the value of $\beta$ is not the start time of the complete global track, but rather the artificially set start time of the local short track. The analysis was restricted to the latest 40 distinct track states, from which we obtained the $\beta$ of each ${\mathbf{X}}_k$.

At the moment k, a sequence of trajectories ${\mathbf{X}}_k=\left\{{\mathbf{X}}_k^1,\dots ,{\mathbf{X}}_k^{N_t}\right\}$ is produced and entered into the re-association module. We assumed that the current target state ${\mathbf{x}}_k$ is related to the target states of the historical M frames. Then, the target state model can be represented by the linear regression relationship:

$${\mathbf{x}}_k=\sum _{i=1}^M{a}_i{\mathbf{x}}_{k-i},$$

(21)

where M is also referred to as the regression model order, which determines the state model complexity, and $a_i$ is the weight of the order i. The re-association method is based on this premise of the regression model. The procedure is displayed in Figure 10.

The motion state (static or moving) of tracks needs to be identified first. The state variation of each short track is calculated, and we labeled it as stationary if the variance was less than the threshold; otherwise, we identified it as a moving track. Next, the moving and stationary tracks will be separately re-assigned.

4.4.1. Track Re-Association for Moving Target

For tracks ${\mathbf{X}}_k^a$ and ${\mathbf{X}}_k^b$ identified as moving targets with non-overlapping time, the time, slope, and state differences ($t_k^{ab}={\beta }_b-{\tau }_a$, $s_k^{ab}=s^a-s^b$, and $x_k^{ab}={\mathbf{x}}_{\beta }^b-{\mathbf{x}}_{\tau }^a$) are calculated. If these three values exceed the thresholds, they are marked as possible homologous track pairings. After obtaining a sequence of potential homology track pairs, the completed track variable is denoted by ${\mathbf{X}}_k^{ab}=\left({\beta }_a,{\tau }_b,{\mathbf{T}}_{{\beta }_a:{\tau }_b}\right)$, and the trajectory is ${\mathbf{T}}_{{\beta }_a:{\tau }_b}=\left[\begin{array}{ccc}{\mathbf{T}}_{{\beta }_a:{\tau }_a}& {\widehat{\mathbf{T}}}_{{\tau }_a:{\beta }_b}& {\mathbf{T}}_{{\beta }_b:{\tau }_b}\end{array}\right]$, where ${\widehat{\mathbf{T}}}_{{\tau }_a:{\beta }_b}=\left({\mathbf{x}}_{\tau }^a,\cdots ,{\mathbf{x}}_{\beta }^b\right)$ stands for the estimated missing trajectories. The n-order Hankel matrix of ${\mathbf{X}}_k^{ab}$ can be obtained as follows:

$${\mathbf{H}}_n^{ab}=\left[\begin{array}{cccc}{\mathbf{x}}_{\beta }^{ab}& {\mathbf{x}}_{\beta +1}^{ab}& \cdots & {\mathbf{x}}_{\beta +n-1}^{ab}\\ {\mathbf{x}}_{\beta +1}^{ab}& {\mathbf{x}}_{\beta +2}^{ab}& \cdots & {\mathbf{x}}_{\beta +n}^{ab}\\ ⋮& ⋮& \ddots & ⋮\\ {\mathbf{x}}_{\tau -n+1}^{ab}& {\mathbf{x}}_{\tau -n}^{ab}& \cdots & {\mathbf{x}}_{\tau }^{ab}\end{array}\right],$$

(22)

where n is produced under the condition that the Hankel matrix arrangement becomes a square matrix; otherwise, the matrix can be interpolated.

As the minimized regression model can be converted to reduce the rank of the Hankel matrix [35], the lost track of the chosen track pair can be calculated by ${\widehat{\mathbf{T}}}_{{\tau }_a:{\beta }_b}=\arg \min \mathrm{rank}\left({\mathbf{H}}_n^{ab}\right)$. This issue can be translated into a semi-positive definite problem as follows:

$$\begin{array}{c}\min \mathrm{Tr}\left(X\right)+\mathrm{Tr}\left(Z\right)\hfill \\ s.t.\left[\begin{array}{cc}X& H_n^{ij}\\ H_n^{ij}& Z\end{array}\right]\ge 0\hfill \end{array},$$

(23)

where X and Z are free variables and $\mathrm{Tr}(·)$ is the trace of the matrix.

Then, for these completed tracks, define the correlation probability of the short track variables ${\mathbf{X}}_k^a$ and ${\mathbf{X}}_k^b$:

$$P_{ab}=\frac{\mathrm{rank}\left({\mathbf{H}}^a\right)+\mathrm{rank}\left({\mathbf{H}}^b\right)}{\mathrm{rank}\left({\mathbf{H}}^{ab}\right)}-1.$$

(24)

The rank of the matrix is solved using SVD decomposition, and the cost matrix $\mathbf{L}$ is derived according to the correlation probability, with the likelihood that the track is associated with itself set to infinity. Finally, we used the Hungarian algorithm [32] to calculate the following optimal allocation problem:

$$\begin{array}{ccc}\min \hfill & tr\left({\mathbf{A}}^T\mathbf{L}\right)& \\ s.t.\hfill & A^{i,j}\in \{0,1\}& i,j\in \{1,\dots ,p\}\times \{1,\dots ,2p\}\\ \hfill & {\displaystyle \sum _{j=1}^{2p}}{A}^{i,j}=1& i\in \{1,\dots ,p\}\\ \hfill & {\displaystyle \sum _{i=1}^{2p}}{A}^{i,j}\le 1& j\in \{1,\dots ,2p\}\end{array},$$

(25)

where $\mathbf{A}$ represents the allocation matrix and p is the number of short tracks participating in the allocation. Once tracks ${\mathbf{X}}_k^a$ and ${\mathbf{X}}_k^b$ are successfully re-associated, ${\mathbf{X}}_k^b$ will be deleted and ${\mathbf{X}}_k^a={\mathbf{X}}_k^{ab}$ will be set in the track management module.

4.4.2. Track Re-Association for Near-Static Target

Short tracks ${\mathbf{X}}_k^a$ and ${\mathbf{X}}_k^b$, marked as stationary targets with non-overlapping time, are identified as possible homologous pairs when their mean states and time difference are both below the thresholds. Indeed, practically stationary tracks frequently break several times. Thus, we decreased the correlation cost by the number of track re-associations $N_H$. Assume the cost of inactive target association as the Euclidean distance divided by the number of re-associations, which can be expressed as

$$l_{ab}=\frac{\mathrm{Distance}\left({\mathbf{x}}_{\beta }^b,{\mathbf{x}}_{\tau }^a\right)}{\log \left(10\times N_H^a{N}_H^b\right)}.$$

(26)

As the time of the re-associations of a stationary target grows, the target is more likely to be an actual, stationary target instead of clutter and, hence, has a lower correlation cost. The optimum allocation problem is then solved using the Hungarian method to produce the association result. In addition, because it is of little necessity to estimate the lost track of a stationary target, linear interpolation is used to calculate the estimated track ${\widehat{\mathbf{T}}}_{{\tau }_a:{\beta }_b}$ and trajectory variable ${\mathbf{X}}_k^{ab}$. After assignment, in the track management module, the track ${\mathbf{X}}_k^b$ will be deleted, and then, we set ${\mathbf{X}}_k^a={\mathbf{X}}_k^{ab}$ and $N_H^a=N_H^a+1$.

4.5. Track Management

To guarantee that the tracking method can retain a high degree of resilience in the case of false tracks, target entrance, exit, etc., the track management module separates the tracking status into five stages based on a combined evaluation of multi-frame association and re-association information: temporary, active, reserved, leaving, and released tracks.

When a new cluster is formed, the new target track is initialized as temporary until it has been successfully associated M times in the previous N frames. When an active trajectory fails to be associated continuously in the last K frames, its status is changed to be reserved to ensure that the two tracks can be re-associated after beginning a new track of the same source. Once it is determined that the track cannot be re-associated in the previous $K^{\prime }$ frames, it will be released. The state of the trajectory determines the values of K and $K^{\prime }$, which is evaluated in Section 4.4. To be specific, these two thresholds have lower values for moving targets and larger values for stationary targets. Moreover, the values are lowest for targets at the field of view bounds, which are referred to as leaving tracks.

5. Experimental Results

In this section, the effectiveness of our strategy was analyzed from many distinct perspectives. The TI IWR6843-ODS [36] was used in the experiments, which has three transmitting and four receiving channels, operating on time domain multiplexing (TDM) from 60 GHz to 64 GHz. Figure 11a depicts the radar and virtual equivalent antenna array. Figure 11b illustrates the $6\mathrm{m}\times 8\mathrm{m}$ experimental area selected, with the radar installed on a bracket approximately $3\mathrm{m}$ high in the center of the area and the radar points directly downward. Simultaneously, we placed a camera facing the trial site to obtain the real target trajectory and the identifying information. Figure 11c is a video screenshot of the seven-person free-motion-counting experiment, which shows the room setup in which we conducted the experiment.

Instead of utilizing integrated DSP to acquire point cloud information, the original radar echo is transmitted straight from the sensor to the computer. On the computer side, the modules for point cloud generation and multi-target tracking are performed to continually capture the trajectory and ID.

Table 1. Radar parameters.

Parameter	Value
Start frequency	60 GHz
Effective bandwidth	960 MHZ
FM slope	30.018 MHz/us
Pulse repetition interval	0.05 ms
No. of sampling points	64
Sample rate	2 MHZ
Duration per frame	50 ms
Number of periods per frame	128
Maximum detectable distance	10 m
Maximum measurable velocity	8.33 m/s

outlines the used radar parameters.

In the experiments, unless otherwise stated in the subsection, human behavior was not constrained, which means that the targets walked as they do daily and were allowed to approach each other. The speed of the targets was the speed at which humans usually walk.

5.1. Evaluation of Tracking Accuracy

In order to verify the accuracy, the error of the tracking results needed to be quantified. Only the single-target scenario was considered here, and the target walked in a straight line according to the pre-set trajectory, as shown in Figure 12, wherein the size of the experimental area was $5\mathrm{m}\times 5\mathrm{m}$. The distance between each adjacent start (end) point was 1 m. The blue and green lines represent the trajectory parallel to the X and Y axes, respectively, and the red line represents the trajectory located diagonally. The target walked back and forth on the path. For each route, 2 sets of data for two targets were collected, and 24 sets of data were obtained.

Assuming that the pre-set path can be expressed as $Ax+By+C=0$ and the estimated position of target at the moment k is $\left(x_k,y_k\right)$, then the position error can be written as

$$e_k=\frac{\left|A{x}_k+B{y}_k+C\right|}{\sqrt{A^2+B^2}}.$$

(27)

where A, B, and C are all constant. The tracking error for each set of data is defined as $err=\frac{1}{K}{\displaystyle \sum _{k=1}^{k=K}}{e}_k$, where K represents the total number of frames. Depending on the distance at which the target is located, the tracking error of different distances can be calculated, as demonstrated in Figure 13.

It can be seen that, as the distance between the target and the radar increased, the tracking error also increased. The reason why the distance started at 3 m was that the radar was 3 m high, and the maximum distance was constrained by the maximum angle that the radar can detect. In addition, the average tracking error for the 24 sets of data was 0.15 m.

5.2. Evaluation of False Track Removal

To evaluate the method against the target splitting problem, we developed the following experimental content: the targets walked freely in the scene while simultaneously shaking their bodies to enhance the degree of target expansion. Owing to the constraints of the DBSCAN algorithm, the conventional approach generates false targets, which are often retained for a long time due to the “points-to-target” association scheme of group tracking. In Figure 14 and Figure 15, the false target produced by extreme expansion and the influence of ID switches on continuous monitoring were successfully removed.

Furthermore, the multi-path effect may generate false alerts when the target is adjacent to a wall. Several targets were instructed to walk freely and were permitted to approach the wall to generate visible ghosts in this subsection. Figure 16a,b show the results. By conducting false target suppression after DBSCAN, the ghost tracks were suppressed, and the temporary path of the real target was initialized sooner, as the false target was no longer associated with the reflection point of the real target.

5.3. Evaluation of Expansion Estimation

Due to the application of extension estimation in AEKF, Track 2 in Figure 16b approaches the wall without interference from ghost reflection points. We selected the x- and y-direction extensions to evaluate the estimated extension. The $R_m$ value that has been alpha-filtered is presented in Figure 17. It can be seen that the degree of expansion impacted the estimate of the observed noise to achieve maximum association with all of the target reflection points. From the track results, one can see that when the expansion of one direction rose, the degree of track oscillation in that direction also increased.

5.4. Evaluation of Targets’ Crossover

In the crossover scenario, our approach displayed remarkable resilience. In the experiment setup, the number of persons present climbed from 2 to 6. For the circumstance where there were more than two persons on the field simultaneously, except for the two targets that intersected, the rest of the people moved freely in the trial area. Both intersecting targets were required to begin from the same side, approach at a predefined point, and then, go straight, make a sharp angle turn, make an obtuse angle turn, or return the same way, as demonstrated in Figure 18.

In the collected dataset, each case of crossover accounted for about a quarter of the dataset. Tracking accuracy is defined as the number of crossings correctly tracked divided by the total number of crossings.

Table 2. The tracking accuracy of the targets crossover.

Number of Targets	Number of Target Crossings	Tracking Accuracy
2	47	95.74%
3	27	96.30%
4	27	96.30%
5	27	90.00%
6	27	92.59%

illustrates the tracking accuracy, which shows that the proposed method achieved more than 90% tracking accuracy in crossover scenarios with six or fewer participants. Case 4 was the crossover case with the most tracking errors for the above four cases. The reason was that, when one of the targets turned back early, its associated reflection point may be misidentified as belonging to another target. The tracking results of the remaining three cases were similar.

During the experiments, it was discovered that the velocity and the distance between the targets at the intersection mainly influenced the tracking accuracy. The association was prone to fail when the targets were close and moved quickly through the intersection, which might cause the target to vanish or move in the wrong direction. When the target vanished, our re-association strategy could recover the missing track. However, the tracking result will be affected if the tracking direction is incorrect.

5.5. Evaluation of Track Re-Association

To demonstrate the consistency and efficacy of monitoring moving targets, we designed scenarios in which six targets moved freely inside the experimental area. When Target 5 walked to the edge of the field of view, the target positioning error was considerable and the SNR was low, causing the track to break, as shown in Figure 19. The track management module retained Track 5, and when Track 7 emerged, it was determined that they were homology track pairs and the optimal missing track was estimated.

When a target was nearly stationary in the scene, its radial velocity was very low, and the information about it was lost during static clutter filtering, leading to an undesirable point cloud effect. The persistent missing association over multiple frames can cause numerous track breaks. In this experimental scenario, ten targets were on the field, nine of which slightly wiggled in place, and an individual wandered freely over the area. In Figure 20, each circle represents a true near-stationary target position and differently colored short tracks within one circle represent multiple breaks of the trajectory. The proposed method can effectively suppress the fracture of stationary targets from the dimension of the track. Due to angle inaccuracy and target limb swing, it was evident that the near-static target still expanded significantly. In addition, the walking person shuttled between them, and due to the continuous estimation of expansion, the trajectory was not affected obviously.

5.6. Analysis of the People Counting and ID Switch Result

In this subsection, the accuracy of the target counting is evaluated. Unrestricted motion datasets were collected with less than eleven individuals. In particular, six sets of data for each distinct number of persons present were gathered, for a total of sixty sets of data. The estimated tracks from the re-association module were counted as an actual target, generating tracks in the counting results. Target counting accuracy A is defined as

$$A=\frac{N_T}{N};$$

(28)

the proportion of the estimation error that is less than or equal to 1 is defined as

$$A^{\prime }=\sum _{t=T-1}^{T+1}{N}_t/N,$$

(29)

where $N_T$ represents the number of frames for which the counting result is T, which is the correct number of people, and N is the total number of frames.

In contrast, the TI group tracking method [37] was also used to process data in addition to the proposed method. In both algorithms, the preprocessing method and the values of the same parameters were set to be precisely the same.

The results of the target counting are demonstrated in Figure 21. The value of A decreased monotonically as the targets increased and dropped significantly when the targets reached eight, which was due to the target body collisions and hesitations. Nevertheless, as shown in Figure 22b, $A^{\prime }$ remained above 95%, proving the high robustness of target counting.

As seen from Figure 22a, our tracking method tended to underestimate the number of targets because when the number of people reached more than eight, the target point cloud information became incomplete. The real target may be challenging to initialize or considered as a false target. Even if it becomes a temporary track, it may still need to be successfully transformed into an active track. In contrast, the counting accuracy of TI method is shown in Figure 22. The accuracy significantly reduced, especially when more than five people were simultaneously on the field. When the number of people reached ten, it can be seen that the counting error became more considerable.

Since the TI method is prone to false targets or breaks, “ID switch” was also added as an evaluation metric. It is defined as the maximum track ID minus the number of actual tracks, revealing the continuity and authenticity of tracked trails. For datasets of five or more people in this subsection, the average ID switch value for each dataset was calculated, as shown in the

Table 3. The ID switch result.

Number of Real Targets	ID Switch of the TI Method	ID Switch of Our Method
5	0	1
6	0.33	1.33
7	0.17	2.17
8	0.5	4.67
9	1.33	5.33
10	1.4	8.4

. The proposed method had a lower ID switch value, which means fewer false targets and fewer breaks.

6. Conclusions

In this paper, we proposed a novel group tracking method, including the signal preprocessing, AEKF, group association, new target initialization, false target suppression, track management, and track re-association methods in sequence. Several experiments were designed to verify the feasibility and effectiveness of the proposed approach. The results showed that the proposed method was significantly superior in continuity and counting accuracy to previous methods, especially in crowded indoor scenes. Specifically, it could guarantee the continuity of moving and static target trajectory, the suppression ability of false targets and splits, and the continuous estimation of target expansion. In the case of target crossing, the tracking accuracy rate can reach more than 90%. In addition, the counting accuracy rate reached more than 80% in the case of seven people or less and still achieved 58% in the case of ten people. In contrast, the TI group tracking method only reached 59% in the seven-person situation and decreased to 33% at ten. Our future work will consist of trying to use deep learning methods to achieve target identification during tracking. Furthermore, we will continue to focus on improving tracking accuracy in complex situations.