Tracking of Multiple Static and Dynamic Targets for 4D Automotive MillimeterWave Radar Point Cloud in Urban Environments
Abstract
:1. Introduction
 This paper proposes a 4D millimeterwave radar point cloudbased multitarget tracking algorithm for estimating the ID, position, velocity, and shape information of targets in continuous time.
 The proposed target tracking solution includes point cloud velocity compensation, clustering, dynamic and static attribute update, dynamic target 3D border generation, static target contour update, and target trajectory management processes.
 To address the issue of the varying size and shape of dynamic and static targets, a binary Bayesian filtering method [24] is utilized to extract static and dynamic targets during the tracking process.
 Kalman filtering is used for dynamic targets such as vehicles, pedestrians, bicycles, and other targets, combined with the target’s track information and radial velocity information to estimate the target’s 3D border information.
 For static targets such as road edges, green belts, buildings, and other nonregular shaped targets, the rolling ball method is employed to estimate and update the shape contour boundaries of the targets.
2. Materials and Methods
2.1. Measurement Modeling
2.2. Target State Modeling
 Position state: The target’s position in threedimensional space ($xyz$).
 Motion state: Since the target’s position in the zaxis direction remains relatively stable in autonomous driving scenarios, the motion state can be simplified to the target’s velocity in the xaxis and yaxis directions on the vehicle motion plane (${v}_{x}{v}_{y}$).
 Profile shape state: This describes the shape and size of the target. For a 3D dynamic target in a road environment, it can be modeled as a 3D cube ($lwh\mathit{\theta}$) since its shape and size states do not change substantially. Its extended state contains the size and rotation direction of the target.
 Position state: The position of the target in the zaxis direction in space ($z$ position).
 Motion state: For static targets, the absolute velocity is zero, and the relative velocity can be estimated as the negative of the velocity of the ego vehicle’s motion (${v}_{xk}{v}_{yk}$).
 The profile shape state of the target: For a 3D static target in a road environment, it can be modeled as a target surrounded by an edge box, which is represented as a set of n 2D enclosing points and their heights ($h{\left\{\begin{array}{cc}{x}_{j}& {y}_{j}\end{array}\right\}}_{j=1}^{n}$).
2.3. Method
2.3.1. Point Cloud Preprocessing
2.3.2. Clustering and Data Association
 Radar Point Cloud Clustering
 Calculation of the number of data points N(p) in the neighborhood of a data point p:
 Determination of whether a data point p is a core point: If $N\left(p\right)\ge MinPts$, then p is a core point.
 Expanding the cluster: Starting from any unvisited core point, find all data points that are densityreachable from the core point, and mark them as belonging to the same cluster.
 Determination of whether a data point is densityreachable: A data point p is densityreachable from a data point q if there exists a core point c such that both c and p are in the neighborhood of q and the distance between c and p is less than ε.
 Marking noise points: Any unassigned data points are marked as noise points.
 Data Association
2.3.3. Target Status Update
 Target Dynamic Static Property Update
 Dynamic Target State Update
Algorithm 1 

Algorithm 2 

 Static Target State Update
 For any point $p$ and rolling ball radius $a$, search for all points within a distance of $2a$ from $p$ in the point cloud, denoted as the set $Q$.
 Select any point ${p}_{1}$(x, y) from $Q$ and calculate the coordinates of the center of the circle passing through $p$ and ${p}_{1}$ with a radius of alpha. There are two possible center coordinates, denoted as ${p}_{2}$ and ${p}_{3}$.
 Remove ${p}_{1}$ from the set $Q$ and calculate the distances between the remaining points and the points ${p}_{2}$ and ${p}_{3}$. If all distances are greater than $a$, the point $p$ is considered a boundary point.
 If all distances are not greater than $a$, iterate over all points in $Q$ as the new $p$ and repeat steps (2) and (3). If a point is found that satisfies the conditions in steps (2) and (3), it is considered a boundary point and the algorithm moves on to the next point. If no such point is found among the neighbors of $p$, then $p$ is considered a nonboundary point.
2.3.4. Track Management
3. Results
3.1. Experiment Setup
3.2. Results and Evaluation
4. Discussion
5. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
 Chen, Q.; Xie, Y.; Guo, S.; Bai, J.; Shu, Q. Sensing System of Environmental Perception Technologies for Driverless Vehicle: A Review of State of the Art and Challenges. Sens. Actuators A Phys. 2021, 319, 112566. [Google Scholar] [CrossRef]
 Ester BarShalom, Y.; Fortmann, T.E.; Cable, P.G. Tracking and Data Association. J. Acoust. Soc. Am. 1990, 87, 918–919. [Google Scholar] [CrossRef]
 Han, Z.; Wang, F.; Li, Z. Research on Nearest Neighbor Data Association Algorithm Based on Target “dynamic” Monitoring Model. In Proceedings of the 2020 IEEE 4th Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), Chongqing, China, 12–14 June 2020; IEEE: New York, NY, USA, 2020. [Google Scholar]
 Konstantinova, P.; Udvarev, A.; Semerdjiev, T. A Study of a Target Tracking Algorithm Using Global Nearest Neighbor Approach. In Proceedings of the 4th International Conference on Computer Systems and Technologies ELearning—CompSysTech’03, Rousse, Bulgaria, 19–20 June 2003; ACM Press: New York, NY, USA, 2003. [Google Scholar]
 Sinha, A.; Ding, Z.; Kirubarajan, T.; Farooq, M. Track Quality Based Multitarget Tracking Approach for Global NearestNeighbor Association. IEEE Trans. Aerosp. Electron. Syst. 2012, 48, 1179–1191. [Google Scholar] [CrossRef]
 Blackman, S.S. Multiple Hypothesis Tracking for Multiple Target Tracking. IEEE Aerosp. Electron. Syst. Mag. 2004, 19, 5–18. [Google Scholar] [CrossRef]
 Kim, C.; Li, F.; Ciptadi, A.; Rehg, J.M. Multiple Hypothesis Tracking Revisited. In Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, 7–13 December 2015; IEEE: New York, NY, USA, 2015. [Google Scholar]
 Rezatofighi, S.H.; Milan, A.; Zhang, Z.; Shi, Q.; Dick, A.; Reid, I. Joint Probabilistic Data Association Revisited. In Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, 7–13 December 2015; IEEE: New York, NY, USA, 2015. [Google Scholar]
 Habtemariam, B.; Tharmarasa, R.; Thayaparan, T.; Mallick, M.; Kirubarajan, T. A MultipleDetection Joint Probabilistic Data Association Filter. IEEE J. Sel. Top. Signal Process. 2013, 7, 461–471. [Google Scholar] [CrossRef]
 Ristic, B.; Beard, M.; Fantacci, C. An Overview of Particle Methods for Random Finite Set Models. Inf. Fusion 2016, 31, 110–126. [Google Scholar] [CrossRef] [Green Version]
 Vo, B.T.; Vo, B.N.; Cantoni, A. Bayesian Filtering with Random Finite Set Observations. IEEE Trans. Signal Process. 2008, 56, 1313–1326. [Google Scholar] [CrossRef] [Green Version]
 Beard, M.; Reuter, S.; Granstrom, K.; Vo, B.T.; Vo, B.N.; Scheel, A. Multiple Extended Target Tracking with Labeled Random Finite Sets. IEEE Trans. Signal Process. 2016, 64, 1638–1653. [Google Scholar] [CrossRef] [Green Version]
 Ester, M.; Kriegel, H.P.; Sander, J.; Xu, X. A DensityBased Algorithm for Discovering Clusters in Large Spatial Databases with Noise. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, Portland, Oregon, 2–4 August 1996; Volume 96, pp. 226–231. [Google Scholar]
 Ankerst, M.; Breunig, M.M.; Kriegel, H.P.; Sander, J. OPTICS: Ordering Points to Identify the Clustering Structure. ACM Sigmod Record 2008, 99. [Google Scholar]
 Campello, R.J.; Moulavi, D.; Sander, J. DensityBased Clustering Based on Hierarchical Density Estimates. In Advances in Knowledge Discovery and Data Mining, Proceedings of the 17th PacificAsia Conference, PAKDD 2013, Gold Coast, Australia, 14–17 April 2013; Proceedings, Part II 17; Springer: Berlin/Heidelberg, Germany, 2013; pp. 160–172. [Google Scholar]
 Koch, J.W. Bayesian Approach to Extended Object and Cluster Tracking Using Random Matrices. IEEE Trans. Aerosp. Electron. Syst. 2008, 44, 1042–1059. [Google Scholar] [CrossRef]
 Haag, S.; Duraisamy, B.; Govaers, F.; Fritzsche, M.; Dickmann, J.; Koch, W. Extended Object Tracking Assisted Adaptive MultiHypothesis Clustering for Radar in Autonomous Driving Domain. In Proceedings of the 2021 21st International Radar Symposium (IRS), Berlin, Germany, 21–22 June 2021; IEEE: New York, NY, USA, 2021. [Google Scholar]
 Baum, M.; Hanebeck, U.D. Extended Object Tracking with Random Hypersurface Models. IEEE Trans. Aerosp. Electron. Syst. 2014, 50, 149–159. [Google Scholar] [CrossRef] [Green Version]
 Wahlstrom, N.; Ozkan, E. Extended Target Tracking Using Gaussian Processes. IEEE Trans. Signal Process. 2015, 63, 4165–4178. [Google Scholar] [CrossRef] [Green Version]
 Knill, C.; Scheel, A.; Dietmayer, K. A Direct Scattering Model for Tracking Vehicles with HighResolution Radars. In Proceedings of the 2016 IEEE Intelligent Vehicles Symposium (IV), Gothenburg, Sweden, 19–22 June 2016; IEEE: New York, NY, USA, 2016. [Google Scholar]
 Scheel, A.; Dietmayer, K. Tracking Multiple Vehicles Using a Variational Radar Model. IEEE Trans. Intell. Transp. Syst. 2019, 20, 3721–3736. [Google Scholar] [CrossRef] [Green Version]
 Cao, X.; Lan, J.; Li, X.R.; Liu, Y. Automotive RadarBased Vehicle Tracking Using DataRegion Association. IEEE Trans. Intell. Transp. Syst. 2022, 23, 8997–9010. [Google Scholar] [CrossRef]
 Yao, G.; Wang, P.; Berntorp, K.; Mansour, H.; Boufounos, P.; Orlik, P.V. Extended Object Tracking with Automotive Radar Using BSpline Chained Ellipses Model. In Proceedings of the ICASSP 2021–2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada, 6–11 June 2021; IEEE: New York, NY, USA, 2021. [Google Scholar]
 Thrun, S. Probabilistic Robotics. Commun. ACM 2002, 45, 52–57. [Google Scholar] [CrossRef]
 Tan, B.; Ma, Z.; Zhu, X.; Li, S.; Zheng, L.; Chen, S.; Huang, L.; Bai, J. 3D Object Detection for MultiFrame 4D Automotive MillimeterWave Radar Point Cloud. IEEE Sens. J. 2022, 23, 1125–11138. [Google Scholar]
 Zhang, X.; Xu, W.; Dong, C.; Dolan, J.M. Efficient LShape Fitting for Vehicle Detection Using Laser Scanners. In Proceedings of the 2017 IEEE Intelligent Vehicles Symposium (IV), Los Angeles, CA, USA, 11–14 June 2017; IEEE: New York, NY, USA, 2017. [Google Scholar]
 Zheng, L.; Ma, Z.; Zhu, X.; Tan, B.; Li, S.; Long, K.; Sun, W.; Chen, S.; Zhang, L.; Wan, M.; et al. TJ4DRadSet: A 4D Radar Dataset for Autonomous Driving. In Proceedings of the 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC), Macau, China, 8–12 October 2022; IEEE: New York, NY, USA, 2022; pp. 493–498. [Google Scholar]
Sensors  Resolution  FOV  

Range  Azimuth  Elevation  Range  Azimuth  Elevation  
4D radar  0.86 m  <1°  <1°  400 m  113°  45° 
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. 
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Tan, B.; Ma, Z.; Zhu, X.; Li, S.; Zheng, L.; Huang, L.; Bai, J. Tracking of Multiple Static and Dynamic Targets for 4D Automotive MillimeterWave Radar Point Cloud in Urban Environments. Remote Sens. 2023, 15, 2923. https://doi.org/10.3390/rs15112923
Tan B, Ma Z, Zhu X, Li S, Zheng L, Huang L, Bai J. Tracking of Multiple Static and Dynamic Targets for 4D Automotive MillimeterWave Radar Point Cloud in Urban Environments. Remote Sensing. 2023; 15(11):2923. https://doi.org/10.3390/rs15112923
Chicago/Turabian StyleTan, Bin, Zhixiong Ma, Xichan Zhu, Sen Li, Lianqing Zheng, Libo Huang, and Jie Bai. 2023. "Tracking of Multiple Static and Dynamic Targets for 4D Automotive MillimeterWave Radar Point Cloud in Urban Environments" Remote Sensing 15, no. 11: 2923. https://doi.org/10.3390/rs15112923