A Bayesian Surprise Approach in Designing Cognitive Radar for Autonomous Driving

Abstract

This article proposes the Bayesian surprise as the main methodology that drives the cognitive radar to estimate a target’s future state (i.e., velocity, distance) from noisy measurements and execute a decision to minimize the estimation error over time. The research aims to demonstrate whether the cognitive radar as an autonomous system can modify its internal model (i.e., waveform parameters) to gain consecutive informative measurements based on the Bayesian surprise. By assuming that the radar measurements are constructed from linear Gaussian state-space models, the paper applies Kalman filtering to perform state estimation for a simple vehicle-following scenario. According to the filter’s estimate, the sensor measures the contribution of prospective waveforms—which are available from the sensor profile library—to state estimation and selects the one that maximizes the expectation of Bayesian surprise. Numerous experiments examine the estimation performance of the proposed cognitive radar for single-target tracking in practical highway and urban driving environments. The robustness of the proposed method is compared to the state-of-the-art for various error measures. Results indicate that the Bayesian surprise outperforms its competitors with respect to the mean square relative error when one-step and multiple-step planning is considered.

1. Introduction

Despite a precipitous drop in driving during the pandemic, the Governors Highway Safety Association (GHSA) of the United States reported that 2020 had the most significant annual increase in pedestrian deaths [1]. This shocking report indicated that the fatality rate for pedestrians spiked by 21% compared to the previous year. Big technology companies have invested in making autonomous radar an integral part of safety systems to prevent such accidents [2].

Current driver assistance technologies use a combination of sensors (e.g., radar, LiDAR (light detection and ranging), camera, GPS, etc.) and software to identify certain safety risks that help the driver to avoid accidents [3,4]. Compared to video cameras and LiDAR, a radar sensor is unaffected by bad weather and light conditions, and it can also detect hidden targets behind other vehicles [5,6]. Undeniably, a well-designed radar system that further advances safety benefits is indispensable for the evolution of automotive technology.

Cognitive radar, first introduced by S. Haykin [7], is an engineering tool to build intelligent tracking sensors which will eventually make autonomous driving a reality [8]. The model was inspired by the perception–action process that takes place in the brain [9]. The cognitive radar continuously interacts with its surroundings to gather information and adapts its operating parameters to ensure accurate target tracking without human control. The sensor selects a transmit waveform that anticipates a better estimate of the target’s state (i.e., distance, velocity) based on the information provided by the received radar measurements. The challenge in designing cognitive radar arises from the uncertain environment [10]. The design objective is to achieve an estimate of the target’s state that minimizes the mean squared error and to gain informative radar measurements in the presence of such disturbances. To this end, this paper focuses on quantifying new information from noisy radar measurements to improve the state estimation process over time.

In the perception and learning literature, surprising events which violate prior expectations encourage information-seeking behaviors that affect learning and decision making [11]. Surprise is an emotion resulting from a discrepancy between an expectation and an actual observation [12]. It measures the amount of information that is associated with an unexpected event [13]. Several definitions and expressions of surprise have been proposed in previous studies [14,15,16,17,18]. The most common forms of surprise are the Shannon surprise [14], the Bayesian surprise [15], and the free energy [16]. For a biological agent, the Shannon surprise measures the unlikeliness of an outcome [14]. Meanwhile, the Bayesian surprise measures how much an agent’s expectation changes when a new observation is made [15]. In [16], the free energy principle suggests that biological agents make decisions by reducing the Shannon surprise, and adjust their (internal) models to make better predictions by minimizing the Bayesian surprise.

This research adopts surprise as the main methodology to measure the amount of new information within the received radar measurements. In particular, the paper considers the Bayesian surprise since it computes how much information a new measurement provides to estimate future states based on prior knowledge. In previous works, the Bayesian surprise has been applied to different models and applications to acquire information from data [13,19,20,21,22]. In [13,19], the Bayesian surprise measures attention and anticipates the human gaze to enhance computer vision applications. A similar attempt is followed in [20], where the Bayesian surprise detects anomalies for autonomous guided vehicles in an unsupervised fashion. In associative learning, the Bayesian surprise is also employed as an error-correction learning rule for the Rescorla–Wagner model [21]. A recent paper considers a Bayesian interpretation of surprise-based learning to perform model estimation [22]. It should be noted that selecting informative measurements using Bayesian surprise to improve state estimation can be viewed as active measurement selection for regression analysis. Similar ideas have been researched and implemented in the literature, and several previously shown results could be interpreted and re-introduced through the Bayesian surprise framework. The interested reader may wish to consult [23,24,25] for input and insight. Our theory is that the Bayesian surprise framework will help understand commonalities amongst methods, lead to interesting connections, and inspire future developments.

For a simple vehicle-following scenario, this article proposes a new design of cognitive radar that operates based on the Bayesian surprise. This research generates radar measurements from a family of linear Gaussian dynamic systems. Assuming that the parameters of the system are known, the Kalman filter [26] is applied as the optimal estimator in the mean squared error sense. Given the current estimated target’s state, the sensor plans by measuring how much information each waveform—available from a predefined set—contributes to state estimation, and selects the one that conveys the maximum information based on the Bayesian surprise. In addition, the paper investigates the estimation algorithms for one-step and multiple-step planning. This proposed design assumes that the sensor is equipped with a predefined set of measurement noise covariances, where each one corresponds to a distinct waveform. Compared to other forms of surprise [14,16], the authors of this paper anticipate that the Bayesian surprise provides sufficient information to minimize the state estimation error.

Despite significant achievements in designing cognitive radar [9,27,28,29], there remains limited literature that compares radar designs based on the choice of information measure and its associated waveform-/measurement-selection procedure. For the first time, this paper systemically analyzes the works in [16,27,30] as alternative methods for designing cognitive radar. Except in [27], where the Shannon entropy is used to quantify the information within radar measurements, the free energy principle [16] and the influence matrix [30] have not been directly addressed to solve the state estimation problem in cognitive radar. In addition, this work re-introduces these methods in the context of linear Gaussian dynamic models and shows how they are related to the proposed approach.

Numerous experiments are carried out to comprehensively evaluate and compare the estimation performance of the proposed cognitive radar with the state-of-the-art. The millimeter-wave radar sensor presumes transmitting frequency-modulated continuous wave (FMCW) signals operating in the 77 GHz frequency band [31]. The paper designs the parameters of the FMCW radar in a manner that supports single-target tracking in highway and urban environments. The paper considers various error measures to examine different aspects of estimation performance. The credibility of the proposed estimation algorithm is ranked based on a pairwise comparison scheme [32]. Simulation results determine whether the tracking performance is improved when the radar switches from one-step planning to multiple-step planning.

The rest of this paper is organized as follows. Section 2 presents the model assumptions and defines the research objective in designing cognitive radar for a simple vehicle-following scenario. Section 3 briefly reviews prior works and demonstrates our proposed method to solve the state estimation problem in cognitive radar. Section 4 evaluates the estimation performance of the proposed approach by emulating real-life driving scenarios. Results are compared to alternative designs for different error measures. Finally, Section 5 concludes the paper.

Notation

In this paper, scalar variables are represented by non-bold lowercase letters (e.g., c), the vectors are denoted by bold lowercase letters (e.g., $\mathbf{x}$), and matrices and sets of vectors are shown as uppercase bold letters, (e.g., $\mathbf{F}$). In addition, tr$\{.\}$, $|.|$, and $\left|\right|.\left|\right|$ represent the trace operator (e.g., tr$\left\{\mathbf{A}\right\}$), the determinant operator (e.g., $\left|\mathbf{A}\right|$), and the norm operator (e.g., ${\left|\right|\mathbf{x}\left|\right|}_{{\mathbf{P}}^{-1}}^2={\mathbf{x}}^T{\mathbf{P}}^{-1}\mathbf{x}$), respectively. Moreover, ${\{.\}}^T$ applies the transpose operation on matrices (e.g., ${\mathbf{F}}^T$).

2. Problem Formulation

Figure 1a illustrates a simple vehicle-following scenario, where a cognitive radar is mounted on the host vehicle, tracking the state dynamics of a target vehicle (i.e., distance, velocity, etc.). Let us consider that something unexpected occurs, and the dynamics of the target vehicle change. To avoid a collision, the cognitive radar must be able to detect these changes to adjust the dynamics of the host vehicle accordingly. The radar signal received at the host vehicle provides information about the target’s state. According to this information, the cognitive radar makes a decision and sends a waveform (or signal) that can provide a better estimate of the target’s state at future time instances.

The goal of the cognitive radar is to ensure reliable and accurate tracking of the target over time. To accomplish this objective, Figure 1b presents a simple design of a cognitive radar as an autonomous system. Inspired by the cognitive dynamic system [27], the model consists of a radar environment, a receiver, an information processor, and a transmitter. The target of interest (i.e., target vehicle) is embedded in the radar environment. It is assumed that the system is equipped with a sensor profile library containing several types of waveforms. Let us consider the stages that the cognitive radar undergoes for a single cycle. Note that the term "cycle" refers to the processes that take place at one time instant. An estimate of a target’s state is determined at the receiver by processing measurements from the radar environment. The information processor measures how much information each waveform—available from the sensor profile library—contributes to estimating the target’s state for the next cycle by planning multiple time steps. Finally, the transmitter selects the waveform that leads to informative radar measurements and provides an improved estimate of the target’s state. The sensor applies the chosen waveform to the radar environment and repeats the same cycle.

This research proposes a holistic methodology to quantify information and maintain informative radar measurements that minimize the state estimation error over time. To this end, the following introduces the assumptions made to model the cognitive radar and formulates the design objectives of this research.

2.1. Model Assumptions

The following presents the model assumptions to construct radar measurements for the simple vehicle-following scenario and addresses the sensor profile library.

2.1.1. Linear Gaussian Dynamic System

For the simple driving case depicted in Figure 1a, the radar measurements at time index k, denoted as ${\mathbf{z}}_k\in {\mathbb{R}}^m$, are obtained from a set of linear Gaussian state-space models [26], expressed as follows:

$$\begin{array}{cc}\hfill {\mathbf{x}}_{k+1}& ={\mathbf{F}}_k{\mathbf{x}}_k+{\mathbf{w}}_k\hfill \\ \hfill {\mathbf{z}}_k& ={\mathbf{H}}_k{\mathbf{x}}_k+{\mathbf{v}}_k\hfill \end{array}$$

(1)

where the evolution of the state vector, denoted as ${\mathbf{x}}_k\in {\mathbb{R}}^n$, follows a first-order Markov chain process. In this problem, the state represents the entities of motion regarding the host and target vehicle, written as

$${\mathbf{x}}_k={\left[\begin{array}{c}{v}_{x,k}^0,a_{x,k}^0,d_{x,k},v_{x,k}^1,a_{x,k}^1\end{array}\right]}^T$$

(2)

where $v_{x,k}^0$ and $a_{x,k}^0$ are the velocity and acceleration of the host vehicle; $v_{x,k}^1$ and $a_{x,k}^1$ are the velocity and acceleration of the target vehicle; $d_{x,k}$ represents the longitude distance between the two cars. In Equation (1), ${\mathbf{F}}_k\in {\mathbb{R}}^{n\times n}$ and ${\mathbf{H}}_k\in {\mathbb{R}}^{m\times n}$ are, respectively, the transition matrix and the measurement matrix. Meanwhile, the state noise ${\mathbf{w}}_k\in {\mathbb{R}}^n$ and measurement noise ${\mathbf{v}}_k\in {\mathbb{R}}^m$ are assumed additive zero-mean white Gaussian processes, where ${\mathbf{Q}}_k\in {\mathbb{R}}^{n\times n}$ and ${\mathbf{R}}_k\in {\mathbb{R}}^{m\times m}$ are the state noise covariance and the measurement noise covariance, respectively. Note that the initial state follows a Gaussian distribution, denoted as ${\mathbf{x}}_0\sim \mathcal{N}(\widehat{\mathbf{x}}\left(0\right|0),\mathbf{P}\left(0\right|0))$, and is mutually uncorrelated with the noise elements.

According to the equations of motion that presume constant acceleration, ${\mathbf{F}}_k$ and ${\mathbf{Q}}_k$ are derived as [33]

$$\begin{array}{c}\hfill {\mathbf{F}}_k=\left[\begin{array}{ccccc}1& T_s& 0& 0& 0\\ 0& 1& 0& 0& 0\\ -T_s& -T_s^2/2& 1& T_s& T_s^2/2\\ 0& 0& 0& 1& T_s\\ 0& 0& 0& 0& 1\end{array}\right],\end{array}$$

(3)

and

$$\begin{array}{c}\hfill {\mathbf{Q}}_k=\left[\begin{array}{ccccc}{T}_s^4/4& T_s^3/2& -T_s^5/12& 0& 0\\ T_s^3/2& T_s^2& -T_s^4/6& 0& 0\\ -T_s^5/12& -T_s^4/6& T_s^6/18& T_s^5/12& T_s^4/6\\ 0& 0& T_s^5/12& T_s^4/4& T_s^3/2\\ 0& 0& T_s^4/6& T_s^3/2& T_s^2\end{array}\right]{\sigma }_q^2\end{array}$$

(4)

where $T_s$ and ${\sigma }_q^2$ refer to the sample time and the state noise variance, respectively. Since the dynamics of the target vehicle is of interest, the measurement matrix is assigned as

$${\mathbf{H}}_k=\left[\begin{array}{ccccc}0& 0& 0& 1& 0\\ 0& 0& 1& 0& 0\end{array}\right]$$

(5)

where the velocity of the target vehicle, $v_{x,k}^1$, and the longitude distance, $d_{x,k}$, are the available radar measurements. The choice of the measurement noise covariance depends on the waveform that the radar sensor conveys for target tracking. FMCW is the most well-known modulation format, where linear frequency ramps with different slopes are transmitted [6]. The FMCW modulation with a Gaussian-shaped pulse is commonly used in designing autonomous radars since it exhibits excellent range and velocity resolution. Thus, for a Gaussian-shaped pulse with FMCW modulation, ${\mathbf{R}}_k$ is defined as follows [34]:

$${\mathbf{R}}_k({\lambda }_{k-1},b_{k-1})=\left[\begin{array}{cc}\frac{c^2}{{\left(2\pi f_c\right)}^2\eta }(\frac{1}{2{\lambda }_{k-1}^2}+2{\lambda }_{k-1}^2{b}_{k-1}^2)& -\frac{c^2{b}_{k-1}{\lambda }_{k-1}^2}{2\pi f_c\eta }\\ -\frac{c^2{b}_{k-1}{\lambda }_{k-1}^2}{2\pi f_c\eta }& \frac{c^2{\lambda }_{k-1}^2}{2\eta }\end{array}\right]$$

(6)

where ${\lambda }_{k-1}$, $b_{k-1}$, c, $f_c$, B, and $\eta$ are the pulse duration, the chirp rate, the speed of light, the carrier frequency, the signal bandwidth, and the received signal-to-noise ratio (SNR), respectively. As shown in Equation (6), the measurement noise covariance depends on the pulse duration and chirp rate at the $k-1$ time index. This indicates that the system’s selection of the transmitted waveform (i.e., ${\lambda }_{k-1}$ and $b_{k-1}$) at the previous time cycle influences the radar measurements (i.e., ${\mathbf{z}}_k$) at the current cycle. Both ${\lambda }_{k-1}$ and $b_{k-1}$ are the design parameters that signify the radar waveform based on the tracking application (e.g., single- or multiple-target tracking). Since the transmitter and the receiver of the radar sensor are both positioned on the host vehicle, the received SNR for the target vehicle located at distance $d=\sqrt{d_x^2+d_y^2}$ may be obtained as [34]

$$\eta ={\left(\frac{d_0}{d}\right)}^4$$

(7)

where $d_y$ is the lateral distance and $d_0$ is the distance at which 0 dB SNR is achieved.

Note that linear Gaussian dynamic systems suffice to model the motion dynamics when simple driving is assumed. However, modeling complex driving situations that consider multiple targets requires switching dynamic models that may not be necessarily expressed as in Equation (1).

2.1.2. Sensor Profile Library

For the model illustrated in Figure 1b, the cognitive radar is assumed to be equipped with a prescribed set of measurement noise covariances, referred to as the sensor profile library. According to Equation (6), the measurement noise covariance is computed based on the waveform parameters: pulse duration and chirp rate. The sensor profile library holds a large set of measurement noise covariances, denoted as $\mathcal{R}$. Since it is computationally expensive and time-consuming to go through the entire library at each time cycle to select the informative measurement (or optimum waveform), a localized set is adopted instead. As a solution, this paper considers a k-nearest neighbors (kNN) method to obtain the localized set ${\mathcal{R}}_k^{\mathcal{L}}=\{{\mathbf{R}}^{\left(1\right)},{\mathbf{R}}^{\left(2\right)},\cdots ,{\mathbf{R}}^{\left(N_{\mathcal{L}}\right)}\}\in \mathcal{R}$, which includes measurement noise covariances that are neighbors to ${\mathbf{R}}_k$. The work in [28] views this localization approach as a form of an attention mechanism, which is one of the basic principles of cognition in modeling intelligent radar sensors.

2.2. Research Objective

The main goal of the cognitive radar is to estimate the target’s state from the uncertain radar measurements while maintaining low estimation error on a cycle-by-cycle basis. Given the model assumptions, this work aims to demonstrate how the cognitive radar can manipulate the waveform signal parameters to improve the target’s state estimate for the next time instant. In this regard, we mathematically express the design objective in terms of a state estimation problem and propose three research questions on modeling the information processor and the measurement-selection mechanism.

Suppose the motion dynamics of the vehicle-following scenario are expressed by Equation (1); the state estimation problem becomes finding the estimated state, denoted as ${\widehat{\mathbf{x}}}_k\sim p\left({\mathbf{x}}_k\right|{\mathbf{Z}}_k)$, that minimizes the following objective function at each time step:

$$\underset{{\widehat{\mathbf{x}}}_k\in {\mathbb{R}}^n}{\arg \min }\mathbb{E}\left[{\tilde{x}}_k^T{\tilde{x}}_k\right]$$

(8)

where ${\tilde{x}}_k={\mathbf{x}}_k-{\widehat{\mathbf{x}}}_k$ is the error between the true state and the estimated state, and $p\left({\mathbf{x}}_k\right|{\mathbf{Z}}_k)$ is the probability density function (PDF) of the estimated target’s state. Given that the radar measurements are available up to time k, ${\mathbf{Z}}_k=\{{\mathbf{z}}_i,i\le k\}$, Equation (8) estimates the target’s state that minimizes the mean squared error. To accomplish this objective, this paper focuses on modeling the information processor and the measurement-selection technique by proposing the following research problems.

Research Problem 1.

Let us assume that the parameters of the model in Equation (1) are known. Determine the amount of new information in radar measurements that contribute to estimating the target’s state ${\widehat{\mathbf{x}}}_k$.

The first research problem captures the essence of the information processor. Computing the information of the estimated target’s state is crucial because the sensor determines the optimum waveform (or the measurement noise covariance) according to the information processor. Meanwhile, the following research problem deals with how the radar sensor can change to improve the estimate of the target’s state.

Research Problem 2.

Let us assume that the measurement noise covariance, ${\mathbf{R}}_k$, can change at any time cycle. Based on Research Problem 1, develop an optimal selection methodology to minimize estimation error and achieve informative measurements with respect to the measurement noise covariance.

This problem represents the measurement-selection procedure that executes a waveform, leading to a better estimate of the target’s state. It solves an optimization problem that depends on the choice of information measure. Finally, we contemplate a general setting, which combines Research Problem 1 and Research Problem 2 in designing the cognitive radar.

Research Problem 3.

Let us consider that a set of measurement noise covariances ${\mathcal{R}}_k^{\mathcal{L}}$ are available (i.e., ${\mathcal{R}}_k^{\mathcal{L}}=\{{\mathbf{R}}^{\left(1\right)},{\mathbf{R}}^{\left(2\right)},\cdots ,{\mathbf{R}}^{\left(N_{\mathcal{L}}\right)}\}$). Derive the algorithm for the information processor and the measurement-selection procedure by looking forward to one time-step ahead. In addition, is it possible to extend the algorithm to acquire informative measurements by planning L steps in advance?

3. Proposed Method

This section briefly reviews the state-of-the-art designs to model the information processor and its corresponding measurement-selection criteria. The paper addresses Haykin’s strategy specific to cognitive radar [27] and, for the first time, discusses alternative approaches that can apply to designing such systems [16,30]. Finally, our proposed solutions to Research Problems 1, 2, and 3 are presented.

3.1. Prior Works

Multiple measures are suggested in statistics and information theory to quantify information [14,15,16,30]. The most common is the Shannon entropy, which measures the amount of self-information or (Shannon surprise) of a particular observation, averaged over all possible outcomes [14]. In [27,28], the authors adopt the Shannon entropy to measure the information of the estimated target’s state and model the information processor as follows:

$${\mathcal{H}}_k=-{\int }_{{\mathbf{x}}_k\in {\mathbb{R}}^n}p\left({\mathbf{x}}_k\right|{\mathbf{Z}}_k)lnp\left({\mathbf{x}}_k\right|{\mathbf{Z}}_k)\mathrm{d}{\mathbf{x}}_k$$

(9)

where $p\left({\mathbf{x}}_k\right|{\mathbf{Z}}_k)$ is the posterior PDF of the estimated state. Haykin derives ${\mathcal{H}}_k$ in terms of the estimated state covariance, $\mathbf{P}\left(k\right|k)$, when the Kalman filter is applied for state estimation (i.e., $p\left({\mathbf{x}}_k\right|{\mathbf{Z}}_k)=\mathcal{N}(\widehat{\mathbf{x}}\left(k\right|k),\mathbf{P}\left(k\right|k))$). According to [27,28], the measurement-selection procedure chooses the measurement noise covariance that minimizes the Shannon entropy.

While Haykin considers an information-theoretic approach to design cognitive radar, refs. [15,16] use surprise as the principal mechanism to acquire information. Surprise measures ”how much wow” one experiences when encountering uncertain events [13]. The Bayesian surprise measures the Kullback–Leibler (KL) information between a prior probability distribution and its update when a new observation is made [15]. Based on the research objective, the Bayesian surprise is defined as

$$\begin{array}{c}\hfill {\mathcal{S}}_k^B\left({\mathbf{z}}_k\right)=D_{KL}(p\left({\mathbf{x}}_k\right|{\mathbf{Z}}_{k-1}),p\left({\mathbf{x}}_k\right|{\mathbf{Z}}_k))={\int }_{{\mathbf{x}}_k\in {\mathbb{R}}^n}p\left({\mathbf{x}}_k\right|{\mathbf{Z}}_{k-1})ln\frac{p\left({\mathbf{x}}_k\right|{\mathbf{Z}}_{k-1})}{p\left({\mathbf{x}}_k\right|{\mathbf{Z}}_k)}\mathbf{d}{\mathbf{x}}_k\end{array}$$

(10)

where it determines the effect of the new radar measurement ${\mathbf{z}}_k$ on the target’s state estimation by measuring the KL distance from the predicted PDF to the posterior PDF. In addition, free energy is another type of surprise that measures the information of a new measurement by weighting and averaging it over all possible models [16]. The free energy is determined as follows:

$$\begin{array}{c}\hfill {\mathcal{F}}_k\left({\mathbf{z}}_k\right)=-{\int }_{{\mathbf{x}}_k\in {\mathbb{R}}^n}p\left({\mathbf{x}}_k\right|{\mathbf{Z}}_{k-1})lnp\left({\mathbf{z}}_k\right|{\mathbf{x}}_k,{\mathbf{Z}}_{k-1})\mathbf{d}{\mathbf{x}}_k={\mathcal{S}}_k^B\left({\mathbf{z}}_k\right)-lnp\left({\mathbf{z}}_k\right|{\mathbf{Z}}_{k-1})\end{array}$$

(11)

where $p\left({\mathbf{z}}_k\right|{\mathbf{x}}_k,{\mathbf{Z}}_{k-1})$ is the probability of the measurements at time k, conditioned on the state and all past measurements. As shown, free energy is expressed in terms of Bayesian surprise and $-lnp\left({\mathbf{z}}_k\right|{\mathbf{Z}}_{k-1})$, which refers to the Shannon surprise within the measurements. Note that the Bayesian surprise and the free energy have been adopted in many works to explain information-seeking behaviors in biological agents [17,19,20,21,22,35,36]. However, these two have not been directly applied to the design of cognitive radar systems.

A classical estimation/control methodology to evaluate the impact of new measurements is computing the trace of the influence matrix that is used in data assimilation for weather forecasting applications [30]. The influence matrix is also suitable for measuring radar measurements’ contribution to estimating the target’s state. For a specific configuration in which the Kalman filter is used for state estimation, the influence matrix is obtained as follows:

$$\begin{array}{c}\hfill {\mathbf{S}}_k=\frac{\partial {\mathbf{H}}_k\widehat{\mathbf{x}}\left(k\right|k)}{\partial {\mathbf{z}}_k}={\mathbf{H}}_k{\mathbf{K}}_k\end{array}$$

(12)

where ${\mathbf{K}}_k$ is the Kalman gain. In [30], the authors determine that the measurement with the maximum trace contributes more information to state estimation. Thus, the measurement noise covariance that maximizes the trace of the influence matrix is selected.

3.2. Solution to Research Problem 1

This paper proposes the Bayesian surprise as the main approach to quantifying the amount of new information within the estimated state. The Bayesian surprise demonstrates how the uncertainty in measurements improves future state estimations given prior estimates and provides valuable information to reduce estimation errors over time.

Since the radar measurements are constructed from linear Gaussian state-space models (see Equation (1)), and it is assumed that the parameters of the model are known, the Kalman filter is adopted for state estimation [26]. The Kalman filter is the optimal estimator in the mean square error sense that solves the research objective given in Equation (8). The filter estimates the state mean, $\widehat{\mathbf{x}}\left(k\right|k)=\mathbb{E}\left[{\mathbf{x}}_k\right|{\mathbf{Z}}_k]$, and its covariance matrix, $\mathbf{P}\left(k\right|k)=\mathbb{E}\left[({\mathbf{x}}_k-\widehat{\mathbf{x}}\left(k\right|k)){({\mathbf{x}}_k-\widehat{\mathbf{x}}\left(k\right|k))}^T\right|{\mathbf{Z}}_k]$, in an iterative manner. Algorithm 1 presents the two-step state prediction and estimation of the Kalman filter.

Algorithm 1: Kalman filter [26].

Measurement update (Estimation): $\widehat{\mathbf{x}}\left(k\right|k)=\widehat{\mathbf{x}}\left(k\right|k-1)+{\mathbf{K}}_k({\mathbf{z}}_k-{\mathbf{H}}_k\widehat{\mathbf{x}}\left(k\right|k-1))$ $\mathbf{P}\left(k\right|k)=({\mathbf{I}}_{n\times n}-{\mathbf{K}}_k{\mathbf{H}}_k)\mathbf{P}\left(k\right|k-1)$ ${\mathbf{K}}_k=\mathbf{P}\left(k\right|k-1){\mathbf{H}}_k^T{\mathbf{P}}_{\tilde{\mathbf{z}}}{\left(k\right|k-1)}^{-1}$ Time update (Prediction): $\widehat{\mathbf{x}}(k+1|k)={\mathbf{F}}_k\widehat{\mathbf{x}}\left(k\right|k)$ $\mathbf{P}(k+1|k)={\mathbf{Q}}_k+{\mathbf{F}}_k\mathbf{P}\left(k\right|k){\mathbf{F}}_k^T$

Given that $p\left({\mathbf{x}}_k\right|{\mathbf{Z}}_{k-1})=\mathcal{N}(\widehat{\mathbf{x}}\left(k\right|k-1),\mathbf{P}\left(k\right|k-1))$ and $p\left({\mathbf{x}}_k\right|{\mathbf{Z}}_k)=\mathcal{N}(\widehat{\mathbf{x}}\left(k\right|k),\mathbf{P}\left(k\right|k))$ are available from the Kalman filter, the following expression for the Bayesian surprise is achieved [37]:

$$\begin{array}{cc}\hfill {\mathcal{S}}_k^B\left({\mathbf{z}}_k\right)& =\frac{1}{2}\left[ln\frac{\left|\mathbf{P}\right(k\left|k\right)|}{\left|\mathbf{P}\right(k|k-1)|}+\mathrm{tr}\left\{\mathbf{P}{\left(k\right|k)}^{-1}\mathbf{P}\left(k\right|k-1)\right\}-n+\left|\right|\widehat{\mathbf{x}}\left(k\right|k)-\widehat{\mathbf{x}}\left(k\right|k-1){\left|\right|}_{\mathbf{P}{\left(k\right|k)}^{-1}}^2\right]\hfill \end{array}$$

(13)

where $\widehat{\mathbf{x}}\left(k\right|k-1)$, $\mathbf{P}\left(k\right|k-1)$, and n are the predicted state mean, the predicted state covariance matrix, and the state space dimension, respectively. While the above expression demonstrates the Bayesian surprise in state space, the nature of the research problem requires rewriting Equation (13) in measurement space. In this regard, we determine the Bayesian surprise in measurement space as

$$\begin{array}{c}\hfill {\mathcal{S}}_k^B\left({\mathbf{z}}_k\right)=\frac{1}{2}\left[ln|{\mathbf{R}}_k{\mathbf{P}}_{\tilde{\mathbf{z}}}{\left(k\right|k-1)}^{-1}|-m+||\tilde{\mathbf{z}}\left(k\right|k-1){\left|\right|}_{{\mathbf{K}}_k^T\mathbf{P}{\left(k\right|k)}^{-1}{\mathbf{K}}_k}^2+\mathrm{tr}\left\{{\left({\mathbf{R}}_k{\mathbf{P}}_{\tilde{\mathbf{z}}}{\left(k\right|k-1)}^{-1}\right)}^{-1}\right\}\right]\end{array}$$

(14)

where $\tilde{\mathbf{z}}\left(k\right|k-1)={\mathbf{z}}_k-{\mathbf{H}}_k\widehat{\mathbf{x}}\left(k\right|k-1)$ is the innovation vector, ${\mathbf{P}}_{\tilde{\mathbf{z}}}\left(k\right|k-1)={\mathbf{R}}_k+{\mathbf{H}}_k\mathbf{P}\left(k\right|k-1){\mathbf{H}}_k^T$ is the innovation covariance, and m is the dimension of the measurement space. Equation (14) clearly shows the connection between the Bayesian surprise and the design parameters of the radar sensor (i.e., ${\mathbf{R}}_k$); it also indicates how the information in a current radar measurement influences the Bayesian surprise (i.e., ${\mathbf{P}}_{\tilde{\mathbf{z}}}{\left(k\right|k-1)}^{-1}$).

As shown in (14), the Bayesian surprise at time k is a function of the measurement ${\mathbf{z}}_k$. In a case where ${\mathbf{z}}_k$ is not available (e.g., multiple-step planning), this paper proposes computing the expectation of Bayesian surprise instead. The expectation of Bayesian surprise with respect to $p\left({\mathbf{z}}_k\right|{\mathbf{Z}}_{k-1})\sim \mathcal{N}({\mathbf{H}}_k\widehat{\mathbf{x}}\left(k\right|k-1),{\mathbf{P}}_{\tilde{\mathbf{z}}}\left(k\right|k-1))$ is obtained as follows:

$$\begin{array}{c}\hfill {\mathbb{E}}_{p\left({\mathbf{z}}_k\right|{\mathbf{Z}}_{k-1})}\left[{\mathcal{S}}_k^B\left({\mathbf{z}}_k\right)\right]=\frac{1}{2}ln\left|{\mathbf{R}}_k{\mathbf{P}}_{\tilde{\mathbf{z}}}{\left(k\right|k-1)}^{-1}\right|+\mathrm{tr}\left\{{\left({\mathbf{R}}_k{\mathbf{P}}_{\tilde{\mathbf{z}}}{\left(k\right|k-1)}^{-1}\right)}^{-1}\right\}-m\end{array}$$

(15)

where ${\mathbb{E}}_{p\left({\mathbf{z}}_k\right|{\mathbf{Z}}_{k-1})}\left[\right||\tilde{\mathbf{z}}{\left(k\right|k-1)\left|\right|}_{{\mathbf{K}}_k^T\mathbf{P}{\left(k\right|k)}^{-1}{\mathbf{K}}_k}^2]$ is simplified to $\mathrm{tr}\left\{{\left({\mathbf{R}}_k{\mathbf{P}}_{\tilde{\mathbf{z}}}{\left(k\right|k-1)}^{-1}\right)}^{-1}\right\}-m$. The measurement noise covariance, ${\mathbf{R}}_k$, and the information within the innovation, ${\mathbf{P}}_{\tilde{\mathbf{z}}}{\left(k\right|k-1)}^{-1}$, are the only terms that appear in Equation (15). Equation (15) shows that the uncertainty in measurements, balanced by what the filter thinks about the measurements (i.e., ${\mathbf{R}}_k{\mathbf{P}}_{\tilde{\mathbf{z}}}{\left(k\right|k-1)}^{-1}$), impacts the expectation of Bayesian surprise.

3.3. Solution to Research Problem 2

This section explores the measurement-selection scheme associated with the choice of the information processor. Since the Bayesian surprise and its expectation are proposed to solve Research Problem 1, the challenge of Research Problem 2 becomes achieving an optimum selection mechanism based on the Bayesian surprise (or its expectation). To this end, let us first refer to the definition of the influence matrix given in Equation (12). A connection exists between the expected Bayesian surprise and the influence matrix. This relation is evident when rewriting Equation (12) in the measurement space. By expressing the Kalman gain as ${\mathbf{K}}_k=\mathbf{P}\left(k\right|k){\mathbf{H}}_k^T{\mathbf{R}}_k^{-1}$ and applying the matrix inversion lemma to Equation (12), the following definition is obtained:

$$\begin{array}{c}\hfill {\mathbf{S}}_k={\mathbf{I}}_{m\times m}-{\mathbf{R}}_k{\mathbf{P}}_{\tilde{\mathbf{z}}}{\left(k\right|k-1)}^{-1}\end{array}$$

(16)

where ${\mathbf{I}}_{m\times m}$ is the identity matrix. The term ${\mathbf{R}}_k{\mathbf{P}}_{\tilde{\mathbf{z}}}{\left(k\right|k-1)}^{-1}$ appears in the expectation of Bayesian surprise as well. The influence matrix is a projection matrix (i.e., ${\mathbf{S}}_k$ is symmetric and idempotent); a positive semi-definite matrix and all its diagonal elements are bounded between 0 and 1 [30]. Since the magnitude of the diagonal values of ${\mathbf{S}}_k$ corresponds to the influence of the measurement, the trace of the influence matrix is acceptable for determining the impact of measurements [30]. An informative measurement that contributes to state estimation maximizes the trace of the influence matrix.

The influence matrix trace provides insight into how to select informative radar measurements with respect to the Bayesian surprise. According to the properties of the influence matrix, its trace does not exceed the dimension of the measurement space m. Therefore, the maximization of $\mathrm{tr}\left\{{\mathbf{S}}_k\right\}$ is equivalent to minimizing $\mathrm{tr}\left\{{\mathbf{R}}_k{\mathbf{P}}_{\tilde{\mathbf{z}}}{\left(k\right|k-1)}^{-1}\right\}$ (see Equation (16)). In this regard, the measurement-selection procedure concerning the influence matrix becomes solving the following optimization problem:

$${\mathbf{R}}_k^{min}=\underset{{\mathbf{R}}_k}{\mathrm{argmin}}\mathrm{tr}\left\{{\mathbf{R}}_k{\mathbf{P}}_{\tilde{\mathbf{z}}}{\left(k\right|k-1)}^{-1}\right\}$$

(17)

where ${\mathbf{R}}_k^{min}$ is obtained when $\mathrm{tr}\left\{{\mathbf{R}}_k{\mathbf{P}}_{\tilde{\mathbf{z}}}{\left(k\right|k-1)}^{-1}\right\}$ is minimized. To do so, the trace is differentiated with respect to the measurement noise covariance and is set equal to zero:

$$\begin{array}{c}\hfill \frac{\partial }{\partial {\mathbf{R}}_k}\left(\mathrm{tr}\left\{{\mathbf{R}}_k{\mathbf{P}}_{\tilde{\mathbf{z}}}{\left(k\right|k-1)}^{-1}\right\}\right)={\mathbf{P}}_{\tilde{\mathbf{z}}}{\left(k\right|k-1)}^{-1}={\mathbf{0}}_{m\times m}\end{array}$$

(18)

The following is obtained by substituting the expression for ${\mathbf{P}}_{\tilde{\mathbf{z}}}\left(k\right|k-1)$ and employing the matrix inversion lemma:

$$\begin{array}{c}\hfill {\mathbf{R}}_k^{-1}({\mathbf{I}}_{m\times m}-{\mathbf{H}}_k\mathbf{P}\left(k\right|k){\mathbf{H}}_k^T{\mathbf{R}}_k^{-1})={\mathbf{0}}_{m\times m}\end{array}$$

(19)

where ${\mathbf{R}}_k^{-1}={\mathbf{0}}_{m\times m}$ is not applicable; hence, ${\mathbf{R}}_k^{min}={\mathbf{H}}_k\mathbf{P}\left(k\right|k){\mathbf{H}}_k$. For ${\mathbf{R}}_k={\mathbf{H}}_k\mathbf{P}\left(k\right|k){\mathbf{H}}_k$, the trace of the influence matrix reaches its maximum.

To demonstrate how the selection criteria for the expectation of Bayesian surprise changes when ${\mathbf{R}}_k={\mathbf{H}}_k\mathbf{P}\left(k\right|k){\mathbf{H}}_k$, it is suitable to revise $\mathbb{E}\left[{\mathcal{S}}_k^B\left({\mathbf{z}}_k\right)\right]$ in terms of the trace operator:

$$\begin{array}{c}\hfill {\mathbb{E}}_{p\left({\mathbf{z}}_k\right|{\mathbf{Z}}_{k-1})}\left[{\mathcal{S}}_k^B\left({\mathbf{z}}_k\right)\right]=\frac{1}{2}\mathrm{tr}\{ln\left({\mathbf{R}}_k{\mathbf{P}}_{\tilde{\mathbf{z}}}{\left(k\right|k-1)}^{-1}\right)\}+\mathrm{tr}\{{\left({\mathbf{R}}_k{\mathbf{P}}_{\tilde{\mathbf{z}}}{\left(k\right|k-1)}^{-1}\right)}^{-1}-{\mathbf{I}}_{m\times m}\}\end{array}$$

(20)

where for any positive semi-definite square matrix $\mathbf{A}$ (i.e., ${\mathbf{R}}_k{\mathbf{P}}_{\tilde{\mathbf{z}}}{\left(k\right|k-1)}^{-1}$), $ln\left|\mathbf{A}\right|=\mathrm{tr}\{ln(\mathbf{A}\left)\right\}$. According to the bounds of the natural logarithm, if $\mathbf{A}<<{\mathbf{I}}_{m\times m}$, then $\mathrm{abs}(ln\mathbf{A})<({\mathbf{A}}^{-1}-{\mathbf{I}}_{m\times m}$). In other words, the growth rate of $({\mathbf{A}}^{-1}-{\mathbf{I}}_{m\times m})$ is higher than $ln\mathbf{A}$ when $\mathbf{A}\to {\mathbf{0}}_{m\times m}$. Applying this condition to Equation (20) for ${\mathbf{R}}_k{\mathbf{P}}_{\tilde{\mathbf{z}}}{\left(k\right|k-1)}^{-1}\to {\mathbf{0}}_{m\times m}$ makes it safe to say that ${\mathbb{E}}_{p\left({\mathbf{z}}_k\right|{\mathbf{Z}}_{k-1})}\left[{\mathcal{S}}_k^B\left({\mathbf{z}}_k\right)\right]\approx \mathrm{tr}\{{\left({\mathbf{R}}_k{\mathbf{P}}_{\tilde{\mathbf{z}}}{\left(k\right|k-1)}^{-1}\right)}^{-1}-{\mathbf{I}}_{m\times m}\}$. Therefore, when ${\mathbf{R}}_k{\mathbf{P}}_{\tilde{\mathbf{z}}}{\left(k\right|k-1)}^{-1}\to {\mathbf{0}}_{m\times m}$ (or ${\mathbf{R}}_k={\mathbf{H}}_k\mathbf{P}\left(k\right|k){\mathbf{H}}_k$), the expectation of Bayesian surprise reaches its maximum value. In an informative radar measurement that decreases the state estimation error, the measurement noise covariance is small, and the inverse of the innovation covariance is maximized.

To further elaborate, we consider a simple example by setting ${\mathbf{R}}_k{\mathbf{P}}_{\tilde{\mathbf{z}}}{\left(k\right|k-1)}^{-1}=\alpha {\mathbf{I}}_{m\times m}$, where $0\le \alpha \le 1$. We examine two extreme cases of $\alpha =0.01$ and $\alpha =0.99$. For $\alpha =0.01$, the expectation of Bayesian surprise becomes

$$\begin{array}{c}\hfill \mathbb{E}\left[{\mathcal{S}}_k^B\left({\mathbf{z}}_k\right)\right]{|}_{\alpha =0.01}=\frac{1}{2}ln\left(\right|0.01{\mathbf{I}}_{m\times m}\left|\right)+\mathrm{tr}\left\{100{\mathbf{I}}_{m\times m}\right\}-m\approx 97m,\end{array}$$

(21)

and for $\alpha =0.99$, $\mathbb{E}\left[{\mathcal{S}}_k^B\left({\mathbf{z}}_k\right)\right]$ is determined as

$$\begin{array}{c}\hfill \mathbb{E}\left[{\mathcal{S}}_k^B\left({\mathbf{z}}_k\right)\right]{|}_{\alpha =0.99}=\frac{1}{2}ln\left(\right|0.99{\mathbf{I}}_{m\times m}\left|\right)+\mathrm{tr}\left\{{\left(0.99\right)}^{-1}{\mathbf{I}}_{m\times m}\right\}-m\approx 0.005m.\end{array}$$

(22)

Evidently, as $\alpha \to 0$, the expectation of Bayesian surprise reaches a higher value than the case where $\alpha \to 1$. Moreover, the final two terms in Equation (21) (i.e., $\mathrm{tr}\left\{100{\mathbf{I}}_{m\times m}\right\}-m=99$m ) are dominant as $\alpha$ approaches zero. Hence, when ${\mathbf{R}}_k{\mathbf{P}}_{\tilde{\mathbf{z}}}{\left(k\right|k-1)}^{-1}\to {\mathbf{0}}_{m\times m}$, the expectation of Bayesian surprise is maximized.

3.4. Solution to Research Problem 3

The solution to the final research problem aligns with the discussions carried out in the last two sections. As assumed, the sensor profile library, depicted in Figure 1b, withholds a set of measurement noise covariances, defined as ${\mathcal{R}}_k^{\mathcal{L}}=\{{\mathbf{R}}^{\left(1\right)},{\mathbf{R}}^{\left(2\right)},\cdots ,{\mathbf{R}}^{\left(N_{\mathcal{L}}\right)}\}$. At time cycle k, the cognitive radar estimates the target’s state from the radar measurements. Through multiple stages of prediction and estimation (i.e., planning), the information processor measures the contribution of each measurement noise covariance to estimating the target’s future state based on the expectation of Bayesian surprise. Eventually, the radar selects the measurement noise covariance with the maximum expectation of Bayesian surprise.

Given that the estimated state covariance, $\mathbf{P}\left(k\right|k)$, is accessible from the state estimation process, the proposed algorithm for one-step planning is summarized in Algorithm 2. Algorithm 2 presents the step-by-step procedure for obtaining the expected Bayesian surprise values corresponding to the i-th measurement noise covariance, $i=1,\cdots ,N_{\mathcal{L}}$. To this end, the one-step measurement-selection mechanism based on the expectation of Bayesian surprise is demonstrated as follows:

$$i^{☆}=\underset{i=\{1,\cdots ,N_{\mathcal{L}}\}}{\mathrm{argmax}}\mathbb{E}\left[{\mathcal{S}}_{k+1}^{B^{\left(i\right)}}\right]$$

(23)

where ${\mathbf{R}}_{k+1}={\mathbf{R}}^{\left(i^{☆}\right)}\in {\mathcal{R}}_k^{\mathcal{L}}$ leads to a better estimate of the target’s state for time $k+1$. The waveform associated with ${\mathbf{R}}_{k+1}$ is applied to the radar environment and sets a repeat of the cycle.

Algorithm 2: Cognitive radar for one-step planning.

1: $\mathbf{P}\left(k\right|k)$ and ${\mathcal{R}}_k^{\mathcal{L}}=\{{\mathbf{R}}^{\left(1\right)},{\mathbf{R}}^{\left(2\right)},\cdots ,{\mathbf{R}}^{\left(N_{\mathcal{L}}\right)}\}$ are available at time k 2: $\mathbf{P}(k+1|k)={\mathbf{Q}}_k+{\mathbf{F}}_k\mathbf{P}\left(k\right|k){\mathbf{F}}_k^T$ 3: for $i=1,\cdots ,N_{\mathcal{L}}$ 4: ${\mathbf{P}}_{\tilde{\mathbf{z}}}^{\left(i\right)}(k+1|k)={\mathbf{R}}_{k+1}^{\left(i\right)}+{\mathbf{H}}_{k+1}\mathbf{P}(k+1|k){\mathbf{H}}_{k+1}^T$ 5: Compute $\mathbb{E}\left[{\mathcal{S}}_{k+1}^{B^{\left(i\right)}}\right]$ from (15) 6: end for 7: $i^{☆}=\underset{i}{\mathrm{argmax}}\mathbb{E}\left[{\mathcal{S}}_{k+1}^{B^{\left(i\right)}}\right]$ for $i=1,\cdots ,N_{\mathcal{L}}$ 8: ${\mathbf{R}}_{k+1}={\mathbf{R}}^{\left(i^{☆}\right)}\in {\mathcal{R}}_k^{\mathcal{L}}$

Table 1. Information processor and measurement-selection designs for one-step planning.

Method	Information Processor	Measurement Selection
Haykin’s Approach	${\mathcal{H}}_{k+1}^{\left(i\right)}=\frac{1}{2}\left[ln|{\mathbf{P}}^{\left(i\right)}{(k+1|k+1)|+ln\left(2\pi e\right)}^n\right]$	$i^{☆}=\underset{i=\{1,\cdots ,N_{\mathcal{L}}\}}{\mathrm{argmin}}{\mathcal{H}}_{k+1}^{\left(i\right)}$
Expectation of Bayesian Surprise	$\mathbf{E}\left[{\mathcal{S}}_{k+1}^{B^{\left(i\right)}}\right]=\frac{1}{2}ln\left|{\mathbf{R}}_{k+1}^{\left(i\right)}{\mathbf{P}}_{\tilde{\mathbf{z}}}^{\left(i\right)}{(k+1|k)}^{-1}\right|+\mathrm{tr}\left\{{\left({\mathbf{R}}_{k+1}^{\left(i\right)}{\mathbf{P}}_{\tilde{\mathbf{z}}}^{\left(i\right)}{(k+1|k)}^{-1}\right)}^{-1}\right\}-m$	$i^{☆}=\underset{i=\{1,\cdots ,N_{\mathcal{L}}\}}{\mathrm{argmax}}\mathbf{E}\left[{\mathcal{S}}_{k+1}^{B^{\left(i\right)}}\right]$
Expectation of Free Energy	$\mathbf{E}\left[{\mathcal{F}}_{k+1}^{\left(i\right)}\right]=\mathbf{E}\left[{\mathcal{S}}_{k+1}^{B^{\left(i\right)}}\right]+\frac{1}{2}[ln|{\mathbf{P}}_{\tilde{\mathbf{z}}}^{\left(i\right)}(k+1|k)|+ln{\left(2\pi e\right)}^m]$	$i^{☆}=\underset{i=\{1,\cdots ,N_{\mathcal{L}}\}}{\mathrm{argmax}}\mathbf{E}\left[{\mathcal{F}}_{k+1}^{\left(i\right)}\right]$
Trace of Influence Matrix	$\mathrm{tr}\left\{{\mathbf{S}}_{k+1}^{\left(i\right)}\right\}=m-\mathrm{tr}\left\{{\mathbf{R}}_{k+1}^{\left(i\right)}{\mathbf{P}}_{\tilde{\mathbf{z}}}^{\left(i\right)}{(k+1|k)}^{-1}\right\}$	$i^{☆}=\underset{i=\{1,\cdots ,N_{\mathcal{L}}\}}{\mathrm{argmax}}\mathrm{tr}\left\{{\mathbf{S}}_{k+1}^{\left(i\right)}\right\}$

summarizes alternative models of the information processor and its corresponding measurement-selection procedure for one-step planning. The expressions are derived when the Kalman filter is presumed for state estimation. It is straightforward to follow Haykin’s design [27,28] and the influence matrix approach [30]. However, a detailed description of solving the research problems with respect to free energy is carried out in Appendix A. The authors make a similar case for using the expectation of free energy to model the information processor instead of the free energy itself. In addition, the measurement noise covariance that maximizes the expectation of free energy minimizes the state estimation error for the upcoming cycle. Haykin’s method requires an additional step to calculate $\mathbf{P}(k+1|k+1)$, while the other three share the same term and exclude this extra step. Although Haykin’s design implies the connection to the measurement noise covariance, his approach is aligned with the research objective. This is because it reduces the estimation error of the target’s state by minimizing the estimated state covariance, $\mathbf{P}(k+1|k+1)$.

This paper also presents the means that extend the one-step planning algorithm to L steps. Since $\mathbf{P}\left(k\right|k)$ and the entire sensor profile library are available, the Kalman algorithm is partially applicable to predict the state covariance and to compute the innovation covariance and the estimated state covariance. The only difference is that the Kalman algorithm repeats L times to capture the influence of L future measurements. In this regard, the expectation of Bayesian surprise at time $k+L$ is calculated as follows:

$$\begin{array}{c}\hfill \mathbb{E}\left[{\mathcal{S}}_{k+L}^{B^{(ij\cdots l)}}\right]=\frac{1}{2}ln\left|{\mathbf{R}}_{k+L}^{\left(l\right)}{\mathbf{P}}_{\tilde{\mathbf{z}}}^{(ij\cdots l)}{(k+L|k+L-1)}^{-1}\right|+\mathrm{tr}\left\{{\left({\mathbf{R}}_{k+L}^{\left(l\right)}{\mathbf{P}}_{\tilde{\mathbf{z}}}^{(ij\cdots l)}{(k+L|k+L-1)}^{-1}\right)}^{-1}\right\}-m\end{array}$$

(24)

where $(ij\cdots l)$ represents the L-length sequence of measurement noise covariances. Note that the expectation is computed with respect to $p\left({\mathbf{z}}_{k+L}\right|{\mathbf{Z}}_{k+L-1})$, with mean and covariance, ${\mathbf{H}}_{k+L}{\widehat{\mathbf{x}}}^{(ij\cdots l)}(k+L|k+L-1)$ and ${\mathbf{P}}_{\tilde{\mathbf{z}}}^{(ij\cdots l)}(k+L|k+L-1)$, respectively. Consequently, the sequence with the maximum expectation of Bayesian surprise leads to informative measurements:

$$(i^{☆}{j}^{☆}\cdots l^{☆})=\underset{i,j,\cdots ,l=\{1,\cdots ,N_{\mathcal{L}}\}}{\mathrm{argmax}}\mathbb{E}\left[{\mathcal{S}}_{k+L}^{B^{(ij\cdots l)}}\right]$$

(25)

where ${\mathbf{R}}_{k+1}={\mathbf{R}}^{\left(i^{☆}\right)}\in {\mathcal{R}}_k^{\mathcal{L}}$. Algorithm 3 illustrates the proposed cognitive radar algorithm for L-step planning. Note that a similar process applies to the other models for L-step planning with some minor modifications.

Algorithm 3: Cognitive radar for L-step planning.

1: $\mathbf{P}\left(k\right|k)$ and ${\mathcal{R}}_k^{\mathcal{L}}=\{{\mathbf{R}}^{\left(1\right)},{\mathbf{R}}^{\left(2\right)},\cdots ,{\mathbf{R}}^{\left(N_{\mathcal{L}}\right)}\}$ are available at time k 2: $\mathbf{P}(k+1|k)={\mathbf{Q}}_k+{\mathbf{F}}_k\mathbf{P}\left(k\right|k){\mathbf{F}}_k^T$ 3: for $i=1,\cdots ,N_{\mathcal{L}}$ 4: ${\mathbf{P}}_{\tilde{\mathbf{z}}}^{\left(i\right)}(k+1|k)={\mathbf{R}}_{k+1}^{\left(i\right)}+{\mathbf{H}}_{k+1}\mathbf{P}(k+1|k){\mathbf{H}}_{k+1}^T$ 5: ${\mathbf{P}}^{\left(i\right)}(k+1|k+1)=\mathbf{P}(k+1|k)-\mathbf{P}(k+1|k){\mathbf{H}}_{k+1}^T{\mathbf{P}}_{\tilde{\mathbf{z}}}^{\left(i\right)}{(k+1|k)}^{-1}{\mathbf{H}}_{k+1}\mathbf{P}(k+1|k)$ 6: for $j=1,\cdots ,N_{\mathcal{L}}$ 7: ${\mathbf{P}}^{\left(i\right)}(k+2|k+1)={\mathbf{F}}_{k+1}{\mathbf{P}}^{\left(i\right)}(k+1|k+1){\mathbf{F}}_{k+1}^T+{\mathbf{Q}}_{k+1}$ 8: ${\mathbf{P}}_{\tilde{\mathbf{z}}}^{\left(ij\right)}(k+2|k+1)={\mathbf{R}}_{k+2}^{\left(j\right)}+{\mathbf{H}}_{k+2}{\mathbf{P}}^{\left(i\right)}(k+2|k+1){\mathbf{H}}_{k+2}^T$ 9: ${\mathbf{P}}^{\left(ij\right)}(k+2|k+2)={\mathbf{P}}^{\left(i\right)}(k+2|k+1)[{\mathbf{I}}_{n\times n}-{\mathbf{H}}_{k+2}^T{\mathbf{P}}_{\tilde{\mathbf{z}}}^{\left(ij\right)}{(k+2|k+1)}^{-1}{\mathbf{H}}_{k+2}{\mathbf{P}}^{\left(i\right)}(k+2|k+1)]$ 10: $\cdots$ 11:  for $l=1,\cdots ,N_{\mathcal{L}}$ 12:  ${\mathbf{P}}^{(ij\cdots r)}(k+L|k+L-1)={\mathbf{F}}_{k+L-1}{\mathbf{P}}^{(ij\cdots r)}(k+L|k+L-1){\mathbf{F}}_{k+L-1}^T+{\mathbf{Q}}_{k+L-1}$ 13:  ${\mathbf{P}}_{\tilde{\mathbf{z}}}^{(ij\cdots rl)}(k+L|k+L-1)={\mathbf{R}}_{k+L}^{\left(l\right)}+{\mathbf{H}}_{k+L}{\mathbf{P}}^{(ij\cdots r)}(k+L|k+L-1){\mathbf{H}}_{k+L}^T$ 14:  Compute $\mathbb{E}\left[{\mathcal{S}}_{k+L}^{B^{(ij\cdots rl)}}\right]$ from (24) 15: end for 16: $\cdots$ 17: end for 18: end for 19: $(i^{☆}{j}^{☆}\cdots r^{☆}{l}^{☆})=\underset{i}{\mathrm{argmax}}\mathbb{E}\left[{\mathcal{S}}_{k+L}^{B^{(ij\cdots rl)}}\right]$ for $i,j,\cdots ,r,l=1,\cdots ,N_{\mathcal{L}}$ 20: ${\mathbf{R}}_{k+1}={\mathbf{R}}^{\left(i^{☆}\right)}\in {\mathcal{R}}_k^{\mathcal{L}}$

4. Numerical Results

In this section, simulation results are presented to compare the state estimation performance of the proposed cognitive radar with the state-of-the-art listed in Table 1. The following demonstrates the experimental setup and parameter settings for generating radar measurements that emulate the simple vehicle-following scenario in Figure 1a. The paper suggests a radar configuration suitable for single-target tracking in highway and urban driving environments. Several error metrics are introduced to examine various aspects of the estimation performance. This section compares the system performance of the proposed one-step planning algorithm to its alternative competitors for different state estimation errors through a series of experiments. In addition, this section also analyzes the impact of multiple-step planning in improving state estimation performance. Results are verified over numerous Monte Carlo runs.

4.1. Simulation Setup and Data Generation

The purpose of the experiment is to evaluate the estimation performance of the proposed cognitive radar. Since the paper adopts the Kalman filter to accomplish state estimation, the model parameters in Equation (1) (i.e., ${\mathbf{F}}_k$, ${\mathbf{Q}}_k$, ${\mathbf{H}}_k$, ${\mathbf{R}}_k$, $\widehat{\mathbf{x}}\left(0\right|0)$ and $\mathbf{P}\left(0\right|0)$) are assumed available. In this regard, the radar sensor configuration and the parameter setting for generating radar measurements are presented.

For the vehicle-following scenario depicted in Figure 1a, the simulation assumes that the two cars are moving forward in the same lane (i.e., $d_y=0$). In this simulation, the FMCW radar sensor is mounted on the host vehicle and operates in the 77 GHz frequency band for short- and long-range applications [31]. The bandwidth of the transmitted radar signal is set to $B=100$ MHz, and 0 dB SNR is achieved at $d_0=2000$ m. According to Equation (6), the measurement noise covariance depends on the choice of pulse duration and the chirp rate, ${\mathbf{R}}_k({\lambda }_{k-1},b_{k-1})$. By assuming that the radar sensor maintains a maximum range of $d_{max}=100$ m and a maximum velocity of $v_{max}=100$ m/s, the sensor profile library consists of measurement noise covariances specified for the following values:

$$\begin{array}{cc}\hfill & {\lambda }_{k-1}\in [1\times {10}^{-6}:{10}^{-7}:10\times {10}^{-6}]\cup [1.1\times {10}^{-5}:{10}^{-6}:10\times {10}^{-5}],\hfill \\ & b_{k-1}\in [-1\times {10}^{12}:0.2\times {10}^{12}:-0.2\times {10}^{12}]\cup [0.2\times {10}^{12}:0.2\times {10}^{12}:1\times {10}^{12}].\hfill \end{array}$$

where ${\lambda }_{k-1}$ and $b_{k-1}$ are configured to simulate a practical radar sensor for single-target tracking applications [6]. The sensor profile library is composed of $N=1810$ measurement noise covariances, denoted as $\mathcal{R}={\left\{{\mathbf{R}}^{\left(i\right)}({\lambda }_{k-1},b_{k-1})\right\}}_{i=1}^N$. Since N is a large number, and going through the entire library at each time instant is cost-ineffective, this paper adopts the kNN method to obtain a smaller set with $N_{\mathcal{L}}=25$ members. Figure 2 illustrates an example of a localized set of measurement noise covariance, ${\mathcal{R}}_k^{\mathcal{L}}$, that is distinguished by pulse duration and chirp rate.

This article demonstrates highway and urban driving to examine the state estimation performance of the proposed cognitive radar for a realistic vehicle-following scenario. Since the true initial state, ${\mathbf{x}}_0$, and its estimation elements (i.e., $\widehat{\mathbf{x}}\left(0\right|0)$, $\mathbf{P}\left(0\right|0)$) depend on the driving environment, without loss of generality, the true initial state for highway driving is set to

$${\mathbf{x}}_0={[25\mathrm{m}/\mathrm{s},3{\mathrm{m}/\mathrm{s}}^2,100\mathrm{m},23\mathrm{m}/\mathrm{s},2{\mathrm{m}/\mathrm{s}}^2]}^T,$$

while the initial estimation of the state mean and its covariance matrix are assumed as

$$\widehat{\mathbf{x}}\left(0\right|0)={[24\mathrm{m}/\mathrm{s},3{\mathrm{m}/\mathrm{s}}^2,80\mathrm{m},23\mathrm{m}/\mathrm{s},2{\mathrm{m}/\mathrm{s}}^2]}^T,$$

$$\mathbf{P}\left(0\right|0)=\mathrm{diag}([100,1,100,100,1]).$$

In the meantime, for the urban driving scenario ${\mathbf{x}}_0$, $\widehat{\mathbf{x}}\left(0\right|0)$, and $\mathbf{P}\left(0\right|0)$ are set to

$$\begin{array}{cc}\hfill & {\mathbf{x}}_0={[13\mathrm{m}/\mathrm{s},1{\mathrm{m}/\mathrm{s}}^2,30\mathrm{m},12\mathrm{m}/\mathrm{s},1{\mathrm{m}/\mathrm{s}}^2]}^T,\hfill \\ \hfill & \widehat{\mathbf{x}}\left(0\right|0)={[12.5\mathrm{m}/\mathrm{s},1{\mathrm{m}/\mathrm{s}}^2,28\mathrm{m},12\mathrm{m}/\mathrm{s},1{\mathrm{m}/\mathrm{s}}^2]}^T,\hfill \\ \hfill & \mathbf{P}\left(0\right|0)=\mathrm{diag}([100,1,100,100,1]).\hfill \end{array}$$

where the values are adjusted according to an in-city driving experience. Note that the estimated initial state ${\widehat{\mathbf{x}}}_0\sim \mathcal{N}(\widehat{\mathbf{x}}\left(0\right|0),\mathbf{P}\left(0\right|0))$ is a random value that changes per Monte Carlo run. This simulation sets the state noise variance to ${\sigma }_q^2=0.01$ and the sample time to $T_s=0.1$ s for computing ${\mathbf{F}}_k$ and ${\mathbf{Q}}_k$ to ensure constant acceleration.

4.2. Evaluation Metrics

This paper considers numerous error measures to evaluate and compare the estimation performance of the proposed cognitive radar to the alternative design in Table 1. The error measures include the root mean square relative error (RMSRE), the average Euclidean relative error (ARE), the harmonic average relative error (HRE), and the geometric average relative error (GRE) [38]. Due to numerical reasons, the logarithm of the GRE is calculated instead, $log\left(\mathrm{GRE}\right)$.

Table 2. Relative error measures and absolute values for performance evaluation [38].

	Relative Error Metric	Absolute of Error Metric
Root mean square relative error (RMSRE)	${\mathrm{RMSRE}}_k=\frac{{\left({\sum }_{j=1}^{N_{mc}}\left|\right|{\tilde{\mathbf{x}}}_k^j{\left|\right|}^2\right)}^{\frac{1}{2}}}{{\left({\sum }_{j=1}^{N_{mc}}\left|\right|{\mathbf{x}}_k^j{\left|\right|}^2\right)}^{\frac{1}{2}}}$	$\mathrm{ARMSRE}=\frac{1}{K}{\sum }_{k=1}^K{\mathrm{RMSRE}}_k$
Average Euclidean relative error (ARE)	${\mathrm{ARE}}_k=\frac{{\sum }_{j=1}^{N_{mc}}\left|\right|{\tilde{\mathbf{x}}}_k^j\left|\right|}{{\sum }_{j=1}^{N_{mc}}\left|\right|{\mathbf{x}}_k^j\left|\right|}$	$\mathrm{AARE}=\frac{1}{K}{\sum }_{k=1}^K{\mathrm{ARE}}_k$
Harmonic average relative error (HRE)	${\mathrm{HRE}}_k=\frac{{\left({\sum }_{j=1}^{N_{mc}}\left|\right|{\tilde{\mathbf{x}}}_k^j{\left|\right|}^{-1}\right)}^{-1}}{{\left({\sum }_{j=1}^{N_{mc}}\left|\right|{\mathbf{x}}_k^j{\left|\right|}^{-1}\right)}^{-1}}$	$\mathrm{AHRE}=\frac{1}{K}{\sum }_{k=1}^K{\mathrm{HRE}}_k$
Geometric average relative error (GRE)	$log\left({\mathrm{GRE}}_k\right)=\frac{1}{N_{mc}}{\sum }_{j=1}^{N_{mc}}log\frac{\left|\right|{\tilde{\mathbf{x}}}_k^j\left|\right|}{\left|\right|{\mathbf{x}}_k^j\left|\right|}$	$log\left(\mathrm{AGRE}\right)=\frac{1}{K}{\sum }_{k=1}^{K}log\left({\mathrm{GRE}}_k\right)$

provides the mathematical expressions of these error measures. In Table 2, ${\tilde{\mathbf{x}}}_k^j={\widehat{\mathbf{x}}}^j\left(k\right|k)-{\mathbf{x}}_k^j$, where ${\mathbf{x}}_k^j$, ${\widehat{\mathbf{x}}}^j\left(k\right|k)$ and ${\tilde{\mathbf{x}}}_k^j$ are, respectively, the state vector, the estimated state vector, and the state estimation error of the j-th Monte Carlo simulation at time step k. $N_{mc}$ represents the number of Monte Carlo simulations. The paper considers relative error measures since they are suitable for the performance evaluation of an estimation algorithm. However, the absolute value of the error metrics mentioned above—which computes the time average—is justified for ranking the overall state estimation performance of the cognitive radar. The absolute error counterparts of the relative error measures are given in Table 2.

4.3. Performance Evaluation and Comparison for One-Step Planning

This section demonstrates the estimation response of the proposed radar design by tracking the velocity of the target vehicle, $v_{x,k}^1$, and the longitude distance, $d_{x,k}$, when one-step planning is involved. The experiment examines target tracking in both highway and urban driving. Results are obtained for $N_{mc}=10,000$ Monte Carlo runs. Multiple attributes are considered for ranking the radar designs’ overall estimation performance, as listed in Table 1. This paper applies a pairwise comparison technique that adopts a ranking vector (RV) to compare different estimation algorithms [32]. This method exploits comparison information based on the probability of the relative closeness of competing estimators to the true quantity. The authors in [32] discuss a variety of approaches for determining a unique RV. Here, order-preserving mapping is considered to obtain the RV for ranking state estimation performance. Since this method is straightforward, the paper solely refers to the results of applying this strategy. In addition, the authors of this paper decided only to present the RMSRE curves of the estimation performance to avoid unnecessary repetition. However, the absolute relative error measures (i.e., ARMSRE, AARE, AHRE, and AGRE) are recorded to evaluate the entire estimation performance.

Figure 3 and Figure 4, respectively, illustrate the RMSRE performance of velocity and longitude distance of the target vehicle for a highway driving experience. The duration of the experiment is set to 10 s. Although the RMSRE results are plotted in a logarithm scale, the figures show that the estimation response of the four radar models are in close proximity. In this regard, the estimation performance is ranked based on a pairwise comparison method for the mentioned error measures.

Table 3. Performance comparison of radar designs versus multiple error measures for estimating the target’s velocity in highway driving.

	$\mathit{E}\left[{\mathcal{S}}_{\mathit{k}+1}^{\mathit{B}}\right]$	$\mathbf{tr}\left\{{\mathit{S}}_{\mathit{k}+1}\right\}$	$\mathit{E}\left[{\mathcal{F}}_{\mathit{k}+1}\right]$	${\mathcal{H}}_{\mathit{k}+1}$
ARMSRE	$0.0202$	$0.0203$	$0.0204$	$0.0204$
AARE	$0.0162$	$0.0162$	$0.0163$	$0.0163$
AHRE	$0.0022$	$0.0023$	$0.0023$	$0.0023$
$log\left(\mathrm{AGRE}\right)$	$-4.68$	$-4.68$	$-4.67$	$-4.67$

provides the absolute error measure values regarding the velocity of the target vehicle. For the results in Table 3, the RV based on order-preserving mapping is computed as follows:

$${\mathbf{r}}_1={\left[\begin{array}{cccc}0.8086& 0.5401& 0.1650& 0.1650\end{array}\right]}^T,$$

where the order of the elements in ${\mathbf{r}}_1$ is similar to the order shown in Table 3. The magnitude reflects the goodness of the approach relative to each other. The larger the value, the better the corresponding estimation performance. According to ${\mathbf{r}}_1$, the rank for the velocity of the target vehicle in the highway scenario is $\mathbf{E}\left[{\mathcal{S}}_{k+1}^B\right]>\mathrm{tr}\left\{{\mathbf{S}}_{k+1}\right\}>\mathbf{E}\left[{\mathcal{F}}_{k+1}\right]={\mathcal{H}}_{k+1}$, indicating that the expectation of Bayesian surprise exceeds the alternative designs of the information processor. ${\mathbf{r}}_1$ also implies that the expectation of free energy and Haykin’s Shannon entropy rank similarly in estimation performance.

Furthermore,

Table 4. Performance comparison of radar designs versus multiple error measures for estimating the longitude distance in highway driving.

	$\mathit{E}\left[{\mathcal{S}}_{\mathit{k}+1}^{\mathit{B}}\right]$	$\mathbf{tr}\left\{{\mathit{S}}_{\mathit{k}+1}\right\}$	$\mathit{E}\left[{\mathcal{F}}_{\mathit{k}+1}\right]$	${\mathcal{H}}_{\mathit{k}+1}$
ARMSRE	$0.0028$	$0.0028$	$0.0028$	$0.0028$
AARE	$0.0022$	$0.0022$	$0.0022$	$0.0022$
AHRE	$3\times {10}^{-4}$	$3\times {10}^{-4}$	$3\times {10}^{-4}$	$3\times {10}^{-4}$
$log\left(\mathrm{AGRE}\right)$	$-6.67$	$-6.67$	$-6.67$	$-6.67$

reports the absolute error measures of the longitude distance for highway driving. Apparently, the four radar designs present identical outcomes for $10,000$ runs of Monte Carlo simulations. This eventually leads to the following RV:

$${\mathbf{r}}_2={\left[\begin{array}{cccc}0.5& 0.5& 0.5& 0.5\end{array}\right]}^T,$$

where it indicates that the estimation performance of the longitude distance ranks the same for all the radar models.

Figure 5 and Figure 6, respectively, depict the RMSRE curves of the velocity and longitude distance when the vehicle-following scenario takes place in an urban environment. In this experiment, results are simulated for 7 s. While the RMSRE curves regarding the four designs converge over time, the estimation response based on the trace of the influence matrix experiences the lowest error at earlier time instances.

Table 5. Performance comparison of radar designs versus multiple error measures for estimating the target’s velocity in urban driving.

	$\mathit{E}\left[{\mathcal{S}}_{\mathit{k}+1}^{\mathit{B}}\right]$	$\mathbf{tr}\left\{{\mathit{S}}_{\mathit{k}+1}\right\}$	$\mathit{E}\left[{\mathcal{F}}_{\mathit{k}+1}\right]$	${\mathcal{H}}_{\mathit{k}+1}$
ARMSRE	$0.0081$	$0.0079$	$0.0083$	$0.0083$
AARE	$0.0059$	$0.0057$	$0.006$	$0.006$
AHRE	$0.001$	$0.001$	$0.001$	$0.001$
$log\left(\mathrm{AGRE}\right)$	$-5.95$	$-5.98$	$-5.92$	$-5.92$

displays the different approaches to modeling cognitive radar versus the absolute relative error metrics for estimating the target’s velocity. As expected, the trace of the influence matrix presents a minimum level of error compared to the alternative designs. As a result, the following RV is achieved:

$${\mathbf{r}}_3={\left[\begin{array}{cccc}0.4926& 0.7896& 0.2587& 0.2587\end{array}\right]}^T,$$

where the influence matrix trace ranks the topmost in estimating the velocity. Our proposed scheme is second on the ranking scale. The expectation of Bayesian surprise experiences a more significant error than the trace of the influence matrix, with AHRE as the only exception. The expectation of free energy and Haykin’s design present the poorest estimation.

Table 6. Performance comparison of radar designs versus multiple error measures for estimating the longitude distance in urban driving.

	$\mathit{E}\left[{\mathcal{S}}_{\mathit{k}+1}^{\mathit{B}}\right]$	$\mathbf{tr}\left\{{\mathit{S}}_{\mathit{k}+1}\right\}$	$\mathit{E}\left[{\mathcal{F}}_{\mathit{k}+1}\right]$	${\mathcal{H}}_{\mathit{k}+1}$
ARMSRE	$0.0014$	$0.0014$	$0.0014$	$0.0014$
AARE	$0.001$	$0.001$	$0.001$	$0.001$
AHRE	$1\times {10}^{-4}$	$1\times {10}^{-4}$	$1\times {10}^{-4}$	$1\times {10}^{-4}$
$log\left(\mathrm{AGRE}\right)$	$-7.74$	$-7.74$	$-7.74$	$-7.74$

provides the overall performance for longitude distance in urban driving. Table 6 shows that all models offer the same outcome for each error measure. Therefore, all designs are ranked equally regarding estimation performance, similar to highway driving.

According to this experiment, the following remarks can be made for one-step planning. In the case of highway driving, the expectation of Bayesian surprise outperforms the other three techniques in estimating the target’s velocity. In the meantime, the trace of influence matrix is a better choice for modeling the information processor in an urban environment. Note that both methods only consider ${\mathbf{R}}_{k+1}{\mathbf{P}}_{\tilde{\mathbf{z}}}{(k+1|k)}^{-1}$ as the means to minimize the state estimation error for the next time instant. This implies that the uncertainty in the measurements balanced by the certainty in innovation provides sufficient information to predict and estimate the target’s dynamic state ahead of time.

4.4. Performance Evaluation and Comparison for L-Step Planning

This experiment evaluates the estimation performance of the proposed cognitive radar when the impact of L future measurements are considered in estimating the target’s state for the upcoming time cycle. The results of this experiment are averaged over $N_{mc}=1000$ Monte Carlo simulations for the highway driving scenario. This section examines how $v_{x,k}^1$ estimation improves when multiple-step planning is assumed. According to the figures and tables in the previous section, the longitude distance seems invariant for the various error measures. To this end, this experiment focuses only on estimating the target’s velocity. Figure 7 illustrates the RMSRE performance of the proposed radar design for $L=\{1,2,3\}$. Figure 7 shows that the estimation error is substantially decreased by increasing the planning step from one to two. Although three-step planning outperforms them all, the amount of errors reduced by changing $L=2$ to $L=3$ is negligible compared to $L=1$ to $L=2$. Additionally, increasing L is associated with a longer simulation run time and higher computational complexity. Thus, two-step planning seems the optimum fit to enhance the state estimation performance of the proposed cognitive radar.

For two-step planning, this section also analyzes the estimation performance of the proposed cognitive radar with alternative designs. Figure 8 compares the RMSRE curves of the four radar designs by setting $L=2$ and $N_{mc}=10,000$ for highway driving. According to Figure 8, the expectation of Bayesian surprise and the expectation of free energy present minimum estimation errors with respect to RMSRE, while surpassing the other two techniques.

Table 7. Performance comparison with respect to the ARMSRE of the target’s velocity for two-step planning in highway driving.

	$\mathit{E}\left[{\mathcal{S}}_{\mathit{k}+1}^{\mathit{B}}\right]$	$\mathbf{tr}\left\{{\mathit{S}}_{\mathit{k}+1}\right\}$	$\mathit{E}\left[{\mathcal{F}}_{\mathit{k}+1}\right]$	${\mathcal{H}}_{\mathit{k}+1}$
ARMSRE	$0.0126$	$0.0182$	$0.0127$	$0.0156$

supports this claim in terms of the absolute RMSRE. The results indicate that, on average, the expectation of Bayesian surprise improves the estimation process when multiple-step planning is considered.

5. Conclusions

This paper proposed a novel design of cognitive radar that plans the estimation response of the system based on the expectation of Bayesian surprise and makes a decision by reducing the estimation error over time. In this work, the radar measurements were expressed for a set of linear Gaussian state-space models to describe the motion dynamics of a simple vehicle-following scenario. Assuming that the model parameters are somehow known, the Kalman filter was applied for state estimation. According to the filter’s estimate, the radar measures how much information each waveform—available from the sensor profile library—contributes to estimating the target’s future state (i.e., velocity, distance), and chooses the one that maximizes the expectation of Bayesian surprise. This research showed that maximizing the expectation of Bayesian surprise leads to informative measurements and successively decreases the state estimation error. In addition, estimation algorithms for one-step planning and multiple-step planning are determined. The paper also demonstrated a unified framework to re-introduce and relate different design methodologies to model cognitive radar systems. Several experiments were carried out to evaluate and compare the estimation performance of the proposed method to alternative designs. Numerical results were implemented to emulate real-life highway and urban driving experiences. The paper examined the credibility of the proposed approach based on a pairwise comparison method for various error measures. Results indicated that the balance between uncertainty in the measurements and the certainty in innovations provides sufficient information for accurate target tracking for one-step planning. The paper also demonstrated that two-step planning improves the estimation error significantly compared to one-step planning. Meanwhile, the proposed radar design exceeds its competitors’ overall estimation performance when two-step planning is applied.