Optimizing the Energy Efficiency of Unreliable Memories for Quantized Kalman Filtering

Abstract

This paper presents a quantized Kalman filter implemented using unreliable memories. We consider that both the quantization and the unreliable memories introduce errors in the computations, and we develop an error propagation model that takes into account these two sources of errors. In addition to providing updated Kalman filter equations, the proposed error model accurately predicts the covariance of the estimation error and gives a relation between the performance of the filter and its energy consumption, depending on the noise level in the memories. Then, since memories are responsible for a large part of the energy consumption of embedded systems, optimization methods are introduced to minimize the memory energy consumption under the desired estimation performance of the filter. The first method computes the optimal energy levels allocated to each memory bank individually, and the second one optimizes the energy allocation per groups of memory banks. Simulations show a close match between the theoretical analysis and experimental results. Furthermore, they demonstrate an important reduction in energy consumption of more than 50%.

1. Introduction

Kalman filtering is a very common recursive estimation task in statistical signal processing [1], and it is often implemented on resource-limited hardware. Applications that require an embedded energy-efficient Kalman filter include air quality monitoring [2], biomedical wearable sensors [3], forest fire detection [4] and vehicle positioning [5]. Energy budgets for embedded systems show that memory access consumes about a hundred-times more energy than integer computations [6]. Therefore, in this paper, we focus on optimizing the energy used by memories in Kalman filters.

All memories used in integrated circuits exhibit a fundamental trade-off between data storage reliability and energy consumption that is related to the inability of perfectly controlling the fabrication process. For example, the energy consumption of static random access memories (SRAMs) can be reduced by lowering its supply voltage; however, this increases the probability that some of the stored bits cannot be retrieved correctly [7]. Following this principle, ref. [8] developed an optimization method to lower the energy consumed by SRAM access by reducing the bit-line voltages. This methodology was also used to decrease the write energy of magnetic random access memories (MRAM) in [9]. In both cases, however, this introduces errors in the words stored in memory.

The robustness to unreliability in computation operations and memories has been investigated for several signal processing and machine-learning applications, including binary recursive estimation [10], binary linear transformation [11], deep neural networks [12,13], multi-agent systems [14] and distributed logistic regression [15]. Moreover, several techniques have been proposed to compensate for faults introduced by unreliable systems. For instance, [16] proposed to add redundancy in the system through algorithmic noise tolerance, and [17] investigated the use of error-correction codes (ECC) for fault correction.

Although Kalman filtering has not previously been investigated under unreliable hardware implementation, some related works considered this filter and other similar models for linear systems under uncertain conditions. These include errors on the filter’s gain [18,19], sensors failures, uncertainties on the observations [20,21], or inaccuracies in the filter parameters [22,23,24]. In these works, new filter equations were derived using the Riccati equations approach to find new bounds or guarantee on the performance of the filter.

Although these models are not relevant for characterizing the effect of unreliable memories, the main lessons they provide are that Kalman filtering is very sensitive to inaccuracies and that one should re-derive the optimal Kalman filter depending on the specifically considered uncertainty model. On a different line of research, other prior works aim at reducing the energy requirements for Kalman filtering by focusing on reduced computational complexity in field-programmable gate arrays (FGPAs) [25,26] and application-specific integrated circuits (ASICs) [27].

Designing a digital hardware implementation requires quantizing all the variables and computational operations. Therefore, to further reduce the memory energy consumption, one option is to properly optimize the quantization to reduce the memory requirements of the implementation. Significant energy gains from optimized quantization have been demonstrated in [28,29,30] for signal processing and digital communications applications and in [31,32,33] for neural networks. The effects of quantization on the Kalman filter were first studied in [34,35] to understand the convergence of filters with reduced precision.

More recently, refs. [36,37,38] considered two distributed quantized Kalman filters, one based on quantized observations and one based on quantized innovations, where sensors process and transmit quantized observations and innovations to a fusion center. Furthermore, ref. [36] proposed to optimize the number of quantization bits at each sensor to minimize the required data transmission energy.

More general linear stochastic systems were also investigated under quantized measurements [39] and quantized innovations [40], where it was shown that the derived quantized filters converged to standard Kalman filters as the number of quantization levels increased. However, none of these theoretical works considered quantized parameters (e.g., quantized Kalman gain matrices, quantized measurement matrices, etc.), in addition to quantized observations/innovations. Therefore, in this paper, we study a fully-quantized Kalman filter and investigate its energy consumption when using unreliable memories.

Here, we aim to optimize the energy consumption of a Kalman filter implemented with fixed-point quantization [41] and with unreliable memories. Fixed-point representations are often preferred in energy-constrained systems as a fixed-point operation can consume 10-times less energy than a floating-point one [6]. We consider the statistical model of [7], which relates the amount of faults introduced in memory for its energy consumption. Then, as a first contribution, we propose a unified framework to analyze the performance of Kalman filters with both quantization errors and faults introduced in the memory.

To develop this framework, we build on the approach of [35], which consists of evaluating the covariance matrix of the estimation error at each filter iteration by considering both error propagation from previous iterations and errors introduced at the current iteration. Our analysis also includes quantized filter parameters and further incorporates the effect of unreliable memories. Determining the covariance matrix of the estimation error has two advantages. First, it allows us to derive the optimal Kalman filter equations under the considered quantization and memory error models. Second, and more specific to our case, it defines a performance criterion that will be used to optimize the memory energy consumption.

As a second contribution, we define two optimization problems to minimize the memory energy consumption while satisfying a target constraint on the estimation performance of the Kalman filter. In the first problem, we optimize the number B of quantization bits and the energy allocated to each bit position to minimize the overall energy consumption of the memory. This optimization problem extends to [8], which was not dedicated to Kalman filtering but derived optimal bitwise energy allocations with a fixed number of quantization bits by considering a generic Mean-Squared Error (MSE) performance criterion when reading a word in memory.

Although a useful baseline, the setting where each bit position can have a different energy allocation is not practical, since each of the B bits should be placed in a different memory bank with its own power supply. This is why we also introduce a second optimization problem in which we fix the number $L<B$ of possible energy levels and optimize the energy value in each level and the mapping of bit positions to an energy level.

At the price of a small energy increase, this optimization problem allows us to build a practical implementation that only requires L memory banks. By using the Karush–Kuhn–Tucker (KKT) conditions, we provide solutions for the two considered optimization problems. Both solutions can be numerically computed using water-filling. Numerical simulations show that, after optimization, the memory energy consumption is reduced by up to $56\%$ compared to uniform allocation.

The main contributions of this paper can be summarized as:

(1)

We develop an error propagation model of the Kalman filter that takes different sources of errors (quantization, unreliable memories) into account and allows us to derive new filter equations to minimize the estimation error. Moreover, these equations accurately predict the filter’s performance, depending on the considered sources of errors and on their parameters.

(2)

We propose a methodology for minimizing the energy of the unreliable memories used in the Kalman filter, under a given performance constraint. This methodology consists of computing the optimal number of quantization levels and bit energy allocation in two setups. The first setup considers that the B energy levels can be chosen freely, while the second one assumes that only $L<B$ energy levels can be set.

A preliminary version of this paper [42] only considered optimizing the energy allocation for each memory bank individually and for a fixed number of bits without taking into account the quantization noise nor trying to reduce the energy consumption by adjusting the number of bits. Since the number of quantization bits also affects the energy consumption of the memory, in this work, we add this parameter to the theoretical analysis and the optimization problems.

The rest of the paper is organized as follows. Section 2 describes the quantized Kalman filter and introduces the uncertainty model for unreliable memories. Section 3 investigates the theoretical performance of the filter. Section 4 formally defines and solves the two considered optimization problems. Section 5 presents the simulation results.

2. System Model

We first review the Kalman filter for estimating dynamic state variables from noisy measurements. We then present the considered implementation of the filter by first introducing its quantization model and then describing its implementation with an unreliable memory.

2.1. Kalman Filter

The process:

$${\mathit{x}}_{k+1}=\mathit{F}{\mathit{x}}_k+{\mathit{u}}_k,$$

(1)

describes the linear dynamic variable $\mathit{x}\in {\mathbb{R}}^c$, where the state vector of the process at step k is noted as ${\mathit{x}}_k$, $\mathit{F}$ is the state transition matrix of size $c\times c$, and ${\mathit{u}}_k\in {\mathbb{R}}^c$ is an additive white noise vector [43]. Observation of the state of $\mathit{x}$ can be obtained through $\mathit{y}\in {\mathbb{R}}^d$, the measurement vector defined as

$${\mathit{y}}_k=\mathit{H}{\mathit{x}}_k+{\mathit{v}}_k.$$

(2)

Here, $\mathit{H}$ is the $d\times c$ measurement model and ${\mathit{v}}_k\in {\mathbb{R}}^d$ is an additive white noise on the measurements, independent from the model noise ${\mathit{u}}_k$. We denote $\mathit{Q}$ and $\mathit{R}$ as the known covariance matrices of the noise vectors ${\mathit{u}}_k$ and ${\mathit{v}}_k$, respectively.

Using the knowledge of the model as well as the the measurement vectors ${\mathit{y}}_k$, the Kalman filter [1] recursively estimates the successive states ${\mathit{x}}_k$. This is done by minimizing the mean squared error between ${\mathit{x}}_k$ and its estimate ${\widehat{\mathit{x}}}_k$ at each step k: $MSE\left(\mathit{x}\right)=\mathbb{E}[{\parallel {\mathit{x}}_k-{\widehat{\mathit{x}}}_k\parallel }^2]$. The filter can be decomposed in two phases: the a priori estimation uses only the known model, and the a posteriori estimation takes into account the measurements.

At each phase, both the estimates ${\widehat{\mathit{x}}}_{k+1|k}$ (for the a priori phase) and ${\widehat{\mathit{x}}}_{k+1|k+1}$ (for the a posteriori phase) of the state vector ${\mathit{x}}_{k+1}$, and the covariance matrices of the estimations errors ${\mathit{P}}_{k+1|k}=\mathrm{Cov}[{\mathit{x}}_{k+1}-{\widehat{\mathit{x}}}_{k+1|k}]$ and ${\mathit{P}}_{k+1|k+1}=\mathrm{Cov}[{\mathit{x}}_{k+1}-{\widehat{\mathit{x}}}_{k+1|k+1}]$ are computed. The recursive equations of the a priori estimation step are [44]:

$$\begin{array}{cc}\hfill & {\widehat{\mathit{x}}}_{k+1|k}=\mathit{F}{\widehat{\mathit{x}}}_{k|k},\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill & {\mathit{P}}_{k+1|k}=\mathit{F}{\mathit{P}}_{k|k}{\mathit{F}}^{\top }+\mathit{Q},\hfill \end{array}$$

(4)

and the recursive equations of the a posteriori estimation step are:

$$\begin{array}{cc}\hfill & {\mathit{K}}_{k+1}={\mathit{P}}_{k+1|k}{\mathit{H}}^{\top }{(\mathit{H}{\mathit{P}}_{k+1|k}{\mathit{H}}^{\top }+\mathit{R})}^{-1},\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill & {\widehat{\mathit{x}}}_{k+1|k+1}={\widehat{\mathit{x}}}_{k+1|k}+{\mathit{K}}_{k+1}({\mathit{y}}_{k+1}-\mathit{H}{\widehat{\mathit{x}}}_{k+1|k}),\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill & {\mathit{P}}_{k+1|k+1}=(\mathit{I}-{\mathit{K}}_{k+1}\mathit{H}){\mathit{P}}_{k+1|k},\hfill \end{array}$$

(7)

where ${\mathit{A}}^{\top }$ denotes the transpose of a matrix $\mathit{A}$. In these equations, the Kalman gain $\mathit{K}$ of size $c\times d$ and the covariance matrices $\mathit{P}$ of size $c\times c$ can be computed offline. On the other hand, the terms ${\widehat{\mathit{x}}}_{k+1|k}$ and ${\widehat{\mathit{x}}}_{k+1|k+1}$ must be computed online as they depend on the measurements ${\mathit{y}}_k$.

2.2. Quantized Implementation of the Filter

In the rest of the paper, we study Kalman filters that are implemented under fixed-point quantization [41]. Under this model, each number is represented as a signed integer coded on $(1+n+m)$ bits, where one bit is used for the sign, n bits are used for the integral part of the number, and m bits are used for its fractional part. Using this model, we can write a given number z as

$$z={(-1)}^{z_n}\sum _{b=-m}^{n-1}{2}^b{z}_b,$$

(8)

where $z_b\in \{0,1\}$ are the bits stored in memory to represent z. In our modeling of the Kalman filter, all variables (including matrix components) involved in Equations (3)–(7) are stored using this quantization model, all with the same values of n and m. The quantization of the variables to this fixed-point model is done using a uniform quantizer. Note that the distribution of the quantized data is not necessarily uniform (the random variables ${\widehat{\mathit{x}}}_{k|k}$ and ${\mathit{y}}_k$ could follow Gaussian distributions for example). However, in [45] it is shown that a uniform quantizer can be applied independently of the probability distribution of the source with only a small difference to an optimal quantizer.

In the considered quantizer, the value of n is chosen to be able to represent the largest possible value in the system. The value of m sets the resolution of the quantization so that the smallest difference between two quantized numbers is $2^{-m}$ [41]. The value of m will be a parameter that is optimized for minimizing the energy in later sections.

In the case of fixed values, such as components of the matrices of the filter, the fixed-point quantized value can be written as $\underset{¯}{f}=f+{\delta }_f$ where ${\delta }_f$ is the quantization error. Using the previously described uniform quantizer, ${\delta }_f<2^{-m}$. In the case of quantized random variables, such as the components of ${\widehat{\mathit{x}}}_{\mathit{k}\mathbf{|}\mathit{k}}$ or ${\mathit{y}}_{\mathit{k}}$, we let ${ϵ}_x$ be the quantization error and express $\underset{¯}{x}=x+{ϵ}_x$. In [46], conditions are given for the quantization error ${ϵ}_x$ to be independent from the quantized variable depending on the distribution of the quantized data.

For the special case of a Gaussian distribution, the quantization step needs to be significantly smaller than the variance of the quantized data. In this case, it can be shown that the quantization error is a white noise following a uniform distribution of variance $\frac{2^{-2m}}{12}$. This independence assumption will be used in the theoretical derivations of Section 3. Note that most existing works on quantized Kalman filters only consider that random quantities, such as ${\widehat{\mathit{x}}}_{\mathit{k}\mathbf{|}\mathit{k}}$ and ${\mathit{y}}_{\mathit{k}}$, are quantized, whereas here, the components of the matrices, e.g., ${\mathit{K}}_k$ of the filter, are also quantized. This will require a new theoretical analysis to treat this case.

2.3. Implementation of the Filter by Using an Unreliable Memory

In order to reduce its energy consumption, the quantized Kalman filter can be implemented on unreliable hardware [8,10,11,12]. Here, we assume, as in [10,12], that only the memory is faulty. In this case, each memory cell of a memory bank has a bit flipping probability p. We then use the model of [7] to express p with respect to the memory bank energy consumption e as

$$p=exp(-ea),$$

(9)

where a is a parameter that depends on the device technology. We assume that bit errors occur independently. This is justified first by the fact that, in many cases of interest, such as the common case of SRAM memories in a CMOS digital circuit, memory failures can be assumed to occur independently for each bit cell [47]. Therefore, we have a spatial independence between each memory cell for one iteration.

However, typically faults are caused by fabrication variations, and therefore this cannot guarantee a temporal independence for successive reads of the same memory cells. To resolve this issue, we can assume that a diversity scheme is implemented at the system level to avoid re-using the same memory location to store the same variable, which can be implemented at very low cost simply by modifying the memory addressing scheme.

Each memory bank has a uniform energy consumption (e.g., single supply voltage) and is used in our case to store the bits at a certain position of all components of matrices that are stored in the unreliable memory. Since the other terms of the filter can be precomputed offline and stored on a reliable memory separately in the system, we assume that only the estimates ${\widehat{\mathit{x}}}_{k+1|k}$ and ${\widehat{\mathit{x}}}_{k+1|k+1}$ are stored in an unreliable memory bank. Therefore, in the Kalman filter, instead of having an estimate component $\widehat{x}$, such as the one computed in (3), we have a possibly incorrect estimate component $\tilde{x}$. We can define an energy per memory bank vector using the binary representation given in (8):

$$\mathit{e}=\left[\begin{array}{c}{e}_{-m},e_{-m+1},\dots ,e_{n-1}\end{array}\right].$$

(10)

We can then express as ${\tilde{x}}_b={\widehat{x}}_b\oplus {\gamma }_b$ a bit at position b stored in the unreliable memory. Here, $p_b=Pr({\gamma }_b=1)=exp(-e_{b}a)$, and ⊕ denotes the modulo-2 addition. As the filter would be particularly sensitive to faults on the sign bit, we consider a sign-preserving model, as in [10,48,49]. This sign-preserving model can be implemented by storing the sign bits in a separate reliable memory.

Using this noise model defined at the bit-level ${\tilde{x}}_b$, we can define a noise model at the symbol level $\tilde{x}$ as

$$\tilde{x}=\widehat{x}+\gamma ,$$

(11)

where $\gamma$ is the noise introduced by the unreliable memory. For the subsequent theoretical analysis, we assume that the mean $\mathbb{E}\left[\gamma \right]$ of this memory noise is negligible compared to its variance $Var\left[\gamma \right]={\sigma }_{\gamma }^2$. We verified this condition with Monte Carlo simulations. The covariance matrix $\mathbf{\Gamma }$ of a memory noise vector $\mathit{\gamma }$ of length c is defined as $\mathbf{\Gamma }=\mathrm{Cov}\left[\mathit{\gamma }\right]={\mathit{I}}_c{\sigma }_{\gamma }^2,$ and has size $c\times c$. The matrix $\mathbf{\Gamma }$ is diagonal since the memory noise variables are considered independent.

3. Error Analysis

As described in Section 2, we consider two types of errors affecting the filter: the quantization error and the unreliable memory noise. In this section, we first describe a generic model of error propagation in the Kalman filter, before studying both types of errors in more detail. Finally, we compute the new covariance matrix ${\mathit{P}}_{\mathit{k}\mathbf{|}\mathit{k}}^{\mathbf{*}}=\mathrm{Cov}\left[{\tilde{\mathit{x}}}_{k|k}-{\mathit{x}}_k\right]$ of the total estimation error by taking both sources of noise (quantization and unreliable memories) into account, compared to a standard Kalman filter, which does not include either.

3.1. Error Propagation Model

Our objective is to compute the total error $\mathbf{\Delta }{\widehat{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}$ on the computation of ${\widehat{\mathit{x}}}_{k+1}$ at step $k+1$ by considering the two types of errors: quantization and unreliable memory. To handle recursion as in [35], we choose to split the error model in two parts: the errors occurring at step k and the errors from the previous steps, which are propagated up to step k.

To compute $\mathbf{\Delta }{\widehat{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}$, we first need to express the total error ${\mathbf{\Delta }\mathit{P}}_{k+1|k}$ on the a posteriori covariance matrix ${\mathit{P}}_{k+1|k}$ after step $k+1$. As in [35], we express this total error as

$$\mathbf{\Delta }{\mathit{P}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}}=f_P\left(\mathbf{\Delta }{\mathit{P}}_{\mathit{k}\mathbf{|}\mathit{k}\mathbf{-}\mathbf{1}}\right)+\mathit{\delta }{\mathit{P}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}},$$

(12)

where the function $f_P$ models the errors propagated from step k, and $\mathit{\delta }{\mathit{P}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}}$ represents the errors occurring at step $k+1$. In this case, according to [35]:

$$f_P\left(\mathbf{\Delta }{\mathit{P}}_{\mathit{k}\mathbf{|}\mathit{k}\mathbf{-}\mathbf{1}}\right)={\mathit{G}}_k\mathbf{\Delta }{\mathit{P}}_{\mathit{k}\mathbf{|}\mathit{k}\mathbf{-}\mathbf{1}}{\mathit{G}}_{\mathit{k}}^{\top }+o\left({\mathbf{\Delta }}^{\mathit{2}}\right),$$

(13)

where $\mathbf{\Delta }={∥\mathit{H}\mathbf{\Delta }{\mathit{P}}_{\mathit{k}\mathbf{|}\mathit{k}\mathbf{-}\mathbf{1}}{\mathit{H}}^{\top }∥}_2\ll {\sigma }_{\min }(\mathit{H}{\mathit{P}}_{k|k-1}{\mathit{H}}^{\top }+\mathit{R})$ and $\mathit{H}{\mathit{P}}_{k|k-1}{\mathit{H}}^{\top }+\mathit{R}$ is a square nonsingular matrix with ${\sigma }_{\min }$ representing the smallest singular value. Therefore, we have the approximation

$$f_P\left(\mathbf{\Delta }{\mathit{P}}_{\mathit{k}\mathbf{|}\mathit{k}\mathbf{-}\mathbf{1}}\right)\approx {\mathit{G}}_k\mathbf{\Delta }{\mathit{P}}_{\mathit{k}\mathbf{|}\mathit{k}\mathbf{-}\mathbf{1}}{\mathit{G}}_{\mathit{k}}^{\top },$$

(14)

where ${\mathit{G}}_k=\mathit{F}(\mathit{I}-{\mathit{K}}_k\mathit{H})$. We then express the total error $\mathbf{\Delta }{\widehat{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}$ on ${\widehat{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}$ by considering the same separation between propagation errors and errors from the current iteration. This gives

$$\mathbf{\Delta }{\widehat{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}=f_x(\mathbf{\Delta }{\widehat{\mathit{x}}}_{\mathit{k}\mathbf{|}\mathit{k}},\mathbf{\Delta }{\mathit{P}}_{\mathit{k}\mathbf{|}\mathit{k}\mathbf{-}\mathbf{1}})+\mathit{\delta }{\widehat{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}},$$

(15)

where the error propagation function $f_x$ is provided in [35], using the same assumption as for (13), as

$$\begin{array}{c}{f}_x(\mathbf{\Delta }{\widehat{\mathit{x}}}_{\mathit{k}\mathbf{|}\mathit{k}},\mathbf{\Delta }{\mathit{P}}_{\mathit{k}\mathbf{|}\mathit{k}\mathbf{-}\mathbf{1}})\approx \hfill \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} (\mathit{I}-{\mathit{K}}_k\mathit{H})(\mathit{F}\mathbf{\Delta }{\widehat{\mathit{x}}}_{\mathit{k}\mathbf{|}\mathit{k}}+\mathbf{\Delta }{\mathit{P}}_{\mathit{k}\mathbf{|}\mathit{k}\mathbf{-}\mathbf{1}}{\mathit{H}}^{\top }{(\mathit{H}{\mathit{P}}_{k,k-1}{\mathit{H}}^{\top }+\mathit{R})}^{-1}({\mathit{y}}_{k+1}-\mathit{H}\mathit{F}{\widehat{\mathit{x}}}_{k,k})).\hfill \end{array}$$

(16)

In this expression, we observe the error propagation from the previous computations of ${\widehat{\mathit{x}}}_{k|k}$${\mathit{P}}_{k+1|k}$. In particular, ${\widehat{\mathit{x}}}_{k+1|k+1}$ depends on ${\mathit{K}}_{k+1}$, which is precomputed from ${\mathit{P}}_{k+1|k}$ at each iteration.

Using the recursive Equations (12) and (15), we now estimate the covariance matrix of the total estimation error ${\mathit{P}}_{\mathit{k}\mathbf{|}\mathit{k}}^{\mathbf{*}}$. Note that [35] considered only quantization errors, while here we consider two sources of errors: quantization and unreliable memories. To evaluate ${\mathit{P}}_{\mathit{k}\mathbf{|}\mathit{k}}^{\mathbf{*}}$, we must first compute the covariance of each term of $\mathbf{\Delta }{\widehat{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}$. By assuming that the two sources of noise (quantization and unreliable memory noise) are statistically independent, we decompose $\mathit{\delta }{\widehat{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}$ as

$$\mathit{\delta }{\widehat{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}=\mathit{\delta }{\widehat{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}^{\mathbf{quant}}+\mathit{\delta }{\widehat{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}^{\mathbf{mem}},$$

(17)

and study the two terms $\mathit{\delta }{\widehat{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}^{\mathbf{quant}}$ and $\mathit{\delta }{\widehat{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}^{\mathbf{mem}}$ separately.

3.2. Quantization Error

We now aim for an analytical expression for $\mathit{\delta }{\widehat{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}^{\mathbf{quant}}$, defined as the difference between the full precision estimate ${\widehat{\mathit{x}}}_{k+1|k+1}$ and its quantized version ${\underset{¯}{\overset{\mathbf{^}}{\mathit{x}}}}_{k+1|k+1}$:

$$\mathit{\delta }{\widehat{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}^{\mathbf{quant}}={\widehat{\mathit{x}}}_{k+1|k+1}-{\underset{¯}{\overset{\mathbf{^}}{\mathit{x}}}}_{k+1|k+1}.$$

(18)

Before expressing $\mathit{\delta }{\widehat{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}^{\mathbf{quant}}$, we first review generic quantization errors expressions [35]. For the scalar fixed-point multiplication of a coefficient $\underset{¯}{s}$ with a random variable $\underset{¯}{t}$ both quantized according to the model presented in Section 2.2, we can show that

$$\begin{array}{cc}\hfill \underset{¯}{st}& =(s+{\delta }_s)(t+{ϵ}_t)+{ϵ}_{st}\hfill \\ \hfill & =st+s{ϵ}_t+t{\delta }_s+{\delta }_s{ϵ}_t+{ϵ}_{st},\hfill \end{array}$$

(19)

where ${\delta }_s=\underset{¯}{s}-s$ and ${ϵ}_t$ and ${ϵ}_{st}$ follow uniform distributions of variance $\frac{2^{-2m}}{12}$. The scalar expression (19) can then be generalized to the case of a product between a matrix of fixed-point coefficients $\underset{¯}{\mathit{A}}$ of size $p\times q$ and a matrix of fixed-point random variables $\underset{¯}{\mathit{B}}$ of size $q\times r$ as

$$\underset{¯}{\mathit{A}\mathit{B}}=\mathit{A}\mathit{B}+\mathit{A}{\mathit{ϵ}}_{\mathit{B}}+\mathit{B}{\mathit{\delta }}_{\mathit{A}}+{\mathit{\delta }}_{\mathit{A}}{\mathit{ϵ}}_{\mathit{B}}+{\mathit{ϵ}}_{\mathit{AB}},$$

(20)

where ${\mathit{ϵ}}_{\mathit{AB}}$ is of size $p\times r$ with ${{ϵ}_{AB}}_{i,j}={\sum }_{k=1}^q{{ϵ}_{AB}}_{i,j,k}$. According to Section 2.2, each ${{ϵ}_{AB}}_{i,j,k}$ follows a uniform distribution of variance $\frac{2^{-2m}}{12}$. In (20), the product ${\mathit{\delta }}_{\mathit{A}}{\mathit{ϵ}}_{\mathit{B}}$ can be considered as negligible compared to the other error terms. Indeed, all scalar quantization errors $ϵ$ and $\delta$ are upper-bounded by $2^{-m-1}$, and since $m\ge 1$, their product is bounded by $2^{-2m-2}$. Thus, given that, for a value of m large enough, the value of $2^{-m-1}$ is much less than 1 and $2^{-2m-2}={\left(2^{-m-1}\right)}^2$, we have that $2^{-2m-2}$ is negligible compared to $2^{-m-1}$. Therefore, in the following derivation, we neglect the products of quantization errors.

We now study quantization errors introduced during the computation of ${\underset{¯}{\overset{\mathbf{^}}{\mathit{x}}}}_{k+1|k+1}$. While existing works, e.g., [36,37,38], assume that only the random quantities ${\underset{¯}{\overset{\mathbf{^}}{\mathit{x}}}}_{k|k}$ and ${\underset{¯}{\mathit{y}}}_{k+1}$ are quantized, we here also consider that the matrices ${\underset{¯}{\mathit{D}}}_{k+1}$ and ${\underset{¯}{\mathit{K}}}_{k+1}$ are quantized as well. This corresponds to a more practical implementation setup and requires a more complex theoretical analysis. We first note that Equation (6) can be rewritten as

$$\begin{array}{c}\hfill {\underset{¯}{\overset{\mathbf{^}}{\mathit{x}}}}_{k+1|k+1}={\underset{¯}{\mathit{D}}}_{k+1}{\underset{¯}{\overset{\mathbf{^}}{\mathit{x}}}}_{k|k}+{\underset{¯}{\mathit{K}}}_{k+1}{\underset{¯}{\mathit{y}}}_{k+1},\end{array}$$

(21)

where both ${\mathit{D}}_k=(\mathit{I}-{\mathit{K}}_k\mathit{H})\mathit{F}$ and the Kalman gains ${\mathit{K}}_k$ can be computed offline. We thus consider that the matrices ${\mathit{K}}_k$ and ${\mathit{D}}_k$ are computed in full precision and then quantized with a fixed point model. Under these conditions, according to (20) and if we consider that the product of quantization errors is negligible, the quantized vector ${\underset{¯}{\overset{\mathbf{^}}{\mathit{x}}}}_{k+1|k+1}$ can be approximated as

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill {\underset{¯}{\overset{\mathbf{^}}{\mathit{x}}}}_{k+1|k+1}& ={\mathit{D}}_{k+1}{\widehat{\mathit{x}}}_{k|k}+{\mathit{\delta }}_{{\mathit{D}}_{\mathit{k}\mathbf{+}\mathbf{1}}}{\widehat{\mathit{x}}}_{k|k}+{\mathit{D}}_{k+1}{\mathit{ϵ}}_{x_{k|k}}\hfill \\ \hfill & + {\mathit{ϵ}}_{{\mathit{D}}_{k+1}{\mathit{x}}_{k|k}}+{\mathit{K}}_{k+1}{\mathit{y}}_{k+1}+{\mathit{\delta }}_{{\mathit{K}}_{\mathit{k}\mathbf{+}\mathbf{1}}}{\mathit{y}}_{k+1}\hfill \\ \hfill & + {\mathit{K}}_{k+1}{\mathit{ϵ}}_{{\mathit{y}}_{\mathit{k}\mathbf{+}\mathbf{1}}}+{\mathit{ϵ}}_{{\underset{¯}{\mathit{K}}}_{k+1}{\underset{¯}{\mathit{y}}}_{k+1}}+o\left(2^{-m-1}\right).\hfill \end{array}\end{array}$$

(22)

We see that the expression of ${\underset{¯}{\overset{\mathbf{^}}{\mathit{x}}}}_{k+1|k+1}$ depends on the full precision vectors ${\widehat{\mathit{x}}}_{k|k}$ and ${\mathit{y}}_{k+1}$ and on the quantization errors and noise. Finally, the quantization error $\mathit{\delta }{\widehat{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}^{\mathbf{quant}}$ defined in (18) can be computed by using (22):

$$\begin{array}{c}\hfill \mathit{\delta }{\widehat{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}^{\mathbf{quant}}\approx {\mathit{\delta }}_{{\mathit{D}}_{\mathit{k}\mathbf{+}\mathbf{1}}}{\widehat{\mathit{x}}}_{k|k}+{\mathit{D}}_{k+1}{\mathit{ϵ}}_{x_{k|k}}+{\mathit{ϵ}}_{{\mathit{D}}_{k+1}{\mathit{x}}_{k|k}}\\ \hfill +{\mathit{\delta }}_{{\mathit{K}}_{\mathit{k}\mathbf{+}\mathbf{1}}}{\mathit{y}}_{k+1}+{\mathit{K}}_{k+1}{\mathit{ϵ}}_{{\mathit{y}}_{\mathit{k}\mathbf{+}\mathbf{1}}}+{\mathit{ϵ}}_{{\mathit{K}}_{k+1}{\mathit{y}}_{k+1}},\end{array}$$

(23)

where the covariance matrix ${\mathbf{\Sigma }}_{\times }$ of ${\mathit{ϵ}}_{\times }={\mathit{ϵ}}_{{\mathit{D}}_{k+1}{\mathit{x}}_{k|k}}+{\mathit{ϵ}}_{{\mathit{K}}_{k+1}{\mathit{y}}_{k+1}}$ is given by

$$\begin{array}{cc}\hfill {\mathbf{\Sigma }}_{\times }& =\mathrm{Cov}[{\mathit{ϵ}}_{{\mathit{D}}_{k+1}{\mathit{x}}_{k|k}}]+\mathrm{Cov}[{\mathit{ϵ}}_{{\underset{¯}{\mathit{K}}}_{k+1}{\underset{¯}{\mathit{y}}}_{k+1}}]\hfill \end{array}$$

(24)

$$\begin{array}{cc}\hfill & ={\mathbf{I}}_c(c+d)\frac{2^{-2m}}{12},\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} \hfill \end{array}$$

(25)

and $\mathrm{Cov}[{\mathit{ϵ}}_{x_{k|k}}]={\mathbf{I}}_c\frac{2^{-2m}}{12}$, $\mathrm{Cov}[{\mathit{ϵ}}_{y_{k+1}}]{=}_d\frac{2^{-2m}}{12}$. Equation (23) gives us the quantization error on the computation of ${\widehat{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}$ based on the unquantized values of ${\widehat{\mathit{x}}}_{\mathit{k}\mathbf{|}\mathit{k}}$, the filter parameters and the quantization resolution m.

3.3. Unreliable Memory Error

We now consider the second source of noise from the unreliable memories and derive an expression for the covariance matrix $\mathbf{\Gamma }=\mathrm{Cov}[\mathit{\delta }{\widehat{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}^{\mathbf{mem}}]$ of the unreliable memory noise $\mathit{\delta }{\widehat{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}^{\mathbf{mem}}$ introduced in (15).

Assuming $\mathbb{E}\left[\gamma \right]\ll Var\left[\gamma \right]$, as discussed in Section 2.3, the variance $Var\left[\gamma \right]={\sigma }_{\gamma }^2$ of the memory noise $\gamma$ can be approximated by the MSE as ${\sigma }_{\gamma }^2\approx \mathbb{E}[{(\tilde{x}-\widehat{x})}^2]$. The value of $\mathbb{E}[{(\tilde{x}-\widehat{x})}^2]$ depends on the error probabilities $p_b$ as well as on the probability distributions of the variables x, which are stored in memory. However, from ([8], Claim 17), if $p_{n-1}\ll \frac{1}{2}$ or $Pr\left({\widehat{x}}_b={\widehat{x}}_{b^{\prime }}\right)\simeq Pr\left({\widehat{x}}_b\ne {\widehat{x}}_{b^{\prime }}\right)$ for any $b\ne b^{\prime }$, then the MSE $\mathbb{E}[{(\tilde{x}-\widehat{x})}^2]$ can be approximated as

$${\sigma }_{\gamma }^2=\mathbb{E}[{(\tilde{x}-\widehat{x})}^2]\approx \sum _{b=-m}^{n-1}{4}^b{p}_b=\sum _{b=-m}^{n-1}{4}^b{e}^{-e_{b}a},$$

(26)

where the last equality is obtained from the noise-versus-energy model (9). Therefore, the probability distributions of the variables x have no significant impact on the value of the MSE.

Equation (26) gives us a relation between the noise variance ${\sigma }_{\gamma }^2$ and the vector $\mathit{e}$ of energy levels defined in (10). Moreover, by using (26), we show that the covariance $\mathbf{\Gamma }$ of the memory noise vector $\mathit{\delta }{\widehat{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}^{\mathbf{mem}}$ is given by

$$\mathbf{\Gamma }={\mathit{I}}_c{\sigma }_{\gamma }^2.$$

(27)

3.4. Total Error

After separately studying the two error terms $\mathit{\delta }{\mathit{x}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}^{\mathbf{quant}}$ and $\mathit{\delta }{\widehat{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}^{\mathbf{mem}}$, we now combine them to get an expression of the total estimation error ${\mathit{e}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}^{\mathbf{*}}={\tilde{\mathit{x}}}_{k+1|k+1}-{\mathit{x}}_{k+1}$. We then provide the covariance matrix ${\mathit{P}}_{k+1|k+1}^{\mathbf{*}}$ of this total error.

By using ${\tilde{\mathit{x}}}_{k|k}$ to denote the faulty estimate of ${\mathit{x}}_k$, we can express

$${\tilde{\mathit{x}}}_{k|k}={\widehat{\mathit{x}}}_{k|k}+\mathbf{\Delta }{\tilde{\mathit{x}}}_{\mathit{k}|\mathit{k}}.$$

(28)

Considering that only the ${\widehat{\mathit{x}}}_{k|k}$ are stored in the unreliable memories, the error propagation model (15) can be rewritten as

$$\begin{array}{cc}\hfill \mathbf{\Delta }{\tilde{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}=& {\mathit{D}}_{k+1}\mathbf{\Delta }{\tilde{\mathit{x}}}_{\mathit{k}\mathbf{|}\mathit{k}}+\mathit{\delta }{\widehat{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}^{\mathbf{quant}}+\mathit{\delta }{\widehat{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}^{\mathbf{mem}}\hfill \end{array}$$

(29)

$$\begin{array}{cc}\hfill \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}=& {\mathit{D}}_{k+1}\mathbf{\Delta }{\tilde{\mathit{x}}}_{\mathit{k}\mathbf{|}\mathit{k}}+{\mathit{\delta }}_{{\mathit{D}}_{\mathit{k}\mathbf{+}\mathbf{1}}}{\tilde{\mathit{x}}}_{k|k}+{\mathit{D}}_{k+1}{\mathit{ϵ}}_{x_{k|k}}\hfill \\ \hfill & +{\mathit{\delta }}_{{\mathit{K}}_{\mathit{k}\mathbf{+}\mathbf{1}}}{\mathit{y}}_{k+1}+{\mathit{K}}_{k+1}{\mathit{ϵ}}_{{\mathit{y}}_{\mathit{k}\mathbf{+}\mathbf{1}}}\hfill \\ \hfill & +{\mathit{ϵ}}_{\times }+\mathit{\delta }{\widehat{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}^{\mathbf{mem}}\hfill \end{array}$$

(30)

$$\begin{array}{cc}\hfill \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} =& ({\mathit{D}}_{k+1}+{\mathit{\delta }}_{{\mathit{D}}_{\mathit{k}\mathbf{+}\mathbf{1}}})\mathbf{\Delta }{\tilde{\mathit{x}}}_{\mathit{k}\mathbf{|}\mathit{k}}+{\mathit{\delta }}_{{\mathit{D}}_{\mathit{k}\mathbf{+}\mathbf{1}}}{\widehat{\mathit{x}}}_{k|k}\hfill \\ \hfill & +{\mathit{D}}_{k+1}{\mathit{ϵ}}_{x_{k|k}}+{\mathit{\delta }}_{{\mathit{K}}_{\mathit{k}\mathbf{+}\mathbf{1}}}{\mathit{y}}_{k+1}+{\mathit{K}}_{k+1}{\mathit{ϵ}}_{{\mathit{y}}_{\mathit{k}\mathbf{+}\mathbf{1}}}\hfill \\ \hfill & +{\mathit{ϵ}}_{\times }+\mathit{\delta }{\widehat{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}^{\mathbf{mem}},\hfill \end{array}$$

(31)

where (30) is obtained by replacing $\mathit{\delta }{\widehat{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}^{\mathbf{quant}}$ by its expression (23), and (31) comes from (28), which allows u to write ${\tilde{\mathit{x}}}_{k|k}={\widehat{\mathit{x}}}_{k|k}+\mathbf{\Delta }{\tilde{\mathit{x}}}_{\mathit{k}\mathbf{|}\mathit{k}}$. Equation (31) provides a recursive form of the error at step k, since $\mathbf{\Delta }{\tilde{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}$ depends on $\mathbf{\Delta }{\tilde{\mathit{x}}}_{\mathit{k}\mathbf{|}\mathit{k}}$ and ${\widehat{\mathit{x}}}_{k|k}$. All the other terms in (31) come from the current iteration $k+1$.

In certain conditions, such as if $\mathit{H}$ and $\mathit{F}$ are only composed of integer components, the total estimation error ${\tilde{\mathit{x}}}_{k+1|k+1}-{\mathit{x}}_{k+1}={\widehat{\mathit{x}}}_{k+1|k+1}+\mathbf{\Delta }{\tilde{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}-{\mathit{x}}_{k+1}$ can be further developed as (see Appendix A for more details):

$$\begin{array}{cc}\hfill {\tilde{\mathit{x}}}_{k+1|k+1}-{\mathit{x}}_{k+1}& =({\mathit{D}}_{k+1}+{\mathit{\delta }}_{{\mathit{D}}_{\mathit{k}\mathbf{+}\mathbf{1}}})({\tilde{\mathit{x}}}_{k|k}-{\mathit{x}}_k)\hfill \end{array}$$

(32)

$$\begin{array}{cc}\hfill & \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} +({\mathit{K}}_{k+1}+{\mathit{\delta }}_{{\mathit{K}}_{\mathit{k}\mathbf{+}\mathbf{1}}}){\mathit{v}}_{k+1}+(({\mathit{K}}_{k+1}+{\mathit{\delta }}_{{\mathit{K}}_{\mathit{k}\mathbf{+}\mathbf{1}}})\mathit{H}-\mathbf{I}){\mathit{u}}_k\hfill \end{array}$$

(33)

$$\begin{array}{cc}\hfill & \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}+{\mathit{D}}_{k+1}{\mathit{ϵ}}_{x_{k|k}}+{\mathit{K}}_{k+1}{\mathit{ϵ}}_{{\mathit{y}}_{\mathit{k}\mathbf{+}\mathbf{1}}}+{\mathit{ϵ}}_{\times }+\mathit{\delta }{\widehat{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}^{\mathbf{mem}}.\hfill \end{array}$$

(34)

This equation gives us a recursive form of the total estimation error ${\tilde{\mathit{x}}}_{k+1|k+1}-{\mathit{x}}_{k+1}$ at step $k+1$, depending on the estimation error ${\tilde{\mathit{x}}}_{k|k}-{\mathit{x}}_k$ at step k and on the quantization resolution $2^{-m}$ and the memory noise $\mathit{\delta }{\widehat{\mathit{x}}}_{\mathit{k}\mathbf{+}\mathbf{1}\mathbf{|}\mathit{k}\mathbf{+}\mathbf{1}}^{\mathbf{mem}}$.

Finally, we can compute the covariance matrix ${\mathit{P}}_{k+1|k+1}^{\mathbf{*}}$ of this error as

$$\begin{array}{cc}\hfill {\mathit{P}}_{k+1|k+1}^{\mathbf{*}}& =\mathrm{Cov}\left[{\tilde{\mathit{x}}}_{k+1|k+1}-{\mathit{x}}_{k+1}\right]\hfill \\ & =({\mathit{D}}_{k+1}+{\mathit{\delta }}_{{\mathit{D}}_{\mathit{k}\mathbf{+}\mathbf{1}}}){\mathit{P}}_{k|k}^{*}{({\mathit{D}}_{k+1}+{\mathit{\delta }}_{{\mathit{D}}_{\mathit{k}\mathbf{+}\mathbf{1}}})}^{\top }\hfill \\ & +({\mathit{K}}_{k+1}+{\mathit{\delta }}_{{\mathit{K}}_{\mathit{k}\mathbf{+}\mathbf{1}}})\mathit{R}{({\mathit{K}}_{k+1}+{\mathit{\delta }}_{{\mathit{K}}_{\mathit{k}\mathbf{+}\mathbf{1}}})}^{\top }\hfill \\ & +(({\mathit{K}}_{k+1}+{\mathit{\delta }}_{{\mathit{K}}_{\mathit{k}\mathbf{+}\mathbf{1}}})\mathit{H}-\mathbf{I})\mathit{Q}{(({\mathit{K}}_{k+1}+{\mathit{\delta }}_{{\mathit{K}}_{\mathit{k}\mathbf{+}\mathbf{1}}})\mathit{H}-\mathbf{I})}^{\top }\hfill \\ & +{\mathit{D}}_{k+1}\mathrm{Cov}[{\mathit{ϵ}}_{x_{k|k}}]{\mathit{D}}_{k+1}^{\top }+{\mathit{K}}_{k+1}\mathrm{Cov}[{\mathit{ϵ}}_{{\mathit{y}}_{\mathit{k}\mathbf{+}\mathbf{1}}}]{\mathit{K}}_{k+1}^{\top }+{\mathbf{\Sigma }}_{\times }+\mathbf{\Gamma },\hfill \end{array}$$

(35)

where all the terms involved, including the covariance matrices, have been explicited in the previous sections. Equation (35) shows that the covariance matrix ${\mathit{P}}_{k+1|k+1}^{\mathbf{*}}$ can be computed recursively.

Equation (35) provides us a measure of the performance of the filter, depending on the quantization resolution and on the energy supplied to the memory. Equipped with this derivation, we can now use the covariance matrix ${\mathit{P}}_{k+1|k+1}^{\mathbf{*}}$ as a performance criterion against which to optimize the energy consumed of the unreliable memory.

4. Energy Optimization

In this section, we optimize the energy consumption of the memory while satisfying a performance constraint defined on the total estimation error of the filter. As parameters to optimize, we consider the number of bits m for the quantization, and the energy vector $\mathit{e}$ of the memory banks. We define two optimization problems, which both seek to minimize the energy consumed by the memory. In the first problem, we find the optimal number of bits m and the corresponding $n+m$ levels of energy to allocate to the memory banks. Although solving this problem provides the minimum energy that needs to be supplied to the memory, it is not very practical since each of the $n+m$ bits should be stored in a different memory bank with a specific voltage supply.

Therefore, in the second problem, we consider that the number of bits m is fixed but that the number of possible energy levels is limited to L possibilities. Both the L energy values and the allocation of each bit to one of the L possible values should be optimized. Solving this problem allows to consider only $L<n+m$ different memory banks.

4.1. Optimization across All the Bits

We first find the optimal level of energy $e_b$ of each memory bank and the optimal number of fractional bits m to minimize the total memory energy consumption. As performance criterion, we consider the covariance matrix ${\mathit{P}}_{N|N}^{*}$ of the total estimation error at step N, where N is chosen to be large enough so that the filter can converge. We further introduce a matrix $\mathcal{V}$ of the same size as ${\mathit{P}}_{N|N}^{*}$ to define the performance constraint for the variances and covariances of estimation error on each component. The optimization problem is then defined as follows:

$$\begin{array}{cc}\hfill \underset{\mathbf{e},m}{min}& e_{\mathrm{tot}}=\sum _{b=-m}^{n-1}{e}_b={\mathbb{𝟙}}^{\top }\mathit{e},\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& {\mathit{P}}_{N|N}^{*}\prec \mathcal{V}\mathrm{and}{e}_b\ge e_{\mathrm{thres}}\forall b\in 〚-m,n-1〛,\hfill \end{array}$$

(36)

where ≺ is a component-wise inequality between the two matrices and where the minimum is taken over all energy vectors $\mathit{e}$ as defined in (10) and for all the possible values of the number of bits m. We consider that $m\in 〚0,M〛$, where M is the maximum number of bits, which could be stored in a memory. The value $e_{\mathrm{thres}}$ is the minimum level of energy for each memory bank to avoid undesired effects, such as circuit delays and energy leakage [7].

Problem (36) involves one discrete parameter m and $m+n$ continuous parameters $\mathit{e}$, which makes it difficult to solve at once. As a first step, we assume that the value of m is fixed and solve the following simplified problem:

$$\begin{array}{cc}\hfill \underset{\mathbf{e}}{min}& e_{\mathrm{tot}}=\sum _{b=-m}^{n-1}{e}_b={\mathbb{𝟙}}^{\top }\mathit{e},\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& {\mathit{P}}_{N|N}^{*}\prec \mathcal{V}\mathrm{and}{e}_b\ge e_{\mathrm{thres}}\forall b\in 〚-m,n-1〛,\hfill \end{array}$$

(37)

by using the Karush–Kuhn–Tucker (KKT) conditions (see Appendix B). From these conditions, we show that the optimal energy level $e_b^{*}$ for bit b has expression

$$e_b^{*}=\left\{\begin{array}{cc}{e}_{\mathrm{thres}},\hfill & \mathrm{if}\lambda <\frac{1}{4^{b}a},\hfill \\ \frac{1}{a}log\left(4^{b}a\lambda \right),\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

(38)

where $\lambda$ is a dual variable. It allows balancing a trade-off between preserving the performance of the system and reducing the energy consumption. A water-filling algorithm [8] can be used to compute the optimal vector ${\mathit{e}}^{*}$ for a fixed desired performance $\mathcal{V}$ of the filter. We can observe that, according to this optimal solution, the energy of the least significant bits will be set to the threshold energy level $e_{\mathrm{thres}}$. The energy levels then increase logarithmically for each bit as their significance increases.

Since m is discrete, the optimal solution (38) is computed using the water-filling algorithm for each possible value of m. We then retain the solution $(m^{\star },{\mathit{e}}^{\star })$, which gives the lowest total energy $e_{\mathrm{tot}}^{\star }={\sum }_{b=-m^{\star }}^n{e}_b^{\star }$. In this method, the influence of the quantization error is taken into account through the performance criterion ${\mathit{P}}_{N|N}^{*}$. For a small number of bits m, quantization errors may make it impossible to satisfy the desired performance constraint, and therefore the water-filling algorithm will not be able to find an optimal solution. In this case, if we detect that the algorithm converges toward a performance value that is still higher than the constraint, the algorithm is stopped, and we proceed to the next value of m in the considered range.

The full optimization process is summarized in Algorithm 1. In this algorithm, the parameter $\beta$ controls the rate at which the energy for each memory bank is increased at each iteration. The value of $\beta$ is chosen either using the precision with which energy can be set in a given device technology, or based on the desired rate of convergence for the water-filling algorithm.

The condition $∥{\mathit{P}}_{\mathrm{prev}}-{\mathit{P}}_{N|N}^{*}∥>\xi$ is used to detect whether the water-filling algorithm has a feasible solution, and thus the value of $\xi$ is set to be low. The computation of ${\mathit{P}}_{N|N}^{*}(\mathit{e},m)$ accounts for most of the computing time of this algorithm. The total run time thus depends on the number of iterations required by the water-filling algorithm (while loop in Algorithm 1). For fixed values of $\beta$ and $\mathcal{V}$, we expect the number of iterations to increase with m.

Algorithm 1: Computing the optimal values for $\mathit{e}$ and m.

Input: $\mathcal{V}$, a, $\beta$, $\xi$, $e_{\mathrm{thres}}$

${\mathit{e}}_{\min }$$\stackrel{}{\leftarrow }$$+\infty$

Result: Optimal number of bits $m_{\mathrm{opti}}$ and optimal energy allocation vector ${\mathit{e}}_{\min }$

4.2. Optimization with a Limited Number of Energy Levels

In practice, the solution of Problem 1 makes the implementation costly as each bit position should be stored in a separate memory bank. Therefore, we define a second optimization problem with only $L<m+n$ possible levels of energy. For implementation purposes, we only consider small values for L ($L<10$). The vector $\mathit{f}=[f_0,\dots ,f_{L-1}]$ contains the L levels of energy. We use $n_{\ell }$ to denote the number of bits allocated to energy level $f_l$, so that ${\sum }_{\ell =0}^{L-1}{n}_{\ell }=n+m$. This means that each memory bank of the energy group ℓ has an energy level $e_b=\frac{f_{\ell }}{n_{\ell }}$. We write $\mathit{n}=[n_0,\dots ,n_{L-1}]$ for the vector containing the L values $n_{\ell }$.

In the following, for simplicity, we consider that the number of bits n and m are fixed, and we seek to optimize the total energy consumption of the unreliable memory for a fixed number of energy levels L. The objective is to reduce the total energy consumed by the unreliable memory by allocating different levels of energy to the L groups of bits. Two parameters are considered in this optimization: the values of each energy level $e_{\ell }$ and the number of bits allocated to each of these energy levels $n_{\ell }$. The optimization problem can be written as

$$\begin{array}{cc}\hfill \underset{\mathit{f},\mathit{n}}{min}& e_{\mathrm{tot}}=\sum _{l=0}^{L-1}{f}_{\ell }={\mathbb{𝟙}}^{\top },\mathit{f}\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& {\mathit{P}}_{N|N}^{*}\prec \mathcal{V}\mathrm{and}\hspace{1em}\hspace{1em}\hspace{1em}\hfill & \hfill f_{\ell }\ge e_{\mathrm{thres}}{n}_{\ell },n_{\ell }\in 〚1,n+m-L〛\forall \ell \in 〚0,L-1〛.\end{array}$$

(39)

First, we solve the optimization problem in the case where we know which bit is allocated to which energy level. This means that the values of $n_{\ell }$ are known and that we only want to compute the optimal values of the energy levels $f_{\ell }$. In this case, the optimization problem can be written as

$$\begin{array}{cc}\hfill \underset{\mathit{f}}{min}& e_{\mathrm{tot}}=\sum _{l=0}^{L-1}{f}_{\ell }={\mathbb{𝟙}}^{\top }\mathit{f},\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& {\mathit{P}}_{N|N}^{*}\prec \mathcal{V}\mathrm{and}\hspace{1em}{f}_{\ell }\ge e_{\mathrm{thres}}{n}_{\ell }\forall \ell \in 〚0,L-1〛.\hfill \end{array}$$

(40)

Problem (40) is quite similar to the one described in Section 4.1 and can be solved using the same method as the one presented in Appendix B, by relying on the KKT conditions. The optimal solution in this case is

$$f_l^{*}=\left\{\begin{array}{cc}{e}_{\mathrm{thres}}{n}_{\ell },\hfill & \mathrm{if}\lambda <\frac{1}{{\sum }_{b=0}^{n_{\ell }}{4}^{b}a},\hfill \\ \frac{1}{a}log\left(\sum _{b=0}^{n_{\ell }}{4}^{b}a\lambda \right),\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(41)

This solution allows us to compute the optimal energy levels $f_{\ell }$ for a given energy allocation across the bits. The second step consists of computing the best allocation of bits to each energy group. Given that we only consider small values of L, we compute the optimal solution from (41) for each possible energy allocation of the bits. Then, the solution with the smallest total energy ${\sum }_{\ell =0}^{L-1}{f}_{\ell }$ is retained. Although Problem 2 leads to a more practical solution, it is expected that the optimal total energy of the memory is higher for Problem 2 than for Problem 1.

5. Simulation Results

In our simulations, unless explicitly stated, we consider a simple tracking problem where the state vector $\mathit{x}$ is composed of two variables representing the position and velocity of an object. Measurements y only consist of noisy observations of the position of the object. The process matrix $\mathit{F}$ and measurement matrix $\mathit{H}$ are defined as

$$\mathit{F}=\left[\begin{array}{cc}1& {\delta }_t\\ 0& 1\end{array}\right],\mathit{H}=\left[\begin{array}{cc}1& 0\end{array}\right],$$

(42)

and the process noise covariance matrix $\mathit{Q}$ and measurement covariance matrix $\mathit{R}$ are given by

$$\mathit{Q}=\left[\begin{array}{cc}{\sigma }_x^2& 0\\ 0& {\sigma }_x^2\end{array}\right],\mathit{R}=\left[\begin{array}{c}{\sigma }_y^2\end{array}\right].$$

(43)

where ${\delta }_t=1$ and ${\sigma }_x=0.01$ and ${\sigma }_y=10$. The factor a in (9) is taken as $a=12.8$ as in [13]. This section is divided into two parts. We first evaluate the accuracy of the proposed theoretical analysis, and we then provide solutions to the two considered optimization problems.

5.1. Accuracy of the Theoretical Analysis

First, to evaluate the accuracy of the proposed theoretical analysis, we perform Monte Carlo simulations ($N_{mc}={10}^7$) and measure the covariance matrix of the error on the estimation at step $N=250$, thus giving enough time for the filter to converge in normal conditions. This covariance matrix is compared with the theoretical expression of the covariance of the estimation error ${\mathit{P}}_{N|N}^{\mathbf{*}}$ computed in Section 3.4. Figure 1 shows the variance of the estimation errors on the position and the velocity for different values of m in the case of a reliable memory, meaning that we consider only the quantization error and not the memory noise.

We observe that the theoretical predictions of the errors closely match the simulations, which shows the accuracy of our theoretical analysis. From Figure 1, we can observe that, for a small number of bits m, the quantization error is large and dominates the total estimation error. However, starting from $m=10$ bits, the estimation error reaches a constant level, which can be interpreted as a minimum bound on the estimation error that one can obtain from a standard Kalman filter for this tracking problem. This shows that, from $m=10$ bits, the quantization errors become negligible compared to the estimation error achieved by the standard full precision Kalman filter. Thus, at this point, using more bits will not result in minimizing the estimation error, justifying the need for optimizing the parameter m.

In a second step, we introduce the memory noise in addition to the quantization. Figure 2 shows the variance of the estimation error on the position depending on the total energy $e_{\mathrm{tot}}$ for different values of m. The variance values were obtained from both Monte Carlo simulations and from the theoretical analysis of Section 3.4. Further, Figure 3 shows the variance of estimation error on the position depending on the number of bits m for different values of the total energy $e_{\mathrm{tot}}$. The comparison between theoretical results and Monte Carlo simulations show the accuracy of the theoretical analysis that predicts the new computed covariance ${\mathit{P}}_{k|k}^{\mathbf{*}}$.

From Figure 2, we can also see that both the number of bits and the total energy can affect the variance of the estimation error. If the number of bits or the energy supplied is too low, then the quantization error or the memory noise will dominate the total estimation error. However, we can see that there is a minimum number of bits, around $m=12$, from which, given enough energy, it will be possible to reach the minimum possible variance of estimation error.

Moreover from Figure 3, we can see that for a low value of supplied energy per variable, the variance of the estimation error will increase with the number of bits as there is too little energy. However, for a larger amount of energy $e_{\mathrm{tot}}>10$, the variance of the estimation error will decrease with the number of bits since the quantization error decreases. Finally for a large enough number of bits, the total estimation error will only depend on the total energy and on the variance of the estimation error of a reliable full precision filter.

As the work presented in this paper was conducted to reduce the energy consumption of the memory of a Kalman filter, it is of a greater utility when the memory is large. For this reason, the results presented before were also tested on a larger Kalman filter with a dimension for the state vector x of $c=20$. For simulations on this large-size example, we use a state transition model that performs a shifting of the entries of the state to the next state at each iteration, such as the one used in [50]. That is,

$${\mathit{F}}_{i,j}=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}i-j=1,\hfill \\ 0,\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

(44)

and ${\mathit{F}}_{c,1}=1$. The initial state vector is drawn from a normal distribution.

In this case, the performance of the filter is measured by the trace of the covariance matrix ${\mathit{P}}_{N|N}$. The results in Figure 4 and Figure 5 show that the same conclusion can be taken from the simulations done on the small-size Kalman filter and that the method presented in this paper can therefore be applied to large-size filters.

5.2. Solutions to the Optimization Problems

We now focus on the optimization problems introduced in Section 4, starting with the first one. Figure 6 shows the amount of energy $e_{\mathrm{tot}}$ needed to store each number in the unreliable memory to achieve a fixed variance of estimation error on the position for each value of m.

The total energy $e_{\mathrm{tot}}$ was calculated both using the optimal allocation from Algorithm 1 and using a uniform energy allocation. From Figure 7, we can see that the total energy $e_{\mathrm{tot}}$ of the memory slightly increases with the number of bits $m+n$. The slight increase in memory consumption is due to the form of the optimal solution (38). Indeed, once the minimum number of bits needed to achieve the performance constraint is reached, then additional bits will be set at the minimum energy threshold $e_{\mathrm{thres}}$.

Figure 7 compares the optimal solution from Figure 6 with a uniform energy allocation. This shows that the optimal energy allocation allows for a significant energy gain compared to the uniform allocation. Here, for the minimum number of bits needed to achieve the performance constraint, the optimal allocation require 56% less energy than the uniform allocation.

We now focus on the second optimization problem defined in Section 4.2 at Equation (40) where only a limited number of energy levels are available. Figure 8 shows the total energy needed for each variable in memory to achieve a fixed level of error depending on the number L of energy level possible. For each considered number of levels $L\in 〚1,7〛$, the total energy $e_{\mathrm{tot}}$ was computed for all possible energy allocations using the optimal solution (41). The minimum energy possible for each value of L was then kept and is shown in Figure 8. This minimum energy is compared with the minimum energy needed for Problem (36) where there are as many energy levels possible as the number of bits. Here, the total number of quantization bits is $B=20$.

We observe that even a small number of energy levels L can lead to significant gains in energy. In this case, only seven levels of energy allow achieving 95% of the maximum energy gain that was obtained in the first optimization problem. When looking at the optimal energy allocation for each value of L bit by bit, we notice that, in most cases, the optimal solution seems to be when the energy levels are uniformly shared between the bits. This means that, if there are B bits and L levels available and L is a divisor of B, then each group of bits assigned to each energy level will have a size of $n_{\ell }=\frac{B}{L}$.

6. Conclusions

In this paper, we studied a quantized Kalman filter implemented with unreliable memories. We provided analytical expressions for the covariance matrix of the estimation error and provided updated filter equations to take into account all considered sources of errors. We proposed and solved two optimization problems that allowed us to find the best trade-offs between the energy consumption and the performance of the filter. The simulation results showed the accuracy of the theoretical analysis and illustrated the significant energy gains provided by our approach.

Due to the generic nature of the considered error propagation model, these results could be used for various realistic noise-versus-energy models of unreliable components. Furthermore, the methodology presented in this work could also be extended to other algorithms where sources of unreliability can be introduced, such as belief propagation [51], binary recursive estimation [10], and multi-agent systems [52].