Exploring Kalman Filtering Applications for Enhancing Artificial Neural Network Learning

Abstract

Kalman filter is a widely used estimation algorithm with numerous applications, including parameter estimation, classification, prediction, pattern recognition, tuning, and filtering. Recently, it has gained attention in artificial intelligence and machine learning as a mathematical framework for the learning process. As a methodology designed for stochastic environments, the Kalman filter effectively manages noise and unstructured data with incomplete information while preventing premature stagnation, enabling faster learning and reducing the need for extensive pre-processing. These characteristics make it ideal for training artificial neural networks and other machine learning techniques. Given its significance, this paper presents a review of Kalman filter applications for artificial neural network learning.

1. Introduction

Working with physical signals, regardless of their context and application, requires the use of underlying techniques to ensure they are properly characterized before processing. This enables the extraction of useful information despite the presence of noise [1], incomplete information [2], stochastic environments [3], control constraints [4], and undesirable components [5]. Over time, different disciplines have made significant contributions to this problem, leading to significant advancements in signal characterization techniques, the scope of problems addressed, and their applications.

The history of filtering has evolved continuously from basic electrical circuits to sophisticated digital and statistical methods. The development of filtering approaches has been primarily driven by the demands of communication systems, control theory, and signal and data processing. From pre-20th-century analog filters to the latest methodologies based on artificial intelligence and machine learning, filtering has become a fundamental tool in engineering, robotics, telecommunications, and data science. As technology advances, filtering will continue to evolve to address increasingly complex problems in dynamic, noisy, unstructured, nonlinear, and non-stationary environments [5].

Early references to filtering appear in works related to Fourier analysis before the 20th century. In 1809, Gauss proposed an optimal filter called the least squares method to determine the trajectory of celestial bodies. This method demonstrated several key advancements, such as not requiring prior knowledge of the signals, making it widely applicable across many scientific fields. According to the literature, this methodology remains a fundamental part for both linear and statistical filtering, playing a crucial role in the foundations of modern filters. It is important to highlight the role of analog filters in the early stages of telegraph communication, where they were used to suppress undesirable frequencies and reduce noise in electrical circuits. Their development continued into the early 20th century, with the introduction of low-pass, high-pass, and band-pass filters, which facilitated more complex applications and paved the way for modern filtering. A major milestone occurred with the advent of digital signal processing in the first half of the 20th century, which led to the formal development of digital filtering methods. As digital filtering advanced in development and the increasing use of digital signals in communications and control systems, researchers began developing more advanced digital filtering techniques. This progress, along with the increasing complexity of applications in areas such as electrical circuits [2], control improvement [6] sophisticated communication systems [7], aeronautics [8], military [9], naval [10], and transportation [9], required solutions that met the evolving demands of technology.

In its early stages, the development of control systems was heavily influenced by filtering techniques, initially through analog filters and later by digital filters, which became essential for implementations enabled by emerging computing technologies. However, at the end of the 19th century, Poincaré, with his seminal work on the “New Methods of Celestial Mechanics” [11], recognized the need to formulate a general theory of dynamic systems based on sets of first-order differential equations. He introduced the now fundamental concept of considering a relevant set of system variables as a trajectory of a point in an n-dimensional space. This approach quickly gained popularity and became known as the state-space method. Thus, the concept of state became dominant in the study of dynamic systems. A critical aspect in this methodology is that your current behavior is influenced by its past history, meaning that the behavior of the system cannot be specified simply as an instantaneous relationship between sets of input and output variables. An additional set of variables, known as state variables, is needed to account for the history of the system, which represents the minimum amount of information necessary to summarize the entire dynamic past of the system. These state variables provide all the information needed to predict the future behavior of the system in response to any input signal. The use of the state-space methodology for control systems design emerged by the mid-20th century; since then, this concept has allowed scientists to formally describe dynamic systems in order to manipulate their behavior through an appropriate controller design. State-space control and optimal control theory marked the beginning of what is known as modern control theory, with wide applications in aerospace technology, robotics, communications, energy manufacturing, and transportation, to name a few. The role of Rudolf Kalman in modern control is of great importance, particularly through the formalization of two key concepts: controllability and observability [12]. The first concept establishes the conditions required to manipulate system behavior according to pre-established operating conditions, while the second concept defines the conditions required to model the evolution of internal variables based on the measurement of inputs and outputs for a given time. Both conditions, along with a third concept known as stability, are essential for the implementation and proper performance of modern control systems. Their development laid the foundation for optimal control techniques and state-estimation methods, which remain fundamental in the design of modern control systems [13].

In 1949, Norbert Wiener [14] introduced the Wiener filter in the frequency domain, which successfully addresses the problem of linear optimal dynamic estimation in stationary stochastic process systems. Despite its importance, this method produces a large computational burden while also requiring that both the estimated signal and the measured signal satisfy stationary stochastic processes, limiting its generalization. In 1960, Rudolf Emil Kalman introduced the Kalman filter (KF) [12], which does not require that the measured signal and noise follow the assumption of a stationary stochastic process. The state equation describes the relationship between input and output, considering the signal process as the result of a linear system that is affected by Gaussian noise in both the input and system state, as well as in its measurement. The Kalman filter provides an optimal estimate in terms of minimum mean-square error for linear filters of non-stationary stochastic processes [3]. Kalman developed the filter for both continuous-time and discrete-time systems. These results represent a unified methodology between the stochastic treatment of signals and the concept of dynamic systems in state-space, achieving a balance between the concepts of modern control and statistical filtering. This filter solves the problem of optimal state estimation of a linear dynamic system in the presence of measurement noise. The Kalman filter is a recursive algorithm, which means that it can process new measurements without needing to store all previous data. This characteristic is inherited from the state-space model, which is the minimum amount of information necessary to describe the complete behavior of a system. As a result, the KF only needs to store the state of the system, as opposed to storing all past signal data, making it computationally efficient. The KF also updates its state estimate by comparing the predicted state with the actual measurement, improving the accuracy of the state estimation in real-time, expanding its use in non-stationary processes, and contributing to the advancements of optimal real-time estimation [5].

The relevance of the work made by Kalman for estimation and optimal control in state-space was fundamental for several reasons: real-time operation, recursive nature and low computational complexity, and robustness to noise and uncertainties. The KF provides an efficient solution for optimal real-time state estimation in dynamical systems. In practice, many systems are not fully observable due to sensor limitations or noisy measurements, but KF enables accurate state estimation even under incomplete and noisy information without requiring storage or reprocessing of historical system data. By assuming Gaussian-distributed system noise, the KF computes optimal estimates based on this assumption. This makes it particularly effective in systems where the dynamics are not perfectly known and noise is present in the system measurements. This is essential in applications where real-time decisions are crucial, such as robotics, aerospace, autonomous vehicles, navigation and guidance systems, biomedical engineering, energy systems, transportation, and communications, among others, especially when computational resources may be limited [5].

The development of quantum systems has expanded rapidily in recent years, and accordingly, KF has also extended. A detailed description of quantum filtering can be found in [15]. A quantum extended Kalman filter (QEKF) is presented in [16], which employs a commutative approximation and a time-varying linearization for nonlinear quantum stochastic differential equations. The quantum Kalman filter (QKF) for linear quantum systems with known parameters has been studied in [17], where an optical system containing an uncertain parameter in the laser probe is described, demonstrating better estimation compared to the classical KF. A deeper review of linear quantum KF is developed in [18]. A bibliography review regarding KF-based approaches for quantum systems has been presented in [19], where an improved method for optimal estimations is demonstrated. Their work also examines a practical scenario involving magnetic field estimations in quantum systems, where nonlinear KFs could be considered as an estimation solution. More recently, ref. [20] proposed an orbit deviation propagation approach based on a deep neural network.

In summary, KF was developed by Rudolph Kalman in 1969 as an optimal state estimator for linear dynamic systems under the presence of Gaussian noise, noisy or incomplete measurements, with uncertainties, and in non-stationary environments. Due to its recursive nature, KF ensures low computational complexity, making it widely applicable and successful in numerous real-time applications. While KF is inherently linear, this work inspired extensions for nonlinear systems, such as extended Kalman filter (EKF) and unscented Kalman filter (UKF). These extensions allow state estimation in nonlinear systems, which are common in practical control applications. Since most real-world applications involve noise, uncertainty, non-measurable signals, unreliable sensors, and time-varying conditions [21], these developments remain highly relevant. Therefore, this work contemplates a somewhat different application of KF: its use for artificial neural networks training. Although this topic is not entirely new, it has gained renewed attention in recent years due to the widespread popularity of applying artificial intelligence and machine learning techniques in complex and relevant problems.

Hence, considering all these facts, this review focuses KF in artificial neural network learning, emphasizing its ability to handle noisy and uncertain data efficiently. By reducing the need for complex preprocessing and speeding up learning, the Kalman filter offers a practical and powerful tool for improving machine learning methods. This review highlights its key advantages and contributions, showing how a classical estimation technique continues to play a vital role in advancing modern artificial intelligence.

This work is organized as follows: In Section 2, the concepts underlying the deduction of the linear Kalman filter and its consequent first-order analytical approximation (EKF) are presented, then different variants of the KF are described for both linear and nonlinear systems based on both analytical and numerical approximations, with references to several key applications in the literature. In Section 3, the training problem of neural networks is presented as an optimization problem, from which the applicability of the FK and its variants in the training of neural networks is derived, including a review of the literature corresponding to different implementations. This is exemplified in Section 4 by the development of the training of three types of widely used neural networks regarding function approximation using neural networks-multilayer perceptron neural networks, radial basis neural networks, and high-order recurrent neural networks-highlighting the elements relevant for the design of KF-based algorithms and their variants in the learning of artificial neural networks. In Section 5, the challenges, limitations, open problems, and future work related to the use of KF and its variants in the learning of neural networks are discussed. Finally, in Section 6, the conclusions of the review carried out are established.

2. Main Variants of Kalman Filter

In this section, concepts underlying the deduction of the linear Kalman filter and its consequent variants are presented, as well as their basis, evolution, and current applications. Since the proposal of the Kalman filter as the solution to the optimal estimation problem in both continuous-time and discrete-time linear dynamic systems, its applicability has been evident. The motivation and mathematical formulation of the KF were already discussed in the previous section.

In 1967 [22,23], one of the most widely used variants of KF was proposed: the extended Kalman filter (EKF). The motivation for this variant was to apply KF methodology in nonlinear systems, approximated by a first-order Taylor series. Since highly nonlinear systems are only approximated, the approximation error can lead to estimation errors. For this reason, EKF cannot be considered an optimal estimator, unlike the original KF. The EKF linearizes nonlinear functions around the current estimated state. Its motivation is justified because nonlinear systems are prevalent in real-world applications where nonlinearities arise due to the construction characteristics of the components in real-world systems, as well as the nature of the systems themselves [5]. Despite the estimation errors caused by linear approximation, this variant remains widely used due to its straightforward and clear mathematical formulation. Another drawback of this variant results from the computation of Jacobians due to the system linearization, which must be evaluated in each state-space point. This results in greater design complexity of the EKF compared to KF, in addition to an increase in computational complexity. These issues have led to various improvements to EKF, such as the robust distributed EKF [24], adaptive square root EKF [25], and lightweight EKF [26].

In 1997, Simon Julier and Jeffrey Uhlmann [8] proposed another variant of KF capable of handling nonlinearities more precisely. This motivation arose from the approximation errors caused by the linearization method used in the EKF. This new variant introduced the unscented transformation, leading to the unscented Kalman filter (UKF). The UKF has better performance for highly nonlinear systems and does not require the computation of Jacobians. However, it does require the selection of sample points in a probability distribution. Furthermore, its computational complexity is significantly higher, which is why it cannot be implemented in applications that require real-time processing [27]. To address these drawbacks, some modifications have been porposed, for example, ref. [28] embedded a new deterministic sampling point set into the UKF framework, and in [18], a new exponential attenuation factor was designed according to changes in noise variance.

In 2009, Arasaratnam and Haykin, ref. [29], introduced another variant of KF to address nonlinearities: the cubature Kalman filter (CKF). The motivation of the CKF lies in selecting a spherical volume of radial type and then using a set of generated cubature points based on sampling to approximately estimate the state of a nonlinear system.

All these KF variants have been developed primarily for nonlinear dynamic systems. These variants, as well as the original KF, are known as white-box models since their design is based on a system model described in state variables. However, for more complex applications, black box models are now commonly used. These models are often obtained experimentally through the acquisition of data representing the behavior of the system under certain conditions. For example, in [30], their use in data science is considered.

Other notable KF variants include the information filter (or inverse Kalman filter), which operates in the information space rather than the state space, using information matrices instead of covariance matrices; the infinite H filter, designed for systems with unknown uncertainties; the federated Kalman filter, developed for distributed systems; and the non-gist Kalman filter with adaptive noise, intended for highly nonlinear systems with varying probabilistic distributions. Other types of filters consist of KF with learning; these type of variants emphasizes its use in applications that require experimental modeling, that is, data-driven models as the work presented by [31] in a simulation data-driven context. In this case, two main groups can be differentiated: in the first group, KF is hybridized with machine learning techniques or artificial intelligence, mainly with artificial neural networks; while the second group involves neural networks in the iterative process of KF either. In this section, the concept’s inherent deduction of the linear Kalman filter and its consequent variants are presented, as well its basis, evolution, and current applications. Since the proposal of the Kalman filter as the solution to the optimal estimation problem in both continuous-time and discrete-time linear dynamic systems, its applicability has been evident. The motivation and mathematical formulation of the KF were already discussed in the previous section.

In 1997, Simon Julier and Jeffrey Uhlmann [8] proposed another variant of KF that is able to handle nonlinearities in a more precise way. Its motivation arises from the approximation errors caused by the linearization method used in the EKF. This new variant proposes the use of the unscented transformation yielding to the unscented Kalman filter (UKF). The UKF has better performance for highly nonlinear systems and does not require the computation of Jacobians. However, it does require the selection of sample points in a probability distribution. Furthermore, its computational complexity is significantly increased, which is why it cannot be implemented in applications that require real-time processing [27]. In this context, various authors have proposed some modifications: ref. [28] embedded a new deterministic sampling point set into the UKF framework and in [18] a new exponential attenuation factor according to change of noise variance is designed.

In 2009, Arasaratnam and Haykin, ref. [29], proposed another variant of KF to address nonlinearities, the so-called cubature Kalman filter (CKF). The motivation of the CKF lies in selecting a spherical volume of radial type and then using a set of generated cubature points based on sampling to approximately estimate the state of a nonlinear system.

All these groups of variants of KF have been developed primarily for nonlinear dynamic systems. These variants, as well as the original KF, are known as white-box models since their design is based on a system model described in state variables. However, for more complex applications, black box models are now commonly used. These models are often obtained experimentally through the acquisition of data that represents the behavior of the system under certain conditions, for example in [30], it is considered for use in data science.

Other variants of KF include the information filter (or inverse Kalman filter), which works in the information space and not in the state space through information matrices and not covariance matrices; the infinite H filter, designed for systems with unknown uncertainties; the federated Kalman filter, for distributed systems; the non-gist Kalman filter with adaptive noise, for highly nonlinear systems with varying probabilistic distributions; KF with learning, this type of variants emphasizes its use in applications that require experimental modeling, that is, data-driven models as the presented by [31] in a simulation data-driven context. In the latter case, two main groups can be differentiated: in the first group, KF hybridizes with machine learning technique or artificial intelligence, mainly artificial neural networks, while the second group involves neural networks in the iterative process of KF, either for parameter estimation or for other internal processes. In [32], similarly to the work proposed in [33], which applied neural networks for nonlinear modeling; [34], which uses neural networks tor time series prediction and [35] with a KF for parameter estimation and state prediction. Figure 1 shows the main variants of KF, described in this section.

3. Neural Network Learning

The advent of artificial neural networks dates back to the work of McCulloch and Pitts in 1943 [36], who formulated a mathematical model for a neural network based on observations of biological neuron behavior made by Santiago Ramon and Cajal in 1911 [37]. However, this model did not include any learning mechanism. It was not until Rosenblatt’s perceptron of 1958 [38] that a learning rule for an artificial neural network was established for the first time. However, this neural network was very limited in terms of its applications. In 1961, Widrow and Hoff [39] addressed the problem of learning in artificial neural networks with supervised learning as an optimization problem. Their approach expanded applications of neural networks, as well as the possible structures and topologies, marking the beginning of artificial neural networks and machine learning as we conceive them today [27,40,41].

Considering neural network learning as an optimization problem allows the use of different optimization and parametric estimation methods in training. From this perspective, the KF can be considered for neural network training, with the specific KF variant depending on the type of learning and the information available to the network [5]. In this sense, an excellent review work can be found in Haykin’s book on the use of KF in neural network training. In this work, a review of KF is made, followed by the presentation of various implementations of KF in the learning process of neural networks, most of them for offline learning, as shown in Table 1, as well as in a graphical representation in Figure 2.

Nowadays, the need to perform real-time learning has increased, which highlights the use of KF for training artificial neural networks. In addition to its recursive nature, KF reduces the need for preprocessing by working directly with noisy, uncertain, and non-stationary signals [42]. It is important to note that KF is considered a second-order gradient optimization algorithm, which reduces the probability of falling into local minima as shown in [43,44,45]. Furthermore, in [43], it is proved that the Kalman filter is a global observer for linear (discrete-time) time-varying systems, and then EKF is shown to act as a quasi-local asymptotic observer for discrete-time nonlinear systems. In [46], a backpropagation training algorithm is shown to be three orders of magnitude less computationally expensive than EFK in terms of the number of floating-point operations. However, in [47], a decoupled extended Kalman filter is proposed in order to decrease the computational effort of the learning algorithm; in its analysis, ref. [47] demonstrates the computational superiority of KF learning algorithms over gradient descent ones. Also, in [41], different approaches are proposed in order to reduce computational complexity implementations of KF variants to train neural networks offline and online, where as [48] demonstrated that KF algorithms converge in fewer iterations compared to backpropagation with the same neural network configurations. In [42], an analysis is carried out that gives insight into the convergence mechanisms, showing that with a modification of the algorithm, global convergence results can be achieved for general cases. The scheme can then be interpreted either as maximization of the likelihood estimation, or as a recursive prediction error algorithm. Additionally, ref. [49], using Lyapunov stability approach, showed that KF-based learning achieves faster convergence than traditional algorithms. Similarly, the work performed by [43] establishes specific conditions for improved convergence of KF as a learning algorithm.

This characteristic has also been exploited in machine learning to improve the performance of the Adam model, leading to the Kadam model, where KF is used to estimate the first and second moments required by the algorithm [50]. The Adam model does first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments [51], while the EKF can be used as a second-order gradient descent algorithm to estimate optimal weights for an RNN, as explained in [44,45], helping the EKF algorithm to avoid local minima.

It is worth highlighting in Table 1 the growing interest in real-time implementations. Such applications are particularly relevant for decentralized approaches as well as for embedded implementations, edge-computing approaches, and reduced communication problems. In Table 2, a comparison is included of reported processing time for different implementations, noting the suitability of EKF-based learning algorithms for online operation with small sampling times. The implementations included in Table 2 are aligned in terms of the elements considered for each scheme, allowing a meaningful comparison.

Table 1. KF variant neural network learning applications.

Author	Type of KF	Main Contribution	Learning	Examples
[52]	EKF	Real-time learning algorithm for a multilayered neural network	Online	Numerical
[53]	DEKF	Feedforward multilayered neural networks based on an EKF	Offline	Simulation
[54]	EKF	Real-time neural controller for three phase induction motors	Online	Experimental
[55]	EKF, UKF	Sequential growing-and-pruning learning algorithm	Offline	Simulation
[41]	KF, EKF, UKF, DEKF	Kalman filtering as applied to the learning and use of neural networks	Offline Online	Numerical Simulation Experimental
[5]	EKF, DEKF	Radial basis neural networks trained with an extended Kalman filter	Online	Simulation
[49]	EKF	State-space recurrent neural networks for nonlinear system identification	Offline	Simulation
[56]	KF	Q-learning with KF for action selection in cooperative control	Offline	Simulation
[57]	KF	Continuous state-space via Q-Learning for Markov decision process	Offline	Numerical
[58]	KF	Online Sequential Extreme Learning Machine and Kalman filter regression	Offline	Simulation
[59]	KF	Kalman filter with iterative learning control	Offline	Simulation
[60]	EKF	Induction motor control, combining EKF with a fuzzy logic controller	Offline	Simulation
[61]	KF	Kalman Filter and a Temporal Differencing	Offline	Simulation
[21]	EKF	Real-time neural controller for autonomous robotic navigation	Online	Experimental
[62]	KF	Kalman filter to update weights of a Single Layer Feedforward Network	Offline	Simulation
[63]	KF	Q-learning is represented in the framework of Kalman filter model	Offline	Simulation
[64]	KF	NN-based learning modules to update a Kalman filter for estimation	Offline	Simulation
[35]	DEKF	Estimation charge for lithium-ion batteries	Online	Experimental
[65]	KF	KF learning for stochastic claims reserving	Offline	Simulation
[66]	KF	Federated Kalman filters are proposed	Offline	Simulation
[67]	KF	Q-learning Approach with Kalman Filter for Self-balancing Robot	Offline	Simulation
[68]	EKF	State estimation algorithm combines the EKF and a Q-learning method	Offline	Simulation
[69]	KF	It is proposed an extreme learning Kalman filter for NN	Offline	Simulation
[70]	KF	Kalman filtering with a dedicated recurrent neural network	Online	Numerical
[71]	KF	KF filter is combined with a NN to predict the transaction throughput in a blockchain	Offline	Experimental
[72]	KF	Reinforcement learning adaptive KF for signal’s autoregressive modeling	Online	Experimental
[73]	EKF	Continuous action learning automata for tuning of Kalman filter	Offline	Experimental
[74]	KF	KF agents that operate sequentially to estimate optimal learning rate	Offline	Simulation
[75]	KF	Kalman filter-based cycle-consistent adversarial learning framework for time series	Offline	Simulation
[76]	EKF	Neural controller applied to an auxiliary energy system for electric vehicles	Online	Experimental
[77]	EKF	Real-time fault-tolerant closed-loop neural controller	Online	Experimental
[78]	KF	A neural network combined with a robust KF	Offline	Simulation

Table 1. KF variant neural network learning applications.

Author	Type of KF	Main Contribution	Learning	Examples
[52]	EKF	Real-time learning algorithm for a multilayered neural network	Online	Numerical
[53]	DEKF	Feedforward multilayered neural networks based on an EKF	Offline	Simulation
[54]	EKF	Real-time neural controller for three phase induction motors	Online	Experimental
[55]	EKF, UKF	Sequential growing-and-pruning learning algorithm	Offline	Simulation
[41]	KF, EKF, UKF, DEKF	Kalman filtering as applied to the learning and use of neural networks	Offline Online	Numerical Simulation Experimental
[5]	EKF, DEKF	Radial basis neural networks trained with an extended Kalman filter	Online	Simulation
[49]	EKF	State-space recurrent neural networks for nonlinear system identification	Offline	Simulation
[56]	KF	Q-learning with KF for action selection in cooperative control	Offline	Simulation
[57]	KF	Continuous state-space via Q-Learning for Markov decision process	Offline	Numerical
[58]	KF	Online Sequential Extreme Learning Machine and Kalman filter regression	Offline	Simulation
[59]	KF	Kalman filter with iterative learning control	Offline	Simulation
[60]	EKF	Induction motor control, combining EKF with a fuzzy logic controller	Offline	Simulation
[61]	KF	Kalman Filter and a Temporal Differencing	Offline	Simulation
[21]	EKF	Real-time neural controller for autonomous robotic navigation	Online	Experimental
[62]	KF	Kalman filter to update weights of a Single Layer Feedforward Network	Offline	Simulation
[63]	KF	Q-learning is represented in the framework of Kalman filter model	Offline	Simulation
[64]	KF	NN-based learning modules to update a Kalman filter for estimation	Offline	Simulation
[35]	DEKF	Estimation charge for lithium-ion batteries	Online	Experimental
[65]	KF	KF learning for stochastic claims reserving	Offline	Simulation
[66]	KF	Federated Kalman filters are proposed	Offline	Simulation
[67]	KF	Q-learning Approach with Kalman Filter for Self-balancing Robot	Offline	Simulation
[68]	EKF	State estimation algorithm combines the EKF and a Q-learning method	Offline	Simulation
[69]	KF	It is proposed an extreme learning Kalman filter for NN	Offline	Simulation
[70]	KF	Kalman filtering with a dedicated recurrent neural network	Online	Numerical
[71]	KF	KF filter is combined with a NN to predict the transaction throughput in a blockchain	Offline	Experimental
[72]	KF	Reinforcement learning adaptive KF for signal’s autoregressive modeling	Online	Experimental
[73]	EKF	Continuous action learning automata for tuning of Kalman filter	Offline	Experimental
[74]	KF	KF agents that operate sequentially to estimate optimal learning rate	Offline	Simulation
[75]	KF	Kalman filter-based cycle-consistent adversarial learning framework for time series	Offline	Simulation
[76]	EKF	Neural controller applied to an auxiliary energy system for electric vehicles	Online	Experimental
[77]	EKF	Real-time fault-tolerant closed-loop neural controller	Online	Experimental
[78]	KF	A neural network combined with a robust KF	Offline	Simulation

Table 2. Comparison for real-time implementation of neural network learning-based algorithms.

Work	Application	Processing Hardware	Processing Time
[54]	Three-phase induction motor	DSP-DS1104	1 ms
[79]	Mobile robot	FPGA Cyclone IV, DE2-115	14 μs
[21]	Mobile robot	DS1104	1 ms
[80]	Smart grid	LAUNCHXL-F28379D	0.5 ms
[77]	Three-phase induction motor	DS1104	1 ms

Table 2. Comparison for real-time implementation of neural network learning-based algorithms.

Work	Application	Processing Hardware	Processing Time
[54]	Three-phase induction motor	DSP-DS1104	1 ms
[79]	Mobile robot	FPGA Cyclone IV, DE2-115	14 μs
[21]	Mobile robot	DS1104	1 ms
[80]	Smart grid	LAUNCHXL-F28379D	0.5 ms
[77]	Three-phase induction motor	DS1104	1 ms

Lastly, KF has a rigorous stability analysis which allows its initialization to be established according to analytically defined stability and convergence conditions [41,43]. In this sense, the literature shows a large number of successful implementations of KF for training neural networks applied mainly to classification, control [6,22,23,54,60], energy [25,35,81,82], estimation [26,69,82,83], forecasting [65,74,84,85], robotics [9,21,56,67], and NN training as presented in Table 1; as well as its combinations, these applications and their interrelationships are depicted in Figure 2. Therefore, in the next section, a neural learning solution from an optimization point of view is described and a solution obtained from the KF context is analyzed.

4. Kalman Filter for Neural Network Learning

In this section, the use of the Kalman filter for neural network learning is introduced as an application of its properties as a state estimator. This approach has been used by several authors to different types of neural networks. In this work, only Multilayer perceptron (MLP), radial basis, and recurrent high order neural networks (RHONN) are considered, whose learning processes are solved with the KF approach. In the literature, other similar learning approaches for other types of neural networks can be found; however, three types of neural networks considered in this work have been experimentally implemented in real time, as shown in Table 2 [5,21,54,76,77].

As previously explained, KF has been formulated for a linear dynamic system in state-space to provide a solution to the linear optimal filtering problem. This solution applies to both stationary and non-stationary environments. Also, as mentioned before, the solution is recursive, meaning each update of the estimated state is calculated using the previous estimate and new input data, requiring only the previous estimate to be stored. This also implies that storing all past data is not necessary. Let us now consider a linear dynamical system in discrete time, as depicted in Figure 3:

4.1. Concepts Prior to KF

Before discussing the formulation of the Kalman filter, it is important to contextualize other significant results.

Optimal Estimation

First, let us review the fundamental concepts of optimal estimation. Consider the following equation:

$$y_k=w_k+v_k$$

(1)

where $w_k$ is an unknown signal and $v_k$ is considered as additive Gaussian noise. ${\widehat{w}}_k$ is a posterior estimated of the signal $w_k$, $y_1,y_2,\dots ,y_k$. Typically, estimate ${\widehat{w}}_k$ is different from unknown signal $w_k$.

The first step for an optimization problem is to define a cost (loss) function, which must satisfy the following requirements:

• Cost function is non-negative.
• Cost function is a non-decreasing function of the estimation error, defined by:

$${\widehat{w}}_k=w_k-{\widehat{w}}_k$$

(2)

These two requirements are satisfied by the expected square error, defined by:

$$J_k=E\left\{{(w_K-{\widehat{w}}_k)}^2\right\}=E\left\{{\tilde{w}}_k^2\right\}$$

(3)

where E is the expected value operator. Cost function $J_k$ is time-dependent with sample instant k, this emphasizes the non-stationary nature of the recursive estimation process. To deduce optimal value for the estimate ${\widehat{w}}_k$, the following theorems are required [13].

Theorem 1.

Conditional expectation estimator. If stochastic processes $w_k$ and $y_k$ are considered as Gaussian, then optimal estimated ${\widehat{w}}_k$ to minimize mean square error $J_k$ is the expected value operator:

$${\widehat{w}}_k=E\left\{w_k\right|y_1,y_2,\dots ,y_k\}$$

(4)

Theorem 2.

Orthogonality principle. Let the processes $w_k$ and $y_k$ be stochastic with zero mean, such that:

$$E\left\{w_k\right\}=E\left\{y_k\right\}=0\forall k$$

(5)

Therefore:

i.

Stochastic processes $w_k$ and $y_k$ are Gaussian, or;

ii.

Optimal estimated ${\widehat{w}}_k$ is restricted to be a linear function of measures $y_1,y_2,\dots ,y_k$ and mean square error cost function.

iii.

Then, optimal estimate $\widehat{w_k}$, with measurements $y_1,y_2,\dots ,y_k$, is orthogonal to the projection of $w_k$ in generated space for such measurements.

4.2. Kalman Filter Realization

• State-space model:

  $$w(k+1)=F(k+1,k)w\left(k\right)+u\left(k\right)$$

  (6)

  $$y\left(k\right)=H\left(k\right)w\left(k\right)+v\left(k\right)$$

  (7)

  where $u\left(k\right)$ and $v\left(k\right)$ are Gaussian independent noises with zero means, where the covariance matrices are $Q\left(k\right)$ and $R\left(k\right)$, respectively.
• Initialization:

  $$\widehat{w}\left(0\right)=E\left\{w\left(0\right)\right\}$$

  (8)

  $$P\left(0\right)=E\left\{[w\left(0\right)-E\left\{w\left(0\right)\right\}]{[w\left(0\right)-E\left\{w\left(0\right)\right\}]}^T\right\}$$

  (9)
• Propagation of estimated state

  $${\widehat{w}}^{-}\left(k\right)=F(k,k-1)\widehat{w}(k-1)$$

  (10)
• Propagation of estimation error covariance

  $$P^{-}\left(k\right)=F(k,k-1)P(k-1)F^T(k,k-1)+Q(k-1)$$

  (11)
• Kalman gain matrix

  $$K\left(k\right)=P^{-}\left(k\right)H^T\left(k\right)[R\left(k\right)+H\left(k\right)P(-)\left(k\right)H^T{\left(k\right)]}^{-}1$$

  (12)
• State estimation update

  $$\widehat{w}\left(k\right)={\widehat{w}}^{-}+K\left(k\right)(y\left(k\right)-H\left(k\right){\widehat{w}}^{-}\left(k\right))$$

  (13)
• Estimation error covariance update

  $$P\left(k\right)=(I-K\left(k\right)H\left(k\right))P^{-}\left(k\right)$$

  (14)

The Kalman filter described earlier assumes a linear model of a dynamic system, as can be seen in Figure 4. However, in most cases the model is nonlinear; next, the use of KF will be extended through a linealization procedure resulting in EKF. Such an extension is feasible since KF is described in terms of differential equations in discrete-time systems.

4.3. Extended Kalman Filter

• State-space model for discrete-time nonlinear systems

  $$w(k+1)=f(k,w(k\left)\right)+u\left(k\right)$$

  (15)

  $$y\left(k\right)=h(k,w(k\left)\right)+v\left(k\right)$$

  (16)

  where $u\left(k\right)$ y $v\left(k\right)$ Gaussian independent noises with zero mean and covariance matrices $Q\left(k\right)$ and $R\left(k\right)$, respectively.
• Initialization:

  $$\widehat{w}\left(0\right)=E\left\{w\left(0\right)\right\}$$

  (17)

  $$P\left(0\right)=E\left\{[w\left(0\right)-E\left\{w\left(0\right)\right\}]{[w\left(0\right)-E\left\{w\left(0\right)\right\}]}^T\right\}$$

  (18)

The basic idea of EKF is to linearize the system model in state-space at each time instant around the most recent estimated state, which can be considered as ${\widehat{w}}_k$ or ${\widehat{w}}_k^{-}$, respectively. Once the linearized model is obtained, KF realization can be applied. Defining a discrete-time nonlinear system as:

$$F(k+1,k)={\displaystyle \frac{\partial f(k,w(k\left)\right)}{\partial w}}{|}_{w=\widehat{w}\left(k\right)}$$

(19)

$$H(k+1,k)={\displaystyle \frac{\partial h(k,w(k\left)\right)}{\partial w}}{|}_{w={\widehat{w}}^{-}\left(k\right)}$$

(20)

EKF realization is given by:

• Propagation of estimated state

  $${\widehat{w}}^{-}\left(k\right)=F(k,k-1)\widehat{w}(k-1)$$

  (21)
• Propagation of estimation error covariance

  $$P^{-}\left(k\right)=F(k,k-1)P(k-1)F^T(k,k-1)+Q(k-1)$$

  (22)
• Kalman gain matrix

  $$K\left(k\right)=P^{-}\left(k\right)H^T\left(k\right){[R\left(k\right)+H\left(k\right)P^{-}\left(k\right)H^T\left(k\right)]}^{-}1$$

  (23)
• State estimation update

  $$\widehat{w}\left(k\right)={\widehat{w}}^{-}\left(k\right)+K\left(k\right)(y\left(k\right)-H\left(k\right){\widehat{w}}^{-}\left(k\right))$$

  (24)
• Estimation error covariance update

  $$P\left(k\right)=(I-K\left(k\right)H\left(k\right))P^{-}\left(k\right)$$

  (25)

Then, following conditional expectation estimator for KF, it is possible to define:

$${\widehat{w}}_k^{-}=E\left\{w_k\right|y_1,\dots ,y_k-1\}$$

(26)

with $w_k=F_{k,k-1}{w}_{k-1}+u_{k-1}$

$$\begin{array}{cc}\hfill {\widehat{w}}_k^{-}& =E\left\{(F_{k,k-1}{w}_{k-1}+u_{k-1})\right|y_1,\dots ,y_{k-1}\}\hfill \\ \hfill & =F_{k,k-1}E\left\{w_{k-1}\right|y_1,\dots ,y_{k-1}\}+E\left\{u_{k-1}\right|y_1,\dots ,y_{k-1}\}\hfill \end{array}$$

By definition:

$$E\left\{w_{k-1}\right|y_1,\dots ,y_{k-1}\}={\widehat{w}}_{k-1}$$

(27)

$$E\left\{u_{k-1}\right|y_1,\dots ,y_{k-1}\}=E\left\{u_{k-1}\right\}=0$$

(28)

Therefore the best a priori estimate of w based on measurements is:

$${\widehat{w}}_k^{-}=F_{k,k-1}{\widehat{w}}_{k-1}$$

(29)

4.4. Relevant Results on KF for Neural Network Learning

From the previous explanation, it is easy to see that KF can be used to train artificial neural networks, as described in this section.

4.4.1. Comparison Between Kalman Filter and Recursive Least Squares Algorithm

It is well-known that most of the supervised neural networks models are trained with the recursive least squares algorithm, where the equations are defined as:

$$\begin{array}{c}\hfill P\left(k\right)=P(k-1)-P(k-1){\displaystyle \frac{\phi \left(k\right){\phi }^T\left(k\right)}{1+{\phi }^T\left(k\right)P(k-1)\phi \left(k\right)}}P(k-1)\end{array}$$

(30)

$$\widehat{\Theta }\left(k\right)=\Theta (k-1)+P\left(k\right)\phi \left(k\right)[y\left(k\right)-{\phi }^T\left(k\right)\widehat{\Theta }(k-1)]$$

(31)

On the other hand, KF equations can be written as:

$$P(k+1)=P\left(k\right)-P\left(k\right)H^T{[HP\left(k\right)H^T+R]}^{-1}HP\left(k\right)+Q$$

(32)

where $\Theta$ is a vector which contains estimated parameters and $\phi$ is a regression vector. Then with $Q=0$ the following is defined:

$$P(k+1)=P\left(k\right)-P\left(k\right)H^T{[HP\left(k\right)H^T+R]}^{-1}HP\left(k\right)$$

(33)

with $\theta (k+1)=\theta \left(k\right)$ it follows:

$$P\left(k\right)=P(k-1)-P(k-1)H^T{[HP(k-1)H^T+R]}^{-1}HP(k-1)$$

(34)

and considering $H={\Theta }^T\left(k\right)$

$$P\left(k\right)=P(k-1)-P(k-1)\phi \left(k\right){[{\phi }^T\left(k\right)P(k-1)\phi \left(k\right)+R]}^{-1}{\phi }^T\left(k\right)P(k-1)$$

(35)

Finally, with $R=1$, (1) can be obtained, which corresponds to the recursive least squares algorithm.

$$P\left(k\right)=P(k-1)-P(k-1){\displaystyle \frac{\phi \left(k\right){\phi }^T\left(k\right)}{1+{\phi }^T\left(k\right)P(k-1)\phi \left(k\right)}}P(k-1)$$

(36)

4.4.2. Retropropagation Versus EKF

The backpropagation algorithm is mostly used in learning for supervised multilayer neural networks [48], including deep learning models, which is an algorithm based on the gradient descent rule [46].

$$\Delta w_{ij}=-\eta {\displaystyle \frac{\partial E_{av}}{\partial w_{ij}}}$$

(37)

$$E_{av}={\displaystyle \frac{1}{2}}{(d_p-z_p)}^2$$

(38)

where $\eta$ is the learning rate parameter, $d_p$ is $p-th$ desired output, and $z_p$ is $p-th$ neural network (approximated) output.

$${\displaystyle \frac{\partial E_{av}}{\partial w_{ij}}}=(d_p-z_p){\displaystyle \frac{\partial z_p}{\partial w_{ij}}}$$

(39)

Then:

$$\Delta w_{ij}=\eta (d_p-z_p){\displaystyle \frac{\partial z_p}{\partial w_{ij}}}$$

(40)

In the EKF algorithm, the following is defined:

$$\Delta w=K(d_p-z_p)$$

(41)

$$K=PH{[H^{T}PH+R]}^{-1}$$

(42)

$$\Delta w=PH{[H^{T}PH+R]}^{-1}(d_p-z_p)$$

(43)

Considering that $P=PI$ and ${[H^{T}PH+R]}^{-1}=aI$ where a and p are scalar values, then, for a particular weight $w_{ij}$, $\Delta w_{ij}=ap(d_p-z_p)h_{ij}$ is defined where $h_{ij}={\displaystyle \frac{\partial z_p}{\partial w_{ij}}}$:

$$\Delta w_{ij}=ap(d_p-z_p){\displaystyle \frac{\partial z_p}{\partial w_{ij}}}$$

(44)

Note that this is the delta rule used in backpropagation algorithm with $\eta =ap$. Therefore, the backpropagation algorithm can be considered a special case of EKF. Since this algorithm is a simplified of EKF, it is possible that the backpropagation algorithm discards information that may be important during weight update [46].

4.5. Multilayer Perceptron Trained with EKF

It is well known that the MLP neural network is one of the most used neural network to deal with complex nonlinear data, traditionally trained with backpropagation-type algorithms and with optimizer-type algorithms [41]; most of their learning implementation works in an offline mode using epoch or batch approaches (see Table 1). In addition, various researchers have proposed learning algorithms for neural networks based on EKF; use of this kind of learning approaches improves learning process without need of hard preprocessing tasks. These applications are based on the training of MLP and can be interpreted as an estimation problem for a nonlinear system, which can be solved with EKF. The use of algorithms based on EKF is recursive due to the gradient descent algorithm. Recursive least squares algorithms, backpopagation algorithms, and others can be considered as specific cases of KF, as is shown in [46].

In KF application for MLP learning, the weight vector is considered as the state to be estimated by the Kalman Filter, where the neural network output is the measurement used by KF. The use of EKF is necessary since MLP is a nonlinear system. When neural networks are considered as an optimal filtering problem, the need for a recursive solution that uses current information and does not require storing the entire weights evolution becomes evident. This is the essence of KF applied to training neural networks.

Estimation is performed recursively, meaning that each update of the estimate (synaptic weights) is calculated using the previous estimate and current data, requiring only the previous estimate to be stored. Training is conducted from a set of N input–output measurements with the objective of finding the optimal weights that minimize prediction error. It is well known that neural networks can be modeled as:

$$\widehat{w}(k+1)=\widehat{w}\left(k\right)$$

(45)

$$\widehat{y}\left(k\right)=\phi (\widehat{w}\left(k\right),u\left(k\right))$$

(46)

where $\phi (•)$ is a nonlinear function of neural weights $\widehat{w}$ and external input u.

According to this, the equations necessary for the development of the learning algorithm based on EKF are:

$$K\left(k\right)=P\left(k\right)H\left(k\right){\left[R\left(k\right)+H\left(k\right)P\left(k\right)H\left(k\right)\right]}^{-1}$$

(47)

$$w(k+1)=w\left(k\right)+K\left(k\right)\left[y\left(k\right)-\widehat{y}\left(k\right)\right]$$

(48)

$$P(k+1)=P\left(k\right)-K\left(k\right)H\left(k\right)P\left(k\right)+Q\left(k\right)$$

(49)

where $P\left(k\right)\in {\Re }^{L\times L}$ represents the estimation error matrix at time $k,w\in {\Re }^L$ is the state vector (weight vector), L represents number of neural network weight (dimension of weight vector) and $\in {\Re }^m$ represents vector of desired outputs, $\widehat{y}\in {\Re }^m$ is the vector of neural network output where m represents its dimension. $K\in {\Re }^{L\times m}$ is the Kalman gain, $Q\in {\Re }^{L\times L}$ is the covariance matrix for weight estimation noise, $R\in {\Re }^{m\times m}$ is the covariance matrix for measurement noise, $H\in {\Re }^{m\times L}$ is a matrix that contains the derivative of each neural output with respect to each neural network weight.

$$H_{ij}\left(k\right)={\left[{\displaystyle \frac{\partial {\widehat{y}}_i\left(k\right)}{\partial w_{ij}\left(k\right)}}\right]}_{w\left(k\right)=w(k-1)};i=1,\dots ,m;j=1,\dots ,L$$

(50)

Typically, $P,R$, and Q are initialized as diagonal matrices where the non-zero elements are defined as $P\left(0\right)$, $R\left(0\right)$ and $Q\left(0\right)$, respectively, considering the MLP depicted in Figure 5.

With p inputs and h hidden networks, the weight vector is defined as:

$$w={\left[w_{10}^{\left(1\right)}\dots w_{1p}^{\left(1\right)}{w}_{20}^{\left(1\right)}\dots w_{h\times p}^{\left(1\right)}{w}_{10}^{\left(2\right)}\dots w_{1h}^{\left(2\right)}\right]}^T$$

(51)

The total number of weight elements L, is defined as:

$$L=(p+1)\times h+(h+1)\times m$$

(52)

where m is the number of neural network outputs, defined as:

$${\sigma }_i={\displaystyle \frac{1}{1+e^{-n_i}}}i=1,\dots ,h$$

(53)

$$n_i=\sum _{j=0}^p{w}_{ij}^{\left(1\right)}{x}_j{x}_o=+1$$

(54)

$$v_1=\sum _{k=0}^h{w}_{ik}^{\left(2\right)}{\sigma }_k{\sigma }_o=+1$$

(55)

$$\widehat{y}=v_1$$

(56)

Then

$${\displaystyle \frac{\partial \widehat{y}}{\partial w}}=\left[{\displaystyle \frac{\partial \widehat{y}}{\partial w_{10}^{\left(1\right)}}}{\displaystyle \frac{\partial \widehat{y}}{\partial w_{11}^{\left(1\right)}}}\dots {\displaystyle \frac{\partial \widehat{y}}{\partial w_{1h}^{\left(2\right)}}}\right]$$

(57)

Therefore,

$$H=\left[\gamma \left(n_1\right)x_0\dots \gamma \left(n_1\right)x_p\gamma \left(n_h\right)x_p{\sigma }_0\dots {\sigma }_h\right]$$

(58)

with

$$\gamma \left(n_i\right)={\displaystyle \frac{w_{1i}^{\left(2\right)}{e}^{-n_i}}{\left(1+e^{-n_i}\right)}}i=1,\dots ,h$$

(59)

which is valid for the MLP depicted in Figure 5 with sigmoid activation functions for hidden neurons and the linear activation function for the output neuron.

4.6. Recurrent High-Order Neural Network Trained with EKF

As a third type of neural network considered in this work, a Recurrent High-Order Neural Network (RHONN) is used to show applicability of EKF as a learning algorithm to train online artificial neural networks using stochastic estimation capabilities of EKF;this type of neural network is based on a Hopfield type neural network [86]. Recurrent neural networks offer a better-suited tool to model data from dynamic environments [54]. Since the seminal paper [87], there has been continuously increasing interest in applying NNs to identification and control of nonlinear systems, specially RHONN, due to their excellent approximation capabilities, using few units. These kind of NNs, compared to the first-order ones, are more flexible and robust when faced with new and/or noisy data patterns [88]. Furthermore, RHONNs performed better than the multilayer first-order ones using a small number of free parameters [89]. Additionally, different authors have demonstrated the feasibility and the advantages of using these architectures in applications for system identification and control. The best-known training approach for RNNs is the backpropagation-through-time learning [48]. However, it is a first-order gradient-descent method, and hence, its learning speed can be very slow. Recently, the extended Kalman filter (EKF)-based algorithms have been introduced to train NNs, in order to improve the learning convergence [41]. Note that it is derived from an EKF-based learning algorithm for RHONN.

Consider an EKF represented by Equations (47)–(49) with:

$$H_{ij}\left(k\right)={\left[{\displaystyle \frac{\partial {\widehat{y}}_i\left(k\right)}{\partial w_{ij}\left(k\right)}}\right]}_{w\left(k\right)=w(k-1)};i=1,\dots ,m;j=1,\dots ,L$$

(60)

For a discrete-time RHONN used as an identifier:

$$\widehat{y}\left(k\right)=\widehat{x}\left(k\right)=w^T\left(k\right)z\left(x\left(k\right),u\left(k\right)\right)$$

(61)

Therefore,

$$H\left(k\right)={\displaystyle \frac{\partial \widehat{y}\left(k\right)}{\partial w}}=z\left(x\left(k\right),u\left(k\right)\right)$$

(62)

As initial weight values are non-dependent of the RHONN state:

$$H_{ij}\left(0\right)=0$$

(63)

4.7. Radial Basis Neural Network Trained with an EKF

From the literature, it is evident that another of the most used neural networks to deal with nonlinear complex data is the radial basis neural network. This type of neural network has a simple structure that allows its use without the need to be an expert on neural networks leaning. Their most commune algorithms are designed for offline learning; however, in [47], they propose an online learning algorithm based on an EKF. In this application, neural network parameters are state variables estimated by EKF. Consider the radial basis neural network (RBF) as depicted in Figure 6:

$$x={\left[x_1{x}_2\dots x_p\right]}^T$$

(64)

$$c={\left[c_1{c}_2\dots c_p\right]}^T$$

(65)

$$i=1,\dots ,m$$

(66)

With p inputs and m hidden neurons, RBF parameters can be defined as:

$$\theta ={\left[w_1\dots w_m{c}_{11}\dots c_{1p}{c}_{21}\dots c_{mp}\right]}^T$$

(67)

$$\widehat{y}=\sum _{i=0}^p\sum _{j=1}^m{w}_{j}G\left(\left|\right|x_i-c_j\left|\right|\right)=WG$$

(68)

$${\displaystyle \frac{\partial \widehat{y}}{\partial \theta }}={\left[{\displaystyle \frac{\partial \widehat{y}}{\partial w_1}}\dots {\displaystyle \frac{\partial \widehat{y}}{\partial w_m}}{\displaystyle \frac{\partial \widehat{y}}{\partial c_{11}}}\dots {\displaystyle \frac{\partial \widehat{y}}{\partial c_{1p}}}{\displaystyle \frac{\partial \widehat{y}}{\partial c_{21}}}\dots {\displaystyle \frac{\partial \widehat{y}}{\partial c_{cmp}}}\right]}^T$$

(69)

In this case, the total number of neural network parameters L, is defined as:

$$L=m+mp$$

(70)

Therefore:

$${\displaystyle \frac{\partial \widehat{y}}{\partial w_1}}=G\left(∥x-c_1∥\right)$$

(71)

⋮

(72)

$${\displaystyle \frac{\partial \widehat{y}}{\partial w_m}}=G\left(∥x-c_m∥\right)$$

(73)

$${\displaystyle \frac{\partial \widehat{y}}{\partial c_1}}=w_1{G}^{\prime }\left(∥x-c_1∥\right)$$

(74)

⋮

(75)

$${\displaystyle \frac{\partial \widehat{y}}{\partial c_m}}=w_m{G}^{\prime }\left(∥x-c_m∥\right)$$

(76)

Considering:

$$\begin{array}{cc}\begin{array}{c}\hfill G(x,c_i)=e^{-{\displaystyle \frac{\Vert x-c_i\Vert }{2{\sigma }^2}}}\end{array}& \end{array}$$

(77)

$$G(x,c_i)=e^{-{\displaystyle \frac{\Vert x-c_i\Vert }{2{\sigma }^2}}}=e^{-{\displaystyle \frac{1}{2{\sigma }^2}}\left[{(x_1-c_{ij})}^2+{(x_2-c_{ij})}^2+\dots +{(x_p-c_{ij})}^2\right]}$$

(78)

$$G^{\prime }(x,c_{ij})=e^{-{\displaystyle \frac{\Vert x-c_i\Vert }{2{\sigma }^2}}}\left[\left(-{\displaystyle \frac{1}{2{\sigma }^2}}\left[-2(x_1-c_{ij})\right]\right)\right]$$

(79)

$$G^{\prime }(x,c_{ij})=e^{-{\displaystyle \frac{\Vert x-c_i\Vert }{2{\sigma }^2}}}\left(x_j-c_{ij}\right)$$

(80)

where $i=1,\dots ,m$ and $j=1,\dots ,p$

Possible Modifications

• Just weight estimation (arbitrary centers of fixed centers).
• Global EKF.
• Decoupled EKF.

  −

  Centers decoupled from weights.

  −

  Weights decoupled from centers and other weights.

• Other combinations.

From these examples of EKF use in neural network learning, it is possible to start the discussion section in order to establish the challenges, limitations, open problems, and future works.

5. Challenges, Limitations, Open Problems and Future Work

Although KF has advantages over traditional learning algorithms, primarily the gradient descent algorithms, many challenges remain, such as computational cost, as using EKF can be resource-intensive, limiting its real-time application with limited hardware. Initializing the learning algorithm also remains a challenge. Regarding implementation limitations, the nonlinear approach to neural network learning also requires a mathematical model for the analytical approximation of EKF, while numerical approaches require training per epoch, batch, or semi-batch, which limits their real-time implementation.

As mentioned in this work, the use of KF and its variants for training neural networks requires a meticulous design tailored to the neural structure used, so the establishment of general-purpose, real-time implementable strategies remains an open problem. Another open problem is the choice of KF initialization parameters, since these design parameters directly impact the algorithm’s convergence conditions. Future work includes relevant applications for engineering such as advanced robotics, autonomous vehicles, energy systems, and biomedical systems, as well as complex innovative problems, including Edge Computing and the hardware implementation of KF and its variants. In this way, it is expected that publications regarding KF variants with a quantum approach can be increased in the coming years.

6. Conclusions

This work provides a practical review of the development of the Kalman Filter and its application to neural network training, including filtering and estimation concepts prior to KF, its main variants, and the modifications introduced to address nonlinear systems. Additionally, an introduction to the development of artificial neural networks and the inherent problems has been included. An analysis is also presented that highlights the KF potential as a learning algorithm for neural networks. Also, the training of three different types of artificial neural networks, MLP, RBF, and RHONN, using EFK was included. This review demonstrates the extensive applicability of KF in a wide range of applications ranging from traditional applications, estimation, and optimal filtering of linear systems to applications that combine the use of KF with artificial intelligence and machine learning techniques. Undoubtedly, in the coming years, an increasing number of implementations that combine the potential of KF with emerging artificial intelligence and machine learning methodologies to solve real-world problems will emerge.