Directed Equilibrium Propagation Revisited

Abstract

Equilibrium Propagation (EP) offers a biologically inspired alternative to backpropagation for training recurrent neural networks, but its reliance on symmetric feedback connections and stability limitations hinders practical adoption. The DirEcted EP (DEEP) model relaxes the symmetry constraint, yet suffers from convergence issues and lacks a principled learning guarantee. In this work, we generalize DEEP by incorporating neuronal leakage, providing new convergence criteria for the network’s dynamics. We additionally propose a novel local learning rule closely linked to the objective function’s gradient and establish sufficient conditions for reliable learning in small networks. Our results resolve longstanding stability challenges and bring energy-based learning models closer to biologically plausible and provably effective neural computation.

1. Introduction

Artificial neural networks are a type of artificial intelligence computing system modeled on the human brain and nervous system. The mathematical theory of neural networks explores how mathematical models can be used to understand and manipulate neural networks. In this regard, statistical learning theory is a valuable tool for studying the behavior of neural networks [1]. However, this paper will focus on a specific learning algorithm that is a biologically plausible alternative to the well-known backpropagation algorithm. We will use for this purpose some results from dynamical systems theory.

The basic building block of all artificial neural networks (ANNs) is the Neuron. Mathematically, a neuron is a function $s,{\mathbb{R}}^n\to \mathbb{R}$, $n\in \mathbb{N}$, parameterized by a weight vector $\mathbf{w}\in {\mathbb{R}}^{n+1}$ and defined by

$$s\left(\mathbf{x}\right)=\rho \left(\sum _{i=1}^n{w}_i{x}_i+w_0\right),$$

(1)

where $\rho$ is a suitable, usually continuous real function, known as the activation function. A feed-forward neural network is built by the composition of many (perhaps billions) of such simple functions, organized in layers. Networks with more than three layers are known in the literature as deep neural networks. For instance, the ANN that was trained to play the game of Chess or Go with superhuman performance, Alpha-zero, is constituted by more than 80 layers, with each layer having thousands of neurons [2,3]. For other applications, like natural language understanding or translation, recurrent networks are used. These networks are not necessarily organized by layers and thus do not form an acyclic graph.

The goal of training such an artificial neural network is to find the correct parameters so that the function implemented by the ANN satisfies certain conditions or minimizes some objective function. For example, in supervised learning, a training set of input–output pairs is used to adjust the parameters until the error between the output of the ANN and the true values is minimized, using gradient descent in the parameter space. Because the error is propagated from the output layer backwards, the algorithm used is known as backpropagation (see, for instance, [4], Chapter 5).

The backpropagation approach to supervised learning stands out as the most successful algorithm for training artificial neural networks. But despite ANNs being originally bio-inspired, backpropagation is widely considered to be biologically implausible [5]. Among other reasons, it (i) lacks local error representation, (ii) uses distinct forward and backward information passes and (iii) requires symmetric feedback weights. One path towards bridging the gap between biology and machine learning is thus to explore other learning paradigms, closer to biological reality.

Equilibrium propagation (EP) is an energy-based model for recurrent neural networks, which operates by minimizing an energy function, uses a local learning rule and just one kind of neural computation, satisfying restrictions (i) and (ii) [5]. This model considers continuous-time neural dynamics, and so is a departure from backpropagation, and makes discrete per-layer updates. However, it also requires symmetric feedback weights to work.

The original idea was followed by several advances addressing the challenges of convergence, scalability, and hardware efficiency while reinforcing theoretical ties to traditional backpropagation.

Laborieux et al. [6] extended EP to deep convolutional networks using symmetric nudging; their work on CIFAR-10 reports an 11.7% test error and lower memory use, though convergence is about 20% slower than backpropagation through time. Martin et al. [7] introduced the EqSpike algorithm for spiking networks and achieved 97.6% accuracy on MNIST. Foroushani et al. [8] implemented EP on an analog circuit and reported a 250-fold acceleration in the relaxation process relative to a Python baseline. Kiraz et al. [9] focused on parameter optimization by analyzing the effects of feedback current and learning rate, while Laborieux and Zenke [10] proposed holomorphic EP, which computes exact gradients using finite teaching signals and matches backpropagation performance on an ImageNet 32 × 32 benchmark.

The problem of symmetric feedback weights was tackled with the introduction of the DirEcted EP (DEEP) model, which allows for asymmetric feedback connections while abandoning the need for a global energy function [11]. The issue in doing so is that the convergence of the neuronal dynamics is no longer assured, because there is no energy function guiding the network’s behavior. In addition, the weight update rule, while biologically inspired, does not seem to have any ties to the gradient of the objective function, and learning is thus, in general, not guaranteed.

The main contribution of the present work is a generalization of the DEEP model by adding leakage to non-input neurons, which solves the stability issues found in the earlier work. New conditions for the convergence of the neuronal dynamics of DEEP’s inference phase are established, and a different local weight update rule is proposed, with close ties to the gradient of the objective function. Moreover, sufficient conditions for a small-sized network to learn following the proposed weight update rule are also determined.

The remainder of the paper is organized as follows. In Section 2, common notation and the EP and DEEP models are introduced. Section 3 goes over the stability problems in the DEEP model. In Section 4, our generalization of the DEEP model is presented along with new conditions for convergence of the inference and learning phases. Section 5 discusses a new learning rule and its ties to the gradient of the objective function. Section 6 concludes the paper.

2. Equilibrium Propagation and Directed Equilibrium Propagation

In this section, we begin by defining some notation to be used throughout this text. Then, we present both the original EP model and the DEEP model.

2.1. Notation

In Equation (1), $w_0$ is commonly referred to as the neuron’s bias. In this text, the bias is represented as a fixed neuron (neuron 0) that maintains a state of one and connects to every non-input neuron. Then, the weight associated with the connection from this neuron to a neuron i becomes neuron i’s bias.

For an architecture consisting of a total of $N+1$ neurons, where P are input neurons, L are hidden neurons, and K are output neurons, both EP and DEEP are entirely described by the following elements:

• A state vector $\mathbf{s}\left(t\right)={\left[s_j\left(t\right)\right]}_{j=0}^N\in {[0,1]}^N$, representing the neuronal activities. This state vector is composed of sub-vectors corresponding to the (fixed) input $\mathbf{x}={\left[s_j\left(t\right)\right]}_{j=1}^P$, the hidden neurons $\mathbf{h}\left(t\right)$, and the output neurons $\widehat{\mathbf{y}}\left(t\right)={\left[s_j\left(t\right)\right]}_{j=N-K+1}^N$, where $s_0\left(t\right)\equiv 1$ corresponds to the bias neuron. The value $s_j$ can be biologically interpreted as the firing rate of neuron j;
• A weight matrix $\mathbf{W}={\left[W_{ij}\right]}_{i,j=0}^N$, where $W_{ij}$ represents the weight of the connection from neuron i to j;
• A system of continuous-time differential equations that governs the dynamics of the network.

From this point onward, the time dependency of the state variable will be omitted to simplify the notation.

2.2. Equilibrium Propagation

Originally introduced in [5], EP is a learning framework for energy-based models that employs a network of recurrently connected neurons with symmetric weights. Its goal is to provide a more biologically plausible alternative to conventional ANNs by circumventing the backpropagation algorithm.

This model is driven by an energy function F, which is the central object of its behavior: all quantities of interest (fixed points, cost function, objective function, gradient formula) can be defined or formulated directly in terms of F.

2.2.1. The Network

The quantity $W_{ij}=W_{ji}$ is the weight associated with the connection between neurons i and j. Self-loops are excluded, meaning $W_{ii}=0$ for all i, and input neurons do not form connections among themselves, i.e., $W_{ij}=0$ for all $i,j\in \{1,\dots ,P\}$.

In Figure 1, the architecture of a three-neuron network of the EP model is represented. The state vector is $\mathbf{s}={\left[s_i\right]}_{i=0}^3$, containing four elements: a constant bias term $\left[s_0\right]=\left[1\right]$, an input term $\mathbf{x}=\left[s_1\right]$, a hidden term $\mathbf{h}=\left[s_2\right]$ and an output term $\widehat{\mathbf{y}}\left(t\right)=\left[s_3\right]$. Each $s_j$ corresponds to the activity of the $j^{th}$ neuron. The synaptic connections are described by the weight matrix $\mathbf{W}={\left[W_{ij}\right]}_{i,j=0}^3$ (note that $W_{01}$ is omitted from Figure 1 since 1 is an input neuron and so $W_{01}=0$).

2.2.2. Auxiliary Functions

There are some functions that need to be defined in order to establish the dynamics of the network. The internal energy function, E, depends only on the state of the network and is defined as

$$E\left(\mathbf{s}\left(t\right)\right)={\displaystyle \frac{1}{2}}\sum _{i=0}^N{s}_i^2\left(t\right)-{\displaystyle \frac{1}{2}}\sum _{i=0}^N\sum _{j=0}^N{W}_{ij}{s}_i\left(t\right)s_j\left(t\right).$$

(2)

It should be noted that the notation in this equation differs slightly from the one presented in [5]. In particular, the notation used was simplified due to the alternative version of stochastic gradient descent employed (detailed in Equation (8)), under which $s_j=\rho \left(s_j\right)$ holds for all j.

The external energy function C, which corresponds to the cost function, is the mean squared error and is defined as

$$C(\mathbf{s},\mathbf{y})={\displaystyle \frac{1}{2}}\sum _{j=P+L+1}^N{(s_j\left(t\right)-y_j)}^2,$$

(3)

where $\mathbf{y}$ is the vector containing the target output for the training input. The clamping factor, denoted by $\beta \in {\mathbb{R}}_0^{+}$, is the parameter that determines the extent to which the target output y influences the network. Incorporating this “influence parameter”, the total energy function F is then defined as

$$F\left(\mathbf{s}\right(t),\mathbf{y},\beta )=E\left(\mathbf{s}\right(t\left)\right)+\beta C\left(\mathbf{s}\right(t),\mathbf{y}).$$

(4)

Finally, the loss function J is defined as the external energy at the equilibrium state ${\mathit{s}}^{★}=\mathbf{s}\left(T_0\right)$, which is achieved after adjusting the neurons over the time interval $[0,T_0]$ to minimize the internal energy function.

2.2.3. The Dynamics and Training Algorithm

The EP dynamics are defined by a set of differential equations. The activity of each adjustable neuron, $s_j$, $j\in \{P+1,\dots ,N\}$, evolves in the negative direction of the gradient of the total energy,

$${\displaystyle \frac{d{s}_j}{dt}}=-{\displaystyle \frac{\partial F\left(\mathbf{s}\right(t),\mathbf{y},\beta )}{\partial s_j}}=-{\displaystyle \frac{\partial E\left(\mathbf{s}\right(t\left)\right)}{\partial s_j}}-\beta {\displaystyle \frac{\partial C\left(\mathbf{s}\right(t),\mathbf{y})}{\partial s_j}},$$

(5)

where

$${\displaystyle \frac{\partial F\left(\mathbf{s}\right(t),\mathbf{y},\beta )}{\partial s_j}}=s_j-\sum _{i=0}^N{W}_{ij}{s}_i+\beta (s_j-y_j){\mathbb{1}}_{\widehat{\mathbf{y}}}\left(s_j\right),$$

(6)

Recall that the indicator function is defined as

$${\mathbb{1}}_A\left(x\right)=\left\{\begin{array}{cc}1,\hfill & \mathrm{x}\in \mathrm{A}\hfill \\ 0,\hfill & \mathrm{x}\notin \mathrm{A}\hfill \end{array}\right..$$

and $s_j\in \widehat{\mathbf{y}}\iff j\in \{P+L+1,\dots ,N\}\iff j$ is an output neuron. As a result, the total energy of the system is guaranteed to decrease over time. To promote convergence and keep the firing rates confined within the interval $[0,1]$, a different form of gradient descent is used in the neuronal update rule. Take $dt$ as the smallest time interval considered between iterations. At iteration step $m+1$, the standard gradient descent update rule takes the form

$$s_j(m+1)⟵s_j\left(m\right)+dt\left(\sum _{i=0}^N{W}_{ij}{s}_i\left(m\right)-s_j\left(m\right)-\beta \left(s_j\left(m\right)-y_j\right){\mathbb{1}}_{\widehat{\mathbf{y}}}\left(s_j\right)\right).$$

(7)

However, in what follows, the update rule is modified to

$$s_j(m+1)⟵\rho \left(s_j\left(m\right)+dt\left(\sum _{i=0}^N{W}_{ij}{s}_i\left(m\right)-s_j\left(m\right)-\beta \left(s_j\left(m\right)-y_j\right){\mathbb{1}}_{\widehat{\mathbf{y}}}\left(s_j\right)\right)\right).$$

(8)

The training algorithm has two distinct phases, corresponding to setting $\beta =0$ (first phase) and $\beta >0$ (second phase).

The first phase, which corresponds to the inference process, consists of, given a fixed input expressed by the values taken by the input neurons, allowing the network to evolve toward an equilibrium state with respect to Equation (5) with $\beta =0$. In the second phase, commonly referred to as the learning phase, a small increment in $\beta$ introduces a perturbation to the system. This slight adjustment causes the output neurons to shift their activities closer to the target values, which, in turn, propagates through the network. The system then relaxes into a new equilibrium state with respect to (5) that reduces the loss function.

Let us denote the first and second equilibrium states by ${\mathbf{s}}^0$ and ${\mathbf{s}}^{\beta }$. The weight updates are then computed based on the gradients of the loss function, which, in the limit $\beta \to 0$, solely depend on ${\mathbf{s}}^0$ and ${\mathbf{s}}^{\beta }$:

$$\Delta W_{ij}=s_i^{\beta }{s}_j^{\beta }-s_i^0{s}_j^0.$$

(9)

This Equation for the loss gradient of $W_{ij}$ was proved in [5].

2.3. DirEcted Equilibrium Propagation (DEEP)

DEEP was proposed in [11] and consists of a generalization of the original EP model. Specifically, DEEP “seeks to improve the biological plausibility of the original EP model, by removing all structural restrictions and enabling the architecture of the model to be a complete directed (asymmetric connections) graph”. Figure 2 shows an example of such a Neural Network.

Due to the lack of symmetry of the synaptic weights, DEEP is not energy-based. This is because its neuronal dynamics are governed by vector fields, which are not gradient fields.

Fixing (clamping) the input neurons’ states, the neuronal dynamics proposed is

$${\displaystyle \frac{d{s}_j}{dt}}=V_j^{\beta }(\mathbf{s},\mathbf{y},\beta )=\sum _{i=0}^N{W}_{ij}{s}_i-s_j\sum _{i=0}^N{W}_{ji}-\beta (s_j\left(t\right)-y_j\left(t\right)){\mathbb{1}}_{\widehat{\mathbf{y}}}\left(s_j\right),$$

(10)

where $\mathbf{y}$ is the vector of target/desired outputs and the cost function, C, is identical to the one specified in (3) for the original formulation. The training algorithm also consists of two distinct phases: the first phase, where $\beta =0$, and the second phase, where $\beta \ne 0$. During both phases, the network reaches equilibrium states, denoted by $s^0$ in the first phase and $s^{\beta }$ in the second. The objective function is defined as the cost when the network is at its first equilibrium state, $J(\mathbf{x},\mathbf{y}):=C({\mathbf{s}}^0,\mathbf{y})$. In the inference phase, the activities of the input neurons are fixed, and the network evolves to the equilibrium state ${\mathbf{s}}^0$, from which the output is read at the corresponding output neurons $\widehat{\mathbf{y}}$.

The learning rule used is

$$\Delta W_{ij}\propto {\displaystyle \frac{1}{M_{\beta }}}\sum _{m=M_0+1}^{M_0+M_{\beta }}{s}_i\left(m\right)\left(s_j\left(m\right)-s_j(m-1)\right),$$

(11)

where $M_0$ and $M_{\beta }$ are the number of steps in a discretization of the first and second phases, respectively, and $s_i\left(m\right)$ is the ith neuron’s state after m steps.

General Stability Test

Consider a nonlinear, time-invariant dynamical system with state variable $s\in {\mathbb{R}}^N$,

$${\displaystyle \frac{d\mathbf{s}\left(t\right)}{dt}}=f\left(\mathbf{s}\left(t\right)\right),$$

(12)

where $f:{\mathbb{R}}^N\to {\mathbb{R}}^N$ is a nonlinear continuously differentiable function (a $C^1$ map).

Definition 1

(Equilibrium state). An equilibrium state ${\mathit{s}}^{★}$ of the system represented by Equation (12) is one for which $f\left({\mathit{s}}^{★}\right)=0$.

Definition 2

(Stable in the sense of Lyapunov). A system is said to be stable in the sense of Lyapunov around ${\mathit{s}}^{★}$ if, for any $ϵ>0$, there exists $\delta \left(ϵ\right)>0$ such that if $\parallel \mathit{s}\left(t_0\right)-{\mathit{s}}^{★}\parallel <\delta \left(ϵ\right)$, then $\parallel \mathit{s}\left(t\right)-{\mathit{s}}^{★}\parallel <ϵ$ for all $t>t_0$.

Definition 3

(Locally asymptotically stable). A system is said to be locally asymptotically stable around ${\mathit{s}}^{★}$ if it is stable in the sense of Lyapunov and there exists $\gamma \left(t_0\right)>0$ such that, if $\parallel \mathit{s}\left(t_0\right)-{\mathit{s}}^{★}\parallel <\gamma \left(t_0\right)$, then ${lim}_{t\to \infty }\mathit{s}\left(t\right)={\mathit{s}}^{★}$.

The following result, which concerns the stability of nonlinear dynamical systems, is stated and proved in [11]:

Theorem 1

(Farinha). Let ${\mathit{s}}^{★}$ be an equilibrium state of Equation (12), $J\in {\mathbb{R}}^{N\times N}$ the Jacobian matrix of f evaluated at ${\mathit{s}}^{★}$ and $R_j={\sum }_{i=1,i\ne j}\left|J_{ji}\right|,j\in \{1,\dots ,N\}$. If, for $j\in \{1,\dots ,N\}$, the following two conditions are satisfied

1. 

$J_{jj}<0$;

2. 

$R_j<\left|J_{jj}\right|$;

then ${\mathit{s}}^{★}$ is locally asymptotically stable.

It yields, as a corollary, sufficient conditions for the stability of the networks’ dynamics during an inference phase:

Corollary 1

(DEEP Inference Sufficient Conditions for Stability Verification). Let ${\mathit{s}}^0$ be an equilibrium state with respect to the dynamics given by Equation (10). If, for $j\in \{P+1,\dots ,N\}$, the following two conditions are satisfied:

1. 

${\sum }_{i=0}^N{W}_{ji}>0$;

2. 

${\sum }_{i=P+1}^N|W_{ij}|<|{\sum }_{i=0}^N{W}_{ji}|$;

then ${\mathit{s}}^0$ is locally asymptotically stable.

3. Stability Issues of the DEEP Model

In this section, two problems regarding the stability of the inference and learning phases of the DEEP model [11] are presented. The first one concerns the mere existence of stable states with regard to Equation (10) when $\beta =0$, while the second one concerns Corollary 1.

3.1. Inexistence of Stable States

During an inference phase, $\beta =0$, so the networks’ dynamics is dictated by

$$V_j\left(s\left(t\right)\right)=\sum _{i=0}^N{W}_{ij}{s}_i\left(t\right)-s_j\left(t\right)\sum _{i=0}^N{W}_{ji}.$$

(13)

In [11], it is noted that a property of DEEP is that, in the absence of external stimuli (when the input and bias neurons have fixed activities), the total sum of firing-rates remains constant over time (i.e., ${\sum }_{j=1}^N\dot{s_j}\left(t\right)=0,t\in {\mathbb{R}}_0^{+}$). Despite this, in an inference phase, the input neurons are clamped (have fixed firing rates), and so the sum of the time derivatives of the firing rates is:

$$\begin{array}{cc}\hfill \sum _{j=0}^N\dot{s_j}& =\sum _{j=P+1}^N\dot{s_j}=\sum _{j=P+1}^N\left(\sum _{i=0}^N{W}_{ij}{s}_i-s_j\sum _{i=0}^N{W}_{ji}\right)=\hfill \end{array}$$

(14)

$$\begin{array}{ccc}\hfill & =\sum _{j=P+1}^N\sum _{i=0}^N{W}_{ij}{s}_i-\sum _{j=P+1}^N\sum _{i=0}^N{W}_{ji}{s}_j=\hfill & \end{array}$$

(15)

$$\begin{array}{cc}\hfill & =\left(\sum _{j=P+1}^N\sum _{i=P+1}^N{W}_{ij}{s}_i-\sum _{j=P+1}^N\sum _{i=P+1}^N{W}_{ji}{s}_j\right)+\sum _{j=P+1}^N\sum _{i=0}^P{W}_{ij}{s}_i-\sum _{j=P+1}^N\sum _{i=0}^P{W}_{ji}{s}_j.\hfill \end{array}$$

(16)

The two terms inside the parentheses cancel each other out, and since $W_{ji}=0$$\forall i\le P$, the last term is 0. We are left with

$$\sum _{j=0}^N\dot{s_j}=\sum _{i=0}^P\sum _{j=P+1}^N{W}_{ij}{s}_i=\mathit{constant}.$$

(17)

As a consequence, without an activation function, the neuronal dynamics in (10) do not have any equilibrium state (its existence requires it to be possible for ${\sum }_{j=0}^N\dot{s_j}$ to equal 0).

In [11], this was avoided with the use of the activation function $\rho :\mathbb{R}\to [0,1]$, defined as

$$\rho \left(s\right)=\left\{\begin{array}{cc}0,\hfill & \mathrm{s}<0\hfill \\ s,\hfill & 0\le \mathrm{s}\le 1\hfill \\ 1,\hfill & \mathrm{s}>1\hfill \end{array}\right..$$

(18)

This function, which is continuous, and differentiable almost everywhere, is known as the hard-sigmoid and constrains the neuron’s activity to lie within the interval $[0,1]$. Nevertheless, the neurons in the DEEP model would saturate (i.e., have a high tendency to only stabilize with an activity of either 0 or 1). Following the tradition in neural networks literature, we will represent by the same symbol the map obtained when applying $\rho$ to each entry of a vector.

3.2. On the Assumptions of Corollary 1

In [11], Theorem 1 is proved using the combination of two major results: Lyapunov’s indirect method and Gershgorin Circle’s Theorem, which states that every eigenvalue of a matrix A lies within at least one of the Gershgorin circles $D_i=\{z\in \mathbb{C}:|z-A_{ii}|\le R_i\}$, where $R_i={\sum }_{j=1,j\ne i}^N\left|A_{ij}\right|$ [12]. The former guarantees that, linearizing the system around an equilibrium state $\mathbf{s}$, in order to establish the local asymptotic stability of $\mathbf{s}$, one must verify that all eigenvalues of the Jacobian matrix J of V (evaluated at $\mathbf{s}$) have strictly negative real parts. The latter provides sufficient conditions for the real parts of every eigenvalue of J to be strictly negative.

The matrix J is:

$$J=\left[\begin{array}{cccc}-{\displaystyle \sum _{i=0}^N}{W}_{P+1,i}& W_{P+2,P+1}& \dots & W_{N,P+1}\\ W_{P+1,P+2}& -{\displaystyle \sum _{i=0}^N}{W}_{P+2,i}& \dots & W_{N,P+2}\\ ⋮& & \ddots & ⋮\\ W_{P+1,N}& \dots & W_{N-1,N}& -{\displaystyle \sum _{i=0}^N}{W}_{N,i}\end{array}\right].$$

(19)

Since $W_{ji}=0$$\forall i\le P$, the sum of the entries of each column of J is 0. As such, the vector $v=(1,1,\dots ,1)$ satisfies $J^T·v=0·v$, meaning $\lambda =0$ is an eigenvalue of $J^T$ and thus of J, regardless of W.

This suggests there should not be any possible conditions given by the Gershgorin Circle Theorem guaranteeing that all eigenvalues of J have strictly negative real parts (because $\lambda =0$ does not satisfy that requirement). So, the following result holds:

Lemma 1.

It is impossible to simultaneously satisfy both conditions stated in Corollary 1.

Proof. 

Considering the first condition and the fact that $W_{ji}=0$ $\forall i\le P$, the second condition yields:

$$\sum _{i=P+1}^N\left|W_{ij}\right|<\left|\sum _{i=0}^N{W}_{ji}\right|=\sum _{i=0}^N{W}_{ji}=\sum _{i=P+1}^N{W}_{ji}.$$

(20)

Summing this inequality with j varying from $P+1$ to N, we obtain

$$\begin{array}{cc}\hfill \sum _{j=P+1}^N\sum _{i=P+1}^N\left|W_{ij}\right|& <\sum _{j=P+1}^N\sum _{i=P+1}^N{W}_{ji}\hfill \end{array}$$

(21)

$$\begin{array}{cc}\hfill \iff \sum _{j=P+1}^N\sum _{i=P+1}^N\left|W_{ji}\right|& <\sum _{j=P+1}^N\sum _{i=P+1}^N{W}_{ji},\hfill \end{array}$$

(22)

which is a contradiction. □

In this section, we showed that the neuronal dynamics in the DEEP model, without an activation function, has no equilibrium state. Furthermore, Lemma 1 shows that Corollary 1 (from [11]) is a vacuously true statement: it claims that given Conditions 1 and 2, $s^0$ is locally asymptotically stable yet those Conditions are impossible to be met simultaneously.

4. Generalizing the DEEP Model

One way to improve the DEEP model and overcome the problem described in the previous Section is to add a leakage term to the neurons. There are several ways of represent this in the model: one would be to introduce a new $\mathit{leakage}$ neuron $N+1$ and letting $W_{i(N+1)}$ be the ith neuron’s leakage term; another, that simplifies the mathematical treatment of the model, is to allow the connections ending in input neurons to be different from 0.

Since these neurons are clamped in both phases of training, they are unaffected by this change. As to each of the remaining neurons, they gain a leakage term ${\mathit{leak}}_{\left(i\right)}:={\sum }_{j=0}^P{W}_{ij}$ (which was previously equal to 0).

The dynamics of the network remain the same as in Equation (10), except now the first $P+1$ terms of the sum ${\sum }_{i=0}^N{W}_{ji}$ are not necessarily 0. We note that, by adding leakage to the model

• Both problems mentioned in the previous Section are solved: the sum of the derivatives of the neurons’ activations, during an inference phase, is not constant anymore, and the conditions provided by Corollary 1 are no longer impossible to hold. In particular, since in [11] it was nowhere used that $W_{ij}=0,j\le P$, Corollary 1 still holds.
• The input neurons stop being different from any other neurons: on a structural level, what distinguished them was having no incoming connections.

We now present results attained for the stability of the altered DEEP model, which incorporates the newly added leakage and may shed light on this model’s need for leakage. We then show that these new stability conditions and Corollary 1 are also enough to ensure stability in the subsequent learning phase ($\beta >0$). Finally, an interesting link between the Corollary 1 and the new conditions is described.

4.1. New Stability Conditions

Let us consider the discretized system, with $dt>0$ as the smallest time interval considered, and let $s\left(n\right)$ be the system’s state at the instant $n·dt$. Then, following the same version of stochastic gradient descent considered in Equation (8), now in vectorial form, we have $s(n+1)=\rho \left(s\left(n\right)+dt·V\left(s\right(n\left)\right)\right).$ This means we can view an inference phase as a fixed point iteration of the function $\mathsf{\Phi }$, defined as

$$\begin{array}{cc}\hfill \mathsf{\Phi }:{\mathbb{R}}^{N+1}& \to {\mathbb{R}}^{N+1}\hfill \\ \hfill s\left(n\right)& \mapsto s(n+1)=\rho \left(s\left(n\right)+dt·V\left(s\right(n\left)\right)\right).\hfill \end{array}$$

An equilibrium state with respect to the network’s dynamics simply becomes a fixed point of $\mathsf{\Phi }$. We assume $\rho$ is continually differentiable and $\rho \left(\mathbb{R}\right)\subseteq [0,1]$ (see end of this subsection). We will need the following theorems (see, for instance, [13,14]):

Theorem 2.

Let X be a convex set in ${\mathbb{R}}^n$ and $\mathsf{\Phi }:X\subset {\mathbb{R}}^n\to {\mathbb{R}}^n$ a $C^1$ function in ${\mathbb{R}}^n$. Letting $J_{\mathsf{\Phi }}$ be the Jacobian of Φ, (The matrix norm 1 of a matrix A is defined as ${\parallel A\parallel }_1={max}_{j=1,\dots ,n}{\sum }_{i=1}^n\left|a_{ij}\right|.$) if

$$\underset{x\in X}{sup}{\parallel J_{\mathsf{\Phi }}\left(x\right)\parallel }_1<1,$$

(23)

then the function Φ is contractive in X.

Theorem 3

(Fixed point theorem in ${\mathbb{R}}^n$). Let E be a finite-dimensional normed space and X a closed convex subset of E. Let Φ be a contractive function in X, such that

$$\mathsf{\Phi }\left(X\right)\subset X.$$

Then, the following statements are valid:

(1) 

Φ has a unique fixed point z in X.

(2) 

If ${\left(x^{\left(k\right)}\right)}_{k\ge 0}$ is the succession of terms in E such that $x^{\left(0\right)}\in X$ and

$$x^{(k+1)}=\mathsf{\Phi }\left(x^{\left(k\right)}\right),\forall k\ge 0,$$

then ${\left(x^{\left(k\right)}\right)}_{k\ge 0}$ converges to z.

We take $E={\mathbb{R}}^{(L+K)}$ and $X={[0,1]}^{(L+K)}$, which clearly satisfy the conditions of Theorem 3. Considering that the activation function $\rho$ restricts neurons’ activations to the interval $[0,1]$, $\mathsf{\Phi }\left(X\right)\subset X$ is immediate. All we have to verify is that $\mathsf{\Phi }$ is contractive in X. We are now ready to prove the following result:

Theorem 4

(DEEP Sufficient Conditions for Stability of an Inference Phase). The inference phase of the DEEP model converges to an equilibrium state if the activation function ρ used is such that $|{\rho }^{\prime }\left(x\right)|\le 1,\forall x\in \mathbb{R}$, and for $j\in \{P+1,\dots ,N\}$,

$${\mathit{leak}}_{\left(j\right)}>2·\sum _{\begin{array}{c}i=P+1\\ W_{ji}<0\end{array}}^N\left|W_{ji}\right|,$$

(24)

where ${\mathit{leak}}_{\left(j\right)}={\sum }_{i=0}^P{W}_{j,i}$.

Proof. 

For any suitable function F, let $J_F$ denote its Jacobian matrix. By Theorem 2,

$$\underset{x\in X}{sup}{\parallel J_{\mathsf{\Phi }}\left(x\right)\parallel }_1<1,$$

(25)

suffices to guarantee convergence of DEEP’s inference phase. Let $x\in X$. Since $\mathsf{\Phi }=\rho \circ (I+dtV)$, where I is the identity operator, we have

$${\parallel J_{\mathsf{\Phi }}\left(x\right)\parallel }_1={\parallel J_{\rho }\left((I+dtV)\left(x\right)\right)J_{(I+dtV)}\left(x\right)\parallel }_1\le {\parallel J_{\rho }\left((I+dtV)x\right)\parallel }_1{\parallel J_{(I+dtV)}\left(x\right)\parallel }_1.$$

(26)

Since ${\displaystyle \frac{\partial {\rho }_i}{\partial x_j}}=0$ if $i\ne j$, and $|{\rho }^{\prime }\left(x\right)|\le 1$, $J_{\rho }$ is a diagonal matrix and ${\parallel J_{\rho }\left(x\right)\parallel }_1\le 1$.

Due to the linearity of the vector field V defined in (13), $J_{(I+dtV)}$ is independent of x. This means ${\parallel J_{(I+dtV)}\left(x\right)\parallel }_1<1$ is enough to ensure (25), since we are taking the supremum over the product of ${\parallel J_{\rho }\left((I+dtV)x\right)\parallel }_1$, which is not greater than 1, and ${\parallel J_{(I+dtV)}\left(x\right)\parallel }_1$, a constant real number strictly less than 1.

We perform the calculations for the $j^{th}$ column of $J_{(I+dtV)}=I+dt{J}_V$ ($J_V$ is matrix (19)):

$$|1-dt\sum _{i=0}^N{W}_{P+j,i}|+\sum _{i=P+1}^N|dt·W_{P+j,i}|<1.$$

(27)

For sufficiently small $dt$, $1-dt{\sum }_{i=0}^N{W}_{P+j,i}>0$ and we can remove the absolute value of the first term. We are left with

$$\begin{array}{cc}\hfill 1-dt(\sum _{i=0}^N{W}_{P+j,i}& -\sum _{i=P+1}^N\left|W_{P+j,i}\right|)<1\hfill \end{array}$$

(28)

$$\begin{array}{cc}\hfill \iff \sum _{i=0}^N{W}_{P+j,i}& -\sum _{i=P+1}^N\left|W_{P+j,i}\right|>0\hfill \end{array}$$

(29)

$$\begin{array}{cc}\hfill \iff \sum _{i=0}^P{W}_{P+j,i}& >2\sum _{\begin{array}{c}i=P+1\\ W_{P+j,i}<0\end{array}}^N\left|W_{P+j,i}\right|,\hfill \end{array}$$

(30)

where, in the last step, we cancelled the weights $W_{ij}$ that appeared on both sums. The left term is familiar and can be written as:

$${\mathit{leak}}_{(P+j)}>2·\sum _{\begin{array}{c}i=P+1\\ W_{P+j,i}<0\end{array}}^N\left|W_{P+j,i}\right|$$

(31)

or, in general, for $j\in \{1,\dots ,N-P\}$,

$${\mathit{leak}}_{(P+j)}>2·\sum _{\begin{array}{c}i=P+1\\ W_{P+j,i}<0\end{array}}^N\left|W_{P+j,i}\right|.$$

(32)

□

This result can be interpreted as the DEEP model needing leakage to compensate for the existence of weights with negative values.

The sigmoid function or the hyperbolic tangent are examples of commonly used activation functions which satisfy Theorem 4’s statement. Note that the activation function introduced in (18) is not $C^1$, however, it is $C^1$ almost everywhere, and so arbitrarily good approximations by differentiable functions can be used.

4.2. On the Stability During Learning Phase ($\beta >0$)

As previously stated, both Corollary 1 and Theorem 4 also ensure convergence of the learning phase, as can be seen in this Section.

4.2.1. DEEP Model

In Section 3.2, it was mentioned that in [11], Theorem 1 was proved using the Gershgorin Circle Theorem to guarantee all eigenvalues of matrix J had strictly negative real part. This was done by imposing the restriction that each circle is centered on the left half of the complex plane, with a radius smaller than the distance from its center to the origin, i.e.,

$$J_{jj}<0 \& R_j<\left|J_{jj}\right|,j\in \{1,\dots ,N\},$$

(33)

where $R_i={\sum }_{j=1,j\ne i}^N\left|A_{ij}\right|$ is the $j^{th}$ Gershgorin circle’s radius.

Let K be the number of output neurons. Noting that J is a square matrix of size $N-P$, and the cost function used is the mean squared error, when $\beta >0$, the only change to J is that $\beta$ is subtracted to the diagonal entries corresponding to output neurons (i.e., $J_{ii}$ with $N-P-K<i\le N-P$).

Assuming Corollary 1’s conditions were satisfied during an inference phase of the DEEP model, then we simply have to check they still are for $(N-K)<j\le N$:

• ${\sum }_{i=0}^N{W}_{ij}>0⟹{\sum }_{i=0}^N{W}_{ij}+\beta >0$
• Given that $\beta >0$ and the previous condition holds, $|{\sum }_{i=0}^N{W}_{ij}+\beta |>|{\sum }_{i=0}^N{W}_{ij}|>{\sum }_{i=P+1}^N\left|W_{ij}\right|$.

Therefore, if the Corollary 1’s stability conditions held during DEEP’s inference phase, they still do during its subsequent learning phase.

4.2.2. Theorem 4

The same is true with the new stability conditions defined in Theorem 4. Going back to Theorem 4’s proof, and again considering the only change in matrix J is that $\beta$ is subtracted from the diagonal entries corresponding to output neurons, we now have

$$|1-dt\sum _{i=0}^N{W}_{P+j,i}-\beta |+\sum _{i=P+1}^N|dt·W_{P+j,i}|<1.$$

(34)

However, since $0<\beta \le 1$, it can be argued that

$$|1-dt\sum _{i=0}^N{W}_{P+j,i}-\beta |\le |1-dt\sum _{i=0}^N{W}_{P+j,i}|.$$

(35)

This is because if $0<\beta <1$, for sufficiently small $dt$, (35) becomes

$$1-dt\sum _{i=0}^N{W}_{P+j,i}-\beta \le 1-dt\sum _{i=0}^N{W}_{P+j,i}\iff \beta \ge 0.$$

(36)

If $\beta =1$, again for sufficiently small $dt$, (35) becomes

$$|dt\sum _{i=0}^N{W}_{P+j,i}|\le 1-dt\sum _{i=0}^N{W}_{P+j,i},$$

(37)

which is trivial given $dt$ is, again, sufficiently small. Either way, (35) holds and

$$\begin{array}{c}|1-dt{\displaystyle \sum _{i=0}^N}{W}_{P+j,i}-\beta |+{\displaystyle \sum _{i=P+1}^N}|dt·W_{P+j,i}|\le \\ |1-dt{\displaystyle \sum _{i=0}^N}{W}_{P+j,i}|+{\displaystyle \sum _{i=P+1}^N}|dt·W_{P+j,i}|<1,\end{array}$$

(38)

where the last inequality comes from our assumption that the network satisfied Theorem 4’s conditions for stability.

4.3. Remark on the Link Between New and Previous Stability Conditions

Theorem 2 also holds if the matrix norm ∞ is used instead (see [13,14]). Then, the same reasoning used in proving Theorem 4 leads one to recover exactly the previous stability conditions (given by Corollary 1). Similarly, one can adapt Theorem 1’s proof. Recall that the Gershgorin Circle Theorem was used to ensure all the eigenvalues of matrix (19) had negative real parts. If we apply the same reasoning to the transpose of matrix (19), which has exactly the same eigenvalues, we get instead the new stability conditions. Two very different approaches to the same problem seem to yield the same results.

In this section, we presented the new DEEP model with leakage and proved stability conditions for both the learning and inference phases. In the next section, we will additionally introduce a learning rule inspired by [15] and analyze its relationship with the gradient of the objective function.

5. Generalization of Equilibrium Propagation to Vector Field Dynamics

In [15], in an attempt to generalize EP to vector field dynamics, the authors used a learning rule which, in the limit $\beta \to 0$, relates to the gradient ${\displaystyle \frac{\partial J}{\partial W}}(x,y,W)$ in a manner made explicit by Theorem 5 (see below Section 5.1).

In this section, the main steps followed in [15] to build up to the new learning rule are presented, as well as the special case of EP. Then, we apply the same reasoning to DEEP, producing a learning rule whose effectiveness we explore for a small network of $N=4$ neurons.

5.1. On the Learning Rule

In [15], the authors consider the simplified Hebbian update rule based on pre- and post-synaptic activity given by

$$d{W}_{ij}\propto \rho \left(s_i\right)d{s}_j.$$

(39)

Since for their vector field $\mu$, it holds that $\rho \left(s_i\right)={\displaystyle \frac{\partial {\mu }_j}{\partial W_{ij}}}\left(s\right)$ and for all $j^{\prime }\ne j$, ${\displaystyle \frac{\partial {\mu }_{j^{\prime }}}{\partial W_{ij}}}=0$, Equation (39) can be written in the concise form (It is important to note that in [15] and in the DEEP model, $\mu$ and V are vector fields from ${\mathbb{R}}^{L+K}$ to ${\mathbb{R}}^{L+K}$, acting on the hidden and output neurons. For that reason, in this section, the vector state s only contains the activities of those $L+K$ neurons)

$$dW\propto {\displaystyle \frac{\partial \mu }{\partial W}}{(W,s)}^T·ds.$$

(40)

The weight update is assumed to occur during the second phase of training (the learning phase), when the network’s state transitions from the first fixed point $s^0$ to the second fixed point $s^{\beta }$. By integrating (40) from $s^0$ to $s^{\beta }$, normalizing by a factor $\beta$, and then taking the limit as letting $\beta \to 0$, we arrive at the update rule $\Delta W\propto \nu \left(W\right)$, where $\nu \left(W\right)$ is the vector defined as

$$\nu \left(W\right)={\displaystyle \frac{\partial \mu }{\partial W}}{(W,s^0)}^T·{\displaystyle \frac{\partial s^{\beta }}{\partial \beta }}\left(0\right).$$

(41)

Formally, $\nu \left(W\right)$ is a vector field in the weight space. The following result is proved in [15]:

Theorem 5.

The gradient ${\displaystyle \frac{\partial J}{\partial W}}(x,y,W)$ and the vector field $\nu \left(W\right)$ can be expressed explicitly in terms of μ and C:

$${\displaystyle \frac{\partial J}{\partial W}}(x,y,W)=-{\displaystyle \frac{\partial C}{\partial s}}\left(s^0\right)·{\left({\displaystyle \frac{\partial \mu }{\partial s}}\left(s^0\right)\right)}^{-1}·{\displaystyle \frac{\partial \mu }{\partial W}}\left(s^0\right)$$

(42)

$$\nu \left(W\right)={\displaystyle \frac{\partial C}{\partial s}}\left(s^0\right){\left({\displaystyle \frac{\partial \mu }{\partial s}}{\left(s^0\right)}^{-1}\right)}^T·{\displaystyle \frac{\partial \mu }{\partial W}}\left(s^0\right),$$

(43)

where $s^0$ is the equilibrium state reached in the inference phase.

This result shows $\nu \left(W\right)$ is related to ${\displaystyle \frac{\partial J}{\partial W}}\left(W\right)$ and that the angle between these two vectors is directly related to the “degree of symmetry” of the Jacobian of $\mu$ in the fixed point $s^0$.

Particular Case of the Original EP

In the original EP [5], we have

$$\nu \left(W_{ij}\right)={\displaystyle \frac{\partial \mu }{\partial W_{ij}}}{\left(s^0\right)}^T·{\displaystyle \frac{\partial s^{\beta }}{\partial \beta }}\left(0\right)={\displaystyle \frac{\partial s_i^{\beta }}{\partial \beta }}\left(0\right)s_j^0+s_i^0{\displaystyle \frac{\partial s_j^{\beta }}{\partial \beta }}\left(0\right).$$

(44)

Since $W_{ij}=W_{ji}$ for all $i,j$, the matrix ${\displaystyle \frac{\partial \mu }{\partial s}}\left(s\right)$ is symmetric and by Theorem 5, $\nu \left(W\right)$ should coincide with the update rule used in [5], given by:

$$\Delta W_{ij}\propto \underset{\beta \to 0}{lim}{\displaystyle \frac{s_i^{\beta }{s}_j^{\beta }-s_i^0{s}_j^0}{\beta }}.$$

(45)

Under mild regularity conditions on $\mu$ and C (recall C is the cost function), the implicit function theorem ensures that, for fixed input and output $(x,y)$, the function $\beta \mapsto s^{\beta }$ is differentiable. So, we use Cauchy’s rule to evaluate (45):

$$\begin{array}{cc}\hfill \underset{\beta \to 0}{lim}{\displaystyle \frac{s_i^{\beta }{s}_j^{\beta }-s_i^0{s}_j^0}{\beta }}& =\underset{\beta \to 0}{lim}{\displaystyle \frac{\frac{\partial s_i^{\beta }}{\partial \beta }{s}_j^{\beta }+s_i^{\beta }\frac{\partial s_j^{\beta }}{\partial \beta }}{1}}=\underset{\beta \to 0}{lim}\left({\displaystyle \frac{\partial s_i^{\beta }}{\partial \beta }}{s}_j^{\beta }+s_i^{\beta }{\displaystyle \frac{\partial s_j^{\beta }}{\partial \beta }}\right)=\hfill \end{array}$$

(46)

$$\begin{array}{cc}\hfill & ={\displaystyle \frac{\partial s_i^{\beta }}{\partial \beta }}\left(0\right)s_j^0+s_i^0{\displaystyle \frac{\partial s_j^{\beta }}{\partial \beta }}\left(0\right)=\nu \left(W_{ij}\right).\hfill \end{array}$$

(47)

We have thus shown that, despite appearing to be different, the update rules $\nu \left(W\right)$ and the one used in [5] are the same.

5.2. $\nu \left(W\right)$ in the DEEP Model

When recreating the steps followed in [15] to reach $\nu \left(W\right)$ in the DEEP model, replacing the vector field $\mu$ for V, instead of Equation (39) one should start from

$$d{W}_{ij}\propto \rho \left(s_i\right)d{s}_j-\rho \left(s_i\right)d{s}_i=\rho \left(s_i\right)(d{s}_j-d{s}_i).$$

(48)

Note that due to the different version of the gradient descent considered (see Equation (8)), $s_i=\rho \left(s_i\right)$ and, similarly to Equation (40), it follows that

$$dW\propto {\displaystyle \frac{\partial V}{\partial W}}{(W,s)}^T·ds.$$

(49)

Then, the same reasoning produces the update rule

$$\nu \left(W_{ij}\right)=s_i^0·{\displaystyle \frac{\partial s_j^{\beta }}{\partial \beta }}\left(0\right)-s_i^0·{\displaystyle \frac{\partial s_i^{\beta }}{\partial \beta }}\left(0\right)=\underset{\beta \to 0}{lim}{s}_i^0{\displaystyle \frac{\left(s_j^{\beta }-s_j^0\right)-\left(s_i^{\beta }-s_i^0\right)}{\beta }},$$

(50)

which is related to ${\displaystyle \frac{\partial J}{\partial W}}(x,y,W)$, precisely as stated in Theorem 5.

Convergence of Training a Small Network

While, for symmetric weights, $\nu \left(W\right)$ equals the negative of the gradient of the objective function, this is no longer true when we allow the network to have asymmetric weights.

In [15], the authors claim that experiments run on the MNIST dataset show that the objective function J consistently decreases, but give no conditions for the alignment between $\nu \left(W\right)$ and ${\displaystyle \frac{\partial J}{\partial W}}(x,y,W)$ to happen. In an attempt to study how aligned these two vectors are in practice, we consider a network of 2 input neurons, 1 intermediate neuron, and 1 output neuron, as well as the bias neuron (see Figure 3). Then, using Theorem 5, we explicitly calculate $\mu \left(W\right)$, $-{\displaystyle \frac{\partial J}{\partial W}}\left(s^0\right)$ and compare them.

Colored in red, we can see the connections added to the DEEP model in this work. The vector field $V=[V_3,V_4]$ and the cost function C are given by

$$\begin{array}{cc}\hfill V_3\left(s\right)& =\sum _{i=0}^4{W}_{i3}{s}_i-s_3\sum _{i=0}^4{W}_{3i}\hfill \end{array}$$

(51)

$$\begin{array}{cc}\hfill V_4\left(s\right)& =\sum _{i=0}^4{W}_{i4}{s}_i-s_4\sum _{i=0}^4{W}_{4i}\hfill \end{array}$$

(52)

$$\begin{array}{cc}\hfill C(s,y)& ={\displaystyle \frac{1}{2}}{(s_4-y)}^2.\hfill \end{array}$$

(53)

To simplify, we assume there is only leakage in neuron 4 and $W_{34},W_{43}\ge 0$. Then,

$${\displaystyle \frac{\partial C}{\partial s}}\left(s^0\right)=\left[\begin{array}{cc}0& s_4^0-y\end{array}\right],$$

(54)

$${\displaystyle \frac{\partial V}{\partial s}}\left(s^0\right)=\left[\begin{array}{cc}-{\displaystyle \sum _{i=0}^4}{W}_{3i}& W_{43}\\ W_{34}& -{\displaystyle \sum _{i=0}^4}{W}_{4i}\end{array}\right],$$

(55)

and

$${\left({\displaystyle \frac{\partial V}{\partial s}}\left(s^0\right)\right)}^{-1}={\displaystyle \frac{1}{\left({\displaystyle \sum _{i=0}^4}{W}_{3i}\right)\left({\displaystyle \sum _{i=0}^4}{W}_{4i}\right)-W_{34}{W}_{43}}}\left[\begin{array}{cc}-{\displaystyle \sum _{i=0}^4}{W}_{4i}& -W_{43}\\ -W_{34}& -{\displaystyle \sum _{i=0}^4}{W}_{3i}\end{array}\right].$$

(56)

Consider the network’s weights in the order $W_{03}$, $W_{04}$, $W_{13}$, $W_{14}$, $W_{23}$, $W_{24}$, $W_{34}$, $W_{41}$, $W_{42}$, and $W_{43}$. Then, ${\displaystyle \frac{\partial V}{\partial W}}\left(s^0\right)$ is the following $2\times 10$ matrix:

$$\left[\begin{array}{cccccccccc}{s}_0& 0& s_1& 0& s_2& 0& -s_3& 0& 0& s_4\\ 0& s_0& 0& s_1& 0& s_2& s_3& -s_4& -s_4& -s_4\end{array}\right].$$

(57)

Using Theorem 5, we can explicitly compute $\nu \left(W\right)$, $-{\displaystyle \frac{\partial J}{\partial W}}\left(W\right)$ and their inner product:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \nu \left(W\right)=& {\displaystyle \frac{s_4^0-y}{det\left(\frac{\partial V}{\partial s}\right)}}(-W_{43}{s}_0,-\sum _{i=1}^4{W}_{3i}{s}_0,-W_{43}{s}_1,-\sum _{i=1}^4{W}_{3i}{s}_1,-W_{43}{s}_2,\hfill \\ \hfill & -\sum _{i=1}^4{W}_{3i}{s}_2,(W_{43}-\sum _{i=1}^4{W}_{3i})s_3,\sum _{i=1}^4{W}_{3i}{s}_4,\sum _{i=1}^4{W}_{3i}{s}_4,(-W_{43}+\sum _{i=1}^4{W}_{3i})s_4)\hfill \end{array}\end{array}$$

(58)

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill -{\displaystyle \frac{\partial J}{\partial W}}\left(W\right)& ={\displaystyle \frac{s_4^0-y}{det\left(\frac{\partial V}{\partial s}\right)}}(-W_{34}{s}_0,-\sum _{i=1}^4{W}_{3i}{s}_0,-W_{34}{s}_1,-\sum _{i=1}^4{W}_{3i}{s}_1,-W_{34}{s}_2,\hfill \\ \hfill & -\sum _{i=1}^4{W}_{3i}{s}_2,(W_{34}-\sum _{i=1}^4{W}_{3i})s_3,\sum _{i=1}^4{W}_{3i}{s}_4,\sum _{i=1}^4{W}_{3i}{s}_4,(-W_{34}+\sum _{i=1}^4{W}_{3i})s_4)\hfill \end{array}\end{array}$$

(59)

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \langle \nu \left(W\right),-{\displaystyle \frac{\partial J}{\partial W}}\left(W\right)\rangle & ={\displaystyle \frac{{(s_4^0-y)}^2}{det{\left(\frac{\partial V}{\partial s}\right)}^2}}(W_{34}{W}_{43}{s}_0^2+(\sum _{i=1}^4{W}_{3i}{)}^2{s}_0^2+W_{34}{W}_{43}{s}_1^2+(\sum _{i=1}^4{W}_{3i}{)}^2{s}_1^2+\hfill \\ \hfill & W_{34}{W}_{43}{s}_2^2+(\sum _{i=1}^4{W}_{3i}{)}^2{s}_2^2+(W_{34}-\sum _{i=1}^4{W}_{3i}\left)\right(W_{43}-\sum _{i=1}^4{W}_{3i})s_3^2+\hfill \\ \hfill & (\sum _{i=1}^4{W}_{3i}{)}^2{s}_4^2+(\sum _{i=1}^4{W}_{3i}{)}^2{s}_4^2+(W_{34}-\sum _{i=1}^4{W}_{3i})(W_{43}-\sum _{i=1}^4{W}_{3i})s_4^2)\hfill \end{array}\end{array}$$

(60)

Neuron 3 has no leakage, so $W_{34}={\displaystyle \sum _{i=1}^4}{W}_{3i}$ and this reduces to

$$\langle \nu \left(W\right),-{\displaystyle \frac{\partial J}{\partial W}}\left(W\right)\rangle ={\displaystyle \frac{{(s_4^0-y)}^2}{det{\left(\frac{\partial V}{\partial s}\right)}^2}}\left(W_{34}{W}_{43}\right(s_0^2+s_1^2+s_2^2)+W_{34}^2(s_0^2+s_1^2+s_2^2+s_4^2\left)\right)>0.$$

(61)

Since $0\le s_i^2\le 1$ and $s_0=1$ (the bias neuron has activity fixed to 1), we have

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \langle \nu \left(W\right),-{\displaystyle \frac{\partial J}{\partial W}}\left(W\right)\rangle & \ge {\displaystyle \frac{{(s_4^0-y)}^2}{det{\left(\frac{\partial V}{\partial s}\right)}^2}}\left(W_{34}{W}_{43}\right(1+0+0)+W_{34}^2(1+0+0+0\left)\right)\hfill \\ \hfill & ={\displaystyle \frac{{(s_4^0-y)}^2}{det{\left(\frac{\partial V}{\partial s}\right)}^2}}{W}_{34}(W_{34}+W_{43}).\hfill \end{array}\end{array}$$

(62)

Analogously,

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \parallel \nu \left(W\right)\parallel =& \left|{\displaystyle \frac{s_4^0-y}{det\left(\frac{\partial V}{\partial s}\right)}}\right|\left(\right(s_0^2+s_1^2+s_2^2\left)\right(W_{34}^2+W_{43}^2)+s_3^2(W_{34}-W_{43}{)}^2+\hfill \\ \hfill & {s_4^2(2W_{34}^2+{(W_{34}-W_{43})}^2))}^{\frac{1}{2}}\hfill \\ \hfill \le & \left|{\displaystyle \frac{s_4^0-y}{det\left(\frac{\partial V}{\partial s}\right)}}\right|\left(\right(1+1+1\left)\right(W_{34}^2+W_{43}^2)+1(W_{34}-W_{43}{)}^2+\hfill \\ \hfill & {1(2W_{34}^2+{(W_{34}-W_{43})}^2))}^{\frac{1}{2}}\hfill \\ \hfill =& \left|{\displaystyle \frac{s_4^0-y}{det\left(\frac{\partial V}{\partial s}\right)}}\right|{(5W_{34}^2+3W_{43}^2+2{(W_{34}-W_{43})}^2)}^{\frac{1}{2}}\hfill \end{array}\end{array}$$

(63)

and

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \parallel {\displaystyle \frac{\partial J}{\partial W}}\left(W\right)\parallel & =\left|{\displaystyle \frac{s_4^0-y}{det\left(\frac{\partial V}{\partial s}\right)}}\right|{\left(\right(s_0^2+s_1^2+s_2^2+s_4^2\left)\right(2W_{34}^2\left)\right)}^{\frac{1}{2}}\hfill \\ \hfill & \le \left|{\displaystyle \frac{s_4^0-y}{det\left(\frac{\partial V}{\partial s}\right)}}\right|{\left(\right(1+1+1+1\left)\right(2W_{34}^2\left)\right)}^{\frac{1}{2}}\hfill \\ \hfill & =\left|{\displaystyle \frac{s_4^0-y}{det\left(\frac{\partial V}{\partial s}\right)}}\right|\sqrt{8}{W}_{34}.\hfill \end{array}\end{array}$$

(64)

These computations allow us to take a lower bound on the cosine of the angle $\theta$ between $\nu \left(W\right)$ and $-{\displaystyle \frac{\partial J}{\partial W}}\left(W\right)$, which ideally would always be close to 1.

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill cos\theta & ={\displaystyle \frac{\langle \nu \left(W\right),-\frac{\partial J}{\partial W}\left(W\right)\rangle }{\parallel \nu \left(W\right)\parallel ·\parallel \frac{\partial J}{\partial W}\left(W\right)\parallel }}\hfill \\ \hfill & >{\displaystyle \frac{\frac{{(s_4^0-y)}^2}{det{\left(\frac{\partial V}{\partial s}\right)}^2}}{\frac{{(s_4^0-y)}^2}{det{\left(\frac{\partial V}{\partial s}\right)}^2}}}{\displaystyle \frac{W_{34}(W_{34}+W_{43})}{\sqrt{8}{W}_{34}\sqrt{5W_{34}^2+3W_{43}^2+2(W_{34}-W_{43}{)}^2}}}\hfill \\ \hfill & ={\displaystyle \frac{W_{34}+W_{43}}{\sqrt{8}\sqrt{5W_{34}^2+3W_{43}^2+2(W_{34}-W_{43}{)}^2}}}\hfill \end{array}\end{array}$$

(65)

Consider the function f defined as

$$\begin{array}{cc}\hfill f:{\mathbb{R}}_{\ge 0}^2& \to \mathbb{R}\hfill \\ \hfill (a,b)& \mapsto {\displaystyle \frac{a+b}{\sqrt{5a^2+3b^2+2{(a-b)}^2}}}.\hfill \end{array}$$

Writing $b=a·k$, $k\in {\mathbb{R}}_{\ge 0}$, yields

$$f(a,b)={\displaystyle \frac{a(1+k)}{\sqrt{a^2(5+3k^2+2{(1-k)}^2)}}}\iff f(a,b)={\displaystyle \frac{1+k}{\sqrt{5+3k^2+2{(1-k)}^2}}}.$$

(66)

It is easy to see that this function has a minimum of ${\displaystyle \frac{1}{\sqrt{7}}}$ when $k=0\iff b=0$ and a maximum of around $0.718$ when $k=1.286$. Furthermore, when $a=0$, $f(0,b)={\displaystyle \frac{1}{\sqrt{5}}}$ and $0.718>{\displaystyle \frac{1}{\sqrt{5}}}>{\displaystyle \frac{1}{\sqrt{7}}}$.

We conclude that

$$cos\theta >{\displaystyle \frac{1}{\sqrt{8}\sqrt{7}}}\iff \theta <82.{33}^{\circ }.$$

(67)

In particular, if $W_{43}=1.286W_{34}$, it holds that

$$\theta <75.{29}^{\circ }.$$

(68)

This guarantees some alignment between the new learning rule $\nu \left(W\right)$, presented for the DEEP model in this example network, and the gradient of the objective function. Further research is necessary to determine more general conditions for the alignment of $\nu \left(W\right)$ and the gradient of the objective function in more general networks.

6. Conclusions

In this paper, we employed dynamical systems theory to rigorously analyze learning dynamics in models of biologically plausible neural networks, encompassing both feed-forward and recurrent architectures. By critically examining the DirEcted Equilibrium Propagation (DEEP) model introduced in [11], we identified inherent stability limitations arising from the absence of an energy function. To address these, we proposed an extension of the DEEP model with a neuronal leakage term and established, via theoretical analysis, that this modification ensures convergence in both the inference and learning phases.

Additionally, we clarified the relationship among various local learning rules, demonstrating that the update mechanisms in equilibrium propagation [5] align with those in [15], reinforcing the broader applicability of these learning principles.

Our investigation into learning with asymmetric (non-symmetric) feedback weights revealed that, although such systems depart from traditional energy-based frameworks, learning can still proceed under certain conditions. Specifically, for a small four-neuron network, we derived an explicit bound on the angle between the update vector and the true gradient, showing that effective learning is possible in this restricted case. However, generalizing these convergence guarantees to larger or more complex architectures remains an open challenge.

The main limitation of our study lies in its focus on small network sizes and specific architectural assumptions. While our theoretical results offer valuable insight, practical implementation and scalability in larger, real-world networks have yet to be established. Further work is needed to derive generalized angle bounds and convergence criteria for networks of arbitrary size and complexity, as well as to explore the empirical performance of these models on benchmark tasks.

We hope that our findings shed new light on the possibilities of biologically plausible learning in neural networks and encourage further research—both theoretical and experimental—into designing robust, scalable, and biologically inspired learning algorithms.