A Consistent Estimator of Nontrivial Stationary Solutions of Dynamic Neural Fields

Abstract

Dynamics of neural fields are tools used in neurosciences to understand the activities generated by large ensembles of neurons. They are also used in networks analysis and neuroinformatics in particular to model a continuum of neural networks. They are mathematical models that describe the average behavior of these congregations of neurons, which are often in large numbers, even in small cortexes of the brain. Therefore, change of average activity (potential, connectivity, firing rate, etc.) are described using systems of partial different equations. In their continuous or discrete forms, these systems have a rich array of properties, among which is the existence of nontrivial stationary solutions. In this paper, we propose an estimator for nontrivial solutions of dynamical neural fields with a single layer. The estimator is shown to be consistent and a computational algorithm is proposed to help carry out implementation. An illustrations of this consistency is given based on different inputs functions, different kernels, and different pulse emission rate functions.

1. Introduction

It is known that any small piece of human or animal cortex contains a vast number of neurons. Therefore, a continuum approach in modeling these large ensembles makes sense and was pioneered by the work of Beurle [1]. This work was designed to accommodate only excitable networks of neurons and was subsequently generalized by Wilson and Cowan [2] to include inhibitory neurons as well. Amari [3] considered ensembles of neurons when studying pattern formation. Since then, there have been applications and extensions of his work in several directions and the birth of the field of dynamic field theory as byproduct. These extensions have for instance enabled analyses of electroencephalograms [4], short-term memory [5], visual hallucinations [6,7], and most recently robotics using dynamics neural fields. Applications to robotics has proven very effective, as shown, for instance, by the works of Bicho, Mallet, and Schöner [8], Erlhangen and Bicho [9], Erlhangen and Schoner [10], and Bicho, Louro, and Erlhagen [11]. The authors of the latter provided studies in which robots to humans interactions were implemented based on information from Dynamic Neural Fields (DNF). The theoretical aspects started in [2,3] and are summarized below.

Let $\Omega \subseteq {\mathbb{R}}^d$ be a manifold. In the presence of neurons located a position $\xi \in \Omega$ at time t arranged on L layers, the average potential function $V_k(\xi ,t)$ is often used to understand the continuous field on the kth layer. $V_k(\xi ,t)$ is the average membrane potential of the neurons located at position $\xi$ at time t of the kth layer. When $L=1$, $V(\xi ,t)$ can also be understood as the synaptic input or activation at time t of a neuron at position or direction $\xi$. It satisfies the Amari equation (see [3]), which is given as

$$\frac{\partial V_k(\xi ,t)}{\partial t}=-V_k(\xi ,t)+\sum _{l=1}^L{\int }_{\Omega }{K}_{kl}(\xi ,y)G\left(V_l(y,t)\right)dy+S_k(\xi ,t),$$

(1)

where $K_{kl}(\xi ,y)$ is the intensity of the connection between a neuron located at position $\xi$ on the kth layer with a neuron a position y on the lth layer and $G\left(V_k(\xi ,y)\right)$ is the pulse emission rate (or activity) at time t of the neuron located at position $\xi$ on the kth layer. G is often chosen as a monotone increasing function. $S_k(\xi ,t)$ represents the intensity of the external stimulus at time t arriving on the neuron at position $\xi$ on the kth layer, see Figure 1 below.

Figure 1. Illustration of the DNF for: single layer (a); and multiple layers (b).

DNFs have also branched out to dynamical systems; for instance, in [12], the authors studied a heterogeneous aspect of DNF and found existence of attractors and saddle nodes for solutions of (1). The existence of solutions of DNF is based on fixed point theory using Hammerstein integral equation (see [13]) such as in [14]. Now, based on recent developments in recurrent neural networks (RNN), Equation (1) can be discretized using nearly exact discretization schemes (see [15]) to give rise to discrete dynamical neural fields as

$$V_{k,n+1}^{\left(i\right)}=F\left(V_{k,n}^{\left(i\right)}\right):={\alpha }_n{V}_{k,n}^{\left(i\right)}+{\beta }_n\sum _{l=1}^L\sum _{j=1}^N{K}_l^{\left(ij\right)}{V}_{k,n}^{\left(j\right)}+{\eta }_{k,n}^{\left(i\right)},$$

(2)

where $V_{k,n}^{\left(i\right)}$ represents the state of the membrane potential on the neuron at position ${\xi }_i$ at time $t_n$ on the lth layer, ${\alpha }_n=1-exp(-{\delta }_n)$ (where ${\delta }_n=t_{n+1}-t_n$) is a time scale parameter, ${\beta }_n=(1-{\alpha }_n)\frac{|\Omega |}{N}$ is a parameter depending on the time scale and the size $|\Omega |$ of the manifold $\Omega$, $K_k^{ij}$ are heterogeneous weights representing the connectivity between a neuron at position ${\xi }_i$ on the kth layer with a neuron at position $y_j$ on the lth layer, and ${\eta }_{k,n}^{\left(i\right)}$ is the intensity of the external stimulus arriving at the neuron at position ${\xi }_i$ at time $t_n$ on the kth layer. We observe that (2) represents a discrete dynamical system. To study the stability analysis of the discrete dynamical system (2), one needs to first find the stationary solutions given as $V_{i,n+1}^{\left(k\right)}=V_{i,n}^{\left(k\right)}:=V_{i,*}^{\left(k\right)}$ and evaluate the derivative of the map F at these stationary solutions. This is a difficult if not impossible task if we do not know how to estimate the stationary solutions for the DNF. This is one of the main motivating factors behind the current paper.

Moreover, from Elman [16], Willams and Zipser [17], and, most recently, Durstewitz [18], this equation is also a RNN. Therefore, the tools of discrete dynamical systems can be applied non only to single-layer DNFs but also to multiple-layers DNFs, where conditions for stability are well-known.

Another interesting aspect of DNFs is that, if we restrict $\Omega$ to the unit circle $\mathbb{T}$ where $\mathbb{T}=\left\{z\in \mathbb{C}:\left|z\right|=1\right\}$, then solutions may exist in the complex unit disk $\mathbb{D}=\left\{z\in \mathbb{C}:\left|z\right|<1\right\}$; with the absence of external stimulus, such solutions would also be solutions of a Dirichlet problem associated with Equation (1) (see [19]). Indeed, suppose that $V=V(z,t)$ and let $F(z,t)=G\left(V\right(z,t\left)\right)$ for $z\in \mathbb{T}$ and $t\ge 0$, for some complex-valued function G. Consider the Poisson kernel on $\mathbb{D}$ is defined as ${\displaystyle K(z,\omega )=\frac{1}{2\pi }\frac{1-{\left|z\right|}^2}{|1-z\overline{\omega }{|}^2}}$, where $z,\omega \in \mathbb{D}$. From the theory of complex analysis (see [20]), consider then the complex single-layer Amari equation

$$\frac{\partial V(z,t)}{\partial t}=-V(z,t)+{\int }_{\mathbb{T}}K(z,\omega )G\left(V(\omega ,t)\right)d\omega +S(z,t).$$

(3)

Suppose F is a smooth function on $\mathbb{T}$ or F is a distribution (in a functional sense) on $\mathbb{T}$. If a nontrivial stationary solution $V\left(z\right)$ for this equation exists, then it satisfies the equation

$$V\left(z\right)=G\left(V\right(z\left)\right)+S\left(z\right).$$

(4)

An obvious corollary is that, if the complex function $G\left(z\right)$ has a fixed point, then a nontrivial solution for the complex Amari Equation (3) without stimulus ($S\left(z\right)=0$) when it exists, is a harmonic function in $\mathbb{D}$, in that the Laplacian operator applied to V is identically zero. Therefore, as a harmonic function, such a solution may be written as a power series ${\displaystyle V\left(z\right)=\sum _{n\in \mathbb{Z}}{a}_n{z}^n}$, where the coefficient $a_n$ are to be determined. This would be an interesting aspect of this non trivial solutions of DNF worth investigating akin to the Lotka–Volterra expansion proposed in [12].

Most analyses of DNFs focus on their applications and theoretical properties. However, given that kernels often used in practice are either Gaussian, Laplacian, or tangent hyperbolic kernels, and the function G is monotone increasing, there are avenues to also study statistical properties of the DNFs, albeit in specific situations. Indeed, the aforementioned kernels can be thought of as density functions of a random variable Y so that the integrand in Equation (1) can be viewed as the average of the random variable $G\left(V\right(Y\left)\right)$ over the manifold $\Omega$. With that understanding at hand, our goal is to use this new statistical paradigm to propose a consistent estimator for nontrivial solutions of DNF. The remainder of the paper is organized as follows. In Section 2, we state the necessary definitions and the main result. In Section 3, we propose a computation algorithm for the implementation of the estimator. In Section 4, we state the technical considerations to be used in the implementation, by proposing other functions G beyond the usual sigmoid function. In Section 5, we perform the Monte Carlo simulations based on different kernel functions. We make concluding remarks in Section 6.

2. Main Results

We observe that non-stationary solutions of (1) are given as

$${\displaystyle V_k\left(\xi \right)=\sum _{l=1}^L{\int }_{\Omega }{K}_{kl}(\xi ,y)G\left(V_l\left(y\right)\right)dy+S_k\left(\xi \right).}$$

Henceforth, for simplicity sake, given that, at up to a kernel, the stationary solutions would have the same form, we consider a dynamic neural field with a single layer, so that (1) becomes

$$\frac{\partial V(\xi ,t)}{\partial t}=-V(\xi ,t)+{\int }_{\Omega }K(\xi ,y)G\left(V(y,t)\right)dy+S(\xi ,t).$$

(5)

Let V be a stationary solution of the integro-differential Equation (5). According to Hammerstein [13], such a nontrivial solution exists if $K(x,y)$ is symmetric positive definite and G satisfies $G\left(V\right)\le {\mu }_{1}V+{\mu }_2$, where ${\mu }_1,{\mu }_2$ are positive constants. We know that V is defined over the domain $\Omega$ as

$$V\left(\xi \right)={\int }_{\Omega }K(\xi ,y)G\left(V\left(y\right)\right)dy+S\left(\xi \right)\mathrm{for}\xi \in \Omega .$$

(6)

Definition 1.

The indicator function $I_A$ of the set A is defined as

$$I_A\left(x\right)=\left\{\begin{array}{cc}1\hfill & if x\in A\hfill \\ 0\hfill & if x\notin A\hfill \end{array}\right..$$

We recall the definition of a consistent estimator.

Definition 2.

Let random sample $X_1,X_3,\cdots ,X_N$ from distribution with parameter θ. Let ${\widehat{\theta }}_N=h(X_1,X_n\cdots ,X_N)$ be an estimator of θ for some function h. Then, ${\widehat{\theta }}_n$ is said to a consistent estimator for θ if for any real number $ϵ>0$

$$\underset{N\to \infty }{lim}Pr\left(\right|{\widehat{\theta }}_N-\theta |\ge ϵ)=0.$$

Theorem 1.

For a given $\xi \in \Omega$, suppose Y is a random variable supported on Ω with probability density function distribution $f\left(y\right|\xi )=K(\xi ,y)$. Suppose G is the cumulative distribution function of some random variable U supported on $V(\Omega )$. Then, given positive integers n and m, define

$$V_{n,m}\left(\xi \right):=\frac{1}{nm}\sum _{i=1}^m\sum _{j=1}^n{I}_{[u_i,\infty )}\left(V\left(y_j\right)\right)+S\left(\xi \right),$$

(7)

where for $1\le i\le m$ and $1\le j\le n$, $u_i$ and $y_j$ are random points from U and Y, respectively.

Then, for $\xi \in \Omega$, we have that

$$V_{n,m}\left(\xi \right) is a consistent estimat or of V\left(\xi \right).$$

Proof. 

From the Markov inequality, we know that, given $\nu >0$, ${\displaystyle Pr\left(\right|V_{n,m}-V|\ge \nu )\le \frac{E\left[\right|V_{n,m}-V\left|\right]}{\nu }}$; therefore, it is enough to prove that, given $\xi \in \Omega$,

$$\underset{n,m\to \infty }{lim}{E}_{Y|\xi }\left[\right|V_{n,m}-V\left|\right]=0.$$

We have that

$$\begin{array}{ccc}\hfill |E_{Y|\xi }[V_{n,m}-V]|& =& |{\int }_{\Omega }(V_{n,m}\left(\xi \right)-V\left(\xi \right))f\left(y\right|\xi )dy|\hfill \\ & \le & |{\int }_{\Omega }\frac{1}{n}\left(\frac{1}{m}\sum _{i=1}^m{I}_{[u_i,\infty )}\left(V\left(y_j\right|\xi )\right)-Pr(U\le V\left(y_j\right|\xi ))\right)f\left(y\right|\xi )dy|+\hfill \\ & +& |{\int }_{\Omega }\frac{1}{n}\sum _{j=1}^n\left(Pr(U\le V\left(y_j\right|\xi ))-G\left(V\left(y\right|\xi )\right)\right)f\left(y\right|\xi )dy|\hfill \\ & +& |{\int }_{\Omega }\left(\frac{1}{n}\sum _{j=1}^{n}G\left(V\left(y\right|\xi )\right)-V\left(\xi \right)\right)f\left(y\right|\xi )dy|\hfill \end{array}$$

Put

$$\begin{array}{ccc}\hfill I_1& =& |{\int }_{\Omega }\frac{1}{n}\sum _{j=1}^n\left(\frac{1}{m}\sum _{i=1}^m{I}_{[u_i,\infty )}\left(V\left(y_j\right|\xi )\right)-Pr(U\le V\left(y_j\right|\xi ))\right)f\left(y\right|\xi )dy|\hfill \\ \hfill I_2& =& |{\int }_{\Omega }\frac{1}{n}\sum _{j=1}^n\left(Pr(U\le V\left(y_j\right|\xi ))-G\left(V\left(y\right|\xi )\right)\right)f\left(y\right|\xi )dy|\hfill \\ \hfill I_3& =& |{\int }_{\Omega }\left(\frac{1}{n}\sum _{j=1}^{n}G\left(V\left(y\right|\xi )\right)-V\left(\xi \right)\right)f\left(y\right|\xi )dy|\hfill \end{array}$$

Since ${\displaystyle F_m\left(x\right)=\frac{1}{m}\sum _{i=1}^m{I}_{[u_i,\infty )}\left(x\right)}$ is the empirical distribution of the random variable U, we have that

$$\underset{m\to \infty }{lim}{F}_m\left(V\left(y_j\right|\xi )\right)=Pr(U\le V\left(y_j\right|\xi ))$$

and since $f\left(y\right|\xi )$ is density function, we have

$$I_1\le \frac{1}{n}\sum _{j=1}^n\left|F_m\left(V\left(y_j\right|\xi )\right)-Pr(U\le V\left(y_j\right|\xi ))\right|\to 0as n,m\to \infty .$$

We observe that by definition $Pr(U\le V\left(y_j\right|\xi ))=G\left(V\left(y_j\right|\xi )\right)$, therefore

$$I_2\le {\int }_{\Omega }\left|\frac{1}{n}\sum _{j=1}^n[G(V\left(y_j\right|\xi )-G\left(V\left(y\right|\xi )\right)]\right|f\left(y\right|\xi )dy.$$

Given $ϵ$, by continuity of G, there exists $\delta >0$ such that

$$|V\left(y_j\right|\xi )-V\left(y\right|\xi )|<\delta \Rightarrow |G(V\left(y_j\right|\xi )-G(V\left(y\right|\xi )|<ϵ.$$

It follows that for any $ϵ>0$

$$I_2\le \frac{ϵ}{n}\to 0as n,m\to \infty .$$

To finish, we note that

$$V\left(\xi \right)={\int }_{\Omega }K(\xi ,y)G\left(V\left(y\right)\right)dy={\int }_{\Omega }G\left(V\left(y\right|\xi )\right)f\left(y\right|\xi )dy=E_{Y|\xi }[G\left(V\left(Y\right|\xi )\right].$$

Therefore,

$$\begin{array}{ccc}\hfill I_3& =& |{\int }_{\Omega }\left(\frac{1}{n}\sum _{j=1}^{n}G\left(V\left(y\right|\xi )\right)-V\left(\xi \right)\right)f\left(y\right|\xi )dy|\hfill \\ & =& |{\int }_{\Omega }\left(G\left(V\right(y\left|\xi \right))-V(\xi )\right)f\left(y\right|\xi )dy|\hfill \\ & =& |{\int }_{\Omega }G(V\left(y\right|\xi )f\left(y\right|\xi )dy-V\left(\xi \right){\int }_{\Omega }f\left(y\right|\xi )dy|\hfill \\ & =& |{\int }_{\Omega }G(V\left(y\right|\xi )f\left(y\right|\xi )dy-V\left(\xi \right)|=0\hfill \end{array}$$

This concludes the proof that $V_{n,m}$ is a consistent estimator for V. □

Remark 1.

(1) We observe that $V_{n,m}\left(\xi \right)$ depends on the knowledge of $V\left(y_i\right)$, which is not known in general. However, we observe that $V_{n,m}\left(y_j\right)$ given in Equation (7) has minimum $V_{n,m}^{min}\left(y_j\right)=S\left(y_j\right)$ and a maximum of $V_{n,m}^{max}\left(y_j\right)=\frac{1}{n}+S\left(y_j\right)$ because ${\displaystyle \frac{1}{m}\sum _{i=1}^m{I}_{[u_i,\infty )}\left(V\left(y_j\right)\right)}$ has minimum 0 and maximum 1. This also means that although we may not know exactly the value of $V_{n,m}\left(y_j\right)$, we can estimate it to be between $S\left(y_j\right)$ and $\frac{1}{n}+S\left(y_j\right)$. We can therefore select $V\left(y_i\right)$ between $S\left(y_j\right)$ and $\frac{1}{n}+S\left(y_j\right)$ for $j=1,2,\cdots ,n$ and the estimate of the nontrivial solution will exist in a small interval of length (bandwidth) ${\displaystyle \frac{1}{n}}$. Hence, if the domain Ω is very dense in points $y_i$’s, then the nontrivial solution will be a small perturbation of the initial external input, and, if the domain is sparse, then perturbation will be greater. Henceforth, for simplicity sake, we assume that $V\left(y_j\right)\sim Unif\left(S\left(y_j\right),\frac{1}{n}+S\left(y_j\right)\right)$. (2) Another observation is that S depends on the position of the neuron ξ in a single-layer system; however, in a multiple-layer system, it would be reasonable to think of S as depending on both the layer k and the position of the neuron on the layer, that is, $S\left(V_k(\xi ,t)\right)$.

3. Computational Algorithm

In this section, we have the setting of Theorem 1. From Remark 1, we can use the following algorithm to estimate $V\left(\xi \right)$:

• Step 1: Select positive integers n and m.
  Here, the experimenter should choose the values of $m,n$ relative to how much computational capabilities ones has, knowing that very large values can lead to a significant slowdown of convergence.
• Step 2: Select $y_1,y_2,\cdots ,y_n$ from the distribution $f\left(y\right|\xi )$.
  Knowing that $f\left(y\right|\xi )=K(\xi ,y)$ is a known probability distribution (Gaussian, Laplace, or tangent hyperbolic, see the section below), this should be achievable with relative ease from any software.
• Step 3: Select $u_1,u_2,\cdots ,u_m$ from the distribution of $g\left(u\right)$ of U associated with G.
  As in the previous step, sampling from a known probability distribution $g\left(u\right)$ should be achievable. However, if G is not given as bounded function between 0, and 1, we can still truncate it adequately to obtain a probability distribution (see Section 4.1).
• Step 4: For $j=1,\cdots ,n$, select $V\left(y_j\right)$ from a uniform distribution $Unif\left(S\left(y_j\right),\frac{1}{n}+S\left(y_j\right)\right)$.
  This step assumes that we have an external stimulus S arriving on the neuron at position $\xi$ given as a function of $\xi$.
• Step 5: For given $\xi \in \Omega$, evaluate ${\displaystyle V_{n,n}\left(\xi \right):=\frac{1}{nm}\sum _{j=1}^n\sum _{i=1}^m{I}_{[u_i,\infty )}\left(V\left(y_j\right)\right)+S\left(\xi \right)}$.
  In this final step, one can choose different values of $\xi$ to plot the estimator in the space $\Omega$.

We use Step 4 only to evaluate $V_{n,m}^{max}\left(\xi \right)$. From the above algorithm, it is clear that the activation function $V\left(\xi \right)$ of the neuron at position $\xi$ is the sum of the average of activations of neurons at position $y_i$ and the external stimulus arriving at $\xi$. Thus, essentially, the function $V\left(\xi \right)$ is a perturbation of the function $S\left(\xi \right)$ by the quantity ${\displaystyle \frac{1}{nm}\sum _{j=1}^n\sum _{i=1}^m{I}_{[u_i,\infty )}\left(V\left(y_i\right)\right)}$ that depends on $\xi$ and possibly of parameters of the distribution of random variables U and Y. These parameters play the role of smoothing to compensate from the noise created by small values of $n,m$ (see Section 5).

4. Technical Considerations

In this section, we discuss the choices for the pulse emission rate function G and the connection intensity $K(x,y)$.

4.1. Pulse Emission Rate Function

We note that Amari considered the dynamic of neural fields with pulse emission function G defined as the sigmoid function. However, the equation still has nontrivial stationary solution even if G is not the sigmoid. In fact, there is a large class of nonlinear functions G for which this is true (see Figure 2 and Table 1 below). For example, the following functions, often used in for training algorithm in artificial neural networks, have been adequately truncated for our purposes. Here, $\theta$ is a positive real number.

Figure 2. Five different activation functions.

Table 1. A list of potential pulse emission rate functions one can consider in applications.

Name	Formulation	Conditions
Sigmoid: $G_1\left(v\right)=$	0	if $v<v_0$
	${\displaystyle \frac{1}{1+e^{-\theta (v-v_0)}}}$	if $v\ge v_0$
Weighted Sigmoid: $G_2\left(v\right)=$	0	if $v<v_0$
	$(v-v_0)G_1\left(v\right)$	if $v\ge v_0$
Hyperbolic Tangent: $G_3\left(v\right)=$	0	if $v<v_0$
	$1-2G_1(-2v)$	if $v\ge v_0$
Tangent inverse: $G_4\left(v\right)=$	$\arctan (v-v_0)$	if $v\ge v_0$
	0	if $v<v_0$
Heaviside: $G_5\left(v\right)=$	0	if $v<v_0$
	1	if $v\ge v_0$
Ramp: $G_6\left(v\right)=$	0	if $v<v_0$
	$\theta (v-v_0)$	$v_0\le v<v_0+{\theta }^{-1}$
	1	if $v\ge v_0+{\theta }^{-1}$

Remark 2.

• (1) We observe that the choice of the sigmoid activation function $G_1\left(v\right)={(1+e^{-\theta (v-v_0)})}^{-1}$ is widely preferred in the literature for its bounded nature, without condition.
• (2) Another reason is the fact that it is also suitable when the$V_n^{\left(i\right)}$s are binary, that is, they may take the value 0 or 1, where 0 represents a non-active neuron at time n and 1 represents an active neuron at time n. In this case,${\displaystyle G_1\left(W_{i0}+\sum _{j=1}^N{W}_{ij}{V}_n^{\left(i\right)}+{\eta }_n^{\left(i\right)}\right)=Pr\left(V_{n+1}^{\left(i\right)}=1\right)}$would represent the probability that there is an activity on neuron at position${\xi }_i$at time$n+1$.
• (3) A third reason, which is important in our situation, is that it has an inverse that can be written in close form, unlike many other activation functions sometime used in the artificial neural networks (see, e.g., [21]) making it easy to generate random numbers from. The other functions would require the use of numerical inversion methods such as the bisection method, the secant method, or the Newton–Raphson method, all of which are computationally intensive (see, e.g., Chapter 4 in [22]).

4.2. Connection Intensity Function

There are various connection intensities functions (or kernel) that one can choose from. These include the Gaussian kernel $K(x,y)=\frac{1}{\sigma \sqrt{2\pi }}{e}^{-\frac{1}{2{\sigma }^2}{∥x-y∥}^2}$ introduced above. One could also consider the Laplacian kernel defined as $K(x,y)=\frac{1}{2\sigma }{e}^{-\frac{1}{\sigma }∥x-y∥}$ or the hyperbolic tangent kernel $K(x,y)=tanh(x-y)$, see Figure 3 below for an illustration.

Figure 3. Illustration of Gaussian, Laplacian, and hyperbolic tangent kernels.

5. Simulations

In each of the simulations below, we used the function $V\left(\xi \right)=sin\left(\xi \right)$ as if it were the true solution, just to evaluate $V\left(y_j\right)$ for $j=1,2,\cdots ,n$. Using the algorithm above, we then compare the estimates of $V\left(\xi \right)$ obtained using our estimator V with V unknown (red curves) and V known (blue curves), using various kernel functions $K(x,y)$ and various external stimulus functions $S\left(x\right)$. In all simulations below, we selected $n=m=100$. We used Gaussian, Laplacian, and hyperbolic tangent kernels with a sigmoid function G. The value of $\sigma$ was set as 1 for the Gaussian and Laplacian kernels.

5.1. Simulation 1: Constant External Stimulus

In this simulation, we illustrate the algorithm above by selection a constant intensity of external stimulus arriving at at point $\xi$. To check if the algorithm is correct, we select $S\left(\xi \right)\equiv 1$, with true function $V\left(\xi \right)=sin\left(\xi \right)$ (see Figure 4, Figure 5 and Figure 6).

Figure 4. The dotted line represents the input $S\left(\xi \right)=1$, which from above is similar to the $V_{n,m}^{min}$. The kernel is a Gaussian. The maximum estimator $V_{n,m}^{min}$ (red) has the same patterns as the external stimulus and the estimator $V_{n,m}$ (blue) takes the form of the true sine function.

Figure 5. The dotted line still represents the input $S\left(\xi \right)=1$, which from above is similar to the $V_{n,m}^{min}$. The kernel is Laplacian. The maximum estimator $V_{n,m}^{min}$ (red) has the same patterns as the external stimulus and the estimator $V_{n,m}$ (blue) takes the form of the true sine function, but noisier at low $n,m$ and progressively smoother at high values of $n,m$.

Figure 6. The dotted line represents the input $S\left(\xi \right)=1$, which from above is similar to the $V_{n,m}^{min}$. The kernel is a Hyperbolic tangent. The maximum estimator $V_{n,m}^{min}$ (red) has the same patterns as the external stimulus and the estimator $V_{n,m}$ (blue) takes the form of the initial external input, unable to replicate the form of the true since function, even at high values of $n,m$.

5.2. Simulation 2: Logarithm External Stimulus

In this simulation, we illustrate the algorithm above by selection a constant intensity of external stimulus arriving at at point $\xi$. To check if the algorithm is correct, we select $S\left(\xi \right)=ln(1+{\xi }^2)$, with true function $V\left(\xi \right)=sin\left(\xi \right)$ (see Figure 7, Figure 8 and Figure 9).

Figure 7. The dotted line represents the input $S\left(\xi \right)=ln(1+x^2)$, which from above is similar to the $V_{n,m}^{min}$. The kernel is Gaussian. The maximum estimator $V_{n,m}^{min}$ (red) has the same patterns as the external stimulus and the estimator $V_{n,m}$ (blue) is a distorted version of the original since function, with distortion that is increased around 0, which is caused by the presence of $S\left(\xi \right)$ that is in the neighborhood of 0 dominating the sine function that we know is close to 0 in a small neighborhood of 0. However, as we get farther from zero, the influence of external stimulus wanes and the estimator starts to take the shape of the true since function.

Figure 8. In this case, we use a Laplacian Kernel and we observe a similar pattern as above. However, the estimator is much noisier. There is also a noticeable phase difference between the estimations from a Gaussian kernel and a laplacian kernel.

Figure 9. In this case, we used hyperbolic tangent kernel and clearly the sine pattern of the true function is never recovered. This suggests that the external input is overwhelming the noise, even after a close look within the interval [5.0,5.1].

5.3. Simulation 3: Exponentially Decaying External Stimulus

In this simulation, we illustrate the algorithm above by selection an intensity of external stimulus arriving at at point $\xi$ as $S\left(\xi \right)=e^{-{\xi }^2}$, with true function $V\left(\xi \right)=sin\left(\xi \right)$ (see Figure 10, Figure 11 and Figure 12).

Figure 10. Clearly, with this external inputs, the situation is different from the above cases. In a small neighborhood of 0, we still have the external stimulus dominating; the noise and the estimates $V_{n,m}$ (blue) remains between $V_{n,m}^{max}$ and $V_{n,m}^{min}$. However, as we move farther away from 0, $V_{n,m}^{max}$ takes the shape of $S\left(\xi \right)$ but it oscillates between $S\left(x\right)$ and $V_{n,m}$. This is explained but the fact that $exp(-{\xi }^2)$ is traded with $sin\left(\xi \right)$ due to the periodic nature of the latter. The estimator on the other hand reproduces the expected pattern is this where a Gaussian kernel is used.

Figure 11. In this case, the kernel is Laplacian and the observations are the same as above. However, we observe much more noise in the estimator, together with a phase shift.

Figure 12. In this case, the expected pattern is not reproduced due to the external stimulus dominating and the estimator serving as noise.

5.4. Simulation 4: Mexican Hat True Function

In [12], the authors used a Gaussian kernel to obtain an estimate of the nontrivial solution that had the form of a Mexican hat function in the space $\Omega$. Our method differs from theirs in two aspects: First, they assumed that $K(x,y)=V_0\left(x\right)·V_0\left(y\right)$ where $V_0\left(x\right)={\int }_{\Omega }K(x,y)G\left(V_0\left(y\right)\right)dy$ which implies that $1={\int }_{\Omega }{V}_0\left(y\right)G\left(V_0\left(y\right)\right)dy$. This would restrict us to only independent random variables X and Y with marginals $V_0\left(x\right)$ and $V_0\left(y\right)$. We do not make such an assumption because there are many kernels (bivariate functions) that cannot be factored as the product of two marginals. Second, we do not make the assumption that $G\left(V\right(y\left)\right)$ has power series about certain state $\overline{V\left(y\right)}$, so that ${\displaystyle G\left(V\left(y\right)\right)=\sum _{n=0}^{\infty }\frac{G^{\left(n\right)}\left(y\right)}{n!}{\left[V\left(y\right)-\overline{V\left(y\right)}\right]}^n}$. This assumption would obviously fail for the Heaviside and Ramp functions. The main reason for the difference is that they were interested in second-order synaptic dynamics, which is not the case here. In this section, we show that our method still yields a comparable estimate even without these assumptions. Indeed, in [12], the authors showed that the true solution obtained with a Gaussian kernel in the space has the form of a Mexican hat function. In this simulation, we use our estimator to verify this fact, that is, we set our external stimulus as a Gaussian distribution with mean zero and standard deviation 0.03 as in their case and compare the estimates obtained from the use of a Gaussian, Laplacian, and hyperbolic tangent kernels (see Figure 13).

Figure 13. The curves are estimates of $V\left(\xi \right)$ using the proposed estimator for different kernels, Gaussian (blue), Laplacian (red), and hyperbolic tangent (green) with a mean zero Gaussian external stimulus with standard deviation 0.03. Clearly, all kernels yield the expected pattern: (a) the sigmoid function $G_1$; (b) the Heaviside function $G_5$; (c) the Ramp function $G_6$ with $\theta =3.5$; and (d) the hyperbolic tangent inverse function $G_3$.

5.5. Discussion

(1) The conclusion we can draw from the first simulation is that Gaussian and Laplacian kernels both fare well when the external input is constant. The latter produces noisier outputs at low resolution values $n,m$ and becomes smother at high resolution values $n,m$.

(2) The major takeaway from the second and third simulations is that the external stimulus can have a significant effect on the estimator, especially near boundary points where there is a significant frequency change between the true value and the external stimulus. In reality, in practical applications, the true value is not known; therefore, a careful choice on the external input is needed if one would like to obtain accurate estimations.

(3) The fourth simulation shows that estimates obtained using a Gaussian kernel are smoother. As for the pulse rate function G, the sigmoid function $G_1$ fares much better for all three kernels used in comparison to other functions.

(4) Ultimately, the point of the first three simulations is the hope to extend this type of estimator beyond solutions of DNF. In fact, Equation (1) can be considered a linear Boltzmann equation with stochastic kernel if, instead of thinking of $\xi$ as the position of a neuron, we think of it as the velocity of a particle and $V(x,t)$ as the velocity distribution overtime. In this case, nontrivial solutions are now solutions of the nonlinear Markov operator $Pg\left(v\right(\xi \left)\right)=v\left(\xi \right)$ where ${\displaystyle Pg\left(v\left(\xi \right)\right):={\int }_{\Omega }K(\xi ,y)g\left(v\left(y\right)\right)dy}$, which by the Hille–Yosida theorem exist. The proposed estimator provides in a sense another way of thinking about this problem using successive approximations (see [23]).

(5) A drawback of the estimator is that it depends on a great first guest of the solutions at points $y_i\in \Omega$. However, if enough of these points are selected, one stands a great chance of obtaining a good approximation.

(6) An advantage of this estimator is that, locally, it is a great point estimator of the value of $V\left(\xi \right)$ for a given $\xi \in \Omega$, and given $y_1,y_2,\cdots ,y_n\in \Omega$. As mentioned in Section 3, although we may not know the values of $V\left(y_i\right),i=1,\cdots ,n$ needed to evaluate $V\left(\xi \right)$, one way to go around the issue is to select them uniformly from a small interval of length $\frac{1}{n}$.

(7) We observe that the computational aspect of our algorithm depends on Monte Carlo Simulations. This is not the only way of efficiently achieve this. One may also choose to achieve this using methods such as sparse grids and Bayesian Monte Carlo with appropriate priors on the parameters (see, e.g., [24,25,26,27,28]).

(8) One other possible use of this estimator is that it can help initialize a RNN algorithm or help find the phase space diagram in a discrete dynamical system with two different layers.

6. Conclusions

In this paper, we propose an estimator for nontrivial solutions of dynamic neural fields. The proposed estimator is shown to be consistent. Moreover, the proposed estimator exists within a small interval depending on the number of points selected in the domain $\Omega$ where these nontrivial solutions are defined.

The choice of the kernel, as in previous studies, is shown to be crucial in determining the accuracy of the estimates.

We also show that Gaussian kernels provide the best balance among accuracy, smoothness, and the number of points used. In the space domain, the estimates obtained are visually similar to those obtained using, for instance, Lotka–Volterra series, as in [12]. The proposed estimator has the advantage that it is simple to implement and may serve as initial guess or initialization when for example using a recurrent neural networks to find nontrivial solutions in the time and space domain. This is particularly important in robotics and to a certain extent in neuroinformatics because it could potentially help with accuracy of movements of robots.

In this paper, we also show how the DNF can be extended to functional and complex analysis, which could further extend theoretical properties of DNFs using techniques from these areas. The proposed estimator in this paper can be used to initialize a discrete dynamical system associated with the DNF.

The present work could be useful for new insights into the connection between DNF and dynamical systems and overall contribute to the literature in these areas and in computational neuroscience.