Inertial Neural Networks with Unpredictable Oscillations

: In this paper, inertial neural networks are under investigation, that is, the second order differential equations. The recently introduced new type of motions, unpredictable oscillations, are considered for the models. The motions continue a line of periodic and almost periodic oscillations. The research is of very strong importance for neuroscience, since the existence of unpredictable solutions proves Poincaré chaos. Sufﬁcient conditions have been determined for the existence, uniqueness, and exponential stability of unpredictable solutions. The results can signiﬁcantly extend the role of oscillations for artiﬁcial neural networks exploitation, since they provide strong new theoretical and practical opportunities for implementation of methods of chaos extension, synchronization, stabilization, and control of periodic motions in various types of neural networks. Numerical simulations are presented to demonstrate the validity of the theoretical results.

Oscillations are in a focus of neuroscience, since they correlate with many cognitive tasks. For example, oscillatory neural networks are productive for investigation of image recognition [13,14], as well as activating network states associated with memory recall [15]. They became the core of interdisciplinary research which unites psychophysics, neuroscience, cognitive psychology, biophysics, and computational modeling [16,17]. Chaoticity in neural networks analysis is reflected by data related to experiments and observations [18][19][20].
Artificial neural networks are involved in computing systems and are designed to simulate the way the human brain analyzes and processes information. They are in the foundation of artificial intelligence and solves problems that would prove impossible or difficult by human or statistical standards. Artificial neural networks allow modeling of nonlinear processes, and they have turned into a very popular and useful tool for solving many problems such as classification, clustering, regression, pattern recognition, structured prediction, machine translation, anomaly detection, decision-making, and visualization [21][22][23][24][25][26].
In the present research, a new type of oscillation, which is described by the unpredictable functions, and was introduced in [27], is investigated. The line of periodic and almost periodic motions of neural networks is continued. The Poincaré chaos has already been approved by the existence of unpredictable solutions for a Hopfield type neural network [28] and shunting inhibitory cellular neural networks [29]. The notion of the unpredictable function is strictly connected to the concept of the unpredictable point [27,30,31]. This has been confirmed in papers by Miller [32], Thakur, and Dus [33] in which unpredictable points are very useful for analysis of Poincaré, strongly Ruelle-Takens and strongly Auslander-Yorke chaos in topological spaces. The unpredictable functions are introduced for the analysis of chaos through methods of differential and discrete equations [34,35]. The study of these functions indicates theoretical advantages and problems that apply to both oscillation theory and chaos theory, and this opens up many interesting perspectives in neuroscience.
Our main purpose is to give the conditions ensuring the existence, uniqueness, and asymptotic stability of the unpredictable oscillations in inertial neural networks.

Preliminaries
Throughout the paper, R and N will stand for the set of real and natural numbers, respectively.
.., p, p ∈ N, will be used. The following definition is the main one in our study.

Definition 1 ([27]).
A uniformly continuous and bounded function v : R → R p is unpredictable if there exist positive numbers 0 , δ and sequences t n , u n both of which diverge to infinity such that v(t + t n ) → v(t) as n → ∞ uniformly on compact subsets of R and v(t + t n ) − v(t) ≥ 0 for each t ∈ [u n − δ, u n + δ] and n ∈ N.
The convergence of the sequence v(t + t n ) is said to be Poisson stability of the unpredictable function or simply Poisson stability as well as existence of the numbers 0 , δ and the sequence u n allow the unpredictability property of the unpredictable function.
In this paper, we consider the following inertial neural networks: where t, x i ∈ R, i = 1, . . . p, p denotes the number of neurons in the network, x i (t) with i = 1, . . . p, corresponds to the state of the unit i at time t, b i > 0, a i > 0 are constants, f i with i = 1, . . . p, denote the measures of activation to its incoming potentials of the unit i at time t, c ij for all i, j = 1, . . . p, are constants, which denote the connection strength between ith neuron and jth neuron, v i (t) are external inputs on the ith neuron at the time t. We assume that the coefficients c ij are real, the activations f i : R → R are continuous functions that satisfy the following condition: By introducing the following variable transformation the neural network (1) can be written as According to the results in [36], the couple x(t) = (x 1 (t), x 2 (t), · · · , x p (t)), y(t) = (y 1 (t), y 2 (t), · · · , y p (t)) is a bounded solution of (3), if and only if the next integral equations are satisfied: where i = 1, · · · , p.
The following conditions will be needed throughout the paper: (C2) the functions v i (t), i = 1, · · · , p, in system (1) are unpredictable; they belong to Σ and there exist positive numbers δ, 0 > 0 and the sequence s n → ∞ as n → ∞, such that |v There exist positive real numbers ξ i , ζ i , such that the following inequalities are valid:

Main Results
In this section, we will use the norm · 1 for dimensions p and 2p. Denote by Σ the set of vector-functions, ϕ(t) = (ϕ 1 , · · · , ϕ 2p ), such that: Proof. For a function φ(t) ∈ Σ and fixed i = 1, 2, · · · , p, we have that Let us fix a positive number and a section [a, b], −∞ < a < b < ∞. We will show that, for sufficiently large n, it is true that Choose numbers c < a and ξ > 0, satisfying the following inequalities: 2H Consider the number n sufficiently large such that |φ (10) Now, inequalities (6)- (9) imply that Πφ(t + t n ) − Πφ(t) 1 < , for t ∈ [a, b]. Since is an arbitrarily small number, condition (K3) is satisfied. Condition (K1) follows from the boundedness of its derivative. The lemma is proved.

Lemma 2. The operator Π is a contraction mapping on Σ.
Proof. For any u, v ∈ Σ, one can attain that The last inequality yields Πu − Πv 1 = max Hence, in accordance with conditions (C4),(C7), the operator Π is contractive.
Proof. Let us check the completeness of the space Σ. Consider a Cauchy sequence φ k (t) in Σ, which converges to a limit function φ(t) on R. Since conditions (K1) and (K2) are easy to verify, it suffices to show that φ(t) satisfies condition (K3). Fix a closed and bounded interval I ⊂ R. We obtain that If n and k are sufficiently large, then each term on the right-hand side of (11) is smaller than 3 for an arbitrary and t ∈ I. This means that φ(t + t n ) uniformly converges to φ(t) on I. The completeness of space Σ is verified. From Lemmas 1, 2 and contraction mapping theorem, it follows that there exists a unique solution ω(t) ∈ Σ of Equation (1).
Next, we prove the unpredictability property. It is true that There exist a positive number κ and integers l and k such that, for each i = 1, · · · , 2p, the following inequalities are valid: Let the numbers κ, l and k as well as numbers n ∈ N, and i = 1, ..., 2p be fixed. Firstly, consider the following two alternatives for i = p + 1, ..., 2p: (i) |ω i (t n + s n ) − ω i (s n )| ≥ 0 /l; (ii) |ω i (t n + s n ) − ω i (s n )| < 0 /l such that the remaining proof falls naturally into two parts.
That is, one can conclude that ω(t) is an unpredictable solution. Now, we will discuss the stability of the solution ω(t). Let us define the 2p-dimensional function z(t) = (x 1 (t), · · · , x p (t), y 1 (t), · · · , y p (t)), and rewrite system (3) in the vector form where It is true that Let ψ(t) = (ψ 1 , ψ 2 , ..., ψ 2p ) be another solution of system (1). One can write We denote by λ = min Then, we have that Applying the Gronwall-Bellman Lemma, one can attain that and condition (C8) implies that ω(t) is a uniformly exponentially stable solution of system (1). The theorem is proved.
In the next section, the following lemmas is necessary.

Lemma 3 ([34]
). If the function φ(t) : R → R is unpredictable, then the function φ(t) + C, where C is a constant, is also unpredictable.

Examples
According to results in [27], the logistic map where i ∈ Z and F µ (s) = µs(1 − s), has an unpredictable solution.
Let Ω(t) be a piecewise constant function which is defined by is the unpredictable solution of system (21). As a function of external inputs, we will use an unpredictable function Θ(t) [28]: Let us take into account the system where i = 1, 2, 3 a 1 = 6, a 2 = 5, is unpredictable in accordance with Lemmas 3 and 4. The conditions (C1)-(C8) hold for the network (23) with ξ 1 = ξ 2 = 2, ξ 3 = 3, ζ 1 = ζ 2 = 4, ζ 3 = 4.4, L = 0.35, M f = 0.56, H = 2. It is not difficult to calculate that L F = 0.57, λ = 1.47. Consequently, there exists the unpredictable solution, ϕ(t), of the system. Since it is not possible to indicate the initial value of the solution, we apply the property of asymptotic stability, since any solution from the domain ultimately approaches the unpredictable oscillation. That is, to visualize the behavior of the unpredictable oscillation, we consider the simulation of another solution. We shall simulate the solution ω(t) with initial conditions ω 1 (0) = 1.023, ω 2 (0) = 1.516, ω 3 (0) = 0.275. Utilizing (20), we have that Thus, if t > 9 10 (5ln10 + ln4) ≈ 11.77, then ϕ(t) − ω(t) 1 < 10 −5 , and we can say that the graphs of the functions match visually, since they have technical conditions [37]. In other words, what is seen in Figures 1 and 2 for a sufficiently large time can be accepted as parts of the graph and trajectory of the unpredictable solution. Both of the figures reveal the irregular dynamics of system (23).

Conclusions
We believe that the theorem that has been proved in the paper creates new circumstances in neuroscience, when extension, control, synchronization, and stabilization of periodic motions in chaotic dynamics can be realized more effectively than by conservative methods. The results are suitable for numerical simulations, and the dynamics can be subdued for more extended numerical analysis similar to the Lyapunov exponents evaluation and bifurcation diagram construction. The huge area of applications for the present research suggestions are nonlinear neural networks. For these models, the averaging method and different types of fixed point theorems can be utilized.
Application of the approach suggested in the present research can be effective with respect to different inertial mechanisms for dynamical systems and artificial neural networks. One can consider the applicability in the light of fractional human models [21] as well as artificial neural networks for uncertainly prediction [24], modeling the human driver [25], decision-making [26], classification [22], and machine translation [23].