Dynamics Analysis of a New Fractional-Order Hopfield Neural Network with Delay and Its Generalized Projective Synchronization

Abstract

In this paper, a new three-dimensional fractional-order Hopfield-type neural network with delay is proposed. The system has a unique equilibrium point at the origin, which is a saddle point with index two, hence unstable. Intermittent chaos is found in this system. The complex dynamics are analyzed both theoretically and numerically, including intermittent chaos, periodicity, and stability. Those phenomena are confirmed by phase portraits, bifurcation diagrams, and the Largest Lyapunov exponent. Furthermore, a synchronization method based on the state observer is proposed to synchronize a class of time-delayed fractional-order Hopfield-type neural networks.

1. Introduction

Fractional calculus has a history of 300 years. In the early days, because of its computational complexity and lack of intuitive physical and geometric explanations, it did not attract the interest of researchers [1,2,3]. Recently, people have discovered that using fractional calculus to describe many natural phenomena will be more accurate, such as biomedical engineering [4], fractional control [5], and specific physical problems [6]. The fractional calculus model is considered as an excellent tool to describe the hereditary and memory characteristics of various processes due to a memory term in the model [1].

The Hopfield neural network (HNN), which can store memories like the human brain does, is considered as one of the most influential neural networks [7]. It is natural to consider that incorporating the two memory terms (fractional calculus and the Hopfield neural network) will potentially be a huge step toward the enhancement of both memory characteristics and the efficiency of information processing [8]. Studies have shown that chaos, which appears in the human brain, plays a vital role in memory storage in a biological neural network [9,10,11]. It is known to us that there exists a delay effect on information processing in intra-neural communication. Therefore, studying the chaotic fractional-order Hopfield neural network (FHNN) with time-delay is a very significant issue that can narrow the gap between biological neuronal systems and artificial neural networks. Due to the complex dynamical behaviors of chaotic systems, they have a wide range of applications such as secure communication [12]. Besides, compared with the integer-order chaotic system, the fractional-order chaotic system exhibits higher nonlinearity and larger key space due to the existence of fractional-order derivatives [13].

Research on synchronization is still attractive due to its potential applications in control processing and secure communication [14]. Various methods of chaotic synchronization have been proposed recently, such as impulse control [15], linear matrix inequalities (LMI) [16], sliding mode control [17] and many others based on observer theory to chaotic synchronization [18]. A well-known method is based on the state observer where the response system is designed to behave as an observer of the drive system [19]. Recently, many researchers have begun to pay attention to the synchronization problem of fractional-order chaotic systems, such as phase synchronization, antiphase synchronization, and generalized projective synchronization.

In the last few years, many papers on FHNN have been published [20,21,22,23,24,25,26,27]. Unfortunately, only a few of them studied the chaotic characteristics and generalized projective synchronization (GPS) schemes of time-delayed FHNN. Motivated by the discussions above, in this paper, we propose a new class of time-delayed FHNN and analyze the rich dynamics, then design its GPS scheme.

The remainder of this paper is organized as follows. In Section 2, the basic definition of fractional derivatives, the stability criterion of the fractional-order system, and the numerical algorithm of time-delayed FHNN are presented. In Section 3, the new time-delayed FHNN with three neurons is introduced. Its complex dynamics like chaos, periodicity, and stability are analyzed. Its chaotic nature is confirmed by phase portraits, bifurcation diagrams, and the Largest Lyapunov exponent. In Section 4 a GPS scheme between two identical time-delayed FHNN based on the state observer is introduced. Finally, in Section 5, some conclusions are given.

2. Preliminaries and Numerical Algorithm

2.1. Preliminaries

There exist some definitions of fractional derivatives; in this article, we will use the Caputo derivative due to its wider range of engineering applications. The Caputo derivation definition is as follows [1]:

$$\begin{array}{c}\hfill D^{q}x(t)=D^{-(n-q)}\frac{d^n}{d{t}^n}x(t)=\frac{1}{\mathsf{\Gamma }(n-q)}{\int }_{t0}^t{(t-\tau )}^{n-q-1}{x}^n(\tau )d\tau \end{array}$$

(1)

where $q(q>0)$ is the fractional-order, $n-1<q<n\in Z^{+},$ and $\mathsf{\Gamma }(·)$ is the gamma function $\mathsf{\Gamma }(x)={\int }_0^{\infty }{t}^{x-1}{e}^{-t}dt.$

The stability theory of fractional derivative system is not the same as that of the integer system. For an n-dimensional fractional-order system:

$$D^q(X)=g(X),$$

(2)

where $D^q(X)={(D^{q_1}(x_1),D^{q_2}(x_2),\cdots ,D^{q_n}(x_n))}^T,X=(x_1,x_2,\cdots ,x_n).$

Like the integer case, the roots of the equation $g(x)=0$ are the equilibrium points of this fractional system.

Theorem 1.

[28] For an n-dimensional fractional-order system, if all the eigenvalues $({\lambda }_1,{\lambda }_2,\cdots ,{\lambda }_n)$ of the Jacobian matrix of some equilibrium points satisfy:

$$\begin{array}{c}\hfill \left|arg({\lambda }_i)\right|>a\pi /2,a=max(q_1,q_2,\cdots ,q_n)(i=1,2,\cdots ,n).\end{array}$$

(3)

then the system is asymptotically stable at one point.

Lemma 1.

If there exists one equilibrium point that is stable, then the fractional system will be stable; if and only if all equilibrium points are unstable, then the fractional system will be unstable, maybe periodic or chaotic.

Lemma 2.

For n = 3, the equilibrium point is called a saddle point with index two, if one of the eigenvalues ${\lambda }_1<0$ and the other two conjugate eigenvalues $\left|arg({\lambda }_2)\right|=\left|arg({\lambda }_3)\right|<q\pi /2$; the equilibrium point is called a saddle point with index one, if one of the eigenvalues ${\lambda }_1>0$ and the other two conjugate eigenvalues $\left|arg({\lambda }_2)\right|=\left|arg({\lambda }_3)\right|>q\pi /2$. Both of these two types of saddle points are unstable.

2.2. Numerical Algorithm

Many numerical algorithms to solve the differential equations of fractional-order (FDE) have been proposed [29,30,31]. We will use the widely-used method, which is modified based on the Adams–Bashforth–Moulton predictor-corrector scheme, to solve time-delayed differential equations of fractional-order (FDDE) [31]. The method is described below.

Consider the general FDDE:

$$\begin{array}{c}{D}_t^{q}y(t)=f(t,y(t),y(t-\tau )),t\in [0,T],q\in (0,1]\hfill \\ y(t)=m(t),t\in [-\tau ,0].\hfill \end{array}$$

(4)

Consider a uniform grid $\{t_n=nh:n=-l,-l+1,\cdots ,-1,0,1,\cdots ,N\}$, in which $l\in Z,N\in Z$ and $l=\tau /h,N=T/h$. Let:

$$y_h(t_k)=m(t_k),k=-l,-l+1,\cdots ,-1,0.$$

(5)

and denote:

$$y_h(t_k-\tau )=y_h(kh-lh)=y_h(t_{k-l}),k=0,1,\cdots ,N.$$

(6)

Suppose that we have already obtained the approximations $y_h(t_k)(k=1,2,\cdots ,n),$ then we will calculate $y_h(t_{n+1})$ according to:

$$y(t_{n+1})=m(0)+\frac{1}{\mathsf{\Gamma }(q)}\underset{0}{\overset{t_{n-1}}{\int }}{(t_{n+1}-\zeta )}^{q+1}f(\zeta ,y(\zeta ),y(\zeta -\tau ))d\zeta .$$

(7)

We obtain Equation (7) by applying fractional integration on both sides of (4). Then, we use approximations $y_h(t_n)$ for $y(t_n)$. Thus, the corrector formula is as follows:

$$\begin{array}{cc}{y}_h(t_{n+1})\hfill & =m(0)+\frac{h^q}{\mathsf{\Gamma }(q+2)}f(t_{n+1},y_h(t_{n+1}),y_h(t_{n+1}-\tau ))+\frac{h^q}{\mathsf{\Gamma }(q+2)}{\displaystyle \sum _{k=0}^n}{a}_{k,n+1}f(t_j,y_h(t_j),y_h(t_j-\tau ))\hfill \\ & =m(0)+\frac{h^q}{\mathsf{\Gamma }(q+2)}f(t_{n+1},y_h(t_{n+1}),y_h(t_{n+1-l}))+\frac{h^q}{\mathsf{\Gamma }(q+2)}{\displaystyle \sum _{k=0}^n}{a}_{k,n+1}f(t_j,y_h(t_j),y_h(t_{j-l})),\hfill \end{array}$$

(8)

where:

$$\begin{array}{c}{a}_{k,n+1}=\left\{\begin{array}{cc}{n}^{q+1}-(n-q){(n+1)}^q,& k=0\\ {(n-k-2)}^{q+1}+{(n-k)}^{q+1}\\ -2{(n-k+1)}^{q+1},& 1<k<n\\ 1,& k=n+1\end{array}\right.\hfill \end{array}$$

(9)

Then, we use a “predictor” $y_h^P(t_{n+1})$ to replace the term $y_h(t_{n+1})$ on the right side of (8).

where:

$$\begin{array}{cc}{y}_h^P(t_{n+1})\hfill & =m(0)+\frac{1}{\mathsf{\Gamma }(q)}{\displaystyle \sum _{k=0}^n}{b}_{k,n+1}f(t_k,y_h(t_j),y_h(t_k-\tau ))\hfill \\ & =m(0)+\frac{1}{\mathsf{\Gamma }(q)}{\displaystyle \sum _{k=0}^n}{b}_{k,n+1}f(t_k,y_h(t_j),y_h(t_{k-l})),\hfill \\ b_{k,n+1}\hfill & =\frac{h^q}{q}({(n+1-k)}^q-{(n-j)}^q).\hfill \end{array}$$

(10)

In the remaining part of this paper, we use the algorithm above for numerical simulation and take the step size h = 0.01.

3. Dynamic Analysis of This New Time-Delayed FHNN

3.1. System Description

The time-delayed FHNN model can be presented as the following fractional-order differential equation:

$$\begin{array}{cc}{D}_t^q(x_i(t))& =-c_i{x}_i(t)+{\displaystyle \sum _{j=1}^n}{W}_{ij}f(x_j(t))+{\displaystyle \sum _{j=1}^n}{T}_{ij}f(x_j(t-{\tau }_{ij}))+I_i,i=1,2,\cdots ,n,\hfill \\ x_i(t)& =m_i(t),t\in [-\tau ,0],0<q<1,\hfill \end{array}$$

(11)

where q is the Caputo differential operator, n represents the number of units in a Hopfield neural network, $c_i$ is a positive real number, $c_i$ represent the self-regulating coefficients of the neurons, $W_{ij}$ and $T_{ij}$ are real constant numbers, $W_{ij}$ represents the synaptic connection coefficient of the neuron j on the neuron i at time t, and $T_{ij}$ represents the synaptic connection coefficient of the neuron j on the neuron i at time $t-{\tau }_{ij}$. $f(x)$ represents the activation to its incoming potentials of the unit j at time t, also known as the activation function, and ${\tau }_{ij}$ is the delay, while $I_i$ represent the external input.

In this article, we consider time-delayed FHNN with three neurons; then, System (11) can be rewritten with the vector form as follows:

$$\begin{array}{cc}{D}_t^q(x(t))& =-Cx(t)+Wf(x(t))+Tf(x(t-\tau ))+I\hfill \\ x(t)& =m(t),t\in [-\tau ,0],0<q<1,\hfill \end{array}$$

(12)

where $x(t)={(x_1(t),x_2(t),x_3(t))}^T$, corresponds to the state vector at time t of three neurons, $f(x(t))=tanh(x(t),$ $f(x(t))$, the activation function that we chose as the hyperbolic tangent, and $f(x(t))=tanh(x(t)$,$m(t)={(0,0,0)}^T$, is the initial condition. $I={(0,0,0)}^T$ is the external input vector. The weight matrix largely determines the characteristics of the entire network, such as good memory characteristics, classification characteristics, and chaos characteristics.

After the elaborate design, our connection weight matrix is defined as follows:

$$\begin{array}{c}\hfill C=\left(\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right)W=\left(\begin{array}{ccc}-1.5& -19.9& -3.2\\ 20& -0.9& 0.7\\ 1.4& 2.79& -2.3\end{array}\right)T=\left(\begin{array}{ccc}0& 19.9& 0\\ 0& 0& 0\\ 0& 0& 0\end{array}\right)\end{array}$$

(13)

The topology connection of System (12) is the full connection as Figure 1 shows.

3.2. Dynamic Analysis

Firstly, we consider the case in which the delay $\tau =0;$ then, System (12) becomes an FHNN with three neurons, and the system of equations becomes:

$$\begin{array}{cc}\left(\begin{array}{c}{D}^q(x_1(t))\hfill \\ D^q(x_2(t))\hfill \\ D^q(x_3(t))\hfill \end{array}\right)& =\left(\begin{array}{c}-x_1(t)\hfill \\ -x_2(t)\hfill \\ -x_3(t)\hfill \end{array}\right)+\left(\begin{array}{ccc}-1.5& 0& -3.2\\ 20& -0.9& 0.7\\ 1.4& 2.79& -2.3\end{array}\right)\ast \left(\begin{array}{c}tanh(x_1(t))\hfill \\ tanh(x_2(t))\hfill \\ tanh(x_3(t))\hfill \end{array}\right)\hfill \\ & =\left(\begin{array}{c}-x_1(t)\hfill \\ -x_2(t)\hfill \\ -x_3(t)\hfill \end{array}\right)+W^{*}\ast \left(\begin{array}{c}tanh(x_1(t))\hfill \\ tanh(x_2(t))\hfill \\ tanh(x_3(t))\hfill \end{array}\right)\hfill \end{array}$$

(14)

Then, the topology connection of System (14) is no longer the full connection, which is shown in Figure 2.

When we consider the equilibrium points of the fractional-order system, it is the same as the integer case. Because all principal minors of $-W^{*}$ are nonnegative, System (14) has a unique equilibrium point [32]. Obviously, the equilibrium point is the origin. The eigenvalues of the corresponding Jacobian matrix are ${\lambda }_1=-8.047,{\lambda }_{2,3}=0.174\pm 4.956i$. According to Lemma 1, if Condition (3) is satisfied, the system will be convergent. Otherwise, the system may be periodic or chaotic once Condition (3) is not satisfied. We can obtain $arg({\lambda }_1)=\pi ,\left|arg({\lambda }_2)\right|=\left|arg({\lambda }_3)\right|=1.536$. If $q>\frac{2}{\pi }·\left|arg({\lambda }_2)\right|=0.977$ is satisfied, the origin will be a saddle point with index two, the system may be periodic or chaotic. If $q<\frac{2}{\pi }·\left|arg({\lambda }_2)\right|=0.977$ is satisfied, the origin will be a stable point, and the system will be convergent. Figure 3 and Figure 4 are the simulation results of System (14) with order 0.9 and 0.99, respectively, consistent with the analysis results above.

Time-delay can cause changes in system stability [19]. Here, we consider system changes at different delays while fixing $q=0.9.$ When $\tau =0.5,$ as shown in Figure 5, the system will be unstable, which is periodic; the system trajectory is periodic and oscillates around the equilibrium point. Increasing the delay to 1.8, we can learn from Figure 6 that the system becomes chaotic, one chaotic attractor exists in the system trajectory. If we continue increasing the delay, we can see the intermittent chaos phenomenon. When delay $\tau =3.3$, the system trajectory, as shown in Figure 7, becomes periodic again. When increasing the delay to four, the system becomes chaotic again, as shown in Figure 8; the chaotic attractor exists once again. In order to see intermittent chaos intuitively, we draw the bifurcation diagrams of $x(t)$ versus $\tau$ shown in Figure 9. In order to further confirm the phenomenon above, we calculate the largest Lyapunov exponents(LLE) of the above system by the TISEAN package [33], shown in Figure 10.

4. Generalized Projective Synchronization of Time-Delayed FHNN

In this section, we discuss the GPS of the system above. Synchronization of the fractional-order system based on the state observer has been reported in many articles [34,35,36,37]. We will extend this method to a class of time-delayed FHNN. A time-delayed FHNN system serves as the driving system, which can be rewritten as follows:

$$\begin{array}{cc}\hfill D_t^q(x(t))& =-Cx(t)+W\{f(x(t))+Zf(x(t-\tau ))\},\hfill \\ \hfill x(t)& =m(t),t\in [-\tau ,0],0<q<1,\hfill \\ \hfill u(t)& =Kx(t)+\{f(x(t))+Zf(x(t-\tau ))\}.\hfill \end{array}$$

(15)

where $x(t)={(x_1(t),x_2(t),\cdots ,x_n(t))}^T,f(x(t))=tanh(x(t)),m(t)={(0,0,\cdots ,0)}^T,C,W$ and $Z\in R^{nxn}$ are constant matrices, $u(t)$ is the synchronized vector signal, and $K\in R^{nxn}$ will be determined later.

The response system can be designed as follows:

$$\begin{array}{cc}\hfill D_t^q(y(t))& =-Cy(t)+W\{f(y(t))+Zf(y(t-\tau ))\}+W(u(t)-v(t)),\hfill \\ \hfill y(t)& =m(t),t\in [-\tau ,0],0<q<1,\hfill \\ \hfill v(t)& =Ky(t)+\{f(y(t))+Zf(y(t-\tau ))\}.\hfill \end{array}$$

(16)

where $y(t)={(y_1(t),y_2(t),\cdots ,y_n(t))}^T,f(y(t))=tanh(y(t)),m(t)={(0,0,\cdots ,0)}^T,C,W$ and $Z\in R^{nxn}$ are constant matrices, $v(t)$ is the observer predictor of $u(t)$, and $K\in R^{nxn}$ will be determined later.

The GPS error signal of System (15) and System (16) can be written as $e(t)=y(t)-\epsilon x(t);\epsilon$ is the scaling factor, which is a non-zero real number. Obviously, if the scaling factor is selected as one, the synchronization will become complete synchronization. The fractional-order error signal is determined as follows:

$$D_t^q(e(t))=D_t^q(y(t)-\epsilon x(t)),$$

according to the basic properties of fractional-order operations, the error signal can be expanded as:

$$D_t^q(e(t))=D_t^q(y(t)-\epsilon x(t))=D_t^q(y(t))-\epsilon D_t^q(x(t)),$$

then we replace the term $D_t^q(y(t))$ and $D_t^q(x(t))$ with System (16) and System (15), respectively. The error system becomes:

$$\begin{array}{cc}\hfill D_t^q(e(t))=& -Cy(t)+W\{f(y(t))+Zf(y(t-\tau ))\}+W(\epsilon u(t)-v(t))\hfill \\ & -\epsilon \{-Cx(t)+W\{f(x(t))+Zf(x(t-\tau ))\}\},\hfill \end{array}$$

now, we replace the term $u(t)$ and $v(t)$ with System (15) and System (16), respectively. The error system can be simplified as:

$$\begin{array}{cc}\hfill D_t^q(e(t))& =-Cy(t)+W\{f(y(t))+Zf(y(t-\tau ))\}+W\{\epsilon \{Kx(t)+\{f(x(t))+Zf(x(t-\tau ))\}\}\hfill \\ & -Ky(t)-\{f(y(t))+Zf(y(t-\tau ))\}\}+\epsilon Cx(t)-\epsilon W\{f(x(t))+Zf(x(t-\tau ))\}\hfill \\ & =(-C-WK)(y(t)-\epsilon x(t))\hfill \\ & =(-C-WK)e(t)\hfill \end{array}$$

(17)

System (15) and System (16) will achieve GPS if the error signal eventually stabilizes at the origin. We can easily get that the origin is one of the equilibrium points of System (17). According to Theorem 1 and Lemma 1, if all the eigenvalues $({\lambda }_1,{\lambda }_2,\cdots ,{\lambda }_n)$ of the Jacobian matrix of origin satisfy:

$$\left|arg({\lambda }_i)\right|>q\pi /2,i=1,2,\cdots ,n,$$

(18)

then the error system will be asymptotically stable at one point. We can select some eigenvalues satisfying Inequality (18), because C and W are known system parameters, and applying some knowledge of matrix theory and with the aid of MATLAB, we can obtain the desired K.

For example, we use the system we proposed in Section 3.2 and set $q=0.9.$ Values for the connection weight matrix C, W, and Z are as follow:

$$C=\left(\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right)W=\left(\begin{array}{ccc}-1.5& -19.9& -3.2\\ 20& -0.9& 0.7\\ 1.4& 2.79& -2.3\end{array}\right)Z=\left(\begin{array}{ccc}0& -0.0021& 0\\ 0& -0.8365& 0\\ 0& -1.0159& 0\end{array}\right).$$

For the simplicity of calculation, we can select eigenvalues as: ${\lambda }_1=-1,{\lambda }_2=-2,{\lambda }_3=-3;$ at this time, Inequality (18) is satisfied. We can get the desired K by solving the following matrix equation:

$$-\left(\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right)-\left(\begin{array}{ccc}-1.5& -19.9& -3.2\\ 20& -0.9& 0.7\\ 1.4& 2.79& -2.3\end{array}\right)\ast K=\left(\begin{array}{ccc}-1& 0& 0\\ 0& -2& 0\\ 0& 0& -3\end{array}\right),$$

after some simple calculations, we obtain:

$$K=\left(\begin{array}{ccc}0& 0.0489& 0.0301\\ 0& -0.0071& 0.1126\\ 0& 0.0212& -0.7146\end{array}\right).$$

(19)

Substituting Equation (19) into Equation (17), the error system becomes:

$$\left(\begin{array}{c}{D}_t^{0.9}(e_1(t))\hfill \\ D_t^{0.9}(e_2(t))\hfill \\ D_t^{0.9}(e_3(t))\hfill \end{array}\right)=\left(\begin{array}{ccc}-1& 0& 0\\ 0& -2& 0\\ 0& 0& -3\end{array}\right)\left(\begin{array}{c}{e}_1(t)\hfill \\ e_2(t)\hfill \\ e_3(t)\hfill \end{array}\right).$$

(20)

Obviously, System (20) will stabilize at the origin. Figure 11 is the time evolution of the error signal, which shows that System (20) will converge to the origin. That is to say, System (15) and System (16) achieve GPS.

5. Conclusions

In this paper, a novel time-delayed FHNN with three neurons is proposed. The rich dynamical behaviors of this new system are analyzed theoretically and verified by numerical simulations. Without delay, this new system presents stability or periodicity with different fractional-orders. Along with the increase of delay, the system presents an intermittent chaos phenomenon when the fractional-order is fixed. The GPS of two time-delayed FHNN with the same structure are achieved by using a synchronization method based on the state observer. The novel time-delayed FHNN can be used as a new chaos generator, which can be applied in many engineering applications such as secure communication. How to implement this system and synchronization method with actual circuits and how to apply them in secure communication will be discussed in the future.