Learning High-Dimensional Chaos Based on an Echo State Network with Homotopy Transformation

Abstract

Learning high-dimensional chaos is a complex and challenging problem because of its initial value-sensitive dependence. Based on an echo state network (ESN), we introduce homotopy transformation in topological theory to learn high-dimensional chaos. On the premise of maintaining the basic topological properties, our model can obtain the key features of chaos for learning through the continuous transformation between different activation functions, achieving an optimal balance between nonlinearity and linearity to enhance the generalization capability of the model. In the experimental part, we choose the Lorenz system, Mackey–Glass (MG) system, and Kuramoto–Sivashinsky (KS) system as examples, and we verify the superiority of our model by comparing it with other models. For some systems, the prediction error can be reduced by two orders of magnitude. The results show that the addition of homotopy transformation can improve the modeling ability of complex spatiotemporal chaotic systems, and this demonstrates the potential application of the model in dynamic time series analysis.

1. Introduction

In recent years, machine learning technologies have been widely applied to a variety of tasks, such as speech recognition, medical diagnosis, autonomous driving, image encryption and recommendation systems [1,2,3,4]. Chaos control has always been the focus of nonlinear research, and using machine learning technology to solve this problem has gradually become a trend [5,6,7,8]. We note that usually only finite time series data from certain dynamic processes are available. Thus, this method of learning only from the data itself is called “model-free” learning. The most commonly used method for model-free learning using dynamic time series is delayed coordinate embedding, which has been well established [9,10,11,12,13].

However, delayed coordinate embedding is too complex, and the results often fail to meet the accuracy required by the project. In 2004, the ESN proposed by Jaeger and Haas achieved impressive results in “model-free” chaotic learning tasks, which was published in Science [14]. In addition, many researchers have subsequently applied ESNs to various chaotic learning tasks. For example, Pathak et al. used reservoir computing to perform model-free estimates of the state evolution of chaotic systems and the Lyapunov exponents [15,16]. Moreover, ESN can also infer the unmeasured state variables from a limited set of continuously measured variables [17]. An ESN is very different from a traditional neural network; the difference is that an ESN only needs to train the output weight, and it overcomes the problem of gradient disappearance and explosion when the traditional neural network uses gradient descent on the weight matrix [18]. Therefore, In the following years, many results using ESNs have emerged [19]. For instance, adaptive reservoir computing can capture critical transitions in dynamical systems. This network has been successful in predicting critical transitions in various low-dimensional dynamical systems or high-dimensional systems with simple parameter structures [20]. Moreover, data-informed reservoir computing, which relies solely on data to enhance prediction accuracy, not only effectively reduces computational costs but also minimizes the cumbersome hyperparameter optimization process in reservoir computing [21].

The above results show that echo state networks can be effectively applied to chaos prediction tasks, and our goal is to achieve long-term and accurate predictions. However, chaotic systems are extremely sensitive to initial conditions, which makes long-term predictions more challenging. In the ESN structure, nonlinear activation can simulate the nonlinear relationship of the chaotic system, model data characteristics, and solve complex problems [22,23], so it is very important for the completion of the task.

The update process of the reservoir state largely depends on the activation function [24,25]. The activation function is a function of the network input, the previous state, and the feedback output. According to the reservoir update equation, the network input plays a crucial role in determining the reservoir state update. Different learning tasks involve distinct input characteristics, necessitating different reservoir update methods. However, in traditional ESN models, regardless of the characteristics of the input data, the activation function usually remains unchanged, typically using fixed nonlinear functions such as tanh or sigmoid [26]. Additionally, when noise or interference in the training set increases, the generalization ability of the ESN may decrease [27]. To overcome the shortcomings of traditional single activation functions, in recent years, the double activation function echo state network (DAF-ESN) [28], the echo state network activation function based on bistable stochastic resonance (SR-ESN) [29], and the deep echo state network with multiple activation functions (MAF-DESN) have been proposed [30]. By linearly combining activation functions, the resulting activation function varies as the coefficients change, providing greater flexibility and adaptability than single activation functions. This enhances the network’s expressive power, allowing the model to better adapt to complex learning tasks.

Recognizing this, in order to learn the key features of spatiotemporal chaotic systems, this paper introduces the homotopy transformation in topological theory and proposes a new chaotic prediction model, called the H-ESN. Under the premise of maintaining basic topological properties, our model achieves the optimal balance between nonlinearity and linearity by continuously transforming between different activation functions and adjusting the homotopy parameter, thereby capturing the key features necessary for learning chaos. In the experimental part of this paper, Our model has been successfully applied to the following classical prototype systems in chaotic dynamics: Lorenz system, MG system, and KS system, and it has obtained the following positive results compared to other models.

• With appropriately chosen parameters, the H-ESN can provide longer prediction times for various high-dimensional chaotic systems.
• Under the same parameter conditions, the H-ESN demonstrates smaller prediction errors compared to other models when predicting different dimensions of chaotic systems.
• Compared to traditional methods, the H-ESN exhibits significant advantages in chaotic prediction tasks, particularly in the estimation of the maximal Lyapunov exponent.

The remainder of this paper is organized as follows: Section 2 introduces the principles and methods of the ESN and H-ESN and provides the sufficient conditions for the H-ESN to satisfy the echo state property. Section 3 discusses the application of the H-ESN to three chaotic system examples and compares its performance with other models, achieving significant results. Section 4 summarizes our research findings and outlines future research directions.

2. Correlation Method

2.1. Echo State Network

The ESN, proposed by Jaeger and Haas, has a relatively simple structure and requires only a small number of parameter adjustments [31]. Compared to traditional neural networks, it trains just a portion of the network’s connection weights, specifically, the output weights, while the input weights and the weights of the recurrent connections within the reservoir are randomly generated and remain fixed [32,33]. This simplification makes the learning process faster and more efficient. It can significantly reduce computational costs and helps mitigate the vanishing gradient problem to some extent. Furthermore, when the number of reservoir nodes is much greater than one, we can expect a wide range of desired outputs. ESN is a machine learning framework that has been shown to reproduce the chaotic attractors of its dynamical systems and includes the fractal dimension and the Lyapunov exponential spectrum [34,35,36].

The operation of the ESN during the training phase is shown in Figure 1a; the D-dimensional input vector $\mathbf{u}\left(t\right)={[u_1\left(t\right),u_2\left(t\right),\cdots ,u_D\left(t\right)]}^T$ is mapped into the reservoir R with $D_r$ nodes via the input coupling $\mathbf{I}/\mathbf{R}$ and the input weight matrix ${\mathbf{W}}_{in}\in {\mathbb{R}}^{D_r\times D}$. The reservoir state evolves according to Equation (1)

$$\begin{array}{c}\hfill \mathbf{r}(t+\Delta t)=(1-\alpha )\mathbf{r}\left(t\right)+\alpha \tan \mathrm{h}(\mathbf{A}\mathbf{r}\left(t\right)+{\mathbf{W}}_{in}\mathbf{u}\left(t\right)+\xi \mathbf{e}),\end{array}$$

(1)

for the vector $\mathbf{g}={[g_1,g_2,\cdots ,g_{D_r}]}^T$, the activation function tanh$\left(\mathbf{g}\right)$ is expressed as (tanh($g_1$), tanh($g_2$), ⋯, tanh${\left(g_{D_r}\right))}^T$, which is the hyperbolic tangent function tanh(x). $\alpha$ is the leakage rate, which adjusts the update speed of the reservoir states and affects the dynamic characteristics of the system. $\xi \mathbf{e}$ is the biased term, where $\mathbf{e}={(1,1,\cdots ,1)}^T\in {\mathbb{R}}^{D_r\times 1}$. This introduces an additional degree of freedom to the reservoir, allowing the network to better fit complex nonlinear dynamic systems. In addition to $\Delta t,\alpha ,\xi ,$ and ${\mathbf{W}}_{in}$ in Equation (1), the reservoir dynamics are also related to the spectral radius $\rho$ and sparsity d of the matrix $\mathbf{A}\in {\mathbb{R}}^{D_r\times D_r}$. The spectral radius $\rho$ determines the stability and memory capacity of the reservoir, while the sparsity d influences the computational efficiency and dynamic diversity of the reservoir by controlling the proportion of non-zero elements in the matrix A. In Equation (2), ${\mathbf{W}}_{out}\in {\mathbb{R}}^{D_v\times D_r}$ is linearly combined with the $D_r$-dimensional vector r(t) to obtain the output vector $\mathbf{v}\left(t\right)\in {\mathbb{R}}^{D_v\times 1}$ as follows

$$\begin{array}{c}\hfill \mathbf{v}(t+\Delta t)={\mathbf{W}}_{out}\mathbf{r}(t+\Delta t).\end{array}$$

(2)

Generally, the output v(t) obtained in Figure 1a is expected to approximate the desired output ${\mathbf{v}}_d\left(t\right)$. During the training phase $-T\le t\le 0$, the data for u(t) and ${\mathbf{v}}_d\left(t\right)$ are already known. The output weight matrix ${\mathbf{W}}_{out}$ is determined by minimizing Equation (3) using ridge regression

$$\begin{array}{c}\hfill \sum _{-T\le t\le 0}\parallel {\mathbf{W}}_{out}\mathbf{r}\left(t\right)-{\mathbf{v}}_d{\left(t\right)\parallel }^2+\beta {\parallel {\mathbf{W}}_{out}\parallel }^2,\end{array}$$

(3)

where $\beta >0$ is the penalty parameter, ${\parallel ·\parallel }^2$ is the sum of squares of each element, and the minimized loss function’s deduced as follows

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \parallel {\mathbf{W}}_{out}\mathbf{r}\left(t\right)-\mathbf{v}{v}_d{\left(t\right)\parallel }^2+\beta {\parallel {\mathbf{W}}_{out}\parallel }^2& ={({\mathbf{W}}_{out}\mathbf{r}\left(t\right)-{\mathbf{v}}_d\left(t\right))}^T({\mathbf{W}}_{out}\mathbf{r}\left(t\right)-{\mathbf{v}}_d\left(t\right))\hfill \\ & +\beta {\mathbf{W}}_{out}^T{\mathbf{W}}_{out},\hfill \end{array}\end{array}$$

(4)

$$\begin{array}{c}\hfill \begin{array}{cc}& {\displaystyle \frac{\partial }{\partial {\mathbf{W}}_{out}}}[{({\mathbf{W}}_{out}\mathbf{r}-{\mathbf{v}}_d)}^T({\mathbf{W}}_{out}\mathbf{r}-{\mathbf{v}}_d)+\beta {\mathbf{W}}_{out}^T{\mathbf{W}}_{out}]=0,\hfill \\ & {\displaystyle \frac{\partial }{\partial {\mathbf{W}}_{out}}}[{\mathbf{r}}^T{\mathbf{W}}_{out}^T{\mathbf{W}}_{out}\mathbf{r}-{\mathbf{r}}^T{\mathbf{W}}_{out}^T{\mathbf{v}}_d-{\mathbf{v}}_d^T{\mathbf{W}}_{out}\mathbf{r}+{\mathbf{v}}_d^T{\mathbf{v}}_d+\beta {\mathbf{W}}_{out}^T{\mathbf{W}}_{out}]=0,\hfill \\ & {\displaystyle \frac{\partial }{\partial {\mathbf{W}}_{out}}}[{\mathbf{r}}^T{\mathbf{W}}_{out}^T{\mathbf{W}}_{out}\mathbf{r}-2{\mathbf{r}}^T{\mathbf{W}}_{out}^T{\mathbf{v}}_d+{\mathbf{v}}_d^T{\mathbf{v}}_d+\beta {\mathbf{W}}_{out}^T{\mathbf{W}}_{out}]=0,\hfill \\ & 2{\mathbf{W}}_{out}\mathbf{r}{\mathbf{r}}^T-2{\mathbf{v}}_d{\mathbf{r}}^T+2\beta {\mathbf{W}}_{out}=0,\hfill \\ & 2{\mathbf{W}}_{out}\mathbf{r}{\mathbf{r}}^T+2\beta {\mathbf{W}}_{out}=2{\mathbf{v}}_d{\mathbf{r}}^T,\hfill \\ & {\mathbf{W}}_{out}(\mathbf{r}{\mathbf{r}}^T+\beta \mathbf{I})={\mathbf{v}}_d{\mathbf{r}}^T,\hfill \\ & {\mathbf{W}}_{out}={\mathbf{v}}_d{\mathbf{r}}^T{(\mathbf{r}{\mathbf{r}}^T+\beta \mathbf{I})}^{-1}.\hfill \end{array}\end{array}$$

(5)

After completing the training phase $-T\le t\le 0$ and obtaining the output weight matrix, the system enters the prediction phase $t>0$. The desired method is to make the expected output equal to the input, i.e., ${\mathbf{v}}_d(t+\Delta t)=\mathbf{u}(t+\Delta t)$, and after obtaining the output weight matrix, at $t=0$, the system transitions from Figure 1a to Figure 1b and autonomously operates according to the following formula

$$\mathbf{r}(t+\Delta t)=(1-\alpha )\mathbf{r}\left(t\right)+\alpha \tan \mathrm{h}(\mathbf{A}\mathbf{r}\left(t\right)+{\mathbf{W}}_{in}\left({\mathbf{W}}_{out}\mathbf{r}\left(t\right)\right)+\xi \mathbf{e}).$$

(6)

2.2. Echo State Network Based on Homotopy Transformation (H-ESN)

2.2.1. Introduction to H-ESN

The activation function refers to the nonlinear function in neurons. The introduction of nonlinear characteristics into artificial neural networks improves the expression ability of the networks [37]. If a network lacks a nonlinear function, it can only perform linear combinations, making the activation function crucial in neural networks.

This section introduces an ESN model based on homotopy theory. The concept of homotopy originated from topological theory and has been applied to recovery algorithms. The core of the homotopy method lies in the use of continuous changing ideas and homotopy paths.

Theorem 1

([38]). Let $f,g:X\to Y$ be a continuous function on a topological space. A homotopy from f to g is a continuous function $F:X\times [0,1]\to Y$, such that for all $x\in X,F(x,0)=f\left(x\right)$ and $F(x,1)=g\left(x\right)$. If such a homotopy exists between f and g, then f is said to be homotopic to g for all $x\in X$. Denote this by $f\simeq g$. F is continuous on t, which means that the transformation from f to g is continuous, that is, the path from one function to another.

Theorem 2

([38]). Let C be a convex set. If for $x,y\in C$, then, there is $\theta \in [0,1],\theta x+(1-\theta )y\in C$, so we consider $F(x,\theta )=\theta f\left(x\right)+(1-\theta )g\left(x\right)$, because C is a convex set, so for any $f\left(x\right),g\left(x\right)\in C$ and for any $\theta \in [0,1],\theta f\left(x\right)+(1-\theta )g\left(x\right)\in C$, that is, a function from $X\times [0,1]$ to C, so F is homotopy.

On the premise of maintaining the basic topological properties, our model can obtain the key features of chaos for learning through a continuous homotopy transformation between different activation functions. The activation function of our model is shown below

$$\begin{array}{c}\hfill F(x,\theta )=(1-\theta )\left({\displaystyle \frac{e^x-e^{-x}}{e^x+e^{-x}}}\right)+\theta x,\end{array}$$

(7)

in which $\theta$ is the homotopy parameter. Figure 2 shows the graph of $F(x,\theta )$ as the function transitions from $tanh$ to x for different values of $\theta$.

To provide a detailed explanation of the H-ESN training and prediction process, the specific flow of the algorithm, as well as its time and space complexity, are presented below (Algorithm 1).

Algorithm 1 H-ESN standard algorithm process.

Require:  Observed data $\mathbf{u}\left(t\right)$; Dimensions of the input data, reservoir, and predicted data is $D\times 1$, $D_r\times D_r$ and $D_v\times 1$. Ensure:  Predicted data $\mathbf{v}\left(t\right)$   1: Fix a random seed and generate the input weight matrix ${\mathbf{W}}_{in}$ and reservoir weight matrix $\mathbf{A}$.   2: Set the spectral radius $\rho$ and sparsity d of $\mathbf{A}$.   3: Determine the homotopy parameter $\theta$.   4: for $i=-T$ to 0 do   5:     Update the reservoir state $\mathbf{r}\left(t\right)$ using Equation (1) with the activation function replaced by $F(x,\theta )$.   6: end for   7: Obtain the output weight matrix ${\mathbf{W}}_{out}$ by minimizing Equation (3).   8: for $t=0$ to $T_{test}$ do   9:     Use Equation (6) with the activation function replaced by $F(x,\theta )$ to predict the output.   10: end for   11: Obtain the H-ESN prediction results.

2.2.2. Echo State Property of the H-ESN

The Echo State Property (ESP) is the core theoretical foundation of the ESN, ensuring that the dynamic behavior of the reservoir has good stability and predictability [39,40]. A sufficient condition for the H-ESN to exhibit the echo state property is given below (

Table 1. Time complexity and space complexity of the H-ESN algorithm.

Time Complexity: $\mathcal{O}(\mathit{d}\times {\mathit{D}}_{\mathit{r}}+(\mathit{T}+{\mathit{T}}_{\mathit{t}\mathit{e}\mathit{s}\mathit{t}})\times {\mathit{D}}_{\mathit{r}}^2+{\mathit{D}}_{\mathit{r}}^3)$	Generating Reservoir Weight Matrix A: $\mathcal{O}(\mathit{d}\times {\mathit{D}}_{\mathit{r}})$
	Reservoir State Update: $\mathcal{O}(T\times D_r^2)$
	Output Weight Matrix: $\mathcal{O}\left(D_r^3\right)$
	Prediction Phase: $\mathcal{O}(T_{test}\times D_r^2)$
Space Complexity: $\mathcal{O}((d+T+D_v)\times D_r+D_v\times T_{test})$	Reservoir Weight Matrix A: $\mathcal{O}(d\times D_r)$
	State Matrix: $\mathcal{O}(T\times D_r)$
	Output Weight Matrix: $\mathcal{O}(D_r\times D_v)$
	Storing Prediction Results: $\mathcal{O}(D_v\times T_{test})$

).

Assumption 1.

The nonlinear activation function $f:X\to Y$ is a function from set X to set Y. There exists a constant $L\ge 0$ such that for any two points $x_1,x_2$ in X, the Lipschitz condition is satisfied

$$\parallel f(u,x_1)-f(u,x_2)\parallel \le L\parallel x_1-x_2\parallel$$

where L is a constant, typically taken as 1.

Theorem 3.

If the following conditions are satisfied for the H-ESN model

(1) 

$F\left(x\right)=(1-\theta )tanh+\theta x$ (the parameter θ satisfies $\theta \in [0,1]$);

(2) 

The spectral radius ${\rho }_A$ of the internal weight matrix A of the reservoir satisfies ${\rho }_A<1$;

The H-ESN model has the ESP.

Proof. 

Consider two arbitrary initial states ${\mathbf{r}}_1\left(0\right)$ and ${\mathbf{r}}_2\left(0\right)$, and the same input sequence $\mathbf{u}\left(n\right)$. As $n\to \infty$, the state difference is $\Delta \mathbf{r}\left(n\right)={\mathbf{r}}_1\left(n\right)-{\mathbf{r}}_2\left(n\right)$. According to the state update equation

$$\begin{array}{cc}\hfill \mathbf{r}(n+1)& =(1-\alpha )\mathbf{r}\left(n\right)+\alpha ((1-\theta )tanh(\mathbf{A}\mathbf{r}\left(n\right)+{\mathbf{W}}_{in}\mathbf{u}\left(n\right)+\xi \mathbf{e})\hfill \\ \hfill & +\theta (\mathbf{A}\mathbf{r}\left(t\right)+{\mathbf{W}}_{in}\mathbf{u}\left(n\right)+\xi \mathbf{e})),\hfill \\ \hfill \Delta \mathbf{r}(n+1)& =(1-\alpha )\Delta \mathbf{r}\left(n\right)+\alpha \left((1-\theta )\right(tanh(\mathbf{A}{\mathbf{r}}_1\left(n\right)+{\mathbf{W}}_{in}\mathbf{u}\left(n\right)+\xi \mathbf{e})\hfill \\ \hfill & -tanh(\mathbf{A}{\mathbf{r}}_2\left(n\right)+{\mathbf{W}}_{in}\mathbf{u}\left(n\right)+\xi \mathbf{e}))+\theta (\mathbf{A}\Delta \mathbf{r}\left(n\right)),\hfill \end{array}$$

The tanh function is Lipschitz continuous, with a Lipschitz constant of 1.

$$\begin{array}{cc}\hfill \parallel \Delta \mathbf{r}(n+1)\parallel & \le (1-\alpha )\parallel \Delta \mathbf{r}\left(n\right)\parallel +\alpha \left(\right(1-\theta )\parallel \mathbf{A}\parallel \parallel \Delta \mathbf{r}(n)\parallel +\theta \parallel \mathbf{A}\parallel \parallel \Delta \mathbf{r}(n)\parallel ),\hfill \\ \hfill & =(1-\alpha )\parallel \Delta \mathbf{r}\left(n\right)\parallel +\alpha \parallel \mathbf{A}\parallel \parallel \Delta \mathbf{r}\left(n\right)\parallel ,\hfill \\ \hfill & \le (1-\alpha )\parallel \Delta \mathbf{r}\left(n\right)\parallel +\alpha {\rho }_A\parallel \Delta \mathbf{r}\left(n\right)\parallel ,\hfill \\ \hfill & <(1-\alpha )\parallel \Delta \mathbf{r}\left(n\right)\parallel +\alpha \parallel \Delta \mathbf{r}\left(n\right)\parallel ,\hfill \\ \hfill & =\parallel \Delta \mathbf{r}\left(n\right)\parallel ,\hfill \\ \hfill \parallel \Delta \mathbf{r}(n+1)\parallel & <\parallel \Delta \mathbf{r}\left(n\right)\parallel .\hfill \end{array}$$

The ESN activated by homotopy transformation Equation (7) is a new chaotic prediction method proposed in this paper. In Section 3, we show the effect of predicting chaotic Lorenz, MG, and KS systems based on the new ESN model (H-ESN). □

3. Results

We will provide three examples, the Lorenz, Mackey–Glass, and Kuramoto–Sivashinsky systems, to illustrate the advantages of using the H-ESN in predicting chaotic systems.

3.1. Lorenz System

The Lorenz system, proposed by Edward Lorenz in 1963 [41], is a three-dimensional nonlinear dynamical system originally designed to study atmospheric convection. As a fundamental model in chaos theory, it is known for its simplicity and complex dynamics. The system’s differential equations are as follows

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \left\{\begin{array}{cc}& \mathrm{d}x/\mathrm{d}t=-ax+ay,\hfill \\ & \mathrm{d}y/\mathrm{d}t=bx-y-xz,\hfill \\ & \mathrm{d}z/\mathrm{d}t=-cz+xy,\hfill \end{array}\right.\end{array}\end{array}$$

(8)

where $a=10,b=28,$ and $c=8/3$. The system variables $x,y,z$ are known, and the input $\mathbf{u}\left(t\right)={(x\left(t\right),y\left(t\right),z\left(t\right))}^T$ is used to obtain the output weight matrix ${\mathbf{W}}_{out}$ through training. Afterward, the system enters the prediction phase for $t>0$. Taking into account the symmetry of the Lorenz equations, Equation (2) is modified to ${\mathbf{v}}_d\left(t\right)={\mathbf{W}}_{out}\tilde{\mathbf{r}}$, where $\tilde{\mathbf{r}}$ is a vector of dimensions $D_r$ in which half of the elements of $\tilde{\mathbf{r}}$ are $\tilde{\mathbf{r}}={\mathbf{r}}_i^2$, with $i^{th}$ representing the components of $\tilde{\mathbf{r}}$. Based on this, we compare the H-ESN with other commonly used ESN models, and the results are illustrated in Figure 3, with the parameters shown in

Table 2. Parameters for the Lorenz system prediction task.

Parameter	Value	Parameter	Value
$D_r$	300	$\alpha$	1
$\rho$	0.7	$\xi$	0
$\Delta$ t	0.02	$\theta$	0.7

. Additionally, the accurate prediction data lengths for the three models on the three variables of the Lorenz system are presented in

Table 3. Accurate prediction data lengths for the three variables of the Lorenz system using the ESN, Deep-ESN, and H-ESN.

	ESN	Deep-ESN	H-ESN
Lorenz-x	378	262	521
Lorenz-y	382	260	522
Lorenz-z	393	273	530

.

According to Figure 3 and Table 3, in the initial stages, all three models—Deep ESN, ESN, and H-ESN—can achieve relatively accurate predictions for the three variables of the Lorenz system. However, as the number of data points increases, the prediction trajectory of the Deep ESN deviates first from the true values, with the purple dotted line diverging from the blue solid line. This is because we selected a Deep ESN with three layers, each containing 100 nodes, which results in weaker nonlinear modeling capability compared to an ESN with a single reservoir (300 nodes). Later, the predicted trajectory of the ESN also starts to deviate from the true state with an increase in data points, with the green dotted line moving away from the blue solid line. In contrast, H-ESN demonstrates a significant advantage in prediction duration compared to the other two models, achieving accurate predictions for approximately 500 data points for the three variables of the Lorenz system. For comparison, we computed the mean squared error (MSE) values of the three models for the three variables of the Lorenz system at different prediction lengths in

Table 4. Comparison table of MSE values for the three dimensions of the Lorenz system at different prediction lengths using the ESN, Deep ESN, and H-ESN.

				MSE
		300	350	400	450	500
Lorenz-x	ESN	0.80367	0.78542	1.0564	3.5907	3.6537
	Deep ESN	2.9228	3.0528	6.9675	25.3735	41.3386
	H-ESN	0.5315	0.51897	0.51815	0.69162	0.65461
Lorenz-y	ESN	1.8692	1.7847	2.6831	8.5373	8.3973
	Deep ESN	7.2936	7.019	12.0632	37.1467	55.8747
	H-ESN	1.2166	1.1686	1.1761	1.6599	1.5405
Lorenz-z	ESN	2.7321	2.6891	2.8106	12.8135	12.4176
	Deep ESN	10.4869	11.1452	27.9924	49.5561	55.6685
	H-ESN	1.8054	1.7662	1.7419	2.4817	2.3127

.

As shown in Table 4, the MSE between the predicted values at 300, 350, 400, 450, and 500 data points and the true values of the three variables of the Lorenz system were calculated separately. Additionally, in

Table 5. MSE percentage improvement of the H-ESN over the ESN.

		300	350	400	450	500
Lorenz-x	H-ESN	33.87%	33.92%	50.95%	80.74%	82.08%
Lorenz-y	H-ESN	34.91%	34.52%	56.17%	80.56%	81.65%
Lorenz-z	H-ESN	33.92%	34.32%	38.02%	80.63%	81.38%

, the average MSE percentage improvement of the H-ESN over the ESN for the three variables of the Lorenz system was calculated based on Equation (9). It can be concluded that the MSE value for the H-ESN model is minimal at different prediction stages, indicating that the model proposed in this paper achieves the best performance in this prediction task.

$$\begin{array}{c}\hfill {\displaystyle \frac{{\mathrm{ESN}}_{\mathrm{MSE}}-{\text{H-ESN}}_{\mathrm{MSE}}}{{\mathrm{ESN}}_{\mathrm{MSE}}}}\times 100\%\end{array}$$

(9)

In chaotic prediction tasks, the focus is on the duration and accuracy of predictions. Effective prediction time (EPT) is an important metric for evaluating the performance of time series prediction models. It refers to the limited period during which accurate predictions can be made using the model in chaotic scenarios. This period is finite because chaotic systems are extremely sensitive to initial conditions, leading to significant uncertainty in long-term predictions. In this paper, the effective prediction time is defined as $\mathrm{EPT}=\left|\mathbf{u}\left(t\right)-\mathbf{v}\left(t\right)\right|$. The prediction is considered invalid at time t, when the prediction error exceeds the set threshold $ϵ$, that is, when $\mathrm{EPT}>ϵ$ ($ϵ$ is a given error).

The parameter $\theta$ is a very important hyperparameter, and its selection significantly affects the system’s predictive performance. Figure 4 shows the EPT for the three variables under different values of $\theta$. Overall, When $\theta$ is small, the EPT tends to decrease because the nonlinearity is too strong, making it difficult for the network to train or generalize effectively. When $\theta$ is large, the EPT also tends to decrease due to excessive linearization, causing the network to fail to capture the key dynamics of the chaotic system. However, after reaching the intermediate region at $\theta =0.65$, the EPT starts to increase and reaches its maximum value, as the balance between nonlinearity and linearity is optimized. To ensure that the three variables have a longer prediction duration, it is recommended to choose a $\theta$ value of approximately 0.7.

The H-ESN introduces a linear component ($\theta x$) through homotopy transformation, finding the optimal balance between nonlinearity and linearity. However, the value of $\theta$ generally varies for different chaotic systems. Currently, we primarily determine the value of $\theta$ through grid search or empirical tuning. While effective, this method can be computationally expensive when dealing with high-dimensional or complex systems. Finding the optimal value of $\theta$ quickly and efficiently is a major challenge faced by the H-ESN.

3.2. Mackey–Glass Equation

The Mackey–Glass (MG) equation is a commonly used delay differential equation used to model complex dynamic behaviors in biological and physical systems with time delays, especially in biology and ecology [42]. Its standard form is as follows

$$\begin{array}{c}\hfill {\displaystyle \frac{\mathrm{d}x}{\mathrm{d}t}}={\displaystyle \frac{\beta x(t-\tau )}{1+x{(t-\tau )}^n}}-\gamma x\left(t\right),\end{array}$$

(10)

where $\beta$ = 0.2, $\gamma$ = 0.1, $\tau$ = 17, and n = 10. The above equation is numerically solved using the Euler method to obtain the chaotic time series of the MG system. The first 2000 data points are used as the training set, and the next 1000 data points as the testing set. Figure 5 shows a comparison of the prediction performance of the ESN and H-ESN on the MG time series, with the parameters listed in

Table 6. Parameters for the MG time series prediction task.

Parameter	Value	Parameter	Value
$D_r$	1000	$\alpha$	0.7
$\rho$	0.9	$\xi$	0
$\Delta$ t	0.1	$\theta$	0.08

.

As shown in Figure 5, both models can make good short-term predictions for the MG time series. The ESN can accurately predict 533 data points, but it fails to accurately predict the peak information in the time interval between 533 and 800 time steps. On the other hand, the H-ESN can predict for 1000 time steps and is capable of capturing the peak information effectively, indicating that the H-ESN has a clear advantage in predicting the MG time series.

The hyperparameters in ESN have a significant impact on the prediction performance of chaotic systems. The following analysis examines how the MSE between the predicted and true values changes with the spectral radius $\rho$ over the first 500 time steps. Overall, as the spectral radius increases, the MSE decreases, and the H-ESN shows higher prediction accuracy compared to the other two models. For small values of $\rho$, the prediction accuracy already reaches $1\times {10}^{-4}$, and the MSE value reaches $1\times {10}^{-7}$ when $\rho$ = 1.3.

In addition to the spectral radius $\rho$, which affects the prediction accuracy of the model, the number of reservoir nodes $D_r$ also plays a crucial role in chaotic prediction. The size of $D_r$ directly influences the complexity of the state space that the network can represent. Generally speaking, the larger the number of reservoir nodes, the more dynamic and complex patterns the network can capture. Figure 6 illustrates how the MSE between the predicted and true values changes with $D_r$ for two different spectral radii. It can be observed that when $\rho$ = 1.2 and $D_r$ = 950, the corresponding MSE is minimized, and the prediction accuracy reaches ${10}^{-9}$. Furthermore, Figure 6 shows that for different reservoir sizes, for $D_r$ = 300, the MSE corresponding to $\rho$ = 1.2 is smaller than that corresponding to $\rho$ = 1.25, which is the opposite trend compared to the one observed for H-ESN in Figure 7. This indicates that different reservoir sizes have a significant impact on the prediction ability of the H-ESN.

3.3. Kuramoto–Sivashinsky Equations

Now consider a modified version of the Kuramoto–Sivashinsky(KS) system defined by the following partial differential Equation [43]

$$\begin{array}{c}\hfill y_t=-y{y}_x-[1+\mu cos\left({\displaystyle \frac{2\pi x}{\lambda }}\right)]y_{xx}-y_{xxx}.\end{array}$$

(11)

If $\mu =0$, this equation is reduced to the standard KS equation, and if $\mu \ne 0$, the cosine term makes the equation spatially inhomogeneous. We will focus on the case where $\mu =0$ below.

We take into consideration the fact that the KS system has periodic boundary conditions at $0\le x<L$, that is, $y(x+L,t)=y(x,t)$, and the KS equation is numerically integrated into a uniformly spaced grid of size Q. The simulated data consists of Q time series with time step $\Delta t$, represented by vector $\mathbf{u}\left(t\right)={(y_1\left(t\right),y_2\left(t\right),\cdots ,y_Q\left(t\right))}^T$, where $y_i\left(t\right)=y(i\Delta t)$, $\Delta x=L/Q$.

Considering that the Kuramoto–Sivashinsky equation has high dimensional spatiotemporal chaos and a certain symmetry, we modify Equation (2) by analogy with the Lorenz system. After the training stage $-T\le t\le 0$, the system uses Tikhonov regularization regression to obtain ${\mathbf{W}}_{out}$. When the output parameters are determined, the system enters the prediction stage $t>0$ and independently evolves according to Figure 1b.

As shown in Figure 8, the ESN model achieves the prediction of 7 ${\Lambda }_{max}t$ steps for the KS system, while the H-ESN model can predict up to 12 ${\Lambda }_{max}t$ steps, in almost twice the duration taken by the ESN model. In terms of prediction accuracy, the error panel of the H-ESN model is close to 0 during the early prediction stages. For comparison, we have plotted the root mean square error (RMSE) of both models at different prediction dimensions Q in Figure 9. The RMSE values of the H-ESN are consistently below 0.15, with relatively small fluctuations. In contrast, the RMSE values of the ESN model are mostly above 0.15, with a significant difference between the maximum and minimum values. Therefore, the H-ESN model is more accurate in prediction and also more stable in terms of prediction performance.

The most important characteristic of chaotic dynamics is their extreme sensitivity to initial conditions. In chaotic systems, long-term predictions of the system’s state are impossible, as even the smallest errors will exponentially amplify, quickly eroding predictive capability. In predicting chaotic systems, not only is it necessary to optimize the hyperparameters to extend the effective prediction time of various variables, but it is also essential to evaluate the prediction accuracy in terms of the system’s inherent chaotic characteristics. The maximum Lyapunov exponent (${\Lambda }_{max}$) is a key metric for measuring the chaotic nature of dynamic systems. It evaluates whether the system exhibits a chaotic behavior by quantifying the rate of divergence of nearby trajectories in the phase space. By comparing the ${\Lambda }_{max}$ of the KS system, as shown in

Table 7. Comparison table of ${\Lambda }_{max}$ for the ESN and H-ESN, ${\Lambda }_{max}$ represents the maximum Lyapunov exponent.

	Actual KS	ESN	H-ESN
${\Lambda }_{max}$	0.0471	0.0488	0.0470
Error (%)		3.61%	0.21%

, it can be observed that the ${\Lambda }_{max}$ estimated from the predicted data of the H-ESN model is closest to the true ${\Lambda }_{max}$ of the KS system, with a difference of 0.0001. In contrast, the ${\Lambda }_{max}$ obtained from the predicted data of the ESN model deviates from the true value by 0.0017. This difference indicates that the H-ESN model demonstrates a stronger ability to capture the chaotic characteristics and sensitivity of the system. Particularly at longer time scales, the H-ESN is able to more accurately reflect the system’s dynamical behavior, highlighting its advantages in modeling complex dynamical systems.

Chaotic systems are often disturbed by noise in practical applications, which can significantly reduce the performance of prediction models. To verify the robustness of the H-ESN under noisy conditions, we added Gaussian noise with varying intensities (noise levels of 0.01, 0.02, and 0.03) to the KS system, simulating real-world measurement errors. The experiment was conducted with the parameters listed in

Table 8. Parameters for the KS system prediction task.

Parameter	Value	Parameter	Value
$D_r$	5000	d	3
$\rho$	0.4	$\xi$	0
Q	60	$\theta$	0.3

, aiming to evaluate the performance of H-ESN under noise conditions.

As shown in Figure 10, when the Gaussian noise intensity is 0.01, the H-ESN can still maintain good predictive capability, with a prediction duration reaching 6 ${\Lambda }_{max}$. However, as the noise intensity increases, the predictive capability of the H-ESN gradually declines. This indicates that in low-noise environments, the H-ESN can effectively handle noise interference and maintain high prediction accuracy; however, under high-noise conditions, the impact of noise on model performance becomes more pronounced.

4. Summary and Future Directions

4.1. Summary

A trained ESN can approximate the ergodic properties of its real system, and an ESN based on homotopy theory has demonstrated better performance for short-term predictions in chaotic systems. As shown in Section 3, with the Lorenz and MG system, the choice of parameters has a significant impact on prediction performance. Once the parameters are properly selected, our method has demonstrated longer prediction durations and better accuracy. For high-dimensional spatiotemporal chaotic systems, the H-ESN has demonstrated better performance in chaotic prediction compared to the ESN, doubling the prediction duration and providing more precise estimates of the Lyapunov exponent. Moreover, the H-ESN exhibits a certain degree of robustness in low-noise environments, where it retains reliable prediction capabilities even under mild noise interference. This resilience not only highlights its practical applicability but also ensures the preservation of chaotic system dynamics, making it a promising tool for real-world scenarios where complete noise elimination is challenging.

4.2. Future Directions

From a broader perspective, this paper reveals that echo state networks based on homotopy theory (H-ESN) represent a highly fruitful and detailed research direction in the field of chaotic system measurement data. However, despite its promising potential, the current H-ESN framework still exhibits certain limitations that need to be addressed.

• Computational inefficiency arises in parameter optimization, particularly in selecting the homotopy parameter $\theta$.
• H-ESN exhibits certain limitations under high-noise conditions, with prediction accuracy gradually decreasing as noise intensity increases.
• Reservoir design varies by task, but the lack of universal guidelines makes selecting the right structure and parameters challenging.

Future research can focus on integrating noise reduction techniques or robust optimization methods to further enhance the performance of the H-ESN in high-noise environments. Additionally, exploring more efficient and direct approaches for selecting the homotopy parameter $\theta$ is crucial for improving the model’s adaptability and reducing computational costs. At the same time, developing adaptive design methods and establishing a theoretical framework can provide systematic guidance for the selection of reservoir structures and parameters in the H-ESN, thereby further enhancing its performance and generalizability across different application scenarios. These research directions not only hold significant theoretical value but also provide new insights for solving practical engineering problems.