# Based on the Improved PSO-TPA-LSTM Model Chaotic Time Series Prediction

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

^{2}) is improved by 0.4. The proposed network demonstrates good adaptability to complex chaotic data, surpassing the accuracy of the LSTM and PSO-LSTM models, thereby achieving higher prediction accuracy.

## 1. Introduction

## 2. IPSO-TPA-LSTM

#### 2.1. Improved Particle Group Optimization Algorithm (IPSO)

_{1}and c

_{2}. The inertia weight ω balances the global search range and the local precise search. The learning factors c

_{1}and c

_{2}. have important impacts on whether the algorithm falls into local optimization and convergence. Therefore, optimizing these three parameters individually can greatly improve the algorithm’s performance. Among them, the standard PSO algorithm’s velocity and position update rules are shown in Equation (1):

_{id}(t) is the initial velocity of the i-th particle, v

_{id}(t + 1) is the current velocity of the i-th particle; x

_{id}(t) is the current position of the i-th particle, x

_{id}(t + 1) is the new position generated by the i-th particle; p

_{id}refers to the best position experienced by each particle, p

_{gd}refers to the optimal position of the entire population. ω is the inertia weight factor; d = 1,2, … n; i = 1,2, … n; n is the current iteration number; c

_{1}and c

_{2}are non-negative constants called learning factors; r

_{1}and r

_{2}are random numbers distributed in the interval (0,1).

#### 2.1.1. The Inertia of Non-Linear Change

_{max}is the initial inertial value, and ω

_{min}is the inertial value value of the maximum evolution. Among them, the value of ω

_{max}is generally 0.9, the ω

_{min}value is 0.3, and t

_{max}is the moment when it iterates to complete its evolution.

#### 2.1.2. Improve Learning Factor Adjustment Strategy

_{1}and C

_{2}values and optimize the model’s performance to some extent [27], due to the highly non-linear and complex nature of chaotic data generated by the Lorenz system, using linearly varying learning factors C

_{1}and C

_{2}can easily lead to the model falling into local optimal solutions or failing to find optimal solutions. Therefore, this article proposes using learning factors that vary sinusoidally with inertia weight, which enables the model to search for solutions in a non-linear manner during optimization, adapting to the characteristics of chaotic data while achieving a better performance. Please refer to Equation (3) for details.

#### 2.2. Long Short-Term Memory Network (LSTM)

_{t}check in the recurrent unit. The cell state is the foundation of the LSTM network, and it can capture some important signals at specific times and retain them during the corresponding time intervals. Therefore, LSTM network has great significance for capturing the time changes of certain parameters and their correlation with other parameters. The internal structure of LSTM-cell unit is shown in Figure 1.

_{t}

_{−1}. The output h

_{t}value not only determines the output of the previous cell but also determines the state of the previous cell. The calculation methods of the three gates are as follows:

_{t}two vector groups in series; [h

_{t}

_{−1}, x

_{t}] is a vector composed of h

_{t}

_{−1}and x

_{t}; W

_{i}, W

_{f}and b

_{i}are the weight matrix of input gate, forgotten gate and output gate, respectively; b

_{f}and b

_{o}are the offset top of forgetting gate and output gate, respectively.

_{c}is the state weight matrix generated by the instant unit state generated at the current moment; b

_{c}is the bias top of the current unit state.

_{t}at the current moment is:

_{t}is as follows:

#### 2.3. Time Mode Attention Mechanism (TPA)

## 3. Build an IPSO-TPA-LSTM Chaos Data Prediction Model

#### 3.1. Input Layer

#### 3.2. LSTM Layer

#### 3.3. Attention Layer

## 4. Chaos Data Sources

#### 4.1. Lorenz Systems

#### 4.2. Evaluation Index of the Chaotic Data Model

^{2}) as the evaluation index of chaotic data prediction accuracy, the smaller the value of RMSE and MAE, the better the performance of the prediction model, meaning that the value of R

^{2}is closer to 1 and the model fitting effect is better. The model evaluation expressions are as follows:

## 5. Experimental Design and Analysis

#### 5.1. Prediction of Chaotic Data for Different Algorithmic Models

#### 5.2. Comparison of Accuracy of Different Prediction Models

^{2}. The initial values (−1, 1, 6), (7, 7, 25), (9, 9, 27) Lorenz system generated XYZ three-axis chaotic data using three models (LSTM, PSO-LSTM, and PSO-TPA-LSTM), and the three index values of the models are shown in Table 1, Table 2 and Table 3. It can be seen from the tables that the evaluation indexes MAE and RMSE of the proposed PSO-TPA-LSTM prediction model are the smallest among the three different initial values generated by Lorenz system. The R

^{2}value is closest to 1, indicating that the proposed algorithm model has the best fitting degree. MAE and RMSE measure the absolute deviation between the true value and the predicted value. The smaller their values, the smaller the absolute deviation of the model is, and the higher the prediction accuracy of the data.

## 6. Discussion and Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## References

- Kumar, U.; Jain, V.K. ARIMA forecasting of ambient air pollutants (O
_{3}, NO, NO_{2}and CO). Stoch. Environ. Res. Risk Assess.**2010**, 24, 751–760. [Google Scholar] [CrossRef] - Garcia, R.C.; Contreras, J.; Van Akkeren, M.; Garcia, J.B.C. A GARCH forecasting model to predict day-ahead electricity prices. IEEE Trans. Power Syst.
**2005**, 20, 867–874. [Google Scholar] [CrossRef] - Su, L.; Li, C. Local prediction of chaotic time series based on polynomial coefficient autoregressive model. Math. Probl. Eng.
**2015**, 2015, 901807. [Google Scholar] [CrossRef] - Tian, Z.D.; Gao, X.W.; Shi, T. Combination kernel function least squares support vector machine for chaotic time series prediction. Acta Phys. Sin.
**2014**, 63, 160508. [Google Scholar] [CrossRef] - Wang, S.Y.; Shi, C.F.; Qian, G.B.; Wang, W.L. Prediction of chaotic time series based on the fractional-order maximum correntropy criterion algorithm. Acta Phys. Sin.
**2018**, 67, 018401. [Google Scholar] [CrossRef] - He, Y.; Xu, Q.; Wan, J.; Yang, S. Electrical load forecasting based on self-adaptive chaotic neural network using Chebyshev map. Neural Comput. Appl.
**2018**, 29, 603–612. [Google Scholar] [CrossRef] - Cheng, W.; Wang, Y.; Peng, Z.; Ren, X.; Shuai, Y.; Zang, S.; Liu, H.; Cheng, H.; Wu, J. High-efficiency chaotic time series prediction based on time convolution neural network. Chaos Solitons Fractals
**2021**, 152, 111304. [Google Scholar] [CrossRef] - Nguyen, N.P.; Duong, T.A.; Jan, P. Strategies of Multi-Step-ahead Forecasting for Chaotic Time Series using Autoencoder and LSTM Neural Networks: A Comparative Study. In Proceedings of the 2023 5th International Conference on Image Processing and Machine Vision, Macau, China, 13–15 January 2023; pp. 55–61. [Google Scholar]
- Yan, J.C.; Zheng, J.Y.; Sun, S.Y. Chaotic time series prediction based on maximum information mining broad learning system. Comput. Appl. Softw.
**2023**, 40, 253–260. [Google Scholar] - Sun, T.B.; Liu, Y.H. Chaotic Time Series Prediction Based on Fuzzy Information Granulation and Hybrid Neural Network. Inf. Control
**2022**, 51, 671–679. [Google Scholar] - Wang, L.; Wang, X.; Li, H.Q. Chaotic time series prediction model of wind power power based on phase space reconstruction and error compensation. Proc. CSU-EPSA
**2017**, 29, 65–69. [Google Scholar] - Li, K.; Han, Y.; Huang, H.Q. Chaotic Time Series Prediction Based on IBH-LSSVM and Its Application to Short-term Prediction of Dynamic Fluid Level in Oil Wells. Inf. Control
**2016**, 45, 241–247, 256. [Google Scholar] - Sareminia, S. A Support Vector Based Hybrid Forecasting Model for Chaotic Time Series: Spare Part Consumption Prediction. Neural Process Lett.
**2023**, 55, 2825–2841. [Google Scholar] [CrossRef] - Wei, A.; Li, X.; Yan, L.; Wang, Z.; Yu, X. Machine learning models combined with wavelet transform and phase space reconstruction for groundwater level forecasting. Comput. Geosci.
**2023**, 177, 105386. [Google Scholar] - Wang, H.; Zhang, Y.; Liang, J.; Liu, L. DAFA-BiLSTM: Deep Autoregression Feature Augmented Bidirectional LSTM network for time series prediction. Neural Netw.
**2023**, 157, 240–256. [Google Scholar] [CrossRef] [PubMed] - Nasiri, H.; Ebadzadeh, M.M. MFRFNN: Multi-functional recurrent fuzzy neural network for chaotic time series prediction. Neurocomputing
**2022**, 507, 292–310. [Google Scholar] [CrossRef] - Wang, L.; Dai, L. Chaotic Time Series Prediction of Multi-Dimensional Nonlinear System Based on Bidirectional LSTM Model. Adv. Theory Simul.
**2023**, 6, 2300148. [Google Scholar] [CrossRef] - Huang, W.J.; Li, Y.T.; Huang, Y. Prediction of chaotic time series using hybrid neural network and attention mechanism. Acta Phys. Sin.
**2021**, 70, 010501. [Google Scholar] [CrossRef] - Fu, K.; Li, H.; Deng, P. Chaotic time series prediction using DTIGNet based on improved temporal-inception and GRU. Chaos Solitons Fractals
**2022**, 159, 112183. [Google Scholar] [CrossRef] - Qi, L.T.; Wang, S.Y.; Shen, M.L.; Huang, G.Y. Prediction of chaotic time series based on Nyström Cauchy kernel conjugate gradient algorithm. Acta Phys. Sin.
**2022**, 71, 108401. [Google Scholar] [CrossRef] - Ong, P.; Zainuddin, Z. An optimized wavelet neural networks using cuckoo search algorithm for function approximation and chaotic time series prediction. Decis. Anal. J.
**2023**, 6, 100188. [Google Scholar] [CrossRef] - Ong, P.; Zainuddin, Z. Optimizing wavelet neural networks using modified cuckoo search for multi-step ahead chaotic time series prediction. Appl. Soft Comput.
**2019**, 80, 374–386. [Google Scholar] [CrossRef] - Jiang, P.; Wang, B.; Li, H.; Lu, H. Modeling for chaotic time series based on linear and nonlinear framework: Application to wind speed forecasting. Energy
**2019**, 173, 468–482. [Google Scholar] [CrossRef] - Mei, Y.; Tan, G.Z.; Liu, Z.T.; Wu, H. Chaotic time series prediction based on brain emotional learning model and self-adaptive genetic algorithm. Acta Phys. Sin.
**2018**, 67, 080502. [Google Scholar] - Awad, M. Forecasting of Chaotic Time Series Using RBF Neural Networks Optimized by Genetic Algorithms. Int. Arab. J. Inf. Technol.
**2017**, 14, 826–834. [Google Scholar] - Long, W.; Zhang, W.Z. Parameter estimation of heavy oil pyrolysis model based on adaptive particle swarm algorithm. J. Chongqing Norm. Univ. Nat. Sci.
**2023**, 30, 128–133. [Google Scholar] - Xu, S.B.; Xia, W.J.; Dai, A.D. A particle swarm algorithm that improves learning factors. Inf. Secur. Technol.
**2012**, 3, 17–19. [Google Scholar] - Li, G.; Li, X.J.; Yang, W.X.; Han, D. Research on TBM boring parameter prediction based on deep learning. Mod. Tunn. Technol.
**2020**, 57, 154–159. [Google Scholar] - Feng, Y.T.; Wu, X.; Xu, X.; Zhang, R.Q. Research on ionospheric parameter prediction based on deep learning. J. Commun.
**2021**, 42, 202–206. [Google Scholar] - Park, J.; Jungsik, J.; Park, Y. Ship trajectory prediction based on Bi-LSTM using spectral-clustered AIS data. J. Mar. Sci. Eng.
**2021**, 9, 1037. [Google Scholar] [CrossRef] - Lorenz, E.N. Deterministic Nonperiodic Flow. J. Atmos. Sci.
**2004**, 20, 130–141. [Google Scholar] [CrossRef]

**Figure 6.**Comparison diagram of simulation prediction of chaotic data on the X axis with initial values (−1, 1, 6).

**Figure 7.**Comparison diagram of simulation prediction of chaotic data on the X axis with initial values (7, 7, 25).

**Figure 8.**Comparison diagram of simulation prediction of chaotic data on the X axis with initial values (9, 9, 27).

**Figure 9.**Comparison diagram of simulation prediction of chaotic data on the Y axis with initial values (−1, 1, 6).

**Figure 10.**Comparison diagram of simulation prediction of chaotic data on the Y axis with initial values (7, 7, 25).

**Figure 11.**Comparison diagram of simulation prediction of chaotic data on the Y axis with initial values (9, 9, 27).

**Figure 12.**Comparison diagram of simulation prediction of chaotic data on the Z axis with initial values (−1, 1, 6).

**Figure 13.**Comparison diagram of simulation prediction of chaotic data on the Z axis with initial values (7, 7, 25).

**Figure 14.**Comparison diagram of simulation prediction of chaotic data on the Z axis with initial values (9, 9, 27).

MAE | RMSE | R^{2} | |
---|---|---|---|

LSTM | 3.874 | 2.736 | 0.83 |

PSO-LSTM | 3.337 | 2.652 | 0.88 |

PSO-TPA-LSTM | 2.983 | 2.231 | 0.92 |

MAE | RMSE | R^{2} | |
---|---|---|---|

LSTM | 6.644 | 4.3822 | 0.87 |

PSO-LSTM | 5.261 | 3.872 | 0.89 |

PSO-TPA-LSTM | 3.823 | 3.523 | 0.93 |

MAE | RMSE | R^{2} | |
---|---|---|---|

LSTM | 5.837 | 4.542 | 0.85 |

PSO-LSTM | 3.982 | 2.953 | 0.86 |

PSO-TPA-LSTM | 2.983 | 2.231 | 0.94 |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Cai, Z.; Feng, G.; Wang, Q.
Based on the Improved PSO-TPA-LSTM Model Chaotic Time Series Prediction. *Atmosphere* **2023**, *14*, 1696.
https://doi.org/10.3390/atmos14111696

**AMA Style**

Cai Z, Feng G, Wang Q.
Based on the Improved PSO-TPA-LSTM Model Chaotic Time Series Prediction. *Atmosphere*. 2023; 14(11):1696.
https://doi.org/10.3390/atmos14111696

**Chicago/Turabian Style**

Cai, Zijian, Guolin Feng, and Qiguang Wang.
2023. "Based on the Improved PSO-TPA-LSTM Model Chaotic Time Series Prediction" *Atmosphere* 14, no. 11: 1696.
https://doi.org/10.3390/atmos14111696