# Dynamically Meaningful Latent Representations of Dynamical Systems

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Models

#### 2.1. fKdV Equation

#### 2.2. Kuramoto–Sivashinksy Equation

## 3. Long Time Dynamics

#### 3.1. fKdV: Effects of Amplitude and Wavenumber

#### 3.2. KS: Long Time Dynamics

## 4. Interpretable Deep Learning-Based Reduced Order Model

#### 4.1. fKdV Latent Representation

#### 4.2. KS Latent Representation

## 5. Topology Preservation

#### 5.1. Persistent Homology

#### 5.2. Persistence Diagrams

## 6. Discussion

## Author Contributions

## Funding

## Data Availability Statement

## Conflicts of Interest

## Appendix A. Autoencoder Model Architecture and Parameters

**Table A1.**Parameters used in neural networks for the autoencoder where ${d}_{i}$ is the input dimension and ${d}_{z}$ is the latent dimension. The “Dimension” column describes the dimension of each layer, and the “Activation” column describes the activation functions between layers. The “Epochs” column describes the number of epochs used for training.

Component | Dimension | Activations | Epochs |
---|---|---|---|

Encoder | ${d}_{i}$:32:64:32:${d}_{z}$ | Linear:ReLU:Linear:ReLU:Linear | 1000 |

Decoder | ${d}_{z}$:32:64:32:${d}_{i}$ | Linear:ReLU:Linear:ReLU:Linear:Sigmoid | 1000 |

## Appendix B. Bifurcation Classification of the fKdV

## References

- Champion, K.; Lusch, B.; Kutz, J.N.; Brunton, S.L. Data-driven discovery of coordinates and governing equations. Proc. Natl. Acad. Sci. USA
**2019**, 116, 22445–22451. [Google Scholar] [CrossRef] - Maulik, R.; Lusch, B.; Balaprakash, P. Reduced-order modeling of advection-dominated systems with recurrent neural networks and convolutional autoencoders. Phys. Fluids
**2021**, 33, 037106. [Google Scholar] [CrossRef] - Brunton, S.L.; Proctor, J.L.; Kutz, J.N. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proc. Natl. Acad. Sci. USA
**2016**, 113, 3932–3937. [Google Scholar] [CrossRef] [PubMed] - Lusch, B.; Kutz, J.N.; Brunton, S.L. Deep learning for universal linear embeddings of nonlinear dynamics. Nat. Commun.
**2018**, 9, 4950. [Google Scholar] [CrossRef] - Raissi, M.; Perdikaris, P.; Karniadakis, G. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys.
**2019**, 378, 686–707. [Google Scholar] [CrossRef] - Lee, K.; Carlberg, K.T. Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders. J. Comput. Phys.
**2020**, 404, 108973. [Google Scholar] [CrossRef] - Linot, A.J.; Graham, M.D. Deep learning to discover and predict dynamics on an inertial manifold. Phys. Rev. E
**2020**, 101, 062209. [Google Scholar] [CrossRef] [PubMed] - Schonsheck, S.; Chen, J.; Lai, R. Chart auto-encoders for manifold structured data. arXiv
**2019**, arXiv:1912.10094. [Google Scholar] - Floryan, D.; Graham, M.D. Data-driven discovery of intrinsic dynamics. Nat. Mach. Intell.
**2022**, 4, 1113–1120. [Google Scholar] [CrossRef] - Zeng, K.; Graham, M.D. Autoencoders for discovering manifold dimension and coordinates in data from complex dynamical systems. arXiv
**2023**, arXiv:2305.01090. [Google Scholar] - Fefferman, C.; Mitter, S.; Narayanan, H. Testing the manifold hypothesis. J. Am. Math. Soc.
**2016**, 29, 983–1049. [Google Scholar] [CrossRef] - Foias, C.; Sell, G.R.; Temam, R. Inertial manifolds for nonlinear evolutionary equations. J. Differ. Equ.
**1988**, 73, 309–353. [Google Scholar] [CrossRef] - Temam, R.; Wang, X. Estimates On The Lowest Dimension Of Inertial Manifolds For The Kuramoto-Sivasbinsky Equation in The General Case. Differ. Integral Equ.
**1994**, 7, 1095–1108. [Google Scholar] - Chepyzhov, V.V.; Višik, M.I. Attractors for Equations of Mathematical Physics; Number 49 in Colloquium Publications; American Mathematical Society: Providence, RI, USA, 2002. [Google Scholar]
- Zelik, S. Attractors. Then and now. arXiv
**2022**, arXiv:2208.12101. [Google Scholar] - Tutueva, A.; Nepomuceno, E.G.; Moysis, L.; Volos, C.; Butusov, D. Adaptive Chaotic Maps in Cryptography Applications. In Cybersecurity: A New Approach Using Chaotic Systems; Abd El-Latif, A.A., Volos, C., Eds.; Springer International Publishing: Cham, Switzerland, 2022; pp. 193–205. [Google Scholar]
- Neamah, A.A.; Shukur, A.A. A Novel Conservative Chaotic System Involved in Hyperbolic Functions and Its Application to Design an Efficient Colour Image Encryption Scheme. Symmetry
**2023**, 15, 1511. [Google Scholar] [CrossRef] - Li, R.; Lu, T.; Wang, H.; Zhou, J.; Ding, X.; Li, Y. The Ergodicity and Sensitivity of Nonautonomous Discrete Dynamical Systems. Mathematics
**2023**, 11, 1384. [Google Scholar] [CrossRef] - Ruelle, D. Small random perturbations of dynamical systems and the definition of attractors. Commun. Math. Phys.
**1981**, 82, 137–151. [Google Scholar] [CrossRef] - Katok, A.; Hasselblatt, B. Introduction to the Modern Theory of Dynamical Systems; Number 54; Cambridge University Press: Cambridge, MA, USA, 1995. [Google Scholar]
- Guckenheimer, J.; Holmes, P. Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields; Springer: Berlin/Heidelberg, Germany, 2013; Volume 42. [Google Scholar]
- Raugel, G. Global Attractors in Partial Diffenrential Equations; Département de Mathématique, Université de Paris-Sud: Orsay, France, 2001. [Google Scholar]
- Mielke, A.; Zelik, S.V. Infinite-Dimensional Hyperbolic Sets and Spatio-Temporal Chaos in Reaction Diffusion Systems in. J. Dyn. Differ. Equ.
**2007**, 19, 333–389. [Google Scholar] [CrossRef] - Doedel, E.J.; Champneys, A.R.; Fairgrieve, T.; Kuznetsov, Y.; Oldeman, B.; Paffenroth, R.; Sandstede, B.; Wang, X.; Zhang, C. Auto-07p: Continuation and Bifurcation Software for Ordinary Differential Equations. 2007. Available online: http://indy.cs.concordia.ca/auto (accessed on 10 September 2023).
- Dhooge, A.; Govaerts, W.; Kuznetsov, Y.A.; Meijer, H.G.E.; Sautois, B. New features of the software MatCont for bifurcation analysis of dynamical systems. Math. Comput. Model. Dyn. Syst.
**2008**, 14, 147–175. [Google Scholar] [CrossRef] - Dankowicz, H.; Schilder, F. Recipes for Continuation; SIAM: Philadelphia, PA, USA, 2013. [Google Scholar]
- Binder, B.; Vanden-Broeck, J.M. Free surface flows past surfboards and sluice gates. Eur. J. Appl. Math.
**2005**, 16, 601–619. [Google Scholar] [CrossRef] - Binder, B.J. Steady Two-Dimensional Free-Surface Flow Past Disturbances in an Open Channel: Solutions of the Korteweg–De Vries Equation and Analysis of the Weakly Nonlinear Phase Space. Fluids
**2019**, 4, 24. [Google Scholar] [CrossRef] - Kuramoto, Y. Diffusion-Induced Chaos in Reaction Systems. Prog. Theor. Phys. Suppl.
**1978**, 64, 346–367. [Google Scholar] [CrossRef] - Kirby, M.; Armbruster, D. Reconstructing phase space from PDE simulations. Z. Angew. Math. Phys.
**1992**, 43, 999–1022. [Google Scholar] [CrossRef] - Rowley, C.W.; Marsden, J.E. Reconstruction equations and the Karhunen–Loève expansion for systems with symmetry. Phys. D Nonlinear Phenom.
**2000**, 142, 1–19. [Google Scholar] [CrossRef] - Lax, P.D. Almost periodic solutions of the KdV equation. SIAM Rev.
**1976**, 18, 351–375. [Google Scholar] [CrossRef] - Kevrekidis, I.G.; Nicolaenko, B.; Scovel, J.C. Back in the saddle again: A computer assisted study of the Kuramoto–Sivashinsky equation. SIAM J. Appl. Math.
**1990**, 50, 760–790. [Google Scholar] [CrossRef] - Otter, N.; Porter, M.A.; Tillmann, U.; Grindrod, P.; Harrington, H.A. A roadmap for the computation of persistent homology. EPJ Data Sci.
**2017**, 6, 1–38. [Google Scholar] [CrossRef] - Chazal, F.; Michel, B. An Introduction to Topological Data Analysis: Fundamental and Practical Aspects for Data Scientists. Front. Artif. Intell.
**2021**, 4. [Google Scholar] [CrossRef] - Rubio, J.; Sergeraert, F. Constructive algebraic topology. Bull. Des Sci. Math.
**2002**, 126, 389–412. [Google Scholar] [CrossRef] - Edelsbrunner, H.; Letscher, D.; Zomorodian, A. Topological persistence and simplification. Discret. Comput. Geom.
**2002**, 28, 511–533. [Google Scholar] [CrossRef] - Vietoris, L. Über den höheren Zusammenhang kompakter Räume und eine Klasse von zusammenhangstreuen Abbildungen. Math. Ann.
**1927**, 97, 454–472. [Google Scholar] [CrossRef] - Skraba, P.; Turner, K. Wasserstein stability for persistence diagrams. arXiv
**2020**, arXiv:2006.16824. [Google Scholar] - Mileyko, Y.; Mukherjee, S.; Harer, J. Probability measures on the space of persistence diagrams. Inverse Probl.
**2011**, 27, 124007. [Google Scholar] [CrossRef] - Moor, M.; Horn, M.; Rieck, B.; Borgwardt, K. Topological autoencoders. Proc. Mach. Learn. Res.
**2020**, 119, 7045–7054. [Google Scholar] - Kuznetsov, Y.A.; Kuznetsov, I.A.; Kuznetsov, Y. Elements of Applied Bifurcation Theory; Springer: New York, NY, USA, 1998; Volume 112. [Google Scholar]

**Figure 1.**Evolutions of trajectories in distance–speed space for initial conditions evolved under the fKdV equation for different initial amplitude values. $A=0.25$ (

**upper left panel**), $A=0.50$ (

**upper right panel**), $A=0.75$ (

**lower left panel**), $A=1.00$ (

**lower right panel**).

**Figure 2.**Evolutions of trajectories in distance–speed space for initial conditions evolved under the fKdV equation for different initial wavenumber values. $k=1.0$ (

**left panel**), $k=2.0$ (

**right panel**).

**Figure 3.**Long time wave profiles for the KS equation. Bursting dynamics (

**left panel**) where $\nu =\frac{16}{71}$, and beating traveling wave dynamics (

**right panel**) where $\nu =\frac{16}{337}$.

**Figure 4.**Fourier representation of the long time dynamics on the two leading spatial modes. Bursting dynamics (

**left panel**) where $\nu =\frac{16}{71}$, and beating traveling wave dynamics (

**right panel**) where $\nu =\frac{16}{337}$. Re1 and Re2 represent the real component of the first and second most dominant Fourier modes, respectively, while Im1 represents the imaginary component of the first dominant mode.

**Figure 5.**Schematic of the classical autoencoder architecture highlighting the encoder, decoder, and latent space (green) components.

**Figure 6.**Original wave profiles (

**left column**) obtained from the numerical simulations of the fKdV and their corresponding latent space representations (

**right column**) obtained using the autoencoder. Upper to lower plot: $A=0.11$, $A=0.26$, $A=0.47$.

**Figure 7.**Original wave profiles from the numerical simulation (

**left panels**) and their corresponding latent representations obtained from the autoencoder. Bursting dynamics (

**upper panels**) where $\nu =\frac{16}{71}$ and beating traveling wave dynamics (

**lower panels**) where $\nu =\frac{16}{337}$.

**Figure 8.**Persistence diagrams for the point cloud data from different dynamical models: (i) fkdv (upper panels); (ii) KS bursting dynamics (middle panels); (iii) KS beating traveling waves (lower panels). Results for the raw high-dimensional dynamical data (

**left column**) and the latent representation obtained from the autoencoder (

**right column**).

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Nasim, I.; Henderson, M.E.
Dynamically Meaningful Latent Representations of Dynamical Systems. *Mathematics* **2024**, *12*, 476.
https://doi.org/10.3390/math12030476

**AMA Style**

Nasim I, Henderson ME.
Dynamically Meaningful Latent Representations of Dynamical Systems. *Mathematics*. 2024; 12(3):476.
https://doi.org/10.3390/math12030476

**Chicago/Turabian Style**

Nasim, Imran, and Michael E. Henderson.
2024. "Dynamically Meaningful Latent Representations of Dynamical Systems" *Mathematics* 12, no. 3: 476.
https://doi.org/10.3390/math12030476