# Entropy-Related Extremum Principles for Model Reduction of Dissipative Dynamical Systems

^{1}

^{2}

## Abstract

**:**

## 1. Introduction

**Figure 1.**Illustration of trajectories relaxing onto a 2-D manifold and successively a 1-D manifold while converging to stable equilibrium point. Figure courtesy of A.N. Al-Khateeb, J.M. Powers, S. Paolucci (private communication).

## 2. Entropy Concepts

**backwards**in time and point out relations to entropy concepts in dynamical systems theory. In the latter context, entropy can generally be interpreted as a measure of the rate of increase in dynamical complexity, e.g., the rate of dissipation. We propose that an extremum principle suitable to characterize trajectories on slow attracting manifolds should incorporate in some sense the idea of minimum dissipativeness (in the sense of “maximum slowness”) of a dynamical system along these trajectories.

**Definition 1**[Topological entropy] [23] Let X be a compact metric space and $g:X\to X,g\left(x\right(t\left)\right)=x(t+1)$ be the time-one map associated with the flow of the dynamical system defined by the vector field $f\left(x\right)$ of the ODE $\dot{x}=f\left(x\right),f\in {C}^{\infty}$. $U\subset X$ is called a $(n,\u03f5)$-separated set, $n\in {\mathbb{Z}}^{+},\u03f5>0$, if

**Definition 2**[Metric entropy] [23] Let $(X,\mathcal{B},\mu )$ be a probability space with the σ-algebra $\mathcal{B}\subset \mathcal{P}\left(X\right)$, an appropriately chosen probability measure μ such that $g:X\to X,g\left(x\right(t\left)\right)=x(t+1)$ is a measure-preserving, i.e. $\mu \left(A\right)=\mu \left({g}^{-1}\left(A\right)\right)$, time-one map diffeomorphism induced by the flow of the dynamical system defined by $\dot{x}=f\left(x\right),f\in {C}^{\infty}$. Let $\alpha :=\left\{{A}_{1},...,{A}_{k}\right\}$ be a finite partition of X, ${g}^{-1}\left(\alpha \right):=\left\{{g}^{-1}\left({A}_{1}\right),...,{g}^{-1}\left({A}_{k}\right)\right\}$ and j is called the α-address of $x\in X$ if $x\in {A}_{j}$. For two partitions $\alpha ,\beta $ we define $\alpha \vee \beta :=\left\{A\cap B:A\in \alpha ,B\in \beta \right\}$, so that elements of ${\bigvee}_{i=0}^{n-1}{g}^{-i}\alpha $ are sets of the form $\left\{x:x\in {A}_{{i}_{0}},{g}^{1}\left(x\right)\in {A}_{{i}_{1}},...,{g}^{n-1}\left(x\right)\in {A}_{{i}_{n-1}}\right\}$ for index tuples $({i}_{0},...,{i}_{n-1})$ which are called the α-addresses of the n-orbit through x. With $H\left(\alpha \right):=-\sum \mu \left({A}_{i}\right)log\mu \left({A}_{i}\right)$ the metric (or measure-theoretic) entropy ${h}_{\mu}\left(g\right)$ is defined as

**Theorem 1**[25] Let $g:X\to X$ be a continuous map of a compact metric space X. Then ${h}_{top}\left(g\right)={sup}_{\mu}{h}_{\mu}\left(g\right)$ over all g-invariant probability measures μ.

**Theorem 2**[24] Let $(g,\mu )$ be ergodic. Then for μ-almost every x it holds

**Theorem 3**[Ruelle’s Inequality] [26] Let $g:M\to M$ be a ${C}^{2}$-diffeomorphism of a compact Riemannian manifold M and μ a g-invariant ergodic probability measure. With the r distinct Lyapunov exponents ${\lambda}_{i},i=1,...,r$ of $(g,\mu )$ and the linear subspaces ${U}_{i}$ corresponding to ${\lambda}_{i}$ with the dimension $dim{U}_{i}$ of the multiplicity of ${\lambda}_{i}$, it holds

**Theorem 4**[Pesin’s Formula] [27] If μ is equivalent to the Riemannian measure on M it holds

**Theorem 5**

- a)
- b)
- [29] For $g\in {C}^{k},k=1,...,\infty $ it holds$${V}_{l,k}\left(g\right)\le {h}_{top}\left(g\right)+\frac{2l}{k}R\left(g\right).$$

## 3. Extremum Principle for Trajectories Approximating Slow Manifolds

#### 3.1. Optimization Criterion

#### 3.2. Numerical Methods

## 4. Results: Application to Model Hydrogen Combustion Reaction Mechanism

#### 4.1. Computation of One-Dimensional Manifolds

**Figure 2.**One-dimensional manifold numerically computed as solution of problem (2) with $\Phi \left(t\right)={\sum}_{j=1}^{6}\frac{{d}_{i}{S}_{j}}{dt}=R({R}_{j}^{f}-{R}_{j}^{r})\mathrm{ln}\left(\frac{{R}_{j}^{f}}{{R}_{j}^{r}}\right)$ (chemical entropy production rate criterion) with reaction rates ${R}_{j}^{f},{R}_{j}^{r}$ for the six forward and backward reactions of mechanism (2) computed according to mass action kinetics. ${c}_{{\mathrm{H}}_{2}\mathrm{O}}$ is chosen as reaction progress variable and varied between 0.1 and 0.5, ${t}^{0}=-0.03,{t}^{f}=0$. Open blue circles represent the final values at ${t}^{f}=0$ of trajectories (blue lines are their continuations to equilibrium) for $t\in [{t}^{0},{t}^{f}]$ computed by solving optimization problem (2), the red dot represents the equilibrium point. Dotted trajectories are started from arbitrary initial values and illustrate the attractiveness of the computed 1-D manifold.

#### 4.2. Computation of Two-Dimensional Manifolds

**Figure 3.**$\Phi \left(t\right)=\parallel {J}_{f}{\left(x\right)\phantom{\rule{0.277778em}{0ex}}\xb7f\parallel}_{2}^{2}$ (curvature criterion). ${c}_{{\mathrm{H}}_{2}\mathrm{O}}$ chosen as reaction progress variable and varied between 0.1 and 0.5, ${t}^{0}=-0.03,{t}^{f}=0$, figure components and notations as in Figure 2.

**Figure 4.**Chemical entropy production rate criterion as in Figure 2, reaction progress variables ${c}_{{\text{H}}_{2}\text{O}}$ and ${c}_{{\text{H}}_{2}}$ varied between 0.01 and 0.69, respectively 0.01 and 0.26, ${t}^{0}=-{10}^{-6},{t}^{f}=0$. Blue circles represent final values of trajectories (at ${t}^{f}=0$) computed as solutions of optimization problem (2), blue lines the corresponding trajectories starting in these points and converging to equilibrium, colored trajectories are started from arbitrary initial values to illustrate attractiveness of the computed manifold spanned by the blue trajectories.

**Figure 5.**Curvature criterion, conditions, notations and figure components as in Figure 4.

## 5. Summary and Conclusions

## Acknowledgments

## References

- Warnatz, J.; Maas, U.; Dibble, R.W. Combustion: Physical and Chemical Fundamentals, Modeling and Simulation, Experiments, Pollutant Formation; Springer-Verlag: Berlin, Germany, 2006. [Google Scholar]
- Shaik, O.S.; Kammerer, J.; Gorecki, J.; Lebiedz, D. Derivation of a quantitative minimal model from a detailed elementary-step mechanism supported by mathematical coupling analysis. J. Chem. Phys.
**2005**, 123, 234103. [Google Scholar] [CrossRef] [PubMed] - Lebiedz, D.; Kammerer, J.; Brandt-Pollmann, U. Automatic network coupling analysis for dynamical systems based on detailed kinetic models. Phys. Rev. E
**2005**, 72, 041911. [Google Scholar] [CrossRef] - Bykov, V.; Gol’dshtein, V.; Maas, U. Simple global reduction technique based on decomposition approach. Combust. Theor. Model.
**2008**, 12, 389–405. [Google Scholar] [CrossRef] - Maas, U.; Pope, S.B. Simplifying chemical kinetics: Intrinsic low-dimensional manifolds in composition space. Combust. Flame
**1992**, 88, 239–264. [Google Scholar] [CrossRef] - Lam, S.H. Singular Perturbation for Stiff Equations using Numerical Methods. In Recent Advances in the Aerospace Sciences; Plenum Press: New York, NY, USA, 1985; pp. 3–20. [Google Scholar]
- Lam, S.H.; Goussis, D.A. The CSP method for simplifying kinetics. Int. J. Chem. Kinet.
**1994**, 26, 461–486. [Google Scholar] [CrossRef] - Davis, M.J.; Skodje, R.T. Geometric investigation of low-dimensional manifolds in systems approaching equilibrium. J. Chem. Phys.
**1999**, 111, 859–874. [Google Scholar] [CrossRef] - Fraser, S.J. The steady state and equilibrium approximations: A geometrical picture. J. Chem. Phys.
**1988**, 88, 4732–4738. [Google Scholar] [CrossRef] - Nguyen, A.H.; Frase, S.J. Geometrical picture of reaction in enzyme kinetics. J. Chem. Phys.
**1989**, 91, 186–193. [Google Scholar] [CrossRef] - Chiavazzo, E.; Gorban, A.N.; Karlin, I.V. Comparison of invariant manifolds for model reduction in chemical kinetics. Commun. Comput. Phys.
**2007**, 2, 964–992. [Google Scholar] - Gorban, A.N.; Karlin, I.V.; Zinovyev, A. Invariant grids: Method of complexity reduction in reaction networks. Complexus
**2005**, 2, 110–127. [Google Scholar] [CrossRef] - Gear, C.W.; Kaper, T.J.; Kevrekidis, I.G.; Zagaris, A. Projecting to a slow manifold: Singularly perturbed systems and legacy codes. SIAM J. Appl. Dyn. Syst.
**2005**, 4, 711–732. [Google Scholar] [CrossRef] - Zagaris, A.; Gear, C.W.; Kaper, T.J.; Kevrekidis, I.G. Analysis of the accuracy and convergence of equation-free projection to a slow manifold. ESAIM-Math. Model. Num.
**2009**, 43, 757–784. [Google Scholar] [CrossRef] - Keck, J.C.; Gillespie, D. Rate-controlled partial-equilibrium method for treating reacting gas mixtures. Combust. Flame
**1971**, 17, 237–241. [Google Scholar] [CrossRef] - Ren, Z.; Pope, S.B. Species reconstruction using pre-image curves. Proc. Combust. Inst.
**2005**, 30, 1293–1300. [Google Scholar] [CrossRef] - Ren, Z.; Pope, S.B.; Vladimirsky, A.; Guckenheimer, J.M. The invariant constrained equilibrium edge preimage curve method for the dimension reduction of chemical kinetics. J. Chem. Phys.
**2006**, 124, 114111. [Google Scholar] [CrossRef] [PubMed] - Delhaye, S.; Somers, L.M.T.; van Oijen, J.A.; de Goey, L.P.H. Simulating transient effects of laminar diffusion flames using flamelet libraries. In Proceedings of the Third European Combustion Meeting, Chania, Crete, April 2007.
- van Oijen, J.A.; de Goey, L.P.H. Modelling of premixed laminar flames using flamelet-generated manifolds. Combust. Sci. Technol.
**2000**, 161, 113–137. [Google Scholar] [CrossRef] - Mease, K.D.; Bharadwaj, S.; Iravanchy, S. Timescale analysis for nonlinear dynamical systems. J. Guid. Control Dynam.
**2003**, 26, 318–330. [Google Scholar] [CrossRef] - Lebiedz, D. Computing minimal entropy production trajectories: An approach to model reduction in chemical kinetics. J. Chem. Phys.
**2004**, 120, 6890–6897. [Google Scholar] [CrossRef] [PubMed] - Lebiedz, D.; Reinhardt, V.; Kammerer, J. Novel trajectory based concepts for model and complexity reduction in (bio)chemical kinetics. In Model Reduction and Coarse-Graining Approaches for Multi-Scale Phenomena; Gorban, A.N., Kazantzis, N., Kevrekidis, I.G., Theodoropoulos, C., Eds.; Springer-Verlag: Berlin, Germany, 2006; pp. 343–364. [Google Scholar]
- Walters, P. An Introduction to Ergodic Theory. Graduate Texts in Math.; Springer Verlag: Berlin, Germany, 1982. [Google Scholar]
- Young, L.S. Entropy in dynamical systems. In Entropy; Greven, A., Keller, G., Warnecke, G., Eds.; Princeton University Press: Princeton, NJ, USA, 2003; pp. 313–328. [Google Scholar]
- Brin, M.; Katok, A. On local entropy. In Geometric Dynamics, Lecture Notes in Math., 1007; Palis, J., Ed.; Springer-Verlag: Berlin, Germany, 1983; pp. 30–38. [Google Scholar]
- Ruelle, D. An inequality of the entropy of differentiable maps. Bol. Sc. Bra. Mat.
**1978**, 98, 83–87. [Google Scholar] [CrossRef] - Pesin, Y. Characteristic Lyapunov exponents and smooth ergodic theory. Russ. Math. Surv.
**1977**, 32, 55–114. [Google Scholar] [CrossRef] - Newhouse, S. Entropy and Volume. Ergod. Theor. Dyn. Syst.
**1988**, 8, 283–299. [Google Scholar] [CrossRef] - Yomdin, Y. Volume growth and entropy. Isr. J. Math.
**1987**, 57, 285–300. [Google Scholar] [CrossRef] - Adrover, A.; Creta, F.; Giona, M.; Valorani, M. Stretching-based diagnostics and reduction of chemical kinetic models with diffusion. J. Comput. Phys.
**2007**, 225, 1442–1471. [Google Scholar] [CrossRef] - Reinhardt, V.; Winckler, M.; Lebiedz, D. Approximation of slow attracting manifolds in chemical kinetics by trajectory-based optimization approaches. J. Phys. Chem. A
**2008**, 112, 1712–1718. [Google Scholar] [CrossRef] [PubMed] - Lebiedz, D.; Reinhardt, V.; Siehr, J. Minimal curvature trajectories: Riemannian geometry concepts for slow manifold computation in chemical kinetics. Submitted to J. Comput. Phys.
**2009**. arXiv:0910.3527. [Google Scholar] [CrossRef] - Powell, M.J.D. A fast algorithm for nonlinearly constrained optimization calculations. In Numerical Analysis, volume 630 of Lecture Notes in Mathematics; Dold, A., Eckmann, B., Eds.; Springer-Verlag: Berlin, Germany, 1978; pp. 144–157. [Google Scholar]
- Wächter, A.; Biegler, L.T. On the Implementation of a Primal-Dual Interior Point Filter Line Search Algorithm for Large-Scale Nonlinear Programming. Math. Program.
**2006**, 106, 25–57. [Google Scholar] [CrossRef] - Gorban, A.N.; Karlin, I.V. The invariance equation in differential form. In Invariant Manifolds for Physical and Chemical Kinetics; Lecture Notes in Physics, Volume 660; Gorban, A.N., Karlin, I.V., Eds.; Springer-Verlag: New York, NY, USA, 2005; pp. 65–69. [Google Scholar]
- Gorban, A.N.; Karlin, I.V.; Zinovyev, A. Constructive methods of invariant manifolds for kinetic problems. Phys. Rep.
**2004**, 396, 197–403. [Google Scholar] [CrossRef] - Chiavazzo, E.; Karlin, I.V. Quasi-equilibrium grid algorithm: Geometric construction for model reduction. J. Comput. Phys.
**2008**, 227, 5535–5560. [Google Scholar] [CrossRef] - Lebiedz, D.; Siehr, J.; Unger, J. A variational principle for computing slow invariant manifolds in dissipative dynamical systems. arXiv:0912.1297. 2009. [Google Scholar]

© 2010 by the author; licensee Molecular Diversity Preservation International, Basel, Switzerland. This article is an open-access article distributed under the terms and conditions of the Creative Commons Attribution license http://creativecommons.org/licenses/by/3.0/.

## Share and Cite

**MDPI and ACS Style**

Lebiedz, D.
Entropy-Related Extremum Principles for Model Reduction of Dissipative Dynamical Systems. *Entropy* **2010**, *12*, 706-719.
https://doi.org/10.3390/e12040706

**AMA Style**

Lebiedz D.
Entropy-Related Extremum Principles for Model Reduction of Dissipative Dynamical Systems. *Entropy*. 2010; 12(4):706-719.
https://doi.org/10.3390/e12040706

**Chicago/Turabian Style**

Lebiedz, Dirk.
2010. "Entropy-Related Extremum Principles for Model Reduction of Dissipative Dynamical Systems" *Entropy* 12, no. 4: 706-719.
https://doi.org/10.3390/e12040706