# Speed Gradient and MaxEnt Principles for Shannon and Tsallis Entropies

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

_{1}, …, x

_{n}}.

## 2. Speed-Gradient Principle

^{n}is the system state vector, u is the vector of input (free) variables t ≥ 0. The problem is to derive the law of variation (evolution) of u(t) that satisfies some criterion of “naturalness” of its behavior to give the model features characterizing a real physical system.

**The speed-gradient principle:**of all possible motions, the system implements the ones for which input variables vary in the direction of the speed-gradient of some “goal” functional Q

_{t}. If the constraints are imposed on a motion of the system then the direction is a speed-gradient vector projection on the admissible directions (the ones that satisfy the constraints) set.

#### 2.1. Example 1: Motion of a Particle in the Potential Field

_{1}, x

_{2}, x

_{3})

^{T}consists of coordinates x

_{1}, x

_{2}, x

_{3}of a particle. Choose smooth Q(x) as the potential energy of a particle and derive the speed-gradient law in the differential form. To this end, calculate the speed

_{3}is the 3×3 identity matrix, we arrive at the Newton’s law $\dot{u}=-{m}^{-1}{\nabla}_{x}Q(x)$or $m\ddot{x}=-{\nabla}_{x}Q(x)$.

^{−1}. Admitting dependence of the metric matrix Γ on x one can obtain evolution laws for complex mechanical systems described by Lagrangian or Hamiltonian formalism.

#### 2.2. Example 2: Wave, Diffusion and Heat Transfer Equations

_{1}, r

_{2}, r

_{3})

^{T}∈ Ω be the temperature field or the concentration of a substance field defined in the domain Ω ∈ R

^{3}. Choose the goal functional evaluating non-uniformity of the field as follows

_{t}yields

#### 2.3. GENERIC and SG-Principle

_{t}= Λ.

## 3. Jaynes’s Maximum Entropy Principle

_{I}(1) is the most objective method to define the distribution.

_{m}can be derived from conditions (15).

## 4. Maximization of the Shannon Entropy with the Speed-Gradient Method

_{i}are changing continuously. Then the law of motion can be represented as:

_{i}= u

_{i}(t) are control functions (6) which has to be determined.

_{i}is the energy of particle in the ith cell and the total energy does not change, then the evolution law has the form:

_{S}is a symmetric m × m matrix defined as follows:

## 5. The Speed-Gradient Dynamics of the Tsallis Entropy Maximization Process

_{1}, …, N

_{m})

^{T}is the state vector of the system.

_{i}considered as frozen parameters.

_{i}. In this mode ${\dot{N}}_{i}=0$. Based on (25) it means that $m{N}_{i}^{q-1}={\displaystyle {\sum}_{i=1}^{m}{N}_{i}^{q-1}}$ which is possible only when all N

_{i}are equal. According to constraint (19) we have that ${N}_{i}=\frac{N}{m}$. This result corresponds to the maximum state of classical entropy and agrees with thermodynamics.

#### 5.1. Equilibrium Stability

_{max}(q) is a maximum possible value for the Tsallis entropy with parameter q.

^{m}:

_{i}are equal. This is the maximum of entropy state. Thus the law (25) provides global asymptotic stability of the maximum entropy state. The physical meaning of this law is nothing but moving along the direction of the maximum entropy production rate (direction of the fastest entropy growth).

## 6. Internal Energy Constraint

_{1}and λ

_{2}are Lagrange multipliers that can be defined by substitution of (27) into (19) and (22).

_{1}and λ

_{2}are given by formulas

_{i}are equal.

_{1}and λ

_{2}from (28) into Equation (27). In abbreviated form we represent this law as

#### 6.1. Equilibrium Stability

_{max}(q) – S(X, q) is Lyapunov function and there is a unique stable equilibrium state of the system in non-degenerate cases. Let us demonstrate it.

_{1}and λ

_{2}from (28) into Equation (27) and the expression for u

_{i}we substitute into (30). Result expression is

_{1}, …, E

_{m}) we get for (31) that $\dot{V}(q)\le 0$. And $\dot{V}(q)=0$ occurs for the only one case when $\exists \mathrm{\lambda},\mu \in \mathbb{R}:\phantom{\rule{0.2em}{0ex}}{N}_{i}^{q-1}=\mathrm{\lambda}{E}_{i}+\mu $ for all i. Due to (27) at the equilibrium state of the system the following equalities hold:

_{1}and λ

_{2}defined in (28).

#### 6.2. Correspondence to the Tsallis Distribution

_{i}from (36) into (19). We get

_{1}in our notation). Following by notation of C. Tsallis (see Equation (10) in [2]) we have that ${\lambda}_{1}=-{\lambda}_{2}\beta (q-1)$. It explains the variable substitution in (39).

_{i}from (36) into (22). Then we get

## 7. Conclusions

**How the system will evolve?**This fact distinguishes the SG-principle from the principle of maximum entropy, the principle of maximum Fisher information and others characterizing the steady-state processes and providing an answer to the questions of

**To where?**and

**How far?**

## Acknowledgments

**PACS classifications:**65.40.Gr; 05.45.-a; 02.30.Yy; 02.50.-r

## Author Contributions

## Conflicts of Interest

## References

- Shannon, C. A mathematical theory of communication. Bell Syst. Tech. J
**1948**, 27, 379–423. [Google Scholar] - Tsallis, C. Possible generalization of Boltzmann–Gibbs statistics. J. Stat. Phys.
**1988**, 52, 479–487. [Google Scholar] - Gorban, A.N.; Gorban, P.A.; Judge, G. Entropy: The Markov ordering approach. Entropy
**2010**, 12, 1145–1193. [Google Scholar] - Cressie, N.; Read, T. Multinomial goodness of fit tests. J. R. Stat. Soc. Ser. B
**1984**, 46, 440–464. [Google Scholar] - Read, T.; Cressie, N. Goodness of Fit Statistics for Discrete Multivariate Data; Springer: New York, NY, USA, 1988. [Google Scholar]
- Tsallis, C. Nonextensive statistical mechanics and thermodynamics: Bibliography. Available online: http://tsallis.cat.cbpf.br/TEMUCO.pdf accessed on 5 March 2015.
- Jaynes, E. The minimum entropy production principle. Ann. Rev. Phys. Chem.
**1980**, 31, 579–601. [Google Scholar] - Jaynes, E. Information theory and statistical mechanics. Phys. Rev.
**1957**, 106, 620–630. [Google Scholar] - Chen, Y. The rank-size scaling law and entropy-maximizing principle. Phys. A Stat. Mech. Appl.
**2012**, 391, 767–778. [Google Scholar] - Martyushev, L.M. Entropy and entropy production: Old misconceptions and new breakthroughs. Entropy
**2013**, 15, 1152–1170. [Google Scholar] - Fanelli, D.; Ninno, G.D.; Turchi, A. On the paradoxical behavior of a cyclic device working with a non-Boltzmannian fluid. Commun. Nonlinear Sci. Numer. Simul.
**2012**, 17, 4174–4180. [Google Scholar] - Fradkov, A.L. Cybernetical Physics: From Control of Chaos to Quantum Control; Springer: Berlin/Heidelberg, Germany, 2007. [Google Scholar]
- Fradkov, A.L. Speed-gradient entropy principle for nonstationary processes. Entropy
**2008**, 10, 757–764. [Google Scholar] - Fradkov, A.; Krivtsov, A. Speed-gradient principle for description of transient dynamics in systems obeying maximum entropy principle, Proceedings of the 30th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering, Chamonix, France, 4–9 July 2010; 1305, pp. 399–406.
- Grmela, M.; Öttinger, H. Dynamics and thermodynamics of complex fluids. I. Development of a general formalism. Phys. Rev. E
**1997**, 56, 6620–6632. [Google Scholar] - Öttinger, H.; Grmela, M. Dynamics and thermodynamics of complex fluids. II. Illustrations of a general formalism. Phys. Rev. E
**1997**, 56, 6633–6655. [Google Scholar] - Morimoto, T. Markov processes and the H-theorem. J. Phys. Soc. Jpn.
**1963**, 12, 328–331. [Google Scholar] - Lanczos, C. The Variational Principles of Mechanics; University of Toronto Press: Toronto, ON, Canada, 1962. [Google Scholar]
- Planck, M. The unity of physical world views. Phys. Z.
**1909**, 10, 62–75. [Google Scholar] - Fradkov, A.; Miroshnik, I.; Nikiforov, V. Nonlinear Adaptive Control Complex Systems; Springer: Berlin/Heidelberg, Germany, 1999. [Google Scholar]
- Tsallis, C; Mendes, R.; Plastino, A. The role of constraints within generalized nonextensive statistics. Physica A
**1998**, 261, 534–554. [Google Scholar] - Esteban, M.; Morales, D. A summary of entropy statistics. Kybernetica
**1995**, 31, 337–346. [Google Scholar] - Rényi, A. On measures of entropy and information, Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability; University of California Press: Berkeley, CA, USA, 1961; 1, pp. 547–561.
- Burg, J. The relationship between maximum entropy spectra and maximum likelihood spectra. Geophysics
**1972**, 37, 375–376. [Google Scholar] - Gorban, A.N. Maxallent: Maximizers of all entropies and uncertainty of uncertainty. Comput. Math. Appl.
**2013**, 65, 1438–1456. [Google Scholar]

© 2015 by the authors; licensee MDPI, Basel, Switzerland This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Fradkov, A.L.; Shalymov, D.S.
Speed Gradient and MaxEnt Principles for Shannon and Tsallis Entropies. *Entropy* **2015**, *17*, 1090-1102.
https://doi.org/10.3390/e17031090

**AMA Style**

Fradkov AL, Shalymov DS.
Speed Gradient and MaxEnt Principles for Shannon and Tsallis Entropies. *Entropy*. 2015; 17(3):1090-1102.
https://doi.org/10.3390/e17031090

**Chicago/Turabian Style**

Fradkov, Alexander L., and Dmitry S. Shalymov.
2015. "Speed Gradient and MaxEnt Principles for Shannon and Tsallis Entropies" *Entropy* 17, no. 3: 1090-1102.
https://doi.org/10.3390/e17031090