# Tsallis Relative Entropy and Anomalous Diffusion

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Introduction to Relative Entropies

## 3. Space-Fractional Diffusion Equation and Stable Distributions

- The stable distribution rescales as$$\mathbf{S}\left(\right)open="("\; close=")">\left(\right)close="|">x\alpha ,\beta ,\gamma ,{\delta}_{n};n$$
- For $n=0,1$ stable distributions satisfy the reflection property [47]$$\mathbf{S}\left(\right)open="("\; close=")">\left(\right)close="|">x\alpha ,\beta ,1,0;n$$
- The position of the mean μ of the distribution is given by$$\mu ={\delta}_{1}={\delta}_{0}-\beta \phantom{\rule{0.166667em}{0ex}}\gamma \phantom{\rule{0.166667em}{0ex}}tan\left(\right)open="("\; close=")">\frac{\pi \phantom{\rule{0.166667em}{0ex}}\alpha}{2}$$
- Stable distributions are unimodal distributions. The mode ${\widehat{x}}_{\alpha}$ depends on the parametrization and on the location parameter. For $\mathbf{S}\left(\right)open="("\; close=")">\left(\right)close="|">x\alpha ,\beta ,\gamma ,{\delta}_{0};0$ it is given by$$\begin{array}{cc}\hfill {\widehat{x}}_{\alpha}& =\gamma \phantom{\rule{0.166667em}{0ex}}m(\alpha ,\beta )+{\delta}_{0}=\gamma \phantom{\rule{0.166667em}{0ex}}m(\alpha ,\beta )+{\delta}_{1}+\beta \phantom{\rule{0.166667em}{0ex}}\gamma \phantom{\rule{0.166667em}{0ex}}tan\left(\right)open="("\; close=")">\frac{\pi \phantom{\rule{0.166667em}{0ex}}\alpha}{2}\hfill \end{array}$$
- For $\alpha \in (0,2)$ the stable distributions do not have a finite second moment, i.e., a finite variance.

## 4. Reference Distribution

**Figure 1.**In this graph the solution ${P}_{\alpha}(x,t)$ for $t=1$ and $D=1$ is shown for different values of $\alpha =2.0,1.5$, and $1.2$. Note that for $\alpha =2$ the notation ${P}_{\mathrm{D}}(x,t)$ is used in the following.

## 5. Kullback–Leibler Entropy

**Figure 2.**Here the Kullback–Leibler entropy $K({P}_{\mathrm{D}},{P}_{\alpha})$, the Tsallis entropy ${S}_{q=0.7}^{\mathrm{T}}\left({P}_{\alpha}\right)$ and the Shannon entropy $S\left({P}_{\alpha}\right)$ are plotted over α at $t=1$. We can see that the Kullback–Leibler entropy shows a monotonic decreasing behavior for increasing α, whereas the Tsallis and the Shannon entropy exhibit a maximum. Thus, $K({P}_{\mathrm{D}},{P}_{\alpha})$ is an appropriate ordering measure for the bridging regime even when other measure candidates are not monotonic.

## 6. Tsallis Relative Entropy

**Figure 3.**The results for the Tsallis relative entropy ${T}_{q}({P}_{\alpha},{P}_{\mathrm{D}})$ are shown over α for four different values of q ($q=0.1,0.3,0.6$, and $0.9$). As expected ${T}_{q}({P}_{\alpha},{P}_{\mathrm{D}})$ goes to zero as α approaches 2, independent of q.

**Figure 4.**For the case of $q=0.6$ the results for the Tsallis relative entropy ${T}_{0.6}({P}_{\alpha},{P}_{\mathrm{D}})$ for different times t is given over α. One can observe that with increasing time the monotonic decreasing behavior is preserved.

**Figure 5.**Here, the Tsallis relative entropy ${T}_{q}({P}_{\mathrm{D}},{P}_{\alpha})$ is depicted over α for different values of q. We find that for $\alpha \to 2$${T}_{q}({P}_{\mathrm{D}},{P}_{\alpha})$ goes down to zero, independent of q. Note that in the case of $q=1$ the corresponding Kullback–Leibler entropy is shown.

**Figure 6.**The log-log-plot of the Kullback–Leibler entropy $K({P}_{\mathrm{D}},{P}_{\alpha})$ over the Tsallis relative entropy ${T}_{q}({P}_{\mathrm{D}},{P}_{\alpha})$ for different q-values is given. A clear monotonic monotonic relationship is observed independent of q.

## 7. Summary and Discussion

## Acknowledgments

## References

- Havlin, S.; Ben-Avraham, D. Diffusion in disordered media. Adv. Phys.
**1987**, 36, 695–798. [Google Scholar] [CrossRef] - Schirmacher, W.; Perm, M.; Suck, J.B.; Heidemann, A. Anomalous diffusion of hydrogen in amorphous metals. Europhys. Lett.
**1990**, 13, 523–529. [Google Scholar] [CrossRef] - Weiss, M.; Elsner, M.; Kartberg, F.; Nilsson, T. Anomalous subdiffusion is a measure for cytoplasmic crowding in living cells. Biophys. J.
**2004**, 87, 3518–3524. [Google Scholar] [CrossRef] [PubMed] - Banks, D.S.; Fradin, C. Anomalous diffusion of proteins due to molecular crowding. Biophys. J.
**2005**, 89, 2960–2971. [Google Scholar] [CrossRef] [PubMed] - Köpf, M.; Corinth, C.; Haferkamp, O.; Nonnenmacher, T.F. Anomalous diffusion of water in biological tissues. Biophys. J.
**1996**, 70, 2950–2958. [Google Scholar] [CrossRef] - Yuste, S.B.; Lindberg, K. Subdiffusion-limited reactions. Chem. Phys.
**2002**, 284, 169–180. [Google Scholar] [CrossRef] - Bénichou, O.; Coppey, M.; Moreau, M.; Suet, P.H.; Voituriez, R. Optimal search strategies for hidden targets. Phys. Rev. Lett.
**2005**, 94, 198101. [Google Scholar] [CrossRef] [PubMed] - Bénichou, O.; Loverdo, C.; Moreau, M.; Voituriez, R. Two-dimensional intermittent search processes: An alternative to Lévy flight strategies. Phys. Rev. E
**2006**, 74, 020102. [Google Scholar] [CrossRef] - Shlesinger, M.F. Mathematical physics: Search research. Nature
**2006**, 443, 281–282. [Google Scholar] [CrossRef] [PubMed] - Solomon, T.H.; Weeks, E.R.; Swinney, H.L. Observation of anomalous diffusion and Lévy flights in a two-dimensional rotating flow. Phys. Rev. Lett.
**1993**, 71, 3975–3978. [Google Scholar] [CrossRef] [PubMed] - Hansen, A.E.; Jullien, M.C.; Paret, J.; Tabeling, P. Dispersion in freely decaying and forced 2D turbulence. In Anomalous Diffusion from Basics to Application; Pekalski, A., Kutner, R., Eds.; Lecture Notes in Physics, 519; Springer-Verlag: Berlin/Heidelberg, Germany, 1999; pp. XVIII, 378 S. [Google Scholar]
- Schneider, W.R.; Wyss, W. Fractional diffusion and wave equation. J. Math. Phys.
**1989**, 30, 134–144. [Google Scholar] [CrossRef] - Giona, M.; Roman, H.E. Fractional diffusion equation for transport phenomena in random media. Physica A
**1992**, 185, 87–97. [Google Scholar] [CrossRef] - Metzler, R.; Glöckle, W.G.; Nonnenmacher, T.F. Fractional model equation for anomalous diffusion. Physica A
**1994**, 211, 13–24. [Google Scholar] [CrossRef] - Metzler, R.; Klafter, J. The random walk’s guide to anomalous diffusion: A fractional dynamics approach. Phys. Rep.
**2000**, 339, 1–77. [Google Scholar] [CrossRef] - Hilfer, R. Applications of Fractional Calculus in Physics; World Scientific Publishing: Singapore, 2000. [Google Scholar]
- del Castillo-Negrete, D. Fractional diffusional models of anomalous transport. In Anomalous Transport: Foundations and Applications; Klages, R., Radons, G., Sokolov, I.M., Eds.; Wiley-VCH: Weinheim, Germany, 2008; Chapter 6; pp. 163–212. [Google Scholar]
- Schulzky, C.; Essex, C.; Davison, M.; Franz, A.; Hoffmann, K.H. The similarity group and anomalous diffusion equations. J. Phys. A: Math. Gen.
**2000**, 33, 5501–5511. [Google Scholar] [CrossRef] - Fischer, A.; Seeger, S.; Hoffmann, K.H.; Essex, C.; Davison, M. Modeling anomalous superdiffusion. J. Phys. A: Math. Gen.
**2007**, 40, 11441–11452. [Google Scholar] [CrossRef] - Hoffmann, K.H.; Prehl, J. Anomalous transport in disordered fractals. In Anomalous Transport—Foundations and Applications, 1st ed.; Klages, R., Radons, G., Sokolov, I.M., Eds.; Wiley-VCH: Weinheim, Germany, 2008. [Google Scholar]
- Malacarne, L.C.; Mendes, R.S.; Pedron, I.T.; Lenzi, E.K. Nonlinear equation for anomalous diffusion: Unified power-law and stretched exponential exact solution. Phys. Rev. E
**2001**, 63, 030101(R). [Google Scholar] [CrossRef] - Pedron, I.T.; Mendes, R.S.; Buratta, T.J.; Malacarne, L.C.; Lenzi, E.K. Logarithmic diffusion and porous media equations: a unified description. Phys. Rev. E
**2005**, 72, 031106. [Google Scholar] [CrossRef] - Li, X.; Essex, C.; Davison, M.; Hoffmann, K.H.; Schulzky, C. Fractional diffusion, irreversibility and entropy. J. Non-Equilib. Thermodyn.
**2003**, 28, 279–291. [Google Scholar] [CrossRef] - Prehl, J.; Essex, C.; Hoffmann, K.H. The superdiffusion entropy production paradox in the space-fractional case for extended entropies. Physica A
**2010**, 389, 214–224. [Google Scholar] [CrossRef] - Hoffmann, K.H.; Essex, C.; Schulzky, C. Fractional diffusion and entropy production. J. Non-Equilib. Thermodyn.
**1998**, 23, 166–175. [Google Scholar] [CrossRef] - Essex, C.; Schulzky, C.; Franz, A.; Hoffmann, K.H. Tsallis and Rényi entropies in fractional diffusion and entropy production. Physica A
**2000**, 284, 299–308. [Google Scholar] [CrossRef] - Kullback, S.; Leibler, R.A. On information and sufficiency. Ann. Math. Stat.
**1951**, 22, 79–86. [Google Scholar] [CrossRef] - Schlögl, F. Probability and Heat; Vieweg: Braunschweig/Wiesbaden, Germany, 1989. [Google Scholar]
- Ebrahimi, N. Testing exponentiality of the residual life, based on dynamic Kullback–Leibler information. IEEE Trans. Reliab.
**1998**, 47, 197–201. [Google Scholar] [CrossRef] - Lim, A.E.B.; Shanthikumar, J.G. Relative entropy, exponential utility, and robust dynamic pricing. Oper. Res.
**2007**, 55, 198–214. [Google Scholar] [CrossRef] - Kekre, H.B.; Bharadi, V.A.; Shaktia, P.; Shah, V.; Ambardekar, A.A. Keystroke dynamic analysis using relative entropy & timing sequence Euclidean distance. In Proceedings of the International Conference and Workshop on Emerging Trends in Technology (ICWET 2011), Mumbai, India, 25–26 February 2011; pp. 220–223.
- Yu, S.; Mehta, P.G. The Kullback–Leibler rate metric for comparing dynamical systems. In Proceedings of the Joint 48th IEEE Conference on Decision and Control and 28th Chinese Control Conference, Shanghai, China, 16–18 December 2009; pp. 8363–8368.
- Do, M.N. Fast approximation of Kullback–Leibler distance for dependence trees and hidden Markov models. IEEE Signal Process. Lett.
**2003**, 10, 115–118. [Google Scholar] [CrossRef] - Llody, S.; Pagels, H. Complexity as thermodynamic depth. Ann. Physics
**1988**, 188, 186–213. [Google Scholar] [CrossRef] - Crutchfield, J.P.; Shalizi, C.R. Thermodynamic depth of casual states: Objective complexity via minimal representation. Phys. Rev. E
**1999**, 59, 275–283. [Google Scholar] [CrossRef] - Vedral, V. The role of relative entropy in quantum information theory. Rev. Mod. Phys.
**2002**, 74, 197–234. [Google Scholar] [CrossRef] - Georgiou, T.T.; Lindquist, A. Kullback–Leibler approximation of spectral density functions. IEEE Trans. Inform. Theor.
**2003**, 49, 2910–2917. [Google Scholar] [CrossRef] - Braunstein, S.L. Geometry of quantum inference. Phys. Lett. A
**1996**, 219, 169–174. [Google Scholar] [CrossRef] - Modi, K.; Paterek, T.; Son, W.; Vedral, V.; Williamson, M. Unified view of quantum and classical correlations. Phys. Rev. Lett.
**2010**, 104, 080501–1–4. [Google Scholar] [CrossRef] [PubMed] - Do, M.N.; Vetterli, M. Wavelet-based texture retrieval using generalized Gaussian density and Kullback–Leibler distance. IEEE Trans. Image Process.
**2002**, 11, 146–158. [Google Scholar] [CrossRef] [PubMed] - Burnham, K.P.; R., A.D. Kullback–Leibler information as a basis for strong inference in ecological studies. Wildl. Res.
**2001**, 28, 111–119. [Google Scholar] [CrossRef] - Tsallis, C. Generalized entropy-based criterion for consistent testing. Phys. Rev. E
**1998**, 58, 1442–1445. [Google Scholar] [CrossRef] - Borland, L.; Plastino, A.R.; Tsallis, C. Information gain within nonextensive thermostatics. J. Math. Phys.
**1998**, 39, 6490–6501. [Google Scholar]Borland, L.; Plastino, A.R.; Tsallis, C. Erratum: “Information gain with generalized termostatistics”[J. Math. Phys. 39, 6490 (1998)]. J. Math. Phys.**1999**, 40, 2196. [Google Scholar] [CrossRef] - Furuichi, S.; Yanagi, K.; Kuriyama, K. Fundamental properties of Tsallis relative entropy. J. Math. Phys.
**2004**, 45, 4868–4877. [Google Scholar] [CrossRef] - Plastino, A.R.; Miller, H.G.; Plastino, A. Minimum Kullback entropy approach to the Fokker–Planck equation. Phys. Rev. E
**1997**, 56, 3927–3934. [Google Scholar] [CrossRef] - Nolan, J.P. Stable Distributions—Models for Heavy Tailed Data; Birkhäuser: Boston, MA, USA, 2012. Unpublished material. Chapter 1: Available online: http://academic2.american.edu/~jpnolan (accessed on 30 March 2012). Chapter 3: Personal communication.
- Samorodnitsky, G.; Taqqu, M.S. Stable non-Gaussian Random Processes; Chapman & Hall: New York, NY, USA, 1994. [Google Scholar]
- Tsallis, C.; Plastino, A.R.; Alvarez-Estrade, R.F. Escort mean values and the characterization of power-law-decaying probability densities. J. Math. Phys.
**2009**, 50, 043303. [Google Scholar] [CrossRef] [Green Version]

© 2012 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/.)

## Share and Cite

**MDPI and ACS Style**

Prehl, J.; Essex, C.; Hoffmann, K.H.
Tsallis Relative Entropy and Anomalous Diffusion. *Entropy* **2012**, *14*, 701-716.
https://doi.org/10.3390/e14040701

**AMA Style**

Prehl J, Essex C, Hoffmann KH.
Tsallis Relative Entropy and Anomalous Diffusion. *Entropy*. 2012; 14(4):701-716.
https://doi.org/10.3390/e14040701

**Chicago/Turabian Style**

Prehl, Janett, Christopher Essex, and Karl Heinz Hoffmann.
2012. "Tsallis Relative Entropy and Anomalous Diffusion" *Entropy* 14, no. 4: 701-716.
https://doi.org/10.3390/e14040701