Entropic Distance for Nonlinear Master Equation

Abstract

More and more works deal with statistical systems far from equilibrium, dominated by unidirectional stochastic processes, augmented by rare resets. We analyze the construction of the entropic distance measure appropriate for such dynamics. We demonstrate that a power-like nonlinearity in the state probability in the master equation naturally leads to the Tsallis (Havrda–Charvát, Aczél–Daróczy) q-entropy formula in the context of seeking for the maximal entropy state at stationarity. A few possible applications of a certain simple and linear master equation to phenomena studied in statistical physics are listed at the end.

1. Definition and Properties of Entropic Distance

Dealing with the dynamics of classical probabilities, we would like to propose a general recipe for defining the corresponding formula for the entropic divergence between two probability distributions. Our goal is to handle complex systems with a stochastic dynamics, generalized to nonlinear dependence on the probabilities. For the study of quantum state probabilities and their distance measures we refer to a recent paper [1] and references therein.

Entropic distance, more properly called “entropic divergence”, is traditionally interpreted as a relative entropy, as a difference between entropies with a prior condition, and without [2]. It is also the Boltzmann–Shannon entropy of a distribution relative to another [3]. Looking at this construction, however, from the viewpoint of a generalized entropy [4], the simple difference or logarithm of a ratio cannot be held as a definition anymore.

Instead, in this paper, we explore a reverse engineering concept: seeking an entropic divergence formula, which is subject to some wanted properties, we consider entropy as a derived quantity. More precisely, we seek entropic divergence formulas appropriate for given stochastic dynamics, shrinking during the approach to a stationary distribution, whenever it exists, and establish the entropy formula from this distance to the uniform distribution. By doing so we serve two goals: (i) having constructed a non-negative entropic distance we derive an entropy formula which is maximal for the uniform distribution; and (ii) we come as near as possible to the classical difference formula for the relative entropy.

Starting from a given master equation, it is far from trivial that which is the most suitable entropy divergence formula for analyzing the stability of a stationary solution. In the present paper we provide a general procedure to obtain a general entropic divergence formula for atypical cases. Although we exemplify only the well-known cases of the logarithmic formula of the Kullback–Leibler and that of the Renyi divergence, our result readily generalizes to an infinite number of cases, distinguished by the dependence on the initial state probability at each transition term.

We start our discussion by contrasting the definition of the metric distance, known from geometry, to the basic properties of an entropic distance. The metric distance possesses the following properties:

• $\rho (P,Q)\ge 0$ for a pair of points P and Q,
• $\rho (P,Q)=0$ only for $P=Q$,
• $\rho (P,Q)=\rho (Q,P)$ symmetric measure,
• $\rho (P,Q)\le \rho (P,R)+\rho (R,Q)$, the triangle inequality in elliptic spaces.

The entropic divergence on the other hand is neither necessarily symmetric, nor can satisfy a triangle inequality. On the other hand it is subject to the second law of thermodynamics, distinguishing the time arrow from the past to the future. We require for a real functional, $\rho [P,Q]$, depending on the distributions $P_n$ and $Q_n$, the followings to hold:

• $\rho [P,Q]\ge 0$ for a pair of distributions $P_n$ and $Q_n$,
• $\rho [P,Q]=0$ only if the distributions coincide $P_n=Q_n$,
• $\frac{\mathrm{d}}{\mathrm{d}t}\rho [P,Q]\le 0$ if $Q_n$ is the stationary distribution,
• $\frac{\mathrm{d}}{\mathrm{d}t}\rho [P,Q]=0$ only for $P_n=Q_n$, i.e., the stationary distribution is unique.

Although this definition is not symmetric in the handling of the normalized distributions $P_n$ and $Q_n$, it is an easy task to consider the symmetrized version, $s[P,Q]\equiv \rho [P,Q]+\rho [Q,P]$. This symmetrized, entropic divergence inherits some properties from the fiducial construction. Considering a scaling trace form entropic divergence, $\rho [P,Q]={\displaystyle \sum _n}\sigma \left({\xi }_n\right)Q_n$ with ${\xi }_n=P_n/Q_n$, to begin with, we identify the following symmetrized kernel function:

$$\mathfrak{s}\left(\xi \right):=\sigma \left(\xi \right)+\xi \sigma (1/\xi ).$$

(1)

The only constraint is to start with a core function, $\sigma \left(\xi \right)$ with a definite concavity. Jensen inequality tells for ${\sigma }^{″}>0$ that

$$\sum _n\sigma \left({\xi }_n\right)Q_n\ge \sigma \left(\sum _n{\xi }_n{Q}_n\right)=\sigma \left(\sum _n{P}_n\right)=\sigma \left(1\right).$$

(2)

For satisfying property 1 and 2 one simply sets $\sigma \left(1\right)=0$. Interestingly enough, this setting suffices also for the satisfaction of the second law of thermodynamics, formulated above as further constraints 3 and 4. As a consequence of the symmetrization, it also follows that $\mathfrak{s}\left(1\right)=0$ and ${\mathfrak{s}}^{″}>0$.

The symmetrized entropic divergence shows some new, emergent properties. We list its derivatives as follows:

$$\begin{array}{ccc}\hfill \mathfrak{s}\left(\xi \right)& =& \sigma \left(\xi \right)+\xi \sigma (1/\xi )\hfill \\ \hfill {\mathfrak{s}}^{\prime }\left(\xi \right)& =& {\sigma }^{\prime }\left(\xi \right)+\sigma (1/\xi )-\frac{1}{\xi }{\sigma }^{\prime }(1/\xi )\hfill \\ \hfill {\mathfrak{s}}^{″}\left(\xi \right)& =& {\sigma }^{″}\left(\xi \right)-\cancel{\frac{1}{{\xi }^2}{\sigma }^{\prime }(1/\xi )}+\cancel{\frac{1}{{\xi }^2}{\sigma }^{\prime }(1/\xi )}+\frac{1}{{\xi }^3}{\sigma }^{″}(1/\xi ).\hfill \end{array}$$

(3)

The consequences, listed below, can be derived from these general relations:

• $\mathfrak{s}\left(1\right)=2\sigma \left(1\right)=0$,
• ${\mathfrak{s}}^{\prime }\left(1\right)=\sigma \left(1\right)=0$,
• ${\mathfrak{s}}^{″}>0\Rightarrow {\xi }_m=1$ is a minimum,
• $\mathfrak{s}\left(\xi \right)\ge 0$.

In this way the kernel function, and hence each summand in the symmetrized entropic divergence formula, is non-negative, not only the total sum.

2. Entropic Distance Evolution Due to Linear Stochastic Dynamics

Now we study properties 3 and 4, by evaluating the rate of change of the entropic divergence in time. This change is based on the dynamics (time evolution) of the evolving distribution, $P_n\left(t\right)$, while the targeted stationary distribution, $Q_n$ is, by definition, time independent. First we consider a class of stochastic evolutions governed by differential equations for ${\dot{P}}_n\left(t\right)\equiv \frac{\mathrm{d}{P}_n}{\mathrm{d}t}$, linear in the distribution, $P_n\left(t\right)$ [5]. We consider the trace form $\rho [P,Q]={\displaystyle \sum _n}{Q}_n\sigma \left(\frac{P_n}{Q_n}\right)$ and the background master equation

$${\dot{P}}_n=\sum _m\left(w_{nm}{P}_m-w_{mn}{P}_n\right).$$

(4)

The antisymmetrized sum in the above equation is merely to ensure the conservation of the norm, ${\displaystyle \sum _n}{P}_n=1$, during the time evolution. Using again the notation ${\xi }_n=P_n/Q_n$ we obtain

$$\dot{\rho }=\sum _n{\sigma }^{\prime }\left({\xi }_n\right){\dot{P}}_n=\sum _{n,m}{\sigma }^{\prime }\left({\xi }_n\right)\left(w_{nm}{\xi }_m{Q}_m-w_{mn}{\xi }_n{Q}_n\right).$$

(5)

The basic trick is to apply the splitting ${\xi }_m={\xi }_n+({\xi }_m-{\xi }_n)$ to get

$$\dot{\rho }=\sum _n{\sigma }^{\prime }\left({\xi }_n\right){\xi }_n\cancel{\sum _m\left(w_{nm}{Q}_m-w_{mn}{Q}_n\right)}+\sum _{n,m}{\sigma }^{\prime }\left({\xi }_n\right)({\xi }_m-{\xi }_n)w_{nm}{Q}_m.$$

(6)

Here the sum in the first term vanishes due to the very definition of the stationary distribution, $Q_n$. For estimating the remaining term, we utilize the Taylor series remainder theorem in the Lagrange form. We recall the Taylor expansion of the kernel function $\sigma \left(\xi \right)$,

$$\sigma \left({\xi }_m\right)=\sigma \left({\xi }_n\right)+{\sigma }^{\prime }\left({\xi }_n\right)({\xi }_m-{\xi }_n)+\frac{1}{2}{\sigma }^{″}\left(c_{mn}\right){({\xi }_n-{\xi }_m)}^2,$$

(7)

with $c_{mn}\in [{\xi }_m,{\xi }_n]$. Here the first derivative term has occurred in Equation (6). This construction delivers

$$\dot{\rho }=\cancel{\sum _{n,m}\left[\sigma \left({\xi }_m\right)-\sigma \left({\xi }_n\right)\right]w_{nm}{Q}_m}-\frac{1}{2}\sum _{n,m}{\sigma }^{″}\left(c_{mn}\right){\left({\xi }_m-{\xi }_n\right)}^2{w}_{nm}{Q}_m.$$

(8)

Here the first sum vanishes again—after exchanging the indices m and n in the first summand, the result is proportional to the total balance expression, which is zero for the stationary distribution. With positive transition rates, $w_{nm}>0$ the approach to stationary distribution, $\dot{\rho }\le 0$ is hence proven for all ${\sigma }^{″}>0$. We note that we never used the detailed balance condition for the transition rates, only the vanishing of the total balance, which defines the stationary distribution.

This proof, without recalling the detailed balance condition as Boltzmann’s famous H-theorem did, is quite general. Any core function with positive second derivative and the scaling trace form co-act to ensure the correct change in time. By using the traditional choice, $\sigma \left(\xi \right)=-ln\xi$, we have ${\sigma }^{\prime }=-1/\xi$ and ${\sigma }^{″}\left(\xi \right)=1/{\xi }^2>0$, satisfying indeed all requirements. The integrated entropic divergence formula (no symmetrization) in this case is given as the Kullback–Leibler divergence :

$$\rho [P,Q]=\sum _n{Q}_{n}ln\frac{Q_n}{P_n}.$$

(9)

There is a rationale behind using the logarithm function. It is the only one being additive for the product form of its argument, mapping factorizing and hence statistically independent distributions to an additive entropic divergence kernel: For $P_n^{\left(12\right)}=P_n^{\left(1\right)}{P}_n^{\left(2\right)}$ also $Q_n^{\left(12\right)}=Q_n^{\left(1\right)}{Q}_n^{\left(2\right)}$ therefore we have ${\xi }_n^{\left(12\right)}={\xi }_n^{\left(1\right)}{\xi }_n^{\left(2\right)}$. Aiming at $\sigma \left({\xi }^{\left(12\right)}\right)=\sigma \left({\xi }^{\left(1\right)}\right)+\sigma \left({\xi }^{\left(2\right)}\right)$, the solution is $\sigma \left(\xi \right)=\alpha ln\xi$. For ${\sigma }^{″}>0$ it must be $\alpha <0$, so without restricting generality one chooses $\alpha =-1$.

Finally, we would like to treat this entropic divergence as an entropy difference. This is achieved when comparing the stationary distribution to the uniform distribution, $U_n=1/W,n=1,2,\dots ,W$. Using the above Kullback–Leibler divergence formula one easily derives

$$\rho [U,Q]=\sum _{n=1}^W{Q}_{n}ln\left(W{Q}_n\right)=lnW+\sum _n{Q}_{n}ln{Q}_n=S_{BG}\left[U\right]-S_{BG}\left[Q\right]$$

(10)

with

$$S_{BG}\left[Q\right]=-\sum _n{Q}_{n}ln{Q}_n,$$

(11)

being the Boltzmann–Gibbs–Planck–Shannon entropy formula. From the Jensen inequality it follows $\rho [U,Q]\ge 0$, so $S_{BG}\left[U\right]\ge S_{BG}\left[Q\right]$.

3. Entropic Divergence Evolution for Nonlinear Master Equations

Detailed balance is also not needed for a more general dynamics. We consider Markovian dynamics, with a master equation nonlinear in the distribution, $P_n$, as

$${\dot{P}}_n=\sum _m\left[w_{nm}a\left(P_m\right)-w_{mn}a\left(P_n\right)\right].$$

(12)

The stationarity condition defines

$$0=\sum _m\left[w_{nm}a\left(Q_m\right)-w_{mn}a\left(Q_n\right)\right].$$

(13)

The entropic distance formula is sought for in the trace form (but this time without the scaling assumption):

$$\rho [P,Q]=\sum _n\sigma (P_n,Q_n),$$

(14)

the dependence on $Q_n$ is fixed by $\rho [Q,Q]=0$. The change of the entropic divergence in this case is given by

$$\dot{\rho }=\sum _{m,n}\frac{\partial \sigma }{\partial P_n}\left[w_{nm}a\left(Q_m\right){\xi }_m-w_{mn}a\left(Q_n\right){\xi }_n\right]$$

(15)

with ${\xi }_n:=a\left(P_n\right)/a\left(Q_n\right)$. We again put ${\xi }_m={\xi }_n+({\xi }_m-{\xi }_n)$ in the first summand:

$$\dot{\rho }=\sum _n\frac{\partial \sigma }{\partial P_n}{\xi }_n\cancel{\sum _m\left[w_{nm}a\left(Q_m\right)-w_{mn}a\left(Q_n\right)\right]}+\sum _{n,m}\frac{\partial \sigma }{\partial P_n}{w}_{nm}a\left(Q_m\right)\left({\xi }_m-{\xi }_n\right)$$

(16)

In order to use the remainder theorem one has to identify

$$\frac{\partial \sigma }{\partial P_n}={\kappa }^{\prime }\left({\xi }_n\right)={\kappa }^{\prime }\left(\frac{a\left(P_n\right)}{a\left(Q_n\right)}\right).$$

(17)

This ensures $\dot{\rho }<0$ for any ${\kappa }^{″}>0$ and $P\ne Q$.

We examine the example of the q–Kullback–Leibler or Rényi divergence. Starting with the classical logarithmic kernel, $\kappa \left(\xi \right)=-ln\xi$, we have ${\kappa }^{″}\left(\xi \right)=1/{\xi }^2>0$. Now having a nonlinear stochastic dynamics, $a\left(P\right)=P^q$, the integrated entropic divergence formula (without symmetrization) delivers the Tsallis divergence [6,7,8],

$$\frac{\partial \sigma }{\partial P_n}=-\frac{Q_n^q}{P_n^q},\Rightarrow \rho [P,Q]=\sum _n{Q}_n{ln}_q\frac{Q_n}{P_n}.$$

(18)

with

$${ln}_q\left(x\right)=\frac{1-x^{q-1}}{1-q}$$

(19)

being the so called deformed logarithm with the real parameter q.

We again would like to interpret this entropic divergence as entropy difference. The entropic divergence of the stationary distribution from the uniform distribution $U_n=1/W,n=1,2,\dots ,W$ is given by:

$$\rho [U,Q]=\sum _{n=1}^W\frac{Q_n}{1-q}\left[1-{\left(W{Q}_n\right)}^{q-1}\right]=W^{q-1}\left(S_T\left[U\right]-S_T\left[Q\right]\right).$$

(20)

with $S_T$ being the Tsallis entropy formula:

$$S_T\left[Q\right]=\frac{1}{1-q}\sum _n(Q_n^q-Q_n)=-\sum _n{Q}_n{ln}_q\left(Q_n\right).$$

(21)

From the Jensen inequality, it follows $\rho [U,Q]\ge 0$ , so $S_T\left[U\right]\ge S_T\left[Q\right]$, i.e., the Tsallis entropy formula is also maximal for the uniform distribution. The factor $W^{q-1}$ signifies non-extensivity, a dependence on the number of states in the relation between the entropic divergence and the relative Tsallis entropy.

4. Master Equation for Unidirectional Growth and Reset

With the particular choice of the transition rates, $w_{nm}={\mu }_m{\delta }_{n-1,m}+{\gamma }_m{\delta }_{n,0}$, one describes a local growth process augmented with direct resetting transitions from any state to the ground state labeled by the index zero [9]. The corresponding master equation

$$\dot{P_n}={\mu }_{n-1}{P}_{n-1}-\left({\mu }_n+{\gamma }_n\right)P_n$$

(22)

is terminated at $n=1$ and the equation for the $n=0$ state takes care of the normalization conservation:

$${\dot{P}}_0=\sum _{n=1}^{\infty }{\gamma }_n{P}_n-{\mu }_0{P}_0.$$

(23)

For the stationary distribution one obtains

$$Q_n=\frac{{\mu }_{n-1}}{{\mu }_n+{\gamma }_n}{Q}_{n-1}=\cdots =\frac{{\mu }_0{Q}_0}{{\mu }_n}\prod _{j=1}^n{\left(1+\frac{{\gamma }_j}{{\mu }_j}\right)}^{-1},$$

(24)

and $Q_0$ has to be obtained from the normalization.

Table 1. Summary of rates and stationary PDFs .

${\mathit{\gamma }}_{\mathit{n}}$, $\mathit{\gamma }\left(\mathit{x}\right)$	${\mathit{\mu }}_{\mathit{n}}$, $\mathit{\mu }\left(\mathit{x}\right)$	${\mathit{Q}}_{\mathit{n}}$, $\mathit{Q}\left(\mathit{x}\right)$
const	const	geometrical → exponential
const	linear	Waring → Tsallis/Pareto
const	sublinear power	Weibull
const	quadratic polynomial	Pearson
const	exp	Gompertz
$ln(x/a)$	$\alpha x$	Log-Normal
linear	const	Gauss
$\alpha (ax-c)$	$\alpha x$	Gamma

summarizes some well known probability density functions, PDFs , which emerge as stationary distribution to this simplified stochastic dynamics upon different choices of the growth and reset rates ${\mu }_n$ and ${\gamma }_n$. In the continuous limit we obtain

$$\frac{\partial }{\partial t}P(x,t)=-\frac{\partial }{\partial x}\left(\mu \left(x\right)P(x,t)\right)-\gamma \left(x\right)P(x,t).$$

(25)

with the stationary distribution

$$Q\left(x\right)=\frac{K}{\mu \left(x\right)}{\mathrm{e}}^{-\underset{0}{\overset{x}{\int }}\frac{\gamma \left(u\right)}{\mu \left(u\right)}du}.$$

(26)

Finally we derive a bound for the entropy production in the continuous model of unidirectional growth with resetting.

First we study the time evolution of the ratio, $\xi (t,x)=P(x,t)/Q\left(x\right)$. Using $P=\xi Q$ we get from Equation (25):

$$Q\frac{\partial \xi }{\partial t}=-\xi \cancel{\frac{\partial \left(\mu Q\right)}{\partial x}}-\mu Q\frac{\partial \xi }{\partial x}-\cancel{\gamma Q}\xi .$$

(27)

Using the same equation for stationary $Q\left(x\right)$ and dividing by Q we obtain

$$\frac{\partial \xi }{\partial t}=-\mu \left(x\right)\frac{\partial \xi }{\partial x}.$$

(28)

Now we turn to the evolution of the entropic divergence,

$$\rho \left(t\right)\equiv \underset{0}{\overset{\infty }{\int }}\mathfrak{s}\left(\xi (t,x)\right)Q\left(x\right)dx,$$

(29)

With the symmetrized kernel, $\mathfrak{s}\left(\xi \right)={\sigma }_{\mathrm{div}}\left(\xi \right)+\xi {\sigma }_{\mathrm{div}}(1/\xi )\ge 0$, one gets using ${\displaystyle \frac{\partial \mathfrak{s}}{\partial t}=-{\displaystyle \mu \left(x\right)}\frac{\partial \mathfrak{s}}{\partial x}}$ the following distance evolution, considering the boundary condition $\xi (t,0)=1$ and $\mathfrak{s}\left(1\right)=0$:

$$\frac{\mathrm{d}\rho }{\mathrm{d}t}=-\underset{0}{\overset{\infty }{\int }}\mathfrak{s}\left(\xi (t,x)\right)Q\left(x\right)\gamma \left(x\right)dx$$

(30)

We note that for the Kullback–Leibler divergence the following symmetrized kernel function has to be used: $\sigma \left(\xi \right)=-ln\xi$ leads to $\mathfrak{s}\left(\xi \right)=(\xi -1)ln\xi$ and in this way ensures $\frac{\mathrm{d}\rho }{\mathrm{d}t}\le 0$.

In order to obtain a lower bound for the speed of the approach to stationarity, we use again the Jensen inequality for $\mathfrak{s}\left(\xi \right)$:

$$\int p\left(x\right)\mathfrak{s}\left(\xi \left(x\right)\right)dx\ge \mathfrak{s}\left(\int p(x)\xi (x)dx\right)$$

(31)

with any arbitrary $p\left(x\right)\ge 0$ satisfying $\int p\left(x\right)dx=1$. For pour purpose we choose $p\left(x\right)=\gamma \left(x\right)Q\left(x\right)/\int \gamma Qdx$. This leads to the following result:

$$\frac{\mathrm{d}\rho }{\mathrm{d}t}\le -{〈\gamma 〉}_{\infty }·\mathfrak{s}\left(\frac{{〈\gamma 〉}_t}{{〈\gamma 〉}_{\infty }}\right)=\left[{〈\gamma 〉}_{\infty }-{〈\gamma 〉}_t\right]·ln\frac{{〈\gamma 〉}_t}{{〈\gamma 〉}_{\infty }}.$$

(32)

Note that the controlling quantity is actually the expectation value of the resetting rate, $\int p\left(x\right)\xi \left(x\right)dx=\int \gamma Pdx={〈\gamma 〉}_t$. Since $\mathfrak{s}\left(\xi \right)$ reaches its minimum with the value zero only at the argument 1, the entropic divergence $\rho \left(t\right)$ stops changing only if the stationary distribution is achieved. In all other cases it shrinks.

5. Conclusions

Summarizing, in this paper we have presented a construction strategy for the entropic distance formula, designed to shrink for a given wide class of stochastic dynamics. The very entropy formula was then derived from inspecting this distance between the uniform distribution and the stationary PDF of the corresponding master equation. In this way, for linear master equations the well-known Kullback–Leibler definition arises, while for nonlinear dependence on the occupation probabilities one always arrives at an accordingly modified expression. In particular, for a general power-like dependence the Tsallis q-entropy occurs as the “natural” relative entropy interpretation of the proper entropic divergence. In the continuous version of the growth and reset master equation, a dissipative probability flow supported with an inflow at the boundary, a lower bound was given for the shrinking speed of the symmetrized entropic divergence using the Jensen inequality.

To finish this paper we would like to make some remarks on real world applications of the above discussed mathematical treatment. Among possible applications of the growth and resetting model, we mention the network degree distributions showing exponential behavior for constant rates and a Tsallis–Pareto distribution [10] (in the discrete version a Waring distribution [11,12]) for having a linear preference in the growth rate, ${\mu }_n=\alpha (n+b)$. For high energy particle abundance (hadron multiplicity) distributions the negative binomial PDF is an excellent approximation [13], when both rates $\mu$ and $\gamma$ are linear functions of the state label. For middle and small settlement size distributions a log-normal PDF arise, achievable with linear growth rate, $\mu \left(x\right)$ and a logarithmic reset rate, $\gamma \left(x\right)\sim lnx$. Citations of scientific papers and Facebook shares and likes also follow a scaling Tsallis–Pareto distribution [14,15], characteristic to constant resetting and linear growth rates. While wealth seems to be distributed according to a Pareto-law tail, the middle class incomes rather show a gamma distribution, stemming from linear reset and growth rates. For a review of such applications see our forthcoming work.