Please select whether you prefer to view the MDPI pages with a view tailored for mobile displays or to view the MDPI
pages in the normal scrollable desktop version. This selection will be stored into your cookies and used automatically
in next visits. You can also change the view style at any point from the main header when using the pages with your
Department of Pure and Applied Mathematics, University of Padova, via Trieste 63 – 35121, Padova, Italy
Received: 14 September 2009 / Accepted: 27 October 2009 / Published: 29 October 2009
In this paper we present a simple model to describe a rather general system in a stationary non-equilibrium state, which is an open system traversed by a stationary flux. The probabilistic description is provided by a non-homogeneous Markov chain, which is not assumed on the basis of a model of the microscopic interactions but rather derived from the knowledge of the macroscopic fluxes traversing the system through a maximum entropy rate principle.
In a recent (2006, ) survey on the Maximum Entropy Production principle, reference is made to an early attempt (1967, [2,3]) to extend (a pioneering paper dealing with this extension problem was written by E.T. Jaynes himself, see ) the celebrated Jaynes’ Maximum Entropy principle to thermodynamic systems in a stationary non-equilibrium. Informally stated, assumed that the dynamic is described by a Markov chain, the authors want to find a stochastic transition matrix such that: (i) it has a prescribed probability distribution as a stationary distribution, (ii) the associated chain evolution satisfies given constraints on the macroscopic scale (admissible microscopic evolution) and (iii) the selected transition matrix generates the maximum number of equally probable microscopic evolution paths. Motivated by their derivation, we proceed here to a self-contained independent approach to the same problem with, we think, more far–reaching consequences.
To state the main ideas, let us begin by recalling a restricted version of the ergodic theorem for Markov chains (). Let us suppose that the system has a finite state space and that its statistical description is given by a stationary, time-homogeneous Markov chain.
Theorem 1Let be a stochastic matrix with positive entries and denote with the power of P. Then there exists a unique distribution π which is stationary for P, i.e., where denotes the transposed matrix of P. Moreover,
and, for every probability distribution ν we have, setting ,
It is immediate to realize from the above Theorem that the stochastic matrix P determines its stationary distribution, while the contrary is false. In this paper we put forth a selection criterion to choose among all stochastic matrices admitting a fixed distribution π as stationary. In a sense, we want to select a preferred dynamics for the approach to equilibrium π. To this, recall that if we take the stationary distribution π as the initial distribution, then the Markov chain defines a discrete-time, finite-states stationary stochastic process. Let be the χ-valued random variable which describe the state of the system at time i. For a stationary process, we can define its entropy rate (see ) as
In a sense, the entropy rate is the thermodynamic N-limit of the entropy of the system formed by all realizations of length N of the process. Therefore, we will call it the system entropy rate in the sequel. Moreover, for the stationary process associated to it holds that (see ,)
Furthermore, in agreement with (ii) above, we assume that the knowledge of the macroscopic fluxes acting on the system in a stationary non-equilibrium state will force constraints on the two-dimensional probability distribution
in the form of constraints on the associated stochastic matrix (see Section 2 below). We can now state our selection criterion, which we call the Maximum Entropy Rate principle and amounts to the following constrained extremum problem:
M.E.R.P.Given a positive probability distribution π, find the stochastic matrix which admits π as a stationary distribution and that has maximum entropy rate .
This principle can be seen as an instance of a Maximum Entropy principle for the case of constraints on two-dimensional distributions. Its justification is provided in  (see also ,) using a large deviation theory type estimate on the empirical second order distributions , in the same spirit of Sanov’ Theorem for first order empirical distributions.
Remark. In the case that the only constraints on P are the stationarity and normalization ones, The answer to this problem can be easily found by using the elementary inequality, see 
with equality if and only if (all rows are equal). This result is a simple consequence of the fact that the entropy can be seen as the entropy rate of a stationary process with i.i.d. random variables and that conditioning reduces the entropy rate. It is immediate to verify that admits π as a stationary distribution. By recalling the ergodic theorem, we see also that for every N.
The starting point of the above M.E.R. principle is the probability distribution π, which is, apart from being positive, completely arbitrary. It affords the statistical description of our system at equilibrium. In case we have some macroscopic information on the system, for example its average energy, that we represent as
where is the energy of the system in state i, then this information can be used to select a probability distribution among all these satisfying the above constraint. By applying the Maximum Entropy principle one gets
the Gibbs probability distribution. Here the inverse temperature is uniquely determined by the value e.
2. Model of the System in a Stationary Non-Equilibrium State
We begin with the model of the energy exchange between our system χ and another system . Let us suppose that the system is described by our stationary Markov chain . We suppose also that the whole system is energetically insulated, that is if and , then the microscopic energy conservation law holds
Let us denote with
the skew-symmetric matrix of the difference of energy between states and with
the joint probability at one time step. The average energy transfer between and χ is
and it follows easily that if π is stationary for P, and if our microscopic energy conservation law holds, then
hence the stationary distribution π describes the system χ at macroscopic equilibrium with .
2.1. Coupling of the system with two environments
We now want to model the coupling of the system with two environments and . As before, the statistical description of the system is given by a Markov chain. As a simplifying assumption, we suppose that at every time step along the evolution of the chain, the system χ is in contact (i.e., the microscopic energy conservation holds) with only one of the environments, alternatively and . Therefore, now the system is described by the non-homogeneous Markov chain ()
We will see that it is not restrictive to suppose that the stochastic matrices have positive entries. Also, notice that the chain “observed” at even times is a time-homogeneous chain described by the positive stochastic matrix , hence the ergodic theorem applies. Let us denote with π the unique stationary distribution for . If π is the distribution describing the system χ at , then we have
Therefore, the probability distribution describing the chain switches between π (at even times) and (at odd times). Let us compute as before the joint probability at one and two time-steps starting from an even time . We have
Reasoning as before, we can compute the averaged energy differences
Modifications of the above formulae for the case that we start the observation of the chain at an odd time are straightforward. By a simple inspection of the above equalities we can draw the following conclusions:
Proposition 1If π is a stationary distribution for , then
Moreover, if π is a stationary distribution for and A, then it is a stationary distribution for B also and
Therefore, with a distribution π stationary for and A we model a system at equilibrium with the two environments, and with a probability distribution π stationary for but not forA, we model a system in a stationary non equilibrium state. Moreover, the condition of stationarity of π with respect to , which is sufficient (but not necessary in the general case) for the balance of energy to hold, is the one that will allow to compute the entropy rate. The value of the macroscopic energy flux can be specified by the macroscopic constraint introduced by the first equality in (2.2)
Note that, as before, we made no assumption on the distribution π. We have to choose matrices A and B satisfying the following constraints (here )
The above constraints do not specify in the general case the matrices A and B. Therefore, we need to invoke a selection criterion, which will be our M.E.R. principle. We are led to investigate the existence of an entropy rate for the stochastic process described by the non-homogeneous Markov chain (2.1) with initial distribution π stationary for . Before turning to this, we look at the solution of our problem in case of zero flux, . It is immediate to see that if we limit ourselves to the case , then the chain is ergodic and
is the maximum entropy rate solution.
2.2. Computation of the entropy rate
It is easy to see that the chain described by (2.1) with initial distribution π stationary for but not for A describes a non-stationary stochastic process. Moreover, the chain is not strongly ergodic, but, with the non restricting assumption that A and B have positive entries it is weakly ergodic (see e.g.,  for the notion of strong and weak ergodicity and , and the bibliography therein for the study of convergence of Markov processes using information theoretic tools.) We now show that the entropy rate is well defined and finite for the process at hand. This is not surprising since the inhomogeneity of the chain is very mild and the chain will become homogeneous with a suitable time-reparameterization, However, we will proceed with the chain as it is for simplicity’ sake. Recall that the probability of a typical sequence of length N of the chain with initial distribution π is
To compute we use the well known chain rule (see ):
Therefore, we have
Hence, if the condition of stationary holds, the terms following alternate between the form of
2.3. Application of the M.E.R. principle and solution
In this section we solve the constrained extremum problem for the objective function subject to the constraints (2.4) using the Lagrange multipliers method. The Lagrange function is
and the necessary condition for the extremum are
By simple computations we get the expression for the solution A and B in terms of the unknown multipliers as
In the following we use as unknowns in place of the Lagrange multipliers. By using the normality constraint on A and B respectively we get
By using the stationarity constraint of π with respect to we get the following equations for , ,
Since the above introduced matrix is a stochastic one with positive entries , it is easy to see that the solution of the above equation for is . Hence, and from (2.7)
Before turning to the inflow constraint to determine the multiplier β, we note that the matrix admits a simpler form using the definition
Therefore , since
Now the inflow constraint can be rewritten as
Hence, for any given π and q, setting , the Lagrange multiplier is uniquely determined by the equation
We conclude by noting that, if the relation is seen as a constraint for the unknown probability distribution π, then the maximum entropy assignation for π is the Gibbs distribution
We have found that the maximum entropy rate assignation for the non-stationary stochastic process described by the non-homogeneous Markov chain (2.1) defined by the stochastic matrices A and B and by the probability distribution π which is stationary for is
where is the Gibbs distribution for and π is the Gibbs distribution for . As expected, the solution depends only on the macroscopic information supplied: the equilibrium energy e and the flow q.
2.4. Entropy rate and entropy production
By a direct computation from (1.1), (2.8), (2.10), the entropy rate of the process is the sum of two terms of the type
Hence, from (2.5)
Since the chain spends “half of its time” in a state with average energy e and the remaining half in a state with average energy , the above formula proposes that the entropy rate be the time average of its “instantaneous” entropies.
Which is the relation between the entropy rate of the stochastic process and the thermodynamic entropy of the system in a stationary non-equilibrium state? If q is small, we can consider the Taylor expansion of w.r.t. q and get
Hence, by the well known identification ,
In the above formula, the entropy rate is the sum of two terms, one of which is non negative while the other has the sign of q. It is appealing to interpret the first as the source term an the other as the flux term.
Moreover, from the relation between the average energy e and the related multiplier , we have that, if q is small
Hence, up to order terms,
2.5. Entropy production of the system
Let us consider the insulated system and let us suppose that the two environments and are two thermostats respectively at temperatures and . The net effect of putting the system χ alternatively in contact with and with is the flow in two time steps of the chain of an average energy amount from a reservoir at higher temperature to a reservoir a lower temperature T leaving the system χ unchanged, since the equilibrium distribution π is stationary for . By a standard non-equilibrium thermodynamics formula (see e.g. ) the entropy production in the cycle is
If we now compute the information divergence (also called relative entropy, see ) of the Gibbs distribution with respect to Gibbs distribution π we find
The information divergence it is not a symmetric function of the two probability distributions while the symmetrized information divergence (see ) of p and q is a symmetric and non negative one
The standard interpretation (see again ) of the symmetrized information divergence is a measure of the difficulty of assessing which is the statistical description (p or q) of the system on the basis of the observations of the system state. Since
we have the following
Proposition 2The entropy production of the closed system is equal to the symmetrized information divergence between the probability distributions π,
Remark. In the literature (see e.g the books , or the papers , ) there is a well established notion of entropy production rate for a stationary Markov chain with countable state space. Let be the transition matrix and its unique stationary distribution respectively. Then the entropy production rate, also called the information gain of the stationary chain with respect to its time reversal, is defined as
We immediately see that if the the chain satisfy the detailed balance condition , then the entropy production rate is zero. Let us show that this is the case for our χ system described by the maximum entropy rate matrices A and B defined in (2.10). Indeed, the chain observed at even times is described by the transition matrix with entries , whose stationary distribution is the same π in (2.9), and this stationary Markov chain satisfy trivially the detailed balance condition. Therefore, the entropy production of the overall system is exclusively due to the energy exchange between the two reservoirs and as computed above.
In this section, we generalize the previous results to the case of a open system with k macroscopic observables with possibly different speed of relaxation to equilibrium and k stationary fluxes .
3.1. The case of k isochronous fluxes and Onsager’ reciprocity relations
The considerations in Section 2 and Section 2.1 developed for E apply without changes for every macroscopic observable . We consider therefore our M.E.R. problem with k constraints of the form (2.4). Since the constraints specifying the values of the k inflows are linear and independent, the last term in the r.h.s. of the Lagrange function (2.6) becomes a sum over the index α
By a simple inspection of the previous computations, one finds that the matrix has the form
where the k Lagrange multipliers are uniquely determined by the k equations
The considerations and the results of Section 2.4. can be reformulated without changes for the vector of fluxes and equilibrium values . In particular, the linear approximation of small fluxes formula (2.12) becomes
and, by a computation like the one done before in Section 2.4., we get
which are Onsager reciprocity relations.
3.2. The case of different speed of relaxation to equilibrium
In this section we sketch how to model a system where there are two observables with different speed of relaxation to equilibrium. As before, let the system χ be alternatively in contact with the two environments and but now we suppose that for the observable X its average value switches from e to and back to e upon the contact with and , while for the observable Y its average value switches from u to and back to u upon the contact with , , , (two cycles). Therefore, the system returns (relaxes) to equilibrium with respect to observable X in a two-step cycle or while, with respect to observable Y relaxes to equilibrium in a full (four step) cycle . Can we infer average values of slow quantity Y at intermediate states B, C from knowledge of average values of fast observable X ?
To this we introduce a periodic Markov chain with four stochastic matrices A, B, C, D generalizing (2.1) and the following set of constraints generalizing (2.4)
By a computation entirely analogous to the previous one, we find the expression of the entropy rate
Moreover, if the equilibrium state is defined by the macroscopic values and , one can show that then system switches between three Gibbsian states
Then, the inferred average value of Y at the intermediate states is
In this paper we have dealt with a simple model describing a system in a stationary non-equilibrium state. The probabilistic description is provided by a non-homogeneous Markov chain, which is not assumed on the basis of a model of the microscopic interactions but rather derived from the knowledge of the macroscopic fluxes traversing the system through a maximum entropy rate principle. With respect to existing applications of the M.E.R. principle, here we have introduced a Markov chain defined by two or more stochastic matrices. In this way we are able to describe a system in a stationary non-equilibrium state that may exhibit different speed of relaxation to equilibrium. We made reference to physical quantities such as the energy for the ease of interpretation of the results, but this is not necessary and the same model can be applied in other domains, such as economy or biology, as it is sufficiently general to take into account macroscopic constraints of different nature.
The author gratefully thanks the referees for useful remarks and comments that have substantially improved the content and presentation of the manuscript.
Martyushev, L.M.; Selenezev, V.D. Maximum entropy production principle in physics, chemistry and biology. Phys. Rep.2006, 426, 1–45. [Google Scholar] [CrossRef]
Filyukov, A.A.; Karpov, V.Y. Description of steady transport processes by the method of the most probable path of evolution. Inzhenerno-Fizicheskii Zhurnal1967, 13, 624–630. [Google Scholar] [CrossRef]
Filyukov, A.A.; Karpov, V.Y. Method of the most probable path of evolution in the theory of stationa ry irreversible processes. Inzhenerno-Fizicheskii Zhurnal1967, 13, 798–804. [Google Scholar]
Jaynes, E.T. The minimum entropy production principle. Ann. Rev. Phys. Chem.1980, 31, 579–601. [Google Scholar] [CrossRef]
Koralov, L.B.; Sinai, Y.G. Theory of Probability and Random Processes, 2nd ed.; Springer-Verlag: Berlin, Germany, 2007. [Google Scholar]
Cover, T.M.; Thomas, J.A. Elements of Information Theory; Wiley & Sons, Inc.: New York, NY, USA, 1991. [Google Scholar]