The Entropy Production Distribution in Non-Markovian Thermal Baths

In this work we study the distribution function for the total entropy production of a Brownian particle embedded in a non-Markovian thermal bath. The problem is studied in the overdamped approximation of the generalized Langevin equation, which accounts for a friction memory kernel characteristic of a Gaussian colored noise. The problem is studied in two physical situations: (i) when the particle in the harmonic trap is subjected to an arbitrary time-dependent driving force; and (ii) when the minimum of the harmonic trap is arbitrarily dragged out of equilibrium by an external force. By assuming a natural non Markovian canonical distribution for the initial conditions, the distribution function for the total entropy production becomes a non Gaussian one. Its characterization is then given through the first three cumulants.


Introduction
The study of fluctuation theorems (FTs) continues to be a topic of great interest in statistical mechanics of small systems out of equilibrium.Those theorems have been studied from the experimental and theoretical point of views, not only in nanosized physical, chemical, and biological systems [1][2][3][4][5][6], but also have been extended to quantum mechanical systems [7,8].The study of FTs for quantities such as work, heat, and entropy production may produce essential results in the description and understanding of nonequilibrium phenomena.Basically the FT relates the probability densities P (A), P (−A) in processes which drive the system out of equilibrium, such that the ratio P (A)/P (−A) = e A , where A represents the work, power, heat, entropy production or some other physical quantity.A few years ago, the stochastic thermodynamics approach allowed to prove the total entropy production fluctuation theorem, for an ordinary Brownian harmonic oscillator assuming an initial canonical distribution [9].Its validity was proven for two exactly solvable models : (i) the Brownian particle is in a harmonic trap and it is subjected to an external time-dependent force and; (ii) the minimum of the trap potential is arbitrarily dragged out of equilibrium with a time-dependent protocol.Also, the total entropy production fluctuation theorem was extended to the case of a charged Brownian harmonic oscillator in crossed electric and magnetic fields [10].In this case, the theorem was studied also in similar physical situations over a finite time interval, namely (i) the charged particle in a two-dimensional harmonic trap is under to the action of an arbitrary time-dependent electric field and; (ii) the minimum of the harmonic trap is arbitrarily dragged by the electric field.We must mention that the theorem for both the ordinary and charged harmonic oscillator was proven only when the initial condition is canonically distributed at equilibrium with the Markovian thermal bath.Both theoretical approaches [9,10] were formulated in the context of the standard Langevin equation (SLE), where the friction force is proportional to the speed with a constant friction coefficient and an additive Gaussian white noise (thermal noise).Those forces, the dissipative and the fluctuating one are originated from the same environment (commonly named to as a thermal bath) at a constant temperature.When a balance between both forces is established, the fluctuation-dissipation relation is valid and consequently the system reaches an equilibrium state.Otherwise, when the fluctuating and dissipative forces are originated from different environments, the system can be driven to a nonequilibrium steady-state.This kind of fluctuating force is known as an external noise and it leads to break down of the fluctuation-dissipation relation.Also the study on Brownian motion and other possible consequences have been widely given in the context of a generalized Langevin equation (GLE), which takes into account the non Markovian effects of the heat bath through a friction force with a memory kernel.Even in this case, the fluctuation-dissipation relation holds and it renders the system to reach an equilibrium state.In contrast as aforementioned, when the system feels the influence of an external Markovian or non Markovian additive noise, such a relation breaks down leading to different dynamical effects [11][12][13][14][15][16][17][18].In nature there are many examples of dynamics that are mostly described in terms of non Markovian processes.For instance, non Markovian processes have been applied in studies of colloidal motion in viscoelastic media [19][20][21], the dynamics of filaments of proteins [22][23][24], and anomalous diffusion in disordered media [25,26].
A unifying characteristic of the problems we have just mentioned is the presence of fluctuations.It is well known that for small systems and short time intervals such fluctuations become extremely important.Then, the approach given by the stochastic thermodynamics represents an alternative way to study situations out of equilibrium with a systematic procedure [4,27,28].In this context, the application of the usual thermodynamic methods to small systems constitutes a generalization of concepts to particle trajectories.It means that the work, heat, internal energy changes as well as changes of entropy among other concepts are extrapolated to consider their calculation in a realization of the stochastic process, followed by the corresponding ensemble average.The FTs have also been studied with such an approach, in particular the entropy change as was mentioned above.Obviously, the introduction of the entropy change in a process has involved all other related concepts such as heat, work, the first-law-like energy balance, etc., all of them calculated for a stochastic trajectory.In fact, stochastic thermodynamics has been developed for mesoscopic systems like colloidal particles or single (bio) molecules driven out of equilibrium by time-dependent forces but still in contact with a heat bath of well-defined temperature.The main results reported in [4,[28][29][30], are the following: the total entropy production (TEP), denoted as ∆s tot , along a single stochastic trajectory, which involves both the particle entropy and entropy production in the surrounding medium, satisfies the integral fluctuation theorem (IFT).It is expressed as e −∆stot = 1, for canonical initial conditions when the particle is arbitrarily driven by time-dependent external forces over a finite time interval (the transient case).It is also shown that in the nonequilibrium steady state over a finite time interval, a stronger fluctuation theorem called the detailed fluctuation theorem (DFT) holds, that is, P (∆s tot )/P (−∆s tot ) = e ∆stot ; where P (∆s tot ) is the probability of entropy change in the direct process whereas P (−∆s tot ) measures the situation in the reverse process.It should be noticed that both cases are valid when the thermal bath is modeled by a Gaussian noise.
It must be mentioned that a considerable number of works related to the FTs has been studied in the context of SLE compared with those given using the GLE.Concerning the studies of FTs, it can be said that several non Markovian treatments base their studies in three main aspects: (a) the preparation of the system when the time dependent protocol starts the time evolution (canonical initial conditions); (b) The construction of the transition probability in both, the forward and the reverse paths; (c) The existence of a stationary distribution function [2,[31][32][33][34][35][36][37].Our main contribution in this work is to show that when the harmonic oscillator is embedded in a non Markovian thermal bath, the change in the total entropy production presents a non-Gaussian distribution function.This characteristic comes from the preparation of the system in the initial condition, which in our case is assumed to satisfy a natural non Markovian canonical distribution consistently with the non Markovian heat bath [38,39].This fact for the initial condition makes the entropy production be also a non Gaussian stochastic process, for which the higher order cumulants are required to specify it.In this work we show that the cumulants of order two and three contain the natural effects of the non Markovian thermal bath through the noise correlation time even at first order of approximation.Consequently, the non-Gaussian characteristic of the total entropy change drives to a breakdown of the usual FTs.
In Section 2 we present the model based on the GLE in the overdamped approximation and its stochastic treatment.The Section 3 is devoted to the calculation of the change in the total entropy for a time-dependent driving force and when the potential center is dragged by an arbitrary time dependent function.Lastly in Section 4 we give some remarks.

GLE and Stochastic Thermodynamics
Let us consider a Brownian particle of mass m trapped in a harmonic potential U (x) = (k/2)x 2 , with k a restitutive constant, embedded in a thermal bath of temperature T .Additionally the particle is subjected to an arbitrary time-dependent driving force f (t).The generalized Langevin equation for the harmonic oscillator embedded in a non Markovian thermal bath can be written as where γ(t − s) is the friction memory kernel, ζ(t) is a Gaussian fluctuating force with zero mean value ζ(t) = 0 which satisfies the fluctuation-dissipation relation with a correlation function given by where k B is the Boltzmann constant.In non Markovian dynamics, the friction memory kernel is usually modeled by γ(t − s) = (γ 0 /τ )e −|t−s|/τ , then the noise term ζ(t) corresponds to the Ornstein-Uhlenbeck (OU) process.In such a case, the noise correlation is explicitly written as , where τ is the correlation time (memory time of the non Markovian dynamics) and γ 0 a constant which is identified with the friction coefficient in the Markovian case.We now proceed to study the generalized Langevin Equation ( 1) by means of an extension of the space of variables (x, v) → (x, v, η), where the new variable obeys the folowing stochastic equation [40] where the stochastic term is described by an Ornstein-Uhlenbeck process and D = γ 0 k B T and ξ(t) is a Gaussian white noise with zero mean value and a correlation function ξ(t)ξ(s) = 2 δ(t − s).In this case the set of Equation ( 1) becomes In the overdamped approximation Equation ( 6) greatly simplifies giving the simple relation η = kx − f (t) and therefore the direct substitution in Equation ( 7) reduces it to a simple effective Langevin equation where It should be noticed that the a parameter as well as the noise intensity D contain the memory correlation time, meaning that the dynamics described by Equation ( 8) is memory dependent.For further considerations, we will require of the fluctuating variable X = x − x , where X and x satisfy the following equations The solution of Equation ( 9) then reads where X(0) = X 0 is the initial condition for position fluctuations, and the solution of Equation (10), by assuming x(0) = x 0 = 0, is We will also need the probability density (PD) P (x, t) given the initial distribution P (x 0 ).To calculate such a PD, the transition probability density (TPD) P (x, t|x 0 ) coming from the Fokker-Planck (FP) equation is required.Thus, the FP equation associated with the Langevin Equation ( 8) reads where λ = D/γ 2 0 (1 + a 0 τ ) 2 .With the initial condition P (x, 0|x 0 ) = δ(x − x 0 ), the solution of Equation ( 13) is well known and given by [41] where k e = k(1 + a 0 τ ) and b(t) = t 0 e as b(s) ds (here, the Boltzmann constant k B has been absorbed in the temperature).The probability density P (x, t) can be calculated for an arbitrary initial condition through the TPD ( 14) and the integral In particular, if we assume that P (x 0 , 0) = P (x 0 ) is canonically distributed at equilibrium with the thermal bath at temperature T in the form then upon substitution of this equation into Equation (15), it can be shown after some algebra that being x the same as that given by Equation (12).It is clear that the PD associated respectively with X and X 0 can be written immediately from Equations ( 16) and ( 17) taking into account that X = x − x , x 0 = X 0 .It should be emphasized that the initial conditions in Equation ( 16) are consistent with the equilibrium between the system and the non Markovian bath.A question which may be interpreted considering two alternatives.First, the preparation of the system occurs in the time interval −∞ < t < 0 in such a way that at t = 0, when the protocol begins its action, the system is in equilibrium with the non Markovian bath.Second, the system and the non-Markovian bath equilibrate between themselves in a negligible time interval.

Stochastic Thermodynamics
According to the stochastic thermodynamics approach [4,42,43], the first law like energy balance states that the work W , the change in internal energy ∆U , and the dissipated heat Q to the bath, can be calculated along a stochastic trajectory x(t) over a finite time interval t.This first-law-like reads as where the work can be calculated from the relation [3,9] and the change in the internal energy is given as ∆U = U (x, t) − U (x 0 ).The change in the environment entropy produced by the heat transfer in the time interval is ∆s M = Q/T .On the other hand, the nonequilibrium Gibbs entropy S of the system in the present problem can be defined as which suggests the extrapolation to a trajectory-dependent entropy for the particle as where the probability density P (x, t) is obtained through the solution of the Fokker-Planck (FP) equation and, it is evaluated along the stochastic trajectory.For a given trajectory x(t), the entropy s(t) depends on the given initial data P (x 0 ) and thus contains information about the whole ensemble.The change in the system's entropy for any trajectory of duration t is then Now, the change in TEP along a trajectory over a finite time interval t is shown to be [4,42] ∆s tot = ∆s M + ∆s Using this definition, Seifert derived the IFT, e −∆stot = 1, where the angular brackets denote average over the statistical ensemble of realizations, or over the ensemble of finite time trajectories.Also he showed that in the nonequilibrium steady state over a finite time interval, the DFT holds.The latter is stated as This theorem has also been proved, even in the transient case, for the ordinary Brownian harmonic oscillator in a Makovian thermal bath (standard Langevin equation with constant friction coefficient) only if the system is initially prepared in equilibrium with the thermal bath [9].In all cases the fluctuating force was modeled with the thermal white noise.In what follows we will show that the DFT (24) does not hold for non Gaussian entropy production in two specific situations.

The Time-Dependent Driving Force
Let us consider the Brownian particle described with the model in Section 2 and study the behavior of the total entropy change.As a first example we take the physical situation in which the effective potential is U (x, t) = (k/2)x 2 − xf (t), where f (t) is an arbitrary function of time.In this case the work is written as and it is seen that its distribution function will be a Gaussian one.On the other hand, the internal energy change reads where f (0) = 0 is assumed for simplicity.Then, the change in total entropy ∆s tot can be calculated explicitly from Equations ( 16), ( 17), ( 18), ( 23) and ( 26), that is After substitution of the k e parameter and grouping terms we can write The result obtained in Equation ( 28) deserves some comments: (a) The change in the total entropy at zeroth order in the noise correlation time τ is a linear combination of Gaussian variables, making the process ∆s tot a Gaussian one; (b) When the noise is modeled as an Ornstein-Uhlenbeck process the time τ = 0, appears a quadratic term in the change of the total entropy, a fact which makes the corresponding process a non-Gaussian one.The lack of Gaussian behavior comes from the initial conditions in which the system were prepared.However, it should be noticed that the preparation was consistent with the non Markovian character of the thermal bath.As a consequence the average and variance of ∆s tot are not enough to calculate its distribution function [41].
To begin the calculation we define the quantity hence the entropy production in the non Markovian case reads Here we just calculate the first three cumulants defined by [41] K 1 = ∆s tot (31) where K 1 is the average, K 2 the variance and K 3 the third order cumulant known as the skewness of the distribution function.The calculation of cumulants proceed in a direct way when we take into account that the stochastic process describing the random variables x or X is a Gaussian one, and its properties are well known.From Equation ( 17) it is clear that X 2 = X 2 0 = T /k e and thus E = 0. From this result we can conclude that, the change in the total entropy production mean value or simply the entropy production mean value in this case is then As can be corroborated, this average value has a very similar structure than that calculated in the Markovian case by Saha et al. [9] (see Equations ( 17) and (21) in that reference).Now, the variance in the change of total entropy is given by the cumulant of second order, where σ 2 W is the work variance and σ W X = (W − W )X .All these quantities can be calculated in a direct way taking into account the properties of the Gaussian characteristics of the x process.The results are as follows and we recall that X 2 = T /k e .As a consequence we obtain Equation ( 39) gives an exact result, where it should be noticed that the production entropy variance contains non Markovian corrections depending on the noise correlation time τ , which vanish when the white noise limit (τ → 0) is taken.
The cumulant K 3 can be further calculated to first order in the noise time correlation τ and it is shown to be given as where we need the averages (W − W ) 2 E , (W − W )XE and X 2 E .They can be readily calculated from the Gaussian properties of X and the work, then As a result, the cumulant K 3 up to first order in the noise time correlation reads as where it obviously contains the influence of the non Markovian noise.Coming back to the density distribution for the change in total entropy, we now write it in terms of the cumulant expansion as usually done [41] P (∆s tot ) = 1 2π which can be written as follows where A(∆s tot ) = 2 i(∆s tot − K 1 )/K 2 .If we define the integral as then the ratio P (∆s tot )/P (−∆s tot ) leads to It is clear that the white noise limit (τ → 0) drives to the usual FT, written as due to the fact that in such a case K 2 = 2K 1 holds.In such a case the initial conditions are distributed according to a canonical distribution at a fixed temperature T .Then, one of the the most important consequences of the non Markovian characteristics of the thermal bath is that the entropy distribution function becomes non Gaussian even to first order in the noise time correlation.Besides, the fluctuation relation as given in Equation ( 49) shows an explicit dependence with the non Markovian properties in the thermal bath.

Dragging of the Harmonic Trap
Let us now consider the physical situation for which the minimum of the harmonic trap is arbitrarily dragged by the time-dependent force.In this case the effective potential now reads U (x, t) = k 2 (x− f (t) k ) 2 , and the work will be given by The change in the internal energy during a time t is now whereas the change in the total entropy production is similar as was calculated in the preceding section, giving as a result After the substituting of the k e expression, we obtain Again, the change in the total entropy contains a quadratic term in the position fluctuations due to the fact that E = X 2 − X 2 0 .Its presence makes the fluctuations in the total entropy change be non Gaussian, as it happened in the previous case.Then, the higher order cumulant must be calculated to have an insight about the behavior in the distribution function.Following the steps done in the previous section we will calculate the cumulants for the change in total entropy, the first one gives us the average which also has a similar structure than that obtained by Saha et al. [9].The second cumulant drives us to the entropy variance It can be shown by a direct substitution of the entropy change and its average in Equation ( 56) that the entropy variance σ 2 s has the same formal expression as given in Equation (35).However the work variance is now given by σ 2 W = 2T W , all other averages have the same structure as in Equations ( 37) and (38).As a result we obtain and at first order in the noise correlation time τ , it reads as Notice that the entropy variance for this case is the same as in the previous example.Also, the third cumulant can be calculated and it is immediate that the result is the same as given in Equation (45).Both examples lead to a non Gaussian distribution function for the change in total entropy as a consequence of the non Markovian character in the thermal bath.Then the FTs do not hold even at the first order in the noise time correlation, it is just in the white noise limit that they are recovered.It should be mentioned that there exists some other alternative theories to study the behavior of phenomena in mesoscopic systems, where additional degrees of freedom can be taken into account.In fact, the so called mesoscopic thermodynamics [44,45] considers such kind of properties by means of phenomenological treatment.Then the change in entropy has a similar expression as the systems entropy we have calculated here.A closer comparison should be done in the future.

Concluding Remarks
The results in this work can be summarized in a short paragraph with the following words.In the examples worked in this paper, we have found a non Gaussian distribution function for the change in the total entropy production.This fact comes from the preparation of the system at the initial time, which was assumed to be as a canonical equilibrium with the non Markovian thermal bath.However, it is important that this canonical initial conditions take explicit account of the time correlation in the bath.It means that the protocol applied to the system starts its effect after the particle has equilibrated with the bath.It should be mentioned explicitly that the models are described by the generalized Langevin equation with a memory kernel taking account of the non Markovian character of the thermal bath.The GLE is a stochastic differential equation linear in the Gaussian variables, then the process has a Gaussian distribution function.On the other hand, the stochastic thermodynamics approach has allowed the calculation of several thermodynamic quantities for a stochastic particle trajectory, in particular the change in the total entropy.This quantity considers the entropy change in the environment as well as the change in the entropy system itself, and when we consider a Gaussian white thermal bath and a canonical initial condition in equilibrium with the bath both FTs are satisfied.However this implies that the thermal bath is a Markovian one.In contrast and consistently with the treatment via the GLE, the Gaussian fluctuations in the the position drive to a non Markovian and most important non Gaussian process in the entropy production.As an immediate consequence, it is necessary to go forward in the characterization of such non Gaussian process by the calculation of higher cumulants.Then, with the non Gaussian distribution functions, the usual fluctuation relations are shown to be not valid.Instead, they are modified with non-Gaussian contributions which do not vanish even to first order in the noise time correlation.It is just in the white noise limit (τ → 0) that both theorems are recovered in their usual structure.