Next Article in Journal
Alloying and Processing Effects on the Aqueous Corrosion Behavior of High-Entropy Alloys
Next Article in Special Issue
Measures of Causality in Complex Datasets with Application to Financial Data
Previous Article in Journal
Microstructures and Crackling Noise of AlxNbTiMoV High Entropy Alloys
Previous Article in Special Issue
Statistical Mechanics Ideas and Techniques Applied to Selected Problems in Ecology
Open AccessArticle

From Random Motion of Hamiltonian Systems to Boltzmann’s H Theorem and Second Law of Thermodynamics: a Pathway by Path Probability

1
Laboratoire de Physique Statistique et Systems Complexes, ISMANS, 44 Ave. F.A., Bartholdi, 72000 Le Mans, France
2
IMMM, Université du Maine, Ave. O. Messiaen, Le Mans 72085, France
*
Author to whom correspondence should be addressed.
Entropy 2014, 16(2), 885-894; https://doi.org/10.3390/e16020885
Received: 25 November 2013 / Revised: 18 December 2013 / Accepted: 23 January 2014 / Published: 13 February 2014
(This article belongs to the Collection Advances in Applied Statistical Mechanics)

Abstract

A numerical experiment of ideal stochastic motion of a particle subject to conservative forces and Gaussian noise reveals that the path probability depends exponentially on action. This distribution implies a fundamental principle generalizing the least action principle of the Hamiltonian/Lagrangian mechanics and yields an extended formalism of mechanics for random dynamics. Within this theory, Liouville’s theorem of conservation of phase density distribution must be modified to allow time evolution of phase density and consequently the Boltzmann H theorem. We argue that the gap between the regular Newtonian dynamics and the random dynamics was not considered in the criticisms of the H theorem.
Keywords: statistical mechanics; random motion; path probability; least action statistical mechanics; random motion; path probability; least action

1. Introduction

It is well known that for regular motion obeying Newtonian mechanics, the path between two given points in configuration space as well as in phase space when the time period is specified is unique [1]. However, it is not the case for random dynamics with a well-known example: Brownian motion. One of the remarkable characteristics of this motion is the non uniqueness of paths between two given points for given time period, which is illustrated in Figure 1 (left).

This phenomenon naturally raises the following questions. What is the probability for a particle, moving between two points, to take a given path or to follow a thin bundle of paths among many others? What are the variables of this probability? How to quantify the dynamical uncertainty in the path probability? Although the path probability of random motion has been noticed since longtime ago [2,3], explicit study of the expression of the path probability and its random variables is relatively new. There are different viewpoints on the question [47] which do not always agree with each other. In a recent work [8], the path probability of an ideal random motion was studied by numerical simulation with particles subject to conservative force and Gaussian noise. This model is ideal since the random motion statistically conserves its mechanical energy in such a way that the Hamiltonian can make sense, such as in a thermodynamically isolated system with internal fluctuation. In practice, any mechanical system with weak dissipation (or friction) can be approximately described by this model within the period in which the energy dissipated is much smaller than the energy of the motion. Figure 1 gives an example of the results. The exponential dependence of the path probability on the action is obvious from this result. The action is denoted by Ak (a,b) for a given path k between two fixed points a and b with the usual definition A k ( a , b ) = a b ( K V ) k d t where K is the kinetic energy and V the potential energy along the path. In that work, we called it Lagrangian action to distinguish it from the integral of Hamiltonian A H = a b ( K + V ) k d t (along the same path (called Hamiltonian action in that work) used to compare the action and energy correlation of the path probability. The result in the right panel of Figure 1 can be expressed by:

p k ( a , b ) = 1 Z a b e γ A k ( a , b )
where γ is the slope of the straight line in the right panel and Zab is the partition function given by Z a b = k e γ A k ( a , b ) summed over all the possible paths between a and b. In general, this summation can be carried out by path integral [9].

In this paper, we present some theoretical consequences of this numerical result. The main point is that this result implies a modification of Liouville’s theorem of conservation of phase volume due to the modification of Hamiltonian equations, which results in other important consequences relative to the entropy of thermodynamics.

2. Stochastic Least Action Principle

Let’s first see the probabilistic uncertainty, or randomness, in the choice of paths by the motion. This uncertainty is represented by the distribution Equation (1) and can be measured by the Shannon information formula S a b = k p k ( a , b ) ln p k ( a , b ). Let’s call it path entropy. It is easy to calculate:

S a b = ln Z a b + γ A ¯ a b
On the other hand, the distribution of Equation (1) maximizes the path entropy. In other words, the vanishing variation δ ( S a b γ A ¯ a b ) = 0 yield Equation (1). Considering that δ ln Z a b = k p k ( a , b ) δ Ak = δ A ¯ and Equation (2), the exponential path probability Equation (1) implies:
δ A ¯ = 0
where δA is a variation of the Lagrangian action and the average is over all possible paths between a and b. This formula has been called stochastic least action principle (SAP) as proposed in our previous work [1013]. Here we have shown that it is in accordance with the distribution of Equation (1), a result of the numerical experiment of random motion of Hamiltonian (non-dissipative) systems.

3. Hamiltonian Mechanics Revisited

It has been shown that the exponential probability distribution of action satisfied the Fokker-Planck equation for normal diffusion in the same way as the Feymann factor of quantum propagator PkeiAk/ℏ satisfies the Schrödinger equation [9] Here we will show a generalized formalism of Hamiltonian mechanics whose equations will be used in the discussion of the Liouville theorem and Boltzmann H theorem.

3.1. Euler-Lagrange Equations

A meaning of the SAP Equation (3) is that for any particular path k, there is no necessarily δAk = 0. In general we have:

δ A k = a b [ t ( L k x ˙ ) L k x ] δ xdt ( or ) 0
where δx is an arbitrary variation of x which is zero at a and b. For δAk ≥ 0 (or ≤ 0), we get:
t ( L k x ˙ ) L k x ( or ) 0
which can be proved by contradiction as follows. Suppose a b f ( t ) δ xdt 0 and f(t) ≤ c ≤ 0 during a small period of time Δt somewhere between a and b. Since δx is arbitrary, let it be zero outside Δt and a positive constant within Δt. We clearly have a b f ( t ) δ x d t c δ x 0, which contradicts our starting assumption. This proves Equation (5).

The Legendre transformation H k = P k x ˙ L k along a path k implies the momentum given by P k = L k x ˙ which can be put into Equation (5) to have:

P ˙ k ( or ) L k x
for δAk ≥(or≤)0.

However, from the path average of Equation (4) and the SAP δ A ¯ = 0, we straightforwardly write:

δ A ¯ = k p k a b [ t ( L k x ˙ ) L k x ] δ x d t = a b [ t ( L k x ˙ ) ¯ L k ¯ x ] δ x d t = 0
which implies:
t ( L k x ˙ ) ¯ L k x ¯ = 0

This is the Euler-Lagrange equation of the random dynamics. We have equivalently:

P ˙ ¯ = L ¯ x
where P ˙ ¯ = k p k P ˙ kand L = k p k L k.

3.2. Hamiltonian Equations

From Legendre transformation, we can have L k x = H k x and the following Hamiltonian equations:

x ˙ k = H k P k and P ˙ k ( or ) H k x
For a path k along which δAk ≥ (or≤)0.

Naturally, Equation (8) means:

P ˙ ¯ = H ¯ x
with the average Hamiltonian H = k p k H k.

4. Liouville’s Theorem

Liouville’s theorem is often involved in the discussions relative to thermodynamic entropy in statistical mechanics. We give an outline below followed by an analysis of the theorem within the present formulation of (probabilistic) Hamiltonian mechanics.

We look at the time change of phase point density ρ(x, P, t) in a, say, 2-dimensional phase space Γ when the system of interest moves on the path of least action [3] ρ(x, P, t) can also be considered as the density of systems of an ensemble of a large number of systems moving in phase space. The time evolution neither creates nor destroys state points or systems, hence the law of state conservation in the phase space is:

ρ t + ( x ˙ ρ ) x + ( P ˙ ρ ) P = 0
which means:
d ρ d t = ρ t + ρ x x ˙ + ρ P P ˙ = ( x ˙ x + P ˙ P ) ρ

For the least action path satisfying Hamiltonian equations [1], the right hand side of the above equation is zero, leading to the Liouville’s theorem:

d ρ dt = 0
i.e., the state density in phase space is a constant of motion. The phase volume Ω available to the system can be calculated by Ω = Γ 1 ρ dn where dn is the number of phase point in an elementary volume dΓ at some point in phase space. The time evolution of the phase volume Ω accessible to the system is then given by:
d Ω dt = d dt Γ 1 ρ dn = Γ 1 ρ 2 d ρ dt dn = 0
meaning that this phase volume is a constant of motion.

The second law of thermodynamics states that the entropy of an isolated system increases or remains constant in time. But Liouville’s theorem implies that if the motion of the system obeys the fundamental laws of mechanics, the Boltzmann entropy defined by S = lnΩ must be constant in time. On the other hand, the probability distribution of states p(x,P) in phase space is proportional to ρ(x,P), meaning that an entropy S(p), as a functional of p(x,P), must be constant in time, which is in contradiction with the second law.

What is then the Liouville’s theorem in the context of random dynamics? From Equation (13), we have d ρ d t | k = ( x ˙ k x + P ˙ k P k ) ρ along a path k. Let us write the second equation of the Equations (10) as P ˙ = H k x + R k where Rk ≥(or≤) 0 for δAk ≥(or≤) 0 is the random force causing the deviation from Newtonian laws. The average Newtonian law P ˙ ¯ = H ¯ x yields k p k R k = 0 for any moment of the process. We then have:

d ρ d t | k = R k P k ρ
and:
d ρ d t = k p k d ρ d t | k = R k ¯ P k ρ
where R k ¯ P k = k p k R k P k is an average over all the possible paths. The solution of this equation is:
ρ ( t ) = ρ ( t 0 ) exp [ ζ ( t , t 0 ) ]
with the function ζ ( t , t 0 ) = t 0 t R k ¯ P k d t.

Now let us see an application to the case of exponential path probability. The relationship k p k R k = 0 implies R k ¯ P k = k R k p k P k. By using the exponential distribution of action, we have p k P k = γ p k A k P k where A k P k = t 0 t L k P k d τ = t 0 t x ˙ k d τ = x k ( t ) (suppose x(t0) = 0) where xk(t) is the displacement of the motion along the path k from t0 to t. Hence R k ¯ P k = γ k p k t 0 t R k x ˙ k d τ = γ W ( t ) ¯ where W ( t ) ¯ = k p k W k ( t ) is the average of the work W k ( t ) = t 0 t R k x ˙ k d τ performed by the random forces Rk (independent from time) over the displacement xk(t). Finally, ζ ( t , t 0 ) = γ t 0 t W ( τ ) ¯ d t = γ W ( t , t 0 ) here W ( t , t 0 ) = t 0 t W ( τ ) ¯ d t is the cumulate average work of the random force performed from t0 and t.

We have:

ρ ( t ) = ρ ( t 0 ) exp [ γ W ( t , t 0 ) ]
meaning that the state density decreases (increases) and the phase volume increases (decreases) whenever the cumulate work W(t,t0) > 0 (<0). ρ(t) is constant only when there is no work of random forces in average.

If the average work W ( t ) ¯ does not depend on time over a process, we can write W(t,t0)=α(tt0) and:

ρ ( t ) = ρ ( 0 ) exp [ λ t ]
where λ=γα is a constant and a Lyapunov-like exponent characterizing the variation of the distances between the state points.

5. Boltzmann H Theorem

The Boltzmann H function can be defined in discrete coarse graining way in phase space by:

H ( t ) = x , P p ( x , P , t ) ln p ( x , P , t )
where the sum is over all the accessible coarse grained phase domain and p(x, P, t) ∝ ρ(x, P, t)dΩ is the probability that a system is found in a phase cell of volume dΩ situated at the point (x,P) at time t.

We have:

p ( x , P , t ) = p ( x 0 , P 0 ) exp [ γ W ( t , t 0 ) ]

It is straightforward to see that the variation of H function from t0 to t is given by ΔH = H(t) − H(t0) = −γW(t,t0). Since γ is positive, it is necessary to prove that the cumulate work WR(t,t0) is positive in order to yield Boltzmann H theorem ΔH ≤ 0.

Here we only provide a proof for W(t,t0) ≥ 0 in the case of ideal gas in which each molecule can be considered as a particle in random motion. The motion of the ensemble of particles is also random, perturbed by internal fluctuation.

Let us consider the free expansion of ideal gas between two equilibrium states at time t1 and t2 with volume V1 and V2, respectively. Let p(x1, P1,t1) and p(x2, P2,t2) be the two equilibrium distributions. We use the entropy expression:

S ( t ) = x , P p ( x , P , t ) ln p ( x , P , t )

For the two equilibrium states, it is straightforward to calculate (let the Boltzmann constant kB=1):

Δ S = S 2 S 1 = N ln V 2 V 1 = ζ ( t 2 , t 1 ) = γ W ( t 2 , t 1 )

Hence the variation of H function is Δ H = H 2 H 1 = Δ S = N ln V 2 V 1, meaning that W ( t 2 , t 1 ) = N γ ln V 2 V 1 0. This is a proof for the H theorem in the special case of ideal gas expansion between two equilibrium states. For the motion between any two states, the property W(t,t0) ≥0 needs a more general proof from its definition above Equation (19).

6. Concluding Remarks

We have studied the theoretical consequences of the path probability exponentially depending on action which was an observed result in the numerical experiment of random motion of Hamiltonian systems. We have indicated that this probability was an emblem of the stochastic mechanics based on a generalization of the least action principle of the classical Hamiltonian/Lagrangian mechanics. Then we have proved a modification of Liouville’s theorem within this generalized mechanics theory for the random dynamics: the distribution function of states in phase space is no more a constant in time as prescribed by Liouville’s theorem before for deterministic dynamics of Hamiltonian systems. The Boltzmann H function can have time evolution allowing entropy increase. This increase can be proved to be a consequence of the work of random forces on the systems. For the special case of free expansion of ideal gas, this random work could be identified through the increase of entropy. More general proof of the H theorem from the definition of the work of random forces can be expected.

The essential idea of this work is to derive Boltzmann’s H theorem, initially proposed to treat thermodynamic phenomena, within a mechanical theory for random motion and on the basis of a characteristic law of random dynamics: the path probability. Although this path probability has been found, by numerical simulation [8], only for a special and ideal random motion of Hamiltonian systems which statistically conserve their energy, the philosophy and the methodology should be suitable for discussing random dynamics and concomitant phenomena.

One of the aims of this work is to question the criticisms of the H theorem by Loschmidt, Poincaré and Zermelo [1416] on the basis of the time reversible Hamiltonian mechanics and Liouville’s theorem for regular and deterministic mechanical motion. Indeed, using Liouville’s theorem to criticize H theorem, one should take it for granted that the state density is proportional to the probability distribution of states. Yet Liouville’s theorem holds only for regular dynamics in which each trajectory can be a priori traced in time with certainty, each state has its unique moment of time to be visited by the system, and the frequency of visit of any phase (state) volume can be predicted exactly. Therefore, when Liouville’s theorem holds, a priori there is no place for probability and entropy or other uncertainty in regular dynamics. On the other hand, all thermodynamic systems undergo random motion with more or less fluctuation and are intrinsically probabilistic and unpredictable. Without introducing probability and unpredictability into the basic laws of the Newtonian mechanics, any effort to relate the second law of thermodynamics to the Newtonian mechanics [17] must encounter the gap between the world of regular, deterministic, predictable and time reversible motion and the world of random, indeterministic, unpredictable and irreversible motion. As is well known Boltzmann has tried to prove the second law of thermodynamics from the H-theorem with the assumption “molecular chaos” (Stosszahlansatz) in formulating the collision term [18]. Boltzmann believes that from this assumption it is possible to break time-reversal symmetry, necessary for the entropy increase, and to solve the Loschmidt paradox [17], but Liouville’s theorem still persists and forbids the H function to change in time.

This work is an effort to generalize the regular Newtonian mechanics into a stochastic mechanics theory through path probability which is a universal character and an emblem of all random dynamics. Within the formalism with the path probability depending on action, Liouville’s theorem has to be modified to allow time evolution of the phase density as is shown in Equation (18). On the other hand, the time reversibility does not exist for Equation (5) or (10) or P ˙ = H k x + R k since the time symmetry along any path k is broken down by the random force Rk whose integration in opposite direction of time should be a priori different.

It should be stressed again that the present work is just a first generalization of Hamiltonian mechanics with an ideal model of random motion: a Hamiltonian system undergoing random motion without, statistically, loss of energy through dissipation. For the time being, this same approach, based on the least action principle, cannot be applied to dissipative systems for which this principle is no more valid. The extension of the least action principle to dissipative system is still under progress [19].

Conflicts of Interest

The authors declare no conflict of interest

References

  1. Arnold, V.I. Mathematical Methods of Classical Mechanics, 2nd ed; Springer-Verlag: New York, NY, USA, 1989; p. 243. [Google Scholar]
  2. Onsager, L; Machlup, S. Fluctuations and irreversible processes. Phys. Rev 1953, 91, 1505. [Google Scholar]
  3. Freidlin, M.I.; Wentzell, A.D. Random Perturbation of Dynamical Systems; Springer-Verlag: New York, NY, USA, 1984. [Google Scholar]
  4. Evans, R.M.L. Detailed balance has a counterpart in non-equilibrium steady states. J. Phys. Math. Gen 2005, 38, 293. [Google Scholar]
  5. Abaimov, S.G. General formalism of non-equilibrium statistical mechanics a path approach. 2009. arXiv:0906.0190. [Google Scholar]
  6. Cohen, E.G.D; Gallavotti, G. Note on two theorems in nonequilibrium statistical mechanics. 1999. arXiv:cond-mat/9903418 v1. [Google Scholar]
  7. Sevick, E.M.; Prabhakar, R.; Williams, S.R.; Searles, D.J. Fluctuations theorems. Annu. Rev. Phys. Chem 2008, 59, 603–633. [Google Scholar]
  8. Lin, T.L.; Wang, R.; Bi, W.P.; El Kaabouchi, A.; Pujos, C.; Calvayrac, F.; Wang, Q.A. Path probability distribution of stochastic motion of non dissipative systems: a classical analog of Feynman factor of path integral. Chaos Soliton. Fract 2013, 57, 129–136. [Google Scholar]
  9. Feynman, R.P.; Hibbs, A.R. Quantum Mechanics and Path Integrals; McGraw-Hill Publishing Company: New York, NY, USA, 1965. [Google Scholar]
  10. Wang, Q.A. Maximum path information and the principle of least action for chaotic system. Chaos Soliton. Fract 2004, 23, 1253–1258. [Google Scholar]
  11. Wang, Q.A. Non quantum uncertainty relations of stochastic dynamics. Chaos Soliton. Fract 2005, 26, 1045–1052. [Google Scholar]
  12. Wang, Q.A. Maximum entropy change and least action principle for nonequilibrium systems. Astrophys. Space Sci 2006, 305, 273. [Google Scholar]
  13. Wang, Q.A.; Tsobnang, F.; Bangoup, S.; Dzangue, F.; Jeatsa, A.; Le Méhauté, A. Reformulation of a stochastic action principle for irregular dynamics. Chaos Soliton. Fract 2009, 40, 2550–2556. [Google Scholar]
  14. Grandy, W.T., Jr. Entropy and the Time Evolution of Macroscopic Systems; Oxford University Press: Oxford, UK, 2008; pp. 142–152. [Google Scholar]
  15. Poincaré, H. Thermodynamique; Gauthier-Villars: Paris, France, 1908; pp. 449–450. [Google Scholar]
  16. Poincaré, H. On the Three-body Problem and the Equations of Dynamics (Sur le Probleme des trios corps ci les equations de dynamique). Acta mathematica 1890, 13, 270. [Google Scholar]Kinetic Theory of Gases; Brush, Stephen G., Hall, Nancy S., Eds.; Imperial College Press: London, UK, 2003; pp. 368–381.
  17. Stenger, I. Cosmopolitiques; La découverte: Paris, France, 2003; p. 169. [Google Scholar]
  18. Dorfman, J.R. An introduction to chaos in nonequilibrium statistical mechanics; Cambridge University Press: Cambridge, UK, 1999. [Google Scholar]
  19. Lin, T.L.; Wang, Q.A. The extrema of an action principle for dissipative mechanical systems. J. Appl. Mech 2013, 81, 031002. [Google Scholar]
Figure 1. An example of the results of numerical simulation of random motion with 109 particles subject to a harmonic force f= −kx and Gaussian noise, where x is the position of the particle with respect to the initial point a. The left panel shows about 100 paths between two given points. The duration of motion is about three halves of a period of the harmonic oscillation. These paths are created through the Gaussian noise around the most probable path which is just the harmonic curve between the points a and b. The magnitude of the noise is controlled in order that the paths are sufficiently far from each other to give sufficiently different energies and actions but not too far to make the paths sufficiently smooth. The right panel shows the correlation of the probability distribution of paths p(Ak) with action (o), compared to the uncorrelation with Hamiltonian action (*). The most probable paths are the paths of least action which is just the ballistic path of the harmonic oscillation. The straight line in the right panel is just guide for eye.
Figure 1. An example of the results of numerical simulation of random motion with 109 particles subject to a harmonic force f= −kx and Gaussian noise, where x is the position of the particle with respect to the initial point a. The left panel shows about 100 paths between two given points. The duration of motion is about three halves of a period of the harmonic oscillation. These paths are created through the Gaussian noise around the most probable path which is just the harmonic curve between the points a and b. The magnitude of the noise is controlled in order that the paths are sufficiently far from each other to give sufficiently different energies and actions but not too far to make the paths sufficiently smooth. The right panel shows the correlation of the probability distribution of paths p(Ak) with action (o), compared to the uncorrelation with Hamiltonian action (*). The most probable paths are the paths of least action which is just the ballistic path of the harmonic oscillation. The straight line in the right panel is just guide for eye.
Entropy 16 00885f1 1024
Back to TopTop