This article is an open-access article distributed under the terms and conditions of the Creative Commons Attribution license http://creativecommons.org/licenses/by/3.0/.

Evidence from climate science suggests that a principle of maximum thermodynamic entropy production can be used to make predictions about some physical systems. I discuss the general form of this principle and an inherent problem with it, currently unsolved by theoretical approaches: how to determine which system it should be applied to. I suggest a new way to derive the principle from statistical mechanics, and present a tentative solution to the system boundary problem. I discuss the need for experimental validation of the principle, and its impact on the way we see the relationship between thermodynamics and kinetics.

Evidence from climate science suggests that, in at least some cases, the dynamics of planetary atmospheres obey a principle of maximum entropy production [

However, the principle as currently formulated suffers from an important conceptual problem, which I term the system boundary problem: it makes different predictions depending on where one draws the boundaries of the system. In this paper I present a new argument based on Edwin Jaynes’ approach to statistical mechanics, which I believe can give a constructive answer to this problem.

If the maximum entropy production principle (MEPP) turns out to be a general property of physical systems under some yet-to-be-determined set of conditions then it could potentially have a huge impact on the way science can deal with far-from-equilibrium systems. This is because planetary atmospheres are extremely complex systems, containing a great number of processes that interact in many ways, but the MEPP allows predictions to be made about them using little information other than knowledge of the boundary conditions. If this trick can be applied to other complex far-from-equilibrium physical systems then we would have a general way to make predictions about complex systems’ overall behaviour. The possibility of applying this principle to ecosystems is particularly exciting. Currently there is no widely accepted theoretical justification for the MEPP, although some significant work has been carried out by Dewar [

However, maximising a system’s entropy production with respect to some parameter will in general give different results depending on which processes are included in the system. In order for the MEPP to make good predictions about the Earth’s atmosphere one must include the entropy produced by heat transport in the atmosphere but not the entropy produced by the absorption of incoming solar radiation [

Like the work of Dewar [

In

This simple argument does not in itself solve the system boundary problem, because it only applies to isolated systems. The Solar system as a whole can be considered an isolated system, but a planetary atmosphere cannot. However, in

The maximum entropy production principle, if valid, will greatly affect the way we conceptualise the relationship between thermodynamics and kinetics. There is also a pressing need for experimental studies of the MEPP phenomenon. I will comment on these issues in

The main arguments of this paper are in sections

The Maximum Entropy Production Principle (MEPP), at least as used in climate science, was first hypothesised by Paltridge [

The lack of a theoretical justification makes it unclear under which circumstances the principle can be applied in atmospheric science, and to what extent it can be generalised to systems other than planetary atmospheres. The aim of this paper is to provide some steps towards a theoretical basis for the MEPP which may be able to answer these questions.

A greatly simplified version of Paltridge’s model has been applied to the atmospheres of Mars and Titan in addition to Earth by Lorenz, Lunine and Withers [

The model I will describe differs from that given in [

In these models, the atmosphere is divided into two boxes of roughly equal surface area, one representing the equatorial and tropical regions, the other representing the temperate and polar regions (see

A diagram showing the components of the two-box atmospheric heat transport model. Labels written next to arrows indicate rates of flow of energy, whereas labels in boxes represent temperature.

Energy flows into the equatorial region at a rate

Putting this together, we have the energy balance equation for the equatorial region

Similarly, for the polar region we have

We now wish to calculate the rate of entropy production due to the atmospheric heat transport. This is given by the rate at which the heat flow

Solving the steady state energy balance equations for

The exact form of this equation is not important, but it has some important properties.

The value of

The two-box atmospheric model described above has an interesting and important feature: the presence of what I will call

Usually in non-equilibrium thermodynamics we would consider one of two different types of boundary condition: we could impose a constant gradient upon the system (corresponding to holding

I use the term “negative feedback boundary conditions” to emphasise that, in situations analogous to these two-box atmospheric models, the gradient is a decreasing function of the flow, which results in a unique maximum value for the entropy production, which occurs when the flow is somewhere between its minimum and maximum possible values.

The notion of gradient can readily be extended to systems where the flow is of something other than heat. In electronics, the flow is the current

If the effectiveness of the two-box model is the result of a genuine physical principle, one might hope that it could be extended in general to some large class of systems under negative feedback constraints. This might include the large class of ecological situations noted by Odum and Pinkerton [

The MEPP cannot apply universally to all systems under negative feedback boundary conditions. As a counter-example, we can imagine placing a resistor in an electrical circuit with the property that the voltage it applies across the resistor is some specific decreasing function of the current that flows through it. In this case we would not expect to find that the current was such as to maximise the entropy production. Instead it would depend on the resistor’s resistance

For this reason, the MEPP is usually thought of as applying only to systems that are in some sense complex, having many “degrees of freedom” [

In Dewar’s approach, as well as in the approach presented in the present paper, this issue is resolved by considering the MEPP to be a predictive principle, akin to the second law in equilibrium thermodynamics: the entropy production is always maximised, but subject to constraints. In equilibrium thermodynamics the entropy is maximised subject to constraints that are usually formed by conservation laws. In Dewar’s theory, the entropy

An interesting consequence of this point of view is that, in the degenerate case of boundary conditions where the gradient is fixed, the MEPP becomes a maximum flow principle. With these boundary conditions the entropy production is proportional to the rate of flow, and so according to the MEPP, the flow will be maximised, subject to whatever macroscopic constraints act upon it. Likewise, for a constant flow, the MEPP gives a prediction of maximal gradient.

However, there is another serious problem which must be solved before a proper theory of maximum entropy production can be formulated, which is that the principle makes non-trivially different predictions depending on where the system boundary is drawn. In the next two sections I will explain this problem, and argue that Dewar’s approach does not solve it.

A substantial criticism of MEPP models in climate science is that of Essex [

This is a serious problem for the MEPP in general. It seems that we must choose the correct system in order to apply the principle, and if we choose wrongly then we will get an incorrect answer. However, there is at present no theory which can tell us which is the correct system.

We can see this problem in action by extending the two-box model to include entropy production due to the absorption of radiation. The incoming solar radiation has an effective temperature of

As in

Thus the results we obtain from applying the MEPP to the Earth system depend on where we draw the boundary. If we draw the system’s boundary so as to include the atmosphere’s heat transport but not the radiative effects then we obtain a prediction that matches the real data, but if we draw the boundary further out, so that we are considering a system that includes the Earth’s surrounding radiation field as well as the Earth itself, then we obtain an incorrect (and somewhat physically implausible) answer.

This problem is not specific to the Earth system. In general, when applying the MEPP we must choose a specific bounded region of space to call the system, and the choice we make will affect the predictions we obtain by using the principle. If the MEPP is to be applied to more general systems we need some general method of determining where the system boundary must be drawn.

Several authors have recently addressed the theoretical justification of the MEPP [

The approach presented here most closely resembles that of Dewar [

Dewar offers two separate derivations of the MEPP from Jaynes’ MaxEnt principle, one in [

Unfortunately I have never been able to follow a crucial step in Dewar’s argument (equations 11–14 in [

In [

In the following sections I will outline a new approach to the MEPP which I believe may be able to avoid these problems. Although not yet formal, this new approach has an intuitively reasonable interpretation, and may be able to give a constructive answer to the system boundary problem.

Both Dewar’s approach and the approach I wish to present here build upon the work of Edwin Jaynes. One of Jaynes’ major achievements was to take the foundations of statistical mechanics, laid down by Boltzmann and Gibbs, and put them on a secure logical footing based on Bayesian probability and information theory [

On one level this is just a piece of conceptual tidying up: it could be seen as simply representing a different justification of the Boltzmann distribution, thereafter changing the mathematics of statistical mechanics very little, except perhaps to express it in a more elegant form. However, on another level it can be seen as unifying a great number of results in equilibrium and non-equilibrium statistical mechanics, putting them on a common theoretical footing. In this respect Jaynes’ work should, in the present author’s opinion, rank among the great unifying theories of

Jaynes gives full descriptions of his technique in [

Like all approaches to statistical mechanics, Jaynes’ is concerned with a probability distribution over the possible microscopic states of a physical system. However, in Jaynes’ approach we always consider this to be an “epistemic” or Bayesian probability distribution: it does not necessarily represent the amount of time the system spends in each state, but is instead used to represent our state of knowledge about the microstate, given the measurements we are able to take. This knowledge is usually very limited: in the case of a gas in a chamber, our knowledge consists of the macroscopic quantities that we can measure—volume, pressure, etc. These values put constraints on the possible arrangements of the positions and velocities of the molecules in the box, but the number of possible arrangements compatible with these constraints is still enormous.

The probability distribution over the microstates is constructed by choosing the distribution with the greatest entropy

Having done this, we can then use the probability distribution to make predictions about the system (for example, by finding the expected value of some quantity that we did not measure originally). There is nothing that guarantees that these predictions will be confirmed by further measurement. If they are not then it follows that our original measurements did not give us enough knowledge to predict the result of the additional measurement, and there is therefore a further constraint that must be taken into account when maximising the entropy. Only in the case where sufficient constraints have been taken into account to make good predictions of all the quantities we can measure should we expect the probability distribution to correspond to the fraction of time the system spends in each state.

In thermodynamics, the constraints that the entropy being maximised is subject to usually take the form of known expectation values for conserved quantities. For example, the quantities in question might be internal energy and volume. We will denote the energy and volume of the

This method can be extended to deal with a continuum of states rather than a set of discrete ones. This introduces some extra subtleties (see [

There is a subtle point regarding the status of the second law in Jaynes’ interpretation of thermodynamics [

Once Jaynes’ approach to thermodynamics is understood, the argument I wish to present for the MEPP is so simple as to be almost trivial. However, there are some aspects to it which have proven difficult to formalise, and hence it must be seen as a somewhat tentative sketch of what the true proof might look like.

Jaynes’ argument tells us that to make the best possible predictions about a physical system, we must use the probability distribution over the system’s microstates that has the maximum possible entropy while remaining compatible with any macroscopic knowledge we have of the system. Now let us suppose that we want to make predictions about the properties of a physical system at time

The application of Jaynes’ procedure in this case seems conceptually quite straight-forward: we maximise the entropy of the system we want to make predictions about—the system at time

If the system in question is isolated then it cannot export entropy to its surroundings. In this case, if we consider the entropy at time

This appears to show that, for isolated systems at least, maximum entropy production is a simple corollary of the maximum entropy principle. However, it should be noted that the entropy which we maximise at time

The procedure thus obtained consists of maximising the thermodynamic entropy subject to constraints acting on the system not only at the present time but also in the past. Our knowledge of system’s kinetics also acts as a constraint when performing such a calculation. This is important because in general systems take some time to respond to a change in their surroundings. In this approach this is dealt with by including our knowledge of the relaxation time among the constraints we apply when maximising the entropy.

There is therefore a principle that could be called “maximum entropy production” which can be derived from Jaynes’ approach to thermodynamics, but it remains to be shown in which, if any, circumstances it corresponds to maximising the thermodynamic entropy. From this point on I will assume that for large macroscopic systems,

This suggests that applying Jaynes’ procedure to make predictions about the future properties of a system yields a principle of maximum production of thermodynamic entropy: it says that the best predictions about the behaviour of isolated systems can be made by assuming that the approach to equilibrium takes place as rapidly as possible. However it should be noted that this argument does not solve the system boundary problem in the way that we might like, since it applies only to isolated systems. The heat transport in the Earth’s atmosphere is not an isolated system, but the Solar system as a whole is, to a reasonable approximation. Thus this argument seems to suggest unambiguously that the MEPP should be applied to the whole Solar system and not to Earth’s atmosphere alone — but this gives incorrect predictions, as shown in

In this section I will demonstrate the application of this technique in the special case of a system with fixed boundary conditions. I will use a technique similar to the one described by Jaynes in [

Consider a system consisting of two large heat baths

We will consider the evolution of the system over a time period from some arbitrary time

At a given time

We now treat our knowledge of the state of the system (including the reservoirs) at time

This reasoning goes through in exactly the same way if the flow is of some other conserved quantity rather than energy. It can also be carried out for multiple flows, where the constraints can include trade-offs between the different steady-state flow rates. In this case the quantity which must be maximised is the steady state rate of entropy production. This is less general than the MEPP as used in climate science, however, since this argument assumed fixed gradient boundary conditions.

The argument presented in

It could be that this reasoning is simply correct, and that there is just something special about planetary atmospheres which happens to make their rates of heat flow match the predictions of the MEPP. However, in this section I will try to argue that there is an extra constraint that must be taken into account when performing this procedure on systems with negative feedback boundary conditions. This constraint arises from the observation that the system’s behaviour should be invariant with respect to certain details of its environment. I will argue that when this constraint is taken account of, we do indeed end up with something that is equivalent to the MEPP as applied in climate science.

We now wish to apply the argument described in

Suppose that our knowledge of this system consists of the functions

We cannot directly apply the procedure from

Two possible ways in which negative feedback boundary conditions could be realised.

Since all processes in this version of the external environment are reversible, the only entropy produced in the extended system is that due to

Although the first of these scenarios seems quite natural and the second somewhat contrived, it seems odd that the MEPP should make different predictions for each of them. After all, their effect on the system

Clearly an extra principle is needed when applying the MEPP to systems of this type: the rate of heat flow

The question now is how to take account of such a constraint when maximising the entropy. It seems that the solution must be to maximise the information entropy of a probability distribution over all possible states of the extended system, subject to knowledge of the system

But this again presents us with a problem that can be solved by maximising an entropy. We know that outside the system

Let us consider again at the two scenarios depicted in

The situation in the second scenario is different. In this case the temperatures of

I now wish to claim that, if we maximise the entropy of

But we now wish to add one more constraint to

Maximising the entropy of a large environment

This will be true no matter which constraints we initially chose for the volume, internal energy and composition of

This result gives us the procedure we need if we want to maximise the total entropy production of the whole system (

Therefore, completing the procedure described in

In this section I have presented a very tentative argument for a way in which the system boundary problem might be resolved. If this turns out the be the correct way to resolve the system boundary problem then its application to the Earth’s atmosphere is equivalent to a hypothesis that the atmosphere would behave in much the same way if heat were supplied to it reversibly rather than as the result of the absorption of high-temperature electromagnetic radiation. It is this invariance with respect to the way in which the boundary conditions are applied that singles the atmosphere out as the correct system to consider when applying MEPP. However, the argument could also be applied in reverse: the fact that the measured rate of heat flow matches the MEPP prediction in this case could be counted as empirical evidence that the atmosphere’s behaviour is independent of the way in which heat is supplied to it.

It is worth mentioning again that although I have presented this argument in terms of heat flows and temperature, it goes through in much the same way if these are replaced by other types of flow and their corresponding thermodynamic forces. Thus the argument sketched here, if correct, should apply quite generally, allowing the same reasoning to be applied to a very wide range of systems. The method is always to vary some unknown parameters so as to maximise the rate of entropy production of a system. The system should be chosen to be as inclusive as possible, while excluding any processes that cannot affect the parameters’ values.

There are two issues which deserve further discussion. The first is the need for experimental study of the MEPP phenomenon, which will help us to judge the validity of theoretical approaches, and the second is the way in which a universally applicable principle of maximum entropy production would affect the way we see the relationship between thermodynamics and kinetics.

It is likely to be difficult to determine the correctness of theoretical approaches to the MEPP without experimental validation of the idea. Currently the main empirical evidence comes from measurements of the atmospheres of Earth and other bodies, but this gives us only a small number of data points. There have also been some simulation-based studies such as [

However, to my knowledge, there is not currently any experimental work which focuses on applying negative feedback boundary conditions to a system. It seems that it should be possible to subject a potentially complex system such as a large volume of fluid to these conditions using an experimental set-up which, in essence, could look like that shown in

There is of course the problem that, regardless of what theoretical justification one might accept, the MEPP cannot make correct predictions about all systems under negative feedback boundary conditions. In the approach presented here, as well as in Dewar’s approach, this is because the entropy production is maximised subject to constraints. Thus if the rate of heat transport in an experimental set-up failed to match the predictions of the MEPP it could be because of additional constraints acting on the system, in addition to the imposed boundary conditions, rather than because the whole theory is wrong.

However, a positive experimental result—some situation in which the MEPP can be used to make accurate predictions about the behaviour of a system under imposed negative feedback boundary conditions—would be of great help to the development of the theory, as well as justifying its use in practice.

One particularly useful experiment would be to vary the experimental setup in various ways until the MEPP predictions based only on the boundary conditions were no longer accurate. This would help to produce a more precise understanding of what is required for a system to be unconstrained enough for the MEPP to apply.

Currently, thermodynamics and kinetics are thought of as somewhat separate subjects. Thermodynamics, in its usual interpretation, tells us that entropy must increase but says nothing about how fast. The study of the rates of chemical reactions and other physical processes is called kinetics, and is thought of as being constrained but not determined by thermodynamics.

If there is a general principle of maximum entropy production which works along the lines described in the present paper then this relationship can be cast in a rather different light. Now the kinetics act as constraints upon the increase of entropy, which always happens at the fastest possible rate. This change in perspective may have an important practical consequence.

In

But this is not how thermodynamics works in practice. The possible energy levels of a large system are usually unknown, but the relationship between the amount of heat added to a system and its temperature can be measured using calorimetry. From this we work backwards, using Equation

But if the MEPP is a consequence of Jaynes’ maximum entropy principle then we should also be able to work backwards from the kinetics of a system to make inferences about its microscopic dynamics.

The methodology would involve something along the lines of a procedure which determines what kind of constraints the observed kinetics put on the underlying microscopic dynamics, and then maximising the entropy production subject to those constraints in order to produce the best possible predictions about the system’s behaviour under other circumstances.

The development of a principled algorithm for doing this is a subject for future research, but the idea could have lasting implications for the way in which the microscopic dynamics of physical and chemical systems are studied.

In this paper I have discussed the general form of the maximum entropy production principle as it is used in climate science, emphasising the notion of negative feedback boundary conditions, the presence of which allows a unique maximum when the entropy function is varied with a flow rate. I have also emphasised that the principle of maximum entropy production has a serious conceptual difficulty in that it gives different results depending on which part of a physical system it is applied to.

I have sketched the beginnings a theoretical approach, based on the MaxEnt principle of Edwin Jaynes, which shows some promise of solving this problem.

Although the theoretical derivation of the MEPP presented here is somewhat tentative, if it or something like it turns out to be correct it will have wide-ranging implications for the study of non-equilibrium systems. This lends great importance to future theoretical and experimental work.

The support of Inman Harvey was instrumental in carrying out this work.