Understanding Atmospheric Behaviour in Terms of Entropy: a Review of Applications of the Second Law of Thermodynamics to Meteorology

The concept of entropy and its relevant principles, mainly the principle of maximum entropy production (MEP), the effect of negative entropy flow (NEF) on the organization of atmospheric systems and the principle of the Second Law of thermodynamics, as well as their applications to atmospheric sciences, are reviewed. Some formulations of sub-grid processes such as diffusion parameterization schemes in computational geophysical fluid dynamics that can be improved based on full-irreversibility are also discussed, although they have not yet been systematically subjected to scrutiny from the perspective of the entropy budgets. A comparative investigation shows that the principle of MEP applies to the entropy production of macroscopic fluxes and determines the most probable state, that is, a system may choose a development meta-stable trajectory with a smaller production since entropy production behavior involves many specific dynamical and thermodynamic processes in the atmosphere and the extremal principles only provide a general insight into the overall configuration of the atmosphere. In contrast to the principle of MEP, the analysis of NEF is able to provide a new insight into the mechanism responsible for the evolution of a weather system as well as a new approach to predicting its track and intensity trend.


Introduction
The atmosphere has been likened to a giant thermodynamic engine in which disorganized energy is transformed into the organized kinetic energy of the winds while the general circulation of the atmosphere can be regarded as simply being driven by temperature differences between the polar and equatorial regions.As mentioned by Sadi Carnot, "We must attribute to heat the great movements that we observe all about us on the Earth.Heat is the cause of currents in the atmosphere, of the rising motion of clouds, of the falling of rain and of other atmospheric phenomena ..." [1][2][3][4][5].
The notions of entropy and its production in equilibrium and nonequilibrium processes not only form the basis of modern thermodynamics and statistical physics, but have always been at the core of various ideological discussions concerned with the evolution of the world, the course of time, etc.These issues were raised by many outstanding scientists, including Clausius, Boltzmann, Gibbs and Onsager.As a result, today we have thousands of books, reviews and papers dedicated to properties of the entropy in different systems [4,6,7].
Some concepts and theories in modern thermodynamics may be relevant and perhaps crucial to enable a deeper understanding of atmospheric dynamics.Recently, more attention has been paid to thermodynamics and statistical mechanics in the atmospheric sciences.In addition to the large number of journal articles [8][9][10][11][12][13][14], several monographs have been published in succession [15][16][17][18].
As known to all, entropy has a different definition and is measured in a different way for the different fields.For example, in classical physics, the entropy of a physical system is proportional to the quantity of energy no longer available to do physical work.Entropy is central to the Second Law of thermodynamics, which states that in an isolated system any activity increases the entropy; in quantum mechanics, von Neumann entropy extends the notion of entropy to quantum systems by means of the density matrix.In probability theory, the entropy of a random variable measures the uncertainty about the value that might be assumed by the variable; in information theory, the compression entropy of a message (e.g., a computer file) quantifies the information content carried by the message in terms of the best lossless compression rate, in the theory of dynamical systems, entropy quantifies the exponential complexity of a dynamical system or the average flow of information per unit of time; in sociology, entropy is the natural decay of structure (such as law, organization, and convention) in a social system; and in the common sense, entropy means disorder or chaos.By saying that entropy is a measure of disorder we mean that the higher the entropy the greater the disorder, which are applied to the cases: when it is regarded as a measure of the unavailability of a system's energy to do work; as a parameter representing the state of disorder of a system at the atomic, ionic, or molecular level; as a measure of disorder in the universe or simply as a measure of disorder in any dynamical system [19,20].
Truly in thermodynamics, entropy is commonly associated with the amount of order, disorder, and/or chaos in a thermodynamic system.This stems from Rudolf Clausius' assertion [21] that any thermodynamic process always "admits to being reduced to the alteration in some way or another of the arrangement of the constituent parts of the working body" and that internal work associated with these alterations is quantified energetically by a measure of "entropy" change, according to the following differential expression: In the years to follow, Ludwig Boltzmann translated these "alterations" into that of a probabilistic view of order and disorder in gas phase molecular systems [22].In 1864, Boltzmann came across Maxwell's paper and was so inspired by it that he spent much of his long and distinguished life developing the subject further.Later, Boltzmann, in efforts to develop a kinetic theory for the behavior of a gas, applied the laws of probability to Maxwell's and Clausius' molecular interpretation of entropy so to begin to interpret entropy in terms of order and disorder.Similarly, in 1882 Hermann von Helmholtz used the word "Unordnung" (disorder) to describe entropy [22].
Entropy and disorder also have associations with equilibrium [23,24].Technically, for example, entropy, from this perspective, is defined as a thermodynamic property which serves as a measure of how close a system is to equilibrium-that is, to perfect internal disorder [23].Likewise, the value of the entropy of a distribution of atoms and molecules in a thermodynamic system is a measure of the disorder in the arrangements of its particles [25].In a stretched out piece of rubber, for example, the arrangement of the molecules of its structure has an "ordered" distribution and has zero entropy, while the "disordered" kinky distribution of the atoms and molecules in the rubber in the non-stretched state has positive entropy.Similarly, in a gas, the order is perfect and the measure of entropy of the system has its lowest value when all the molecules are in one place, whereas when more points are occupied the gas is all the more disorderly and the measure of the entropy of the system has its largest value [25].
Finally, it is needed to mention that Shannon entropy has been related by physicist Léon Brillouin to a concept sometimes called negentropy early in 1960s.In his 1962 book Science and Information Theory [26], Brillouin described the Negentropy Principle of Information or NPI, the gist of which is that acquiring information about a system's microstates is associated with a decrease in entropy (work is needed to extract information, erasure leads to increase in thermodynamic entropy).There is no violation of the Second Law of thermodynamics, according to Brillouin, since a reduction in any local system's thermodynamic entropy results in an increase in thermodynamic entropy elsewhere.Negentropy was considered as controversial because its earlier understanding can yield Carnot efficiency higher than one.In 2003 Weiss and Weiss came on a background of psychometric data and theoretical considerations to the conclusion that the negentropy principle underlies the information processing by the brain [27].In 2008 this has been empirically confirmed by a group of neurobiologists [28].In 2009, Mahulikar and Herwig redefined thermodynamic negentropy as the specific entropy deficit of the dynamically ordered sub-system relative to its surroundings [29].This definition enabled the formulation of the Negentropy Principle, which is mathematically shown to follow from the Second Law of Thermodynamics, during order existence.
In this paper we address only the entropy measures related to meteorology, which include thermodynamic entropy, the entropy of statistical mechanics (Boltzmann entropy or Gibbs entropy), and information entropy (Shannon entropy).In doing so, we understand that there exists an intrinsic connection between Boltzmann entropy and classical thermodynamic entropy as well as between Gibbs and Shannon entropy.
It should be pointed out emphatically that the importance of entropy theory lies not only in the entropy concept itself, but also in the extremal principles developed therein.Glansdorff and Prigogine wrote "Dissipative structures have a quite different [from equilibrium structures] status: they are formed and maintained through the effect of exchange of energy and matter in non-equilibrium conditions" [30].They were referring to the dissipation function of Rayleigh [31] that was used also by Onsager [32,33].Glansdorff and Prigogine concluded that at a stable steady state, the dissipation function was minimum [30].10 years later, Sawada, postulating a principle of largest amount of entropy increment per unit time [34], cites work in fluid mechanics by Malkus and Veronis [35] as having "proven a principle of maximum heat current, which in turn is a maximum entropy production for a given boundary condition".On the other hand, using the information theoretical formalism of Jaynes, Dewar claimed to have derived a principle of maximum entropy production applying to nonlinear stationary states [36,37].With a background of previous literature, a number of authors [38] cited Dewar's purported proof.In particular, Beretta, in an attempt to unify quantum mechanics and thermodynamics [39][40][41], proposed a generalization of the time-dependent Schrödinger equation of quantum mechanics, that postulates and implements at the single particle level the principle of maximum entropy production (even though, at that time, he named it "steepest entropy ascent dynamics").Beretta has also extended his steepest-entropy-ascent nonlinear dynamical equation to model the time evolution of non-equilibrium probability distributions in discrete, non-quantum contexts [42].
As a result, this paper will first focus more attention to the works related the maximum entropy production principle (MEPP) and its applications to meteorology.On the other hand, since entropy production is only one component of the whole entropy budget another important component called entropy flow in the entropy balance equation will be intensively addressed here as well.Recently the applications at this direction show an increasing trend owing largely to enlightening by the theory of dissipative structures created by Prigogine, e.g., [30].
In fact, entropy flow is a core concept in the modern theory of dissipative structures just as entropy is in classical thermodynamics and statistical physics.According to the Second Law of thermodynamics, an isolated system will evolve spontaneously into equilibrium with maximum entropy in which the order or organization of the system is at the minimum [43][44][45].As a result, negative entropy is very important for a system to keep itself far from its equilibrium, which is also true in biological systems (for a biological individual, equilibrium means death [46] as reflected in the statement that life's existence depends on its continuous gain of 'negentropy' from its surroundings [47]).
It is noticed that there has been little research based directly on the Second Law of thermodynamics itself in atmospheric sciences.As well-known to all, this law is usually expressed in terms of an inequation, which makes its applications inconvenient, but some progress has nevertheless been reached in this field.
This review paper is arranged as follows: a number of major concepts of entropy used in meteorology are presented in the next section; in Section 3 starting from the entropy balance equation one introduces briefly the issues regarding entropy budgets and isentropic analyses as well as isentropic coordinates employed in meteorological numerical models; Sections 4 and 5 are devoted to several research results concerning entropy production and entropy flow, respectively; Section 6 discusses applications of the principle of the Second Law to meteorology; and some discussions and conclusion are given in the last section.

Thermodynamic Entropy
The entropy of a system is defined as a summation of "heat supplied" divided by its "temperature" [48].If a certain small amount of heat įQ is supplied quasi-statically to a system with an absolute temperature of T, then the entropy of the system will increase by: where S is the entropy of the system, d represents an infinitesimal small change of a state function, and į represents that of a path function.Heat can be supplied by conduction, convection, or radiation.The entropy of the system will increase by equation (1), no matter which way we may choose.When we extract the heat from the system, the entropy of the system will decrease by the same amount.Thus the entropy of a diabatic system, which exchanges heat with its surrounding system, can either increase or decrease, depending on the direction of the heat exchange.This is not a violation of the Second Law of thermodynamics since the entropy increase in the surrounding system is larger.The Second Law (the law of entropy increase) is valid for a whole (isolated) system.When we sum up all the changes of entropy of interacting subsystems, the total change must be nonnegative.This is a statement of the Second Law of thermodynamics.

The Entropy of Statistical Mechanics
In statistical mechanics, the Boltzmann entropy is an approximation to the normal Gibbs entropy which refers to the statistical formulation of entropy for an ideal gas system of particles as found in Willard Gibbs' 1902 book Elementary Principles in Statistical Mechanics [49].
The Boltzmann entropy is obtained if one assumes one can treat all the component particles of a thermodynamic system as statistically independent.The probability distribution of the system as a whole then factorises into the product of N separate identical terms, one term for each particle; and the Gibbs entropy simplifies to the Boltzmann entropy: where the summation is taken over each possible state in the 6-dimensional phase space of a single particle (rather than the 6N-dimensional phase space of the system as a whole) and p i is the probability of a microstate with k B as the Boltzmann constant.This reflects the original statistical entropy function introduced by Ludwig Boltzmann in 1872 [50].For the special case of an ideal gas it exactly corresponds to the proper thermodynamic entropy mentioned in Section 2.1 above.

Information Entropy
In information theory, entropy is a measure of the uncertainty associated with a random variable.The term by itself in this context usually refers to the Shannon entropy (Shannon information entropy) [51,52], which quantifies, in the sense of an expected value, the information contained in a message, usually in units such as bits.
The entropy H of a discrete random variable X with possible values {x 1 , ..., x n } is: ) where E is the expected value, and I is the information content of X. I(X) is itself a random variable.If p denotes the probability mass function of X then the entropy can explicitly be written as: where b is the base of the logarithm used.
By the way, the inspiration for adopting the word entropy in information theory came from the close resemblance between Shannon's formula and a very similar formula from thermodynamics.In statistical thermodynamics, the most general formula for the thermodynamic entropy S of a thermodynamic system is the Gibbs entropy: which was defined by J. Willard Gibbs in 1878 [53] after earlier work by Boltzmann [50].Note that Gibbs entropy also translates over almost unchanged into the world of quantum physics to give the von Neumann entropy introduced by John von Neumann in 1927 [19]: where ȡ is the density matrix of the quantum mechanical system and Tr is the trace.
Consider a discrete probability distribution among m mutually exclusive propositions.The most informative distribution would occur when one of the propositions was known to be true.In that case, the information entropy would be equal to zero.The least informative distribution would occur when there is no reason to favor any one of the propositions over the others.In that case, the only reasonable probability distribution would be uniform, and then the information entropy would be equal to its maximum possible value, log m.The information entropy can therefore be seen as a numerical measure which describes how uninformative a particular probability distribution is, ranging from zero (completely informative) to log m (completely uninformative).
The principle was first expounded by E.T. Jaynes in two papers in 1957 [54,55] where he emphasized a natural correspondence between statistical mechanics and information theory.In particular, Jaynes offered a new and very general rationale why the Gibbsian method of statistical mechanics works.He argued that the entropy of statistical mechanics, and the information entropy of information theory, are principally the same thing.Consequently, statistical mechanics should be seen just as a particular application of a general tool of logical inference and information theory.
As a matter of fact, the explanation of these results the fluctuation theorem, maximum entropy production and self-organized criticality as in the classical thermodynamics on general information theoretic grounds underlines their relevance to a broad class of stationary, non-equilibrium systems.In turn, the accumulating empirical evidence for these results lends support to Jaynes' formalism (i.e., Jaynes' information theory formalism of statistical mechanics) as a common predictive framework for equilibrium and non-equilibrium statistical mechanics [56].

On Entropy Balance
According to classical thermodynamics, the entropy change in an open system consists of two contributory parts: one is entropy flow that is caused by the positive or negative entropy exchange with its surroundings; and, the other is positive definite entropy production owing to irreversible processes within the system.This can be described by the entropy balance equation which is derived from the Gibbs relation and local equilibrium assumption and, for entropy per unit volume, S, is of the form: where ı denotes the entropy production, and the "div"-term is the entropy flow with w denoting the partial differentiation [45,57].In the following sections (Section 4 and Section 5) discussions on the classical thermodynamic entropy production and entropy flow are just based on equation ( 2) above.
In physics, a variety of conservative laws for physical quantity such as mass, momentum and energy have been proved.However, entropy is a very special quantity to which there is no corresponding conservation law.With the local equilibrium assumption employed an entropy balance relation can be derived as mentioned above.As known to all, for an isolated system, its entropy will either increase spontaneously if it is apart from equilibrium or keep unchanged if it is at equilibrium.For an open system, even if the entropy inside the system decreases owing to negative entropy flow into the system from its surroundings a greater system that comprises the original system and its surroundings will experience an entropy-increasing process [45].
Indeed, according to [58], in the long-term mean the outgoing flux of longwave radiation to space equals the incoming flux of solar energy almost exactly, but the corresponding entropy fluxes are quite different.In fact, the entropy exported by the outgoing longwave radiation is found to be about 22 times larger than the entropy associated with the incoming solar radiation.
At the same time, Peixoto et al. claim that the net export of entropy to space by the climate system must be equal to the internal production of entropy by the irreversible processes that occur within the system [58].As a result, the total entropy for the whole climate system should be an invariant.
Usually, the entropy S is expressed as C p lnΫ with C p being the specific heat at constant pressure and Ϋ the potential temperature that relates temperature T by: T where R is the gas constant for dry air and p 0 reference pressure that is usually taken as 1000 hPa.In meteorological applications, not only is the total entropy for the whole climate system regarded as being an invariant but also entropy is assumed to be unchanged in many cases.
Indeed the continuous improvement of the isentropic vertical coordinate favors the further understanding of the nature of atmospheric motion and the mechanism behind it.On the other hand, Arakawa and Lamb have previously emphasized that global conservation of lnΫ is important for climate modeling [82].This suggests that if entropy itself were the thermodynamic property and isentropic coordinates are utilized, not only will global conservation of lnΫ be assured, but also local aphysical sources of entropy by numerical mixing would simply be precluded.However, as Sommerfeld states, "in spite of their not being real, isentropic processes are most important in thermodynamics because definite equations can only be obtained by considering reversible changes" [83].Later Johnson et al. further discussed numerical uncertainties in the simulation of reversible isentropic processes and the relevant issues related to entropy conservation [84,85].Here, we do not intend to go into their detailed conclusions but just recall the fact that the isentropic motion assumption requires entropy conservation, which can only be valid if the motion is both adiabatic and frictionless, along with the classical thermodynamic entropy definition itself requires the process (the atmospheric motion in our context) to be experienced at equilibrium (that's why the local equilibrium assumption is adopted).As a result, due caution should be exercised when the isentropic approach is employed.

Entropy Production
From the entropy balance equation (2) above, local changes in entropy cover two parts: one of them is entropy flow and the other is entropy production.We will specifically discuss, in this section, the entropy production that has been more intensively addressed in the field of atmospheric sciences.
Let us first review Paltridge's work in 1975 in which he suggested a well-know relevant principle, the principle of maximum entropy production (MEP) [86,87].Indeed, according to Tuck, "On the macroscopic level, the way of MEP has been pioneered by a meteorologist" [86,88].Initially Paltridge used the terminology "minimum entropy exchange" [86], but after that, he used the current terminology "MEP" to describe the same thing [88][89][90].This point is clarified in the review by Ozawa et al. [4].
Studies on the MEP principle (MEPP) originated from meteorology but covered various fields, including physics, chemistry, astronomy, geoscience and life science as well as social sciences, and some reviews have been made on the MEPP recently e.g., [4,7].From the basic concepts, Ozawa et al. [4] introduced first a number of significant works on global climate including Paltridge's, and then discussed the relevant results in the fields such as climates on other planets and mantle convection in planets as well as a simple theory of turbulence that was recently shown to be consistent with the hypothesis of MEP [91].Martyushev and Seleznev summarized and analyzed the studies dealing with MEPP and, also starting from the thermodynamic and statistical basis of the principle (including the relationship of MEPP with the Second Law of thermodynamics and Prigogine's principle), presented various existing applications of the principle in the analysis of nonequilibrium systems, including climate systems [7].
Early in the 1970s Paltridge proposed that Earth's climate structure might be explained from a hypothesis of MEP [86,92].If correct, this proposal would be of crucial importance to future climate research because it provides the hitherto missing global constraint of the Second Law of thermodynamics [93].As pointed out [86], the most probable macroscopic flux configuration did be very close to real observations.Figure 1 shows a schematic of energy transport processes in a planetary system composed of the Earth and the Sun.Shortwave radiation emitted from the Sun with a brightness temperature of about 5,760 K is absorbed by the Earth, mainly in the equatorial region.This energy is transported poleward through direct motions of the atmosphere and oceans (the general circulation).The energy is finally reemitted to space via longwave radiations from the atmosphere at a temperature of about 252 K, from clouds at 259 K and from the Earth's surface at 288 K. Thus there is a flow of energy from the hot Sun to cold space through the Earth.In the Earth's system the energy is transported from the warm equatorial region to the cool polar regions by the atmosphere and oceans.Then, according to Carnot, a part of the heat energy is converted into potential energy which is the source of the kinetic energy of the atmosphere and oceans.In this respect, the Earth's system can be regarded as a heat engine operating between thermal reservoirs with different temperatures (equator and poles) [4].On the other hand, the atmospheric system, especially its troposphere, can be regarded as a heat engine operating between two thermal reservoirs that is the surface at a higher temperature and the tropopause at a lower temperature.That seems to explain why severe weather phenomena, including severe convection, occur in the bottom sphere of the Earth atmosphere-the troposphere.The determination of the strength of the circulation, and hence the rate of heat transport, constitutes a fundamental problem in thermodynamics of the general circulation, e.g., [94].A schematic of energy transport processes in the planetary system of the Earth, the Sun and space.The Earth receives the shortwave radiation from the hot Sun and emits longwave radiation into space.The atmosphere and oceans act as a fluid system that transports heat from the hot region to cold regions via general circulation.
As mentioned by Ozawa et al. [4], Lorenz suspected that the Earth's atmosphere operates in such a manner as to generate available potential energy at a possible maximum rate [95].The available potential energy is defined as the amount of potential energy that can be converted into kinetic energy.Independently, Paltridge suggested [86,92] that the mean state of the present climate is reproducible as a state with a maximum rate of entropy production due to horizontal heat transport in the atmosphere and oceans [4].Figure 2 shows such an example [86].Without considering the detailed dynamics of the system, the predicted distributions (air temperature, cloud amount, and meridional heat transport) show remarkable agreement with observations [4].Later on, several researchers investigated Paltridge's work and obtained essentially the same result [96][97][98][99][100][101][102][103].Recently an information theoretical formulation of nonequilibrium statistical mechanics (MaxEnt, i.e., the constrained maximization of Shannon information entropy) is used to derive the fluctuation theorem (FT) concerning the probability of Second Law violating phase-space paths [104].Dewar presented a more rigorous and general mathematical derivation of MEP from MaxEnt, and the relationship between MEP and the FT is clarified thereby.Specifically, it is shown that the FT allows a general orthogonality property of maximum information entropy to be extended to entropy production itself, from which MEP then follows.The new derivation highlights MEP and the FT as generic properties of MaxEnt probability distributions involving anti-symmetric constraints, independently of any physical interpretation [105].Physically, MEP applies to the entropy production of those macroscopic fluxes that are free to vary under the imposed constraints, and corresponds to selection of the most probable macroscopic flux configuration.On the other hand, here we would like to mention that, as Grinstein and Linsker showed, "the question of the existence of possible extremal principles (and in particular, of MaxEP) that might apply to far-from-equilibrium regimes (having non-linear constitutive relations) has not been settled" [106][107][108].We will go into further discussions on it in the Section 7 below.By the way, rooted in information theory, the maximum entropy method (MEM) seeks to extract as much information from a measurement as is justified by the data's signal-to-noise ratio [109].It has been used successfully in a variety of fields, including radio astronomy, NMR, fluorescence, and ligand rebinding [110][111][112][113][114][115][116].The method is by no means restricted to conventional imaging problems, but the word 'image' will be used to refer to any function to be extracted from the data.
Although there is no consensus on the MEM numerous works have been directed at the principle of MEP as well as the MEM and their applications to meteorology, e.g., [117][118][119][120][121][122] henceforth.Despite the different details most of them are focused on the climate modeling based on the MEPP, subgrid parameterization scheme and numerical restriction from the entropy budget caused substaintially by entropy production.In addition to the exact thermodynamic origin, some of them are related to the extreme sensitivity of a climate model's temperature response to aphysical entropy sources introduced by spurious numerical dispersion diffusion, Gibbs oscillations, parameterizations, and other factors, see also [123][124][125][126][127][128][129][130][131][132][133][134][135][136][137][138][139].The results in combination with Carathéordary's statement of the Second Law reveal in the strict sense that the presence of positive definite aphysical sources of entropy in a climate model precludes the simulation of unbiased distributions of the additional heat and temperature [140][141][142].
Finally, we would like to particularly review the works concerning the potential mechanism for MEP.For instance, in addition to a regulation mechanism by turbulent fluctuations suggested by Paltridge [86,88,143], Ozawa et al. discussed qualitatively a mechanism by which a turbulent fluid system adjust itself to a state of maximum generation of available potential energy or, equivalently, MEP [4].As a matter of fact, it is obvious from the point of view of information theory that, by choosing to use the distribution with the maximum entropy allowed by our information, the argument goes, we are choosing the most uninformative distribution possible.To choose a distribution with lower entropy would be to assume information we do not possess; to choose one with a higher entropy would violate the constraints of the information we do possess.Thus the maximum entropy distribution is the only reasonable distribution.
By the way, as mentioned in the theory of dissipative structures [30,144], the stability of a multi-body system is able to be judged in terms of excess entropy production that is the second order variation of entropy, į 2 s, where s is specific entropy with į 2 as the second order variational operator.Liu and Tao suggested a new stability criterion based on the Liapunov second method with the excess entropy production į 2 s as a generalized Liapunov function by which the stability characteristics of a series of weather conditions are revealed [145].On the other hand, Pujol and Llebot used a second differential of the entropy as a criterion for the stability in low-dimensional climate models [146].

Entropy Flow
As pointed out in Section 3 above, the "div"-term in equation ( 2) describes the entropy flow entering into a system through its boundaries from the surroundings.The Earth system exchanges radiation with space, which drives and maintains almost all the processes within the Earth system.As shown in Figure 1, the essence of the Earth's radiation exchange with space is that the high-energy photons with low entropy from the Sun enter the Earth system and the low-energy photons with high entropy are emitted from the Earth system to space, see also [147].In general, the Earth system as a whole can be thought of as a multi-body system that is closed to nonradiation material exchange and open to radiation exchange with space.For such a complex system, the net entropy flux resulting from the radiation exchange constrains the Earth system's internal entropy production rate, a fundamental measure of the overall activities within the Earth system, including oceanic, atmospheric, and biological processes [148].
Because of the entropy in a system determining the order of the system to some extent, the evolution of a specific atmospheric system should be related with entropy flow.
In 1900 Bénard investigated a thin layer of fluid at rest originally, with a free surface at a lower temperature T 1 , heated from its bottom at a higher temperature T 2 (see Figure 3), and noticed that a rather regular cellular pattern of hexagonal convective cells was abruptly organised when the temperature difference (T 2 í T 1 ) reaching the value of the threshold of primary instability [30,44,45,149], suggesting that every system that obtains heat at a higher temperature but loses heat at a lower temperature will experience net negative entropy flow since in this case the entropy exchange ( e s G ) of the system with its environment or entropy flow entering into the system through its boundaries is as follows: where Q 1 and Q 2 are the heat fluxes through the top and bottom of the layer of fluid, respectively, and Q is the flux when the system becomes steady so that Later, in the 1940s, Schrödinger stated in his monograph "What Is Life?" that life's existence depends on its continuous gain of 'negentropy' from its surroundings [105].This implies that negative entropy flow is something very significant for a system, whether it is living or nonliving [30,44,45,47,150,151].That is, negative entropy flow will cause a system initially at equilibrium or even at rest to be organized, or lead a system already at non-equilibrium to a state further from equilibrium (i.e., cause it to strengthen).2) above), as a logical component of the entropy balance equation [30,44,45], and calculated the three-dimensional instantaneous entropy flow field of a number of the tropical cyclones [1,2,[152][153][154][155][156].The entropy flow vector s J * in -div J * s that is just the "div"-term in equation ( 2) or e s G mentioned above can be written as: where q J * , T, k P and k J * are the heat flow, temperature, chemical potential and diffusive flow for component k, respectively, e.g., [156].It is necessary to modify the entropy flow formula so as to make it be suitable for the computation of an atmospheric system such as hurricane.As a result, the entropy flow formula finally becomes: where u and v are the velocities in the xand y-direction at constant pressure layers, respectively; Z the vertical velocity in p-coordinates; I the geopotential height with I = gz in which g is the gravitational acceleration; Õ the thermal conductivity; and s the specific entropy [156].Equation ( 5) is the fundamental expression for computing entropy flow employed.
Here, we will introduce the computational results for one of the most typical cases-Hurricane Katrina (2005) that is extraordinarily powerful and deadly with a total damage cost of approximately $81 billion being roughly double that of Hurricane Andrew (1992) [157] and the total number of fatalities over 1800 and whose track is shown in Figure 4 [156,158,159] and then discuss the relationship of entropy flow with the development of the hurricane.
Let's first have a look at changes in the total entropy flow of Hurricane Katrina based on the National Centers for Environmental Prediction/National Center for Atmospheric Research (NCEP/NCAR) 1° × 1° (latitude-longitude) resolution reanalysis data (Figure 5) [160].
Figure 5 shows changes, at 6-hourly intervals, in the accumulative total entropy flow for Katrina as well as the corresponding maximum sustained wind speed representative of the intensity of the hurricane.It is seen from Figure 5 that there exists a general trend of anti-correlation between these two curves with the phase in the total entropy flow preceding that in the maximum wind speed.For example, Katrina experienced more and more increasing net negative entropy flow before 00:00 UTC 28 August 2005 and, which was followed closely by its continuous intensification with very rapid strengthening occurring on the morning of 28 August.On the other hand, the values of negative entropy flow for Katrina decreased distinctly and even turned to positive entropy flow after 00:00 UTC 28 August, which was followed by the gradual weakening of Katrina from its peak intensity of 150 kt later at 18:00 UTC 28 August.Katrina experienced a dramatic decrease of negative entropy flow and a successive turn to the stage of positive entropy flow during the period from 18:00 UTC 28 to 06:00 UTC 29 August when the very rapid weakening of Katrina followed accordingly 6 hours later starting with 12:00 UTC 29 August.As a result, the total entropy flow of a hurricane may contain some significant information conducive to its intensity forecast.In addition, it is found that the asymmetries of the entropy flow pattern around Katrina's centre in the lower troposphere (e.g., at 850 hPa as are shown in Figure 6 for the two times of 18 UTC 26 and 28 August which are marked also with the red spot on the Katrina's track in Figure 4 for easy reference) contained some significant information on hurricane track prediction.In fact in [156], there are four points (corresponding to four times of 18:00 UTC 26, 18:00 UTC 28, 18:00 UTC 29 and 00:00 UTC 30 August, respectively) along the Katrina track chosen and discussed to make the finding that Katrina (2005) tends to move towards the side of its centre with stronger negative entropy flows more robust and convincible, since in that way all the four potential moving directions, east-, south-, west-and north-wards, are covered.As stated in [156], the finding is the same for all of these four points along the track chosen.As a matter of fact, it occurs throughout the hurricane lifecycle, and is also valid for the other severe tropical storms (see also e.g., [1,2] in our MS).These results imply that classical thermodynamic entropy may serve as an order parameter for an atmospheric system and that entropy flow analysis might provide a new insight into the mechanism responsible for the life cycle of the system.In view of that such severe tropical storms tend to move towards the side of its centre with stronger negative entropy flows, the entropy flow patterns in the lower levels (e.g., 850 hPa) might act as an indicator of the hurricane track.This is understandable from the theory of dissipative structures [161]: negative entropy flow entering into an open system will potentially enable the system to depart further from equilibrium or cause it to strengthen, and so here a region of stronger negative entropy flow might become a likely area that the hurricane will approach.
To sum up, it might be argued that negative entropy flow is something very positive which can be regarded as a kind of forces to drive a system to a state further from equilibrium.Next we will give some examples in the Earth's atmosphere.
An estimation of the entropy budget of the atmosphere shows that the entropy flow to the Earth's atmosphere is negative [162], which recall spontaneously the fact that the troposphere of the Earth atmosphere experiences a double negative entropy flow process because it gets heat at the tropics with higher temperature but loses heat at both poles with lower temperature horizontally, and also it gets heat at the surface of the Earth with higher temperature but loses heat at the tropopause with lower temperature vertically.This might contribute to the causes, as implied by the Beҁ nard convection shown in Figure 3, why the worldwide extreme weather/climate events are getting more and more frequent [163][164][165][166] (see also  which are based on the data in [153], showing the increasing numbers of the severe weather events such as tornados, hails, and thunderstorms with high winds, in the United States, including Wisconsin which is one of the states in the US with the least number of devastating weather disasters in its history, between 1950 and 2003 with a dramatic rise in the numbers after 1990).

Applications of the Principle of the Second Law
So far there has been little research in meteorology based directly on the Second Law of thermodynamics.A plausible cause might be attributed to the fact that this law is expressed in the form of an inequality rather than an equation, which makes its use restricted.
Paltridge proposed Earth's climate structure could be explained via a hypothesis of MEP [86,89] which provides global constraint on the Second Law of thermodynamics [167].However, the fact that the Earth's climate system transports heat as does a system in a MEP state does not prove that the Earth's climate system is necessarily seeking such a state [4].The constraint of the Second Law on the thermodynamic behavior of the atmosphere is quite different from that as seen in a "common" physical law such as Newton's Second Law and Energy Conservation law.
Among the studies of the Second Law and its applications, some of them proposed a certain relevant thermodynamic parameter or approximation in a dynamical model that should satisfy or be fully consistent with the Second Law of thermodynamics and some even suggested a generalized Second Law of thermodynamics, including the Second Law of Eddy Thermodynamics [168].
Next we show some recent results with the emphases on illustrating how the principle of the Second Law of thermodynamics is employed to improve the dynamic framework of a general circulation model (and thus its simulation accuracy) and the intensity forecasts of a hurricane through a full-irreversability restraint applied on the diffusion scheme in a meso-scale numerical model.as Also we try to understand the potential mechanism responsible for climatic changes in the tropical cyclone number over the western North Pacific in an environment associated with global warming [169][170][171][172][173][174][175].
From the very beginning of numerical modeling of atmospheric motions, it was noticed that various mixing/diffusion parameterizations in a model are questionable in the sense that such schemes of parameterization are non-monotonic, which violates the Second Law of thermodynamics, and, dissipative heating associated with vertical momentum diffusion is often included, but in a way that is inconsistent with the Second Law of thermodynamics [172,[176][177][178][179][180][181][182][183].As a result, a number of new schemes are suggested based on the principle of the Second Law of thermodynamics.For example, the forecast accuracy of a numerical weather prediction model is improved by emulating physical dissipation rather than using the original diffusion scheme.
The ability of the new physics-based scheme to improve model accuracy is demonstrated first via the case of the one-dimensional viscous Burgers equation that can be written as: where u Ł u(x, t) is the flow velocity and R e the Renolds number.In the case of the initial condition taken as the linear distribution: u íx at t 0, í < x < with the boundary condition as: there exists a special solution: u íth[(R e /2)x], which is a steady solution containing no time.Figure 9 shows that the final solution with the original diffusion scheme has a Gibbs oscillation [184] near the "shock wave" where the velocity gradient augments suddenly.On the other hand, the numerical solution consistent with the analytic one is reached when the scheme is changed to the physics-based scheme that meets the restriction of the Second Law of thermodynamics.Using such a scheme, a series of numerical simulations, including the well-known 1998 successive torrential rains along the Yangtze River valley and 365 continuous 24-h integrations for 2005-06 improved with decreased root-mean-square errors and more realistic forecasts in all of the simulations [176][177][178][179]. Figure10 shows the intensity forecasts of a hurricane, Hurricane Andrew (1992), improved through a full-irreversability restraint applied on the diffusion scheme in a meso-scale numerical model (the fifth-generation Pennsylvania State University-National Center for Atmospheric Research Mesoscale Model (MM5) [185] where a fourth order scheme originally-used created upgradient mass or heat transport, which is spurious and violates the principle of the Second Law of thermodynamics that controls the evolutionary direction of a multi-body system like the atmosphere [182].It is seen from Figure 10 that the surface maximum sustained wind velocity forecasts based on the new scheme that incorporated the Second Law are distinctly closer to observations than those based on the old one, especially for the period before 54 h.A global spectral model has also been improved by incorporating the second law of thermodynamics in a slightly different way from the above [186].
In the end of this section, one would like to mention an application of the Second Law to typhoon climate, in which Liu et al. proposed a potential mechanism responsible for changes in the tropical cyclone number during the recent 60 years or so over the western North Pacific in an environment associated with global warming based on the principle of this Law [187].

Discussion and Conclusions
The Second Law of Thermodynamics is one of the most fundamental principles in thermodynamics that controls the evolutionary direction of a multi-body such as the Earth's atmosphere.Entropy is a crucial physical concept originated from this law.As a result, applications of entropy and the Second Law to atmospheric sciences are significantly important.For the sake of convenience, this paper introduces the works under the subtitles of Entropy balance, Entropy production, Entropy flow, and Applications of the principle of the Second Law, respectively.
Since the 1950s a series of studies of atmospheric dynamics and thermodynamics into the entropy theory and its applications with stress put on entropy production have been made.However, till now the studies on entropy production and its applications are still very preliminary.Early in 1980 Nicolis and Nicolis discussed Paltridge's work and commented that the behaviour of entropy production is far from simple and universal [188].On the other hand, entropy production behaviour involves many specific dynamical and thermodynamic processes in the atmosphere, the extremal principles can only provide a general insight into the overall configuration of the atmosphere.More relevant works need to be done in this direction, see also [182].In fact, later Lucia offers a proof of a theorem for the eventual steady state of a constrained open system: "The principle of maximum irreversible entropy: The irreversible entropy reaches its maximum at the stability" [189].What is more, Lucia's proof is limited to the theory of so-called "rational thermodynamics" in which there is not necessarily an empirically verifiable definition of entropy, let alone entropy production.Sometimes the MEPP mentioned above is used by many researchers who guide themselves by their intuition and are often unaware of the general formalism supporting the validity of MEPP.The point is that MEPP determines just the most probable state and a system may choose a development trajectory with a smaller production, but it will be meta-stable.
In nature, there is much room for improving the entropy theory, including the extremal principles of entropy or its production, as well as the Second Law itself.For example, as Romps mentioned, questions like "Can the entropy budget place stringent bounds on the intensity of storms?" have not yet been addressed, but given the importance of wind-generated dissipation to those living in the atmosphere's boundary layer, these questions certainly deserve attention [175].Recently emphasis has been given to the entropy principle in computational fluid dynamics, although within a different context.Several questions and avenues of inquiry are raised.Specifically, while there are many different formulations of sub-grid processes in the literature on fluid turbulence, they have not yet been systematically subjected to scrutiny from the perspective of the entropy budgets, among which, e.g., the first-order down-gradient Smagorinsky formulation for the sub-grid transport of dry static energy clearly fails to comply with expectations that sub-grid heat transport be a net source of entropy or dry entropy.This would merit further research to assess existing turbulence schemes for their ability to produce entropy sources of the correct sign and magnitude [175].It needs to be pointed out that the studies mentioned in Section 5 in this paper, on the forecast accuracy of a meso-scale numerical weather prediction model improved by emulating physical dissipation as suggested by the Second Law of thermodynamics, are done in a different way in which the newly-constructed diffusion scheme with a full-irreversability restraint is applied based directly on the principle of the Second Law.
It is noticed that the importance of the entropy budget in relation to its structure, its exchange, and its sources and sinks has received almost no attention in both medium-range weather prediction and climate modeling.
The studies on the effect of negative entropy flow on the self-organization of the atmospheric systems, discussed in Section 4 in this paper, provide a new insight into the mechanism responsible for the evolution of a weather system as well as a new tool to predict its track and intensity change trend.However, the entropy balance equation and the entropy formulism suitable for the atmospheric applications remain to be further improved e.g., the components of the specific entropy remain to be extended so as to cover all of the water phases.
Lastly, one would like to mention that a variety of computable entropy measures (e.g., block entropy, Kolmogorov entropy, Kolmogorov-Sinai entropy, T complexity, Ledrappier-Young Entropy, Pesin Entropy, Topological entropy, von Neumann entropy, Black hole entropy, and approximate entropy) have been proposed in various subjects [19,20,36,37].The theory of entropy has been developing itself and, its applications to and the proposition of a definition of entropy perfect for meteorology both certainly deserve attention.

Figure 1 .
Figure 1.A schematic of energy transport processes in the planetary system of the Earth, the Sun and space.The Earth receives the shortwave radiation from the hot Sun and emits longwave radiation into space.The atmosphere and oceans act as a fluid system that transports heat from the hot region to cold regions via general circulation.

Figure 2 .
Figure 2. Latitudinal distributions of (a) mean air temperature, (b) cloud cover, and (c) meridional heat transport in the Earth.Solid line curves indicate those predicted with the constraint of maximum entropy production (equation (9)), and dashed lines indicate those observed.(Reproduced with minor changes based on [4])

Figure 3 .
Figure 3.The schematic diagram for Bénard convection as the prototype of self-organization.

Figure 4 .
Figure 4. Best track positions for Hurricane Katrina, 23-30 August 2005.The red spot marks are for the corresponding position of the hurricane centres to that shown in Figure 6 (Reproduced with minor changes based on [158]).

Figure 5 .
Figure 5.The evolution of total entropy flow (TEF, in 10 6 J K í1 s í1 ) for Hurricane Katrina at 6-hourly intervals, with changes in the maximum sustained wind speed (V max, in kt) as a reference.

Figure 6 .
Figure 6.The entropy flow fields at 850 hPa at 18:00 UTC 26 (left panel) and at 18:00 UTC 29 (right one) August 2005.The red spot stands for the corresponding position of the hurricane centre's to the respective time.Green lines are for contours at 850 hPa and black ones are for isopleths of entropy flow with the areas of negative value shaded.

Figure 7 .
Figure 7.The number of tornados in the United States/Wisconsin between 1950 and 2003.

Figure 8 .
Figure 8.The numbers of thunderstorms, high winds with speed of at least 75 knots and hails with size of at least 2.5 inches in Wisconsin, between 1950 and 2003.

Figure 9 .
Figure 9.Comparison of analytic solution of the one-dimensional viscous Burger's equation that runs to a steady flow with the numerical solutions between the original and physics-based diffusion schemes used.

Figure 10 .
Figure 10.Comparison of the surface maximum sustained wind velocity between the different schemes with observations.

1 )V
obs -observed surface maximum wind velocity V old -old MM5 simulated surface maximum wind velocity V new -new MM5 simulated surface maximum wind velocity