Please select whether you prefer to view the MDPI pages with a view tailored for mobile displays or to view the MDPI
pages in the normal scrollable desktop version. This selection will be stored into your cookies and used automatically
in next visits. You can also change the view style at any point from the main header when using the pages with your
Health Effects Laboratory Division, National Institute for Occupational Safety and Health, Morgantown, West Virginia 26505, USA
Center for Advanced Research in Biotechnology, University of Maryland Biotechnology Institute, Rockville, Maryland 20850, USA
Current Address: Skaggs School of Pharmacy and Pharmaceutical Sciences, University of California San Diego, 9500 Gilman Drive, MC 0736, La Jolla, California 92093, USA.
Authors to whom correspondence should be addressed.
Received: 17 December 2009; in revised form: 27 February 2010 / Accepted: 2 March 2010 / Published: 16 March 2010
The differential Shannon entropy of information theory can change under a change of variables (coordinates), but the thermodynamic entropy of a physical system must be invariant under such a change. This difference is puzzling, because the Shannon and Gibbs entropies have the same functional form. We show that a canonical change of variables can, indeed, alter the spatial component of the thermodynamic entropy just as it alters the differential Shannon entropy. However, there is also a momentum part of the entropy, which turns out to undergo an equal and opposite change when the coordinates are transformed, so that the total thermodynamic entropy remains invariant. We furthermore show how one may correctly write the change in total entropy for an isothermal physical process in any set of spatial coordinates.
The Gibbs entropy of classical statistical thermodynamics is, apart from some non-essential constants, the differential Shannon entropy  of the probability density function (pdf) in the phase space of the system under consideration. However, whereas the thermodynamic entropy is not expected to depend upon the choice of variables, the differential entropy can be changed by a transformation of variables. In particular, the differential entropy of a spatial pdf depends on the choice of the coordinates used to describe the spatial configuration of the system. Moreover, a change of variables can change not only the absolute differential entropy, but also its change on a change in the pdf, as shown below. This sensitivity to coordinates appears paradoxical, since a physically meaningful quantity ought to be independent of the choice of spatial coordinates. A similar concern was previously expressed in a critique of the concept of the differential entropy itself .
Here, we demonstrate that, for the thermodynamic entropy, a transformation of the spatial coordinates is accompanied by a compensating change of the entropy of the canonically conjugate momenta so that the full thermodynamic entropy remains invariant. The invariance of the full entropy stems from the fact that the Jacobian of a canonical transformation equals unity, and explicit demonstration of this invariance yields a simple formula for correcting a spatial entropy computed with transformed coordinates to yield correct full entropy changes. These results have application in calculations of spatial entropy from molecular simulations when Cartesian coordinates are transformed, for example to bond-angle-torsion  coordinates.
The paper is structured as follows. Section 2 reviews the formalism of entropy in classical statistical thermodynamics, defines a splitting of the full thermodynamic entropy into momentum and spatial parts for the case in which the spatial coordinates used are Cartesian, and shows that then the change in the spatial entropy equals the change in the total entropy, for an isothermal process. (Note that the spatial entropy of the solute part of a solute-solvent system is often termed the solute’s configurational entropy.) Section 3 and Section 4 investigate the effects on the spatial and momentum entropy, respectively, of a transformation of the Cartesian coordinates to general spatial coordinates. Section 5, discusses how one may evaluate the change in the full thermodynamic entropy due to an isothermal process in terms of the change in the spatial entropy evaluated in non-Cartesian coordinates. Finally, Section 6 draws conclusions.
2. Spatial Entropy in Cartesian Coordinates
In classical statistical thermodynamics, the entropy S of a system described by coordinates and the canonically conjugate momenta , briefly p and q, respectively, is given in terms of the system’s pdf in the phase space as 
Here, is Boltzmann’s constant, and the factor , where is Planck’s constant (quasi-classically, the number of states in a volume element of the phase space is , see e.g., ), ensures that the argument of the logarithm is dimensionless. The probability distribution function is given by the Boltzmann-Gibbs distribution
where , with T being the absolute temperature, is the system’s energy, and
is the partition function, which is the distribution’s normalization constant divided by to make it dimensionless. The entropy S can be written in terms of the partition function Z as
is the mean (expectation) value of the energy .
Let us assume that the coordinates q are Cartesian. Then the energy is the sum of a kinetic energy, K, which is a function of the momenta p only, , where are the masses associated with degrees of freedom i, and a potential energy , which depends only on spatial coordinates. The probability distribution (2) and the partition function (3) now factorize as
are momentum and spatial probability distributions with normalization constants
which can be termed, respectively, the momentum and spatial partition functions. It should be noted that neither of these partition functions are dimensionless; the treatment of the factor is discussed later in this section.
Accordingly, the full entropy (1) can now be separated as
where is a momentum entropy which can be evaluated in closed form,
and is a spatial entropy,
Similarly to (4), the spatial entropy can be written as
is the mean value of the potential energy .
A factor of is included in the definition of the momentum entropy (13) so that the full entropy has the correct physical dimensions of energy/temperature. The association of this factor with the momentum entropy is arbitrary (as was its inclusion in the momentum partition function (10)); it could instead have been included in the spatial entropy. Unfortunately, the factor cannot be split so that both parts of the full entropy are dimensionless. Thus, neither of the “partial" entropies and is correctly dimensioned. Nonetheless, the troubling dimensions cancel for differences in and arising from changes in the pdf, since the relevant terms appear in the arguments of logarithms, and so such differences are physically meaningful.
It is evident from Equation (13) that the momentum entropy does not depend upon the spatial pdf , but only upon the momentum pdf , which in turn depends only on the atomic masses and the temperature. As a consequence, for an isothermal physical process with a fixed set of atoms, the change in total entropy equals the change in spatial entropy:
More generally, this equation holds in any coordinate system for which the kinetic energy K is independent of the spatial coordinates, .
Note that the spatial entropy defined here is akin to the configurational entropy. However, the latter usually refers specifically to the entropy associated with the conformational fluctuations of a molecule in solution, and is therefore exclusive of the solvent entropy . The spatial entropy is more general, as it may refer to the whole system or any of its parts.
3. Spatial Entropy under a Coordinate Transformation
It is sometimes of interest to compute the change in total entropy, , associated with an isothermal molecular process, such as protein-ligand binding or protein-folding. Equation (17) shows that the change in total entropy can be obtained by computing the change in the spatial entropy in Cartesian coordinates. However, Cartesian coordinates are not always optimal for this purpose, because the pdf in Cartesian coordinates includes many coordinate dependencies that are rather easily removed by transforming to more natural coordinates, such as bond-angle-torsion (BAT) coordinates . For example, even the Cartesian coordinates of a single atom, , can be strongly correlated with each other, due to the natural tendency of each atom to move in a circular trajectory corresponding to a bond-rotation. This motion is readily captured by a single torsional variable. For this reason, a transformation from Cartesian coordinates to suitably defined internal coordinates of the molecular system under consideration, plus the coordinates of the translation and rotation of the system as a whole, is often performed.
Transforming from Cartesian coordinates q to new coordinates (with an inverse ), transforms the probability density function (pdf) into a pdf of the new coordinates Q according to a rule of general probability theory as
where is the Jacobian of the transformation. The differential Shannon entropy of information theory, defined as 
can be written in terms of the new coordinates as
is the Shannon entropy of the transformed pdf and
is the expectation value of the logarithm of the Jacobian .
Based upon Equation (20), the spatial part of the thermodynamic entropy (14) in Cartesian coordinates q, , transforms to a different value on changing to internal coordinates Q:
where . This result may seem troubling, since the thermodynamic entropy of a physical system should not depend on the choice of the spatial coordinates used. Nonetheless, this Jacobian correction is real and is not even guaranteed to cancel when one computes the difference between two entropies associated with a change in the potential function and a consequent change in the physical pdf from ρ to some . That is, the change in the Cartesian spatial entropy arising from a change in the pdf of q does not necessarily equal the change in the transformed spatial entropy arising from the corresponding change in the transformed pdf of Q. This is because the Jacobian can vary with Q, so its contribution to the entropy for ρ (22) can differ from that for
We note that a similar treatment of the transformation of spatial entropy under a change of coordinates has been given in .
4. Momentum Entropy under a Coordinate Transformation
The coordinate transformation is called a point transformation because the new coordinates Q are functions only of the old coordinates q; that is, they do not involve the old conjugate momenta p. (A general point transformation may also involve an explicit dependence on time.) A point transformation of the spatial coordinates is associated with a canonical transformation of the full phase space coordinate system; i.e., of both the spatial and momentum coordinates, such that , , with the inverses , . In a canonical transformation, the new variables remain canonically conjugate, which means that
where , with , is the system’s Lagrangian expressed in terms of Q. Classical statistical thermodynamics is based on the Hamilton formalism of mechanics, and property (26) ensures that the Hamilton equations in terms of retain their canonical form. Here, an important property of a canonical transformation is that its Jacobian equals unity . This means that the phase-space volume element of an integration in the full phase space is invariant:
A point transformation itself has in general a Jacobian , and so . Thus, for invariance (27) to hold, the momentum volume element must transform in a canonical transformation as
Using (27), we can write the full entropy (1) in terms of an integral in the new phase space variables simply as
is the pdf of the new canonical variables . Note that because in general , the kinetic energy K is now a function of not only the momenta P, but also of the non-Cartesian coordinates Q: . Like any joint pdf, (30) may be factorized by means of the product rule as
is the marginal pdf of the coordinates Q, and
is the conditional pdf of P given Q. Using (30), (28) and (10), it can be verified that the marginal pdf (32) equals the spatial probability density (18) that was introduced in Sec. 2 under the same symbol:
Here, the transformation was performed, which reverts the kinetic energy to being a function of the Cartesian momenta p only, , since this transformation is the inverse of that which made the kinetic energy a function of both P and Q. (The transformation has an inverse that is such that is in fact a function of p only.) Since [see Equation (7)], the last line of (34) indeed yields the spatial pdf (18).
Using (31) and the fact that, like any conditional pdf, the distribution (33) is properly normalized (i.e., at any Q), the full entropy (29) can be written as
is the mean (expectation) value with respect to Q of the Shannon entropy of the conditional pdf ,
and is the Shannon entropy (21) of . In analogy with (12), Equation (35) in turn can be written as
may be termed the momentum entropy associated with the new momenta P, and
is the spatial entropy in the new spatial coordinates Q. Equations (12) and (38), together with (23), straightforwardly yield the transformation of the momentum entropy:
This relation complements the transformation (23) of the spatial entropy so that, on a point transformation , a change in the spatial entropy is compensated by a change in the momentum entropy and the full entropy remains invariant: .
5. Total Entropy in Terms of the Spatial Entropy in Non-Cartesian Coordinates
It is now easy to show how the change in the spatial entropy of a molecular system, calculated using a non-Cartesian system of coordinates, can be corrected to provide the corresponding change in the full thermodynamic entropy of the system, which is the quantity of physical interest. We consider the entropy change arising from some physicochemical process which changes the phase-space pdf of the system from ρ to . The molecular species of interest is considered to be present at a standard concentration , which corresponds to an isolated molecule in a container of volume . For an isothermal process, and with Cartesian coordinates, the momentum pdf is unchanged, so that there is no change in the momentum entropy, . The change of the full thermodynamic entropy therefore equals the change in the Cartesian spatial entropy :
If non-Cartesian coordinates Q are used, as in many methods for calculating the spatial entropy, a change in the entropy of the coordinates Q obtained in this way can be corrected to the change in the Cartesian spatial entropy using (23):
Here, the correction term
is the difference between the means, evaluated with the changed and original spatial distributions and , respectively, of the logarithm of the Jacobian of the transformation . If molecular simulations are used for the evaluation of spatial entropy, then is evaluated easily as a simple arithmetic mean,
where is a sample of the coordinates obtained from snapshots of the simulation trajectory.
We now consider the specific case where Q represents bond-angle-torsion (BAT) coordinates. For N atoms, Q comprises 3 external translational coordinates , 3 external rotational coordinates , and internal coordinates of bond lengths , bond angles , and torsional angles , where the subscripts indicate the atoms to which the internal coordinates correspond. The Jacobian for this transformation is given as 
and the spatial pdf in Cartesian coordinates , , is thus transformed into the following pdf in BAT coordinates:
where we have used expression (9) for , along with the fact that the potential energy of a molecule of N atoms that is not located in an external field depends only on its internal BAT coordinates . The right-hand side of (47) may be written as a product of two factors, one depending on only the external coordinate and the other on only the internal coordinates . Therefore, the joint pdf (47) can be factorized as
Here, , the marginal pdf of the external coordinates, is clearly normalized because
and , the marginal pdf of the internal BAT coordinates, is normalized based on the definition of [Equation(48)].
The entropy of the joint pdf (49) now separates as
where and are the entropies of the marginal pdf’s and , respectively. Using expression (50) for , the external entropy can be written as
where is the mean of for a uniform distribution of molecular orientations. Finally, using Equations (23), (46), (53) and (54), we have for the spatial entropy in Cartesian coordinates, :
In previous work , has been written in terms of the BAT coordinates as
where is a distribution function of the internal BAT coordinates that becomes the normalized marginal pdf on multiplication by ,
Expression (56) is entirely consistent with the presented formalism, as it can be rewritten in terms of the properly normalized marginal pdf and to give
which is identical with (55). The results of  and other work that adopted a similar evaluation of as that in (56) are thus not affected by our findings.
The approaches outlined above for evaluating the spatial entropy in Cartesian coordinates in BAT coordinates avoid potential shortcomings that may arise from approximating as a constant equal to its value at the equilibrium values and of the internal BAT coordinates. In particular, although most of these coordinates are “hard” and therefore make nearly constant contributions to the Jacobian, this is by no means the case for the pseudo-bond and pseudo-angles often used to define the position and orientation of one molecule relative to another in a noncovalent complex .
One circumstance in which the Jacobian can be approximated as constant is that in which the molecule occupies only a single, reasonably narrow energy well with its local minimum at internal coordinates . One may then make the approximation , in which case the pdf of Equation (51) simplifies to
If, as a further approximation, the harmonic approximation is used for the potential energy , this pdf becomes a multivariate normal (Gaussian) distribution, the entropy of which can be evaluated in closed form, yielding for the entropy of internal coordinates the following estimate:
where is the Hessian matrix of the harmonic potential energy at the energy minimum. The widely used quasiharmonic approximation for estimation of the configurational entropy from molecular simulations was based in its original formulation  on the assumption that , where Σ is the covariance matrix of a simulation sample of internal coordinates. Using now Equations (55) and (60), the spatial entropy in Cartesian coordinates is obtained as
With these approximations, an isothermal process that changes the conformation of the system produces a change in the spatial entropy given by
where the primed quantities pertain to the changed system. Clearly, the Jacobian-dependent term may be neglected only when the equilibrium Jacobians of the two conformations are approximately the same. As noted above, this approximation holds to good accuracy when and
Another perspective on this condition is also of interest. We first note that , where is the total kinetic-energy matrix in the BAT coordinates, and are the masses of the atoms; , like , is a function of the BAT coordinates. This expression follows from the fact that , where and is a matrix whose elements are the partial derivatives of all the BAT coordinates with respect to the Cartesian coordinates, so that (see, e.g., ). Using this identity, along with, e.g., Equations (2.34) and (3.5) of , one may show that
where is the matrix of the instantaneous inertia tensor and is the kinetic-energy matrix of the internal degrees of freedom. Note that , where are the principal moments of inertia. The Jacobian term in (62) therefore may be written as
We have addressed the situation in which one wishes to compute for a classically treated, isothermal physical process where the phase-space probability distribution goes to . In the biophysical context, this might be a binding or folding process. If are Cartesian, then we can factorize as and accordingly decompose the entropy S into . The momentum entropy, , is not affected by the physical process, so . (If the temperature T changes, then there is a contribution from as well, which can be computed analytically.) In practical applications, it is often preferable to compute spatial entropy in non-Cartesian coordinates, Q, but questions arise regarding the correct way to treat the entropy under a coordinate transformation because the differential Shannon entropy of information theory, which has the same mathematical form as the thermodynamic entropy, is not invariant under a change of coordinates. This lack of invariance appears problematic, because a simple change of coordinates must not affect the change in the entropy computed for a physical process.
This paradox is reconciled when one recognizes that the thermodynamic spatial entropy does in fact transform in the same manner as the differential Shannon entropy, but that the change in the transformed spatial entropy, , is not in general equal to the change in total entropy, . The reason is that, in the new coordinates, unlike in Cartesians, the physical process also produces a change in the entropy associated with the conjugate momenta, where is defined as an average of the momentum entropy over all values of the spatial coordinates. This change in the transformed momentum entropy precisely cancels the change in the spatial entropy associated with the transformation of coordinates, so that the change in total entropy due to the physical process is invariant under the transformation of coordinates.
The present analysis furthermore has provided useful expressions for the total entropy change for an isothermal physical process in terms of the spatial entropy in any set of spatial coordinates.
The authors thank Dan S. Sharp for many helpful discussions, and the anonymous referees for their insightful comments. This research was supported by grant no. 212-2006-M-16936 from the National Institute for Occupational Safety and Health to MKG and grant no. GM 61300 from the National Institute of General Medical Sciences to MKG. The findings of this report are solely those of the authors and do not necessarily represent the views of the National Institute for Occupational Safety and Health or the National Institutes of Health.
Campbell, J. Grammatical Man; Information, Entropy, Language & Life; Simon and Schuster: New York, NY, USA, 1982. [Google Scholar]
Cover, T.M.; Thomas, J.A. Elements of Information Theory; Wiley: New York, NY, USA, 1991. [Google Scholar]