Common Probability Patterns Arise from Simple Invariances

Abstract

Shift and stretch invariance lead to the exponential-Boltzmann probability distribution. Rotational invariance generates the Gaussian distribution. Particular scaling relations transform the canonical exponential and Gaussian patterns into the variety of commonly observed patterns. The scaling relations themselves arise from the fundamental invariances of shift, stretch and rotation, plus a few additional invariances. Prior work described the three fundamental invariances as a consequence of the equilibrium canonical ensemble of statistical mechanics or the Jaynesian maximization of information entropy. By contrast, I emphasize the primacy and sufficiency of invariance alone to explain the commonly observed patterns. Primary invariance naturally creates the array of commonly observed scaling relations and associated probability patterns, whereas the classical approaches derived from statistical mechanics or information theory require special assumptions to derive commonly observed scales.

“It is increasingly clear that the symmetry [invariance] group of nature is the deepest thing that we understand about nature today. I would like to suggest something here that I am not really certain about but which is at least a possibility: that specifying the symmetry group of nature may be all we need to say about the physical world, beyond the principles of quantum mechanics.

The paradigm for symmetries of nature is of course the group symmetries of space and time. These are symmetries that tell you that the laws of nature don’t care about how you orient your laboratory, or where you locate your laboratory, or how you set your clocks or how fast your laboratory is moving.”

(Weinberg [1] (p. 73))

“For the description of processes taking place in nature, one must have a system of reference.”

(Landau and Lifshitz [2] (p. 1))

1. Introduction

I argue that three simple invariances dominate much of observed pattern. First, probability patterns arise from invariance to a shift in scaled measurements. Second, the scaling of measurements satisfies invariance to uniform stretch. Third, commonly observed scales are often invariant to rotation.

Feynman [3] described the shift invariant form of probability patterns as:

$$\frac{q\left(\mathcal{E}\right)}{q\left({\mathcal{E}}^{\prime }\right)}=\frac{q\left(\mathcal{E}+a\right)}{q\left({\mathcal{E}}^{\prime }+a\right)}$$

(1)

in which $q\left(\mathcal{E}\right)$ is the probability associated with a measurement, $\mathcal{E}$. Here, the ratio of probabilities for two different measurements, $\mathcal{E}$ and ${\mathcal{E}}^{\prime }$, is invariant to a shift by a. Feynman derived this invariant ratio as a consequence of Boltzmann’s equilibrium distribution of energy levels, $\mathcal{E}$, that follows from statistical mechanics:

$$q\left(\mathcal{E}\right)=\lambda e^{-\lambda \mathcal{E}}$$

(2)

Here, $\lambda =1/〈\mathcal{E}〉$ is the inverse of the average measurement.

Feynman presented the second equation as primary, arising as the equilibrium from the underlying dynamics of particles and the consequent distribution of energy, $\mathcal{E}$. He then mentioned in a footnote that the first equation of shift invariance follows as a property of equilibrium. However, one could take the first equation of shift invariance as primary. The second equation for the form of the probability distribution then follows as a consequence of shift invariance.

What is primary in the relation between these two equations: equilibrium statistical mechanics or shift invariance? The perspective of statistical mechanics, with Equation (2) as the primary equilibrium outcome, dominates treatises of physics.

Jaynes [4,5] questioned whether statistical mechanics is sufficient to explain why patterns of nature often follow the form of Equation (2). Jaynes emphasized that the same probability pattern often arises in situations for which physical theories of particle dynamics make little sense. In Jaynes’ view, if most patterns in economics, biology and other disciplines follow the same distributional form, then that form must arise from principles that transcend the original physical interpretations of particles, energy and statistical mechanics [6].

Jaynes argued that probability patterns derive from the inevitable tendency for systems to lose information. By that view, the equilibrium form expresses minimum information, or maximum entropy, subject to whatever constraints may act in particular situations. In maximum entropy, the shift invariance of the equilibrium distribution is a consequence of the maximum loss of information under the constraint that total probability is conserved.

Here, I take the view that shift invariance is primary. My argument is that shift invariance and the conservation of total probability lead to the exponential-Boltzmann form of probability distributions, without the need to invoke Boltzmann’s equilibrium statistical mechanics or Jaynes’ maximization of entropy. Those secondary special cases of Boltzmann and Jaynes follow from primary shift invariance and the conservation of probability. The first part of this article develops the primacy of shift invariance.

Once one adopts the primacy of shift invariance, one is faced with the interpretation of the measurement scale, $\mathcal{E}$. We must abandon energy, because we have discarded the primacy of statistical mechanics, and we must abandon Jaynes’ information, because we have assumed that we have only general invariances as our basis.

We can of course end up with notions of energy and information that derive from underlying invariance. However, that leaves open the problem of how to define the canonical scale, $\mathcal{E}$, that sets the frame of reference for measurement.

We must replace the scaling relation $\mathcal{E}$ in the above equations by something that derives from deeper generality: the invariances that define the commonly observed scaling relations.

In essence, we start with an underlying scale for observation, z. We then ask what transformed scale, $z\mapsto {\mathrm{T}}_z\equiv \mathcal{E}$, achieves the requisite shift invariance of probability pattern, arising from the invariance of total probability. It must be that shift transformations, ${\mathrm{T}}_z\mapsto a+{\mathrm{T}}_z$, leave the probability pattern invariant, apart from a constant of proportionality.

Next, we note that a stretch of the scale, ${\mathrm{T}}_z\mapsto b{\mathrm{T}}_z$, also leaves the probability pattern unchanged, because the inverse of the average value in Equation (2) becomes $\lambda =1/b〈{\mathrm{T}}_z〉$, which cancels the stretch in the term $\lambda \mathcal{E}=\lambda {\mathrm{T}}_z$. Thus, the scale ${\mathrm{T}}_z$ has the property that the associated probability pattern is invariant to the affine transformation of shift and stretch, ${\mathrm{T}}_z\mapsto a+b{\mathrm{T}}_z$. That affine invariance generates the symmetry group of scaling relations that determine the commonly observed probability patterns [7,8,9].

The final part of this article develops rotational invariance of conserved partitions. For example, the Pythagorean partition ${\mathrm{T}}_z=x^2\left(s\right)+y^2\left(s\right)$ splits the scaled measurement into components that add invariantly to ${\mathrm{T}}_z$ for any value of s. The invariant quantity defines a circle in the $xy$ plane with a conserved radius $R_z=\sqrt{{\mathrm{T}}_z}$ that is invariant to rotation around the circle, circumscribing a conserved area $\pi R_z^2=\pi {\mathrm{T}}_z$. Rotational invariance allows one to partition a conserved quantity into additive components, which often provides insight into underlying process.

If we can understand these simple shift, stretch and rotational invariances, we will understand much about the intrinsic structure of pattern. An explanation of natural pattern often means an explanation of how particular processes lead to particular forms of invariance.

2. Background

This section introduces basic concepts and notation. I emphasize qualitative aspects rather than detailed mathematics. The final section of this article provides historical background and alternative perspectives.

2.1. Probability Increments

Define $q\left(z\right)\equiv q_z$, such that the probability associated with z is $q_z\Delta {\psi }_z$. This probability is the area of a rectangle with height $q_z$ and incremental width $\Delta {\psi }_z$.

The total probability is constrained to be one, as the sum of the rectangular areas over all values of z, which is $\sum q_z\Delta {\psi }_z=1$. When the z values are discrete quantities or qualitative labels for events, then the incremental measure is sometimes set to one everywhere, $\Delta {\psi }_z\equiv 1$, with changes in the measure $\Delta {\psi }_z$ made implicitly by adjusting $q_z$. The conservation of probability becomes $\sum q_z=1$.

If a quantitative scale z has values that are close together, then the incremental widths are small, $\Delta {\psi }_z\to \mathrm{d}{\psi }_z$, and the distribution becomes essentially continuous in the limit. The probability around each z value is $q_z\mathrm{d}{\psi }_z$. Writing the limiting sum as a integral over z, the conservation of total probability is $\int q_z\mathrm{d}{\psi }_z=1$.

The increments may be constant-sized steps $\mathrm{d}{\psi }_z=\mathrm{d}z$ on the z scale, with probabilities $q_z\mathrm{d}{\psi }_z=q_z\mathrm{d}z$ in each increment. One may transform z in ways that alter the probability expression, $q_z$, or the incremental widths, $\mathrm{d}{\psi }_z$, and study how those changes alter or leave invariant properties associated with the total probability, $q_z\mathrm{d}{\psi }_z$.

2.2. Parametric Scaling Relations

A probability pattern, $q_z\mathrm{d}{\psi }_z$, may be considered as a parametric description of two scaling relations, $q_z$ and ${\psi }_z$, with respect to the parameter z. Geometrically, $q_z\mathrm{d}{\psi }_z$ is a rectangular area defined by the parametric height, $q_z$, with respect to the parameter, z, and the parametric width, $\mathrm{d}{\psi }_z$, with respect to the parameter, z.

We may think of z as a parameter that defines a curve along the path $\left({\psi }_z,q_z\right)$, relating a scaled input measure, ${\psi }_z$, to a scaled output probability, $q_z$. The followings sections describe how different invariances constrain these scaling relations.

3. Shift Invariance and the Exponential Form

I show that shift invariance and the conservation of total probability lead to the exponential form of probability distributions in Equation (2). Thus, we may consider the main conclusions of statistical mechanics and maximum entropy as secondary consequences that follow from the primacy of shift invariance and conserved total probability.

3.1. Conserved Total Probability

This section relates shift invariance to the conservation of total probability. Begin by expressing probability in terms of a transformed scale, $z\mapsto {\mathrm{T}}_z$, such that $q_z=k_{0}f\left({\mathrm{T}}_z\right)$ and:

$$\int q_z\mathrm{d}{\psi }_z=\int k_{0}f\left({\mathrm{T}}_z\right)\mathrm{d}{\psi }_z=1$$

The term $k_0$ is independent of z and adjusts to satisfy the conservation of total probability.

If we assume that the functional form f is invariant to a shift of the transformed scale by a constant, a, then by the conservation of total probability:

$$\int k_{0}f\left({\mathrm{T}}_z\right)\mathrm{d}{\psi }_z=\int k_{a}f({\mathrm{T}}_z+a)\mathrm{d}{\psi }_z=1$$

(3)

The proportionality constant, $k_a$, is independent of z and changes with the magnitude of the shift, a, in order to satisfy the constraint on total probability.

Probability expressions, $q\left(z\right)\equiv q_z$, are generally not shift invariant with respect to the scale, z. However, if our transformed scale, $z\mapsto {\mathrm{T}}_z$ is such that we can write Equation (3) for any magnitude of shift, a, solely by adjusting the constant, $k_a$, then the fact that the conservation of total probability sets the adjustment for $k_a$ means that the condition for ${\mathrm{T}}_z$ to be a shift-invariant canonical scale for probability is:

$$q_z=k_{0}f\left({\mathrm{T}}_z\right)=k_{a}f({\mathrm{T}}_z+a)$$

(4)

which holds over the entire domain of z.

The key point here is that $k_a$ is an adjustable parameter, independent of z, that is set by the conservation of total probability. Thus, the conservation of total probability means that we are only required to consider shift invariance in relation to the proportionality constant $k_a$ that changes with the magnitude of the shift, a, independently of the value of z. Appendix A provides additional detail about the conservation of total probability and the shift-invariant exponential form.

3.2. Shift-Invariant Canonical Coordinates

This section shows the equivalence between shift invariance and the exponential form for probability distributions.

Let $x\equiv {\mathrm{T}}_z$, so that we can write the shift invariance of f in Equation (4) as:

$$f(x+a)={\alpha }_{a}f\left(x\right)$$

By the conservation of total probability, ${\alpha }_a$ depends only on a and is independent of x.

If the invariance holds for any shift, a, then it must hold for an infinitesimal shift, $a=ϵ$. By Taylor series, we can write:

$$f(x+ϵ)=f\left(x\right)+ϵf^{\prime }\left(x\right)={\alpha }_{ϵ}f\left(x\right)$$

Because ϵ is small and independent of x, and ${\alpha }_0=1$, we can write ${\alpha }_{ϵ}=1-\lambda ϵ$ for a constant λ. Then, the previous equation becomes:

$$f^{\prime }\left(x\right)=-\lambda f\left(x\right)$$

This differential equation has the solution:

$$f\left(x\right)=\widehat{k}{e}^{-\lambda x}$$

in which $\widehat{k}$ may be determined by an additional constraint. Using this general property for shift-invariant f in Equation (4), we obtain the classical exponential-Boltzmann form for probability distributions in Equation (2) as:

$$q_z=k{e}^{-\lambda {\mathrm{T}}_z}$$

(5)

with respect to the canonical scale, ${\mathrm{T}}_z$. Thus, expressing observations on the canonical shift-invariant scale, $z\mapsto {\mathrm{T}}_z$, leads to the classical exponential form. If one accepts the primacy of invariance, the “energy,” $\mathcal{E}$, of the Boltzmann form in Equation (2) arises as a particular interpretation of the generalized shift-invariant canonical coordinates, ${\mathrm{T}}_z$.

3.3. Entropy as a Consequence of Shift Invariance

The transformation to obtain the shift-invariant coordinate ${\mathrm{T}}_z$ follows from Equation (5) as:

$$-log{q}_z=\lambda {\mathrm{T}}_z-logk$$

This logarithmic expression of probability leads to various classical definitions of entropy and information [3,10]. Here, the linear relation between the logarithmic scale and the canonical scale follows from the shift invariance of probability with respect to the canonical scale, ${\mathrm{T}}_z$, and the conservation of total probability.

I interpret shift invariance and the conservation of total probability as primary aspects of probability patterns. Entropy and information interpretations follow as secondary consequences.

One can of course derive shift invariance from physical or information theory perspectives. My only point is that such extrinsic concepts are unnecessary. One can begin directly with shift invariance and the conservation of total probability.

3.4. Example: The Gamma Distribution

Many commonly observed patterns follow the gamma probability distribution, which may be written as:

$$q_z=k{z}^{\alpha \lambda }{e}^{-\lambda z}$$

This distribution is not shift invariant with respect to z, because $z\mapsto a+z$ alters the pattern:

$$q_z=k{z}^{\alpha \lambda }{e}^{-\lambda z}\ne k_a{(a+z)}^{\alpha \lambda }{e}^{-\lambda (a+z)}$$

There is no value of $k_a$ for which this expression holds for all z.

If we write the distribution in canonical form:

$$q_z=k{e}^{-\lambda {\mathrm{T}}_z}=k{e}^{-\lambda \left(z-\alpha logz\right)}$$

(6)

then the distribution becomes shift invariant on the canonical scale, ${\mathrm{T}}_z=z-\alpha logz$, because ${\mathrm{T}}_z\mapsto a+{\mathrm{T}}_z$ yields:

$$q_z=k{e}^{-\lambda (a+{\mathrm{T}}_z)}=k_a{e}^{-\lambda {\mathrm{T}}_z}$$

with $k_a=k{e}^{-\lambda a}$. Thus, a shift by a leaves the pattern unchanged apart from an adjustment to the constant of proportionality that is set by the conservation of total probability.

The canonical scale, ${\mathrm{T}}_z=z-\alpha logz$, is log-linear. It is purely logarithmic for small z, purely linear for large z, and transitions between the log and linear domains through a region determined by the parameter α.

The interpretation of process in relation to pattern almost always reduces to understanding the nature of invariance. In this case, shift invariance associates with log-linear scaling. To understand the gamma pattern, one must understand how a process creates a log-linear scaling relation that is shift invariant with respect to probability pattern [7,8,9].

4. Stretch Invariance and Average Values

4.1. Conserved Average Values

Stretch invariance means that multiplying the canonical scale by a constant, ${\mathrm{T}}_z\mapsto b{\mathrm{T}}_z$, does not change probability pattern. This condition for stretch invariance associates with the invariance of the average value.

To begin, note that for the incremental measure $\mathrm{d}{\psi }_z=\mathrm{d}{\mathrm{T}}_z$, the constant in Equation (5) to satisfy the conservation of total probability is $k=\lambda$, because:

$${\int }_0^{\infty }\lambda e^{-\lambda {\mathrm{T}}_z}\mathrm{d}{\mathrm{T}}_z=1$$

when integrating over ${\mathrm{T}}_z$.

Next, define ${〈X〉}_{\psi }$ as the average value of X with respect to the incremental measure $\mathrm{d}{\psi }_z$. Then, the average of $\lambda {\mathrm{T}}_z$ with respect to $\mathrm{d}{\mathrm{T}}_z$ is:

$$\lambda {〈\mathrm{T}〉}_{\mathrm{T}}=\int {\lambda }^2{\mathrm{T}}_z{e}^{-\lambda {\mathrm{T}}_z}\mathrm{d}{\mathrm{T}}_z=1$$

(7)

The parameter λ must satisfy the equality. This invariance of $\lambda {〈\mathrm{T}〉}_{\mathrm{T}}$ implies that any stretch transformation ${\mathrm{T}}_z\mapsto b{\mathrm{T}}_z$ will be canceled by $\lambda \mapsto \lambda /b$. See Appendix A for further details.

We may consider stretch invariance as a primary attribute that leads to the invariance of the average value, $\lambda {〈\mathrm{T}〉}_{\mathrm{T}}$, or we may consider invariance of the average value as a primary attribute that leads to stretch invariance.

4.2. Alternative Measures

Stretch invariance holds with respect to alternative measures, $\mathrm{d}{\psi }_z\ne \mathrm{d}{\mathrm{T}}_z$. Note that for $q_z$ in Equation (5), the conservation of total probability fixes the value of k, because we must have:

$$\int k{e}^{-\lambda {\mathrm{T}}_z}\mathrm{d}{\psi }_z=1$$

The average value of $\lambda {\mathrm{T}}_z$ with respect to $\mathrm{d}{\psi }_z$ is:

$$\int \lambda {\mathrm{T}}_{z}k{e}^{-\lambda {\mathrm{T}}_z}\mathrm{d}{\psi }_z=\lambda {〈\mathrm{T}〉}_{\psi }$$

Here, we do not have any guaranteed value of $\lambda {〈\mathrm{T}〉}_{\psi }$, because it will vary with the choice of the measure $\mathrm{d}{\psi }_z$. If we assume that ${〈\mathrm{T}〉}_{\psi }$ is a conserved quantity, then λ must be chosen to satisfy that constraint, and from the fact that $\lambda {\mathrm{T}}_z$ occurs as a pair, $\lambda {〈\mathrm{T}〉}_{\psi }$ is a conserved quantity. The conservation of $\lambda {〈\mathrm{T}〉}_{\psi }$ leads to stretch invariance, as in the prior section. Equivalently, stretch invariance leads to the conservation of the average value.

4.3. Example: The Gamma Distribution

The gamma distribution from the prior section provides an example. If we transform the base scale by a stretch factor, $z\mapsto bz$, then:

$$q_z=k_b{e}^{-\lambda \left(bz-\alpha logbz\right)}$$

There is no altered value of λ for which this expression leaves $q_z$ invariant over all z. By contrast, if we stretch with respect to the canonical scale, ${\mathrm{T}}_z\mapsto b{\mathrm{T}}_z$, in which ${\mathrm{T}}_z=z-\alpha logz$ for the gamma distribution, we obtain:

$$q_z=k{e}^{-{\lambda }_{b}b{\mathrm{T}}_z}=k{e}^{-\lambda {\mathrm{T}}_z}$$

for ${\lambda }_b=\lambda /b$. Thus, if we assume that the distribution is stretch invariant with respect to $\mathrm{d}z$, then the average value $\lambda {〈\mathrm{T}〉}_z=\lambda 〈z-\alpha logz〉$ is a conserved quantity. Alternatively, if we assume that the average value:

$$\lambda {〈\mathrm{T}〉}_z=\lambda 〈z-\alpha logz〉=\lambda 〈z〉-\lambda \alpha 〈logz〉$$

is a conserved quantity, then stretch invariance of the canonical scale follows.

In this example of the gamma distribution, conservation of the average value with respect to the canonical scale is associated with conservation of a linear combination of the arithmetic mean, $〈z〉$, and the geometric mean, $〈logz〉$, with respect to the underlying values, z. In statistical theory, one would say that the arithmetic and geometric means are sufficient statistics for the gamma distribution.

5. Consequences of Shift and Stretch Invariance

5.1. Relation between Alternative Measures

We can relate alternative measures to the canonical scale by $\mathrm{d}{\mathrm{T}}_z={\mathrm{T}}^{\prime }\mathrm{d}{\psi }_z$, in which ${\mathrm{T}}^{\prime }=|\mathrm{d}{\mathrm{T}}_z/\mathrm{d}\psi |$ is the absolute value of the rate of change of the canonical scale with respect to the alternative scale. Starting with Equation (7) and substituting $\mathrm{d}{\mathrm{T}}_z={\mathrm{T}}^{\prime }\mathrm{d}{\psi }_z$, we have:

$$\lambda {〈\mathrm{T}{\mathrm{T}}^{\prime }〉}_{\psi }=\int {\lambda }^2{\mathrm{T}}_z{\mathrm{T}}^{\prime }{e}^{-\lambda {\mathrm{T}}_z}\mathrm{d}{\psi }_z=1$$

Thus, we recover a universally conserved quantity with respect to any valid alternative measure, $\mathrm{d}{\psi }_z$.

5.2. Entropy

Entropy is defined as the average value of $-log{q}_z$. From the canonical form of $q_z$ in Equation (2), we have:

$$-log{q}_z=\lambda {\mathrm{T}}_z-logk$$

(8)

Average values depend on the incremental measure, $\mathrm{d}{\psi }_z$, so we may write entropy [11] as:

$${〈-log{q}_z〉}_{\psi }={〈\lambda {\mathrm{T}}_z-logk〉}_{\psi }=\lambda {〈\mathrm{T}〉}_{\psi }-log{k}_{\psi }$$

The value of $log{k}_{\psi }$ is set by the conservation of total probability, and λ is set by stretch invariance. The value of ${〈\mathrm{T}〉}_{\psi }$ varies according to the measure $\mathrm{d}{\psi }_z$. Thus, the entropy is simply an expression of the average value of the canonical scale, $\mathrm{T}$, with respect to some incremental measurement scale, ψ, adjusted by a term for the conservation of total probability, k.

When $\psi \equiv \mathrm{T}$, then $k_{\psi }=\lambda$, and we have the classic result for the exponential distribution:

$${〈-log{q}_z〉}_{\mathrm{T}}=\lambda {〈\mathrm{T}〉}_{\mathrm{T}}-log\lambda =1-log\lambda =loge/\lambda$$

in which the conserved value $\lambda {〈\mathrm{T}〉}_{\mathrm{T}}=1$ was given in Equation (7) as a consequence of stretch invariance.

5.3. Cumulative Measure

Shift and stretch invariance lead to an interesting relation between $-log{q}_z$ and the scale at which probability accumulates. From Equation (8), we have:

$$-\frac{1}{\lambda }\mathrm{d}log{q}_z=\mathrm{d}{\mathrm{T}}_z={\mathrm{T}}^{\prime }\mathrm{d}{\psi }_z$$

Multiplying both sides by $q_z$, the accumulation of probability with each increment of the associated measure is:

$$-\frac{1}{\lambda }{q}_z\mathrm{d}log{q}_z=q_z\mathrm{d}{\mathrm{T}}_z=q_z{\mathrm{T}}^{\prime }\mathrm{d}{\psi }_z$$

The logarithmic form for the cumulative measure of probability simplifies to:

$$-\frac{1}{\lambda }{q}_z\mathrm{d}log{q}_z=-\frac{1}{\lambda }\mathrm{d}{q}_z=q_z\mathrm{d}{\mathrm{T}}_z$$

This expression connects the probability weighting, $q_z$, for each incremental measure, to the rate at which probability accumulates in each increment, $\mathrm{d}{q}_z=-\lambda q_z\mathrm{d}{\mathrm{T}}_z$. This special relation follows from the expression for $q_z$ in Equation (2), arising from shift and stretch invariance and the consequent canonical exponential form.

5.4. Affine Invariance and the Common Scales

Probability patterns are invariant to shift and stretch of the canonical scale, ${\mathrm{T}}_z$. Thus, affine transformations ${\mathrm{T}}_z\mapsto a+b{\mathrm{T}}_z$ define a group of related canonical scales. In previous work, we showed that essentially all commonly observed probability patterns arise from a simple affine group of canonical scales [7,8,9]. This section briefly summarizes the concept of affine invariant canonical scales. Appendix B provides some examples.

A canonical scale $\mathrm{T}\left(z\right)\equiv \mathrm{T}$ is affine invariant to a transformation $G\left(z\right)$ if:

$$\mathrm{T}\left[G\left(z\right)\right]=a+b\mathrm{T}\left(z\right)$$

for some constants a and b. We can abbreviate this notion of affine invariance as:

$$\mathrm{T}\circ G\sim \mathrm{T}$$

(9)

in which “∼” means affine invariance in the sense of equivalence for some constants a and b.

We can apply the transformation G to both sides of Equation (9), yielding the new invariance $\mathrm{T}\circ G\circ G\sim \mathrm{T}\circ G$. In general, we can apply the transformation G repeatedly to each side any number of times, so that:

$$\mathrm{T}\circ G^n\sim \mathrm{T}\circ G^m$$

for any nonnegative integers n and m. Repeated application of G generates a group of invariances, a symmetry group. Often, in practical application, the base invariance in Equation (9) does not hold, but asymptotic invariance:

$$\mathrm{T}\circ G^{(n+1)}\sim \mathrm{T}\circ G^n$$

holds for large n. Asymptotic invariance is a key aspect of pattern [12].

6. Rotational Invariance and the Gaussian Radial Measure

The following sections provide a derivation of the Gaussian form and some examples. This section highlights a few results before turning to the derivation.

Rotational invariance transforms the total probability $q_z\mathrm{d}{\mathrm{T}}_z$ from the canonical exponential form into the canonical Gaussian form:

$$\lambda e^{-\lambda {\mathrm{T}}_z}\mathrm{d}{\mathrm{T}}_z\mapsto v{e}^{-\pi v^2{R}_z^2}\mathrm{d}{R}_z$$

(10)

This transformation follows from the substitution $\lambda {\mathrm{T}}_z\mapsto \pi v^2{R}_z^2$, in which the stretch-invariant canonical scale, $\lambda {\mathrm{T}}_z$, becomes the stretch-invariant circular area, $\pi v^2{R}_z^2$, with squared radius $v^2{R}_z^2$. The new incremental scale, $v\mathrm{d}{R}_z$, is the stretch-invariant Gaussian radial measure.

We can, without loss of generality, let $v=1$ and write $\Lambda =\pi R_z^2$ as the area of a circle. Thus, the canonical Gaussian form:

$$q_z\mathrm{d}{\psi }_z=e^{-\Lambda }\mathrm{d}{R}_z$$

(11)

describes the probability, $-log{q}_z=\Lambda$, in terms of the area of a circle, Λ, and the incremental measurement scale, $\mathrm{d}{\psi }_z$, in terms of the radial increments, $\mathrm{d}{R}_z$.

Feynman [3] noted the relation between entropy, radial measure and circular area. In my notation, that relation may be summarized as:

$${〈-log{q}_z〉}_R={〈\Lambda 〉}_R$$

However, Feynman considered the circular expression of entropy as a consequence of the underlying notion of statistical mechanics. Thus, his derivation followed from an underlying canonical ensemble of particles

By contrast, my framework derives from primary underlying invariances. An underlying invariance of rotation leads to the natural Gaussian expression of circular scaling. To understand how rotational invariance leads to the Gaussian form, it is useful to consider a second parametric input dimension, θ, that describes the angle of rotation [13]. Invariance with respect to rotation means that the probability pattern that relates $q(z,\theta )$ to $\psi (z,\theta )$ is invariant to the angle of rotation.

6.1. Gaussian Distribution

I now show that rotational invariance transforms the canonical shift and stretch invariant exponential form into the Gaussian form, as in Equation (10). To begin, express the incremental measure in terms of the Gaussian radial measure as:

$$\lambda \mathrm{d}{\mathrm{T}}_z=\pi v^2\mathrm{d}{R}_z^2=2\pi v^2{R}_z\mathrm{d}{R}_z$$

from which the canonical exponential form $q_z\mathrm{d}{\mathrm{T}}_z=\lambda e^{-\lambda {\mathrm{T}}_z}\mathrm{d}{\mathrm{T}}_z$ may be expressed in terms of the radial measure as:

$$\lambda e^{-\lambda {\mathrm{T}}_z}\mathrm{d}{\mathrm{T}}_z=2\pi v^2{R}_z{e}^{-\pi v^2{R}_z^2}\mathrm{d}{R}_z$$

(12)

Rotational invariance means that for each radial increment, $v\mathrm{d}{R}_z$, the total probability in that increment given in Equation (12) is spread uniformly over the circumference $2\pi v{R}_z$ of the circle at radius $v{R}_z$ from a central location.

Uniformity over the circumference implies that we can define a unit of incremental length along the circumferential path with a fraction $1/2\pi v{R}_z$ of the total probability in the circumferential shell of width $v\mathrm{d}{R}_z$. Thus, the probability along an increment $v\mathrm{d}{R}_z$ of a radial vector follows the Gaussian distribution:

$$\left(1/2\pi v{R}_z\right)q_z\mathrm{d}{\mathrm{T}}_z=v{e}^{-\pi v^2{R}_z^2}\mathrm{d}{R}_z$$

invariantly of the angle of orientation of the radial vector.

Here, the total probability of the original exponential form, $q_z\mathrm{d}{\mathrm{T}}_z$, is spread evenly over the two-dimensional parameter space $\left(z,\theta \right)$ that includes all rotational orientations. The Gaussian expression describes the distribution of probability along each radial vector, in which a vector intersects a constant-sized area of each circumferential shell independently of distance from the origin.

The Gaussian distribution varies over all positive and negative values, $R_z\in \left(-\infty ,\infty \right)$, corresponding to an initial exponential distribution in squared radii, $R_z^2={\mathrm{T}}_z\in \left(0,\infty \right)$. We can think of radial vectors as taking positive or negative values according to their orientation in the upper or lower half planes.

6.2. Radial Shift and Stretch Invariance

The radial value, $R_z$, describes the distance from the central location. Thus, the average radial value is zero, ${〈R〉}_R=0$, when evaluated over all positive and negative radial values. Shift invariance associates with no change in radial distance as the frame of reference shifts the location of the center of the circle to maintain constant radii.

Stretch invariance associates with the conserved value of the average circular area:

$$\lambda {〈\mathrm{T}〉}_R=\pi v^2{〈R^2〉}_R=\pi v^2{\sigma }^2=\frac{1}{2}$$

in which the variance, ${\sigma }^2$, is traditionally defined as the average of the squared deviations from the central location. Here, we have squared radial deviations from the center of the circle averaged over the incremental radial measure, $\mathrm{d}{R}_z$.

When $\lambda =v^2=1$, we have ${\sigma }^2=1/2\pi$, and we obtain the elegant expression of the Gaussian as the relation between circular area and radial increments in Equation (11). This result corresponds to an average circular area of one, because $〈2\pi R^2〉=2\pi {\sigma }^2=1$.

It is common to express the Gaussian in the standard normal form, with ${\sigma }^2=1$, which yields $v^2=1/2\pi$ and the associated probability expression obtained by substituting this value into Equation (10).

6.3. Transforming Distributions to Canonical Gaussian Form

Rotational invariance transforms the canonical exponential form into the Gaussian form, as in Equation (10). If we equate $R_z=\sqrt{{\mathrm{T}}_z}$ and $\lambda =\pi v^2$, we can write the Gaussian form as:

$$q_z\mathrm{d}{R}_z=\sqrt{\frac{\lambda }{\pi }}{e}^{-\lambda {\mathrm{T}}_z}\mathrm{d}\sqrt{{\mathrm{T}}_z}$$

(13)

in which:

$${\tilde{\sigma }}^2={〈\mathrm{T}〉}_{\sqrt{\mathrm{T}}}$$

is a generalized notion of the variance.

The expression in Equation (13) may require a shift of ${\mathrm{T}}_z$ so that ${\mathrm{T}}_z\in \left(0,\infty \right)$, with associated radial values $R_z=\pm \sqrt{{\mathrm{T}}_z}$. The nature of the required shift is most easily shown by example.

6.4. Example: The Gamma Distribution

The gamma distribution may be expressed as $q_z\mathrm{d}{\psi }_z$ with respect to the parameter z when we set ${\mathrm{T}}_z=z-\alpha logz$ and $\mathrm{d}{\psi }_z=\mathrm{d}z$, yielding:

$$q_z\mathrm{d}z=k{e}^{-\lambda \left(z-\alpha logz\right)}\mathrm{d}z$$

for $z\ge 0$. To transform this expression to the Gaussian radial scale, we must shift ${\mathrm{T}}_z$ so that the corresponding value of $R_z$ describes a monotonically increasing radial distance from a central location.

For the gamma distribution, if we use the shift ${\mathrm{T}}_z\mapsto {\mathrm{T}}_z-\alpha =\left(z-\alpha logz\right)-\alpha$ for $\alpha \ge 0$, then the minimum of ${\mathrm{T}}_z$ and the associated maximum of $q_z$ correspond to $R_z=0$, which is what we need to transform into the Gaussian form. In particular, the parametric plot of the points $\left(\pm R_z,q_z\right)$ with respect to the parameter $z\in \left(0,\infty \right)$ follows the Gaussian pattern.

In addition, the parametric plot of the points $\left({\mathrm{T}}_z,q_z\right)$ follows the exponential-Boltzmann pattern. Thus, we have a parametric description of the probability pattern $q_z$ in terms of three alternative scaling relations for the underlying parameter z: the measure $\mathrm{d}z$ corresponds to the value of z itself and the gamma pattern; the measure $\mathrm{d}{R}_z$ corresponds to the Gaussian radial measure; and the measure $\mathrm{d}{\mathrm{T}}_z$ corresponds to the logarithmic scaling of $q_z$ and the exponential-Boltzmann pattern. Each measure expresses particular invariances of scale.

6.5. Example: The Beta Distribution

A common form of the beta distribution is:

$$q_z\mathrm{d}z=k{z}^{\alpha -1}{(1-z)}^{\beta -1}\mathrm{d}z$$

for $z\in \left(0,1\right)$. We can express this distribution in canonical exponential form $k{e}^{-\lambda {\mathrm{T}}_z}$ by the scaling relation:

$$-\lambda {\mathrm{T}}_z=(\alpha -1)logz+(\beta -1)log(1-z)$$

with $\lambda >0$. For α and β both greater than one, this scaling defines a log-linear-log pattern [8], in the sense that $-\lambda {\mathrm{T}}_z$ scales logarithmically near the endpoints of zero and one and transitions to a linear scaling interiorly near the minimum of ${\mathrm{T}}_z$ at:

$$z^{*}=\frac{\alpha -1}{\alpha +\beta -2}$$

(14)

When $0<\alpha <1$, the minimum (extremum) of ${\mathrm{T}}_z$ is at $z^{*}=0$. For our purposes, it is useful to let $\alpha =\lambda$ for $\lambda >0$ and to assume $\beta >1$.

Define ${\mathrm{T}}^{*}$ as the value of ${\mathrm{T}}_z$ evaluated at $z^{*}$. Thus, ${\mathrm{T}}^{*}$ is the minimum value of ${\mathrm{T}}_z$, and ${\mathrm{T}}_z$ increases monotonically from its minimum. If we shift ${\mathrm{T}}_z$ by its minimum, ${\mathrm{T}}_z\mapsto {\mathrm{T}}_z-{\mathrm{T}}^{*}$, and use the shifted value of ${\mathrm{T}}_z$, we obtain the three standard forms of a distribution in terms of the parameter $z\in (0,1)$, as follows.

The measure $\mathrm{d}z$ and parametric plot $\left(z,q_z\right)$ are the standard beta distribution form; the measure $\mathrm{d}{R}_z$ and parametric plot $\left(\pm R_z,q_z\right)$ are the standard Gaussian form; and the measure $\mathrm{d}{\mathrm{T}}_z$ and parametric plot $\left({\mathrm{T}}_z,q_z\right)$ are the standard exponential-Boltzmann form.

7. Rotational Invariance and Partitions

The Gaussian radial measure often reveals the further underlying invariances that shape pattern. Those invariances appear from the natural way in which the radial measure can be partitioned into additive components.

7.1. Overview

Conserved quantities may arise from an underlying combination of processes. For example, we might know that a conserved quantity, $R^2=x+y$, arises as the sum of two underlying processes with values x and y. We do not know x and y, only that their conserved sum is invariantly equal to $R^2$.

The partition of an invariant quantity into a sum may be interpreted as rotational invariance, because:

$$R^2=x+y={\sqrt{x}}^2+{\sqrt{y}}^2$$

defines a circle with conserved radius R along the positive and negative values of the coordinates $\left(\sqrt{x},\sqrt{y}\right)$. That form of rotational invariance explains much of observed pattern, many of the classical results in probability and dynamics, and the expression of those results in the context of mechanics.

The partition can be extended to a multidimensional sphere of radius R as:

$$R^2=\sum {\sqrt{x_i}}^2$$

(15)

One can think of rotational invariance in two different ways. First, one may start with a variety of different dimensions, with no conservation in any particular dimension. However, the aggregate may satisfy a conserved total that imposes rotational invariance among the components.

Second, every conserved quantity can be partitioned into various additive components. That partition starts with a conserved quantity, and then, by adding dimensions that satisfy the total conservation, one induces a higher dimensional rotational invariance. Thus, every conserved quantity associates with higher-dimensional rotational invariance.

7.2. Rotational Invariance of Conserved Probability

In the probability expression $q_z\mathrm{d}{\psi }_z$, suppose the incremental measure $\mathrm{d}{\psi }_z$ is constant, and we have a finite number of values of z with positive probability. We may write the conserved total probability as ${\sum }_z{q}_z=1$. Then, from Equation (15), we can write the conservation of total probability as a partition of $R^2=1$ confined to the surface of a multidimensional sphere:

$$\sum _z{\sqrt{q_z}}^2=1$$

There is a natural square root spherical coordinate system, $\sqrt{q_z}$, in which to express conserved probability. Square roots of probabilities arise in a variety of fundamental expressions of physics, statistics and probability theory [14,15].

7.3. Partition of the Canonical Scale

The canonical scale equals the square of the Gaussian radial scale, ${\mathrm{T}}_z=R_z^2$. Thus, we can write a two-dimensional partition from Equation (15) as:

$${\mathrm{T}}_z={\sqrt{x_1}}^2+{\sqrt{x_2}}^2$$

Define the two dimensions as:

$$\begin{array}{cc}\hfill \sqrt{x_1}& =w\equiv w(z,s)\hfill \\ \hfill \sqrt{x_2}& =\dot{w}\equiv \dot{w}(z,s)\hfill \end{array}$$

yielding the partition for the canonical scale as:

$${\mathrm{T}}_z=w^2+{\dot{w}}^2$$

(16)

This expression takes the input parameter z and partitions the resulting value of ${\mathrm{T}}_z=R_z^2$ into a circle of radius $R_z$ along the path $\left(w,\dot{w}\right)$ traced by the parameter s.

The radial distance, $R_z$, and associated canonical scale value, ${\mathrm{T}}_z=R_z^2$, are invariant with respect to s. In general, for each dimension we add to a partition of ${\mathrm{T}}_z$, we can create an additional invariance with respect to a new parameter.

7.4. Partition into Location and Rate

A common partition separates the radius into dimensions of location and rate. Define $\dot{w}=\partial w/\partial s$ as the rate of change in the location w with respect to the parameter s. Then, we can use the notational equivalence $H_z\equiv {\mathrm{T}}_z=R_z^2$ to emphasize the relation to a classic expression in physics for a conserved Hamiltonian as:

$$H_z=w^2+{\dot{w}}^2$$

(17)

in which this conserved square of the radial distance is partitioned into the sum of a squared location, $w^2$, and a squared rate of change in location, ${\dot{w}}^2$. The squared rate, or velocity, arises as a geometric consequence of the Pythagorean partitioning of a squared radial distance into squared component dimensions. Many extensions of this Hamiltonian interpretation can be found in standard textbooks of physics.

With the Hamiltonian notation, $H_z\equiv {\mathrm{T}}_z$, our canonical exponential-Boltzmann distribution is:

$$q_z\mathrm{d}{H}_z=\lambda e^{-\lambda H_z}\mathrm{d}{H}_z$$

The value H is often interpreted as energy, with $\mathrm{d}H$ as the Gibbs measure. For the simple circular partition of Equation (17), the total energy is often split into potential, $w^2$, and kinetic, ${\dot{w}}^2$, components.

In this article, I emphasize the underlying invariances and their geometric relations as fundamental. From my perspective, the interpretation of energy and its components are simply one way in which to describe the fundamental invariances.

The Hamiltonian interpretation is, however, particularly useful. It leads to a natural expression of dynamics with respect to underlying invariance. For example, we can partition a probability pattern into its currently observable location and its rate of change:

$$e^{-\lambda H_z}=e^{-\lambda w^2}{e}^{-\lambda {\dot{w}}^2}$$

The first component, $w^2$, may be interpreted as the observable state of the probability pattern at a particular time. The second component, ${\dot{w}}^2$, may be interpreted as the rate of change in the probability pattern. Invariance applies to the combination of location and rate of change, rather than to either component alone. Thus, invariance does not imply equilibrium.

8. Summary of Invariances

Probability patterns, $q_z$, express invariances of shift and stretch with respect to a canonical scale, ${\mathrm{T}}_z$. Those invariances lead to an exponential form:

$$q_z\mathrm{d}{\psi }_z=k{e}^{-\lambda {\mathrm{T}}_z}\mathrm{d}{\psi }_z$$

with respect to various incremental measures, $\mathrm{d}{\psi }_z$. This probability expression may be regarded parametrically with respect to z. The parametric view splits the probability pattern into two scaling relations, $q_z$ and ${\psi }_z$, with respect to z, forming the parametric curve defined by the points $\left({\psi }_z,q_z\right)$.

For the canonical scale, ${\mathrm{T}}_z$, we may consider the sorts of transformations that leave the scale shift and stretch (affine) invariant, $\mathrm{T}\circ G\sim \mathrm{T}$, as in Equation (9). Essentially, all of the canonical scales of common probability patterns [7,8,9] arise from the affine invariance of $\mathrm{T}$ and a few simple types of underlying invariance with respect to z.

For the incremental measure scale, $\mathrm{d}{\psi }_z$, four alternatives highlight different aspects of probability pattern and scale.

The scale $\mathrm{d}z$ leads to the traditional expression of probability pattern, $q_z\mathrm{d}z$, which highlights the invariances that set the canonical scale, ${\mathrm{T}}_z$.

The scale $\mathrm{d}{\mathrm{T}}_z$ leads to the universal exponential-Boltzmann form, $q_z\mathrm{d}{\mathrm{T}}_z$, which highlights the fundamental shift and stretch invariances in relation to the conservation of total probability.

This conservation of total probability may alternatively be described by a cumulative probability measure, $\mathrm{d}{q}_z=-\lambda q_z\mathrm{d}{\mathrm{T}}_z$.

Finally, rotational invariance leads to the Gaussian radial measure, $\mathrm{d}{R}_z$. That radial measure transforms many probability scalings, $q_z$, into Gaussian distributions, $q_z\mathrm{d}{R}_z$.

Invariances typically associate with conserved quantities [16]. For example, the rotational invariance of the Gaussian radial measure is equivalent to the conservation of the average area circumscribed by the radial measure. That average circular area is proportional to the traditional definition of the variance. Thus, rotational invariance and conserved variance are equivalent in the Gaussian form.

The Gaussian radial measure often reveals the further underlying invariances that shape pattern. That insight follows from the natural way in which the radial measure can be partitioned into additive components.

9. The Primacy of Invariance and Symmetry

“It was Einstein who radically changed the way people thought about nature, moving away from the mechanical viewpoint of the nineteenth century toward the elegant contemplation of the underlying symmetry principles of the laws of physics in the twentieth century.”

(Lederman and Hill [17] (p. 153))

The exponential-Boltzmann distribution in Equation (2) provides the basis for statistical mechanics, Jaynesian maximum entropy and my own invariance framework. These approaches derive the exponential form from different assumptions. The underlying assumptions determine how far one may extend the exponential-Boltzmann form toward explaining the variety of commonly observed patterns.

I claim that one must begin solely with the fundamental invariances in order to develop a proper understanding of the full range of common patterns. By contrast, statistical mechanics and Jaynesian maximum entropy begin from particular assumptions that only partially reflect the deeper underlying invariances.

9.1. Statistical Mechanics

Statistical mechanics typically begins with an assumed, unseen ensemble of microscopic particles. Each particle is often regarded as identical in nature to the others. Statistical averages over the underlying microscopic ensemble lead to a macroscopic distribution of measurable quantities. The exponential-Boltzmann distribution is the basic equilibrium macroscopic probability pattern.

In contrast with the mechanical perspective of statistical physics, my approach begins with fundamental underlying invariances (symmetries). Both approaches arrive at roughly the same intermediate point of the exponential-Boltzmann form. That canonical form expresses essentially the same invariances, no matter whether one begins with an underlying mechanical perspective or an underlying invariance perspective.

From my point of view, the underlying mechanical perspective happens to be one particular way in which to uncover the basic invariances that shape pattern. However, the mechanical perspective has limitations associated with the unnecessarily particular assumptions made about the underlying microscopic ensemble.

For example, to derive the log-linear scaling pattern that characterizes the commonly observed gamma distribution in Equation (6), a mechanical perspective must make special assumptions about the interactions between the underlying microscopic particles.

Some may consider the demand for explicit mechanical assumptions about the underlying particles to be a benefit. However, in practice, those explicit assumptions are almost certainly false and, instead, simply serve as a method by which to point in the direction of the deeper underlying invariance that shapes the scaling relations and associated probability patterns.

I prefer to start with the deeper abstract structure shaped by the key invariances. Then, one may consider the variety of different particular mechanical assumptions that lead to the key invariances. Each set of particular assumptions that are consistent with the key invariances define a special case.

There have been many powerful extensions to statistical mechanics in recent years. Examples include generalized entropies based on assumptions about underlying particle mechanics [18], superstatistics as the average over heterogeneous microscopic sets [19] and invariance principles applied to the mechanical aspects of particle interactions [20].

My own invariance and scaling approach subsumes essentially all of those results in a simple and elegant way, and goes much further with regard to providing a systematic understanding of the commonly observed patterns [7,8,9]. However, it remains a matter of opinion whether an underlying mechanical framework based on an explicit microscopic ensemble is better or worse than a more abstract approach based purely on invariances.

9.2. Jaynesian Maximum Entropy

Jaynes [4,5] replaced the old microscopic ensemble of particles and the associated mechanical entropy with a new information entropy. He showed that maximum entropy, in the sense of information rather than particle mechanics, leads to the classic exponential-Boltzmann form. A large literature extends the Jaynesian framework [21]. Axiomatic approaches transcend the original justifications based on intuitive notions of information [22].

Jaynes’ exponential form has a kind of canonical scale, ${\mathrm{T}}_z$. In Jaynes’ approach, one sets the average value over the canonical scale to a fixed value, in our notation a fixed value of ${〈\mathrm{T}〉}_z$. That conserved average value defines a constraint (an invariance) that determines the associated probability pattern [23]. The Jaynesian algorithm is the maximization of entropy, subject to a constraint on the average value of some quantity, ${\mathrm{T}}_z$.

Jaynes struggled to go beyond the standard constraints of the mean or the variance. Those constraints arise from fixing the average values of ${\mathrm{T}}_z=z$ or ${\mathrm{T}}_z=z^2$, which lead to the associated exponential or Gaussian forms. Jaynes did discuss a variety of additional invariances [6] and associated probability patterns. However, he never achieved any systematic understanding of the common invariances and the associated commonly observed patterns and their relations.

I regarded Jaynes’ transcendence of the particle-based microscopic ensemble as a strong move in the right direction. I followed that direction for several years [7,8,9,12]. In my prior work, I developed the intrinsic affine invariance of the canonical scale, ${\mathrm{T}}_z$, with respect to the exponential-Boltzmann distribution of maximum entropy. The recognition of that general affine invariance plus the variety of common invariances of scale [24,25] led to my systematic classification of the common probability patterns and their relationships [7,8,9].

In this article, I have taken the next step by doing away with the Jaynesian maximization of entropy. I replaced that maximization with the fundamental invariances of shift and stretch, from which I obtained the canonical exponential-Boltzmann form.

With the exponential-Boltzmann distribution derived from shift and stretch invariance rather than Jaynesian maximum entropy, I added my prior work on the general affine invariance of the canonical scale and the additional particular invariances that define the common scaling relations and probability patterns. We now have a complete system based purely on invariances.

9.3. Conclusions

Shift and stretch invariance set the exponential-Boltzmann form of probability patterns. Rotational invariance transforms the exponential pattern into the Gaussian pattern. These fundamental forms define the abstract structure of pattern with respect to a canonical scale.

In a particular application, observable pattern arises by the scaling relation between the natural measurements of that application and the canonical scale. The particular scaling relation derives from the universal affine invariance of the canonical scale and from the additional invariances that arise in the particular application.

Together, these invariances define the commonly observed scaling relations and associated probability patterns. The study of pattern often reduces to the study of how particular generative processes set the particular invariances that define scale.

Diverse and seemingly unrelated generative processes may reduce to the same simple invariance and, thus, to the same scaling relation and associated pattern. To test hypotheses about generative process and to understand the diversity of natural pattern, one must understand the central role of invariance. Although that message has been repeated many times, it has yet to be fully deciphered.