Information Geometry Conﬂicts With Independence †

: Information Geometry conﬂicts with the independence that is required for science and for rational inference generally.


Introduction
Information Geometry [1] assigns a geometrical relationship between probability distributions, using the local curvature (Hessian) of the Kullback-Leibler formula H(p; q) = ∑ i p(i) log p(i) q(i) as the covariant geometrical metric tensor [2,3] between q and p.On a n-dimensional manifold p(θ) specified by parameters θ 1 , . . ., θ n , this n×n Riemannian metric g is Geodesic lengths and invariant volumes V follow from (d ) 2 = ∑ g jk dθ j dθ k and dV = det g d n θ.Necessarily, lengths are symmetric (p, q) = (q, p) between source and destination, so cannot be isomorphic to H which is from-to asymmetric.Yet (1) is the only connection which preserves independence of separate distributions, H(x×p ; y×q) = H(x; y) + H(p; q).Specifically, when H is used to assign an optimal p (meaning minimally distorted from q) under constraints, that "maximum entropy" selection also depends on separate optimisation x-from-y unless H has the form (1) [4,5].
It follows that any imposed geometrical connection must introduce interference between supposedly separate distributions.That behaviour is incompatible with the practice of scientific inference, and is confirmed by a counter-example.

Counter-Example
Consider the 2-parameter family of probability distributions [6] Parameters v (location) and w (width) lie between 0 and 1.The function f (Figure 1) is monotonically increasing so that it rises from f (0) at t = v to f (1) at t = v+w (mod 1) before falling back to f (0) at t = v+1 (mod 1).It is positive and normalised to 1 0 f (u)du = 1 so that the p vw (•)'s can be probability distributions on the interval (0,1) -which could model growth and decay in a periodic system.The 2 × 2 information-geometry metric evaluates to where, by construction, AC − B 2 > 0. The total invariant volume is infinite.

One Parameter w
If v had been fixed, p would have been confined to a submanifold p w (•) parameterised by w alone.The information-geometry metric reduces to The invariant volume element follows as where, by construction, C > 0. The total invariant volume is finite.

Comparison of One and Two Parameters
Both shape (( 5) versus ( 8)) and integral (( 6) versus ( 9)) over w differ qualitatively according to whether or not v is held fixed.

Treatment of v influences invariant volumes over w [Geometry]
That is a mathematical fact of information geometry.

Science
For scientific application, (3) defines a wraparound translation-invariant model in which v does not affect w.
Treatment of v should not influence inference about w That is a science requirement.Any observational consequence of information-geometry's invariant volumes would be rejected by the informed scientist.If there were such consequence, then observation of width w could be used to infer something about location v, contrary to the intention of the formulation.

Conclusions
Information geometry is not science.It denies the independence of separate parameters even though such independence is a fundamental requirement of scientific inquiry.The assumption of a geometrical connection between distributions is unnecessary for science and it fails under test.
Information geometry is a self-consistent mathematical structure which (like any other piece of mathematics) may find specialised application within science, but is not fundamental to it.The only fundamental connection is the Kullback-Leibler, which is from-to asymmetric hence not geometric.