^{1}

^{2}

^{*}

^{3}

^{1}

^{2}

This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).

We evaluate the information geometric complexity of entropic motion on low-dimensional Gaussian statistical manifolds in order to quantify how difficult it is to make macroscopic predictions about systems in the presence of limited information. Specifically, we observe that the complexity of such entropic inferences not only depends on the amount of available pieces of information but also on the manner in which such pieces are correlated. Finally, we uncover that, for certain correlational structures, the impossibility of reaching the most favorable configuration from an entropic inference viewpoint seems to lead to an information geometric analog of the well-known frustration effect that occurs in statistical physics.

One of the main efforts in physics is modeling and predicting natural phenomena using relevant information about the system under consideration. Theoretical physics has had a general measure of the uncertainty associated with the behavior of a probabilistic process for more than 100 years: the Shannon entropy [

Along a similar avenue we may set Entropic Dynamics [

This quantity might help uncover connections between microscopic dynamics and experimentally observable macroscopic dynamics which is a fundamental issue in physics [

The investigation of the macroscopic behavior of complex systems in terms of the underlying statistical structure of its microscopic degrees of freedom also reveals effects due to the presence of microcorrelations [

The layout of the article is as follows. In Section 2 we introduce a general statistical model discussing its geometry and describing both its dynamics and information geometry complexity. In Section 3, Gaussian statistical models (up to a trivariate model) are considered. There, we compute the asymptotic temporal behaviors of their IGCs. Finally, in Section 4 we draw our conclusions by outlining our findings and proposing possible further investigations.

Given _{1},…, _{n}^{n}

let us consider a family ^{1},…, ^{m}

where Θ ⊆ ℝ^{m}_{θ}^{n}.

The mapping
_{θ}^{i}^{∞}, we can turn ^{∞}

The values _{1},…, _{n}^{1},…, ^{m}

Let
_{ij}

with _{i}_{ij}dθ^{i}^{j}_{ij}

Given the Riemannian manifold
^{m}

where ^{kl}

The idea of curvature is the fundamental tool to understand the geometry of the manifold

with

The recipe to compute some curvatures at a point
_{Π} swiped out by them inheriting a Riemannian metric from
_{Π} at

For any basis (ξ,

where _{ijkl}dθ^{i}^{j}^{k}^{l}

The sectional curvature is directly related to the topology of the manifold; along this direction the ^{n}

We can consider upon the statistical manifold
^{initial} and ^{final}, respectively. The trajectory can be equally traversed in both directions [

where

Taking into consideration these facts, we single out as suitable indicator of dynamical (temporal) complexity, the information geometric complexity defined as the average dynamical statistical volume [

where

with

where ^{k}^{k}^{k}

Relevant properties, concerning complexity of geodesic paths on curved statistical manifolds, of the quantity (10) compared to the Jacobi vector field are discussed in [

In the following we devote our attention to a Gaussian statistical model _{1},…, _{n}

where _{1}),…,
_{n}_{ij}_{i}X_{j}_{i}_{j}

Hereafter we consider the statistical model given by _{i}σ_{j}

We may further notice that the function _{ij}_{i}_{j}_{i}

and, therefore, the differentiation does not affect variables _{i}

where the exponential denotes the power series over its argument (the differential operator).

We now start to apply the concepts of the previous section to a Gaussian statistical model of ^{2} =
^{2}

Consider the monovariate model with only ^{2}_{1}_{2}, where _{1}, _{2} ∈ ℝ. Hence, the volume of _{i}_{2} > 0. Finally, the asymptotic behavior of the IGC (9) is

This shows that the complexity linearly increases in time meaning that acquiring information about

Consider now the monovariate Gaussian statistical model of _{1} exp [_{2}

Again, to have positive volume we have to assume _{1}, _{2} > 0. Finally, the (asymptotic) IGC (9) becomes

This shows that also in this case the complexity linearly increases in time meaning that acquiring information about

The take home message of the previous cases is that we have to account for both mean

The dimension of the Riemannian manifold

The integration of the above coupled differential equations is non-trivial. We follow the method described in [

where _{0} and _{1} are real constants. Then, using (21), the volume of

Since the last quantity must be positive, we assume _{1} > 0. Finally, employing the above expression into

We can now see a reduction in time of the complexity meaning that acquiring information about both

Hence, comparing

Consider now the Gaussian statistical model _{1}) =
_{2}) = _{1} − ^{2} =
_{2} − ^{2} ^{2}. Furthermore, in this case we have also to take into account the possible presence of (micro) correlations, which appear at the level of macro-states as off-diagonal terms in the covariance matrix. In short, this implies considering the following probability distribution function

where

Thanks to _{11}^{2} + _{22}^{2} with,

The only non trivial Christoffel coefficients (4) are

whose solutions are,

Using (27) in

To have it positive we have to assume _{1} > 0. Finally, employing (28) in (9) leads to the IGC,

with

where “strong” stands for the fully connected lattice underlying the micro-variables. The ratio

While the temporal behavior of the IGC (29) is similar to the IGC in (23), here correlations play a fundamental role. From

Intuitively, this is due to the fact that for anticorrelated variables, an increase in one variable implies a decrease in the other one (different directional change): variables become more distant, thus more distinguishable in the Fisher–Rao information metric sense. Similarly, for positively correlated variables, an increase or decrease in one variable always predicts the same directional change for the second variable: variables do not become more distant, thus more distinguishable in the Fisher–Rao information metric sense. This may lead us to guess that in the presence of anticorrelations, motion on curved statistical manifolds via the Maximum Entropy updating methods becomes less complex.

In this section we consider a Gaussian statistical model _{1}, _{2}, _{3}) distributed according to a trivariate Gaussian, then
_{1}) =
_{2}) =
_{3}) = _{1} − ^{2} =
_{2} − ^{2} =
_{2} − ^{2} ^{2}. Therefore, the space of the parameters of

The manifold

First, we consider the trivariate Gaussian statistical model of _{1}. Then proceeding like in Section 3.2 we have _{11}^{2} + _{22}^{2}, where

where
_{1} ∈ ℝ. We remark that

requiring _{1} > 0 for its positivity. Finally, using (33) in (9) we arrive at the asymptotic behavior of the IGC

Comparing (34) in the presence and in the absence of correlations yields

where “weak” stands for low degree of connection in the lattice underlying the micro-variables Notice that

When the trivariate Gaussian statistical model of _{2}, the condition _{11}^{2} + _{22}^{2}, where

where
_{1} ∈ ℝ. We remark that

We have to set _{1} > 0 for the positivity of the volume (37), and using it in (9) we arrive at the asymptotic behavior of the IGC

Then, comparing (38) in the presence and in the absence of correlations yields

where “mildly weak” stands for a lattice (underlying micro-variables) neither fully connected nor with minimal connection.

This is a function of the argument

Last, we consider the trivariate Gaussian statistical model of the _{3}. In this case, the condition _{11}^{2} + _{22}^{2}, where

where
_{1} ∈ ℝ. We note that

Also in this case we need to assume _{1} > 0 to have positive volume. Finally, substituting

The comparison of (42) in the presence and in the absence of correlations yields

where “strong” stands for a fully connected lattice underlying the (three) micro-variables. We remark the latter ratio is a monotonically increasing function of the argument

The behaviors of

The

Where

Specifically, carrying out entropic inferences on the higher-dimensional manifold in the presence of anti-correlations, that is for

In summary, we considered low dimensional Gaussian statistical models (up to a trivariate model) and have investigated their dynamical (temporal) complexity. This has been quantified by the volume of geodesics for parameters characterizing the probability distribution functions. To the best of our knowledge, there is no

For the sake of completeness, we also point out that the relevance of volumes in quantifying the

In this work we uncover that in order to have a reduction in time of the complexity one has to consider both mean and variance as macro-variables. This leads to different topological structures of the parameter space in (13); in particular, we have to consider at least a 2-dimensional manifold in order to have effects such as a power law decay of the complexity. Hence, the minimal hypothesis in a multivariate Gaussian model consists in considering all mean values equal and all covariances equal. In such a case, however, the complexity shows interesting features depending on the correlation among micro-variables (as summarized in

Specifically, recall that a geometrically frustrated system cannot simultaneously minimize all interactions because of geometric constraints [

In this article, we observe a significant change in the behavior of the information geometric complexity ratios as a function of the correlation coefficient in the trivariate Gaussian statistical models. Specifically, in the fully connected trivariate case, no peak arises and a monotonic behavior in _{peak} ≥ 0. This dramatic disparity of behavior can be ascribed to the fact that when carrying out statistical inferences with positively correlated Gaussian random variables, the maximum entropy favorable scenario is incompatible with these working hypothesis. Thus, the system appears frustrated.

These considerations lead us to conclude that we have uncovered a very interesting information geometric resemblance of the more standard geometric frustration effect in Ising spin models. However, for a conclusive claim of the existence of an information geometric analog of the frustration effect, we feel we have to further deepen our understanding. A forthcoming research project along these lines will be a detailed investigation of both arbitrary triangular and square configurations of correlated Gaussian random variables where we take into consideration both the presence of different intensities and signs of pairwise interactions (_{ij}_{ik}

Domenico Felice and Stefano Mancini acknowledge the financial support of the Future and Emerging Technologies (FET) programme within the Seventh Framework Programme for Research of the European Commission, under the FET-Open grant agreement TOPDRIM, number FP7-ICT-318121.

The authors have equally contributed to the paper. All authors read and approved the final manuscript.

The authors declare no conflict of interest.

Ratio