# Scaling Behaviour and Critical Phase Transitions in Integrated Information Theory

^{1}

^{2}

## Abstract

**:**

## 1. Introduction

## 2. Model

#### 2.1. IIT 3.0

#### Working Assumptions

#### 2.2. Kinetic Ising Model with Homogeneous Regions

#### 2.3. Integrated Information in the Kinetic Ising Model with Homogeneous Regions

## 3. Results

#### 3.1. Dynamics and Temporal Span of Integrated Information

#### 3.2. Integrated Information of the Cause Repertoire

#### 3.3. Divergence of Integrated Information: Wasserstein and Kullback–Leibler Distance Measures

#### 3.4. Situatedness: Effect of the Environment of a System

#### 3.5. System-Level or Mechanism-Level Integration: Big Phi versus Small Phi

#### 3.6. Values versus Tendencies of Integration

## 4. Discussion

## Funding

## Acknowledgments

## Conflicts of Interest

## Appendix A. List of Assumptions and Experiments

Assumptions | Description |
---|---|

Homogeneous connectivity | In order to simplify the computation of probability distributions and the MIP in the thermodynamic limit, we assume that the system is divided into a number of homogeneous regions. All units within a region share the same inter/intra-region coupling values. |

Equal mechanism and purview | To simplify calculations, we assume that the purview of the system is equal to the mechanism. In contrast, IIT 3.0 selects the purview that yields maximum integration $\phi $. |

MIP cuts either a single node or entire regions | In the thermodynamic limit, when all couplings are positive, the MIP of a homogeneous Ising model is either a partition that cuts a single node from the mechanism or one that separates an entire region (see Appendix B.3). We assume that the same applies to finite systems. |

Initial noise injection | When transition probability matrices describe several updates, IIT 3.0 assumes that partitions only inject noise in the initial state. |

Continuous noise injection | In contrast with IIT 3.0, in some cases we assume that partitions inject noise at every update of the system. |

Independent prior | In order to compute the cause repertoire of a mechanism, IIT 3.0 assumes a uniform prior distribution to apply Bayes rule (Equation (17)) |

Stationary prior | Alternative, in some cases we assume a stationary prior to compute cause repertoires of a mechanism (Section 3.2). |

Wasserstein distance | In IIT 3.0, distances between distributions are computed using the Wasserstein distance. |

Kullback–Leibler divergence | Many other alternative measures of integration (including previous versions of IIT) are based on the Kullback–Leibler divergence. |

**Table A2.**List of assumptions considered by IIT 3.0 and by the results of the different experiments in the article.

Assumptions & Experiments | IIT 3.0 | Figure 3A | Figure 3B | Figure 4A | Figure 4B | Figure 5A | Figure 5B,C | Figure 6A and Figure 8 | Figure 6B,C | Figure 7A,B |
---|---|---|---|---|---|---|---|---|---|---|

Homogeneous connectivity | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | |

Equal mechanism and purview | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | |

MIP is a single node or entire regions | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | |

Initial noise injection | ✓ | ✓ | ||||||||

Continuous noise injection | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ||

Independent prior | ✓ | ✓ | ||||||||

Stationary prior | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ||||

Wasserstein distance | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | |

KL divergence | ✓ | |||||||||

Environment decoupling | ✓ | ✓ | ||||||||

Environment coupling | ✓ | ✓ |

## Appendix B. Wasserstein Distance

#### Appendix B.1. Finite Size

#### Appendix B.2. Infinite Size

#### Appendix B.3. Minimum Information Partition in the Thermodynamic Limit

#### Appendix B.4. Minimum Information Partition in the Thermodynamic Limit for Computing Φ

## Appendix C. Kullback–Leibler Divergence

#### Appendix C.1. Finite Size

#### Appendix C.2. Infinite Size

#### Appendix C.3. Minimum Information Partition in the Thermodynamic Limit for the Kullback–Leibler Divergence

## References

- Oizumi, M.; Albantakis, L.; Tononi, G. From the phenomenology to the mechanisms of consciousness: Integrated information theory 3.0. PLoS Comput. Biol.
**2014**, 10, e1003588. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Marshall, W.; Kim, H.; Walker, S.I.; Tononi, G.; Albantakis, L. How causal analysis can reveal autonomy in models of biological systems. Phil. Trans. R. Soc. A
**2017**, 375, 20160358. [Google Scholar] [CrossRef] [PubMed] - Miyahara, K.; Witkowski, O. The integrated structure of consciousness: Phenomenal content, subjective attitude, and noetic complex. Phenom. Cogn. Sci.
**2019**, 18, 731–758. [Google Scholar] [CrossRef] [Green Version] - Cerullo, M.A. The Problem with Phi: A Critique of Integrated Information Theory. PLoS Comput. Biol.
**2015**, 11. [Google Scholar] [CrossRef] [Green Version] - Barrett, A.B.; Mediano, P.A. The Phi measure of integrated information is not well-defined for general physical systems. J. Conscious. Stud.
**2019**, 26, 11–20. [Google Scholar] - Mediano, P.A.M.; Seth, A.K.; Barrett, A.B. Measuring Integrated Information: Comparison of Candidate Measures in Theory and Simulation. Entropy
**2019**, 21, 17. [Google Scholar] [CrossRef] [Green Version] - Salinas, S.R.A. The Ising Model. In Introduction to Statistical Physics; Salinas, S.R.A., Ed.; Graduate Texts in Contemporary Physics; Springer: New York, NY, USA, 2001; pp. 257–276. [Google Scholar] [CrossRef]
- Salinas, S.R.A. Scaling Theories and the Renormalization Group. In Introduction to Statistical Physics; Springer: New York, NY, USA, 2001; pp. 277–304. [Google Scholar]
- Beggs, J.M. The criticality hypothesis: How local cortical networks might optimize information processing. Philos. Trans. R. Soc. A
**2007**, 366, 329–343. [Google Scholar] [CrossRef] - Barrett, A.B.; Seth, A.K. Practical Measures of Integrated Information for Time-Series Data. PLoS Comput. Biol.
**2011**, 7, e1001052. [Google Scholar] [CrossRef] [Green Version] - Oizumi, M.; Amari, S.i.; Yanagawa, T.; Fujii, N.; Tsuchiya, N. Measuring Integrated Information from the Decoding Perspective. PLoS Comput. Biol.
**2016**, 12, e1004654. [Google Scholar] [CrossRef] - Hoel, E.P.; Albantakis, L.; Marshall, W.; Tononi, G. Can the macro beat the micro? Integrated information across spatiotemporal scales. Neurosci. Conscious.
**2016**, 2016. [Google Scholar] [CrossRef] [Green Version] - Kochmański, M.; Paszkiewicz, T.; Wolski, S. Curie-Weiss magnet: A simple model of phase transition. Eur. J. Phys.
**2013**, 34, 1555–1573. [Google Scholar] [CrossRef] [Green Version] - Aguilera, M.; Di Paolo, E. Integrated information in the thermodynamic limit. Neural Netw.
**2019**. [Google Scholar] [CrossRef] [PubMed] - Pressé, S.; Ghosh, K.; Lee, J.; Dill, K.A. Principles of maximum entropy and maximum caliber in statistical physics. Rev. Mod. Phys.
**2013**, 85, 1115–1141. [Google Scholar] [CrossRef] [Green Version] - Tegmark, M. Consciousness as a state of matter. Chaos Soliton. Fract.
**2015**, 76, 238–270. [Google Scholar] [CrossRef] [Green Version] - Marshall, W.; Albantakis, L.; Tononi, G. Black-boxing and cause-effect power. PLoS Comput. Biol.
**2018**, 14, e1006114. [Google Scholar] [CrossRef] [Green Version] - Chiel, H.J.; Beer, R.D. The brain has a body: Adaptive behavior emerges from interactions of nervous system, body and environment. Trends Neurosci.
**1997**, 20, 553–557. [Google Scholar] [CrossRef] - Clark, A. The Dynamical Challenge. Cogn. Sci.
**1997**, 21, 461–481. [Google Scholar] [CrossRef] - Aguilera, M.; Bedia, M.G.; Santos, B.A.; Barandiaran, X.E. The situated HKB model: How sensorimotor spatial coupling can alter oscillatory brain dynamics. Front. Comput. Neurosci.
**2013**, 7. [Google Scholar] [CrossRef] [Green Version] - Di Paolo, E.; Buhrmann, T.; Barandiaran, X. Sensorimotor Life: An Enactive Proposal; Oxford University Press: Oxford, UK, 2017. [Google Scholar]
- Di Paolo, E.A. Autopoiesis, Adaptivity, Teleology, Agency. Phenomenol. Cogn. Sci.
**2005**, 4, 429–452. [Google Scholar] [CrossRef] - Albantakis, L.; Hintze, A.; Koch, C.; Adami, C.; Tononi, G. Evolution of integrated causal structures in animats exposed to environments of increasing complexity. PLOS Comput. Biol.
**2014**, 10, e1003966. [Google Scholar] [CrossRef] - Tononi, G.; Boly, M.; Massimini, M.; Koch, C. Integrated information theory: From consciousness to its physical substrate. Nat. Rev. Neurosci.
**2016**, 17, 450–461. [Google Scholar] [CrossRef] [PubMed] - Thompson, E.; Varela, F.J. Radical embodiment: Neural dynamics and consciousness. Trends Cogn. Sci.
**2001**, 5, 418–425. [Google Scholar] [CrossRef] - Albantakis, L.; Tononi, G. Causal Composition: Structural Differences among Dynamically Equivalent Systems. Entropy
**2019**, 21, 989. [Google Scholar] [CrossRef] [Green Version]

**Figure 1.**(

**A**) Description of the infinite size kinetic Ising model. (

**B**) Description of the partition schema used to define perturbations. Partitioned connections (black arrows) are injected with random noise. Nonpartitioned connections operate normally or are independent sources of noise (see Section 3.4).

**Figure 2.**Description of the behaviour of the homogeneous Ising model with one region and coupling J, showing a critical point at $J=1$. (

**A**) Values of mean firing rate m for the stationary solution of the kinetic Ising model with one homogeneous region. (

**B**) Value of $\frac{\partial m}{\partial J}$ for the positive stationary solution of the kinetic Ising model with one homogeneous region, diverging at the critical point.

**Figure 3.**Integration of the effect repertoire ${\phi}_{\mathrm{effect}}\left(\tau \right)$ of the largest mechanism of a homogeneous Ising model with one region of size $N=256$ and couplings J with different temporal spans $\tau $, assuming (

**A**) initial injection of noise and (

**B**) continuous injection of noise. Note that $\tau =1$, in both cases ${\phi}_{\mathrm{effect}}$, has the same value.

**Figure 4.**Integration of the cause repertoire ${\phi}_{\mathrm{cause}}\left(\tau \right)$ of the largest mechanism of a homogeneous Ising model with one region of size $N=256$ and couplings J with different temporal spans $\tau $, assuming (

**A**) an independent prior and (

**B**) the stationary distribution as a prior. Continuous noise injection is assumed.

**Figure 5.**Integrated information $\phi \left(\tau \right)$ for the cause (black lines) and effect (grey lines) repertoires of the largest mechanism of a homogeneous kinetic Ising models with one region of size N (and infinite size when $N\to \infty $) and coupling J using (

**A**) the Wasserstein distance. (

**B**) The Kullback–Leibler divergence, and (

**C**) values of $\phi N$ using the Kullback–Leibler divergence. Note that in all cases $\phi \left(\tau \right)={\phi}_{\mathrm{cause}}\left(\tau \right)$. All cases are computed with $\tau =10{log}_{2}N$ for finite systems and $\tau \to \infty $ for infinite systems. Continuous noise injection and a stationary prior are assumed.

**Figure 6.**Effects of the environment in integrated information. Integrated information ${\phi}_{\mathcal{M}}\left(\tau \right)$ (black lines) of a mechanism $\mathcal{M}$ of size $\frac{3N}{4}$ of a homogeneous kinetic Ising model with one region of size N and coupling J, assuming that elements outside of the mechanism operate (

**A**) normally, (

**B**) as independent sources of noise and (

**C**) as static input fields. Values of ${\phi}_{\mathcal{M}}\left(\tau \right)$ are compared with ${\phi}_{\mathcal{M},\mathrm{effect}}(\tau \to \infty )$ (grey line) to show diverging tendencies of the effect repertoire. Note that tendencies of ${\phi}_{\mathcal{M},\mathrm{effect}}(\tau \to \infty )$ are larger than values of ${\phi}_{\mathcal{M}}\left(\tau \right)$, as the effect repertoire tends to show larger values. Values of $\phi $ are computed with $\tau =10{log}_{2}N$ for finite systems and $\tau \to \infty $ for infinite systems. Continuous noise injection and a stationary prior are assumed.

**Figure 7.**Mechanims and system-level integration in a homogeneous system with one region of size N and coupling J. Values of (

**A**) $\phi $ of the largest mechanism and (

**B**) values of $\mathrm{\Phi}$ for the whole system. Measures with $\tau =10{log}_{2}N$, assuming continuous noise injection, stationary priors and environment coupling.

**Figure 8.**Integrated information in a system coupled to an environment. (

**A**) Structure of couplings between the two regions $\mathcal{A},\mathcal{B}$ of size ${N}_{\mathcal{A}}={N}_{\mathcal{E}}=\frac{N}{2}$ of a homogeneous kinetic Ising models with couplings ${J}_{\mathcal{AA}}={J}_{R},{J}_{\mathcal{EE}}=0,{J}_{\mathcal{AE}}={J}_{C},{J}_{\mathcal{EA}}=2{J}_{C}$. (

**B**–

**E**) Integrated information of the mechanism $\mathcal{A}$, ${\phi}_{\mathcal{A}}$ and mechanism $\mathcal{AE}$, ${\phi}_{\mathcal{AE}}$, for values of ${J}_{R}=1$ and ${J}_{C}=0.8$ and ${J}_{C}=1.2$, respectively. Values of $\phi $ are computed for $\tau =10{log}_{2}N$ for finite systems and $\tau \to \infty $ for infinite systems. Continuous noise injection, stationary priors and environment coupling are assumed.

© 2019 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Aguilera, M.
Scaling Behaviour and Critical Phase Transitions in Integrated Information Theory. *Entropy* **2019**, *21*, 1198.
https://doi.org/10.3390/e21121198

**AMA Style**

Aguilera M.
Scaling Behaviour and Critical Phase Transitions in Integrated Information Theory. *Entropy*. 2019; 21(12):1198.
https://doi.org/10.3390/e21121198

**Chicago/Turabian Style**

Aguilera, Miguel.
2019. "Scaling Behaviour and Critical Phase Transitions in Integrated Information Theory" *Entropy* 21, no. 12: 1198.
https://doi.org/10.3390/e21121198