Central to integrated information theory (IIT) is the postulate that, in order to exist, a system in a state must have cause-effect power, since there is no point in assuming that something exists if nothing can make a difference to it or it does not make a difference to anything. To exist from its own intrinsic perspective, the system moreover must have cause-effect power upon itself. To that end, the system must be comprised of mechanisms, elements that have cause-effect power on the system, alone or in combination.
Our objectives here are to assess whether and how much certain isolated and adaptive discrete dynamical systems exist (have irreducible cause-effect power) from their own intrinsic perspective and to determine how their cause-effect structures relate to their dynamic complexity. With our results we want to shed light on the distinction between the intrinsic perspective of the system itself and the extrinsic perspective of an observer, and highlight key aspects of the IIT formalism, such as the notions of causal selectivity, composition, and irreducibility, and their significance in the analysis of discrete dynamical systems.
2.2. Behavior and Cause-Effect Power of Adapting Animats
Cellular automata are typically considered as isolated systems. In this section, we examine the cause-effect structures of small, adaptive logic-gate systems (“animats”), which are conceptually similar to discrete, deterministic cellular automata. By contrast to typical CA, however, the animats are equipped with two sensor and two motor elements, which allow them to interact with their environment (
Figure 8A). Moreover, the connections between an animat’s elements, as well as the update rules of its four hidden elements and two motors are evolved through mutation and selection within a particular task-environment over several 10,000 generations (
Figure 8B). We demonstrated above, that isolated ECA require a sufficiently complex, integrated cause-effect structure for complex global dynamics. Given sufficiently complex inputs from the environment, the animats, however, can in principle exhibit complex dynamics even if their internal structure is causally trivial and/or reducible (e.g., unconnected COPY gates of the sensors). Consequently, the dynamic behavior and intrinsic cause-effect structures of these non-isolated systems may be dissociated. Nevertheless, in adaptive systems, evolution to an environment with a rich causal structure provides a link between the system’s dynamics and its intrinsic cause-effect structures.
In the following, we review evidence from [
19], which shows that, under constraints on the number of internal elements, environments that require context-sensitivity and memory favor the evolution of integrated systems with rich cause-effect structures. Building on these data, we show that, although the animats are very small systems, their average transient lengths in isolation from the environment tend to correlate with
<ΦMax>, as observed in the isolated ECA systems. Finally, we discuss how the adaptive advantages of integrated animats, such as their higher economy in terms of mechanisms per element, and larger degeneracy in architecture and function, are related to the animats’ behavioral and dynamical repertoire.
An animated example of animat evolution and task simulation can be found on the IIT website [
31]. The task environments the animats were exposed to are variants of “Active Categorical Perception” (ACP) tasks, where moving blocks of different sizes have to be distinguished [
19,
32,
33]. Solving the ACP tasks successfully requires combining different sensory inputs and past experience. Adaptation is measured as an increase in fitness: the percentage of correctly classified blocks (“catch” or “avoid”).
Figure 8.
Fitness,
<#concepts>,
<ΦMax>, and state entropy
H, of animats adapting to four task environments with varying requirements for internal memory. (
A) Schematic of animat in example environment. On each trial, the animat has to recognize the size of the downward moving block and either catch or avoid it. Blocks continuously move downward and either to the right or left, at a speed of one unit per time step (periodic boundary conditions). The animat has two sensors with a space of 1 unit between them and thus a total width of three units. Its two motors can move it one unit to the left or right, respectively; (
B) Animat evolution. Each animat is initialized at generation 0 without connections between elements. Through mutation and fitness selection, the animats develop complex network structures with mechanisms that enable them to solve their task. Animats were let to evolve for 60,000 generations; (
C) Illustration of the four Task environments with increasing difficulty and requirements for internal memory from left to right; (
D) The final fitness achieved by the animats after 60,000 generations corresponds to the task difficulty. The two red traces show data from a subset of Task 3 and Task 4 trials with the same high average fitness as Task 1 and 2. Animats that evolved to the more difficult tasks, particularly Task 4, developed significantly more concepts, higher
<ΦMax> values, and more state entropy
H, than those animats that evolved to Task 1. Shaded areas around curves denote SEM across 50 independent evolutions (LODs); (
E) Scatter plots of all evaluated generations of animats from all 50 LODs of Task 4 illustrating the relation of
<#concepts>,
<ΦMax>, and
H to fitness, and
H to
<ΦMax>. The circle size is proportional to the number of animats with the same pair of values. Red dots denote the final generation of animats from all 50 independent evolutions. Panels A, C, and D were adapted with permission from [
19].
Figure 8.
Fitness,
<#concepts>,
<ΦMax>, and state entropy
H, of animats adapting to four task environments with varying requirements for internal memory. (
A) Schematic of animat in example environment. On each trial, the animat has to recognize the size of the downward moving block and either catch or avoid it. Blocks continuously move downward and either to the right or left, at a speed of one unit per time step (periodic boundary conditions). The animat has two sensors with a space of 1 unit between them and thus a total width of three units. Its two motors can move it one unit to the left or right, respectively; (
B) Animat evolution. Each animat is initialized at generation 0 without connections between elements. Through mutation and fitness selection, the animats develop complex network structures with mechanisms that enable them to solve their task. Animats were let to evolve for 60,000 generations; (
C) Illustration of the four Task environments with increasing difficulty and requirements for internal memory from left to right; (
D) The final fitness achieved by the animats after 60,000 generations corresponds to the task difficulty. The two red traces show data from a subset of Task 3 and Task 4 trials with the same high average fitness as Task 1 and 2. Animats that evolved to the more difficult tasks, particularly Task 4, developed significantly more concepts, higher
<ΦMax> values, and more state entropy
H, than those animats that evolved to Task 1. Shaded areas around curves denote SEM across 50 independent evolutions (LODs); (
E) Scatter plots of all evaluated generations of animats from all 50 LODs of Task 4 illustrating the relation of
<#concepts>,
<ΦMax>, and
H to fitness, and
H to
<ΦMax>. The circle size is proportional to the number of animats with the same pair of values. Red dots denote the final generation of animats from all 50 independent evolutions. Panels A, C, and D were adapted with permission from [
19].

An animat’s behavior is deterministically guided by the sensory stimuli it receives from the environment. An animat sensor turns on if a block is located vertically above it; otherwise it is off. The hidden and motor elements are binary Markov variables, whose value is specified by a deterministic input-output logic. However, an animat’s reaction to a specific sensor configuration is context-dependent, in the sense that it also depends on the current state of the animat’s hidden elements, which can be considered as memories of previous sensor and hidden element configurations. In [
19], we evaluated the cause-effect structure and integrated conceptual information
Φ of animats evolved to four task environments that differed primarily in their requirements for internal memory (
Figure 8C). For each task environment, we simulated 50 independent evolutions with 100 animats at each generation. The probability of an animat to be selected into the next generation was proportional to an exponential measure of the animat’s fitness (roulette wheel selection) [
19,
33]. At the end of each evolution, the line of descent (LOD) of one animat from the final generation was traced back through all generations and the cause-effect structures of its ancestors were evaluated every 512 generations. For a particular animat generation in one LOD, the IIT measures were evaluated across all network states experienced by the animat during the 128 test trials, weighted by their probability of occurrence. As shown in
Figure 8D, adapted from [
19], in the more difficult task environments that required more internal memory to be solved (Task 3 and particularly Task 4), the animats developed overall more concepts and higher
<ΦMax> than in the simpler task environments (Task 1 and 2). This is even more evident when the tasks are compared at the same level of fitness (red lines in
Figure 8D). Note that
<#concepts> shown in
Figure 8 was evaluated across all of the animat’s elements including sensors and motors in order to capture all fitness relevant causal functions, while
ΦMax is the integrated conceptual information of the set of elements that forms the major complex (MC) in an animat’s “brain” (values for the number of MC concepts behave similarly and can be found in [
19]).
As an indicator for the dynamical repertoire (dynamical complexity) of the animats in their respective environments, we measured the state entropy
of the animats’ hidden and motor elements for the different task environments, displayed in the right panel of
Figure 8D. The animats’ state entropy increases with adaptation across generations, and also with task difficulty across the different task environments, similar to the IIT measures. The maximum possible entropy for six binary elements is
H = 6, if all system states have equal probability to occur. Note that the animats are initialized without connections between elements and elements without inputs cannot change their state. During adaptation, the number of connected elements increases, particularly in the more difficult tasks that require more memory. More internal elements mean a greater capacity for memory, entropy, and also a higher number of concepts and integrated conceptual information. In this way, fitness, dynamical complexity, and causal complexity are tied together, particularly if the requirement for internal memory is high, even though, in an arbitrary, non-isolated system, the state entropy
H could be dissociated from the system’s cause-effect structure. This relation is illustrated in
Figure 8E, where the
<#concepts>,
<ΦMax>, and the state entropy
H are plotted against fitness for every animat of all 50 LODs of Task 4. All three measures are positively correlated with fitness (ρ = 0.80/0.79/0.54 Spearman’s rank correlation coefficient for H/<#concepts>/
<ΦMax> with
p < 0.001). Note that animats from the same LOD are related. The red dots in
Figure 8E highlight the final generation of each LOD, which are independent of each other. Taking only the final generation into account, H and <#concepts> still correlate significantly with fitness. However, the correlation for
<ΦMax> is not significant after correcting for multiple comparisons (ρ = 0.63/0.56 for H/<#concepts> with
p < 0.001), since having more
<ΦMax> even at lower fitness levels has no cost for the animats.
In contrast to the state entropy
H, the entropy of the sensor states
HSen is mostly task dependent: during adaptation
HSen increases only slightly for Tasks 3 and 4 and decreases slightly for Tasks 1 and 2 (see Figure S4 of [
19]). The entropy of the motor states
HMot represents the behavioral repertoire (behavioral complexity) of the animats and is included in
H.
HMot increases during adaptation, but asymptotes at similar values (~1.6) for all tasks. This reflects the fact that the behavioral requirements (“catch” and “avoid”) are similar in all task environments (see Figure S4 of [
19]).
More elements allow for a higher capacity for state entropy
H and also higher
<ΦMax>. Nevertheless,
H is also directly related to
<ΦMax>, since the highest level of entropy for a fixed number of elements is achieved if, for each element, the probability to be in state “0” or “1” is balanced. As we saw above for elementary cellular automata, balanced rules that output “0” or “1” with equal probability are more likely to achieve high values of
<ΦMax> (
Figure 5B, λ parameter). This is because mechanisms with balanced cause-effect repertoires have on average higher
φ values and lead to more higher-order concepts, and thus cause-effect structures with higher
<ΦMax>. Likewise, as shown for Task 4 in
Figure 8E, right panel, animats with high
<ΦMax> also have high entropy
H (ρ = 0.66,
p < 0.001; taking only the final generation into account the correlation is still almost significant after correcting for multiple comparisons with ρ = 0.44,
p = 0.053).
In the last section, we noted that for isolated ECA systems, having a certain level of <ΦMax> and <#concepts> is necessary in order to have the potential for complex dynamics, and thus high state entropy. The animats, however, receive sensory inputs that can drive their internal dynamics. Consequently, also animats with modular, mainly feedforward structures (Φ = 0) can have high state entropy H while they are behaving in their world. Keeping the sensory inputs constant, animats converge to steady states or periodic dynamics of small cycle length within at most seven time-steps. The average length of these transients, measured for the final generation of animats of all 50 LODs, tends to correlate with the average <ΦMax> calculated from all states experienced during the 128 test trials especially in the simpler tasks 1 and 2 (ρ = 0.45/0.46/0.43/0.39 Spearman’s rank correlation coefficient for Task 1–4 with p = 0.04/0.03/0.067/0.19 after correcting for multiple comparisons). Interestingly, there is no correlation between the transient length and the animats’ fitness. This is because, in general, high fitness only requires a rich behavioral repertoire while interacting with the world, but not in isolation.
In addition to the state entropy, in [
19] we also assessed how the sensory-motor mutual information (
ISMMI) [
34] and predictive information (
IPred) [
35] of the animats as defined in [
19,
36] evolved during adaptation.
ISMMI measures the differentiation of the observed input-output behavior of the animats’ sensors and motors.
IPred, the mutual information between observed past and future system states, measures the differentiation of the observed internal states of the animats’ hidden and motor elements. Both, high
ISMMI and high
IPred, should be advantageous during adaptation to a complex environment, since they reflect the animats’ behavioral and dynamical repertoire, in particular how deterministically one state leads to another.
IPred in the animats is indeed closely tied to the state entropy: it increases during adaptation with increasing fitness and a higher number of internal elements.
ISMMI, however, may actually decrease during adaptation in the animats, since an increase in internal memory may reduce the correlation between sensors and motors, which are restricted to two each (see Figure S4 of [
19]). Both
ISMMI and
IPred, are correlational measures, which depend on the observed distributions of system states. By contrast, analyzing the cause-effect structure of a system requires system perturbations that reveal the causal properties of the system’s mechanisms under all possible initial states. The cause-effect structure thus takes the entire set of possible circumstances the animat might be exposed to into account and not just those observed in a given setting. As for cellular automata, an animat’s cause-effect structures, evaluated by its
<#concepts> and
<ΦMax> quantify its intrinsic causal complexity and its dynamical
potential.
Under external constraints on the number of available internal elements, having many concepts and high integrated conceptual information
Φ proved advantageous for animats in more complex environments (
Figure 8D and [
19]). While the simpler Tasks 1 and 2 could be solved (100% fitness) by animats with either integrated (
Φ > 0) or modular (
Φ = 0) network architectures, only animats with integrated networks reached high levels of fitness in the more difficult Tasks 3 and particularly Task 4, which required more internal computations and memory [
19]. This is because integrated systems can implement more functions (concepts) for the same number of elements, since they can make use of higher-order concepts—irreducible mechanisms specified by combinations of elements.
When particular concepts are selected for during adaptation, higher-order concepts become available at no extra cost in terms of elements or wiring. This degeneracy in concepts may prove beneficial to respond to novel events and challenges in changing environments. Degeneracy here refers to different structures that perform the same function in a certain context [
37,
38]. Contrary to redundant structures, degenerate structures can diverge in function under different contexts. Animats with integrated networks with many degenerate concepts may already be equipped to master novel situations. In principle, this allows them to adapt faster to unpredicted changes in the environment than animats with modular structures, which first have to expand and rearrange their mechanisms and connectivity [
39].
In the context of changing environments, large behavioral and dynamical repertoires are advantageous not only at the level of individual organisms, but also at the population level. In [
19] we found that the variety of network connectomes, mechanisms, and distinct behaviors was much higher among animats that solved Task 1 and 2 perfectly with integrated network structures (
<ΦMax> > 0, high degeneracy) than among animats with the same perfect fitness, but
<ΦMax> = 0 (low degeneracy)
. In Task 1, for example, integrated solutions were encountered in six out of 50 lines of descent (LODs); modular solutions in seven out of 50 LODs. Nevertheless, analyzing all animats with perfect fitness across all generations and LODs, animats with
<ΦMax> > 0 showed 332 different behavioral strategies, while animats with
<ΦMax> = 0 only produced 44 different behavioral strategies. The reason is that integrated networks are more flexible and allow for neutral mutations that do not lead to a decrease in fitness. By contrast, modular networks showed very little variability once a solution was encountered. Having more potential solutions should give a probabilistic selective advantage to integrated networks, and should also lead to more heterogeneous populations, which provide an additional advantage in the face of environmental change.
Taken together, in causally rich environments that foster memory and sensitivity to context, integrated systems should have an adaptive advantage over modular systems. This is because under naturalistic constraints on time, energy, and substrates, integrated systems can pack more mechanisms for a given number of elements, exhibit higher degeneracy in function and architecture, and demonstrate greater sensitivity to context and adaptability. These prominent features of integrated systems also link intrinsic cause-effect power to behavioral and dynamical complexity at the level of individuals and populations.