# On the Criticality of Adaptive Boolean Network Robots

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Materials and Methods

#### 2.1. Robots Configuration

#### 2.2. Experimental Settings

**Task I**—The first task is a simple navigation with collision avoidance, and it consists of moving as straight as possible, minimizing the number of curves, and at the same time avoiding obstacles that may be encountered. The arena configuration employed for this task is represented in Figure 1. To obtain robots performing this task, we introduced the following objective function, which is typically used in evolutionary robotics [32]:

**Task II**—The second task differs from the first task in the presence of two virtual regions that subdivide the arena, as depicted in Figure 2. Each robot will be assigned to a region in which to navigate and avoid obstacles, avoiding entering the wrong one. The score is calculated as in Equation (1) by introducing a penalty factor $\u03f5$ which takes the value 1 when the robot is in the correct region and $-1$ otherwise. The resulting objective function is the following:

**Task III**—It is a foraging task where robots must pick up a virtual object in the blue zone and bring it to the green zone while avoiding obstacles; see Figure 3. To do this, the robot is provided with a virtual hook controlled by a binary signal: pick up or deposit.

**00**for the neutral zone,

**01**for the green zone and

**10**for the blue zone.

#### 2.3. Adaptive Mechanisms

**In–Out****Mapping**- The internal structure of the Boolean network remains unchanged; what this mechanism changes is, at the end of each epoch, the coupling between BN nodes and robot actuators and sensors. In particular, driven by the objective function that we are going to present later, the robot replaces two input nodes and one output node in a random fashion. This causes the network to be perturbed by sensor values in different regions of the BN and the output for the actuators to be derived from a different combination of nodes.
**Mutation**- At the end of each epoch, the robot changes the internal structure of the Boolean network as follows: 1% of the truth table entries (80 bits, result of networks with 1000 nodes and Boolean functions with 8 entries each) is negated, and 1% of the connections between nodes (30 arcs) are redistributed. Both entries and edges are chosen randomly. The internal structure of the network thus changes, while the set of input and output nodes remains unchanged.
**Hybrid**- The combined effect of the previous techniques.

## 3. Results

#### Comparison of the Adaptive Mechanisms

**Mean of max objective function per****epoch**- To represent the robots’ ability to adapt as the length of the adaptation phase increases, we take for each epoch the maximum score (returned value by the objective function) achieved so far by the robot, taking into account all previous epochs. Subsequently, we calculate the average so as to obtain for each epoch, 500 in total, a point that represents the average of the maximum score obtained so far by all robots. The trend will always be increasing.
**Area under the****curve**- To provide quantitative information about the differential performances of all the combinations of adaptive mechanisms and dynamical regimes, we calculate and rank from highest to lowest the area under the curve of the trend traced by the mean of the maximum objective function along the epochs.
**Box-plot of max objective function****scores**- In this type of statistic, each box-plot is populated by the maximum objective function score of each robot. The maximum score is the highest score that a robot was able to reach in one of the 500 epochs available.
**Mean rank of max****scores**- For this analysis, we sort all the robot’s max scores from highest to lowest and assign a rank. Afterwards, we compute the mean of the rank of every experimental case by grouping them by the mechanism and dynamical regime. For this statistic, the lower, the better.

**nominal**dynamical regime, i.e., the one statistically possessed by the networks generated randomly at the beginning of the adaptation phase, against the

**actual**one, i.e., the operating dynamical regime expressed by the BNs of the best robots. The

**actual dynamical regime**is the result of the particular adaptive pressure exerted by the objective function on the environment–robot system. Indeed, having undergone modifications to the internal structure through the mutation and hybrid mechanisms, Boolean networks belonging to the $p=\{0.1,0.9\}$ configuration that have performance comparable to the critical ones, initially ordered, may have undergone a change to their dynamical regime of operation, which, arguably, could have led them to resemble, dynamically, those in the critical regime. In general, structural modifications to the Boolean networks introduced by the mutation and hybrid adaptive techniques no longer allow us to consider the networks as part of the random Boolean network ensemble from which they were initially sampled, since the bias selected by the adaptation techniques could have undergone substantial changes. So, a precise characterization of the dynamical regime of the networks that have been subjected to adaptation is needed. In this regard, we study their Derrida parameter [34] $\lambda $ at one step.

- We identify the epoch in which the robot obtained the best score;
- We extrapolate the state of the network for all the 800 steps composing the epoch;
- We calculate the Derrida value for each of the 800 states and on all the possible perturbations, $n=100$. Then, an average value of all states belonging to the robot is computed;
- Finally, the average value is used to generate a point on the scatter plot.

**nominal**dynamical regime, we considered as ordered the BNs generated with $p=\{0.1,0.9\}$, critical those with $p=\{0.21,0.79\}$ and chaotic with $p=\left\{0.5\right\}$. Meanwhile, for the

**actual**dynamical regime, we considered them ordered if $D<1-\u03f5$, critical if $D\ge 1-\u03f5$ and $D\le 1+\u03f5$, and, lastly, chaotic if $D>1+\u03f5$, with $\u03f5=0.3$. The results are reported in Table 3 for Task III, while those concerning the dynamical regime shift observed in Tasks I and II are presented in Appendix D, in Table A6 and Table A7. In general, we can note how all the configurations with (initially) ordered networks that are present in the first positions of the “nominal” column (remember that in this analysis, the lower, the better) are superseded by the critical ones when we consider the actual operating dynamical regimes expressed by the BNs that control the robots.

## 4. Discussion

#### 4.1. Remarks on the Criticality of Adaptive BN Robots

- (i)
- Behavior: optimal balance between the system’s repertoire of actions and their reliability with respect to external inputs.
- (ii)
- Evolvability: trade-off between robustness against mutation and phenotypic innovation.

#### 4.2. Criticality and Phenotypic Plasticity

## 5. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## Appendix A. Experimental Parameters

#### Appendix A.1. Simulations Parameters

**Table A1.**In the table, we describe the parameters and their relative values used in the ARGoS simulator to perform the experiments.

Parameter | Value |
---|---|

no. of robots for each run | 10 |

no. of steps per second ($\Delta t$) | 10 |

epoch duration (in seconds) | 80 |

no. of epochs per run | 500 |

maximum wheel speed (in cm/s) | 25 |

no. output nodes controlling robot wheels | 2 |

binarization threshold for proximity sensors values | 0.1 |

no. of input nodes for proximity sensors | 8 |

binarization threshold for light sensors | 0.1 |

no. of input nodes for light sensors | 0 for Task I and II, 8 for Task III |

#### Appendix A.2. Parameters of the Objective Function

Parameter | Description |
---|---|

$E=d/\Delta t=800$ | number of steps in each epoch |

n | actual step |

$\theta \left(n\right)\in [0,1]$ | max proximity value read by sensors step at time n |

$l\left(n\right)\in \{0,1\}$ | left motor power at time n |

$r\left(n\right)\in \{0,1\}$ | right motor power at time n |

#### Appendix A.3. Parameters of the Adaptive Mechanisms

**Table A3.**Parameters used in the case of the application of the mutation and hybrid adaptive schemes.

Mutation and Hybrid Mechanisms | |
---|---|

Parameter | Value |

no. of edges mutation per epoch | 30 |

self-loops allowed after edges mutation | False |

$k=3$ distinct incoming nodes per node after edges mutation | True |

no. of bit flips in Boolean functions per epoch. | 80 |

In–Out Mapping Mechanism | |
---|---|

Parameter | Value |

no. of input nodes to rewire per epoch | 2 |

no. of output nodes to rewire per epoch | 1 |

allow input nodes to be used as output nodes and vice versa | False |

#### Appendix A.4. Random Boolean Networks Parameters

Parameter | Value |
---|---|

n | 1000 |

k | 3 |

p | ∈$\{0.1,0.21,0.5,0.79,0.9\}$ |

self-loops allowed | False |

$k=3$ distinct incoming nodes per node | True |

p of output nodes | 0.5 |

allow input nodes to be used as output nodes and vice versa | False |

## Appendix B. Derrida Against Fitness

**Figure A1.**Scatter plot of Derrida values (x-axis) against the relative objective function scores (y-axis) obtained by the robots in their best epoch. Noting that we used RBNs with $k=3$ as robot controllers, red points represent Boolean networks initially generated using $p=0.9$, while the green dots represent RBNs generated using $p=0.79$, and lastly, blue ones have been sampled from the ensemble of chaotic RBNs, i.e., using $p=0.5$. (

**a**)

**Task I**: Mutation (

**b**)

**Task II**: Mutation (

**c**)

**Task III**: Mutation (

**d**)

**Task I**: Hybrid (

**e**)

**Task II**: Hybrid (

**f**)

**Task III**: Hybrid (

**g**)

**Task I**: In-Out Mapping (

**h**)

**Task II**: In-Out Mapping (

**i**)

**Task III**: In-Out Mapping.

## Appendix C. Statistical Significance Tests

**Figure A2.**Graphical representation of the results of the Wilcoxon test for the distributions presented in Figure 5. Blue cells represent p-value < 0.05 and red ones represent values ≥ 0.05.

## Appendix D. Performance of Nominal vs. Actual Dynamical Regime

**Table A6.**Mean of the ranking of the maximum scores for Task I; in this analysis, the lower, the better. [

**i-o**] stands for in–out mapping mechanism, while [

**h**] stands for hybrid adaptation and [

**m**] stands for mutation scheme. For the

**nominal**dynamical regime, we considered as ordered the BNs generated with $p=\{0.1,0.9\}$, critical those with $p=\{0.21,0.79\}$ and chaotic with $p=\left\{0.5\right\}$. Meanwhile, for the

**actual**dynamical regime, we considered it ordered if $D<1-\u03f5$, critical if $D\ge 1-\u03f5$ and $D\le 1+\u03f5$, and, lastly, chaotic if $D>1+\u03f5$, with $\u03f5=0.3$.

Task I | ||||
---|---|---|---|---|

Nominal | Actual | |||

Rank | Case | $\mathbf{<}\mathit{Rank}\mathbf{>}$ | Case | $\mathbf{<}\mathit{Rank}\mathbf{>}$ |

1 | [i-o] critical | $2592.96$ | [i-o] critical | $2592.96$ |

2 | [h] ordered | 2762 | [h] critical | $3203.58$ |

3 | [i-o] ordered | $3588.44$ | [i-o] ordered | $3588.42$ |

4 | [h] critical | $3709.62$ | [h] ordered | $3862.18$ |

5 | [m] critical | $4922.71$ | [m] critical | $4637.69$ |

6 | [m] ordered | $5207.89$ | [h] chaotic | $6728.48$ |

7 | [h] chaotic | $6758.2$ | [m] chaotic | $6748.2$ |

8 | [m] chaotic | $6978.95$ | [i-o] chaotic | $7078.08$ |

9 | [i-o] chaotic | $7078.09$ | [m] ordered | $8246.23$ |

**Table A7.**Mean of the ranking of the maximum scores for Task II; in this analysis, the lower, the better. [

**i-o**] stands for in–out mapping mechanism, while [

**h**] stands for hybrid adaptation and [

**m**] stands for mutation scheme. For the

**nominal**dynamical regime, we considered as ordered the BNs generated with $p=\{0.1,0.9\}$, critical those with $p=\{0.21,0.79\}$ and chaotic with $p=\left\{0.5\right\}$. Meanwhile, for the

**actual**dynamical regime, we considered them ordered if $D<1-\u03f5$, critical if $D\ge 1-\u03f5$ and $D\le 1+\u03f5$, and, lastly, chaotic if $D>1+\u03f5$, with $\u03f5=0.3$.

Task II | ||||
---|---|---|---|---|

Nominal | Actual | |||

Rank | Case | $\mathbf{<}\mathit{Rank}\mathbf{>}$ | Case | $\mathbf{<}\mathit{Rank}\mathbf{>}$ |

1 | [i-o] critical | $2602.67$ | [i-o] critical | $2602.64$ |

2 | [h] ordered | $3418.02$ | [h] critical | $3333.58$ |

3 | [h] critical | $3560.18$ | [m] critical | $3428.67$ |

4 | [m] critical | $3696.67$ | [m] chaotic | $5850.27$ |

5 | [m] ordered | $4315.74$ | [i-o] ordered | $6063.59$ |

6 | [i-o] ordered | $6063.65$ | [h] ordered | $6081.39$ |

7 | [m] chaotic | $6118.02$ | [h] chaotic | $6324.15$ |

8 | [h] chaotic | $6332.71$ | [i-o] chaotic | $6617.91$ |

9 | [i-o] chaotic | $6617.91$ | [m] ordered | $8054.52$ |

## Appendix E. Dynamical Regime Shift with Random Mutation

**Figure A3.**Expected number of ones in the truth tables along the adaptation epochs according to the model (A1). Blue lines represent RBNs with $k=3$ initially ordered ${N}_{t=0}^{1}\in \{8000\times 0.1,8000\times 0.9\}$, green ones represent those initially critical ${N}_{t=0}^{1}\in \{8000\times 0.21,8000\times 0.79\}$, and eventually, the red line represents chaotic ones ${N}_{t=0}^{1}=8000\times 0.5$.

## References

- Hidalgo, J.; Grilli, J.; Suweis, S.; Muñoz, M.; Banavar, J.; Maritan, A. Information-based fitness and the emergence of criticality in living systems. Proc. Natl. Acad. Sci. USA
**2014**, 111, 10095–10100. [Google Scholar] [CrossRef] [PubMed] - Munoz, M.A. Colloquium: Criticality and dynamical scaling in living systems. Rev. Mod. Phys.
**2018**, 90, 031001. [Google Scholar] [CrossRef] - Roli, A.; Villani, M.; Filisetti, A.; Serra, R. Dynamical criticality: Overview and open questions. J. Syst. Sci. Complex.
**2018**, 31, 647–663. [Google Scholar] [CrossRef] - Balleza, E.; Alvarez-Buylla, E.; Chaos, A.; Kauffman, S.; Shmulevich, I.; Aldana, M. Critical dynamics in genetic regulatory networks: Examples from four kingdoms. PLoS ONE
**2008**, 3, e2456. [Google Scholar] [CrossRef] [PubMed] - Nykter, M.; Price, N.; Aldana, M.; Ramsey, S.; Kauffman, S.; Hood, L.; Yli-Harja, O.; Shmulevich, I. Gene expression dynamics in the macrophage exhibit criticality. Proc. Natl. Acad. Sci. USA
**2008**, 105, 1897–1900. [Google Scholar] [CrossRef] - Daniels, B.C.; Kim, H.; Moore, D.; Zhou, S.; Smith, H.B.; Karas, B.; Kauffman, S.A.; Walker, S.I. Criticality distinguishes the ensemble of biological regulatory networks. Phys. Rev. Lett.
**2018**, 121, 138102. [Google Scholar] [CrossRef] - Villani, M.; La Rocca, L.; Kauffman, S.; Serra, R. Dynamical criticality in gene regulatory networks. Complexity
**2018**, 2018, 14. [Google Scholar] [CrossRef] - Bertschinger, N.; Natschläger, T. Real-Time Computation at the Edge of Chaos in Recurrent Neural Networks. Neural Comput.
**2004**, 16, 1413–1436. [Google Scholar] [CrossRef] - Legenstein, R.; Maass, W. Edge of chaos and prediction of computational performance for neural circuit models. Neural Netw.
**2007**, 20, 323–334. [Google Scholar] [CrossRef] - Boedecker, J.; Obst, O.; Lizier, J.T.; Mayer, N.M.; Asada, M. Information processing in echo state networks at the edge of chaos. Theory Biosci.
**2012**, 131, 205–213. [Google Scholar] [CrossRef] - Kauffman, S. Investigations; Oxford University Press: Oxford, UK, 2000. [Google Scholar]
- Fusco, G.; Minelli, A. Phenotypic plasticity in development and evolution: Facts and concepts. Philos. Trans. R. Soc. B Biol. Sci.
**2010**, 365, 547–556. [Google Scholar] [CrossRef] [PubMed] - Kelly, S.; Panhuis, T.; Stoehr, A. Phenotypic plasticity: Molecular mechanisms and adaptive significance. Compr. Physiol.
**2011**, 2, 1417–1439. [Google Scholar] - Kauffman, S. Metabolic stability and epigenesis in randomly constructed genetic nets. J. Theor. Biol.
**1969**, 22, 437–467. [Google Scholar] [CrossRef] - Huang, S.; Ernberg, I.; Kauffman, S. Cancer attractors: A systems view of tumors from a gene network dynamics and developmental perspective. In Seminars in Cell & Developmental Biology; Elsevier: Amsterdam, The Netherlands, 2009; Volume 20, pp. 869–876. [Google Scholar]
- Huang, S.; Eichler, G.; Bar-Yam, Y.; Ingber, D. Cell Fates as High-Dimensional Attractor States of a Complex Gene Regulatory Network. Phys. Rev. Lett.
**2005**, 94, 128701. [Google Scholar] [CrossRef] [PubMed] - Villani, M.; Barbieri, A.; Serra, R. A dynamical model of genetic networks for cell differentiation. PLoS ONE
**2011**, 6, e17703. [Google Scholar] [CrossRef] [PubMed] - Serra, R.; Villani, M.; Graudenzi, A.; Kauffman, S. Why a simple model of genetic regulatory networks describes the distribution of avalanches in gene expression data. J. Theor. Biol.
**2007**, 246, 449–460. [Google Scholar] [CrossRef] - Montagna, S.; Braccini, M.; Roli, A. The Impact of Self-Loops on Boolean Networks Attractor Landscape and Implications for Cell Differentiation Modelling. IEEE ACM Trans. Comput. Biol. Bioinform.
**2021**, 18, 2702–2713. [Google Scholar] [CrossRef] - Braccini, M.; Roli, A.; Villani, M.; Serra, R. Dynamical properties and path dependence in a gene-network model of cell differentiation. Soft Comput.
**2021**, 25, 6775–6787. [Google Scholar] [CrossRef] - Braccini, M.; Roli, A.; Villani, M.; Montagna, S.; Serra, R. A simplified model of chromatin dynamics drives differentiation process in Boolean models of GRN. In Proceedings of the 2019 Conference on Artificial Life, ALIFE 2019, Online, 29 July–2 August 2019; pp. 211–217. [Google Scholar] [CrossRef]
- Goudarzi, A.; Teuscher, C.; Gulbahce, N.; Rohlf, T. Emergent Criticality through Adaptive Information Processing in Boolean Networks. Phys. Rev. Lett.
**2012**, 108, 128702. [Google Scholar] [CrossRef] - Benedettini, S.; Villani, M.; Roli, A.; Serra, R.; Manfroni, M.; Gagliardi, A.; Pinciroli, C.; Birattari, M. Dynamical regimes and learning properties of evolved Boolean networks. Neurocomputing
**2013**, 99, 111–123. [Google Scholar] [CrossRef] - Echlin, M.; Aguilar, B.; Notarangelo, M.; Gibbs, D.; Shmulevich, I. Flexibility of Boolean Network Reservoir Computers in Approximating Arbitrary Recursive and Non-Recursive Binary Filters. Entropy
**2018**, 20, 954. [Google Scholar] [CrossRef] [PubMed] - Braccini, M.; Roli, A.; Kauffman, S. Online adaptation in robots as biological development provides phenotypic plasticity. arXiv
**2020**, arXiv:2006.02367. [Google Scholar] - Bonani, M.; Longchamp, V.; Magnenat, S.; Rétornaz, P.; Burnier, D.; Roulet, G.; Vaussard, F.; Bleuler, H.; Mondada, F. The marXbot, a miniature mobile robot opening new perspectives for the collective-robotic research. In Proceedings of the 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, Taipei, Taiwan, 18–22 October 2010; pp. 4187–4193. [Google Scholar]
- Roli, A.; Villani, M.; Serra, R.; Benedettini, S.; Pinciroli, C.; Birattari, M. Dynamical properties of artificially evolved Boolean network robots. In AI*IA2015: Advances in artificial intelligence, Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2015; Volume 9336, pp. 45–57. [Google Scholar]
- Roli, A.; Benedettini, S.; Birattari, M.; Pinciroli, C.; Serra, R.; Villani, M. A preliminary study on BN-robots’ dynamics. In Proceedings of the Italian Workshop on Artificial Life and Evolutionary Computation (WIVACE 2012), Parma, Italy, 20–21 February 2012; pp. 1–4. [Google Scholar]
- Roli, A.; Braccini, M. Attractor Landscape: A Bridge between Robotics and Synthetic Biology. Complex Syst.
**2018**, 27, 229–248. [Google Scholar] [CrossRef] - Roli, A.; Manfroni, M.; Pinciroli, C.; Birattari, M. On the design of Boolean network robots. In Applications of Evolutionary Computation; Springer: Berlin/Heidelberg, Germany, 2011; pp. 43–52. [Google Scholar]
- Luque, B.; Solé, R. Phase transitions in random networks: Simple analytic determination of critical points. Phys. Rev. E
**1997**, 55, 257. [Google Scholar] [CrossRef] - Nolfi, S.; Floreano, D. Evolutionary Robotics; The MIT Press: Cambridge, MA, USA, 2000. [Google Scholar]
- Braitenberg, V. Vehicles: Experiments in Synthetic Psychology; MIT Press: Cambridge, MA, USA, 1986. [Google Scholar]
- Bastolla, U.; Parisi, G. A Numerical Study of the Critical Line of Kauffman Networks. J. Theor. Biol.
**1997**, 187, 117–133. [Google Scholar] [CrossRef] [PubMed] - Kauffman, S. The Origins of Order: Self-Organization and Selection in Evolution; Oxford University Press: Oxford, UK, 1993. [Google Scholar]
- Kauffman, S. At Home in the Universe; Oxford University Press: Oxford, UK, 1996. [Google Scholar]
- Shmulevich, I.; Kauffman, S.; Aldana, M. Eukaryotic cells are dynamically ordered or critical but not chaotic. Proc. Natl. Acad. Sci. USA
**2005**, 102, 13439–13444. [Google Scholar] [CrossRef] - Ashby, W. Design for a Brain; Chapman & Hall: London, UK, 1952. [Google Scholar]
- Cariani, P. Some epistemological implications of devices which construct their own sensors and effectors. In Toward a Practice of Autonomous Systems: Proceedings of the First European Conference on Artificial Life, Paris, France, 11–13 December 1991; MIT Press: Cambridge, MA, USA, 1992; pp. 484–493. [Google Scholar]
- Cariani, P. Creating new informational primitives in minds and machines. In Computers and Creativity; Springer: Berlin/Heidelberg, Germany, 2012; pp. 383–417. [Google Scholar]
- Beggs, J. The criticality hypothesis: How local cortical networks might optimize information processing. Phil. Trans. R. Soc. A Math. Phys. Eng. Sci.
**2008**, 366, 329–343. [Google Scholar] [CrossRef] - Beggs, J.; Timme, N. Being critical of criticality in the brain. Front. Physiol.
**2012**, 3, 163. [Google Scholar] [CrossRef] - Chialvo, D. Emergent complex neural dynamics. Nat. Phys.
**2010**, 6, 744–750. [Google Scholar] [CrossRef] - Lizier, J. The Local Information Dynamics of Distributed Computation in Complex Systems; Springer Theses Series; Springer: Berlin/Heidelberg, Germany, 2013. [Google Scholar]
- Ay, N.; Bertschinger, N.; Der, R.; Güttler, F.; Olbrich, E. Predictive information and explorative behavior of autonomous robots. Eur. Phys. J. B
**2008**, 63, 329–339. [Google Scholar] [CrossRef] - Edlund, J.; Chaumont, N.; Hintze, A.; Koch, C.; Tononi, G.; Adami, C. Integrated information increases with fitness in the evolution of animats. PLoS Comput. Biol.
**2011**, 7, e1002236. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Lizier, J.; Prokopenko, M.; Zomaya, A. The Information Dynamics of Phase Transitions in Random Boolean Networks. In Proceedings of the ALIFE, Winchester, UK, 5–8 August 2008; pp. 374–381. [Google Scholar]
- Longo, G. How Future Depends on Past and Rare Events in Systems of Life. Found. Sci.
**2018**, 23, 443–474. [Google Scholar] [CrossRef] - Butz, M.; Wörgötter, F.; van Ooyen, A. Activity-dependent structural plasticity. Brain Res. Rev.
**2009**, 60, 287–305. [Google Scholar] [CrossRef] [PubMed] - Kriegman, S.; Blackiston, D.; Levin, M.; Bongard, J. A scalable pipeline for designing reconfigurable organisms. Proc. Natl. Acad. Sci. USA
**2020**, 117, 1853–1859. [Google Scholar] [CrossRef] [PubMed] - Pinciroli, C.; Trianni, V.; O’Grady, R.; Pini, G.; Brutschy, A.; Brambilla, M.; Mathews, N.; Ferrante, E.; Di Caro, G.; Ducatelle, F.; et al. ARGoS: A modular, multi-engine simulator for heterogeneous swarm robotics. Swarm Intell.
**2012**, 6, 271–295. [Google Scholar] [CrossRef] [Green Version]

**Figure 1.**Top view of the arena used for Task I. The gray objects are the obstacles, and the blue objects are the robots. The arena consists of a square perimeter with a square obstacle in the center.

**Figure 2.**Top view of the arena used for Task II. As in the first arena, the gray objects are the obstacles, the blue objects are the robots and the arena consists of a rectangular perimeter with a square obstacle in the center. Unlike the first arena, this one has, in addition, two virtual areas, red and blue.

**Figure 3.**Top view of the arena for Task III. The third arena features an octagonal perimeter with obstacles in the center; the blue and green areas indicate two virtual areas needed to accomplish a foraging task. Moreover, it also presents a light source located on the outer boundary of the green area. Purple and green lines indicate whether or not the robots are perceiving light and were used for debugging purposes.

**Figure 4.**Comparison of average maximum score trends obtained in each epoch by all robots with respect to the p parameter and the three adaptive techniques. Graphs are divided by the task they address. Given the very low average max score obtained by chaotic parameters in the first epochs (subplot), the Task III main graph shows the values above to $-50$.

**Figure 5.**Comparison of the actual distributions of maximum objective function values obtained by all the robots, subdivided by task typology.

**Figure 6.**Scatter plot of Derrida values (x-axis) against the relative objective function scores (y-axis) obtained by the robots in their best epoch. Noting that we used RBNs with $k=3$ as robot controllers, red points represent Boolean networks initially generated using $p=0.1$, while the green dots represent RBNs generated using $p=0.21$, and lastly, blue ones have been sampled from the ensemble of chaotic RBNs, i.e., using $p=0.5$. (

**a**)

**Task I**: Mutation (

**b**)

**Task II**: Mutation (

**c**)

**Task III**: Mutation (

**d**)

**Task I**: Hybrid (

**e**)

**Task II**: Hybrid (

**f**)

**Task III**: Hybrid (

**g**)

**Task I**: In-Out Mapping (

**h**)

**Task II**: In-Out Mapping (

**i**)

**Task III**: In-Out Mapping.

$n=100$, $k=3$, $p\in \{0.1,0.21,0.5,0.79,0.9\}$, (no self loops allowed) |

$\Delta t=0.1$, $d=80$ s; |

Population of 10 robots per each run; |

Adaptive mechanisms $\in \{$In-Out Mapping, Mutation, Hybrid}; |

Tasks $\in \{$I, II, III}; |

50 runs for each parameters combination |

**Table 2.**Ranking, for each task, of the first 5 values of the area under the curves of the trends shown in Figure 4. [

**i-o**] stands for in–out mapping mechanism, while [

**h**] stands for hybrid adaptation and [

**m**] stands for mutation scheme.

Task I | Task II | Task III | ||||
---|---|---|---|---|---|---|

Rank | Case | Area | Case | Area | Case | Area |

1 | [i-o] $p=0.21$ | 28,431.61 | [h] $p=0.21$ | 23,302.72 | [i-o] $p=0.79$ | 35,406.36 |

2 | [i-o] $p=0.79$ | 27,921.29 | [i-o] $p=0.21$ | 23,283.64 | [h] $p=0.79$ | 34,519.52 |

3 | [h] $p=0.79$ | 26,603.61 | [h] $p=0.79$ | 23,148.49 | [i-o] $p=0.21$ | 33,832.64 |

4 | [h] $p=0.21$ | 26,414.22 | [m] $p=0.79$ | 22,482.42 | [h] $p=0.21$ | 33,781.08 |

5 | [h] $p=0.1$ | 26,079.86 | [i-o] $p=0.79$ | 22,410.69 | [h] $p=0.1$ | 33,630.86 |

**Table 3.**Mean of the ranking of the maximum scores for Task III; in this analysis, the lower, the better. [

**i-o**] stands for in–out mapping mechanism, while [

**h**] stands for hybrid adaptation and [

**m**] stands for mutation scheme. For the

**nominal**dynamical regime, we considered as ordered the BNs generated with $p=\{0.1,0.9\}$, critical those with $p=\{0.21,0.79\}$ and chaotic those with $p=\left\{0.5\right\}$. Meanwhile, for the

**actual**dynamical regime, we considered them ordered if $D<1-\u03f5$, critical if $D\ge 1-\u03f5$ and $D\le 1+\u03f5$, and, lastly, chaotic if $D>1+\u03f5$, with $\u03f5=0.3$.

Task III | ||||
---|---|---|---|---|

Nominal | Actual | |||

Rank | Case | $\mathbf{<}\mathit{Rank}\mathbf{>}$ | Case | $\mathbf{<}\mathit{Rank}\mathbf{>}$ |

1 | [i-o] critical | $2593.62$ | [i-o] critical | $2600.79$ |

2 | [h] ordered | $2975.89$ | [h] critical | $2956.95$ |

3 | [h] critical | $3185.77$ | [m] critical | $4002.53$ |

4 | [m] critical | $4366.05$ | [i-o] ordered | $4779.87$ |

5 | [m] ordered | $4435.57$ | [h] ordered | $5468.71$ |

6 | [i-o] ordered | $4804.36$ | [m] ordered | $6687.97$ |

7 | [h] chaotic | $6967.89$ | [h] chaotic | $6734.92$ |

8 | [i-o] chaotic | $7134.18$ | [m] chaotic | $7063.64$ |

9 | [m] chaotic | $7557.92$ | [i-o] chaotic | $7134.17$ |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Braccini, M.; Roli, A.; Barbieri, E.; Kauffman, S.A.
On the Criticality of Adaptive Boolean Network Robots. *Entropy* **2022**, *24*, 1368.
https://doi.org/10.3390/e24101368

**AMA Style**

Braccini M, Roli A, Barbieri E, Kauffman SA.
On the Criticality of Adaptive Boolean Network Robots. *Entropy*. 2022; 24(10):1368.
https://doi.org/10.3390/e24101368

**Chicago/Turabian Style**

Braccini, Michele, Andrea Roli, Edoardo Barbieri, and Stuart A. Kauffman.
2022. "On the Criticality of Adaptive Boolean Network Robots" *Entropy* 24, no. 10: 1368.
https://doi.org/10.3390/e24101368