# Topological Characterization of Complex Systems: Using Persistent Entropy

^{1}

^{2}

^{3}

^{4}

^{*}

## Abstract

**:**

## 1. Introduction

**Figure 1.**On the left, a simplicial complex formed by one two-simplex (yellow filled triangle) and three one-simplices (segments forming the non-filled triangle) and three zero-simplices (the vertices). On the right, an invalid simplicial complex; the intersection between the yellow filled triangle and the one-simplex belonging to the upper triangle is not empty, and they do not share a common face.

**Figure 2.**Graphical representation of our methodology. All of the details are explained in the text.

## 2. Case Study

## 3. Methodology

#### 3.1. From Data to Weighted Graphs

- nodes represent the interacting components;
- an edge, equipped with a weight, expresses an interaction (or a distance) between two components.

#### 3.2. From Weighted Graphs to Filtered Simplicial Complexes

- -
- 0-simplex, geometrically represented by a vertex;
- -
- 1-simplex, geometrically represented by an edge;
- -
- 2-simplex, geometrically represented by a filled triangle;
- -
- 3-simplex, geometrically represented by a filled tetrahedron formed by filled triangles;
- -
- ⋯

**Figure 3.**On the left, an undirected graph formed by eight vertices $V=\{{v}_{0},{v}_{1},\cdots ,{v}_{7}\}$ and twelve edges with weights $W=\{{w}_{0},{w}_{1},\cdots ,{w}_{11}\}$ with ${w}_{0}\le {w}_{1}\cdots \le {w}_{11}$. Highlighted by dashed circles, the four 3-maximal cliques $\{{v}_{0},{v}_{1},{v}_{2}\}$, $\{{v}_{2},{v}_{3},{v}_{4}\}$, $\{{v}_{4},{v}_{5},{v}_{6}\}$ and $\{{v}_{6},{v}_{7},{v}_{0}\}$ identified by the Bron–Kerbosch algorithm. On the right, the simplicial complex formed by the four 2-simplices (filled triangles) corresponding to the four 3-maximal cliques.

- list all maximal cliques of G with the Bron–Kerbosch algorithm;
- for each maximal clique, select the minimum (or the maximum) value of the weights of its edges;
- for each maximal clique, assign the value selected in Equation (2) to the corresponding simplicial complex as the filter value.

**Figure 4.**On the right, the filtered simplicial complex derived from the algorithm where the filter values are selected as the minimum of the weights. As a result, we set ${t}_{0}={w}_{0},{t}_{1}={w}_{3},{t}_{2}={w}_{6},{t}_{3}={w}_{9}$, because we want a filter set $F=\{{t}_{0},{t}_{1},{t}_{2},{t}_{3}\}$, such that ${t}_{0}<{t}_{1}<{t}_{2}<{t}_{3}$.

#### 3.3. Computation of Persistent Homology on Filtered Simplicial Complexes

- -
- ${\beta}_{0}$ corresponds to the number of connected components;
- -
- ${\beta}_{1}$ corresponds to the number of planar holes;
- -
- ${\beta}_{2}$ corresponds to the number of voids in solid objects (2-dimensional holes);
- -
- …

`Dim`

`0`and

`Dim`

`1`) is equipped with two pieces of information: the lifespan, in terms of appearance and disappearance, of a Betti number in the form of an interval $[a;b)$ and the corresponding set of generators. If the line persists all over the procedure, i.e., it survives also during the subsequent introduction of new simplices, it is called persistent, and $b=\infty $. Otherwise, it is classified as topological noise.

**Figure 5.**Computation of persistent homology over a filtered simplicial complex: at filter value ${t}_{0}$, the first 2-simplex (blue filled triangle) is introduced. The topological space is characterized by one connected component; the Betti numbers are ${\beta}_{0}=1,\phantom{\rule{3.33333pt}{0ex}}{\beta}_{i}=0$ for all $i\ge 1$. These Betti numbers are the same until ${t}_{3}$ when the last 2-simplex (green filled triangle) is introduced, and it is connected to the others. For filter value ${t}_{3}$, the simplicial complex is still characterized by one connected component ${\beta}_{0}=1$, but also by one 1-dimensional hole, i.e., ${\beta}_{1}=1$. The generator of the connected component is $\left\{\left[{v}_{0}\right]\right\}$, while the generators of the persistent hole are $\{[{v}_{0},{v}_{2}],[{v}_{2},{v}_{4}],[{v}_{4},{v}_{6}],[{v}_{6},{v}_{0}]\}$.

`Dim`

`0`line of the Betti barcode. Similarly, a 1-dimensional hole appeared at filter value ${t}_{3}$, and its set of generators is $\{[{v}_{0},{v}_{2}],[{v}_{2},{v}_{4}],[{v}_{4},{v}_{6}],[{v}_{6},{v}_{0}]\}$; it is persistent, as indicated in the

`Dim`

`1`line of the Betti barcode. Note that in this example, all of the lines in the barcode are persistent. However, this is not the case in general, because topological noise can be present.

#### 3.4. From Persistent Homology to Computational Agents: Local Interactions

#### 3.5. From Persistent Homology to Persistent Entropy: Global Information

`Dim`

`0`and 1 is the index of the unique line in

`Dim`

`1`(both persistent in this case); then $m=4$; the line 0 is $[0,4)$; the line 1 is $[3,4)$; yielding an entropy $H=0.5$.

#### 3.6. The Derived S[B] Model

#### 3.7. Persistent Entropy Automaton

- R is a set of steady states;
- Λ is a set of names for the transitions;
- ${\rho}_{0}\in R$ is the initial steady state;
- H is the observable variable, corresponding to the value of persistent entropy;
- ${\stackrel{}{\to}}_{S}\subseteq R\times \Lambda \times R$ is a labeled transitions relation among steady states;
- $L(\rho )$ is a labeling function associating each state $\rho \in R$ with its equilibrium condition, depending on the values of H.

## 4. S[B] Model of the Case Study

- a lifespan of 2190 discrete time ticks, where a tick corresponds to three days;
- a repertoire of at most ${10}^{12}$ antibodies, i.e., the maximum number of antibodies available during the whole simulation;
- an antigen volume $V=10$ μL.

#### From Data to Persistent Homology

**Figure 6.**Example of the immune network at the end of a simulation. The thickness of the arcs is proportional to their weight; the diameter of the nodes is proportional to the number of incident edges.

**Table 1.**Example of the output of jHoles with idiotypic network (IN) simulation data as the input. One connected component appeared at filtration value 0.0, which is persistent. The generator is the vertex called 16. Three 1-dimensional holes appeared at filtration values 7.0, 6.0 and 8.0, which are persistent. The four edges generating them are reported after the intervals.

β_{0} | [0.0; ∞) : | {[16]} |

β_{1} | [7.0; ∞) : | {[320, 3775], [256, 3775], [320, 3839], [256, 3839]} |

[6.0; ∞) : | {[256, 3839], [256, 3711], [384, 3711], [384, 3839]} | |

[8.0; ∞) : | {[260, 3835], [260, 3839], [256, 3835], [256, 3839]} |

#### 4.1. Idiotype Agent Behaviors as Higher Dimensional Automata

**Figure 7.**A classical finite state automaton representing the interleaving of actions a and b (

**left**); the corresponding higher dimensional automaton (

**right**).

**Figure 8.**On the left, a classical automaton together with the two-length alphabet of statuses and its Chu space representation as a matrix. On the right, a higher dimensional automaton (HDA) with its Chu space representation using the three-length alphabet.

#### Applying Domain-Based Knowledge

**Table 2.**Chu space representation of an HDA modeling a generic antibody $A{b}_{i}$ performing two actions on the same target $A{b}_{j}$.

r | ${s}_{0}$ | ${s}_{1}$ | ${s}_{2}$ | ${s}_{3}$ | ${s}_{4}$ | ${s}_{5}$ | ${s}_{6}$ | ${s}_{7}$ | ${s}_{8}$ |
---|---|---|---|---|---|---|---|---|---|

$(\mathit{elicit},A{b}_{i},A{b}_{j})$ | 0 | 0 | 0 | 1 | 1 | 1 | 2 | 2 | 2 |

$(\mathit{reduce},A{b}_{i},A{b}_{j})$ | 0 | 1 | 2 | 0 | 1 | 2 | 0 | 1 | 2 |

r | ${s}_{1}$ | ${s}_{2}$ | ${s}_{3}$ | ${s}_{4}$ | ${s}_{5}$ | ${s}_{6}$ | ${s}_{7}$ | ${s}_{8}$ | ${s}_{9}$ | ${s}_{10}$ | ${s}_{11}$ | ${s}_{12}$ | ${s}_{13}$ |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|

$(\mathit{elicit},A{b}_{1},A{b}_{13})$ | 1 | 0 | 1 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 1 |

$(\mathit{reduce},A{b}_{1},A{b}_{13})$ | 0 | 0 | 0 | 0 | 1 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

$(\mathit{elicit},A{b}_{13},A{b}_{1})$ | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 2 | 0 | 0 | 1 | 0 | 2 |

$(\mathit{reduce},A{b}_{13},A{b}_{1})$ | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 2 | 0 | 1 | 0 |

$\mathit{r}$ | ${\mathit{s}}_{\mathbf{14}}$ | ${\mathit{s}}_{\mathbf{15}}$ | ${\mathit{s}}_{\mathbf{16}}$ | ${\mathit{s}}_{\mathbf{17}}$ | ${\mathit{s}}_{\mathbf{18}}$ | ${\mathit{s}}_{\mathbf{19}}$ | ${\mathit{s}}_{\mathbf{20}}$ | ${\mathit{s}}_{\mathbf{21}}$ | ${\mathit{s}}_{\mathbf{22}}$ | ${\mathit{s}}_{\mathbf{23}}$ | ${\mathit{s}}_{\mathbf{24}}$ | ${\mathit{s}}_{\mathbf{25}}$ | |

$(\mathit{elicit},A{b}_{1},A{b}_{13})$ | 2 | 2 | 2 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |

$(\mathit{reduce},A{b}_{1},A{b}_{13})$ | 0 | 0 | 0 | 0 | 1 | 1 | 1 | 1 | 2 | 2 | 2 | 2 | |

$(\mathit{elicit},A{b}_{13},A{b}_{1})$ | 1 | 0 | 2 | 0 | 1 | 0 | 2 | 0 | 1 | 0 | 2 | 0 | |

$(\mathit{reduce},A{b}_{13},A{b}_{1})$ | 0 | 1 | 0 | 2 | 0 | 1 | 0 | 2 | 0 | 1 | 0 | 2 |

#### 4.2. Persistent Entropy Automaton for the Idiotypic Network

**Figure 11.**Persistent entropy of the immune system. The difference between the peaks amplitude is motivated by the fact that before the second peaks, the antibodies have been already stimulated, and the immune memory has been reached, so the system is more reactive and is faster in the suppression of the antigen; thus, the the entropy value for the steady state is $H=2.87$.

**Figure 12.**Persistent entropy automaton representing the S level of the $S\left[B\right]$ model of the idiotypic network.

## 5. Conclusions

## Acknowledgments

## Author Contributions

## Conflicts of Interest

## References

- Stein, D.; Newman, C. Nature versus nurture in complex and not-so-complex systems. In ISCS 2013: Interdisciplinary Symposium on Complex Systems; Springer: Berlin Heidelberg, Germany, 2014; pp. 57–63. [Google Scholar]
- Boccara, N. Modeling Complex Systems, 2nd ed.; Springer-Verlag: New York, NY, USA, 2010. [Google Scholar]
- Jennings, N.R. An agent-based approach for building complex software systems. Commun. ACM
**2001**, 44, 35–41. [Google Scholar] [CrossRef] - Carlsson, G. Topology and data. Bull. Am. Math. Soc.
**2009**, 46, 255–308. [Google Scholar] [CrossRef] - Zomorodian, A.J. Topology for computing. In Cambridge Monographs on Applied and Computational Mathematics; Cambridge University Press: Cambridge, UK, 2009; Volume 16. [Google Scholar]
- Ibekwe, A.M.; Ma, J.; Crowley, D.E.; Yang, C.H.; Johnson, A.M.; Petrossian, T.C.; Lum, P.Y. Topological data analysis of escherichia coli o157: H7 and non-o157 survival in soils. Front. Cell. Infect. Microbiol.
**2014**, 4, 122. [Google Scholar] [CrossRef] [PubMed] - De Silva, V.; Ghrist, R. Coverage in sensor networks via persistent homology. Algebraic Geom. Topol.
**2007**, 7, 24. [Google Scholar] [CrossRef] - Chan, J.M.; Carlsson, G.; Rabadan, R. Topology of viral evolution. Proc. Natl. Acad. Sci. USA
**2003**, 110, 18566–18571. [Google Scholar] [CrossRef] [PubMed] - Petri, G.; Expert, P.; Turkheimer, F.; Carhart-Harris, R.; Nutt, D.; Hellyer, P.J.; Vaccarino, F. Homological scaffolds of brain functional networks. J. R. Soc. Interf.
**2014**, 11. [Google Scholar] [CrossRef] - Edelsbrunner, H.; Harer, J. Computational Topology: An Introduction; American Mathematical Soc.: Providence, RI, USA, 2010. [Google Scholar]
- Merelli, E.; Pettini, M.; Rasetti, M. Topology driven modeling: The IS metaphor. Nat. Comput.
**2015**, 14, 421–430. [Google Scholar] [CrossRef] [PubMed] - Merelli, E.; Paoletti, N.; Tesei, L. Adaptability Checking in Complex Systems. Sci. Comput. Program.
**2015**. [Google Scholar] [CrossRef] - Jerne, N.K. Towards a network theory of the immune system. Ann. Immunol.
**1974**, 125, 373–389. [Google Scholar] - Holland, J.H. Complex adaptive systems. Daedalus
**1992**, 121, 17–30. [Google Scholar] - Parisi, G. A simple model for the immune network. Proc. Natl. Acad. Sci. USA
**1990**, 87, 429–433. [Google Scholar] [CrossRef] [PubMed] - Bernaschi, M.; Castiglione, F. Design and implementation of an immune system simulator. Comput. Biol. Med.
**2001**, 31, 303–331. [Google Scholar] [CrossRef] - Hoffmann, G. A theory of regulation and self-nonself discrimination in an immune network. Eur. J. Immunol.
**1975**, 5, 638–647. [Google Scholar] [CrossRef] [PubMed] - Mikk, E.; Lakhnech, Y.; Siegel, M. Hierarchical Automata as Model for Statecharts. In Advances in Computing Science (ASIAN’97); No. 1345 in Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 1997; pp. 181–106. [Google Scholar]
- Jonsson, J. Simplicial Complexes of Graphs; Springer-Verlag: Berlin Heidelberg, Germany, 2008. [Google Scholar]
- Bron, C.; Kerbosch, J. Algorithm 457: Finding all cliques of an undirected graph. Commun. ACM
**1973**, 16, 575–577. [Google Scholar] [CrossRef] - Bandelt, H.J.; Chepoi, V. Metric graph theory and geometry: A survey. Contemp. Math.
**2008**, 453, 49–86. [Google Scholar] - Adams, H.; Tausz, A.; Vejdemo-Johansson, M. javaPlex: A research software package for persistent (co) homology. In Mathematical Software-ICMS 2014; Springer: Berlin Heidelberg, Germany, 2014; pp. 129–136. [Google Scholar]
- Petri, G.; Scolamiero, M.; Donato, I.; Vaccarino, F. Topological strata of weighted complex networks. PLoS ONE
**2003**, 8, e66506. [Google Scholar] [CrossRef] [PubMed] - Binchi, J.; Merelli, E.; Rucco, M.; Petri, G.; Vaccarino, F. jholes: A tool for understanding biological complex networks via clique weight rank persistent homology. Electron. Notes Theor. Comput. Sci.
**2014**, 306, 5–18. [Google Scholar] [CrossRef] - Merelli, E.; Rasetti, M. Non locality, topology, formal languages: New global tools to handle large data sets. Proced. Comput. Sci.
**2013**, 18, 90–99. [Google Scholar] [CrossRef] - Pratt, V. Modeling concurrency with geometry. In Proceedings of the 18th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, Orlando, FL, USA, 21–23 January 1991; pp. 311–322.
- Rucco, M.; Castiglione, F.; Merelli, E.; Pettini, M. Characterisation of the idiotypic immune network through persistent entropy. Proc. Complex.
**2015**, in press. [Google Scholar] - Hopcroft, J.E.; Motwani, R.; Ullman, J.D. Introduction to automata theory, languages, and computation. ACM SIGACT News
**2001**, 32, 60–65. [Google Scholar] [CrossRef] - Bernardo, M.; Tesei, L. Encoding timed models as uniform labeled transition systems. In Computer Performance Engineering; Springer: Berlin/Heidelberg, Germany, 2013; pp. 104–118. [Google Scholar]
- Henzinger, T.A. The Theory of Hybrid Automata; Springer: Berlin/Heidelberg, Germany, 2000. [Google Scholar]
- Mancini, E.; Castiglione, F.; Bernaschi, M.; de Luca, A.; Sloot, P. HIV reservoirs and immune surveillance evasion cause the failure of structured treatment interruptions: A computational study. PLoS ONE
**2012**, 4, e36108. [Google Scholar] [CrossRef] [PubMed] - Milner, R. Communication and Concurrency; Prentice-Hall, Inc.: Upper Saddle River, NJ, USA, 1989. [Google Scholar]
- Gupta, V. CHU Spaces: A Model of Concurrency. Ph.D. Thesis, Stanford University, Palo Alto, CA, USA, 1994. [Google Scholar]
- Pratt, V. Higher dimensional automata revisited. Math. Struct. Comput. Sci.
**2000**, 10, 525–548. [Google Scholar] [CrossRef] - Stewart, W.J. Probability, Markov Chains, Queues, and Simulation: The Mathematical Basis of Performance Modeling; Princeton University Press: Princeton, NJ, USA, 2009. [Google Scholar]
- Rasetti, M.; Merelli, E. The topological field theory of data: A program towards a novel strategy for data mining through data language. J. Phys. Conf. Ser.
**2015**, 626, 012005. [Google Scholar] [CrossRef]

© 2015 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Merelli, E.; Rucco, M.; Sloot, P.; Tesei, L.
Topological Characterization of Complex Systems: Using Persistent Entropy. *Entropy* **2015**, *17*, 6872-6892.
https://doi.org/10.3390/e17106872

**AMA Style**

Merelli E, Rucco M, Sloot P, Tesei L.
Topological Characterization of Complex Systems: Using Persistent Entropy. *Entropy*. 2015; 17(10):6872-6892.
https://doi.org/10.3390/e17106872

**Chicago/Turabian Style**

Merelli, Emanuela, Matteo Rucco, Peter Sloot, and Luca Tesei.
2015. "Topological Characterization of Complex Systems: Using Persistent Entropy" *Entropy* 17, no. 10: 6872-6892.
https://doi.org/10.3390/e17106872