# Information Theory of Networks

## Abstract

**:**

## 1. Introduction

## 2. Graph Entropies

#### 2.1. Measures Based on Equivalence Criteria and Graph Invariants

- Topological information content due to Rashevsky [19]:$${I}_{a}\left(G\right):=-\sum _{i=1}^{k}\frac{|{N}_{i}|}{\left|V\right|}log\left(\right)open="("\; close=")">\frac{|{N}_{i}|}{\left|V\right|}$$
- Symmetry index for graphs due to Mowshowitz et al. [47]:$$S\left(G\right):=\left(\right)open="("\; close=")">log\left(\right|V\left|\right)-{I}_{a}\left(G\right)$$
- Chromatic information content due to Mowshowitz [15,16]:$${I}_{c}\left(G\right):=\underset{\widehat{V}}{min}\left(\right)open="\{"\; close="\}">-\sum _{i=1}^{h}\frac{{n}_{i}\left(\widehat{V}\right)}{\left|V\right|}log\left(\right)open="("\; close=")">\frac{{n}_{i}\left(\widehat{V}\right)}{\left|V\right|}$$
- Magnitude-based information indices due to Bonchev et al. [49]:$$\begin{array}{ccc}\hfill {I}_{D}\left(G\right)& :=& -\frac{1}{\left|V\right|}log\left(\right)open="("\; close=")">\frac{1}{\left|V\right|}-\sum _{i=1}^{\rho \left(G\right)}\frac{2{k}_{i}}{{\left|V\right|}^{2}}log\left(\right)open="("\; close=")">\frac{2{k}_{i}}{{\left|V\right|}^{2}}\hfill \end{array}$$$$\begin{array}{ccc}\hfill {I}_{D}^{W}\left(G\right)& :=& W\left(G\right)log\left(W\left(G\right)\right)-\sum _{i=1}^{\rho \left(G\right)}i{k}_{i}log\left(i\right)\hfill \end{array}$$
- Vertex degree equality-based information index found by Bonchev [8]:$${I}_{\mathrm{deg}}\left(G\right):=\sum _{i=1}^{\overline{k}}\frac{|{N}_{i}^{{k}_{v}}|}{\left|V\right|}log\left(\right)open="("\; close=")">\frac{|{N}_{i}^{{k}_{v}}|}{\left|V\right|}$$
- Overall information indices found by Bonchev [46,51]:$$\begin{array}{cc}\hfill OX\left(G\right)& :=\sum _{k=0}^{\left|E\right|}{}^{k}X;\phantom{\rule{1.em}{0ex}}\left\{X\right\}:=\{{}^{0}X,{}^{1}X,\dots ,{}^{\left|E\right|}X\}\hfill \end{array}$$$$\begin{array}{cc}\hfill I(G,OX)& :=OXlog\left(OX\right)-\sum _{k=0}^{\left|E\right|}{}^{k}Xlog\left(\right)open="("\; close=")">{}^{k}X\hfill \end{array}$$

#### 2.2. Körner Entropy

#### 2.3. Entropy Measures Using Information Functionals

- What kind of structural features (e.g., vertices, edges, degrees, distances etc.) should be used to derive meaningful information functionals?
- In this context, what does “meaningful” mean?
- In case the functional is parametric, how can the parameters be optimized?
- What kind of structural information does the functional as well as the resulting entropy detect?

#### 2.4. Information-Theoretic Measures for Trees

#### 2.5. Other Information-Theoretic Network Measures

## 3. Structural Interpretation of Graph Measures

## 4. Summary and Conclusion

## Acknowledgements

## References

- Allen, E.B. Measuring Graph Abstractions of Software: An Information-Theory Approach. In Proceedings of the 8-th International Symposium on Software Metrics; IEEE Computer Society: Ottawa, ON, Canada, 4–7 June 2002; p. 182.
- Cover, T.M.; Thomas, J.A. Elements of Information Theory; Wiley & Sons: Hoboken, NJ, USA, 2006. [Google Scholar]
- McDonnell, M.D.; Ikeda, S.; Manton, J.H. An introductory review of information theory in the context of computational neuroscience. Biol. Cybern.
**2011**, 105, 55–70. [Google Scholar] [CrossRef] [PubMed] - Mathar, R.; Schmeink, A. A Bio-Inspired Approach to Condensing Information. In Proceedings of the IEEE International Symposium on Information Theory (ISIT); IEEE Xplore: Saint-Petersburg, Russia, 31 July–5 August 2011; pp. 2524–2528.
- Shannon, C.E.; Weaver, W. The Mathematical Theory of Communication; University of Illinois Press: Champaign, IL, USA, 1949. [Google Scholar]
- Grosse, I.; Galván, P.B.; Carpena, P.; Roldán, R.R.; Oliver, J.; Stanley, H.E. Analysis of symbolic sequences using the Jensen-Shannon divergence. Phys. Rev. E
**2002**, 65, 041905:1–041905:16. [Google Scholar] [CrossRef] [PubMed] - Anand, K.; Bianconi, G. Entropy measures for networks: Toward an information theory of complex topologies. Phys. Rev. E
**2009**, 80, 045102(R):1–045102(R):4. [Google Scholar] [CrossRef] [PubMed] - Bonchev, D. Information Theoretic Indices for Characterization of Chemical Structures; Research Studies Press: Chichester, UK, 1983. [Google Scholar]
- Dehmer, M. Information processing in complex networks: Graph entropy and information functionals. Appl. Math. Comput.
**2008**, 201, 82–94. [Google Scholar] [CrossRef] - Hirata, H.; Ulanowicz, R.E. Information theoretical analysis of ecological networks. Int. J. Syst. Sci.
**1984**, 15, 261–270. [Google Scholar] [CrossRef] - Kim, D.C.; Wang, X.; Yang, C.R.; Gao, J. Learning biological network using mutual information and conditional independence. BMC Bioinf.
**2010**, 11, S9:1–S9:8. [Google Scholar] [CrossRef] [PubMed] - Barnett, L.; Buckley, L.; Bullock, C.L. Neural complexity and structural connectivity. Phys. Rev. E
**2009**, 79, 051914:1–051914:12. [Google Scholar] [CrossRef] [PubMed] - Morowitz, H. Some order-disorder considerations in living systems. Bull. Math. Biophys.
**1953**, 17, 81–86. [Google Scholar] [CrossRef] - Trucco, E. A note on the information content of graphs. Bull. Math. Biol.
**1956**, 18, 129–135. [Google Scholar] [CrossRef] - Mowshowitz, A. Entropy and the complexity of the graphs I: An index of the relative complexity of a graph. Bull. Math. Biophys.
**1968**, 30, 175–204. [Google Scholar] [CrossRef] - Mowshowitz, A. Entropy and the complexity of graphs II: The information content of digraphs and infinite graphs. Bull. Math. Biophys.
**1968**, 30, 225–240. [Google Scholar] [CrossRef] [PubMed] - Mowshowitz, A. Entropy and the complexity of graphs III: Graphs with prescribed information content. Bull. Math. Biophys.
**1968**, 30, 387–414. [Google Scholar] [CrossRef] - Mowshowitz, A. Entropy and the complexity of graphs IV: Entropy measures and graphical structure. Bull. Math. Biophys.
**1968**, 30, 533–546. [Google Scholar] [CrossRef] - Rashevsky, N. Life, information theory, and topology. Bull. Math. Biophys.
**1955**, 17, 229–235. [Google Scholar] [CrossRef] - Bonchev, D. Information Theoretic Measures of Complexity. In Encyclopedia of Complexity and System Science; Meyers, R., Ed.; Springer: Berlin, Heidelberg, Germany, 2009; Volume 5, pp. 4820–4838. [Google Scholar]
- Dehmer, M. A novel method for measuring the structural information content of networks. Cybern. Syst.
**2008**, 39, 825–842. [Google Scholar] [CrossRef] - Mowshowitz, A.; Mitsou, V. Entropy, Orbits and Spectra of Graphs. In Analysis of Complex Networks: From Biology to Linguistics; Dehmer, M., Emmert-Streib, F., Eds.; Wiley-VCH: Hoboken, NJ, USA, 2009; pp. 1–22. [Google Scholar]
- Bonchev, D.; Rouvray, D.H. Complexity in Chemistry, Biology, and Ecology; Springer: New York, NY, USA, 2005. [Google Scholar]
- Dancoff, S.M.; Quastler, H. Information Content and Error Rate of Living Things. In Essays on the Use of Information Theory in Biology; Quastler, H., Ed.; University of Illinois Press: Champaign, IL, USA, 1953; pp. 263–274. [Google Scholar]
- Linshitz, H. The Information Content of a Battery Cell. In Essays on the Use of Information Theory in Biology; Quastler, H., Ed.; University of Illinois Press: Urbana, IL, USA, 1953. [Google Scholar]
- Latva-Koivisto, A.M. Finding a Complexity Measure for Business Process Models; Helsinki University of Technology: Espoo, Finland, 2001. [Google Scholar]
- MacArthur, R.H. Fluctuations of animal populations and a measure of community stability. Ecology
**1955**, 36, 533–536. [Google Scholar] [CrossRef] - Solé, R.V.; Montoya, J.M. Complexity and Fragility in Ecological Networks. Proc. R. Soc. Lond. B
**2001**, 268, 2039–2045. [Google Scholar] [CrossRef] - Ulanowicz, R.E. Information theory in ecology. Comput. Chem.
**2001**, 25, 393–399. [Google Scholar] [CrossRef] - Körner, J. Coding of an Information Source Having Ambiguous Alphabet and the Entropy of Graphs. In Proceedings of the Transactions of the 6th Prague Conference on Information Theory; Academia: Prague, Czechoslovakia, 19–25 September 1973; pp. 411–425.
- Mehler, A. Social Ontologies as Generalized Nearly Acyclic Directed Graphs: A Quantitative Graph Model of Social Tagging. In Towards an Information Theory of Complex Networks: Statistical Methods and Applications; Dehmer, M., Emmert-Streib, F., Mehler, A., Eds.; Birkhäuser: Boston, MA, USA, 2011; pp. 259–319. [Google Scholar]
- Mehler, A.; Weiß, P.; Lücking, A. A network model of interpersonal alignment. Entropy
**2010**, 12, 1440–1483. [Google Scholar] [CrossRef] - Balch, T. Hierarchic social entropy: An information theoretic measure of robot group diversity. Auton. Robot.
**2001**, 8, 209–237. [Google Scholar] [CrossRef] - Butts, C.T. The complexity of social networks: Theoretical and empirical findings. Soc. Netw.
**2001**, 23, 31–71. [Google Scholar] [CrossRef] - Sommerfeld, E.; Sobik, F. Operations on Cognitive Structures—Their Modeling on the Basis of Graph Theory. In Knowledge Structures; Albert, D., Ed.; Springer: Berlin, Heidelberg, Germany, 1994; pp. 146–190. [Google Scholar]
- Krawitz, P.; Shmulevich, I. Entropy of complex relevant components of Boolean networks. Phys. Rev. E
**2007**, 76, 036115:1–036115:7. [Google Scholar] [CrossRef] [PubMed] - Sanchirico, A.; Fiorentino, M. Scale-free networks as entropy competition. Phys. Rev. E
**2008**, 78, 046114:1–046114:10. [Google Scholar] [CrossRef] [PubMed] - Bonchev, D.; Mekenyan, O.; Trinajstić, N. Topological characterization of cyclic structures. Int. J. Quantum Chem.
**1980**, 17, 845–893. [Google Scholar] [CrossRef] - Dehmer, M.; Sivakumar, L.; Varmuza, K. Uniquely discriminating molecular structures using novel eigenvalue-based descriptors. MATCH Commun. Math. Comput. Chem.
**2012**, 67, 147–172. [Google Scholar] - Konstantinova, E.V. The discrimination ability of some topological and information distance indices for graphs of unbranched hexagonal systems. J. Chem. Inf. Comput. Sci.
**1996**, 36, 54–57. [Google Scholar] [CrossRef] - Dehmer, M.; Barbarini, N.; Varmuza, K.; Graber, A. A large scale analysis of information-theoretic network complexity measures using chemical structures. PLoS One
**2009**, 4. [Google Scholar] [CrossRef] [PubMed] - Bonchev, D. Information indices for atoms and molecules. MATCH Commun. Math. Comp. Chem.
**1979**, 7, 65–113. [Google Scholar] - Bertz, S.H. The first general index of molecular complexity. J. Am. Chem. Soc.
**1981**, 103, 3241–3243. [Google Scholar] [CrossRef] - Basak, S.C.; Magnuson, V.R. Molecular topology and narcosis. Drug Res.
**1983**, 33, 501–503. [Google Scholar] - Basak, S.C. Information-Theoretic Indices of Neighborhood Complexity and their Applications. In Topological Indices and Related Descriptors in QSAR and QSPAR; Devillers, J., Balaban, A.T., Eds.; Gordon and Breach Science Publishers: Amsterdam, The Netherlands, 1999; pp. 563–595. [Google Scholar]
- Bonchev, D. Overall connectivities and topological complexities: A new powerful tool for QSPR/QSAR. J. Chem. Inf. Comput. Sci.
**2000**, 40, 934–941. [Google Scholar] [CrossRef] [PubMed] - Mowshowitz, A.; Dehmer, M. A symmetry index for graphs. Symmetry Cult. Sci.
**2010**, 21, 321–327. [Google Scholar] - Dehmer, M.; Varmuza, K.; Borgert, S.; Emmert-Streib, F. On entropy-based molecular descriptors: Statistical analysis of real and synthetic chemical structures. J. Chem. Inf. Model.
**2009**, 49, 1655–1663. [Google Scholar] [CrossRef] [PubMed] - Bonchev, D.; Trinajstić, N. Information theory, distance matrix and molecular branching. J. Chem. Phys.
**1977**, 67, 4517–4533. [Google Scholar] [CrossRef] - Dehmer, M.; Grabner, M.; Varmuza, K. Information indices with high discrimination power for arbitrary graphs. PLoS One submitted for publication.
**2011**. [Google Scholar] - Bonchev, D. The Overall Topological Complexity Indices. In Advances in Computational Methods in Science and Engineering; Simos, T., Maroulis, G., Eds.; VSP Publications: Boulder, USA, 2005; Volume 4B, pp. 1554–1557. [Google Scholar]
- Bonchev, D. My life-long journey in mathematical chemistry. Internet Electron. J . Mol. Des.
**2005**, 4, 434–490. [Google Scholar] - Todeschini, R.; Consonni, V.; Mannhold, R. Handbook of Molecular Descriptors; Wiley-VCH: Weinheim, Germany, 2002. [Google Scholar]
- Passerini, F.; Severini, S. The von Neumann entropy of networks. Int. J. Agent Technol. Syst.
**2009**, 1, 58–67. [Google Scholar] [CrossRef] - Simonyi, G. Graph Entropy: A Survey. In Combinatorial Optimization; Cook, W., Lovász, L., Seymour, P., Eds.; ACM: New York, NY, USA, 1995; Volume 20, pp. 399–441. [Google Scholar]
- Bang-Jensen, J.; Gutin, G. Digraphs. Theory, Algorithms and Applications; Springer: Berlin, Heidelberg, Germany, 2002. [Google Scholar]
- Dehmer, M. Information-theoretic concepts for the analysis of complex networks. Appl. Artif. Intell.
**2008**, 22, 684–706. [Google Scholar] [CrossRef] - Dijkstra, E.W. A note on two problems in connection with graphs. Numer. Math.
**1959**, 1, 269–271. [Google Scholar] [CrossRef] - Brandes, U.; Erlebach, T. Network Analysis; Springer: Berlin, Heidelberg, Germany, 2005. [Google Scholar]
- Emmert-Streib, F.; Dehmer, M. Information theoretic measures of UHG graphs with low computational complexity. Appl. Math. Comput.
**2007**, 190, 1783–1794. [Google Scholar] [CrossRef] - Lyons, R. Identities and inequalities for tree entropy. Comb. Probab. Comput.
**2010**, 19, 303–313. [Google Scholar] [CrossRef] - Solé, R.V.; Valverde, S. Information Theory of Complex Networks: On Evolution and Architectural Constraints; Springer: Berlin, Heidelberg, Germany, 2004; Volume 650, pp. 189–207. [Google Scholar]
- Wilhelm, T.; Hollunder, J. Information theoretic description of networks. Physica A
**2007**, 388, 385–396. [Google Scholar] [CrossRef] - Ulanowicz, R.E.; Goerner, S.J.; Lietaer, B.; Gomez, R. Quantifying sustainability: Resilience, efficiency and the return of information theory. Ecol. Complex.
**2009**, 6, 27–36. [Google Scholar] [CrossRef] - Claussen, J.C. Characterization of networks by the offdiagonal complexity. Physica A
**2007**, 365–373, 321–354. [Google Scholar] - Kim, J.; Wilhelm, T. What is a complex graph? Physica A
**2008**, 387, 2637–2652. [Google Scholar] [CrossRef] - Bonchev, D. Topological order in molecules 1. Molecular branching revisited. J. Mol. Strut. THEOCHEM
**1995**, 336, 137–156. [Google Scholar] [CrossRef] - Janežić, D.; Miležević, A.; Nikolić, S.; Trinajstić, N. Topological Complexity of Molecules. In Encyclopedia of Complexity and System Science; Meyers, R., Ed.; Springer: Berlin, Heidelberg, Germany, 2009; Volume 5, pp. 9210–9224. [Google Scholar]
- Wiener, H. Structural determination of paraffin boiling points. J. Am. Chem. Soc.
**1947**, 69, 17–20. [Google Scholar] [CrossRef] [PubMed] - Balaban, A.T.; Mills, D.; Kodali, V.; Basak, S.C. Complexity of chemical graphs in terms of size, branching and cyclicity. SAR QSAR Environ. Res.
**2006**, 17, 429–450. [Google Scholar] [CrossRef] [PubMed] - Finn, J.T. Measures of ecosystem structure and function derived from analysis of flows. J. Theor. Biol.
**1976**, 56, 363–380. [Google Scholar] [CrossRef] - Garrido, A. Symmetry of complex networks. Adv. Model. Optim.
**2009**, 11, 615–624. [Google Scholar] [CrossRef] - Li, X.; Li, Z.; Wang, L. The inverse problems for some topological indices in combinatorial chemistry. J. Comput. Biol.
**2003**, 10, 47–55. [Google Scholar] [CrossRef] - Bonchev, D.; Mekenyan, O.; Trinajstić, N. Isomer discrimination by topological information approach. J. Comput. Chem.
**1981**, 2, 127–148. [Google Scholar] [CrossRef] - Diudea, M.V.; Ilić, A.; Varmuza, K.; Dehmer, M. Network analysis using a novel highly discriminating topological index. Complexity
**2011**, 16, 32–39. [Google Scholar] [CrossRef] - Konstantinova, E.V.; Paleev, A.A. Sensitivity of topological indices of polycyclic graphs. Vychisl. Sist.
**1990**, 136, 38–48. [Google Scholar]

© 2011 by the author; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0/).

## Share and Cite

**MDPI and ACS Style**

Dehmer, M.
Information Theory of Networks. *Symmetry* **2011**, *3*, 767-779.
https://doi.org/10.3390/sym3040767

**AMA Style**

Dehmer M.
Information Theory of Networks. *Symmetry*. 2011; 3(4):767-779.
https://doi.org/10.3390/sym3040767

**Chicago/Turabian Style**

Dehmer, Matthias.
2011. "Information Theory of Networks" *Symmetry* 3, no. 4: 767-779.
https://doi.org/10.3390/sym3040767