# Allometric Scaling of Mutual Information in Complex Networks: A Conceptual Framework and Empirical Approach

^{1}

^{2}

^{3}

^{4}

^{*}

^{†}

## Abstract

**:**

## 1. Introduction

#### 1.1. The Context

#### 1.2. The Complexity Framework: Allometric Nature of Mutual Information

#### 1.2.1. Structural Mutual Information: SMI

#### 1.2.2. Total Mutual Information, I

#### 1.3. The Economic Dimension

## 2. Results

#### 2.1. Data Analysis

#### 2.1.1. Macro Features of Entropy and Mutual Information

#### 2.1.2. The Geographical Perspective of Mutual Information

## 3. The Allometric Nature of Mutual Information

#### Entropy and Mutual Information Micro Features

## 4. Conclusions and Discussion

## 5. Materials and Methods

#### 5.1. Measuring Entropy and Mutual Information

#### 5.2. Network Randomisation and Rewiring Process

#### 5.3. Methods Underpinning the Complexity Framework

#### 5.3.1. Structural Mutual Information, SMI

#### 5.3.2. Total Mutual Information I

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## Abbreviations

SMI | Structural Mutual Information |

I | Total Mutual Information |

H | Entropy |

J | Total Joint Entropy |

GDP | Gross Domestic Product |

## References

- Shannon, C.E. A mathematical theory of communication. Bell Syst. Techn. J.
**1948**, 27, 379–423. [Google Scholar] [CrossRef][Green Version] - von Neumann, J. Papers of John Von Neumann on Computing and Computer Theory; MIT Press: Cambridge, MA, USA, 1987. [Google Scholar]
- von Neumann, J. Collected Works/Vol. V, Design of Computers, Theory of Automata and Numerical Analysis; Pergamon Press: Oxford, UK, 1963. [Google Scholar]
- Dehmer, M.; Emmert-Streib, F.; Mehler, A. The Central Role of Information Theory in Ecology. In Towards an Information Theory of Complex Networks: Statistical Methods and Applications, 1st ed.; Birkhäuser Boston: Boston, MA, USA, 2011; pp. 153–167. [Google Scholar]
- Ulanowicz, R.E. Reckoning the nonexistent: Putting the science right. Ecol. Model.
**2014**, 293, 22–30. [Google Scholar] [CrossRef] - West, G.B. A general model for the origin allometric scaling laws in biology. Science
**1997**, 276, 122–126. [Google Scholar] [CrossRef] [PubMed] - Bettencourt, L.M.A.; Lobo, J.; Helbing, D.; Kühnert, C.; West, G.B. Growth, innovation, scaling, and the pace of life in cities. Proc. Natl. Acad. Sci. USA
**2007**, 104, 7301. [Google Scholar] [CrossRef] [PubMed][Green Version] - West, G.B. Santa Fe Institute studies in the sciences of complexity. In Scaling in Biology; Oxford University Press: Oxford, UK, 2000. [Google Scholar]
- Goto, H.; Viegas, E.; Jensen, H.J.; Takayasu, H.; Takayasu, M. Appearance of Unstable Monopoly State Caused by Selective and Concentrative Mergers in Business Networks. Sci. Rep.
**2017**, 7, 5064. [Google Scholar] [CrossRef] [PubMed][Green Version] - Goto, H.; Viegas, E.; Jensen, H.; Takayasu, H.; Takayasu, M. Smoluchowski Equation for Networks: Merger Induced Intermittent Giant Node Formation and Degree Gap. J. Stat. Phys.
**2018**, 172, 1086–1100. [Google Scholar] [CrossRef][Green Version] - Takayasu, M.; Sameshima, S.; Watanabe, H.; Ohnishi, T.; Iyatomi, H.; Iino, T.; Kobayashi, Y.; Kamehama, K.; Ikeda, Y.; Takayasu, H.; et al. Massive Economics Data Analysis by Econophysics Method-The Case of Companies’ Network Structure; Annual Report of the Earth Simulator Center; Earth Simulator Center: Yokohama, Japan, 2008; pp. 237–241. [Google Scholar]
- Viegas, E.; Goto, H.; Takayasu, H.; Takayasu, M.; Jensen, H. Assembling real networks from synthetic and unstructured subsets: The corporate reporting case. Sci. Rep.
**2019**, 9, 1–10. [Google Scholar] [CrossRef] [PubMed][Green Version] - Watanabe, H.; Takayasu, H.; Takayasu, M. Relations between allometric scalings and fluctuations in complex systems: The case of Japanese firms. Physica A
**2013**, 392, 741–756. [Google Scholar] [CrossRef][Green Version] - Huang, J.; Ulanowicz, R. Ecological Network Analysis for Economic Systems: Growth and Development and Implications for Sustainable Development. PLoS ONE
**2014**, 9, e100923. [Google Scholar] [CrossRef] [PubMed] - Arkolakis, C. Market Penetration Costs and the New Consumers Margin in International Trade; Working Paper No. 14214; NBER: Washington, DC, USA, Augest 2008. [Google Scholar]
- Price, D.D.S. A general theory of bibliometric and other cumulative advantage processes. J. Assoc. Inf. Sci. Technol.
**1976**, 27, 292–306. [Google Scholar] [CrossRef][Green Version] - Barabási, A.L.; Albert, R. Emergence of Scaling in Random Networks. Science
**1999**, 286, 509–512. [Google Scholar] [CrossRef] [PubMed][Green Version] - Sanchirico, A.; Fiorentino, M. Scale-free networks as entropy competition. Phys. Rev. E
**2008**, 78, 046114. [Google Scholar] [CrossRef] [PubMed] - Sutton, J. Sunk Costs and Market Structure: Price Competition, Advertising, and the Evolution of Concentration; MIT Press: Cambridge, MA, USA, 1991. [Google Scholar]
- Viegas, E.; Cockburn, S.P.; Jensen, H.J.; West, G.B. The dynamics of mergers and acquisitions: Ancestry as the seminal determinant. Proc. Math. Phys. Eng. Sci.
**2014**, 470, 20140370. [Google Scholar] [CrossRef] [PubMed][Green Version] - Aitsahlia, F.; Yoon, J.H. Information stages in efficient markets. J. Bank. Finance
**2016**, 69, 84–94. [Google Scholar] [CrossRef] - Ito, M.; Noda, A.; Wada, T. The evolution of stock market efficiency in the US: A non-Bayesian time-varying model approach. Appl. Econ.
**2016**, 48, 621–635. [Google Scholar] [CrossRef][Green Version] - Lee, S.H.; Kim, P.J.; Jeong, H. Statistical properties of sampled networks. Phys. Rev. E
**2006**, 73, 016102. [Google Scholar] [CrossRef] [PubMed][Green Version] - West, G. Scale: The Universal Laws of Growth, Innovation, Sustainability, and the Pace of Life in Organisms, Cities, Economies, and Companies; Penguin Press: New York, NY. USA, 2017. [Google Scholar]
- Boettke, P.J. What Happened to “Efficient Markets”? Indep. Rev.
**2010**, 14, 363–375. [Google Scholar]

**Figure 1.**Entropy and mutual information prefectures in Japan between 1994 and 2018. Plot (

**a**) shows the evolution of the total entropy H as a function of the total number of companies (nodes) $\left|C\right|$ during the period 1994–2018. Similarly, plots (

**b**,

**c**) show the equivalent joint entropy J and mutual information I as a function of total number of edges E. Each grey line represents the path taken single prefecture in Japan during the period 1994–2018. The coloured lines highlighting representative prefectures, selected from the largest (i.e., Tokyo) to smallest (i.e., Tottori) in terms of GDP (gross domestic product) and maintaining similar ranking intervals in between (Fukuoka, Kagoshima, and Wakayama). The circles point to the year 1994 whereas the x-cross relates to 2018.

**Figure 2.**Average decline/increase rate of the mutual information within prefectures in Japan, 1994 to 2018. The map on the left consists of a geographical heatmap for the average yearly rate of decline (

**red**) or increase (

**light green**) of the mutual information for each of the 47 prefectures as approximated by a linear fitting ${I}_{t}=a+b(t-{t}_{0})$. Each graph on the side corresponds to the evolution of the total mutual information I (y-axis) over the period 1994–2018 (x-axis) for (

**a**) Tokyo, (

**b**) Wakayama and (

**c**) Tottori prefectures.

**Figure 3.**Observed and estimated values for the structural mutual information $SMI$ and total mutual information I for the years 2013–2018. The

**top left**panel shows the comparative results between the values observed for structural mutual information $SMI$ (or $\tilde{I}$ as described within Methods) on the y-axis against those estimated by making use of Equation (11) on the x-axis. Each dot consists of a single prefecture within Japan at a specific year, with years colour mapped from lighter to darker shades, older to the most recent. The diagonal line represents the point where $y=x$. Similarly, the

**bottom left**panel shows the observed mutual information I on the y-axis, calculated in accordance with Equation (7), against those estimated by the model by making use of Equation (12) on the x-axis. On the

**right side**, the differences between the estimates to the actual values (x-axes) are ranked and plotted against the cumulative function of a normally distributed curve as shown by the red lines.

**Figure 4.**Pointwise contribution to the mutual information, ${I}_{(i,j)}$, for real and randomised networks. Each panel represents a heatmap of the pointwise contribution to mutual information for the directional edge combination ‘i’ (vertical axis) to ‘j’ (horizontal axis), calculated in accordance with Equation (8). Both axes are equal in value, consisting of the ranked sequence of the total degree distribution of companies for the relevant representative prefectures, Osaka in the top row and Kagoshima in the bottom row. The

**left**(all degrees) and

**centre**(zoomed degrees up to 20) panels show the contribution to the mutual information for the real network, whereas the

**right**panels show contribution related to the randomised network. The colour maps on the right show the intensity of the contribution, with different scales by prefecture, but the same for all panels for the selected prefecture. Darker colours are associated with higher numbers with blue being negative values, red being positive, and totally white being zero.

**Figure 5.**Average degree, population distribution, and cumulative mutual information values for selected Japanese prefectures in 2018. Panels (

**a**,

**b**) show the average degree of the neighbouring nodes [23] ‘${k}_{nn}\left(k\right)$’ (y-axis) of companies with total degree ‘k’ (x-axis, on a lognormal scale) for three selected prefectures: Osaka (magenta), Kagoshima (turquoise), and Saitama (red). Each dot represents the aggregate of companies of total degree ’k’ and the average of their neighbours ‘${k}_{nn}\left(k\right)$’ generated through a binning process with a minimum of 1000 edges (i.e., datapoints) per bin. Whereas (

**a**) relates to data extracted directly from the real network, (

**b**) shows the average values for 1000 randomised realisations. Panel (

**c**) consists of the total degree distribution of companies for the selected prefectures plotted on a log-log scale. The bottom panels (

**d,e**and

**f**) show the cumulative of the total mutual information I within the y-axis as a function of the degree distributions of companies, within the x-axis, on a lognormal scale. The left panel (

**d**) relates to data from the real network, whereas the centre panels (

**e**) consists of the average value of the mutual information I for each of the 1000 realisations adopting ${P}_{ij}={w}_{ij}$ as calculated by Equations (3) and (7). In contrast, the right panel (

**f**) consists of the calculation of a single value for mutual information ${I}_{{r}^{th}}$ for all aggregated realisations of ${P}_{ij}$ as described in Equation (10).

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Viegas, E.; Goto, H.; Kobayashi, Y.; Takayasu, M.; Takayasu, H.; Jensen, H.J.
Allometric Scaling of Mutual Information in Complex Networks: A Conceptual Framework and Empirical Approach. *Entropy* **2020**, *22*, 206.
https://doi.org/10.3390/e22020206

**AMA Style**

Viegas E, Goto H, Kobayashi Y, Takayasu M, Takayasu H, Jensen HJ.
Allometric Scaling of Mutual Information in Complex Networks: A Conceptual Framework and Empirical Approach. *Entropy*. 2020; 22(2):206.
https://doi.org/10.3390/e22020206

**Chicago/Turabian Style**

Viegas, Eduardo, Hayato Goto, Yuh Kobayashi, Misako Takayasu, Hideki Takayasu, and Henrik Jeldtoft Jensen.
2020. "Allometric Scaling of Mutual Information in Complex Networks: A Conceptual Framework and Empirical Approach" *Entropy* 22, no. 2: 206.
https://doi.org/10.3390/e22020206