# Redundancy in Systems Which Entertain a Model of Themselves: Interaction Information and the Self-Organization of Anticipation

## Abstract

**:**

## 1. Introduction

_{ABC→AB:AC:BC}measures the (Shannon-type) interaction information generated in the system in addition to the lower-level (i.e., bilateral) interactions. In other words, R is generated at the level of the model entertained by an observer because of the assumption that the probability distributions are independent. Given this assumption of independence, the algebraic derivation of Q is valid because circular relations among the variables cannot occur under this condition and all probabilities add up to unity. However, R measures the error caused by this assumption if the distributions are not independent.

_{ABC→AB:AC:BC}in the modeled system. In a final section, I provide empirical examples of how one can measure, for example, the effects of intellectual organization as an order of expectations of textual organization in interactions among (three or more) attributes of documents.

## 2. Dually Layered Systems Entertaining a Model of Themselves

_{t+}

_{1}= ½ ± ½ √[1 – (4/a) x

_{t}]

## 3. The Measurement of Redundancy and Interaction Information

_{x}= −Σ

_{x}p

_{x}log

_{2}p

_{x}and H

_{xy}= −Σ

_{xy}p

_{xy}log

_{2}p

_{xy}[18]. When the distributions Σ

_{x}p

_{x}and Σ

_{y}p

_{y}are independent, T

_{xy}= 0 and H

_{xy}= H

_{x}+ H

_{y}. In all other cases, H

_{xy}< H

_{x}+ H

_{y}, and therefore T

_{xy}is positive [19]. The uncertainty which prevails when two probability distributions are combined is reduced by the transmission or mutual information between these distributions.

_{i}p

_{i}= 1) is violated.

_{ABC→AB:AC:BC}) is Shannon-type information: it is the surplus information potentially generated in the three-way interactions which cannot be accounted for by the two-way interactions.

_{ABC→AB:AC:BC}is necessarily generated. Q can be positive or negative (or zero) depending on the difference between R and I.

## 4. An Empirical Interpretation of R

_{ABC→AB:AC:BC}, Krippendorff’s iterative algorithm is available, and Q can be computed using Equation 4. Because I

_{ABC→AB:AC:BC}is a Shannon-type information it cannot be negative. μ* (= –Q) has to be added to (or subtracted from) I in order to find R.

_{ABC→AB:AC:BC}, μ*, and therefore R. From this perspective, the difference between R and I can be considered as the remaining redundancy of the model that is not consumed by the Shannon information contained in the empirical distributions. If R < I, this difference can be considered as remaining uncertainty.

## 5. Methods and Materials

**Figure 2.**Two matrices for n documents with m authors and k words can be combined to a third matrix of n documents versus (m + k) variables.

_{ABC→AB:AC:BC}, and therefore R as the bias of the model. Note that the variables generate interactions by relating to the dimensions in terms of factor loadings.

**Figure 3.**Forty-eight title words in rotated vector space [29].

^{3}cell values. This transformation into discrete data can be expected to generate another source of error. However, we develop our reasoning into empirical studies using bibliometric data. The various measures (I, R, and μ*) then can be appreciated in the bibliometrically informed analysis.

## 6. Results

**Figure 5.**Cosine-normalized network among the 43 title words occurring more than twice in the document set of Social Networks (2006–2008); cosine ≥ 0.2. The size of the nodes is proportionate to the logarithm of the frequency of occurrence; the width of lines is proportionate to the cosine values; colors are based on the k-core algorithm; layout is based on energy minimization in a system of springs [32].

**Figure 6.**Map based on bibliographic coupling of 395 references in the 102 articles from Social Networks; cosine ≥ 0.5; [30]. For the sake of readability a selection of 136 nodes (for the partitions 4 ≤ k ≤ 10) is indicated with legends.

#### 6.1. Social Networks (2006–2008)

**Figure 7.**Interaction information (I

_{ABC→AB:AC:BC}) and remaining redundancy (–μ* or Q) among the three main components in different dimensions and combinations of dimensions on the basis of Social Networks (2006–2008).

#### 6.2. Social Studies of Science (2004–2008)

**Figure 8.**Interaction information (I

_{ABC→AB:AC:BC}) and remaining redundancy (–μ* or Q) among the three main components in different dimensions and combinations of dimension on the basis of Social Studies of Science (2004–2008).

## 7. Discussion and Conclusions

## Acknowledgements

## References and Notes

- Krippendorff, K.W. Ross Ashby’s Information Theory: A Bit of History, Some Solutions to Problems, and What We Face Today. Int. J. Gen. Syst.
**2009**, 38, 189–212. [Google Scholar] [CrossRef] - Krippendorff, K. Information of Interactions in Complex Systems. Int. J. Gen. Syst.
**2009**, 38, 669–680. [Google Scholar] [CrossRef] - Leydesdorff, L. Configurational Information as Potentially Negative Entropy: The Triple Helix Model. Entropy
**2008**, 10, 391–410. [Google Scholar] [CrossRef] - Leydesdorff, L. Interaction Information: Linear and Nonlinear Interpretations. Int. J. Gen. Syst.
**2009**, 38, 681–685. [Google Scholar] [CrossRef] - Coveney, P.; Highfield, R. The Arrow of Time; Allen: London, UK, 1990. [Google Scholar]
- Rosen, R. Anticipatory Systems: Philosophical, Mathematical and Methodological Foundations; Pergamon Press: Oxford, UK, 1985. [Google Scholar]
- Dubois, D.M. Computing Anticipatory Systems with Incursion and Hyperincursion. In Computing Anticipatory Systems, CASYS-First International Conference; Dubois, D.M., Ed.; AIP: Woodbury, NY, USA, 1998; Volume 437, pp. 3–29. [Google Scholar]
- The following steps are included in the derivation:
- The steady state can be found by solving x
_{t}= x_{t}_{+1}as follows: - Leydesdorff, L. The Communication of Meaning in Anticipatory Systems: A Simulation Study of the Dynamics of Intentionality in Social Interactions. In Proceedings of the 8th Intern. Conf. on Computing Anticipatory Systems CASYS'07; Dubois, D.M., Ed.; AIP: Melville, NY, USA, 2008; Volume 1051, pp. 33–49. [Google Scholar]
- Leydesdorff, L.; Franse, S. The Communication of Meaning in Social Systems. Syst. Res. Behav. Sci.
**2009**, 26, 109–117. [Google Scholar] [CrossRef] - The second law of thermodynamics holds equally for probabilistic entropy, since S = k
_{B}H and k_{B}is a constant (the Boltzmann constant). The development of S over time is a function of the development of H, and vice versa. - Giddens, A. The Constitution of Society; Polity Press: Cambridge, UK, 1984. [Google Scholar]
- Monge, P.R.; Contractor, N.S. Theories of Communication Networks; Oxford University Press: New York, NY, USA, 2003. [Google Scholar]
- Ulanowicz, R.E. Growth and Development: Ecosystems Phenomenology; Springer-Verlag: San Jose, CA, USA, 1986. [Google Scholar]
- McGill, W.J. Multivariate Information Transmission. Psychometrika
**1954**, 19, 97–116. [Google Scholar] [CrossRef] - Yeung, R.W. Information Theory and Network Coding; Springer: New York, NY, USA, 2008. [Google Scholar]
- Shannon, C.E. A Mathematical Theory of Communication. Bell Syst. Techn. J.
**1948**, 27, 379–423, 623–356. [Google Scholar] [CrossRef] - Theil, H. Statistical Decomposition Analysis; North-Holland: Amsterdam/London, The Netherlands/UK, 1972. [Google Scholar]
- Abramson, N. Information Theory and Coding; McGraw-Hill: New York, NY, USA, 1973. [Google Scholar]
- Watanabe, S. Information Theoretical Analysis of Multivariate Correlation. IBM J. Res. Dev.
**1960**, 4, 66–82. [Google Scholar] [CrossRef] - Jakulin, A. Machine Learning Based on Attribute Interactions. Ph.D. Dissertation, University of Ljubljana, Ljubljana, Slovenia, 2005. [Google Scholar]
- Krippendorff, K.Q. An Interpreation of the Information Theoretical Q-Measures. In Progress in Cybernetics and Systems Research; Trappl, R., Klir, G.J., Pichler, F., Eds.; Hemisphere: New York, NY, USA, 1980; Volume VIII, pp. 63–67. [Google Scholar]
- Krippendorff, K. Information Theory. Structural Models for Qualitative Data; Sage: Beverly Hills, CA, USA, 1986. [Google Scholar]
- The factor model is generated in terms of independent dimensions among the cases; this independence can be measured in terms of factor scores.
- Unlike principal component analysis—which essentially rewrites the variation in the matrix—factor analysis assumes the specification of a model.
- Maturana, H.R. The Nature of the Laws of Nature. Syst. Res. Behav. Sci.
**2000**, 17, 459–468. [Google Scholar] [CrossRef] - Maturana, H.R. Biology of Language: The Epistemology of Reality. In Psychology and Biology of Language and Thought. Essays in Honor of Eric Lenneberg; Miller, G.A., Lenneberg, E., Eds.; Academic Press: New York, NY, USA, 1978; pp. 27–63. [Google Scholar]
- De Nooy, W.; Leydesdorff, L. How Can Configurations Be Studied in 2-Mode Networks? Configurational Information and “Structuration”. In Two-Mode Social Network Analysis Conference, Free University Amsterdam, Amsterdam, The Netherlands, 1–2 October 2009.
- Van Heurs, B.; Leydesdorff, L.; Wyatt, S. Turning to Ontology in STS? Turning to STS through “Ontology”. Unpublished work. 2009. [Google Scholar]
- Ahlgren, P.; Jarneving, B.; Rousseau, R. Requirement for a Cocitation Similarity Measure, with Special Reference to Pearson’s Correlation Coefficient. J. Am. Soc. Inf. Sci. Technol.
**2003**, 54, 550–560. [Google Scholar] [CrossRef] - Kamada, T.; Kawai, S. An Algorithm for Drawing General Undirected Graphs. Inf. Process. Lett.
**1989**, 31, 7–15. [Google Scholar] [CrossRef] - Pajek, version 1.24. Program for Large Network Analysis, freely available for academic use at http://vlado.fmf.uni-lj.si/pub/networks/pajek/ (accessed on 10 October 2009).
- Leydesdorff, L. Words and Co-Words as Indicators of Intellectual Organization. Res. Policy
**1989**, 18, 209–223. [Google Scholar] [CrossRef] - Kessler, M.M. Bibliographic Coupling between Scientific Papers. Am. Document.
**1963**, 14, 10–25. [Google Scholar] [CrossRef] - Leydesdorff, L.; Akdag Salah, A.A. Maps on the Basis of the Arts & Humanities Citation Index. J. Am. Soc. Inf. Sci. Technol.
**2010**, in press. [Google Scholar] - Nederhof, A.J. Bibliometric monitoring of research performance in the social sciences and the humanities: A review. Scientometrics
**2006**, 66, 81–100. [Google Scholar] [CrossRef] - Von Foerster, H. Observing Systems; (with an introduction of Francisco Varela ed.); Intersystems Publications: Seaside, CA, USA, 1982. [Google Scholar]
- Leydesdorff, L. Why Words and Co-Words Cannot Map the Development of the Sciences. J. Am. Soc. Inf. Sci. Technol.
**1997**, 48, 418–427. [Google Scholar] [CrossRef] - Husserl, E. Cartesian Meditations and the Paris Lectures; Martinus Nijhoff: The Hague, The Netherlands, 1929. [Google Scholar]

© 2010 by the authors. Licensee Molecular Diversity Preservation International, Basel, Switzerland.. This article is an open-access article distributed under the terms and conditions of the Creative Commons Attribution license ( http://creativecommons.org/licenses/by/3.0/).

## Share and Cite

**MDPI and ACS Style**

Leydesdorff, L.
Redundancy in Systems Which Entertain a Model of Themselves: Interaction Information and the Self-Organization of Anticipation. *Entropy* **2010**, *12*, 63-79.
https://doi.org/10.3390/e12010063

**AMA Style**

Leydesdorff L.
Redundancy in Systems Which Entertain a Model of Themselves: Interaction Information and the Self-Organization of Anticipation. *Entropy*. 2010; 12(1):63-79.
https://doi.org/10.3390/e12010063

**Chicago/Turabian Style**

Leydesdorff, Loet.
2010. "Redundancy in Systems Which Entertain a Model of Themselves: Interaction Information and the Self-Organization of Anticipation" *Entropy* 12, no. 1: 63-79.
https://doi.org/10.3390/e12010063