# Self-Organized Complexity and Coherent Infomax from the Viewpoint of Jaynes’s Probability Theory

^{1}

^{2}

## Abstract

**:**

## 1. Introduction

## 2. Organized Complexity

## 3. Coherent Infomax

## 4. Jaynes’s Probability Theory

## 5. Relations Between Jaynes’s Probability Theory and Adaptively Self-Organized Complexity

## 6. Relations Between Jaynes’s Probability Theory and Coherent Infomax

#### 6.1. Challenges Faced by Theories of Self-Organized Inference in Neural Systems

#### 6.2. How Coherent Infomax Responds to These Challenges

#### 6.3. Can the Asymmetry Between the Effects of Contextual and Driving Inputs Be Related to Jaynes’s Probability Theory?

#### 6.4. What Are the Major Transitions in the Evolution of Inferential Capabilities?

## 7. Does Self-Organized Inference in Living Things Have an Objective?

## Acknowledgments

## References

- Jaynes, E.T. Probability Theory: The Logic of Science; Bretthorst, G.L., Ed.; Cambridge University Press: Cambridge, UK, 2003. [Google Scholar]
- Engel, C.; Singer, W. Better than Conscious? MIT Press: Cambridge, MA, USA, 2008. [Google Scholar]
- Phillips, W.A.; Kay, J.; Smyth, D. The discovery of structure by multi-stream networks of local processors with contextual guidance. Netw. Comput. Neural Syst.
**1995**, 6, 225–246. [Google Scholar] [CrossRef] - Kay, J.; Floreano, D.; Phillips, W.A. Contextually guided unsupervised learning using local multivariate binary processors. Neural Network.
**1998**, 11, 117–140. [Google Scholar] [CrossRef] - Kay, J.; Phillips, W.A. Activation functions, computational goals and learning rules for local processors with contextual guidance. Neural Comput.
**1997**, 9, 895–910. [Google Scholar] [CrossRef] - Kay, J.; Phillips, W.A. Coherent Infomax as a computational goal for neural systems. Bull. Math. Biol.
**2011**, 73, 344–372. [Google Scholar] [CrossRef] - Sporns, O. Complexity. Scholarpedia
**2007**, 2, 1623. [Google Scholar] [CrossRef] - von der Malsburg, C.; Singer, W. Principles of Cortical Network Organization. In Neurobiology of Neocortex; Rakic, P., Singer, W., Eds.; John Wiley & Sons: Chichester, UK, 1988. [Google Scholar]
- Rolston, H., III. Three Big Bangs: Matter-Energy, Life, Mind; Columbia University Press: New York, NY, USA, 2010. [Google Scholar]
- Phillips, W.A.; von der Malsburg, C.; Singer, W. Dynamic Coordination in Brain and Mind. In Dynamic Coordination in the Brain: From Neurons to Mind; von der Malsburg, C., Phillips, W.A., Singer, W., Eds.; MIT Press: Cambridge, MA, USA, 2010. [Google Scholar]
- Friston, K.J. The free-energy principle: A unified brain theory? Nat. Rev. Neurosci.
**2010**, 11, 127–138. [Google Scholar] [CrossRef] - Finger, S. Origins of Neuroscience; Oxford University Press: New York, NY, USA, 1994. [Google Scholar]
- Phillips, W.A.; Singer, W. In search of common foundations for cortical computation. Behav. Brain Sci.
**1997**, 20, 657–722. [Google Scholar] [CrossRef] - Dynamic Coordination in the Brain: From Neurons to Mind; von der Malsburg, C.; Phillips, W.A.; Singer, W. (Eds.) MIT Press: Cambridge, MA, USA, 2010.
- Phillips, W.A.; Silverstein, S.M. Convergence of biological and psychological perspectives on cognitive coordination in schizophrenia. Behav. Brain Sci.
**2003**, 26, 65–138. [Google Scholar] - Becker, S.; Hinton, G.E. A self-organizing neural network that discovers surfaces in random-dot stereograms. Nature
**1992**, 335, 161–163. [Google Scholar] - Creutzig, F.; Globerson, A.; Tishby, N. Past-future information bottleneck in dynamical systems. Phys. Rev.
**2009**, 79, 041925. [Google Scholar] - Kelso, J.A.S. Dynamic Patterns: The Self-Organization of Brain and Behavior; MIT Press: Cambridge, MA, USA, 1995. [Google Scholar]
- Körding, K.P.; König, P. Learning with two sites of synaptic integration. Netw. Comput. Neural Syst.
**2000**, 11, 1–15. [Google Scholar] [CrossRef] - Salinas, E.; Sejnowski, T.J. Gain modulation in the central nervous system: Where behavior, neurophysiology, and computation meet. Neuroscientist
**2001**, 7, 430–440. [Google Scholar] [CrossRef] - Spratling, M.W. Predictive-coding as a model of biased competition in visual attention. Vis. Res.
**2008**, 48, 1391–1408. [Google Scholar] [CrossRef] - Spratling, M.W.; Johnson, M.H. A feedback model of perceptual learning and categorisation. Vis. Cogn.
**2006**, 13, 129–165. [Google Scholar] [CrossRef] - Tononi, G.; Sporns, O.; Edelman, G.M. Complexity and coherency: Integrating information in the brain. Trends Cogn. Sci.
**1998**, 2, 474–484. [Google Scholar] [CrossRef] - Fiorillo, C.D. Towards a general theory of neural computation based on prediction by single neurons. PLoS One
**2008**, 3, e3298. [Google Scholar] [CrossRef] - Fiorillo, C.D. A neurocentric approach to Bayesian inference. Nat. Rev. Neurosci.
**2010**, 11, 605. [Google Scholar] - Jaynes, E.T. Probability Theory as Logic. In Maximum Entropy and Bayesian Methods; Fougere, P.H., Ed.; Kluwer Academic Publishers: Dordrecht, The Netherlands, 1990. [Google Scholar]
- Jaynes, E.T. Where do We Stand on Maximum Entropy? In The Maximum Entropy Formalism; Levine, R.D., Tribus, M., Eds.; MIT Press: Cambridge, MA, USA, 1979. [Google Scholar]
- Jaynes, E.T. Information theory and statistical mechanics. Phys. Rev.
**1957**, 106, 120–130. [Google Scholar] - Jaynes, E.T. Information theory and statistical mechanics. II. Phys. Rev.
**1957**, 108, 171–191. [Google Scholar] [CrossRef] - Friston, K.J.; Stephan, K.E. Free-energy and the brain. Synthese
**2007**, 159, 417–458. [Google Scholar] [CrossRef][Green Version] - Bellman, R.E. Adaptive Control Processes; Princeton University Press: Princeton, NJ, USA, 1961. [Google Scholar]
- Barlow, H.B. Possible Principles Underlying the Transformation of Sensory Messages. In Sensory Communication; Rosenblith, W.A., Ed.; MIT Press: Cambridge, MA, USA, 1961. [Google Scholar]
- Linsker, R. Self-organization in a perceptual network. Computer
**1988**, 21, 105–107. [Google Scholar] [CrossRef] - Dechter, R. From local to global consistency. Artif. Intell.
**1992**, 55, 87–107. [Google Scholar] - Silver, R.A. Neuronal arithmetic. Nat. Rev. Neurosci.
**2010**, 11, 474–489. [Google Scholar] [CrossRef] - Berger, A.L.; Della Pietra, S.A.; Della Pietra, V.J. A maximum entropy approach to natural language processing. Comput. Linguist.
**1996**, 22, 39–71. [Google Scholar] - Myers, C.L.; Troyanskaya, O.G. Context-sensitive data integration and prediction of biological networks. Bioinformatics
**2007**, 23, 2322–2330. [Google Scholar] [CrossRef] - Ginter, F.; Boberg, J.; Jarvinen, J.; Salakoski, T. New techniques for disambiguation in natural language and their application to biological text. J. Mach. Learn. Res.
**2004**, 5, 605–621. [Google Scholar] - Oaksford, M.; Chater, N. Bayesian rationality: The probabilistic approach to human reasoning. Behav. Brain Sci.
**2009**, 32, 69–120. [Google Scholar] [CrossRef] - Knill, D.C.; Pouget, A. The Bayesian Brain: The role of uncertainty in neural coding and computation. Trends Neurosci.
**2004**, 27, 712–719. [Google Scholar] [CrossRef] - Jones, M.; Love, B.C. Bayesian fundamentalism or enlightenment? On the explanatory status and theoretical contributions of Bayesian models to cognition. Behav. Brain Sci.
**2011**, 34, 169–188. [Google Scholar] [CrossRef] - Szmatháry, E.; Smith, J.M. The major evolutionary transitions. Nature
**1995**, 374, 227–232. [Google Scholar] - Phillips, W.A. How do Neural Systems Use Probabilistic Inference that is Context-Sensitive to Create and Preserve Organized Complexity? In Integral Biomathics: Tracing the Road to Reality; Simeonov, P.L., Smith, L.S., Ehresmann, A.C., Eds.; Springer: Berlin, Germany, 2011. [Google Scholar]
- Berhendt, R.-P.; Young, C. Hallucinations in schizophrenia, sensory impairment, and brain disease: A unifying model. Behav. Brain Sci.
**2004**, 27, 771–830. [Google Scholar] - Phillips, W.A. Belief in the primacy of fantasy is misleading and unnecessary. Behav. Brain Sci.
**2004**, 27, 802–803. [Google Scholar] - Fiorillo, C.D. Beyond Bayes: On the need for a unified and Jaynesian definition of probability and information within neuroscience. Information
**2012**. submitted for publication. [Google Scholar]

© 2012 by the authors; licensee MDPI, Basel, Switzerland. This article is an open-access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).

## Share and Cite

**MDPI and ACS Style**

Phillips, W.A. Self-Organized Complexity and Coherent Infomax from the Viewpoint of Jaynes’s Probability Theory. *Information* **2012**, *3*, 1-15.
https://doi.org/10.3390/info3010001

**AMA Style**

Phillips WA. Self-Organized Complexity and Coherent Infomax from the Viewpoint of Jaynes’s Probability Theory. *Information*. 2012; 3(1):1-15.
https://doi.org/10.3390/info3010001

**Chicago/Turabian Style**

Phillips, William A. 2012. "Self-Organized Complexity and Coherent Infomax from the Viewpoint of Jaynes’s Probability Theory" *Information* 3, no. 1: 1-15.
https://doi.org/10.3390/info3010001