# Critical Dynamics Mediate Learning of New Distributed Memory Representations in Neuronal Networks

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Consolidation of New Memory Near Criticality in Attractor Neural Networks

## 3. Consolidation of a Fear Memory Results in Subcritical Neural Dynamics in the Mouse Hippocampus

## 4. Discussion

#### Experimental Methods

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Chialvo, D. Emergent Complex Neural Dynamics. Nat. Phys.
**2010**, 6, 744–750. [Google Scholar] [CrossRef] - Jiang, H.; Wang, S.; Huang, Y.; He, X.; Cui, H.; Zhu, X.; Zheng, Y. Phase transition of spindle-associated protein regulate spindle apparatus assembly. Cell
**2015**, 163, 108–122. [Google Scholar] [CrossRef] - Jacobs, W.M.; Frenkel, D. Phase Transitions in Biological Systems with Many Components. Biophys. J.
**2017**, 112, 683–691. [Google Scholar] [CrossRef] [PubMed][Green Version] - Feldman, D.E. The Spike-timing Dependence of Plasticity. Neuron
**2012**, 75, 556–571. [Google Scholar] [CrossRef] [PubMed] - Hubener, M.; Bonhoeffer, T. Searching for Engrams. Neuron
**2010**, 67, 363–371. [Google Scholar] [CrossRef] [PubMed][Green Version] - Baeg, E.H.; Kim, Y.B.; Kim, J.; Ghim, J.W.; Kim, J.J.; Jung, M.W. Learning-induced enduring changes in the functional connectivity among prefrontal cortical neurons. J. Neurosci.
**2007**, 27, 909–918. [Google Scholar] [CrossRef] - Lashley, K.S. In Search of the Engram. In Society of Experimental Biology, Physiological mechanisms in animal behavior (Society’s Symposium IV); Cambridge University Press: Cambridge, UK, 1980; pp. 454–482. [Google Scholar]
- Goshen, I.; Brodsky, M.; Prakash, R.; Wallace, J.; Gradinaru, V.; Ramakrishnan, C.; Deisseroth, K. Dynamics of Retrieval Strategies for Remote Memories. Cell
**2011**, 147, 678–689. [Google Scholar] [CrossRef][Green Version] - Adrian, E.D.; Zotterman, Y. The Impulses Produced by Sensory Nerve-endings. J. Physiol.
**1926**, 61, 151–171. [Google Scholar] [CrossRef] - Hopfield, J.J. Neural Networks and Physical Systems with Emergent Collective Computational Abilities. Proc. Natl. Acad. Sci. USA
**1982**, 79, 2554–2558. [Google Scholar] [CrossRef] - Ryan, T.J.; Roy, D.S.; Pignatelli, M.; Arons, A.; Tonegawa, S. Memory Engram Cells Retain Memory Under Retrograde Amnesia. Science
**2015**, 348, 1007–1013. [Google Scholar] [CrossRef] - Attardo, A.; Fitzgerald, J.E.; Schnitzer, M.J. Impermanence of Dendritic Spines in Live Adult CA1 Hippocampus. Nature
**2015**, 523, 592–596. [Google Scholar] [CrossRef] [PubMed] - Ognjanovski, N.; Schaeffer, S.; Wu, J.; Mofakham, S.; Maruyama, D.; Zochowski, M.; Aton, S.J. Parvalbumin-expressing Interneurons Coordinate Hippocampal Network Dynamics Required for Memory Consolidation. Nat. Commun.
**2017**, 8, 15039. [Google Scholar] [CrossRef] [PubMed] - Nicola, W.; Clopath, C. A Diversity of Interneurons and Hebbian Plasticity Facilitate Rapid Compressible Learning in the Hippocampus. Nat. Neurosci.
**2019**, 22, 1168–1181. [Google Scholar] [CrossRef] [PubMed] - Landau, L.D.; Lifshitz, E.M. Statistical Physics; Elsevier Science and Technology: New York, NY, USA, 2011. [Google Scholar]
- Beggs, J.M. The criticality hypothesis: how local cortical networks might optimize information processing. Philos. Trans. A Math. Phys. Eng. Sci.
**2008**, 366, 329–343. [Google Scholar] [CrossRef] - Hesse, J.; Gross, T. Self-organized criticality as a fundamental property of neural systems. Front. Syst Neurosci.
**2014**, 8, 166. [Google Scholar] [CrossRef] - Beggs, J.M.; Plenz, D. Neuronal Avalanches in Neocortical Circuits. J. Neurosci.
**2003**, 23, 11167–11177. [Google Scholar] [CrossRef][Green Version] - Petermann, T.; Thiagarajan, T.C.; Lebedev, M.A.; Nicolelis, M.A.L.; Chialvo, D.R.; Plenz, D. Spontaneous cortical activity in awake monkeys composed of neuronal avalanches. Proc. Natl. Acad. Sci. USA
**2009**, 106, 15921–15926. [Google Scholar] [CrossRef][Green Version] - Pasquale, V.; Massobrio, P.; Bologna, L.L.; Chiappalone, M.; Martinoia, S. Self-organization and neuronal avalanches in networks of dissociated cortical neurons. Neuroscience
**2008**, 153, 1354–1369. [Google Scholar] [CrossRef] - Gireesh, E.D.; Plenz, D. Neuronal avalanches organize as nested theta-and beta/gamma-oscillations during development of cortical layer 2/3. Proc. Natl. Acad. Sci. USA
**2008**, 105, 7576–7581. [Google Scholar] [CrossRef] - Tetzlaff, C.; Okujeni, S.; Egert, U.; Wörgötter, F.; Butz, M. Self-organized criticality in developing neuronal networks. PLoS Comput. Biol.
**2010**, 6, e1001013. [Google Scholar] [CrossRef] - Friedman, N.; Ito, S.; Brinkman, B.A.; Shimono, M.; Deville, R.E.; Dahmen, K.A.; Beggs, J.M.; Butler, T.C. Universal critical dynamics in high resolution neuronal avalanche data. Phys. Rev. Lett.
**2012**, 108, 208102. [Google Scholar] [CrossRef] [PubMed] - Priesemann, V.; Valderrama, M.; Wibral, M.; Le Van Quyen, M. Neuronal Avalanches Differ from Wakefulness to Deep Sleep–Evidence from Intracranial Depth Recordings in Humans. PLoS Comput. Biol.
**2013**, 9, e1002985. [Google Scholar] [CrossRef] [PubMed] - Fontenele, A.J.; de Vasconcelos, N.A.P.; Feliciano, T.; Aguiar, L.A.A.; Soares-Cunha, C.; Coimbra, B.; Porta, L.D.; Ribeiro, S.; Rodrigues, A.J.; Sousa, N.; et al. Criticality between Cortical States. Phys. Rev. Lett.
**2019**, 122, 208101. [Google Scholar] [CrossRef] [PubMed][Green Version] - Poil, S.S.; Hardstone, R.; Mansvelder, H.D.; Linkenkaer-Hansen, K. Critical-state dynamics of avalanches and oscillations jointly emerge from balanced excitation/inhibition in neuronal networks. J. Neurosci.
**2012**, 32, 9817–9823. [Google Scholar] [CrossRef] [PubMed] - Priesemann, V.; Wibral, M.; Valderrama, M.; Propper, R.; Le Van Quyen, M.; Geisel, T.; Triesch, J.; Nikolic, D.; Munk, M.H. Spike Avalanches in vivo Suggest a Driven, Slightly Subcritical Brain State. Front. Syst. Neurosci.
**2014**, 8, 108. [Google Scholar] [CrossRef] [PubMed] - Shew, W.L.; Yang, H.; Yu, S.; Roy, R.; Plenz, D. Information capacity and transmission are maximized in balanced cortical networks with neuronal avalanches. J. Neurosci.
**2011**, 31, 55–63. [Google Scholar] [CrossRef] [PubMed] - Yang, H.; Shew, W.L.; Roy, R.; Plenz, D. Maximal variability of phase synchrony in cortical networks with neuronal avalanches. J. Neurosci.
**2012**, 32, 1061–1072. [Google Scholar] [CrossRef] - Timme, N.M.; Marshall, N.J.; Bennett, N.; Ripp, M.; Lautzenhiser, E.; Beggs, J.M. Criticality maximizes complexity in neural tissue. Front. Physiol.
**2016**, 7, 425. [Google Scholar] [CrossRef] - Gautam, S.H.; Hoang, T.T.; McClanahan, K.; Grady, S.K.; Shew, W.L. Maximizing Sensory Dynamic Range by Tuning the Cortical State to Criticality. PLoS Comput. Biol.
**2015**, 11, e1004576. [Google Scholar] [CrossRef] - Shew, W.L.; Plenz, D. The functional benefits of criticality in the cortex. Neuroscientist
**2013**, 19, 88–100. [Google Scholar] [CrossRef] - Ognjanovski, N.; Maruyama, D.; Lashner, N.; Zochowski, M.; Aton, S.J. CA1 Hippocampal Network Activity Changes During Sleep-Dependent Memory Consolidation. Front. Syst. Neurosci.
**2014**, 8, 61. [Google Scholar] [CrossRef] [PubMed] - Graves, L.A.; Heller, E.A.; Pack, A.I.; Abel, T. Sleep Deprivation Selective Impairs Memory Consolidation for Contextual Fear Conditioning. Learn. Mem.
**2003**, 10, 168–176. [Google Scholar] [CrossRef] [PubMed] - Rasch, B.; Born, J. About Sleep’s Role in Memory. Physiol. Rev.
**2013**, 93, 681–766. [Google Scholar] [CrossRef] [PubMed] - Amit, D.J.; Gutfreund, H.; Sompolinsky, H. Statistical Mechanics of Neural Networks Near Saturation. Ann. Phys.
**1987**, 173, 30–67. [Google Scholar] [CrossRef] - Amit, D.J. Modeling Brain Function: The World of Attractor Neural Networks; Cambridge University Press: Cambridge, NY, USA, 1989; pp. 304–308. [Google Scholar]
- Watts, D.J.; Strogatz, S.H. Collective Dynamics of ″Small-World″ Networks. Nature
**1998**, 393, 440–442. [Google Scholar] [CrossRef] [PubMed] - Giri, B.; Miyawaki, H.; Mizuseki, K.; Cheng, S.; Diba, K. Hippocampal Reactivation Extends for Several Hours Following Novel Experience. J. Neurosci.
**2019**, 39, 866–875. [Google Scholar] [CrossRef] [PubMed] - Wilson, M.A.; McNaughton, B.L. Reactivation of Hippocampal Ensemble Memories During Sleep. Science
**1994**, 265, 676–679. [Google Scholar] [CrossRef][Green Version] - Del Papa, B.; Priesemann, V.; Triesch, J. Criticality Meets Learning: Criticality Signatures in a Self-Organizing Recurrent Neural Network. PLoS ONE
**2017**, 12, e0178683. [Google Scholar] [CrossRef] - Bi, G.; Poo, M. Synaptic Modification by Correlated Activity: Hebb’s Postulate Revisted. Annu. Rev. Neurosci.
**2001**, 24, 139–166. [Google Scholar] [CrossRef] - Shew, W.L.; Yang, H.; Petermann, T.; Roy, R.; Plenz, D. Neuronal Avalanches Impoly Maximum Dynamic Range in Cortical Networks at Criticality. J. Neurosci.
**2009**, 29, 15595–15600. [Google Scholar] [CrossRef] - Bak, P.; Tang, C.; Wiesenfeld, K. Self-organized criticality: an explanation of the 1/f Noise. Phys. Rev. Lett.
**1987**, 59, 381–384. [Google Scholar] [CrossRef] [PubMed] - Wilting, J.; Priesemann, V. Inferring collective dynamical states from widely unobserved systems. Nat. Commun.
**2018**, 9, 2325. [Google Scholar] [CrossRef] [PubMed] - Wilting, J.; Dehning, J.; Pinheiro Neto, J.; Rudelt, L.; Wibral, M.; Zierenberg, J.; Priesemann, V. Operating in a Reverberating Regime Enables Rapid Tuning of Network States to Task Requirements. Front. Syst. Neurosci.
**2018**, 12, 1–8. [Google Scholar] [CrossRef] [PubMed] - Munoz, M.A.; Juhasz, R.; Castellano, C.; Odor, G. Griffiths Phases on Complex Networks. Phys. Rev. Lett.
**2010**, 105, 128701. [Google Scholar] [CrossRef] - Hsu, D.; Beggs, J.M. Neuronal Avalanches and Criticality: A Dynamical Model for Homeostasis. Neurocomputing
**2012**, 69, 1134–1136. [Google Scholar] [CrossRef] - De Andrade Costa, A.; Copelli, M.; Kinouchi, O. Can dynamical synapses produce true self-organized criticality? J. Stat. Mech.
**2015**, 2015. [Google Scholar] [CrossRef] - Uhlig, M.; Levina, A.; Geisel, T.; Herrmann, J.M. Critical Dynamics in Associative Memory Networks. Front. Comput. Neurosci.
**2013**, 7, 87. [Google Scholar] [CrossRef] - Hengen, K.B.; Torrado Pacheco, A.; McGregor, J.N.; Van Hooser, S.D.; Turrigiano, G.G. Neuronal Firing Rate Homeostasis is Inhibited by Sleep and Promoted by Wake. Cell
**2016**, 165, 180–191. [Google Scholar] [CrossRef] - Tononi, G.; Cirelli, C. Sleep and the price of plasticity: from Synaptic and Cellular Homeostasis to Memory Consolidation and Integration. Neuron
**2014**, 81, 12–34. [Google Scholar] [CrossRef] - Kossio, F.Y.K.; Goedeke, S.; van den Akker, B.; Ibarz, B.; Memmesheimer, R.M. Growing Critical: Self-Organized Criticality in a Developing Neural System. Phys. Rev. Lett.
**2018**, 121, 058301. [Google Scholar] [CrossRef][Green Version] - Zierenberg, J.; Wilting, J.; Priesemann, V. Homeostatic Plasticity and External Input Shape Neural Network Dynamics. Phys. Rev. X
**2018**, 8, 1–15. [Google Scholar] [CrossRef] - Prince, T.M.; Wimmer, M.; Choi, J.; Havekes, R.; Aton, S.; Abel, T. Sleep Deprivation During a Specific 3-Hour Time Window Post-Training Impairs Hippocampal Synaptic Plasticity and Memory. Neurobiol. Learn. Mem.
**2014**, 109, 122–130. [Google Scholar] [CrossRef] [PubMed]

**Figure 1.**New memory consolidation occurs only near criticality. (

**A**) Overlap of the system with the native configuration without external input (solid black line) and with external input (dashed black line), as well as overlap with the new configuration (red) represented by external input, as a function of temperature before learning. Note that maximal susceptibility of the new configuration only occurs near the initial critical temperature of the system, where overlap with the native configuration declines. Here, we define the critical temperature to be the temperature where the order parameter (Overlap) reaches its half-maximal value, as indicated by the blue line. (

**B**) Overlap of the new configuration after learning for neurons grouped based on their number of connections to the input. Colors represent pre-learning sub- (blue) super- (black) and critical (red) temperatures. (

**C**) Overlap of the system with the native (black) and new (red) configurations as a function of system temperature after learning. Few changes in overlap occur before the initial critical temperature, after which (near criticality) the system aligns to the new configuration. Note also that the new configuration overlap occurs for larger values of temperature, indicating consolidation and a shift in critical temperature due to learning. All error bars in (

**A**–

**C**) represent the standard error of the mean.

**Figure 2.**Robustness of new memory consolidation as a function of input strength. (

**A**) Peak change in overlap of the new state between pre- and post-learning as a function of input size (percentage of fixed nodes in the network) and strength (${w}^{e}$). The asterisk represents the parameters used to generate the data showed in Figure 1. (

**B**) Change in overlap (color) between the new and native configurations post-learning as a function of temperature for increasing values of external field strength applied during learning. Blue colors represent cases where the native configuration is still stable after learning, red colors are where the new configuration is stable, and green is where neither configuration is stable. Note that for sufficiently high field strength, we see a slight increase in the maximal critical temperature.

**Figure 3.**Dynamical properties of consolidating new information. (

**A**) Time (steps) required for the system to consolidate the new configuration, as a function of temperature. Values not shown (on the left and right sides) indicate timescales longer than the simulation runtime, i.e. that it takes a prohibitively long time to consolidate a new memory. (

**B**) Data and fit sigmoidal functions for mean number of changes in the neurons’ state S

_{i}per iteration as a function of temperature, pre- (solid black line) and post-learning (dashed lines); the learning rate ε increases left-to-right and from darker to lighter colors of the dashed lines. Error bars represent the standard error of the mean. The horizontal line labeled Tc represents the half-maximal point of the where we calculate the critical temperature via linear interpolation. (

**C**) Change in observed critical temperature ${T}_{c}$, calculated using the sigmoidal half-maximum values (

**B**) as a function of the learning rate ε. Colors correspond to the curves shown in (

**B**). (

**D**) Critical temperature ${T}_{c}$ as a function of the memories per degree distribution $\alpha $ before (black points) and after (red shaded region) learning for a new-state connectivity strength of ${w}^{e}=3.0$. Note that the minimal value of the critical temperature for the new configuration post-learning closely matches the critical temperature pre-learning, but that the effect of learning is a broadening of the stable regime.

**Figure 4.**Branching parameter and its changes as a function of quality of memory consolidation during SWS sleep. (

**A**) Percentage of freezing behavior observed in mice before (baseline) and after learning (post-cond.) for sham (circles) and CFC (squares) groups. Different colors represent different mice. (

**B**) Branching parameters σ during SWS before (baseline) and after learning (post-cond.). Colors and shapes are conserved as in (

**A**). Error bars represent the standard error of the mean, calculated for each mouse over all intervals. (

**C**) Change in freezing behavior vs change in branching parameter across the learning interval. Error bars represent the propagation of standard errors between Pre and Post in (

**B**). (

**D**) Mean change in branching parameter within each group. Error bars represent the standard error of the mean. * p < 0.10 confidence interval that the reduction was significant; ** p < 0.02 confidence interval that the reduction was significant, using the one-way T test.

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Skilling, Q.M.; Ognjanovski, N.; Aton, S.J.; Zochowski, M.
Critical Dynamics Mediate Learning of New Distributed Memory Representations in Neuronal Networks. *Entropy* **2019**, *21*, 1043.
https://doi.org/10.3390/e21111043

**AMA Style**

Skilling QM, Ognjanovski N, Aton SJ, Zochowski M.
Critical Dynamics Mediate Learning of New Distributed Memory Representations in Neuronal Networks. *Entropy*. 2019; 21(11):1043.
https://doi.org/10.3390/e21111043

**Chicago/Turabian Style**

Skilling, Quinton M., Nicolette Ognjanovski, Sara J. Aton, and Michal Zochowski.
2019. "Critical Dynamics Mediate Learning of New Distributed Memory Representations in Neuronal Networks" *Entropy* 21, no. 11: 1043.
https://doi.org/10.3390/e21111043