# Thermodynamic Neural Network

## Abstract

**:**

## 1. Introduction

## 2. Results

#### 2.1. Model Concepts

- A node j is characterized by a state ${e}_{j}$ representing a potential. In general, the model supports any number of values of the node state on the interval ${e}_{j}\in [\phantom{\rule{0.166667em}{0ex}}-1,1]\phantom{\rule{0.166667em}{0ex}}$. For example, a binary node might assume a state ${e}_{j}\in \{-1,1\}$.
- An edge connecting nodes i and j is characterized by a real, symmetric weight, ${w}_{ij}$, describing a capacity to transport charge.
- Node i may apply a potential ${e}_{i}$ to an edge weight ${w}_{ij}$ and generate an edge charge ${q}_{ij}={e}_{i}{w}_{ij}$ that becomes the input to node j. Similarly, node j may apply a potential ${e}_{j}$ to the same edge weight and generate an edge charge ${q}_{ji}={e}_{j}{w}_{ij}$ that becomes the input to node i. In order to clarify the relationship between potentials and edges, we may sometimes designate potentials with two subscripts as ${e}_{ij}={e}_{i}$ and ${e}_{ji}={e}_{j}$.
- Positive and negative charges are independently conserved (i.e., they never sum to cancel each other) and communicate potential along which charges of opposite polarity should flow. By this means, externally applied potentials are able to diffuse through the network and connect to complementary potentials.
- For a given node j, charge conservation requires the aggregation of input node charges into four compartments, ${q}_{j}^{\pm \pm}={\sum}_{i}{e}_{i}^{\pm}{w}_{ij}^{\pm}$, distinguished by the signs of potentials ${e}_{i}^{\pm}$ and weights ${w}_{ij}^{\pm}$ that create the charge. Depending on the inputs, anywhere from one to four compartments may be populated with charge at the time of the node state decision.
- Node state selection optimizes the transfer of charge between pairs of competing compartments using Boltzmann statistics, illustrated as a “switch” in Figure 1.
- If two nodes are connected by an edge, then configurations in which the nodes have opposite potentials will be favored. Hence, if the nodes are arranged on a regular grid and connected locally, then domains of “anti-ferromagnetic” order typically emerge.

- The node state decision selects and connects two compartments and transfers complementary charge between them, while leaving unselected compartments disconnected. If the compartments are populated such that no complementary pairs exist, then the state selection becomes uniformly random. Residual charge is that portion of the charge remaining on the selected compartments after charge transfer, which is, in general, unavoidable, owing to the thermal fluctuations in the network.
- When a node is near equilibrium, residual charge in the selected compartments is dissipated to the thermal reservoir via updates to the associated edge weights according to a Boltzmann distribution. In general, these updates improve the ability of the node to transport charge among the selected compartments if similar conditions are encountered in the future.
- Edge weights associated with unselected compartments are not updated and the charge accumulated in the unselected compartments is retained. In this way, the node retains memory of “contrary” inputs (charges) and correlations (weights) that may be important to future decisions, which is intended to address the long standing “forgetting problem” in artificial neural networks.

- Node states are updated in a continuous round-robin Markov chain, which guarantees that at the time of updating any particular node all the other nodes in the network have already updated and thereby preserves temporal consistency within the network interactions.
- Updates may be either reversible or irreversible, depending on the node’s state of equilibration at the time of the update. A node can determine its state of equilibration by examining the fluctuations in its energy over time. If those fluctuations are small compared to the temperature, then the node may be deemed to be equilibrated and vice-versa.
- A reversible update, which happens when the node is non-equilibrated, corresponds to the node temporarily updating its compartments with new input charges and communicating its state to its connected nodes without updating edge weights. The purpose of the reversible update is to generate fluctuations in the network that explore its configuration space to search for an equilibrium without destroying its previously acquired structure.
- An irreversible update, which happens when the node is equilibrated, corresponds to the node permanently updating its compartments with new input charges, updating the edge weights as described above, and communicating its state to its connected nodes. The purpose of the irreversible update is to adapt the network to make it more effective at transporting charge in the future.
- This method of updating the network creates a continuous cycle of fluctuation, equilibration, dissipation and adaptation that connects and refines features at large spatial/short temporal scale (i.e., the collection of network node states), intermediate spatial/intermediate temporal scale (i.e., compartment charges) and small spatial/long temporal scale (i.e., edge weights) as the network evolves. In this way, the network can rapidly equilibrate to large-scale changes in its environment through the continuous refinement of its multiscale, internal organization.
- A range of network topologies are possible, including multi-dimensional grid networks with near-neighbor connectivity, probabilistically connected, gridded networks with a metric that determines the probability of connection, and random networks. In general, there is no imposition of hierarchy or “layers” upon the network, as is common in most neural network models, but these kinds of networks can also be supported. Because connected nodes are driven to orient anti-ferromagnetically, most network configurations are inherently “frustrated”, in that the nodes cannot find a way to satisfy this orientation with all their connected nodes. For a special class of networks that are partitioned into two groups (bi-partitioned networks), in which nodes of one partition can connect only to nodes in the opposite partition, this frustration can be avoided. Nearest neighbor grid networks are inherently bi-partitioned and are also attractive to study because they are easy to visualize.

#### 2.2. Model Details

#### 2.2.1. Network Model

- -
**e**refers to the set of node potentials {${e}_{1},{e}_{2}...{e}_{n}$},- -
**w**refers to the set of edge weights {${\mathit{w}}_{1},{\mathit{w}}_{2}...{\mathit{w}}_{n}$},- -
- ${\mathit{w}}_{j}$ refers to the set of weights {${w}_{1j},{w}_{2j}...{w}_{{m}_{j}j}$} associated with node j,
- -
**q**refers to the set of edge charges {${\mathit{q}}_{1},{\mathit{q}}_{2}...{\mathit{q}}_{n}$},- -
- ${\mathit{q}}_{j}$ refers to the set of edge charges {${q}_{1j},{q}_{2j}...{q}_{{m}_{j}j}$} associated with node j,
- -
- n refers to the number of nodes in the network, and
- -
- ${n}_{j}$ refers to the number of edges associated with node j.

**w**and

**q**and, thereby, sample the space of fluctuations in

**e**without modifying Equation (2). Conversely, irreversible updates do modify the edge states

**w**and

**q**and, thereby, also modify Equation (2).

#### 2.2.2. Compartment Model

#### 2.2.3. Node Model

- -
- minimizing residual charge on the node
- -
- maximizing charge transport through the node
- -
- avoiding attractors to node states ${e}_{j}=0$
- -
- employing “kinetic” factors to sharpen state decisions and direct residual charge dissipation and accumulation processes

#### 2.2.4. Edge Model

#### 2.2.5. Network Evolution Model

#### 2.2.6. External Bias Model

#### 2.2.7. Other Network Effects

#### 2.3. Model Simulation

#### 2.3.1. Isolated Networks

#### 2.3.2. Externally Biased Networks

## 3. Discussion

#### 3.1. Model Features

#### 3.2. Limitations and Speculation on Future Opportunities

## 4. Materials and Methods

## 5. Conclusions

## Supplementary Materials

## Funding

## Acknowledgments

## Conflicts of Interest

## References

- Glauber, R.J. Time-dependent statistics of the Ising model. J. Math. Phys.
**1963**, 4, 294–307. [Google Scholar] [CrossRef] - Suzuki, M.; Kubo, R. Dynamics of the Ising model near the critical point. I. J. Phys. Soc. Jpn.
**1968**, 24, 51–60. [Google Scholar] [CrossRef] - Kirkpatrick, S.; Sherrington, D. Infinite-ranged models of spin-glasses. Phys. Rev. B
**1978**, 17, 4384. [Google Scholar] [CrossRef] - Hopfield, J.J. Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. USA
**1982**, 79, 2554–2558. [Google Scholar] [CrossRef] [Green Version] - Amit, D.J.; Gutfreund, H.; Sompolinsky, H. Spin-glass models of neural networks. Phys. Rev. A
**1985**, 32, 1007. [Google Scholar] [CrossRef] - Bruce, A.; Gardner, E.; Wallace, D. Dynamics and statistical mechanics of the Hopfield model. J. Phys. Math. Gen.
**1987**, 20, 2909. [Google Scholar] [CrossRef] - Sompolinsky, H. Statistical mechanics of neural networks. Phys. Today
**1988**, 41, 70–80. [Google Scholar] [CrossRef] [Green Version] - Gutfreund, H. From statistical mechanics to neural networks and back. Phys. A Stat. Mech. Appl.
**1990**, 163, 373–385. [Google Scholar] [CrossRef] - Ackley, D.H.; Hinton, G.E.; Sejnowski, T.J. A learning algorithm for Boltzmann machines. Cogn. Sci.
**1985**, 9, 147–169. [Google Scholar] [CrossRef] - Hinton, G.E.; Sejnowski, T.J. Learning and relearning in Boltzmann machines. In Parallel Distributed Processing: Explorations in the Microstructure of Cognition; MIT Press: Cambridge, MA, USA, 1986; pp. 282–317. [Google Scholar]
- Levin, E.; Tishby, N.; Solla, S.A. A statistical approach to learning and generalization in layered neural networks. Proc. IEEE
**1990**, 78, 1568–1574. [Google Scholar] [CrossRef] - Albert, R.; Barabási, A.L. Statistical mechanics of complex networks. Rev. Mod. Phys.
**2002**, 74, 47. [Google Scholar] [CrossRef] [Green Version] - Watkin, T.L.; Rau, A.; Biehl, M. The statistical mechanics of learning a rule. Rev. Mod. Phys.
**1993**, 65, 499. [Google Scholar] [CrossRef] [Green Version] - Clark, J.W. Statistical mechanics of neural networks. Phys. Rep.
**1988**, 158, 91–157. [Google Scholar] [CrossRef] - Schrödinger, E. The Physical Aspect of the Living Cell; Cambridge University press: Cambridge, UK, 1944. [Google Scholar]
- Schneider, E.D.; Kay, J.J. Life as a manifestation of the second law of thermodynamics. Math. Comput. Model.
**1994**, 19, 25–48. [Google Scholar] [CrossRef] - Glansdorff, P.; Prigogine, I. On a general evolution criterion in macroscopic physics. Physica
**1964**, 30, 351–374. [Google Scholar] [CrossRef] - Schlögl, F. On the statistical foundation of the thermodynamic evolution criterion of Glansdorff and Prigogine. Ann. Phys.
**1967**, 45, 155–163. [Google Scholar] [CrossRef] - Jun, J.K.; Hübler, A.H. Formation and structure of ramified charge transportation networks in an. Proc. Natl. Acad. Sci. USA
**2005**, 102, 536–540. [Google Scholar] [CrossRef] [Green Version] - Yufik, Y.M. How the mind works: An exercise in pragmatism. In Proceedings of the 2002 International Joint Conference on Neural Networks, Honolulu, HI, USA, 12–17 May 2002; Volume 3, pp. 2265–2269. [Google Scholar]
- Onsager, L. Reciprocal relations in irreversible processes. I. Phys. Rev.
**1931**, 37, 405. [Google Scholar] [CrossRef] - Callen, H.B.; Welton, T.A. Irreversibility and generalized noise. Phys. Rev.
**1951**, 83, 34. [Google Scholar] [CrossRef] - Jarzynski, C. Nonequilibrium equality for free energy differences. Phys. Rev. Lett.
**1997**, 78, 2690. [Google Scholar] [CrossRef] [Green Version] - Crooks, G.E. Entropy production fluctuation theorem and the nonequilibrium work relation for free energy differences. Phys. Rev. E
**1999**, 60, 2721. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Still, S.; Sivak, D.A.; Bell, A.J.; Crooks, G.E. Thermodynamics of prediction. Phys. Rev. Lett.
**2012**, 109, 120604. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Paperin, G.; Green, D.G.; Sadedin, S. Dual-phase evolution in complex adaptive systems. J. R. Soc. Interface
**2010**, 8, 609–629. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Wolpert, D.H.; Macready, W.G. No free lunch theorems for optimization. IEEE Trans. Evol. Comput.
**1997**, 1, 67–82. [Google Scholar] [CrossRef] [Green Version] - Wang, Z.; Joshi, S.; Savel’ev, S.E.; Jiang, H.; Midya, R.; Lin, P.; Hu, M.; Ge, N.; Strachan, J.P.; Li, Z.; et al. Memristors with diffusive dynamics as synaptic emulators for neuromorphic computing. Nat. Mater.
**2017**, 16, 101. [Google Scholar] [CrossRef] [Green Version] - Hylton, T.; Conte, T.; DeBenedictis, E.; Ganesh, N.; Still, S.; Strachan, J.P.; Williams, R.S.; Alemi, A.; Altenberg, L.; Crooks, G.; et al. Thermodynamic Computing: Report Based on a CCC Workshop Held on 3–5 January 2019; Technical Report; Computing Community Consortium: Washington, DC, USA, 2019. [Google Scholar]
- DeBole, M.V.; Taba, B.; Amir, A.; Akopyan, F.; Andreopoulos, A.; Risk, W.P.; Kusnitz, J.; Otero, C.O.; Nayak, T.K.; Appuswamy, R.; et al. TrueNorth: Accelerating From Zero to 64 Million Neurons in 10 Years. Computer
**2019**, 52, 20–29. [Google Scholar] [CrossRef]

**Figure 1.**(

**left**) The Ising model is a network of nodes in which the node states (“spins”) ${e}_{i}$ interact via symmetric weights ${w}_{ij}$. Low-energy states are those that align the node state ${e}_{j}$ with the net total of the weighted interactions. (

**right**) The TNN model is also a network of nodes in which the node states (“potentials”) ${e}_{i}$ interact via symmetric weights ${w}_{ij}$, but nodes interact via the exchange of charge and low-energy node states are those that effectively transport charge among the node’s inputs while independently conserving positive and negative charge. Node state selection is a competition to connect two different pairs of compartments having opposite charge polarity and the same node input state polarity, illustrated as rotating switch.

**Figure 2.**(

**left**) Node state selection results in a “setting of the switch” that connects a pair of selected compartments (light pink) and relieves accumulated charge while unselected compartments (dark pink) retain their accumulated charge. (

**right**) A network of internal, adapting nodes (light blue) and external, biased nodes (green). A round robin Markov chain method (dashed blue arrows) continuously updates the node states, while edge states update only when a node is near equilibrium (red arrows) and executes an irreversible update (see Section 2.2.5). During these irreversible updates, edge weights associated with the selected compartments adapt through the dissipation of residual compartment charge (red arrows,

**left**and

**right**), while edge weights associated with unselected compartments are not updated. In order to transport complementary charge, node states favor connections to nodes with opposite polarities (nearest neighbor network shown).

**Figure 3.**The energy landscape of a node using Heaviside kinetic factors, which promote state selection near ${e}_{j}=\pm 1$. The energy is piecewise linear in the state, which is a convenient choice that allows a simple spacing of node energies by a characteristic energy scale (temperature).

**Figure 4.**Snapshots from the evolution of a two-dimensional, bi-partitioned network of 900 nodes with each node connected to its 4 nearest neighbors in the opposite partition with ${T}_{node}/{T}_{edge}=1$. Networks are dynamic and noisy owing to contact with the thermal bath. The propensity for the nodes to organize anti-symmetrically is evident in the checkboard appearance of the various regions of the images. Video S1 and at https://youtu.be/3WFO41aV9lg.

**Figure 5.**A different simulation of the same network as in Figure 4. Because antisymmetric order is challenging to visualize, in these images the order is displayed by reversing the sign of the node state in one of the partitions (i.e., on every other square on the checkboard). This change is applied only to the image display—the underlying order is still antiferromagnetic. When displaying images this way, domains appear as preferentially “white” or “black”. Video S2 and at https://youtu.be/8_dvWLFr4mA.

**Figure 6.**Three frames from the evolution (early to late from left to right) of a two dimensional, bi-partitioned network of 40,000 nodes with each node connected to its 16 nearest neighbors in the opposite partition at ${T}_{node}/{T}_{edge}=80$. The network evolves to a large single domain with local excitations. Video S3 and at https://youtu.be/1Dj59El93KE.

**Figure 7.**Three frames in the evolution of the same network as in Figure 6 at ${T}_{node}/{T}_{edge}=100$. The network evolves complex multiscale dynamics with domain nucleation and growth. Video S4 and at https://youtu.be/Ca9XEGyBytg.

**Figure 8.**Three frames in the evolution of the same network as in Figure 6 at ${T}_{node}/{T}_{edge}=120$. The network is dominated by faster, smaller scale dynamics as compared to Figure 7. Video S5 and at https://youtu.be/UUA08xwcMAQ.

**Figure 9.**Temporal evolution of selected network averages for the simulation of Figure 7—node energy, node entropy, order, and fluctuations vs simulation time. Energy is the node energy of Equation (5) averaged over all the nodes. Entropy is the sum of the nodes entropies derived from Equation (17) normalized to a maximum value of 40,000. Order is the average over all edges of (the negative of) the product of the edge’s connected node states. Fluctuations are the percentage of time that nodes choose a reversible update averaged over all the nodes. Ordering in the network is consistent with decreased node energy and entropy and increased order and reversible node updates.

**Figure 10.**A bi-partitioned network of 40,000 nodes randomly connected to 16 nodes in the opposite partition at ${T}_{node}/{T}_{edge}=120$ uniformly oscillates periodically between mostly dark and light states. (

**left and center**) Two frames from the evolution (https://youtu.be/1YA5xauI5Y0) of the network showing dark and light states. (

**right**) Color shows the average state value of the network nodes and illustrates the regularity of the oscillations. Video S6 and at https://youtu.be/1YA5xauI5Y0.

**Figure 11.**Snapshots from the evolution of a bi-partitioned network of 900 nodes with 4 nearest neighbor connections and a single recurrent connection per node at ${T}_{node}/{T}_{edge}=1$ with a single externally biased node polarizing the region in its vicinity. The sequence of images is from three different simulations with increasing bias strength (increasing size of fixed edge weights connecting the bias node to its neighbors) from left to right in the images. Larger bias creates a larger region of polarization. Domain polarization changes polarity as the biasing node changes sign. Videos S7–S9 and at https://youtu.be/8kiLYNyOMZ8, https://youtu.be/HD_kJCEqrYA, https://youtu.be/pZy6S5Huph4.

**Figure 12.**In the same network as in Figure 11, two biased nodes interact through the network. If the nodes are of opposite polarity and opposite partition (

**top left**), the network evolves a connection. If the nodes are of opposite polarity and the same partition (

**top right**) or of the same polarity and opposite partition (

**bottom left**), the network evolves a domain wall. If the nodes are of the same polarity and same partition (

**bottom right**), the nodes jointly polarize their surrounding region, but do not grow strong weights between them. Videos S10–S13 and at https://youtu.be/nj-juPln5b0, https://youtu.be/JdBHyPSUFL0, https://youtu.be/2N8zqGX0swM, https://youtu.be/tVmCSKUA8dQ.

**Figure 13.**Frames from the evolution of a 10,000-node, bi-partitioned, 16 nearest neighbor network with 2 recurrent connections per node and 10 pairs of bias nodes at ${T}_{node}/{T}_{edge}=100$. Each bias pair is composed of complementary nodes (opposite polarity and opposite partition) that change periodically in time, each of the 10 pairs with different periods. These four images show different configurations of the network as it adapts to changes in its inputs from early to late in the network evolution (left to right and top to bottom). As the edge weights grow, the domains become sharper and better connected. As the input nodes change polarity, the network rapidly adapts by creating, destroying and moving domain walls. In general, the network is challenged to connect and separate nodes into black and white domains according to their polarity and partition (see Figure 12). Video S14 and at https://youtu.be/xy-eivZ2vJg.

**Figure 14.**Temporal evolution of selected network averages for the simulation of Figure 13—node energy, node entropy, order, and fluctuations vs simulation time. Energy is the node energy of Equation (6) averaged over all the nodes. Entropy is the sum of the node entropies derived from Equation (17) normalized to a maximum value of 10,000. Order is the average over all edges of (the negative of) the product of the edge’s connected node states. Fluctuations are the percentage of time that nodes choose a reversible update averaged over all the nodes. Even as the bias nodes change polarities, the large-scale behavior of the network is well behaved. As compared to the unbiased network example of Figure 7 and Figure 9, this network shows lower energy and entropy and a higher degree of order and fluctuation, which is consistent with the application of external bias to the network.

**Figure 15.**Frames from the evolution of a network like that of Figure 13 at ${T}_{node}/{T}_{edge}=120$ and using smaller fixed weights to bias the network. The simulation periodically applies and removes the bias potentials: bias nodes are green when on and red when off. (

**upper left**) The network is not yet exposed to any bias inputs and the dynamics are like those of Figure 8. (

**upper right**) Bias has been applied and the network evolves to connect bias nodes as in Figure 13. (

**lower left & right**) With the bias removed, the network dynamics continue but are clearly influenced by the modular structure that emerged during the bias phase, seemingly resembling both the unbiased and biased stages of its evolution. Edge weight magnitudes are largely preserved during the unbiased phases of evolution even as they are continuously updated (not shown). Video S15 and at https://youtu.be/ctmWAu09qTE.

**Figure 16.**Temporal evolution of selected network averages of a 10,000-node, single partition randomly connected network with 16 connections per node plus 2 recurrent connections per node and 10 pairs of bias nodes at ${T}_{edge}/{T}_{node}=100$. Each pair is composed of opposite polarity nodes that change periodically in time, each of the 10 pairs with different periods. Unlike the networks of Figure 13 and Figure 15, which are bi-partitioned and locally connected, this network is inherently frustrated and must “carve out” connections amid a vast web of competing interactions. (

**top left**) Energy is the node energy of Equation (6) averaged over all the nodes. (

**top right**) Entropy is the sum of the nodes entropies from Equation (17) normalized to a maximum value of 10,000. (

**bottom left**) Order is the average over all edges of (the negative of) the product of the edge’s connected node states. (

**bottom left**) Fluctuations are the percentage of time that nodes choose a reversible update averaged over all the nodes. Although inherently frustrated by its connectivity, as evidenced by it relatively low order, the statistics of the network evolution are similar to those of the nearest-neighbor, bi-partitioned network of Figure 14, indicating that the network can effectively evolve connectivity among the bias nodes. Video S16 and at https://youtu.be/PEFAkAcMdVk.

© 2020 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Hylton, T.
Thermodynamic Neural Network. *Entropy* **2020**, *22*, 256.
https://doi.org/10.3390/e22030256

**AMA Style**

Hylton T.
Thermodynamic Neural Network. *Entropy*. 2020; 22(3):256.
https://doi.org/10.3390/e22030256

**Chicago/Turabian Style**

Hylton, Todd.
2020. "Thermodynamic Neural Network" *Entropy* 22, no. 3: 256.
https://doi.org/10.3390/e22030256