# Memory Systems, the Epistemic Arrow of Time, and the Second Law

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Albert and Loewer on the Asymmetry of Records

The SM [i.e., statistical–mechanical] probability distribution embodies a way in which “the future” (i.e., the temporal direction away from the time at which PH [i.e., the Past Hypothesis] obtains) is “open” at least insofar as macro states are being considered. Since all histories must satisfy the PH, they are very constrained at one boundary condition, but there is no similar constraint at other times. It is true that (almost) all histories eventually end up in an equilibrium state (there is a time at which almost all histories are in an equilibrium state), but this is not a constraint, it is a consequence of the dynamics and the PH, and it is not very constraining (almost all states are equilibrium states). Another feature of the SM distribution when applied to the macro state of the kind of world we find ourselves in is that the macro state of the world at any time is compatible with micro states that lead to rather different macro futures. For example, conditional on the present macro state of the world, the SM probability distribution may assign substantial chances both to it raining and not raining tomorrow. On the other hand, there is typically much less branching towards the past. The reason is that the macro states that arise in our world typically contain many macroscopic signatures (i.e., macro states/events that record other macro states/events) of past events but fewer macroscopic signatures of future states/events. Newspapers are much more accurate in recording past weather than in predicting future weather. Of course, these two features of the SM distribution—that histories are very constrained at one boundary condition but not at other times and that they branch much more to the future (direction away from the PH)—are related.(pp. 302–303, [15])

## 3. Three Types of Memory Systems

#### 3.1. Intuitive Examples of Memory Systems

#### 3.2. How Memory Systems Work

## 4. Formal Definitions of Memory Systems

**memory systems**. We consider three types of memory systems, which differ from one another depending on whether the memory is based on value ${m}_{0}$, on value ${w}_{0}$, or on value ${m}_{0}$ combined with some knowledge about how the laws of physics arise in the joint dynamics of $M\times W$.

#### 4.1. Restricted Mutual Information

**restricted**mutual information between A and B. We write it as ${I}_{\mathcal{C}}(A;B)$, with value $c=1$ being implicit.

#### 4.2. The Three Types of Memory Systems

**Definition**

**1.**

**Type-1**memory is any stochastic process over space $M\times W$ where there is some set ${M}^{*}$ such that ${I}_{{m}_{0}\in {M}^{*}}({W}_{1};{M}_{0})$ is large.

**Definition**

**2.**

**Type-2**memory is any stochastic process over space $M\times W$ where there is some set ${W}^{*}$ such that ${I}_{{w}_{0}\in {W}^{*}}({W}_{1};{M}_{0})$ is large.

**Definition**

**3.**

**Type-3**memory is any stochastic process over space $M\times W$ where:

- (1)
- There is an ${m}^{\u2020}\in M$ and a set ${M}^{*}$ such that ${I}_{{m}_{1}={m}^{\u2020},{m}_{0}\in {M}^{*}}({W}_{1};{M}_{0})$ is large.
- (2)
- There is a set ${M}^{\prime}\subseteq M$ such that for all ${m}_{0}\in {M}^{*}$,
- (a)
- $P({m}_{2}\in {M}^{\prime}\phantom{\rule{0.166667em}{0ex}}|\phantom{\rule{0.166667em}{0ex}}{m}_{0})$ is close to 1.
- (b)
- $P\left({m}_{1}\phantom{\rule{0.166667em}{0ex}}\right|\phantom{\rule{0.166667em}{0ex}}{m}_{2},{m}_{0})$ is a highly peaked distribution about ${m}_{1}={m}^{\u2020}$, for all ${m}_{2}\in {M}^{\prime}$.
- (c)
- ${w}_{1}$ is conditionally independent from ${m}_{2}$, given ${m}_{0}$ and given that ${m}_{1}={m}^{\u2020}$. In other words,$$\begin{array}{c}P\left({w}_{1}\phantom{\rule{0.166667em}{0ex}}\right|\phantom{\rule{0.166667em}{0ex}}{m}_{0},{m}_{1}={m}^{\u2020},{m}_{2})=P\left({w}_{1}\phantom{\rule{0.166667em}{0ex}}\right|\phantom{\rule{0.166667em}{0ex}}{m}_{0},{m}_{1}={m}^{\u2020})\end{array}$$

**Lemma**

**1.**

- (1)
- For any ${m}_{0}\in {M}^{*}$ and any ${w}_{1}$,$$\begin{array}{c}P\left({w}_{1}\phantom{\rule{0.166667em}{0ex}}\right|\phantom{\rule{0.166667em}{0ex}}{m}_{0})\simeq P\left({w}_{1}\phantom{\rule{0.166667em}{0ex}}\right|\phantom{\rule{0.166667em}{0ex}}{m}_{0},{m}_{1}={m}^{\u2020})\end{array}$$and since this holds for all ${m}_{0}\in {M}^{*}$,$$\begin{array}{cc}P\left({w}_{1}\phantom{\rule{0.166667em}{0ex}}\right|\phantom{\rule{0.166667em}{0ex}}{m}_{0},{m}_{0}\in {M}^{*})& \simeq P\left({w}_{1}\phantom{\rule{0.166667em}{0ex}}\right|\phantom{\rule{0.166667em}{0ex}}{m}_{0},{m}_{1}={m}^{\u2020},{m}_{0}\in {M}^{*})\hfill \end{array}$$
- (2)
- For any ${m}_{1}$,$$\begin{array}{cc}P\left({m}_{1}\right|{m}_{0}\in {M}^{*})& \simeq \delta ({m}_{1},{m}^{\u2020})\hfill \end{array}$$
- (3)
- For any ${m}_{0}$,$$\begin{array}{cc}P\left({m}_{0}\right|{m}_{0}\in {M}^{*})& \simeq P\left({m}_{0}\right|{m}_{1}={m}^{\u2020},{m}_{0}\in {M}^{*})\hfill \end{array}$$
- (4)
- For any ${w}_{1}$,$$\begin{array}{cc}P\left({w}_{1}\phantom{\rule{0.166667em}{0ex}}\right|\phantom{\rule{0.166667em}{0ex}}{m}_{0}\in {M}^{*})& \simeq P\left({w}_{1}\phantom{\rule{0.166667em}{0ex}}\right|\phantom{\rule{0.166667em}{0ex}}{m}_{1}={m}^{\u2020},{m}_{0}\in {M}^{*})\hfill \end{array}$$

**Proof.**

**Theorem**

**1.**

**Proof.**

#### 4.3. Illustrations of Our Formal Definitions

- Consider an image on a chemical photographic film in an instant camera. M is the possible patterns on the surface of the film; ${M}^{*}$ is all such patterns aside from those that indicate the camera holding the film was incorrectly exposed to the outside world, e.g., resulting in a fogged image on the surface of the film. ${m}^{\u2020}$ is the initialized state of the film, with no image, before exposure of any sort. It has low entropy, and is formed in an entropy-increasing chemical initialization process that involves some external set of chemicals, K. W is an external photon field, which results in an image being made some time between ${t}_{1}$ and ${t}_{0}$ if the camera exposes the film correctly, i.e., if ${m}_{0}\in {M}^{*}$.
- Suppose we come across a cave and find that inside of it, some of the stones scattered about the floor (which evidently had originally been part of the roof) are arranged in letters, spelling “Help!”. In this case, M is (a coarse-graining of) the possible patterns of stones on the floor of the cave. ${m}^{\u2020}$ is the pattern where the stones are scattered uniformly randomly. We rely on the second law to presume that the joint state of the cave (including, in particular, its roof and the pattern of stones on its floor) was in ${m}^{\u2020}$ some time in the past. This allows inferring that some subsystem of W (in this case, some English-speaking human) interfered with M at some time between when in the past it was initialized to ${m}^{\u2020}$, and the present, when the stones spell out “Help!”. Intuitively, this example is just like footprints on the beach, where the analog of the smoothed beach surface is the initially random positions of stones on the cave floor (notice that this is a high-entropy state!), and the analog of the trail of footprints is some of the stones being arranged to spell “Help!”.
- Suppose we took some photographs through a telescope of the positions of the planets of the solar system which (together with other recorded information gathered from different positions on the surface of the Earth) allow us inferring their current positions and velocities. Those photographs and recordings are jointly a Type-3 memory system (see discussion just above of the Type-3 memory system of an image on a photographic film). Note that we can evolve what we infer from the current state of this memory system—the current phase space position of the planets in the solar system—into the future, after time ${t}_{0}$. In this, the current value, ${m}_{0}$, of the memory system provides information about the future, not just the past. However, the key is that the recordings are a Type-3 memory system, and they provide information about the (recent) past. The fact that that information provides predictions concerning the future is a red herring.

#### 4.4. Discussion of Our Formal Definitions

## 5. Memory Systems, Records, and the Epistemic Arrow

^{15}(?) dynamically relevant degrees of freedom. So setting M to be part of the memory of just one of those servers, $\left|W\right|$ is on the order of Avogadro’s number. Yet, such computer systems are examples of Type-2 memory systems.) However, running a Type-2 memory system with a large W seems to require a huge number of energy barriers keeping trajectories of $M\times {Z}_{2}$ well separated during the evolution of the joint system, with high probability, i.e., such systems use a huge amount of error correction; this is certainly true in cloud computers. Systems with this property seem to only arise with careful engineering by humans. In contrast, memory systems like footprints on a beach do not rely on anything close to that number of energy barriers, allowing the stochastic process governing the dynamics of microstate trajectories spreading out more readily. This may be why they can occur in systems that are not artificially constructed; see discussion of the Past Hypothesis in Section 6.

## 6. The Past Hypothesis and the Second Law

**Example**

**1.**

## 7. Future Work and Open Issues

## Author Contributions

## Funding

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## References

- Carroll, S. From Eternity to Here: The Quest for the Ultimate Theory of Time; Dutton: New York, NY, USA, 2010. [Google Scholar]
- Grünbaum, A. Philosophical Problems of Space and Time; Reidel: Boston, MA, USA, 1963. [Google Scholar]
- Hartle, J.B. The physics of now. Am. J. Phys.
**2005**, 73, 101–109. [Google Scholar] [CrossRef] - Hawking, S. The no-boundary proposal and the arrow of time. Vistas Astron.
**1993**, 37, 559–568. [Google Scholar] [CrossRef] - Horwich, P. Asymmetries in Time: Problems in the Philosophy of Science; MIT Press: Cambridge, MA, USA, 1987. [Google Scholar]
- Reichenbach, H. The Direction of Time; Dover Publications: Mineola, NY, USA, 1956. [Google Scholar]
- Rovelli, C. The Order of Time; Riverhead Books: New York, NY, USA, 2018. [Google Scholar]
- Rovelli, C. Memory and entropy. Entropy
**2022**, 24, 1022. [Google Scholar] [CrossRef] [PubMed] - Schulman, L.S. A computer’s arrow of time. Entropy
**2005**, 7, 221–233. [Google Scholar] [CrossRef] - Stradis, A. Memory, the fork asymmetry, and the initial state. Synthese
**2021**, 199, 9523–9547. [Google Scholar] [CrossRef] - Albert, D.Z. Time and Chance; Harvard University Press: Cambridge, MA, USA, 2000. [Google Scholar]
- Albert, D.Z. The sharpness of the distinction between the past and the future. In Chance and Temporal Asymmetry; Wilson, A., Ed.; Oxford University Press: Oxford, UK, 2014. [Google Scholar]
- Albert, D.Z. After Physics; Harvard University Press: Cambridge, MA, USA, 2015. [Google Scholar]
- Albert, D.Z. Conclusion. In The Probability Map of the Universe: Essays on David Albert’s Time and Chance; Loewer, B., Weslake, B., Winsberg, E., Eds.; Harvard University Press: Cambridge, MA, USA, 2023; pp. 351–374. [Google Scholar]
- Loewer, B. Counterfactuals and the second law. In Causation, Physics, and the Constitution of Reality: Russell’s Republic Revisited; Price, H., Corry, R., Eds.; Oxford University Press: Oxford, UK, 2007. [Google Scholar]
- Loewer, B. The emergence of time’s arrows and special science laws from physics. Interface Focus
**2012**, 2, 13–19. [Google Scholar] [CrossRef] [PubMed] - Loewer, B. Two accounts of laws and time. Philos. Stud.
**2012**, 160, 115–137. [Google Scholar] [CrossRef] - Wolpert, D.H. Memory systems, computation, and the second law of thermodynamics. Int. J. Theor. Phys.
**1992**, 31, 743–785. [Google Scholar] [CrossRef] - Davies, P.C.W. The Physics of Time Asymmetry; University of California Press: Berkeley, CA, USA, 1977. [Google Scholar]
- Jaynes, E.T.; Bretthorst, G.L. Probability Theory: The Logic of Science; Cambridge University Press: Cambridge, UK, 2003. [Google Scholar]
- Earman, J. The Past Hypothesis: Not even false. Stud. Hist. Philos. Mod. Phys.
**2006**, 37, 399–430. [Google Scholar] [CrossRef] - Frisch, M. Counterfactuals and the past hypothesis. Philos. Sci.
**2005**, 72, 739–750. [Google Scholar] [CrossRef] - Frisch, M. Does a low-entropy constraint prevent us from influencing the past? In Time, Chance, and Reduction: Philosophical Aspects of Statistical Mechanics; Hüttemann, A., Ernst, G., Eds.; Cambridge University Press: Cambridge, UK, 2007; pp. 13–33. [Google Scholar]
- Frisch, M. Causes, randomness, and the past hypothesis. In The Probability Map of the Universe: Essays on David Albert’s Time and Chance; Loewer, B., Weslake, B., Winsberg, E., Eds.; Harvard University Press: Cambridge, MA, USA, 2023; pp. 294–311. [Google Scholar]
- Huggett, N. Reading the past in the present. In The Probability Map of the Universe: Essays on David Albert’s Time and Chance; Loewer, B., Weslake, B., Winsberg, E., Eds.; Harvard University Press: Cambridge, MA, USA, 2023; pp. 271–293. [Google Scholar]
- Winsberg, E. Can conditioning on the Past Hypothesis militate against the reversibility objections? Philos. Sci.
**2004**, 71, 489–504. [Google Scholar] [CrossRef] - Kleidon, A. Maximum entropy production and general trends in biospheric evolution. Paleontol. J.
**2009**, 43, 980–985. [Google Scholar] [CrossRef] - Kleidon, A. Nonequilibrium thermodynamics and maximum entropy production in the earth system: Applications and implications. Naturwissenschaften
**2009**, 96, 1–25. [Google Scholar] [CrossRef] [PubMed] - Loewer, B. The mentaculus vision. In Statistical Mechanics and Scientific Explanation: Determinism, Indeterminism and Laws of Nature; World Scientific: Singapore, 2020; pp. 3–29. [Google Scholar]
- Mlodinow, L.; Brun, T.A. Relation between the psychological and thermodynamic arrows of time. Phys. Rev. E
**2014**, 89, 052102. [Google Scholar] [CrossRef] [PubMed] - Sagawa, T. Thermodynamic and logical reversibilities revisited. J. Stat. Mech. Theory Exp.
**2014**, 2014, P03025. [Google Scholar] [CrossRef] - Wolpert, D.H. The stochastic thermodynamics of computation. J. Phys. A Math. Theor.
**2019**, 52, 193001. [Google Scholar] [CrossRef] - Hemmo, M.; Shenker, O. The second law of thermodynamics and the psychological arrow of time. Br. J. Philos. Sci.
**2022**, 73, 85–107. [Google Scholar] [CrossRef] - Adam, S.P.; Alexandropoulos, S.A.N.; Pardalos, P.M.; Vrahatis, M.N. No free lunch theorem: A review. In Approximation and Optimization: Algorithms, Complexity and Applications; Springer: Cham, Switzerland, 2019; pp. 57–82. [Google Scholar]
- Wikipedia Contributors. No Free Lunch Theorem—Wikipedia, the Free Encyclopedia. Online. 2024. Available online: https://en.wikipedia.org/wiki/No_free_lunch_theorem (accessed on 18 November 2023).
- Wolpert, D.H. The implications of the no-free-lunch theorems for meta-induction. J. Gen. Philos. Sci.
**2023**, 54, 421–432. [Google Scholar] [CrossRef] - Lawler, G.F. Introduction to Stochastic Processes; CRC Press: Boca Raton, FL, USA, 2018. [Google Scholar]
- Serfozo, R. Basics of Applied Stochastic Processes; Springer Science & Business Media: Berlin/Heidelberg, Germany, 2009. [Google Scholar]

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Wolpert, D.H.; Kipper, J.
Memory Systems, the Epistemic Arrow of Time, and the Second Law. *Entropy* **2024**, *26*, 170.
https://doi.org/10.3390/e26020170

**AMA Style**

Wolpert DH, Kipper J.
Memory Systems, the Epistemic Arrow of Time, and the Second Law. *Entropy*. 2024; 26(2):170.
https://doi.org/10.3390/e26020170

**Chicago/Turabian Style**

Wolpert, David H., and Jens Kipper.
2024. "Memory Systems, the Epistemic Arrow of Time, and the Second Law" *Entropy* 26, no. 2: 170.
https://doi.org/10.3390/e26020170