On the Change of Measure for Brownian Processes

Abstract

Sometimes, limits can be singular, implying that they take on different values depending on the order of arithmetic operations. In other words, the limit map lacks commutativity. While all such limits are mathematically valid, only one can be the physical limit. The change of measure for Brownian processes illustrates this phenomenon. A substantial body of elegant mathematics centered around continuous-time Brownian processes has been embraced by the physics community to investigate the nonequilibrium and equilibrium thermodynamics of systems composed of atoms and molecules. In this paper, we derive the continuous-time limit of discrete-time Brownian dynamics, specifically focusing on the change of measure. We demonstrate that this result yields the physical limit that differs from the commonly used expression. Consequently, the concepts of “the most probable path”, “minimum thermodynamic action”, and “the small-noise limit” are unphysical mathematical artifacts.

1. Introduction

Some observations of the movement of a particle show that it seems to move in a zig-zag pattern, as if driven by random, thermal forces. This “Brownian” motion is named after Robert Brown [1], who observed this behavior in 1827. The mathematical description of the thermal motion of a particle embedded in an external potential and buffeted by random forces is described by a stochastic differential equation (SDE), the overdamped Langevin equation, often referred to as Brownian dynamics. The mathematical basis was initiated by Wiener [2,3] in the 1920s. In the ensuing century, many textbooks have been written covering the mathematical developments; as an example, see Reference [4].

Among the many theorems that have been developed over the last century, the ones regarding the continuous-time process stand out as being particularly important. In particular, the Itô–Girsanov expression for the Radon–Nikodym derivative is the basis for the equivalency of measures associated with continuous-time Brownian processes with differing deterministic forces.

Here, we re-examine the change of measure, not disputing the major theorems mentioned above. Instead of considering continuous-time Brownian processes, we start with interpreting the discrete-time realization of the continuous-time process as the physical picture and then consider the continuous-time limit of such processes. One might ask, why bother? Often, it is illuminating to numerically construct particle motion as it moves from one free-energy basin to another to uncover transition pathways. In computer algorithms designed to sample such paths, time advances in discrete steps. As one approaches the continuous-time limit, one encounters divergent results for various properties. To alleviate this problem by isolating the singularity, a quantity called the “change of measure” was introduced. As we show below, the formula for the change of measure depends on the route one takes to evaluate the continuous-time limit.

Historically, the first route to consider was forming the change of measure for continuous-time processes. However, when following this route, unphysical features have emerged in computer calculations. For instance, Malsom and Pinski [5,6,7] used the Itô–Girsanov expression to examine paths for a particle moving in a one-dimensional potential with two degenerate wells, one of which was significantly broader than the other. In these calculations, they found that the paths indicated that the particle would spend most of its time in the narrow well rather than the broad well. This unphysical result was directly attributed to the “Laplacian” term in the Itô–Girsanov expression. In a more recent paper [7], the author analyzed doubly pinned paths for a particle traversing a two-dimensional potential landscape. This potential featured two distinct free-energy wells connected by a very narrow channel devoid of an energy barrier. Using the Itô–Girsanov (IG) expression, the sampled paths indicated that the particle gets stuck in the narrow channel. This unphysical result was traced to the substantial value of the Laplacian term along the channel. It is worth noting that both computational algorithms found sensible, physical paths when using the discrete-time form of the Onsager–Machlup (OM) functional and the same numerical parameters.

The above then motivates a different way to determine the continuous-time limit, which is to evaluate the change of measure for discrete-time processes before taking the continuous-time limit. This alternative approach is what we investigate here. Both limiting procedures are mathematically correct; however, as we show below, the latter one provides the physical limit. In this physical limit, many of the commonly used concepts, such as the MPP [8] (most probable path), do not exist.

This paper is organized as follows. In Section 2, we specify the notation used, and define the “common-noise” and “common-path” constructions. In this section, we also calculate the expression for the change of measure between two processes, and the concomitant the Kullback–Leibler (KL) divergence for each of the two constructions. In Section 3, we present our main result: the physical continuous-time limit of the change of measure between two Brownian processes is given by the common-noise construction. We continue with a discussion of the results in Section 4, where we explore some numerical issues and the concept of “thermodynamic action”. Finally, in Section 5, we end with a summary and additional ramifications of the current work.

2. Model

In this section, for both continuous and discrete time, we define the stochastic (Brownian) processes in terms of mathematical equations. Then, we proceed to examine the “common-noise” and “common-path” constructions and produce expressions for the change of measure and KL divergence between two processes. Although this paper examines a single spatial dimension, the conclusions derived herein are readily extendable to higher dimensions.

2.1. Preliminaries

Consider the concrete example of a particle moving in one spatial dimension under the influence of an external potential, $U_{\alpha }\left(x\right)$ at a temperature $ϵ$. The motion is described by Brownian dynamics; namely, the stochastic differential equation provides the position x as a function of time t:

$$d{x}_t=f_{\alpha }\left(x_t\right)dt+\sqrt{2ϵ}d{W}_t,$$

(1)

with $f_{\alpha }\left(x\right)=-\partial U_{\alpha }/\partial x$ and $d{W}_t$ being the standard Wiener process. For a nonzero time step, $\Delta t>0$, the Euler–Maruyama expression [9] is

$$x_i-x_{i-1}=f_{\alpha }\left(x_{i-1}\right)\Delta t+\sqrt{2ϵ\Delta t}{\xi }_i,$$

(2)

where the index i is related to the time $t=i\Delta t$, and ${\xi }_i$ are Gaussian distributed random variates with mean zero and unit variance. To be consistent with thermodynamics, the external potential $U_{\alpha }\left(x\right)$ must belong to the Kato class [10], which includes smooth functions bounded from below.

2.2. The Physical Model

Usually, Equation (2) is considered as an approximation to Equation (1). Here, we attach meaning to Equation (2) and use it to represent a physical model. The system of interest is connected to a thermal reservoir, kept at a temperature $ϵ$. This reservoir is the source of the noise. The noise is Gaussian-distributed with mean zero and unit variance. We define the noise history of any path with $N_t$ time steps as the sequence $\mathbb{N}=\left\{{\xi }_i\right\}$; the sequence $\mathbb{N}$ has $N_t$ elements.

Unless explicitly stated, we will be working with paths described in ${\mathbb{R}}^N$, where $N=N_t+1$ and $N_t$ is the number of time steps. In addition, the noise is due to an external thermal reservoir. Thus, the noise is independent of the system being studied; the noise and the particle positions are not correlated. The path positions for a single trajectory may be highly correlated with the noise. However, for a point $x_P$ and an ensemble of paths each containing $x_P$, the value of the noise acting at $x_P$ will be Gaussian-distributed.

For a path of (time) length T, with $N_t\Delta t=T$, the probability density (Lebesgue measure) of the noise history, $\left\{{\xi }_i\right\}$, is given by

$${\mathbb{P}}_{N_t}\left(\left\{{\xi }_i\right\}\right)\prod _{i=1}^{N_t}d{\xi }_i={\left(2\pi \right)}^{-N_t/2}\prod _{i=1}^{N_t}\left(\exp (-{\displaystyle \frac{1}{2}}{\xi }_i^2)d{\xi }_i\right).$$

(3)

The probability density can be expressed in terms of the path positions, ${\mathbb{P}}_{N_t}^{OM}\left(\left\{x_i\right\}\right)$, and given by

$${\mathbb{P}}_{N_t}^{OM}\left(\left\{x_i\right\}\right)\prod _{i=1}^{N_t}d{x}_i={(4ϵ\Delta t\pi )}^{-N_t/2}\exp \left(-{\mathbb{I}}_{N_t,\alpha }\left(\left\{x_i\right\}\right)\right)\prod _{i=1}^{N_t}d{x}_i$$

(4)

where ${\mathbb{I}}_{N_t,\alpha }\left(\left\{x_i\right\}\right)$ is the Onsager–Machlup (OM) functional [11] and is given by

$${\mathbb{I}}_{N_t,\alpha }\left(\left\{x_i\right\}\right)={\displaystyle \frac{\Delta t}{4ϵ}}\sum _{i=1}^{N_t}{\left({\displaystyle \frac{x_i-x_{i-1}}{\Delta t}}-f_{\alpha }\left(x_{i-1}\right)\right)}^2.$$

(5)

Here, we again emphasize that the path density is determined by the noise, Equation (3) or (4), which is emanating from a heat reservoir. The noise is Gaussian-distributed and is independent of the system being studied. Since

$${\xi }_i=\sqrt{{\displaystyle \frac{\Delta t}{2ϵ}}}\left({\displaystyle \frac{x_i-x_{i-1}}{\Delta t}}-f_{\alpha }\left(x_{i-1}\right)\right),$$

(6)

the OM functional is also Gaussian-distributed with mean and variance $N_t/2$. It is the multiplicity of paths that determines the optimal pathways for a particle to move from one free-energy basin to another.

To better grasp this concept, visualize a pair of standard, balanced six-sided dice, one black and the other red. Each individual throw yields an outcome with a one-in-thirty-six probability of occurring. The most probable sum of their faces is seven, as it emerges most frequently from the thirty-six possible outcomes. Similarly, the noise generated by the heat reservoir is represented by a sequence of independent and identically distributed (iid) Gaussian random variables. It is not the probabilities of individual paths that determine the most probable pathways between two configurations, but rather the multiplicity, the sheer number of paths.

2.3. Common-Noise History

We have now laid the groundwork to understand how the measure changes from one Brownian process to another. By examining Equation (3), it becomes evident that the measure is independent of the force (the drift). Consequently, for discrete time, two Brownian measures are identically Gaussian. The underlying physical concept behind this mathematical statement is that the noise is a characteristic of the external heat bath and is independent of the system being studied. This idea is further reinforced by considering the following thought experiment. Connect a constant-temperature heat reservoir to two (or more) separated systems, ensuring that all the systems receive the same noise input, as schematically shown in Figure 1. Despite being subjected to the same noise history, distinct Brownian processes generate different paths. However, the measure of each of the processes corresponds to the same Gaussian measure, and thus such paths are connected through a bijective map.

To make the above ideas concrete, consider the two discrete-time processes:

$${\mathbb{Q}}_1:d{x}_t=f_1\left(x_t\right)dt+\sqrt{2ϵ}d{W}_t\mathrm{and}$$

(7)

$${\mathbb{Q}}_2:d{z}_t=f_2\left(z_t\right)dt+\sqrt{2ϵ}d{W}_t.$$

(8)

We use the equations for the discrete-time process given by Equation (2). Both processes use identical values for the (time) length, T, the time increment $\Delta t$, and the temperature, $ϵ$. We assume that the starting points $x_0$ and $z_0$ are both zero. If one uses the same noise history $\mathbb{N}=\left\{{\xi }_i\right\}$, one would find that the ending positions differ; $x_T\ne z_T$, almost surely, as long as $f_1$ and $f_2$ differ.

Looking at Equation (3), the path probability only depends on the noise history, the sequence $\mathbb{N}$. Clearly, measures for the discrete-time processes, ${\mathbb{Q}}_1$ and ${\mathbb{Q}}_2$, are identical.

2.4. Common-Noise Construction Revisited

Typically, when one considers the processes ${\mathbb{Q}}_1$ and ${\mathbb{Q}}_2$, one generates paths and considers them instead of the noise histories. Consider a path $Z=\left\{z_i\right\}$ generated by process $Q_2$ by a noise history ${\mathbb{N}}_2=\left\{{\xi }_i^{\left(2\right)}\right\}$, and a path $X=\left\{x_i\right\}$ generated by process $Q_1$ by a noise ${\mathbb{N}}_1=\left\{{\xi }_i^{\left(1\right)}\right\}$. The set of all possible paths X is identical to the set of all possible paths Z. Consider a particular path Z generated by $Q_2$ by the noise history ${\mathbb{N}}_2$. Using this same noise history, the path $\tilde{X}=\left\{{\tilde{x}}_i\right\}$ is generated by $Q_1$. This then defines the bijective map, B, namely $\tilde{X}=B\left(Z\right)$. The change of measure, the reweighting, $\mathbb{L}\left(Z\right)$, of the probabilities, is given by

$$\mathbb{L}\left(Z\right)={\displaystyle \frac{{\mathbb{Q}}_1}{{\mathbb{Q}}_2}}=\exp \left({\mathbb{I}}_{N_t,2}\left(\left\{{\tilde{x}}_i\right\}\right)-{\mathbb{I}}_{N_t,1}\left(\left\{z_i\right\}\right)\right)=1.$$

(9)

Equation (9) is simply another a restatement of the physical idea expressed in Section 2.3, and leads to the vanishing of the Kullback–Leibler (KL) divergence. From the change of measure, one can obtain the Kullback–Leibler (KL) divergence, given by considering

$$\begin{array}{c}\hfill D_{KL}({\mathbb{Q}}_2\left|\right|{\mathbb{Q}}_1)=-{\mathbb{E}}_2\ln {\displaystyle \frac{{\mathbb{Q}}_2}{{\mathbb{Q}}_1}}=0.\end{array}$$

(10)

Clearly, the KL divergence is zero since each term in Equation (10) vanishes. We note the importance of the bijective map B. The same noise history ${\mathbb{N}}_2$ generates Z and $\tilde{X}$, using the processes $Q_2$ and $Q_1$, respectively.

Consistent with the results of the previous section, the KL divergence vanishes, indicating that the measures of two Brownian processes are equivalent. This manner of evaluating the KL divergence will be denoted as the “common-noise construction”.

2.5. Common-Path Construction

Instead of using a common noise history, one can construct the ratio ${\mathbb{Q}}_1/{\mathbb{Q}}_2$ using identical paths. The Equations (7) and (8) must be modified since the Wiener processes are different in the two SDEs. A path $Z=\left\{z_i\right\}$ can be generated by process $Q_2$ using the noise ${\mathbb{N}}_2$. The same path Z can be generated by the process $Q_1$ using a different noise ${\mathbb{N}}_1$. We can calculate the common-path construction for the Kullback–Leibler (KL) divergence:

$$\begin{array}{c}\hfill {\tilde{D}}_{KL}({\mathbb{Q}}_2\left|\right|{\mathbb{Q}}_1)={\mathbb{E}}_2\left({\mathbb{I}}_{N_t,2}\left(\left\{z_i\right\}\right)-{\mathbb{I}}_{N_t,1}\left(\left\{z_i\right\}\right)\right).\end{array}$$

(11)

In Equation (11), the far right-hand-side term depends on the difference ${\mathbb{I}}_{N_t,2}\left(\left\{z_i\right\}\right)-{\mathbb{I}}_{N_t,1}\left(\left\{z_i\right\}\right)$; some terms cancel in this difference. We then define $\mathbb{J}$ eliminating these canceling terms; namely,

$$\begin{array}{c}\hfill {\mathbb{J}}_{N_t,\alpha }\left(Z\right)={\displaystyle \frac{\Delta t}{2ϵ}}\sum _{i=1}^{N_t}\left({\displaystyle \frac{1}{2}}{f}_{\alpha }^2\left(z_{i-1}\right)-\left({\displaystyle \frac{z_i-z_{i-1}}{\Delta t}}\right)f_{\alpha }\left(z_{i-1}\right)\right).\end{array}$$

(12)

And the KL divergence becomes

$$\begin{array}{c}\hfill {\tilde{D}}_{KL}({\mathbb{Q}}_2\left|\right|{\mathbb{Q}}_1)=-{\mathbb{E}}_2\left({\mathbb{J}}_{N_t,1}\left(Z\right)-{\mathbb{J}}_{N_t,2}\left(Z\right)\right),\end{array}$$

(13)

where Z is a realization of the path generated by process ${\mathbb{Q}}_2$. This cancelation eliminates the possibly of a singularity in the KL divergence due to $\mathbb{I}$ (see Equation (4)) when evaluating the continuous-time limit.

We can now take the continuous-time (CT) limit $(\Delta t\to 0)$ of ${\mathbb{J}}_{N_t,\alpha }\left(Z\right)$ by applying the integration-by-parts formula from Itô calculus. See Appendix B for details. The resulting expression becomes

$$\begin{array}{c}\hfill {\mathbb{J}}_{\alpha ;T}^{CT}\left(Z\right)=\underset{\Delta t\to 0}{lim}{\mathbb{J}}_{N_t,\alpha }\left(\left\{z_i\right\}\right)={\displaystyle \frac{1}{2ϵ}}(U_{\alpha }\left(z_T\right)-U_{\alpha }\left(z_0\right))\\ \hfill +{\displaystyle \frac{1}{2ϵ}}{\int }_0^{T}dt\left({\displaystyle \frac{1}{2}}{f}_{\alpha }^2\left(z_t\right)-ϵU_{\alpha }^{″}\left(z_t\right)\right).\end{array}$$

(14)

The KL divergence within the common-path construction is given by

$${\tilde{D}}_{KL}^{CT}(Q_2\left|\right|Q_1)={\mathbb{E}}_2\left({\mathbb{J}}_{1;,T}^{CT}\left(Z\right)-{\mathbb{J}}_{2;T}^{CT}\left(Z\right)\right).$$

(15)

In contrast, the above result differs from the continuous limit of the KL divergence within the common-noise construction as within that construction $D_{KL}({\mathbb{Q}}_2\left|\right|{\mathbb{Q}}_1) =0$ for any non-negative $\Delta t$, no matter how small it is.

Note that Equations (13) and (14) are consistent with the Itô–Girsanov expression for the Radon–Nikodym (RN) derivative when considering continuous-time Brownian processes. We attach Itô’s name to this expression to emphasize the use of Itô’s lemma to treat the stochastic integral in the standard Girsanov expression. As indicated in Appendix A, the RN derivative is necessary to eliminate divergences when taking the continuous-time limit. The equations for the common-path construction are valid only when all possible endpoints are included. In Section 3.2, we examine the case where the endpoint is constrained. With such bridge paths, we find a different result. Below, in Section 3.1, for two examples, we determine the KL divergence for the common-path construction and obtain nonzero results.

3. Results

In this section, we present two illustrative examples that demonstrate that the common-noise construction produces both the physical continuous-time limit of the change of measure and the KL divergence between two Brownian processes. Furthermore, we investigate the prevalent practice of sampling transition paths by exclusively using bridge paths. When using such a restricted subset of paths, Jensen’s inequality is violated. Thus, such sampling methods lack a sound mathematical foundation.

3.1. The Physical Limit

Here, we show two examples where it is clear that the common-noise construction produces the physical continuous-time limit. As noted above, if one uses the common-noise procedure to generate the KL divergence, then one finds that it vanishes. First, we examine the change of measure as one switches from free Brownian motion to a process with constant drift. Consider discrete-time processes $Q_c$ and $Q_0$ corresponding to

$${\mathbb{Q}}_c:x_i-x_{i-1}-c\Delta t=\sqrt{2ϵ\Delta t}{\eta }_i\mathrm{and}$$

(16)

$${\mathbb{Q}}_0:z_i-z_{i-1}=\sqrt{2ϵ\Delta t}{\tilde{\eta }}_i.$$

(17)

We see that $x_t=z_t+ct$ defines the bijective map that links path pairs with a common-noise history (${\eta }_i={\tilde{\eta }}_i$), with $x_0=z_0=0$, and $x_T=z_T+cT$. The value of $D_{KL}(Q_0\left|\right|Q_c)$ vanishes. Now consider the Fokker–Planck equation for the process $Q_c$, which is given by

$${\displaystyle \frac{\partial }{\partial t}}\rho (x,t)=-c{\displaystyle \frac{\partial }{\partial x}}\rho (x,t)+ϵ{\displaystyle \frac{{\partial }^2}{\partial x^2}}\rho (x,t).$$

(18)

Making the substitution $x=z+ct$, Equation (18) reduces to the diffusion equation for the shifted probability function. Thus, the solution to the Fokker–Planck equation for the process $Q_c$ is consistent with this bijective map.

However, here we show that if one uses the common-path procedure, one finds the KL divergence to be positive and thus the measures are not identical. Note that for the common-path construction, in Equations (16) and (17), ${\eta }_i\ne {\tilde{\eta }}_i$. We use the processes $Q_c$ and $Q_0$ defined above and consider Equation (12), and note that $\mathbb{J}$ vanishes when the external force is zero. We generate paths Z that correspond to free Brownian motion (zero force). Then the path Z is inserted into ${\mathbb{J}}_{N_t,c}\left(Z\right)$ for the process $Q_c$. Thus, ${\tilde{D}}_{KL}^{CT}(Q_0\left|\right|Q_c)$ is given by

$${\tilde{D}}_{KL}^{CT}(Q_0\left|\right|Q_c)={\mathbb{E}}_0\left({\displaystyle \frac{1}{2ϵ}}{\int }_0^T{\displaystyle \frac{1}{2}}{c}^{2}dt\right)={\displaystyle \frac{1}{4ϵ}}{c}^{2}T$$

(19)

where T is the (time) length of the path, ${\mathbb{E}}_0(..)$ is the expected value of $(..)$ for the ensemble of free Brownian paths starting at the origin, and $z_0=0$, ${\mathbb{E}}_0{z}_t=0$, and ${\mathbb{E}}_0{z}_t^2=2ϵt$. Within the common-path construction, the expression for ${\tilde{D}}_{KL}^{CT}(Q_0\left|\right|Q_c)$ is positive when c is nonzero and T is positive.

The result in Equation (19) demonstrates that, in the continuous-time limit, the two constructions produce different results for the KL divergence. Since in the continuous-time limit, $\Delta t$ never becomes zero but only approaches it, neither construction produces the KL divergence for continuous-time Brownian processes. Clearly, the continuous-time limit depends on the manner in which the KL divergence is constructed.

Now, consider the Ornstein–Uhlenbeck (OU) process. The discrete-time OU process is given by

$${\mathbb{Q}}_{OU}:x_i-x_{i-1}+A{x}_{i-1}\Delta t=\sqrt{2ϵ\Delta t}{\eta }_i,$$

(20)

with $A>0$ being the OU constant. By combining Equations (17) and (20), we see that the common-noise construction implies that

$$x_i-x_{i-1}+A{x}_{i-1}\Delta t=z_i-z_{i-1}.$$

(21)

Using $y_t=\exp \left(At\right)x_t$, one is able to generate the analytic solution to the continuous-time OU process. With the continuous-time limit of Equation (21), we arrive at

$$d{x}_t+A{x}_{t}dt=\exp \left(-At\right)d{y}_t=d{z}_t=\sqrt{2ϵ}d{W}_t.$$

(22)

and thus

$$y_t=y_0+\sqrt{2ϵ}{\int }_0^t\exp \left(As\right)d{W}_s.$$

(23)

This shows that the analytical solution is consistent with the common-noise construction with a vanishing KL divergence. On the other hand, using Itô calculus, the common-path construction uses produces non-vanishing KL divergences, namely

$$\begin{array}{c}\hfill {\tilde{D}}_{KL}(Q_0\left|\right|Q_{OU})={\displaystyle \frac{1}{4}}{A}^2{T}^2\mathrm{and}\\ \hfill {\tilde{D}}_{KL}(Q_{OU}\left|\right|Q_0)={\displaystyle \frac{1}{8}}\left(\exp (-2AT)-1+2AT\right).\end{array}$$

(24)

3.2. Bridge Paths

In this section, we examine the common-path construction when the paths are constrained to start and end at the origin, $z_0=z_T=0$. In particular, we consider Brownian Bridge and the OU bridge processes. Here, we look at ${\tilde{D}}_{KL}^{CT}(Q_0^{BB}\left|\right|Q_{OU}^B)$, which becomes

$$\begin{array}{c}{\tilde{D}}_{KL}^{CT}(Q_0^{BB}\left|\right|Q_{OU}^B) ={\mathbb{E}}_0^{BB}{\mathbb{J}}_{N_t,OU}^B\left(Z\right)\hfill \\ ={\mathbb{E}}_0^{BB}\left({\displaystyle \frac{1}{2ϵ}}{\int }_0^{T}dt\left({\displaystyle \frac{A^2}{2}}{z}_t^2-ϵA\right)\right),\hfill \end{array}$$

(25)

where the symbols $Q_0^{BB}$ and $Q_{OU}^B$ denote the Brownian Bridge and the OU bridge processes, respectively, and ${\mathbb{E}}_0^{BB}(..)$ denotes the expected value of $(..)$ generated by the Brownian Bridge process. The time-dependent variance of Brownian Bridges is ${\mathbb{E}}_0^{BB}{x}_t^2=2ϵt(1-t/T).$ Thus, we find that

$$\begin{array}{c}\hfill {\tilde{D}}_{KL}^{CT}(0_{BB}\left|\right|A_{OUB})={\displaystyle \frac{AT}{12}}\left(AT-6\right).\end{array}$$

(26)

This result indicates that something is amiss, as the KL divergence is to be non-negative, which is not the case when $AT<6$. Equation (26) represents a violation of Jensen’s inequality. The source of this odd result lies in the truncation of the path space caused by the ending constraint. Paths that start at the origin will have (infinitely) many possible endings. The percentage of paths that end near the origin will depend on the underlying potential. This then shows that it is not correct to use the common-path construction to calculate the KL divergence for measures associated with doubly constrained paths, so-called bridge paths. Furthermore, similar considerations exist in interpreting results generated by reweighting schemes [12,13], as bridge paths are used.

4. Discussion

In this section, we delve into the results presented in Section 3. Firstly, we address numerical discrepancies that distinguish the results obtained using the common-path construction from those obtained using the common-noise construction. Subsequently, we explore the concept of “thermodynamic action” and show that it is a mathematical artifact.

4.1. Numerical Details

We now examine the implications for the KL divergence of using the common-noise construction, Equation (10), or using the common-path construction, given by Equation (15). In Equation (10), we use the following equality,

$$\begin{array}{c}\hfill \sum _{i=1}^{N_t}{\left((x_i-x_{i-1})-\Delta t{f}_1\left(x_{i-1}\right)\right)}^2\\ \hfill =\sum _{i=1}^{N_t}{\left((z_i-z_{i-1})-\Delta t{f}_2\left(z_{i-1}\right)\right)}^2.\end{array}$$

(27)

The above implies that

$$\begin{array}{c}\hfill \sum _{i=1}^{N_t}\left({(x_i-x_{i-1})}^2-{(z_i-z_{i-1})}^2\right)\\ \hfill =\mathcal{O}(\Delta t)+\mathrm{higher}\mathrm{order}\mathrm{terms}.\end{array}$$

(28)

Consider the continuous-time limit of the quadratic variation, namely,

$${\left[X\right]}_T=\underset{\Delta t\to 0}{lim}\sum _{i=1}^{T/\Delta t}{\left(x_i-x_{i-1}\right)}^2,$$

Then, ${\left[X\right]}_T={\left[Z\right]}_T=2ϵT$ almost surely. By reordering the sum, one may choose to use the continuous-time limits of the quadratic variation to eliminate all the terms on the left-hand side of Equation (28). However, the quantity described in Equation (10) does not vanish in the continuous-time limit as

$$\begin{array}{c}\hfill {\displaystyle \frac{1}{\Delta t}}\sum _{i=1}^{N_t}\left({(x_i-x_{i-1})}^2-{(z_i-z_{i-1})}^2\right)\\ \hfill =\mathcal{O}\left(1\right)+\mathrm{higher}\mathrm{order}\mathrm{terms}.\end{array}$$

(29)

Evidently, the common-path construction is consistent with the continuous-time limit of the quadratic variation being used in the calculation of the KL divergence. This lies in contrast to the common-noise construction where the continuous-time limit is taken only after the KL divergence is formed. Changing the order of arithmetic operations produces different results.

We have explored two constructions for the KL divergence for discrete-time Brownian processes. In the continuous-time limit, one construction produces a vanishing value for $D_{KL}$; the other produces ${\tilde{D}}_{KL}$, which looks similar to the Itô–Girsanov expression for the KL divergence and produces a non-vanishing value.

4.2. Thermodynamic Action

It is of great interest to understand how particles move from one free-energy basin to another and the size of the intervening energy barriers. Historically, path integrals gave an intuitive description of quantum mechanics [14]. There, one samples all doubly constrained paths to determine the transition amplitude. The similarity of the diffusion equation and Schrödinger equation inspired the use of the OM function as a “thermodynamic action” to investigate barrier hopping transitions driven by Brownian dynamics. The OM functional (Equation (4)) was modified in the 1970s to use the continuous-time limit using the Itô–Girsanov (IG) expression of the Radon–Nikodym derivative [8,15,16,17,18,19,20,21,22,23,24,25,26,27]. In particular, this modified version of the OM functional ${\mathbb{I}}^{IG}$ is given by

$$\begin{array}{c}\hfill {\mathbb{I}}^{IG}={\displaystyle \frac{\Delta t}{2ϵ}}\sum _{i=1}^{N_t}\left\{{\displaystyle \frac{1}{2}}{\left({\displaystyle \frac{x_i-x_{i-1}}{\Delta t}}\right)}^2+G\left(x_{i-1}\right)\right\}\\ \hfill \Rightarrow {\displaystyle \frac{1}{2ϵ}}{\int }_0^{T}dt\left\{{\displaystyle \frac{1}{2}}{\left({\displaystyle \frac{d{x}_t}{dt}}\right)}^2+G\left(x_t\right)\right\},\end{array}$$

(30)

where G is the path potential and is defined (in one dimension) as

$$G\left(x\right)={\displaystyle \frac{1}{2}}{f}^2\left(x\right)-ϵU^{″}\left(x\right).$$

(31)

This form of the OM functional has been in methods for sampling rare events by constructing doubly pinned diffusion paths [16,28,29,30,31,32,33]. Graham [16] and Eyink [34] used the “least-action” principle to discuss the minimization of the transformed OU functional as a “thermodynamic action”, as the value of $\int dtG\left(x_t\right)$ depends on the path.

As previously pointed out [6], Equations (3) and (4) show that the OM functional is independent of the path. The values of the OM functional are Gaussian-distributed with the mean and variance being $N_t/2$. Any path dependence would be a consequence of correlation between the noise and the particle positions. Such correlations are unphysical; the noise is a property of the thermal bath and is independent of the system being studied. Looking at the form of Equations (30) and (31), the minimum of ${\mathbb{I}}^{IG}$ can be dominated by the maximum of $G\left(x\right)$. Such maxima can lead to MPPs (most probable paths) that are unphysical [35] or lead to an unphysical ensemble of paths when using Hybrid Monte Carlo sampling methods [5,6,7]. By using Equations (30) and (31), one has introduced an uncontrollable approximation.

Now, consider the method known as MinActionPath, which was introduced in the paper [36]. This method is employed to calculate the details of molecular transformations. It involves approximating the potential energy in multiple free-energy basins using quadratic polynomials. The fundamental principle behind this approach is that since the Laplacian is a constant, it can be eliminated from Equation (31). However, it is important to recognize that for each well the constant will be different. Taking this difference into account makes the minimization of the OM functional over a specific time interval much more complex. Additionally, this method ignores the role of saddle points. This method introduce additional uncontrolled approximations. Contrary to the comment in the work of Carter et al. [37], calculations based on MinActionPath have little to say about the physical meaning of the minimizer of the Itô–Girsanov expression.

We turn to compare classical path integrals with their quantum counterparts, as used in the path-integral formulation of quantum mechanics [14]. When considering paths with fixed endpoints, the minimization of the quantum action determines the physical properties of such paths. This contrasts with the “thermodynamic action”, which is a property of the noise transmitted from the thermal reservoir and is independent of the path. Typically, the Wick rotation (from imaginary time to inverse temperature) is used to connect quantum mechanics to statistical mechanics. In this case, blindly applying the Wick rotation would obscure this distinction.

Even though the noise does not contain information about the system, paths generated in discrete time are not without thermodynamic information. For ergodic systems, the equivalence of the microcanonical and canonical ensembles allows one to extract such information. In particular, consider a particle moving in a system with two free-energy basins, A and B. For long paths, the times the particle spends in either basins, $t_A$ and $t_B$, are related to the free-energy difference $F_A-F_B$ by

$$F_B-F_A=-ϵ\ln \left({\displaystyle \frac{t_B}{t_A}}\right).$$

(32)

Using an ensemble of shorter (time) doubly constrained paths, the free-energy differences can be extracted in a similar manner [7]. The form of ${\mathbb{I}}^{IG}$, Equation (30), was derived using continuous-time Brownian processes. As shown above, for discrete-time processes, the use of the Itô–Girsanov expression for the Radon–Nikodym derivative leads to a change of measure with a nonzero KL divergence, and leads to an uncontrolled approximation to the underlying measure. Computer algorithms are based on discrete-time processes, where such an uncontrolled approximation can lead to unphysical results.

Now, consider the frequency dependence of the (time) Fourier transform of the Brownian paths. Above some finite frequency cutoff, the high-frequency components of the path are independent of the force (drift), as the potential (and the force) must be smooth. The path multiplicity depends exponentially on the number of frequency components. As the time increment, $\Delta t$, approaches zero, the logarithm of the multiplicity diverges. One can make an analogy with the standard physical interpretation of thermodynamics. To do so, we use the Boltzmann definition of the Entropy as the logarithm of the (path) multiplicity, and equate the energy with the cost function, the logarithm of the probability, here given in Equation (4). For smooth paths, only a relatively small number of frequency components are nonzero. As one decreases the time step, $\Delta t$, of the numerical representation of Brownian paths, the number of frequency components increases. However, when comparing the paths for any nonzero $\Delta t$ to those of a continuous-time process, one finds that an infinite number of frequency components are discarded. The multiplicity of paths grows exponentially as $\Delta t$ decreases. For coarse time grids, the multiplicity, and thus the Entropy, is small, and the distribution of paths is dominated by the most probable ones. The multiplicity increases exponentially as the time grid becomes finer; the cost function becomes less important.

Remember that the origin of the OM functional is based on Equation (3), which only depends on the number of Gaussian random variables; this functional is independent of the prefactor of the Weiner function in Equation (1). Thus, the “small-noise” limit of Huang, Huang, and Duan [38] does not have any physical meaning. Finally, in continuous-time processes, the multiplicity and the Entropy diverge and, concomitantly, the cost function becomes irrelevant. Thus, all paths generated by continuous-time Brownian processes are equally likely, independent of the drift, the deterministic component of the force.

We conclude that the use of Equation (30) for the OM functional is troublesome. If used as the basis of computer algorithms, unphysical results may be generated. The path integral over $G\left(x_t\right)$ is only correct if one uses paths generated by continuous-time processes. But for continuous-time processes, $d{x}_t/dt$ is not defined. If one uses discrete-time processes, $d{x}_t/dt$ can be approximated, but the use of $G\left(x_t\right)$ is an uncontrolled approximation, as shown in the above sections.

5. Conclusions

In this analysis, we examined the KL divergence, denoted as $D_{KL}$, between two discrete-time Brownian processes and then at the limit as $\Delta t$ approaches zero. We discovered that the KL divergence vanishes only when we consider common-noise histories. However, when we consider ${\tilde{D}}_{KL}$ derived from the common-path construction, we obtain a formula that reduces, in the continuous-time limit, to one that resembles the Itô–Girsanov expression for the Radon–Nikodym derivative, and does not vanish. On the other hand, all continuous-time Brownian processes have equivalent measures.

How do we reconcile these results? The limit map is not commutative. How the limit is taken matters. Analyzing the continuous-time processes produces a result different from the procedure of taking the limit of discrete-time processes as the value of the time step vanishes. It is evident that the results obtained using the common-noise construction align with physical considerations, as demonstrated by the two examples presented in Section 3.1

The two limiting processes differ. For discrete-time processes, as shown in Section 3.1, the physical limit must be the one that employs the common-noise construction to create path pairs, as the KL divergence vanishes for any value of $\Delta t>0$ no matter how small $\Delta t$ becomes. Remember that in such a limiting process, $\Delta t$ never becomes zero; continuous-time processes are never considered. This is an example of what Berry [39] called a “singular limit”.

This work has significant ramifications in numerous domains. We consider two. First, consider computer algorithms that aim to explore transition pathways by numerically generating an ensemble of paths that are constrained to start in one free-energy basin and end in another. If one uses the Itô–Girsanov expression for the Radon–Nikodym derivative, and uses a nonzero time step, one has introduced an uncontrolled approximation. The resulting computer simulations may become widely unphysical, as shown previously [5,6,7].

A second byproduct of the current work is as follows: the OM functional (Equation (4)) cannot be the thermodynamic action [17] that can be minimized, and the MPP, most probable path [8,40], is unphysical; it is simply a mathematical artifact. The OM functional is a property of the noise and has a mean and variance independent of the path. When the functional is expressed in terms of the path positions, and when the Itô–Girsanov expression is used, the functional only seems to have minima. However, as indicated above, the infinite multiplicity for continuous-time processes relegates all extrema to a set of measure zero. In numerical approaches, one may try to compensate for the structure of the “cost” function by decreasing the size of $\Delta t$. Even with the modern Hybrid Monte Carlo methods [7,41] that have a dimension-free acceptance rate, this presents a computational hurdle since the size of the phase space increases exponentially as the size of $\Delta t$ decreases.

All thermodynamically allowed paths are equally probable. It is the multiplicity of paths that determines what reaction pathways dominate. Simply put, more paths pass over small barriers than over large ones. More paths pass through broad free-energy valleys than through narrow ones. Additionally, processes described by discrete time steps are significantly different from continuous-time processes, as in the former, no matter how small $\Delta t$ is, an infinite number of frequency components have been discarded. Finally, consider doubly constrained paths: all paths starting at $x_A$ and, after a time T, ending at $x_B$. For quantum path integrals, the extremum of the quantum action governs the physical trajectory of the particle as it transverse from $x_A$ to $x_B$. In contrast, for classical path integrals, the “thermodynamic action” yields a value that is independent of the endpoints. This distinction would not be discernible by merely applying the Wick rotation.

We now state an observation based on the following: $\left(1\right)$ the continuous-time limit of the quadratic variation does not depend on the drift, $\left(2\right)$ for small values of $\Delta t$, the noise term dominates the drift term in Equation (2), $\left(3\right)$ for continuous-time Brownian processes, the drift can be transformed away, and $\left(4\right)$ the multiplicity of paths diverges in the limit $\Delta t\to 0$. Thus, the paths generated by continuous-time Brownian processes are independent of the drift, as all such paths correspond to the Gaussian process with zero drift. For continuous-time processes, the random buffeting due to the noise overwhelms the effects of the deterministic force. The change of measure for continuous-time processes is given in terms of the Radon–Nikodym derivative, which differs from the value for the continuous-time limit of discrete-time processes.

With apologies to P.W. Anderson [42], when it comes to the high-frequency modes of Brownian paths, the statement “Infinitely more is singularly different” holds true.