Information Length as a New Diagnostic of Stochastic Resonance^†

Abstract

Stochastic resonance is a subtle, yet powerful phenomenon in which noise plays an interesting role of amplifying a signal instead of attenuating it. It has attracted great attention with a vast number of applications in physics, chemistry, biology, etc. Popular measures to study stochastic resonance include signal-to-noise ratios, residence time distributions, and different information theoretic measures. Here, we show that the information length provides a novel method to capture stochastic resonance. The information length measures the total number of statistically different states along the path of a system. Specifically, we consider the classical double-well model of stochastic resonance in which a particle in a potential $V(x,t)=[-x^2/2+x^4/4-Asin\left(\omega t\right)x]$ is subject to an additional stochastic forcing that causes it to occasionally jump between the two wells at $x\approx \pm 1$. We present direct numerical solutions of the Fokker–Planck equation for the probability density function $p(x,t)$ for $\omega ={10}^{-2}$ to ${10}^{-6}$, and $A\in [0,0.2]$ and show that the information length shows a very clear signal of the resonance. That is, stochastic resonance is reflected in the total number of different statistical states that a system passes through.

1. Introduction

Mutual interaction between two systems is most effective when the time scales of the two match, i.e., at the resonance. Time scale matching is manifested in a more subtle manner in stochastic resonance [1,2,3,4,5,6,7] as it occurs when the escape time scale [8,9] due to stochasticity matches the modulation time scale. One of the interesting consequences of stochastic resonance is that a noise plays an unusual role of amplifying a signal instead of attenuating it. Since the pioneering work by Benzi et al. [1] and Nicolis and Nicolis [2] (stochastic resonance could explain the relatively regular occurrence of ice ages by weak periodic modulations in the Earth’s orbital eccentricity), it has attracted great attention with numerous applications in physics, chemistry, biology, etc. [3,4,5,6,7]. Most popular diagnostic quantities that have been used to understand stochastic resonances include the signal-to-noise ratios, residence time distributions, escape time [8,9] and different information theoretic measures, e.g., [3,10,11,12,13,14,15,16,17].

The purpose of this paper is to introduce a new way of describing stochastic resonance from the perspective of information geometry. We do this by mapping the evolution of a stochastic system to a statistical space and interpret stochastic resonance in terms of the total number of statistically different states that a system passes through in time over a cycle of the modulation. The latter is quantified by information length $\mathcal{L}$ [18,19,20,21,22,23,24,25,26]. Specifically, for a time-dependent Probability Density Function (PDF) $p(x,t)$ for a stochastic variable, we define the (time-dependent) correlation time $\tau \left(t\right)$ of the PDF by

$$\mathcal{E}\left(t\right)\equiv \frac{1}{{\tau }^2}=\int \frac{1}{p(x,t)}{\left[\frac{\partial p(x,t)}{\partial t}\right]}^{2}dx.$$

(1)

Since $\tau \left(t\right)$ in general depends on time t, we measure the infinitesimal time $dt$ in unit of $\tau \left(t\right)$ and integrate over the time $[0,t]$ to calculate the total number of statistically different states as

$$\mathcal{L}\left(t\right)\equiv \int \frac{dt}{\tau \left(t\right)}=\int \sqrt{\int \frac{1}{p(x,t)}{\left[\frac{\partial p(x,t)}{\partial t}\right]}^{2}dx}dt.$$

(2)

Note that the differential element $d\mathcal{L}=dt/\tau \left(t\right)$ measures the rate at which new information is being generated during the evolution of p. We can relate $\mathcal{L}$ as the time integral of the infinitesimal relative entropy (see Appendix A). Unlike the relative entropy or Kullback–Leibler divergence [27], which only compares the initial and final PDFs, $\mathcal{L}$ is a path-dependent quantity between some initial and final times.

For stochastic resonance which is a periodically forced problem, we calculate $\mathcal{L}$ over one period $T=\frac{2\pi }{\omega }$ as $\mathsf{\Delta }\mathcal{L}=\mathcal{L}(t+T)-\mathcal{L}\left(t\right)$. We show below that stochastic resonance is accompanied by a rapid change in PDF, generating a new source of information. The latter is captured by $\mathsf{\Delta }\mathcal{L}$.

2. Double-Well Potential Model

We consider an over-damped system for a stochastic variable x under the influence of a periodic potential $V(x,t)$ and a short-correlated Gaussian noise $\xi$

$$\frac{dx}{dt}=-\frac{\partial }{\partial x}V+\xi =x-x^3+Asin\left(\omega t\right)+\xi ,$$

(3)

where

$$V(x,t)=-\frac{x^2}{2}+\frac{x^4}{4}-Asin\left(\omega t\right)x,$$

(4)

and

$$\langle \xi \left(t\right)\rangle =0,\langle \xi \left(t_1\right)\xi \left(t_2\right)\rangle =2D\delta (t_1-t_2).$$

(5)

Here, x and t in Equation (3) are given in dimensionless form (in space and time); the angular brackets in Equation (5) denote the average over $\xi$ and $\langle \xi \rangle =0$; D in Equation (5) is the strength of the stochastic noise $\xi$.

It is useful to note that for $A=0$, two stable equilibrium points are located at $x=\pm 1$, separated by an unstable point at $x=0$. The potential barrier between the two wells has height $V\left(0\right)-V(\pm 1)=1/4$. For $0<A\ll 1$, the stable equilibria move as $x\approx \pm 1+(A/2)sin\left(\omega t\right)$, while the depth of the wells fluctuates as $-1/4\mp Asin\left(\omega t\right)$ to leading order in A. That is, $A\ne 0$ causes the asymmetry in the two potential wells, each well being slightly deeper than the other one for half of the cycle. As D increases, the probability of switching between the two potential wells increases with more particles overcoming the potential barrier.

For a stationary potential $V\left(x\right)$ with $A=0$, the switching between the two wells is quantified by the following Kramers rate [28]

$$r_K=\frac{1}{\pi \sqrt{2}}exp\left[-\frac{1}{4D}\right].$$

(6)

In the limit of $D\to 0$, Equation (6) gives $r_K\to 0$, confirming no switching without $\xi$.

To investigate numerically the effect of A, D and $\omega$ on stochastic resonance, we consider the Fokker–Planck equation [29] for $p(x,t)$ corresponding to Equation (3)

$$\frac{\partial }{\partial t}p(x,t)=-\frac{\partial }{\partial x}\left(\left(x-x^3+Asin\left(\omega t\right)\right)p(x,t)\right)+D\frac{{\partial }^2}{\partial x^2}p(x,t).$$

(7)

In the limit of the adiabatic or quasi-static approximation, $p(x,t)$ is determined by the instantaneous potential as

$$p=p_{0}exp\left[-V(x,t)/D\right],$$

(8)

where $p_0$ is the normalization constant for ${\int }_{-\infty }^{+\infty }pdx=1$.

For the results presented in the following, we solve Equation (7) numerically by using second-order accurate finite differences in both space and time. We take a sufficiently large interval in x as $x\in [-3,3]$ so that we can impose $p=0$ boundary conditions at $x=\pm 3$. We check that the total probability ${\int }_{-3}^{3}pdx=1$ within ${10}^{-7}$. We also use sufficiently small grid size $\mathsf{\Delta }x={10}^{-3}$ and time steps $\mathsf{\Delta }t=\pi ·{10}^{-3}$ to ensure a good numerical accuracy.

3. Results

3.1. Probability Density Functions

We begin by presenting snapshots of the PDFs for small $A=0.04$ and $\omega ={10}^{-4}$ at the times $t=0$, $T/4$, $T/2$ and $3T/4$, $\mathrm{mod}\left(T\right)$ in Figure 1 around $x\approx 1$, where $T=\frac{2\pi }{\omega }$ is the period of the oscillatory force. Note that the behaviour around $x\approx -1$ is exactly the same as that around $x\approx 1$ by symmetry.

Notable in Figure 1 is the oscillation of the peak position in time as $(A/2)sin\left(\omega t\right)$ despite the fact that the particle switching between the wells is not necessarily following the adiabatic approximation (Equation (8)).That is, for small $\omega$, the behaviour within each well is almost adiabatic. In comparison, for large $\omega =O\left(1\right)$, the potential does vary so quickly that the PDFs are not adiabatic, the peak positions oscillating with an amplitude $A/\sqrt{4+{\omega }^2}$ (see, e.g., [7]). We also observe in Figure 1 that the widths of the PDFs increases with D. While for $D=0.01$ the PDFs are very close to Gaussian, for larger $D=0.1$, the PDF are not Gaussian in general.

Furthermore, while the amplitudes of p are of similar magnitude for small $D=0.01$, they become drastically different for larger $D=0.0324$ and $0.1$. In particular, for $D=0.0324$, the PDFs at $t=T/4$ and $3T/4$, and even $t=0$ and $T/2$, are significantly different, with $t=T/4$ yielding a far higher peak than $t=3T/4$. These results suggest that ${\int }_0^{\infty }p(x,t)dx$ can vary considerably throughout the cycle.

Figure 2 shows the maximum over the cycle of ${\int }_0^{\infty }p(x,t)dx$ for $\omega ={10}^{-2}$ to ${10}^{-6}$ and $D\in [0.01,0.1]$. We can see clearly that if D is too small, this maximum remains essentially 0.5 since there is no preference for one well over the other at any time in the cycle, and hence no synchronization. For larger D however, the values suddenly rise and then slowly decrease again. For smaller $\omega$ the rise is more abrupt, and occurs at smaller values of D. For the gradual decrease after the maximum value has been reached, all frequencies $\omega$ converge to the same curve.

To understand these results, we use $\omega =r_K$ and solve $\omega =exp(-1/4D)/\left(\pi \sqrt{2}\right)$ for $D=D_{\mathrm{res}}$ to obtain

$$D_{\mathrm{res}}=\frac{-1}{4ln\left(\pi \sqrt{2}\omega \right)},$$

(9)

where the subscript ‘res’ indicates the resonant value. In particular, we see in Figure 2 that the maxima of the individual $\omega$ curves all occur very close to their corresponding $D_{\mathrm{res}}$ values.

In comparison, the thick dashed lines in Figure 2 show the equivalent results for the adiabatic expression (8) obtained by numerically evaluating quantity

$${\int }_0^{\infty }exp\left[-V(x,T/4)/D\right]dx/{\int }_{-\infty }^{+\infty }exp\left[-V(x,T/4)/D\right]dx.$$

(10)

That is, at $t=T/4$, when the asymmetry between the two wells is greatest, we evaluate the probability of being in the lower well, according to the adiabatic PDF.

From Figure 2, we conclude that if $D\ll D_{\mathrm{res}}$, then $\omega \gg r_K$. That is, when the modulation is too rapid, the system cannot effectively respond, and the probability of being in either well remains practically 0.5 throughout the entire cycle with no synchronization between the stochastic switching and the modulation. In contrast, if $D\gg D_{\mathrm{res}}$, then $\omega \ll r_K$. The imposed modulation is then so slow that the adiabatic relation Equation (8) holds. As the results from (10) show, Equation (8) exhibits synchronization, and in particular stronger synchronization for smaller D, explaining why D should be as small as possible, but not much less than $D_{\mathrm{res}}$, which would switch the resonance off.

These results suggest that a resonant peak occurs only if a scan is done over the noise level D, as presented above. If a scan is done over $\omega$ for a fixed D, then the ‘resonance’ becomes a simple on-off phenomenon. This is because if $\omega \gg r_K$ there is no synchronization; if $\omega \ll r_K$ there is synchronization, at whatever level (10) yields for the given D (and A), but no variation with $\omega$, since (10) does not involve $\omega$.

3.2. Escape Times

A popular tool to study stochastic resonance is the escape time [8,9,29]. For this, we solve the same Fokker–Planck Equation (7) as before, but now only on the interval $x\in [0,3]$ with the boundary condition $p=0$ at $x=0$. Due to the probability loss, $p(x,t+T)=cp(x,t)$, where $c<1$ is some factor that depends on the parameters D, $\omega$ and A, but is the same for each subsequent period once this behaviour has emerged.

Figure 3 shows this reduction by a constant factor each period, starting from such initialized solutions (rescaled so that they start out with $\int pdx=1$ again). The top row shows the survival probability $G\left(t\right)={\int }_0^{\infty }p(x,t)dx$. The bottom row shows the escape rate $W\left(t\right)=-\frac{d}{dt}G\left(t\right)$. For all choices of D and A, we see that increasing either D or A yields a smaller c with more rapid loss of particles.

3.3. Information Length

Figure 4 shows $\mathcal{E}$ in Equation (1) for the previous solutions from Figure 1. Recalling that the position of the peaks varies as $(A/2)sin\left(\omega t\right)$, we see that $\mathcal{E}$ is consistently greatest when the peaks are moving fastest. The greatest values of $\mathcal{E}$ are for the intermediate case $D=0.0324$. It is important to note that Figure 4 includes the information about how the PDFs move back and forth within a given well.

Figure 5 shows the information length $\mathcal{L}$ in Equation (2) per cycle ($\mathsf{\Delta }\mathcal{L}=\mathcal{L}(t+T)-\mathcal{L}\left(t\right)$) associated with the results from Figure 2. We see that $\mathcal{L}$ exhibits a beautiful signal of the resonance phenomenon, just as clear as the probabilities themselves in Figure 2. To interpret these results, we start with the thick dashed lines, which are simply $4A/1.4D^{1/2}$. To understand the significance of this formula, we first recall that the peaks move according to $(A/2)sin\left(\omega t\right)$. The total distance each peak moves throughout a cycle is therefore $2A$. Next, the two PDF peaks at $x\approx \pm 1$ each have standard deviation $\sigma \approx \sqrt{D/2}$, with the approximation becoming better for smaller D, where the peaks are increasingly close to Gaussian (Figure 1). The width of each peak is therefore $2\sigma =1.4D^{1/2}$. (As we can also see in Figure 1, the width actually varies slightly throughout the period, but to obtain a lowest order estimate of what $\mathcal{L}$ should be, just the average width is sufficient.) So, if each peak moves a total distance $2A$, and has width $1.4D^{1/2}$, then the number of statistically distinguishable states it moves through is just $2A/1.4D^{1/2}$. The final factor of 2 is simply due to the fact that there are two peaks, each undergoing the same motion. We see therefore that in this small D regime before the resonance sets in, $\mathcal{L}$ is measuring precisely the motion of the peaks, and there is no other source of information length.

At the beginning of the resonance there is a sudden increase in $\mathcal{L}$ because of the additional information and associated number of statistically distinguishable states caused by the synchronization behaviour; at the synchronization, the probability of being in one well or the other at the appropriate times in the cycle becomes significantly different from 0.5.

Finally, for sufficiently large D, $\mathcal{L}$ is decreasing again, in fact, more rapidly than the previous $D^{-1/2}$ scaling. This is because in addition to the previous $D^{-1/2}$ factor, the contribution from the synchronization also decreases again, since according to Figure 2 the synchronization itself decreases once the solutions are back to satisfying the adiabatic condition (10).

4. Conclusions

We presented direct numerical solutions of the Fokker–Planck equation governing the classical double-well model of stochastic resonance. Stochastic resonance was shown to be accompanied by a rapid change in PDF, generating a new source of information. The latter was beautifully captured by the information length.