Energy Conversion and Entropy Production in Biased Random Walk Processes—From Discrete Modeling to the Continuous Limit

Abstract

We considered discrete and continuous representations of a thermodynamic process in which a random walker (e.g., a molecular motor on a molecular track) uses periodically pumped energy (work) to pass N sites and move energetically downhill while dissipating heat. Interestingly, we found that, starting from a discrete model, the limit in which the motion becomes continuous in space and time ($N\to \infty$) is not unique and depends on what physical observables are assumed to be unchanged in the process. In particular, one may (as usually done) choose to keep the speed and diffusion coefficient fixed during this limiting process, in which case, the entropy production is affected. In addition, we also studied processes in which the entropy production is kept constant as $N\to \infty$ at the cost of a modified speed or diffusion coefficient. Furthermore, we also combined this dynamics with work against an opposing force, which made it possible to study the effect of discretization of the process on the thermodynamic efficiency of transferring the power input to the power output. Interestingly, we found that the efficiency was increased in the limit of $N\to \infty$. Finally, we investigated the same process when transitions between sites can only happen at finite time intervals and studied the impact of this time discretization on the thermodynamic variables as the continuous limit is approached.

1. Introduction

Nonequilibrium thermodynamics deals with general laws of a (driven) system transferring energy from one or more heat bath(s) to useful work. The second law, however, restricts this transformation as only part of the input energy may be “accessible”, as the entropy production, related to heat production, must not decrease [1,2,3]. The “system” is usually described by many degrees of freedom, e.g., ${10}^{23}$ gas particles with individual fluctuating trajectories, where the exchanged heat and the extracted work are determined from statistical averages over all these particles trajectories [4]. For “small” systems, such as (bio)polymers, colloid particle, enzymes, or molecular motors, the dynamics is described by only few degrees of freedom and the fluctuation of individual “state” trajectories becomes more prominent [3,5,6]. Notably, it has been shown, e.g., see Refs. [7,8,9], that the resulting (relative) fluctuations of a statistical averaged observable, like the number of steps R of a molecular motor mimicked by a random walk along a track of molecules, can be related to the entropy production. A relative uncertainty or fluctuation $\Delta R^2=2D/v^2$ of the motor steps along the molecular track with the diffusion coefficient D and velocity v requires at least an entropy production rate $\dot{\sigma }$ of $2k_B/\Delta R^2$. This leads to the inequality, known as the thermodynamic uncertainty relation (TUR) [7,8]:

$$\begin{array}{c}\hfill \dot{\sigma }\frac{2D}{v^2}\ge 2k_B.\end{array}$$

(1)

Different approaches are used for describing small system dynamics including fluctuations and the resulting (stochastic) thermodynamic properties. Two of these approaches have been prominently elaborated in recent years [3]:

(i) Dynamics on a discrete set of states: A system is described by its microstates whose dynamics is captured by a master equation. The rates to interchange between microstates is governed by the local detailed balance relation determined by a thermal bath [3];

(ii) Dynamics on continuous trajectories: Consideration of individual continuous trajectories of a (driven) colloidal particle whose velocity is described by a Langevin equation where the probability distributions of a particle at position x with velocity v and diffusion D are determined by the Fokker–Planck equation.

Starting from discrete set of states, we can approximate the dynamics by a continuous description, such as the Fokker–Planck equation by standard procedures like the Kramers–Moyal expansion, when (infinite) many states may be visited during the time of observation, e.g., a chemical reaction network of multiple reaction steps [10,11,12,13,14]. Similarly, starting from a continuous description for the dynamics, a discrete representation is obtained by standard mathematical steps of replacing derivatives by finite difference quotients. While such mathematical transformations are expected to lead to equivalent descriptions of the underlying physics, we show below that not all physical observables can be kept invariant under such transformations. In particular, we show that the discrete to continuous limiting process is not unique and depends on which observables are chosen to be invariant under this process. Recent studies have already shown that the entropy production might differ under different coarse-graining schemes since, under coarse-graining, some “information” is lost while also the mathematical derivation of the differential entropy starting from the discrete Shannon entropy has revealed some discrepancy (Remark: Note that the Shannon entropy $S=-k_B{\sum }_i{p}_{i}ln{p}_i$ is defined for a discrete propability distribution of i states (each state with the respective probability $p_i$). Going to the continuous state-space description for the probability distribution $p_i\to p\left(x_i\right)\Delta x$, the continuous entropy $s^{\Delta x}=-k_B\int dxp\left(x\right)lnp\left(x\right)-li{m}_{\Delta x\to 0}{k}_B{\sum }_{i}p\left(x_i\right)\Delta xln(\Delta x)$ differs by a potentially infinite offset (since $ln(\Delta x)\to -\infty$ for $\Delta x\to 0$), which needs to be substracted, see discussion in [15]. However for the discussion on entropy production (change in entropy and its change per time (entropy production rate)), we note that the infinte offset vanishes) [15,16,17]. Here, we suggest a different point of view by showing that it is possible to impose an invariant entropy production (or a given heat exchanged with the thermal environment) when proceeding from the discrete to the continuous description of the system dynamics. Under such restriction, some other system observable cannot be kept invariant. Alternatively, one may even ask whether this observation may be translated into realistic system processes. Put differently, can we, by adding intermediate states between an initial and a target state, optimize the process of transferring input energy into useful work.

The aim of the present manuscript is to investigate the transition from a discrete to a continuous state-space system for an exactly solvable master equation by keeping distinct system observables constant while studying the impact on other observables. Explicitly, we consider a simple model: a Brownian particle moving on a downhill slope with an energy-pumping step taking place at constant length intervals L that restores the energy of the particle into its initial value, which represents a cycle. In the discrete representation, the particle is a random walker moving among N equally spaced sites per fixed length L. $N\to \infty$ represents the continuous limit for the cyclic dynamics. More details about the model are provided in Section 2. Other processes with period boundary conditions can be mapped into this form. One realization of such a process for walking in discrete steps is that of molecular motors, which transform free energy liberated in a chemical reaction by a succession of steps on a track into mechanical work (motion) [5,8]. Driven rotational Brownian particles through periodic potential wells to experimentally test thermodynamics laws represent another example, though there are many others [18]. Given the specific chosen conditions, like constant speed v and the entropy production rate of the cycle, we can study the diffusion coefficient D in dependence of the number of sites N per cycle and the continuous limit when $N\to \infty$. Interestingly, under an additional action of an opposing force during the process under study, we can also determine the impact of the number of sites per cycle on thermodynamic performance by transferring input energy into useful work. Furthermore, when keeping the energy drop per cycle and the speed constant, we interestingly found that, in the limiting case $N\to \infty$, the diffusion coefficient (equivalent to the variance in the site distribution) approaches the thermal Einstein relation when $N\to \infty$, which is usually only expected for small velocities in the linear response limit of small driving.

The paper is structured as follows. In Section 2, we describe the cyclic process as biased random between (energy) sites governed by a master equation restricted by periodic boundary conditions. We further discuss the implication for the entropy and heat production. In Section 3, we study the entropy production rate and heat transfer rate into the environment of the cyclic process given the velocity v and diffusion constant D for different number of sites N in the dynamical (relaxation from initial site) and in a steady state. We then determine the performance in transferring input power to useful output power for the cycle process under an opposing force by varying N and, so, by going from the discrete to the continuous ($N\to \infty$) state space. In Section 4, we study the transition to $N\to \infty$ given constant entropy production for the process, and, under either constant velocity v or diffusion coefficient D. The impact on the respective system variable when increasing N is discussed. Section 5 is devoted to examining the randomness parameter of the cycle, which is dependent on the number of sites N, while maintaining various physical observables as constants. Through this randomness factor, we can also determine the variance in the cycle completion time and study its dependence on N. In Section 6, we investigate the process by the same biased random walk, but where the system evolves at finite time intervals, and study the impact of this time discretization on the entropy production and diffusion coefficient. Section 7 concludes this work.

2. Cyclic Process

We consider a cyclic process, where the $n=1,\dots ,N$ sites are aligned on a circle such that each site has two neighboring sites with the periodic boundary condition $N+n=n$ with $N\ge 3$. The process dynamics is captured by a biased one-step random walk with forward and backward transfer rates $\alpha$ and $\beta$, which are equal at each site, such that the classical (Markovian) master equation for the probability distribution of the n-site of the cycle reads [10,13,14]

$$\begin{array}{c}\hfill \dot{P}(n,t)=\alpha P(n-1,t)+\beta P(n+1,t)-(\alpha +\beta )P(n,t).\end{array}$$

(2)

Starting from a well-defined site at $t=0$, one finds in the limit of long times $t\to \infty$ the (nonequilibrium) steady-state distribution for the n site to be $P(n,t\to \infty )=1/N$ (see Appendix C). When writing the master Equation (2) in the form $\dot{\mathbf{P}}=M\mathbf{P}$, we see that the matrix M is irreducible with the dominant eigenvalue $\lambda =0$ while all other eigenvalues have a strictly negative real part according to the Perron–Frobenius theorem, signaling the existence of a stable steady-state vector [19,20]. While our cyclic process can take place in the state space of any given system, it is convenient to consider the equivalent random walk with forward and backward rates $\alpha$ and $\beta$ on a cycle of constant circumference $L=2\pi R$. In this equivalent random walk problem, the velocity v and diffusion constant D at steady state are defined by [14]

$$\begin{array}{c}\hfill v\equiv \frac{\langle n\rangle \Delta x}{t}=(\alpha -\beta )\Delta x\end{array}$$

(3)

and

$$\begin{array}{c}\hfill 2D\equiv \frac{(\langle n^2\rangle -{\langle n\rangle }^2)\Delta x^2}{t}=(\alpha +\beta )\Delta x^2,\end{array}$$

(4)

where $\Delta x=2\pi R/N$ is the equidistant step size, with R and N being the radius of the cycle and the number of total sites N of the cycle, respectively. Evidently, the time for completing a full cycle is $\tau =N{(\alpha -\beta )}^{-1}$.

The transition between the neighboring sites, $n\to n\pm 1$, are considered as autonomous Markov jump processes where each site has its energy $E\left(n\right)$. Thermodynamic consistency is introduced by the local detailed balance condition

$$\begin{array}{c}\hfill \frac{\alpha }{\beta }=e^{\frac{1}{k_{B}T}\left(E\left(n\right)-E(n+1)\right)},\end{array}$$

(5)

where, for simplicity, we assume isothermal conditions with T on all sites, and where $\Delta E=E\left(n\right)-E(n+1)>0$ is taken to be the same for all nearest neighbor sites. Note that we count the heat exchange with the bath as $Q\equiv -\Delta E$, i.e., if $Q<0$, the amount of heat is taken from the system to the bath; while the environment provides heat to the system if $Q>0$. This implies that during a cycle, the amount of heat $\Delta Q\equiv -N\Delta E=k_{B}TNln\left(\frac{\alpha }{\beta }\right)$ is dissipated, so $E(N+1)\equiv E\left(1\right)-N\Delta E$ may be compared to a downhill process of energy loss $N\Delta E$. To remain consistent with the periodic boundary conditions, we further assume that, between sites N and $N+1=1$, an upward energy jump occurs, in which the same amount of work, $W=N\Delta E$, is returned to the system.

Next, we calculate the time-dependent change of the Boltzmann–Gibbs entropy, $\dot{S}\left(t\right)=-k_B{\sum }_n\dot{P}(n,t)lnP(n,t)$, [2,21] for the system using the master Equation (2). We obtain

$$\begin{array}{cc}\hfill \dot{S}\left(t\right)& =-k_B\sum _{m=1}^N\sum _{n=1}^N[\alpha P(n-1,t)+\beta P(n+1,t)\hfill \\ \hfill & -(\alpha +\beta )P(n,t)]ln\frac{P(n,t)}{P(m,t)}\hfill \end{array}$$

(6)

$$\begin{array}{c}\hfill =k_B\sum _{m=1}^N\sum _{n=m-1}^{N-1}\left[\alpha P(n,t)-\beta P(m,t)\right]ln\frac{P(n,t)}{P(m,t)}\end{array}$$

(7)

$$\begin{array}{cc}\hfill & =k_B\sum _{m,n=m-1}\left[\alpha P(n,t)-\beta P(m,t)\right]ln\frac{P(n,t)\alpha }{P(m,t)\beta }\hfill \\ \hfill & -k_B\sum _{m,n=m-1}\left[\alpha P(n,t)-\beta P(m,t)\right]ln\frac{\alpha }{\beta }.\hfill \end{array}$$

(8)

Note that in Equation (6), we could replace $lnP(n,t)$ by $ln\left[P\right(n,t)/P(m,t\left)\right]$ because the sum that multiplies $lnP(m,t)$ vanishes. The result (8) can be recast in the form $\dot{S}\left(t\right)={\dot{S}}_e+\dot{\sigma }\left(t\right)$, [22,23] where, using ${\sum }_{n}P(n,t)=1$,

$$\begin{array}{cc}\hfill {\dot{S}}_e& =-k_B\sum _{m,n=m-1}\left[\alpha P(n,t)-\beta P(m,t)\right]ln\frac{\alpha }{\beta }\hfill \\ \hfill & =-k_B\left[\alpha -\beta \right]ln\frac{\alpha }{\beta },\hfill \end{array}$$

(9)

and

$$\begin{array}{cc}\hfill \dot{\sigma }\left(t\right)& =k_B[\left[\alpha -\beta \right]ln\frac{\alpha }{\beta }\hfill \\ \hfill & +\sum _{m,n=m-1}\left[\alpha P(n,t)-\beta P(m,t)\right]ln\frac{P(n,t)}{P(m,t)}].\hfill \end{array}$$

(10)

Next, we show that the first term ${\dot{S}}_e$ (Equation (9)) is the entropy flow into the environment, while the second term is the entropy production rate $\dot{\sigma }\left(t\right)$ (Equation (10)).

Consider first Equation (9). Because the rates $\alpha$ and $\beta$ were assumed not to depend on the site identity, ${\dot{S}}_e$ is time-independent. ${\dot{S}}_e$ can be written as

$$\begin{array}{c}\hfill {\dot{S}}_e=J\frac{Q}{T},\end{array}$$

(11)

where $J\equiv \alpha -\beta$ is the cumulative flux: the sum over nearest neighbor site-pair fluxes (see first expression in Equation (9)). Q is the heat exchanged during a single nearest neighbor transfer event with the environment according to the local detailed balance relation in Equation (5). The product $JQ$ in Equation (11) is the heat flux into the thermal environment of temperature T per cycle. For the given model, this heat exchange is time-independent.

According to the Clausius principle, the change in system entropy is bounded by the (negative) heat amount exchanged with the environment $\dot{S}\ge \frac{JQ}{T}$ where equality is reached for reversible processes [1]. Motivated by this inequality, one defines the total entropy production rate by $\dot{\sigma }\left(t\right)=\dot{S}-\frac{JQ}{T}\ge 0$. Indeed, the second term $\dot{\sigma }\left(t\right)$ (Equation (10)) meets the two important properties: (i) It is non-negative because the first term in Equation (8) can be recast into $(x-y)ln(x/y)\ge 0$; and (ii) it vanishes for thermal equilibrium, when microscopic reversibility or the detailed balance condition, $\alpha P(m,t)=\beta P(n,t)$, is obeyed and no entropy is produced. In a (nonequilibrium) steady state, Equation (6) is zero and the entropy production equals the negative entropy flow into the environment $\dot{\sigma }=-{\dot{S}}_e$, Refs. [23,24].

In the following, we investigate the entropy production rate $\dot{\sigma }\left(t\right)$ and the physical measurable entropy or heat flow ${\dot{S}}_e$ into the environment for the prototype N-site cyclic process given different conditions. In particular, we are interested in the limit $N\to \infty$ to investigate the transition from the discrete to the continuous state space.

3. N-Site Cyclic Process under Constant Velocity v and Diffusion Constant D

We first consider the dependence of the number of sites N per cycle on the system that is performed under the conditions that (a) the speed v (Equation (3)) and (b) the diffusion constant D (Equation (4)) are kept constant. The first condition requires that the time, $\tau =N{(\alpha -\beta )}^{-1}$, for a full completion of the cycle remains constant. Together with the second condition, the forward and backward rate must depend on N. We obtain

$$\begin{array}{cc}\hfill \alpha & =\frac{D{N}^2}{4{\pi }^2{R}^2}+\frac{vN}{4\pi R},\hfill \end{array}$$

(12)

$$\begin{array}{cc}\hfill \beta & =\frac{D{N}^2}{4{\pi }^2{R}^2}-\frac{vN}{4\pi R}.\hfill \end{array}$$

(13)

To ensure the positivity of the rate (13), $\beta \ge 0$, it immediately follows that $v\le ND/R\pi$. Note that under the conditions restricted by Equations (12) and (13), the detailed balance relation $\Delta E=k_{B}Tln\left(\frac{\alpha }{\beta }\right)$ (Equation (5)) will be a nonlinear function of N.

Consider the entropy production rate $\dot{\sigma }\left(t\right)$ (Equation (10)) given a well-defined initial cycle site $n=1$ at $t=0$, so $P(n=1,t=0)=1$. The entropy production rate (the entropy produced per unit time) decreases over time before reaching a (N-dependent) steady-state value, see Figure 1.

This observation can be understood as follows. Initially, the system starts at a given site and evolves in its cyclic dynamics according to the master Equation (2). At later time t, the system will be found at a given site n with probability $P(n,t)$ (see Appendix A, Appendix B and Appendix C). The loss of the initial “knowledge” about the exact system site n increases the entropy of the system. As the probability distribution relaxes to its steady-state distribution $P_{n,SS}\equiv P(n,t\to \infty )=1/N$ (see Appendix C), the entropy production rate, $\dot{\sigma }$, decreases to its steady-state value

$$\begin{array}{cc}\hfill {\dot{\sigma }}_{SS}& =k_B(\alpha -\beta )ln\left[\frac{\alpha }{\beta }\right]=\frac{k_{B}N}{2\pi R}vln\left[\frac{\frac{2DN}{2\pi R}+v}{\frac{2DN}{2\pi R}-v}\right].\hfill \end{array}$$

(14)

Note that entropy is produced at constant rate when running the cyclic process under steady-state conditions. Interestingly, as seen in Figure 1, the more sites N are included in the cycle of finite length the faster the entropy production rate decreases to its steady-state value. At steady state, the entropy production equals the negative (in this model time-independent) entropy flow into the environment ${\dot{\sigma }}_{SS}=-{\dot{S}}_e$.

Consider the N-dependence of the the steady-state entropy production rate ${\dot{\sigma }}_{SS}$ for an N-site cycle, Equation (14). The dependence is shown in Figure 2. For $N\to \infty$, the steady-state entropy production rate (14) is reduced to $k_B{v}^2/D$. This minimum entropy production rate in this limit can be understood as the forward and backward rates (Equations (12) and (13)) become more alike. Therefore, consecutive transitions between sites n and $n+1$ become more time-symmetric and less entropy per step, $\Delta E/T=ln\left[\alpha \right(N)/\beta (N\left)\right]/T$, is produced.

Interestingly, for $N\to \infty$, the TUR relation, the minimal required entropy production rate $\dot{\sigma }$ for a given relative fluctuation $2D/v^2$ (as introduced in Equation (1) for a molecular motor moving along a molecular track) states equality, $\dot{\sigma }2D/v^2=2k_B$. In this limit, the dynamics of the cyclic process is comparable to a continuous Brownian diffusive motion of a particle of constant speed v and diffusion D described by a Fokker–Planck equation [25].

Assume that a forward (downhill) step on the cycle takes place against a constant applied force f. Then, the local detail balance relation (5) must be redefined by $\Delta \tilde{E}\equiv E\left(n\right)-E(n+1)-f\Delta x=\Delta E-f\Delta x>0$, as part of the energy per step $\Delta E$ is transferred to work $f\Delta x$, where $\Delta x$ is the step size. We defined the system’s heat exchange per step with the bath as $Q\equiv -\Delta \tilde{E}$, while $Q<0$ is the amount of heat taken from the system to the bath. At steady state, the heat flow from the system into the bath is $-\dot{Q}=T{\dot{\sigma }}_{SS}$, see discussion in Section 2. The power output per cycle when running against the force at steady state is

$$\begin{array}{cc}\hfill P\equiv \sum _{m,n=m-1}& \left[\alpha P\left(n\right)-\beta P\left(m\right)\right]f\Delta x=(\alpha -\beta )\Delta xf\hfill \\ \hfill & =vf.\hfill \end{array}$$

(15)

In Equation (15), $\alpha P\left(n\right)-\beta P(n+1)$ is the probability flux between neighboring sites and where ${\sum }_{n}P\left(n\right)=1$. By the first law of thermodynamics, the total supplied power must be $P-\dot{Q}$ such that we can define the thermodynamic efficiency as [8]

$$\begin{array}{cc}\hfill \eta \left(N\right)& \equiv \frac{P}{P-\dot{Q}}=\frac{fv}{fv+T{\dot{\sigma }}_{SS}\left(N\right)},\hfill \end{array}$$

(16)

where ${\dot{\sigma }}_{SS}\left(N\right)$, Equation (14), depends on the total number of sites N per cycle. The cycle can be compared to a process of going down a slope against a constant force and with friction. The friction force is usually taken as $F_{fr}=\gamma v$ such that the related heat dissipated in the cycle per unit time is $F_{fr}v=\gamma v^2$. By identifying $\gamma v^2\equiv T{\dot{\sigma }}_{SS}(v,N)$, we can calculate the friction coefficient $\gamma \left(v\right)=T{\dot{\sigma }}_{SS}\left(v\right)/v^2$ for the present cycle process using Equation (14). Note that, as expected, the friction coefficient $\gamma$ goes to its linear response value for a small velocity, $\gamma (v\to 0)\to k_{B}T/D$, independent of N. Surprisingly, as depicted in the inset of Figure 3, $\gamma$ also takes the same value for finite v in the limit of $N\to \infty$. In this limit, the system assumes some features similar to equilibrium even though the flux is finite. Not only in the limit $v\to 0$, but also in the limiting process can $N\to \infty$ be compared to thermodynamic cycles where the system changes adiabatically slowly to always be in thermal equilibrium throughout the process.

Equivalently, in the limit $N\to \infty$, we can write the thermodynamic efficiency (16) as

$$\begin{array}{c}\hfill \eta =\frac{1}{1+k_{B}Tv/Df}.\end{array}$$

(17)

Note that in the limit of linear response the velocity v and force f are linearly related by the mobility $\mu f=v$. Assuming the Einstein relation $D=\mu k_{b}T$ holds under linear response such that the efficiency in the limit $N\to \infty$ reach $\eta =1/2$.

Figure 3 portrays the efficiencies for different N. As expected, the efficiency increases with N as less heat $-\dot{Q}=T{\dot{\sigma }}_{SS}$ will be produced per cycle given constant power output P. Interestingly, the slower the chosen velocity v, the more one can reach maximal efficiency. The efficiency is bound from above, $\eta \le 1$, while equality is reached for $v=0$ or $f\to \infty$, $D\to \infty$, see Equation (17). The last conditions, however, do not produce useful output power, as the cycle will stop.

4. Cyclic Process with Constant Velocity v or Diffusion Constant D and Constant Energy Drop Per Cycle

As stated in the introduction, the process of going to the continuous description for the state-space dynamics ($N\to \infty$) is not unique. We, therefore, use the same methodology as above to describe the unicyclic process as a 1D random walk at steady state but where we now require the total entropy produced per cycle to be constant (given by the constant energy drop per cycle) and either (A) the velocity v or (B) the diffusion coefficient D to be constant. We assume that all energy invested into the system is dissipated as heat to the environment, so $W=\sigma T$. The steady-state entropy production $\sigma$ is

$$\begin{array}{c}\hfill \sigma =k_{B}Nln\left(\frac{\alpha }{\beta }\right),\end{array}$$

(18)

where N is the total number of sites. At steady state, the entropy production equals the heat going into the environment, see Section 2.

(A) Given the constant velocity v, Equation (3), the forward and backward rates are

$$\begin{array}{cc}\hfill \alpha & =\frac{vN}{2\pi R\left(1-e^{\frac{-\sigma }{N{k}_B}}\right)},\hfill \end{array}$$

(19)

$$\begin{array}{cc}\hfill \beta & =\frac{vN}{2\pi R\left(e^{\frac{\sigma }{N{k}_B}}-1\right)},\hfill \end{array}$$

(20)

where R is the radius of the cycle (see Section 2).

(B) Given the constant diffusion coefficient D, Equation (4), the forward and backward rates are

$$\begin{array}{cc}\hfill \alpha & =\frac{2D{N}^2}{{\left(2\pi R\right)}^2\left(1+e^{\frac{-\sigma }{N{k}_B}}\right)},\hfill \end{array}$$

(21)

$$\begin{array}{cc}\hfill \beta & =\frac{2D{N}^2}{{\left(2\pi R\right)}^2\left(e^{\frac{\sigma }{N{k}_B}}+1\right)}.\hfill \end{array}$$

(22)

Consider first case (A). Equation (4) with Equations (19) and (20) leads to

$$\begin{array}{cc}\hfill D& =\frac{1}{2}(\alpha +\beta ){\left(\frac{2\pi R}{N}\right)}^2=\frac{v\pi R}{N}coth\left[\frac{\sigma }{2N{k}_B}\right],\hfill \end{array}$$

(23)

which is shown in Figure 4. The diffusion coefficient decreases with increasing N. As the diffusion coefficient is the variance of the site distribution on our equivalent cycle, see Equation (4), the related fluctuations in localization of a site is reduced during a cycle with an increase in N. Assuming that the energy falls uniformly along the cycle, so that $\Delta E=T\sigma /N$, we find that, in the regime of linear response $v\to 0$ and in the limit $N\to \infty$, the diffusion coefficient is captured by the (Einstein) relation $D=v/F{k}_{B}T=\mu k_{B}T$ with the mobility $\mu =v/F$ and the related force $F=T\sigma /2\pi R$ in analogue to the friction coefficient (as discussed in Section 3) [26,27].

Next, consider case (B). Equation (3) with Equations (21) and (22) leads to the velocity on N

$$\begin{array}{c}\hfill v=(\alpha -\beta )\frac{2\pi R}{N}=\frac{DN}{\pi R}tanh\left[\frac{\sigma }{2N{k}_B}\right].\end{array}$$

(24)

The velocity v increases with N and so the time of a full completion of the system cycle $\tau =2\pi R/v$ is reduced and minimizes for $N\to \infty$, see Figure 5.

5. Randomness Parameter and Variance in Cycle Completion Time

In the previous Section 3 and Section 4, we analyzed, for a biased random process, the dependence of different physical observables on the number intermediate sites N taken to complete a given cycle of operation, as a way to demonstrate the nonuniqueness of going to the continuous limit ($N\to \infty$) of this process. Another perspective of this problem is studied in Refs. [28,29,30,31], where the values of physical observables associated with enzyme-catalyzed cycles were used to set bounds on the number of intermediate cycle steps (Note that in difference to enzymic cycles where the step size is considered as constant, we reduce the stepsize $\Delta x=L/N$ in a cycle of finite lenght $L=2\pi R$ with increasing N to study the impact of the limit $N\to \infty$ on the physical observables). It was pointed out that, in addition to the average speed v and diffusion coefficient D, their ratio provides important information on the observed walk statistics [28,29,30,31]. Explicitly, we characterize the random process in terms of their forward and backward rates, $\alpha$ and $\beta$, respectively, and an equidistant step length $\Delta x$. If the process starts at $n=1$ and a random site n is reached at time t, then the process will be on average at site $\langle n\rangle =vt/\Delta x$, whilst the random diffusive process produces a variance in the site by $\langle \delta n^2\rangle \equiv \langle n^2\rangle -{\langle n\rangle }^2=2Dt/\Delta x^2$, see Equations (3) and (4) in the limit $t\to \infty$. These two quantities can be combined into a randomness parameter, which, for the given step size $\Delta x$, reads [28,29,30,31]

$$\begin{array}{c}\hfill r\equiv \frac{\langle \delta n^2\rangle }{\langle n\rangle }=\frac{2D}{\Delta xv}=\frac{\alpha +\beta }{\alpha -\beta },\end{array}$$

(25)

where v and D are defined by Equations (3) and (4), respectively. Alternatively, we may consider the random passage time $\tau$ at which, starting from $n=1$, the walk reached the site N for the first time, namely, a distance $N\Delta x$ from the starting point. For walks of uniform step length and finite bias, it has been shown [28,29] that, for a large enough N, the randomness parameter can be expressed in terms of the first two moments of the passage time distribution

$$\begin{array}{c}\hfill r=\frac{\langle {\tau }^2\rangle -{\langle \tau \rangle }^2}{{\langle \tau \rangle }^2}=\frac{\langle \delta {\tau }^2\rangle }{{\langle \tau \rangle }^2},\end{array}$$

(26)

where $\langle \tau \rangle$ is the average time for a cycle completion and $\langle \delta {\tau }^2\rangle$ is its variance. Note that for many enzyme reaction cycles, the backward reaction rates are often sufficiently low as to be negligible. In such cases, the pathway consists of a sequence of N forward reactions only and the randomness parameter ($r=N_{min}^{-1}$) can be used to estimate the minimal number of kinetic sites that compose the underlying kinetic model [29,31]. In general, when considering forward and backward steps and using the average cycle completion time $\langle \tau \rangle$, we can calculate the variance in cycle completion time to

$$\begin{array}{c}\hfill \langle \delta {\tau }^2\rangle =r{\langle \tau \rangle }^2.\end{array}$$

(27)

We can now apply the results of Section 3 and Section 4 to examine the behavior of these observables in our different limiting cases. In our case, N corresponds to the number of sites per cycle and, consequently, $\tau$ is the time for the process to complete the cycle. Increasing N is achieved by eventually approaching a continuous description, so the cycle length $N\Delta x=2\pi R$ is kept fixed.

Consider first the condition of a constant velocity and diffusion constant by increasing the number of sites N, see Section 3. Using Equations (12) and (13) in (25), the randomness parameter is

$$\begin{array}{c}\hfill r=\frac{D}{\pi Rv}N\end{array}$$

(28)

and is linear in N.

Next, for the condition of constant entropy production $\sigma$ per cycle under either constant velocity v or constant diffusion D, see Section 4, we find in both cases, using the respective rates in Equations (19) and (20) or Equations (21) and (22), the randomness parameter to be

$$\begin{array}{c}\hfill r=coth\left[\frac{\sigma }{2N{k}_B}\right].\end{array}$$

(29)

The randomness parameter (29) is determined by the number of sites N and the thermodynamic entropy production or heat dissipation into the environment, which equals, when neglecting the movement against an external force, the energy drop per cycle, see Section 2. Note that r tends to infinity in the limit $\sigma \to 0$ and in the continuous limit $N\to \infty$ since both limits reflect the equilibrium situation where forward and backward rates will be alike. In the limit $\sigma \to \infty$, given finite N, we find $r=1$, which is expected for the so-called “Poisson” motion since the infinite energy drop per cycle leads to an unidirectional motion [28].

With the randomness parameter at hand, we can now study the variance in the cycle completion time (Equation (27)). It has been shown that, for a biased random walk for a large N, the average completion time $\langle \tau \rangle =N{(\alpha -\beta )}^{-1}$ [32]. Consider first the cases (A) of keeping the velocity and diffusion coefficient constant and (B) keeping the entropy production per cycle and velocity constant. We find for (A) the given Equation (28), together with Equations (12) and (13) in Equation (27), where $\langle \tau \rangle =N{(\alpha -\beta )}^{-1}$, the variance in cycle completion time to be

$$\begin{array}{c}\hfill \langle \delta {\tau }^2\rangle =\frac{4D\pi R}{v^3}N,\end{array}$$

(30)

and, equivalent for case (B), by using Equation (29), together with Equations (19) and (20) in Equation (27)

$$\begin{array}{c}\hfill \langle \delta {\tau }^2\rangle =coth\left[\frac{\sigma }{2N{k}_B}\right]\frac{4{\pi }^2{R}^2}{v^2}.\end{array}$$

(31)

In both Equations (30) and (31), the variance in the cycle completion time increases monotonously with the site number N. This reflects the fact that, with an increasing number of sites N per cycle, the intersite rates become more alike, which increases the overall “randomness”, and, thus, $\langle \delta {\tau }^2\rangle$ for the total cycle completion.

In contrast, when N is changed while keeping a constant diffusion coefficient and entropy production, Equation (26), together with (24) and (29) lead to

$$\begin{array}{c}\hfill \langle \delta {\tau }^2\rangle ={coth}^3\left[\frac{\sigma }{2N{k}_B}\right]\frac{4{\left(\pi R\right)}^4}{D^2{N}^2}.\end{array}$$

(32)

Interestingly, $\langle \delta {\tau }^2\rangle$ goes through a minimum with increasing site number N, see Figure 6. It should be kept in mind, however, that Equation (26) and, consequently, Equation (27), were derived under the assumption that N is large so that this observation should not be regarded as conclusive.

6. N-Site Process with Time Step Discretization

Consider now the same biased random walk process in which a cycle of length $2\pi R$ is traversed in N steps, so that $N\Delta x=2\pi R$, but where the system is restricted to move (by intersite distance $\Delta x$) only at finite time intervals $\Delta t$. Indeed, small systems which are periodically driven can be thought of as discrete-time processes, see [33]. As shown below, $\Delta x$ and $\Delta t$ are not independent of each other but some freedom exists in their choices. $N\Delta E$ is the energy drop per such cycle (see Section 2, and recall that $\Delta E$ determines the detailed balance ratio of the forward and backward rates according to Equation (5)). The probability to be at site n, namely, at position $x=n\Delta x$ on the cycle at time $t=M\Delta t$, is governed by the Makrov chain (Note that in contrast to other descriptions of random walks in forms of a discrete Markov chain, e.g,. for waiting time distributions [34], where after a time interval, sampled from such a distribution, a “jump” always happens with constant probabilities p and $1-p$ for a forward backward jump, respectively, here, the “walker” can also remain at its original position after $\Delta t$ and where the probabilities depend linearly on $\Delta t$) [34]

$$\begin{array}{cc}\hfill P(x,t+\Delta t)& =\alpha \Delta tP(x-\Delta x,t)+\beta \Delta tP(x+\Delta x,t)\hfill \\ \hfill & +(1-\alpha \Delta t-\beta \Delta t)P(x,t).\hfill \end{array}$$

(33)

Here, $\alpha \Delta t$ and $\beta \Delta t$ are the probabilities (both assumed linear in $\Delta t$) to move a step forward and backward, respectively. Note that $(\alpha +\beta )\Delta t\le 1$ has to be imposed in Equation (33) to ensure positivity. As before, the process has periodic boundaries so that, after the final site $n=N$ has been reached, it restarts at the beginning $n=1$ and its original energy is restored by some external work reservoir between sites N and $N+1=1$ (see discussion in Section 2). We use this model to study the effect of time discretization on the dynamical properties of the process. To calculate the velocity v and diffusion coefficient D, we determine the generating function $P(s,t)={\sum }_x{s}^{x}P(x,t)$ where the moments can be calculated by $\langle x^m\rangle ={(s\partial /\partial s)}^{m}P(s,t){|}_{s=1}$. For the initial condition $P(x=0,t=0)=1$ (so that $P(s,t=0)=1$), we find the generating function to be

$$\begin{array}{cc}\hfill P(s,t=& M\Delta t)\hfill \\ \hfill & ={[\alpha \Delta t{s}^{\Delta x}+\beta \Delta t{s}^{-\Delta x}+(1-\alpha \Delta t-\beta \Delta t)]}^M.\hfill \end{array}$$

(34)

The velocity and diffusion coefficient are determined as follows. Starting at $x=0$, we find in the long time limit (at steady state) $t\to \infty$ (see details in Appendix D)

$$\begin{array}{cc}\hfill v& =\frac{\langle x\rangle }{t}=(\alpha -\beta )\Delta x;\hfill \end{array}$$

(35)

$$\begin{array}{cc}\hfill 2D& =\frac{\langle x^2\rangle -{\langle x\rangle }^2}{t}=(\alpha +\beta )\Delta x^2-{(\alpha \Delta x-\beta \Delta x)}^2\Delta t\hfill \\ \hfill & =(\alpha +\beta )\Delta x^2-v^2\Delta t.\hfill \end{array}$$

(36)

The velocity v, Equation (35), is the same as in the continuous time case (Equation (3)), whereas the diffusion coefficient in Equation (36) is smaller by $v^2\Delta t$ in comparison to the continuous time case of Equation (4). Refs. [35,36] associate the bigger variance in the continuous-time master equation with higher fluctuations in the total number of hops observed in a given time interval. Next, consider the process as N increases. As in Section 3 and Section 4, we may consider an increase in N while keeping v and D constant or while keeping only one of them together with $N\Delta E$ constant. As examples of the effect of moving in discrete time steps, we study the cases (A) constant v and D and (B) constant v and $N\Delta E$.

(A) Keeping v and D constant, we scale the rates again analog to Equations (19) and (20) with the total site number N (given by the intersite distance $\Delta x=2\pi R/N$)

$$\begin{array}{cc}\hfill \alpha & =\frac{(D+v^2\Delta t/2)N^2}{{\left(2\pi R\right)}^2}+\frac{vN}{4\pi R},\hfill \end{array}$$

(37)

$$\begin{array}{cc}\hfill \beta & =\frac{(D+v^2\Delta t/2)N^2}{{\left(2\pi R\right)}^2}-\frac{vN}{4\pi R}.\hfill \end{array}$$

(38)

Note that the modification of the rates $\alpha$ and $\beta$ depends on $\Delta t$. As before, this rescaling also implies a change in $\Delta E$ (see discussion in Section 3) so that the detailed balance relation is maintained. The condition $(\alpha +\beta )\Delta t\le 1$ in Equation (33) together with Equations (37) and (38) restricts the choices for $\Delta t$ given $\Delta x=2\pi R/N$ to

$$\begin{array}{c}\hfill 0\le \Delta t\le \frac{-D}{v^2}+\sqrt{\frac{D^2}{v^4}+\frac{\Delta x^2}{v^2}},\end{array}$$

(39)

which implies that $\Delta t$ and $\Delta x$ cannot be assigned independently of each other. In the limit $N\to \infty$ ($\Delta x\to 0$), this inequality (Equation (39)) becomes $0\le \Delta t\le \Delta x^2/2D$.

Next, consider the entropy production for this discrete hopping process. The average entropy change per step is

$$\begin{array}{c}\hfill \Delta \sigma =k_B\alpha ln\left[\frac{\alpha }{\beta }\right]+k_B\beta ln\left[\frac{\beta }{\alpha }\right],\end{array}$$

(40)

where the two terms represent the entropy change in a forward and backward step multiplied by the probabilities that the respective step occurs. The rate of entropy change at steady state (the entropy production rate) is given by

$$\begin{array}{cc}\hfill {\dot{\sigma }}_{SS}& =k_B\frac{\Delta \sigma }{\Delta t}=k_B(\alpha -\beta )ln\left[\frac{\alpha }{\beta }\right]\hfill \\ \hfill & =\frac{k_{B}N}{2\pi R}vln\left[\frac{\frac{(D+v^2\Delta t/2)N^2}{{\left(2\pi R\right)}^2}+v\frac{N}{4\pi R}}{\frac{(D+v^2\Delta t/2)N^2}{{\left(2\pi R\right)}^2}-v\frac{N}{4\pi R}}\right].\hfill \end{array}$$

(41)

Interestingly, comparing the resulting expression (41) to its analog (14) for the continuous master equation, we obtain a similar result, but with an additional term $v^2\Delta t/2$ added to the diffusion constant. The additional term effectively modifies the TUR relation (Equation (1)) as the relative uncertainty $2D/v^2$ changes. A similar observation was made in Ref. [37]. The resulting entropy production rate in Equation (41) for a given site number N per cycle is reduced if we choose a finite $\Delta t$ (given the restriction on choices of $\Delta t$ by Equation (39)), see Figure 7. This might be understood since, during a given time interval, the variance in position x on the cycle is reduced ((Equation (36)) by allowing intersite hops only in intervals $\Delta t$. In the continuous limit ($N\to \infty$, $\Delta x\to 0$ and $\Delta t\to 0$), however, Equation (41) yields

$$\begin{array}{c}\hfill {\dot{\sigma }}_{SS}=k_B\left[\frac{v^2}{D}\right],\end{array}$$

(42)

which is the same result as that obtained in this limit in Section 3 (see Equation (14)).

(B) Next, consider the dependence on N under the condition of a constant energy drop $N\Delta E$ per cycle and constant velocity v. This is the analogue consideration as in Section 4, but where the intersite hops are only allowed at time intervals $\Delta t$ (which are restricted by the given N according to Equation (46) below). We assume that all energy invested into the system is dissipated as heat into the environment, so the steady-state entropy production per cycle is

$$\begin{array}{c}\hfill \sigma =N\frac{\Delta E}{T}=k_{B}Nln\left(\frac{\alpha }{\beta }\right),\end{array}$$

(43)

where N is the total number per cycle.

Given the constant velocity, Equation (35), the forward and backward rates are equivalent to Equations (19) and (20):

$$\begin{array}{cc}\hfill \alpha & =\frac{vN}{2\pi R\left(1-e^{\frac{-\sigma }{N{k}_B}}\right)},\hfill \end{array}$$

(44)

$$\begin{array}{cc}\hfill \beta & =\frac{vN}{2\pi R\left(e^{\frac{\sigma }{N{k}_B}}-1\right)}.\hfill \end{array}$$

(45)

The restriction $(\alpha +\beta )\Delta t\le 1$ in Equation (33), together with Equations (44) and (45), limits the choices for $\Delta t$ given $\Delta x=2\pi R/N$ to

$$\begin{array}{c}\hfill 0\le \Delta t\le \frac{\Delta x}{v}tanh\left[\frac{\sigma }{2N{k}_B}\right].\end{array}$$

(46)

In the limit $N\to \infty$ ($\Delta x\to 0$ and $\Delta t\to 0$), the inequality in Equation (46) becomes $0\le \Delta t\le \Delta x\sigma /\left(vN{k}_B\right)$.

For a given N and $\Delta x=2\pi R/N$, the time step $\Delta t$ needs to satisfy the inequality (46). Here, we take

$$\begin{array}{c}\hfill \Delta t=a\frac{\Delta x}{v}tanh\left[\frac{\sigma }{2N{k}_B}\right]\end{array}$$

(47)

with $0\le a\le 1$ and use Equation (36) to obtain

$$\begin{array}{cc}\hfill D& =\frac{1}{2}\left((\alpha +\beta )\Delta x^2-v^2\Delta t\right)\hfill \\ \hfill & =\frac{v\pi R}{N}\left(coth\left[\frac{\sigma }{2N{k}_B}\right]-atanh\left[\frac{\sigma }{2N{k}_B}\right]\right).\hfill \end{array}$$

(48)

Interestingly, the dependence on time discretization translates here to a dependence of D on the choice of a, see Figure 8. Given N for different finite time $\Delta t$ (scaled between $0\le a\le 1$), the diffusion coefficient, and consequently, the variance in $x=n\Delta x$, are strongly reduced for increasing $\Delta t$ (less fluctuations in the total number of transitions for finite $\Delta t$). As expected, in the limit $N\to \infty$ and $\Delta t\to 0$, the diffusion constant takes the form $D=v/F{k}_{B}T=\mu k_{B}T$ with the mobility $\mu =v/F$ and the corresponding force $F=T\sigma /2\pi R$, [26,27], see discussion in Section 4. Therefore, given the chosen $\Delta t$ and its above-discussed effect on D, the latter increases or decreases with N to the final value $D=\mu k_{B}T$ as depicted in Figure 8.

To summarize this section, when describing the dynamics of a (cyclic) process in discrete time intervals, the thermodynamic properties of the process, e.g., the entropy production or diffusion coefficient, are affected by this time discretization. The discretization in time, however, cannot be chosen arbitrarily but must obey the bounds given by the state-space discretization of the process. If the process dynamics become continuous state-space dynamics ($N\to \infty$), the time evolution needs to be described by intervals $\Delta t\to 0$, i.e., equivalently by a continuous time scale and all effects vanish.

7. Conclusions

In this paper, we investigate a cyclic and unithermal thermodynamic process using a model of a biased random walk between N sites on a cycle of a given length with an exact solvable master equation. We note that many dynamical site (state)-space processes with periodic boundary conditions may be equivalently mapped to this process. The limit $N\to \infty$ corresponds to the continuous limit that is usually captured by a Fokker–Planck equation. This limit is taken by keeping low order moments of system observables, i.e., the velocity v and diffusion coefficient D, constant. We show that the entropy produced, or, equivalently, the energy drop per cycle, is reduced when moving towards this continuous description. This has direct consequences for the efficiency of transferring input power into useful output power when an opposing force acts on the cyclic process. In particular, more power can be extracted from the process with an increasing number of sites N per cycle length.

An important outcome of our analysis is that the procedure in going to the continuous description of the process is not unique and depends on the physical observables that are assumed to be invariant under this limiting process. In addition to taking the limit $N\to \infty$ while keeping v and D constant, we also analyzed this limiting process while keeping v or D and $N\Delta E$ constant. Interestingly, considering the limiting process under constant v and $N\Delta E$, we show that the diffusion coefficient D for a finite cycle velocity v in the limit $N\to \infty$ takes the same value as in linear response $v\to 0$ limit. Additionally, when analyzing the cycle randomness statistics and, in particular, the variance in the cycle completion time $\langle \delta {\tau }^2\rangle$, we found that, with increasing N, $\langle \delta {\tau }^2\rangle$ increases, signaling the increasing randomness in the cycle. Finally, we studied the dependence on N in the case where the transitions between sites are only allowed at fixed time intervals $\Delta t$. We found that not only the entropy production rate per cycle (when v and D are kept constant), but also the diffusion coefficient D (when v and $N\Delta E$ are kept constant), are strongly affected by the way time discretization is introduced for a given N.

In conclusion, one can use the total site number N as a control parameter to design “useful” physical and thermodynamic (cycle) processes by keeping desirable observables constant and affecting others. It may provide valuable insights into the engineering of small (molecular) machines capable of performing specific tasks with high efficiency and precision. Further investigations of the discrete to continuous transition in state space and its potential impact on information-to-work conversion are avenues for future research.