Medium Entropy Reduction and Instability in Stochastic Systems with Distributed Delay

Abstract

Many natural and artificial systems are subject to some sort of delay, which can be in the form of a single discrete delay or distributed over a range of times. Here, we discuss the impact of this distribution on (thermo-)dynamical properties of time-delayed stochastic systems. To this end, we study a simple classical model with white and colored noise, and focus on the class of Gamma-distributed delays which includes a variety of distinct delay distributions typical for feedback experiments and biological systems. A physical application is a colloid subject to time-delayed feedback control, which is, in principle, experimentally realizable by co-moving optical traps. We uncover several unexpected phenomena in regard to the system’s linear stability and its thermodynamic properties. First, increasing the mean delay time can destabilize or stabilize the process, depending on the distribution of the delay. Second, for all considered distributions, the heat dissipated by the controlled system (e.g., the colloidal particle) can become negative, which implies that the delay force extracts energy and entropy of the bath. As we show here, this refrigerating effect is particularly pronounced for exponential delay. For a specific non-reciprocal realization of a control device, we find that the entropic costs, measured by the total entropy production of the system plus controller, are the lowest for exponential delay. The exponential delay further yields the largest stable parameter regions. In this sense, exponential delay represents the most effective and robust type of delayed feedback.

1. Introduction

In addition to the omnipresent noise, many complex systems are governed by dynamical equations that involve some sort of memory or time delay. The latter may stem from delayed response in the communication between (active) constituents, retardation effects due to finite signal transmission times, or maturation times in population growth. Another common source of delay is the presence of feedback loops, which typically involve a delay due to the time lag between feeding a signal and receiving the response [1,2,3,4,5,6,7]. Feedback is a widespread mechanism in nature, encountered in various biological systems (such as gene regulatory networks [8,9,10,11,12,13,14], the cell metabolism [15,16], or the chemotactal motion of bacteria [17,18,19]); and is further implemented in numerous artificial systems (such as robots [5], autonomously driving cars [20], or quantum devices [21,22]). Recent experiments on small-scale, fluctuating systems successfully apply feedback control involving video microscopy and co-moving laser traps to create virtual potentials [23,24] or realize arbitrary interactions between colloidal particles [25,26]. Such experiments give rise to a stochastic motion with time delay of the feedback-controlled colloidal particles [2,23,24,25,26,27,28,29].

As a most simple way to mathematically describe stochastic motion with time delay, we will consider the overdamped Langevin equation

$$\begin{array}{cc}\hfill {\dot{X}}_0\left(t\right)=& {\int }_0^{t}K(t-t^{\prime })X_0\left(t^{\prime }\right)\mathrm{d}{t}^{\prime }-V^{\prime }+\xi \left(t\right),\hfill \end{array}$$

(1)

with a noise term $\xi$, an external potential V, and a linear delay force, ${\int }_0^{t}K(t-t^{\prime })X_0\left(t^{\prime }\right)\mathrm{d}{t}^{\prime }$, where K defines the distribution of the delay. The delay distribution depends on the details of the underlying (feedback) mechanism. For example, in biological systems, the delay often has a rather broad distribution [11,30,31], while in control theory, a delta-distributed delay is more common, $K(t-t^{\prime })=\delta (t-t^{\prime }-\tau )$, giving rise to a single delay time, $\tau$. The prime physical example motivating this paper is a colloidal particle under time-delayed feedback control, where the delay distribution is, in principle, externally tunable by the involved computer.

From a practical point of view, delay is often unwanted, as it may destroy the system’s stability [32,33,34,35,36,37,38] or induce chaos [39,40]. It furthermore dramatically increases the difficulty of the mathematical description, as the combination of delay and noise yields non-Markovian stochastic processes with infinite-dimensionality [31,32,37,38,41], that are, moreover, automatically far from thermal equilibrium [42,43]. At the same time, it is known that delay can also be beneficial, for example, it can stabilize period solutions [3,44,45,46] or induce coherent oscillations [43,47], which is why delay is sometimes intentionally implemented, see [45,48,49]. Furthermore, it has recently been shown that delay may yield intriguing thermodynamic behavior. For example, a delay force acting on a colloid can induce a net heat flow from the heat bath to the colloid, i.e., a reduction of the medium entropy [43,50]. This refrigeration effect, called feedback cooling, is closely related to feedback cooling and “entropy pumping” in Markovian inertial systems with velocity-dependent feedback [51,52,53,54,55,56] (specifically, the delay force has a comparable effect as a velocity-dependent force, and both induce entropy pumping, see [43,57]). While these effects are, in principle, known, we are still only at the beginning of a full understanding of the impact of delay on stochastic dynamics. Given the omnipresence of time delay in many fluctuating systems, deepening our theoretical understanding seems indeed crucial.

In this paper, we discuss the influence of the distribution of the delay (i.e., the functional form of K in Equation (1)) on dynamical and thermodynamical properties in the overdamped case. In contrast to algebraically decaying delay distributions, we here focus on “short-ranged” ones that eventually decay exponentially fast in the long time limit, which are more important in the context of feedback and naturally arise in biological systems. For a recent discussion of other kernels and corresponding stability issues, see [58]. Specifically, we will perform a linear stability analysis (around local extrema of an external potential) and study how the stability boundaries shift when we change nothing but the distribution of the delay while keeping the mean delay time and the weight of the kernel constant. Gamma-distributed delays turn out to be particularly suitable to study these questions. They are versatile (including single-exponential, and delta-distributed delay, as well as distributions with a pronounced maximum of finite width), and, at the same time, are embeddable in a higher-dimensional Markovian system. We will exploit both properties in this paper. As a prominent example for thermodynamic behavior induced by delay, we will further consider how the distribution of delay influences the aforementioned “negative dissipation”. We also study the impact of colored noise. Lastly, we discuss the total entropy production and the thermodynamic efficiency for a specific realization of a feedback controller involving non-reciprocal interactions between some linear, stochastic internal degrees of freedom (which we interpret as the “memory cells”) and gives rise to a delay force with Gamma-distributed delay.

2. Model

We have already introduced the overdamped Langevin equation (LE) in Equation (1) that can be used, e.g., to describe the motion of a colloidal particle ($X_0$ being the colloidal position) under the impact of a time-delayed feedback controller. More specifically, we will consider

$$\begin{array}{cc}\hfill {\dot{X}}_0\left(t\right)=& {\int }_0^{t}K(t-t^{\prime })X_0\left(t^{\prime }\right)\mathrm{d}{t}^{\prime }-V^{\prime }+{\xi }_0+\nu ,\hfill \end{array}$$

(2)

which accounts for two types of noise, $\xi ={\xi }_0+\nu$. First, ${\xi }_0$ is a zero-mean Gaussian white noise with $\langle {\xi }_0\left(t\right){\xi }_0\left(t^{\prime }\right)\rangle =2k_{\mathrm{B}}{\mathcal{T}}_0/{\gamma }_0{\delta }_{ij}\delta (t-t^{\prime })$ with the Boltzmann and friction constants $k_{\mathrm{B}}$, ${\gamma }_0$, and $\langle \dots \rangle$ denoting the ensemble average. This white noise stems from the surrounding heat bath at temperature ${\mathcal{T}}_0$. Second, $\nu$ is a Gaussian colored noise which may account for a finite precision of the feedback controller [as we elaborate below, and define in Equation (6)]. The term ${\int }_0^{t}K(t-t^{\prime })X_0\left(t^{\prime }\right)\mathrm{d}{t}^{\prime }$ models the delay force with distributed delay. Note that the linearity in $X_0$ is indeed a typical assumption in the experimental realizations of feedback by co-moving laser traps [59,60].

The external potential V in Equation (2) depends on the specific system at hand. An important example is $V=a_{}{X}_0^2/2$, giving rise to a linear force $-a_{}{X}_0$, which could represent an approximate potential gained by linearization of a more complicated V around a (local) extremum, or it could represent an actual parabolic potential. For $a>0$, the LE (2) may be used to describe a colloidal particle in an (optical) trap, while $a<0$ corresponds to a colloid on a “parabolic mountain”, see Figure 1b.

A typical feature of feedback is a time delay $\tau$. The focus of this paper is to study the impact of different delay distributions. For this purpose, a class of Gamma-distributions defined by [61]

$$\begin{array}{cc}\hfill K(\Delta t)& =k\frac{n^n\Delta t^{n-1}}{{\tau }^n(n-1)!}{e}^{-\frac{n\Delta t}{\tau }},\hfill \end{array}$$

(3)

(with $\Delta t\ge 0$) turns out to be particularly suitable for the following reasons:

(i)

By changing n, we can model various common types of delay with distinct characteristics, see Figure 2. For $n=1$, the kernel is a simple exponential decay. For $n>1$, $K(\Delta t)$ is peaked around $\Delta t=\tau$, with peaks becoming sharper upon increasing n. In the limit $n\to \infty$, it reaches a delta-distribution, ${lim}_{n\to \infty }K(\Delta t)=k\delta (\Delta t-\tau )$ [41]. Thus the ansatz Equation (3) includes the case of a discrete (single) delay.

(ii)

The weight of K, which defines the feedback gain k, is identical for all n: ${\int }_0^{\infty }K(\Delta t)\mathrm{d}\Delta t=k$.

(iii)

The mean delay time τ is identical for all n:

$$\begin{array}{c}\hfill \frac{{\int }_0^{\infty }K(\Delta t)\Delta t\mathrm{d}\Delta t}{{\int }_0^{\infty }K(\Delta t)\mathrm{d}\Delta t}=\tau .\end{array}$$

(4)

(iv)

There exists a corresponding Markovian system that generates a memory of the form Equation (3) (as explained in the following section). Using this route allows us to obtain various analytical expressions that would be notoriously difficult to derive based on the non-Markovian LE (2) alone.

2.1. Markovian Representation

Alternative to the non-Markovian LE (2), one may formulate a Markovian equation by embedding the system in a higher-dimensional space (adding “auxiliary variables”) [41]. We note that such a Markovian representation is not unique, i.e., different Markovian models lead, after projection onto $X_0$, to the same non-Markovian Equation (2). Here, we will consider the $(n+1)$-dimensional Markovian system

$$\begin{array}{cc}\hfill \underline{\dot{X}}& =\mathbf{A}\underline{X}+\underline{\xi }+\underline{\varphi }\hfill \end{array}$$

(5a)

$$\begin{array}{cc}\hfill \mathbf{A}& =\left(\begin{array}{ccccc}-a_{}& 0& 0& ...& k\\ n/\tau & -n/\tau & 0& ...& 0\\ 0& n/\tau & -n/\tau & ...& 0\\ & & ⋮& & \\ 0& ...& 0& n/\tau & -n/\tau \end{array}\right)\hfill \end{array}$$

(5b)

with the vector $\underline{X}={(X_0{X}_1\cdots X_n)}^T\in {\mathbb{R}}^{n+1}$, and the noise vector $\underline{\xi }={({\xi }_0{\xi }_1\cdots {\xi }_n)}^T$ containing zero-mean Gaussian white noises with $\langle {\xi }_i\left(t\right){\xi }_{j>0}\left(t^{\prime }\right)\rangle =2k_{\mathrm{B}}{\mathcal{T}}^{\prime }{\delta }_{ij}\delta (t-t^{\prime })$ and $\langle {\xi }_0\left(t\right){\xi }_0\left(t^{\prime }\right)\rangle =2k_{\mathrm{B}}{\mathcal{T}}_0/{\gamma }_0{\delta }_{ij}\delta (t-t^{\prime })$. In the case ${\mathcal{T}}^{\prime }=0$, this embedding is called the “linear chain trick” [31,62]. The asymmetry of the coupling matrix $\mathbf{A}$ is associated with non-reciprocal interactions between the $X_j$ [57]. Further, if the external potential in Equation (2) is parabolic, $\underline{\varphi }=0$, otherwise $\underline{\varphi }=[a{X}_0-V^{\prime }]{(10\cdots 0)}^T$.

In the last part of this paper (Section 5), we will employ a physical interpretation of the Markovian representation (5). Specifically, one may interpret the variables $X_{j>0}$ as the memory cells of a feedback controller, which has a shift register architecture. This idea is illustrated in Figure 3, and further elucidated in Appendix A.1. The noise terms ${\xi }_{j>0}$ then give the controller degrees of freedom a finite temperature ${\mathcal{T}}^{\prime }$. Alternatively, one may interpret the terms ${\xi }_{j>0}$ as errors of the memory device, in which it seems appropriate to define their amplitudes as n-dependent quantities, see the discussion in Section 2.4.

To return from Equations (2)–(5) one can employ a projection [41,63], as we derive in Appendix A. This gives rise in Equation (2) to the delay distribution (Equation (3)) and the Gaussian colored noise $\nu$ with noise correlations (for $\Delta t\ge 0$)

$$C_{\nu }(\Delta t)=\langle \nu \left(t\right)\nu (t+\Delta t)\rangle =\frac{k_{\mathrm{B}}{\mathcal{T}}^{\prime }}{k^{-2}}\sum _{p=0}^{n-1}\sum _{l=0}^p\frac{2^{l-2p}(2p-l)!}{p!(p-l)!l!}\frac{e^{-n\Delta t/\tau }\Delta t^l}{{(\tau /n)}^{l-1}}.$$

(6)

We emphasize that the dynamics of $X_0$ is identical in both representations (2) and (5).

2.2. Colored Noise

Let us take a closer look at the colored noise (Equation (6)). It is zero for ${\mathcal{T}}^{\prime }\equiv 0$ (corresponding to a deterministic controller), and otherwise positive. Interestingly and contrary to the delay distribution K, which dramatically changes between $n=1$ and $n=2$, the correlations of the colored noise (Equation (6)) remain very similar for all n. They decay monotonically in all cases (see Figure 2). Indeed, as we have explicitly shown in [57], the system with colored noise (Equation (6)) and delay distribution (Equation (3)) never fulfills the fluctuation-dissipation relation (FDR) of second kind, indicating the inherent non-equilibrium nature of the non-Markovian process $X_0$ (Equation (2)). An exception is the case $n=1$ with

$$\begin{array}{c}\hfill k\tau ={\mathcal{T}}_0/{\mathcal{T}}^{\prime },\end{array}$$

(7)

where FDR is fulfilled. As we have discussed in [57], a way to understand the broken FDR are the involved non-reciprocal, interactions in the corresponding Markovian representation (Equation (5)). We have further shown in [57] that the detailed balance (DB) condition is as well only fulfilled for $n=1$ and condition (Equation (7)).

2.3. Limit of Infinitely Large System

In many physical and biological systems, a delay distribution with a non-zero width is the more realistic scenario [61,64]. For example, in the context of feedback, the non-zero width may come from a finite precision and resolution of the control device. In our interpretation of $X_{j>0}$, this corresponds to a finite number n, i.e., the controller has a finite memory capacity. However, in theoretical studies, the delay is often assumed to be discrete, i.e., infinitely sharp. In our approach, this corresponds to the limit $n\to \infty$, where $K(\Delta t)\to \delta (\Delta t-\tau )$ [41].

In this limit, we find a surprising result for the colored noise, that is, the noise correlation entirely vanishes

$$\underset{n\to \infty }{lim}{C}_{\nu }(\Delta t)=0$$

(8)

(irrespective of the value of ${\mathcal{T}}^{\prime }$), implying that the colored noise $\nu$ itself vanishes. Hence, Equation (2) becomes a delay equation with white noise, reading

$${\dot{X}}_0\left(t\right)=-a_{}{X}_0+k{X}_0(t-\tau )+{\xi }_0.$$

(9)

Proof. 

We perform the limit of $n\to \infty$ of the noise correlation given in Equation (6), which read for $\Delta t\ge 0$

$$\frac{C_{\nu }(\Delta t)}{k_{\mathrm{B}}{\mathcal{T}}^{\prime }{k}^2}=\sum _{p=0}^{n-1}\sum _{l=0}^p\frac{n^{l-1}(2p-l)!}{{\tau }^{l-1}{2}^{2p-l}p!(p-l)!l!}{e}^{-n\Delta t/\tau }\Delta t^l.$$

(10)

To this end, we calculate the weight of Equation (10), i.e., $\Psi ={\int }_0^{\infty }{C}_{\nu }/\left(k_{\mathrm{B}}{\mathcal{T}}^{\prime }{k}^2\right)\mathrm{d}(\Delta t)$, yielding

$$\Psi =\sum _{p=0}^{n-1}\sum _{l=0}^p\frac{2^{l-2p}(2p-l)!}{{(\tau /n)}^{l-1}p!(p-l)!l!}\frac{{\tau }^{l+1}}{n^{l+1}}{\int }_0^{\infty }{e}^{-u}{u}^l\mathrm{d}u$$

(11)

This can further be simplified to

$$\Psi =\frac{{\tau }^2}{n^2}\sum _{p=0}^{n-1}\sum _{l=0}^p\frac{1}{2^{2p-l}}\frac{(2p-l)!}{p!(p-l)!l!}l!=\frac{{\tau }^2}{n^2}\sum _{p=0}^{n-1}\sum _{l=0}^p\frac{(2p-l)!}{p!(p-l)!}{\left[\frac{1}{2}\right]}^{2p-l}.$$

(12)

Now we use the binomial theorem to further simplify this expression, and find

$$\Psi =\frac{{\tau }^2}{n^2}\sum _{m=1}^n\sum _{r=0}^p\left(\genfrac{}{}{0pt}{}{p}{r}\right){\left[\frac{1}{2}\right]}^r{\left[\frac{1}{2}\right]}^{p-r}=\frac{{\tau }^2}{n^2}\sum _{m=1}^n{\left[\frac{1}{2}+\frac{1}{2}\right]}^{2(n-m)}=\frac{{\tau }^2}{n^2}\sum _{m=1}^{n}1=\frac{{\tau }^2}{n}.$$

(13)

Hence, the weight vanishes as $n\to \infty$. Since $C_{\nu }$ is obviously a non-negative function of $\Delta t$, this readily implies (8). □

2.4. Alternative Choice of ${\mathcal{T}}^{\prime }$

So far, we have considered an n-independent temperature of the auxiliary variables, ${\mathcal{T}}^{\prime }$, which seems a reasonable assumption when ${\mathcal{T}}^{\prime }$ describes the temperature of the “controller degrees of freedom” $X_{j>0}$. Alternatively, one may scale the temperatures of the auxiliary heat baths with n, specifically, ${\mathcal{T}}^{\prime }=n{\mathcal{T}}^{″}$ in Equation (5). In Appendix A.1, we provide a reasoning why this could be a better choice when the noises ${\xi }_{j>0}$ are interpreted as errors of the controller. Using the same reasoning as before one may show that, in the alternative choice, the colored noise in Equation (2) has a weight $\Psi$ which is independent of n [contrary to the result Equation (13)]. The corresponding equation for $n\to \infty$, i.e., Equation (9), then contains a white noise at temperature ${\mathcal{T}}_0+{\mathcal{T}}^{\prime }{\left(k\tau \right)}^2$. Thus, this choice of ${\mathcal{T}}^{\prime }$ results in controller errors whose effect does not vanish as $n\to \infty$, which seems indeed more plausible than the standard choice, where the controller errors vanish in this limit. We note that all of the following results also apply to this alternative choice just by setting ${\mathcal{T}}^{\prime }=n{\mathcal{T}}^{″}$.

3. Stability for Different n

Let us now turn to the dynamics of $X_0$. A prominent property of systems with delay are delay-induced oscillations (which can generally not occur in first order ODEs, but arise in first order DDEs [33]). In linear systems, they are either damped, such that a stable steady state is reached in the limit $t\to \infty$, or their amplitude increases with time yielding instability. Here we explore whether, for a given $a_{}$, k, $\tau$ and n, the system reaches a stable steady state (for more mathematical discussions of noise-free systems with Gamma-distributed delay, see [65,66]). For a stochastic system (2) with a general potential V, the following analysis determines the linear stability of the fixed points. As an illustration, imagine the task to stabilize a particle on a parabolic mountain (when $a_{}<0$) by means of a delayed feedback force (Figure 1). As we now show, the system’s stability significantly depends on the distribution of the delay, even if the feedback gain and mean delay time stay the same.

Determining the stability of the noisy process $X_0$ amounts to checking whether the moments stay finite or diverge as $t\to \infty$. In the here considered examples, the steady state probability density is Gaussian [32,67] (with zero mean due to the system’s symmetry). Thus, we need to consider the second moment, $\langle X_0^2\left(t\right)\rangle$. A calculation of the latter for different n is provided in Appendix B. For $n>1$, this yields cumbersome integral expressions. Interestingly, we found that the stability boundaries, however, do not depend on the temperatures ${\mathcal{T}}^{\prime }$ and ${\mathcal{T}}_0$. We conclude that the type of noise or the presence of noise at all does not have any impact on the linear stability of the system. This includes, in particular, the colored noise and the related additional memory.

Owing to these insights, we here discuss a simpler route to check the stability based on the noise-free case (${\mathcal{T}}_0={\mathcal{T}}^{\prime }\equiv 0$), where (5) reduces to

$$\underline{\dot{X}}=\mathbf{A}\underline{X}.$$

(14)

This linear matrix equation has solutions of the form $\underline{X}=\underline{X}\left(0\right)e^{\mathbf{A}t}$, such that the stability boundaries can be simply determined by calculating the real part of the largest eigenvalue of the coupling matrix $\mathbf{A}$, called $\mathsf{\lambda }$. If $\mathsf{\lambda }>0$, the system is unstable, while it is stable if $\mathsf{\lambda }<0$.

Due to the sparseness of $\mathbf{A}$, this strategy allows to determine the stability boundaries up to very large values of n. However, it is still limited to finite n. To determine to stability in the limit $n\to \infty$, we consider directly the noise-free delay Equation (9), that is

$${\dot{X}}_0=-a_{}{X}_0\left(t\right)+k{X}_0(t-\tau ).$$

(15)

This equation can also be solved by an exponential ansatz $X_0\left(t\right)=X_0\left(0\right)e^{-\widehat{\mathsf{\lambda }}t}$, yielding the transcendental equation [68,69]

$$\widehat{\mathsf{\lambda }}-a_{}=-k{e}^{\widehat{\mathsf{\lambda }}\tau }.$$

(16)

In the trivial cases $k=0$ and $\tau =0$, the solutions are $\widehat{\mathsf{\lambda }}=a_{}$ and $\widehat{\mathsf{\lambda }}=a_{}-k$, respectively, immediately giving some stability boundaries. Further, if $k=a_{}$, a solution is $\widehat{\mathsf{\lambda }}=0$. In general, the solutions of Equation (16) read [68,69]

$${\widehat{\mathsf{\lambda }}}_m=-\frac{1}{\tau }[W_m\left(k{e}^{a_{}\tau }\tau \right)-a_{}\tau ],$$

(17)

with the infinitely many branches of the Lambert-W function $W_m$, $m\in \mathbb{Z}$. The branch $m=0$ has the highest real part. Thus, the stability of the DDE changes at ${\widehat{\mathsf{\lambda }}}_m=0$.

Figure 4 and Figure 5 show the stability boundaries for various n of the deterministic system with $a_{}>0$ and $a_{}<0$, respectively, in the plane spanned by the feedback gain k and mean delay time $\tau$ (given in units of $\left|a\right|$ and $1/\left|a\right|$, respectively). Note that the figures hold for arbitrary values of $a_{}>0$ and $a_{}<0$, respectively. The parameter $\left|a\right|$ can be scaled out from the noise-free Langevin equation upon rescaling the time by $t\to t/\left|a\right|$. The system in Figure 4 (where $a_{}>0$) corresponds to a colloidal particle in a harmonic trap, which is as expected stable in the absence of delay force ($k=0$ or $\tau =0$). In contrast, the system displayed in Figure 5 (where $a_{}<0$) without delay force corresponds to a colloidal particle in a reversed trap, i.e., on a “parabolic mountain”, which is not stable, consistent with our expectation. However, in the presence of delayed feedback, stable and unstable regions emerge for both signs of a and for all values of n.

Figure 4 and Figure 5 reveal that there is a critical k value of $k^{*}=a_{}$ above which the system is generally unstable for all n. To get an intuitive understanding for this critical value, we assume for a moment that the particle would for all past times $t^{\prime }<t$ stand still, i.e., $X_0\left(t^{\prime }\right)\equiv X_0\left(t\right)$. Under this assumption ${\int }_0^{t}K(t-t^{\prime })X_0\left(t^{\prime }\right)\mathrm{d}{t}^{\prime }=k{X}_0\left(t\right),\forall n$ because ${\int }_0^{t}K(\Delta t)\mathrm{d}\Delta t=k$, such that Equation (2) reduces to

$${\dot{X}}_0\left(t\right)=(k-a_{})X_0\left(t\right),$$

(18)

irrespective of the value of n (including finite n and $n\to \infty$), and irrespective of the values of k, a and $\tau \ge 0$. Now, it becomes clear that the system is generally unstable if $k>a_{}$, and that $k^{*}=a_{}$ is a critical value (for all n).

We now focus on the regions below $k^{*}$, where the system with $\tau =0$ is stable, but may be destabilized by the delay. In this region, we make several interesting observations.

Most prominently, the system with exponentially distributed delay ($n=1$) has the largest stable area for positive and negative $a_{}$. Further, the stability region of every $n=\tilde{n}$, includes the whole $n=\tilde{n}+1$ region, while the reverse is never found. This means, the system generally becomes less stable as the delay distribution gets more localized around the delay time $\tau$. This trend (which has also been found in the mathematical analysis of corresponding deterministic systems [65]) is very robust, and also holds for higher n (e.g., $n=4,5,6$) and other $a_{}$. To achieve stability, it appears to be beneficial to take into account a “larger fraction of the past”. Thus, if the aim of a controller was to stabilize $X_0$ at a given location, the feedback with exponential delay would be the most effective and robust one.

Another remarkable aspect is the behavior for large mean delay times, $\tau$. Only if $a_{}<0$ (“particle on the parabolic mountain”), increasing the mean delay time seems to generally destabilize the system. Then there even exists a maximal $\tau$ value of ${\tau }^{*}=-1/a_{}$, where the system is unstable for all n and k, see Figure 5. For $a_{}>0$ (“particle in the trap”), the behavior is very different. Here, an increased mean delay time can be both, disadvantageous or advantageous for stability (depending on n and k). Interestingly, for n values $1<n<\infty$, we observe a reentrant behavior: by increasing the mean delay time, the linear stability is destroyed, but by further increasing $\tau$, the system becomes stable again. We suspect that this behavior is a consequence of an interplay with an internal timescale, specifically, the relaxation time within the parabolic potential well, which is of order $\sim 1/|a_{}|$. (Note that this estimate stems from solving just the conservative part on the right hand side of the LE ${\dot{X}}_0=-a{X}_0$. Strictly speaking, in the presence of the delay force, all timescales may be shifted, including the relaxation times.). If the mean delay time matches this internal timescale, the delay-induced oscillations might be resonantly enhanced, destroying the stability. However, this reentrant behavior is not observed for $n\to \infty$ and $n=1$.

4. Delay-Induced Heat Flow

Let us now turn to a thermodynamic consideration of the systems. To this end, we utilize and extend results from Ref. [57], where we have explored the connection between coupling topology $\mathbf{A}$ and the thermodynamic properties for generic linear systems of the type (5). We note that more general coupling topologies also allow to model other physical systems, as active particles [57]. Here we will focus on the topology Equation (5b), which yields the Gamma-distributed delay distributions. We will further give two special cases our main attention: the case of isothermal conditions, i.e., system and controller have the same temperature ${\mathcal{T}}^{\prime }={\mathcal{T}}_0$ (red symbols in the Figure 6 and the figures below), and the case of a “perfect” (noise-less) controller, i.e., ${\mathcal{T}}^{\prime }=0$ (black symbols in Figure 6).

In steady state, the mean Shannon entropy of the system, i.e., $\langle -k_{B}ln{\rho }_0\left(x\right)\rangle$ [70] is, per se, conserved (because ${\partial }_t{\rho }_0=0$). However, nonequilibrium steady states are typically characterized by an entropy flow between system and its bath associated with heat flow. We therefore start by studying the magnitude and sign of the steady-state mean heat flow ${\dot{Q}}_0$ between $X_0$ and its bath induced by the delay force, which is proportional to the medium entropy production rate ${\dot{S}}_m={\dot{Q}}_0/{\mathcal{T}}_0$. According to the framework by Sekimoto [71], the stochastic heat $\delta q_0$ flowing from $X_0$ to its bath, i.e., the dissipation of $X_0$ per infinitesimal time step $\mathrm{d}t$ is given by [71]

$$\delta q_0/\mathrm{d}t=[{\dot{X}}_0-{\xi }_0]\circ {\dot{X}}_0.$$

(19)

Note that we define the heat such that $\delta q_0>0$ indicates energy transfer from the particle to the bath (different from [71]). The heat flux (19) is independent of the question whether the non-Markovian (2) or the Markovian description (5) is employed. We will thus exploit the latter, allowing us to handle the system analytically in the case of linear forces. We are particularly interested in the mean heat production or dissipation rate, and find from Equations (5) and (19)

$${\dot{Q}}_0:=\langle \delta q_0/\mathrm{d}t\rangle \stackrel{\left(5\right)}{=}\langle [-a_{}{X}_0+k{X}_n]\circ {\dot{X}}_0\rangle .$$

(20)

Further, in the steady state, $\langle X_j{\dot{X}}_j\rangle$-correlations vanish, since $2\langle X_i\dot{X_i}\rangle =\mathrm{d}\langle X_i^2\rangle /\mathrm{d}t=0$, such that the mean heat production can be expressed based on the positional correlations only

$${\dot{Q}}_0=k^2\langle X_n^2\rangle -k{a}_{}\langle X_0{X}_n\rangle .$$

(21)

From Equation (21), analytical expressions for ${\dot{Q}}_0$ can be derived for any n, as outlined in Appendix B. Noteworthy, the steady-state heat production rate equals the work applied to the system by the external delay force [70,71]. Using the closed-form expressions $\langle X_1^2\rangle$, $\langle X_0{X}_1\rangle$ from [57,72], one finds from Equation (21) for the case $n=1$

$$\frac{{\dot{Q}}_0}{k_{\mathrm{B}}}=\frac{{\mathcal{T}}_0(k^2/{\tau }^2)-{\mathcal{T}}^{\prime }{k}^3/\tau +{\mathcal{T}}^{\prime }a{k}^2/\tau -{\mathcal{T}}_{0}k{a}_{}/{\tau }^2}{(a+1/\tau )(a/\tau -k/\tau )}.$$

(22)

From this expression, we see that the heat flow rate diverges at both previously determined stability boundaries $k^{*}=a$, or ${\tau }^{*}=-1/a$. This shows that, in the unstable regions, the transient dynamics is accompanied by a medium entropy production rate that is unboundedly growing with time. Analogous behavior at the stability boundaries is also found for $n>1$. Within the stable regions, we can further simplify Equation (22) and find

$${\dot{Q}}_0=k_{\mathrm{B}}\frac{k^2{\mathcal{T}}^{\prime }-(k/\tau ){\mathcal{T}}_0}{a_{}+1/\tau }.$$

(23)

From Equation (23), one immediately sees that for $n=1$ the heat flow between $X_0$ and its bath vanishes when the condition for the FDR (7) is fulfilled, consistent with the expectation that the system then reaches equilibrium.

Let us now discuss the general behavior for different n, i.e., different delay distributions. Figure 6 shows analytical results for the mean heat rate as functions of n, for three different values of the feedback gain k. We here mainly focus on the case $a>0$, such that we can consider both negative and positive k (in stable regions). Note that, in experiments with optical traps, usually the case $k>0$ is of more practical relevance. First, we notice that the net heat flow of $X_0$ is generally nonzero, even if the temperature of $X_0$ and all $X_{j>0}$ is the same (red symbols), showing that the systems considered here are out of equilibrium. An exception is the case $k=1/\tau$ and ${\mathcal{T}}_0={\mathcal{T}}^{\prime }$, where the medium entropy production rate vanishes. For this special case, the FDR is approximately fulfilled for $n=1$ (but violated for $n>1$ [57]). Further, we find that the magnitude of the induced heat flow is generally maximal for $n=1$, and decreases for larger n. Thus, the exponentially decaying delay generally yields the highest entropy flow between system and bath.

Another observation from Figure 6 is that for $k>0$, the heat flow from the colloid to the bath is negative (${\dot{Q}}_0<0$), corresponding to medium entropy reduction. This means that we have a steady state in which the delay force consistently extracts energy from the heat bath, i.e., feedback cooling. A Markovian external force could not have this impact on the stochastic system (as follows from the second law of thermodynamics). Importantly, this phenomenon also occurs when the “controller” $X_{j>0}$ has the same temperature as the colloid, i.e., isothermal conditions (Figure 6c), or even if the controller is hotter (then the heat flow is “reversed”) (Figure 6b). From Equation (23) follows for the case $n=1$ the following condition to find feedback cooling and medium entropy reduction

$$0<k\tau <{\mathcal{T}}_0/{\mathcal{T}}^{\prime }.$$

(24)

On the level of the Markovian “supersystem” (colloid plus controller), the negative heat flow can be explained by the involved non-reciprocal couplings [57]. Using the non-Markovian model alone, the explanation is more subtle. An extensive discussion of the origin of the heat flow, which is connected to non-trivial information flow from the colloidal system to the controller appearing in the generalized second law for the controlled system, is provided in [57]. As an example for a nonlinear system, we have further numerically explored the heat flow in a bistable potential $V\propto (x_0^4-4x_0)$; and again found regimes where a delay force induces a negative heat flow (for all values of n).

In Figure 7, we display the heat flow ${\dot{Q}}_0$ as a function of k, for positive and negative a, for the systems with $n=1$ and $n\to \infty$. We observe that feedback cooling is only possible for $a>0$. The reason is that only in this case, stable parameter regimes with $k>0$ exist, for all n (see Section 3). Further, we observe that the heat flow ${\dot{Q}}_0$ displays very similar behavior in the systems with $n=1$ and $n\to \infty$ for all considered values of k and $\tau$.

Concerning the connection to the stability analysis presented in the previous section, we note that the parameter combination considered in Figure 6c corresponds for all n almost to the stability boundary $k^{*}=a=1$ (see Figure 4 for a comparable case); i.e., slight increase of k would result in an unstable dynamics. In that case, the heat rate diverges due to the divergence of correlations entering Equation (21). However, when approaching this boundary from the left, the heat rate does not exhibit any noticeable difference compared to the parameter settings in the middle of the stability region. On the contrary, if the stability boundary at ${\tau }^{*}=-1/a_{}$ is approached, we notice very high values of the heat rate, see Figure 7b for an example.

Let us now consider the limit $n\to \infty$. Recall that in this limit the colored noise $\nu$ vanishes, while the delay distribution becomes a delta-distribution around $\tau$ [see Equation (9)]. We have intensely studied the heat flow in such a system in [43], for linear and nonlinear systems. The data in Figure 6 indeed seems to converge to the result from [43] (given in Appendix D) as n increases (as follows from the reasoning in Section 2.3). We expect the convergence to be more obvious in a regime of n larger than the regime that we can study on the bases of our analytical solutions. Preliminary numerical simulation results confirm this expectation, but we have not studied this question rigorously yet, and feel that it is beyond the scope of this manuscript. Remarkably, the approach is fastest for isothermal conditions ${\mathcal{T}}_0={\mathcal{T}}^{\prime }$. In fact, for this specific case, the heat flow behaves very similar to the the heat flow in the limit $n\to inf$ for all k values, see Figure 7c. To us, the reason for this fastest convergence for ${\mathcal{T}}_0={\mathcal{T}}^{\prime }$ is not clear and represents an interesting perspective of future research.

Finally, we consider the impact of the colored noise $\nu$, which in our model stems from the noise terms ${\xi }_j$ in Equation (5) and, thus, from a finite controller temperature, or from controller errors (depending on the interpretation). Figure 6 shows that the heat flow of $X_0$ generally increases with the magnitude of the colored noise, ${\mathcal{T}}^{\prime }$. Thus, this additional noise generally yields an additional positive contribution to the heat dissipated by the colloid into its heat bath (irrespective of the sign of the heat flow induced by the delay force). This trend persists (not shown here) when we employ the alternative interpretation of the noises associated to $X_{j>0}$ (see Section 2.4), i.e., when we set ${\mathcal{T}}^{\prime }=n\mathcal{T}″$.

Taken together, we conclude that, if the aim of the controller was to cool down the surrounding fluid (like a microscopic refrigerator), the feedback with exponential delay ($n=1$) and without controller errors (${\mathcal{T}}^{\prime }=0$) would be the most effective one.

5. The Total Entropy Production

Recent literature for (underdamped) systems with discrete delay [42,50,73] has pointed out strategies (and related difficulties) in calculating measures of irreversibility based on the non-Markovian dynamics of $X_0$ alone. Here, we rather focus on the total entropy production of the supersystem consisting of particle plus controller, assuming that all variables in Equation (5) are physical. In other words, we from now on interpret the $X_{j>0}$ as the internal degrees of freedom of the controller, which are non-reciprocally coupled among each other and generate a time-delayed feedback force on the particle $X_0$ (as illustrated in Figure 3). A main goal of the subsequent analysis is to explore how the total entropy production depends on n and thus, on the sharpness of the delay distribution.

To calculate the dissipated energy of the total supersystem (particle plus controller), we employ the standard formalism from stochastic thermodynamics [70,74]. The total EP along a fluctuating trajectory $\underline{\mathbf{X}}$$=\{X_0\left(t^{\prime }\right),..,X_n\left(t^{\prime }\right)\},$$t^{\prime }\in [t_s,t_f]$ is given by

$$\frac{\Delta s_{\mathrm{tot}}\left[\underline{\mathbf{X}}\right]}{k_{\mathrm{B}}}=ln\frac{\mathcal{P}\left[\underline{\mathbf{X}}\right]}{\widehat{\mathcal{P}}\left[\widehat{\underline{\mathbf{X}}}\right]}=\frac{\Delta s_{\mathrm{sh}}}{k_{\mathrm{B}}}+ln\frac{\mathcal{P}\left[\underline{\mathbf{X}}\right|{\underline{x}}_s]}{\widehat{\mathcal{P}}\left[\widehat{\underline{\mathbf{X}}}\right|{\underline{x}}_f]},$$

(25)

involving the multivariate joint Shannon entropy $s_{\mathrm{sh}}=-k_{\mathrm{B}}ln\left[{\rho }_{n+1}\left(\underline{x}\right)\right]$ of the $(n+1)$-point joint probability density function (pdf) ${\rho }_{n+1}$, and the path probabilities $\mathcal{P}$ and $\widehat{\mathcal{P}}$ for forward and backward process. $\mathcal{P}$ conditioned on the starting point $\underline{X}\left(t_s\right)={\underline{x}}_s$, is essentially the exponential of the Onsager–Machlup action, stated in Appendix C. We assume that $X_{j>0}$ are even under time-reversal, like positions, which is more suitable when $X_{j>0}$ model memory cells (odd parity would imply information storage in particle velocities). This assumption implies

$$ln\frac{\mathcal{P}\left[\underline{\mathbf{X}}\right|{\underline{x}}_s]}{\widehat{\mathcal{P}}\left[\widehat{\underline{\mathbf{X}}}\right|{\underline{x}}_f]}={\int }_{t_s}^{t_f}\frac{\delta q_0}{{\mathcal{T}}_0}+\sum _{j=1}^n{\int }_{t_s}^{t_f}\frac{n/\tau (X_{j-1}-X_j)\circ \mathrm{d}{X}_j}{{\mathcal{T}}^{\prime }},$$

(26)

where $\delta q_0$ can be identified as the heat flow from $X_0$ to its bath (20). In the steady state, $\langle \Delta s_{\mathrm{sh}}\rangle :=\Delta S_{\mathrm{sh}}\equiv 0$. Applying the analogous steps as between Equations (20) and (21), we find that the ensemble average of the EP (25) reads

$$\begin{array}{cc}\hfill {\dot{S}}_{\mathrm{tot}}=& \frac{{\dot{Q}}_0}{{\mathcal{T}}_0}+\sum _{j=1}^n\frac{{\dot{Q}}_j}{{\mathcal{T}}^{\prime }}\ge 0,\hfill \end{array}$$

(27)

$$\begin{array}{cc}\hfill {\dot{Q}}_j=& \frac{\langle X_{j-1}^2\rangle -\langle X_{j-1}{X}_j\rangle }{{(\tau /n)}^2}.\hfill \end{array}$$

(28)

with ${\dot{Q}}_0$ from (21). The nonnegativity of ${\dot{S}}_{\mathrm{tot}}$ follows by construction from Equation (25), see [70]. Equation (27) yields analytical expressions for the total EP for any n, see Appendix B. For $n=1$ (which was also discussed in [72]), this yields a particularly simple closed-form expression, which diverges in the unstable regions ($k\ge k^{*}$, or $\tau \ge {\tau }^{*}$), and in the stable region reads

$${\dot{S}}_{\mathrm{tot}}=\frac{k_{\mathrm{B}}}{{\mathcal{T}}_0{\mathcal{T}}^{\prime }}\frac{{(\tau {\mathcal{T}}_0-k{\mathcal{T}}^{\prime })}^2}{a_{}+1/\tau }.$$

(29)

The mean EP (27) has a contribution stemming from the dissipation of $X_0$ to its bath, ${\dot{Q}}_0$, and a contribution from the $X_{j>0}$, i.e., the dissipation of the additional d.o.f., ${\dot{Q}}_j$. This second contribution can be considered as the “entropic cost” of the memory [42,50,73] (where the “memory” arises in the form of colored noise and distributed delay). It should be emphasized, however, that memory is not unequivocally connected to EP. For example, the $n=1$ case with parameters such that FDR is fulfilled, also has memory in the process $X_0$, but no EP, see Equation (29). From Equation (29), one immediately sees that the EP vanishes if, and only if, the condition for the FDR (7) is fulfilled. In all other cases, the process (5) has entropic cost.

Note that the total EP is also positive for isothermal conditions ${\mathcal{T}}^{\prime }={\mathcal{T}}_0$ (red symbols) where no temperature gradients are present in the system. The EP is closely connected with the non-reciprocal couplings between the $X_j$, which are non-conservative interactions [57]. We further note that the total EP is positive even in the cases where we detected the reversed heat flow, showing that the second law of thermodynamics holds for the supersystem (“the colloidal system plus the controller”), as expected. In Figure 7, we further show for the case $n=1$, as a function of k, both heat flows (${\dot{Q}}_0$ and ${\dot{Q}}_1$) and the total entropy production, which grows like $\sim k^2$ (as also can be seen from Equation (29)).

Now we discuss the dependency on n. The left panels of Figure 8 show that, in sharp contrast to the ${\dot{Q}}_0$ saturating at large n, the total EP generally increases with n, specifically we observe a quadratic increase$\sim n^2$. This observation is robust against the details of the system, like a, k, $\tau$. In fact, it even holds for nonlinear systems, as exemplarily shown in Figure 9 for a bistable system. We further note that the $n^2$ dependence persists within in the aforementioned alternative interpretation (see Section 2.4), when we scale the heat baths of $X_{j>0}$ with n, i.e., ${\mathcal{T}}^{\prime }=n\mathcal{T}″$ (not shown here). The observed $n^2$ dependency implies a divergent EP. Taking the perspective that the $X_{j>0}$ model a memory device, where each memory cell contributes to EP, a divergent total EP is indeed expected due to the infinite system size. From an information-theoretical perspective, it is indeed also not surprising, as for $n\to \infty$ the controller stores a full stochastic trajectory, which has, due to the white noise, infinitely many jumps at each time interval, giving it an infinite information content.

The total EP also diverges if ${\mathcal{T}}^{\prime }$ goes to zero, corresponding to the cases of vanishing controller errors. Thus, an infinite precision has an infinite cost. This is in accordance with the result from [75], whose limit of “precise and infinitely fast” control corresponds in our time-continuous model to the limit ${\mathcal{T}}^{\prime }\to 0$. (Mathematically, the divergence of the entropy production rate as ${\mathcal{T}}^{\prime }\to 0$, can simply be seen as a consequence of the definition ${\dot{S}}_{\mathrm{tot}}\propto {\dot{Q}}_j/{\mathcal{T}}^{\prime }$, where one should note that the heat fluxes themselves do not vanish in this limit).

Lastly, let us briefly comment on the total EP fluctuations (discussed in Appendix E). The characteristics of the entropy distributions are found to be very similar for exponential vs. non-monotonic delay distributions (specifically, Figure A1 shows $n=1,2$). In both cases, they exhibit exponentially decaying tails and fulfilling an integral fluctuation theorem.

6. Conclusions

In this work, we have studied single-particle stochastic systems under the impact of a linear delay force. We have focused on the question how the distribution of the delay affects the stability and entropy production of the stochastic system.

To this end, we have considered a one-variable Langevin equation for $X_0$ with white and colored noise, conservative forces stemming from an external potential, and a linear delay force that involves a Gamma-distributed delay. Gamma-distributions turn out to be particularly suitable to study these questions, because they are versatile and moreover allow for a finite representation in the form of a set of $n+1$ Markovian first-order differential equation (involving n “auxiliary variables”). This set of equations resembles a unidirectionally coupled ring network of length $n+1$ [41]. We note that for a general functional form of the delay distribution, a finite-dimensional representation is not guaranteed. By exploiting the Markovian representation of the dynamics, we could find analytical results for ensemble-averaged quantities in the long-time limit, where the non-Markovian process reaches a steady state.

We have found that the distribution of the delay has a tremendous impact on the linear stability, even if mean delay time and weight of delay distribution remain unchanged. Interestingly, the regime of parameter where the system is stable is smallest in the case of a delta-distributed delay, while the exponentially decaying delay represents the type of delay with the largest stability region. Another remarkable observation is that for delay distributions with a peak of nonzero width around the mean delay time, we observe a reentrant behavior by varying the mean delay time. In particular, upon increasing the mean delay time, a system may become unstable, while an even further increase of delay time stabilizes the system again. This reentrant behavior is however not observed for delta-distributed or exponentially decaying delay. While in this paper, we have discussed the linear stability of fixed points of (nonlinear) time-delay processes, it would further be very interesting, as a next step, to systematically consider the stability regions of different dynamical regimes of nonlinear delay systems (e.g., for different n). For example, nonlinear delay systems may exhibit bifurcations where stable periodic orbits emerge, associated with rich dynamical phenomena such as coherence resonance and stochastic resonance [43,58]. In fact, our preliminary numerical studies show that the coherence resonance regime that we have described in [43] for $n\to \infty$ also exists for finite values of n.

Studying the heat flow of $X_0$ in the stable parameter regions, we found that for all types of delay distribution, the mean steady-state heat rate can be positive or negative. Negative heat flow implies that the delay force continuously cools down the heat bath by extracting energy from it and thereby reduces the medium entropy production. This points out the fundamentally different character of a delay force and a Markovian drive, which can never on average extract energy as stated by the second law. We have shown that the exponentially decaying delay yields the largest magnitude of heat flow, i.e., extracts the most energy from the heat bath.

Last, we have considered the total entropy production of all variables in the Markovian representation of the process. This entropy production is only meaningful if we interpret the “auxiliary variables” as actual physical processes. However, this consideration offered an interesting new perspective on the feedback cooling. First, this phenomenon can be traced back to the involved non-reciprocal couplings [57]. Second, it is not due to (hidden) temperature gradients. Specifically, one might have argued that a “colder feedback controller” is expected to extract energy from a system it is coupled to. However, here we have clearly shown that also an isothermal “controller” may extract energy. We found that the total entropy production increases quadratically with n, implying that the controller with $n=1$, which yields exponentially decaying delay, has the lowest entropic cost. Taken together, the exponentially decaying delay force is the one with the largest linear stability region, which may extract the most energy from the bath, and has, at the same time, the lowest entropic cost, making it the most “robust” and “efficient” one.

The special case of discrete delay with white noise emerges as the limit $n\to \infty$ in our approach. We find that the heat exchange of the system of interest, $X_0$, approaches a known result from the literature [43]. On the contrary, the total entropy production diverges, suggesting that the cost of storing an infinite amount of information is unbounded.

While we have here mostly looked at linear systems, our numerical investigations demonstrate that the main effects can also be found in nonlinear systems.

A particular interesting application of our results concerns the approximation of discrete delay processes by finite numbers of n. Indeed, if a non-Markovian process with a single delay time of type (Equation (A16)) is the original process, one may consider the n-dimensional Markovian process with small numbers of n as an approximation to the latter (which becomes exact as $n\to \infty$). One generally expects that, the smaller n, the “worse” this approximation. Our results confirm this expectation. However, they moreover show that the approximation can be improved upon setting ${\mathcal{T}}^{\prime }={\mathcal{T}}_0$. Furthermore, Figure 7c,d demonstrates that this approximation is actually already surprisingly good, even for $n=1$, in terms of heat flow (and, thus, average applied work by the feedback). This is somewhat surprising, given that heat and work are “sensitive” to dynamical aspects, as they depend on two-time correlation functions. Our results thus demonstrate a powerful way to approximate the discrete delay case by a very low-dimensional representation.

As a physical application for our theoretical considerations, we have here considered a colloidal particle under time-delayed feedback control, which is, in principle, experimentally realizable with optical feedback. More generally, our results carry over to other real-world systems, describable by delayed Langevin equations. Possible applications include the dynamics (and associated thermodynamics) of tracer particles in nonequilibrium viscoelastic environments [76], active environments [77], or granular media [78]. It will further be interesting to think about the physical implications of the negative heat flow reported here in other systems.

By generalizing the coupling matrix in the Markovian representation, one could use the approach presented here to further analyze more complicated delay distributions (e.g., multi-peaked ones, oscillatory delay), broadening the range of applications. We have undertaken first steps in this direction in [79], where we studied correlation functions and the nonequilibrium dynamics of systems similar to the one considered here, with up to two auxiliary variables. Further, several conclusions are indeed expected to carry over to other types of (non-Gamma) delay distributions. Indeed we have numerically observed that the $X_0$-dynamics essentially remains the same upon moderate variations of the delay distribution. An exception represent delay distributions that are not “short-ranged", like algebraically decaying delay distributions. These are, moreover known to yield anomalous diffusion, which might also result in interesting thermodynamic behavior and non-trivial stability bounds, representing a worthwhile perspective of future research.

Lastly, various interesting open questions concerning the thermodynamic analysis remain. For example, we have here only considered the EP of the Markovian supersystem, but we have not discussed irreversibility measures based on the non-Markovian dynamics of $X_0$ alone. Thermodynamic notions of systems with discrete delay were indeed recently also discussed by Rosinberg, Munakata and Tarjus [42,50,73]. They introduced such a framework of irreversibility measures based solely on the non-Markovian dynamics of $X_0$, focusing on the case of underdamped motion. In this context, one may for example investigate if, upon marginalization of the total entropy production calculated here, an effective irreversibility measure for the delay process, similar to [80], can be constructed.

Next, while we have so far mostly focused on the mean values, one could further explore the distributions. The mean value of the heat flow considered here is equivalent to the mean work performed on the system. For Markovian systems, the fluctuations of the work are known to fulfill useful fluctuation theorems such as the Jarzynski relation. It would be interesting to discuss generalization of such work fluctuations theorems to non-Markovian systems, taking the approach presented here as a possible starting point. For non-Makovian systems that fulfill FDR, such generalizations have indeed been carried out successfully in the past: see [81,82]. Similarly, this approach might also be useful to investigate thermodynamic uncertainty [83,84] relations in non-Markovian systems [85,86].