^{1}

^{*}

^{2}

^{*}

This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).

Malliavin weight sampling (MWS) is a stochastic calculus technique for computing the derivatives of averaged system properties with respect to parameters in stochastic simulations, without perturbing the system’s dynamics. It applies to systems in or out of equilibrium, in steady state or time-dependent situations, and has applications in the calculation of response coefficients, parameter sensitivities and Jacobian matrices for gradient-based parameter optimisation algorithms. The implementation of MWS has been described in the specific contexts of kinetic Monte Carlo and Brownian dynamics simulation algorithms. Here, we present a general theoretical framework for deriving the appropriate MWS update rule for any stochastic simulation algorithm. We also provide pedagogical information on its practical implementation.

Malliavin weight sampling (MWS) is a method for computing derivatives of averaged system properties with respect to parameters in stochastic simulations [

MWS works by introducing an auxiliary stochastic quantity, the Malliavin weight, for each parameter of interest. The Malliavin weights are updated alongside the system’s usual (unperturbed) dynamics, according to a set of rules. The derivative of any system function,

MWS has so far been discussed only in the context of specific simulation algorithms. In this paper, we present a pedagogical and generic approach to the construction of Malliavin weights, which can be applied to any stochastic simulation scheme. We further describe its practical implementation in some detail using as our example one dimensional Brownian motion in a force field.

The rules for the propagation of Malliavin weights have been derived for the kinetic Monte-Carlo algorithm [

We suppose that our system evolves in some state space, and a point in this state space is denoted as _{S′}_{eq}(_{eq}(_{eq}(_{1}_{n}

Let us now consider the average of some quantity

Let us now suppose that we track in our simulation not only the physical state of the system, but also an auxiliary stochastic variable, which we term _{λ}_{λ}_{λ}_{λ}_{λ}_{S}_{λ}

Our aim is to define a set of rules for updating _{λ}_{λ}_{S}_{λ}_{λ}

How do we go about finding the correct updating rule? If the Malliavin weight exists, we should be able to derive its updating rule from the system’s underlying stochastic equations of motion. We obtain an important clue from differentiating _{λ}

From a practical point of view, for each time step, we implement the following procedure:

propagate the system from its current state

with knowledge of _{λ}

update the Malliavin weight according to _{λ}_{λ}_{λ}_{λ}

At the start of the simulation, the Malliavin weight is usually initialised to _{λ}

Let us first suppose that our system is not in steady state. However rather the quantity 〈_{λ}_{λ}_{i}_{λ,i}

If, instead, our system is in steady state, the procedure needs to be modified slightly. This is because the variance in the values of _{λ}_{λ}_{λ}

Returning to a more theoretical perspective, it is interesting to note that the rule for updating the Malliavin weight, _{λ}_{λ}_{λ}_{S}_{λ}_{S}_{S}_{S}_{S}_{λ}_{S}_{λ}_{S}_{λ}_{S}

Up to now, we have assumed that the quantity _{λ}

We can also extend our analysis further to allow us to compute higher derivatives with respect to the parameters. These may be useful, for example, for increasing the efficiency of gradient-based parameter optimisation algorithms. Taking the derivative of _{λμ}_{λμ}_{λ}q_{μ}

Steady state problems can be approached by extending the correlation function method to second order weights. Define,

As in the first order case, in steady state we expect ^{2}〈

We now demonstrate this machinery by way of a practical but very simple example, namely one-dimensional (overdamped) Brownian motion in a force field. In this case, the state space is specified simply by the particle position _{λ}_{λ}_{λ}_{λ}_{λ}_{λ}

If we differentiate _{λμ}_{λμ}_{λμ}_{λ}_{μ}_{λμ}_{λ}q_{μ}_{λ}_{μ}

Now let us consider the simplest case of a particle in a linear force field _{0} at _{0} = 1. The weighted averages in ^{5} samples as in

As discussed briefly above, in this procedure the sampling error in the computation of _{h}_{κ}_{h}_{λ}^{2}〉 = 1_{hκ}_{hκ}_{h}q_{κ}^{2}〉 ∼ ^{2}, although the transient behaviour is complicated. Full expressions are given in Section 5. This suggests that computation of second order derivatives is likely to suffer more severely from statistical sampling problems than the computation of first order derivatives.

In this paper, we have provided an outline of the generic use of Malliavin weights for sampling derivatives in stochastic simulations, with an emphasis on practical aspects. The usefulness of MWS for a particular simulation scheme hinges on the simplicity or otherwise of constructing the propagator

Rosalind J. Allen is supported by a Royal Society University Research Fellowship.

The authors declare no conflict of interest.

The MATLAB script in

MATLAB script used to generate

Here is a brief explanation of the script.

Here, we present analytic results for the growth in time of the mean square Malliavin weights. We can express the rate of growth of the mean of a generic function _{h}, q_{κ}, q_{hκ}_{hκ}^{2}〉 = 2 ^{3}〉 = 0, 〈^{4}〉 = 12 ^{2},

Time-dependent derivatives, ^{2}〈

Growth of mean square Malliavin weights with time. The solid lines are from simulations and the dashed lines are from