^{1}

^{*}

^{2}

This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).

We review a selection of methods for performing enhanced sampling in molecular dynamics simulations. We consider methods based on collective variable biasing and on tempering, and offer both historical and contemporary perspectives. In collective-variable biasing, we first discuss methods stemming from thermodynamic integration that use mean force biasing, including the adaptive biasing force algorithm and temperature acceleration. We then turn to methods that use bias potentials, including umbrella sampling and metadynamics. We next consider parallel tempering and replica-exchange methods. We conclude with a brief presentation of some combination methods.

The purpose of molecular dynamics (MD) is to compute the positions and velocities of a set of interacting atoms at the present time instant given these quantities one time increment in the past. Uniform sampling from the discrete trajectories one can generate using MD has long been seen as synonymous with sampling from a statistical-mechanical ensemble; this just expresses our collective wish that the ergodic hypothesis holds at finite times. Unfortunately, most MD trajectories are not ergodic and leave many relevant regions of configuration space unexplored. This stems from the separation of high-probability “metastable” regions by low-probability “transition” regions and the inherent difficulty of sampling a 3

This review concerns a selection of methods to use MD simulation to enhance the sampling of configuration space. A central concern with any enhanced sampling method is guaranteeing that the statistical weights of the samples generated are known and correct (or at least correctable) while simultaneously ensuring that as much of the relevant regions of configuration space are sampled. Because of the tight relationship between probability and free energy, many of these methods are known as “free-energy” methods. To be sure, there are a large number of excellent reviews of free-energy methods in the literature (e.g., [

We divide our review into three main sections. In the first, we discuss enhanced sampling approaches that rely on

For our purposes, the term “collective variable” or CV refers to any multidimensional function _{i}_{1}(_{2}(_{M}_{j}_{B}

Local minima in

Given the limitations of standard MD, how does one “discover” such states in a proposed CV space? A perfectly ergodic (infinitely long) MD trajectory would visit these minima much more frequently than it would the intervening spaces, allowing one to tally how often each point in CV space is visited; normalizing this histogram into a probability

In this section, we focus on methods of enhanced sampling of CVs based on MD simulations that are directly biased on those CVs; that is, we focus on methods in which an investigator must identify the CVs of interest as an input to the calculation. We have chosen to limit discussion to two broad classes of biasing: those whose objective is direct computation of the gradient of the free energy (

Naively, one way to have an MD system visit a hard-to-reach point

visit a select number of local points in that space, and at each one,

compute the mean force, then

use numerical integration to reconstruct

The discussion so far leaves open the correct way to compute the local free-energy gradients. A gradient is a local quantity, so a natural choice is to compute it from an MD simulation localized at a point in CV space by a constraint. Consider a long MD simulation with a holonomic constraint fixing the system at the point _{θ}_{(}_{x}_{)=}_{z}_{j}_{jk}_{j}_{j}

Although nowhere near exhaustive, below is a listing of common types of problems to which blue-moon sampling has been applied with some representative examples:

sampling conformations of small flexible molecules and peptides [

environmental effects on covalent bond formation/breaking (usually in combination with

solvation and non-covalent binding of small molecules in solvent [

protein dimerization [

The blue-moon approach requires multiple independent constrained MD simulations to cover the region of CV space in which one wants internal statistics. The care taken in choosing these quadrature points can often dictate the accuracy of the resulting free energy reconstruction. It is therefore sometimes advantageous to consider ways to avoid having to choose such points ahead of time, and adaptive methods attempt to address this problem. One example is the adaptive-biasing force (ABF) algorithm of Darve _{x}θ_{j}_{j}

The first of those two ideas is motivated by the fact that “forces” that keep normal MD simulations effectively confined to free energy minima are mean forces on the collective variables projected onto the atomic coordinates, and balancing those forces against their exact opposite should allow for thermal motion to take the system out of those minima. The second idea is a bit more subtle; after all, in a running MD simulation with no CV constraints, the constrained ensemble expression for the mean force (_{θ}_{θ}_{i}/∂x_{j}_{j}

ABF has found rather wide application in CV-based free energy calculations in recent years. Below is a representative sample of some types of problems subjected to ABF calculations in the recent literature:

Peptide backbone angle sampling [

Nucleoside [

Interactions of small molecules with polymers in water [

Molecule/ion transport through protein complexes [

Calculation of octanol-water partition coefficients [

Large-scale protein conformational changes [

Protein-nanotube [

Both blue-moon sampling and ABF are based on statistics in the constrained ensemble. However, estimation of mean forces need not only use this ensemble. One can instead relax the constraint and work with a “mollified” version of the free energy:
_{κ}_{B}T_{βκ→∞}_{κ}_{βκ→∞} _{κ}

Because the Gaussian has continuous gradients, it can be used directly in an MD simulation. Suppose we have a CV space ^{2} can be stably handled, and values of around 100 kcal/mol/Å^{2} are typically adequate.)

Temperature-accelerated MD (TAMD) [_{i}_{i}^{−1}:
_{j}_{j}^{−1} ≠ ^{−1}.

The advantage of TAMD is that if (1) ^{−1} acted on by the “potential” ^{−1}. That is, under TAMD, ^{−1}>^{−1}, which has the effect of attenuating the ruggedness of

Unlike TI, ABF, and the methods of umbrella sampling and metadynamics discussed in the next section, TAMD is not a method for direct calculation of the free energy. Rather, it is a way to overcome free energy barriers in a chosen CV space quickly without visiting irrelevant regions of CV space. (However, we discuss briefly a method in Section 4.2.2. in which TAMD gradients are used in a spirit similar to ABF to reconstruct a free energy.) That is, we consider TAMD a way to efficiently explore relevant regions CV space that are practically inaccessible to standard MD simulation. It is also worth pointing out that, unlike ABF, TAMD does not operate by opposing the natural gradients in free energy, but rather by using them to guide accelerated sampling. ABF can only use forces in locations in CV space the trajectory has visited, which means nothing opposes the trajectory going to regions of very high free energy. However, under TAMD, an acceleration of ^{−1} = 6 kcal/mol on the CVs will greatly accelerate transitions over barriers of 6-12 kcal/mol, but will still not (in theory) accelerate excursions to regions requiring climbs of hundreds of kcal/mol. TAMD and ABF have in common the ability to handle rather high-dimensional CVs.

Although it was presented theoretically in 2006 [

TAMD-enhanced flexible fitting of all-atom protein and RNA models into low-resolution electron microscopy density maps [

Large-scale (interdomain) protein conformational sampling [

Loop conformational sampling in proteins [

Mapping of diffusion pathways for small molecules in globular proteins [

Vacancy diffusion [

Conformational sampling and packing in dense polymer systems [

Finally, we mention briefly that TAMD can be used as a quick way to generate trajectories from which samples can be drawn for subsequent mean-force estimation for later reconstruction of a multidimensional free energy; this is the essence of the single-sweep method [_{2}O in myoglobin [

In the previous section, we considered methods that achieve enhanced sampling by using mean forces: in TI, these are integrated to reconstruct a free energy; in ABF, these are built on-the-fly to drive uniform CV sampling; and in TAMD, these are used on-the-fly to guide accelerated evolution of CVs. In this section, we consider methods that achieve enhanced sampling by means of controlled bias potentials. As a class, we refer to these as

Non-Boltzmann sampling is generally a way to derive statistics on a system whose energetics differ from the energetics used to perform the sampling. Imagine we have an MD system with bare interatomic potential _{b}^{−βΔV}〉 as a constant we can get from normalizing _{b}^{βΔV(z)}.

How does one choose Δ^{βΔV(z)} = ^{−βF(z)} = _{b}

We usually do not know the free energy ahead of time; if we did, we would already know the statistics of CV space and no enhanced sampling would be necessary. Moreover, perfectly uniform sampling of the entire CV space is usually far from necessary, since most CV spaces have many irrelevant regions that should be ignored. And in reference to the mean-force methods of the last section, uniform sampling is likely not necessary to achieve accurate mean force values; how good an estimate of ∇_{0} should not depend on how well we sampled at some other point _{1}. Yet achieving uniform sampling is an idealization since, if we do, this means we know the free energy. We now consider two other biasing methods that aim for this ideal, either in relatively small regions of CV space using fixed biases, or over broader extents using adaptive biases.

Umbrella sampling is the standard way of using non-Boltzmann sampling to overcome free energy barriers. In its debut [

Coming up with the umbrella potential that would enable exploration of CV space with a single umbrella sampling simulation that takes the system far from its initial point is not straightforward. Akin to TI, it is therefore advantageous to combine results from several independent trajectories, each with its own umbrella potential that localizes it to a small volume of CV space that overlaps with nearby volumes. The most popular way to combine the statistics of such a set of independent umbrella sampling runs is the weighted-histogram analysis method (WHAM) [

To compute statistics of CV space using WHAM, one first chooses the points in CV space that define the little local neighborhoods, or “windows” to be sampled and chooses the bias potential used to localize the sampling. Not knowing how the free energy changes in CV space makes the first task somewhat challenging, since more densely packed windows are preferred in regions where the free energy changes rapidly; however, since the calculations are independent, more can be added later if needed. A convenient choice for the bias potential is a simple harmonic spring that tethers the trajectory to a reference point _{i}_{i}_{i}

Each window-restrained trajectory is directly histogrammed to yield apparent (_{b,i}_{i}_{i}_{b,i}_{i}

WHAM solves this problem by renormalizing the probabilities in each window into a single composite probability. Where there is overlap among windows, WHAM renormalizes such that the statistical variance of the probability is minimal. That is, it treats the factor
_{i}

The literature on umbrella sampling is vast (by simulation standards), so we present here a very condensed listing of some of its more recent application areas with representative citations:

Small molecule conformational sampling [

Protein-folding [

Protein-protein/peptide-peptide interactions [

DNA conformational changes [

Binding and association free-energies [

Adsorption on and permeation through lipid bilayers [

Adsorption onto inorganic surfaces/interfaces [

Water ionization [

Phase transitions [

Enzymatic mechanisms [

Molecule/ion transport through protein complexes [

As already mentioned, one of the difficulties of the umbrella sampling method is the choice and construction of the bias potential. As we already saw with the relationship among TI, ABF, and TAMD, an adaptive method for building a bias potential in a running MD simulation may be advantageous. Metadynamics [

Metadynamics is rooted in the original idea of “local elevation” [

In metadynamics, configurational variables _{0} is the bare interatomic potential and Δ_{G}

The difference between the metadynamics estimate of the free energy and the true free energy can be shown to be related to the diffusion coefficient of the collective variables and to the rate at which the bias is grown. A possible way to decrease this error as a simulation progresses is to decrease the growth rate of the bias. Well-tempered metadynamics [_{0} is the initial “deposition rate”, measured Gaussian height per unit time, and Δ

In the well-tempered ensemble, the parameter Δ

Metadynamics is a powerful method whose popularity continues to grow. In either its original formulation or in more recent variants, metadynamics has been employed successfully in several fields, some of which we point out below with some representative examples:

Chemical reactions [

Peptide backbone angle sampling [

Protein folding [

Protein aggregation [

Molecular docking [

Conformational rearrangement of proteins [

Crystal structure prediction [

Nucleation and crystal growth [

and proton diffusion [

Given a potential

Metastable states and transition states must be unambiguously identified as

The CV space must not contain hidden barriers.

The first of these may seem obvious: CVs are chosen to provide a low-dimensional description of some important process, say a conformational change or a chemical reaction or a binding event, and one can not describe a process without being able to discriminate states. However, it is not always easy to find CVs that do this. Even given representative configurations of two distinct metastable states, standard MD from these two different initial configurations may sample partially overlapping regions of CV space, making ambiguous the assignation of an arbitrary configuration to a state. It may be in this case that the two representative configurations actually belong to the same state, or that if there are two states, that no matter what CV space is overlaid, the barrier separating them is so small that, on MD timescales, they can be considered rapidly exchanging substates of some larger state.

However, a third possibility exists: the two MD simulations mentioned above may in fact represent very different states. The overlap might just be an artifact of neglecting to include one or more CVs that are truly necessary to distinguish those states. If there is a significant free energy barrier along this neglected variable, an MD simulation will not cross it, yet may still sample regions in CV space also sampled by an MD simulation launched from the other side of this hidden barrier. And it is even worse: if TI or umbrella sampling is used along a pathway in CV space that neglects an important variable, the free-energy barriers along that pathway might be totally meaningless.

Hidden barriers can be a significant problem in CV-based free-energy calculations. Generally speaking, one only learns of a hidden barrier after postulating its existence and testing it with a new calculation. Detecting them is not straightforward and often involves a good deal of CV space exploration. Methods such as TAMD and well-tempered metadynamics offer this capability, but much more work could be done in the automated detection of hidden barriers and the “right” CVs (e.g., [

An obvious way of reducing the likelihood of hidden barriers is to use increase the dimensionality of CV space. TAMD is well-suited to this because it is a gradient method, but standard metadynamics, because it is a histogram-filling method, is not. A recent variant of metadynamics termed “reconnaissance metadynamics” [

There are very few “best practices” codified for choosing CVs for any given system. Most CVs are developed ad hoc based on the processes that investigators would like to study, for instance, center-of-mass distance between two molecules for studying binding/unbinding, or torsion angles for studying conformational changes, or number of contacts for studying order-disorder transitions. Cartesian coordinates of centers of mass of groups of atoms are also often used as CVs, as they are functions of these coordinates.

The potential energy

Coarse-graining of particle positions onto Eulerian fields was used early on in enhanced sampling [

We finally mention the possibility of building collective variables based on set of frames which might be available from experimental data or generated by means of previous MD simulations. Some of these variables are based on the idea of computing the distances between the present configuration and a set of precomputed snapshots. These distances, here indicated with _{i}

A generalization to multidimensional paths (_{i}

“Tempering” refers to a class of methods based on increasing the temperature of an MD system to overcome barriers. Tempering relies on the fact that according to the Arrhenius law the rate at which activated (barrier-crossing) events happen is strongly dependent on the temperature. Thus, an annealing procedure where the system is first heated and then cooled allows one to produce quickly samples which are largely uncorrelated. The root of all these ideas indeed lies in the simulated annealing procedure [

Simulated annealing is a form of Markov-chain Monte Carlo sampling where the temperature is artificially modified during the simulation. In particular, sampling is initially done at a temperature high enough that the simulation can easily overcome high free-energy barriers. Then, the temperature is decreased as the simulation proceeds, thus smoothly bringing the simulation to a local energy minimum. In simulated annealing, a critical parameter is the cooling speed. Indeed, the probability to reach the global minimum grows as this speed is decreased.

The search for the global minimum can be interpreted in the same way as sampling an energy landscape at zero temperature. One could thus imagine to use simulated annealing to generate conformations at, e.g., room temperature by slowly cooling conformations starting at high temperature. However, the resulting ensemble will strongly depend on the cooling speed, thus possibly providing a biased result. A better approach consists of the the so-called simulated tempering methods [_{i}_{i}

Estimating these weights _{i}

A smart way to alleviate the issue of finding the correct weights is that of simulating several replicas at the same time [

The calculation of the acceptance is very cheap as it is based on the potential energy which is often computed alongside force evaluation. Thus, one could in theory exploit also a large number of virtual, rejected exchanges so as to enhance statistical sampling [

Parallel tempering is popular in simulations of protein conformational sampling [

The difference between the replicas is not restricted to be a change in temperature. Any control parameter can be changed, and even the expression of the Hamiltonian can be modified [

Several recipes for choosing the modified Hamiltonian have been proposed in the literature [^{−1}. Hamiltonian scaling however can take advantage of the fact that the total energy of the system is an extensive property. Thus, one can limit the scaling to the portion of the system which is considered to be interesting and which has the relevant bottlenecks. With solute tempering, the solute energy is scaled whereas the solvent energy is left unchanged. This is equivalent to keeping the solute at a high effective temperature and the solvent at the physical temperature. Since in the simulation of solvated molecules most of the atoms belong to the solvent, this turns in a much smaller modification to the explored ensemble when compared with parallel tempering. In spite of this, the effect on the solute resemble much that of increasing the physical temperature.

A sometimes-overlooked subtlety in solute tempering is the choice for the treatment of solvent-solute interactions. Indeed, whereas solute-solute interactions are scaled with a factor

In general, the advantage of these tempering methods over straighforward sampling can be rationalized as follows. A simulation is evolved so as to sample a modified ensemble by e.g., raising temperature or artificially modifying the Hamiltonian. The change in the ensemble could be drastic, so that trying to extract canonical averages by reweighting from such a simulation would be pointless. For this reason, a ladder of intermediate ensembles is built, interpolating between the physical one (

Tempering methods are thus relying on the ergodicity of the most modified ensemble. This assumption is not always correct. A very simple example is parallel tempering used to accelerate the sampling over an entropic barrier. Since the height of an entropic barrier grows with the temperature, in this conditions the barrier in the most modified ensembles are unaffected [

The number of intermediate steps in the ladder can be shown to grow with the square root of the specific heat of the system in the case of parallel tempering simulations. No general relationship can be drawn in the case of Hamiltonian replica exchange, but one can expect approximately that the number of replicas should be proportional to the square root of the number of degrees of freedom affected by the modification of the Hamiltonian. Thus, Hamiltonian replica exchange methods could be much more effective than simple parallel tempering as they allow the effort to be focused and the number of replicas to be minimized.

Parallel tempering has the advantage that all the replicas can be analyzed to obtain meaningful results, e.g., to predict the melting curve of a molecule. This procedure should be used with caution, especially with empirically parametrized potentials, which are often tuned to be realistic only at room temperature. On the other hand, Hamiltonian replica exchange often relies on unphysically modified ensembles which have no interest but for the fact that they increase ergodicity.

As a final note, we observe that data obtained at different temperature (or with modified Hamiltonians) could be combined to enhance statistics at the physical temperature [

The algorithms presented in Section 3 and based on tempering are typically considered to be simpler to apply when compared with those discussed in Section 2 and based on biasing the sampling of selected collective variables. Indeed, by avoiding the problem of choosing collective variables which properly describe the reaction path, most of the burden of setting up a simulation is removed. However, this comes at a price: considering the computational cost, tempering methods are extremely expensive. This cost is related to the fact that they are able to accelerate all degrees of freedom to the same extent, without an

Because of the conceptual difference between tempering methods and CV-based methods, these approaches can be easily and efficiently combined. As an example, the combination of metadynamics and parallel tempering can be used to take advantage of the known bottlenecks with biased collective variables at the same time accelerating the overall sampling with parallel tempering [_{B}T

It is also possible to combine metadynamics with the solute tempering method so as to decrease the number of required replicas and the computational cost [

The string method is generally an approach to find pathways of minimal energy connecting two points in phase space [_{s}_{s}_{s}_{jk}

In contrast, in the on-the-fly variant of string method in CVs, the friction on the _{s}_{s}

Because TAMD provides mean-force estimates as it is exploring CV space, it stands to reason that those mean forces could be used to compute a free energy. In contrast, in the single-sweep method [_{1}, _{2}, …, _{M}

Chen, Cuendet, and Tuckermann developed a very similar approach that in addition to parameterizing a free energy using d-AFED-computed gradients uses a metadynamics-like bias on the potential [

In this review, we have summarized some of the current and emerging enhanced sampling methods that sit atop MD simulation. These have been broadly classified as methods that use collective variable biasing and methods that use tempering. CV biasing is a much more prevalent approach than tempering, due partially to the fact that it is perceived to be cheaper, since tempering simulations are really only useful for enhanced sampling of configuration space when run in parallel. CV-biasing also reflects the desire to rein in the complexity of all-atom simulations by projecting configurations into a much lower dimensional space. (Parallel tempering can be thought of as increasing the dimensionality of the system by a factor equal to the number of simulated replicas.) But the drawback of all CV-biasing approaches is the risk that the chosen CV space does not provide the most faithful representation of the true spectrum of metastable subensembles and the barriers that separate them. Guaranteeing that sampling of CV space is not stymied by hidden barriers must be of paramount concern in the continued evolution of such methods. For this reason, methods that specifically allow broad exploration of CV space, like TAMD (which can handle large numbers of CVs) and well-tempered metadynamics will continue to be valuable. So too will parallel tempering because its broad sampling of configuration space can be used to inform the choice of better CVs. Accelerating development of combined CV-tempering methods bodes well for enhanced sampling generally.

Although some of these methods involve time-varying forces (ABF, TAMD, and metadynamics), all methods we’ve discussed have the underlying rationale of the equilibrium ensemble. TI uses the constrained ensemble, ABF and metadynamics ideally converge to an ensemble in which a bias erases free-energy variations, and TAMD samples an attenuated/mollified equilibrium ensemble. There is an entirely separate class of methods that inherently rely on

Finally, we have also not really touched on any of the practical issues of implementing and using these methods in conjunction with modern MD packages (e.g., NAMD [

CFA would like to acknowledge support of NSF (DMR-1207389) and NIH (1R01GM100472). GB would like to acknowledge the European Research Council (Starting Grant S-RNA-S, no. 306662) for financial support. Both authors would like to acknowledge NSF support of a recent Pan-American Advanced Studies Institute Workshop “Molecular-based Multiscale Modeling and Simulation” (OISE-1124480; PI: W. J. Pfaednter, U. Washington) held in Montevideo, Uruguay, 12–15 September 2012, where the authors met and began discussions that influenced the content of this review.

The authors declare no conflict of interest.

_{2}from classical and ab initio metadynamics