Bayesian Regression Quantifies Uncertainty of Binding Parameters from Isothermal Titration Calorimetry More Accurately Than Error Propagation

Abstract

We compare several different methods to quantify the uncertainty of binding parameters estimated from isothermal titration calorimetry data: the asymptotic standard error from maximum likelihood estimation, error propagation based on a first-order Taylor series expansion, and the Bayesian credible interval. When the methods are applied to simulated experiments and to measurements of Mg(II) binding to EDTA, the asymptotic standard error underestimates the uncertainty in the free energy and enthalpy of binding. Error propagation overestimates the uncertainty for both quantities, except in the simulations, where it underestimates the uncertainty of enthalpy for confidence intervals less than 70%. In both datasets, Bayesian credible intervals are much closer to observed confidence intervals.

1. Introduction

Isothermal titration calorimetry (ITC) is widely used to characterize binding processes involving biomolecules, including proteins [1], small organic molecules [2], DNA/RNA [3,4], and lipids [5]. ITC data are routinely analyzed to estimate thermodynamic parameters—the Gibbs free energy $\Delta G$ and the enthalpy $\Delta H$—for simple binding processes. Based on the relation $\Delta G=\Delta H-T\Delta S$ that includes the temperature T, the entropy $\Delta S$ may also be obtained. These parameters have often been estimated using what we will refer to as the standard procedure: a nonlinear least squares regression method implemented in the Origin software package that is distributed with the MicroCal VP-ITC instrument and its successors. The software yields a maximum likelihood estimate (MLE) of the parameters and asymptotic standard error (ASE). Unfortunately, the ASE underestimates the uncertainty by as much as an order of magnitude [6]!

The severe underestimation of uncertainty is mainly a consequence of ignoring the error in the titrant concentration. In the standard procedure, the error in the titrand concentration (in the sample cell) is handled by assigning the stoichiometry n as a free parameter. On the other hand, the titrant concentration (in the syringe) is treated as a constant. This assumption is made because ITC data can only be used to estimate the ratio of the titrant:titrand concentrations as opposed to individual values [7]. However, it is a poor assumption, because large errors (10–20%) in titrant concentration between laboratories have been observed [6].

In 2015, Boyce et al. suggested that the uncertainty of the standard procedure could be adjusted based on error propagation [8]. Specifically, based on the Taylor expansion, errors in $K_a$, $\Delta H$, and the site parameter n may be corrected by the relative error of the titrant concentration. However, they did not show that the resulting uncertainty estimate leads to accurate confidence intervals. Confidence intervals are accurate when the X% confidence interval includes the true value X% of the time, where X is a confidence level. In 2018, Nguyen et al. described the analysis of ITC data with Bayesian regression. They found that Bayesian credible intervals (BCIs)—regions that contain a specified percentage of the Bayesian posterior—are more accurate confidence intervals than those based on the ASE [9]. There was no comparison to confidence intervals based on the ASE augmented with error propagation. The purpose of this short manuscript is to address this oversight.

2. Results and Discussion

2.1. Bayesian Posteriors Are Converged

As in Nguyen et al. [9], the Markov chain Monte Carlo protocol leads to converged BCIs. In a representative run for one of the 1000 simulated integrated heat curves (Figure S1), fewer than 10% of samples are required before estimated percentiles of the posterior density are stable (Figure S2). Comparable convergence behavior is observed when sampling the Bayesian posterior for the ITC experiments (Figure S3).

2.2. Error Propagation Expands Confidence Intervals to Be Larger Than Bayesian Credible Intervals

For each of the 14 experiments, 95% CIs of $\Delta G$ and $\Delta H$ are shown in Figure 1. Because the true value of parameters is unknown, the median estimate is shown as a proxy. Panels (a) and (b) reproduce Figure 6 from Nguyen et al. [9]. The 95% CIs of $\Delta G$ encompass the median value in nearly every experiment. For CIs based on the ASE, 95% CIs of $\Delta H$ are too small. Panel (c) shows that error propagation increases CIs to encompass the median, but the CIs appear to be larger than necessary.

2.3. Even with Error Propagation, BCIs Provide More Accurate CIs Than the ASE

The accuracy of CIs was more carefully assessed by coverage plots, in which predicted confidence intervals are plotted against the percentage of BCIs and CIs that contain the true values of $\Delta G$ and $\Delta H$. For accurate CIs, points should lie along the diagonal. If points are below the diagonal, CIs are underestimated. If points are above the diagonal, CIs are overestimated.

Coverage plots were generated for 1000 simulations with high error (Figure 2) and low error (Figure S4) and for 14 experiments (Figure 3). In all of the coverage plots, Bayesian credible intervals are closest to the diagonal. As expected, the ASE consistently underestimates confidence intervals. For $\Delta G$, the ASE with error propagation overestimates confidence intervals for nearly all confidence levels. For $\Delta H$, the story is more subtle. In the simulations at both high and low error, confidence intervals are underestimated for confidence levels less than 70% but somewhat overestimated for higher confidence levels. In the experiments, confidence intervals are overestimated for confidence intervals greater than 30%.

3. Materials and Methods

3.1. Integrated Heat Data

Our data are integrated heat curves, $\mathcal{D}\in \left\{q_1,q_2,\dots ,q_N\right\}$, where $q_n$ is the integrated heat of injection n. We analyzed simulations as well as ITC experiments that were previously described [9].

Simulations are useful because it is inexpensive to collect large amounts of data and because thermodynamic parameters are known exactly. Simulations of simple 1:1 binding were performed in a similar way as in Nguyen et al. [9]. A total of 1000 integrated heat curves with 24 injections each were modeled based on the free energy of binding $\Delta G$, the enthalpy of binding $\Delta H$, the enthalpy of dilution and stirring per injection $\Delta H_0$, the concentration of receptor (titrand) ${\left[R\right]}_0$, the concentration of ligand (titrant) ${\left[L\right]}_s$, and the standard deviation of the measurement error $\sigma$. The thermodynamic parameters and enthalpy of injection were fixed at $\Delta G$ = −10 kcal/mol, $\Delta H$ = −5 kcal/mol, and $\Delta H_0$ = 0.5 $\mathsf{\mu }$cal; ${\left[R\right]}_0$ and ${\left[L\right]}_s$ were sampled from lognormal distributions with mean values of 0.1 and 1.0 mM, respectively. Based on the uncertainty of 10% observed by Myszka et al. [6], the variance was set at either small (5% of the mean) or large (10% of the mean). Measurement error was modeled as normally distributed with a zero mean and standard deviation of $\sigma$ = 1 $\mathsf{\mu }$cal.

We also analyzed 14 integrated heat curves from previously performed experiments in which MgCl${}_2$ was titrated into a sample cell containing EDTA in a MicroCal VP-ITC calorimeter [9].

3.2. Regression

Data were analyzed via Bayesian regression and maximum likelihood estimation. In both procedures, integrated heat curves for simple 1:1 binding were modeled as previously described [9]. They are functions of the aforementioned parameters,

$$\begin{array}{c}\hfill \mathit{\theta }\equiv (\Delta G,\Delta H,\Delta H_0,{\left[R\right]}_0,{\left[L\right]}_s,\sigma ).\end{array}$$

(1)

Observed injection heat $q_n$ was treated as normally distributed about the true heat $q_n^{*}\left(\theta \right)$,

$$\begin{array}{c}\hfill q_n\sim \mathcal{N}(q_n^{*}\left(\theta \right),{\sigma }^2).\end{array}$$

(2)

Thus, the likelihood function of an integrated heat curve $\mathcal{D}\in \left\{q_1,q_2,\dots ,q_N\right\}$ is

$$\begin{array}{c}\hfill p\left(\mathcal{D}\right|\mathit{\theta })=\frac{1}{{\left(2\pi \right)}^{N/2}{\sigma }^N}exp\left[-\frac{1}{2{\sigma }^2}\sum _{n=1}^N{(q_n-q_n^{*}\left(\theta \right))}^2\right].\end{array}$$

(3)

3.2.1. Bayesian Regression

For the Bayesian regression, the prior of parameters was independent, such that $p\left(\theta \right)={\prod }_{i}p\left({\theta }_i\right)$. As in Nguyen et al. [9], uniform priors were used for $\Delta G$, $\Delta H$, and $\Delta H_0$. Lognormal priors were used for the concentrations of the ligand and the receptor,

$$\begin{array}{c}\hfill ln{\left[\mathcal{X}\right]}_0\sim \mathcal{LN}({\left[\mathcal{X}\right]}_0,{\left(\delta {\left[\mathcal{X}\right]}_0\right)}^2),\end{array}$$

(4)

where ${\left[\mathcal{X}\right]}_0\in \{{\left[R\right]}_0,{\left[L\right]}_s\}$ is the stated value of each quantity; $\delta$ was assumed to be either 5% or 10%. The uninformative Jeffreys prior was used for $\sigma$ [10]:

$$\begin{array}{c}\hfill p\left(\sigma \right)\propto \frac{{\sigma }_0}{\sigma },\end{array}$$

(5)

where ${\sigma }_0=1\mathrm{cal}$. For this model, the posterior probability density is

$$\begin{array}{c}\hfill p\left(\mathit{\theta }\right|\mathcal{D})\propto p(\mathcal{D}\left|\mathit{\theta }\right)p\left(\mathit{\theta }\right),\end{array}$$

(6)

Sampling from the Bayesian posterior was performed using a Markov chain Monte Carlo method, as in Nguyen et al. [9], but with a few small adjustments. Instead of using pymc3, the regression was implemented in numpyro [11,12]. After 2000 warm-up moves, 10,000 (as opposed to 5000 [9]) samples from four chains were stored. The X% BCI of each parameter was calculated based on the smallest interval that contains X% of the posterior samples. Additionally, the uncertainty $\delta$ was set at either 5% and 10%, as opposed to only 10%.

3.2.2. Maximum Likelihood Estimation

For the MLE, parameter estimates were obtained as

$$\begin{array}{c}\hfill \widehat{\mathit{\theta }}=\arg \underset{\mathit{\theta }}{max}logp\left(\mathcal{D}\right|\mathit{\theta }).\end{array}$$

(7)

The covariance matrix of the asymptotic standard error (ASE) was estimated based on the inverse Fisher information matrix,

$$\begin{array}{c}\hfill cov\left(\widehat{\mathit{\theta }}\right)\approx -\frac{1}{N}{\left[\frac{\partial log{\mathcal{L}}_{\mathcal{N}}}{\partial \mathit{\theta }\partial {\mathit{\theta }}^{\top }}\left|{}_{\mathit{\theta }=\widehat{\mathit{\theta }}}\right.\right]}^{-1}.\end{array}$$

(8)

We used scipy.optimize.minimize function from the python package scipy [13] to implement this MLE model, estimate the parameters, and automatically calculate the covariance matrix for parameter uncertainty. The X% CI of each parameter was defined by an interval in which the lower bound was the 1 − X/2 percentile, and the upper bound was the 1 + X/2 percentile of the normal distribution with a mean as the estimated value and standard deviation as the ASE.

3.2.3. Maximum Likelihood Estimation with Error Propagation

We performed error propagation to augment the ASE of MLE parameters based on the formula provided by Boyce et al. [8],

$$\begin{array}{c}\hfill {\left(\frac{s_{\theta }}{\theta }\right)}^2={\left(\frac{s_{\theta ,ASE}}{\theta }\right)}^2+{\left(\frac{s_{{\left[L\right]}_s}}{{\left[L\right]}_S}\right)}^2\end{array}$$

(9)

In this equation, s are standard errors and $\theta \in \{\Delta G,\Delta H\}$ are the parameters affected by the uncertainty of ligand concentration ${\left[L\right]}_S$; $s_{\theta ,ASE}$ is the ASE, $s_{{\left[L\right]}_s}$ is the standard error in the ligand concentration, and $s_{\theta }$ is the error estimate of the parameter $\theta$ that incorporates ligand concentration error. The uncertainty of the ligand concentration $s_{{\left[L\right]}_s}$ can be estimated by another experiment or based on previous estimates. Considering uncertainty in both protein and ligand concentrations, we used either 5% or 10% for both $s_{{\left[R\right]}_0}/{\left[R\right]}_0$ and $s_{{\left[L\right]}_S}/{\left[L\right]}_S$. CIs of this procedure were estimated similarly to the MLE procedure.

4. Conclusions

In both ITC simulations and experiments, BCIs provide more accurate uncertainty estimates for thermodynamic binding parameters than the ASE, without or with error propagation. The ASE underestimates the uncertainties of all datasets. Error propagation overestimates the uncertainties in the experimental dataset, but in simulations it underestimates the uncertainty of enthalpy for confidence intervals less than 70%.