Curve Registration of Functional Data for Approximate Bayesian Computation

Abstract

Approximate Bayesian computation is a likelihood-free inference method which relies on comparing model realisations to observed data with informative distance measures. We obtain functional data that are not only subject to noise along their y axis but also to a random warping along their x axis, which we refer to as the time axis. Conventional distances on functions, such as the L₂ distance, are not informative under these conditions. The Fisher–Rao metric, previously generalised from the space of probability distributions to the space of functions, is an ideal objective function for aligning one function to another by warping the time axis. We assess the usefulness of alignment with the Fisher–Rao metric for approximate Bayesian computation with four examples: two simulation examples, an example about passenger flow at an international airport, and an example of hydrological flow modelling. We find that the Fisher–Rao metric works well as the objective function to minimise for alignment; however, once the functions are aligned, it is not necessarily the most informative distance for inference. This means that likelihood-free inference may require two distances: one for alignment and one for parameter inference.

1. Introduction

In statistical problems, what constitutes the sample space is sometimes a matter of perspective. One perspective is that the data comprise a set of functions rather than simply numbers [1]. For instance, using ultraviolet-visible spectroscopy, suppose we record the absorbance for a series of samples over a wide range of frequencies. We might regard the sample number and frequency as explanatory variables and absorbance as the response variable; alternatively, for each sample, we could consider the entire functional form of the absorption spectrum as the response variable, in which case the only explanatory variable is the sample number. The sample space could comprise the output range of the machine (a subset of the real line) or the function space of the set of all possible spectra. Functional data analysis (FDA) proceeds with this latter approach [2,3]; these data are termed functional data.

A familiar problem when analysing functional data is misalignment [4,5,6], where there is difficultly comparing observations or deriving meaningful statistics, since the features of the functions do not correspond with one another on the domain. We refer to the domain, by convention, as the time axis, since we consider only functions mapping a bounded subset of $\mathbb{R}$ to $\mathbb{R}$. A simple example of issues with misalignment is the comparison of handwriting samples—if the samples are not on top of each other, then it is difficult to compare them [7].

In cases with misaligned data, it is often helpful to align the functions before proceeding with standard statistical analysis. Standard statistical methods assume that only the amplitude of a function is subject to noise. However, it is possible that the time axis itself is subject to noise in a way that preserves the ordering of observations (warping). The fact that the time axis is warped creates misalignment between the functions. This situation can be elegantly described as a combination of amplitude and phase variation within a functional dataset [7,8].

Dynamic time warping [9,10] and curve registration [11,12] are equivalent terms used in the engineering and statistical literature (respectively), for aligning functions by warping the time axis. Applications of curve registration include alignment of growth curves to improve the usefulness of statistical summaries [13] and alignment of medical equipment signals to optimise surgical workflow [14]. In this paper, we use curve registration to align the data.

In curve registration, an elastic distance between functions displaying both amplitude and phase variation is used. The elastic distance we adopt has mathematically convenient properties [11]. This elastic distance generalises the Fisher–Rao metric [15,16] from the space of probability density functions to the space of absolutely continuous functions. This is referred to as the generalised Fisher–Rao metric (FR).

Parameter inference with functional data is developed under many different names, including self-modelling [17], functional-modelling [3] and longitudinal data analysis [18]. However, these methods require that the likelihood function can be evaluated. In many statistical models, the likelihood function is either unavailable or intractable, but it is possible to draw realisations from the model conditional on parameter values. One approach to parameter inference in these situations is approximate Bayesian computation (ABC). ABC is a likelihood-free approach to parameter inference where the proposed parameter values are accepted when a distance between model realisations and observed data is smaller than some threshold [19].

In ABC, the distance metric employed is usually defined as the Euclidean distance between summary statistics of the observed data and the model realisation. The summary statistics can be obtained in a variety of ways. For instance, one can compute summary statistics based on increments at each step [20], one could use wavelet compression to transform functional data into low-dimensional summary statistics [21] or they could be transformed into estimators for parameters from an autoregressive model [22]. Choosing the most effective summary statistics is problem specific, and might also be dataset-specific [23]. In recent work, summary statistics have been replaced by estimators for metrics on probability measures, such as the Wasserstein distance [24], maximum mean discrepancy (MMD) [25], and the Fisher–Rao metric [26]. These estimators have been used within ABC samplers [24,27]. We compare the FR and the MMD for the purposes of ABC, however, in all cases, the curve registration is itself performed using the FR.

In this paper, we consider problems where the likelihood function is unavailable and the shape and characteristics of the response variables’ functional form encode important information for parameter inference, considering that this functional form is subject to variation in peak locations, which can be modelled as a stochastic warping of the time axis. We incorporate curve registration into the distance computation of ABC (Figure 1).

In the following Section, we provide some background to functional data analysis and the use of curve registration as well as describing the Fisher–Rao metric and introducing notation. In Section 3, we take a closer look at approximate Bayesian computation and introduce a distance sampler as well as relevant notation. Our approach combining curve registration and ABC is demonstrated with four examples; two simulation examples with random distributions that shift position of peaks, where we test whether ABC with alignment outperforms ABC without alignment (Section 4); a model of passenger flow in an international airport, where we use an algorithm for queueing-based simulation using data consisting of records of passenger numbers passing through specific checkpoints for each minute of the day (Section 5); and a hydrological model, where water flows within catchment areas in response to for example rainfall are forecasted using the GR4J simulation model (Section 6). In Section 7 we summarize our results and conclude with our findings.

2. Functional Data Analysis and Curve Registration

Functional data analysis (FDA), in which datapoints are functions, is a well-established field of study [3]. Special cases of functional data include longitudinal data and time series; however, the field of FDA is much broader [28]. For simplicity, we consider problems in which the functional data comprise a single functional random variable $\mathsf{f}$. These variables $\mathsf{f}$ take values in a function space $\mathcal{F}$ with probability measure p [29]. In practice, $\mathsf{f}$ is not directly observed; instead, we observe an empirical function f, with error, on a finite subset of the domain. This is referred to as an empirical functional random variable (EFRV). An EFRV f is a set $f_1,f_2,\cdots ,f_m$ of pairs $f_i=(t_i^f,y_i^f)$, representing the sampling location $t_i^f$ and corresponding function output $y_i^f$. We append the superscript f to associate each EFRV with its sampling points and functional outputs. The set of all sampling locations for f is $t^f$, similarly $y^f$ is the set of all functional outputs.

When analysing functional data, estimating the mean function $z\left(t\right)=\underset{\mathsf{f}\sim p}{\mathsf{E}}\left[\mathsf{f}\right]\left(t\right)$ is often of interest. In the case of multiple EFRVs $f^1,\dots ,f^n$, which share the same sampling locations $t^{f^1}=\cdots =t^{f^n}$, a standard estimator is $\widehat{z}\left(t_i\right)=\frac{1}{n}{\sum }_{j=1}^n{y}^{f_i^j}$, with $i=1,\dots ,m$. In many cases, however, it is possible that the resulting function does not lie within $\mathcal{F}$ due to misalignment caused by time warping (phase variation). For instance, if $\mathcal{F}$ is the set of Gaussian functions $\varphi \left(t\right|\mu ,\sigma )$ with $\mu \sim U(0,5)$ and $\sigma =1$, then the mean function $z\left(t\right)=\frac{1}{5}{\int }_0^5\varphi \left(t\right|\mu ,\sigma )\mathrm{d}\mu$ is not a Gaussian function on the time axis.

The functional data can be aligned using curve registration. We assume that $\mathcal{F}$ is the space of real-valued functions defined on the interval $[0,T]$, for some $T>0$. Curve registration aligns elements of $\mathcal{F}$ with increasing and invertible automorphisms on the time axis. These automorphisms are called warping functions $\gamma :[0,T]\mapsto [0,T]$, such that elements of the set $\mathcal{G}:=\{\mathsf{f}\circ \gamma |\mathsf{f}\in \mathcal{F}\}$ have aligned features according to some criteria. We adopt the elastic functions approach of Srivastava et al. (2011) [11], who align $\mathsf{f}$ to another functional random variable $\mathsf{g}$ using the generalised Fisher–Rao (FR) metric:

$$\begin{array}{c}\hfill d_{\mathrm{FR}}(\mathsf{f},\mathsf{g})=\int {[q^{\mathsf{f}}\left(t\right)-q^{\mathsf{g}}\left(t\right)]}^2\mathrm{d}t,\end{array}$$

(1)

as the objective function, where $q^{\mathsf{f}}\left(t\right)=\mathrm{sign}\left({\mathsf{f}}^{\prime }\left(t\right)\right)\times \sqrt{|{\mathsf{f}}^{\prime }\left(t\right)|}$. Curve $\mathsf{f}$ is aligned to $\mathsf{g}$ by finding the value of $\gamma$ which minimises $d_{\mathrm{FR}}(\mathsf{f}\circ \gamma ,\mathsf{g})$. Since this distance takes phase variation into account, it is called an elastic distance. One advantage of using $d_{\mathrm{FR}}$ for curve registration is that:

$$\begin{array}{c}\hfill d_{\mathrm{FR}}(\mathsf{f},\mathsf{g})=d_{\mathrm{FR}}(\mathsf{f}\circ \gamma ,\mathsf{g}\circ \gamma );\end{array}$$

in other words, the FR is invariant with respect to shared warpings. This implies that the distance of amplitudes between $\mathsf{f}$ and $\mathsf{g}$, defined as

$$\begin{array}{c}\hfill d_{\mathrm{amp}}(\mathsf{f},\mathsf{g}):=\underset{\gamma \in \Gamma }{inf}{d}_{\mathrm{FR}}(\mathsf{f}\circ \gamma ,\mathsf{g}),\end{array}$$

is symmetric [11]. The R package fdasrvf [30], which we use, implements this approach. Since $\mathsf{f}$ is not directly observed (rather, the EFRV f is observed), a cubic spline approximation to ${\mathsf{f}}^{\prime }\left(t\right)$ is used to compute the FR distance.

3. Approximate Bayesian Computation

Suppose it is possible to draw samples x from a probability distribution $p(·|\theta )$, but it is not possible to directly evaluate $p\left(x\right|\theta )$ for a particular x. Furthermore, suppose we wish to perform inference on $\theta$. In such cases, so-called likelihood-free approaches are necessary; a popular approach is approximate Bayesian computation (ABC).

The theoretical foundation of ABC rests on the fact that, if we draw samples $\theta$ independently of the prior $\pi \left(\theta \right)$ and use each $\theta$ to generate a corresponding model realisation $x_{\theta }$ from $p(·|\theta )$, then the set $\left\{\theta \right|x_{\theta }=x\}$ is distributed as $\pi \left(\theta \right|x)$. Such an approach would never work in practice, as the proportion of samples $x_{\theta }$ equal to x for models with continuous support is generally zero. For this reason, a distance d between x and $x_{\theta }$ is used as a decision rule to accept or reject $\theta$. In other words, we sample the set $\left\{\theta \right|d(x,x_{\theta })\le ϵ\}$ for some fixed threshold $ϵ>0$. We should say dissimilarity [31], rather than distance, since, for ABC, a weaker notion than distance is needed, without the triangle inequality and where $x=y\Rightarrow d(x,y)=0$ is true, but the converse $d(x,y)=0\Rightarrow x=y$ is not true in general.

The FR (Equation (1)) is a distance metric for functions. Another distance, which we use, is the estimator for maximum mean discrepancy ($d_{\mathrm{MMD}}$), a non-parametric two-sample statistic for testing whether samples come from the same distribution [32]. This has previously been used as a distance within an ABC sampler [27,33]. In this paper, inputs to distances are EFRV. For the purposes of defining $d_{\mathrm{MMD}}$ we denote these inputs as f and g; however, when used within the d-sampler, one is the observed data f and the other is the model realisation $f_{\theta }$. The definition of $d_{\mathrm{MMD}}(f,g)$ is as follows:

$$\begin{array}{cc}\hfill d_{\mathrm{MMD}}(f,g)& =\frac{1}{m^2}\sum _{i=1}^m\sum _{i^{\prime }=1}^m\left[k(f_i,f_{i^{\prime }})+k(g_i,g_{i^{\prime }})-2k(f_i,g_{i^{\prime }})\right],\hfill \end{array}$$

(2)

where k is some kernel function. The kernel function that we use in this paper is the Gaussian kernel, $k(f_i,g_{i^{\prime }})=exp\left[-0.5\sqrt{{(f_i-g_{i^{\prime }})}^T{S}^{-1}(f_i-g_{i^{\prime }})}\right]$, where S is a fixed tuning covariance matrix. This distance, $d_{\mathrm{MMD}}$, is equivalent to a kernel-smoothed L₂ metric between EFRVs [25]. Therefore, we use $d_{\mathrm{MMD}}$ as a distance on EFRVs rather than probability measures.

Many varieties of ABC samplers exist in the literature, including accept-reject [34], replenishment [35], and simulated annealing [36]. We find it useful in this paper to introduce an additional notion called a distance sampler (d-sampler) (Algorithm 1), which helps to simplify the language used later. The d-sampler is simply the distance viewed as a random variable, which is conditional on $\theta$, and observed functional data f.

Algorithm 1 d-sampler.

Input: $\theta ,f$   Output: $d_{\theta }$   $f_{\theta }\sim p(·|\theta )$   $d_{\theta }=d(f,f_{\theta })$

This notion makes ABC samplers easier to describe. For instance, the rejection sampler can be written as a loop over i with $\left({\theta }_i\sim \pi (·),d_i=\mathrm{d}-\mathrm{sampler}({\theta }_i,f)\right)$ keeping only pairs with $d_i$ less than $ϵ$. In this paper, we adopt the replenishment ABC sampler [35].

To develop an ABC sampler for functional data, we develop d-samplers on $\theta$ and f. We align a model realisation $f_{\theta }$ to f with a warping ${\gamma }^{\ast }$ [11], as discussed in the previous Section. The distance between f and the aligned model realisation $f_{\theta }\circ {\gamma }^{\ast }$ is computed with either the FR or MMD, as is shown in Algorithm 2. In the next section, we compare these d-samplers against their unregistered versions (Algorithm 1) for two simulation examples of shifting peaks.

Algorithm 2 Registered d-sampler.

Input: $\theta ,f$   Output: $d_{\theta }$   $f_{\theta }\sim p(·|\theta )$   ${\gamma }^{\ast }=arg{inf}_{\gamma \in \Gamma }{d}_{\mathrm{FR}}(f,f_{\theta }\circ \gamma )$   $d_{\theta }=d(f,f_{\theta }\circ {\gamma }^{\ast })$

4. Peak Shift

4.1. Methods

To demonstrate our method, we construct two simulation examples. The functional data of the examples are a set of peaks with shifting positions. We test whether ABC with alignment (alignment of f with $f_{\theta }$ before distance computation) outperforms ABC without alignment. The first example is as follows: the mean function is a sum over five Gaussian density functions $\varphi$ and four Cauchy density functions C, i.e., $z\left(t\right)={\sum }_{u=1}^5\varphi \left(t\right|{\mu }_u,{\sigma }_{\varphi })+{\sum }_{u=6}^9\mathrm{C}\left(t\right|{\mu }_u,{\sigma }_{\mathrm{C}})$. This is related to peaks in the spectroscopy, which are either Gaussian or Cauchy (Lorentzian) peaks [37]. The observed functional output corresponding to fixed sampling locations $t=(0,0.5,\cdots ,200)$, is independent and normally distributed $y_j|t_j,{\mu }_u\sim N(z\left(t_j\right),{\sigma }_{ϵ})$, conditional on the time axis value $t_j$ and peak positions ${\mu }_u$. What makes this problem of interest is that the location parameters ${\mu }_u$ are nuisance parameters, treated as random effects to be integrated out. Instead of integration, we use curve registration to produce a distance which is invariant with respect to ${\mu }_u$, the nuisance parameters. These ${\mu }_u$ have distribution $N({\alpha }_u,{\sigma }_u)$ for known values for ${\sigma }_u$ and ${\alpha }_u$. The fact that ${\mu }_u$ is random and high-dimensional makes inelastic distances, such as the L${}^2$ distance, inappropriate.

The prior distributions for the parameters of interest are ${\sigma }_{\varphi }\sim U(0,3)$, ${\sigma }_{\mathrm{C}}\sim U(0,2)$ and ${\sigma }_{ϵ}\sim U(0,0.1)$. An example of a model realisation from this distribution is shown in Figure 2. To test the effectiveness of registered d-samplers in this situation, we compare all four combinations: MMD and FR with and without registration. We set the covariance matrix S for MMD as the $2\times 2$ diagonal matrix with elements 9 and $1\times {10}^{-4}$. These values within S were chosen based on the scale of the data (see Figure 2).

The second simulation example that we construct is similar to the first, except we use a set of skewed normal density functions SN [38]. The purpose of this example is to further test the robustness of our technique. The mean function here is $z\left(t\right)={\sum }_{u=1}^9\mathrm{SN}\left(t\right|{\mu }_u,{\sigma }_{\mathrm{SN}},\eta )$, where ${\sigma }_{\mathrm{SN}}$ is the scale parameter and $\eta$ is the skewness parameter. This is called the centred parameterisation [39]. The prior distributions for the parameters of interest are ${\sigma }_{\mathrm{SN}}\sim U(0,10)$, $\eta \sim U(-0.9,0.9)$, and ${\sigma }_{ϵ}\sim U(0,0.02)$.

4.2. Results

Density plots of posterior samples arising from the ABC sampler with all four d-samplers, registered and unregistered, are shown for the first example in Figure 3 and the second example in Figure 4. The plots show that the registered d-samplers outperform their unregistered counterparts; although the FR is an ideal distance for alignment of functions, when used as a distance for ABC, it is sometimes outperformed by MMD. This suggests that the best distance for registration is not always the best distance for ABC. The reason that MMD outperforms the FR for ${\sigma }_{ϵ}$ is its penalty term. The first term in Equation (2) penalises the concentration of $y^{f_{\theta }}$, such that the algorithm tends towards a higher value of ${\sigma }_{ϵ}$ to match the noise in $y^f$. The same pattern of density plots is seen between the two examples, with the registered distances outperforming others for peak shape parameters (${\sigma }_{\varphi },{\sigma }_C,{\sigma }_{SN}$, and $\eta$), and the MMD distance (both registered and unregistered) outperforms the Fisher–Rao distance for the noise parameter ${\sigma }_{ϵ}$.

The relative merits of each distance are not our primary concern. We are interested in the effect of registration rather than the distance. It is interesting, however, that we can use a different distance for ABC than is used for curve registration and, depending on the parameter of interest, obtain superior results. In further examples, we proceed only with MMD. We now demonstrate a more complex example with a dynamic queueing network model of an international airport terminal.

5. Passenger Processing at an International Airport

Rising demand for air travel and enhanced security screening place pressure on existing airport infrastructure. Airport terminal infrastructure, for financial and geographical reasons, can be difficult to upgrade the rate needed to match this demand. With this in mind, operational planners at airports seek to optimise day-to-day operations. However, optimisation frameworks require models with some degree of realism. These models will necessarily become complicated and this, combined with the data collection scheme, inevitably leads to intractable likelihoods. Simulators are, on the other hand, straightforward to construct, although computationally demanding.

We use a fast algorithm for queueing based simulations, called QDC [40], to construct an ABC sampler for this problem [33]. However, there were problems with misalignment, as noted in their discussion. We consider a simplified (for the sake of exposition) version of this model and data. The purpose of this model is to predict passenger flows through an airport terminal in response to particular flight schedules and staff rosters at an immigration checkpoint. The data comprise records of passenger numbers passing through certain checkpoints for each minute of the day. The flight schedule and staff rosters can be thought of as explanatory variables, since they are known and affect the response variable (passenger flows) to some degree, which we wish to estimate.

We model the arrivals terminal of an international airport. For one day, we have a list of flights u along with their arrival gate $g_u$ and arrival time $a_u$. The walking distance $m_u$ from gate $g_u$ to immigration is also known. The flight arrival information $\mathbf{x}=(u,g_u,a_u)$, collectively, are a type of forcing data; in other words, they are inputs to a simulation model which are themselves subject to observational error [41]. The time that a flight arrives is known, but the time at which a flight opens its doors to deplane passengers is not recorded [33]. This leads to more uncertainty regarding the dynamics of passenger processing. It is of interest whether curve registration helps to improve parameter inference under these conditions.

5.1. Methods

To model passenger flows, we simulate every passenger from every flight. The time at which passenger v from flight u deplanes is $a_{u,v}^{\mathrm{dpl}}=a_u+t_{u,v}^{\mathrm{dpl}}+b_u$, where $t_{u,v}^{\mathrm{dpl}}$ is the time taken for the passenger to leave the aircraft and $b_u\sim U(0,20)$ is a nuisance parameter representing an unobserved amount of time taken by the crew after the recorded arrival time to open the doors, letting passengers out. Note that, in this example, $b_u$ plays the same role as ${\mu }_u$ from the peak shift example, i.e., that of a high-dimensional latent variable, shifting peaks independently.

Once passengers have deplaned, they walk to be processed at immigration and queue for processing. The times at which customers arrive to the queueing system are $a_{u,v}^{\mathrm{imm}}=a_{u,v}^{\mathrm{dpl}}+t_{u,v}^{\mathrm{imm}}$, where $t_{u,v}^{\mathrm{imm}}$ is the walking time to immigration from the arrival gate. The distribution of $t_{u,v}^{\mathrm{imm}}|m_u$ is Gamma$(\alpha ,\beta /m_u)$. This is equivalent to sampling from Gamma$(\alpha ,\beta )$ and multiplying the result by $m_u$. We transform these parameters, for interpretability to mean $\rho =\frac{\alpha }{\beta }$, and variance to mean squared ratio $\nu =\frac{1}{\alpha }$. There are two parallel queueing systems: one for local and one for foreign passengers. The proportion $l_u$ of local passengers on each flight is known and encoded in the Boolean variable ${\mathrm{nat}}_{u,v}\sim \mathrm{Bern}\left(l_u\right)$. Each queueing system proceeds at a different rate ${\lambda }_{{\mathrm{nat}}_{u,v}}$ and has a different number of servers which are non-interchangeable. This is an accurate depiction of real systems, including staff and automated systems working in parallel and which cannot be interchanged.

We require a queueing simulator to compute the times at which passengers leave their assigned queueing system (local or foreign). A queueing simulator (QueueSim) is a function which (conditional on arrival times, service times and number of servers) deterministically computes the times at which passengers leave their queueing system (departure times), see [40]. First, we partition the arrival times $a_{u,v}^{\mathrm{imm}}$ by ${\mathrm{nat}}_{u,v}$; then, we sample service times $s_{u,v}\sim \mathrm{Exp}\left({\lambda }_{{\mathrm{nat}}_{u,v}}\right)$. There is a separate rate parameter for the local passengers queue ${\lambda }_l$ and the foreign passengers queue ${\lambda }_f$. The number of servers for each queue $K_{\mathrm{nat}}$ is known. The departure times from the queueing systems are ${\mathbf{z}}_{\mathrm{nat}}^{\mathrm{imm}}=\mathrm{QueueSim}({\mathbf{a}}_{\mathrm{nat}}^{\mathrm{imm}},{\mathbf{s}}_{\mathrm{nat}},K_{\mathrm{nat}})$, where the variables are written in boldface to denote the full vectors partitioned by nationality, since the movements of each passenger, once queueing begins, are no longer independent. The synthetic data were constructed in the same way so that we could know the true parameter values, which are $\rho =0.02$, $\nu =0.64$, ${\lambda }_l=0.5$, and ${\lambda }_f=0.4$. The priors are independent and uniform for all parameters $\rho \sim U(0,0.05)$, $\nu \sim U(0,1)$, ${\lambda }_l\sim U(0,1)$, ${\lambda }_f\sim U(0,1)$. We bin ${\mathbf{a}}_{\theta }^{\mathrm{imm}}$ and ${\mathbf{z}}^{\mathrm{imm}}$ by minute to construct EFRVs in the form of histograms $f_{\theta }^a$ and $f_{\theta }^z$, respectively. The two functional variables $f_{\theta }^a$ and $f_{\theta }^z$ represent, respectively, the input and output of the immigration system.

Considering that variations in flight arrival times significantly affect the dynamics of queues within the immigration system, we cannot register $f_{\theta }^a$ and $f_{\theta }^z$ independently to $f^a$ and $f^z$, respectively, without introducing modelling error. Our solution here is to “correct” ${\mathbf{a}}_{\theta }^{\mathrm{imm}}$ for forcing data uncertainty with reference to the warping function. We, therefore, obtain three different types of posterior samples (registered, unregistered and corrected). Since ${\mathbf{a}}_{\theta }^{\mathrm{imm}}$ themselves are not functional data, we refer to these as corrected arrival times and denote them as ${\breve{\mathbf{a}}}_{\theta }^{\mathrm{imm}}$. We use curve registration to find the warping function $\gamma$ from $f_{\theta }^a$ to $f^a$ and apply this to the latent variable ${\mathbf{a}}^{\mathrm{imm}}$, correcting ${\mathbf{z}}^{\mathrm{imm}}$ and $f_{\theta }^z$ (Algorithm 3). The type argument encodes whether the d-sampler is unregistered, registered, or corrected. For the registered d-sampler, only the distance on $f_{\theta }^a$ is registered. The corrected d-sampler uses a registered distance on $f_{\theta }^a$ and uses the warping function $\gamma$ to correct the input for a more accurate output sample ${\mathbf{z}}^{\mathrm{imm}}$.

Algorithm 3 Airport d-sampler.

Input: $\theta ,f^a,f^z,\mathrm{type},\mathbf{x}$   Output: $d_{\theta }$   1: ${\mathbf{a}}_{\theta }^{\mathrm{imm}}\sim p_a(·|\rho ,\nu ,\mathbf{x})$   2: $f_{\theta }^a=\mathrm{hist}\left({\mathbf{a}}_{\theta }^{\mathrm{imm}}\right)$   3: ${\breve{\mathbf{a}}}_{\theta }^{\mathrm{imm}}={\mathbf{a}}_{\theta }^{\mathrm{imm}}$   4: if type = registered or corrected then   5:     $\gamma =arg{inf}_{\gamma \in \Gamma }{d}_{\mathrm{FR}}(f^a,f_{\theta }^a\circ \gamma )$   6:     $d_{\theta }^a=d(f^a,f_{\theta }^a\circ \gamma )$   7:     if type = corrected then   8:         ${\breve{\mathbf{a}}}_{\theta }^{\mathrm{imm}}=\gamma \left({\mathbf{a}}_{\theta }^{\mathrm{imm}}\right)$ 9:     end if  10: else  11:     $d_{\theta }^a=d(f^a,f_{\theta }^a)$   12: end if  13: ${\mathbf{z}}_{\theta }^{\mathrm{imm}}\sim p_a(·|{\breve{\mathbf{a}}}_{\theta }^{\mathrm{imm}},\lambda ,K)$   14: $f_{\theta }^z=\mathrm{hist}\left({\mathbf{z}}_{\theta }^{\mathrm{imm}}\right)$   15: $d_{\theta }=d_{\theta }^a+d(f^z,f_{\theta }^z)$

5.2. Results

Density plots of posterior samples for all parameters of interest, for each d-sampler, are shown in Figure 5. We see that the registered d-sampler outperforms the unregistered d-sampler for $\rho$ and $\nu$, which relates to the walking speed, as well as outperforming the unregistered d-sampler for ${\lambda }_f$ and is as least as good as the unregistered d-sampler for ${\lambda }_l$. Furthermore, we see that applying the correction to the input to the immigration system improves the accuracy of the output since the posterior density for the corrected d-sampler outperforms all others for all parameters. Curve registration therefore helps improve parameter inference under the stated conditions and can even help to improve inference by aligning “correcting” forcing data. Models involving a stochastic transformation from unobserved latent variables may benefit from using this technique.

6. Hydrological Modelling

Water flows within catchment areas, in response to rainfall and other weather events, are often modelled and forecast with run-off models, where water flows are represented by hydrographs showing volumetric flow-rates at a particular point (see Figure 6) [42,43]. In particular, the Génie Rural à 4 paramètres Journalier (GR4J) model of [44] is widely used. Parameter estimation for hydrological methods is an active area of research [45]. Once parameters are estimated, parameter uncertainty itself contributes little to overall uncertainty compared to the residual error structure [46].

Hydrographs are EFRVs; their functional form is controlled by model parameters $\theta$ and forcing data $\mathbf{x}$. For the GR4J model, $\mathbf{x}$ are rainfall patterns [47]. The peak positions within the functional data are affected by $\theta$ and $\mathbf{x}$. However, unlike the airport example, $\mathbf{x}$ are assumed to be measured without error. As in the previous example, we are interested in whether curve registration helps improve parameter inference.

6.1. Methods

In this example we use the GR4J simulation model. This model contains four parameters: the first three $({\theta }_1,{\theta }_2,{\theta }_3)$ are capacities in units of length (mm); and the fourth parameter (${\theta }_4$) represents a lagged effect in units of time (days). Predictions from the GR4J, given $\theta =({\theta }_1,{\theta }_2,{\theta }_3,{\theta }_4)$ and $\mathbf{x}$, are denoted as $\mathcal{H}(\theta ,\mathbf{x})$. As in many run-off models, the GR4J model is deterministic. To provide useful prediction intervals, hydrologists typically add an error structure to the deterministic model. Run-off predictions are positively valued, and errors are heteroscedastic [45]. A common strategy is to perform a Box-Cox transformation $B(x,\lambda )=(x^{\lambda }-1)/\lambda$ of the deterministic output, add noise $ϵ$ and then invert the Box-Cox transformation:

$$\begin{array}{cc}\hfill f^z& =B^{-1}[B\left[\mathcal{H}(\theta ,\mathbf{x})\right]+ϵ],\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill ϵ& \sim N(0,{\sigma }^2).\hfill \end{array}$$

(4)

The model is an input/output model like the airport example; however, in this case, the input is not a latent variable, but observed directly, so we do not include a corrected d-sampler. We use the same d-samplers used in the peak shift simulation examples (Algorithms 1 and 2).

6.2. Results

As the previous examples, this example includes peak shifting. However, in this case, we see from density plots of posterior samples (Figure 7) that the unregistered d-sampler outperforms the registered d-sampler. In the previous examples, there was a high-dimensional latent variable ${\mu }_u$ affecting the peak locations; in this case, there is only one variable affecting the peak locations, i.e., ${\theta }_4$. Since the peak positions shift according to a one-dimensional parameter of interest ${\theta }_4$, rather than a high-dimensional nuisance parameter, the process of registration is unnecessary here, and adds noise to the d-sampler, flattening the posterior densities. Curve registration will therefore not improve parameter inference, since the ABC sampler correctly identifies appropriate values for ${\theta }_4$.

7. Conclusions

In this work, curve registration techniques are applied to the problem of misaligned functional data for likelihood-free inference for the first time. Interested readers can find the code we used at https://github.com/AnthonyEbert/CurveRegistration, accessed on 3 September 2021. We used the FR as an objective function to align model realisations to observed data by warping the time axis. Once functional data were aligned, we compared the FR and MMD as distances for posterior inference within an ABC sampler. We tested this approach with four examples: two peak shift examples (Section 4), the airport example (Section 5), and a real hydrology example (Section 6). The examples highlight different aspects and use-cases of applying curve registration techniques to ABC.

In the peak shift examples of Section 4, we found that the registered d-samplers based on both the FR and MMD outperform the unregistered d-samplers, although the MMD, in some cases, outperformed FR. This means that the distance for alignment is not necessarily the best distance for ABC samplers once alignment is performed. In the airport example of Section 5, we explored the effect of stochasticity in forcing data affecting peak positions. We found that ABC inference based on curve registered functional data increased posterior concentration near the true value. Furthermore, we found that we can “correct” ${\breve{\mathbf{a}}}_{\theta }^{\mathrm{imm}}$, a latent variable computed within the model itself, by using the warping function computed from the original alignment. This further improves posterior inference, counteracting the effect of forcing data uncertainty. Finally, in the hydrological example (Section 6), the ABC sampler correctly identifies appropriate values for ${\theta }_4$, making curve registration unnecessary in this example; when peak location stochasticity is entirely controlled by parameters of interest, then curve registration ceases to improve posterior inference for those parameters.

While the Fisher–Rao metric is an exceedingly appropriate distance for curve registration, in these cases, it was not necessarily the best distance to use for approximate Bayesian computation (ABC). This means that two distances may be ideal for different purposes within an ABC sampler.

ABC samplers suffer in high dimensions; therefore, the purpose of curve registration is to build distances which are invariant with respect to certain types of high-dimensional nuisance parameters. We find, illustrated by the four examples, that curve registration provides a valuable technique for posterior inference based on misaligned functional data, where variation in peak positions is controlled by nuisance parameters.