Integrated Model Selection and Scalability in Functional Data Analysis Through Bayesian Learning

Abstract

Functional data, including one-dimensional curves and higher-dimensional surfaces, have become increasingly prominent across scientific disciplines. They offer a continuous perspective that captures subtle dynamics and richer structures compared to discrete representations, thereby preserving essential information and facilitating the more natural modeling of real-world phenomena, especially in sparse or irregularly sampled settings. A key challenge lies in identifying low-dimensional representations and estimating covariance structures that capture population statistics effectively. We propose a novel Bayesian framework with a nonparametric kernel expansion and a sparse prior, enabling the direct modeling of measured data and avoiding the artificial biases from regridding. Our method, Bayesian scalable functional data analysis (BSFDA), automatically selects both subspace dimensionalities and basis functions, reducing the computational overhead through an efficient variational optimization strategy. We further propose a faster approximate variant that maintains comparable accuracy but accelerates computations significantly on large-scale datasets. Extensive simulation studies demonstrate that our framework outperforms conventional techniques in covariance estimation and dimensionality selection, showing resilience to high dimensionality and irregular sampling. The proposed methodology proves effective for multidimensional functional data and showcases practical applicability in biomedical and meteorological datasets. Overall, BSFDA offers an adaptive, continuous, and scalable solution for modern functional data analysis across diverse scientific domains.

1. Introduction

The emergence of big data across diverse fields, such as biomedicine, finance, and physical modeling, has catalyzed the need for advanced analytical methodologies capable of handling complex, high-dimensional datasets that conventional discrete data analysis approaches cannot always process effectively. Such datasets often require analysis that captures and interprets their continuous and potentially high-dimensional complexities—a central promise of functional data analysis (FDA) [1,2]. Foundational work established FDA’s capacity to treat each observation as an entire function [3], be it a curve, surface, or higher-dimensional structure, thereby extracting richer insights than conventional discrete point analyses. Over the past decade, FDA’s scope has widened significantly to accommodate high-dimensional applications, with theoretical and computational advances emerging across various contexts [4,5].

A pivotal technique within FDA is functional principal component analysis (fPCA), which serves as a dimension reduction tool similar to classical PCA and factor analysis. Unlike classical PCA, however, fPCA operates, in principle, in an infinite-dimensional function space to capture dominant modes of variation and reduce complexity [6]. Despite its conceptual elegance, existing fPCA and similar FDA models often assume that data are observed on a shared, finite grid, often relying on heuristic imputation or posterior estimation to handle any missing entries [7,8,9]. This assumption conveniently facilitates the adoption of established linear algebraic methods but compromises the integrity of FDA by introducing significant information loss and high-computational demands in high-dimensional applications.

An ideal approach would represent each function at its naturally sampled measurement points rather than forcing all observations onto a shared grid, thus preserving crucial information and avoiding the need for heuristic resampling. This point is critical when considering that, given only a finite number of data points, infinitely many functions can interpolate these points, each reflecting different inductive biases about smoothness or shape [10]. Conventional smoothing or regridding methods (e.g., polynomial interpolation) introduce biases that may distort the underlying function’s actual behavior. In contrast, we will achieve more accurate and unbiased predictions by concurrently updating the function estimation and the population-level statistics governing the estimation, such as those encoded in the covariance operator. Such an approach requires the direct modeling of the function from its original measurement points, rather than imposing artificial grids.

To mitigate these limitations, several studies have proposed alternative strategies. For instance, Ref. [3] developed a nonparametric technique for the estimation of the mean and covariance for functional data under smoothness assumptions while also discussing a continuous formulation and the necessary discretization in practical applications. In [11,12], the authors extended fPCA to sparse and irregular longitudinal designs by smoothing the covariance estimate and then discretizing. Nonetheless, classical discretization steps often result in significant information loss and computational burdens.

As functional data’s size and complexity grew, researchers turned to flexible basis expansions, including sinusoids (Fourier), wavelets, polynomials, and B-splines, for a finite-dimensional representation of functional data that is convenient and accurate in computation, avoiding the drawbacks of explicit approximation and resampling [2,13,14]. However, a core challenge remains in selecting a suitable model. For instance, researchers must choose the number and form (e.g., smoothness), along with the dimensionality of the representational subspace. In approximation, the placement of basis functions is also essential. Evenly spaced nodes remain popular for their simplicity but may be suboptimal. Alternative node allocations may be better, such as Chebyshev nodes for superior accuracy [15] or sparse grids to reduce the combinatorial growth of the computational complexity [16].

Existing studies tend to rely on choosing the hyperparameters manually [6] or on cross-validation [3,14,17], which are known to be computationally prohibitive. Others employ approximated cross-validation [13] or marginal likelihood [8], but these still require the exhaustive testing of all candidate models. Methods with sparse Bayesian priors [7,18] for model selection allow model selection with a single optimization. In [19,20], the authors use shrinkage or sparse priors for data-adaptive basis selection to ensure minimal but effective sets of basis functions. Notably, Ref. [21] proposed the Bayesian and Akaike information criteria, demonstrating state-of-the-art performance in simulation studies for sparse and dense functional data.

In addition, probabilistic FDA emerges as a sophisticated adaptation of probabilistic methods tailored to incorporate the flexibility of latent variable models to manage functional data. A Bayesian latent factor regression model (LFRM) [18], for example, extends conventional regression to accommodate complex structures and dependencies in functional data, providing a robust framework to handle the complexities inherent in functional data. However, these Bayesian approaches are often limited by the computational demands of Monte Carlo methods in high dimensions [8]. To address increasingly high-dimensional FDA problems, recent efforts have emphasized scalability. For instance, Ref. [6] introduced FDA for images with a fixed basis or grid. In [17], they further reduced the complexity in 2D fPCA via tensor product B-splines. Meanwhile, Ref. [22] applied a Bayesian framework with basis expansion, adaptive regularization, and Gibbs sampling to 2D functional data in the form of EEG studies on children with autism. Furthermore, Ref. [23] leveraged a parsimonious basis representation and variational Bayes to achieve computational efficiency, making it suitable for 3D brain imaging data. A Bayesian nonparametric model [24] leverages variational inference for efficient computation in high-dimensional functional time series and uses an Indian buffet process to automatically select latent factors. Nonetheless, it focuses on 1D functional observations with temporal dependencies and a common sampling grid.

In parallel, the broader method of principal component analysis (PCA) remains a fundamental and effective tool. Classical PCA, rooted in eigendecomposition [25], effectively extracts dominant modes of variation in many settings but does not inherently accommodate the probabilistic nature of real-world data and their inherent uncertainties. Thus, Ref. [26] introduced probabilistic PCA (PPCA), which incorporates a probability distribution to manage these uncertainties more effectively. PCA has since evolved to address missing data [27], model selection [28], and complex data types [29]. In the context of functional data, these concepts motivate new approaches that unify probabilistic methodologies, latent factor models, and kernel expansions for continuous domains [1].

Within Bayesian machine learning, various priors have been proposed for sparse or robust formulations of PCA. Specifically, sparse Bayesian learning (SBL) [30], with its mechanism automatic relevance determination (ARD) [31,32], has proven adept in promoting parsimonious solutions [33]. SBL has emerged in Bayesian PCA [28], applying an iterative method to evaluate the relevance of each component and select the internal dimensionality by disregarding the redundant ones. In [34], the authors applied SBL to optimize the combination of base kernels to enhance model performance. A matrix completion method [35] uses ARD to select the factorization rank and dual graph priors to promote smoothness along rows and columns for the effective interpolation of missing entries, although they are more relaxed than a strict continuity constraint for FDA. These methods often exploit variational techniques or accelerated optimization [36], thereby balancing model complexity with computational tractability. In functional data contexts, where representations are infinite-dimensional, SBL offers a compelling framework for advanced FDA methods by efficiently handling sparse expansions and adaptively adjusting the model complexity.

In summary, despite these efforts to advance functional data analysis, several challenges persist. Existing methods often exhibit limitations in accuracy and efficiency when sampling is sparse, automatic model selection is essential, and the dimensionality is high [23]. Concurrently, probabilistic PCA and SBL frameworks illustrate powerful strategies to incorporate versatility and adaptivity for such data complexities, while their adaptation to FDA is still evolving. These gaps underscore the need for a robust, flexible, and computationally feasible approach, unifying ideas from FDA, PPCA, and SBL, that manages the continuous and high-dimensional nature of modern datasets.

1.1. Contributions

This manuscript proposes a novel Bayesian framework for functional principal component analysis that leverages nonparametric kernel expansions, sparse Bayesian learning for model selection, and efficient variational inference (VI). We abbreviate the proposed method as BSFDA (Bayesian scalable functional data analysis). (The code is available at https://github.com/WeeenZh/BSFDA, accessed on 21 March 2025). BSFDA addresses critical gaps in existing FDA techniques with irregular sampling, high-dimensional scalability, and the selection of both basis functions and principal components. Specifically, our approach offers the following:

• Joint selection of optimum latent factors and sparse basis functions: This eliminates constraints on parametric representation dimensionality, avoids information loss from discretization, and extends naturally to higher dimensions or non-Euclidean spaces through nonparametric kernel expansion. It further enhances the interpretability by adaptively choosing the model complexity without testing multiple models separately. We achieve these improvements using a Bayesian paradigm that provides robust and accurate posterior estimates while supporting uncertainty quantification.
• Scalability across domain dimensionality and data size: The proposed method uses VI for faster computation compared to Markov chain Monte Carlo (MCMC) methods, while still being accurate in terms of the estimation of the intrinsic dimensionality and overall covariance structure. BSFDA reduces the overall computation by partitioning the parameters into smaller update groups and introducing a slack variable to further subdivide the weighting matrix (which is part of the kernel structure) into even smaller parts [18], updating fewer blocks at a time and considering all model options. Introducing a slack variable makes the optimization process more efficient by separating different variable groups. This approach scales well with the data size and works efficiently even with large, complex datasets. We demonstrate this on the 4D global oceanic temperature dataset (ARGO), which consists of 127 million data points spanning across the globe for 27 years, with depths of up to 200 m [37].

1.2. Outline

Together, these contributions position our work at the intersection of functional principal component analysis [1] and sparse Bayesian learning [30], enabling the robust, flexible, and computationally feasible analysis of high-dimensional functional data. The remainder of this paper is organized as follows. In Section 2, we describe the proposed Bayesian functional PCA framework in detail, highlighting the nonparametric kernel expansions and sparse Bayesian priors. Next, in Section 3 and Section 4, we discuss the variational inference procedure and the reduced active block updating step, illustrating how these techniques jointly provide scalability and accuracy. In Section 5, we then present extensive empirical studies demonstrating the factor selection accuracy, covariance operator estimation, and performance in large-scale 4D applications. Finally, in Section 6, we conclude with a discussion of potential extensions and open directions, emphasizing the broader implications of our work for large-scale, high-dimensional functional data analysis.

2. Formulation

2.1. Generative Model

In conventional fPCA, the data are assumed to be samples of functions that are elements of an appropriately smooth function space [1]. Using this assumption, data samples acquired at discrete points are typically interpolated to the continuum using tools such as splines, Fourier basis functions, or wavelets. In our work, we assume that the functions $y_i:R^M\mapsto R$ are outcomes of an M-dimensional stochastic process. As in classical fPCA, we assume that $y_i$ is in a class of functions that can be approximated through a truncated, finite expansion, which is a weighted summation of K kernel functions ${\left\{{\varphi }_k\right\}}_{k=1}^K$:

$$\begin{array}{c}\hfill y_i\left(x\right)=\sum _{k=1}^K{w}_{ik}{\varphi }_k\left(x\right),\end{array}$$

(1)

with $w_{ik}$ being random variables, ${\varphi }_k\left(x\right)=\mathcal{K}(x,{\mathbb{X}}_k)$, $\mathcal{K}$ is the kernel function, and ${\mathbb{X}}_k$ is the k-th location.

Thus, $y_i\left(x\right)$s are realizations of a finite-dimensional stochastic process. Conventionally, the covariance operator of these functions is discretized, and the leading eigenfunctions form the estimated principal component loadings, following the Karhunen–Loève theorem [1]. By contrast, we establish a flexible Bayesian framework of fPCA following the form of probabilistic PCA [26], where the principal subspace is identifiable up to an arbitrary rotation and does not enforce the orthogonality of the loadings. Nevertheless, it is straightforward to recover classical eigenfunctions from the final covariance estimation over an arbitrary grid in this Bayesian framework.

The observed data are P independent, noisy samples of the functions ${\left\{y_i\right\}}_{i=1}^P$ at index ${\{X_i\in R^{N_i\times M}\}}_{i=1}^P$, where $N_i$ is the number of measured samples for the ith function $y_i$ and $X_{in}\in R^M$ is the location of the nth measurement in the domain of the sample. The observations are ${\left\{Y_i\right\}}_{i=1}^P$, where $Y_{in}=y_i\left(X_{in}\right)+E_{in}$, where $E_{in}$ is white Gaussian noise of variance ${\sigma }^2$.

We also assume that the functions span a low-dimensional subspace of dimension $J<<K$. We model this stochastically by assuming that the weights, $w_i\in R^K$, are given by $w_{ik}={\sum }_{j=1}^J{Z}_{ij}{W}_{jk}+{\overline{Z}}_k,$ where $W\in R^{J\times K}$ are the principal component loading coefficients and $Z_i\in R^J$ are standard normal variables [26]. This model is therefore

$$\begin{array}{cc}\hfill Y_{in}& =\sum _{k=1}^K\left(\left(\sum _{j=1}^J\left(Z_{ij}{W}_{jk}\right)+{\overline{Z}}_k\right){\varphi }_k\left(X_{in}\right)\right)+E_{in}=(Z_{i}W+\overline{Z}){\Phi }_{i·n}+E_{in},\hfill \end{array}$$

(2)

where ${\Phi }_{i·n}={[{\varphi }_1\left(X_{in}\right),\dots ,{\varphi }_K\left(X_{in}\right)]}^T$ are the evaluations of the basis functions at the n-th index of the i-th sample function.

The choice of the kernel family usually benefits from knowledge of the dataset’s characteristics, such as the periodicity or domain geometry. Our framework is flexible across various kernel families, but we employ Gaussian kernels for both one-dimensional and multidimensional data by default, with the initial length-scale selection carried out through cross-validation, which will be refined through our sparse prior described below. To avoid disproportionately favoring larger length scales, we normalize each kernel’s scaling coefficient using its square integral over the observational domain.

2.2. Sparse Prior

For effective model selection, we introduce a sparse prior over the coefficients of the basis functions [28]. The sparse prior in the proposed model is based on automatic relevance detection (ARD) [28]. ARD evaluates the importance of a feature with a precision parameter estimated from the data. The model uses ${\left\{{\alpha }_j\right\}}_{j=1}^J$ and ${\left\{{\beta }_k\right\}}_{k=1}^K$ for the numbers of components and basis functions, respectively, while $\eta$ signifies the overall magnitude of the mean coefficients:

$$\begin{array}{cc}\hfill {\overline{Z}}_k& \sim \mathcal{N}(0,{\eta }^{-1}{\beta }_k^{-1}),\forall k=1:K\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill W_{jk}& \sim \mathcal{N}(0,{\alpha }_j^{-1}{\beta }_k^{-1}),\forall j=1:J,k=1:K\hfill \end{array}$$

(4)

In the model, ${\alpha }_j,{\beta }_k,\eta ,{\sigma }^{-2}$ are all variables of precision parameters, coming naturally with a conjugate prior of Gamma distribution that facilitates efficient posterior optimization. The probabilistic graphical model is depicted in Figure 1. Setting $a_0,b_0$ to a small value yields a vague Gamma prior that approximates a noninformative (Jeffreys-type) prior.

3. Methods

Based on the proposed formulation in Section 2, we estimate $\mathrm{Pr}[\Theta |X,Y,a_0,b_0]$, the posterior of the unobserved values $\Theta =\{Z,W,\overline{Z},\sigma ,\alpha ,\beta ,\eta \}$. This inference gives the point estimates of $\Theta$ and the posterior predictive distribution of new data. For notational convenience, $X,a_0,$ and $b_0$ are omitted.

Using Bayes’ theorem, $\mathrm{Pr}[\Theta |Y]={\displaystyle \frac{\mathrm{Pr}\left[Y\right|\Theta \left]\mathrm{Pr}\right[\Theta ]}{\mathrm{Pr}\left[Y\right]}}$, but the exact posterior distribution is intractable because the evidence $\mathrm{Pr}\left[Y\right]=\int \mathrm{Pr}[\Theta ,Y]d\Theta$ is intractable. Therefore, an approximate inference strategy is proposed. To facilitate this, we utilize variational inference (VI) [38], choosing a surrogate density from a parameterized family, denoted as $\mathcal{Q}$, to approximate the posterior. Compared with classical methods such as Markov chain Monte Carlo (MCMC) sampling, VI is typically faster [38]. In our experiments, VI is about 85 times faster for the original Bayesian PCA formulation [28], as shown in Appendix F.2.

3.1. Variational Bayesian Inference

Variational inference optimizes $\mathcal{Q}$ by maximizing the lower bound $\mathcal{L}$ (minimizing the KL divergence between the actual and surrogate distributions):

$$\begin{array}{c}\hfill {\mathbb{E}}_{\mathcal{Q}}\left[\ln {\displaystyle \frac{\mathrm{Pr}[\Theta ,Y]}{\mathcal{Q}(\Theta )}}\right]=-\mathrm{KL}\left(\mathcal{Q}(\Theta )\right|\left|\mathrm{Pr}[\Theta |Y]\right)+\ln \mathrm{Pr}\left[Y\right]\propto -\mathrm{KL}\left(\mathcal{Q}(\Theta )\right|\left|\mathrm{Pr}[\Theta |Y]\right).\end{array}$$

(5)

The mean field variational family is used for $\mathcal{Q}$. It simplifies the optimization by assuming that the surrogate posterior distributions are independent, allowing each variable in the posterior to be optimized independently: ${\mathcal{Q}}_{\Theta }={\prod }_i{\mathcal{Q}}_{{\Theta }_i}$. The posterior for each variable is chosen to be conjugate, further simplifying the optimization. Thus, the posteriors of the component scores Z, the weighting matrix W, and the mean weights $\overline{Z}$ are normal distributions. Here, W is vectorized via $\mathrm{vec}\left(W\right)$ without altering its normality assumption. Meanwhile, the posteriors of the precision variables of noise ${\sigma }^{-2}$, components $\alpha$, basis functions $\beta$, and mean weights $\eta$ are Gamma distributions:

$$\begin{array}{cc}\hfill {\mathcal{Q}}_Z\left(Z\right)& =\prod _i{\mathcal{Q}}_{Z_i}\left(Z_i\right)=\prod _i\mathcal{N}\left(Z_i\right|{\mu }_{Z_i},{\Sigma }_{Z_i})\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill {\mathcal{Q}}_W\left(W\right)& =\mathcal{N}\left(\mathrm{vec}\left(W\right)\right|{\mu }_{\mathrm{vec}\left(W\right)},{\Sigma }_{\mathrm{vec}\left(W\right)})\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill {\mathcal{Q}}_{\overline{Z}}\left(\overline{Z}\right)& =\mathcal{N}\left(\overline{Z}\right|{\mu }_{\overline{Z}},{\Sigma }_{\overline{Z}})\hfill \end{array}$$

(8)

$$\begin{array}{cc}\hfill {\mathcal{Q}}_{\sigma }\left(\sigma \right)& =\Gamma \left({\sigma }^{-2}\right|a_{\sigma },b_{\sigma })\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill {\mathcal{Q}}_{\alpha }\left(\alpha \right)& =\prod _j{\mathcal{Q}}_{{\alpha }_j}=\prod _j\Gamma \left({\alpha }_j\right|a_{{\alpha }_j},b_{{\alpha }_j})\hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill {\mathcal{Q}}_{\beta }\left(\beta \right)& =\prod _k{\mathcal{Q}}_{{\beta }_k}=\prod _k\Gamma \left({\beta }_k\right|a_{{\beta }_k},b_{{\beta }_k})\hfill \end{array}$$

(11)

$$\begin{array}{cc}\hfill {\mathcal{Q}}_{\eta }\left(\eta \right)& =\Gamma (\eta ,|a_{\eta },b_{\eta })\hfill \end{array}$$

(12)

Update Steps

In mean field approximation using the surrogate posterior ${\mathcal{Q}}_{\Theta }={\prod }_i{\mathcal{Q}}_{{\Theta }_i}$ conditioned on observations Y, the lower bound is maximized with respect to each unknown ${\Theta }_i$. With the conjugate prior, the optimal updates (denoted with “←”) make the moments of ${\mathcal{Q}}_{{\Theta }_i}$ equal to the moments conditioned on the remaining parts of ${\mathcal{Q}}_{\Theta }$ [38]:

$$\begin{array}{c}\hfill {\mathcal{Q}}_{{\Theta }_i}\leftarrow {\displaystyle \frac{\exp \left({\mathbb{E}}_{{\mathcal{Q}}_{/{\Theta }_i}}\left[\ln \left(\mathrm{Pr}[Y,\Theta ]\right)\right]\right)}{\int \exp \left({\mathbb{E}}_{{\mathcal{Q}}_{/{\Theta }_i}}\left[\ln \left(\mathrm{Pr}[Y,\Theta ]\right)\right]\right)d{\Theta }_i}}\end{array}$$

(13)

From Equation (13), detailed update rules for each variable are presented subsequently, and the derivations of these formulas are given in the Appendix part.

Updates for the parameters of the posterior for the precision of components ${\mathcal{Q}}_{{\alpha }_j},\forall j=1:J$:

$$\begin{array}{cc}\hfill a_{{\alpha }_j}& \leftarrow a_0+{\displaystyle \frac{K}{2}},\hfill \end{array}$$

(14)

$$\begin{array}{cc}\hfill b_{{\alpha }_j}& \leftarrow b_0+{\displaystyle \frac{1}{2}}\sum _{k=1}^K{\mathbb{E}}_{{\mathcal{Q}}_{/{\alpha }_j}}\left[W_{jk}^2{\beta }_k\right]=b_0+{\displaystyle \frac{1}{2}}\sum _{k=1}^K\left(\left({\Sigma }_{W_{jk}}+{\mu }_{W_{jk}}^2\right){\displaystyle \frac{a_{{\beta }_k}}{b_{{\beta }_k}}}\right),\hfill \end{array}$$

(15)

where Equation (14) calculates the corrected degrees of freedom and Equation (15) calculates the corrected sum of squares. As $a_0$ and $b_0$ approach 0, the expectation of precision ${\alpha }_j$, which is ${\mathbb{E}}_{{\mathcal{Q}}_{{\alpha }_j}}\left[{\alpha }_j\right]={\displaystyle \frac{a_{{\alpha }_j}}{b_{{\alpha }_j}}}$, is exactly the inverse of the empirical or sample variance.

Updates for the parameters of the posterior of the precision of the mean weights ${\mathcal{Q}}_{\eta }$:

$$\begin{array}{cc}\hfill a_{\eta }& \leftarrow a_0+{\displaystyle \frac{K}{2}},\hfill \end{array}$$

(16)

$$\begin{array}{cc}\hfill b_{\eta }& \leftarrow b_0+{\displaystyle \frac{1}{2}}\sum _{k=1}^K{\mathbb{E}}_{{\mathcal{Q}}_{/\eta }}\left[{\overline{Z}}_k^2{\beta }_k\right]=b_0+{\displaystyle \frac{1}{2}}\sum _{k=1}^K\left(\left({{\Sigma }_{\overline{Z}}}_k+{{\mu }_{\overline{Z}}}_k^2\right){\displaystyle \frac{a_{{\beta }_k}}{b_{{\beta }_k}}}\right)\hfill \end{array}$$

(17)

Updates for the parameters of the posterior of the precision of basis functions ${\mathcal{Q}}_{{\beta }_k},\forall k=1:K$:

$$\begin{array}{cc}\hfill a_{{\beta }_k}& \leftarrow a_0+{\displaystyle \frac{J+1}{2}},\hfill \end{array}$$

(18)

$$\begin{array}{cc}\hfill b_{{\beta }_k}& \leftarrow b_0+{\displaystyle \frac{1}{2}}{\mathbb{E}}_{{\mathcal{Q}}_{/{\beta }_k}}[{\overline{Z}}_k^2\eta +\sum _{j=1}^J{W}_{jk}^2{\alpha }_j]\hfill \\ \hfill & =b_0+{\displaystyle \frac{1}{2}}\left(\left({{\Sigma }_{\overline{Z}}}_{kk}+{{\mu }_{\overline{Z}}}_k^2\right){\displaystyle \frac{a_{\eta }}{b_{\eta }}}+\sum _{j=1}^J\left(\left({\Sigma }_{W_{jk}}+{\mu }_{W_{jk}}^2\right){\displaystyle \frac{a_{{\alpha }_j}}{b_{{\alpha }_j}}}\right)\right)\hfill \end{array}$$

(19)

Updates for the parameters of the posterior of the mean weights ${\mathcal{Q}}_{\overline{Z}}$:

$$\begin{array}{cc}\hfill {\Sigma }_{\overline{Z}}& \leftarrow {\left({\mathbb{E}}_{{\mathcal{Q}}_{{/}_{\overline{Z}}}}\left[{\sigma }^{-2}\sum _{i=1}^P{\Psi }_i+\eta \mathrm{diag}\left(\beta \right)\right]\right)}^{-1}={\left({\displaystyle \frac{a_{\sigma }}{b_{\sigma }}}\sum _{i=1}^P{\Psi }_i+{\displaystyle \frac{a_{\eta }}{b_{\eta }}}\mathrm{diag}\left({\displaystyle \frac{a}{b}}\right)\right)}^{-1},\hfill \end{array}$$

(20)

$$\begin{array}{cc}\hfill {\mu }_{\overline{Z}}& \leftarrow \left({\mathbb{E}}_{{\mathcal{Q}}_{{/}_{\overline{Z}}}}\left[{\sigma }^{-2}\right]\sum _{i=1}^P(Y_i-{\mathbb{E}}_{{\mathcal{Q}}_{{/}_{\overline{Z}}}}\left[Z_{i}W\right]{\Phi }_i){\Phi }_i^T\right){\Sigma }_{\overline{Z}}=\left({\displaystyle \frac{a_{\sigma }}{b_{\sigma }}}\sum _{i=1}^P(Y_i-{\mu }_{Z_i}{\mu }_W{\Phi }_i){\Phi }_i^T\right){\Sigma }_{\overline{Z}}\hfill \end{array}$$

(21)

where $\mathrm{diag}\left(\beta \right)$ denotes the diagonal matrix with diagonal entries given by $\beta$. Equation (20) indicates that the eigenvectors of ${\Sigma }_{\overline{Z}}$ are solely determined by the sum of Gram matrices ${\sum }_{i=1}^P{\Psi }_i$, where ${\Psi }_i={\Phi }_i{\Phi }_i^T$, while the eigenvalues of ${\Sigma }_{\overline{Z}}$ have a negative correlation with the scale of ${\sum }_{i=1}^P{\Psi }_i$, the prior $\eta \mathrm{diag}\left(\beta \right)$, and data-dependent term ${\sigma }^{-2}$. This is sensible because, for instance, large noise would result in large uncertainty in $\overline{Z}$. In Equation (21), the data residuals, excluding the component scores, are projected into the K-dimensional space through the inner product, with ${\Phi }_i$ and summed over all sample functions to calculate the mean weights.

Updates for the parameters of the posterior of the weights ${\mathcal{Q}}_W$:

$$\begin{array}{cc}\hfill {\Sigma }_{\mathrm{vec}\left(W\right)}& \leftarrow {\mathbb{E}}_{{\mathcal{Q}}_{{/}_W}}{\left[{\sigma }^{-2}\sum _{i=1}^P\left({\Psi }_i^T\otimes \left(Z_i^T{Z}_i\right)\right)+\mathrm{diag}\left(\beta \right)\otimes \mathrm{diag}\left(\alpha \right)\right]}^{-1}\hfill \\ \hfill & ={\left({\displaystyle \frac{a_{\sigma }}{b_{\sigma }}}\sum _{i=1}^P\left({\Psi }_i^T\otimes ({{\mu }_{Z_i}}^T{\mu }_{Z_i}+{\Sigma }_{Z_i})\right)+\mathrm{diag}\left({\displaystyle \frac{a}{b}}\right)\otimes \mathrm{diag}\left({\displaystyle \frac{c}{d}}\right)\right)}^{-1},\hfill \end{array}$$

(22)

$$\begin{array}{cc}\hfill {\mu }_{\mathrm{vec}\left(W\right)}& \leftarrow {\mathbb{E}}_{{\mathcal{Q}}_{{/}_W}}\left[-{\sigma }^{-2}\sum _{i=1}^P\mathrm{vec}{\left({\left({\Phi }_i({\Phi }_i^T{\overline{Z}}^T-Y_i^T)Z_i\right)}^T\right)}^T\right]{\Sigma }_{\mathrm{vec}\left(W\right)}\hfill \\ \hfill & =-{\displaystyle \frac{a_{\sigma }}{b_{\sigma }}}\sum _{i=1}^P\mathrm{vec}{\left({\left({\Phi }_i({\Phi }_i^T{{\mu }_{\overline{Z}}}^T-Y_i^T){\mu }_{Z_i}\right)}^T\right)}^T{\Sigma }_{\mathrm{vec}\left(W\right)}\hfill \end{array}$$

(23)

Equation (22) is similar to Equation (20), because it is correlated with ${\Phi }_i$, its prior $\mathrm{diag}\left(\beta \right)\otimes \mathrm{diag}\left(\alpha \right)$, and data-dependent terms ${\sigma }^{-2}$ and $Z_i$. In Equation (23), the data residual excluding the mean function is used to estimate the expectation of W.

Updates for the parameters of the posterior of the component scores ${\mathcal{Q}}_{Z_i}$:

$$\begin{array}{cc}\hfill H_{ijk}& \leftarrow {\mathbb{E}}_{{\mathcal{Q}}_{{/}_{Z_i}}}\left[W_j{\Psi }_i{W}_k^T\right]=\mathrm{Tr}\left({\mathbb{E}}_{{\mathcal{Q}}_{{/}_{Z_i}}}\left[W_k^T{W}_j\right]{\Psi }_i\right)\hfill \\ \hfill & =Tr\left(\left({\Sigma }_{[W_k,W_j]}+{\mu }_{\left[W_j\right]}^T{\mu }_{\left[W_k\right]}\right){\Psi }_i\right),\forall j=1:K,k=1:K,\hfill \end{array}$$

(24)

$$\begin{array}{cc}\hfill {\Sigma }_{Z_i}& \leftarrow {\left({\mathbb{E}}_{{\mathcal{Q}}_{{/}_{Z_i}}}[{\sigma }^{-2}W{\Psi }_i{W}^T+I]\right)}^{-1}={[{\displaystyle \frac{a_{\sigma }}{b_{\sigma }}}{H}_i+I]}^{-1},\hfill \end{array}$$

(25)

$$\begin{array}{cc}\hfill {\mu }_{Z_i}& \leftarrow {\mathbb{E}}_{{\mathcal{Q}}_{{/}_{Z_i}}}\left[{\sigma }^{-2}(Y_i-\overline{Z}{\Phi }_i){\Phi }_i^T{W}^T\right]{\Sigma }_{Z_i}={\displaystyle \frac{a_{\sigma }}{b_{\sigma }}}(Y_i-{\mu }_{\overline{Z}}{\Phi }_i){\Phi }_i^T{\left({\mu }_W\right)}^T{\Sigma }_{Z_i},\hfill \end{array}$$

(26)

where $H_i$ is a temporary variable denoting the Gram matrix of weighted kernel functions $W{\Phi }_i$, and ${\Sigma }_{[W_k,W_j]}$ denotes the covariance between $W_k^T$ and $W_j$ in $\mathcal{Q}$.

Updates for the parameters of the posterior of the noise ${\mathcal{Q}}_{\sigma }$:

$$\begin{array}{cc}\hfill a_{\sigma }& \leftarrow a_0+{\displaystyle \frac{1}{2}}\sum _i{N}_i,\hfill \end{array}$$

(27)

$$\begin{array}{cc}\hfill b_{\sigma }& \leftarrow b_0+{\displaystyle \frac{1}{2}}{\mathbb{E}}_{{\mathcal{Q}}_{/\sigma }}\left[\sum _i\left|\right|Y_i-(Z_{i}W+\overline{Z}){\Phi }_i{\left|\right|}_2^2\right]\hfill \\ \hfill & =b_0+{\displaystyle \frac{1}{2}}\sum _i(Y_i{Y}_i^T-2Y_i{\left({\mu }_{Z_i}{\mu }_W{\Phi }_i\right)}^T-2Y_i{\left({\mu }_{\overline{Z}}{\Phi }_i\right)}^T+2{\mu }_{Z_i}{\mu }_W{\Psi }_i{\left({\mu }_{\overline{Z}}\right)}^T\hfill \\ \hfill & +\mathrm{Tr}\left(\left({\Sigma }_{\overline{Z}}+{\left({\mu }_{\overline{Z}}\right)}^T{\mu }_{\overline{Z}}\right){\Psi }_i\right))+{\displaystyle \frac{1}{2}}{\mathrm{vec}\left(H^T\right)}^T\sum _i\mathrm{vec}\left(\mathrm{vec}\left({\Psi }_i\right)\mathrm{vec}({\Sigma }_{Z_i}+{{\mu }_{Z_i}}^T{\mu }_{Z_i}{)}^T\right),\hfill \end{array}$$

(28)

where H is a temporary variable that is updated by

$$\begin{array}{cc}\hfill H_{j+kM}& \leftarrow {\mathbb{E}}_{{\mathcal{Q}}_{/\sigma }}\left[\mathrm{vec}{\left(W_k{W_j}^T\right)}^T\right]=\mathrm{vec}{({\Sigma }_{[W_k,W_j]}+{\mu }_{\left[W_j\right]}^T{\mu }_{\left[W_k\right]})}^T,\forall j=1:K,k=1:K\hfill \end{array}$$

(29)

Nearly noninformative (vague) priors, i.e., with almost zero $a_0,b_0$, introduce an inherent identifiability ambiguity in our formulation—specifically, in the product of the precision parameters $\alpha ,\beta ,$ and $\eta$ (Equations (20) and (22)). In our model, scaling $\alpha$ and $\eta$ by a specific factor while inversely scaling $\beta$ leaves the product (and hence the lower bound in Equation (5) unchanged. This inherent ambiguity can lead $\alpha$, $\beta$, and $\eta$ to converge to extreme values, thereby challenging the numerical stability during optimization. To mitigate this issue, we adopt a heuristic constraint to ensure that the smallest values of $\alpha$ and $\beta$ remain within one order of magnitude of each other. Specifically, we enforce $\left|{\log }_{10}^{\left({\displaystyle \frac{\min \left(\alpha \right)}{\min \left(\beta \right)}}\right)}\right|\le 1$. If an update to any ${\alpha }_j$ or ${\beta }_k$ would violate this constraint, this particular update is skipped, and the rest of the parameters remain updated. This strategy does not alter the algorithm’s overall structure but stabilizes the optimization by preventing unnecessary flexibility in the precision parameters.

3.2. Scalable Update Strategy

The scalability of our algorithm so far is primarily challenged by the need to optimize the variational lower bound, $\mathcal{L}$, over K basis functions. As indicated by Equation (22), the time complexity is $\mathcal{O}\left(K^6\right)$ (or, alternatively, $\mathcal{O}\left(K^{2}P{\max }_i\left(N_i\right)\right)$, typically dominated by the former), which becomes prohibitive when K is large. In practice, however, only a small subset of these basis functions is necessary for an accurate representation—those with non-negligible weights under our sparse prior.

To address this, we focus the updates on the subspace of active basis functions, denoted as $K^{\left(a\right)}$, which comprises only those functions with non-negligible weights. The remaining basis functions, whose influence is minimal, are held fixed during optimization. Furthermore, the number of active principal components is noted as $J^{\left(\mathrm{a}\right)}$ and set equal to $K^{\left(a\right)}$, ensuring that the model spans the full range of possible ranks from 1 to $K^{\left(\mathrm{a}\right)}$. Consequently, we optimize ${\mathcal{Q}}^{\left(\mathrm{a}\right)}$ using updates derived with regard to the objective $K^{\left(a\right)}$-dimensional lower bound ${\mathcal{L}}^{\left(a\right)}$ as an efficient surrogate of the full updates of $\mathcal{Q}$ with regard to the full lower bound $\mathcal{L}$, using only $K^{\left(a\right)}$ active basis functions. Meanwhile, the active dimensionality of the model is adjusted dynamically during optimization by activating or deactivating basis functions based on their precision parameters. For clarity, variables associated with the active subspace are annotated with the superscript $\left(a\right)$ (e.g., $a_{{\alpha }_j}^{\left(\mathrm{a}\right)}=a_0+{\displaystyle \frac{K^{\left(\mathrm{a}\right)}}{2}}$ versus $a_{{\alpha }_j}=a_0+{\displaystyle \frac{K}{2}}$).

3.2.1. Implicit Factorization

For notational clarity, we reorder the rows and columns of our parameter matrices to separate active components from inactive ones. Specifically, we partition the matrices as follows:

$$\begin{array}{cc}\hfill Z_i& =\left(\begin{array}{cc}{Z}_{iA}& Z_{iB}\end{array}\right),\overline{Z}=\left(\begin{array}{cc}{\overline{Z}}_A& {\overline{Z}}_B\end{array}\right),\alpha =\left(\begin{array}{cc}{\alpha }_A& {\alpha }_B\end{array}\right),\beta =\left(\begin{array}{cc}{\beta }_A& {\beta }_B\end{array}\right),\hfill \\ \hfill W& =\left(\begin{array}{cc}{W}_A& W_B\\ W_C& W_D\end{array}\right),{\Phi }_i=\left(\begin{array}{c}{\Phi }_{iA}\\ {\Phi }_{iB},\end{array}\right)\hfill \end{array}$$

(30)

Here, the subscript A denotes variables belonging to the active subspace (i.e., those corresponding to $K^{\left(a\right)}$ basis functions), and $B,C,$ and D denote the inactive components. Notably, the cross terms $W_B$ and $W_C$ involve both active and inactive components; these are updated implicitly, as proven in the Appendix part.

Following the strategy in [39], a basis function is deemed inactive if its precision exceeds a high threshold, i.e., ${\alpha }_j>{ϵ}^{-1}$ and ${\beta }_k>{ϵ}^{-1}$ as $ϵ\to 0$. In the limit, the inactive basis functions decouple from the active ones, leading to the following mean field factorization:

$$\begin{array}{c}{\mathcal{Q}}_W ={\mathcal{Q}}_{W_A}{\mathcal{Q}}_{W_B}{\mathcal{Q}}_{W_C}{\mathcal{Q}}_{W_D}\end{array}$$

(31)

$$\begin{array}{c}{\mathcal{Q}}_{\overline{Z}} ={\mathcal{Q}}_{{\overline{Z}}_A}{\mathcal{Q}}_{{\overline{Z}}_B}\end{array}$$

(32)

$$\begin{array}{c}{\mathcal{Q}}_{Z_i} ={\mathcal{Q}}_{Z_{iA}}{\mathcal{Q}}_{Z_{iB}}\end{array}$$

(33)

The factorization of $\alpha$ and $\beta$ was already obtained in Equations (10) and (11). These factorizations allow us to decouple the update for the active subspace with the proof provided in the Appendix part.

It implies that only updates for ${\mathcal{Q}}_{Z_{iA}},{\mathcal{Q}}_{W_A},{\mathcal{Q}}_{W_B},{\mathcal{Q}}_{W_C},{\mathcal{Q}}_{{\overline{Z}}_A},{\mathcal{Q}}_{{\alpha }_A},{\mathcal{Q}}_{{\beta }_A},{\mathcal{Q}}_{\sigma },{\mathcal{Q}}_{\eta }$ are required, as shown in Figure 2. This strategy reduces the computational complexity from $\mathcal{O}\left(K^6\right)$ to $\mathcal{O}\left({K^{\left(a\right)}}^6\right)$. Moreover, we initialize W as an identity matrix and set the active ${\alpha }_A$ to all ones and the inactive ${\alpha }_B$ to infinite. In this way, we can initialize the remaining active dimensions $K^{\left(\mathrm{a}\right)}$, e.g., $Z_{iA}$, using a modified, multi-instance version of a relevance vector machine [30], which performs fast analytical maximum-likelihood updates, as detailed in Appendix E in the Appendix part.

3.2.2. Low-Dimensional Lower Bound

This section shows how to optimize these active surrogates, e.g., ${\mathcal{Q}}_{{\overline{Z}}_A}$, using updates of ${\mathcal{Q}}^{\left(\mathrm{a}\right)}$ with regard to the $K^{\left(a\right)}$-dimensional lower bound ${\mathcal{L}}^{\left(a\right)}$, which ultimately optimizes the full lower bound $\mathcal{L}$. To distinguish between the two, we denote the active surrogate posterior for the full model as ${\mathcal{Q}}_{{\overline{Z}}_A}$ and that for the reduced $K^{\left(a\right)}$-dimensional model as ${\mathcal{Q}}_{{\overline{Z}}_A}^{\left(\mathrm{a}\right)}$. The active Gaussian surrogate posteriors are shared, e.g., ${\mathcal{Q}}_{{\overline{Z}}_A}={\mathcal{Q}}_{{\overline{Z}}_A}^{\left(\mathrm{a}\right)}=\mathcal{N}\left({\overline{Z}}_A\right|{{\mu }_{\overline{Z}}}_A,{{\Sigma }_{\overline{Z}}}_A)$. This sharing implies that updating ${\mathcal{Q}}^{\left(\mathrm{a}\right)}$ is equivalent to updating $\mathcal{Q}$, so we set the moments of the active distributions of the full model to match those of the reduced model. However, the surrogate posterior Gamma distributions differ between the two models. For example, the update of ${\mathbb{E}}_{{\mathcal{Q}}^{\left(\mathrm{a}\right)}}\left[{\alpha }_A\right]$ depends solely on ${\mathcal{Q}}_{W_A}$, whereas ${\mathbb{E}}_{\mathcal{Q}}\left[{\alpha }_A\right]$ also incorporates a cross term ${\mathcal{Q}}_{W_B}$ corresponding to the remaining $(K-K^{\left(a\right)})$ dimensions. This difference is reflected in how the scale parameters depend on the number of active versus total basis functions, as shown in Equations (14), (16) and (18). Nonetheless, we prove that, in the limit $ϵ\to 0$, the fixed point of the $K^{\left(a\right)}$-dimensional updates of the complete surrogate $\mathcal{Q}$ equals that of the reduced surrogate ${\mathcal{Q}}^{\left(\mathrm{a}\right)}$. Consequently, the updates for ${\mathcal{Q}}_{{\alpha }_A},{\mathcal{Q}}_{{\beta }_A}$, and ${\mathcal{Q}}_{\eta }$ are derived directly from the expectations of the reduced model ${\mathcal{Q}}_{{\alpha }_A}^{\left(\mathrm{a}\right)},{\mathcal{Q}}_{{\beta }_A}^{\left(\mathrm{a}\right)},{\mathcal{Q}}_{\eta }^{\left(\mathrm{a}\right)}$:

$$\begin{array}{cc}\hfill {\mathbb{E}}_{\mathcal{Q}}\left[{\alpha }_A\right]\leftarrow {\mathbb{E}}_{{\mathcal{Q}}^{\left(\mathrm{a}\right)}}\left[{\alpha }_A\right]& \iff b_{{\alpha }_j}\leftarrow {\displaystyle \frac{a_{{\alpha }_j}}{a_{{\alpha }_j}^{\left(\mathrm{a}\right)}}}{b}_{{\alpha }_j}^{\left(\mathrm{a}\right)},\forall j\le J^{\left(\mathrm{a}\right)},\hfill \end{array}$$

(34)

$$\begin{array}{cc}\hfill {\mathbb{E}}_{\mathcal{Q}}\left[{\beta }_A\right]\leftarrow {\mathbb{E}}_{{\mathcal{Q}}^{\left(\mathrm{a}\right)}}\left[{\beta }_A\right]& \iff b_{{\beta }_k}\leftarrow {\displaystyle \frac{a_{{\beta }_k}}{b_{{\beta }_k}}}{b}_{{\beta }_k}^{\left(\mathrm{a}\right)},\forall k\le K^{\left(\mathrm{a}\right)},\hfill \end{array}$$

(35)

$$\begin{array}{cc}\hfill {\mathbb{E}}_{\mathcal{Q}}\left[\eta \right]\leftarrow {\mathbb{E}}_{{\mathcal{Q}}^{\left(\mathrm{a}\right)}}\left[\eta \right]& \iff b_{\eta }\leftarrow {\displaystyle \frac{b_{\eta }^{\left(\mathrm{a}\right)}}{a_{\eta }^{\left(\mathrm{a}\right)}}}{a}_{\eta }\hfill \end{array}$$

(36)

These update Equations (34)–(36) are proven to optimize $\mathcal{L}$ in Theorems A1 and A2 in the Appendix part.

3.2.3. Heuristic for Activation of Basis Functions

The proposed method selects a relatively small set of basis functions from a potentially extensive set of possibilities. The computational costs are mitigated by recognizing that inactive basis functions do not interact with those that are active (with non-negligible weights). The inactive components are removed in the final model and excluded from the active subspace optimization. However, our method essentially optimizes over the full space, and thus we have the algorithm allowing for the reactivation of inactive basis functions to ensure optimization across the entire subspace. Due to computational constraints, we consider functions for activation sequentially rather than all at once. Thus, we propose Algorithm 1 to introduce unseen basis functions into the active set using a selective strategy akin to the heuristic approach described in [39].

The algorithm selects the top function, ${\varphi }_{Bk}$, from the inactive basis functions ${\left\{{\varphi }_{Bk}\right\}}_k$ by gauging their correlations with residuals and applying an angle-based threshold ${\tau }_{\mathrm{ang}}$ relative to the subspace of ${\varphi }_A$. The correlation with residuals for ${\varphi }_{Bk}$ is measured by ${\sum }_i{\left({\Phi }_{iBk}{(Y_i-{\mathbb{E}}_{{\mathcal{Q}}^{\left(\mathrm{a}\right)}}[Z_{iA}{W}_A+{\overline{Z}}_A]{\Phi }_{iA})}^T\right)}^2$. The angle-based threshold ensures a meaningful distinction from active functions. Next, the current active surrogate posterior is expanded by a dimension for $\widetilde{{\varphi }_{Bk}}$, initiating optimization from the numerical maximum ${\tau }_{\max }$. Postoptimization, the function is retained if it falls below ${\tau }_{\max }$. Otherwise, the algorithm terminates. Efficiently, in trial optimization, the approach replaces one function with precision ${\tau }_{\max }$, if present.

Algorithm 1 Search for new basis functions to activate

Sort inactive basis functions ${\left\{{\varphi }_{Bk}\right\}}_k$ by correlation with residuals.    Filter through ${\left\{{\varphi }_{Bk}\right\}}_k$, selecting the most correlated one as ${{\varphi }_B}_k$.    Copy current active surrogate ${\mathcal{Q}}^{\left(\mathrm{a}\right)}(\Theta )$ posterior to ${{\mathcal{Q}}_k}^{{\left(a\right)}^{\prime }}(\Theta )$.    Expand dimension in ${{\mathcal{Q}}_k}^{{\left(a\right)}^{\prime }}(\Theta )$ for ${{\varphi }_B}_k$.    Optimize ${{\mathcal{Q}}_k}^{{\left(a\right)}^{\prime }}(\Theta )$ for three iterations using mean field approximation.    if expected precision is within threshold then          ${\mathcal{Q}}^{\left(\mathrm{a}\right)}(\Theta )\leftarrow {{\mathcal{Q}}_k}^{{\left(a\right)}^{\prime }}(\Theta )$.    end if

4. Faster Variant

To enhance the computational efficiency of our primary algorithm, we introduce a faster variant, denoted as ${\mathrm{BSFDA}}^{\mathrm{Fast}}$. This approach leverages conditional independence among the columns of W, enabling separate updates and thereby reducing the computational complexity. Similar strategies have been described in [18,28]. The model is defined with an introduced variable ${\zeta }_i$ for the coefficient noise as follows:

$${\theta }_i=Z_{i}W+{\zeta }_i,$$

(37)

$${\zeta }_{ik}\sim \mathcal{N}(0,{\varsigma }_k^2{\beta }_k^{-1}).$$

(38)

Similarly to the above, we assign a conjugate Gamma prior to the precision:

$${\varsigma }_k^{-2}\sim \Gamma (a_0,b_0).$$

(39)

This formulation ensures that the columns of W are conditionally independent, allowing the variational distribution to factorize as ${\mathcal{Q}}_{\mathcal{W}}={\prod }_k{\mathcal{Q}}_{W_{·k}}$, thereby facilitating separate updates for each column. Consequently, the time complexity is reduced from $\mathcal{O}\left({K^{\left(a\right)}}^6\right)$ to $\mathcal{O}\left({K^{\left(a\right)}}^3\right)$.

To align with the original model, it is necessary for $\zeta$ and the associated variance parameters $\varsigma$ to approach zero. Having $\varsigma$ too high would allow the coefficient noise to corrupt the signal, biasing the model toward underestimating the true signal levels, particularly because this noise operates in the coefficient space where it introduces smooth, correlated variation (low entropy, like signals), which is harder to eliminate than high-frequency white noise (maximum entropy). Injecting the same amount of noise leads to the unbiased estimation of the signals but increases the estimation variance. Conversely, as $\varsigma$ decreases, the columns of W become dependent, violating the independence assumption inherent in variational inference. This dependency degrades the approximation quality and slows down the optimization process. Such dependency issues are well documented in both the variational inference and MCMC literature—with recent efforts addressing them via structured VI [38] or blocked/collapsed Gibbs sampling [40]. Empirical validations of this noise impact are conducted with both ${\mathrm{BSFDA}}^{\mathrm{Fast}}$ in Section 5.1 and with Bayesian PCA [28] in Appendix F.2 in the Appendix part.

To balance the trade-off between the optimization speed and accuracy, we adopt a strategy of gradually decreasing the values of ${\varsigma }_k$ during the optimization iterations. Specifically, we initialize ${\varsigma }_k$ with a relatively large value and linearly decrease it from ${10}^{-2}$ to ${10}^{-5}$ over the first half of the iterations. After reaching ${10}^{-5}$, ${\varsigma }_k$ is fixed for the remaining iterations. This gradual reduction ensures that the algorithm initially maintains its efficiency, with benefits from minimizing the interdependency among the columns of W to accelerating convergence while later preserving the quality of the approximation by preventing the noise from obscuring the signal components. We unify the scales by scaling the basis functions so that $Z_i$ is standard normal and W is an identity matrix in initialization. Empirical evaluations indicate that the strategy above is effective in most applications.

By implementing these modifications, ${\mathrm{BSFDA}}^{\mathrm{Fast}}$ offers a practical solution that substantially accelerates the algorithm without a significant loss in accuracy, making it well suited for large-scale, high-dimensional functional data analysis.

5. Results

The proposed method demonstrates its effectiveness through simulations and applications to observed datasets.

5.1. Simulation Results

In the simulations, we evaluate the functional data analysis performance in terms of model selection, the estimated covariance accuracy, and extendability to multidimensional domains.

The model selection metric is the accuracy in estimating the number of principal components, which is the dimension of the compact subspace of signal variations. The configuration of the simulations in this section aligns with that established in [21], covering various scenarios. The simulated datasets are derived from a latent generative model with variables ${\mathcal{Z}}_i$ with dimension r for the i-th sample function and noise corruption with a standard deviation of $\sigma$: $Y_i={\sum }_{j=1}^r\left({\mathcal{Z}}_{ij}{f}_j\left(X_i\right)\right)+g\left(X_i\right)+E_i,{\mathcal{Z}}_{ij}\sim \mathcal{N}(0,v_j),E_j\sim \mathcal{N}(0,{\sigma }^{2}I),$ where ${\left\{f_j\right\}}_{j=1}^r$ represent eigenfunctions, ${\left\{v_j\right\}}_{j=1}^r$ are the eigenvalues, and $g:R\mapsto R$ signifies the mean function. Here, we consider five scenarios.

Scenario 1: Data generated with $g=5{(x-0.6)}^2$, $r=3$, $v=(0.6,0.3,0.1)$, ${\sigma }^2=0.2$, $f_1\left(x\right)=1,f_2\left(x\right)=\sqrt{2}\sin \left(2\pi x\right),f_3\left(x\right)=\sqrt{2}\cos \left(2\pi x\right).$ Here, $v_3<{\sigma }^2$, i.e., the noise has larger variance than the smallest signal.

Scenario 2: Similar to Scenario 1, but the third eigenfunction is replaced by a function with higher frequencies $f_3\left(x\right)=\sqrt{2}\cos \left(4\pi x\right)$, and the principal component scores follow a skewed Gaussian mixture model. Specifically, the j-th component score has a one-in-three probability of following a $\mathcal{N}(2\sqrt{v_j/3},v_j/3)$ distribution and a two-in-three probability of following $\mathcal{N}(-\sqrt{v_j/3},v_j)$, for $j=1,2,3$.

Scenario 3: Data generated with $g=12.5{(x-0.5)}^2-1.25$, $r=3$, $v=(4,2,1)$, ${\sigma }^2=0.5$, $f_1\left(x\right)=1,f_2\left(x\right)=\sqrt{2}\cos \left(2\pi x\right),f_3\left(x\right)=\sqrt{2}\sin \left(4\pi x\right).$

Scenario 4: Same as Scenario 3, but the component scores are generated from a Gaussian mixture model as in Scenario 2.

Scenario 5: Data from $g=12.5{(x-0.5)}^2-1.25$, $r=6$, $v=(4,3.5,3,2.5,2,1.5)$, ${\sigma }^2=0.5$, $f_1\left(x\right)=1,f_{2k}\left(x\right)=\sqrt{2}\sin \left(2k\pi x\right)\mathrm{for}k=1,2,3,f_{2k+1}\left(x\right)=\sqrt{2}\cos \left(2k\pi x\right)\mathrm{for}k=1,2$, j-th component score obeying $\mathcal{N}(0,v_j)$.

In each scenario, the simulations produce 200 sample functions. We investigate three cases with sparse, medium, and dense sampling by assigning the number of observations per sample function $N_i=\{5,10,50\}$. Each case in each scenario is repeated 200 times. The method’s performance is compared to that of fpca from [13], the AIC and BIC in the 2022 release of pace [11], the modified AIC and BIC in [21], and all the competing methods in [21]. For fpca, we set the candidate numbers of basis functions as [8,10,15,20] and the candidate dimensions of the process as [2,3,4,5] for Scenarios 1–4 and [4,5,6,7,8] for Scenario 5. The other parameters are all set to the defaults. Due to its consistent overestimation of the true number of components—likely resulting from interference by correlated noise and less sparse precision priors—we exclude LFRM [18] from further comparisons (see Appendix F.1.1 in the Appendix part).

Each estimation uses 10 length scales of functions, which are selected using cross-validation and k-means clustering. This adaptive strategy allows the algorithm to choose distinct length scales at different locations of the definition domain, thereby accommodating varying smoothness characteristics inherent in complex functional data—a level of flexibility that is not possible when using a regular grid that forces a single length scale across the entire domain [18]. Sparse sampling in Scenario 5 uses five length scales to avoid overfitting. Figure 3 shows the length scales and centers of the selected kernel basis functions for three different numbers of sample points, $N_i$, in a random repetition of Scenario 5. The results reveal that the selected length scales mainly concentrate around 0.07, with a few as high as 0.35—suggesting that the lower length scales capture finer, high-frequency variations. The higher length scales model the overall, lower-frequency quadratic mean structure and the constant baseline component. Furthermore, the estimated density functions of the selected length scales exhibit consistent patterns across the three sampling densities, and the method selects 9, 11, and 12 basis functions, respectively, demonstrating the algorithm’s adaptive fidelity and complexity based on the available observations. The Appendix part showcases the uncertainty evaluation in Figure A2.

Table 1. Proportion of accurate estimations for Scenario 1 (r = 3).

${\mathit{N}}_{\mathit{i}}$	${\mathrm{AIC}}_{\mathrm{PACE}}$	AIC	BIC	${\mathbf{PC}}_{\mathit{p}1}$	${\mathbf{IC}}_{\mathit{p}1}$	${\mathbf{AIC}}_{\mathrm{PACE}}^{2022}$	${\mathbf{BIC}}_{\mathrm{PACE}}^{2022}$	fpca	BSFDA	${\mathrm{BSFDA}}^{\mathrm{Fast}}$
5	0.000	0.580	0.380	0.410	0.735	0.650	0.880	0.645	0.995	0.015
10	0.000	0.980	0.670	0.955	0.985	0.880	0.920	0.645	1.000	0.910
50	0.000	1.000	0.830	1.000	1.000	1.000	1.000	0.890	0.980	0.945

,

Table 2. Proportion of accurate estimations for Scenario 2 (r = 3).

${\mathit{N}}_{\mathit{i}}$	${\mathbf{AIC}}_{\mathbf{PACE}}$	AIC	BIC	${\mathbf{PC}}_{\mathit{p}1}$	${\mathbf{IC}}_{\mathit{p}1}$	${\mathbf{AIC}}_{\mathbf{PACE}}^{2022}$	${\mathbf{BIC}}_{\mathbf{PACE}}^{2022}$	fpca	BSFDA	${\mathbf{BSFDA}}^{\mathbf{Fast}}$
5	0.005	0.630	0.245	0.375	0.605	0.570	0.620	0.475	1.000	0.040
10	0.000	0.710	0.665	0.570	0.805	0.825	0.850	0.640	1.000	0.995
50	0.000	0.630	0.795	0.955	0.945	1.000	1.000	0.950	1.000	0.950

,

Table 3. Proportion of accurate estimations for Scenario 3 (r = 3).

${\mathit{N}}_{\mathit{i}}$	${\mathbf{AIC}}_{\mathbf{PACE}}$	AIC	BIC	${\mathbf{PC}}_{\mathit{p}1}$	${\mathbf{IC}}_{\mathit{p}1}$	${\mathbf{AIC}}_{\mathbf{PACE}}^{2022}$	${\mathbf{BIC}}_{\mathbf{PACE}}^{2022}$	fpca	BSFDA	${\mathbf{BSFDA}}^{\mathbf{Fast}}$
5	0.005	0.720	0.325	0.640	0.590	0.320	0.400	0.450	0.995	0.945
10	0.000	0.580	0.770	0.965	0.665	0.740	0.755	0.440	0.995	1.000
50	0.000	1.000	0.775	1.000	1.000	1.000	1.000	0.765	0.980	0.920

,

Table 4. Proportion of accurate estimations for Scenario 4 (r = 3).

${\mathit{N}}_{\mathit{i}}$	${\mathbf{AIC}}_{\mathbf{PACE}}$	AIC	BIC	${\mathbf{PC}}_{\mathit{p}1}$	${\mathbf{IC}}_{\mathit{p}1}$	${\mathbf{AIC}}_{\mathbf{PACE}}^{2022}$	${\mathbf{BIC}}_{\mathbf{PACE}}^{2022}$	fpca	BSFDA	${\mathbf{BSFDA}}^{\mathbf{Fast}}$
5	0.015	0.710	0.410	0.640	0.560	0.515	0.575	0.370	1.000	0.975
10	0.000	0.830	0.775	0.920	0.900	0.750	0.760	0.350	0.995	0.990
50	0.000	0.945	0.835	1.000	1.000	1.000	1.000	0.730	0.950	0.935

and

Table 5. Proportion of accurate estimations for Scenario 5 (r = 6).

${\mathit{N}}_{\mathit{i}}$	${\mathbf{AIC}}_{\mathbf{PACE}}$	AIC	BIC	${\mathbf{PC}}_{\mathit{p}1}$	${\mathbf{IC}}_{\mathit{p}1}$	${\mathbf{AIC}}_{\mathbf{PACE}}^{2022}$	${\mathbf{BIC}}_{\mathbf{PACE}}^{2022}$	fpca	BSFDA	${\mathbf{BSFDA}}^{\mathbf{Fast}}$
5	0.705	0.470	0.090	0.070	0.545	0.425	0.410	0.855	0.925	0.160
10	0.065	0.570	0.525	0.775	0.705	0.575	0.575	0.500	1.000	0.930
50	0.000	0.260	0.590	0.980	0.965	0.870	0.770	0.695	0.995	0.925

show the results. The results for the first five methods are from [21]. Of 15 cases, the proposed BSFDA exhibits the highest accuracy in 12. In the other three cases, the accuracy of BSFDA is comparable to the best result and is always above 0.950. ${\mathrm{BSFDA}}^{\mathrm{Fast}}$ demonstrates performance comparable to that of BSFDA when applied to medium-density and dense datasets with significantly higher efficiency, which we detail in Figure 4. However, its efficacy diminishes with sparse data. This limitation arises because the parameter $\varsigma$ can bias model estimation in scenarios with insufficient data evidence, leading to an underestimation of the signal variance. Consequently, ${\mathrm{BSFDA}}^{\mathrm{Fast}}$ tends to underestimate the number of components, particularly those capturing nuanced variations, in the presence of sparse observations. Nonetheless, with adequate data, ${\mathrm{BSFDA}}^{\mathrm{Fast}}$ achieves performance on par with that of the original model.

5.1.1. Mean Squared Error in Covariance Operator

The mean squared error across $X_{\mathrm{grid}}$, a grid of 1000 index points,

$${\displaystyle \frac{\left|\right|\mathrm{cov}(X_{\mathrm{grid}},X_{\mathrm{grid}})-\widehat{\mathrm{cov}}(X_{\mathrm{grid}},X_{\mathrm{grid}}){\left|\right|}_F^2}{1000\times 1000}},$$

where $\left|\right|·{\left|\right|}_F$ is the Frobenius norm, measures the accuracy of the estimated covariance. The quadratic measure of the error with the Frobenius norm for covariance estimators has been used by [41]. The methods compared include fpca of [13], pace of [11] with the AIC and BIC, and refund-sc of [12]. Only cases in Scenario 5 are used because of the time constraints (e.g., refund-sc takes 6 h for 20 repetitions with 50 points in Scenario 5). As the most challenging, Scenario 5 should provide the most compelling comparison. The results in

Table 6. Mean squared error of covariance ${\mathrm{Error}}_{\mathrm{CovFunc}}$ for Scenario 5.

${\mathit{N}}_{\mathit{i}}$	${\mathbf{AIC}}_{\mathbf{PACE}}^{2022}$	${\mathbf{BIC}}_{\mathbf{PACE}}^{2022}$	fpca	refund.sc	BSFDA	${\mathbf{BSFDA}}^{\mathbf{Fast}}$
5	12.373 ± 4.026	12.377 ± 4.031	5.192 ± 6.166	8.833 ± 4.730	5.814 ± 3.535	10.292 ± 12.717
10	10.391 ± 2.521	10.391 ± 2.521	2.098 ± 1.425	5.314 ± 3.501	2.068 ± 1.427	2.656 ± 1.712
50	9.054 ± 1.683	9.054 ± 1.683	1.642 ± 1.240	N/A	1.638 ± 1.247	1.770 ± 1.275

demonstrate that the proposed method is comparable to the best work in terms of the estimated covariance accuracy. Specifically, dense sampling becomes prohibitive for refund-sc. The results highlight the benefit of continuous formulations, as seen in both fpca and the proposed method, over the grid-based optimization in conventional methods. ${\mathrm{BSFDA}}^{\mathrm{Fast}}$ again performs comparably well when the data are adequate.

5.1.2. Multidimensional Functional Data Simulation

A simulation experiment with a 4D index set reveals the proposed method’s advantages for high-dimensional data, where the gridding strategies of previous methods are impractical. The settings are as follows, with a length scale $l_{\mathrm{s}}=0.33$:

$$\begin{array}{cc}\hfill Z_i& \sim \mathcal{N}(0,I)\in R^{1\times 3}\hfill \end{array}$$

(40)

$$\begin{array}{cc}\hfill {\varphi }_0\left(x\right)& ={\left(\pi l_{\mathrm{s}}^2\right)}^{-2}\exp \left(-{\displaystyle \frac{1}{2}}{∥{\displaystyle \frac{x-[0.5,0.5,0.5,0.5]}{l_{\mathrm{s}}}}∥}_2^2\right)\hfill \end{array}$$

(41)

$$\begin{array}{cc}\hfill {\varphi }_1\left(x\right)& ={\left(\pi l_{\mathrm{s}}^2\right)}^{-2}\exp \left(-{\displaystyle \frac{1}{2}}{∥{\displaystyle \frac{x-[0.4,0.4,0.4,0.4]}{l_{\mathrm{s}}}}∥}_2^2\right)\hfill \end{array}$$

(42)

$$\begin{array}{cc}\hfill {\varphi }_2\left(x\right)& ={\left(\pi l_{\mathrm{s}}^2\right)}^{-2}\exp \left(-{\displaystyle \frac{1}{2}}{∥{\displaystyle \frac{x-[0.6,0.6,0.6,0.6]}{l_{\mathrm{s}}}}∥}_2^2\right)\hfill \end{array}$$

(43)

$$\begin{array}{cc}\hfill y_i\left(x\right)& =Z_{i0}\ast \sqrt{0.6}\ast ({\varphi }_0\left(x\right)-{\varphi }_1\left(x\right))+Z_{i1}\ast \sqrt{0.3}\ast {\varphi }_1\left(x\right)+Z_{i2}\ast \sqrt{0.4}\ast {\varphi }_2\left(x\right)\hfill \end{array}$$

(44)

The observations include additive noise with a sigma of 4.472 × 10⁻¹. The cross-validation selects a length scale of 0.405. The estimated noise sigma is 4.637 × 10⁻¹. The proposed method correctly estimates the number of principal components as three and selects 31 basis functions. As shown in Figure 5, the eigenfunctions are correctly estimated. In addition, the estimated mean function is zero, which is accurate.

Next, we present a convergence comparison between BSFDA and ${\mathrm{BSFDA}}^{\mathrm{Fast}}$ under four schedules for the coefficient noise ${\varsigma }_k$. Specifically, we compare the default diminishing schedule from ${10}^{-2}$ to ${10}^{-5}$ with three fixed settings: ${10}^{-2}$, ${10}^{-3}$, and ${10}^{-5}$. We evaluate the covariance error and the discrepancy between the estimated/true dimensionality in one replicate of each sample density in Scenario 5 and the 4D simulation. For the 4D case, we adopt a default initial ${\varsigma }_k$ of ${10}^{-3}$. As illustrated in Figure 4, ${\mathrm{BSFDA}}^{\mathrm{Fast}}$ achieves comparable accuracy to BSFDA while converging significantly faster than BSFDA in terms of both covariance errors and component estimation for medium and densely sampled data. In the 4D case, ${\mathrm{BSFDA}}^{\mathrm{Fast}}$ converges in covariance estimation after approximately 10,000 s and in dimensionality after around 4000 s, compared to roughly 100,000 s and 13,000 s, respectively, for BSFDA. However, for sparse data, ${\mathrm{BSFDA}}^{\mathrm{Fast}}$ exhibits reduced estimation accuracy and underestimates the number of components by one. A similar decline in accuracy is observed in the 4D simulation when data sparsity is high. This limitation arises because the introduction of coefficient noise $\varsigma$ biases the model toward eliminating signals that are deemed insignificant. Moreover, when comparing the three fixed-${\varsigma }_k$ variants of the fast algorithm, a clear trade-off emerges: a smaller ${\varsigma }_k$ reduces the overall error but slows down the optimization due to increased dependency among variables. These results collectively demonstrate the effectiveness of our chosen ${\varsigma }_k$ schedule in ${\mathrm{BSFDA}}^{\mathrm{Fast}}$, as it balances both efficiency and accuracy.

5.2. Results on Public Datasets

The proposed method’s practicality was validated with two application datasets, CD4 and wind speed measurements.

5.2.1. CD4

CD4 data, a classical form of functional data, have received attention in [1,11,13]. CD4 cell counts gauge the immune system’s response to human immunodeficiency virus (HIV) infection, which leads to a progressive reduction in CD4 cell counts. The Multicenter AIDS Cohort Study (MACS) [42] provided the CD4 data. This dataset consists of CD4 percentages from 283 male human subjects that were HIV-positive, each with 1 to 14 repeated measurements over time in years. Subjects were scheduled for reevaluation at least semiannually. However, missed visits caused the sparse and uneven distribution of measurements. The proposed method used five length scales selected from cross-validation and k-means clustering. Finally, the model selected nine basis functions. Figure 6 displays the estimated mean function, eigenfunctions, and curves of the observations. The mean function reflects the overall decreasing tendency with the progression of the disease. The eigenfunctions are obtained by applying the singular value decomposition of the covariance operator that is discretized (for visualization purposes only) with a grid of 50 evenly spaced points over the whole timeline. The first eigenfunction is relatively flat and mainly captures the subject-specific average magnitude of the CD4 counts, consistent with the findings of [1,11,13]. The second eigenfunction captures the simple linear trend of the variations, as described in [13]. The third eigenfunction captures the piece-wise linear time trend with a breakpoint near 2.5 years from the baseline. Refs. [1,11] found similar eigenfunctions.

5.2.2. Wind Speed

The wind speed data, collected from 110 locations across Utah’s Salt Lake Valley, range between 11 and 1440 measurements. The proposed method leverages 10 length scales selected from cross-validation and k-means clustering. Figure 7 illustrates the estimated mean function, curves of the observations, eigenfunctions, and covariance. The horizontal axis represents the seconds starting from 12:00 AM Greenwich Mean Time (GMT) on 15 June 2023, which corresponds to 6:00 PM in Salt Lake City. In Figure 7a, the estimated mean function depicts two pronounced peaks observed approximately at 8:00 PM and 6:00 AM, as well as two troughs around 12:00 AM and 12:00 PM. This pattern aligns with the diurnal cycle, particularly highlighting the thermal activity associated with sunset and sunrise. The peaks during sunset and sunrise are due to the interplay of topographical features, which result in specific breezes, such as the land breeze near the Great Salt Lake and the distinct mountain and valley breezes. The troughs, on the other hand, reflect moments when the atmosphere is at its most stable, with minimal thermal activity disrupting wind patterns. The complexity of the data is distilled and represented using 12 descriptors with 17 basis functions. As Figure 7b shows, the primary eigenfunction is relatively level, indicating that the most significant variation is the location-specific average magnitude. Its profile echoes the influence of sunrise and sunset observed in the mean function, with elevations around 7:00 PM and 5:00 AM and subdued patterns during other times, indicative of similar atmospheric stability. The estimated covariance in Figure 7c highlights variance peaks around 8 PM and 5 AM, as well as a strong correlation between these periods. This underscores the effects of location-specific topographic factors on the wind speed.

5.2.3. Modeling Large-Scale, Dynamic, Geospatial Data

Here, we demonstrate the scalability regarding both the size of the measurements and the dimensionality of our framework. For this, we apply it to the ARGO dataset, which consists of ocean temperature measurements from more than 4000 locations, at multiple depths and time points [37]. ARGO is a nearly global observing system for the ocean temperature, salinity, and other key variables via autonomous profiling floats. As of 2019, ARGO has generated over 338 gigabytes of data from 15,231 floats [37]. We focused on high-quality (“research” mode option in the database API) data from 1998 to 2024 for depths between 0 and 200 m in the open-access snapshot of Argo GDAC of 9 November 2024 [43]. The number of measurement points per year varies widely—from 38,931 to over 11 million, with 127 million in total. Figure 8 illustrates a global map of the sea surface temperature measurements from February 2021, highlighting the dataset’s extensive spatial coverage.  

In our modeling, each year’s data are treated as a single underlying function of four variables: latitude, longitude (on the spherical Earth), depth, and intra-annual time (modeled as a periodic variable). Note that the spatial data lie on a sphere and the time is a circle, assuming the periodicity of the time of the year. Our approach models these measurements holistically—without resorting to moving windows or submodeling—thereby preserving the continuous nature of the data and enabling the extraction of meaningful global, seasonal, and depth-dependent trends. Furthermore, the unique geospatial and temporal structure of the ARGO data, with spatial coordinates on a sphere and time exhibiting periodicity, necessitates specialized modeling techniques. Given that our model is 4D, the 4D kernel is defined as a product of the following kernels, following the design strategy for climatological data in [10]. The geospatial kernel on the sphere is a radial basis function (RBF) on geodesic distances. To ensure periodicity, the temporal kernel is an Exp-Sine-Squared $k(x,x^{\prime })=\exp \left(-{\displaystyle \frac{2{\sin }^2\left(\pi |x-x^{\prime }|\right)}{l_{\mathrm{s}}}}\right)$, where $l_{\mathrm{s}}$ is the length scale. For depth, we use a Gaussian kernel.

The numeric data (excluding metadata) as input to the model were approximately 4 GB. For length scale selection, we used Gaussian process regression on a small subset of 2000 randomly selected data in 2016 (medium size of measurements) for a cross-validated RMSE, which we optimized with a grid search. The specific length scales were set as follows: geodesic length scale of $2\times {10}^3$ km, depth length scale of 70 m, time length scale of 3, and periodicity of 1. For evaluation, we held out 10% of the depth profiles (a single round trip of a buoy from the surface to a depth at the same coordinate) from each year as testing data, following [44]. The total training set contained roughly 114 million points. Because the sample spacing was typically small relative to the selected length scales, we applied agglomerative clustering to 10,000 randomly chosen index points, reducing them to 2000 candidate basis functions. These candidate basis functions—precomputed for efficiency—took roughly 1.7 TB of memory. Computations were performed with 24 threads on a server equipped with 192 Intel^® Xeon^® Platinum 8360H CPUs @ 3.00 GHz and 3 TB RAM. Initialization was conducted using the modified RVM for 200 iterations for initial basis functions, using stochastic optimization with a 1000 batch size per year. Then, ${\mathrm{BSFDA}}^{\mathrm{Fast}}$ was executed for 10,000 iterations, where the heuristic to include new bases also used a 1000 batch size per year. With these computational strategies and heuristics, the entire modeling process was completed in 15.

The proposed approach selected 163 effective basis functions and condensed them into 16 principle components. The final model occupied merely 50 MB of storage. The interpolation yielded a root mean square error (RMSE) of 1.95 and an $R^2$ of 94.2% on the testing data, reflecting a reasonable balance between global dimension reduction and fidelity. The estimated white noise level was also 1.95, indicating that the training data adequately covered the underlying variability in the ARGO observations, and the final model was reasonably generalizable.

Figure 9 presents 2D visualizations of geospatial interpolations at three depths (in decibars, roughly meters) and a specific time (29 May 2021) around 1° S and 30° W, each with three views. We have chosen one measurement as the central point, denoted by the red circle, and selected a narrow window (±1 decibar, ±1 day) around this center. The cyan and fuchsia circles represent training and testing data, respectively, within this window. Their sizes indicate the distance along the unplotted dimensions (depth and time here), reflecting variations in these dimensions. The visualizations show that the temperatures are warmer near the equator and decrease with depth. The match between the interpolated values and actual measurements demonstrates consistency in capturing broad spatial and vertical variations.

Figure 10 complements this by illustrating interpolation in the depth–time slices while holding the geospatial coordinates fixed, focusing on mixed layer characteristics. The “mixed layer” refers to a region of nearly uniform temperature, which is crucial in understanding thermodynamic potential and nutrient cycling [45]. Here, the plot uses a window of 50 km to include actual measurements, and the circle sizes denote the geodesic distance from the chosen center.

We plot every fifth measurement vertically to reduce overlap and improve the clarity. Figure 10a uses the same center point, 1° S and 30° W, as in Figure 9, exhibiting a shallow mixed layer with pronounced vertical gradients. In contrast, Figure 10b adopts a center at a higher latitude, 49° N and 29° W, where the model reveals a deeper mixed layer. The temperature here remains relatively stable below the surface. The dominant variations are cyclic seasonal changes, which are warmer near the surface around September. As is shown, the vertical sequence of the center and the nearby testing sequence match the interpolation closely. These results confirm that the mid-latitudes exhibit a stronger seasonal cycle [45] and that ${\mathrm{BSFDA}}^{\mathrm{Fast}}$ accurately approximates the actual measurements.

To our knowledge, this is the first time that the ARGO dataset has been modeled in a full 4D principal component model, with the correct domain topology. We incorporate the entire period of 27 years, rather than shorter spans (e.g., 2004–2008 or 2007–2016) [44,46,47]. Instead of segmenting the dataset into localized spatiotemporal windows, we process the entire 4D domain (latitude, longitude, depth, and intra-annual time) in a single holistic framework. Previous studies were typically tailored to ARGO datasets and handled each depth, month, or spatial region separately, restricting the correlation estimates to limited windows (e.g., 1000 km and three months) while excluding data with large offsets [44,47]. In addition, they required repeated on-demand model fitting, which can hinder scalability. Our kernel-based framework, by contrast, is broadly applicable to general functional data, only requiring kernel definitions for the domain. Although global dimension reduction inevitably introduces some residual noise, the kernel-based design is extensible to finer spacing or multiple length scales if higher precision is needed. Furthermore, inference with our model is simply the evaluation of the active 163 active basis functions weighted by the 16 principal components. Interpolation over a 300 × 300 grid only takes about two seconds. By contrast, previous methods with Gaussian process regression require a weighted sum of all measured data within a certain window. The parametric representations also facilitate straightforward derivative and integration calculations, which are essential in investigating ocean temperature stratification and heat content [44]. In summary, the ARGO dataset provides an ideal testbed for our method, as it captures the dynamic behavior of high-dimensional geospatial data in a continuous framework. A more comprehensive study of ARGO was beyond this study’s scope. Nonetheless, the results here confirm the clear advantages of the proposed method for large-scale, high-dimensional functional data.

6. Discussion and Conclusions

This paper proposes BSFDA, a novel framework for functional data analysis with irregular sampling, integrating model selection and scalability in one unique, coherent, and effective algorithm. Our extensive empirical studies, including both simulations and real-world applications, show that BSFDA offers superior covariance estimation accuracy with remarkable efficiency.

In terms of accuracy, our method excels in model selection, consistently achieving top-tier performance. The accuracy of the covariance operator estimation also rivals that of the best existing methodologies in the field. This shows that our approach can not only handle large and complex datasets but also ensure high accuracy and precision in the results that it produces. Our method’s superiority compared to existing techniques is expected owing to the inherent iterative nature of data smoothing and covariance estimation in our approach.

In terms of scalability, our method demonstrates linear growth in time complexity with the size of the dataset, and, impressively, the computations are executed in a small, $K^{\left(a\right)}$-dimensional subspace. This property ensures that, as the datasets grow larger and more complex, the performance of our model remains robust and efficient. Additionally, we introduce a faster variant, ${\mathrm{BSFDA}}^{\mathrm{Fast}}$, which performs similarly to BSFDA on medium and dense datasets with a significantly reduced computational cost. This leap in efficiency enabled the full 4D functional modeling, for the first time, of a large-scale oceanic temperature dataset across 27 years (ARGO) [37]. Although ${\mathrm{BSFDA}}^{\mathrm{Fast}}$ can underestimate the signal strength under very sparse sampling, the vanilla BSFDA effectively complements and alleviates this issue.

Although the proposed framework proves effective in various real-world scenarios, it relies on the specific scheduling of coefficient noise levels. This schedule transitions from a faster, more biased model to a slower, less biased one, balancing the convergence speed against the estimation accuracy. Whereas empirical tests validate its advantages, there remain exploratory directions for further enhancement. For instance, the proper incorporation of structured variational inference or injection of artificial, compensatory noise in the observations could enable fast inference with reduced bias at the same time. However, it is necessary to address the increased complexity in optimization and variance in the estimation. Additionally, variational inference based on mean field approximations may underestimate the posterior variance, which is acceptable in many tasks [38] but leaves an open question when the independence assumptions are severely violated. Moreover, variational inference prioritizes computational efficiency over strict theoretical optimality [23,38], and the inherent infinite-dimensionality of fPCA further complicates any formal asymptotic analysis, even though our empirical studies demonstrate decreasing errors as the sampling density increases.

Looking ahead, it would be interesting to explore how extensions of regular PCA, such as simplified PCA and robust PCA [25], can be integrated within our proposed framework. Domains such as finance, in particular, include large-time-series datasets that often contain more outliers. These extensions will enhance the flexibility and robustness of our method, further improving its adaptability to various data conditions. In addition, we see potential in examining the extensions of functional PCA, such as time warping, dynamics, and manifold learning [1]. In particular, shape analysis emerges as a direct application of time warping. Such extensions would push the boundaries of what our proposed method could achieve, potentially enabling it to handle an even wider array of data structures and complexities.

In conclusion, our research findings affirm the proposed framework’s effectiveness and adaptability in advanced functional data analysis. Nonetheless, the method’s potential remains broad, and future work promises to widen its scope and refine its performance. By unifying sparse Bayesian learning, kernel-based expansions, and efficient variational inference, BSFDA offers a powerful foundation for large-scale, high-dimensional FDA challenges.