Multivariate Uncertainty Quantification with Tomographic Quantile Forests

Abstract

Quantifying predictive uncertainty is essential for safe and trustworthy real-world AI deployment. However, the fully nonparametric estimation of conditional distributions remains challenging for multivariate targets. We propose Tomographic Quantile Forests (TQF), a nonparametric, uncertainty-aware, tree-based regression model for multivariate targets. TQF learns conditional quantiles of directional projections ${\mathbf{n}}^{\top }\mathbf{y}$ as functions of the input $\mathbf{x}$ and the direction n. At inference, it aggregates quantiles across many directions and reconstructs the multivariate conditional distribution by minimizing the sliced Wasserstein distance via an efficient alternating scheme with convex subproblems. Unlike classical directional-quantile approaches that typically produce only convex quantile regions and require training separate models for different directions, TQF covers all directions with a single model to reconstruct the full conditional distribution itself, naturally overcoming any convexity restrictions. We evaluate TQF on synthetic and real-world datasets, and release the source code on GitHub.

1. Introduction

In machine learning, probabilistic prediction aims to estimate an output distribution rather than a single point estimate, thereby enabling the quantification of predictive uncertainty [1,2,3,4,5,6,7,8,9,10]. Such approaches are essential in high-stakes domains, including medical treatment, autonomous driving, and financial risk assessment, where understanding decision risk is critical. Uncertainty estimation is also central to Bayesian experimental design, in which one seeks to maximize information gain from new observations under cost constraints [11]. Deep neural networks (NNs) have driven substantial progress in many of these areas, largely due to their ability to handle high-dimensional data effectively. However, in the setting of tabular data, whether NNs consistently outperform traditional machine learning models remains actively debated [12,13,14,15,16,17,18]. In practice, gradient boosting machines [19,20] often perform comparably to, or even better than, deep learning approaches while being more computationally efficient. This motivates the development of uncertainty quantification methods based on non-NN models.

For regression with a univariate target $y\in \mathbb{R}$, predictive uncertainty is commonly summarized using confidence intervals or quantiles, or modeled using parametric distributions. In contrast, uncertainty quantification for a multivariate target $\mathbf{y}\in {\mathbb{R}}^d$ is considerably more challenging. A naïve approach is to model each component independently, but the resulting predictive distribution fails to capture dependencies among components; in general, $p\left(\mathbf{y}\right|\mathbf{x})\ne {\prod }_{i=1}^{d}p\left(y_i\right|\mathbf{x})$. A didactic example is shown in Figure 1: the three datasets have clearly different joint distributions $p(y_1,y_2)$, yet their marginal distributions $p\left(y_1\right)$ and $p\left(y_2\right)$ coincide exactly. Ignoring such dependence can severely undermine reliability in downstream decision-making. Although conventional models assuming a Gaussian predictive distribution extend readily to multivariate targets, they inherently fail to capture multi-modality and heavy tails. More flexible parametric models such as Gaussian mixtures are often difficult to fit reliably and may converge to poor local optima. Recent work has therefore focused on NN-based approaches that address these limitations [21,22,23,24,25,26,27]. By comparison, multivariate uncertainty quantification with non-NN models has received less attention.

To bridge this gap, we introduce a non-NN method for learning nonparametric, potentially multimodal predictive distributions in tabular-data settings. Our approach, a tomographic multivariate uncertainty quantification method termed Tomographic Quantile Forests (TQF), first produces probabilistic predictions for one-dimensional projections of the target, $p\left({\mathbf{n}}^{\top }\mathbf{y}\right|\mathbf{x})$, over directions $\mathbf{n}\in {\mathbb{S}}^{d-1}:=\{\mathbf{x}\in {\mathbb{R}}^d\mid {\parallel \mathbf{x}\parallel }_2=1\}$. It then aggregates these directional predictions to construct a multivariate predictive distribution $p\left(\mathbf{y}\right|\mathbf{x})$, rather than estimating geometric quantile regions. TQF is motivated by the Cramér–Wold theorem [28], which states that a probability measure on ${\mathbb{R}}^d$ is uniquely determined by its one-dimensional projections. While TQF can in principle use any probabilistic model for univariate targets as a backbone, we find it particularly effective to build on a variant of Quantile Regression Forests [29], given their strong performance and favorable computational cost on large tabular datasets. We further show that TQF approximately minimizes the sliced 1-Wasserstein distance between the true conditional distribution and the predicted distribution. Finally, we validate TQF empirically on multiple benchmark problems and observe favorable performance. The source code for this paper is publicly available at GitHub (https://github.com/TaTKSM/TQF, accessed on 1 March 2026).

The remainder of this paper is organized as follows. Section 2 reviews prior work on uncertainty quantification in machine learning. Section 3 introduces a modified forest approach, QRF++, and evaluates it numerically. Section 4 presents the proposed method, TQF, and its core component, the Quantile-Matching Empirical Measure (QMEM) algorithm, which reconstructs a probability distribution from directional quantile data. Section 5 specifically evaluates the standalone performance of the QMEM algorithm. Section 6 reports numerical results for TQF on synthetic and real-world datasets to empirically validate performance. Finally, Section 7 presents conclusions. Appendix A summarizes technical background on distributional distance metrics, the Radon transform, and proper scoring rules.

2. Related Work

Table 1. An inexhaustive list of statistical and machine-learning models for distributional prediction with univariate or multivariate target variables. The proposed method is listed in the bottom row. The column “Nonparametric” indicates whether a method can predict arbitrary (i.e., nonparametric) probability distributions; it does not refer to whether the input–output relationship is modeled nonparametrically.

	Admits $\mathbf{dim}\mathit{y}>1$	Nonpara- Metric	Use NNs	Use Decision Trees	Limited to Convex Predictive Regions	Curse of Dimensionality When $\mathbf{dim}\mathit{x}$ Is High
Rasmussen & Williams 2005 [30]	Yes	No	No	No	Yes	No
Meinshausen 2006 [29]	No	Yes	No	Yes	No	No
Sugiyama et al., 2010 [31]	Yes	Yes	No	No	No	Yes
Hallin et al., 2010 [32]	Yes	Yes	No	No	Yes	No
Paindaveine & Šiman 2011 [33]	Yes	Yes	No	No	Yes	No
Kong & Mizera 2012 [34]	Yes	Yes	No	No	Yes	No
Bouchacourt et al., 2016 [21]	Yes	Yes	Yes	No	No	No
Schlosser et al., 2019 [35]	No	No	No	Yes	No	No
Athey et al., 2019 [36]	No	Yes	No	Yes	No	No
Duan et al., 2020 [37] and O’Malley et al., 2021 [38]	Yes	No	No	Yes	No	No
Hothorn & Zeileis 2017, 2021 [39,40]	No	No	No	Yes	No	No
Du et al., 2021 [41]	Yes	Yes	No	Yes	No	No
März 2019 [42], März & Kneib 2022 [43], and März 2022 [44]	Yes	Yes	No	Yes	No	No
Russell & Reale [22]	Yes	No	Yes	No	Yes	No
Kanazawa & Gupta 2022 [24]	Yes	Yes	Yes	No	No	No
Kan et al., 2022 [23]	Yes	Yes	Yes	No	No	No
Vedula et al., 2023 [25]	Yes	Yes	Yes	No	No	No
Cevid et al., 2022 [45] and Näf et al., 2023 [46]	Yes	Yes	No	Yes	No	No
Feldman et al., 2023 [26]	Yes	Yes	Yes	No	No	No
Chen & Müller 2023 [47]	Yes	Yes	No	No	No	Yes
Matsubara 2024 [48]	No	Yes	No	Yes	No	No
Barrio et al., 2024 [49]	Yes	Yes	No	No	No	Yes
This Work	Yes	Yes	No	Yes	No	No

summarizes prior work on probabilistic prediction models, with an emphasis on non-neural approaches. We briefly review these methods in the following subsections. Readers interested in a more comprehensive overview of uncertainty quantification are referred to the survey articles in Refs. [1,3,4,6,7,8,9,10].

2.1. Univariate Target Variable

Quantile regression [50] and expectile regression [51] have been among the most widely used methods in statistics for the evaluation of predictive errors and uncertainty. Since no parametric form of noise is assumed, they can in principle estimate heteroscedastic and multimodal predictive distributions. It is noteworthy that expectiles are the only risk measures that are both coherent and elicitable [52,53], making them particularly well-suited for financial risk management. As linear regression [50,51] can only estimate linear dependence on predictors, various nonlinear generalizations have been explored. For example, quantile regression NNs [54,55] adopt artificial NNs, whereas Quantile Regression Forests (QRF) [29] train an ensemble of decision trees. In contrast to the original random forests [56] that record only the mean of the observations in each leaf, QRF keeps the value of all observations for subsequent quantile estimation. Generalized random forests [36] is another generalization of random forests, which utilizes a tailored splitting criterion of trees and treats forests as a type of adaptive nearest neighbor estimator, demonstrating strong empirical performance. Gradient boosting decision trees such as XGBoost [19] and LightGBM [20] can also minimize quantile losses, but one model per one quantile is necessary, which works fine for confidence interval prediction but incurs high computational costs when the entire predictive distribution is to be estimated.

While classical Kernel Density Estimation (KDE) and density-ratio methods (e.g., Ref. [31]) offer theoretically sound approaches for conditional density estimation, we do not treat them as direct baselines in our setting. Traditional KDE evaluating $P\left(Y\right|X)=P(X,Y)/P(X)$ suffers from zero-denominator instabilities and meaninglessly inflated distance metrics in the presence of irrelevant features. Although adapting bandwidths component-wise can mitigate this, it incurs prohibitive computational costs. Tree-based methods inherently bypass these limitations through adaptive spatial partitioning, making them more robust for high-dimensional inputs [57].

There are also models that exploit the Wasserstein distance, which stems from the optimal transport theory and has gained popularity in the machine learning community [58,59,60,61,62]. Wasserstein random forests [41] employ a novel splitting criterion of trees based on the Wasserstein distance. Wasserstein gradient boosting [48] trains weak learners successively via Wasserstein gradients and nonparametrically solves regression problems with distribution-valued responses.

In addition to these nonparametric density estimation methods, there is another class of methods that aim to fit a parametric family of probability distributions to the target. A classical and well-established example is the Gaussian Processes (GP) [30]. Gradient boosting-based models such as NGBoost [37] and XGBoostLSS [42] allow fitting various parametric (e.g., Gaussian, Laplace, Lognormal, and Gaussian mixture) distributions. Distributional regression forests [35] learn parameters of a zero-censored Gaussian distribution from data, with a focus on meteorological applications. Transformation forests [39,40] posit a monotone transformation that maps the target to a known base distribution and fit its parameters locally with trees using likelihood-based splits, yielding a smooth predictive CDF.

Recent years have witnessed remarkable progress in uncertainty quantification techniques for deep learning, such as Bayesian NNs, Monte Carlo dropout, and deep ensembles, as reviewed in Refs. [2,3,4,6,7,8,10]. A fundamental insight emerging from these lines of work is that uncertainty in machine learning generally comprises both aleatoric uncertainty and epistemic uncertainty [3]. The former is related to intrinsic stochasticity of the data generating process, which can be precisely modeled with quantile regression or parametric distribution fitting, whereas the latter represents our lack of knowledge and can be reduced by gathering more data; the latter uncertainty is high for out-of-distribution samples and hence its quantification is crucial in the task of anomaly (or outlier) detection. Whether NNs are used or not, it is generally difficult to capture both types of uncertainty within a single model. Tree-based methods, including gradient boosting and random forests, are not well-suited for measuring epistemic uncertainty because trees are poor extrapolators by construction. Our work in this paper also uses trees, and we assume that the main application of our model should be in the area of aleatoric uncertainty quantification.

2.2. Multivariate Target Variable

Multioutput regression models have a long history of research [63,64,65]. It is notoriously difficult to extend the concept of quantiles to more than one dimension [66,67], hampering nonparametric distributional prediction for multidimensional outputs. Directional quantile regression (DQR) [32,33,34] is an approach that computes directional half-spaces and takes their intersections to construct quantile regions in arbitrary dimensions. However, DQR by construction only yields convex quantile regions and cannot describe general nonconvex features. More recent work [49,68] developed the theory of center-outward quantile regions, which propose a novel multivariate quantile concept and overcome DQR’s limitation of convexity. However, its current implementation [49] is based on classic kernel weighting and is vulnerable to the curse of dimensionality when the predictor resides in high dimensions. Unlike these prior methods that primarily target the geometric boundaries of quantile regions, our focus is on estimating the full conditional distribution $p\left(\mathbf{y}\right|\mathbf{x})$.

There are several lines of work in deep learning that addresses the challenge of multivariate probabilistic predictions. Refs. [23,25] propose to formulate multivariate quantile functions as the gradient of partially input-convex NNs. Ref. [26] enhances DQR by representation learning via a conditional variational autoencoder, so that arbitrary nonconvex probability regions can be predicted. Ref. [22] integrates NNs with a Kalman filter for applications to a visual tracking problem. Refs. [21,24] employ implicit generative NNs, which receive a noise vector as additional input and generate dispersed predictions. There are more references in the domain of probabilistic multivariate time-series forecasting, e.g., [69].

Compared to deep learning, tree-based approaches to multivariate probabilistic predictions have been less explored. Refs. [38,44] extend gradient boosting so that parameters of multivariate distributions can be learned. Wasserstein random forests [41] offer a natural extension of random forests to multivariate targets. Distributional Random Forest (DRF) [45,46] grows an ensemble of decision trees with a distribution-aware splitting criterion based on the Maximal Mean Discrepancy metric [70,71]. This is in marked contrast to QRF [29], which uses the plain CART splitting criterion of the original random forests [56]. DRF dynamically decides weights for each training data point and estimates the predictive distribution by an empirical measure determined by training points and their weights.

Fréchet regression [72] is a general framework for regressing targets that lie in non-Euclidean metric spaces. Ref. [47] extended this framework to multivariate distributions by using the sliced Wasserstein distance [73,74], which provides a computationally efficient surrogate for the bona fide Wasserstein distance. Specifically, Ref. [47] proposed four variants: GSWW, GSAW, LSWW, and LSAW. The global variants (GSWW and GSAW) generalize linear regression to metric spaces, whereas the local variants (LSWW and LSAW) use kernel smoothing to perform nonparametric local linear regression.

We highlight two technical differences between Ref. [47] and the present work. First, Ref. [47] provides strong theoretical guarantees, but its linear modeling assumptions may limit flexibility and can exacerbate the curse of dimensionality in high-dimensional settings. In contrast, our approach uses fully nonparametric, tree-based models, which can alleviate these limitations in practice. Second, GSWW and LSWW in Ref. [47] apply Fréchet regression independently to each Radon slice, whereas our method trains a single model that jointly covers all Radon slices.

3. QRF++: A Simplified DRF Implementation

3.1. Definition

This section introduces a slight modification of the original QRF model [29] with output-space augmentation. Let ${\left\{({\mathbf{x}}_i,y_i)\right\}}_{i=1}^N$ be training data with a univariate response $y_i\in \mathbb{R}$. QRF builds an ensemble of CART trees using bootstrapped data and the squared-error splitting criterion; at prediction time, QRF estimates the conditional CDF $F\left(y\right|\mathbf{x})$ by aggregating training points that fall into the same leaves as the query $\mathbf{x}$. As emphasized by Refs. [36,45], such univariate targets make splits primarily sensitive to changes in the conditional mean and they may overlook other distributional changes (e.g., variance, tail or multi-modality shifts).

While more sophisticated forest models [36,45] overcome such a limitation, we here propose a minimalist’s remedy that requires no algorithmic changes and can be implemented with any off-the-shelf QRF package: augment the learning target from $y\in \mathbb{R}$ to

$$\begin{array}{c}\hfill (y,\cos {\displaystyle \frac{y}{w_1}},\sin {\displaystyle \frac{y}{w_1}},\dots ,\cos {\displaystyle \frac{y}{w_T}},\sin {\displaystyle \frac{y}{w_T}})\in {\mathbb{R}}^{2T+1},\end{array}$$

(1)

followed by a per-component normalization so that all $2T+1$ components have comparable variance. We then train a multioutput forest using the average sum of squares across multiple outputs as the splitting criterion [75,76]. The parameters $w_t>0$ determine the inverse frequency of random Fourier features. At prediction time, we simply take the first component of the model’s output and ignore the rest. We refer to this method as QRF++.

The rationale behind QRF++ is as follows. Consider a trivial example where $p\left(y\right|x)$ changes abruptly at $x=0$, from $\mathcal{N}(0,1^2)$ for $x<0$ to ${\displaystyle \frac{1}{2}}{\delta }_{-\pi }+{\displaystyle \frac{1}{2}}{\delta }_{+\pi }$ for $x>0$. Since both sides have $\mathbb{E}\left[y\right|x]=0$, a univariate CART split need not occur at $x=0$. In contrast, $\mathbb{E}\left[\cos y\right|x]$ jumps from ${\mathrm{e}}^{-1/2}\simeq 0.606$ to $-1$ at $x=0$, creating a strong signal. More generally, one can uniquely determine the full distribution once the characteristic function $\mathbb{E}\left[{\mathrm{e}}^{i\alpha y}\right|x]$ is obtained for every $\alpha \in \mathbb{R}$. Sampling a few frequencies as an approximation leads to (1).

Conceptually, QRF++ is a variant of DRF [45]. DRF selects splits by maximizing MMD in a chosen RKHS. If one chooses a linear kernel over the feature space (1), the resulting DRF would match QRF++. Equation (1) can also be viewed as random Fourier features stemming from some stationary kernel through Bochner’s theorem. For mathematical details, we refer the reader to Ref. [45], Theorem 1. Thus, QRF++ is in essence a compact lightweight DRF implementation (To be precise, QRF++ does not follow the honesty principle [77,78] in building trees, unlike DRF. Moreover, QRF++ uses bootstrapping, while DRF uses subsampling without replacement).

An important caveat is that (1) does not define a characteristic kernel: it is not possible to distinguish two arbitrary distributions based on (1) as long as $T<\infty$. Nevertheless, as our experiments in the next subsection show, the minimal modification (1) substantially improves QRF’s sensitivity to various distributional changes that are overlooked by vanilla QRF.

3.2. Evaluation of QRF++

We compare QRF and QRF++ on four benchmark datasets. In all the datasets, the predictor $\mathbf{x}\in {\mathbb{R}}^{40}$ is sampled from a uniform distribution over ${[-1,1]}^{40}$. Of these, 39 features are just noise, and only 1 feature is associated with the target y. Characteristics of the datasets (a)–(d) are summarized below:

(a) 

$y|x\sim \mathcal{N}(0,1^2\mathrm{)}$ for $x<0$ and $\mathcal{N}(0,3^2)$ for $x\ge 0$. The mean is constant but the variance jumps at $x=0$. There are 1500 samples.

(b) 

$y|x\sim \mathcal{N}(0,1^2)$ for $x<0$ and ${\displaystyle \frac{1}{2}}\left(\mathcal{N}\right(\mu ,{\sigma }^2)+\mathcal{N}(-\mu ,{\sigma }^2\left)\right)$ for $x\ge 0$, where $\mu =0.95$ and $\sigma =\sqrt{1-{\mu }^2}$. The mean and variance are 0 and 1 for all x. The distribution is unimodal for $x<0$ and bimodal for $x\ge 0$. There are 3000 samples.

(c) 

$y|x\sim \mathrm{Uniform}\left(\right[-{(x+2.5)}^2,{(x+2.5)}^2\left]\right)$. This dataset exhibits a funnel-like shape when plotted in the $xy$-plane. There are 3500 samples.

(d) 

$y|x\sim 1-\mathrm{Exp}(1)$ for $x<0$ and $\mathrm{Exp}\left(1\right)-1$ for $x\ge 0$, where $\mathrm{Exp}\left(\lambda \right)$ is the exponential distribution with the rate parameter $\lambda$. The mean and variance are 0 and 1 for all x. There are 1700 samples.

While there are a few publicly available QRF implementations [79,80], we use the Cython-optimized Python package quantile-forest [81]. We fix the number of trees to 100 and prevent overfitting by setting min_samples_leaf $= 30$. As for (1), we set $T=3$ and let $(w_1,w_2,w_3)=(w_{\mathrm{med}}/2,w_{\mathrm{med}},2w_{\mathrm{med}})$, where $w_{\mathrm{med}}$ is the median pairwise distance of $\left\{y_i\right\}$, motivated by the median heuristic [71,82,83].

Numerical results are shown in Figure 2. Across experiments (a)–(d), the vanilla QRF produces poor approximations to the true quantiles, consistent with the fact that a constant zero mean prevents the standard CART splitting criterion from capturing distributional variation. By contrast, QRF++ delivers accurate predictions in all cases. In particular, in (b) and (d), QRF++ sharply recovers the jumps in the quantiles, even though the mean and variance remain unchanged.

We examined the average tree depth and training time for both models. On average, QRF++ trees were 18% shallower than those of QRF, and QRF++ trained 14% faster. The likely explanation is that the output-augmented splits in QRF++ identify stronger partitions earlier, yielding more balanced trees. Although multioutput targets add a small constant overhead to each split evaluation, the reduction in per-node sample counts dominates, resulting in the observed speedup.

It is natural to ask which components of the augmented target most strongly shape the trees. We quantify this via a component-wise decomposition of the impurity decrease at each split under the multi-output CART criterion (summed squared error). Let a node be split into left/right children with sample sets ${\mathcal{I}}_P,{\mathcal{I}}_L,{\mathcal{I}}_R$ and let $Y^{\left(m\right)}$ denote the m-th target coordinate. Writing ${\overline{Y}}_{\#}^{\left(m\right)}$ for the (weighted) mean on each set, the reduction in the sum of squared errors attributable to coordinate m at that split is

$$\begin{array}{c}\hfill {\Delta }^{\left(m\right)}=\sum _{i\in {\mathcal{I}}_P}{\left(Y_i^{\left(m\right)}-{\overline{Y}}_P^{\left(m\right)}\right)}^2-\sum _{i\in {\mathcal{I}}_L}{\left(Y_i^{\left(m\right)}-{\overline{Y}}_L^{\left(m\right)}\right)}^2-\sum _{i\in {\mathcal{I}}_R}{\left(Y_i^{\left(m\right)}-{\overline{Y}}_R^{\left(m\right)}\right)}^2.\end{array}$$

(2)

The node’s total gain is $\Delta ={\sum }_m{\Delta }^{\left(m\right)}$. For a given tree t, we define the (raw) target importance of coordinate m as the sum over all internal nodes,

$$\begin{array}{c}\hfill G_t^{\left(m\right)}=\sum _{\mathrm{nodes} v\in t}{\Delta }_v^{\left(m\right)},\end{array}$$

(3)

and normalize to obtain

$$\begin{array}{c}\hfill I_t^{\left(m\right)}={\displaystyle \frac{G_t^{\left(m\right)}}{{\sum }_{m^{\prime }}{G}_t^{\left(m^{\prime }\right)}}},\sum _m{I}_t^{\left(m\right)}=1.\end{array}$$

(4)

Averaging over all trees in the forest yields the forest-level importance ${\overline{I}}^{\left(m\right)}={\displaystyle \frac{1}{T}}{\sum }_{t=1}^T{I}_t^{\left(m\right)}$, which sums to one across m and reflects the relative contribution of each target coordinate to split selection during training. This notion is directly analogous to the feature importance (defined for input variables), except that here the roles of inputs and outputs are reversed. It is recommended to standardize each target coordinate before training so that importances are not dominated by scale.

The resulting target importances for the four synthetic datasets are shown in Figure 3. Across all cases, the raw target y carries an importance of about $0.1$ or less, indicating that the $2T$ auxiliary components dominate split decisions. In panels (a), (b), and (c), the importances of the sin components are small, which is consistent with the symmetry of the true conditional distributions around $y=0$. In panel (d), by contrast, the sin components receive large importances, as expected from the distributional flip $p(y\mid x>0)=p(-y\mid x<0)$ that creates an odd (sign-reversing) signal.

3.3. Prediction Interval Coverage and Width

Prediction Intervals (PIs) provide an intuitive approach to addressing the limitations of point predictions and are widely adopted in real-world forecasting tasks. Given their practical importance, it is crucial to verify that our proposed model, QRF++, generates reliable PIs. To evaluate this capability, we employ the Prediction Interval Coverage Probability (PICP) and Mean Prediction Interval Width (MPIW), two standard metrics widely recognized in the literature [7,10,84,85,86,87]. Suppose that a model generates a PI $[{\widehat{y}}_i^{\mathrm{L}},{\widehat{y}}_i^{\mathrm{U}}]$ with bounds ${\widehat{y}}_i^{\mathrm{L}}<{\widehat{y}}_i^{\mathrm{U}}$ at a nominal confidence level $\lambda \in [0,1]$ for each observation $(x_i,y_i)$ in a dataset of size N. These metrics are defined as follows:

$$\begin{array}{cc}\hfill \mathrm{PICP}\left(\lambda \right)& ={\displaystyle \frac{1}{N}}\sum _{i=1}^N{\mathbb{1}}_{\{{\widehat{y}}_i^{\mathrm{L}}\le y_i\le {\widehat{y}}_i^{\mathrm{U}}\}},\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill \mathrm{MPIW}\left(\lambda \right)& ={\displaystyle \frac{1}{N}}\sum _{i=1}^N({\widehat{y}}_i^{\mathrm{U}}-{\widehat{y}}_i^{\mathrm{L}}).\hfill \end{array}$$

(6)

Intuitively, a well-calibrated prediction should yield an empirical coverage that closely matches the nominal level, meaning $\mathrm{PICP}\left(\lambda \right)\approx \lambda$ for any specified $\lambda$. Furthermore, MPIW quantifies the average width of the PIs, serving as a measure of predictive sharpness. Since artificially widening the intervals trivially guarantees high coverage at the cost of informational value, and excessively narrow intervals fail to achieve the target coverage, a high-quality model must strike an optimal balance—minimizing the MPIW while strictly maintaining the target PICP.

In QRF++, a PI at a confidence level of $\lambda$ is constructed directly from the lower $(1-\lambda )/2$ and upper $(1+\lambda )/2$ quantiles of the predictive distribution. For instance, setting $\lambda =0.8$ yields a quantile range of $[0.1,0.9]$.

Table 2. Datasets with univariate target, used to evaluate PI in this study. The concrete dataset contained multiple duplicate instances, and their removal reduces the dataset size to 1005.

Dataset	# Instances	# Input Features	Reference
Concrete	1030	8	[88]
Diabetes	442	10	[89]
Airfoil	1503	5	[90]

summarizes the three real-world datasets used in our evaluation. While the concrete and airfoil datasets are relatively easy to predict (${\mathrm{R}}^2\gtrsim 0.9$), the diabetes dataset presents a more challenging task (${\mathrm{R}}^2\lesssim 0.5$). Note that our evaluation emphasizes PI quality over point forecast accuracy. As baselines, we evaluate both GP and standard QRF; however, since QRF and QRF++ perform similarly on these datasets, we report only the results of GP and QRF++ for brevity. Finally, all data were standardized prior to training to ensure numerical stability.

• Hyperparameters

For the GP model, we utilized a Matérn kernel ($\nu =3/2$) with automatic relevance determination to adjust feature-specific length scales. In QRF++, the forest size was set to 300 trees for the concrete and airfoil datasets, and 100 trees for the diabetes dataset. We evaluated the models using repeated random sub-sampling, partitioning the data into 80% training, 10% validation, and 10% test sets over 20 independent trials, and reported the averaged results. While QRF++ regularization was managed via the max_leaf_nodes parameter, the model interestingly achieved its best ${\mathrm{R}}^2$ performance on these datasets without any regularization.

• Results

Figure 4 displays the results for the concrete dataset. For the GP model (leftmost panel), the actual coverage is notably larger than the nominal confidence level, implying that the model is under-confident. For QRF++, we observe that, when the regularization is strong (max_leaf_nodes $=300$), the PICP curve closely matches the oracle line, indicating good calibration. Conversely, under weak regularization (max_leaf_nodes$=500$), the PICP deviates from the oracle line at low confidence levels. This suggests that pursuing the best ${\mathrm{R}}^2$ score may hinder achieving optimal calibration. Finally, the MPIW curves (rightmost panel) for the three models closely overlap.

Figure 5 compares the PIs produced by the GP model and unregularized QRF++. While the GP model yields PIs of nearly constant width, the widths of the QRF++ intervals vary considerably across samples.

Figure 6 displays the results for the diabetes dataset. In this case, the GP model is well-calibrated, whereas QRF++ appears to be slightly over-confident, as its actual coverage falls slightly below the nominal confidence level. Unlike the results for the concrete dataset, the QRF++ curves did not exhibit a noticeable dependence on regularization.

Figure 7 displays the results for the airfoil dataset. The coverage of the GP model shows a strong deviation from the nominal confidence level, while QRF++ is better calibrated. Interestingly, in this case, the PI widths produced by these models differ substantially (rightmost panel), with the GP model yielding sharper PIs. One plausible hypothesis for the GP model yielding a narrower MPIW and higher coverage is that the underlying target function is highly smooth, making it inherently well-suited for GP, whereas the piecewise-constant nature of QRF++ might be less compatible with such continuous characteristics. Finally, test samples ordered by PI widths are shown in Figure 8.

In summary, QRF++ generally produces reliable PIs, and the deviation between the empirical coverage and the nominal confidence level ($\left|\mathrm{PICP}\right(\lambda )-\lambda |$) is quantitatively small, if present. However, we find that the calibration quality and the trends in MPIW are highly dataset-dependent. Pursuing further improvements by employing methods such as conformal prediction [91,92,93] merits future study, although it is beyond the scope of this work.

4. Proposed Method

In this section, we define and elaborate on Tomographic Quantile Forests (TQF), a regression framework tailored for multivariate uncertainty quantification. The pseudocode is displayed in Algorithms 1–3 below. TQF is fully nonparametric, and leverages QRF++ as a backbone to mitigate the curse of dimensionality and suppress the impact of noisy features. Rather than outputting a full distribution directly, TQF employs a post-processing step that constructs an empirical measure of weighted points that approximates the conditional distribution $p\left(\mathbf{y}\right|\mathbf{x})$.

Learning the distribution of Radon-transformed projections could be vulnerable to scale differences among different components of $\mathbf{y}$. For example, a meteorological record $\mathbf{y}$ might comprise a vector of (solar radiation, precipitation, wind speed, temperature) $\in {\mathbb{R}}^4$, which are measured in entirely different units. To mitigate this problem, we standardize each component of $\mathbf{y}$ to zero mean and unit variance before training starts (line 1 in Algorithm 1).

Algorithm 1 Tomographic Quantile Forests (TQF)

Input:  Training dataset $\mathcal{D}={\left\{({\mathbf{x}}_n,{\mathbf{y}}_n)\right\}}_{n=1}^N$, target dimensionality $d=dim\left({\mathbf{y}}_n\right)\in \mathbb{N}$, sample augmentation factor $G\in \mathbb{N}$, feature augmentation factor $\tilde{G}\in \mathbb{N}$, QRF++ model ${\mathcal{F}}_Q$ with a parameter $T\in \mathbb{N}\cup \left\{0\right\}$, quantile levels ${\left(q_m\right)}_{m=1}^M\in {(0,1)}^M$, number of projection axes for prediction $K\in \mathbb{N}$, test sample ${\mathbf{x}}_{\mathrm{new}}$ Output:  Weighted point cloud ${\left\{(w_j,{\mathbf{y}}_j)\right\}}_{j=1}^J$ representing the conditional distribution $p\left(\mathbf{y}\right|{\mathbf{x}}_{\mathrm{new}})$ # Training 1: Normalize the target data:    ${\displaystyle \tilde{\mathcal{D}}\leftarrow {\left\{{\mathbf{x}}_n,{\mathbf{z}}_n\right\}}_{n=1}^N:={\left\{{\mathbf{x}}_n,\mathtt{scaler}\left({\mathbf{y}}_n\right)\right\}}_{n=1}^N}$ 2: Replicate each record in $\tilde{\mathcal{D}}$ exactly G times: $${\tilde{\mathcal{D}}}_{*}\leftarrow {\left\{\left({\widehat{\mathbf{x}}}_{\mathcal{l}},{\widehat{\mathbf{z}}}_{\mathcal{l}}\right)\right\}}_{\mathcal{l}=1}^{GN}:=\left\{\underset{G}{\underbrace{({\mathbf{x}}_1,{\mathbf{z}}_1),\dots ,({\mathbf{x}}_1,{\mathbf{z}}_1)}},\underset{G}{\underbrace{({\mathbf{x}}_2,{\mathbf{z}}_2),\dots ,({\mathbf{x}}_2,{\mathbf{z}}_2)}},({\mathbf{x}}_3,{\mathbf{z}}_3),\dots \right\}$$ 3: Draw unit vectors ${\left\{{\mathbf{n}}_{\mathcal{l}}\right\}}_{\mathcal{l}=1}^{GN}$ randomly from ${\mathbb{S}}^{d-1}$ 4: Draw $\tilde{G}-1$ orthogonal matrices $O_1,O_2,\dots ,O_{\tilde{G}-1}$ randomly from the Haar measure on the orthogonal group $\mathrm{O}\left(d\right)$ 5: Perform projective augmentation of ${\tilde{\mathcal{D}}}_{*}$: $${\displaystyle {\tilde{\mathcal{D}}}_{**}\leftarrow {\left\{\left({\widehat{\mathbf{x}}}_{\mathcal{l}}\parallel {\mathbf{n}}_{\mathcal{l}}\parallel O_1{\mathbf{n}}_{\mathcal{l}}\parallel \dots \parallel O_{\tilde{G}-1}{\mathbf{n}}_{\mathcal{l}},{\mathbf{n}}_{\mathcal{l}}^{\top }{\widehat{\mathbf{z}}}_{\mathcal{l}}\right)\right\}}_{\mathcal{l}=1}^{GN}}$$ where ‖ denotes concatenation of vectors 6: Train ${\mathcal{F}}_Q$ on ${\tilde{\mathcal{D}}}_{**}$ 7: Construct a new model ${\mathcal{F}}_Q^{*}$ based on ${\mathcal{F}}_Q$: $${\displaystyle {\mathcal{F}}_Q^{*}(\mathbf{x},\mathbf{n},q):=\frac{1}{2}\left[{\mathcal{F}}_Q\left(\mathbf{x},\mathbf{n},q\right)-{\mathcal{F}}_Q\left(\mathbf{x},-\mathbf{n},1-q\right)\right]}$$ # Inference for a new predictor${\mathbf{x}}_{\mathrm{new}}$ 8: Draw unit vectors ${\left\{{\mathbf{n}}_k\right\}}_{k=1}^K$ randomly from ${\mathbb{S}}^{d-1}$ 9: ${\mathcal{D}}_{\mathrm{slice}}\leftarrow \varnothing$ 10: for  $k\leftarrow 1,\dots ,K$  do 11:     for $m\leftarrow 1,\dots ,M$ do 12:         Predict the quantile as    ${\mathcal{Q}}_{k,m}\leftarrow {\mathcal{F}}_Q^{*}({\mathbf{x}}_{\mathrm{new}},{\mathbf{n}}_k,q_m)$ 13:        ${\mathcal{D}}_{\mathrm{slice}}\leftarrow {\mathcal{D}}_{\mathrm{slice}}\bigcup \left\{{(}_k,q_m,{\mathcal{Q}}_{k,m})\right\}$ 14:     end for 15: end for 16: ${\left\{(w_j,{\mathbf{z}}_j)\right\}}_{j=1}^J\leftarrow \mathrm{QMEM}\left({\mathcal{D}}_{\mathrm{slice}}\right)$                     ▶Algorithm 2 17: Undo normalization: ${\left\{{\mathbf{y}}_j\right\}}_{j=1}^J\leftarrow {\left\{{\mathtt{scaler}}^{-1}\left({\mathbf{z}}_j\right)\right\}}_{j=1}^J$  18: return ${\left\{(w_j,{\mathbf{y}}_j)\right\}}_{j=1}^J$

4.1. Stage I: Model Fitting

The core idea is to consider multiple direction vectors $\mathbf{n}\in {\mathbb{S}}^{d-1}$ and, for each direction, model the conditional quantiles of the projected target ${\mathbf{n}}^{\top }\mathbf{y}$ as a function of both the covariates $\mathbf{x}$ and the direction n. At test time, the model predicts directional quantiles for multiple sampled directions and then reconstructs a multivariate conditional distribution by optimizing the locations and weights of a point cloud to approximately minimize the sliced Wasserstein distance ((A3) in Appendix A.1) to the true distribution. Given a weighted point cloud that represents a distribution, a broad range of statistical analyses becomes straightforward. For example, one can compute the mean, median, covariance, and expectiles; estimate marginal distributions; and estimate the copula. One can apply kernel density estimation to find a smooth interpolation of the empirical sample density.

Algorithm 2 Quantile-Matching Empirical Measure (QMEM)

Input:  Radon-transformed quantile values ${\mathcal{D}}_{\mathrm{slice}}=\left\{\left({\mathbf{n}}_k,q_m,{\mathcal{Q}}_{k,m}\right)\mid k=1..K,m=1..M\right\}$, loss function $\mathcal{L}(·,{\mathcal{D}}_{\mathrm{slice}})$, initial point cloud size $N_0\in \mathbb{N}$, regular point cloud size $N_1\in \mathbb{N}$, ensemble size $E\in \mathbb{N}$ Output:  Weighted point cloud ${\left\{(w_j,{\mathbf{z}}_j)\right\}}_{j=1}^J$ 1: $w_j\leftarrow 1/N_0$  for  $j\in \{1,\dots ,N_0\}$ 2: ${\displaystyle \left\{{\mathbf{z}}_j\right\}\leftarrow \underset{{\mathbf{z}}^{\prime }}{\mathrm{argmin}}\mathcal{L}\left({\left\{(w_j,{\mathbf{z}}_j^{\prime })\right\}}_{j=1}^{N_0},{\mathcal{D}}_{\mathrm{slice}}\right)}$ 3: $\mathcal{Z}\leftarrow {\left\{(w_j,{\mathbf{z}}_j)\right\}}_{j=1}^{N_0}$ 4: $\mathtt{loss}\leftarrow \mathcal{L}\left(\mathcal{Z},{\mathcal{D}}_{\mathrm{slice}}\right)$ 5: repeat 6:     Fit a Gaussian KDE to $\mathcal{Z}$ 7:     Draw samples ${\left\{{\mathbf{z}}_j\right\}}_{j=1}^{N_1}$ from the KDE 8:     ${\displaystyle \left\{w_j\right\}\leftarrow \underset{w^{\prime }}{\mathrm{argmin}}\mathcal{L}\left({\left\{(w_j^{\prime },{\mathbf{z}}_j)\right\}}_{j=1}^{N_1},{\mathcal{D}}_{\mathrm{slice}}\right)}$ 9:     $\mathcal{Z}\leftarrow {\left\{(w_j,{\mathbf{z}}_j)\right\}}_{j=1}^{N_1}$ 10:    $\mathtt{loss}\leftarrow \mathcal{L}\left(\mathcal{Z},{\mathcal{D}}_{\mathrm{slice}}\right)$ 11: until loss stops decreasing 12: Fit a Gaussian KDE to $\mathcal{Z}$ 13: ${\mathcal{Z}}_{*}\leftarrow \varnothing$ 14: for  $e\leftarrow 1,\dots ,E$  do 15:     Draw samples ${\left\{{\mathbf{z}}_j^{*}\right\}}_{j=1}^{N_1}$ from the KDE 16:     ${\displaystyle \left\{w_j^{*}\right\}\leftarrow \underset{w^{\prime }}{\mathrm{argmin}}\mathcal{L}\left({\left\{(w_j^{\prime },{\mathbf{z}}_j^{*})\right\}}_{j=1}^{N_1},{\mathcal{D}}_{\mathrm{slice}}\right)}$ 17:     ${\mathcal{Z}}^{\left(e\right)}\leftarrow {\left\{(w_j^{*}/E,{\mathbf{z}}_j^{*})\right\}}_{j=1}^{N_1}$ 18:     ${\mathcal{Z}}_{*}\leftarrow {\mathcal{Z}}_{*}\bigcup {\mathcal{Z}}^{\left(e\right)}$ 19: end for 20: ${\mathcal{Z}}_{*}\leftarrow \mathrm{Prune}\left({\mathcal{Z}}_{*}\right)$                     ▶Algorithm 3 21: return ${\mathcal{Z}}_{*}$

Algorithm 3 Prune a Point Cloud

Input:  Loss function $\mathcal{L}(·,{\mathcal{D}}_{\mathrm{slice}})$, weighted point cloud ${\left\{(w_j,{\mathbf{z}}_j)\right\}}_{j=1}^J$ Output:  Pruned point cloud $\left\{(w_j^{\prime },{\mathbf{z}}_j)\right\}$ 1: Sort the points in descending order by weight and relabel the indices so that $w_1\ge w_2\ge \dots \ge w_J$ 2: for  $\tilde{j}\leftarrow 1,\dots ,J$  do 3:     $C\leftarrow {\sum }_{j=1}^{\tilde{j}}{w}_j$ 4:     for $j\leftarrow 1,\dots ,\tilde{j}$ do 5:         $w_j^{\prime }\leftarrow w_j/C$ 6:     end for 7:     $L_{\tilde{j}}\leftarrow \mathcal{L}\left({\left\{(w_j^{\prime },{\mathbf{z}}_j)\right\}}_{j=1}^{\tilde{j}},{\mathcal{D}}_{\mathrm{slice}}\right)$ 8: end for 9: $j^{*}\leftarrow {\mathrm{argmin}}_{\tilde{j}}{L}_{\tilde{j}}$ 10: $C\leftarrow {\sum }_{j=1}^{j^{*}}{w}_j$ 11: for  $j=1,\dots ,j^{*}$  do 12:      $w_j^{\prime }\leftarrow w_j/C$ 13: end for 19: return${\left\{(w_j^{\prime },{\mathbf{z}}_j)\right\}}_{j=1}^{j^{*}}$

A naïve implementation would require fitting $K\times M$ separate quantile regressors for each of K directions and M quantile levels in $(0,1)$. That will inflate implementation effort and training time. Moreover, projected values ${\left({\mathbf{n}}^{\top }\mathbf{y}\right)}_q$ at nearby quantile levels q and nearby directions n are strongly correlated (i.e., ${\left({\mathbf{n}}^{\top }\mathbf{y}\right)}_q\simeq {\left({{\mathbf{n}}^{\prime }}^{\top }\mathbf{y}\right)}_{q^{\prime }}$ if n ≃ n’ and $q\simeq q^{\prime }$), making independent modeling statistically inefficient. We therefore adopt two design choices. First, we use QRF++ (introduced in Section 3) to learn multiple quantile levels simultaneously. Second, we embed the direction n as part of the input, concatenating $\mathbf{x}$ and n and feeding the joint vector $\mathbf{x}\parallel \mathbf{n}$ to the QRF++ model. It is worthwhile to mention an advantage of using QRF/QRF++: the monotonicity of estimated quantiles. Methods that fit independent models at different quantile levels are prone to quantile crossing due to broken monotonicity [94,95]; in contrast, QRF++ reads all quantiles from one estimated conditional CDF, which ensures monotonicity.

The resulting projective augmentation proceeds as follows (lines 2–5 in Algorithm 1). For each training pair $(\mathbf{x},\mathbf{y})$, sample $G>1$ unit vectors ${\left\{{\mathbf{n}}_g\right\}}_{g=1}^G$ uniformly from the hypersphere ${\mathbb{S}}^{d-1}$. In theory, a concatenated vector $\mathbf{x}\parallel {\mathbf{n}}_g$ carries complete information on $\mathbf{x}$ and ${\mathbf{n}}_g$. However, in practice, this naïve representation might be suboptimal, leading to RF’s excessive partitions or slower convergence. For instance, consider curving out the interval $-1\le x\le 1$ from $\mathbb{R}$. A decision tree would need two splits at $x=-1$ and $x=+1$. However, if $x^2$ is present as an extra feature, then only a single partition at $x^2=1$ will work. This way, feature enrichment can substantially help to make trees simpler. Motivated by this observation, we propose to enrich the input $\mathbf{x}\parallel {\mathbf{n}}_g$ as follows. Let $\tilde{G}\ge 1$ and draw $\tilde{G}-1$$d\times d$ orthogonal matrices $O_1,O_2,\dots ,O_{\tilde{G}-1}$ randomly from the orthogonal group $\mathrm{O}\left(d\right)$. Then, we rotate ${\mathbf{n}}_g$ by $O_1,O_2,\dots$ and append them as extra features, resulting in the long augmented vector $\mathbf{x}\parallel {\mathbf{n}}_g\parallel O_1{\mathbf{n}}_g\parallel O_2{\mathbf{n}}_g\parallel \dots \parallel O_{\tilde{G}-1}{\mathbf{n}}_g$. (If $\tilde{G}=1$, no feature augmentation takes place.) This process is repeated for each $g=1,\dots ,G$, yielding G augmented training pairs. Summarizing, the projective augmentation is a transformation

$$\begin{array}{c}\hfill (\mathbf{x},\mathbf{y})\mapsto {\left\{\left(\mathbf{x}\parallel {\mathbf{n}}_g\parallel O_1{\mathbf{n}}_g\parallel \dots \parallel O_{\tilde{G}-1}{\mathbf{n}}_g,{\mathbf{n}}_g^{\top }\mathbf{y}\right)\right\}}_{g=1}^G.\end{array}$$

(7)

Iterating this for all the training pairs $\left\{\right(\mathbf{x},\mathbf{y}\left)\right\}$ will enlarge the dataset volume by a factor of G. Increasing G would make it easier for the model to learn -dependence, but the training cost would increase in proportion. Determining the optimal choice of G and $\tilde{G}$ on general grounds is difficult; we believe it would be sensitive to the dataset size and characteristics, the dimensionality of $\mathbf{y}$, and the QRF++ hyperparameters. In Section 6.1, we examine this numerically.

Because projections onto n and −n differ only by sign, one could restrict the sampling of directions to a hemisphere without any loss of generality. However, it is known that decision-tree-based predictors are prone to boundary (“edge”) effects, with predictions flattening near the edge of the covariate domain [36]. To mitigate such boundary effects in QRF++, we sample directions over the entire sphere ${\mathbb{S}}^{d-1}$.

By the definitions of projection and quantiles, one has $-{\left({\mathbf{n}}^{\top }\mathbf{y}\right)}_q={\left({(-\mathbf{n})}^{\top }\mathbf{y}\right)}_{1-q}$. The predictions by QRF++, however, respect this symmetry only approximately. To enforce this, we symmetrize the predictions by taking the average of the two “partner” predictions (line 7 in Algorithm 1). This symmetrization ensures that the symmetry holds exactly by construction.

Our approach outlined above is partly inspired by prior work on offline reinforcement learning, specifically Hindsight Experience Replay [96], where the training of agents on sparse rewards is facilitated by augmenting state transition samples from a replay buffer with multiple goals and corresponding rewards, thereby reusing the same experience for diverse targets without additional environment interactions. It is also noteworthy that the method for constructing linear combinations of existing multiple targets has been proposed before [97,98], but their interest was not in uncertainty quantification.

• Computational Complexity

It is important to explicitly note the computational overhead introduced by the projective augmentation and target expansion in TQF. As detailed by Louppe [99], training a standard random forest with $n_{trees}$ trees requires an average time complexity of roughly $\mathcal{O}(n_{trees}·p·N{log}^{2}N)$ for N samples and p features. In TQF, the effective sample size is multiplied by G, the number of features increases to $p+d\tilde{G}$, and the multi-output split criterion in QRF++ evaluates $2T+1$ targets simultaneously. Assuming $N\gg G$, the theoretical average time complexity increases by a multiplicative factor of approximately $G(1+d\tilde{G}/p)(2T+1)$ compared to a standard univariate random forest. It is worth noting that this worst-case overhead can sometimes be partially offset in practice; as observed in Section 3.2, expanding the target dimension may lead to stronger early partitions and shallower trees, effectively reducing the total number of node splits. Furthermore, because individual trees are built independently, the training process is embarrassingly parallel, allowing the actual runtime to be significantly reduced by leveraging multi-core processors. In any case, while TQF introduces a noticeable constant-factor overhead in memory and time, its overall asymptotic scaling with respect to the sample size N remains $\mathcal{O}\left(N{log}^{2}N\right)$, ensuring that the method remains computationally tractable for typical tabular datasets.

4.2. Stage II: Distribution Reconstruction

Once the QRF++ model is trained, we use their predictions at a new input ${\mathbf{x}}_{\mathrm{new}}$ to assemble a weighted point cloud that approximates the conditional distribution $p\left(\mathbf{y}\right|{\mathbf{x}}_{\mathrm{new}})$. The procedure is outlined in Algorithms 2 and 3; we refer to it as the Quantile-Matching Empirical Measure (QMEM). We choose the point locations and weights by minimizing a loss that compares the model-predicted directional quantiles with those induced by the point cloud itself. Let $\mathcal{Z}$ denote a weighted point cloud and write its directional quantile as $\mathcal{Q}(\mathbf{n},q;\mathcal{Z})$. If we had access to the true distribution p, the sliced 1-Wasserstein distance between $\mathcal{Z}$ and p would be (cf. (A3) in Appendix A.1)

$$\begin{array}{}\mathrm{(8)}& \hfill {\mathrm{SW}}_1(\mathcal{Z},p)& ={\int }_{{\mathbb{S}}^{d-1}}\mathrm{d}{\mathrm{W}}_1\left({\widehat{R}}_{\mathbf{n}♯}\mathcal{Z},{\widehat{R}}_{\mathbf{n}♯}p\right)\hfill \mathrm{(9)}& & ={\int }_{{\mathbb{S}}^{d-1}}\mathrm{d}{\int }_0^1\mathrm{d}q|\mathcal{Q}(\mathbf{n},q;\mathcal{Z})-F_{{\widehat{R}}_{\mathbf{n}♯}p}^{-1}\left(q\right)|\hfill \mathrm{(10)}& & \simeq {\displaystyle \frac{1}{K}}\sum _{{\mathbf{n}}_k}{\displaystyle \frac{1}{M}}\sum _{q_m}|\mathcal{Q}({\mathbf{n}}_k,q_m;\mathcal{Z})-F_{{\widehat{R}}_{{\mathbf{n}}_k♯}p}^{-1}\left(q_m\right)|\hfill \mathrm{(11)}& & \simeq {\displaystyle \frac{1}{K}}\sum _{{\mathbf{n}}_k}{\displaystyle \frac{1}{M}}\sum _{q_m}|\mathcal{Q}({\mathbf{n}}_k,q_m;\mathcal{Z})-{\mathcal{Q}}_{k,m}|.\hfill \end{array}$$

In the last step, the inverse cumulative distribution function is replaced by the QRF++ predictions ${\mathcal{Q}}_{k,m}$. We adopt this discretized sliced Wasserstein objective as our loss and minimize it with respect to $\mathcal{Z}$. $\mathcal{Q}$ requires evaluating a sample quantile function, for which multiple conventions exist [100]. The choice is particularly consequential when the sample size is small. We adopt Hazen’s formula (Type 5 in Ref. [100]), which is the only definition that satisfies all desiderata listed therein.

Point-cloud optimization has been used to compute Wasserstein barycenters [73,74,101], where one seeks a distribution minimizing a weighted sum of squared Wasserstein distances to several input distributions. Our setting differs in that barycenter problems assume sample access to each input distribution, while here we only observe directional-quantile summaries ${\mathcal{Q}}_{k,m}$ of the target conditional distribution.

The point cloud $\mathcal{Z}={\left\{(w_j,{\mathbf{z}}_j)\right\}}_j$ consists of nonnegative weights $w_j$ and points ${\mathbf{z}}_j$, subject to the normalization ${\sum }_j{w}_j=1$. As noted in (A5) in Appendix A.1, the sliced Wasserstein distance ${\mathrm{SW}}_1$ is convex in its distributional argument; consequently, our loss (after discretizing the integral over $q\in [0,1]$) is approximately convex. In particular, optimizing the weights with fixed support points is an approximately convex problem, which can be solved stably and efficiently.

By contrast, the loss viewed as a function of the locations $\left\{{\mathbf{z}}_j\right\}$ is nonconvex; direct nonlinear optimization is expensive and prone to poor local minima. To mitigate this, we employ an alternating procedure that iterates point updates and weight optimization. In the point-update step, given the current weighted cloud, we fit a KDE and sample a refreshed set of support points from it. In the weight-optimization step, we reoptimize the weights for the new support so as to minimize the loss. Repeating these two steps drives the loss to a plateau.

To further improve robustness, we form an ensemble of E point clouds: we run E parallel instances of the above alternation (sampling from the KDE and reoptimizing the weights), yielding E weighted clouds whose losses are, by construction, approximately equal (each run is stopped at a similar plateau). We then merge them by pooling supports and aggregating weights. Because the loss is (approximately) convex (cf. (A5) in Appendix A.1), Jensen’s inequality gives

$$\mathcal{L}\left(\mathrm{merged}\right)\le {\displaystyle \frac{1}{E}}\sum _{e=1}^E\mathcal{L}\left({\mathcal{Z}}^{\left(e\right)}\right).$$

Since the ensemble members have nearly the same loss, the average on the right is essentially that common level; hence, the merged cloud achieves a loss that is no greater (and in practice often strictly lower) than that of any single member. The merged cloud may contain many negligible-weight points; we remove such points in the pruning step (Algorithm 3). This completes the reconstruction phase.

The reconstruction steps described above are closely related to the cross-entropy method (CEM) [102]. CEM is a global, gradient-free optimization scheme, widely used in evolutionary computation, that alternates between (i) generating a population of candidate solutions and evaluating their objective values and (ii) updating the sampling distribution so as to increase the probability of drawing elite samples. In our QMEM procedure, the weights $w_j$ play the role of performance-based importance assigned to candidates, analogous to fitness scores in CEM.

• Construction of Quantile Regions

Although TQF outputs a discrete empirical measure rather than a continuous geometric contour, one can straightforwardly construct highly flexible probability regions from the predictions. By applying KDE to the output point cloud, one can extract the Highest-Density Region (HDR) [103], covering a specified probability mass $\tau \in (0,1)$. Because the TQF point cloud captures complex topologies (including multimodality and holes), the resulting HDRs naturally form nonconvex or even disjoint regions. In this sense, they serve as useful alternatives to standard geometric multivariate quantile regions, practically overcoming the inherent convexity limitation of DQR.

5. Empirical Evaluation of the QMEM Algorithm

We assess the reconstruction fidelity of QMEM by comparing the reconstructed conditional distributions with the ground truth on synthetic datasets. The discrepancy is measured by ED ((A4) in Appendix A.1). The weight optimization for support points uses the SLSQP algorithm as implemented in the scipy.optimize package of Scipy [104].

In the first experiment, we use the 2D “two moons” dataset from scikit-learn [105], comprising 5000 samples with a Gaussian noise of standard deviation $0.1$. The dataset is shown in Figure 9a. We apply QMEM with parameters $K=25$, $M=25$, $N_0=9$, $N_1=150$, and $E=20$ mModern multi-slice CT scanners acquire on the order of ${10}^7$–${10}^8$ measurement points per rotation [106]. By contrast, our experiments use only $25\times 25=625$ points, i.e., a data volume smaller by ${10}^4$–${10}^5$ times. Consequently, reconstruction methods that are well-suited to clinical CT (with extremely dense measurements) are not appropriate in our markedly sparse setting). The computation took about 40 seconds on our machine (Intel Core i7-1165G7, 2.80 GHz, with 16 GB of RAM). No parallelization was used. The reconstructed points are overlaid on a smoothed rendering of the true distribution in Figure 9b–e, in the algorithmic order of progression; each panel reports the ED score (lower is better). The initial fit in (b) is poor, but accuracy improves as the iterations proceed, and the support points gradually concentrate around the support of the true distribution. Finally, merging the 20 clouds with pruning in (e) yields a marked improvement in accuracy, as is evident in the figure. The multimodal structure of the two moons is captured precisely.

What is the oracle value range for the ED score? To estimate this, we drew 2456 points with uniform weights from the ground-truth two-moons distribution, repeated over 20 random seeds, and computed the ED between each point set and the main dataset in Figure 9a. The average ED across trials was $0.0103$ with standard deviation $7.2\times {10}^{-4}$, reflecting the finite-sample bias of ED, which vanishes as the sample size tends to infinity. This indicates that the ED $0.0118$ achieved in Figure 9e is close to the optimal value for this dataset.

• Hyperparameter Dependence

To examine sensitivity to the hyperparameters, we varied K and M and measured performance. (Recall that K is the number of projection axes and M is the number of quantiles to match.) As shown in Figure 10, smaller K and/or M degrade reconstruction quality, as expected. In particular, the impact of a small M appears stronger than that of a small K. Figure 11 shows the reconstruction for $K=M=5$; fine-scale structure in the original dataset is missed, and ED is larger by a factor of $5.7$ relative to Figure 9e. The dependence on the number of pooled clouds is reported in Figure 12: larger E yields lower ED (i.e., higher quality) at higher computational cost, with performance appearing to saturate around $E=15–20$ on this dataset.

In the second experiment, we used the 2D letter “A” dataset, consisting of 2029 points with each coordinate standardized. The scatter plot is shown in Figure 13left. Using the same parameters as before, we reconstructed the distribution, yielding 2570 support points with optimized weights. A smoothed KDE of the reconstruction is shown in Figure 13right.

In the third experiment, we used the 2D letter “P” dataset, consisting of 2059 points with each coordinate standardized. The scatter plot is shown in Figure 14left. Using the same parameters as before, we reconstructed the distribution, yielding 2785 support points with optimized weights. A smoothed KDE of the reconstruction is shown in Figure 14right. The reconstructions in Figure 13 and Figure 14 appear to be of good quality.

6. Experimental Results for the TQF Algorithm

6.1. Evaluation on Synthetic Data I

In this subsection, we evaluate TQF on a synthetic dataset.

• Dataset

We draw $\mathbf{x}={\left(x^{\left(\mathcal{l}\right)}\right)}_{\mathcal{l}}\in {\mathbb{R}}^p$ from a uniform distribution over ${[-2,2]}^p$, compute the coordinate average ${\mathbf{x}}_{\mathrm{avg}}:={\displaystyle \frac{1}{p}}{\sum }_{\mathcal{l}=1}^p{x}^{\left(\mathcal{l}\right)}$, and set $a\left(\mathbf{x}\right):=\sigma \left(1.5{\mathbf{x}}_{\mathrm{avg}}\right)\in (0,1)$, where $\sigma$ denotes the sigmoid function. The function a is illustrated in Figure 15. Conditional on $a=a\left(\mathbf{x}\right)$, the target variable $\mathbf{y}=(y_1,y_2)$ is supported on a rectangle whose four vertices are $\left({\displaystyle \frac{s_1(1-a)-s_{2}a}{\sqrt{2}}},{\displaystyle \frac{s_1(1-a)+s_{2}a}{\sqrt{2}}}\right)$ with $s_1,s_2\in \{\pm 1\}$, and is further perturbed by small isotropic noise $\mathcal{N}(\mathbf{0},0.{08}^2{\mathbb{I}}_2)$. Snapshots of $\mathbf{y}$ are shown in Figure 16. The distribution is clearly non-Gaussian and highly non-unimodal, making it difficult to reconstruct from the marginals of $y_1$ and $y_2$ alone. For the experiment, we generate ${\left\{({\mathbf{x}}_i,{\mathbf{y}}_i)\right\}}_{i=1}^N$ with $p=2$ and $N=20,000$.

• Model

We configure TQF as follows. We compute all pairwise distances among the scalar target y (i.e., projection ${\mathbf{n}}^{\top }\mathbf{y}$ of the original targets) and choose the T parameters $\left\{w_t\right\}$ in (1) as the quantiles of this distance distribution at levels $(2t-1)/\left(2T\right)$ for $t=1,\dots ,T$. In our setup we use 50 trees, min_samples_leaf$=300$, $T=5$, $G=15$, and $\tilde{G}=10$. For QMEM we use $K=M=30$, $N_0=9$, $N_1=100$, and $E=20$. The training and output computation of TQF takes about 2 min on our machine.

• Evaluation Metric

ED is used to quantify the discrepancy between the predicted and the true distributions.

• Results

Numerical results are shown in Figure 17. We observe that TQF’s distributional predictions closely match the oracle data in Figure 16. In particular, TQF accurately recovers the holes in the support of the conditional distribution, which are known to be difficult to capture with conventional directional-quantile methods [32,33,34].

To probe the internal behavior of TQF, we compare its predictions with those of a k-nearest-neighbor (KNN) regressor trained on the original dataset (i.e., without projective augmentation). For a test pair $(\mathbf{x},\mathbf{n})$, KNN assigns weights to training samples based on their distances to $\mathbf{x}$, projects the corresponding responses via ${\mathbf{y}}_i\mapsto {\mathbf{n}}^{\top }{\mathbf{y}}_i$, and then computes weighted quantiles of ${\mathbf{n}}^{\top }{\mathbf{y}}_i$. As shown in Figure 18, the two methods produce closely matching estimates across varying $\mathbf{x}$ and n, providing quantitative evidence that TQF behaves as intended. (Unlike forest-based regressors, KNN regression typically degrades rapidly in high-dimensional settings or in the presence of many uninformative noisy features [57]. The present setting is favorable to KNN due to highly informative features; accordingly, KNN may not serve as a reliable baseline in more general tabular problems.)

We further compare TQF with several popular baselines for multivariate distribution estimation; the results are reported in

Table 3. Comparison of methods as measured by ED (lower is better), for the dataset in Figure 16. Scores are averaged over 10 random seeds for TQF, and 100 random seeds for the rest. Digits in parentheses denote one standard deviation. The best result in each row (except for Oracle), as determined using Welch’s t test at the 5% significance level, is shown in boldface.

a(x)	Point	Naïve	GMM1	GMM2	GMM3	TQF	Oracle
0.1	0.586(5)	0.205(5)	0.079(6)	0.035(5)	0.023(5)	0.038(7)	0.025(9)
0.2	0.601(5)	0.166(6)	0.090(7)	0.051(4)	0.040(4)	0.033(6)	0.027(8)
0.3	0.619(3)	0.127(6)	0.106(5)	0.072(4)	0.056(5)	0.038(9)	0.025(8)
0.4	0.633(2)	0.098(4)	0.118(5)	0.086(4)	0.055(4)	0.039(6)	0.025(8)
0.5	0.638(1)	0.086(3)	0.121(4)	0.094(4)	0.054(3)	0.043(12)	0.026(7)

. We show only the cases with $a\left(\mathbf{x}\right)\ge 0.5$ because the datasets for $a<0.5$ and $a>0.5$ are mirror images of each other (see Figure 16). In Table 3, “Point” denotes a non-probabilistic predictor that outputs only the mean of the ground-truth distribution. “Naïve” approximates the joint distribution as the product of marginals, $p(y_1,y_2|\mathbf{x})=p\left(y_1\right|\mathbf{x})p\left(y_2\right|\mathbf{x})$, thereby ignoring correlations. “GMM_k” fits a mixture of k Gaussians. “Oracle” reports ED computed between two datasets of size 2000 sampled from the ground-truth distribution at the same $\mathbf{x}$ using different random seeds. We emphasize that the GMM₁/GMM₂/GMM₃ entries achieve lower scores than standard Gaussian mixture regression because, in this experiment, the mixtures are fit directly to 2000 samples from the ground-truth distribution at fixed $\mathbf{x}$, and thus incur no regression error.

In Table 3, it is clear that TQF performs significantly better than the others, except for $a\left(\mathbf{x}\right)=0.1$, for which the true distribution is a relatively simple rectangle. It is intriguing that “Naïve” works better than GMM₁ and GMM₂ at $a\left(\mathbf{x}\right)=0.5$. Not surprisingly, the score of “Point” is always far worse than any other probabilistic method shown here.

The actual fitted distributions from all the methods are juxtaposed in Figure 19. We observe that the annulus shape of the true distribution is hard to approximate, even with a superposition of multiple Gaussians, while TQF provides an accurate estimation.

We systematically investigated the effects of sample augmentation and feature augmentation by varying the hyperparameters G and $\tilde{G}$. Varying G changes the effective training-set size for QRF++, and we therefore scale min_samples_leaf proportionally with G (e.g., min_samples_leaf$=20$ when $G=1$). The experimental results are reported in

Table 4. Hyperparameter dependence of TQF on the dataset in Figure 16, quantified by ED. Digits in parentheses denote one standard deviation over 9 trials with different random seeds. The best and the second-best results in each row are shown in boldface.

$\mathit{a}\left(\mathit{x}\right)$	$(\mathit{G},\tilde{\mathit{G}})$
	(15,1)	(15,5)	(15,10)	(15,15)	(1,10)	(5,10)	(10,10)	(20,10)
0.1	0.067(23)	0.042(16)	0.036(14)	0.039(18)	0.050(13)	0.046(22)	0.037(13)	0.053(26)
0.2	0.055(10)	0.037(5)	0.035(5)	0.038(4)	0.046(7)	0.035(6)	0.038(6)	0.036(5)
0.3	0.045(11)	0.046(11)	0.040(13)	0.043(11)	0.060(16)	0.043(10)	0.043(7)	0.045(14)
0.4	0.054(12)	0.045(10)	0.043(10)	0.045(10)	0.055(9)	0.045(8)	0.045(7)	0.041(7)
0.5	0.061(17)	0.048(17)	0.051(22)	0.051(18)	0.059(16)	0.053(16)	0.050(17)	0.051(17)

. The configurations $(G,\tilde{G})=(15,1)$ and $(1,10)$ yield noticeably worse scores than the other settings, suggesting that using both $G>1$ and $\tilde{G}>1$ is beneficial for TQF. Moreover, performance appears largely insensitive to the specific values of G and $\tilde{G}$, provided they exceed 1. Overall, Table 4 indicates that both sample augmentation (via G) and feature augmentation (via $\tilde{G}$) are important for TQF to achieve strong performance.

6.2. Evaluation on Synthetic Data II

Next, we test TQF on another synthetic dataset.

• Dataset

We draw $\mathbf{x}\in {\mathbb{R}}^{p_1+p_2}$ uniformly from ${[-2,2]}^{p_1+p_2}$ and map it to the unit interval via $a\left(\mathbf{x}\right):=\sigma \left(1.5{\mathbf{x}}_{\mathrm{avg}}^{\prime }\right)\in (0,1)$ (cf. Figure 15), where ${\mathbf{x}}_{\mathrm{avg}}^{\prime }:={\displaystyle \frac{1}{p_1}}{\sum }_{\mathcal{l}=1}^{p_1}{x}^{\left(\mathcal{l}\right)}$. The remaining components $(x^{(p_1+1)},\dots ,x^{(p_1+p_2)})\in {\mathbb{R}}^{p_2}$ are unused and serve as noise features. The conditional distribution $p\left(\mathbf{y}\right|\mathbf{x})$ forms a thick “7”-shaped pattern that is rotated about the origin by an angle proportional to $a\left(\mathbf{x}\right)$. Concretely, $\mathbf{y}\in {\mathbb{R}}^2$ is sampled uniformly from a fixed 7-shaped point cloud consisting of 1312 points, and the entire cloud is then rotated clockwise by ${\displaystyle \frac{\pi }{2}}a\left(\mathbf{x}\right)$ radians. The resulting data are visualized in Figure 20. In our experiments, we set $(p_1,p_2)=(2,3)$ and generate ${({\mathbf{x}}_i,{\mathbf{y}}_i)}_{i=1}^N$ with $N=20,000$. Unlike the dataset in Section 6.1, whose conditional mean is fixed at $\mathbf{y}=\mathbf{0}$, the conditional mean in the present dataset varies with $\mathbf{x}$.

• Model

We configure TQF as follows. We use 120 trees with min_samples_leaf $=100$, $G=20$, $\tilde{G}=10$, and $T=2$. We found TQF to be sensitive to the choice of the parameters $\left\{w_t\right\}$ in (1); after some trial and error, we set $(w_1,w_2)=(0.3,0.4)$. For QMEM, we use the same hyperparameters as in Section 6.1. The training and output computation of TQF takes 9 min on our machine.

• Results

Numerical results are shown in Figure 21. As $\mathbf{x}$ varies, even in the presence of noise features, TQF accurately captures the rotating, multimodal support of the conditional distribution; qualitatively, the overlap between the predictions and the ground truth appears satisfactory.

In Figure 22, we compare TQF with a KNN quantile regressor. For a fairer comparison, the KNN regressor uses only the first two components of $\mathbf{x}$ (i.e., the informative features) to compute neighbor distances. We observe good quantitative agreement between the two across a wide range of x and $\theta$.

Table 5. Same as Table 3 but for the dataset in Figure 20. The best results in each row (except for Oracle), as determined using Welch’s t test at the 5% significance level, are shown in boldface.

a(x)	Point	Naïve	GMM1	GMM2	GMM3	TQF	Oracle
0.1	0.920(8)	0.208(6)	0.196(7)	0.110(25)	0.058(9)	0.056(12)	0.035(11)
0.2	0.920(8)	0.207(6)	0.196(7)	0.110(25)	0.058(9)	0.050(14)	0.035(11)
0.3	0.920(8)	0.192(5)	0.196(7)	0.110(25)	0.058(9)	0.060(19)	0.035(11)
0.4	0.920(8)	0.164(4)	0.196(7)	0.110(25)	0.058(9)	0.063(16)	0.035(11)
0.5	0.920(8)	0.136(4)	0.196(7)	0.110(25)	0.058(9)	0.065(20)	0.035(11)

compares TQF with several baseline methods. For the “Oracle” column, we draw two bootstrap samples of size 1312 from the underlying 7-shaped point cloud (with replacement) and report ED between them. We note that the scores for “Point,” “GMM_k,” and “Oracle” are independent of $a\left(\mathbf{x}\right)$ because the true distribution is identical up to rotation. Overall, TQF performs comparably to GMM₃.

Figure 23 visualizes the predictive distributions produced by the probabilistic methods in Table 5, “Naïve,” GMM₁, GMM₂, GMM₃, and TQF (from left to right), together with the corresponding ED score. We see that the nonparametric TQF approach achieves performance comparable to GMM₃, which benefits from a carefully specified parametric form and sufficient mixture complexity. In contrast, “Naïve” and GMM_1,2 fail to capture finer geometric features of the true conditional distribution.

6.3. Comparing TQF and DRF on Small Data

As both TQF and DRF are tree-based models for distributional prediction, we conduct a numerical experiment in this subsection to highlight their differences. Recall that DRF is consistent under certain conditions ([45] Theorem 2), suggesting that, with sufficiently large training data, its predictions converge to the optimal (or near-optimal) conditional distribution. From the perspective of model comparison, it is therefore particularly informative to study the scarce-data regime. How do TQF and DRF perform when the training set is extremely small?

• Dataset

We construct a dataset ${(x_i,{\mathbf{y}}_i)}_{i=1}^N$ with only $N=30$ samples. Each $x_i\in \mathbb{R}$ is drawn uniformly from $[0,1]$. Conditional on $x_i$, we sample ${\mathbf{y}}_i\in {\mathbb{R}}^2$ uniformly from the unit disk centered at $(2x_i,2x_i)$. A scatter plot of the resulting ${\mathbf{y}}_i$ is shown in Figure 24.

• Model

To tune the hyperparameters of TQF and DRF, we generated 30 independent pairs of a training set (30 samples) and a validation set (1000 samples). We then averaged ES ((A10) in Appendix A.3) over the 30 runs to assess performance. For TQF, we used 10 trees, fixed $G=20$, $\tilde{G}=10$, $T=0$, $K=20$, $M=20$, and $E=10$, and varied min_samples_leaf over the grid $\{5,10,15,20,25,30\}$. We found min_samples_leaf$=15$ to perform best. For DRF, we used 10 trees and searched over the following hyperparameter grid:

• honesty: {True, False}.
• min_node_size: $\{1,2,\dots ,15\}$.
• sample_fraction: $\{0.3,0.5,0.7,0.9\}$.

We found DRF to be relatively stable with respect to these hyperparameters, consistent with the numerical results reported in Ref. [107]. The best configuration in our search was honesty $=\mathrm{True}$, min_node_size$=4$, and sample_fraction$=0.7$. For both TQF and DRF, increasing the number of trees did not yield a noticeable improvement in performance.

• Evaluation Metric

We generated 300 independent training datasets and trained TQF and DRF on each of them. For each trained model, we drew $x_{\mathrm{test}}\sim \mathrm{Unif}\left([0,1]\right)$ and produced a predictive distribution $\tilde{p}\left(\mathbf{y}\right|x_{\mathrm{test}})$. Prediction accuracy was quantified using ED and NLL, evaluated on 50,000 and 10,000 test samples, respectively, drawn from the true conditional distribution $p_{\mathrm{true}}\left(\mathbf{y}\right|x_{\mathrm{test}})$. To compute NLL numerically, we estimated the predictive density using gaussian_kde in SciPy [104] with default parameters. In addition, ES in (A10) was computed on a test set ${\left\{(x_k,{\mathbf{y}}_k)\right\}}_{k=1}^{1000}$.

• Results

The results are reported in

Table 6. Mean and standard deviation of performance metrics for 300 runs with different random seeds for the sliding-disk dataset. The bottom row shows p-values of the Wilcoxon signed-rank test. Best values are shown in boldface. Lower scores are better.

	ED (↓)	ES (↓)	NLL (↓)
TQF	0.290(133)	0.538(18)	2.01(69)
DRF	0.345(109)	0.544(21)	1.86(26)
p-value	$1.32\times {10}^{-13}$	$1.46\times {10}^{-6}$	0.238

and Figure 25. Table 6 shows that TQF achieves better ED and ES scores, as supported by the very small p-values from the Wilcoxon signed-rank test, which rejects the null hypothesis that the median scores of the two methods are equal. For NLL, DRF appears slightly better than TQF, although the difference is not statistically significant. NLL values should be interpreted with caution because they depend on the choice of KDE settings, such as the bandwidth.

Figure 25a visualizes the score distributions across the 300 runs. A number of outliers are present, especially for NLL, consistent with the general observation that NLL penalizes poor predictions more strongly than ES. Panels (b) and (c) compare representative predictive distributions produced by the two models trained on the same dataset. Because DRF can only reweight training samples and cannot generate new support points, its distributional predictions may deteriorate when the training data are scarce. In contrast, TQF produces denser predictions, since the QMEM reconstruction step is designed to generate support points flexibly beyond the observed samples.

6.4. Evaluation on Real-World Data

Next, we train the models on a real-world dataset and examine the characteristics of their distributional predictions.

• Dataset

We use the California Housing dataset [108] (https://scikit-learn.org/stable/modules/generated/sklearn.datasets.fetch_california_housing.html, accessed on 27 December 2025), which is based on the 1990 California census. It contains 20,640 records with eight covariates, and is commonly used for univariate regression, with the median house value (MedHouseVal) as the prediction target and the remaining attributes (MedInc, HouseAge, AveRooms, AveBedrms, Population, AveOccup, Latitude, Longitude) as features. In contrast, we use {Latitude, Longitude} as the prediction targets and treat the other seven variables (including MedHouseVal) as features. Although HouseAge is capped at 52 years and MedHouseVal at 5.00001 (in units of $100,000), we keep these values unchanged, since any additional treatment of censored observations could introduce bias into the subsequent analysis.

Although inherently spatial, we treat this dataset purely as a generic multi-output tabular regression task to demonstrate TQF’s general applicability, rather than competing with specialized spatial statistics methods.

The dataset is spatially imbalanced, with records concentrated around major metropolitan areas such as Los Angeles and the San Francisco Bay Area, which can distort the learning problem. To mitigate this effect, we apply a nonlinear quantile transformation (scikit-learn’s QuantileTransformer) to Longitude and Latitude so that their marginal distributions become approximately uniform on $[0,1]$. The transformed distribution is shown in Figure 26. Overall, higher house values tend to be observed closer to the coast.

• Models

We compare the following models: “Simple”, RF*, LightGBM*, KNN, GP, NGBoost, and TQF. Details are given below—models marked with * output point predictions:

• “Simple”: A trivial baseline that returns the entire training set as the predictive distribution for every test input.
• RF*: A random forest regressor with 200 trees. We tune min_samples_leaf by grid search over $\{1,5,10,20,50,100,200,300\}$ using the ${\mathcal{l}}_2$ loss, and select min_samples_leaf $=10$.
• LightGBM*: A gradient-boosted decision-tree model [20]. We tune the hyperparameters over the grids below, using the ${\mathcal{l}}_2$ loss:

  –

  learning_rate: $\{0.005,0.01,0.02,0.03,0.05,0.1\}$    → select $0.05$.

  –

  min_data_in_leaf: $\{1,5,10,20,50,100,200,300\}$    → select 50.

  The number of boosting rounds is chosen automatically via early stopping. Since LightGBM does not natively support multivariate targets, we train two independent models, one per target component.
• KNN: As we show below, RF feature importances suggest that MedHouseVal and AveOccup are the two most informative features. We standardize these two variables and train a KNN regressor using them. For a test input, we return the responses of the k nearest training points with uniform weights as the predictive distribution. Grid search selects $k=100$.
• GP: A Gaussian-process-based baseline. Because exact GP inference is prohibitively expensive for $N>{10}^4$, we use a scalable approximation: we generate 1000 random Fourier features using scikit-learn’s RBFSampler and fit a Bayesian linear model in the resulting feature space. To facilitate training, we first apply QuantileTransformer so that each feature is approximately standard normal. To obtain bivariate predictions, we fit two independent models (one per target component), each outputting a predictive mean and standard deviation. This independent modeling ignores cross-correlation between the two target components.
• NGBoost: Natural Gradient Boosting [37,38] (https://stanfordmlgroup.github.io/projects/ngboost/, accessed on 27 December 2025). It models predictive uncertainty by fitting a parametric distribution and optimizing its parameters using natural gradients. In our experiments, we use a bivariate normal distribution. We tune the hyperparameters over the grids below, using the ${\mathcal{l}}_2$ loss. We do not constrain max_depth. The following hyperparameters are used:

  –

  learning_rate: $\{0.001,0.005,0.01,0.02,0.05,0.1,0.2,0.3,0.4,0.5\}$    → select $0.1$.

  –

  min_samples_leaf: $\{5,10,20,50,100,200,300\}$    → select 100.

  The number of boosting rounds is determined via early stopping.
• TQF: We use 50 trees. We tune the hyperparameters over the grids below:

  –

  G: $\{5,10\}$    → select 10.

  –

  $\tilde{G}$: $\{1,5,10\}$    → select 1.

  –

  T: $\{0,3,6\}$    → select 3.

  –

  min_samples_leaf: $\{1,2,3,4,5,10,25,50,100,200,300,500\}$    → select 5.

• Evaluation Metric

We use ${\mathrm{R}}^2$ and ES to quantify mean-prediction accuracy and distributional-prediction accuracy, respectively. We perform 10-fold cross-validation (CV) and report average scores. During hyperparameter tuning, each model is trained on $8/9$ of the training data, with the remaining $1/9$ used as a validation set.

• Results

Table 7. Prediction performance on the California Housing dataset with bivariate targets (Longitude, Latitude). Parentheses report one standard deviation across the 10 CV folds for ${\mathrm{R}}^2$, and across all 20,640 test predictions aggregated over 10-fold CV for ES. Best scores are shown in boldface.

	Simple	RF*	LightGBM*	KNN	GP	NGBoost	TQF
${\mathrm{R}}^2$ $(\uparrow )$	$0.0\left(0\right)$	0.202(7)	$0.196\left(9\right)$	$0.094\left(9\right)$	$0.191\left(10\right)$	$0.191\left(7\right)$	0.204(8)
ES $(\downarrow )$	$0.244\left(90\right)$	$0.311\left(190\right)$	$0.314\left(189\right)$	$0.221\left(117\right)$	$0.229\left(122\right)$	0.215(122)	0.216(128)

summarizes the results. (For RF and LightGBM, ES is computed using the fact that, for point predictions, ES reduces to the mean absolute error.) Overall, this regression task is challenging: no method attains ${\mathrm{R}}^2$ above 0.3. The best-performing models in terms of ${\mathrm{R}}^2$ are RF and TQF, and their difference is not statistically significant ($p=0.22$; paired t-test). We also compute SHAP (https://shap.readthedocs.io/en/latest/index.html, accessed on 27 December 2025) values [109] for RF (Figure 27). The most important feature is MedHouseVal, which is consistent with the spatial pattern in Figure 26, followed by AveOccup (average occupancy per household).

For ES, the trivial baseline (“Simple”) attains 0.244, which serves as a natural reference level: any informative distributional predictor should achieve ES below 0.244. The best-performing models are NGBoost and TQF, with ES around 0.215–0.216, and their difference is not statistically significant ($p=0.44$; Wilcoxon signed-rank test).

Figure 28 illustrates how distributional predictions differ across the models. NGBoost, which fits a bivariate Gaussian, produces a narrow, tilted ellipse, highlighting the importance of the joint modeling of Longitude and Latitude. By contrast, the GP baseline fits the two components independently and therefore yields a broader, axis-aligned predictive distribution that ignores cross-correlation between the targets. For this particular test input, TQF produces a distribution with a shape qualitatively similar to that of NGBoost. The fact that the two methods achieve comparable ES on average, despite the nonparametric nature of TQF, may suggest that a unimodal Gaussian approximation is already adequate for capturing uncertainty in this dataset. Nevertheless, we emphasize that TQF attains the highest ${\mathrm{R}}^2$ among the tested models, indicating that its uncertainty estimation does not come at the expense of point-prediction accuracy.

7. Conclusions

In this paper, we presented Tomographic Quantile Forests (TQF), a new random forest-based regression model for multivariate targets that provides nonparametric predictive uncertainty estimates. To the best of our knowledge, this is the first work to implement CT-principle-based distributional prediction using tree-based models. Whereas conventional directional-quantile methods typically require training a separate model for each direction in the target space, TQF covers all directions within a single forest model. We hope that this work complements the rapidly growing literature on deep-learning-based uncertainty-aware modeling and encourages further research on more sophisticated non-neural methods tailored to tabular data.

It should be emphasized that TQF is positioned as a multivariate extension of tree-based regressors (e.g., QRF) specifically tailored for tabular data, rather than a full generative model.

TQF involves several hyperparameters, and tuning them can be time-consuming in practical applications; thus, making the method more robust to hyperparameter choices is an important direction for future research. Furthermore, while we relied heavily on synthetic data in this study to rigorously evaluate the recovery of ground-truth distributions, extensive benchmarking on diverse real-world datasets is a necessary next step. In this context, we plan to explore applications to complex, large-scale spatiotemporal data, such as those arising in weather forecasting, where comparisons with domain-specific spatial methods will be highly relevant.