EPSOM-Hyb: A General Purpose Estimator of Log-Marginal Likelihoods with Applications in Probabilistic Graphical Models

Abstract

We consider the estimation of the marginal likelihood in Bayesian statistics, with primary emphasis on Gaussian graphical models, where the intractability of the marginal likelihood in high dimensions is a frequently researched problem. We propose a general algorithm that can be widely applied to a variety of problem settings and excels particularly when dealing with near log-concave posteriors. Our method builds upon a previously posited algorithm that uses MCMC samples to partition the parameter space and forms piecewise constant approximations over these partition sets as a means of estimating the normalizing constant. In this paper, we refine the aforementioned local approximations by taking advantage of the shape of the target distribution and leveraging an expectation propagation algorithm to approximate Gaussian integrals over rectangular polytopes. Our numerical experiments show the versatility and accuracy of the proposed estimator, even as the parameter space increases in dimension and becomes more complicated.

1. Introduction

In Bayesian inference, many tasks, such as model selection, averaging, and comparison, rely on being able to compute the marginal likelihood (or model evidence) in order to form the Bayes factor. The marginal likelihood is the normalizing constant of a posterior distribution, which plays a particularly important role in probabilistic graphical models. Suppose $\gamma$ is a general Gibbs distribution defined on $\mathcal{U}\subseteq {\mathbb{R}}^d$, $\gamma \left(u\right)={\mathcal{Z}}^{-1}{e}^{-\Psi \left(u\right)}$, with 

$$\begin{array}{cc}\hfill & \mathcal{Z}={\int }_{\mathcal{U}}{e}^{-\Psi \left(u\right)}du<+\infty ,\mathcal{U}\subseteq {\mathbb{R}}^d.\hfill \end{array}$$

(1)

In a Bayesian setup, $\Psi :{\mathbb{R}}^d\to \mathbb{R}$ is the negative log posterior. Since $\Psi$ is typically complicated, and the space over which we are integrating tends to be high-dimensional, the resulting integral is usually intractable. This calculation is exacerbated further for Gaussian graphical models (GGM) [1,2] because the integral is taken over a specialized subset of positive definite matrices.

Generic algorithms for approximating the marginal likelihood include Laplace’s method [3], the harmonic mean estimator [4,5], the corrected arithmetic mean estimator [6], annealed importance sampling [7], Chib’s method [8], (warp) bridge sampling [9,10], and  nested sampling [11]. More recent developments include the integrated nested Laplace approximation [12] and the expectation propagation–approximate Bayesian computation algorithm [13]. While these methods can be applied to many marginal likelihood estimation problems, they do not provide a straightforward way to deal with the specificity of GGMs and the positive definite restriction on the precision matrices. Additionally, many of these aforementioned methods tend to rely on high-quality MCMC samples and have numerous problem-specific model settings that limit the overall practicality.

While there have been developments that specifically target the marginal likelihood of GGMs [14], the stringent prior restrictions (inverse-Wishart and hyper-inverse Wishart) prevent many of these approaches from being used in broader contexts. Furthermore, there exists substantial literature on inference for both decomposable [15] and non-decomposable [16,17,18] graphs; however, the need for more general and scalable algorithms is ever present. This has consequently led to dedicated methods for sampling from the G-Wishart distribution and conducting model comparison [19,20,21]. Also relevant is the availability of the G-Wishart normalizing constant in closed form [22], but the viability of these results is limited when considering most high-dimensional graphs of practical interest. Bhadra et al. [23] propose an application of Chib’s method and a telescoping decomposition of the precision matrix that simplifies the ensuing marginal likelihood calculations, but the main advantages of this approach are seen with element-wise priors, and in most cases, the time complexity is on par with other GGM-specific methods.

A recent approach that combines some of the probabilistic ideas from MCMC-based methods while drawing inspiration from quadrature is the hybrid estimator [24]. This method moves away from an over-reliance on MCMC samples, as these can be time-consuming to obtain if the likelihood is expensive to evaluate. Rather, the MCMC samples are utilized during a preliminary step to learn a partition of the parameter space in order to identify regions of posterior concentration. By making local approximations to the negative log posterior over these partition sets, the the hybrid algorithm simplifies the integral and forms a piecewise estimate of the marginal likelihood. The high-probability partition of the parameter space yields multiple benefits. First, the areas of the parameter space that admit finer partitions encourage precise approximations to the log posterior. More importantly, this partitioning routine redirects attention away from regions that have little to no contribution to the posterior distribution, saving both time and computation.

The hybrid estimator establishes the foundation for a promising way to estimate the marginal likelihood, and the experimental results demonstrate its competitiveness with other well-known estimators in various problem settings [24]. Moreover, the methodology’s generality makes it convenient to refine the algorithm so that it can be applied to more specific problem setups. With the ultimate goal of accurately computing the marginal likelihood for GGMs, we introduce an alternative parametrization of precision matrices and restrict our scope to target the normalizing constants of a specific class of densities—unimodal densities that are approximately log-concave around the mode. In this paper, we leverage core ideas of the hybrid estimator in conjunction with higher order approximations to the negative log posterior for accuracy, as well as expectation propagation techniques for scalability. This results in a novel estimator that is suitably equipped for high-dimensional problems. We call the estimator that arises from these modifications the EP-guided second-order modified hybrid estimator (EPSOM-Hyb).

After verifying the accuracy of EPSOM-Hyb in initial experiments for decomposable graphs, we further extend the methodology to better handle non-decomposable GGMs. By incorporating junction tree (JT) representations of connected graphs, we enhance the computational efficiency of the marginal likelihood calculation. This leads to a GGM-specific estimator, denoted EPSOM-HybJT. Our contribution is multifaceted; despite the introduction of higher-order terms that inherently bring additional computational challenges to the hybrid estimator framework, we maintain practical utility in the EPSOM-Hyb methodology. This approach leads to an estimator that not only excels in highly specific problem settings, but also outperforms well-established estimators. Our experiments indicate that EPSOM-HybJT is more accurate and more than 100 times faster in higher dimensions than other viable estimators.

The outline of the paper is as follows. In Section 2, we provide the background for the hybrid estimator so that the modifications that we propose in Section 3 to formulate the EPSOM-Hyb estimator have relevant context. In Section 4, we investigate the performance of the EPSOM-Hyb estimator for GGMs, and we also develop the EPSOM-HybJT estimator. In Section 5, we conclude and briefly discuss future work.

2. The Hybrid Estimator

We first review the salient details of the hybrid estimator as a means of providing background on the existing work and establishing context for our development of the EPSOM-Hyb estimator. As an initial step, the hybrid algorithm learns a dyadic partition of the parameter space that helps identify high-probability regions of the target distribution. In order to accomplish this, we form covariate–response pairs, ${\left\{\left(u_j,\Psi \left(u_j\right)\right)\right\}}_{1\le j\le J}$, using MCMC samples from the target distribution $\gamma$ and evaluating them with $\Psi$. Clearly, we make the assumption of being able to sample from $\gamma$ and evaluate $\Psi$, but both of these are basic requirements in many MCMC-based algorithms. We can then use a regression tree algorithm to obtain optimal splits of the parameter space, where each of the partition sets is the hyper-rectangle that defines the corresponding leaf node in the fitted tree. The hybrid estimator uses the classification and regression tree (CART) algorithm [25], which returns a partition of the parameter space that can be further restricted to the compactification A of $\mathcal{U}$ by setting A to be bounding box defined by the range of the posterior samples: $A={\prod }_{l=1}^d[{min}_j\left\{u_j^{\left(l\right)}\right\},{max}_j\left\{u_j^{\left(l\right)}\right\}]$, where $u_j^{\left(l\right)}$ is the l-th coordinate of the j-th sample. Thus, the resulting partition $\mathcal{A}$ is a dyadic partition of the compact set A. We work with a compactification of the parameter space as a means of eliminating low-probability regions of the domain whose contributions to the integral in Equation  (1) are negligible. This is particularly useful in Bayesian contexts where the posterior concentrates with increasing sample size [26,27].

Next, we formulate the piecewise estimator to $\Psi$ defined over the compact set $A\subseteq \mathcal{U}$,

$$\begin{array}{cc}\hfill & \Psi \left(u\right)\approx \widehat{\Psi }\left(u\right)=\sum _{k=1}^K{\widehat{\Psi }}_k\left(u\right){\mathbb{1}}_{A_k}\left(u\right).\hfill \end{array}$$

(2)

where $\mathcal{A}=\{A_1,\dots ,A_K\}$, $A={\bigcup }_{k=1}^K{A}_k$, and $A_k\cap A_{k^{\prime }}=\varnothing$ for all $k\ne k^{\prime }$. We set ${\widehat{\Psi }}_k\left(u\right)=c_k^{★}$, a representative point in $A_k$ that is constant in u. Since each $A_k={\prod }_{l=1}^d[a_k^{\left(l\right)},b_k^{\left(l\right)}]$ is a hyper-rectangle, the hybrid estimator reduces to

$$\begin{array}{c}\hfill {\int }_A{e}^{-\Psi \left(u\right)}du\approx {\int }_A{e}^{-\widehat{\Psi }\left(u\right)}du=\sum _{k=1}^K{e}^{-c_k^{★}}·\mu \left(A_k\right)=:{\widehat{\mathcal{Z}}}_{\mathrm{Hyb}}\end{array}$$

(3)

Here, $\mu \left(B\right)={\int }_{B}1du$ denotes the d-dimensional volume of a set B.

3. The Epsom-Hyb Estimator

In the discourse that follows, we propose some key modifications to the hybrid algorithm to develop a novel estimator better suited for tackling higher-dimensional problems. For the initial development, we do not restrict our analysis to posterior distributions and illustrate the modified algorithm in the general case of log-concave target densities. Our additional assumption about the shape of $\Psi$ is employed only to ensure that the Hessian is positive definite everywhere and can, in fact, be relaxed to unimodal densities that are approximately log-concave in a suitable neighborhood around the mode. This assumption is widely satisfied by Bayesian posteriors in regular parametric models, and due to the Bernstein–von Mises phenomenon, most regular posterior distributions are approximately log-concave with sufficiently large sample size [28].

As before, we work with a compactification of the parameter space, for which we have a dyadic partition $\mathcal{A}$ obtained through the same tree-fitting procedure as before. However, instead of the constant approximation to $\Psi$ within each partition set $A_k$ in $\mathcal{A}$, we first retain the general estimator given in Equation (2),

$$\begin{array}{cc}\hfill & \mathcal{Z}\approx {\int }_A{e}^{-\Psi \left(u\right)}du\approx {\int }_A{e}^{-\widehat{\Psi }\left(u\right)}du=\sum _{k=1}^K{\int }_{A_k}{e}^{-{\widehat{\Psi }}_k\left(u\right)}du.\hfill \end{array}$$

(4)

Below, we highlight three fundamental features of the EPSOM-Hyb algorithm that distinguish it from the hybrid algorithm, followed by detailed discussions of each step. See Algorithm 1 for a formal statement of the EPSOM-Hyb estimation procedure.

• In Equation (2), we take ${\widehat{\Psi }}_k$ to be a local approximation to $\Psi$ constructed using a second-order Taylor expansion of $\Psi$ around a representative point $u_k\in A_k$.
• The second-order Taylor approximation complicates the calculation of the integrals in Equation (4). We address the intractability of the resulting integral using an expectation propagation (EP) algorithm that targets high-dimensional Gaussian integrals.
• We exploit the unimodality of $\gamma$ to identify suitable points within each partition set around which we perform the Taylor expansion required for ${\widehat{\Psi }}_k$.

Algorithm 1: EPSOM-Hyb

3.1. Local Approximation Using a Taylor Expansion

We revisit Equation (2) and consider the following piecewise quadratic approximation to $\Psi$,

$$\begin{array}{cc}\hfill \Psi \left(u\right)& \approx \sum _k\left[\Psi \left(u_k\right)+{\left(u-u_k\right)}^{\prime }\nabla \Psi \left(u_k\right)+\frac{1}{2}{\left(u-u_k\right)}^{\prime }{\nabla }^2\Psi \left(u_k\right)\left(u-u_k\right)\right]{\mathbb{1}}_{A_k}\left(u\right),\hfill \end{array}$$

(5)

where $u_k$ is a representative point of $A_k$. This improves upon the initial idea of the piecewise constant approximation in the hybrid methodology by introducing a second-order Taylor approximation. By introducing higher-order terms, we trade the convenient simplification of the integral in Equation (3) for increased local accuracy of the approximation. Upon exponentiating the approximation in Equation (5), we observe that the second exponential in the summation below is proportional to a Gaussian density,

$$\begin{array}{c}\hfill e^{-\widehat{\Psi }\left(u\right)}=\sum _{k}exp\left\{-\Psi \left(u_k\right)+u_k^{\prime }{\lambda }_k-\frac{1}{2}{u}_k^{\prime }{H}_k{u}_k\right\}exp\left\{-\frac{1}{2}{u}^{\prime }{H}_{k}u+{(H_k{u}_k-{\lambda }_k)}^{\prime }u\right\}{\mathbb{1}}_{A_k}\left(u\right),\end{array}$$

where ${\lambda }_k:=\nabla \Psi \left(u_k\right)$ and $H_k:={\nabla }^2\Psi \left(u_k\right)$ represent the gradient vector and the Hessian matrix of $\Psi$ evaluated at $u_k$, respectively. Since $\gamma$ is log-concave, the Hessian matrix is positive definite, and hence invertible. Integrating the exponential above, we arrive at the following approximation to $\mathcal{Z}$:

$$\begin{array}{c}\hfill {\int }_A{e}^{-\Psi \left(u\right)}du\approx {\int }_A{e}^{-\widehat{\Psi }\left(u\right)}du=\sum _k{C}_k{\int }_{A_k}\mathcal{N}\left(u\mid H_k^{-1}{b}_k,H_k^{-1}\right)du=:{\widehat{\mathcal{Z}}}_{\mathrm{EPSOM}-\mathrm{Hyb}}.\end{array}$$

(6)

Here, $C_k=exp\left(-\Psi \left(u_k\right)+{\lambda }_k^{\prime }{u}_k-0.5u_k^{\prime }{H}_k{u}_k+0.5m_k^{\prime }{H}_k{m}_k\right)$, $b_k=H_k{u}_k-{\lambda }_k$, and $m_k=H_k^{-1}{b}_k$. Before ${\widehat{\mathcal{Z}}}_{\mathrm{EPSOM}-\mathrm{Hyb}}$ can serve as a viable estimator for $\mathcal{Z}$, we must address the remaining questions of how to determine the value of each $u_k\in A_k$ and how to compute the intractable Gaussian integrals in Equation (6). In the next two sections, we provide scalable methods to accomplish these tasks. It is worth noting that the original hybrid estimator does not face the same intractability issue due to the simplistic nature of the constant approximation used in Equation (3). Our incorporation of higher-order terms complicates the subsequent calculation, which necessitates a scalable algorithm like EP to compute the truncated Gaussian integrals.

3.2. Estimating Truncated Gaussian Probabilities

Despite the prevalence of Gaussian densities in statistical modeling, multivariate Gaussian probabilities remain difficult to compute, as they typically require high-dimensional integration. Numerical integration [29] is a common solution, but it is prohibitively expensive in the number of points required, thus preventing it from being a scalable solution in the high-dimensional space in which we wish to operate. A more recent alternative that specifically targets truncated normal distributions is the minimax tilting method [30], and while we used this method in the initial development of our algorithm, it failed to produce accurate results beyond trivial settings. This ultimately led us to consider more computationally efficient integration techniques, such as expectation propagation [31], which is widely used to compute approximate integrals. To better understand how the expectation propagation (EP) algorithm can be applied to the intractable Gaussian integral in the previous section, we lay out some preliminary groundwork for the EP algorithm. Starting with the Gaussian distribution $p_0\left(x\right)=\mathcal{N}\left(x\mid m,K\right)$, we define the un-normalized truncated distribution $p\left(x\right)=p_0\left(x\right){\mathbb{1}}_{\mathcal{A}}\left(x\right)$. The Gaussian probability of interest can be written as:

$$F\left(\mathcal{A}\right)={\int }_{\mathcal{A}}{p}_0\left(x\right)dx=\int p\left(x\right)dx.$$

(7)

Cunningham et al. [32] propose a framework for approximating $F\left(\mathcal{A}\right)$ that consists of two main approximations. The first approximation is a tractable distribution q that minimizes the Kullback–Leibler (KL) divergence between p and q, denoted $D\left(p\left|\right|q\right)$. However, the intractability of p complicates the minimization and subsequently requires another approximation, for which we instead work with a simplified representation of p. Namely, we replace $p\left(x\right)$ with the product of a prior distribution $p_0\left(x\right)$ and factors $t_i\left(x\right)$, such that

$$\begin{array}{c}\hfill p\left(x\right)=p_0\left(x\right)\prod _i{t}_i\left(x\right).\end{array}$$

In order to facilitate tractability, we assume $t_i\left(x\right)=\mathbb{1}\left\{a_i<x_i<b_i\right\}$, where $x_i$ is the i-th coordinate of x, and $a_i,b_i$ are simply the lower and upper bounds of integration, respectively. Then, the target integral in Equation (7) can be written as

$$\begin{array}{c}\hfill F\left(\mathcal{A}\right)=\int p\left(x\right)dx=\int p_0\left(x\right)\prod _{i=1}^d{t}_i\left(x_i\right)d{x}_i.\end{array}$$

We proceed to approximate each of the intractable factors $t_i$ with a tractable, un-normalized Gaussian ${\tilde{t}}_i\left(x\right)$, which produces the final approximation q of p. More specifically, we take q to mirror the product form of p,

$$\begin{array}{c}\hfill q\left(x\right)=p_0\left(x\right)\prod _i{\tilde{t}}_i\left(x\right)=p_0\left(x\right)\prod _i{\tilde{Z}}_i\mathcal{N}\left(x\mid {\tilde{\mu }}_i,{\tilde{\sigma }}_i^2\right)=Z\mathcal{N}\left(x\mid \mu ,\Sigma \right),\end{array}$$

where the parameters of these unnormalized Gaussian distributions, $\{{\tilde{\mu }}_i,{\tilde{\sigma }}_i^2,{\tilde{Z}}_i\}$, admit closed form updates that are the result of an iterative moment matching scheme. From this, we observe that by estimating the normalizing constant of q, we also obtain an approximation for the normalizing constant of the target distribution p. See Equations (21)–(23) from Cunningham et al. [32] for the closed-form updates for each of these parameters and more details regarding the relevant notation. Essentially, the EP algorithm iteratively constructs the approximating distribution $q\left(x\right)$ to minimize $D\left(t_i{q}^{\setminus i}\left|\right|{\tilde{t}}_i{q}^{\setminus i}\right)$, which in turn approximately minimizes $D\left(p\left|\right|q\right)$. Here, $q^{\setminus i}\left(x\right)=q\left(x\right)/{\tilde{t}}_i\left(x\right)$ is defined as the cavity distribution. By running the EP algorithm to convergence, we can calculate the following mean and covariance parameters of the normal distribution q,

$$\begin{array}{c}\hfill \mu =\Sigma \left(K^{-1}m+\sum _{i=1}^d\frac{{\tilde{\mu }}_i}{{\tilde{\sigma }}_i^2}{e}_i\right),\Sigma ={\left(K^{-1}+\sum _{i=1}^d\frac{1}{{\tilde{\sigma }}_i^2}{c}_i{c}_i^{\prime }\right)}^{-1},\end{array}$$

where $e_i$ is the i-th standard basis vector. With this, we also obtain a closed-form expression for the normalizing constant of q,

$$\begin{array}{cc}\hfill logZ=& -\frac{1}{2}\left(m^{\prime }{K}^{-1}m+log\left|K\right|\right)+\sum _{i=1}^d\left(log{\tilde{Z}}_i-\frac{1}{2}\left(\frac{{\tilde{\mu }}_i^2}{{\tilde{\sigma }}_i^2}+log{\tilde{\sigma }}_i^2+log\left(2\pi \right)\right)\right)\hfill \\ \hfill & +\frac{1}{2}\left({\mu }^{\prime }{\Sigma }^{-1}\mu +log|\Sigma |\right),\hfill \end{array}$$

which approximates the Gaussian probability $F\left(\mathcal{A}\right)$ in Equation (7). It is worth noting that the algorithm is still valid for arbitrary factors $t_i\left(x\right)$, albeit with different parameter updates. Our simplistic choice of $t_i\left(x\right)$ to be the indicator function defined over the constant lower and upper limits of integration reflects the rectangular nature of the partition sets used in the Hybrid-EP estimator.

With this, recall the Gaussian integral in Equation (6),

$${\int }_{A_k}\mathcal{N}\left(u\mid H_k^{-1}{b}_k,H_k^{-1}\right)du.$$

(8)

Taking $p_0\left(u\right)\equiv \mathcal{N}\left(u\mid H_k^{-1}{b}_k,H_k^{-1}\right)$, we observe that the above integral is exactly the one given in Equation (7). Thus, we can directly use the $\mathrm{EPMGP}$ algorithm to obtain an estimate for the Gaussian probability in Equation (8). Another benefit of using the EPMGP algorithm is its exceptional accuracy when the constraint set is rectangular, which coincides with our setup, where $A_k={\prod }_{l=1}^d[a_k^{\left(l\right)},b_k^{\left(l\right)}]$ is a d-dimensional hyper-rectangle.

3.3. Selecting the Candidate Point in Each Partition Set

The final piece of EPSOM-Hyb is the point $u_k$ used in the piecewise Taylor approximation $\widehat{\Psi }$ in Equation (5). In our unimodal setup, a natural choice for each partition set’s representative point is one that is closest to the global mode of $\gamma$. More specifically, if $u_0$ is the global mode, then $u_k={\mathrm{argmin}}_{u\in A_k}{\parallel u-u_0\parallel }_1$. Conveniently, we can obtain the global mode of $\gamma$ using Newton’s method with little additional effort because we already have expressions for the gradient and Hessian of $\Psi$ as part of the approximation in Equation (6). With this, all components of EPSOM-Hyb are accounted for and can be summarized in Algorithm 1.

4. Application to Gaussian Graphical Models

Next, we investigate the performance of EPSOM-Hyb on marginal likelihood estimation for GGMs. Let $G=(V,E)$ be an undirected graph with vertex set $V=\{1,\dots ,p\}$ and edge set E. Define ${\mathbb{S}}^p$ as the set of symmetric $p\times p$ matrices and ${\mathbb{S}}_{\succ 0}^p$ as the cone of positive definite $p\times p$ matrices in ${\mathbb{S}}^p$. Let $X\sim \mathcal{N}\left(\mu ,\Sigma \right)$, ${\Sigma }^{-1}\in {\mathbb{S}}_{\succ 0}^p\left(G\right)$, where ${\mathbb{S}}_{\succ 0}^p\left(G\right)=\{M=\left(M_{ij}\right)\in {\mathbb{S}}_{\succ 0}^p\mid M_{ij}=0,\forall (i,j)\notin E\}$. Then, the likelihood can be written as follows,

$$\begin{array}{cc}\hfill & L\left(\Sigma \right)={\left(2\pi \right)}^{-np/2}det{\left(\Sigma \right)}^{-n/2}{e}^{-tr\left({\Sigma }^{-1}B\right)/2},B=\sum _{i=1}^n{x}_i{x}_i^{\prime }.\hfill \end{array}$$

(9)

Further, X satisfies the GGM with graph G, where G dictates the conditional dependence structure and restricts the sparse concentration matrix $\Omega ={\left({\omega }_{ij}\right)}_{p\times p}={\Sigma }^{-1}$ so that $(i,j)\in E$ if and only if ${\omega }_{ij}\ne 0$, and $x^{\left(i\right)}$ and $x^{\left(j\right)}$ are conditionally independent if and only if ${\omega }_{ij}=0$. Hence, an undirected graphical model corresponding to G restricts the inverse covariance matrix $\Omega$ to a linear subspace of the cone of positive definite matrices.

We first consider decomposable GGMs where the tractability of the marginal likelihood makes it easy to evaluate estimates. In Section 4.2, we broaden our focus to general, non-decomposable graphs. To address the additional computational challenges that accompany non-decomposable graphs and high-dimensional parameter spaces, we propose an extension to EPSOM-Hyb that facilitates dimension reduction and drastically reduces the time complexity of the normalizing constant calculation. In the following GGM experiments, we refer to the estimator from Atay-Kayis and Massam [17] as GNORM, which is well-established for general graphs and can be computed easily using BDgraph [33].

4.1. Hyper Inverse-Wishart Induced Cholesky Factor Density

We first consider the case where G is decomposable. The hyper-inverse Wishart (HIW) distribution [34] for $\Sigma ={\Omega }^{-1}$ and the corresponding induced class of distributions [14] for $\Omega$ are attractive choices for priors. Given G, we place a hyper-inverse Wishart prior ${\mathrm{HIW}}_G\left(\delta ,\Lambda \right)$ on $\Omega ={\Sigma }^{-1}$, where $\delta >2$ is the degrees of freedom and $\Lambda \in {\mathbb{S}}_{\succ 0}^p$ is fixed. The HIW distribution is defined over the cone of $p\times p$ positive definite matrices, with the corresponding density:

$$\begin{array}{c}\hfill f\left(\Omega \mid G\right)\propto {|\Omega |}^{(\delta -2)/2}{e}^{-tr\left(\Omega \Lambda \right)/2}.\end{array}$$

(10)

Despite being able to sample from the posterior distribution, ${\mathrm{HIW}}_G\left(\delta +n,\Lambda +B\right)$, we cannot directly carry out the EPSOM-Hyb algorithm. Since samples are drawn from a sub-manifold of ${\mathbb{R}}^{p\times p}$, proceeding to obtain a partition over ${\mathbb{R}}^{p\times p}$ does not guarantee that a given point in the partition can be reconstructed to form a valid covariance matrix. As such, we circumvent this issue by working with an alternative representation of $\Sigma$ given by the Cholesky decomposition ${\Sigma }^{-1}=\Omega ={\varphi }^{\prime }\varphi$. Provided that the vertices of G are enumerated according to a perfect vertex elimination scheme, the upper triangular matrix $\varphi$ observes the same sparsity as $\Omega$. Consequently, we need only compute the induced prior on the nonzero elements of $\varphi$, which, together with the likelihood in Equation (9), gives us an explicit expression for the negative log posterior $\Psi$. The determinant of the Jacobian matrix J of the transformation $\Omega \to \varphi$ is given by $\left|J\right|=2^p{\prod }_{i=1}^p{\varphi }_{ii}^{{\nu }_i+1}$, where the i-th row of $\varphi$ has exactly ${\nu }_i+1$ nonzero elements [14]. Further, the distribution of $\Sigma$ has the strong hyper-Markov property [35], so we can ascertain mutual independence of the rows of $\varphi$. Then, we can specify the joint density of the free elements of $\varphi$ as follows,

$$\begin{array}{cc}\hfill \pi \left(\varphi \right)& =\left[\prod _{i=1}^p\frac{2^{-(\delta +{\nu }_i)/2}}{\Gamma \left((\delta +{\nu }_i)/2\right)}{\varphi }_{ii}^{\delta +{\nu }_i-2}{e}^{-\frac{1}{2}{\varphi }_{ii}^2}\left(2{\varphi }_{ii}\right)\right]\times \left[\prod _{(r,s):s>r,(v_s,v_r)\in E}\frac{1}{\sqrt{2\pi }}{e}^{-\frac{1}{2}{\varphi }_{rs}^2}\right].\hfill \end{array}$$

(11)

Recall that we also require expressions for the gradient and Hessian of $\Psi$, both of which are provided in Appendix D. Putting these ideas together, we can employ the EPSOM-Hyb estimation framework by drawing samples from the posterior distribution, taking the Cholesky factor of each sample, and computing $\Psi \left(\varphi \right)$ using the likelihood in Equation (9) and the prior in Equation (11). While this procedure appears to be quite simple, the implications are significant; if we have a different prior on $\Sigma$ for which we can do the posterior computation, all other aspects of the algorithm would remain the same. All that is required is a way to sample from the posterior of $\Omega$ and an expression for the Jacobian of the corresponding transformation.

In our simulations, we emulate a high-dimensional setting by stacking the adjacency matrix of the decomposable graph $G_9$ in Figure 1 multiple times along a block diagonal to construct larger graphs, e.g., $A_{G_p}=I_r\otimes A_{G_9}$, where $A_{G_p}$ is the adjacency matrix of $G_p$ and r is the number of times we stack the graph. For $r=8,10$, we form the graphs $G_{72} \mathrm{and} G_{90}$, which have $d=200,250$ free elements, respectively. Drawing data that satisfy their corresponding GGMs and taking $\delta =3,n=100$, and $\Lambda =I_p$, we can sample from the posterior distribution and compute the marginal likelihood estimates. Among the generic methods mentioned in Section 1, only the bridge sampling estimator (BSE) and the warp bridge sampling estimator (WBSE) are viable and accurate methods for computing the marginal likelihood. As seen in

Table 1. Mean, average error (AE), and root mean squared error (RMSE) for the approximations to the log normalizing constant of the ${\mathrm{HIW}}_G\left(\delta +n,I_p+B\right)$ distribution. $G_{72}$ has 72 vertices with 200 parameters and $G_{90}$ has 90 vertices and 250 parameters. Estimates are reported over 100 replications, each using 1000 samples from the true posterior.

Dimension	Method	Mean	AE	RMSE
$p=72,d=200$	EPSOM-HybJT	−6230.59	0.41	0.45
	BSE	−6230.56	−0.74	1.93
	WBSE	−6230.59	−0.70	2.22
$p=90,d=250$	EPSOM-HybJT	−7881.46	0.51	0.56
	BSE	−7875.70	−5.26	6.31
	WBSE	−7875.76	−5.19	6.09

, both BSE and WBSE are competitive with EPSOM-Hyb when $d=200$, but the quality of the former two estimators deteriorates when $d=250$. The relatively small error of EPSOM-Hyb in this high-dimensional graphical models setting is particularly encouraging. Interestingly, GNORM fails to produce sensible estimates for these high-dimensional graphs.

4.2. G-Wishart Prior for General Graphs

In contrast with the previous section, we now broaden the scope of examples and consider general, non-decomposable graphs. For the conditional general graph G, a popular choice for the prior is the G-Wishart prior on $\Omega$, $GW\left(\delta ,\Lambda \right)$, which has the following density,

$$\begin{array}{c}\hfill f\left(\Omega \mid G\right)\propto {|\Omega |}^{(\delta -2)/2}{e}^{-\mathrm{tr}\left(\Omega \Lambda \right)/2}.\end{array}$$

(12)

Here, $\Omega ,\Lambda$ are $p\times p$ positive definite matrices, and $\delta >2$. While the density expression is similar to that of the HIW density, the tractability of the normalizing constant is no longer assumed. The normalizing constant that we wish to compute is of the form

$$\begin{array}{cc}\hfill C_G\left(\delta ,\Lambda \right)& =2^{\frac{1}{2}p\delta +\left|E\right|}·I_G\left((\delta -2)/2,\Lambda \right),\hfill \end{array}$$

(13)

where $I_G\left(\delta ,\Lambda \right)={\int }_{{\mathbb{S}}_{\succ 0}^p}{|\Omega |}^{\delta }exp\left(-tr\left(\Omega \Lambda \right)\right)d\Omega$. Here, $d\Omega ={\prod }_{i=1}^{p}d{\omega }_{ii}·{\prod }_{i<j,(i,j)\in E}d{\omega }_{ij}$.

G-Wishart-Induced Cholesky Factor Density

We first establish some of the notations relevant to the GW density. For $G=(V,E)$, define $\mathcal{V}=\left\{\right(i,j):i\le j\mathrm{where}i=j,i\in V\mathrm{or}(i,j)\in E\}$ and $\mathcal{W}=\left\{\right(i,j):i,j\in V,i\le j\}.$ Then, let $\overline{\mathcal{V}}=\mathcal{W}\setminus \mathcal{V}$, and $A_{p\times p}=\left(a_{ij}\right)$, where $a_{ij}=0$ if $(i,j)\in \overline{\mathcal{V}}$ or if $i=j$, and $a_{ij}=1$ otherwise. Set $k_i$ to be the number of 1’s in the i-th column of A. Proceeding in a similar fashion as in the HIW example, we take the Cholesky decomposition of $\Omega ={\varphi }^{\prime }\varphi$ and $\Lambda ={\left(T^{\prime }T\right)}^{-1}$, where $T={\left(t_{ij}\right)}_{1\le i\le j\le p}$. We make an additional change of variable $\zeta =\varphi T^{-1}$. The Jacobian of the first transformation $\Omega \to \varphi$ is identical to the one given in Section 4.1, and the determinant of the Jacobian of the second transformation $\varphi \to \zeta$ is given by ${\prod }_{i=1}^p{t}_{ii}^{k_i+1}$. Putting this all together, we can rewrite the normalizing constant as an integral over the free variables of $\zeta ={\left({\zeta }_{ij}\right)}_{1\le i\le j\le p}$,

$$\begin{array}{cc}\hfill C_G\left(\delta ,\Lambda \right)& =2^p\prod _{i=1}^p{\left(t_{ii}^2\right)}^{(\delta +b_i-1)/2}\int exp\left(-\frac{1}{2}\sum _{(i,j)\in \overline{\mathcal{V}}}{\zeta }_{ij}^2\right)\prod _{i=1}^p{\left({\zeta }_{ii}^2\right)}^{(\delta +{\nu }_i-1)/2}exp\left(-\frac{1}{2}\sum _{i=1}^p{\zeta }_{ii}^2\right)\hfill \\ \hfill & \times exp\left(-\frac{1}{2}\sum _{(i,j)\in \mathcal{V},i\ne j}{\zeta }_{ij}^2\right)\prod _{i=1}^{p}d{\zeta }_{ii}\prod _{(i,j)\in \mathcal{V},i\ne j}d{\zeta }_{ij},\hfill \end{array}$$

(14)

where $b_i={\nu }_i+k_i+1$, ${\nu }_i=|\mathrm{ne}\left(i\right)\cap \{i+1,\dots ,p\}|$, and $\mathrm{ne}\left(i\right)=\{j\in V:(i,j)\in E\}$. Taking the logarithm of the integrand above, we can write the following expression for $\Psi \left(\zeta \right)$,

$$\begin{array}{cc}\hfill & \Psi \left(\zeta \right)=C+\sum _{i=1}^p(\delta +{\nu }_i-1)log{\zeta }_{ii}-\frac{1}{2}\sum _{i=1}^p{\zeta }_{ii}^2-\frac{1}{2}\sum _{(i,j)\in \overline{\mathcal{V}}}{\zeta }_{ij}^2-\frac{1}{2}\sum _{(i,j)\in \mathcal{V},i\ne j}{\zeta }_{ij}^2,\hfill \end{array}$$

(15)

where $C=plog\left(2\right)+{\sum }_{i=1}^p(\delta +b_i-1)log{t}_{ii}$. Another key difference between the GW and HIW setups is how the non-free elements interact with the objective function $\Psi$. In the case of decomposable G, the Cholesky factor $\varphi$ observes the same sparsity pattern as the adjacency matrix of G. This simplifies the evaluation of $\Psi$, as $\Psi$ can then be evaluated over the nonzero elements of $\varphi$. However, in the case of non-decomposable G, the sparsity in G is not necessarily reflected in the Cholesky factors. Therefore, $\Psi$ has nonzero contribution from the non-free elements, denoted as ${\zeta }_{ij},(i,j)\in \overline{\mathcal{V}}$. Using Lemma 2 [17], we can express the non-free elements, ${\zeta }_{rs}$ for $(r,s)\in \overline{\mathcal{V}}$ and $r<s$, recursively in terms of the free elements,

$$\begin{array}{c}\hfill {\zeta }_{rs}=\sum _{j=r}^{s-1}\left(-{\zeta }_{rj}\frac{{\lambda }_{js}}{{\lambda }_{ss}}\right)-\sum _{i=1}^{r-1}\left(\frac{{\zeta }_{ir}+{\sum }_{j=i}^{r-1}{\zeta }_{ij}\frac{{\lambda }_{jr}}{{\lambda }_{rr}}}{{\zeta }_{rr}}\right)\left({\zeta }_{is}+\sum _{j=i}^{s-1}{\zeta }_{ij}\frac{{\lambda }_{js}}{{\lambda }_{ss}}\right).\end{array}$$

(16)

Subsequent expressions for the gradient and Hessian of $\Psi \left(\zeta \right)$ can be found in Appendix E.

4.3. Junction Tree Representation

In order to combat the computational overhead associated with larger graphs, we modify EPSOM-Hyb to first break down the graph into subgraphs according to its topology using a junction tree representation, which exists for all connected graphs. By leveraging the attractive properties of junction tree representations, we not only achieve significant computational speedup in the marginal likelihood calculation, but we can also more adequately handle general graphs. We briefly discuss the properties of decomposable versus non-decomposable graphs and how their respective junction tree representations can simplify the normalizing constant calculation. A junction tree representation of a graph decomposes the graph into a sequence of interconnecting subgraphs separated by complete subgraphs [36], which has the following form:

$$\begin{array}{c}\hfill G\mapsto {\mathcal{J}}_G=\{P_1,S_2,P_2,S_3,\dots ,P_{m-1},S_m,P_m\}.\end{array}$$

(17)

In ${\mathcal{J}}_G$, each prime component $P_i$ is a proper subgraph of G, each separator $S_i$ is a complete subgraph of G, and each $S_i$ is the intersection of $P_i$ with all the previous components $\{P_1,P_2,\dots ,P_{i-1}\}$. If any of the prime components found by this decomposition procedure is not complete and cannot be further decomposed, then G is non-decomposable [37]. With these concepts in place, we recall the original goal of marginal likelihood estimation but look to include the intermediate step of obtaining the junction tree representation of the graph. By working with the smaller prime components, we can overcome the complexity associated with the original high-dimensional graph. Moreover, we can take advantage of the distributional properties of the complete prime components.

Concretely, suppose we have data $X=\{x_1,\dots ,x_n\}$ satisfying the GGM with a graph G and corresponding junction tree ${\mathcal{J}}_G$, as given in Equation (17). Let $\mathcal{P}$ and $\mathcal{S}$ represent the prime component and separator sequences, respectively. If all of the prime components are complete (cliques), we denote the clique sequence as $\mathcal{C}$. Regardless of the decomposability of G, we can use ${\mathcal{J}}_G$ to express the joint density of X given $\Sigma$,

$$\begin{array}{c}\hfill p\left(X\mid \Sigma \right)=\frac{{\prod }_{P\in \mathcal{P}}p\left(X_P\mid {\Sigma }_P\right)}{{\prod }_{S\in \mathcal{S}}p\left(X_S\mid {\Sigma }_S\right)},\end{array}$$

(18)

where ${\Sigma }_P$ and ${\Sigma }_S$ represent the submatrices corresponding to the subgraphs P and S, respectively. We first consider the case where G is decomposable, with ${\Sigma }^{-1}\sim {\mathrm{HIW}}_G\left(\delta ,\Lambda \right)$. Then, the prior density factorizes over the cliques and separators,

$$\begin{array}{c}\hfill p\left(\Sigma \mid G\right)=\frac{{\prod }_{C\in \mathcal{C}}p\left({\Sigma }_C\mid G\right)}{{\prod }_{S\in \mathcal{S}}p\left({\Sigma }_S\mid G\right)}.\end{array}$$

(19)

The completeness of the prime components admits distributional properties that make the normalizing constants tractable. In particular, the prior density on ${\Sigma }_C$ is the inverse-Wishart (IW). Putting this together with the likelihood given in Equation (18), we deduce that the marginal likelihood also factorizes over the cliques and separators as follows,

$$\begin{array}{ccc}\hfill p\left(X\mid G\right)& ={\int }_{\Sigma \mid G}p\left(X\mid G\right)p\left(\Sigma \mid G\right)d\Sigma \hfill & \\ \hfill & ={\left(2\pi \right)}^{-\frac{np}{2}}\frac{h\left(G,\delta ,\Lambda \right)}{h\left(G,\delta +n,\Lambda +B\right)}\hfill & \left(20\right)\\ \hfill & ={\left(2\pi \right)}^{-\frac{np}{2}}\frac{{\prod }_{C\in \mathcal{C}}w\left(C\right)}{{\prod }_{S\in \mathcal{S}}w\left(S\right)}.\hfill & \left(21\right)\end{array}$$

Here, $B={\sum }_i{x}_i{x}_i^{\prime }$. For a decomposable graph, the factorization of the likelihood and IW prior over the cliques and separators yields a product of tractable normalizing constants. Exact formulae for $h\left(G,\delta ,\Lambda \right)$, $w\left(C\right)$ and $w\left(S\right)$ can be found in Equations (A4) and (A5).

We can easily generalize the calculations above to accommodate non-decomposable graphs with GW priors. Since the GW density in Equation (12) has a similar form to the HIW density, the marginal likelihood calculation also factorizes over prime components and separators. The difference from the previous calculation is that for decomposable G, the product in Equation (21) is taken over the cliques $\mathcal{C}$, whereas in the GW case, the product is taken over prime components $\mathcal{P}$, which may or may not be complete. Since normalizing constants for the non-complete prime components are generally intractable, they typically require MCMC estimation.

Essentially, the factorized representation of the marginal likelihood induced by the junction tree representation of G splits the original calculation involving the entire graph G into sub-problems involving the prime components and separators. When $P\in \mathcal{P}$ is complete, we can rely on the closed form IW normalizing constant. For non-complete $P\in \mathcal{P}$, we take the corresponding parameters, ${\Sigma }_P^{-1}={\Omega }_P,B_P,{\Lambda }_P$, and write the normalizing constant for the non-complete prime component as

$$\begin{array}{cc}\hfill & h\left(P,\delta ,{\Lambda }_P\right)=\int {\left|{\Omega }_P\right|}^{\frac{\delta -2}{2}}exp\left(-tr\left({\Omega }_P{\Lambda }_P\right)/2\right)d{\Omega }_P.\hfill \end{array}$$

Although this quantity remains intractable, it poses a lower dimensional problem for which we can use the procedure outlined in Section 4.2 along with the EPSOM-Hyb algorithm. The posterior normalizing constant $h\left(P,\delta +n,{\Lambda }_P+B_P\right)$ can be similarly computed. After computing the normalizing constants for each of the prime components and separators, the individual approximations are summed together to form the log normalizing constant approximation corresponding to the original graph G. These steps make up the EPSOM-HybJT algorithm, which is outlined in Algorithm 2.

4.3.1. Experiment 1: Block Diagonal Graph Structure

Consider a graph $G_p$ formed by stacking the adjacency matrix $A_{G_5}$ of $G_5$ (given in Figure 1) r times along a block diagonal. Using the notation in Section 4.1, $A_{G_p}=I_r\otimes A_{G_5}$, for $r=1,10,20,30$. Similarly, for a randomly generated scale matrix $\Lambda \in {\mathbb{R}}^{5\times 5}$, the corresponding scale matrix for $G_p$ is ${\Lambda }_p=I_r\otimes \Lambda$, where $p=5r$ denotes the dimension of the vertex set of $G_p$. For specific graphs, the corresponding GW normalizing constants can be computed analytically. In particular, for $G_5$, we can use the formula in Equation (4.1) from Uhler et al. [22] to compute $I_{G_5}\left(\delta ,\Lambda \right)$, which can then be plugged into Equation (13) to obtain the normalizing constant of ${\mathrm{GW}}_{G_5}\left(\delta ,\Lambda \right)$. Since $G_p$ explicitly uses $G_5$ to create its dependence structure, we can then easily calculate the normalizing constant of ${\mathrm{GW}}_{G_p}\left(\delta ,{\Lambda }_r\right)$.

The results in

Table 2. Mean estimate, mean relative error (MRE), and relative runtime of EPSOM-HybJT and GNORM approximations to the log normalizing constant of the ${\mathrm{GW}}_{G_p}\left(\delta ,{\Lambda }_p\right)$ density. $G_p$ has $p=5r$ vertices and $d=12r$ free elements. Here, $\delta =100$ and ${\Lambda }_p=I_r\otimes \Lambda$. Estimates are reported over 20 replications, each using 1000 samples from the target distribution. The runtime of the GNORM algorithm is calculated relative to the runtime of the EPSOM-HybJT algorithm.

Dimension	Method	Mean	MRE	Runtime
$p=5,d=12$	EPSOM-HybJT	920.16	$6.46\times {10}^{-4}$	000001
	GNORM	00920.17	$6.63\times {10}^{-4}$	0000.03
$p=50,d=120$	EPSOM-HybJT	9201.55	$6.44\times {10}^{-4}$	0009.60
	GNORM	09201.74	$6.63\times {10}^{-4}$	0035.94
$p=100,d=240$	EPSOM-HybJT	18,403.14	$6.45\times {10}^{-4}$	19.26
	GNORM	18,403.46	$6.63\times {10}^{-4}$	0796.07
$p=150,d=360$	EPSOM-HybJT	27,604.72	$6.46\times {10}^{-4}$	28.92
	GNORM	27,605.18	$6.63\times {10}^{-4}$	4587.46

indicate that both GNORM and EPSOM-HybJT produce accurate estimates, with EPSOM-HybJT having a slightly lower relative error. However, the time savings that accompany EPSOM-HybJT are substantial. In the case of ${GW}_{G_5}\left(\delta ,\Lambda \right)$, GNORM is notably faster, but for high-dimensional graphs, there is a dramatic slowdown. Indeed, for $p=150$, GNORM is more than 100 times slower than EPSOM-HybJT.

Algorithm 2: EPSOM-HybJT

4.3.2. Experiment 2: Normalizing Constants of General Graphs

Next, we investigate the performance of the two estimators for more general graphs. The results of the previous experiment showed that both GNORM and EPSOM-HybJT are very close to the true normalizing constants. For general graphs that do not satisfy the conditions that previously allowed us to calculate the exact GW normalizing constant, we compare the values of the GNORM and EPSOM-HybJT estimates against each other and keep track of the runtime relative to that of the EPSOM-HybJT algorithm when $p=10$. In 

Table 3. Mean estimate, standard deviation, and relative runtime of EPSOM-HybJT and GNORM approximations to the log normalizing constant of the ${\mathrm{GW}}_{G_p}\left(\delta ,\Lambda \right)$. Here, $\Lambda$ is not block-diagonal. Estimates are reported over 20 replications, each using 1000 samples from the corresponding G-Wishart distribution. The runtime of the GNORM algorithm is calculated relative to the runtime of the EPSOM-HybJT algorithm.

# Vertices	Method	Mean	SD	Runtime
$p=10$	EPSOM-HybJT	−2477.40	0.003	000.01
	GNORM	0−2468.19	0.000	000.02
$p=30$	EPSOM-HybJT	−7450.69	0.012	002.99
	GNORM	0−7428.00	0.773	012.34
$p=40$	EPSOM-HybJT	−10,030.95	0.015	4.68
	GNORM	−10,003.81	1.596	042.76
$p=50$	EPSOM-HybJT	−12,563.87	0.014	6.76
	GNORM	−12,528.42	2.385	108.87
$p=60$	EPSOM-HybJT	−15,170.05	0.017	15.26
	GNORM	-Inf	—	231.32

, for $p\le 50$, very little separates the log normalizing constant estimates, and GNORM even demonstrates its strength in low dimensions with a runtime that is about 50 times faster than EPSOM-HybJT for $p=10$. However, EPSOM-HybJT flips the script in all subsequent experiments and scales better as p grows. For $p=60$, the GNORM estimator fails to give a finite estimate for the log normalizing constant.

In these experiments, each graph is generated using Bernoulli draws to determine the existence of its edges and checked to ensure that the graph is non-decomposable; otherwise, this would simply reduce the problem to a summation of tractable IW constants and would be unable to fairly assess the approximating ability of EPSOM-HybJT. Contrast these results with the examples from the previous section where the scale matrix was assumed to be block-diagonal and the dependence structure was relatively simple. In those simpler settings, GNORM retains its accuracy even for high-dimensional graphs, albeit with a much slower runtime. Evidently, the added complexity induced by a nontrivial dependence structure contributes to the computational burden that cannot be easily overcome using standard methods.

4.4. Software Contribution

With the architecture in place for both the EPSOM-Hyb algorithm and its junction tree extension, we are equipped with the necessary tools for computing normalizing constants for general cases, as well as GGM-specific examples. For ease of use, EPSOM-Hyb is implemented in the hybrid package. Note that the hyperparameter initialization in line 26 of Figure A1 refers to problem-specific hyperparameters, rather than algorithm-specific hyperparameters or settings. We emphasize that other than supplying a sampler for the target distribution and functions to evaluate $\Psi$ and its gradient and Hessian, the EPSOM-Hyb methodology does not require hyperparameter tuning or convergence monitoring, beyond Newton’s method for finding the global mode of the unimodal target distribution, as described in Section 3.3. Recognizing the intricacy of the EPSOM-HybJT algorithm and the difficulty of having to manually combine the EPSOM-Hyb and the junction tree methodologies, we have developed an additional package dedicated specifically to estimating the normalizing constant of G-Wishart densities. The R package graphml serves as a black box method that seamlessly integrates the junction tree algorithm into the existing EPSOM-Hyb methodology without any user input other than the adjacency matrix representation of the graph and the G-Wishart density parameters. See Appendix A and Appendix B for more details regarding the installation and use of these packages. Additionally, in the package repositories, we provide the R programs that replicate the experimental results reported in this paper.

5. Discussion

Our methodological contributions are twofold: firstly, we developed EPSOM-Hyb as a general algorithm for computing the normalizing constant of log-concave densities, and secondly, we extended this methodology to specifically target the marginal likelihood of non-decomposable GGMs. While the GNORM estimator from Atay-Kayis and Massam [17] is a valuable tool that performs well for simpler and lower-dimensional graph structures, it is evident that more scalable and robust solutions are needed in the nontrivial problem settings presented in this paper. By taking advantage of junction tree representations of connected graphs and pairing them with the EPSOM-Hyb methodology, we significantly simplify the normalizing constant calculation for GGMs. The resulting EPSOM-HybJT algorithm is not only able to compete with other widely accepted estimators, but it also presents itself as a scalable solution for high-dimensional scenarios where other methods either fail to produce sensible results or succumb to time complexity constraints.

We recognize that the junction tree representation of a connected graph can be integrated into the GNORM methodology to create a more robust algorithm, leading to substantial time savings in subsequent normalizing constant calculations. However, such an implementation is presently unavailable. In the meantime, our contributions represent a universal method for marginal likelihood estimation that can be easily adapted to be as accurate and significantly faster than specialized methods for graphical models. Finally, while our methodologies empirically demonstrate their competitiveness across a variety of examples, many popular and long-established marginal likelihood estimation methods are accompanied by strong theoretical guarantees. As such, this remains an area of future work and development.