The Convergent Indian Buffet Process

Abstract

We propose a new Bayesian nonparametric prior for latent feature models, called the Convergent Indian Buffet Process (CIBP). We show that under the CIBP, the number of latent features is distributed as a Poisson distribution, with the mean monotonically increasing but converging to a certain value as the number of objects goes to infinity. That is, the expected number of features is bounded above even when the number of objects goes to infinity, unlike the standard Indian Buffet Process, under which the expected number of features increases with the number of objects. We provide two alternative representations of the CIBP based on a hierarchical distribution and a completely random measure, which are of independent interest. The proposed CIBP is assessed on a high-dimensional sparse factor model.

1. Introduction

1.1. Motivation and Background

Latent feature models provide a flexible framework for representing objects that may simultaneously possess multiple latent characteristics. The Indian Buffet Process (IBP), first introduced by [1], has become one of the most essential Bayesian nonparametric priors for such models, owing to its elegant construction as an exchangeable distribution over binary matrices with infinitely many columns. Each column corresponds to a potential latent feature, and each row to an object possessing a subset of these features. This approach has been widely applied to diverse areas, including factor analysis, link prediction, and relational data modeling [2,3,4]. Although its original development is more than a decade ago, recent research demonstrates that IBP-based models remain an active and evolving field within Bayesian nonparametrics, with extensions including hierarchical spike-and-slab IBP [5], similarity-based feature learning [6], and more general product feature allocation formulations [7].

Despite its popularity, a key limitation of the standard IBP and its two- and three-parameter extensions [8,9] lies in the unbounded growth of the expected number of features as the number of objects increases. For a latent feature matrix $\Xi \in {\mathbb{R}}^{p\times \infty }$, each row corresponds to an object and each column to a potential feature. The number of active features, denoted by the number of nonzero columns in $\Xi$, is defined as

$$K^{+}:=\sum _{k=1}^{\infty }\mathbb{I}\left({\mathit{\xi }}_{\left(k\right)}\ne 0\right),$$

where ${\mathit{\xi }}_{\left(k\right)}$ is the k-th column of $\Xi$ and $\mathbb{I}(·)$ is the indicator function. Under the standard IBP and its two- or three-parameter generalizations, the expected number of features $\mathbb{E}\left[K^{+}\right]$ increases either logarithmically or polynomially as the number of objects p grows:

$$\mathbb{E}\left[K^{+}\right]=\left\{\begin{array}{cc}O(logp),\hfill & \mathrm{for}\mathrm{the}\mathrm{one-}\mathrm{or}\mathrm{two-parameter}\mathrm{IBP},\hfill \\ O\left(p^a\right),\hfill & \mathrm{for}\mathrm{the}\mathrm{three-parameter}\mathrm{IBP}\mathrm{with}a>0.\hfill \end{array}\right.$$

Hence, existing IBP variants can produce redundant and uninformative features as data size increases, reducing interpretability and predictive performance. This property may be undesirable for applications where the number of underlying latent factors is believed to be finite or asymptotically stable.

In many practical domains, the assumption of infinitely increasing latent features is not realistic. For example, in macroeconomic or financial modeling, the co-movements among a large number of asset returns can often be explained by a fixed and small number of underlying factors [10,11]. Similarly, in biological and social network analysis, the generative mechanisms are often governed by a finite set of latent processes that do not proliferate with additional data [12]. However, the standard IBP tends to allocate unnecessary features as the data size grows, which complicates interpretability and reduces statistical efficiency.

This motivates the need for a new nonparametric prior that maintains the flexibility of IBP while ensuring a convergent (bounded) number of features, i.e., the distribution of $K^{+}$ converges to a finite limit as $p\to \infty$. Such a prior would bridge the gap between classical finite feature models and existing nonparametric counterparts, combining parsimony with theoretical tractability.

In this context, we propose the Convergent Indian Buffet Process (CIBP), a novel three-parameter extension of the IBP that possesses a convergent expected number of features. The CIBP retains exchangeability and constructive elegance of the IBP while introducing an additional parameter that governs the rate of convergence. Specifically, the total number of active features follows a Poisson distribution whose mean does not diverge as the number of objects tends to infinity.

1.2. Contributions and Organization

This paper makes both methodological and theoretical contributions to Bayesian nonparametric modeling. First, we introduce the Convergent Indian Buffet Process (CIBP), a new stochastic process that extends the Indian Buffet Process by ensuring a bounded expected number of latent features. The CIBP maintains the desirable exchangeability property of the IBP while incorporating an additional parameter that controls convergence. This enables a parsimonious latent feature inference even as the number of observations increases.

Second, we establish the theoretical foundations of the CIBP. We derive its distributional properties, including the asymptotic convergence of the number of active features, and provide two equivalent formulations: a hierarchical representation based on Poisson–Beta–Bernoulli layers, and a completely random measure construction using the Beta process. These representations clarify the relationship between the CIBP and existing nonparametric priors, offering new insight into the structure of latent feature models.

Finally, we demonstrate the utility of the CIBP through its application to Bayesian sparse factor models. The empirical results show that the CIBP yields stable and interpretable factor inference while preventing overfitting in high-dimensional data.

The remainder of this paper is organized as follows: Section 2 introduces the Convergent Indian Buffet Process and analyzes its distributional properties. Section 3 and Section 4 develop the hierarchical and random measure formulations, respectively. Section 5 presents an application to Bayesian sparse factor modeling, and Section 6 concludes the paper.

2. Convergent Indian Buffet Process

In this section, we formalize the construction of the proposed Convergent Indian Buffet Process (CIBP). The CIBP extends the classical IBP by introducing a “convergence” parameter that ensures the total number of latent features remains bounded as the number of observations increases. While the standard IBP provides an elegant and exchangeable distribution over infinite binary matrices, its expected number of features grows unboundedly with data size. The CIBP modifies this mechanism while preserving exchangeability, thereby providing a regularized version of the IBP suitable for parsimonious latent feature inference.

2.1. Restaurant Analogy

The Convergent Indian Buffet Process (CIBP) is defined by the restaurant analogy given below. We denote by $B(a,b)$ the beta function for $a>0$ and $b>0$.

Definition 1 (The restaurant analogy of the CIBP). 

Let $\gamma >0$, $\alpha >0$ and $\kappa >0$. We refer to the following stochastic process, the restaurant analogy of $\mathtt{CIBP}$$(\gamma ,\alpha ,\kappa )$:

1. 

The first customer tries $K_1^{+}$ dishes with $\mathtt{Poisson}$$\left(\gamma B\right(\alpha +1,\kappa )/B(\alpha ,\kappa \left)\right)$

2. 

For every $j=2,\dots ,p$, the j-th customer does:

• For every $k=1,\dots ,K_{j-1}^{+}$, the j-th customer tries the k-th dish if ${\xi }_{j,k}=1$ and does not otherwise, where

  $${\xi }_{j,k}\sim \mathtt{Bernoulli}\left({\displaystyle \frac{m_{j,k}+\alpha }{j-1+\kappa +\alpha }}\right).$$

  (1)

  Here, $m_{j,k}$ denotes the number of previous customers before the j-th customer, who have tried the k-th dish.
• The j-th customer tries $K_j^{\mathrm{new}}$ many new dishes with

  $$K_j^{\mathrm{new}}\sim \mathtt{Poisson}\left(\gamma {\displaystyle \frac{B(\alpha +1,\kappa +j-1)}{B(\alpha ,\kappa )}}\right).$$

  (2)
• Set $K_j^{+}=K_{j-1}^{+}+K_j^{new}$.

Each realization of the above restaurant process is a binary matrix, where the number of rows corresponds to the number of customers and the columns are unbounded. In this matrix, the element at position $(j,k)$ is set to one if the j-th customer has sampled the k-th dish, and zero if they have not. The distribution of this binary matrix, as generated by the restaurant analogy, is denoted by $\mathtt{CIBP}(\gamma ,\alpha ,\kappa )$.

We provide an intuition behind the converging property of the number of features under the CIBP. Under the CIBP, the expected number of new dishes taken by the j-th customer is

$$\begin{array}{cc}\gamma {\displaystyle \frac{B(\alpha +1,\kappa +j-1)}{B(\alpha ,\kappa )}}& = \gamma {\displaystyle \frac{\Gamma (\alpha +1)\Gamma (\kappa +j-1)}{\Gamma (\alpha +\kappa +j)}}{\displaystyle \frac{\Gamma (\alpha +\kappa )}{\Gamma \left(\alpha \right)\Gamma \left(\kappa \right)}}\\ & = \gamma {\displaystyle \frac{\Gamma (\alpha +1)}{\Gamma \left(\alpha \right)}}{\displaystyle \frac{\Gamma (\alpha +\kappa )}{\Gamma (\alpha +\kappa +j)}}{\displaystyle \frac{\Gamma (\kappa +j-1)}{\Gamma \left(\kappa \right)}}\\ & = \gamma {\displaystyle \frac{\alpha }{\alpha +\kappa +j-1}}{\displaystyle \prod _{h=1}^{j-1}}{\displaystyle \frac{\kappa +h-1}{\alpha +\kappa +h-1}},\end{array}$$

(3)

where Γ(·) denotes the gamma function. As the simplest case, when α = 1, the last display becomes

$$\begin{array}{c}\hfill {\displaystyle \frac{\gamma }{\kappa +j}}{\displaystyle \frac{\kappa }{\kappa +j-1}}=O \left({\displaystyle \frac{1}{j^2}}\right).\end{array}$$

since $\sum _{j=1}^{\infty }1/j^2<\infty$ we can expect that the expected number of features converges to a finite number as p tends to ∞. Indeed, this convergent property holds for any α > 0, which we will show in the next subsection.

Throughout the paper, we let ${\psi }_{a_1,b_1}^{a_2,b_2}$ denote the ratio of two beta functions defined as

$${\psi }_{a_1,b_1}^{a_2,b_2}:={\displaystyle \frac{B(a_1+a_2,b_1+b_2)}{B(a_1,b_1)}}.$$

for $a_1,b_1>0$ and $a_2,b_2\ge 0$ for notational simplicity. For example, the Poisson distribution in (2) may be written as $\mathtt{Poisson}\left(\gamma {\psi }_{\alpha ,\kappa }^{1,j-1}\right)$.

2.2. Distribution of the Number of Features Under the CIBP

In this subsection, we show that the number of features under the CIBP follows a Poisson distribution with the mean being fixed as the number of objects increases. The name, convergent IBP, is named after this property.

Theorem 1. 

If $\Xi \sim \mathtt{CIBP}(\gamma ,\alpha ,\kappa )$, then the number of active features is distributed as

$$K^{+}\sim \mathtt{Poisson}\left(\gamma \right(1-{\psi }_{\alpha ,\kappa }^{0,p}\left)\right),$$

(4)

where ${\psi }_{\alpha ,\kappa }^{0,p}:=B(\alpha ,\kappa +p)/B(\alpha ,\kappa )\le 1.$ Moreover, the mean $\gamma (1-{\psi }_{\alpha ,\kappa }^{0,p})$ is monotonically increasing and tends to γ as $p\to \infty$. This implies that $K^{+}$ converges to the random variable K following $\mathtt{Poisson}\left(\gamma \right)$ in distribution.

Proof. 

From the restaurant analogy of the CIBP, we have that

$$K^{+}\stackrel{d}{=}\sum _{j=1}^P{K}_j^{\mathrm{new}}, \mathrm{where} K_j^{\mathrm{new}}\stackrel{\mathrm{ind}}{\sim }\mathrm{Poisson} \left(\gamma {\displaystyle \frac{B(\alpha +1,\kappa +j-1)}{B(\alpha ,\kappa )}}\right)$$

Therefore, by the additive property of independent Poisson random variables,

$$K^{+}\sim \mathtt{Poisson} \left({\displaystyle \frac{\gamma }{B(\alpha ,\kappa )}}\sum _{j=1}^{p}B(\alpha +1,\kappa +j-1)\right)$$

From the identity $B(x,y)-B(x,y+1)=B(x+1,y)$, we have

$$\begin{array}{cc}\hfill \sum _{j=1}^{p}B(\alpha +1,\kappa +j-1)& =\sum _{j=1}^p \left\{B\right(\alpha ,\kappa +j-1)-B(\alpha ,\kappa +j\left)\right\}\hfill \\ \hfill & =B(\alpha ,\kappa )-B(\alpha ,\kappa +p),\hfill \end{array}$$

which implies (4).

For the second assertion, note that

$$\begin{array}{cc}\hfill {\psi }_{\alpha ,\kappa }^{0,p}& ={\displaystyle \frac{\Gamma \left(\alpha \right)\Gamma (\kappa +p)}{\Gamma (\alpha +\kappa +p)}}{\displaystyle \frac{\Gamma (\alpha +\kappa )}{\Gamma \left(\alpha \right)\Gamma \left(\kappa \right)}}\hfill \\ \hfill & ={\displaystyle \frac{\Gamma (\alpha +\kappa )}{\Gamma (\alpha +\kappa +p)}}{\displaystyle \frac{\Gamma (\kappa +p)}{\Gamma \left(\kappa \right)}}\hfill \\ \hfill & =\prod _{j=1}^p{\displaystyle \frac{\kappa +j-1}{\alpha +\kappa +j-1}}.\hfill \end{array}$$

Since $\alpha >0$, it follows that ${\psi }_{\alpha ,\kappa }^{0,p}\downarrow 0$ as $p\to \infty$. □

We provide a simple numerical example supporting (4). In

Table 1. Empirical and theoretical probabilities for the number of active features under $\mathtt{CIBP}(1,1,1)$ with $p=500$.

k	Empirical Prob.	Poisson Prob.	Abs. Diff.
0	0.369	0.369	0.002
1	0.368	0.368	0.000
2	0.181	0.184	0.002
3	0.057	0.061	0.004
4	0.017	0.015	0.002
≥5	0.008	0.005	0.001

, we compare the empirical distribution of $K^{+}$ obtained from 5000 draws of feature matrices under $\mathtt{CIBP}(1,1,1)$ with $p=500$ to the distribution of $\mathtt{Poisson}(1-{\psi }_{1,1}^{0,500})$. The absolute differences are below $0.005$ across all categories, providing numerical support for the distribution of the number of active features under the CIBP.

2.3. Connection to the Two-Parameter IBP

We give the restaurant analogy of $\mathtt{IBP}({\gamma }^{†},\kappa )$. The first customer tries $\mathtt{Poisson}\left({\gamma }^{†}\right)$ dishes. For $j\ge 2$, the j-th customer tries each previously tasted dish according to

$$\begin{array}{c}\hfill \mathtt{Bernoulli} \left({\displaystyle \frac{m_{j,k}}{j-1+\kappa }}\right),\end{array}$$

and tries $K_j^{\mathrm{new}}$ many new dishes with $K_j^{\mathrm{new}}$ following

$$\begin{array}{c}\hfill \mathtt{Poisson} \left({\displaystyle \frac{{\gamma }^{†}{\kappa }^{†}}{j-1+\kappa }}\right).\end{array}$$

We denote by $\mathtt{IBP}({\gamma }^{†},\kappa )$ the distribution induced by the above restaurant analogy.

The next theorem reveals the relationship between $\mathtt{CIBP}(\gamma ,\alpha ,\kappa )$ and $\mathtt{IBP}({\gamma }^{†},\kappa )$.

Theorem 2. 

For two $p\times \infty$-dimensional binary matrices $\Xi \sim \mathtt{CIBP}(\gamma ,\alpha ,\kappa )$ and ${\Xi }_0\sim \mathtt{IBP}({\gamma }^{†},\kappa ), \Xi$ converges to ${\Xi }_0$ in distribution as $\alpha \to 0$ and $\gamma \alpha /\kappa \to {\gamma }^{†}$.

Proof. 

Note that

$${\displaystyle \frac{m_{j,k}+\alpha }{j-1+\kappa +\alpha }}\to {\displaystyle \frac{m_{j,k}}{j-1+\kappa }}$$

as $\alpha \to 0$. Moreover,

$$\begin{array}{cc}\hfill \gamma {\displaystyle \frac{B(\alpha +1,\kappa +j-1)}{B(\alpha ,\kappa )}}& =\gamma {\displaystyle \frac{\alpha }{\alpha +\kappa +j-1}}\prod _{h=1}^{j-1}{\displaystyle \frac{\kappa +h-1}{\alpha +\kappa +h-1}}\hfill \\ \hfill & \to {\displaystyle \frac{{\gamma }^{†}\kappa }{j-1+\kappa }},\hfill \end{array}$$

as $\alpha \to 0$ and $\gamma \alpha /\kappa \to {\gamma }^{†}$, where the equality was shown in (3). These two displays imply that the means of the Bernoulli distribution in (1) and the Poisson distribution in (2) converge to the corresponding quantities for $\mathtt{IBP}({\gamma }^{†},\kappa )$. This gives the desired result. □

Figure 1 shows some realizations of $\mathtt{CIBP}({\gamma }^{†}\kappa /\alpha ,\alpha ,\kappa )$ $\alpha =5$, $\alpha =1$, $\alpha =0.5$ and $\mathtt{IBP}({\gamma }^{†},\kappa )$, with ${\gamma }^{†}=5$, where $\kappa =4$ is fixed. It is evident that the IBP generates more active features than the CIBP.

3. Hierarchical Representation

In this section, we prove that the restaurant analogy of the CIBP can be expressed as some hierarchical distribution.

Definition 2 (Hierarchical representation of the CIBP). 

Let $\gamma >0$, $\alpha >0$, and $\kappa >0$. We refer to the following hierarchical probability distribution as the hierarchical representation of $\mathtt{CIBP}(\gamma ,\alpha ,\kappa )$:

$$\begin{gathered} K\sim \mathtt{Poisson}\left(\gamma \right), \\ {\theta }_k\stackrel{\mathrm{iid}}{\sim }\mathtt{Beta} (\alpha ,\kappa ),k\in \left[K\right] \\ {\xi }_{jk}|{\theta }_k\stackrel{\mathrm{ind}}{\sim }\mathtt{Bernoulli}\left({\theta }_k\right)j\in \left[p\right],k\in \left[K\right]. \end{gathered}$$

(5)

To articulate the result precisely, we introduce the notion of lof-equivalence classes. In the context of the latent feature model, the sequence of features does not influence the likelihood. Therefore, we consider two $p\times \infty$ dimensional binary matrices to be equivalent if they can be converted into each other by rearranging their columns. It is practical to select a representative for each equivalence class through the left-ordering procedure. This procedure transforms each $p\times \infty$ dimensional binary matrix into its left-ordered form, arranging the columns based on the score $s_k$, which is defined as follows:

$$s_k:=\sum _{j=1}^p{\xi }_{jk}{2}^{p-j}$$

i.e., the columns are ordered so that $s_1\ge s_2\ge \dots .$ The equivalence class defined by the left-ordering procedure is called the lof-equivalence class and denoted by $[\Xi ]$.

We introduce additional notations. Let $\Delta :={\{0,1\}}^p$ and ${\Delta }_1:=\Delta \setminus \left\{\mathbf{0}\right\}$. Moreover, for $\mathbf{u}\in {\Delta }_1$, we define

$$\begin{array}{cc}\hfill K_{\mathbf{u}}& :=\sum _{k=1}^{\infty }\mathbb{I}({\mathit{\xi }}^{\left(k\right)}=\mathbf{u}),\hfill \end{array}$$

(6)

with ${\mathit{\xi }}^{\left(k\right)}$ being the k-th column of $\Xi$, which is the number of columns of $\Xi$ equal to $\mathbf{u}$. We then have $K^{+}:={\sum }_{k=1}^{\infty }\mathbb{I}({\mathit{\xi }}^{\left(k\right)}\ne \mathbf{0})={\sum }_{\mathbf{u}\in {\Delta }_1}{K}_{\mathbf{u}}$. We define

$$m_k:=\sum _{j=1}^p{\xi }_{jk},$$

which is the number of rows that possess the k-th feature.

The following theorem provides the probability mass function of the lof-equivalence class $[\Xi ]$.

Theorem 3. 

The probability mass function of a $p\times \infty$-dimensional random binary matrix $\Xi \equiv {\left({\xi }_{jk}\right)}_{j\in \left[p\right],k\in \mathbb{N}}$ generated from the distribution in (5) is given by

$$\mathbb{P}\left([\Xi ]\right)={\displaystyle \frac{{\gamma }^{K^{+}}}{{\prod }_{\mathbf{u}\in {\Delta }_1}{K}_{\mathbf{u}}!}}{e}^{-\gamma {\sum }_{j=1}^p{\psi }_{\alpha ,\kappa }^{1,j-1}}\left[\prod _{k=1}^{K^{+}}{\psi }_{\alpha ,\kappa }^{m_k,p-m_k}\right].$$

(7)

Proof. 

Recall that $m_k:={\sum }_{j=1}^p{\xi }_{jk}$. If $K\ge K^{+}$, we have that

$$\begin{array}{cc}\hfill \mathbb{P}(\Xi |K)& =\prod _{k=1}^K{\displaystyle \frac{B(m_k+\alpha ,p-m_k+\kappa )}{B(\alpha ,\kappa )}}\hfill \\ \hfill & = {\left({\displaystyle \frac{B(\alpha ,p+\kappa )}{B(\alpha ,\kappa )}}\right)}^{K-K^{+}}\times \prod _{k=1}^{K^{+}}{\displaystyle \frac{B(m_k+\alpha ,p-m_k+\kappa )}{B(\alpha ,\kappa )}}\hfill \\ \hfill & = {\left({\psi }_{\alpha ,\kappa }^{0,p}\right)}^{K-K^{+}}\times \prod _{k=1}^{K^{+}}{\psi }_{\alpha ,\kappa }^{m_k,p-m_k},\hfill \end{array}$$

where for the second equality, we reorder the columns such that $m_k>0$ if $k\le K^{+}$ and $m_k=0$ otherwise. Therefore, since the cardinality of the lof-equivalence class is $\left|[\Xi ]\right]=K!/{\prod }_{\mathbf{u}\in \Delta }{K}_{\mathbf{u}}!$, the (conditional) probability mass function of $[\Xi ]$ given $K\ge K^{+}$ is given by

$$\mathbb{P}\left([\Xi ]\right|K)={\displaystyle \frac{K!}{{\prod }_{\mathbf{u}\in \Delta }{K}_{\mathbf{u}}!}}{ \left({\psi }_{\alpha ,\kappa }^{0,p}\right)}^{K-K^{+}}\prod _{k=1}^{K^{+}}{\psi }_{\alpha ,\kappa }^{m_k,p-m_k}$$

If $K<K^{+}$, we have $\mathbb{P}(\Xi |K)=0$. Since $K\sim \mathtt{Poisson}\left(\gamma \right)$, we have

$$\begin{array}{c}\hfill \mathbb{P}\left([\Xi ]\right)={\displaystyle \frac{1}{{\prod }_{\mathbf{u}\in {\Delta }_1}{K}_{\mathbf{u}}!}} \left[\prod _{k=1}^{K^{+}}{\psi }_{\alpha ,\kappa }^{m_k,p-m_k}\right]\sum _{K=K^{+}}^{\infty }{\displaystyle \frac{K!}{K_{\mathbf{0}}!}}{\left({\psi }_{\alpha ,\kappa }^{0,p}\right)}^{K-K^{+}}{p}_K\left(K\right).\end{array}$$

where $p_K$ denotes the probability mass function of $\mathtt{Poisson}\left(\gamma \right)$, i.e., $p_K\left(k\right):=e^{-\gamma }{\gamma }^k/k!$ for $k=0,1,2,\dots$. Note that

$$\begin{gathered} \sum _{K=K^{+}}^{\infty }{\displaystyle \frac{K!}{K_{\mathbf{0}}!}}{ \left({\psi }_{\alpha ,\kappa }^{0,p}\right)}^{K-K^{+}}{p}_K\left(k\right) \\ =e^{-\gamma }{\gamma }^{K^{+}}\sum _{K=K^{+}}^{\infty }{\displaystyle \frac{1}{(K-K^{+})!}}{ \left(\gamma {\psi }_{\alpha ,\kappa }^{0,p}\right)}^{K-K^{+}} \\ ={\gamma }^{K^{+}}{e}^{-\gamma (1-{\psi }_{\alpha ,\kappa }^{0,p})}. \end{gathered}$$

(8)

Moreover, by the identity $B(x,y)-B(x,y+1)=B(x+1,y)$, we have

$$\begin{array}{cc}\hfill 1-{\psi }_{\alpha ,\kappa }^{0,p}& =1-{\displaystyle \frac{B(\alpha ,p+\kappa )}{B(\alpha ,\kappa )}}\hfill \\ & ={\displaystyle \frac{1}{B(\alpha ,\kappa )}} \left\{B\right(\alpha ,\kappa )-B(\alpha ,p+\kappa \left)\right\}\hfill \\ & ={\displaystyle \frac{1}{B(\alpha ,\kappa )}}{\displaystyle \sum _{j=1}^p} \left\{B\right(\alpha ,\kappa +j-1)-B(\alpha ,\kappa +j\left)\right\}\hfill \\ & ={\displaystyle \frac{1}{B(\alpha ,\kappa )}}{\displaystyle \sum _{j=1}^p}B(\alpha +1,\kappa +j-1)\hfill \\ & ={\displaystyle \sum _{j=1}^p}{\psi }_{\alpha ,\kappa }^{1,j-1}.\hfill \end{array}$$

(9)

Combining (8) and (9), we obtain the desired result.     □

By Theorem 3, it can be shown that Definitions 1 and 2 are equivalent.

Theorem 4.

For a $p\times \infty$-dimensional binary matrix Ξ following $\mathtt{CIBP}(\gamma ,\alpha ,\kappa )$, the probability mass function of the lof-equivalence class $[\Xi ]$ is the same as (7). Therefore, the distribution in (5) is equivalent to $\mathtt{CIBP}(\gamma ,\alpha ,\kappa )$.

Proof. 

Let ${\mathit{\xi }}_j$ be the j-th row of $\Xi$. We have

$$\mathbb{P}\left({\mathit{\xi }}_1\right)={\displaystyle \frac{1}{K_1^{+}!}}{ \left(\gamma {\psi }_{\alpha ,\kappa }^{1,0}\right)}^{K_1^{+}}{\mathrm{e}}^{-\gamma {\psi }_{\alpha ,\kappa }^{1,0}},$$

where $K_1^{+}$ denotes the number of nonzero elements in ${\mathit{\xi }}_1$. It is same as (7) with $p=1$ and $K^{+}=K_1^{+}$.

For $p\ge 2$, the conditional distribution of ${\mathit{\xi }}_p$ given ${\mathit{\xi }}_1,\dots ,{\mathit{\xi }}_{p-1}$ is given by

$$\begin{array}{cc}\hfill \mathbb{P}\left({\mathit{\xi }}_p\right|{\mathit{\xi }}_1,\dots ,{\mathit{\xi }}_{p-1})& ={\mathrm{e}}^{-\gamma {\psi }_{\alpha ,\kappa }^{1,p-1}}{\displaystyle \frac{{\left(\gamma {\psi }_{\alpha ,\kappa }^{1,p-1}\right)}^{K_p^{\mathrm{new}}}}{K_p^{\mathrm{new}}!}}\hfill \\ & \times {\displaystyle \prod _{k\in J_p}}{\displaystyle \frac{m_{p,k}+\alpha }{p-1+\kappa +\alpha }}{\displaystyle \prod _{k\notin J_p}}{\displaystyle \frac{p-1-m_{p,k}+\kappa }{p-1+\kappa +\alpha }},\hfill \end{array}$$

(10)

where $m_{p,k}:=\sum _{j=1}^{p-1}{\xi }_{jk},K_p^{\mathrm{new}}$ is the number of new dishes taken by the p-th customer and J_p is the set of dishes taken by the p-th customer, i.e., $J_p:=\{k\in \left[K_{p-1}^{+}\right]:{\xi }_{pk}=1\}. \mathrm{Let} K_p^{+}:=\sum _{j=1}^p{K}_j^{\mathrm{new}}=K_{p-1}^{+}+K_p^{\mathrm{new}} \mathrm{and} K_1^{\mathrm{new}}=K_1^{+}.$ By the inductive hypothesis, we have

$$\begin{array}{cc}\hfill \mathbb{P}({\mathit{\xi }}_1,\dots ,{\mathit{\xi }}_p)& =\mathbb{P}\left({\mathit{\xi }}_p\right|{\mathit{\xi }}_1,\dots ,{\mathit{\xi }}_{p-1})\mathbb{P}({\mathit{\xi }}_1,\dots ,{\mathit{\xi }}_{p-1})\hfill \\ \hfill & =e^{-\gamma {\sum }_{j=1}^p{\psi }_{\alpha ,\kappa }^{0,j-1}}{\displaystyle \frac{{\gamma }^{K_p^{+}}}{{\prod }_{j=1}^p{K}_j^{\mathrm{new}}!}}\prod _{k\in J_p}{\displaystyle \frac{m_{p,k}+\alpha }{p-1+\kappa +\alpha }}{\psi }_{\alpha ,\kappa }^{m_{p,k},p-1-m_{p,k}}\hfill \\ \hfill & \times \prod _{k\notin J_p}{\displaystyle \frac{p-1-m_{p,k}}{p-1+\kappa +\alpha }}{\psi }_{\alpha ,\kappa }^{m_{p,k},p-1-m_{p,k}}\times {\left({\psi }_{\alpha ,\kappa }^{1,p-1}\right)}^{K_p^{\mathrm{new}}}\hfill \end{array}$$

since $m_k=m_{p,k}+1 \mathrm{for} k\in J_p \mathrm{and} m_k=m_{p,k}$ otherwise, we have

$$\begin{array}{cc}\hfill {\displaystyle \frac{m_{p,k}+\alpha }{p-1+\kappa +\alpha }}{\psi }_{\alpha ,\kappa }^{m_{p,k},p-1-m_{p,k}}& ={\displaystyle \frac{m_{p,k}+\alpha }{p-1+\kappa +\alpha }}{\displaystyle \frac{B(m_{p,k}+\alpha ,p-1-m_{p,k}+\kappa )}{B(\alpha ,\kappa )}}\hfill \\ \hfill & ={\displaystyle \frac{B(m_{p,k}+1+\alpha ,p-1-m_{p,k}+\kappa )}{B(\alpha ,\kappa )}}\hfill \\ \hfill & ={\psi }_{\alpha ,\kappa }^{m_{p,k}+1,p-1-m_{p,k}}\hfill \\ \hfill & ={\psi }_{\alpha ,\kappa }^{m_k,p-m_k}\hfill \end{array}$$

and similarly,

$$\begin{array}{c}\hfill {\displaystyle \frac{p-1-m_{p,k}}{p-1+\kappa +\alpha }}{\psi }_{\alpha ,\kappa }^{m_{p,k},p-1-m_{p,k}}={\psi }_{\alpha ,\kappa }^{m_{p,k},p-m_{p,k}}={\psi }_{\alpha ,\kappa }^{m_k,p-m_k}.\end{array}$$

Therefore,

$$\begin{array}{cc}\hfill \mathbb{P}({\mathit{\xi }}_1,\dots ,{\mathit{\xi }}_p)& ={\mathrm{e}}^{-\gamma {\sum }_{j=1}^p{\psi }_{\alpha ,\kappa +1}^{1,j-1}}{\displaystyle \frac{{\gamma }^{K_p^{+}}}{{\prod }_{j=1}^p{K}_j^{\mathrm{new}}!}}{\displaystyle \prod _{k\in J_p}}{\psi }_{\alpha ,\kappa }^{m_k,p-m_k}\hfill \\ & \times {\displaystyle \prod _{k\notin J_p}}{\psi }_{\alpha ,\kappa }^{m_k,p-m_k}\times {\left({\psi }_{\alpha ,\kappa }^{1,p-1}\right)}^{K_p^{\mathrm{new}}}\hfill \\ & =e^{-\gamma {\sum }_{j=1}^p{\psi }_{\alpha ,\kappa }^{1,j-1}}{\displaystyle \frac{{\gamma }^{K_p^{+}}}{{\prod }_{j=1}^p{K}_j^{\mathrm{new}}!}}{\displaystyle \prod _{k=1}^{K_p^{+}}}{\psi }_{\alpha ,\kappa }^{m_k,p-m_k}.\hfill \end{array}$$

(11)

since $\prod _{j=1}^p{K}_j^{\mathrm{new}}!/\prod _{\mathbf{u}\in {\Delta }_1}{K}_{\mathbf{u}}$ many matrices generated by the above process have the same left-ordered form, we obtain $\mathbb{P}\left([\Xi ]\right)$ by multiplying $\prod _{j=1}^p{K}_j^{\mathrm{new}}!/\prod _{\mathbf{u}\in {\Delta }_1}{K}_{\mathbf{u}}\mathrm{and}\mathbb{P}({\mathit{\xi }}_1,\dots ,{\mathit{\xi }}_p)$ in (11).    □

Using the hierarchical representation, we can explicitly obtain the distribution of $m_k$.

Corollary 1.

Assume $\Xi \sim \mathtt{CIBP}(\gamma ,\alpha ,\kappa )$. Let $F_k:=\mathbb{P}(K\le k)$ for $K\sim \mathtt{Poisson}\left(\gamma \right)$. Then, the (marginal) probability mass function of $m_k:={\sum }_{j=1}^p{\xi }_{jk}$ is given by

$$\begin{array}{c}\hfill \mathbb{P}(m_k=m)=F_k\left({\displaystyle \genfrac{}{}{0pt}{}{p}{m}}\right){\psi }_{\alpha ,\kappa }^{m,p-m}\mathbb{I}(m=0,1,\dots ,p)+(1-F_k)\mathbb{I}(m=0)\end{array}$$

for each $k\in \mathbb{N}$.

Proof. 

In the hierarchical representation of the CIBP, we have ${\sum }_{j=1}^p{\xi }_{j,k}|{\theta }_k\sim \mathtt{Binom}\left({\theta }_k\right)$ when $K\ge k$. Thus,

$$\begin{array}{cc}\hfill \mathbb{P}(m_k=m|K\ge k)& =\left({\displaystyle \genfrac{}{}{0pt}{}{p}{m}}\right)\mathbb{E}\left[{\theta }^m{(1-\theta )}^{p-m}\right]\hfill \\ \hfill & =\left({\displaystyle \genfrac{}{}{0pt}{}{p}{m}}\right){\displaystyle \frac{B(\alpha +m,\kappa +p-m)}{B(\alpha ,\kappa )}}=\left({\displaystyle \genfrac{}{}{0pt}{}{p}{m}}\right){\psi }_{\alpha ,\kappa }^{m,p-m}\hfill \end{array}$$

On the other hand, we have $\mathbb{P}(m_k=0|K<k)=1$. Combining these two derivations, we obtain the desired. □

Exchangeability

The exchangeability of the IBP ensures that the associated posterior inference algorithms is tractable. As established in the following corollary, the CIBP is likewise an exchangeable distribution. This result follows directly from the hierarchical formulation of the CIBP given in Theorem 4.

Corollary 2.

The row vectors ${\mathit{\xi }}_1,\dots ,{\mathit{\xi }}_p$ of a $p\times \infty$-dimensional binary matrix Ξ generated from $\mathtt{CIBP}(\gamma ,\alpha ,\kappa )$ are exchangeable.

4. Construction from Random Measures

In this section, we provide the random measure construction of the CIBP. We first briefly review the concept of completely random measures. We refer to Appendix J of [13] for more details. Let $(\mathsf{\Omega },\mathcal{A})$ be the pair of a Polish space with its Borel $\sigma$-field. Let $(M,\mathcal{M})$ be the pair of a set of all measures on $(\mathsf{\Omega },\mathcal{A})$ with its Borel $\sigma$-field. A completely random measure (CRM) $\mu$ on $(\mathsf{\Omega },\mathcal{A})$ is a random measure satisfying that $\mu \left(A_1\right),\dots ,\mu \left(A_k\right)$ are mutually independent for all disjoint measurable sets $A_1,\dots ,A_k\in \mathcal{A}$. Every CRM consists of three independent parts:

$$\begin{array}{c}\hfill \mu ={\mu }_0+\sum _{k=1}^K{q}_k{\delta }_{{\omega }_k}+\sum _{(q,\omega )\in \mathsf{\Phi }}q{\delta }_{\omega },\end{array}$$

where ${\mu }_0$ is a non-random measure, ${\left({\omega }_k\right)}_{k\in \left[K\right]}$ are fixed points in $\mathsf{\Omega }$, ${\left(q_k\right)}_{k\in \left[K\right]}$ are independent random variables on ${\mathbb{R}}_{+}$ and $\mathsf{\Phi }$ is a Poisson process on ${\mathbb{R}}_{+}\times \mathsf{\Omega }$. In this paper, we consider purely atomic CRMs with ${\mu }_0=0$. If $\mu$ is a purely atomic CRM written as $\mu ={\sum }_{k=1}^K{q}_k{\delta }_{{\omega }_k}+{\sum }_{(q,\omega )\in \mathsf{\Phi }}q{\delta }_{\omega }$, we write

$$\begin{array}{c}\hfill \mu \sim \mathtt{CRM} (\Lambda ,{({\omega }_k,P_k)}_{k\in \left[K\right]}),\end{array}$$

where $P_k$ is a probability measure on ${\mathbb{R}}_{+}$, such that $q_k\stackrel{\mathrm{ind}}{\sim }{P}_k$ for each $k\in \left[K\right]$ and $\Lambda$ is a probability measure on ${\mathbb{R}}_{+}\times \mathsf{\Omega }$, such that $\mathbb{E}\mathsf{\Phi }=\Lambda$, i.e., $\Lambda$ is the intensity measure of the Poisson process $\mathsf{\Phi }$. We write $\mu \sim \mathtt{CRM}(\Lambda )$ if $\mu ={\sum }_{(q,\omega )\in \mathsf{\Phi }}q{\delta }_{\omega }$ with $\mathbb{E}\mathsf{\Phi }=\Lambda$.

We introduce two specific CRMs. The Bernoulli process with mean probability measure ${\mu }^{\prime }$, denoted by $\mathtt{BeP}\left({\mu }^{\prime }\right)$, is the CRM on $(\mathsf{\Omega },\mathcal{A})$ with intensity measure

$$\begin{array}{c}\hfill {\Lambda }_{\mathtt{BeP}\left({\mu }^{\prime }\right)}(\mathrm{d}q,\mathrm{d}\omega )={\delta }_1\left(\mathrm{d}q\right){\mu }^{\prime }\left(\mathrm{d}\omega \right),\end{array}$$

where ${\delta }_1$ denotes a point mass at 1. The beta process with parameters $a,b>0$ and base measure ${\Lambda }_0$ is the CRM on $(\mathsf{\Omega },\mathcal{A})$ with intensity measure

$$\begin{array}{c}\hfill {\Lambda }_{\mathtt{BP}(a,b,{\Lambda }_0)}(\mathrm{d}q,\mathrm{d}\omega )={\displaystyle \frac{1}{B(a,b)}}{q}^{a-2}{(1-q)}^{b-1}\mathrm{d}q{\Lambda }_0\left(\mathrm{d}\omega \right).\end{array}$$

It is known that $\mathtt{IBP}({\gamma }^{†},\kappa )$ with ${\gamma }^{†}>0$ and $\kappa >0$ is described by a random measure as follows:

$$\begin{gathered} {\mathit{\xi }}_j|\mu \stackrel{\mathrm{iid}}{\sim }\mathtt{BeP}\left(\mu \right),j\in \left[p\right] \\ \mu \sim \mathtt{BP}(1,\kappa ,{\gamma }^{†}{\Lambda }_0) \end{gathered}$$

(12)

for some smooth probability measure ${\Lambda }_0$. Note that the intensity measure of $\mathtt{BP}(1,\kappa ,{\gamma }^{†}{\Lambda }_0)$ is given by

$$\begin{array}{c}\hfill {\Lambda }_{\mathtt{BP}(1,\kappa +1,{\gamma }^{†}{\Lambda }_0)}(\mathrm{d}q,\mathrm{d}\omega )={\gamma }^{†}\kappa q^{-1}{(1-q)}^{\kappa -1}\mathrm{d}q{\Lambda }_0\left(\mathrm{d}\omega \right).\end{array}$$

We introduce another random measure representation, which turns out to be related to the CIBP.

Definition 3 (Random measure representation of the CIBP). 

Let $\gamma >0$, $\alpha >0$, and $\kappa \ge 0$. We refer to the following stochastic process the random measure representation of $\mathtt{CIBP}(\gamma ,\alpha ,\kappa )$:

$$\begin{gathered} {\mathit{\xi }}_j|\mu \stackrel{\mathrm{iid}}{\sim }\mathtt{BeP}\left(\mu \right),j\in \left[p\right] \\ \mu \sim \mathtt{BP}(\alpha +1,\kappa ,{\displaystyle \frac{\gamma \alpha }{\alpha +\kappa }}{\Lambda }_0) \end{gathered}$$

(13)

for some smooth probability measure ${\Lambda }_0$. Note that the intensity measure of $\mathtt{BP} \left(\alpha +1,\kappa ,{\displaystyle \frac{\gamma \alpha }{\alpha +\kappa }}{\Lambda }_0\right)$ is given by

$$\begin{array}{c}\hfill {\Lambda }_{\mathtt{BP}(\alpha +1,\kappa ,{\displaystyle \frac{\gamma \alpha }{\alpha +\kappa }}{\Lambda }_0)}(dq,d\omega )={\displaystyle \frac{\gamma }{B(\alpha ,\kappa )}}{q}^{\alpha -1}{(1-q)}^{\kappa -1}dq{\Lambda }_0\left(d\omega \right)\end{array}$$

(14)

As the function $q\mapsto q^{\alpha -1}{(1-q)}^{\kappa -1}$ is integrable on $[0,1]$ for $\alpha >0$, there would be a finite number of features under the distribution of (13). In contrast, this function is not integrable for $\alpha =0$, thus there would be an infinite number of features under the two-parameter IBP in (12).

In the following theorem, we show that Definitions 2 and 3 are equivalent.

Theorem 5.

The joint distribution of random measures ${\mathit{\xi }}_1,\dots ,{\mathit{\xi }}_p$ generated as (13) is given by

$$\mathbb{P}({\mathit{\xi }}_1,\dots ,{\mathit{\xi }}_p)={\mathrm{e}}^{-\gamma {\sum }_{j=1}^p{\psi }_{\alpha ,\kappa }^{1,j-1}} \left[\prod _{k=1}^{K^{+}}{\psi }_{\alpha ,\kappa }^{m_k,p-m_k}{\lambda }_0\left({\omega }_k^{*}\right)\right],$$

(15)

where there are K⁺ atoms ${\omega }_1^{*},\dots ,{\omega }_{K^{+}}^{*}$ such that $m_k:=\sum _{j=1}^p{\mathit{\xi }}_j\left({\omega }_k^{*}\right)\ge 1 for k\in \left[K^{+}\right], and {\lambda }_0$ denotes the density of Λ₀.

Proof. 

By the well-known conjugacy result (Theorem 3.3 of Kim [14]),

$$\mu |{\mathit{\xi }}_1,\dots ,{\mathit{\xi }}_{p-1}\sim \mathtt{CRM}({\Lambda }_p,{\{{\omega }_k^{*},P_k\}}_{k=1}^K),$$

where ${\omega }_1^{*},\dots ,{\omega }_K^{*}$ are unique atoms that ${\mathit{\xi }}_1,\dots ,{\mathit{\xi }}_{p-1}$ possess,

$$\begin{array}{cc}\hfill P_k\left(\mathrm{d}q\right)& :={\displaystyle \frac{q^{m_{p,k}+\alpha -1}{(1-q)}^{p-1-m_{p,k}+\kappa -1}\mathrm{d}q}{{\int }_{(0,1]}{q}^{m_{p,k}+\alpha -1}{(1-q)}^{p-1-m_{p,k}+\kappa -1}\mathrm{d}q}}\hfill \\ \hfill & ={\displaystyle \frac{1}{B(m_{p,k}+\alpha ,p-1-m_{p,k}+\kappa )}}{q}^{m_{p,k}+\alpha -1}{(1-q)}^{p-1-m_{p,k}+\kappa -1}\mathrm{d}q,\hfill \end{array}$$

with $m_{p,k}:={\sum }_{j=1}^{p-1}{\mathit{\xi }}_j\left({\omega }_k^{*}\right)$, and

$${\Lambda }_p(\mathrm{d}q,\mathrm{d}\omega ):={\displaystyle \frac{\gamma }{B(\alpha ,\kappa )}}{q}^{\alpha -1}{(1-q)}^{p-1+\kappa -1}\mathrm{d}q{\Lambda }_0\left(\mathrm{d}\omega \right).$$

Thus, for each atom ${\omega }_k^{*}$, we have that

$$\begin{array}{cc}& \mathbb{P}({\mathit{\xi }}_p\left({\omega }_k^{*}\right)=1|{\mathit{\xi }}_1,\dots ,{\mathit{\xi }}_{p-1})\hfill \\ & ={\displaystyle \frac{1}{B(m_{p,k}+\alpha ,p-1-m_{p,k}+\kappa )}}{\int }_{(0,1]}{q}^{m_{p,k}+\alpha }{(1-q)}^{p-1-m_{p,k}+\kappa -1}\mathrm{d}q\hfill \\ & ={\displaystyle \frac{1}{B(m_{p-1,k}+\alpha ,p-1-m_{p-1,k}+\kappa )}}B(m_{p,k}+1+\alpha ,p-1-m_{p,k}+\kappa )\hfill \\ & ={\displaystyle \frac{m_{p,k}+\alpha }{p-1+\kappa +\alpha }}.\hfill \end{array}$$

(16)

On the other hand, for a small change dω in $\omega \in \mathsf{\Omega }\setminus \{{\omega }_1^{*},\dots ,{\omega }_K^{*}\}$, we have

$$\begin{array}{cc}\hfill \mathbb{P}\left({\mathit{\xi }}_p\left(\mathrm{d}\omega \right)=1|{\mathit{\xi }}_1,\dots ,{\mathit{\xi }}_{p-1}\right)& =\mathbb{E}\left[{\mathit{\xi }}_p\left(d\omega \right)|{\mathit{\xi }}_1,\dots ,{\mathit{\xi }}_{p-1}\right]\hfill \\ & ={\displaystyle \frac{\gamma }{B(\alpha ,\kappa )}}{\int }_{(0,1]}{q}^{\alpha }{(1-q)}^{p-1+\kappa -1}\mathrm{d}q{\Lambda }_0\left(\mathrm{d}\omega \right)\hfill \\ & =\gamma {\displaystyle \frac{B(\alpha +1,p-1+\kappa )}{B(\alpha ,\kappa )}}{\Lambda }_0\left(\mathrm{d}\omega \right)\hfill \\ & =\gamma {\psi }_{\alpha ,\kappa }^{1,p-1}{\Lambda }_0\left(\mathrm{d}\omega \right).\hfill \end{array}$$

(17)

Therefore, since  ${\mathit{\xi }}_p$  is completely random and Λ₀ is smooth,  ${\mathit{\xi }}_p$  is a Poisson process with intensity measure  $\gamma {\psi }_{\alpha ,\kappa }^{1,p-1}{\Lambda }_0 \mathrm{on} \mathsf{\Omega }\setminus \{{\omega }_1^{*},\dots ,{\omega }_K^{*}\}$.  This implies that the number of new atoms in  ${\mathit{\xi }}_p$ follows a Poisson distribution with mean  $\gamma {\psi }_{\alpha ,\kappa }^{1,p-1}$.  Combining (16) and (17), we complete the proof.   □

5. Application to Bayesian Sparse Factor Models

In this section, as an application, we consider a Bayesian factor model with the CIBP prior.

5.1. Model and Prior

We assume that p-dimensional random vector $\mathbf{Y}$ is distributed as

$$\mathbf{Y}|\mathbf{Z}=\mathbf{z}\sim {\mathtt{N}}_p(\mathbf{B}\mathbf{z},{\sigma }^2\mathbf{I}),\mathbf{Z}\sim {\mathtt{N}}_K(\mathbf{0},\mathbf{I}),$$

(18)

where B is a factor loading matrix, Z is a latent factor and  ${\sigma }^2>0$  is a noise variance. Let  ${\beta }_{jk}$  be the (j, k)-th element of B. We impose the CIBP prior distribution on B as

$$\begin{array}{cc}\hfill {\beta }_{jk}|{\xi }_{jk}& \stackrel{\mathrm{ind}}{\sim }(1-{\xi }_{jk}){\delta }_0+{\xi }_{jk}\mathtt{N}(0,\tau ),j\in \left[p\right],k\in \left[K\right]\hfill \\ \hfill {\xi }_{jk}|{\theta }_k& \stackrel{\mathrm{ind}}{\sim }\mathtt{Bernoulli} \left({\theta }_k\right),j\in \left[p\right],k\in \left[K\right]\hfill \\ \hfill {\theta }_k& \stackrel{\mathrm{iid}}{\sim }\mathtt{Beta}(\alpha ,\kappa ),k\in \left[K\right]\hfill \\ \hfill K& \sim \mathtt{Poisson}\left(\gamma \right)\hfill \end{array}$$

with κ ≥ 0 and τ > 0. This means that we impose $\mathtt{CIBP}(\gamma ,\alpha ,\kappa )$  on the binary matrix   $\Xi :={\left({\xi }_{jk}\right)}_{j\in \left[p\right],k\in \mathbb{N}}.$ The above prior distribution on B is named as   ${\mathtt{SSCIBP}}_p(\gamma ,\alpha ,\kappa ,\tau ),$  which is a shorthand for spike−and−slab CIBP.

5.2. Posterior Computation

In this sectino, we describe a Markov chain Monte Carlo (MCMC) algorithm for computing the posterior distribution under the ${\mathtt{SSCIBP}}_p(\gamma ,\alpha ,\kappa ,\tau )$ prior on $\mathbf{B}$ and inverse Gamma prior $\mathtt{IG}(a,b)$ on ${\sigma }^2$. We denote by $K^{+}$ the number of nonzero columns of the loading matrix $\mathbf{B}$.

Sample ${\beta }_{jk}$ for $j\in \left[p\right]$ and $k\in \left[K^{+}\right]$. 

We sample ${\beta }_{jk}$ ss

$${\beta }_{jk}|-\sim \left\{\begin{array}{cc}\mathtt{N}({\widehat{\beta }}_{jk},{\widehat{\tau }}_k)\hfill & \mathrm{if}{\xi }_{jk}=1\hfill \\ {\delta }_0\hfill & \mathrm{if}{\xi }_{jk}=0,\hfill \end{array}\right.$$

where

$$\begin{array}{cc}\hfill {\widehat{\tau }}_k& :={ ({\sigma }^{-2}\sum _{i=1}^n{Z}_{ik}^2+{\tau }^{-1})}^{-1}\hfill \\ \hfill {\widehat{\beta }}_{jk}& :={\widehat{\tau }}_k\{{\sigma }^{-2}\sum _{i=1}^n{Z}_{ik}(Y_{ij}-\sum _{h\in \left[K^{+}\right]:h\ne k}{Z}_{ih}{\beta }_{jh})\}.\hfill \end{array}$$

Sample ${\xi }_{jk}$ for $j\in \left[p\right]$ and $k\in \mathbb{N}$. 

Since the CIBP is exchangeable, without loss of generality, we can assume that the j-th customer is the last customer. For each $k\in \left[K^{+}\right]$, ${\xi }_{jk}$ is sampled with probability

$${\displaystyle \frac{\Pi ({\xi }_{jk}=1|-)}{\Pi ({\xi }_{jk}=0|-)}}={\displaystyle \frac{m_{j,k}+\alpha }{\kappa +p-1-m_{j,k}}}\sqrt{{\displaystyle \frac{{\widehat{\tau }}_k}{\tau }}}exp\left({\displaystyle \frac{1}{2{\widehat{\tau }}_k}}{\widehat{\beta }}_{jk}^2\right),$$

where $m_{j,k}:={\sum }_{l\in \left[p\right]:l\ne j}{\xi }_{lk}$. Next, we sample ${\xi }_{jk}$ for $k>K^{+}$. by the Metropolis–Hastings (MH) step as follows: We generate the proposal $K_j^{*}\in \mathbb{N}\cup \left\{0\right\}$ and ${\mathit{\beta }}_j^{*}:=({\beta }_{j,1}^{*},\dots ,{\beta }_{j,K_j^{*}}^{*})\in {\mathbb{R}}^{K_j^{*}}$ from the distribution

$$J\left(K_j^{*}\right)J\left({\beta }_j^{*}\right|K_j^{*})=\mathtt{Poisson}\left(1\right)\mathtt{N}{(0,\tau )}^{K_j^{*}}.$$

and then we accept the proposal with probability

$$min\left\{1, {\left|{\mathbf{M}}_j\right|}^{-n/2}exp ({\displaystyle \frac{1}{2}}{\left({\mathit{\beta }}_j^{*}\right)}^{\top }{\mathbf{M}}_j^{-1}{\mathit{\beta }}_j^{*}\sum _{i=1}^n{E}_{ij}^2){ \left(\gamma {\psi }_{\alpha ,\kappa }^{1,p-1}\right)}^{K_j^{*}}\right\},$$

where

$$\begin{array}{cc}\hfill {\mathbf{M}}_j& :={\sigma }^{-2}{\mathit{\beta }}_j^{*}{\left({\mathit{\beta }}_j^{*}\right)}^{\top }+\mathbf{I}\hfill \\ \hfill E_{ij}& :={\sigma }^{-2} (Y_{ij}-\sum _{k=1}^{K^{+}}{Z}_{ik}{\beta }_{jk}).\hfill \end{array}$$

If the proposal is accepted, we update

$$\begin{array}{cc}\hfill \mathbf{B}& \leftarrow (\mathbf{B},{({\beta }_{j,k}^{*}\mathbb{I}(j^{\prime }=j))}_{j^{\prime }\in \left[p\right],k\in \left[K_j^{*}\right]})\hfill \\ \hfill K^{+}& \leftarrow K^{+}+K_j^{*}.\hfill \end{array}$$

Sample ${\mathbf{Z}}_i$ for $i\in \left[n\right]$. 

We sample ${\mathbf{Z}}_i$ as

$${\mathbf{Z}}_i|-\sim \mathtt{N} ({\sigma }^{-2}{\widehat{\mathsf{\Sigma }}}_{\mathbf{Z}}{\mathbf{B}}^{\top }{\mathbf{Y}}_i,{\widehat{\mathsf{\Sigma }}}_{\mathbf{Z}}),$$

where ${\widehat{\mathsf{\Sigma }}}_{\mathbf{Z}}:={({\sigma }^{-2}{\mathbf{B}}^{\top }\mathbf{B}+\mathbf{I})}^{-1}$.

Sample ${\sigma }^2$. 

We sample the noise variance ${\sigma }^2$ as

$${\sigma }^2|-\sim \mathtt{IG} (a+{\displaystyle \frac{np}{2}},b+{\displaystyle \frac{1}{2}}\sum _{i=1}^n\sum _{j=1}^p{ (Y_{ij}-\sum _{k=1}^{K^{+}}{Z}_{ik}{\beta }_{jk})}^2).$$

The dominant computational cost of our sampler is $\mathcal{O}\left(np{K}^{+}\right)$ per iteration, and since $K^{+}$ remains stochastically bounded under the CIBP, the complexity is linear in p, unlike in the classical IBP where $K^{+}=\mathcal{O}(logp)$.

5.3. Simulation

We perform a simulation study to compare the CIBP and the two-parameter IBP when they serve as prior distributions in the sparse factor model. We generate simulated datasets as follows: For each value $p\in \{100,250,500,1000\}$, we generate a $p\times 5$-dimensional loading matrix ${\mathbf{B}}_0$ having 10 nonzero rows. Each factor loading in the sampled nonzero rows are generated from the uniform distribution on $[-3,3]$. We generate $n=50$ random vectors from the normal distribution with mean $\mathbf{0}$ and variance ${\mathbf{B}}_0{\left({\mathbf{B}}_0\right)}^{\top }+\mathbf{I}$ independently. For each configuration of p, we generate 100 independent datasets.

In all experiments, we compare $\mathtt{CIBP}(\gamma ,\alpha ,\kappa )$ with $\gamma =0.1$, $\alpha =10$ and $\kappa =10$ and $\mathtt{IBP}({\gamma }^{†},\kappa )$ with ${\gamma }^{†}=0.1$ and $\kappa =10$. Note that ${\gamma }^{†}=0.1$ is equal to $\gamma \alpha /\kappa =0.1\times 10/10$ used for the CIBP. Posterior inference is performed using the MCMC scheme described in Section 5.2. We run 10,000 iterations with 5000 burn-in and thin every 10th draw.

Figure 2 gives the posterior distributions of the number of nonzero factors under the $\mathtt{CIBP}(\gamma ,\alpha ,\kappa )$ and $\mathtt{IBP}({\gamma }^{†},\kappa )$ priors, respectively, aggregated over 100 replications for each $p\in \{100,250,500,1000\}$. As the dimension p grows, the $\mathtt{IBP}({\gamma }^{†},\kappa )$ prior tends to considerably overestimate the number of factors. In contrast, the $\mathtt{CIBP}(\gamma ,\alpha ,\kappa )$ prior yields accurate estimates of the number of factors for all considered values of p.

In

Table 2. Comparison between CIBP and IBP priors in sparse factor modeling. Reported are averages over 10 datasets with standard errors in parentheses.

p	$\mathbb{E}\left[{\mathit{K}}^{+}\right]$		Sparsity (%)		Predictive Log-Likelihood
	CIBP	IBP	CIBP	IBP	CIBP	IBP
100	5.12 (0.18)	6.54 (0.22)	81.82 (0.38)	84.97 (0.52)	$-1.72$ (0.02)	$-1.74$ (0.02)
250	5.18 (0.19)	6.81 (0.29)	85.05 (0.34)	82.87 (0.61)	$-1.70$ (0.02)	$-1.76$ (0.03)
500	5.29 (0.21)	7.92 (0.41)	95.21 (0.30)	88.15 (0.73)	$-1.69$ (0.02)	$-1.79$ (0.03)
1000	5.24 (0.23)	7.53 (0.56)	97.46 (0.27)	87.09 (0.82)	$-1.68$ (0.02)	$-1.83$ (0.04)

reports the averages of the following metrics across the 100 replications: (1) posterior mean $\mathbb{E}\left[K^{+}\right]$ of the number of active features (nonzero columns of $\mathbf{B}$), (2) the proportion of zero entries in the posterior median of $\mathbf{B}$ (sparsity), and (3) average test log-likelihood per observation, evaluated on an independent test set of size 100. Under the CIBP prior, $\mathbb{E}\left[K^{+}\right]$ remains stable around the true value five across all values of p. In contrast, the IBP prior yields an increasing number of active features as p grows. For the sparsity of the loading matrix, the CIBP consistently produces sparser loading matrices than the IBP. Despite using fewer features and sparser loading matrix, the CIBP yields equal or better predictive log-likelihood compared to the IBP. These results support the claim that the CIBP provides a parsimonious yet predictive alternative to the IBP when the underlying number of latent features does not diverge.

In Appendix A, we provide the results of a sensitivity analysis regarding the choice of hyperparameters, as well as convergence diagnostics for our MCMC algorithm.

6. Conclusions

In this paper, we introduced the CIBP, a novel three-parameter extension of the IBP. The key distinction between the CIBP and standard IBPs is that, under the CIBP, the distribution of the number of features converges to a stable limiting distribution as the number of objects increases. This property helps the CIBP avoid superfluous features in both interpretation and prediction. We evaluated the proposed CIBP within a high-dimensional sparse factor model and provided empirical evidence that it outperforms the IBP in estimating the number of factors.

Although our focus in this work is on posterior sampling, the proposed framework can be combined with several strategies to further improve scalability. First, stochastic subsampling may be used to approximate likelihood contributions when p is large, following advances in subsampling-based MCMC for large-scale Bayesian inference [15]. Second, collapsed or partially collapsed sampling can be employed, which is known to improve mixing in nonparametric feature allocation models, including IBP [16]. Finally, variational inference offers a substantially faster alternative for large-scale applications and can be adapted from the existing variance inference method for the standard IBP [17]. A systematic investigation of these computational refinements is left for future research.