A Review: Construction of Statistical Distributions

Abstract

Statistical modeling is fundamentally based on probability distributions, which can be discrete or continuous and univariate or multivariate. This review focuses on the methods used to construct these distributions, covering both traditional and newly developed approaches. We first examine classic distributions such as the normal, exponential, gamma, and beta for univariate data, and the multivariate normal, elliptical, and Dirichlet for multidimensional data. We then address how, in recent decades, the demand for more flexible modeling tools has led to the creation of complex meta-distributions built using copula theory.

1. Introduction

Statistical distributions form the foundation of probability theory, statistical inference, and predictive modeling. They provide a mathematical framework for describing the randomness, variability, and uncertainty inherent in data. Different types of data—univariate or multivariate, discrete or continuous—require different distributions, including mixture models for complex patterns.

Numerous books have offered comprehensive introductions to useful distributions. The seminal four-volume work by Johnson et al. [1,2,3,4] provides an exceptionally detailed encyclopedia of distributions and their interrelations, though its sheer scope can make it challenging for readers to locate specific information efficiently. Other key texts include Fang and Xu’s book [5] (in Chinese, which limits its accessibility), Fang et al. [6] focusing treatment on multivariate symmetric distributions, Johnson [7] reviewing statistical simulation for multivariate distributions, and the new work by Fang et al. [8] offering a comprehensive study on representative points of distributions and their applications.

The purpose of this review is to synthesize various methods for constructing statistical distributions. By presenting a unified overview of both traditional and modern techniques, we aim to equip readers with the knowledge to understand existing distributions and develop new ones to meet evolving data modeling needs.

This paper is structured as follows. Section 2 introduces construction methods for univariate distributions, encompassing both traditional and newer models. Section 3 briefly reviews methods for constructing discrete distributions, including both univariate and multivariate frameworks. Section 4.4 then details construction techniques for multivariate statistical distributions, specifically the families of elliptical distributions and meta-distributions created via copula methods. The paper concludes with final remarks in Section 5.

2. Construction of Univariate Statistical Distributions

This section introduces several approaches for constructing univariate continuous distributions:

(1)

Distribution families, including location-scale families, the Pearson system, and exponential families;

(2)

Functions of random variables via stochastic representations;

(3)

Parameter expansion generalizes existing families by adding parameters. For example, normal→ skew normal; exponential → Weibull.

(4)

Generalized families via transformation of the cdf. A powerful and prolific approach for generating new families of distributions is through the functional transformation of an existing baseline cdf, F(x). These transformations introduce new parameters, thereby creating more flexible distributions capable of modeling complex data behaviors.

(5)

Combination methods, such as mixture distributions, compound distributions, convolutions, and copula-based techniques.

2.1. Distribution Families

A distribution family is defined by a specific structural property. Among many such families, the following three are particularly well established in the literature.

2.1.1. Location-Scale Distribution Family

A distribution F(x; μ, σ) belongs to the location-scale family if its probability density function (pdf) can be expressed as

$$f(x)={\displaystyle \frac{1}{\sigma }}{f}_0\left({\displaystyle \frac{x-\mu }{\sigma }}\right),$$

(1)

where μ is the location parameter, σ > 0 is the scale parameter, and f₀ is the standard density of the distribution Z ∼ F(x; 0, 1). This implies that for any X ∼ F(x; μ, σ), the standardized variable $Z={\displaystyle \frac{(X-\mu )}{\sigma }}$ follows the standard distribution F(x; 0, 1).

A key property of this family is its invariance under affine transformations, implying that if X ∼ F(x; μ, σ), then a + bX ∼ F(x; a + bμ, |b|σ). The normal distribution is a classic example of the location-scale distribution. Other prominent examples include the Cauchy and logistic distributions.

• Cauchy distribution: Its pdf is given by

  $$\begin{array}{c}\hfill f(x;\theta ,\lambda )={\displaystyle \frac{1}{\pi \lambda \left[1+{\left(\frac{x-\theta }{\lambda }\right)}^2\right]}},\end{array}$$

  where θ is the location parameter and λ is the scale parameter. Note that the Cauchy distribution does not have a defined mean or variance.
• Logistic distribution: Named for its relationship to the logistic (sigmoid) function, the logistic distribution resembles the normal distribution in shape but possesses heavier tails. It finds applications in logistic regression, growth models, and economics. Its cumulative distribution function (cdf) is given by

  $$\begin{array}{c}\hfill F(x;\alpha ,\beta )={\displaystyle \frac{1}{1+\exp (-(x-\alpha )/\beta )}},x,\alpha \in \mathbb{R},\beta >0.\end{array}$$

There are more other distributions in this category. For further reading, see [9,10].

2.1.2. Pearson Distribution Family

The Pearson family is a system of continuous probability distributions, introduced by Karl Pearson [11], designed to model a wide range of empirical data characterized by properties such as skewness and kurtosis that deviate from the normal distribution. The system is defined by the Pearson differential equation:

$$\begin{array}{c}\hfill {\displaystyle \frac{df(x)}{dx}}=f(x){\displaystyle \frac{x-\lambda }{a{x}^2+bx+c}},\end{array}$$

(2)

where f(x) is a probability density function and the parameters a, b, c, λ determine the specific distribution type.

Roots of the denominator (ax² + bx + c) classify distributions into eight or more types listed in

Table 1. Classification of Pearson distributions.

Type	Condition	Distribution	PDF Example
0	a = 0, b = 0	Normal distribution	$f(x)\propto e^{-{(x-\mu )}^2/2{\sigma }^2}$
I	b2 − 4ac > 0	Beta distribution	f(x) ∝ (1 + x)r1(1 − x)r2, |x| ≤ 1
II	b2 − 4ac = 0, λ = − b/2a	Symmetric beta	$f(x)\propto {(1-x^2)}^r,\left|x\right|\le 1$
III	a = 0	Gamma distribution	f(x) ∝ xk−1e−x/θ. x > 0
IV	b2 − 4ac < 0	Skewed heavy-tailed	Complex form
V	c = 0	Inverse gamma	f(x) ∝ xk−1e−θ/x, x > 0
VI	b2 − 4ac > 0, $\lambda \ne \mathrm{root}$	Beta prime, F-distribution	f(x) ∝ xr1(1 + x)−(r1+r2)
VII	b = 0	Student’s t-distribution	$f(x)\propto {(1+x^2/\nu )}^{-(\nu +1)/2}$

, where type 0 is the normal distribution; type I distribution is the beta distribution; type II distribution curve is a symmetrical U-shaped curve; type III distribution curve is the gamma distribution; type IV distribution curve is skewed heavy-tailed; type V is the shift inverse gamma distribution; type VI is an inverse beta distribution; type VII is the t-distribution. Some references consider more types, like types VIII and IX, which are special cases of the power function distribution, and type X, which is the exponential distribution. More details can be found in Chapter 12 of [1,12].

2.1.3. The Exponential Distribution Family

Consider it as a “family” in the biological sense, implying that though its members, such as the normal, exponential, gamma, beta, and Poisson distributions, may appear distinct characteristics, they share a common underlying mathematical structure. This shared “DNA” makes them exceptionally powerful and convenient to use in statistical modeling and machine learning.

Formally, the exponential family comprises distributions whose probability density (or mass) functions can be expressed in the following canonical form:

$$\begin{array}{c}\hfill f(x;\mathit{\theta })=h(x)\exp \left\{{\mathit{\theta }}^{\top }\mathbf{T}(x)-B(\mathit{\theta })\right\},\end{array}$$

(3)

where h(x) is a function as base measure that depends only on the data X, not the parameters $\mathit{\theta }={({\theta }_1,\dots ,{\theta }_k)}^{\top }$. $\mathbf{T}(x)={(T_1(x),\dots ,T_k(x))}^{\top }$ is a sufficient statistic, and B(θ) is a log-partition function and a normalization constant that ensures that the function integrates/sums to 1. B(θ) also generates the moments (mean, variance) of the distribution.

This common structure leads to several powerful properties:

(1)

All members have a low-dimensional sufficient statistic T(x).

(2)

In Bayesian statistics, the exponential family always has a natural conjugate prior. This makes Bayesian updating mathematically neat and computationally tractable.

(3)

Distributions in the exponential family are the maximum entropy distributions given constraints on the expected values of the sufficient statistics. For example, the normal distribution has the maximum entropy for a given mean and variance.

Further details can be found in references such as [1,9]. Many additional families of univariate distributions have also been cataloged in [1].

2.2. Distributions Produced by a Function of Random Variables

When the distribution of a random variable X or a random vector $\mathit{X}=(X_1,\dots ,X_k)$ is known, applying a continuous function Y = g(X) or Y = h(X) in order to yield a new distribution is a prevalent distribution construction approach. In this context, the original distribution is termed the initial distribution, while the resulting distribution is called the target distribution. The inverse transformation method [13] is a widely used technique for statistical simulation, based on the following theorem.

Theorem 1.

Let X have a continuous cdf F(x) and a uniform random variable U ∼ U(0, 1). Then we have

(a) 

F⁻¹(U) ∼ F(·);

(b) 

F(X) ∼ U(0, 1).

Following Theorem 1, we can leverage this connection between any continuous distribution and the uniform distribution to derive the distribution of a transformed random variable Y = g(X). The key insight is to use the probability integral transform from part (b) of the theorem. We know that F_X(X) ∼ U(0, 1). Therefore, the cdf of the transformation Y = g(X) can be found by relating the event {Y ≤ y} back to this uniform distribution. Specifically, the cdf of Y is

$$F_Y(y)=P(Y\le y)=P(g(X)\le y).$$

The solution to this probability depends on the nature of the function g. For monotonic and differentiable transformations, this approach leads directly to the well-known change of variables formula for the pdf of Y:

$$f_Y(y)=f_X(x)\left|{\displaystyle \frac{dx}{dy}}\right|=f_X(g^{-1}(y))\left|{\displaystyle \frac{d}{dy}}{g}^{-1}(y)\right|,$$

where x = g⁻¹(y) is the inverse function. This formula can be derived by differentiating the cdf F_Y(y), which itself is computed using the distribution of X that Theorem 1 helps us to understand and manipulate through the uniform distribution.

A systematic study and generalization of the inverse transformation method can be found in Li et al. [14]. Numerous distributions have been constructed using this approach. The following are some functions commonly employed in the literature for this purpose.

2.2.1. Transformation of Random Variables

Some transformation examples are presented later.

• Linear transformation Y = a + bX, for example, used in the location-scale distributions;
• Power transformation Y = X^r, for example, from the normal to the χ² distribution by Y = X²;
• Exponential transformation Y = e^X, for example, from the normal to the lognormal distribution by Y = e^X;
• Inverse transformation Y = 1/X, for example, the inverse gamma distribution.
• Transformation $Y=g(X_1,\dots ,X_m)$, for example, the F-distribution and beta distribution are produced by functions of two χ² distribution variables.

2.2.2. Order Statistics

These include fundamental objects in statistics, for example, the sample minimum, maximum, and median. They deal with the properties and applications of sorted random samples. Let $X_1,\dots ,X_n$ be a random sample from a population with cdf F and pdf f. The order statistics, denoted by $X_{(1)}\le X_{(2)}\le \cdots \le X_{(n)}$, represent the sorted values of the sample. The cdf of the kth order statistic, X_(k), is given by

$$\begin{array}{c}\hfill P(X_{(k)}\le x)=\sum _{j=k}^n\left({\displaystyle \genfrac{}{}{0pt}{}{n}{j}}\right){\left[F(x)\right]}^j{[1-F(x)]}^{n-j}.\end{array}$$

(4)

The density functions for specific order statistics are provided in

Table 2. Distributions of the order statistics.

Distribution	Density $\mathit{f}(\mathit{x})$
The general k-th order statistic 1	$({\displaystyle \genfrac{}{}{0pt}{}{n}{k-1}}){\left[F(x)\right]}^{k-1}{[1-F(x)]}^{n-k}f(x)$
Minimum (X(1))	n[1 − F(x)]n−1f(x)
Maximum (X(n))	n[F(x)]n−1f(x)

¹ Note: The pdf for the median exists in a closed form only when n is odd (n = 2m + 1), given by $p_{\mathrm{median}}(x)={\displaystyle \frac{n!}{{(m!)}^2}}{\left[F(x)\right]}^m{[1-F(x)]}^{m}f(x)$. For even n, the median is defined as ${\displaystyle \frac{1}{2}}(X_{(n/2)}+X_{(n/2+1)})$ and its distribution is derived from the joint distribution of these two order statistics.

. Order statistics have wide-ranging applications, including robust statistics, sample range and interquartile range, quantiles and Q–Q plots, reliability engineering, and non-parametric statistical tests (e.g., the Wilcoxon rank-sum test). A comprehensive introduction to order statistics can be found in [15].

2.2.3. Extreme Value Theory (EVT)

EVT is a statistical framework for assessing the probabilities of extreme events. The fundamental result in EVT is the extreme value theorem, stating that the distribution of properly normalized maxima (or minima) converges to one of three types of distributions, regardless of the underlying distribution of the original data. These three types are collectively known as the generalized extreme value (GEV) distribution. The cdf for the GEV is given by

$$\begin{array}{c}\hfill F(x)=\exp \left\{-{[1+\xi ({\displaystyle \frac{x-\mu }{\sigma }})]}^{-1/\xi }\right\},\mathrm{if}1+\xi {\displaystyle \frac{x-\mu }{\sigma }}>0,\end{array}$$

(5)

where parameters are μ for location, σ > 0 for scale, and ξ for shape. The three special cases of the GEV distribution are

• Type I (Gumbel distribution) Gu(μ, σ) light-tailed, with ξ = 0. Its pdf is

  $${\displaystyle \frac{1}{\sigma }}\exp \left\{-{\displaystyle \frac{x-\mu }{\sigma }}-\exp \left\{-{\displaystyle \frac{x-\mu }{\sigma }}\right\}\right\},\sigma >0.$$
• Type II (Fr$\acute{\mathit{e}}$chet distribution) heavy-tailed, with ξ > 0.
• Type III (Weibull distribution) short-tailed and bounded with ξ < 0.

The extreme value distribution has been used in finance, environmental science, engineering, and reliability testing. For a rigorous treatment of the GEV distribution and extreme value theory, see [16].

2.3. Generalized Families via Transformation of the cdf

Given a baseline cdf F(x), the exponentiated (or power-transformed) family is defined by

$$G(x)={\left[F(x)\right]}^{\alpha },\alpha >0.$$

This construction, detailed in [17], generates a vast range of new distributions. If $X_1,X_2,\dots ,X_n$ is a random sample from F(x), then G(x) is the cdf of the sample maximum, $max(X_1,\dots ,X_n)$, when α = n is an integer. Notable members include the exponentiated Weibull and exponentiated exponential distributions.

A dual transformation yields the following complementary exponentiated family:

$$G(x)=1-{[1-F(x)]}^{\beta },\beta >0.$$

This is the cdf of the sample minimum, $min(X_1,\dots ,X_n)$, when β = n. It is also known as the proportional odds model in survival analysis. Together, the exponentiated and complementary exponentiated families form the foundation of the broader class of maximum–minimum stable distributions.

Further generalizations have been proposed to introduce additional flexibility and skewness. The Alpha Power Family (or T-X family), defined by

$$G(x)={\displaystyle \frac{{\alpha }^{F(x)}-1}{\alpha -1}},\alpha >0,\alpha \ne 1,$$

effectively introduces skewness to the baseline distribution [18]. A highly flexible generalization is the Kumaraswamy-G family [19], given by

$$G(x)=1-{\left[1-F{(x)}^a\right]}^b,a>0,b>0.$$

This family, which contains the exponentiated family as a special case (b = 1), enjoys the advantage of having a quantile function in a simple closed form.

These methods, along with other systematic frameworks like the T-X family [20], demonstrate that simple transformations of a baseline cdf can serve as an engine for generating numerous new and useful distributions, greatly expanding the toolbox for statistical modeling.

2.4. Combination Methods

There are several approaches to constructing a distribution through combining the existing distributions.

2.4.1. Types of Mixtures

• Finite mixture $f(x)={\sum }_{i=1}^k{w}_i{f}_i(x)$, which has various applications in statistical inference with flexibility in density fitting [21], for example, a mixture of two normals [22] or two Weibull distributions [23].
• Continuous mixture f(x) = ∫ f(x; θ)g(θ)dθ. One example is the scale mixture. A primary method to create such continuous mixtures is through compounding, which we detail in the next section.

The mixing distribution g(θ) in an infinite mixture is not required to be continuous. When g(θ) is a discrete distribution, the resulting mixture, often called a countable mixture or discrete mixture, integrates over a countable set of parameters. This approach is technically distinct from the continuous mixture defined above but remains a powerful tool for deriving flexible distributions. Prominent examples of distributions derived from a discrete mixing mechanism include the following:

• The negative binomial distribution, which can be derived as a Poisson-gamma mixture where the Poisson rate parameter is governed by a gamma distribution. While the gamma is continuous, a discrete analogue exists (e.g., a Poisson mixture with a discrete log-gamma distribution would yield a similar over-dispersed count distribution).
• The Student’s t-distribution, which is a scale mixture of normal distributions where the precision (inverse variance) parameter follows a gamma distribution.
• The beta-binomial distribution, which arises when the success probability p of a binomial distribution follows a beta distribution.
• The compound Poisson distribution and related models, where the intensity or other parameters are driven by a latent discrete process.

2.4.2. Compounding

It is a fundamental concept in probability theory that a parameter of one distribution is itself a random variable following another distribution (see [1]). This approach generates a more flexible distribution that incorporates uncertainty in the parameter. It is important to note that the marginal distribution resulting from compounding is mathematically equivalent to a continuous mixture distribution, as defined in Section 2.4.1. The compounding framework provides a hierarchical data-generation perspective for the same model. Formally, if X|θ ∼ F₁(θ) and θ ∼ F₂(λ), then the distribution of X is the marginal distribution of a compound distribution. Examples are the beta-binomial, gamma-Poisson (negative binomial), normal–normal, Dirichlet–multinomial, compound Poisson process models. The normal–normal model is as follows:

• First stage (conditional distribution): $X|\mu \sim \mathcal{N}(\mu ,{\sigma }^2)$. The observation X is normally distributed with mean μ and a known variance σ²;
• Second stage (parameter distribution): $\mu \sim \mathcal{N}({\mu }_0,{\tau }^2)$. The mean μ itself is assumed to be normally distributed with prior mean μ₀ and variance τ²;
• Compound distribution: The unconditional distribution of $X\sim \mathcal{N}({\mu }_0,{\sigma }^2+{\tau }^2)$.

2.4.3. Convolution

The random variable Z = X + Y is termed the convolution of X and Y, and it has a density function given by

$$f_{X+Y}(z)={\int }_{-\infty }^{+\infty }{f}_{X,Y}(x,z-x)dx,$$

where f_X,Y(·, ·) is the joint density of (X, Y). If X and Y are independent, this simplifies to $f_{X+Y}(z)=(f_X\ast f_Y)(z)={\int }_{-\infty }^{+\infty }{f}_X(x)f_Y(z-x)dx$.

A distribution of significant practical importance arises when X follows a normal distribution N(μ, σ²) and Y follows an exponential distribution Exp(λ), with X and Y being independent. This specific convolution forms the cornerstone of the normal–exponential stochastic frontier model [24], widely used in econometrics to measure the efficiency of firms and other decision-making units. In this context, the normal random variable represents the random noise (e.g., measurement error, luck), while the exponential random variable captures one-sided inefficiency.

Then the density function of Z = X + Y is given by

$$f_Z(z;\mu ,\sigma ,\lambda )={\displaystyle \frac{\lambda }{2}}{e}^{{\displaystyle \frac{\lambda }{2}}(2\mu +\lambda {\sigma }^2-2z)}·\mathrm{erfc}\left({\displaystyle \frac{\mu +\lambda {\sigma }^2-z}{\sqrt{2}\sigma }}\right),$$

where erfc(z) represents the complementary error function defined as

$$\mathrm{erfc}(z)={\displaystyle \frac{2}{\sqrt{\pi }}}{\int }_z^{\infty }{e}^{-t^2}dt.$$

2.5. Some Key Univariate Distributions and Related Distributions

The first continuous statistical distribution to be formally studied was the uniform distribution, dating back to Huygens [25]. However, the most historically significant early continuous distribution is the normal distribution, also referred to as the Gaussian distribution. Although the uniform distribution provided the simplest continuous model, it was the normal distribution that first underwent extensive analytical study. The development of the normal distribution emerged later through the work of de Moivre [26] and Gauss [27] during the 18th and 19th centuries. Its fundamental role in error theory and the central limit theorem established it as the cornerstone of continuous probability theory.

2.5.1. Distributions Related to the Normal Distribution

The probability density function of a normal distribution X ∼ N(μ, σ²) is given by

$$\begin{array}{c}\hfill f(x;\mu ,\sigma )={\displaystyle \frac{1}{\sqrt{2\pi }\sigma }}{e}^{-{\displaystyle \frac{1}{2}}{({\displaystyle \frac{x-\mu }{\sigma }})}^2},-\infty <x<\infty ,\end{array}$$

(6)

where parameters μ and σ² are the mean and variance of X, respectively. When μ = 0 and σ² = 1, the distribution is referred to as the standard normal distribution, denoted by Z ∼ N(0, 1). Several distributions derived from the normal distribution are listed in Table 3 and can be constructed as follows:

(1)

Lognormal distribution: X ∼ LN(μ, σ²) if ln(X) ∼ N(μ, σ²);

(2)

Skew-normal distribution: proposed by Azzalini [28] with the pdf

$$\begin{array}{c}\hfill f(z;\alpha )=2\varphi (z)\mathrm{\Phi }(\alpha z),z\in \mathbb{R},\alpha \in \mathbb{R},\end{array}$$

(7)

where ϕ(z) is the standard normal pdf, Φ(αz) is is the standard normal cdf evaluated at αz and α is a shape parameter. For more general cases, refer to Table 3. Skew-normal distribution is one of the skew-elliptical distributions, and there are more such distributions including skew-t, skew-Cauchy, skew-logistic, and skew-Laplace distributions; see [29,30]. There is more discussion about this topic in Section 4.6.

(3)

Chi-square distribution: If $X_1,\dots ,X_{\nu }$ are independent and identically distributed (i.i.d.) random variables of N(0, 1), then $X=X_1^2+\dots +X_{\nu }^2\sim {\chi }^2(\nu )$, the χ²-distribution with degrees of freedom ν;

(4)

Chi distribution: If $X_1,\dots ,X_{\nu }$ are i.i.d. N(0, 1), then $X=\sqrt{X_1^2+\dots +X_{\nu }^2}\sim \chi (\nu )$ the χ-distribution with degrees of freedom ν;

(5)

Beta distribution (special case): If Y ∼ χ²(2m) and Z ∼ χ²(2n) are independent, then $X={\displaystyle \frac{Y}{Y+Z}}\sim Be(m,n)$, referred to Table 3;

(6)

Beta prime distribution: If X ∼ Be(α, β), then ${\displaystyle \frac{X}{1-X}}\sim BP(\alpha ,\beta )$;

(7)

Arcsine distribution: It is defined as $Be({\displaystyle \frac{1}{2}},{\displaystyle \frac{1}{2}})$;

(8)

F-distribution: If Y ∼ χ²(m) and Z ∼ χ²(n) are independent, then $X={\displaystyle \frac{nY}{mZ}}\sim F(m,n)$;

(9)

t-distribution: If Y ∼ χ²(ν) and Z ∼ N(0, 1) are independent, then $T=\sqrt{\nu }{\displaystyle \frac{Z}{\sqrt{Y}}}\sim t_{\nu }$.

Table 3. Some continuous distributions constructed by or closely related to N(μ, σ²).

Distribution	Notation	Density $\mathit{f}(\mathit{x})$
Normal	N(μ, σ2)	${\displaystyle \frac{1}{\sqrt{2\pi }\sigma }}\exp \left\{-{\displaystyle \frac{{(x-\mu )}^2}{2{\sigma }^2}}\right\},x,\mu \in \mathbb{R},\sigma >0$
Lognormal	LN(μ, σ2)	${\displaystyle \frac{1}{x\sigma \sqrt{2\pi }}}\exp \left(-{\displaystyle \frac{{(lnx-\mu )}^2}{2{\sigma }^2}}\right)$, $x>0,\mu \in \mathbb{R},\sigma >0$
Skew-normal	SN(μ, σ, α)	$f(x;\mu ,\sigma ,\alpha )={\displaystyle \frac{2}{\sigma }}\varphi \left({\displaystyle \frac{x-\mu }{\sigma }}\right)\mathrm{\Phi }\left(\alpha \left({\displaystyle \frac{x-\mu }{\sigma }}\right)\right),\sigma >0,x,\mu ,\alpha \in \mathbb{R}$
χ2(ν)	χ2(ν)	${\left[2^{\nu /2}\mathrm{\Gamma }({\displaystyle \frac{\nu }{2}})\right]}^{-1}{x}^{\nu /2-1}{e}^{-x/2},x>0,\nu >0$
χ(ν)	χν	${\left[2^{\nu /2-1}\mathrm{\Gamma }({\displaystyle \frac{\nu }{2}})\right]}^{-1}{x}^{\nu -1}{e}^{-x^2/2},x>0,\nu >0$
Beta	Be(a, b)	${\displaystyle \frac{1}{B(a,b)}}{x}^{a-1}{(1-x)}^{b-1},0\le x\le 1,a,b>0$
Beta prime	BP(α, β)	${\displaystyle \frac{1}{\mathrm{B}(\alpha ,\beta )}}{\displaystyle \frac{x^{\alpha -1}}{{(1+x)}^{\alpha +\beta }}},x>0,\alpha ,\beta >0$
Arcsine	arcsine	${\displaystyle \frac{1}{\pi \sqrt{x(1-x)}}},0<x<1$
F	F(m, n)	${\displaystyle \frac{1}{\mathrm{B}(\frac{m}{2},\frac{n}{2})}}{\left({\displaystyle \frac{m}{n}}\right)}^{{\displaystyle \frac{m}{2}}}{x}^{{\displaystyle \frac{m}{2}}-1}{\left(1+{\displaystyle \frac{m}{n}}x\right)}^{-{\displaystyle \frac{m+n}{2}}},x>0,m,n>0$
Student-t	tν	${\displaystyle \frac{\mathrm{\Gamma }(\frac{\nu +1}{2})}{\sqrt{\nu \pi }\mathrm{\Gamma }(\frac{\nu }{2})}}{\left(1+{\displaystyle \frac{x^2}{\nu }}\right)}^{-{\displaystyle \frac{\nu +1}{2}}},\nu >0,x\in \mathbb{R}$

Note: The title “constructed by” implies a direct derivation from the normal distribution. For distributions like χ², χ, F, and t, this direct construction (e.g., as a sum of squares of independent standard normal variables) is most straightforward when their shape/degrees of freedom parameters (v, m, n) are positive integers. The density functions listed here represent the analytic continuations of these distributions to positive real-valued parameters, which are their standard definitions. The beta and arcsine distributions are included due to their relationship with normal random variables via the transformation of ratios or functions thereof.

Table 3. Some continuous distributions constructed by or closely related to N(μ, σ²).

Distribution	Notation	Density $\mathit{f}(\mathit{x})$
Normal	N(μ, σ2)	${\displaystyle \frac{1}{\sqrt{2\pi }\sigma }}\exp \left\{-{\displaystyle \frac{{(x-\mu )}^2}{2{\sigma }^2}}\right\},x,\mu \in \mathbb{R},\sigma >0$
Lognormal	LN(μ, σ2)	${\displaystyle \frac{1}{x\sigma \sqrt{2\pi }}}\exp \left(-{\displaystyle \frac{{(lnx-\mu )}^2}{2{\sigma }^2}}\right)$, $x>0,\mu \in \mathbb{R},\sigma >0$
Skew-normal	SN(μ, σ, α)	$f(x;\mu ,\sigma ,\alpha )={\displaystyle \frac{2}{\sigma }}\varphi \left({\displaystyle \frac{x-\mu }{\sigma }}\right)\mathrm{\Phi }\left(\alpha \left({\displaystyle \frac{x-\mu }{\sigma }}\right)\right),\sigma >0,x,\mu ,\alpha \in \mathbb{R}$
χ2(ν)	χ2(ν)	${\left[2^{\nu /2}\mathrm{\Gamma }({\displaystyle \frac{\nu }{2}})\right]}^{-1}{x}^{\nu /2-1}{e}^{-x/2},x>0,\nu >0$
χ(ν)	χν	${\left[2^{\nu /2-1}\mathrm{\Gamma }({\displaystyle \frac{\nu }{2}})\right]}^{-1}{x}^{\nu -1}{e}^{-x^2/2},x>0,\nu >0$
Beta	Be(a, b)	${\displaystyle \frac{1}{B(a,b)}}{x}^{a-1}{(1-x)}^{b-1},0\le x\le 1,a,b>0$
Beta prime	BP(α, β)	${\displaystyle \frac{1}{\mathrm{B}(\alpha ,\beta )}}{\displaystyle \frac{x^{\alpha -1}}{{(1+x)}^{\alpha +\beta }}},x>0,\alpha ,\beta >0$
Arcsine	arcsine	${\displaystyle \frac{1}{\pi \sqrt{x(1-x)}}},0<x<1$
F	F(m, n)	${\displaystyle \frac{1}{\mathrm{B}(\frac{m}{2},\frac{n}{2})}}{\left({\displaystyle \frac{m}{n}}\right)}^{{\displaystyle \frac{m}{2}}}{x}^{{\displaystyle \frac{m}{2}}-1}{\left(1+{\displaystyle \frac{m}{n}}x\right)}^{-{\displaystyle \frac{m+n}{2}}},x>0,m,n>0$
Student-t	tν	${\displaystyle \frac{\mathrm{\Gamma }(\frac{\nu +1}{2})}{\sqrt{\nu \pi }\mathrm{\Gamma }(\frac{\nu }{2})}}{\left(1+{\displaystyle \frac{x^2}{\nu }}\right)}^{-{\displaystyle \frac{\nu +1}{2}}},\nu >0,x\in \mathbb{R}$

Note: The title “constructed by” implies a direct derivation from the normal distribution. For distributions like χ², χ, F, and t, this direct construction (e.g., as a sum of squares of independent standard normal variables) is most straightforward when their shape/degrees of freedom parameters (v, m, n) are positive integers. The density functions listed here represent the analytic continuations of these distributions to positive real-valued parameters, which are their standard definitions. The beta and arcsine distributions are included due to their relationship with normal random variables via the transformation of ratios or functions thereof.

2.5.2. Distributions Related to the Exponential Distribution

The exponential distribution, defined on the positive real number space, was extensively studied in the early 19th century due to its valuable properties, such as memorylessness and its connection to the Poisson process for modeling rare events (see [1]). Several distributions derived from the exponential distribution are listed in Table 4.

(1)

Laplace distribution: Also known as the double exponential distribution, the Laplace distribution extends the exponential distribution symmetrically to the entire real number space, $\mathbb{R}$.

(2)

Weibull distribution: The Weibull distribution is formed by introducing location δ and sharp α parameters to the density of the exponential density. It constitutes a flexible family of distributions widely used for modeling time-to-failure, lifespan, and other positive-valued measurements.

(3)

Rayleigh distribution: Proposed by Rayleigh [31] for modeling wave amplitudes, the Rayleigh distribution is a special case of the Weibull distribution with Wei(2, β, 0). It also arises from the normal distribution, N(0, σ²), by $X=\sqrt{Y^2+Z^2}\sim Ray(\sigma )$, where Y and Z are i.i.d. N(0, σ²) and β = 2σ².

(4)

Topp–Leone exponential distribution (TLED): Introduced by [32], the TLED adds a shape parameter α to the exponential distribution, yielding the cumulative distribution function $F(x)={(1-e^{-2x/\beta })}^{\alpha }$. Its density is provided in Table 4. The TLED generalizes certain distributions. For instance, TLED includes the Burr type as TLED(2, β) and Frèchet distribution as TLED(1/2, β).

Table 4. Some continuous distributions constructed by the exponential distribution.

Distribution	Notation	Density $\mathit{p}(\mathit{x})$
Exponential	Exp(β)	${\displaystyle \frac{1}{\beta }}{e}^{-x/\beta },x\ge 0,\beta >0$
Laplace	Laplace(μ, β)	${\displaystyle \frac{1}{2\beta }}{e}^{-{\displaystyle \frac{|x-\mu |}{\beta }}},x\in \mathbb{R},\mu \in \mathbb{R},\beta >0$
Weibull	Wei(α, β, δ)	${\displaystyle \frac{\alpha }{\beta }}{\left({\displaystyle \frac{x-\delta }{\beta }}\right)}^{\alpha -1}{e}^{-{\left({\displaystyle \frac{x-\delta }{\beta }}\right)}^{\alpha }},x\ge \delta ,\delta \in \mathbb{R},\alpha ,\beta >0$
Rayleigh	Ray(β) or Ray(σ)	${\displaystyle \frac{2x}{\beta }}{e}^{-x^2/\beta },x\ge 0,\beta =2{\sigma }^2$
TLED	TLED(α, β)	${\displaystyle \frac{2\alpha }{\beta }}{e}^{-2x/\beta }{(1-e^{-2x/\beta })}^{\alpha -1},x>0,\alpha ,\beta >0$

Table 4. Some continuous distributions constructed by the exponential distribution.

Distribution	Notation	Density $\mathit{p}(\mathit{x})$
Exponential	Exp(β)	${\displaystyle \frac{1}{\beta }}{e}^{-x/\beta },x\ge 0,\beta >0$
Laplace	Laplace(μ, β)	${\displaystyle \frac{1}{2\beta }}{e}^{-{\displaystyle \frac{|x-\mu |}{\beta }}},x\in \mathbb{R},\mu \in \mathbb{R},\beta >0$
Weibull	Wei(α, β, δ)	${\displaystyle \frac{\alpha }{\beta }}{\left({\displaystyle \frac{x-\delta }{\beta }}\right)}^{\alpha -1}{e}^{-{\left({\displaystyle \frac{x-\delta }{\beta }}\right)}^{\alpha }},x\ge \delta ,\delta \in \mathbb{R},\alpha ,\beta >0$
Rayleigh	Ray(β) or Ray(σ)	${\displaystyle \frac{2x}{\beta }}{e}^{-x^2/\beta },x\ge 0,\beta =2{\sigma }^2$
TLED	TLED(α, β)	${\displaystyle \frac{2\alpha }{\beta }}{e}^{-2x/\beta }{(1-e^{-2x/\beta })}^{\alpha -1},x>0,\alpha ,\beta >0$

It is worth mentioning that the exponential distribution and the Poisson distribution share a fundamental symbiotic relationship within the context of a Poisson process. If the number of events occurring in a fixed time interval follows a Poisson distribution with rate parameter λ, then the waiting time between consecutive events follows an exponential distribution with the same rate parameter $\lambda ={\displaystyle \frac{1}{\beta }}$. Thus, the former models event counts, and the latter models inter-arrival times.

2.5.3. Distributions Related to the Gamma Distribution

The gamma distribution, Gamma(α, β), was formally defined in the 20th century and can be considered as an extension of the exponential distribution. It retains several desirable properties of the exponential distribution, such as additivity: the sum of independently and identically distributed variables X_i ∼ Gamma(α_i, β) for $i=1,\dots ,m$ is distributed as $Gamma({\sum }_{i=1}^m{\alpha }_i,\beta )$. The gamma distribution is foundational, as the chi-square distribution ${\chi }_{\nu }^2$ is a special case, Gamma(ν/2, 1/2), and the beta distribution Be(a, b) can be defined by ${\displaystyle \frac{Y}{Y+Z}}$, where Y ∼ Gamma(a, 1) and Z ∼ Gamma(b, 1) are independent. Table 5 lists several important extensions of the gamma distribution.

(1)

Location-shift gamma: Adding a location parameter δ forms the three-parameter distribution Gamma(α, β, δ).

(2)

Generalized gamma: This distribution exists in several forms. The version shown with shape a, scale d, and power p parameters was introduced by Stacy [33]. It includes several common distributions as special cases: the standard gamma (p = 1), the Weibull (a = 1), the Rayleigh (a = p = 2), and the exponential (a = p = 1).

(3)

Inverse gamma: Defined as Y = 1/X for X ∼ Gamma(α, β), this two-parameter distribution is positive-skewed. It is primarily used in Bayesian statistics as the conjugate prior for the variance of a normal distribution.

(4)

Beta prime distribution: The ratio X/Y of two independent variables, X ∼ Gamma(α, θ) and Y ∼ Gamma(β, θ), follows a beta prime distribution BP(α, β).

(5)

General beta prime distribution: This is a further extension of the beta prime distribution, incorporating two additional parameters.

Table 5. Some continuous distributions constructed by the gamma distribution.

Distribution	Notation	Density $\mathit{f}(\mathit{x})$
Gamma	Gamma(α, β)	${\displaystyle \frac{{\beta }^{\alpha }}{\mathrm{\Gamma }(\alpha )}}{x}^{\alpha -1}{e}^{-\beta x},x>0,\alpha ,\beta >0$
Location-shift gamma	Gamma(α, β, δ)	${\displaystyle \frac{{\beta }^{\alpha }}{\mathrm{\Gamma }(\alpha )}}{(x-\delta )}^{\alpha -1}{e}^{-\beta (x-\delta )}$, $x\ge \delta ,\delta \in \mathbb{R}$, α, β > 0
Generalized gamma	G − gamma(a, d, p)	${\displaystyle \frac{p}{d^a\mathrm{\Gamma }(a/p)}}{x}^{a-1}{e}^{-{(x/d)}^p},x>0,a,d,p>0$
Inverse gamma	Inv − Gamma(α, β)	${\displaystyle \frac{{\beta }^{\alpha }}{\mathrm{\Gamma }(\alpha )}}{x}^{-(\alpha +1)}{e}^{-\beta /x},x>0,\alpha ,\beta >0$
Beta prime	BP(α, β)	${\displaystyle \frac{1}{\mathrm{B}(\alpha ,\beta )}}{\displaystyle \frac{x^{\alpha -1}}{{(1+x)}^{\alpha +\beta }}},x>0,\alpha ,\beta >0$
General beta prime	GBP(α, β, p, q)	${\displaystyle \frac{\left|\alpha \right|{(1+{(x/\beta )}^{\alpha })}^{-p-q}{(x/\beta )}^{p\alpha -1}}{\beta B(p,q)}}$, $\alpha \in \mathbb{R},\alpha \ne 0$, p, q, β > 0, x > 0.

Table 5. Some continuous distributions constructed by the gamma distribution.

Distribution	Notation	Density $\mathit{f}(\mathit{x})$
Gamma	Gamma(α, β)	${\displaystyle \frac{{\beta }^{\alpha }}{\mathrm{\Gamma }(\alpha )}}{x}^{\alpha -1}{e}^{-\beta x},x>0,\alpha ,\beta >0$
Location-shift gamma	Gamma(α, β, δ)	${\displaystyle \frac{{\beta }^{\alpha }}{\mathrm{\Gamma }(\alpha )}}{(x-\delta )}^{\alpha -1}{e}^{-\beta (x-\delta )}$, $x\ge \delta ,\delta \in \mathbb{R}$, α, β > 0
Generalized gamma	G − gamma(a, d, p)	${\displaystyle \frac{p}{d^a\mathrm{\Gamma }(a/p)}}{x}^{a-1}{e}^{-{(x/d)}^p},x>0,a,d,p>0$
Inverse gamma	Inv − Gamma(α, β)	${\displaystyle \frac{{\beta }^{\alpha }}{\mathrm{\Gamma }(\alpha )}}{x}^{-(\alpha +1)}{e}^{-\beta /x},x>0,\alpha ,\beta >0$
Beta prime	BP(α, β)	${\displaystyle \frac{1}{\mathrm{B}(\alpha ,\beta )}}{\displaystyle \frac{x^{\alpha -1}}{{(1+x)}^{\alpha +\beta }}},x>0,\alpha ,\beta >0$
General beta prime	GBP(α, β, p, q)	${\displaystyle \frac{\left|\alpha \right|{(1+{(x/\beta )}^{\alpha })}^{-p-q}{(x/\beta )}^{p\alpha -1}}{\beta B(p,q)}}$, $\alpha \in \mathbb{R},\alpha \ne 0$, p, q, β > 0, x > 0.

2.6. The Generalized Hyperbolic Distribution Family

Schrdinger [34] described the probability distribution of the first passage time in Brownian motion. Thirty years later, Tweedie [35] gave the inverse relationship between the cumulant generating function of the first passage time distribution and that of the normal distribution, and named it the inverse Gaussian (IG) distribution. This new distribution was proposed in the top journal Nature, which shows its importance. The IG distribution, denoted as IG(μ, λ), is defined for x > 0, μ > 0, and λ > 0 by the probability density function:

$$\begin{array}{c}\hfill f(x;\mu ,\lambda )=\sqrt{{\displaystyle \frac{\lambda }{2\pi x^3}}}\exp \left\{{\displaystyle \frac{-\lambda {(x-\mu )}^2}{2{\mu }^{2}x}}\right\}.\end{array}$$

(8)

The IG distribution models the first passage time of a Brownian motion, making it highly useful for analyzing lifetime data and the frequency of event occurrences in various fields. For a comprehensive reference, see [36]. The IG distribution has many extensions, among which the generalized hyperbolic distribution (GH) is a more general one.

The GH distribution is a continuous probability distribution that represents a highly flexible family of models for describing data with skewness and heavy tails. Its key strength lies in its ability to nest several other important distributions as special or limiting cases, providing a unified framework for statistical modeling. It was introduced by [37] to model the grain size distributions of wind-blown sand. However, its remarkable fit to financial data catapulted it to prominence in financial econometrics and risk management (see [38]).

A random variable X is said to follow a GH distribution GH(λ, α, β, δ, μ) if its pdf is

$$\begin{array}{c}\hfill f(x)=C(\lambda ,\alpha ,\beta ,\delta ){({\delta }^2+{(x-\mu )}^2)}^{(\lambda -0.5)/2}{K}_{\lambda -0.5}\left(\alpha \sqrt{{\delta }^2+{(x-\mu )}^2}\right)\exp (\beta (x-\mu )),\end{array}$$

(9)

where $x\in \mathbb{R}$ and

$$\begin{array}{c}\hfill C(\lambda ,\alpha ,\beta ,\delta )={\displaystyle \frac{{({\alpha }^2-{\beta }^2)}^{\lambda /2}}{\sqrt{2\pi }{\alpha }^{\lambda -0.5}{\delta }^{\lambda }{K}_{\lambda }(\delta \sqrt{{\alpha }^2-{\beta }^2})}},\alpha >0,\left|\beta \right|<\alpha ,\delta >0,\mu ,\lambda ,\in \mathbb{R}.\end{array}$$

The parameter λ influences the tail behavior and the subfamily classification; α controls the heaviness of the tails and the steepness; β controls the asymmetry; δ is the scale parameter related to the dispersion; and μ is the location parameter. K_λ(·) is the modified Bessel function of the second kind of order λ defined by

$$\begin{array}{c}\hfill K_{\lambda }(x)={\displaystyle \frac{1}{2}}{\int }_0^{\infty }{t}^{\lambda -1}\exp \left(-{\displaystyle \frac{x}{2}}(t+{\displaystyle \frac{1}{t}})\right)dt.\end{array}$$

The GH distribution involves many distributions as its special cases: hyperbolic distribution (λ = 1), normal inverse Gaussian distribution (λ = −0.5), variance gamma distribution (λ > 0, δ = 0), etc. A normal variance-mean mixture distribution, termed the normal inverse Gaussian distribution proposed by [39], is used to construct stochastic processes that appear of interest for statistical modeling purposes, particularly in turbulence and finance. Its pdf is defined as

$$\begin{array}{c}\hfill f(x;\alpha ,\beta ,\delta ,\mu )=C(\alpha ,\beta ,\delta )q{\left({\displaystyle \frac{x-\mu }{\delta }}\right)}^{-1}{K}_1\left(\alpha \delta q\left({\displaystyle \frac{x-\mu }{\delta }}\right)\right)\exp (\beta (x-\mu )),x\in \mathbb{R},\end{array}$$

where

$$\begin{array}{c}\hfill C(\alpha ,\beta ,\delta )={\displaystyle \frac{\alpha }{\pi }}\exp \left(\delta \sqrt{{\alpha }^2-{\beta }^2}\right),q(t)=\sqrt{1+t^2},\end{array}$$

and K₁ is the modified Bessel function of the second kind of order 1. The parameters have the following role: $\mu \in \mathbb{R}$ (location) and δ > 0 (scale); α > 0 (tail heaviness) controls the steepness of the distribution. A larger α leads to lighter tails and α > |β|, where β (asymmetry/skewness) controls the skewness. β = 0 gives a symmetric distribution, β > 0 implies positive skewness, and β < 0 implies negative skewness.

The GH distribution is a normal variance-mean mixture of the generalized inverse Gaussian (GIG) distributions. This construction is the source of its flexibility and mathematical tractability. The GIG distribution is a continuous probability distribution that generalizes several important distributions and plays a critical role as a mixing distribution in normal variance-mean mixtures. The GIG distribution denoted by GIG(λ, χ, ψ) is defined by the given pdf

$$\begin{array}{c}\hfill f(x;\lambda ,\chi ,\psi )={\displaystyle \frac{{(\psi /\chi )}^{\lambda /2}}{2K_{\lambda }(\sqrt{\psi \chi })}}{x}^{\lambda -1}\exp \left(-{\displaystyle \frac{1}{2}}\left({\displaystyle \frac{\chi }{x}}+\psi x\right)\right),x>0,\chi ,\psi \ge 0,\end{array}$$

(10)

where parameter λ determines the shape of the distribution and is deeply connected to the index of the Bessel function. χ is the scale parameter often related to the “left-hand” behavior, with the constraint: λ > 0 whenever χ = 0. ψ is another scale parameter that is often related to the “right-hand” tail with the constraint: λ < 0 whenever ψ = 0, and ψ = χ = 0 is not allowed. The GIG involves inverse Gaussian distribution (λ = −0.5), reciprocal inverse Gaussian distribution (λ = 0.5), and gamma distribution (χ = 0, λ > 0).

A random variable Y follows a GH distribution GH(λ, α, β, δ, μ) if it has the following stochastic representation:

$$\begin{array}{c}\hfill Y=\mu +\beta X+\sigma \sqrt{X}Z,\end{array}$$

where Z ∼ N(0, 1), and X ∼ GIG(λ, χ, ψ) is independent of Z. The parameters of the resulting GH(λ, α, β, δ, μ) distribution are identified as follows: $\alpha =\sqrt{\psi {\sigma }^2+{\beta }^2},\delta =\sqrt{\chi }$. By choosing different GIG subfamilies as the mixer, we obtain the different GH subfamilies:

(1)

Mixing with the inverse Gaussian (λ = −0.5) distribution results in the normal inverse Gaussian distribution.

(2)

Mixing with the gamma (χ = 0, λ > 0) distribution results in the variance gamma distribution.

(3)

Mixing with the GIG (λ = 1) distribution results in the hyperbolic distribution.

2.7. Beta-Generated Distributions

A significant research area is the creation of flexible univariate distribution classes that extend classical models. The beta-generated (BG) distributions by Arnold et al. [40,41] achieve this by merging the beta distribution with a chosen baseline distribution.

Definition 1.

For a baseline cdf F(x) with pdf f(x) on a domain D, the beta-generated (BG) distributions have the following pdf:

$$\begin{array}{c}\hfill g_F(x;a,b)={\displaystyle \frac{1}{B(a,b)}}F{(x)}^{a-1}{[1-F(x)]}^{b-1}f(x),x\in D,a>0,b>0.\end{array}$$

(11)

We denote this by X ∼ BG(a, b; F). The corresponding cdf is derived via the substitution y = F(t):

$$\begin{array}{c}\hfill G_F(x;a,b)={\displaystyle \frac{1}{B(a,b)}}{\int }_0^{F(x)}{y}^{a-1}{(1-y)}^{b-1}dy=I_{F(x)}(a,b),\end{array}$$

which is the incomplete beta function ratio.

A key property of the BG(a, b; F) distribution is its stochastic representation:

$$\begin{array}{c}\hfill X\stackrel{d}{=}{F}^{-1}(B),\mathrm{for}B\sim Be(a,b).\end{array}$$

(12)

This representation provides a foundation for applying the generalized inverse transformation method in statistical simulation using representative points. It offers a direct method for simulating from the density (11) and for constructing multivariate extensions. The moments of X are consequently given by

$$\begin{array}{c}\hfill E(X^r)=E({\left[F^{-1}(B)\right]}^r),r>0.\end{array}$$

(13)

The beta-generated family encompasses a wide range of distributions depending on the choice of the baseline F. To demonstrate the flexibility and utility of this class, we select two specific examples. For the generalization of this generating mechanism, see [42].

2.7.1. The Beta-Normal Distribution

If F(x) is the cdf of N(μ, σ²), the corresponding BG distribution is the beta-normal distribution, denoted as X ∼ BN(a, b; μ, σ). As studied by Eugene et al. [43], its cdf is

$$\begin{array}{c}\hfill G(x)={\displaystyle \frac{1}{B(a,b)}}{\int }_0^{\mathrm{\Phi }({\displaystyle \frac{x-\mu }{\sigma }})}{t}^{a-1}{(1-t)}^{b-1}dt,\end{array}$$

(14)

where Φ(·) is the standard normal cdf. For the standard case BN(a, b; 0, 1), the pdf is

$$\begin{array}{c}\hfill g(x)={\displaystyle \frac{1}{B(a,b)}}\mathrm{\Phi }{(x)}^{a-1}{[1-\mathrm{\Phi }(x)]}^{b-1}\varphi (x),x\in \mathbb{R}.\end{array}$$

(15)

Its moments are given by the formula in (13) (Gupta and Nadarajah [44]).

This four-parameter distribution defines location, scale, and shape, and can be unimodal or bimodal. Its fitting performance has been compared to that of normal mixture models in simulation studies. Note that the beta distribution is a special case of the Dirichlet distribution (cf. Section 4.4.2).

2.7.2. The Beta-Weibull Distribution

The beta-Weibull distribution, denoted by BW(a, b; c, γ), has the following pdf:

$$\begin{array}{c}\hfill f_{BW}(x)={\displaystyle \frac{1}{B(a,b)}}{\displaystyle \frac{c}{\gamma }}{\left({\displaystyle \frac{x}{\gamma }}\right)}^{c-1}{\left[1-\exp \left\{-{\left({\displaystyle \frac{x}{\gamma }}\right)}^c\right\}\right]}^{a-1}\exp \left\{-b{\left({\displaystyle \frac{x}{\gamma }}\right)}^c\right\},x>0.\end{array}$$

(16)

A system study on this distribution refer to Famege et al. [45] and Mdziniso [46]. The beta-Weibull distribution includes many useful distributions, including the Weibull distribution (a = b = 1), beta-exponential distribution (c = 1), Rayleigh distribution (a = b = 1, c = 2), two-parameter Burr-type distribution (b = 1, c = 2), etc.

The beta-Weibull distribution has many good properties.

(1)

The limit of beta-Weibull density (16) is

$$\begin{array}{c}\hfill \underset{x\to \infty }{lim}{f}_{BW}(x)=\left\{\begin{array}{cc}\infty ,\hfill & ac<1,\hfill \\ {\displaystyle \frac{c\mathrm{\Gamma }(a+b)}{\gamma \mathrm{\Gamma }(a)\mathrm{\Gamma }(b)}},\hfill & ac=1,\hfill \\ 0,\hfill & ac>1.\hfill \end{array}\right.\end{array}$$

(2)

Let Y ∼ Be(a, b), then the random variable $X=\gamma {\left[-log(1-Y)\right]}^{1/c}$ follows BW(a, b, c, γ), where c, γ > 0. It provides a way to generate a random sample from the BW distribution.

(3)

Let Y follow a beta-exponential distribution with parameters (a, b, θ), then random variable X = θ(Y/θ)^1/c follows a beta-Weibull distribution.

(4)

The beta-Weibull distribution is unimodal. The mode is at x₀ = 1 whenever ac < 1 or ac = 1 and b ≥ (c − 1)/2c. For other cases, x₀ is the solution of the equation

$$\begin{array}{c}\hfill bc-(c-1){(x_0/\gamma )}^{-c}=c(a-1){\left[\exp {(x_0/\gamma )}^c\right]}^{-1}.\end{array}$$

(5)

The rth moments of BW(a, b, c, γ) is given by

$$\begin{array}{c}\hfill E\left[{\left({\displaystyle \frac{X}{\gamma }}\right)}^r\right]={\displaystyle \frac{1}{B(a,b)}}{\int }_0^{\infty }{t}^{r/c}{\left(1-e^{-t}\right)}^{a-1}{e}^{-bt}dt,\end{array}$$

by the transformation t = (x/γ)^c.

3. Construction of Univariate Discrete Distributions and an Approximation

A discrete distribution is characterized by its support points and their probabilities:

$$\begin{array}{c}\hfill \begin{array}{c}\begin{array}{ccccc}X& x_1& x_2& \cdots & x_k\end{array}\\ \begin{array}{ccccc}{p}_i& p_1& p_2& \cdots & p_k\end{array}\end{array}\end{array}$$

(17)

where the random variable X has support $x_1,\dots ,x_k$ and $P(X=x_i)=p_i,i=1,\dots ,k.$

3.1. Classical Univariate Discrete Distributions

Standard discrete distribution model outcomes from fundamental random processes/trials:

• Bernoulli(p)/Binomial(n, p): The Bernoulli distribution, Bernoulli(p), models a single trial, and Binomial(n, p) counts successes in n independent trials with sample success probability p.
• Geometric(p): It models the number of Bernoulli trials needed to obtain the first success, representing the waiting time for a success. It is a special case of the negative binomial (with r = 1).
• Negative Binomial(r, p): As a generalization of the geometric distribution, it models the number of Bernoulli trials needed to achieve r successes. The negative binomial distribution presented here in its classic form (modeling the number of trials) is equivalent to the parameterization modeling the number of failures before the r-th success (with support $y=0,1,2,\dots$), which is prevalent in many modern software packages and generalized linear models.
• Poisson(λ): It models the number of events occurring in a fixed interval of time or space, given a constant average rate (λ) and independence between events. It can be derived as a limiting case of the binomial distribution when n → ∞ and p → 0 such that np → λ.
• Hypergeometric(s, n, N, M): It represents the number s of successes in n draws without replacement from a population with N size with M success items, where max(0, n − (N − M)) ≤ s ≤ min(M, n). This differs from the binomial distribution, where trials are independent (i.e., with replacement).

See Johnson and Kotz [47] for an extensive catalog. A guide to selecting common discrete probability distributions is illustrated in Figure 1, with probability mass functions summarized in

Table 6. Famous discrete distributions.

Distribution	Support Points	Probability Mass Function ${\mathit{p}}_{\mathit{i}}(\mathit{x})$
Bernoulli(p)	0, 1	p0 = 1 − p, p1 = p
Binomial(n, p)	$0,1,2,\dots ,n$	$p_i=\left({\displaystyle \genfrac{}{}{0pt}{}{n}{i}}\right)p^i{(1-p)}^{n-i},i=0,1,2,\dots ,n$
Geometric(p)	$1,2,\dots$	$p_i=p{(1-p)}^{i-1},i=1,2,\dots$ (Number of trials until first success)
Negative Binomial(r, p)	$r,r+1,\dots$	$p_i=\left({\displaystyle \genfrac{}{}{0pt}{}{i-1}{r-1}}\right)p^r{(1-p)}^{i-r},i=r,r+1,\dots$ (Number of trials until r-th success)
	$0,1,2,\dots$	$p_i=\left({\displaystyle \genfrac{}{}{0pt}{}{i+r-1}{i}}\right)p^r{(1-p)}^i,i=0,1,2,\dots$ (Number of failures before r-th success)
Poisson(λ)	$0,1,2\dots$	$p_i=e^{-\lambda }{\displaystyle \frac{{\lambda }^i}{i!}},i=0,1,2,\dots$
Hypergeometric(s, n, N, M)	max (0, n − (N − M)), ⋯, min (M, n)	$p_i=\left({\displaystyle \genfrac{}{}{0pt}{}{M}{i}}\right)\left({\displaystyle \genfrac{}{}{0pt}{}{N-M}{n-i}}\right)/\left({\displaystyle \genfrac{}{}{0pt}{}{N}{n}}\right)$

Note on alternative derivations and parameterizations: The distributions listed above can often be derived through multiple probabilistic mechanisms, which leads to different (but equivalent) parameterizations and interpretations. These alternative derivations highlight the interconnectedness of discrete probability distributions and provide flexibility in choosing the most appropriate model for a given data-generating process.

.

Consider a random variable X following the discrete distribution defined in (17) and a continuous function g(·). The transformed variable Y = g(X) follows a distribution with the following structure:

$$\begin{array}{c}\begin{array}{ccccc}Y& g(x_1)& g(x_2)& \cdots & g(x_k)\end{array}\\ \begin{array}{ccccc}{p}_i& p_1& p_2& \cdots & p_k\end{array}\end{array}.$$

(18)

In other words, the law of Y is determined by mapping the support points of X via g while preserving the associated probabilities. This elegant property facilitates the analysis of transformed discrete variables. Li et al. [14] utilized this property to compute nearly representative points for the distribution of g(X).

3.2. Distributions Generated by Mixtures

A powerful method for constructing flexible probability models is through mixture distributions, where the parameter of one distribution is itself governed by another distribution. This approach naturally incorporates heterogeneity and over-dispersion, commonly observed in real-world data.

3.2.1. Poisson Mixtures

When the rate parameter λ of a Poisson distribution is not fixed but follows a continuous distribution G(λ) with support on ${\mathbb{R}}^{+}$, the resulting marginal distribution is a Poisson mixture. Its probability mass function is given by

$$P(X=k)={\int }_0^{\infty }{\displaystyle \frac{e^{-\lambda }{\lambda }^k}{k!}}dG(\lambda ),k=0,1,2,\dots$$

The most prominent example is the negative binomial distribution, which arises as a Poisson-gamma mixture. Specifically, if

$$X\mid \lambda \sim \mathrm{Poisson}(\lambda )\mathrm{and}\lambda \sim \mathrm{Gamma}(r,\beta =(1-p)/p),$$

then the marginal distribution of X is negative binomial with pmf:

$$P(X=k)={\displaystyle \frac{\mathrm{\Gamma }(r+k)}{k!\mathrm{\Gamma }(r)}}{p}^r{(1-p)}^k,k=0,1,2,\dots$$

This model is widely used for count data where the variance exceeds the mean.

3.2.2. Binomial Mixtures

Similarly, allowing the success probability p of a binomial distribution to vary according to a continuous distribution F(p) on [0,1] leads to a binomial mixture. The marginal pmf is

$$P(X=k)={\int }_0^1\left({\displaystyle \genfrac{}{}{0pt}{}{n}{k}}\right)p^k{(1-p)}^{n-k}dF(p),k=0,1,\dots ,n$$

The beta-binomial distribution is the canonical example, where p ∼ Beta(α, β). Its pmf is

$$P(X=k)=\left({\displaystyle \genfrac{}{}{0pt}{}{n}{k}}\right){\displaystyle \frac{\mathrm{B}(k+\alpha ,n-k+\beta )}{\mathrm{B}(\alpha ,\beta )}}$$

where B(·, ·) is the beta function. This distribution effectively models counts with extra-binomial variation.

Other notable mixtures include the Poisson-inverse Gaussian and the negative binomial-beta distributions [48].

3.3. Distributions Generated by Random Sums

Distributions generated by random sums, also known as compound distributions, provide a framework for modeling the aggregate outcome of a random number of events. Let N be a non-negative integer-valued random variable with pmf p_N(n), and let {X_i} be a sequence of i.i.d. random variables, independent of N. The random sum is defined as

$$S=X_1+X_2+\cdots +X_N,(\mathrm{with}\mathrm{the}\mathrm{convention}\mathrm{that}S=0\mathrm{if}N=0).$$

The distribution of S is called the compound distribution of N and X_i. Its probability generating function (pgf) is given by G_S(t) = G_N(G_X(t)), from which moments and probabilities can be derived.

Prominent examples include the following:

• Compound Poisson distribution: If N ∼ Poisson(λ), then S is compound Poisson. This is a highly flexible family used in insurance (to model total claim amounts) and queueing theory. Special cases include the Poisson-exponential and Poisson-gamma (which is equivalent to the negative binomial under a specific parameterization) distributions.
• Geometric sum: If N ∼ Geometric(p), and X_i ∼ Exp(θ), then S ∼ Exp(θp). This result is foundational in renewal theory.

The family of phase-type distributions can also be constructed using random sums of exponential random variables, demonstrating the versatility of this method for generating distributions with specific tail behaviors [49].

3.4. Approximation to Univariate Continuous Distributions by Representative Points

Consider a random variable X ∼ F(x), where F(x) is a univariate cumulative distribution function. The empirical distribution function F_k(x) is defined as

$$\begin{array}{c}\hfill F_k(x)={\displaystyle \frac{1}{k}}\sum _{i=1}^{k}I\left\{x_i\le x\right\},\end{array}$$

(19)

where I_A is the indicator function of A. The empirical distribution F_k(x) is a discrete distribution with support points $x_1,\dots ,x_k$, each assigned an equal probability mass of 1/k. It serves as a consistent approximation of F(x), meaning F_k(x) → F(x) in distribution as k → ∞. This distribution can be represented as

$$\begin{array}{c}\begin{array}{ccccc}{Y}_{MC}& x_1& x_2& \dots & x_k\end{array}\\ \begin{array}{ccccc}{p}_i& {\displaystyle \frac{1}{k}}& {\displaystyle \frac{1}{k}}& \dots & {\displaystyle \frac{1}{k}}\end{array}\end{array}.$$

(20)

Alternatively, one can construct other approximations using the so-called representative points (RPs) as support points. For a given distribution F(x) in $\mathbb{R}$, several types of RPs exist, including

• Monte Carlo RPs (MC-RPs): the empirical sample points.
• Quasi-Monte Carlo RPs (QMC-RPs): deterministic points based on low-discrepancy sequences.
• Mean square error RPs (MSE-RPs): points minimizing a mean squared error criterion.

For a univariate continuous distribution F(x), the QMC-RP approximation is given by the following distribution:

$$\begin{array}{c}\begin{array}{ccccc}{Y}_{QMC}& F^{-1}({\displaystyle \frac{1}{2k}})& F^{-1}({\displaystyle \frac{3}{2k}})& \dots & F^{-1}({\displaystyle \frac{2k-1}{2k}})\end{array}\\ \begin{array}{ccccc}{p}_i& {\displaystyle \frac{1}{k}}& {\displaystyle \frac{1}{k}}& \dots & {\displaystyle \frac{1}{k}}\end{array}\end{array}.$$

(21)

MSE-RPs and related approximation distribution is

$$\begin{array}{c}\begin{array}{ccccc}{Y}_{MSE}& {\xi }_1& {\xi }_2& \dots & {\xi }_k\end{array}\\ \begin{array}{ccccc}{p}_i& p_1& p_2& \dots & p_k\end{array}\end{array}.$$

(22)

where the support points $\{{\xi }_1,\dots ,{\xi }_k\}$ are determined by

$$\begin{array}{c}\hfill \mathrm{MSE}(\mathcal{Y})={\int }_{-\infty }^{+\infty }\underset{i}{min}{(x-{\xi }_i^{(k)})}^{2}p(x)dx=\sum _{j=1}^k{\int }_{I_j}{(x-{\xi }_i^{(k)})}^{2}p(x)dx,\end{array}$$

(23)

where

$$\begin{array}{cc}\hfill & I_1=(a_1,a_2),I_2=(a_2,a_3),\dots ,I_k=(a_k,a_{k+1}),\\ \hfill & a_1=-\infty ,a_i=\left({\xi }_{i-1}^{(k)}+{\xi }_i^{(k)}\right)/2,i=2,\cdots ,k,a_{k+1}=\infty .\end{array}$$

(24)

Fang and Pan [50] and Fang et al. [8] provided a comprehensive review of the theory, generating algorithms, and applications of representative points in statistical inference.

3.4.1. Difference Between Two Distributions

The Kullback–Leibler (KL) divergence, or relative entropy, of two probability distributions with probability density functions p(x) and q(x) on $\mathcal{X}\subset {\mathbb{R}}^p$ is defined as the expected logarithmic ratio of densities concerning p(x):

$$\begin{array}{c}\hfill D_{KL}(p(x)|q(x))={\int }_{\mathcal{X}}log\left({\displaystyle \frac{p(x)}{q(x)}}\right)p(x)dx=E_{p(x)}\left[log\left({\displaystyle \frac{p(x)}{q(x)}}\right)\right].\end{array}$$

(25)

The KL divergence is non-negative and is equal to 0 if and only if the two distributions are identical.

Now, consider the univariate case. For any discrete distribution, we can find a corresponding density estimator, denoted by $\widehat{p}(x)$. Let F(x) be a cdf with pmf p(x), and let $\widehat{F}(x)$ with pmf $\widehat{p}(x)$ be an approximating distribution. Several criteria in the literature measure the distance between F and $\widehat{F}$:

$$\begin{array}{ccc}\hfill \mathrm{KL} \mathrm{divergence}(KL):\mathrm{KL}(p\Vert \widehat{p})& =& \sum _{x}log\left({\displaystyle \frac{p(x)}{\widehat{p}(x)}}\right)p(x);\hfill \\ \hfill L^2\mathrm{-norm} \mathrm{of} \mathrm{pmfs}(L^2\mathrm{pdf}):\parallel p,\widehat{p}\parallel & =& {\left(\sum _x{\left[p(x)-\widehat{p}(x)\right]}^2\right)}^{1/2};\hfill \\ \hfill L^2\mathrm{-norm} \mathrm{of} \mathrm{cdfs}(L^2\mathrm{cdf}):\parallel F,\widehat{F}\parallel & =& {\left({\int }_{\mathbb{R}}{\left[F(x)-\widehat{F}(x)\right]}^{2}dx\right)}^{1/2}.\hfill \end{array}$$

Another metric for the distance between F(x) and $\widehat{F}(x)$ is the absolute bias index (ABI), which evaluates the overall bias in parameter estimation. For a population distribution F(x; θ) with parameter vector $\mathit{\theta }=({\theta }_1,\dots ,{\theta }_m)$ and its estimate $\widehat{\mathit{\theta }}=({\widehat{\theta }}_1,\dots ,{\widehat{\theta }}_m)$ under $\widehat{F}(x)$, the ABI is defined as follows:

$$\begin{array}{c}\hfill ABI={\displaystyle \frac{1}{m}}\sum _{i=1}^m\left|{\displaystyle \frac{{\theta }_i-{\widehat{\theta }}_i}{{\theta }_i}}\right|.\end{array}$$

(26)

3.4.2. Discrete Analogues

Discrete analogs aim to create new discrete distributions that preserve one or more defining characteristics of an original continuous distribution. An approximation $\widehat{F}(x)$ is considered successful if it maintains as much information from F(x) as possible, whether measured by the distance between the distributions, the similarity of their moments, or the properties of simulated samples.

As a continuous random variable X can be characterized by its pdf, moments, generating functions, or other features, the construction of a discrete analog Y (with pmf) from X (with density f(x)) focuses on matching these properties. The primary methods for this construction, summarized by [51], are classified as follows.

(1)

Discrete analogs of the Pearson differential equation. This approach constructs discrete probability distributions by employing difference equations that mirror the structure of the Pearson differential equation, which defines the continuous Pearson family [52,53].

(2)

The probability mass function (pmf) of Y retains the form of the pdf of X, and the support of Y is determined from the full range of X. The pmf is defined as

$$\begin{array}{c}\hfill P(Y=k)={\displaystyle \frac{f(k)}{{\sum }_{j=-\infty }^{\infty }f(j)}},k=0,\pm 1,\pm 2,\dots .\end{array}$$

This approach has been used to create discrete analogs of various continuous distributions, including the gamma, general Dirichlet, normal, lognormal, exponential, Laplace, generalized exponential, and gamma distributions [54,55,56].

(3)

The survival function (SF) of Y retains the form of the survival function of X, and the support of Y is determined from the full range of X. The discrete survival function is defined as S_Y(k) = P(Y ≥ k), and its corresponding cdf is F_Y(k) = P(Y ≤ k), related by S_Y(k) = 1 − F_Y(k − 1). This technique has generated numerous discrete distributions, such as the discrete exponential, Weibull, geometric Weibull, normal, Rayleigh, Maxwell, extended exponential, and Lindley distributions [57,58].

(4)

The hazard (failure) rate function of Y retains the form of the hazard rate function of X. This method preserves the hazard rate function. For a continuous random variable X with survival function S_X(x) = P(X ≥ x) and hazard rate function λ_X(x) = f_X(x)/S_X(x), the survival function of the discrete analog Y is given by

$$\begin{array}{c}\hfill P(Y\ge k)=\prod _{i=1}^{k-1}\left(1-{\lambda }_X(i)\right),k=1,\dots ,m,\end{array}$$

and its pmf is

$$\begin{array}{c}\hfill P(Y=k)={\lambda }_X(k)\prod _{i=1}^{k-1}\left(1-{\lambda }_X(i)\right).\end{array}$$

This approach has been used to propose new discrete distributions, such as the discrete Lomax, Weibull, and Rayleigh [59].

(5)

The moments of Y and X coincide up to a certain order. This method requires that Y and X share the same finite r^th moment for $r=0,1,\dots ,2N-1$, and that their cdfs coincide at least at M + 1 points. The support of the discrete random variable Y is $\{y_1,\dots ,y_{MN}\}$. It has been used to define new discrete uniform, normal, gamma, and beta distributions [60].

There are more other construction methods, for instance, an extensive review on discrete distributions defined for integer data is provided in [61].

4. Multivariate Distributions

The commonly used discrete distributions can be extended to multivariate cases, including multinomial and multivariate negative binomial, hypergeometric, and Poisson distributions.

4.1. Multinomial Distribution

An extension of the binomial distribution is the so-called multinomial distribution. It is the fundamental model for counting events that fall into multiple categories, making it essential in fields like statistics, genetics, machine learning (e.g., topic modeling with naive Bayes), and survey analysis. Considering an experiment consists of n identical, independent trials, each trial results in one of k mutually exclusive and exhaustive categories with probabilities $p_i,i=1,\dots ,k;p_1+\dots +p_k=1$. Let random vector $X=(X_1,\dots ,X_k)$ follow a multinomial distribution $Multinomial(n;p_1,\dots ,p_k)$. Its probability mass function is

$$\begin{array}{c}\hfill P(X_1=n_1,\dots ,X_k=n_k)={\displaystyle \frac{n!}{n_1!\cdots n_k!}},n_1,\dots ,n_k\in {\mathbb{N}}_0,n_1+\dots +n_k=n.\end{array}$$

(27)

The multinomial distribution has many nice properties. One is related to Poisson distribution, that lets $X_1,\dots ,X_k$ be independent and $X_i\sim Poisson({\lambda }_i),i=1,\dots ,k$. Then the conditional distribution of $(X_1,\dots ,X_k|X_1+\cdots +X_k=n)$ follows $Multinomial(n;p_1,\dots ,p_k)$ where $p_i={\lambda }_i/({\lambda }_1+\cdots +{\lambda }_k),i=1,\dots ,k$. The multinomial distribution has been used in many statistical applications. One is in the Pearson’s chi-square test for goodness-of-fit, using the fact that the test statistic, calculated from observed multinomial counts and expected counts (np_i), follows a chi-square distribution asymptotically (as n grows large).

4.2. Multivariate Hypergeometric Distribution

The multivariate hypergeometric distribution is the generalization of the hypergeometric distribution to more than two categories. It describes the probability of drawing specific numbers of items from each category without replacement. Considering a population with size N, of which $M_1,\dots ,M_k$ are the numbers of items in the k mutually exclusive and exhaustive categories with ${\sum }_{i=1}^k{M}_i=N$, suppose that the n random samples are sampling without replacement containing $n_1,\dots ,n_k$ items of each corresponding category. The joint distribution of $X=(X_1,\dots ,X_k)$ has pmf

$$\begin{array}{c}\hfill P(X_1=n_1,\cdots ,n_k)={\displaystyle \frac{{\prod }_{i=1}^k\left(\genfrac{}{}{0pt}{}{M_i}{n_i}\right)}{\left(\genfrac{}{}{0pt}{}{N}{n}\right)}},\sum _{i=1}^k{n}_i=n,0\le n_i\le M_i,i=1,\cdots ,k.\end{array}$$

(28)

The distribution denoted as $Mult.Hypg.(n;M_1,\dots ,M_k)$ is the so-called multivariate hypergeometric distribution. Without requiring the parameters in (28) to be non-negative, multivariate inverse hypergeometric, negative hypergeometric, and negative inverse hypergeometric distributions are then defined; see [62]. One property is that if X and Y are independent and each distributed as $Multinomial(n;p_1,\dots ,p_k)$, then X|(X + Y = N) follows Mult. Hypg. (n, N).

4.3. Multivariate Poisson Distribution

If the parameters $p_1,\cdots ,p_{k-1}$ in the multinomial distribution are allowed to tend to 0, this implies that p_k tends 1 as n → ∞, such that $n{p}_i\to {\lambda }_i,i=1,2,\cdots ,k-1$. Hence, the probability of obtaining the event ${\cap }_{i=1}^{k-1}(X_i=n_i)$ tends to

$$\begin{array}{c}\hfill P(X_1=n_1,\cdots ,X_{k-1}=n_{k-1})=\prod _{i=1}^{k-1}\left\{e^{-{\lambda }_i}{\displaystyle \frac{{\lambda }_i^{n_i}}{n_i!}}\right\}.\end{array}$$

(29)

This is the joint distribution of k − 1 mutually independent Poisson random variables with means ${\lambda }_1,\cdots ,{\lambda }_{k-1}$. The distribution can also be obtained as a limiting form of negative multinomial distributions.

The simplest case, as derived from the multinomial limit, is one of independence, but more flexible distributions allowing for correlation are of greater practical interest. A more general and widely used approach to constructing a multivariate Poisson distribution with a covariance structure is to use a common mixture or latent variable model. A standard construction introduces a set of independent latent Poisson variables and defines the observed variables as their sums.

A classical bivariate Poisson distribution for random variables (X, Y) can be defined as follows:

$$X=Z_1+Z_0,Y=Z_2+Z_0,$$

where Z₀, Z₁, Z₂ are independent Poisson random variables with rates λ₀, λ₁, λ₂ ≥ 0, respectively. The joint probability mass function of (X, Y) is given by

$$P(X=x,Y=y)=\exp (-({\lambda }_1+{\lambda }_2+{\lambda }_0))\sum _{k=0}^{min(x,y)}{\displaystyle \frac{{\lambda }_1^{x-k}}{(x-k)!}}{\displaystyle \frac{{\lambda }_2^{y-k}}{(y-k)!}}{\displaystyle \frac{{\lambda }_0^k}{k!}}.$$

The marginal distributions are X ∼ Poisson(λ₁ + λ₀) and Y ∼ Poisson(λ₂ + λ₀), and the covariance between X and Y is Cov(X, Y) = λ₀ ≥ 0. This model can only account for non-negative correlation, which is often sufficient for many applications, such as modeling the number of insurance claims across different policy types or goals scored by two teams in a match.

This construction can be generalized to higher dimensions by introducing more common components, leading to a richer covariance structure [3,63,64]. These correlated multivariate Poisson models are essential tools in modern multivariate count data analysis.

4.4. Continuous Multivariate Distributions

Johnson et al. [4] provided a comprehensive compilation of many multivariate distributions. Johnson [7] offered an in-depth introduction to the generation of multivariate statistical simulations. Palmitesta and Provasi [65] reviewed a range of multivariate distributions and their computer generation methods. Their survey encompassed the Dirichlet distribution, the multivariate Johnson distribution system, multivariate beta and gamma distributions, the multivariate extreme value distribution, the inverse Gaussian multivariate distribution, the Cook–Johnson family of multivariate uniform distributions, and elliptically contoured distributions. They also examined techniques for generating random vectors with specified marginal distributions and correlation matrices. Fang, Kotz, and Ng [6] conducted a comprehensive study on symmetric multivariate distributions. Due to space limitations, this section focuses on selected multivariate distributions from the literature and meta-distributions constructed via copulas, see [66]. For more construction methods for multivariate distribution, see [67].

4.4.1. Stochastic Representation

The traditional definition of the multivariate normal distribution relies on its density function. This approach, however, has several weaknesses:

(i)

It assumes that the explicit form of the density function is known.

(ii)

It requires the analytical evaluation of high-dimensional integrals to compute probabilities, moments, and marginal distributions.

The stochastic representation operator, denoted by $\stackrel{d}{=}$, is a powerful tool for constructing new distributions. If two random vectors X and Y have the same distribution, we write $\mathit{X}\stackrel{d}{=}\mathit{Y}$.

Examples

$U\stackrel{d}{=}1-U$, where U ∼ U(0, 1);

$X\stackrel{d}{=}-X$, where X ∼ N(0, σ²);

$X\stackrel{d}{=}\mu +\sigma Y$, where X ∼ N(μ, σ²) and Y ∼ N(0, 1).

Properties

(1)

$\mathbf{X}\stackrel{\mathbf{d}}{=}\mathbf{Y}$$\Rightarrow (f_1(\mathbf{X}),\dots ,f_m(\mathbf{X}))\stackrel{d}{=}(f_1(\mathbf{Y}),\dots ,f_m(\mathbf{Y})),$ where f_j are measurable functions.

Examples are ${\mathit{X}}^{\top }\mathit{X}\stackrel{d}{=}{\mathit{Y}}^{\top }\mathit{Y}$ and $\mathbf{X}/\parallel \mathbf{X}\parallel \stackrel{d}{=}\mathbf{Y}/\parallel \mathbf{Y}\parallel .$

(2)

X, Y, and Z are random variables,

$X\stackrel{d}{=}Y$ can imply $X+Z\stackrel{d}{=}Y+Z,$ if Z is independent of (X, Y).

(3)

If $X+Z\stackrel{d}{=}Y+Z$, and Z is independent of (X, Y), it is not necessary for $X\stackrel{d}{=}Y$.

(4)

If Z is independent of (X, Y), then

• $X\stackrel{d}{=}Y$ implies $XZ\stackrel{d}{=}YZ$;
• $XZ\stackrel{d}{=}YZ$ implies $X\stackrel{d}{=}Y$ under some conditions.

(5)

$X\stackrel{d}{=}Y$ if and only if $X^{+}\stackrel{d}{=}{Y}^{+}$ and $X^{-}\stackrel{d}{=}{Y}^{-}$.

Example 1.

The multivariate normal distribution, denoted by X ∼ N_p(μ, Σ), has the following stochastic representation:

$$\begin{array}{c}\hfill \mathit{X}\stackrel{d}{=}\mathit{\mu }+{\mathbf{A}}^{\top }\mathbf{Z},\end{array}$$

(30)

where $\mathit{\mu }={({\mu }_1,\dots ,{\mu }_p)}^{\top },{\mathbf{A}}^{\top }\mathbf{A}=\mathbf{\Sigma }$, and $\mathbf{Z}={(Z_1,\dots ,Z_p)}^{\top }$with i.i.d. elements Z_i ∼ N(0, 1). It is easy to find density function, moments, and marginal distributions of Z and then transfer these to X. For example, from the density function of Z, ${\displaystyle \frac{1}{{(\sqrt{2\pi })}^p}}{e}^{-{\displaystyle \frac{1}{2}}{\mathit{z}}^{\top }\mathit{z}}$, it is straightforward to find the density of X as

$$\begin{array}{c}\hfill f(x)={(2\pi )}^{-p/2}{\left|\mathbf{\Sigma }\right|}^{-1/2}\exp \left\{-{\displaystyle \frac{1}{2}}{(\mathit{x}-\mathit{\mu })}^{\top }(\mathit{x}-\mathit{\mu })\right\}.\end{array}$$

(31)

As the mean and covariance of Z are, respectively, $\mathbf{0}$ and I_p, one can immediately find E(X) = μ and Cov(X) = Σ, respectively. It is also easy to find that all the marginal and affine transformations of X are normally distributed. For more detailed discussion and applications, see [68].

Many efforts to extend constructing methods from univariate to multivariate, for example, Pearson system, exponential family, and many others, can be found in [4].

4.4.2. Dirichlet Distribution

Let X₁, …, X_m−1 and X_m be independent random variables, and X_i ∼ Gamma(α_i), i = 1, …, m. Set

$$\begin{array}{c}\hfill Y=X_1+\cdots +X_m,Y_j={\displaystyle \frac{X_j}{Y}},j=1,\dots ,m.\end{array}$$

The distribution of Y = (Y₁, …, Y_m)^⊤ is called the Dirichlet distribution with parameters ${\alpha }_1,\dots ,{\alpha }_m$ and is written as (Y₁, …, Y_m) ∼ D_m(α₁, …, α_m).

The beta distribution Be(α, β) is a special case of Dirichlet distribution. As $Y_1+\dots +Y_m=1$, the random vector $(Y_1,\dots ,Y_m)$ does not have a density on ${\mathbb{R}}^m$. However, the density of the first m − 1 components (with respect to Lebesgue measure on the simplex) is given by

$$\begin{array}{c}\hfill f(y_1,\dots ,y_{m-1})={\displaystyle \frac{1}{B_m(\mathit{\alpha })}}\prod _{i=1}^m{y}_i^{{\alpha }_i-1},\sum _{i=1}^m{y}_i=10\le y_i\le 1,i=1,\dots ,m,\end{array}$$

where α = (α₁, …, α_m) and

$$\begin{array}{c}\hfill B_m({\alpha }_1,\dots ,{\alpha }_m)={\displaystyle \frac{\mathrm{\Gamma }({\alpha }_1)\cdots \mathrm{\Gamma }({\alpha }_m)}{\mathrm{\Gamma }({\alpha }_1+\cdots +{\alpha }_m)}}.\end{array}$$

Moreover, Y is independent of $\{Y_j,j=1,\dots ,m-1\}$.

There is a fundamental relationship between the Dirichlet distribution and the uniform distribution on the unit sphere S^p = {x : x^⊤x = 1} [68].

Theorem 2.

Suppose that U ∼ U(S^p) in ${\mathbb{R}}^p$. Then U has a stochastic representation

$$\begin{array}{c}\hfill \mathit{U}=\left[\begin{array}{c}{\mathit{U}}_1\\ ⋮\\ {\mathit{U}}_m\end{array}\right]\stackrel{d}{=}\left[\begin{array}{c}{D}_1{\mathit{U}}^{(1)}\\ ⋮\\ D_m{\mathit{U}}^{(m)}\end{array}\right],\end{array}$$

(32)

where ${\mathit{U}}^{(j)}\in {\mathbb{R}}^{t_j},t_1+\cdots +t_m=p$, and

(i)

U^(j)∼ U(S^t_j), the uniform distribution on $S^{t_j},j=1,\cdots ,m$;

(ii)

${\mathit{U}}^{(1)},\cdots ,{\mathit{U}}^{(m)}$ and $(D_1,\cdots ,D_m)$ are mutually independent;

(iii)

$D_j>0,j=1,\cdots ,m$, and $(D_1^2,\cdots ,D_m^2)\sim D_m(t_1/2,\cdots ,t_m/2)$.

Based on the Dirichlet distribution, the multivariate Liouville distribution is defined on ${\mathbb{R}}_{+}^m$ by the stochastic representation $\mathbf{X}\stackrel{d}{=}R\mathbf{Y}$, where R > 0 is a random variable independent of Y and $\mathbf{Y}\sim D_m({\alpha }_1,\dots ,{\alpha }_m)$. For a comprehensive study of the Liouville distribution, see Chapter 6 of [6].

4.4.3. Spherical and Elliptical Distributions

There are a number of equivalent ways to define the spherical distribution. Let X be a p × 1 random vector. The following statements are equivalent [6]:

(i)

$\mathit{X}\stackrel{d}{=}\mathit{P}\mathit{X}$ for each P ∈ O(p), where O(p) is the set of orthogonal matrices of order p;

(ii)

The characteristic function (c.f.) of X is of the form ψ(t^⊤t) for some ψ ∈ Ψ_p where

$$\begin{array}{c}\hfill {\mathrm{\Psi }}_p=\{\psi (·):\psi (t_1^2+\dots +t_p^2)\mathrm{is} a \mathrm{p-dimensional} \mathrm{c.f.}\};\end{array}$$

(33)

(iii)

X has a stochastic representation $X\stackrel{d}{=}R{U}^{(p)}$, where random variable R ≥ 0 is independent of U^(p), U^(p) is uniformly distributed on the unit sphere S^p and does not have a density;

(iv)

For any $\mathbf{a}\in S^p\mathrm{and}\mathbf{b}\in S^p$, we have ${\mathbf{a}}^{\top }X\stackrel{d}{=}{\mathbf{b}}^{\top }X$.

Theorem 3.

Let $X\stackrel{d}{=}R{\mathbf{U}}^{(p)}$ and $P(X=\mathbf{0})=0$. Then

$$\Vert \mathit{X}\Vert \stackrel{d}{=}R,\mathit{X}/\Vert \mathit{X}\Vert \stackrel{d}{=}{\mathbf{U}}^{(p)},$$

and they are independent.

Example 2.

The theorem has two immediate applications. First, it provides a method to generate a random sample from the uniform distribution on the sphere, U(S^p): simply sample X from $N_p(\mathbf{0},{\mathit{I}}_p)$and normalize it to obtain ${\displaystyle \frac{\mathit{X}}{\Vert \mathit{X}\Vert }}$. Second, it allows the mixed moments of U^(p)to be easily calculated from the known mixed moments of the standard normal vector X and the moments of its radius R = ||X||. A detailed discussion can be found in Section 3.1.2 of [6].

Theorem 4.

Let X ∼ S_p(ψ) denote a spherically symmetric distribution in ${\mathbb{R}}^p$, characterized by its characteristic function ψ(t^⊤t) as (33). It is not necessary that X has a density. The random vector $\mathit{X}\stackrel{d}{=}R{\mathit{U}}^{(p)}$ has a density which must be of the form g(x^⊤x), if and only if the radial variable R has a density, denoted by f(r). The relationship between g and f is given by

$$f(r)={\displaystyle \frac{2{\pi }^{p/2}}{\mathrm{\Gamma }(p/2)}}{r}^{p-1}g(r^2),$$

where g(·) is called the density generator. We write X ∼ S_p(g).

The necessary condition that X ∼ S_p(ψ) has a density is

$${\int }_0^{\infty }{y}^{p/2-1}g(y)dy<\infty$$

as

$$\int g(x^{\top }x)dx={\displaystyle \frac{{\pi }^{p/2}}{\mathrm{\Gamma }(p/2)}}{\int }_0^{\infty }{y}^{p/2-1}g(y)dy=1.$$

Table 7. Some subclasses of p-dimensional spherical distributions.

Type	Density Function $\mathit{g}({\mathit{x}}^{\mathit{\top }}\mathit{x})$ or c.f. $\mathit{\psi }({\mathit{t}}^{\mathit{\top }}\mathit{t})$
Multinormal	$g({\mathit{x}}^{\top }\mathit{x})=c\exp (-{\displaystyle \frac{1}{2}}{\mathit{x}}^{\top }\mathit{x})$
Kotz-type	$g({\mathit{x}}^{\top }\mathit{x})=c{({\mathit{x}}^{\top }\mathit{x})}^{N-1}\exp [-r{({\mathit{x}}^{\top }\mathit{x})}^s],r,s>0,2N+p>2$
Pearson type VII	$g({\mathit{x}}^{\top }\mathit{x})=c{(l+{\mathit{x}}^{\top }\mathit{x}/s)}^{-N},N>p/2,s>0$
Multivariate t	$g({\mathit{x}}^{\top }\mathit{x})=c{(l+{\mathit{x}}^{\top }\mathit{x}/v)}^{-(p+v)/2},v>0$
Multivariate Cauchy	$g({\mathit{x}}^{\top }\mathit{x})=c{(l+{\mathit{x}}^{\top }\mathit{x})}^{-(p+1)/2}$
Pearson Type II	$g({\mathit{x}}^{\top }\mathit{x})=c{(l-{\mathit{x}}^{\top }\mathit{x})}^q,q>0$ for x⊤x < 1
Logistics	$g({\mathit{x}}^{\top }\mathit{x})=c\exp (-{\mathit{x}}^{\top }\mathit{x})/{[1+\exp (-{\mathit{x}}^{\top }\mathit{x})]}^2$
Scale mixture	$g({\mathit{x}}^{\top }\mathit{x})=c{\int }_0^{\infty }{t}^{-p/2}\exp (-{\mathit{x}}^{\top }\mathit{x}/2t)dG(t),G(t)a \mathrm{cdf}.$

gives many useful spherical distributions from [6].

Elliptical Distribution

A random vector X has an elliptical distribution, denoted as X ∼ EC_p(μ, Σ), with parameters μ and Σ = A^⊤A, if it can be expressed as

$$\begin{array}{c}\hfill \mathbf{X}\stackrel{d}{=}\mu +{\mathbf{A}}^{\top }\mathbf{Y}\stackrel{d}{=}\mu +R{\mathbf{A}}^{\top }{\mathbf{U}}^{(p)},\end{array}$$

(34)

where Y ∼ S_p(ψ) is spherical. This relationship generalizes the way a normal variable X ∼ N(μ, σ²) is constructed from a standard normal Z ∼ N(0, 1). Consequently, any subclass of spherical distributions in Table 7 defines a corresponding subclass of elliptical distributions.

4.4.4. Approximation to Multivariate Continuous Distributions by Representative Points

For multivariate continuous distributions, the QMC-RPs and MSE-RPs can be defined in a similar way to the univariate case.Suppose that a random vector X ∼ F(·) in ${\mathbb{R}}^n$ has a density function f(·) with a finite mean vector and covariance matrix. A set of points $\mathrm{\Xi }=\{{\mathit{\xi }}_j,j=1,\dots ,k\}$ is called mean squared error representative points if it minimizes the mean squared error, $MSE(\mathrm{\Xi })={\int }_{{\mathbb{R}}^n}{min}_{j=1,\dots ,k}{\parallel \mathit{x}-{\mathit{\xi }}_j\parallel }^{2}f(\mathit{x})d\mathit{x},$ where ∥ · ∥ denotes the L²-norm. For information regarding the construction of approximation distributions of a continuous multivariate distribution by their QMC-RPs, one can refer to [8,69,70]. For information regarding the construction of approximation distributions of a continuous multivariate distribution by their MSE-RPs, one can refer to [8,70,71,72,73]. Due to the complexity of computation in high-dimensional multivariate distribution, the related research works about MSE-RPs of this kind of distribution are limited. Representative points also have many applications for discrete data, especially for discrete massive data. In Chapter 9 of [8], some cases where the sample size is hugely larger than the number of variables have been considered, and some model-based and model-free subsampling methods are included.

4.5. Basic Statistics Under the Multivariate Population

Let X follow a multivariate continuous distribution F(x; θ) where θ represents parameters. A random sample ${\mathit{x}}_1,\dots ,{\mathit{x}}_n$ is drawn from this population. The distributions of basic statistics, such as the mean vector, the covariance matrix, and various test statistics, are fundamental to statistical inference and its applications. For example, when the population distribution is multivariate normal, N_p(μ, Σ), the standard unbiased estimators for μ and Σ are sample mean and the sample covariance, given, respectively, by

$$\begin{array}{c}\hfill \widehat{\mathit{\mu }}=\overline{\mathit{x}}={\displaystyle \frac{1}{n}}\sum _{i=1}^n{\mathit{x}}_i,\mathbf{S}={\displaystyle \frac{1}{n-1}}\sum _{i=1}^n{({\mathit{x}}_i-\overline{\mathit{x}})}^{\top }({\mathit{x}}_i-\overline{\mathit{x}}).\end{array}$$

(35)

It is known that $\overline{\mathit{x}}\sim N_p(\mathit{\mu },{\displaystyle \frac{1}{n-1}}\mathbf{\Sigma })$. Other key related distributions are described below.

Wishart distribution: The Wishart distribution, denoted by W_p(n, Σ), is the multivariate generalization of the chi-square distribution. It is defined by $W={\sum }_{i=1}^n{\mathbf{Z}}_i{\mathbf{Z}}_i^{\top }$ and ${\mathbf{Z}}_i\sim N_p(\mathbf{0},\mathbf{\Sigma })$ are i.i.d. The sample covariance matrix S follows Wishart distribution: (n − 1)S ∼ W_p(n − 1, Σ). This distribution was developed by John Wishart [74], who, in his seminal 1928 paper “The Generalised Product Moment Distribution in samples from a normal multivariate population”, published in Biometrika, derived the distribution of the matrix of sums of squares and cross-products. A crucial result for normally distributed data is that the sample mean $\overline{\mathit{x}}$ and the sample covariance matrix S are independent, generalizing the corresponding univariate property.

Distributions of the eigenvalues of S: Unlike the sample covariance matrix itself, the distributions of its individual eigenvalues are complex and do not follow simple, universal forms like the normal or Wishart distributions.

Hotelling’s T-squared distribution: It is the multivariate generalization of the Student’s t-distribution, used primarily for hypothesis testing on the mean vector. For a random vector X ∼ N_p(μ, Σ) independent of S ∼ W_p(n, Σ), the Hotelling’s T² statistic is denoted as

$$\begin{array}{c}\hfill T^2=n{\mathit{X}}^{\top }{\mathbf{S}}^{-1}\mathit{X}.\end{array}$$

(36)

The resulting T² statistic, despite being derived from multivariate data, is a scalar. Therefore, its sampling distribution, denoted by T²(p, n, μ), is a univariate distribution. When $\mathit{\mu }=\mathbf{0}$, the distribution is central and denoted by T²(p, n). The T² statistic is related to the F-distribution through the following identity: ${\displaystyle \frac{n-p+1}{np}}{T}^2(p,n)\sim F(p,n-p+1).$

Wilks’ lambda distribution: This distribution generalizes the beta distribution and is used in multivariate analysis of variance and likelihood ratio tests. Let A₁ ∼ W_p(m, Σ) and A₂ ∼ W_p(n, Σ) be independent. The Wilks’ lambda statistic, defined as $\mathsf{\Lambda }={\displaystyle \frac{A_2}{A_1+A_2}}$, is a scalar. Thus, its sampling distribution, denoted by Λ(p, m, n), is a univariate distribution. Wilks’ lambda has close relationships with the T² distribution and simplifies to an F or beta distribution when p = 1.

When the population X follows an elliptical distribution (a broader class than the normal), deriving the distributions of the aforementioned statistics becomes significantly more challenging. A comprehensive discussion of this topic can be found in [68].

Multivariate L₁-Norm Symmetric Distribution

The elliptical distribution can be regarded as an extension of the multivariate normal distribution. Similarly, the exponential distribution can be generalized to a multivariate setting. Let $X_1,\dots ,X_p$ be i.i.d. random variables from the standard exponential distribution, Exp(1) (see Table 4). Consider the random vector $\mathit{X}={(X_1,\dots ,X_p)}^{\top }$ and its L₁-norm defined by ${\Vert \mathbf{X}\Vert }_1=|X_1|+\dots +|X_p|$. Let $\mathbf{U}={\displaystyle \frac{\mathbf{X}}{{\Vert \mathbf{X}\Vert }_1}}$. Fang and Fang [75] defined a class of multivariate distributions via the following stochastic representation:

$$\begin{array}{c}\hfill \mathbf{X}\stackrel{d}{=}R\mathbf{U},\end{array}$$

(37)

where the radial variable R is independent of the directional vector U. This class of distributions was studied systematically in [75,76] and in Chapter 5 of [6].

4.6. Definition by Distribution, Density or Survival Functions

4.6.1. Meta-Skew Symmetric Distributions

A traditional approach to defining a probability distribution is through its distribution function, density function, or survival function. For instance, the multivariate normal distribution in Anderson’s book [77] is defined by its density function. A general univariate form of an asymmetric distribution, known as the skew-symmetric family, can be represented as a function that combines a cumulative distribution function and a probability density function; see [78,79].

Definition 2.

A random variable X, whose density function has the form

$$\begin{array}{c}\hfill p(x)=2f(x)G(w(x)),\end{array}$$

(38)

is called skew symmetric, where f(x) is a density function centrally symmetric about 0, w(x) is an odd real-valued function such that w(−x) = −w(x), and G(w(·)) is a cumulative distribution function on $\mathbb{R}$such that $g=G^{\prime }$ is an even density function, providing G is differentiable.

4.6.2. Skew-Normal Distribution

The multivariate skew-normal (MSN) distribution, introduced in Table 3, is a fundamental and powerful generalization of both the multivariate normal and the univariate skew-normal distributions. It provides a flexible framework for modeling multivariate data that exhibits asymmetry (skewness).

A p-dimensional random vector Y follows a multivariate skew-normal distribution with location vector μ, positive definite scale matrix Σ, and skewness vector α if its probability density function is given by

$$\begin{array}{c}\hfill f(y;\mathit{\mu },\mathbf{\Sigma },\mathit{\alpha })=2{\varphi }_p(y-\mathit{\mu };\mathbf{\Sigma })\mathrm{\Phi }({\mathit{\alpha }}^{\top }{\mathit{\omega }}^{-1}(y-\mathit{\mu })),\end{array}$$

where ϕ_p(z; Σ) is the pdf of p-variate normal distribution with mean $\mathbf{0}$ and covariance matrix Σ; Φ(·) is the cdf of N(0, 1); ω is the diagonal matrix formed from the standard deviations of Σ (the square roots of the diagonal elements), and the vector α is the shape or skewness parameter vector. It denotes Y ∼ SN_p(μ, Σ, α).

The MSN distribution admits an intuitive stochastic representation. Consider a (p + 1)-dimensional normal random vector ${(X_0,{\mathit{X}}_1^{\top })}^{\top }$,

$$\begin{array}{c}\hfill \left[\begin{array}{c}{X}_0\\ {\mathit{X}}_{\mathbf{1}}\end{array}\right]\sim N_{1+p}\left[\left(\begin{array}{c}0\\ \mathbf{0}\end{array}\right),\left(\begin{array}{cc}1& {\mathit{\delta }}^{\top }\\ \mathit{\delta }& \mathbf{\Sigma }\end{array}\right)\right].\end{array}$$

(39)

Here, X₀ is a scalar and X₁ is a p-vector. The vector δ controls the correlation between X₀ and each component of X₁. Then

$$\begin{array}{c}\hfill Y\stackrel{d}{=}\mathit{\mu }+\mathit{\omega }({\mathit{X}}_1|X_0>0)\sim S{N}_p(\mathit{\mu },\mathbf{\Sigma },\mathit{\alpha }),\end{array}$$

where the skewness parameter α relates to δ through the following equation:

$$\begin{array}{c}\hfill \mathit{\alpha }={\displaystyle \frac{1}{\sqrt{1-{\mathit{\delta }}^{\top }{\mathbf{\Sigma }}^{-1}\mathit{\delta }}}}{\mathbf{\Sigma }}^{-1}\mathit{\delta }.\end{array}$$

For comprehensive details on the theory and applications of the multivariate skew-normal distribution, see [80,81].

4.6.3. Skew-Elliptical Distributions

Skew-elliptical distributions form a broad family that generalizes symmetric elliptical distributions by introducing skewness (asymmetry). This extension moves beyond the familiar world of symmetric, bell-shaped curves into the asymmetric realm, providing a principled and mathematically sound framework for modeling a wider range of real-world data patterns. Among these, the skew-t distribution is often a particularly useful starting point for applied work due to its flexibility in handling both skewness and heavy tails.

The general form of the pdf for a skew-elliptical distribution is

$$\begin{array}{c}\hfill f(z)=2f_0(z)\mathrm{\Pi }(z),\end{array}$$

(40)

where f₀(z) is the pdf of a symmetric elliptical distribution (the “parent”), and Π(z) is a skewing function that maps onto the interval [0, 1] and satisfies Π(z) + Π(−z) = 1. This function controls the direction and amount of skewness. The skew-elliptical family is vast, but several members are particularly important: skew-normal, skew-t, skew-Cauchy, and skew-Laplace distributions. The skew-elliptical distributions are used anywhere that data deviates from symmetry and normality, such as in finance, environmental science, biology, psychometrics, and social sciences. For a comprehensive study, refer to [82].

4.6.4. Definition by Survival Function

The survival function of a random variable X ∼ F(x) is defined as $\overline{F}(x)=P(X>x)$. This function is particularly useful in the analysis of biometrical data. Within this domain, the exponential and Weibull distributions play an important role.

A foundational model for dependent failures is the Marshall–Olkin multivariate exponential distribution [83]. Its strength lies in an intuitive shock-based construction and analytical tractability. The model assumes that shocks to the system components arrive according to three independent Poisson processes with rates λ₁, λ₂, and λ₁₂, corresponding to shocks that fatally affect only component 1, only component 2, or both components simultaneously. The arrival times of these shocks, Z₁, Z₂, and Z₁₂, are, therefore, independent exponential random variables with parameters λ₁, λ₂, and λ₁₂, respectively.

We set X = min{Z₁, Z₁₂} and Y = min{Z₂, Z₁₂}. The joint survival function of the random vector (X, Y) is defined as $\overline{H}(x,y)=P(X>x,Y>y)$. For any x, y ≥ 0 the survival function is given by

$$\begin{array}{ccc}\hfill \overline{H}(x,y)& =& P(Z_1>x)P(Z_2>y)P(Z_{12}>max(x,y))\hfill \\ & =& \exp [-{\lambda }_{1}x-{\lambda }_{2}y-{\lambda }_{12}max(x,y)].\hfill \end{array}$$

Its marginal survival functions are

$$\begin{array}{c}\hfill \overline{F}(x)=\exp (-({\lambda }_1+{\lambda }_{12})x),\mathrm{and} \overline{G}(y)=\exp (-({\lambda }_2+{\lambda }_{12})y).\end{array}$$

Hence, X and Y are exponential random variables with parameters λ₁ + λ₁₂ and λ₂ + λ₁₂, respectively. This elegant construction can be extended to higher dimensions.

4.7. Meta-Distributions via Copula Techniques

Various statistical distributions play a key role in modeling real-world data. The rapid development of data science has increased the demand for a wider variety of multivariate distributions. A common challenge is to construct a multivariate distribution from given one-dimensional marginal distributions while incorporating a known dependence structure. Such constructions arise not only from statistical theory but also from practical problems, particularly in risk and decision analysis for complex multivariate systems. The copula is a powerful technique designed for this purpose.

Copulas are mathematical functions that fully capture the dependence structure among random variables, offering great flexibility in building multivariate stochastic models. Since their introduction in the 1950s, copulas have gained significant popularity in several applied fields such as finance, insurance, and reliability theory. Nowadays, they are a well-recognized tool for market and credit modeling, risk aggregation, portfolio selection, and more.

The following definition is based on probability theory. For a more general treatment, see [66,84]. As most research works and applications are interested in the copulas with respect to continuous distributions, and it becomes complicated for discrete distributions, this section only covers the continuous case.

Definition 3.

If $C(z_1,\dots ,z_p)$is the cumulative distribution function on the unit hypercube [0, 1]^p with uniform U(0, 1) margins, then C is called a copula.

The following Lemma demonstrates the existence of the copula function.

Lemma 1.

Let $H(x_1,\dots ,x_p)$be a continuous distribution with p continuous margins F_i on the domain D_i for $i=1,\dots ,p$. Then there exists a unique copula C such that

$$\begin{array}{c}\hfill C(z_1,\dots ,z_p)=H(F_1^{-1}(z_1),\dots ,F_p^{-1}(z_p)).\end{array}$$

(41)

It is well known that for a given set of p marginal distributions ${F_i(x_i)}_{i=1}^p$, there exist infinitely many p-dimensional distributions with these margins. If the dependence structure of the joint distribution is known, one may find a unique or a small family of such joint distributions $H(x_1,\dots ,x_p)$. The copula technique is a fundamental tool for this purpose.

Suppose we have p random variables X_i ∼ F_i(x_i) for $i=1,\dots ,p$, and we wish to construct a multivariate distribution H with these margins and a given dependence structure defined by a copula $C(z_1,\dots ,z_p)$. This method is called the copula technique. The resulting distribution is termed a meta-distribution and is denoted by $F(C;F_1,\dots ,F_p)$.

Theorem 5.

Sklar’s Theorem (Multivariate Case) Let X be a continuous random vector with joint distribution $H(x_1,\dots ,x_p)$ and continuous marginal distributions F_i on domains D_i for $i=1,\dots ,p$. Then there exists a unique copula C such that

$$\begin{array}{c}\hfill H(x_1,\dots ,x_p)=C(F_1(x_1),\dots ,F_p(x_p)).\end{array}$$

(42)

Conversely, if C is a copula and $F_1,\dots ,F_p$ are univariate distribution functions, then the function H defined by (42) is a p-dimensional distribution function with margins $F_1,\dots ,F_p$.

If the margins F_i are continuous, then C is unique. The joint density function of X derived from (42) is given by

$$\begin{array}{c}\hfill h(x_1,\dots ,x_p)=c(F_1(x_1),\dots ,F_p(x_p))\prod _{i=1}^p{f}_i(x_i),\end{array}$$

(43)

where f_i is the probability density function of X_i, and c(·) is the copula density, i.e., the p-th mixed partial derivative of C.

4.7.1. Some Examples of Copula

Many copula examples exist for lower dimensions. This subsection presents selected examples from Nelsen [66] to provide a tutorial for beginners to comprehend the ideas of copula. Examples 3 and 4 have different joint distributions but have the same copula.

Example 3.

Adapted from Nelsen [66]

Consider the joint distribution

$$\begin{array}{c}\hfill H(x,y)={\displaystyle \frac{(x+1)(e^y-1)}{x+2e^y-1}},(x,y)\in [-1,1]\times [0,\infty ).\end{array}$$

Its marginal distributions are X ∼ U(−1, 1) with cdf $F(x)={\displaystyle \frac{x+1}{2}}$ and Y ∼ Exp(1), the standard exponential distribution with cdf G(y) = 1 − e^−y, and copula

$$\begin{array}{c}\hfill C(u,v)={\displaystyle \frac{uv}{u+v-uv}}.\end{array}$$

We now demonstrate how to derive the copula C from H, and, conversely, how to reconstruct H from C, F, G.

Deriving C from H:

Let u = F(x) and v = G(y), which implies $u={\displaystyle \frac{x+1}{2}}$and v = 1 − e^−y or $e^y={\displaystyle \frac{1}{1-v}}$.

$$\begin{array}{ccc}\hfill C(u,v)& & =H(F^{-1}(u),G^{-1}(v))={\displaystyle \frac{(F^{-1}(u)+1)(e^{G^{-1}(v)}-1)}{F^{-1}(u)+2e^{G^{-1}(v)}-1}}\hfill \\ & & ={\displaystyle \frac{[(2u-1)+1][e^{-ln(1-v)}-1]}{2u-1+2e^{-ln(1-v)}-1}}\hfill \\ & & ={\displaystyle \frac{2u(\frac{1}{1-v}-1)}{2u-1+2\frac{1}{1-v}-1}}={\displaystyle \frac{2u[1-(1-v)]}{(2u-1)(1-v)+2-(1-v)}}\hfill \\ & & ={\displaystyle \frac{uv}{u+v-uv}}.\hfill \end{array}$$

Deriving H from {C, F, G}:

Note that $e^y={\displaystyle \frac{1}{1-v}}$; we have

$$\begin{array}{ccc}\hfill H(x,y)& & =C(u,v)\hfill \\ & & ={\displaystyle \frac{\frac{x+1}{2}(1-e^{-y})}{\frac{x+1}{2}+1-e^{-y}-\frac{x+1}{2}(1-e^{-y})}}\hfill \\ & & ={\displaystyle \frac{(x+1)(1-e^{-y})}{x+1+2(1-e^{-y})-(x+1)(1-e^{-y})}}\hfill \\ & & ={\displaystyle \frac{(x+1)(e^y-1)}{(x+1)+2e^y-2}}={\displaystyle \frac{(x+1)(e^y-1)}{x+2e^y-1}}.\hfill \end{array}$$

Example 4.

Gumbel’s bivariate logistic distribution [85].

$$\begin{array}{c}\hfill H(x,y)={(1+e^{-x}+e^{-y})}^{-1},x,y\ge 0.\end{array}$$

Its two margins are

$$\begin{array}{c}\hfill F(x)={(1+e^{-x})}^{-1},G(y)={(1+e^{-y})}^{-1}.\end{array}$$

Its copula is

$$\begin{array}{c}\hfill C(u,v)={\displaystyle \frac{uv}{u+v-uv}},\end{array}$$

which is identical to the copula in Example 3, despite the marginal distributions being quite different.

For multivariate copulas, higher-dimensional margins are defined as per the following example.

Example 5.

Consider the following three-dimensional distribution H:

$$\begin{array}{c}\hfill H(x,y,z)={\displaystyle \frac{(x+1)(e^z-1)sin(z)}{x+2e^y-1}},on[-1,1]\times [0,\infty ]\times [0,{\displaystyle \frac{\pi }{2}}].\end{array}$$

Its three one-dimensional marginal distributions are

$$\begin{array}{c}\hfill H_1(x)={\displaystyle \frac{x+1}{2}},H_2(y)=1-e^{-y},H_3(z)=sin(z).\end{array}$$

The three two-dimensional marginal distributions are

$$\begin{array}{ccc}& & H_{12}(x,y)={\displaystyle \frac{(x+1)(e^y-1)}{x+2e^y-1}},\hfill \\ & & H_{23}(y,z)=(1-e^{-y})sin(z),\hfill \\ & & H_{13}(x,z)={\displaystyle \frac{1}{2}}(x+1)sin(z).\hfill \end{array}$$

Example 6.

Fang et al. [86] provided a comprehensive study of the copula defined for α > 0 and β ∈ [0, 1]:

$$\begin{array}{c}\hfill C(u,v;\alpha ,\beta )={\displaystyle \frac{uv}{{[1-\beta (1-u^{1/\alpha })(1-v^{1/\alpha })]}^{\alpha }}}.\end{array}$$

(44)

For different values of α and β, we have the following:

1. 

β = 0; U and V are independent.

2. 

β = 1; it reduces to the bivariate Clayton distribution [87].

3. 

α = 1; it reduces to the Ali–Mikhail–Haq copula [88].

Its multivariate version is given by

$$\begin{array}{c}\hfill C(u_1,\dots ,u_n;\alpha ,\beta )={\displaystyle \frac{{\prod }_{i=1}^n{u}_i}{{[1-\beta {\prod }_{i=1}^n(1-u_i^{1/\alpha })]}^{\alpha }}}.\end{array}$$

(45)

These examples demonstrate that the copula technique provides a powerful method for constructing numerous new distributions useful for statistical modeling.

4.7.2. Gaussian Copula and Meta-Gaussian Distribution

When H(·) is a multivariate normal distribution, the associated copula is called a Gaussian copula. When H(·) is an elliptical distribution, the corresponding copula is known as an elliptical copula. This family includes the Gaussian copula, the t-copula, and the Kotz-type copula, among others; for a comprehensive overview, see [89].

From the definition of a copula, the Gaussian copula for N_p(μ, Σ) is identical to that of $\mathbf{Z}=(Z_1,\dots ,Z_p)\sim N_p(\mathbf{0},\mathit{R})$, where R is the correlation matrix of Z. Clearly, all marginal distributions of Z are standard normal N(0, 1), yet its components have correlation matrix R = (r_ij). Denote the cdf of Z by $G_{\mathrm{normal}}(z_1,\dots ,z_p\mid \mathit{R})$ and define a function C as follows:

$$\begin{array}{c}\hfill C_{normal}(u_1,\dots ,u_p)=G_{normal}({\mathrm{\Phi }}^{-1}(u_1),\dots ,{\mathrm{\Phi }}^{-1}(u_p)),{(u_1,\dots ,u_p)}^{\top }\in {[0,1]}^p,\end{array}$$

(46)

where Φ(·) is the cdf of N(0, 1). This function C is the Gaussian copula. Kelly and Krzysztofowicz [90] studied the meta-Gaussian (or meta-normal) distribution. A more detailed analysis can be found in Section 2.6, “Multivariate Gaussian/Normal”, of [91].

Definition 4.

The meta-Gaussian or meta-normal distribution with a set of given marginal cdfs $F_1,\dots ,F_p$ is defined by its cdf

$$\begin{array}{c}\hfill F_{X_1,\dots ,X_p}(x_1,\dots ,x_p)=G_{Z_1,\dots ,Z_p}({\mathrm{\Phi }}^{-1}(F_1(x_1)),\dots ,{\mathrm{\Phi }}^{-1}(F_p(x_p))).\end{array}$$

(47)

We denote this distribution as $\mathit{X}\sim M{N}_p(\mathbf{0},\mathit{R};F_1,\dots ,F_p)$.

The pdf of $M{N}_p(\mathbf{0},\mathit{R};F_1,\dots ,F_p)$ is

$$\begin{array}{c}\hfill h(x_1,\dots ,x_p)=c_{normal}(F_1(x_1),\dots ,F_p(x_p))\prod _{i=1}^p{f}_i(x_i),\end{array}$$

where c_normal is the density of the Gaussian copula and f_i is the pdf corresponding to F_i. In the case where the correlation matrix has all off-diagonal elements equal to ρ, Bouy’e [92] provided illustrative plots for various values of ρ.

Let $\mathit{X}\sim M{N}_2(\mathbf{0},\mathit{R};F_1,F_2)$, where the correlation matrix with Corr(X₁, X₂) = ρ. Then the normal copula is given by

$$\begin{array}{c}\hfill C_{normal}(u,v)={\displaystyle \frac{1}{2\pi \sqrt{1-{\rho }^2}}}{\int }_{-\infty }^{{\mathrm{\Phi }}^{-1}(u)}{\int }_{-\infty }^{{\mathrm{\Phi }}^{-1}(v)}\exp \left\{-{\displaystyle \frac{s^2-2\rho st+t^2}{2(1-{\rho }^2)}}\right\}dsdt,\end{array}$$

where 0 ≤ u, v ≤ 1.

The correlation coefficient has been used in multivariate analysis for a long time. However, the copula and its related meta-distributions might have different correlation coefficients due to different marginal distributions. Therefore, the so-called “measure of concordance” is considered in the literature. It requests that the copula and its meta-distributions have the same values of the measure of concordance. The Kendall’s τ and the Spearman’s rho are the most popular ones used in the applications. Their definitions can refer to [66]. Spearman’s ρ for the binormal copula is given by ${\displaystyle \frac{6}{\pi }}arcsin\left({\displaystyle \frac{\rho }{2}}\right)$ (see [93]). This result does not extend to other elliptical distributions, which are introduced in the next section.

The meta-normal distribution $\mathit{X}\sim M{N}_2(\mathbf{0},\mathit{R};F_1,F_2)$ is given by

$$\begin{array}{ccc}\hfill F(x_1,x_2)& & =C_{normal}(F_1(x_1),F_2(x_2))\hfill \\ & & ={\displaystyle \frac{1}{2\pi \sqrt{1-{\rho }^2}}}{\int }_{-\infty }^{{\mathrm{\Phi }}^{-1}(F_1(x_1))}{\int }_{-\infty }^{{\mathrm{\Phi }}^{-1}(F_2(x_2))}\exp \left\{-{\displaystyle \frac{s^2-2\rho st+t^2}{2(1-{\rho }^2)}}\right\}dsdt.\hfill \end{array}$$

(48)

Mai and Scherer [94], in Example 1.7, discussed the Gaussian copula and meta-Gaussian distributions for the general case X ∼ N_p(μ, Σ). In this case, the marginal distributions of X_i ($i=1,\dots ,p$) are not necessarily identical if the variances Var(X_i) differ. Now, consider the two-dimensional normal distribution $N_2(\mathbf{0},\mathit{R})$ with correlation matrix R and correlation coefficient ρ. Let ξ_i = Φ⁻¹(u_i) for i = 1, 2. The density of the Gaussian copula, as given in Chapter 7 of [95], is

$$\begin{array}{c}\hfill c_{normal}(\mathit{\xi })=c_{normal}({\xi }_1,{\xi }_2)={\left|\mathit{R}\right|}^{-1/2}\exp \left(-{\displaystyle \frac{1}{2}}{\mathit{\xi }}^{\top }({\mathit{R}}^{-1}-\mathit{I})\mathit{\xi }\right).\end{array}$$

(49)

This expression can be factorized into two independent components

$$\begin{array}{c}\hfill c_{normal}(\mathit{\xi })={\displaystyle \frac{1}{2\pi \sqrt{1-{\rho }^2}}}\exp \left(-{\displaystyle \frac{1}{2}}{\mathit{\xi }}^{\top }{\mathit{R}}^{-1}\mathit{\xi }\right)[2\pi ·\exp ({\displaystyle \frac{1}{2}}({\xi }_1^2+{\xi }_2^2))].\end{array}$$

The first factor is the density of $N_2(\mathbf{0},\mathit{R})$, while the second factor is the product of the densities of two independent standard normal variables (up to a constant),

$$\sqrt{2\pi }\exp ({\displaystyle \frac{1}{2}}{\xi }_1^2)\mathrm{and}\sqrt{2\pi }\exp ({\displaystyle \frac{1}{2}}{\xi }_2^2).$$

Recently, He et al. [96] derived the covariance matrix of the Gaussian copula and related properties.

4.7.3. Marshall–Olkin Copula

The Marshall–Olkin multivariate exponential distribution was introduced in Section 4.6.4. We now derive its corresponding survival copula, denoted by ${\widehat{C}}_{MO}(u,v)$. We begin by expressing the joint survival function $\overline{H}(x,y)$ in terms of the marginal survival functions $u=\overline{F}(x)$ and $v=\overline{G}(y)$, recalling the relationship $\overline{H}(x,y)=\widehat{C}(\overline{F}(x),\overline{G}(y))$, where $\widehat{C}$ is the survival copula. Using the identity max(x, y) = x + y − min(x, y), we can rewrite the survival function as follows:

$$\begin{array}{ccc}\hfill \overline{H}(x,y)& =& \widehat{C}(\overline{F}(x),\overline{G}(y))\hfill \\ & =& \exp [-({\lambda }_1+{\lambda }_{12})x-({\lambda }_2+{\lambda }_{12})y+{\lambda }_{12}min(x,y)]\hfill \\ & =& \overline{F}(x)\overline{G}(y)min\{\exp ({\lambda }_{12}x),\exp ({\lambda }_{12}y)\}.\hfill \end{array}$$

As $u^{{\displaystyle \frac{-{\lambda }_{12}}{{\lambda }_1+{\lambda }_{12}}}}=exp({\lambda }_{12}x)$ and $v^{{\displaystyle \frac{-{\lambda }_{12}}{{\lambda }_2+{\lambda }_{12}}}}=exp({\lambda }_{12}y)$, let $\alpha ={\displaystyle \frac{{\lambda }_{12}}{{\lambda }_1+{\lambda }_{12}}}$ and $\beta ={\displaystyle \frac{{\lambda }_{12}}{{\lambda }_2+{\lambda }_{12}}}$. Obviously, 0 < α, β < 1, and let

$$\begin{array}{ccc}\hfill u& =& \overline{F}(x)=e^{-({\lambda }_1+{\lambda }_{12})x}=e^{-{\lambda }_{12}x/\alpha },u^{-\alpha }=e^{{\lambda }_{12}x},\hfill \\ \hfill v& =& \overline{G}(y)=e^{-({\lambda }_2+{\lambda }_{12})y}=e^{-{\lambda }_{12}y/\beta },v^{-\beta }=e^{{\lambda }_{12}y}.\hfill \end{array}$$

The survival copula for the Marshall–Olkin distribution, known as the Marshall–Olkin copula, is given by

$$\begin{array}{ccc}\hfill {\widehat{C}}_{MO}(u,v)& & =uvmin\left\{u^{{\displaystyle \frac{-{\lambda }_{12}}{{\lambda }_1+{\lambda }_{12}}}},v^{{\displaystyle \frac{-{\lambda }_{12}}{{\lambda }_2+{\lambda }_{12}}}}\right\}=min\left\{v{u}^{1-{\displaystyle \frac{{\lambda }_{12}}{{\lambda }_1+{\lambda }_{12}}}},u{v}^{1-{\displaystyle \frac{{\lambda }_{12}}{{\lambda }_2+{\lambda }_{12}}}}\right\}\hfill \\ & & =uvmin(u^{-\alpha },v^{-\beta })=min(u^{1-\alpha }v,u{v}^{1-\beta }).\hfill \end{array}$$

(50)

Therefore, the survival copula for the Marshall–Olkin bivariate exponential distribution forms a two-parameter family of copulas given by

$$\begin{array}{c}\hfill C_{\alpha ,\beta }(u,v)=\left\{\begin{array}{cc}\hfill u^{1-\alpha }v,& \hfill \mathrm{if}{u}^{\alpha }\ge v^{\beta },\\ \hfill u{v}^{1-\beta },& \hfill \mathrm{if}{u}^{\alpha }\le v^{\beta }.\end{array}\right.\end{array}$$

(51)

The application of copulas and meta-distributions is a cornerstone of modern quantitative finance, risk management, and many other fields that deal with multivariate dependence. They provide a powerful toolkit for modeling complex relationships beyond simple correlation.

5. Conclusions

The development of statistical distributions, driven by the need to model real data, is a cornerstone of statistical science. With a vast and well-established body of literature, a complete review of construction methods is beyond our scope. This article, instead, focuses on recent advancements in the field, particularly in simulation techniques and computational algorithms. Our aim is to offer readers an expanded perspective on the evolving landscape of statistical distributions.