Bayesian Inference for Skew-Symmetric Distributions

Abstract

Skew-symmetric distributions are a popular family of flexible distributions that conveniently model non-normal features such as skewness, kurtosis and multimodality. Unfortunately, their frequentist inference poses several difficulties, which may be adequately addressed by means of a Bayesian approach. This paper reviews the main prior distributions proposed for the parameters of skew-symmetric distributions, with special emphasis on the skew-normal and the skew-t distributions which are the most prominent skew-symmetric models. The paper focuses on the univariate case in the absence of covariates, but more general models are also discussed.

1. Introduction

The need to model skewed data led to the development of many skewed distributions which are obtained by adding to a symmetric distribution a parameter that controls skewness [1]. Arguably, the best known example is the skew-normal distribution introduced by Azzalini (1985) [2]. Its probability density function (pdf) is

$$sn(x;\mu ,\sigma ,\lambda )=\frac{2}{\sigma }\varphi \left(\frac{x-\mu }{\sigma }\right)\mathsf{\Phi }\left(\lambda \frac{x-\mu }{\sigma }\right),x\in \mathbb{R},$$

(1)

where $\mu \in \mathbb{R}$ is the location parameter, $\sigma \in {\mathbb{R}}^{+}$ the scale parameter, both inherited from the standard normal distribution with pdf denoted by $\varphi$ and (cumulative distribution function) cdf $\mathsf{\Phi }$, and $\lambda \in \mathbb{R}$ is called the skewness parameter given that density (1) is asymmetric for $\lambda \ne 0$ and reduces to the standard normal pdf for $\lambda =0$. Several extensions and generalizations followed, see [3], a very general and highly popular being the skew-symmetric distributions of Wang et al. (2004) [4] with pdf

$$\begin{array}{c}\hfill s_{f;\mathrm{\Pi }}(x;\mu ,\sigma ,\lambda )=\frac{2}{\sigma }f\left(\frac{x-\mu }{\sigma }\right)\mathrm{\Pi }\left(\frac{x-\mu }{\sigma },\lambda \right),x\in \mathbb{R},\end{array}$$

where f is the symmetric density to be skewed and $\mathrm{\Pi }:\mathbb{R}\times \mathbb{R}\to [0,1]$ is a so-called skewing function satisfying $\mathrm{\Pi }(z,\lambda )+\mathrm{\Pi }(-z,\lambda )=1\forall z,\lambda \in \mathbb{R}$ and $\mathrm{\Pi }(z,0)=1/2\forall z\in \mathbb{R}$. The most widely used subfamily of skew-symmetric distributions has densities of the form

$$\begin{array}{c}\hfill s_{f;G}(x;\mu ,\sigma ,\lambda )=\frac{2}{\sigma }f\left(\frac{x-\mu }{\sigma }\right)G\left(\lambda \frac{x-\mu }{\sigma }\right),x\in \mathbb{R},\end{array}$$

(2)

where G is any symmetric, univariate, absolutely continuous cumulative distribution function. In (2), $\mu \in \mathbb{R}$ is a location, $\sigma \in {\mathbb{R}}^{+}$ a scale and $\lambda \in \mathbb{R}$ a skewness parameter. The function G might be replaced by a function $w\left(·\right)$ satisfying $0\le w\left(-x\right)=1-w\left(x\right)\le 1$, as done in [4,5]. Different skew models might then be obtained by different choices for f and G, as for example the pdf and the cdf of the power exponential distribution [6], the Student t distribution [5] and the logistic distribution [7].

The skew-normal distribution admits several stochastic representations, of which a very simple is given as follows [8]: let U and V be independent, standard normal variables, then

$$Z_{\lambda }=\frac{\lambda }{\sqrt{1+{\lambda }^2}}\left|U\right|+\frac{1}{\sqrt{1+{\lambda }^2}}V$$

(3)

has the standard skew-normal distribution $SN\left(\lambda \right)=SN\left(0,1,\lambda \right)$. This representation models departures from normality, allows for simple generation of random numbers and the calculation of odd moments. Despite its numerous appealing probabilistic properties [3], the skew-normal distribution suffers from serious inferential problems, as remarked already by Azzalini (1985). In the first place, the method of moments might lead to complex estimates of the parameters. In the second place, when $\mu =0$, $\sigma =1$ and all observations are positive (negative) the likelihood function will be monotonically increasing (decreasing) in $\lambda$, thus making the maximum likelihood estimate (MLE) of $\lambda$ (minus) infinite. In the third place, the MLE is quite unstable even in the presence of negative and positive observations. In the fourth place, skew-normal data offer little help in discriminating between different values of the shape parameter $\lambda$: very different values of $\lambda$ might correspond to very similar skew-normal densities. In the fifth place, the sampling distribution of the MLE does not allow for analytically tractable standard errors and confidence intervals. In the sixth place, if all three parameters are assumed to be unknown, the profile likelihood function for $\lambda$ has a stationary point at $\lambda =0$, regardless of the observed sample, and the Fisher information matrix is singular when $\lambda$ approaches zero.

The latter problem, also known as the singularity problem, is very serious and has motivated an active line of research both for multivariate and semiparametric generalizations of the skew-normal distribution. It has been discussed in various papers such as [9,10,11,12,13,14,15]. Different parameterizations have been proposed by, among others, [2,11,13,16]. While most authors were pointing at some special status for the normal distribution as symmetric base distribution, Ref. [12] showed that this singularity can occur for very general symmetric densities f and provided a full characterization, in a general multivariate context, of the singularity problem, showing that it is due to unfortunate mismatches between the symmetric density f and the skewing function $\mathrm{\Pi }(·,·)$. It is to be noted that in the context of models (2) singularity occurs only when f is normal, irrespective of the choice of function G. However, even the proposed solutions of reparameterization do not remove all problems as remarked by Azzalini and Capitanio (1999) [9]: “there are cases where the likelihood shape and the MLE are problematic. We are not referring here to difficulties with numerical maximization, but to the intrinsic properties of the likelihood function, not removable by change of parameterization. In case of this sort, the behaviour of the MLE appears quite unsatisfactory, and an alternative estimation method is called for”.

These difficulties arise from the shape of the likelihood function, which can be modified by appropriate weighting functions [17]. The most natural and best known approach to weight the likelihood function is the Bayesian one, where the prior distribution plays the role of the weight function. The Bayesian approach might adequately address both point estimation and hypothesis testing of the skewness parameter $\lambda$. Most papers in this area focused on objective priors for the skew-normal distribution, whereas only a few of them dealt with its multivariate or semiparametric generalizations. Bayesian inference and prior elicitation of $\lambda$ for skew-symmetric distributions are challenging since $\lambda$ not only controls symmetry but also spread, modes and tail behaviour. This paper is not meant to be a comprehensive review of this very active research topic, but rather a handy source for interested readers to the Bayesian analysis of skew-symmetric distributions. The paper is structured as follows. Section 2 recalls some basic concepts regarding the default choices for the prior distribution. Section 3 and Section 4 review the prior distributions for the parameters indexing the skew-normal and some other skew-symmetric models, respectively, while Section 5 reviews some of the literature on generalizations of the univariate skew-symmetric model. Final comments are provided in Section 6.

2. Default Prior Choices in Bayesian Statistics

The Bayesian approach to quantify uncertainty in statistical inference can be broken down into three steps [18]. The first step consists in choosing a joint probability distribution for observable and unobservable quantities, consistently with the available knowledge about the underlying scientific problem and the data collection process. The second step is to condition on the observed data, which is carried out by means of several computational techniques. The third step is assessing the model’s fit and interpreting the implications of the resulting posterior distribution. In this section we focus on the first step and review the most common default choices for the prior distribution. We use the following notation in the sequel: $\theta$ is our parameter of interest, $\pi \left(\theta \right)$ is the prior distribution, $\pi \left(\theta \right|t)$ is the posterior distribution given data information t, $p\left(t\right|\theta )$ is the data likelihood, $p(t,\theta )=p\left(t\right|\theta \left)\pi \right(\theta )$ is the joint distribution of t and $\theta$, and $p\left(t\right)$ is the marginal distribution of t. With this notation in hand, we of course have that the posterior equals

$$\pi \left(\theta \right|t)=\frac{\pi \left(\theta \right)p\left(t\right|\theta )}{p\left(t\right)}.$$

2.1. Jeffreys Priors

In Bayesian analysis there are situations in which the available prior information is too vague to be formalized into a probability distribution, too controversial to be acceptable in scientific communities or too complicated to allow for a reliable statistical analysis. Hence the need for priors with minimal effect on the posterior distribution, so that “the chosen prior would let the data speak for themselves” [19]. Reference analysis aims at an “objective” Bayesian solution to statistical inference in the same way as conventional statistical methods, where solutions only depend on model assumptions and observed data.

One of the earliest non-informative (objective) priors is the uniform prior for the Binomial proportion [20,21]. Unfortunately, this prior suffers from its lack of invariance under one-to-one reparameterization. Jeffreys’ prior is a non-informative prior which is invariant under one-to-one reparameterization and is proportional to the positive square root of the Fisher information associated with the parameter of interest. For regular models where asymptotic normality holds, the Jeffreys prior enjoys some optimality properties in the absence of nuisance parameters, but suffers from serious difficulties in the presence of nuisance parameters. As a first example, in the Neyman–Scott problem it leads to a strong inconsistency in Bayes estimation of the error variance [22]. As a second example, when estimating the product of two independent normal means, a circular symmetric prior was found to be superior to Jeffreys’ prior [23]. As a third example, Jeffreys himself supported the use of another prior for location-scale models.

2.2. Reference Priors

Intuitively, a reference prior for some real-valued parameter $\theta$ is a prior of the form $\pi \left(\theta \right)=\pi \left(\theta \right|\mathcal{T},\mathcal{P})$ which maximizes the missing information about $\theta$ within the class $\mathcal{P}$ of prior distributions compatible with the available prior knowledge $\mathcal{T}$ [19]. More formally, let D be a set of observations, generated by some random mechanism $p\left(D\right|\theta )$ that only depends on a real-valued parameter $\theta \in \mathrm{\Theta }$. Furthermore, let $t=t\left(D\right)\in \mathcal{T}$ be any sufficient statistic (which may be the complete data set D). In Shannon’s general information theory the amount of information $I^{\theta }\left[\mathcal{T},\pi \left(\theta \right)\right]$, which may be expected to be provided by D or equivalently by $t\left(D\right)$ about $\theta$, is

$$I^{\theta }\left[\mathcal{T},\pi \left(\theta \right)\right]={\int }_{\mathcal{T}}{\int }_{\mathrm{\Theta }}p(t,\theta )log\frac{p(t,\theta )}{p\left(t\right)\pi \left(\theta \right)}d\theta dt={\mathbb{E}}_t\left[{\int }_{\mathrm{\Theta }}\pi \left(\theta \right|t)log\frac{\pi \left(\theta \right|t)}{\pi \left(\theta \right)}d\theta \right],$$

(4)

which is the expected Kullback–Leibler divergence of the prior from the posterior (here ${\mathbb{E}}_t$ indicates that the expectation is taken on the t-part). The functional $I^{\theta }[\mathcal{T},\pi \left(\theta \right)]$ is concave, non-negative and invariant under one-to-one transformations of $\theta$. Lindley (1956) [24] and Bernardo (1979) [19] defined the reference prior as the prior maximizing (4). There are some situations where we need the asymptotic maximization of the above expectation, since for a fixed n, its maximization might lead to a discrete prior with finitely many jumps, which is hardly compatible with the concept of diffuse prior [25]. Ref. [19] proved that, in the absence of any nuisance parameter, Jeffreys’ prior yields the necessary maximization.

2.3. Matching Priors

Matching priors allow for posterior probability statements which have an interpretation as confidence statements in the sampling model. Matching priors aim at achieving a compromise between Bayesian and frequentist inference based on some order of approximation, thus providing default priors for routine use in Bayesian inference and possibly more palatable to frequentist statisticians. The concept of matching prior appears to have been proposed first by Lindley (1958) [26] and several matching priors have been proposed since, such as for example quantile matching priors, matching priors for distribution functions, highest probability density matching priors and matching priors associated with likelihood ratio statistics [27].

In this subsection we illustrate the approach to matching priors introduced by Welch and Peers in the seminal paper [28]. Suppose that $Y_1,\cdots ,Y_n$ are i.i.d. random variables with pdf $f\left(Y\right|\theta )$, where $\theta$ is real-valued. In addition, assume all the regularity conditions which allow to expand the posterior around the MLE ${\widehat{\theta }}_n$. Furthermore, for $0<\alpha <1$, let ${\theta }_{1-\alpha }^{\pi }(Y_1,\cdots ,Y_n)\equiv {\theta }_{1-\alpha }^{\pi }$ denote the $(1-\alpha )$-th asymptotic posterior quantile of $\theta$ based on the prior $\pi$, that is

$$P^{\pi }[\theta \le {\theta }_{1-\alpha }^{\pi }|Y_1,\cdots ,Y_n]=1-\alpha +O_p\left(n^{-r}\right)$$

for some $r>0$. If $r=1$, $\pi$ is called a first-order matching prior and if $r=3/2$ the prior $\pi$ will be a second-order probability matching prior. For instance, the Jeffreys prior is a first-order probability matching prior in the absence of nuisance parameters. We illustrate this appealing property with an example from [29]. Suppose that $Y_1,\cdots ,Y_n$ are i.i.d. with pdf $N(\theta ,1)$ and that $\pi \left(\theta \right)=1$ with $-\infty <\theta <\infty$. Then the posterior density $\pi \left(\theta \right|Y_1,\cdots ,Y_n)$ stems from the $N({\overline{Y}}_n,n^{-1})$. By considering $z_{1-\alpha }$ as the $100(1-\alpha )\%$ quantile of the $N(0,1)$ distribution, we have

$$P[\sqrt{n}(\theta -\overline{Y_n})\le z_{1-\alpha }|Y_1,\cdots ,Y_n]=1-\alpha =P[\sqrt{n}(\overline{Y_n}-\theta )\ge -z_{1-\alpha }|\theta ].$$

Therefore, the one-sided credible interval $\overline{Y_n}+z_{1-\alpha }/\sqrt{n}$ for $\theta$ has exact frequentist coverage probability $(1-\alpha )$. This exact matching does not always exist. However, if $Y_1,\cdots ,Y_n$ are i.i.d. random variables then

$$\widehat{{\theta }_n}|\theta \stackrel{a}{\sim }N\left(\theta ,{\mathcal{I}}^{-1}/n\right)$$

where $\mathcal{I}$ is the expected Fisher information and $\stackrel{a}{\sim }$ means asymptotically equivalent in distribution. Using the delta method we have

$$g\left({\widehat{\theta }}_n\right)|\theta \stackrel{a}{\sim }N\left[g\left(\theta \right),{\left(g^{\prime }\left(\theta \right)\right)}^2{\mathcal{I}}^{-1}/n\right].$$

Therefore, if $g^{\prime }\left(\theta \right)={\mathcal{I}}^{1/2}\left(\theta \right)$, then $g\left(\theta \right)={\int }_{\theta }{\mathcal{I}}^{1/2}\left(t\right)dt$ and $\left.\sqrt{n}\left[g\left(\widehat{{\theta }_n}\right)-g\left(\theta \right)\right]\right|\theta$ is asymptotically $N(0,1)$. In the absence of nuisance parameters, a first-order matching prior for $\theta$ is a solution of the differential equation

$$\frac{d}{d\theta }\left[\pi \left(\theta \right){\mathcal{I}}^{-1/2}\left(\theta \right)\right]=0,$$

so that the Jeffreys prior is the unique first-order matching prior, but it does not always hold for the second-order matching probability [27].

To obtain the second-order matching prior we need an asymptotic expansion of the posterior distribution function up to $O\left(n^{-1}\right)$ and the differential equation given by Mukerjee and Dey (1993) [30] and Mukerjee and Ghosh (1997) [31], that is

$$\frac{1}{3}\frac{d}{d\theta }\left[(\pi \left(\theta \right){\mathcal{I}}^{-2}\left(\theta \right)g_3\left(\theta \right)\right]+\frac{d^2}{d{\theta }^2}\left[(\pi \left(\theta \right){\mathcal{I}}^{-1}\left(\theta \right)\right]=0$$

(5)

where $g_3\left(\theta \right)=-\mathbb{E}\left[\left.\frac{d^{3}logf\left(Y_1\right|\theta )}{d{\theta }^3}\right|\theta \right]$. Jeffreys’ prior is the unique second-order matching prior if it satisfies (5), as it happens for the location-scale families: for ${\pi }_J\left(\theta \right)={\mathcal{I}}^{1/2}\left(\theta \right)$, (5) converts to

$$\frac{1}{3}\frac{d}{d\theta }\left[{\mathcal{I}}^{-3/2}\left(\theta \right)g_3\left(\theta \right)\right]+\frac{d^2}{d{\theta }^2}{\mathcal{I}}^{-1/2}\left(\theta \right)=0$$

which requires

$$\frac{1}{3}{\mathcal{I}}^{-3/2}\left(\theta \right)g_3\left(\theta \right)+\frac{d}{d\theta }{\mathcal{I}}^{-1/2}\left(\theta \right)$$

to be constant for all values of $\theta$. We refer the reader to [29] for more details on cases where, in the absence of nuisance parameters, there is not a second-order probability matching prior, and where, in the presence of nuisance parameters, there are first- and second-order matching priors and where there is not a second-order matching prior.

3. Prior Choices for the Skew-Normal Distribution

This section reviews the prior distributions proposed for Bayesian inference on the parameters of the skew-normal distribution: the reference prior by Liseo and Loperfido (2006) [32], the matching prior by Cabras et al. (2012) [33] and the informative prior by Canale and Scarpa (2013) [34].

3.1. The Reference Prior

Liseo and Loperfido (2006) [32] first proposed a default prior for the shape parameter of the location-scale-free (standard) skew-normal model $sn(z;\lambda )=2\varphi \left(z\right)\mathrm{\Phi }\left(\lambda z\right),z\in \mathbb{R}$. The associated Jeffreys prior is

$${\pi }_J\left(\lambda \right)\propto {\mathcal{I}}^{1/2}\left(\lambda \right)\mathrm{where}\mathcal{I}\left(\lambda \right)={\int }_{-\infty }^{\infty }2z^2\varphi \left(z\right)\frac{{\varphi }^2\left(\lambda z\right)}{\mathrm{\Phi }\left(\lambda z\right)}dz.$$

This prior is proper, symmetric about $\lambda =0$, decreasing in $\left|\lambda \right|$ and its tails are of order $O\left({\lambda }^{-3/2}\right)$. This prior is therefore suitable for testing the hypothesis of symmetry, which might be formalized in the skew-normal framework as $H_0:\lambda =0$ versus $H_1:\lambda \ne 0$. The same authors investigated the frequentist performances of this prior with simulated data, concluding that the Bayesian approach might be beneficial in easing some inferential difficulties of the frequentist approach for the standard skew-normal distribution.

Ref. [32] also considered a default Bayes analysis for the general scalar case (1), where $\lambda$ is the parameter of interest and the location parameter $\mu$ and the scale parameter $\sigma$ are the nuisance parameters. They are assumed to be independent of $\lambda$ and to have a normal-inverse gamma distribution:

$$\mu |\sigma \sim N\left({\mu }_0,\frac{{\sigma }^2}{\tau }\right)\mathrm{with}{\mu }_0\in \mathbb{R},\tau >0,{\sigma }^{-2}\sim Gamma(\alpha ,\beta )$$

where $Gamma(\alpha ,\beta )$ is a Gamma distribution with parameters $\alpha ,\beta >0$. The default prior $\pi (\mu ,\sigma )\propto {\sigma }^{-1}$ is a limiting case and is the conditional reference prior for $(\mu ,\sigma )$ given $\lambda$. These assumptions allow for a closed-form expression of the marginal likelihood for $\lambda$. The proposed method has been successfully applied to the infamous “frontier” dataset (see http://azzalini.stat.unipd.it/SN/frontier.dat), where the maximum likelihood estimate of the skewness parameter $\lambda$ is infinite.

Bayes and Branco (2007) [35] highlighted the advantages of the Bayesian approach and proposed two priors. They considered the stochastic representation (3) of the skew-normal distribution and, following the Bayes–Laplace rule, chose the uniform distribution on the interval $\left[-1,1\right]$ as a prior for $\lambda /\sqrt{1-{\lambda }^2}$, thus leading to a $t(0,0.5,2)$ distribution as prior for $\lambda$, where $t(a,b,c)$ denotes the Student t distribution centered in $a\in \mathbb{R}$ with scale $b>0$ and $c>0$ degrees of freedom, which is a non-vague and non-subjective prior. They further proposed the tractable approximation $t(0,{\pi }^2/4,1/2)$ for the Jeffreys prior from [32]. They motivated it by the following approximation (see [36]):

$$\frac{1}{\pi }\frac{\varphi \left(x\right)}{\sqrt{\mathrm{\Phi }\left(x\right)(1-\mathrm{\Phi }(x\left)\right)}}\approx \frac{1}{\sqrt{2\pi }(\pi /2)}exp\left(-\frac{2x^2}{{\pi }^2}\right).$$

3.2. The Matching Prior

Cabras et al. (2012) [33] proposed another approach towards Bayesian inference about the shape parameter of the skew-normal distribution. It is based on a pseudo-likelihood function and a matching prior distribution for the shape parameter when location and scale parameters are unknown. First, they derive the marginal likelihood

$$\begin{array}{c}\hfill L_m\left(\lambda \right)={\int }_0^{\infty }{\int }_{-\infty }^{+\infty }\frac{L(\lambda ,\eta )}{\sigma }d\mu d\sigma ,\end{array}$$

(6)

where $L(\lambda ,\eta )={\prod }_{i=1}^{n}sn(y_i;\eta ,\lambda )$ is the skew-normal likelihood function, $\eta =(\mu ,\sigma )$ is the nuisance parameter and ${\sigma }^{-1}$ is the right-invariant Haar measure on the location-scale group of transformations, whose action on the parameter space leaves $\lambda$ unchanged. By considering the fact that the marginal likelihood (6) can be approximated by the modified profile likelihood $L_{mp}\left(\lambda \right)$ of [37] since $L_m\left(\lambda \right)=L_{mp}\left(\lambda \right)(1+O\left(n^{-1}\right))$ (see [38]) and by invoking results about the use of pseudo-likelihood functions in Bayesian analysis, the matching prior $\pi \left(\lambda \right)$ is simply proportional to the square root of the inverse of the asymptotic variance of the MLE of $\lambda$. Based on Ventura et al. (2009) [39], the matching prior for $\lambda$ corresponding to (6) is

$$\pi \left(\lambda \right)\propto I_{\lambda \lambda .\eta }{(\lambda ,{\widehat{\eta }}_{\lambda })}^{1/2},$$

(7)

where ${\widehat{\eta }}_{\lambda }$ is the constrained MLE of $\eta$ for a given $\lambda$ and $I_{\lambda \lambda .\eta }(\lambda ,\eta )=I_{\lambda \lambda }(\lambda ,\eta )-I_{\lambda \eta }(\lambda ,\eta )I_{\eta \eta }{(\lambda ,\eta )}^{-1}{I}_{\eta \lambda }(\lambda ,\eta )$ is the partial information with $I_{\lambda \lambda }(\lambda ,\eta )$, $I_{\lambda \eta }(\lambda ,\eta )$, $I_{\eta \eta }(\lambda ,\eta )$ and $I_{\eta \lambda }(\lambda ,\eta )$ blocks of the expected Fisher information

$$I(\lambda ,\eta )=\left[\begin{array}{cc}{I}_{\lambda \lambda }(\lambda ,\eta )& I_{\lambda \eta }(\lambda ,\eta )\\ I_{\eta \lambda }(\lambda ,\eta )& I_{\eta \eta }(\lambda ,\eta )\end{array}\right].$$

For the interested reader we provide the detailed quantities of this matrix:

$$\begin{array}{c}{I}_{\lambda \lambda }(\lambda ,\eta )=a_2,I_{\lambda \mu }(\lambda ,\eta )=\frac{1}{\sigma }\left(\frac{b}{A^{3/2}}-\lambda a_1\right),I_{\lambda \sigma }(\lambda ,\eta )=-\frac{\lambda a_2}{\sigma },I_{\mu \mu }(\lambda ,\eta )=\frac{(1+{\lambda }^2{a}_0)}{{\sigma }^2},\\ \\ I_{\mu \sigma }(\lambda ,\eta )=\frac{1}{{\sigma }^2}\left[\frac{b\lambda (1+2{\lambda }^2)}{A^{3/2}}+{\lambda }^2{a}_1\right],I_{\sigma \sigma }(\lambda ,\eta )=\frac{2+{\lambda }^2{a}_2}{{\sigma }^2},\end{array}$$

where $b=\sqrt{\frac{2}{\pi }},A=1+{\lambda }^2$, and $a_i=\mathbb{E}\left[Z^i{\left(\frac{\varphi \left(\lambda Z\right)}{\mathrm{\Phi }\left(\lambda Z\right)}\right)}^2\right],i=0,1,2,$ with Z following the standard skew-normal with parameter $\lambda$. However, the prior (7) might be data dependent because of the presence of ${\widehat{\eta }}_{\lambda }$. A prior for $\lambda$ which does not suffer from this problem is proportional to

$$\begin{array}{c}\sqrt{\frac{a_2{A}^2[\pi (1+a_0{\lambda }^4)+{\lambda }^2(\pi (1+a_0)-4)]+2\sqrt{2\pi }{a}_1\lambda A^{3/2}-\pi a_1^2{\lambda }^2{A}^3-2}{\pi A^3[2+{\lambda }^2(2a_0+a_2)+{\lambda }^4(a_0{a}_2-a_1^2)]-2{(\lambda +2{\lambda }^3)}^2-2\sqrt{(}2\pi )a_1{\lambda }^3\sqrt{A}(1+3{\lambda }^2+2{\lambda }^4)}}.\end{array}$$

This prior is proper, symmetric at the origin and with tails of order $O\left({\lambda }^{-3/2}\right)$. It also compensates for the possible monotonicity of the modified profile likelihood (6) and possesses good frequentist properties.

3.3. The Informative Priors of Canale and Scarpa (2013)

Canale and Scarpa (2013) [34] discuss two informative priors for the skewness parameter of the skew-normal distribution. Their study is motivated by an interesting data set on marks referring to first-year undergraduate students for the program in Economics at the University of Padua. The skew-normal model is implemented on students’ grades in the first mandatory class of Statistics. Making inference on the grades of the previous years shows that the distribution of Statistics grades is skewed to the right around a certain mean, which explains why they need informative priors for their endeavour.

The first prior is the normal density with hyperparameters reflecting prior beliefs about the expectation and variance of $\lambda$ in order to center the prior on a particular guess for $\lambda$. The resulting posterior belongs to the family of unified skew-normal (SUN) distributions, introduced in [40]. The explicit expressions for the mean and the variance of the posterior are not very tractable but they allow for a simple interpretation. The second informative prior is itself a skew-normal, motivated by the distribution of grades of university examinations [41]. The skew-normal prior includes location and scale hyperparameters as well as a skewness hyperparameter reflecting the beliefs on the direction of skewness. The posterior distribution also belongs to the class of SUN distributions. The authors set the location hyperparameter of the skew-normal prior equal to zero in order to have a rough prior information only on the skewness side of the distribution of the data: considering negative or positive values for the skewness hyperparameter puts more prior mass on the positive or negative semi-axis. In both cases the resulting posteriors are intractable, but the SUN parametrization eases efficient sampling methods for posterior computation via Markov Chain Monte Carlo (MCMC). For both prior choices for $\lambda$, they have specified an independent normal inverse gamma distribution for the location and scale parameters. To perform the related Bayesian inference, the authors have presented an algorithm to simulate the full conditional distribution of the skewness parameter $\lambda$ given the location and scale parameter. This algorithm uses a Gibbs sampler for the stochastic representation of the SUN model. To get the posterior, the authors introduced normal latent variables, say ${\eta }_1,\dots ,{\eta }_n$. Conditionally on these latent variables, the generic i-th observation will be normally distributed with a specific mean and variance. This way of constructing the Gibbs sampler leads to conjugacy for the location and scale parameters. For the detailed computations we refer the reader to [34]. This sampling method is useful in MCMC methods to approximate the posterior distribution.

We also wish to mention that generally the MCMC method in Bayesian statistics bears a particular importance in model selection. Suppose we have a set of models reflecting competing hypotheses about the underlying data set, where each model is characterized by a specific vector of parameters of interest. From the Bayesian viewpoint, these models are compared pairwise through their Bayes factor which is the ratio of relative marginal likelihoods. Obviously, finding the marginal likelihood is often not feasible in particular analytically. We refer the reader to [42] and references therein for estimation methods of the marginal likelihood, specifically in general non-nested models.

4. Prior Choices for Other Skew-Symmetric Distributions

There exists a wide literature on the Bayesian analysis of skew-symmetric distributions different form the skew-normal. Azzalini (1986) [6] and Naranjo et al. (2012) [43] provided a Bayesian analysis of a skewed exponential power distribution. This family includes the symmetric exponential distribution as well as the skew-normal distribution, and provides flexible distributions with lighter and heavier tails. Interestingly, this family of densities can fit each tail separately. Hossianzadeh and Zare (2016) [44] estimated the parameter of the discrete skewed Laplace distribution by an empirical Bayesian analysis and compared it with the maximum likelihood approach. In what follows, we will first consider the popular skew-t distribution and then focus on two general approaches for skew-symmetric densities.

4.1. Jeffreys’ Prior for Skew-t Distributions

Skew-t distributions are the best-known alternatives to skew-normal ones, due to their flexibility: they can model any level of skewness and excess kurtosis. However, they pose some further inferential problems, which we illustrate in the simpler case of the Student t distribution with known location and scale parameters. Ref. [45] discussed that the likelihood function approaches infinity when the degrees of freedom go to zero, and showed that the supremum of the likelihood function may be achieved when the degrees of freedom go to infinity. There have been several frequentist attempts to solve the inferential problems of the skew-t distribution with all parameters unknown. Sartori (2006) [46] used the modified score function, which requires the degrees of freedom to be fixed. Azzalini and Genton (2008) [10] proposed a deviance approach which is only partially satisfactory, since its implementation might not be straightforward. We illustrate the problem with the univariate skew-t distribution. The deviance approach replaces the boundary maximum likelihood estimate of $\left(\lambda ,\upsilon \right)$ by the smallest vector $\left({\lambda }_0,{\upsilon }_0\right)$ for which the null hypothesis $H_0:\left(\lambda ,\upsilon \right)=\left({\lambda }_0,{\upsilon }_0\right)$ is not rejected. The deviance approach assumes that such smallest vector exists but neither theoretical results nor simulation studies support this assumption. For these reasons we cannot exclude the existence of samples admitting two vectors $\left({\lambda }_a,{\upsilon }_a\right)$ and $\left({\lambda }_b,{\upsilon }_b\right)$ satisfying ${\lambda }_a>{\lambda }_b$ and ${\upsilon }_a<{\upsilon }_b$ for which the hypotheses $H_0:\left(\lambda ,\upsilon \right)=\left({\lambda }_a,{\upsilon }_a\right)$ and $H_0:\left(\lambda ,\upsilon \right)=\left({\lambda }_b,{\upsilon }_b\right)$ are not rejected, while no vector $\left({\lambda }_0,{\upsilon }_0\right)$ exists satisfying $\left({\lambda }_0,{\upsilon }_0\right)<\left({\lambda }_a,{\upsilon }_a\right)$, $\left({\lambda }_0,{\upsilon }_0\right)<\left({\lambda }_b,{\upsilon }_b\right)$ and for which the null hypothesis $H_0:\left(\lambda ,\upsilon \right)=\left({\lambda }_0,{\upsilon }_0\right)$ is not rejected. We reckon that this situation is likely to happen, given that when either of the parameters $\lambda$ or $\upsilon$ is large the shape of the skew-t density function remains almost unchanged if one parameter is substantially increased while the other is substantially decreased.

Given these shortcomings, Branco, Genton and Liseo (2012) [47] studied Bayesian analysis for various forms of skew-t distributions. Denoting by $\nu >0$ the degrees of freedom parameter, they first considered skew-t densities of the form

$$2t\left(z\right|\nu \left)T\right(\lambda z\left|\nu \right),z\in \mathbb{R},$$

where $t(·|\nu )$ and $T(·|\nu )$ are the pdf and the cdf of a Student t distribution with $\nu$ degrees of freedom. The corresponding Jeffreys prior for the skewness parameter $\lambda$ when $\nu$ is known and finite is

$${\pi }_J\left(\lambda \right|\nu )\propto \sqrt{{\int }_{-\infty }^{\infty }{z}^{2}t\left(z\right|\nu )\frac{t^2\left(\lambda z\right|\nu )}{T\left(\lambda z\right|\nu )}dz}.$$

It is proper, symmetric about zero and with tails of order $O\left({\lambda }^{-3/2}\right)$. The same authors further investigated the case of the skew-t distribution of [5] with pdf

$$2t\left(z\right|\nu )T\left(\left.\lambda z\sqrt{\frac{\nu +1}{\nu +z^2}}\right|\nu +1\right),z\in \mathbb{R}.$$

The corresponding Jeffreys prior for $\lambda$ for known and finite $\nu$ is

$${\pi }_J\left(\lambda \right|\nu )\propto \sqrt{{\int }_0^{\infty }\frac{z^{2}t\left(z\right|\nu )t^2\left(\lambda z\right|\nu )}{(\nu +z^2)T\left(\left.\lambda z\sqrt{\frac{\nu +1}{\nu +z^2}}\right|\nu +1\right)\left[1-T\left(\left.\lambda z\sqrt{\frac{\nu +1}{\nu +z^2}}\right|\nu +1\right)\right]}dz}.$$

Again, this prior is proper, symmetric about zero and the tails are of order $O\left({\lambda }^{-3/2}\right)$.

4.2. Jeffreys Prior for General Skew-Symmetric Models

Rubio and Liseo (2014) [48] investigated the Jeffreys prior for the skewness parameter of a general class of scalar skew-symmetric models. The Jeffreys prior cannot be used for some skew-symmetric models at $\lambda =0$ because of the singularity of the Fisher information at this point; see the Introduction for details about this issue. They showed that under mild conditions, including knowledge of location and scale parameters, the Jeffreys prior of the skewness parameter $\lambda$ in the skew-symmetric model is proper, symmetric about zero and tails are of order of $O\left(\right|\lambda {|}^{-3/2})$. They used these results to construct the independence Jeffreys prior for the model including the location and scale parameters: it is the product of the Jeffreys prior of each parameter, under the assumption that the remaining parameters are held fixed. The same authors also provided sufficient conditions for the existence of the posterior distribution and briefly discussed the existence of a proper independence Jeffreys prior for the skew-logistic model described in [7] and gave a Student t approximation to that prior.

The approach in [48] might be sketched as follows. The Fisher information for the shape parameter in (2) is

$$I\left(\lambda \right)={\int }_{-\infty }^{\infty }2z^{2}f\left(z\right)\frac{g^2\left(\lambda z\right)}{G\left(\lambda z\right)}dz={\int }_0^{+\infty }2z^{2}f\left(z\right){\left[\pi h\left(\lambda z\right)\right]}^{2}dz$$

where

$$h\left(z\right)=\frac{1}{\pi }\frac{g\left(z\right)}{\sqrt{G\left(z\right)\left[1-G\left(z\right)\right]}}$$

(8)

and therefore in this case the Jeffreys prior for $\lambda$ is ${\pi }_J\left(\lambda \right)\propto \sqrt{I\left(\lambda \right)}$. The first step for approximating the function $h\left(z\right)$ in (8) is to see that the transformed random variable $Z=G^{-1}\left(X\right)\sim h\left(z\right)$, where the random variable $X\sim Beta(1/2,1/2)$ and G is a cumulative distribution function of an absolutely continuous symmetric random variable, that is $G(-z)=1-G\left(z\right)$ for all $z\in \mathbb{R}$. The corresponding cumulative distribution function H is

$$H\left(z\right)=\frac{2}{\pi }arcsin\left[\sqrt{G\left(z\right)}\right].$$

An approximation of h in terms of g might then be achieved by choosing the scale parameter $\sigma$ such that

$$h\left(z\right)\approx \frac{1}{\sigma }g\left(\frac{z}{\sigma }\right).$$

The quality of this approximation depends on the thickness of the tails of g. The authors illustrate this point by considering the case of $g\left(z\right)$ having a Student t distribution with $\nu$ degrees of freedom and comparing the approximations using quantiles. Alternatively, $\sigma$ might be chosen to minimize the Kullback-Leibler divergence between $h\left(z\right)$ and $g(z/\sigma )/\sigma$. Ref. [32] approximated the Jeffreys prior using the parameterization $\delta =\lambda /\sqrt{1+{\lambda }^2}$. They also proposed to use the symmetric Beta prior $Beta(\tau ,\tau )$ for $\beta =\left(\delta -1\right)/2$, thus leading to the Student t prior for $\lambda$

$${\pi }_J\left(\lambda \right)=\frac{\mathrm{\Gamma }\left(2\tau \right)}{2^{2\tau -1}\mathrm{\Gamma }\left(\tau \right)\mathrm{\Gamma }\left(\tau \right){(1+{\lambda }^2)}^{\tau +1/2}}$$

which reduces to the Cauchy distribution for $\tau =0.5$, see [32].

4.3. Distance-Based Priors

As already mentioned, the shape parameter $\lambda$ does not only impact the skewness in skew-symmetric models, but also the mean, the variance, the modes and the kurtosis. Dette et al. (2018) [49] dealt with this issue by assigning a prior distribution on the perturbation effect of the skewness parameter, quantified by the Total Variation distance between the symmetric density f and its skew-symmetric counterpart $2f\left(x\right)G\left[\lambda w\left(x\right)\right]$, where w is an odd function, rather than on the skewness parameter itself. The rationale behind this choice is that such a distance is more easily interpretable than the parameter $\lambda$, and hence informative as well as non-informative priors can more readily be found for the effect of $\lambda$ than for $\lambda$ itself. The Total Variation distance between two probability measures $\mu (·)$ and $\nu (·)$ on $\mathbb{R}$ is

$$d_{TV}(\mu ,\nu )=\underset{A\subset \mathbb{R}}{sup}|\mu \left(A\right)-\nu \left(A\right)|,$$

that is, the maximum difference between the probabilities assigned to the same event by the two measures. It is bounded between zero and one, $0\le d_{TV}(\mu ,\nu )\le 1$. The Total Variation distance between f and $2f\left(x\right)G\left[\lambda w\left(x\right)\right]$ is given by

$$d_{TV}(f,G|\lambda )=\frac{1}{2}{\int }_{\mathbb{R}}|2G\left[\lambda w\left(x\right)\right]-1|f\left(x\right)dx.$$

The symmetry of G implies that $d_{TV}(f,G|\lambda )$ is not a one-to-one function of the parameter $\lambda$: $d_{TV}(f,G|\lambda )=d_{TV}(f,G|-\lambda )$. It is therefore convenient to use $M_{TV}\left(\lambda \right)=sign\left(\lambda \right)d_{TV}(f,G|\lambda )$ as a measure of perturbation, due to its appealing properties: $M_{TV}\left(0\right)=0$, the largest/smallest value of $M_{TV}\left(\lambda \right)$ is $\pm 0.5$ (attained when $\lambda \to \pm \infty$) so that $M_{TV}\left(\lambda \right)\in (-1/2,1/2)$, and $M_{TV}\left(\lambda \right)$ is invariant under affine transformations. Moreover, $M_{TV}\left(\lambda \right)=0.5\left[1-2S_{f;G}(0;\lambda )\right]$, where $S_{f;G}$ is the cdf associated with $s_{f;G}$, which means that $M_{TV}\left(\lambda \right)$ is a re-scaling of the difference between the mass cumulated on either side of the origin for a fixed choice of f and G by the distribution $S_{f;G}$. In summary, $M_{TV}\left(\lambda \right)$ quantifies the impact of the parameter $\lambda$ on the relocation of the probability mass on either side of the symmetry center of f.

The proposed measure $M_{TV}$ allows to build both informative and non-informative priors for the perturbation parameter $\lambda$ in skew-symmetric models. Since $M_{TV}\in (-1/2,1/2)$ is an injective function of $\lambda$ any prior option for $\lambda$ induces a proper prior. Ref. [49] proposed, for $M_{TV}\left(\lambda \right)$, Beta priors with support on the interval $(-1/2,1/2)$ and with density

$$\frac{1}{Beta(\alpha ,\beta )}{\left(u+\frac{1}{2}\right)}^{\alpha -1}{\left(\frac{1}{2}-u\right)}^{\beta -1},$$

where $Beta(\alpha ,\beta )$ is a beta function with hyperparameters $\alpha ,\beta >0$. This induces on $\lambda$ the proper prior with pdf

$$\pi \left(\lambda \right|\alpha ,\beta )=\frac{1}{Beta(\alpha ,\beta )}{\left(M_{TV}\left(\lambda \right)+\frac{1}{2}\right)}^{\alpha -1}{\left(\frac{1}{2}-M_{TV}\left(\lambda \right)\right)}^{\beta -1}\frac{d}{d\lambda }{M}_{TV}\left(\lambda \right).$$

Priors of this type are called Beta Total Variation priors and are denoted by $BTV(\alpha ,\beta )$; they are flexible, interpretable and lead to tractable posterior distributions. The behaviour of the prior $BTV(\alpha ,\beta )$ is well illustrated by the special case $BTV(1,1)$, that is a uniform prior giving equal probability mass to any pair of subintervals of equal length belonging to the support. If g is a bounded pdf and ${\int }_0^{1}w\left(x\right)f\left(x\right)dx<\infty$, then $BTV(1,1)$ is well-defined for all $\lambda$ and is given by

$${\pi }_{TV}\left(\lambda \right|1,1)=2{\int }_0^{\infty }w\left(x\right)f\left(x\right)g\left[\left(\lambda w\right(x)\right]dx.$$

(9)

Since (9) does not have a closed-form, the authors proposed to approximate it by a Cauchy distribution centered at the origin and with scale parameter equal to $0.92$. A Monte Carlo study showed that the proposed non-informative prior induces a posterior distribution with good frequentist properties and similar to those of the Jeffreys prior.

4.4. Prior Choices in the Presence of Kurtosis Parameters

Rubio and Steel (2015) [50] have proposed a general strategy for constructing weakly informative priors for kurtosis parameters by assigning a uniform prior to a bounded measure of kurtosis applied to the symmetric baseline density $f(·|\delta )$ in which $\delta$ is the tail parameter and is a one-to-one function of the kurtosis. This methodology, used in [51], induces a proper prior on $\delta$ that can be interpreted as weakly non-informative prior, in that it assigns a flat prior on a function that incorporates the influence of the parameter $\delta$ on the shape of the density. This prior can be coupled with the Jeffreys prior for the skewness parameter in order to produce a joint prior for $(\delta ,\lambda )$ in skew-symmetric models by using $p(\lambda ,\delta )=p\left(\lambda \right|\delta \left)p\right(\delta )$ where

$$p\left(\lambda \right|\delta )\propto \sqrt{{\int }_0^{\infty }\frac{x^{2}f\left(x\right|\delta )g{\left(\lambda x\right)}^2}{G\left(\lambda x\right)[1-G(\lambda x\left)\right]}dx}.$$

For each value of $\delta$ the tails are of order $O\left(\right|\lambda {|}^{-3/2})$. A simulation study showed that this prior produces a posterior density with good frequentist properties.

5. Overview on Related Topics

So far this paper has focussed on the univariate case without covariates. This section briefly reviews some of the literature on more general settings related to skew-symmetric distributions.

Ref. [52] proposed a general population Monte Carlo algorithm in order to conduct a full Bayesian analysis of the multivariate skew-normal distribution, also in the presence of constrained parameters. Since the prior distribution approximates the actual reference prior for the shape parameter vector, this approach can be considered as a weakly informative prior. In addition, a generalization to the matrix variate regression model with skew-normal error is also provided.

Ref. [53] carried out a Bayesian analysis of a p-variate skew-t distribution by providing a new parameterization, considering a set of non-informative priors and a sampler designed to obtain the posterior model based on the parameters. The methodology can be extended to multivariate regression models with skewed errors and also stochastic frontier models.

Ref. [54] investigated the time series of electricity spot prices, which exhibit heavy tails and skewness. The authors conducted Bayesian inference on the multivariate skew-t distribution by putting a normal prior on the multi-dimensional skewness parameter.

Ref. [51] proposed a general non-informative structure for regression models with skew-symmetric errors, showed that under some mild conditions the resulting posterior distribution is proper and extended the results to the cases where the response variables are censored. The authors also investigated accelerated failure time models, which are relevant in survival analysis. Different prior distributions have been implemented on the skewness parameter of the skew-normal model including a Jeffreys prior, a matching prior, an informative prior and a uniform, noninformative prior on the parameter $\delta =\lambda /\sqrt{1+{\lambda }^2}$, leading to the proper prior

$$\pi \left(\lambda \right)\propto \frac{1}{{(1+{\lambda }^2)}^{3/2}}.$$

Ref. [55] used finite mixtures of skewed distributions to model flow cytometry data, in order to describe their skewness, kurtosis and heterogeneity. The authors developed Bayesian inference based on data augmentation and MCMC sampling using the aforementioned model. Data augmentation in this case is based on stochastic representation of the skew-normal distribution in terms of a random-effects model with truncated normal random effects. Finite mixtures of skew-normals provide a Gibbs sampling scheme that can be drawn from standard densities only. The same MCMC scheme is extended to mixtures of skew-t distributions by considering the skew-t distribution as a scale mixture of skew-normals.

Ref. [56] proposed a new class of distributions by introducing a skewness parameter in multivariate elliptically symmetric densities. This class of densities contains many standard families such as skew-t and skew-normal distributions. They condition on some unobserved variables commonly used in regression modelling and model stock market returns, security options or risky financial assets subject to shocks. Within the Bayesian realm, they show inter alia that there exist posterior distributions and moments for regression coefficients derived under improper priors.

Linear mixed models (LMM) are commonly used to analyze repeated measure data since they allow for flexible modelling of within-subject correlations. Mostly LMM for continuous responses assume that the random effects and the within-subjects errors are normally distributed, which can be unrealistic. Ref. [57] considered the less restrictive assumption of skew-normality and Bayesian inference based on prior distributions very similar to non-informative ones. They illustrated the proposed approach with the Framingham cholesterol data, obtained from a well-known long-term study aimed at investigating the relationship between various risk factors and diseases and to characterize the natural history of chronic circulatory diseases.

6. Discussion

In this paper we have provided an overview on the various proposals of Bayesian inference within skew-symmetric models. We hope that the reader will consider it as a helpful tool and source of information on this research domain. We refer the interested reader to the simulation study and real data analysis of the recent paper [49] for a performance comparison between several of the above described prior proposals. Digging further into performance comparisons is a promising research task in order to get a more complete picture on which prior to ideally use in which situation when dealing with skew-symmetric distributions.

A referee remarked that in the general case the posterior distribution is multimodal and it is therefore necessary to impose some conditions ensuring unimodality. Log-concavity implies unimodality and it is preserved under convolution, marginalization, affine transformations and conditioning. For example, the assumption that the joint distribution of the parameter and the observations is log-concave implies that the posterior distribution is log-concave, too. We illustrate this point with a simple example. Assume that we sampled just one observation from a standard skew-normal distribution and that our prior distribution on the shape parameter is standard normal: $f\left(z|\lambda \right)=2\varphi \left(z\right)\mathrm{\Phi }\left(\lambda z\right)$ and $\pi \left(\lambda \right)=\varphi \left(\lambda \right)$. The joint distribution of the observation and the parameter is then $f\left(z,\lambda \right)=2\varphi \left(z\right)\varphi \left(\lambda \right)\mathrm{\Phi }\left(\lambda z\right)$. A little calculation shows that $f\left(z,\lambda \right)$ is log-concave. Hence, without further calculation, we know that $f\left(\lambda |z\right)$ is log-concave and hence unimodal, too. MAP estimates are then uniquely defined and can be easily derived by noticing that the posterior distribution is skew-normal: $\pi \left(\lambda |z\right)=2\varphi \left(\lambda \right)\mathrm{\Phi }\left(\lambda z\right)$. Ref. [58] provides a thorough review of the literature on log-concavity, both in the univariate and in the multivariate case. Would the assumption of log-concavity be too restrictive, one could resort to other multivariate generalizations of unimodality, as for example block-unimodality, which already appeared in the Bayesian literature ([59]).