The Flexible Gumbel Distribution: A New Model for Inference about the Mode

Abstract

A new unimodal distribution family indexed via the mode and three other parameters is derived from a mixture of a Gumbel distribution for the maximum and a Gumbel distribution for the minimum. Properties of the proposed distribution are explored, including model identifiability and flexibility in capturing heavy-tailed data that exhibit different directions of skewness over a wide range. Both frequentist and Bayesian methods are developed to infer parameters in the new distribution. Simulation studies are conducted to demonstrate satisfactory performance of both methods. By fitting the proposed model to simulated data and data from an application in hydrology, it is shown that the proposed flexible distribution is especially suitable for data containing extreme values in either direction, with the mode being a location parameter of interest. Using the proposed unimodal distribution, one can easily formulate a regression model concerning the mode of a response given covariates. We apply this model to data from an application in criminology to reveal interesting data features that are obscured by outliers.

1. Introduction

The mean, median, and mode are the three most commonly used measures of central tendency of data. When data contain outliers that cause heavy tails or are potentially skewed, the mode is a more sensible representation of the central location of data than the mean or median. The timely review on mode estimation and its application by Chacón [1] and references therein provide many examples in various fields of research where the mode serves as a more informative representative value of data. Most existing methods developed to draw inference for the mode are semi-/non-parametric in nature, starting from early works on direct estimation in the 1960s [2,3,4] to more recent works based on kernel density estimation [5] and quantile-based methods [6,7]. Two main factors contribute to the enduring preference for semi-/non-parametric methods for mode estimation, despite the typically less straightforward implementation and lower efficiency compared to parametric counterparts. First, parametric models often impose strict constraints on the relationship between the mode and other location parameters, which may not hold in certain applications. Second, very few existing named distribution families that allow the inclusion of both symmetric and asymmetric distributions in the same family can be parameterized so that it is indexed by the mode as the location parameter along with other parameters, such as shape or scale parameters. In this study, we alleviate concerns raised by both reasons that discourage the use of parametric methods for mode estimation by formulating a flexible distribution indexed via the (unique) mode and parameters controlling the shape and scale.

When it comes to modeling heavy-tailed data, the Gumbel distribution [8] is arguably one of the most widely used models in many disciplines. Indeed, as a case of the generalized extreme value distribution [9], the Gumbel distribution for the maximum (or minimum) is well-suited for modeling extremely large (or small) events that produce heavy-tailed data. For example, it is often used in hydrology to predict extreme rainfall and flood frequency [10,11,12]. In econometrics, Gumbel distribution plays an important role in modeling extreme movements of stock prices and large changes in interest rates [13,14]. The Gumbel distribution is indexed by the mode and a scale parameter, and thus is convenient for mode estimation. However, the Gumbel distribution for the maximum (or minimum) is right-skewed (or left-skewed) with the skewness fixed at around $1.14$ (or $-1.14$), and the kurtosis fixed at 5.4 across the entire distribution family. Thus, it may be too rigid for scenarios where the direction and extremeness of outliers presented in data are initially unclear, or when the direction and level of skewness are unknown beforehand. Constructions of more flexible distributions that overcome these limitations have been proposed. In particular, Cooray [15] applied a logarithmic transformation on a random variable following the odd Weibull distribution to obtain the so-called generalized Gumbel distribution that includes the Gumbel distribution as a subfamily. But the mode of the generalized Gumbel distribution is not indexed by a location parameter, or an explicit function of other model parameters. Shin et al. [16] considered mixture distributions with one of the components being the Gumbel distribution and the other component(s) being Gumbel of the same skewness direction or a different distribution, such as the gamma distribution. Besides the same drawback pointed out for the generalized Gumbel distribution, it is difficult to formulate a unimodal distribution following their construction of mixtures, and thus their proposed models are unsuitable when unimodality is a feature required to make inferring the mode meaningful, such as in a regression setting, as in modal regression [5,17,18].

With heavy-tailed data in mind and the mode as the location parameter of interest, we construct a new unimodal distribution that does not impose stringent constraints on how the mode relates to other central tendency measures, while allowing a range of kurtosis wide enough to capture heavy tails at either direction, as well as different degrees and directions of skewness. This new distribution, called the flexible Gumbel (FG) distribution, is presented in Section 2, where we study properties of the distribution and discuss the identifiability of the model. We present a frequentist method and a Bayesian method for estimating parameters in the FG distribution in Section 3. The finite sample performance of these methods is inspected in a simulation study in Section 4, followed by an application of the FG distribution in hydrology in Section 5. Section 6 demonstrates fitting a modal regression model based on the FG distribution to data from a criminology study. Section 7 highlights the contributions of our study and outlines future research directions.

2. The Flexible Gumbel Distribution

The probability density function (pdf) of the Gumbel distribution for the maximum is given by

$$\begin{array}{cc}\hfill f(x;\theta ,\sigma )& =\frac{1}{\sigma }\exp \left\{-\frac{x-\theta }{\sigma }-\exp \left(-\frac{x-\theta }{\sigma }\right)\right\},\hfill \end{array}$$

(1)

where $\theta$ is the mode and $\sigma >0$ is a scale parameter. The pdf of the Gumbel distribution for the minimum with mode $\theta$ and a scale parameter $\sigma$ is given by

$$\begin{array}{cc}\hfill f(x;\theta ,\sigma )& =\frac{1}{\sigma }\exp \left\{\frac{x-\theta }{\sigma }-\exp \left(\frac{x-\theta }{\sigma }\right)\right\}.\hfill \end{array}$$

(2)

We define a unimodal distribution for a random variable Y via a mixture of the two Gumbel distributions specified by (1) and (2) that share the same mode $\theta$ while allowing different scale parameters, ${\sigma }_1$ and ${\sigma }_2$, in the two components. We call the resultant distribution the flexible Gumbel distribution FG for short with the pdf given by

$$\begin{array}{cc}\hfill f\left(y\right)=& w\times \frac{1}{{\sigma }_1}\exp \left\{-\frac{x-\theta }{{\sigma }_1}-\exp \left(-\frac{x-\theta }{{\sigma }_1}\right)\right\}+\hfill \\ \hfill & (1-w)\times \frac{1}{{\sigma }_2}\exp \left\{\frac{x-\theta }{{\sigma }_2}-\exp \left(\frac{x-\theta }{{\sigma }_2}\right)\right\},\hfill \end{array}$$

(3)

where $w\in [0,1]$ is the mixing proportion parameter. Henceforth, we state that $Y\sim \mathrm{FG}(\theta ,{\sigma }_1,{\sigma }_2,w)$ if Y follows the distribution specified by the pdf in (3).

For each component distribution of FG, the mean and median are both some simple shift of the mode, with each shift solely determined by the scale parameter. Because the two components in (3) share a common mode $\theta$, the mode of Y is also $\theta$, and thus the FG distribution is convenient to use when one aims to infer the mode as a central tendency measure, or to formulate parametric modal regression models [19,20,21]. One can easily show that the mean of Y is $E\left(Y\right)=w(\theta +{\sigma }_1\gamma )+(1-w)(\theta -{\sigma }_2\gamma )=\theta +\{w({\sigma }_1+{\sigma }_2)-{\sigma }_2\}\gamma$, where $\gamma \approx 0.5772$ is the Euler–Mascheroni constant. Thus, the discrepancy between the mode and the mean of FG depends on three other parameters that control the scale and shape of the distribution. The median of Y, denoted by m, is the solution to the following equation,

$$w\exp \left\{-\exp \left(-\frac{m-\theta }{{\sigma }_1}\right)\right\}+(1-w)\left[1-\exp \left\{-\exp \left(\frac{m-\theta }{{\sigma }_2}\right)\right\}\right]=0.5.$$

Even though this equation cannot be solved for m explicitly to reveal the median in closed form, it is clear that $m-\theta$ also depends on all three other parameters of FG. In conclusion, the relationships between the three central tendency measures of FG are more versatile than those under a Gumbel distribution for the maximum or a Gumbel distribution for the minimum.

The variance of Y is $V\left(Y\right)=\{w{\sigma }_1^2+(1-w){\sigma }_2^2\}{\pi }^2/6+w(1-w){({\sigma }_1+{\sigma }_2)}^2{\gamma }^2$, which does not depend on the mode parameter $\theta$. Obviously, by setting $w=0$ or 1, $\mathrm{FG}(\theta ,{\sigma }_1,{\sigma }_2,w)$ reduces to one of the Gumbel components. Unlike a Gumbel distribution that only has one direction of skewness at a fixed level (of $\pm 1.14)$, an FG distribution can be left-skewed, or right-skewed, or symmetric. More specifically, with the mode fixed at zero when studying the skewness and kurtosis of FG, one can show (as outlined in Appendix A) that the third central moment of Y is given by

$$w\overline{w}{({\sigma }_1+{\sigma }_2)}^2\gamma \left\{{\gamma }^2(\overline{w}-w)({\sigma }_1+{\sigma }_2)+0.5{\pi }^2({\sigma }_1-{\sigma }_2)\right\}+2\zeta \left(3\right)\left(w{\sigma }_1^3-\overline{w}{\sigma }_2^3\right),$$

(4)

where $\overline{w}=1-w$, and $\zeta \left(3\right)\approx 1.202$ is Apéry’s constant. Although the direction of skewness is not immediately clear from (4), one may consider a special case with $w=0.5$ where (4) reduces to $({\sigma }_1-{\sigma }_2)\{\gamma {\pi }^2{({\sigma }_1+{\sigma }_2)}^2/8+\zeta \left(3\right)({\sigma }_1^2+{\sigma }_1{\sigma }_2+{\sigma }_2^2)\}$. Now one can see that $\mathrm{FG}(\theta ,{\sigma }_1,{\sigma }_2,0.5)$ is symmetric if and only if ${\sigma }_1={\sigma }_2$, and it is left-skewed (or right-skewed) when ${\sigma }_1$ is less (or greater) than ${\sigma }_2$. The kurtosis of Y can also be derived straightforwardly, with a more lengthy expression than (4) that we omit here, which may not shed much light on its magnitude except that it varies as the scale parameters and the mixing proportion vary, instead of fixing at 5.4 as for a Gumbel distribution. An R Shiny app depicting the pdf of $\mathrm{FG}(\theta ,{\sigma }_1,{\sigma }_2,w)$ with user-specified parameter values is available here: https://qingyang.shinyapps.io/gumbel_mixture/ (accessed on 6 March 2024), created and maintained by the first author. Along with the density function curve, the Shiny app provides skewness and kurtosis of the depicted FG density. From there, one can see that the skewness can be much lower than $-1.14$ or higher than $1.14$, and the kurtosis can be much higher than 5.4, suggesting that inference based on FG can be more robust to outliers than when a Gumbel distribution is assumed for data at hand, without imposing stringent assumptions on the skewness of the underlying distribution.

The flexibility of a mixture distribution usually comes with concerns relating to identifiability [22,23,24]. In particular, there is the notorious issue of label switching when fitting a finite mixture model [25]. Take the family of two-component normal mixture (NM) distributions as an example, defined by $\{\mathrm{NM}({\mu }_1,{\sigma }_1,{\mu }_2,{\sigma }_2,w):w\mathcal{N}({\mu }_1,{\sigma }_1^2)+(1-w)\mathcal{N}({\mu }_1,{\sigma }_2^2),\mathrm{for} {\sigma }_1,{\sigma }_2>0 \mathrm{and} w\in [0,1]\}$. When fitting a dataset assuming a normal mixture distribution, one cannot distinguish between, for instance, $\mathrm{NM}(1,2,3,4,0.2)$ and $\mathrm{NM}(3,4,1,2,0.8)$, since the likelihood of the data is identical under these two mixture distributions. As another example, for data from a normal distribution, a two-component normal mixture with two identical normal components and an arbitrary mixing proportion $w\in [0,1]$ leads to the same likelihood, and thus w cannot be identified. Teicher [23] showed that imposing a lexicographical order for the normal components resolves the issue of non-identifiability, which also excludes mixtures with two identical components in the above normal mixture family. Unlike normal mixtures of which all components are in the same family of normal distributions, the FG distribution results from mixing two components from different families, i.e., a Gumbel distribution for the maximum and a Gumbel distribution for the minimum, with weight w on the former component. By construction, FG does not have the label-switching issue. And we show in Appendix B by invoking Theorem 1 in Teicher [23] that the so-constructed mixture distribution is always identifiable even when the true distribution is a (one-component) Gumbel distribution.

3. Statistical Inference

3.1. Frequentist Inference Method

Based on a random sample of size n from the FG distribution, $\mathbf{y}$ $={\left\{y_i\right\}}_{i=1}^n$, maximum likelihood estimators (MLEs) of all model parameters in $\Omega$ $=(\theta ,{\sigma }_1,{\sigma }_2,w)$ can be obtained via the expectation-maximization (EM) algorithm [26]. To apply the EM algorithm, we introduce a latent variable Z that follows Bernoulli(w) such that the joint likelihood of $(Y,Z)$ is

$$f_{Y,Z}(y,z)={\left\{w{f}_1(y;\theta ,{\sigma }_1)\right\}}^z{\left\{(1-w)f_2(y;\theta ,{\sigma }_2)\right\}}^{1-z},$$

(5)

where $f_1(y;\theta ,{\sigma }_1)$ is the pdf in (1) evaluated at y with the scale parameter $\sigma ={\sigma }_1$, and $f_2(y;\theta ,{\sigma }_2)$ is the pdf in (2) evaluated at y with the scale parameter $\sigma ={\sigma }_2$. A random sample of size n from Bernoulli(w), $\mathbf{z}={\left\{z_i\right\}}_{i=1}^n$, is viewed as missing data, and ${\left\{(y_i,z_i)\right\}}_{i=1}^n$ are viewed as the complete data in the EM algorithm. It can be shown ([27] Section 2.6.3a) that integrating z out from (5) indeed gives the density of Y in (3). The log-likelihood based on the density in (3) is usually not well-behaved as an objective function to be maximized with respect to $\Omega$. By considering the complete-data log-likelihood based on (5), one can often maximize an objective function that is better-behaved as we demonstrate next. More specifically, the complete-data log-likelihood is

$$\mathcal{l}(\Omega ;\mathbf{y},\mathbf{z})={\displaystyle \sum _{i=1}^n}\{z_i\log \left(w{f}_1(y_i;\theta ,{\sigma }_1)\right)+(1-z_i)\log \left((1-w)f_2(y_i;\theta ,{\sigma }_2)\right)\}.$$

(6)

Starting from an initial estimate of $\Omega$ (at the zero-th iteration), denoted by ${\Omega }^{\left(0\right)}$, one iterates two steps referred to as the E-step and the M-step until a convergence criterion is met. In the E-step at the $(t+1)$-th iteration, one computes the conditional expectation of (6) given $\mathbf{y}$ while assuming the true parameter value to be ${\Omega }^{\left(t\right)}=({\theta }^{\left(t\right)},{\sigma }_1^{\left(t\right)},{\sigma }_2^{\left(t\right)},w^{\left(t\right)})$, that is, ${\mathrm{E}}_{{\Omega }^{\left(t\right)}}\left\{\mathcal{l}(\Omega ;\mathbf{y},\mathbf{z})\right|\mathbf{y}\}$. This conditional expectation can be shown to be

$$Q\left(\Omega \left|{\Omega }^{\left(t\right)}\right.\right)={\displaystyle \sum _{i=1}^n}\left\{T_i^{\left(t\right)}\log \left(w{f}_1\left(y_i;\theta ,{\sigma }_1\right)\right)+\left(1-T_i^{\left(t\right)}\right)\log \left((1-w)f_2\left(y_i;\theta ,{\sigma }_2\right)\right)\right\},$$

(7)

where

$$T_i^{\left(t\right)}={\mathrm{E}}_{{\Omega }^{\left(t\right)}}\left(Z\right|Y=y_i)=\frac{w^{\left(t\right)}{f}_1(y_i;{\theta }^{\left(t\right)},{\sigma }_1^{\left(t\right)})}{w^{\left(t\right)}{f}_1(y_i;{\theta }^{\left(t\right)},{\sigma }_1^{\left(t\right)})+(1-w^{\left(t\right)})f_2(y_i;{\theta }^{\left(t\right)},{\sigma }_2^{\left(t\right)})}.$$

(8)

In the M-step at the $(t+1)$-th iteration, one maximizes $Q(\Omega |{\Omega }^{\left(t\right)})$ with respect to $\Omega$ to obtain an updated estimate ${\Omega }^{(t+1)}=({\theta }^{(t+1)},{\sigma }_1^{(t+1)},{\sigma }_2^{(t+1)},w^{(t+1)})$, in which $w^{(t+1)}={\sum }_{i=1}^n{T}_i^{\left(t\right)}/n$, and the other three updated estimates in ${\Omega }^{(t+1)}$ are obtained numerically.

The EM algorithm avoids directly maximizing the log-likelihood based on (3) by (iteratively) maximizing the better-behaved $Q(\Omega |{\Omega }^{\left(t\right)})$ in (7). To further improve the numerical efficiency, we exploit the expectation-conditional maximization (ECM) algorithm [28], which replaces the M-step with a sequence of simpler conditional maximizations referred to as the CM-step. Essentially, within each M-step, we update w via $w^{(t+1)}={\sum }_{i=1}^n{T}_i^{\left(t\right)}/n$, then we update $\theta$ using $w^{(t+1)}$ along with $({\sigma }_1^{\left(t\right)},{\sigma }_2^{\left(t\right)})$, followed by updating ${\sigma }_1$ using $w^{(t+1)}$, the recently updated $\theta$, and ${\sigma }_2^{\left(t\right)}$; lastly, we update ${\sigma }_2$ using $w^{(t+1)}$ and the recently updated $\theta$ and ${\sigma }_1$. There is no closed-form updating formula for the latter three updates, but each of them can now be easily updated by most well-accepted one-dimensional optimization algorithms, such as the Newton–Raphson method. To ensure convergence at the global maximum, as recommended by Wu [29], one should implement the ECM algorithm several rounds with different starting values ${\Omega }^{\left(0\right)}$.

After obtaining the MLE of $\Omega$, denoted by $\widehat{\Omega }$, we propose to use the sandwich variance estimator ([27] Chapter 7) to estimate the variance-covariance matrix of $\widehat{\Omega }$. One may also estimate the variance-covariance of $\widehat{\Omega }$ based on the observed information matrix as described in Louis [30] and Oakes [31]. The benefit of using the sandwich variance estimator is its robustness to model misspecification. Finally, the EM and ECM algorithms bear a strong resemblance to data augmentation [32] in the Bayesian framework, which we turn to next for inferring $\Omega$.

3.2. Bayesian Inference Method

In the Bayesian framework, we formulate hierarchical models starting with the FG distribution,

$$Y|\theta ,{\sigma }_1,{\sigma }_2,w\sim \mathrm{FG}(\theta ,{\sigma }_1,{\sigma }_2,w),$$

followed by independent weakly informative or non-informative priors for elements in $\Omega$,

$$\begin{array}{cc}\hfill \theta & \sim \mathcal{N}(0,{10}^4),\hfill \\ \hfill {\sigma }_j& \sim \mathrm{inv-Gamma}(1,1),\mathrm{for}j=1,2,\hfill \\ \hfill w& \sim \mathrm{Uniform}(0,1),\hfill \end{array}$$

where inv-Gamma refers to the inverse Gamma distribution. We choose the above prior for the scale parameters by following the prior selection for variance parameters suggested in Gelman [33].

We employ the Metropolis-within-Gibbs sampler [34,35] to obtain an estimate of $\Omega$ from the posterior distribution of $\Omega$ given observed data $\mathbf{y}$. Similar to the EM/ECM algorithm in Section 3.1, the latent variable Z is also introduced as a device to carry out data augmentation. The iterative algorithm presented next is based on the following two conditional distributions that can be easily proved,

$$\begin{array}{cc}\hfill z_i|\theta ,{\sigma }_1,{\sigma }_2,w,{\mathbf{z}}_{-i},\mathbf{y}& \sim \mathrm{Bernoulli}\left(\frac{w{f}_1(y_i;\theta ,{\sigma }_1)}{w{f}_1(y_i;\theta ,{\sigma }_1)+(1-w)f_2(y_i;\theta ,{\sigma }_2)}\right),\hfill \\ \hfill w|\theta ,{\sigma }_1,{\sigma }_2,\mathbf{z},\mathbf{y}& \sim \mathrm{Beta}\left(1+{\displaystyle \sum _{i=1}^n}{z}_i,n+1-{\displaystyle \sum _{i=1}^n}{z}_i\right),\hfill \end{array}$$

where ${\mathbf{z}}_{-i}$ results from dropping $z_i$ from $\mathbf{z}$, and the first result above is also the one from which (8) is deduced.

The Metropolis-within-Gibbs sampler at the $(t+1)$-th iteration involves four steps outlined below.

• Step 1: For $i=1,\cdots ,n$, draw $z_i^{(t+1)}$ from $\mathrm{Bernoulli}\left(T_i^{\left(t\right)}\right)$, where $T_i^{\left(t\right)}$ is given in (8).
• Step 2: Draw $w^{(t+1)}$ from $\mathrm{Beta}\left(1+{\sum }_{i=1}^n{z}_i^{(t+1)},n+1-{\sum }_{i=1}^n{z}_i^{(t+1)}\right).$
• Step 3: Draw $\tilde{\theta }$ from $\mathcal{N}({\theta }^{\left(t\right)},{\tau }_0)$, and update ${\theta }^{\left(t\right)}$ to ${\theta }^{(t+1)}$ according to the following decision rule,

  $${\theta }^{(t+1)}=\left\{\begin{array}{c}{\displaystyle \tilde{\theta },\mathrm{with}\mathrm{probability}q=\min \left\{\frac{p\left(\tilde{\theta }\right|w^{(t+1)},{\sigma }_1^{\left(t\right)},{\sigma }_2^{\left(t\right)},\mathbf{y})}{p\left({\theta }^{\left(t\right)}\right|w^{(t+1)},{\sigma }_1^{\left(t\right)},{\sigma }_2^{\left(t\right)},\mathbf{y})},1\right\},}\hfill \\ {\theta }^{\left(t\right)},\mathrm{with}\mathrm{probability}1-q.\hfill \end{array}\right.$$
• Step 4: For $j=1,2$, draw ${\tilde{\sigma }}_j$ from $\mathcal{N}({\sigma }_j^{\left(t\right)},{\tau }_j)$, and update ${\sigma }_j^{\left(t\right)}$ to ${\sigma }_j^{(t+1)}$ according to the following decision rule, for $k\ne j$,

  $${\sigma }_j^{(t+1)}=\left\{\begin{array}{c}{\displaystyle {\tilde{\sigma }}_j,\mathrm{with}\mathrm{probability}q=\min \left\{\frac{p\left({\tilde{\sigma }}_j\right|{\theta }^{(t+1)},{\sigma }_k^{\left(t\right)},w^{(t+1)},\mathbf{y})}{p\left({\sigma }_j^{\left(t\right)}\right|{\theta }^{(t+1)},{\sigma }_k^{\left(t\right)},w^{(t+1)},\mathbf{y})},1\right\},}\hfill \\ {\sigma }_j^{\left(t\right)},\mathrm{with}\mathrm{probability}1-q.\hfill \end{array}\right.$$

In Steps 3 and 4, $p(·|·)$ refers to a conditional pdf generically, ${\tau }_0$, ${\tau }_1$, and ${\tau }_2$ are three positive tuning parameters whose values should be chosen so that the acceptance rate at each step is around $23\%$ [36]. To draw samples from the joint posterior distribution, there are numerous ways to design the Markov chain Monte Carlo (MCMC) sampler. Instead of the Metropolis-within-Gibbs sampler we adopt here, one may use other existing MCMC software, such as STAN [37], JAGS [38], and BUGS [39,40], the former two of which are demonstrated in the GitHub repository (https://github.com/rh8liuqy/flexible_Gumbel, accessed on 6 March 2024)). After obtaining enough high-quality samples from the joint posterior distribution $p(\theta ,{\sigma }_1,{\sigma }_2,w|\mathbf{y})$, Bayesian inference is straightforward, including point estimation, interval estimation, and uncertainty assessment.

4. Simulation Study

Large-sample properties of MLEs and likelihood-based Bayesian inference under a correct model for data have been well studied. To assess finite-sample performance of the frequentist method and Bayesian method proposed in Section 3, we carried out a simulation study with two specific aims: first, to compare inference results from the two methods; second, to compare goodness of fit for data from distributions outside of the FG family when one assumes an FG distribution and when one assumes a two-component normal mixture distribution for the data.

In the first experiment, denoted as (E1) hereafter, we considered two FG distributions as true data-generating mechanisms, $\mathrm{FG}(\theta =1,{\sigma }_1=1,{\sigma }_2=1,w=0.4)$ and FG$(\theta =0,{\sigma }_1=1,{\sigma }_2=5,w=0.5)$. This design creates two FG distributions with the second one more skewed and variable than the first. Based on a random sample of size $n=50$ from the first FG distribution, we estimated $\Omega$ by applying the ECM algorithm and the Metropolis-within-Gibbs algorithm. Similarly, based on a random sample of size $n\in \{100,200\}$, we implemented the two algorithms to estimate $\Omega$. The former algorithm produced the MLE of $\Omega$, and we used the median of the posterior distribution of $\Omega$ at convergence of the latter algorithm as another point estimate of $\Omega$.

Table 1. Frequentist and Bayesian inference results in experiment (E1) across 1000 Monte Carlo replicates. Here, point.est stands for the average of 1000 point estimates for each parameter from each method, $\widehat{\mathrm{s}.\mathrm{d}.}$ stands for the average of the corresponding 1000 estimated standard deviations, and s.d. refers to the empirical standard deviation of the 1000 point estimates from each method. Numbers in parentheses are $100\times$ Monte Carlo standard errors associated with the averages of 1000 estimates of the standard deviation associated with a point estimator.

Sample Size	Parameter	Frequentist			Bayesian
		point.est	$\widehat{\mathbf{s}.\mathbf{d}.}$	s.d.	point.est	$\widehat{\mathbf{s}.\mathbf{d}.}$	s.d.
$n=50$	$\theta$	0.990	0.197 (0.40)	0.209	0.965	0.250 (0.18)	0.224
	${\sigma }_1$	1.106	0.272 (0.77)	0.419	1.045	0.638 (2.30)	0.296
	${\sigma }_2$	1.047	0.192 (0.49)	0.216	1.082	0.459 (2.12)	0.465
	w	0.411	0.190 (0.46)	0.207	0.435	0.207 (0.12)	0.187
$n=100$	$\theta$	0.002	0.198 (0.20)	0.201	0.013	0.205 (0.15)	0.203
	${\sigma }_1$	0.979	0.204 (0.41)	0.216	1.014	0.224 (0.27)	0.214
	${\sigma }_2$	4.932	0.590 (0.56)	0.613	4.813	0.666 (0.44)	0.615
	w	0.495	0.091 (0.09)	0.090	0.484	0.090 (0.04)	0.088
$n=200$	$\theta$	0.008	0.136 (0.08)	0.129	0.011	0.137 (0.07)	0.130
	${\sigma }_1$	0.999	0.143 (0.21)	0.144	1.013	0.144 (0.10)	0.141
	${\sigma }_2$	4.993	0.435 (0.32)	0.431	4.940	0.457 (0.20)	0.434
	w	0.500	0.064 (0.04)	0.063	0.495	0.063 (0.02)	0.062

presents summary statistics of these estimates of $\Omega$ and estimates of the corresponding standard deviation across 1000 Monte Carlo replicates under each simulation setting specified by the design of an FG distribution and the level of n.

According to Table 1, all parameter estimates in $\Omega$ are reasonably close to the true values. When the sample size is as small as 50, estimates for $\Omega$ resulting from the frequentist method are still similar to those from the Bayesian inference method, although estimates for the standard deviations of these point estimators can be fairly different. We do not find such discrepancy surprising because, for the frequentist method where we use the sandwich variance estimator to infer the uncertainty of an MLE for $\Omega$, the asymptotic properties associated with MLEs that support the use of a sandwich variance estimator may not take effect yet at the current sample size; and, for the Bayesian method, the quantification of standard deviation can be sensitive to the choice of priors when n is small. These are confirmed by the diminishing discrepancy between the two sets of standard deviation estimates when $n=100$, 200. A closer inspection of the reported empirical mean of estimates for $\Omega$ along with their empirical standard error suggests that, when $n=100$, the Bayesian method may slightly underestimate ${\sigma }_2$, the larger of the two scale parameters of FG. We believe that this is due to the inverse gamma prior imposed on the scale parameters that is sharply peaked near zero, and thus the posterior median of the larger scale parameter tends to be pulled downwards when the sample size is not large. As the sample size increases to 200, this trend of underestimation appears to diminish. The empirical means of the standard deviation estimates from both methods are close to the corresponding empirical standard deviations, which indicate that the variability of a point estimator is accurately estimated when n is not small, whether it is based on the sandwich variance estimator in the frequentist framework, or based on the posterior sampling in the Bayesian framework. In summary, the methods proposed in Section 3 under both frameworks provide reliable inference for $\Omega$ along with accurate uncertainty assessment of the point estimators when data arise from an FG distribution.

Among all existing mixture distributions, normal mixtures probably have the longest history and are most referenced in the literature. In another experiment, we compared the model fitting of normal mixture with that of FG when data arise from three heavy-tailed distributions: (E2) Laplace with the location parameter equal to zero and the scale parameter equal to 2; (E3) a mixture of two Gumbel distributions for the maximum, with a common mode at zero, scale parameters in the two components equal to 2 and 6, respectively, and the mixing proportion equal to 0.5; (E4) a Student-t distribution with degrees of freedom equal to 5. From each of the three distributions in (E2)–(E4), we generated a random sample of size $n=200$, following which we fit a two-component normal mixture model via the EM algorithm implemented using the R package mixtools (version: 2.0.0), and also fit an FG model via the two algorithms described in Section 3. This model fitting exercise was repeated for 1000 Monte Carlo replicates under each of (E2)–(E4).

We used an empirical version of the Kullback–Leibler divergence as the metric to assess the quality of modeling fitting. We denote the true density function as $p(·)$, and let $\widehat{p}(·)$ be a generic estimated density resulting from one of the three considered model fitting strategies. Under each setting in (E2)–(E4), a random sample of size 50,000, $(x_1,\dots ,x_{50,000})$, was generated from the true distribution, and an empirical version of the Kullback–Leibler divergence from $\widehat{p}(·)$ to $p(·)$ is given by $D_{\mathrm{KL}}=(1/50,000){\sum }_{i=1}^{50,000}\log (p\left(x_i\right)/\widehat{p}\left(x_i\right))$. Figure 1 shows the boxplots of $D_{\mathrm{KL}}$ across 1000 Monte Carlo replicates corresponding to each model fitting scheme under (E2)–(E4).

Judging from Figure 1, the FG distribution clearly outperforms the normal mixture when fitting data from any of the three heavy-tailed distributions in (E2)–(E4), and results from the frequentist method are comparable with those from the Bayesian method for fitting an FG model. When implementing the ECM algorithm for fitting the FG model and the EM algorithm for fitting the normal mixture, we set a maximum number of iterations at 1000. Our ECM algorithm always converged in the simulation, i.e., converged to a stationary point within 1000 iterations. However, the EM algorithm for fitting a normal mixture often had trouble achieving that, with more difficulty when data come from a heavier-tailed distribution. More specifically, under (E4), which has the highest kurtosis (equal to 9) among the three settings, the EM algorithm failed to converge in 59.9% of all Monte Carlo replicates; under (E2), which has the second highest kurtosis (equal to 6), it failed to converge in 6.7% of the replicates. Results associated with the normal mixture from these failing replicates were not included when producing the boxplots in Figure 1. In conclusion, the FG distribution is more suitable for symmetric or asymmetric heavy-tailed data than the normal mixture distribution.

5. An Application in Hydrology

Daily maximum water elevation changes of a water body, such as ocean, lake, and wetland, are of interest in hydrologic research. These changes may be close to zero on most days but can be extremely large or small under extreme weather. From the National Water Information System (https://waterdata.usgs.gov/), we downloaded water elevation data for Lake Murray near Columbia, South Carolina, United States, recorded from 18 September 2020 to 18 September 2021. The water elevation change of a given day was calculated by contrasting the maximum elevation and the minimum elevation on that day, returning a positive (negative) value if the maximum record of the day comes after (before) the minimum record on the same day. We fit the FG distribution to the resultant data with $n=366$ records using the frequentist method and the Bayesian method, with results presented in

Table 2. Frequentist and Bayesian inferences about daily maximum water elevation changes of Lake Murray, South Carolina, United States. Besides parameter estimates (under point.est) and the estimated standard deviations of these parameter estimates (under $\widehat{\mathrm{s}.\mathrm{d}.}$), 95% confidence intervals of the parameters from the frequentist method, and 95% credible intervals from the Bayesian method are also provided (under lower 95 and upper 95).

Parameter	Frequentist				Bayesian
	point.est	$\widehat{\mathbf{s}.\mathbf{d}.}$	Lower 95	Upper 95	point.est	$\widehat{\mathbf{s}.\mathbf{d}.}$	Lower 95	Upper 95
$\theta$	−0.795	0.796	−2.355	0.764	−0.485	0.695	−1.670	0.979
${\sigma }_1$	5.186	0.541	4.124	6.247	5.400	0.655	4.520	6.910
${\sigma }_2$	6.237	1.735	2.836	9.638	5.733	1.036	4.390	8.030
w	0.698	0.169	0.367	1.029	0.629	0.141	0.327	0.846

. The two inference methods produced very similar estimates for most parameters, although small differences were observed. For example, one would estimate the mode of daily maximum water elevation change to be $-0.795$ feet based on the frequentist method, but estimate it to be $-0.485$ feet using the Bayesian method. The discrepancy between these two mode estimates is minimal considering that the daily maximum water elevation changes range from $-38$ feet to 49.4 feet within this one year. Taking into account the uncertainty in these point estimates, we do not interpret any of these differences as statistically significant because a parameter estimate from one method always falls in the interval estimate for the same parameter from the other method according to Table 2. Using parameter estimates in Table 2 in the aforementioned R Shiny app, we obtained an estimated skewness of $-0.102$ and an estimated kurtosis of 6.384 based on the frequentist inference results, whereas the Bayes inference yielded an estimated skewness of 0.058 and an estimated kurtosis of 6.074. Combining these two sets of results, we concluded that the underlying distribution of daily maximum water elevation change may be nearly symmetrical, with outliers on both tails that cause tails heavier than that of a Gumbel distribution.

Figure 2 presents the estimated density functions from these two methods, in contrast with the estimated density curve resulting from fitting the data to a two-component normal mixture, and a kernel density estimate using a Gaussian kernel with the bandwidth selected according to the method proposed by Sheather and Jones [41]. The last estimate is fully nonparametric and served as a benchmark against which the other three density estimates were assessed graphically. The kernel density estimate is more flexible at describing varying tail behaviors, but such flexibility comes at the cost of statistical efficiency and interpretability. With the wiggly tails evident in Figure 2 for this estimate, we suspected a certain level of overfitting of the kernel density estimate. This often happens to kernel-based estimation of a function around a region where data are scarce, with a bandwidth not large enough for the region. Between the two FG density estimates, the difference is almost negligible. They both track the kernel density estimate closely over a wide range of the support around the mode. The mode of the estimated normal mixture density is close to the other three mode estimates, but the tails are much lighter than those of the other three estimated densities.

Besides comparing the three parametric density estimates pictorially, we also used the Monte-Carlo-based one-sample Kolmogorov–Smirnov test to assess the goodness of fit. The p-values from this test are 0.223, 0.312, and 0.106 for the frequentist FG density estimate, the Bayesian FG density estimate, and the estimated normal mixture density, respectively. Although none of the p-values are low enough to indicate a lack of fit (at significance level 0.05, for example), the p-value associated with the normal mixture is much lower than those for FG. Hence, between the two null hypotheses, with one assuming an FG distribution and the other claiming a normal mixture for this dataset, we find even weaker evidence to reject the former than data evidence against the latter. It is also worth noting that the Kolmogorov–Smirnov test is known to have low power to detect deviations from a posited distribution that occurs in the tails [42]. This may explain the above-0.05 p-value for the normal mixture fit of the data even though the tail of this posited distribution may be too thin for the current data. Finally, as suggested by a referee, we computed the Akaike information criterion (AIC) and the Bayesian information criterion (BIC) after fitting the FG distribution and the normal mixture distribution to the data. When assuming an FG distribution, we obtained an AIC/BIC of 2506.028/2521.638 from the frequentist method, and 2506.299/2521.909 from the Bayesian method. When assuming a mixture normal, we found the values of AIC and BIC to be 2499.821 and 2519.334, respectively. Even though the fitted normal mixture distribution produces a lower AIC/BIC than the fitted FG distribution, we argue that these metrics focus more on the overall goodness of fit, and can be more forgiving when it comes to a relatively poor fit for certain feature of a distribution, such as the tail behavior.

We used STAN to implement the Bayesian inference for the Lake Murray data. The code can be found here: https://github.com/rh8liuqy/flexible_Gumbel/blob/main/FG_MLR.stan, accessed on 6 March 2024), where the JAGS code for fitting the FG distribution is also provided. The posterior output is given in Appendix C. The output provided there indicates that our MCMC chain has converged (see the Rhat statistics).

6. An Application in Criminology

With the location parameter $\theta$ signified in the FG distribution as the mode, it is straightforward to formulate a modal regression model that explores the relationship between the response variable and predictors. To demonstrate the formulation of a modal regression model based on the FG distribution, we analyzed a dataset from Agresti et al. [43] in the area of criminology. This dataset contains the percentage of college education, poverty percentage, metropolitan rate, and murder rate for the 50 states in the United States and the District of Columbia from the year 2003. The poverty percentage is the percentage of the residents with income below the poverty level; the metropolitan rate is defined as the percentage of the population living in the metropolitan area; and the murder rate is the annual number of murders per 100,000 people in the population.

We fit the following modal regression model to investigate the association between the murder rate (Y) and the aforementioned demographic variables,

$$Y\mid \beta ,{\sigma }_1,{\sigma }_2\sim \mathrm{FG}({\beta }_0+{\beta }_1\times \mathrm{college}+{\beta }_2\times \mathrm{poverty}+{\beta }_3\times \mathrm{metropolitan},{\sigma }_1,{\sigma }_2,w),$$

where $\beta ={[{\beta }_0,{\beta }_1,{\beta }_2,{\beta }_3]}^{\top }$ includes all regression coefficients. For the prior elicitation in Bayesian inference, we assume that ${\beta }_0,\cdots ,{\beta }_3\stackrel{\mathrm{i}.\mathrm{i}.\mathrm{d}}{\sim }\mathcal{N}(0,{10}^4)$ and use the same priors for ${\sigma }_1$, ${\sigma }_2$, and w as those in Section 3.2. As a more conventional regression analysis to compare with our modal regression, we also fit the mean regression model assuming mean-zero normal model error to the data.

Table 3. Frequentist and Bayesian modal regression models based on the FG distribution fitted to the crime data. Besides parameter estimates (under point.est) and the estimated standard deviations of these parameter estimates (under $\widehat{\mathrm{s}.\mathrm{d}.}$), 95% confidence intervals of the parameters from the frequentist method, and 95% credible intervals from the Bayesian method are also provided (under lower 95 and upper 95).

Parameter	Frequentist				Bayesian
	point.est	$\widehat{\mathbf{s}.\mathbf{d}.}$	Lower 95	Upper 95	point.est	$\widehat{\mathbf{s}.\mathbf{d}.}$	Lower 95	Upper 95
${\beta }_1$	−0.166	0.072	−0.306	−0.026	−0.162	0.079	−0.312	−0.003
${\beta }_2$	0.216	0.110	−0.000	0.432	0.232	0.120	−0.007	0.479
${\beta }_3$	0.067	0.013	0.042	0.093	0.067	0.014	0.039	0.095
${\sigma }_1$	1.600	0.180	1.247	1.954	1.690	0.214	1.206	2.686
${\sigma }_2$	51.882	45.034	−36.384	140.148	19.3	19.300	0.187	133.047

shows the inference results from the modal regression model, and

Table 4. Mean regression model based on the normal distribution fitted to the crime data. Besides parameter estimates (under point.est) and the estimated standard deviations of these parameter estimates (under $\widehat{\mathrm{s}.\mathrm{d}.}$), 95% confidence intervals of the parameters are also provided (under lower 95 and upper 95).

Parameter	point.est	$\widehat{\mathbf{s}.\mathbf{d}.}$	Lower 95	Upper 95
${\beta }_1$	0.467	0.161	0.142	0.792
${\beta }_2$	1.140	0.224	0.689	1.591
${\beta }_3$	0.068	0.034	0.000	0.136

presents the inference results from the mean regression model. At $5\%$ significance level, both frequentist and Bayesian modal regression analyses confirm that there exists a negative association between the percentage of college education and the murder rate, as well as a positive association between the metropolitan rate and the murder rate. In contrast, according to the inferred mean regression model, there is a positive association between the percentage of college education and the murder rate. Such claimed positive association is intuitively difficult to justify and contradicts many published results in criminology [44,45].

The scatter plot of the data in Figure 3 can shed some light on why one reaches such a drastically different conclusion on a covariate effect when mean regression is considered in place of modal regression. As shown in Figure 3, there exists an obvious outlier, the District of Columbia (D.C.), in panels of the first row of the scatter plot matrix, for instance. D.C. not only exhibited the highest murder rate but also the highest percentage of college-educated individuals. These dual characteristics position D.C., as an outlier within the dataset. Mean regression reacts to this one extreme outlier by inflating the covariate effect associated with the percentage of college education in the inferred mean regression function. Thanks to the heavy-tailed feature of the FG distribution, modal regression based on this distribution is robust to outliers; it strives to capture data features suggested by the majority of the data and is not distracted by the extreme outlier when inferring covariate effects in this application.

Lastly, to compare their overall goodness of fit for the current data, we computed AIC and BIC following fitting each regression model. Adopting the frequentist and Bayesian methods, the modal regression analysis yields AIC/BIC equal to 239.394/252.917 and 238.710/252.233, respectively. The mean regression analysis leads to AIC and BIC equal to 303.154 and 312.813, respectively. Appendix C contains the convergence diagnosis for the Bayesian inferential method applied to this dataset, from which we see no concerns about convergence.

7. Discussion

The mode had been an overlooked location parameter in statistical inference until recently when the statistics community witnessed a revived interest in modal regression among statisticians [1,5,46,47,48,49,50]. Historically, statistical inference for the mode has been mostly developed under the nonparametric framework for reasons we point out in Section 1. Existing semiparametric methods for modal regression only introduce parametric ingredients in the regression function, i.e., the conditional mode of the response, with the mode-zero error distribution left in a nonparametric form [18,51,52,53,54,55,56,57]. The few recently proposed parametric modal regression models all impose stringent parametric assumptions on the error distribution [19,20,21]. Our proposed flexible Gumbel distribution greatly alleviates concerns contributing to data scientists’ reluctance to adopt a parametric framework when drawing inferences for the mode. This new distribution is a heterogeneous mixture in the sense that the two components in the mixture belong to different Gumbel distribution families, which is a feature that shields it from the non-identifiability issue most traditional mixture distributions face, such as the normal mixtures. The proposed distribution is indexed by the mode along with shape and scale parameters, and thus is convenient to use to draw inferences for the mode while remaining flexible. It is also especially suitable for modeling heavy-tailed data, whether the heaviness in tails is due to extremely large or extremely small observations, or both. These are virtues of FG that cannot be achieved by the popular normal mixture and many other existing mixture distributions.

We develop the numerically efficient and stable ECM algorithm for frequentist inference for the FG distribution, and a reliable Bayesian inference method that can be easily implemented using free software, including STAN, JAGS, and BUGS. Compared with the more widely adopted mean regression framework, the modal regression model based on FG we entertained in Section 6 shows great potential in revealing meaningful covariate effects potentially masked by extreme outliers. With these advances made in this study, we open up new directions for parametric modal regression and semiparametric modal regression with a fully parametric yet flexible error distribution, and potentially nonparametric ingredients incorporated in the regression function.