Bayesian Estimation of Variance-Based Information Measures and Their Application to Testing Uniformity

Abstract

Entropy and extropy are emerging concepts in machine learning and computer science. Within the past decade, statisticians have created estimators for these measures. However, associated variability metrics, specifically varentropy and varextropy, have received comparably less attention. This paper presents a novel methodology for computing varentropy and varextropy, drawing inspiration from Bayesian nonparametric methods. We implement this approach using a computational algorithm in R and demonstrate its effectiveness across various examples. Furthermore, these new estimators are applied to test uniformity in data.

1. Introduction

In this section, we begin by reviewing the concepts of entropy and extropy, along with some existing estimators from the literature. Subsequently, we introduce estimators of varentropy and varextropy developed using a frequentist approach.

Entropy is a fundamental concept in information theory that was originally introduced in [1]. It has found numerous applications in various fields, such as thermodynamics, communication theory, reliability, computer science, biology, economics, and statistics [2,3]. Let X be a continuous random variable with support on $\mathcal{S}$, cumulative distribution function (CDF) F, and probability density function (PDF) f. The entropy $H\left(F\right)$ of X is defined as

$$H\left(F\right)=E_f\left(-logf\left(x\right)\right)=-{\int }_{\mathcal{S}}f\left(x\right)logf\left(x\right)dx,$$

(1)

where log is the natural logarithm.

Extropy, a concept introduced by [4], represents a relatively recent development in the field of statistics that serves as a dual counterpart to entropy. Its significance has been demonstrated in various studies, including those conducted by [5,6], where it has found applications in the context of goodness-of-fit tests.

For a continuous random variable X supported on $\mathcal{S}$ with CDF F and PDF f, the extropy of X, denoted by $J\left(F\right)$, is defined as

$$J\left(F\right)=-\frac{1}{2}{E}_f\left(f\left(x\right)\right)=-\frac{1}{2}{\int }_{\mathcal{S}}{f}^2\left(x\right)dx$$

(2)

In most practical scenarios, the true PDF f is unknown, and hence, we need to estimate the entropy (1) and extropy (2) from the available data, which can be a challenging task. Several frequentist methods are available for entropy estimation in the literature. Among different approaches, Ref. [7] estimator has gained wide popularity due to its simplicity. Ref. [7] derived the expression for (1) in terms of the inverse of the distribution function F, given by

$$H\left(F\right)={\int }_0^{1}log\left(\frac{d}{dt}{F}^{-1}\left(t\right)\right)dt.$$

Using the empirical distribution function $F_n$ instead of the unknown F, Ref. [7] proposed an estimator for $H\left(F\right)$ based on the difference operator rather than the differential operator. The derivative of $F^{-1}\left(t\right)$ is estimated using a function of the order statistics. Specifically, if $X_1,X_2,\dots ,X_n$ is a sample from F, then [7] estimator is given by

$$\begin{array}{c}\hfill H{1}_{m,n}=n^{-1}\sum _{i=1}^{n}log\left(\frac{X_{(i+m)}-X_{(i-m)}}{2m/n}\right),\end{array}$$

(3)

where m is a positive integer smaller than $n/2$, and $X_{\left(1\right)}\le X_{\left(2\right)}\le \cdots \le X_{\left(n\right)}$ are the order statistics of $X_1,X_2,\dots ,X_n$ with $X_{(i-m)}=X_{\left(1\right)}$ if $i\le m$ and $X_{(i+m)}=X_{\left(n\right)}$ if $i\ge n-m$. Ref. [7] showed that $H{1}_{m,n}\stackrel{p}{\to }H\left(F\right)$ as $n\to \infty$, $m\to \infty$, and $\frac{m}{n}\to 0$, where $\stackrel{p}{\to }$ denotes convergence in probability. Note that the expression inside the log in (3) is the slope of the straight line that passes through the points $\left(\frac{i+m}{n},X_{(i+m)}\right)$ and $\left(\frac{i-m}{n},X_{(i-m)}\right)$, where $F_n\left(X_{(i+m)}\right)=\frac{i+m}{n}$ and $F_n\left(X_{(i-m)}\right)=\frac{i-m}{n}$. Ref. [8] proposed a modification to the estimator (3) as it does not provide the correct formula for the slope when $i\le m$ or $i\ge n-m+1$. The proposed estimator, denoted by $H{2}_{m,n}$, is given by

$$\begin{array}{c}\hfill H{2}_{m,n}=n^{-1}\sum _{i=1}^{n}log\left(\frac{X_{(i+m)}-X_{(i-m)}}{c_{i}m/n}\right),\end{array}$$

(4)

where

$$\begin{array}{cc}\hfill c_i& =\left\{\begin{array}{cc}\frac{m+i-1}{m}\hfill & \mathrm{if}1\le i\le m\hfill \\ 2\hfill & \mathrm{if}m+1\le i\le n-m\hfill \\ \frac{n+m-i}{m}\hfill & \mathrm{if}n-m+1\le i\le n\hfill \end{array}\right..\hfill \end{array}$$

(5)

As for extropy estimation, Ref. [5] noticed that the extropy, first defined in (2) can be rewritten as

$$J\left(F\right)=-\frac{1}{2}{\int }_0^1{\left[\frac{d}{dt}{F}^{-1}\left(t\right)\right]}^{-1}dt$$

and proposed the following estimator for $J\left(F\right)$:

$$J{1}_{m,n}=-\frac{1}{2n}\sum _{i=1}^n\frac{2m/n}{X_{(i+m)}-X_{(i-m)}},$$

As in $H{1}_{m,n}$, Ref. [5] found that $J{1}_{m,n}$ gives incorrect estimates for $i\le m$ or $i\ge n-m+1$. Therefore, they proposed the revised estimator $J{2}_{m,n}$, where

$$J{2}_{m,n}=-\frac{1}{2n}\sum _{i=1}^n\frac{c_{i}m/n}{X_{(i+m)}-X_{(i-m)}}.$$

Here, $c_i$ is defined as specified in Equation (5). Ref. [5] proved that $J{1}_{m,n}$ and $J{2}_{m,n}$ converges in probability to $J\left(F\right)$ under the same conditions as for $H{1}_{m,n}$ or $H{2}_{m,n}$.

Other frequentist nonparametric estimators of entropy include those proposed by  [9,10,11,12,13,14]. A comprehensive review of nonparametric entropy estimators can be found in [15]. For extropy estimation, alternative approaches are provided by [16] as well as [17].

Bayesian estimation of entropy has not received as much attention as the frequentist approach. However, Ref. [18] developed a Bayes estimator of $H\left(F\right)$ based on the Dirichlet process [19]. Recently, Refs. [6,20,21] proposed an estimator of entropy and extropy based on an approximation of the Dirichlet process $DP(a,G)$ introduced by [22], where $a>0$ and G is a known CDF. The approximation is defined as

$$P_N(·)=\sum _{i=1}^N{w}_{i,N}{\delta }_{Y_i}(·),$$

(6)

In this equation, the weights $(w_{1,N},\dots ,w_{N,N})$ follow a Dirichlet distribution with parameters $(a/N,\dots ,a/N)$, while $Y_1,\dots ,Y_N$ are independent and identically distributed from the distribution G. The notation ${\delta }_{Y_i}$ represents the Dirac measure at the point $Y_i$. The sequences $\left(W_{i,N}\right)1\le i\le N$ and $\left(Y_i\right)1\le i\le N$ are independent. We refer to $\left(Y_i\right)1\le i\le N$ as the data points of $P_N$. Let

$$\begin{array}{c}\hfill H_{m,N,a}=\frac{1}{N}\sum _{i=1}^{N}log\left(\frac{Y_{(i+m)}-Y_{(i-m)}}{c_{i,a}}\right)\end{array}$$

(7)

and

$$\begin{array}{c}\hfill J_{m,N,a}=-\frac{1}{2N}\sum _{i=1}^N\frac{c_{i,a}}{Y_{(i+m)}-Y_{(i-m)}},\end{array}$$

(8)

where

$$\begin{array}{c}\hfill c_{i,a}=\left\{\begin{array}{cc}\sum _{k=2}^{i+m}{w}_{k,N}\hfill & 1\le i\le m,\hfill \\ \sum _{k=i-m+1}^{i+m}{w}_{k,N}\hfill & m+1\le i\le N-m,\hfill \\ \sum _{k=i-m+1}^N{w}_{k,N}\hfill & N-m+1\le i\le N.\hfill \end{array}\right.\end{array}$$

(9)

As $N\to \infty$, $m\to \infty$, $\frac{m}{N}\to 0$ and $a\to \infty$, Refs. [6,20] showed that

$$\begin{array}{c}\hfill H_{m,N,a}\stackrel{p}{\to }H\left(G\right)=-{\int }_{\mathcal{S}}g\left(x\right)logg\left(x\right)dx\end{array}$$

and

$$\begin{array}{c}\hfill J_{m,N,a}\stackrel{p}{\to }J\left(G\right)=-\frac{1}{2}{\int }_{\mathcal{S}}{g}^2\left(x\right)dx,\end{array}$$

where $G^{\prime }\left(x\right)=g\left(x\right)$. Observe that the slope of the straight line connecting the two points $\left(P_N\left(Y_{(i-m)}\right),Y_{(i-m)}\right)$ and $\left(P_N\left(Y_{(i+m)}\right),Y_{(i+m)}\right)$ is

$$\begin{array}{c}\hfill \frac{Y_{(i+m)}-Y_{(i-m)}}{P_N\left(Y_{(i+m)}\right)-P_N\left(Y_{(i-m)}\right)}=\frac{Y_{(i+m)}-Y_{(i-m)}}{c_{i,a}}.\end{array}$$

Let $\mathit{X}=(X_1,X_2,\dots ,X_n)$ be a sample from F and $DP(a,G)$ be a prior of F. Consider $H_{m,N,a}|\mathit{X}$ to be the posterior version of $H_{m,N,a}$ as defined in (7) with $P_N$ replaced by $P_N|\mathit{X}$, an approximation of $DP(a+n,G_{\mathit{X}})$, where

$$\begin{array}{c}\hfill G_{\mathit{X}}=a{(a+n)}^{-1}G+n{(a+n)}^{-1}{F}_n.\end{array}$$

(10)

Then, as $N\to \infty$, $m\to \infty$, $n\to \infty$, $\frac{m}{N}\to 0$, and $\frac{a}{n}\to 0$, we have [6,20]

$$\begin{array}{c}\hfill H_{m,N,a}|\mathit{X}\stackrel{p}{\to }H\left(F\right)=-{\int }_{\mathcal{S}}f\left(x\right)logf\left(x\right)dx,\end{array}$$

and

$$\begin{array}{c}\hfill J_{m,N,a}|\mathit{X}\stackrel{p}{\to }J\left(F\right)=-\frac{1}{2}{\int }_{\mathcal{S}}{f}^2\left(x\right)dx,\end{array}$$

where $F^{\prime }\left(x\right)=f\left(x\right)$.

Recently, there has been significant interest in studying the variability of information measures in the literature. In certain situations, it is possible to encounter two random variables with identical entropy or extropy. As a result of these circumstances, researchers ponder whether entropy or extropy would be the most suitable criterion for measuring uncertainty. One way of determining which probability distribution is more suitable is to check its variance. This scenario serves as a motivation for exploring two variance measures associated with entropy and extropy, known as varentropy and varextropy, respectively.

For a random variable X, the varentropy, denoted by $VH\left(F\right)$, is defined as follows:

$$\begin{array}{cc}\hfill VH\left(F\right)={\mathrm{Var}}_f(log\left(f\left(X\right)\right))& ={\int }_{S}f\left(x\right){\left[\log f\left(x\right)\right]}^{2}dx-{\left[{\int }_{S}f\left(x\right)\log f\left(x\right)dx\right]}^2\hfill \\ \hfill & =E_f\left(log\left({\left(f\left(x\right)\right)}^2\right)\right)-{\left[H\left(F\right)\right]}^2.\hfill \end{array}$$

Ref. [23] introduced varentropy as a compelling alternative to the kurtosis measure, particularly for comparing heavy-tailed distributions. In fact, varentropy has proven to be a valuable tool for assessing heavy-tailed distributions instead of relying solely on kurtosis. Subsequently, varentropy has found diverse applications across various fields. In computer science, varentropy plays an instrumental role in data compression, studying the variability of uncertainty measures [24], testing uniformity [25], and analyzing statistical issues [26]. Moreover, researchers have effectively employed varentropy to explore the variability of interval entropy measures [27]. Additionally, varentropy has proven valuable in applications related to proportional hazard rate models [24] and residual lifetime distributions [26].

Ref. [25] presented six estimators for computing varentropy. In this context, we will focus on two specific estimators based on the estimators (3) and (4). Specifically, $VH\left(F\right)$ can be expressed as

$$\begin{array}{c}\hfill VH\left(F\right)={\int }_0^1{log}^2\left(\frac{d}{dt}{F}^{-1}\left(t\right)\right)dt-{\left[{\int }_0^{1}log\left(\frac{d}{dt}{F}^{-1}\left(t\right)\right)dt\right]}^2,\end{array}$$

(11)

the two estimators of [25] are

$$\begin{array}{cc}\hfill VH{1}_{mn}& =\frac{1}{n}\sum _{i=1}^n{log}^2\left(\frac{X_{(i+m)}-X_{(i-m)}}{2m/n}\right)-{\left[H{1}_{m,n}\right]}^2\hfill \\ \hfill & =\frac{1}{n}\sum _{i=1}^n{\log }^2\left(X_{(i+m)}-X_{(i-m)}\right)-{\left[\frac{1}{n}\sum _{i=1}^n\log (X_{(i+m)}-X_{(i-m)})\right]}^2\hfill \end{array}$$

(12)

and

$$\begin{array}{cc}\hfill VH{2}_{mn}& =\frac{1}{n}\sum _{i=1}^n{\log }^2\left(\frac{X_{(i+m)}-X_{(i-m)}}{c_{i}m/n}\right)-{\left[H{2}_{m,n}\right]}^2,\hfill \end{array}$$

where $H{1}_{m,n}$, $H{2}_{m,n}$, and  $c_i$ are defined in (3), (4), and (5), respectively. Noughabi and Noughabi (2023) showed that $VH{1}_{mn}$ and $VH{2}_{mn}$ converge in probability to $VH\left(F\right)$ under the same conditions as for $H{1}_{m,n}$ or $H{2}_{m,n}$.

Another measure of information variability is the varextropy. Let X be an absolutely continuous random variable, then its varextropy, denoted as $VJ\left(X\right)$, is defined as follows [28]:

$$\begin{array}{cc}\hfill VJ\left(X\right)& \hfill ={\mathrm{Var}}_f(-\frac{1}{2}f\left(X\right))=\frac{1}{4}{E}_f\left({\left(f\left(X\right)\right)}^2\right)-{\left(J\left(X\right)\right)}^2\hfill \end{array}$$

(13)

$$\begin{array}{cc}\hfill & \hfill =\frac{1}{4}{\int }_{\mathcal{S}}{f}^3\left(x\right)dx-\frac{1}{4}{\left[{\int }_{\mathcal{S}}{f}^2\left(x\right)dx\right]}^2.\hfill \end{array}$$

(14)

Unlike the varentropy $VH\left(F\right)$, the estimation of $VJ\left(F\right)$ has not been intensively discussed in the literature. Notice that

$$\begin{array}{c}\hfill VJ\left(F\right)=\frac{1}{4}{\int }_0^1{\left[\frac{d}{dt}{F}^{-1}\left(t\right)\right]}^{-2}dt-\frac{1}{4}{\left[{\int }_0^1{\left[\frac{d}{dt}{F}^{-1}\left(t\right)\right]}^{-1}dt\right]}^2.\end{array}$$

(15)

Accordingly, our proposed estimators of $VJ\left(X\right)$ based on the two estimator of [5] are:

$$\begin{array}{cc}\hfill VJ{1}_{mn}& \hfill =\frac{1}{4n}\sum _{i=1}^n{\left[\frac{2m/n}{x_{(i+m)}-x_{(i-m)}}\right]}^2-{\left[J{1}_{m,n}\right]}^2\hfill \end{array}$$

(16)

and

$$\begin{array}{cc}\hfill VJ{2}_{mn}& \hfill =\frac{1}{4n}\sum _{i=1}^n{\left[\frac{c_{i}m/n}{x_{(i+m)}-x_{(i-m)}}\right]}^2-{\left[J{2}_{m,n}\right]}^2,\hfill \end{array}$$

(17)

where $c_i$ is defined in (5). The convergence in probability of $VJ{1}_{mn}$ and $VJ{2}_{mn}$ to $VJ\left(F\right)$ straightforwardly follows from the consistency of $J{1}_{m,n}$ and $J{2}_{m,n}$.

The rest of this paper is structured as follows. Section 2 presents the proposed Bayesian estimator based on the Dirichlet process. Section 3 details the proposed approach, including a computational algorithm. In Section 4, a test for uniformity is developed. Section 5 presents several examples to illustrate the approach. Finally, Section 6 contains concluding remarks and discussions.

2. Bayesian Estimation of Varentropy and Varextropy

In this section, we derive Bayesian nonparametric estimators for varentropy and varextropy. Define the following two quantities:

$$\begin{array}{cc}\hfill V{H}_{m,N,a}& \hfill =\frac{1}{N}\sum _{i=1}^N{log}^2\left(\frac{Y_{(i+m)}-Y_{(i-m)}}{c_{i,a}}\right)-{\left[H_{m,N,a}\right]}^2\hfill \end{array}$$

(18)

and

$$\begin{array}{cc}\hfill V{J}_{m,N,a}& \hfill =\frac{1}{4N}\sum _{i=1}^N{\left[\frac{c_{i,a}}{Y_{(i+m)}-Y_{(i-m)}}\right]}^2-{\left[J_{m,N,a}\right]}^2,\hfill \end{array}$$

(19)

where $c_{i,a}$, $H_{m,N,a},$ and $J_{m,N,a}$ are defined in (9), (7), and (8), respectively. The following proposition presents the prior formulation of varentropy and varextropy. The proof follows from the consistency of $H_{m,N,a}$ and $J_{m,N,a}$.

Lemma 1.

Let $V{H}_{m,N,a}$ and $V{J}_{m,N,a}$ be defined as in (18) and (19), respectively. Let $P_N$ be an approximation of the $DP(a,G)$ as defined (6). As $N\to \infty$, $m\to \infty$, $\frac{m}{N}\to 0$ and $a\to \infty$, we have

$$\begin{array}{c}\hfill V{H}_{m,N,a}\stackrel{p}{\to }VH\left(G\right)\end{array}$$

and

$$\begin{array}{c}\hfill V{J}_{m,N,a}\stackrel{p}{\to }VJ\left(G\right),\end{array}$$

where $G^{\prime }\left(x\right)=g\left(x\right)$.

The following proposition demonstrates that as the sample size increases (with the concentration parameter a being relatively small compared to the sample size n), the posterior distributions of $V{H}_{m,N,a}$ and $V{J}_{m,N,a}$ converge in probability to $VH\left(F\right)$ and $VJ\left(F\right)$, respectively. The proof follows from consistency of $H_{m,N,a}|\mathit{X}$ and $J_{m,N,a}|\mathit{X}$.

Lemma 2.

Let $\mathit{X}=(X_1,\dots ,X_n)$ be a sample from F. Let the prior on F be $DP(a,H)$. Let $V{H}_{m,N,a}$ and $V{J}_{m,N,a}$ be as defined in (18) and (19), respectively. As $N\to \infty$, $m\to \infty$, $n\to \infty$, $\frac{m}{N}\to 0$, and $\frac{a}{n}\to 0$, we have

$$\begin{array}{c}\hfill V{H}_{m,N,a}|\mathit{X}\stackrel{p}{\to }VH\left(F\right)\end{array}$$

and

$$\begin{array}{c}\hfill V{J}_{m,N,a}|\mathit{X}\stackrel{p}{\to }VJ\left(F\right),\end{array}$$

where $VH\left(F\right)$ and $VJ\left(F\right)$ are defined in (11) and (14), respectively, with $F^{\prime }\left(x\right)=f\left(x\right)$.

3. Computational Algorithms

Let $\mathit{X}=(X_1,\dots ,X_n)$ be a sample from a continuous distribution F. The objective is to approximate $VH\left(F\right)$ and $JH\left(F\right)$ using the approximation discussed in Section 2. To proceed with this approximation, it is important to determine the values of m, a, and G. We begin by considering the choice of m. A commonly used formula, proposed by [29], is given by

$$\begin{array}{c}\hfill m=\lfloor \sqrt{N}+0.5\rfloor ,\end{array}$$

(20)

where ⌊y⌋ denotes the largest integer less than or equal to y. Note that the value of m in (20) is used for the prior. For the posterior, the value of N should be replaced with the number of distinct data points in $P_N|\mathit{X}$, an approximation of $F|\mathit{X}$. It is worth noting that, from (10), if $a/n$ is close to zero, the number of distinct data points in $P_N|\mathit{X}$ will be approximately n.

Regarding the hyperparameters a and G of the Dirichlet process, their selection depends on the specific application of interest. For varentropy and varextropy estimation, any choice of a such that $a/n$ is close to zero should be suitable, regardless of the choice of G. This property is evident from (10), as when $a/n$ approaches 0, the sample will dominate the prior guess G. Consequently, the approach becomes invariant to the choice of G. As an illustrative example, by setting $a=0.01$ and $n=20$ in (10), we obtain

$$G_{\mathit{X}}=0.0005G+0.9995F_n.$$

This suggests a 99.95% likelihood of selecting a sample from the gathered data rather than drawing a new sample from G. To facilitate estimation, we will consider G as the uniform distribution over $(0,1)$ and set $a=0.01$, although alternative choices are certainly possible. Within Section 4, we will include an example that investigates the sensitivity of the approach to the choices of a and G.

For a given observed data set $\mathit{X}=(X_1,X_2,\dots ,X_n)$, we employ the following computational algorithm to estimate $VH\left(F\right)$ and $VJ\left(F\right)$ based on Equations (18) and (19).

Algorithm 1.

(Nonparametric Estimation of Varentropy and Varextropy):

(i) 

Generate a sample from $P_N$, where $P_N$ is an approximation of $DP(a=0.01,G=U(0,1\left)\right)$.

(ii) 

Generate a sample from $P_N|\mathit{X}$, where $P_N|\mathit{X}$ is an approximation of ∼$DP(a+n,G_{\mathit{X}})$.

(iii) 

Compute $V{H}_{m,N,a}|\mathit{X}$ and $V{J}_{m,N,a}|\mathit{X}$ as specified in Lemma 2.

(iv) 

Repeat steps (i) and (iii) to obtain a sample of r values from $V{H}_{m,N,a}|\mathit{X}$ and $V{J}_{m,N,a}|\mathit{X}$. As r increases, the average of the generated r values becomes the estimator for varentropy and varextropy.

4. Testing for Uniformity

Suppose that $\mathit{X}=(X_1,\dots ,X_n)$ is a sample from an unknown continuous distribution F. The objective is to test the hypothesis ${\mathcal{H}}_0:F\left(x\right)=F_0\left(x\right)$, for all $x\in \mathbb{R}$, where $F_0$ represents a fully specified distribution. By utilizing the probability integral transform property, we can deduce that $F\left(X_1\right),\dots ,F\left(X_n\right)$ follows the uniform distribution on the interval $(0,1)$. Therefore, testing the null hypothesis is equivalent to testing ${\mathcal{H}}_0:U\left(x\right)=x$ for all $x\in (0,1)$, where $U\left(x\right)$ represents the CDF of a uniform $(0,1)$ random variable. For further details on testing uniformity, please consult the work of [30].

For any random variable X, it holds that $VH\left(F\right)\ge 0$ and $VJ\left(F\right)\ge 0$. The following proposition demonstrates that equality is achieved when F corresponds to the CDF of a uniform distribution on the interval $(0,1)$. This property plays a crucial role in testing the hypothesis $H_0$.

Proposition 1.

Let f be a probability density function with support in $[0,1]$, we have

(i) 

$VH\left(F\right)=0$ if and only if $f\left(x\right)=1$ for all $x\in (0,1)$ (i.e., f is the PDF of the uniform random variable on $(0,1)$).

(ii) 

$VJ\left(F\right)=0$ if and only if $f\left(x\right)=1$ for all $x\in (0,1)$ (i.e., f is the PDF of the uniform random variable on $(0,1)$).

Proof. 

For the proof of (i), see Theorem 4.1 of [25]. For (ii), if $f\left(x\right)=1$ for all $x\in (0,1)$, then $VJ\left(F\right)=\mathrm{Var}(-0.5f(X\left)\right)=0$. Also, if $VJ\left(F\right)=0$, then $f\left(x\right)=c$, for all $x\in (0,1)$. Since ${\int }_0^{1}f\left(x\right)dx=1$, we have $c=1$. Hence, $f\left(x\right)=1$ for all $x\in (0,1)$.    □

The proposed test for uniformity involves comparing $V{H}_{m,N,a}|\mathit{X}$ and $V{J}_{m,N,a}|\mathit{X}$. When the null hypothesis ${\mathcal{H}}_0$ is true, it is expected that $V{H}_{m,N,a}|\mathit{X}\approx V{J}_{m,N,a}|\mathit{X}\approx 0$. Conversely, if there is evidence that $V{H}_{m,N,a}|\mathit{X}$ or $V{J}_{m,N,a}|\mathit{X}$ deviates significantly from zero, ${\mathcal{H}}_0$ is rejected.

5. Examples

5.1. Simulation Study

In this subsection, we focus on investigating the efficiency and robustness of the proposed estimator for varentropy and varextropy. Additionally, we demonstrate the implementation of the uniformity test using these estimations. To evaluate the performance of our proposed Bayesian estimator, we compare it with the non-Bayesian counterparts obtained from (12), (13), (16), and (17). The comparison between Bayesian and non-Bayesian methods holds particular significance in this context, particularly when we consider the scenario where $a=0.01$. In this case, the estimator remains unaffected by the selection of the prior guess G, as demonstrated in Section 4.

To carry out the computations, we implemented the required program codes in the programming language R, and these codes are made available by the authors. For demonstration purposes, we constructed a demo for the algorithms and presented them using R Shiny, as shown here: https://annaly.shinyapps.io/BayesianVarentropyVarextropy/ (accessed on 15 July 2023). In Algorithm 1, we set the parameters $r=1000$ and $N=500$ to ensure accurate and sufficient evaluations. Additionally, to ensure reproducibility, the set.seed(100) function in R was utilized for all examples.

Throughout this section, we use the following notation: $N(\mu ,{\sigma }^2)$ denotes the normal distribution with mean $\mu$ and standard deviation $\sigma$, $t_r$ represents the t distribution with r degrees of freedom, $\mathrm{Exp}\left(\lambda \right)$ corresponds to the exponential distribution with mean $1/\lambda$, $U(a,b)$ signifies the uniform distribution over the interval $(a,b)$, and $\mathrm{beta}(\alpha ,\beta )$ denotes the beta distribution with parameters $\alpha$ and $\beta$.

In

Table 1. Varentropy Measure Estimates.

			${\mathit{VH}}_{\mathit{m},\mathit{n},\mathit{a}}|\mathit{X}$	$\mathit{VH}{1}_{\mathit{m},\mathit{n}}$	$\mathit{VH}{2}_{\mathit{m},\mathit{n}}$
Distribution	$\mathit{n}$	$\mathit{m}$	Est(RMSE)
U(0, 1)	20	4	0.2055(0.2248)	0.1749(0.218)	0.1099(0.1465)
	50	7	0.1266(0.1334)	0.1059(0.1214)	0.0641(0.0769)
	100	10	0.0919(0.0954)	0.0783(0.0873)	0.0469(0.0536)
Exp(1)	20	4	0.7446(0.4699)	0.6663(0.5105)	0.6586(0.5258)
	50	7	0.9025(0.3148)	0.7929(0.3454)	0.8344(0.3396)
	100	10	0.9671(0.2315)	0.8641(0.2533)	0.9203(0.2417)
$N(0,1)$	20	4	0.2646(0.2621)	0.1221(0.3885)	0.1620(0.3574)
	50	7	0.3051(0.2245)	0.1449(0.3633)	0.2351(0.2864)
	100	10	0.3666(0.1677)	0.2136(0.2971)	0.319(0.2076)

and

Table 2. Varextropy Measure Estimates.

			${\mathit{VJ}}_{\mathit{m},\mathit{n},\mathit{a}}|\mathit{X}$	$\mathit{JQ}{1}_{\mathit{m},\mathit{n}}$	$\mathit{JQ}{2}_{\mathit{m},\mathit{n}}$
Distribution	$\mathit{n}$	$\mathit{m}$	Est(RMSE)
U(0, 1)	20	4	0.0828(0.1159)	0.1698(0.3129)	0.0502(0.0875)
	50	7	0.0404(0.0460)	0.0642(0.0943)	0.0217(0.0291)
	100	10	0.0262(0.0283)	0.0352(0.0460)	0.0135(0.0166)
Exp(1)	20	4	0.0589(0.1451)	0.1419(0.5516)	0.0505(0.1466)
	50	7	0.0370(0.0291)	0.0621(0.0696)	0.0311(0.0238)
	100	10	0.0337(0.0197)	0.0514(0.0435)	0.0294(0.0161)
$N(0,1)$	20	4	0.0063(0.0065)	0.0062(0.0136)	0.0043(0.0047)
	50	7	0.0053(0.0034)	0.0033(0.0022)	0.0041(0.0026)
	100	10	0.0069(0.0044)	0.0037(0.0018)	0.0043(0.0021)

, for each sample size ($n=20,50,100$), 1000 samples were generated. We have considered three distributions: uniform on $(0,1)$ (exact varentropy and varextopy are both 0), exponential with mean 1 (exact varentropy and varextropy are 1 and $1/48\approx 0.0208$, respectively), $N(0,1)$ (exact varentropy and exact varextropy are 0.5 and $(2-\sqrt{3})/16\pi \sqrt{3}\approx 0.0031$, respectively). The estimators and their root mean squared errors are computed and reported in Table 1 and Table 2. The reported value of the estimator (Est) is the average of the 1000 estimates. On the other hand, the root mean squared error (RMSE) is computed as follows: $\sqrt{{\sum }_{i=1}^{1000}{({\mathrm{Est}}_i-\mathrm{true}\mathrm{value})}^2/1000}$, where ${\mathrm{Est}}_i$ is the estimated value based on the ith sample.

Based on the findings presented in Table 1 and Table 2, it is evident that the estimators for varentropy and varextropy demonstrate overall good performance.

It is also of interest to examine the impact of utilizing different base measures G and concentration parameters a on the methodology. To explore this, we consider two distinct values for a, namely 0.01 and 5, and examine various choices for G. In our analysis, we utilize a dataset generated from the exponential distribution with a mean of 1. Based on the findings presented in

Table 3. Analysis of the impact of different values of a and G on the proposed estimators.

G	a	${\mathit{VH}}_{\mathit{m},\mathit{n},\mathit{a}}|\mathit{X}$	${\mathit{VJ}}_{\mathit{m},\mathit{n},\mathit{a}}|\mathit{X}$
$N(0,1)$	0.01	0.8758	0.0663
	5	2.4917	0.0255
$N(3,9)$	0.01	0.8877	0.0667
	5	3.2328	0.0073
$t_1$	0.01	0.8932	0.0681
	5	7.7335	0.0215
$\mathrm{Exp}\left(1\right)$	0.001	0.8587	0.0657
	5	1.6113	0.0725
$U(0,1)$	0.01	0.8699	0.0668
	5	0.7525	0.0776

, it can be concluded that the estimators exhibit robustness to the choice of G when $a=0.01$.

In this last example, we generated samples of sizes $n=20,50,$ and 100 from the uniform distribution on the interval $(0,1)$. The goal is to test the hypothesis $H_0:F\left(x\right)=F_0\left(x\right)$ using the proposed test of uniformity. To achieve this, we considered a range of candidate distribution functions $F_0\left(x\right)$ as outlined in

Table 4. Goodness-of-Fit Test.

n	${\mathit{F}}_0$	Est. of $\mathit{VH}$	Est. of $\mathit{VJ}$	Threshold of $\mathit{VH}$	Threshold of $\mathit{VJ}$	p-Value
20	$A_{1.5}$	0.1573	0.0423	0.2461	0.0955	0.2953
	$A_2$	0.2494	0.0839			0.0442
	$B_{1.5}$	0.1322	0.0337			0.7472
	$B_2$	0.1553	0.0421			0.4646
	$B_3$	0.2934	0.1163			0.1469
	$C_{1.5}$	0.3082	0.1687			0.1958
	$C_2$	0.5543	0.6276			0.0745
	$U(0,1)$	0.1529	0.0484			0.6307
	$\mathrm{Exp}\left(2\right)$	0.2604	0.1169			0.1300
	$\mathrm{beta}(3,1)$	1.1438	2.6605			0.0000
	$N(0,1)$	0.1944	0.3892			0.0000
50	$A_{1.5}$	0.2378	0.1044	0.1469	0.0475	0.0073
	$A_2$	0.6105	0.7342			0.0000
	$B_{1.5}$	0.1176	0.0391			0.4119
	$B_2$	0.2564	0.1421			0.0857
	$B_3$	0.7699	1.6625			0.0048
	$C_{1.5}$	0.2449	0.1046			0.0430
	$C_2$	0.54717	0.4817			0.0045
	$U(0,1)$	0.0876	0.0232			0.3349
	$\mathrm{Exp}\left(2\right)$	0.3819	0.2553			0.0013
	$\mathrm{beta}(3,1)$	1.0189	1.6964			0.0000
	$N(0,1)$	0.1361	0.2849			0.0000
100	$A_{1.5}$	0.2073	0.0873	0.1042	0.03107	0.0019
	$A_2$	0.5906	0.8231			0.0000
	$B_{1.5}$	0.0958	0.0329			0.6042
	$B_2$	0.2602	0.1665			0.0764
	$B_3$	0.9135	3.0551			0.0011
	$C_{1.5}$	0.2877	0.1323			0.0080
	$C_2$	0.7074	0.8427			0.0001
	$U(0,1)$	0.0755	0.0193			0.2672
	$\mathrm{Exp}\left(2\right)$	0.3507	0.1926			0.0000
	$\mathrm{beta}(3,1)$	1.3669	4.1632			0.0000
	$N(0,1)$	0.1006	0.1951			0.0000

, where

$$\begin{array}{cc}{A}_k:F_0\left(x\right)\hfill & =1-{(1-x)}^k,0\le x\le 1(\mathrm{for}k=1.5,2);\hfill \\ B_k:F_0\left(x\right)\hfill & =\left\{\begin{array}{cc}{2}^{k-1}{x}^k\hfill & 0\le x\le 0.5,\hfill \\ 1-2^{k-1}{(1-x)}^k\hfill & 0.5\le x\le 1\hfill \end{array}\right.(\mathrm{for}k=1.5,2,3);\hfill \\ C_k:F_0\left(x\right)\hfill & =\left\{\begin{array}{cc}0.5-2^{k-1}{(0.5-x)}^k\hfill & 0\le x\le 0.5,\hfill \\ 0.5+2^{k-1}{(0.5-x)}^k\hfill & 0.5\le x\le 1\hfill \end{array}\right.(\mathrm{for}k=1.5,2).\hfill \end{array}$$

These candidate distribution functions $A_k,B_k,$ and $C_k$ of $F_0\left(x\right)$ have been previously studied by various authors, including [25,31]. The distributions of $\mathrm{Exp}\left(2\right)$ and $N(0,1)$ are included here to explore cases with support different from $[0,1]$. The results are presented in Table 4, where we also included the p-values obtained from the Kolmogorov–Smirnov test.

Using Monte Carlo simulation, we can determine an appropriate cut-off for both $V{H}_{m,n,a}|\mathit{X}$ and $V{J}_{m,n,a}|\mathit{X}$ in the test of uniformity under $\mathcal{H}0$. We recommend using $Q_3$, the third quartile, as a suitable threshold. In Table 4, we present the thresholds for $n=20,50$, and 100, based on 5000 values of $VHm,n,a|\mathit{X}$ and $V{J}_{m,n,a}|\mathit{X}$. If both estimates are less than their respective thresholds, it is advisable not to reject the null hypothesis ${\mathcal{H}}_0$. However, if one of the estimates is greater than its threshold, it is recommended to reject $H_0$.

For example, when $n=50$ and $F_0=A_{1.5}$, as $V{H}_{m,n,a}|\mathit{X}>0.1469$ (or $V{J}_{m,n,a}|\mathit{X}>0.0475$), $\mathcal{H}0$ is rejected. Conversely, when $F_0=B_{1.5}$, as both $V{H}_{m,n,a}|\mathit{X}<0.1469$ and $V{J}_{m,n,a}|\mathit{X}<0.0475$, ${\mathcal{H}}_0$ is not rejected.

5.2. Real Data Examples

Military Personnel Carriers Dataset [32]: The following data represents mileages for 19 military personnel carriers that failed in service. The mileages are as follows:

• 162, 200, 271, 320, 393, 508, 539, 629, 706, 778, 884, 1003, 1101, 1182, 1463, 1603, 1984, 2355, 2880.

The aim of the study is to test whether the data follows an exponential distribution with a mean of 998. Employing Algorithm 1 with parameters $n=19$, $N=500$, and $r=1000$, we obtained $V{H}_{m,n,a}|\mathit{X}=0.1124$ and $V{J}_{m,n,a}|\mathit{X}=0.0331$. Since both of these values are significantly lower than their respective thresholds (0.2509 for $V{H}_{m,n,a}|\mathit{X}$ and 0.0969 for $V{J}_{m,n,a}|\mathit{X}$), we cannot reject the hypothesis that the failure time is exponentially distributed with a mean of 998. This conclusion aligns with the findings of [33].

Chick Dataset [30]: The dataset below represents the weights of 20 chicks in grams:

• 156, 162, 168, 182, 186, 190, 190, 196, 202, 210, 214, 220, 226, 230, 230, 236, 236, 242, 246, 270.

The goal of this study is to test whether the data follow a normal distribution with a mean of 200 and a variance of 1225. Using Algorithm 1 with parameters $n=20$, $N=500$, and $r=1000$, we obtained $V{H}_{m,n,a}|\mathit{X}=0.1396$ and $V{J}_{m,n,a}|\mathit{X}=0.0000$. Both of these values are significantly lower than their respective thresholds (0.2461 for $V{H}_{m,n,a}|\mathit{X}$ and 0.0955 for $V{J}_{m,n,a}|\mathit{X}$). Consequently, we cannot reject the hypothesis that the data follow a normal distribution with a mean of 200 and a variance of 1225. This conclusion is consistent with the findings of [30].

6. Conclusions

In this paper, we introduced a novel estimator for varentropy and varextropy, drawing inspiration from Bayesian nonparametric statistical methods. This method exhibits flexibility as it does not rely on any specific assumptions about the underlying distribution. Furthermore, we also presented a goodness-of-fit test. Extensive testing and validation of our estimator were conducted using multiple simulated examples and a real-life application. The results clearly demonstrate that our estimator displays favorable and accurate performance.

Moreover, the applicability of our approach is not limited to varentropy and varextropy alone. It is possible to extend the results presented in this paper to study other dispersion indices. For instance, dispersion indices based on Kerridge inaccuracy measure and Kullback–Leibler divergence, as studied by [34], can be explored using a similar Bayesian nonparametric framework.