A New Index for Measuring the Non-Uniformity of a Probability Distribution

Abstract

This paper proposes a new index, the “distribution non-uniformity index (DNUI)”, for quantitatively measuring the non-uniformity or unevenness of a probability distribution relative to a baseline uniform distribution. The proposed DNUI is a normalized, distance-based metric ranging between 0 and 1, with 0 indicating perfect uniformity and 1 indicating extreme non-uniformity. It satisfies our axioms for an effective non-uniformity index and is applicable to both discrete and continuous probability distributions. Several examples are presented to demonstrate its application and to compare it with two distance measures, namely, the Hellinger distance (HD) and the total variation distance (TVD), and two classical evenness measures, namely, Simpson’s evenness and Buzas and Gibson’s evenness.

1. Introduction

Non-uniformity, or unevenness, is an inherent characteristic of probability distributions, as outcomes or values from a probability system are typically not distributed uniformly or evenly. Although the shape of a distribution can offer an intuitive sense of its non-uniformity, researchers often require a quantitative measure to assess this property. Such a measure is valuable for constructing distribution models and for comparing the non-uniformity across different distributions in a consistent and interpretable way.

A probability distribution is considered uniform when all outcomes have equal probability, in the discrete case, or when the probability density is constant, in the continuous case. Therefore, the uniform distribution serves as the natural baseline for assessing the non-uniformity of any given distribution, and non-uniformity is referred to as the degree to which a distribution deviates from this uniform benchmark. It is essential to ensure that the distribution being evaluated and the baseline uniform distribution share the same support. This requirement is especially important in the continuous case, where a fixed and clearly defined support is crucial for meaningful comparison.

The Kullback–Leibler (KL) divergence or the χ² divergence may be used as a metric for measuring the non-uniformity of a given distribution by quantifying how different the distribution is from a baseline uniform distribution. For a discrete random variable X with probability mass function (PMF) $P\left(x\right)$ and n possible outcomes, the KL divergence relative to the uniform distribution with PMF 1/n is given by

$$\mathrm{K}\mathrm{L}={\sum }_{i=1}^{n}P\left(x_i\right)\mathrm{l}\mathrm{o}\mathrm{g}{\displaystyle \frac{P\left(x_i\right)}{(1/n)}}=\mathrm{l}\mathrm{o}\mathrm{g}(n)+{\sum }_{i=1}^{n}P\left(x_i\right)\mathrm{l}\mathrm{o}\mathrm{g}P\left(x_i\right)$$

(1)

The χ² divergence is given by

$${\mathrm{\chi }}^2 \mathrm{d}\mathrm{i}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{g}\mathrm{e}\mathrm{n}\mathrm{c}\mathrm{e}= {\sum }_{i=1}^n{\displaystyle \frac{{P\left(x_i\right)}^2}{\left(\frac{1}{n}\right)}}-1=n{\sum }_{i=1}^n{P\left(x_i\right)}^2-1$$

(2)

While a KL or χ² divergence value of zero indicates perfect uniformity, there is no natural upper bound that allows us to specify how non-uniform a distribution is. Furthermore, as shown in Equations (1) and (2), the KL or χ² divergence will tend to infinity as the number of possible outcomes (n) goes to infinity, regardless of the distribution (except for the uniform distribution). The lack of an upper bound can make interpretation difficult, especially when comparing different distributions or when the scale of the divergence matters. Therefore, we will not discuss the KL and χ² divergence further in this paper.

The Hellinger distance (HD) and the total variation distance (TVD), two well-known distances, may be used to measure the non-uniformity of a given distribution relative to a baseline uniform distribution. For the discrete case, the HD, as a non-uniformity measure, is given by (relative to a uniform distribution with PMF 1/n)

$$\mathrm{H}\mathrm{D}=\sqrt{\frac{1}{2}{\sum }_{i=1}^n{\left(\sqrt{P\left(x_i\right)}-\frac{1}{\sqrt{n}}\right)}^2 }$$

(3)

The TVD as a non-uniformity measure is given by

$$\mathrm{T}\mathrm{V}\mathrm{D}=\frac{1}{2}{\sum }_{i=1}^n|P\left(x_i\right)-\frac{1}{n}|$$

(4)

The HD and TVD range between 0 and 1 and do not require standardization or normalization. This is a desirable property for non-uniformity metrics. However, to the best of the author’s knowledge, the HD and TVD have not been used to measure distribution non-uniformity. Therefore, their performance is unknown.

In recent work, Rajaram et al. [1,2] proposed a measure called the “degree of uniformity (DOU)” to quantify how evenly the probability mass or density is distributed across available outcomes or support. Specifically, they defined the DOU for a partial distribution on a fixed interval as the ratio of the exponential of the Shannon entropy to the coverage probability of that interval [1,2].

$$\mathrm{D}\mathrm{O}\mathrm{U}={\displaystyle \frac{D_{\mathrm{P}}}{c_{\mathrm{P}}}} ={\displaystyle \frac{1}{c_{\mathrm{P}}}}\mathrm{e}\mathrm{x}\mathrm{p}(H_{\mathrm{P}})$$

(5)

where the subscript “P” denotes “part”, referring to the partial distribution on the fixed interval, $c_{\mathrm{P}}$ is the coverage probability of the interval, $H_P$ is the entropy of the partial distribution, and $D_{\mathrm{P}} = \mathrm{e}\mathrm{x}\mathrm{p}\left(H_{\mathrm{P}}\right)$ is the entropy-based diversity of the partial distribution. When the entire distribution is considered, $c_{\mathrm{P}} = 1$, and thus, the DOU equals the entropy-based diversity $\mathrm{e}\mathrm{x}\mathrm{p}\left(H\right)$. It should be noted that the DOU is neither standardized nor normalized and does not explicitly measure the deviation of a given distribution from a uniform benchmark. Therefore, we will not discuss the DOU further in this paper.

Classical evenness measures, such as Simpson’s evenness and Buzas and Gibson’s evenness, are essentially diversity ratios. For a discrete random variable X with PMF $P\left(x\right)$ and n possible outcomes, Simpson’s evenness is defined as (e.g., [3])

$$E_{S2}={\displaystyle \frac{1/{\sum }_{i=1}^n{\left[P\left(x_i\right)\right]}^2}{n}}$$

(6)

where $1/{\sum }_{i=1}^n{\left[P\left(x_i\right)\right]}^2$ is called Simpson’s diversity, representing the effective number of distinct elements in the probability system $\{X, P(x\left)\right\}$, and n is the maximum diversity that corresponds to a uniform distribution with PMF 1/n. The concept of effective number is the core of diversity measures in biology [4].

Buzas and Gibson’s evenness is defined as [5].

$$E_{BG}={\displaystyle \frac{\mathrm{e}\mathrm{x}\mathrm{p}\left[H\left(X\right)\right]}{\mathrm{e}\mathrm{x}\mathrm{p}\left[\mathrm{l}\mathrm{n}\right(n\left)\right]}}={\displaystyle \frac{\mathrm{e}\mathrm{x}\mathrm{p}\left[H\left(X\right)\right]}{n}}$$

(7)

where $H\left(X\right)$ is the Shannon entropy of X, $H\left(X\right) =-{\sum }_{i=1}^{n}P\left(x_i\right)\ln P\left(x_i\right)$, and $\mathrm{l}\mathrm{n}\left(n\right)$ is the entropy of the baseline uniform distribution. The exponential of the Shannon entropy $\mathrm{e}\mathrm{x}\mathrm{p}\left[H\left(X\right)\right]$ is the entropy-based diversity and is also considered to be an effective number of elements in the probability system $\{X, P(x\left)\right\}$.

Both $E_{S2}$ and $E_{BG}$ are normalized by n, the maximum diversity corresponding to the baseline uniform distribution. Therefore, these indices range between 0 and 1, with 0 indicating extreme unevenness and 1 indicating perfect evenness. Since evenness is negatively correlated with unevenness, we consider the complement of $E_{S2}$ and $E_{BG}$ as unevenness (i.e., non-uniformity) indices. That is, we denote ${(1-E}_{S2})$ as Simpson’s unevenness and $(1-E_{BG})$ as Buzas and Gibson’s unevenness, with 0 indicating perfect evenness (uniformity) and 1 indicating extreme unevenness (non-uniformity).

However, as Gregorius and Gillet [6] pointed out, “Diversity-based methods of assessing evenness cannot provide information on unevenness, since measures of diversity generally do not produce characteristic values that are associated with states of complete unevenness.” This limitation arises because diversity measures are primarily designed to capture internal distribution characteristics, such as concentration and relative abundance within the distribution. For example, the quantity ${\sum }_{i=1}^n{\left[P\left(x_i\right)\right]}^2$ is often called the “repeat rate” [7] or Simpson concentration [4]; it has historically been used as a measure of concentration [7]. Moreover, since diversity metrics are not constructed within a comparative distance framework, they inherently lack the ability to quantify deviations from uniformity in a meaningful or interpretable way. This limitation significantly diminishes their effectiveness when the goal is specifically to detect or describe high degrees of non-uniformity.

The aim of this study is to develop a new normalized, distance-based index that can effectively quantify the non-uniformity or unevenness of a probability distribution. In the following sections, Section 2 describes the proposed distribution non-uniformity index (DNUI). Section 3 presents several examples to compare the proposed NDUI with the Hellinger distance (HD), the total variation distance (TVD), Simpson’s unevenness, and Buzas and Gibson’s unevenness. Section 4 and Section 5 provide discussion and conclusion, respectively.

2. The Proposed Distribution Non-Uniformity Index (DNUI)

The mathematical formulation of the proposed distribution non-uniformity index (DNUI) differs for discrete and continuous random variables.

2.1. Discrete Cases

Consider a discrete random variable X with PMF$P\left(x\right)$ and n possible outcomes. Let $X_U$ denote the uniform distribution with the same possible outcomes, so that its PMF $P_U\left(x\right)$ $= \frac{1}{n}$ for all x. We use this uniform distribution as the baseline for measuring the non-uniformity of the distribution of X.

The difference between the two PMFs $P\left(x\right)$ and $P_U\left(x\right)$ is given by

$${\Delta }_{P(x)}=P(x)-P_U(x)=P(x)-\frac{1}{n}$$

(8)

Thus, $P(x)$ can be written as

$$P\left(x\right)={\Delta }_{P\left(x\right)}+\frac{1}{n}$$

(9)

Taking squares on both sides of Equation (9) yields

$${P\left(x\right)}^2={\Delta }_{P\left(x\right)}^2+{\displaystyle \frac{2}{n}}{\Delta }_{P\left(x\right)}+{\displaystyle \frac{1}{n^2}}$$

(10)

Then, taking the expectation on both sides of Equation (10) yields

$${\mathrm{E}[P(x)}^2]=\mathrm{E}\left({\Delta }_{P\left(x\right)}^2\right)+{\displaystyle \frac{2}{n}}\mathrm{E}\left({\Delta }_{P\left(x\right)}\right)+{\displaystyle \frac{1}{n^2}}={\mathsf{\omega }}_{P\left(x\right)}^2+{\displaystyle \frac{1}{n^2}}$$

(11)

In Equation (11), the second moment ${\mathrm{E}\left[P\right(x)}^2]$ is expressed as the sum of the total variance ${\mathsf{\omega }}_{P\left(x\right)}^2$ and the baseline term 1/n², where ${\omega }_{P\left(x\right)}$ is called the total deviation given by

$${\omega }_{P(x)}=\sqrt{\mathrm{E}({\Delta }_{P(x)}^2)+{\displaystyle \frac{2}{n}}\mathrm{E}({\Delta }_{P(x)})}$$

(12)

where $\mathrm{E}\left({\Delta }_{P\left(x\right)}^2\right)$ is the variance of $P\left(x\right)$ relative to the baseline uniform distribution, given by

$$\mathrm{E}\left({\Delta }_{P\left(x\right)}^2\right)=\mathrm{E}\{{\left[P\right(x)-P_U(x\left)\right]}^2\}={\sum }_{i=1}^n{P\left(x_i\right)[P\left(x_i\right)-{\displaystyle \frac{1}{n}}]}^2$$

(13)

$\mathrm{E}\left({\Delta }_{P\left(x\right)}\right)$ is the bias of $P\left(x\right)$ relative to the baseline uniform distribution, given by

$$\mathrm{E}\left({\Delta }_{P\left(x\right)}\right)=\mathrm{E}\left[P\left(x\right)-P_U\left(x\right)\right]={\sum }_{i=1}^n{P\left(x_i\right)}^2-\frac{1}{n}=\beta \left(X\right)-\frac{1}{n}$$

(14)

where $\beta \left(X\right) = {\sum }_{i=1}^n{P\left(x_i\right)}^2$ is called the (discrete) informity of X in the theory of informity proposed by Huang [8], which is the expectation of the PMF $P\left(x\right)$. The informity of the baseline uniform distribution of $X_U$ is $\beta \left(X_U\right) = \mathrm{E}[P_U(x)]=\frac{1}{n}$. Therefore, $\mathrm{E}\left({\Delta }_{P\left(x\right)}\right)$ is the difference between the two discrete informities.

Definition 1. 

The proposed DNUI (denoted by $\rho \left(X\right)$) for the distribution of X is given by

$$\rho \left(X\right)={\displaystyle \frac{{\omega }_{P(x)}}{\sqrt{{\mathrm{E}[P(x)}^2]}}}=\sqrt{{\displaystyle \frac{{\mathrm{E}[P(x)}^2]-\frac{1}{n^2}}{{\mathrm{E}[P(x)}^2]}}}=\sqrt{{\displaystyle \frac{\mathrm{E}({\Delta }_{P(x)}^2)+\frac{2}{n}\mathrm{E}({\Delta }_{P(x)})}{\mathrm{E}({\Delta }_{P(x)}^2)+\frac{2}{n}\mathrm{E}({\Delta }_{P(x)})+\frac{1}{n^2}}}}$$

(15)

where  $\sqrt{{\mathrm{E}[P(x)}^2]}$  is the root mean square (RMS) of  $P(x)$. The second moment  ${\mathrm{E}[P(x)}^2]$  can be calculated as

$${\mathrm{E}[P(x)}^2]={\sum }_{i=1}^n{P\left(x_i\right)P\left(x_i\right)}^2={\sum }_{i=1}^n{P\left(x_i\right)}^3$$

(16)

2.2. Continuous Cases

Consider a continuous random variable Y with probability density function (PDF) $p\left(y\right)$ defined on an unbounded support, such as $(-\infty ,\infty$). Since there is no baseline uniform distribution defined over an unbounded support, we cannot measure the non-uniformity of the entire distribution. Instead, we examine parts of the distribution on a fixed interval ${[y}_1,y_2]$, which allows us to assess local non-uniformity.

According to Rajaram et al. [1], the PDF of a partial distribution on ${[y}_1,y_2]$ is given by the renormalization of the original PDF

$$p^{\prime }\left(y\right)={\displaystyle \frac{p\left(y\right)}{P_{{(y}_1,y_2)}}}$$

(17)

where $P_{{(y}_1,y_2)}={\int }_{y_1}^{y_2}p\left(y\right)dy$, which is the coverage probability of the interval ${[y}_1,y_2]$.

Let $Y_U$ denote the uniform distribution on ${[y}_1,y_2]$ with PDF $p_U\left(y\right)$ $={\displaystyle \frac{1}{{{(y}_2-y}_1)}}$. We use this uniform distribution as the baseline for measuring the non-uniformity of the partial distribution.

Similar to the discrete case, the difference between the two PDFs $p^{\prime }\left(y\right)$ and $p_U\left(y\right)$ is given by

$${\Delta }_{p^{\prime }\left(y\right)}=p^{\prime }\left(y\right)-p_U\left(y\right)=p^{\prime }\left(y\right)-{\displaystyle \frac{1}{{{(y}_2-y}_1)}}$$

(18)

Thus, $p^{\prime }\left(y\right)$ can be written as

$$p^{\prime }\left(y\right)={\Delta }_{p^{\prime }\left(y\right)}+{\displaystyle \frac{1}{{{(y}_2-y}_1)}}$$

(19)

Taking squares on both sides of Equation (19) yields

$${p^{\prime }\left(y\right)}^2={\Delta }_{p^{\prime }\left(y\right)}^2+{\displaystyle \frac{2}{{{(y}_2-y}_1)}}{\Delta }_{p^{\prime }\left(y\right)}+{\displaystyle \frac{1}{{{{(y}_2-y}_1)}^2}}$$

(20)

Then, taking the expectation on both sides of Equation (20) yields

$${\mathrm{E}[p^{\prime }\left(y\right)}^2]=\mathrm{E}\left({\Delta }_{p^{\prime }\left(y\right)}^2\right)+{\displaystyle \frac{2}{{{(y}_2-y}_1)}}\mathrm{E}\left({\Delta }_{p^{\prime }\left(y\right)}\right)+{\displaystyle \frac{1}{{{{(y}_2-y}_1)}^2}}={\mathsf{\omega }}_{p^{\prime }\left(y\right)}^2+{\displaystyle \frac{1}{{{{(y}_2-y}_1)}^2}}$$

(21)

The total deviation ${\omega }_{p^{\prime }\left(y\right)}$ is given by

$${\omega }_{p^{\prime }\left(y\right)}=\sqrt{\mathrm{E}({\Delta }_{p^{\prime }\left(y\right)}^2)+{\displaystyle \frac{2}{{{(y}_2-y}_1)}}\mathrm{E}\left({\Delta }_{p^{\prime }\left(y\right)}\right)}$$

(22)

where $\mathrm{E}\left({\Delta }_{p^{\prime }\left(y\right)}^2\right)$ is the variance in $p^{\prime }\left(y\right)$ relative to $p_U\left(y\right)$, given by

$$\mathrm{E}\left({\Delta }_{p^{\prime }\left(y\right)}^2\right)=\mathrm{E}\{{[p^{\prime }\left(y\right)-p_U(y\left)\right]}^2\}={\int }_{y_1}^{y_2}{\displaystyle \frac{p\left(y\right)}{P_{{(y}_1,y_2)}}}{\left[{\displaystyle \frac{p\left(y\right)}{P_{{(y}_1,y_2)}}}-{\displaystyle \frac{1}{{{(y}_2-y}_1)}}\right]}^{2}dy$$

(23)

and $\mathrm{E}\left({\Delta }_{p^{\prime }\left(y\right)}\right)$ is the bias of $p^{\prime }\left(y\right)$ relative to $p_U\left(y\right)$, given by

$$\mathrm{E}\left({\Delta }_{p^{\prime }\left(y\right)}\right)=\mathrm{E}\left[p^{\prime }\left(y\right)-p_U\left(y\right)\right]={\int }_{y_1}^{y_2}{\displaystyle \frac{p\left(y\right)}{P_{{(y}_1,y_2)}}}\left[{\displaystyle \frac{p\left(y\right)}{P_{{(y}_1,y_2)}}}-{\displaystyle \frac{1}{{{(y}_2-y}_1)}}\right]dy$$

(24)

Definition 2. 

The proposed DNUI for the partial distribution on ${[y}_1,y_2]$ (denoted by $\rho (y_1,y_2)$) is given by

$$\rho \left(y_1,y_2\right)={\displaystyle \frac{{\omega }_{p^{\prime }\left(y\right)}}{\sqrt{{\mathrm{E}[p^{\prime }\left(y\right)}^2]}}}=\sqrt{{\displaystyle \frac{{\mathrm{E}[p^{\prime }\left(y\right)}^2]-\frac{1}{{{{(y}_2-y}_1)}^2}}{{\mathrm{E}[p^{\prime }\left(y\right)}^2]}}}=\sqrt{{\displaystyle \frac{\mathrm{E}({\Delta }_{p^{\prime }\left(y\right)}^2)+\frac{2}{{{(y}_2-y}_1)}\mathrm{E}({\Delta }_{p^{\prime }\left(y\right)})}{\mathrm{E}({\Delta }_{p^{\prime }\left(y\right)}^2)+\frac{2}{{{(y}_2-y}_1)}\mathrm{E}({\Delta }_{p^{\prime }\left(y\right)})+\frac{1}{{{{(y}_2-y}_1)}^2}}}}$$

(25)

where  ${\mathrm{E}[p^{\prime }\left(y\right)}^2]$  is the second moment of  $p^{\prime }\left(y\right)$, given by

$${\mathrm{E}[p^{\prime }\left(y\right)}^2]={\int }_{y_1}^{y_2}{p}^{\prime }\left(y\right){\left[p^{\prime }\left(y\right)\right]}^{2}dy={\int }_{y_1}^{y_2}{\left[{\displaystyle \frac{p\left(y\right)}{P_{{(y}_1,y_2)}}}\right]}^{3}dy$$

(26)

Definition 3. 

If the continuous distribution is defined on the fixed support $[-a,a]$, $P_{(-a,a)}=1$ and ${{(y}_2-y}_1)=2a$, the proposed DNUI for the entire distribution of Y (denoted by $\rho \left(Y\right)$) is given by

$$\rho \left(Y\right)={\displaystyle \frac{{\omega }_{p\left(y\right)}}{\sqrt{{\mathrm{E}[p\left(y\right)}^2]}}}=\sqrt{{\displaystyle \frac{{\mathrm{E}[p\left(y\right)}^2]-\frac{1}{4a^2}}{{\mathrm{E}[p\left(y\right)}^2]}}}=\sqrt{{\displaystyle \frac{\mathrm{E}({\Delta }_{p\left(y\right)}^2)+\frac{1}{a}\mathrm{E}({\Delta }_{p\left(y\right)})}{\mathrm{E}({\Delta }_{p\left(y\right)}^2)+\frac{1}{a}\mathrm{E}({\Delta }_{p\left(y\right)})+\frac{1}{4a^2}}}}$$

(27)

where  ${\mathrm{E}[p\left(y\right)}^2]$  is the second moment of  $p\left(y\right)$, given by

$${\mathrm{E}[p\left(y\right)}^2]={\int }_{-a}^a{p\left(y\right)}^{3}dy$$

(28)

the variance  $\mathrm{E}\left({\Delta }_{p\left(y\right)}^2\right)$  is given by

$$\begin{gathered} \mathrm{E}({\Delta }_{p\left(y\right)}^2)={\int }_{-a}^a{p\left(y\right)\left[p\left(y\right)-{\displaystyle \frac{1}{2a}}\right]}^{2}dy \\ ={\int }_{-a}^a{p\left(y\right)}^{3}dy-{\displaystyle \frac{1}{a}}{\int }_{-a}^a{p\left(y\right)}^{2}dy+{\displaystyle \frac{1}{{4a}^2}} \end{gathered}$$

(29)

and the bias  $\mathrm{E}\left({\Delta }_{p\left(y\right)}\right)$  is given by

$$\mathrm{E}({\Delta }_{p\left(y\right)})=\mathrm{E}[p\left(y\right)-p_U(y)]={\int }_{-a}^{a}p\left(y\right)[p\left(y\right)-{\displaystyle \frac{1}{2a}}]dy={\int }_{-a}^a{p\left(y\right)}^{2}dy-{\displaystyle \frac{1}{2a}}$$

(30)

The quantity ${\int }_{-a}^a{p\left(y\right)}^{2}dy$ is denoted by $\beta \left(Y\right)$ and is called the continuous informity of Y in the theory of informity [8]. The continuous informity of the baseline uniform distribution of $Y_U$ is $\beta \left(Y_U\right)=\mathrm{E}[p_U(y)]={\displaystyle \frac{1}{2a}}$. Therefore, $\mathrm{E}\left({\Delta }_{p\left(y\right)}\right)$ is the difference between the two continuous informities.

3. Examples

3.1. Coin Tossing

Consider tossing a coin, which is a simplest two-state probability system: {X; P(x)} = {head, tail; P(head), P(tail)}, where $P\left(\mathrm{t}\mathrm{a}\mathrm{i}\mathrm{l}\right)=1-P\left(\mathrm{h}\mathrm{e}\mathrm{a}\mathrm{d}\right)$. The DNUI for the distribution of X is given by

$$\rho \left(X\right)=\sqrt{{\displaystyle \frac{{\mathrm{E}\left[P\right(x)}^2]-\frac{1}{2^2}}{{\mathrm{E}\left[P\right(x)}^2]}}}$$

(31)

where the second moment ${\mathrm{E}\left[P\right(x)}^2]$ can be calculated as

$${\mathrm{E}\left[P\right(x)}^2]={\left[P\left(\mathrm{h}\mathrm{e}\mathrm{a}\mathrm{d}\right)\right]}^3+{\left[P\left(\mathrm{t}\mathrm{a}\mathrm{i}\mathrm{l}\right)\right]}^3$$

(32)

Figure 1 shows the DNUI for the distribution of X as a function of the bias represented by $P\left(\mathrm{h}\mathrm{e}\mathrm{a}\mathrm{d}\right)$. The HD, TVD, ${(1-E}_{S2})$, and $(1-E_{BG})$ are also shown in Figure 1 for comparison.

Figure 1. The DNUI for the distribution of X as a function of the bias represented by the probability of heads, compared with the Hellinger distance (HD), the total variation distance (TVD), Simpson’s unevenness ${(1-E}_{S2})$, and Buzas and Gibson’s unevenness $(1-E_{BG}).$

As shown in Figure 1, when the coin is fair (i.e., $P\left(\mathrm{h}\mathrm{e}\mathrm{a}\mathrm{d}\right) = P\left(\mathrm{t}\mathrm{a}\mathrm{i}\mathrm{l}\right) = 0.5$), the DNUI, HD, and TVD, ${(1-E}_{S2})$, and $(1-E_{BG})$ are all 0, indicating perfect uniformity or evenness. As the coin becomes increasingly biased toward either heads or tails, all indices increase. In the extreme case where $P\left(\mathrm{t}\mathrm{a}\mathrm{i}\mathrm{l}\right)=1$ or $P\left(\mathrm{h}\mathrm{e}\mathrm{a}\mathrm{d}\right)=1$, the DNUI reaches a maximum value of $\rho \left(X\right)=0.866$, reflecting a high degree of non-uniformity. However, the HD reaches a maximum value of 0.541, and the TVD, ${(1-E}_{S2})$, and $(1-E_{BG})$ reach a maximum value of 0.5, which are significantly smaller than 1, indicating that these indices fail to capture the high degree of non-uniformity.

3.2. Three Frequency Data Series

JJC [9] posted a question on Cross Validated about quantifying distribution non-uniformity. He supplied three frequency datasets (Series A, B, and C), each containing 10 values (Table 1). Visually, Series A is almost perfectly uniform, Series B is nearly uniform, and Series C is heavily skewed by a single outlier (0.6). Table 1 lists these datasets alongside the corresponding DNUI, HD, TVD, ${(1-E}_{S2})$, and $(1-E_{BG})$ values.

Table 1. Three frequency data series and the corresponding DNUI, HD, TVD, ${(1-E}_{S2})$, and $(1-E_{BG})$ values.

Series	$\mathit{\rho }\left(\mathit{X}\right)$	HD	TVD	${1-\mathit{E}}_{\mathit{S}2}$	$1-{\mathit{E}}_{\mathit{B}\mathit{G}}$
A: {0.1, 0.11, 0.1, 0.09, 0.09, 0.11, 0.1, 0.1, 0.12, 0.08}	0.1864	0.0389	0.04	0.0119	0.0060
B: {0.1, 0.1, 0.1, 0.08, 0.12, 0.12, 0.09, 0.09, 0.12, 0.08}	0.2499	0.0524	0.06	0.0215	0.0109
C: {0.03, 0.02, 0.6, 0.02, 0.03, 0.07, 0.06, 0.05, 0.05, 0.07}	0.9767	0.4121	0.5	0.7375	0.5455

From Table 1, we can see that the DNUI value for Series A is 0.1864, confirming its high uniformity, while the DNUI value for Series B is 0.2499, indicating near-uniformity. In contrast, the DNUI value for Series C is 0.9767 (close to 1), signaling extreme non-uniformity. These results align well with intuitive expectations. The HD, TVD, ${(1-E}_{S2})$, and $(1-E_{BG})$ values range from 0.0060 to 0.04 for Series A and from 0.0109 to 0.06 for Series B, which may be considered to reflect the uniformity of these two series fairly well. However, the HD, TVD, ${(1-E}_{S2})$, and $(1-E_{BG})$ values range from 0.4121 to 0.7375 for Series C, which are too high to adequately reflect the severity of the non-uniformity.

3.3. Five Continuous Distributions with Fixed Support $[-a,a]$

Consider five continuous distributions with fixed support $[-a,a]$: uniform, triangular, quadratic, raised cosine, and half-cosine. Table 2 summarizes their PDFs, variances, biases, second moments, and DNUIs.

Table 2. The PDF $p\left(y\right)$, variance $\mathrm{E}({\Delta }_{p\left(y\right)}^2)$, bias $\mathrm{E}({\Delta }_{p\left(y\right)})$, second moment ${\mathrm{E}[p\left(y\right)}^2]$, and DNUI $\rho \left(Y\right)$ for five continuous distributions with fixed support $[-a,a].$

Distribution	$\mathit{p}\left(\mathit{y}\right)$	$\mathbf{E}\left({\Delta }_{\mathit{p}\left(\mathit{y}\right)}^2\right)$	$\mathbf{E}\left({\Delta }_{\mathit{p}\left(\mathit{y}\right)}\right)$	${\mathbf{E}[\mathit{p}\left(\mathit{y}\right)}^2]$	$\mathit{\rho }\left(\mathit{Y}\right)$
Uniform	${\displaystyle \frac{1}{2a}}$	0	0	${\displaystyle \frac{1}{{4a}^2}}$	0
Triangular	$\left\{\begin{array}{c}{\displaystyle \frac{(y+a)}{a^2}}, -a\le y\le 0\\ {\displaystyle \frac{(a-y)}{a^2}}, 0\le y\le a\\ \end{array}\right.$	${\displaystyle \frac{1}{{12a}^2}}$	${\displaystyle \frac{1}{6a}}$	${\displaystyle \frac{1}{2a^2}}$	0.7071
Quadratic	${\displaystyle \frac{3}{4a}}\left[1-{\left({\displaystyle \frac{1}{a}}y\right)}^2\right]$	${\displaystyle \frac{1}{{28a}^2}}$	${\displaystyle \frac{1}{10a}}$	${\displaystyle \frac{27}{70a^2}}$	0.5932
Raised cosine	${\displaystyle \frac{1}{2a}}\left[1+\cos \left({\displaystyle \frac{\pi }{a}}y\right)\right]$	${\displaystyle \frac{1}{{8a}^2}}$	${\displaystyle \frac{1}{4a}}$	${\displaystyle \frac{5}{8a^2}}$	0.7746
Half-cosine	${\displaystyle \frac{\pi }{4a}}\cos \left({\displaystyle \frac{\pi }{2a}}y\right)$	$\left({\displaystyle \frac{1}{4}}-{\displaystyle \frac{{\pi }^2}{48}}\right){\displaystyle \frac{1}{a^2}}$	$\left({\displaystyle \frac{{\pi }^2}{16}}-{\displaystyle \frac{1}{2}}\right){\displaystyle \frac{1}{a}}$	${\displaystyle \frac{1}{{4a}^2}}$	0.6262

As shown in Table 2, the DNUI is independent of the scale parameter a, which is a desirable property for a measure of distribution non-uniformity. By definition, the DNUI for the uniform distribution is 0. In contrast, the DNUI values for the other four distributions range from 0.5932 to 0.7746, indicating moderate to high non-uniformity. These results align well with intuitive expectations. Notably, the raised cosine distribution has the highest DNUI value among the five distributions, suggesting it exhibits the greatest non-uniformity.

3.4. Exponential Distribution

The PDF of the exponential distribution with support $[0,\infty )$ is

$$p\left(y\right)=\lambda e^{-\lambda y}$$

(33)

where $\lambda$ is the shape parameter.

We consider a partial exponential distribution on the interval [$0,b]$ (i.e., $y_1=0$ and $y_2=b$), and b is the length of the interval. Thus, the DNUI for the partial exponential distribution is given by

$$\rho \left(0,b\right)=\sqrt{{\displaystyle \frac{{\mathrm{E}[p^{\prime }\left(y\right)}^2]-\frac{1}{b^2}}{{\mathrm{E}[p^{\prime }\left(y\right)}^2]}}}$$

(34)

where the second moment ${\mathrm{E}[p^{\prime }\left(y\right)}^2]$ is given by

$${\mathrm{E}[p^{\prime }\left(y\right)}^2]={\displaystyle \frac{1}{{{[P}_{(0,b)}]}^3}}{\int }_0^b{p\left(y\right)}^{3}dy$$

(35)

The coverage probability of the interval [$0,b]$ is given by

$$P_{(0,b)}={\int }_0^b\lambda exp(-\lambda y)dy=1-e^{-\lambda b}$$

(36)

The integral ${\int }_0^b{p\left(y\right)}^{3}dy$ can be solved as

$${\int }_0^b{p\left(y\right)}^{3}dy={\int }_0^b{\left\{\lambda e^{-\lambda y})\right\}}^{3}dy={\lambda }^3{\int }_0^b{e}^{-3\lambda y}dy={\displaystyle \frac{{\lambda }^2}{3}}(1-e^{-3\lambda b})$$

(37)

Figure 2 shows the plot of the DNUI for the partial exponential distribution with $\lambda =1$ as a function of the interval length b. It also shows the PDF of the original exponential distribution, Equation (33) with $\lambda =1$, as a function of y.

Figure 2. Plots of the DNUI for the partial exponential distribution with $\lambda =1$ and the PDF of the original exponential distribution.

As shown in Figure 2, when the interval length b is very small (approaching 0), the DNUI is close to 0, reflecting the high local uniformity within small intervals. As the interval length b increases, the DNUI also increases, indicating the growing local non-uniformity with larger intervals. When the interval length b becomes very large, the DNUI approaches 1, indicating that the distribution over a large interval is extremely non-uniform. These observations align well with intuitive expectations.

4. Discussion

4.1. Axioms for an Effective Non-Uniformity Index

It is important to note that non-uniformity indices require an axiomatic foundation to ensure their validity and meaningful interpretation. This foundation should be built upon a set of axioms that any acceptable non-uniformity index should satisfy. We propose the following four axioms for an effective non-uniformity index:

• Normalization: The index should range between 0 and 1 (or approximately), with 0 indicating perfect uniformity and 1 (or near 1) indicating extreme non-uniformity.
• Sensitivity to Deviations: The index should be sensitive to any deviations from a baseline uniform distribution, producing a value that reflects the extent of non-uniformity.
• Consistency and Comparability: The index should yield consistent results when applied to similar distributions and enable comparisons across different distributions.
• Intuitive Interpretation: The index should be easy to understand and interpret, providing a clear indication of how close a distribution is to perfect uniformity.

Of the eight non-uniformity measures evaluated in this paper, the DOU, KL divergence, and χ² divergence fail to meet Axiom 1 (normalization), as noted in the Introduction. The Hellinger distance (HD), total variation distance (TVD), Simpson’s unevenness ${(1-E}_{S2})$, and Buzas and Gibson’s unevenness $(1-E_{BG})$ do not satisfy Axiom 2 (sensitivity to deviations), as demonstrated in Examples 3.1 and 3.2. Only the proposed NDUI satisfies all four axioms, making it a robust and effective measure.

4.2. Normalization, Benchmarks for Defining Non-Uniformity Levels, and Invariance to Probability Permutations

The definition of the proposed NUDI is both mathematically sound and intuitively interpretable. It is a normalized, distance-based metric derived from the total deviation defined in Equations (11) and (21). Importantly, this total deviation incorporates two components, namely, variance and bias, both measured relative to the baseline uniform distribution. The particular normalization using the second moment of the PMF or PDF provides a natural and robust scaling factor, which ensures that the NDUI consistently reflects deviations from uniformity across diverse distributions while maintaining a normalized range of [0, 1], as demonstrated in the presented examples.

The proposed DNUI ranges between 0 and 1, with 0 indicating perfect uniformity and 1 indicating extreme non-uniformity. Lower DNUI values (near 0) suggest a more uniform or flatter distribution, while higher values (near 1) suggest a greater degree of non-uniformity or unevenness. Since there are no universally accepted benchmarks for defining levels of non-uniformity, we tentatively propose DNUI values of 0.25, 0.5, and 0.75 to represent low, moderate, and high non-uniformity, respectively. The proposed thresholds are determined by the DNUI’s normalized [0, 1] range, which approximately divide it into quartiles and aligns with the empirical DNUI values observed in Examples 3.1 and 3.2, where values near 0.25 indicate minor deviations, 0.5 indicate moderate deviations, and 0.75 or higher indicate significant deviations from uniformity.

Note that the DNUI (similar to other indices) depends solely on the probability values and not on the associated outcomes (or scores) or their order. This property can be illustrated using the frequency data from Series C in Section 3.2: {0.03, 0.02, 0.6, 0.02, 0.03, 0.07, 0.06, 0.05, 0.05, 0.07}. If, for example, the second and third values are swapped, the DNUI value remains unchanged. Therefore, the DNUI is not a one-to-one function of the distribution; it can “collapse” different distributions into the same value. This property is analogous to how different distributions can share the same mean or variance. The invariance of the DNUI to probability permutations implies that it may not distinguish distributions with identical probability sets but different arrangements, suggesting that in applications like clustering or anomaly detection, the DNUI should be complemented with order-sensitive metrics when structural differences are critical.

4.3. Upper Bounds of Non-Uniformity Indices in the Discrete Case

In the discrete case, when X follows a uniform distribution, the DNUI, HD, TVD, ${(1-E}_{S2})$, and $(1-E_{BG})$ are all 0 regardless of the number of possible outcomes. However, in the extreme case, where all outcomes have probability 0 except one with probability 1, the upper bound of these indices depends on the number of possible outcomes. The upper bound of the DNUI is given by

$${\rho \left(X\right)}_{\mathrm{u}\mathrm{p}\mathrm{p}\mathrm{e}\mathrm{r}-\mathrm{b}\mathrm{o}\mathrm{u}\mathrm{n}\mathrm{d}}=\sqrt{1-{\displaystyle \frac{1}{n^2}}}$$

(38)

The upper bound of the HD is given by

$${\mathrm{H}\mathrm{D}}_{\mathrm{u}\mathrm{p}\mathrm{p}\mathrm{e}\mathrm{r}-\mathrm{b}\mathrm{o}\mathrm{u}\mathrm{n}\mathrm{d}}=\sqrt{{\displaystyle \frac{1}{2}}{\sum }_{i=1}^{n-1}{\displaystyle \frac{1}{n}}+{\displaystyle \frac{1}{2}}{\left(1-{\displaystyle \frac{1}{\sqrt{n}}}\right)}^2}=\sqrt{{\displaystyle \frac{n-1}{2n}}+{\displaystyle \frac{1}{2}}{\left(1-{\displaystyle \frac{1}{\sqrt{n}}}\right)}^2}$$

(39)

The upper bound of the TVD is given by

$${\mathrm{T}\mathrm{V}\mathrm{D}}_{\mathrm{u}\mathrm{p}\mathrm{p}\mathrm{e}\mathrm{r}-\mathrm{b}\mathrm{o}\mathrm{u}\mathrm{n}\mathrm{d}}={\displaystyle \frac{1}{2}}{\sum }_{i=1}^{n-1}{\displaystyle \frac{1}{n}}+{\displaystyle \frac{1}{2}}(1-\frac{1}{n})=1-\frac{1}{n}$$

(40)

The upper bound of ${(1-E}_{S2})$ is given by

$${{(1-E}_{S2})}_{\mathrm{u}\mathrm{p}\mathrm{p}\mathrm{e}\mathrm{r}-\mathrm{b}\mathrm{o}\mathrm{u}\mathrm{n}\mathrm{d}}=1-\frac{1}{n}$$

(41)

The upper bound of $(1-E_{BG})$ is given by

$${(1-E_{BG})}_{\mathrm{u}\mathrm{p}\mathrm{p}\mathrm{e}\mathrm{r}-\mathrm{b}\mathrm{o}\mathrm{u}\mathrm{n}\mathrm{d}}=1-\frac{1}{n}$$

(42)

Note that the TVD, ${(1-E}_{S2})$, and $(1-E_{BG})$ have the same upper bound. Figure 3 shows plots of the upper bounds of the five indices as functions of the number of possible outcomes. It can be seen that among the five indices, the NDUI has the largest upper bound at n = 2 (where the upper bound is minimum), which increases rapidly to 1 as n increases. In contrast, the other indices have a very low upper bound at n = 2, less than 0.541, which increases slowly to 1 as n increases. Our common sense tells us that this extreme case represents a very high degree of non-uniformity and should be represented by an index value of 1 or close to 1. Therefore, the NDUI performs best among the five non-uniformity indices.

Figure 3. Plots of the upper bounds of the five indices as functions of the number of possible outcomes.

5. Conclusions

Four axioms for an effective non-uniformity index are proposed: normalization, sensitivity to deviations, consistency and comparability, and intuitive interpretation. Among the eight non-uniformity measures evaluated in this paper, the degree of uniformity (DOU), KL divergence, and χ² divergence fail to satisfy Axiom 1 (normalization). The Hellinger distance (HD), total variation distance (TVD), Simpson’s unevenness ${(1-E}_{S2})$, and Buzas and Gibson’s unevenness $(1-E_{BG})$ do not satisfy Axiom 2 (sensitivity to deviations). Only the proposed NDUI satisfies all four axioms.

The proposed DNUI provides an effective metric for quantifying the non-uniformity or unevenness of probability distributions. It is applicable to any distribution, discrete or continuous, defined on a fixed support. It can also be applied to partial distributions on fixed intervals to examine local non-uniformity, even when the overall distribution has unbounded support. The presented examples have demonstrated the effectiveness of the proposed DNUI in capturing and quantifying distribution non-uniformity.

It is important to emphasize that the NDUI, as a normalized and axiomatically grounded measure of non-uniformity, could be applied to fields such as ecological modeling, information theory, and machine learning. For example, the NDUI’s sensitivity to deviations and intuitive interpretation could support its use as an alternative to diversity-based evenness measures in evenness/unevenness analysis in ecology. The scope of application of the NDUI needs to be further studied and expanded.