Kernel-Based Non-Parametric Tests for Exponentiality Against Increasing or Decreasing Residual Entropy Alternatives ^†

Abstract

Testing exponentiality is a fundamental problem in reliability and survival analysis. This paper proposes a kernel-based nonparametric test for exponentiality against monotonic residual uncertainty alternatives. The procedure relies on kernel estimators of Shannon entropy and residual entropy, providing smoother estimation than histogram-based approaches. The test statistic is shown to converge almost surely under the null hypothesis, while critical values are obtained by Monte Carlo simulation. Simulation results under Weibull and Gamma alternatives demonstrate satisfactory finite-sample performance and high power. Pitman efficiency comparisons show that the proposed test performs competitively with existing entropy-based procedures for small sample sizes and stay efficient for large ones.

1. Introduction

The exponential distribution occupies a central position in reliability theory, survival analysis, and lifetime data analysis because of its unique memoryless property and constant hazard rate. It is frequently adopted as a benchmark model for modelling lifetimes of systems and components [1,2,3].

A wide variety of goodness-of-fit procedures for exponentiality have been proposed over the past decades [4]. Among them, information-theoretic methods have received considerable attention owing to their ability to quantify the uncertainty associated with lifetime distributions [5,6,7,8,9]. In particular, Shannon entropy and residual entropy have proved to be powerful tools for describing aging properties and for deriving distributional characterizations [10,11,12,13,14,15,16]. Unlike Shannon entropy, which measures the global uncertainty of a lifetime distribution, residual entropy quantifies the uncertainty associated with the remaining lifetime of a component that has survived up to a given time.

Among entropy-based procedures, Ebrahimi [4] introduced one of the earliest nonparametric tests of exponentiality against decreasing residual uncertainty (DURL) alternatives. His procedure was based on histogram estimation of the density and distribution functions and its asymptotic properties were established under a differentiability framework specific to its characterization. This pioneering work demonstrated that residual entropy provides an effective criterion for detecting departures from the exponential model.

More recently, Benaoudia and Aissani [17,18] proposed a new nonparametric test capable of detecting both decreasing and increasing residual uncertainty (DURL/IURL) alternatives. Their methodology relies on a different characterization of the exponential distribution involving residual entropy. The corresponding test statistic was constructed using histogram estimators and was shown to possess attractive asymptotic and numerical properties.

The present work extends the methodology of Benaoudia and Aissani [18] by replacing the histogram estimators with the kernel-based estimators of the Shannon entropy and residual entropy introduced by Belzunce, Navarro and Ruiz [19]. These entropy estimators are obtained using kernel estimators of the probability density functions, yielding smoother estimation and reducing the dependence on arbitrary histogram partitioning. The proposed approach preserves the original information-theoretic characterization while improving the estimation stage on which the test statistic is built.

The resulting kernel-based statistic, denoted by S, is therefore obtained by replacing the unknown Shannon entropy and residual entropy appearing in the theoretical characterization with their corresponding kernel estimators. Under suitable regularity conditions, the statistic is shown to converge almost surely to its theoretical counterpart. Empirical critical values are computed by Monte Carlo simulation.

The finite-sample performance of the proposed procedure is investigated through an extensive simulation study under Weibull and Gamma alternatives. The results indicate that the proposed kernel-based test exhibits satisfactory finite-sample behavior and high empirical power, for both decreasing and increasing residual uncertainty alternatives. Pitman efficiency comparisons expressed favorable conclusions.

The remainder of the paper is organized as follows. Section 2 briefly recalls the characterization underlying the proposed procedure and describes the construction of the kernel-based statistic. Section 3 summarizes its main asymptotic properties. Section 4 presents the Monte Carlo study, including the empirical critical values, the power analysis and the Pitman efficiency study. Finally, Section 5 concludes the paper and discusses several directions for future research.

2. Construction of the Kernel-Based Test Statistic

The proposed procedure is based on an information-theoretic characterization of the exponential distribution involving Shannon entropy and residual entropy. Let X be a non-negative continuous random variable with probability density function f, cumulative distribution function F, and survival function $\overline{F}=1-F$. The Shannon entropy measures the global uncertainty associated with the lifetime distribution, whereas the residual entropy quantifies the uncertainty remaining after the component has survived up to a given time.

In the present work, we extend the framework of Benaoudia and Aissani 2023 [18] by using the kernel-based entropy and residual entropy estimators introduced by Belzunce, Navarro and Ruiz [19]. This modification preserves the theoretical characterization of the exponential distribution while providing smoother estimates of the entropy functionals involved in the test construction.

More precisely, kernel estimators of the probability density function are first obtained using Gaussian kernel and an appropriate bandwidth. This estimator is then employed to compute the corresponding kernel estimators of Shannon entropy and residual entropy.

The test statistic, denoted by S, is constructed by replacing the unknown Shannon entropy and residual entropy appearing in the theoretical characterization with their corresponding kernel estimates. Therefore, S may be viewed as a kernel estimate of the theoretical functional defining the exponentiality criterion. Under the null hypothesis, the statistic is expected to remain close to its theoretical value, whereas systematic deviations indicate departures from exponentiality. Positive or negative values of the statistic provide evidence in favor of increasing or decreasing residual uncertainty alternatives, respectively.

Let X₁, …, X_n be i.i.d. positive random variables with distribution function F and density f. We test

H₀: F is exponential vs. 

$$H_1:F\in \mathrm{DURL}/\mathrm{IURL}\left(\mathrm{but\; not\; exponential\; distribution}\right).$$

The residual Shannon entropy at time t ≥ 0, as defined by Ebrahimi [14], is

$$H(f;t)=-{\int }_t^{\mathrm{\infty }}{\displaystyle \frac{f\left(x\right)}{\bar{F}\left(t\right)}}\log {\displaystyle \frac{f\left(x\right)}{\bar{F}\left(t\right)}}\hspace{0.17em}\mathit{dx}$$

(1)

where $\bar{F}\left(t\right)=1-F\left(t\right)$ is the survival function.

Under differentiability assumptions, the derivative of the residual entropy is [14]

H^′(f; t) = λ_F(t)[H(f; t) − 1 + logλ_F(t)]

(2)

with ${\lambda }_F\left(t\right)=f\left(t\right)/\bar{F}\left(t\right)$ being the hazard rate. Under H₀ (exponentiality), H^′(f; t) = 0 [15]. We define

g(t) = H(f; t) − 1 + logλ_F(t)

(3)

We know that λ_F(t) ≠ 0,∀t > 0, so that g(t) = 0 under H₀.

Instead of this, we propose the theoretical statistic as the expectation of g(X) [18]:

$$S=\mathbb{E}[g(X)]={\int }_0^{\mathrm{\infty }}g(t)f(t)\hspace{0.17em}dt$$

(4)

As introduced by Belzunce et al. [19], the estimated Shannon entropy and residual entropy are respectively:

$$\widehat{H}=-{\displaystyle \frac{1}{n}}{\displaystyle {\sum }_{i=1}^n}\log {\widehat{f}}_n(X_i)$$

(5)

$$\widehat{H}(t)=\log {\displaystyle \frac{m_t}{n}}-{\displaystyle \frac{1}{m_t}}{\displaystyle {\sum }_{i=1}^n}\mathbb{I}(X_i>t)\log {\widehat{f}}_n(X_i) \mathrm{where}$$

(6)

We estimate the density f using a leave-one-out kernel estimator:

$${\widehat{f}}_n(x_i)={\displaystyle \frac{1}{(n-1)h_n}}{\displaystyle {\sum }_{\begin{array}{c}k=1\\ k\ne i\end{array}}^n}K\left({\displaystyle \frac{x_i-X_k}{h_n}}\right)$$

(7)

where $m_t={\displaystyle {\sum }_{i=1}^n}\mathbb{I}\left(X_i>t\right)$.

K is a kernel function (e.g., Gaussian: $K(u)={\displaystyle \frac{1}{\sqrt{2\pi }}}{e}^{-u^2/2}$), h_n is the bandwidth with the formula $h_n=1.06\sigma n^{{\displaystyle \frac{-1}{5}}}$ [20,21,22], and σ is a standard deviation.

The statistic S is obtained by replacing the unknown quantities involved in the above functional with the corresponding kernel estimators of Shannon entropy and residual entropy.

3. Asymptotic Properties

The theoretical validity of the proposed procedure relies on the asymptotic properties of the kernel estimators of Shannon entropy and residual entropy introduced by Belzunce, Navarro and Ruiz [19]. Since the proposed statistic S_n is constructed from these estimators, its asymptotic behavior follows from their strong consistency together with the convergence properties of the underlying entropy functionals.

Theorem 1.

Under the null hypothesis, assume that the regularity conditions on the kernel function and the bandwidth sequence are satisfied [22]. Then, as the sample size tends to infinity,

$$S_n\stackrel{\mathrm{a}.\mathrm{s}.}{\to }S,$$

where S denotes the theoretical functional characterizing the exponential distribution.

The previous theorem establishes the strong consistency of the proposed statistic and ensures that the kernel-based estimator converges almost surely to its theoretical counterpart.

The above results provide the theoretical justification for the proposed testing procedure. They guarantee that the statistic behaves as expected under the null hypothesis and forms a consistent basis for testing exponentiality against monotonic residual uncertainty alternatives.

4. Monte Carlo Simulation Study

The finite-sample behavior of the proposed statistic was investigated through an extensive Monte Carlo simulation study. Since the exact null distribution of S_n is analytically intractable, empirical critical values were obtained by repeated sampling from the standard exponential distribution. All simulations were performed using GNU Octave, version 11.3.0.

For each sample size, N = 10,000 independent samples were generated. The empirical quantiles of the simulated values of S_n were determined at the significance levels α = 0.10, 0.05, 0.025, and 0.01. Throughout the simulation study, the Gaussian kernel was employed together with the bandwidth specification adopted in the theoretical development.

4.1. Empirical Critical Values

Table 1 presents the empirical critical values of the proposed statistic obtained from the Monte Carlo simulations under the null hypothesis of exponentiality. These critical values constitute the practical decision thresholds for implementing the proposed test in finite samples.

Table 1. Empirical quantiles of Sn and empirical size.

n	0.90	0.95	0.975	0.99	Size (θ = 1)
10	2.2770	2.3610	2.4344	2.5021	0.0514
20	2.4306	2.5069	2.5688	2.6411	0.0521
40	2.6158	2.6739	2.7268	2.7872	0.0485

Under H₀, we have g(t) = 0, whereas under H₁: DURL, g(t) < 0, as follows from the derivative-based criterion stated in [17,18]. Consequently, S_n(k) is equal to zero under H₀ and takes negative values under H₁. Therefore, smaller values of S_n(k) provide evidence in favor of H₁. Equivalently, larger values of −S_n(k) support the alternative hypothesis. Accordingly, H₀ is rejected in favor of H₁ at significance level α whenever −S_n(k) ≥ C_k,_n(α). Here, C_k,_n(α) denotes the critical value obtained from the (1 − α)-quantile of the distribution of −S_n(k) under the exponentiality hypothesis. Under H₁: IURL, g(t) > 0. Therefore, H₀ is rejected in favor of H₁ at significance level α whenever S_n(k) ≥ C_k,_n(1 − α), where C_k,_n(1 − α) denotes the (1 − α)-quantile of the distribution of S_n(k) under the exponentiality hypothesis.

As expected, the critical values become progressively more stable as the sample size increases, reflecting the convergence of the finite-sample distribution of the statistic toward its asymptotic behavior. For θ = 1, corresponding to the exponential null hypothesis, the empirical rejection probabilities are close to the nominal 5% significance level for the DURL settings. For the DURL alternative, the empirical sizes range from 0.0485 to 0.0514. These results indicate overall satisfactory control of the empirical size under the null hypothesis. The same results are obtained for the IURL alternative.

4.2 Empirical Power Analysis

The empirical power of the proposed kernel-based test was investigated under Weibull and Gamma alternatives for different parameter values and sample sizes. These distributions were selected because they represent classical alternatives to exponentiality and generate both decreasing and increasing residual uncertainty behaviors. For θ > 1, both the Weibull and Gamma distributions are IFR, and therefore correspond to DURL alternatives. For θ < 1, they are DFR and hence correspond to IURL alternatives. When θ = 1, both distributions reduce to the exponential distribution Exp(1).

As shown in Figure 1, the empirical power increases steadily with the sample size and rapidly approaches one as the departure from exponentiality becomes more pronounced. These results indicate that the proposed statistic efficiently detects departures associated with Weibull lifetime models.

Figure 1. Empirical power of the proposed DURL test for different values of θ and sample sizes under Weibull alternatives.

A similar behavior is observed under Gamma alternatives. The power of the proposed procedure improves with increasing sample size and remains satisfactory over the range of parameter values considered in the simulation study under increasing residual uncertainty alternatives. In this setting, the proposed statistic exhibits particularly high empirical power, even for moderate sample sizes. This behavior confirms the ability of the kernel-based procedure to detect this class of alternatives.

4.3. Pitman Relative Efficiencies

Let J_n be the statistic proposed by Ebrahimi (1997) [13], with W_n being the statistic introduced by Benaoudia and Aissani (2023) [18]. Comparing W_n and J_n, as shown in Figure 2; the statistic J_n generally outperforms W_n for moderate sample sizes, but asymptoticaly W_n performs better than J_n. Overall, these findings establish S_n as the most efficient test statistic among the three in the Pitman sense, for small sample sizes, whereas W_n performs better for large ones.

Figure 2. Pitman relative efficiencies of S_n compared to J_n (Ebrahimi) and W_n for various sample sizes n, considering DURL test.

The horizontal reference line at y = 1 in Figure 3, indicates equal efficiency between the two statistics. Values above this line show that −S_n is more efficient than −W_n under IURL alternatives, for small sample sizes. The Pitman relative efficiencies obtained for the statistics −S_n and −W_n in testing exponentiality against the IURL alternative show that the kernel test is locally more efficient. In contrast, for the large sample sizes considered, the relative efficiencies are less than unity, showing that the tests based on the histogram method are more efficient than the kernel entropy estimator-based test.

Figure 3. Finite-sample Pitman efficiency of −W_n and −S_n for IURL test.

The relative Pitman efficiency results presented in Figure 4 lead to the same conclusion for IURL alternatives.

Figure 4. Relative Pitman efficiency of −S_n with respect to −W_n under IURL alternatives.

5. Illustrative Example

To illustrate the proposed testing procedure, we use a data set consisting of the survival times of 50 cancer patients reported by Lai and Xie [2]. For this sample, the value of the proposed statistic is found to be S_n = 2.6104. This value exceeds all the critical values reported for the DURL test, leading to the rejection of the null hypothesis of exponentiality in favor of the DURL alternative.

Conversely, the corresponding value of −S_n is smaller than all the critical values associated with the IURL test. These results provide evidence in favor of the DURL alternative, while the IURL alternative is not supported by the data.

6. Conclusion and Future Work

This paper has presented a kernel-based nonparametric test for exponentiality against monotonic residual uncertainty alternatives. The proposed methodology extends the histogram-based procedure introduced by Benaoudia and Aissani (2023) [18], replacing the Shannon entropy and residual entropy with the kernel-based entropy estimators proposed by Belzunce, Navarro and Ruiz (2001) [19]. This modification preserves the underlying information-theoretic characterization of the exponential distribution while providing a smoother and more flexible estimation framework. Its asymptotic validity was justified through strong consistency results, while empirical critical values were obtained by Monte Carlo simulations because of the absence of a closed-form null distribution. The empirical power increases with the sample size and with the degree of departure from exponentiality. The kernel-based statistic exhibits good performance under both monotonic residual uncertainty alternatives. The Pitman efficiency study gives good results compared with competitive tests, but the histogram method gives relatively more efficient tests. Furthermore, the present work demonstrates that kernel estimation can successfully be incorporated into entropy-based exponentiality testing without modifying the underlying theoretical characterization, thereby providing a natural and effective alternative to the histogram-based approach.

Future research will focus on extending the proposed methodology to more general reliability settings, in particular, investigating its application to multivariate lifetime distributions and censored data.