Interval Entropy and Informative Distance

Abstract

The Shannon interval entropy function as a useful dynamic measure of uncertainty for two sided truncated random variables has been proposed in the literature of reliability. In this paper, we show that interval entropy can uniquely determine the distribution function. Furthermore, we propose a measure of discrepancy between two lifetime distributions at the interval of time in base of Kullback-Leibler discrimination information. We study various properties of this measure, including its connection with residual and past measures of discrepancy and interval entropy, and we obtain its upper and lower bounds.

1. Introduction

Recently, information theory has attracted the attention of statisticians. Sunoj et al. [1] have explored the use of information measures for doubly truncated random variables which plays a significant role in studying the various aspects of a system when it fails between two time points. In reliability theory and survival analysis, the residual entropy was considered in Ebrahimi and Pellerey [2], which basically measures the expected uncertainty contained in remaining lifetime of a system. The residual entropy has been used to measure the wear and tear of components and to characterize, classify and order distributions of lifetimes by Belzunce et al. [3] and Ebrahimi [4]. The notion of past entropy, which can be viewed as the entropy of the inactivity time of a system, was introduced in Di Crescenzo and Longobardi [5].

Ebrahimi and Kirmani [6] introduced the residual discrimination measure and studied the minimum discrimination principle. Di Crescenzo and Longobardi [7] have considered the past discrepancy measure and presented a characterization of the proportional reversed hazards model. Furthermore, the use of information measures for doubly truncated random variables was explored by Misagh and Yari [8,9]. In this paper, continuing their work, we propose a new measure of discrepancy between two doubly truncated life distributions. The remaining of this paper is organized as follows: in Section 2, some results, including uniqueness of interval entropy and its invariance property are presented. Section 3 is devoted to definitions of dynamic measures of discrimination, including residual and past lifetimes and also the notion of interval discrimination measure is introduced. In Section 4 we present some results and properties of interval entropy and interval discrimination measures. Some conclusions are given in Section 4. Throughout this paper we consider absolutely continuous random variables.

2. Interval Entropy

Let $X$ be a non-negative random variable describing a system failure time. We denote the probability density function of $X$ as $f\left(x\right)$, the failure distribution as $F\left(x\right)=P\left(X\le x\right)$ and the survival function as $\overline{F}\left(x\right)=P\left(X>x\right)$. The Shannon [10] information measure of uncertainty is defined as:

$$H\left(X\right)=-E\left(\log f\left(X\right)\right)=-{\int }_0^{\infty }f\left(x\right)\log f\left(x\right)dx$$

(1)

where log denotes the natural logarithm. Ebrahimi and Pellerey [2] considered the residual entropy of the non-negative random variable $X$ at time $t$ as:

$$H_X\left(t\right)=-{\int }_t^{\infty }\frac{f\left(x\right)}{\overline{F}\left(t\right)}\log \frac{f\left(x\right)}{\overline{F}\left(t\right)}dx$$

(2)

Given that a system has survived up to time $t$, $H_X\left(t\right)$ essentially measures the uncertainty represented by the remaining lifetime. The residual entropy has been used to measure the wear and tear of systems and to characterize, classify and order distributions of lifetimes. See Belzunce et al. [3], Ebrahimi [4] and Ebrahimi and Kirmani [6]. Di Crescenzo and Longobardi [5] introduced the notion of past entropy and motivated its use in real-life situations. They also discussed its relationship with the residual entropy. Formally, the past entropy of $X$ at time $t$ is defined as follows:

$${\overline{H}}_X\left(t\right)=-{\int }_0^t\frac{f\left(x\right)}{F\left(t\right)}\log \frac{f\left(x\right)}{F\left(t\right)}dx$$

(3)

Given that the system $X$ has failed at time $t$, ${\overline{H}}_X\left(t\right)$ measures the uncertainty regarding its past lifetime. Now Recall that the probability density function of $\left(X|t_1<X<t_2\right)$ for all $0<t_1<t_2$ is given by $f\left(x\right)/\left(F\left(t_2\right)-F\left(t_1\right)\right)$. Sunoj et al. [1] considered the notion of interval entropy of $X$ in the interval $\left(t_1,t_2\right)$ as the uncertainty contained in $\left(X|t_1<X<t_2\right)$ which is denoted by:

$$IH\left(t_1,t_2\right)=-{\int }_{t_1}^{t_2}\frac{f\left(x\right)}{F\left(t_2\right)-F\left(t_1\right)}\log \frac{f\left(x\right)}{F\left(t_2\right)-F\left(t_1\right)}dx$$

(4)

We can rewrite the interval entropy as:

$$\begin{array}{c}IH\left(t_1,t_2\right)=1-\frac{1}{F\left(t_2\right)-F\left(t_1\right)}{\int }_{t_1}^{t_2}f\left(x\right)\log r\left(x\right)dx\\ +\frac{1}{F\left(t_2\right)-F\left(t_1\right)}\{\overline{F}\left(t_2\right)\log \overline{F}\left(t_2\right)-F\left(t_1\right)\log F\left(t_1\right)\\ + \left[F\left(t_2\right)-F\left(t_1\right)\right]\log \left[F\left(t_2\right)-F\left(t_1\right)\right] \}\end{array}$$

where $r\left(x\right)=f\left(x\right)/\overline{F}\left(x\right)$ is the hazard function of $X$. Note that interval entropy can be negative and also it can be $-\infty$ or $+\infty$. Given that a system has survived up to time $t_1$, and has been found to be down at time $t_2$, $IH\left(t_1,t_2\right)$ measures the uncertainty about its lifetimes between $t_1$ and $t_2$. Misagh and Yari [9] introduced a shift-dependent version of $IH\left(t_1,t_2\right)$. The entropy (4) has been used to characterize and ordering random lifetime distributions. See Misagh and Yari [8] and Sunoj et al. [1].

The general characterization problem is to obtain when the interval entropy uniquely determines the distribution function. The following proposition attempts to solve this problem. We first give definition of general failure rate (GFR) functions extracted from Navarro and Ruiz [11].

Definition 2.1. The GFRs of a random variable $X$ having density function $f\left(x\right)$ and cumulative distribution function $F\left(x\right)$ are given by $h_1^X\left(t_1,t_2\right)=\frac{f\left(t_1\right)}{F\left(t_2\right)-F\left(t_1\right)}$ and $h_2^X\left(t_1,t_2\right)=\frac{f\left(t_2\right)}{F\left(t_2\right)-F\left(t_1\right)}$.

Remark 2.1. GFR functions determine distribution function uniquely. See Navarro and Ruiz [11].

Proposition 2.1. Let $X$ be a non-negative random variable, and assume $IH\left(t_1,t_2\right)$ be increasing with respect to $t_1$ and decreasing with respect to $t_2$, then $IH\left(t_1,t_2\right)$ uniquely determines $F\left(x\right)$.

Proof. By differentiating $IH\left(t_1,t_2\right)$ with respect to $t_j$, we have:

$$\frac{\partial I{H}_X\left(t_1,t_2\right)}{\partial t_1}=h_1\left(t_1,t_2\right)[I{H}_X\left(t_1,t_2\right) -1+\log h_1\left(t_1,t_2\right)]$$

and:

$$\frac{\partial I{H}_X\left(t_1,t_2\right)}{\partial t_2}=-h_2\left(t_1,t_2\right)[I{H}_X\left(t_1,t_2\right)-1+\log h_2\left(t_1,t_2\right)]$$

Thus, for fixed $t_1$ and arbitrary $t_2$, $h_1\left(t_1,t_2\right)$ is a positive solution of the following equation:

$$g\left(x_{t_2}\right)=x_{t_2}\left[I{H}_X\left(t_1,t_2\right)-1+\log x_{t_2}\right]-\frac{\partial IH\left(t_1,t_2\right)}{\partial t_1}=0$$

(5)

Similarly, for fixed $t_2$ and arbitrary $t_1$, we have $h_2\left(t_1,t_2\right)$ as a positive solution of the following equation:

$$\gamma \left(y_{t_1}\right)=y_{t_1}\left[I{H}_X\left(t_1,t_2\right)-1+\log y_{t_1}\right]+\frac{\partial IH\left(t_1,t_2\right)}{\partial t_2}=0$$

(6)

By differentiating $g$ and $\gamma$ with respect to $x_{t_2}$ and $y_{t_1},$ we get $\frac{\partial g\left(x_{t_2}\right)}{\partial x_{t_2}}=\log x_{t_2}+IH\left(t_1,t_2\right)$, and $\frac{\partial \gamma \left(y_{t_1}\right)}{\partial y_{t_1}}=\log y_{t_1}+IH\left(t_1,t_2\right).$ Furthermore, second-order derivatives of $g$ and $\gamma$ with respect to $x_{t_2}$ and $y_{t_1}$ are $\frac{1}{x_t}>0$ and $\frac{1}{y_t}>0$ respectively. Then the functions $g$ and $\gamma$ are minimized at points $x_{t_2}=e^{-IH\left(t_1,t_2\right)}$ and $y_{t_1}=e^{-IH\left(t_1,t_2\right)}$ respectively. In addition, $g\left(0\right)=-\frac{\partial IH\left(t_1,t_2\right)}{\partial t_1}<0 , g(\infty )=\infty$ and $\gamma \left(0\right)=-\frac{\partial IH\left(t_1,t_2\right)}{\partial t_1}<0 , \gamma (\infty )=\infty$. So, both functions $g$ and $\gamma$ first decrease and then increase with respect to $x_{t_2}$ and $y_{t_1}$ respectively. which conclude that Equations (5) and (6) has unique roots $h_1\left(t_1,t_2\right)$ and $h_2\left(t_1,t_2\right)$ respectively. Now, $IH\left(t_1,t_2\right)$ uniquely determines GFRs and by virtue of Remark 2.1, the distribution function.

The effect of monotone transformations on the residual and past entropy has been discussed in Ebrahimi and Kirmani [6] and Di Crescenzo and Longobardi [5] respectively. Following proposition gives similar results for interval entropy.

Proposition 2.2. Suppose $X$ be a non-negative random variable with cumulative distribution function $F$ and survival function $\overline{F}$; let $Y=\phi \left(X\right)$, with $\phi$, strictly increasing and differentiable function. Then for all $0<t_1<t_2<\infty$:

$$\begin{array}{c}I{H}_Y\left(t_1,t_2\right)=I{H}_X\left({\phi }^{-1}\left(t_1\right),{\phi }^{-1}\left(t_2\right)\right)\\ +\frac{1}{F\left({\phi }^{-1}\left(t_2\right)\right)-F\left({\phi }^{-1}\left(t_1\right)\right)}\{ E\left(\log {\phi }^{\prime }\left(X\right)\right) \\ -F\left({\phi }^{-1}\left(t_1\right)\right)E\left(\log {\phi }^{\prime }\left(X\right)|X<{\phi }^{-1}\left(t_1\right)\right)\\ -\overline{F}\left({\phi }^{-1}\left(t_2\right)\right)E\left(\log {\phi }^{\prime }\left(X\right)|X>{\phi }^{-1}\left(t_2\right)\right) \}\end{array}$$

Proof. Recalling (1), The Shannon information of $X$ and $Y$ can be expressed as:

$$H\left(Y\right)=H\left(X\right)+E\left(\log {\phi }^{\prime }\left(X\right)\right)$$

(7)

From Theorem 2 of Ebrahimi and Kirmani [6] and Proposition 2.4 of Di Crescenzo and Longobardi [5], we have:

$$H_Y\left(t_1\right)=H_X\left({\phi }^{-1}\left(t_2\right)\right)+E\left(\log {\phi }^{\prime }\left(X\right)|X>{\phi }^{-1}\left(t_1\right)\right)$$

(8)

and:

$${\overline{H}}_Y\left(t_1\right)={\overline{H}}_X\left({\phi }^{-1}\left(t_1\right)\right)+E\left(\log {\phi }^{\prime }\left(X\right)|X<{\phi }^{-1}\left(t_1\right)\right)$$

(9)

Due to Equation 2.8 of Sunoj et al. [1], there holds:

$$\begin{array}{c}H\left(Y\right)=\mathscr{H}\left(G\left(t_1\right), \overline{G}\left(t_2\right),1-G\left(t_1\right)-\overline{G}\left(t_2\right)\right)\\ +G\left(t_1\right){\overline{H}}_Y\left(t_1\right)+\overline{G}\left(t_2\right)H_Y\left(t_2\right)\\ +\left[1-G\left(t_1\right)-\overline{G}\left(t_2\right)\right] I{H}_Y\left(t_1,t_2\right)\end{array}$$

(10)

where $G$ and $\overline{G}$ denote distribution and survival functions of $Y$ respectively. Substituting $H\left(Y\right)$, ${\overline{H}}_Y\left(t_1\right)$ and $H_Y\left(t_1\right)$ in (7), (8) and (9) into terms of (10), we get:

$$\begin{gathered} H\left(X\right)+ E\left(\log {\phi }^{\prime }\left(X\right)\right)=\left(F\left({\phi }^{-1}\left(t_2\right)\right)-F\left({\phi }^{-1}\left(t_1\right)\right)\right)I{H}_Y\left(t_1,t_2\right) \\ -F\left({\phi }^{-1}\left(t_1\right)\right)E\left(\log {\phi }^{\prime }\left(X\right)|X<{\phi }^{-1}\left(t_1\right)\right) \\ -\overline{F}\left({\phi }^{-1}\left(t_2\right)\right)E\left(\log {\phi }^{\prime }\left(X\right)|X>{\phi }^{-1}\left(t_2\right)\right) \\ +F\left({\phi }^{-1}\left(t_1\right)\right){\overline{H}}_X\left({\phi }^{-1}\left(t_1\right)\right) \\ +\overline{F}\left({\phi }^{-1}\left(t_2\right)\right)H_X\left({\phi }^{-1}\left(t_2\right)\right) \\ +\mathscr{H}(F\left({\phi }^{-1}\left(t_1\right)\right),\overline{F}\left({\phi }^{-1}\left(t_2\right)\right),F\left({\phi }^{-1}\left(t_2\right)\right)-F\left({\phi }^{-1}\left(t_1\right)\right)) \end{gathered}$$

(11)

Three terms of the right hand side of (11) are equal to:

$$\left[F\left({\phi }^{-1}\left(t_2\right)\right)-F\left({\phi }^{-1}\left(t_1\right)\right)\right]I{H}_X\left({\phi }^{-1}\left(t_1\right),{\phi }^{-1}\left(t_2\right)\right)$$

and the proof is complete.

Remark 2.2. Suppose $\phi \left(X\right)=F\left(X\right)$, then the function $\phi$ satisfies the assumptions of Proposition 2.2 and uniformly distributed over $\left(0,1\right)$, then:

$$\begin{gathered} \begin{array}{c}I{H}_{F\left(X\right)}\left(t_1,t_2\right)=I{H}_X\left(F^{-1}\left(t_1\right),F^{-1}\left(t_2\right)\right)\\ -\frac{1}{t_2-t_1}\{ H\left(X\right)+t_{1}E\left(\log f\left(X\right)|X<F^{-1}\left(t_1\right)\right) \\ +t_{2}E\left(\log f\left(X\right)|X>F^{-1}\left(t_2\right)\right) \}\end{array} \end{gathered}$$

Remark 2.3. For all $0<\theta <t_1$, we get $I{H}_{X+\theta }\left(t_1,t_2\right)=I{H}_X\left(t_1-\theta ,t_2-\theta \right)$.

Remark 2.4. Let $Y=aX$ where $a>0$, then, we have $I{H}_{aX}\left(t_1,t_2\right)=I{H}_X\left(\frac{t_1}{a},\frac{t_2}{a}\right)+\log a.$

3. Informative Distance

In this section, we review some basic definitions and facts for measures of discrimination between two residual and past lifetime distributions. We introduce a measure of discrepancy between two random variables at an interval of time.

Let $X$ and $Y$ are two non-negative random variables describing times to failure of two systems. We denote the probability density functions of $X$ and $Y$ as $f\left(x\right)$ and$g\left(y\right)$, failure distributions as $F\left(x\right)=P\left(X\le x\right)$ and $G\left(y\right)=G\left(Y\le y\right)$ and the survival functions as $\overline{F}\left(x\right)=P\left(X>x\right)$ and $\overline{G}\left(y\right)=G\left(Y>y\right)$ respectively, with $F\left(0\right)=G\left(0\right)=1$. Kullback-Leibler [12] informative distance between $F$ and $G$ is defined by:

$$I_{X,Y}={\int }_0^{\infty }f\left(x\right)\log \frac{f\left(x\right)}{g\left(x\right)}dx$$

(12)

where log denotes natural logarithm. $I_{X,Y}$ is known as relative entropy and it is shift and scale invariant. However it is not metric, since symmetrization and triangle inequality does not hold. We point out the Jensen-Shannon divergence (JSD) which is based on the Kullback-Leibler divergence, with the notable differences that it is always a finite value and its square root is a metric. See Nielsen [13] and Amari et al. [14]. The application of $I_{X,Y}$ as an informative distance in residual and past lifetimes has increasingly studied in recent years. In particular, Ebrahimi and Kirmani [6] considered the residual Kullback-Leibler discrimination information of non-negative lifetimes of the systems $X$ and $Y$ at time $t$ as:

$$I_{X,Y}\left(t\right)={\int }_t^{\infty }\frac{f\left(x\right)}{\overline{F}\left(t\right)}\log \frac{f\left(x\right)/\overline{F}\left(t\right)}{g\left(x\right)/\overline{G}\left(t\right)}dx$$

(13)

Given that both systems have survived up to time $t$, $I_{X,Y}\left(t\right)$ identifies with the relative entropy of remaining lifetimes $\left(X|X>t\right)$ and $\left(Y|Y>t\right).$ Furthermore, the Kullback-Leibler distance for two past lifetimes was studied in Di Crescenzo and Longobardi [7] which is dual to (13) in the sense that it is an informative distance between past lifetimes $\left(X|X<t\right)$ and $\left(Y|Y<t\right)$. Formally, the past Kullback-Leibler distance of non-negative random lifetimes of the systems $X$ and $Y$ at time $t$ is defined as:

$${\overline{I}}_{X,Y}\left(t\right)={\int }_0^t\frac{f\left(x\right)}{F\left(t\right)}\log \frac{f\left(x\right)/F\left(t\right)}{g\left(x\right)/G\left(t\right)}dx$$

(14)

Given that at time $t$, both systems have been found to be down, ${\overline{I}}_{X,Y}\left(t\right)$ measures the informative distance between their past lives.

Along a similar line, we define a new discrepancy measure that completes studying informative distance between two random lifetimes.

Definition 3.1. The interval distance between random lifetimes $X$ and $Y$ at interval $\left(t_1,t_2\right)$ is the Kullback-Leibler discrimination measure between the truncated lives $\left(X|t_1<X<t_2\right)$ and $\left(Y|t_1<Y<t_2\right)$:

$$I{D}_{X,Y}\left(t_1,t_2\right)={\int }_{t_1}^{t_2}\frac{f\left(x\right)}{F\left(t_2\right)-F\left(t_1\right)}\log \frac{f\left(x\right)/\left[F\left(t_2\right)-F\left(t_1\right)\right]}{g\left(x\right)/\left[G\left(t_2\right)-G\left(t_1\right)\right]}dx$$

(15)

Remark 3.1. Clearly $I{D}_{X,Y}\left(0,t\right)={\overline{I}}_{X,Y}\left(t\right)$, $I{D}_{X,Y}\left(t,\infty \right)=I_{X,Y}\left(t\right)$ and $I{D}_{X,Y}\left(0,\infty \right)=I_{X,Y}$.

Given that both systems $X$ and $Y$ have survived up to time $t_1$ and have seen to be down at time $t_2$, $I{D}_{X,Y}\left(t_1,t_2\right)$ measures the discrepancy between their unknown failure times in the interval $\left(t_1,t_2\right)$. $I{D}_{X,Y}\left(t_1,t_2\right)$ satisfies all properties of Kullback-Leibler discrimination measure and can be rewritten as:

$$I{D}_{X,Y}\left(t_1,t_2\right)=-{\int }_{t_1}^{t_2}\frac{f\left(x\right)}{F\left(t_2\right)-F\left(t_1\right)}\log \frac{g\left(x\right)}{G\left(t_2\right)-G\left(t_1\right)}dx-I{H}_X\left(t_1,t_2\right)$$

(16)

where $I{H}_X\left(t_1,t_2\right)$ is the interval entropy of $X$ in (4).

An alternative way of writing (16) is the following:

$$I{D}_{X,Y}\left(t_1,t_2\right)=\log \frac{G\left(t_2\right)-G\left(t_1\right)}{F\left(t_2\right)-F\left(t_1\right)}+ \frac{1}{F\left(t_2\right)-F\left(t_1\right)}{\int }_{t_1}^{t_2}f\left(x\right)\log \frac{f\left(x\right)}{g\left(x\right)}dx$$

(17)

The following example clarifies the effectiveness of the interval discrimination measure.

Example 3.1. Suppose $X$ and $Y$ be random lifetimes of two systems with joint density function:

$$f\left(x,y\right)=\frac{1}{4}, 0<x<2, 0<y<4-2x$$

and that the marginal densities of $X$ and $Y$ are $f\left(x\right)=\frac{1}{2}\left(2-x\right) , 0<x<2$ and $g\left(y\right)=\frac{1}{8}\left(4-y\right),$$0<y<4,$ respectively. Because $X$ and $Y$, belongs to different domains, using relative entropy to measure the informative distance between $X$ and $Y$ is not interpretable. The interval distance between $X$ and $Y$ in the intervals $\left(0,1.5\right)$ and $\left(1.5,2\right)$ are 0.01 and 0.16 respectively. Hence, the informative distance between $X$ and $Y$ in the interval $\left(1.5,2\right)$ is greater than of it in the interval $\left(0,1.5\right)$.

In the following proposition we decompose the Kullback-Leibler discrimination function in terms of residual, past and interval discrepancy measure. The proof is straightforward.

Proposition 3.1. Let $X$ and $Y$ are two non-negative random lifetimes of two systems. For all $0\le t_1<t_2<\infty ,$ the Kullback-Leibler discrimination measure is decomposed as follows:

$$I_{X,Y}=\left[F\left(t_2\right)-F\left(t_1\right)\right]I{D}_{X,Y}\left(t_1,t_2\right)+F\left(t_1\right){\overline{I}}_{X,Y}\left(t_1\right)+ \overline{F}\left(t_2\right)I_{X,Y}\left(t_2\right)+I_{U,V}\left(t_1,t_2\right)$$

(18)

where:

$$I_{U,V}\left(t_1,t_2\right)=F\left(t_1\right)\log \frac{F\left(t_1\right)}{G\left(t_1\right)}+\overline{F}\left(t_2\right)\log \frac{\overline{F}\left(t_2\right)}{\overline{G}\left(t_2\right)}+\left[F\left(t_2\right)-F\left(t_1\right)\right]\log \frac{F\left(t_2\right)-F\left(t_1\right)}{G\left(t_2\right)-G\left(t_1\right)}$$

is the Kullback-Leibler distance between two trivalent discrete random variables.

Proposition 3.1 admits the following interpretation: the Kullback-Leibler discrepancy measure between random lifetimes of systems $X$ and $Y$ is composed from four parts: (i) the discrepancy between the past lives of two systems at time $t_1$; (ii) the discrepancy between residual lifetimes of $X$ and $Y$ that have both survived up to time $t_2$; (iii) the discrepancy between the lifetimes of both systems in the interval $\left(t_1,t_2\right)$; (iv) the discrepancy between two random variables which determines if the systems have been found to be failing before $t_1$, between $t_1$ and $t_2$ or after $t_2$.

4. Some Results on Interval Based Measures

In this section we study the properties of $ID\left(t_1,t_2\right)$ and point out certain similarities with those of $I_{X,Y}\left(t\right)$ and ${\overline{I}}_{X,Y}\left(t\right)$. The following proposition gives lower and upper bounds for the interval distance. We first give definition of likelihood ratio ordering.

Definition 4.1. $X$is said to be larger than$Y$in likelihood ratio ($X\stackrel{LR}{\ge }Y$) if $\frac{f\left(x\right)}{g\left(x\right)}$ is increasing in $x$ over the union of the supports of $X$ and $Y$.

Several results regarding the ordering in Definition 4.1. was provided in Ebrahimi and Pellerey [2].

Proposition 4.1. Let $X$ and $Y$ are random variables with common support $\left(0,\infty \right)$. Then:

(i) $X\stackrel{LR}{\ge }Y$ implies:

$$\log \frac{h_1^X\left(t_1,t_2\right)}{h_1^Y\left(t_1,t_2\right)}\le I{D}_{X,Y}\left(t_1,t_2\right)\le \log \frac{h_2^X\left(t_1,t_2\right)}{h_2^Y\left(t_1,t_2\right)}$$

(19)

when $\frac{f\left(x\right)}{g\left(x\right)}$ is decreasing in $x>0$, then the inequalities in (19) are reversed.

(ii) Decreasing $g\left(x\right)$ in $x>0$, implies:

$$\log \frac{1}{h_1^Y\left(t_1,t_2\right)}\le I{D}_{X,Y}\left(t_1,t_2\right)+I{H}_X\left(t_1,t_2\right)\le \log \frac{1}{h_2^Y\left(t_1,t_2\right)}$$

(20)

for increasing $g\left(x\right)$ then the inequalities in (20) are reversed.

Proof. Because of increasing $\frac{f\left(x\right)}{g\left(x\right)}$ in $>0$, from (15), we have:

$$I{D}_{X,Y}\left(t_1,t_2\right)\le {\displaystyle {\int }_{t_1}^{t_2}\frac{f(x)}{F(t_2)-F(t_1)}}\log \frac{f(t_2)/\left[F(t_2)-F(t_1)\right]}{g(t_2)/\left[G(t_2)-G(t_1)\right]}dx=\log \frac{h_2^X\left(t_1,t_2\right)}{h_2^Y\left(t_1,t_2\right)}$$

and:

$$I{D}_{X,Y}\left(t_1,t_2\right)\ge {\displaystyle {\int }_{t_1}^{t_2}\frac{f(x)}{F(t_2)-F(t_1)}}\log \frac{f(t_1)/\left[F(t_2)-F(t_1)\right]}{g(t_1)/\left[G(t_2)-G(t_1)\right]}dx=\log \frac{h_1^X\left(t_1,t_2\right)}{h_1^Y\left(t_1,t_2\right)}$$

which gives (19). When $\frac{f\left(x\right)}{g\left(x\right)}$ is decreasing, the proof is similar. Furthermore, for all $t_1<x<t_2$ decreasing $g\left(x\right)$ in $x>0$ implies $g\left(t_2\right)<g\left(x\right)<g\left(t_1\right)$, then from (16) we get:

$$I{D}_{X,Y}\left(t_1,t_2\right)\le -\log h_2^Y\left(t_1,t_2\right)-I{H}_X\left(t_1,t_2\right)$$

and:

$$I{D}_{X,Y}\left(t_1,t_2\right)\ge -\log h_1^Y\left(t_1,t_2\right)-I{H}_X\left(t_1,t_2\right)$$

so that (20) holds. When $g\left(x\right)$ is increasing the proof is similar.

Remark 4.1. Consider $X$ and $Y$ are two non-negative random variables corresponding to weighted exponential distributions with positive rates $\lambda$ and $\mu$ respectively and with common positive real weight function $\omega (\cdot )$. The densities of $X$ and $Y$ are $f\left(x\right)=\frac{\omega \left(x\right)e^{-\lambda x}}{h\left(\lambda \right)}$, and $g\left(x\right)=\frac{\omega \left(x\right)e^{-\mu x}}{h\left(\mu \right)}$ respectively, where $h(\cdot )$ denotes the Laplace transform of $\omega (\cdot )$ given by $h\left(\theta \right)={\int }_0^{\infty }\omega \left(x\right)e^{-\theta x} dx , \theta >0$, therefore, for $\lambda \ne \mu$ the interval distance between $X$ and $Y$ at interval $\left(t_1,t_2\right)$ is the following:

$$I{D}_{X,Y}\left(t_1,t_2\right)=\log \frac{G\left(t_2\right)-G\left(t_1\right)}{F\left(t_2\right)-F\left(t_1\right)}+\log \frac{h\left(\mu \right)}{h\left(\lambda \right)}-\left(\lambda -\mu \right)E\left(X|t_1<X<t_2\right)$$

(21)

Remark 4.2. Let $X$ be a non-negative random lifetime with density function $f\left(x\right)$ and cumulative distribution function $F\left(t\right)=P\left(X\le t\right)$. Then the density function and cumulative distribution function for the weighted random variable $X_{\omega }$ associated to a positive real function $\omega (\cdot )$ are $f_{\omega }\left(x\right)=\frac{\omega \left(x\right)}{E\left(\omega \left(X\right)\right)}f\left(x\right)$, and $F_{\omega }\left(t\right)=\frac{E\left(\omega \left(X\right)|X\le t\right)}{E\left(\omega \left(X\right)\right)}F\left(t\right)$, respectively, where $E\left(\omega \left(X\right)\right)={\int }_0^{\infty }\omega \left(x\right)f\left(x\right)dx.$ Then, from (17) we have:

$$I{D}_{X,X_{\omega }}\left(t_1,t_2\right)=\log E\left(\omega \left(X\right)|t_1<X<t_2\right)-E\left(\log \left(\omega \left(X\right)\right)|t_1<X<t_2\right)$$

(22)

A similar expression is available in Maya and Sunoj [15] for past life time. Due to (22) and from non-negativity of $I{D}_{X,X_{\omega }}\left(t_1,t_2\right)$ we have:

$$\log E\left(\omega \left(X\right)|t_1<X<t_2\right)\ge E\left(\log \left(\omega \left(X\right)\right)|t_1<X<t_2\right)$$

which is a direct result of Markov inequality for concave functions.

Example 4.1. For $\omega \left(x\right)=x^{n-1}$ and $h\left(\theta \right)=\left(n-1\right)!/{\theta }^n$ the distributions of random variables in Remark 4.1 called Erlang distributions with scale parameters $\lambda$ and $\mu$ and with common shape parameter. The conditional mean of $\left(X|t_1<X<t_2\right)$ is the following:

$$E\left(X|t_1<X<t_2\right)=\frac{1}{F\left(t_2\right)-F\left(t_1\right)}{\displaystyle {\int }_{t_1}^{t_2}x\frac{x^{n-1}{\lambda }^n{e}^{-\lambda x}}{\left(n-1\right)!}}dx=\frac{\gamma \left(n+1,\lambda t_2\right)-\gamma \left(n+1,\lambda t_1\right)}{\lambda \left(n-1\right)!\left(F\left(t_2\right)-F\left(t_1\right)\right)}$$

where $\gamma \left(\alpha ,x\right)={\int }_0^x{e}^{-u}{u}^{\alpha -1}du$ is the incomplete Gamma function. From (21) we obtain:

$$I{D}_{X,Y}\left(t_1,t_2\right)=\log \frac{\gamma \left(n,\mu t_2\right)-\gamma \left(n,\mu t_1\right)}{\gamma \left(n,\lambda t_2\right)-\gamma \left(n,\lambda t_1\right)}-n\hspace{0.17em}\log \frac{\lambda }{\mu }+\left(\lambda -\mu \right)\frac{\gamma \left(n+1,\lambda t_2\right)-\gamma \left(n+1,\lambda t_1\right)}{\lambda \left(n-1\right)!\left(F\left(t_2\right)-F\left(t_1\right)\right)}$$

In the following proposition, sufficient condition for $I{D}_{X_1,Y}\left(t_1,t_2\right)$ to be smaller than $I{D}_{X_2,Y}\left(t_1,t_2\right)$ is provided.

Proposition 4.2. Consider three non-negative random variables $X_1$, $X_2$ and $Y$ with probability density functions $f_1$, $f_2$ and $g$ respectively. $X_1\stackrel{LR}{\ge }Y$ implies $I{D}_{X_1,Y}\left(t_1,t_2\right)\le I{D}_{X_2,Y}\left(t_1,t_2\right)$.

Proof. From (17) we have:

$$\begin{array}{ll}I{D}_{X_1,Y}\left(t_1,t_2\right)\hfill & -\hspace{0.17em}\hspace{0.17em}I{D}_{X_2,Y}\left(t_1,t_2\right)\hfill \\ \hfill & =-I{D}_{X_2,X_1}\left(t_1,t_2\right)+{\displaystyle {\int }_{t_1}^{t_2}\left(\frac{f_1\left(x\right)}{F_1\left(t_2\right)-F_1\left(t_1\right)}-\frac{f_2\left(x\right)}{F_2\left(t_2\right)-F_2\left(t_1\right)}\right)\log \frac{f_1\left(x\right)}{g\left(x\right)}dx}\hfill \\ \hfill & \le \underset{t_1}{\overset{t_2}{{\displaystyle \int }}}\left(\frac{f_1\left(x\right)}{F_1\left(t_2\right)-F_1\left(t_1\right)}-\frac{f_2\left(x\right)}{F_2\left(t_2\right)-F_2\left(t_1\right)}\right)\log \frac{f_1\left(x\right)}{g\left(x\right)}dx\hfill \\ \hfill & \le \log \frac{f_1\left(t_2\right)}{g\left(t_2\right)}\underset{t_1}{\overset{t_2}{{\displaystyle \int }}}\left(\frac{f_1\left(x\right)}{F_1\left(t_2\right)-F_1\left(t_1\right)}-\frac{f_2\left(x\right)}{F_2\left(t_2\right)-F_2\left(t_1\right)}\right)dx=0\hfill \end{array}$$

where the first inequality comes from the fact that $I{D}_{X_2,X_1}\left(t_1,t_2\right)\ge 0$ and the second one follows from the increasing $\frac{f_1\left(x\right)}{g\left(x\right)}$ in $x>0$.

Example 4.2. Let $\left\{N\left(t\right), t\ge 0\right\}$ be a non-homogeneous Poisson process with a differentiable mean function $M\left(t\right)=E\left(N\left(t\right)\right)$ such that $M\left(t\right)$ tends to $\infty$ as $t$ tends to $\infty$. Let $R_n, n=1,2,3$ denote the time of the occurrence of the $n$-th event in such a process. Then $R_n$ has the following density function:

$$f_n\left(x\right)=\frac{{\left(M\left(t\right)\right)}^{n-1}}{\left(n-1\right)!}{f}_1\left(x\right), x>0$$

where:

$$f_1\left(x\right)=-\frac{d}{dx}\exp \left(-M\left(x\right)\right), x>0$$

clearly $f_n\left(x\right)/f_1\left(x\right)$ is increasing in $x>0$. It follows from Proposition 4.2 that for all $m\le n$:

$$I{D}_{X_n,X_1}\left(t_1,t_2\right)\le I{D}_{X_m,X_1}\left(t_1,t_2\right)$$

Proposition 4.3. Let $X$ and $Y$ were random variables with common support $\left(0,\infty \right)$. Let $\phi$ be a continuous and increasing function, then:

$$I{D}_{\phi \left(X\right),\phi \left(Y\right)}\left(t_1,t_2\right)=I{D}_{X,Y}\left({\phi }^{-1}\left(t_1\right),{\phi }^{-1}\left(t_2\right)\right)$$

Proof. The proof is straightforward.

The following remarks clarify the invariance of interval discrimination measure under location and scale transformation.

Remark 4.3. For all $0\le \theta <t_1$, we get $I{D}_{X+\theta ,Y+\theta }\left(t_1,t_2\right)=I{D}_{X,Y}\left(t_1-\theta ,t_2-\theta \right)$.

Remark 4.4. Let $Y=aX$ where $>0$, $I{D}_{aX,aY}\left(t_1,t_2\right)=I{D}_{X,Y}\left(\frac{t_1}{a},\frac{t_2}{a}\right)$.

4. Conclusions

In this paper, we presented two novel measures of information which are based on a time interval and are more general than the well-known Shannon’s differential entropy and Kullback-Leibler divergence measure. These new measures are consistent in that they are valid in both past and residual lifetimes. We call these measures of information the interval entropy and the informative distance. We obtain the requirements that interval entropy can uniquely determine the distribution function. We presented several propositions and remarks, some of which parallel those for Shannon entropy and Kullback-Leibler divergence and others that are more general. The advantages of interval entropy and informative distance were outlined as well. We believe that interval basic measures will have many applications in reliability, stochastic processes and other areas in the near future. The results presented here are by no means comprehensive but hopefully will pave the way for studying the entropy in a different and more general setting.