A Generalized Measure of Cumulative Residual Entropy

Abstract

In this work, we introduce a generalized measure of cumulative residual entropy and study its properties. We show that several existing measures of entropy, such as cumulative residual entropy, weighted cumulative residual entropy and cumulative residual Tsallis entropy, are all special cases of this generalized cumulative residual entropy. We also propose a measure of generalized cumulative entropy, which includes cumulative entropy, weighted cumulative entropy and cumulative Tsallis entropy as special cases. We discuss a generating function approach, using which we derive different entropy measures. We provide residual and cumulative versions of Sharma–Taneja–Mittal entropy and obtain them as special cases this generalized measure of entropy. Finally, using the newly introduced entropy measures, we establish some relationships between entropy and extropy measures.

1. Introduction

The uncertainty associated with a random variable can be evaluated using information measures. In many practical situations in lifetime data analysis, experimental physics, econometrics and demography, measuring the uncertainty associated with a random variable is very important. The seminal work on information theory started with the concept of Shannon entropy or differential entropy introduced by Shannon (1948) [1]. For an absolutely continuous non-negative random variable X, the differential entropy is given by

$$H\left(X\right)=-E(logf\left(X\right))=-{\int }_0^{\infty }f\left(x\right)logf\left(x\right)dx,$$

where $f\left(x\right)$ is the probability density function of X and “log” stands for the natural logarithm, with $0log0$ taken as 0.

Several measures of entropy have been introduced in the literature since then, each one being suitable for some specific situations. The widely used measures of entropy are cumulative residual entropy (CRE) [2], cumulative entropy (CE) [3] and the corresponding weighed measures by Mirali et al. (2016) [4] and Mirali and Baratpour (2017) [5]. A unified formulation of entropy has been put forward by Balakrishnan et al. (2022) [6] recently. For a non-negative random variable X with distribution function $F\left(x\right)$, the cumulative residual entropy, which measures the uncertainty in the future of a lifetime of a system, is defined as

$$\mathcal{E}\left(X\right)=-{\int }_0^{\infty }\overline{F}\left(x\right)log\overline{F}\left(x\right)dx,$$

(1)

where $\overline{F}\left(x\right)=1-F\left(x\right)$ is the survival function of X. Asadi and Zohrevand (2007) [7] gave a representation of (1) based on the mean residual life function as

$$\mathcal{E}\left(X\right)=E\left(r\right(X\left)\right),$$

where $r\left(t\right)$ is the mean residual life function of X at time t given by

$$r\left(t\right)=E(X-t|X>t)=\frac{{\int }_t^{\infty }\overline{F}\left(u\right)du}{\overline{F}\left(t\right)}.$$

Di Crescenzo and Longobardi (2009) [3] introduced cumulative entropy for estimating the uncertainty in the past lifetime of a system as

$$\mathcal{CE}\left(X\right)=-{\int }_0^{\infty }F\left(x\right)logF\left(x\right)dx.$$

(2)

The weighted versions of $\mathcal{E}\left(X\right)$ and $\mathcal{CE}\left(X\right)$ have been studied in the literature as well. Weighted cumulative residual entropy, introduced by Mirali et al. (2016) [4], is defined as

$${\mathcal{E}}^w\left(X\right)=-{\int }_0^{\infty }x\overline{F}\left(x\right)log\overline{F}\left(x\right)dx.$$

(3)

Mirali and Baratpour (2017) [5] introduced weighted cumulative entropy as

$${\mathcal{CE}}^w\left(X\right)=-{\int }_0^{\infty }xF\left(x\right)logF\left(x\right)dx.$$

(4)

A detailed discussion on weighted entropies has been made by Suhov and Sekeh (2015) [8]. Some additional recent developments in this area were due to [9,10,11,12].

Recently, Kharazmi and Balakrishnan (2020) [13] proposed Jensen cumulative residual entropy, which is an extension of (1). Kharazmi and Balakrishnan (2020) [14] then studied cumulative residual and relative cumulative residual Fisher information measures and their properties. More general cumulative residual-information-generating and relative cumulative residual-information-generating measures have been introduced and studied by Kharazmi and Balakrishnan (2021) [15]. Fractional generalized cumulative entropy and its dynamic version have been proposed by Di Crescenzo et al. (2021) [16].

Several extensions of Shannon entropy are available in the literature, obtained by introducing some additional parameters so that these measures become sensitive to different characteristics and shapes of probability distributions. One important generalization of Shannon entropy is due to Tsallis (1988) [17], known as generalized Tsallis entropy of order $\alpha$. For a continuous random variable X, the generalized Tsallis entropy of order $\alpha$ is defined as [17]

$$T_{\alpha }\left(X\right)=\frac{1}{\alpha -1}\left(1-{\int }_0^{\infty }{f}^{\alpha }\left(x\right)dx\right),\alpha \ne 1.$$

Many extensions or modifications have also been provided for $T_{\alpha }\left(X\right)$. Sati and Gupta (2015) [18] proposed a cumulative Tsallis residual entropy of order $\alpha$, and Rajesh and Sunoj (2019) [19] modified it and defined cumulative residual Tsallis entropy of order $\alpha$ as

$$CR{T}_{\alpha }\left(X\right)=\frac{1}{\alpha -1}{\int }_0^{\infty }(\overline{F}\left(x\right)-{\overline{F}}^{\alpha }\left(x\right))dx,\alpha >0,\alpha \ne 1.$$

(5)

For a non-negative continuous random variable X, Chakraborty and Pradhan (2021) [20] defined weighted cumulative residual Tsallis entropy (WCRTE) of order $\alpha$ as

$$WCR{T}_{\alpha }\left(X\right)=\frac{1}{\alpha -1}{\int }_0^{\infty }x(\overline{F}\left(x\right)-{\overline{F}}^{\alpha }\left(x\right))dx,\alpha >0,\alpha \ne 1.$$

(6)

They also introduced dynamic weighted cumulative residual Tsallis entropy of order $\alpha$.

Calì et al. (2017) [9] introduced cumulative Tsallis past entropy as

$$C{T}_{\alpha }\left(X\right)=\frac{1}{\alpha -1}{\int }_0^{\infty }(F\left(x\right)-F^{\alpha }\left(x\right))dx,\alpha >0,\alpha \ne 1.$$

(7)

Calì et al. (2021) [21] subsequently introduced a family of mean past weighted entropies of order $\alpha$, using the concept of mean inactivity time. Chakraborty and Pradhan (2021) [20] defined weighted cumulative Tsallis entropy (WCTE) of order $\alpha$ as

$$WC{T}_{\alpha }\left(X\right)=\frac{1}{\alpha -1}{\int }_0^{\infty }x(F\left(x\right)-F^{\alpha }\left(x\right))dx,\alpha >0,\alpha \ne 1.$$

(8)

They also studied dynamic weighted cumulative Tsallis entropy of order $\alpha$. As is evident from the description above, several entropy measures are available in the literature. Recently, Balakrishnan et al. (2022) [22] have provided a unified formulation of entropy and demonstrated its applications.

In the present work, we define a generalized cumulative residual entropy and study its properties in Section 2. We show that cumulative residual entropy, weighted cumulative residual entropy, cumulative residual Tsallis entropy and weighted cumulative residual Tsallis entropy are all special cases of the proposed measure. We also propose a new generalized cumulative entropy measure and discuss some of its properties. We use the generating function approach to obtain some new entropy measures. In Section 3, we provide cumulative (residual) versions of Sharma–Taneja–Mittal entropy and obtain them as special cases of the generalized measures of entropy introduced in Section 2. In Section 4, we establish some relationships between entropy and extropy measures. Finally, we make some concluding remarks in Section 5.

2. Generalized Cumulative Entropy

In this section, we introduce generalized cumulative (residual) entropy measures. We then show that several entropy measures are special cases of the proposed entropies. Some generalizations of CRE and CE have been discussed in the literature and we now review these briefly. Drissi et al. (2008) [23] generalized the definition of CRE, given by Rao et al. (2004) [2], to the case of distributions with general support. They also showed that this generalized CRE can be used as an alternative to differential entropy. Kayal (2016) [24] introduced a generalization of CE proposed by Di Crescenzo and Longobardi (2009) [3] and their dynamic versions. Their definition is related to lower records and the reversed relevation transform. Psarrakos and Navarro (2013) [25] proposed a generalized cumulative residual entropy (GCRE), related to record values from a sequence of independent and identical random variables and with the relevation transform. Some properties and applications of the GCRE in actuarial risk measures have been discussed by Psarrakos and Toomaj (2017) [26]. Under some assumptions, Navarro and Psarrakos (2017) [27] proved that the GCRE function of a fixed order n uniquely determines the distribution function. Consequently, some characterizations of particular probability models have been obtained from this general result. Di Crescenzo and Toomaj (2017) [28] obtained some further results associated with generalized cumulative entropy, such as stochastic orders, bounds and characterization results. Some characterizations for the dynamic generalized cumulative entropy have also been derived. Recently, Di Crescenzo et al. (2021) [16] proposed the fractional generalized cumulative entropy and its dynamic version.

We introduce here generalized CRE and CE, which encompass most of the existing variations of these measures, as demonstrated below.

2.1. Generalized Cumulative Residual Entropy

Let X be a non-negative random variable with absolutely continuous distribution function $F\left(x\right)$ and $\overline{F}\left(x\right)$ as the survival function. We assume that the mean $\mu =E\left(X\right)$ is finite.

Definition 1.

Let X be a non-negative random variable with absolutely continuous distribution function F. Further, let $\varphi (.)$ be a function of X and $w(.)$ be a weight function. Then, the generalized cumulative residual entropy of X is defined as

$$\mathcal{GE}\left(X\right)=\int w\left(u\right)E\left[\varphi \left(X\right)-\varphi \left(u\right)|X>u\right]dF\left(u\right),$$

(9)

where w and ϕ can be chosen arbitrarily under the existence of the above integral such that $\mathcal{GE}\left(X\right)$ becomes concave.

Entropy is defined as a measure of uncertainty associated with a model. Strict concavity implies that entropy will increase under averaging. The general definition of entropy we have given contains two arbitrary functions $\varphi$ and w, and though it is difficult to state general conditions under which concavity of the general measure will hold, one can chose these functions so that $\mathcal{GE}\left(X\right)$ becomes concave.

However, with different choices of weight function $w(.)$ and $\varphi (.)$, we can introduce several measures of entropy. First, we show that the new measure reduces to the cumulative residual entropy of Rao et al. (2004) [2] and the weighted cumulative residual entropy of Mirali et al. (2016) [4] for some specific choices of $\varphi (.)$ and $w(.)$.

Using integration by parts, from (1), we obtain (see [29])

$$\begin{array}{ccc}\hfill \mathcal{E}\left(X\right)& =& {\int }_0^{\infty }x[-log\overline{F}\left(x\right)]dF\left(x\right)-E\left(X\right).\hfill \end{array}$$

Let us denote the hazard rate of X by

$$\lambda \left(x\right)=\frac{f\left(x\right)}{\overline{F}\left(x\right)},$$

where f is the density function of X. Then, the cumulative hazard function $\Lambda \left(x\right)$ can be expressed as

$$\Lambda \left(x\right)={\int }_0^x\lambda \left(u\right)du=-log\overline{F}\left(x\right).$$

Now, with $w\left(u\right)=1$ and $\varphi \left(x\right)=x$, the expression in (9) becomes

$$\begin{array}{ccc}\hfill \mathcal{GE}\left(X\right)& =& {\int }_0^{\infty }E(X-u|X>u)dF\left(u\right)\hfill \\ & =& {\int }_0^{\infty }E\left(X\right|X>u)f\left(u\right)du-{\int }_0^{\infty }udF\left(u\right)\hfill \\ & =& {\int }_0^{\infty }\frac{f\left(u\right)}{\overline{F}\left(u\right)}{\int }_u^{\infty }xdF\left(x\right)du-E\left(X\right)\hfill \\ & =& {\int }_0^{\infty }x{\int }_0^x\frac{f\left(u\right)}{\overline{F}\left(u\right)}dudF\left(x\right)-E\left(X\right)\hfill \\ & =& {\int }_0^{\infty }x{\int }_0^x\lambda \left(u\right)dudF\left(x\right)-E\left(X\right)\hfill \\ & =& {\int }_0^{\infty }x[-log\overline{F}\left(x\right)]dF\left(x\right)-E\left(X\right).\hfill \end{array}$$

Thus, $\mathcal{GE}\left(X\right)$ reduces in this case to the cumulative residual entropy of Rao et al. (2004) [2].

The weighted cumulative residual entropy defined in (3) can be written as [29]

$$\begin{array}{ccc}\hfill {\mathcal{E}}^w\left(X\right)& =& \frac{1}{2}{\int }_0^{\infty }{x}^2[-log\overline{F}\left(x\right)]dF\left(x\right)-\frac{1}{2}E\left(X^2\right).\hfill \end{array}$$

If we choose $w\left(u\right)=1/2$ and $\varphi \left(x\right)=x^2$ and proceed as above, we can show that (9) becomes

$$\begin{array}{ccc}\hfill \mathcal{GE}\left(X\right)& =& \frac{1}{2}{\int }_0^{\infty }E(X^2-u^2|X>u)dF\left(u\right)\hfill \\ & =& \frac{1}{2}{\int }_0^{\infty }{x}^2[-log\overline{F}\left(x\right)]dF\left(x\right)-\frac{1}{2}E\left(X^2\right).\hfill \end{array}$$

Thus, $\mathcal{GE}\left(X\right)$ reduces in this case to the weighted cumulative residual entropy, ${\mathcal{E}}^w\left(X\right)$.

Next, we show that the cumulative residual Tsallis entropy of order $\alpha$ is a special case of $\mathcal{GE}\left(X\right)$. An alternative representation of $CR{T}_{\alpha }\left(X\right)$ is [19]

$$CR{T}_{\alpha }\left(X\right)=E\left(r\left(X\right){\overline{F}}^{\alpha -1}\left(X\right)\right),$$

where $r\left(X\right)=E(X-x|X>x)$. If we now choose $w\left(u\right)={\overline{F}}^{\alpha -1}\left(u\right)$ and $\varphi \left(x\right)=x$, (9) becomes

$$\mathcal{GE}\left(X\right)={\int }_0^{\infty }E(X-u|X>u){\overline{F}}^{\alpha -1}\left(u\right)dF\left(u\right)=E\left(r\left(X\right){\overline{F}}^{\alpha -1}\left(X\right)\right).$$

Chakraborty and Pradhan (2021) [20] expressed a weighted version of cumulative residual Tsallis entropy of order $\alpha$ in (6) as

$$WCR{T}_{\alpha }\left(X\right)={\int }_0^{\infty }\frac{1}{\overline{F}\left(t\right)}{\int }_t^{\infty }x\overline{F}\left(x\right)dx{\overline{F}}^{\alpha -1}\left(t\right)dF\left(t\right).$$

(10)

Upon noting that the integral

$$\begin{array}{ccc}\hfill \frac{1}{\overline{F}\left(t\right)}{\int }_t^{\infty }x\overline{F}\left(x\right)dx& =& \frac{1}{\overline{F}\left(t\right)}{\int }_t^{\infty }x{\int }_x^{\infty }dF\left(y\right)dx\hfill \\ & =& \frac{1}{\overline{F}\left(t\right)}{\int }_t^{\infty }{\int }_t^{y}xdxdF\left(y\right)\hfill \\ & =& \frac{1}{\overline{F}\left(t\right)}{\int }_t^{\infty }\frac{1}{2}(y^2-t^2)dF\left(y\right)\hfill \\ & =& \frac{1}{2}E(X^2-t^2|X>t),\hfill \end{array}$$

(10) becomes

$$WCR{T}_{\alpha }\left(X\right)=\frac{1}{2}{\int }_0^{\infty }E(X^2-t^2|X>t){\overline{F}}^{\alpha -1}\left(t\right)dF\left(t\right).$$

Now, for the choices of $w\left(u\right)={\overline{F}}^{\alpha -1}\left(u\right)$ and $\varphi \left(x\right)=\frac{1}{2}{x}^2$, from (9), we obtain the above expression. Thus, $WCR{T}_{\alpha }\left(X\right)$ is a special case of $\mathcal{GE}\left(X\right)$ as well. The special cases of $\mathcal{GE}\left(X\right)$ discussed here are all listed in Table 1.

Table 1. Special cases of generalized residual entropy.

Entropy Measure	$\mathit{w}\left(\mathit{u}\right)$	$\mathit{\varphi }\left(\mathit{x}\right)$
Cumulative residual entropy	1	x
Weighted cumulative residual entropy	$\frac{1}{2}$	$x^2$
Cumulative residual Tsallis entropy	${\overline{F}}^{\alpha -1}\left(u\right)$	x
Weighted cumulative Tsallis residual entropy	${\overline{F}}^{\alpha -1}\left(u\right)$	$\frac{x^2}{2}$

Next, we derive expressions for $\mathcal{GE}\left(X\right)$ for some specific distributions.

Consider the exponential distribution with mean $\lambda$. Then, it is well-known that mean residual life is equal to mean. Thus, when $w\left(u\right)=1$, we have

$$\mathcal{GE}\left(X\right)={\int }_0^{\infty }w\left(u\right)\lambda dF\left(u\right)=\lambda .$$

In general, $\mathcal{GE}\left(X\right)$ is a constant for any weight function. For the standard exponential, taking $\varphi \left(x\right)=x^2$, we have

$$\begin{array}{ccc}\hfill \mathcal{GE}\left(X\right)& =& {\int }_0^{\infty }w\left(u\right)E(X^2-u^2|X>u)e^{-u}du\hfill \\ & =& {\int }_0^{\infty }w\left(u\right)(u+1)e^{-u}du.\hfill \end{array}$$

Thus, $\mathcal{GE}\left(X\right)=2$ when $w\left(u\right)=1$ and $\mathcal{GE}\left(X\right)=1.5$ when $w\left(u\right)=\overline{F}\left(u\right)$.

Next, let us consider the standard uniform distribution with pdf $f\left(x\right)=1$, $0<x<1.$ Then,

$$\begin{array}{ccc}\hfill \mathcal{GE}\left(X\right)& =& {\int }_0^{1}w\left(u\right)\frac{1}{1-u}{\int }_u^1(1-x)dxdu\hfill \\ & =& {\int }_0^{1}w\left(u\right)\frac{1}{2(1-u)}{(u-1)}^{2}du\hfill \\ & =& {\int }_0^{1}w\left(u\right)(1-u)du.\hfill \end{array}$$

We thus obtain the residual entropy as $\frac{1}{4}$. Additionally, when $w\left(u\right)=1$ and $\varphi \left(x\right)=x^2/2$, we get

$$\begin{array}{ccc}\hfill \mathcal{GE}\left(X\right)& =& \frac{1}{6}{\int }_0^1(1-u)(2u+1)du=\frac{5}{36},\hfill \end{array}$$

as given in Example 5 of Balakrishnan et al. (2022) [29].

2.2. Generalized Cumulative Entropy

In this sub-section, we introduce a generalized cumulative entropy and discuss some of its properties.

Definition 2.

Let X be a non-negative random variable with absolutely continuous distribution function F. Further, let $\varphi (.)$ be a function of X and $w(.)$ be a weight function. Then, the generalized cumulative entropy is defined as

$$\mathcal{GCE}\left(X\right)={\int }_0^{\infty }w\left(u\right)E\left[\varphi \left(u\right)-\varphi \left(X\right)|X\le u\right]dF\left(u\right),$$

(11)

where w and ϕ can be chosen arbitrarily under the existence of the above integral such that $\mathcal{GCE}\left(X\right)$ becomes concave.

For the choices of $w\left(u\right)=1$ and $\varphi \left(x\right)=x$, (11) reduces to $\mathcal{CE}\left(X\right)$ [3]. Similarly, $\mathcal{GCE}\left(X\right)$ in (11) reduces to the weighted cumulative entropy of Mirali and Baratpour (2017) [5] for the choices of $w\left(u\right)=1/2$ and $\varphi \left(x\right)=x^2$.

The reversed hazard rate function of X, denoted by $h(.)$, is defined as

$$h\left(x\right)=\frac{f\left(x\right)}{F\left(x\right)},$$

which yields the cumulative reversed hazard rate function as

$$H\left(x\right)={\int }_x^{\infty }\frac{f\left(u\right)}{F\left(u\right)}du=-logF\left(x\right).$$

The cumulative entropy in (2) can be expressed as (see [29])

$$\begin{array}{ccc}\hfill \mathcal{CE}\left(X\right)& =& {\int }_0^{\infty }x[-logF\left(x\right)]dF\left(x\right)+E\left(X\right).\hfill \end{array}$$

Thus, by using the cumulative reversed hazard rate function, we can express

$$\begin{array}{ccc}\hfill \mathcal{CE}\left(X\right)& =& -{\int }_0^{\infty }x[logF\left(x\right)]dF\left(x\right)+E\left(X\right)\hfill \\ & =& E\left(X\right)-{\int }_0^{\infty }x{\int }_x^{\infty }\frac{f\left(u\right)}{F\left(u\right)}dudF\left(x\right)\hfill \\ & =& E\left(X\right)-{\int }_0^{\infty }\frac{1}{F\left(u\right)}{\int }_0^{u}xdF\left(x\right)dF\left(u\right)\hfill \\ & =& {\int }_0^{\infty }E(u-X|X\le u)dF\left(u\right),\hfill \end{array}$$

which is the special case of the generalized cumulative entropy in (11) for the choices of $w\left(u\right)=1$ and $\varphi \left(x\right)=x$. Proceeding similarly, we can show that $\mathcal{GCE}\left(X\right)$ reduces to the weighted cumulative entropy [5] for the choices of $w\left(u\right)=1/2$ and $\varphi \left(x\right)=x^2$.

Next, we show that the cumulative Tsallis entropy of order $\alpha$ is a special case of $\mathcal{GCE}\left(X\right)$ in (11). The mean inactivity time function of a random variable X, at time x, is defined as

$$m\left(x\right)=E(x-X|X\le x)=\frac{1}{F\left(x\right)}{\int }_0^{x}ydF\left(y\right).$$

Using $m\left(x\right)$, $C{T}_{\alpha }\left(X\right)$ can be expressed as [9]

$$C{T}_{\alpha }\left(X\right)=E\left(m\left(X\right)F^{\alpha -1}\left(X\right)\right).$$

Now, for the choices of $w\left(u\right)=F^{\alpha -1}\left(u\right)$ and $\varphi \left(x\right)=x$, (11) yields

$$\mathcal{GCE}\left(X\right)={\int }_0^{\infty }{F}^{\alpha -1}\left(u\right)E\left[u-X|X\le u\right]dF\left(u\right)=E\left(m\left(X\right)F^{\alpha -1}\left(X\right)\right).$$

An alternative expression for the weighted cumulative Tsallis entropy of order $\alpha$ is given by [20]

$$WC{T}_{\alpha }\left(X\right)={\int }_0^{\infty }\frac{1}{F\left(t\right)}{\int }_t^{\infty }xF\left(x\right)dx{F}^{\alpha -1}\left(t\right)dF\left(t\right).$$

(12)

As in Section 2, simple algebraic manipulations yield

$$WC{T}_{\alpha }\left(X\right)=\frac{1}{2}{\int }_0^{\infty }E(t^2-X^2|X\le t)F^{\alpha -1}\left(t\right)dF\left(t\right).$$

Again, for the choices of $w\left(u\right)=F^{\alpha -1}\left(u\right)$ and $\varphi \left(x\right)=\frac{1}{2}{x}^2$, (11) yields

$$\mathcal{GCE}\left(X\right)=\frac{1}{2}{\int }_0^{\infty }E\left[u^2-X^2|X\le u\right]F^{\alpha -1}\left(u\right)dF\left(u\right).$$

Thus, $WC{T}_{\alpha }\left(X\right)$ is a special case of $\mathcal{GCE}\left(X\right)$. In Table 2, we list the cumulative entropies derived from $\mathcal{GCE}\left(X\right)$.

Table 2. Special cases of generalized entropy.

Entropy Measure	$\mathit{w}\left(\mathit{u}\right)$	$\mathit{\varphi }\left(\mathit{x}\right)$
Cumulative entropy	1	x
Weighted cumulative entropy	$\frac{1}{2}$	$x^2$
Cumulative Tsallis entropy	$F^{\alpha -1}\left(u\right)$	x
Weighted cumulative Tsallis entropy	$F^{\alpha -1}\left(u\right)$	$\frac{x^2}{2}$

Next, we derive expressions for $\mathcal{GCE}\left(X\right)$ for some specific distributions. Consider the standard exponential distribution with mean $\lambda =1$. Then, for the choices of $w\left(u\right)=1$ and $\varphi \left(x\right)=x$, we have

$$\begin{array}{ccc}\hfill \mathcal{GCE}\left(X\right)& =& {\int }_0^{\infty }\frac{1}{1-e^{-u}}{\int }_0^u(u-x)e^{-x}dxdF\left(u\right)\hfill \\ & =& {\int }_0^{\infty }\frac{1}{1-e^{-u}}(u-(1-e^{-u})e^{-u}du\hfill \\ & =& {\int }_0^{\infty }\frac{1}{1-e^{-u}}u{e}^{-u}du-1=\frac{{\pi }^2}{6}-1.\hfill \end{array}$$

Next, let us consider the standard uniform distribution with pdf $f\left(x\right)=1$, $0<x<1.$ Then, for the choices of $w\left(u\right)=1$ and $\varphi \left(x\right)=x$, we obtain

$$\begin{array}{ccc}\hfill \mathcal{GCE}\left(X\right)& =& {\int }_0^1\frac{1}{u}{\int }_0^u(u-x)dxdu\hfill \\ & =& \frac{1}{2}{\int }_0^{1}udu=1/4.\hfill \end{array}$$

These two examples have been presented earlier by Balakrishnan et al. (2022) [29].

2.3. Generating Function

We now introduce generating function related to generalized entropy measures discussed in the preceding sections.

Definition 3.

Let X be a non-negative random variable with absolutely continuous distribution function F. Further, let $\varphi (.)$ be a function of X and $w(.)$ be a weight function. We then define a generating function for the generalized cumulative residual entropy measure as

$$G{f}_{re}\left(t\right)={\int }_0^{\infty }w\left(u\right)E\left(\left[exp\left(t\varphi \left(X\right)\right)-exp\left(t\varphi \left(u\right)\right)\right]|X>u\right)dF\left(u\right).$$

(13)

Being a function of t, we can interpret $G{f}_{re}\left(t\right)$ as a generating function for the general entropy measure introduced in Section 2. If we differentiate the expression in (13) with respect to t once, we obtain

$$G{f}_{re}^{{}^{\prime }}\left(t\right)={\int }_0^{\infty }w\left(u\right)E\left(\left[\varphi \left(X\right)exp\left(t\varphi \left(X\right)\right)-\varphi \left(u\right)exp\left(t\varphi \left(u\right)\right)\right]|X>u\right)dF\left(u\right).$$

Now, by setting $t=0$ in the above expression, we obtain the generalized entropy measure in (9). We, therefore, refer to it as generalized cumulative residual entropy of order 1. Higher-order derivatives with respect to t would similarly give rise to generalized cumulative residual entropies of orders 2, 3 and so on. For example, the generalized cumulative residual entropy of order two is given by

$$G{f}_{re}\left(2\right)={\int }_0^{\infty }w\left(u\right)E\left({\varphi }^2\left(X\right)-{\varphi }^2\left(u\right)|X>u\right)dF\left(u\right).$$

(14)

For the choice of $w\left(u\right)=1/2$ and $\varphi \left(x\right)=x$, $G{f}_{re}\left(2\right)$ reduces to the weighted cumulative residual entropy of Mirali et al. (2016) [4].

In a similar manner, we define the generating function for the generalized cumulative entropy as follows.

Definition 4.

Let X be a non-negative random variable with absolutely continuous distribution function F. Further, let $\varphi (.)$ be a function of X and $w(.)$ be a weight function. We then define the generating function for the generalized cumulative entropy as

$$G{f}_{ce}\left(t\right)={\int }_0^{\infty }w\left(u\right)E\left(\left[exp\left(t\varphi \left(u\right)-exp\left(t\varphi \left(X\right)\right)\right)\right]|X\le u\right)dF\left(u\right).$$

(15)

Once again, differentiating (15) with respect to t once and setting $t=0$, we obtain the generalized cumulative entropy (of order 1) measure in (11). From (15), we can similarly obtain the generalized cumulative entropy of order 2 to be

$$G{f}_{ce}\left(2\right)={\int }_0^{\infty }w\left(u\right)E\left({\varphi }^2\left(u\right)-{\varphi }^2\left(X\right)|X\le u\right)dF\left(u\right).$$

(16)

The weighted cumulative entropy of Mirali and Baratpour (2017) [5] can be obtained from (16) for the choices of $w\left(u\right)=1/2$ and $\varphi \left(x\right)=x$. Naturally, higher-order derivatives with respect to t would give rise to generalized cumulative entropies of orders 3, 4 and so on.

3. Sharma–Taneja–Mittal Entropy

In this section, we introduce the cumulative (residual) versions of Sharma–Taneja–Mittal (STM) entropy. We then show that these are indeed special cases of the generalized residual and the past entropy introduced in Section 2.

Sharma and Taneja (1975) [30] and Mittal (1975) [31] independently introduced an entropy of the form

$$S_{\alpha ,\beta }=\frac{1}{\beta -\alpha }{\int }_0^{\infty }(f^{\alpha }\left(x\right)-f^{\beta }\left(x\right))dx,\alpha ,\beta >0,(\alpha ,\beta )\ne (1,1).$$

(17)

For different choices of $\alpha$ and $\beta$, from (17), we obtain some entropy measures discussed in the literature. In particular, for $\alpha =1-\kappa$ and $\beta =1+\kappa$, we obtain Kaniadakis entropy [32], as a special case of Sharma-Mittal entropy. For more details, see [33] along with Table 1 of Ilic et al. (2021) [34].

3.1. Sharma–Taneja–Mittal Cumulative Residual Entropy

In this sub-section, we introduce the cumulative residual version of the STM entropy.

Definition 5.

Let X be a non-negative random variable with absolutely continuous survival function $\overline{F}$. Then, the cumulative residual STM entropy is defined as

$$S{R}_{\alpha ,\beta }=\frac{1}{\beta -\alpha }{\int }_0^{\infty }({\overline{F}}^{\alpha }\left(x\right)-{\overline{F}}^{\beta }\left(x\right))dx,\alpha ,\beta >0,(\alpha ,\beta )\ne (1,1).$$

(18)

We also introduce the weighted cumulative residual STM entropy as follows.

Definition 6.

Let X be a non-negative random variable with absolutely continuous survival function $\overline{F}$. Then, the cumulative weighted residual STM entropy is defined as

$$SR{W}_{\alpha ,\beta }=\frac{1}{\beta -\alpha }{\int }_0^{\infty }x({\overline{F}}^{\alpha }\left(x\right)-{\overline{F}}^{\beta }\left(x\right))dx,\alpha ,\beta >0,(\alpha ,\beta )\ne (1,1).$$

(19)

Next, we show that $S{R}_{\alpha ,\beta }$ and $SR{W}_{\alpha ,\beta }$ are indeed special cases of $\mathcal{GE}\left(X\right)$. Consider

$$\begin{array}{ccc}S{R}_{\alpha ,\beta }& =& \frac{1}{\beta -\alpha }{\int }_0^{\infty }({\overline{F}}^{\alpha }\left(x\right)-{\overline{F}}^{\beta }\left(x\right))dx\hfill \\ & =& \frac{1}{\beta -\alpha }{\int }_0^{\infty }(\overline{F}\left(x\right)-{\overline{F}}^{\beta }\left(x\right)-(\overline{F}\left(x\right)-{\overline{F}}^{\alpha }\left(x\right)))dx.\hfill \end{array}$$

(20)

Now, consider

$$\begin{array}{ccc}\hfill {\int }_0^{\infty }(\overline{F}\left(x\right)-{\overline{F}}^{\beta }\left(x\right))dx& =& {\int }_0^{\infty }\overline{F}\left(x\right)(1-{\overline{F}}^{\beta -1}\left(x\right))dx\hfill \\ & =& {\int }_0^{\infty }\overline{F}\left(x\right){\int }_0^x(\beta -1){\overline{F}}^{\beta -2}\left(t\right)dF\left(t\right)dx\hfill \\ & =& (\beta -1){\int }_0^{\infty }{\overline{F}}^{\beta -2}\left(t\right){\int }_x^{\infty }\overline{F}\left(x\right)dxdF\left(t\right)\hfill \\ & =& (\beta -1){\int }_0^{\infty }{\overline{F}}^{\beta -1}\left(t\right)\frac{1}{\overline{F}\left(t\right)}{\int }_t^{\infty }\overline{F}\left(x\right)dxdF\left(t\right)\hfill \\ & =& (\beta -1){\int }_0^{\infty }{\overline{F}}^{\beta -1}\left(t\right)r\left(t\right)dF\left(t\right)\hfill \\ & =& (\beta -1)E\left(r\left(X\right){\overline{F}}^{\beta -1}\left(X\right)\right).\hfill \end{array}$$

(21)

Similarly, we obtain

$$\begin{array}{ccc}\hfill {\int }_0^{\infty }(\overline{F}\left(x\right)-{\overline{F}}^{\alpha }\left(x\right))dx& =& (\alpha -1)E\left(r\left(X\right){\overline{F}}^{\alpha -1}\left(X\right)\right).\hfill \end{array}$$

(22)

Upon substituting (21) and (22) in (20), we obtain

$$\begin{array}{ccc}\hfill S{R}_{\alpha ,\beta }& =& \frac{1}{\beta -\alpha }E\left(r\left(X\right)\left((\beta -1){\overline{F}}^{\beta -1}\left(X\right)-(\alpha -1){\overline{F}}^{\alpha -1}\left(X\right)\right)\right).\hfill \end{array}$$

(23)

Now, for the choices of $w\left(u\right)=\frac{1}{\beta -\alpha }((\beta -1){\overline{F}}^{\beta -1}\left(u\right)-(\alpha -1){\overline{F}}^{\alpha -1}\left(u\right))$ and $\varphi \left(x\right)=x$, from (9), we obtain $S{R}_{\alpha ,\beta }$.

By following the same steps as performed for (23), the cumulative weighted residual STM entropy can be expressed as

$$SR{W}_{\alpha ,\beta }=\frac{1}{\beta -\alpha }{\int }_0^{\infty }E(X^2-x^2|X>x)\left((\beta -1){\overline{F}}^{\beta -1}\left(x\right)-(\alpha -1){\overline{F}}^{\alpha -1}\left(x\right)\right)dF\left(x\right).$$

(24)

Again, for the choices of $w\left(u\right)=\frac{1}{\beta -\alpha }((\beta -1){\overline{F}}^{\beta -1}\left(u\right)-(\alpha -1){\overline{F}}^{\alpha -1}\left(u\right))$ and $\varphi \left(x\right)=x^2/2$, from (9), we arrive at (24).

3.2. Sharma–Taneja–Mittal Cumulative Entropy

In this sub-section, we introduce cumulative and weighted cumulative STM entropies, and then show that they are indeed special cases of the generalized entropy.

Definition 7.

Let X be a non-negative random variable with absolutely continuous distribution function F. Then, the cumulative STM entropy is defined as

$$S{P}_{\alpha ,\beta }=\frac{1}{\beta -\alpha }{\int }_0^{\infty }(F^{\alpha }\left(x\right)-F^{\beta }\left(x\right))dx,\alpha ,\beta >0,(\alpha ,\beta )\ne (1,1).$$

(25)

In this case, the weighted cumulative STM entropy is defined as follows.

Definition 8.

Let X be a non-negative random variable with absolutely continuous distribution function F. Then, the cumulative weighted STM entropy is defined as

$$SP{W}_{\alpha ,\beta }=\frac{1}{\beta -\alpha }{\int }_0^{\infty }x(F^{\alpha }\left(x\right)-F^{\beta }\left(x\right))dx,\alpha ,\beta >0,(\alpha ,\beta )\ne (1,1).$$

(26)

Of course, in the special cases of $\alpha =1-\kappa$ and $\beta =1+\kappa$, the above definitions would result in the corresponding generalized versions of Kaniadakis entropy. Following the same steps as those used to obtain alternative expressions for $S{P}_{\alpha ,\beta }$ and $SP{W}_{\alpha ,\beta }$, we can express $S{P}_{\alpha ,\beta }$ and $SP{W}_{\alpha ,\beta }$, respectively, as

$$\begin{array}{ccc}\hfill S{P}_{\alpha ,\beta }& =& \frac{1}{\beta -\alpha }E\left(m\left(X\right)\left((\beta -1){\overline{F}}^{\beta -1}\left(X\right)-(1-\alpha ){\overline{F}}^{\alpha -1}\left(X\right)\right)\right),\hfill \end{array}$$

$$SP{W}_{\alpha ,\beta }=\frac{1}{\beta -\alpha }{\int }_0^{\infty }E(x^2-X^2|X\le x)\left((\beta -1){\overline{F}}^{\beta -1}\left(x\right)-(1-\alpha ){\overline{F}}^{\alpha -1}\left(x\right)\right)dF\left(x\right).$$

(27)

Now, let $w\left(u\right)=\frac{1}{\beta -\alpha }((\beta -1){\overline{F}}^{\beta -1}\left(u\right)-(\alpha -1){\overline{F}}^{\alpha -1}\left(u\right))$. Then, from (11), we obtain $S{P}_{\alpha ,\beta }$ and $SP{W}_{\alpha ,\beta }$ in (27) and (27) by taking $\varphi \left(x\right)=x$ and $\varphi \left(x\right)=x^2/2$, respectively.

4. Connection between Entropy and Extropy

Apart from entropy, extropy and its properties have also been studied for quantifying the uncertainty associated with a random variable X. Using the new entropy measures introduced in the preceding sections, we establish some relationships between entropy and extropy measures in this section.

For a non-negative random variable X, extropy is defined as [35]

$$J\left(X\right)=-\frac{1}{2}{\int }_0^{\infty }{f}^2\left(x\right)dx.$$

Now, we briefly discuss some recent developments associated with extropy measures. Jahanshahi et al. [36] defined the cumulative residual extropy as

$$\mathcal{CRJ}\left(X\right)=-\frac{1}{2}{\int }_0^{\infty }{\overline{F}}^2\left(x\right)dx,$$

(28)

and the cumulative extropy is defined as [37]

$$\mathcal{CJ}\left(X\right)=-\frac{1}{2}{\int }_0^{\infty }(1-F^2\left(x\right))dx.$$

(29)

Sudheesh and Sreedevi (2022) [38] discussed non-parametric estimation of $\mathcal{CRJ}\left(X\right)$ and $\mathcal{CJ}\left(X\right)$ for right-censored data.

Recently, Balakrishnan et al. [29], Bansal and Gupta [39] and Sathar and Nair [40,41,42] introduced different weighted versions of extropy. The weighted version of the survival extropy is given by [41]

$$J(X,w)=-\frac{1}{2}{\int }_0^{\infty }x{\overline{F}}^2\left(x\right)dx.$$

These authors also introduced the weighted version of the cumulative extropy as

$$H(X,w)=-\frac{1}{2}{\int }_0^{\infty }x(1-F^2\left(x\right))dx,$$

and Sathar and Nair [42] subsequently defined dynamic survival extropy as

$$J_t\left(X\right)=-\frac{1}{2{\overline{F}}^2\left(t\right)}{\int }_t^{\infty }{\overline{F}}^2\left(x\right)dx.$$

For various properties of $J_t\left(X\right)$, one may see [36]. Sudheesh and Sreedevi (2022) [38] proposed simple alternative expressions for different extropy measures. Using these expressions, they established relationships between different dynamic and weighted extropy measures, and reliability concepts. In particular, they expressed $J_t\left(X\right)$ as

$$J_t\left(X\right)=-\frac{1}{2}E\left(min(X_1,X_2)-t|min(X_1,X_2)>t\right).$$

(30)

Thus, $-2J_t\left(X\right)$ is the mean residual life function of a series system having two identical components.

Sathar and Nair [41] defined weighted dynamic survival extropy as

$$J_t(X,w)=-\frac{1}{2{\overline{F}}^2\left(t\right)}{\int }_t^{\infty }x{\overline{F}}^2\left(x\right)dx.$$

Kundu (2021) [43] introduced dynamic cumulative extropy as

$$H_t\left(X\right)=-\frac{1}{2F^2\left(t\right)}{\int }_0^t{F}^2\left(x\right)dx,$$

while Sathar and Nair [41] defined the weighted dynamic cumulative extropy as

$$H_t(X,w)=-\frac{1}{2F^2\left(x\right)}{\int }_0^{t}x{F}^2\left(x\right)dx.$$

Sudheesh and Sreedevi (2022) [38] expressed $H_t\left(X\right)$ as

$$H_t\left(X\right)=-\frac{1}{2}E\left(t-max(X_1,X_2)|max(X_1,X_2)\le t\right).$$

(31)

Thus, $-2H_t\left(X\right)$ is the mean past life function of a parallel system having two identical components, where the mean past life function of a random variable X is defined as $E(t-X|X\le t).$

Next, we establish some connections between different entropy and extropy measures. For the choice of $w\left(u\right)=\overline{F}\left(u\right)$ and $\varphi \left(x\right)=x$, from (9), we obtain

$$\mathcal{GE}\left(X\right)={\int }_0^{\infty }\overline{F}\left(u\right)F\left(u\right)du={\int }_0^{\infty }\overline{F}\left(u\right)du-{\int }_0^{\infty }{\overline{F}}^2\left(u\right)du.$$

Thus, using (28), we have the relationship

$$\mathcal{GE}\left(X\right)=2\mathcal{CRJ}\left(X\right)+E\left(X\right).$$

Again, for the choices of $w\left(u\right)=\overline{F}\left(u\right)$ and $\varphi \left(x\right)=x^2$, from (9), we obtain

$$\mathcal{GE}\left(X\right)={\int }_0^{\infty }2u\overline{F}\left(u\right)F\left(u\right)du={\int }_0^{\infty }2u\overline{F}\left(u\right)du-{\int }_0^{\infty }2u{\overline{F}}^2\left(u\right)du.$$

For a non-negative random variable X, we have

$$E\left(X^2\right)={\int }_0^{\infty }{x}^{2}dF\left(x\right)={\int }_0^{\infty }2x\overline{F}\left(x\right)dx.$$

Thus, in this case, we obtain the relationship

$$\mathcal{GE}\left(X\right)=4\mathcal{CRJ}(X,w)+E\left(X^2\right).$$

For the choices of $w\left(u\right)=-F\left(u\right)$ and $\varphi \left(x\right)=x$, from (12), we obtain

$$\mathcal{GCE}\left(X\right)=-{\int }_0^{\infty }\overline{F}\left(u\right)F\left(u\right)du.$$

(32)

Using the identity $1-F^2\left(x\right)=\overline{F}\left(x\right)+\overline{F}\left(x\right)F\left(x\right)$, from (29) and (32), we have the relationship

$$\mathcal{GCE}\left(X\right)=2\mathcal{CJ}\left(X\right)+E\left(X\right).$$

Additionally, for the choices of $w\left(u\right)=-F\left(u\right)$ and $\varphi \left(x\right)=x^2$, from (12), we obtain

$$\mathcal{GCE}\left(X\right)=-{\int }_0^{\infty }2u\overline{F}\left(u\right)F\left(u\right)du.$$

Thus, in this case, we obtain the relationship

$$\mathcal{GCE}\left(X\right)=4\mathcal{CJ}(X,w)+E\left(X^2\right).$$

Let $X_1$ and $X_2$ be two independent and identical random variables having the same distribution function F. Let $Z=min(X_1,X_2)$ be the lifetime of a series system having two identical components. Using (9), we define the generalized residual entropy associated with Z as

$$\mathcal{GE}\left(Z\right)=\int 2w\left(u\right)E\left[\varphi \left(Z\right)-\varphi \left(u\right)|Z>u\right]\overline{F}\left(u\right)dF\left(u\right),$$

(33)

where $\varphi (.)$ is a function of Z and $w(.)$ is a weight function. Now, for the choices of $w\left(u\right)=-\frac{1}{4}$ and $\varphi \left(z\right)=z$, from (33), we obtain

$$\mathcal{GE}\left(Z\right)=-\frac{1}{2}\int E\left[min(X_1,X_2)-u|min(X_1,X_2)>u\right]\overline{F}\left(u\right)dF\left(u\right).$$

Thus, the generalized residual entropy associated with Z is the weighted average of the dynamic survival extropy in (30).

Next, let $Z=max(X_1,X_2)$ be the lifetime of a parallel system having two identical components. Then, the generalized cumulative entropy associated with Z is defined as

$$\mathcal{GCE}\left(Z\right)={\int }_0^{\infty }2w\left(u\right)E\left[\varphi \left(u\right)-\varphi \left(X\right)|X\le u\right]F\left(u\right)dF\left(u\right).$$

(34)

Again, for the choices of $w\left(u\right)=-\frac{1}{4}$ and $\varphi \left(z\right)=z$, from (34), we obtain

$$\mathcal{GCE}\left(Z\right)=-\frac{1}{2}{\int }_0^{\infty }E\left[u-max(X_1,X_2)|max(X_1,X_2)\le u\right]F\left(u\right)dF\left(u\right).$$

Thus, the generalized cumulative entropy associated with Z is the weighted average of the dynamic cumulative extropy in (31).

5. Concluding Remarks

In this work, we have introduced two general measures of entropy, viz., generalized cumulative residual entropy and generalized cumulative entropy. Several entropy measures known in the literature were all shown to be special cases of these generalized measures. Cumulative residual entropy, weighted cumulative residual entropy, cumulative residual Tsallis entropy and weighted cumulative residual Tsallis entropy are all special cases of the generalized cumulative residual entropy. Cumulative entropy, weighted cumulative entropy, cumulative Tsallis entropy and weighted cumulative Tsallis entropy are all special cases of the generalized cumulative entropy.

We have presented a generating function approach to obtain generalized measures of higher-order. We have shown that the generalized cumulative residual entropy of order two reduces to the weighted cumulative residual entropy of Mirali et al. (2016) [4]. Moreover, the weighted cumulative entropy of Mirali and Baratpour (2017) [5] is a special case of the generalized cumulative entropy of order two. We have also established some relationships between entropy and extropy measures.

In information theory literature, conditional entropy is the amount of information required to describe the outcome of one random variable Y, given the value of another random variable X. Conditional entropy, as a measure of information, can be defined through any measure, such as the Shannon entropy measure (denoted by $H\left(Y\right|X)$). The conditional entropy defined through Shannon entropy measure, for example, is given by

$$H\left(Y\right|X)=-{\int }_0^{\infty }{\int }_0^{\infty }f(x,y)log\frac{f(x,y)}{f\left(x\right)}dydx.$$

In this way, we can define the conditional entropy measures even in a generalized form. The generalized versions we have introduced in the present work can thus be extended to conditional entropy notions. However, we plan to carry out a detailed study of this in our future work. It will be of interest to develop some inferential methods for these measures as well. We are currently working in these directions and hope to report these finding in a future paper.