A Shift-Dependent Measure of Extended Cumulative Entropy and Its Applications in Blind Image Quality Assessment

Recently, Tahmasebi and Eskandarzadeh introduced a new extended cumulative entropy (ECE). In this paper, we present results on shift-dependent measure of ECE and its dynamic past version. These results contain stochastic order, upper and lower bounds, the symmetry property and some relationships with other reliability functions. We also discuss some properties of conditional weighted ECE under some assumptions. Finally, we propose a nonparametric estimator of this new measure and study its practical results in blind image quality assessment.


Introduction
Differential entropy is a basic concept in the field of information theory. The central idea of information theory revolves around the concept of uncertainty introduced by Shannon [1]. If X is a random variable representing the lifetime of a system with probability density function (PDF) f , then the Shannon entropy of X is given by Later, Rényi [2] introduced another extension of the Shannon entropy that is more flexible than Shannon entropy and has a wide range of applications in many fields. The Rényi entropy of X, which we denote by H α (X), is defined as follows: By replacing the PDF by the survival functionF = 1 − F in (1), Rao et al. [3] defined an alternate information measure called the cumulative residual entropy (CRE) given by where Λ(x) = − logF(x). Di Crescenzo and Longobardi [4] introduced a new information measure similar to E (X) as follows: whereΛ(x) = − log F(x). Recently Di Crescenzo and Toomaj [5] discussed some properties of a new weighted distribution based on a cumulative entropy (CE) function. Psarrakos and Navarro [6] generalized the concept of CRE, relating this concept with the mean time between record values and with the concept of relevation transform, and also considered a dynamic version of this new measure (for more details see Calí, Longobardi and Psarrakos, [7]). Moharana and Kayal [8] obtained some results on the weighted extended cumulative residual entropy of k-th upper record values. Tahmasebi et al. [9] considered a shift-dependent measure of generalized cumulative entropy and its dynamic version in the case where the weight is a general non-negative function. An important concept of ordered random variables which arises in many areas of applications is the concept of record values. Consider the sequence {X n , n ≥ 1} of independent and identically distributed random variables with cumulative distribution function (CDF) F and PDF f . An observation X j is called a lower record if X j < X i for every i < j. For a fixed positive integer k, the sequence {L n(k) , n ≥ 1} of k-th lower record times for {X n , n ≥ 1} is defined by Dziubdziela and Kopocinski [10] as follows: L 1(k) = 1, L n+1(k) = min{j > L n(k) : X k:L n(k) +k−1 > X k:k+j−1 }, where X j:m denotes the j-th order statistic in a sample of size m. Then X n(k) := X k:L n(k) +k−1 is called a sequence of k-th lower record values of {X n , n ≥ 1}. Aditionally, the PDF and CDF of X n(k) , which are denoted by f n(k) and F n(k) , respectively, are given by Now, if we defineμ n,k (x) = +∞ 0 F n(k) (x)dx, from (5) Tahmasebi and Eskandarzadeh [11] defined a further extension of CE as follows: , f or n = 1, 2, ..., k ≥ 1, where r(.) = f (.) F(.) is the reversed failure rate of F. This new CE is presented on the idea of GCRE introduced by Psarrakos and Navarro [6]. They named it extended cumulative entropy (ECE).
Non-reference image quality assessment (IQA) methods give quality estimates without prior knowledge of the reference image, and quality assessment is done based on the test images only. Image quality approaches largely depend on the intended imaging area. However, making an objective general quality assessment of image information based on a physical measurement of the image is interesting. Shannon entropy is classically used as a value to indicate the amount of uncertainty or information in a source. Quality and entropy are related issues. However, a barrier to entropy as the quality indicator is that it can't distinguish the noise of an image from the desired information. Hence, Shannon entropy is not a good indicator of image quality by itself. Overcoming this problem is presented by anisotropy as an appropriate measure of image quality. Degradation processes damage the directional scene's information. Hence, anisotropy, as a directionally dependent quality, is reduced by adding further damage to the image. Using neuroscience research, the local receptive field (LRF) in the primary visual cortex is highly adaptable to extract local features for visual comprehension, and simple cells in the LRF can be described as being used for localization and spatial orientation. In other words, the LRF is very sensitive to changes in intensity and orientation [12]. Therefore, visual information of an image can be represented by the local intensity and local orientation of the image. Thus, an image quality index must consider the local intensity and local orientation information of an image. The anisotropic quality index (AQI) uses Rényi entropy as the basic criterion for the measuring of the image information content using the local intensity. For this purpose, as a first step the pseudo-winger distribution of the symmetric neighbors of each pixel at different directions is calculated. Then the Rényi entropy of the obtained values is computed. Furthermore, AQI calculates the entropy at different directions in order to consider the local orientation information of an image. Although it seems that AQI considers the local intensity and local orientation information of an image for image quality estimation, Rényi entropy only uses the distribution of the local intensity of pixels, and exact value of pixels are not used. Hence in this paper we propose a novel entropy measure which considers the distribution and exact value of pixels simultaneously. The results on three test images show the benefits of the proposed new measure of entropy. For this purpose, we present results on a shift-dependent measure of ECE and its dynamic past version. We also study the numerical results of ECE in blind image quality assessment. Therefore, the rest of this paper is organized as follows: In Section 2, we present some basic properties and the stochastic ordering of a weighted ECE (denoted by WECE). We also obtain some results from the dynamic version of the WECE. In Section 3, we study some properties of the conditional WECE. In Section 4, we state some relationships of the WECE with other concepts of reliability functions. Finally, in Section 5, using the nonparametric estimator of WECE, numerical results of a blind image quality assessment are presented.

Some Results on WECE and Its Dynamic Past Version
In this section, we first present some properties of the WECE and then consider the dynamic past version of this concept. Definition 1. Let X be a non-negative random variable with CDF F. Then, the WECE is defined as follows: Furthermore, from (5) we can obtain an alternative expression as Remark 1. Let X be a non-negative absolutely continuous random variable: i. If X is uniformly distributed in [0, θ], then, CE w n,k (X) = θ 2 ( k k+2 ) n+1 . ii. If X has the Fréchet distribution with F(x) = e −θ x , then for n > 2 we have CE w n,k (X) = k 3 θ 2 n(n − 1)(n − 2) = k 3 CE w n,1 (X).
In the following, we prove important properties of the WECE using stochastic ordering. For that we present the following definition: Definition 2. Let X and Y be the non-negative random variables with CDFs F and G, respectively, then 1. X is smaller than Y in the usual stochastic order (denoted by is increasing in x; 3. X is smaller than Y in the reversed hazard rate order, denoted by X ≤ rhr Y, if r X (x) ≥ r Y (x) for all x; 4. X is smaller than Y in the decreasing convex order, denoted by X ≤ dcx Y, if E(φ(X)) ≤ E(φ(Y)) for all decreasing convex functions φ such that the expectations exist; 5. X is smaller than Y in the dispersive order, denoted by where F −1 and G −1 are right continuous inverses of F and G, respectively; 6. A non-negative random variable X is said to have a decreasing reversed hazard rate (DRHR) if r X (x) = f (x) F(x) is decreasing in x; 7. A non-negative random variable X is said to have a decreasing reversed hazard rate average (DRHRA) if x is a decreasing function in x > 0. Note that DRHR classes of distributions are included in DRHRA classes of distributions. Theorem 1. Let X be an absolutely continuous non-negative random variable with CDF F. If X is DRHRA, Proof.
Since the ratio . This is equivalent (Shaked and Shanthikumar [13], (p. 4)) to having E(φ(X n+2(k) )) ≤ E(φ(X n+1(k) )), for all increasing functions φ such that these expectations exist. Hence, if X is DRHRA and r X is its reversed hazard rate, then we have and this completes the proof.

Remark 2.
Assume that the non-negative random variable X is DRHRA, then we have Remark 3. Let X and Y be two non-negative random variables with finit functions CE w n,k (X) and CE w n,k (Y), respectively. If X ≤ rhr Y and x r X (x) is an increasing function of x, then Proposition 1. Let X and Y be non-negative random variables with CDFs F and G, respectively. If X ≤ disp Y, then we have Proof. See Lemma 3 in Klein et al. [14].

Proposition 2.
Let X and Y be two independent non-negative random variables with distribution functions F and G, respectively. If X and Y have log-concave densities, then Proof. See Theorem 3.2 of Di Crescenzo and Toomaj [5].
Proposition 3. Let X be a non-negative absolutely continuous random variable with CDF F. Then, Proof. The proof is similar to that Proposition 4.3 of Di Crescenzo and Longobardi [4].

Proposition 4.
Let X be a non-negative random variable with CDF F, then for any k ≥ 1 we have where CE w n (X) is the shift-dependent GCE of order n (see Kayal and Moharana [15]).
Assume that X and Y are the lifetimes of two components of a system with joint distribution function F(x, y). Then the bivariate WECE can be defined as whereΛ(x, y) = − log F(x, y). Using the binomial expansion in (14), we obtain the following proposition.
Proposition 5. Let X and Y be the independent random variables with joint distribution function F(x, y), then using the symmetry property we have Suppose that X ia a random lifetime of a system with CDF F, then we state that X [t] = (t − X | X < t) describes the inactivity time of the system. Analogously, we can also consider the dynamic past version of WECE for for n = 1, 2, . . . , and k ≥ 1. This function is called a weighted dynamic extension cumulative entropy (WDECE).
Proposition 6. Let X be a non-negative absolutely continuous random variable with CDF F. Then, iii.
Proposition 7. Suppose that the non-negative random variable X is DRHRA, then for t > 0 we have Proof. We recall that if X is DRHRA, then X [t] is DRHRA and the proof follows from Theorem 1.

Remark 4.
Assume that the non-negative random variable X is DRHRA, then we have Theorem 2. Let X be a non-negative absolutely continuous random variable with CDF F , then Proof. The proof is similar to that Theorem 4 of Tahmasebi et al. [9].
Proposition 8. Let X be a non-negative random variable with CDF F, then we have Proposition 9. Suppose that the non-negative random variable X is DRHRA, then CE w n,k (X; t) is increasing in t > 0 for n = 1, 2, ... and k ≥ 1.

Definition 3.
We state that the non-negative random variable X has an increasing WDECE of order n (denoted by IWDECE n ) if CE w n,k (X; t) is increasing in t.

Remark 5.
Let X be a non-negative random variable with CDF F. If X is DRHRA, then it is IWDECE n for n = 1, 2, . . . and k ≥ 1.
Proof. Suppose that X is IWDECE n−1 . Then, by recalling Proposition 8 we have Furthermore, (19) implies that ∂ ∂t CE w n,1 (X; t) ≥ 0 and X is IWDECE n .

Properties of Conditional WECE
Let X be a random variable on a probability space (Ω, F , P) such that E|X| < ∞. We denote by E(X|G) the conditional expectation of X given sub σ-field G, where G ⊂ F . Here, we define the conditional WECE and discuss some of its properties.

Definition 4.
Suppose that X is a non-negative random variable with CDF F. Then for a given σ-field F , the conditional WECE is defined as follows:

Lemma 1.
Suppose that X is a non-negative random variable with CDF F. If F = {φ, Ω}, then CE w n,k (X|F ) = CE w n,k (X).
Proof. The proof follows by applying Jensen's inequality for the convex function x k (−logx) n , 0 < x < 1 as and the result follows.
Lemma 2. Let X, Y and Z be the non-negative random variables. If X → Y → Z is a Markov chain, then we have Proof.
(i) By using the Markov property and definition of CE w n,k (Z|Y, X), the result follows. (ii) Let G = σ(X) and F = σ(X, Y), then from (20) we have and the result follows.
Theorem 3. Let X ∈ L p for some p > 2 be a non-negative random variable with CDF F and F be a σ− field. Then E(CE w n,k (X|F )) = 0 if X is F -measurable.
Proof. Let E(CE w n,k (X|F )) = 0, then CE w n,k (X|F ) = 0. By using the definition of CE w n,k (X|F ) we conclude that E(I (X≤x) |F ) = 0 or 1. Thus, using the relation (24) of Rao et al. [3], X is F -measurable. Supposing that X is F -measurable, again using relation (24) of Rao et al. [3], we have P(X ≤ x|F ) = 0 or 1 for almost all x ∈ R + , thus the result follows.
Theorem 4. Let X be a non-negative random variable with CDF F and F be a σ− field, then we have E(CE w n,k (X|F )) ≤ CE w n,k (X), (21) and the equality holds if, and only if, X is independent of F . (21) follows from (20) by taking F = {φ, Ω}. Assume that X is independent of F , then clearly

Proof. The inequality
By using Definition 4 and (20), we have E(CE w n,k (X|F )) = CE w n,k (X).
Conversely, suppose that there is equality in (21). We put W :

Relationships with Other Reliability Functions
In this section, we state some relationships of CE w n,k (X) and CE w n,k (X; t) with other concepts such as the reversed hazard rate function and the weighted mean inactivity time of the random variable [t − X n(k) | X n(k) < t].
Theorem 5. Let X be an absolutely continuous non-negative random variable with PDF f and CDF F. Then for n ≥ 1 we have Proof. By (8) and the relationΛ(x) = ∞ x r(z)dz, we have Using Fubini's theorem, we obtain and the result follows. Now, we define the weighted mean inactivity time of the random variable [t − X n(k) | X n(k) < t] as follows:M w n,k (t) = M w n,k (t) is analogous to the mean residual waiting time used in reliability analysis (Bdair and Raqab [16]).

Theorem 6.
For a non-negative absolutely continuous random variable X with CE w n,k (X) < ∞, we have Proof. From relation (23) and (24), we get and this completes the proof.
From (24), we can obtain the following result as where and To obtain a connection betweenM w n,k (t) and CE w n,k (X; t) we need the following lemma.
Lemma 3. Let X be a non-negative random variable with CDF F. Then we have Proof. From (26), we have In the following, we can obtain the connection betweenM w n,k (t) and CE w n,k (X; t).
Theorem 7. Let X be a non-negative random variable with CDF F, then for n ≥ 1 we havẽ Proof. By (25) and (28), we havẽ and this completes the proof.
Theorem 8. Let X be a non-negative random variable with CDF F, then for any n ≥ 1 we have whereM w n,k (t) = 1 F n(k) (t) t 0 xF n(k) (x)dx, n = 1, 2, 3, ... is the weighted mean inactivity time of X n(k) .

Proof. From (5), we see that
Substituting this equation in (23) we have and the result follows.

Remark 6.
For a non-negative absolutely continuous random variable X with CE w n,k (X) < ∞, we have

Application of CE w n,k (X) in Blind Image Quality Assessment
Suppose that X 1 , X 2 , ..., X m is a random sample of size m from CDF F(x). If X (1) ≤ X (2) ≤ ... ≤ X (m) represent the order statistics of X 1 , X 2 , ..., X m , then the empirical measure of F(x) for i = 1, 2, ..., m − 1 is defined asF x ≥ X (m) .
Thus the empirical measure of CE w n,k (X) is obtained as Theorem 14 of Tahmasebi et al. [9]). In the following example we present applications of CE w n,k (F m ) in blind image quality assessment. (Blind Image Quality Assessment). In this example a modified anisotropic image quality (AIQ) measure based on the WECE is used as a blind image quality index, which we call WECE-AIQ. The old AIQ is based on the using of Rényi entropy and the normalized pseudo-Wigner distribution [17]. We call this measure Rényi-AIQ. Dataset [18,19] is used in this example for blind image quality assessment. The dataset contains distorted images of three grayscale reference images: a horse, a harbor and a baby (Figure 1). The size and pixel values of the images are 512 × 512 and in the range 0-255, respectively. The reference images are distorted using "flat allocation"; quantization of the LH sub-bands of a 5-level DWT of the image with equal distortion contrast at each scale (FLT), baseline JPEG compression (JPG), baseline JPEG-2000 compression (JP2), JPEG-2000+DCQ compression (DCQ), Gaussian blur filter (BLR) and additive Gaussian white noise (AGWN). These distortions are utilized to reference images in three levels: low quality (LQ), mid quality (MQ) and good quality (GQ). In this example, WECE is used instead of Rényi entropy for the estimation of the AIQ metric, and k and n are selected as 2 and 4 for WECE, respectively. For the assessment of the Rényi-AIQ and WECE-AIQ metrics, some full-reference image quality metrics are needed: PSNR, WSNR, a weighted SNR [20], a universal quality index (UQI) [21], a noise quality measure (NQM) [22], a structural similarity metric (SSIM) [23], a visual information fidelity (VIF) metric [24] and a visual SNR (VSNR) [18]. A bigger value of each of these metrics indicates a better quality of an image. The values of these metrics are available for images of the database used in this example [19]. Note that only gray scale images are considered in this example. For color images, only spatial structures cannot properly demonstrate the quality of an image. Visual damage caused by distortion of the image's color must be considered. Therefore, a criterion for color distortion must be used. The color image can be decomposed into different color spaces such as RGB, CIE, YCbCr, YIQ, HIS etc. [25]. LMN space, with the optimized weights that are suitable for the human visual system (HVS), can be a good choice [25]. L is the luminance channel for evaluating the structure distortions of the images, and M and N are two chrominance channels which are used to characterize the image quality degradation caused by color distortions. an image quality metric is applied on the L channel for structure distortions measurement and on the M and N channels for color distortions measurement. The values of Rényi-AIQ, WECE-AIQ and full-reference metrics are depicted in Table 1. The biggest value of Rényi-AIQ and WECE-AIQ metrics are shown using bold numbers for each image. The performance of WECE-AIQ and Rényi-AIQ is measured using the times in which a full-reference criterion of the selected image of each approach is larger than in the other approaches. It can be seen from Table 1 which WECE-AIQ displayed a better performance than Rényi-AIQ for the "Horse (GQ)", "Horse (LQ)", "Harbor (GQ)", "Harbor (MQ)" and "Harbor (LQ)" images. This shows that the quality of the selected images using the WECE-AIQ metric is better than the ones which were selected using the Rényi-AIQ metric. For visual analysis of the results of Table 1, corresponding images with the biggest values of Rényi-AIQ and WECE-AIQ metrics are shown in Figures 2-4. It can be seen that in most cases, the visual quality of images which were selected using the WECE-AIQ metric was higher than the ones which were chosen using Rényi-AIQ. For more analysis of the results of Table 1, Spearman's rank correlation coefficient (SRCC) was used in this example [26]. The results of this measure are shown in Table 2. Table 2 shows the SRCC between full-reference and blind image quality metrics for each image. Bold numbers show the bigger SRCC value of each full-reference metrics. In general, the Spearman's rank correlation coefficient range is [−1, 1]. In this example, each blind image quality metric that has a bigger Spearman's rank correlation coefficient value than others is more useful for image quality assessment. Table 2 shows that for all images, the performance of WECE-AIQ was better than Rényi-AIQ. Additionally, the performance of WECE-AIQ for the harbor image was better than for the horse and baby image. The corresponding SRCC values of WECE-AIQ for the harbor image were positive in most cases. This shows that the quality ranks of images, which are selected using WECE-AIQ, are very similar to the quality ranks of full-reference metrics. Hence it seems that WECE-AIQ has worked much more effectively than Rényi-AIQ on the harbor image. Indeed, none of the full-reference image quality metric had a high correlation with the HVS. The accuracy of each one depends on the distortion type, context and texture of the distorted image. Therefore in general, the quality of a distorted image is evaluated using some of the full-reference image quality criteria. For further investigation of this subject, the Spearman's rank correlation coefficients (SRCCs) between each of the full-reference criterions of the horse image are illustrated at Table 3. Contrary to what was expected, it is seen that the correlation between the full-reference criteria was not high in most cases. Additionally, as can be seen in Table 2, the correlation of Rényi-AIQ and WECE-AIQ with the full-reference image quality criteria was not high. This is due to the fact that each criterion evaluates the distorted image from a different point of view compared with the others. For example, PSNR calculates the difference between the distorted and reference images, while SSIM is based on the structural similarity between them. Indeed, none of the full-reference image quality criteria consider all of the properties of HVS. Therefore, in this research the performance of Rényi-AIQ and WECE-AIQ have been evaluated using the correlation between them and all of the full-reference image quality criteria.  Best quality images that were selected using the Rényi-AIQ metric from GQ, MQ and LQ distorted horse images (a, c and e, respectively), and best quality images that were selected using the WECE-AIQ metric from GQ, MQ and LQ distorted horse images (b, d and f, respectively).

Figure 3.
Best quality images that were selected using the Rényi-AIQ metric from GQ, MQ and LQ distorted harbor images (a, c and e, respectively), and best quality images that were selected using the WECE-AIQ metric from GQ, MQ and LQ distorted harbor images (b, d and f, respectively).

Figure 4.
Best quality images that were selected using the Rényi-AIQ metric from GQ, MQ and LQ distorted baby images (a, c and e, respectively), and best quality images that were selected using the WECE-AIQ metric from GQ, MQ and LQ distorted baby images (b, d and f, respectively).

Conclusions
In this paper, we have presented some results of the WECE and its dynamic past version. These results included stochastic ordering, bounds and some relationships with other reliability concepts.
Additionally, we examined the conditional WECE, which can be applied in measuring the uncertainty in blind image quality assessment. Finally, we proposed a nonparametric estimator of WECE and studied the numerical results of WECE in a blind image quality assessment. Furthermore, it can be seen that in most cases, the visual quality of images that were selected using the WECE-AIQ metric was higher than for images that were chosen using the Rényi-AIQ metric. It was shown that the quality rank of images which are selected using the WECE-AIQ are very similar to the quality ranks of the full-reference metrics.
Author Contributions: All authors contributed equally to the manuscript. All authors have read and agreed to the published version of the manuscript.
Funding: This research received no external funding.