Residual and Past Discrete Tsallis and Renyi Extropy with an Application to Softmax Function

Abstract

In this paper, based on the discrete lifetime distribution, the residual and past of the Tsallis and Renyi extropy are introduced as new measures of information. Moreover, some of their properties and their relation to other measures are discussed. Furthermore, an example of a uniform distribution of the obtained models is given. Moreover, the softmax function can be used as a discrete probability distribution function with a unity sum. Thus, applying those measures to the softmax function for simulated and real data is demonstrated. Besides, for real data, the softmax data are fit to a convenient ARIMA model.

1. Introduction

The entropy function has resulted in a substantial turn in the theory of information, such as the measure of uncertainty information. For a discrete random variable (R.V.) $X=\{x_1,x_2,\dots ,x_{\mathfrak{N}}\}$, and $\mathbf{p}$ $=\{p_1,p_2,\dots ,p_{\mathfrak{N}}\}$ is the corresponding probability vector, $p_i=P(X=x_i),$$i=1,2,\dots ,\mathfrak{N}$. Thus, Shannon [1] produced the non-negative discrete entropy function as follows:

$$S\left(\mathbf{p}\right)=-\sum _{i=1}^{\mathfrak{N}}{p}_{i}ln{p}_i.$$

(1)

Lad et al. [2] originated the extropy as a dual model of uncertainty. The non-negative extropy of the discrete R.V. X is given by

$$ES\left(\mathbf{p}\right)=-\sum _{i=1}^{\mathfrak{N}}(1-p_i)ln(1-p_i).$$

(2)

Several measures of entropy and its generalization have been presented in the literature. Through the various generalizations of uncertainty, Tsallis [3] introduced the Tsallis entropy. The Tsallis entropy of the discrete R.V. X, $1\ne \alpha >0$, is given by

$$T_{\alpha }\left(\mathbf{p}\right)=\frac{1}{\alpha -1}\left(1-\sum _{i=1}^{\mathfrak{N}}{p}_i^{\alpha }\right)=\frac{1}{\alpha -1}\left(\sum _{i=1}^{\mathfrak{N}}{p}_i-\sum _{i=1}^{\mathfrak{N}}{p}_i^{\alpha }\right),$$

(3)

when $\alpha$ is 1, then ${lim}_{\alpha \to 1}{T}_{\alpha }\left(\mathbf{p}\right)=S\left(\mathbf{p}\right)$.

Recently, Xue and Deng [4] proposed the measure Tsallis extropy, as a complementary dual of the Tsallis uncertainty function and studied its maximum value. Then, Balakrishnan et al. [5] studied the Tsallis extropy and applied it to pattern recognition. The Tsallis extropy of the discrete R.V. X, $1\ne \alpha >0$, is given by

$$TE{S}_{\alpha }\left(\mathbf{p}\right)=\frac{1}{\alpha -1}\left(\sum _{i=1}^{\mathfrak{N}}(1-p_i)-\sum _{i=1}^{\mathfrak{N}}{(1-p_i)}^{\alpha }\right)=\frac{1}{\alpha -1}\left(\mathfrak{N}-1-\sum _{i=1}^{\mathfrak{N}}{(1-p_i)}^{\alpha }\right),$$

(4)

when $\alpha$ is 1, then ${lim}_{\alpha \to 1}TE{S}_{\alpha }\left(\mathbf{p}\right)=E{S}_{\alpha }\left(\mathbf{p}\right)$.

Based on the continuous lifetime distribution, Ebrahimi [6] discussed the measure of the entropy of the residual lifetime distribution. Furthermore, Di Crescenzo and Longobardi [7] presented the measure of the entropy of the past lifetime distribution.

Based on the discrete lifetime distribution, Gao and Shang [8] developed the generalized past entropy via the grain exponent based on oscillations. Li and Shang [9] introduced the modified discrete generalized past entropy via the grain exponent.

The innovation of this paper lies in presenting the residual and past versions of the Tsallis and Renyi extropy based on the discrete lifetime distribution. Moreover, we apply those models to the softmax function for simulated and real data. Besides, we discuss the residuals of the softmax real data and give a suitable ARIMA model to fit the data. The remaining article is structured as follows: Section 2 gives the suggested models with their properties and relation to other measures. Besides, we give an example of the models for a uniform distribution. Section 3 applies the softmax function to the suggested models on simulated and real data. Furthermore, we discuss the residuals and the fit ARIMA model to the softmax U.S. consumption data. Finally, Section 4 finishes the article with some conclusions.

2. The Suggested Models

Extropy is a favorite instrument to enhance the Tsallis entropy, which is the dual model of information entropy. The Tsallis extropy combines the advantages of the Tsallis entropy and extropy. In this section, we present the residual and past versions of the Tsallis extropy based on the discrete lifetime distribution.

Let the discrete R.V. X supported with $S=\{x_1,x_2,\dots ,x_{\mathfrak{N}}\}$, and the corresponding probability vector $\mathbf{p}=\{p_1,p_2,\dots ,p_{\mathfrak{N}}\}$. Then, the residual and past discrete Tsallis entropy, $1\ne \alpha >0$, is given, respectively, by

$$R{T}_{\alpha }(\mathbf{p};t)=\frac{1}{\alpha -1}\left(1-\sum _{i=t}^{\mathfrak{N}}{\left(\frac{p_i}{{\overline{P}}_t}\right)}^{\alpha }\right)=\frac{1}{\alpha -1}\left(\sum _{i=1}^{\mathfrak{N}}{p}_i-\sum _{i=t}^{\mathfrak{N}}{\left(\frac{p_i}{{\overline{P}}_t}\right)}^{\alpha }\right),$$

(5)

$$P{T}_{\alpha }(\mathbf{p};t)=\frac{1}{\alpha -1}\left(1-\sum _{i=1}^t{\left(\frac{p_i}{P_t}\right)}^{\alpha }\right)=\frac{1}{\alpha -1}\left(\sum _{i=1}^{\mathfrak{N}}{p}_i-\sum _{i=1}^t{\left(\frac{p_i}{P_t}\right)}^{\alpha }\right),$$

(6)

where $P_t=P(X\le t)={\sum }_{i=1}^t{p}_i$ and ${\overline{P}}_t=1-P_t$, and the interceptive parameter t is between 1 and $\mathfrak{N}$. Moreover, the residual and past discrete extropy is given, respectively, by

$$RES(\mathbf{p};t)=-\sum _{i=t}^{\mathfrak{N}}\left(1-\frac{p_i}{{\overline{P}}_t}\right)ln\left(1-\frac{p_i}{{\overline{P}}_t}\right),$$

(7)

$$PES(\mathbf{p};t)=-\sum _{i=1}^t\left(1-\frac{p_i}{P_t}\right)ln\left(1-\frac{p_i}{P_t}\right).$$

(8)

Motivated by the concept of the Tsallis extropy and the discrete lifetime distribution, we introduce the residual and past of the Tsallis extropy, respectively, as follows:

$$RTE{S}_{\alpha }(\mathbf{p};t)=\frac{1}{\alpha -1}\left(\sum _{i=1}^{\mathfrak{N}}(1-p_i)-\sum _{i=t}^{\mathfrak{N}}{\left(1-\frac{p_i}{{\overline{P}}_t}\right)}^{\alpha }\right)=\frac{1}{\alpha -1}\left(\mathfrak{N}-1-\sum _{i=t}^{\mathfrak{N}}{\left(1-\frac{p_i}{{\overline{P}}_t}\right)}^{\alpha }\right),$$

(9)

$$PTE{S}_{\alpha }(\mathbf{p};t)=\frac{1}{\alpha -1}\left(\sum _{i=1}^{\mathfrak{N}}(1-p_i)-\sum _{i=1}^t{\left(1-\frac{p_i}{P_t}\right)}^{\alpha }\right)=\frac{1}{\alpha -1}\left(\mathfrak{N}-1-\sum _{i=1}^t{\left(1-\frac{p_i}{P_t}\right)}^{\alpha }\right).$$

(10)

In the following proposition, we illustrate the past and residual Tsallis extropy behavior according to the value of $\alpha$.

Proposition 1.

Let the discrete R.V. X supported with $S=\{x_1,x_2,\dots ,x_{\mathfrak{N}}\}$, and the corresponding probability vector $\mathit{p}=\{p_1,p_2,\dots ,p_{\mathfrak{N}}\}$. Then, from the past and residual Tsallis extropy given in Equations (10) and (9), respectively, we have:

1. 

The past Tsallis extropy is positive (negative) if $\alpha >1$ ($0<\alpha <1$).

2. 

The residual Tsallis extropy is positive (negative) if $\alpha >1$ ($0<\alpha <1$).

Proof. 

• The past Tsallis extropy given in Equation (10) can be rewritten as follows:

  $$\begin{array}{cc}\hfill PTE{S}_{\alpha }(\mathbf{p};t)& =\frac{1}{\alpha -1}\left(\sum _{i=1}^{\mathfrak{N}}(1-p_i)-\sum _{i=1}^t{\left(1-\frac{p_i}{P_t}\right)}^{\alpha }\right)\hfill \\ & =\frac{1}{\alpha -1}\left(\sum _{i=1}^{\mathfrak{N}}(1-P_t+P_t-p_i)-\sum _{i=1}^t{\left(1-\frac{p_i}{P_t}\right)}^{\alpha }\right)\hfill \\ & =\frac{1}{\alpha -1}\left(\mathfrak{N}(1-P_t)+\sum _{i=t+1}^{\mathfrak{N}}(P_t-p_i)+\sum _{i=1}^t(P_t-p_i)\left(1-\frac{1}{P_t}{\left(1-\frac{p_i}{P_t}\right)}^{\alpha -1}\right)\right).\hfill \end{array}$$
  Moreover, we have that the term $\mathfrak{N}(1-P_t)\ge 0$ is greater than the other two terms, and $P_t\ge p_i,i=1,2,\dots ,\mathfrak{N}$. Then, the proof is obtained.
• The residual Tsallis extropy given in Equation (9) can be rewritten as follows:

  $$\begin{array}{cc}\hfill RTE{S}_{\alpha }(\mathbf{p};t)& =\frac{1}{\alpha -1}\left(\sum _{i=1}^{\mathfrak{N}}(1-{\overline{P}}_t+{\overline{P}}_t-p_i)-\sum _{i=t}^{\mathfrak{N}}{\left(1-\frac{p_i}{{\overline{P}}_t}\right)}^{\alpha }\right)\hfill \\ & =\frac{1}{\alpha -1}\left(\mathfrak{N}(1-{\overline{P}}_t)+\sum _{i=1}^{t-1}({\overline{P}}_t-p_i)+\sum _{i=t}^{\mathfrak{N}}({\overline{P}}_t-p_i)\left(1-\frac{1}{{\overline{P}}_t}{\left(1-\frac{p_i}{{\overline{P}}_t}\right)}^{\alpha -1}\right)\right).\hfill \end{array}$$
  Moreover, the proof is obtained similarly.

□

Example 1.

Suppose that the discrete R.V. X has a uniform distribution over $\{1,\dots ,\mathfrak{N}\}$. Then, the residual and past Tsallis extropy are given, respectively, by

$$\begin{array}{cc}\hfill RTE{S}_{\alpha }(\mathit{p};t)& =\frac{1}{\alpha -1}\left(\mathfrak{N}-1-\sum _{i=t}^{\mathfrak{N}}{\left(1-\frac{1}{\mathfrak{N}-t}\right)}^{\alpha }\right)\hfill \\ & =\frac{1}{\alpha -1}\left(\mathfrak{N}-1-(\mathfrak{N}-t+1){\left(1-\frac{1}{\mathfrak{N}-t}\right)}^{\alpha }\right).\hfill \end{array}$$

$$\begin{array}{cc}\hfill PTE{S}_{\alpha }(\mathit{p};t)& =\frac{1}{\alpha -1}\left(\mathfrak{N}-1-\sum _{i=1}^t{\left(1-\frac{1}{t}\right)}^{\alpha }\right)\hfill \\ & =\frac{1}{\alpha -1}\left(\mathfrak{N}-1-t{\left(1-\frac{1}{t}\right)}^{\alpha }\right).\hfill \end{array}$$

In the following proposition, we will establish the relation between the past and residual Tsallis extropy and the past and residual extropy.

Proposition 2.

Suppose that the discrete R.V. X has finite support with corresponding probability vector $\mathit{p}=\{p_1,p_2,\dots ,p_{\mathfrak{N}}\}$. Then, from the past and residual Tsallis extropy given in Equations (10) and (9), respectively, and the past and residual extropy given in Equations (8) and (7), respectively, we have:

1. 

From Equations (10) and (8), we have

$$\underset{\alpha \to 1}{lim}PTE{S}_{\alpha }(\mathit{p};t)=PES(\mathit{p};t).$$

2. 

From Equations (9) and (7), we have

$$\underset{\alpha \to 1}{lim}RTE{S}_{\alpha }(\mathit{p};t)=RES(\mathit{p};t).$$

Proof. 

• From (10) and applying $L^{\prime }H\widehat{o}pita{l}^{\prime }s$ rule, we have

  $$\begin{array}{cc}\hfill \underset{\alpha \to 1}{lim}PTE{S}_{\alpha }(\mathbf{p};t)& =\underset{\alpha \to 1}{lim}\frac{1}{\alpha -1}\left(\mathfrak{N}-1-\sum _{i=1}^t{\left(1-\frac{p_i}{P_t}\right)}^{\alpha }\right)\hfill \\ \hfill & =\underset{\alpha \to 1}{lim}-\sum _{i=1}^t{\left(1-\frac{p_i}{P_t}\right)}^{\alpha }ln\left(1-\frac{p_i}{P_t}\right).\hfill \\ \hfill & =-\sum _{i=1}^t\left(1-\frac{p_i}{P_t}\right)ln\left(1-\frac{p_i}{P_t}\right)\hfill \\ \hfill & =PES(\mathbf{p};t).\hfill \end{array}$$
• From (9) and applying $L^{\prime }H\widehat{o}pita{l}^{\prime }s$ rule, we have

  $$\begin{array}{cc}\hfill \underset{\alpha \to 1}{lim}RTE{S}_{\alpha }(\mathbf{p};t)& =\underset{\alpha \to 1}{lim}\frac{1}{\alpha -1}\left(\mathfrak{N}-1-\sum _{i=t}^{\mathfrak{N}}{\left(1-\frac{p_i}{{\overline{P}}_t}\right)}^{\alpha }\right)\hfill \\ \hfill & =\underset{\alpha \to 1}{lim}-\sum _{i=t}^{\mathfrak{N}}{\left(1-\frac{p_i}{{\overline{P}}_t}\right)}^{\alpha }ln\left(1-\frac{p_i}{{\overline{P}}_t}\right).\hfill \\ \hfill & =-\sum _{i=t}^{\mathfrak{N}}\left(1-\frac{p_i}{{\overline{P}}_t}\right)ln\left(1-\frac{p_i}{{\overline{P}}_t}\right)\hfill \\ \hfill & =RES(\mathbf{p};t).\hfill \end{array}$$

□

In the next proposition, we will obtain the relation between the past and residual Tsallis extropy and the past and residual Tsallis entropy when the choice of the parameter $\alpha =2$.

Proposition 3.

Suppose that the discrete R.V. X has finite support with the corresponding probability vector $\mathit{p}=\{p_1,p_2,\dots ,p_{\mathfrak{N}}\}$. Then, from the past and residual Tsallis extropy given in Equations (10) and (9), respectively, and the past and residual Tsallis entropy given in Equations (6) and (5), respectively, we have:

1. 

From Equations (10) and (6), we have

$$PTE{S}_2(\mathit{p};t)=\mathfrak{N}-t+P{T}_2(\mathit{p};t),$$

if $t=\mathfrak{N}$, then $PTE{S}_2(\mathit{p};t)=P{T}_2(\mathit{p};t)$.

2. 

From Equations (9) and (5), we have

$$RTE{S}_2(\mathit{p};t)=t-1+R{T}_2(\mathit{p};t),$$

if $t=1$, then $RTE{S}_2(\mathit{p};t)=R{T}_2(\mathit{p};t)$.

Proof. 

• From (10), when $\alpha =2$, we have

  $$\begin{array}{cc}\hfill PTE{S}_2(\mathbf{p};t)& =\frac{1}{2-1}\left(\mathfrak{N}-1-\sum _{i=1}^t{\left(1-\frac{p_i}{P_t}\right)}^2\right)\hfill \\ & =\mathfrak{N}-1-\sum _{i=1}^t\left(1-2\frac{p_i}{P_t}+{\left(\frac{p_i}{P_t}\right)}^2\right).\hfill \\ & =\mathfrak{N}-t+1-\sum _{i=1}^t{\left(\frac{p_i}{P_t}\right)}^2\hfill \\ & =\mathfrak{N}-t+P{T}_2(\mathbf{p};t).\hfill \end{array}$$
• From (9), when $\alpha =2$, we have

  $$\begin{array}{cc}\hfill RTE{S}_2(\mathbf{p};t)& =\frac{1}{2-1}\left(\mathfrak{N}-1-\sum _{i=t}^{\mathfrak{N}}{\left(1-\frac{p_i}{{\overline{P}}_t}\right)}^2\right)\hfill \\ & =\mathfrak{N}-1-\sum _{i=t}^{\mathfrak{N}}\left(1-2\frac{p_i}{{\overline{P}}_t}+{\left(\frac{p_i}{{\overline{P}}_t}\right)}^2\right)\hfill \\ & =t-1+1-\sum _{i=t}^{\mathfrak{N}}{\left(\frac{p_i}{{\overline{P}}_t}\right)}^2\hfill \\ & =t-1+R{T}_2(\mathbf{p};t).\hfill \end{array}$$

□

In the following theorem, we will show the relation between the past and residual Tsallis extropy and past and residual Tsallis entropy according to the value of $\alpha$.

Theorem 1.

Assume that the discrete R.V. X has finite support $\{x_1,x_2,\dots ,x_{\mathfrak{N}}\}$ with the corresponding probability vector $\mathit{p}=\{p_1,p_2,\dots ,p_{\mathfrak{N}}\}$. Then, from the past and residual Tsallis extropy given in Equations (10) and (9), respectively, and the past and residual Tsallis entropy given in Equations (6) and (5), respectively, we have:

1. 

For $0<\alpha <1$, we obtain

$$P{T}_{\alpha }(\mathit{p};t)>PTE{S}_{\alpha }(\mathit{p};t),andR{T}_{\alpha }(\mathit{p};t)>RTE{S}_{\alpha }(\mathit{p};t).$$

2. 

For $\alpha >1$, we obtain

$$P{T}_{\alpha }(\mathit{p};t)<PTE{S}_{\alpha }(\mathit{p};t),andR{T}_{\alpha }(\mathit{p};t)<RTE{S}_{\alpha }(\mathit{p};t).$$

Proof. 

From Equations (9) and (5), we have

$$\begin{array}{cc}\hfill \frac{P{T}_{\alpha }(\mathbf{p};t)}{PTE{S}_{\alpha }(\mathbf{p};t)}& =\frac{1-{\sum }_{i=1}^t{\left(\frac{p_i}{P_t}\right)}^{\alpha }}{\mathfrak{N}-1-{\sum }_{i=1}^t{\left(1-\frac{p_i}{P_t}\right)}^{\alpha }}\hfill \\ & =\frac{P_t^{\alpha }-{\sum }_{i=1}^t{p}_i^{\alpha }}{(\mathfrak{N}-1)P_t^{\alpha }-{\sum }_{i=1}^t{(P_t-p_i)}^{\alpha }}\hfill \\ & =\frac{{\left({\sum }_{i=1}^t{p}_i\right)}^{\alpha }-{\sum }_{i=1}^t{p}_i^{\alpha }}{(\mathfrak{N}-1){\left({\sum }_{i=1}^t{p}_i\right)}^{\alpha }-{\sum }_{i=1}^t{(\left({\sum }_{i=1}^t{p}_i\right)-p_i)}^{\alpha }}.\hfill \end{array}$$

It can be simply noted that: if $0<\alpha <1$, then ${\left({\sum }_{i=1}^t{p}_i\right)}^{\alpha }<{\sum }_{i=1}^t{p}_i^{\alpha }$, and if $\alpha >1$, then ${\left({\sum }_{i=1}^t{p}_i\right)}^{\alpha }>{\sum }_{i=1}^t{p}_i^{\alpha }$. Moreover, we have $(\mathfrak{N}-1)P_t^{\alpha }>P_t^{\alpha }-{\sum }_{i=1}^t{p}_i^{\alpha }>-{\sum }_{i=1}^t{(P_t-p_i)}^{\alpha }$, $\forall \alpha \ne 1$. In the sequel, the results follow. □

Residual and Past Discrete Renyi Extropy

In the same manner, we can discuss the residual and past discrete Renyi extropy. Let the discrete R.V. X supported with $S=\{x_1,x_2,\dots ,x_{\mathfrak{N}}\}$, and the corresponding probability vector $\mathbf{p}=\{p_1,p_2,\dots ,p_{\mathfrak{N}}\}$. Then, the Renyi extropy was introduced by Liu and Xiao [10] as follows:

$$\begin{array}{cc}\hfill RE{S}_{\alpha }(\mathbf{p};t)& =\frac{1}{1-\alpha }\left(-(\mathfrak{N}-1)ln(\mathfrak{N}-1)+(\mathfrak{N}-1)ln\left(\sum _{i=1}^{\mathfrak{N}}{(1-p_i)}^{\alpha }\right)\right),1\ne \alpha >0.\hfill \end{array}$$

(11)

Motivated by the concept of the Renyi extropy and the discrete lifetime distribution, we introduce the residual and past of the Renyi extropy, respectively, as follows:

$$RRE{S}_{\alpha }(\mathbf{p};t)=\frac{1}{1-\alpha }\left(-(\mathfrak{N}-t)ln(\mathfrak{N}-t)+(\mathfrak{N}-t)ln\left(\sum _{i=t}^{\mathfrak{N}}{\left(1-\frac{p_i}{{\overline{P}}_t}\right)}^{\alpha }\right)\right),$$

(12)

$$PRE{S}_{\alpha }(\mathbf{p};t)=\frac{1}{1-\alpha }\left(-(t-1)ln(t-1)+(t-1)ln\left(\sum _{i=1}^t{\left(1-\frac{p_i}{P_t}\right)}^{\alpha }\right)\right).$$

(13)

In the following proposition, we will show the relation between the past and residual Renyi extropy and the past and residual extropy.

Proposition 4.

Assume that the discrete R.V. X has finite support with the corresponding probability vector $\mathit{p}=\{p_1,p_2,\dots ,p_{\mathfrak{N}}\}$. Then, from the past and residual Renyi extropy given in Equations (13) and (12), respectively, and the past and residual extropy given in Equations (8) and (7), respectively, we have:

1. 

From Equations (13) and (8), we have

$$\underset{\alpha \to 1}{lim}PRE{S}_{\alpha }(\mathit{p};t)=PES(\mathit{p};t).$$

2. 

From Equations (12) and (7), we have

$$\underset{\alpha \to 1}{lim}RRE{S}_{\alpha }(\mathit{p};t)=RES(\mathit{p};t).$$

Example 2.

Suppose that the discrete R.V. X has a uniform distribution over $\{1,\dots ,\mathfrak{N}\}$. Then, the residual and past Renyi extropy are given, respectively, by

$$\begin{array}{cc}\hfill RRE{S}_{\alpha }(\mathit{p};t)& =\frac{1}{\alpha -1}\left(-(\mathfrak{N}-t)\left(ln(\mathfrak{N}-t)-ln\left({\left(\frac{-1+\mathfrak{N}-t}{\mathfrak{N}-t}\right)}^{\alpha }(1+\mathfrak{N}-t)\right)\right)\right).\hfill \end{array}$$

$$\begin{array}{cc}\hfill PRE{S}_{\alpha }(\mathit{p};t)& =\frac{1}{\alpha -1}\left(-(-1+t)\left(ln(-1+t)-tln{\left(\frac{-1+t}{t}\right)}^{\alpha }\right)\right).\hfill \end{array}$$

3. Applications

In this section, we will use different data sources and the softmax function to obtain the corresponding probability vector $\mathbf{p}$, then discuss the behavior of the residual and past Tsallis extropy.

3.1. Softmax Function

It has been noted in the real world that the actual R.V. may be continuous in disposition, but discrete when we observe it, for example in a hospital, the number of days a patient stays or a patient survives after treatment. Accordingly, it is appropriate to fit these positions by discrete distributions developed from continuous distributions. One of the most well-known functions in engineering and science is the softmax function. It has usage in many fields such as game theory [11,12,13], reinforcement learning [14], and machine learning [15,16]. This normalized exponential function is used to transform a vector $\mathbf{y}={(y_1,y_2,\dots ,y_{\mathfrak{N}})}^t\in {\mathbb{R}}^{\mathfrak{N}}$ into a unit sum vector $\in {(0,1)}^{\mathfrak{N}}$ as follows

$$\begin{array}{cc}\hfill SMAX{\left(\mathbf{y}\right)}_i& =\frac{e^{y_i}}{{\sum }_{l=1}^{\mathfrak{N}}{e}^{y_l}}\hfill \\ & =e^{y-log\left({\sum }_{l=1}^{\mathfrak{N}}{e}^{y_l}\right)},i=1,2,\dots ,\mathfrak{N}.\hfill \end{array}$$

(14)

3.1.1. Standard Normal Distribution

In this part, we generate data from standard normal distribution ($N(0,1)$) and use the softmax function to obtain the corresponding probability vector with a unity sum. Figure 1 and Figure 2 show the simulated data of $N(0,1)$, besides the softmax data when $\mathfrak{N}$ = 30 and 100, respectively. For the softmax data when $\mathfrak{N}=30$, we see the mean = 0.0333 and the variance = 0.001195, and it shows the quantile–quantile plot of the softmax data, which can be noted to be far from normality. When $\mathfrak{N}=100$, we see that the mean = 0.01 and the variance = 0.0002753. Therefore, we can see that the variance is decreasing by increasing the sample. Furthermore, in the quantile–quantile plot of the softmax data, it starts to be closer to normality, but still, the extremely standardized residuals (on both ends) are more extensive than they would be for normal data. Moreover, Figure 3 and Figure 4 show the past and residual Tsallis extropy when $\alpha =0.2,0.8,3,15$, which can be noted as follows: By increasing t, the past Tsallis extropy decreases (increases) for $\alpha >1$$(0<\alpha <1)$. Furthermore, the residual Tsallis extropy increases (decreases) for $\alpha >1$$(0<\alpha <1)$.

Figure 1. Simulated data of $N(0,1)$ and the softmax data, $\mathfrak{N}=30$.

Figure 2. Simulated data of $N(0,1)$ and the softmax data, $\mathfrak{N}=100$.

Figure 3. Past and residual Tsallis extropy, $\mathfrak{N}=30$, $\alpha =0.2,0.8,3,15$.

Figure 4. Past and residual Tsallis extropy, $\mathfrak{N}=100$, $\alpha =0.2,0.8,3,15$.

3.1.2. Real Data: U.S. Consumption and Personal Income Quarterly Changes

This part presents the quarterly changes in U.S. consumption and personal income data from 1970 to 2016. First, we want to discuss the softmax data obtained from our data. Figure 5 displays the U.S. consumption and income data and the corresponding softmax data. Figure 6 shows the data analysis of the softmax U.S. consumption and personal income, such that the mean of softmax U.S. consumption is 0.00534759 and the variance is 0.000011557 and the mean softmax of U.S. personal income is 0.00534759 and the variance is 0.0000149023. Moreover, the extremely standardized residuals (on both ends) are larger than they would be for normal data.

Figure 5. Quarterly changes in U.S. consumption and personal income data from 1970 to 2016 and the corresponding softmax data.

Figure 6. Data analysis of softmax U.S. consumption (left panel) and softmax U.S. personal income (right panel).

In Figure 7, based on the softmax U.S. consumption data, we see the residuals of the data before fitting them to a preferred model. Therefore, there are significant spikes in the ACF according to its residuals. Then, our aim now is to find a suitable model based on the ACF. Correspondingly, Figure 7 shows the residuals of the model fit to the data. Thus, the best model that fits the data is $ARIMA(0,1,1){(2,0,2)}_4$ with p-value = 0.1502, AIC = 350.51, and it accepts the Ljung–Box test. Accordingly, all the spikes are within the significant ACF limits, except for a single spike that exceeds a small limit. Note that the general form of a seasonal ARIMA model is $ARIMA(p,b,q){(p,b,q)}_s$, where ${(p,b,q)}_s$ and $(p,b,q)$ are the seasonal and non-seasonal part of of the model, respectively, and s is the number of observations per year.

Figure 7. Residuals of softmax U.S. consumption and its fit model $ARIMA(0,1,1){(2,0,2)}_4$.

Moreover, Figure 8 shows the past and residual Tsallis extropy of the softmax U.S. consumption data, which can be noted as follows: By increasing t, the past Tsallis extropy decreases (increases) for $\alpha >1$ $(0<\alpha <1)$. Furthermore, the residual Tsallis extropy increases (decreases) for $\alpha >1$ $(0<\alpha <1)$.

Figure 8. Past and residual Tsallis extropy of softmax U.S. consumption.

4. Conclusions

In this article, we offered the residual and past discrete Tsallis and Renyi extropy. Moreover, we discussed some properties and relations of the residual and past discrete Tsallis entropy with respect to our models. Moreover, we chose the softmax function as a discrete probability distribution function and studied its behavior for simulated and real data. We discussed the residuals of the softmax data for real data and suggested that the appropriate ARIMA model fits the data. Furthermore, we obtained the past and residual Tsallis extropy and studied its increasing and decreasing according to the value of $\alpha$. Finally, we can apply those models to ordered variables and their concomitants for future work. For more details, see [17,18,19]. For applications, see [20,21,22].