Measuring Dispersion and Serial Dependence in Ordinal Time Series Based on the Cumulative Paired ϕ-Entropy

Abstract

The family of cumulative paired $\varphi$-entropies offers a wide variety of ordinal dispersion measures, covering many well-known dispersion measures as a special case. After a comprehensive analysis of this family of entropies, we consider the corresponding sample versions and derive their asymptotic distributions for stationary ordinal time series data. Based on an investigation of their asymptotic bias, we propose a family of signed serial dependence measures, which can be understood as weighted types of Cohen’s $\kappa$, with the weights being related to the actual choice of $\varphi$. Again, the asymptotic distribution of the corresponding sample ${\kappa }_{\varphi }$ is derived and applied to test for serial dependence in ordinal time series. Using numerical computations and simulations, the practical relevance of the dispersion and dependence measures is investigated. We conclude with an environmental data example, where the novel $\varphi$-entropy-related measures are applied to an ordinal time series on the daily level of air quality.

1. Introduction

During the last years, ordinal data in general [1] and ordinal time series in particular [2] received a great amount of interest in research and applications. Here, a random variable X is said to be ordinal if X has a bounded qualitative range exhibiting a natural order among the categories. We denote the range as $\mathcal{S}=\{s_0,s_1,\dots ,s_m\}$ with some $m\in \mathbb{N}=\{1,2,\dots \}$, and we assume that $s_0<\dots <s_m$. The realized data are denoted as $x_1,\dots ,x_n$ with $n\in \mathbb{N}$. They are assumed to stem either from independent and identically distributed (i. i. d.) replications of X (then, we refer to $x_1,\dots ,x_n$ as an ordinal random sample), or from a stationary ordinal stochastic process ${\left(X_t\right)}_{\mathbb{Z}=\{\dots ,-1,0,1,\dots \}}$ (then, $x_1,\dots ,x_n$ are said to be an ordinal time series).

In what follows, we take up several recent works on measures of dispersion and serial dependence in ordinal (time series) data. Regarding ordinal dispersion, the well-known measures such as variance or mean absolute deviation cannot be used as the data are not quantitative. Therefore, several tailor-made measures for ordinal dispersion have been developed and investigated in the literature, see, among others, [3,4,5,6,7,8,9,10,11,12,13,14,15]. The unique feature of all these measures is that they rely on the cumulative distribution function (CDF) of X, i.e., on $\mathit{f}={(f_0,\dots ,f_{m-1})}^{\top }$ with $f_i=P(X\le s_i)$ for $i=0,\dots ,m$ ($f_m$ is omitted in f as it necessarily equals one). They classify any one-point distribution on $\mathcal{S}$ as a scenario of minimal dispersion, i.e., if all probability mass concentrates on one category from $\mathcal{S}$ (maximal consensus):

$${\mathit{f}}_{one}\in \left\{\left(\begin{array}{c}1\\ 1\\ ⋮\\ 1\end{array}\right),\left(\begin{array}{c}0\\ 1\\ ⋮\\ 1\end{array}\right),\dots ,\left(\begin{array}{c}0\\ ⋮\\ 0\\ 1\end{array}\right),\left(\begin{array}{c}0\\ ⋮\\ 0\\ 0\end{array}\right)\right\}.$$

By contrast, maximal dispersion is achieved exactly in the case of the extreme two-point distribution (polarized distribution), ${\mathit{f}}_{two}={(\frac{1}{2},\dots ,\frac{1}{2})}^{\top }$, where we have 50 % probability mass in each of the outermost categories (maximal dissent). Further details on ordinal dispersion measures are presented in Section 2 below.

Building upon earlier works by Klein [16], Yager [17], it was recently shown by Klein Doll [18], Klein et al. [19] that the aforementioned ordinal dispersion measures can be subsumed under the family of so-called “cumulative paired $\varphi$-entropies” (see Section 2), abbreviated as ${\mathrm{CPE}}_{\varphi }$, which constitutes the starting point of the present article. Our first main task is to derive the asymptotic distribution of the corresponding sample version, ${\widehat{\mathrm{CPE}}}_{\varphi }$, for both i. i. d. and time series data, and to check the finite sample performance of the resulting approximate distribution, see Section 3 and Section 5.

In the recent paper by Weiß [20] on the asymptotics of some well-known dispersion measures for nominal data (i.e., qualitative data without a natural ordering), it turned out that the corresponding dispersion measures—if these are applied to time series data—are related to specific measures of serial dependence. Therefore, our second main task is to explore for a similar relation in the ordinal case, if considering the ${\mathrm{CPE}}_{\varphi }$-family for measuring dispersion. Ordinal dependence measures can be defined in analogy to the popular autocorrelation function (ACF) for quantitative time series, namely by using the lagged bivariate CDF $f_{ij}\left(h\right)=P(X_t\le s_i,X_{t-h}\le s_j)$ for time lags $h\in \mathbb{N}$ as their base [14]. For the novel family of ${\kappa }_{\varphi }\left(h\right)$ measures, which cover the existing ordinal Cohen’s $\kappa$ [14,15] as a special case, we derive the asymptotics under the null hypothesis of i. i. d. time series data, see Section 4. This result is used in Section 5 to test for significant serial dependence, in analogy to the application of the sample ACF to quantitative time series. In Section 6, we discuss an illustrative real-world data example from an environmental application, namely regarding the daily level of air quality. The article concludes in Section 7.

2. The Family of Cumulative Paired $\varphi$-Entropies

Klein Doll [18], Klein et al. [19] proposed and investigated a family of cumulative paired $\varphi$-entropies. Although their main focus was on continuously distributed random variables, they also referred to the ordinal case and pointed out that many well-known ordinal dispersion measures are included in this family. Here, we exclusively concentrate on the ordinal case as introduced in Section 1, and we define the (normalized) cumulative paired $\varphi$-entropy as (see Section 2.3 in Klein et al. [19])

$${\mathrm{CPE}}_{\varphi }\left(f\right)=\frac{1}{2m\varphi (1/2)}\sum _{i=0}^{m-1}\left(\varphi \left(f_i\right)+\varphi (1-f_i)\right).$$

(1)

The entropy generating function (EGF) $\varphi$ is defined on $[0;1]$, it satisfies $\varphi \left(0\right)=\varphi \left(1\right)=0$, and it is assumed to be concave on $[0;1]$. Later in Section 3, when deriving the asymptotic distribution of the sample counterpart ${\widehat{\mathrm{CPE}}}_{\varphi }={\mathrm{CPE}}_{\varphi }\left(\widehat{\mathit{f}}\right)$, we shall also require that $\varphi$ is (twice) differentiable. As pointed out in Section 2.3 and 3.1 of Klein et al. [19], several well-known measures of ordinal dispersion can be expressed by (1) with an appropriate choice of $\varphi$.

• Leik’s ordinal variation [11] corresponds to the choice $\varphi \left(z\right)=min\{z,1-z\}$ (which is not differentiable in $z=1/2$) because of the equality $|2z-1|=1-2min\{z,1-z\}$:

  $$\mathrm{LOV}=1-\frac{1}{m}{\displaystyle \sum _{i=0}^{m-1}}|2f_i-1|=\frac{2}{m}{\displaystyle \sum _{i=0}^{m-1}}min\{f_i,1-f_i\}.$$

  (2)
• An analogous argument applies to the whole family of ordinal variation measures, ${\mathrm{OV}}_q$ with $q\ge 1$ [3,9,10,13]. Choosing ${\varphi }_q\left(z\right)=1-{\left(1-2min\{z,1-z\}\right)}^q=1-{|2z-1|}^q$ with ${\varphi }_q(1/2)=1$, we have the relation

  $${\mathrm{OV}}_q=1-{\left(\frac{1}{m}{\displaystyle \sum _{i=0}^{m-1}}{|2f_i-1|}^q\right)}^{1/q}=1-{{\displaystyle (}1-{\mathrm{CPE}}_{\varphi }\left(f\right){\displaystyle )}}^{1/q}.$$

  (3)
  Note that $q=1$ leads to the LOV, and the case $q=2$ is known as the coefficient of ordinal variation, $\mathrm{COV}={\mathrm{OV}}_2$ [4,8].
• Related to the previous case ${\mathrm{OV}}_q$ with $q=2$, ${\mathrm{CPE}}_{\varphi }\left(\mathit{f}\right)$ becomes the widely-used index of ordinal variation [3,7,8] if choosing $\varphi \left(z\right)=z(1-z)$:

  $$\mathrm{IOV}=\frac{4}{m}{\displaystyle \sum _{i=0}^{m-1}}{f}_i(1-f_i)=1-\frac{1}{m}{\displaystyle \sum _{i=0}^{m-1}}{(2f_i-1)}^2.$$

  (4)
• The cumulative paired (Shannon) entropy [12] corresponds to the choice $\varphi \left(z\right)=-zlnz$ (with the convention $0ln0=0$):

  $$\mathrm{CPE}=\frac{-1}{mln2}{\displaystyle \sum _{i=0}^{m-1}}{\displaystyle (}{f}_{i}ln{f}_i+(1-f_i)ln(1-f_i){\displaystyle )}.$$

  (5)
• $\varphi \left(z\right)=-zlnz$ can be embedded into the family of a-entropies [21,22],

  $${\varphi }_a\left(z\right)=\frac{z-z^a}{a-1}\mathrm{with}a>0\mathrm{and}a\ne 1,$$

  (6)

  as the boundary case $a\to 1$. Plugging-in (6) into (1), one obtains

  $${\mathrm{CPE}}_a=\frac{2^{a-1}}{m(2^{a-1}-1)}{\displaystyle \sum _{i=0}^{m-1}}{\displaystyle (}1-f_i^a-{(1-f_i)}^a{\displaystyle )}.$$

  (7)
  Note that both $a=2$ and $a=3$ in (7) lead to the IOV (4).

The EGFs $\varphi$ involved in (2)–(4) are symmetric around $1/2$, i.e., they satisfy $\varphi \left(z\right)=\varphi (1-z)$. This is also illustrated by Figure 1, where the cases $q=2$ (left) and $a=2$ (right; both in bold black) agree with each other except a scaling factor. The plotted EGFs ${\varphi }_q\left(z\right)$ are maximal in $1/2$ with ${\varphi }_q(1/2)=1$. The EGF ${\varphi }_a\left(z\right)$ is maximal in $a^{1/(1-a)}$ with ${\varphi }_a\left(a^{1/(1-a)}\right)=a^{a/(1-a)}$ for $a\ne 1$, whereas in the boundary case $a\to 1$, ${\varphi }_a\left(z\right)$ takes its maximal value $1/e$ at $1/e$. However, since ${\mathrm{CPE}}_{\varphi }$ in (1) is normalized, it is not necessary to care for a possible rescaling of ${\varphi }_a\left(z\right)$ if computing ${\mathrm{CPE}}_{\varphi }$.

Figure 1. Plot of EGFs $\varphi \left(z\right)$ against z. (Left): ${\varphi }_q\left(z\right)=1-{|2z-1|}^q$; (right): ${\varphi }_a\left(z\right)=(z-z^a)/(a-1)$.

Remark 1.

The dotted curve in the right part of Figure 1, which connects the maxima of ${\varphi }_a\left(z\right)$ for different a, is computed by using the Lambert W function (also referred to as the product logarithm). This can be seen as follows: ${\varphi }_a\left(z\right)$ is maximal in $z_0=a^{1/(1-a)}$ with ${\varphi }_a\left(z_0\right)=z_0^a$. It holds that

$$a·z_0^a=a^{1+a/(1-a)}=z_0.$$

Using that $z_0^a=exp(aln{z}_0)$, this implies that

$$(aln{z}_0)·exp(aln{z}_0)=z_{0}ln{z}_0.$$

The Lambert W function is defined to solve the equation $wexpw=z$ as $w=W\left(z\right)$, so we get

$$aln{z}_0=W(z_{0}ln{z}_0)\iff a=W(z_{0}ln{z}_0)/ln{z}_0.$$

Thus, since ${\varphi }_a\left(z_0\right)=a^{a/(1-a)}=a^{1/(1-a)}/a=z_0/a$, the dotted curve in Figure 1 follows the function $zlnz/W(zlnz)$. More precisely, since $zlnz$ is minimal in $z=1/e$ with minimal value $-1/e$, we have to use the principal branch $W\left(z\right)=W_0\left(z\right)$ for $z\le 1/e$, and the secondary branch $W\left(z\right)=W_{-1}\left(z\right)$ for $z>1/e$.

Let us conclude this section with some examples of ${\mathrm{CPE}}_{\varphi }$ measures (see Figure 2). For all examples, we set $m=4$, i.e., we have five ordinal categories like, for example, in the case of a simple Likert scale. In the left part of Figure 2, f was computed according to the binomial distribution $\mathrm{Bin}(4,p)$, which has maximal dispersion for $p=0.5$. This is also recognized by any of the plotted measures ${\mathrm{CPE}}_{\varphi }$, with their maximal dispersion values varying around 0.6. This medium level of dispersion is plausible because $\mathrm{Bin}(4,0.5)$ is far away from the extreme two-point distribution. The right part of Figure 2, by contrast, shows the ${\mathrm{CPE}}_{\varphi }$ for the two-point distribution with $f_0=p$ ($=f_1=\cdots =f_{m-1}$). So $p=0$ corresponds to a one-point distribution in $s_m$ (minimal dispersion), and $p=0.5$ to the extreme two-point distribution (maximal dispersion). Accordingly, all measures reach their extreme values 0 and 1, respectively. It is interesting to note that outside these extreme cases, the dispersion measures judge the actual dispersion level quite differently; see the related discussion in Kvålseth [10], Weiß [13].

Figure 2. Plot of ${\mathrm{CPE}}_{\varphi }$ against p for specific cases of ${\varphi }_a\left(z\right)=(z-z^a)/(a-1)$ and ${\varphi }_q\left(z\right)=1-{|2z-1|}^q$. (Left): binomial distribution $\mathrm{Bin}(4,p)$; (Right): two-point distribution with $f_0=p$.

3. Asymptotic Distribution of Sample ${\mathrm{CPE}}_{\varphi }$

From now on, we focus on the sample version of ${\mathrm{CPE}}_{\varphi }$ from (1), i.e., on ${\widehat{\mathrm{CPE}}}_{\varphi }={\mathrm{CPE}}_{\varphi }\left(\widehat{\mathit{f}}\right)$, where $\widehat{\mathit{f}}$ denotes the vector of cumulative relative frequencies computed from $X_1,\dots ,X_n$. To derive the asymptotic distribution of ${\widehat{\mathrm{CPE}}}_{\varphi }$, which is to be used as an approximation to the true distribution of ${\widehat{\mathrm{CPE}}}_{\varphi }$, we recall the following results from Weiß [14]. Provided that the data-generating process (DGP) satisfies appropriate mixing conditions, e.g., $\alpha$-mixing with exponentially decreasing weights (which includes the i. i. d.-case), it holds that

$$\begin{array}{c}\sqrt{n}\left(\widehat{f}-f\right)\stackrel{\mathrm{d}}{\to }N(0,\Sigma )\mathrm{with}\Sigma ={\left({\sigma }_{ij}\right)}_{i,j=0,\dots ,m-1},\mathrm{where}\hfill \\ {\sigma }_{ij}\stackrel{a}{=}{f}_{min\{i,j\}}-f_i{f}_j+{\displaystyle \sum _{h=1}^{\infty }}{\displaystyle (}{f}_{ij}\left(h\right)+f_{ji}\left(h\right)-2f_i{f}_j{\displaystyle )}.\hfill \end{array}$$

(8)

For an analogous result in the presence of missing data, see Theorem 1 in Weiß [15]. In (8), finite (co)variances are ensured if we require that ${\sum }_{h=1}^{\infty }\left(f_{ij}\left(h\right)-f_i{f}_j\right)<\infty$ holds for all $i,j$ (“short memory”). In particular, all sums ${\sum }_{h=1}^{\infty }\left(f_{ij}\left(h\right)+f_{ji}\left(h\right)-2f_i{f}_j\right)$ vanish in the i. i. d.-case. Otherwise, they account for the serial dependence of the DGP. This can be seen by considering the trace of $\Sigma$, which equals

$${\displaystyle \sum _{i=0}^{m-1}}\left(f_i(1-f_i)+2{\displaystyle \sum _{h=1}^{\infty }}\left(f_{ii}\left(h\right)-f_i^2\right)\right)={\displaystyle \sum _{i=0}^{m-1}}{f}_i(1-f_i)\left(1+2{\displaystyle \sum _{h=1}^{\infty }}{\kappa }_{\mathrm{ord}}\left(h\right)\right).$$

Here, the term ${\sum }_{i=0}^{m-1}{f}_i(1-f_i)$ agrees with the IOV in (4) except the normalizing factor $\frac{4}{m}$, i.e., it corresponds to ${\mathrm{CPE}}_{\varphi }$ with $\varphi \left(z\right)=z(1-z)$. The term ${\kappa }_{\mathrm{ord}}\left(h\right)$, in turn, is the ordinal Cohen’s $\kappa$ [14] defined by

$${\kappa }_{\mathrm{ord}}\left(h\right)=\frac{{\sum }_{i=0}^{m-1}\left(f_{ii}\left(h\right)-f_i^2\right)}{{\sum }_{i=0}^{m-1}{f}_i(1-f_i)}.$$

(9)

It is a measure of signed serial dependence, which evaluates the extent of (dis)agreement between $X_t$ and $X_{t-h}$ by positive (negative) values.

Based on Taylor expansions of ${\widehat{\mathrm{CPE}}}_{\varphi }={\mathrm{CPE}}_{\varphi }\left(\widehat{\mathit{f}}\right)$ in f, we shall now study its asymptotic behavior. To establish asymptotic normality and to derive the asymptotic variance of ${\widehat{\mathrm{CPE}}}_{\varphi }$, we need $\varphi$ to be differentiable. For an asymptotic bias correction, which relies on a second-order Taylor expansion, $\varphi$ even has to be twice differentiable (then, the concavity of $\varphi$ implies that ${\varphi }^{″}\left(z\right)<0$).

Remark 2.

From the examples discussed in Section 2, the EGF corresponding to the LOV (i.e., ${\varphi }_q$ with $q=1$) is not differentiable (in $z=1/2$). ${\varphi }_q$ is only once differentiable for $1<q<2$, while $q\ge 2$ ensures ${\varphi }_q$ to be at least twice differentiable; see Example 1 below. Accordingly, in these cases, it is not possible to establish asymptotic normality ($q=1$) or an asymptotic bias correction ($1<q<2$), respectively. In fact, Weiß [13] was faced with the same problem when studying the asymptotics of the sample ${\mathrm{OV}}_q$, and a solution to it was not possible. In simulations, he showed that even modified asymptotics (using a folded-normal distribution) did not lead to an acceptable approximation quality. We shall therefore exclude such cases from our further discussion. If, in applications, the cases $q=1$ or $1<q<2$ appear to be relevant, a bootstrap approach might be an option to gain insight into the sample distribution of ${\widehat{\mathrm{CPE}}}_{\varphi }$.

Assuming $\varphi$ to be (twice) differentiable, the partial derivatives of ${\mathrm{CPE}}_{\varphi }\left(\mathit{f}\right)$ according to (1) are

$$\begin{array}{cc}\hfill \frac{\partial }{\partial f_k}CP{E}_{\varphi }\left(\mathit{f}\right)=& \frac{1}{2m\varphi (1/2)}\left({\varphi }^{\prime }\left(f_k\right)-{\varphi }^{\prime }(1-f_k)\right)=:d_k,\hfill \\ \hfill \frac{{\partial }^2}{{\partial }^2{f}_k}CP{E}_{\varphi }\left(\mathit{f}\right)=& \frac{1}{2m\varphi (1/2)}\left({\varphi }^{″}\left(f_k\right)+{\varphi }^{″}(1-f_k)\right)=:h_{kk},\hfill \\ \hfill \frac{{\partial }^2}{\partial f_k\partial f_l}CP{E}_{\varphi }\left(\mathit{f}\right)=& 0\mathrm{for}k\ne l.\hfill \end{array}$$

(10)

We denote the gradient (Jacobian) of ${\mathrm{CPE}}_{\varphi }\left(\mathit{f}\right)$ by $\mathbf{D}=(d_0,\dots ,d_{m-1})$, and the Hessian equals $\mathbf{H}=diag(h_{00},\dots ,h_{m-1,m-1})$. Note that if $\varphi$ is symmetric around $1/2$, i.e., if $\varphi \left(z\right)=\varphi (1-z)$, then $d_k={\varphi }^{\prime }\left(f_k\right)/\left(m\varphi (1/2)\right)$ and $h_{kk}={\varphi }^{″}\left(f_k\right)/\left(m\varphi (1/2)\right)$.

Example 1.

Let us compute the derivatives required in (10) for the EGF examples presented in Section 2.

For ${\varphi }_a\left(z\right)=\frac{z-z^a}{a-1}$, the constant factor becomes $\frac{1}{2m{\varphi }_a(1/2)}=\frac{2^{a-1}(a-1)}{m(2^{a-1}-1)}$, and the derivatives are ${\varphi }_a^{\prime }\left(z\right)=\frac{1-a{z}^{a-1}}{a-1}$ and ${\varphi }_a^{″}\left(z\right)=-a{z}^{a-2}$. Here, ${\varphi }_a^{\prime }\left(z\right)$ exists in the boundary value $z=0$ only if $a>1$, and ${\varphi }_a^{″}\left(z\right)$ if $a\ge 2$. Important special cases are

$$\varphi \left(z\right)=-zlnz\Rightarrow \frac{1}{2m\varphi (1/2)}=\frac{1}{mln2},{\varphi }^{\prime }\left(z\right)=-1-lnz,{\varphi }^{″}\left(z\right)=-1/z\mathrm{for}a\to 1,$$

$$\varphi \left(z\right)=z(1-z)\Rightarrow \frac{1}{2m\varphi (1/2)}=\frac{2}{m},{\varphi }^{\prime }\left(z\right)=1-2z,{\varphi }^{″}\left(z\right)=-2\mathrm{for}a=2,$$

and for $a=3$,

$$\varphi \left(z\right)=\frac{1}{2}z(1-z^2)\Rightarrow \frac{1}{2m\varphi (1/2)}=\frac{8}{3m},{\varphi }^{\prime }\left(z\right)=\frac{1}{2}(1-3z^2),{\varphi }^{″}\left(z\right)=-3z.$$

Note that both $a=2,3$ lead to the same expressions for $d_k,h_{kk}$ in (10); see Table 1. This is in accordance with the equivalence of these cases as discussed after (7).

For ${\varphi }_q\left(z\right)=1-{\left(1-2min\{z,1-z\}\right)}^q=1-{|2z-1|}^q$ with $\frac{1}{2m{\varphi }_q(1/2)}=\frac{1}{2m}$, the derivatives are expressed using the sign function, $\mathrm{sgn}(·)$, which is not continuous at 0. Note that the following relations hold:

$$\frac{d}{dx}\left|x\right|=\mathrm{sgn}\left(x\right)forx\ne 0,x=\mathrm{sgn}\left(x\right)·\left|x\right|,\left|x\right|=\mathrm{sgn}\left(x\right)·x.$$

For $q\ge 2$, it then follows by applying the chain rule and the product rule that

$$\begin{array}{cc}\hfill {\varphi }_q^{\prime }\left(z\right)=& {-2q|2z-1|}^{q-1}\mathrm{sgn}(2z-1)=-2q(2z-1){|2z-1|}^{q-2},\hfill \\ \hfill {\varphi }_q^{″}\left(z\right)=& {-4q|2z-1|}^{q-2}-4q(q-2)(2z-1){|2z-1|}^{q-3}\mathrm{sgn}(2z-1)\hfill \\ \hfill =& {-4q(q-1)|2z-1|}^{q-2}.\hfill \end{array}$$

Note that these derivatives are continuous in $z=1/2$ for $q\ge 2$, using that $0^0=1$. The final expressions for (10) are summarized in Table 1.

Table 1. Partial derivatives (10) for EGFs discussed in Example 1.

EGF $\mathit{\varphi }\left(\mathit{z}\right)$	${\mathit{d}}_{\mathit{k}}$	${\mathit{h}}_{\mathbf{kk}}$
$(z-z^a)/(a-1)$	$\frac{2^{a-1}a}{m(2^{a-1}-1)}\left({(1-f_k)}^{a-1}-f_k^{a-1}\right)$	$-\frac{2^{a-1}a(a-1)}{m(2^{a-1}-1)}\left(f_k^{a-2}+{(1-f_k)}^{a-2}\right)$
$-zlnz$	$\frac{1}{mln2}{\displaystyle (}ln(1-f_k)-ln{f}_k{\displaystyle )}$	$\frac{-1}{mln2}{\displaystyle (}1/f_k+1/(1-f_k){\displaystyle )}$
$z(1-z),\frac{1}{2}z(1-z^2)$	$\frac{4}{m}(1-2f_k)$	$-\frac{8}{m}$
${1-|2z-1|}^q$	$-\frac{2q}{m}(2f_k-1){|2f_k-1|}^{q-2}$	$-\frac{4q(q-1)}{m}{|2f_k-1|}^{q-2}$

Table 1. Partial derivatives (10) for EGFs discussed in Example 1.

EGF $\mathit{\varphi }\left(\mathit{z}\right)$	${\mathit{d}}_{\mathit{k}}$	${\mathit{h}}_{\mathbf{kk}}$
$(z-z^a)/(a-1)$	$\frac{2^{a-1}a}{m(2^{a-1}-1)}\left({(1-f_k)}^{a-1}-f_k^{a-1}\right)$	$-\frac{2^{a-1}a(a-1)}{m(2^{a-1}-1)}\left(f_k^{a-2}+{(1-f_k)}^{a-2}\right)$
$-zlnz$	$\frac{1}{mln2}{\displaystyle (}ln(1-f_k)-ln{f}_k{\displaystyle )}$	$\frac{-1}{mln2}{\displaystyle (}1/f_k+1/(1-f_k){\displaystyle )}$
$z(1-z),\frac{1}{2}z(1-z^2)$	$\frac{4}{m}(1-2f_k)$	$-\frac{8}{m}$
${1-|2z-1|}^q$	$-\frac{2q}{m}(2f_k-1){|2f_k-1|}^{q-2}$	$-\frac{4q(q-1)}{m}{|2f_k-1|}^{q-2}$

Using (10), the second-order Taylor expansion equals

$${\mathrm{CPE}}_{\varphi }\left(\widehat{\mathit{f}}\right)\approx {\mathrm{CPE}}_{\varphi }\left(\mathit{f}\right)+{\sum }_{k=0}^{m-1}{d}_k\left({\widehat{f}}_k-f_k\right)+\frac{1}{2}{\sum }_{k=0}^{m-1}{h}_{kk}{\left({\widehat{f}}_k-f_k\right)}^2.$$

(11)

According to (8), the linear term in (11) is asymptotically normally distributed (“Delta method”), provided that $\mathbf{D}$ does not vanish (see Remark 3 below). Then, we conclude from (8) that

$$\sqrt{n}\left({\mathrm{CPE}}_{\varphi }\left(\widehat{\mathit{f}}\right)-{\mathrm{CPE}}_{\varphi }\left(\mathit{f}\right)\right)\stackrel{\mathrm{d}}{\to }N\left(0,\mathbf{D}\Sigma \mathbf{D}{}^{\top }\right),\mathbf{D}\Sigma \mathbf{D}{}^{\top }={\displaystyle \sum _{i,j=0}^{m-1}}{d}_i{d}_j{\sigma }_{ij}.$$

(12)

The approximate bias correction relies on the quadratic term in (11), because $E[{\widehat{f}}_k-f_k]=0$. Using (8), we conclude that

$$nE\left[{\mathrm{CPE}}_{\varphi }(\widehat{\mathit{f}})-{\mathrm{CPE}}_{\varphi }\left(\mathit{f}\right)\right]\approx \frac{1}{2}{\sum }_{i=0}^{m-1}{h}_{ii}{\sigma }_{ii}.$$

(13)

Let us summarize the results implied by (12) and (13) in the following theorem.

Theorem 1.

Under the mixing assumptions stated before (8), assuming the EGF ϕ to be twice differentiable, and recalling the $d_k,h_{kk}$ from (10) where $\mathbf{D}$ must not vanish, it holds that

$$\begin{array}{c}\sqrt{n}\left({\mathrm{CPE}}_{\varphi }(\widehat{\mathit{f}}\right)-{\mathrm{CPE}}_{\varphi }\left(\mathit{f}\right))\stackrel{\mathrm{d}}{\to }N\left(0,{\sigma }_{\varphi }^2\right),{\sigma }_{\varphi }^2={\sigma }_{\varphi ,iid}^2\left(1+2{\displaystyle \sum _{h=1}^{\infty }}{\vartheta }_{\varphi }\left(h\right)\right)\hfill \\ with{\sigma }_{\varphi ,iid}^2={\displaystyle {\sum }_{i,j=0}^{m-1}}{d}_i{d}_j\left(f_{min\{i,j\}}-f_i{f}_j\right){\displaystyle and{\vartheta }_{\varphi }\left(h\right)=\frac{{\sum }_{i,j=0}^{m-1}{d}_i{d}_j\left(f_{ij}\left(h\right)-f_i{f}_j\right)}{{\sum }_{i,j=0}^{m-1}{d}_i{d}_j\left(f_{min\{i,j\}}-f_i{f}_j\right)}}.\hfill \end{array}$$

In addition, the bias-corrected mean of ${\mathrm{CPE}}_{\varphi }(\widehat{\mathit{f}})$ is

$$\begin{array}{c}E\left[{\mathrm{CPE}}_{\varphi }\left(\widehat{\mathit{f}}\right)\right]\approx {\mathrm{CPE}}_{\varphi }\left(\mathit{f}\right)+\frac{1}{2n}\left({\displaystyle \sum _{i=0}^{m-1}}{h}_{ii}{f}_i(1-f_i)\right)\left(1+2{\displaystyle \sum _{h=1}^{\infty }}{\kappa }_{\varphi }\left(h\right)\right),\hfill \\ {\displaystyle where{\kappa }_{\varphi }\left(h\right)=\frac{{\sum }_{i=0}^{m-1}{h}_{ii}\left(f_{ii}\left(h\right)-f_i^2\right)}{{\sum }_{i=0}^{m-1}{h}_{ii}{f}_i(1-f_i)}.}\hfill \end{array}$$

Note that the second-order derivatives are negative due to the concavity of $\varphi$, so ${\mathrm{CPE}}_{\varphi }(\widehat{\mathit{f}})$ exhibits a negative bias. ${\vartheta }_{\varphi }\left(h\right)$ expresses the effect of serial dependence on ${\sigma }_{\varphi }^2$. For i. i. d. ordinal data, ${\vartheta }_{\varphi }\left(h\right)=0$, so Theorem 1 simplifies considerably in this case, namely to ${\sigma }_{\varphi }^2={\sigma }_{\varphi ,iid}^2$. The bias of ${\mathrm{CPE}}_{\varphi }(\widehat{\mathit{f}})$ is affected by serial dependence via ${\kappa }_{\varphi }\left(h\right)$, which is a $\kappa$-type measure reflecting the extent of (dis)agreement between lagged observations, recall (9). More precisely, ${\kappa }_{\varphi }\left(h\right)$ can be interpreted as weighted type of ${\kappa }_{\mathrm{ord}}\left(h\right)$, where the weights $h_{ii}$ depend on the particular choice of $\varphi$. It thus provides a novel family of measures of signed serial dependence, the asymptotics of which are analyzed in Section 4 below.

Example 2.

In the special case $\varphi \left(z\right)=z(1-z)$ (as well as for $\varphi \left(z\right)=\frac{1}{2}z(1-z^2)$), which corresponds to the IOV in (4), $h_{ii}=-\frac{8}{m}$ is constant (see Table 1). Thus, ${\kappa }_{\varphi }\left(h\right)={\kappa }_{\mathrm{ord}}\left(h\right)$ in this case (see (9)). Furthermore, the first factor of the bias in Theorem 1 becomes

$$\frac{1}{2n}{\sum }_{i=0}^{m-1}{h}_{ii}{f}_i(1-f_i)=-\frac{1}{n}·\frac{4}{m}{\sum }_{i=0}^{m-1}{f}_i(1-f_i)=-\frac{1}{n}\mathrm{IOV}.$$

Hence, the bias is determined by both the serial dependence and the dispersion of the process.

As another simple example, consider the choice $\varphi \left(z\right)=-zlnz$ for the $\mathrm{CPE}$ in (5). Then, using $h_{ii}=\frac{-1}{mln2}(\frac{1}{f_i}+\frac{1}{1-f_i})=\frac{-1}{mln2}\frac{1}{f_i(1-f_i)}$ from Table 1, we get

$$\frac{1}{2n}{\sum }_{i=0}^{m-1}{h}_{ii}{f}_i(1-f_i)=\frac{1}{2n}\frac{-1}{mln2}{\sum }_{i=0}^{m-1}1=\frac{-1}{(2ln2)n}.$$

Thus, we have a unique i. i. d.-bias, independent of the actual marginal CDF f. Under serial dependence, we get

$${\kappa }_{\varphi }\left(h\right)=\frac{{\sum }_{i=0}^{m-1}{h}_{ii}\left(f_{ii}\left(h\right)-f_i^2\right)}{{\sum }_{i=0}^{m-1}{h}_{ii}{f}_i(1-f_i)}=\frac{1}{m}{\displaystyle \sum _{i=0}^{m-1}}\frac{f_{ii}\left(h\right)-f_i^2}{f_i(1-f_i)}=:{\kappa }_{ord}^{*}\left(h\right).$$

(14)

So, besides the pair $\left(\mathrm{IOV},{\kappa }_{\mathrm{ord}}\left(h\right)\right)$, $\left(\mathrm{CPE},{\kappa }_{\mathrm{ord}}^{*}\left(h\right)\right)$ also belongs to the $\left({\mathrm{CPE}}_{\varphi },{\kappa }_{\varphi }\left(h\right)\right)$-family.

A few examples are plotted in Figure 3, where the DGP $X_t=s_{I_t}$ assumes the rank counts $I_t$ to have $\mathrm{Bin}(4,p)$-marginals. In the top panel, the DGP is i. i. d., whereas $\left(I_t\right)$ follows a so-called first-order binomial autoregressive (BAR$\left(1\right)$) model with dependence parameter $\rho =0.4$ ([2] Section 3.3) in the lower panel, i.e., the DGP exhibits a medium level of positive dependence. While the resulting dependence structure is investigated in more detail in Section 4, Figure 3 considers the asymptotic standard error (SE) ${\sigma }_{\varphi }$ and bias $n(E[{\mathrm{CPE}}_{\varphi }(\widehat{\mathit{f}})]-{\mathrm{CPE}}_{\varphi }\left(\mathit{f}\right))$ according to Theorem 1. Obviously, an increase of serial dependence causes an increase of both SE and bias. While most measures have a rather stable SE for varying p (except for extremely small p, where we are close to a one-point distribution), the EGF ${\varphi }_a$ with $a=1/2$ varies a lot. In particular, the bias takes rather extreme values with decreasing p for this case, which can be explained by the strongly negative exponents at $f_k,1-f_k$ in $h_{kk}$ from Table 1. Thus, choices $a<1$ seem not advisable for practice. The boundary case $a=1$ has a constant bias for an i. i. d. DGP. For ${\varphi }_q$ with $q=4$, we note an oscillating behavior of both bias and SE. The lowest bias and SE are achieved for the cases ${\varphi }_a$ with $a>1$.

Figure 3. Plots for specific cases of ${\varphi }_a\left(z\right)=(z-z^a)/(a-1)$ and ${\varphi }_q\left(z\right)=1-{|2z-1|}^q$: ${\sigma }_{\varphi }$ (left) and $n(E[CP{E}_{\varphi }(\widehat{f})]-CP{E}_{\varphi }\left(f\right))$ (right) against p of marginal $\mathrm{Bin}(4,p)$, where i. i. d. DGP in (a,b), and BAR$\left(1\right)$ DGP with dependence parameter $\rho =0.4$ in (c,d).

The newly obtained measure ${\kappa }_{\mathrm{ord}}^{*}\left(h\right)$ from (14) constitutes a counterpart to the nominal measures ${\kappa }^{*}\left(h\right),{\kappa }^{☆}\left(h\right)$ in Weiß [20]. It is worth pointing out that the latter measures were derived from the nominal entropy and extropy, respectively, while the $\mathrm{CPE}$ in (5) can be interpreted as a combination of cumulative entropy and extropy. It has to be noted that ${\kappa }_{\mathrm{ord}}^{*}\left(h\right)$ also shares a disadvantage with ${\kappa }^{*}\left(h\right)$: if only one of the $f_i$ equals 0 or 1, we suffer from a division by 0 in (14). For ${\kappa }_{\mathrm{ord}}\left(h\right)$ according to (9), by contrast, a division by 0 only happens in the (deterministic) case of a one-point distribution. As a workaround when computing ${\kappa }_{\mathrm{ord}}^{*}\left(h\right)$, it is recommended to replace the affected summands in (14) by 0.

Remark 3.

If $f=f_{two}$, then all $d_k=0$ in (10). Therefore, the linear term in (11) vanishes. In fact, for any two-point distribution on $s_0$ and $s_m$, we necessarily have $f_0=\dots =f_{m-1}$ and ${\widehat{f}}_0=\dots ={\widehat{f}}_{m-1}$. Therefore, ${\mathrm{CPE}}_{\varphi }(\widehat{\mathit{f}})$ reduces to ${\mathrm{CPE}}_{\varphi }(\widehat{\mathit{f}})=\frac{1}{2\varphi (1/2)}(\varphi \left({\widehat{f}}_0\right)+\varphi (1-{\widehat{f}}_0))$, and the quadratic term in (11) to $\frac{m}{2}{h}_{00}{({\widehat{f}}_0-f_0)}^2$. Hence, in this special case,

$$n({\mathrm{CPE}}_{\varphi }(\widehat{\mathit{f}})-{\mathrm{CPE}}_{\varphi }\left(\mathit{f}\right))\stackrel{a}{\sim }\frac{m}{2}{h}_{00}{\sigma }_{00}·{\chi }_1^2.$$

For example, plugging-in $h_{00}=-\frac{8}{m}$ for $\varphi \left(z\right)=z(1-z)$ corresponding to the IOV in (4), we obtain the result in Remark 7.1.2 in Weiß [14].

4. Asymptotic Distribution of Sample ${\mathit{\kappa }}_{\mathit{\varphi }}\left(\mathit{h}\right)$

The bias equation in Theorem 1 gives rise to a novel family of serial dependence measures for ordinal time series, namely

$${\kappa }_{\varphi }\left(h\right)=\frac{{\sum }_{i=0}^{m-1}\left({\varphi }^{″}\left(f_i\right)+{\varphi }^{″}(1-f_i)\right)\left(f_{ii}\left(h\right)-f_i^2\right)}{{\sum }_{i=0}^{m-1}\left({\varphi }^{″}\left(f_i\right)+{\varphi }^{″}(1-f_i)\right)f_i(1-f_i)}$$

(15)

for a given EGF $\varphi$. Some examples are plotted in the left part of Figure 4, where the DGP $X_t=s_{I_t}$ assumes that the rank counts $I_t$ follow the BAR$\left(1\right)$ model with marginal distribution $\mathrm{Bin}(4,0.3)$ and dependence parameter $\rho$; recall the discussion of Figure 3. So, the rank counts ${\left(I_t\right)}_{\mathbb{Z}}$ have the first-order ACF $\rho$, whereas the plotted ${\kappa }_{\varphi }\left(1\right)$ have absolute value $\le \left|\rho \right|$.

Figure 4. Plots for specific cases of ${\varphi }_a\left(z\right)=(z-z^a)/(a-1)$ and ${\varphi }_q\left(z\right)=1-{|2z-1|}^q$: ${\kappa }_{\varphi }\left(1\right)$ against BAR$\left(1\right)$’s dependence parameter $\rho$ with marginal $\mathrm{Bin}(4,0.3)$ (left); ${\sigma }_{\kappa }$ against p of marginal $\mathrm{Bin}(4,0.3)$ (right).

In practice, the sample version of this measure, ${\widehat{\kappa }}_{\varphi }\left(h\right)$, is particularly important, where the cumulative (bivariate) probabilities are replaced by the corresponding relative frequencies. For uncovering significant deviations from the null hypothesis of serial independence (then, ${\kappa }_{\varphi }\left(h\right)=0$), the asymptotic distribution of ${\widehat{\kappa }}_{\varphi }\left(h\right)$ under the null of i. i. d. time series data is required. For its derivation, we proceed in an analogous way as in Section 3. As the starting point, we have to extend the asymptotics of the marginal sample CDF in (8) by also considering the bivariate sample CDF ${\widehat{f}}_{ii}\left(h\right)$. Let ${\mathit{f}}_{\left(h\right)}={\left(f_0,\dots ,f_{m-1},f_{00}\left(h\right),\dots ,f_{m-1,m-1}\left(h\right)\right)}^{\top }$, and denote its sample version by ${\widehat{\mathit{f}}}_{\left(h\right)}$. Then, under the same mixing conditions as in Section 3, Weiß [14] established the asymptotic normality

$$\sqrt{n}\left({\widehat{\mathit{f}}}_{\left(h\right)}-{\mathit{f}}_{\left(h\right)}\right)\stackrel{\mathrm{d}}{\to }\mathrm{N}\left(0,{\Sigma }^{\left(h\right)}\right)\mathrm{with}{\Sigma }^{\left(h\right)}={\left({\sigma }_{i,j}^{\left(h\right)}\right)}_{i,j=0,\dots ,2m-1},$$

(16)

and he derived general expressions for the asymptotic (co)variances ${\sigma }_{i,j}^{\left(h\right)}$. Analogous results for the case of missing data can be found in Supplement S.3 of Weiß [15]. For the present task, the asymptotics of the i. i. d.-case are sufficient. Then, $f_{ii}\left(h\right)=f_i^2$ for all $i=0,\dots ,m-1$, and the covariances in (16) are given by

$$\begin{array}{cc}\hfill {\sigma }_{i,j}^{\left(h\right)}={\sigma }_{i,j}=& f_{min\{i,j\}}-f_i{f}_j(\mathrm{see}(8)),\hfill \\ \hfill {\sigma }_{i,m+j}^{\left(h\right)}=& 2f_j\left(f_{min\{i,j\}}-f_i{f}_j\right),\hfill \\ \hfill {\sigma }_{m+i,m+j}^{\left(h\right)}=& \left(f_{min\{i,j\}}+3f_i{f}_j\right)\left(f_{min\{i,j\}}-f_i{f}_j\right)\hfill \end{array}\mathrm{for}i,j\in \{0,\dots ,m-1\},$$

(17)

see Weiß [14] (as well as p. 8 in Supplement S.3 of Weiß [15] if being concerned with missing data).

Next, we derive the asymptotics of ${\widehat{\kappa }}_{\varphi }\left(h\right)$ under the i. i. d.-null, and this requires to derive the second-order Taylor expansion for ${\widehat{\kappa }}_{\varphi }\left(h\right)$; details are postponed to Appendix A.1. As higher-order derivatives of $\varphi$, which are initially used while deriving a bias correction of ${\widehat{\kappa }}_{\varphi }\left(h\right)$, cancel out, the final result still relies on derivatives of $\varphi$ up to order 2 only.

Theorem 2.

Under the null hypothesis of i. i. d. data, i.e., if ${\kappa }_{\varphi }\left(h\right)=0$ for all lags $h\in \mathbb{N}$, and assuming the EGF ϕ to be twice differentiable, it holds that

$$\begin{array}{c}\sqrt{n}\left({\widehat{\kappa }}_{\varphi }\left(h\right)-{\kappa }_{\varphi }\left(h\right)\right)\stackrel{\mathrm{d}}{\to }N\left(0,{\sigma }_{\kappa }^2\right)with{\sigma }_{\kappa }^2={\displaystyle \sum _{j,k=0}^{m-1}}{u}_j{u}_k{\left(f_{min\{j,k\}}-f_j{f}_k\right)}^2,\hfill \\ {\displaystyle where{u}_j=\frac{{\varphi }^{″}\left(f_j\right)+{\varphi }^{″}(1-f_j)}{{\sum }_{i=0}^{m-1}\left({\varphi }^{″}\left(f_i\right)+{\varphi }^{″}(1-f_i)\right)f_i(1-f_i)}.}\hfill \end{array}$$

In addition, the bias-corrected mean of ${\widehat{\kappa }}_{\varphi }\left(h\right)$ is ${\displaystyle E\left[{\widehat{\kappa }}_{\varphi }\left(h\right)\right]\approx -\frac{1}{n}}$.

Note that we have a unique bias correction for any of the measures ${\widehat{\kappa }}_{\varphi }\left(h\right)$, independent of the choice of the EGF $\varphi$. Thus, for application in practice, it remains to compute the asymptotic variance ${\sigma }_{\kappa }^2$ in Theorem 2. This only requires knowledge about ${\varphi }^{″}\left(z\right)$ to evaluate the $u_j$, but not about higher-order derivatives of the EGF $\varphi$ (see Example 3 for illustration). Further examples are plotted in the right part of Figure 4, where ${\sigma }_{\kappa }$ was computed for the marginal distribution $\mathrm{Bin}(4,p)$. The oscillating behavior of ${\sigma }_{\kappa }$ for ${\varphi }_q\left(z\right)$ with $q=4$ is quite striking. It is also interesting to note that among the plotted $\kappa$-measures, the novel ${\widehat{\kappa }}_{\mathrm{ord}}^{*}\left(h\right)$ (case $a=1$) has the lowest variance.

Example 3.

While we have a unique bias correction for ${\widehat{\kappa }}_{\varphi }\left(h\right)$, the asymptotic variance ${\sigma }_{\kappa }^2$ according to Theorem 2 differs for different choices of the EGF ϕ, as the involved $u_j$ depend on ${\varphi }^{″}\left(z\right)$. For example,

• for ${\varphi }_a\left(z\right)=\frac{z-z^a}{a-1}$, we have ${\varphi }_a^{″}\left(z\right)=-a{z}^{a-2}$ according to Example 1,
• while for ${\varphi }_q\left(z\right)=1-{|2z-1|}^q$ with $q\ge 2$, we have ${\varphi }_q^{″}\left(z\right)=-4q(q-1){|2z-1|}^{q-2}$.

Important special cases are:

• For $\varphi \left(z\right)=z(1-z)$, i.e., for the basic ${\kappa }_{ord}\left(h\right)$ according to (9), we have

  $${\varphi }^{″}\left(z\right)=-2,\mathrm{so}{u}_j={\left({\sum }_{i=0}^{m-1}{f}_i(1-f_i)\right)}^{-1}.$$
  Thus, we get

  $${\sigma }_{\kappa }^2=\frac{{\sum }_{j,k=0}^{m-1}{\left(f_{min\{j,k\}}-f_j{f}_k\right)}^2}{{\left({\sum }_{i=0}^{m-1}{f}_i(1-f_i)\right)}^2};$$
  see Theorem 7.2.1 in Weiß [14].
• For $\varphi \left(z\right)=-zlnz$, i.e., for the novel ${\kappa }_{ord}^{*}\left(h\right)$ according to (14), we have

  $${\varphi }^{″}\left(z\right)=-\frac{1}{z},\mathrm{so}{u}_j=\frac{1}{m}\frac{1}{f_j(1-f_j)}.$$
  Thus, we get

  $${\sigma }_{\kappa }^2=\frac{1}{m^2}{\displaystyle \sum _{j,k=0}^{m-1}}\frac{{\left(f_{min\{j,k\}}-f_j{f}_k\right)}^2}{f_j(1-f_j)f_k(1-f_k)}=\frac{1}{m}+\frac{2}{m^2}\sum _{j<k}\frac{f_j(1-f_k)}{f_k(1-f_j)}.$$

For any other choice of ${\varphi }_a\left(z\right)$ and ${\varphi }_q\left(z\right)$, ${\sigma }_{\kappa }^2$ is easily computed using the aforementioned expressions for ${\varphi }_a^{″}\left(z\right)$ and ${\varphi }_q^{″}\left(z\right)$ from Example 1. Since the obtained closed-form formulae do not much simplify, further details are omitted here.

5. Simulation Results

In what follows, we discuss results from a simulation study, being tabulated in Appendix B, where ${10}^4$ replications per scenario were used throughout. In view of our previous findings, achieved when discussing the asymptotics plotted in Figure 3 and Figure 4, we do not further consider the choice $a=1/2<1$ for the EGF ${\varphi }_a$, but we use $a=5/2>2$ instead. The latter choice, in turn, was not presented before as the resulting asymptotic curves could hardly be distinguished from the case $a=2$. So, altogether, $a=1,3/2,2,5/2$ as well as $q=4$ were taken into account for simulations. The ordinal data were generated via binomial rank counts, $X_t=s_{I_t}$ with $I_t\sim \mathrm{Bin}(m,p)$, which either exhibit serial dependence caused by a BAR$\left(1\right)$ DGP with dependence parameter $\rho$, or which are i. i. d. (corresponding to $\rho =0$).

Let us start with the ordinal dispersion measures ${\widehat{\mathrm{CPE}}}_{\varphi }$. Table A1 presents the simulated means (upper part) and SEs (lower part) for the case of i. i. d. ordinal data, and these are compared to the asymptotic values obtained from Theorem 1. Generally, we have an excellent agreement between simulated and asymptotic values, i.e., the derived asymptotic approximation to the true distribution of ${\widehat{\mathrm{CPE}}}_{\varphi }$ works well in practice. This is even more remarkable as also the low sample size $n=50$ is included. There is a somewhat larger deviation only for the mean in the case $a=1$, i.e., for the CPE (5), if $n\le 100$ and $p=0.1$. In this specific case, the simulated sample distribution might be quite close to a one-point distribution, which might cause computational issues for (5); recall that the convention $0ln0=0$ has to be used. However, as the approximation quality is good throughout, a pivotal argument for the choice of $\varphi$ in practice might be that the least SEs are observed if using ${\varphi }_a$ with $a=3/2,2,5/2$.

Table A2 considers exactly the same marginals as before, but now in the presence of additional serial dependence ($\rho =0.4$). Comparing Table A1 and Table A2, it becomes clear that the additional dependence causes increased bias and SE. However, and this is the crucial point for practice, the asymptotic approximations from Theorem 1 work as well as they do in the i. i. d.-case. If there are visible deviations at all, then these happen again mainly for $p=0.1$ and low sample sizes. Overall, we have an excellent approximation quality throughout, but with least SEs again for $a=3/2,2,5/2$.

While the ${\mathrm{CPE}}_{\varphi }$-type dispersion measures and their asymptotics perform well, essentially for any choice of $\varphi$, the gap becomes wider when looking at the serial dependence measure ${\widehat{\kappa }}_{\varphi }\left(h\right)$. The asymptotics in Theorem 2 refer to the i. i. d.-case, which is used as the null hypothesis ($H_0$) if testing for significant serial dependence. Thus, let us start by investigating again the mean and SE of ${\widehat{\kappa }}_{\varphi }\left(1\right)$ for i. i. d. data (same DGPs as in Table A1); see the results in Table A3. For the asymptotic mean, we have the unique approximation $-1/n$, and this works well except for $p=0.1$ and low sample sizes. In particular, for ${\varphi }_a$ with $a=1$, i.e., for ${\widehat{\kappa }}_{\mathrm{ord}}^{*}\left(1\right)$, we get notable deviations. The reason is given by the computation of ${\widehat{\kappa }}_{\mathrm{ord}}^{*}\left(1\right)$, where division by zero might happen (in the simulations, this was circumvented by replacing a zero by ${10}^{-6}$). In a weakened form, we observe a similar issue for the case $a=3/2$; generally, we are faced with the zero problem if $a<2$ because of the second-order derivatives of ${\varphi }_a\left(z\right)$. Analogous deviations are observed for the SEs. Here, generally, the simulated SEs tend to be larger than the asymptotic ones. As a consequence, if using the asymptotic SEs for calculating the critical values when testing $H_0$, we expect a tendency to oversizing.

If looking at the simulated rejection rates in Table A4, first at the size values ($\rho =0$) being printed in italic font, we indeed see sizes being slightly larger than the nominal 5%-level, as long as $n\le 100$. For larger sample sizes, by contrast, the ${\widehat{\kappa }}_{\varphi }\left(1\right)$-test satisfies the given size requirement quite precisely. Here, we computed the critical values by plugging-in the respective sample CDF $\widehat{\mathit{f}}$ into the formula for ${\sigma }_{\kappa }^2$. In Table A4, power values for $\rho \ne 0$ are also shown. Note that for a BAR$\left(1\right)$ process, $\rho$ can take any positive value in $(0;1)$, but the attainable range of negative values is bounded from below by $max\left\{-\frac{1-p}{p},-\frac{p}{1-p}\right\}$ [2]. Thus, only $\rho =-0.4,-0.2$ are considered in Table A4. Generally, the ${\widehat{\kappa }}_{\varphi }\left(1\right)$-tests are powerful regarding both positive and negative dependencies, but the actual power performance differs a lot for different $\varphi$. The worst power is observed for ${\varphi }_q$ with $q=4$, followed by ${\varphi }_a$ with $a=1$. Regarding the remaining ${\varphi }_a$-cases, $a=3/2$ does slightly worse than $a=2,5/2$, and we often have a slight advantage for $a=5/2$, especially for negative dependencies.

To sum up, while the whole $\left({\mathrm{CPE}}_{\varphi },{\kappa }_{\varphi }\left(h\right)\right)$-family is theoretically appealing, and while there are hardly any noteworthy problems with the sample dispersion measures ${\widehat{\mathrm{CPE}}}_{\varphi }$, the performance of ${\widehat{\kappa }}_{\varphi }\left(h\right)$ clearly depends on the choice of $\varphi$. It is recommended to use the family of a-entropies (6), and there, $a\ge 2$ is preferable. The measure ${\widehat{\kappa }}_{\mathrm{ord}}^{*}\left(1\right)$ from (14), for example, although theoretically appealing as a combination of entropy and extropy, has a relatively bad finite-sample performance. The probably most well-known pair, $\left(\mathrm{IOV},{\kappa }_{\mathrm{ord}}\left(h\right)\right)$, has a good performance, although there appears to be a slight advantage if choosing a somewhat larger than 2, such as $a=5/2$ (recall that $a=3$ leads back to the case $a=2$).

6. Data Application

Ordinal time series are observed in quite diverse application areas. Economic examples include time series on credit ratings [14] or on fear states at the stock market [20], and a climatological example is the level of cloudiness of the sky [23]. Health-related examples are time series of electroencephalographic (EEG) sleep states [24], the pain severity of migraine attacks, and the level of perceived stress [15]. In this section, we are concerned with an environmental application, namely the level of air quality. Different definitions of air quality have been reported in the literature. In Chen Chiu [25], the air quality index (AQI) is used for expressing the daily air quality, with levels ranging from $s_0=“\mathrm{good}”$ to $s_5=“\mathrm{hazardous}”$. Another case study is reported by Liu et al. [26], who use the classification of the Chinese government, which again distinguishes $m+1=6$ levels, but now ranging from $s_0=“\mathrm{excellent}”$ to $s_5=“\mathrm{severely}\mathrm{polluted}”$. The latter article investigates daily time series from thirty Chinese cities for the period December 2013–July 2019, i.e., the sample size equals $n=2068$. In what follows, we use one of the time series studied by Liu et al. [26], namely the daily air quality levels $x_1,\dots ,x_n$ in Shanghai, for illustrating our novel results about cumulative paired $\varphi$-entropies.

The considered time series is plotted in the top panel of Figure 5. The bottom left graph shows the sample version of the probability mass function (PMF) $P(X=s_i)$, i.e., the relative frequencies of the categories. It exhibits a unimodal shape with mode (=median) in $s_1=“\mathrm{good}”$. The serial dependence structure is analyzed in the bottom right graph, where ${\widehat{\kappa }}_{\varphi }\left(h\right)$ with a-entropy having $a=5/2$ is used, as this is the recommended choice according to Section 5. All of the plotted ${\widehat{\kappa }}_{\varphi }\left(h\right)$-values are significantly different from 0 at the 5 %-level, where the critical values (plotted as dashed lines in Figure 5) are computed as $\{-0.029,0.028\}$ according to Theorem 2 (and by plugging-in the sample CDF). We recognize a medium level of dependence (${\widehat{\kappa }}_{\varphi }\left(1\right)\approx 0.378$), which quickly decreases with increasing time lag h, similar to an AR-type process.

Figure 5. Daily air quality level in Shanghai: plot of time series $\left(x_t\right)$ in top panel; plot of sample PMF (left) and ${\widehat{\kappa }}_{\varphi }\left(h\right)$ (right) in bottom panel; ${\widehat{\kappa }}_{\varphi }\left(h\right)$ uses a-entropy with $a=5/2$.

Let us now have a closer look at the dispersion properties of the Shanghai series. The different choices of the ${\mathrm{CPE}}_{\varphi }$-measure considered so far provide slightly different results regarding the extent of dispersion. In accordance with Figure 2, the largest point estimates are computed for $a=1/2$ (0.514) and $q=4$ (0.465), followed by $a=1$ with 0.394, whereas $a=3/2$ (0.349), $a=2$ (0.332), and $a=5/2$ (0.328) lead to similar but clearly lower values. Comparing the sample PMF in Figure 5 to the extreme scenarios of a one-point and an extreme two-point distribution, the PMF appears to be more close to a one-point than to a two-point distribution, i.e., the lower ones among the above dispersion values seem to be more realistic here.

The novel asymptotics of Theorem 1 allow to judge the estimation uncertainty for the above point estimates. To keep the discussion simple, let us focus again on the case $a=5/2$. In the first step, we compute the i. i. d.-approximations of bias and SE, $\frac{1}{2n}({\sum }_{i=0}^{m-1}{h}_{ii}{f}_i(1-f_i))$ and $n^{-1/2}{\sigma }_{\varphi ,iid}$, respectively. By plugging-in the sample CDF, these are obtained as $-1.54·{10}^{-4}$ and $7.83·{10}^{-3}$, respectively. However, these i. i. d.-results are misleading in the present example as the data exhibit significant serial dependence (recall Figure 5). As we know from Theorem 1, the bias has to be increased by the factor $1+2{\sum }_{h=1}^{\infty }{\kappa }_{\varphi }\left(h\right)$, and the SE by $\left(1+2{\sum }_{h=1}^{\infty }{\vartheta }_{\varphi }\left(h\right)\right){}^{1/2}$. These factors shall now be computed based on the so-called “ZOBPAR model” proposed by Liu et al. [26], which constitutes a rank-count approach, $X_t=s_{I_t}$. In view of the AR$\left(1\right)$-like dependence structure and the high frequency for $s_1$, namely 0.560, the conditional distribution of $I_t|I_{t-1},\dots$ is assumed to be a truncated Poisson distribution, truncated to the range $\{0,\dots ,5\}$, with time-varying Poisson parameter ${\lambda }_t=0.3489+0.7594I_{t-1}$ and additional one-inflation parameter 0.3463 ([26] Table III). For this model fit, we compute

$$1+2{\sum }_{h=1}^{\infty }{\kappa }_{\varphi }\left(h\right)\approx 1.907,\sqrt{1+2{\sum }_{h=1}^{\infty }{\vartheta }_{\varphi }\left(h\right)}\approx 1.343.$$

Thus, an approximate 95 %-confidence interval (CI) for ${\mathrm{CPE}}_{\varphi }$ is given by $\approx (0.308;0.349)$. CIs for the remaining ${\mathrm{CPE}}_{\varphi }$-measures are computed analogously, leading to $(0.487;0.544)$ for $a=1/2$, to $(0.372;0.418)$ for $a=1$, to $(0.328;0.371)$ for $a=3/2$, to $(0.312;0.353)$ for $a=2$, and to $(0.439;0.492)$ for $q=4$.

7. Conclusions

In this article, we considered the family of cumulative paired $\varphi$-entropies. For each appropriate choice of the EGF $\varphi$, an ordinal dispersion measure ${\mathrm{CPE}}_{\varphi }\left(\mathit{f}\right)$ is implied. For example, particular choices from the families of a-entropies or q-entropies, respectively, lead to well-known dispersion measures from the literature. The first main contribution of this work was the derivation of the asymptotic distribution of the sample version ${\widehat{\mathrm{CPE}}}_{\varphi }$ for ordinal time series data. These asymptotics can be used to approximate the true distribution of ${\widehat{\mathrm{CPE}}}_{\varphi }$, e.g., to compute approximate confidence intervals. Simulations showed that these asymptotics lead to an excellent finite-sample performance. Based on the obtained expression for the asymptotic bias of ${\widehat{\mathrm{CPE}}}_{\varphi }$, we recognized that each EGF $\varphi$ also implies a $\kappa$-type serial dependence measures, i.e., altogether, we have a matched pair $\left({\mathrm{CPE}}_{\varphi },{\kappa }_{\varphi }\left(h\right)\right)$ for each EGF $\varphi$. Again, we analyzed the asymptotics of the sample version ${\widehat{\kappa }}_{\varphi }\left(h\right)$, and these can be utilized for testing for significant serial dependence in the given ordinal time series. This time, however, the finite-sample performance clearly depends on the choice of $\varphi$. Choosing ${\widehat{\kappa }}_{\varphi }\left(h\right)$ based on an a-entropy with $a\ge 2$, such as $a=5/2$, ensures good finite-sample properties. The practical application of the measures $\left({\mathrm{CPE}}_{\varphi },{\kappa }_{\varphi }\left(h\right)\right)$ and their asymptotics was demonstrated with an ordinal time series on the daily level of air quality in Shanghai.

Appendix A.1. Proof of Theorem 2

First, we derive the second-order Taylor expansion for ${\widehat{\kappa }}_{\varphi }\left(h\right)$. For this purpose, define $g:{(0;1)}^{2m}\to \mathbb{R}$ by mapping $\mathit{y}=(y_0,\dots ,y_{2m-1})$ onto

$$g\left(\mathit{y}\right)=\frac{{\sum }_{i=0}^{m-1}\left({\varphi }^{″}\left(y_i\right)+{\varphi }^{″}(1-y_i)\right)\left(y_{m+i}-y_i^2\right)}{{\sum }_{i=0}^{m-1}\left({\varphi }^{″}\left(y_i\right)+{\varphi }^{″}(1-y_i)\right)y_i(1-y_i)}.$$

So, according to (15), $g\left({\mathit{f}}_{\left(h\right)}\right)={\kappa }_{\varphi }\left(h\right)$ and $g\left({\widehat{\mathit{f}}}_{\left(h\right)}\right)={\widehat{\kappa }}_{\varphi }\left(h\right)$. In fact, because of the i. i. d.-assumption, we even have $g\left({\mathit{f}}_{\left(h\right)}\right)=0$.

For the intended Taylor expansion of ${\widehat{\kappa }}_{\varphi }\left(h\right)$, we have to compute all partial derivatives of $g\left(\mathit{y}\right)$ up to order 2, and to evaluate these derivatives in $\mathit{y}={\mathit{f}}_{\left(h\right)}$, using that $f_{ii}\left(h\right)=f_i^2$ for all $i=0,\dots ,m-1$ under the i. i. d.-assumption, and thus $g\left({\mathit{f}}_{\left(h\right)}\right)=0$. We use the notations

$$u_j:=\frac{{\varphi }^{″}\left(f_j\right)+{\varphi }^{″}(1-f_j)}{{\sum }_{i=0}^{m-1}\left({\varphi }^{″}\left(f_i\right)+{\varphi }^{″}(1-f_i)\right)f_i(1-f_i)},v_j:=\frac{{\varphi }^{‴}\left(f_j\right)-{\varphi }^{‴}(1-f_j)}{{\sum }_{i=0}^{m-1}\left({\varphi }^{″}\left(f_i\right)+{\varphi }^{″}(1-f_i)\right)f_i(1-f_i)}.$$

(A1)

Note that if $\varphi$ is symmetric around $1/2$, i.e., if $\varphi \left(z\right)=\varphi (1-z)$, then $u_j,v_j$ simplify because of ${\varphi }^{″}\left(f_j\right)+{\varphi }^{″}(1-f_j)=2{\varphi }^{″}\left(f_j\right)$ and ${\varphi }^{‴}\left(f_j\right)-{\varphi }^{‴}(1-f_j)=2{\varphi }^{‴}\left(f_j\right)$. Using (A1), we get for $0\le j,k\le m-1$ that

$$\begin{array}{l}{d}_j^{\left(\kappa \right)}:=\frac{\partial }{\partial y_j}g\left(f_{\left(h\right)}\right)=-2f_j{u}_j,\\ d_{m+j}^{\left(\kappa \right)}:=\frac{\partial }{\partial y_{m+j}}g\left(f_{\left(h\right)}\right)=u_j,\\ h_{j,k}^{\left(\kappa \right)}:=\frac{{\partial }^2}{\partial y_j\partial y_k}g\left(f_{\left(h\right)}\right)=2f_j{f}_k\left((1-f_k)u_j{v}_k+(1-f_j)u_k{v}_j\right)\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}+2(f_j+f_k-4f_j{f}_k)u_j{u}_k-2{\delta }_{j,k}(u_j+2f_j{v}_j),\\ h_{j,m+k}^{\left(\kappa \right)}:=\frac{{\partial }^2}{\partial y_j\partial y_{m+k}}g\left(f_{\left(h\right)}\right)=\left({\delta }_{j,k}-f_j(1-f_j)u_k\right)v_j-(1-2f_j)u_j{u}_k,\\ h_{m+j,m+k}^{\left(\kappa \right)}:=\frac{{\partial }^2}{\partial y_{m+j}\partial y_{m+k}}g\left(f_{\left(h\right)}\right)=0.\end{array}$$

(A2)

The proof of Equation (A2) is provided by Appendix A.2. We denote the gradient (Jacobian) of $g\left(f_{\left(h\right)}\right)$ by ${\mathbf{D}}_{\kappa }=(d_0^{\left(\kappa \right)},\dots ,d_{2m-1}^{\left(\kappa \right)})$, and the Hessian equals ${\mathbf{H}}_{\kappa }=diag(h_{00}^{\left(\kappa \right)},\dots ,h_{2m-1,2m-1}^{\left(\kappa \right)})$.

Example A1.

Let us pick up Example 1 and continue with the derivatives of the EGFs, as required for evaluating (A1) and (A2).

For ${\varphi }_a\left(z\right)=\frac{z-z^a}{a-1}$, we have ${\varphi }_a^{″}\left(z\right)=-a{z}^{a-2}$, and thus ${\varphi }_a^{‴}\left(z\right)=-a(a-2)z^{a-3}$. Specific examples are

• $a\to 1$ corresponding to the EGF $\varphi \left(z\right)=-zlnz$, then

  $${\varphi }^{″}\left(z\right)=-\frac{1}{z},{\varphi }^{″}\left(z\right)=\frac{1}{z^2},\mathrm{so}{u}_j=\frac{1}{m}\frac{1}{f_j(1-f_j)},v_j=\frac{1}{m}\frac{2f_j-1}{f_j^2{(1-f_j)}^2};$$
• $a=2$ corresponding to the EGF $\varphi \left(z\right)=z(1-z)$, then

  $${\varphi }^{″}\left(z\right)=-2,{\varphi }^{″\prime }\left(z\right)=0,\mathrm{so}{u}_j={\left({\sum }_{i=0}^{m-1}{f}_i(1-f_i)\right)}^{-1},v_j=0;$$
• $a=3$ corresponding to the EGF $\varphi \left(z\right)=\frac{1}{2}z(1-z^2)$, then

  $${\varphi }^{″}\left(z\right)=-3z,{\varphi }^{‴}\left(z\right)=-3,\mathrm{so}{u}_j,v_j\mathrm{are}\mathrm{as}\mathrm{before}.$$
  For the equivalence of the $a=2,3$, recall (7).

For ${\varphi }_q\left(z\right)=1-{\left(1-2min\{z,1-z\}\right)}^q=1-{|2z-1|}^q$ with $q>3$, we get

$$\begin{array}{cc}\hfill {\varphi }_q^{″}\left(z\right)=& {-4q(q-1)|2z-1|}^{q-2},{\varphi }_q^{‴}\left(z\right)=-8q(q-1)(q-2){|2z-1|}^{q-3}\mathrm{sgn}(2z-1).\hfill \end{array}$$

Recall that (A1) simplifies because of the symmetry of ${\varphi }_q\left(z\right)$ around $1/2$.

Using (A2), the second-order Taylor expansion of ${\widehat{\kappa }}_{\varphi }\left(h\right)$ equals

$${\widehat{\kappa }}_{\varphi }\left(h\right)\approx {\kappa }_{\varphi }\left(h\right)+{\mathbf{D}}_{\kappa }\left({\widehat{\mathit{f}}}_{\left(h\right)}-{\mathit{f}}_{\left(h\right)}\right)+\frac{1}{2}{\left({\widehat{\mathit{f}}}_{\left(h\right)}-{\mathit{f}}_{\left(h\right)}\right)}^{\top }{\mathbf{H}}_{\kappa }\left({\widehat{\mathit{f}}}_{\left(h\right)}-{\mathit{f}}_{\left(h\right)}\right).$$

(A3)

In analogy to Section 3, we now conclude that

$$\begin{array}{c}\sqrt{n}{\displaystyle (}{\widehat{\kappa }}_{\varphi }\left(h\right)-{\kappa }_{\varphi }\left(h\right){\displaystyle )}\stackrel{\mathrm{d}}{\to }\mathrm{N}\left(0,{\sigma }_{\kappa }^2\right)\mathrm{with}{\sigma }_{\kappa }^2={\mathbf{D}}_{\kappa }{\Sigma }^{\left(h\right)}{\mathbf{D}}_{\kappa }^{\top },\hfill \\ \mathrm{where}{\sigma }_{\kappa }^2={\displaystyle {\sum }_{j,k=0}^{m-1}}{u}_j{u}_k{\left(f_{min\{j,k\}}-f_j{f}_k\right)}^2.\hfill \end{array}$$

(A4)

The proof of Equation (A4) is provided by Appendix A.3. ${\sigma }_{\kappa }^2$ is computed explicitly by plugging-in (A1).

An approximate bias correction is obtained from (A3) (see the analogous arguments in Section 3). Using that $h_{m+j,m+k}^{\left(\kappa \right)}=0$, we get that

$$\begin{array}{c}E\left[{\widehat{\kappa }}_{\varphi }\left(h\right)\right]\approx 0+\frac{1}{2n}{\displaystyle {\sum }_{j,k=0}^{m-1}}\left(h_{j,k}^{\left(\kappa \right)}{\sigma }_{j,k}+2h_{j,m+k}^{\left(\kappa \right)}{\sigma }_{j,m+k}^{\left(h\right)}\right)=-\frac{1}{n},\hfill \end{array}$$

(A5)

see the proof in Appendix A.4. Using (A4) and (A5), the derivation of Theorem 2 is complete.

Appendix A.2. Proof of Equation (A2)

For

$$g\left(y\right)=\frac{{\sum }_{i=0}^{m-1}\left({\varphi }^{″}\left(y_i\right)+{\varphi }^{″}(1-y_i)\right)\left(y_{m+i}-y_i^2\right)}{{\sum }_{i=0}^{m-1}\left({\varphi }^{″}\left(y_i\right)+{\varphi }^{″}(1-y_i)\right)y_i(1-y_i)},$$

we compute all partial derivatives up to order 2. For doing this, we have to require that the EGF $\varphi$ is even four times differentiable. Then,

$$\frac{\partial }{\partial y_{m+j}}g\left(y\right)=\frac{{\varphi }^{″}\left(y_j\right)+{\varphi }^{″}(1-y_j)}{{\sum }_{i=0}^{m-1}\left({\varphi }^{″}\left(y_i\right)+{\varphi }^{″}(1-y_i)\right)y_i(1-y_i)}\mathrm{and}\frac{{\partial }^2}{\partial y_{m+j}\partial y_{m+k}}g\left(\mathit{y}\right)=0\mathrm{for}\mathrm{all}0\le j,k\le m-1.$$

Next, using the product and chain rule,

$$\frac{\partial }{\partial y_j}g\left(\mathit{y}\right)=\frac{\frac{\partial }{\partial y_j}\left({\varphi }^{″}\left(y_j\right)+{\varphi }^{″}(1-y_j)\right)\left(y_{m+j}-y_j^2\right)}{{\sum }_{i=0}^{m-1}\left({\varphi }^{″}\left(y_i\right)+{\varphi }^{″}(1-y_i)\right)y_i(1-y_i)}-\frac{g\left(y\right)\frac{\partial }{\partial y_j}\left({\varphi }^{″}\left(y_j\right)+{\varphi }^{″}(1-y_j)\right)y_j(1-y_j)}{{\sum }_{i=0}^{m-1}\left({\varphi }^{″}\left(y_i\right)+{\varphi }^{″}(1-y_i)\right)y_i(1-y_i)}.$$

Here,

$$\begin{array}{c}\frac{\partial }{\partial y_j}\left({\varphi }^{″}\left(y_j\right)+{\varphi }^{″}(1-y_j)\right)\left(y_{m+j}-y_j^2\right)\hfill \\ =\left({\varphi }^{‴}\left(y_j\right)-{\varphi }^{‴}(1-y_j)\right)\left(y_{m+j}-y_j^2\right)-2y_j\left({\varphi }^{″}\left(y_j\right)+{\varphi }^{″}(1-y_j)\right),\hfill \end{array}$$

and

$$\begin{array}{c}\frac{\partial }{\partial y_j}\left({\varphi }^{″}\left(y_j\right)+{\varphi }^{″}(1-y_j)\right)y_j(1-y_j)\hfill \\ =\left({\varphi }^{‴}\left(y_j\right)-{\varphi }^{‴}(1-y_j)\right)y_j(1-y_j)+(1-2y_j)\left({\varphi }^{″}\left(y_j\right)+{\varphi }^{″}(1-y_j)\right).\hfill \end{array}$$

Thus,

$$\begin{array}{cc}\hfill \frac{\partial }{\partial y_j}g\left(\mathit{y}\right)=& {\left({\displaystyle \sum _{i=0}^{m-1}}\left({\varphi }^{″}\left(y_i\right)+{\varphi }^{″}(1-y_i)\right)y_i(1-y_i)\right)}^{-1}\hfill \\ & ·{\displaystyle (}({\varphi }^{‴}(y_j)-{\varphi }^{‴}(1-y_j))(y_{m+j}-y_j^2-g(y)y_j(1-y_j))\hfill \\ & -({\varphi }^{″}(y_j)+{\varphi }^{″}(1-y_j))(2y_j+g(y)(1-2y_j)){\displaystyle )}.\hfill \end{array}$$

Consequently,

$$\begin{array}{c}\frac{{\partial }^2}{\partial y_j\partial y_{m+k}}g\left(\mathit{y}\right)=\frac{{\delta }_{j,k}\left({\varphi }^{‴}\left(y_j\right)-{\varphi }^{‴}(1-y_j)\right)}{{\sum }_{i=0}^{m-1}\left({\varphi }^{″}\left(y_i\right)+{\varphi }^{″}(1-y_i)\right)y_i(1-y_i)}-\frac{\frac{\partial }{\partial y_{m+k}}g\left(y\right)}{{\sum }_{i=0}^{m-1}\left({\varphi }^{″}\left(y_i\right)+{\varphi }^{″}(1-y_i)\right)y_i(1-y_i)}\hfill \\ ·{\displaystyle (}\left({\varphi }^{‴}\left(y_j\right)-{\varphi }^{‴}(1-y_j)\right)y_j(1-y_j)+(1-2y_j)\left({\varphi }^{″}\left(y_j\right)+{\varphi }^{″}(1-y_j)\right){\displaystyle )}.\hfill \end{array}$$

Finally,

$$\begin{array}{c}\frac{{\partial }^2}{\partial y_j\partial y_k}g\left(\mathit{y}\right)=-\frac{\frac{\partial }{\partial y_j}g\left(\mathit{y}\right)\frac{\partial }{\partial y_k}\left({\varphi }^{″}\left(y_k\right)+{\varphi }^{″}(1-y_k)\right)y_k(1-y_k)}{{\sum }_{i=0}^{m-1}\left({\varphi }^{″}\left(y_i\right)+{\varphi }^{″}(1-y_i)\right)y_i(1-y_i)}\hfill \\ +{\left({\displaystyle \sum _{i=0}^{m-1}}\left({\varphi }^{″}\left(y_i\right)+{\varphi }^{″}(1-y_i)\right)y_i(1-y_i)\right)}^{-1}\hfill \\ \left.\begin{array}{c}·\frac{\partial }{\partial y_k}(\left({\varphi }^{‴}\left(y_j\right)-{\varphi }^{‴}(1-y_j)\right)\left(y_{m+j}-y_j^2-g\left(\mathit{y}\right)y_j(1-y_j)\right)\hfill \\ -\left({\varphi }^{\prime \prime }\left(y_j\right)+{\varphi }^{\prime \prime }(1-y_j)\right)\left(2y_j+g\left(\mathit{y}\right)(1-2y_j)\right))\hfill \end{array}\right\}=:A.\hfill \end{array}$$

Here, we have for $k\ne j$ that

$$\begin{array}{cc}\hfill A=-(& y_j(1-y_j)\left({\varphi }^{‴}\left(y_j\right)-{\varphi }^{‴}(1-y_j)\right)+(1-2y_j)\left({\varphi }^{″}\left(y_j\right)+{\varphi }^{″}(1-y_j)\right))\frac{\partial }{\partial y_k}g\left(\mathit{y}\right).\hfill \end{array}$$

For $k=j$, we have that

$$\begin{array}{cc}\hfill A=& \frac{\partial }{\partial y_j}\left({\varphi }^{‴}\left(y_j\right)-{\varphi }^{‴}(1-y_j)\right)\left(y_{m+j}-y_j^2-g\left(\mathit{y}\right)y_j(1-y_j)\right)\hfill \\ & -\frac{\partial }{\partial y_j}\left({\varphi }^{″}\left(y_j\right)+{\varphi }^{″}(1-y_j)\right)\left(2y_j+g\left(\mathit{y}\right)(1-2y_j)\right)\hfill \\ \hfill =& \left({\varphi }^{\left(4\right)}\left(y_j\right)+{\varphi }^{\left(4\right)}(1-y_j)\right)\left(y_{m+j}-y_j^2-g\left(\mathit{y}\right)y_j(1-y_j)\right)\hfill \\ & -\left({\varphi }^{‴}\left(y_j\right)-{\varphi }^{‴}(1-y_j)\right)y_j(1-y_j)\frac{\partial }{\partial y_j}g\left(y\right)\hfill \\ & -2\left({\varphi }^{‴}\left(y_j\right)-{\varphi }^{‴}(1-y_j)\right)\left(2y_j+g\left(\mathit{y}\right)(1-2y_j)\right)\hfill \\ & -\left({\varphi }^{″}\left(y_j\right)+{\varphi }^{″}(1-y_j)\right)\left(2-2g\left(\mathit{y}\right)+(1-2y_j)\frac{\partial }{\partial y_j}g\left(\mathit{y}\right)\right).\hfill \end{array}$$

So, altogether,

$$\begin{array}{cc}\hfill A=& -{\delta }_{jk}{\displaystyle (}2\left({\varphi }^{″}\left(y_j\right)+{\varphi }^{″}(1-y_j)\right)\left(1-g\left(y\right)\right)+2\left({\varphi }^{‴}\left(y_j\right)-{\varphi }^{‴}(1-y_j)\right)\left(2y_j+g\left(\mathit{y}\right)(1-2y_j)\right)\hfill \\ & -\left({\varphi }^{\left(4\right)}\left(y_j\right)+{\varphi }^{\left(4\right)}(1-y_j)\right)\left(y_{m+j}-y_j^2-g\left(\mathit{y}\right)y_j(1-y_j)\right))\hfill \\ & -{\displaystyle (}{y}_j(1-y_j)\left({\varphi }^{‴}\left(y_j\right)-{\varphi }^{‴}(1-y_j)\right)+(1-2y_j)\left({\varphi }^{″}\left(y_j\right)+{\varphi }^{″}(1-y_j)\right){\displaystyle )}\frac{\partial }{\partial y_k}g\left(\mathit{y}\right).\hfill \end{array}$$

Hence,

$$\begin{array}{c}\frac{{\partial }^2}{\partial y_j\partial y_k}g\left(\mathit{y}\right)=-\frac{\left(y_k(1-y_k)\left({\varphi }^{‴}\left(y_k\right)-{\varphi }^{‴}(1-y_k)\right)+(1-2y_k)\left({\varphi }^{″}\left(y_k\right)+{\varphi }^{″}(1-y_k)\right)\right)\frac{\partial }{\partial y_j}g\left(y\right)}{{\sum }_{i=0}^{m-1}\left({\varphi }^{″}\left(y_i\right)+{\varphi }^{″}(1-y_i)\right)y_i(1-y_i)}\hfill \\ -\frac{\left(y_j(1-y_j)\left({\varphi }^{‴}\left(y_j\right)-{\varphi }^{‴}(1-y_j)\right)+(1-2y_j)\left({\varphi }^{″}\left(y_j\right)+{\varphi }^{″}(1-y_j)\right)\right)\frac{\partial }{\partial y_k}g\left(y\right)}{{\sum }_{i=0}^{m-1}\left({\varphi }^{″}\left(y_i\right)+{\varphi }^{″}(1-y_i)\right)y_i(1-y_i)}\hfill \\ -{\delta }_{jk}{\left({\displaystyle \sum _{i=0}^{m-1}}\left({\varphi }^{″}\left(y_i\right)+{\varphi }^{″}(1-y_i)\right)y_i(1-y_i)\right)}^{-1}\hfill \\ ·(2\left({\varphi }^{″}\left(y_j\right)+{\varphi }^{″}(1-y_j)\right)\left(1-g\left(y\right)\right)+2\left({\varphi }^{‴}\left(y_j\right)-{\varphi }^{‴}(1-y_j)\right)\left(2y_j+g\left(\mathit{y}\right)(1-2y_j)\right)\hfill \\ -\left({\varphi }^{\left(4\right)}\left(y_j\right)+{\varphi }^{\left(4\right)}(1-y_j)\right)\left(y_{m+j}-y_j^2-g\left(\mathit{y}\right)y_j(1-y_j)\right)).\hfill \end{array}$$

For the required Taylor expansion, we have to evaluate all these derivatives in $\mathit{y}={\mathit{f}}_{\left(h\right)}$, using that $f_{ii}\left(h\right)=f_i^2$ for all $i=0,\dots ,m-1$, and thus $g\left({\mathit{f}}_{\left(h\right)}\right)=0$. We use the notations introduced in (A1), i.e.,

$$u_j:=\frac{{\varphi }^{″}\left(f_j\right)+{\varphi }^{″}(1-f_j)}{{\sum }_{i=0}^{m-1}\left({\varphi }^{″}\left(f_i\right)+{\varphi }^{″}(1-f_i)\right)f_i(1-f_i)},v_j:=\frac{{\varphi }^{‴}\left(f_j\right)-{\varphi }^{‴}(1-f_j)}{{\sum }_{i=0}^{m-1}\left({\varphi }^{″}\left(f_i\right)+{\varphi }^{″}(1-f_i)\right)f_i(1-f_i)}.$$

Then, we get for $0\le j\le m-1$ that

$$\begin{array}{cc}\hfill d_j^{\left(\kappa \right)}:=& \frac{\partial }{\partial y_j}g\left({\mathit{f}}_{\left(h\right)}\right)=\frac{0-2f_j\left({\varphi }^{″}\left(f_j\right)+{\varphi }^{″}(1-f_j)\right)}{{\sum }_{i=0}^{m-1}\left({\varphi }^{″}\left(f_i\right)+{\varphi }^{″}(1-f_i)\right)f_i(1-f_i)}=-2f_j{u}_j,\hfill \\ \hfill d_{m+j}^{\left(\kappa \right)}:=& \frac{\partial }{\partial y_{m+j}}g\left({\mathit{f}}_{\left(h\right)}\right)=u_j.\hfill \end{array}$$

For the second-order derivatives, we get for $0\le j,k\le m-1$ that

$$\begin{array}{cc}\hfill h_{j,k}^{\left(\kappa \right)}:=& \frac{{\partial }^2}{\partial y_j\partial y_k}g\left({\mathit{f}}_{\left(h\right)}\right)\hfill \\ \hfill =& -\frac{\left(f_k(1-f_k)\left({\varphi }^{‴}\left(f_k\right)-{\varphi }^{‴}(1-f_k)\right)+(1-2f_k)\left({\varphi }^{″}\left(f_k\right)+{\varphi }^{″}(1-f_k)\right)\right)d_j^{\left(\kappa \right)}}{{\sum }_{i=0}^{m-1}\left({\varphi }^{″}\left(f_i\right)+{\varphi }^{″}(1-f_i)\right)f_i(1-f_i)}\hfill \\ & -\frac{\left(f_j(1-f_j)\left({\varphi }^{‴}\left(f_j\right)-{\varphi }^{‴}(1-f_j)\right)+(1-2f_j)\left({\varphi }^{″}\left(f_j\right)+{\varphi }^{″}(1-f_j)\right)\right)d_k^{\left(\kappa \right)}}{{\sum }_{i=0}^{m-1}\left({\varphi }^{″}\left(f_i\right)+{\varphi }^{″}(1-f_i)\right)f_i(1-f_i)}\hfill \\ & -\frac{{\delta }_{jk}2\left(\left({\varphi }^{″}\left(f_j\right)+{\varphi }^{″}(1-f_j)\right)+2f_j\left({\varphi }^{‴}\left(f_j\right)-{\varphi }^{‴}(1-f_j)\right)\right)}{{\sum }_{i=0}^{m-1}\left({\varphi }^{″}\left(f_i\right)+{\varphi }^{″}(1-f_i)\right)f_i(1-f_i)}\hfill \\ \hfill =& -f_k(1-f_k)v_k{d}_j^{\left(\kappa \right)}-(1-2f_k)u_k{d}_j^{\left(\kappa \right)}\hfill \\ & -f_j(1-f_j)v_j{d}_k^{\left(\kappa \right)}-(1-2f_j)u_j{d}_k^{\left(\kappa \right)}-2{\delta }_{jk}{u}_j-4{\delta }_{jk}{f}_j{v}_j\hfill \\ \hfill =& -f_k(1-f_k)(-2f_j{u}_j)v_k-(1-2f_k)(-2f_j{u}_j)u_k\hfill \\ & -f_j(1-f_j)(-2f_k{u}_k)v_j-(1-2f_j)(-2f_k{u}_k)u_j-2{\delta }_{jk}(u_j+2f_j{v}_j)\hfill \\ \hfill =& 2f_j{f}_k(1-f_k)u_j{v}_k+2f_j{f}_k(1-f_j)u_k{v}_j\hfill \\ & +2f_j(1-2f_k)u_j{u}_k+2f_k(1-2f_j)u_j{u}_k-2{\delta }_{jk}(u_j+2f_j{v}_j)\hfill \\ \hfill =& 2f_j{f}_k\left((1-f_k)u_j{v}_k+(1-f_j)u_k{v}_j\right)+2(f_j+f_k-4f_j{f}_k)u_j{u}_k-2{\delta }_{j,k}(u_j+2f_j{v}_j).\hfill \end{array}$$

Similarly,

$$\begin{array}{cc}\hfill h_{j,m+k}^{\left(\kappa \right)}:=& \frac{{\partial }^2}{\partial y_j\partial y_{m+k}}g\left({\mathit{f}}_{\left(h\right)}\right)=\frac{{\delta }_{j,k}\left({\varphi }^{‴}\left(f_j\right)-{\varphi }^{‴}(1-f_j)\right)}{{\sum }_{i=0}^{m-1}\left({\varphi }^{″}\left(f_i\right)+{\varphi }^{″}(1-f_i)\right)f_i(1-f_i)}\hfill \\ & -\frac{d_{m+k}^{\left(\kappa \right)}\left({\varphi }^{‴}\left(f_j\right)-{\varphi }^{‴}(1-f_j)\right)f_j(1-f_j)}{{\sum }_{i=0}^{m-1}\left({\varphi }^{″}\left(f_i\right)+{\varphi }^{″}(1-f_i)\right)f_i(1-f_i)}-\frac{d_{m+k}^{\left(\kappa \right)}(1-2f_j)\left({\varphi }^{″}\left(f_j\right)+{\varphi }^{″}(1-f_j)\right)}{{\sum }_{i=0}^{m-1}\left({\varphi }^{″}\left(f_i\right)+{\varphi }^{″}(1-f_i)\right)f_i(1-f_i)}\hfill \\ \hfill =& \left({\delta }_{j,k}-d_{m+k}^{\left(\kappa \right)}{f}_j(1-f_j)\right)v_j-d_{m+k}^{\left(\kappa \right)}(1-2f_j)u_j\hfill \\ \hfill =& \left({\delta }_{j,k}-f_j(1-f_j)u_k\right)v_j-(1-2f_j)u_j{u}_k.\hfill \end{array}$$

Finally,

$$h_{m+j,m+k}^{\left(\kappa \right)}:=\frac{{\partial }^2}{\partial y_{m+j}\partial y_{m+k}}g\left({\mathit{f}}_{\left(h\right)}\right)=0.$$

This completes the proof of (A2).

Appendix A.3. Proof of Equation (A4)

${\sigma }_{\kappa }^2={\mathbf{D}}_{\kappa }{\Sigma }^{\left(h\right)}{\mathbf{D}}_{\kappa }^{\top }$ is computed as follows:

$$\begin{array}{cc}\hfill {\sigma }_{\kappa }^2=& {\displaystyle \sum _{j,k=0}^{m-1}}\left(d_j^{\left(\kappa \right)}{d}_k^{\left(\kappa \right)}{\sigma }_{j,k}+2d_j^{\left(\kappa \right)}{d}_{m+k}^{\left(\kappa \right)}{\sigma }_{j,m+k}^{\left(h\right)}+d_{m+j}^{\left(\kappa \right)}{d}_{m+k}^{\left(\kappa \right)}{\sigma }_{m+j,m+k}^{\left(h\right)}\right)\hfill \\ \hfill \stackrel{\left(\mathrm{A}2\right)}{=}& {\displaystyle \sum _{j,k=0}^{m-1}}\left(4f_j{f}_k{u}_j{u}_k{\sigma }_{j,k}-4f_j{u}_j{u}_k{\sigma }_{j,m+k}^{\left(h\right)}+u_j{u}_k{\sigma }_{m+j,m+k}^{\left(h\right)}\right)\hfill \\ \hfill \stackrel{\left(17\right)}{=}& {\displaystyle \sum _{j,k=0}^{m-1}}{\displaystyle (}-4f_j{f}_k{u}_j{u}_k\left(f_{min\{j,k\}}-f_j{f}_k\right)+u_j{u}_k\left(f_{min\{j,k\}}+3f_j{f}_k\right)\left(f_{min\{j,k\}}-f_j{f}_k\right){\displaystyle )}\hfill \\ \hfill =& {\displaystyle \sum _{j,k=0}^{m-1}}{u}_j{u}_k{\left(f_{min\{j,k\}}-f_j{f}_k\right)}^2.\hfill \end{array}$$

This completes the proof of (A4).

Appendix A.4. Proof of Equation (A5)

Using that $h_{m+j,m+k}^{\left(\kappa \right)}=0$, we get that

$$\begin{array}{cc}\hfill E\left[{\widehat{\kappa }}_{\varphi }\left(h\right)\right]\approx & 0+\frac{1}{2n}{\displaystyle {\sum }_{j,k=0}^{m-1}}\left(h_{j,k}^{\left(\kappa \right)}{\sigma }_{j,k}+2h_{j,m+k}^{\left(\kappa \right)}{\sigma }_{j,m+k}^{\left(h\right)}\right)\hfill \\ \hfill \stackrel{\left(17\right)}{=}& \frac{1}{2n}{\displaystyle {\sum }_{j,k=0}^{m-1}}{\sigma }_{j,k}\left(h_{j,k}^{\left(\kappa \right)}+4f_k{h}_{j,m+k}^{\left(\kappa \right)}\right)\hfill \\ \hfill \stackrel{\left(\mathrm{A}2\right)}{=}& \frac{1}{2n}{\displaystyle {\sum }_{j,k=0}^{m-1}}{\sigma }_{j,k}(2f_j{f}_k\left((1-f_k)u_j{v}_k+(1-f_j)u_k{v}_j\right)\hfill \\ & +2(f_j+f_k-4f_j{f}_k)u_j{u}_k-2{\delta }_{j,k}(u_j+2f_j{v}_j)\hfill \\ & +4f_k\left({\delta }_{j,k}-f_j(1-f_j)u_k\right)v_j-4f_k(1-2f_j)u_j{u}_k)\hfill \\ \hfill =& \frac{1}{2n}{\displaystyle {\sum }_{j,k=0}^{m-1}}{\sigma }_{j,k}(4f_j{f}_k(1-f_j)u_k{v}_j+4f_k(1-2f_j)u_j{u}_k-2{\delta }_{j,k}{u}_j-4{\delta }_{j,k}{f}_j{v}_j\hfill \\ & +4{\delta }_{j,k}{f}_j{v}_j-4f_k{f}_j(1-f_j)u_k{v}_j-4f_k(1-2f_j)u_j{u}_k)\hfill \\ \hfill =& \frac{1}{2n}{\displaystyle {\sum }_{j,k=0}^{m-1}}{\sigma }_{j,k}\left(0-2{\delta }_{j,k}{u}_j\right)=-\frac{1}{n}{\displaystyle {\sum }_{j=0}^{m-1}}{u}_j{\sigma }_{j,j}=-\frac{1}{n},\hfill \end{array}$$

where the last equality follows from (A1) by using ${\sigma }_{j,j}=f_j(1-f_j)$ according to (17). This completes the proof of (A5).