Fair Numerical Algorithm of Coset Cardinality Spectrum for Distributed Arithmetic Coding

Abstract

As a typical symbol-wise solution of asymmetric Slepian-Wolf coding problem, Distributed Arithmetic Coding (DAC) non-linearly partitions source space into disjoint cosets with unequal sizes. The distribution of DAC coset cardinalities, named the Coset Cardinality Spectrum (CCS), plays an important role in both theoretical understanding and decoder design for DAC. In general, CCS cannot be calculated directly. Instead, a numerical algorithm is usually used to obtain an approximation. This paper first finds that the contemporary numerical algorithm of CCS is theoretically imperfect and does not finally converge to the real CCS. Further, to solve this problem, we refine the original numerical algorithm based on rigorous theoretical analyses. Experimental results verify that the refined numerical algorithm amends the drawbacks of the original version.

1. Introduction

As an important branch of network information theory, Distributed Source Coding (DSC) can find broad potential applications in many scenarios (e.g., wireless sensor network, genome compression, etc.). Just as traditional source coding, DSC has two forms. Lossless DSC is also called Slepian-Wolf Coding (SWC) [1]. The general form of lossy DSC is referred to as Berger-Tung coding [2], while a special case of asymmetric lossy DSC with side information at the decoder is referred to as Wyner-Ziv Coding (WZC) [3]. Up to now, the most important works on SWC have focused on its asymmetric form. Let X and Y be two correlated discrete random variables. Given Y available only at the decoder, the achievable rates of compressing X without loss are bounded by $R\ge H\left(X\right|Y)$. Different from SWC, an important property of WZC is that it usually suffers rate loss when compared to traditional lossy coding with side information available at both the encoder and the decoder. However, if the difference between source and side information is an independent Gaussian random variable, there is no rate loss [4,5].

This paper treats only asymmetric SWC. The asymmetric SWC problem is in essence a channel coding problem [6,7,8,9]. To show this point, one can take Y as a noisy version of X corrupted by virtual channel noise and take the bitstream of X as the index of the coset containing X. If the elements in each coset are spaced as far (in Hamming distance) as possible and Y is near (in Hamming distance) enough to X, then X can be recovered from its bitstream with the help of Y. From this viewpoint, the bitstream of X is actually the syndrome of a coset code. Hence traditionally, asymmetric SWC was implemented by channel codes (e.g., Turbo codes [10], Low-Density Parity-Check (LDPC) codes [11], and polar codes [12], etc.).

Arithmetic coding is usually deemed as the most important method for lossless data compression [13,14]. Due to the duality between source coding and channel coding, arithmetic coding can be easily modified to achieve the purpose of error detection and correction [15,16,17]. In 2007, people found that after some slight modifications, arithmetic coding can also serve as a coset code to implement asymmetric SWC. There are mainly two approaches: One is bit puncturing [18] and the other is interval enlarging [19,20,21]. This paper will focus only on the latter approach, which is often referred to as Distributed Arithmetic Coding (DAC), while ignoring the former. For independent and identically-distributed (i.i.d.) binary sources, DAC is in general inferior to channel codes, e.g., LDPC codes, polar codes, etc., as shown by the experimental results in [22]. However, for binary sources with memory or nonbinary sources, DAC performs significantly better than channel codes [23,24,25,26].

In nature, DAC is a many-to-one nonlinear mapping that partitions source space into disjoint cosets of unequal cardinalities. Then an important problem is how DAC coset cardinality is distributed, which can be answered by the so-called Coset Cardinality Spectrum (CCS). The idea of CCS budded in [27,28] and was formally defined for uniform and nonuniform binary sources in [29,30], respectively. The concept of CCS is very useful as it not only can serve as a theoretical tool to analyze the properties of DAC [31,32], but also can be used to derive correct decoding formulae [31,32] (See the discussion in the last paragraph of Section 2).

If the stream of source symbols is grouped into length-n blocks, then DAC CCS is a tuple of $n+1$ probability density functions (pdfs). It is impossible to deduce the exact closed form for each pdf directly. Instead, one can begin with the final pdf, which is usually simple and calculable, and then derive each pdf via a backward recursion. In [29], a numerical algorithm was proposed to implement the backward recursion for uniform binary sources, and then it was generalized to nonuniform binary sources in [30].

However, the numerical algorithm proposed in [29,30] is very primitive and lacks theoretical justification. This paper will make an in-depth analysis of the numerical algorithm proposed in [29,30] and observe the results carefully. It will be found that the numerical algorithm proposed in [29,30] does not exactly converge to the real CCS. After a strict theoretical analysis, this paper will propose a novel numerical algorithm, which perfectly overcomes the drawbacks of the original numerical algorithm.

The rest of this paper is arranged as below. Section 2 briefly reviews the background knowledge of DAC and CCS. Section 3 presents the original numerical algorithm and its trivial upgrade version. Section 4 proposes the novel numerical algorithm for DAC CCS based on solid theoretical analyses. Section 5 reports some experimental results to compare three numerical algorithms. Finally, Section 6 concludes this paper.

2. Review on DAC and CCS

Let $X^n\triangleq (X_1,\dots ,X_n)$ be a length-n binary source block with bias probability $Pr(X_i=1)=p$. The entropy of X is $H\left(X\right)=-p{log}_{2}p-(1-p){log}_2(1-p)$. The DAC codec recursively maps every source symbol onto an interval in $[0,1)$ according to the following rule

$$\begin{array}{c}\hfill x\in \mathbb{B}\triangleq \{0,1\}\to [x(1-p^r),(1-x){(1-p)}^r+x)\subset [0,1),\end{array}$$

(1)

where $r\in [0,1]$ is called the normalized rate or overlapping factor. Let $[l\left(X^n\right),h\left(X^n\right))\subset [0,1)$ denote the mapping interval of $X^n$. In theory, $[l\left(X^n\right),h\left(X^n\right))$ can be represented by $-{log}_2(h\left(X^n\right)-l\left(X^n\right))\in \mathbb{R}$ bits. However, due to the indivisibility of a bit, the bitstream of $X^n$ actually includes $-\lfloor {log}_2(h\left(X^n\right)-l\left(X^n\right))\rfloor \in \mathbb{Z}$ bits, which can be explained as a real number $U_0$ in $[l\left(X^n\right),h\left(X^n\right))$. Hence, there is a rate loss of $\delta$ bits, where

$$\begin{array}{c}\hfill \delta \triangleq {log}_2(h\left(X^n\right)-l\left(X^n\right))-\lfloor {log}_2(h\left(X^n\right)-l\left(X^n\right))\rfloor \in [0,1).\end{array}$$

(2)

If we decode the bitstream $U_0$ along the path $X^n$, then we will obtain a tuple of $n+1$ real numbers $U_0^n\triangleq (U_0,U_1,\cdots ,U_n)$ which can be deduced recursively. The forward recursion of $U_0^n$ can be easily transformed into an equivalent backward recursion as shown below [30].

$$\begin{array}{c}\hfill \left\{\begin{array}{cc}{U}_{i-1}={(1-p)}^r{U}_i,\hfill & X_i=0\hfill \\ U_{i-1}=p^r{U}_i+(1-p^r),\hfill & X_i=1\hfill \end{array}\right..\end{array}$$

(3)

The pdf of $U_i$ is called the i-th CCS and denoted by $f_{n,i}\left(u\right)$, $0\le u<1$, which is usually simplified as $f_i\left(u\right)$. Especially, $f_0\left(u\right)$ is called the initial CCS and $f_n\left(u\right)$ is called the final CCS.

At this point, we have two choices: One is beginning from the initial CCS $f_0\left(u\right)$ and then deducing $f_i\left(u\right)$ for $0<i\le n$ via a forward recursion; the other is beginning from the final CCS $f_n\left(u\right)$ and then deducing $f_i\left(u\right)$ for $0\le i<n$ via a backward recursion. Unfortunately, it is impossible to deduce the initial CCS $f_0\left(u\right)$ directly, so forward recursion is infeasible. Instead, it is proved in [30] that depending on the parameters p and r, $f_n\left(u\right)$ will tend to be a piecewise uniform function or

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}{f}_n\left(u\right)=\frac{{log}_{2}e}{1+|1-2u|}.\end{array}$$

(4)

Once $f_n\left(u\right)$ is known, $f_i\left(u\right)$ for $0\le i<n$ can be deduced via a backward recursion [30].

$$\begin{array}{c}\hfill f_{i-1}\left(u\right)={(1-p)}^{1-r}{f}_i\left(u{(1-p)}^{-r}\right)+p^{1-r}{f}_i\left((u-(1-p^r))p^{-r}\right).\end{array}$$

(5)

In general, it is impossible to deduce $f_{i-1}\left(u\right)$ from $f_i\left(u\right)$ analytically, even though the closed form of $f_i\left(u\right)$ is known. Hence, we have to resort to a numerical algorithm, which mimics (5) in a discrete way.

Regarding the physical meaning of CCS, there is a detailed explanation in [28]. Let us take uniform binary sources as an example. For any real number $u\in [0,1)$, if u is fed into a DAC decoder with overlapping factor r, then we will get an incomplete binary tree. In this tree, the number of level-i nodes is roughly $\left({lim}_{n\to \infty }{f}_{n,n-i}\right)·2^{i(1-r)}$. If we implement the decoder in a breadth-first way, i.e., with the M-algorithm, for every level-i node, the sub-tree grown from this level-i node will have about $\left({lim}_{n\to \infty }{f}_{n,i}\left(u\right)\right)·2^{(n-i)(1-r)}$ leaf nodes, where u is the real number at the level-i node. Obviously, those level-i nodes with larger $\left({lim}_{n\to \infty }{f}_{n,i}\left(u\right)\right)$ should be more likely, and vice versa. That is why DAC CCS can be used to derive correct decoding formulae.

3. Original Numerical Algorithms

3.1. Rounding Numerical Algorithm

The first version of the numerical algorithm was proposed in [28,30] for uniform and nonuniform binary sources, respectively. This algorithm divides the interval $[0,1)$ into N segments and then uses a finite number of $f_i(j/N)$’s, where $j\in [0:N)\triangleq \{0,\dots ,N-1\}$, to approximate $f_i\left(u\right)$. For simplicity, we use ${\widehat{f}}_i\left(j\right)$ to denote the approximation of $f_i(j/N)$. Then (5) can be discretized as

$$\begin{array}{c}\hfill {\widehat{f}}_{i-1}\left(j\right)={(1-p)}^{1-r}{\widehat{f}}_i\left(j^{\prime }\right)+p^{1-r}{\widehat{f}}_i\left(j^{″}\right).\end{array}$$

(6)

Now the key is to find a good mapping from j to $j^{\prime }$ and $j^{″}$. Let $\lfloor ·\rceil$ denote the rounding operation. This problem was solved by a brute-force method in [28,30] as follows:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{j}^{\prime }=\lfloor j{(1-p)}^{-r}\rceil \hfill \\ j^{″}=\lfloor (j-N(1-p^r))p^{-r}\rceil \hfill \end{array}\right..\end{array}$$

(7)

For some $j\in [0:N)$, we will have $j^{\prime }\notin [0:N)$ or $j^{″}\notin [0:N)$. It does not matter because we have $\widehat{f}\left(j\right)\equiv 0$ for any $j\notin [0:N)$ according to the definition of CCS. Since the core of (7) is the rounding operation, this method will be formally referred to as rounding numerical algorithm below.

Though the rounding numerical algorithm works well at first glance [28,30], its rationality is indeed flawed. On one hand, since ${(1-p)}^{-r}>1$, we have $j{(1-p)}^{-r}\ge j$ and thus among N points of ${\widehat{f}}_i\left(j\right)$’s, only about $N{(1-p)}^r<N$ points, i.e., $0\le j<j_{max}\approx N{(1-p)}^r$, are used for the first line of (7). On the other hand,

$$\begin{array}{cc}\hfill (j-N(1-p^r))p^{-r}& =N-(N-j)p^{-r}\hfill \\ \hfill & =(N-j)-(N-j)p^{-r}+j\hfill \\ \hfill & =(N-j)(1-p^{-r})+j<j,\hfill \end{array}$$

(8)

where the last inequality is due to $N>j$ and $p^{-r}>1$. Thus among N points of ${\widehat{f}}_i\left(j\right)$’s, only about $N{p}^r<N$ points, i.e., $N(1-p^r)\approx j_{min}\le j<N$, are used for the second line of (7). In summary, not all N points of ${\widehat{f}}_i\left(j\right)$’s will be used to generate ${\widehat{f}}_{i-1}\left(j\right)$’s. In other words, some points of ${\widehat{f}}_i\left(j\right)$’s are discarded without being used when we try to deduce ${\widehat{f}}_{i-1}\left(j\right)$ according to (6), which is equivalent to the case that partial information of $f_i\left(u\right)$ is lost when we try to deduce $f_{i-1}\left(u\right)$ according to (5). This is imperfect in theory and may bring two negative effects in practice:

• The rounding numerical algorithm cannot generate an accurate approximation of CCS. This phenomenon will be observed in the experimental results of Section 5.
• Even if ${\sum }_{j=0}^{N-1}{\widehat{f}}_i\left(j\right)=N$, (6) does not strictly satisfy the normalization condition ${\sum }_{j=0}^{N-1}{\widehat{f}}_{i-1}\left(j\right)=N$. Thus an extra re-normalization step is needed after (6).

3.2. Linear Numerical Algorithm

To improve the performance of the rounding numerical algorithm, more points of ${\widehat{f}}_i\left(j\right)$’s should be involved in generating ${\widehat{f}}_{i-1}\left(j\right)$’s. This problem can be solved by linear interpolation. Below we will propose a trivial upgrade of the rounding numerical algorithm, which we will refer to as linear numerical algorithm. Let $\lfloor ·\rfloor$ and $\lceil ·\rceil$ denote the flooring and ceiling operations, respectively. Let us define $\alpha \triangleq j{(1-p)}^{-r}\ge j$ and $\beta \triangleq (j-N(1-p^r))p^{-r}<j$. In general, (6) can be refined as

$$\begin{array}{c}\hfill {\widehat{f}}_{i-1}\left(j\right)={(1-p)}^{1-r}{g}_0\left(\alpha \right)+p^{1-r}{g}_1\left(\beta \right),\end{array}$$

(9)

where

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{g}_0\left(\alpha \right)=(\lceil \alpha \rceil -\alpha ){\widehat{f}}_i\left(\lfloor \alpha \rfloor \right)+(\alpha -\lfloor \alpha \rfloor ){\widehat{f}}_i\left(\lceil \alpha \rceil \right)\hfill \\ g_1\left(\beta \right)=(\lceil \beta \rceil -\beta ){\widehat{f}}_i\left(\lfloor \beta \rfloor \right)+(\beta -\lfloor \beta \rfloor ){\widehat{f}}_i\left(\lceil \beta \rceil \right)\hfill \end{array}\right..\end{array}$$

(10)

Then we discuss some special cases:

• If $\alpha \in \mathbb{Z}$, then $g_0\left(\alpha \right)={\widehat{f}}_i\left(\alpha \right)$. If $\beta \in \mathbb{Z}$, then $g_1\left(\beta \right)={\widehat{f}}_i\left(\beta \right)$.
• If $\lfloor \alpha \rfloor \ge N$, then $g_0\left(\alpha \right)=0$. If $\lceil \beta \rceil <0$, then $g_1\left(\beta \right)=0$.
• If $\lfloor \alpha \rfloor <N$ and $\lceil \alpha \rceil \ge N$, then $g_0\left(\alpha \right)={\widehat{f}}_i\left(\lfloor \alpha \rfloor \right)$. If $\lfloor \beta \rfloor <0$ and $\lceil \beta \rceil \ge 0$, then $g_1\left(\beta \right)={\widehat{f}}_i\left(\lceil \beta \rceil \right)$.

However, it must be pointed out that the linear numerical algorithm still does not guarantee that all N points of ${\widehat{f}}_i\left(j\right)$’s are used to generate ${\widehat{f}}_{i-1}\left(j\right)$’s, and a renormalization step is still needed after (9).

4. Fair Numerical Algorithm

In the rounding/linear numerical algorithms, we take ${\widehat{f}}_i\left(j\right)$ as an approximation of $f_i(j/N)$, i.e., ${\widehat{f}}_i\left(j\right)\approx f_i(j/N)$ for any $j\in [0:N)$, and for each ${\widehat{f}}_{i-1}\left(j\right)$, we try to find the corresponding ${\widehat{f}}_i\left(j^{\prime }\right)$. This understanding is however incorrect. The correct physical meaning of ${\widehat{f}}_i\left(j\right)$ should be an approximation of the scaled-up probability of $U_i$ falling into the interval $[j/N,(j+1)/N))$, i.e.,

$$\begin{array}{c}\hfill {\widehat{f}}_i\left(j\right)\approx N{\int }_{j/N}^{(j+1)/N}{f}_i\left(u\right)du.\end{array}$$

(11)

According to (5), we have

$$\begin{array}{c}\hfill {\int }_{j/N}^{(j+1)/N}{f}_{i-1}\left(u\right)du={\int }_{j/N}^{(j+1)/N}\left({(1-p)}^{1-r}{f}_i\left(u{(1-p)}^{-r}\right)+p^{1-r}{f}_i\left((u-(1-p^r))p^{-r}\right)\right)du.\end{array}$$

(12)

Let $u^{\prime }\triangleq u{(1-p)}^{-r}$ and $u^{″}\triangleq (u-(1-p^r))p^{-r}$. Let

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{\mathcal{I}}_0\triangleq [j{(1-p)}^{-r}/N,(j+1){(1-p)}^{-r}/N)\hfill \\ {\mathcal{I}}_1\triangleq [(j-N(1-p^r))p^{-r}/N,(j+1-N(1-p^r))p^{-r}/N)\hfill \end{array}\right..\end{array}$$

(13)

Then we can obtain

$$\begin{array}{cc}\hfill {\int }_{j/N}^{(j+1)/N}{f}_{i-1}\left(u\right)du& =(1-p){\int }_{u^{\prime }\in {\mathcal{I}}_0}{f}_i\left(u^{\prime }\right)d{u}^{\prime }+p{\int }_{u^{″}\in {\mathcal{I}}_1}{f}_i\left(u^{″}\right)d{u}^{″}\hfill \\ \hfill & =(1-p){\int }_{u\in {\mathcal{I}}_0}{f}_i\left(u\right)du+p{\int }_{u\in {\mathcal{I}}_1}{f}_i\left(u\right)du.\hfill \end{array}$$

(14)

Naturally, we have

$$\begin{array}{c}\hfill {\widehat{f}}_{i-1}\left(j\right)=(1-p)·g_0\left(j\right)+p·g_1\left(j\right),\end{array}$$

(15)

where

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{g}_0\left(j\right)\approx N{\int }_{u\in {\mathcal{I}}_0}{f}_i\left(u\right)du\hfill \\ g_1\left(j\right)\approx N{\int }_{u\in {\mathcal{I}}_1}{f}_i\left(u\right)du\hfill \end{array}\right..\end{array}$$

(16)

In plain words, $g_0\left(j\right)$ is an approximation of the scaled-up probability of $U_i$ falling into the interval ${\mathcal{I}}_0$, and $g_1\left(j\right)$ is an approximation of the scaled-up probability of $U_i$ falling into the interval ${\mathcal{I}}_1$.

4.1. Calculation of $g_0\left(j\right)$

Let ${\lambda }_0\triangleq j{(1-p)}^{-r}\ge 0$ and ${\eta }_0\triangleq (j+1){(1-p)}^{-r}>{\lambda }_0\ge 0$. Then we can obtain

$$\begin{array}{c}\hfill N{\int }_{u\in {\mathcal{I}}_0}{f}_i\left(u\right)du=N{\int }_{{\lambda }_0/N}^{(\lfloor {\lambda }_0\rfloor +1)/N}{f}_i\left(u\right)du+N{\int }_{(\lfloor {\lambda }_0\rfloor +1)/N}^{(\lfloor {\lambda }_0\rfloor +2)/N}{f}_i\left(u\right)du+\cdots \\ \hfill +N{\int }_{(\lfloor {\eta }_0\rfloor -1)/N}^{\lfloor {\eta }_0\rfloor /N}{f}_i\left(u\right)du+N{\int }_{\lfloor {\eta }_0\rfloor /N}^{{\eta }_0/N}{f}_i\left(u\right)du.\end{array}$$

(17)

For N sufficiently large, $f_i\left(u\right)$ can be taken as uniform over $[j/N,(j+1)/N)$. Hence we have

$$\begin{array}{cc}\hfill N{\int }_{{\lambda }_0/N}^{(\lfloor {\lambda }_0\rfloor +1)/N}{f}_i\left(u\right)du& \approx (\lfloor {\lambda }_0\rfloor +1-{\lambda }_0)·N{\int }_{\lfloor {\lambda }_0\rfloor /N}^{(\lfloor {\lambda }_0\rfloor +1)/N}{f}_i\left(u\right)du\hfill \\ \hfill & \approx (\lfloor {\lambda }_0\rfloor +1-{\lambda }_0)·{\widehat{f}}_i\left(\lfloor {\lambda }_0\rfloor \right)\hfill \end{array}$$

(18)

and

$$\begin{array}{cc}\hfill N{\int }_{\lfloor {\eta }_0\rfloor /N}^{{\eta }_0/N}{f}_i\left(u\right)du& \approx ({\eta }_0-\lfloor {\eta }_0\rfloor )·N{\int }_{\lfloor {\eta }_0\rfloor /N}^{(\lfloor {\eta }_0\rfloor +1)/N}{f}_i\left(u\right)du\hfill \\ \hfill & \approx ({\eta }_0-\lfloor {\eta }_0\rfloor )·{\widehat{f}}_i\left(\lfloor {\eta }_0\rfloor \right).\hfill \end{array}$$

(19)

According to the above analysis, we can obtain the following results.

• $N\le {\lambda }_0<{\eta }_0$: It is easy to know $g_0\left(j\right)\equiv 0$.
• ${\lambda }_0<N\le {\eta }_0$: In general, we have

  $$\begin{array}{c}\hfill g_0\left(j\right)=(1-({\lambda }_0-\lfloor {\lambda }_0\rfloor ))·{\widehat{f}}_i\left(\lfloor {\lambda }_0\rfloor \right)+\sum _{j^{\prime }=\lfloor {\lambda }_0\rfloor +1}^{N-1}{\widehat{f}}_i\left(j^{\prime }\right).\end{array}$$

  (20)
  Especially, if ${\lambda }_0\in \mathbb{Z}$, then

  $$\begin{array}{c}\hfill g_0\left(j\right)=\sum _{j^{\prime }={\lambda }_0}^{N-1}{\widehat{f}}_i\left(j^{\prime }\right).\end{array}$$

  (21)
• ${\lambda }_0<{\eta }_0<N$: In general, we have

  $$\begin{array}{c}\hfill g_0\left(j\right)=(1-({\lambda }_0-\lfloor {\lambda }_0\rfloor ))·{\widehat{f}}_i\left(\lfloor {\lambda }_0\rfloor \right)+({\eta }_0-\lfloor {\eta }_0\rfloor )·{\widehat{f}}_i\left(\lfloor {\eta }_0\rfloor \right)+\sum _{j^{\prime }=\lfloor {\lambda }_0\rfloor +1}^{\lfloor {\eta }_0\rfloor -1}{\widehat{f}}_i\left(j^{\prime }\right).\end{array}$$

  (22)
  Let us consider three special cases:

  —

  If ${\lambda }_0\in \mathbb{Z}$ and ${\eta }_0\notin \mathbb{Z}$, then

  $$\begin{array}{c}\hfill g_0\left(j\right)=({\eta }_0-\lfloor {\eta }_0\rfloor )·{\widehat{f}}_i\left(\lfloor {\eta }_0\rfloor \right)+\sum _{j^{\prime }={\lambda }_0}^{\lfloor {\eta }_0\rfloor -1}{\widehat{f}}_i\left(j^{\prime }\right).\end{array}$$

  (23)

  —

  If ${\lambda }_0\notin \mathbb{Z}$ and ${\eta }_0\in \mathbb{Z}$, then

  $$\begin{array}{c}\hfill g_0\left(j\right)=(\lceil {\lambda }_0\rceil -{\lambda }_0)·{\widehat{f}}_i\left(\lfloor {\lambda }_0\rfloor \right)+\sum _{j^{\prime }=\lceil {\lambda }_0\rceil }^{{\eta }_0-1}{\widehat{f}}_i\left(j^{\prime }\right).\end{array}$$

  (24)

  —

  If ${\lambda }_0\in \mathbb{Z}$ and ${\eta }_0\in \mathbb{Z}$, then

  $$\begin{array}{c}\hfill g_0\left(j\right)=\sum _{j^{\prime }={\lambda }_0}^{{\eta }_0-1}{\widehat{f}}_i\left(j^{\prime }\right).\end{array}$$

  (25)

4.2. Calculation of $g_1\left(j\right)$

Let ${\eta }_1\triangleq (j+1-N(1-p^r))p^{-r}\le N$ and ${\lambda }_1\triangleq (j-N(1-p^r))p^{-r}<{\eta }_1\le N$. Similarly, we can get the following results.

• ${\lambda }_1<{\eta }_1<0$: It is easy to know $g_1\left(j\right)\equiv 0$.
• ${\lambda }_1<0\le {\eta }_1$: In general, we have

  $$\begin{array}{c}\hfill g_1\left(j\right)=({\eta }_1-\lfloor {\eta }_1\rfloor )·{\widehat{f}}_i\left(\lfloor {\eta }_1\rfloor \right)+\sum _{j^{\prime }=0}^{\lfloor {\eta }_1\rfloor -1}{\widehat{f}}_i\left(j^{\prime }\right).\end{array}$$

  (26)
  Especially, if ${\eta }_1\in \mathbb{Z}$, then

  $$\begin{array}{c}\hfill g_1\left(j\right)=\sum _{j^{\prime }=0}^{{\eta }_1-1}{\widehat{f}}_i\left(j^{\prime }\right).\end{array}$$

  (27)
• $0\le {\lambda }_1<{\eta }_1$: In general, we have

  $$\begin{array}{c}\hfill g_1\left(j\right)=(1-({\lambda }_1-\lfloor {\lambda }_1\rfloor ))·{\widehat{f}}_i\left(\lfloor {\lambda }_1\rfloor \right)+({\eta }_1-\lfloor {\eta }_1\rfloor )·{\widehat{f}}_i\left(\lfloor {\eta }_1\rfloor \right)+\sum _{j^{\prime }=\lfloor {\lambda }_1\rfloor +1}^{\lfloor {\eta }_1\rfloor -1}{\widehat{f}}_i\left(j^{\prime }\right).\end{array}$$

  (28)
  Let us consider three special cases:

  —

  If ${\lambda }_1\in \mathbb{Z}$ and ${\eta }_1\notin \mathbb{Z}$, then

  $$\begin{array}{c}\hfill g_1\left(j\right)=({\eta }_1-\lfloor {\eta }_1\rfloor )·{\widehat{f}}_i\left(\lfloor {\eta }_1\rfloor \right)+\sum _{j^{\prime }={\lambda }_1}^{\lfloor {\eta }_1\rfloor -1}{\widehat{f}}_i\left(j^{\prime }\right).\end{array}$$

  (29)

  —

  If ${\lambda }_1\notin \mathbb{Z}$ and ${\eta }_1\in \mathbb{Z}$, then

  $$\begin{array}{c}\hfill g_1\left(j\right)=(\lceil {\lambda }_1\rceil -{\lambda }_1)·{\widehat{f}}_i\left(\lfloor {\lambda }_1\rfloor \right)+\sum _{j^{\prime }=\lceil {\lambda }_1\rceil }^{{\eta }_1-1}{\widehat{f}}_i\left(j^{\prime }\right).\end{array}$$

  (30)

  —

  If ${\lambda }_1\in \mathbb{Z}$ and ${\eta }_1\in \mathbb{Z}$, then

  $$\begin{array}{c}\hfill g_1\left(j\right)=\sum _{j^{\prime }={\lambda }_1}^{{\eta }_1-1}{\widehat{f}}_i\left(j^{\prime }\right).\end{array}$$

  (31)

4.3. Discussion

From the above analysis, it can be found that all N points of ${\widehat{f}}_i\left(j\right)$’s are made use of to generate ${\widehat{f}}_{i-1}\left(j\right)$’s, and such treatment is fair for every ${\widehat{f}}_i\left(j\right)$. For this reason, we will formally refer to this method as fair numerical algorithm. Given ${\sum }_{j=0}^{N-1}{\widehat{f}}_i\left(j\right)=N$, it is easy to know ${\sum }_{j=0}^{N-1}{g}_0\left(j\right)={\sum }_{j=0}^{N-1}{g}_1\left(j\right)=N$. Hence, (15) satisfies the normalization property by itself and no renormalization step is needed after (15).

Let us briefly discuss the convergence of numerical algorithms. Let $f\left(u\right)$ denote the asymptotic form of $f_{n,0}\left(u\right)$ as $n\to \infty$. According to (5), it is obvious that

$$\begin{array}{c}\hfill f\left(u\right)={(1-p)}^{1-r}f\left(u{(1-p)}^{-r}\right)+p^{1-r}f\left((u-(1-p^r))p^{-r}\right).\end{array}$$

(32)

For the rounding/linear numerical algorithms, as analyzed above, partial parts of $f\left(u\right)$ have been lost, so the discrete version of (32) will not exactly hold, while for the fair numerical algorithm, since all information of $f\left(u\right)$ is reserved, the discrete version of (32) will exactly hold as the number of segments N goes to infinity.

5. Experimental Results

We use a classical CCS to compare three numerical algorithms. Given $p=r=0.5$, for n sufficiently large, as i increases, $f_{n-i}$ will tend to be ladder-shaped and we have [27,28].

$$\begin{array}{c}\hfill \underset{i\to \infty }{lim}{f}_{\infty -i}\left(u\right)=\left\{\begin{array}{cc}\frac{u}{3\sqrt{2}-4},\hfill & 0\le u<\sqrt{2}-1\hfill \\ \frac{1}{2-\sqrt{2}},\hfill & \sqrt{2}-1\le u<2-\sqrt{2}\hfill \\ \frac{1-u}{3\sqrt{2}-4},\hfill & 2-\sqrt{2}\le u<1\hfill \end{array}\right..\end{array}$$

(33)

For simplicity, we set the final CCS $f_n\left(u\right)=\Pi \left(u\right)$, where $\Pi \left(u\right)$ is a uniform function over $[0,1)$. Some results are included in Figure 1, Figure 2 and Figure 3. In these figures, we set $n=128$ and $N=1024$. It can be observed that for small i (e.g., 1, 2, and 3), the rounding numerical algorithm performs almost as well as the fair numerical algorithm, while the linear numerical algorithm performs very poorly as there are some big spikes. When $i=8$, small spikes are also observed for the rounding numerical algorithm, but no spike is observed for the fair numerical algorithm. Finally, for large i (e.g., 16, 32, and 64), the resulting curves of the rounding numerical algorithm always fluctuate slightly along the real CCS given by (33), and more experiments show that there is no trend that these curves will finally converge to the real CCS as i increases. On the contrary, for both the linear numerical algorithm and the fair numerical algorithm, their curves coincide with the real CCS very well for large i, and there is a trend that these curves will finally converge to the real CCS as i increases.

6. Conclusions

As an important property of DAC, CCS finds its broad applications in many scenarios. However, CCS is usually incalculable so we have to resort to numerical algorithms. This paper finds that the original numerical algorithm proposed in our previous papers is not sound in theory and does not work well in practice (fails to generate an accurate approximation of CCS). Based on a strict theoretical analysis, this paper proposes a novel numerical algorithm that overcomes the weaknesses of the original numerical algorithm. The superiority of the newly-proposed numerical algorithm is well validated by experimental results.

A software package of source codes to reproduce the experimental results in this paper has been released in [33].