Numerical and Non-Asymptotic Analysis of Elias’s and Peres’s Extractors with Finite Input Sequences ^†

Abstract

Many cryptographic systems require random numbers, and the use of weak random numbers leads to insecure systems. In the modern world, there are several techniques for generating random numbers, of which the most fundamental and important methods are deterministic extractors proposed by von Neumann, Elias, and Peres. Elias’s extractor achieves the optimal rate (i.e., information-theoretic upper bound) $h\left(p\right)$ if the block size tends to infinity, where $h(·)$ is the binary entropy function and p is the probability that each bit of input sequences occurs. Peres’s extractor achieves the optimal rate $h\left(p\right)$ if the length of the input and the number of iterations tend to infinity. Previous research related to both extractors has made no reference to practical aspects including running time and memory size with finite input sequences. In this paper, based on some heuristics, we derive a lower bound on the maximum redundancy of Peres’s extractor, and we show that Elias’s extractor is better than Peres’s extractor in terms of the maximum redundancy (or the rates) if we do not pay attention to the time complexity or space complexity. In addition, we perform numerical and non-asymptotic analysis of both extractors with a finite input sequence with any biased probability under the same environments. To do so, we implemented both extractors on a general PC and simple environments. Our empirical results show that Peres’s extractor is much better than Elias’s extractor for given finite input sequences under a very similar running time. As a consequence, Peres’s extractor would be more suitable to generate uniformly random sequences in practice in applications such as cryptographic systems.

1. Introduction

Many cryptographic systems require random numbers, and the use of weak random numbers leads to insecure systems. In fact, many past security problems were due to the use of weak random numbers [1,2,3,4]. This tells us that random number generation is very important in cryptography, in particular to ensure that secret keys are random and unpredictable. In the modern world, there are several techniques for generating random numbers. A natural source such as physical phenomena, the stock market, or Bitcoin [5] can produce unpredictable random sequences, although such sequences are not uniformly random at the source (i.e., biased). However, there is a solution to solve this problem, namely, to use deterministic extractors. A deterministic extractor is a function which takes a non-uniformly random sequence as input and outputs a uniformly random sequence. The deterministic extractors have been studied in mathematics, information theory, and cryptography. In information theory, those extractors can also be treated for the intrinsic randomness problem (i.e., the problem of generating truly random numbers). Furthermore, as applications in cryptography, the output sequence of those extractors can be used as secret keys in information-theoretic cryptography or symmetric key cryptography. The extractors by von Neumann [6], Elias [7], and Peres [8] are fundamental and important ones. In particular, Elias’s and Peres’s extractors are interesting, since they can achieve the optimal rate (or redundancy), if we suppose that input size tends to infinity (i.e., from an asymptotic viewpoint). However, it is not easy to conclude which one is better, since those are constructed by completely different approaches. The main purpose of this paper is to investigate those with finite inputs (i.e., from a non-asymptotic viewpoint) by numerical analysis to make it clear which is better for practical use.

1.1. Related Work

Several works have proposed methods for extracting uniform random sequences from non-uniform random sequences. The most famous among them is von Neumann’s extractor [6] proposed in 1951. He demonstrated a simple procedure for extracting independent unbiased bits from a sequence of independent, identically distributed (i.i.d.) and biased bits. The technique by von Neumann guarantees that the output sequences are independent and uniform if the input sequence is independent and constantly biased, while this cannot be guaranteed if the bias is not constant (e.g., see [9]).

An improved algorithm of von Neumann’s extractor was proposed by Elias [7] in 1971. Elias’s extractor utilizes a block coding technique to improve the rate (or redundancy) of von Neumann’s extractor; however, the straightforward implementation of this extractor requires exponential time and exponential memory size with respect to N, where N is the block size, to store all $2^N$ input sequences with their assignment of output sequences. In 2000, Ryabko and Matchikina [10] proposed an extension of Elias’s extractor that improved time complexity and space complexity by using the enumerative encoding technique from [11] and the Schönhage–Strassen algorithm [12] for fast integer multiplication in order to compute the assignment of output sequences. In this paper, we call this improved method the RM method.

Peres’s extractor is another extended algorithm of von Neumann’s extractor. In 1992, Peres [8] proposed a procedure which improved upon von Neumann’s extractor. The basic idea of Peres’s extractor is to reuse the discarded bits in von Neumann’s extractor by iterating similar procedures in von Neumann’s extractor.

The extractors by von Neumann, Elias, and Peres are the most fundamental and important ones using a single source. In particular, Elias’s and Peres’s extractors are interesting, since they can achieve the optimal rate (i.e., information-theoretic upper bound) $h\left(p\right)$ if input size tends to infinity (i.e., in an asymptotic case), where each bit of input sequences from a single source occurs with probability $p\in (0,1)$ and $h(·)$ is the binary entropy function. In this paper, we are interested in the non-asymptotic case, namely, the achievable rate for finite input-sizes. The rate of Elias’s extractor for finite input-sizes can be observed in the work [7], but the rate of Peres’s extractor for finite input-sizes is not explicitly known. As a work related to Peres’s extractor, Pae [13] reported a recursion formula to compute the rate for finite input-sizes, but it is difficult to give the rate function with finite input-sizes since the recursion formula is complicated. Pae also computed the rate by the recursion formula in the case $p=1/3$, compared the rates of Peres’s extractor and Elias’s extractor, and concluded that the rate of Peres’s extractor increased much slower than that of Elias’s extractor via numerical analysis. However, it is not explicitly known which extractor is better to use in practice, if we take into account the running time, implementation cost, and memory size required in the extractors, as mentioned in [13].

There are several works for constructing extractors using multiple sources (i.e., not a single source). Bourgain [14] provided a 2-source extractor under the condition that the two sources are independent and each source has min-entropy $0.499n$, where n is the bit-length of the output of the sources. Raz [15] proposed an improvement in terms of total min-entropy, and constructed 2-source extractors with the condition that one source has min-entropy more than $n/2$ and the other source requires min-entropy $O(logn)$. In 2015, Cohen [16] constructed a 3-source extractor, where one source has min-entropy $\delta n$, the second source has min-entropy $O(logn)$ and the third source has min-entropy $O(loglogn)$. In 2016, Chattopadhyay and Zuckerman [17] proposed a general 2-source extractor, where each source has a polylogarithmic min-entropy. They combined two weak random sequences into a single sequence by using K-Ramsey graphs and resilient functions. Their extractor has only one-bit output and achieves negligible error and higher complexity than Peres’s extractor or Elias’s extractor.

Furthermore, there are various reports about extracting random bits in the real world. In particular, in 2009, Bouda et al. [18] used mobile phones or pocket computers to generate random data that is close to truly random data. They took 12 pictures per second then used their function to obtain four random bits in each picture, and then applied Carter–Wegman universal${}_2$ hash functions. Halprin and Naor [19] presented the idea of using human game-play as a randomness source in 2009. They constructed the Hide and Seek game that produced approximately 17 bits of raw data per click, and then generated with a pairwise independent hash function a 128-bit string which is $2^{64}$-close to the random one in less than two minutes. In 2011, Voris et al. [20] investigated the use of accelerators on the RFID tags as a source. They implemented a two-stage extractor on the RFID tags. It can produce 128 random bits in 1.5 s by storing a Toeplitz matrix on the RFID tags and performing matrix multiplications.

1.2. Our Contribution

In this paper, we revisit the extractors by von Neumann, Elias, and Peres, since they are fundamental and only require a single source. In the studies on those extractors, it is normal to asymptotically analyze the rate or redundancy of the extractors in the literature, where the rate is the average bit-length of outputs per bit of input (see Section 2 for details). Specifically, the rate of von Neumann’s extractor is $p(1-p)$ that is far from the optimal rate (i.e., information-theoretic upper bound) $h\left(p\right)$. Meanwhile, the rate of Elias’s extractor converges to $h\left(p\right)$ if the block size tends to infinity. Specifically, Elias’s extractor outputs a uniformly random sequence at a high rate, when it takes a long block size equal to the input length. However, it has a trade-off between the rates and computational resources such as time complexity and memory size. On the other hand, Peres’s extractor achieves the optimal rate $h\left(p\right)$ if the length of input and the number of iterations tend to infinity, and it requires smaller time complexity and memory size. However, it would be hard to explicitly derive the exact rate for finite input sequences. Thus, it is not easy to conclude which is a more suitable extractor for practical use in general. Among related work, only one, by Pae [13], compared both extractors as mentioned in Section 1.1, but it does not completely answer the question, since it analyzed the performance of both extractors only for restricted parameters, in particular, the case where each bit of input sequences occurs with probability $p=1/3$ and did not consider the running time. In this paper, we will perform non-asymptotic analysis for the wide range of parameters for Elias’s and Peres’s extractors, to answer the following question: which is more suitable for practical use in real-world applications? To do this, we evaluate the numerical performance of Peres’s extractor and Elias’s extractor with the RM method in terms of practical aspects including achievable rates (or redundancy) and running time with finite input sequences. Specifically, the contribution of this paper is as follows:

(i)

Based on some heuristics, we derive a lower bound on the maximum redundancy of Peres’s extractor in Section 3. This result shows that the maximum redundancy of Elias’s extractor is superior to Peres’s extractor in general, if we focus only on redundancy (or rates) and we do not pay attention to the time complexity or space complexity.

(ii)

By numerical analysis, we design our experiments by comparing both extractors with finite input sequences of which each bit occurs with any biased probability $p\in (0,1)$ under the same environments in terms of practical aspects. Both extractors are implemented on a general PC and do not require any special resources, libraries, or frameworks for computation. Our implementation and results will be explained in Section 4. We calibrate our implementation by comparing the theoretical and experimental redundancy of both extractors. Afterwards, we analyze the time complexity of both extractors with respect to the bit-length of input sequences from 100 to 5000. We compare the redundancy of both extractors, and our implementation shows that Peres’s extractor is much better than Elias’s extractor under a very similar running time. As a result, Peres’s extractor would be more suitable for generating uniformly random sequences for practical use in applications.

The primary version of this paper appeared in CISS2017 [21], and this paper is an extended and full version of it. The difference between the primary version [21] and this paper is as follows: This paper contains the above result (i) in Section 3, and reports more detailed implementation results for (ii) in Section 4. In particular, we implemented and confirmed the results (ii) at a larger scale (see Section 4 for details) in addition to obtaining new figures in Section 4.1.

2. Preliminaries

Throughout this paper, we assume that $log(·):={log}_2(·)$ and $ln(·):={log}_e(·)$, and we define $0log0:=0$. $h(·)$ is the binary entropy function defined by $h\left(p\right)=-plogp-(1-p)log(1-p)$ for $p\in [0,1]$. Let $\left(\genfrac{}{}{0pt}{}{n}{k}\right)$ be a binomial coefficient defined by $\left(\genfrac{}{}{0pt}{}{n}{k}\right):=\frac{n(n-1)(n-2)\cdots (n-k+1)}{k(k-1)(k-2)\cdots 1}$ for nonnegative integers n and k, and $\left(\genfrac{}{}{0pt}{}{n}{0}\right):=1$ for any $n\ge 0$ (see [22] for an extension of the traditional definition of binomial coefficients). Note that $\left(\genfrac{}{}{0pt}{}{n}{k}\right)>0$ if $k\le n$, and $\left(\genfrac{}{}{0pt}{}{n}{k}\right)=0$ if $k>n$.

The first deterministic extractor was constructed by von Neumann [6] in 1951, and later improved ones were proposed by Elias [7] in 1971, and by Peres [8] in 1992. The prior work [6,7,8] considered Bernoulli source Bern$\left(p\right)$ from which input sequences were generated, namely Bern$\left(p\right)$ outputs i.i.d. $(x_1,x_2,\dots ,x_n)\in {\{0,1\}}^n$ according to $Pr(x_i=1)=p$ and $Pr(x_i=0)=q=1-p$ for some unknown $p\in (0,1)$.

A deterministic extractor A takes $(x_1,x_2,\dots ,x_n)$ $\in {\{0,1\}}^n$ as input and outputs $(y_1,y_2,\dots ,y_{\ell })\in {\{0,1\}}^{\ell }$, and its average bit-length of output is denoted by $\overline{\ell }\left(n\right)$ which is a function of n, and defines its rate function by $r^{\mathsf{A}}\left(p\right):={lim}_{n\to \infty }\overline{\ell }\left(n\right)/n$. Additionally, for a deterministic extractor A, we define the redundancy function by $f^{\mathsf{A}}\left(p\right):=h\left(p\right)-r^{\mathsf{A}}\left(p\right)$, and the maximum redundancy by $\mathsf{\Gamma }:={sup}_{p\in (0,1)}{f}^{\mathsf{A}}\left(p\right)$. Note that the above definition of redundancy functions is meaningful, since $h\left(p\right)$ is shown to be the information-theoretic upper bound of the extractors in [7,8]. Furthermore, in this paper we define a non-asymptotic rate function $r^{\mathsf{A}}(p,n):=\overline{\ell }\left(n\right)/n$, a non-asymptotic redundancy function $f^{\mathsf{A}}(p,n):=h\left(p\right)-r^{\mathsf{A}}(p,n)$, and the non-asymptotic maximum redundancy $\mathsf{\Gamma }\left(n\right):={sup}_{p\in (0,1)}{f}^{\mathsf{A}}(p,n)$, which will be used in our non-asymptotic analysis.

2.1. Von Neumann’s Extractor

Von Neumann’s extractor was a simple algorithm for extracting independent unbiased bits from biased bits. This algorithm divides the input sequences $(x_1,x_2,x_3,x_4,\dots ,x_n)$ into the pairs (If n is odd, we discard the last bit.) $(\left(x_1{x}_2\right),\left(x_3{x}_4\right),\dots )$ and maps each pair with a mapping as follows:

$$00\mapsto \wedge ,01\mapsto 0,10\mapsto 1,11\mapsto \wedge ,$$

(1)

where ∧ means no output was generated. After that, it concatenates all resulting outputs of (1). To facilitate understanding, we give an example as follows.

Example 1.

Suppose that an input sequence is $(x_1,x_2,x_3,$ $\dots ,x_8)=(1,0,0,1,$ $0,0,1,1)$. Firstly, divide it into the pairs as $\left(\right(1,0),(0,1),(0,0),(1,1\left)\right)$. Next, map each pair with the mapping (1). Finally, the extractor outputs $(y_1,y_2)=(1,0)$.

Complexity: Von Neumann’s extractor is efficient in the sense that both time complexity and space complexity are small such that time complexity is evaluated as $O\left(n\right)$, and space complexity is evaluated as $O\left(1\right)$.

Redundancy: Von Neumann’s extractor is not desirable, since the maximum redundancy is far from zero. In fact, the rate function $r^{\mathsf{vN}}\left(p\right)$ of von Neumann’s extractor is evaluated by $r^{\mathsf{vN}}\left(p\right)={lim}_{n\to \infty }np(1-p)/n=p(1-p)$, which is $1/4$ at $p=1/2$ and less elsewhere. In addition, the (non-asymptotic) rate functions, (non-asymptotic) redundancy functions, and the (non-asymptotic) maximum redundancy is evaluated as follows: $f^{\mathsf{vN}}(p,n)=f^{\mathsf{vN}}\left(p\right)=h\left(p\right)-p(1-p)$, ${\mathsf{\Gamma }}^{\mathsf{vN}}\left(n\right)={\mathsf{\Gamma }}^{\mathsf{vN}}=3/4$.

2.2. Elias’s Extractor

Elias [7] improved von Neumann’s extractor by using a block coding technique in 1971. Let $N\in \mathbb{N}$ $(N\ge 2)$ be the block size in Elias’s extractor. For all binary sequences with bit-length N, partition them into $N+1$ sets $S_k$ ($k=0,1,2,\dots ,N$), where $S_k$ consists of all the $\left(\genfrac{}{}{0pt}{}{N}{k}\right)$ sequences of length N which have k ones and $N-k$ zeros. Here, we note that each sequence of $S_k$ is equiprobable, i.e., the probability $p^k{q}^{N-k}$.

We consider binary representation of the nonnegative integer $|S_k|=\left(\genfrac{}{}{0pt}{}{N}{k}\right)$ as follows: $\left(\genfrac{}{}{0pt}{}{N}{k}\right)={\alpha }_m{2}^m+{\alpha }_{m-1}{2}^{m-1}+\dots +{\alpha }_0{2}^0$, where $m=\lfloor log\left(\genfrac{}{}{0pt}{}{N}{k}\right)\rfloor$, ${\alpha }_j\in \{0,1\}$, and ${\alpha }_m=1$. In this case, we briefly write $|S_k|=\left(\genfrac{}{}{0pt}{}{N}{k}\right)=({\alpha }_m,{\alpha }_{m-1},\dots ,{\alpha }_0)$. For each j ($1\le j\le m$) such that ${\alpha }_j=1$, we assign $2^j$ distinct output sequences of length j to $2^j$ distinct sequences of $S_k$ which have not already been assigned. If ${\alpha }_0=1$, one sequence of $S_k$ is assigned to ∧. In particular, since $|S_0|=|S_N|=1$, two sequences $(0,0,\dots ,0)$ and $(1,1,\dots ,1)$ are assigned to ∧. For instance, we show a procedure of Elias’s extractor in Example 2.

Example 2.

Suppose that the given input sequence $x=(1,0,0,1,0,0,1,1)$ with block size $N=4$, is the same as in Example 1. Firstly, we partition the set ${\{0,1\}}^4$ of possible input sequences into the following subsets:

$$\begin{array}{cc}\hfill S_0& =\left\{\right(0,0,0,0\left)\right\},\hfill \\ \hfill S_1& =\left\{\right(1,0,0,0),(0,1,0,0),(0,0,1,0),(0,0,0,1\left)\right\},\hfill \\ \hfill S_2& =\left\{\right(0,0,1,1),(0,1,0,1),(0,1,1,0),(1,1,0,0),(1,0,1,0),(1,0,0,1\left)\right\},\hfill \\ \hfill S_3& =\left\{\right(1,1,1,0),(1,0,1,1),(1,1,0,1),(0,1,1,1\left)\right\},\hfill \\ \hfill S_4& =\left\{\right(1,1,1,1\left)\right\}.\hfill \end{array}$$

Then, we have $|S_0|=|S_4|$ $=1=\left(1\right),|S_1|=|S_3|=4=(1,0,0),|S_2|=6=(1,1,0)$. We consider the following assignment of output sequences:

$$\begin{array}{ccc}\hfill & (0,0,0,0)\mapsto \wedge ,\hfill & (1,1,1,1)\mapsto \wedge ,\hfill \\ \hfill & (1,0,0,0)\mapsto (0,0),\hfill & (1,1,1,0)\mapsto (0,0),\hfill \\ \hfill & (0,1,0,0)\mapsto (0,1),\hfill & (1,0,1,1)\mapsto (1,0),\hfill \\ \hfill & (0,0,1,0)\mapsto (1,0),\hfill & (1,1,0,1)\mapsto (1,1),\hfill \\ \hfill & (0,0,0,1)\mapsto (1,1),\hfill & (0,1,1,1)\mapsto (0,1),\hfill \\ \hfill & (0,0,1,1)\mapsto (0,1),\hfill & (1,0,1,0)\mapsto (1,0),\hfill \\ \hfill & (0,1,1,0)\mapsto (0,0),\hfill & (1,0,0,1)\mapsto (1,1),\hfill \\ \hfill & (0,1,0,1)\mapsto \left(0\right),\hfill & (1,1,0,0)\mapsto \left(1\right).\hfill \end{array}$$

Suppose that an input sequence $x=(1,0,0,1,0,0,1,1)$ is given. Since the block size $N=4$, the sequence is divided as $x=\left(\right(1,0,0,1),(0,0,1,1\left)\right)$. By the above assignment of output sequences, the output sequence is $y=\left(\right(1,1\left)\right(0,1\left)\right)=(1,1,0,1)$. Furthermore, there are several ways to assign $m_k$ bits to binary output sequences with the same probability that affect the output sequence y. Thus, the output sequence of 10010011 will not be 1101, if we use another assignment. Note that Elias’s extractor with block size $N=2$ is equivalent to von Neumann’s extractor, or equivalently the mapping (1). In this sense, Elias’s extractor is an extension of von Neumann’s extractor.

Complexity: It can be seen that the straightforward implementation of Elias’s extractor requires much space and time complexity to make a table of the assignment of output sequences, as illustrated by Example 2. Specifically, it requires exponential time and exponential memory size with respect to N to store all $2^N$ binary sequences with their assignment of output sequences. To reduce the time and space complexity of Elias’s extractor, Ryabko and Matchikina [10] proposed a method that is extended from Elias’s extractor, which we call the RM method in this paper. The RM method utilizes the enumerative encoding technique from [11] and the Schönhage–Strassen algorithm [12] for fast integer multiplication in order to compute the assignment of output sequences without making the table large. The procedure of the RM method is described as follows.

Firstly, suppose that a binary input sequence $x^N=(x_1,x_2,\dots ,x_N)$ contains k ones and $N-k$ zeros. Let $Num\left(x^N\right)\ge 0$ be a number which is defined by $x^N$ depending on the lexicographical order set $S_k$. Namely, if $x^N$ has k ones, then the number Num$\left(x^N\right)\ge 0$ is defined by

$$\mathrm{Num}\left(x^N\right)=\sum _{t=1}^N\left(\genfrac{}{}{0pt}{}{x_{t}N-t}{k-{\displaystyle \sum _{i=1}^{t-1}}{x}_i}\right),$$

(2)

where the summation is taken over all $1\le t\le N$ such that $x_t=1$, and Num$\left(0^N\right):=0$. Then, we calculate a binary codeword $code\left(x^N\right)$ of $x^N$, which is the assignment of an output sequence of $x^N$ as follows:

(i)

Compute Num$\left(x^N\right)$ in the set $S_k$, if $x^N$ contains k ones.

(ii)

Let $|S_k|=\left(\genfrac{}{}{0pt}{}{N}{k}\right)=2^{j_0}+2^{j_1}+\dots +2^{j_m}$ for $0\le j_0<j_1<\dots <j_m$.

(iii)

If $j_0=0$ and Num$\left(x^N\right)=0$, then $code\left(x^N\right)=\wedge$.

(iv)

If $0\le \mathrm{Num}\left(x^N\right)<2^{j_0}$, then $code\left(x^N\right)$ is defined to be the $j_0$ low-order binary string of Num$\left(x^N\right)$.

(v)

If ${\displaystyle \sum _{s=0}^t}{2}^{j_s}\le \mathrm{Num}\left(x^N\right)<{\displaystyle \sum _{s=0}^t}{2}^{j_s}+2^{j_{t+1}}$ for some $0\le t\le m$, then $code\left(x^N\right)$ is defined to be the suffix consisting of the $j_{t+1}$ binary string of Num$\left(x^N\right)$.

Example 3.

Suppose that the block size $N=4$, and the given input sequence is $x=(1,0,0,1,0,0,1,1)$, which is the same as all previous examples. After that, the sequence is divided as $x=\left(\right(1,0,0,1),(0,0,1,1\left)\right)$. Next, compute Num$\left(x^N\right)$ following Equation (2):

$$\begin{array}{ccc}\hfill \mathrm{Num}\left(\right(1,0,0,1\left)\right)& =& \left(\genfrac{}{}{0pt}{}{4-1}{2}\right)+\left(\genfrac{}{}{0pt}{}{4-4}{2-1}\right)=3,\hfill \\ \hfill \mathrm{Num}\left(\right(0,0,1,1\left)\right)& =& \left(\genfrac{}{}{0pt}{}{4-3}{2}\right)+\left(\genfrac{}{}{0pt}{}{4-4}{2-1}\right)=0.\hfill \end{array}$$

Then, the RM method computes $code(1,0,0,1)=(1,1)$ and $code(0,0,1,1)=\left(0\right)$. Finally, it outputs $y=(1,1,0)$ by concatenating $code(1,0,0,1)$ and $code(0,0,1,1)$.

The time and space complexity of Elias’s extractor with the RM method are $O(N{log}^{3}NloglogN)$ and $O\left(N{log}^{2}N\right)$, respectively (see [10] for details).

Redundancy: Generally, the rate function and redundancy function of Elias’s extractor depend on block size N. For a given n-bit input sequence, if we take the block size equal to the length of input sequence $N:=n$, the rate function (or redundancy) achieve the best value. For simplicity, we assume that $N=n$ in the following explanation. Then, the rate function $r^{\mathsf{E}}(p,n)$ is evaluated by

$$\begin{array}{c}\hfill r^{\mathsf{E}}(p,n)\approx \frac{1}{n}\sum _{k=0}^n\left(\genfrac{}{}{0pt}{}{n}{k}\right)p^k{(1-p)}^{n-k}log\left(\genfrac{}{}{0pt}{}{n}{k}\right).\end{array}$$

(3)

Elias’s extractor takes i.i.d. with non-uniform distribution as the input, and it will output i.i.d. with uniform distribution such that its rate is given by Equation (3). Elias [7] showed that the rate function $r^{\mathsf{E}}(p,n)$ of Elias’s extractor converges to $h\left(p\right)$ as $n\to \infty$, or equivalently, the redundancy function $f^{\mathsf{E}}(p,n):=h\left(p\right)-r^{\mathsf{E}}(p,n)$ converges to zero as $n\to \infty$. More precisely, it was shown that $f^{\mathsf{E}}(p,n)=O(1/n)$ for any fixed p. Therefore, for a given n-bit input sequence, if we set the maximum block size to be the input size, the non-asymptotic maximum redundancy ${\mathsf{\Gamma }}^{\mathsf{E}}\left(n\right)$ converges to zero not slower than $1/n$.

2.3. Peres’s Extractor

Peres’s extractor is another method that improved the rates (or redundancy) of von Neumann’s extractor. The basic idea behind Peres’s extractor is to reuse the discarded bits in the mapping (1). In the following, we denote von Neumann’s extractor by ${\mathsf{\Psi }}_1$. For an n-bit sequence $(x_1,x_2,\dots ,x_n)$, we describe von Neumann’s extractor by ${\mathsf{\Psi }}_1(x_1,x_2,\dots ,x_n)$ $=(y_1,y_2,\dots ,y_{\ell }),$ where $y_i=x_{2m_i-1}$ and $m_1<m_2<\cdots <m_{\ell }$ are all the indices satisfying $x_{2m_i-1}\ne x_{2m_i}$ with $m_i\le n/2$. In Peres’s extractor, ${\mathsf{\Psi }}_{\nu }$ ($\nu \ge 2$) is defined inductively as follows: For an even n,

$$\begin{array}{c}\hfill {\mathsf{\Psi }}_{\nu }(x_1,x_2,\dots ,x_n)={\mathsf{\Psi }}_1(x_1,x_2,\dots ,x_n)\ast {\mathsf{\Psi }}_{\nu -1}(u_1,u_2,\dots ,u_{\frac{n}{2}})\ast {\mathsf{\Psi }}_{\nu -1}(v_1,v_2,\dots ,v_{\frac{n}{2}-\ell }),\end{array}$$

(4)

where ∗ is concatenation; $u_j=x_{2j-1}\oplus x_{2j}$ for $1\le j\le n/2$; $v_s=x_{2i_s-1}$ and $i_1<i_2<\cdots <i_{\frac{n}{2}-\ell }$ are all the indices satisfying $x_{2i_s-1}=x_{2i_s}$ with $i_s\le n/2$. For an odd input size n, ${\mathsf{\Psi }}_{\nu }(x_1,x_2,\dots ,x_n):={\mathsf{\Psi }}_{\nu }(x_1,x_2,\dots ,x_{n-1})$, i.e., the last bit is discarded and the above case of an even n is utilized.

Note that the number of iterations $\nu$ is at most $⌊logn⌋$, since ${\mathsf{\Psi }}_{\nu }$ for every $\nu \ge 2$ is defined by ${\mathsf{\Psi }}_{\nu -1}$ having an input sequence whose bit-length is at most $n/2$, i.e., the bit-length of both $(u_1,u_2,\dots ,u_{\frac{n}{2}})$ and $(v_1,v_2,\dots ,v_{\frac{n}{2}-\ell })$ in Equation (4) is at most $n/2$. Obviously, Peres’s extractor with $\nu =1$ is the same as von Neumann’s extractor. In addition, Peres’s extractor with a large $\nu$ is considered to be an elegantly improved version of von Neumann’s extractor by utilizing a recursion mechanism.

Example 4.

Suppose that an input sequence is given as $x=(1,0,0,1,0,0,1,1)$, which is the same as all previous examples. The number of iterations satisfies $\nu \le ⌊log8⌋=3$. Then, Peres’s extractor is executed as follows:

$$\begin{array}{ccc}\hfill {\mathsf{\Psi }}_1\left(x\right)& =& (1,0),\hfill \\ \hfill {\mathsf{\Psi }}_2\left(x\right)& =& {\mathsf{\Psi }}_1\left(x\right)\ast {\mathsf{\Psi }}_1(1,1,0,0)\ast {\mathsf{\Psi }}_1(0,1)=(1,0,0),\hfill \\ \hfill {\mathsf{\Psi }}_3\left(x\right)& =& {\mathsf{\Psi }}_1\left(x\right)\ast {\mathsf{\Psi }}_2(1,1,0,0)\ast {\mathsf{\Psi }}_2(0,1)\hfill \\ & =& {\mathsf{\Psi }}_1\left(x\right)\ast ({\mathsf{\Psi }}_1(1,1,0,0)\ast {\mathsf{\Psi }}_1(0,0)\ast {\mathsf{\Psi }}_1(1,0))\ast ({\mathsf{\Psi }}_1(0,1)\ast {\mathsf{\Psi }}_1\left(1\right))\hfill \\ & =& (1,0,1,0).\hfill \end{array}$$

Complexity: We denote the time complexity of ${\mathsf{\Psi }}_{\nu }$ by $T_{\nu }\left(n\right)$. By Equation (4), we have

$$\begin{array}{c}\hfill T_{\nu }\left(n\right)=T_1\left(n\right)+n/2+T_{\nu -1}(n/2)+T_{\nu -1}(n/2-\ell ),\end{array}$$

(5)

and $T_1\left(n\right)=O\left(n\right)$ (see Section 2.1 for the time complexity of von Neumann’s extractor). From the condition (5), we obtain $T_{\nu }\left(n\right)=O\left(\nu n\right)$ for ${\mathsf{\Psi }}_{\nu }$ with $1\le \nu \le ⌊logn⌋$. In particular, the time complexity of Peres’s extractor with the maximum iterations $\nu =⌊logn⌋$ is evaluated as $T_{\nu }\left(n\right)=O(nlogn)$ and the space complexity is $O\left(1\right)$.

Redundancy: The rate function $r_{\nu }^{\mathsf{P}}\left(p\right)$ of Peres’s extractor can be computed inductively by the equation

$$\begin{array}{cc}\hfill r_{\nu }^{\mathsf{P}}\left(p\right)& =pq+\frac{1}{2}{r}_{\nu -1}^{\mathsf{P}}(p^2+q^2)+\frac{1}{2}(p^2+q^2)r_{\nu -1}^{\mathsf{P}}\left(\frac{p^2}{p^2+q^2}\right)\end{array}$$

(6)

for $\nu \ge 2$, and $r_1^{\mathsf{P}}\left(p\right)=pq$. Note that $r_1^{\mathsf{P}}\left(p\right)$ is the rate of von Neumann’s extractor. Peres’s extractor takes i.i.d. with non-uniform distribution as input, and it will output i.i.d. with uniform distribution such that its rate is given by Equation (6) if $n\to \infty$. It is shown in [8] that $r_{\nu }^{\mathsf{P}}\left(p\right)\le r_{\nu +1}^{\mathsf{P}}\left(p\right)$ for all $\nu \in \mathbb{N}$, $p\in (0,1)$, and ${\displaystyle \underset{\nu \to \infty }{lim}{r}_{\nu }^{\mathsf{P}}\left(p\right)=h\left(p\right)}$ uniformly in $p\in (0,1)$.

In other words, the above result is described in terms of redundancy as follows:

$$\begin{array}{ccc}\hfill f_{\nu }^{\mathsf{P}}\left(p\right)& =& h\left(p\right)-r_{\nu }^{\mathsf{P}}\left(p\right)\hfill \\ & =& \frac{1}{2}{f}_{\nu -1}^{\mathsf{P}}(p^2+q^2)+\frac{1}{2}(p^2+q^2)f_{\nu -1}^{\mathsf{P}}\left(\frac{p^2}{p^2+q^2}\right)\hfill \end{array}$$

(7)

for $\nu \ge 2$ and $f_1^{\mathsf{P}}\left(p\right)=h\left(p\right)-p(1-p)$, where the above Equation (7) follows from Equation (6). Furthermore, it holds that $f_{\nu }^{\mathsf{P}}\left(p\right)\ge f_{\nu +1}^{\mathsf{P}}\left(p\right)$ for all $\nu \in \mathbb{N}$, $p\in (0,1)$, and ${\displaystyle \underset{\nu \to \infty }{lim}{f}_{\nu }^{\mathsf{P}}\left(p\right)=0}$ uniformly in $p\in (0,1)$. Suppose that we take the maximum $\nu =⌊logn⌋$ and $n\to \infty$, and then, we have ${\mathsf{\Gamma }}^{\mathsf{P}}\left(n\right)=o\left(1\right)$.

In

Table 1. Comparison of extractors.

	Redundancy $\mathsf{\Gamma }\left(\mathit{n}\right)$	Time Complexity	Space Complexity
von Neumann’s extractor	$3/4$	$O\left(n\right)$	$O\left(1\right)$
Elias’s extractor	$O(1/n)$	$O(n{log}^{3}nloglogn)$	$O\left(n{log}^{2}n\right)$
(with maximum block-size)	(by [7])	(by [10])	(by [10])
Peres’s extractor	$O\left(1\right)$	$O(nlogn)$	$O\left(1\right)$
(with maximum iterations)	(by [8])	(by [8])	(by [8])

, we summarize the redundancy, time complexity and space complexity (memory size) for von Neumann’s, Elias’s, and Peres’s extractors.

3. Lower Bound on Redundancy of Peres’s Extractor

Although it is shown that ${\mathsf{\Gamma }}^{\mathsf{P}}\left(n\right)=o\left(1\right)$ in Peres’s extractor (i.e., ${\mathsf{\Gamma }}^{\mathsf{P}}\left(n\right)$ converges to zero as $n\to \infty$), it is not known whether ${\mathsf{\Gamma }}^{\mathsf{P}}\left(n\right)$ converges to zero rapidly or slowly. To investigate it, we analyze the non-asymptotic redundancy function $f_{\nu }^{\mathsf{P}}(p,n)$ and non-asymptotic maximum redundancy ${\mathsf{\Gamma }}^{\mathsf{P}}\left(n\right)$. In particular, we derive a lower bound on ${\mathsf{\Gamma }}^{\mathsf{P}}\left(n\right)$ based on some heuristics.

Let $f_{\nu }^{\mathsf{P}}\left(p\right)=h\left(p\right)-r_{\nu }^{\mathsf{P}}\left(p\right)$ be the redundancy function for Peres’s extractor with $\nu$ iterations. Then, we first show that $f_{\nu }^{\mathsf{P}}\left(p\right)$ is not concave in $p\in (0,1)$ for $\nu \ge 5$ as follows. The proof is given in Appendix A.

Proposition 1.

The redundancy function $f_{\nu }^{\mathsf{P}}\left(p\right)$ in Peres’s extractor with ν iterations is not concave in $p\in (0,1)$ if $\nu \ge 5$. More generally, for Peres’s extractor with ν iterations, the redundancy function $f_{\nu }^{\mathsf{P}}\left(p\right)$ satisfies

$$\begin{array}{c}\hfill \frac{d^2{f}_{\nu }^{\mathsf{P}}\left(\frac{1}{2}\right)}{d{p}^2}=8-\frac{4}{ln2}-6{\left(\frac{3}{4}\right)}^{\nu -1}.\end{array}$$

(8)

In particular, $\frac{d^2{f}_{\nu }^{\mathsf{P}}}{d{p}^2}\left(\frac{1}{2}\right)<0$ for $1\le \nu \le 4$ and $\frac{d^2{f}_{\nu }^{\mathsf{P}}}{d{p}^2}\left(\frac{1}{2}\right)>0$ for $\nu \ge 5$.

Here, we assume that the following proposition, Proposition 2, holds true. Although it is not easy to provide proof, it seems to be true from our experimental results that are provided in Appendix B. In Figure A1 in Appendix B, we depict the difference values $f_{\nu }^{\mathsf{P}}(p,n)-f_{\nu }^{\mathsf{P}}\left(p\right)$ with input bit-length $n=80,100,\dots ,200$ and iterations $1\le \nu \le ⌊logn⌋$. We note that Proposition 2 states that $f_{⌊logn⌋}^{\mathsf{P}}(p,n)-f_{⌊logn⌋}^{\mathsf{P}}\left(p\right)\ge 0$ for $p\in (0,1)$, and we can observe that it holds true for input bit-length $n=80,100,\dots ,200$ by our experimental results given in Figure A1.

Proposition 2 (Heuristics).

Suppose $\nu =⌊logn⌋$. Then, we have $f_{\nu }^{\mathsf{P}}(p,n)\ge f_{\nu }^{\mathsf{P}}\left(p\right)$, or equivalently $r_{\nu }^{\mathsf{P}}(p,n)\le r_{\nu }^{\mathsf{P}}\left(p\right)$, for a sufficiently large n and any $p\in (0,1)$.

The following theorem shows a lower bound on ${\mathsf{\Gamma }}^{\mathsf{P}}\left(n\right)$ that is derived based on Proposition 2.

Theorem 1.

Suppose that Proposition 2 holds true. Then, in Peres’s extractor with the maximum iterations $\nu =⌊logn⌋$, we have ${\mathsf{\Gamma }}^{\mathsf{P}}\left(n\right)>1/n^{2-log3}$. In particular, ${\mathsf{\Gamma }}^{\mathsf{P}}\left(n\right)=\omega (1/n)$.

Proof. 

Let n be a large natural number. For a natural number $\nu \in \mathbb{N}$ with $1\le \nu \le logn$, we define $a_{\nu }:=r_{\nu }(1/2)$. Then, by Equation (6) we have

$$\begin{array}{c}\hfill a_1=\frac{1}{4},a_{\nu }=\frac{1}{4}+\frac{3}{4}{a}_{\nu -1}\mathrm{for}\nu \ge 2.\end{array}$$

By solving the above equation, we have

$$\begin{array}{c}\hfill a_{\nu }=1-{\left(\frac{3}{4}\right)}^{\nu }\mathrm{for}\nu \ge 1.\end{array}$$

(9)

Thus, for $\nu =⌊logn⌋$, we obtain

$$\begin{array}{ccc}\hfill f_{\nu }^{\mathsf{P}}(1/2,n)& \ge & f_{\nu }^{\mathsf{P}}(1/2)\hfill \end{array}$$

(10)

$$\begin{array}{cc}=& {(3/4)}^{\nu }\hfill \\ \ge & {(3/4)}^{logn}\hfill \\ =& \frac{1}{n^{2-log3}},\hfill \end{array}$$

(11)

where the inequality (10) follows from Proposition 2, and the equality (11) follows from (9).

Therefore, we have

$$\begin{array}{ccc}\hfill {\mathsf{\Gamma }}^{\mathsf{P}}\left(n\right)& =& \underset{p\in (0,1)}{sup}{f}_{⌊logn⌋}^{\mathsf{P}}(p,n)\hfill \\ & >& f_{⌊logn⌋}^{\mathsf{P}}(\frac{1}{2},n)\hfill \\ & \ge & \frac{1}{n^{2-log3}},\hfill \end{array}$$

(12)

where the inequality (12) follows from Proposition 1. ☐

Theorem 1 shows that the non-asymptotic maximum redundancy ${\mathsf{\Gamma }}^{\mathsf{P}}\left(n\right)$ does converge to zero slower than $1/n$. This means that Peres’s extractor is worse than Elias’s extractor in terms of the maximum redundancy, since ${\mathsf{\Gamma }}^{\mathsf{E}}\left(n\right)=O(1/n)$ if block size is set to be n. However, this result does not always mean that Peres’s extractor is worse than Elias’s extractor, since the time complexity and space complexity of Peres’s extractor are better than those of Elias’s extractor, as shown in Table 1. In this sense, it is not easy to conclude which extractor is superior. In the next section, from a viewpoint of practicality including running time, we compare both extractors and show that Peres’s extractor is better than Elias’s extractor by numerical analysis with various parameters.

4. Implementation and Numerical Analysis

In this section, we describe our experimental results of Peres’s extractor and Elias’s extractor with the RM method. We used Java language version 1.8 to implement both extractors and evaluated the performance on a desktop PC with Intel Core i3 3.70 GHz and 4 GB of RAM. Our experiments could also be performed on a general PC and do not require any special resources, libraries, or frameworks for computation. In fact, we can use other languages instead of Java language, but Java language can evaluate it on every platform without any support software. Thereby, we used Java language for implementation. To compare Peres’s extractor and Elias’s extractor with the RM method with finite input sequences in terms of non-asymptotic viewpoints, we consider the following four questions.

(1)

Is theoretical redundancy the same as experimental redundancy in both extractors?

(2)

Is the experimental redundancy of Elias’s extractor with the RM method better than the experimental redundancy of Peres’s extractor?

(3)

What is the exact running time of both extractors?

(4)

Which extractor achieves better redundancy (or rate) under the very similar running time?

To answer the above questions, we design our experiments as follows.

To answer the Questions (1) and (2), we evaluate the theoretical and experimental redundancy of Peres’s extractor and Elias’s extractor by using a pseudorandom number generation program rand() in MATLAB [23] to obtain biased input sequences while controlling the probability (See Section 4.1 and Section 4.2). This experiment used rand() to generate input sequences because we can control the probability p for each input sequence. Therefore, we vary the probability $p=0.1,0.2,\dots ,0.9$. We show the results for a finite input sequence with 180 bits that would be used in various cryptographic algorithms. In fact, we implemented various bit-lengths of input sequences such as $n=80,100,\dots ,200$ bit-length, and obtained very similar results to the case of 180-bit length (In the primary version [21], we implemented only the case of 180-bit length, and in this paper we further investigated $n=80,100,\dots ,200$ bit-length.). Hence, we will describe only the input length with 180 bits, and we omit the cases of other bit-length in this paper. In addition, to investigate the efficiency of Elias’s extractor, the input size should be divided by a reasonable block size. Therefore, the 180 bit-length is also suitable, because it can be divided by many simple block-sizes 10, 20, 30, 60, 90, 180. To compute $\left(\genfrac{}{}{0pt}{}{N}{k}\right)$ in Elias’s extractor with the RM method, we consider the following:

• The Schönhage–Strassen multiplication algorithm requires $O\left(N^{1+ϵ}\right)$ which is asymptotically faster than the normal multiplication requiring $O\left(N^2\right)$;
• To avoid multiplication, we use only the addition operation because it is simple and makes the basic operation lighter so that it can be used in various applications and environments.

Additionally, we use the recursive formula $\left(\genfrac{}{}{0pt}{}{N}{k}\right)=\left(\genfrac{}{}{0pt}{}{N-1}{k-1}\right)+\left(\genfrac{}{}{0pt}{}{N-1}{k}\right)$ for $10\le N\le 180$ in order to compute $\left(\genfrac{}{}{0pt}{}{N}{k}\right)$ only by additions and also by dynamic programming. To compute experimental redundancy with finite input sequences, we use 180 bit-length of inputs and generate 100 times for each probability p. The rand() will produce different sequences in every time under the same probability, thus we repeat the process to generate input sequences 100 times and calculate the average experimental redundancy. In fact, we repeated the process to generate input sequences 100, 1000, and 2000 times, but all the results on the average of experimental redundancy are almost the same, and hence, we focus on generating input sequences 100 times only (In the primary version [21], we repeated the process to generate input sequences only 100 times, and in this paper we conducted further investigations when repeating the process 1000 and 2000 times.). Next, we note that the number of iterations satisfies $\nu \le \lfloor log180\rfloor =7$ for Peres’s extractor in Section 4.1, and we take the block size $N=10,20,30,60,90,180$ for Elias’s extractor with the RM method in Section 4.2. Then, we calculate the average on the redundancy function $f_{\nu }^{\mathsf{P}}\left(p\right)$ of Peres’s extractor by using (7) and the redundancy function $f^{\mathsf{E}}(p,N)=h\left(p\right)-r^E(p,N)$ of Elias’s extractor with the RM method by using (3) for each probability p.

To answer the Question (3), we investigate running time to extract uniformly random sequences for both extractors (See Section 4.3). Time complexity depends on the length of input sequences, and thus the probability is not a parameter in this investigation. Thereby, this experiment changes the random number generator for input sequences to RANDOM.ORG [24] to generate input sequences. This random number generator can produce a sequence that is very close to a true random number with unknown probability p by using the randomness of atmospheric noises. In addition, it can produce 131,072 random bits in each time. This experiment takes $n=100,200,400,600,800,1000,2000,3000,$ $4000,5000$ as the bit-length of input sequences (In the primary version [21], we took only $n=100$, $200,400,600,800,1000$, and we further investigated longer bit-length $2000,3000,4000,5000$ in this paper.). For reliability of our experiment, we repeated the process to extract unbiased random sequences 100 times for each n, and then calculated their average running time.

By analyzing all the results of the experiments above, we can answer the Question (4): we can compare the redundancy of both extractors under the very similar running time (see Section 4.4).

4.1. Analysis of the Redundancy of Peres’s Extractor

In Figure 1a, we show the redundancy of Peres’s extractor from theoretical aspects, that is, we calculated the redundancy $f_{\nu }^{\mathsf{P}}\left(p\right)$ of Peres’s extractor by using (7) with the iterations $\nu =1,2,\dots ,7$ and the probability $p=0.1,0.2,\dots ,0.9$. We depicted the graphs of redundancy $f_{\nu }^{\mathsf{P}}\left(p\right)$, where the x-axis means probability p and the y-axis means redundancy. It can be easily seen that the redundancy becomes smaller as the number of iterations becomes bigger, for all $p\in (0,1)$. Furthermore, we showed the experimental redundancy of Peres’s extractor with 180 bit-length of input sequences in Figure 1b. As a result, the theoretical redundancy in Figure 1a is almost the same as the experimental redundancy in Figure 1b.

Figure 2 depicts the graphs of theoretical redundancy $f_{\nu }^{\mathsf{P}}\left(p\right)$ with $\nu =5,6$ around $p=1/2$, namely, $0.450\le p\le 0.550$. Both graphs support Proposition 1 from a geometric viewpoint. In addition, our experiment shows that $f_5^{\mathsf{P}}\left(p\right)$ would approximately take the maximum $0.2373467$ at $p\approx 0.476$ and $p\approx 0.524$, and $f_6^{\mathsf{P}}\left(p\right)$ would approximately take the maximum $0.1781326$ at $p\approx 0.459$ and $p\approx 0.541$.

Figure 3 is provided to observe the difference or similarity between $f_{\nu }^{\mathsf{P}}\left(p\right)$ and $f_{\nu }^{\mathsf{P}}(p,n)$ ($\nu \ge 5$) for a large fixed n. Figure 3 shows experimental redundancy with probability $0.450\le p\le 0.550$ at the x-axis as in Figure 2. When we observe $f_{\nu }^{\mathsf{P}}\left(p\right)$ and $f_{\nu }^{\mathsf{P}}(p,n)$ by a rough scale, those graphs are very similar as shown in Figure 1; however, when we observe $f_{\nu }^{\mathsf{P}}\left(p\right)$ and $f_{\nu }^{\mathsf{P}}(p,n)$ with $n=180$ and $\nu =5,6$ by a fine scale, we can see the difference between those graphs. Actually, the graphical forms of $f_5^{\mathsf{P}}(p,180)$ and $f_6^{\mathsf{P}}(p,180)$ are quite different from $f_5^{\mathsf{P}}\left(p\right)$ and $f_6^{\mathsf{P}}\left(p\right)$, respectively, as shown in Figure 2 and Figure 3, although there is the fluctuation in Figure 3 depending on our experiments. This implies that we need to analyze a non-asymptotic function $f_{\nu }^{\mathsf{P}}(p,n)$ much more from a theoretical aspect in the future, and this analysis is also important to validate the assumption given in Proposition 2.

4.2. Analysis of the Redundancy of Elias’s Extractor with the RM Method

In Figure 4a, we show the redundancy of Elias’s extractor with the RM method from theoretical aspects, that is, we calculated the theoretical redundancy $f^{\mathsf{E}}(p,N)=h\left(p\right)-r^E(p,N)$ of Elias’s extractor with the RM method by using (3) with probability $p=0.1,0.2,\dots ,0.9$ and the block size $N=10,20,30,60,$ $90,180$. It can be seen that the redundancy becomes smaller as block size becomes larger, for all $p\in (0,1)$. In spite of the fact that there is a slight difference between theoretical redundancy in Figure 4b and experimental redundancy in Figure 4a, we can say that most of them are similar.

As a result, the redundancy of Elias’s extractor with large block size is better than that of Peres’s extractor, which is an answer to our second question. Moreover, we can observe that the theoretical redundancy is almost the same as the experimental redundancy in both extractors, which is an answer to our first question. Therefore, we can rely on our implementation, and we will use this implementation for analyzing the running time in the next section.

4.3. Analysis of the Time Complexity of Both Extractors

This section will answer the third question. In Figure 5a, we show the running time of Peres’s extractor with iterations $\nu =1,2,\dots ,7$ and bit-length of input sequences $n=100$, 200, 400, 600, 800, 1000, $2000,3000,4000,5000$. We depicted the graphs of the running time, where the x-axis is the bit-length of input sequences and the y-axis is the running time in the second unit. It is clearly seen that an increase in the number of iterations leads to a large running time. The running time increases almost linearly but the slope depends on the iterations $\nu$, as supported by a theoretical estimate of time complexity $O\left(\nu n\right)$. Additionally, the running time of iterations $\nu =7$ and the bit-length of input sequences $n=5000$ leads to the largest running time ($1.425$ milliseconds), which means that it can be used in real-world applications.

In Figure 5a, we show the running time of Elias’s extractor with the RM method with block size $N=2,4,6,8,10,12,16,20$. It can be seen that an increase in the block size leads to a large running time. The running time increases linearly, but the slope depends on the block size N, as supported by a theoretical estimate of time complexity $O(N{log}^{3}NloglogN)$. In addition, the running time with block size $N=20$ and bit-length of input sequences $n=5000$ leads to the largest running time ($33.155$ milliseconds), which is much larger than that of Peres’s extractor.

By comparing the running time of both extractors, the running time of Peres’s extractor is better than that of Elias’s extractor with the RM method at the same bit-length of input sequences. In the case of a long bit-length of input sequences, the difference between running time of both extractors can be seen more clearly. Therefore, we can conclude that Peres’s extractor is faster than Elias’s extractor with the RM method at the same bit-length of input sequences. On the other hand, according to the results in Section 4.1 and Section 4.2, we have seen that the redundancy of Elias’s extractor with the RM method is better than that of Peres’s extractor. Thus, we analyze the comparison of redundancy (or rate) under the very similar running time in the next section.

4.4. Comparison under the Very Similar Running Time

In all previous experiments, we have observed that the redundancy of Elias’s extractor with the RM method is better than that of Peres’s extractor; however, the time complexity of Peres’s extractor is better than that of Elias’s extractor with the RM method. Therefore, we will answer the fourth question by comparing the running time in Figure 6a and redundancy under the very similar running time in Figure 6b.

In Figure 6a, we show the comparison of the running time of Peres’s extractor with iterations $\nu =4,5,6$ and the running time of Elias’s extractor with the RM method with block size $N=2,10,20$. The running time of Peres’s extractor with iterations $\nu =6$ (the yellow line) is almost the same as the running time of Elias’s extractor with the RM method having block size $N=2$ (the black dashed line). Thereby, we can compare the experimental redundancy of Peres’s extractor and that of Elias’s extractor with the RM method under the very similar running time, that is, $f_6^{\mathsf{P}}(p,180)$ and $f^{\mathsf{E}}(p,2)$ in Figure 6b. It is clearly seen that $f_6^{\mathsf{P}}(p,180)$ (the yellow line) is much better than $f^{\mathsf{E}}(p,2)$ (the black dashed line), and $f_6^{\mathsf{P}}(p,180)$ is close to $f^{\mathsf{E}}(p,20)$ (the green dashed line). However, the running time of Elias’s extractor with the RM method with block size $N=20$ is much larger than the running time of Peres’s extractor with iterations $\nu =6$, as seen in Figure 6a. In addition, we can observe that the redundancy $f_4^{\mathsf{P}}(p,180)$ of Peres’s extractor with iterations $\nu =4$ (the red line) is close to the redundancy $f^{\mathsf{E}}(p,10)$ of Elias’s extractor with the RM method with block size $N=10$ (the blue dashed line), but the running time of Elias’s extractor with the RM method with block size $N=10$ is approximately 16 times larger than that of Peres’s extractor with iterations $\nu =4$, as seen in Figure 6a (i.e., the blue dashed line and the red line). As a result, we can conclude that Peres’s extractor achieves a better rate (or redundancy) than Elias’s extractor with the RM method under the very similar running time.

5. Conclusions

It is known that Elias’s extractor achieves the optimal rate if the block size tends to infinity. We considered an improved version of Elias’s extractor from Ryabko and Matchikina [10] to reduce both the time complexity and space complexity. Peres’s extractor achieves the optimal rate if the length of the input and the number of iterations tend to infinity. These are the results of asymptotic analysis, but it is important and interesting to non-asymptotically analyze and compare both extractors for finite input sequences, since the resulting information will be useful in applications (e.g., cryptography) in practice.

In this paper, we evaluated the numerical performance of Peres’s extractor and Elias’s extractor with the RM method in terms of practical aspects. Firstly, we derived a lower bound on the maximum redundancy of Peres’s extractor based on some heuristics, and we showed that the maximum redundancy of Elias’s extractor (with the RM method) was superior to that of Peres’s extractor in general, if we do not pay attention to the time complexity or space complexity. We also found that $f_{\nu }^{\mathsf{P}}\left(p\right)$ is not concave in $p\in (0,1)$ for every $\nu \ge 5$. Afterwards, we evaluated the numerical performance of Peres’s extractor and Elias’s extractor with the RM method for finite input sequences. Our implementation evaluated it on a general PC and did not require any special resources, libraries, or frameworks for computation. Our empirical results showed that Peres’s extractor is much better than Elias’s extractor for given finite input sequences under a very similar running time. As a consequence, Peres’s extractor would be more suitable to generate uniformly random sequences in practice in applications such as cryptographic systems.