Guessing with a Bit of Help ^†

Abstract

What is the value of just a few bits to a guesser? We study this problem in a setup where Alice wishes to guess an independent and identically distributed (i.i.d.) random vector and can procure a fixed number of k information bits from Bob, who has observed this vector through a memoryless channel. We are interested in the guessing ratio, which we define as the ratio of Alice’s guessing-moments with and without observing Bob’s bits. For the case of a uniform binary vector observed through a binary symmetric channel, we provide two upper bounds on the guessing ratio by analyzing the performance of the dictator (for general $k\ge 1$) and majority functions (for $k=1$). We further provide a lower bound via maximum entropy (for general $k\ge 1$) and a lower bound based on Fourier-analytic/hypercontractivity arguments (for $k=1$). We then extend our maximum entropy argument to give a lower bound on the guessing ratio for a general channel with a binary uniform input that is expressed using the strong data-processing inequality constant of the reverse channel. We compute this bound for the binary erasure channel and conjecture that greedy dictator functions achieve the optimal guessing ratio.

1. Introduction

In the classical guessing problem, Alice wishes to learn the value of a discrete random variable (r.v.) X as quickly as possible by sequentially asking yes/no questions of the form “Is $X=x$?”, until she makes a correct guess. A guessing strategy corresponds to an ordering of the alphabet of X according to which the guesses are made and induces a random guessing time. It is well known and simple to verify that the guessing strategy which simultaneously minimizes all the positive moments of the guessing time is to order the alphabet according to a decreasing order of probability. Formally, for any $s>0$, the minimal sth-order guessing-time moment of X is

$$G_s\left(X\right):=\mathbb{E}\left({ORD}_X^s\left(X\right)\right),$$

(1)

where ${ORD}_X\left(x\right)$ returns the index of the symbol x relative to the order induced by sorting the probabilities in a descending order, with ties broken arbitrarily. For brevity, we refer to $G_s\left(X\right)$ as the guessing-moment of X.

Several motivating problems for studying guesswork are fairness in betting games, computational complexity of sequential decoding [1], computational complexity of lossy source coding and database search algorithms (see the introduction of Reference [2] for a discussion), secrecy systems [3,4,5], and crypt-analysis (password cracking) [6,7]. The guessing problem was first introduced and studied in an information-theoretic framework by Massey [8], who drew a relation between the average guessing time of an r.v. to its entropy. It was later explored more systematically by Arikan [1], who also introduced the problem of guessing with side information. In this problem, Alice is in possession of another r.v. Y that is jointly distributed with X, and then, the optimal conditional guessing strategy is to guess by decreasing order of conditional probabilities. Hence, the associated minimal conditional sth-order guessing-time moment of X given Y is

$$G_s\left(X\right|Y):=\mathbb{E}\left({ORD}_{X|Y}^s(X\mid Y)\right),$$

(2)

where ${ORD}_{X|Y}(x\mid y)$ returns the index of x relative to the order induced by sorting the conditional probabilities of X given that $Y=y$ in a descending order. Arikan showed that, as intuition suggests, side information reduces the guessing-moments ([1], Corollary 1)

$$G_s\left(X\right|Y)\le G_s\left(X\right).$$

(3)

Furthermore, he showed that, if ${\left\{(X_i,Y_i)\right\}}_{i=1}^n$ is an i.i.d. sequence, then ([1], Proposition 5)

$${\displaystyle \underset{n\to \infty }{lim}}\frac{1}{n}log{G}_s^{1/s}\left(X^n\right|Y^n)=H_{\frac{1}{1+s}}(X_1\mid Y_1),$$

(4)

where $H_{\alpha }(X\mid Y)$ is the Arimoto-Rényi conditional entropy of order $\alpha$. As was noted by Arikan a few years later [9], the guessing moments are related to the large deviations behavior of the random variable $\frac{1}{n}log{ORD}_{X^n|Y^n}(X^n\mid Y^n)$. However, in Reference [9], he was only able to obtain right-tail large deviation bounds since asymptotically tight bounds on $G_s(X^n\mid Y^n)$ were only known for positive moments ($s>0$). Large deviation principle for the normalized logarithm of the guessing time was later established in Reference [10] using substantial results from References [11,12]. Throughout the years, information-theoretic analysis of the guessing problem was extended in multiple directions, such as guessing until the distortion between the guess and the true value is below a certain threshold [2], guessing under source uncertainty [13], and improved bounds at finite blocklength [14,15,16], to name a few.

In the conditional setting described above, one may think of $Y^n$ as side information observed by a “helper”, say Bob, who sends his observations to Alice. Nonetheless, as other problems employing a helper (e.g., source coding [17,18]), it is more realistic to impose communication constraints and to assume that Bob can only send a compressed description of $Y^n$ to Alice. This setting was recently addressed by Graczyk and Lapidoth [19,20], who considered the case where Bob encodes $Y^n$ at a positive rate using $nR$ bits before sending this description to Alice. They then characterized the best possible guessing-moments attained by Alice for general distributions as a function of the rate R. In this paper, we take this setting to its extreme and attempt to quantify the value of k bits in terms of reducing the guessing-moments by allowing Bob to use only a k-bit description of $Y^n$. The major difference from previous work is that, here, k is finite and does not increase with n, and for some of our results, we further concentrate on the extreme case of $k=1$—a single bit of help. To that end, we define (Section 2) the guessing ratio, which is the (asymptotically) best possible ratio of the guessing-moments of $X^n$ obtained with and without observing a function $f\left(Y^n\right)\in {\{0,1\}}^k$, i.e., the minimal possible ratio $G_s(X^n\mid f\left(Y^n\right))/G_s\left(X^n\right)$ as a function of $s>0$, in the limit of large n.

Sharply characterizing the guessing ratio appears to be a difficult problem in general. Here, we mostly focus on the special case where $X^n$ is uniformly distributed over the Boolean cube ${\{0,1\}}^n$ and $Y^n$ is obtained by passing $X^n$ through a memoryless binary symmetric channel (BSC) with crossover probability $\delta$ (Section 3). We derive two upper bounds and two lower bounds on the guessing ratio in this case. The upper bounds are derived by analyzing the ratio attained by two specific functions, k-Dictator, to wit $f\left(Y^n\right)=Y^k$, and Majority, to wit $f\left(Y^n\right)=𝟙({\sum }_{i=1}^n{Y}_i>\frac{n}{2})$, where $𝟙(·)$ is the indicator function, and for simplicity, we henceforth assume that n is odd when discussing majority functions. For $k=1$, we demonstrate that neither of these functions is better than the other for all values of the moment order s. The first lower bound is based on relating the guessing-moment to entropy using maximum-entropy arguments (generalizing a result of Reference [8]), and the second one on Fourier-analytic techniques combined with a hypercontractivity argument [21]. Furthermore, for the restricted class of functions for which the constituent k-bit functions operate on disjoint sets of bits, a general method is proposed for transforming a lower bound valid for $k=1$ to a lower bound valid for any $k\ge 1$. Nonetheless, we remark that our bounds are valid for $s>0$ and obtaining similar bounds for $s<0$ in order to obtain large deviation principle for the normalized logarithm of the guessing time remains an open problem. In Section 4, we briefly discuss the more general case where $X^n$ is still uniform over the Boolean cube, but $Y^n$ is obtained from $X^n$ via a general binary-input, arbitrary-output channel. We generalize our entropy lower bound to this case using the strong data-processing inequality (SDPI) applied to the reverse channel (from Y to X). We then discuss the case of the binary erasure channel (BEC), for which we also provide an upper bound by analyzing the greedy dictator function, namely where Bob sends the first bit that has not been erased. We conjecture that this function minimizes the guessing-moments simultaneously at all erasure parameters and all moments s.

Related Work. As mentioned above, Graczyk and Lapidoth [19,20] considered the same guessing question if Bob can communicate with Alice at some positive rate R, i.e., can use $k=nR$ bits to describe $Y^n$. This setup facilitates the use of large-deviation-based information-theoretic techniques, which allowed the authors to characterize the optimal reduction in the guessing-moments as a function of R to the first order in the exponent. This type of argument cannot be applied in our setup of finite number of bits. Furthermore, as we shall see, in our setup, the exponential order of the guessing moment with help is equal to the one without it and the performance is therefore more finely characterized by bounding the ratio of the guessing-moments. For a single bit of help $k=1$, characterizing the guessing ratio in the case of the BSC with a uniform input can also be thought of as a guessing variant of the most informative Boolean function problem introduced by Kumar and Courtade [22]. There, the maximal reduction in the entropy of $X^n$ obtainable by observing a Boolean function $f\left(Y^n\right)$ is sought after. It was conjectured in Reference [22] that a dictator function, e.g., $f\left(y^n\right)=y_1$, is optimal simultaneously at all noise levels; see References [23,24,25,26] for some recent progress. As in the guessing case, allowing Bob to describe $Y^n$ using $nR$ bits renders the problem amenable to an exact information-theoretic characterization [27]. In another related work [28], we have asked about the Boolean function $Y^n$ that maximizes the reduction in the sequential mean-squared prediction error of $X^n$ and showed that the majority function is optimal in the noiseless case. There is, however, no single function that is simultaneously optimal at all noise levels. Finally, in a recent line of works [29,30], the average guessing time using the help of a noisy version of $f\left(X^n\right)$ has been considered. The model in this paper is different since the noise is applied to the inputs of the function rather than to its output.

2. Problem Statement

Let $X^n$ be an i.i.d. vector from a distribution $P_X$, which is transmitted over a memoryless channel of conditional distribution $P_{Y|X}$. A helper observes $Y^n\in {\mathcal{Y}}^n$ at the output of the channel and can send k bits $f\left(Y^n\right),f:{\mathcal{Y}}^n\to {\{0,1\}}^k$ to a guesser of $X^n$. Our goal is to characterize the best possible multiplicative reduction in guessing-moments offered by a function f, in the limit of large n. Precisely, we wish to characterize the guessing ratio, defined as

$${\gamma }_{s,k}(P_X,P_{Y|X}):={\displaystyle \underset{n\to \infty }{lim\; sup}}{\displaystyle \underset{f:{\mathcal{Y}}^n\to {\{0,1\}}^k}{min}}\frac{G_s(X^n\mid f\left(Y^n\right))}{G_s\left(X^n\right)}$$

(5)

for an arbitrary $s>0$. In this paper, we are mostly interested in the case where $P_X=(1/2,1/2)$, i.e., $X^n$ is uniformly distributed over ${\{0,1\}}^n$, and where the channel is a BSC with crossover probability $\delta \in [0,1/2]$. With a slight abuse of notation, we denote the guessing ratio in this case by ${\gamma }_{s,k}\left(\delta \right)$. Furthermore, some of the results will be restricted to the case of a single bit of help ($k=1$), and in this case, we will further abbreviate the notation from ${\gamma }_{s,1}\left(\delta \right)$ to ${\gamma }_s\left(\delta \right)$. We note the following basic facts.

Proposition 1.

The following properties hold:

1. 

The minimum in Equation (5) is achieved by a sequence of deterministic functions.

2. 

${\gamma }_{s,k}\left(\delta \right)$ is a non-decreasing function of $\delta \in [0,1/2]$ which satisfies ${\gamma }_{s,k}\left(0\right)=2^{-sk}$ and ${\gamma }_{s,k}(1/2)=1$. In addition, ${\gamma }_{s,k}\left(0\right)$ is attained by any sequence of functions $f_n$ such that $f_n\left(Y^n\right)$ is a uniform Bernoulli vector, i.e., $Pr(f_n\left(Y^n\right)=b^k)=2^{-k}$ for all $b^k\in {\{0,1\}}^k$.

3. 

For a BSC $P_{Y|X}$, the limit-supremum in Equation (5) defining ${\gamma }_{s,k}\left(\delta \right)$ is a regular limit.

4. 

If $k=1$ and $X^n$ is a uniformly distributed vector, then the optimal guessing order given that $f\left(Y^n\right)=0$ is reversed to the optimal guessing order when $f\left(Y^n\right)=1$.

Proof. 

See Appendix A. □

3. Guessing Ratio for a Binary Symmetric Channel

3.1. Main Results

We begin by presenting the bound on the guessing ratio ${\gamma }_{s,k}\left(\delta \right)$ obtained by k-dictator functions and then proceed to the bound obtained by majority functions for a single bit of help, $k=1$. The proofs are given in the next two subsections.

Theorem 1.

Let $L_{k,w}:={\sum }_{v=0}^w\left(\genfrac{}{}{0pt}{}{k}{v}\right)$ for $w\in \{0,1,\dots ,k\}$. The guessing ratio is upper bounded as

$${\gamma }_{s,k}\left(\delta \right)\le (1-2\delta )·2^{-sk}·{\displaystyle \sum _{w=0}^{k-1}}{(1-\delta )}^{k-1-w}{\delta }^w·L_{k,w}^{s+1}+{\left(2\delta \right)}^k,$$

(6)

and this upper bound is achieved by k-dictator functions, $f\left(y^n\right)=y^k$.

Specifically, for $k=1$, Theorem 1 implies

$${\gamma }_s\left(\delta \right)\le (1-2\delta )·2^{-s}+2\delta .$$

(7)

Theorem 2.

Let $\beta :=\frac{1-2\delta }{\sqrt{4\delta (1-\delta )}}$ and $Z\sim \mathcal{N}(0,1)$, and denote by $Q(·)$ the tail distribution function of the standard normal distribution. Then, the guessing ratio is upper bounded as

$${\gamma }_s\left(\delta \right)\le 2·(s+1)·\mathbb{E}\left[Q\left(\beta Z\right)·{\left(1-Q\left(Z\right)\right)}^s\right],$$

(8)

and this upper bound is achieved by majority functions, $f\left(y^n\right)=𝟙({\sum }_{i=1}^n{y}_i>\frac{n}{2})$.

We remark that, if $k=1$, the guessing ratio of functions similar to the dictator and majority functions, such as single-bit dictator on $j>1$ inputs ($f\left(y^n\right)=1$ if and only if $y^j=1^j$) or unbalanced majority ($f\left(y^n\right)=𝟙({\sum }_{i=1}^n{y}_i>t)$ for some t), may also be analyzed in a similar way. However, numerical computations indicate that they do not improve the bounds of Theorems 1 and 2, and thus, their analysis is omitted.

We next present two lower bounds on the guessing ratio ${\gamma }_{s,k}\left(\delta \right)$. The first is based on maximum-entropy arguments, and the second is based on Fourier-analytic arguments.

Theorem 3.

The guessing ratio satisfies the following lower bound:

$${\gamma }_{s,k}\left(\delta \right)\ge e^{-1}·\frac{s^{s-1}·(s+1)}{{\mathsf{\Gamma }}^s\left(\frac{1}{s}\right)}·2^{-sk{(1-2\delta )}^2}$$

(9)

where $\mathsf{\Gamma }\left(z\right):={\int }_0^{\infty }{t}^{z-1}{e}^{-t}\mathrm{d}t$ is Euler’s Gamma function (defined for $\Re \left\{z\right\}>0$).

Remark 1.

When restricted to $k=1$, the proof of Theorem 3 utilizes the bound $H\left(X^n\right|f\left(Y^n\right))\ge n-{(1-2\delta )}^2$ (see Equation (63)). For balanced functions, this bound was improved in Reference [23] for $1/2(1-1/\sqrt{3})\le \delta \le 1/2$. Using this improved bound here leads to an immediate improvement in the bound of Theorem 3. Furthermore, it is known [24] that there exists ${\delta }_0$ such that the most informative Boolean function conjecture holds for all ${\delta }_0\le \delta \le 1/2$. For such crossover probabilities,

$$H\left(X^n\right|f\left(Y^n\right))\ge n-1+h\left(\delta \right)$$

(10)

holds, and then, Theorem 3 may be improved to

$${\gamma }_s\left(\delta \right)\ge e^{-1}·\frac{s^{s-1}·(s+1)}{{\mathsf{\Gamma }}^s\left(\frac{1}{s}\right)}·2^{-s\left(1-h\left(\delta \right)\right)}.$$

(11)

Our Fourier-based bound for $k=1$ is as follows:

Theorem 4.

Let $\tau :=1+{(1-2\delta )}^{2(1-\lambda )}$. The guessing ratio is lower bounded as

$${\gamma }_s\left(\delta \right)\ge {\displaystyle \underset{0\le \lambda \le 1}{max}}\left[1-\frac{(s+1)·{(1-2\delta )}^{\lambda }}{{\left(\tau s+1\right)}^{1/\tau }}\right].$$

(12)

This bound can be weakened by the possibly suboptimal choice $\lambda =1$, which leads to a simpler yet explicit bound:

Corollary 1.

$${\gamma }_s\left(\delta \right)\ge 1-\frac{(s+1)·(1-2\delta )}{\sqrt{1+2s}}.$$

(13)

The bound in Theorem 4 is only valid for the case $k=1$. An interesting problem is to find a general way of “transforming” a lower bound which assumes $k=1$ to a bound useful for $k>1$. In principle, such a result could stem from the observation that a k bit function provides k different conditional optimal guessing orders for each of its output bits. For a general function, however, distilling a useful bound from this observation seems challenging since the relation between the optimal guessing order induced by each of the bits and the optimal guessing order induced by all k bits might be involved. Nonetheless, such a result is possible to obtain if each of the k single-bit functions operate on a different set of input bits. For this restricted set of functions, there is a simple bound which relates the optimal ordering given each of the bits and all the k bits together. It is reasonable to conjecture that this restricted sub-class is optimal or at least close to optimal, since it seems that more information is transferred to the guesser when the k functions operate on different sets of bits, which make the k functions statistically independent.

Specifically, let us specify a k-bit function $f:{\mathcal{Y}}^n\to {\{0,1\}}^k$ by its k constituent one-bit functions $f_j:{\mathcal{Y}}^n\to \{0,1\}$, $j\in \left[k\right]$. Let ${\mathcal{F}}_k$ be the set of sequences of functions $\left\{f^{\left(n\right)}\right\}$, $f^{\left(n\right)}:{\mathcal{Y}}^n\to {\{0,1\}}^k$, such that each specific sequence of functions $\left\{f^{\left(n\right)}\right\}$ satisfies the following property: There exists a sequence of partitions ${\{{\{I_j^{\left(n\right)}\}}_{j\in \left[k\right]}\}}_{n=1}^{\infty }$ of $\left[n\right]$, such that, for all $n\ge 1$ and $j\in \left[k\right]$, $f_j^{\left(n\right)}\left(Y^n\right)$ only depends on ${\left\{Y_i\right\}}_{i\in I_j^{\left(n\right)}}$ and ${lim}_{n\to \infty }\left|I_j^{\left(n\right)}\right|=\infty$ for all $j\in \left[k\right]$. In particular, this implies that ${\left\{f_j^{\left(n\right)}\left(Y^n\right)\right\}}_{j\in \left[k\right]}$ is mutually independent for all $n\ge 1$. For example, when $k=2$, $f_1\left(x^n\right)=x_1$, and $f_2\left(x^n\right)=x_2$, we can choose $I_j^{\left(n\right)}$ to be the odd/even indices. For $f_1=Maj\left(y_1^{n/2}\right)$ and $f_2=Maj\left(y_{n/2+1}^n\right)$, the sets are the first and second halves of $\left[n\right]$. As in Equation (5), we may define the guessing ratio of this constrained set of functions as

$${\tilde{\gamma }}_{s,k}\left(\delta \right):={\displaystyle \underset{{\left\{f^{\left(n\right)}\right\}}_{n=1}^{\infty }\in {\mathcal{F}}_k}{min}}{\displaystyle \underset{n\to \infty }{lim\; sup}}\frac{G_s(X^n\mid f^{\left(n\right)}\left(Y^n\right))}{G_s\left(X^n\right)},$$

(14)

where, in general, ${\tilde{\gamma }}_{s,k}\left(\delta \right)\ge {\gamma }_{s,k}\left(\delta \right)$.

Proposition 2.

$${\tilde{\gamma }}_{s,k}\left(\delta \right)\ge \frac{{\tilde{\gamma }}_{s,1}^k\left(\delta \right)}{{(s+1)}^{k-1}}.$$

(15)

We demonstrate our results for $k=1$ in Figure 1 (resp. Figure 2) which display the bounds on ${\gamma }_s\left(\delta \right)$ for fixed values of s (resp. $\delta$). The numerical results show that, for the upper bounds, when $s\lesssim 3.5$, dictator dominates majority (for all values of $\delta$), whereas for $s\gtrsim 4.25$, majority dominates dictator. For $3.5\lesssim s\lesssim 4.25$, there exists ${\delta }_s^{\prime }$ such that majority is better for $\delta \in (0,{\delta }_s^{\prime })$ and dictator is better for $\delta \in ({\delta }_s^{\prime },1/2)$. Figure 2 demonstrates the switch from dictator to majority as s increases (depending on $\delta$). As for lower bounds, we first remark that the conjectured maximum-entropy bound (Equation (11)) is also plotted (see Remark 1). The numerical results show that the maximum-entropy bound is better for low values of $\delta$ whereas the Fourier-analysis bound is better for high values of $\delta$. As a function of s, the maximum-entropy bound (resp. Fourier-analysis bound) is better for high (resp. low) values of s. We also mention that, in these figures, the maximizing parameter in the Fourier-based bound (Theorem 4) is $\lambda =1$ and the resulting bound is as in Equation (13). However, for values of s as low as 10, the maximizing $\lambda$ may be far from 1, and in fact, it continuously and monotonically increases from 0 to 1 as $\delta$ increases from 0 to $1/2$. Finally, Figure 3 demonstrates the behavior of the k-dictator and maximum-entropy bounds on ${\gamma }_{s,k}\left(\delta \right)$ as a function of k.

3.2. Proofs of the Upper Bounds on ${\gamma }_{s,k}\left(\delta \right)$

Let $a,b\in \mathbb{N}$, $a\le b$ be given. The following sum will be useful for the proofs in the rest of the paper:

$$K_s(a,b):=\frac{1}{b-a}{\displaystyle \sum _{i=a+1}^b}{i}^s,$$

(16)

where we will abbreviate $K_s\left(b\right):=K_s(0,b)$. For a pair of sequences ${\left\{a_n\right\}}_{n=1}^{\infty }$, {$b_n{\}}_{n=1}^{\infty }$, we will let $a_n\doteq b_n$ mean that ${lim}_{n\to \infty }\frac{a_n}{b_n}=1$.

Lemma 1.

Let ${\left\{a_n\right\}}_{n=1}^{\infty }$ and ${\left\{b_n\right\}}_{n=1}^{\infty }$ be non-decreasing integer sequences such that $a_n<b_n$ for all n and ${lim}_{n\to \infty }(a_n+1)/b_n=0$. Then,

$$K_s(a_n,b_n)\doteq \frac{1}{s+1}·\frac{b_n^{s+1}-a_n^{s+1}}{b_n-a_n}.$$

(17)

Specifically, $G_s\left(X^n\right)=K_s\left(2^n\right)\doteq \frac{2^{sn}}{s+1}$.

Proof. 

See Appendix A. □

We next prove Theorem 1.

Proof of Theorem 1.

Consider a k-dictator function which directly outputs k of the bits of $y^n$, say, without loss of generality (w.l.o.g.) $f\left(y^n\right)=y^k$. Let $d_H(x^n,y^n)$ be the Hamming distance of $x^n$ and $y^n$, and recall the assumption $0<\delta <1/2$. It is easily verified that the optimal guessing order of $X^n$ given $y^k$ has $k+1$ parts, such that the wth part, $w\in \{0,1,\dots ,k\}$, is comprised of an arbitrary ordering of the $\left(\genfrac{}{}{0pt}{}{k}{w}\right)·2^{n-k}$ vectors for which $d_H(x^k,y^k)=w$. From symmetry, $G_s(X^n\mid f\left(Y^n\right))=G_s(X^n\mid f\left(Y^n\right)=b^k)$ for any $b^k\in {\{0,1\}}^k$. Then, from Lemma 1

$$\begin{array}{cc}\hfill G_s(X^n\mid f\left(Y^n\right)=b^k)& ={\displaystyle \sum _{w=0}^k}\left(\genfrac{}{}{0pt}{}{k}{w}\right){(1-\delta )}^{k-w}{\delta }^w·K_s(2^{n-k}·L_{k,w-1},2^{n-k}·L_{k,w})\hfill \end{array}$$

(18)

$$\begin{array}{cc}\hfill & ={\displaystyle \sum _{w=0}^k}\left(\genfrac{}{}{0pt}{}{k}{w}\right){(1-\delta )}^{k-w}{\delta }^w·K_s(2^{n-k}·L_{k,w-1},2^{n-k}·L_{k,w})\hfill \end{array}$$

(19)

$$\begin{array}{cc}\hfill & \doteq {\displaystyle \sum _{w=0}^k}\left(\genfrac{}{}{0pt}{}{k}{w}\right){(1-\delta )}^{k-w}{\delta }^w·\frac{2^{s(n-k)}}{s+1}·\frac{L_{k,w}^{s+1}-L_{k,w-1}^{s+1}}{\left(\genfrac{}{}{0pt}{}{k}{w}\right)}\hfill \end{array}$$

(20)

$$\begin{array}{cc}\hfill & =\frac{2^{s(n-k)}}{s+1}{\displaystyle \sum _{w=0}^k}{(1-\delta )}^{k-w}{\delta }^w·\left[L_{k,w}^{s+1}-L_{k,w-1}^{s+1}\right]\hfill \end{array}$$

(21)

$$\begin{array}{cc}\hfill & =\frac{2^{s(n-k)}}{s+1}\left((1-2\delta ){\displaystyle \sum _{w=0}^{k-1}}{(1-\delta )}^{k-1-w}{\delta }^w·L_{k,w}^{s+1}+{\delta }^k{2}^{k(s+1)}\right)\hfill \end{array}$$

(22)

where in the first equality, $L_{k,-1}:=0$, and the last equality is obtained by telescoping the sum. The result then follows from Equation (5) and Lemma 1. □

We next prove Theorem 2.

Proof of Theorem 2.

Recall that we assume for simplicity that n is odd. The analysis for an even n is not fundamentally different. To evaluate the guessing-moment, we first need to find the optimal guessing strategy. To this end, we let $W_H\left(x^n\right)$ be the Hamming weight of $x^n$ and note that the posterior probability is given by

$$\begin{array}{cc}\hfill Pr(X^n=x^n\mid Maj\left(Y^n\right)=1)& =\frac{Pr(Maj\left(Y^n\right)=1\mid X^n=x^n)·Pr(X^n=x^n)}{Pr(Maj\left(Y^n\right)=1)}\hfill \end{array}$$

(23)

$$\begin{array}{cc}\hfill & =2^{1-n}·Pr\left({\displaystyle \sum _{i=1}^n}{Y}_i>n/2\mid X^n=x^n\right)\hfill \end{array}$$

(24)

$$\begin{array}{cc}\hfill & =2^{1-n}·Pr\left({\displaystyle \sum _{i=1}^n}{Y}_i>n/2\mid W_H\left(X^n\right)=W_H\left(x^n\right)\right)\hfill \end{array}$$

(25)

$$\begin{array}{cc}\hfill & =:2^{1-n}·r_n\left(W_H\left(x^n\right)\right),\hfill \end{array}$$

(26)

where Equation (25) follows from symmetry. Evidently, $r_n\left(w\right)$ is an increasing function of $w\in \{0,1,\dots ,n\}$. Indeed, let $\mathrm{Bin}(n,\delta )$ be a binomial r.v. of n trials and success probability $\delta$. Then, for any $w\le n-1$, as $\delta \le 1/2$,

$$\begin{array}{cc}\hfill & r_n(w+1)\hfill \end{array}$$

$$\begin{array}{cc}\hfill & =Pr\left(\mathrm{Bin}(w+1,1-\delta )+\mathrm{Bin}(n-w-1,\delta )>n/2\right)\hfill \end{array}$$

(27)

$$\begin{array}{cc}\hfill & =Pr\left(\mathrm{Bin}(w,1-\delta )+\mathrm{Bin}(1,1-\delta )+\mathrm{Bin}(n-w-1,\delta )>n/2\right)\hfill \end{array}$$

(28)

$$\begin{array}{cc}\hfill & \ge Pr\left(\mathrm{Bin}(w,1-\delta )+\mathrm{Bin}(1,\delta )+\mathrm{Bin}(n-w-1,\delta )>n/2\right)\hfill \end{array}$$

(29)

$$\begin{array}{cc}\hfill & =Pr\left(\mathrm{Bin}(w,1-\delta )+\mathrm{Bin}(n-w,\delta )>n/2\right)\hfill \end{array}$$

(30)

$$\begin{array}{cc}\hfill & =r_n\left(w\right),\hfill \end{array}$$

(31)

where, in each of the above probabilities, the summation is over an independent binomial r.v. Hence, we deduce that, whenever $Maj\left(Y^n\right)=1$ (resp. $Maj\left(Y^n\right)=0$), the optimal guessing strategy is by decreasing (resp. increasing) Hamming weight (with arbitrary order for inputs of equal Hamming weight).

We can now turn to evaluate the guessing-moment for the optimal strategy given the majority of $Y^n$. Let $M_{n,w}:={\sum }_{v=0}^w\left(\genfrac{}{}{0pt}{}{n}{v}\right)$ for $w\in \{0,1,\dots ,n\}$. From symmetry,

$$\begin{array}{cc}\hfill G_s(X^n\mid Maj\left(Y^n\right))& =G_s(X^n\mid Maj\left(Y^n\right)=1)\hfill \end{array}$$

(32)

$$\begin{array}{cc}\hfill & ={\displaystyle \sum _{w=0}^n}\left(\genfrac{}{}{0pt}{}{n}{w}\right)2^{1-n}{r}_n\left(w\right){\displaystyle \sum _{i=M_{n,w-1}+1}^{M_{n,w}}}{i}^s\hfill \end{array}$$

(33)

where $M_{n,-1}:=0$. Thus,

$$\begin{array}{cc}\hfill G_s(X^n\mid Maj\left(Y^n\right))& \ge {\displaystyle \sum _{w=0}^n}\left(\genfrac{}{}{0pt}{}{n}{w}\right)2^{1-n}{r}_n\left(w\right)M_{n,w-1}^s\hfill \end{array}$$

(34)

$$\begin{array}{cc}\hfill & =2^{sn+1}·\mathbb{E}\left[r_n\left(W\right){\left(\frac{M_{n,W-1}}{2^n}\right)}^s\right]\hfill \end{array}$$

(35)

$$\begin{array}{cc}\hfill & =2^{sn+1}·\mathbb{E}\left[r_n\left(W\right)Pr{\left(W^{\prime }\le W-1\right)}^s\right],\hfill \end{array}$$

(36)

where $W,W^{\prime }\sim \mathrm{Bin}(n,1/2)$ and is independent. For evaluating the asymptotic behavior (for large n) of this expression, we note that the Berry–Esseen central-limit theorem ([31], Chapter XVI.5, Theorem 2) leads to (see, e.g., Reference [28], proof of Lemma 15)

$$r_n\left(w\right)=Q\left(\beta ·\frac{2}{\sqrt{n}}\left[\frac{n}{2}-w\right]\right)+\frac{a_{\delta }}{\sqrt{n}},$$

(37)

for some universal constant $a_{\delta }$. Using the Berry–Esseen central-limit theorem again, we have that $\frac{2}{\sqrt{n}}(\frac{n}{2}-W^{\prime })\stackrel{d}{\to }Z$, where $Z\sim \mathcal{N}(0,1)$ and $\stackrel{d}{\to }$ denote convergence in distribution. Thus for a given w,

$$\begin{array}{cc}\hfill Pr\left(W^{\prime }\le w-1\right)& =1-Pr\left(\frac{2}{\sqrt{n}}\left(\frac{n}{2}-W^{\prime }\right)\ge \frac{2}{\sqrt{n}}\left(\frac{n}{2}-w-1\right)\right)\hfill \end{array}$$

(38)

$$\begin{array}{cc}\hfill & =1-Q\left(\frac{2}{\sqrt{n}}\left(\frac{n}{2}-w-1\right)\right)-\frac{a_{1/2}}{\sqrt{n}}\hfill \end{array}$$

(39)

$$\begin{array}{cc}\hfill & =1-Q\left(\frac{2}{\sqrt{n}}\left(\frac{n}{2}-w\right)\right)-O\left(\frac{1}{\sqrt{n}}\right),\hfill \end{array}$$

(40)

where the last equality follows from the fact that $|Q^{\prime }\left(t\right)|\le \frac{1}{\sqrt{2\pi }}$ for all $t\in \mathbb{R}$. Using the Berry–Esseen theorem once again, we have that $\frac{2}{\sqrt{n}}(\frac{n}{2}-w)\stackrel{d}{\to }Z$. Hence, Portmanteau’s lemma (e.g., Reference [31], Chapter VIII.1, Theorem 1) and the fact the $Q\left(t\right)$ is continuous and bounded result in the following:

$$G_s(X^n\mid Maj\left(Y^n\right))\ge 2^{sn+1}·\mathbb{E}\left[Q\left(\beta N\right)·{\left(1-Q\left(N\right)\right)}^s\right]+O\left(\frac{1}{n^{s/2}}\right).$$

(41)

Similarly to Equation (34), the upper bound

$$G_s(X^n\mid Maj\left(Y^n\right))\le {\displaystyle \sum _{w=0}^n}\left(\genfrac{}{}{0pt}{}{n}{w}\right)2^{1-n}{r}_n\left(w\right)M_w^s,$$

(42)

holds, and a similar analysis leads to an expression which asymptotically coincides with the right-hand side (r.h.s.) of Equation (41). The result then follows from Equation (5) and Lemma 1. □

3.3. Proofs of the Lower Bounds on ${\gamma }_{s,k}\left(\delta \right)$

To prove Theorem 3, we first prove the following maximum entropy result. With a standard abuse of notation, we will write the guessing-moment and the entropy of a random variable as functions of its distribution.

Lemma 2.

The maximal entropy under guessing-moment constraint satisfies

$${\displaystyle \underset{P:G_s\left(P\right)=g}{max}}H\left(P\right)=log\left(e^{1/s}{s}^{(1-s)/s}·G_s^{1/s}\left(P\right)·\mathsf{\Gamma }\left(\frac{1}{s}\right)\right)+o\left(1\right),$$

(43)

where $o\left(1\right)$ vanishes as $g\to \infty$.

Proof. 

To solve the maximum entropy problem ([32], Chapter 12) in Equation (43) (note that the support of P is only restricted to be countable), we first relax the constraint $G_s\left(P\right)=g$ to

$${\displaystyle \sum _{i=1}^{\infty }}P\left(i\right)·i^s=g,$$

(44)

i.e., we omit the requirement that $\left\{P\right(i\left)\right\}$ is a decreasing sequence. Assuming momentarily that the entropy is measured in nats, it is easily verified (e.g., using the theory of exponential families ([33], Chapter 3) or by Lagrange duality ([34], Chapter 5)) that the entropy maximizing distribution is

$$P_{\lambda }\left(i\right):=\frac{exp(-\lambda i^s)}{Z\left(\lambda \right)}$$

(45)

for $i\in {\mathbb{N}}_{+}$, where $Z\left(\lambda \right):={\sum }_{i=1}^{\infty }exp(-\lambda i^s)$ is the partition function and $\lambda >0$ is chosen such that ${\sum }_{i=1}^{\infty }{P}_{\lambda }\left(i\right)·i^s=g$. Evidently, $P_{\lambda }\left(i\right)$ is in decreasing order (and so is $G_s\left(P_{\lambda }\right)=g$) and is therefore the solution to Equation (43). The resulting maximum entropy is then given in a parametric form as

$$H\left(P_{\lambda }\right)=\lambda G_s\left(P_{\lambda }\right)+lnZ\left(\lambda \right).$$

(46)

Evidently, if $g=G_s\left(P_{\lambda }\right)\to \infty$, then $\lambda \to 0$. In this case, we may approximate the limit of the partition function as $\lambda \to 0$ by a Riemann integral. Specifically, by the monotonicity of $e^{-\lambda i^s}$ in $i\in \mathbb{N}$,

$$\begin{array}{cc}\hfill Z\left(\lambda \right)& ={\displaystyle \sum _{i=1}^{\infty }}{e}^{-\lambda i^s}\hfill \end{array}$$

(47)

$$\begin{array}{cc}\hfill & =\frac{1}{2}\left({\displaystyle \sum _{i=-\infty }^{\infty }}exp\left(-{\left(\frac{\left|i\right|}{{\lambda }^{-1/s}}\right)}^s\right)-1\right)\hfill \end{array}$$

(48)

$$\begin{array}{cc}\hfill & \ge \frac{1}{2}\left({\int }_{-\infty }^{\infty }exp\left(-{\left(\frac{\left|t\right|}{{\lambda }^{-1/s}}\right)}^s\right)\mathrm{d}t-1\right)\hfill \end{array}$$

(49)

$$\begin{array}{cc}\hfill & =\frac{1}{s}{\lambda }^{-1/s}·\mathsf{\Gamma }\left(\frac{1}{s}\right)-\frac{1}{2},\hfill \end{array}$$

(50)

where the last equality follows from the definition of the Gamma function (see Theorem 3) or from the identification of the integral as an unnormalized generalized Gaussian distribution of zero mean, scale parameter ${\lambda }^{-1/s}$, and shape parameter s [35]. Further, by the convexity of $e^{-\lambda t^s}$ in $t\in {\mathbb{R}}_{+}$, Jensen’s inequality implies that

$$e^{-\lambda i^s}\le {\int }_{-i-1/2}^{i+1/2}exp\left(-{\lambda \left|t\right|}^s\right)\mathrm{d}t$$

(51)

for every $i\ge 1$ (the r.h.s. can be considered as averaging over a uniform random variable $[i-1/2,i+1/2]$) and so, similarly to Equation (50),

$$\begin{array}{cc}\hfill Z\left(\lambda \right)& \le \frac{1}{2}\left({\int }_{-\infty }^{\infty }exp\left(-{\left(\frac{\left|t\right|}{{\lambda }^{-1/s}}\right)}^s\right)\mathrm{d}t\right).\hfill \end{array}$$

(52)

Therefore,

$$Z\left(\lambda \right)=(1+a_{\lambda })·\frac{1}{s}{\lambda }^{-1/s}·\mathsf{\Gamma }\left(\frac{1}{s}\right)$$

(53)

where $a_{\lambda }\to 0$ as $\lambda \to 0$. In the same spirit,

$$\begin{array}{cc}\hfill G_s\left(P_{\lambda }\right)& ={\displaystyle \sum _{i=1}^{\infty }}{i}^s·\frac{exp(-\lambda i^s)}{Z\left(\lambda \right)}\hfill \end{array}$$

(54)

$$\begin{array}{cc}\hfill & =\frac{{\int }_0^{\infty }{t}^{s}exp\left(-{\left(\frac{\left|t\right|}{{\lambda }^{-1/s}}\right)}^s\right)\mathrm{d}t+b_{\lambda }}{(1+a_{\lambda })\frac{1}{s}{\lambda }^{-1/s}·\mathsf{\Gamma }\left(\frac{1}{s}\right)}\hfill \end{array}$$

(55)

$$\begin{array}{cc}\hfill & =\frac{\frac{1}{s}{\lambda }^{-\frac{s+1}{s}}·\mathsf{\Gamma }\left(\frac{s+1}{s}\right)+b_{\lambda }}{(1+a_{\lambda })\frac{1}{s}{\lambda }^{-1/s}·\mathsf{\Gamma }\left(\frac{1}{s}\right)}\hfill \end{array}$$

(56)

$$\begin{array}{cc}\hfill & =\frac{\frac{1}{s^2}{\lambda }^{-\frac{s+1}{s}}·\mathsf{\Gamma }\left(\frac{1}{s}\right)+b_{\lambda }}{(1+a_{\lambda })\frac{1}{s}{\lambda }^{-1/s}·\mathsf{\Gamma }\left(\frac{1}{s}\right)}\hfill \end{array}$$

(57)

$$\begin{array}{cc}\hfill & =\frac{1}{s\lambda }(1+c_{\lambda }),\hfill \end{array}$$

(58)

where in Equation (56), $b_{\lambda }\to 0$ as $\lambda \to 0$; in Equation (57), the identity $\mathsf{\Gamma }(t+1)=t\mathsf{\Gamma }\left(t\right)$ for $t\in {\mathbb{R}}_{+}$ was used; and in Equation (58), $c_{\lambda }\to 0$ as $\lambda \to 0$.

Returning to measure entropy in bits, we thus obtain that, for any distribution P,

$$H\left(P\right)\le log\left(\frac{e^{1/s}}{s}{s}^{1/s}·G_s^{1/s}\left(P\right)·\mathsf{\Gamma }\left(\frac{1}{s}\right)\right)+o\left(1\right),$$

(59)

or, equivalently,

$$G_s\left(P\right)\ge {\mathsf{\Psi }}_s·2^{sH\left(P\right)}·(1+o\left(1\right)),$$

(60)

where ${\mathsf{\Psi }}_s:=e^{-1}·\frac{s^{s-1}}{{\mathsf{\Gamma }}^s\left(\frac{1}{s}\right)}$ and $o\left(1\right)$ is a vanishing term as $G_s\left(P\right)\to \infty$. In the same spirit, Equation (60) holds whenever $H\left(P\right)\to \infty$. □

Remark 2.

In Reference [8], the maximum-entropy problem was studied for $s=1$. In this case, the maximum-entropy distribution is readily identified as the geometric distribution. The proof above generalizes that result to any $s>0$.

Proof of Theorem 3.

Assume that f is taken from a sequence of functions which achieves the minimum in Equation (5). Using Lemma 2 when conditioning on $f\left(Y^n\right)=b^k$ for each of possible $b^k$, we get (see a rigorous justification to Equation (61) in Appendix A)

$$\begin{array}{cc}\hfill G_s(X^n\mid f\left(Y^n\right))& \ge {\ell }_n·{\mathsf{\Psi }}_s·{\displaystyle \sum _{b^k\in {\{0,1\}}^k}}Pr(f\left(Y^n\right)=b^k)·2^{sH(X^n\mid f\left(Y^n\right)=b^k)}\hfill \end{array}$$

(61)

$$\begin{array}{cc}\hfill & \ge {\ell }_n·{\mathsf{\Psi }}_s·2^{sH\left(X^n\right|f\left(Y^n\right))}\hfill \end{array}$$

(62)

$$\begin{array}{cc}\hfill & \ge {\ell }_n·{\mathsf{\Psi }}_s·2^{s[n-k{(1-2\delta )}^2]}\hfill \end{array}$$

(63)

where in Equation (61), ${\ell }_n\doteq 1$ and Equation (62) follows from Jensen’s inequality. For $k=1,$ the bound in Equation (63) is directly related to the Boolean function conjecture [22] and may be proved in several ways, e.g., using Mrs. Gerber’s Lemma ([36], Theorem 1); see ([23], Section IV), References [27,37]. For general $k\ge 1$, the bound $H\left(X^n\right|f\left(Y^n\right))\ge n-k{(1-2\delta )}^2$ was established in Reference ([27], Corollary 1). □

Before presenting the proof of the Fourier-based bound, we briefly remind the reader of the basic definitions and results of Fourier analysis of Boolean functions [21], and to that end, it is convenient to replace the binary alphabet $\{0,1\}$ by $\{-1,1\}$. An inner product between two real-valued functions on the Boolean cube $f,g:{\{-1,1\}}^n\to \mathbb{R}$ is defined as

$$〈f,g〉:=\mathbb{E}\left(f\left(X^n\right)g\left(X^n\right)\right),$$

(64)

where $X^n\in {\{-1,1\}}^n$ is a uniform Bernoulli vector. A character associated with a set of coordinates $S\subseteq \left[n\right]:=\{1,2,\dots ,n\}$ is the Boolean function $x^S:={\prod }_{i\in S}{x}_i$, where by convention, $x^{\varnothing }:=1$. It can be shown ([21], Chapter 1) that the set of all characters forms an orthonormal basis with respect to the inner product (Equation (64)). Furthermore,

$$f\left(x^n\right)={\displaystyle \sum _{S\subseteq \left[n\right]}}{\widehat{f}}_S·x^S,$$

(65)

where ${\left\{{\widehat{f}}_S\right\}}_{S\subseteq \left[n\right]}$ are the Fourier coefficients of f, given by ${\widehat{f}}_S=\langle x^S,f\rangle =\mathbb{E}(X^S·f\left(X^n\right))$. Plancherel’s identity then states that $\langle f,g\rangle =\mathbb{E}\left(f\left(X^n\right)g\left(X^n\right)\right)={\sum }_{S\in \left[n\right]}{\widehat{f}}_S{\widehat{g}}_S$. The p norm of a function f is defined as ${\parallel f\parallel }_p:=\left[\mathbb{E}\right|f\left(X^n\right){{|}^p]}^{1/p}$.

The noise operator operating on a Boolean function f is defined as

$$T_{\rho }f\left(x^n\right)=\mathbb{E}(f\left(Y^n\right)\mid X^n=x^n)$$

(66)

where $\rho :=1-2\delta$ is the correlation parameter. The noise operator has a smoothing effect on the function which is captured by the so-called hypercontractivity theorems. Specifically, we shall use the following version.

Theorem 5

([21], p. 248). Let $f:{\{-1,1\}}^n\to \mathbb{R}$ and $0\le \rho \le 1$. Then, $\parallel T_{\rho }{f\parallel }_2\le {\parallel f\parallel }_{{\rho }^2+1}$.

With the above, we can prove Theorem 4.

Proof of Theorem 4.

From Bayes law (recall that $f\left(x^n\right)\in \{-1,1\}$)

$$Pr(X^n=x^n\mid f\left(Y^n\right)=b)=2^{-(n+1)}·\frac{1+b{T}_{\rho }f\left(x^n\right)}{Pr(f\left(Y^n\right)=b)},$$

(67)

and from the law of total expectation

$$G_s(X^n\mid f\left(Y^n\right))=Pr(f\left(Y^n\right)=1)·G_s(X^n\mid f\left(Y^n\right)=1)+Pr(f\left(Y^n\right)=-1)·G_s(X^n\mid f\left(Y^n\right)=-1).$$

(68)

Let us denote ${\widehat{f}}_{\varphi }=\mathbb{E}f\left(X^n\right)$ and $g:=f-{\widehat{f}}_{\varphi }$ and abbreviate ${ORD}_f\left(x^n\right):={ORD}_{X^n\mid f\left(Y^n\right)}(x^n\mid 1)$. Then, the first addend on the r.h.s. of Equation (68) is given by

$$\begin{array}{cc}\hfill Pr(f\left(Y^n\right)=1)·G_s(X^n\mid f\left(Y^n\right)=1)& =2^{-(n+1)}{\displaystyle \sum _{x^n}}\left(1+{\widehat{f}}_{\varphi }+T_{\rho }g\left(x^n\right)\right)·{ORD}_{T_{\rho }g}^s\left(x^n\right)\hfill \end{array}$$

(69)

$$\begin{array}{cc}\hfill & =\frac{(1+{\widehat{f}}_{\varphi })}{2}·\mathbb{E}\left({ORD}_{T_{\rho }g}^s\left(X^n\right)\right)+\frac{1}{2}\langle T_{\rho }g,{ORD}_{T_{\rho }g}^s\rangle \hfill \end{array}$$

(70)

$$\begin{array}{cc}\hfill & =\frac{(1+{\widehat{f}}_{\varphi })}{2}·K_s\left(2^n\right)+\frac{1}{2}\langle T_{\rho }g,{ORD}_{T_{\rho }g}^s\rangle \hfill \end{array}$$

(71)

$$\begin{array}{cc}\hfill & =\frac{(1+{\widehat{f}}_{\varphi })}{2}·{\ell }_n·\frac{2^{sn}}{s+1}+\frac{1}{2}\langle T_{\rho }g,{ORD}_{T_{\rho }g}^s\rangle ,\hfill \end{array}$$

(72)

where, in the last equality, ${\ell }_n\doteq 1$ (Lemma 1). Let $\lambda \in [0,1]$, and denote ${\rho }_1:={\rho }^{\lambda }$ and ${\rho }_2={\rho }^{1-\lambda }$. Then, the inner-product term in Equation (72) is upper bounded as

$$\begin{array}{cc}\hfill \left|\langle T_{\rho }g,{ORD}_{T_{\rho }g}^s\rangle \right|& =\left|\langle T_{{\rho }_1}g,T_{{\rho }_2}{ORD}_{T_{\rho }g}^s\rangle \right|\hfill \end{array}$$

(73)

$$\begin{array}{cc}\hfill & \le \parallel T_{{\rho }_1}{g\parallel }_2·{\parallel T_{{\rho }_2}{ORD}_{T_{\rho }g}^s\parallel }_2\hfill \end{array}$$

(74)

$$\begin{array}{cc}\hfill & \le {\rho }_1·\sqrt{1-{\widehat{f}}_{\varphi }^2}·{\parallel T_{{\rho }_2}{ORD}_{T_{\rho }g}^s\parallel }_2\hfill \end{array}$$

(75)

$$\begin{array}{cc}\hfill & \le {\rho }_1·\sqrt{1-{\widehat{f}}_{\varphi }^2}·{\parallel {ORD}_{T_{\rho }g}^s\parallel }_{1+{\rho }_2^2}\hfill \end{array}$$

(76)

$$\begin{array}{cc}\hfill & ={\rho }_1·\sqrt{1-{\widehat{f}}_{\varphi }^2}·{\left(K_{(1+{\rho }_2^2)s}\left(2^n\right)\right)}^{1/(1+{\rho }_2^2)}\hfill \end{array}$$

(77)

$$\begin{array}{cc}\hfill & ={\rho }_1·\sqrt{1-{\widehat{f}}_{\varphi }^2}·{\left[k_n·\frac{1}{(1+{\rho }_2^2)s+1}\right]}^{1/(1+{\rho }_2^2)}·2^{sn},\hfill \end{array}$$

(78)

where Equation (73) holds since $T_{\rho }$ is a self-adjoint operator and Equation (74) follows from the Cauchy–Schwarz inequality. To justify Equation (75), we note that

$$\begin{array}{cc}\hfill \parallel T_{\rho }{g\parallel }_2^2& =\langle T_{\rho }g,T_{\rho }g\rangle \hfill \end{array}$$

(79)

$$\begin{array}{cc}\hfill & ={\displaystyle \sum _{S\in \left[n\right]}}{\rho }^{2\left|S\right|}{\widehat{g}}_S^2\hfill \end{array}$$

(80)

$$\begin{array}{cc}\hfill & ={\displaystyle \sum _{S\in \left[n\right]\backslash \varphi }}{\rho }^{2\left|S\right|}{\widehat{f}}_S^2\hfill \end{array}$$

(81)

$$\begin{array}{cc}\hfill & \le {\rho }^2·(1-{\widehat{f}}_{\varphi }^2),\hfill \end{array}$$

(82)

where Equation (80) follows from Plancherel’s identity, Equation (81) is since ${\widehat{g}}_S={\widehat{f}}_S$ for all $S\ne \varphi$ and ${\widehat{g}}_{\varphi }=0$, and Equation (82) follows from ${\sum }_{S\in \left[n\right]}{\widehat{f}}_S^2={\parallel f\parallel }_2^2=\mathbb{E}{f}^2=1$. Equation (76) follows from Theorem 5, and in Equation (78), $k_n\doteq 1$. The second addend on the r.h.s. of Equation (68) can be bounded in the same manner. Hence,

$$\begin{array}{cc}\hfill G_s(X^n\mid f\left(Y^n\right))& \ge {\displaystyle \underset{0\le \lambda \le 1}{max}}{2}^{sn}·\left[{\ell }_n·\frac{1}{s+1}-{\rho }^{\lambda }·\sqrt{1-{\widehat{f}}_{\varphi }^2}·{\left[k_n·\frac{1}{(1+{\rho }^{2(1-\lambda )})s+1}\right]}^{1/(1+{\rho }^{2(1-\lambda )})}\right]\hfill \end{array}$$

(83)

$$\begin{array}{cc}\hfill & \ge {\displaystyle \underset{0\le \lambda \le 1}{max}}{2}^{sn}·\left[{\ell }_n·\frac{1}{s+1}-{\rho }^{\lambda }{\left[k_n·\frac{1}{(1+{\rho }^{2(1-\lambda )})s+1}\right]}^{1/(1+{\rho }^{2(1-\lambda )})}\right]\hfill \end{array}$$

(84)

$$\begin{array}{cc}\hfill & \to 2^{sn}·{\displaystyle \underset{0\le \lambda \le 1}{max}}\left[\frac{1}{s+1}-\frac{{\rho }^{\lambda }}{{\left[(1+{\rho }^{2(1-\lambda )})s+1\right]}^{1/(1+{\rho }^{2(1-\lambda )})}}\right]\hfill \end{array}$$

(85)

as $n\to \infty$. □

We close this section with the following proof of Proposition 2:

Proof of Proposition 2.

Let $I=(i_1,\dots ,i_L)$ be a vector of indices in $\left[n\right]$ such that $1\le i_1<i_2<\cdots <i_L\le n$, and let $x^n\left(I\right)=(x_{i_1},\dots ,x_{i_L})$ be the components of $x^n$ in those indices. Further, let ${\left\{f^{\left(n\right)}\right\}}_{n=1}^{\infty }\in {\mathcal{F}}_k$. Then, it holds that

$$Pr\left[X^n=x^n,f^{\left(n\right)}\left(Y^n\right)\right]={\displaystyle \prod _{j=1}^k}Pr\left[X^n\left(I_j\right)=x^n\left(I_j\right),f_j^{\left(n\right)}\left(y^n\right)\right],$$

(86)

as well as

$${ORD}_{X^n\mid f^{\left(n\right)}\left(Y^n\right)}(x^n\mid b^k)\ge {\displaystyle \prod _{j=1}^k}\left[{ORD}_{X^n\left(I_j\right)\mid f_j^{\left(n\right)}\left(Y^n\right)}(x^n\left(I_j\right)\mid b_j)-1\right].$$

(87)

Hence,

$$\begin{array}{cc}\hfill G_s(X^n\mid f^{\left(n\right)}\left(Y^n\right))& \ge {\displaystyle \prod _{j=1}^k}[G_s^{\left(n\right)}(X^n\left(I_j\right)\mid f_j^{\left(n\right)}\left(Y^n\right))-1]\hfill \end{array}$$

(88)

and the stated bound is deduced after taking limits and normalizing by $G_s\left(X^n\right)\doteq \frac{2^{sn}}{s+1}$. □

4. Guessing Ratio for a General Binary Input Channel

In this section, we consider the guessing ratio for general channels with a uniform binary input. The lower bound of Theorem 3 can be easily generalized to this case. To that end, consider the SDPI constant [38,39] of the reverse channel $(P_Y,P_{X|Y})$, given by

$$\eta (P_Y,P_{Y|X}):={\displaystyle \underset{Q_Y:Q_Y\ne P_Y}{sup}}\frac{D\left(Q_X\right|\left|P_X\right)}{D\left(Q_Y\right|\left|P_Y\right)},$$

(89)

where $Q_X$ is the X-marginal of $Q_Y\circ P_{X|Y}$. As was shown in Reference ([40], Theorem 2), the SDPI constant of $(P_Y,P_{X|Y})$ is also given by

$$\eta (P_Y,P_{Y|X})={\displaystyle \underset{P_{W|Y}:W-Y-X,I(W;Y)>0}{sup}}\frac{I(W;X)}{I(W;Y)}.$$

(90)

Theorem 6.

We have

$${\gamma }_{s,k}(P_X,P_{Y|X})\ge e^{-1}·\frac{s^{s-1}·(s+1)}{{\mathsf{\Gamma }}^s\left(\frac{1}{s}\right)}·2^{-s·k·\eta (P_Y,P_{X|Y})}.$$

(91)

Proof. 

See Appendix A. □

Remark 3.

The bound for the BSC case (Theorem 3) is indeed a special case of Theorem 6 as the reverse BSC channel is also a BSC with uniform input and the same crossover probability. For BSCs, it is well known that the SDPI constant is ${(1-2\delta )}^2$ ([38], Theorem 9).

Next, we consider in more detail the case where the observation channel is a BEC. We restrict the discussion to the case of a single bit of help, $k=1$.

4.1. Binary Erasure Channel

Suppose that $Y^n\in {\{0,1,\mathrm{e}\}}^n$ is obtained from $X^n$ by erasing each bit independently with probability $ϵ\in [0,1]$. As before, Bob observes the channel output $Y^n$ and can send one bit $f:{\{0,1,\mathrm{e}\}}^n\to \{0,1\}$ to Alice, who wishes to guess $X^n$. With a slight abuse of notation, the guessing ratio in Equation (5) will be denoted by ${\gamma }_s\left(ϵ\right)$.

To compute the lower bound of Theorem 6, we need to find the SDPI constant associated with the reverse channel, which is easily verified to be

$$P_{X|Y=y}\left(x\right)=\left\{\begin{array}{cc}𝟙(x=y),\hfill & y=0\mathrm{or}y=1\hfill \\ \mathrm{Ber}(1/2),\hfill & y=\mathrm{e},\hfill \end{array}\right.$$

(92)

with an input distribution $P_Y=(\frac{1-ϵ}{2},ϵ,\frac{1-ϵ}{2})$. Letting $Q_Y\left(y\right)=q_y$ for $y\in \{0,1,\mathrm{e}\}$ yields $Q_X\left(x\right)=q_x+\frac{q_{\mathrm{e}}}{2}$ for $x\in \{0,1\}$. The computation of $\eta (P_Y,P_{X|Y})$ is now a simple three-dimensional constrained optimization problem. We plotted the resulting lower bound for $s=1$ in Figure 4.

Let us now turn to upper bounds and focus for simplicity on the average guessing time, i.e., the guessing-moment for $s=1$. To begin, let S represent the set of indices of the symbols that were not erased, i.e., $i\in S$ if and only if $Y_i\ne \mathrm{e}$. Any function $f:{\{0,1,\mathrm{e}\}}^n\to \{0,1\}$ is then uniquely associated with a set of Boolean functions ${\left\{f_S\right\}}_{S\in \left[n\right]}$, where $f_S:{\{0,1\}}^{\left|S\right|}\to \{0,1\}$ designates the operation of the function when S is the set of non-erased symbols. We also let $Pr\left(S\right)={(1-ϵ)}^{\left|S\right|}·{ϵ}^{|S^c|}$ be the probability that the non-erased symbols have index set S. Then, the joint probability distribution is given by

$$\begin{array}{cc}\hfill Pr(X^n=x^n,f\left(Y^n\right)=1)& =Pr(X^n=x^n)·Pr(f\left(Y^n\right)=1\mid X^n=x^n)\hfill \end{array}$$

(93)

$$\begin{array}{cc}\hfill & =2^{-n}·{\displaystyle \sum _{S\subseteq \left[n\right]}}Pr\left(S\right)·Pr(f\left(Y^n\right)=1\mid X^n=x^n,S)\hfill \end{array}$$

(94)

$$\begin{array}{cc}\hfill & =2^{-n}·{\displaystyle \sum _{S\subseteq \left[n\right]}}Pr\left(S\right)·f_S\left(x^n\right),\hfill \end{array}$$

(95)

and, similarly,

$$\begin{array}{cc}\hfill Pr(X^n=x^n,f\left(Y^n\right)=0)& =2^{-n}·{\displaystyle \sum _{S\subseteq \left[n\right]}}Pr\left(S\right)·(1-f_S\left(x^n\right))\hfill \end{array}$$

(96)

$$\begin{array}{cc}\hfill & =2^{-n}-2^{-n}·{\displaystyle \sum _{S\subseteq \left[n\right]}}Pr\left(S\right)·f_S\left(x^n\right).\hfill \end{array}$$

(97)

In accordance with Proposition 1, the optimal guessing order given that $f\left(Y^n\right)=0$ is reversed to the optimal guessing order when $f\left(Y^n\right)=1$. It is also apparent that the posterior probability is determined by a mixture of $2^n$ different Boolean functions ${\left\{f_S\right\}}_{S\in \left[n\right]}$. This may be contrasted with the BSC case, in which the posterior is determined by a single Boolean function (though with noisy input).

A seemingly natural choice is a greedy dictator function, for which $f\left(Y^n\right)$ sends the first non-erased bit. Concretely, letting

$$k\left(y^n\right):=\left\{\begin{array}{cc}n+1,\hfill & y^n={\mathrm{e}}^n\hfill \\ min\left\{i:y_i\ne \mathrm{e}\right\},\hfill & \mathrm{otherwise}\hfill \end{array}\right.,$$

(98)

the greedy dictator function is defined by

$$\text{G-Dict}\left(y^n\right):=\left\{\begin{array}{cc}\mathrm{Ber}(1/2),\hfill & y^n={\mathrm{e}}^n\hfill \\ y_{k\left(y^n\right)},\hfill & \mathrm{otherwise}\hfill \end{array}\right.,$$

(99)

where $\mathrm{Ber}\left(\alpha \right)$ is a Bernoulli r.v. of success probability $\alpha$. From an analysis of the posterior probability, it is evident that, conditioned on $f\left(Y^n\right)=0$, an optimal guessing order must satisfy that $x^n$ is guessed before $z^n$ whenever

$${\displaystyle \sum _{i=1}^n}{ϵ}^{i-1}·x_i\le {\displaystyle \sum _{i=1}^n}{ϵ}^{i-1}·z_i,$$

(100)

(see Appendix A for a proof of Equation (100)). This rule can be loosely thought of as comparing the “base $1/ϵ$ expansion” of $x^n$ and $z^n$. Furthermore, when $ϵ$ is close to 1, then the optimal guessing order tends toward a minimum Hamming weight rule (or maximum Hamming weight in case $f=1$).

The greedy dictator function is “locally optimal” when $ϵ\in [0,1/2]$, in the following sense:

Proposition 3.

If $ϵ\in [0,1/2]$, then an optimal guessing order conditioning on $\text{G-Dict}\left(Y^n\right)=0$ (resp. $\text{G-Dict}\left(Y^n\right)=1$) is lexicographic (reverse lexicographic). Also, given lexicographic (resp. reverse lexicographic) order when the received bit is 0 (resp. 1), the optimal function f is a greedy dictator.

Proof. 

See Appendix A. □

The guessing ratio of the greedy dictator function can be evaluated for $s=1$, and the analysis leads to the following upper bound:

Theorem 7.

For $s=1$, the guessing ratio is upper bounded as

$${\gamma }_1\left(ϵ\right)\le \frac{1}{2-ϵ},$$

(101)

and the r.h.s. is achieved with equality by the greedy dictator function in Equation (99) for $ϵ\in [0,1/2]$.

Proof. 

See Appendix A. □

The upper bound of Theorem 7 is plotted in Figure 4. Based on Proposition 3 and numerical computations for moderate values of n, we conjecture:

Conjecture. 1.

Greedy dictator functions attain ${\gamma }_s\left(ϵ\right)$ for the BEC.

Supporting evidence for this conjecture include the local optimality property stated in Proposition 3 (although there are other locally optimal choices) as well as the following heuristic argument: Intuitively, Bob should reveal as much as possible regarding the bits he has seen and as little as possible regarding the erasure pattern. So, it seems reasonable to find a smallest possible set of balanced functions from which to choose all the functions $f_S$, so that they coincide as much as possible. Greedy dictator is a greedy solution to this problem: it uses the function $x_1$ for half of the erasure patterns, which is the maximum possible. Then, it uses the function $x_2$ for half of the remaining patterns, and so on. Indeed, we were not able to find a better function than G-Dict for small values of n.

However, applying standard techniques in attempt to prove Conjecture 1 has not been fruitful. One possible technique is induction. For example, assume that the optimal functions for dimension $n-1$ are $f_S^{(n-1)}$. Then, it might be perceived that there exists a bit, say $x_1$, such that the optimal functions for dimension n satisfy $f_S^{\left(n\right)}=f_S^{(n-1)}$ if $x_1$ is erased; in that case, it remains only to determine $f_S^{\left(n\right)}$ when $x_1$ is not erased. However, observing Equation (95), it is apparent that the optimal choice of $f_S^{\left(n\right)}$ should satisfy two contradicting goals—on the one hand, to match the order induced by

$${\displaystyle \sum _{S\subseteq \left[n\right]:1\notin S}}Pr\left(S\right)·f_S\left(x^n\right)$$

(102)

and, on the other hand, to minimize the average guessing time of

$${\displaystyle \sum _{S\subseteq \left[n\right]:1\in S}}Pr\left(S\right)·f_S\left(x^n\right).$$

(103)

It is easy to see that taking a greedy approach toward satisfying the second goal would result in $f_S^{\left(n\right)}\left(x^n\right)=x_1$ if $1\in S$ and performing the recursion steps would indeed lead to a greedy dictator function. Interestingly, taking a greedy approach toward satisfying the first goal would also lead to a greedy dictator function, but one which operates on a cyclic permutation of the inputs (specifically, Equation (99) applied to $(y_2^n,y_1)$). Nonetheless, it is not clear that choosing ${\left\{f_S^{\left(n\right)}\right\}}_{S:1\in S}$ with some loss in the average guessing time induced by Equation (103) could not lead to a gain in the second goal (matching the order of Equation (102)), which outweighs that loss.

Another possible technique is majorization. It is known that, if one probability distribution majorizes another, then all the nonnegative guessing-moments of the first are no greater than the corresponding moments of the second ([29], Proposition 1). (The proof in Reference [29] is only for $s=1$, but it is easily extended to the general $s>0$ case.) Hence, one approach toward identifying the optimal function could be to try and find a function in which induced posterior distributions majorize the corresponding posteriors that induces by any other functions with the same bias (it is of course not clear that such a function even exists). This approach unfortunately fails for the greedy dictator. For example, the posterior distributions induced by setting $f_S$ to be majority functions are not always majorized by those induces by the greedy dictator (although they seem to be “almost” majorized) even though the average guessing time of greedy dictator is lower (this happens, e.g., for $n=5$ and $ϵ=0.4$). In fact, the guessing moments for greedy dictator seem to be better than these of majority irrespective of the value of s.