Distributed Hypothesis Testing over a Noisy Channel: Error-Exponents Trade-Off

Abstract

A two-terminal distributed binary hypothesis testing problem over a noisy channel is studied. The two terminals, called the observer and the decision maker, each has access to n independent and identically distributed samples, denoted by $\mathbf{U}$ and $\mathbf{V}$, respectively. The observer communicates to the decision maker over a discrete memoryless channel, and the decision maker performs a binary hypothesis test on the joint probability distribution of $(\mathbf{U},\mathbf{V})$ based on $\mathbf{V}$ and the noisy information received from the observer. The trade-off between the exponents of the type I and type II error probabilities is investigated. Two inner bounds are obtained, one using a separation-based scheme that involves type-based compression and unequal error-protection channel coding, and the other using a joint scheme that incorporates type-based hybrid coding. The separation-based scheme is shown to recover the inner bound obtained by Han and Kobayashi for the special case of a rate-limited noiseless channel, and also the one obtained by the authors previously for a corner point of the trade-off. Finally, we show via an example that the joint scheme achieves a strictly tighter bound than the separation-based scheme for some points of the error-exponents trade-off.

1. Introduction

Hypothesis testing (HT), which refers to the problem of choosing between one or more alternatives based on available data, plays a central role in statistics and information theory. Distributed HT (DHT) problems arise in situations where the test data are scattered across multiple terminals, and need to be communicated to a central terminal, called the decision maker, which performs the hypothesis test. The need to jointly optimize the communication scheme and the hypothesis test makes DHT problems much more challenging than their centralized counterparts. Indeed, while an efficient characterization of the optimal hypothesis test and its asymptotic performance is well known in the centralized setting, thanks to [1,2,3,4,5], the same problem in even the simplest distributed setting remains open, except for some special cases (see [6,7,8,9,10,11]).

In this work, we consider a DHT problem with two parties, an observer and a decision maker, such that the former communicates to the latter over a noisy channel. The observer and the decision maker each has access to independent and identically distributed samples, denoted by $\mathbf{U}$ and $\mathbf{V}$, respectively. Based on the information received from the observer and its own observations $\mathbf{V}$, the decision maker performs a binary hypothesis test on the joint distribution of $(\mathbf{U},\mathbf{V})$. Our goal is to characterize the trade-off between the best achievable rate of decay (or exponent) of the type I and type II error probabilities with respect to the sample size. We will refer to this problem as DHT over a noisy channel, and its special instance with the noisy channel replaced by a rate-limited noiseless channel as DHT over a noiseless channel.

1.1. Background

Distributed statistical inference problems were first conceived in [12] and the information-theoretic study of DHT over a noiseless channel was first investigated in [6], where the objective is to characterize Stein’s exponent ${\kappa }_{\mathrm{se}}\left(ϵ\right)$, i.e., the optimal type II error-exponent subject to the type I error probability constrained to be at most $ϵ\in (0,1)$. The authors therein established a multi-letter characterization of this quantity including a strong converse, which shows that ${\kappa }_{\mathrm{se}}\left(ϵ\right)$ is independent of $ϵ$. Furthermore, a single-letter characterization of ${\kappa }_{\mathrm{se}}\left(ϵ\right)$ is obtained for a special case of HT known as testing against independence (TAI), in which the joint distribution factors as a product of the marginal distributions under the alternative hypothesis. Improved lower bounds on ${\kappa }_{\mathrm{se}}\left(ϵ\right)$ were subsequently obtained in [7,8], respectively, and the strong converse was extended to zero-rate settings [13]. While all the aforementioned works focus on ${\kappa }_{\mathrm{se}}\left(ϵ\right)$, the trade-off between the exponents of both the type I and type II error probabilities in the same setting was first explored in [14].

In the recent years, there has been a renewed interest in distributed statistical inference problems motivated by emerging machine learning applications to be served at the wireless edge, particularly in the context of semantic communications in 5G/6G communication systems [15,16]. Several extensions of the DHT over a noiseless channel problem have been studied, such as generalizations to multi-terminal settings [9,17,18,19,20,21], DHT under security or privacy constraints [22,23,24,25], DHT with lossy compression [26], interactive settings [27,28], successive refinement models [29], and more. Improved bounds have been obtained on the type I and type II error-exponents region [30,31], and on ${\kappa }_{\mathrm{se}}\left(ϵ\right)$ for testing correlation between bivariate standard normal distributions [32]. In the simpler zero-rate communication setting, there has been some progress in terms of second-order optimal schemes [33], geometric interpretation of type I and type II error-exponent region [34], and characterization of ${\kappa }_{\mathrm{se}}\left(ϵ\right)$ for sequential HT [35]. DHT over noisy communication channels with the goal of characterizing ${\kappa }_{\mathrm{se}}\left(ϵ\right)$ has been considered in [10,11,36,37].

1.2. Contributions

In this work, our objective is to explore the trade-off between the type I and type II error-exponents for DHT over a noisy channel. This problem is a generalization of [14] from noiseless rate-limited channels to noisy channels, and also of [10,11] from a type I error probability constraint to a positive type I error-exponent constraint.

Our main contributions can be summarized as follows:

(i)

We obtain an inner bound (Theorem 1) on the error-exponents trade-off by using a separate HT and channel coding scheme (SHTCC) that is a combination of a type-based (type here refers to the empirical probability distribution of a sequence, see [38]) quantize-bin strategy and unequal error-protection scheme of [39]. This result is shown to recover the bounds established in [10,14]. Furthermore, we evaluate Theorem 1 for two important instances of DHT, namely TAI and its opposite, i.e., testing against dependence (TAD) in which the joint distribution under the null hypothesis factors as a product of marginal distributions.

(ii)

We also obtain a second inner bound (Theorem 2) on the error-exponents trade-off by using a joint HT and channel coding scheme (JHTCC) based on hybrid coding [40]. Subsequently, we show via an example that the JHTCC scheme strictly outperforms the SHTCC scheme for some points on the error-exponent trade-off.

While the above schemes are inspired from those in [10], which have been proposed with the goal of maximizing the type II error-exponent, novel modifications in its design and analysis are required when considering both of the error-exponents. More specifically, the schemes presented here perform separate quantization-binning or hybrid coding on each individual source sequence type at the observer/encoder (as opposed to a typical ball in [10]) with the corresponding reverse operation implemented at the decision-maker/decoder. This necessitates a different analysis to compute the probabilities of the various error events contributing to the overall error-exponents. We finally mention that the DHT problem considered here was recently investigated in [41], where an inner bound on the error-exponents trade-off (Theorem 2 in [41]) is obtained using a combination of a type-based quantization scheme and unequal error protection scheme of [42] with two special messages. A qualitative comparison between Theorem 2 and Theorem 2 in [41] seems to suggest that the JHTCC scheme here uses a stronger decoding rule depending jointly on the source-channel statistics. In comparison, the metric used at the decoder for the scheme in [41] factors as the sum of two metrics, one which depends only on the source statistics, and the other which depends only on the channel statistics. Importantly, this hints that the inner bound achieved by JHTCC scheme is not subsumed by that in [41]. That said, a direct computational comparison appears difficult, as evaluating the latter requires optimization over several parameters as mentioned in the last paragraph of [41].

The remainder of the paper is organized as follows. Section 2 formulates the operational problem along with the required definitions. The main results are presented in Section 3. The proofs are furnished in Section 4. Finally, concluding remarks are given in Section 5.

2. Preliminaries

2.1. Notation

We use the following notation. All logarithms are with respect to the natural base e. $\mathbb{N}$, $\mathbb{R}$, ${\mathbb{R}}_{\ge 0}$, and $\overline{\mathbb{R}}$ denotes the set of natural, real, non-negative real and extended real numbers, respectively. For $a,b\in {\mathbb{R}}_{\ge 0}$, $[a:b]:=\{n\in \mathbb{N}:a\le n\le b\}$ and $\left[b\right]:=[1:b]$. Calligraphic letters, e.g., $\mathcal{X}$, denote sets, while ${\mathcal{X}}^c$ and $\left|\mathcal{X}\right|$ stands for its complement and cardinality, respectively. For $n\in \mathbb{N}$, ${\mathcal{X}}^n$ denotes the n-fold Cartesian product of $\mathcal{X}$, and $x^n=(x_1,\cdots ,x_n)$ denotes an element of ${\mathcal{X}}^n$. Bold-face letters denote vectors or sequences, e.g., $\mathbf{x}$ for $x^n$; its length n will be clear from the context. For $i,j\in \mathbb{N}$ such that $i\le j$, $x_i^j:=(x_i,x_{i+1},\cdots ,x_j)$, the subscript is omitted when $i=1$. ${\mathrm{𝟙}}_{\mathcal{A}}$ denotes the indicator of set $\mathcal{A}$. For a real sequence ${\left\{a_n\right\}}_{n\in \mathbb{N}}$, $a_n\stackrel{\left(n\right)}{\to }b$ stands for ${lim}_{n\to \infty }{a}_n=b$, while $a_n\gtrsim b$ denotes ${lim}_{n\to \infty }{a}_n\ge b$. Similar notations apply for other inequalities. $O(·)$, $\Omega (·)$ and $o(·)$ denote standard asymptotic notations.

Random variables and their realizations are denoted by uppercase and lowercase letters, respectively, e.g., X and x. Similar conventions apply for random vectors and their realizations. The set of all probability mass functions (PMFs) on a finite set $\mathcal{X}$ is denoted by $\mathcal{P}\left(\mathcal{X}\right)$. The joint PMF of two discrete random variables X and Y is denoted by $P_{XY}$; the corresponding marginals are $P_X$ and $P_Y$. The conditional PMF of X given Y is represented by $P_{X|Y}$. Expressions such as $P_{XY}=P_X{P}_{Y|X}$ are to be understood as pointwise equality, i.e., $P_{XY}(x,y)=P_X\left(x\right)P_{Y|X}\left(y\right|x)$, for all $(x,y)\in \mathcal{X}\times \mathcal{Y}$. When the joint distribution of a triple $(X,Y,Z)$ factors as $P_{XYZ}=P_{XY}{P}_{Z|X}$, these variables form a Markov chain $X-Y-Z$. When X and Y are statistically independent, we write $X⫫Y$. If the entries of $X^n$ are drawn in an independent and identically distributed manner, i.e., if $P_{X^n}\left(\mathbf{x}\right)={\prod }_{i=1}^n{P}_X\left(x_i\right)$, $\forall \mathbf{x}\in {\mathcal{X}}^n$, then the PMF $P_{X^n}$ is denoted by $P_X^{\otimes n}$. Similarly, if $P_{Y^n|X^n}\left(\mathbf{y}\right|\mathbf{x})={\prod }_{i=1}^n{P}_{Y|X}\left(y_i\right|x_i)$ for all $(\mathbf{x},\mathbf{y})\in {\mathcal{X}}^n\times {\mathcal{Y}}^n$, then we write $P_{Y|X}^{\otimes n}$ for $P_{Y^n|X^n}$. The conditional product PMF given a fixed $\mathbf{x}\in {\mathcal{X}}^n$ is designated by $P_{Y|X}^{\otimes n}(·|\mathbf{x})$. The probability measure induced by a PMF P is denoted by ${\mathbb{P}}_P$. The corresponding expectation is designated by ${\mathbb{E}}_P$.

The type or empirical PMF of a sequence $\mathbf{x}\in {\mathcal{X}}^n$ is designated by $P_{\mathbf{x}}$, i.e., $P_{\mathbf{x}}\left(x\right):=\frac{1}{n}{\sum }_{i=1}^n{\mathrm{𝟙}}_{\{x_i=x\}}$. The set of n-length sequences $\mathbf{x}\in {\mathcal{X}}^n$ of type $P_X$ is ${\mathcal{T}}_n(P_X,{\mathcal{X}}^n):=\{\mathbf{x}\in {\mathcal{X}}^n:P_{\mathbf{x}}=P_X\}$. Whenever the underlying alphabet ${\mathcal{X}}^n$ is clear from the context, ${\mathcal{T}}_n(P_X,{\mathcal{X}}^n)$ is simplified to ${\mathcal{T}}_n\left(P_X\right)$. The set of all possible types of n-length sequences $\mathbf{x}\in {\mathcal{X}}^n$ is $\mathcal{T}\left({\mathcal{X}}^n\right):=\left\{P_X\in \mathcal{P}\left(\mathcal{X}\right):\left|{\mathcal{T}}_n(P_X,{\mathcal{X}}^n)\right|\ge 1\right\}$. Similar notations are used for larger combinations, e.g., $P_{\mathbf{x}\mathbf{y}}$, ${\mathcal{T}}_n(P_{XY},\mathcal{X}\times \mathcal{Y})$ and $\mathcal{T}({\mathcal{X}}^n\times {\mathcal{Y}}^n)$. For a given $\mathbf{x}\in {\mathcal{T}}_n(P_X,{\mathcal{X}}^n)$ and a conditional PMF $P_{Y|X}$, ${\mathcal{T}}_n(P_{Y|X},\mathbf{x}):=\{\mathbf{y}\in {\mathcal{Y}}^n:(\mathbf{x},\mathbf{y})\in {\mathcal{T}}_n(P_{XY},{\mathcal{X}}^n\times {\mathcal{Y}}^n)\}$ stands for the $P_{Y|X}$-conditional type class of $\mathbf{x}$.

For PMFs $P,Q\in \mathcal{P}\left(\mathcal{X}\right)$, the Kullback–Leibler (KL) divergence between P and Q is $D\left(P\left|\right|Q\right):={\sum }_{x\in \mathcal{X}}P\left(x\right)log\left(P\left(x\right)/Q\left(x\right)\right)$. The conditional KL divergence between $P_{Y|X}$ and $Q_{Y|X}$ given $P_X$ is $D\left(P_{Y|X}\left|\right|Q_{Y|X}|P_X\right):={\sum }_{x\in \mathcal{X}}{P}_X\left(x\right)D\left(P_{Y|X}(·|x)\left|\right|Q_{Y|X}(·|x)\right)$. The mutual information and entropy terms are denoted by $I_P(·)$ and $H_P(·)$, respectively, where P denotes the PMF of the relevant random variables. When the PMF is clear from the context, the subscript is omitted. For $(\mathbf{x},\mathbf{y})\in {\mathcal{X}}^n\times {\mathcal{Y}}^n$, the empirical conditional entropy of $\mathbf{y}$ given $\mathbf{x}$ is $H_e\left(\mathbf{y}\right|\mathbf{x}):=H_P\left(\tilde{Y}\right|\tilde{X})$, where $P_{\tilde{X}\tilde{Y}}=P_{\mathbf{x}\mathbf{y}}$. For a given function $f:\mathcal{Z}\to \mathbb{R}$ and a random variable $Z\sim P_Z$, the log-moment generating function of Z with respect to f is ${\psi }_{P_Z,f}\left(\lambda \right):=log{\mathbb{E}}_{P_Z}\left[e^{\lambda f\left(Z\right)}\right]$ whenever the expectation exists. Finally, let

$$\begin{array}{c}\hfill {\psi }_{P_Z,f}^{*}\left(\theta \right):=\underset{\lambda \in \mathbb{R}}{sup}\theta \lambda -{\psi }_{P_Z,f}\left(\lambda \right),\end{array}$$

(1)

denote the rate function (see, e.g., Definition 15.5 in [43]).

2.2. Problem Formulation

Let $\mathcal{U}$, $\mathcal{V}$, $\mathcal{X}$ and $\mathcal{Y}$ be finite sets, and $n\in \mathbb{N}$. The DHT over a noisy channel setting is depicted in Figure 1. Herein, the observer and the decision maker observe n independent and identically distributed samples, denoted by $\mathbf{u}$ and $\mathbf{v}$, respectively. Based on its observations $\mathbf{u}$, the observer outputs a sequence $\mathbf{x}\in {\mathcal{X}}^n$ as the channel input sequence (note that the ratio of the number of channel uses to the number of data samples, termed the bandwidth ratio, is taken to be 1 for simplicity; however, our results easily generalize to arbitrary bandwidth ratios). The discrete memoryless channel (DMC) with transition kernel $P_{Y|X}$ produces a sequence $\mathbf{y}\in {\mathcal{Y}}^n$ according to the probability law $P_{Y|X}^{\otimes n}(·|\mathbf{x})$ as its output. We will assume that $P_{Y|X}(·|x)\ll P_{Y|X}(·|x^{\prime })$, $\forall (x,x^{\prime })\in {\mathcal{X}}^2$, where $P\ll Q$ indicates the absolute continuity of P with respect to Q. Based on its observations, $\mathbf{y}$ and $\mathbf{v}$, the decision maker performs binary HT on the joint probability distribution of $(\mathbf{U},\mathbf{V})$ with the null ($H_0$) and alternative ($H_1$) hypotheses given by

$$\begin{array}{cc}\hfill & H_0:(\mathbf{U},\mathbf{V})\sim P_{UV}^{\otimes n},\hfill \end{array}$$

(2a)

$$\begin{array}{cc}\hfill & H_1:(\mathbf{U},\mathbf{V})\sim Q_{UV}^{\otimes n}.\hfill \end{array}$$

(2b)

Figure 1. DHT over a noisy channel. The observer observes an n-length independent and identically distributed sequence $\mathbf{U}$, and transmits $\mathbf{X}$ over the DMC $P_{Y|X}^{\otimes n}$. Based on the channel output $\mathbf{Y}$ and the n-length independent and identically distributed sequence $\mathbf{V}$, the decision maker performs a binary HT to determine whether $(\mathbf{U},\mathbf{V})\sim P_{UV}^{\otimes n}$ or $(\mathbf{U},\mathbf{V})\sim Q_{UV}^{\otimes n}$.

The decision maker outputs $\hat{h}\in \hat{\mathcal{H}}:=\{0,1\}$ as the decision of the hypothesis test, where 0 and 1 denote $H_0$ and $H_1$, respectively.

A length-n DHT code $c_n$ is a pair of functions $(f_n,g_n)$, where

(i)

$f_n:{\mathcal{U}}^n\to \mathcal{P}\left({\mathcal{X}}^n\right)$ denotes the encoding function;

(ii)

$g_n:{\mathcal{V}}^n\times {\mathcal{Y}}^n\to \hat{\mathcal{H}}$ denotes a deterministic decision function specified by an acceptance region (for null hypothesis $H_0$) ${\mathcal{A}}_n\subseteq {\mathcal{V}}^n\times {\mathcal{Y}}^n$ as $g_n(\mathbf{v},\mathbf{y})=1-{\mathrm{𝟙}}_{\{(\mathbf{v},\mathbf{y})\in {\mathcal{A}}_n\}},\forall (\mathbf{v},\mathbf{y})\in {\mathcal{V}}^n\times {\mathcal{Y}}^n$.

We emphasize at this point that there is no loss in generality in restricting our attention to a deterministic decision function for the objective of characterizing the error-exponents trade-off in HT (for e.g., see Lemma 3 in [24])).

A code $c_n=(f_n,g_n)$ induces the joint PMFs $P_{\mathbf{U}\mathbf{V}\mathbf{X}\mathbf{Y}\hat{H}}^{\left(c_n\right)}$ and $Q_{\mathbf{U}\mathbf{V}\mathbf{X}\mathbf{Y}\hat{H}}^{\left(c_n\right)}$ under the null and alternative hypotheses, respectively, where

$$\begin{array}{c}\hfill P_{\mathbf{U}\mathbf{V}\mathbf{X}\mathbf{Y}\hat{H}}^{\left(c_n\right)}(\mathbf{u},\mathbf{v},\mathbf{x},\mathbf{y},\hat{h}):=P_{UV}^{\otimes n}(\mathbf{u},\mathbf{v})f_n\left(\mathbf{x}\right|\mathbf{u})P_{Y|X}^{\otimes n}\left(\mathbf{y}\right|\mathbf{x}){\mathrm{𝟙}}_{\left\{g_n(\mathbf{v},\mathbf{y})=\hat{h}\right\}},\end{array}$$

(3)

and

$$\begin{array}{c}\hfill Q_{\mathbf{U}\mathbf{V}\mathbf{X}\mathbf{Y}\hat{H}}^{\left(c_n\right)}(\mathbf{u},\mathbf{v},\mathbf{x},\mathbf{y},\hat{h}):=Q_{UV}^{\otimes n}(\mathbf{u},\mathbf{v})f_n\left(\mathbf{x}\right|\mathbf{u})P_{Y|X}^{\otimes n}\left(\mathbf{y}\right|\mathbf{x}){\mathrm{𝟙}}_{\left\{g_n(\mathbf{v},\mathbf{y})=\hat{h}\right\}},\end{array}$$

(4)

respectively. For a given code $c_n$, the type I and type II error probabilities are ${\alpha }_n\left(c_n\right):={\mathbb{P}}_{P^{\left(c_n\right)}}(\hat{H}=1)$ and ${\beta }_n\left(c_n\right):={\mathbb{P}}_{Q^{\left(c_n\right)}}(\hat{H}=0)$ respectively. The following definition formally states the error-exponents trade-off we aim to characterize.

Definition 1 

(Error-exponent region).An error-exponent pair $({\kappa }_{\alpha },{\kappa }_{\beta })\in {\mathbb{R}}_{\ge 0}^2$ is said to be achievable if there exists a sequence of codes ${\left\{c_n\right\}}_{n\in \mathbb{N}}$ such that

$$\underset{n\to \infty }{lim\; inf}-\frac{1}{n}log{\alpha }_n\left(c_n\right)\ge {\kappa }_{\alpha },$$

(5a)

$$\underset{n\to \infty }{lim\; inf}-\frac{1}{n}log{\beta }_n\left(c_n\right)\ge {\kappa }_{\beta }.$$

(5b)

The error-exponent region $\overline{\mathcal{R}}$ is the closure of the set of all achievable error-exponent pairs $({\kappa }_{\alpha },{\kappa }_{\beta })$. Set $\mathcal{R}:=\{\left({\kappa }_{\alpha },\kappa \left({\kappa }_{\alpha }\right)\right):{\kappa }_{\alpha }\in (0,{\kappa }_{\alpha }^{★})\}$, where ${\kappa }_{\alpha }^{★}=inf\{{\kappa }_{\alpha }:\kappa \left({\kappa }_{\alpha }\right)=0\}$ and $\kappa \left({\kappa }_{\alpha }\right):=sup\{{\kappa }_{\beta }:({\kappa }_{\alpha },{\kappa }_{\beta })\in \overline{\mathcal{R}}\}$.

We are interested in a computable characterization of $\mathcal{R}$, which pertains to the region of positive error-exponents (i.e., excluding the boundary points corresponding to Stein’s exponent). To this end, we present two inner bounds on $\mathcal{R}$ in the next section.

3. Main Results

In this section, we obtain two inner bounds on $\mathcal{R}$, first using a separation-based scheme which performs independent HT and channel coding, termed the SHTCC scheme, and the second via a joint HT and channel coding scheme that uses hybrid coding for communication between the observer and the decision maker.

3.1. Inner Bound on $\mathcal{R}$ via SHTCC Scheme

Let $\mathcal{S}=\mathcal{X}$ and $P_{SXY}=P_{SX}{P}_{Y|X}$ be a PMF under which $S-X-Y$ forms a Markov chain. For $x\in \mathcal{X}$, let ${\Lambda }_{x,P_{SXY}}\left(y\right):=log\left(P_{Y|X=x}\left(y\right)/P_{Y|S=x}\left(y\right)\right)$ and define

$$\begin{array}{c}\hfill E_{\mathrm{sp}}(P_{SX},\theta ):=\sum _{s\in \mathcal{S}}{P}_S\left(s\right){\psi }_{P_{Y|S=s},{\Lambda }_{s,P_{SXY}}}^{*}\left(\theta \right),\end{array}$$

where the rate function ${\psi }^{*}$ is defined in (1). For a fixed $P_{SX}$ and $R\ge 0$, let

$$\begin{array}{cc}{E}_{\mathrm{ex}}(R,P_{SX}):=\underset{\rho \ge 1}{max}-\rho R-\rho log(& \sum _{s,x,\tilde{x}}{P}_S\left(s\right)P_{X|S}\left(x\right|s)P_{X|S}\left(\tilde{x}\right|s)\hfill \\ & {\left(\sum _y{\left(P_{Y|X}\left(y\right|x)P_{Y|X}\left(y\right|\tilde{x})\right)}^{\frac{1}{2}}\right)}^{\frac{1}{\rho }}),\hfill & \end{array}$$

denote the expurgated exponent [38,44]. Let $\mathcal{W}$ be a finite set and $\mathfrak{F}$ denote the set of all continuous mappings from $\mathcal{P}\left(\mathcal{U}\right)$ to $\mathcal{P}\left(\mathcal{W}\right|\mathcal{U})$, where $\mathcal{P}\left(\mathcal{W}\right|\mathcal{U})$ is the set of all conditional distributions $P_{W|U}$. Set ${\theta }_{\mathrm{l}}\left(P_{SX}\right):={\sum }_{s\in \mathcal{S}}{P}_S\left(s\right)D\left(P_{Y|S=s}\left|\right|P_{Y|X=s}\right)$, ${\theta }_{\mathrm{u}}\left(P_{SX}\right):={\sum }_{s\in \mathcal{S}}{P}_S\left(s\right)D\left(P_{Y|X=s}\left|\right|P_{Y|S=s}\right)$, $\Theta \left(P_{SX}\right):=\left(-{\theta }_{\mathrm{l}}\left(P_{SX}\right),{\theta }_{\mathrm{u}}\left(P_{SX}\right)\right)$. Denote an arbitrary element of $\mathfrak{F}\times {\mathbb{R}}_{\ge 0}\times \mathcal{P}(\mathcal{S}\times \mathcal{X})\times \Theta \left(P_{SX}\right)$ by $(\omega ,R,P_{SX},\theta )$, and set

$$\begin{array}{cc}\hfill & \mathcal{L}\left({\kappa }_{\alpha }\right):=\left\{\begin{array}{cc}\hfill (\omega ,R,P_{SX},\theta ):& \zeta ({\kappa }_{\alpha },\omega )-\rho ({\kappa }_{\alpha },\omega )\le R<I_P(X;Y|S),P_{SXY}=P_{SX}{P}_{Y|X}\hfill \\ \hfill & \min \left\{E_{\mathrm{sp}}(P_{SX},\theta ),E_{\mathrm{ex}}\left(R,P_{SX}\right),E_{\mathrm{b}}({\kappa }_{\alpha },\omega ,R)\right\}\ge {\kappa }_{\alpha }\hfill \end{array}\right\},\hfill \end{array}$$

$$\begin{array}{cc}\hfill & \hat{\mathcal{L}}({\kappa }_{\alpha },\omega ):=\left\{P_{\hat{U}\hat{V}\hat{W}}:D\left(P_{\hat{U}\hat{V}\hat{W}}\left|\right|P_{UV\hat{W}}\right)\le {\kappa }_{\alpha },P_{\hat{W}|\hat{U}}=\omega \left(P_{\hat{U}}\right),P_{UV\hat{W}}=P_{UV}{P}_{\hat{W}|\hat{U}}\right\},\hfill \\ \hfill & E_{\mathrm{b}}({\kappa }_{\alpha },\omega ,R):=\left\{\begin{array}{cc}R-\zeta ({\kappa }_{\alpha },\omega )+\rho ({\kappa }_{\alpha },\omega ),\hfill & \mathrm{if} 0\le R<\zeta ({\kappa }_{\alpha },\omega ),\hfill \\ \infty ,\hfill & \mathrm{otherwise},\hfill \end{array}\right.\hfill \end{array}$$

(6a)

$$\begin{array}{cc}\hfill & \zeta ({\kappa }_{\alpha },\omega ):=\underset{\begin{array}{c}{P}_{\hat{U}\hat{W}}:\exists P_{\hat{V}},P_{\hat{U}\hat{V}\hat{W}}\in \hat{\mathcal{L}}({\kappa }_{\alpha },\omega )\end{array}}{max}{I}_P(\hat{U};\hat{W}),\hfill \end{array}$$

(6b)

$$\begin{array}{cc}\hfill & \rho ({\kappa }_{\alpha },\omega ):=\underset{\begin{array}{c}{P}_{\hat{V}\hat{W}}:\exists P_{\hat{U}},P_{\hat{U}\hat{V}\hat{W}}\in \hat{\mathcal{L}}({\kappa }_{\alpha },\omega )\end{array}}{\min }{I}_P(\hat{V};\hat{W}),\hfill \\ \hfill & E_1({\kappa }_{\alpha },\omega ):=\underset{\begin{array}{c}(P_{\tilde{U}\tilde{V}\tilde{W}},Q_{\tilde{U}\tilde{V}\tilde{W}})\in {\mathcal{T}}_1({\kappa }_{\alpha },\omega )\end{array}}{\min }D(P_{\tilde{U}\tilde{V}\tilde{W}}\left|\right|Q_{\tilde{U}\tilde{V}\tilde{W}}),\hfill \end{array}$$

(6c)

$$\begin{array}{cc}\hfill & E_2({\kappa }_{\alpha },\omega ,R):=\left\{\begin{array}{cc}{\displaystyle \underset{\begin{array}{c}(P_{\tilde{U}\tilde{V}\tilde{W}},Q_{\tilde{U}\tilde{V}\tilde{W}})\in {\mathcal{T}}_2({\kappa }_{\alpha },\omega )\end{array}}{\min }}D(P_{\tilde{U}\tilde{V}\tilde{W}}\left|\right|Q_{\tilde{U}\tilde{V}\tilde{W}})+E_{\mathrm{b}}({\kappa }_{\alpha },\omega ,R),\hfill & \mathrm{if} R<\zeta ({\kappa }_{\alpha },\omega ),\hfill \\ \infty ,\hfill & \mathrm{otherwise},\hfill \end{array}\right.\hfill \end{array}$$

$$\begin{array}{cc}\hfill & E_3({\kappa }_{\alpha },\omega ,R,P_{SX}):=\left\{\begin{array}{cc}{\displaystyle \underset{\begin{array}{c}(P_{\tilde{U}\tilde{V}\tilde{W}},Q_{\tilde{U}\tilde{V}\tilde{W}})\in {\mathcal{T}}_3({\kappa }_{\alpha },\omega )\end{array}}{\min }}D(P_{\tilde{U}\tilde{V}\tilde{W}}\left|\right|Q_{\tilde{U}\tilde{V}\tilde{W}})+E_{\mathrm{b}}({\kappa }_{\alpha },\omega ,R)+E_{\mathrm{ex}}\left(R,P_{SX}\right),\hfill & \\ \mathrm{if} R<\zeta ({\kappa }_{\alpha },\omega ),\hfill & \\ {\displaystyle \underset{\begin{array}{c}(P_{\tilde{U}\tilde{V}\tilde{W}},Q_{\tilde{U}\tilde{V}\tilde{W}})\in {\mathcal{T}}_3({\kappa }_{\alpha },\omega )\end{array}}{\min }}D(P_{\tilde{U}\tilde{V}\tilde{W}}\left|\right|Q_{\tilde{U}\tilde{V}\tilde{W}})+\rho ({\kappa }_{\alpha },\omega )+E_{\mathrm{ex}}\left(R,P_{SX}\right),\hfill & \\ \mathrm{otherwise},\hfill \end{array}\right.\hfill \end{array}$$

$$\begin{array}{cc}\hfill & E_4({\kappa }_{\alpha },\omega ,R,P_{SX},\theta ):=\left\{\begin{array}{cc}{\displaystyle \underset{P_{\hat{V}}:P_{\hat{U}\hat{V}\hat{W}}\in \hat{\mathcal{L}}({\kappa }_{\alpha },\omega )}{\min }}D(P_{\hat{V}}\left|\right|Q_V)+E_{\mathrm{b}}({\kappa }_{\alpha },\omega ,R)+E_{\mathrm{m}}(P_{SX},\theta )-\theta ,\hfill & \\ \mathrm{if} R<\zeta ({\kappa }_{\alpha },\omega ),\hfill & \\ {\displaystyle \underset{P_{\hat{V}}:P_{\hat{U}\hat{V}\hat{W}}\in \hat{\mathcal{L}}({\kappa }_{\alpha },\omega )}{\min }}D(P_{\hat{V}}\left|\right|Q_V)+\rho ({\kappa }_{\alpha },\omega )+E_{\mathrm{m}}(P_{SX},\theta )-\theta ,\hfill & \\ \mathrm{otherwise},\hfill \end{array}\right.\hfill \\ \hfill & \mathrm{where},\hfill \end{array}$$

$$\begin{array}{cc}\hfill & {\mathcal{T}}_1({\kappa }_{\alpha },\omega ):=\left\{\begin{array}{cc}(P_{\tilde{U}\tilde{V}\tilde{W}},Q_{\tilde{U}\tilde{V}\tilde{W}}):\hfill & P_{\tilde{U}\tilde{W}}=P_{\hat{U}\hat{W}},P_{\tilde{V}\tilde{W}}=P_{\hat{V}\hat{W}},Q_{\tilde{U}\tilde{V}\tilde{W}}\hfill \\ & \hfill :=Q_{UV}{P}_{\tilde{W}|\tilde{U}}\mathrm{for} \mathrm{some}{P}_{\hat{U}\hat{V}\hat{W}}\in \hat{\mathcal{L}}({\kappa }_{\alpha },\omega )\end{array}\right\},\hfill \\ \hfill & {\mathcal{T}}_2({\kappa }_{\alpha },\omega ):=\left\{\begin{array}{cc}(P_{\tilde{U}\tilde{V}\tilde{W}},Q_{\tilde{U}\tilde{V}\tilde{W}}):\hfill & P_{\tilde{U}\tilde{W}}=P_{\hat{U}\hat{W}},P_{\tilde{V}}=P_{\hat{V}},H_P\left(\tilde{W}\right|\tilde{V})\ge H_P\left(\hat{W}\right|\hat{V}),\hfill \\ & Q_{\tilde{U}\tilde{V}\tilde{W}}:=Q_{UV}{P}_{\tilde{W}|\tilde{U}}\mathrm{for} \mathrm{some}{P}_{\hat{U}\hat{V}\hat{W}}\in \hat{\mathcal{L}}({\kappa }_{\alpha },\omega )\hfill \end{array}\right\},\hfill \\ \hfill & {\mathcal{T}}_3({\kappa }_{\alpha },\omega ):=\left\{\begin{array}{cc}(P_{\tilde{U}\tilde{V}\tilde{W}},Q_{\tilde{U}\tilde{V}\tilde{W}}):\hfill & P_{\tilde{U}\tilde{W}}=P_{\hat{U}\hat{W}},P_{\tilde{V}}=P_{\hat{V}},Q_{\tilde{U}\tilde{V}\tilde{W}}:=Q_{UV}{P}_{\tilde{W}|\tilde{U}}\hfill \\ & \mathrm{for} \mathrm{some}{P}_{\hat{U}\hat{V}\hat{W}}\in \hat{\mathcal{L}}({\kappa }_{\alpha },\omega )\hfill \end{array}\right\}.\hfill \end{array}$$

(6d)

We have the following lower bound for $\kappa \left({\kappa }_{\alpha }\right)$, which translates to an inner bound for $\mathcal{R}$.

Theorem 1 

(Inner bound via SHTCC scheme). $\kappa \left({\kappa }_{\alpha }\right)\ge {\kappa }_{\mathrm{s}}^{★}\left({\kappa }_{\alpha }\right),$ where

$$\begin{array}{cc}\hfill {\kappa }_{\mathrm{s}}^{★}\left({\kappa }_{\alpha }\right)& :=\underset{\begin{array}{c}(\omega ,R,P_{SX},\theta )\in \mathcal{L}\left({\kappa }_{\alpha }\right)\end{array}}{max}\min \{E_1({\kappa }_{\alpha },\omega ),E_2({\kappa }_{\alpha },\omega ,R),E_3({\kappa }_{\alpha },\omega ,R,P_{SX}),\hfill \\ \hfill & E_4({\kappa }_{\alpha },\omega ,R,P_{SX},\theta )\}.\hfill \end{array}$$

(7)

The proof of Theorem 1 is presented in Section 4.1. The SHTCC scheme, which achieves the error-exponent pair $({\kappa }_{\alpha },{\kappa }_{\mathrm{s}}^{★}\left({\kappa }_{\alpha }\right))$, is a coding scheme analogous to separate source and channel coding for the lossy transmission of a source over a communication channel with correlated side-information at the receiver [45], however, with the objective of reliable HT. In this scheme, the source samples are first compressed to an index, which acts as the message to be transmitted over the channel. But, in contrast to standard communication problems, there is a need to protect certain messages more reliably than others; hence, an unequal error-protection scheme [39,42] is used. To describe briefly, the SHTCC scheme involves $\left(i\right)$ the quantization and binning of $\mathbf{u}$ sequences, whose type $P_{\mathbf{u}}$ is within a ${\kappa }_{\alpha }$-neighborhood (in terms of KL divergence) of $P_U$, using $\mathbf{V}$ as side information at the decision maker for decoding, and $\left(ii\right)$ unequal error-protection channel coding scheme in [39] for protecting a special message which informs the decision maker that $P_{\mathbf{u}}$ lies outside the ${\kappa }_{\alpha }$-neighborhood of $P_U$. The output of the channel decoder is processed by an empirical conditional entropy decoder which recovers the quantization codeword with the least conditional entropy with $\mathbf{V}$. Since this decoder depends only on the empirical distributions of the observations, it is universal and useful in the hypothesis testing context, where multiple distributions are involved (as was first noted in [8]). The various factors $E_1$ to $E_4$ in (7) have natural interpretations in terms of events that could possibly result in a hypothesis testing error. Specifically, $E_1$ and $E_2$ correspond to the error events arising due to quantization and binning, respectively, while $E_3$ and $E_4$ correspond to the error events of wrongly decoding an ordinary channel codeword and special message codeword, respectively.

Remark 1 

(Generalization of Han–Kobayashi inner bound). In Theorem 1 in [14], Han and Kobayashi obtained an inner bound on $\mathcal{R}$ for DHT over a noiseless channel. At a high level, their coding scheme involves type-based quantization of $\mathbf{u}\in {\mathcal{U}}^n$ sequences, whose type $P_{\mathbf{u}}$ lies within a ${\kappa }_{\alpha }$-neighborhood of $P_U$, where ${\kappa }_{\alpha }$ is the desired type I error-exponent. As a corollary, Theorem 1 recovers the lower bound for $\kappa \left({\kappa }_{\alpha }\right)$ obtained in [14] by $\left(i\right)$ setting $E_{\mathrm{ex}}\left(R,P_{SX}\right)$, $E_{\mathrm{m}}(P_{SX},\theta )$ and $E_{\mathrm{m}}(P_{SX},\theta )-\theta$ to ∞, which hold when the channel is noiseless, and $\left(ii\right)$ maximizing over the set $\left\{(\omega ,R,P_{SX},\theta )\in \mathfrak{F}\times {\mathbb{R}}_{\ge 0}\times \mathcal{P}(\mathcal{S}\times \mathcal{X})\times \Theta \left(P_{SX}\right):\zeta ({\kappa }_{\alpha },\omega )\le R<I_P(X;Y|S),P_{SXY}:=P_{SX}{P}_{Y|X}\right\}\subseteq \mathcal{L}\left({\kappa }_{\alpha }\right)$ in (7). Then, note that the terms $E_2({\kappa }_{\alpha },\omega ,R)$, $E_3({\kappa }_{\alpha },\omega ,R,P_{SX})$ and $E_4({\kappa }_{\alpha },\omega ,R,P_{SX},\theta )$ all equal ∞, and thus the inner bound in Theorem 1 reduces to that given in Theorem 1 in [14].

Remark 2 

(Improvement via time-sharing). Since the lower bound on $\kappa \left({\kappa }_{\alpha }\right)$ in Theorem 1 is not necessarily concave, a tighter bound can be obtained using the technique of time-sharing similar to Theorem 3 in [14]. We omit its description, as it is cumbersome, although straightforward.

Theorem 1 also recovers the lower bound for the optimal type II error-exponent for a fixed type I error probability constraint established in Theorem 2 in [10] by letting ${\kappa }_{\alpha }\to 0$. The details are provided in Appendix A. Further, specializing the lower bound in Theorem 1 to the case of TAI, i.e., when $Q_{UV}=P_U{P}_V$, we obtain the following corollary which characterizes the optimal type II error-exponent for TAI established in Proposition 7 in [10] as a special case.

Corollary 1 

(Inner bound for TAI). Let $P_{UV}\in \mathcal{P}(\mathcal{U}\times \mathcal{V})$ be an arbitrary distribution and $Q_{UV}=P_U{P}_V$. Then,

$$\begin{array}{c}\hfill \kappa \left({\kappa }_{\alpha }\right)\ge {\kappa }_{\mathrm{s}}^{★}\left({\kappa }_{\alpha }\right)\ge {\kappa }_{\mathrm{i}}^{★}\left({\kappa }_{\alpha }\right),\end{array}$$

(8)

where

$$\begin{array}{cc}\hfill & {\kappa }_{\mathrm{i}}^{★}\left({\kappa }_{\alpha }\right):=\underset{\begin{array}{c}(\omega ,P_{SX},\theta )\in {\mathcal{L}}^{★}\left({\kappa }_{\alpha }\right)\end{array}}{max}\min \left\{E_1^{\mathrm{i}}({\kappa }_{\alpha },\omega ),E_2^{\mathrm{i}}({\kappa }_{\alpha },\omega ,P_{SX}),E_3^{\mathrm{i}}({\kappa }_{\alpha },\omega ,P_{SX},\theta )\right\},\hfill \\ \hfill & {\mathcal{L}}^{★}\left({\kappa }_{\alpha }\right):=\left\{\begin{array}{cc}\hfill & (\omega ,P_{SX},\theta )\in \mathfrak{F}\times \mathcal{P}(\mathcal{S}\times \mathcal{X})\times \Theta \left(P_{SX}\right):\zeta ({\kappa }_{\alpha },\omega )<I_P(X;Y|S),\hfill \\ \hfill & P_{SXY}:=P_{SX}{P}_{Y|X},\min \left\{E_{\mathrm{sp}}(P_{SX},\theta ),E_{\mathrm{ex}}\left(\zeta ({\kappa }_{\alpha },\omega ),P_{SX}\right)\right\}\ge {\kappa }_{\alpha }\hfill \end{array}\right\},\hfill \\ \hfill & E_1^{\mathrm{i}}({\kappa }_{\alpha },\omega ):=\underset{\begin{array}{c}{P}_{\hat{V}\hat{W}}:\exists P_{\hat{U}\hat{V}\hat{W}}\in \hat{\mathcal{L}}({\kappa }_{\alpha },\omega )\end{array}}{\min }{I}_P(\hat{V};\hat{W})+D(P_{\hat{V}}\left|\right|P_V),\hfill \\ \hfill & E_2^{\mathrm{i}}({\kappa }_{\alpha },\omega ,P_{SX}):=\rho ({\kappa }_{\alpha },\omega )+E_{\mathrm{ex}}\left(\zeta ({\kappa }_{\alpha },\omega ),P_{SX}\right),\hfill \\ \hfill & E_3^{\mathrm{i}}({\kappa }_{\alpha },\omega ,P_{SX},\theta ):=\rho ({\kappa }_{\alpha },\omega )+E_{\mathrm{sp}}(P_{SX},\theta )-\theta ,\hfill \end{array}$$

(9)

and $\hat{\mathcal{L}}({\kappa }_{\alpha },\omega )$, $\zeta ({\kappa }_{\alpha },\omega )$ and $\rho ({\kappa }_{\alpha },\omega )$ are defined in (6a), (6b) and (6c), respectively. In particular,

$$\begin{array}{c}\hfill \underset{{\kappa }_{\alpha }\to 0}{lim}\kappa \left({\kappa }_{\alpha }\right)={\kappa }_{\mathrm{s}}^{★}\left(0\right)={\kappa }_{\mathrm{i}}^{★}\left(0\right)=\underset{\begin{array}{c}{P}_{W|U}:I_P(U;W)\le C\left(P_{Y|X}\right),\\ P_{UVW}=P_{UV}{P}_{W|U}\end{array}}{max}{I}_P(V;W),\end{array}$$

(10)

where $\left|\mathcal{W}\right|\le \left|\mathcal{U}\right|+1$ and $C\left(P_{Y|X}\right)$ denotes the capacity of the channel $P_{Y|X}$.

The proof of Corollary 1 is given in Section 4.2. Its achievability follows from a special case of the SHTCC scheme without binning at the encoder.

Next, we consider testing against dependence (TAD) for which $Q_{UV}$ is an arbitrary joint distribution and $P_{UV}=Q_U{Q}_V$. Theorem 1 specialized to TAD gives the following corollary.

Corollary 2 

(Inner bound for TAD). Let $Q_{UV}\in \mathcal{P}(\mathcal{U}\times \mathcal{V})$ be an arbitrary distribution and $P_{UV}=Q_U{Q}_V$. Then,

$$\begin{array}{c}\hfill \kappa \left({\kappa }_{\alpha }\right)\ge {\kappa }_{\mathrm{s}}^{★}\left({\kappa }_{\alpha }\right)={\kappa }_{\mathrm{d}}^{★}\left({\kappa }_{\alpha }\right):=\underset{\begin{array}{c}(\omega ,P_{SX},\theta )\\ \in {\mathcal{L}}^{★}\left({\kappa }_{\alpha }\right)\end{array}}{max}\min \left\{E_1^{\mathrm{d}}({\kappa }_{\alpha },\omega ),E_2^{\mathrm{d}}({\kappa }_{\alpha },\omega ,P_{SX}),E_3^{\mathrm{d}}(P_{SX},\theta )\right\},\end{array}$$

(11)

where

$$\begin{array}{cc}\hfill & E_1^{\mathrm{d}}({\kappa }_{\alpha },\omega ):=\underset{\begin{array}{c}(P_{\tilde{U}\tilde{V}\tilde{W}},Q_{\tilde{U}\tilde{V}\tilde{W}})\\ \in {\mathcal{T}}_1({\kappa }_{\alpha },\omega )\end{array}}{\min }D(P_{\tilde{U}\tilde{V}\tilde{W}}\left|\right|Q_{\tilde{U}\tilde{V}\tilde{W}})\ge \underset{\begin{array}{c}(P_{\hat{V}\hat{W}},Q_{V\hat{W}}):P_{\hat{U}\hat{V}\hat{W}}\in \hat{\mathcal{L}}({\kappa }_{\alpha },\omega ),\\ Q_{UV\hat{W}}=Q_{UV}{P}_{\hat{W}|\hat{U}}\end{array}}{\min }D(P_{\hat{V}\hat{W}}\left|\right|Q_{V\hat{W}}),\hfill \\ \hfill & E_2^{\mathrm{d}}({\kappa }_{\alpha },\omega ,P_{SX}):=E_{\mathrm{ex}}\left(\zeta ({\kappa }_{\alpha },\omega ),P_{SX}\right),\hfill \\ \hfill & E_3^{\mathrm{d}}(P_{SX},\theta ):=E_{\mathrm{sp}}(P_{SX},\theta )-\theta ,\hfill \end{array}$$

and $\hat{\mathcal{L}}({\kappa }_{\alpha },\omega )$, ${\mathcal{T}}_1({\kappa }_{\alpha },\omega )$ and ${\mathcal{L}}^{★}\left({\kappa }_{\alpha }\right)$ are given in (6a), (6d) and (9), respectively. In particular,

$$\begin{array}{c}\hfill \underset{{\kappa }_{\alpha }\to 0}{lim}\kappa \left({\kappa }_{\alpha }\right)\ge {\kappa }_{\mathrm{s}}^{★}\left(0\right)={\kappa }_{\mathrm{d}}^{★}\left(0\right)\ge {\kappa }_{\mathrm{TAD}}^{★},\end{array}$$

(12)

where

$$\begin{array}{c}\hfill {\kappa }_{\mathrm{TAD}}^{★}=\underset{\begin{array}{c}(P_{W|U},P_{SX}):\\ I_Q(W;U)\le I_P(X;Y|S),\\ Q_{UVW}=Q_{UV}{P}_{W|U},\\ P_{SXY}=P_{SX}{P}_{Y|X}\end{array}}{max}\min \left\{D(Q_V{Q}_W\left|\right|Q_{VW}),E_{\mathrm{ex}}\left(I_Q(U;W),P_{SX}\right),{\theta }_{\mathrm{l}}\left(P_{SX}\right)\right\},\end{array}$$

and $\left|\mathcal{W}\right|\le \left|\mathcal{U}\right|+1$.

The proof of Corollary 2 is given in Section 4.3. Note that the expression for ${\kappa }_{\mathrm{s}}^{★}\left({\kappa }_{\alpha }\right)$ given in (11) is relatively simpler to compute compared to that in Theorem 1. This will be handy in showing that the JHTCC scheme strictly outperforms the SHTCC scheme, which we highlight via an example in Section 3.3 below.

3.2. Inner Bound via JHTCC Scheme

It is well known that joint source-channel coding schemes offer advantages over separation-based coding schemes in several information theoretic problems, such as the transmission of correlated sources over a multiple-access channel [40,46] and the error-exponent in the lossless or lossy transmission of a source over a noisy channel [42,47]. Recently, it was shown via an example in [10] that joint schemes also achieve a strictly larger type II error-exponent in DHT problems compared to a separation-based scheme in some scenarios. Motivated by this, we present an inner bound on $\mathcal{R}$ using a generalization of the JHTCC scheme in [10].

Let $\mathcal{W}$ and $\mathcal{S}$ be arbitrary finite sets, and ${\mathfrak{F}}^{\prime }$ denote the set of all continuous mappings from $\mathcal{P}(\mathcal{U}\times \mathcal{S})$ to $\mathcal{P}\left(\mathcal{W}\right|\mathcal{U}\times \mathcal{S})$, where $\mathcal{P}\left(\mathcal{W}\right|\mathcal{U}\times \mathcal{S})$ is the set of all conditional distributions $P_{W|US}$. Let $\left(P_S,{\omega }^{\prime }(·,P_S),P_{X|USW},P_{X^{\prime }|US}\right)$ denote an arbitrary element of $\mathcal{P}\left(\mathcal{S}\right)\times {\mathfrak{F}}^{\prime }\times \mathcal{P}\left(\mathcal{X}\right|\mathcal{U}\times \mathcal{S}\times \mathcal{W})\times \mathcal{P}\left(\mathcal{X}\right|\mathcal{U}\times \mathcal{S})$, and define $\begin{array}{cc}& {\mathcal{L}}_h\left({\kappa }_{\alpha }\right):=\left\{\left(P_S,{\omega }^{\prime }(·,P_S),P_{X|USW},P_{X^{\prime }|US}\right):E_{\mathrm{b}}^{\prime }({\kappa }_{\alpha },{\omega }^{\prime },P_S,P_{X|USW})\ge {\kappa }_{\alpha }\right\},\hfill \\ & {\hat{\mathcal{L}}}_h({\kappa }_{\alpha },{\omega }^{\prime },P_S,P_{X|USW})\hfill \\ & :=\left\{\begin{array}{cc}\hfill (P_{\hat{U}\hat{V}\hat{W}\hat{Y}S}:& D(P_{\hat{U}\hat{V}\hat{W}\hat{Y}|S}\left|\right|P_{UV\hat{W}Y|S}\left|P_S\right)\le {\kappa }_{\alpha },P_{SUV\hat{W}XY}:=P_S{P}_{UV}{P}_{\hat{W}|\hat{U}S}{P}_{X|USW}{P}_{Y|X},\hfill \\ & P_{\hat{W}|\hat{U}S}={\omega }^{\prime }(P_{\hat{U}},P_S)\hfill \end{array}\right\},\hfill \\ & E_{\mathrm{b}}^{\prime }({\kappa }_{\alpha },{\omega }^{\prime },P_S,P_{X|USW}):={\rho }^{\prime }({\kappa }_{\alpha },{\omega }^{\prime },P_S,P_{X|USW})-{\zeta }_q^{\prime }({\kappa }_{\alpha },{\omega }^{\prime },P_S),\hfill \\ & {\zeta }^{\prime }({\kappa }_{\alpha },{\omega }^{\prime },P_S):=\underset{\begin{array}{c}{P}_{\hat{U}\hat{W}S}:\exists P_{\hat{V}\hat{Y}}\mathrm{s}.\mathrm{t}.\\ P_{\hat{U}\hat{V}\hat{W}\hat{Y}S}\in {\hat{\mathcal{L}}}_h({\kappa }_{\alpha },{\omega }^{\prime },P_S,P_{X|USW})\end{array}}{max}{I}_P(\hat{U};\hat{W}|S),\hfill \\ & {\rho }^{\prime }({\kappa }_{\alpha },{\omega }^{\prime },P_S,P_{X|USW}):=\underset{\begin{array}{c}{P}_{\hat{V}\hat{W}\hat{Y}S}:\exists P_{\hat{U}}\mathrm{s}.\mathrm{t}.\\ P_{\hat{U}\hat{V}\hat{W}\hat{Y}S}\in {\hat{\mathcal{L}}}_h({\kappa }_{\alpha },{\omega }^{\prime },P_S,P_{X|USW})\end{array}}{\min }{I}_P(\hat{Y},\hat{V};\hat{W}|S),\hfill \\ & E_1^{\prime }({\kappa }_{\alpha },{\omega }^{\prime }):=\underset{\begin{array}{c}(P_{\tilde{U}\tilde{V}\tilde{W}\tilde{Y}S},Q_{\tilde{U}\tilde{V}\tilde{W}\tilde{Y}S})\in {\mathcal{T}}_1^{\prime }({\kappa }_{\alpha },{\omega }^{\prime })\end{array}}{\min }D(P_{\tilde{U}\tilde{V}\tilde{W}\tilde{Y}|S}\left|\right|Q_{\tilde{U}\tilde{V}\tilde{W}\tilde{Y}|S}\left|P_S\right),\hfill \\ & E_2^{\prime }({\kappa }_{\alpha },{\omega }^{\prime },P_S,P_{X|USW}):=\underset{\begin{array}{c}(P_{\tilde{U}\tilde{V}\tilde{W}\tilde{Y}S},Q_{\tilde{U}\tilde{V}\tilde{W}\tilde{Y}S})\in {\mathcal{T}}_2^{\prime }({\kappa }_{\alpha },{\omega }^{\prime },P_S,P_{X|USW})\end{array}}{\min }D(P_{\tilde{U}\tilde{V}\tilde{W}\tilde{Y}|S}\left|\right|Q_{\tilde{U}\tilde{V}\tilde{W}\tilde{Y}|S}\left|P_S\right)\hfill \\ & +E_{\mathrm{b}}^{\prime }({\kappa }_{\alpha },{\omega }^{\prime },P_S,P_{X|USW}),\hfill \\ & E_3^{\prime }({\kappa }_{\alpha },{\omega }^{\prime },P_S,P_{X|USW},P_{X^{\prime }|US}):=\underset{\begin{array}{c}{P}_{\hat{V}\hat{Y}S}:P_{\hat{U}\hat{V}\hat{W}\hat{Y}S}\in {\hat{\mathcal{L}}}_h({\kappa }_{\alpha },{\omega }^{\prime },P_S,P_{X|USW})\end{array}}{\min }D(P_{\hat{V}\hat{Y}|S}\left|\right|Q_{V{Y}^{\prime }|S}\left|P_S\right)\hfill \\ & +E_{\mathrm{b}}^{\prime }({\kappa }_{\alpha },{\omega }^{\prime },P_S,P_{X|USW}),\hfill \\ & Q_{SUV{X}^{\prime }{Y}^{\prime }}:=P_S{Q}_{UV}{P}_{X^{\prime }|US}{P}_{Y^{\prime }|X^{\prime }},P_{Y^{\prime }|X^{\prime }}:=P_{Y|X},\hfill \\ & {\mathcal{T}}_1^{\prime }({\kappa }_{\alpha },{\omega }^{\prime },P_S,P_{X|USW}):=\left\{\begin{array}{cc}(P_{\tilde{U}\tilde{V}\tilde{W}\tilde{Y}S},\hfill & P_{\tilde{U}\tilde{W}S}=P_{\hat{U}\hat{W}S},P_{\tilde{V}\tilde{W}\tilde{Y}S}=P_{\hat{V}\hat{W}\hat{Y}S},\hfill \\ Q_{\tilde{U}\tilde{V}\tilde{W}\tilde{Y}S}):\hfill & Q_{S\tilde{U}\tilde{V}\tilde{W}\tilde{X}\tilde{Y}}:=P_S{Q}_{UV}{P}_{\tilde{W}|\tilde{U}S}{P}_{X|USW}{P}_{Y|X}\hfill \\ & \mathrm{for} \mathrm{some}{P}_{\hat{U}\hat{V}\hat{W}\hat{Y}S}\in {\hat{\mathcal{L}}}_h({\kappa }_{\alpha },{\omega }^{\prime },P_S,P_{X|USW})\hfill \end{array}\right\},\hfill \\ & {\mathcal{T}}_2^{\prime }({\kappa }_{\alpha },{\omega }^{\prime },P_S,P_{X|USW}):=\left\{\begin{array}{cc}(P_{\tilde{U}\tilde{V}\tilde{W}\tilde{Y}S},\hfill & P_{\tilde{U}\tilde{W}S}=P_{\hat{U}\hat{W}S},P_{\tilde{V}\tilde{Y}S}=P_{\hat{V}\hat{Y}S},\hfill \\ Q_{\tilde{U}\tilde{V}\tilde{W}\tilde{Y}S}):\hfill & H_P\left(\tilde{W}\right|\tilde{V},\tilde{Y},S)\ge H_P\left(\hat{W}\right|\hat{V},\hat{Y},S),\hfill \\ & Q_{S\tilde{U}\tilde{V}\tilde{W}\tilde{X}\tilde{Y}}:=P_S{Q}_{UV}{P}_{\tilde{W}|\tilde{U}S}{P}_{X|USW}{P}_{Y|X}\hfill \\ & \mathrm{for} \mathrm{some}{P}_{\hat{U}\hat{V}\hat{W}\hat{Y}S}\in {\hat{\mathcal{L}}}_h({\kappa }_{\alpha },{\omega }^{\prime },P_S,P_{X|USW})\hfill \end{array}\right\}.\hfill & \end{array}$

Then, we have the following result.

Theorem 2 

(Inner bound via JHTCC scheme).

$$\begin{array}{cc}& \kappa \left({\kappa }_{\alpha }\right)\ge max\left\{{\kappa }_{\mathrm{h}}^{★}\left({\kappa }_{\alpha }\right),{\kappa }_{\mathrm{u}}^{★}\left({\kappa }_{\alpha }\right)\right\},\hfill \end{array}$$

(13)

where

$$\begin{array}{cc}& {\kappa }_{\mathrm{h}}^{★}\left({\kappa }_{\alpha }\right):=\underset{\begin{array}{c}(P_S,{\omega }^{\prime },P_{X|USW},P_{X^{\prime }|US})\in {\mathcal{L}}_h\left({\kappa }_{\alpha }\right)\end{array}}{max}\min \{E_1^{\prime }({\kappa }_{\alpha },{\omega }^{\prime }),E_2^{\prime }({\kappa }_{\alpha },{\omega }^{\prime },P_S,P_{X|USW}),\hfill \end{array}$$

$$\begin{array}{cc}& E_3^{\prime }({\kappa }_{\alpha },{\omega }^{\prime },P_S,P_{X|USW},P_{X^{\prime }|US})\},\hfill \\ & {\kappa }_{\mathrm{u}}^{★}\left({\kappa }_{\alpha }\right):=\underset{\begin{array}{c}(P_S,P_{X|US})\in \mathcal{P}\left(\mathcal{S}\right)\times \mathcal{P}\left(\mathcal{X}\right|\mathcal{S}\times \mathcal{U})\end{array}}{max}{\kappa }_{\mathrm{u}}({\kappa }_{\alpha },P_S,P_{X|US}),\hfill \\ & {\kappa }_{\mathrm{u}}({\kappa }_{\alpha },P_S,P_{X|US}):=\underset{\begin{array}{c}{P}_S{P}_{\hat{V}\hat{Y}}:D\left(P_{\hat{V}\hat{Y}|S}\left|\right|P_{VY|S}|P_S\right)\le {\kappa }_{\alpha }\end{array}}{\min }D\left(P_{\hat{V}\hat{Y}|S}\left|\right|Q_{VY|S}|P_S\right),\hfill \\ & P_{SUVXY}=P_S{P}_{UV}{P}_{X|US}{P}_{Y|X}and{Q}_{SUVXY}=P_S{Q}_{UV}{P}_{X|US}{P}_{Y|X}.\hfill \end{array}$$

The proof of Theorem 2 is given in Section 4.4, and utilizes a generalization of hybrid coding scheme [40] to achieve the stated inner bound. Specifically, the error-exponent pair $\left({\kappa }_{\alpha },{\kappa }_{\mathrm{h}}^{★}\left({\kappa }_{\alpha }\right)\right)$ is achieved using type-based hybrid coding, while $\left({\kappa }_{\alpha },{\kappa }_{\mathrm{u}}^{★}\left({\kappa }_{\alpha }\right)\right)$ is realized by uncoded transmission, in which the channel input $\mathbf{X}$ is generated as the output of a DMC $P_{X|U}$ with input $\mathbf{U}$ (along with time sharing). In standard hybrid coding, the source sequence is first quantized via joint typicality and the channel input is then chosen as a function of both the original source sequence and its quantization. At the decoder, the quantized codeword is first recovered using the channel output and side information via joint typicality decoding, and an estimate of the source sequence is output as a function of the channel output and recovered codeword. The quantization part forms the digital part of the scheme, while the use of the source sequence for encoding and channel output for decoding comprises the analog part. The scheme derives its name from these joint hybrid digital-analog operations. In the HT context considered here, the aforementioned source quantization is replaced by a type-based quantization at the encoder, and the joint typicality decoder is replaced by a universal empirical conditional entropy decoder. We note that Theorem 2 recovers the lower bound on the optimal type II error-exponent proved in Theorem 5 in [10]. The details are provided in Appendix B.

Next, we provide a comparison between the SHTCC and JHTCC bounds via an example as mentioned earlier.

3.3. Comparison of Inner Bounds

We compare the inner bounds established in Theorem 1 and Theorem 2 for a simple setting of TAD over a BSC. For this purpose, we will use the inner bound ${\kappa }_{\mathrm{d}}^{★}\left({\kappa }_{\alpha }\right)$ stated in Corollary 2 and ${\kappa }_{\mathrm{u}}^{★}\left({\kappa }_{\alpha }\right)$ that is achieved by uncoded transmission. Our objective is to illustrate that the JHTCC scheme achieves a strictly tighter bound on $\mathcal{R}$ compared to the SHTCC scheme, at least for some points of the trade-off.

Example 1. 

Let $p,q\in [0,0.5]$, $\mathcal{U}=\mathcal{V}=\mathcal{X}=\mathcal{Y}=\mathcal{S}=\{0,1\}$,

$$Q_{UV}=\left[\begin{array}{cc}q& 0.5-q\\ 0.5-q& q\end{array}\right],P_{Y|X}=\left[\begin{array}{cc}1-p& p\\ p& 1-p\end{array}\right],and P_{UV}=Q_U{Q}_V.$$

A comparison of the inner bounds achieved by the SHTCC and JHTCC schemes for the above example are shown in Figure 2 and Figure 3, where we plot the error-exponents trade-off achieved by uncoded transmission (a lower bound for the JHTCC scheme), and the expurgated exponent at a zero rate:

$$\begin{array}{c}\hfill E_{\mathrm{ex}}\left(0\right):=\underset{P_{SX}\in \mathcal{P}(\mathcal{S}\times \mathcal{X})}{max}{E}_{\mathrm{ex}}(P_{SX},0)=-0.25log\left(4p(1-p)\right),\end{array}$$

which is an upper bound on ${\kappa }_{\mathrm{d}}^{★}\left({\kappa }_{\alpha }\right)$ for any ${\kappa }_{\alpha }\ge 0$. To compute $E_{\mathrm{ex}}\left(0\right)$, we used the closed-form expression for $E_{\mathrm{ex}}(·)$ given in Problem 10.26(c) in [38]. Clearly, it can be seen that the JHTCC scheme outperforms SHTCC scheme for ${\kappa }_{\alpha }$ below a threshold, which depends on the source and channel distributions. In particular, the threshold below which improvement is seen is reduced when the channel or the source becomes more uniform. The former behavior can be seen directly by comparing the subplots in Figure 2 and Figure 3, while the latter can be noted by comparing Figure 2a with Figure 3a, or Figure 2b with Figure 3b.

Figure 2. Comparison of the error-exponents trade-off achieved by the SHTCC and JHTCC schemes for TAD over a BSC in Example 1 with parameters $p=0.25,q=0$ for (a) and $p=0.35,q=0$ for (b). The red curve shows $\left({\kappa }_{\alpha },{\kappa }_u^{★}\left({\kappa }_{\alpha }\right)\right)$ pairs achieved by uncoded transmission while the blue line plots $\left({\kappa }_{\alpha },E_{\mathrm{ex}}\left(0\right)\right)$. The joint scheme clearly achieves a better error-exponent trade-off for values of ${\kappa }_{\alpha }$ below a threshold which depends on the transition kernel of the channel. In particular, a more uniform channel results in a lesser threshold.

Figure 3. Comparison of the error-exponents trade-off achieved by the SHTCC and JHTCC schemes for Example 1 with parameters $p=0.25,q=0.05$ for (a) and $p=0.35,q=0.05$ for (b). The JHTCC scheme improves over the separation based scheme for small values of ${\kappa }_{\alpha }$; however, the region of improvement is reduced compared to Figure 2 as the source is more uniformly distributed.

4. Proofs

4.1. Proof of Theorem 1

We will show the achievability of the error-exponent pair $({\kappa }_{\alpha },{\kappa }_{\mathrm{s}}^{★}\left({\kappa }_{\alpha }\right))$ by constructing a suitable ensemble of HT codes, and showing that the expected type I and type II error probabilities (over this ensemble) satisfy (5) for the pair $({\kappa }_{\alpha },{\kappa }_{\mathrm{s}}^{★}\left({\kappa }_{\alpha }\right))$. Then, an expurgation argument [44] will be used to show the existence of a HT code that satisfies (5) for the same error-exponent pair, thus showing that $({\kappa }_{\alpha },{\kappa }_{\mathrm{s}}^{★}\left({\kappa }_{\alpha }\right))\in \mathcal{R}$ as desired.

Let $n\in \mathbb{N}$, $\left|\mathcal{W}\right|<\infty$, ${\kappa }_{\alpha }>0$, $(\omega ,R,P_{SX},\theta )\in \mathcal{L}\left({\kappa }_{\alpha }\right)$, $R^{\prime }:=\zeta ({\kappa }_{\alpha },\omega )$, and $\eta >0$ be a small number. Additionally, suppose that $R\ge 0$ satisfies

$$\begin{array}{cc}\hfill & \zeta ({\kappa }_{\alpha },\omega )-\rho ({\kappa }_{\alpha },\omega )\le R<I_P(X;Y|S),\hfill \end{array}$$

(14)

where $\zeta ({\kappa }_{\alpha },\omega )$ and $\rho ({\kappa }_{\alpha },\omega )$ are defined in (6b) and (6c), respectively. The SHTCC scheme is as follows:

• Encoding: The observer’s encoder is composed of two stages, a source encoder followed by a channel encoder.
• Source encoder: The source encoding comprises a quantization scheme followed by binning to reduce the rate if necessary.
• Quantization codebook: Let

$$\begin{array}{c}\hfill {\mathcal{D}}_n(P_U,\eta ):=\left\{P_{\hat{U}}\in \mathcal{T}\left({\mathcal{U}}^n\right):D(P_{\hat{U}}\left|\right|P_U)\le {\kappa }_{\alpha }+\eta \right\}.\end{array}$$

(15)

Consider some ordering on the types in ${\mathcal{D}}_n(P_U,\eta )$ and denote the elements as $P_{{\hat{U}}_i}$ for $i\in \left[|{\mathcal{D}}_n(P_U,\eta )|\right]$. For each type $P_{{\hat{U}}_i}\in {\mathcal{D}}_n(P_U,\eta )$, $i\in \left[|{\mathcal{D}}_n(P_U,\eta )|\right]$, choose a joint type variable $P_{{\hat{U}}_i{\hat{W}}_i}\in \mathcal{T}({\mathcal{U}}^n\times {\mathcal{W}}^n)$ such that

$$\begin{array}{cc}\hfill & D\left(P_{{\hat{W}}_i|{\hat{U}}_i}\left|\right|P_{W_i|U}|P_{{\hat{U}}_i}\right)\le \frac{\eta }{3},\hfill \end{array}$$

(16a)

$$\begin{array}{cc}\hfill & I_P\left({\hat{U}}_i;{\hat{W}}_i\right)\le R^{\prime }+\frac{\eta }{3},\hfill \end{array}$$

(16b)

where $P_{W_i|U}=\omega \left(P_{{\hat{U}}_i}\right)$. Note that this is always possible for n sufficiently large by the definition of $R^{\prime }$.

Let

$$\begin{array}{cc}\hfill {\mathcal{D}}_n(P_{UW},\eta )& :=\left\{P_{{\hat{U}}_i{\hat{W}}_i}:i\in \left[|{\mathcal{D}}_n(P_U,\eta )|\right]\right\},\hfill \end{array}$$

(17a)

$$\begin{array}{cc}\hfill \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}{R}_i^{\prime }& :=I_P\left({\hat{U}}_i;{\hat{W}}_i\right)+(\eta /3),i\in \left[|{\mathcal{D}}_n(P_U,\eta )|\right],\hfill \end{array}$$

(17b)

$$\begin{array}{cc}\hfill \hspace{1em}\hspace{1em}\hspace{1em}{\mathcal{M}}_i^{\prime }& :=\left[1+\sum _{k=1}^{i-1}{e}^{n{R}_k^{\prime }}:\sum _{k=1}^i{e}^{n{R}_k^{\prime }}\right],\hfill \end{array}$$

(17c)

and ${\mathbb{B}}_{W,n}=\left\{\mathbf{W}\left(j\right),1\le j\le {\sum }_{i=1}^{|{\mathcal{D}}_n(P_U,\eta )|}\left|{\mathcal{M}}_i^{\prime }\right|\right\}$ denote a random quantization codebook such that the codeword $\mathbf{W}\left(j\right)\sim \mathsf{Unif}\left[{\mathcal{T}}_n\left(P_{{\hat{W}}_i}\right)\right]$, if $j\in {\mathcal{M}}_i^{\prime }$ for some $i\in \left[|{\mathcal{D}}_n(P_U,\eta )|\right]$. Denote a realization of ${\mathbb{B}}_{W,n}$ by ${\mathcal{B}}_{W,n}=\left\{\mathbf{w}\left(j\right)\in {\mathcal{W}}^n,1\le j\le {\sum }_{i=1}^{|{\mathcal{D}}_n(P_U,\eta )|}\left|{\mathcal{M}}_i^{\prime }\right|\right\}$.

Quantization scheme: For a given codebook ${\mathcal{B}}_{W,n}$ and $\mathbf{u}\in {\mathcal{T}}_n\left(P_{{\hat{U}}_i}\right)$ such that $P_{{\hat{U}}_i}\in {\mathcal{D}}_n(P_U,\eta )$ for some $i\in \left[|{\mathcal{D}}_n(P_U,\eta )|\right]$, let

$$\tilde{M}\left(\mathbf{u},{\mathcal{B}}_{W,n}\right):=\left\{j\in {\mathcal{M}}_i^{\prime }:\mathbf{w}\left(j\right)\in {\mathcal{B}}_{W,n},(\mathbf{u},\mathbf{w}\left(j\right))\in {\mathcal{T}}_n\left(P_{{\hat{U}}_i{\hat{W}}_i}\right),P_{{\hat{U}}_i{\hat{W}}_i}\in {\mathcal{D}}_n(P_{UW},\eta )\right\}.$$

If $|\tilde{M}(\mathbf{u},{\mathcal{B}}_{W,n})|\ge 1$, let $M^{\prime }\left(\mathbf{u},{\mathcal{B}}_{W,n}\right)$ denote an index selected uniformly at random from the set $\tilde{M}(\mathbf{u},{\mathcal{B}}_{W,n})$, otherwise, set $M^{\prime }\left(\mathbf{u},{\mathcal{B}}_{W,n}\right)=0$. Denoting the support of $M^{\prime }\left(\mathbf{u},{\mathcal{B}}_{W,n}\right)$ by ${\mathcal{M}}^{\prime }$, we have for sufficiently large n that

$$\begin{array}{cc}\hfill |{\mathcal{M}}^{\prime }|\le 1+\sum _{i=1}^{|{\mathcal{D}}_n(P_U,\eta )|}{e}^{n{R}_i^{\prime }}& \le 1+|{\mathcal{D}}_n(P_U,\eta )|e^{{\stackrel{max}{P_{\hat{U}\hat{W}}\in {\mathcal{D}}_n(P_{UW},\eta )}}^{nI(\hat{U};\hat{W})+(n\eta /3)}}\le e^{n(R^{\prime }+\eta )},\hfill \end{array}$$

(18)

where the last inequality uses (16b) and $|{\mathcal{D}}_n(P_U,\eta )|\le {(n+1)}^{\left|\mathcal{U}\right|}$. Binning: If $|{\mathcal{M}}^{\prime }|>|\mathcal{M}|$, then the source encoder performs binning as described below. Let $R_n:=log\left(e^{nR}/\left|{\mathcal{D}}_n(P_U,\eta )\right|\right)$, ${\mathcal{M}}_i:=[1+(i-1)R_n:i{R}_n],i\in \left[|{\mathcal{D}}_n(P_U,\eta )|\right]$, and $\mathcal{M}:=\left\{0\right\}\bigcup \left\{{\cup }_{i\in \left[\right|{\mathcal{D}}_n(P_U,\eta )\left|\right]}{\mathcal{M}}_i\right\}.$ Note that

$$\begin{array}{c}\hfill e^{n{R}_n}\ge e^{nR-\left|\mathcal{U}\right|log(n+1)}.\end{array}$$

(19)

Let $f_{\mathbb{B}}$ denote the random binning function such that for each $j\in {\mathcal{M}}_i^{\prime }$, $f_{\mathbb{B}}\left(j\right)\sim \mathsf{Unif}\left[\right|{\mathcal{M}}_i\left|\right]$ for $i\in \left[|{\mathcal{D}}_n(P_U,\eta )|\right]$, and $f_{\mathbb{B}}\left(0\right)=0$ with probability one. Denote a realization of $f_{\mathbb{B}}\left(j\right)$ by $f_b$, where $f_b:{\mathcal{M}}^{\prime }\to \mathcal{M}$. Given a codebook ${\mathcal{B}}_{W,n}$ and binning function $f_b$, the source encoder outputs $M=f_b\left(M^{\prime }\left(\mathbf{u},{\mathcal{B}}_{W,n}\right)\right)$ for $\mathbf{u}\in {\mathcal{U}}^n$. If $|{\mathcal{M}}^{\prime }|\le |\mathcal{M}|$, then $f_b$ is taken to be the identity map (no binning), and in this case, $M=M^{\prime }\left(\mathbf{u},{\mathcal{B}}_{W,n}\right)$.

Channel codebook: Let ${\mathbb{B}}_{X,n}:=\{\mathbf{X}\left(m\right)\in {\mathcal{X}}^n,m\in \mathcal{M}\}$ denote a random channel codebook generated as follows. Without loss of generality, denote the elements of the set $\mathcal{S}=\mathcal{X}$ as $1,\dots ,\left|\mathcal{X}\right|$. The codeword length n is divided into $\left|\mathcal{S}\right|=\left|\mathcal{X}\right|$ blocks, where the length of the first block is $\lceil P_S\left(1\right)n\rceil$, the second block is $\lceil P_S\left(2\right)n\rceil$, so on and so forth, and the length of the last block is chosen such that the total length is n. For $i\in \left[\right|\mathcal{X}\left|\right]$, let $k_i:={\sum }_{l=1}^{i-1}\lceil P_S\left(l\right)n\rceil +1$ and ${\overline{k}}_i:={\sum }_{l=1}^i\lceil P_S\left(l\right)n\rceil$, where the empty sum is defined to be zero. Let $\mathbf{s}\in {\mathcal{X}}^n$ be such that $s_{k_i}^{{\overline{k}}_i}=i$, i.e., the elements of $\mathbf{s}$ equal i in the $i^{th}$ block for $i\in \left[\right|\mathcal{X}\left|\right]$. Let $\mathbf{X}\left(0\right)=\mathbf{s}$ with probability one, and the remaining codewords $\mathbf{X}\left(m\right),m\in \mathcal{M}\setminus \left\{0\right\}$ be constant composition codewords [38] selected such that $X_{k_i}^{{\overline{k}}_i}\left(m\right)\sim \mathsf{Unif}\left[{\mathcal{T}}_{\lceil P_S\left(i\right)n\rceil }\left({\hat{P}}_{X|S}(·|i)\right)\right]$, where ${\hat{P}}_{X|S}$ is such that ${\mathcal{T}}_{\lceil P_S\left(i\right)n\rceil }\left({\hat{P}}_{X|S}(·|i)\right)$ is non-empty and $D({\hat{P}}_{X|S}\left|\right|P_{X|S}\left|P_S\right)\le \frac{\eta }{3}$. Denote a realization of ${\mathbb{B}}_{X,n}$ by ${\mathcal{B}}_{X,n}:=\{\mathbf{x}\left(m\right)\in {\mathcal{X}}^n,m\in \mathcal{M}\}$. Note that for $m\in \mathcal{M}\setminus \left\{0\right\}$ and large n, the codeword pair $\left(\mathbf{x}\right(0),\mathbf{x}(m\left)\right)$ has joint type (approx) $P_{\mathbf{x}\left(0\right)\mathbf{x}\left(j\right)}={\hat{P}}_{SX}:=P_S{\hat{P}}_{X|S}$.

Channel encoder: For a given ${\mathcal{B}}_{X,n}$, the channel encoder outputs $\mathbf{x}=\mathbf{x}\left(m\right)$ for output m from the source encoder. Denote this map by $f_{{\mathcal{B}}_{X,n}}:\mathcal{M}\to {\mathcal{X}}^n$.

Encoder: Denote by $f_n:{\mathcal{U}}^n\to \mathcal{P}\left({\mathcal{X}}^n\right)$ the encoder induced by all the above operations, i.e., $f_n(·|\mathbf{u})=f_{{\mathcal{B}}_{X,n}}\circ f_b\left(M^{\prime }(\mathbf{u},{\mathcal{B}}_{W,n})\right)$.

Decision function: The decision function consists of three parts, a channel decoder, a source decoder and a tester.

Channel decoder: The channel decoder first performs a Neyman–Pearson test on the channel output $\mathbf{y}$ according to ${\tilde{\Pi }}_{\theta }:{\mathcal{Y}}^n\to \{0,1\}$, where

$$\begin{array}{cc}\hfill {\tilde{\Pi }}_{\theta }\left(\mathbf{y}\right)& :=\mathrm{𝟙}\left(\sum _{i=1}^{n}log\left(\frac{P_{Y|X}\left(y_i\right|s_i)}{P_{Y|S}\left(y_i\right|s_i)}\right)\ge n\theta \right).\hfill \end{array}$$

(20)

If ${\tilde{\Pi }}_{\theta }\left(\mathbf{y}\right)=1$, then $\hat{M}=0$. Else, for a given ${\mathcal{B}}_{X,n}$, maximum likelihood (ML) decoding is done on the remaining set of codewords $\left\{\mathbf{x}\right(m),m\in \mathcal{M}\setminus \{0\left\}\right\}$, and $\hat{M}$ is set equal to the ML estimate. Denote the channel decoder induced by the above operations by $g_{{\mathcal{B}}_{X,n}}$, where $g_{{\mathcal{B}}_{X,n}}:{\mathcal{Y}}^n\to \mathcal{M}$.

For a given codebook ${\mathcal{B}}_{X,n}$, the channel encoder–decoder pair described above induces a distribution

$$\begin{array}{c}\hfill P_{\mathbf{X}\mathbf{Y}\hat{M}|M}^{\left({\mathcal{B}}_{X,n}\right)}(m,\mathbf{x},\mathbf{y},\hat{m}|m):={\mathrm{𝟙}}_{\left\{f_{{\mathcal{B}}_{X,n}}\left(m\right)=\mathbf{x}\right\}}{P}_{Y|X}^{\otimes n}\left(\mathbf{y}\right|\mathbf{x}){\mathrm{𝟙}}_{\left\{\hat{m}=g_{{\mathcal{B}}_{X,n}}\right\}}.\end{array}$$

Note that $P_{\mathbf{x}\left(0\right)\mathbf{x}\left(m\right)}={\hat{P}}_{SX}$, $\mathbf{Y}\sim {\prod }_{i=1}^{\left|\mathcal{X}\right|}{P}_{Y|X}^{\otimes \lceil P_S\left(i\right)n\rceil }(·|i)$ for $M=0$ and $\mathbf{Y}\sim {\prod }_{i=1}^{\left|\mathcal{X}\right|}{P}_{Y|S}^{\otimes \lceil P_S\left(i\right)n\rceil }(·|i)$ for $M=m\ne 0$. Then, it follows by an application of Proposition A1 proved in Appendix C that for any ${\mathcal{B}}_{X,n}$ and n sufficiently large, the Neyman–Pearson test in (20) yields

$$\begin{array}{cc}\hfill \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}{\mathbb{P}}_{P^{\left({\mathcal{B}}_{X,n}\right)}}\left(\hat{M}=0|M=m\right)& \le e^{-n\left(E_{\mathrm{sp}}(P_{SX},\theta )-\eta \right)},m\in \mathcal{M}\setminus \left\{0\right\},\hfill \end{array}$$

(21a)

$$\begin{array}{cc}\hfill {\mathbb{P}}_{P^{\left({\mathcal{B}}_{X,n}\right)}}\left(\hat{M}\ne 0|M=0\right)& \le e^{-n\left(E_{\mathrm{sp}}(P_{SX},\theta )-\theta -\eta \right)}.\hfill \end{array}$$

(21b)

Moreover, given $\hat{M}\ne 0$, a random coding argument over the ensemble of ${\mathbb{B}}_X^n$ (see Exercise 10.18, 10.24 in [38,44]) shows that there exists a deterministic codebook ${\mathcal{B}}_{X,n}$ such that (21a) and (21b) holds, and the ML decoding described above asymptotically achieves

$$\begin{array}{cc}\hfill {\mathbb{P}}_{P^{\left({\mathcal{B}}_{X,n}\right)}}\left(\hat{M}\ne m|M=m\ne 0,\hat{M}\ne 0\right)& \le e^{-n\left(E_{\mathrm{ex}}\left(R,P_{SX}\right)-\eta \right)}.\hfill \end{array}$$

(22)

This deterministic codebook ${\mathcal{B}}_{X,n}$ is used for channel coding.

Source decoder: For a given codebook ${\mathcal{B}}_{W,n}$ and inputs $\hat{M}=\hat{m}$ and $\mathbf{V}=\mathbf{v}$, the source decoder first decodes for the quantization codeword $\mathbf{w}\left({\hat{m}}^{\prime }\right)$ (if required) using the empirical conditional entropy decoder, and then declares the output $\hat{H}$ of the hypothesis test based on $\mathbf{w}\left({\hat{m}}^{\prime }\right)$ and $\mathbf{v}$. More specifically, if binning is not performed, i.e., if $\left|\mathcal{M}\right|\ge |{\mathcal{M}}^{\prime }|$, ${\hat{M}}^{\prime }=\hat{m}$. Otherwise, ${\hat{M}}^{\prime }={\hat{m}}^{\prime }$, where ${\hat{m}}^{\prime }=0$ if $\hat{m}=0$ and ${\hat{m}}^{\prime }={arg\; min}_{j:f_b\left(j\right)=\hat{m}}{H}_e\left(\mathbf{w}\left(j\right)\right|\mathbf{v})$ otherwise. Denote the source decoder induced by the above operations by $g_{{\mathcal{B}}_{W,n}}:\mathcal{M}\times {\mathcal{V}}^n\to {\mathcal{M}}^{\prime }$.

Testing and Acceptance region: If ${\hat{m}}^{\prime }=0$, $\hat{H}=1$ is declared. Otherwise, $\hat{H}=0$ or $\hat{H}=1$ is declared depending on whether $({\hat{m}}^{\prime },\mathbf{v})\in {\mathcal{A}}_n$ or $({\hat{m}}^{\prime },\mathbf{v})\notin {\mathcal{A}}_n$, respectively, where ${\mathcal{A}}_n$ denotes the acceptance region for $H_0$ as specified next. For a given codebook ${\mathcal{B}}_{W,n}$, let ${\mathcal{O}}_{m^{\prime }}$ denote the set of $\mathbf{u}$ such that the source encoder outputs $m^{\prime }$, $m^{\prime }\in {\mathcal{M}}^{\prime }\setminus \left\{0\right\}$. For each $m^{\prime }\in {\mathcal{M}}^{\prime }\setminus \left\{0\right\}$ and $\mathbf{u}\in {\mathcal{O}}_{m^{\prime }}$, let

$$\begin{array}{c}\hfill {\mathcal{Z}}_{m^{\prime }}\left(\mathbf{u}\right)=\{\mathbf{v}\in {\mathcal{V}}^n:(\mathbf{w}\left(m^{\prime }\right),\mathbf{u},\mathbf{v})\in {\mathcal{J}}_n({\kappa }_{\alpha }+\eta ,P_{W_{m^{\prime }}UV})\},\end{array}$$

where ${\mathcal{J}}_n(r,P_X):=\{\mathbf{x}\in {\mathcal{X}}^n:D\left(P_{\mathbf{x}}\left|\right|P_X\right)\le r\}$,

$$\begin{array}{c}\hfill P_{UV{W}_{m^{\prime }}}:=P_{UV}{P}_{W_{m^{\prime }}|U} \mathrm{and} P_{W_{m^{\prime }}|U}=\omega \left(P_{\mathbf{u}}\right).\end{array}$$

(23)

For $m^{\prime }\in {\mathcal{M}}^{\prime }\setminus \left\{0\right\}$, set ${\mathcal{Z}}_{m^{\prime }}:=\{\mathbf{v}:\mathbf{v}\in {\mathcal{Z}}_{m^{\prime }}\left(\mathbf{u}\right)\mathrm{for} \mathrm{some}\mathbf{u}\in {\mathcal{O}}_{m^{\prime }}\}$, and define the acceptance region for $H_0$ at the decision maker as ${\mathcal{A}}_n:={\cup }_{m^{\prime }\in {\mathcal{M}}^{\prime }\setminus 0}{m}^{\prime }\times {\mathcal{Z}}_{m^{\prime }}$ or equivalently as ${\mathcal{A}}_n^e:={\cup }_{m^{\prime }\in {\mathcal{M}}^{\prime }\setminus 0}{\mathcal{O}}_{m^{\prime }}\times {\mathcal{Z}}_{m^{\prime }}$. Note that ${\mathcal{A}}_n$ is the same as the acceptance region for $H_0$ in Theorem 1 in [14]. Denote the decision function induced by $g_{{\mathcal{B}}_{X,n}}$, $g_{{\mathcal{B}}_{W,n}}$ and ${\mathcal{A}}_n$ by $g_n:{\mathcal{Y}}^n\times {\mathcal{V}}^n\to \hat{\mathcal{H}}$.

Induced probability distribution: The PMFs induced by a code $c_n=(f_n,g_n)$ with respect to codebook ${\mathcal{B}}_n:=\left({\mathcal{B}}_{W,n},f_b,{\mathcal{B}}_{X,n}\right)$ under $H_0$ and $H_1$ are

$$\begin{array}{cc}\hfill & P_{\mathbf{UV}{M}^{\prime }M\mathbf{XY}\hat{M}{\hat{M}}^{\prime }\hat{H}}^{({\mathcal{B}}_n,c_n)}(\mathbf{u},\mathbf{v},m^{\prime },m,\mathbf{x},\mathbf{y},\hat{m},{\hat{m}}^{\prime },\hat{h})\hfill \\ \hfill & :=P_{UV}^{\otimes n}(\mathbf{u},\mathbf{v}){\mathrm{𝟙}}_{\left\{M^{\prime }\left(\mathbf{u},{\mathcal{B}}_{W,n}\right)=m^{\prime },f_b\left(m^{\prime }\right)=m\right\}}{P}_{\mathbf{X}\mathbf{Y}\hat{M}|M}^{\left({\mathcal{B}}_{X,n}\right)}(\mathbf{x},\mathbf{y},\hat{m}|m){\mathrm{𝟙}}_{\left\{g_{{\mathcal{B}}_{W,n}}(m,\mathbf{v})={\hat{m}}^{\prime },\hat{h}={\mathrm{𝟙}}_{\left\{({\hat{m}}^{\prime },\mathbf{v})\in {\mathcal{A}}_n^c\right\}}\right\}},\hfill \\ & Q_{\mathbf{UV}{M}^{\prime }M\mathbf{XY}\hat{M}{\hat{M}}^{\prime }\hat{H}}^{({\mathcal{B}}_n,c_n)}(\mathbf{u},\mathbf{v},m^{\prime },m,\mathbf{x},\mathbf{y},\hat{m},{\hat{m}}^{\prime },\hat{h})\hfill \\ \hfill & :=Q_{UV}^{\otimes n}(\mathbf{u},\mathbf{v}){\mathrm{𝟙}}_{\left\{M^{\prime }\left(\mathbf{u},{\mathcal{B}}_{W,n}\right)=m^{\prime },f_b\left(m^{\prime }\right)=m\right\}}{P}_{\mathbf{X}\mathbf{Y}\hat{M}|M}^{\left({\mathcal{B}}_{X,n}\right)}(\mathbf{x},\mathbf{y},\hat{m}|m){\mathrm{𝟙}}_{\left\{g_{{\mathcal{B}}_{W,n}}(m,\mathbf{v})={\hat{m}}^{\prime },\hat{h}={\mathrm{𝟙}}_{\left\{({\hat{m}}^{\prime },\mathbf{v})\in {\mathcal{A}}_n^c\right\}}\right\}},\hfill \end{array}$$

respectively. For simplicity, we will denote the above distributions by $P^{\left({\mathcal{B}}_n\right)}$ and $Q^{\left({\mathcal{B}}_n\right)}$. Let ${\mathbb{B}}_n:=\left({\mathbb{B}}_{W,n},f_{\mathbb{B}},{\mathcal{B}}_{X,n}\right)$, ${\mathfrak{B}}_n$, and ${\mu }_n$ denote the random codebook, its support, and the probability measure induced by its random construction, respectively. Additionally, define ${\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}:={\mathbb{E}}_{{\mu }_n}\left[{\mathbb{P}}_{P^{\left({\mathbb{B}}_n\right)}}\right]$ and ${\overline{\mathbb{P}}}_{Q^{\left({\mathbb{B}}_n\right)}}:={\mathbb{E}}_{{\mu }_n}\left[{\mathbb{P}}_{Q^{\left({\mathbb{B}}_n\right)}}\right]$.

Analysis of the type I and type II error probabilities: We analyze the type I and type II error probabilities averaged over the random ensemble of quantization and binning codebooks $({\mathbb{B}}_W,f_{\mathbb{B}})$. Then, an expurgation technique [44] guarantees the existence of a sequence of deterministic codebooks ${\left\{{\mathcal{B}}_n\right\}}_{n\in \mathbb{N}}$ and a code ${\{c_n=(f_n,g_n)\}}_{n\in \mathbb{N}}$ that achieves the lower bound given in Theorem 1.

Type I error probability: In the following, random sets where the randomness is induced due to ${\mathbb{B}}_n$ will be written using blackboard bold letters, e.g., ${\mathbb{A}}_n$ for the random acceptance region for $H_0$. Note that a type I error can occur only under the following events:

(i)

${\mathcal{E}}_{\mathrm{EE}}:=\stackrel{\bigcup }{P_{\hat{U}}\in {\mathcal{D}}_n(P_U,\eta )}\stackrel{\bigcup }{\mathbf{u}\in {\mathcal{T}}_n\left(P_{\hat{U}}\right)}\hspace{1em}{\mathcal{E}}_{\mathrm{EE}}\left(\mathbf{u}\right),\mathrm{where}$

$\begin{array}{cc}\hfill {\mathcal{E}}_{\mathrm{EE}}\left(\mathbf{u}\right)& :=\{\nexists j\in {\mathcal{M}}^{\prime }\setminus \left\{0\right\}\mathrm{s}.\mathrm{t}.(\mathbf{u},\mathbf{W}\left(j\right))\in {\mathcal{T}}_n\left(P_{{\hat{U}}_i{\hat{W}}_i}\right),P_{{\hat{U}}_i}=P_{\mathbf{u}},\hfill \\ & P_{{\hat{U}}_i{\hat{W}}_i}\in {\mathcal{D}}_n(P_{UW},\eta )\},\hfill & \end{array}$

(ii)

${\mathcal{E}}_{\mathrm{NE}}:=\{{\hat{M}}^{\prime }=M^{\prime } \mathrm{and} ({\hat{M}}^{\prime },\mathbf{V})\notin {\mathbb{A}}_n\},$

(iii)

${\mathcal{E}}_{\mathrm{OCE}}:=\{M^{\prime }\ne 0,\hat{M}\ne M \mathrm{and} ({\hat{M}}^{\prime },\mathbf{V})\notin {\mathbb{A}}_n\},$

(iv)

${\mathcal{E}}_{\mathrm{SCE}}:=\{M^{\prime }=M=0,\hat{M}\ne M \mathrm{and} ({\hat{M}}^{\prime },\mathbf{V})\notin {\mathbb{A}}_n\},$

(v)

${\mathcal{E}}_{\mathrm{BE}}:=\{M^{\prime }\ne 0,\hat{M}=M,{\hat{M}}^{\prime }\ne M^{\prime } \mathrm{and} ({\hat{M}}^{\prime },\mathbf{V})\notin {\mathbb{A}}_n\}$.

Here, ${\mathcal{E}}_{\mathrm{EE}}$ corresponds to the event that there does not exist a quantization codeword corresponding to atleast one sequence $\mathbf{u}$ of type $P_{\mathbf{u}}\in {\mathcal{D}}_n(P_U,\eta )$; ${\mathcal{E}}_{\mathrm{NE}}$ corresponds to the event, in which, there is neither an error at the channel decoder nor at the empirical conditional entropy decoder; ${\mathcal{E}}_{\mathrm{OCE}}$ and ${\mathcal{E}}_{\mathrm{SCE}}$ corresponds to the case, in which there is an error at the channel decoder (hence also at the empirical conditional entropy decoder); and ${\mathcal{E}}_{\mathrm{BE}}$ corresponds to the case that there is an error (due to binning) only at the empirical conditional entropy decoder. For the event ${\mathcal{E}}_{\mathrm{EE}}$, it follows from a slight generalization of the type-covering lemma (Lemma 9.1 in [38]) that

$$\begin{array}{c}\hfill {\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left({\mathcal{E}}_{\mathrm{EE}}\right)\le e^{-e^{n\Omega \left(\eta \right)}}.\end{array}$$

(24)

Since $e^{n\Omega \left(\eta \right)}/n\stackrel{\left(n\right)}{\to }\infty$ for $\eta >0$, the event ${\mathcal{E}}_{\mathrm{EE}}$ may be safely ignored from the analysis of the error-exponents. Given that ${\mathcal{E}}_{\mathrm{EE}}^c$ holds for some ${\mathcal{B}}_{W,n}$, it follows from Equation 4.22 in [14] that

$$\begin{array}{c}\hfill {\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left({\mathcal{E}}_{\mathrm{NE}}|{\mathcal{E}}_{\mathrm{EE}}^c\right)\le e^{-n{\kappa }_{\alpha }},\end{array}$$

(25)

for sufficiently large n since the acceptance region is the same as that in Theorem 1 in [14].

Next, consider the event ${\mathcal{E}}_{\mathrm{OCE}}$. We have for sufficiently large n that

$$\begin{array}{cc}{\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left({\mathcal{E}}_{\mathrm{OCE}}\right)& \le {\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left(M^{\prime }\ne 0\right){\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left(\hat{M}\ne M|M^{\prime }\ne 0\right)\hfill \\ & \stackrel{\left(a\right)}{\le }{\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left(\hat{M}\ne M|M\ne 0\right)\hfill \\ & \le {\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left(\hat{M}=0|M\ne 0\right)+{\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left(\hat{M}\ne M|M\ne 0,\hat{M}\ne 0\right)\hfill \\ & \stackrel{\left(b\right)}{\le }{e}^{-n\left(E_{\mathrm{m}}(P_{SX},\theta )-\eta \right)}+e^{-n\left(E_{\mathrm{ex}}\left(R,P_{SX}\right)-\eta \right)}\hfill \\ & =e^{-n\left(\min \left\{E_{\mathrm{m}}(P_{SX},\theta ),E_{\mathrm{ex}}\left(R,P_{SX}\right)\right\}-\eta \right)},\hfill & \end{array}$$

(26)

where

(a)

holds since the event $\{M^{\prime }\ne 0\}$ is equivalent to $\{M\ne 0\}$;

(b)

holds due to (21a) and (22), which holds for ${\mathcal{B}}_{X,n}$.

Additionally, the probability of ${\mathcal{E}}_{\mathrm{SCE}}$ can be upper bounded as

$$\begin{array}{cc}{\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left({\mathcal{E}}_{\mathrm{SCE}}\right)& \le {\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left(M^{\prime }=0\right)\hfill \\ & \le {\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left(M^{\prime }=0|\mathbf{U}\in {\mathcal{D}}_n(P_U,\eta )\right)+{\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left(\mathbf{U}\notin {\mathcal{D}}_n(P_U,\eta )\right)\hfill \\ & ={\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left({\mathcal{E}}_{\mathrm{EE}}\right)+{\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left(\mathbf{U}\notin {\mathcal{D}}_n(P_U,\eta )\right)\hfill \\ & \le e^{-n{\kappa }_{\alpha }},\hfill & \end{array}$$

(27)

where (27) is due to (24), the definition of ${\mathcal{D}}_n(P_U,\eta )$ in (15) and Lemma 2.2 and Lemma 2.6 in [38].

Finally, consider the event ${\mathcal{E}}_{\mathrm{BE}}$. Note that this event occurs only when $\left|\mathcal{M}\right|\le |{\mathcal{M}}^{\prime }|$. Additionally, $M=0$ iff $M^{\prime }=0$, and hence $M^{\prime }\ne 0$ and $\hat{M}=M$ implies that $\hat{M}\ne 0$. Let

$$\begin{array}{cc}\hfill {\mathcal{D}}_n(P_{VW},\eta )& :=\left\{P_{\hat{V}\hat{W}}:\begin{array}{cc}& \exists (\mathbf{w},\mathbf{u},\mathbf{v})\in \underset{m^{\prime }\in {\mathcal{M}}^{\prime }\setminus \left\{0\right\}}{\cup }{\mathcal{J}}_n({\kappa }_{\alpha }+\eta ,P_{W_{m^{\prime }}UV}),P_{W_{m^{\prime }}UV}\mathrm{satisfies}\hfill \\ \hfill & \left(23\right) \mathrm{and} P_{\mathbf{w}\mathbf{u}\mathbf{v}}=P_{\hat{W}\hat{U}\hat{V}}\hfill \end{array}\right\}.\hfill \end{array}$$

We have

$$\begin{array}{cc}\hfill {\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left({\mathcal{E}}_{\mathrm{BE}}\right)& ={\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left({\mathcal{E}}_{\mathrm{BE}},(M^{\prime },\mathbf{V})\in {\mathbb{A}}_n\right)+{\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left({\mathcal{E}}_{\mathrm{BE}},(M^{\prime },\mathbf{V})\notin {\mathbb{A}}_n\right).\hfill \end{array}$$

(28)

The second term in (28) can be upper bounded as

$$\begin{array}{cc}{\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left({\mathcal{E}}_{\mathrm{BE}},(M^{\prime },\mathbf{V})\notin {\mathbb{A}}_n\right)& \le {\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left((M^{\prime },\mathbf{V})\notin {\mathbb{A}}_n,{\mathcal{E}}_{\mathrm{EE}}\right)+{\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left((M^{\prime },\mathbf{V})\notin {\mathbb{A}}_n,{\mathcal{E}}_{\mathrm{EE}}^c\right)\hfill \\ & \le e^{-e^{n\Omega \left(\eta \right)}}+{\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left((M^{\prime },\mathbf{V})\notin {\mathbb{A}}_n|{\mathcal{E}}_{\mathrm{EE}}^c\right)\hfill \\ & \le e^{-e^{n\Omega \left(\eta \right)}}+{\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left((\mathbf{U},\mathbf{V})\notin {\mathbb{A}}_n^e\right)\hfill \\ & \le e^{-e^{n\Omega \left(\eta \right)}}+e^{-n{\kappa }_{\alpha }},\hfill & \end{array}$$

(29)

where the inequality in (29) follows from Equation (4.22) in [14] for sufficiently large n since the acceptance region ${\mathbb{A}}_n^e$ is the same as that in [14]. To bound the first term in (28), define ${\mathcal{D}}_n(P_V,\eta ):=\{P_{\hat{V}}:\exists P_{\hat{V}\hat{W}}\in {\mathcal{D}}_n(P_{VW},\eta )\}$, and observe that since $(M^{\prime },\mathbf{V})\in {\mathbb{A}}_n$ implies $M^{\prime }\ne 0$, we have

$$\begin{array}{cc}& {\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left({\mathcal{E}}_{\mathrm{BE}},(M^{\prime },\mathbf{V})\in {\mathbb{A}}_n\right)\hfill \\ & =\sum _{(m^{\prime },m)\in {\mathcal{M}}^{\prime }\times \mathcal{M}}{\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left({\mathcal{E}}_{\mathrm{BE}},(M^{\prime },\mathbf{V})\in {\mathbb{A}}_n,M=m,M^{\prime }=m^{\prime }\right)\hfill \\ & =\sum _{(m^{\prime },m)\in {\mathcal{M}}^{\prime }\times \mathcal{M}}{\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left(M=m,M^{\prime }=m^{\prime },\hat{M}=M\right)\hfill \\ & {\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left({\hat{M}}^{\prime }\ne M^{\prime },({\hat{M}}^{\prime },\mathbf{V})\notin {\mathbb{A}}_n,(M^{\prime },\mathbf{V})\in {\mathbb{A}}_n|M^{\prime }=m^{\prime },M=m,\hat{M}=M\right)\hfill \\ & \le \sum _{(m^{\prime },m)\in {\mathcal{M}}^{\prime }\times \mathcal{M}}{\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left(M=m,M^{\prime }=m^{\prime },\hat{M}=M\right)\hfill \\ & {\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left({\hat{M}}^{\prime }\ne M^{\prime },(M^{\prime },\mathbf{V})\in {\mathbb{A}}_n|M^{\prime }=m^{\prime },M=m,\hat{M}=M\right)\hfill \\ & \stackrel{\left(a\right)}{=}{\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left({\hat{M}}^{\prime }\ne M^{\prime },(M^{\prime },\mathbf{V})\in {\mathbb{A}}_n|M^{\prime }=1,M=1,\hat{M}=M\right)\hfill \\ & \stackrel{\left(b\right)}{\le }\sum _{\begin{array}{c}{P}_{\mathbf{v}}\in {\mathcal{D}}_n(P_V,\eta )\end{array}}\sum _{\mathbf{v}\in P_{\mathbf{v}}}{\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}(\mathbf{V}=\mathbf{v}|M^{\prime }=1)\hfill \end{array}$$

(30)

$$\begin{array}{ccc}& {\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left(\exists j\in f_{\mathbb{B}}^{-1}\left(1\right),j\ne 1,H_e\left(\mathbf{W}\left(j\right)\right|\mathbf{v})\le H_e\left(\mathbf{W}\left(1\right)\right|\mathbf{v})|M^{\prime }=1,\mathbf{V}=\mathbf{v}\right),\hfill & \end{array}$$

(31)

where $\left(a\right)$ follows since by the symmetry of the source encoder, binning function and random codebook construction, the term in (30) is independent of $(m,m^{\prime })$; $\left(b\right)$ holds since $(M^{\prime },\mathbf{V})\in {\mathbb{A}}_n$ implies that $P_{\mathbf{v}}\in {\mathcal{D}}_n(P_V,\eta )$ and $(\mathbf{V},{\mathbb{B}}_W)-M^{\prime }-(M,\hat{M})$ form a Markov chain. Defining $P_{\hat{V}}=P_{\mathbf{v}}$, and the event ${\mathcal{E}}_1^{\prime }:=\{M^{\prime }=1,\mathbf{V}=\mathbf{v}\}$, we obtain

$$\begin{array}{cc}& {\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left(\exists j\in f_{\mathbb{B}}^{-1}\left(1\right),j\ne 1,H_e\left(\mathbf{W}\left(j\right)\right|\mathbf{v})\le H_e\left(\mathbf{W}\left(1\right)\right|\mathbf{v})|{\mathcal{E}}_1^{\prime }\right)\hfill \\ & =\sum _{j\in {\mathcal{M}}^{\prime }\setminus \{0,1\}}{\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left(f_{\mathbb{B}}\left(j\right)=1,H_e\left(\mathbf{W}\left(j\right)\right|\mathbf{v})\le H_e\left(\mathbf{W}\left(1\right)\right|\mathbf{v})|{\mathcal{E}}_1^{\prime }\right)\hfill \\ & \stackrel{\left(a\right)}{\le }\frac{1}{e^{n{R}_n}}\sum _{j\in {\mathcal{M}}^{\prime }\setminus \{0,1\}}{\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left(H_e\left(\mathbf{W}\left(j\right)\right|\mathbf{v})\le H_e\left(\mathbf{W}\left(1\right)\right|\mathbf{v})|{\mathcal{E}}_1^{\prime }\right)\hfill \\ & \stackrel{\left(b\right)}{\le }\frac{1}{e^{n{R}_n}}\sum _{j\in {\mathcal{M}}^{\prime }\setminus \{0,1\}}\sum _{\begin{array}{c}{P}_{\hat{W}}:P_{\hat{V}\hat{W}}\in {\mathcal{D}}_n(P_{VW},\eta )\end{array}}\sum _{\begin{array}{c}\mathbf{w}:(\mathbf{v},\mathbf{w})\in {\mathcal{T}}_n\left(P_{\hat{V}\hat{W}}\right)\end{array}}{\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left(\mathbf{W}\left(1\right)=\mathbf{w}|{\mathcal{E}}_1^{\prime }\right)\hfill \\ & \sum _{\begin{array}{c}\tilde{\mathbf{w}}\in {\mathcal{T}}_n\left(P_{\hat{W}}\right):H_e\left(\tilde{\mathbf{w}}\right|\mathbf{v})\le H\left(\hat{W}\right|\hat{V})\end{array}}{\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left(\mathbf{W}\left(j\right)=\tilde{\mathbf{w}}|{\mathcal{E}}_1^{\prime }\cup \{\mathbf{W}\left(1\right)=\mathbf{w}\}\right)\hfill \\ & \stackrel{\left(c\right)}{\le }\frac{1}{e^{n{R}_n}}\sum _{j\in {\mathcal{M}}^{\prime }\setminus \{0,1\}}\sum _{\begin{array}{c}{P}_{\hat{W}}:P_{\hat{V}\hat{W}}\in {\mathcal{D}}_n(P_{VW},\eta )\end{array}}\sum _{\begin{array}{c}\mathbf{w}:(\mathbf{v},\mathbf{w})\in {\mathcal{T}}_n\left(P_{\hat{V}\hat{W}}\right)\end{array}}{\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left(\mathbf{W}\left(1\right)=\mathbf{w}|{\mathcal{E}}_1^{\prime }\right)\hfill \\ & \sum _{\begin{array}{c}\tilde{\mathbf{w}}\in {\mathcal{T}}_n\left(P_{\hat{W}}\right):H_e\left(\tilde{\mathbf{w}}\right|\mathbf{v})\le H\left(\hat{W}\right|\hat{V})\end{array}}2{\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left(\mathbf{W}\left(j\right)=\tilde{\mathbf{w}}\right),\hfill \end{array}$$

(32)

where

(a)

follows since $f_{\mathbb{B}}(·)$ is the uniform binning function independent of ${\mathbb{B}}_{W,n}$;

(b)

holds due to the fact that if $P_{\mathbf{v}}\in {\mathcal{D}}_n(P_V,\eta )$, then $M^{\prime }=1$ implies that $(\mathbf{W}\left(1\right),\mathbf{v})\in {\mathcal{T}}_n\left(P_{\hat{V}\hat{W}}\right)$ with probability one for some $P_{\hat{V}\hat{W}}\in {\mathcal{D}}_n(P_{VW},\eta )$;

(c)

holds since ${\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left(\mathbf{W}\left(j\right)=\tilde{\mathbf{w}}|{\mathcal{E}}_1^{\prime }\cup \{\mathbf{W}\left(1\right)=\mathbf{w}\}\right)\le 2{\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left(\mathbf{W}\left(j\right)=\tilde{\mathbf{w}}\right)$, which follows similarly to Equation (101) in [10].

Continuing, we can write for sufficiently large n,

$$\begin{array}{cc}& {\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left(\exists j\in f_{\mathbb{B}}^{-1}\left(1\right),j\ne 1,H_e\left(\mathbf{W}\left(j\right)\right|\mathbf{v})\le H_e\left(\mathbf{W}\left(1\right)\right|\mathbf{v})|{\mathcal{E}}_1^{\prime }\right)\hfill \\ & \stackrel{\left(a\right)}{\le }\frac{1}{e^{n{R}_n}}\sum _{j\in {\mathcal{M}}^{\prime }\setminus \{0,1\}}\sum _{\begin{array}{c}{P}_{\hat{W}}:P_{\hat{V}\hat{W}}\in {\mathcal{D}}_n(P_{VW},\eta )\end{array}}\sum _{\begin{array}{c}\mathbf{w}:(\mathbf{v},\mathbf{w})\in {\mathcal{T}}_n\left(P_{\hat{V}\hat{W}}\right)\end{array}}{\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left(\mathbf{W}\left(1\right)=\mathbf{w}|{\mathcal{E}}_1^{\prime }\right)\hfill \\ & \sum _{\begin{array}{c}\tilde{\mathbf{w}}\in {\mathcal{T}}_n\left(P_{\hat{W}}\right):H_e\left(\tilde{\mathbf{w}}\right|\mathbf{v})\le H\left(\hat{W}\right|\hat{V})\end{array}}2e^{-n(H\left(\hat{W}\right)-\eta )}\hfill \\ & \stackrel{\left(b\right)}{\le }\frac{1}{e^{n{R}_n}}\sum _{j\in {\mathcal{M}}^{\prime }\setminus \{0,1\}}\sum _{\begin{array}{c}{P}_{\hat{W}}:P_{\hat{V}\hat{W}}\in {\mathcal{D}}_n(P_{VW},\eta )\end{array}}\sum _{\begin{array}{c}\mathbf{w}:(\mathbf{v},\mathbf{w})\in {\mathcal{T}}_n\left(P_{\hat{V}\hat{W}}\right)\end{array}}{\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left(\mathbf{W}\left(1\right)=\mathbf{w}|{\mathcal{E}}_1^{\prime }\right)\hfill \\ & {(n+1)}^{\left|\mathcal{V}\right|\left|\mathcal{W}\right|}{e}^{nH\left(\hat{W}\right|\hat{V})}2e^{-n(H\left(\hat{W}\right)-\eta )}\hfill \\ & \le \frac{1}{e^{n{R}_n}}\sum _{j\in {\mathcal{M}}^{\prime }\setminus \{0,1\}}\sum _{\begin{array}{c}{P}_{\hat{W}}:P_{\hat{V}\hat{W}}\in {\mathcal{D}}_n(P_{VW},\eta )\end{array}}2{(n+1)}^{\left|\mathcal{V}\right|\left|\mathcal{W}\right|}{e}^{-n\left(I(\hat{W};\hat{V})-\eta \right)}\hfill \\ & \stackrel{\left(c\right)}{\le }\frac{1}{e^{n{R}_n}}\sum _{j\in {\mathcal{M}}^{\prime }\setminus \{0,1\}}2{(n+1)}^{\left|\mathcal{W}\right|}{(n+1)}^{\left|\mathcal{V}\right|\left|\mathcal{W}\right|}{e}^{-n\left(\underset{P_{\hat{V}\hat{W}}\in {\mathcal{D}}_n(P_{VW},\eta )}{min}I(\hat{W};\hat{V})-\eta \right)}\hfill \\ & \stackrel{\left(d\right)}{\le }{e}^{-n(R-R^{\prime }+{\rho }_n-{\eta }_n^{\prime })},\hfill \end{array}$$

(33)

where ${\rho }_n:={\min }_{P_{\hat{V}\hat{W}}\in {\mathcal{D}}_n(P_{VW},\eta )}I(\hat{V};\hat{W})$ and ${\eta }_n^{\prime }:=3\eta +o\left(1\right)$. In the above,

(a) 

used Lemma 2.3 in [38] and the fact that the codewords are chosen uniformly at random from ${\mathcal{T}}_n\left(P_{\hat{W}}\right)$;

(b) 

follows since the total number of sequences $\tilde{\mathbf{w}}\in {\mathcal{T}}_n\left(P_{\hat{W}}\right)$ such that $P_{\tilde{\mathbf{w}}\mathbf{v}}=P_{\tilde{W}\tilde{V}}$ and $H\left(\tilde{W}\right|\tilde{V})\le H\left(\hat{W}\right|\hat{V})$ is upper bounded by $e^{nH\left(\hat{W}\right|\hat{V})}$, and $|\mathcal{T}({\mathcal{W}}^n\times {\mathcal{V}}^n)|\le {(n+1)}^{\left|\mathcal{V}\right|\left|\mathcal{W}\right|}$;

(c) 

holds due to Lemma 2.2 in [38];

(d) 

follows from $R^{\prime }:=\zeta ({\kappa }_{\alpha }$, (14), (18) and (19).

Thus, since ${\rho }_n\to \rho ({\kappa }_{\alpha },\omega )+O\left(\eta \right)$, we have from (28), (29), (31), (33) for large enough n that

$$\begin{array}{c}\hfill {\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left({\mathcal{E}}_{\mathrm{BE}}\right)\le e^{-n\left(\min \left\{{\kappa }_{\alpha },R-\zeta ({\kappa }_{\alpha },\omega )+\rho ({\kappa }_{\alpha },\omega )-O\left(\eta \right)\right\}\right)}.\end{array}$$

(34)

By choice of $(\omega ,P_{SX},\theta )\in \mathcal{L}\left({\kappa }_{\alpha }\right)$, it follows from (24), (25), (26), (27) and (34) that the type I error probability is upper bounded by $e^{-n\left({\kappa }_{\alpha }-O\left(\eta \right)\right)}$ for large n.

Type II error probability: We analyze the type II error probability averaged over ${\mathbb{B}}_n$. A type II error can occur only under the following events:

(i)

${\mathcal{E}}_a:=\left\{\begin{array}{cc}& \hat{M}=M,{\hat{M}}^{\prime }=M^{\prime }\ne 0,(\mathbf{U},\mathbf{V},\mathbf{W}\left(M^{\prime }\right))\in {\mathcal{T}}_n\left(P_{\hat{U}\hat{V}\hat{W}}\right)\hfill \\ & \mathrm{s}.\mathrm{t}.P_{\hat{U}\hat{W}}\in {\mathcal{D}}_n(P_{UW},\eta ) \mathrm{and} P_{\hat{V}\hat{W}}\in {\mathcal{D}}_n(P_{VW},\eta )\hfill \end{array}\right\},$

(ii)

${\mathcal{E}}_b:=\left\{\begin{array}{cc}& M^{\prime }\ne 0,\hat{M}=M,{\hat{M}}^{\prime }\ne M^{\prime },f_{\mathbb{B}}\left({\hat{M}}^{\prime }\right)=f_{\mathbb{B}}\left(M^{\prime }\right),(\mathbf{U},\mathbf{V},\mathbf{W}\left(M^{\prime }\right),\hfill \\ & \mathbf{W}\left({\hat{M}}^{\prime }\right))\in {\mathcal{T}}_n\left(P_{\hat{U}\hat{V}\hat{W}{\hat{W}}_d}\right)\mathrm{s}.\mathrm{t}.P_{\hat{U}\hat{W}}\in {\mathcal{D}}_n(P_{UW},\eta ),\hfill \\ & P_{\hat{V}{\hat{W}}_d}\in {\mathcal{D}}_n(P_{VW},\eta ) \mathrm{and} H_e\left(\mathbf{W}\left({\hat{M}}^{\prime }\right)|\mathbf{V}\right)\le H_e\left(\mathbf{W}\left(M^{\prime }\right)|\mathbf{V}\right)\hfill \end{array}\right\},$

(iii)

${\mathcal{E}}_c:=\left\{\begin{array}{cc}& M^{\prime }\ne 0,\hat{M}\ne M \mathrm{o}\mathrm{r} 0,(\mathbf{U},\mathbf{V},\mathbf{W}\left(M^{\prime }\right),\mathbf{W}\left({\hat{M}}^{\prime }\right))\in {\mathcal{T}}_n\left(P_{\hat{U}\hat{V}\hat{W}{\hat{W}}_d}\right)\mathrm{s}.\mathrm{t}.\hfill \\ & P_{\hat{U}\hat{W}}\in {\mathcal{D}}_n(P_{UW},\eta ) \mathrm{and} P_{\hat{V}{\hat{W}}_d}\in {\mathcal{D}}_n(P_{VW},\eta )\hfill \end{array}\right\},$

(iv)

${\mathcal{E}}_d:=\left\{M=M^{\prime }=0,\hat{M}\ne M,(\mathbf{V},\mathbf{W}\left({\hat{M}}^{\prime }\right))\in {\mathcal{T}}_n\left(P_{\hat{V}{\hat{W}}_d}\right)\mathrm{s}.\mathrm{t}.P_{\hat{V}{\hat{W}}_d}\in {\mathcal{D}}_n(P_{VW},\eta )\right\}.$

Similar to (24), it follows that ${\overline{\mathbb{P}}}_{Q^{\left({\mathbb{B}}_n\right)}}\left({\mathcal{E}}_{\mathrm{EE}}\right)\le e^{-e^{n\Omega \left(\eta \right)}}$. Hence, we may assume that ${\mathcal{E}}_{\mathrm{EE}}^c$ holds for the type II error-exponent analysis. It then follows from the analysis in Equations (4.23)–(4.27) in [14] that for sufficiently large n,

$$\begin{array}{c}\hfill {\overline{\mathbb{P}}}_{Q^{\left({\mathbb{B}}_n\right)}}\left({\mathcal{E}}_a|{\mathcal{E}}_{\mathrm{EE}}^c\right)\le e^{-n\left(E_1({\kappa }_{\alpha },\omega )-O\left(\eta \right)\right)}.\end{array}$$

The analysis of the error events ${\mathcal{E}}_b$, ${\mathcal{E}}_c$ and ${\mathcal{E}}_d$ follows similarly to that in the proof of Theorem 2 in [10], and results in $\begin{array}{cc}& -\frac{1}{n}log\left({\overline{\mathbb{P}}}_{Q^{\left({\mathbb{B}}_n\right)}}\left({\mathcal{E}}_b\right)\right)\hfill \\ & \gtrsim \left\{\begin{array}{cc}\underset{\begin{array}{c}(P_{\tilde{U}\tilde{V}\tilde{W}},Q_{\tilde{U}\tilde{V}\tilde{W}})\in {\mathcal{T}}_2({\kappa }_{\alpha },\omega )\end{array}}{\min }D(P_{\tilde{U}\tilde{V}\tilde{W}}\left|\right|Q_{\tilde{U}\tilde{V}\tilde{W}})+E_{\mathrm{b}}({\kappa }_{\alpha },\omega ,R)-O\left(\eta \right),\hfill & \mathrm{if} R<\zeta ({\kappa }_{\alpha },\omega )+\eta ,\hfill \\ \infty ,\hfill & \mathrm{otherwise},\hfill \end{array}\right.\hfill \\ & =E_2({\kappa }_{\alpha },\omega ,R)-O\left(\eta \right).\hfill \\ & \frac{-1}{n}log\left({\overline{\mathbb{P}}}_{Q^{\left({\mathbb{B}}_n\right)}}\left({\mathcal{E}}_c\right)\right)\hfill \\ & \gtrsim \left\{\begin{array}{cc}\underset{\begin{array}{c}(P_{\tilde{U}\tilde{V}\tilde{W}},Q_{\tilde{U}\tilde{V}\tilde{W}})\in {\mathcal{T}}_3({\kappa }_{\alpha },\omega )\end{array}}{\min }D(P_{\tilde{U}\tilde{V}\tilde{W}}\left|\right|Q_{\tilde{U}\tilde{V}\tilde{W}})+E_{\mathrm{b}}({\kappa }_{\alpha },\omega ,R)\hfill & \mathrm{if} R<\zeta ({\kappa }_{\alpha },\omega )+\eta \hfill \\ +E_{\mathrm{ex}}\left(R,P_{SX}\right)-O\left(\eta \right),\hfill & \hfill \\ \underset{\begin{array}{c}(P_{\tilde{U}\tilde{V}\tilde{W}},Q_{\tilde{U}\tilde{V}\tilde{W}})\in {\mathcal{T}}_3({\kappa }_{\alpha },\omega )\end{array}}{\min }D(P_{\tilde{U}\tilde{V}\tilde{W}}\left|\right|Q_{\tilde{U}\tilde{V}\tilde{W}})+\rho ({\kappa }_{\alpha },\omega )\hfill & \mathrm{otherwise},\hfill \\ +E_{\mathrm{ex}}\left(R,P_{SX}\right)-O\left(\eta \right),\hfill \end{array}\right.\hfill \\ & =E_3({\kappa }_{\alpha },\omega ,R,P_{SX})-O\left(\eta \right).\hfill \\ & \frac{-1}{n}log\left({\overline{\mathbb{P}}}_{Q^{\left({\mathbb{B}}_n\right)}}\left({\mathcal{E}}_d\right)\right)\hfill \\ & \gtrsim \left\{\begin{array}{cc}\underset{P_{\tilde{V}}:P_{\tilde{V}\tilde{W}}\in {\mathcal{D}}_n(P_{VW},\eta )}{\min }D(P_{\tilde{V}}\left|\right|Q_V)+E_{\mathrm{b}}({\kappa }_{\alpha },\omega ,R)\hfill & \mathrm{if} R<\zeta ({\kappa }_{\alpha },\omega )+\eta ,\hfill \\ +E_{\mathrm{sp}}(P_{SX},\theta )-\theta -O\left(\eta \right),\hfill & \hfill \\ {\min }_{\begin{array}{c}{P}_{\tilde{V}}:P_{\tilde{V}\tilde{W}}\in {\mathcal{D}}_n(P_{VW},\eta )\end{array}}D(P_{\tilde{V}}\left|\right|Q_V)+\rho ({\kappa }_{\alpha },\omega )\hfill & \mathrm{otherwise},\hfill \\ +E_{\mathrm{sp}}(P_{SX},\theta )-\theta -O\left(\eta \right),\hfill \end{array}\right.\hfill \\ & =E_4({\kappa }_{\alpha },\omega ,R,P_{SX},\theta )-O\left(\eta \right).\hfill & \end{array}$ Since the exponent of the type II error probability is lower bounded by the minimum of the exponent of the type II error-causing events, we have shown from the above that for a fixed $(\omega ,R,P_{SX},\theta )\in \mathcal{L}\left({\kappa }_{\alpha }\right)$ and sufficiently large n,

$$\begin{array}{cc}\hfill {\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left(\hat{H}=1\right)& \le e^{-n({\kappa }_{\alpha }-O\left(\eta \right))},\hfill \end{array}$$

(35a)

$$\begin{array}{cc}\hfill \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}{\overline{\mathbb{P}}}_{Q^{\left({\mathbb{B}}_n\right)}}\left(\hat{H}=0\right)& \le e^{-n({\overline{\kappa }}_s({\kappa }_{\alpha },\omega ,R,P_{SX},\theta )-O\left(\eta \right))},\hfill \end{array}$$

(35b)

where

$$\begin{array}{c}\hfill {\overline{\kappa }}_s({\kappa }_{\alpha },\omega ,R,P_{SX},\theta ):=\min \left\{E_1({\kappa }_{\alpha },\omega ),E_2({\kappa }_{\alpha },\omega ,R),E_3({\kappa }_{\alpha },\omega ,R,P_{SX}),E_4({\kappa }_{\alpha },\omega ,R,P_{SX},\theta )\right\}.\end{array}$$

Expurgation: To complete the proof, we extract a deterministic codebook ${\mathcal{B}}_n^{★}$ that satisfies

$$\begin{array}{cc}\hfill {\mathbb{P}}_{P^{\left({\mathcal{B}}_n^{★}\right)}}\left(\hat{H}=1\right)& \le e^{-n({\kappa }_{\alpha }-O\left(\eta \right))},\hfill \\ \hfill {\mathbb{P}}_{Q^{\left({\mathcal{B}}_n^{★}\right)}}\left(\hat{H}=0\right)& \le e^{-n({\overline{\kappa }}_s({\kappa }_{\alpha },\omega ,R,P_{SX},\theta )-O\left(\eta \right))}.\hfill \end{array}$$

For this purpose, remove a set ${\mathfrak{B}}_n^{\prime }\subset {\mathfrak{B}}_n$ of highest type I error probability codebooks such that the remaining set ${\mathfrak{B}}_n\setminus {\mathfrak{B}}_n^{\prime }$ has a probability of $\tau \in (0.25,0.5)$, i.e., ${\mu }_n\left({\mathfrak{B}}_n\setminus {\mathfrak{B}}_n^{\prime }\right)=\tau$. Then, it follows from (35a) and (35b) that for all ${\mathcal{B}}_n\in {\mathfrak{B}}_n\setminus {\mathfrak{B}}_n^{\prime }$,

$$\begin{array}{cc}\hfill {\mathbb{P}}_{P^{\left({\mathcal{B}}_n\right)}}\left(\hat{H}=1\right)& \le 2e^{-n({\kappa }_{\alpha }-O\left(\eta \right))},\hfill \\ \hfill {\tilde{\mathbb{P}}}_{Q^{\left({\mathbb{B}}_n\right)}}\left(\hat{H}=0\right)& \le 4e^{-n({\overline{\kappa }}_s({\kappa }_{\alpha },\omega ,R,P_{SX},\theta )-O\left(\eta \right))},\hfill \end{array}$$

where ${\tilde{\mathbb{P}}}_{Q^{\left({\mathbb{B}}_n\right)}}=\frac{1}{\tau }{\mathbb{E}}_{{\mu }_n}\left[{\mathbb{P}}_{Q^{{\mathbb{B}}_n}}{\mathrm{𝟙}}_{\left\{{\mathbb{B}}_n\in {\mathfrak{B}}_n\setminus {\mathfrak{B}}_n^{\prime }\right\}}\right]$ is a PMF. Perform one more similar expurgation step to obtain ${\mathcal{B}}_n^{★}=\left({\mathcal{B}}_{W,n}^{★},f_b^{★},{\mathcal{B}}_{X,n}^{★}\right)\in {\mathfrak{B}}_n\setminus {\mathfrak{B}}_n^{\prime }$ such that for all sufficiently large n

$$\begin{array}{cc}\hfill {\mathbb{P}}_{P^{\left({\mathcal{B}}_n^{★}\right)}}\left(\hat{H}=1\right)& \le 2e^{-n({\kappa }_{\alpha }-O\left(\eta \right))}\le e^{-n\left({\kappa }_{\alpha }-O\left(\eta \right)-(log2/n)\right)},\hfill \\ \hfill {\mathbb{P}}_{Q^{\left({\mathcal{B}}_n^{★}\right)}}\left(\hat{H}=0\right)& \le 4e^{-n\left({\overline{\kappa }}_s({\kappa }_{\alpha },\omega ,R,P_{SX},\theta )-O\left(\eta \right)\right)}\le e^{-n\left({\overline{\kappa }}_s({\kappa }_{\alpha },\omega ,R,P_{SX},\theta )-O\left(\eta \right)-(log4/n)\right)}.\hfill \end{array}$$

Maximizing over $(\omega ,R,P_{SX},\theta )\in \mathcal{L}\left({\kappa }_{\alpha }\right)$ and noting that $\eta >0$ is arbitrary completes the proof.

4.2. Proof of Corollary 1

Consider $(\omega ,P_{SX},\theta )\in {\mathcal{L}}^{★}\left({\kappa }_{\alpha }\right)$ and $R=\zeta ({\kappa }_{\alpha },\omega )$. Then, $(\omega ,R,P_{SX},\theta )\in \mathcal{L}\left({\kappa }_{\alpha }\right)$. Additionally, for any $(P_{\tilde{U}\tilde{V}\tilde{W}},Q_{\tilde{U}\tilde{V}\tilde{W}})\in {\mathcal{T}}_1({\kappa }_{\alpha },\omega )$, we have

$$\begin{array}{cc}D\left(P_{\tilde{U}\tilde{V}\tilde{W}}\right|\left|Q_{\tilde{U}\tilde{V}\tilde{W}}\right)& =D(P_{\tilde{U}\tilde{W}}\left|\right|Q_{\tilde{U}\tilde{W}})+D\left(P_{\tilde{V}|\tilde{U}\tilde{W}}\left|\right|Q_{\tilde{V}|\tilde{U}\tilde{W}}|P_{\tilde{U}\tilde{W}}\right)\hfill \\ & \stackrel{\left(a\right)}{\ge }D\left(P_{\tilde{V}|\tilde{U}\tilde{W}}\left|\right|P_V|P_{\tilde{U}\tilde{W}}\right)\hfill \\ & =D\left(P_{\tilde{V}\tilde{U}\tilde{W}}\left|\right|P_V{P}_{\tilde{U}\tilde{W}}\right)\hfill \\ & \stackrel{\left(b\right)}{\ge }D\left(P_{\tilde{V}\tilde{W}}\left|\right|P_V{P}_{\tilde{W}}\right)\hfill \\ & \stackrel{\left(c\right)}{=}D\left(P_{\hat{V}\hat{W}}\left|\right|P_V{P}_{\hat{W}}\right)\hfill \\ & =I_P(\hat{V};\hat{W})+D(P_{\hat{V}}\left|\right|P_V),\hfill & \end{array}$$

(36)

where $\left(a\right)$ is due to the non-negativity of KL divergence and since $Q_{\tilde{V}|\tilde{U}\tilde{W}}=P_V$; $\left(b\right)$ is because of the monotonicity of KL divergence Theorem 2.14 in [43]; $\left(c\right)$ follows since for $(P_{\tilde{U}\tilde{V}\tilde{W}},Q_{\tilde{U}\tilde{V}\tilde{W}})\in {\mathcal{T}}_1({\kappa }_{\alpha },\omega )$, $P_{\tilde{V}\tilde{W}}=P_{\hat{V}\hat{W}}$ for some $P_{\hat{U}\hat{V}\hat{W}}\in \hat{\mathcal{L}}({\kappa }_{\alpha },\omega )$. Minimizing over all $P_{\hat{U}\hat{V}\hat{W}}\in \hat{\mathcal{L}}({\kappa }_{\alpha },\omega )$ yields that

$$\begin{array}{cc}{E}_1({\kappa }_{\alpha },\omega )& =\underset{\begin{array}{c}(P_{\tilde{U}\tilde{V}\tilde{W}},Q_{\tilde{U}\tilde{V}\tilde{W}})\in {\mathcal{T}}_1({\kappa }_{\alpha },\omega )\end{array}}{\min }D(P_{\tilde{U}\tilde{V}\tilde{W}}\left|\right|Q_{\tilde{U}\tilde{V}\tilde{W}})\hfill \\ & \ge \underset{P_{\hat{U}\hat{V}\hat{W}}\in \hat{\mathcal{L}}({\kappa }_{\alpha },\omega )}{\min }\left[I_P(\hat{V};\hat{W})+D(P_{\hat{V}}\left|\right|P_V)\right]\hfill \\ & =\underset{\begin{array}{c}{P}_{\hat{V}\hat{W}}:P_{\hat{U}\hat{V}\hat{W}}\in \hat{\mathcal{L}}({\kappa }_{\alpha },\omega )\end{array}}{\min }\left[I_P(\hat{V};\hat{W})+D(P_{\hat{V}}\left|\right|P_V)\right]:=E_1^{\mathrm{i}}({\kappa }_{\alpha },\omega ),\hfill & \end{array}$$

where the inequality above follows from (36). Next, since $\zeta ({\kappa }_{\alpha },\omega )=R$, we have that $E_2({\kappa }_{\alpha },\omega ,R)=\infty$. Additionally, by the non-negativity of KL divergence

$$\begin{array}{cc}{E}_3({\kappa }_{\alpha },\omega ,R,P_{SX})& =\underset{\begin{array}{c}(P_{\tilde{U}\tilde{V}\tilde{W}},Q_{\tilde{U}\tilde{V}\tilde{W}})\\ \in {\mathcal{T}}_3({\kappa }_{\alpha },\omega )\end{array}}{\min }D(P_{\tilde{U}\tilde{V}\tilde{W}}\left|\right|Q_{\tilde{U}\tilde{V}\tilde{W}})+\rho ({\kappa }_{\alpha },\omega )+E_{\mathrm{ex}}\left(R,P_{SX}\right)\hfill \\ & \ge \rho ({\kappa }_{\alpha },\omega )+E_{\mathrm{ex}}\left(\zeta ({\kappa }_{\alpha },\omega ),P_{SX}\right):=E_2^{\mathrm{i}}({\kappa }_{\alpha },\omega ,P_{SX}),\hfill \\ E_4({\kappa }_{\alpha },\omega ,P_{SX},\theta )& =\underset{P_{\hat{V}}:P_{\hat{U}\hat{V}\hat{W}}\in \hat{\mathcal{L}}({\kappa }_{\alpha },\omega )}{\min }D(P_{\hat{V}}\left|\right|P_V)+\rho ({\kappa }_{\alpha },\omega )+E_{\mathrm{m}}(P_{SX},\theta )-\theta \hfill \\ & =\rho ({\kappa }_{\alpha },\omega )+E_{\mathrm{m}}(P_{SX},\theta )-\theta :=E_3^{\mathrm{i}}({\kappa }_{\alpha },\omega ,P_{SX},\theta ),\hfill & \end{array}$$

where the final equality is since $P_{UV}{P}_{W|U}\in \hat{\mathcal{L}}({\kappa }_{\alpha },\omega )$ for $P_{W|U}:=\omega \left(P_U\right)$. The claim in (8) now follows from Theorem 1.

Next, we prove (10). Note that $\hat{\mathcal{L}}(0,\omega )=\{P_{UVW}=P_{UV}{P}_{W|U}:P_{W|U}=\omega \left(P_U\right)\}$ and ${\mathcal{L}}^{★}\left(0\right)=\{(\omega ,P_{SX},\theta )\in \mathfrak{F}\times \mathcal{P}(\mathcal{S}\times \mathcal{X})\times \Theta \left(P_{SX}\right):I_P(U;W)<I_P(X;Y|S),P_{W|U}=\omega \left(P_U\right),P_{SXY}:=P_{SX}{P}_{Y|X}\}$ since $E_{\mathrm{sp}}(P_{SX},\theta )\ge 0$ and $E_{\mathrm{ex}}\left(I_P(U;W),P_{SX}\right)\ge 0$. Hence, we have

$$\begin{array}{cc}\hfill E_1^{\mathrm{i}}(0,\omega )& \ge \underset{P_{\hat{U}\hat{V}\hat{W}}\in \hat{\mathcal{L}}(0,\omega )}{\min }{I}_P(\hat{V};\hat{W})=I_P(V;W).\hfill \end{array}$$

Additionally, $\rho (0,\omega )=I_P(V;W)$, $E_2^{\mathrm{i}}(0,\omega ,P_{SX})\ge \rho (0,\omega )$ and $E_3^{\mathrm{i}}(0,\omega ,P_{SX},\theta )\ge \rho (0,\omega )$. By choosing $P_{XS}=P_X^{★}{P}_S$ where $P_X^{★}$ is the capacity achieving input distribution, we have $I_P(X;Y|S)=C$. Then, it follows from (8) and the continuity of $E_1^{\mathrm{i}}({\kappa }_{\alpha },\omega )$, $E_2^{\mathrm{i}}({\kappa }_{\alpha },\omega ,P_{SX})$ and $E_3^{\mathrm{i}}({\kappa }_{\alpha },\omega ,P_{SX},\theta )$ in ${\kappa }_{\alpha }$ that ${lim}_{{\kappa }_{\alpha }\to 0}\kappa \left({\kappa }_{\alpha }\right)\ge {\kappa }_{\mathrm{i}}^{★}\left(0\right)$. On the other hand, ${lim}_{{\kappa }_{\alpha }\to 0}\kappa \left({\kappa }_{\alpha }\right)\le {\kappa }_{\mathrm{i}}^{★}\left(0\right)$ follows from the converse proof in Proposition 7 in [10]. The proof of the cardinality bound $\left|\mathcal{W}\right|\le \left|\mathcal{U}\right|+1$ follows from a standard application of the Eggleston–Fenchel–Carathéodory theorem (Theorem18 in [48]), thus completing the proof.

4.3. Proof of Corollary 2

Specializing Theorem 1 to TAD, note that $\rho ({\kappa }_{\alpha },\omega )=0$ since $P_{\hat{U}\hat{V}\hat{W}}=Q_U{Q}_V{P}_{\hat{W}|\hat{U}}\in \hat{\mathcal{L}}({\kappa }_{\alpha },\omega )$ and $I_P(\hat{V};\hat{W})=0$. Additionally, for $R\ge \zeta ({\kappa }_{\alpha },\omega )$, $E_{\mathrm{b}}({\kappa }_{\alpha },\omega ,R)=\infty$. Hence,

$$\begin{array}{cc}\hfill & \mathcal{L}\left({\kappa }_{\alpha }\right)-=-\left\{(\omega ,R,P_{SX},\theta ):\begin{array}{cc}\hfill & \zeta ({\kappa }_{\alpha },\omega )\le R<I_P(X;Y|S),P_{SXY}=P_{SX}{P}_{Y|X},\hfill \\ \hfill & \min \left\{E_{\mathrm{sp}}(P_{SX},\theta ),E_{\mathrm{ex}}\left(R,P_{SX}\right)\right\}\ge {\kappa }_{\alpha }\hfill \end{array}\right\},\hfill \\ \hfill & \hat{\mathcal{L}}({\kappa }_{\alpha },\omega ):=\left\{P_{\hat{U}\hat{V}\hat{W}}:D\left(P_{\hat{U}\hat{V}\hat{W}}\left|\right|P_{UV\hat{W}}\right)\le {\kappa }_{\alpha },P_{\hat{W}|\hat{U}}=\omega \left(P_{\hat{U}}\right),P_{UV\hat{W}}=Q_U{Q}_V{P}_{\hat{W}|\hat{U}}\right\}.\hfill \end{array}$$

Then, we have

$$\begin{array}{cc}{E}_1({\kappa }_{\alpha },\omega ):=E_1^{\mathrm{d}}({\kappa }_{\alpha },\omega )& :=\underset{\begin{array}{c}(P_{\tilde{U}\tilde{V}\tilde{W}},Q_{\tilde{U}\tilde{V}\tilde{W}})\in {\mathcal{T}}_1({\kappa }_{\alpha },\omega )\end{array}}{\min }D(P_{\tilde{U}\tilde{V}\tilde{W}}\left|\right|Q_{\tilde{U}\tilde{V}\tilde{W}})\hfill \\ & \stackrel{\left(a\right)}{\ge }\underset{\begin{array}{c}(P_{\tilde{U}\tilde{V}\tilde{W}},Q_{\tilde{U}\tilde{V}\tilde{W}})\in {\mathcal{T}}_1({\kappa }_{\alpha },\omega )\end{array}}{\min }D(P_{\tilde{V}\tilde{W}}\left|\right|Q_{\tilde{V}\tilde{W}})\hfill \\ & \stackrel{\left(b\right)}{=}\underset{\begin{array}{c}(P_{\hat{V}\hat{W}},Q_{V\hat{W}}):P_{\hat{U}\hat{V}\hat{W}}\in \hat{\mathcal{L}}({\kappa }_{\alpha },\omega ),\\ Q_{UV\hat{W}}=Q_{UV}{P}_{\hat{W}|\hat{U}}\end{array}}{\min }D(P_{\hat{V}\hat{W}}\left|\right|Q_{V\hat{W}}),\hfill & \end{array}$$

(37)

where $\left(a\right)$ follows due to the data-processing inequality for KL divergence Theorem 2.15 in [43]; $\left(b\right)$ is since $(P_{\tilde{U}\tilde{V}\tilde{W}},Q_{\tilde{U}\tilde{V}\tilde{W}})\in {\mathcal{T}}_1({\kappa }_{\alpha },\omega )$ implies that $P_{\tilde{V}\tilde{W}}=P_{\hat{V}\hat{W}}$ and $Q_{\tilde{U}\tilde{V}\tilde{W}}=Q_{UV}{P}_{\hat{W}|\hat{U}}$ for some $P_{\hat{U}\hat{V}\hat{W}}\in \hat{\mathcal{L}}({\kappa }_{\alpha },\omega )$. Next, note that since $R\ge \zeta ({\kappa }_{\alpha },\omega )$, $E_2({\kappa }_{\alpha },\omega ,R)=\infty$. Additionally,

$$\begin{array}{cc}\hfill E_3({\kappa }_{\alpha },\omega ,R,P_{SX})& =\underset{\begin{array}{c}(P_{\tilde{U}\tilde{V}\tilde{W}},Q_{\tilde{U}\tilde{V}\tilde{W}})\in {\mathcal{T}}_3({\kappa }_{\alpha },\omega )\end{array}}{\min }D(P_{\tilde{U}\tilde{V}\tilde{W}}\left|\right|Q_{\tilde{U}\tilde{V}\tilde{W}})+E_{\mathrm{ex}}\left(R,P_{SX}\right)\hfill \end{array}$$

(38a)

$$\begin{array}{cc}& \stackrel{\left(a\right)}{=}{E}_{\mathrm{ex}}\left(R,P_{SX}\right),\hfill \end{array}$$

(38b)

$$\begin{array}{cc}{E}_4({\kappa }_{\alpha },\omega ,P_{SX},\theta )& =\underset{P_{\hat{V}}:P_{\hat{U}\hat{V}\hat{W}}\in \hat{\mathcal{L}}({\kappa }_{\alpha },\omega )}{\min }D(P_{\hat{V}}\left|\right|Q_V)+E_{\mathrm{m}}(P_{SX},\theta )-\theta \hfill \end{array}$$

$$\begin{array}{ccc}& \stackrel{\left(b\right)}{=}{E}_{\mathrm{m}}(P_{SX},\theta )-\theta =:E_3^{\mathrm{d}}(P_{SX},\theta ),\hfill & \end{array}$$

(38c)

where

(a)

is obtained by taking $P_{\hat{U}\hat{V}\hat{W}}=Q_U{Q}_V{P}_{W|U}\in \hat{\mathcal{L}}({\kappa }_{\alpha },\omega )$ and $P_{W|U}=\omega \left(Q_U\right)$ in the definition of ${\mathcal{T}}_3({\kappa }_{\alpha },\omega )$. This implies that $(P_{\tilde{U}\tilde{V}\tilde{W}},Q_{\tilde{U}\tilde{V}\tilde{W}})=(Q_{UV}{P}_{W|U},Q_{UV}{P}_{W|U})\in {\mathcal{T}}_3({\kappa }_{\alpha },\omega )$, and hence that the first term in the right hand side (RHS) of (38a) is zero;

(b)

is due to $Q_U{Q}_V{P}_{W|U}\in \hat{\mathcal{L}}({\kappa }_{\alpha },\omega )$ for $P_{W|U}=\omega \left(Q_U\right)$.

Since $E_{\mathrm{ex}}\left(R,P_{SX}\right)$ is a non-increasing function of R and $R\ge \zeta ({\kappa }_{\alpha },\omega )$, selecting $R=\zeta ({\kappa }_{\alpha },\omega )$ maximizes $E_3({\kappa }_{\alpha },\omega ,R,P_{SX})$. Then, (11) follows from (37), (38b) and (38c).

Next, we prove (12). Note that $\zeta (0,\omega )=I_Q(U;W)$, where $Q_{UW}=Q_U{P}_{W|U}$, $P_{W|U}=\omega \left(Q_U\right)$, and since $E_{\mathrm{sp}}(P_{SX},\theta )\ge 0$ and $E_{\mathrm{ex}}\left(I_Q(U;W),P_{SX}\right)\ge 0$,

$$\begin{array}{cc}{\mathcal{L}}^{★}\left(0\right)& =\left\{\begin{array}{cc}& (\omega ,P_{SX},\theta )\in \mathfrak{F}\times \mathcal{P}(\mathcal{S}\times \mathcal{X})\times \Theta \left(P_{SX}\right):I_Q(U;W)<I_P(X;Y|S),\hfill \\ & Q_{UVW}=Q_{UV}{P}_{W|U},P_{W|U}=\omega \left(Q_U\right),P_{SXY}:=P_{SX}{P}_{Y|X}\hfill \end{array}\right\}.\hfill \end{array}$$

Additionally, $\hat{\mathcal{L}}(0,\omega )=\left\{Q_U{Q}_V{P}_{W|U}:P_{W|U}=\omega \left(Q_U\right)\right\}$. By choosing $\theta =-{\theta }_{\mathrm{l}}\left(P_{SX}\right)$ (defined above (6a)) that maximizes $E_3^{\mathrm{d}}(P_{SX},\theta )$, we have

$$\begin{array}{cc}& E_1^{\mathrm{d}}(0,\omega )\ge \underset{\begin{array}{c}(P_{\hat{V}\hat{W}},Q_{V\hat{W}}):P_{\hat{U}\hat{V}\hat{W}}\in \hat{\mathcal{L}}(0,\omega ),\\ Q_{UV\hat{W}}=Q_{UV}{P}_{\hat{W}|\hat{U}}\end{array}}{\min }D(P_{\hat{V}\hat{W}}\left|\right|Q_{V\hat{W}})\hfill \\ & =\underset{\begin{array}{c}(P_{W|U},P_{SX}):I_Q(U;W)\le I_P(X;Y|S),\\ Q_{UVW}=Q_{UV}{P}_{W|U},P_{SXY}=P_{SX}{P}_{Y|X}\end{array}}{\min }D(Q_V{Q}_W\left|\right|Q_{VW}),\hfill \end{array}$$

(39a)

$$\begin{array}{cc}& E_2^{\mathrm{d}}(0,\omega ,P_{SX})=E_{\mathrm{ex}}\left(I_Q(U;W),P_{SX}\right),\hfill \end{array}$$

(39b)

$$\begin{array}{ccc}& E_3^{\mathrm{d}}(P_{SX},-{\theta }_{\mathrm{l}}\left(P_{SX}\right))=E_{\mathrm{m}}(P_{SX},-{\theta }_{\mathrm{l}}\left(P_{SX}\right))+{\theta }_{\mathrm{l}}\left(P_{SX}\right)={\theta }_{\mathrm{l}}\left(P_{SX}\right),\hfill & \end{array}$$

(39c)

where (39c) is due to $E_{\mathrm{m}}(P_{SX},-{\theta }_{\mathrm{l}}\left(P_{SX}\right))=0$. The latter in turn follows similar to (A10) and (A11) from the definition of $E_{\mathrm{m}}(·,·)$. From (11), (39a,39b,39c), and the continuity of $E_1^{\mathrm{d}}({\kappa }_{\alpha },\omega )$, $E_2^{\mathrm{d}}({\kappa }_{\alpha },\omega ,P_{SX})$ in ${\kappa }_{\alpha }$, (12) follows. The proof of the cardinality bound $\left|\mathcal{W}\right|\le \left|\mathcal{U}\right|+1$ in the RHS of (39a) follows via a standard application of the Eggleston–Fenchel–Carathéodory Theorem (Theorem 18 in [48]). To see this, note that it is sufficient to preserve $\{Q_U\left(u\right),u\in \mathcal{U}\}$, $D\left(Q_V{Q}_W\right|\left|Q_{VW}\right)$ and $H_Q\left(U\right|W)$, all of which can be written as a linear combination of functionals of $Q_{U|W}(·|w)$ with weights $Q_W\left(w\right)$. Thus, it requires $\left|\mathcal{U}\right|-1$ points to preserve $\{Q_U\left(u\right),u\in \mathcal{U}\}$ and one each for $D\left(Q_V{Q}_W\right|\left|Q_{VW}\right)$ and $H_Q\left(U\right|W)$. This completes the proof.

4.4. Proof of Theorem 2

We will show that the error-exponent pairs $\left({\kappa }_{\alpha },{\kappa }_{\mathrm{h}}^{★}\left({\kappa }_{\alpha }\right)\right)$ and $\left({\kappa }_{\alpha },{\kappa }_{\mathrm{u}}^{★}\left({\kappa }_{\alpha }\right)\right)$ are achieved by a hybrid coding scheme and uncoded transmission scheme, respectively. First, we describe the hybrid coding scheme.

Let $n\in \mathbb{N}$, $\left|\mathcal{W}\right|<\infty$, ${\kappa }_{\alpha }>0$, and $(P_S,{\omega }^{\prime }(·,P_S),P_{X|USW},$$P_{X^{\prime }|US})\in {\mathcal{L}}_h\left({\kappa }_{\alpha }\right)$. Further, let $\eta >0$ be a small number, and choose a sequence $\mathbf{s}\in {\mathcal{T}}_n\left(P_{\hat{S}}\right)$, where $P_{\hat{S}}$ satisfies $D\left(P_{\hat{S}}\left|\right|P_S\right)\le \eta$. Set $R^{\prime }:={\zeta }^{\prime }({\kappa }_{\alpha },{\omega }^{\prime },P_{\hat{S}})$.

Encoding: The encoder performs type-based quantization followed by hybrid coding [40]. The details are as follows:

Quantization codebook: Let ${\mathcal{D}}_n(P_U,\eta )$ be as defined in (15). Consider some ordering on the types in ${\mathcal{D}}_n(P_U,\eta )$ and denote the elements as $P_{{\hat{U}}_i}$, $i\in \left[|{\mathcal{D}}_n(P_U,\eta )|\right]$. For each joint type $P_{\hat{S}{\hat{U}}_i}$ such that $P_{{\hat{U}}_i}\in {\mathcal{D}}_n(P_U,\eta )$ and $\hat{S}⫫{\hat{U}}_i$, choose a joint type variable $P_{\hat{S}{\hat{U}}_i{\hat{W}}_i}$, $P_{{\hat{W}}_i}\in \mathcal{T}\left({\mathcal{W}}^n\right)$, such that $D\left(P_{{\hat{W}}_i|{\hat{U}}_i\hat{S}}\left|\right|P_{W_i|U\hat{S}}|P_{{\hat{U}}_i\hat{S}}\right)\le \eta /3$ and $I(\hat{S},{\hat{U}}_i;{\hat{W}}_i)\le R^{\prime }+(\eta /3)$, where $P_{W_i|U,S}={\omega }^{\prime }(P_{{\hat{U}}_i},P_{\hat{S}})$. Define ${\mathcal{D}}_n(P_{SUW},\eta ):=\left\{P_{\hat{S}{\hat{U}}_i{\hat{W}}_i}:i\in \left[|{\mathcal{D}}_n(P_U,\eta )|\right]\right\}$, $R_i^{\prime }:=I_P(\hat{S},{\hat{U}}_i;{\hat{W}}_i)+(\eta /3)$ for $i\in \left[|{\mathcal{D}}_n(P_U,\eta )|\right]$ and ${\mathcal{M}}_i^{\prime }:=\left[1+{\sum }_{m=1}^{i-1}{e}^{n{R}_m^{\prime }}:{\sum }_{m=1}^i{e}^{n{R}_m^{\prime }}\right],i\in \left[|{\mathcal{D}}_n(P_U,\eta )|\right]$. Let ${\mathbb{B}}_{W,n}=\left\{\mathbf{W}\left(j\right)\in {\mathcal{W}}^n,1\le j\le {\sum }_{i=1}^{|{\mathcal{D}}_n(P_U,\eta )|}{e}^{n{R}_i^{\prime }}\right\}$ denote a random quantization codebook such that for $i\in \left[|{\mathcal{D}}_n(P_U,\eta )|\right]$, each codeword $\mathbf{W}\left(j\right)$, $j\in {\mathcal{M}}_i^{\prime }$, is independently selected from ${\mathcal{T}}_n\left(P_{{\hat{W}}_i}\right)$ according to uniform distribution, i.e., $\mathbf{W}\left(j\right)\sim \mathsf{Unif}\left[{\mathcal{T}}_n\left(P_{{\hat{W}}_i}\right)\right]$. Let ${\mathcal{B}}_{W,n}$ denote a realization of ${\mathbb{B}}_{W,n}$.

Type-based hybrid coding: For $\mathbf{u}\in {\mathcal{T}}_n\left(P_{{\hat{U}}_i}\right)$ such that $P_{{\hat{U}}_i}\in {\mathcal{D}}_n(P_U,\eta )$ for some $i\in \left[|{\mathcal{D}}_n(P_U,\eta )|\right]$, let

$$\overline{M}\left(\mathbf{u},{\mathcal{B}}_{W,n}\right):=\left\{j\in {\mathcal{M}}_i^{\prime }:\mathbf{w}\left(j\right)\in {\mathcal{B}}_{W,n},(\mathbf{s},\mathbf{u},\mathbf{w}\left(j\right))\in {\mathcal{T}}_n\left(P_{\hat{S}{\hat{U}}_i{\hat{W}}_i}\right),P_{\hat{S}{\hat{U}}_i{\hat{W}}_i}\in {\mathcal{D}}_n(P_{SUW},\eta )\right\}.$$

If $|\overline{M}\left(\mathbf{u},{\mathcal{B}}_{W,n}\right)|\ge 1$, let $M^{\prime }\left(\mathbf{u},{\mathcal{B}}_{W,n}\right)$ denote an index selected uniformly at random from the set $\overline{M}\left(\mathbf{u},{\mathcal{B}}_{W,n}\right)$; otherwise, set $M^{\prime }\left(\mathbf{u},{\mathcal{B}}_{W,n}\right)=0$. Given ${\mathcal{B}}_{W,n}$ and $\mathbf{u}\in {\mathcal{U}}^n$, the quantizer outputs $M^{\prime }=M^{\prime }\left(\mathbf{u},{\mathcal{B}}_{W,n}\right)$, where the support of $M^{\prime }$ is ${\mathcal{M}}^{\prime }:=\left\{0\right\}{\bigcup }_{i=1}^{|{\mathcal{D}}_n(P_U,\eta )|}{\mathcal{M}}_i^{\prime }$. Note that for sufficiently large n, it follows similarly to (18) that $|{\mathcal{M}}^{\prime }|\le e^{n(R^{\prime }+\eta )}$. For a given ${\mathcal{B}}_{W,n}$ and $\mathbf{u}\in {\mathcal{U}}^n$, the encoder transmits $\mathbf{X}\sim P_{X|USW}^{\otimes n}(·|\mathbf{u},\mathbf{s},\mathbf{w}\left(m^{\prime }\right))$ if $M^{\prime }=m^{\prime }\ne 0$, and ${\mathbf{X}}^{\prime }\sim P_{X^{\prime }|US}^{\otimes n}(·|\mathbf{u},\mathbf{s})$ if $M^{\prime }=0$.

Acceptance region: For a given codebook ${\mathcal{B}}_{W,n}$ and $m^{\prime }\in {\mathcal{M}}^{\prime }\setminus \left\{0\right\}$, let ${\mathcal{O}}_{m^{\prime }}$ denote the set of $\mathbf{u}$ such that $M^{\prime }\left(\mathbf{u},{\mathcal{B}}_{W,n}\right)=m^{\prime }$. For each $m^{\prime }\in {\mathcal{M}}^{\prime }\setminus \left\{0\right\}$ and $\mathbf{u}\in {\mathcal{O}}_{m^{\prime }}$, set

$$\begin{array}{c}\hfill {\mathcal{Z}}_{m^{\prime }}^{\prime }\left(\mathbf{u}\right)=\left\{(\mathbf{v},\mathbf{y})\in {\mathcal{V}}^n\times {\mathcal{Y}}^n:(\mathbf{s},\mathbf{u},{\overline{\mathbf{w}}}_{m^{\prime }},\mathbf{v},\mathbf{y})\in {\mathcal{J}}_n\left({\kappa }_{\alpha }+\eta ,P_{\hat{S}U{W}_{m^{\prime }}VY}\right)\right\},\end{array}$$

where recall that ${\mathcal{J}}_n(r,P_X):=\{\mathbf{x}\in {\mathcal{X}}^n:D\left(P_{\mathbf{x}}\left|\right|P_X\right)\le r\}$, and

$$P_{\hat{S}U{W}_{m^{\prime }}VXY}=P_{\hat{S}}{P}_{UV}{P}_{W_{m^{\prime }}|U\hat{S}}{P}_{X|U\hat{S}{W}_{m^{\prime }}}{P}_{Y|X},$$

(40a)

$$P_{W_{m^{\prime }}|U\hat{S}}={\omega }^{\prime }(P_{\mathbf{u}},P_{\hat{S}}) \mathrm{and} P_{X|U\hat{S}{W}_{m^{\prime }}}=P_{X|USW}.$$

(40b)

For $m^{\prime }\in {\mathcal{M}}^{\prime }\setminus \left\{0\right\}$, define ${\mathcal{Z}}_{m^{\prime }}^{\prime }:=\{(\mathbf{v},\mathbf{y}):(\mathbf{v},\mathbf{y})\in {\mathcal{Z}}_{m^{\prime }}^{\prime }\left(\mathbf{u}\right)\mathrm{for} \mathrm{some}\mathbf{u}\in {\mathcal{O}}_{m^{\prime }}\}.$ The acceptance region for $H_0$ is given by ${\mathcal{A}}_n:={\cup }_{m^{\prime }\in {\mathcal{M}}^{\prime }\setminus 0}\mathbf{s}\times m^{\prime }\times {\mathcal{Z}}_{m^{\prime }}^{\prime }$ or equivalently as ${\mathcal{A}}_n^e:={\cup }_{m^{\prime }\in {\mathcal{M}}^{\prime }\setminus 0}\mathbf{s}\times {\mathcal{O}}_{m^{\prime }}\times {\mathcal{Z}}_{m^{\prime }}^{\prime }.$

Decoding: Given codebook ${\mathcal{B}}_{W,n}$, $\mathbf{Y}=\mathbf{y}$, and $\mathbf{V}=\mathbf{v}$, if $(\mathbf{v},\mathbf{y})\in {\bigcup }_{m^{\prime }\in {\mathcal{M}}^{\prime }\setminus \left\{0\right\}}{\mathcal{Z}}_{m^{\prime }}^{\prime }$, then ${\hat{M}}^{\prime }={\hat{m}}^{\prime }$, where ${\hat{m}}^{\prime }:={arg\; min}_{j\in {\mathcal{M}}^{\prime }\setminus 0}{H}_e\left(\mathbf{w}\left(j\right)\right|\mathbf{v},\mathbf{y},\mathbf{s})$. Otherwise, ${\hat{M}}^{\prime }=0$. Denote the decoder induced by the above operations by $g_{{\mathcal{B}}_{W,n}}:{\mathcal{S}}^n\times {\mathcal{V}}^n\times {\mathcal{Y}}^n\to {\mathcal{M}}^{\prime }$.

Testing: If ${\hat{M}}^{\prime }=0$, $\hat{H}=1$ is declared. Otherwise, $\hat{H}=0$ or $\hat{H}=1$ is declared depending on whether $(\mathbf{s},{\hat{m}}^{\prime },\mathbf{v},\mathbf{y})\in {\mathcal{A}}_n$ or $(\mathbf{s},{\hat{m}}^{\prime },\mathbf{v},\mathbf{y})\notin {\mathcal{A}}_n$, respectively. Denote the decision function induced by $g_{{\mathcal{B}}_{W,n}}$ and ${\mathcal{A}}_n$ by $g_n:{\mathcal{S}}^n\times {\mathcal{V}}^n\times {\mathcal{Y}}^n\to \hat{\mathcal{H}}$.

Induced probability distribution: The PMFs induced by a code $c_n=(f_n,g_n)$ with respect to codebook ${\mathcal{B}}_{W,n}$ under $H_0$ and $H_1$ are

$$\begin{array}{cc}& P_{\mathbf{UV}{M}^{\prime }\mathbf{XY}{\hat{M}}^{\prime }\hat{H}}^{({\mathcal{B}}_{W,n},c_n)}(\mathbf{u},\mathbf{v},m^{\prime },\mathbf{x},\mathbf{y},{\hat{m}}^{\prime },\hat{h})\hfill \\ & :=\left\{\begin{array}{c}{P}_{UV}^{\otimes n}(\mathbf{u},\mathbf{v}){\mathrm{𝟙}}_{\left\{M^{\prime }\left(\mathbf{u},{\mathcal{B}}_{W,n}\right)=m^{\prime }\right\}}{P}_{X|USW}^{\otimes n}\left(\mathbf{x}\right|\mathbf{s},\mathbf{u},\mathbf{w}\left(m^{\prime }\right))P_{Y|X}^{\otimes n}\left(\mathbf{y}\right|\mathbf{x})\hfill \\ {\mathrm{𝟙}}_{\left\{g_{{\mathcal{B}}_{W,n}}(\mathbf{v},\mathbf{y},\mathbf{s})={\hat{m}}^{\prime }\right\}}{\mathrm{𝟙}}_{\left\{\hat{h}={\mathrm{𝟙}}_{\left\{(\mathbf{s},{\hat{m}}^{\prime },\mathbf{v},\mathbf{y})\in {\mathcal{A}}_n^c\right\}}\right\}},\hfill & \mathrm{if} m^{\prime }\ne 0,\hfill \\ P_{UV}^{\otimes n}(\mathbf{u},\mathbf{v}){\mathrm{𝟙}}_{\left\{M^{\prime }\left(\mathbf{u},{\mathcal{B}}_{W,n}\right)=m^{\prime }\right\}}{P}_{X^{\prime }|US}^{\otimes n}\left(\mathbf{x}\right|\mathbf{s},\mathbf{u})P_{Y|X}^{\otimes n}\left(\mathbf{y}\right|\mathbf{x})\hfill \\ {\mathrm{𝟙}}_{\left\{g_{{\mathcal{B}}_{W,n}}(\mathbf{v},\mathbf{y},\mathbf{s})={\hat{m}}^{\prime }\right\}}{\mathrm{𝟙}}_{\left\{\hat{h}={\mathrm{𝟙}}_{\left\{(\mathbf{s},{\hat{m}}^{\prime },\mathbf{v},\mathbf{y})\in {\mathcal{A}}_n^c\right\}}\right\}},\hfill & \mathrm{otherwise},\hfill \end{array}\right.\hfill & \end{array}$$

and

$$\begin{array}{cc}& Q_{\mathbf{UV}{M}^{\prime }\mathbf{XY}{\hat{M}}^{\prime }\hat{H}}^{({\mathcal{B}}_{W,n},c_n)}(\mathbf{u},\mathbf{v},m^{\prime },\mathbf{x},\mathbf{y},{\hat{m}}^{\prime },\hat{h})\hfill \\ & :=\left\{\begin{array}{c}{Q}_{UV}^{\otimes n}(\mathbf{u},\mathbf{v}){\mathrm{𝟙}}_{\left\{M^{\prime }\left(\mathbf{u},{\mathcal{B}}_{W,n}\right)=m^{\prime }\right\}}{P}_{X|USW}^{\otimes n}\left(\mathbf{x}\right|\mathbf{s},\mathbf{u},\mathbf{w}\left(m^{\prime }\right))P_{Y|X}^{\otimes n}\left(\mathbf{y}\right|\mathbf{x})\hfill \\ {\mathrm{𝟙}}_{\left\{g_{{\mathcal{B}}_{W,n}}(\mathbf{v},\mathbf{y},\mathbf{s})={\hat{m}}^{\prime }\right\}}{\mathrm{𝟙}}_{\left\{\hat{h}={\mathrm{𝟙}}_{\left\{(\mathbf{s},{\hat{m}}^{\prime },\mathbf{v},\mathbf{y})\in {\mathcal{A}}_n^c\right\}}\right\}},\hfill & \mathrm{if} m^{\prime }\ne 0,\hfill \\ Q_{UV}^{\otimes n}(\mathbf{u},\mathbf{v}){\mathrm{𝟙}}_{\left\{M^{\prime }\left(\mathbf{u},{\mathcal{B}}_{W,n}\right)=m^{\prime }\right\}}{P}_{X^{\prime }|US}^{\otimes n}\left(\mathbf{x}\right|\mathbf{s},\mathbf{u})P_{Y|X}^{\otimes n}\left(\mathbf{y}\right|\mathbf{x})\hfill \\ {\mathrm{𝟙}}_{\left\{g_{{\mathcal{B}}_{W,n}}(\mathbf{v},\mathbf{y},\mathbf{s})={\hat{m}}^{\prime }\right\}}{\mathrm{𝟙}}_{\left\{\hat{h}={\mathrm{𝟙}}_{\left\{(\mathbf{s},{\hat{m}}^{\prime },\mathbf{v},\mathbf{y})\in {\mathcal{A}}_n^c\right\}}\right\}},\hfill & \mathrm{otherwise},\hfill \end{array}\right.\hfill & \end{array}$$

respectively. For brevity, we will denote ${\mathcal{B}}_{W,n}$ by ${\mathcal{B}}_n$, ${\mathbb{B}}_{W,n}$ by ${\mathbb{B}}_n$, and the above probability distributions by $P^{\left({\mathcal{B}}_n\right)}$ and $Q^{\left({\mathcal{B}}_n\right)}$. Let ${\mathfrak{B}}_n$ and ${\mu }_n$ stand for the support and probability measure of ${\mathbb{B}}_n$, respectively, and set ${\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}:={\mathbb{E}}_{{\mu }_n}\left[{\mathbb{P}}_{P^{\left({\mathbb{B}}_n\right)}}\right]$, ${\overline{\mathbb{P}}}_{Q^{\left({\mathbb{B}}_n\right)}}:={\mathbb{E}}_{{\mu }_n}\left[{\mathbb{P}}_{Q^{\left({\mathbb{B}}_n\right)}}\right]$

Analysis of the type I and type II error probabilities: We analyze the expected type I and type II error probabilities, where the expectation is with respect to the randomness of ${\mathbb{B}}_n$, followed by the expurgation technique to extract a sequence of deterministic codebooks ${\left\{{\mathcal{B}}_n\right\}}_{n\in \mathbb{N}}$ and a code ${\{c_n=(f_n,g_n)\}}_{n\in \mathbb{N}}$ that achieves the lower bound in Theorem 2.

Type I error probability: Denoting by ${\mathbb{A}}_n$ the random acceptance region for $H_0$, note that a type I error can occur only under the following events:

(i)

${\mathcal{E}}_{\mathrm{EE}}^{\prime }:={\displaystyle \bigcup _{P_{\hat{U}}\in {\mathcal{D}}_n(P_U,\eta )}}{\displaystyle \bigcup _{\mathbf{u}\in {\mathcal{T}}_n\left(P_{\hat{U}}\right)}}{\mathcal{E}}_{\mathrm{EE}}^{\prime }\left(\mathbf{u}\right)$, where

$$\begin{array}{ccc}\hfill {\mathcal{E}}_{\mathrm{EE}}^{\prime }\left(\mathbf{u}\right):=\left\{\begin{array}{cc}& \nexists j\in {\mathcal{M}}^{\prime }\setminus \left\{0\right\}\mathrm{s}.\mathrm{t}.(\mathbf{s},\mathbf{u},\mathbf{W}\left(j\right))\in {\mathcal{T}}_n\left(P_{\hat{S}{\hat{U}}_i{\hat{W}}_i}\right),P_{\hat{S}{\hat{U}}_i}=P_{\mathbf{s}\mathbf{u}},\hfill \\ & P_{\hat{S}{\hat{U}}_i{\hat{W}}_i}\in {\mathcal{D}}_n(P_{SUW},\eta )\hfill \end{array}\right\},& & \hfill \end{array}$$

(ii)

${\mathcal{E}}_{\mathrm{NE}}^{\prime }:=\{{\hat{M}}^{\prime }=M^{\prime }$ and $(\mathbf{s},{\hat{M}}^{\prime },\mathbf{V},\mathbf{Y})\notin {\mathbb{A}}_n\}$,

(iii)

${\mathcal{E}}_{\mathrm{ODE}}^{\prime }:=\{M^{\prime }\ne 0$, ${\hat{M}}^{\prime }\ne M^{\prime }$ and $(\mathbf{s},{\hat{M}}^{\prime },\mathbf{V},\mathbf{Y})\notin {\mathbb{A}}_n\}$,

(iv)

${\mathcal{E}}_{\mathrm{SDE}}^{\prime }:=\{M^{\prime }=0$, ${\hat{M}}^{\prime }\ne M^{\prime }$ and $(\mathbf{s},{\hat{M}}^{\prime },\mathbf{V},\mathbf{Y})\notin {\mathbb{A}}_n\}$.

By definition of $R_i^{\prime }$, we have, similar to (24), the following:

$$\begin{array}{c}\hfill {\overline{\mathbb{P}}}_{P^{{\mathbb{B}}_n}}\left({\mathcal{E}}_{\mathrm{EE}}^{\prime }\right)\le e^{-e^{n\Omega \left(\eta \right)}}.\end{array}$$

(41)

Next, the event ${\mathcal{E}}_{\mathrm{NE}}^{\prime }$ can be upper bounded as

$$\begin{array}{cc}{\overline{\mathbb{P}}}_{P^{{\mathbb{B}}_n}}\left({\mathcal{E}}_{\mathrm{NE}}^{\prime }|{\mathcal{E}}_{\mathrm{EE}}^{\prime c}\right)& \le {\overline{\mathbb{P}}}_{P^{{\mathbb{B}}_n}}\left((\mathbf{s},{\hat{M}}^{\prime },\mathbf{V},\mathbf{Y})\notin {\mathbb{A}}_n|{\hat{M}}^{\prime }=M^{\prime },{\mathcal{E}}_{EE}^{\prime c}\right)\hfill \\ & =1-{\overline{\mathbb{P}}}_{P^{{\mathbb{B}}_n}}\left((\mathbf{s},\mathbf{U},\mathbf{V},\mathbf{Y})\in {\mathbb{A}}_n^e|{\mathcal{E}}_{EE}^{\prime c}\right).\hfill \end{array}$$

(42)

For $\mathbf{u}\in {\mathbb{O}}_{m^{\prime }}$, note that, similar to Equation 4.17 in [14], we have

$$\begin{array}{c}{\overline{\mathbb{P}}}_{P^{{\mathbb{B}}_n}}\left((\mathbf{V},\mathbf{Y})\in {\mathbb{Z}}_{m^{\prime }}^{\prime }\left(\mathbf{u}\right)|\mathbf{U}=\mathbf{u},\mathbf{W}\left(m^{\prime }\right)={\overline{\mathbf{w}}}_{m^{\prime }},{\mathcal{E}}_{EE}^{\prime c}\right)\ge 1-e^{-n\left({\kappa }_{\alpha }+\frac{\eta }{3}-D(P_{\mathbf{u}}\left|\right|P_U)\right)}.\end{array}$$

From this and (15), we obtain, similar to Equation (4.22) in [14] that

$$\begin{array}{c}\hfill {\overline{\mathbb{P}}}_{P^{{\mathbb{B}}_n}}((\mathbf{s},\mathbf{U},\mathbf{V},\mathbf{Y})\in {\mathbb{A}}_e^n|{\mathcal{E}}_{EE}^{\prime c})\ge 1-e^{-n{\kappa }_{\alpha }}.\end{array}$$

(43)

Substituting (43) in (42) yields

$$\begin{array}{cc}\hfill & {\overline{\mathbb{P}}}_{P^{{\mathbb{B}}_n}}\left({\mathcal{E}}_{\mathrm{NE}}^{\prime }|{\mathcal{E}}_{\mathrm{EE}}^{\prime c}\right)\le e^{-n{\kappa }_{\alpha }}.\hfill \end{array}$$

(44)

Next, we bound the probability of the event ${\mathcal{E}}_{\mathrm{ODE}}^{\prime }$ as follows:

$$\begin{array}{cc}& {\overline{\mathbb{P}}}_{P^{{\mathbb{B}}_n}}\left({\mathcal{E}}_{\mathrm{ODE}}^{\prime }\right)\hfill \\ & ={\overline{\mathbb{P}}}_{P^{{\mathbb{B}}_n}}\left(M^{\prime }\ne 0,{\hat{M}}^{\prime }\ne M^{\prime },(\mathbf{s},M^{\prime },\mathbf{V},\mathbf{Y})\in {\mathbb{A}}_n,(\mathbf{s},{\hat{M}}^{\prime },\mathbf{V},\mathbf{Y})\notin {\mathbb{A}}_n\right)\hfill \\ & +{\overline{\mathbb{P}}}_{P^{{\mathbb{B}}_n}}\left(M^{\prime }\ne 0,{\hat{M}}^{\prime }\ne M^{\prime },(\mathbf{s},M^{\prime },\mathbf{V},\mathbf{Y})\notin {\mathbb{A}}_n,(\mathbf{s},{\hat{M}}^{\prime },\mathbf{V},\mathbf{Y})\notin {\mathbb{A}}_n\right)\hfill \\ & \le {\overline{\mathbb{P}}}_{P^{{\mathbb{B}}_n}}\left(M^{\prime }\ne 0,{\hat{M}}^{\prime }\ne M^{\prime },(\mathbf{s},M^{\prime },\mathbf{V},\mathbf{Y})\in {\mathbb{A}}_n,(\mathbf{s},{\hat{M}}^{\prime },\mathbf{V},\mathbf{Y})\notin {\mathbb{A}}_n\right)\hfill \\ & +{\overline{\mathbb{P}}}_{P^{{\mathbb{B}}_n}}\left(M^{\prime }\ne 0,{\hat{M}}^{\prime }\ne M^{\prime },(\mathbf{s},M^{\prime },\mathbf{V},\mathbf{Y})\notin {\mathbb{A}}_n\right)\hfill \end{array}$$

$$\begin{array}{cc}& \stackrel{\left(a\right)}{\le }{\overline{\mathbb{P}}}_{P^{{\mathbb{B}}_n}}\left(M^{\prime }\ne 0,{\hat{M}}^{\prime }\ne M^{\prime },(\mathbf{s},M^{\prime },\mathbf{V},\mathbf{Y})\in {\mathbb{A}}_n,(\mathbf{s},{\hat{M}}^{\prime },\mathbf{V},\mathbf{Y})\notin {\mathbb{A}}_n\right)+e^{-e^{n\Omega \left(\eta \right)}}+e^{-n{\kappa }_{\alpha }}\hfill \end{array}$$

(45)

$$\begin{array}{cc}& \le {\overline{\mathbb{P}}}_{P^{{\mathbb{B}}_n}}\left({\hat{M}}^{\prime }\ne M^{\prime }|M^{\prime }\ne 0,(\mathbf{s},M^{\prime },\mathbf{V},\mathbf{Y})\in {\mathbb{A}}_n\right)+e^{-e^{n\Omega \left(\eta \right)}}+e^{-n{\kappa }_{\alpha }},\hfill \end{array}$$

$$\begin{array}{cc}& \stackrel{\left(b\right)}{=}{\overline{\mathbb{P}}}_{P^{{\mathbb{B}}_n}}\left({\hat{M}}^{\prime }\ne M^{\prime }|M^{\prime }\ne 0,{\hat{M}}^{\prime }\ne 0,(\mathbf{s},M^{\prime },\mathbf{V},\mathbf{Y})\in {\mathbb{A}}_n\right)\hfill \end{array}$$

(46)

$$\begin{array}{cc}& \stackrel{\left(c\right)}{\le }{e}^{-n\left({\rho }^{\prime }({\kappa }_{\alpha },{\omega }^{\prime },P_S,P_{X|USW})-{\zeta }^{\prime }({\kappa }_{\alpha },{\omega }^{\prime },P_{\hat{S}})-O\left(\eta \right)\right)},\hfill \end{array}$$

(47)

where $\left(a\right)$ follows similar to (29) using (41) and (43); $\left(b\right)$ is since $(\mathbf{s},M^{\prime },\mathbf{V},\mathbf{Y})\in {\mathbb{A}}_n$ implies that ${\hat{M}}^{\prime }\ne 0$; and $\left(c\right)$ follows similar to (33). Further,

$$\begin{array}{cc}{\overline{\mathbb{P}}}_{P^{{\mathbb{B}}_n}}\left({\mathcal{E}}_{\mathrm{SDE}}^{\prime }\right)& \le {\overline{\mathbb{P}}}_{P^{{\mathbb{B}}_n}}\left(M^{\prime }=0\right)\hfill \\ & \le {\overline{\mathbb{P}}}_{P^{{\mathbb{B}}_n}}\left(M^{\prime }=0|{\mathcal{E}}_{EE}^{\prime c}\right)+{\overline{\mathbb{P}}}_{P^{{\mathbb{B}}_n}}\left({\mathcal{E}}_{EE}^{\prime }\right)\hfill \\ & =\sum _{\begin{array}{c}\mathbf{u}:P_{\mathbf{u}}\notin {\mathcal{D}}_n(P_U,\eta )\end{array}}{P}_U^{\otimes n}\left(\mathbf{u}\right)+{\overline{\mathbb{P}}}_{P^{{\mathbb{B}}_n}}\left({\mathcal{E}}_{EE}^{\prime }\right)\hfill \\ & \le e^{-n{\kappa }_{\alpha }}+e^{-e^{n\Omega \left(\eta \right)}},\hfill & & \hfill \end{array}$$

(48)

where the penultimate equality is since given ${\mathcal{E}}_{EE}^{\prime c}$, $M^{\prime }=0$ occurs only for $\mathbf{U}=\mathbf{u}$ such that $P_{\mathbf{u}}\notin {\mathcal{D}}_n(P_U,\eta )$, and the final inequality follows from (41), the definition of ${\mathcal{D}}_n(P_U,\eta )$ and Lemma 1.6 in [38]. From (41), (44), (47) and (48), the expected type I error probability satisfies $e^{-n({\kappa }_{\alpha }-O\left(\eta \right))}$ for sufficiently large n via the union bound.

Type II error probability: Next, we analyze the expected type II error probability over ${\mathbb{B}}_n$. Let

$$\begin{array}{cc}\hfill & {\mathcal{D}}_n(P_{SVWY},\eta ):=\left\{\begin{array}{cc}{P}_{\hat{S}\hat{V}\hat{W}\hat{Y}}:\hfill & \exists (\mathbf{s},\mathbf{u},\mathbf{v},\overline{\mathbf{w}},\mathbf{y})\in \underset{m^{\prime }\in {\mathcal{M}}^{\prime }\setminus \left\{0\right\}}{\cup }{\mathcal{J}}_n\left({\kappa }_{\alpha }+\eta ,P_{\hat{S}UV{W}_{m^{\prime }}Y}\right),\hfill \\ & P_{\hat{S}UV{W}_{m^{\prime }}Y}\mathrm{satisfies}\left(40\right) \mathrm{and} P_{\mathbf{s}\mathbf{u}\mathbf{v}\overline{\mathbf{w}}\mathbf{y}}=P_{\hat{S}\hat{U}\hat{V}\hat{W}\hat{Y}}\hfill \end{array}\right\},\hfill \\ \hfill & {\mathcal{F}}_{1,n}^{\prime }\left(\eta \right):=\left\{\begin{array}{cc}{P}_{\hat{S}\tilde{U}\tilde{V}\tilde{W}\tilde{Y}}\in \mathcal{T}\left({\mathcal{S}}^n\times {\mathcal{U}}^n\times {\mathcal{V}}^n\times {\mathcal{W}}^n\times {\mathcal{Y}}^n\right)\hfill & :P_{\hat{S}\tilde{U}\tilde{W}}\in {\mathcal{D}}_n(P_{SUW},\eta ),\hfill \\ \hfill & P_{\hat{S}\tilde{V}\tilde{W}\tilde{Y}}\in {\mathcal{D}}_n(P_{SVWY},\eta )\hfill \end{array}\right\}.\hfill \end{array}$$

A type II error can occur only under the following events:

(a) 

${\mathcal{E}}_a^{\prime }:=\left\{\begin{array}{cc}& {\hat{M}}^{\prime }=M^{\prime }\ne 0,(\mathbf{s},\mathbf{U},\mathbf{V},\mathbf{W}\left(M^{\prime }\right),\mathbf{Y})\in {\mathcal{T}}_n\left(P_{\hat{S}\hat{U}\hat{V}\hat{W}\hat{Y}}\right)\hfill \\ & \mathrm{s}.\mathrm{t}.P_{\hat{U}\hat{W}}\in {\mathcal{D}}_n(P_{SUW},\eta ) \mathrm{and} P_{\hat{S}\hat{V}\hat{W}\hat{Y}}\in {\mathcal{D}}_n(P_{SVWY},\eta )\hfill \end{array}\right\}$,

(b) 

${\mathcal{E}}_b^{\prime }:=\left\{\begin{array}{cc}& M^{\prime }\ne 0,{\hat{M}}^{\prime }\ne M^{\prime },(\mathbf{s},\mathbf{U},\mathbf{V},\mathbf{W}\left(M^{\prime }\right),\mathbf{Y},\mathbf{W}\left({\hat{M}}^{\prime }\right))\in {\mathcal{T}}_n\left(P_{\hat{S}\hat{U}\hat{V}\hat{W}\hat{Y}{\hat{W}}_d}\right)\hfill \\ & \mathrm{s}.\mathrm{t}.P_{\hat{S}\hat{U}\hat{W}}\in {\mathcal{D}}_n(P_{SUW},\eta ),P_{\hat{S}\hat{V}{\hat{W}}_d\hat{Y}}\in {\mathcal{D}}_n(P_{SVWY},\eta ),\hfill \\ & \mathrm{and} H_e\left(\mathbf{W}\left({\hat{M}}^{\prime }\right)|\mathbf{s},\mathbf{V},\mathbf{Y}\right)\le H_e\left(\mathbf{W}\left(M^{\prime }\right)|\mathbf{s},\mathbf{V},\mathbf{Y}\right)\hfill \end{array}\right\}$,

(c) 

${\mathcal{E}}_c^{\prime }:=\left\{\begin{array}{cc}& M^{\prime }=0,{\hat{M}}^{\prime }\ne M^{\prime },(\mathbf{s},\mathbf{V},\mathbf{Y},\mathbf{W}\left({\hat{M}}^{\prime }\right))\in {\mathcal{T}}_n\left(P_{\hat{S}\hat{V}\hat{Y}{\hat{W}}_d}\right)\hfill \\ & \mathrm{s}.\mathrm{t}.P_{\hat{S}\hat{V}{\hat{W}}_d\hat{Y}}\in {\mathcal{D}}_n(P_{SVWY},\eta )\hfill \end{array}\right\}.$

Considering the event ${\mathcal{E}}_a^{\prime }$, we have

$$\begin{array}{cc}& {\overline{\mathbb{P}}}_{Q^{{\mathbb{B}}_n}}\left({\mathcal{E}}_a^{\prime }\right)\hfill \\ & \le \sum _{\begin{array}{c}{P}_{\hat{S}\tilde{U}\tilde{V}\tilde{W}\tilde{Y}}\\ \in {\mathcal{F}}_{1,n}^{\prime }\left(\eta \right)\end{array}}\sum _{\begin{array}{c}(\mathbf{u},\mathbf{v},\overline{\mathbf{w}},\mathbf{y}):\\ (\mathbf{s},\mathbf{u},\mathbf{v},\overline{\mathbf{w}},\mathbf{y})\\ \in {\mathcal{T}}_n\left(P_{\hat{S}\tilde{U}\tilde{V}\tilde{W}\tilde{Y}}\right)\end{array}}\sum _{\begin{array}{c}{m}^{\prime }\in {\mathcal{M}}^{\prime }\setminus \left\{0\right\}\end{array}}{\overline{\mathbb{P}}}_{Q^{{\mathbb{B}}_n}}\left(\mathbf{U}=\mathbf{u},\mathbf{V}=\mathbf{v},M^{\prime }=m^{\prime },\mathbf{W}\left(m^{\prime }\right)=\overline{\mathbf{w}},\mathbf{Y}=\mathbf{y}|\mathbf{S}=\mathbf{s}\right)\hfill \\ & \le \sum _{\begin{array}{c}{P}_{\hat{S}\tilde{U}\tilde{V}\tilde{W}\tilde{Y}}\in {\mathcal{F}}_{1,n}^{\prime }\left(\eta \right)\end{array}}\sum _{\begin{array}{c}(\mathbf{u},\mathbf{v},\overline{\mathbf{w}},\mathbf{y}):\\ (\mathbf{s},\mathbf{u},\mathbf{v},\overline{\mathbf{w}},\mathbf{y})\in {\mathcal{T}}_n\left(P_{\hat{S}\tilde{U}\tilde{V}\tilde{W}\tilde{Y}}\right)\end{array}}\sum _{m^{\prime }\in {\mathcal{M}}^{\prime }\setminus \left\{0\right\}}{\overline{\mathbb{P}}}_{Q^{{\mathbb{B}}_n}}\left(\mathbf{U}=\mathbf{u},\mathbf{V}=\mathbf{v},M^{\prime }=m^{\prime }|\mathbf{S}=\mathbf{s}\right)\hfill \\ & {\overline{\mathbb{P}}}_{Q^{{\mathbb{B}}_n}}\left(\mathbf{W}\left(m^{\prime }\right)=\overline{\mathbf{w}}|\mathbf{U}=\mathbf{u},\mathbf{V}=\mathbf{v},M^{\prime }=m^{\prime },\mathbf{S}=\mathbf{s}\right)\hfill \\ & {\overline{\mathbb{P}}}_{Q^{{\mathbb{B}}_n}}\left(\mathbf{Y}=\mathbf{y}|\mathbf{U}=\mathbf{u},\mathbf{V}=\mathbf{v},M^{\prime }=m^{\prime },\mathbf{W}\left(m^{\prime }\right)=\overline{\mathbf{w}},\mathbf{S}=\mathbf{s}\right)\hfill \\ & \stackrel{\left(a\right)}{\le }\sum _{\begin{array}{c}{P}_{\hat{S}\tilde{U}\tilde{V}\tilde{W}\tilde{Y}}\in {\mathcal{F}}_{1,n}^{\prime }\left(\eta \right)\end{array}}\sum _{\begin{array}{c}(\mathbf{u},\mathbf{v},\overline{\mathbf{w}},\mathbf{y}):\\ (\mathbf{s},\mathbf{u},\mathbf{v},\overline{\mathbf{w}},\mathbf{y})\in {\mathcal{T}}_n\left(P_{\hat{S}\tilde{U}\tilde{V}\tilde{W}\tilde{Y}}\right)\end{array}}{e}^{-n\left(H(\tilde{U},\tilde{V})+D\left(P_{\tilde{U}\tilde{V}}\left|\right|Q_{UV}\right)\right)}{e}^{-n\left(H\left(\tilde{W}\right|\hat{S},\tilde{U})-\eta \right)}\hfill \\ & e^{-n\left(H\left(\tilde{Y}\right|\tilde{U},\hat{S},\tilde{W})+D\left(P_{\tilde{Y}|\tilde{U}\hat{S}\tilde{W}}\left|\right|P_{Y|USW}|P_{\tilde{U}\hat{S}\tilde{W}}\right)\right)}\hfill \\ & \le \sum _{\begin{array}{c}{P}_{\hat{S}\tilde{U}\tilde{V}\tilde{W}\tilde{Y}}\in {\mathcal{F}}_{1,n}^{\prime }\left(\eta \right)\end{array}}{e}^{nH(\tilde{U},\tilde{V},\tilde{W},\tilde{Y}|\hat{S})}{e}^{-n\left(H(\tilde{U},\tilde{V})+D\left(P_{\tilde{U}\tilde{V}}\left|\right|Q_{UV}\right)\right)}{e}^{-n\left(H\left(\tilde{W}\right|\hat{S},\tilde{U})-\eta \right)}\hfill \\ & e^{-n\left(H\left(\tilde{Y}\right|\tilde{U},\hat{S},\tilde{W})+D\left(P_{\tilde{Y}|\tilde{U}\hat{S}\tilde{W}}\left|\right|P_{Y|USW}|P_{\tilde{U}\hat{S}\tilde{W}}\right)\right)}\hfill \\ & \le e^{-n{\mathsf{E}}_{1,n}^{\prime }},\hfill & \end{array}$$

(49)

where

$$\begin{array}{cc}\hfill {\mathsf{E}}_{1,n}^{\prime }& :=\underset{P_{\hat{S}\tilde{U}\tilde{V}\tilde{W}\tilde{Y}}\in {\mathcal{F}}_{1,n}^{\prime }\left(\eta \right)}{\min }H(\tilde{U},\tilde{V})+D\left(P_{\tilde{U}\tilde{V}}\left|\right|Q_{UV}\right)+H\left(\tilde{W}\right|\hat{S},\tilde{U})-\eta +H\left(\tilde{Y}\right|\tilde{U},\hat{S},\tilde{W})\hfill \\ \hfill & +D\left(P_{\tilde{Y}|\tilde{U}\hat{S}\tilde{W}}\left|\right|P_{Y|USW}|P_{\tilde{U}\hat{S}\tilde{W}}\right)-H(\tilde{U},\tilde{V},\tilde{W},\tilde{Y}|\hat{S})-\frac{1}{n}\left|\right|\mathcal{U}\left|\right|\mathcal{V}\left|\right|\mathcal{W}\left|\right|\mathcal{Y}|log(n+1)\hfill \\ \hfill & \gtrsim \underset{\begin{array}{c}(P_{\tilde{U}\tilde{V}\tilde{W}\tilde{Y}S},Q_{\tilde{U}\tilde{V}\tilde{W}\tilde{Y}S})\\ \in {\mathcal{T}}_1^{\prime }({\kappa }_{\alpha },{\omega }^{\prime },P_S,P_{X|USW})\end{array}}{\min }D(P_{\tilde{U}\tilde{V}\tilde{W}\tilde{Y}|S}\left|\right|Q_{UVWY|S}\left|P_S\right)-O\left(\eta \right)\hfill \\ \hfill & =E_1^{\prime }({\kappa }_{\alpha },{\omega }^{\prime })-O\left(\eta \right).\hfill \end{array}$$

For the inequality in $\left(a\right)$ above, we used ${\sum }_{}{\overline{\mathbb{P}}}_{Q^{{\mathbb{B}}_n}}\left(M^{\prime }=m^{\prime }|\mathbf{U}=\mathbf{u},\mathbf{V}=\mathbf{v},\mathbf{S}=\mathbf{s}\right)\le 1$ and

$$\begin{array}{c}\hfill {\overline{\mathbb{P}}}_{Q^{{\mathbb{B}}_n}}\left(\mathbf{W}\left(m^{\prime }\right)=\overline{\mathbf{w}}|\mathbf{U}=\mathbf{u},\mathbf{V}=\mathbf{v},\mathbf{S}=\mathbf{s},M^{\prime }=m^{\prime }\right)\le \left\{\begin{array}{cc}{e}^{-n\left(H\left(\tilde{W}\right|\hat{S},\tilde{U})-\eta \right)},\hfill & \mathrm{if} \overline{\mathbf{w}}\in {\mathcal{T}}_n\left(\tilde{W}\right),\hfill \\ 0,\hfill & \mathrm{otherwise},\hfill \end{array}\right.\end{array}$$

which in turn follows from the fact that given $M^{\prime }=m^{\prime }$ and $\mathbf{U}=\mathbf{u}$, $\mathbf{W}\left(m^{\prime }\right)$ is uniformly distributed in the set ${\mathcal{T}}_n\left(P_{\tilde{W}|\hat{S}\tilde{U}},\mathbf{s},\mathbf{u}\right)$ and that for sufficiently large n$\left|{\mathcal{T}}_n\left(P_{\tilde{W}|\hat{S}\tilde{U}},\mathbf{s},\mathbf{u}\right)\right|\ge e^{n\left(H\left(\tilde{W}\right|\hat{S},\tilde{U})-\eta \right)}$.

Next, we analyze the probability of the event ${\mathcal{E}}_b^{\prime }$. Let

$$\begin{array}{ccc}\hfill {\mathcal{F}}_{2,n}^{\prime }\left(\eta \right):=\left\{\begin{array}{cc}{P}_{\hat{S}\tilde{U}\tilde{V}\tilde{W}\tilde{Y}{\tilde{W}}_d}:\hfill & P_{\hat{S}\tilde{U}\tilde{W}}\in {\mathcal{D}}_n(P_{SUW},\eta ),P_{\hat{S}\tilde{V}{\tilde{W}}_d\tilde{Y}}\in {\mathcal{D}}_n(P_{SVWY},\eta )\hfill \\ & H\left({\tilde{W}}_d|\hat{S},\tilde{V},\tilde{Y}\right)\le H\left(\tilde{W}|\hat{S},\tilde{V},\tilde{Y}\right)\hfill \end{array}\right\}.& \hfill & \hfill \end{array}$$

Then,

$$\begin{array}{cc}& {\overline{\mathbb{P}}}_{Q^{{\mathbb{B}}_n}}\left({\mathcal{E}}_b^{\prime }\right)\hfill \\ & \le \sum _{\begin{array}{c}{P}_{\hat{S}\tilde{U}\tilde{V}\tilde{W}\tilde{Y}{\tilde{W}}_d}\\ \in {\mathcal{F}}_{2,n}^{\prime }\left(\eta \right)\end{array}}\sum _{\begin{array}{c}(\mathbf{u},\mathbf{v},\overline{\mathbf{w}},\mathbf{y},{\mathbf{w}}^{\prime }):\\ (\mathbf{s},\mathbf{u},\mathbf{v},\overline{\mathbf{w}},\mathbf{y},{\mathbf{w}}^{\prime })\\ \in {\mathcal{T}}_n\left(P_{\hat{S}\tilde{U}\tilde{V}\tilde{W}\tilde{Y}{\tilde{W}}_d}\right)\end{array}}\sum _{\begin{array}{c}{m}^{\prime }\in \\ {\mathcal{M}}^{\prime }\setminus \left\{0\right\}\end{array}}{\overline{\mathbb{P}}}_{Q^{{\mathbb{B}}_n}}\left(\mathbf{U}=\mathbf{u},\mathbf{V}=\mathbf{v},M^{\prime }=m^{\prime },\mathbf{W}\left(m^{\prime }\right)=\overline{\mathbf{w}},\mathbf{Y}=\mathbf{y}|\mathbf{S}=\mathbf{s}\right)\hfill \\ & \sum _{\begin{array}{c}{\hat{m}}^{\prime }\in {\mathcal{M}}^{\prime }\setminus \{0,m^{\prime }\}\end{array}}{\overline{\mathbb{P}}}_{Q^{{\mathbb{B}}_n}}\left(\overline{\mathbf{W}}\left({\hat{m}}^{\prime }\right)={\mathbf{w}}^{\prime }|\mathbf{U}=\mathbf{u},M^{\prime }=m^{\prime },\mathbf{W}\left(m^{\prime }\right)=\overline{\mathbf{w}},\mathbf{S}=\mathbf{s}\right)\hfill \\ & \le \sum _{\begin{array}{c}{P}_{\hat{S}\tilde{U}\tilde{V}\tilde{W}\tilde{Y}{\tilde{W}}_d}\\ \in {\mathcal{F}}_{2,n}^{\prime }\left(\eta \right)\end{array}}\sum _{\begin{array}{c}(\mathbf{u},\mathbf{v},\overline{\mathbf{w}},\mathbf{y}):\\ (\mathbf{s},\mathbf{u},\mathbf{v},\overline{\mathbf{w}},\mathbf{y})\in {\mathcal{T}}_n\left(P_{\hat{S}\tilde{U}\tilde{V}\tilde{W}\tilde{Y}}\right)\end{array}}{e}^{-n\left(H(\tilde{U},\tilde{V})+D\left(P_{\tilde{U}\tilde{V}}\left|\right|Q_{UV}\right)\right)}{e}^{-n\left(H\left(\tilde{W}\right|\hat{S},\tilde{U})-\eta \right)}\hfill \\ & e^{-n\left(H\left(\tilde{Y}\right|\tilde{U},\hat{S},\tilde{W})+D\left(P_{\tilde{Y}|\tilde{U}\hat{S}\tilde{W}}\left|\right|P_{Y|USW}|P_{\tilde{U}\hat{S}\tilde{W}}\right)\right)}{e}^{n\left({\zeta }^{\prime }({\kappa }_{\alpha },{\omega }^{\prime },P_{\hat{S}})+\eta \right)}\frac{2e^{nH\left({\tilde{W}}_d\right|\hat{S},\tilde{V},\tilde{Y})}}{e^{n\left(H\left({\tilde{W}}_d\right)-\eta \right)}}\hfill \\ & \le e^{-n{\mathsf{E}}_{2,n}^{\prime }},\hfill & \end{array}$$

(50)

where

$$\begin{array}{cc}{\mathsf{E}}_{2,n}^{\prime }& \gtrsim \underset{\begin{array}{c}(P_{\tilde{U}\tilde{V}\tilde{W}\tilde{Y}S},Q_{\tilde{U}\tilde{V}\tilde{W}\tilde{Y}S})\in {\mathcal{T}}_2^{\prime }({\kappa }_{\alpha },{\omega }^{\prime },P_S,P_{X|USW})\end{array}}{\min }D(P_{\tilde{U}\tilde{V}\tilde{W}\tilde{Y}|S}\left|\right|Q_{UVWY|S}\left|P_S\right)+{\rho }^{\prime }({\kappa }_{\alpha },{\omega }^{\prime },P_S,P_{X|USW})\hfill \\ & -{\zeta }^{\prime }({\kappa }_{\alpha },{\omega }^{\prime },P_S)-O\left(\eta \right)\hfill \\ & =E_2^{\prime }({\kappa }_{\alpha },{\omega }^{\prime },P_S,P_{X|USW})-O\left(\eta \right).\hfill & \end{array}$$

Finally, considering the event ${\mathcal{E}}_c^{\prime }$, we have

$$\begin{array}{cc}& {\overline{\mathbb{P}}}_{Q^{{\mathbb{B}}_n}}\left({\mathcal{E}}_c^{\prime }\right)=\sum _{\begin{array}{c}\mathbf{u}\in {\mathcal{T}}_n\left(P_{\tilde{U}}\right):\\ P_{\tilde{U}}\in {\mathcal{D}}_n(P_U,\eta )\end{array}}{\overline{\mathbb{P}}}_{Q^{{\mathbb{B}}_n}}\left(\mathbf{U}=\mathbf{u},{\mathcal{E}}_{\mathrm{EE}}^{\prime },{\mathcal{E}}_c^{\prime }|\mathbf{S}=\mathbf{s}\right)+\sum _{\begin{array}{c}\mathbf{u}\in {\mathcal{T}}_n\left(P_{\tilde{U}}\right):\\ P_{\tilde{U}}\notin {\mathcal{D}}_n(P_U,\eta )\end{array}}{\overline{\mathbb{P}}}_{Q^{{\mathbb{B}}_n}}\left(\mathbf{U}=\mathbf{u},{\mathcal{E}}_c^{\prime }|\mathbf{S}=\mathbf{s}\right).\hfill \end{array}$$

The first term in the RHS decays double exponentially as $e^{-e^{n\Omega \left(\eta \right)}}$, while the second term can be handled as follows:

$$\begin{array}{cc}& \sum _{\begin{array}{c}\mathbf{u}\in {\mathcal{T}}_n\left(P_{\tilde{U}}\right):P_{\tilde{U}}\notin {\mathcal{D}}_n(P_U,\eta )\end{array}}{\overline{\mathbb{P}}}_{Q^{{\mathbb{B}}_n}}\left(\mathbf{U}=\mathbf{u},{\mathcal{E}}_c^{\prime }|\mathbf{S}=\mathbf{s}\right)\hfill \\ & \le \sum _{\begin{array}{c}\mathbf{u}\in {\mathcal{T}}_n\left(P_{\tilde{U}}\right):\\ P_{\tilde{U}}\notin {\mathcal{D}}_n(P_U,\eta )\end{array}}\sum _{\begin{array}{c}(\mathbf{v},\mathbf{y},{\mathbf{w}}^{\prime }):\\ (\mathbf{s},\mathbf{v},\mathbf{y},{\mathbf{w}}^{\prime })\in {\mathcal{T}}_n\left(P_{\hat{S}\tilde{V}\tilde{Y}{\tilde{W}}_d}\right),\\ P_{\hat{S}\tilde{V}{\tilde{W}}_d\tilde{Y}}\in {\mathcal{D}}_n(P_{SVWY},\eta )\end{array}}\sum _{\hat{m}\in \mathcal{M}\setminus \left\{0\right\}}{\overline{\mathbb{P}}}_{Q^{{\mathbb{B}}_n}}\left(\mathbf{U}=\mathbf{u},\mathbf{V}=\mathbf{v},M^{\prime }=0,\mathbf{Y}=\mathbf{y}|\mathbf{S}=\mathbf{s}\right)\hfill \\ & \sum _{{\hat{m}}^{\prime }\in {\mathcal{M}}^{\prime }\setminus \left\{0\right\}}{\overline{\mathbb{P}}}_{Q^{{\mathbb{B}}_n}}\left(\mathbf{W}\left({\hat{m}}^{\prime }\right)=\overline{\mathbf{w}}\right)\hfill \\ & \le \sum _{\begin{array}{c}{P}_{\tilde{U}\hat{S}\tilde{V}{\tilde{W}}_d\tilde{Y}}\in {\mathcal{D}}_n{(P_U,\eta )}^c\\ \times {\mathcal{D}}_n(P_{SVWY},\eta )\end{array}}{e}^{nH(\tilde{U},\tilde{V},\tilde{Y}|\hat{S})}{e}^{-n\left(H(\tilde{U},\tilde{V},\tilde{Y}|\hat{S})+D\left(P_{\tilde{U}\tilde{V}\tilde{Y}|\hat{S}}\left|\right|Q_{UV{Y}^{\prime }|\hat{S}}|P_{\hat{S}}\right)\right)}\frac{e^{nH\left({\tilde{W}}_d\right|\hat{S},\tilde{V},\tilde{Y})}{e}^{n(R^{\prime }+\eta )}}{e^{n\left(H\left({\tilde{W}}_d\right)-\eta \right)}}\hfill \\ & \le e^{-n{\mathsf{E}}_{3,n}^{\prime }},\hfill & \end{array}$$

(51)

where

$$\begin{array}{cc}\hfill {\mathsf{E}}_{3,n}^{\prime }& \gtrsim \underset{\begin{array}{c}{P}_{\hat{V}\hat{Y}S}:P_{\hat{U}\hat{V}\hat{W}\hat{Y}S}\in \\ {\hat{\mathcal{L}}}_h({\kappa }_{\alpha },{\omega }^{\prime },P_S,P_{X|USW})\end{array}}{\min }-D\left(P_{\hat{V}\hat{Y}|S}\left|\right|Q_{V{Y}^{\prime }|S}|P_S\right)+{\rho }^{\prime }({\kappa }_{\alpha },{\omega }^{\prime },P_S,P_{X|USW})-{\zeta }^{\prime }({\kappa }_{\alpha },{\omega }^{\prime },P_S)-O\left(\eta \right)\hfill \\ \hfill & =E_3^{\prime }({\kappa }_{\alpha },{\omega }^{\prime },P_S,P_{X|USW},P_{X^{\prime }|US})-O\left(\eta \right).\hfill \end{array}$$

Since the exponent of the type II error probability is lower bounded by the minimum of the exponent of the type II error-causing events, it follows from (49), (50) and (51) that for a fixed $\left(P_S,{\omega }^{\prime }(·,P_S),P_{X|USW},P_{X^{\prime }|US}\right)\in {\mathcal{L}}_h\left({\kappa }_{\alpha }\right)$

$$\begin{array}{cc}\hfill {\overline{\mathbb{P}}}_{P^{\left({\mathbb{B}}_n\right)}}\left(\hat{H}=1\right)& \le e^{-n({\kappa }_{\alpha }-O\left(\eta \right))},\hfill \end{array}$$

(52a)

$$\begin{array}{cc}\hfill {\overline{\mathbb{P}}}_{Q^{\left({\mathbb{B}}_n\right)}}\left(\hat{H}=0\right)& \le e^{-n\left({\overline{\kappa }}_h({\kappa }_{\alpha },{\omega }^{\prime },P_S,P_{X|USW},P_{X^{\prime }|US})-O\left(\eta \right)\right)},\hfill \end{array}$$

(52b)

where ${\overline{\kappa }}_h=\min \left\{E_1^{\prime }({\kappa }_{\alpha },{\omega }^{\prime }),E_2^{\prime }({\kappa }_{\alpha },{\omega }^{\prime },P_S,P_{X|USW}),E_3^{\prime }({\kappa }_{\alpha },{\omega }^{\prime },P_S,P_{X|USW},P_{X^{\prime }|US})\right\}$. Performing expurgation as in the proof of Theorem 1 to obtain a deterministic codebook ${\mathcal{B}}_n$ satisfying (52a, 52b), maximizing over $\left(P_S,{\omega }^{\prime }(·,P_S),P_{X|USW},P_{X^{\prime }|US}\right)\in {\mathcal{L}}_h\left({\kappa }_{\alpha }\right)$ and noting that $\eta >0$ is arbitrary yields $\kappa \left({\kappa }_{\alpha }\right)\ge {\kappa }_{\mathrm{h}}^{★}\left({\kappa }_{\alpha }\right)$.

Finally, we show that $\kappa \left({\kappa }_{\alpha }\right)\ge {\kappa }_{\mathrm{u}}^{★}\left({\kappa }_{\alpha }\right)$, which will complete the proof. Fix $P_{X|US}$ and let $P_{UVXY}:=P_{UV}{P}_{X|US}{P}_{Y|X}$ and $Q_{UVXY}:=Q_{UV}{P}_{X|US}{P}_{Y|X}$. Consider an uncoded transmission scheme in which the channel input $\mathbf{X}\sim f_n(·|\mathbf{u})=P_{X|US}^{\otimes n}(·|\mathbf{u},\mathbf{s})$. Let the decision rule $g_n$ be specified by the acceptance region ${\mathcal{A}}_n=\left\{(\mathbf{s},\mathbf{v},\mathbf{y}):D\left(P_{\mathbf{v}\mathbf{y}|\mathbf{s}}\left|\right|P_{VY|S}|P_{\mathbf{s}}\right)\le {\kappa }_{\alpha }+\eta \right\}$ for some small $\eta >0$. Then, it follows from Lemma 2.6 in [42] that for sufficiently large n,

$$\begin{array}{cc}\hfill {\alpha }_n(f_n,g_n)& =P_{VY|S}^{\otimes n}\left({\mathcal{A}}_n^c|\mathbf{s}\right)\le e^{-n{\kappa }_{\alpha }},\hfill \\ \hfill {\beta }_n(f_n,g_n)& =Q_{VY|S}^{\otimes n}\left({\mathcal{A}}_n|\mathbf{s}\right)\le e^{-n({\kappa }_{\mathrm{u}}^{★}\left({\kappa }_{\alpha }\right)-O\left(\eta \right))}.\hfill \end{array}$$

The proof is complete by noting that $\eta >0$ is arbitrary.

5. Conclusions

This work explored the trade-off between the type I and type II error-exponents for distributed hypothesis testing over a noisy channel. We proposed a separate hypothesis testing and channel coding scheme as well as a joint scheme utilizing hybrid coding, and analyzed their performance resulting in two inner bounds on the error-exponents trade-off. The separate scheme recovers some of the existing bounds in the literature as special cases. We also showed via an example of testing against dependence that the joint scheme strictly outperforms the separate scheme at some points of the error-exponents trade-off. An interesting avenue for future research is the exploration of novel outer bounds that could shed light on the scenarios where the separate or joint schemes are tight.