On the Non-Adaptive Zero-Error Capacity of the Discrete Memoryless Two-Way Channel ^†

Abstract

We study the problem of communicating over a discrete memoryless two-way channel using non-adaptive schemes, under a zero probability of error criterion. We derive single-letter inner and outer bounds for the zero-error capacity region, based on random coding, linear programming, linear codes, and the asymptotic spectrum of graphs. Among others, we provide a single-letter outer bound based on a combination of Shannon’s vanishing-error capacity region and a two-way analogue of the linear programming bound for point-to-point channels, which, in contrast to the one-way case, is generally better than both. Moreover, we establish an outer bound for the zero-error capacity region of a two-way channel via the asymptotic spectrum of graphs, and show that this bound can be achieved in certain cases.

1. Introduction

The problem of reliable communication over a discrete memoryless two-way channel (DM-TWC) was originally introduced and investigated by Shannon [1] in a seminal paper that has marked the inception of multi-user information theory. A DM-TWC is characterized by a quadruple of finite input and output alphabets ${\mathcal{X}}_1$, ${\mathcal{X}}_2$, ${\mathcal{Y}}_1$, ${\mathcal{Y}}_2$, and a conditional probability distribution $P_{Y_1,Y_2|X_1,X_2}(y_1,y_2|x_1,x_2)$, where $x_1\in {\mathcal{X}}_1$, $x_2\in {\mathcal{X}}_2$, $y_1\in {\mathcal{Y}}_1$, $y_2\in {\mathcal{Y}}_2$. The channel is memoryless in the sense that channel uses are independent, that is, for any i,

$$\begin{array}{cc}\hfill & P_{Y_{1i},Y_{2i}|X_1^i,X_2^i,Y_1^{i-1},Y_2^{i-1}}(y_{1i},y_{2i}|x_1^i,x_2^i,y_1^{i-1},y_2^{i-1})=P_{Y_1,Y_2|X_1,X_2}(y_{1i},y_{2i}|x_{1i},x_{2i}).\hfill \end{array}$$

In [1], Shannon provided inner and outer bounds for the vanishing-error capacity region of the DM-TWC, in the general setting where the users are allowed to adapt their transmissions on the fly based on past observations. We note that Shannon’s inner bound is tight for non-adaptive schemes, namely when the users map their messages to codewords in advance. The non-adaptive DM-TWC is also sometimes called the restricted DM-TWC [2]. Shannon’s inner and outer bounds have later been improved by utilizing auxiliary random variable techniques [3,4,5], and sufficient conditions under which his bounds coincide have been obtained [6,7]. However, despite much effort, the capacity region of a general DM-TWC under the vanishing-error criterion remains elusive. In fact, a strong indicator for the inherent difficulty of the problem can be observed in Blackwell’s binary multiplying channel, a simple, deterministic, common-output channel whose capacity remains unknown hitherto [4,5,8,9,10].

In yet another seminal work, Shannon proposed and studied the zero-error capacity of the point-to-point discrete memoryless channel [2], also known as the Shannon capacity of a graph. This problem has been extensively studied by others, most notably in [11,12], yet remains generally unsolved. In this paper, we consider the problem of zero-error communication over a DM-TWC. We limit our discussion to the case of non-adaptive schemes, for which the capacity region is known in the vanishing-error case [1]. Despite the obvious difficulty of the problem (the point-to-point zero-error capacity is a special case), its two-way nature adds a new combinatorial dimension that renders it interesting to study. To the best of our knowledge, this problem has not been addressed before, except in the special case of the binary multiplying channel, where upper and lower bounds on non-adaptive zero-error sum capacity have been obtained [13,14,15]. Our bounds are partially based on generalizations of these ideas and an earlier short version [16].

The problem of non-adaptive communication over a DM-TWC can be formulated as follows. Alice and Bob would like to simultaneously convey messages $m_1\in [2^{n{R}_1}]$ and $m_2\in [2^{n{R}_2}]$ respectively to each other, over n uses of the DM-TWC $P_{Y_1,Y_2|X_1,X_2}$. To that end, Alice maps her message to an input sequence (codeword) $x_1^n\in {\mathcal{X}}_1^n$ using an encoding function $f_1:[2^{n{R}_1}]\to {\mathcal{X}}_1^n$, and Bob maps his message into an input sequence (codeword) $x_2^n\in {\mathcal{X}}_2^n$ using an encoding function $f_2:[2^{n{R}_2}]\to {\mathcal{X}}_2^n$. We call the pair of codeword collections $(f_1([2^{n{R}_1}]),f_2([2^{n{R}_2}]))$ a codebook pair. Note that the encoding functions depend only on the messages, and not on the observed outputs during the transmission, hence the name non-adaptive. When transmissions end, Alice and Bob observe the resulting (random) channel outputs $Y_1^n\in {\mathcal{Y}}_1^n$ and $Y_2^n\in {\mathcal{Y}}_2^n$ respectively, and attempt to decode the message sent by their counterpart, without error. When this is possible, that is, when there exist decoding functions ${\varphi }_1:[2^{n{R}_1}]\times {\mathcal{Y}}_1^n\to [2^{n{R}_2}]$ and ${\varphi }_2:[2^{n{R}_2}]\times {\mathcal{Y}}_2^n\to [2^{n{R}_1}]$ such that $m_2={\varphi }_1(m_1,Y_1^n)$ and $m_1={\varphi }_2(m_2,Y_2^n)$, for all $m_1,m_2$, with probability one, then the codebook pair (or the encoding functions) is called $(n,R_1,R_2)$ uniquely decodable. A rate pair $(R_1,R_2)$ is achievable for the DM-TWC if an $(n,R_1,R_2)$ uniquely decodable code exists for some n. The non-adaptive zero-error capacity region of a DM-TWC $P_{Y_1,Y_2|X_1,X_2}$ is the closure of the set of all achievable rate pairs, and is denoted here by ${\mathcal{C}}_{ze}(P_{Y_1,Y_2|X_1,X_2})$. Moreover, the non-adaptive zero-error sum-capacity of a DM-TWC $P_{Y_1,Y_2|X_1,X_2}$, denoted by ${\mathcal{C}}_{ze}^{sum}(P_{Y_1,Y_2|X_1,X_2})$, is the supremum of the sum-rate $R_1+R_2$ taken over all achievable rate pairs.

The main objective of this paper is to provide several single-letter outer and inner bounds on the non-adaptive zero-error capacity region of the DM-TWC. The remainder of this paper is organized as follows. In Section 2, we provide some necessary mathematical preliminaries, discussing in particular the characterization of zero-error DM-TWC capacity via confusion graphs, behavior under graph homomorphisms, and one-shot zero-error communication. Section 3 is devoted to three general outer bounds of the zero-error capacity region of DM-TWC, which are based on Shannon’s vanishing-error non-adaptive capacity region, a two-way analogue of the linear programming bound for point-to-point channels, and the Shannon capacity of a graph. In Section 4, we provide two general inner bounds using random coding and random linear codes respectively. In Section 5, we establish outer bounds for certain types of DM-TWC via the asymptotic spectra of graphs, and also explicitly construct the uniquely decodable codebook pairs achieving the outer bound. Some concluding remarks appear in Section 6.

2. Preliminaries

2.1. Shannon Capacity of a Graph

Let $G=(V,E)$ be a graph with vertex set V and edge set E. Two vertices $v_1,v_2$ are adjacent, denoted as $v_1\sim v_2$, if there is an edge between $v_1$ and $v_2$, that is, $\{v_1,v_2\}\in E$. An independent set in G is a subset of pairwise non-adjacent vertices. A maximum independent set is an independent set with the largest possible number of vertices. The size of a maximum independent set in G is called the independence number of G, denoted by $\alpha (G)$. The complement of a graph G, denoted by $\overline{G}$, is a graph with the same vertex set, where two distinct vertices of $\overline{G}$ are adjacent if and only if they are not adjacent in G. We write $K_n$ and ${\overline{K}}_n$ for the complete graph (containing all possible edges) and the empty graph (containing no edges) over n vertices, respectively.

Let $G=(V(G),E(G))$ and $H=(V(H),E(H))$ be two graphs. The strong product (or normal product) $G⊠H$ of the graphs G and H is a graph such that

(1)

the vertex set of $G⊠H$ is the Cartesian product $V(G)\times V(H)$;

(2)

two vertices $(u,u^{\prime })$ and $(v,v^{\prime })$ are adjacent if and only if one of the followings holds: (a) $u=v$ and $u^{\prime }\sim v^{\prime }$; (b) $u\sim v$ and $u^{\prime }=v^{\prime }$; (c) $u\sim v$ and $u^{\prime }\sim v^{\prime }$.

The n-fold strong product of graph G with itself is denoted as $G^n$. The Shannon capacity of graph G was defined in [2] to be:

$$\mathsf{\Theta }(G)\triangleq \underset{n}{sup}\frac{1}{n}log\alpha (G^n)=\underset{n\to \infty }{lim}\frac{1}{n}log\alpha (G^n),$$

where the limit exists by Fekete’s lemma. We note that throughout the paper all logarithms are taken to base 2.

The disjoint union $G\bigsqcup H$ of the graphs G and H is a graph such that $V(G\bigsqcup H)=V(G)\bigsqcup V(H)$ and $E(G\bigsqcup H)=E(G)\bigsqcup E(H)$. A graph homomorphism from G to H, denoted by $G\to H$, is a mapping $\phi :V(G)\to V(H)$ such that if $g_1\sim g_2$ in G, then $\phi (g_1)\sim \phi (g_2)$ in H. We write $G\preccurlyeq H$ if there exists a graph homomorphism $\overline{G}\to \overline{H}$ from the complement of G to the complement of H.

In [17], Zuiddam introduced the asymptotic spectrum of graphs notion, and provided a dual characterisation of the Shannon capacity of a graph by applying Strassen’s theory of asymptotic spectra, which includes the Lovász theta number [12], the fractional clique cover number, the complement of the fractional orthogonal rank [18], and the fractional Haemers’ bound over any field [11,19,20] as specific elements of the asymptotic spectrum (also called spectral points).

Theorem 1

([17]). Let $\mathcal{G}$ be a collection of graphs that is closed under the disjoint union ⊔ and the strong product ⊠, and also contains the graph with a single vertex $K_1$. Define the asymptotic spectrum $\Delta (\mathcal{G})$ as the set of all mappings $\eta :\mathcal{G}\to {\mathbb{R}}_{\ge 0}$ such that for all $G,H\in \mathcal{G}$:

(1) 

if $G\preccurlyeq H$, then $\eta (G)\le \eta (H)$;

(2) 

$\eta (G\bigsqcup H)=\eta (G)+\eta (H)$;

(3) 

$\eta (G⊠H)=\eta (G)\eta (H)$;

(4) 

$\eta (K_1)=1$.

Then, $\mathsf{\Theta }(G)=\underset{\eta \in \Delta (\mathcal{G})}{inf}log\eta (G)$. In other words, $\underset{\eta \in \Delta (\mathcal{G})}{inf}\eta (G)=2^{\mathsf{\Theta }(G)}$ and $\alpha (G)\le \underset{\eta \in \Delta (\mathcal{G})}{inf}\eta (G)$.

As remarked in [17], $2^{\mathsf{\Theta }(G)}$ is in general not an element of $\Delta (\mathcal{G})$. In fact, $2^{\mathsf{\Theta }(G)}$ is not additive under ⊔ by a result of Alon [21], and also not multiplicative under ⊠ by a result of Haemers [11]. In Section 3.3, to derive an outer bound for zero-error capacity of a DM-TWC, we will employ the multiplicativity of $\eta (G)$ for $\eta \in \Delta (\mathcal{G})$ under the ⊠ operation.

2.2. Confusion Graphs of Channels

In this subsection, we characterize the zero-error capacity of a discrete memoryless point-to-point channel, as well as the zero-error capacity region of a DM-TWC, in terms of suitably defined graphs. The point-to-point characterization is well known and goes back to Shannon [2], and the DM-TWC case is a natural generalization thereof.

A discrete memoryless point-to-point channel consists of a finite input alphabet $\mathcal{X}$, a finite output alphabet $\mathcal{Y}$, and a conditional probability distribution $P_{Y|X}(y|x)$, where $x\in \mathcal{X}$, $y\in \mathcal{Y}$. The channel is memoryless in the sense that $P_{Y_i|X^i,Y^{i-1}}(y_i|x^i,$ $y^{i-1})=P_{Y|X}(y_i|x_i)$ for the ith channel use. Suppose that a transmitter would like to convey a message $m\in [2^{nR}]$ to a receiver over the channel. To that end, the transmitter sends an input sequence $x^n\in {\mathcal{X}}^n$ using an encoding function $f:[2^{nR}]\to {\mathcal{X}}^n$, and the receiver, after observing the corresponding channel outputs $y^n\in {\mathcal{Y}}^n$, guesses the message using a decoding function $\varphi :{\mathcal{Y}}^n\to [2^{nR}]$. This pair $(f,\varphi )$ is called an $(n,R)$ code, and such a code is called uniquely decodable if $m=\varphi (y^n)$ holds for any $m\in [2^{nR}]$ and any correspondingly possible $y^n$. A rate R is called achievable if an $(n,R)$ uniquely decodable code exists for some n. The zero-error capacity of the channel is defined as the supremum of all achievable rates.

A channel $P_{Y|X}$ is associated with a confusion graph G, whose vertex set is the input alphabet $\mathcal{X}$, and two vertices $x,x^{\prime }\in \mathcal{X}$ are adjacent, denoted as $x\sim x^{\prime }$, if and only if there exists $y\in \mathcal{Y}$ that is possible under both of them, that is, such that $P_{Y|X}(y|x)>0$ and $P_{Y|X}(y|x^{\prime })>0$. It is easy to verify that $\mathcal{C}$ is an $(n,R)$ uniquely decodable code if and only if $\mathcal{C}$ is an independent set of the graph $G^n$, the n-fold strong product of graph G. Consequently, the zero-error capacity of a point-to-point channel is equal to the Shannon capacity of its confusion graph G. Note that there are infinitely many distinct channels with the same confusion graph, and all of these channels have the same zero-error capacity.

We now proceed to similarly associate a DM-TWC with a collection of confusion graphs, which would then be shown to characterize its zero-error capacity region. To that end, note that when Alice sends a letter $x_1\in {\mathcal{X}}_1$, the resulting channel from Bob back to Alice at that same instant is the point-to-point channel $P_{Y_1|X_1=x_1,X_2}$. This channel is associated with a confusion graph $G_{x_1}$, whose vertex set is ${\mathcal{X}}_2$ and where two vertices $x_2,x_2^{\prime }\in {\mathcal{X}}_2$ are adjacent, denoted in this case by $x_2\stackrel{x_1}{\sim }{x}_2^{\prime }$, if and only if there exists some $y_1\in {\mathcal{Y}}_1$ such that both

$$P_{Y_1|X_1,X_2}(y_1|x_1,x_2)>0,P_{Y_1|X_1,X_2}(y_1|x_1,x_2^{\prime })>0,$$

where

$$P_{Y_1|X_1,X_2}(y_1|x_1,x_2)\triangleq \sum _{y_2\in {\mathcal{Y}}_2}{P}_{Y_1,Y_2|X_1,X_2}(y_1,y_2|x_1,x_2).$$

Symmetrically, when Bob sends a letter $x_2\in {\mathcal{X}}_2$, the resulting channel from Alice to Bob at that same instant is associated with a confusion graph $H_{x_2}$, whose vertex set is ${\mathcal{X}}_1$, and where two vertices $x_1,x_1^{\prime }\in {\mathcal{X}}_1$ are adjacent, denoted in this case by $x_1\stackrel{x_2}{\sim }{x}_1^{\prime }$, if and only if there exists some $y_2\in {\mathcal{Y}}_2$ such that both:

$$P_{Y_2|X_1,X_2}(y_2|x_1,x_2)>0,P_{Y_2|X_1,X_2}(y_2|x_1^{\prime },x_2)>0,$$

where

$$P_{Y_2|X_1,X_2}(y_2|x_1,x_2)\triangleq \sum _{y_1\in {\mathcal{Y}}_1}{P}_{Y_1,Y_2|X_1,X_2}(y_1,y_2|x_1,x_2).$$

Based on the foregoing discussion, a DM-TWC $P_{Y_1,Y_2|X_1,X_2}$ can be decomposed into a collection of discrete memoryless point-to-point channels, and hence is associated with a corresponding collection of confusion graphs, denoted by $[G_1,\dots ,G_{|{\mathcal{X}}_1|};H_1,$ $\dots ,H_{|{\mathcal{X}}_2|}]$, where $V(G_1)=\cdots =V(G_{|{\mathcal{X}}_1|})={\mathcal{X}}_2$ and $V(H_1)=\cdots =V(H_{|{\mathcal{X}}_2|})={\mathcal{X}}_1$. The following useful observation is immediate, and in particular shows that the zero-error capacity region of a DM-TWC is a function of its confusion graphs only. Thus, from here and on, we will sometimes identify the channel with its collection of confusion graphs.

Proposition 1.

Consider a DM-TWC $P_{Y_1,Y_2|X_1,X_2}$, associated with the collection of confusion graphs $[G_1,\dots ,G_{|{\mathcal{X}}_1|};$ $H_1,\dots ,$ $H_{|{\mathcal{X}}_2|}]$. A codebook pair $(\mathcal{A},\mathcal{B})$ is uniquely decodable for $P_{Y_1,Y_2|X_1,X_2}$ if and only if for any $a^n=(a_1,\dots ,a_n)\in \mathcal{A}$ and $b^n=(b_1,\dots ,b_n)\in \mathcal{B}$, it holds that $\mathcal{B}$ is an independent set of $G_{a_1}⊠\cdots ⊠G_{a_n}$, and $\mathcal{A}$ is an independent set of $H_{b_1}⊠\cdots ⊠H_{b_n}$.

In particular, we see that the capacity region ${\mathcal{C}}_{ze}(P_{Y_1,Y_2|X_1,X_2})$ depends only on the corresponding confusion graphs $[G_1,\dots ,G_{|{\mathcal{X}}_1|};H_1,$ $\dots ,H_{|{\mathcal{X}}_2|}]$. Hence, in the sequel, we will write ${\mathcal{C}}_{ze}([G_1,\dots ,G_{|{\mathcal{X}}_1|};H_1,\dots ,H_{|{\mathcal{X}}_2|}])$ and ${\mathcal{C}}_{ze}^{sum}([G_1,\dots ,G_{|{\mathcal{X}}_1|};H_1,$ $\dots ,H_{|{\mathcal{X}}_2|}])$ to represent ${\mathcal{C}}_{ze}(P_{Y_1,Y_2|X_1,X_2})$ and ${\mathcal{C}}_{ze}^{sum}(P_{Y_1,Y_2|X_1,X_2})$, respectively. We will also often identify the channel with its confusion graphs, and refer to it as $[\{G_i\};\{H_j\}]$, when it is clear from the context. This also leads to the following immediate observation, analogues to the point-to-point case.

Proposition 2.

If $P_{Y_1,Y_2|X_1,X_2}$ and $Q_{Y_1,Y_2|X_1,X_2}$ have the same confusion graphs up to some relabeling on input symbols, then ${\mathcal{C}}_{ze}(P_{Y_1,Y_2|X_1,X_2})={\mathcal{C}}_{ze}(Q_{Y_1,Y_2|X_1,X_2})$.

This further immediately implies:

Proposition 3.

${\mathcal{C}}_{ze}(P_{Y_1,Y_2|X_1,X_2})$ depends only on the conditional marginal distributions $P_{Y_1|X_1,X_2}$ and $P_{Y_2|X_1,X_2}$.

The strong product of two DM-TWCs $[G_1,\dots ,G_{|{\mathcal{X}}_1|};H_1,\dots ,H_{|{\mathcal{X}}_2|}]$ and $[G_1^{\prime },\dots ,$ $G_{|{\mathcal{X}}_1^{\prime }|}^{\prime };$ $H_1^{\prime },\dots ,H_{|{\mathcal{X}}_2^{\prime }|}^{\prime }]$, denoted by $[\{G_i\};\{H_j\}]⊠[\{G_i^{\prime }\};\{H_j^{\prime }\}]$, refers to a DM-TWC having input alphabets ${\mathcal{X}}_1\times {\mathcal{X}}_1^{\prime }$ and ${\mathcal{X}}_2\times {\mathcal{X}}_2^{\prime }$, as well as confusion graphs

$$[\{G_i⊠G_{i^{\prime }}^{\prime }:i\in {\mathcal{X}}_1,i^{\prime }\in {\mathcal{X}}_1^{\prime }\};\{H_j⊠H_{j^{\prime }}^{\prime }:j\in {\mathcal{X}}_2,j^{\prime }\in {\mathcal{X}}_2^{\prime }\}].$$

Considering the zero-error sum-capacity with respect to the strong product, we have the lemma below.

Lemma 1.

$${\mathcal{C}}_{ze}^{sum}\left([\{G_i\};\{H_j\}]⊠[\{G_i^{\prime }\};\{H_j^{\prime }\}]\right)\ge {\mathcal{C}}_{ze}^{sum}\left([\{G_i\};\{H_j\}]\right)+{\mathcal{C}}_{ze}^{sum}\left([\{G_i^{\prime }\};\{H_j^{\prime }\}]\right).$$

Proof. 

To prove this lemma, it is sufficient to prove that, for any $(n,R_1,R_2)$ (resp. $(n,R_1^{\prime },$ $R_2^{\prime })$) uniquely decodable codebook pair $(\mathcal{A},\mathcal{B})$ (resp. $({\mathcal{A}}^{\prime },{\mathcal{B}}^{\prime })$) for channel $[\{G_i\};\{H_j\}]$ (resp. $[\{G_i^{\prime }\};\{H_j^{\prime }\}]$), there exists an $(n,R_1+R_1^{\prime },R_2+R_2^{\prime })$ uniquely decodable codebook pair for the associated product channel $[\{G_i\};\{H_j\}]⊠[\{G_i^{\prime }\};\{H_j^{\prime }\}]$. To that end, let

$$\begin{array}{cc}\hfill {\mathcal{A}}^{*}& =\{((a_1,a_1^{\prime }),\dots ,(a_n,a_n^{\prime })):a^n\in \mathcal{A},a^{\prime n}\in {\mathcal{A}}^{\prime }\},\hfill \\ \hfill {\mathcal{B}}^{*}& =\{((b_1,b_1^{\prime }),\dots ,(b_n,b_n^{\prime })):b^n\in \mathcal{B},b^{\prime n}\in {\mathcal{B}}^{\prime }\}.\hfill \end{array}$$

It is easy to verify that $({\mathcal{A}}^{*},{\mathcal{B}}^{*})$ is uniquely decodable for the product channel. Moreover, $|{\mathcal{A}}^{*}|=|\mathcal{A}||{\mathcal{A}}^{\prime }|=2^{n(R_1+R_1^{\prime })}$ and $|{\mathcal{B}}^{*}|=|\mathcal{B}||{\mathcal{B}}^{\prime }|=2^{n(R_2+R_2^{\prime })}$. The lemma follows. □

2.3. Dual Graph Homomorphisms

In this subsection we study the behavior of the zero-error capacity region of a DM-TWC under graph homomorphisms, generalizing a similar analysis from the point-to-point channel case [2]. Let $[\{G_i\};\{H_j\}]$ and $[\{G_i^{\prime }\};\{H_j^{\prime }\}]$ be two collections of confusion graphs corresponding to two DM-TWCs such that $V(G_i)=V(G)$, $V(H_j)=V(H)$, $V(G_i^{\prime })=V(G^{\prime })$ and $V(H_j^{\prime })=V(H^{\prime })$. A dual graph homomorphism from $[\{G_i\};\{H_j\}]$ to $[\{G_i^{\prime }\};\{H_j^{\prime }\}]$, denoted by $[\{G_i\};\{H_j\}]\to [\{G_i^{\prime }\};\{H_j^{\prime }\}]$, is a pair of mappings $(\phi ,\psi )$, where $\phi :V(H)\to V(H^{\prime })$ and $\psi :V(G)\to V(G^{\prime })$, such that

(1)

if $v_1\sim v_2$ in $G_i$, then $\psi (v_1)\sim \psi (v_2)$ in $G_{\phi (i)}^{\prime }$; and

(2)

if $u_1\sim u_2$ in $H_j$, then $\phi (u_1)\sim \phi (u_2)$ in $H_{\psi (j)}^{\prime }$.

It is easy to see that the dual graph homomorphism is a natural generalization of the standard graph homomorphism of two graphs, in the sense that they are both adjacency preserving. We write $[\{G_i\};\{H_j\}]⪯[\{G_i^{\prime }\};\{H_j^{\prime }\}]$ if there exists a dual graph homomorphism from $[\{{\overline{G}}_i\};\{{\overline{H}}_j\}]$ to $[\{{\overline{G}}_i^{\prime }\};\{{\overline{H}}_j^{\prime }\}]$. Then:

Lemma 2.

If $[\{G_i\};\{H_j\}]⪯[\{G_i^{\prime }\};\{H_j^{\prime }\}]$, and ${\overline{G}}_i$ and ${\overline{H}}_j$ do not have self-loops, then

$${\mathcal{C}}_{ze}([\{G_i\};\{H_j\}])\subseteq {\mathcal{C}}_{ze}([\{G_i^{\prime }\};\{H_j^{\prime }\}]).$$

Proof. 

Suppose $(\phi ,\psi ):[\{{\overline{G}}_i\};\{{\overline{H}}_j\}]\to [\{{\overline{G}}_i^{\prime }\};\{{\overline{H}}_j^{\prime }\}]$ and $(\mathcal{A},\mathcal{B})$ is a uniquely decodable codebook pair of length n for the DM-TWC $[\{G_i\};\{H_j\}]$. Let

$$\begin{array}{cc}\hfill \mathsf{\Phi }(\mathcal{A})& =\{\phi (a^n)=(\phi (a_1),\dots ,\phi (a_n)):a^n\in \mathcal{A}\},\hfill \\ \hfill \mathsf{\Psi }(\mathcal{B})& =\{\psi (b^n)=(\psi (b_1),\dots ,\psi (b_n)):b^n\in \mathcal{B}\}.\hfill \end{array}$$

We now show that $(\mathsf{\Phi }(\mathcal{A}),\mathsf{\Psi }(\mathcal{B}))$ is a uniquely decodable codebook pair for the DM-TWC $[\{G_i^{\prime }\};\{H_j^{\prime }\}]$. To that end, it suffices to show that for any distinct $a^n,{\tilde{a}}^n\in \mathcal{A}$ and $b^n,{\tilde{b}}^n\in \mathcal{B}$, we have

$$\begin{array}{cc}\hfill \phi (a^n)& \nsim \phi ({\tilde{a}}^n)\mathrm{in}{H}_{\psi (b_1)}⊠\cdots ⊠H_{\psi (b_n)},\hfill \\ \hfill \psi (b^n)& \nsim \psi ({\tilde{b}}^n)\mathrm{in}{G}_{\phi (a_1)}⊠\cdots ⊠G_{\phi (a_n)}.\hfill \end{array}$$

(1)

Indeed, since $(\mathcal{A},\mathcal{B})$ is a uniquely decodable codebook pair, there exist coordinates $i,j\in [n]$ such that $a_i\nsim {\tilde{a}}_i$ in $H_{b_i}$ and $b_j\nsim {\tilde{b}}_j$ in $G_{a_j}$. By the definition of $(\phi ,\psi )$, we have that $\phi (a_i)\nsim \phi ({\tilde{a}}_i)$ in $H_{\psi (b_i)}$ and $\psi (b_j)\nsim \psi ({\tilde{b}}_j)$ in $G_{\phi (a_j)}$, implying (1). It is also evident that $|\mathsf{\Phi }(\mathcal{A})|=|\mathcal{A}|$ and $|\mathsf{\Psi }(\mathcal{B})|=|\mathcal{B}|$. The lemma now follows by taking the union over all uniquely decodable codebook pairs $(\mathcal{A},\mathcal{B})$ for $[\{G_i\};\{H_j\}]$. □

2.4. One-Shot Zero-Error Communication

In this subsection, we consider the problem of zero-error communication over a DM-TWC with only a single channel use by the two parties (i.e., $n=1$). We refer to the associated set of achievable rate pairs as the one-shot zero-error capacity region, and the associated sum-rate as the one-shot zero-error sum-capacity. Recall that the one-shot zero-error capacity of a point-to-point channel is simply the logarithm of the independence number of its confusion graph; this quantity yields a lower bound on the zero-error capacity of the channel, and also provides an infinite-letter expression for the capacity when evaluated over the product graph. It is therefore interesting to study the analogue of the independence number in the two-way case, which in particular would yield an inner bound on the zero-error capacity region of the DM-TWC. For simplicity of exposition, we will focus here on the one-shot zero-error sum-capacity only.

For convenience we define some notions first. Let $[\{G_i\};\{H_j\}]$ be a DM-TWC such that $V(G_i)={\mathcal{X}}_2$ and $V(H_j)={\mathcal{X}}_1$. A pair $(S,T)$ of subsets $S\subseteq {\mathcal{X}}_1$ and $T\subseteq {\mathcal{X}}_2$ is called a dual clique pair of the DM-TWC if $t\stackrel{s}{\sim }{t}^{\prime }$ and $s\stackrel{t}{\sim }{s}^{\prime }$ for any distinct $s,s^{\prime }\in S$ and distinct $t,t^{\prime }\in T$, that is, S is a clique in each $H_t$ for $t\in T$, and T is a clique in each $G_s$ for $s\in S$. A pair $(S,T)$ of subsets $S\subseteq {\mathcal{X}}_1$ and $T\subseteq {\mathcal{X}}_2$ is called a dual independent pair of the DM-TWC if T is an independent set of the graph $G_s$ for each $s\in S$, and S is an independent set of the graph $H_t$ for each $t\in T$. A maximum dual independent pair is a dual independent pair $(S,T)$ with the largest possible product of sizes $|S||T|$. This product is called the independence product of $[\{G_i\};\{H_j\}]$, denoted by $\pi (\{G_i\};\{H_j\})$. According to the definition, the one-shot zero-error sum-capacity of the DM-TWC is $log\pi (\{G_i\};\{H_j\})$. It is also readily seen that if two channels have the same confusion graphs up to some relabeling of input symbols, then they have the same collections of dual clique pairs and dual independent pairs, and hence the same one-shot zero-error sum-capacity.

For two graphs $G_1$ and $G_2$, let $G_1\cup G_2$ be the union of $G_1$ and $G_2$ such that $V(G_1\cup G_2)=V(G_1)\cup V(G_2)$ and $E(G_1\cup G_2)=E(G_1)\cup E(G_2)$. Notice that the graph disjoint union ⊔ in Section 2.1 is a special case of the union ∪, when the vertex sets of $G_1$ and $G_2$ are disjoint. For notation convenience, in the rest of this subsection we let $|{\mathcal{X}}_1|=m_1$ and $|{\mathcal{X}}_2|=m_2$. The following simple observations are now in order.

Proposition 4.

Suppose $(S,T)$ is a dual independent pair of $[G_1,\dots ,G_{m_1};H_1,\dots ,H_{m_2}]$. Then:

(1) 

If $|S|=1$, then $|T|\le \underset{1\le i\le m_1}{max}\alpha (G_i)$. The equality holds by taking $S=\{s\}$ and T be a maximum independent set of $G_s$, where $s\in arg{max}_{1\le i\le m_1}\alpha (G_i)$.

(2) 

$|S|\le \underset{t\in T}{min}\alpha (H_t)$.

(3) 

S is an independent set of ${\bigcup }_{t\in T}{H}_t$.

Proof. 

The results follow directly from the definition of dual independent pairs. □

Lemma 3.

Let $[G_1,\dots ,G_{m_1};H_1,\dots ,H_{m_2}]$ be a DM-TWC and G, H be graphs such that $V(G)={\mathcal{X}}_2$, $V(H)={\mathcal{X}}_1$. Then:

(1) 

$max\left\{\underset{1\le i\le m_1}{max}\alpha (G_i),\underset{1\le j\le m_2}{max}\alpha (H_j)\right\}\le \pi (G_1,\dots ,G_{m_1};H_1,\dots ,H_{m_2})\le \underset{1\le i\le m_1}{max}\alpha (G_i)·\underset{1\le j\le m_2}{max}\alpha (H_j).$

(2) 

$\pi (G,\dots ,G;H,\dots ,H)=\alpha (G)\alpha (H).$

(3) 

$\pi ({\overline{K}}_{m_2},G,\dots ,G;{\overline{K}}_{m_1},H,\dots ,H)=max\{\alpha (G)\alpha (H),m_1,m_2\}.$

(4) 

$\pi (G_1,\dots ,G_{m_1};K_{m_1},\dots ,K_{m_1})=\underset{1\le i\le m_1}{max}\alpha (G_i)$.

Proof. 

(1) The lower bound follows from Proposition 4 (1) and the symmetry of S and T. From Proposition 4 (2), we have

$$\begin{array}{cc}\hfill |S|& \le \underset{t\in T}{min}\alpha (H_t)\le \underset{1\le j\le m_2}{max}\alpha (H_j),\hfill \\ \hfill |T|& \le \underset{s\in S}{min}\alpha (G_s)\le \underset{1\le i\le m_1}{max}\alpha (G_i),\hfill \end{array}$$

yielding the upper bound.

(2) From claim (1) above, we have $\pi (G,\dots ,G;H,\dots ,H)\le \alpha (G)\alpha (H)$. The equality holds by taking S and T as the maximum independent sets of H and G respectively.

(3) From claims (1) and (2) above, we have

$$\pi ({\overline{K}}_{m_2},G,\dots ,G;{\overline{K}}_{m_1},H,\dots ,H)\ge max\{\alpha (G)\alpha (H),m_1,m_2\}.$$

On the other hand, suppose $(S,T)$ is a dual independent pair. We have the following three cases: (i) If $|S|=1$ then by Proposition 4, claim (1), we have $|S||T|\le m_2$. (ii) If $|T|=1$, similar to case (i), we have $|S||T|\le m_1$. (iii) If $|S|\ge 2$ and $|T|\ge 2$, then by Proposition 4, claim (2), we obtain $|S||T|\le \alpha (G)\alpha (H)$. Thus $\pi ({\overline{K}}_{m_2},G,\dots ,G;{\overline{K}}_{m_1},H,\dots ,H)\le max\{\alpha (G)\alpha (H),m_1,m_2\}$.

(4) is a direct consequence of claim (1) above. The lemma follows. □

By graph homomorphisms we immediately have:

Proposition 5.

If $[\{G_i\};\{H_j\}]⪯[\{G_i^{\prime }\};\{H_j^{\prime }\}]$, then

$$\pi (\{G_i\};\{H_j\})\le \pi (\{G_i^{\prime }\};\{H_j^{\prime }\}).$$

Next, we shall provide an upper bound for $\pi (\{G_i\};\{H_j\})$ via a generalization of the Lovász theta number [12]. Let $\mathsf{\Gamma }$ be an arbitrary $(m_1+m_2)\times (m_1+m_2)$ positive semi-definite matrix (i.e., $\mathsf{\Gamma }⪰0$), and ${\mathsf{\Gamma }}_{i,j}$ be its $(i,j)$th entry. Let J be an $m_1\times m_2$ all-one matrix, and $I_n$ be an $n\times n$ identity matrix. For any matrices A and B, denote $⟨A,B⟩=\mathrm{trace}(AB)$ and denote $A^T$ as the transpose of matrix A. Now define $\rho (\{G_i\},\{H_j\})$ as

$$\begin{array}{cc}\hfill \mathrm{maximize}& ⟨\left(\begin{array}{cc}0& J\\ J^T& 0\end{array}\right),\mathsf{\Gamma }⟩\hfill \\ \hfill \mathrm{subject}\mathrm{to}& ⟨\left(\begin{array}{cc}{I}_{m_1}& 0\\ 0& 0\end{array}\right),\mathsf{\Gamma }⟩=1,\hfill \\ \hfill & ⟨\left(\begin{array}{cc}0& 0\\ 0& I_{m_2}\end{array}\right),\mathsf{\Gamma }⟩=1,\hfill \\ \hfill & {\mathsf{\Gamma }}_{i,j+m_1}·{\mathsf{\Gamma }}_{i,k+m_1}=0,\forall i\in {\mathcal{X}}_1,j,k\in {\mathcal{X}}_2,j\ne k,j\sim k\mathrm{in}{G}_i\hfill \\ \hfill & {\mathsf{\Gamma }}_{i+m_1,j}·{\mathsf{\Gamma }}_{i+m_1,k}=0,\forall i\in {\mathcal{X}}_2,j,k\in {\mathcal{X}}_1,j\ne k,j\sim k\mathrm{in}{H}_i\hfill \\ \hfill & {\mathsf{\Gamma }}_{i,k+m_1}·{\mathsf{\Gamma }}_{j,k+m_1}=0,\forall i,j\in {\mathcal{X}}_1,k\in {\mathcal{X}}_2,i\ne j,i\sim j\mathrm{in}{H}_k\hfill \\ \hfill & {\mathsf{\Gamma }}_{i+m_1,k}·{\mathsf{\Gamma }}_{j+m_1,k}=0,\forall i,j\in {\mathcal{X}}_2,k\in {\mathcal{X}}_1,i\ne j,i\sim j\mathrm{in}{G}_k\hfill \\ \hfill & \mathsf{\Gamma }⪰0.\hfill \end{array}$$

(2)

Lemma 4.

$\pi (\{G_i\},\{H_j\})\le {\left(\frac{1}{2}\rho (\{G_i\},\{H_j\})\right)}^2$.

Proof. 

Suppose $(S,T)$ with $S\subseteq {\mathcal{X}}_1$, $T\subseteq {\mathcal{X}}_2$ is a maximum dual independent pair such that $|S||T|=\pi (\{G_i\},\{H_j\})$. For a number m and a set S, denote $m+S=\{m+s:s\in S\}$. Let $\mathsf{\Gamma }$ be an $(m_1+m_2)\times (m_1+m_2)$ matrix such that

$${\mathsf{\Gamma }}_{i,j}=\left\{\begin{array}{cc}\frac{1}{|S|},\hfill & \mathrm{if}i\in S,j\in S\hfill \\ \frac{1}{\sqrt{|S||T|}},\hfill & \mathrm{if}i\in S,j\in m_1+T,\mathrm{or}i\in m_1+T,j\in S\hfill \\ \frac{1}{|T|},\hfill & \mathrm{if}i\in m_1+T,j\in m_1+T\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

Notice that for any vector $x^{m_1+m_2}=(x_1,\dots ,x_{m_1+m_2})$ we have

$$x^{m_1+m_2}·\mathsf{\Gamma }·{(x^{m_1+m_2})}^T={\left(\frac{1}{\sqrt{|S|}}\sum _{i\in S}{x}_i+\frac{1}{\sqrt{|T|}}\sum _{j\in T}{x}_{m_1+j}\right)}^2\ge 0.$$

This shows that $\mathsf{\Gamma }$ is a positive semi-definite matrix satisfying the equality constraints in (2). Accordingly, $\mathsf{\Gamma }$ is a feasible solution for program (2) and

$$\begin{array}{cc}\hfill \rho (\{G_i\},\{H_j\})& \ge ⟨\left(\begin{array}{cc}0& J\\ J^T& 0\end{array}\right),\mathsf{\Gamma }⟩\hfill \\ \hfill & =2\sqrt{|S||T|}\hfill \\ \hfill & =2\sqrt{\pi (\{G_i\},\{H_j\})},\hfill \end{array}$$

implying the result. This completes the proof. □

2.5. Information-Theoretic Notations

We recall some standard information-theoretic quantities that will be used in the sequel. Let $X,Y$ be two discrete random variables taking values from sets $\mathcal{X},\mathcal{Y}$ according to a joint probability distribution $P_{XY}$. Let $P_X$ denote the marginal probability distribution for X, where $P_X(x)={\sum }_{y\in \mathcal{Y}}{P}_{XY}(x,y)$, and $P_Y$ be the marginal probability distribution for Y similarly. The Shannon entropy of X is denoted by $H(X)$ where $H(X)=-{\sum }_{x\in \mathcal{X}}{P}_X(x)log{P}_X(x)$. In particular, the binary entropy function is written as $h(x)=-xlogx-(1-x)log(1-x)$, where $0\le x\le 1$. The conditional entropy of X given Y is written as $H(X|Y)$ where $H(X|Y)=-P_{XY}(x,y)log\frac{P_{XY}(x,y)}{P_Y(y)}$. The mutual information between X and Y is $I(X;Y)=H(X)-H(X|Y)$. The conditional mutual information of $X,Y$ given another random variable Z is $I(X;Y|Z)=H(X|Z)-H(X|Y,Z)$. The following basic properties will be used in the arguments afterwards.

Proposition 6.

(1) $H(X)\ge 0,I(X;Y)\ge 0$. (Non-negativity)

(2) $H(X|Y)\le H(X)$, $I(X;Y|Z)\le I(X;Y)$. (Conditioning reduces entropy)

(3) $H(X_1,X_2,\dots ,X_n)={\sum }_{i=1}^{n}H(X_i|X_1,\dots ,X_{i-1})$. (Entropy chain rule)

3. Outer Bounds

In this section, we provide single-letter outer bounds for the non-adaptive zero-error capacity region of the DM-TWC. First in Section 3.1, we present two simple outer bounds, one based on Shannon’s vanishing-error non-adaptive capacity region and the other on a two-way analogue of the linear programming bound for point-to-point channels. Next in Section 3.2, we combine the two bounds given in Section 3.1 and obtain an outer bound that is generally better than both. Finally, in Section 3.3 we derive another single-letter outer bound via the asymptotic spectra of graphs.

3.1. Simple Bounds

It is trivial to see that Shannon’s vanishing-error non-adaptive capacity region of the DM-TWC ([1], Theorem 3) contains its zero-error counterpart. First recall Shannon’s bound in [1].

Lemma 5

([1]). The vanishing-error non-adaptive capacity region of a DM-TWC $P_{Y_1,Y_2|X_1,X_2}$ is the convex hull of the set:

$$\bigcup _{P_{X_1},P_{X_2}}\{(R_1,R_2):R_1\ge 0,R_2\ge 0,R_1=I(X_1;Y_2|X_2),R_2=I(X_2;Y_1|X_1)\}$$

where the union is taken over all product input probability distributions $P_{X_1}\times P_{X_2}$.

Together with Proposition 2, this immediately yields the following outer bound.

Lemma 6.

${\mathcal{C}}_{ze}(P_{Y_1,Y_2|X_1,X_2})$ is contained in

$$\begin{array}{cc}\hfill \bigcap _{Q_{Y_1,Y_2|X_1,X_2}}\bigcap _{0\le \lambda \le 1}\{(R_1,& R_2):R_1\ge 0,R_2\ge 0,\lambda R_1+(1-\lambda )R_2\le \underset{P_{X_1},P_{X_2}}{max}ϵ(\lambda )\},\hfill \end{array}$$

(3)

where

$$ϵ(\lambda )\triangleq \lambda I(X_1;Y_2|X_2)+(1-\lambda )I(X_2;Y_1|X_1).$$

(4)

The first intersection is taken over all DM-TWCs $Q_{Y_1,Y_2|X_1,X_2}$ with the same adjacency as $P_{Y_1,Y_2|X_1,X_2}$, and the maximum is taken over all product input probability distributions $P_{X_1}\times P_{X_2}$.

Remark 1.

The bound (3) can also be written in the standard form

$$\begin{array}{cc}\hfill \bigcap _{Q_{Y_1,Y_2|X_1,X_2}}\bigcup _{P_{X_1},P_{X_2}}\{(R_1,R_2):& R_1\ge 0,R_2\ge 0,\hfill \\ \hfill & R_1\le I(X_1;Y_2|X_2),\hfill \\ \hfill & R_2\le I(X_2;Y_1|X_1)\}.\hfill \end{array}$$

Here we prefer however to use the form (3), for ease of comparison with forthcoming bounds.

We now proceed to obtain a combinatorial outer bound. Recall that a dual clique pair of a DM-TWC is a pair $(S,T)$ of subsets $S\subseteq {\mathcal{X}}_1$ and $T\subseteq {\mathcal{X}}_2$ such that $t\stackrel{s}{\sim }{t}^{\prime }$ and $s\stackrel{t}{\sim }{s}^{\prime }$ for any distinct $s,s^{\prime }\in S$ and distinct $t,t^{\prime }\in T$. In the sequel, we adopt the convention that $0^0=1$.

Lemma 7.

${\mathcal{C}}_{ze}(P_{Y_1,Y_2|X_1,X_2})$ is contained in:

$$\bigcap _{0\le \lambda \le 1}\{(R_1,R_2):R_1\ge 0,R_2\ge 0,\lambda R_1+(1-\lambda )R_2\le \underset{P_{X_1},P_{X_2}}{max}-logl(\lambda )\},$$

(5)

where

$$l(\lambda )\triangleq \underset{S,T}{max}{\left(\sum _{x_1\in S}{P}_{X_1}(x_1)\right)}^{\lambda }{\left(\sum _{x_2\in T}{P}_{X_2}(x_2)\right)}^{1-\lambda }$$

(6)

and the maximum in (5) is taken over all the input probability distributions $P_{X_1}$ and $P_{X_2}$, and the maximum in (6) is taken over all the dual clique pairs $(S,T)$ of $P_{Y_1,Y_2|X_1,X_2}$.

Proof. 

Let $(\mathcal{A},\mathcal{B})$ be a uniquely decodable codebook pair of length n. We will show that:

$${|\mathcal{A}|}^{\lambda }{|\mathcal{B}|}^{1-\lambda }\le \kappa ·{\left(\frac{1}{l(\lambda )}\right)}^n$$

(7)

by induction on n, where $\kappa$ is a constant independent of n.

Indeed, for the base case $n=1$, one could take subsets $\mathcal{A}\subseteq {\mathcal{X}}_1$, $\mathcal{B}\subseteq {\mathcal{X}}_2$ such that for any distinct $a,a^{\prime }\in \mathcal{A}$ and distinct $b,b^{\prime }\in \mathcal{B}$, we have $a\stackrel{b}{\nsim }{a}^{\prime }$ and $b\stackrel{a}{\nsim }{b}^{\prime }$. Clearly, $|\mathcal{A}||\mathcal{B}|\le |{\mathcal{X}}_1||{\mathcal{X}}_2|$ and (7) follows by taking $\kappa$ sufficiently large.

Assume that (7) holds for every length $n^{\prime }\le n-1$, and let us proceed to prove for length n. Suppose $(\mathcal{A},\mathcal{B})\subseteq {\mathcal{X}}_1^n\times {\mathcal{X}}_2^n$ is a uniquely decodable codebook pair of length n. For a vector $x^n$, let $x^{n\setminus i}\triangleq (x_1,\dots ,x_{i-1},x_{i+1},\dots ,x_n)$ be its projection over all coordinates not equal to i. For each coordinate $1\le i\le n$ and each $x_1\in {\mathcal{X}}_1$, $x_2\in {\mathcal{X}}_2$, let

$$\begin{array}{cc}\hfill {\mathcal{A}}_i(x_1)& \triangleq \{a^{n\setminus i}:a^n\in \mathcal{A},a_i=x_1\},\hfill \\ \hfill {\mathcal{B}}_i(x_2)& \triangleq \{b^{n\setminus i}:b^n\in \mathcal{B},b_i=x_2\}\hfill \end{array}$$

(8)

be the projections of each codebook obtained by fixing the ith coordinate. Define the distributions induced by these projections over ${\mathcal{X}}_1$ and ${\mathcal{X}}_2$ respectively to be

$$P_{X_1}^i(x_1)\triangleq \frac{|{\mathcal{A}}_i(x_1)|}{|\mathcal{A}|},P_{X_2}^i(x_2)\triangleq \frac{|{\mathcal{B}}_i(x_2)|}{|\mathcal{B}|}.$$

(9)

Furthermore, for any two subsets $S\subseteq {\mathcal{X}}_1$ and $T\subseteq {\mathcal{X}}_2$, define the codebooks induced by the unions over S and T of the respective projected codebooks, to be

$${\mathcal{A}}_i(S)\triangleq \bigcup _{x_1\in S}{\mathcal{A}}_i(x_1),{\mathcal{B}}_i(T)\triangleq \bigcup _{x_2\in T}{\mathcal{B}}_i(x_2).$$

(10)

Note that if $(S,T)$ is a dual clique pair such that ${\mathcal{A}}_i(S)\ne \varnothing$ and ${\mathcal{B}}_i(T)\ne \varnothing$, then the unions in (10) are disjoint, as otherwise this would contradict the assumption that $(\mathcal{A},\mathcal{B})$ is uniquely decodable. Hence

$$\begin{array}{c}\hfill |{\mathcal{A}}_i(S)|=\sum _{x_1\in S}|{\mathcal{A}}_i(x_1)|,|{\mathcal{B}}_i(T)|=\sum _{x_2\in T}|{\mathcal{B}}_i(x_2)|,\end{array}$$

(11)

and also, for any $i\in [n]$ it must hold that $({\mathcal{A}}_i(S),{\mathcal{B}}_i(T))$ is a uniquely decodable codebook pair of length $n-1$. Combining (8), (9) and (11) gives

$$\begin{array}{cc}\hfill {|\mathcal{A}|}^{\lambda }{|\mathcal{B}|}^{1-\lambda }& ={\left(\frac{|{\mathcal{A}}_i(S)|}{{\sum }_{s\in S}{P}_{X_1}^i(s)}\right)}^{\lambda }·{\left(\frac{|{\mathcal{B}}_i(T)|}{{\sum }_{t\in T}{P}_{X_2}^i(t)})\right)}^{1-\lambda }.\hfill \end{array}$$

(12)

By the inductive hypothesis, we obtain

$$\begin{array}{cc}\hfill {|\mathcal{A}|}^{\lambda }{|\mathcal{B}|}^{1-\lambda }& \le \frac{\kappa ·{\left(\frac{1}{l(\lambda )}\right)}^{n-1}}{{\left({\sum }_{s\in S}{P}_{X_1}^i(s)\right)}^{\lambda }·{\left({\sum }_{t\in T}{P}_{X_2}^i(t)\right)}^{1-\lambda }}\hfill \\ \hfill & \le \frac{\kappa ·{\left(\frac{1}{l(\lambda )}\right)}^{n-1}}{l(\lambda )}=\kappa ·{\left(\frac{1}{l(\lambda )}\right)}^n,\hfill \end{array}$$

where the second inequality follows from the definition of $l(\lambda )$ in (6). This completes the proof. □

The following is a trivial corollary of Lemmas 6 and 7.

Corollary 1.

${\mathcal{C}}_{ze}(P_{Y_1,Y_2|X_1,X_2})$ is contained in

$$\begin{array}{cc}\hfill \bigcap _{Q_{Y_1,Y_2|X_1,X_2}}\bigcap _{0\le \lambda \le 1}\{(R_1,& R_2):R_1\ge 0,R_2\ge 0,\lambda R_1+(1-\lambda )R_2\le t(\lambda )\},\hfill \end{array}$$

(13)

where

$$t(\lambda )\triangleq min\left\{\underset{P_{X_1},P_{X_2}}{max}ϵ(\lambda ),\underset{P_{X_1},P_{X_2}}{max}-logl(\lambda )\right\}.$$

(14)

3.2. An Improved Bound

We now provide a single-letter outer bound, in which the order of the minimum and the maximum in (14) is swapped. This generally yields a tighter outer bound due to the max–min inequality. In fact, our bound can be seen as a generalization of the one obtained by Holzman and Körner for the binary multiplying channel [13], in which case the max–min is indeed strictly tighter than the min–max.

Theorem 2.

${\mathcal{C}}_{ze}(P_{Y_1,Y_2|X_1,X_2})$ is contained in

$$\begin{array}{cc}\hfill \bigcap _{Q_{Y_1,Y_2|X_1,X_2}}\bigcap _{0\le \lambda \le 1}\{(R_1,& R_2):R_1\ge 0,R_2\ge 0,\lambda R_1+(1-\lambda )R_2\le \theta (\lambda )\},\hfill \end{array}$$

(15)

where

$$\theta (\lambda )\triangleq \underset{P_{X_1},P_{X_2}}{max}min\{ϵ(\lambda ),-logl(\lambda )\}.$$

(16)

The first intersection is taken over all DM-TWCs $Q_{Y_1,Y_2|X_1,X_2}$ with the same adjacency as $P_{Y_1,Y_2|X_1,X_2}$, and the maximum is taken over all product input probability distributions $P_{X_1}\times P_{X_2}$.

Proof. 

The intersection over all $Q_{Y_1,Y_2|X_1,X_2}$ follows from Proposition 2. Hence without loss of generality, we prove that for $P_{Y_1,Y_2|X_1,X_2}$, each achievable rate pair $(R_1,R_2)$ satisfies $\lambda R_1+(1-\lambda )R_2\le \theta (\lambda )$, where $0\le \lambda \le 1$.

To that end, for each uniquely decodable codebook pair $(\mathcal{A},\mathcal{B})$ of length n, we will show that:

$${|\mathcal{A}|}^{\lambda }{|\mathcal{B}|}^{1-\lambda }\le \kappa ·2^{n\theta (\lambda )}$$

(17)

by induction on n, where $\kappa$ is a constant independent of n. The base case of $n=1$, follows in the same way as in the proof of the base case in Lemma 7. Assume that (17) holds for all length $n^{\prime }\le n-1$, and let us prove it also holds for length n. Suppose that $(\mathcal{A},\mathcal{B})\subseteq {\mathcal{X}}_1^n\times {\mathcal{X}}_2^n$ is a uniquely decodable codebook pair of length n. Following the same steps of (8)–(12) in the argument of Lemma 7, we also have:

$$\begin{array}{cc}\hfill {|\mathcal{A}|}^{\lambda }{|\mathcal{B}|}^{1-\lambda }& ={\left(\frac{|{\mathcal{A}}_i(S)|}{{\sum }_{s\in S}{P}_{X_1}^i(s)}\right)}^{\lambda }·{\left(\frac{|{\mathcal{B}}_i(T)|}{{\sum }_{t\in T}{P}_{X_2}^i(t)})\right)}^{1-\lambda }.\hfill \end{array}$$

(18)

Now, if there exists a dual clique pair $(S,T)$ and a coordinate $1\le i\le n$ such that

$${\left(\sum _{s\in S}{P}_{X_1}^i(s)\right)}^{\lambda }{\left(\sum _{t\in T}{P}_{X_2}^i(t)\right)}^{1-\lambda }\ge 2^{-\theta (\lambda )},$$

(19)

then (18) implies

$$\begin{array}{c}\hfill {|\mathcal{A}|}^{\lambda }{|\mathcal{B}|}^{1-\lambda }\le \frac{\kappa 2^{(n-1)\theta (\lambda )}}{2^{-\theta (\lambda )}}=\kappa 2^{n\theta (\lambda )},\end{array}$$

where the inequality follows from the inductive hypothesis and (19). Therefore, we conclude that (17) holds under condition (19).

Assume now that condition (19) is not satisfied, that is,

$$\underset{i\in [n]}{max}\underset{S,T}{max}{\left(\sum _{s\in S}{P}_{X_1}^i(s)\right)}^{\lambda }{\left(\sum _{t\in T}{P}_{X_2}^i(t)\right)}^{1-\lambda }<2^{-\theta (\lambda )}.$$

(20)

Let $A^n$ and $B^n$ be codewords chosen from $\mathcal{A}$ and $\mathcal{B}$ respectively, uniformly at random, and let $Y_1^n$, $Y_2^n$ be the corresponding channel outputs. Since $(\mathcal{A},\mathcal{B})$ is a uniquely decodable codebook pair of length n, it must be that:

$$\begin{array}{cc}\hfill log|\mathcal{A}|& =I(Y_2^n;A^n|B^n),\hfill \\ \hfill log|\mathcal{B}|& =I(Y_1^n;B^n|A^n).\hfill \end{array}$$

(21)

On the other hand, we have:

$$\begin{array}{cc}\hfill I(Y_1^n;B^n|A^n)& =H(Y_1^n|A^n)-H(Y_1^n|A^n,B^n)\hfill \end{array}$$

(22)

$$\begin{array}{cc}\hfill & =\sum _{i=1}^{n}H(Y_{1,i}|Y_{1,1},\dots ,Y_{1,i-1},A^n)-\sum _{i=1}^{n}H(Y_{1,i}|A_i,B_i)\hfill \end{array}$$

(23)

$$\begin{array}{cc}\hfill & \le \sum _{i=1}^{n}H(Y_{1,i}|A_i)-\sum _{i=1}^{n}H(Y_{1,i}|A_i,B_i)\hfill \end{array}$$

(24)

$$\begin{array}{cc}\hfill & =\sum _{i=1}^{n}I(Y_{1,i};B_i|A_i),\hfill \end{array}$$

(25)

where (23) follows from the entropy chain rule and the memorylessness of the channel, and (24) follows from the fact that conditioning reduces entropy. Similarly,

$$I(Y_2^n;A^n|B^n)\le \sum _{i=1}^{n}I(Y_{2,i};A_i|B_i).$$

(26)

Combining (20)–(26), we obtain

$$\begin{array}{cc}\hfill & {log|\mathcal{A}|}^{\lambda }{|\mathcal{B}|}^{1-\lambda }\hfill \\ \hfill & =\lambda log|\mathcal{A}|+(1-\lambda )log|\mathcal{B}|\hfill \\ \hfill & \le \sum _{i=1}^n\lambda I(Y_{2,i};A_i|B_i)+(1-\lambda )I(Y_{1,i};B_i|A_i)\hfill \\ \hfill & \le \underset{\genfrac{}{}{0pt}{}{P_{X_1},P_{X_2},}{l(\lambda )<2^{-\theta (\lambda )}}}{max}n[\lambda I(Y_2;X_1|X_2)+(1-\lambda )I(Y_1;X_2|X_1)]\hfill \\ \hfill & =\underset{\genfrac{}{}{0pt}{}{P_{X_1},P_{X_2},}{l(\lambda )<2^{-\theta (\lambda )}}}{max}n·ϵ(\lambda ),\hfill \end{array}$$

(27)

where $ϵ(\lambda )$ and $l(\lambda )$ are defined in (4) and (6), respectively, and the maximum is taken over all product input probability distributions $P_{X_1}\times P_{X_2}$ such that $l(\lambda )<2^{-\theta (\lambda )}$, following condition (20).

By the definition of $\theta (\lambda )$, we have:

$$\begin{array}{cc}\hfill \theta (\lambda )& =\underset{P_{X_1},P_{X_2}}{max}min\{ϵ(\lambda ),-logl(\lambda )\}\hfill \end{array}$$

(28)

$$\begin{array}{cc}\hfill & \ge \underset{\genfrac{}{}{0pt}{}{P_{X_1},P_{X_2},}{l(\lambda )<2^{-\theta (\lambda )}}}{max}min\{ϵ(\lambda ),-logl(\lambda )\}.\hfill \end{array}$$

(29)

Note that for any input distributions $P_{X_1},P_{X_2}$ such that $l(\lambda )<2^{-\theta (\lambda )}$, we have

$$-logl(\lambda )>\theta (\lambda ).$$

(30)

Combining (29) and (30), we obtain

$$\underset{\genfrac{}{}{0pt}{}{P_{X_1},P_{X_2},}{l(\lambda )<2^{-\theta (\lambda )}}}{max}ϵ(\lambda )\le \theta (\lambda ).$$

(31)

Substituting (31) into (27), we have ${log|\mathcal{A}|}^{\lambda }{|\mathcal{B}|}^{1-\lambda }\le n\theta (\lambda )$, completing the proof. □

We remark that Theorem 2 immediately implies, in particular, the following upper bound on the zero-error capacity of the point-to-point discrete memoryless channel.

Corollary 2.

The zero-error capacity of the discrete memoryless channel $P_{Y|X}$ is upper bounded by

$$\underset{Q_{Y|X}}{min}\underset{P_X}{max}min\left\{I(X;Y),-log\underset{C}{max}\sum _{x\in C}{P}_X(x)\right\}.$$

The outer minimum is taken over all the $Q_{Y|X}$ having the same confusion graph as $P_{Y|X}$, the outer maximum is taken over all the input distributions $P_X$, and the inner maximum is taken over all the cliques C of the confusion graph of the channel.

As it turns out, the upper bound in Corollary 2 coincides with the linear programming bound on the zero-error capacity of a point-to-point discrete memoryless channel in [2]. Namely,

$$\underset{Q_{Y|X}}{min}\underset{P_X}{max}I(X;Y)=\underset{P_X}{max}\left\{-log\underset{C}{max}\sum _{x\in C}{P}_X(x)\right\}$$

for any point-to-point discrete memoryless channel $P_{Y|X}$. This fact was originally conjectured by Shannon [2] and later proved by Ahlswede [22]. In other words, this means that in the point-to-point case, Corollary 1 yields exactly the same bound as Theorem 2. However, this is not the case in general for the DM-TWC. For example, recall that Holzman and Körner [13] derived the bound in Theorem 2 in the special case of the (deterministic) binary multiplying channel (using $\lambda =0.5$) and numerically showed that it is strictly better than what can be obtained from Corollary 1. Next we give another example showing that Theorem 2 outperforms Corollary 1 for a noisy (i.e., non-deterministic) DM-TWC as well.

Example 1.

Let ${\mathcal{X}}_1=\{0,1,2\},{\mathcal{X}}_2={\mathcal{Y}}_1={\mathcal{Y}}_2=\{0,1\}$, and the conditional probability distribution $P_{Y_1,Y_2|X_1,X_2}$ be

$y_1{y}_2$	$x_1{x}_2$
	00	01	10	11	20	21
00	1	1	0	0	0	0
01	0	0	0	0	1	0
10	0	0	$\delta$	0	0	1
11	0	0	$1-\delta$	1	0	0

where $\delta \in (0,1)$. Corollary 1 gives the upper bound

$$R_1+R_2\le min\left\{\underset{P_{X_1},P_{X_2}}{max}{ϵ}^{*},\underset{P_{X_1},P_{X_2}}{max}-log{l}^{*}\right\}\approx 1.2933,$$

where

$$\begin{array}{cc}\hfill {ϵ}^{*}& =I(X_1;Y_2|X_2)+I(X_2;Y_1|X_1)\hfill \\ \hfill & =P_{X_1}(2)·h(P_{X_2}(0))+P_{X_2}(0)·h(P_{X_1}(0)+\delta ·P_{X_1}(1))-P_{X_1}(1)·P_{X_2}(0)·h(\delta )\hfill \\ \hfill & +P_{X_2}(1)·h(P_{X_1}(1)),\hfill \\ \hfill l^{*}& =\underset{S,T}{max}\left(\sum _{x_1\in S}{P}_{X_1}(x_1)\right)\left(\sum _{x_2\in T}{P}_{X_2}(x_2)\right)\hfill \\ \hfill & =max\{P_{X_1}(0),P_{X_1}(1),P_{X_2}(0)·(P_{X_1}(0)+P_{X_1}(1)),P_{X_2}(0)·(P_{X_1}(1)+P_{X_1}(2)),\hfill \\ \hfill & P_{X_2}(1)·(P_{X_1}(0)+P_{X_1}(2))\},\hfill \end{array}$$

and $h(x)=-xlogx-(1-x)log(1-x)$. In contrast, Theorem 2 yields a tighter upper bound of

$$R_1+R_2\le \underset{P_{X_1},P_{X_2}}{max}min\{{ϵ}^{*},-log{l}^{*}\}\approx 1.2910.$$

3.3. An Outer Bound via Shannon Capacity of a Graph

Based on Lemma 3 and the Shannon capacity of a graph, we immediately have the following bound.

Lemma 8.

$$\begin{array}{c}\hfill {\mathcal{C}}_{ze}^{sum}([G_1,\dots ,G_{|{\mathcal{X}}_1|};H_1,\dots ,H_{|{\mathcal{X}}_2|}])\le \underset{x_1\in {\mathcal{X}}_1,x_2\in {\mathcal{X}}_2}{max}\mathsf{\Theta }(G_{x_1})+\mathsf{\Theta }(H_{x_2}).\end{array}$$

It is worth noting that the above bound could be optimal in the sense that when all $G_i=G$ and $H_j=H$, it is easily verified that ${\mathcal{C}}_{ze}^{sum}([G,\dots ,G;H,\dots ,H])=\mathsf{\Theta }(G)+\mathsf{\Theta }(H)$. However, the bound in Lemma 8 is not tight in general. Later in Section 5, we will improve the bound in Lemma 8 for certain scenarios and show that the improved bound (in Theorem 5) could outperform Theorem 2 (see Example 3), and be achieved in special cases (see Theorem 7).

4. Inner Bounds

In this section, we present two inner bounds for the non-adaptive zero-error capacity region of the DM-TWC, one based on random coding and the other on linear codes.

4.1. Random Coding

The random coding for DM-TWC is standard and generalizes a known bound by Shannon for the one-way case [2]. To obtain the random coding inner bound, we need the following lemma from [1].

Lemma 9

([1]). Let X be a random variable taking values in $[N]$, and ${\{f_i:[N]\to {\mathbb{R}}_{+}\}}_{i\in [d]}$ be a collection of nonnegative functions. Then there exists $x\in [N]$ such that $f_i(x)\le d·\mathbb{E}[f_i(X)]$ for all $i\in [d]$.

Theorem 3.

${\mathcal{C}}_{ze}(P_{Y_1,Y_2|X_1,X_2})$ contains the region:

$$\begin{array}{cc}\hfill \bigcup _{P_{X_1},P_{X_2}}\{(R_1,R_2):& R_1\ge 0,R_2\ge 0,\hfill \\ \hfill & R_1\le -\frac{1}{2}log\sum _{\genfrac{}{}{0pt}{}{x_1\stackrel{x_2}{\sim }{x}_1^{\prime }\vee x_1=x_1^{\prime },}{x_1,x_1^{\prime }\in {\mathcal{X}}_1,x_2\in {\mathcal{X}}_2}}{P}_{X_1}(x_1)P_{X_1}(x_1^{\prime })P_{X_2}(x_2),\hfill \\ \hfill & R_2\le -\frac{1}{2}log\sum _{\genfrac{}{}{0pt}{}{x_2\stackrel{x_1}{\sim }{x}_2^{\prime }\vee x_2=x_2^{\prime },}{x_1\in {\mathcal{X}}_1,x_2,x_2^{\prime }\in {\mathcal{X}}_2}}{P}_{X_1}(x_1)P_{X_2}(x_2)P_{X_2}(x_2^{\prime }),\}\hfill \end{array}$$

(32)

where the union is taken over all input distributions $P_{X_1}$, $P_{X_2}$.

Proof. 

We randomly draw a codebook pair $(\mathcal{A},\mathcal{B})$, such that $\mathcal{A}$ (resp. $\mathcal{B}$) consists of $M_1$ (resp.$M_2$) statistically independent words, where each word is generated i.i.d. according to a probability distribution $P_{X_1}$ (resp.$P_{X_2}$). A word $a^n\in \mathcal{A}$ is called bad, if there exist two words, $b^n,{\tilde{b}}^n\in \mathcal{B}$ that are either equal or adjacent in $G_{a_1}⊠\cdots ⊠G_{a_n}$. For any particular words $a^n\in \mathcal{A}$, $b^n,{\tilde{b}}^n\in \mathcal{B}$ and coordinate $i\in [n]$, the probability that $b_i\sim {\tilde{b}}_i$ in $G_{a_i}$ is upper bounded by:

$$\sum _{\genfrac{}{}{0pt}{}{x_2\stackrel{x_1}{\sim }{x}_2^{\prime }\vee x_2=x_2^{\prime },}{x_1\in {\mathcal{X}}_1,x_2,x_2^{\prime }\in {\mathcal{X}}_2}}{P}_{X_1}(x_1)P_{X_2}(x_2)P_{X_2}(x_2^{\prime }).$$

Since all the coordinates are independent, the probability that $b^n\sim {\tilde{b}}^n$ in $G_{a_1}⊠\cdots ⊠G_{a_n}$ is at most:

$${\left(\sum _{\genfrac{}{}{0pt}{}{x_2\stackrel{x_1}{\sim }{x}_2^{\prime }\vee x_2=x_2^{\prime },}{x_1\in {\mathcal{X}}_1,x_2,x_2^{\prime }\in {\mathcal{X}}_2}}{P}_{X_1}(x_1)P_{X_2}(x_2)P_{X_2}(x_2^{\prime })\right)}^n.$$

(33)

Denote by $\mathrm{Bad}(a^n)$ the number of 2-subsets $\{b^n,{\tilde{b}}^n\}\subseteq \mathcal{B}$ such that $b^n\sim {\tilde{b}}^n$ in $G_{a_1}⊠\cdots ⊠G_{a_n}$. Then,

$$\begin{array}{cc}\hfill \mathrm{Pr}\{a^n\mathrm{is}\mathrm{bad}\}& =\mathrm{Pr}\{\mathrm{Bad}(a^n)\ge 1\}\hfill \\ \hfill & \le \mathbb{E}[\mathrm{Bad}(a^n)]\hfill \\ \hfill & \le \left(\genfrac{}{}{0pt}{}{M_2}{2}\right){\left(\sum _{\genfrac{}{}{0pt}{}{x_2\stackrel{x_1}{\sim }{x}_2^{\prime }\vee x_2=x_2^{\prime },}{x_1\in {\mathcal{X}}_1,x_2,x_2^{\prime }\in {\mathcal{X}}_2}}{P}_{X_1}(x_1)P_{X_2}(x_2)P_{X_2}(x_2^{\prime })\right)}^n,\hfill \end{array}$$

where the first inequality is by Markov’s inequality, and the second inequality follows from (33) and the linearity of expectation. Similarly, a word $b^n\in \mathcal{B}$ is called bad, if there exist two words $a^n,{\tilde{a}}^n\in \mathcal{A}$ that are equal or adjacent in $H_{b_1}⊠\cdots ⊠H_{b_n}$, and we have

$$\mathrm{Pr}\{b^n\mathrm{is}\mathrm{bad}\}\le \left(\genfrac{}{}{0pt}{}{M_1}{2}\right){\left(\sum _{\genfrac{}{}{0pt}{}{x_1\stackrel{x_2}{\sim }{x}_1^{\prime }\vee x_1=x_1^{\prime },}{x_1,x_1^{\prime }\in {\mathcal{X}}_1,x_2\in {\mathcal{X}}_2}}{P}_{X_1}(x_1)P_{X_1}(x_1^{\prime })P_{X_2}(x_2)\right)}^n.$$

Let $f_1(\mathcal{A},\mathcal{B})$, $f_2(\mathcal{A},\mathcal{B})$ be the number of bad words in $\mathcal{A}$ and $\mathcal{B}$ respectively. Then, we have:

$$\begin{array}{cc}\hfill \mathbb{E}[f_1(\mathcal{A},\mathcal{B})]& \le M_1\left(\genfrac{}{}{0pt}{}{M_2}{2}\right){\left(\sum _{\genfrac{}{}{0pt}{}{x_2\stackrel{x_1}{\sim }{x}_2^{\prime }\vee x_2=x_2^{\prime },}{x_1\in {\mathcal{X}}_1,x_2,x_2^{\prime }\in {\mathcal{X}}_2}}{P}_{X_1}(x_1)P_{X_2}(x_2)P_{X_2}(x_2^{\prime })\right)}^n,\hfill \end{array}$$

(34)

$$\begin{array}{cc}\hfill \mathbb{E}[f_2(\mathcal{A},\mathcal{B})]& \le M_2\left(\genfrac{}{}{0pt}{}{M_1}{2}\right){\left(\sum _{\genfrac{}{}{0pt}{}{x_1\stackrel{x_2}{\sim }{x}_1^{\prime }\vee x_1=x_1^{\prime },}{x_1,x_1^{\prime }\in {\mathcal{X}}_1,x_2\in {\mathcal{X}}_2}}{P}_{X_1}(x_1)P_{X_1}(x_1^{\prime })P_{X_2}(x_2)\right)}^n.\hfill \end{array}$$

(35)

By Lemma 9, there exists a pair $({\mathcal{A}}^{*},{\mathcal{B}}^{*})$ such that

$$f_1({\mathcal{A}}^{*},{\mathcal{B}}^{*})\le 2\mathbb{E}[f_1(\mathcal{A},\mathcal{B})],f_2({\mathcal{A}}^{*},{\mathcal{B}}^{*})\le 2\mathbb{E}[f_2(\mathcal{A},\mathcal{B})].$$

(36)

Remove all the bad words in ${\mathcal{A}}^{*}$ and ${\mathcal{B}}^{*}$ respectively, yielding a codebook pair $({\mathcal{A}}^{\prime },{\mathcal{B}}^{\prime })$ such that:

$$|{\mathcal{A}}^{\prime }|=M_1-f_1({\mathcal{A}}^{*},{\mathcal{B}}^{*})\mathrm{and}|{\mathcal{B}}^{\prime }|=M_2-f_2({\mathcal{A}}^{*},{\mathcal{B}}^{*}).$$

(37)

It is readily seen that $({\mathcal{A}}^{\prime },{\mathcal{B}}^{\prime })$ is a uniquely decodable codebook pair.

Now let

$$\begin{array}{cc}\hfill M_1& ={(1-{\xi }_1)}^{\frac{n}{2}}{\left(\sum _{\genfrac{}{}{0pt}{}{x_1\stackrel{x_2}{\sim }{x}_1^{\prime }\vee x_1=x_1^{\prime },}{x_1,x_1^{\prime }\in {\mathcal{X}}_1,x_2\in {\mathcal{X}}_2}}{P}_{X_1}(x_1)P_{X_1}(x_1^{\prime })P_{X_2}(x_2)\right)}^{-\frac{n}{2}},\hfill \end{array}$$

(38)

$$\begin{array}{cc}\hfill M_2& ={(1-{\xi }_2)}^{\frac{n}{2}}{\left(\sum _{\genfrac{}{}{0pt}{}{x_2\stackrel{x_1}{\sim }{x}_2^{\prime }\vee x_2=x_2^{\prime },}{x_1\in {\mathcal{X}}_1,x_2,x_2^{\prime }\in {\mathcal{X}}_2}}{P}_{X_1}(x_1)P_{X_2}(x_2)P_{X_2}(x_2^{\prime })\right)}^{-\frac{n}{2}},\hfill \end{array}$$

(39)

where ${\xi }_1,{\xi }_2$ are arbitrarily small positive numbers. Combining (34)–(39), we obtain:

$$\begin{array}{cc}\hfill |{\mathcal{A}}^{\prime }|& \ge (1-{(1-{\xi }_2)}^n){(1-{\xi }_1)}^{\frac{n}{2}}{\left(\sum _{\genfrac{}{}{0pt}{}{x_1\stackrel{x_2}{\sim }{x}_1^{\prime }\vee x_1=x_1^{\prime },}{x_1,x_1^{\prime }\in {\mathcal{X}}_1,x_2\in {\mathcal{X}}_2}}{P}_{X_1}(x_1)P_{X_1}(x_1^{\prime })P_{X_2}(x_2)\right)}^{-\frac{n}{2}},\hfill \\ \hfill |{\mathcal{B}}^{\prime }|& \ge (1-{(1-{\xi }_1)}^n){(1-{\xi }_2)}^{\frac{n}{2}}{\left(\sum _{\genfrac{}{}{0pt}{}{x_2\stackrel{x_1}{\sim }{x}_2^{\prime }\vee x_2=x_2^{\prime },}{x_1\in {\mathcal{X}}_1,x_2,x_2^{\prime }\in {\mathcal{X}}_2}}{P}_{X_1}(x_1)P_{X_2}(x_2)P_{X_2}(x_2^{\prime })\right)}^{-\frac{n}{2}}.\hfill \end{array}$$

Since ${\xi }_1,{\xi }_2$ are arbitrarily small, by taking n as sufficiently large, we can find an $(n,R_1,R_2)$ uniquely decodable codebook pair arbitrarily close to (32), as desired. □

4.2. Linear Codes

In this subsection, we present a construction of uniquely decodable codes via linear codes, which generalizes a known result for the binary multiplying channel [15]. Let us introduce some notations first. Suppose D is a set of letters, $x^n$ and $y^n$ are vectors of length n, and $\mathcal{C}$ is a collection of vectors of length n. Let:

$${\mathrm{ind}}_D(x^n)\triangleq \{1\le i\le n:x_i\in D\}$$

(40)

denote the collection of indices where $x_i\in D$. For $I\subseteq [n]$ let $y^n{|}_I$ denote the vector obtained from $y^n$ by projecting onto the coordinates in I, and denote

$${\mathcal{C}}_{|I}\triangleq \{c^n{|}_I:c^n\in \mathcal{C}\}.$$

Let $P_{Y_1,Y_2|X_1,X_2}$ be a DM-TWC. We say that $x_1\in {\mathcal{X}}_1$ is a detecting symbol, if $x_2\stackrel{x_1}{\nsim }{x}_2^{\prime }$ for any distinct $x_2,x_2^{\prime }\in {\mathcal{X}}_2$. A detecting symbol $x_2\in {\mathcal{X}}_2$ is defined analogously. Let $D_1\subseteq {\mathcal{X}}_1$ and $D_2\subseteq {\mathcal{X}}_2$ denote the sets of all detecting symbols in ${\mathcal{X}}_1$ and ${\mathcal{X}}_2$, respectively. A vector $a^n\in {\mathcal{X}}_1^n$ is called a detecting vector for $\mathcal{B}\subseteq {\mathcal{X}}_2^n$ if

$$\left|{\mathcal{B}}_{|{\mathrm{ind}}_{D_1}(a^n)}\right|=|\mathcal{B}|.$$

(41)

Similarly, a vector $b^n\in {\mathcal{X}}_2^n$ is a detecting vector for $\mathcal{A}\subseteq {\mathcal{X}}_1^n$ if

$$|{\mathcal{A}}_{|{\mathrm{ind}}_{D_2}(b^n)}|=|\mathcal{A}|.$$

(42)

The following claim is immediate.

Proposition 7.

Let $\mathcal{A}\subseteq {\mathcal{X}}_1^n$, $\mathcal{B}\subseteq {\mathcal{X}}_2^n$. If each $a^n\in \mathcal{A}$ is a detecting vector for $\mathcal{B}$ and each $b^n\in \mathcal{B}$ is a detecting vector for $\mathcal{A}$, then $(\mathcal{A},\mathcal{B})$ is a uniquely decodable codebook pair.

Proposition 7 provides a sufficient condition for a uniquely decodable code, which is not always necessary (see Example 2). Nevertheless, this sufficient condition furnishes us with a way of constructing uniquely decodable codes by employing linear codes.

Example 2.

Suppose that ${\mathcal{X}}_1=\{a_0,a_1,a_2\}$, ${\mathcal{X}}_2=\{b_0,b_1\}$ such that $D_1=\{a_0,a_1,a_2\}$, $D_2=\{b_1\}$, and $a_0\stackrel{b_0}{\sim }{a}_1$, $a_0\stackrel{b_0}{\sim }{a}_2$, $a_1\stackrel{b_0}{\nsim }{a}_2$. Let $\mathcal{A}=\{a_0{a}_0{a}_0,a_1{a}_1{a}_1,a_0{a}_1{a}_2\}$ and $\mathcal{B}=\{b_0{b}_1{b}_0\}$. It is easy to verify that $(\mathcal{A},\mathcal{B})$ is a uniquely decodable codebook pair. However, ${\mathrm{ind}}_{D_2}(b_0{b}_1{b}_0)=\{2\}$ and $|{\mathcal{A}}_{|\{2\}}|=|\{a_0,a_1\}|=2<|\mathcal{A}|=3$, implying that $b_0{b}_1{b}_0$ is not a detecting vector for $\mathcal{A}$.

Assume that $|{\mathcal{X}}_1|=q_1$ and $|{\mathcal{X}}_2|=q_2$, where $q_1,q_2$ are prime powers, and let us think of the alphabets as ${\mathbb{F}}_{q_1}$ and ${\mathbb{F}}_{q_2}$, respectively. The following theorem gives an inner bound on the capacity region, which is a generalization of the Tolhuizen’s construction for the Blackwell’s multiplying channel [15].

Theorem 4.

Let $P_{Y_1,Y_2|X_1,X_2}$ be a DM-TWC with input alphabet sizes $|{\mathcal{X}}_1|=q_1$, $|{\mathcal{X}}_2|=q_2$, where $q_1$, $q_2$ are prime powers. If ${\mathcal{X}}_1$ and ${\mathcal{X}}_2$ contain ${\tau }_1$ and ${\tau }_2$ detecting symbols respectively, then ${\mathcal{C}}_{ze}(P_{Y_1,Y_2|X_1,X_2})$ contains the region

$$\begin{array}{cc}\hfill \bigcup _{0\le \alpha ,\beta \le 1}\{(R_1,R_2):& R_1\ge 0,R_2\ge 0,\hfill \\ \hfill & R_1\le h(\alpha )+\alpha log{\tau }_2+(1-\alpha )log(q_2-{\tau }_2)-(1-\beta )log{q}_2,\hfill \\ \hfill & R_2\le h(\beta )+\beta log{\tau }_1+(1-\beta )log(q_1-{\tau }_1)-(1-\alpha )log{q}_1\},\hfill \end{array}$$

(43)

where $h(x)\triangleq -xlogx-(1-x)log(1-x)$ is the binary entropy function.

To prove this theorem, we need the following lemma. The case that $q_1=q_2=2$ and $\tau =1$ was proved in ([15], Theorem 3). Lemma 10 follows from similar argument.

Lemma 10.

Let q, $q^{\prime }$ be prime powers, $n,k$ be positive integers such that $1\le k\le n$, and $D\subseteq {\mathbb{F}}_{q^{\prime }}$ with cardinality $|D|=\tau$. Then there exists a pair $(\mathcal{C},\mathrm{Y}(\mathcal{C}))$ satisfying that:

(1) 

$\mathcal{C}$ is a q-ary $[n,k]$ linear code;

(2) 

$\mathrm{Y}(\mathcal{C})\subseteq {\mathbb{F}}_{q^{\prime }}^n$ such that

$$|\mathrm{Y}(\mathcal{C})|\ge \left(\genfrac{}{}{0pt}{}{n}{k}\right){\tau }^k{(q^{\prime }-\tau )}^{n-k}\prod _{i=1}^{\infty }(1-q^{-i});$$

(3) 

for each $x^n\in \mathrm{Y}(\mathcal{C})$, we have $|{ind}_D(x^n)|=k$ and $\left|{\mathcal{C}}_{|{ind}_D(x^n)}\right|=|\mathcal{C}|$.

Proof. 

Let A be a $k\times n$ matrix of full rank over ${\mathbb{F}}_q$, then $\mathcal{C}(A)\triangleq \{y^{k}A:y^k\in {\mathbb{F}}_q^k\}$ is a q-ary $[n,k]$ linear code generated by A. Recall that for every $x^n\in {\mathbb{F}}_{q^{\prime }}^n$, ${\mathrm{ind}}_D(x^n)=\{i\in [n]:x_i\in D\}$ as in (40). Denote:

$$\mathrm{Y}(\mathcal{C}(A))\triangleq \left\{x^n\in {\mathbb{F}}_{q^{\prime }}^n:|{\mathrm{ind}}_D(x^n)|=k,|{\mathcal{C}}_{|{\mathrm{ind}}_D(x^n)}|=|\mathcal{C}|\right\}.$$

Let ${A|}_{{\mathrm{ind}}_D(x^n)}$ denote the $k\times |{\mathrm{ind}}_D(x^n)|$ submatrix of A with columns indexed by ${\mathrm{ind}}_D(x^n)$. It is easy to see that $|{\mathcal{C}}_{|{\mathrm{ind}}_D(x^n)}|=|\mathcal{C}|$ is equivalent to $\mathrm{rank}(A{|}_{{\mathrm{ind}}_D(x^n)})=k$. Denote:

$$\mathcal{P}\triangleq \left\{(A,x^n):A\in {\mathbb{F}}_q^{k\times n},x^n\in {\mathbb{F}}_{q^{\prime }}^n,|{\mathrm{ind}}_D(x^n)|=k,\mathrm{rank}(A{|}_{{\mathrm{ind}}_D(x^n)})=k\right\},$$

(44)

and let us proceed by double counting the cardinality of $\mathcal{P}$.

On the one hand, the number of vectors $x^n\in {\mathbb{F}}_{q^{\prime }}^n$ such that $|{\mathrm{ind}}_D(x^n)|=k$ is $\left(\genfrac{}{}{0pt}{}{n}{k}\right){\tau }^k{(q^{\prime }-\tau )}^{n-k}$. For each such $x^n$, there are $q^{k(n-k)}{I}_q(k)$ corresponding $k\times n$ matrices $A\in {\mathbb{F}}_q^{k\times n}$ such that $\mathrm{rank}(A{|}_{{\mathrm{ind}}_D(x^n)})=k$, where $I_q(k)={\prod }_{i=0}^{k-1}(q^k-q^i)$ is the number of $k\times k$ invertible matrices over ${\mathbb{F}}_q$, see ([15], Lemma 3). Hence, we have:

$$|\mathcal{P}|=\left(\genfrac{}{}{0pt}{}{n}{k}\right){\tau }^k{(q^{\prime }-\tau )}^{n-k}{q}^{k(n-k)}{I}_q(k).$$

On the other hand, the number of matrices $A\in {\mathbb{F}}_q^{k\times n}$ is $q^{nk}$. By (44) and the pigeonhole principle, there exist a matrix $A^{*}\in {\mathbb{F}}_q^{k\times n}$ and a corresponding code $\mathcal{C}(A^{*})$ such that $|\mathrm{Y}(\mathcal{C}(A))|\ge |\mathcal{P}|/q^{nk}$. Letting $\mathcal{C}=\mathcal{C}(A^{*})$, the lemma follows. □

Proof of Theorem 4.

For $i=1,2$, let us identify ${\mathcal{X}}_i$ with ${\mathbb{F}}_{q_i}$, and let the respective sets of all detecting symbols be $D_i\subseteq {\mathbb{F}}_{q_i}$ with $|D_i|={\tau }_i$.

To prove the existence of a uniquely decodable codebook pair based on Proposition 7, we first use Lemma 10 to find two “one-sided” uniquely decodable linear codebook pairs, and then combine them to the desired codebook pair by employing their cosets in ${\mathbb{F}}_{q_1}^n$ and ${\mathbb{F}}_{q_2}^n$.

First, letting $q=q_1$, $q^{\prime }=q_2$, $G=D_2$ and $\tau ={\tau }_2$ in Lemma 10, we have a pair $({\mathcal{C}}_1,\mathrm{Y}({\mathcal{C}}_1))$ satisfying that ${\mathcal{C}}_1$ is a $q_1$-ary $[n,k_1]$ linear code and $\mathrm{Y}({\mathcal{C}}_1)\subseteq {\mathbb{F}}_{q_2}^n$ such that

$$|\mathrm{Y}({\mathcal{C}}_1)|\ge \left(\genfrac{}{}{0pt}{}{n}{k_1}\right){{\tau }_2}^{k_1}{(q_2-{\tau }_2)}^{n-k_1}\prod _{i=1}^{\infty }(1-q_1^{-i}).$$

(45)

Similarly, letting $q=q_2$, $q^{\prime }=q_1$, $G=D_1$ and $\tau ={\tau }_1$ in Lemma 10, we have a pair $({\mathcal{C}}_2,\mathrm{Y}({\mathcal{C}}_2))$ satisfying that ${\mathcal{C}}_2$ is a $q_2$-ary $[n,k_2]$ linear code and $\mathrm{Y}({\mathcal{C}}_2)\subseteq {\mathbb{F}}_{q_1}^n$ such that

$$|\mathrm{Y}({\mathcal{C}}_2)|\ge \left(\genfrac{}{}{0pt}{}{n}{k_2}\right){{\tau }_1}^{k_2}{(q_1-{\tau }_1)}^{n-k_2}\prod _{i=1}^{\infty }(1-q_2^{-i}).$$

(46)

The property $(3)$ in Lemma 10 implies that each $x^n\in \mathrm{Y}({\mathcal{C}}_i)$ is a detecting vector for ${\mathcal{C}}_i$ for $i=1,2$. Note that if $\mathsf{\Xi }({\mathcal{C}}_i)\subseteq {\mathbb{F}}_{q_i}^n$ is a coset of ${\mathcal{C}}_i$, then each $x^n\in \mathrm{Y}({\mathcal{C}}_i)$ is also a detecting vector for $\mathsf{\Xi }({\mathcal{C}}_i)$.

Now we are going to combine the two pairs $({\mathcal{C}}_1,\mathrm{Y}({\mathcal{C}}_1))$ and $({\mathcal{C}}_2,\mathrm{Y}({\mathcal{C}}_2))$. Since ${\mathcal{C}}_i$ has $q_i^{n-k_i}$ cosets, then by the pigeonhole principle there exists coset $\mathsf{\Xi }({\mathcal{C}}_i)$ of ${\mathcal{C}}_i$ such that:

$$\begin{array}{cc}\hfill \mathcal{A}& \triangleq \mathrm{Y}({\mathcal{C}}_1)\cap \mathsf{\Xi }({\mathcal{C}}_2),|\mathcal{A}|\ge \frac{|\mathrm{Y}({\mathcal{C}}_1)|}{q_2^{n-k_2}},\hfill \\ \hfill \mathcal{B}& \triangleq \mathrm{Y}({\mathcal{C}}_2)\cap \mathsf{\Xi }({\mathcal{C}}_1),|\mathcal{B}|\ge \frac{|\mathrm{Y}({\mathcal{C}}_2)|}{q_1^{n-k_1}}.\hfill \end{array}$$

(47)

We now notice that each vector in $\mathcal{A}$ (resp.$\mathcal{B}$) is a detecting vector for $\mathcal{B}$ (resp.$\mathcal{A}$), hence by Proposition 7 $(\mathcal{A},\mathcal{B})$ is a uniquely decodable codebook pair. Moreover, for fixed $q_1$, $q_2$, we have:

$$\begin{array}{cc}\hfill \frac{log|\mathcal{A}|}{n}\ge & h(\frac{k_1}{n})+(\frac{k_1}{n})log{\tau }_2+(1-\frac{k_1}{n})log(q_2-{\tau }_2)-(1-\frac{k_2}{n})log{q}_2-O(\frac{1}{n}),\hfill \end{array}$$

$$\begin{array}{cc}\hfill \frac{log|\mathcal{B}|}{n}\ge & h(\frac{k_2}{n})+(\frac{k_2}{n})log{\tau }_1+(1-\frac{k_2}{n})log(q_1-{\tau }_1)-(1-\frac{k_1}{n})log{q}_1-O(\frac{1}{n}),\hfill \end{array}$$

which follows from (45)–(47). Letting $\alpha =k_1/n\in [0,1]$, $\beta =k_2/n\in [0,1]$, we obtain:

$$\begin{array}{cc}\hfill R_1& =\underset{n\to \infty }{lim}\frac{log|\mathcal{A}|}{n}\ge h(\alpha )+\alpha log{\tau }_2+(1-\alpha )log(q_2-{\tau }_2)-(1-\beta )log{q}_2,\hfill \\ \hfill R_2& =\underset{n\to \infty }{lim}\frac{log|\mathcal{B}|}{n}\ge h(\beta )+\beta log{\tau }_1+(1-\beta )log(q_1-{\tau }_1)-(1-\alpha )log{q}_1.\hfill \end{array}$$

Therefore, (43) follows, as desired. □

We note that for any DM-TWC, one could only exploit part of input symbols ${\mathcal{X}}_1^{\prime }\subseteq {\mathcal{X}}_1$, ${\mathcal{X}}_2^{\prime }\subseteq {\mathcal{X}}_2$ to meet the requirements in Theorem 4. Hence we in fact have the following more general bound.

Corollary 3.

Let $P_{Y_1,Y_2|X_1,X_2}$ be a DM-TWC with input alphabets ${\mathcal{X}}_1$, ${\mathcal{X}}_2$. Then ${\mathcal{C}}_{ze}(P_{Y_1,Y_2|X_1,X_2})$ contains the region:

$$\begin{array}{cc}\hfill \bigcup _{\genfrac{}{}{0pt}{}{{\mathcal{X}}_1^{\prime }\subseteq {\mathcal{X}}_1,}{{\mathcal{X}}_2^{\prime }\subseteq {\mathcal{X}}_2}}\bigcup _{0\le \alpha ,\beta \le 1}\{(R_1,R_2):& R_1\ge 0,R_2\ge 0,\hfill \\ \hfill & R_1\le h(\alpha )+\alpha log{\tau }_2^{\prime }+(1-\alpha )log(q_2^{\prime }-{\tau }_2^{\prime })-(1-\beta )log{q}_2^{\prime },\hfill \\ \hfill & R_2\le h(\beta )+\beta log{\tau }_1^{\prime }+(1-\beta )log(q_1^{\prime }-{\tau }_1^{\prime })-(1-\alpha )log{q}_1^{\prime }\},\hfill \end{array}$$

where the first union is taken over all ${\mathcal{X}}_1^{\prime }\subseteq {\mathcal{X}}_1$, ${\mathcal{X}}_2^{\prime }\subseteq {\mathcal{X}}_2$ such that $|{\mathcal{X}}_1^{\prime }|$ and $|{\mathcal{X}}_2^{\prime }|$ are prime powers, and contain ${\tau }_1^{\prime }$ and ${\tau }_2^{\prime }$ detecting symbols, respectively.

Notice that the region (43) relies on the number $q_1$, $q_2$ of symbols being used and the corresponding numbers ${\tau }_1,{\tau }_2$ of detecting symbols. It is thus possible that using only a smaller subset of channel inputs would yield higher achievable rates (when using our linear coding strategy) than those obtained by using larger subsets. For Example 1, Corollary 3 shows that a lower bound on the maximum sum-rate $R_1+R_2$ is $1.17$, which is better than the random coding lower bound $1.0907$.

5. Certain Types of DM-TWC

In this section, we consider the DM-TWC in the scenario that the communication in one direction is stable (in particular, noiseless). First we briefly review the probabilistic refinement of the Shannon capacity of a graph in Section 5.1. Then in Section 5.2, we provide an outer bound on the zero-error capacity region via the asymptotic spectrum of graphs. In Section 5.3, we present explicit constructions that attain our outer bound in certain special cases.

5.1. Probabilistic Refinement of the Shannon Capacity of a Graph

We first recall some basic notions and results from the method-of-types. Let $x^n\in {\mathcal{X}}^n$ be a sequence and $N(a|x^n)$ be the number of times that $a\in \mathcal{X}$ appears in sequence $x^n$. The type $P_{x^n}$ of $x^n$ is the relative proportion of each symbol in $\mathcal{X}$, that is, $P_{x^n}(a)\triangleq \frac{N(a|x^n)}{n}$ for all $a\in \mathcal{X}$. Let ${\mathcal{P}}_n$ denote the collection of all possible types of sequences of length n. For every $P\in {\mathcal{P}}_n$, the type class$T^n(P)$ of P is the set of sequences of type P, that is, $T^n(P)\triangleq \{x^n:P_{x^n}=P\}$. The ϵ-typical set of P is

$$T_{ϵ}^n(P)\triangleq \{x^n\in {\mathcal{X}}^n:|P_{x^n}(a)-P(a)|<ϵ,\forall a\in \mathcal{X}\}.$$

The joint type $P_{x^n,y^n}$ of a pair of sequences $(x^n,y^n)$ is the relative proportion of occurrences of each pair of symbols of $\mathcal{X}\times \mathcal{Y}$, that is, $P_{x^n,y^n}\triangleq \frac{N(a,b|x^n,y^n)}{n}$ for all $a\in \mathcal{X}$ and $b\in \mathcal{Y}$. By the Bayes’ rule, the conditional type $P_{x^n|y^n}$ is defined as:

$$P_{x^n|y^n}(a,b)\triangleq \frac{N(a,b|x^n,y^n)}{N(b|y^n)}=\frac{P_{x^n,y^n}(a,b)}{P_{y^n}(b)}.$$

Lemma 11

([23]). $|{\mathcal{P}}_n{|\le (n+1)}^{|\mathcal{X}|}$.

Lemma 12

([23]). $\forall P\in {\mathcal{P}}_n$, we have $\frac{2^{nH(X)}}{{(n+1)}^{|\mathcal{X}|}}\le |T^n(P)|\le 2^{nH(X)}$.

In [24], Csiszár and Körner introduced the probabilistic refinement of the Shannon capacity of a graph, imposing that the independent set consists of sequences of the (asymptotically) same type. Let $G_{ϵ}^n[P]$ denote the subgraph of $G^n$ induced by $T_{ϵ}^n(P)$. The Shannon capacity of graph G relative to P is defined as

$$\mathsf{\Theta }(G,P)\triangleq \underset{ϵ\to 0}{lim}\underset{n\to \infty }{lim\; sup}\frac{1}{n}log\alpha (G_{ϵ}^n[P]).$$

Let $G^n[P]$ denote the subgraph of $G^n$ induced by $T^n(P)$. Then, it is readily seen that:

$$\underset{n\to \infty }{lim\; sup}\frac{1}{n}log\alpha (G^n[P])\le \mathsf{\Theta }(G,P).$$

For each $\eta \in \Delta (\mathcal{G})$, define

$$\widehat{\eta }(G,P)\triangleq \underset{ϵ\to 0}{lim}\underset{n\to \infty }{lim\; sup}\frac{1}{n}log\eta (G_{ϵ}^n[P]).$$

(48)

If $G={\overline{K}}_n$, then according to Lemma 12, we have

$$\widehat{\eta }({\overline{K}}_n,P)=H(X)$$

(49)

for any $\eta \in \Delta (\mathcal{G})$. Very recently, Vrana [25] proved the following results on $\widehat{\eta }(G,P)$.

Lemma 13

([25]). The limit in (48) exists and

(1) 

$\mathsf{\Theta }(G,P)=\underset{\eta \in \Delta (\mathcal{G})}{min}\widehat{\eta }(G,P)$;

(2) 

$log\eta (G)=\underset{P}{max}\widehat{\eta }(G,P)$ for $\eta \in \Delta (\mathcal{G})$.

According to Lemma 11, it is easily seen that:

$$\mathsf{\Theta }(G)=\underset{P}{max}\mathsf{\Theta }(G,P).$$

Here, we would like to mention that the probabilistic refinement of the Lovász theta number was introduced and investigated by Marton in [26] via a non-asymptotic formula, and the probabilistic refinement of the fractional clique cover number was studied in relation to the graph entropy in [27].

5.2. An Outer Bound via the Asymptotic Spectrum of Graphs

In this subsection, we derive an outer bound for the case when all $\{H_j\}$ are the same, namely, $H_j=H$ for all $j\in {\mathcal{X}}_2$.

Theorem 5.

${\mathcal{C}}_{ze}([G_1,\dots ,G_{|{\mathcal{X}}_1|};H,\dots ,H])$ is contained in the region

$$\begin{array}{c}\hfill \left\{(R_1,R_2):R_1\ge 0,R_2\ge 0,R_1+R_2\le \underset{P_{X_1}}{max}\sum _{x_1\in {\mathcal{X}}_1}{P}_{X_1}(x_1)\mathsf{\Theta }(G_{x_1})+\mathsf{\Theta }(H,P_{X_1})\right\}.\end{array}$$

Proof. 

Suppose that $(\mathcal{A},\mathcal{B})\subseteq {\mathcal{X}}_1^n\times {\mathcal{X}}_2^n$ is a uniquely decodable codebook pair of length n. For any $a^n\in \mathcal{A}$ and $b^n\in \mathcal{B}$, let $P_{a^n,b^n}$ denote the joint type of the pair $(a^n,b^n)$ and

$$J^n(P_{X_1,X_2})\triangleq \{(a^n,b^n):a^n\in \mathcal{A},b^n\in \mathcal{B},P_{a^n,b^n}=P_{X_1,X_2}\}.$$

By Lemma 11, there are at most ${(n+1)}^{|{\mathcal{X}}_1||{\mathcal{X}}_2|}$ different joint types over $(\mathcal{A},\mathcal{B})$. Thus by the pigeonhole principle, there exists one joint type $P_{X_1,X_2}^{*}$ such that:

$$|J^n(P_{X_1,X_2}^{*})|\ge \frac{|\mathcal{A}||\mathcal{B}|}{{(n+1)}^{|{\mathcal{X}}_1||{\mathcal{X}}_2|}}.$$

(50)

Notice that for each $(a^n,b^n)\in J^n(P_{X_1,X_2}^{*})$, we have:

$$\begin{array}{cc}\hfill P_{a^n}& =P_{X_1}^{*}=\sum _{x_2\in {\mathcal{X}}_2}{P}_{X_1,X_2=x_2}^{*},\hfill \\ \hfill P_{b^n}& =P_{X_2}^{*}=\sum _{x_1\in {\mathcal{X}}_1}{P}_{X_1=x_1,X_2}^{*}.\hfill \end{array}$$

Now we are going to upper bound the cardinality of $J^n(P_{X_1,X_2}^{*})$. Let ${\mathcal{A}}^{*}$ (resp. ${\mathcal{B}}^{*}$) denote the collection of $a^n\in \mathcal{A}$ (resp. $b^n\in \mathcal{B}$) that appears in $J^n(P_{X_1,X_2}^{*})$, that is, there exists $b^n\in \mathcal{B}$ (resp. $a^n\in \mathcal{A}$) such that $P_{a^n,b^n}=P_{X_1,X_2}^{*}$. Then we immediately have

$$|J^n(P_{X_1,X_2}^{*})|\le |{\mathcal{A}}^{*}||{\mathcal{B}}^{*}|.$$

(51)

Let us now turn to upper bound the cardinalities of ${\mathcal{A}}^{*}$ and ${\mathcal{B}}^{*}$. Since $(\mathcal{A},\mathcal{B})$ is uniquely decodable, by Proposition 1, for any $a^n\in {\mathcal{A}}^{*}$ it must hold that ${\mathcal{B}}^{*}$ is an independent set of $G_{a_1}⊠G_{a_2}⊠\cdots ⊠G_{a_n}$. Accordingly,

$$\begin{array}{cc}\hfill |{\mathcal{B}}^{*}|& \le \alpha \left(G_1^{n{P}_{X_1}^{*}(1)}⊠G_2^{n{P}_{X_1}^{*}(2)}⊠\cdots ⊠G_{|{\mathcal{X}}_1|}^{n{P}_{X_1}^{*}(|{\mathcal{X}}_1|)}\right).\hfill \end{array}$$

(52)

Also, for $b^n\in {\mathcal{B}}^{*}$, we notice that ${\mathcal{A}}^{*}$ is an independent set of $H^n$ with type $P_{X_1}^{*}$. To be precise, we have:

$$\begin{array}{cc}\hfill |{\mathcal{A}}^{*}|& \le \alpha \left(H^n[P_{X_1}^{*}]\right).\hfill \end{array}$$

(53)

Therefore we have:

$$\begin{array}{cc}\hfill & \underset{n\to \infty }{lim\; sup}\frac{1}{n}log|\mathcal{A}||\mathcal{B}|\hfill \end{array}$$

$$\begin{array}{cc}\hfill & \le \underset{n\to \infty }{lim\; sup}\frac{1}{n}log\left({(n+1)}^{|{\mathcal{X}}_1||{\mathcal{X}}_2|}|J^n(P_{X_1,X_2}^{*})|\right)\hfill \end{array}$$

(54)

$$\begin{array}{cc}\hfill & =\underset{n\to \infty }{lim\; sup}\frac{1}{n}log|J^n(P_{X_1,X_2}^{*})|\hfill \end{array}$$

(55)

$$\begin{array}{cc}\hfill & \le \underset{n\to \infty }{lim\; sup}\frac{1}{n}log|{\mathcal{A}}^{*}||{\mathcal{B}}^{*}|\hfill \end{array}$$

(56)

$$\begin{array}{cc}\hfill & \le \underset{n\to \infty }{lim\; sup}\frac{1}{n}\left[log\left(\alpha \left(G_1^{n{P}_{X_1}^{*}(1)}⊠\cdots ⊠G_{|{\mathcal{X}}_1|}^{n{P}_{X_1}^{*}(|{\mathcal{X}}_1|)}\right)\right)+log\left(\alpha \left(H^n[P_{X_1}^{*}]\right)\right)\right]\hfill \end{array}$$

(57)

$$\begin{array}{cc}\hfill & \le \underset{n\to \infty }{lim\; sup}\underset{\eta ,{\eta }^{\prime }\in \Delta (\mathcal{G})}{min}\frac{1}{n}\left[log\left(\eta \left(G_1^{n{P}_{X_1}^{*}(1)}⊠\cdots ⊠G_{|{\mathcal{X}}_1|}^{n{P}_{X_1}^{*}(|{\mathcal{X}}_1|)}\right)\right)+log\left({\eta }^{\prime }\left(H^n[P_{X_1}^{*}]\right)\right)\right]\hfill \end{array}$$

(58)

$$\begin{array}{cc}\hfill & \le \underset{n\to \infty }{lim\; sup}\underset{\eta ,{\eta }^{\prime }\in \Delta (\mathcal{G})}{min}\frac{1}{n}\left[\sum _{x_1\in {\mathcal{X}}_1}n{P}_{X_1}^{*}(x_1)log\left(\eta \left(G_{x_1}\right)\right)+log\left({\eta }^{\prime }\left(H^n[P_{X_1}^{*}]\right)\right)\right]\hfill \end{array}$$

(59)

$$\begin{array}{cc}\hfill & \le \underset{P_{X_1}}{max}\sum _{x_1\in {\mathcal{X}}_1}{P}_{X_1}(x_1)\mathsf{\Theta }(G_{x_1})+\mathsf{\Theta }(H,P_{X_1}),\hfill \end{array}$$

(60)

where (54) follows from (50); (55) follows from the fact that $|{\mathcal{X}}_1|$, $|{\mathcal{X}}_2|$ are fixed when n tends to infinity; (56) follows from (51); (57) follows from (52) and (53); (58) follows from Theorem 1 that $\alpha (G)\le {min}_{\eta \in \Delta (\mathcal{G})}\eta (G)$ for any graph G; (59) follows from Theorem 1 that each $\eta \in \Delta (\mathcal{G})$ is multiplicative with respect to the strong product; and (60) follows from Theorem 1 and Lemma 13.

This completes the proof. □

In particular, considering the DM-TWC such that $|{\mathcal{X}}_1|=2$, $H={\overline{K}}_2$, $G_1={\overline{K}}_{|{\mathcal{X}}_2|}$ and $G_2=G$ is a general graph, we have the following result.

Theorem 6.

${\mathcal{C}}_{ze}([{\overline{K}}_{|{\mathcal{X}}_2|},G;{\overline{K}}_2,\dots ,{\overline{K}}_2])$ is contained in the region

$$\begin{array}{c}\hfill \left\{(R_1,R_2):R_1\ge 0,R_2\ge 0,R_1+R_2\le log\left(|{\mathcal{X}}_2|+2^{\mathsf{\Theta }(G)}\right)\right\}.\end{array}$$

Proof. 

Recall that: $\mathsf{\Theta }({\overline{K}}_n)=log(n)$ and $\mathsf{\Theta }({\overline{K}}_n,P)=H(X)$. According to Theorem 5, we have:

$$\begin{array}{cc}\hfill R_1+R_2& \le \underset{P_{X_1}}{max}\sum _{x_1\in {\mathcal{X}}_1}{P}_{X_1}(x_1)\mathsf{\Theta }(G_{x_1})+\mathsf{\Theta }({\overline{K}}_2,P_{X_1})\hfill \\ \hfill & =\underset{P_{X_1}}{max}\left\{P_{X_1}(1)·log|{\mathcal{X}}_2|+P_{X_1}(2)·\mathsf{\Theta }(G)+H(X_1)\right\}\hfill \\ \hfill & =log\left(|{\mathcal{X}}_2|+2^{\mathsf{\Theta }(G)}\right),\hfill \end{array}$$

where the last equality is achieved by taking $P_{X_1}(1)=\frac{|{\mathcal{X}}_2|}{|{\mathcal{X}}_2|+2^{\mathsf{\Theta }(G)}}$ and $P_{X_1}(2)=\frac{2^{\mathsf{\Theta }(G)}}{|{\mathcal{X}}_2|+2^{\mathsf{\Theta }(G)}}$. □

We remark that Theorem 6 (hence also Theorem 5) could outperform Theorem 2, see the following example.

Example 3.

Consider the channel $[{\overline{K}}_5,C_5;{\overline{K}}_2,\dots ,{\overline{K}}_2]$ where $C_5$ is the Pentagon graph. It is well known from [2,12] that $\mathsf{\Theta }(C_5)=\frac{1}{2}log5$. Then by Theorem 6 we have an upper bound on the sum-rate $R_1+R_2\le log(5+\sqrt{5})\approx 2.8552$, while Theorem 2 only gives an upper bound $R_1+R_2\le 2.9069$.

5.3. Explicit Constructions

In this subsection, we present explicit constructions of uniquely decodable codebook pairs which could attain the outer bound of Theorem 6 in certain special cases.

Theorem 7.

Let m be a prime power, $|{\mathcal{X}}_2|=q=ms$ and $G=K_m\bigsqcup \cdots \bigsqcup K_m$ be a disjoint union of s cliques. Then ${\mathcal{C}}_{ze}^{sum}\left([{\overline{K}}_q,G;{\overline{K}}_2,\dots ,{\overline{K}}_2]\right)=log(q+s)$.

Proof. 

First by Theorem 6, we have an upper bound on the sum-capacity given by

$${\mathcal{C}}_{ze}^{sum}\left([{\overline{K}}_q,G;{\overline{K}}_2,\dots ,{\overline{K}}_2]\right)\le log\left(|{\mathcal{X}}_2|+2^{\mathsf{\Theta }(G)}\right)=log(q+s).$$

(61)

Next, we consider the lower bound. Notice that $G=K_m⊠{\overline{K}}_s$. We can reformulate the channel accordingly as:

$$[{\overline{K}}_q,G;{\overline{K}}_2,\dots ,{\overline{K}}_2]=[{\overline{K}}_m,K_m;{\overline{K}}_2,\dots ,{\overline{K}}_2]⊠[{\overline{K}}_s;K_1,\dots ,K_1],$$

where the first $[{\overline{K}}_m,K_m;{\overline{K}}_2,\dots ,{\overline{K}}_2]$ corresponds to a channel with input alphabets ${\mathcal{X}}_1^{(1)}=\{1,2\}$ and ${\mathcal{X}}_2^{(1)}=\{1,\dots ,m\}$; and the second $[{\overline{K}}_s;K_1,\dots ,K_1]$ is with input alphabets ${\mathcal{X}}_1^{(2)}=\{1\}$ and ${\mathcal{X}}_2^{(2)}=\{1,\dots ,s\}$. Together with Lemma 1, we have:

$${\mathcal{C}}_{ze}^{sum}\left([{\overline{K}}_q,G;{\overline{K}}_2,\dots ,{\overline{K}}_2]\right)\ge {\mathcal{C}}_{ze}^{sum}\left([{\overline{K}}_m,K_m;{\overline{K}}_2,\dots ,{\overline{K}}_2]\right)+{\mathcal{C}}_{ze}^{sum}\left([{\overline{K}}_s;K_1,\dots ,K_1]\right).$$

(62)

On the one hand, it is easy to see that:

$${\mathcal{C}}_{ze}^{sum}\left([{\overline{K}}_s;K_1,\dots ,K_1]\right)=logs$$

(63)

since this is a clean channel and Alice and Bob could always communicate without error. On the other hand, by Lemma 10, we obtain:

$${\mathcal{C}}_{ze}^{sum}\left([{\overline{K}}_m,K_m;{\overline{K}}_2,\dots ,{\overline{K}}_2]\right)\ge log(m+1).$$

(64)

In fact, letting $q=m$, $q^{\prime }=2$ and $\tau =1$ in Lemma 10, we have a pair $(\mathcal{C},\mathrm{Y}(\mathcal{C}))$ satisfying that $\mathcal{C}$ is an m-ary $[n,k]$ linear code and $\mathrm{Y}(\mathcal{C})\subseteq {\mathbb{F}}_2^n$ such that

$$|\mathrm{Y}(\mathcal{C})|\ge \left(\genfrac{}{}{0pt}{}{n}{k}\right)\prod _{i=1}^{\infty }(1-m^{-i}).$$

(65)

Now let $\mathcal{A}=\mathrm{Y}(\mathcal{C})$ and $\mathcal{B}=\mathcal{C}$, then it is easy to see that $(\mathcal{A},\mathcal{B})$ is a uniquely decodable codebook pair with respect to the channel $[{\overline{K}}_m,K_m;{\overline{K}}_2,\dots ,{\overline{K}}_2]$. The corresponding sum-rate is

$$\begin{array}{cc}\hfill \underset{n\to \infty }{lim}\frac{1}{n}log|\mathcal{A}||\mathcal{B}|& =\underset{n\to \infty }{lim}\frac{1}{n}\left(log{m}^k+log\left(\genfrac{}{}{0pt}{}{n}{k}\right)\prod _{i=1}^{\infty }(1-m^{-i})\right)\hfill \\ \hfill & =\underset{n\to \infty }{lim}\frac{k}{n}logm+h\left(\frac{k}{n}\right).\hfill \end{array}$$

Taking $k/n=m/(m+1)$, we obtain a lower bound $log(m+1)$ on the best possible sum-rate, that is, (64).

Combining (62)–(64), we have ${\mathcal{C}}_{ze}^{sum}\left([{\overline{K}}_q,G;{\overline{K}}_2,\dots ,{\overline{K}}_2]\right)\ge log(m+1)+logs=log(q+s)$, which also implies an explicit uniquely decodable codebook pair for the channel $[{\overline{K}}_q,G;{\overline{K}}_2,\dots ,{\overline{K}}_2]$ based on the argument of Lemma 1. Then, together with (61), the proof is complete. □

6. Concluding Remarks

In this paper, we investigated the non-adaptive zero-error capacity region of the DM-TWC and provided several single-letter inner and outer bounds, some of which coincide in certain special cases. Determining the exact zero-error capacity region of a general DM-TWC remains an open problem, and clearly a difficult one, since it includes the notorious Shannon capacity of a graph as a special case. Despite this inherent difficulty, the problem is richer than the graph capacity setting, and we believe it deserves further study in order to obtain tighter bounds and smarter constructions.

One appealing direction is to extend the Lovász’s semi-definite relaxation approach in order to obtain tighter outer bounds, mimicking the graph capacity case. This, however, does not seem to be a simple task. In particular, one may ask whether the natural quantity $\rho (\{G_i\},\{H_j\})$ defined in (2), which upper-bounds the one-shot zero-error sum-capacity, is sub-multiplicative with respect to the graph strong product, in which case it would also serve as an upper bound for the zero-error sum-capacity. This is however not evident, in part since the problem (2) is not a semi-definite program. We have also considered other variations of the program (2). In particular, we have attempted to modify the non-linear constraints $⟨E_{i,j},\mathsf{\Gamma }⟩⟨E_{i,m},\mathsf{\Gamma }⟩=0$ to be of a linear form $⟨A,\mathsf{\Gamma }⟩=0$ for some suitable symmetric matrix A. We have also looked at some variants of the orthonormal representation. For example, we considered the case where each graph vertex a is labelled by a unit vector $v_a$, and if two vertices a and $a^{\prime }$ are nonadjacent $a\stackrel{b}{\nsim }{a}^{\prime }$ if and only if $b\in F$ for some set F, then the vector projections of $v_a$ and $v_{a^{\prime }}$ onto the subspace spanned by the vectors in F are orthogonal. However, proving sub-multiplicativity in any of these settings has so far resisted our best efforts.

It would be also of much interest to consider the adaptive zero-error capacity of the DM-TWC. Allowing Alice and Bob to adapt their transmissions on the fly can in general enlarge the zero-error capacity region. As a simple example, note that a point-to-point channel with noiseless feedback is a special case of the DM-TWC (where Bob has no information to send). In [2], Shannon explicitly derived the zero-error capacity with feedback for the point-to-point channel, and pointed out that for the channel corresponding to Pentagon graph this capacity is given by $log(5/2)\approx 1.32$. This is strictly larger than the zero-error capacity without feedback $(log5)/2\approx 1.16$, which can be thought of in this case as the non-adaptive zero-error capacity of the channel. Exploring the differences between the adaptive and non-adaptive zero-error capacity regions of a general DM-TWC remains a challenging future work.