The Computability of the Channel Reliability Function and Related Bounds

Abstract

The channel reliability function is a crucial tool for characterizing the dependable transmission of messages across communication channels. In many cases, the only upper and lower bounds of this function are known. We investigate the computability of the reliability function and its associated functions, demonstrating that the reliability function is not Turing computable. This also holds true for functions related to the sphere packing bound and the expurgation bound. Additionally, we examine the $R_{\infty }$ function and zero-error feedback capacity, as they are vital in the context of the reliability function. Both the $R_{\infty }$ function and the zero-error feedback capacity are not Banach–Mazur computable.

1. Introduction

In [1], C. Shannon established the foundations of information theory by characterizing the key mathematical properties of communication channels. For a transmission rate R that is less than the channel capacity C, the probability of erroneous decoding with respect to an optimal code decreases exponentially as the code length $n\in \mathbb{N}$ increases. Shannon introduced the channel reliability function $E\left(R\right)$ as being the exponent governing this exponential decrease in relation to the transmission rate R.

A major goal in information theory is to find a closed-form expression for the channel reliability function. This expression should be computable and fully determined by the parameters of the communication task. Naturally, we must define what constitutes a closed-form expression. In [2], Chow, and in [3], Borwein and Crandall discuss different approaches to defining closed-form expressions. All of the various representations satisfy the requirement that the corresponding functions can be computed algorithmically using a digital computer. This can be achieved with great precision, depending on the inputs within their domain of definition.

Shannon’s characterization of the capacity for message transmission via the discrete memoryless channel (DMC) in [1], Ahlswede’s characterization of the capacity for message transmission via the multiple access channel in [4], and Ahlswede and Dueck’s characterization of the identification capacity for DMCs in [5] are all significant examples of closed-form solutions using elementary functions. These provide important instances of the computability of the corresponding performance functions, as defined in the previous context. The precise definition of computability, as outlined by Turing, is presented in Section 2.

Lovász’s characterization of the zero-error capacity for the pentagram also represents a closed-form number according to Chow’s definition in [2], which can be computed algorithmically—an outcome that is desirable. However, the characterization of the zero-error capacity for a cyclical heptagon remains an open problem. Moreover, it is still unclear whether the zero-error capacities of DMCs can take computable values for computable channels. Additionally, the algorithmic computability of the broadcast capacity region is still uncertain.

In the age of artificial intelligence, it is increasingly important to determine whether a digital computer can solve a given problem or compute a given function. Since every function that can be computed by a digital computer can also be computed by a Turing machine (as will be discussed in more detail below), this question is reduced to asking whether a function is computable. It is therefore crucial to distinguish between determining how to compute the zero-error capacity and whether it is computable at all. In this work, we focus on the latter: the computability of the zero-error capacity.

The Lovász $\vartheta$-function for graphs was analyzed in [6] from three distinct research perspectives related to various graph invariants. This investigation resulted in new insights into the Shannon capacity of graphs, observations on cospectral and nonisomorphic graphs, and bounds on graph invariants while also serving as a tutorial in zero-error information theory and algebraic graph theory. Further observations on the Lovász $\vartheta$-function are provided by the author in [7].

In this paper, we provide a negative answer to the question of whether the channel reliability function and several related bounds are algorithmically computable by Turing machines.

Significant research has been conducted on the channel reliability function, but many aspects of its behavior remain unresolved (see surveys [8] and [9]). In fact, a complete characterization of the channel reliability function is still unknown for binary-input binary-output channels. As a result, considerable efforts have been made to derive computable lower and upper bounds for the function (see [10,11,12]).

Determining the behavior of the channel reliability function across the entire interval $(0,C)$ is a challenging problem. Various approaches have attempted to compute the reliability function algorithmically by constructing sequences of upper and lower bounds. The first significant contribution in this direction was made by Shannon, Gallager, and Berlekamp in [13].

A fundamental question that arises is whether the reliability function can be computed in this manner. To investigate this, we employ the framework of Turing computability [14]. In general, a function is considered Turing computable if there exists an algorithm capable of computing it. The Turing machine serves as the most fundamental and powerful model of computation, underpinning theoretical computer science. Unlike physical computers, which have practical constraints, a Turing machine is an abstract mathematical construct that can be rigorously analyzed using formal mathematical methods.

It is important to note that the Turing machine represents the ultimate performance limit of current digital computers, including supercomputers. A Turing machine models an algorithm or a program, where computation consists of step-by-step manipulation of symbols or characters that are read from and written to a memory tape according to a set of rules. These symbols can be interpreted in various ways, including as numbers. To perform computations on abstract sets, the elements of the set must be encoded as strings of symbols on the tape. This approach allows Turing computability to be defined for real and complex numbers.

The use of digital computers to compute approximations of channel capacities or channel reliability functions has been a prominent topic in information theory. The computation of channel capacity for discrete memoryless channels (DMCs) is a convex optimization problem, and in 1972, an algorithm for approximating the capacity of a DMC on digital computers was independently published in [15] and [16].

Even for binary symmetric channels with rational crossover probabilities (excluding the case $p={\displaystyle \frac{1}{2}}$), the channel capacity is a transcendental number. As a result, despite the relative simplicity of these channels, their capacity can only be approximated with finite precision by digital computers. In contrast to the problem of computing channel capacity, determining the behavior of the channel reliability function over the entire interval $(0,C)\subset \mathbb{R}$ is a significantly more complex task. A common approach to this challenge involves considering sequences of upper and lower bounds for $E\left(R\right)$ (see [13]).

In general, the channel reliability function is a well-studied topic in information theory. Originally introduced and analyzed for discrete memoryless channels (DMCs), the concept has since been significantly extended to various other scenarios and channel models. In [17], the reliability function of a DMC was studied at rates above capacity. Subsequent refinements and theoretical bounds were proposed, such as the Poor–Verdú upper bound addressed in [18]. Extensions beyond DMCs include continuous channels and channels with feedback or secrecy constraints. For instance, upper bounds for Gaussian channels were developed in [19], while the role of feedback in Poisson and Gaussian channels was explored in [20,21]. The impact of signal constraints was analyzed in [22], and improved Gaussian channel bounds were proposed in [23]. Secrecy considerations and cost constraints were incorporated into the analysis of the reliability and secrecy functions in [24]. The reliability function in the presence of side information, as in the Gelfand–Pinsker channel, was considered in [25]. More recently, a new upper bound for DMCs was given in [26], and noisy feedback for binary symmetric channels was studied in [27]. These developments culminated in the analysis of reliability functions in quantum communication settings. Foundational work includes [28,29], and recent advancements include [30].

In this work, we explore whether it is possible to compute the channel reliability function in this manner using a mathematically rigorous formalization of computability. Specifically, our analysis is based on the theory of Turing machines and recursive functions.

In many cases, there is no direct characterization of the behavior of a general function over an abstract set in terms of an algorithm on a Turing machine. Consequently, a common strategy is to approximate the function successively using a sequence of computable upper and lower bounds, for which an algorithm is available. One can then ask the weaker question of whether it is possible to approximate the function in a computable manner. This requires computable sequences of computable upper and lower bounds. This approach is also necessary for the reliability function, and we conducted this analysis. Unfortunately, our results show that the channel reliability function is not a Turing computable performance function when the channel is considered as input.

We also examine several other closely related functions, including the $R_{\infty }$ function, the sphere packing bound function, the expurgation bound function, and the zero-error feedback capacity, all of which are closely tied to the reliability function. We treat all of these functions as functions of the channel.

As envisioned, the sixth generation (6G) of mobile networks will introduce a wide range of new features [31]. These innovations bring new challenges to the design of wireless communication systems. Specifically, the Tactile Internet will enable not only the control of data but also the manipulation of physical and virtual objects [31]. With such applications, there arises an increased need to ensure the trustworthiness of the system and its services [32,33].

6G will impose more diverse and demanding quality-of-service (QoS) requirements on network resilience, reliability, service availability, and delay [31]. The channel reliability function plays a vital role in the reliability and delay performance analysis of communication systems. It is therefore of interest to explore whether the reliability and delay performance of communication systems can be verified automatically on digital hardware [33]. Analyzing the channel reliability function with respect to Turing computability becomes crucial in this context. The question of Turing computability for performance functions is a central issue in information theory, as closed-form expressions are only known for a few performance functions. It is therefore important to compute corresponding performance functions on available computers with provable performance, ensuring the strict requirements for future communication systems [31,33].

The structure of this paper is as follows. In Section 2, we begin by presenting the basic definitions and known results that will be used throughout the paper. Section 3 focuses on the $R_{\infty }$ function. We examine the decidability of connected sets with the $R_{\infty }$ function and demonstrate that only an approximation from below is possible. This has implications for the sphere packing bound, and we show that it is not a Turing computable performance function.

In Section 4, we analyze the reliability function and prove that it is also not Turing computable. The same result holds for the expurgation bound. In Section 5, we investigate the zero-error feedback capacity, which is closely related to the $R_{\infty }$ function. We first address a question posed by Alon and Lubetzky in [34] regarding the zero-error capacity with feedback, specifically for the case without feedback (which was examined in [35]). We then show that the zero-error feedback capacity is not Banach–Mazur computable and cannot be approximated by computable increasing sequences of computable functions. Additionally, we characterize the superadditivity of the zero-error feedback capacity and demonstrate that the $R_{\infty }$ function is additive.

In Section 6, we analyze the behavior of the expurgation bound rates. Finally, we conclude by summarizing the implications of our results for the channel reliability function. Our findings indicate that, in general, there cannot be a simple recursive closed-form expression for the channel reliability function over a very precise interval.

Some of the results in this paper were presented at the IEEE International Symposium on Information Theory in Espoo, as noted in [36].

2. Definitions and Basic Results

2.1. Basic Concepts of Computability Theory

In this section, we present the basic definitions and results from computability theory that are necessary for this work. We begin with the fundamental definitions of computability, starting with the concept of a Turing machine [14].

A Turing machine serves as a mathematical model for what we intuitively understand as computation machines. In this sense, they provide an abstract idealization of modern-day computers. Any algorithm that can be executed by a real-world computer can, in principle, be simulated by a Turing machine, and vice versa. However, unlike real-world computers, Turing machines are not constrained by limitations such as energy consumption, computation time, or memory size. Furthermore, all computation steps on a Turing machine are assumed to be executed flawlessly, with no possibility of error.

Recursive functions, more specifically known as μ-recursive functions, form a special subset of the set ${\bigcup }_{n=0}^{\infty }\left\{f:{\mathbb{N}}^n\hookrightarrow \mathbb{N}\right\}$, where the symbol “↪” denotes a partial mapping. The set of recursive functions provides an alternative characterization of the notion of computability. Turing machines and recursive functions are equivalent in the following sense: a function $f:{\mathbb{N}}^n\hookrightarrow \mathbb{N}$ is computable by a Turing machine if and only if it is a partial recursive function.

Next, we introduce several key definitions from computable analysis [37,38,39], which we will apply in the subsequent sections.

Definition 1.

A sequence of rational numbers ${\left\{r_n\right\}}_{n\in \mathbb{N}}$ is called a computable sequence if there exist recursive functions $a,b,s:\mathbb{N}\to \mathbb{N}$ with $b\left(n\right)\ne 0$ for all $n\in \mathbb{N}$ and

$$r_n={(-1)}^{s\left(n\right)}{\displaystyle \frac{a\left(n\right)}{b\left(n\right)}},n\in \mathbb{N}.$$

Definition 2.

We say that a computable sequence ${\left\{r_n\right\}}_{n\in \mathbb{N}}$ of rational numbers converges effectively, i.e., computably, to a number x, if a recursive function $a:\mathbb{N}\to \mathbb{N}$ exists such that $|x-r_n|<{\displaystyle \frac{1}{2^N}}$ for all $N\in \mathbb{N}$ and all $n\in \mathbb{N}$ with $n\ge a\left(N\right)$ applies.

We can now introduce computable numbers.

Definition 3.

A real number x is said to be computable if there exists a computable sequence of rational numbers ${\left\{r_n\right\}}_{n\in \mathbb{N}}$, such that $|x-r_n|<2^{-n}$ for all $n\in \mathbb{N}$. We denote the set of computable real numbers using ${\mathbb{R}}_c$.

Next, we need suitable subsets of the natural numbers.

Definition 4.

A set $A\subset \mathbb{N}$ is called recursive if there exists a recursive function f, such that $f\left(x\right)=1$ if $x\in A$ and $f\left(x\right)=0$ if $x\in A^c$, where $A^c$ stands for the complement set of A.

Definition 5.

A set $A\subset \mathbb{N}$ is recursively enumerable if there exists a recursive function whose domain is exactly A.

Remark 1.

For the definition of recursive and partial recursive functions, see [37]. Recursive functions $f:\mathbb{N}\to \mathbb{N}$ are the building blocks to develop the framework for computing theory on rational numbers, on real numbers, and on related functions defined over these number fields. This theory captures exactly what can be achieved in theory with digital computers in these number fields. We next introduce the concept of computable performance functions on the basis of computability theory. It is important to note that computability theory formalizes exactly what is computable with perfect digital computers.

2.2. Basic Concepts of Information Theory

To define the reliability function and its related functions, we first need the definition of a discrete memoryless channel. In the theory of transmission, the receiver must be in a position to successfully decode all the messages transmitted by the sender.

Let $\mathcal{X}$ be a finite alphabet, and denote the set of all probability distributions on $\mathcal{X}$ using $\mathcal{P}\left(\mathcal{X}\right)$. We define the set of computable probability distributions, ${\mathcal{P}}_c\left(\mathcal{X}\right)$, as the subset of $\mathcal{P}\left(\mathcal{X}\right)$ consisting of all distributions $P\in \mathcal{P}\left(\mathcal{X}\right)$ for which $P\left(x\right)\in {\mathbb{R}}_c$ holds for all $x\in \mathcal{X}$.

Furthermore, for finite alphabets $\mathcal{X}$ and $\mathcal{Y}$, let $\mathcal{CH}$ denote the set of all conditional probability distributions (or channels) $P_{Y|X}:\mathcal{X}\to \mathcal{P}\left(\mathcal{Y}\right)$. We define ${\mathcal{CH}}_c$ as the set of computable conditional probability distributions, i.e., those for which $P_{Y|X}(·|x)\in {\mathcal{P}}_c\left(\mathcal{Y}\right)$ holds for every $x\in \mathcal{X}$.

Let $M\subset {\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})$. We call M semi-decidable if and only if there is a Turing machine $T{M}_M$ that either stops or computes forever, depending on whether $W\in M$ is true. This means $T{M}_M$ exactly accepts the elements of M, and for an input $W\in M^c={\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})\setminus M$, it computes forever.

Definition 6.

A discrete memoryless channel (DMC) is a triple $(\mathcal{X},\mathcal{Y},W)$, where $\mathcal{X}$ is the finite input alphabet, $\mathcal{Y}$ is the finite output alphabet, and $W\left(y\right|x)\in \mathcal{C}H(\mathcal{X},\mathcal{Y})$, with $x\in \mathcal{X}$, $y\in \mathcal{Y}$. The probability that a sequence $y^n\in {\mathcal{Y}}^n$ is received if $x^n\in {\mathcal{X}}^n$ was sent is defined by

$$W^n\left(y^n\right|x^n)=\prod _{j=1}^{n}W\left(y_j\right|x_j).$$

Definition 7.

A (deterministic) block code $\mathcal{C}\left(n\right)$ with rate R and block length n consists of

• A message set $\mathcal{M}=\{1,2,...,M\}$ with $M=2^{nR}\in \mathbb{N}$;
• An encoding function $e:\mathcal{M}\to {\mathcal{X}}^n$;
• aAdecoding function $d:{\mathcal{Y}}^n\to \mathcal{M}$.

We call such a code an $(R,n)$-code.

Definition 8.

Let $(\mathcal{X},\mathcal{Y},W)$ be a DMC. Using $\mathcal{C}\left(n\right)$, we denote a block code with the block length n and message set $\mathcal{M}$.

1.

The individual message probability of error is defined by the conditional probability of error, given that message $m\in \mathcal{M}$ is transmitted:

$$P_e(\mathcal{C}\left(n\right),W,m)=Pr\{d\left(Y^n\right)\ne m|X^n=e\left(m\right)\}.$$

2.

We define the average probability of error by

$$P_{e,\mathrm{av}}(\mathcal{C}\left(n\right),W)={\displaystyle \frac{1}{\left|\mathcal{M}\right|}}\sum _{m\in \mathcal{M}}{P}_e(\mathcal{C}\left(n\right),W,m).$$

$P_{e,\mathrm{av}}(W,R,n)$ denotes the minimum error probability $P_{e,\mathrm{av}}(\mathcal{C}\left(n\right),W)$ over all block codes $\mathcal{C}\left(n\right)$ of block length n and with message set $\mathcal{M}=2^{nR}$.

3.

We define the maximal probability of error by

$$P_{e,max}(\mathcal{C}\left(n\right),W)=\underset{m\in \mathcal{M}}{max}{P}_e(\mathcal{C}\left(n\right),W,m).$$

$P_{e,max}(W,R,n)$ denotes the minimum error probability $P_{e,max}(\mathcal{C}\left(n\right),W)$ over all block codes $\mathcal{C}\left(n\right)$ of block length n and with message set $\mathcal{M}=2^{nR}$.

4.

The Shannon capacity for a channel $W\in \mathcal{C}H(\mathcal{X},\mathcal{Y})$ is defined by

$$C\left(W\right):=sup\{R:\underset{n\to \infty }{lim}{P}_{e,max}(W,R,n)=0\}.$$

5.

The zero-error capacity for a channel $W\in \mathcal{C}H(\mathcal{X},\mathcal{Y})$ is defined by

$$C_0\left(W\right):=sup\{R:P_{e,max}(W,R,n)=0\mathit{for}\mathit{some}n\}.$$

Remark 2.

For R with $C_0\left(W\right)<R<C\left(W\right)$, there exists $A(W,R),B(W,R)\in {\mathbb{R}}^{+}$, such that

$$2^{-nA(W,R)+o\left(1\right)}\le P_{e,max}(W,R,n)\le 2^{-nB(W,R)+o\left(1\right)}.$$

We also define the discrete memoryless channel with noiseless feedback (DMCF). By this, we mean that, in addition to the DMC, there exists a return channel that sends the element of $\mathcal{Y}$ actually received back from the receiving point to the transmitting point. It is assumed that this information is received at the transmitting point before the next letter is sent and can therefore be used to choose the next letter to be sent. We assume that this feedback is noiseless. We denote the feedback capacity of a channel W by $C^{FB}\left(W\right)$ and the zero-error feedback capacity by $C_0^{FB}\left(W\right)$. Shannon proved in [40] that $C\left(W\right)=C^{FB}\left(W\right)$. This is, in general, not true for the zero-error capacity. We see that the zero-error (feedback) capacity is related to the reliability function, which we analyze in this paper. It is defined as follows.

Definition 9.

The channel reliability function (error exponent) is defined by

$$E(W,R)=\underset{n\to \infty }{lim\; sup}-{\displaystyle \frac{1}{n}}{log}_2{P}_{e,max}(W,R,n).$$

(1)

Remark 3.

We make use of the common convention that ${log}_{2}0:=-\infty$.

Remark 4.

We need the lim sup in (1), because it is not known whether the limit value, i.e., the limts on the right-hand side of (1), exist.

The first simple observation is that for $R>C\left(W\right)$, we have $E(W,R)=0$, and if $C_0\left(W\right)>0$ for $0\le R<C_0\left(W\right)$, we have $E(W,R)=+\infty$. One well-known upper bound is the sphere packing bound, which can be defined as follows (see [10]).

Definition 10.

Let $\mathcal{X},\mathcal{Y}$ be finite alphabets, and $(\mathcal{X},\mathcal{Y},W)$ be a DMC. Then, for all $R\in (0,C(W\left)\right)$, we define the sphere packing bound function:

$$E_{SP}(W,R)=\underset{\rho >0}{sup}\underset{P\in \mathcal{P}\left(\mathcal{X}\right)}{max}\left(-log\sum _y{\left(\sum _{x}P\left(x\right)W{\left(y\right|x)}^{{\displaystyle \frac{1}{1+\rho }}}\right)}^{1+\rho }-\rho R\right).$$

(2)

Theorem 1

(Fano 1961, Shannon, Gallager, Berlekamp 1967). For any DMC W and for all $R\in (0,C(W\left)\right)$, it holds that

$$E(W,R)\le E_{SP}(W,R).$$

The sphere packing upper bound is an important upper bound. The following two lower bounds of the reliability function are also very important. In [41], the random coding bound was defined as follows:

Definition 11.

Let $\mathcal{X},\mathcal{Y}$ be finite alphabets, and $(\mathcal{X},\mathcal{Y},W)$ be a DMC. Then, for all $R\in (0,C(W\left)\right)$, we define the random coding bound function as

$$\begin{array}{cc}\hfill E_{\mathrm{r}}(W,R)& =\underset{0\le \rho \le 1}{max}{E}_0(W,\rho )-\rho R,\mathrm{where}\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill E_0\left(\rho \right)& =\underset{P\in \mathcal{P}\left(\mathcal{X}\right)}{max}\left[-log\sum _y{\left(\sum _{x}P\left(x\right)W{\left(y\right|x)}^{1/(1+\rho )}\right)}^{1+\rho }\right].\hfill \end{array}$$

(4)

Theorem 2.

Let $\mathcal{X},\mathcal{Y}$ be finite alphabets and $(\mathcal{X},\mathcal{Y},W)$ be a DMC; then,

$$E(W,R)\ge E_r(W,R).$$

Gallager also defined in [41] the k-letter expurgation bound as follows:

Definition 12.

Let $\mathcal{X},\mathcal{Y}$ be finite alphabets and $(\mathcal{X},\mathcal{Y},W)$ be a DMC; then, for all $R\in (0,C(W\left)\right)$, we define the k-letter expurgation bound function:

$$\begin{array}{cc}\hfill E_{\mathrm{ex}}(W,R,k)& =\underset{\rho \ge 1}{sup}{E}_{\mathrm{x}}(\rho ,k)-\rho R\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill E_{\mathrm{x}}(\rho ,k)& =-{\displaystyle \frac{\rho }{k}}log\underset{P_{X^k}\in \mathcal{P}\left({\mathcal{X}}^k\right)}{min}{Q}^k(\rho ,P_{X^k})\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill Q^k(\rho ,P_{X^k})& =\sum _{x^k,x^{\prime k}}{P}_{X^k}\left(x^k\right)P_{X^k}\left(x^{\prime k}\right)g_k{(x^k,x^{\prime k})}^{{\displaystyle \frac{1}{\rho }}}\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill g_k(x^k,x^{\prime k})& =\sum _{y^k}\sqrt{W^k\left(y^k\right|x^k)W^k\left(y^k\right|x^{\prime k})}.\hfill \end{array}$$

(8)

Theorem 3.

Let $\mathcal{X},\mathcal{Y}$ be finite alphabets and $(\mathcal{X},\mathcal{Y},W)$ be a DMC. Then, for all $R\in (0,C(W\left)\right)$, we have

$$E(W,R)\ge \underset{k\to \infty }{lim}{E}_{\mathrm{ex}}(W,R,k).$$

(9)

The inequality in (9) follows from Fekete’s lemma.

The smallest value of R, at which the convex curve $E_{SP}(W,R)$ meets its supporting line of slope -1, is called the critical rate and is denoted by $R_{crit}$ [9]. For the certain interval $[R_{crit},C]$, the random coding lower bound corresponds to the sphere packing upper bound. The channel reliability function is therefore known for this interval. The channel reliability function is generally not known for the interval $[0,R_{crit}]$. For the interval $[0,R_{crit}]$, there are also better lower bounds than the random coding lower bound. $R_{\infty }\left(W\right)$ is the infimum of all rates $\underline{R}$ such that $E_{SP}(W,\underline{R})$ is finite on the open interval $(\underline{R},C\left(W\right))$. $C_0\left(W\right)\le R_{\infty }\left(W\right)$ applies if $C_0\left(W\right)>0$. The following representation of $R_{\infty }$ exists (see [9]):

$$R_{\infty }\left(W\right)=\underset{Q\in \mathcal{P}\left(\mathcal{Y}\right)}{min}\underset{x\in \mathcal{X}}{max}{log}_2{\displaystyle \frac{1}{{\sum }_{y:W\left(y\right|x)>0}Q\left(y\right)}}.$$

(10)

There exist alphabets $\mathcal{X},\mathcal{Y}$ and channels $W\in \mathcal{CH}$ such that $C_0\left(W\right)=0$ while $R_{\infty }\left(W\right)>0$.

Moreover, for the zero-error feedback capacity $C_0^{FB}$, it holds that $C_0^{FB}\left(W\right)=R_{\infty }\left(W\right)$ whenever $C_0\left(W\right)>0$. However, if $C_0\left(W\right)=0$, there exists a channel W for which $C_0^{FB}\left(W\right)=0$ while $R_{\infty }\left(W\right)>0$ (see [9]).

For the zero-error feedback capacity, the following is known.

Theorem 4

(Shannon 1956, [40]). Let $W\in \mathcal{C}H(\mathcal{X},\mathcal{Y})$; then,

$$C_0^{FB}\left(W\right)=\left\{\begin{array}{cc}0\hfill & \mathit{if}{C}_0\left(W\right)=0\hfill \\ \underset{P\in \mathcal{P}\left(\mathcal{X}\right)}{max}\underset{y}{min}{log}_2{\displaystyle \frac{1}{{\sum }_{x:W\left(y\right|x)>0}P\left(x\right)}}\hfill & \mathit{otherwise}.\hfill \end{array}\right.$$

(11)

2.3. Lower and Upper Bounds on the Reliability Function for the Typewriter Channel

As mentioned before, Shannon, Gallager, and Berlekamp assumed in [13] that the expurgation is bound tight. Katsman, Tsfasman, and Vladut showed in [42] a counterexample for the symmetric q-ary channel when $q\ge 49$. Dalai and Polyanskiy found a simpler counterexample in [43]. They showed that the conjecture is already wrong for the q-ary typewriter channel for $q\ge 4$. We would like to briefly present their results here.

Definition 13.

Let $\mathcal{X}=\mathcal{Y}={\mathbb{Z}}_q$ and $0\le ϵ\le {\displaystyle \frac{1}{2}}$. The typewriter channel $W_{ϵ}$ is defined by

$$W_{ϵ}\left(y\right|x)=\left\{\begin{array}{cc}1-ϵ\hfill & y=x\hfill \\ ϵ\hfill & y=x+1modq.\hfill \end{array}\right.$$

(12)

The extension of the channel $W_{ϵ}^n$ is defined by

$$W_{ϵ}^n\left(y^n\right|x^n)=\prod _{k=1}^n{W}_{ϵ}\left(y_i\right|x_i).$$

(13)

For the reliability function of this channel, the interval $(C_0\left(W_{ϵ}\right),C\left(W_{ϵ}\right))$ is of interest. The capacity of a typewriter channel $W_{ϵ}$ has the formula

$$C\left(W_{ϵ}\right)=log\left(q\right)-h_2\left(ϵ\right),$$

where $h_2$ is the binary entropy function. Shannon showed in [40] that $C_0\left(W_{ϵ}\right)$ is positive if $q\ge 4$. He showed that for even q, it holds that $C_0\left(W_{ϵ}\right)=log\left({\displaystyle \frac{q}{2}}\right)$. It is difficult to get a formula for odd q. Lovász proved in [44] that Shannon’s lower bound for $q=5$: $C_0\left(W_{ϵ}\right)=log\sqrt{5}$ is tight. For general odd q, Lovász proved

$$C_0\left(W_{ϵ}\right)\le log{\displaystyle \frac{cos\left(\pi q\right)}{1+cos\left(\pi q\right)}}q.$$

It is only known for $q=5$ that this bound is tight. In general, this is not true. For special q, there are special results outlined in [44,45,46,47].

Dalai and Polyanskiy provide upper and lower bounds on the reliability function in [43]. They observed that the zero-error capacity of the pentagon can be determined by a careful study of the expurgated bound.

They present an improved lower bound for the case of even and odd q, showing that it also is a precisely shifted version of the expurgated bound for the BSC. Their result also provides a new elementary disproof of the conjecture suggested in [13] that the expurgated bound is asymptotically tight when computed on arbitrarily large blocks. Furthermore, in [43], Dalai and Polyanskiy present a new upper bound for the case of odd q based on the minimum distance of codes. They use Delsarte’s linear programming method [48] (see also [49]), combining the construction used by Lovász [44] for bounding the graph capacity with the construction used by McEliece–Rodemich–Rumsey–Welch [50] for bounding the minimum distance of codes in Hamming spaces. In the special case $ϵ=1/2$, they give another improved upper bound for the case of odd q, following the ideas of Litsyn [51] and Barg–McGregor [52], which in turn are based on estimates for the spectra of codes originated by Kalai–Linial [53].

2.4. Computable Channels and Computable Performance Functions

We need further basic concepts for computability. We want to investigate the function $E(W,R)$ and the upper bounds like $E_{SP}(W,R)$ and $E_{ex}(W,R)$ for $k\in \mathbb{N}$ as functions of W and R. These functions are generally only well defined for fixed channels W on sub-intervals of $[0,C(W\left)\right]$ as functions depending on R. For example, for $W\in \mathcal{C}H(\mathcal{X},\mathcal{Y})$ with $C_0\left(W\right)>0$, $E(W,R)$ is infinite for $R<C_0\left(W\right)$. Hence, $E(W,R)$ must be examined and computed as a function of R on the interval $(C_0\left(W\right),C\left(W\right)]$. Similar statements also apply to the other functions that have already been introduced. We now fix non-trivial alphabets $\mathcal{X},\mathcal{Y}$ and the corresponding set ${\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})$ of the computable channels and $R\in {\mathbb{R}}_c$.

Definition 14

(Turing computable channel function). We call a function $f:{\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})\to {\mathbb{R}}_c$ a Turing computable channel function if there is a Turing machine that converts any program for the representation of $W\in {\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})$ into a program for the computation of $f\left(W\right)$—that is, $f\left(W\right)=T{M}_f\left(W\right)$, $W\in {\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})$.

We want to determine whether there is a closed form for the channel reliability function. For this, we need the following definition, which we discuss in more detail in Remark 5 below.

Definition 15

(Turing computable performance function). Let ⊥ be a symbol. We call a function $F:\mathcal{C}H{(\mathcal{X},\mathcal{Y})}_c\times {\mathbb{R}}_c^{+}\to {\mathbb{R}}_c\cup \{\perp \}$ a Turing computable performance function if there are two Turing computable channel functions $\underline{f}$ and $\overline{f}$ with $\underline{f}\left(W\right)\le \overline{f}\left(W\right)$ for $W\in {\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})$, and a Turing machine $T{M}_F$, which is defined for input $R\in {\mathbb{R}}_c^{+}$ and $W\in {\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})$. The Turing machine $T{M}_F$ stops for the variables $R\in {\mathbb{R}}_c^{+}$ and $W\in {\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})$ and any representation for W and R as input if and only if $R\in (\underline{f}\left(W\right), \overline{f}\left(W\right))$ and the Turing machine $T{M}_F$ delivers $F(W,R)=T{M}_F(W,R)$. If $R\notin (\underline{f}\left(W\right), \overline{f}\left(W\right))$, then $T{M}_F$ does not stop.

Remark 5.

The requirement for function $F:\mathcal{C}H{(\mathcal{X},\mathcal{Y})}_c\times {\mathbb{R}}_c^{+}\to {\mathbb{R}}_c\cup \{\perp \}$ to be a Turing computable performance function is relatively weak. For example, let us take W and R as inputs. Then, the interval $(\underline{f}\left(W\right), \overline{f}\left(W\right))$ is computed first. If R is now in the interval $((\underline{f}\left(W\right), \overline{f}\left(W\right))$, then the Turing machine $T{M}_F$ must stop for the input $(W,R)$ and deliver the result for $F(W,R)$. We impose no requirements on the behavior of the Turing machine for input W and $R\notin (\underline{f}\left(W\right), \overline{f}\left(W\right))$. In particular, the Turing machine $T{M}_F$ does not have to stop for the input $(W,R)$ in this case.

Take, for example, any Turing computable function $G:\mathcal{C}H{(\mathcal{X},\mathcal{Y})}_c\times {\mathbb{R}}_c^{+}\to {\mathbb{R}}_c\cup \{\perp \}$ with the corresponding Turing machine $T{M}_G$. Furthermore, let $\underline{TM}:{\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})\to {\mathbb{R}}_c$ and $\overline{TM}:{\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})\to {\mathbb{R}}_c$ be any two TMs, so that $\underline{TM}\left(W\right)\le \overline{TM}\left(W\right)$ always holds for all $W\in {\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})$. Then, the following Turing machine $TM:{\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})\times {\mathbb{R}}_c\to {\mathbb{R}}_c\cup \{\perp \}$ defines a Turing computable performance function.

1.

For any input $W\in {\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})$ and $R\in {\mathbb{R}}_c$, first compute $\underline{f}\left(W\right)=\underline{TM}\left(W\right)$ and $\overline{f}\left(W\right)=\overline{TM}\left(W\right)$.

2.

Compute the following two tests in parallel:

(a) 

Use the Turing machine $T{M}_{>\underline{f}\left(W\right)}$ and test $R>\underline{f}\left(W\right)$ using $T{M}_{>\underline{f}\left(W\right)}$ for input $R\in {\mathbb{R}}_c$.

(b) 

Use the Turing machine $T{M}_{<\overline{f}\left(W\right)}$ and test $R<\overline{f}\left(W\right)$ using $T{M}_{<\overline{f}\left(W\right)}$ for input $R\in {\mathbb{R}}_c$.

Let these two tests run until both Turing machines stop. If both Turing machines stop in 2, then compute $G(W,R)$ and set $TM(W,R)=G(W,R)$.

$TM$ actually generates a Turing computable performance function, and the Turing machine $TM$ stops for the input $(W,R)$ if and only if $R\in (\underline{f}\left(W\right), \overline{f}\left(W\right))$ applies. Then, it gives the value $G(W,R)$ as output. This follows from the fact that the Turing machine $T{M}_{>\underline{f}\left(W\right)}$ stops for input $R\in {\mathbb{R}}_c$ if and only if $R>\underline{f}\left(W\right)$. The second Turing machine $T{M}_{<\overline{f}\left(W\right)}$ from 2 stops exactly when $R<\overline{f}\left(W\right)$, i.e. the Turing machine $TM$ in 2., which simulates $T{M}_{>\underline{f}\left(W\right)}$ and $T{M}_{<\overline{f}\left(W\right)}$ in parallel, stops exactly when $R\in (\underline{f}\left(W\right), \overline{f}\left(W\right))$ applies.

Remark 6.

Using the above approach, we can try, for example, to find upper and lower bounds for the channel reliability function by allowing general Turing computable functions $G:\mathcal{C}H{(\mathcal{X},\mathcal{Y})}_c\times {\mathbb{R}}_c^{+}\to {\mathbb{R}}_c\cup \{\perp \}$ and algorithmically determine the interval from ${\mathbb{R}}_c^{+}$ for which the function $G(W,·)$ delivers lower or upper bounds for the channel reliability function.

Definition 16

(Banach–Mazur computable channel function). We call $f:{\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})\to {\mathbb{R}}_c$ a Banach–Mazur computable channel function if every computable sequence ${\left\{W_r\right\}}_{r\in \mathbb{N}}$ from ${\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})$ is mapped by f into a computable sequence from ${\mathbb{R}}_c$.

For practical applications, it is necessary to have performance functions that satisfy Turing computability. Depending on W, the channel reliability function or the bounds for this function should be computed. This computation is carried out by an algorithm that also receives W as input. This means that the algorithm should also be recursively dependent on W; otherwise, a special algorithm would have to be developed for each W (depending on W but not recursively dependent), since the channel reliability function for this channel, or a bound for this function, is computed.

It is now clear that when defining the Turing computable performance function, the Turing computable channel functions $\underline{f}, \overline{f}$ cannot be dispensed with, because the channel reliability function depends on the specific channel and the permissible rate region for which the function can be computed. For $\overline{f}$, one often has the representation $\overline{f}\left(W\right)=C\left(W\right)$ with $W\in {\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})$. For $\underline{f}$, the choice $\underline{f}\left(W\right)=C_0\left(W\right)$ with $W\in {\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})$ for the channel reliability function is a natural choice, because the channel reliability function is only useful for this interval. (We note that we showed in [35] that $C_0\left(W\right)$ is not Turing computable in general.)

For the Turing computability of the channel reliability function or corresponding upper and lower bounds, it is therefore a necessary condition that the dependency of the relevant rate intervals on W be Turing computable—that is, recursive.

Remark 7.

As noted in the Introduction, very few closed-form expressions for performance functions are known in information theory. Even for relatively simple scenarios, such as secure message transmission over a wiretap channel with an active jammer, closed-form solutions are not available (see [54,55,56]). Existing methods in information theory provide convergent multi-letter sequences for determining capacity. While these sequences enable the investigation of important properties of the capacity (see [54,57,58]), they are not yet suitable for direct numerical computation of the capacity. This is due to the reliance on Fekete’s lemma to prove the existence of the limit of these sequences. However, it was shown in [59] that Fekete’s lemma is not constructive, meaning no algorithm can effectively compute the associated limit values.

Moreover, the problem of finding simple optimizers for performance functions is generally not algorithmically solvable [60,61]. For instance, the Blahut–Arimoto algorithm can be used to compute an infinite sequence of input distributions that converge to an optimal distribution. However, there is no way to halt the process based on a reliable approximation error, making it impossible to stop the computation at a specific point (see [60,61]).

3. Results for the Rate Function $R_{\infty }$ and Applications on the Sphere Packing Bound

In this section, we analyze the function $R_{\infty }$ and its implications for the sphere packing bound. Specifically, we demonstrate that $R_{\infty }$ is not a Turing computable performance function.

We begin by expressing $R_{\infty }\left(W\right)$ as

$$R_{\infty }\left(W\right)=\underset{Q\in \mathcal{P}\left(\mathcal{Y}\right)}{min}\underset{x\in \mathcal{X}}{max}{log}_2{\displaystyle \frac{1}{{\sum }_{y:W\left(y\right|x)>0}Q\left(y\right)}}.$$

(14)

From this, we derive the equivalent representations:

$$\begin{array}{ccc}\hfill R_{\infty }\left(W\right)& =& \underset{Q\in \mathcal{P}\left(\mathcal{Y}\right)}{min}\underset{x\in \mathcal{X}}{max}{log}_2{\displaystyle \frac{1}{{\sum }_{y:W\left(y\right|x)>0}Q\left(y\right)}}\hfill \\ & =& \underset{Q\in \mathcal{P}\left(\mathcal{Y}\right)}{min}{log}_2{\displaystyle \frac{1}{{min}_{x\in \mathcal{X}}{\sum }_{y:W\left(y\right|x)>0}Q\left(y\right)}}\hfill \\ & =& {log}_2\underset{Q\in \mathcal{P}\left(\mathcal{Y}\right)}{min}{\displaystyle \frac{1}{{min}_{x\in \mathcal{X}}{\sum }_{y:W\left(y\right|x)>0}Q\left(y\right)}}\hfill \\ & =& {log}_2{\displaystyle \frac{1}{{max}_{Q\in \mathcal{P}\left(\mathcal{Y}\right)}{min}_{x\in \mathcal{X}}{\sum }_{y:W\left(y\right|x)>0}Q\left(y\right)}}\hfill \\ & =& {log}_2{\displaystyle \frac{1}{{\mathrm{\Psi }}_{\infty }\left(W\right)}},\hfill \end{array}$$

where

$$\begin{array}{c}\hfill {\mathrm{\Psi }}_{\infty }\left(W\right)=\underset{Q\in \mathcal{P}\left(\mathcal{Y}\right)}{max}\underset{x\in \mathcal{X}}{min}\sum _{y:W\left(y\right|x)>0}Q\left(y\right).\end{array}$$

(15)

In summary, the following holds true: let $\mathcal{X},\mathcal{Y}$ be arbitrary non-trivial finite alphabets; then, for $W\in {\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})$

$$R_{\infty }\left(W\right)={log}_2{\displaystyle \frac{1}{{\mathrm{\Psi }}_{\infty }\left(W\right)}}.$$

(16)

Lemma 1.

It holds that

$$R_{\infty }:{\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})\to {\mathbb{R}}_c.$$

Proof. 

Let W be fixed. We consider the vector ${\left(\begin{array}{ccc}Q\left(1\right)& \dots & Q\left(\right|\mathcal{Y}\left|\right)\end{array}\right)}^{\mathrm{T}}$ of the convex set

$${\mathcal{M}}_{Prob}=\{u\in {\mathbb{R}}^{\left|\mathcal{Y}\right|}:u=\left(\begin{array}{c}{u}_1\\ ⋮\\ u_{\left|\mathcal{Y}\right|}\end{array}\right),u_l\ge 0,l=1,\dots ,\left|\mathcal{Y}\right|,\sum _l{u}_l=1\}.$$

$G\left(u\right):={min}_x{\sum }_{y:W\left(y\right|x)>0}{u}_y$ is a computable continuous function on ${\mathcal{M}}_{Prob}$. Thus, for ${\mathrm{\Psi }}_{\infty }\left(W\right)={max}_{u\in {\mathcal{M}}_{Prob}}G\left(u\right)$, we always have ${\mathrm{\Psi }}_{\infty }\left(W\right)\in {\mathbb{R}}_c$ with ${\mathrm{\Psi }}_{\infty }\left(W\right)>0$, and thus $R_{\infty }\left(W\right)\in {\mathbb{R}}_c$. □

Remark 8.

We do not know whether $C_0:{\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})\to {\mathbb{R}}_c$ holds for any finite $\mathcal{X},\mathcal{Y}$. This statement holds for $max\left\{\right|\mathcal{X}|,|\mathcal{Y}\left|\right\}\le 5$, but the general case is open.

For finite alphabets $\mathcal{X},\mathcal{Y}$ and $\lambda \in {\mathbb{R}}_c$ with $\lambda >0$, we want to analyze the set

$$\{W\in {\mathcal{C}H}_c(\mathcal{X},\mathcal{Y}):R_{\infty }\left(W\right)>\lambda \}.$$

To accomplish this, we refer to the proof of Theorem 23 in [35]. Along the same lines, one can show that the following holds true:

Theorem 5.

Let $\mathcal{X},\mathcal{Y}$ be non-trivial finite alphabets. For all $\lambda \in {\mathbb{R}}_c$ with $0<\lambda <{log}_2(min\{\left|\mathcal{X}\right|,\left|\mathcal{Y}\right|\left\}\right)$, the set

$$\{W\in {\mathcal{C}H}_c(\mathcal{X},\mathcal{Y}):R_{\infty }\left(W\right)>\lambda \}$$

is not semi-decidable.

The following theorem can be derived from a combination of the proof of Theorem 5 and Theorem 24 in [35]. The proof is carried out in the same way as the proof of Theorem 24 in [35].

Theorem 6.

Let $\mathcal{X},\mathcal{Y}$ be non-trivial finite alphabets. The function $R_{\infty }:{\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})\to \mathbb{R}$ is not Banach–Mazur computable.

We now prove a stronger result then what we were able to show for $C_0$ in [35] so far. We show that the analogous question, like the question in [34] for $C_0$ for the function $R_{\infty }$, can be answered positively.

We need a concept of distance for $W_1,W_2\in \mathcal{C}H(\mathcal{X},\mathcal{Y})$. Therefore, for fixed and finite alphabets $\mathcal{X},\mathcal{Y}$, we define the distance between $W_1$ and $W_2$ based on the total variation distance

$$d_C(W_1,W_2)=\underset{x\in \mathcal{X}}{max}\sum _{y\in \mathcal{Y}}|W_1\left(y\right|x)-W_2\left(y\right|x)|.$$

(17)

Definition 17.

A function $f:\mathcal{C}H(\mathcal{X},\mathcal{Y})\to \mathbb{R}$ is called computable continuously if the following are true:

1.

f is sequentially computable, i.e., f maps every computable sequence ${\left\{W_n\right\}}_{n\in \mathbb{N}}$ with $W_n\in {\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})$ into a computable sequence ${\left\{f\left(W_n\right)\right\}}_{n\in \mathbb{N}}$ of computable numbers,

2.

f is effectively uniformly continuous, i.e., there is a recursive function $d:\mathbb{N}\to \mathbb{N}$ such that for all $W_1,W_2\in {\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})$ and all $N\in \mathbb{N}$ with $d_C(W_1,W_2)\le {\displaystyle \frac{1}{d\left(N\right)}}$, it holds that $|f\left(W_1\right)-f\left(W_2\right)|\le {\displaystyle \frac{1}{2^N}}.$

Theorem 7.

Let $\mathcal{X},\mathcal{Y}$ be finite alphabets with $\left|\mathcal{X}\right|\ge 2$ and $\left|\mathcal{Y}\right|\ge 2$. There exists a computable sequence of computable continuous functions ${\left\{F_N\right\}}_{N\in \mathbb{N}}$ on ${\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})$ with

1.

$F_N\left(W\right)\ge F_{N+1}\left(W\right)$ with $W\in \mathcal{C}H(\mathcal{X},\mathcal{Y})$ and $N\in \mathbb{N}$,

2.

${lim}_{N\to \infty }{F}_N\left(W\right)=R_{\infty }\left(W\right)$ for all $W\in \mathcal{C}H(\mathcal{X},\mathcal{Y})$.

Proof. 

We consider the function

$${\mathrm{\Phi }}_N\left(W\right)=\underset{Q\in \mathcal{P}\left(\mathcal{Y}\right)}{max}\underset{x\in \mathcal{X}}{min}\sum _{y\in \mathcal{Y}}{\displaystyle \frac{NW\left(y\right|x)}{1+NW\left(y\right|x)}}Q\left(y\right)$$

for $N\in \mathbb{N}$. For all $x\in \mathcal{X}$ we have for all $Q\in \mathcal{P}\left(\mathcal{Y}\right)$

$$\sum _{y\in \mathcal{Y}}{\displaystyle \frac{NW\left(y\right|x)}{1+NW\left(y\right|x)}}Q\left(y\right)\le \sum _{y\in \mathcal{Y}:W\left(y\right|x)>0}Q\left(y\right),$$

(18)

and for all $N\in \mathbb{N}$, we have for all $x\in \mathcal{X}$ and $Q\in \mathcal{P}\left(\mathcal{Y}\right)$

$$\sum _{y\in \mathcal{Y}}{\displaystyle \frac{NW\left(y\right|x)}{1+NW(y<x)}}Q\left(y\right)\le \sum _{y\in \mathcal{Y}:W\left(y\right|x)>0}{\displaystyle \frac{(N+1)W\left(y\right|x)}{1+(N+1)W\left(y\right|x)}}Q\left(y\right).$$

(19)

${\mathrm{\Phi }}_N$ is a computable continuous function, and ${\left\{{\mathrm{\Phi }}_N\right\}}_{N\in \mathbb{N}}$ is a computable sequence of computable continuous functions. So,

$$F_N\left(W\right)={log}_2{\displaystyle \frac{a}{{\mathrm{\Phi }}_N\left(W\right)}},$$

for $N\in \mathbb{N}$ and $W\in \mathcal{C}H(\mathcal{X},\mathcal{Y})$. $F_N$ satisfies all properties of the theorem, and point 1 is shown.

It holds

$$\begin{array}{ccc}& & \left|\sum _{y\in \mathcal{Y}:W\left(y\right|x)>0}Q\left(y\right)-\sum _{y\in \mathcal{Y}}{\displaystyle \frac{NW\left(y\right|x)}{1+NW\left(y\right|x)}}Q\left(y\right)\right|\hfill \\ & =& \left|\sum _{y\in \mathcal{Y}:W\left(y\right|x)>0}{\displaystyle \frac{1}{1+NW\left(y\right|x)}}Q\left(y\right)\right|\hfill \\ & \le & {\displaystyle \frac{1}{1+N{min}_{y\in \mathcal{Y}:W\left(y\right|x)>0}W\left(y\right|x)}}.\hfill \end{array}$$

Therefore, we have

$$\begin{array}{ccc}\hfill \sum _{y\in \mathcal{Y}:W\left(y\right|x)>0}Q\left(y\right)& \le & {\displaystyle \frac{1}{1+N{min}_{y\in \mathcal{Y}:W\left(y\right|x)>0}W\left(y\right|x)}}\hfill \\ & & +\sum _{y\in \mathcal{Y}}{\displaystyle \frac{NW\left(y\right|x)}{1+NW\left(y\right|x)}}Q\left(y\right).\hfill \end{array}$$

(20)

Because of (18), we have

$${\mathrm{\Phi }}_N\left(W\right)\le {\mathrm{\Psi }}_{\infty }\left(W\right)$$

for all $W\in {\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})$. (20) yields

$$\begin{array}{ccc}\hfill \sum _{y\in \mathcal{Y}:W\left(y\right|x)>0}Q\left(y\right)& \le & {\displaystyle \frac{1}{1+N{min}_{x\in \mathcal{X}}\left({min}_{y\in \mathcal{Y}:W\left(y\right|x)>0}W\left(y\right|x)\right)}}\hfill \\ & & +\sum _{y\in \mathcal{Y}}{\displaystyle \frac{NW\left(y\right|x)}{1+NW\left(y\right|x)}}Q\left(y\right).\hfill \end{array}$$

So,

$$\begin{array}{ccc}\hfill \underset{x\in \mathcal{X}}{min}\sum _{y\in \mathcal{Y}:W\left(y\right|x)>0}Q\left(y\right)& \le & {\displaystyle \frac{1}{1+N{min}_{x\in \mathcal{X}}\left({min}_{y\in \mathcal{Y}:W\left(y\right|x)>0}W\left(y\right|x)\right)}}\hfill \\ & & +\underset{x\in \mathcal{X}}{min}\sum _{y\in \mathcal{Y}}{\displaystyle \frac{NW\left(y\right|x)}{1+NW\left(y\right|x)}}Q\left(y\right)\hfill \end{array}$$

and

$$\begin{array}{ccc}\hfill {\mathrm{\Psi }}_{\infty }\left(W\right)& \le & {\displaystyle \frac{1}{1+N{min}_{x\in \mathcal{X}}\left({min}_{y\in \mathcal{Y}:W\left(y\right|x)>0}W\left(y\right|x)\right)}}+{\mathrm{\Phi }}_N\left(W\right)\hfill \end{array}$$

holds. So, we have

$$0\le {\mathrm{\Psi }}_{\infty }\left(W\right)-{\mathrm{\Phi }}_N\left(W\right)\le {\displaystyle \frac{1}{1+N{min}_{x\in \mathcal{X}}\left({min}_{y\in \mathcal{Y}:W\left(y\right|x)>0}W\left(y\right|x)\right)}}.$$

 □

We now want to prove that the corresponding question in [34] can be answered positively for $R_{\infty }$.

Theorem 8.

Let $\mathcal{X},\mathcal{Y}$ be finite alphabets with $\left|\mathcal{X}\right|\ge 2$ and $\left|\mathcal{Y}\right|\ge 2$. For all $\lambda \in {\mathbb{R}}_c$ with $0<\lambda <{log}_2(min\{\left|\mathcal{X}\right|,\left|\mathcal{Y}\right|\left\}\right)$, the set

$$\{W\in {\mathcal{C}H}_c(\mathcal{X},\mathcal{Y}):R_{\infty }\left(W\right)<\lambda \}$$

is semi-decidable.

Proof. 

We use the computable sequences of computable continuous functions $F_N$ from Theorem 7. It holds that

$$W\in \{W\in {\mathcal{C}H}_c(\mathcal{X},\mathcal{Y}):R_{\infty }\left(W\right)<\lambda \}$$

if and only if there is an $N_0$ such that $F_{N_0}<\lambda$ holds. As in the proof of Theorem 28 from [35], we now use the construction of a Turing machine $T{M}_{R_{\infty },<\lambda }$, which exactly accepts the set

$$\{W\in {\mathcal{C}H}_c(\mathcal{X},\mathcal{Y}):R_{\infty }\left(W\right)<\lambda \}.$$

 □

We now consider the approximability “from below” (this can be seen as a kind of reachability). We have shown that $R_{\infty }(·)$ can always be represented as a limit value of monotonically decreasing computable sequences of computable continuous functions. From this, it can be concluded that the sequence is then also a computable sequence of Banach–Mazur computable functions. We now have the following:

Theorem 9.

Let $\mathcal{X},\mathcal{Y}$ be finite alphabets with $\left|\mathcal{X}\right|\ge 2$ and $\left|\mathcal{Y}\right|\ge 2$. There does not exist a sequence of Banach–Mazur computable functions ${\left\{F_N\right\}}_{N\in \mathbb{N}}$ with

1.

$F_N\left(W\right)\le F_{N+1}\left(W\right)$ with $W\in {\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})$ and $N\in \mathbb{N}$;

2.

${lim}_{N\to \infty }{F}_N\left(W\right)=R_{\infty }\left(W\right)$ for all $W\in \mathcal{C}H(\mathcal{X},\mathcal{Y})$.

Proof. 

We assume that such a sequence ${\left\{F_N\right\}}_{N\in \mathbb{N}}$ does exist. Then, from Theorem 7 and the assumptions from this theorem, it can be concluded that $R_{\infty }$ is a Banach–Mazur computable function. This has created a contradiction.  □

With this, we immediately get the following:

Corollary 1.

Consider finite alphabets $\mathcal{X},\mathcal{Y}$ with $\left|\mathcal{X}\right|\ge 2,\left|\mathcal{Y}\right|\ge 2$, and let ${\left\{F_N\right\}}_{N\in \mathbb{N}}$ be a sequence of Banach–Mazur computable functions that satisfies the following:

1.

$F_N\left(W\right)\le F_{N+1}\left(W\right)$ with $W\in {\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})$ and $N\in \mathbb{N}$,

2.

${lim}_{N\to \infty }{F}_N\left(W\right)=R_{\infty }\left(W\right)$ for all $W\in \mathcal{C}H(\mathcal{X},\mathcal{Y})$.

Then, there exists $\widehat{W}\in {\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})$ such that ${lim}_{N\to \infty }{F}_N\left(\widehat{W}\right)<R_{\infty }\left(\widehat{W}\right)$ holds true.

We now want to apply the results for $R_{\infty }$ to the sphere packing bound as an application. With the results via the rate function, we immediately get

Theorem 10.

Let $\mathcal{X},\mathcal{Y}$ be finite alphabets with $\left|\mathcal{X}\right|\ge 2$ and $\left|\mathcal{Y}\right|\ge 2$. The sphere packing bound $E_{SP}(·,·)$ is not a Turing computable performance function for ${\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})\times {\mathbb{R}}_c^{+}$.

Proof. 

Assuming that the statement of the theorem is incorrect, then $R_{\infty }$ is a Turing computable performance function on ${\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})\times {\mathbb{R}}_c^{+}$. But then the channel functions $\underline{f}\left(W\right)=R_{\infty }\left(W\right)$ for $W\in {\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})$ and $\overline{f}\left(W\right)=C\left(W\right)$ for $W\in {\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})$ must be Turing computable channel functions. As was already shown, however, $R_{\infty }$ is not Banach–Mazur computable. We have thus created a contradiction.  □

4. Computability of the Channel Reliability Function and the Sequence of Expurgation Bound Functions

In this section, we consider the reliability function and the expurgation bound and show that these functions are not Turing computable performance functions.

With the help of the results from [35] for $C_0$ for noisy channels, we immediately get the following theorem:

Theorem 11.

Let $\mathcal{X},\mathcal{Y}$ be finite alphabets with $\left|\mathcal{X}\right|\ge 2$ and $\left|\mathcal{Y}\right|\ge 2$. The channel reliability function $E(·,·)$ is not a Turing computable performance function for ${\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})\times {\mathbb{R}}_c$.

Proof. 

Here, $\underline{f}\left(W\right)=C_0\left(W\right)$ for $W\in {\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})$ is a Turing computable function, according to Definition 14. We already know that $C_0$ is not Banach–Mazur computable on ${\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})$. This gives the proof in the same way as for the sphere packing bound, i.e., the proof of Theorem 10.  □

Now, we consider the rate function for the expurgation bound. The k-letter expurgation bound $E_{ex}(W,R,k)$ as a function of W and R is a lower bound for the channel reliability function. The latter can only be finite for certain intervals $(R_k^{ex}\left(W\right),C\left(W\right))$. Thus, we want to compute the function in these intervals. In their famous paper [13], Shannon, Gallager, and Berlekamp examined the sequence of functions ${\left\{E_{ex}(·,·,k)\right\}}_{k\in \mathbb{N}}$ and analyzed the relationship to the channel reliability function. They conjectured that for all $W\in \mathcal{C}H(\mathcal{X},\mathcal{Y})$ for all R with $E(W,R)<+\infty$ (one would have convergence and also $E_{ex}(W,R,k)<+\infty$), the relation

$$\underset{k\to \infty }{lim}{E}_{ex}(W,R,k)=E(W,R)$$

holds. This conjecture was first refuted in [42] and later refuted by a simpler example in [43].

It was already clear with the introduction of the channel reliability function that it had a complicated behavior. A closed-form formula for the channel reliability function is not yet known, and the results of this paper show that such a formula cannot exist. Shannon, Gallager, and Berlekamp tried in [13] in 1967 to find sequences of seemingly simple formulas for the approximation of the channel reliability function. It seems that they considered the sequence of the k-letter expurgation bounds to be very good channel data for its approximation. It was hoped that these sequences could be computed more easily with the use of new powerful digital computers.

Let us now examine the sequence ${\left\{E_{ex}(·,·,k)\right\}}_{k\in \mathbb{N}}$. We have already introduced the concept of computable sequences of computable continuous channel functions. We now introduce the concept of computable sequences of Turing computable performance functions.

Definition 18.

A sequence ${\left\{F_k\right\}}_{k\in \mathbb{N}}$ of Turing computable performance functions is called a computable sequence if there is a Turing machine that generates the description of $F_k$ for input k according to the definition of the function $F_k$ for the values for which the function is defined.

In the following theorem, we prove that the sequence of the k-letter expurgation bounds is not a computable sequence of computable performance functions. So, the hope mentioned above cannot be fulfilled.

Theorem 12.

Let $\mathcal{X},\mathcal{Y}$ be finite alphabets with $\left|\mathcal{X}\right|\ge 2$ and $\left|\mathcal{Y}\right|\ge 2$. The sequence of the expurgation lower bounds ${\left\{E_{ex}(·,·,k)\right\}}_{k\in \mathbb{N}}$ is not a computable sequence of Turing computable performance functions.

Proof. 

We prove the theorem by contradiction, assuming that there exists a Turing machine $T{M}_{*}$ that generates a description of the function $E_{ex}(·,·,k)$ for a given input k, as defined in its formulation. This implies that the sequence ${\left\{R_k^{ex}\right\}}_{k\in \mathbb{N}}$ is computable, since we have an algorithm that can generate each function in the sequence.

Notably, we can express ${\underline{f}}_k(·)$ as $R_k^{ex}(·)$. Given an input k, the Turing machine $T{M}_{*}$ produces the description of $E_{ex}(·,·,k)$, from which $R_k^{ex}$ can be directly obtained via projection (in the sense of primitive recursive functions).

According to Shannon, Gallager, and Berlekamp [13], the following limit holds:

$$\underset{k\to \infty }{lim}{R}_k^{ex}\left(W\right)=C_0\left(W\right)$$

for all $W\in \mathcal{C}H(\mathcal{X},\mathcal{Y})$. Furthermore, the sequence ${\left\{R_k^{ex}\left(W\right)\right\}}_{k\in \mathbb{N}}$ is monotonically increasing, i.e.,

$$R_k^{ex}\left(W\right)\le R_{k+1}^{ex}\left(W\right)\mathrm{for}\mathrm{all}k\in \mathbb{N}\mathrm{and}W\in \mathcal{C}H(\mathcal{X},\mathcal{Y}).$$

Let us consider the set

$$\{W\in {\mathcal{C}H}_c(\mathcal{X},\mathcal{Y}):C_0\left(W\right)>\lambda \}$$

for $\lambda \in {\mathbb{R}}_c$ with $0<\lambda <{log}_2(min\{\left|\mathcal{X}\right|,\left|\mathcal{Y}\right|\left\}\right)$. We are now constructing a Turing machine $T{M}_{*}$ with only one holding state, “stop”, which means that it either stops or computes forever. $T{M}_{*}$ should stop for input $W\in {\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})$ if and only if $C_0\left(W\right)$ applies, that is, $T{M}_{*}$ stops if W is in the above set. According to the assumption, ${\left\{R_k^{ex}(·)\right\}}_{k\in \mathbb{N}}$ is a computable sequence of Turing computable channel functions. For the input W, we can generate the computable sequence ${\left\{R_k^{ex}\left(W\right)\right\}}_{k\in \mathbb{N}}$ of computable numbers. We now use the Turing machine $T{M}_{\lambda }^1$, which receives an arbitrary computable number x as input and stops if and only if $x>\lambda$, i.e., $T{M}_{\lambda }^1$ has only one hold state and accepts exactly the computable numbers x as input for which $x>\lambda$ holds. We now use this program for the following algorithm.

• We start with $l=1$ and let $T{M}_{\lambda }^1$ compute one step for input $R_1^{ex}\left(W\right)$. If $T{M}_{\lambda }^1\left(R_1^{ex}\left(W\right)\right)$ stops; then, we stop the algorithm.
• If $T{M}_{\lambda }^1\left(R_1^{ex}\left(W\right)\right)$ does not stop, we set $l=l+1$ and compute $l+1$ steps $T{M}_{\lambda }^1\left(R_r^{ex}\left(W\right)\right)$ for $1\le r\le l+1$. If one of these Turing machines stops, then the algorithm stops; if not, we set $l=l+1$ and repeat the second computation.

The above algorithm stops if and only if there is a $\widehat{k}\in \mathbb{N}$ such that $R_{ex}^{\widehat{k}}\left(W\right)>\lambda$. But this is the case (because of the monotony of the sequence ${\left\{R_k^{ex}\left(W\right)\right\}}_{k\in \mathbb{N}}$) if and only if $C_0\left(W\right)>\lambda$. But with this, the set

$$\{W\in {\mathcal{C}H}_c(\mathcal{X},\mathcal{Y}):C_0\left(W\right)>\lambda \}$$

is semi-decidable. So, we have shown that this is not the case. We have thus created a contradiction.  □

5. Computability of the Zero-Error Capacity of Noisy Channels with Feedback

In this section, we consider the zero-error capacity for noisy channels with feedback. In our paper [35], we examined the properties of the zero-error capacity without feedback. Let $W\in \mathcal{C}H(\mathcal{X},\mathcal{Y})$. We already noted that Shannon showed in [40] that

$$C_0^{FB}=\left\{\begin{array}{cc}0\hfill & \mathrm{if}{C}_0\left(W\right)=0\hfill \\ \underset{P}{max}\underset{y}{min}{log}_2{\displaystyle \frac{1}{{\sum }_{x:W\left(y\right|x)>0}P\left(x\right)}}\hfill & otherwise.\hfill \end{array}\right.$$

(21)

From (15), recall that

$${\mathrm{\Psi }}_{\infty }\left(W\right)=\underset{p\in \mathcal{P}\left(\mathcal{X}\right)}{max}\underset{y\in \mathcal{Y}}{min}\sum _{x:W\left(y\right|x)>0}P\left(x\right).$$

(22)

Then, we have for W with $C_0\left(W\right)\ne 0$,

$$C_0^{FB}={log}_2{\displaystyle \frac{1}{{\mathrm{\Psi }}_{\infty }\left(W\right)}}.$$

We know that $C_0^{FB}\left(W\right)=R_{\infty }\left(W\right)$ if $C_0\left(W\right)>0$. If $C_0\left(W\right)=0$, then there is a channel W with $C_0^{FB}\left(W\right)=0$ and $R_{\infty }>0$. Like in Lemma 1, we can show the following:

Lemma 2.

Let $\mathcal{X},\mathcal{Y}$ be finite non-trivial alphabets. It holds that

$$C_0^{FB}:{\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})\to {\mathbb{R}}_c.$$

From Theorem 5 and the relationship between $C_0$ and $C_0^{FB}$, we get the following results for $C_0^{FB}$, which we have already proved for $C_0$ in [35].

Theorem 13.

Let $\mathcal{X},\mathcal{Y}$ be finite alphabets with $\left|\mathcal{X}\right|\ge 2$ and $\left|\mathcal{Y}\right|\ge 2$. For all $\lambda \in {\mathbb{R}}_c$ with $0\le \lambda <{log}_{2}min\left\{\right|\mathcal{X}|,|\mathcal{Y}\left|\right\}$, the sets $\{W\in {\mathcal{C}H}_c(\mathcal{X},\mathcal{Y}):C_0^{FB}\left(W\right)>\lambda \}$ are not semi-decidable.

Theorem 14.

Let $\mathcal{X},\mathcal{Y}$ be finite alphabets with $\left|\mathcal{X}\right|\ge 2$ and $\left|\mathcal{Y}\right|\ge 2$. Then, $C_0^{FB}:{\mathcal{C}H}_c(\mathcal{X},\mathcal{Y})\to \mathbb{R}$ is not Banach-Mazur computable.

Now, we will prove the following:

Theorem 15.

Let $\mathcal{X},\mathcal{Y}$ be finite alphabets with $\left|\mathcal{X}\right|\ge 2$ and $\left|\mathcal{Y}\right|\ge 2$. There is a computable sequence of computable continuous functions G with

1.

$G_N\left(W\right)\ge G_{N+1}\left(W\right)$ for $W\in \mathcal{C}H(\mathcal{X},\mathcal{Y})$ and $N\in \mathbb{N}$;

2.

${lim}_{n\to \infty }{G}_N\left(W\right)=C_0^{FB}\left(W\right)$ for $W\in \mathcal{C}H(\mathcal{X},\mathcal{Y})$.

Proof. 

We use for $N\in \mathbb{N}$, $y\in \mathcal{Y}$ and $P\in \mathcal{P}\left(\mathcal{X}\right)$ the function

$$\sum _{x\in \mathcal{X}}{\displaystyle \frac{NW\left(y\right|x)}{1+NW\left(y\right|x)}}P\left(x\right).$$

Then, for

$${\mathrm{\Phi }}_N\left(W\right)=\underset{P\in \mathcal{P}\left(\mathcal{X}\right)}{min}\underset{y\in \mathcal{Y}}{max}\sum _{x\in \mathcal{X}:W\left(y\right|x)>0}P\left(x\right),$$

we have the same properties as in Theorem 7 and

$$U_N\left(W\right)={log}_2{\displaystyle \frac{1}{{\mathrm{\Phi }}_n\left(W\right)}}$$

is an upper bound for $C_0^{FB}$, which is monotonically decreasing. Now, the relation $C_0^{FB}\left(W\right)>0$ holds for $W\in \mathcal{C}H(\mathcal{X},\mathcal{Y})$ if and only if there are two $x_1,x_2\in \mathcal{X}$ so that

$$\sum _{y\in \mathcal{Y}}W\left(y\right|x_1)W\left(y\right|x_2)=0$$

holds. We now set $g(\widehat{x},x)={\sum }_{y\in \mathcal{Y}}W\left(y\right|\widehat{x})W\left(y\right|x)=g(W,\widehat{x},x)$ and have $0\le g(\widehat{x},x)\le 1$ for $x,\widehat{x}\in \mathcal{X}$. g is a computable continuous function with respect to $W\in \mathcal{C}H(\mathcal{X},\mathcal{Y})$. Now, we set

$$V_N\left(W\right)=\left(1-\prod _{x,\widehat{x}}g{(W,\widehat{x},x)}^N\right)U_N\left(W\right)$$

for $N\in \mathbb{N}$. ${\left\{V_N\right\}}_{N\in \mathbb{N}}$ is thus a computable sequence of computable continuous functions. Obviously, $V_N\left(W\right)\ge V_{N+1}\left(W\right)$ for $W\in \mathcal{C}H(\mathcal{X},\mathcal{Y})$ and $N\in \mathbb{N}$ is satisfied.

$${(1-\prod _{x,\widehat{x}}g(W,x,hx))}^N=1$$

if and only if $C_0^{FB}>0$. So, for $C_0^{FB}\left(W\right)=0$, we always have

$$\underset{N\to \infty }{lim}{V}_N\left(W\right)=0.$$

For W with $C_0^{FB}\left(W\right)$,

$$\underset{N\to \infty }{lim}{V}_N\left(W\right)=\underset{N\to \infty }{lim}{U}_N\left(W\right)=C_0^{FB}\left(W\right).$$

This is shown in the proof of Theorem 7.  □

This immediately gives us the following theorem.

Theorem 16.

Let $\mathcal{X},\mathcal{Y}$ be finite alphabets with $\left|\mathcal{X}\right|\ge 2$ and $\left|\mathcal{Y}\right|\ge 2$. For all $\lambda \in {\mathbb{R}}_c$ with $0\le \lambda <{log}_{2}min\left\{\right|\mathcal{X}|,|\mathcal{Y}\left|\right\}$, the sets $\{W\in {\mathcal{C}H}_c(\mathcal{X},\mathcal{Y}):C_0^{FB}\left(W\right)<\lambda \}$ are semi-decidable.

Now, we want to look at the consequences of the results above for $C_0^{FB}$. The same statements apply here as in Section 3 for $R_{\infty }$ with regard to the approximation from below. $C_0^{FB}$ cannot be approximated by monotonically increasing sequences.

There is an elementary relationship between $R_{\infty }$ and $C_0^{FB}$, which we use in the following. Again, we assume that $\mathcal{X},\mathcal{Y}$ are finite non-trivial alphabets. We remember the following functions:

$$R_{\infty }\left(W\right)={log}_2{\displaystyle \frac{1}{{\mathrm{\Psi }}_{\infty }\left(W\right)}},$$

(23)

where ${\mathrm{\Psi }}_{\infty }\left(W\right)={max}_{Q\in \mathcal{P}\left(\mathcal{Y}\right)}{min}_{x\in \mathcal{X}}{\sum }_{y:W\left(y\right|x)>0}Q\left(y\right).$

$$C_0^{FB}=\left\{\begin{array}{cc}0\hfill & C_0\left(W\right)=0\hfill \\ G\left(W\right)\hfill & C_0\left(W\right)>0\hfill \end{array}\right.,$$

(24)

where $G\left(W\right)={log}_2{\displaystyle \frac{1}{{\mathrm{\Psi }}_{\infty }\left(W\right)}}$ and

$${\mathrm{\Psi }}_{\infty }\left(W\right)=\underset{p\in \mathcal{P}\left(\mathcal{X}\right)}{min}\underset{y\in \mathcal{Y}}{min}\sum _{x:W\left(y\right|x)>0}P\left(x\right).$$

(25)

Let $A\left(W\right)$ be the $\left|\mathcal{Y}\right|\times \left|\mathcal{X}\right|$ matrix with ${\left(A\left(W\right)\right)}_{kl}\in \{0,1\}$ for $1\le k\le \left|\mathcal{Y}\right|$ and $1\le l\le \left|\mathcal{X}\right|$, such that ${\left(A\left(W\right)\right)}_{kl}=1$ if and only if $W\left(k\right(l\left)\right)>0$. Furthermore, let

$${\mathcal{M}}_{\mathcal{X}}=\left\{u\in {\mathbb{R}}^{\left|\mathcal{X}\right|}:u=\left(\begin{array}{c}{u}_1\\ \dots \\ u_{\left|\mathcal{X}\right|}\end{array}\right),u_l\ge 0,\sum _{l=1}^{\left|\mathcal{X}\right|}{u}_l=1\right\}$$

(26)

and

$${\mathcal{M}}_{\mathcal{Y}}=\left\{v\in {\mathbb{R}}^{\left|\mathcal{Y}\right|}:v=\left(\begin{array}{c}{v}_1\\ \dots \\ v_{\left|\mathcal{Y}\right|}\end{array}\right),v_l\ge 0,\sum _{l=1}^{\left|\mathcal{Y}\right|}{v}_l=1\right\}.$$

(27)

For $v\in {\mathbb{R}}^{\left|\mathcal{Y}\right|}$ and $u\in {\mathbb{R}}^{\left|\mathcal{X}\right|}$, we consider the function $F(v,u)=v^{T}A\left(W\right)u$. The function F is concave in $v\in {\mathcal{M}}_{\mathcal{Y}}$ and convex in $u\in {\mathcal{M}}_{\mathcal{X}}$. ${\mathcal{M}}_{\mathcal{Y}}$ and ${\mathcal{M}}_{\mathcal{X}}$ are closed convex and compact sets, and $F(v,u)$ is continuous in both variables. So,

$$\underset{v\in {\mathcal{M}}_{\mathcal{Y}}}{max}\underset{u\in {\mathcal{M}}_{\mathcal{X}}}{min}F(v,u)=\underset{u\in {\mathcal{M}}_{\mathcal{X}}}{min}\underset{v\in {\mathcal{M}}_{\mathcal{Y}}}{max}F(v,u).$$

(28)

Let $v\in {\mathcal{M}}_{\mathcal{Y}}$ be fixed. Then,

$$\begin{array}{ccc}\hfill F(v,u)& =& \left(\sum _{l=1}^{\left|\mathcal{X}\right|}\left(\sum _{k=1}^{\left|\mathcal{Y}\right|}{v}_k{A}_{kl}\left(W\right)\right)u_l\right)\hfill \end{array}$$

(29)

$$\begin{array}{ccc}\hfill F(v,u)& =& \left(\sum _{l=1}^{\left|\mathcal{X}\right|}{d}_l\left(v\right)u_l\right),\hfill \end{array}$$

(30)

with $d_l\left(v\right)={\sum }_{k=1}^{\left|\mathcal{Y}\right|}{v}_k{A}_{kl}\left(W\right)$. Now, $d_l\left(v\right)\ge 0$ for $1\le l\le \left|\mathcal{X}\right|$. Hence,

$$\underset{u\in {\mathcal{M}}_{\mathcal{X}}}{min}F(v,u)=\underset{1\le l\le \left|\mathcal{X}\right|}{min}{d}_l\left(v\right)=\underset{1\le l\le \left|\mathcal{X}\right|}{min}\sum _{k:A_{k}l\left(W\right)>0}{v}_k=\underset{x\in \mathcal{X}}{min}\sum _{y:W\left(y\right|x)>0}{Q}_v\left(y\right),$$

with $Q_v\left(y\right)=v_y$ for $y\in \{1,\dots ,|\mathcal{Y}\left|\right\}$. So,

$$\underset{v\in {\mathcal{M}}_{\mathcal{Y}}}{max}\underset{u\in {\mathcal{M}}_{\mathcal{X}}}{min}F(v,u)=\underset{Q\in \mathcal{P}\left(\mathcal{Y}\right)}{max}\underset{x\in \mathcal{X}}{min}\sum _{y:W\left(y\right|x)>0}{Q}_v\left(y\right)={\mathrm{\Psi }}_{\infty }\left(W\right).$$

Furthermore, for $u\in {\mathcal{M}}_{\mathcal{X}}$ fixed,

$$\begin{array}{ccc}\hfill F(v,u)& =& \left(\sum _{k=1}^{\left|\mathcal{Y}\right|}\left(\sum _{l=1}^{\left|\mathcal{X}\right|}{u}_l{A}_{kl}\left(W\right)\right)v_k\right)\hfill \\ & =& \left(\sum _{k=1}^{\left|\mathcal{Y}\right|}{\beta }_k\left(u\right)v_k\right),\hfill \end{array}$$

with ${\beta }_k\left(u\right)={\sum }_{l=1}^{\left|\mathcal{X}\right|}{u}_l{A}_{kl}\left(W\right)\ge 0$ and $1\le k\le \left|\mathcal{Y}\right|$. Therefore,

$$\underset{v\in {\mathcal{M}}_{\mathcal{Y}}}{max}F(v,u)=\underset{1\le k\le \left|\mathcal{Y}\right|}{max}{\beta }_k\left(u\right)=\underset{1\le k\le \left|\mathcal{Y}\right|}{max}\sum _{l:A_{kl}\left(W\right)>0}{u}_l=\underset{y\in \mathcal{Y}}{max}\sum _{x:W\left(Y\right|x)>0}{p}_u\left(x\right)$$

with $p_u\left(x\right)=u_x$ for $1\le x\le \left|\mathcal{X}\right|$. It follows that

$$\underset{u\in {\mathcal{M}}_{\mathcal{X}}}{min}\underset{v\in {\mathcal{M}}_{\mathcal{Y}}}{max}F(v,u)=\underset{p\in \mathcal{P}\left(\mathcal{X}\right)}{min}\underset{y\in \mathcal{Y}}{max}\sum _{x:W\left(y\right|x)>0}P\left(x\right)={\mathrm{\Psi }}_{\infty }\left(W\right).$$

We get the following lemma.

Lemma 3.

Let $W\in \mathcal{C}H(\mathcal{X},\mathcal{Y})$; then,

$$R_{\infty }\left(W\right)=G\left(W\right).$$

We want to investigate the behavior of $E(·,R)$ for the input $W_1\otimes W_2$, where $W_1\otimes W_2$ denotes the Kronecker product of the matrices $W_1$ and $W_2$ compared to $E(W_1,R)$ and $E(W_2,R)$. For this purpose, let ${\mathcal{X}}_1,{\mathcal{Y}}_1,{\mathcal{X}}_2,{\mathcal{Y}}_2$ be arbitrary finite non-trivial alphabets, and we consider $W_l\in \mathcal{C}H({\mathcal{X}}_l,{\mathcal{Y}}_l)$ for $l=1,2$.

Theorem 17.

Let ${\mathcal{X}}_1,{\mathcal{Y}}_1,{\mathcal{X}}_2,{\mathcal{Y}}_2$ be arbitrary finite non-trivial alphabets, and $W_l\in \mathcal{C}H({\mathcal{X}}_l,{\mathcal{Y}}_l)$ for $l=1,2$. Then, we have

$$R_{\infty }(W_1\otimes W_2)=R_{\infty }\left(W_1\right)+R_{\infty }\left(W_2\right).$$

Proof. 

We use the ${\mathrm{\Psi }}_{\infty }$ function. It applies to $Q=Q_1·Q_2$ with $Q_1\in \mathcal{P}\left({\mathcal{Y}}_1\right)$ and ${\mathcal{Q}}_2\in \mathcal{P}\left(Y_2\right)$, so that

$$\begin{array}{ccc}& & \underset{x_1\in {\mathcal{X}}_1,x_2\in {\mathcal{X}}_2}{min}\sum _{y_1:W_1\left(y_1\right|x_1)>0}\sum _{y_2:W_2\left(y_2\right|x_2)}{Q}_1\left(y_1\right)Q_2\left(y_2\right)\hfill \\ & =& \left(\underset{x_1\in {\mathcal{X}}_1}{min}\sum _{y_1:W_1\left(y_1\right|x_1)>0}{Q}_1\left(y_1\right)\right)\left(\underset{x_1\in {\mathcal{X}}_1}{min}\sum _{y_2:W_2\left(y_2\right|x_2)>0}{Q}_2\left(y_2\right)\right).\hfill \end{array}$$

This applies to all $Q_1\in \mathcal{P}\left({\mathcal{Y}}_1\right)$ and $Q_2\in \mathcal{P}\left({\mathcal{Y}}_2\right)$ arbitrarily. So,

$${\mathrm{\Psi }}_{\infty }(W_1\otimes W_2)\ge {\mathrm{\Psi }}_{\infty }\left(W_1\right)·{\mathrm{\Psi }}_{\infty }\left(W_2\right).$$

Also, we have

$$\begin{array}{ccc}& & {\mathrm{\Psi }}_{\infty }(W_1\otimes W_2)\hfill \\ & =& \underset{P\in \mathcal{P}({\mathcal{X}}_1\times {\mathcal{X}}_2)}{min}\underset{(y_1,y_2)\in {\mathcal{Y}}_1\times {\mathcal{Y}}_2}{max}\sum _{x_1:W_1\left(y_1\right|x_1)>0}\sum _{x_2:W_2\left(y_2\right|x_2)>0}P(x_1,y_2)\hfill \\ & \le & {\mathrm{\Psi }}_{\infty }\left(W_1\right)·{\mathrm{\Psi }}_{\infty }\left(W_2\right)\hfill \end{array}$$

as well. So,

$${\mathrm{\Psi }}_{\infty }(W_1\otimes W_2)={\mathrm{\Psi }}_{\infty }\left(W_1\right)·{\mathrm{\Psi }}_{\infty }\left(W_2\right)$$

and the theorem is proven.  □

We want to investigate the behavior of $C_0^{FB}$ for the input $W_1\otimes W_2$ compared to $C_0^{FB}\left(W_1\right)$ and $C_0^{FB}\left(W_2\right)$. For this purpose, let ${\mathcal{X}}_1,{\mathcal{Y}}_1,{\mathcal{X}}_2,{\mathcal{Y}}_2$ be arbitrary finite non-trivial alphabets and consider $W_l\in \mathcal{C}H({\mathcal{X}}_l,{\mathcal{Y}}_l)$ for $l=1,2$.

Theorem 18.

Let ${\mathcal{X}}_1,{\mathcal{Y}}_1,{\mathcal{X}}_2,{\mathcal{Y}}_2$ be arbitrary finite non-trivial alphabets, and $W_l\in \mathcal{C}H({\mathcal{X}}_l,{\mathcal{Y}}_l)$ for $l=1,2$. Then, we have

1.

$$C_0^{FB}(W_1\otimes W_2)\ge C_0^{FB}\left(W_1\right)+C_0^{FB}\left(W_2\right)$$

(31)

2.

$$C_0^{FB}(W_1\otimes W_2)>C_0^{FB}\left(W_1\right)+C_0^{FB}\left(W_2\right)$$

(32)

if and only if

$$\underset{1\le l\le 2}{min}{C}_0^{FB}\left(W_l\right)=0\mathit{and}\underset{1\le l\le 2}{max}{C}_0^{FB}\left(W_l\right)>0\mathit{and}\underset{1\le l\le 2}{min}{R}_{\infty }\left(W_l\right)>0.$$

(33)

Remark 9.

The condition (33) is equivalent to

$$\underset{1\le l\le 2}{min}{C}_0\left(W_l\right)=0\mathit{and}\underset{1\le l\le 2}{max}{C}_0\left(W_l\right)>0\mathit{and}\underset{1\le l\le 2}{min}{R}_{\infty }\left(W_l\right)>0.$$

(34)

Proof. 

(31) follows directly from the operational definition of C. Let (33) now be fulfilled. Then, $C_0^{FB}(W_1\otimes W_2)>0$ must be fulfilled. Without loss of generality, we assume $C_0^{FB}\left(W_1\right)=0$, $C_0^{FB}\left(W_2\right)>0$ and $R_{\infty }\left(W_1\right)>0$, $R_{\infty }\left(W_2\right)>0$. Since $C_0^{FB}(W_1\otimes W_2)>0$,

$$\begin{array}{ccc}\hfill C_0^{FB}(W_1\otimes W_2)& =& R_{\infty }(W_1\otimes W_2)\hfill \\ & =& R_{\infty }\left(W_1\right)+R_{\infty }\left(W_2\right)\hfill \\ & =& R_{\infty }\left(W_1\right)+C_0^{FB}\left(W_2\right)\hfill \\ & >& 0+C_0^{FB}\left(W_2\right)\hfill \\ & =& C_0^{FB}\left(W_1\right)+C_0^{FB}\left(W_2\right).\hfill \end{array}$$

If (32) is fulfilled, then $C_0^{FB}(W_1\otimes W_2)>0$. Then, ${max}_{1\le l\le 2}{C}_0^{FB}\left(W_l\right)>0$ must be, because if ${max}_{1\le l\le 2}{C}_0^{FB}\left(W_l\right)=0$, then ${max}_{1\le l\le 2}{C}_0\left(W_l\right)=0$, and thus $C_0(W_1\otimes W_2)=0$ also (since the $C_0$ capacity has no super-activation). This means that $C_0^{FB}(W_1\otimes W_2)=0$, which would be a contradiction.

If ${min}_{1\le 2}{C}_0^{FB}\left(W_l\right)>0$, then

$$\begin{array}{ccc}\hfill C_0^{FB}(W_1\otimes W_2)& =& R_{\infty }(W_1\otimes W_2)\hfill \\ & =& R_{\infty }\left(W_1\right)+R_{\infty }\left(W_2\right)\hfill \\ & =& C_0^{FB}\left(W_1\right)+C_0^{FB}\left(W_2\right).\hfill \end{array}$$

This is a contradiction, and thus ${min}_{1\le 2}{C}_0^{FB}\left(W_l\right)=0$. Furthermore, ${min}_{1\le l\le 2}{R}_{\infty }\left(W_l\right)>0$ must apply, because if ${min}_{1\le l\le 2}{R}_{\infty }\left(W_l\right)=0$, then $R_{\infty }\left(W_1\right)=0$ without loss of generality. Then,

$$\begin{array}{ccc}\hfill C_0^{FB}(W_1\otimes W_2)& =& R_{\infty }(W_1\otimes W_2)\hfill \\ & =& R_{\infty }\left(W_1\right)+R_{\infty }\left(W_2\right)\hfill \\ & =& 0+R_{\infty }\left(W_2\right)\hfill \\ & =& 0+C_0^{FB}\left(W_2\right)\hfill \\ & =& C_0^{FB}\left(W_1\right)+C_0^{FB}\left(W_2\right),\hfill \end{array}$$

because $C_0^{FB}\left(W_1\right)=0$ when $R_{\infty }\left(W_1\right)=0$. This is again a contradiction. With this, we have proven the theorem.  □

We still want to show for which alphabet sizes the behavior according to Theorem 18 can occur.

Theorem 19.

1.

If $|{\mathcal{X}}_1|=|{\mathcal{X}}_2|=|Y_1|=|Y_2|=2$, then for all $W_l\in \mathcal{C}H({\mathcal{X}}_l,{\mathcal{Y}}_l)$ with $l=1,2$, we have

$$C_0^{FB}(W_1\otimes W_2)=C_0^{FB}\left(W_1\right)+C_0^{FB}\left(W_2\right).$$

(35)

2.

If ${\mathcal{X}}_1,X_2,{\mathcal{Y}}_1,{\mathcal{Y}}_2$ are non-trivial alphabets with

$$max\{min\{|{\mathcal{X}}_1|,|{\mathcal{Y}}_1\left|\right\},min\left\{\right|X_2|,|{\mathcal{Y}}_2\left|\right\}\}\ge 3,$$

then there exists ${\widehat{W}}_l\in \mathcal{C}H({\mathcal{X}}_l,{\mathcal{Y}}_l)$ with $l=1,2$, such that

$$C_0^{FB}({\widehat{W}}_1\otimes {\widehat{W}}_2)>C_0^{FB}\left({\widehat{W}}_1\right)+C_0^{FB}\left({\widehat{W}}_2\right).$$

(36)

Proof. 

• If $C_0\left(W_1\right)=C_0\left(W_2\right)$, then (35) holds, since $C_0(W_1\otimes W_2)=0$.
  If $max\{C_0\left(W_1\right),C_0\left(W_2\right)\}>0$, we can assume without loss of generality that $C_0\left(W_1\right)=0$. In this case, $W_1$ must be either

  $$W_1=\left(\begin{array}{cc}1& 0\\ 0& 1\end{array}\right)\mathrm{or}{W}_1=\left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right),$$

  which implies that $C_0\left(W_2\right)=1$, and consequently, $C_0^{FB}\left(W_2\right)=1$. Furthermore, if $R_{\infty }\left(W_2\right)>0$, then $W_2$ must also be one of the two matrices above, ensuring that (35) holds. If instead $R_{\infty }\left(W_2\right)=0$, Theorem 17 guarantees that (35) remains valid.
• We now prove (36) under the assumption that $|{\mathcal{X}}_1|=|{\mathcal{Y}}_1|=2$ and $|{\mathcal{X}}_2|=|{\mathcal{Y}}_2|=3$. If we have found channels ${\widehat{W}}_1,{\widehat{W}}_2$ for this case, such that (36) holds, then it is also clear how general case 2 can be proved. We set ${\widehat{W}}_1=\left(\begin{array}{cc}1& 0\\ 0& 1\end{array}\right)$, which means $C_0\left({\widehat{W}}_1\right)=C_0^{FB}\left({\widehat{W}}_1\right)=R_{\infty }\left({\widehat{W}}_1\right)=1$. For ${\widehat{W}}_2$, we take the three-ary typewriter channel ${\widehat{W}}_2\left(ϵ\right)$ with ${\mathcal{X}}_2={\mathcal{Y}}_2=\{0,1,2\}$ (see [43]):

  $${\widehat{W}}_2\left(ϵ\right)\left(y\right|x)=\left\{\begin{array}{cc}1-ϵ\hfill & y=x,\hfill \\ ϵ\hfill & y=x+1mod3.\hfill \end{array}\right.$$
  Let $ϵ\in (0,{\displaystyle \frac{1}{2}})$ be arbitrary, then $C\left({\widehat{W}}_2\left(ϵ\right)\right)={log}_2\left(3\right)-H_2\left(ϵ\right)$. We have $R_{\infty }\left({\widehat{W}}_2\left(ϵ\right)\right)={log}_2{\displaystyle \frac{3}{2}}$ and $C_0\left({\widehat{W}}_2\left(ϵ\right)\right)=0$. This means that $C_0^{FB}\left({\widehat{W}}_2\left(ϵ\right)\right)=0$. Thus, because $C_0({\widehat{W}}_1\times {\widehat{W}}_2\left(ϵ\right))\ge C_0\left({\widehat{W}}_1\right)=1$,

  $$\begin{array}{ccc}\hfill C_0^{FB}({\widehat{W}}_1\otimes {\widehat{W}}_2\left(ϵ\right))& =& R_{\infty }\left({\widehat{W}}_1\right)=R_{\infty }\left({\widehat{W}}_2\left(ϵ\right)\right)\hfill \\ & =& 1+{log}_2\left({\displaystyle \frac{3}{2}}\right)>C_0^{FB}\left({\widehat{W}}_1\right)+C_0^{FB}\left({\widehat{W}}_2\left(ϵ\right)\right)\hfill \end{array}$$

  and we have proven case 2.

 □

6. Behavior of the Expurgation-Bound Rates

In this section, we consider the behavior of the expurgation-bound rate. $R_{ex}^k$ occurs in the expurgation bound as a lower bound for the channel reliability function, where k is the parameter for the k-letter description. Let ${\mathcal{X}}_1,{\mathcal{Y}}_1,{\mathcal{X}}_2,{\mathcal{Y}}_2$ be arbitrary finite non-trivial alphabets, and $W_l\in \mathcal{C}H({\mathcal{X}}_l,{\mathcal{Y}}_l)$ for $l=1,2$. We want to examine $R_{ex}^k$.

Theorem 20.

There exist non-trivial alphabets ${\mathcal{X}}_1,{\mathcal{Y}}_1,{\mathcal{X}}_2,{\mathcal{Y}}_2$ and channels $W_l\in \mathcal{C}H({\mathcal{X}}_l,{\mathcal{Y}}_l)$ for $l=1,2$, such that for all $\widehat{k}$, there exists $k\ge \widehat{k}$ with

$$R_{ex}^k(W_1\otimes W_2)\ne R_{ex}^k\left(W_1\right)+R_{ex}^k\left(W_2\right).$$

Proof. 

Assume that for all ${\mathcal{X}}_1,{\mathcal{Y}}_1,{\mathcal{X}}_2,{\mathcal{Y}}_2$ and $W_l\in \mathcal{C}H({\mathcal{X}}_l,{\mathcal{Y}}_l)$ with $l=1,2$ for all $k\in \mathbb{N}$,

$$R_{ex}^k(W_1\otimes W_2)=R_{ex}^k\left(W_1\right)+R_{ex}^k\left(W_2\right).$$

We now take ${\mathcal{X}}_1^{\prime },{\mathcal{Y}}_1^{\prime },{\mathcal{X}}_2^{\prime },{\mathcal{Y}}_2^{\prime }$ such that $C_0$ is superadditive. Then, we have for certain $W_1^{\prime },W_2^{\prime }$ with $W_l^{\prime }\in \mathcal{C}H({\mathcal{X}}_l^{\prime },{\mathcal{Y}}_l^{\prime })$,

$$C_0(W_1^{\prime }\otimes W_2^{\prime })>C_0\left(W_1^{\prime }\right)+C_0\left(W_2^{\prime }\right).$$

(37)

Then,

$$\begin{array}{ccc}\hfill C_0(W_1^{\prime }\otimes W_2^{\prime })& =& \underset{k\to \infty }{lim}{R}_{ex}^k(W_1^{\prime }\otimes W_2^{\prime })\hfill \\ & =& \underset{k\to \infty }{lim}{R}_{ex}^k\left(W_1^{\prime }\right)+R_{ex}^k\left(W_2^{\prime }\right)\hfill \\ & =& C_0\left(W_1^{\prime }\right)+C_0\left(W_2^{\prime }\right).\hfill \end{array}$$

This is a contradiction, and thus the theorem is proven.  □

We improve the statement of Theorem 20 with the following theorem.

Theorem 21.

There exist non-trivial alphabets ${\mathcal{X}}_1,{\mathcal{Y}}_1,{\mathcal{X}}_2,{\mathcal{Y}}_2$ and channels $W_l\in \mathcal{C}H({\mathcal{X}}_l,{\mathcal{Y}}_l)$ for $l=1,2$ and a $\widehat{k}$, such that for all $k\ge \widehat{k}$,

$$R_{ex}^k(W_1\otimes W_2)>R_{ex}^k\left(W_1\right)+R_{ex}^k\left(W_2\right)$$

holds true.

Proof. 

Assume the statement of the theorem is false, which means for all channels $W_l\in \mathcal{C}H({\mathcal{X}}_l,{\mathcal{Y}}_l)$ with $l=1,2$, the following applies: There exists a sequence ${\left\{k_j\right\}}_{j\in \mathbb{N}}\subset \mathbb{N}$ with ${lim}_{j\to \infty }{k}_j=+\infty$, such that

$$R_{ex}^{k_l}(W_1\otimes W_2)\le R_{ex}^{k_l}\left(W_1\right)+R_{ex}^{k_l}\left(W_2\right)$$

for $l\in \mathbb{N}$. We now take ${\widehat{\mathcal{X}}}_1,{\widehat{\mathcal{Y}}}_1,{\widehat{\mathcal{X}}}_2,{\widehat{\mathcal{Y}}}_2$ so that $C_0$ is superadditive for these alphabets. Then, we have for certain ${\widehat{W}}_1,{\widehat{W}}_2$ with ${\widehat{W}}_l\in \mathcal{C}H({\mathcal{X}}_l,{\mathcal{Y}}_l)$ for $l=1,2$,

$$C_0({\widehat{W}}_1\otimes {\widehat{W}}_2)>C_0\left({\widehat{W}}_1\right)+C_0\left({\widehat{W}}_2\right).$$

(38)

Then,

$$\begin{array}{ccc}\hfill C_0({\widehat{W}}_1\otimes {\widehat{W}}_2)=\underset{j\to \infty }{lim}{R}_{ex}^{k_j}({\widehat{W}}_1\otimes {\widehat{W}}_2)& \le & \underset{j\to \infty }{lim}\left(R_{ex}^{k_j}\left({\widehat{W}}_1\right)+R_{ex}^{k_j}\left({\widehat{W}}_2\right)\right)\hfill \\ & =& C_0\left({\widehat{W}}_1\right)+C_0\left({\widehat{W}}_2\right).\hfill \end{array}$$

This is a contradiction to (38), and thus the theorem is proven.  □

We have already observed that the function $E(W,·)$ exhibits significantly different behavior over certain rate intervals $[R,\widehat{R}]$. In particular, we have analyzed the impact of the channel product $W_1\otimes W_2$ on the intervals $(R_{\infty }(W_1\otimes W_2),C(W_1\otimes W_2))$ and $(E_{ex}^k(W_1\otimes W_2),C(W_1\otimes W_2))$ for $k\in \mathbb{N}$.

For the first interval, we established the relation

$$(R_{\infty }(W_1\otimes W_2),C(W_1\otimes W_2))=(R_{\infty }\left(W_1\right)+R_{\infty }\left(W_2\right),C\left(W_1\right)+C\left(W_2\right)).$$

However, for the second interval, we have shown that such a simple additive behavior does not hold. Given the proof of Theorem 18, we conclude that there exist channels $W_1^{\prime },W_2^{\prime }$ for which

$$R_{ex}^k(W_1^{\prime }\otimes W_2^{\prime })>R_{ex}^k\left(W_1^{\prime }\right)+R_{ex}^k\left(W_2^{\prime }\right)$$

is satisfied for all $k\ge \widehat{k}$.

Another important aspect is understanding the conditions under which the interval $[0,\widehat{R})$ causes $E(W,r)$ to become infinite. This occurs if and only if $C_0\left(W\right)>0$, in which case, the interval is given by $[0,C_0\left(W\right))$. Consequently, there exist channels $W_1^{\prime },W_2^{\prime }$ such that for the function $E(W_1^{\prime }\otimes W_2^{\prime },·)$, this interval extends beyond $[0,C_0\left(W_1^{\prime }\right)+C_0\left(W_2^{\prime }\right)]$.

Thus, we conclude that $C_0$ is generally superadditive.

7. Conclusions

We have shown that the channel reliability function is not a Turing computable performance function. The same conclusion holds for the functions associated with the sphere packing bound and the expurgation bound.

An interesting aspect of our work is that the constraints we impose on Turing computable performance functions are strictly weaker than those typically required for Turing computable functions. Specifically, we do not require that the Turing machine halt for all inputs $(W,R)\in {\mathcal{C}}_h\times {\mathbb{R}}_c^{+}$. This means we allow the Turing machine to compute indefinitely for certain inputs, i.e., it may never halt for some inputs. Consequently, we permit performance functions that are not defined for all $(W,R)\in {\mathcal{C}}_h\times {\mathbb{R}}_c^{+}$. However, we do require the Turing machine to halt for inputs $(W,R)\in {\mathcal{C}}_h\times {\mathbb{R}}_c$ whenever the performance function F is defined, and in such cases, the machine must return the computable value $F(W,R)$ as output. This ensures that the algorithm generated corresponds to the number $F(W,R)$ according to Definition 15.

Additionally, we considered the $R_{\infty }$ function and the zero-error feedback capacity, both of which play a critical role in the context of the channel reliability function. We demonstrated that neither the $R_{\infty }$ function nor the zero-error feedback capacity is Banach–Mazur computable. Furthermore, we proved that the $R_{\infty }$ function is additive.

We also established that for all finite alphabets $\mathcal{X},\mathcal{Y}$ with $\left|\mathcal{X}\right|\ge 2$ and $\left|\mathcal{Y}\right|\ge 2$, the channel reliability function itself is not a Turing computable performance function. Moreover, we showed that the commonly studied bounds, which have been extensively examined in the literature, are also not Turing computable performance functions. It remains unclear whether non-trivial upper bounds for the channel reliability function that are Turing computable even exist.

In [13], the sequence of k-letter expurgation bounds was considered an effective method for approximating the channel reliability function. It was hoped that these sequences could be computed more efficiently using modern digital computers. However, we have shown that this is not the case.

Table 1. Overview of results.

Problem	Result
Computability of capacity	$R_{\infty }$: Yes $R_0$: Unknown $R_0$ for alphabet size $<5$: Yes
Semi-decidability of capacity $>\lambda$	$R_{\infty }$: No $C_0^{FB}$: No
Semi-decidability of capacity $<\lambda$	$R_{\infty }$: Yes $C_0^{FB}$: Yes
Computability of capacity function	$R_{\infty }$: No $C_0^{FB}$: No
Computability of performance function	$E(W,R)$: No ${\left\{E_{ex}(·,·,k)\right\}}_{k\in \mathbb{N}}$: No
Additivity	$R_{\infty }$: Yes $C_0^{FB}$: Unknown $C_0^{FB}$ for alphabet size 2: Yes $R_{ex}^k$: No

gives an overview of the main results of the paper.

As mentioned in the Introduction, future communication systems, such as 6G, will face stringent requirements for trustworthiness. Ultra-reliability, along with the corresponding performance functions, is central to 6G, and this paper addresses that challenge. It is currently unclear how the non-Turing computability of performance functions will impact the system evaluation and certification of future communication systems. A recent study [62] showed that the non-Turing computability of performance functions in artificial intelligence (AI) leads to digital AI algorithms being unable to meet essential legal requirements. It is an intriguing research question whether similar issues might arise in the context of communication systems.

This work does not claim that machine learning or artificial intelligence (AI) approaches are useless for computing capacity functions. Rather, it demonstrates that certain solutions cannot be found by such methods, or that a computer may not be able to assess how close a given result is to the optimum. Nevertheless, employing machine learning tools remains valuable; one must simply be aware that these approaches do not always guarantee optimality. In such cases, alternative theoretical frameworks may be necessary.

Turing computability and Banach–Mazur computability are two central notions in the theory of computation. Every function that is Turing computable is also Banach–Mazur computable, meaning that Banach–Mazur computability subsumes Turing computability. However, the converse does not hold: not every Banach–Mazur computable function is Turing computable. In fact, if a function is not Banach–Mazur computable, then it cannot be computable under any other standard notion of computability. This underscores the foundational and maximal character of Banach–Mazur computability within the hierarchy of computability concepts. Moreover, as shown in [63], there exist even total functions—functions defined on all computable real numbers—that are Banach–Mazur computable but not Turing computable. For readers interested in a deeper understanding of computability theory—how to determine whether a function is computable, along with illustrative examples and detailed explanations—we recommend the comprehensive work by Soare [64] and Cooper’s New Computational Paradigms [65]. Practical implications of these theoretical analyses, especially their relevance to real-world applications, are further explored in [66], which may be of particular interest to those seeking connections between theory and practice.