Continuity of Channel Parameters and Operations under Various DMC Topologies ^†

Abstract

We study the continuity of many channel parameters and operations under various topologies on the space of equivalent discrete memoryless channels (DMC). We show that mutual information, channel capacity, Bhattacharyya parameter, probability of error of a fixed code and optimal probability of error for a given code rate and block length are continuous under various DMC topologies. We also show that channel operations such as sums, products, interpolations and Arıkan-style transformations are continuous.

1. Introduction

This paper is an extended version of our paper that is published in the International Symposium on Information Theory 2017 (ISIT 2017) [1].

Let $\mathcal{X}$ and $\mathcal{Y}$ be two finite sets, and let W be a fixed channel with input alphabet $\mathcal{X}$ and output alphabet $\mathcal{Y}$. It is well known that the input-output mutual information is continuous on the simplex of input probability distributions. Many other parameters that depend on the input probability distribution were shown to be continuous on the simplex in [2].

Polyanskiy studied in [3] the continuity of the Neyman–Pearson function for a binary hypothesis test that arises in the analysis of channel codes. He showed that for arbitrary input and output alphabets, this function is continuous in the input distribution in the total variation topology. He also showed that under some regularity assumptions, this function is continuous in the weak-∗ topology.

If $\mathcal{X}$ and $\mathcal{Y}$ are finite sets, the space of channels with input alphabet $\mathcal{X}$ and output alphabet $\mathcal{Y}$ can naturally be endowed with the topology of the Euclidean metric, or any other equivalent metric. It is well known that the channel capacity is continuous in this topology. If $\mathcal{X}$ and $\mathcal{Y}$ are arbitrary, one can construct a topology on the space of channels using the weak-∗ topology on the output alphabet. It was shown in [4] that the capacity is lower semi-continuous in this topology.

The continuity results that are mentioned in the previous paragraph do not take into account “equivalence” between channels. Two channels are said to be equivalent if they are degraded from each other. This means that each channel can be simulated from the other by local operations at the receiver. Two channels that are degraded from each other are completely equivalent from an operational point of view: both channels have exactly the same probability of error under optimal decoding for any fixed code. Moreover, any sub-optimal decoder for one channel can be transformed to a sub-optimal decoder for the other channel with the same probability of error and essentially the same computational complexity. This is why it makes sense, from an information-theoretic point of view, to identify equivalent channels and consider them as one point in the space of “equivalent channels”.

In [5], equivalent binary-input channels were identified with their L-density (i.e., the density of log-likelihood ratios). The space of equivalent binary-input channels was endowed with the topology of convergence in distribution of L-densities. Since the symmetric capacity (the symmetric capacity is the input-output mutual information with uniformly-distributed input) and the Bhattacharyya parameter can be written as an integral of a continuous function with respect to the L-density [5], it immediately follows that these parameters are continuous in the L-density topology.

In [6], many topologies were constructed for the space of equivalent channels sharing a fixed input alphabet. In this paper, we study the continuity of many channel parameters and operations under these topologies. The continuity of channel parameters and operations might be helpful in the following two problems:

• If a parameter (such as the optimal probability of error of a given code) is difficult to compute for a channel W, one can approximate it by computing the same parameter for a sequence of channels ${(W_n)}_{n\ge 0}$ that converges to W in some topology where the parameter is continuous.
• The study of the robustness of a communication system against the imperfect specification of the channel.

In Section 2, we introduce the preliminaries for this paper. In Section 3, we recall the main results of [6] that we need here. In Section 4, we introduce the channel parameters and operations that we investigate in this paper. In Section 5, we study the continuity of these parameters and operations in the quotient topology of the space of equivalent channels with fixed input and output alphabets. The continuity in the strong topology of the space of equivalent channels sharing the same input alphabet is studied in Section 6. Finally, the continuity in the noisiness/weak-∗ and the total variation topologies is studied in Section 7.

2. Preliminaries

We assume that the reader is familiar with the basic concepts of General Topology. The main concepts and theorems that we need can be found in the Preliminaries Section of [6].

2.1. Set-Theoretic Notations

For every integer $n\ge 1$, we denote the set $\{1,\dots ,n\}$ as $[n]$.

The set of mappings from a set A to a set B is denoted as $B^A$.

Let A be a subset of B. The indicator mapping ${𝟙}_{A,B}:B\to \{0,1\}$ of A in B is defined as:

$${𝟙}_{A,B}(x)={𝟙}_{x\in A}=\left\{\begin{array}{cc}1\hfill & \mathrm{if}x\in A,\hfill \\ 0\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

If the superset B is clear from the context, we simply write ${𝟙}_A$ to denote the indicator mapping of A in B.

The power set of B is the set of subsets of B. Since every subset of B can be identified with its indicator mapping, we denote the power set of B as ${\{0,1\}}^B=2^B$.

Let ${(A_i)}_{i\in I}$ be a collection of arbitrary sets indexed by I. The disjoint union of ${(A_i)}_{i\in I}$ is defined as ${\displaystyle \coprod _{i\in I}{A}_i=\bigcup _{i\in I}(A_i\times \{i\})}$. For every $i\in I$, the i-th-canonical injection is the mapping ${\displaystyle {\varphi }_i:A_i\to \coprod _{j\in I}{A}_j}$ defined as ${\varphi }_i(x_i)=(x_i,i)$. If no confusions can arise, we can identify $A_i$ with $A_i\times \{i\}$ through the canonical injection. Therefore, we can see $A_i$ as a subset of ${\displaystyle \coprod _{j\in I}{A}_j}$ for every $i\in I$.

Let R be an equivalence relation on a set T. For every $x\in T$, the set $\widehat{x}=\{y\in T:xRy\}$ is the R-equivalence class of x. The collection of R-equivalence classes, which we denote as $T/R$, forms a partition of T, and it is called the quotient space of T by R. The mapping ${Proj}_R:T\to T/R$ defined as ${Proj}_R(x)=\widehat{x}$ for every $x\in T$ is the projection mapping onto $T/R$.

2.2. Topological Notations

A topological space $(T,\mathcal{U})$ is said to be contractible to $x_0\in T$ if there exists a continuous mapping $H:T\times [0,1]\to T$ such that $H(x,0)=x$ and $H(x,1)=x_0$ for every $x\in T$, where $[0,1]$ is endowed with the Euclidean topology. $(T,\mathcal{U})$ is strongly contractible to $x_0\in T$ if we also have $H(x_0,t)=x_0$ for every $t\in [0,1]$.

Intuitively, T is contractible if it can be “continuously shrinked” to a single point $x_0$. If this “continuous shrinking” can be done without moving $x_0$, T is strongly contractible.

Note that contractibility is a very strong notion of connectedness: every contractible space is path-connected and simply connected. Moreover, all its homotopy, homology and cohomology groups of order $\ge 1$ are zero.

Let ${\{(T_i,{\mathcal{U}}_i)\}}_{i\in I}$ be a collection of topological spaces indexed by I. The product topology on ${\displaystyle \prod _{i\in I}{T}_i}$ is denoted by ${\displaystyle \underset{i\in I}{⨂}{\mathcal{U}}_i}$. The disjoint union topology on ${\displaystyle \coprod _{i\in I}{T}_i}$ is denoted by ${\displaystyle \underset{i\in I}{⨁}{\mathcal{U}}_i}$.

The following lemma is useful to show the continuity of many functions.

Lemma 1.

Let $(S,\mathcal{V})$ and $(T,\mathcal{U})$ be two compact topological spaces, and let $f:S\times T\to \mathbb{R}$ be a continuous function on $S\times T$. For every $s\in \mathcal{S}$ and every $ϵ>0$, there exists a neighborhood $V_s$ of s such that for every $s^{\prime }\in V_s$, we have:

$$\underset{t\in T}{sup}|f(s^{\prime },t)-f(s,t)|\le ϵ.$$

Proof. 

See Appendix A. ☐

2.3. Quotient Topology

Let $(T,\mathcal{U})$ be a topological space, and let R be an equivalence relation on T. The quotient topology on $T/R$ is the finest topology that makes the projection mapping ${Proj}_R$ continuous. It is given by:

$$\mathcal{U}/R=\left\{\widehat{U}\subset T/R:{Proj}_R^{-1}(\widehat{U})\in \mathcal{U}\right\}.$$

Lemma 2.

Let $f:T\to S$ be a continuous mapping from $(T,\mathcal{U})$ to $(S,\mathcal{V})$. If $f(x)=f(x^{\prime })$ for every $x,x^{\prime }\in T$ satisfying $xR{x}^{\prime }$, then we can define a transcendent mapping $f:T/R\to S$ such that $f(\widehat{x})=f(x^{\prime })$ for any $x^{\prime }\in \widehat{x}$. f is well defined on $T/R$. Moreover, f is a continuous mapping from $(T/R,\mathcal{U}/R)$ to $(S,\mathcal{V})$.

Let $(T,\mathcal{U})$ and $(S,\mathcal{V})$ be two topological spaces, and let R be an equivalence relation on T. Consider the equivalence relation $R^{\prime }$ on $T\times S$ defined as $(x_1,y_1)R^{\prime }(x_2,y_2)$ if and only if $x_{1}R{x}_2$ and $y_1=y_2$. A natural question to ask is whether the canonical bijection between $\left((T/R)\times S,(\mathcal{U}/R)\otimes \mathcal{V}\right)$ and $\left((T\times S)/R^{\prime },(\mathcal{U}\otimes \mathcal{V})/R^{\prime }\right)$ is a homeomorphism. It turns out that this is not the case in general. The following theorem, which is widely used in Algebraic Topology, provides a sufficient condition:

Theorem 1.

[7] If $(S,\mathcal{V})$ is locally compact and Hausdorff, then the canonical bijection between $((T/R)\times S$, $(\mathcal{U}/R)\otimes \mathcal{V})$ and $\left((T\times S)/R^{\prime },(\mathcal{U}\otimes \mathcal{V})/R^{\prime }\right)$ is a homeomorphism.

Corollary 1.

Let $(T,\mathcal{U})$ and $(S,\mathcal{V})$ be two topological spaces, and let $R_T$ and $R_S$ be two equivalence relations on T and S, respectively. Define the equivalence relation R on $T\times S$ as $(x_1,y_1)R(x_2,y_2)$ if and only if $x_1{R}_T{x}_2$ and $y_1{R}_S{y}_2$. If $(S,\mathcal{V})$ and $(T/R_T,\mathcal{U}/R_T)$ are locally compact and Hausdorff, then the canonical bijection between $\left((T/R_T)\times (S/R_S),(\mathcal{U}/R_T)\otimes (\mathcal{V}/R_S)\right)$ and $\left((T\times S)/R,(\mathcal{U}\otimes \mathcal{V})/R\right)$ is a homeomorphism.

Proof. 

We just need to apply Theorem 1 twice. Define the equivalence relation $R_T^{\prime }$ on $T\times S$ as follows: $(x_1,y_1)R_T^{\prime }(x_2,y_2)$ if and only if $x_1{R}_T{x}_2$ and $y_1=y_2$. Since $(S,\mathcal{V})$ is locally compact and Hausdorff, Theorem 1 implies that the canonical bijection from $\left((T/R_T)\times S,(\mathcal{U}/R_T)\otimes \mathcal{V}\right)$ to $\left((T\times S)/R_T^{\prime },(\mathcal{U}\otimes \mathcal{V})/R_T^{\prime }\right)$ is a homeomorphism. Let us identify these two spaces through the canonical bijection.

Now, define the equivalence relation $R_S^{\prime }$ on $(T/R_T)\times S$ as follows: $({\widehat{x}}_1,y_1)R_S^{\prime }({\widehat{x}}_2,y_2)$ if and only if ${\widehat{x}}_1={\widehat{x}}_2$ and $y_1{R}_S{y}_2$. Since $(T/R_T,\mathcal{U}/R_T)$ is locally compact and Hausdorff, Theorem 1 implies that the canonical bijection from $\left((T/R_T)\times (S/R_S),(\mathcal{U}/R_T)\otimes (\mathcal{V}/R_S)\right)$ to $\left(((T/R_T)\times S)/R_S^{\prime },((\mathcal{U}/R_T)\otimes \mathcal{V})/R_S^{\prime }\right)$ is a homeomorphism.

Since we identified $\left((T/R_T)\times S,(\mathcal{U}/R_T)\otimes \mathcal{V}\right)$ and $\left((T\times S)/R_T^{\prime },(\mathcal{U}\otimes \mathcal{V})/R_T^{\prime }\right)$ through the canonical bijection (which is a homeomorphism), $R_S^{\prime }$ can be seen as an equivalence relation on $\left((T\times S)/R_T^{\prime },(\mathcal{U}\otimes \mathcal{V})/R_T^{\prime }\right)$. It is easy to see that the canonical bijection from $\left(\left((T\times S)/R_T^{\prime }\right)/R_S^{\prime },\left((\mathcal{U}\otimes \mathcal{V})/R_T^{\prime }\right)/R_S^{\prime }\right)$ to $\left((T\times S)/R,(\mathcal{U}\otimes \mathcal{V})/R\right)$ is a homeomorphism. We conclude that the canonical bijection from $\left((T/R_T)\times (S/R_S),(\mathcal{U}/R_T)\otimes (\mathcal{V}/R_S)\right)$ to $\left((T\times S)/R,(\mathcal{U}\otimes \mathcal{V})/R\right)$ is a homeomorphism. ☐

2.4. Measure-Theoretic Notations

If $(M,\mathsf{\Sigma })$ is a measurable space, we denote the set of probability measures on $(M,\mathsf{\Sigma })$ as $\mathcal{P}(M,\mathsf{\Sigma })$. If the $\sigma$-algebra $\mathsf{\Sigma }$ is known from the context, we simply write $\mathcal{P}(M)$ to denote the set of probability measures.

If $P\in \mathcal{P}(M,\mathsf{\Sigma })$ and $\{x\}$ is a measurable singleton, we simply write $P(x)$ to denote $P(\{x\})$.

For every $P_1,P_2\in \mathcal{P}(M,\mathsf{\Sigma })$, the total variation distance between $P_1$ and $P_2$ is defined as:

$$\parallel P_1-P_2{\parallel }_{TV}=\underset{A\in \mathsf{\Sigma }}{sup}|P_1(A)-P_2(A)|.$$

• The push-forward probability measure:
  Let P be a probability measure on $(M,\mathsf{\Sigma })$, and let $f:M\to M^{\prime }$ be a measurable mapping from $(M,\mathsf{\Sigma })$ to another measurable space $(M^{\prime },{\mathsf{\Sigma }}^{\prime })$. The push-forward probability measure of P by f is the probability measure $f_{\#}P$ on $(M^{\prime },{\mathsf{\Sigma }}^{\prime })$ defined as $(f_{\#}P)(A^{\prime })=P(f^{-1}(A^{\prime }))$ for every $A^{\prime }\in {\mathsf{\Sigma }}^{\prime }$.
  A measurable mapping $g:M^{\prime }\to \mathbb{R}$ is integrable with respect to $f_{\#}P$ if and only if $g\circ f$ is integrable with respect to P. Moreover,

  $${\int }_{M^{\prime }}g·d(f_{\#}P)={\int }_M(g\circ f)·dP.$$
  The mapping $f_{\#}$ from $\mathcal{P}(M,\mathsf{\Sigma })$ to $\mathcal{P}(M^{\prime },{\mathsf{\Sigma }}^{\prime })$ is continuous if these spaces are endowed with the total variation topology:

  $$\begin{array}{cc}\hfill \parallel f_{\#}P-f_{\#}{P}^{\prime }{\parallel }_{TV}& \stackrel{(a)}{\le }{\parallel P-P^{\prime }\parallel }_{TV},\hfill \end{array}$$

  where (a) follows from Property 1 of [8].
• Probability measures on finite sets:
  We always endow finite sets with their finest $\sigma$-algebra, i.e., the power set. In this case, every probability measure is completely determined by its value on singletons, i.e., if P is a probability measure on a finite set $\mathcal{X}$, then for every $A\subset \mathcal{X}$, we have:

  $$P(A)=\sum _{x\in A}P(x).$$
  If $\mathcal{X}$ is a finite set, we denote the set of probability distributions on $\mathcal{X}$ as ${\Delta }_{\mathcal{X}}$. Note that ${\Delta }_{\mathcal{X}}$ is an $(|\mathcal{X}|-1)$-dimensional simplex in ${\mathbb{R}}^{\mathcal{X}}$. We always endow ${\Delta }_{\mathcal{X}}$ with the total variation distance and its induced topology. For every $p_1,p_2\in {\Delta }_{\mathcal{X}}$, we have:

  $$\parallel p_1-p_2{\parallel }_{TV}=\frac{1}{2}\sum _{x\in \mathcal{X}}|p_1(x)-p_2(x)|=\frac{1}{2}{\parallel p_1-p_2\parallel }_1.$$
• Products of probability measures:
  We denote the product of two measurable spaces $(M_1,{\mathsf{\Sigma }}_1)$ and $(M_2,{\mathsf{\Sigma }}_2)$ as $(M_1\times M_2,{\mathsf{\Sigma }}_1\otimes {\mathsf{\Sigma }}_2)$. If $P_1\in \mathcal{P}(M_1,{\mathsf{\Sigma }}_1)$ and $P_2\in \mathcal{P}(M_2,{\mathsf{\Sigma }}_2)$, we denote the product of $P_1$ and $P_2$ as $P_1\times P_2$.
  If $\mathcal{P}(M_1,{\mathsf{\Sigma }}_1)$, $\mathcal{P}(M_2,{\mathsf{\Sigma }}_2)$ and $\mathcal{P}(M_1\times M_2,{\mathsf{\Sigma }}_1\otimes {\mathsf{\Sigma }}_2)$ are endowed with the total variation topology, the mapping $(P_1,P_2)\to P_1\times P_2$ is a continuous mapping (see Appendix B).
• Borel sets and the support of a probability measure:
  Let $(T,\mathcal{U})$ be a Hausdorff topological space. The Borel $\sigma$-algebra of $(T,\mathcal{U})$ is the $\sigma$-algebra generated by $\mathcal{U}$. We denote the Borel $\sigma$-algebra of $(T,\mathcal{U})$ as $\mathcal{B}(T,\mathcal{U})$. If the topology $\mathcal{U}$ is known from the context, we simply write $\mathcal{B}(T)$ to denote the Borel $\sigma$-algebra. The sets in $\mathcal{B}(T)$ are called the Borel sets of T.

The support of a measure $P\in \mathcal{P}(T,\mathcal{B}(T))$ is the set of all points $x\in T$ for which every neighborhood has a strictly positive measure:

$$supp(P)=\{x\in T:P(O)>0\mathrm{for}\mathrm{every}\mathrm{neighborhood}O\mathrm{of}x\}.$$

If P is a probability measure on a Polish space, then $P\left(T\backslash supp(P)\right)=0$.

2.5. Random Mappings

Let M and $M^{\prime }$ be two arbitrary sets, and let ${\mathsf{\Sigma }}^{\prime }$ be a $\sigma$-algebra on $M^{\prime }$. A random mapping from M to $(M^{\prime },{\mathsf{\Sigma }}^{\prime })$ is a mapping R from M to $\mathcal{P}(M^{\prime },{\mathsf{\Sigma }}^{\prime })$. For every $x\in M$, $R(x)$ can be interpreted as the probability distribution of the random output given that the input is x.

Let $\mathsf{\Sigma }$ be a $\sigma$-algebra on M. We say that R is a measurable random mapping from $(M,\mathsf{\Sigma })$ to $(M^{\prime },{\mathsf{\Sigma }}^{\prime })$ if the mapping $R_B:M\to \mathbb{R}$ defined as $R_B(x)=(R(x))(B)$ is measurable for every $B\in {\mathsf{\Sigma }}^{\prime }$.

Note that this definition of measurability is consistent with the measurability of ordinary mappings: let f be a mapping from M to $M^{\prime }$, and let $D_f:M\to \mathcal{P}(M^{\prime },{\mathsf{\Sigma }}^{\prime })$ be the random mapping defined as $D_f(x)={\delta }_{f(x)}$ for every $x\in M$, where ${\delta }_{f(x)}\in \mathcal{P}(M^{\prime },{\mathsf{\Sigma }}^{\prime })$ is a Dirac measure centered at $f(x)$. We have:

$$\begin{array}{cc}\hfill D_f\mathrm{is}\mathrm{measurable}& \iff {(D_f)}_B\mathrm{is}\mathrm{measurable},\forall B\in {\mathsf{\Sigma }}^{\prime }\hfill \\ & \iff {({(D_f)}_B)}^{-1}(B^{\prime })\in \mathsf{\Sigma },\forall B^{\prime }\in \mathcal{B}(\mathbb{R}),\forall B\in {\mathsf{\Sigma }}^{\prime }\hfill \\ & \stackrel{(a)}{\iff }{({(D_f)}_B)}^{-1}(\{1\})\in \mathsf{\Sigma },\forall B\in {\mathsf{\Sigma }}^{\prime }\hfill \\ & \stackrel{(b)}{\iff }{f}^{-1}(B)\in \mathsf{\Sigma },\forall B\in {\mathsf{\Sigma }}^{\prime }\hfill \\ & \iff f\mathrm{is}\mathrm{measurable},\hfill \end{array}$$

where (a) and (b) follow from the fact that $({(D_f)}_B)(x)$ is either one or zero, depending on whether $f(x)\in B$ or not.

Let P be a probability measure on $(M,\mathsf{\Sigma })$, and let R be a measurable random mapping from $(M,\mathsf{\Sigma })$ to $(M^{\prime },{\mathsf{\Sigma }}^{\prime })$. The push-forward probability measure of P by R is the probability measure $R_{\#}P$ on $(M^{\prime },{\mathsf{\Sigma }}^{\prime })$ defined as:

$$(R_{\#}P)(B)={\int }_M{R}_B·dP,\forall B\in {\mathsf{\Sigma }}^{\prime }.$$

Note that this definition is consistent with the push-forward of ordinary mappings: if f and $D_f$ are as above, then for every $B\in {\mathsf{\Sigma }}^{\prime }$, we have:

$$({(D_f)}_{\#}P)(B)={\int }_M{(D_f)}_B·dP={\int }_M({𝟙}_B\circ f)·dP={\int }_{M^{\prime }}{𝟙}_B·d(f_{\#}P)=(f_{\#}P)(B).$$

Proposition 1.

Let R be a measurable random mapping from $(M,\mathsf{\Sigma })$ to $(M^{\prime },{\mathsf{\Sigma }}^{\prime })$. If $g:M^{\prime }\to {\mathbb{R}}^{+}\cup \{+\infty \}$ is a ${\mathsf{\Sigma }}^{\prime }$-measurable mapping, then the mapping ${\displaystyle x\to {\int }_{M^{\prime }}g(y)·d(R(x))(y)}$ is a measurable mapping from $(M,\mathsf{\Sigma })$ to ${\mathbb{R}}^{+}\cup \{+\infty \}$. Moreover, for every $P\in \mathcal{P}(M,\mathsf{\Sigma })$, we have:

$${\int }_{M^{\prime }}g·d(R_{\#}P)={\int }_M\left({\int }_{M^{\prime }}g(y)·d(R(x))(y)\right)dP(x).$$

Proof. 

See Appendix C. ☐

Corollary 2.

If $g:M^{\prime }\to \mathbb{R}$ is bounded and ${\mathsf{\Sigma }}^{\prime }$-measurable, then the mapping:

$${\displaystyle x\to {\int }_{M^{\prime }}g(y)·d(R(x))(y)}$$

is bounded and Σ-measurable. Moreover, for every $P\in \mathcal{P}(M,\mathsf{\Sigma })$, we have:

$${\int }_{M^{\prime }}g·d(R_{\#}P)={\int }_M\left({\int }_{M^{\prime }}g(y)·d(R(x))(y)\right)dP(x).$$

Proof. 

Write $g=g^{+}-g^{-}$ (where $g^{+}=max\{g,0\}$ and $g^{-}=max\{-g,0\}$), and use the fact that every bounded measurable function is integrable over any probability distribution. ☐

Lemma 3.

For every measurable random mapping R from $(M,\mathsf{\Sigma })$ to $(M^{\prime },{\mathsf{\Sigma }}^{\prime })$, the push-forward mapping $R_{\#}$ is continuous from $\mathcal{P}(M,\mathsf{\Sigma })$ to $\mathcal{P}(M^{\prime },{\mathsf{\Sigma }}^{\prime })$ under the total variation topology.

Proof. 

See Appendix D. ☐

Lemma 4.

Let $\mathcal{U}$ be a Polish (This assumption can be dropped. We assumed that $\mathcal{U}$ is Polish just to avoid working with Moore–Smith nets.) topology on M, and let ${\mathcal{U}}^{\prime }$ be an arbitrary topology on $M^{\prime }$. Let R be a measurable random mapping from $(M,\mathcal{B}(M))$ to $(M^{\prime },\mathcal{B}(M^{\prime }))$. Moreover, assume that R is a continuous mapping from $(M,\mathcal{U})$ to $\mathcal{P}(M^{\prime },\mathcal{B}(M^{\prime }))$ when the latter space is endowed with the weak-∗ topology. Under these assumptions, the push-forward mapping $R_{\#}$ is continuous from $\mathcal{P}(M,\mathcal{B}(M))$ to $\mathcal{P}(M^{\prime },\mathcal{B}(M^{\prime }))$ under the weak-∗ topology.

Proof. 

See Appendix D. ☐

2.6. Meta-Probability Measures

Let $\mathcal{X}$ be a finite set. A meta-probability measure on $\mathcal{X}$ is a probability measure on the Borel sets of ${\Delta }_{\mathcal{X}}$. It is called a meta-probability measure because it is a probability measure on the space of probability distributions on $\mathcal{X}$.

We denote the set of meta-probability measures on $\mathcal{X}$ as $\mathcal{MP}(\mathcal{X})$. Clearly, $\mathcal{MP}(\mathcal{X})=\mathcal{P}({\Delta }_{\mathcal{X}})$.

A meta-probability measure MP on $\mathcal{X}$ is said to be balanced if it satisfies:

$${\int }_{{\Delta }_{\mathcal{X}}}p·dMP(p)={\pi }_{\mathcal{X}},$$

where ${\pi }_{\mathcal{X}}$ is the uniform probability distribution on $\mathcal{X}$.

We denote the set of all balanced meta-probability measures on $\mathcal{X}$ as ${\mathcal{MP}}_b(\mathcal{X})$. The set of all balanced and finitely-supported meta-probability measures on $\mathcal{X}$ is denoted as ${\mathcal{MP}}_{bf}(\mathcal{X})$.

The following lemma is useful to show the continuity of functions defined on $\mathcal{MP}(\mathcal{X})$.

Lemma 5.

Let $(S,\mathcal{V})$ be a compact topological space, and let $f:S\times {\Delta }_{\mathcal{X}}\to \mathbb{R}$ be a continuous function on $S\times {\Delta }_{\mathcal{X}}$. The mapping $F:S\times \mathcal{MP}(\mathcal{X})\to \mathbb{R}$ defined as:

$$F(s,MP)={\int }_{{\Delta }_{\mathcal{X}}}f(s,p)·dMP(p)$$

is continuous, where $\mathcal{MP}(\mathcal{X})$ is endowed with the weak-∗ topology.

Proof. 

See Appendix E. ☐

Let f be a mapping from a finite set $\mathcal{X}$ to another finite set ${\mathcal{X}}^{\prime }$. f induces a push-forward mapping $f_{\#}$ taking probability distributions in ${\Delta }_{\mathcal{X}}$ to probability distributions in ${\Delta }_{{\mathcal{X}}^{\prime }}$. $f_{\#}$ is continuous because ${\Delta }_{\mathcal{X}}$ and ${\Delta }_{{\mathcal{X}}^{\prime }}$ are endowed with the total variation distance. $f_{\#}$ in turn induces another push-forward mapping taking meta-probability measures in $\mathcal{MP}(\mathcal{X})$ to meta-probability measures in $\mathcal{MP}({\mathcal{X}}^{\prime })$. We denote this mapping as $f_{\#\#}$, and we call it the meta-push-forward mapping induced by f. Since $f_{\#}$ is a continuous mapping from ${\Delta }_{\mathcal{X}}$ to ${\Delta }_{{\mathcal{X}}^{\prime }}$, $f_{\#\#}$ is a continuous mapping from $\mathcal{MP}(\mathcal{X})$ to $\mathcal{MP}({\mathcal{X}}^{\prime })$ under both the weak-∗ and the total variation topologies.

Let ${\mathcal{X}}_1$ and ${\mathcal{X}}_2$ be two finite sets. Let $Mul:{\Delta }_{{\mathcal{X}}_1}\times {\Delta }_{{\mathcal{X}}_2}\to {\Delta }_{{\mathcal{X}}_1\times {\mathcal{X}}_2}$ be defined as $Mul(p_1,p_2)=p_1\times p_2$. For every ${MP}_1\in \mathcal{MP}({\mathcal{X}}_1)$ and ${MP}_2\in \mathcal{MP}({\mathcal{X}}_2)$, we define the tensor product of ${MP}_1$ and ${MP}_2$ as ${MP}_1\otimes {MP}_2={Mul}_{\#}({MP}_1\times {MP}_2)\in \mathcal{MP}({\mathcal{X}}_1\times {\mathcal{X}}_2)$.

Note that since ${\Delta }_{{\mathcal{X}}_1}$, ${\Delta }_{{\mathcal{X}}_2}$ and ${\Delta }_{{\mathcal{X}}_1\times {\mathcal{X}}_2}$ are endowed with the total variation topology, $Mul(p_1,p_2)=p_1\times p_2$ is a continuous mapping from ${\Delta }_{{\mathcal{X}}_1}\times {\Delta }_{{\mathcal{X}}_2}$ to ${\Delta }_{{\mathcal{X}}_1\times {\mathcal{X}}_2}$. Therefore, ${Mul}_{\#}$ is a continuous mapping from $\mathcal{P}({\Delta }_{{\mathcal{X}}_1}\times {\Delta }_{{\mathcal{X}}_2})$ to $\mathcal{P}({\Delta }_{{\mathcal{X}}_1\times {\mathcal{X}}_2})=\mathcal{MP}({\mathcal{X}}_1\times {\mathcal{X}}_2)$ under both the weak-∗ and the total variation topologies. On the other hand, Appendix B and Appendix F imply that the mapping $({MP}_1,{MP}_2)\to {MP}_1\times {MP}_2$ from $\mathcal{MP}({\mathcal{X}}_1)\times \mathcal{MP}({\mathcal{X}}_2)$ to $\mathcal{P}({\Delta }_{{\mathcal{X}}_1}\times {\Delta }_{{\mathcal{X}}_2})$ is continuous under both the weak-∗ and the total variation topologies. We conclude that the tensor product is continuous under both of these topologies.

3. The Space of Equivalent Channels

In this section, we summarize the main results of [6].

3.1. Space of Channels from $\mathcal{X}$ to $\mathcal{Y}$

A discrete memoryless channel W is a three-tuple $W=(\mathcal{X},\mathcal{Y},p_W)$ where $\mathcal{X}$ is a finite set that is called the input alphabet of W, $\mathcal{Y}$ is a finite set that is called the output alphabet of W and $p_W:\mathcal{X}\times \mathcal{Y}\to [0,1]$ is a function satisfying ${\displaystyle \forall x\in \mathcal{X},\sum _{y\in \mathcal{Y}}{p}_W(x,y)=1}$.

For every $(x,y)\in \mathcal{X}\times \mathcal{Y}$, we denote $p_W(x,y)$ as $W(y|x)$, which we interpret as the conditional probability of receiving y at the output, given that x is the input.

Let ${DMC}_{\mathcal{X},\mathcal{Y}}$ be the set of all channels having $\mathcal{X}$ as the input alphabet and $\mathcal{Y}$ as the output alphabet.

For every $W,W^{\prime }\in {DMC}_{\mathcal{X},\mathcal{Y}}$, define the distance between W and $W^{\prime }$ as follows:

$$d_{\mathcal{X},\mathcal{Y}}(W,W^{\prime })=\frac{1}{2}\underset{x\in \mathcal{X}}{max}\sum _{y\in \mathcal{Y}}|W^{\prime }(y|x)-W(y|x)|.$$

We always endow ${DMC}_{\mathcal{X},\mathcal{Y}}$ with the metric distance $d_{\mathcal{X},\mathcal{Y}}$. This metric makes ${DMC}_{\mathcal{X},\mathcal{Y}}$ a compact path-connected metric space. The metric topology on ${DMC}_{\mathcal{X},\mathcal{Y}}$ that is induced by $d_{\mathcal{X},\mathcal{Y}}$ is denoted as ${\mathcal{T}}_{\mathcal{X},\mathcal{Y}}$.

3.2. Equivalence between Channels

Let $W\in {DMC}_{\mathcal{X},\mathcal{Y}}$ and $W^{\prime }\in {DMC}_{\mathcal{X},\mathcal{Z}}$ be two channels having the same input alphabet. We say that $W^{\prime }$ is degraded from W if there exists a channel $V\in {DMC}_{\mathcal{Y},\mathcal{Z}}$ such that:

$$W^{\prime }(z|x)=\sum _{y\in \mathcal{Y}}V(z|y)W(y|x).$$

W and $W^{\prime }$ are said to be equivalent if each one is degraded from the other.

Let ${\Delta }_{\mathcal{X}}$ and ${\Delta }_{\mathcal{Y}}$ be the space of probability distributions on $\mathcal{X}$ and $\mathcal{Y}$, respectively. Define $P_W^o\in {\Delta }_{\mathcal{Y}}$ as ${\displaystyle P_W^o(y)=\frac{1}{|\mathcal{X}|}\sum _{x\in \mathcal{X}}W(y|x)}$ for every $y\in \mathcal{Y}$. The image of W is the set of output-symbols $y\in \mathcal{Y}$ having strictly positive probabilities:

$$Im(W)=\{y\in \mathcal{Y}:P_W^o(y)>0\}.$$

For every $y\in Im(W)$, define $W_y^{-1}\in {\Delta }_{\mathcal{X}}$ as follows:

$${\displaystyle W_y^{-1}(x)=\frac{W(y|x)}{|\mathcal{X}|P_W^o(y)},\forall x\in \mathcal{X}.}$$

For every $(x,y)\in \mathcal{X}\times Im(W)$, we have $W(y|x)=|\mathcal{X}|P_W^o(y)W_y^{-1}(x)$. On the other hand, if $x\in \mathcal{X}$ and $y\in \mathcal{Y}\backslash Im(W)$, we have $W(y|x)=0$. This shows that $P_W^o$ and the collection ${\{W_y^{-1}\}}_{y\in Im(W)}$ uniquely determine W.

The Blackwell measure (denoted ${MP}_W$) of W is a meta-probability measure on $\mathcal{X}$ defined as:

$${MP}_W=\sum _{y\in Im(W)}{P}_W^o(y){\delta }_{W_y^{-1}},$$

where ${\delta }_{W_y^{-1}}$ is a Dirac measure centered at $W_y^{-1}$. In an earlier version of this work, I called ${MP}_W$ the posterior meta-probability distribution of W. Maxim Raginsky thankfully brought to my attention the fact that ${MP}_W$ is called the Blackwell measure.

It is known that a meta-probability measure MP on $\mathcal{X}$ is the Blackwell measure of some discrete memoryless channels (DMC) with input alphabet $\mathcal{X}$ if and only if it is balanced and finitely supported [9].

It is also known that two channels $W\in {DMC}_{\mathcal{X},\mathcal{Y}}$ and $W^{\prime }\in {DMC}_{\mathcal{X},\mathcal{Z}}$ are equivalent if and only if ${MP}_W={MP}_{W^{\prime }}$ [9].

3.3. Space of Equivalent Channels from $\mathcal{X}$ to $\mathcal{Y}$

Let $\mathcal{X}$ and $\mathcal{Y}$ be two finite sets. Define the equivalence relation $R_{\mathcal{X},\mathcal{Y}}^{(o)}$ on ${DMC}_{\mathcal{X},\mathcal{Y}}$ as follows:

$$\forall W,W^{\prime }\in {DMC}_{\mathcal{X},\mathcal{Y}},W{R}_{\mathcal{X},\mathcal{Y}}^{(o)}{W}^{\prime }\iff W\mathrm{is}\mathrm{equivalent}\mathrm{to}{W}^{\prime }.$$

The space of equivalent channels with input alphabet $\mathcal{X}$ and output alphabet $\mathcal{Y}$ is the quotient of ${DMC}_{\mathcal{X},\mathcal{Y}}$ by the equivalence relation:

$${DMC}_{\mathcal{X},\mathcal{Y}}^{(o)}={DMC}_{\mathcal{X},\mathcal{Y}}/R_{\mathcal{X},\mathcal{Y}}^{(o)}.$$

Quotient topology:

We define the topology ${\mathcal{T}}_{\mathcal{X},\mathcal{Y}}^{(o)}$ on ${DMC}_{\mathcal{X},\mathcal{Y}}^{(o)}$ as the quotient topology ${\mathcal{T}}_{\mathcal{X},\mathcal{Y}}/R_{\mathcal{X},\mathcal{Y}}^{(o)}$. We always associate ${DMC}_{\mathcal{X},\mathcal{Y}}^{(o)}$ with the quotient topology ${\mathcal{T}}_{\mathcal{X},\mathcal{Y}}^{(o)}$.

We have shown in [6] that ${DMC}_{\mathcal{X},\mathcal{Y}}^{(o)}$ is a compact, path-connected and metrizable space.

If ${\mathcal{Y}}_1$ and ${\mathcal{Y}}_2$ are two finite sets of the same size, there exists a canonical homeomorphism between ${DMC}_{\mathcal{X},{\mathcal{Y}}_1}^{(o)}$ and ${DMC}_{\mathcal{X},{\mathcal{Y}}_2}^{(o)}$ [6]. This allows us to identify ${DMC}_{\mathcal{X},\mathcal{Y}}^{(o)}$ with ${DMC}_{\mathcal{X},[n]}^{(o)}$, where $n=|\mathcal{Y}|$ and $[n]=\{1,\dots ,n\}$.

Moreover, for every $1\le n\le m$, there exists a canonical subspace of ${DMC}_{\mathcal{X},[m]}^{(o)}$ that is homeomorphic to ${DMC}_{\mathcal{X},[n]}^{(o)}$ [6]. Therefore, we can consider ${DMC}_{\mathcal{X},[n]}^{(o)}$ as a compact subspace of ${DMC}_{\mathcal{X},[m]}^{(o)}$.

Noisiness metric:

For every $m\ge 1$, let ${\Delta }_{[m]\times \mathcal{X}}$ be the space of probability distributions on $[m]\times \mathcal{X}$. Let $\mathcal{Y}$ be a finite set, and let $W\in {DMC}_{\mathcal{X},\mathcal{Y}}$. For every $p\in {\Delta }_{[m]\times \mathcal{X}}$, define $P_c(p,W)$ as follows:

$$P_c(p,W)=\underset{D\in {DMC}_{\mathcal{Y},[m]}}{sup}\sum _{\begin{array}{c}u\in [m],\\ x\in \mathcal{X},\\ y\in \mathcal{Y}\end{array}}p(u,x)W(y|x)D(u|y).$$

The quantity $P_c(p,W)$ depends only on the $R_{\mathcal{X},\mathcal{Y}}^{(o)}$-equivalence class of W (see [6]). Therefore, if $\widehat{W}\in {DMC}_{\mathcal{X},\mathcal{Y}}^{(o)}$, we can define $P_c(p,\widehat{W}):=P_c(p,W^{\prime })$ for any $W^{\prime }\in \widehat{W}$.

Define the noisiness distance $d_{\mathcal{X},\mathcal{Y}}^{(o)}:{DMC}_{\mathcal{X},\mathcal{Y}}^{(o)}\times {DMC}_{\mathcal{X},\mathcal{Y}}^{(o)}\to {\mathbb{R}}^{+}$ as follows:

$$d_{\mathcal{X},\mathcal{Y}}^{(o)}({\widehat{W}}_1,{\widehat{W}}_2)=\underset{\begin{array}{c}m\ge 1,\\ p\in {\Delta }_{[m]\times \mathcal{X}}\end{array}}{sup}|P_c(p,{\widehat{W}}_1)-P_c(p,{\widehat{W}}_2)|.$$

We have shown in [6] that $({DMC}_{\mathcal{X},\mathcal{Y}}^{(o)},{\mathcal{T}}_{\mathcal{X},\mathcal{Y}}^{(o)})$ is topologically equivalent to $({DMC}_{\mathcal{X},\mathcal{Y}}^{(o)},d_{\mathcal{X},\mathcal{Y}}^{(o)})$.

3.4. Space of Equivalent Channels with Input Alphabet $\mathcal{X}$

The space of channels with input alphabet $\mathcal{X}$ is defined as:

$${DMC}_{\mathcal{X},\ast }={\displaystyle \coprod _{n\ge 1}}{DMC}_{\mathcal{X},[n]}.$$

We define the equivalence relation $R_{\mathcal{X},\ast }^{(o)}$ on ${DMC}_{\mathcal{X},\ast }$ as follows:

$$\forall W,W^{\prime }\in {DMC}_{\mathcal{X},\ast },W{R}_{\mathcal{X},\ast }^{(o)}{W}^{\prime }\iff W\mathrm{is}\mathrm{equivalent}\mathrm{to}{W}^{\prime }.$$

The space of equivalent channels with input alphabet $\mathcal{X}$ is the quotient of ${DMC}_{\mathcal{X},\ast }$ by the equivalence relation:

$${DMC}_{\mathcal{X},\ast }^{(o)}={DMC}_{\mathcal{X},\ast }/R_{\mathcal{X},\ast }^{(o)}.$$

For every $n\ge 1$ and every $W\in {DMC}_{\mathcal{X},[n]}$, we identify the $R_{\mathcal{X},[n]}^{(o)}$-equivalence class of W with the $R_{\mathcal{X},\ast }^{(o)}$-equivalence class of it. This allows us to consider ${DMC}_{\mathcal{X},[n]}^{(o)}$ as a subspace of ${DMC}_{\mathcal{X},\ast }^{(o)}$. Moreover,

$${DMC}_{\mathcal{X},\ast }^{(o)}=\bigcup _{n\ge 1}{DMC}_{\mathcal{X},[n]}^{(o)}.$$

Since any two equivalent channels have the same Blackwell measure, we can define the Blackwell measure of $\widehat{W}\in {DMC}_{\mathcal{X},\ast }^{(o)}$ as ${MP}_{\widehat{W}}={MP}_{W^{\prime }}$ for any $W^{\prime }\in \widehat{W}$. The rank of $\widehat{W}\in {DMC}_{\mathcal{X},\ast }^{(o)}$ is the size of the support of its Blackwell measure:

$$rank(\widehat{W})=|supp({MP}_{\widehat{W}})|.$$

We have:

$${DMC}_{\mathcal{X},[n]}^{(o)}=\{\widehat{W}\in {DMC}_{\mathcal{X},\ast }^{(o)}:rank(\widehat{W})\le n\}.$$

A topology $\mathcal{T}$ on ${DMC}_{\mathcal{X},\ast }^{(o)}$ is said to be natural if and only if it induces the quotient topology ${\mathcal{T}}_{\mathcal{X},[n]}^{(o)}$ on ${DMC}_{\mathcal{X},[n]}^{(o)}$ for every $n\ge 1$.

Every natural topology is $\sigma$-compact, separable and path-connected [6]. On the other hand, if $|\mathcal{X}|\ge 2$, a Hausdorff natural topology is not Baire, and it is not locally compact anywhere [6]. This implies that no natural topology can be completely metrized if $|\mathcal{X}|\ge 2$.

Strong topology on ${DMC}_{\mathcal{X},\ast }^{(o)}$:

We associate ${DMC}_{\mathcal{X},\ast }$ with the disjoint union topology ${\displaystyle {\mathcal{T}}_{s,\mathcal{X},\ast }:=\underset{n\ge 1}{⨁}{\mathcal{T}}_{\mathcal{X},[n]}}$. The space $({DMC}_{\mathcal{X},\ast },{\mathcal{T}}_{s,\mathcal{X},\ast })$ is disconnected, metrizable and $\sigma$-compact [6].

The strong topology ${\mathcal{T}}_{s,\mathcal{X},\ast }^{(o)}$ on ${DMC}_{\mathcal{X},\ast }^{(o)}$ is the quotient of ${\mathcal{T}}_{s,\mathcal{X},\ast }$ by $R_{\mathcal{X},\ast }^{(o)}$:

$${\mathcal{T}}_{s,\mathcal{X},\ast }^{(o)}={\mathcal{T}}_{s,\mathcal{X},\ast }/R_{\mathcal{X},\ast }^{(o)}.$$

We call open and closed sets in $({DMC}_{\mathcal{X},\ast }^{(o)},{\mathcal{T}}_{s,\mathcal{X},\ast }^{(o)})$ as strongly-open and strongly-closed sets, respectively. If A is a subset of ${DMC}_{\mathcal{X},\ast }^{(o)}$, then A is strongly open if and only if $A\cap {DMC}_{\mathcal{X},[n]}^{(o)}$ is open in ${DMC}_{\mathcal{X},[n]}^{(o)}$ for every $n\ge 1$. Similarly, A is strongly closed if and only if $A\cap {DMC}_{\mathcal{X},[n]}^{(o)}$ is closed in ${DMC}_{\mathcal{X},[n]}^{(o)}$ for every $n\ge 1$.

We have shown in [6] that ${\mathcal{T}}_{s,\mathcal{X},\ast }^{(o)}$ is the finest natural topology. The strong topology is sequential, compactly generated and $T_4$ [6]. On the other hand, if $|\mathcal{X}|\ge 2$, the strong topology is not first-countable anywhere [6]; hence, it is not metrizable.

Noisiness metric:

Define the noisiness metric on ${DMC}_{\mathcal{X},\ast }^{(o)}$ as follows:

$$d_{\mathcal{X},\ast }^{(o)}(\widehat{W},{\widehat{W}}^{\prime }):=d_{\mathcal{X},[n]}^{(o)}(\widehat{W},{\widehat{W}}^{\prime })\mathrm{where}n\ge 1\mathrm{satisfies}\widehat{W},{\widehat{W}}^{\prime }\in {DMC}_{\mathcal{X},[n]}^{(o)}.$$

$d_{\mathcal{X},\ast }^{(o)}(\widehat{W},{\widehat{W}}^{\prime })$ is well-defined because $d_{\mathcal{X},[n]}^{(o)}(\widehat{W},{\widehat{W}}^{\prime })$ does not depend on $n\ge 1$ as long as $\widehat{W},{\widehat{W}}^{\prime }\in {DMC}_{\mathcal{X},[n]}^{(o)}$. We can also express $d_{\mathcal{X},\ast }^{(o)}$ as follows:

$$d_{\mathcal{X},\ast }^{(o)}(\widehat{W},{\widehat{W}}^{\prime })=\underset{\begin{array}{c}m\ge 1,\\ p\in {\Delta }_{[m]\times \mathcal{X}}\end{array}}{sup}|P_c(p,\widehat{W})-P_c(p,{\widehat{W}}^{\prime })|.$$

The metric topology on ${DMC}_{\mathcal{X},\ast }^{(o)}$ that is induced by $d_{\mathcal{X},\ast }^{(o)}$ is called the noisiness topology on ${DMC}_{\mathcal{X},\ast }^{(o)}$, and it is denoted as ${\mathcal{T}}_{\mathcal{X},\ast }^{(o)}$. We have shown in [6] that ${\mathcal{T}}_{\mathcal{X},\ast }^{(o)}$ is a natural topology that is strictly coarser than ${\mathcal{T}}_{s,\mathcal{X},\ast }^{(o)}$.

Topologies from Blackwell measures:

The mapping $\widehat{W}\to {MP}_{\widehat{W}}$ is a bijection from ${DMC}_{\mathcal{X},\ast }^{(o)}$ to ${\mathcal{MP}}_{bf}(\mathcal{X})$. We call this mapping the canonical bijection from ${DMC}_{\mathcal{X},\ast }^{(o)}$ to ${\mathcal{MP}}_{bf}(\mathcal{X})$.

Since ${\Delta }_{\mathcal{X}}$ is a metric space, there are many standard ways to construct topologies on $\mathcal{MP}(\mathcal{X})$. If we choose any of these standard topologies on $\mathcal{MP}(\mathcal{X})$ and then relativize it to the subspace ${\mathcal{MP}}_{bf}(\mathcal{X})$, we can construct topologies on ${DMC}_{\mathcal{X},\ast }^{(o)}$ through the canonical bijection.

In [6], we studied the weak-∗ and the total variation topologies. We showed that the weak-∗ topology is exactly the same as the noisiness topology.

The total-variation metric distance $d_{TV,\mathcal{X},\ast }^{(o)}$ on ${DMC}_{\mathcal{X},\ast }^{(o)}$ is defined as:

$$d_{TV,\mathcal{X},\ast }^{(o)}(\widehat{W},{\widehat{W}}^{\prime })={\parallel {MP}_{\widehat{W}}-{MP}_{{\widehat{W}}^{\prime }}\parallel }_{TV}.$$

The total-variation topology ${\mathcal{T}}_{TV,\mathcal{X},\ast }^{(o)}$ is the metric topology that is induced by $d_{TV,\mathcal{X},\ast }^{(o)}$ on ${DMC}_{\mathcal{X},\ast }^{(o)}$. We proved in [6] that if $|\mathcal{X}|\ge 2$, we have:

• ${\mathcal{T}}_{TV,\mathcal{X},\ast }^{(o)}$ is not natural, nor Baire, hence it is not completely metrizable.
• ${\mathcal{T}}_{TV,\mathcal{X},\ast }^{(o)}$ is not locally compact anywhere.

4. Channel Parameters and Operations

4.1. Useful Parameters

Let ${\Delta }_{\mathcal{X}}$ be the space of probability distributions on $\mathcal{X}$. For every $p\in {\Delta }_{\mathcal{X}}$ and every $W\in {DMC}_{\mathcal{X},\mathcal{Y}}$, define $I(p,W)$ as the mutual information $I(X;Y)$, where X is distributed as p and Y is the output of W when X is the input. The mutual information is computed using the natural logarithm. The capacity of W is defined as ${\displaystyle C(W)=\underset{p\in {\Delta }_{\mathcal{X}}}{sup}I(p,W)}$.

For every $p\in {\Delta }_{\mathcal{X}}$, the error probability of the MAP decoder of W under prior p is defined as:

$$P_e(p,W)=1-\sum _{y\in \mathcal{Y}}\underset{x\in \mathcal{X}}{max}\{p(x)W(y|x)\}.$$

Clearly, $0\le P_e(p,W)\le 1$.

For every $W\in {DMC}_{\mathcal{X},\mathcal{Y}}$, define the Bhattacharyya parameter of W as:

$$Z(W)=\left\{\begin{array}{cc}{\displaystyle \frac{1}{|\mathcal{X}|·(|\mathcal{X}|-1)}\sum _{\begin{array}{c}{x}_1,x_2\in \mathcal{X},\\ x_1\ne x_2\end{array}}\sum _{y\in \mathcal{Y}}\sqrt{W(y|x_1)W(y|x_2)},}\hfill & \mathrm{if}|\mathcal{X}|\ge 2\hfill \\ 0\hfill & \mathrm{if}|\mathcal{X}|=1.\hfill \end{array}\right.$$

It is easy to see that $0\le Z(W)\le 1$.

It was shown in [10,11] that ${\displaystyle \frac{1}{4}Z{(W)}^2\le P_e({\pi }_{\mathcal{X}},W)\le (|\mathcal{X}|-1)Z(W)}$, where ${\pi }_{\mathcal{X}}$ is the uniform distribution on $\mathcal{X}$.

An $(n,M)$-code $\mathcal{C}$ on the alphabet $\mathcal{X}$ is a subset of ${\mathcal{X}}^n$ such that $|\mathcal{C}|=M$. The integer n is the block length of $\mathcal{C}$, and M is the size of the code. The rate of $\mathcal{C}$ is $\frac{1}{n}logM$, and it is measured in nats. The error probability of the ML decoder for the code $\mathcal{C}$ when it is used for a channel $W\in {DMC}_{\mathcal{X},\mathcal{Y}}$ is given by:

$$P_{e,\mathcal{C}}(W)=1-\frac{1}{|\mathcal{C}|}\sum _{y_1^n\in {\mathcal{Y}}^n}\underset{x_1^n\in \mathcal{C}}{max}\left\{\prod _{i=1}^{n}W(y_i|x_i)\right\}.$$

The optimal error probability of $(n,M)$-codes for a channel W is given by:

$$P_{e,n,M}(W)=\underset{\begin{array}{c}\mathcal{C}\subset {\mathcal{X}}^n,\\ |\mathcal{C}|=M\end{array}}{min}{P}_{e,\mathcal{C}}(W).$$

The following proposition shows that all the above parameters are continuous:

Proposition 2.

We have:

• $I:{\Delta }_{\mathcal{X}}\times {DMC}_{\mathcal{X},\mathcal{Y}}\to {\mathbb{R}}^{+}$ is continuous, concave in p and convex in W.
• $C:{DMC}_{\mathcal{X},\mathcal{Y}}\to {\mathbb{R}}^{+}$ is continuous and convex.
• $P_e:{\Delta }_{\mathcal{X}}\times {DMC}_{\mathcal{X},\mathcal{Y}}\to [0,1]$ is continuous, concave in p and concave in W.
• $Z:{DMC}_{\mathcal{X},\mathcal{Y}}\to [0,1]$ is continuous.
• For every code $\mathcal{C}$ on $\mathcal{X}$, $P_{e,\mathcal{C}}:{DMC}_{\mathcal{X},\mathcal{Y}}\to [0,1]$ is continuous.
• For every $n>0$ and every $1\le M\le {|\mathcal{X}|}^n$, the mapping $P_{e,n,M}:{DMC}_{\mathcal{X},\mathcal{Y}}\to [0,1]$ is continuous.

Proof. 

These facts are well known, especially the continuity of I, its concavity in p and its convexity in W [12]. Since C is the supremum of a family of mappings that are convex in W, it is also convex in W. For a proof of the continuity of C, see Appendix G. The continuity of Z, $P_e$ and $P_{e,\mathcal{C}}$ follows immediately from their definitions. Moreover, since $P_{e,n,M}$ is the minimum of a finite number of continuous mappings, it is continuous. The concavity of $P_e$ in p and in W can also be easily seen from the definition. ☐

4.2. Channel Operations

If $W\in {DMC}_{\mathcal{X},\mathcal{Y}}$ and $V\in {DMC}_{\mathcal{Y},\mathcal{Z}}$, we define the composition $V\circ W\in {DMC}_{\mathcal{X},\mathcal{Z}}$ of W and V as follows:

$$(V\circ W)(z|x)=\sum _{y\in \mathcal{Y}}V(z|y)W(y|x).\forall x\in \mathcal{X},\forall z\in \mathcal{Z}.$$

For every function $f:\mathcal{X}\to \mathcal{Y}$, define the deterministic channel $D_f\in {DMC}_{\mathcal{X},\mathcal{Y}}$ as follows:

$$D_f(y|x)=\left\{\begin{array}{cc}1\hfill & \mathrm{if}y=f(x),\hfill \\ 0\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

It is easy to see that if $f:\mathcal{X}\to \mathcal{Y}$ and $g:\mathcal{Y}\to \mathcal{Z}$, then $D_g\circ D_f=D_{g\circ f}$.

For every two channels $W_1\in {DMC}_{{\mathcal{X}}_1,{\mathcal{Y}}_1}$ and $W_2\in {DMC}_{{\mathcal{X}}_2,{\mathcal{Y}}_2}$, define the channel sum $W_1\oplus W_2\in {DMC}_{{\mathcal{X}}_1\coprod {\mathcal{X}}_2,{\mathcal{Y}}_1\coprod {\mathcal{Y}}_2}$ of $W_1$ and $W_2$ as:

$$(W_1\oplus W_2)(y,i|x,j)=\left\{\begin{array}{cc}{W}_i(y|x)\hfill & \mathrm{if}i=j,\hfill \\ 0\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

$W_1\oplus W_2$ arises when the transmitter has two channels $W_1$ and $W_2$ at its disposal, and it can use exactly one of them at each channel use. It is an easy exercise to check that $e^{C(W_1\oplus W_2)}=e^{C(W_1)}+e^{C(W_2)}$ (remember that we compute the mutual information using the natural logarithm).

We define the channel product $W_1\otimes W_2\in {DMC}_{{\mathcal{X}}_1\times {\mathcal{X}}_2,{\mathcal{Y}}_1\times {\mathcal{Y}}_2}$ of $W_1$ and $W_2$ as:

$$(W_1\otimes W_2)(y_1,y_2|x_1,x_2)=W_1(y_1|x_1)W_2(y_2|x_2).$$

$W_1\otimes W_2$ arises when the transmitter has two channels $W_1$ and $W_2$ at its disposal, and it uses both of them at each channel use. It is an easy exercise to check that $C(W_1\otimes W_2)=C(W_1)+C(W_2)$, or equivalently $e^{C(W_1\otimes W_2)}=e^{C(W_1)}·e^{C(W_2)}$. Channel sums and products were first introduced by Shannon in [13].

For every $W_1\in {DMC}_{\mathcal{X},{\mathcal{Y}}_1}$, $W_2\in {DMC}_{\mathcal{X},{\mathcal{Y}}_2}$ and every $0\le \alpha \le 1$, we define the $\alpha$-interpolation $[\alpha W_1,(1-\alpha )W_2]\in {DMC}_{\mathcal{X},{\mathcal{Y}}_1\coprod {\mathcal{Y}}_2}$ between $W_1$ and $W_2$ as:

$$[\alpha W_1,(1-\alpha )W_2](y,i|x)=\left\{\begin{array}{cc}\alpha W_1(y|x)\hfill & \mathrm{if}i=1,\hfill \\ (1-\alpha )W_2(y|x)\hfill & \mathrm{if}i=2.\hfill \end{array}\right.$$

Channel interpolation arises when a channel behaves as $W_1$ with probability $\alpha$ and as $W_2$ with probability $1-\alpha$. The transmitter has no control on which behavior the channel chooses, but on the other hand, the receiver knows which one was chosen. Channel interpolations were used in [14] to construct interpolations between polar codes and Reed–Muller codes.

Now, fix a binary operation ∗ on $\mathcal{X}$. For every $W\in {DMC}_{\mathcal{X},\mathcal{Y}}$, define $W^{-}\in {DMC}_{\mathcal{X},{\mathcal{Y}}^2}$ and $W^{+}\in {DMC}_{\mathcal{X},{\mathcal{Y}}^2\times \mathcal{X}}$ as:

$$W^{-}(y_1,y_2|u_1)=\frac{1}{|\mathcal{X}|}\sum _{u_2\in \mathcal{X}}W(y_1|u_1\ast u_2)W(y_2|u_2),$$

and:

$$W^{+}(y_1,y_2,u_1|u_2)=\frac{1}{|\mathcal{X}|}W(y_1|u_1\ast u_2)W(y_2|u_2).$$

These operations generalize Arıkan’s polarization transformations [15].

Proposition 3.

We have:

• The mapping $(W,V)\to V\circ W$ from ${DMC}_{\mathcal{X},\mathcal{Y}}\times {DMC}_{\mathcal{Y},\mathcal{Z}}$ to ${DMC}_{\mathcal{X},\mathcal{Z}}$ is continuous.
• The mapping $(W_1,W_2)\to W_1\oplus W_2$ from ${DMC}_{{\mathcal{X}}_1,{\mathcal{Y}}_1}\times {DMC}_{{\mathcal{X}}_2,{\mathcal{Y}}_2}$ to the space ${DMC}_{{\mathcal{X}}_1\coprod {\mathcal{X}}_2,{\mathcal{Y}}_1\coprod {\mathcal{Y}}_2}$ is continuous.
• The mapping $(W_1,W_2)\to W_1\otimes W_2$ from ${DMC}_{{\mathcal{X}}_1,{\mathcal{Y}}_1}\times {DMC}_{{\mathcal{X}}_2,{\mathcal{Y}}_2}$ to ${DMC}_{{\mathcal{X}}_1\times {\mathcal{X}}_2,{\mathcal{Y}}_1\times {\mathcal{Y}}_2}$ is continuous.
• The mapping $(W_1,W_2,\alpha )\to [\alpha W_1,(1-\alpha )W_2]$ from ${DMC}_{\mathcal{X},{\mathcal{Y}}_1}\times {DMC}_{\mathcal{X},{\mathcal{Y}}_2}\times [0,1]$ to ${DMC}_{\mathcal{X},{\mathcal{Y}}_1\coprod {\mathcal{Y}}_2}$ is continuous.
• For any binary operation ∗ on $\mathcal{X}$, the mapping $W\to W^{-}$ from ${DMC}_{\mathcal{X},\mathcal{Y}}$ to ${DMC}_{\mathcal{X},{\mathcal{Y}}^2}$ is continuous.
• For any binary operation ∗ on $\mathcal{X}$, the mapping $W\to W^{+}$ from ${DMC}_{\mathcal{X},\mathcal{Y}}$ to ${DMC}_{\mathcal{X},{\mathcal{Y}}^2\times \mathcal{X}}$ is continuous.

Proof. 

The continuity immediately follows from the definitions. ☐

5. Continuity on ${\mathbf{DMC}}_{\mathcal{X},\mathcal{Y}}^{(\mathit{o})}$

It is well known that the parameters defined in Section 4.1 depend only on the $R_{\mathcal{X},\mathcal{Y}}^{(o)}$-equivalence class of W. Therefore, we can define those parameters for any $\widehat{W}\in {DMC}_{\mathcal{X},\mathcal{Y}}^{(o)}$ through the transcendent mapping (defined in Lemma 2). The following proposition shows that those parameters are continuous on ${DMC}_{\mathcal{X},\mathcal{Y}}^{(o)}$:

Proposition 4.

We have:

• $I:{\Delta }_{\mathcal{X}}\times {DMC}_{\mathcal{X},\mathcal{Y}}^{(o)}\to {\mathbb{R}}^{+}$ is continuous and concave in p.
• $C:{DMC}_{\mathcal{X},\mathcal{Y}}^{(o)}\to {\mathbb{R}}^{+}$ is continuous.
• $P_e:{\Delta }_{\mathcal{X}}\times {DMC}_{\mathcal{X},\mathcal{Y}}^{(o)}\to [0,1]$ is continuous and concave in p.
• $Z:{DMC}_{\mathcal{X},\mathcal{Y}}^{(o)}\to [0,1]$ is continuous.
• For every code $\mathcal{C}$ on $\mathcal{X}$, $P_{e,\mathcal{C}}:{DMC}_{\mathcal{X},\mathcal{Y}}^{(o)}\to [0,1]$ is continuous.
• For every $n>0$ and every $1\le M\le {|\mathcal{X}|}^n$, the mapping $P_{e,n,M}:{DMC}_{\mathcal{X},\mathcal{Y}}^{(o)}\to [0,1]$ is continuous.

Proof. 

Since the corresponding parameters are continuous on ${DMC}_{\mathcal{X},\mathcal{Y}}$ (Proposition 2), Lemma 2 implies that they are continuous on ${DMC}_{\mathcal{X},\mathcal{Y}}^{(o)}$. The only cases that need a special treatment are those of I and Z. We will only prove the continuity of I since the proof of continuity of Z is similar.

Define the relation R on ${\Delta }_{\mathcal{X}}\times {DMC}_{\mathcal{X},\mathcal{Y}}$ as:

$$(p_1,W_1)R(p_2,W_2)\iff p_1=p_2\mathrm{and}{W}_1{R}_{\mathcal{X},\mathcal{Y}}^{(o)}{W}_2.$$

It is easy to see that $I(p,W)$ depends only on the R-equivalence class of $(p,W)$. Since I is continuous on ${\Delta }_{\mathcal{X}}\times {DMC}_{\mathcal{X},\mathcal{Y}}$, Lemma 2 implies that the transcendent mapping of I is continuous on $({\Delta }_{\mathcal{X}}\times {DMC}_{\mathcal{X},\mathcal{Y}})/R$. On the other hand, since ${\Delta }_{\mathcal{X}}$ is locally compact, Theorem 1 implies that $({\Delta }_{\mathcal{X}}\times {DMC}_{\mathcal{X},\mathcal{Y}})/R$ can be identified with ${\Delta }_{\mathcal{X}}\times ({DMC}_{\mathcal{X},\mathcal{Y}}/R_{\mathcal{X},\mathcal{Y}}^{(o)})={\Delta }_{\mathcal{X}}\times {DMC}_{\mathcal{X},\mathcal{Y}}^{(o)}$, and the two spaces have the same topology. Therefore, I is continuous on ${\Delta }_{\mathcal{X}}\times {DMC}_{\mathcal{X},\mathcal{Y}}^{(o)}$. ☐

With the exception of channel composition, all the channel operations that were defined in Section 4.2 can also be “quotiented”. We just need to realize that the equivalence class of the resulting channel depends only on the equivalence classes of the channels that were used in the operation. Let us illustrate this in the case of channel sums:

Let $W_1,W_1^{\prime }\in {DMC}_{{\mathcal{X}}_1,{\mathcal{Y}}_1}$ and $W_2,W_2^{\prime }\in {DMC}_{{\mathcal{X}}_2,{\mathcal{Y}}_2}$ and assume that $W_1$ is degraded from $W_1^{\prime }$ and $W_2$ is degraded from $W_2^{\prime }$. There exists $V_1\in {DMC}_{{\mathcal{Y}}_1,{\mathcal{Y}}_1}$ and $V_2\in {DMC}_{{\mathcal{Y}}_2,{\mathcal{Y}}_2}$ such that $W_1=V_1\circ W_1^{\prime }$ and $W_2=V_2\circ W_2^{\prime }$. It is easy to see that $W_1\oplus W_2=(V_1\oplus V_2)\circ (W_1^{\prime }\oplus W_2^{\prime })$, which shows that $W_1\oplus W_2$ is degraded from $W_1^{\prime }\oplus W_2^{\prime }$. This was proven by Shannon in [16].

Therefore, if $W_1$ is equivalent to $W_1^{\prime }$ and $W_2$ is equivalent to $W_2^{\prime }$, then $W_1\oplus W_2$ is equivalent to $W_1^{\prime }\oplus W_2^{\prime }$. This allows us to define the channel sum for every ${\widehat{W}}_1\in {DMC}_{{\mathcal{X}}_1,{\mathcal{Y}}_1}^{(o)}$ and every ${\overline{W}}_2\in {DMC}_{{\mathcal{X}}_2,{\mathcal{Y}}_2}^{(o)}$ as ${\widehat{W}}_1\oplus {\overline{W}}_2=\widetilde{W_1^{\prime }\oplus W_2^{\prime }}\in {DMC}_{{\mathcal{X}}_1\coprod {\mathcal{X}}_2,{\mathcal{Y}}_1\coprod {\mathcal{Y}}_2}^{(o)}$ for any $W_1^{\prime }\in {\widehat{W}}_1$ and any $W_2^{\prime }\in {\overline{W}}_2$, where $\widetilde{W_1^{\prime }\oplus W_2^{\prime }}$ is the $R_{{\mathcal{X}}_1\coprod {\mathcal{X}}_2,{\mathcal{Y}}_1\coprod {\mathcal{Y}}_2}^{(o)}$-equivalence class of $W_1^{\prime }\oplus W_2^{\prime }$.

With the exception of channel composition, we can “quotient” all the channel operations of Section 4.2 in a similar fashion. Moreover, we can show that they are continuous:

Proposition 5.

We have:

• The mapping $({\widehat{W}}_1,{\overline{W}}_2)\to {\widehat{W}}_1\oplus {\overline{W}}_2$ from ${DMC}_{{\mathcal{X}}_1,{\mathcal{Y}}_1}^{(o)}\times {DMC}_{{\mathcal{X}}_2,{\mathcal{Y}}_2}^{(o)}$ to ${DMC}_{{\mathcal{X}}_1\coprod {\mathcal{X}}_2,{\mathcal{Y}}_1\coprod {\mathcal{Y}}_2}^{(o)}$ is continuous.
• The mapping $({\widehat{W}}_1,{\overline{W}}_2)\to {\widehat{W}}_1\otimes {\overline{W}}_2$ from ${DMC}_{{\mathcal{X}}_1,{\mathcal{Y}}_1}^{(o)}\times {DMC}_{{\mathcal{X}}_2,{\mathcal{Y}}_2}^{(o)}$ to ${DMC}_{{\mathcal{X}}_1\times {\mathcal{X}}_2,{\mathcal{Y}}_1\times {\mathcal{Y}}_2}^{(o)}$ is continuous.
• The mapping $({\widehat{W}}_1,{\overline{W}}_2,\alpha )\to [\alpha {\widehat{W}}_1,(1-\alpha ){\overline{W}}_2]$ from ${DMC}_{\mathcal{X},{\mathcal{Y}}_1}^{(o)}\times {DMC}_{\mathcal{X},{\mathcal{Y}}_2}^{(o)}\times [0,1]$ to ${DMC}_{\mathcal{X},{\mathcal{Y}}_1\coprod {\mathcal{Y}}_2}^{(o)}$ is continuous.
• For any binary operation ∗ on $\mathcal{X}$, the mapping $\widehat{W}\to {\widehat{W}}^{-}$ from ${DMC}_{\mathcal{X},\mathcal{Y}}^{(o)}$ to ${DMC}_{\mathcal{X},{\mathcal{Y}}^2}^{(o)}$ is continuous.
• For any binary operation ∗ on $\mathcal{X}$, the mapping $\widehat{W}\to {\widehat{W}}^{+}$ from ${DMC}_{\mathcal{X},\mathcal{Y}}^{(o)}$ to ${DMC}_{\mathcal{X},{\mathcal{Y}}^2\times \mathcal{X}}^{(o)}$ is continuous.

Proof. 

We only prove the continuity of the channel sum because the proof of continuity of the other operations is similar.

Let $Proj:{DMC}_{{\mathcal{X}}_1\coprod {\mathcal{X}}_2,{\mathcal{Y}}_1\coprod {\mathcal{Y}}_2}\to {DMC}_{{\mathcal{X}}_1\coprod {\mathcal{X}}_2,{\mathcal{Y}}_1\coprod {\mathcal{Y}}_2}^{(o)}$ be the projection onto the $R_{{\mathcal{X}}_1\coprod {\mathcal{X}}_2,{\mathcal{Y}}_1\coprod {\mathcal{Y}}_2}^{(o)}$-equivalence classes. Define the mapping $f:{DMC}_{{\mathcal{X}}_1,{\mathcal{Y}}_1}\times {DMC}_{{\mathcal{X}}_2,{\mathcal{Y}}_2}\to {DMC}_{{\mathcal{X}}_1\coprod {\mathcal{X}}_2,{\mathcal{Y}}_1\coprod {\mathcal{Y}}_2}^{(o)}$ as $f(W_1,W_2)=Proj(W_1\oplus W_2)$. Clearly, f is continuous.

Now, define the equivalence relation R on ${DMC}_{{\mathcal{X}}_1,{\mathcal{Y}}_1}\times {DMC}_{{\mathcal{X}}_2,{\mathcal{Y}}_2}$ as:

$$(W_1,W_2)R(W_1^{\prime },W_2^{\prime })\iff W_1{R}_{{\mathcal{X}}_1,{\mathcal{Y}}_1}^{(o)}{W}_1^{\prime }\mathrm{and}{W}_2{R}_{{\mathcal{X}}_2,{\mathcal{Y}}_2}^{(o)}{W}_2^{\prime }.$$

The discussion before the proposition shows that $f(W_1,W_2)=Proj(W_1\oplus W_2)$ depends only on the R-equivalence class of $(W_1,W_2)$. Lemma 2 now shows that the transcendent map of f defined on $({DMC}_{{\mathcal{X}}_1,{\mathcal{Y}}_1}\times {DMC}_{{\mathcal{X}}_2,{\mathcal{Y}}_2})/R$ is continuous.

Notice that $({DMC}_{{\mathcal{X}}_1,{\mathcal{Y}}_1}\times {DMC}_{{\mathcal{X}}_2,{\mathcal{Y}}_2})/R$ can be identified with ${DMC}_{{\mathcal{X}}_1,{\mathcal{Y}}_1}^{(o)}\times {DMC}_{{\mathcal{X}}_2,{\mathcal{Y}}_2}^{(o)}$. Therefore, we can define f on ${DMC}_{{\mathcal{X}}_1,{\mathcal{Y}}_1}^{(o)}\times {DMC}_{{\mathcal{X}}_2,{\mathcal{Y}}_2}^{(o)}$ through this identification. Moreover, since ${DMC}_{{\mathcal{X}}_1,{\mathcal{Y}}_1}$ and ${DMC}_{{\mathcal{X}}_2,{\mathcal{Y}}_2}^{(o)}$ are locally compact and Hausdorff, Corollary 1 implies that the canonical bijection between $({DMC}_{{\mathcal{X}}_1,{\mathcal{Y}}_1}\times {DMC}_{{\mathcal{X}}_2,{\mathcal{Y}}_2})/R$ and ${DMC}_{{\mathcal{X}}_1,{\mathcal{Y}}_1}^{(o)}\times {DMC}_{{\mathcal{X}}_2,{\mathcal{Y}}_2}^{(o)}$ is a homeomorphism.

Now, since the mapping f on ${DMC}_{{\mathcal{X}}_1,{\mathcal{Y}}_1}^{(o)}\times {DMC}_{{\mathcal{X}}_2,{\mathcal{Y}}_2}^{(o)}$ is just the channel sum, we conclude that the mapping $({\widehat{W}}_1,{\overline{W}}_2)\to {\widehat{W}}_1\oplus {\overline{W}}_2$ from ${DMC}_{{\mathcal{X}}_1,{\mathcal{Y}}_1}^{(o)}\times {DMC}_{{\mathcal{X}}_2,{\mathcal{Y}}_2}^{(o)}$ to ${DMC}_{{\mathcal{X}}_1\coprod {\mathcal{X}}_2,{\mathcal{Y}}_1\coprod {\mathcal{Y}}_2}^{(o)}$ is continuous. ☐

6. Continuity in the Strong Topology

The following lemma provides a way to check whether a mapping defined on $({DMC}_{\mathcal{X},\ast }^{(o)},{\mathcal{T}}_{s,\mathcal{X},\ast }^{(o)})$ is continuous:

Lemma 6.

Let $(S,\mathcal{V})$ be an arbitrary topological space. A mapping $f:{DMC}_{\mathcal{X},\ast }^{(o)}\to S$ is continuous on $({DMC}_{\mathcal{X},\ast }^{(o)},{\mathcal{T}}_{s,\mathcal{X},\ast }^{(o)})$ if and only if it is continuous on $({DMC}_{\mathcal{X},[n]}^{(o)},{\mathcal{T}}_{\mathcal{X},[n]}^{(o)})$ for every $n\ge 1$.

Proof. 

$$\begin{array}{cc}\hfill f\mathrm{is}\mathrm{continuous}\mathrm{on}({DMC}_{\mathcal{X},\ast }^{(o)},{\mathcal{T}}_{s,\mathcal{X},\ast }^{(o)})& \iff f^{-1}(V)\in {\mathcal{T}}_{s,\mathcal{X},\ast }^{(o)}\forall V\in \mathcal{V}\hfill \\ & \iff f^{-1}(V)\cap {DMC}_{\mathcal{X},[n]}^{(o)}\in {\mathcal{T}}_{\mathcal{X},[n]}^{(o)}\forall n\ge 1,\forall V\in \mathcal{V}\hfill \\ & \iff f\mathrm{is}\mathrm{continuous}\mathrm{on}({DMC}_{\mathcal{X},[n]}^{(o)},{\mathcal{T}}_{\mathcal{X},[n]}^{(o)})\forall n\ge 1.\hfill \end{array}$$

☐

Since the channel parameters I, C, $P_e$, Z, $P_{e,\mathcal{C}}$ and $P_{e,n,M}$ are defined on ${DMC}_{\mathcal{X},[n]}^{(o)}$ for every $n\ge 1$ (see Section 5), they are also defined on ${DMC}_{\mathcal{X},\ast }^{(o)}={\displaystyle \bigcup _{n\ge 1}}{DMC}_{\mathcal{X},[n]}^{(o)}$. The following proposition shows that those parameters are continuous in the strong topology:

Proposition 6.

Let ${\mathcal{U}}_{\mathcal{X}}$ be the standard topology on ${\Delta }_{\mathcal{X}}$. We have:

• $I:{\Delta }_{\mathcal{X}}\times {DMC}_{\mathcal{X},\ast }^{(o)}\to {\mathbb{R}}^{+}$ is continuous on $({\Delta }_{\mathcal{X}}\times {DMC}_{\mathcal{X},\ast }^{(o)},{\mathcal{U}}_{\mathcal{X}}\otimes {\mathcal{T}}_{s,\mathcal{X},\ast }^{(o)})$ and concave in p.
• $C:{DMC}_{\mathcal{X},\ast }^{(o)}\to {\mathbb{R}}^{+}$ is continuous on $({DMC}_{\mathcal{X},\ast }^{(o)},{\mathcal{T}}_{s,\mathcal{X},\ast }^{(o)})$.
• $P_e:{\Delta }_{\mathcal{X}}\times {DMC}_{\mathcal{X},\ast }^{(o)}\to [0,1]$ is continuous on $({\Delta }_{\mathcal{X}}\times {DMC}_{\mathcal{X},\ast }^{(o)},{\mathcal{U}}_{\mathcal{X}}\otimes {\mathcal{T}}_{s,\mathcal{X},\ast }^{(o)})$ and concave in p.
• $Z:{DMC}_{\mathcal{X},\ast }^{(o)}\to [0,1]$ is continuous on $({DMC}_{\mathcal{X},\ast }^{(o)},{\mathcal{T}}_{s,\mathcal{X},\ast }^{(o)})$.
• For every code $\mathcal{C}$ on $\mathcal{X}$, $P_{e,\mathcal{C}}:{DMC}_{\mathcal{X},\ast }^{(o)}\to [0,1]$ is continuous on $({DMC}_{\mathcal{X},\ast }^{(o)},{\mathcal{T}}_{s,\mathcal{X},\ast }^{(o)})$.
• For every $n>0$ and every $1\le M\le {|\mathcal{X}|}^n$, the mapping $P_{e,n,M}:{DMC}_{\mathcal{X},\ast }^{(o)}\to [0,1]$ is continuous on $({DMC}_{\mathcal{X},\ast }^{(o)},{\mathcal{T}}_{s,\mathcal{X},\ast }^{(o)})$.

Proof. 

The continuity of $C,Z,P_{e,C}$ and $P_{e,n,M}$ immediately follows from Proposition 4 and Lemma 6. Since the proofs of the continuity of I and Z are similar, we only prove the continuity for I.

Due to the distributivity of the product with respect to disjoint unions, we have:

$$\begin{array}{c}\hfill {\Delta }_{\mathcal{X}}\times {DMC}_{\mathcal{X},\ast }={\displaystyle \coprod _{n\ge 1}}\left({\Delta }_{\mathcal{X}}\times {DMC}_{\mathcal{X},[n]}\right),\end{array}$$

and:

$$\begin{array}{c}\hfill {\mathcal{U}}_{\mathcal{X}}\otimes {\mathcal{T}}_{s,\mathcal{X},\ast }={\displaystyle \underset{n\ge 1}{⨁}}\left({\mathcal{U}}_{\mathcal{X}}\otimes {\mathcal{T}}_{\mathcal{X},[n]}\right).\end{array}$$

Therefore, $({\Delta }_{\mathcal{X}}\times {DMC}_{\mathcal{X},\ast },{\mathcal{U}}_{\mathcal{X}}\otimes {\mathcal{T}}_{s,\mathcal{X},\ast })$ is the disjoint union of the spaces ${({\Delta }_{\mathcal{X}}\times {DMC}_{\mathcal{X},[n]})}_{n\ge 1}$. Moreover, I is continuous on ${\Delta }_{\mathcal{X}}\times {DMC}_{\mathcal{X},[n]}$ for every $n\ge 1$. We conclude that I is continuous on $({\Delta }_{\mathcal{X}}\times {DMC}_{\mathcal{X},\ast },{\mathcal{U}}_{\mathcal{X}}\otimes {\mathcal{T}}_{s,\mathcal{X},\ast })$.

Define the relation R on ${\Delta }_{\mathcal{X}}\times {DMC}_{\mathcal{X},\ast }$ as follows: $(p_1,W_1)R(p_2,W_2)$ if and only if $p_1=p_2$ and $W_1{R}_{\mathcal{X},\ast }^{(o)}{W}_2$. Since $I(p,W)$ depends only on the R-equivalence class of $(p,W)$, Lemma 2 shows that the transcendent map of I is a continuous mapping from $\left(({\Delta }_{\mathcal{X}}\times {DMC}_{\mathcal{X},\ast })/R,({\mathcal{U}}_{\mathcal{X}}\otimes {\mathcal{T}}_{s,\mathcal{X},\ast })/R\right)$ to ${\mathbb{R}}^{+}$. On the other hand, since ${\Delta }_{\mathcal{X}}$ is locally compact and Hausdorff, Theorem 1 implies that $\left(({\Delta }_{\mathcal{X}}\times {DMC}_{\mathcal{X},\ast })/R,({\mathcal{U}}_{\mathcal{X}}\otimes {\mathcal{T}}_{s,\mathcal{X},\ast })/R\right)$ can be identified with $\left({\Delta }_{\mathcal{X}}\times ({DMC}_{\mathcal{X},\ast }/R_{\mathcal{X},\ast }^{(o)}),{\mathcal{U}}_{\mathcal{X}}\otimes ({\mathcal{T}}_{s,\mathcal{X},\ast }/R_{\mathcal{X},\ast }^{(o)})\right)=({\Delta }_{\mathcal{X}}\times {DMC}_{\mathcal{X},\ast }^{(o)},{\mathcal{U}}_{\mathcal{X}}\otimes {\mathcal{T}}_{s,\mathcal{X},\ast }^{(o)})$. Therefore, I is continuous on $({\Delta }_{\mathcal{X}}\times {DMC}_{\mathcal{X},\ast }^{(o)},{\mathcal{U}}_{\mathcal{X}}\otimes {\mathcal{T}}_{s,\mathcal{X},\ast }^{(o)})$. ☐

It is also possible to extend the definition of all the channel operations that were defined in Section 5 to ${DMC}_{\mathcal{X},\ast }^{(o)}$. Moreover, it is possible to show that many channel operations are continuous in the strong topology:

Proposition 7.

Assume that all equivalent channel spaces are endowed with the strong topology. We have:

• The mapping $({\widehat{W}}_1,{\overline{W}}_2)\to {\widehat{W}}_1\oplus {\overline{W}}_2$ from ${DMC}_{{\mathcal{X}}_1,\ast }^{(o)}\times {DMC}_{{\mathcal{X}}_2,{\mathcal{Y}}_2}^{(o)}$ to ${DMC}_{{\mathcal{X}}_1\coprod {\mathcal{X}}_2,\ast }^{(o)}$ is continuous.
• The mapping $({\widehat{W}}_1,{\overline{W}}_2)\to {\widehat{W}}_1\otimes {\overline{W}}_2$ from ${DMC}_{{\mathcal{X}}_1,\ast }^{(o)}\times {DMC}_{{\mathcal{X}}_2,{\mathcal{Y}}_2}^{(o)}$ to ${DMC}_{{\mathcal{X}}_1\times {\mathcal{X}}_2,\ast }^{(o)}$ is continuous.
• The mapping $({\widehat{W}}_1,{\overline{W}}_2,\alpha )\to [\alpha {\widehat{W}}_1,(1-\alpha ){\overline{W}}_2]$ from ${DMC}_{\mathcal{X},\ast }\times {DMC}_{\mathcal{X},{\mathcal{Y}}_2}^{(o)}\times [0,1]$ to ${DMC}_{\mathcal{X},\ast }^{(o)}$ is continuous.
• For any binary operation ∗ on $\mathcal{X}$, the mapping $\widehat{W}\to {\widehat{W}}^{-}$ from ${DMC}_{\mathcal{X},\ast }^{(o)}$ to ${DMC}_{\mathcal{X},\ast }^{(o)}$ is continuous.
• For any binary operation ∗ on $\mathcal{X}$, the mapping $\widehat{W}\to {\widehat{W}}^{+}$ from ${DMC}_{\mathcal{X},\ast }^{(o)}$ to ${DMC}_{\mathcal{X},\ast }^{(o)}$ is continuous.

Proof. 

We only prove the continuity of the channel interpolation because the proof of the continuity of other operations is similar.

Let $\mathcal{U}$ be the standard topology on $[0,1]$. Due to the distributivity of the product with respect to disjoint unions, we have:

$${DMC}_{\mathcal{X},\ast }\times {DMC}_{\mathcal{X},{\mathcal{Y}}_2}\times [0,1]={\displaystyle \coprod _{n\ge 1}}({DMC}_{\mathcal{X},[n]}\times {DMC}_{\mathcal{X},{\mathcal{Y}}_2}\times [0,1]),$$

and:

$${\mathcal{T}}_{s,\mathcal{X},\ast }\otimes {\mathcal{T}}_{\mathcal{X},{\mathcal{Y}}_2}\otimes \mathcal{U}={\displaystyle \underset{n\ge 1}{⨁}}\left({\mathcal{T}}_{\mathcal{X},[n]}\otimes {\mathcal{T}}_{\mathcal{X},{\mathcal{Y}}_2}\otimes \mathcal{U}\right).$$

Therefore, the space ${DMC}_{\mathcal{X},\ast }\times {DMC}_{\mathcal{X},{\mathcal{Y}}_2}\times [0,1]$ is the topological disjoint union of the spaces ${({DMC}_{\mathcal{X},[n]}\times {DMC}_{\mathcal{X},{\mathcal{Y}}_2}\times [0,1])}_{n\ge 1}$.

For every $n\ge 1$, let ${Proj}_n$ be the projection onto the $R_{\mathcal{X},[n]\coprod {\mathcal{Y}}_2}^{(o)}$-equivalence classes, and let $i_n$ be the canonical injection from ${DMC}_{\mathcal{X},[n]\coprod {\mathcal{Y}}_2}^{(o)}$ to ${DMC}_{\mathcal{X},\ast }^{(o)}$.

Define the mapping $f:{DMC}_{\mathcal{X},\ast }\times {DMC}_{\mathcal{X},{\mathcal{Y}}_2}\times [0,1]\to {DMC}_{\mathcal{X},\ast }^{(o)}$ as:

$$f(W_1,W_2,\alpha )=i_n({Proj}_n([\alpha W_1,(1-\alpha )W_2]))=[\alpha {\widehat{W}}_1,(1-\alpha ){\overline{W}}_2],$$

where n is the unique integer satisfying $W_1\in {DMC}_{\mathcal{X},[n]}$. ${\widehat{W}}_1$ and ${\overline{W}}_2$ are the $R_{\mathcal{X},[n]}^{(o)}$ and $R_{\mathcal{X},{\mathcal{Y}}_2}^{(o)}$-equivalence classes of $W_1$ and $W_2$, respectively.

Due to Proposition 3 and due to the continuity of ${Proj}_n$ and $i_n$, the mapping f is continuous on ${DMC}_{\mathcal{X},[n]}\times {DMC}_{\mathcal{X},{\mathcal{Y}}_2}\times [0,1]$ for every $n\ge 1$. Therefore, f is continuous on $({DMC}_{\mathcal{X},\ast }\times {DMC}_{\mathcal{X},{\mathcal{Y}}_2}\times [0,1],{\mathcal{T}}_{s,\mathcal{X},\ast }\otimes {\mathcal{T}}_{\mathcal{X},{\mathcal{Y}}_2}\otimes \mathcal{U})$.

Let $R^{\prime }$ be the equivalence relation defined on ${DMC}_{\mathcal{X},\ast }\times {DMC}_{\mathcal{X},{\mathcal{Y}}_2}$ as follows: $(W_1,W_2)R^{\prime }(W_1^{\prime },W_2^{\prime })$ if and only if $W_1{R}_{\mathcal{X},\ast }^{(o)}{W}_1^{\prime }$ and $W_2{R}_{\mathcal{X},{\mathcal{Y}}_2}^{(o)}{W}_2^{\prime }$. Furthermore, define the equivalence relation R on ${DMC}_{\mathcal{X},\ast }\times {DMC}_{\mathcal{X},{\mathcal{Y}}_2}\times [0,1]$ as follows: $(W_1,W_2,\alpha )R(W_1^{\prime },W_2^{\prime },{\alpha }^{\prime })$ if and only if $(W_1,W_2)R^{\prime }(W_1^{\prime },W_2^{\prime })$ and $\alpha ={\alpha }^{\prime }$.

Since $f(W_1,W_2,\alpha )$ depends only on the R-equivalence class of $(W_1,W_2,\alpha )$, Lemma 2 implies that the transcendent mapping of f is continuous on $({DMC}_{\mathcal{X},\ast }\times {DMC}_{\mathcal{X},{\mathcal{Y}}_2}\times [0,1])/R$.

Since $[0,1]$ is Hausdorff and locally compact, Theorem 1 implies that the canonical bijection from $({DMC}_{\mathcal{X},\ast }\times {DMC}_{\mathcal{X},{\mathcal{Y}}_2}\times [0,1])/R$ to $\left(({DMC}_{\mathcal{X},\ast }\times {DMC}_{\mathcal{X},{\mathcal{Y}}_2})/R^{\prime }\right)\times [0,1])$ is a homeomorphism. On the other hand, since $({DMC}_{\mathcal{X},\ast },{\mathcal{T}}_{s,\mathcal{X},\ast })$ and ${DMC}_{\mathcal{X},{\mathcal{Y}}_2}^{(o)}={DMC}_{\mathcal{X},{\mathcal{Y}}_2}/R_{\mathcal{X},{\mathcal{Y}}_2}^{(o)}$ are Hausdorff and locally compact, Corollary 1 implies that the canonical bijection from ${DMC}_{\mathcal{X},\ast }^{(o)}\times {DMC}_{\mathcal{X},{\mathcal{Y}}_2}^{(o)}$ to $({DMC}_{\mathcal{X},\ast }\times {DMC}_{\mathcal{X},{\mathcal{Y}}_2})/R^{\prime }$ is a homeomorphism. We conclude that the channel interpolation is continuous on $({DMC}_{\mathcal{X},\ast }^{(o)}\times {DMC}_{\mathcal{X},{\mathcal{Y}}_2}^{(o)}\times [0,1],{\mathcal{T}}_{s,\mathcal{X},\ast }^{(o)}\otimes {\mathcal{T}}_{\mathcal{X},\mathcal{Y}}^{(o)}\otimes \mathcal{U})$. ☐

Corollary 3.

$({DMC}_{\mathcal{X},\ast }^{(o)},{\mathcal{T}}_{s,\mathcal{X},\ast }^{(o)})$ is strongly contractible to every point in ${DMC}_{\mathcal{X},\ast }^{(o)}$.

Proof. 

Fix ${\widehat{W}}_0\in {DMC}_{\mathcal{X},\ast }^{(o)}$. Define the mapping $H:{DMC}_{\mathcal{X},\ast }^{(o)}\times [0,1]\to {DMC}_{\mathcal{X},\ast }^{(o)}$ as $H(\widehat{W},\alpha )=[\alpha {\widehat{W}}_0,(1-\alpha )\widehat{W}]$. H is continuous by Proposition 7. We also have $H(\widehat{W},0)=\widehat{W}$ and $H(\widehat{W},1)={\widehat{W}}_0$ for every $\widehat{W}\in {DMC}_{\mathcal{X},\ast }^{(o)}$. Moreover, $H({\widehat{W}}_0,\alpha )={\widehat{W}}_0$ for every $0\le \alpha \le 1$. Therefore, $({DMC}_{\mathcal{X},\ast }^{(o)},{\mathcal{T}}_{s,\mathcal{X},\ast }^{(o)})$ is strongly contractible to every point in ${DMC}_{\mathcal{X},\ast }^{(o)}$. ☐

The reader might be wondering why channel operations such as the channel sum were not shown to be continuous on the whole space ${DMC}_{{\mathcal{X}}_1,\ast }^{(o)}\times {DMC}_{{\mathcal{X}}_2,\ast }^{(o)}$ instead of the smaller space ${DMC}_{{\mathcal{X}}_1,\ast }^{(o)}\times {DMC}_{{\mathcal{X}}_2,{\mathcal{Y}}_2}^{(o)}$. The reason is because we cannot apply Corollary 1 to ${DMC}_{{\mathcal{X}}_1,\ast }\times {DMC}_{{\mathcal{X}}_2,\ast }$ and ${DMC}_{{\mathcal{X}}_1,\ast }^{(o)}\times {DMC}_{{\mathcal{X}}_2,\ast }^{(o)}$ since neither ${DMC}_{{\mathcal{X}}_1,\ast }^{(o)}$, nor ${DMC}_{{\mathcal{X}}_2,\ast }^{(o)}$ is locally compact (under the strong topology).

One potential method to show the continuity of the channel sum on $({DMC}_{{\mathcal{X}}_1,\ast }^{(o)}\times {DMC}_{{\mathcal{X}}_2,\ast }^{(o)},{\mathcal{T}}_{s,{\mathcal{X}}_1,\ast }^{(o)}\otimes {\mathcal{T}}_{s,{\mathcal{X}}_2,\ast }^{(o)})$ is as follows: let R be the equivalence relation on ${DMC}_{{\mathcal{X}}_1,\ast }\times {DMC}_{{\mathcal{X}}_2,\ast }$ defined as $(W_1,W_2)R(W_1^{\prime },W_2^{\prime })$ if and only if $W_1{R}_{{\mathcal{X}}_1,\ast }^{(o)}{W}_1^{\prime }$ and $W_2{R}_{{\mathcal{X}}_2,\ast }^{(o)}{W}_2^{\prime }$. We can identify $({DMC}_{{\mathcal{X}}_1,\ast }\times {DMC}_{{\mathcal{X}}_2,\ast })/R$ with ${DMC}_{{\mathcal{X}}_1,\ast }^{(o)}\times {DMC}_{{\mathcal{X}}_2,\ast }^{(o)}$ through the canonical bijection. Using Lemma 2, it is easy to see that the mapping $({\widehat{W}}_1,{\overline{W}}_2)\to {\widehat{W}}_1\oplus {\overline{W}}_2$ is continuous from $\left({DMC}_{{\mathcal{X}}_1,\ast }^{(o)}\times {DMC}_{{\mathcal{X}}_2,\ast }^{(o)},({\mathcal{T}}_{s,{\mathcal{X}}_1,\ast }\otimes {\mathcal{T}}_{s,{\mathcal{X}}_2,\ast })/R\right)$ to $({DMC}_{{\mathcal{X}}_1\coprod {\mathcal{X}}_2,\ast }^{(o)},{\mathcal{T}}_{s,{\mathcal{X}}_1\coprod {\mathcal{X}}_2,\ast }^{(o)})$.

It was shown in [17] that the topology $({\mathcal{T}}_{s,{\mathcal{X}}_1,\ast }\otimes {\mathcal{T}}_{s,{\mathcal{X}}_2,\ast })/R$ is homeomorphic to $\kappa ({\mathcal{T}}_{s,{\mathcal{X}}_1,\ast }^{(o)}\otimes {\mathcal{T}}_{s,{\mathcal{X}}_2,\ast }^{(o)})$ through the canonical bijection, where $\kappa ({\mathcal{T}}_{s,{\mathcal{X}}_1,\ast }^{(o)}\otimes {\mathcal{T}}_{s,{\mathcal{X}}_2,\ast }^{(o)})$ is the coarsest topology that is both compactly generated and finer than ${\mathcal{T}}_{s,{\mathcal{X}}_1,\ast }^{(o)}\otimes {\mathcal{T}}_{s,{\mathcal{X}}_2,\ast }^{(o)}$. Therefore, the mapping $({\widehat{W}}_1,{\overline{W}}_2)\to {\widehat{W}}_1\oplus {\overline{W}}_2$ is continuous on $\left({DMC}_{{\mathcal{X}}_1,\ast }^{(o)}\times {DMC}_{{\mathcal{X}}_2,\ast }^{(o)},\kappa ({\mathcal{T}}_{s,{\mathcal{X}}_1,\ast }^{(o)}\otimes {\mathcal{T}}_{s,{\mathcal{X}}_2,\ast }^{(o)})\right)$. This means that if ${\mathcal{T}}_{s,{\mathcal{X}}_1,\ast }^{(o)}\otimes {\mathcal{T}}_{s,{\mathcal{X}}_2,\ast }^{(o)}$ is compactly generated, we will have ${\mathcal{T}}_{s,{\mathcal{X}}_1,\ast }^{(o)}\otimes {\mathcal{T}}_{s,{\mathcal{X}}_2,\ast }^{(o)}=\kappa ({\mathcal{T}}_{s,{\mathcal{X}}_1,\ast }^{(o)}\otimes {\mathcal{T}}_{s,{\mathcal{X}}_2,\ast }^{(o)})$, and so, the channel sum will be continuous on $({DMC}_{{\mathcal{X}}_1,\ast }^{(o)}\times {DMC}_{{\mathcal{X}}_2,\ast }^{(o)},{\mathcal{T}}_{s,{\mathcal{X}}_1,\ast }^{(o)}\otimes {\mathcal{T}}_{s,{\mathcal{X}}_2,\ast }^{(o)})$. Note that although ${\mathcal{T}}_{s,{\mathcal{X}}_1,\ast }^{(o)}$ and ${\mathcal{T}}_{s,{\mathcal{X}}_2,\ast }^{(o)}$ are compactly generated, their product ${\mathcal{T}}_{s,{\mathcal{X}}_1,\ast }^{(o)}\otimes {\mathcal{T}}_{s,{\mathcal{X}}_2,\ast }^{(o)}$ might not be compactly generated.

7. Continuity in the Noisiness/Weak-∗ and the Total Variation Topologies

We need to express the channel parameters and operations in terms of the Blackwell measures.

7.1. Channel Parameters

The following proposition shows that many channel parameters can be expressed as an integral of a continuous function with respect to the Blackwell measure:

Proposition 8.

For every $\widehat{W}\in {DMC}_{\mathcal{X},\ast }^{(o)}$, we have:

$$\forall p\in {\Delta }_{\mathcal{X}},I(p,\widehat{W})=H(p)-|\mathcal{X}|·{\int }_{{\Delta }_{\mathcal{X}}}\left(\sum _{x\in \mathcal{X}}p(x)p^{\prime }(x)log\frac{p(x)p^{\prime }(x)}{{\displaystyle \sum _{x^{\prime }}p(x^{\prime })p^{\prime }(x^{\prime })}}\right)·d{MP}_{\widehat{W}}(p^{\prime }),$$

$$\forall p\in {\Delta }_{\mathcal{X}},P_e(p,\widehat{W})=1-|\mathcal{X}|{\int }_{{\Delta }_{\mathcal{X}}}\underset{x\in \mathcal{X}}{max}\left\{p(x)\times p^{\prime }(x)\right\}·d{MP}_{\widehat{W}}(p^{\prime }),$$

$$\mathrm{if}|\mathcal{X}|\ge 2,Z(\widehat{W})=\frac{1}{|\mathcal{X}|-1}\sum _{\begin{array}{c}x,x^{\prime }\in \mathcal{X},\\ x\ne x^{\prime }\end{array}}{\int }_{{\Delta }_{\mathcal{X}}}\sqrt{p(x)p(x^{\prime })}·d{MP}_{\widehat{W}}(p),$$

$$\mathit{For}\mathit{every}\mathit{code}\mathcal{C}\subset {\mathcal{X}}^n,P_{e,\mathcal{C}}(\widehat{W})=1-\frac{{|\mathcal{X}|}^n}{|\mathcal{C}|}{\int }_{{\Delta }_{\mathcal{X}}^n}\underset{x_1^n\in \mathcal{C}}{max}\left\{\prod _{i=1}^n{p}_i(x_i)\right\}d{MP}_{\widehat{W}}^n(p_1^n),$$

where $H(p)$ is the entropy of p and ${MP}_{\widehat{W}}^n$ is the product measure on ${\Delta }_{\mathcal{X}}^n$ obtained by multiplying ${MP}_{\widehat{W}}$ with itself n times. Note that we adopt the standard convention that $0log\frac{0}{0}=0$.

Proof. 

By choosing any representative channel $W\in \widehat{W}$ and replacing $W(y|x)$ by $|\mathcal{X}|P_W^o(y)W_y^{-1}(x)$ in the definitions of the channel parameters, all the above formulas immediately follow. Let us show how this works for $P_e$:

$$\begin{array}{cc}\hfill P_e(p,\widehat{W})& =P_e(p,W)\stackrel{(a)}{=}1-\sum _{y\in Im(W)}\underset{x\in \mathcal{X}}{max}\{p(x)W(y|x)\}\hfill \\ & =1-\sum _{y\in Im(W)}\underset{x\in \mathcal{X}}{max}\left\{p(x)·|\mathcal{X}|·P_W^o(y)W_y^{-1}(x)\right\}\hfill \\ & =1-|\mathcal{X}|\sum _{y\in Im(W)}\underset{x\in \mathcal{X}}{max}\{p(x)W_y^{-1}(x)\}·P_W^o(y)\hfill \\ & =1-|\mathcal{X}|{\int }_{{\Delta }_{\mathcal{X}}}\underset{x\in \mathcal{X}}{max}\{p(x)p^{\prime }(x)\}·d{MP}_W(p^{\prime })\hfill \\ & =1-|\mathcal{X}|{\int }_{{\Delta }_{\mathcal{X}}}\underset{x\in \mathcal{X}}{max}\{p(x)p^{\prime }(x)\}·d{MP}_{\widehat{W}}(p^{\prime }),\hfill \end{array}$$

where (a) is true because $W(y|x)=0$ for $y\notin Im(W)$. ☐

Proposition 9.

Let ${\mathcal{U}}_{\mathcal{X}}$ be the standard topology on ${\Delta }_{\mathcal{X}}$. We have:

• $I:{\Delta }_{\mathcal{X}}\times {DMC}_{\mathcal{X},\ast }^{(o)}\to {\mathbb{R}}^{+}$ is continuous on $({\Delta }_{\mathcal{X}}\times {DMC}_{\mathcal{X},\ast }^{(o)},{\mathcal{U}}_{\mathcal{X}}\otimes {\mathcal{T}}_{\mathcal{X},\ast }^{(o)})$ and concave in p.
• $C:{DMC}_{\mathcal{X},\ast }^{(o)}\to {\mathbb{R}}^{+}$ is continuous on $({DMC}_{\mathcal{X},\ast }^{(o)},{\mathcal{T}}_{\mathcal{X},\ast }^{(o)})$.
• $P_e:{\Delta }_{\mathcal{X}}\times {DMC}_{\mathcal{X},\ast }^{(o)}\to [0,1]$ is continuous on $({\Delta }_{\mathcal{X}}\times {DMC}_{\mathcal{X},\ast }^{(o)},{\mathcal{U}}_{\mathcal{X}}\otimes {\mathcal{T}}_{\mathcal{X},\ast }^{(o)})$ and concave in p.
• $Z:{DMC}_{\mathcal{X},\ast }^{(o)}\to [0,1]$ is continuous on $({DMC}_{\mathcal{X},\ast }^{(o)},{\mathcal{T}}_{\mathcal{X},\ast }^{(o)})$.
• For every code $\mathcal{C}$ on $\mathcal{X}$, $P_{e,\mathcal{C}}:{DMC}_{\mathcal{X},\ast }^{(o)}\to [0,1]$ is continuous on $({DMC}_{\mathcal{X},\ast }^{(o)},{\mathcal{T}}_{\mathcal{X},\ast }^{(o)})$.
• For every $n>0$ and every $1\le M\le {|\mathcal{X}|}^n$, the mapping $P_{e,n,M}:{DMC}_{\mathcal{X},\ast }^{(o)}\to [0,1]$ is continuous on $({DMC}_{\mathcal{X},\ast }^{(o)},{\mathcal{T}}_{\mathcal{X},\ast }^{(o)})$.

Proof. 

We associate the space $\mathcal{MP}(\mathcal{X})$ with the weak-∗ topology. Define the mapping:

$$\overline{I}:{\Delta }_{\mathcal{X}}\times \mathcal{MP}(\mathcal{X})\to {\mathbb{R}}^{+}$$

as follows:

$$\overline{I}(p,MP)=H(p)-|\mathcal{X}|·{\int }_{{\Delta }_{\mathcal{X}}}\left(\sum _{x\in \mathcal{X}}p(x)p^{\prime }(x)log\frac{p(x)p^{\prime }(x)}{{\displaystyle \sum _{x^{\prime }}p(x^{\prime })p^{\prime }(x^{\prime })}}\right)·dMP(p^{\prime }),$$

Lemma 5 implies that $\overline{I}$ is continuous. On the other hand, Proposition 8 shows that $I(p,\widehat{W})=\overline{I}(p,{MP}_{\widehat{W}})$. Therefore, I is continuous on $({\Delta }_{\mathcal{X}}\times {DMC}_{\mathcal{X},\ast }^{(o)},{\mathcal{U}}_{\mathcal{X}}\otimes {\mathcal{T}}_{\mathcal{X},\ast }^{(o)})$. We can prove the continuity of $P_e$ and Z similarly.

Now, define the mapping $\overline{C}:\mathcal{MP}(\mathcal{X})\to \mathbb{R}$ as:

$$\overline{C}(MP)=\underset{p\in {\Delta }_{\mathcal{X}}}{sup}\overline{I}(p,MP).$$

Fix $MP\in \mathcal{MP}(\mathcal{X})$, and let $ϵ>0$. Since $\mathcal{MP}(\mathcal{X})$ is compact (under the weak-∗ topology), Lemma 1 implies the existence of a weakly-∗ open neighborhood $U_{MP}$ of MP such that $|\overline{I}(p,MP)-\overline{I}(p,{MP}^{\prime })|<ϵ$ for every ${MP}^{\prime }\in U_{MP}$ and every $p\in {\Delta }_{\mathcal{X}}$. Therefore, for every ${MP}^{\prime }\in U_{MP}$ and every $p\in {\Delta }_{\mathcal{X}}$, we have:

$$\overline{I}(p,MP)<\overline{I}(p,{MP}^{\prime })+ϵ\le \overline{C}({MP}^{\prime })+ϵ,$$

hence,

$$\overline{C}(MP)=\underset{p\in {\Delta }_{\mathcal{X}}}{sup}\overline{I}(p,MP)\le \overline{C}({MP}^{\prime })+ϵ.$$

Similarly, we can show that $\overline{C}({MP}^{\prime })\le \overline{C}(MP)+ϵ$. This shows that $|\overline{C}({MP}^{\prime })-\overline{C}(MP)|\le ϵ$ for every ${MP}^{\prime }\in U_{MP}$. Therefore, $\overline{C}$ is continuous. However, $C(\widehat{W})=\overline{C}({MP}_{\widehat{W}})$, so C is continuous on $({DMC}_{\mathcal{X},\ast }^{(o)},{\mathcal{T}}_{\mathcal{X},\ast }^{(o)})$.

Now for every $0\le i\le n$, define the mapping $f_i:{\Delta }_{\mathcal{X}}^i\times \mathcal{MP}(\mathcal{X})\to \mathbb{R}$ backward-recursively as follows:

• ${\displaystyle f_n(p_1^n,MP)=\underset{x_1^n\in \mathcal{C}}{max}\left\{\prod _{i=1}^n{p}_i(x_i)\right\}}$.
• For every $0\le i<n$, define:

  $$f_i(p_1^i,MP)={\int }_{{\Delta }_{\mathcal{X}}}{f}_{i+1}(p_1^{i+1},MP)·dMP(p_{i+1}).$$

Clearly $f_n$ is continuous. Now, let $0\le i<n$, and assume that $f_{i+1}$ is continuous. If we let $S={\Delta }_{\mathcal{X}}^i\times \mathcal{MP}(\mathcal{X})$, Lemma 5 implies that the mapping $F_i:{\Delta }_{\mathcal{X}}^i\times \mathcal{MP}(\mathcal{X})\times \mathcal{MP}(\mathcal{X})$ defined as:

$$F_i(p_1^i,MP,{MP}^{\prime })={\int }_{{\Delta }_{\mathcal{X}}}{f}_{i+1}(p_1^{i+1},MP)·d{MP}^{\prime }(p_{i+1})$$

is continuous. However, $f_i(p_1^i,MP)=F_i(p_1^i,MP,MP)$, so $f_i$ is also continuous. Therefore, $f_0$ is continuous. By noticing that ${\displaystyle P_{e,\mathcal{C}}(\widehat{W})=1-\frac{{|\mathcal{X}|}^n}{|\mathcal{C}|}{f}_0({MP}_{\widehat{W}})}$, we conclude that $P_{e,\mathcal{C}}$ is continuous on $({DMC}_{\mathcal{X},\ast }^{(o)},{\mathcal{T}}_{\mathcal{X},\ast }^{(o)})$. Moreover, since $P_{e,n,M}$ is the minimum of a finite family of continuous mappings, it is continuous. ☐

It is worth mentioning that Proposition 6 can be shown from Proposition 9 because the noisiness topology is coarser than the strong topology.

Corollary 4.

All the mappings in Proposition 9 are also continuous if we replace the noisiness topology ${\mathcal{T}}_{\mathcal{X},\ast }^{(o)}$ with the total variation topology ${\mathcal{T}}_{TV,\mathcal{X},\ast }^{(o)}$.

Proof. 

This is true because ${\mathcal{T}}_{TV,\mathcal{X},\ast }^{(o)}$ is finer than ${\mathcal{T}}_{\mathcal{X},\ast }^{(o)}$. ☐

7.2. Channel Operations

In the following, we show that we can express the channel operations in terms of Blackwell measures. We have all the tools to achieve this for the channel sum, channel product and channel interpolation. In order to express the channel polarization transformations in terms of the Blackwell measures, we need to introduce new definitions.

Let $\mathcal{X}$ be a finite set, and let ∗ be a binary operation on a finite set $\mathcal{X}$. We say that ∗ is uniformity preserving if the mapping $(a,b)\to (a\ast b,b)$ is a bijection from ${\mathcal{X}}^2$ to itself [18]. For every $a,b\in \mathcal{X}$, we denote the unique element $c\in \mathcal{X}$ satisfying $c\ast b=a$ as $c=a{/}^{\ast }b$. Note that ${/}^{\ast }$ is a binary operation, and it is uniformity preserving. ${/}^{\ast }$ is called the right-inverse of ∗. It was shown in [11] that a binary operation is polarizing if and only if it is uniformity preserving and its inverse is strongly ergodic.

Binary operations that are not uniformity preserving are not interesting for polarization theory because they do not preserve the symmetric capacity [11]. Therefore, we will only focus on polarization transformations that are based on uniformity preserving operations.

Let ∗ be a fixed uniformity preserving operation on $\mathcal{X}$. Define the mapping $C^{-,\ast }:{\Delta }_{\mathcal{X}}\times {\Delta }_{\mathcal{X}}\to {\Delta }_{\mathcal{X}}$ as

$$(C^{-,\ast }(p_1,p_2))(u_1)=\sum _{u_2\in \mathcal{X}}{p}_1(u_1\ast u_2)p_2(u_2).$$

The probability distribution $C^{-,\ast }(p_1,p_2)$ can be interpreted as follows: let $X_1$ and $X_2$ be two independent random variables in $\mathcal{X}$ that are distributed as $p_1$ and $p_2$, respectively, and let $(U_1,U_2)$ be the random pair in ${\mathcal{X}}^2$ defined as $(U_1,U_2)=(X_1{/}^{\ast }{X}_2,X_2)$, or equivalently $(X_1,X_2)=(U_1\ast U_2,U_2)$. $C^{-,\ast }(p_1,p_2)$ is the probability distribution of $U_1$.

Clearly, $C^{-,\ast }$ is continuous. Therefore, the push-forward mapping $C_{\#}^{-,\ast }$ is continuous from $\mathcal{P}({\Delta }_{\mathcal{X}}\times {\Delta }_{\mathcal{X}})$ to $\mathcal{P}({\Delta }_{\mathcal{X}})=\mathcal{MP}(\mathcal{X})$ under both the weak-∗ and the total variation topologies (see Section 2.6). For every ${MP}_1,{MP}_2\in \mathcal{MP}(\mathcal{X})$, we define the $(-,\ast )$-convolution of ${MP}_1$ and ${MP}_2$ as:

$${({MP}_1,{MP}_2)}^{-,\ast }=C_{\#}^{-,\ast }({MP}_1\times {MP}_2)\in \mathcal{MP}(\mathcal{X}).$$

Since the product of meta-probability measures is continuous under both the weak-∗ and the total variation topologies (Appendix B and Appendix F), the $(-,\ast )$-convolution is also continuous under these topologies.

For every $p_1,p_2\in {\Delta }_{\mathcal{X}}$ and every $u_1\in supp(C^{-,\ast }(p_1,p_2))$, define $C^{+,u_1,\ast }(p_1,p_2)\in {\Delta }_{\mathcal{X}}$ as:

$$(C^{+,u_1,\ast }(p_1,p_2))(u_2)=\frac{p_1(u_1\ast u_2)p_2(u_2)}{(C^{-,\ast }(p_1,p_2))(u_1)}.$$

The probability distribution $C^{+,u_1,\ast }(p_1,p_2)$ can be interpreted as follows: if $X_1,X_2,U_1$ and $U_2$ are as above, $C^{+,u_1,\ast }(p_1,p_2)$ is the conditional probability distribution of $U_2$ given $U_1=u_1$.

Define the mapping $C^{+,\ast }:{\Delta }_{\mathcal{X}}\times {\Delta }_{\mathcal{X}}\to \mathcal{P}({\Delta }_{\mathcal{X}})=\mathcal{MP}(\mathcal{X})$ as follows:

$$C^{+,\ast }(p_1,p_2)=\sum _{u_1\in supp(C^{-,\ast }(p_1,p_2))}(C^{-,\ast }(p_1,p_2))(u_1)·{\delta }_{C^{+,u_1,\ast }(p_1,p_2)},$$

where ${\delta }_{C^{+,u_1,\ast }(p_1,p_2)}$ is a Dirac measure centered at $C^{+,u_1,\ast }(p_1,p_2)$.

If $X_1,X_2,U_1$ and $U_2$ are as above, $C^{+,\ast }(p_1,p_2)$ is the meta-probability measure that describes the possible conditional probability distributions of $U_2$ that are seen by someone having knowledge of $U_1$. Clearly, $C^{+,\ast }$ is a random mapping from ${\Delta }_{\mathcal{X}}\times {\Delta }_{\mathcal{X}}$ to ${\Delta }_{\mathcal{X}}$. In Appendix H, we show that $C^{+,\ast }$ is a measurable random mapping. We also show in Appendix H that $C^{+,\ast }$ is a continuous mapping from ${\Delta }_{\mathcal{X}}\times {\Delta }_{\mathcal{X}}$ to $\mathcal{MP}(\mathcal{X})$ when the latter space is endowed with the weak-∗ topology. Lemmas 3 and 4 now imply that the push-forward mapping $C_{\#}^{+,\ast }$ is continuous under both the weak-∗ and the total variation topologies.

For every ${MP}_1,{MP}_2\in \mathcal{MP}(\mathcal{X})$, we define the $(+,\ast )$-convolution of ${MP}_1$ and ${MP}_2$ as:

$${({MP}_1,{MP}_2)}^{+,\ast }=C_{\#}^{+,\ast }({MP}_1\times {MP}_2)\in \mathcal{MP}(\mathcal{X}).$$

Since the product of meta-probability measures is continuous under both the weak-∗ and the total variation topologies (Appendix B and Appendix F), the $(+,\ast )$-convolution is also continuous under these topologies.

Proposition 10.

We have:

• For every ${\widehat{W}}_1\in {DMC}_{{\mathcal{X}}_1,\ast }^{(o)}$ and ${\overline{W}}_2\in {DMC}_{{\mathcal{X}}_2,\ast }^{(o)}$, we have:

  $${MP}_{{\widehat{W}}_1\oplus {\overline{W}}_2}=\frac{|{\mathcal{X}}_1|}{|{\mathcal{X}}_1|+|{\mathcal{X}}_2|}{MP}_{{\widehat{W}}_1}^{\prime }+\frac{|{\mathcal{X}}_2|}{|{\mathcal{X}}_1|+|{\mathcal{X}}_2|}{MP}_{{\overline{W}}_2}^{\prime },$$

  where ${MP}_{{\widehat{W}}_1}^{\prime }$ (respectively ${MP}_{{\widehat{W}}_2}^{\prime }$) is the meta-push-forward of ${MP}_{{\widehat{W}}_1}$ (respectively ${MP}_{{\widehat{W}}_2}$) by the canonical injection from ${\mathcal{X}}_1$ (respectively ${\mathcal{X}}_2$) to ${\mathcal{X}}_1\coprod {\mathcal{X}}_2$.
• For every ${\widehat{W}}_1\in {DMC}_{{\mathcal{X}}_1,\ast }^{(o)}$ and ${\overline{W}}_2\in {DMC}_{{\mathcal{X}}_2,\ast }^{(o)}$, we have:

  $${MP}_{{\widehat{W}}_1\otimes {\overline{W}}_2}={MP}_{{\widehat{W}}_1}\otimes {MP}_{{\overline{W}}_2}.$$
• For every $\alpha \in [0,1]$ and every ${\widehat{W}}_1,{\widehat{W}}_2\in {DMC}_{\mathcal{X},\ast }^{(o)}$, we have:

  $${MP}_{[\alpha {\widehat{W}}_1,(1-\alpha ){\widehat{W}}_2]}=\alpha {MP}_{{\widehat{W}}_1}+(1-\alpha ){MP}_{{\widehat{W}}_2}.$$
• For every uniformity preserving binary operation ∗ on $\mathcal{X}$, and every $\widehat{W}\in {DMC}_{\mathcal{X},\ast }^{(o)}$, we have:

  $${MP}_{{\widehat{W}}^{-}}={({MP}_{\widehat{W}},{MP}_{\widehat{W}})}^{-,\ast }.$$
• For every uniformity preserving binary operation ∗ on $\mathcal{X}$ and every $\widehat{W}\in {DMC}_{\mathcal{X},\ast }^{(o)}$, we have:

  $${MP}_{{\widehat{W}}^{+}}={({MP}_{\widehat{W}},{MP}_{\widehat{W}})}^{+,\ast }.$$

Proof. 

See Appendix I. ☐

Note that the polarization transformation formulas in Proposition 10 generalize the formulas given by Raginsky in [19] for binary-input channels.

Proposition 11.

Assume that all equivalent channel spaces are endowed with the noisiness/weak-∗ or the total variation topology. We have:

• The mapping $({\widehat{W}}_1,{\overline{W}}_2)\to {\widehat{W}}_1\oplus {\overline{W}}_2$ from ${DMC}_{{\mathcal{X}}_1,\ast }^{(o)}\times {DMC}_{{\mathcal{X}}_2,\ast }^{(o)}$ to ${DMC}_{{\mathcal{X}}_1\coprod {\mathcal{X}}_2,\ast }^{(o)}$ is continuous.
• The mapping $({\widehat{W}}_1,{\overline{W}}_2)\to {\widehat{W}}_1\otimes {\overline{W}}_2$ from ${DMC}_{{\mathcal{X}}_1,\ast }^{(o)}\times {DMC}_{{\mathcal{X}}_2,\ast }^{(o)}$ to ${DMC}_{{\mathcal{X}}_1\times {\mathcal{X}}_2,\ast }^{(o)}$ is continuous.
• The mapping $({\widehat{W}}_1,{\overline{W}}_2,\alpha )\to [\alpha {\widehat{W}}_1,(1-\alpha ){\overline{W}}_2]$ from ${DMC}_{\mathcal{X},\ast }\times {DMC}_{\mathcal{X},\ast }^{(o)}\times [0,1]$ to ${DMC}_{\mathcal{X},\ast }^{(o)}$ is continuous.
• For every uniformity preserving binary operation ∗ on $\mathcal{X}$, the mapping $\widehat{W}\to {\widehat{W}}^{-}$ from ${DMC}_{\mathcal{X},\ast }^{(o)}$ to ${DMC}_{\mathcal{X},\ast }^{(o)}$ is continuous.
• For every uniformity preserving binary operation ∗ on $\mathcal{X}$, the mapping $\widehat{W}\to {\widehat{W}}^{+}$ from ${DMC}_{\mathcal{X},\ast }^{(o)}$ to ${DMC}_{\mathcal{X},\ast }^{(o)}$ is continuous.

Proof. 

The proposition directly follows from Proposition 10 and the fact that all the meta-probability measure operations that are involved in the formulas are continuous under both the weak-∗ and the total variation topologies. ☐

Corollary 5.

Both $({DMC}_{\mathcal{X},\ast }^{(o)},{\mathcal{T}}_{\mathcal{X},\ast }^{(o)})$ and $({DMC}_{\mathcal{X},\ast }^{(o)},{\mathcal{T}}_{TV,\mathcal{X},\ast }^{(o)})$ are strongly contractible to every point in ${DMC}_{\mathcal{X},\ast }^{(o)}$.

Proof. 

We can use the same proof of Corollary 3. ☐

8. Discussion and Conclusions

Section 5 and Section 6 show that the quotient topology is relatively easy to work with. If one is interested in the space of equivalent channels sharing the same input and output alphabets, then using the quotient formulation of the topology seems to be the easiest way to prove theorems.

The continuity of the channel sum and the channel product on the whole product space $({DMC}_{{\mathcal{X}}_1,\ast }^{(o)}\times {DMC}_{{\mathcal{X}}_2,\ast }^{(o)},{\mathcal{T}}_{s,{\mathcal{X}}_1,\ast }^{(o)}\otimes {\mathcal{T}}_{s,{\mathcal{X}}_2,\ast }^{(o)})$ remains an open problem. As we mentioned in Section 6, it is sufficient to prove that the product topology ${\mathcal{T}}_{s,{\mathcal{X}}_1,\ast }^{(o)}\otimes {\mathcal{T}}_{s,{\mathcal{X}}_2,\ast }^{(o)}$ is compactly generated.