Conditioning in Tropical Probability Theory

Abstract

We define a natural operation of conditioning of tropical diagrams of probability spaces and show that it is Lipschitz continuous with respect to the asymptotic entropy distance.

1. Introduction

In [1,2], we have initiated the study of tropical probability spaces and their diagrams. In [1], we endowed (commutative) diagrams of probability spaces with the intrinsic entropy distance and, in [2], we defined tropical diagrams as points in the asymptotic cone of the metric space. They are represented by certain sequences of diagrams of probability spaces.

We expect that tropical diagrams will be helpful in the study of information optimization problems, such as the ones considered in [3,4,5,6,7,8], and we have indeed applied them to derive a dimension-reduction result for the shape of the entropic cone in [9].

In this present article, we introduce the notion of conditioning on a space in a tropical diagram and show that the operation is Lipschitz continuous with respect to the asymptotic entropy distance.

It is a rather technical result, and we have, therefore, decided to treat it in this separate article, but it is an important ingredient in the theory and, in particular, we need it for the dimension-reduction result mentioned before.

Given a tuple of finite-valued random variables ${\left({\mathsf{X}}_i\right)}_{i=1}^n$ and a random variable $\mathsf{Y}$, one may "condition" the collection $\left({\mathsf{X}}_i\right)$ on $\mathsf{Y}$. The result of this operation is a family of n-tuples of random variables denoted ${\left({\mathsf{X}}_i\right|\mathsf{Y})}_{i=1}^n$ parameterized by those values of $\mathsf{Y}$ that have positive probability. Each tuple of random variables in this family is defined on a separate probability space.

When passing to the tropical setting, the situation is different in the sense that when we condition a tropical diagram $\left[\mathcal{X}\right]$ on a space $\left[Y\right]$, the result is again a tropical diagram $\left[\mathcal{X}\right|Y]$ rather than a family. After recalling some preliminaries in Section 2, we describe the operation of conditioning and prove that the result depends in a Lipschitz way on the original diagram in Section 3.

2. Preliminaries

Our main objects of study are commutative diagrams of probability spaces and their tropical counterparts. In this section, we recall briefly the main definitions and results.

2.1. Probability Spaces and Their Diagrams

2.1.1. Probability Spaces

By a finite probability space, we mean a set with a probability measure that has finite support. A reduction from one probability space to another is an equivalence class of measure-preserving maps. Two maps are equivalent if they coincide on a set of full measures. We call a point x in a probability space $X=(\underline{X},p)$ an atom if it has positive weight, and we write $x\in X$ to mean x is an atom in X (as opposed to $x\in \underline{X}$ for points in the underlying set). For a probability space X, we denote by $\left|X\right|$ the cardinality of the support of the probability measure.

2.1.2. Indexing Categories

To record the combinatorial structure of commutative diagrams of probability spaces and reductions, we use an object that we call an indexing category. By an indexing category, we mean a finite category $\mathbf{G}$ such that for any pair of objects $i,j\in \mathbf{G}$, there is at most one morphism between them either way. In addition, we will assume it satisfies one additional property that we will describe after introducing some terminology. For a pair of objects $i,j\in \mathbf{G}$ such that there is a morphism ${\gamma }_{ij}:i\to j$, object i will be called an ancestor of j and object j will be called a descendant of i. The subcategory of all descendants of an object $i\in \mathbf{G}$ is called an ideal generated by i and will be denoted $⌈i⌉$, while we will call the subcategory consisting of all ancestors of i together with all the morphisms in it a co-ideal generated by i and denote it by $⌊i⌋$. (The term filter is also used for a co-ideal in the literature about lattices).

The additional property that an indexing category has to satisfy is that for any pair of objects $i,j\in \mathbf{G}$, there exists a minimal common ancestor $\widehat{𝚤}$, and $\widehat{𝚤}$ is an ancestor for both i and j and any other ancestor of them both is also an ancestor of $\widehat{𝚤}$; in other words, $\mathbf{G}$ is an upper semi-lattice.

An equivalent formulation of the property above is the following: the intersection of the co-ideals generated by two objects $i,j\in \mathbf{G}$ is also a co-ideal generated by some object $\widehat{𝚤}\in \mathbf{G}$.

Any indexing category $\mathbf{G}$ is necessarily initial, which means that there exists an initial object, that is an object $i_0$ such that $\mathbf{G}=⌈i_0⌉$.

A fan in a category is a pair of morphisms with the same domain. A fan $(i\leftarrow k\to j)$ is called minimal if for any other fan $(i\leftarrow l\to j)$ included in a commutative diagram

the vertical arrow must be an isomorphism; in other words, k is a minimal common ancestor of i and j.

For any pair of objects $i,j$ in an indexing category $\mathbf{G}$, there exists a unique minimal fan $(i\leftarrow \widehat{𝚤}\to j)$ in $\mathbf{G}$.

2.1.3. Diagrams

We denote by $\mathbf{Prob}$ the category of finite probability spaces and reductions, i.e., the equivalence classes of measure-preserving maps. For an indexing category $\mathbf{G}=\left\{i;{\gamma }_{ij}\right\}$, a $\mathbf{G}$-diagram is a functor $\mathcal{X}:\mathbf{G}\to \mathbf{Prob}$. A reduction f from one $\mathbf{G}$-diagram $\mathcal{X}=\left\{X_i;{\chi }_{ij}\right\}$ to another $\mathcal{Y}=\left\{Y_i;{\upsilon }_{ij}\right\}$ is a natural transformation between the functors. It amounts to a collection of reductions $f_i:X_i\to Y_i$, such that the big diagram consisting of all spaces $X_i$, $Y_i$ and all morphisms ${\chi }_{ij}$, ${\upsilon }_{ij}$ and $f_i$ is commutative. The category of $\mathbf{G}$-diagrams and reductions will be denoted as $\mathbf{Prob}〈\mathbf{G}〉$. The construction of diagrams could be iterated; thus, we can consider $\mathbf{H}$-diagrams of $\mathbf{G}$-diagrams and denote the corresponding category $\mathbf{Prob}〈\mathbf{G}〉〈\mathbf{H}〉=\mathbf{Prob}〈\mathbf{G},\mathbf{H}〉$. Every $\mathbf{H}$-diagram of a $\mathbf{G}$-diagram can also be considered as $\mathbf{G}$-diagrams of $\mathbf{H}$-diagrams; thus, there is a natural equivalence of categories $\mathbf{Prob}〈\mathbf{G},\mathbf{H}〉\cong \mathbf{Prob}〈\mathbf{H},\mathbf{G}〉$.

A $\mathbf{G}$-diagram $\mathcal{X}$ will be called minimal if it maps minimal fans in $\mathbf{G}$ to minimal fans in the target category. The subspace of all minimal $\mathbf{G}$-diagrams will be denoted $\mathbf{Prob}{〈\mathbf{G}〉}_{\mathsf{m}}$. In [1], we have shown that for any fan in $\mathbf{Prob}$ or in $\mathbf{Prob}〈\mathbf{G}〉$, its minimization exists and is unique up to isomorphism.

2.1.4. Tensor Product

The tensor product of two probability spaces $X=(\underline{X},p)$ and $Y=(\underline{Y},q)$ is their independent product $X\otimes Y:=(\underline{X}\times \underline{Y},p\otimes q)$. For two $\mathbf{G}$-diagrams $\mathcal{X}=\left\{X_i;{\chi }_{ij}\right\}$ and $\mathcal{Y}=\left\{Y_i;{\upsilon }_{ij}\right\}$, we define their tensor product to be $\mathcal{X}\otimes \mathcal{Y}=\left\{X_i\otimes \mathcal{Y};{\chi }_{ij}\times {\upsilon }_{ij}\right\}$.

2.1.5. Constant Diagrams

Given an indexing category $\mathbf{G}$ and a probability space, we can form a constant diagram $X^{\mathbf{G}}$ that has all spaces equal to X and all reductions equal to the identity isomorphism. Sometimes, when such a constant diagram is included in a diagram with other $\mathbf{G}$-diagrams (such as, for example, a reduction $\mathcal{X}\to X^{\mathbf{G}}$), we will write simply X in place of $X^{\mathbf{G}}$.

2.1.6. Entropy

Evaluating entropy on every space in a $\mathbf{G}$-diagram, we obtain a tuple of non-negative numbers indexed by objects in $\mathbf{G}$; thus, entropy gives a map

$${\mathsf{Ent}}_{*}:\mathbf{Prob}〈\mathbf{G}〉\to {\mathbb{R}}^{\mathbf{G}},$$

where the target space ${\mathbb{R}}^{\mathbf{G}}$ is a space of real-valued functions on the set of objects in $\mathbf{G}$ endowed with the ${\ell }^1$-norm. Entropy is a homomorphism in that it satisfies

$${\mathsf{Ent}}_{*}(\mathcal{X}\otimes \mathcal{Y})={\mathsf{Ent}}_{*}\left(\mathcal{X}\right)+{\mathsf{Ent}}_{*}\left(\mathcal{Y}\right).$$

2.1.7. Entropy Distance

Let $\mathbf{G}$ be an indexing category and $\mathcal{K}=(\mathcal{X}\leftarrow \mathcal{Z}\to \mathcal{Y})$ be a fan of $\mathbf{G}$-diagrams. We define the entropy distance as

$$\mathrm{kd}\left(\mathcal{K}\right):={∥{\mathsf{Ent}}_{*}\mathcal{Z}-{\mathsf{Ent}}_{*}\mathcal{X}∥}_1+{∥{\mathsf{Ent}}_{*}\mathcal{Z}-{\mathsf{Ent}}_{*}\mathcal{Y}∥}_1.$$

The intrinsic entropy distance between two $\mathbf{G}$-diagrams is defined to be the infimal entropy distance of all fans with terminal diagrams $\mathcal{X}$ and $\mathcal{Y}$:

$$\mathbf{k}(\mathcal{X},\mathcal{Y}):=inf\left\{\mathrm{kd}\left(\mathcal{K}\right):\mathcal{K}=(\mathcal{X}\leftarrow \mathcal{Z}\to \mathcal{Y})\right\}.$$

The intrinsic entropy distance was introduced in [10,11] for probability spaces.

In [1], it is shown that the infimum is attained, that the optimal fan is minimal, that $\mathbf{k}$ is a pseudo-distance, which vanishes if, and only if, $\mathcal{X}$ and $\mathcal{Y}$ are isomorphic, and that ${\mathsf{Ent}}_{*}$ is a 1-Lipschitz linear functional with respect to $\mathbf{k}$.

2.2. Diagrams of Sets, Distributions, and Empirical Reductions

2.2.1. Distributions on Sets

For a set S, we denote by $\Delta S$ the collection of all finitely-supported probability distributions on S. For a pair of distributions ${\pi }_1,{\pi }_2\in \Delta S$, we denote by ${∥{\pi }_1-{\pi }_2∥}_1$ the total variation distance between them.

For a map $f:S\to S^{\prime }$ between two sets, we denote by $f_{*}:\Delta S\to \Delta S^{\prime }$ the induced affine map (the map-preserving convex combinations).

For $n\in \mathbb{N}$, we define the empirical map $\mathbf{q}:S^n\to \Delta S$ by the assignment below. For $\overline{s}=(s_1,\cdots ,s_n)\in S^n$ and $A\subset S$, we define

$$\mathbf{q}\left(\overline{s}\right)\left(A\right):=\frac{1}{n}·|\left\{k:s_k\in A\right\}|.$$

For a finite probability space $X=(S,p)$, the empirical distribution on $\Delta X$ is the push-forward ${\tau }_n:={\mathbf{q}}_{*}{p}^{\otimes n}$. Thus,

$$\mathbf{q}:X^n\to (\Delta X,{\tau }_n)$$

is a reduction of finite probability spaces. The construction of empirical reduction is functorial, which is for a reduction between two probability spaces $f:X\to Y$, the diagram of the reductions

commutes.

2.2.2. Distributions on Diagrams of Sets

Let $\mathbf{Set}$ denote the category of sets and surjective maps. For an indexing category $\mathbf{G}$, we denote by $\mathbf{Set}〈\mathbf{G}〉$ the category of $\mathbf{G}$-diagrams in $\mathbf{Set}$. The objects in $\mathbf{Set}〈\mathbf{G}〉$ are commutative diagrams of sets indexed by $\mathbf{G}$, and the spaces in such a diagram are sets, where the arrows represent surjective maps, subject to commutativity relations.

For a diagram of sets $\mathcal{S}=\left\{S_i;{\sigma }_{ij}\right\}$, we define the space of distributions on the diagram $\mathcal{S}$ by

$$\Delta \mathcal{S}:=\left\{\left({\pi }_i\right)\in \prod _i\Delta S_i:{\left({\sigma }_{ij}\right)}_{*}{\pi }_i={\pi }_j\right\}.$$

If $S_0$ is the initial set of $\mathcal{S}$, then there is an isomorphism of

$$\begin{array}{cc}\hfill \Delta S_0& \stackrel{\cong }{\leftrightarrow }\Delta \mathcal{S}\hfill \\ \hfill \Delta S_0\ni {\pi }_0& \mapsto \left\{{\left({\sigma }_{0i}\right)}_{*}{\pi }_0\right\}\in \Delta \mathcal{S}\hfill \\ \hfill \Delta S_0\ni {\pi }_0& ↤\left\{{\pi }_i\right\}\in \Delta \mathcal{S}.\hfill \end{array}$$

(1)

Given a $\mathbf{G}$-diagram of sets $\mathcal{S}=\left\{S_i;{\sigma }_{ij}\right\}$ and an element $\pi \in \Delta \mathcal{S}$, we can construct a $\mathbf{G}$-diagram of probability spaces $(\mathcal{S},\pi ):=\left\{(S_i,{\pi }_i);{\sigma }_{ij}\right\}$. Note that any diagram $\mathcal{X}$ of probability spaces has this form.

2.3. Conditioning

Consider a $\mathbf{G}$-diagram of probability spaces $\mathcal{X}=(\mathcal{S},\pi )$, where $\mathcal{S}$ is a diagram of sets and $\pi \in \Delta \mathcal{S}$. Let $X_0=(S_0,{\pi }_0)$ be the initial space in $\mathcal{X}$ and $U:=X_i$ be another space in $\mathcal{X}$. Since $S_0$ is initial, there is a map ${\sigma }_{0,i}:S_0\to S_i$. Fix an atom $u\in U$ and define the conditioned distribution ${\pi }_0(·|u)$ on $S_0$ as the distribution supported in ${\sigma }_{0,i}^{-1}\left(u\right)$ and for every $s\in {\sigma }_{0,i}^{-1}\left(u\right)$ defined by

$${\pi }_0\left(s\right|u):=\frac{{\pi }_0\left(s\right)}{{\pi }_0\left({\sigma }_{0,i}^{-1}\left(u\right)\right)}.$$

Let $\pi (·|u)\in \Delta \mathcal{S}$ be the distribution corresponding to ${\pi }_0(·|u)$ under the isomorphism in (1). We define the conditioned $\mathbf{G}$-diagram as $\mathcal{X}|u:=(\mathcal{S},\pi (·|u\left)\right)$.

2.4. The Slicing Lemma

In [1], we prove the so-called Slicing Lemma that allows us to estimate the intrinsic entropy distance between two diagrams in terms of distances between conditioned diagrams. Among the corollaries of the Slicing Lemma is the following inequality.

Proposition 1. 

Let $(\mathcal{X}\leftarrow \widehat{\mathcal{X}}\to U^{\mathbf{G}})\in \mathbf{Prob}〈\mathbf{G},{\mathsf{\Lambda }}_2〉$ be a fan of $\mathbf{G}$-diagrams of probability spaces and $\mathcal{Y}\in \mathbf{Prob}〈\mathbf{G}〉$ be another diagram. Then,

$$\mathbf{k}(\mathcal{X},\mathcal{Y})\le {\int }_U\mathbf{k}\left(\mathcal{X}\right|u,\mathcal{Y})dp\left(u\right)+2\left[\left[\mathbf{G}\right]\right]·\mathsf{Ent}U.$$

The fan in the assumption of the proposition above can often be constructed in the following manner. Suppose $\mathcal{X}$ is a $\mathbf{G}$-diagram and $U:=X_{\iota }$ is a space in it for some $\iota \in \mathbf{G}$. We can construct a fan $\left(\mathcal{X}\stackrel{f}{\leftarrow }\widehat{\mathcal{X}}\stackrel{g}{\to }{U}^{\mathbf{G}}\right)\in \mathbf{Prob}〈\mathbf{G},{\mathsf{\Lambda }}_2〉$ by assigning ${\widehat{X}}_i$ to be the initial space of the (unique) minimal fan in $\mathcal{X}$ with terminal spaces $X_i$ and U and $f_i$ and $g_i$ to be left and right reductions in that fan for any $i\in \mathbf{G}$.

2.5. Tropical Diagrams

A detailed discussion of the topics in this section can be found in [2].

The asymptotic entropy distance between two diagrams of the same combinatorial type is defined by

$$\mathit{\kappa }(\mathcal{X},\mathcal{Y}):=lim\frac{1}{n}\mathbf{k}({\mathcal{X}}^n,{\mathcal{Y}}^n).$$

A tropical $\mathbf{G}$-diagram is an equivalence class of certain sequences of $\mathbf{G}$-diagrams of probability spaces. Below, we describe the type of sequences and the equivalence relation.

A function $\phi :\mathbb{R}\ge 1\to \mathbb{R}\ge 0$ is called an admissible function if $\phi$ is non-decreasing and there is a constant $D_{\phi }$, such that for any $t\ge 1$:

$$t·{\int }_t^{\infty }\frac{\phi \left(s\right)}{s^2}ds\le D_{\phi }·\phi \left(t\right).$$

An example of an admissible function will be $\phi \left(t\right)=t^{\alpha }$ for $\alpha \in [0,1)$.

A sequence $\overline{\mathcal{X}}=(\mathcal{X}\left(n\right):n\in {\mathbb{N}}_0)$ of diagrams of probability spaces will be called quasi-linear with the defect bounded by an admissible function $\phi$ if for some $C>0$ and all $m,n\in \mathbb{N}$, it satisfies

$$\mathit{\kappa }\left(\mathcal{X}(n+m),\mathcal{X}\left(n\right)\otimes \mathcal{X}\left(m\right)\right)\le C·\phi (n+m).$$

For example for a diagram $\mathcal{X}$, the sequence $\overrightarrow{\mathcal{X}}:=({\mathcal{X}}^n:n\in {\mathbb{N}}_0)$ is $\phi$-quasi-linear for $\phi \equiv 0$ (and for any admissible $\phi$). Sequences with zero defect are called linear, and the space of all linear sequences in $\mathbf{Prob}〈\mathbf{G}〉$ is denoted by $\mathsf{L}\left(\mathbf{Prob}〈\mathbf{G}〉\right)$.

The asymptotic entropic distance between two $\phi$-quasi-linear sequences $\overline{\mathcal{X}}=\left(\mathcal{X}\left(n\right):n\in {\mathbb{N}}_0\right)$ and $\overline{\mathcal{Y}}=\left(\mathcal{Y}\left(n\right):n\in {\mathbb{N}}_0\right)$ is defined to be

$$\mathit{\kappa }(\overline{\mathcal{X}},\overline{\mathcal{Y}}):=\underset{n\to \infty }{lim}\frac{1}{n}\mathbf{k}(\mathcal{X}\left(n\right),\mathcal{Y}\left(n\right)),$$

and the sequences are called asymptotically equivalent if $\mathit{\kappa }(\overline{\mathcal{X}},\overline{\mathcal{Y}})=0$. An equivalence class of a sequence $\overline{\mathcal{X}}$ will be denoted as $\left[\mathcal{X}\right]$, and the totality of all the classes as $\mathbf{Prob}\left[\mathbf{G}\right]$. We have shown in [2] that the space of equivalence classes of $\phi$-quasi-linear sequences does not depend on the choice of a non-zero admissible function $\phi$.

The sum of two such equivalence classes is defined to be the equivalence class of the sequence obtained by tensor-multiplying representative sequences of the summands term-wise. In addition, there is a doubly transitive action of ${\mathbb{R}}_{\ge 0}$ on $\mathbf{Prob}\left[\mathbf{G}\right]$. In [2], the following theorem is proven.

Theorem 1. 

Let $\mathbf{G}$ be an indexing category. Then:

1. 

The space $\mathbf{Prob}\left[\mathbf{G}\right]$ does not depend on the choice of a positive admissible function φ up to isometry.

2. 

The space $\mathbf{Prob}\left[\mathbf{G}\right]$ is metrically complete.

3. 

The map $\mathcal{X}\mapsto \overrightarrow{\mathcal{X}}$ is a $\mathit{\kappa }$-$\mathit{\kappa }$-isometric embedding. The space of linear sequences, i.e., the image of the map above, is dense in $\mathbf{Prob}\left[\mathbf{G}\right]$.

4. 

There is a distance-preserving homomorphism from $\mathbf{Prob}\left[\mathbf{G}\right]$ into a Banach space B, whose image is a closed convex cone in B.

5. 

The entropy functional

$$\begin{array}{cc}\hfill {\mathsf{Ent}}_{*}:\mathbf{Prob}\left[\mathbf{G}\right]& \to {\mathbb{R}}^{\mathbf{G}}\hfill \\ \hfill [{\left(\mathcal{X}\left(n\right)\right)}_{n\in {\mathbb{N}}_0}]& \mapsto \underset{n\to \infty }{lim}\frac{1}{n}{\mathsf{Ent}}_{*}\mathcal{X}\left(n\right)\hfill \end{array}$$

is a well-defined 1-Lipschitz linear map.

2.6. Asymptotic Equipartition Property for Diagrams

Among all $\mathbf{G}$-diagrams, there is a special class of maximally symmetric ones. We call such diagrams homogeneous; see below for the definition. Homogeneous diagrams come in very handy in many considerations, because their structure is easier to describe than that of general diagrams. We show below that among the tropical diagrams, those that have homogeneous representatives are dense. It means, in particular, that when considering continuous functionals in the space of diagrams, it suffices to only study them in the space of all homogeneous diagrams.

2.6.1. Homogeneous Diagrams

A $\mathbf{G}$-diagram $\mathcal{X}$ is called homogeneous if the automorphism group $\mathrm{Aut}\left(\mathcal{X}\right)$ acts transitively on every space in $\mathcal{X}$, by which we mean that the action is transitive on the support of the probability measure. Homogeneous probability spaces are isomorphic to uniform spaces. For more complex indexing categories, this simple description is not sufficient.

2.6.2. Tropical Homogeneous Diagrams

The subcategory of all homogeneous $\mathbf{G}$-diagrams will be denoted $\mathbf{Prob}{〈\mathbf{G}〉}_{\mathsf{h}}$, and we write $\mathbf{Prob}{〈\mathbf{G}〉}_{\mathsf{h},\mathsf{m}}$ for the category of minimal homogeneous $\mathbf{G}$-diagrams. These spaces are invariant under the tensor product; thus, they are metric Abelian monoids, and the general "tropicalization" described in [2] can be performed. Passing to the tropical limit, we obtain spaces of tropical (minimal) homogeneous diagrams, which we denote by $\mathbf{Prob}{\left[\mathbf{G}\right]}_{\mathsf{h}}$ and $\mathbf{Prob}{\left[\mathbf{G}\right]}_{\mathsf{h},\mathsf{m}}$, respectively.

2.6.3. Asymptotic Equipartition Property

For an indexing category $\mathbf{G}$, denote by $\left[\right[G\left]\right]$ the number of objects in $\mathbf{G}$. In [1], the following theorem is proven.

Theorem 2. 

Suppose $\mathcal{X}\in \mathbf{Prob}〈\mathbf{G}〉$ is a $\mathbf{G}$-diagram of probability spaces for some fixed indexing category $\mathbf{G}$. Then, there exists a sequence $\overline{\mathcal{H}}={\left({\mathcal{H}}_n\right)}_{n=0}^{\infty }$ of homogeneous $\mathbf{G}$-diagrams, such that

$$\frac{1}{n}\mathbf{k}({\mathcal{X}}^n,{\mathcal{H}}_n)\le C\left(\right|X_0|,\left[\left[\mathbf{G}\right]\right])·\sqrt{\frac{{ln}^{3}n}{n}},$$

(2)

where $C\left(\right|X_0|,\left[\left[\mathbf{G}\right]\right])$ is a constant only depending on $|X_0|$ and $\left[\right[\mathbf{G}\left]\right]$.

The approximating sequence of homogeneous diagrams is evidently quasi-linear with the defect bounded by the admissible function

$$\phi \left(t\right):=2C\left(\right|X_0|,\left[\left[\mathbf{G}\right]\right])·t^{3/4}\ge 2C\left(\right|X_0|,\left[\left[\mathbf{G}\right]\right])·t^{1/2}·{ln}^{3/2}t.$$

Thus, Theorem 2 above states that $\mathsf{L}\left(\mathbf{Prob}〈\mathbf{G}〉\right)\subset \mathbf{Prob}{\left[\mathbf{G}\right]}_{\mathsf{h}}$. On the other hand, we have shown in [2] that the space of linear sequences $\mathsf{L}\left(\mathbf{Prob}〈\mathbf{G}〉\right)$ is dense in $\mathbf{Prob}\left[\mathbf{G}\right]$. Combining the two statements, we obtain the following theorem.

Theorem 3. 

For any indexing category $\mathbf{G}$, the space $\mathbf{Prob}{\left[\mathbf{G}\right]}_{\mathsf{h}}$ is dense in $\mathbf{Prob}\left[\mathbf{G}\right]$. Similarly, the space $\mathbf{Prob}{\left[\mathbf{G}\right]}_{\mathsf{h},\mathsf{m}}$ is dense in $\mathbf{Prob}{\left[\mathbf{G}\right]}_{\mathsf{m}}$.

3. Conditioning of Tropical Diagrams

3.1. Motivation

Let $\mathcal{X}\in \mathbf{Prob}〈\mathbf{G}〉$ be a $\mathbf{G}$-diagram of probability spaces containing probability space $U=X_{i_0}$ indexed by an object $i_0\in \mathbf{G}$.

Given an atom $u\in U$, we can define a conditioned diagram $\mathcal{X}|u$. If the diagram $\mathcal{X}$ is homogeneous, then the isomorphism class of $\mathcal{X}|u$ is independent of u, so that $\left(\mathcal{X}\right|u:u\in U)$ is a constant family. On the other hand, we have shown that the power of any diagram can be approximated by homogeneous diagrams, thus suggesting that in the tropical setting $\mathcal{X}|U$ should be a well-defined tropical diagram, rather than a family. Below, we give a definition of the tropical conditioning operation and prove its consistency.

3.2. Classical-Tropical Conditioning

Here, we define the operation of conditioning of the classical diagram, such that the result is a tropical diagram. Let $\mathcal{X}$ be a $\mathbf{G}$-diagram of probability spaces and U be a space in $\mathcal{X}$. We define the conditioning map

$$[·|·]:\mathbf{Prob}〈\mathbf{G}〉\to \mathbf{Prob}\left[\mathbf{G}\right]$$

by conditioning $\mathcal{X}$ by $u\in U$ and averaging the corresponding tropical diagrams:

$$\left[\mathcal{X}\right|U]:={\int }_{u\in U}\overrightarrow{\left(\mathcal{X}\right|u)}d{p}_U\left(u\right),$$

where $\overrightarrow{\left(\mathcal{X}\right|u)}$ is the tropical diagram represented by a linear sequence generated by $\mathcal{X}|u$; see Section 2.5. Note that the integral on the right-hand side is just a finite convex combination of tropical diagrams. Expanding all the definitions, we will obtain for $\left[\mathcal{Y}\right]:=\left[\mathcal{X}\right|U]$, the representative sequence

$$\mathcal{Y}\left(n\right)=\underset{u\in U}{⨂}{\left(\mathcal{X}\right|u)}^{\lfloor n·p\left(u\right)\rfloor }.$$

3.3. Properties

3.3.1. Conditioning of Homogeneous Diagrams

If the diagram $\mathcal{X}$ is homogeneous, then for any atom $u\in U$, with a positive weight,

$$\left[\mathcal{X}\right|U]\check{=}\overrightarrow{\left(\mathcal{X}\right|u)}.$$

3.3.2. Entropy

By definition, the conditioned entropy is

$${\mathsf{Ent}}_{*}\left(\mathcal{X}\right|U):={\int }_U{\mathsf{Ent}}_{*}\left(\mathcal{X}\right|u)d{p}_U\left(u\right).$$

Now that $\left[\mathcal{X}\right|U]$ is a tropical diagram, the expression ${\mathsf{Ent}}_{*}\left(\mathcal{X}\right|U)$ can be interpreted in two, a priori different, ways: by the formula above and as the entropy of the object introduced in the previous subsection. Fortunately, the numeric value of it does not depend on the interpretation since the entropy is a linear functional on $\mathbf{Prob}\left[\mathbf{G}\right]$.

3.3.3. Additivity

If $\mathcal{X}$ and $\mathcal{Y}$ are two $\mathbf{G}$-diagrams with $U:=X_{\iota }$, $V:=Y_{\iota }$ for some $\iota \in \mathbf{G}$, then

$$\left[\right(\mathcal{X}\otimes \mathcal{Y}\left)\right|(U\otimes V)]=[\mathcal{X}\left|U\right]+\left[\mathcal{Y}\right|V].$$

Proof. 

$$\begin{array}{cc}\hfill \left[\right(\mathcal{X}\otimes & \mathcal{Y})|(U\otimes V)]={\int }_{U\otimes V}\overrightarrow{(\mathcal{X}\otimes \mathcal{Y})|(u,v)}dp\left(u\right)dp\left(v\right)\hfill \\ \hfill & ={\int }_{U\otimes V}(\overrightarrow{\mathcal{X}|u}+\overrightarrow{\mathcal{Y}|v})dp\left(u\right)dp\left(v\right)={\int }_U\overrightarrow{\mathcal{X}|u}dp\left(u\right)+{\int }_V\overrightarrow{\mathcal{Y}|v}dp\left(v\right)\hfill \\ \hfill & =[\mathcal{X}|U]+[\mathcal{Y}|V]\hfill \end{array}$$

□

3.3.4. Homogeneity

It follows that for any diagram $\mathcal{X}$, a space U in $\mathcal{X}$ and $n\in {\mathbb{N}}_0$ holds

$$\left[{\mathcal{X}}^n\right|U^n]=n·\left[\mathcal{X}\right|U].$$

3.4. Continuity and Lipschitz Property

Proposition 2. 

Let $\mathbf{G}$ be an indexing category, $\mathcal{X},\mathcal{Y}\in \mathbf{Prob}〈\mathbf{G}〉$ be two $\mathbf{G}$ diagrams, and $U:=X_{\iota }$ and $V:=Y_{\iota }$ be two spaces in $\mathcal{X}$ and $\mathcal{Y}$, respectively, indexed by some $\iota \in \mathbf{G}$. Then,

$$\mathit{\kappa }\left(\left[\mathcal{X}\right|U],[\mathcal{Y}\left|V\right]\right)\le (2·\left[\left[\mathbf{G}\right]\right]+1)·\mathbf{k}\left(\mathcal{X},\mathcal{Y}\right).$$

Using the homogeneity property of conditioning, Section 3.3.4, we can obtain the following stronger inequality.

Corollary 1. 

In the setting of Proposition 2, the following holds:

$$\mathit{\kappa }\left(\left[\mathcal{X}\right|U],[\mathcal{Y}\left|V\right]\right)\le (2·\left[\left[\mathbf{G}\right]\right]+1)·\mathit{\kappa }\left(\mathcal{X},\mathcal{Y}\right).$$

Before we prove Proposition 2, we will need some preparatory lemmas.

Lemma 1. 

Let $\mathcal{A}$ be a $\mathbf{G}$-diagram of probability spaces and E be a space in it. Let $\mathbf{q}:E^n\to (\Delta E,{\tau }_n)$ be the empirical reduction. Then, for any $n\in \mathbb{N}$ and any $\overline{e},{\overline{e}}^{\prime }\in ,E^n$

$$\mathbf{k}({\mathcal{A}}^n|\overline{e},{\mathcal{A}}^n|{\overline{e}}^{\prime })\le n·\parallel {\mathsf{Ent}}_{*}{\left(\mathcal{A}\right)\parallel }_1·{\parallel \mathbf{q}\left(\overline{e}\right)-\mathbf{q}\left({\overline{e}}^{\prime }\right)\parallel }_1.$$

Proof. 

To prove the lemma, we construct a coupling between ${\mathcal{A}}^n|\overline{e}$ and ${\mathcal{A}}^n|{\overline{e}}^{\prime }$ in the following manner. Note that there exists a permutation $\sigma \in S_n$, such that

$$|\left\{i:e_i\ne e_{\sigma i}^{\prime }\right\}|=\frac{n}{2}·{\parallel \mathbf{q}\left(\overline{e}\right)-\mathbf{q}\left({\overline{e}}^{\prime }\right)\parallel }_1.$$

Let

$$\begin{array}{cc}\hfill I& =\left\{i:e_i=e_{\sigma i}^{\prime }\right\}\hfill \\ \hfill \tilde{I}& =\left\{i:e_i\ne e_{\sigma i}^{\prime }\right\}.\hfill \end{array}$$

Using that $|\tilde{I}|=\frac{n}{2}·{\parallel \mathbf{q}\left(\overline{e}\right)-\mathbf{q}\left({\overline{e}}^{\prime }\right)\parallel }_1$, we can estimate

$$\begin{array}{cc}\hfill \mathbf{k}\left({\mathcal{A}}^n|\overline{e},{\mathcal{A}}^n|{\overline{e}}^{\prime }\right)& =\mathbf{k}\left(\underset{i=1}{\overset{n}{⨂}}\left(\mathcal{A}\right|e_i),\underset{i=1}{\overset{n}{⨂}}\left(\mathcal{A}\right|e_{\sigma i}^{\prime })\right)\hfill \\ \hfill & \le \sum _{i\in I}\mathrm{kd}\left(\mathcal{A}\right|e_i\stackrel{=}{⟷}\mathcal{A}|e_{\sigma i}^{\prime })+\sum _{i\in \tilde{I}}\mathrm{kd}\left(\mathcal{A}\right|e_i\stackrel{\otimes }{⟷}\mathcal{A}\left|e_{\sigma i}^{\prime }\right)\hfill \\ \hfill & \le n·\parallel {\mathsf{Ent}}_{*}{\left(\mathcal{A}\right)\parallel }_1·{\parallel \mathbf{q}\left(\overline{e}\right)-\mathbf{q}\left({\overline{e}}^{\prime }\right)\parallel }_1,\hfill \end{array}$$

where $\mathcal{A}\stackrel{=}{\leftrightarrow }\mathcal{B}$ denotes the isomorphism coupling of two naturally isomorphic diagrams, while $\mathcal{A}\stackrel{\otimes }{\leftrightarrow }\mathcal{B}$ denotes the “independence” coupling. □

Lemma 2. 

Let $\mathcal{A}$ be a $\mathbf{G}$-diagram of probability spaces and E be a space in $\mathcal{A}$. Then,

$${\int }_{E^n}\mathbf{k}({\mathcal{A}}^n,{\mathcal{A}}^n|\overline{e})dp\left(\overline{e}\right)\le 2n·\left[\left[\mathbf{G}\right]\right]·\mathsf{Ent}\left(E\right)+\mathcal{o}\left(n\right).$$

Proof. 

First, we apply Proposition 1 slicing the first argument:

$$\begin{array}{c}{\int }_{E^n}\mathbf{k}({\mathcal{A}}^n,{\mathcal{A}}^n|\overline{e})dp\left(\overline{e}\right)\hfill \\ \le {\int }_{E^n}{\int }_{E^n}\mathbf{k}({\mathcal{A}}^n|{\overline{e}}^{\prime },{\mathcal{A}}^n|\overline{e})dp\left({\overline{e}}^{\prime }\right)dp\left(\overline{e}\right)+2n·\left[\left[\mathbf{G}\right]\right]·\mathsf{Ent}\left(E\right).\hfill \end{array}$$

We will argue now that the double integral on the right-hand side grows sub-linearly with n. We estimate the double integral by applying Lemma 1 to the integrand

$$\begin{array}{c}{\int }_{E^n}{\int }_{E^n}\mathbf{k}({\mathcal{A}}^n|{\overline{e}}^{\prime },{\mathcal{A}}^n\left|\overline{e}\right)dp\left({\overline{e}}^{\prime }\right)dp\left(\overline{e}\right)\hfill \\ \le {\int }_{E^n}{\int }_{E^n}n·\left[\left[\mathbf{G}\right]\right]·|{\mathsf{Ent}}_{*}{\left(\mathcal{A}\right)|}_1·{|\mathbf{q}\left(\overline{e}\right)-\mathbf{q}\left({\overline{e}}^{\prime }\right)|}_{1}dp\left({\overline{e}}^{\prime }\right)dp\left(\overline{e}\right)\hfill \\ \hfill =n·\left[\left[\mathbf{G}\right]\right]·|{\mathsf{Ent}}_{*}{\left(\mathcal{A}\right)|}_1·{\int }_{\Delta E}{\int }_{\Delta E}{|\pi -{\pi }^{\prime }|}_{1}d{\tau }_n\left(\pi \right)d{\tau }_n\left({\pi }^{\prime }\right)=\mathcal{o}\left(n\right),\end{array}$$

where the convergence to zero of the last double integral follows from Sanov’s theorem. □

Corollary 2. 

Let $\mathcal{A}$ be a $\mathbf{G}$-diagram and E a probability space included in $\mathcal{A}$. Then,

$$\mathit{\kappa }\left(\overrightarrow{\mathcal{A}},\left[\mathcal{A}\right|E]\right)\le 2\left[\left[\mathbf{G}\right]\right]·\mathsf{Ent}\left(E\right).$$

Proof. 

Let $n\in \mathbb{N}$. Then,

$$\begin{array}{cc}\hfill \mathit{\kappa }\left(\overrightarrow{\mathcal{A}},\left[\mathcal{A}\right|E]\right)& =\frac{1}{n}\mathit{\kappa }\left(\overrightarrow{{\mathcal{A}}^n},\left[{\mathcal{A}}^n\right|E^n]\right)\hfill \\ \hfill & =\frac{1}{n}\mathit{\kappa }\left(\overrightarrow{{\mathcal{A}}^n},{\int }_{E^n}\overrightarrow{{\mathcal{A}}^n|\overline{e}}dp\left(\overline{e}\right)\right)\hfill \\ \hfill & \le \frac{1}{n}{\int }_{E^n}\mathit{\kappa }\left(\overrightarrow{{\mathcal{A}}^n},\overrightarrow{{\mathcal{A}}^n|\overline{e}}\right)dp\left(\overline{e}\right)\hfill \\ \hfill & =\frac{1}{n}{\int }_{E^n}\mathit{\kappa }({\mathcal{A}}^n,{\mathcal{A}}^n|\overline{e})dp\left(\overline{e}\right)\hfill \\ \hfill & \le 2·\left[\left[\mathbf{G}\right]\right]·\mathsf{Ent}\left(E\right)+\mathcal{o}\left(n^0\right),\hfill \end{array}$$

where we used Lemma 2 and the fact that $\mathit{\kappa }\le \mathbf{k}$ in the last line. We finish the proof by taking the limit $n\to \infty$. □

Proof of Proposition 2.

We start with a note on general terminology: a reduction $f:A\to B$ of probability spaces can also be considered as a fan $\mathcal{F}:=\left(A\stackrel{=}{\leftarrow }A\stackrel{f}{\to }B\right)$. Then, the entropy distance of f is

$$\mathrm{kd}\left(f\right):=\mathrm{kd}\left(\mathcal{F}\right)=\mathsf{Ent}A-\mathsf{Ent}B.$$

If the reduction f is a part of a bigger diagram containing also space U, then the following inequality holds:

$${\int }_U\mathrm{kd}\left(f\right|u)dp\left(u\right)\le \mathrm{kd}\left(f\right).$$

Let $\mathcal{K}\in \mathbf{Prob}〈\mathbf{G},{\mathsf{\Lambda }}_2〉$

$$\mathcal{K}=\left(\mathcal{X}\stackrel{f}{\leftarrow }\mathcal{Z}\stackrel{g}{\to }\mathcal{Y}\right)\in \mathbf{Prob}〈\mathbf{G},{\mathsf{\Lambda }}_2〉=\mathbf{Prob}〈{\mathsf{\Lambda }}_2,\mathbf{G}〉$$

be an optimal coupling between $\mathcal{X}$ and $\mathcal{Y}$. It can also be viewed as a $\mathbf{G}$-diagram of two fans, $\mathcal{K}={\left\{{\mathcal{K}}_i\right\}}_{i\in \mathbf{G}}$, each of which is a minimal coupling between $X_i$ and $Y_i$. Among them is the minimal fan $\mathcal{W}:={\mathcal{K}}_{\iota }=\left(U\stackrel{f_{\iota }}{⟵}W\stackrel{g_{\iota }}{⟶}V\right)$.

We use the triangle inequality to bound the distance $\mathit{\kappa }\left(\left[\mathcal{X}\right|U],[\mathcal{Y}\left|V\right]\right)$ by four summands as follows:

$$\begin{array}{cc}\hfill \mathit{\kappa }\left(\left[\mathcal{X}\right|U],[\mathcal{Y}\left|V\right]\right)\le & \mathit{\kappa }\left(\left[\mathcal{X}\right|U],[\mathcal{Z}\left|U\right]\right)+\mathit{\kappa }\left(\left[\mathcal{Z}\right|U],[\mathcal{Z}\left|W\right]\right)+\hfill \\ & \mathit{\kappa }\left(\left[\mathcal{Z}\right|W],[\mathcal{Z}\left|V\right]\right)+\mathit{\kappa }\left(\left[\mathcal{Z}\right|V],[\mathcal{Y}\left|V\right]\right).\hfill \end{array}$$

We will estimate each of the four summands separately. The bound for the first one is as follows:

$$\begin{array}{c}\mathit{\kappa }(\left[\mathcal{X}\right|U],\left[\mathcal{Z}\right|U])=\mathit{\kappa }\left({\int }_U\overrightarrow{\mathcal{X}|u}dp\left(u\right),{\int }_U\overrightarrow{\mathcal{Z}|u}dp\left(u\right)\right)\hfill \\ \le {\int }_U\mathit{\kappa }\left(\overrightarrow{\mathcal{X}|u},\overrightarrow{\mathcal{Z}|u}\right)dp\left(u\right)={\int }_U\mathit{\kappa }\left(\mathcal{X}|u,\mathcal{Z}|u\right)dp\left(u\right)\hfill \\ \le {\int }_U\mathbf{k}\left(\mathcal{X}|u,\mathcal{Z}|u\right)dp\left(u\right)\le {\int }_U\mathrm{kd}\left(f\right|u)dp\left(u\right)\hfill \\ \le \sum _{i\in \mathbf{G}}{\int }_U\mathrm{kd}\left(f_i\right|u)dp\left(u\right)=\sum _{i\in \mathbf{G}}\mathrm{kd}\left(f_i\right)=\mathrm{kd}\left(f\right).\hfill \end{array}$$

An analogous calculation shows that

$$\mathit{\kappa }\left(\left[\mathcal{Z}\right|V],[\mathcal{Y}\left|V\right]\right)\le \mathrm{kd}\left(g\right).$$

To bound the second summand, we will use Corollary 2:

$$\begin{array}{cc}\hfill \mathit{\kappa }\left(\left[\mathcal{Z}\right|U],[\mathcal{Z}\left|W\right]\right)& =\mathit{\kappa }\left({\int }_U\overrightarrow{\mathcal{Z}|u}dp\left(u\right),{\int }_W\overrightarrow{\mathcal{Z}|w}dp\left(w\right)\right)\hfill \\ \hfill & =\mathit{\kappa }\left({\int }_U\overrightarrow{\mathcal{Z}|u}dp\left(u\right),{\int }_U{\int }_{W|u}\overrightarrow{\mathcal{Z}|w}dp\left(w\right|u)dp\left(u\right)\right)\hfill \\ \hfill & \le {\int }_U\mathit{\kappa }\left(\overrightarrow{\mathcal{Z}|u},{\int }_{W|u}\overrightarrow{\mathcal{Z}|w}dp\left(w\right|u)\right)dp\left(u\right).\hfill \end{array}$$

We will now use Corollary 2 with $\mathcal{A}=\mathcal{Z}|u$ and $E=W|u$ to estimate the integrand. Then,

$$\begin{array}{cc}\hfill \mathit{\kappa }\left(\left[\mathcal{Z}\right|U],[\mathcal{Z}\left|W\right]\right)& ={\int }_U\mathit{\kappa }\left(\overrightarrow{\mathcal{Z}|u},{\int }_{W|u}\overrightarrow{\mathcal{Z}|w}dp\left(w\right|u)\right)dp\left(u\right)\hfill \\ \hfill & \le 2\left[\left[\mathbf{G}\right]\right]·{\int }_U\mathsf{Ent}\left(W\right|u)dp\left(u\right)\hfill \\ \hfill & \le 2\left[\right[\mathbf{G}\left]\right]·\mathsf{Ent}\left(W\right|U)\le 2[\left[\mathbf{G}\right]]·\mathrm{kd}(f).\hfill \end{array}$$

Similarly,

$$\mathit{\kappa }\left(\left[\mathcal{Z}\right|W],[\mathcal{Z}\left|V\right]\right)\le 2\left[\left[\mathbf{G}\right]\right]·\mathrm{kd}\left(g\right).$$

Combining the estimates, we obtain

$$\mathit{\kappa }\left(\left[\mathcal{X}\right|U],[\mathcal{Y}\left|V\right]\right)\le (2\left[\left[\mathbf{G}\right]\right]+1)·(\mathrm{kd}\left(f\right)+\mathrm{kd}\left(g\right))=(2\left[\left[\mathbf{G}\right]\right]+1)·\mathbf{k}(\mathcal{X},\mathcal{Y}).$$

□

3.5. Tropical Conditioning

Let $\left[\mathcal{X}\right]$ be a tropical $\mathbf{G}$-diagram and $\left[U\right]=\left[X_{\iota }\right]$ for some $\iota \in \mathbf{G}$. Choose a representative ${\left(\mathcal{X}\left(n\right)\right)}_{n\in {\mathbb{N}}_0}$ and denote $u\left(n\right):=X_{\iota }\left(n\right)$. We define now a conditioned diagram $\left[\mathcal{X}\right|U]$ by the following limit:

$$\left[\mathcal{X}\right|U]:=\underset{n\to \infty }{lim}\frac{1}{n}\left[\mathcal{X}\left(n\right)\right|U\left(n\right)].$$

Proposition 1 guarantees that the limit exists and is independent of the choice of representative. For a fixed $\iota \in \mathbf{G}$, the conditioning is a linear Lipschitz map of

$$[·|{·}_{\iota }]:\mathbf{Prob}\left[\mathbf{G}\right]\to \mathbf{Prob}\left[\mathbf{G}\right].$$