Entropic Matroids and Their Representation

Abstract

This paper investigates entropic matroids, that is, matroids whose rank function is given as the Shannon entropy of random variables. In particular, we consider p-entropic matroids, for which the random variables each have support of cardinality p. We draw connections between such entropic matroids and secret-sharing matroids and show that entropic matroids are linear matroids when $p=2,3$ but not when $p=9$. Our results leave open the possibility for p-entropic matroids to be linear whenever p is prime, with particular cases proved here. Applications of entropic matroids to coding theory and cryptography are also discussed.

1. Introduction

Matroid theory generalizes the notion of independence and rank beyond vector spaces. In a graphical matroid, for example, the rank of a subset of edges is the size of an acyclic spanning set of edges; analogous to the rank of a subset of vectors, which is the size of a spanning set of linearly independent vectors. It is natural to ask whether such combinatorial structures can also be obtained from probabilistic notions of independence, based on random variables. In particular, the entropy can be used to measure dependencies between random variables and it can be used to define a matroid rank function as discussed below. One can then investigate how such entropic matroids relate to other matroids, in particular whether they admit linear representations as graphical matroids do. Before giving formal definitions of such entropic matroids, we give some general definitions for matroids.

1.1. Definitions

We recall a few standard definitions related to matroids, see, for example, Oxley [1]. A matroid is a pair $M=(E,r)$, where the ground set E is a finite set (typically $E=\left[m\right]$, $m\in {\mathbb{Z}}_{+}$) and where the rank function $r:2^E\to {\mathbb{Z}}_{+}$ satisfies

• For any $A\subseteq E$, $r\left(A\right)\le \left|A\right|$ (normalization);
• For any $A\subseteq B\subseteq E$, $r\left(A\right)\le r\left(B\right)$ (monotonicity);
• For any $A,B\subseteq E$, $r(A\cup B)+r(A\cap B)\le r\left(A\right)+r\left(B\right)$ (submodularity).

The submodularity property can be interpreted as a diminishing return property: for every $A\subseteq B$ and $x\in E$,

$$\begin{array}{c}\hfill r(A\cup x)-r\left(A\right)\ge r(B\cup x)-r\left(B\right),\end{array}$$

(1)

that is, the larger the set, the smaller the increase in rank when adding a new element. Independent sets in a matroid are the subsets $S\subseteq E$ such that $r\left(S\right)=\left|S\right|$ and maximal independent sets are called bases, whereas minimal dependent sets are called circuits.

A matroid $M=(E,r)$ is linear if there is a vector space V and a map $f:E\to V$ such that $r\left(S\right)=rank\left(f\right(S\left)\right)$ for all $S\subseteq E$, where $rank$ denotes the rank function of V, that is, $rank\left(f\right(S\left)\right)=dimspan\left(f\right(S\left)\right)$. We say that a matroid is $\mathbb{F}$-representable if in addition, V can be chosen as a vector space over the field $\mathbb{F}$.

Given a matroid M, a minor of $M=(E,\mathcal{F})$ is a matroid that can be obtained from M by a finite sequence of the following two operations:

• Restriction: Given $A\subseteq E$, we define the matroid $M|A=(A,\mathcal{F}\cap 2^A)$.
• Contraction: Given an independent set $A\in \mathcal{F}$, we define the matroid $M/A=(E\setminus A,\left\{B\subseteq E\setminus A:B\cup A\in \mathcal{F}\right\})$.

We define the dual $M^{*}=(E,r^{*})$ of a matroid $M=(E,r)$ is defined by letting $r^{*}\left(A\right)=r(E\setminus A)+\left|A\right|-r\left(E\right)$ for all $A\subseteq E$. A matroid property is a dual property if M has the property if and only if $M^{*}$ does.

Theorem 1

(Woodall [2]). Being an F-representable matroid is a dual property, thas is, M is F-representable if and only if $M^{*}$ is.

1.2. Entropic Matroids

One may expect that matroids could also result from probabilistic structures. Perhaps the first possibility would be to define a matroid to be ‘probabilistic’ if its elements can be represented by random variables (with a joint distribution on some domain), such that a subset S is independent if the random variables indexed by S are mutually independent. This, however, does not necessarily give a matroid. For example, let $X_1$ and $X_2$ be independent random variables (for example, normally distributed) and let $X_3=X_1+X_2$. Let $A=\left\{3\right\}$, $B=\{1,3\}$ and $x=\left\{2\right\}$. Then $r(A\cup x)-r\left(A\right)=0$ since $X_2$ and $X_3$ are dependent but $r(B\cup x)-r\left(B\right)=1$ since $B\cup x=\{1,2,3\}$ contains two independent random variables. So this violates the submodularity requirement.

On the other hand, it is well known that the entropy function satisfies the monotonicity and submodularity properties [3,4]. Namely, for a probability measure $\mu$ on a discrete set $\mathcal{X}$, the entropy of μ in base q is defined by

$$\begin{array}{cc}\hfill H\left(\mu \right)& =-\sum _{x\in \mathcal{X}}\mu \left(x\right){log}_q\mu \left(x\right).\hfill \end{array}$$

(2)

For two random variables X and Y with values in $\mathcal{X}$ and $\mathcal{Y}$ respectively and with joint distribution $\mu$, we define the conditional entropy

$$\begin{array}{c}\hfill H\left(X\right|Y)=\sum _{x\in \mathcal{X},y\in \mathcal{Y}}\mu (x,y)log\frac{\mu (x,y)}{{\sum }_{u\in \mathcal{X}}\mu (u,y)}.\end{array}$$

(3)

In particular, we have the chain rule of entropy $H\left(X\right|Y)=H(X,Y)-H(Y)$. We also define the Hamming distance of two vectors x and y as $d(x,y)=|\left\{1\le i\le n:x_i\ne y_i\right\}|$ and the Hamming ball of radius r around x as $B_r\left(x\right)=\left\{y:d(x,y)\le r\right\}$.

Furthermore, for a probability measure $\mu$ of m random variables defined each on a domain $\mathcal{X}$, that is, for a probability distribution $\mu$ on ${\mathcal{X}}^m$, one can define the function

$$\begin{array}{c}\hfill r\left(S\right)=H\left({\mu }_S\right),S\subseteq \left[m\right],\end{array}$$

(4)

where ${\mu }_S$ is the marginal of $\mu$ on S, that is,

$$\begin{array}{cc}\hfill {\mu }_S\left(x\left[S\right]\right)& =\sum _{x_i\in \left[q\right]:i\notin S}\mu \left(x\right),x\left[S\right]=\{x_i:i\in S\}.\hfill \end{array}$$

(5)

By choosing the base q for the entropy in (4) to be $\left|\mathcal{X}\right|$, we also get that $r\left(S\right)\le \left|S\right|$, with equality for uniform measures. Therefore, the above r satisfies the three axioms of a rank function, with the exception that r is not necessarily integral. In fact this defines a polymatroid (and r is also called a $\beta$-function [5]) and entropic polymatroids (i.e., polymatroids derived from such entropic $\beta$-functions) have been studied extensively in the literature; see References [6,7,8,9] and references therein. Using the Shannon entropy to study matroid structures already emerged in the works [10,11], where the family of pairs of sets $(i,j)$ and K such that $K\subseteq \left[m\right]$, $i,j\in \left[m\right]\setminus K$ is called probabilistically representable if there exit random variables ${\left\{X_k\right\}}_{k\in \left[m\right]}$ such that $X_i$ and $X_j$ are conditionally independent given $X_K$, with the latter expressed in terms of the Shannon entropy as $r(i,K)+r(j,K)-r(i,j,K)-r\left(K\right)=0$.

However, we can also investigate what happens if this function r is in fact integral. This is the object of study in this paper.

Definition 1.

Let $q\in {\mathbb{Z}}_{+}$. A matroid $M=\left(\right[m],r)$ is q-entropicif there is a probability distribution μ on ${\left[q\right]}^m$ such that for any $S\subseteq \left[m\right]$,

$$\begin{array}{c}\hfill r\left(S\right)=H\left({\mu }_S\right),\end{array}$$

(6)

where ${\mu }_S$ is the marginal of μ on S and H is the Shannon entropy in base q.

Note that the entropy does not depend on the support of the random variables but only on their joint distribution. For this reason, the restriction that $\mu$ is taking values in ${\left[q\right]}^m$ is in fact equivalent to requiring that each random variable has a support of cardinality at most q. When working with the m underlying random variables $X_1,\cdots ,X_m$ distributed according to $\mu$, we write $H\left(S\right)=H\left(X\left[S\right]\right)=H(X_i:i\in S)=H\left({\mu }_S\right)$.

With the integrality constraint, the random variables representing a q-entropic matroid must be marginally either uniformly distributed or deterministic, each pair of random variables must be either independent or a deterministic function of each other, and so on. These represent therefore extremal dependencies. As discussed in Section 8, such distributions (with extremal dependencies) have recently emerged in the context of polarization theory and multi-user polar codes [12], which has motivated in part this paper. In Section 4, we also comment on the connection between secret sharing from cryptography.

It is well-known and easy to check that entropic matroids generalize linear matroids, see, for example, References [7,13]. For completeness we recall the proof, making explicit the dependency on the field size.

Lemma 1.

Let $\mathbb{F}$ be a finite field. If a matroid is $\mathbb{F}$-representable then it is $\left|\mathbb{F}\right|$-entropic.

Proof. 

Let M be an $\mathbb{F}$-representable matroid and A be a matrix in ${\mathbb{F}}^{\left|E\right|\times n}$ whose rows correspond to elements of E so that a subset of rows is linearly independent in ${\mathbb{F}}^n$ if and only if the corresponding subset of E is independent in M. Let $Y_1,\cdots ,Y_n$ be mutually independent and uniformly distributed random variables over $\mathbb{F}$ and let $Y=(Y_1,\cdots ,Y_n)$. Then the vector of random variables $(X_1,\cdots ,X_{\left|E\right|})=A·Y$ satisfies that for any $B\subseteq E$, $H\left(\left\{X_i:i\in B\right\}\right)=\mathrm{rank}\left\{A_i:i\in B\right\}$. Thus the entropy function on $X_1,\cdots ,X_{\left|E\right|}$ recovers the rank function of M and M is $\left|\mathbb{F}\right|$-entropic. □

Our main goal throughout the remainder of this paper is to investigate whether entropic matroids are always representable over fields. As discussed in next section, we will approach this question by checking whether the forbidden minors of representable matroids are entropic or not. This strategy is justified by the fact that for the Shannon entropy, entropic matroids are a minor-closed class, as we will show in Lemma 2.

1.3. Results

We prove that for every p, a matroid is p-entropic if and only if it is secret-sharing with a ground set of size p, which is equivalent to being the matroid of an almost affine code with alphabet size p. Furthermore, we prove that for every p, being p-entropic is closed under taking matroid minors.

We give alternative proofs that for $p=2$ and $p=3$, being p-entropic is equivalent to being ${\mathbb{F}}_p$-representable by examining known forbidden minor characterizations. We also make some partial progress towards proving the same for other primes p. In the final section of the paper, we mention some applications of entropic matroids in coding.

2. Further Related Literature

Matroid representations and forbidden minors were studied in Reference [14] for GF(3), Reference [15,16] for GF(4) and some results for general fields were obtained in References [17,18,19]. Linear representable matroids are also intimately related to linear solutions to network coding problems, in particular in Reference [20], in which a network-constrained matroid enumeration algorithm is developed, as well as Reference [21] that considers integer-valued polymatroids and representable polymatroids in References [22,23]. Matroid’s minors and the connection to Zhang-Yeung inequality was discussed in Reference [24], which shows in particular that almost entropic matroids have infinitely many excluded minor. Matroids, secret sharing and linearity are also discussed in several papers as mentioned in part earlier. Reference [25] gave the first example of an access structure (i.e., the parties that can recover the secret from their share) induced by a matroid, namely the Vamos matroid, that is non-ideal (a measure of optimality of the secret shares lengths); Reference [26] presented the first non-trivial lower bounds on the size of the domain of the shares for secret-sharing schemes realizing an access structure induced by the Vamos matroid and this is later improved in Reference [27] using using non-Shannon inequalities for the entropy function. As mentioned earlier, an important line of work is also dedicated to understanding the representation of entropic polymatroids for a fixed ground set cardinality [9], which is well-understood for cardinality 2 and 3 and more complicated for larger cardinality with the non-Shannon inequalities emerging.

3. Minors of Entropic Matroids

In this section, we prove the following:

Lemma 2.

Let M be an entropic matroid on random variables $X_1,\cdots ,X_m$ with values in ${\mathbb{F}}_p$ and with entropy H and joint distribution μ.

(i) 

For any $A\subseteq \left\{X_1,\cdots ,X_m\right\}$, $M|A$ is entropic.

(ii) 

For any $X_i\in \left\{X_1,\cdots ,X_m\right\}$ with $H\left(X_i\right)=1$, $M/\left\{X_i\right\}$ is entropic.

(iii) 

For any independent set A, $M/A$ is entropic.

Proof. 

For each of the claims, we construct random variables and a probability distribution whose entropy agrees with the rank function of the matroid in question.

To prove (i), we consider the variable set A with the marginal distribution given by $\mu$. Then H is integral on any subset of A, since it is integral on any subset of $\left\{X_1,\cdots ,X_m\right\}$. This implies (i).

To prove (ii), we consider two cases. If for any $B\subseteq \left\{X_1,\cdots ,X_m\right\}$ with $X_i\notin B$ we have $H(X_i,B)=H\left(B\right)+1$, then $X_i$ is independent of all other variables. In particular, any set is independent in M if any only if its union with $\left\{X_i\right\}$ is. Therefore, $M/\left\{X_i\right\}=M|\left\{X_1,\cdots ,X_{i-1},X_{i+1},\cdots ,X_m\right\}$ in this case and the result follows from (i).

Otherwise, we define a distribution on $\left\{X_1,\cdots ,X_{i-1},X_{i+1},\cdots ,X_m\right\}$ by fixing any value x for $X_i$ with $\mathbb{P}\left[X_i=x\right]>0$ and considering the probability distribution obtained by conditioning on the event $\{X_i=x\}$. Now let $A\subseteq \left\{X_1,\cdots ,X_{i-1},X_{i+1},\cdots ,X_m\right\}$. There are two cases. If there is no circuit C with $X_i\in C$ such that A contains $C\setminus \left\{X_i\right\}$ as a subset, then $H\left(A\right)+1=H(A,X_i)=H\left(A\right)+H\left(X_i\right|A)$, therefore $H\left(X_i\right|A)=1$ and so $X_i$ and A are independent. In this case, $H\left(A\right|X_i=x)=H\left(A\right)$, thus H agrees with the rank function of $M/\left\{X_i\right\}$.

If adding $X_i$ to A creates a circuit, then $H(A,X_i)=H\left(A\right)$ and $H\left(A\right|X_i)=H\left(A\right)-1$. Let $X\left(A\right)$ denote the vector with components $X_j,j\in A$ and let $\mathcal{Y}={\mathbb{F}}_p^A$ denote the set of possible values of $X\left(A\right)$.

Suppose first that $H\left(A\right|X_i=k)<H\left(A\right)-1$ for some $k\in \mathbb{F}$. Now let B be a basis in A, that is, $\left|B\right|=H\left(B\right)=H\left(A\right)$. We have that $H\left(A\right|X_i=k)=H\left(B\right|X_i=k)+H\left(A\right|B,X_i=k)$ and $H\left(A\right|B,X_i=k)\le H(A,X_i|B)=H\left(A\right|B)=0$. Therefore, $H\left(B\right|X_i=k)<|B|-1$.

Now let C be the unique circuit in $B\cup \left\{i\right\}$. It follows that $H\left(C\right)=H(C\setminus \left\{X_i\right\})=\left|C\right|-1$ and $H(B\setminus C|C)=H(B)-H(C)=|B\setminus C|$. In particular, the variables in $B\setminus C$ are independent of $X_i$ in the marginal distribution on B and thus

$$H\left(B\right|X_i=k)=H(B\setminus C)+H(C\setminus \left\{X_i\right\}|X_i=k,B\setminus C)=|B\setminus C|+H\left(C\right|X_i=k).$$

This implies that $H\left(C\right|X_i=k)<|B|-|B\setminus C|-1=|C|-2$. But $\mathbb{P}\left[X_i=k|X(C\setminus \left\{X\right\})=c\right]\in \left\{0,1\right\}$ and $\mathbb{P}\left[X(C\setminus \left\{X_i\right\})=c\right]=p^{-\left|C\right|+1}$, which implies that $\mathbb{P}\left[X\left(C\right)=c\right]\in \left\{0,p^{-\left|C\right|+1}\right\}$ and $\mathbb{P}\left[X(C\setminus \left\{X_i\right\})=c|X_i=k\right]\in \left\{0,p^{-\left|C\right|+2}\right\}$. Since these probabilities add up to one, it follows that exactly $p^{\left|C\right|-2}$ of them are non-zero, which yields

$$\begin{array}{cc}\hfill H\left(C\right|X_i=k)& =\sum _c\mathbb{P}\left[X(C\setminus \left\{X_i\right\})=c|X_i=k\right]{log}_p\left(\frac{1}{\mathbb{P}\left[X(C\setminus \left\{X_i\right\})=c|X_i=k\right]}\right).\hfill \\ \hfill & =p^{\left|C\right|-2}\left(p^{-\left|C\right|+2}{log}_p\left(\frac{1}{p^{-\left|C\right|+2}}\right)\right)\hfill \\ \hfill & =\left|C\right|-2,\hfill \end{array}$$

a contradiction to the assumption $H\left(C\right)<\left|C\right|-2$.

This implies that $H\left(A\right|X_i=k)\ge H\left(A\right)-1$ for all A. Since

$$\begin{array}{cc}\hfill H\left(A\right)-1& =H\left(A\right|X_i)=\sum _{k=0}^{p-1}\mathbb{P}\left[X_i=k\right]H\left(A\right|X_i=k)\hfill \\ \hfill & =\sum _{k=0}^{p-1}\frac{1}{p}H\left(A\right|X_i=k)\ge p·\frac{1}{p}(H\left(A\right)-1)=H\left(A\right)-1,\hfill \end{array}$$

it follows that we have $H\left(A\right|X_i=k)=H\left(A\right)-1$ for all summands. This implies that the entropy of the conditional distribution yields the entropic matroid $M/\left\{X_i\right\}$ and this proves (ii).

Finally, (iii) follows by applying (ii) repeatedly. □

This lemma proves that the property of being an entropic matroid is closed under taking minors. This means that in order to show entropic matroids belong to a minor-closed class of matroids, it suffices to show that the forbidden minors of this class are not entropic.

4. Secret-Sharing and Almost Affine Matroids

Secret-sharing matroids were introduced in Reference [28]. These matroids are motivated by the problem of secret-sharing in cryptography [29,30], which refers to distributing a secret among a collection of parties via secret shares such that the secret can be reconstructed by combining a sufficient number (of possibly different types) of secrete shares, while individual shares being of no use on their own.

We use the following definitions from Reference [25]: Let $A\in S^{I\times E}$ be a matrix, where $S,I$ and E are finite sets. For $i\in I$, $e\in E$ and $Y\subseteq E\setminus \left\{e\right\}$, we define $n(i,e,Y)=\left\{a_{je}:j\in I,a_{jy}=a_{iy}forally\in Y\right\}$. Then A is a secret-sharing matrix if for $e\in E$ and $Y\subseteq E\setminus \left\{e\right\}$, either $n(i,e,Y)=S$ for all $i\in I$ or $\left|n\right(i,e,Y\left)\right|=1$ for all $i\in I$. Any secret-sharing matrix induces a secret-sharing matroid with ground set E and rank function $r\left(Y\right)$ the logarithm with base $\left|S\right|$ of the number of distinct rows of the submatrix $A\left[Y\right]=(a_{ij}:i\in I,j\in Y)$ of A. In particular, Y is independent if and only if $A\left[Y\right]$ contains all vectors in $S^Y$.

The interpretation is as follows. Suppose some row $i\in I$ has been chosen in A but its value has been kept secret. Knowing A, one wishes to determine as much as possible about the values $a_{ie},e\in E$, without knowing which row has been selected. If by some means one has been able to determine the values $a_{if}$ for all $f\in Y\subseteq E$. Then the possible values of $a_{ie}$ for some $e\in E\setminus Y$, consistent with the available information, are precisely the members of $n(i,e,Y)$ (and this set can be determined despite not knowing i).

Secret-sharing matroids were connected to entropy rank functions in Reference [31], as further discussed below. We now formally connect the two classes of matroids.

Lemma 3.

If a matroid is p-entropic, then it is a secret-sharing matroid with a ground set of size p.

Proof. 

Given a p-entropic matroid M with ground set E and rank (entropy) function H, we let A be the matrix containing all vectors in ${\mathbb{Z}}_p^E$ which correspond to outcomes of positive probability in M. For every set Y of variables, $A\left[Y\right]$ contains the possible outcomes of these variables. These outcomes are all equally likely and the number of distinct outcomes with positive probability is $p^{H\left(Y\right)}$. This implies that to prove that M is a secret-sharing matroid, it suffices to prove that A is a secret-sharing matrix.

Let $e\in E$ and $Y\subseteq E\setminus \left\{e\right\}$. Then $n(i,e,Y)$ is the number of possible values of the random variable $X_e\in E$ associated with e when Y is fixed to its values in outcome i. But $H\left(X_e\right|Y)\in \left\{0,1\right\}$ and if $H\left(X_e\right|Y)=0$ then $X_e$ is determined by the values of Y and $\left|n\right(i,e,Y\left)\right|=1$ for all i; if $H\left(X_e\right|Y)=1$ then $X_e$ is independent of the values of the variables in Y and thus $n(i,e,Y)={\mathbb{Z}}_p$. This proves that A is a secret-sharing matrix. □

Note that this proof remains true for any $p\in {\mathbb{N}}_{\ge 1}$, that is, it does not require the ground set to be a field. The converse of Lemma 3 is true as well: every secret-sharing matroid is p-entropic for some p. This was observed in Reference [31] and we include a proof for completeness. Together, this observation and Lemma 3 provide an alternative characterization of entropic matroids as secret-sharing matroids.

Lemma 4.

Every secret-sharing matroid with ground set S is $\left|S\right|$-entropic.

Proof. 

Let M be a secret-sharing matroid and A a secret-sharing matrix inducing M. Without loss of generality, we may assume that A does not contain two identical rows, since this does not affect the structure of the matroid. The definition of secret-sharing matroids implies that the number of rows of A is a power ${\left|S\right|}^r$ of $\left|S\right|$. We define a probability distribution on the set of random variables $\left\{X_e:e\in E\right\}$ by setting the probability that ${\left(X_e\right)}_{e\in E}=a$ as ${\left|S\right|}^{-r}$ for every row a of A.

We proceed by induction on $|E\setminus Y|$ to show that $H\left(Y\right)$ (with the Shannon entropy with base $\left|S\right|$) is integral for every $Y\subseteq E$ and moreover, that the resulting probability distribution on Y is the uniform distribution on the distinct rows of $A\left[Y\right]$. This is clearly true for $Y=E$, since $H\left(E\right)=r$. Let $Y\subset E$ and let $e\in E\setminus Y$, then by the induction hypothesis, $H(Y\cup \left\{e\right\})=k\in \mathbb{N}$. The matrix $A[Y\cup \left\{e\right\}]$ has ${\left|S\right|}^k$ distinct rows and each distinct row has the same probability ${\left|S\right|}^{-k}$. If $H\left(X_e\right|Y)=0$, then $H\left(Y\right)=k$ and distinct rows in $A[Y\cup \left\{e\right\}]$ are distinct rows of $A\left[Y\right]$ and thus the distribution of the variables in Y is the same as for the variables of $Y\cup \left\{e\right\}$. Therefore, we may assume that fixing the values of the variables in Y does not always determine $X_e$. This means that $n(i,e,Y)=\left|S\right|$ for all i. In particular, every distinct row of $A\left[Y\right]$ gives rise to $\left|S\right|$ distinct rows in $A[Y\cup \left\{e\right\}]$ and thus $A\left[Y\right]$ has ${\left|S\right|}^{k-1}$ distinct rows. Each distinct row has the same multiplicity ${\left|S\right|}^{r-k}$ in $A[Y\cup \left\{e\right\}]$ by the induction hypothesis and thus each distinct row of $A\left[Y\right]$ has multiplicity ${\left|S\right|}^{r-k+1}$. Now the resulting distribution of the variables in Y is a uniform distribution with ${\left|S\right|}^{k-1}$ distinct outcomes, therefore $H\left(Y\right)=k-1$. Clearly, $r_M\left(Y\right)=k-1$ and therefore this induction allows us to conclude that the rank in M coincides with the entropy of the constructed distribution. This implies the result. □

Seymour [25] proved that the Vamos matroid is not a secret sharing matroid. This implies that it is not an entropic matroid for any p.

Moreover, there is a secret-sharing matroid which is not representable over the corresponding field (with $\left|S\right|$ elements) and which has been discovered by Simonis and Ashikhmin [32]. This example is the non-Pappus matroid, shown in Figure 1. This matroid has nine elements $\left\{1,\cdots ,9\right\}$ as its ground set E and each $X\subseteq E$ has rank $min\left(\right|X|,3)$ with the exception of the eight 3-elements sets shown as colored lines, which each have rank 2. Pappus’ theorem proves that this matroid is not representable over any field.

Simonis and Ashikhmin [32] show that the row space of the matrix

$$\left[\begin{array}{ccccccccc}10& 10& 00& 10& 00& 10& 10& 10& 00\\ 01& 01& 00& 01& 00& 01& 01& 01& 00\\ 00& 00& 00& 10& 10& 21& 01& 10& 10\\ 00& 00& 00& 02& 01& 20& 12& 02& 01\\ 00& 10& 10& 01& 00& 01& 00& 11& 10\\ 00& 01& 01& 21& 00& 21& 00& 10& 01\end{array}\right]$$

is a secret-sharing matrix, where each entry of the matrix is considered as an element of ${\mathbb{F}}_3^2$. They introduce another definition of entropic matroids via codes: a code (subset) $\mathcal{C}\subseteq S^E$ is almost affine if $r\left(Y\right):={log}_{\left|S\right|}\left(\right|{\mathcal{C}}_Y\left|\right)\in {\mathbb{N}}_0$ for all $Y\subseteq E$, where ${\mathcal{C}}_Y$ denotes the projection of $\mathcal{C}$ to the variables in Y. The corresponding matroid M with ground set E and rank function r is called an almost affine matroid. It is not hard to see that this definition coincides with secret-sharing matroids by using the codewords in C as the rows of the secret-sharing matrix A and vice versa. These results show that not all entropic matroids are representable by giving a 9-entropic matroid which is not representable over any field.

5. The Case $\mathbf{p}=\mathbf{2}$

An ${\mathbb{F}}_2$-representable matroid is called binary. The goal of this section is to prove the following.

Theorem 2.

Every 2-entropic matroid is binary.

To prove this, we use the characterization of binary matroids proved by Tutte [33] stating that a matroid is binary if and only if it has no $U_{2,4}$-minor. $U_{2,4}$ is the uniform matroid of rank two on four elements: $E=\left[4\right]$ and $\mathcal{F}$ consists of all subsets of E of cardinality at most two. Using Tutte’s characterization, the theorem follows from the next lemma.

Lemma 5.

$U_{2,4}$ is not 2-entropic.

Proof. 

Suppose for a contradiction that $\mu$ is a probability distribution on four random variables $X_1,\cdots ,X_4$ whose entropy is the rank function of $U_{2,4}$, then $H\left(X_i\right)=1$ for all i and $H(X_i,X_j)=2$ for all $i\ne j$; furthermore $H(X_1,X_2,X_3,X_4)=2$. This implies that $\mathbb{P}\left[X_i=a,X_j=b\right]=\frac{1}{4}$ for all $i\ne j$ and $a,b\in {\mathbb{F}}_2$, because the marginal distribution of $X_i$ and $X_j$ has to be the product of two independent $Ber\left(\frac{1}{2}\right)$ distributions to achieve an entropy of two.

Furthermore, $H(X_i,X_j|X_k,X_l)=0$ for $\left\{i,j,k,l\right\}=\left[4\right]$ by the chain rule and therefore $\mathbb{P}\left[X_1=a,X_2=b,X_3=c,X_4=d\right]\in \left\{0,\frac{1}{4}\right\}$ for all $a,b,c,d$. Without loss of generality, we may assume that $\mathbb{P}\left[X_1=0,X_2=0,X_3=0,X_4=0\right]=\frac{1}{4}$ but then every other event in which at least two different variables $X_i$ and $X_j$ are zero must have probability zero, since $\mathbb{P}\left[X_i=0,X_j=0\right]=\frac{1}{4}$. Since $\mathbb{P}\left[X_i=0,X_j=1\right]=\frac{1}{4}$, it follows that all outcomes with three ones have probability $\frac{1}{4}$. Now $\frac{1}{4}=\mathbb{P}\left[X_1=1,X_2=1\right]\ge \mathbb{P}\left[X_1=1,X_2=1,X_3=0,X_4=1\right]+\mathbb{P}\left[X_1=1,X_2=1,X_3=1,X_4=0\right]=\frac{1}{2}$, a contradiction. □

6. The Case $\mathbf{p}=\mathbf{3}$

An ${\mathbb{F}}_3$-representable matroid is called ternary. The following structure theorem has been proved independently by Seymour [34] and Bixby [35], who attributed it to Reid.

Theorem 3

(Seymour [34], Bixby [35]). A matroid is ternary if and only if it contains no minor isomorphic to $U_{2,5}$, $U_{3,5}$, the Fano plane $F_7$ or its dual.

The Fano plane, shown in Figure 2, has a ground set $E=\left[7\right]$ and can be represented over ${\mathbb{F}}_2$ by the column vectors of the matrix $\left[\begin{array}{ccccccc}1& 1& 0& 0& 0& 1& 1\\ 1& 0& 1& 0& 1& 0& 1\\ 1& 0& 0& 1& 1& 1& 0\end{array}\right]$, that is, a set is independent if and only if it contains at most three vectors and it does not contain all three vectors on any line (including the circle).

Lemma 6.

$U_{2,5}$ is not 3-entropic.

Proof. 

Suppose for a contradiction that there exist $X=(X_1,\cdots ,X_5)$ such that $H\left(A\right)=min\left\{\left|A\right|,2\right\}$ for all $A\subseteq \left\{X_1,\cdots ,X_5\right\}$. Then, for any choice of $\left\{a,b,c,d,e\right\}=\left\{1,2,3,4,5\right\}$, we have that $H(X_a,X_b,X_c|X_d,X_e)=0$ and thus for any vector $x\in {\mathbb{F}}_3^5$,

$$\mathbb{P}\left[X_a=x_a,X_b=x_b,X_c=x_c|X_d=x_d,X_e=x_e\right]\in \left\{0,1\right\}$$

and $\mathbb{P}\left[X=x\right]\in \left\{0,\frac{1}{9}\right\}$.

As in the proof for $U_{2,4}$, we may assume that $\mathbb{P}\left[X=0\right]=\frac{1}{9}$ but then any other event with at least two zeros must have probability 0. This leaves six events, five with one zero and one with no zeros; but each of them has probability at most $\frac{1}{9}$, thus the total probabilities add up to at most $\frac{7}{9}$, a contradiction. □

Lemma 7.

$U_{3,5}$ is not 3-entropic.

Proof. 

As before, we suppose for a contradiction that there is a vector $X=(X_1,\cdots ,X_5)$ of random variables such that $H\left(A\right)=min\left\{\left|A\right|,3\right\}$ for all $A\subseteq \left\{X_1,\cdots ,X_5\right\}$.

Every three distinct variables are independent and they determine the other two variables. It follows that, for every event, its probability is either zero or $\frac{1}{27}$. But there are only 81 outcomes and 27 of them occur with positive probability. Each of those 27 must differ from the others in at least three places, because if two outcomes are equal in three positions, the other two are determined and thus equal. This means that the Hamming balls of radius 1 around the outcomes with positive probability are disjoint. Each of these Hamming balls contains 11 elements: the outcome with positive probability and the outcomes in which one variable is flipped to one of the two other possible values. Therefore, we have at least $27\times 11=297$ outcomes, a contradiction. □

Lemma 8.

The Fano plane is not 3-entropic.

Proof. 

Suppose for a contradiction that the Fano plane is 3-entropic and that $X=\left\{X_1,\cdots ,X_7\right\}$ is a set of random variables whose entropy corresponds to their rank in the Fano matroid as shown in Figure 2. Since the maximum size of an independent set in the Fano matroid is three, any three independent variables determine the values of all the others; in particular, there are at most 27 outcomes with positive probability, which we denote by their values on the independent set $X_1,X_2,X_3$. Since $H(X_1,X_2,X_3)=3$, each of these outcomes has probability $\frac{1}{27}$, whereas all other outcomes have probability zero. It follows that we have a map $f:{\mathbb{F}}_3^3\to {\mathbb{F}}_3^4$ mapping the values on $X_1,X_2,X_3$ to the values on $X_4,\cdots ,X_7$, where $X_2$ and $X_3$ determine $X_7$, $X_1$ and $X_2$ determine $X_5$ and $X_3$ and $X_1$ determine $X_6$ but every change of one of $X_1,X_2,X_3$ must change $X_4$.

We consider the set of nine assignments of $X_1,X_2,X_3$ for which $X_4=0$. If every two of these have pairwise distance at least three, we can only have three distinct assignments. This implies that we may assume that there are two assignments with distance two. Furthermore, if we fix any two digits, exactly one choice is valid for the remaining digit. Therefore, up to isomorphism (exchanging symbols), the set looks as follows: $\left\{000,012,021,102,111,120,201,210,222\right\}$; and thus $X_4=X_1+X_2+X_3$.

The random variables $X_2,X_3,X_4$ determine $X_5,X_6,X_7$ and $X_1$. In particular, both of the pairs $X_1$, $X_1+X_2+X_3$ and $X_2$, $X_3$ determine $X_7$.

Changing $X_1$ does not change $X_7$ and neither does simultaneously adding k to $X_2$ and subtracting k from $X_3$. Therefore, keeping $X_2+X_3$ constant will keep $X_7$ constant and $H\left(X_7\right|X_2+X_3)=0$, and $H(X_2+X_3|X_7)=0$. This implies that there is a one-to-one correspondence between $X_7$ and $X_2+X_3$ and similarly between $X_6$ and $X_2+X_4$ and between $X_5$ and $X_3+X_4$. But then $X_5,X_6,X_7$ allow us to find $X_2+X_3$, $X_2+X_4$ and $X_3+X_4$ and thus $2X_2+2X_3+2X_4$ and $X_2+X_3+X_4$ (since $2\ne 0$ in ${\mathbb{F}}_3$), which is $X_1$. This shows that $H\left(X_1\right|X_5,X_6,X_7)=0$ and thus $3=H(X_1,X_5,X_6,X_7)=H(X_5,X_6,X_7)=2$, a contradiction. □

The above proof actually shows that the Fano plane is not p-entropic for any $p>2$, which gives an alternative proof that it is not ${\mathbb{F}}_p$-representable for $p>2$ either.

The dual $F_7^{*}$ of the Fano plane is ${\mathbb{F}}_2$-representable and a representation is given by the columns of the matrix $\left[\begin{array}{ccccccc}1& 1& 1& 1& 1& 1& 1\\ 1& 1& 0& 0& 0& 1& 1\\ 1& 0& 1& 0& 1& 0& 1\\ 1& 0& 0& 1& 1& 1& 0\end{array}\right].$ This shows that every 3-element set is independent in $F_7^{*}$, thus its circuits are exactly the complements of the three-element circuits of the Fano plane. To give a better understanding of these matroids, we expanded the symmetrical representation of $F_7$ given in Reference [36] and shown in Figure 3a to $F_7^{*}$. The result is shown in Figure 3b. Each color connects the elements of a circuit in one figure and the corresponding circuit given by its complement in the other figure. The cyclical order of the nodes in Figure 3a yields a rainbow Hamilton cycle (one edge of each color) in Figure 2.

Lemma 9.

The dual of the Fano plane is not 3-entropic.

Proof. 

Suppose for a contradiction that $X=(X_1,\cdots ,X_7)$ is a vector of random variables whose entropy coincides with the rank function of $F_7^{*}$. Since $H(X_2,X_3,X_4,X_5)=4$ and $H\left(X\right)=4$, $\mathbb{P}\left[X=x\right]\in \left\{0,\frac{1}{81}\right\}$ for all $x\in {\mathbb{F}}_3^7$. We refer to the events with positive probability as outcomes.

By permuting the symbols, we may assume that 0000000 is a possible outcome. We consider the other outcomes $(X_1,X_6,X_7)$ for $X_2=0$. No two of these outcomes can have distance one, because $X_1,X_2,X_6,X_7$ is a cycle, so for fixed $X_2$, any two distinct possible outcomes must have distance at least two on their restriction to $(X_1,X_6,X_7)$. In the proof of the previous lemma, we have already to shown that by switching digits, we may assume that the set of images is $\left\{000,012,021,102,111,120,201,210,222\right\}$. As shown in Figure 4, this also determines the other two sets (but not necessarily which of them is which). This shows that $X_1+X_6+X_7$ is sufficient to determine $X_2$ and vice versa; by flipping symbols 1 and 2 for $X_2$, we may assume that $X_1+X_6+X_7=X_2$.

We now fix $X_3$. Then $X_4$ is determined by either $X_1,X_2=X_1+X_6+X_7$ or $X_6,X_7$ and thus changing $X_1$ or adding k to $X_6$ and subtracting it from $X_7$ does not change $X_4$. This implies that $X_4$ depends only on $X_6+X_7$ (and $X_3$) and thus $H(X_3,X_4,X_6+X_7)=2$. Analogously, $H(X_3,X_5,X_1+X_7)=2$ and $H(X_4,X_5,X_1+X_6)=2$. Therefore, $X_3$, $X_4$ and $X_5$ determine $X_6+X_7+X_1+X_7+X_1+X_6=2(X_1+X_6+X_7)=2X_2$ and since $2\ne 0$ in ${\mathbb{F}}_3$, this shows that $H(X_2,X_3,X_4,X_5)=3$, contradicting the assumption that X had the entropy function given by the rank in $F_7^{*}$. □

Combining these four lemmas with the characterization of ternary matroids, we have proved the following theorem (the interesting part being the only if part).

Theorem 4.

A matroid is 3-entropic if and only if it is ${\mathbb{F}}_3$-representable.

7. Comments for General Primes $\mathbf{p}$

For ground sets of arbitrary size p, being p-representable is a stronger assumption than being p-entropic as the example of Simonis and Ashikhmin [32] of the non-Pappus matroid (see Figure 1) shows. However, no counterexamples exist in the case where the ground set has prime order.

In this section, we show that for primes p, every p-entropic matroid of rank at most two is linear, that is, let M be an entropic matroid with ground set E and $H\left(E\right)\le 2$, then M is linear. If $H\left(E\right)<2$, this is true since any basis has at most one element. Furthermore, we may assume that every $X\in E$ satisfies $H\left(X\right)=1$, for otherwise X is deterministic and is represented by the zero vector in every linear representation.

Lemma 10.

Let M be a p-entropic matroid of rank 2. If there are two elements X and Y in the ground set E with $H(X,Y)=1$, then M is ${\mathbb{F}}_p$-linear if and only if $M\setminus \left\{X\right\}$ is.

Proof. 

If M is ${\mathbb{F}}_p$-representable, then so is $M\setminus \left\{X\right\}$, since it is a minor-closed property. Suppose that $M\setminus \left\{X\right\}$ is representable and let $f:E\setminus \left\{X\right\}\to V$ be a representation and let $g:E\to V$ be defined as $f\left(Z\right)$ for $Z\ne X$ and $f\left(X\right)=f\left(Y\right)$. Let $S\subseteq E$. Then $dim\left(span\right(g\left(S\right)\left)\right)=H\left(S\right)$ for $X\notin E$. If $X\in S$ but $Y\notin S$, then $dim\left(span\left(g\left(S\right)\right)\right)=dim\left(span\left(f(S\cup \left\{Y\right\})\right)\right)=H(S\cup \left\{Y\right\})$ and

$$\begin{array}{cc}\hfill H(S\cup \left\{Y\right\})& =H\left(S\right)+H\left(Y\right|S)\hfill \\ \hfill & =H\left(S\right)+H\left(X\right|S)+H(X,Y\left|S\right)-H\left(X\right|S)+H(Y\left|S\right)-H(X,Y|S)\hfill \\ \hfill & =H\left(S\right)+H\left(X\right|S)+H\left(Y\right|X,S)-H\left(X\right|Y,S)=H(S\cup \left\{X\right\}).\hfill \end{array}$$

If $X,Y\in S$, then $dim\left(span\left(g\left(S\right)\right)\right)=dim\left(span\left(f(S\setminus \left\{X\right\})\right)\right)=H(S\setminus \left\{X\right\})=H\left(S\right)$ by applying submodularity to the sets $\left\{X,Y\right\}$ and $S\setminus \left\{X\right\}$. This proves that g is an ${\mathbb{F}}_p$-representation of M. □

With the above lemma, we have reduced the problem to considering uniform matroids. For any prime p, the uniform matroid $U_{2,p+1}$ is ${\mathbb{F}}_p$-representable by choosing the images of E as

$$(0,1),(1,0),(1,1),(1,2),\cdots ,(1,p-1)\in {\mathbb{F}}_p^2.$$

Each pair of these $p+1$ vectors is independent and a basis of ${\mathbb{F}}_p^2$, thus they represent $U_{2,p+1}$. The following lemma shows that any larger uniform matroid is neither p-entropic nor ${\mathbb{F}}_p^2$-representable.

Lemma 11.

The uniform matroid $U_{2,p+2}$ is not p-entropic for any $p\in {\mathbb{N}}_{\ge 2}$.

Proof. 

Suppose not and let C denote the set of possible outcomes for a probability distribution on $p+2$ variables representing $U_{2,p+2}$. By changing symbols, we may assume that $(0,\cdots ,0)$ is a possible outcome. Furthermore, there are $p^2$ outcomes and hence p of them begin with a zero. These p outcomes have the same value at the first coordinate $X_1$ but all other values are distinct (i.e., each $X_i$ for $i>1$ takes all of its p possible values exactly once among these p outcomes, including value zero for outcome $(0,\cdots ,0)$). Therefore, we can simultaneously change the other symbols so that these p outcomes become $(0,0,\cdots ,0),(0,1,\cdots ,1),(0,2,\cdots ,2),\cdots ,(0,p-1,\cdots ,p-1)$. But then any other outcome not starting with zero satisfies that $X_2,\cdots ,X_{p+2}$ all take different values in ${\mathbb{Z}}_p$. Since there are only p values but $p+1$ variables, this is a contradiction. □

This shows that line matroids, which are among the forbidden minors of binary and ternary matroids, are p-entropic if and only if they are ${\mathbb{F}}_p$-linear.

8. Application: Entropic Matroids in Coding

We recall here a result proved in Reference [12] that makes entropic matroids emerge in a probabilistic context and which gives further motivations to studying entropic matroids. The result gives in particular a rate-optimal code for compressing correlated sources, similarly to the channel counter-part developed in Reference [37].

Let $X^n=(X_1,\cdots ,X_n)$ be an i.i.d. sequence of discrete random variables taking values in ${\mathcal{X}}^m$. That is, $X^n$ is an $m\times n$ random matrix with i.i.d. columns of distribution $\mu$ on ${\mathcal{X}}^m$. One can assume that the support of $\mathcal{X}$ is finite (countable supports can be handled with truncation arguments) and to further simplify, we assume that $\mathcal{X}$ is binary, associating each element in the binary field, that is, $\mathcal{X}=GF\left(2\right)$.

Due to the i.i.d. nature of the sequence, the entropy of $X^n$ is the sum of each components’ entropies $H\left(\mu \right)$, i.e.,

$$\begin{array}{c}\hfill H\left(X^n\right)=nH\left(\mu \right).\end{array}$$

(7)

The next result shows that it is possible to transform the sequence $X^n$ with an invertible map that extracts the entropy in subsets of the components. In words, the transformation takes the i.i.d. vectors under an arbitrary $\mu$ to a sequence of distributions that correspond in the limit to entropic matroids.

Theorem 5

(Abbe [12]). Let m be a positive integer, n be a power of 2 and $X^n$ be an $m\times n$ random matrix with i.i.d. columns of distribution μ on ${\mathbb{F}}_2^m$. Let $Y^n=X^n{G}_n$ over ${\mathbb{F}}_2$, where $G_n={\left[\begin{array}{cc}1& 0\\ 1& 1\end{array}\right]}^{\otimes {log}_2\left(n\right)}$. For any $\epsilon =O\left(2^{-n^{\beta }}\right)$, $\beta <1/2$, we have

$$\begin{array}{c}\hfill \left|\right\{i\in \left[n\right]:H\left(Y_i\left[S\right]\right|Y^{i-1})\notin \mathbb{Z}\pm \epsilon ,\mathrm{for}\mathrm{any}S\subseteq \left[m\right]\left\}\right|=o\left(n\right).\end{array}$$

(8)

In other words, one starts with an i.i.d. sequence of random vectors under a distribution $\mu$ that defines an entropic polymatroid$\left[m\right]\supseteq S\mapsto H\left(S\right)$ and after the transformation $G_n$, one obtains a sequence of random vectors which is no longer i.i.d. but where each random vector given the past defines an entropic matroid in the limit. Having a matroid structure is of course much easier to handle for compression purposes, one simply has to pick a basis for each matroid, store the components in that basis and the other components are fully dependent on these so they can be recovered without being stored. Of course, in practice n is large but finite, and each random vector defines a polymatroid that is close to a matroid but a continuity argument allows to show that the components outside of the bases can still be recovered but only with high probability. Since a compression code is allowed to fail with a low probability of error, this is not an issue. Understanding the structure of these entropic matroids allows then one to better understand how the stored components can be allocated over the different components—see Reference [12] for further details.