Matroidal Entropy Functions: A Quartet of Theories of Information, Matroid, Design, and Coding

In this paper, we study the entropy functions on extreme rays of the polymatroidal region which contain a matroid, i.e., matroidal entropy functions. We introduce variable strength orthogonal arrays indexed by a connected matroid M and positive integer v which can be regarded as expanding the classic combinatorial structure orthogonal arrays. It is interesting that they are equivalent to the partition-representations of the matroid M with degree v and the (M,v) almost affine codes. Thus, a synergy among four fields, i.e., information theory, matroid theory, combinatorial design, and coding theory is developed, which may lead to potential applications in information problems such as network coding and secret-sharing. Leveraging the construction of variable strength orthogonal arrays, we characterize all matroidal entropy functions of order n≤5 with the exception of log10·U2,5 and logv·U3,5 for some v.


Introduction
Given N := {1, 2, · · · , n} and discrete random vector X := (X i : i ∈ N), the set function h X : 2 N → R defined by h X (A) = H(X A ), ∀A ⊆ N is called the entropy function of X, where X A := (X i : i ∈ A) and H(X ∅ ) = 0 by convention. We also say X characterizes h X , or X is the characterizing random vector of h X . An entropy function h can be considered as a vector in the entropy space H n := R 2 N . (For a set A and a finite set B, A B denotes the |B| Cartesian product of A with each coordinate indexed by b ∈ B. When A = F is a field, F B is a |B|-dimensional vector space over F with each coordinate indexed by b ∈ B.) We say H n and the vectors in it have order n. The set of all entropy functions of order n, denoted by Γ * n , is called the entropy region of order n. The closure of Γ * n , denoted by Γ * n , is called almost entropic region. It is a convex cone [1]. A vector h ∈ H n is called entropic if h ∈ Γ * n , almost entropic if h ∈ Γ * n , and non-entropic if h ∈ Γ * n . Characterization of entropy functions, i.e., for a vector h ∈ H n , determining whether it is in Γ * n or Γ * n , is of fundamental importance in information theory.  (N, h) is called a matroid. By frequent abuse of terminology, we do not distinguish a (poly)matroid and its rank function if there is no ambiguity. See Section 2.1 for a more detailed discussion on matroids.
The set of all polymatroids in H n , denoted by Γ n , is called the polymatroidal region of order n. It is proved in [2] that any entropy function is a polymatroid, thus Γ n is an outer bound of Γ * n . As Γ n is closed, it is also an outer bound of Γ * n . As inequalities bounding Γ n are equivalent to the nonnegativity of Shannon information measures, they are called Shannon-type information inequalities, and Γ n is also called the Shannon outer bound of Γ * n and Γ * n . For more about entropy functions and information inequalities, readers are referred to [3], (Chapter [13][14][15] [4,5]. It is well known that Γ * n Γ n when n ≥ 4 due to the existence of non-Shannon-type inequalities, e.g., Zhang-Yeung inequality [6]. However, though Γ * 3 = Γ 3 , Zhang and Yeung also discovered that on an extreme ray of Γ 3 , only countably many vectors are entropic, which implies that Γ * 3 Γ * 3 , and therefore there exists a gap between Γ * n and Γ * n [1]. Given a random vector X = (X 1 , X 2 , X 3 ) with X i , i = 1, 2, 3 mutually independent and each of them the function of the other two, it is proved in [1] that X i must be uniformly distributed on a finite set, say On the other hand, for each integer v ≥ 1, they proved that polymatroid h with h(A) = log v · min{2, |A|}, A ⊆ {1, 2, 3} is entropic: let X 1 and X 2 uniformly distributed on Z v and X 3 ≡ X 1 + X 2 (mod v), then h is the entropy function of (X 1 , X 2 , X 3 ).
As the rank function of U 2,3 is equal to min{2, |A|}, A ⊆ {1, 2, 3}, Zhang-Yeung indeed proved that for any vector h = c · U 2,3 on the ray R U 2,3 := {c · U 2,3 : c ≥ 0}, h is entropic if and only if c = log v for some positive integer v. In [7], Matúš proved that, for any extreme ray R M := {c · M : c ≥ 0} of Γ n containing a connected matroid M with rank ≥ 2, h = c · M is entropic only if c = log v for some positive integer v. However, on the other hand, h = c · M is not entropic for all positive integers. For example, we will see in Section 4 that h = log v · U 2,4 is non-entropic when v = 2, 6. Definition 1. For a connected matroid M with rank ≥ 2, we call the set χ M of all positive integers v such that h = log v · M is entropic the probabilistically (p-)characteristic set of M.
The term p-characteristic set of a matroid M is first coined in [7]. As discussed above, χ U 2,3 = Z + , the set of all positive integers, and χ U 2,4 = {v ∈ Z + : v = 2, 6}. In this paper, we study the p-characteristic set of an arbitrary connected matroid with rank ≥ 2.

Definition 2.
For a connected matroid M with rank ≥ 2 and a positive integer v, if v ∈ χ M , we call the entropy function h = log v · M a matroidal entropy function induced by M with degree v.
It can be seen in the proof of Zhang-Yeung, characterizing random vectors of matroidal entropy functions on R U 2,3 is constructed by the multiplication table of an additive group on Z v . It is not difficult to see that a random vector constructed by any quasigroup on Z v , or equivalently, a Latin square with symbols in Z v , or equivalently, an orthogonal array OA (2, 3, v), characterizes log v · U 2,3 . (See Section 2.2 for the definition of an orthogonal array.) More generally, an OA(t, n, v) can be used to construct a characterizing random vector of the matroidal entropy function log v · U t,n with t ≥ 2. It is a natural question to ask whether such construction can be generalized to an arbitrary connected matroid M with rank ≥ 2? In [8], partition-representations ξ i , i ∈ N of a matroid M = (N, r) with degree v was defined, where each ξ i is partition of a set Ω with cardinality v h(N) . See more details in Sections 3.1.2. Characterizing random vectors of h = log v · r can be obtained by the uniform distributions on the blocks of ξ i . In [9], an equivalent definition in coding theory called almost affine code was defined. In this paper, in coordinate with the language in combinatorial design, we introduce variable strength orthogonal arrays(VOA) indexed by matroid M and integer v ≥ 2, which is equivalent to a partition-representation of M with degree v and an (M, v) almost affine code. We denoted it by VOA(M, v). A VOA(M, v) can be regarded as expanding the concept of orthogonal array. If a VOA(M, v) exists, we will prove that the matroidal entropy function log v · M is entropic and a characterizing random vector of log v · M can be constructed by VOA(M, v). On the other hand, if VOA(M, v) does not exist, log v · M is non-entropic.
It is well known that orthogonal arrays with index unity in design theory are equivalent to maximum distance saperable (MDS) codes in coding theory. In discussions of our paper, we also see a more generalized equivalence, i.e., the equivalence between a VOA and an almost affine code. Thus, we review and develop the correspondences and equivalences in literatures such as [8,9] among four fields, i.e., information theory, matroid theory, combinatorial design, and coding theory, which may help them benefit from each other. In this paper, VOAs are also leveraged to characterize matroidal entropy functions induced by matroids of order n ≤ 5.
The rest of this paper is organized as follows. Section 2 gives the preliminaries on matroid theory and orthogonal arrays. In Section 3, we first define variable strength orthogonal arrays and show their equivalence to the partition representation of a matroid and almost affine codes. Then we characterize matroid entropy functions via variable strength orthogonal arrays. in Section 4 , we characterize matroidal entropy functions of order n ≤ 5. A discussion of the applications and further research is in Section 5.

Matroids
There exist various cryptomorphic definitions of a matroid. In this paper we discuss matroid theory mainly from the perspective of rank functions. For a detailed treatment of matroid theory, readers are referred to [10,11]. In Section 1, we defined matroids as special cases of polymatroids. Here we restate the definition in the following.

Definition 3.
A matroid M is an ordered pair (N, r), where the ground set N is a finite set and the rank function r is a set function on 2 N , and they satisfy the conditions that: for any A, B ⊆ N, With a slight abuse of terminology and notations, we do not distinguish a matroid and its rank function. So M, r M and r may all denote the rank function of M when there is no ambiguity.

Definition 4.
For integer n ≥ 1 and 0 ≤ t ≤ n, the uniform matroid U t,n with rank t and order n is defined by U t,n (A) := min{t, |A|} ∀A ⊆ N.
we call A a parallel class. If |A| ≥ 2, the parallel class is called non-trivial. A matroid is called simple if it contains no loops and no non-trivial classes. For a matroid M, if we delete its loops and in each non-trivial parallel class, we delete all elements but one, then we obtain a simple matroid M . We call M the simplification of M.
For a matroid M = (N, r), a nonempty C ⊆ N is called a circuit with size |C| of M if r(C − x) = r(C) = |C| − 1 for any x ∈ C. It can be seen that any loop of M is a circuit of size 1 and a parallel pair {i, j} is a circuit of size 2. For a uniform matroid U t,n , circuits are exactly those (t + 1)-subsets C of N. In particular, U 0,n contains n loops, any two elements of U 1,n are parallel, and the ground set of U n−1,n forms the unique circuit of U n−1,n .

Definition 5.
A matroid is connected if any two elements in the ground set are contained in a circuit.
It is easy to be verified that any uniform matroid U t,n with 1 ≤ t ≤ n − 1 is connected. This is because any x ∈ N is contained in a t + 1 subset of N which is a circuit of U t,n .
An extreme ray R of a convex cone C is a subset of C and for any r ∈ R such that r = c 1 + c 2 and c 1 , c 2 ∈ C, we have c 1 , c 2 ∈ R, where c 1 = ar and c 2 = (1 − a)r for some a ∈ R. Lemma 1. [12] A loopless matroid is connected if and only if M is contained in an extreme ray of Γ n .
Each extreme ray of Γ n contains an integer-valued polymatroid, some of which are matroids. Such a matroid on an extreme ray is either a loopless connected matroid as stated in the above lemma, or a matroid obtained by adding loops to a connected matroids.

Orthogonal Arrays
Orthogonal array is a well studied topic in design theory. In this paper, orthogonal arrays are leveraged to characterize matroidal entropy functions. For a detailed treatment of this topic, readers are referred to [13].

Definition 6.
A λv t × n array T with entries from Z v is called an orthogonal array of strength t, factor n, level v and index λ if for any λv t × t subarray T of T, each t-tuple in Z t v occurs in the rows of T exactly λ times. We call T an OA(λ × v t ; t, n, v). When λ = 1, we say such orthogonal array has index unity and call it an OA(t, n, v) for short.
By the definition, for any In this paper, we only consider the strength of the orthogonal array largest possible.
An important research problem of orthogonal arrays is the existence of an OA(t, n, v). The following lemmas state some results of this problem, in which Lemmas 2-4 can be found in Handbook [14]. This lemma is a folklore in the combinatorial design community. For self-contain, we prove it in the following.
Proof. We prove the non-existence of OA (3, 5, v) for v = 2, 6 by contradiction. Assume there exists an OA(3, 5, 2) A, i.e., a 2 3 × 5 array whose each 2 3 × 3 subarray contains each 3-tuple in Z 3 2 as a row exactly one time. By permuting the rows of A, we obtain an OA(3, 5, 2) A such that the entries in the first 2 2 rows and the 5-th column of A are all 0. Let c i , 1 ≤ i ≤ 5 be the 5 columns of A and c i be the vector consisting of the first 2 2 entries in c i . Now consider the subarray [c i , c j , c 5 ] with 1 ≤ i < j ≤ 4. As its rows are exactly all 3-tuples in Z 3 2 and c 5 is a zero vetor, it can be seen that the rows of [c i , c j ] are exactly all 2-tuples in Z 2 2 . Thus, [c 1 , c 2 , c 3 , c 4 ] forms an OA(2, 4, 2) which contradicts Lemma 3. The non-existence of OA (3,5,6) can be proved similarly.
For OA (3,5,3), assume such an array B exists. As each 3 3 × 3 subarray of B contains each 3-tuple in Z 3 3 as a row exactly one time, for each two rows of the 3 3 = 27 rows of B, their Hamming distance must be ≥ 3. Therefore, any two Hamming balls with center a row of B and radius 1 are disjoint. As there are 27 such Hamming balls with each size 11, there are at least 27 × 11 = 297 5-tuples, which contradicts the fact that only 3 5 = 243 5-tuples exist.

Lemma 7 ([16])
. Let x be an arbitrary odd positive integer. Let g be an arbitrary positive integer whose prime power factors are all ≥ 7 such that g ≡ 3 (mod 4). Then 1.

Characterizing Matroidal Entropy Functions via Voa
In this section, we introduce variable strength orthogonal arrays and then show that they are equivalent to partition-representations of a matroid and almost affine code. We then characterize matroidal entropy functions via variable strength orthogonal arrays. However, on the other hand, two OA(λv t ; t, n, v)s may be VOAs induced by two distinct matroids as long as they have the same rank and the same size of the smallest circuit. This is because the rank of a matroid provides richer parameters in VOA description than strength and index in OA description. The VOA description provides accurate information on the vary of strength on different set of columns of the array. This is why we term it "variable strength orthogonal array". Example 2. Let M 2 = (N, r 2 ) be a matroid with N = {1, 2, 3, 4, 5} and rank function Then VOA(M 1 , v) and VOA(M 2 , v) are both OA(v 3 ; 2, 5, v).
However, when the matroid is uniform, the two descriptions are equivalent. For a uniform matroid U t,n , as any circuit has size t + 1, a VOA(U t,n , v) has strength t and index λ = 1, i.e., an OA(t, n, v). On the other hand, it can be seen any OA(t, n, v) is a VOA(U t,n , v). So in the following of this paper, we write VOA(U t,n , v) as OA(t, n, v) for simplicity.

Partition-Representation of A Matroid
We will see that a partition-representation of a matroid M with degree v defined in [8] is equivalent to an VOA(M, v).

Definition 8. Let M = (N, r) be a matroid with ground set N and rank function r.
Let v ∈ Z + . The matroid M is partition representable of degree v if there exist a finite set Ω of cardinality v r(N) and partitions ξ i of Ω, i ∈ N, such that for any A ⊆ N, the meet-partition ξ A = ∧ i∈A ξ i has v r(A) blocks all the same cardinality.
Let Ω be the set of all rows of an VOA(M, v). Let ξ i be a partition of Ω such that the rows in each block of ξ i have the same entry in the i-th column. It can be seem that ξ i , i ∈ N is a partition-representation of M with degree v.
On the other hand, let ξ i , i ∈ N, be a partition-representation of a loopless matroid M with degree v, living on Ω. As each ξ i has v blocks, we label them from 0 to v − 1. Now for each x ∈ Ω, it is labelled by an |N|-tuple (x i , i ∈ N) where x i is the label of the block of ξ i to which x belong. Let A be an array whose rows are exactly the labels of all x ∈ Ω. It can be checked that A is an VOA(M, v).

Almost Affine Codes
Almost affine codes were introduced in [9]. For vector space F N q over finite field F q , where q is a prime power, a linear subspace of F N q forms a linear code of length n, while each coset of a linear code are called an affine code. For an affine code C ⊂ F N q and any A ⊆ N, let C A be the projection of C onto F A q , it can be seen that |C A | is a power of q. But there are other codes satisfy this property even if they are not codes over a finite fields.
is an integer for all A ⊆ N.
For any almost affine code C, (N, r) forms a matroid M, where the rank function r is defined in (1). We call such almost affine code an (M, v) (almost affine) code.
For an (M, v) code, if M is a uniform matroid U t,n , it coincides with an (n, t, v) maximum distance separable (MDS) code.
By checking the definition of a VOA(M, v) and an (M, v) almost affine code, it can be seen that rows of a VOA(M, v) are exactly codewords of an (M, v) almost affine code and vice versa. In particular, the rows of a OA(t, n, v) are exactly codewords of an (n, t, v)-MDS code and vice versa.
If there exists an (M, v) almost affine code, M is called almost affinely representable with degree v.

Characterizing Matroidal Entropy Functions via VOA
Given a random vector (X i , N), let p X N (·) denote its joint probability mass function, and for any A ⊆ N, p X A (·) be the marginal distribution function on A. Without loss of generality, we assume each random variable X i is distributed on Z v i and for each Hence h X (A) = log v · r(A). It proves that X characterizes log v · M and thus the "if part" of the theorem.
For the "only if part", let X = (X i : i ∈ N) be a characterizing random vector of log v · M. Take C ⊆ N be a circuit of M of with cardinality n ≥ 3. WLOG, we asume C = {1, 2, · · · , n }. Then for each A C, X i , i ∈ A are mutually independent, and for each i ∈ C, X i is a function of (X j : j ∈ C − i). Then for each A C and In particular, p X C (x 1 , x 2 · · · , x n ) = p X C−1 (x 2 , · · · , x n ) = p X 2 (x 2 )...p X n (x n ) (2) and Equating (2) and (3), we have Let x 1 ∈ X 1 and x 1 = x 1 , with the same argument, we have As x 1 and x 1 are arbitrary chosen from Z v 1 , X 1 is uniformly distributed on it. Since Since M is a connected matroid with r(N) ≥ 2, each element is contained in a circuit with size not less than 3. Hence for all i ∈ N, X i is uniformly distributed on Z v . Thus, in the following X can be considered to distributed on Z N v and for any A ⊆ N, X A is distributed on Z A v . Now let B ⊆ N be a base of M, i.e., r(B) = |B| = r(N). Since h X (B) = log v · r(N), any It implies that there exists at least v r(N) n-tuples x ∈ Z N v with p N (x) > 0 and the marginal distribution of them on B is uniform. As h X (N) = h X (B) = log v · r(N), each x ∈ Z N v with p N (x) > 0 is uniquely determined by their entries indexed by B, and so there are exactly v r(N) n-tuples x ∈ Z N v with p N (x) = v −r(N) and other n-tuples has zero probability. Furthermore, for any A ⊆ N, as h X (A) = log v · r(A), by taking sub-tuples indexed by A of these v r(N) n-tuples, we obtain v r(A) distinct |A|-tuple in Z |A| v , each of which occur exactly v r(N)−r(A) times. Therefore, if we put these n-tuples in an array and each as a row, they form a VOA(M, v).

Corollary 1.
A random vector X = (X i : i ∈ N) characterizes matroidal entropy function log v · U t,n with 2 ≤ t ≤ n − 1 if and only if random variable Y = X is uniformly distributed on the rows of an OA(t, n, v).

P-Characteristic Set of Matroids with Order n ≤ 5
Rank 1 matroids of order n are exactly those matroids containing U 1,n on N ⊆ N as a submatroid and other elements loops. Let X be an arbitrary random variable. Let (X i : i ∈ N) be defined by It can be seen that (X i : i ∈ N) characterizes h on the ray {h ∈ H n : h = c · M} as long as we let H(X) = c.
Armed with the results of orthogonal arrays in Section 2.2 and Theorem 1, we can characterize the matroidal entropy functions log v · M for a connected matroid M with rank ≥ 2. In this section, we determine the p-characteristic set χ M for all connected matroids M = (N, r) with rank r(N) ≥ 2 and order n ≤ 5. For a disconnected matroid M with each connected component M i rank ≥ 2, χ M is the intersections of all χ M i . Thus, it is sufficient to consider connected matroids and take them as building blocks. It matches the fact that matroidal entropy functions indexed by a connected matroid live on an extreme rays of Γ n (see Lemma 1), while those indexed by a disconnected matroid can be written as the sum of the matroidal entropy functions indexed by its connected components.
Among all connected matroids, we only need to consider those simple matroids since the p-characteristic set of a matroid is the same as its simplification. For a matroid M = (N, r) and its simplification M = (N , and let X i be a constant if i is a loop of M. Then (X i : i ∈ N) characterizes log v · M. On the other hand, if (X i : i ∈ N) characterizes log v · M, by the reverse method, we obtain (Y j : j ∈ N ) characterizing M . Hence they have the same p-characteristic set.
Non-isomorphic simple matroids with order ≤ 8 is listed in [17] (A simple matroid is also called a combinatorial geometry.). Here we consider connected simple matroids with rank r(N) ≥ 2 and order n ≤ 5. Before that we first consider U n−1,n for general n ≥ 3. By Lemma 2 and Theorem 1, we have the following proposition.
When n = 3, the case U 2,3 is also proved by Zhang-Yeung [1] as we discussed in Section 1. As U 2,3 is the only case we need to consider for matroids with oder n = 3, in the following we discuss the cases for n = 4 and 5.

n = 4
For n = 4, besides U 3,4 , one more matroid we need to consider is U 2,4 . By Theorem 1 together and Lemma 3, we have the following propositions.
For U 2,5 , by Theorem 1 and Lemma 4, we have the following propositions.
For U 3,5 , by Theorem 1 and Lemmas 5-7, we have the following propositions.
) where x is an arbitrary odd positive integer, and g is an arbitrary positive integer whose prime power factors are all ≥ 7 such that g ≡ 3 (mod 4).
For M 1 , we give a VOA(M 1 , 2) in Example 1, thus 2 ∈ χ M 1 . We will have in the following proposition on the existence of VOA(M 1 , v) for an arbitrary v ≥ 2.
The following proposition determines the p-characteristic set of M 2 Proposition 6.

Applications
Matroidal entropy functions and its characterizations have many potential applications in information theory. In the following we discuss the applications to network coding and secret sharing.

Network Coding
A method of building networks from matroids was given in [18]. In a matroidal network G, messages generated in the source nodes and transmitted on the edges are mapped to the ground set of a matroid M (See Section V.B of [18]). By the same mapping, a VOA(M, v) with v ≥ 2 can be considered as a (1, 1) coding solution with alphabet size v of the network G. This coding solution is scalar but may not need to be linear.

Secret Sharing
Let M be a connected matroid with rank ≥ 2 and N be its ground set. Let 1 ∈ N be the special element. Let A m = {C \ {1} : 1 ∈ C, C is a circuit of M} and A = {A ⊆ N : ∃B ∈ A m s.t. B ⊆ A}. It can the checked that a VOA(M, v) forms an ideal secret sharing scheme of the access structure A, where the dealer is indexed by 1 and other participants are indexed by x ∈ N \ {1}. Such constructions can be seen in literatures such as [19][20][21][22][23].

Further Research
In this paper, we review and developed correspondences among matroidal entropy functions, connected matroids with rank ≥ 2, variable strength orthogonal arrays and almost affine codes. These correspondences can make them benefit from each other, and therefore yield more research topics in the following facets.

1.
Results of orthogonal arrays can be leveraged to characterize matroidal entropy functions as we do in Section 4 for those of order ≤ 5.

2.
Abundant tools in matroid theory can be used to study matroidal entropy functions, VOAs and almost affine codes. For example, in the proof of Lemma 5 and Proposition 6 , we implicitly use the fact that U 2,4 is minor of U 2,5 and M 2 , and U 2,4 is a forbidden minor for characteristic 2 and 6. 3.
Matroid representability is an important and well-studied area in matroid theory. See [11], (Chapter 6). A matroid M = (N, r) is called representable over a field F if there exists a matrix T with entries in F whose columns are indexed by N, and for each A ⊆ N, the rank of the submatrix consisting of the columns indexed by A is equal to r(A). As we discussed in Section 3.1.2 and 3.1.3, a matroid is called partitionrepresentable [8] or almost affinely representable [9] with degree v if there exists a VOA(M, v). Obviously, an F q -representable matroid is also partition-representable with degree q. However, the converse of the statement may not hold in general.

4.
The construction of an OA(t, n, v) is also an important problem in combinatorial design. For some parameters, say OA(2, 5, 10), the problem is extremely difficult. The definition of VOA provides more tools to attack the problem.

5.
Matroidal entropy functions induced by U t,n are called symmetric matroidal entropy functions. They are special cases of the p-symmetrical entropy functions, where p is the trivial partition of N with N being the only block. In general, for an arbitrary permutation group G on N, symmetries of an G-symmetric matroidal entropy function, i.e., an entropy function that is G-symmetric [24] and matroidal, can be utilized to construct its characterizing random vectors via VOA. [25]. orthogonal arrays. Finally the authors thank IEEE Information Theory Society Guangzhou Chapter for proposing opportunities to discuss the results of this paper.

Conflicts of Interest:
The authors declare no conflict of interest.

Abbreviations
The following abbreviations are used in this manuscript: OA Orthogonal array VOA variable strength orthogonal array MDS Maximum distance separable WLOG With loss of generality