This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/.)

For fixed ^{j}

Let _{3}(2) is assigned a label which is a substring of

The leaves of the tree, traversed left to right, are labeled with the respective left-to-right entries of the data string

For each non-leaf vertex _{L}, x_{R}_{L}_{R}_{L}_{R}_{R}_{L}

In _{L}_{R}_{L}, x_{R}_{R}_{L}

Before discussing the nature of the results to be obtained in this paper, we need some definitions and notation. Fix integers _{m}_{1}, _{2}, …, _{m}^{j}_{m}^{1}, ^{2}, …, ^{k}^{1}*^{2}*…*^{k}_{1}, _{2}, …, _{k}_{1} * _{2} * … * _{k}^{1} * ^{2} * … * ^{k}^{i}_{i}_{m}

We wish to formally define the family
_{m,k} of all hierarchical type classes in which the alphabet cardinality parameter is ^{1}, …, ^{k}_{i}^{i}_{k}_{m,k} is then the set of all hierarchical type classes, of all orders.

We define the type of _{1}, …, _{m}_{i}_{i}_{j}_{j}^{+}(_{j}_{j}^{j}

_{m,k}_{m,k} whose strings are of type _{m,k}(

_{m,k} be a hierarchical type class of order _{1}, _{2}, …, _{k}_{i}

The _{m,k} → [0, ∞) such that

Let _{1}, _{2}, …, _{k}

Represent

We see now how to inductively compute entropy values

Let {_{j}_{m,k} such that _{j}_{j}^{j}_{j}^{j}_{j}_{m,k}(_{m}^{m}_{m}_{λ}_{m}_{m}_{m}_{j}_{j}_{j}_{j→∞} _{λj}_{m}

This section is devoted to the discussion of hierarchical sources. The concept of hierarchical source was informally described in the Introduction. In Section 2.1., we make this concept formal. In Section 2.2., we define the entropy-stable hierarchical sources, which are the hierarchical sources that induce hierarchical entropy functions. In Section 2.3., we introduce a particular type of entropy-stable hierarchical source called finitary hierarchical source. The finitary hierarchical sources induce the hierarchical entropy functions that are the subject of this paper.

Let
_{m,k}

_{1}, _{2}, …, _{k}

We discuss how the Consistency Condition gives us a way to describe every possible hierarchical source. Let Λ(^{k}^{+} → Λ(^{k}_{1}, _{2}, …, _{k}

If _{i}

Each ^{ϕ}^{ϕ}

If ^{ϕ}_{i}_{i}_{m} whose type is

If ^{+}, assume class ^{ϕ}_{1}, _{2}, …, _{k}

From the Consistency Condition, all possible hierarchical sources arise in this way, that is, given any Λ(^{ϕ}

Another advantage of the Consistency Condition is that it allows the entropies of the classes in a hierarchical source to be recursively computed. To see this, let
^{ϕ}_{ϕ}_{0}(^{+}(_{1}, …, _{k}

The concept of entropy-stable source discussed in this section allows us to formally define the concept of hierarchical entropy function.

For each _{λ}_{m}

Let
_{m}

Suppose we have a hierarchical source
_{j}_{m}

Suppose
_{m}_{j}_{j}_{m}_{m}

If _{1}, _{2}, …, _{m}^{+}, define

ℛ(

Ψ(_{1}, …, _{m}_{1}, _{2}, …, _{m}_{1} + _{2} + … + _{m}

If _{1}, _{2}, …, _{m}^{+}, let _{1}, _{2}, …, _{k}

Suppose ^{ϕ}

Note that (1122) belongs to ℛ(4, 3). Suppose

Note that (7758) ∈ Λ^{+}(4, 3), and that

Since ⌊(7758)/3⌋ = (2212), we see that _{1}, _{2}, _{3}), where

Note that the splitting up of (7758) into the three types (3312), (2223), (2223) indeed does make sense because these latter three types sum to (7758) and are of order 2, one less than the order of (7758).

Fix the alphabet cardinality parameter to be 2, and fix the partitioning parameter _{1}, _{2}) belong to ℛ(2, _{1}, _{2}) = (0, 0) or (b) _{1} + _{2} = _{1}, _{2}) to be the _{1}, _{2}) to be the _{1} rows are (1, 0) and whose last _{2} rows are (0, 1). Letting ^{ϕ}

Now fix the alphabet cardinality parameter to be 3, and fix the partitioning parameter _{1}, _{2}, _{3}) belong to ℛ(3, _{1}, _{2}, _{3}) = (0, 0, 0); (b) _{1} + _{2} + _{3} = _{1} + _{2} + _{3} = 2_{1}, _{2}, _{3}) to be the _{1}, _{2}, _{3}) to be the _{1} rows are (100), whose next _{2} rows are (010), and whose last _{3} rows are (001). In case (c), we define _{1}, _{2}, _{3}) to be the _{1} rows are (011), whose next _{2} rows are (101), and whose last _{3} rows are (110). Letting ^{ϕ}

For each fixed

The source defined in Example 3 is the unique finitary Λ(2,

The source defined in Example 4 is the unique finitary Λ(3,

This is because the matrices employed in these examples are unique up to row permutation.

Let _{m}

Theorem 1 is proved in

Fix _{2,k} : Λ(2,

For later use, we remark that

The hierarchical entropy function induced by this source maps ℙ_{2} into [0, ∞) and shall be denoted _{2,k}. The relationship between functions _{2,k} and _{2,k} is

Fix _{3,k} : Λ(3,

For later use, we remark that

The hierarchical entropy function induced by this source maps ℙ_{3} into [0, ∞) and shall be denoted _{3,k}. The relationship between functions _{3,k} and _{3,k} is

In Section 3, we show that hierarchical entropy function _{2,k} is self-affine for each _{3,k} is self-affine for each

An iterated function system (IFS) on a closed nonempty subset Ω of a finite-dimensional Euclidean space is a finite nonempty set of mappings which map Ω into itself and are each contraction mappings. Given an IFS

Suppose _{m}_{m}_{m}^{m}^{+1}. We define _{m}

Each mapping in

The attractor of
_{m}

For the rest of this section, _{2,k} : ℙ_{2} → [0, ∞) is self-affine, where _{2,k} is the hierarchical entropy function induced by the unique finitary Λ(2,

For each

Define the matrix

Define

Define the vector

Define _{i}_{2} → Ω_{2} to be the mapping
_{i}

It is clear that the set of mappings
_{2}. This fact allows one to prove (Lemma B.3 of _{i}_{2}. This result is the first part of the following theorem.

Let

_{0}, _{1}, …, _{k−1}} is an IFS on Ω_{2}.

_{2,k} is self-affine and its graph is the attractor of the IFS in (a).

Our proof of Theorem 2 requires the following lemma.

Let ^{ϕ}

_{i}_{i}

^{+} and _{1}, _{2}, …, _{k}

Property (a.1), whose proof we omit, is a simple consequence of the fact that _{i}^{+}. Letting _{1}, _{2}, …, _{k}_{i}_{1}, _{2}, …, _{k}_{s}_{s}M_{i}_{1}, _{2}). As remarked in Example 3, either _{1} = _{2} = 0, or _{1} + _{2} = _{1} + _{2} =

It is easy to show that

It follows that

For 1 ≤ _{1}, we have

For _{1} + 1 ≤

The remaining case _{1} = _{2} = 0 is much easier. We have

We first derive part(c) and then part(b) (part(a) is already taken care of, as remarked previously). We derive part(c) by establishing _{2,k} denotes the entropy function _{ϕ}_{i}

We first show that

Our proof of _{i}

Similarly, if

By the induction hypothesis,

Adding,

By Lemma 2,

Appealing to _{1}_{i}_{k}M_{i}_{2} is the number of permutations of the _{1}, …, _{k}_{i}_{2}. Substituting the right hand sides of the previous two equations into _{i}

It is easy to see that

Therefore,

_{2} and _{2,k} is a continuous function on ℙ_{2}, completing the derivation of part(c) of Theorem 2. All that remains is to prove part(b) of Theorem 2. Let _{2,k}(_{2}} be the graph of _{2,k}. Part(c) is equivalent to the property that

This property, together with the fact that
_{2} with attractor ℙ_{2}, allows us to conclude that _{0}, …, _{i}_{−1}} (Lemma B.1 of _{2,k} is self-affine because the _{i}

For each

We can obtain ^{n}_{i}_{2} given in Theorem 2, such that the attractor of this IFS is the graph of _{2,k}. Let _{0}(_{1}(_{2}(_{n}^{3} by the recursion

Then _{n}^{n}_{2,k}(^{n}_{n}

The plot of
_{24} (2), consisting of 2^{24} = 16777216 points, computed in 4.2 seconds.

The plot of
_{15} (3) consisting of 3^{15} = 14348907 points, computed in 3.3 seconds.

The plot of
_{12}(4) consisting of 4^{12} = 16777216 points, computed in 3.5 seconds.

We point out that the functions

Fix _{3,k} : ℙ_{3} → [0, ∞), the hierarchical entropy function induced by the unique finitary Λ(3, ^{3}, let _{k}_{k}_{k}^{2} congruent equilateral triangles, formed as follows. Partition each of the three sides of _{k}_{k}_{k}^{2} congruent equilateral triangles of the quadratic partition. See _{3} into nine sub-triangles.

Let _{1} be the set of all points (_{k}_{1}. For each _{1}, let _{1,v} be the 3 × 3 matrix

For each _{1}, the convex hull of the rows of _{1,v} is one of the sub-triangles in the quadratic partition of _{k}_{1}. This gives us a total of _{k}_{1} sub-triangles of the partition. Let _{2} be the set of all (_{k}_{2}. For each _{2}, let _{2,v} be the 3 × 3 matrix

For each _{2}, the convex hull of the rows of _{2,v} is one of the sub-triangles in the quadratic partition of _{k}_{2}. This gives us a total of _{k}_{2} sub-triangles of the partition. The _{1} sub-triangles are all translations of each other; the _{2} sub-triangles are all translations of each other and each one can be obtained by rotating a _{1} sub-triangle about its center 180 degrees, followed by a translation. Together, the _{1} sub-triangles and the _{2} sub-triangles constitute all ^{2} sub-triangles in the quadratic partition of _{k}

We define ℳ(^{2} matrices

Each row sum of each matrix in ℳ(_{M}_{3} → Ω_{3} in which

It is clear that the set of ^{2} mappings
_{3}. This fact allows one to prove (Lemma B.4 of ^{2} mappings {_{M}_{3}. In the following example, we exhibit this IFS in a special case.

Let

Following _{i}^{3} be the vector whose components are the _{3,3} entropies of the rows of _{i}_{2} 3 and _{2} 6,

Following _{i}_{3} → Ω_{3} be the mapping defined by

Theorem 3 which follows will tell us that the graph of _{3,3} is the attractor of the IFS

Let

_{M}_{3}.

_{3,k} is self-affine and its graph is the attractor of the IFS in (a).

Our proof of Theorem 3 requires a couple of lemmas, which follow.

Let ^{+}, let _{1}, _{2}, …, _{k}_{i}_{i}_{1} + _{2} + _{3} = 2

If each _{i} < _{i} = _{i}_{i}_{1} =

Since _{2}, _{3} ∈ {1, 2, …, _{2} + _{3} =

_{1} = 0.

Let

^{+}, and _{1}, _{2}, …, _{k}_{1}_{2}_{k}M

Property (a.1), whose proof we omit, is a simple consequence of the fact that each matrix in ℳ(^{+} and fix _{1}, _{2}, _{3}), and let

_{1} + _{2} + _{3})/

We have

Note that

If

From

Property (a.2) then follows if the equations

Finally, if

Property (a.2) then follows if the equations

We have

Note that

If

From

In view of the fact that

Thus, Property (a.2) holds. Finally, suppose that

In view of the fact that

Thus, Property (a.2) holds.

We first derive part(c) and then part(b) (part(a) is already taken care of, as remarked previously). We derive part(c) by establishing _{3,k} denotes the entropy function _{ϕ}_{M}

We first show that

The proof is by induction on ‖_{1}, _{2}, …, _{k}

Adding,

By Lemma 4, _{1}_{2}_{k}M_{1}_{k}M_{2} is the number of permutations of the _{1}, …, _{k}_{2}. Substituting the right hand sides of the previous two equations into

It is easy to see that

Therefore,

_{3} and _{3,k} is a continuous function on ℙ_{3}, completing the derivation of part(c) of Theorem 3. All that remains is to prove part(b) of Theorem 3. Letting _{3,k}(_{3}} be the graph of _{3,k}, part(c) is equivalent to the property that

Note that
_{3}, and so ℙ_{3} must be the attractor of the IFS
_{M}_{3,k} is self-affine because the _{M}

We conclude the paper with a discussion of some properties of the self-affine hierarchical entropy functions _{2,k} and _{3,k}. For each _{m,k}

_{m,k}_{m}

_{1}, _{2} in ℙ_{m}

_{m}_{m,k}

_{m}

Properties P1-P4 are simple consequences of what has gone before. For example, to see why the symmetry property P2 is true, first observe that _{m,k}_{1}) = _{m,k}_{2}) if _{1}, _{2} are types which are permutations of each other; this symmetry property for entropy on types then extends to ℙ_{m}_{m,k}

The well-known Shannon entropy function _{m}_{m}_{i}_{2} _{i}_{i}_{m}_{m}_{m,k}_{2,2} and _{2,4} do not reach their maximum at (1/2, 1/2), but _{2,3} does. It is an open problem to determine the maximum value of each _{2,k} and _{3,k} and to see where the maximum is attained.

The inequality
_{m}

Example 1 Tree Representations and Codeword Table.

Plots of

Quadratic Partition Of Triangle _{3}.

The work of the author was supported in part by National Science Foundation Grant CCF-0830457.

In this Appendix, we prove Theorem 1. In the following, the infinity norm ‖_{∞} of a vector _{1}, _{2}, …, _{m}^{m}_{i} |_{i}

Let

If

We show there exists

_{0}, _{j}_{0} − _{∞}, there exist types _{1}, _{2}, …, _{I}_{j}_{I}_{0} to _{j}

_{0} ∈ Λ_{j}_{1}, _{2}, …, _{M}_{j}_{M}/k_{j−1}(_{j}

Let _{1}, _{2} belong to
_{1} − _{2}‖_{∞} ≤ ^{−J}. Fix _{1}, _{2} in Λ_{J′}_{1} = _{λ1} and _{2} = _{λ2}. Starting at _{1} and applying property (p.2) repeatedly (that is, for each

Similarly, we find

By the triangle inequality, we have

Applying property (p.1),

Let

It is easily seen that
_{m}_{j} ∊_{j}_{j}^{ϕ}_{j}_{∞} ≤ 1. Letting
_{1}, _{2} are positive integers ≤

We conclude that

This Appendix proves some auxiliary results useful for proving Theorems 2–3. Henceforth, ‖_{2} shall denote the Euclidean norm of a vector

Let
_{m}_{m}_{m}_{m}_{m}_{m}_{m}_{h}_{m}

Then _{h}

Let _{h}_{h}_{h}_{m}_{h}_{m}_{h}_{h}

Let _{m}_{m}

Let _{1}, _{2}, …, _{m}^{m}_{m} → Ω_{m} be the mapping

Then ^{−2}(1 − ^{2})^{2}.

By the intermediate value theorem, there is a real number

Then _{m}

The right hand side of

Therefore, we will be done if we can show that
_{m}_{p,q}_{p,q}^{2} is the positive number ^{2} − ^{2}. Therefore, _{p,q}

It follows that _{m}_{1}, _{2}, …, _{m}^{m}

It is a simple exercise in Lagrange multipliers, which we omit, to show that the vector _{1}, …, _{m}

For this choice of

But this is true with equality, by

Let _{i}_{2} → Ω_{2} defined in Section 3 is a contraction.

Fix ^{−1}. Applying Lemma B.2 with ^{−1}, _{i}

It is easy to compute that
_{1} = _{2} =

Using the fact that

Choosing the smallest value of

Using calculus, it is easy to show that

Thus,

Let _{M}_{3} → Ω_{3} defined in Section 4 is a contraction.

The mapping
^{−1}. Applying Lemma B.2 with ^{−1}, we have to show that various variances are all less than (^{−1})^{2}. Specifically, for each (_{1} we wish to show
_{2} we wish to show

Using

Let _{1} = _{2} = _{3} =

Using the fact that

Choosing the smallest value of

Similarly, the variance on the left side of _{2}(_{2} _{2}

Using calculus, it is easy to show that

Thus, for each (_{1}, we have the desired inequality
_{2}, we have the desired inequality