Next Article in Journal
Graphical Gaussian Models Associated to a Homogeneous Graph with Permutation Symmetries
Previous Article in Journal
Adaptive Importance Sampling for Equivariant Group-Convolution Computation

Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Proceeding Paper

# Information Properties of a Random Variable Decomposition through Lattices †

Institute of Mathematics, Statistics and Scientific Computing (IMECC), University of Campinas, Campinas 13083-859, Brazil
*
Author to whom correspondence should be addressed.
Presented at the 41st International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering, Paris, France, 18–22 July 2022.
Phys. Sci. Forum 2022, 5(1), 19; https://doi.org/10.3390/psf2022005019
Published: 5 December 2022

## Abstract

:
A full-rank lattice in the Euclidean space is a discrete set formed by all integer linear combinations of a basis. Given a probability distribution on $R n$, two operations can be induced by considering the quotient of the space by such a lattice: wrapping and quantization. For a lattice $Λ$, and a fundamental domain $D$, which tiles $R n$ through $Λ$, the wrapped distribution over the quotient is obtained by summing the density over each coset, while the quantized distribution over the lattice is defined by integrating over each fundamental domain translation. These operations define wrapped and quantized random variables over $D$ and $Λ$, respectively, which sum up to the original random variable. We investigate information-theoretic properties of this decomposition, such as entropy, mutual information and the Fisher information matrix, and show that it naturally generalizes to the more abstract context of locally compact topological groups.

## 1. Introduction

Lattices are discrete sets in $R n$ formed by all integer linear combinations of a set of independent vectors, and have found different applications, such as in information theory and communications [1,2,3]. Given a probability distribution in $R n$, two operations can be induced by considering the quotient of the space by a lattice: wrapping and quantization.
The wrapped distribution over the quotient is obtained by summing the probability density over each coset. It is used to define parameters for lattice coset coding, particularly for AWGN and wiretap channels, such as the flatness factor, which is, up to a constant, the $L ∞$ distance from a wrapped probability distribution to a uniform one [4,5]. This factor is equivalent to the smoothing parameter, used in post-quantum lattice-based cryptography [6]. In the context of directional statistics, wrapping has been used as a standard way to construct distributions on a circle and on a torus [7].
The quantized distribution over the lattice can be defined by integrating over each fundamental domain translation, thus corresponding to the distribution of the fundamental domains after lattice-based quantization is applied. Lattice quantization has different uses in signal processing and coding: for instance, it can achieve the optimal rate-distortion trade-off and can be used for shaping in channel coding [2]. A special case of interest is when the distribution on the fundamental region is uniform, which amounts to high-resolution quantization or dithered quantization [8,9].
In this work, we relate these two operations by remarking that the random variables induced by wrapping and quantization sum up to the original one. We study information properties of this decomposition, both from classical information theory [10] and from information geometry [11], and provide some examples for the exponential and Gaussian distributions. We also propose a generalization of these ideas to locally compact groups. Probability distributions on these groups have been studied in [12], and some information-theoretic properties have been investigated in [13,14,15]. In addition to probability measures, one can also define the notions of lattice and fundamental domains on them, thereby generalizing the Euclidean case. We show that wrapping and quantization are also well defined, and provide some illustrative examples.

## 2. Lattices, Wrapping and Quantization

#### 2.1. Lattices and Fundamental Domains

A lattice $Λ$ in $R n$ is a discrete additive subgroup of $R n$, or, equivalently, the set $Λ = { α 1 b 1 + ⋯ + α k b k | α 1 , ⋯ , α k ∈ Z }$ formed by all integer linear combinations of a set of linearly independent vector ${ b 1 , ⋯ , b k } ⊂ R n$, called a basis of $Λ$. A matrix B whose column vectors form a basis is called a generator matrix of $Λ$, and we have $Λ = B Z k$. The lattice dimension is k, and, if $k = n$, the lattice is said to be full-rank; we henceforth consider full-rank lattices. A lattice $Λ$ defines an equivalence relation in $R n$: $x ∼ y ⇔ x − y ∈ Λ$. The associated equivalence classes are denoted by $x ¯$ or $x + Λ$. The set of all equivalence classes is the lattice quotient $R n / Λ$, and we denote the standard projection $π : R n → R n / Λ , π ( x ) = x ¯$.
Let $D$ be a Lebesgue-measurable set of $R n$ and $Λ$ a lattice. We say that $D$ is a fundamental domain or a fundamental region of $Λ$, or that $D$ tiles $R n$ by $Λ$, if (1) $⋃ λ ∈ Λ ( λ + D ) = R n$, and (2) $( λ + D ) ∩ ( λ ˜ + D ) = ∅$, for all $λ ≠ λ ˜$ in $Λ$ (it is often only asked that this intersection has Lebesgue measure zero, but we require it to be empty). Given a fundamental domain $D$, each coset $x ¯ ∈ R n / Λ$ has a unique representative in $D$, i.e., the measurable map $π | D : D → R n / Λ$ is a bijection. This fact suggests using a fundamental domain to represent the quotient. Each fundamental domain contains exactly one lattice point, which may be chosen as the origin. An example of a fundamental domain is the fundamental parallelotope with respect to a basis ${ b 1 , ⋯ , b n }$, namely $P ( Λ ) := { x = α 1 b 1 + ⋯ + α n b n | α 1 , ⋯ , α n ∈ 0 , 1 }$. Another one is the Voronoi region $V ( Λ )$ of the origin, given by the points that are closer to the origin than to any other lattice point, with an appropriate choice for ties. It is a well-known fact that every fundamental domain has the same volume, denoted by $covol Λ := vol D = | det B |$, for any generator matrix B of $Λ$.

#### 2.2. Wrapping and Quantization

Consider $R n$ with the Lebesgue measure $μ$, and P a probability measure such that $P ≪ μ$. Then the probability density function (pdf) of P is $p = d P d μ$, the Radon–Nikodym derivative. For fixed full-rank lattice $Λ$ and fundamental domain $D$, the wrapping of P by $Λ$ is the distribution $P π := π * P$ on $R n / Λ$, given by $P π ( A ) = P ( π − 1 A )$. For simplicity, we identify $R n / Λ$ with $D$ to regard $P π$ as a distribution over $D$, and then we have $π : R n → D$ given by $( y + λ ) ↦ y$, for all $y ∈ D , λ ∈ Λ$. Using this identification, the wrapping has density $p π = d P π d μ$ given by
$p π ( y ) = ∑ λ ∈ Λ p ( y + λ ) .$
A construction that is, in some sense, dual to wrapping is quantization. Note that each fundamental domain $D$ partitions the space as $R n = ⨆ λ ∈ Λ ( λ + D )$. The quantization function is the measurable map $Q : R n → Λ$, given by $( y + λ ) ↦ λ$, for $y ∈ D$ and $λ ∈ Λ$. The quantized probability distribution of P on the discrete set $Λ$ is $P Q := Q * P$, given by $P Q ( A ) := P ( Q − 1 A )$. The probability mass function of the quantized distribution is then
$p Q ( λ ) = ∫ D p ( y + λ ) d y .$
Letting X be a vector random variable in $R n$ with distribution p, we define $X π := π ( X )$ and $X Q := Q ( X )$ the wrapped and quantized random variables, respectively. By definition, they are distributed according to $p π$ and $p Q$. Interestingly, they sum up to the original one:
$X = X π + X Q ,$
since $π + Q = id R n$. Note also that $X π + X Q$ has the same distribution as $( X π , X Q )$, by the bimeasurable bijection $y + λ ↦ ( y , λ )$. These factors, however, are not independent, since, in general, $p ( y + λ ) ≠ p π ( y ) p Q ( λ )$. The difference between $p ( x )$ and $( p π ⊗ p Q ) ( x ) := p π ( π ( x ) ) p Q ( Q ( x ) )$ shall be illustrated in the following examples. Note that the expression for the quantized distribution depends on the choice of fundamental domain, while the wrapped distribution does not, up to a lattice translation.
We say a random variable X over $[ 0 , ∞ )$ is memoryless if $C ¯ ( t ) = C ¯ ( t + s ) / C ¯ ( s )$ for all $t , s$, where $C ¯ ( t ) := P [ X > t ]$ is the tail distribution function. In particular, a memoryless distribution satisfies $C ¯ ( y + λ ) = C ¯ ( y ) C ¯ ( λ )$ for all $y ∈ D$, $λ ∈ Λ$, which implies $p = p π ⊗ p Q$. The converse, however, is not true; for example, independence holds whenever p is constant on each region $λ + D$, for $λ ∈ Λ$.
Example 1.
The exponential distribution, parametrized by $ν > 0$, is defined as
$p ( x ) = ν e − ν x 1 0 , + ∞ ( x ) ,$
where $1 A ( x )$ takes value 1 if $x ∈ A$, and 0 otherwise. Choosing the lattice $Λ = α Z$, $α ∈ R +$, and the fundamental domain $D = 0 , α$, one can write closed-form expressions for the wrapped and quantized distributions:
$p π ( y ) = ν e − ν y 1 − e − ν α , y ∈ D and p Q ( λ ) = e − ν λ 1 − e − ν α , λ ∈ Λ ∩ R + .$
Note that, in this special case, $p = p π ⊗ p Q$, as a consequence of memorylessness. The wrapped distribution with $α = 2 π$, which amounts to a distribution on the unitary circle, is well studied in [16].
Example 2.
Consider the univariate Gaussian distribution
$p ( x ) = 1 2 π σ 2 exp − ( x − μ ) 2 2 σ 2$
and the lattice $Λ = α Z$, with fundamental domain $D = − α 2 , α 2$, $α ∈ R +$. The wrapped and quantized distributions are given respectively by
$p π ( y ) = 1 2 π σ 2 ∑ i ∈ Z e − ( y − μ + α i ) 2 2 σ 2 , y ∈ D and p Q ( λ ) = 1 2 π σ 2 ∫ λ − α 2 λ + α 2 e − ( x − μ ) 2 2 σ 2 d x , λ ∈ Λ .$
The value $α = 2 π$ for the wrapped distribution on a unitary circle is usually considered in directional statistics [7]. Figure 1 illustrates the original, wrapped, quantized and product distributions for different zero-mean Gaussian distributions. As can be seen in the figure, in this case, $p ( x ) ≠ p π ( y ) p Q ( λ )$.
A straightforward consequence of the decomposition (3) is
• $E [ X ] = E [ X π ] + E [ X Q ]$;
• $Var [ X ] = Var [ X π ] + Var [ X Q ] + Cov [ X π , X Q ] + Cov [ X Q , X π ]$,
where $E [ · ]$, $Var [ · ]$ and $Cov [ · , · ]$ denote, respectively, the expectation, the variance and the cross-covariance operators.
We note that different types of discretization have also been studied, other then integrating over a fundamental domain [17]. For instance, in [4,18,19], the discretized distribution is defined by restricting the original pdf $p ( x )$ to the lattice $Λ$, and then normalizing:
$D Λ , c ( λ ) := p ( c + λ ) ∑ λ ˜ ∈ Λ p ( c + λ ˜ ) ,$
for a fixed $c ∈ D$. This discretization is nothing other than the conditional distribution of $X Q$ given that $X π = c$, expressed as $p Q | π ( λ | c ) = p ( c + λ ) / p π ( c )$. Moreover, when $p = p π ⊗ p Q$, such as in the exponential distribution, cf. example 1, then $D Λ , c ( λ ) = p Q ( λ )$.

## 3. Information Properties

#### 3.1. Information-Theoretic Measures

Let us consider a random variable X with distribution p and the induced wrapped and quantized ones, respectively, $X π ∼ p π$ and $X Q ∼ p Q$. The mutual information between $X π$ and $X Q$ is defined as the Kullback–Leibler divergence $I ( X π ; X Q ) := D KL p ∥ p π ⊗ p Q$, and is a measure of how non-independent the marginal distributions $p π$ and $p Q$ are [10]. Using the theorem of change of variables, we have
$I ( X π ; X Q ) = E X log p ( X ) p π ⊗ p Q ( X ) = E X [ log p ( X ) ] − E X [ log p π ( X π ) ] − E X [ log p Q ( X Q ) ] = E X [ log p ( X ) ] − E X π [ log p π ( X π ) ] − E X Q [ log p Q ( X Q ) ] = h ( X π ) + H ( X Q ) − h ( X ) .$
Note that, from this decomposition, we have $h ( X ) ≤ h ( X π ) + H ( X Q )$.
Proposition 1.
Let X be a real random variable, and $X π$ and $X Q$ the respective wrapped and quantized random variables, using the lattice $α Z$. Denote $μ Q := E [ X Q ]$ and $σ Q 2 := Var [ X Q ]$. If X has support $[ 0 , ∞ )$, then the mutual information $I ( X π ; X Q )$ between $X π$ and $X Q$ is upper-bounded by
$I ( X π ; X Q ) < log e μ Q + α / 2 − h ( X ) .$
If X has support $R$, then $I ( X π ; X Q )$ is upper-bounded by
$I ( X π ; X Q ) < 1 2 log ( 2 π e σ Q 2 ) + 2 log e exp ( 2 π 2 α − 2 σ Q 2 ) − 1 − h ( X ) .$
Proof.
First, $h ( X π ) ≤ log α$, since the uniform distribution maximizes entropy on a bounded support. Then, note that the mean and variance of the integer-valued random variable $α − 1 X Q$ are $α − 1 μ Q$ and $α − 2 σ Q 2$, respectively. For (7), use that, for positive integer random variables, $H ( X Q ) < log e μ Q / α + 1 / 2$, as in [20] (Theorem 8); for (8), the upper-bound for integer-valued random variables from [20] (Theorem 10) gives us $H ( X Q ) < 1 2 log ( 2 π e α − 2 σ Q 2 ) + 2 log e exp ( 2 π 2 α − 2 σ Q 2 ) − 1$. Replacing the corresponding inequalities in (6) yields the desired results. □
The following lemma can be found in [2] (Appendix 3).
Lemma 1.
$h ( X π ) ≤ h ( X )$.
Proof.
$h ( X ) = h ( X π ) + H ( X Q | X π )$, and $H ( X Q | X π ) ≥ 0$, since it is a discrete entropy. □
Proposition 2.
Let $Λ α := α Λ$, $α > 0$, be a family of lattices, with fundamental domains $D α := α D$.
1.
If $D$ is connected, and p is continuous and Riemann-integrable, then $lim α → 0 I ( X π ; X Q ) = 0$.
2.
If 0 is an interior point of $D$, then $lim α → + ∞ I ( X π ; X Q ) = 0$.
Proof.
For $α → 0$, the proof is an adaptation of [10] (Theorem 8.3.1). Since $D$ is connected and p is continuous, we can use the mean value theorem: for every $λ ∈ Λ$ there exists an $x λ , α ∈ ( λ + D α )$ such that $p ( x λ , α ) vol D α = p Q ( λ )$. Therefore, we can write $H ( X Q ) = − ∑ λ ∈ Λ α p ( x λ , α ) log ( p ( x λ , α ) ) vol D α − log ( vol D α )$, using that $∑ λ ∈ Λ α p ( x λ , α ) vol D α = 1$. The first term is an n-dimensional Riemann sum, and converges to $h ( X )$ when $α → 0$, while the second term becomes arbitrarily small. Therefore, $0 ≤ I ( X π ; X Q ) ≤ H ( X Q ) + log ( vol D α ) − h ( X ) → 0$, so $I ( X π ; X Q ) → 0$.
For $α → + ∞$, note that, from Lemma 1, $I ( X π ; X Q ) ≤ H ( X Q )$. However, by choosing $α$ sufficiently large, we can make $p Q ( 0 ) = ∫ D α p ( x ) d x$ arbitrarily close to 1, since 0 is in the interior of $D α$. Therefore, $H ( X Q )$ can be made arbitrarily small. □
Example 3.
In the case of the exponential distributions, as in Example 1, the distributions of $X π$ and $X Q$ are independent, i.e., $p = p π ⊗ p Q$, therefore $I ( X π ; X Q ) = 0$. The mutual information and the corresponding upper bound (7) are plotted in Figure 2a, as function of the parameter ν.
Example 4.
In the case of the univariate zero-mean Gaussian distributions, as in Example 2, one can use (6) to numerically compute the mutual information $I ( X π ; X Q )$, as a function of the standard deviation σ, and compare it with the upper bound (8) (Figure 2b). Interestingly, $I ( X π ; X Q )$ vanishes as $σ → 0$ or $σ → + ∞$, which is equivalent to choosing a lattice $Λ = α Z$ with $α → 0$ or $α → + ∞$, cf. Proposition 2. The mutual information attains a maximum in $σ ≈ 0.38$, showing this is the value for which $X π$ and $X Q$ are the least independent.

#### 3.2. Fisher Information

Let $M = { p θ : θ ∈ Θ }$ be a family of probability densities $p θ : R n → R +$ smoothly parametrized by $θ$ in an open set $Θ ⊂ R d$. The Fisher information matrix is defined as the positive semi-definite matrix $G ( θ )$ with coefficients $g i j ( θ ) = E p θ [ ∂ i ℓ θ ∂ j ℓ θ ]$, where $ℓ θ ( x ) := log p θ ( x )$. When M is a manifold satisfying certain regularity conditions [11], and G is positive definite, it becomes a Riemannian manifold with the metric given by $g i j ( θ )$, called a statistical manifold. Let ⪯ denote the Loewner partial order for matrices, given by $A ⪯ B$ if, and only if, $B − A$ is positive semi-definite. The following results justify the name information matrix given to this quantity.
Proposition 3
([11,21]). Let X be a random variable distributed according to a distribution parametrized by θ, and $G ( θ )$ its information matrix. The following hold.
1.
Monotonicity:if $F : X → Y$ is a measurable function (i.e., a statistic) and $G F ( θ )$ is the information matrix of $F ( X )$, then $G F ( θ ) ⪯ G ( θ )$, with equality if, and only if, F is a sufficient statistic for θ.
2.
Additivity:if $X , Y$ are independent random variables, then the joint information matrix satisfies $G ( X , Y ) ( θ ) = G X ( θ ) + G Y ( θ )$.
Let X be a random variable on $R n$, and $X π$ and $X Q$ its wrapped and quantized factors, respectively. We denote their respective Fisher information matrices by $G ( θ )$, $G π ( θ )$ and $G Q ( θ )$. By additivity, the Fisher information of $p π ⊗ p Q$ is $G ˜ ( θ ) := G π ( θ ) + G Q ( θ )$, and, by monotonicity, we have both $G π ( θ ) ⪯ G ( θ )$ and $G Q ( θ ) ⪯ G ( θ )$. It follows immediately that
$G ˜ ( θ ) 2 = G π ( θ ) + G Q ( θ ) 2 ⪯ G ( θ ) .$
Example 5.
In the family of exponential distributions, as in Example 1, the independence of $X π$ and $X Q$ implies that the Fisher information matrix is additive. Indeed, for $Λ = α Z$:
$G ( ν ) = 1 ν 2 , G π ( ν ) = 1 ν 2 + α 2 2 ( 1 − cosh ( α ν ) ) , and G Q ( ν ) = α 2 2 ( cosh ( α ν ) − 1 ) .$

## 4. A Generalization to Topological Groups

A topological group is a topological space $( G , τ G )$ that is also a group with respect to some operation · called product, and such that the inverse $g − 1$ and product $g · h$ are continuous. As additional requisites, we ask G to be locally compact, Hausdorff and second-countable (i.e., has a countable basis) [22]. Let $B G$ be the the Borel $σ$-algebra of G. Haar’s theorem says there is a unique (up to a constant) Radon measure on G that is invariant by left translations—we will suppose a fixed normalization, and denote both the measure and integration with respect to it by dg. The group G is said to be unimodular if dg is also invariant by right translations. Since G is $σ$-compact, the Haar measure is $σ$-finite [12].
Let $Γ$ be a discrete subgroup of G, which is necessarily closed, since G is Hausdorff, and countable, since G is second-countable. Let us also consider the quotient space of left cosets $G / Γ = { g ¯ = g Γ | g ∈ G } ,$ which has a natural projection $π : G → G / Γ$, given by $π ( g ) = g ¯$. We call $Γ$ a lattice if the induced Haar measure on $G / Γ$ is finite and bi-invariant. A particular case is when the quotient $G / Γ$ is compact; then $Γ$ is said to be a uniform lattice. A cross-section is defined as a set $D ⊂ G$ of representatives of $G / Γ$ such that all cosets are uniquely represented. A fundamental domain is a measurable cross-section. It can be shown that $Γ$ is a lattice if, and only if, it admits a fundamental domain. Furthermore, every fundamental domain has the same measure [23,24].
Let P be a probability measure on the space $( G , B G )$ that is absolutely continuous with respect to the Haar measure dg. By the Radon–Nikodym theorem, we can define a density function $p = d P d g ∈ L 1 ( G )$, such that $p ≥ 0$ and $P ( A ) = ∫ A p ( g ) d g$, for all $A ∈ B G$. The original measure can be represented as $P = p d g$, and we consider the family of all such densities
${ P ( G ) = p ∈ L 1 ( G ) | p ≥ 0 d g - a . e . , ∫ p d g = 1 } .$
Probability distributions on locally compact groups have been studied in [12], and some information-theoretic properties have been investigated in [13,14,15]. The result that allows us to consider wrapped distributions in this context is the Weil formula, taken as a particular case of [24] (Theorem 3.4.6):
Theorem 1.
For any $f ∈ L 1 ( G )$, the wrapping $f π ∈ L 1 ( G / Γ )$, $f π ( g ¯ ) := ∑ λ ∈ Γ f ( g λ )$ is well defined $d g ¯$-almost everywhere, belongs to $L 1 ( G / Γ )$, and
$∫ G f ( g ) d g = ∫ G / Γ ∑ λ ∈ Γ f ( g λ ) d g ¯ .$
As a consequence, for every probability density $p ∈ P ( G )$, we can consider its wrapping $p π ( g ¯ ) = ∑ λ ∈ Γ p ( g λ )$, which is $L 1 ( G / Γ )$, non-negative and is also a probability density: $∫ G / Γ p π d g ¯ = 1$. The associated probability measure over $( G / Γ , B G / Γ )$ is $P π = p π d g ¯$. This notation, suggesting $P π$ as the push-forward measure by $π$, is not a coincidence, since, from Theorem 1,
$π * P ( A ) = ∫ G 1 A ( π ( g ) ) p ( g ) d g = ∫ G / Γ ∑ λ ∈ Γ 1 A ( π ( g ) ) p ( g λ ) d g ¯ = ∫ G / Γ 1 A ( g ¯ ) p π ( g ¯ ) d g ¯ = P π ( A ) .$
Analogously, given a fundamental domain $D$, it is possible to define a quantization map $Q : G → Λ$ by $Q ( g λ ) = λ$, for every $g ∈ D , λ ∈ Γ$, which is unique since $G = ⨆ g ∈ D g Γ$. The quantized probability distribution is the discrete probability measure $P Q$ over $Λ$, defined by the mass function $p Q ( λ ) = ∫ D p ( g λ ) d g$, or as the push-forward measure $Q * P$.
If X is distributed according to p, and $X π = π ( X ) ∼ p π$, $X Q = Q ( X ) ∼ p Q$, then $X = X π · X Q$, again, as a consequence of $g ↦ ( π ( g ) , Q ( g ) )$ being a measurable bijection whose inverse is the product $π ( g ) · Q ( g )$. Despite being an abstract definition, this framework expands the scope of the previous approach, cf. examples below. In the following, let $Λ ⊂ R n$ be a full-rank lattice, and $Λ s ⊂ Λ$ be a full-rank sublattice, as defined in Section 2.
Example 6.
Let $G = R n$ and $Γ = Λ$. This recovers the approach from Section 2 as a particular case.
Example 7.
Let $G = Λ$ and $Γ = Λ s$. A fundamental domain is a choice $D = { d 1 , ⋯ , d k }$ of $k = | Λ / Λ s |$ points, where each point corresponds to a coset $λ ¯ = ( λ + Λ s ) ∈ Λ / Λ s$. Of particular interest are Voronoi constellations [25,26] where the coset leaders are selected, with some choice made for ties. Since Λ is discrete, the Haar measure is the counting measure $μ ( A ) = | A |$, and $p : Λ → [ 0 , 1 ]$. The wrapped and quantized distributions are $p π ( λ ¯ ) = ∑ λ s ∈ Λ s p ( λ + λ s )$, and $p Q ( λ s ) = ∑ i = 1 k p ( d i + λ s )$.
Example 8.
Let $G = R n / Λ s$ (a torus) and $Γ = π s ( Λ )$ (the projection of Λ to G). Then $π s ( Λ )$ consists of a finite family of cosets $λ ¯ 1 , ⋯ , λ ¯ k$, for $k = | Λ / Λ s |$, and a choice of fundamental domain $D ¯$ is the projection of a fundamental domain $D$ of Λ. There are some standard choices for the distribution on G, such as a wrapping from the Euclidean space and the bivariate von Mises distribution [7] (Section 11.4). Then, $p π ( x ¯ ) = ∑ i = 1 k p ( x ¯ + λ ¯ i )$ and $p Q ( λ ¯ i ) = ∫ D ¯ p ( x ¯ + λ ¯ k ) d x ¯$, and, in the particular case where $p ( x ¯ ) = ∑ λ s ∈ Λ s p ( x + λ s )$ is a $Λ s$-wrapped distribution, they become $p π ( x ¯ ) = ∑ i = 1 k ∑ λ s ∈ Λ s p ( x + λ s + λ i )$ and $p Q ( λ ¯ i ) = ∑ λ s ∈ Λ s ∫ D p ( x + λ s + λ i ) d x$.
Example 9.
Let $G = F q n$ (a finite field) or $G = Z q n$, and $Γ = C$ (any linear block code). A fundamental domain can be a finite set of points that tiles the space by $C$. The distributions then become finite sums, such as in Example 7.
Example 10.
Let $G = SL ( n , R )$, the Lie group of square matrices with determinant 1, and $Γ = SL ( n , Z )$ (the subgroup of integer matrices). This is, in fact, a lattice, since for $n = 2$, $vol ( G / Γ ) = 2 ζ ( 2 )$ where ζ is the Riemann zeta function, and for $n > 2$ the finite covolume is calculated in [27], where descriptions of fundamental domains are also given.

## 5. Conclusions

In this work, we have studied the decomposition of a random variable through lattices into its wrapping and quantization terms. Generalization of examples and of Proposition 1 to higher dimensions constitutes a work in progress. We have also proposed a generalization of this decomposition to topological groups; in particular, this allows one to study information theory on such abstract spaces, which is another perspective for future work.

## Author Contributions

Conceptualization, all authors; investigation, F.C.C.M. and H.K.M.; writing—original draft preparation, F.C.C.M. and H.K.M.; writing—review and editing, all authors; supervision, S.I.R.C. All authors have read and agreed to the published version of the manuscript.

## Funding

This work was partly supported by Brazilian National Council for Scientific and Technological Development (CNPq) grants 141407/2020-4 and 314441/2021-2, and by São Paulo Research Foundation (FAPESP) grant 2021/04516-8.

Not applicable.

Not applicable.

Not applicable.

## Acknowledgments

The authors are grateful to Max Costa for fruitful discussions, and to the reviewer for the relevant suggestions.

## Conflicts of Interest

The authors declare no conflict of interest.

## References

1. Conway, J.H.; Sloane, N.J.A. Sphere Packings, Lattices and Groups; Springer: New York, NY, USA, 1999. [Google Scholar]
2. Zamir, R. Lattice Coding for Signals and Networks: A Structured Coding Approach to Quantization, Modulation and Multiuser Information Theory; Cambridge University Press: Cambridge, UK, 2014. [Google Scholar]
3. Costa, S.I.R.; Oggier, F.; Campello, A.; Belfiore, J.C.; Viterbo, E. Lattices Applied to Coding for Reliable and Secure Communications; Springer: Cham, Switzerland, 2017. [Google Scholar]
4. Ling, C.; Belfiore, J.C. Achieving AWGN channel capacity with lattice Gaussian coding. IEEE Trans. Inf. Theory 2014, 60, 5918–5929. [Google Scholar] [CrossRef][Green Version]
5. Damir, M.T.; Karrila, A.; Amorós, L.; Gnilke, O.W.; Karpuk, D.; Hollanti, C. Well-rounded lattices: Towards optimal coset codes for Gaussian and fading wiretap channels. IEEE Trans. Inf. Theory 2021, 67, 3645–3663. [Google Scholar] [CrossRef]
6. Chung, K.M.; Dadush, D.; Liu, F.H.; Peikert, C. On the lattice smoothing parameter problem. In Proceedings of the 2013 IEEE Conference on Computational Complexity, Stanford, CA, USA, 5–7 June 2013; pp. 230–241. [Google Scholar]
7. Mardia, K.V.; Jupp, P.E. Directional Statistics; Wiley: New York, NY, USA, 2000. [Google Scholar]
8. Zamir, R.; Feder, M. On lattice quantization noise. IEEE Trans. Inf. Theory 1996, 42, 1152–1159. [Google Scholar] [CrossRef]
9. Ling, C.; Gan, L. Lattice quantization noise revisited. In Proceedings of the 2013 IEEE Information Theory Workshop (ITW), Sevilla, Spain, 9–13 September 2013; pp. 1–5. [Google Scholar]
10. Cover, T.M.; Thomas, J.A. Elements of Information Theory, 2nd ed.; Wiley: Hoboken, NJ, USA, 2006. [Google Scholar]
11. Amari, S.; Nagaoka, H. Methods of Information Geometry; American Mathematical Society: Providence, RI, USA, 2000. [Google Scholar]
12. Heyer, H. Probability Measures on Locally Compact Groups; Springer: Berlin/Heidelberg, Germany, 1977. [Google Scholar]
13. Chirikjian, G.S. Stochastic Models, Information Theory, and Lie Groups; Birkhäuser: Boston, MA, USA, 2009. [Google Scholar]
14. Johnson, O.; Suhov, Y. Entropy and convergence on compact groups. J. Theor. Probab. 2000, 13, 843–857. [Google Scholar] [CrossRef]
15. Chirikjian, G.S. Information-theoretic inequalities on unimodular Lie groups. J. Geom. Mech. 2010, 2, 119–158. [Google Scholar] [CrossRef] [PubMed]
16. Jammalamadaka, S.R.; Kozubowski, T.J. New families of wrapped distributions for modeling skew circular data. Commun. Stat.–Theory Methods 2004, 33, 2059–2074. [Google Scholar] [CrossRef]
17. Chakraborty, S. Generating discrete analogues of continuous probability distributions-A survey of methods and constructions. J. Stat. Distrib. Appl. 2015, 2, 6. [Google Scholar] [CrossRef][Green Version]
18. Nielsen, F. The Kullback–Leibler divergence between lattice Gaussian distributions. J. Indian Inst. Sci. 2022. [Google Scholar] [CrossRef]
19. Luzzi, L.; Vehkalahti, R.; Ling, C. Almost universal codes for MIMO wiretap channels. IEEE Trans. Inf. Theory 2018, 64, 7218–7241. [Google Scholar] [CrossRef][Green Version]
20. Rioul, O. Variations on a Theme by Massey. IEEE Trans. Inf. Theory 2022, 68, 2813–2828. [Google Scholar] [CrossRef]
21. Kagan, A.; Smith, P.J. Multivariate normal distributions, Fisher information and matrix inequalities. Int. J. Math. Educ. Sci. Technol. 2001, 32, 91–96. [Google Scholar] [CrossRef]
22. Pontryagin, L.S. Topological Groups, 3rd ed.; Gordon and Breach Science Publishers: Montreux, Switzerland, 1986. [Google Scholar]
23. Raghunathan, M.S. Discrete Subgroups of Lie Groups; Springer: New York, NY, USA, 1972. [Google Scholar]
24. Reiter, H.; Stegeman, J.D. Classical Harmonic Analysis and Locally Compact Groups, 2nd ed.; Clarendon Press: Oxford, UK, 2000. [Google Scholar]
25. Forney, G.D. Multidimensional constellations. II. Voronoi constellations. IEEE J. Sel. Areas Commun. 1989, 7, 941–958. [Google Scholar] [CrossRef]
26. Boutros, J.J.; Jardel, F.; Méasson, C. Probabilistic shaping and non-binary codes. In Proceedings of the 2017 IEEE International Symposium on Information Theory (ISIT), Aachen, Germany, 25–30 June 2017; pp. 2308–2312. [Google Scholar]
27. Paula, G.T. Comparison of volumes of Siegel sets and fundamental domains for SLn(). Geom. Dedicata 2019, 199, 291–306. [Google Scholar] [CrossRef]
Figure 1. Example of zero-mean Gaussian distributions and their corresponding wrapped, quantized and product distributions, with $Λ = Z$ and $D = [ − 1 2 , 1 2 [$ for different variances: $σ 2 = 0.25$ (blue), $σ 2 = 1$ (orange), $σ 2 = 4$ (green).
Figure 1. Example of zero-mean Gaussian distributions and their corresponding wrapped, quantized and product distributions, with $Λ = Z$ and $D = [ − 1 2 , 1 2 [$ for different variances: $σ 2 = 0.25$ (blue), $σ 2 = 1$ (orange), $σ 2 = 4$ (green).
Figure 2. Mutual information $I ( X π ; X Q )$ and its upper bound.
Figure 2. Mutual information $I ( X π ; X Q )$ and its upper bound.
 Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Share and Cite

MDPI and ACS Style

Meneghetti, F.C.C.; Miyamoto, H.K.; Costa, S.I.R. Information Properties of a Random Variable Decomposition through Lattices. Phys. Sci. Forum 2022, 5, 19. https://doi.org/10.3390/psf2022005019

AMA Style

Meneghetti FCC, Miyamoto HK, Costa SIR. Information Properties of a Random Variable Decomposition through Lattices. Physical Sciences Forum. 2022; 5(1):19. https://doi.org/10.3390/psf2022005019

Chicago/Turabian Style

Meneghetti, Fábio C. C., Henrique K. Miyamoto, and Sueli I. R. Costa. 2022. "Information Properties of a Random Variable Decomposition through Lattices" Physical Sciences Forum 5, no. 1: 19. https://doi.org/10.3390/psf2022005019