2.1. Bayesian Information Geometry
In this subsection, we generalize the setting of the information geometry. Take a smooth family of volume forms with finite total volumes on
. We regard each of the volume forms as a point of a manifold
, namely, a point
presents a volume form
smoothly depending on
y. Let
be the space of volume forms with finite total volumes on
. We take a volume form
V in
. Given a point
z on
, we regard the value
of the density as a function
, and multiply the volume form
V by the function
. This defines the updating map
We notice that a volume form with finite total volume is proportional to a probability measure. Thus the function
is proportional to the likelihood, and System (
1) presents Bayes’ rule.
A proper subset
is called a (generalized) conjugate prior if it satisfies
Suppose that we have a conjugate prior
which is a smooth manifold, and further that, by using the hypersurface
, it can be written as
. We define on
the following “distance”
, which satisfies non of the axioms of distance.
Note that the restriction
satisfies the separation axiom, and is called the Kullback–Leibler divergence. We have to fix the coordinate in the
k-direction which presents the time. Then we write the quadratic term of the Taylor expansion of
as
, where
. Suppose that
is a metric on
. Let
be the Levi-Civita connection with respect to
. We write the cubic term of the expansion of
symmetrically as
. This defines the line of (generalized)
-connections
with affine parameter
, where
denotes the contraction
by the contravariant metric
. Note that
has no torsion. Restricting all of the above notions with tilde to the hypersurface
, we obtain the notions without tilde in the usual information geometry [
1]. Here
can be identified with a space
U of probability distributions.
2.2. The Geometry of Normal Distributions
In this subsection we consider the space
U of multivariate normal distributions. The pair of a vector
and an upper triangular matrix
with positive diagonal entries determines an
n-variate normal distribution by declaring that
presents the mean and
the Cholesky decomposition of the covariance matrix. We put
Note that
is unitriangular, i.e., it is a triangular matrix whose diagonal entries are all 1. Considering
and
as parameters, we can write the probability density of the
n-variate normal distribution at
as
Then the relative entropy defines the premetric
where
denotes the sum of squares (i.e.,
the Frobenius norm). Thus,
where
is the unit, and
the difference
. Let
be the entries of the inverse matrix of
. Then we have
The Fisher information
g appears in
as the quadratic form
which is presented by a block diagonal
, where
and
(
). Lowering the upper indices of the
-connection by
, we have
and
and thus we also have
The coefficients for the e-connection all vanish with respect to the natural parameter , where is the upper half of . Dually, the coefficients for the m-connection all vanish with respect to the expectation parameter , where is the upper half of . Now we fix the third component r of , and change the others. We take the natural projection and modify the coordinates on the fiber into in the next proposition.
Proposition 1. The fiber is an affine subspace of U with respect to the e-connection . It can be parametrized by affine parameters and , where and .
The fiber satisfies the following two properties.
Proposition 2. is closed under the convolution * and the normalized pointwise product · between the probability densities.
Proposition 3. The fiber with the induced metric from g admits a Kähler complex structure.
We write the restriction
of the premetric
D using the coordinates
as
We take the product
of two copies of the space
U. Then the products
of the fibers foliate
. We call this the primary foliation of
. For each
, we have the coordinate system
on the leaf
. From the Kähler forms
respectively, on
and
, we define the symplectic forms
on
. We fix their primitive 1-forms
The symplectic structures on the primary foliation defines a pair of regular Poisson structures.
Now we take the
-dimensional submanifolds
of the leaf
for
and
. The secondary foliation of
foliates any leaf
by the
-dimensional submanifolds
for
. The tertiary foliation of
foliates all leaves
of the secondary foliation by the
-dimensional submanifolds
for
. We take the hypersurface
which inherits the contact forms
. We can prove the following propositions.
Proposition 4. With respect to the Kähler form , the tertiary leaves are Lagrangian correspondences.
Proposition 5. For any ε and δ with , is a disjoint union of n-dimensional submanifolds which are integral submanifolds of the contact hyperplane distribution on N.
For each point
, we have the diffeomorphism
sending
to
with
. We put
where
. Then we have
For any
, we define the diffeomorphism
which preserves the 1-forms
. It is easy to prove
Proposition 6. In the case where for , the diffeomorphism preserves .
For each , we take the set , and consider it as a structure of the secondary leaf . Then we can prove
Proposition 7. For any , the diffeomorphism preserves the set for any . In the case where ζ satisfies , the diffeomorphism also preserves the hypersurface N.
Hereafter we fix
. For any
, the diffeomorphism
interchanges the operation
with the operation
Namely,
Proposition 8. If , then A curve is a geodesic with respect to the e-connection if and only if and are affine functions of t for .
Definition 1. We say that an e-geodesic is intensive if it admits an affine parametrization such that are linear for .
Note that any e-geodesic is intensive in the case where . We show
Proposition 9. Given an intensive e-geodesic , we can parametrize its image under the diffeomorphism to obtain an intensive e-geodesic.
We have the hypersurface carrying the contact forms . We state the main result.
Theorem 1. The contact Hamiltonian vector field X of the restriction of the function to the hypersurface N on any leaf of the primary foliation of with respect to the contact form on N coincides with that for the other contact form . The vector field X is tangent to the tertiary leaves and defines flows on them. Here each flow line presents a correspondence between intensive e-geodesics as is described in Proposition 9. Particularly, for and any , the flow on the leaf presents the iteration of the operation ∗ on the first factor of and that of the operation · on the second factor.
Finally, we consider the transverse unitriangular group. We have the orthonormal frame
with the relations
of the unitriangular algebra. Using the dual coframe
, the relations can be expressed as
. The transverse section of the primary foliation of
is the product of two copies of the unitriangular Lie group, which we would like to call the bi-unitriangular group. We fix the frame (resp. the coframe) of the transverse section consisting of the above
(resp.
) in the first factor
and their copies
(resp.
) in the second factor
. The quotient manifold carries the
-plectic structure
which satisfies
and
. We notice that, in the symplectic case where
, the quotient manifold admits no Kähler structure (see [
6]).