Geometry of Statistical Manifolds

Abstract

A statistical manifold M can be defined as a Riemannian manifold each of whose points is a probability distribution on the same support. In fact, statistical manifolds possess a richer geometric structure beyond the Fisher information metric defined on the tangent bundle $TM$. Recognizing that points in M are distributions and not just generic points in a manifold, $TM$ can be extended to a Hilbert bundle $HM$. This extension proves fundamental when we generalize the classical notion of a point estimate—a single point in M—to a function on M that characterizes the relationship between observed data and each distribution in M. The log likelihood and score functions are important examples of generalized estimators. In terms of a parameterization $\theta :M\to \mathrm{\Theta }\subset {\mathbb{R}}^k$, $\widehat{\theta }$ is a distribution on $\mathrm{\Theta }$ while its generalization $g_{\widehat{\theta }}=\widehat{\theta }-E\widehat{\theta }$ as an estimate is a function over $\mathrm{\Theta }$ that indicates inconsistency between the model and data. As an estimator, $g_{\widehat{\theta }}$ is a distribution of functions. Geometric properties of these functions describe statistical properties of $g_{\widehat{\theta }}$. In particular, the expected slopes of $g_{\widehat{\theta }}$ are used to define $\mathrm{\Lambda }\left(g_{\widehat{\theta }}\right)$, the $\mathrm{\Lambda }$-information of $g_{\widehat{\theta }}$. The Fisher information I is an upper bound for the $\mathrm{\Lambda }$-information: for all g, $\mathrm{\Lambda }\left(g\right)\le I$. We demonstrate the utility of this geometric perspective using the two-sample problem.

1. Introduction

Statistical manifolds provide a geometric framework for understanding families of probability distributions. While traditionally defined as Riemannian manifolds equipped with the Fisher information metric, their structure extends beyond this basic framework. Lauritzen [1] identified an additional skewness tensor, and Amari [2] also noticed this additional structure which he used to define a family of connections including both the metric connection and a dual pair—the mixture and exponential connections. This duality, first observed by Efron [3], reveals geometric structure beyond the Riemannian setting, though this previous work remained confined to the tangent bundle.

Amari [4] introduced a Hilbert space extension of the tangent bundle which Amari and Kumon [5] applied to estimating functions. Kass and Vos (Section 10.3) [6] also describe statistical Hilbert bundles which Pistone [7] extends to other statistical bundles in the nonparametric setting where extra care is required when the sample space is not finite. Recent developments have expanded the geometric perspective on the role of the Hilbert bundle in parametric inference when the traditional approach to statistical inference is replaced with Fisher’s view of estimation.

Classical statistical inference separates estimation and hypothesis testing into distinct frameworks. Point estimators map from the sample space to the parameter space, with their local properties described through the tangent bundle. Test statistics similarly rely on tangent bundle geometry. The log likelihood and its derivative, the score function, bridge these approaches by providing both estimation methods (maximum likelihood) and testing procedures (likelihood ratio and score tests). Godambe [8] extended the score’s role in estimation through estimating equations, yet the fundamental separation between testing and estimation persisted.

Building on Fisher’s [9] conception of estimation as a continuum of hypothesis tests, Vos [10] unified these approaches by replacing point estimators with generalized estimators—functions on the parameter space that geometrically represent surfaces over the manifold. These generalized estimators shift the inferential focus from individual parameter values to entire functions, whose properties are naturally characterized within the Hilbert bundle framework.

This paper demonstrates the advantages of generalized estimators and the utility of the Hilbert bundle perspective specifically for the two-sample problem. We show how the orthogonalized score achieves information bounds as a consequence of its membership in the tangent bundle, while other generalized estimators, residing only in the larger Hilbert bundle, suffer information loss measured by their angular deviation from the tangent space.

2. Statistical Manifolds

Let $M^{\mathcal{X}}$ be a family of probability measures with common support $\mathcal{X}$. While $\mathcal{X}$ can be an abstract space, for most applications, $\mathcal{X}\subset {\mathbb{R}}^d$. Each point in $M^{\mathcal{X}}$ represents a candidate model for a population whose individuals take values in $\mathcal{X}$.

We consider inference based on a sample denoted by y, with corresponding sample space $\mathcal{Y}$. The relationship between $\mathcal{X}$ and $\mathcal{Y}$ depends on three factors: the sampling plan, any conditioning applied, and dimension reduction through sufficient statistics. In the simplest case—a simple random sample of size n without conditioning or dimension reduction—we have $\mathcal{Y}={\mathcal{X}}^n$.

Let $M=M^{\mathcal{Y}}$ denote the family of probability measures on $\mathcal{Y}$ induced by $M^{\mathcal{X}}$ through the sampling plan. For the simple random sampling case:

$$M=\left\{m:m\left(y\right)=\prod _{i=1}^n{m}^{\mathcal{X}}\left(x_i\right),m^{\mathcal{X}}\in M^{\mathcal{X}}\right\}.$$

For any real-valued measurable function h, we define its expected value at $m\in M$ as

$$E_{m}h=\left\{\begin{array}{cc}{\int }_{\mathcal{Y}}h\left(y\right)m\left(y\right)d\mu \hfill & \mathrm{when}\mathcal{Y}\mathrm{is}\mathrm{continuous}\hfill \\ \sum _{y\in \mathcal{Y}}h\left(y\right)m\left(y\right)\hfill & \mathrm{when}\mathcal{Y}\mathrm{is}\mathrm{discrete}\hfill \end{array}\right.$$

The Hilbert space associated with M consists of all square-integrable functions:

$$H_M=\left\{h:E_m{h}^2<\infty ,\forall m\in M\right\}.$$

This space carries a family of inner products indexed by points in M:

$${⟨h,h^{\prime }⟩}_m=E_m\left(h{h}^{\prime }\right)\mathrm{for}\mathrm{all}h,h^{\prime }\in H_M.$$

When ${⟨h,h^{\prime }⟩}_m=0$, we say that h and $h^{\prime }$ are m-orthogonal and write $h{\perp }_m{h}^{\prime }$.

We construct the Hilbert bundle over M by associating a copy of $H_M$ to each point:

$$HM=M\times H_M.$$

The fiber at m, denoted $H_m$ or $H_{m}M$, inherits the inner product ${⟨·,·⟩}_m$. For inference purposes, we decompose each fiber into the space of constant functions and its orthogonal complement:

$$H_m=H_m^{\perp }\oplus H_m^0\mathrm{where}{H}_m^{\perp }{\perp }_m{H}_m^0.$$

(1)

Here, $H_m^{\perp }=\{h\in H_m:E_{m}h=0\}$ consists of centered functions, while $H_m^0$ contains the constants. Note that $E_{m}h={⟨h,1⟩}_m$ and $H_m^0$ is independent of m. As decomposition (1) holds fiberwise, we obtain a global decomposition:

$$HM=H^{\perp }M\oplus H^{0}M\mathrm{where}{H}^{\perp }M\perp H^{0}M.$$

(2)

The bundle $HM$ extends the tangent bundle $TM$, which emerges naturally through parameterization. We assume that M admits a global parameterization—while not strictly necessary, this simplifies our exposition by avoiding coordinate charts. We require this parameterization to be a diffeomorphism.

Consider a parameterization $\theta :M\to \mathrm{\Theta }\subset {\mathbb{R}}^d$ with inverse ${\theta }^{-1}:\mathrm{\Theta }\to M$. For a specific distribution $m_{\circ }\in M$, we write ${\theta }_{\circ }=\theta \left(m_{\circ }\right)$ for its parameter value. When considering all distributions simultaneously, we write $\theta =\theta \left(m\right)$, where context distinguishes between $\theta$ as a point in $\mathrm{\Theta }$ (left side) and as a function (right side).

For notational convenience, we denote the distribution corresponding to parameter value ${\theta }_{\circ }$ as $m_{{\theta }_{\circ }}={\theta }^{-1}\left({\theta }_{\circ }\right)$. This allows us to write the following:

$$m_{\theta }={\theta }^{-1}\left(\theta \right)$$

where, again, context clarifies whether $m_{\theta }$ refers to the function ${\theta }^{-1}$ or its value.

With this parameterization, the Hilbert bundle can be expressed as

$$HM=\mathrm{\Theta }\times H_M$$

allowing us to index fibers by parameter values: $H_{\theta }M=H_{m_{\theta }}M$.

The log likelihood function plays a fundamental role in our geometric framework. On M, it is the function ${\ell }_M:\mathcal{Y}\times M\to \mathbb{R}$ defined by ${\ell }_M(y,m)=logm\left(y\right)$. Through the parameterization, this induces ${\ell }_{\mathrm{\Theta }}:\mathcal{Y}\times \mathrm{\Theta }\to \mathbb{R}$ given by ${\ell }_{\mathrm{\Theta }}(y,\theta )=log{m}_{\theta }\left(y\right)$. When the parameterization is clear from context, we simply write ℓ for ${\ell }_{\mathrm{\Theta }}$.

The partial derivatives of ℓ with respect to the parameters,

$$\left\{{\displaystyle \frac{\partial \ell }{\partial {\theta }^1}},{\displaystyle \frac{\partial \ell }{\partial {\theta }^2}},\dots ,{\displaystyle \frac{\partial \ell }{\partial {\theta }^d}}\right\},$$

evaluated at ${\theta }_{\circ }$, form a basis for the tangent space $T_{{\theta }_{\circ }}M$. For all $i=1,\dots ,d$ and all $m\in M$, $0<E_m{(\partial \ell /\partial {\theta }^i)}^2<\infty$, ensuring that $TM\subset HM$. In fact, $TM\subset H^{\perp }M$ as $E_m(\partial \ell /\partial {\theta }^i)$ vanishes on M.

3. Functions on $\mathbf{M}$

The log likelihood function ℓ and its derivatives are central to statistical inference. Traditionally, these serve as tools to find point estimates—particularly the maximum likelihood estimate (MLE)—and to characterize the estimator’s properties. We adopt a different perspective: we treat ℓ and its derivatives as primary inferential objects rather than mere computational tools. This approach aligns with Fisher’s conception of estimation as a continuum of tests.

As the log likelihood ratio for comparing models with parameters ${\theta }_1$ and ${\theta }_2$ is the difference $log{m}_{{\theta }_1}\left(y\right)-log{m}_{{\theta }_2}\left(y\right)$ and adding an arbitrary constant to each term does not affect this difference, we define the log likelihood so that $\ell \le 0$ for each fixed y. Thus we work with

$$\ell (y,\theta )=log{m}_{\theta }\left(y\right)-\underset{m\in M}{sup}logm\left(y\right).$$

As an inferential function, $|\ell (y,\theta \left)\right|$ quantifies the dissonance between observation y and distribution $m_{\theta }$. While the MLE set $\{\theta :\ell (y,\theta )=0\}$ identifies parameters with minimal dissonance, our emphasis shifts to characterizing the full landscape of dissonance across the manifold. While “dissonance” lacks a precise mathematical definition, it can be thought of as the evidence in y against the model at $\theta$—essentially, a test statistic evaluated at y for the null hypothesis specifying $m_{\theta }$. We use the notation $\widehat{\theta }$ for the MLE when it is unique. When the MLE set is empty we say that the MLE does not exist. Note that ℓ is defined even when the MLE does not exist or is not unique; the only requirement is that ${sup}_{m}logm\left(y\right)<\infty$.

The log likelihood exemplifies a broader class of generalized estimators: functions $G:\mathcal{Y}\times \mathrm{\Theta }\to \mathbb{R}$ where, for almost every $y\in \mathcal{Y}$, the function $G(y,·)$ measures dissonance between y and distributions across M. Like ℓ, we can normalize G so that $G\le 0$.

Consider the geometric interpretation. For a function $f:\mathrm{\Theta }\to \mathbb{R}$, let $\gimel \left(f\right)=\left\{\right(\theta ,f\left(\theta \right)):\theta \in \mathrm{\Theta }\}\subset \mathrm{\Theta }\times \mathbb{R}$ denote its graph. The graphs $\gimel (-\ell (y,·\left)\right)$ and $\gimel (-G(y,·\left)\right)$ form d-dimensional surfaces over $\mathrm{\Theta }\subset {\mathbb{R}}^d$. We compare these surfaces through their gradients:

$$s(y,\theta )=\nabla \ell (y,\theta )\mathrm{and}g(y,\theta )=\nabla G(y,\theta )$$

where $\nabla ={(\partial /\partial {\theta }^1,\dots ,\partial /\partial {\theta }^d)}^{\mathrm{t}}$.

Viewing these as estimators requires replacing the fixed observation y with the random variable Y. Then $\gimel (-\ell (Y,·\left)\right)$ and $\gimel (-G(Y,·\left)\right)$ become random surfaces, while $s(Y,\theta )$ and $g(Y,\theta )$ become random gradient fields. The score components ${\left\{s^i(Y,\theta )\right\}}_{i=1}^d$ span the tangent space: $\mathrm{span}\left\{s(Y,\theta )\right\}=T_{\theta }M$. The key difference between generalized estimation described in this paper and the estimating equations of Godambe and Thompson [11] lies in their inferential approach: the former focuses on the distribution of graph slopes (gradients in a linear space), while the latter examines the distribution of where graphs intersect the horizontal axis (roots of g).

Under mild regularity conditions, the components of $g(Y,\theta )$ span a subspace of $H_{\theta }M$ of dimension d although generally not $T_{\theta }M$. Strictly speaking, $\mathrm{span}\left\{s\right(Y,\theta \left)\right\}$ is isomorphic rather than equal to $T_{\theta }M$, as the former consists of vectors attached to the surface $\gimel (-\ell (y,·\left)\right)$ while the latter are attached to M (equivalently, to $\mathrm{\Theta }$). As shown in Vos and Wu [12], this precise relationship between the log likelihood surface and the manifold ensures that score-based estimators attain the information bound.

This perspective fundamentally shifts our focus. Rather than comparing point estimators through their variance or mean squared error on the parameter space $\mathrm{\Theta }$, we compare the linear spaces spanned by the components of generalized estimators within the Hilbert bundle $HM$.

For point estimator $\check{\theta }$, define its associated generalized estimator:

$$g_{\check{\theta }}=\check{\theta }-E\check{\theta }.$$

The estimator must have nonzero variance, $0<V\left(\check{\theta }\right)<\infty$ for all $\theta \in \mathrm{\Theta }$, so that $\check{\theta }\in H_M$. Instead of traditional comparisons between $\widehat{\theta }$ (the MLE) and $\check{\theta }$, we compare the spaces spanned by $s=s(Y,\theta )$ and $g_{\check{\theta }}=g_{\check{\theta }}(Y,\theta )$ through their $\mathrm{\Lambda }$-information—a generalization of Fisher information to arbitrary generalized estimators. Geometrically, the relationship between s and $g_{\check{\theta }}$ is characterized by angles between their component vectors. Statistically, this translates to correlations between the corresponding random variables. The $\mathrm{\Lambda }$-information is defined by the left-hand side of Equation (13) which also shows the role of the correlation.

Generalized estimators offer particular advantages when nuisance parameters are present. For point estimators, one seeks a parameterization where nuisance and interest parameters are orthogonal—a goal not always achievable. When working in $HM$ rather than $\mathrm{\Theta }$, orthogonalization remains important but becomes more flexible: the choice of nuisance parameterization becomes immaterial as orthogonalization occurs within $HM$ itself.

The information bound for the interest parameter is attained by restricting generalized estimators to be orthogonal to the nuisance parameter’s score components. The general framework is developed in Vos and Wu [12]; we illustrate the approach through the special case for comparing two populations in the following section.

4. Comparing Two Populations

We now develop the general framework for comparing two distributions from the same parametric family. The next section applies this framework to the more specific case of $2\times 2$ contingency tables.

Let $M^{\mathcal{X}}$ be a one-parameter family of distributions on $\mathcal{X}\subset \mathbb{R}$, and let $M^{\mathcal{Y}}$ be the corresponding family of sampling distributions on $\mathcal{Y}$. While we work primarily with sampling distributions in $M^{\mathcal{Y}}$, we use superscripts to distinguish when necessary: $m^{\mathcal{Y}}$ denotes a sampling distribution obtained from population distribution $m^{\mathcal{X}}$.

For simple random sampling outside exponential families, $\mathcal{Y}={\mathcal{X}}^n$. Within exponential families, $\mathcal{Y}$ represents the support of the sufficient statistic. For example, when $M^{\mathcal{X}}$ consists of Bernoulli distributions with success probability $p\in (0,1)$, the family $M^{\mathcal{Y}}$ consists of binomial distributions for n trials with sample space $\mathcal{Y}=\{0,1,2,\dots ,n\}$.

Let ${\theta }_{\mathcal{Y}}:M^{\mathcal{Y}}\to \mathrm{\Theta }\subset \mathbb{R}$ parameterize $M^{\mathcal{Y}}$. We define the population parameterization ${\theta }_{\mathcal{X}}$ to ensure consistency: ${\theta }_{\mathcal{Y}}^{-1}\circ {\theta }_{\mathcal{X}}\left(m^{\mathcal{X}}\right)=m^{\mathcal{Y}}$. Thus, each parameter value simultaneously labels both a population distribution in $M^{\mathcal{X}}$ and its corresponding sampling distribution in $M^{\mathcal{Y}}$. As our focus is on sampling distributions, we simplify notation by dropping the subscript $\theta ={\theta }_{\mathcal{Y}}$.

The score function for parameter $\theta$ is

$${\ell }_{/\theta }={\displaystyle \frac{\partial \ell }{\partial \theta }}=\alpha Z$$

where we factor the score into its magnitude $\alpha =\sqrt{V\left({\ell }_{/\theta }\right)}$ and its standardized version Z with $EZ=0$ and $V\left(Z\right)=1$. Both $\alpha$ and Z depend on $\theta$ and, thus, vary across $M^{\mathcal{Y}}$.

Under reparameterization $\xi$ of $\theta$, the standardized score Z remains invariant while the coefficient transforms as $\alpha \partial \theta /\partial \xi$. The coefficient $\alpha$ equals the square root of the total Fisher information: $\alpha =\sqrt{I_{\theta }}=\sqrt{n{i}_{\theta }}$ where $i_{\theta }$ is the Fisher information per observation. For the binomial family, $Z=(Y-np)/\sqrt{np(1-p)}$.

Now consider independent samples of sizes $n_1$ and $n_2$ from two distributions in $M^{\mathcal{X}}$. The manifold of joint population distributions is

$$M^{\mathcal{X}\times \mathcal{X}}=\left\{m=m_1{m}_2:m_1,m_2\in M^{\mathcal{X}}\right\}$$

with corresponding manifold of joint sampling distributions:

$$M^{{\mathcal{Y}}_1\times {\mathcal{Y}}_2}=\left\{m=m_1{m}_2:m_1\in M^{{\mathcal{Y}}_1},m_2\in M^{{\mathcal{Y}}_2}\right\}.$$

The parameterization ${\theta }_{\mathcal{X}}$ of $M^{\mathcal{X}}$ induces natural parameterizations ${\theta }_{\mathcal{X}\times \mathcal{X}}={({\theta }_{\mathcal{X}},{\theta }_{\mathcal{X}})}^{\mathrm{t}}$ on $M^{\mathcal{X}\times \mathcal{X}}$ and ${\theta }_{{\mathcal{Y}}_1\times {\mathcal{Y}}_2}={({\theta }_{{\mathcal{Y}}_1},{\theta }_{{\mathcal{Y}}_2})}^{\mathrm{t}}$ on $M^{{\mathcal{Y}}_1\times {\mathcal{Y}}_2}$. These share the same image ${\mathrm{\Theta }}_1\times {\mathrm{\Theta }}_2$ where ${\mathrm{\Theta }}_1={\mathrm{\Theta }}_2={\theta }_{\mathcal{X}}\left(M^{\mathcal{X}}\right)$. Setting $M=M^{{\mathcal{Y}}_1\times {\mathcal{Y}}_2}$ and $\theta ={({\theta }_1,{\theta }_2)}^{\mathrm{t}}$, each point in $\mathrm{\Theta }$ labels both a joint sampling distribution and its generating population distribution.

The hypothesis that both samples arise from the same distribution corresponds to the diagonal submanifold:

$$M_{diag}^{\mathcal{X}\times \mathcal{X}}=\left\{m=m_1{m}_2:m_1=m_2\in M^{\mathcal{X}}\right\}$$

with parameter space:

$${\mathrm{\Theta }}_{diag}=\left\{{({\theta }_1,{\theta }_2)}^{\mathrm{t}}\in \mathrm{\Theta }:{\theta }_1={\theta }_2\right\}.$$

The joint parameter $\theta ={({\theta }_1,{\theta }_2)}^{\mathrm{t}}$ yields two score functions:

$${\ell }_{/{\theta }_1}={\alpha }_1{Z}_1\mathrm{and}{\ell }_{/{\theta }_2}={\alpha }_2{Z}_2$$

where $Z_1$ and $Z_2$ are orthonormal at each $m\in M$:

$${⟨Z_1,Z_2⟩}_m=E_m\left(Z_1{Z}_2\right)=0.$$

To compare the distributions, we reparameterize using the difference $\delta ={\theta }_1-{\theta }_2$ as our interest parameter and $\tau ={\theta }_1+{\theta }_2$ as the nuisance parameter. The inverse transformation gives ${\theta }_1={\displaystyle \frac{1}{2}}(\tau +\delta )$ and ${\theta }_2={\displaystyle \frac{1}{2}}(\tau -\delta )$, yielding scores:

$$\begin{array}{cc}\hfill {\ell }_{/\delta }& ={\displaystyle \frac{\partial {\theta }_1}{\partial \delta }}{\ell }_{/{\theta }_1}+{\displaystyle \frac{\partial {\theta }_2}{\partial \delta }}{\ell }_{/{\theta }_2}={\displaystyle \frac{1}{2}}{\alpha }_1{Z}_1-{\displaystyle \frac{1}{2}}{\alpha }_2{Z}_2\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill {\ell }_{/\tau }& ={\displaystyle \frac{\partial {\theta }_1}{\partial \tau }}{\ell }_{/{\theta }_1}+{\displaystyle \frac{\partial {\theta }_2}{\partial \tau }}{\ell }_{/{\theta }_2}={\displaystyle \frac{1}{2}}{\alpha }_1{Z}_1+{\displaystyle \frac{1}{2}}{\alpha }_2{Z}_2.\hfill \end{array}$$

(4)

Let $Z_{\nu }$ denote the unit vector in the direction of ${\ell }_{/\tau }$, satisfying $⟨Z_{\nu },{\ell }_{/\tau }⟩>0$, $E{Z}_{\nu }=0$, and $|Z_{\nu }|=1$. As $Z_{\nu }={\ell }_{/\tau }/\left|{\ell }_{/\tau }\right|$ remains invariant under monotonic reparameterizations of $\tau$, we use the subscript $\nu$ (for nuisance). In terms of the basis $\{Z_1,Z_2\}$:

$$Z_{\nu }={\displaystyle \frac{{\alpha }_1{Z}_1+{\alpha }_2{Z}_2}{\sqrt{{\alpha }_1^2+{\alpha }_2^2}}}={\displaystyle \frac{I_{{\theta }_1}^{1/2}{Z}_1+I_{{\theta }_2}^{1/2}{Z}_2}{\sqrt{I_{{\theta }_1}+I_{{\theta }_2}}}}.$$

Let h be a point estimator or test statistic for $\delta$. The function h is a generalized pre-estimator provided $h-Eh$ is a generalized estimator. For any pre-estimator h of $\delta$, define its orthogonalized version:

$$h^{\perp }=(h-Eh)-⟨h,Z_{\nu }⟩Z_{\nu }=\left|h\right|(Z_h-{\rho }_{h\nu }{Z}_{\nu })$$

where $Z_h=(h-Eh)/\left|h\right|$ is the standardized direction and ${\rho }_{h\nu }=⟨Z_h,Z_{\nu }⟩$ is the correlation with the nuisance direction.

To ensure that inference is independent of the nuisance parameter, we work with orthogonalized generalized estimators $g=h^{\perp }$:

$$\begin{array}{ccc}\hfill g& =& \left|h\right|(Z_h-{\rho }_{h\nu }{Z}_{\nu })\hfill \\ & =& \left|g\right|Z_g=\sqrt{1-{\rho }_{h\nu }^2}\left|h\right|Z_g.\hfill \end{array}$$

(5)

When h is the score for $\delta$, the orthogonalized score becomes

$$s^{\perp }=\sqrt{I_{\delta }^{\perp }}{Z}_{s^{\perp }}=\sqrt{(1-{\rho }_{s\nu }^2)I_{\delta }}{Z}_{s^{\perp }}$$

(6)

where $I_{\delta }^{\perp }$ is the information after orthogonalization. The proportion of information loss due to the nuisance parameter is the square of the correlation between the interest and nuisance parameters

$${\rho }_{s\nu }^2={\displaystyle \frac{I_{\delta }-I_{\delta }^{\perp }}{I_{\delta }}}.$$

(7)

This loss cannot be recovered by reparameterization. Geometrically, ${\rho }_{s\nu }=cos\angle (Z_s,Z_{\nu })$, so the proportional information loss equals the squared cosine of the angle between the score and the tangent space of $M_{{\delta }_{\circ }}={\delta }^{-1}\left({\delta }_{\circ }\right)$. The submanifold $M_{{\delta }_{\circ }}$ depends on the choice of interest parameter and is integral to the inference problem.

The orthogonalized Fisher information $I_{\delta }^{\perp }$ is additive on the reciprocal scale:

$${\left(I_{\delta }^{\perp }\right)}^{-1}=I_{{\theta }_1}^{-1}+I_{{\theta }_2}^{-1}.$$

(8)

Equation (8) is established as follows. The orthogonalized score is a linear combination of the orthonormal basis vectors $Z_1$ and $Z_2$,

$$\begin{array}{ccc}\hfill s^{\perp }& =& {\ell }_{/\delta }-{\displaystyle \frac{⟨{\ell }_{/\delta },{\ell }_{/\tau }⟩}{⟨{\ell }_{/\tau },{\ell }_{/\tau }⟩}}{\ell }_{/\tau }\hfill \\ & =& \left({\displaystyle \frac{1}{2}}{\alpha }_1{Z}_1-{\displaystyle \frac{1}{2}}{\alpha }_2{Z}_2\right)-\left({\displaystyle \frac{{\alpha }_1^2-{\alpha }_2^2}{{\alpha }_1^2+{\alpha }_2^2}}\right)\left({\displaystyle \frac{1}{2}}{\alpha }_1{Z}_1+{\displaystyle \frac{1}{2}}{\alpha }_2{Z}_2\right)\hfill \\ & =& {\displaystyle \frac{{\alpha }_1{\alpha }_2}{{\alpha }_1^2+{\alpha }_2^2}}\left({\alpha }_2{Z}_1-{\alpha }_1{Z}_2\right)\hfill \\ & =& {\displaystyle \frac{{\alpha }_1{\alpha }_2}{\sqrt{{\alpha }_1^2+{\alpha }_2^2}}}{Z}_{s^{\perp }}.\hfill \\ \hfill \end{array}$$

(9)

As $|s^{\perp }{|}^2=I_{\delta }^{\perp }$ and ${\alpha }_i^2=I_{{\theta }_i}$:

$$I_{\delta }^{\perp }={\displaystyle \frac{{\alpha }_1^2{\alpha }_2^2}{{\alpha }_1^2+{\alpha }_2^2}}={\displaystyle \frac{I_{{\theta }_1}{I}_{{\theta }_2}}{I_{{\theta }_1}+I_{{\theta }_2}}}$$

(10)

and taking the reciprocal of both sides of (10) gives (8). Substituting into (7) with $I_{\delta }={\displaystyle \frac{1}{4}}(I_{{\theta }_1}+I_{{\theta }_2})$ shows

$${\rho }_{s\nu }^2={\displaystyle \frac{{(I_{{\theta }_1}-I_{{\theta }_2})}^2}{I_{{\theta }_1}+I_{{\theta }_2}}}$$

which means that the information loss due to the nuisance parameter is proportional to the squared difference in the Fisher information for the distributions being compared. Using Equation (9), the orthogonalized score in terms of the basis vectors $Z_1$ and $Z_2$ is

$$\begin{array}{cc}\hfill s^{\perp }& =I_{\delta }^{\perp }\left({\displaystyle \frac{Z_1}{\sqrt{I_{{\theta }_1}}}}-{\displaystyle \frac{Z_2}{\sqrt{I_{{\theta }_2}}}}\right).\hfill \end{array}$$

The basis $\{Z_{s^{\perp }},Z_{\nu }\}$ is obtained from $\{Z_1,Z_2\}$ using the linear transformation

$$\left(\begin{array}{c}{Z}_{s^{\perp }}\\ Z_{\nu }\end{array}\right)={\displaystyle \frac{1}{\sqrt{{\alpha }_1^2+{\alpha }_2^2}}}\left(\begin{array}{cc}{\alpha }_2& -{\alpha }_1\\ {\alpha }_1& {\alpha }_2\end{array}\right)\left(\begin{array}{c}{Z}_1\\ Z_2\end{array}\right)$$

(11)

which is a rotation through an angle of ${cos}^{-1}\sqrt{I_{{\theta }_2}/(I_{{\theta }_1}+I_{{\theta }_2})}$. When $\theta$ is a location parameter, $I_{{\theta }_1}$ and $I_{{\theta }_2}$ are constant on M. With equal sample sizes ($n_1=n_2$), the rotation angle is $\pi /4$ and $Z_{s^{\perp }}\propto (Z_1-Z_2)$.

While $Z_{s^{\perp }}\in TM$, for general estimators g we have $Z_g\in HM$ but $Z_g\notin TM$ unless $g=s^{\perp }$. This distinction explains why general estimators fail to achieve the information bound. The $\mathrm{\Lambda }$-information of g is $\mathrm{\Lambda }\left(g\right)={\rho }_{g{s}^{\perp }}^2{I}_{\delta }^{\perp }$, where ${\rho }_{g{s}^{\perp }}$ is the correlation between $Z_g$ and $Z_{s^{\perp }}$.

The null hypothesis $H_0:\delta =0$ deserves special attention. While $H:\delta ={\delta }_{\circ }$ generally depends on the parameterization choice, $H_0:\delta =0$ is parameterization-invariant as it is equivalent to $H_0:{\theta }_1={\theta }_2$. Under simple random sampling with $I_{\theta }=n{I}_{\theta }$, the standardized orthogonalized score on $M_0={\delta }^{-1}\left(0\right)$ becomes

$$Z_{s^{\perp }}={\displaystyle \frac{n_2^{1/2}{Z}_1-n_1^{1/2}{Z}_2}{\sqrt{n_1+n_2}}}$$

which is invariant across all parameterizations $\theta$. This invariance does not hold for test statistics based on point estimators like ${\widehat{\theta }}_1-{\widehat{\theta }}_2$, whose form depends on whether we parameterize using proportions, log-proportions, or log-odds.

5. Comparing Two Bernoulli Distributions

We now specialize the general framework to comparing two Bernoulli distributions, establishing the geometric structure that underlies inference for $2\times 2$ contingency tables.

For the Bernoulli sample space $\mathcal{X}=\{0,1\}$, the manifold of population distributions is

$$M^{\mathcal{X}}=\{m:0<m\left(1\right)<1,m\left(0\right)+m\left(1\right)=1\}$$

with natural parameterization $p_{\mathcal{X}}\left(m\right)=m\left(1\right)$. For a sample of size n, the sufficient statistic has support $\mathcal{Y}=\{0,1,\dots ,n\}$, yielding the manifold of binomial sampling distributions:

$$M^{\mathcal{Y}}=\left\{m:m\left(y\right)=\left({\displaystyle \genfrac{}{}{0pt}{}{n}{y}}\right)p^y{(1-p)}^{n-y},0<p<1\right\}.$$

A natural bijection exists between $M^{\mathcal{X}}$ and $M^{\mathcal{Y}}$: each population distribution determines a unique sampling distribution. We define $p_{\mathcal{Y}}$ to make this bijection $p_{\mathcal{Y}}^{-1}\circ p_{\mathcal{X}}$. Similarly, for any alternative parameterization ${\theta }_{\mathcal{X}}$ (such as ${\theta }_{\mathcal{X}}\left(m\right)=log(m\left(1\right)/m\left(0\right))$), we define ${\theta }_{\mathcal{Y}}$ so that the bijection equals ${\theta }_{\mathcal{Y}}^{-1}\circ {\theta }_{\mathcal{X}}$.

For independent samples of sizes $n_1$ and $n_2$, the joint manifolds are

$$\begin{array}{cc}\hfill M^{\mathcal{X}\times \mathcal{X}}& =\{m=m_1{m}_2:m_1,m_2\in M^{\mathcal{X}}\}\hfill \\ \hfill M=M^{{\mathcal{Y}}_1\times {\mathcal{Y}}_2}& =\{m=m_1{m}_2:m_1\in M^{{\mathcal{Y}}_1},m_2\in M^{{\mathcal{Y}}_2}\}.\hfill \end{array}$$

Using the proportion parameterization $p={(p_1,p_2)}^{\mathtt{t}}\in {(0,1)}^2$, the sampling distribution at p is

$$m(y_1,y_2)=\left({\displaystyle \genfrac{}{}{0pt}{}{n_1}{y_1}}\right)\left({\displaystyle \genfrac{}{}{0pt}{}{n_2}{y_2}}\right)p_1^{y_1}{(1-p_1)}^{n_1-y_1}{p}_2^{y_2}{(1-p_2)}^{n_2-y_2}$$

for ${(y_1,y_2)}^{\mathtt{t}}\in {\mathcal{Y}}_1\times {\mathcal{Y}}_2$, with corresponding population distribution:

$$m_{\mathcal{X}\times \mathcal{X}}(x_1,x_2)=p_1^{x_1}{(1-p_1)}^{1-x_1}{p}_2^{x_2}{(1-p_2)}^{1-x_2}$$

for ${(x_1,x_2)}^{\mathtt{t}}\in {\{0,1\}}^2$.

The Hilbert space for this manifold consists of all real-valued functions on the finite sample space:

$$H_M=\left\{h:{\mathcal{Y}}_1\times {\mathcal{Y}}_2\to \mathbb{R}:\sum _{y_1,y_2}{h}^2(y_1,y_2)m(y_1,y_2)<\infty ,\forall m\in M\right\}.$$

(12)

As the support is finite, $H_M$ includes all finite-valued functions. The tangent space at m is the two-dimensional subspace:

$$T_{m}M=\mathrm{span}\{Y_1-n_1{p}_1,Y_2-n_2{p}_2\}$$

where $p_1=p_1\left(m\right)$ and $p_2=p_2\left(m\right)$.

Table 1 summarizes the Fisher information per observation for three common parameterizations of the Bernoulli distribution, each offering different advantages for inference.

Table 1. Fisher information per observation, $i_{\theta }$, for common parameterizations of the Bernoulli distribution expressed in the success parameter p.

Parameter $\mathit{\theta }$	Information ${\mathit{i}}_{\mathit{\theta }}$ per Observation
$log\left({\displaystyle \frac{p}{1-p}}\right)$	$p(1-p)$
p	${\displaystyle \frac{1}{p(1-p)}}$
$logp$	${\displaystyle \frac{1-p}{p}}$

We illustrate our geometric framework using data from Mendenhall et al. [13], who conducted a retrospective analysis of laryngeal carcinoma treatment. Disease was controlled in 18 of 23 patients treated with surgery alone and 11 of 19 patients treated with irradiation alone ($y_1=18$, $n_1=23$, $y_2=11$, $n_2=19$). We use this data to compare the orthogonalized score $s^{\perp }$ with other generalized estimators when the interest parameter is the log odds ratio $\delta =log(p_1/(1-p_1))-log(p_2/(1-p_2))$.

5.1. Orthogonalized Score

The score has two key properties: at each point in the sample space ${\ell }_{/\delta }$ is a smooth function on the parameter space, and at each each point in the manifold ${\ell }_{/\delta }$ is a distribution on the sample space. Formally, for y fixed ${\ell }_{/\delta }={\ell }_{/\delta }(y,·)\in C^1(\Delta )$ and for $\delta$ fixed ${\ell }_{/\delta }={\ell }_{/\delta }(·,\delta )\in H_{m}M$ when there is no nuisance parameter. As $\delta$ is the interest parameter we use the notation s for the score ${\ell }_{/\delta }$. These properties persist after orthogonalization and standardization to obtain $s^{\perp }$ and $Z_{s^{\perp }}$.

Figure 1 illustrates these properties for $Z_{s^{\perp }}$ using the cancer data. The black curve shows $Z_{s^{\perp }}$ evaluated at the observed sample $(y_1=18,y_2=11)$ as a function of $\delta$, with the nuisance parameter fixed at ${\xi }_{\circ }=29$. Each of the 480 points in the sample space ${\mathcal{Y}}_1\times {\mathcal{Y}}_2$ generates such a curve; two additional examples appear in gray. We distinguish the family of curves $Z_{s^{\perp }}$ (uppercase) from the specific observed curve $z_{s^{\perp }}$ (lowercase).

Figure 1. Cancer data—orthogonalized score. Standardized orthogonal score $z_{s^{\perp }}$ as a function of the log odds difference $\delta$. The observed data (18 of 23 surgery successes, 11 of 19 irradiation successes) yields the black curve. Two additional sample points shown in gray illustrate the family of 480 possible curves. The nuisance parameter is fixed at $\xi =n_1{p}_1+n_2{p}_2=29$. Horizontal lines at $z=\pm 2$ intersect the observed curve to yield an approximate 95% confidence interval for $\delta$.

For any fixed ${\delta }_{\circ }$, the vertical line $\delta ={\delta }_{\circ }$ intersects all 480 curves, yielding a distribution of $Z_{s^{\perp }}$ values. Together with the probability mass function $m_{{\delta }_{\circ },{\xi }_{\circ }}$, this defines the sampling distribution of $Z_{s^{\perp }}$ when $\delta ={\delta }_{\circ }$ and ${\xi }_{\circ }=29$. Crucially, every such vertical distribution has mean zero and variance one, reflecting the standardization of the score.

The intersection of horizontal lines with $z_{s^{\perp }}$ provides confidence intervals through inversion. The lines $z=\pm 2$ intersect the observed curve at points $({\delta }_{lo},2)$ and $({\delta }_{hi},-2)$, partitioning the parameter space into three regions:

• For $\delta \le {\delta }_{lo}$: the observed $s^{\perp }$ exceeds 2 standard deviations above its expectation.
• For ${\delta }_{lo}<\delta <{\delta }_{hi}$: the observed $s^{\perp }$ lies within 2 standard deviations.
• For $\delta \ge {\delta }_{hi}$: the observed $s^{\perp }$ falls below −2 standard deviations.

The interval $({\delta }_{lo},{\delta }_{hi})$ forms an approximate 95% confidence interval for $\delta$. The approximation quality depends on the normality of the vertical distributions, while the interval width depends on the slope of $z_{s^{\perp }}$—steeper slopes yield narrower intervals.

These calculations are conditional on $\xi =29$. Different nuisance parameter values yield different intervals, motivating our choice of the orthogonal parameterization $(\delta ,\xi )$ where $\xi =n_1{p}_1+n_2{p}_2$. With this choice, the one-dimensional submanifolds $M_{{\xi }_{\circ }}={\xi }^{-1}\left({\xi }_{\circ }\right)$ and $M_{\delta }={\delta }^{-1}\left(\delta \right)$ intersect transversally, and their tangent spaces are orthogonal at the intersection point.

Varying the horizontal line height provides confidence intervals at different levels. For all $z\ne 0$, these lines intersect each of the 480 curves, ensuring that confidence intervals exist for every sample point. The intersection of all confidence levels can be interpreted as a point estimate for $\delta$. For sample points other than $(0,0)$ and $(n_1,n_2)$, this intersection equals the MLE—the point where $z_{s^{\perp }}$ crosses zero. At the boundary points $(0,0)$ and $(n_1,n_2)$, the curves never cross zero, yielding an empty intersection that corresponds to the nonexistence of the MLE.

The 2-standard deviation confidence interval ($z=\pm 2)$ for the log odds ratio $\delta$ is (−0.35, 2.27). The exact 95% confidence interval is (−0.40, 2.40) for nuisance parameter $\xi$ equal to 29. This interval is a function of $\xi$. To obtain an interval that is the same for all values of the nuisance parameter, we take the union of intervals as $\xi$ takes all values to obtain (−0.46, 2.42). The exact 95% confidence interval from Fisher’s exact test is (−0.57, 2.55).

5.2. Other Generalized Estimators

Point estimators naturally induce generalized estimators, though the relationship depends on the parameterization. For a parameterization $\theta$ and point estimator $\widehat{\theta }$, if $\widehat{\theta }\in H_M$, then the generalized estimator is $g_{\widehat{\theta }}=\widehat{\theta }-E\widehat{\theta }$ when no nuisance parameters exist, or $g_{\widehat{\theta }}={(\widehat{\theta }-E\widehat{\theta })}^{\perp }$ with nuisance parameters present.

Consider the binomial family with proportion parameter p. The MLE $\widehat{p}=y/n$ yields $g_{\widehat{p}}=\widehat{p}-p$, which is proportional to the score. However, for the log odds parameterization $\eta =log(p/(1-p\left)\right)$, the MLE $\widehat{\eta }=log(\widehat{p}/(1-\widehat{p}))$ satisfies $\widehat{\eta }\left(0\right)=-\infty$ and $\widehat{\eta }\left(n\right)=+\infty$, so $\widehat{\eta }\notin H_M$. No generalized estimator exists for the unmodified log odds MLE.

A standard remedy adds a small constant $ϵ$ to each cell, yielding the modified MLE:

$$\tilde{\eta }\left(y\right)=log\left({\displaystyle \frac{y+ϵ}{n-y+ϵ}}\right)\in H_M.$$

This modification ensures finite values throughout the sample space, enabling construction of the corresponding generalized estimator.

While the proportion MLE $\widehat{p}$ could similarly be modified, this is rarely performed despite the MLE’s failure at the boundaries. The MLE’s parameter invariance allows its definition without reference to any specific parameterization: for $y\notin \{0,n\}$,

$${\widehat{m}}_y=arg\underset{m\in M}{max}m\left(y\right).$$

This coordinate-free definition emphasizes the MLE’s geometric nature but obscures its boundary behavior.

For comparing two populations using log odds, the modified MLE yields the difference estimator $\widehat{\delta }={\tilde{\eta }}_1-{\tilde{\eta }}_2$, with orthogonalized generalized estimator:

$$g=g_{\widehat{\delta }}={(\widehat{\delta }-E\widehat{\delta })}^{\perp }.$$

Like the orthogonalized score, g exhibits smoothness in parameters and distributional properties in the sample space: $g=g(y,·)\in C^1(\Delta )$ for fixed y, and $g(·,\delta )\in H_{m}M$ for fixed $\delta$. Both are orthogonal to the nuisance space. The critical distinction lies in their geometric location: while $s^{\perp }\in TM$, generally $g\notin TM$ unless $g=s^{\perp }$.

Figure 2 illustrates this distinction for the cancer data. The black curve shows $z_g$ for the observed sample with ${ϵ}_1={ϵ}_2=0.5$ (adding 0.5 to each cell) and nuisance parameter ${\xi }_{\circ }=29$. Each of the 480 sample points generates a smooth curve, with two shown in gray. Vertical lines at any ${\delta }_{\circ }$ intersect these curves to yield distributions with mean zero and unit variance.

Figure 2. Cancer data—modified MLE estimator. Standardized generalized estimator $z_g$ based on the modified log odds difference $\widehat{\delta }$, where 0.5 is added to each cell count. The observed data (18 of 23 surgery successes, 11 of 19 irradiation successes) yields the black curve. Two additional sample points are shown in gray. The nuisance parameter is fixed at $\xi =29$. Compare with Figure 1 to observe the flatter slope indicating lower $\mathrm{\Lambda }$-information.

As with $z_{s^{\perp }}$, horizontal lines at $z=\pm z_0$ determine confidence intervals through their intersections with $z_g$. Steeper slopes produce narrower intervals, making the expected slope a natural efficiency measure. Differentiating the identity $E{Z}_g=0$ with respect to $\delta$ yields

$$E{\displaystyle \frac{\partial Z_g}{\partial \delta }}+E\left(Z_g{Z}_{s^{\perp }}\right)\sqrt{I_{\delta }^{\perp }}=0$$

Rearranging gives the fundamental inequality

$${\left(E{\displaystyle \frac{\partial Z_g}{\partial \delta }}\right)}^2={\rho }_{g{s}^{\perp }}^2{I}_{\delta }^{\perp }\le I_{\delta }^{\perp }$$

(13)

where ${\rho }_{g{s}^{\perp }}$ is the correlation between g and $s^{\perp }$. Vos and Wu [12] define the left-hand side of (13) as the ${\mathrm{\Lambda }}_{\delta }\left(g\right)$, $\mathrm{\Lambda }$-information in g for parameter $\delta$. The bound is attained only when $g=s^{\perp }$, establishing the optimality of the orthogonalized score:

$${\mathrm{\Lambda }}_{\delta }\left(g\right)={\rho }_{g{s}^{\perp }}^2{I}_{\delta }^{\perp }.$$

(14)

The square of the correlation is the same for any reparameterization of $\delta$, so we can define the $\mathrm{\Lambda }$-efficiency of g as

$${\mathrm{Eff}}^{\mathrm{\Lambda }}\left(g\right)={\rho }_{g{s}^{\perp }}^2.$$

$\mathrm{\Lambda }$-efficiency is independent of the choice of interest or nuisance parameter. For example, $\mathrm{\Lambda }$-efficiency will be the same whether we use the odds ratio or the log odds ratio. $\mathrm{\Lambda }$-information, like Fisher information $I_{\delta }^{\perp }$, is a tensor.

The geometric interpretation is revealing: ${\rho }_{g{s}^{\perp }}$ measures the cosine of the angle between g and $s^{\perp }$ in $H_{m}M$. The information loss $(1-{\rho }_{g{s}^{\perp }}^2)I_{\delta }^{\perp }$ equals the squared sine of this angle times the total information. Estimators achieve full $\mathrm{\Lambda }$-efficiency only when perfectly aligned with the orthogonalized score.

A crucial distinction emerges when testing $H_0:\delta =0$. Under this null hypothesis with simple random sampling, the standardized orthogonalized score becomes

$$Z_{s^{\perp }}={\displaystyle \frac{n_2^{1/2}{Z}_1-n_1^{1/2}{Z}_2}{\sqrt{n_1+n_2}}}$$

which remains invariant across all parameterizations $\theta$. This invariance reflects the geometric fact that $H_0:\delta =0$ is equivalent to $H_0:{\theta }_1={\theta }_2$ regardless of the choice of $\theta$.

In contrast, test statistics based on point estimators like ${\widehat{\theta }}_1-{\widehat{\theta }}_2$ depend critically on the parameterization. Tests based on proportions, log proportions, and log odds yield different statistics with different null distributions, even though they test the same hypothesis. The orthogonalized score provides a canonical, parameterization-invariant test that achieves maximum power against local alternatives.

The 2-standard deviation confidence interval ($z=\pm 2)$ for the log odds ratio $\delta$ is (−1.09, 2.95). The exact 95% confidence interval is (−0.43, 2.39) for nuisance parameter $\xi$ equal to 29. The union of intervals over values for $\xi$ is at least (−0.68, 2.47).

5.3. Discussion

Table 2 presents confidence intervals for the log odds difference $\delta$ computed using various methods. These intervals reveal substantial variation in both width and location, highlighting the importance of understanding the underlying geometric principles.

Table 2. Confidence intervals for the log odds difference $\delta$ using cancer treatment data. Computed using R version 4.4.1 with package exact2x2 version 1.6.8 [14]. All methods except Fisher’s exact test are unconditional; Fisher’s conditions on the observed total of 29 successes.

Method	R Function	95% Confidence Interval
Fisher-type adjusted	uncondExact2x2	($-\infty$, 0.57)
Simple asymptotic	uncondExact2x2	($-3.90$, 2.48)
Score-based	uncondExact2x2	($-2.99$, 0.43)
Orthogonalized score $s^{\perp }$	–	($-0.46$, 2.42)
Fisher’s exact test	fisher.test	($-0.57$, 2.55)

The orthogonalized score interval, whether computed at $\xi =29$ or maximized over all nuisance parameter values, falls within both the modified MLE and Fisher’s exact test intervals for this particular dataset. However, this nesting relationship is sample-specific and should not guide method selection. The choice among methods should depend on their theoretical properties rather than their behavior for any particular observed data.

The orthogonalized score offers three key advantages:

• It attains the Fisher information bound, achieving maximum $\mathrm{\Lambda }$-efficiency.
• It requires no ad hoc modifications to handle boundary cases (unlike the MLE for log odds).
• It provides parameterization-invariant inference for $H_0:\delta =0$, yielding identical test statistics whether we parameterize using proportions, log proportions, or log odds.

The R (version 4.4.1) package (version 4.4.1) exact2x2 (version 1.6.8) [14,15] implements several additional unconditional methods, each corresponding to different generalized estimators. While this diversity offers flexibility, it also highlights the need for principled comparison methods.

The geometric framework of generalized estimation provides this principled approach. By working in the Hilbert bundle, we obtain

• Unified treatment: Point estimators and test statistics become special cases of generalized estimators.
• Parameter invariance: Generalized estimators transform properly under reparameterization.
• Linear structure: The Hilbert bundle provides a natural vector space framework for combining and comparing estimators.
• Consistent comparison: $\mathrm{\Lambda }$-information offers a single efficiency measure, replacing the multiple criteria (bias, variance, MSE) used for point estimators.

This geometric perspective reveals why the orthogonalized score achieves optimality: it lies in the tangent bundle $TM$, while other generalized estimators reside only in the larger Hilbert bundle $HM$. The information loss of any estimator equals $I_{\delta }^{\perp }$ times the squared sine of its angle from the tangent space—a geometric characterization that unifies and clarifies classical efficiency results.

6. Conclusions

This paper has demonstrated how the Hilbert bundle structure of statistical manifolds provides a unified geometric framework for statistical inference. By recognizing that points in a statistical manifold are probability distributions rather than abstract points, we extend the traditional tangent bundle framework to encompass a richer geometric structure that naturally accommodates both estimation and hypothesis testing.

The central insight is that generalized estimators—functions on the parameter space—serve as the fundamental inferential objects. The $\mathrm{\Lambda }$-information of a generalized estimator g captures both its smooth structure across models in M and its distributional properties at each point. These dual aspects require different geometric descriptions: the smooth structure manifests through the graph of $Z_g$ in the $(\delta ,z)$ plane, while the distributional properties are naturally characterized within the Hilbert bundle $HM$.

The information bound emerges as a geometric principle: the mean slope of $-Z_g$ equals $\sqrt{\mathrm{\Lambda }\left(g\right)}$, and this slope is maximized precisely when g lies in the tangent bundle $TM$. Statistically, the bound is attained when $g=s^{\perp }$, the orthogonalized score. For any other generalized estimator, the information loss equals $(1-{\rho }_{g{s}^{\perp }}^2)I_{\delta }^{\perp }$, where ${\rho }_{g{s}^{\perp }}$ measures the correlation between g and $s^{\perp }$ as elements of $HM$. This correlation has a direct geometric interpretation: it equals the cosine of the angle between these functions in the Hilbert space.

The presence of nuisance parameters introduces an additional layer of geometric structure. Information loss due to nuisance parameters equals ${\rho }_{s\nu }^2{I}_{\delta }$, where ${\rho }_{s\nu }$ is the correlation between the score s and the nuisance direction $Z_{\nu }$. Crucially, this correlation—and hence the information loss—remains invariant under reparameterization of either interest or nuisance parameters. This invariance reflects a fundamental geometric fact: specifying a value ${\delta }_{\circ }$ for the interest parameter defines a submanifold $M_{{\delta }_{\circ }}={\delta }^{-1}\left({\delta }_{\circ }\right)$ rather than a single point in M. The increased inferential difficulty is precisely quantified by ${\rho }_{s\nu }^2$, the squared correlation between the score and the tangent space of $M_{{\delta }_{\circ }}$.

Our analysis of $2\times 2$ contingency tables illustrates these principles concretely. The orthogonalized score achieves three key advantages over traditional approaches: it attains the information bound, requires no ad hoc modifications for boundary cases, and provides parameterization-invariant inference. The geometric framework explains why different confidence interval methods yield different results—they correspond to different generalized estimators with varying degrees of alignment with the tangent bundle.

This geometric perspective resolves longstanding tensions between estimation and testing frameworks. Rather than treating these as separate endeavors united only by computational tools like the likelihood function, we see them as complementary aspects of a single geometric structure. Point estimators, test statistics, and estimating equations all become special cases of generalized estimators, whose efficiency is uniformly measured by their $\mathrm{\Lambda }$-information.

The Hilbert bundle framework thus provides both conceptual clarity and practical benefits. It reveals why certain statistical procedures are optimal, quantifies the cost of using suboptimal methods, and suggests principled ways to construct new inferential procedures. By shifting focus from points in parameter space to functions on the manifold, we gain a richer, more complete understanding of statistical evidence and its geometric foundations.