Geometric Residual Projection in Linear Regression: Rank-Aware Operators and a Geometric Multicollinearity Index

Abstract

Residuals play a central role in linear regression, yet their geometry is often hidden by inverse- and pseudoinverse-based formulas. We develop a rank-aware framework for residual projection that makes the underlying orthogonality explicit. When the design matrix has codimension one, the unexplained component of the response lies along a single unit normal to the predictor space, and the residual projector reduces to the rank-one operator $n{n}^{\top }$, avoiding matrix inversion. For general designs, the residual lies in a higher-dimensional orthogonal complement spanned by an orthonormal basis N, and the residual projector factorizes as $N{N}^{\top }$. Using cross-products, wedge products, and Gram determinants, we provide basis-independent characterizations of the residual subspace. We further introduce the Geometric Multicollinearity Index (GMI), a scale-invariant diagnostic derived from the polar sine that quantifies the collapse of predictor-space volume under multicollinearity. Synthetic perturbation studies and an illustrative real-data experiment show that the proposed projectors reproduce ordinary least squares residuals, that GMI responds predictably to controlled collinearity, and that the projector viewpoint clarifies the distinction between regression residuals and PCA reconstruction residuals in both full-rank and rank-deficient settings.

1. Introduction

Linear regression remains one of the most widely used modeling tools in statistics, data science, and machine learning [1]. In its familiar form

$$Y=X\beta +\epsilon ,\mathbb{E}\left[\epsilon \right]=0,X\in {\mathbb{R}}^{n\times p},Y\in {\mathbb{R}}^n,\beta \in {\mathbb{R}}^p,$$

(1)

the ordinary least squares (OLS) estimator chooses coefficients that minimize the squared discrepancy between observations and predictions. When the design matrix X has the full column rank, the fitted values and the associated orthogonal projector onto the column space $C\left(X\right)$ take the classical form

$$\widehat{\beta }={\left(X^{\top }X\right)}^{-1}{X}^{\top }Y,\widehat{Y}=X\widehat{\beta },P=X{\left(X^{\top }X\right)}^{-1}{X}^{\top },$$

(2)

with P symmetric and idempotent [2]. More generally, the orthogonal projector onto $C\left(X\right)$ can be written as $P=X{X}^{+}$ in terms of the Moore–Penrose pseudoinverse $X^{+}$ [2].

The fitted component and residual are

$$\widehat{Y}=PY,r=(I-P)Y,$$

(3)

so that $r\in C{\left(X\right)}^{\perp }$ and

$$Y=PY+r,$$

is the orthogonal decomposition of the response into a part explained by $C\left(X\right)$ and a part orthogonal to it. This algebraic formulation is standard, but it can obscure the underlying geometry that governs the behavior of the residuals, and inversion-based formulas can be fragile in high-dimensional or nearly singular regimes [2].

A classical identity from numerical linear algebra makes the geometry explicit:

$$I-P=N{N}^{\top },$$

(4)

where the columns of N form an orthonormal basis of $C{\left(X\right)}^{\perp }$ [2]. When $\rho =n-1$, the orthogonal complement is one-dimensional and the projector reduces to the rank-one form

$$I-P=n{n}^{\top },$$

(5)

where n is the unit vector (ambiguous sign) perpendicular to $C\left(X\right)$. In this codimension one setting, the residual is the projection of Y into a single direction, and the operator (5) completely avoids matrix inversion. For higher codimension, the residual lies in a subspace of dimension $k=n-\rho$, and the factorization (4) makes this structure explicit.

The construction of normal directions motivates generalizations of the classical cross-product to ${\mathbb{R}}^n$. Belhaouari et al. formalize a determinant-based vector product that yields a vector orthogonal to $n-1$ linearly independent vectors in ${\mathbb{R}}^n$ [3]. In regression problems with $dimC{\left(X\right)}^{\perp }=1$, a single normal vector uniquely defines the rank-one projector (5). For higher codimensions, these constructions naturally combine with null-space and QR-based methods to form an orthonormal residual basis N.

Multicollinearity is a persistent challenge in regression modeling, as highly dependent predictors inflate variances, destabilize coefficient estimates, and impede interpretation [4,5]. Classical diagnostics such as the variance inflation factor (VIF) quantify this effect but offer limited geometric insight. We therefore introduce the Geometric Multicollinearity Index (GMI), a scale-invariant, volume-based measure derived from the polar sine and the Gram determinant. For a full column-rank matrix $X=[x_1,\dots ,x_p]$ with $p\le n$, the polar sine normalizes $det\left(X^{\top }X\right)$ by the product of the column norms, taking values in $[0,1]$: it equals 1 for orthogonal predictors and approaches 0 as multicollinearity increases.

In rank-deficient or over-parameterized regimes, the Gram determinant vanishes and the GMI saturates at 1, indicating extreme geometric degeneracy. As a normalized volume measure of the parallelotope spanned by the columns of X, the GMI provides a compact, scale-invariant geometric diagnostic across dimensions. In this regime, $\mathrm{GMI}=1$ should be interpreted as a diagnostic flag for complete geometric collapse, rather than as a measure of severity beyond rank deficiency.

Although both regression and principal component analysis (PCA) rely on orthogonal projections, their objectives differ fundamentally [6]. PCA projects X onto directions of maximum variance, yielding residuals $(I-P_{\mathrm{PCA}})X$ that remain in $C\left(X\right)$. By contrast, regression projects the response, decomposing Y into a fitted component $PY$ and a residual $r=(I-P)Y\in C{\left(X\right)}^{\perp }$ [2]. This distinction between feature-space and response-space residuals is essential for interpreting linear-model diagnostics.

Scope of the present paper is the identities $I-P=N{N}^{\top }$ and $I-P=n{n}^{\top }$ are standard results from orthogonal projection theory. The contribution of this paper lies in a rank-aware geometric interpretation of these identities via multivector and cross-product constructions, together with the introduction of the volume-based Geometric Multicollinearity Index (GMI) as a complementary geometric diagnostic.

This work is theoretical in nature. We develop a rank-aware residual projector, characterize the associated residual subspaces, and formalize GMI as an intrinsic measure of multicollinearity. Algorithmic, large-scale, regularized, and nonlinear extensions are beyond the scope of this paper and are left for future work.

The main contributions of this theoretical paper are as follows:

• Rank-aware geometric formulation is the first one where we develop a unified, rank-aware description of the residual projector based on the identity $I-P=N{N}^{\top }$, which reduces to the rank-one form $n{n}^{\top }$ when $dimC{\left(X\right)}^{\perp }=1$ and generalizes to arbitrary rank.
• Multivector perspective on residual spaces such that we reformulate $C{\left(X\right)}^{\perp }$ and the associated projector using multivectors, wedge products, and cross-products, yielding a basis-independent geometric characterization valid in both codimension-one and higher-codimension settings.
• In Geometric Multicollinearity Index (GMI) We introduce a scale-invariant, volume-based diagnostic derived from the Gram determinant and polar sine, which quantifies geometric degeneracy in both full-rank and rank-deficient regimes and complements classical VIF-type measures.
• Also, We clarify the conceptual distinction between regression residuals and PCA reconstruction residuals, emphasizing the impact of projecting the response versus the predictors on residual geometry and interpretation.
• Finally, we provide simple numerical examples illustrating rank-one and multivector residual projections and the behavior of GMI under controlled perturbations.

The organization of the paper is provieded as follows Section 2 introduces the notation and basic geometric concepts. Section 3 develops the main theoretical results on residual projectors and the geometric structure of $C{\left(X\right)}^{\perp }$. Section 4 presents numerical illustrations, including the behavior of GMI under controlled multicollinearity. Section 5 and Section 6 discuss implications and conclude.

2. Notation and Preliminaries

We briefly fix notation and recall the algebraic and geometric concepts that underpin the proposed framework. We follow standard linear algebra conventions as in Strang [7], Lay et al. [8], and Golub and Van Loan [2].

2.1. Basic Notation and Subspaces

The transpose of a matrix or vector is denoted by the superscript ⊤. Lowercase letters (e.g., x, n) denote column vectors, and uppercase letters (e.g., X, N) denote matrices. For a matrix $X\in {\mathbb{R}}^{n\times p}$, we write $\mathcal{C}\left(X\right)$ for its column space and denote its rank by $\rho$. Throughout, r denotes the residual vector while $\rho$ denotes the rank of X, to avoid notational ambiguity. All notation is defined once and used consistently throughout the manuscript; in particular, N denotes an orthonormal basis of the orthogonal complement $\mathcal{C}{\left(X\right)}^{\perp }$.

We reserve r exclusively for the residual vector in (8).

The orthogonal complement of a subspace $V\subseteq {\mathbb{R}}^n$ is denoted by $V^{\perp }$, and the null space of a matrix A is

$$\mathcal{N}\left(A\right)=\{z\in {\mathbb{R}}^m:Az=0\}.$$

A summary of all symbols and subspace conventions is provided in Abbreviations.

Given $n\in {\mathbb{R}}^n$, the matrix $n{n}^{\top }\in {\mathbb{R}}^{n\times n}$ denotes its outer product. The identity matrix in ${\mathbb{R}}^{n\times n}$ is denoted by I.

2.2. Projection and Residual Operators

Let $X\in {\mathbb{R}}^{n\times p}$ have rank $\rho \le min\{n,p\}$; when X has full column rank ($\rho =p$), the orthogonal projector admits the classical closed form $P=X{\left(X^{\top }X\right)}^{-1}{X}^{\top }$.

The orthogonal projector onto the column space $\mathcal{C}\left(X\right)$ is the unique symmetric idempotent matrix P that satisfies

$$\mathcal{C}\left(P\right)=\mathcal{C}\left(X\right),P^{\top }=P,P^2=P.$$

When X has full column rank $\rho$, this projector admits the well-known closed form

$$P=X{\left(X^{\top }X\right)}^{-1}{X}^{\top },$$

(6)

which is symmetric and idempotent [2]. In the general case, including rank-deficient designs, P can be written as

$$P=X{X}^{+},$$

(7)

where $X^{+}$ denotes the Moore–Penrose pseudoinverse of X [2]. Throughout, $X^{+}$ denotes the Moore–Penrose pseudoinverse of X, computed via an SVD-based routine.

Both (6) and (7) implement the same geometric operation: orthogonal projection onto $\mathcal{C}\left(X\right)$.

When $X^{\top }X\succ 0$, the same orthogonal projector can equivalently be constructed by Cholesky factorization of $X^{\top }X$, producing identical residuals while requiring positive definiteness of the Gram matrix.

For any response vector $Y\in {\mathbb{R}}^n$, the fitted component is

$$\widehat{Y}=PY,$$

and the residual is

$$r=(I-P)Y,$$

(8)

which obeys $X^{\top }r=0$ and hence lies in the orthogonal complement $\mathcal{C}{\left(X\right)}^{\perp }$ [2].

Let $\rho =rank\left(X\right)$ and set $k=n-\rho =dim\mathcal{C}{\left(X\right)}^{\perp }$. Choose any matrix $N\in {\mathbb{R}}^{n\times k}$ whose columns form an orthonormal basis of $\mathcal{C}{\left(X\right)}^{\perp }$:

$$N^{\top }N=I_k,X^{\top }N=0.$$

(9)

Then the residual projector admits the geometric factorization

$$I-P=N{N}^{\top },$$

(10)

and the null space of $X^{\top }$ is

$$\mathcal{N}\left(X^{\top }\right)=\{z\in {\mathbb{R}}^n:X^{\top }z=0\}=\mathcal{C}{\left(X\right)}^{\perp }.$$

(11)

This characterization is standard in numerical linear algebra and regression theory [2] and forms the algebraic backbone of our geometric residual formulation. Equation (10) makes explicit that the residual r lies in a k-dimensional orthogonal complement spanned by the columns of N.

2.3. Cross-Product and Cross–Wedge–QR Method

To construct explicit orthogonal directions, especially in the case of codimension-one where $dim\mathcal{C}{\left(X\right)}^{\perp }=1$, we employ the cross-product in ${\mathbb{R}}^n$ [3]. Given $n-1$ linearly independent vectors

$$x_1,\dots ,x_{n-1}\in {\mathbb{R}}^n,$$

the determinant-based construction of Belhaouari et al. produces

$$cross(x_1,\dots ,x_{n-1})\in {\mathbb{R}}^n,$$

which is orthogonal to each $x_i$:

$$x_i^{\top }cross(x_1,\dots ,x_{n-1})=0,i=1,\dots ,n-1.$$

This extends earlier n-dimensional vector products and related constructions in eigenanalysis and geometry [9,10].

In the regression setting, when $\rho =n-1$ the predictor space $\mathcal{C}\left(X\right)$ has codimension one and $\mathcal{C}{\left(X\right)}^{\perp }$ is spanned by a single unit vector n. Choosing any basis of $\mathcal{C}\left(X\right)$ and applying the cross-product yields a nonzero vector orthogonal to $\mathcal{C}\left(X\right)$; normalizing it gives a unit normal n spanning $\mathcal{C}{\left(X\right)}^{\perp }$. In this case, the residual projector reduces to the rank-one form

$$I-P=n{n}^{\top },$$

(12)

and the residual is simply $r=\left(n^{\top }Y\right)n$.

In this codimension-one setting, the rank-one projector $I-P=n{n}^{\top }$ coincides with the classical construction of the cross-product, and we refer to this case as the residual projector of the cross-product (rank one).

A key geometric property of the cross-product is that its norm equals the $(n-1)$-dimensional volume of the parallelotope spanned by its arguments [3]:

$$∥cross(v_1,v_2,\dots ,v_{n-1})∥={vol}_{n-1}(v_1,\dots ,v_{n-1}).$$

(13)

Thus, the cross-product simultaneously provides an orthogonal direction and encodes a volume. When $dim\mathcal{C}{\left(X\right)}^{\perp }>1$, a single cross-product yields at most one null direction; in that case additional null directions must be obtained by other means (e.g., null-space methods and QR), as discussed later.

2.4. Wedge Products, Gram Matrices, and Polar Sine

To describe volumes and linear independence in a coordinate-free manner, we rely on standard constructions from linear algebra and multilinear geometry [2,7,8]. Given vectors $u_1,\dots ,u_k\in {\mathbb{R}}^n$, their wedge product $u_1\wedge \cdots \wedge u_k$ encodes the oriented k-dimensional parallelotope spanned by $\left\{u_i\right\}$. Geometrically, $u_1\wedge \cdots \wedge u_k=0$ if and only if the vectors are linearly dependent, and its norm $\parallel u_1\wedge \cdots \wedge u_k\parallel$ equals the volume k of the associated parallelotope. This interpretation underlies many geometric treatments of subspaces, volumes, and orthogonality in ${\mathbb{R}}^n$ [2,7].

For computational purposes, such volumes can be expressed using Gram matrices. Given vectors $v_1,\dots ,v_m\in {\mathbb{R}}^n$, the associated Gram matrix is

$$G=\left[\begin{array}{ccc}{v}_1^{\top }{v}_1& \cdots & v_1^{\top }{v}_m\\ ⋮& \ddots & ⋮\\ v_m^{\top }{v}_1& \cdots & v_m^{\top }{v}_m\end{array}\right]={[v_1,\dots ,v_m]}^{\top }[v_1,\dots ,v_m].$$

(14)

The m-dimensional volume of the parallelotope spanned by $\left\{v_i\right\}$ is given by

$${vol}_m(v_1,\dots ,v_m)=\sqrt{det\left(G\right)},$$

(15)

a classical identity that plays a central role in covariance analysis and principal component analysis [2,6].

We quantify the angular separation of p predictor vectors using the polar sine. Let $X=[x_1,\dots ,x_p]\in {\mathbb{R}}^{n\times p}$ with Gram matrix $G=X^{\top }X$. Following geometric constructions based on determinants and cross-products [3,9,10], the polar sine is defined as

$$psin(x_1,\dots ,x_p)={\displaystyle \frac{\sqrt{det\left(X^{\top }X\right)}}{\parallel x_1\parallel \cdots \parallel x_p\parallel }}.$$

(16)

When $p\le n$ and the columns of X are linearly independent, $det\left(X^{\top }X\right)>0$ and $psin(x_1,\dots ,x_p)\in (0,1]$; it equals 1 if and only if the vectors are mutually orthogonal. If the columns of X are linearly dependent or $p>n$, then $X^{\top }X$ is singular and $det\left(X^{\top }X\right)=0$, and we adopt the convention $psin(x_1,\dots ,x_p)=0$. Thus, the polar sine takes values in $[0,1]$, is scale-invariant, and decreases toward zero as the set of columns becomes geometrically degenerate.

Within the present framework, the polar sine provides a normalized geometric measure of predictor-space collapse under multicollinearity and forms the basis of the Geometric Multicollinearity Index (GMI) introduced in the following section.

2.5. Projection Strategies: QR and Cross-Product Formulations

Two projection strategies play a central role in the remainder of the paper.

First, QR decomposition is used when an entire orthonormal basis of $\mathcal{C}\left(X\right)$ or $\mathcal{C}{\left(X\right)}^{\perp }$ is required. If $X=QR$ is a (thin) QR factorization with orthonormal columns in Q, then

$$P=Q{Q}^{\top },$$

is the orthogonal projector onto $\mathcal{C}\left(X\right)$, and orthonormal complements can be obtained by extending Q to an orthogonal basis of ${\mathbb{R}}^n$ [2]. QR-based projectors are numerically stable and well suited for high-dimensional or nearly singular problems.

Second, when the residual space is one-dimensional (for example, when $\rho =n-1$), the cross-product provides a compact alternative. It produces a normal vector n directly from which the residual projector is derived:

$$I-P=n{n}^{\top }.$$

This follows without forming or inverting $X^{\top }X$ [3]. In higher-codimension settings ($dim\mathcal{C}{\left(X\right)}^{\perp }>1$), cross-products can still be used to generate individual null directions, but a full residual basis is obtained more naturally from $\mathcal{N}\left(X^{\top }\right)$ and orthonormalised via QR.

When null directions are generated sequentially via cross-products and accumulated into a multivector basis that is subsequently orthonormalised using QR, we refer to the resulting construction as the Recursive Cross–Wedge–QR method.

These geometric primitives—orthonormal bases, cross-products, wedge products, Gram matrices, and polar sine—constitute the toolkit on which the residual projection framework developed in the following sections is built.

Here s denotes the sketch size (number of random probe vectors) used in the Gaussian range-finding procedure.

Sketch–QR Baseline (Reproducibility)

The Sketch–QR baseline follows a standard randomized range-finding procedure. Given $X\in {\mathbb{R}}^{n\times p}$ and a sketch size s, we draw a Gaussian sketching matrix $\Omega \in {\mathbb{R}}^{p\times s}$ with independednt and identically distributed $\mathcal{N}(0,1)$ entries and form $Y=X\Omega \in {\mathbb{R}}^{n\times s}$. We then compute a thin QR factorization $Y=\tilde{Q}R$, where $\tilde{Q}\in {\mathbb{R}}^{n\times s}$ has orthonormal columns. The approximate projector is $\tilde{P}=\tilde{Q}{\tilde{Q}}^{\top }$, and the approximate residual is $\tilde{r}=(I-\tilde{P})y$. Unless stated otherwise, we use a single sketch without power iterations or oversampling [11,12].

3. Core Theory

3.1. Rank-One Residual Projection

Assume that $X\in {\mathbb{R}}^{n\times p}$ has rank $n-1$. Then $dim\mathcal{C}\left(X\right)=n-1$ and $dim\mathcal{C}{\left(X\right)}^{\perp }=1$. There exists a (unique up to sign) unit vector $n\in {\mathbb{R}}^n$ such that

$$X^{\top }n=0,\parallel n\parallel =1.$$

Proposition 1 

(Rank-One Residual Projection). Let $X\in {\mathbb{R}}^{n\times p}$ have $\rho =n-1$, and let P denote the orthogonal projector onto $\mathcal{C}\left(X\right)$ (for example, $P=X{X}^{+}$ in terms of the Moore–Penrose pseudoinverse $X^{+}$). Then the orthogonal residual projector is

$$I-P=n{n}^{\top }.$$

Proof. 

Because $\mathcal{C}{\left(X\right)}^{\perp }$ is one-dimensional, any residual $r\in \mathcal{C}{\left(X\right)}^{\perp }$ can be written as $r=\alpha n$ for some scalar $\alpha$. Since $n\in \mathcal{C}{\left(X\right)}^{\perp }$ and P is the orthogonal projector onto $\mathcal{C}\left(X\right)$, we have $Pn=0$ and therefore $(I-P)n=n$. For any $Y\in {\mathbb{R}}^n$, we may write $Y=u+v$ with $u\in \mathcal{C}\left(X\right)$ and $v\in \mathcal{C}{\left(X\right)}^{\perp }=span\left\{n\right\}$. Then

$$(I-P)Y=(I-P)(u+v)=v,$$

while

$$n{n}^{\top }Y=n{n}^{\top }(u+v)=n{n}^{\top }u+n{n}^{\top }v=0+v=v,$$

because $n^{\top }u=0$ and v is a scalar multiple of n. Thus, $(I-P)Y=n{n}^{\top }Y$ for all Y, and therefore $I-P=n{n}^{\top }$, which is symmetric and idempotent.    □

3.2. Generalization to n Dimensions

The rank-one projector $I-P=n{n}^{\top }$ extends naturally to the general setting in which the predictor matrix $X\in {\mathbb{R}}^{n\times p}$ has $\rho <n$. In this case, the orthogonal complement $\mathcal{C}{\left(X\right)}^{\perp }$ has dimension $k=n-\rho$, and the residual is no longer confined to a single direction but lies in a k-dimensional subspace.

Theorem 1 

(Multivector Residual Projection). Let X have rank $\rho <n$ and set $k:=n-\rho$. Let $N\in {\mathbb{R}}^{n\times k}$ have orthonormal columns spanning $\mathcal{C}{\left(X\right)}^{\perp }$ (so $N^{\top }N=I_k$). Then, the residual projector is $I-P=N{N}^{\top }$ and $r=(I-P)y=N{N}^{\top }y$.

Proof. 

Since $N^{\top }N=I_{n-\rho }$, the matrix $N{N}^{\top }$ is symmetric and satisfies

$${\left(N{N}^{\top }\right)}^2=N\left(N^{\top }N\right)N^{\top }=N{I}_{n-\rho }{N}^{\top }=N{N}^{\top },$$

so $N{N}^{\top }$ is an orthogonal projector. Its range equals $\mathcal{C}\left(N\right)=\mathcal{C}{\left(X\right)}^{\perp }$.

On the other hand, P is the orthogonal projector onto $\mathcal{C}\left(X\right)$, so $I-P$ is the orthogonal projector onto $\mathcal{C}{\left(X\right)}^{\perp }$. For any $Y\in {\mathbb{R}}^n$, write $Y=u+v$ with $u\in \mathcal{C}\left(X\right)$ and $v\in \mathcal{C}{\left(X\right)}^{\perp }$. Then

$$(I-P)Y=v.$$

Since $v\in \mathcal{C}\left(N\right)$ and the columns of N are orthonormal, we also have

$$N{N}^{\top }Y=N{N}^{\top }(u+v)=N{N}^{\top }u+N{N}^{\top }v=0+v=v,$$

because $N^{\top }u=0$ and $N{N}^{\top }$ acts as the identity on $\mathcal{C}\left(N\right)$. Thus, $(I-P)Y=N{N}^{\top }Y$ for all Y, and hence $I-P=N{N}^{\top }$.    □

Figure 1 illustrates the situation in a simple three-dimensional example. The predictor space occupies a lower-dimensional subspace (blue), while the residual lies in its orthogonal complement (orange), spanned by multiple orthonormal directions when $dim\mathcal{C}{\left(X\right)}^{\perp }>1$.

3.2.1. Geometric Multicollinearity Index (GMI)

The multivector formulation of the residual projector reflects the angular configuration of the columns of X. When the predictors are nearly linearly dependent, the volume of the parallelotope spanned by these columns collapses, signaling instability. To capture this behavior geometrically, we introduce the Geometric Multicollinearity Index (GMI), a scale-invariant measure based on the polar sine.

Definition 1 

(Polar Sine and GMI). Let $X=[x_1,\dots ,x_p]\in {\mathbb{R}}^{n\times p}$ have nonzero columns $x_j\ne 0$, and let $psin(x_1,\dots ,x_p)$ be the polar sine defined in Equation (16).

We adopt the shorthand

$$psin\left(X\right):=psin(x_1,\dots ,x_p),$$

(17)

and define the Geometric Multicollinearity Index by

$$GMI\left(X\right)=1-psin\left(X\right).$$

(18)

Lemma 1 

(Basic properties). Let $X=[x_1,\dots ,x_p]\in {\mathbb{R}}^{n\times p}$ have nonzero columns, and let $G=X^{\top }X$ denote its Gram matrix. Then:

(i) 

$\mathrm{psin}\left(X\right)\in [0,1]$ and hence $\mathrm{GMI}\left(X\right)\in [0,1]$;

(ii) 

$\mathrm{psin}\left(X\right)=1$ (and $\mathrm{GMI}\left(X\right)=0$) if and only if the columns of X are mutually orthogonal;

(iii) 

$\mathrm{psin}\left(X\right)=0$ (and $\mathrm{GMI}\left(X\right)=1$) whenever the columns are linearly dependent or $p>n$.

Proof. 

By Hadamard’s inequality,

$$det\left(G\right)\le \prod _{j=1}^p{\parallel x_j\parallel }^2,$$

with equality if and only if the columns of X are mutually orthogonal. Substituting this bound into the polar-sine definition Equation (16) yields

$$0\le \mathrm{psin}\left(X\right)={\displaystyle \frac{\sqrt{det\left(G\right)}}{\parallel x_1\parallel \cdots \parallel x_p\parallel }}\le 1,$$

which proves (i). Moreover, $\mathrm{psin}\left(X\right)=1$ holds precisely when equality in Hadamard’s inequality holds, i.e., when the columns are pairwise orthogonal, giving (ii).

If the columns of X are linearly dependent or $p>n$, then G is singular and $det\left(G\right)=0$, so by definition $\mathrm{psin}\left(X\right)=0$ in that case, and (iii) follows immediately. The corresponding statements for $\mathrm{GMI}\left(X\right)$ are then direct consequences of the definition $\mathrm{GMI}\left(X\right)=1-\mathrm{psin}\left(X\right)$.    □

Lemma 2 

(Scale invariance). Let $A=\mathrm{diag}(a_1,\dots ,a_p)$ with each $a_j\ne 0$. Then $\mathrm{psin}\left(XA\right)=\mathrm{psin}\left(X\right)$ and hence $\mathrm{GMI}\left(XA\right)=\mathrm{GMI}\left(X\right)$.

Proof. 

We have ${\left(XA\right)}^{\top }\left(XA\right)=A^{\top }GA$ and $det\left(A^{\top }GA\right)=det{\left(A\right)}^{2}det\left(G\right)$, while $\parallel a_j{x}_j\parallel =|a_j|\parallel x_j\parallel$ for each j. The scale factors cancel in the quotient (17), so $\mathrm{psin}\left(XA\right)=\mathrm{psin}\left(X\right)$, and (18) yields the claim for GMI.    □

Remark 1. 

For orthonormal predictors, $\mathrm{psin}\left(X\right)=1$ and $\mathrm{GMI}\left(X\right)=0$. For nearly collinear predictors, $det\left(G\right)\approx 0$, so $\mathrm{psin}\left(X\right)\approx 0$ and $\mathrm{GMI}\left(X\right)\approx 1$. Intermediate angular configurations (e.g., vectors separated by ≈45°) yield values such as $\mathrm{psin}\approx 0.71$ and $\mathrm{GMI}\approx 0.29$, as illustrated in Figure 2. In this sense, GMI is a normalized version of determinant-based multicollinearity measures (such as the generalized variance $det\left(G\right)$ or det of the correlation matrix), rescaled to lie in $[0,1]$ and made explicitly scale-invariant via the polar-sine normalization.

GMI provides a direct geometric diagnostic that complements classical VIF-type measures. Whereas VIF assesses how much the variance of each individual regression coefficient is inflated by linear dependence among the predictors, GMI treats the entire predictor set as a single geometric object and summarizes its angular degeneracy in a single number in $[0,1]$. Because it depends only on the angular configuration of the predictors and is invariant under independent rescaling of the columns, it can be compared meaningfully across differently scaled datasets.

3.2.2. Uniqueness of the Residual Projection

The multivector formulation $I-P=N{N}^{\top }$ expresses the residual projector in terms of any orthonormal basis of the orthogonal complement $\mathcal{C}{\left(X\right)}^{\perp }$. The next result confirms that this projector is independent of the specific basis chosen.

Remark 2 

(Independence of the residual projector from the basis). Let $X\in {\mathbb{R}}^{n\times p}$ have rank $r<n$, and let $N,\tilde{N}\in {\mathbb{R}}^{n\times (n-\rho )}$ have orthonormal columns spanning $\mathcal{C}{\left(X\right)}^{\perp }$. Then there exists an orthogonal $Q\in {\mathbb{R}}^{(n-\rho )\times (n-\rho )}$ such that $\tilde{N}=NQ$, and hence

$$\tilde{N}{\tilde{N}}^{\top }=NQ{Q}^{\top }{N}^{\top }=N{N}^{\top }.$$

In particular, the representation $I-P=N{N}^{\top }$ from Theorem 1 is independent of the orthonormal basis chosen of $\mathcal{C}{\left(X\right)}^{\perp }$: it is simply the (unique) orthogonal projector onto $\mathcal{C}{\left(X\right)}^{\perp }$.

3.3. Generalized Projection in Rank-Deficient Spaces via Null-Spaces and Wedge Products

When the design matrix X is rank-deficient or when $n-\rho >1$, the orthogonal complement $\mathcal{C}{\left(X\right)}^{\perp }$ has dimension greater than one, and a single normal vector—as in the rank-one case—is no longer sufficient to describe the residual direction. In such settings, a full orthonormal basis of $\mathcal{C}{\left(X\right)}^{\perp }$ is required. This subsection explains how standard null-space computations, combined with QR orthonormalization, yield such a basis, and how wedge products provide a coordinate-free way to reason about independence and volume.

Definition 2 

(Residual Projection in the Rank-Deficient Case). Let $X\in {\mathbb{R}}^{n\times p}$ have rank $r<n$, and set $k=n-\rho =dim\mathcal{C}{\left(X\right)}^{\perp }$. If $N\in {\mathbb{R}}^{n\times k}$ has orthonormal columns spanning $\mathcal{C}{\left(X\right)}^{\perp }$, then the residual projector is

$$I-P=N{N}^{\top }.$$

Proposition 2 

(Limitation of the cross-product). The cross-product of Belhaouari [3] produces a single vector orthogonal to $n-1$ independent vectors in ${\mathbb{R}}^n$. This suffices when $dim\mathcal{C}{\left(X\right)}^{\perp }=1$, but for $n-r>1$ it yields only one null direction. Additional directions are needed to span the entire residual space.

Proof. 

If $n-\rho =1$, then $\mathcal{C}{\left(X\right)}^{\perp }$ is one-dimensional, and a single cross-product applied to a basis of $\mathcal{C}\left(X\right)$ recovers the unique (up to sign) normal vector n, giving $I-P=n{n}^{\top }$ as in the rank-one case. When $n-\rho >1$, the orthogonal complement has dimension $k>1$, so multiple linearly independent null directions are required. A single cross-product produces at most one such direction and therefore cannot span a k-dimensional space. Additional vectors must be constructed, for instance, via null-space computations and subsequent orthonormalization.    □

Theorem 2 

(Construction of a Residual Basis via null-space and QR). Let $X\in {\mathbb{R}}^{n\times p}$ have rank r and set $k=n-\rho$, so that $\mathcal{C}{\left(X\right)}^{\perp }=\mathcal{N}\left(X^{\top }\right)$ has dimension k. Then there exists an orthonormal basis $N\in {\mathbb{R}}^{n\times k}$ of $\mathcal{C}{\left(X\right)}^{\perp }$ such that

$$I-P=N{N}^{\top },$$

where P is the orthogonal projector onto $\mathcal{C}\left(X\right)$. Moreover, such an N can be obtained by the null-space + QR procedure in Algorithm 1.

Algorithm 1 Residual Projector via Null Space and QR (Explicit SVD/RRQR)

Require: Design matrix $X\in {\mathbb{R}}^{n\times p}$; numerical tolerance $\tau >0$. Ensure: Orthonormal basis N of $\mathcal{N}\left(X^{\top }\right)$ and residual projector $(I-P)=N{N}^{\top }$.    1: Estimate numerical rank: compute $\rho$ using either SVD or RRQR (below); set $k\leftarrow n-\rho$.    2: Construct null-space basis: compute $B\in {\mathbb{R}}^{n\times k}$ spanning $\mathcal{N}\left(X^{\top }\right)$ using exactly one of the following: (SVD option: robust) Compute a full SVD $X=U\Sigma V^{\top }$ with singular values ${\sigma }_1\ge {\sigma }_2\ge \cdots$. Set $\rho :=\#\{i:{\sigma }_i\ge \tau {\sigma }_1\}$ and $k:=n-\rho$. Set $B\leftarrow U(:,\rho +1:n)$▹$\mathcal{N}\left(X^{\top }\right)$ spanned by left singular vectors with near-zero ${\sigma }_i$ (RRQR option: efficient) Compute a column-pivoted QR factorization $X^{\top }\Pi =QR$. Set $\rho :=max\{i:|R_{ii}|\ge \tau |R_{11}\left|\right\}$ and $k:=n-\rho$. Partition $R=\left[\begin{array}{cc}{R}_{11}& R_{12}\\ 0& R_{22}\end{array}\right]$. Solve $R_{11}Z=R_{12}$ for Z (do not form $R_{11}^{-1}$). Form $B_0=\left[\begin{array}{c}-Z\\ I_k\end{array}\right]$ and set $B\leftarrow \Pi B_0$. Implementation note: Row reduction is sufficient for symbolic or toy examples; for floating-point computations, use SVD (robust) or RRQR (fast).    3: (Optional) Verify $rank\left(B\right)=k$.    4: Compute a thin QR factorization $B=Q_B{R}_B$ with orthonormal columns in $Q_B$.    5: Set $N\leftarrow Q_B$.    6: return N and the residual projector $(I-P)=N{N}^{\top }$.

Proof. 

By construction, the columns of B span $\mathcal{N}\left(X^{\top }\right)$, which is equal to $\mathcal{C}{\left(X\right)}^{\perp }$. If $b_1\wedge \cdots \wedge b_k\ne 0$ then the $b_i$’s are linearly independent and form a basis of this k-dimensional subspace. The thin QR factorization $B=QR$ with orthonormal columns in Q then produces $N:=Q$ with $N^{\top }N=I_k$ and $\mathcal{C}\left(N\right)=\mathcal{C}\left(B\right)=\mathcal{C}{\left(X\right)}^{\perp }$. The matrix $N{N}^{\top }$ is therefore the orthogonal projector onto $\mathcal{C}{\left(X\right)}^{\perp }$, so $I-P=N{N}^{\top }$ by the uniqueness of the orthogonal projector onto a fixed subspace.    □

Remark 3. 

In practice, the null-space basis in Step 1 is typically obtained from a numerically stable factorization such as SVD or a rank-revealing QR. The wedge product need not be computed explicitly; it serves primarily as a geometric way to interpret the independence check. When $dim\mathcal{C}{\left(X\right)}^{\perp }=1$, the construction reduces to the case of rank one, where a normal vector can be obtained either from the null space of $X^{\top }$ or by applying a cross-product to a basis of $\mathcal{C}\left(X\right)$. The numerical tolerance used for the determination of rank (e.g., $\rho =\tau ={10}^{-12}$ relative to the largest singular value) follows standard practice and does not affect the conceptual geometric results, which hold exactly at the algebraic level.

Remark 4 

(Implementation note: computing $\mathcal{N}\left(X^{\top }\right)$ in floating-point arithmetic). The instruction “compute a basis of $\mathcal{N}\left(X^{\top }\right)$” requires (i) a numerical rank decision and (ii) an explicit extraction rule. In this work we use either:

(a) SVD. Compute $X^{\top }=U\Sigma V^{\top }$, choose the numerical rank $\rho =\#\{i:{\sigma }_i\ge \tau {\sigma }_1\}$, and set $B=V(:,r+1:n)$.

(b) RRQR. Compute a column-pivoted QR factorization $X^{\top }\Pi =QR$, choose rank ρ from the diagonal of R (e.g., $|R_{ii}|\ge \tau |R_{11}|$), partition $R=\left[\begin{array}{cc}{R}_{11}& R_{12}\\ 0& R_{22}\end{array}\right]$, and set $B=\Pi \left[\begin{array}{c}-R_{11}^{-1}{R}_{12}\\ I_{n-\rho }\end{array}\right]$.

SVD is typically more robust under severe ill conditioning, while RRQR is often faster when pivoted QR is sufficient. Row reduction is used only for symbolic/toy demonstrations.

Figure 3 illustrates the residual subspace and two independent null directions before orthonormalization.

3.4. Projection Method Variants and Complexity

Definition 3 

(Variants of Residual Projection). Let $X\in {\mathbb{R}}^{n\times p}$ be a real-valued design matrix of rank $\rho <n$. The residual projection operator $I-P$ onto the orthogonal complement $\mathcal{C}{\left(X\right)}^{\perp }$ can be constructed by several exact procedures, each suited to different regimes of dimension, conditioning, and interpretability.

The following variants highlight that null-space, wedge-based, and determinant-based approaches provide geometrically transparent alternatives to inversion-based formulas in linear regression.

Proposition 3 

(Wedge–QR Null-Space Variant). Let $X\in {\mathbb{R}}^{n\times p}$ have rank ρ, and let $k=n-\rho$. Using the null-space + QR construction of Theorem 2, we obtain an orthonormal residual basis $N\in {\mathbb{R}}^{n\times k}$ spanning $\mathcal{C}{\left(X\right)}^{\perp }$ and hence the projector $I-P=N{N}^{\top }$. If, in addition, we compute a thin QR factorization $X=QR$, then the Gram matrix satisfies

$$G=X^{\top }X=R^{\top }R,det\left(G\right)=det{\left(R\right)}^2,$$

so the squared volume of the parallelotope spanned by the columns of X is given directly by $det{\left(R\right)}^2$. The general complexity of forming N and R is $\mathcal{O}(n{p}^2+nk)$.

Proof. 

The existence of an orthonormal basis N of $\mathcal{C}{\left(X\right)}^{\perp }$ and the identity $I-P=N{N}^{\top }$ follow from Theorem 2. The factorization $X=QR$ implies $G=X^{\top }X=R^{\top }R$, so $det\left(G\right)=det{\left(R\right)}^2$, which yields the stated volume formula. The complexity estimate is inherited from the null space + QR construction.    □

When geometric interpretations involving volume are necessary—such as evaluating multicollinearity through the Geometric Multicollinearity Index (GMI)—factorizations of the Gram matrix are particularly convenient.

Proposition 4 

(Cholesky-Based Volume and Projection). Let $G=X^{\top }X$ and assume that X has full-column-rank so that $G\succ 0$. Compute the Cholesky factorization $G=L{L}^{\top }$, which yields

$$det\left(G\right)=det{\left(L\right)}^2=\prod _i{L}_{ii}^2.$$

Let N be any orthonormal basis of $\mathcal{C}{\left(X\right)}^{\perp }$ constructed as in Theorem 2. Then $I-P=N{N}^{\top }$ is the orthogonal residual projector. The dominant cost is $\mathcal{O}(p^3+nk)$: $\mathcal{O}\left(p^3\right)$ for the Cholesky factorization and $\mathcal{O}\left(nk\right)$ for forming and applying N.

Proof. 

The positive definiteness of G ensures the existence of a unique Cholesky factor L with positive diagonal entries, and the determinant formula follows from $det\left(G\right)=det{\left(L\right)}^2={\prod }_i{L}_{ii}^2$. The construction of N and the identity $I-P=N{N}^{\top }$ is provided by Theorem 2; any such N produces an orthogonal projector onto $\mathcal{C}{\left(X\right)}^{\perp }$.    □

In the special case where the residual subspace is one-dimensional, the projector admits an especially simple representation.

Proposition 5 

(Cross-product (Rank-One Case)). If $dim\left(\mathcal{C}{\left(X\right)}^{\perp }\right)=1$, then

$$I-P=n{n}^{\top },$$

(19)

where $n\in {\mathbb{R}}^n$ satisfies $X^{\top }n=0$ and $\parallel n\parallel =1$. The one-time cost of building n is $\mathcal{O}\left(n^2\right)$, and each subsequent projection $(I-P)Y$ costs $\mathcal{O}\left(n\right)$.

Proof. 

When $\mathcal{C}{\left(X\right)}^{\perp }$ is one-dimensional, any nonzero vector in this space is a scalar multiple of a unit normal n. The orthogonal projector onto the line spanned by n is precisely $n{n}^{\top }$. Symmetry and idempotence follow from $\parallel n\parallel =1$, and orthogonality is ensured by $X^{\top }n=0$.    □

Example 1. 

Consider $n=4$, $p=3$ and the residual codimension $k=n-p=1$. In this codimension-one setting, the classical OLS residual projector based on ${\left(X^{\top }X\right)}^{-1}$ (assuming that X has full column rank) involves a matrix inversion together with several matrix–vector multiplications, which require on the order of a few hundred floating-point operations in this small example. By contrast, computing a unit normal vector n via the geometric cross-product costs $\mathcal{O}\left(n^2\right)$ operations, and each application of the rank-one projector $n{n}^{\top }$ to a new response Y costs only $\mathcal{O}\left(n\right)$ operations. Once n is constructed, the projection of additional responses is therefore substantially cheaper than recomputing residuals through ${\left(X^{\top }X\right)}^{-1}$, illustrating the efficiency of geometric formulation in symbolic or resource-limited regimes.

Unless stated otherwise, we use a single Gaussian sketch without power iterations or oversampling; orthonormalization is performed via a standard thin QR factorization, see Algorithm 2.

Algorithm 2 Sketch–QR Baseline (Gaussian Range Finder)

Require: Design matrix $X\in {\mathbb{R}}^{n\times p}$; sketch size $s\ll n$. Ensure: Approximate orthonormal basis $\tilde{Q}$ and projector $\tilde{P}$.    1: Draw a Gaussian sketching matrix $\Omega \in {\mathbb{R}}^{p\times s}$ with i.i.d. $\mathcal{N}(0,1)$ entries.    2: Form the sketched matrix $Y=X\Omega \in {\mathbb{R}}^{n\times s}$.    3: Compute a thin QR factorization $Y=\tilde{Q}R$.    4: Define the approximate projector $\tilde{P}=\tilde{Q}{\tilde{Q}}^{\top }$.    5: Compute the approximate residual $\tilde{r}=(I-\tilde{P})y$.

Table 1. Exact residual projection variants for $X\in {\mathbb{R}}^{n\times p}$ with rank $\rho$ and residual dimension $k=n-\rho$. The “one-off” cost refers to constructing the projector, while the “per-response” cost refers to applying the residual mapping $(I-P)Y$ to a new response vector Y. Methods involving ${\left(X^{\top }X\right)}^{-1}$ assume that X has full column rank so that $X^{\top }X$ is invertible.

Method	Residual $(\mathit{I}-\mathit{P})\mathit{Y}$	One-Off Cost	Per-Response Cost
Classical OLS (via ${\left(X^{\top }X\right)}^{-1}$)	$\left(I-X{\left(X^{\top }X\right)}^{-1}{X}^{\top }\right)Y$	$\mathcal{O}(n{p}^2+p^3)$	$\mathcal{O}\left(np\right)$
QR/null-space basis	$N{N}^{\top }Y$	$\mathcal{O}(n{p}^2+nk)$	$\mathcal{O}\left(nk\right)$
Cholesky-based (when $X^{\top }X\succ 0$)	$\left(I-X{\left(X^{\top }X\right)}^{-1}{X}^{\top }\right)Y$	$\mathcal{O}\left(p^3\right)$ (form $X^{\top }X$ and $L{L}^{\top }$)	$\mathcal{O}\left(np\right)$
Rank-one cross-product ($dim\mathcal{C}{\left(X\right)}^{\perp }=1$)	$n{n}^{\top }Y$	$\mathcal{O}\left(n^2\right)$ (construct n)	$\mathcal{O}\left(n\right)$

Notes: Classical OLS and the Cholesky-based variant treat the residual subspace implicitly. The QR/null space basis method also applies to rank-deficient X and explicitly constructs a basis N of $\mathcal{C}{\left(X\right)}^{\perp }$. The rank-one cross-product method applies when $dim\mathcal{C}{\left(X\right)}^{\perp }=1$ and is particularly efficient once the normal n is known.

compares four exact ways to build the residual projector $(I-P)$ for a given design matrix X. Classical OLS and the Cholesky-based variant both work through ${\left(X^{\top }X\right)}^{-1}$ and treat the residual subspace only implicitly, with a cubic dependence on the number of predictors p and the restriction that $X^{\top }X$ be invertible. In contrast, the Wedge–QR and rank-one cross-product methods construct $(I-P)$ from an explicit basis of $\mathcal{C}{\left(X\right)}^{\perp }$; the former handles general rank-deficient cases, while the latter is tailored to the codimension-one setting and achieves the lowest per-response cost when many responses must be projected.

In summary, Wedge–QR and Cholesky-based constructions are well suited for when one needs both an exact residual projector and access to volume-based quantities such as $det\left(X^{\top }X\right)$ or the GMI. The formulation of a rank-one cross-product is preferable whenever $dim\mathcal{C}{\left(X\right)}^{\perp }=1$, as it produces a simple geometric projector $n{n}^{\top }$ with linear complexity per response and, when $p\ll n$, is usually the cheapest option to repeatedly project many responses. For general rank-deficient designs, a QR/null-space construction offers a numerically safer alternative to explicitly forming and inverting $X^{\top }X$. Together, the four variants in Table 1 provide a practical menu of exact geometric projectors that avoid unnecessary matrix inversion and can be chosen according to the rank of X, the conditioning of $X^{\top }X$ and the available computational budget.

We emphasize that the exact residual projector $I-P$ can be constructed via several algebraically equivalent routes (normal-equations OLS when applicable, rank-one cross products, null-space/QR, wedge-based constructions and Cholesky-based variants), which differ only in numerical assumptions and computational cost; see Table 1.

4. Illustrative Numerical Examples

When the design matrix X is rank deficient, the normal equations $X^{\top }X\beta =X^{\top }Y$ do not admit a unique solution. In all rank-deficient experiments, we therefore compute the minimum-norm least-squares solution using the Moore–Penrose pseudoinverse, i.e., $\widehat{\beta }=X^{+}Y$, and refer to this baseline as OLS (Moore–Penrose pseudoinverse). The full implementation and experimental scripts are publicly available [13].

Predictive performance is assessed using K-fold cross-validation with $K=5$, reporting CV MSE and CV $R^2$. Unless stated otherwise, all randomized procedures use a fixed random seed (seed = 42) to ensure reproducibility.

Coefficient stability is evaluated via $B=100$ perturbation trials with the design matrix X held fixed. In each trial, the response is perturbed as

$${\tilde{y}}^{\left(b\right)}=y+{\eta }^{\left(b\right)},{\eta }^{\left(b\right)}\sim \mathcal{N}\left(0,{\sigma }_y^{2}I\right),{\sigma }_y=0.01\mathrm{std}\left(y\right),b=1,\dots ,B.$$

The model is refit for each trial to obtain coefficients ${\widehat{\beta }}^{\left(b\right)}$. We let ${\mathrm{SD}}_j$ denote the sample standard deviation of the jth coefficient ${\{{\widehat{\beta }}_j^{\left(b\right)}\}}_{b=1}^B$ across trials. We summarize coefficient stability by

$$\mathrm{medianSD}\left(\widehat{\beta }\right):={\mathrm{median}}_{j=1,\dots ,p}\left({\mathrm{SD}}_j\right),$$

i.e., the median across coefficients of their across-trial standard deviations.

To validate the geometric projector, we verify agreement between the classical OLS residual $r_{\mathrm{OLS}}=(I-P)y$ and the geometric residual $r_{\mathrm{geo}}=N{N}^{\top }y$ by monitoring $\parallel r_{\mathrm{OLS}}-r_{\mathrm{geo}}{\parallel }_2$ up to numerical tolerance.

Sketch–QR [11] yields an approximate basis $\tilde{Q}$ and projector $\tilde{P}=\tilde{Q}{\tilde{Q}}^{\top }$, producing the approximate residual $\tilde{r}=(I-\tilde{P})y$. Agreement with the exact residual is summarized using cosine similarity $cos(r_{\mathrm{OLS}},\tilde{r})$ and the residual discrepancy $\parallel \tilde{r}-r_{\mathrm{OLS}}{\parallel }_2$ (and, when reported, a principal-angle distortion between $\mathcal{C}\left(X\right)$ and $\mathcal{C}\left(\tilde{Q}\right)$).

4.1. Rank-One Residual Projection in ${\mathbb{R}}^3$

This example explicitly shows how the normality of the cross-product reproduces the ordinary least-squares (OLS) residual in the rank-one case, in line with Proposition 1.

We consider a rank-one residual setting where the orthogonal complement of the column space is one-dimensional. Let us

$$\mathbf{X}=\left[\begin{array}{cc}1& 2\\ 1& 3\\ 1& 5\end{array}\right],\mathbf{Y}=\left[\begin{array}{c}1\\ 2\\ {\displaystyle \frac{11}{2}}\end{array}\right],$$

with $\mathbf{X}\in {\mathbb{R}}^{3\times 2}$ of full-column-rank (so ${\mathbf{X}}^{\top }\mathbf{X}$ is invertible). The ordinary least-squares projection matrix is

$$\mathbf{P}=\mathbf{X}{\left({\mathbf{X}}^{\top }\mathbf{X}\right)}^{-1}{\mathbf{X}}^{\top },{\mathbf{r}}_{\mathrm{OLS}}=(\mathbf{I}-\mathbf{P})\mathbf{Y}.$$

A direct calculation yields

$${\mathbf{r}}_{\mathrm{OLS}}=\left[\begin{array}{c}{\displaystyle \frac{3}{14}}\\ -{\displaystyle \frac{9}{28}}\\ {\displaystyle \frac{3}{28}}\end{array}\right],{\mathbf{r}}_{\mathrm{OLS}}\in \mathcal{C}{\left(\mathbf{X}\right)}^{\perp }.$$

To recover the same residual in purely geometric form, we construct a unit normal vector to the column space of $\mathbf{X}$. Let us

$${\mathbf{x}}_1=\left[\begin{array}{c}1\\ 1\\ 1\end{array}\right],{\mathbf{x}}_2=\left[\begin{array}{c}2\\ 3\\ 5\end{array}\right],{\mathbf{n}}_0={\mathbf{x}}_1\times {\mathbf{x}}_2=\left[\begin{array}{c}2\\ -3\\ 1\end{array}\right],\mathbf{n}={\displaystyle \frac{{\mathbf{n}}_0}{\parallel {\mathbf{n}}_0\parallel }}={\displaystyle \frac{1}{\sqrt{14}}}\left[\begin{array}{c}2\\ -3\\ 1\end{array}\right].$$

Then ${\mathbf{X}}^{\top }\mathbf{n}=0$ and $\parallel \mathbf{n}\parallel =1$, so Proposition 1 applies, and

$$\mathbf{I}-\mathbf{P}=\mathbf{n}{\mathbf{n}}^{\top }.$$

The residual can therefore be written as the rank-one projection

$$\mathbf{r}=(\mathbf{I}-\mathbf{P})\mathbf{Y}=\mathbf{n}{\mathbf{n}}^{\top }\mathbf{Y}=\left({\mathbf{n}}^{\top }\mathbf{Y}\right)\mathbf{n}=\left[\begin{array}{c}{\displaystyle \frac{3}{14}}\\ -{\displaystyle \frac{9}{28}}\\ {\displaystyle \frac{3}{28}}\end{array}\right],$$

which coincides with ${\mathbf{r}}_{\mathrm{OLS}}$ and lies on the line spanned by $\mathbf{n}$.

From a computational point of view, this example highlights the attraction of the geometric formulation in low dimension: the cross-product and the multiplication of a scalar vector are sufficient to obtain the residual, avoiding explicit matrix inversion. Once $\mathbf{n}$ is known, applying the projector $(\mathbf{I}-\mathbf{P})=\mathbf{n}{\mathbf{n}}^{\top }$ to a new response vector reduces to one dot product and one scaled copy of $\mathbf{n}$, i.e., $\mathcal{O}\left(n\right)$ work per residual.

4.2. Multivector Residual Projection in ${\mathbb{R}}^4$

This example illustrates the multivector residual projection via an orthonormal basis of $\mathcal{C}{\left(\mathbf{X}\right)}^{\perp }$ as in Theorem 2 and links it to the Geometric Multicollinearity Index.

We now consider a multivector setting in which the residual subspace has a dimension greater than one. Let us

$$\mathbf{X}=\left[\begin{array}{cc}1& 1\\ 1& 2\\ 1& 3\\ 1& 4\end{array}\right],\mathbf{Y}=\left[\begin{array}{c}1\\ 2\\ 3\\ 5\end{array}\right],\mathbf{X}\in {\mathbb{R}}^{4\times 2},\rho =2.$$

The ordinary least-squares projector is again

$$\mathbf{P}=\mathbf{X}{\left({\mathbf{X}}^{\top }\mathbf{X}\right)}^{-1}{\mathbf{X}}^{\top },{\mathbf{r}}_{\mathrm{OLS}}=(\mathbf{I}-\mathbf{P})\mathbf{Y},$$

and a straightforward computation gives

$${\mathbf{r}}_{\mathrm{OLS}}=\left[\begin{array}{c}{\displaystyle \frac{1}{5}}\\ -{\displaystyle \frac{1}{10}}\\ -{\displaystyle \frac{2}{5}}\\ {\displaystyle \frac{3}{10}}\end{array}\right].$$

Since $\rho =2$, its orthogonal complement has dimension $dim\mathcal{C}{\left(\mathbf{X}\right)}^{\perp }=2$. A basis for the null space of ${\mathbf{X}}^{\top }$ is

$${\mathbf{n}}_1=\left[\begin{array}{c}1\\ -2\\ 1\\ 0\end{array}\right],{\mathbf{n}}_2=\left[\begin{array}{c}2\\ -3\\ 0\\ 1\end{array}\right],{\mathbf{X}}^{\top }{\mathbf{n}}_i=0,i=1,2.$$

Applying Gram–Schmidt yields an orthonormal basis

$${\mathbf{u}}_1={\displaystyle \frac{1}{\sqrt{6}}}\left[\begin{array}{c}1\\ -2\\ 1\\ 0\end{array}\right],{\mathbf{u}}_2={\displaystyle \frac{1}{\sqrt{30}}}\left[\begin{array}{c}2\\ -1\\ -4\\ 3\end{array}\right],\mathbf{N}=\left[{\mathbf{u}}_1{\mathbf{u}}_2\right]\in {\mathbb{R}}^{4\times 2},$$

with ${\mathbf{N}}^{\top }\mathbf{N}={\mathbf{I}}_2$ and ${\mathbf{X}}^{\top }\mathbf{N}=0$. The general residual projection theorem then gives

$$\mathbf{I}-\mathbf{P}=\mathbf{N}{\mathbf{N}}^{\top },{\mathbf{r}}_{\mathrm{geom}}=\mathbf{N}{\mathbf{N}}^{\top }\mathbf{Y}.$$

Evaluating ${\mathbf{r}}_{\mathrm{geom}}$ for this $\mathbf{X}$ and $\mathbf{Y}$ reproduces the residual OLS:

$${\mathbf{r}}_{\mathrm{geom}}={\mathbf{r}}_{\mathrm{OLS}}=\left[\begin{array}{c}{\displaystyle \frac{1}{5}}\\ -{\displaystyle \frac{1}{10}}\\ -{\displaystyle \frac{2}{5}}\\ {\displaystyle \frac{3}{10}}\end{array}\right],$$

making explicit that the residual lives in the two-dimensional subspace spanned by ${\mathbf{u}}_1$ and ${\mathbf{u}}_2$ and realizing the null space + QR construction in Theorem 2, in a concrete low-dimensional setting.

From a complexity point of view, building $\mathbf{N}$ on a null-space basis followed by QR (or Gram–Schmidt) is a one-off cost of order $\mathcal{O}(n{p}^2+nk)$ for $\mathbf{X}\in {\mathbb{R}}^{n\times p}$ with residual dimension $k=n-\rho$, while applying the projector $(\mathbf{I}-\mathbf{P})=\mathbf{N}{\mathbf{N}}^{\top }$ to a new response vector requires only $\mathcal{O}\left(nk\right)$ operations.

4.2.1. GMI for the ${\mathbb{R}}^4$ Example

For this design matrix $\mathbf{X}=[{\mathbf{x}}_1,{\mathbf{x}}_2]$, the Gram matrix is

$$\mathbf{G}={\mathbf{X}}^{\top }\mathbf{X}=\left[\begin{array}{cc}4& 10\\ 10& 30\end{array}\right],det\left(\mathbf{G}\right)=20.$$

The column norms are $\parallel {\mathbf{x}}_1\parallel =2$ and $\parallel {\mathbf{x}}_2\parallel =\sqrt{30}$, so the polar sine and the Geometric Multicollinearity Index are

$$\mathrm{psin}({\mathbf{x}}_1,{\mathbf{x}}_2)={\displaystyle \frac{\sqrt{det\left({\mathbf{X}}^{\top }\mathbf{X}\right)}}{\parallel {\mathbf{x}}_1\parallel \parallel {\mathbf{x}}_2\parallel }}={\displaystyle \frac{\sqrt{20}}{2\sqrt{30}}}={\displaystyle \frac{\sqrt{6}}{6}},\mathrm{GMI}=1-{\displaystyle \frac{\sqrt{6}}{6}}\approx 0.59.$$

In this small example, the GMI value $\mathrm{GMI}\approx 0.59$ indicates a moderate degree of collinearity between the two regressors; the residual subspace $\mathcal{C}{\left(\mathbf{X}\right)}^{\perp }$ has dimension two (as $n-r=2$), and, compared to the orthonormal case ($\mathrm{GMI}=0$), the fitted coefficients are more sensitive to perturbations of $\mathbf{Y}$.

This example isolates the effect of controlled collinearity on GMI in a minimal symbolic setting.

Consider the family of matrices

$${\mathbf{X}}_{\epsilon }=\left[\begin{array}{cc}1& 0\\ 0& 1\\ \epsilon & \epsilon \end{array}\right],\epsilon \in [0,1].$$

(20)

For every $\epsilon \in [0,1]$ the two columns are linearly independent, so ${\mathbf{X}}_{\epsilon }^{\top }{\mathbf{X}}_{\epsilon }$ is positive definite, and the polar-sine is well defined. A direct computation gives

$$\mathrm{psin}\left({\mathbf{X}}_{\epsilon }\right)={\displaystyle \frac{\sqrt{1+2{\epsilon }^2}}{1+{\epsilon }^2}},$$

(21)

and hence

$$\mathrm{GMI}\left(\epsilon \right)=1-{\displaystyle \frac{\sqrt{1+2{\epsilon }^2}}{1+{\epsilon }^2}}.$$

(22)

As $\epsilon$ increases, the columns become more collinear, $\mathrm{psin}$ decreases, and $\mathrm{GMI}$ increases, reflecting the growing multicollinearity in a purely geometric way. In families where columns eventually become exactly collinear, the Gram determinant vanishes, the polar sine drops to zero, and $\mathrm{GMI}$ reaches 1, in agreement with the rank-deficient discussion in Section 3.

4.2.2. Large-Scale Multivector Residual Projection

To complement the low-dimensional symbolic example above, we consider a larger synthetic design in which the residual subspace has a dimension greater than one and the multivector formulation $I-P=N{N}^{\top }$ is essential.

We generate a rank-deficient design matrix $X\in {\mathbb{R}}^{n\times p}$ with $n=1000$, $p=50$, and $\rho =45$, so that $dim\mathcal{C}{\left(X\right)}^{\perp }=n-\rho =955$. All predictors are standardized, and the response is constructed so that the exact residual $r_{\mathrm{exact}}$ has a prescribed norm, ensuring a controlled comparison between methods.

Table 2. Large-scale synthetic multivector residual projection (core methods). The design matrix has $n=1000$, $p=50$, $\rho =45$, and residual codimension $dim\mathcal{C}{\left(X\right)}^{\perp }=955$. All predictors are standardized. Exact projector variants reproduce the same residual up to numerical precision, while Sketch–QR produces a controlled approximation whose agreement is summarized by cosine similarity. Entries marked as FAIL indicate that the algebraic assumptions of the method (e.g., positive definiteness of $X^{\top }X$ for Cholesky-based constructions) are violated, rather than an algorithmic failure.

#	Method	${\parallel \mathit{r}\parallel }_2$	$cos(\mathit{r},{\mathit{r}}_{\mathbf{exact}})$	Runtime (s)	Status
1	OLS (Moore–Penrose pseudoinverse)	0.5092	1.000	0.0002	OK
2	Cross-product (rank one)	NaN	NaN	NaN	N/A
3	Multivector QR (Exact)	0.5092	1.000	0.0000	OK
4	Recursive Cross–Wedge–QR	0.5092	1.000	0.0008	OK
5	Wedge–QR (+GMI)	0.5092	1.000	0.0000	OK
6	Cholesky–QR	NaN	NaN	0.0016	FAIL
7	Sketch–QR (Gaussian, $s=150$)	0.5092	1.000	0.0111	OK
Dataset diagnostics: $\mathrm{cond}\left(X\right)\approx 6.28\times {10}^{13}$, $\kappa \left(X^{\top }X\right)\approx 3.95\times {10}^{15}$, $\mathrm{GMI}\left(X\right)=1.000$.

reports residual norms, cosine similarity to the exact residual, and run times for the variants of the Article 1 projector only. The exact constructions (null space/QR, rank-one cross-product when applicable, and the Moore–Penrose pseudoinverse baseline for OLS) reproduce the same residual up to numerical precision, confirming the multivector identity $I-P=N{N}^{\top }$ in a high-dimensional setting.

Sketch–QR yields an approximate residual, whose agreement with the exact residual is summarized by cosine similarity. Entries marked NaN or N/A indicate methods that are not applicable in this regime (e.g., rank-one cross-product when $dim\mathcal{C}{\left(X\right)}^{\perp }\ne 1$), while FAIL indicates that the required algebraic conditions (e.g., $X^{\top }X\succ 0$) are violated.

This experiment demonstrates that the geometric residual formulation extends naturally from symbolic low-dimensional examples to realistic large-n, rank-deficient designs, without relying on regularization or nonlinear modeling.

Together with the symbolic R⁴ example, this large-scale synthetic experiment confirms that the multivector residual projector $N{N}^{\top }$ is exact in dimensions, while Sketch–QR provides a principled approximation when a reduced basis is desired, setting the stage for the real-data illustrations in Section 4.3.

4.3. Illustrative Real Data Use of GMI

Real-data experiments use two regression benchmarks: Boston Housing and Auto MPG. Both are standardized via centering and scaling before forming X, and they provide realistic multicollinearity patterns suitable for illustrating GMI as a purely geometric, response-independent diagnostic.

This example illustrates how GMI can serve as a purely geometric multicollinearity diagnostic on real data and how it can be interpreted alongside coefficient stability.

We use the Boston Housing dataset because it is a widely used regression benchmark with well-known predictor correlations, making it a convenient setting for illustrating multicollinearity diagnostics and residual geometry on realistic data.

To illustrate the geometric diagnostics on real data, we use the classical Boston Housing dataset, with median house price as the response and a set of standardized socio-economic and structural predictors. The protocol used in this subsection is as follows:

(a)

Standardize all predictors to zero mean and unit variance, and form the design matrix X.

(b)

For selected subsets of predictors, form the corresponding design matrix X, compute the Gram matrix $G=X^{\top }X$, and evaluate the polar sine and GMI using the definitions in (18).

(c)

Report predictive metrics using five-fold cross-validation (CV) so that MSE and $R^2$ are out-of-sample summaries rather than in-sample fit statistics.

(d)

Fit ordinary least-squares models on these subsets and monitor the variation of the estimated coefficients under small perturbations of Y (e.g., additive Gaussian noise or resampling).

The purpose of this subsection is to demonstrate (i) that the geometric residual projector $(I-P)=N{N}^{\top }$ reproduces the classical OLS residuals on a realistic dataset and (ii) that GMI provides a compact, response-independent summary of predictor-space degeneracy.

4.3.1. Boston Housing and Auto MPG: Increasing Model Size

Real-data experiments use two regression benchmarks: Boston Housing and Auto MPG. Both are standardized through centering and scaling before forming X and provide realistic multicollinearity patterns suitable to illustrate GMI as a purely geometric and response-independent diagnostic.

Table 3. Boston Housing illustration (standardized predictors). Predictive metrics are reported using 5-fold cross-validation (CV MSE and CV $R^2$). Geometric diagnostics (GMI) are response-independent and depend only on the design matrix X. We additionally report max VIF and $\kappa \left(X^{\top }X\right)$, and summarize coefficient stability by $\mathrm{medianSD}\left(\widehat{\beta }\right)$ under resampling/perturbation, to connect GMI with classical multicollinearity measures.

Model	p	CV MSE	$\mathbf{CV}{\mathit{R}}^2$	GMI	max VIF	$\mathit{\kappa }\left({\mathit{X}}^{\mathit{\top }}\mathit{X}\right)$	$\mathbf{medianSD}\left(\widehat{\mathit{\beta }}\right)$
$p=1$ (Area/RM → Price/MEDV)	1	44.02	0.467	0.000	1.00	1.00	0.342
$p=4$ (Area/RM, TAX, PTRATIO, LSTAT)	4	28.02	0.655	0.434	2.09	7.85	0.257
$p\ge 6$ (Area/RM, LSTAT, PTRATIO, TAX, NOX, INDUS)	6	28.20	0.653	0.790	3.25	17.22	0.266
Full model (all predictors)	13	23.67	0.712	0.988	9.01	96.47	0.235

and

Table 4. Auto MPG illustration (standardized predictors), using the same preprocessing and evaluation protocol as in Section 4.3. Predictive metrics are reported using 5-fold cross-validation (CV MSE and CV $R^2$). Geometric diagnostics (GMI) are response-independent and depend only on the design matrix X. We additionally report max VIF and $\kappa \left(X^{\top }X\right)$, and summarize coefficient stability by $\mathrm{medianSD}\left(\widehat{\beta }\right)$ under resampling/perturbation.

Model	p	CV MSE	$\mathbf{CV}{\mathit{R}}^2$	GMI	max VIF	$\mathit{\kappa }\left({\mathit{X}}^{\mathit{\top }}\mathit{X}\right)$	$\mathbf{medianSD}\left(\widehat{\mathit{\beta }}\right)$
$p=1$ model (Auto)	1	18.91	0.682	0.000	1.00	1.00	0.118
$p=3$ engine-core (Auto)	3	18.22	0.693	0.843	10.31	45.06	0.367
$p\ge 6$ model (Auto)	6	12.05	0.797	0.973	19.64	117.03	0.325
Full model (Auto)	7	11.31	0.810	0.980	21.84	136.84	0.355

report results for increasing the size of the model ($p=1$, $p=4$, $p\ge 6$, and in full where applicable). For each subset, we report out-of-sample predictive metrics using five-fold cross-validation (CV MSE and CV $R^2$), geometric multicollinearity (GMI), and classical diagnostics (max VIF and $\kappa \left(X^{\top }X\right)$), along with a summary of coefficient stability.

We restrict algorithmic comparisons to the projector variants defined in Table 1 (OLS/Cholesky when applicable, null space/QR, rank-one cross-product when $\mathrm{codim}=1$, and Sketch–QR).

Table 3 and Table 4 show that increasing the size of the model improves predictive performance (lower CV MSE, higher CV $R^2$) while typically increasing multicollinearity. In Boston, moving from $p=1$ to the full model reduces CV MSE from $44.02$ to $23.67$ and increases CV $R^2$ from $0.467$ to $0.712$, while GMI increases from 0 to $0.988$ and $\kappa \left(X^{\top }X\right)$ increases from 1 to $96.47$. In Auto MPG the collinearity is even stronger: the full model attains CV MSE $11.31$ and CV $R^2$ $0.810$, but the corresponding diagnostics are GMI $\approx 0.980$, max VIF $\approx 21.84$, and $\kappa \left(X^{\top }X\right)\approx 136.84$.

These results illustrate the intended role of GMI: it is computed purely from X (independent of Y) and tracks familiar multicollinearity measures (VIF and condition number), while the coefficient stability summary provides a concrete link between predictor space degeneracy and sensitivity of fitted parameters.

4.3.2. Implementation Note (Stable GMI Computation)

When the Gram matrix $G=X^{\top }X$ is poorly conditioned, the evaluation of direct determinants is numerically fragile. We therefore compute the polar sine and GMI in the log-domain with a small jitter $\epsilon >0$:

$$log{\mathrm{psin}}_{\epsilon }\left(G\right)={\displaystyle \frac{1}{2}}logdet(G+\epsilon I)-{\displaystyle \frac{1}{2}}\sum _{i=1}^{p}log(G_{ii}+\epsilon ),{\mathrm{GMI}}_{\epsilon }\left(G\right)=1-exp\left(log{\mathrm{psin}}_{\epsilon }\left(G\right)\right).$$

In implementation, $logdet(G+\epsilon I)$ is evaluated using a numerically stable log-determinant routine (equivalently, a Cholesky factorization when $G+\epsilon I\succ 0$). Unless stated otherwise, we use $\epsilon ={10}^{-12}$.

Figure 4 makes the equivalence of the exact geometric projector and classical OLS residuals explicit: the discrepancy curve for the exact QR-basis construction is at a numerical noise level in both the $p=1$ and $p=4$ models. In contrast, Sketch–QR constructs an approximate projector $\tilde{P}$, so its residual $\tilde{r}$ differs slightly from $r_{\mathrm{OLS}}$; the magnitude of this discrepancy decreases as the size of the sketch s increases. Overall, this visualization complements Table 3 and Table 4 by separating the exact residual-geometry identities (Article 1 methods) from the approximate Sketch–QR construction while keeping the presentation compact and reproducible.

5. Discussion

This paper presents a rank-aware geometric interpretation of linear regression residuals via orthogonal projectors. In the case of codimension one, the residual is fully determined by a unit normal $n\in \mathcal{C}{\left(X\right)}^{\perp }$, resulting in $r=(I-P)Y=n{n}^{\top }Y$; in general rank settings, the residual lies in a higher-dimensional orthogonal complement spanned by an orthonormal basis N, with $(I-P)=N{N}^{\top }$. This formulation makes the residual subspace explicit and decouples basis construction from repeated projection.

An immediate consequence is that all exact computational routes—normal-equation OLS, Cholesky-based solvers, and null space or QR constructions—must yield identical residuals up to numerical tolerance, as they implement the same projector onto $\mathcal{C}{\left(X\right)}^{\perp }$. We confirm this numerically through residual agreement metrics and discrepancy visualizations, showing that differences arise only from conditioning and the computational regime. In contrast, Sketch–QR yields an approximate projector and residual, whose deviation from the exact solution is quantified using cosine similarity and norm-based discrepancies.

We further link residual geometry to predictor-space conditioning through the Geometric Multicollinearity Index (GMI), a scale-invariant, normalized volume measure derived from the polar sine. Across the Boston Housing and Auto MPG benchmarks, higher GMI values consistently align with larger max VIF and $\kappa \left(X^{\top }X\right)$, while coefficient stability summaries illustrate increased parameter sensitivity under geometric degeneracy.

Although both regression and PCA rely on orthogonal projections, their objectives differ fundamentally: PCA projects predictors to minimize reconstruction error, with residuals remaining in $\mathcal{C}\left(X\right)$, whereas regression projects the response, yielding residuals in $\mathcal{C}{\left(X\right)}^{\perp }$.

Finally, this work focuses on unregularized least squares and exact orthogonal projection. With regularization, the operators are no longer orthogonal and the residual subspace becomes non-Euclidean or data-adaptive, motivating future work on regularized and sketch-aware residual geometry while preserving the geometric diagnostics enabled by GMI.

6. Conclusions

This paper presented a rank-aware geometric interpretation of linear regression residuals. In the case of codimension-one, the residual projector reduces to the rank-one form $(I-P)=n{n}^{\top }$, where n spans $\mathcal{C}{\left(X\right)}^{\perp }$. For general rank, the residual lies in a $k=n-\rho$ dimensional orthogonal complement, with a projector $(I-P)=N{N}^{\top }$ for an orthonormal basis N of $\mathcal{C}{\left(X\right)}^{\perp }$. Although algebraically equivalent to the classical OLS formulation, this representation makes the residual subspace explicit and decouples basis construction from repeated projection.

Building on this viewpoint, we introduced the Geometric Multicollinearity Index (GMI), a scale-invariant diagnostic derived from the polar sine and Gram determinant that quantifies predictor-space degeneracy. Synthetic experiments confirm predictable behavior under controlled perturbations, while real-data benchmarks on Boston Housing and Auto MPG show that increasing model size typically improves predictive performance while increasing multicollinearity; GMI captures this trade-off and aligns with max VIF and $\kappa \left(X^{\top }X\right)$. Across all experiments, exact projector constructions yield identical residuals up to numerical tolerance, whereas Sketch–QR produces a controlled approximation whose deviation is quantified via cosine similarity and residual discrepancy norms.

In general, these results elevate $\mathcal{C}{\left(X\right)}^{\perp }$ to a central geometric object in regression analysis and position GMI as a concise indicator of departure from orthogonal designs. Extensions to sketch-based, large-scale, streaming, and nonlinear settings are deferred to companion work.