A Safeguarded Transport-Extrapolated Proximal Gradient Method on the Stiefel Manifold

Abstract

Motivated by sparse principal component analysis (PCA), this paper proposes ATE-ManPG, a safeguarded transport-extrapolated proximal gradient method for minimizing the sum of a smooth function and a convex, possibly nonsmooth function over the Stiefel manifold. The method augments the manifold proximal gradient method (ManPG) with two ingredients: momentum obtained by transporting the previous proximal search direction to the current tangent space, protected by a monotone restart safeguard, and a bounded adaptive proximal parameter. For the exact idealized iteration, global convergence to stationary points and $O(1/{\epsilon }^2)$ iteration complexity are proved; under a proximal Kurdyka–Łojasiewicz condition with exponent $1/2$ and a relative-error condition, both stated as explicit hypotheses, the convergence is R-linear. Experiments on a sparse-PCA model, validated by an external residual certificate at the returned point, show a $3.6\times$ mean iteration-count reduction relative to fixed-parameter ManPG; a factorial ablation attributes most of this reduction to the adaptive parameter, while transported momentum adds a further $6.4\%$ on average, concentrated in one paired trial.

1. Introduction

The Stiefel manifold $\mathrm{St}(n,r)=\{X\in {\mathbb{R}}^{n\times r}:X^{\top }X=I_r\}$ captures orthogonality constraints essential to sparse principal component analysis (PCA) [1], compressed modes [2], feature selection [3], and blind deconvolution [4]. This paper considers the composite problem

$$\underset{X\in \mathrm{St}(n,r)}{\min }F\left(X\right):=f\left(X\right)+h\left(X\right),$$

(1)

where f is smooth with L-Lipschitz gradient and h is convex and possibly nonsmooth.

Chen, Ma, So, and Zhang [5] proposed the manifold proximal gradient method (ManPG), a retraction-based algorithm that solves a strongly convex subproblem in the tangent space at each iteration via a semismooth Newton (SSN) method. ManPG converges globally to stationary points with $O(1/{\epsilon }^2)$ iteration complexity and was selected for republication in the SIGEST section of SIAM Review in 2024 [6], testifying to its practical impact and its role in unifying the theory of nonsmooth Riemannian optimization.

1.1. Motivation and Related Work

Two lines of research motivate the present work: acceleration on manifolds and convergence under geometric conditions.

Accelerated proximal gradient on manifolds. Huang and Wei [7] extended the fast iterative shrinkage-thresholding algorithm (FISTA) [8,9] to the Stiefel manifold via inverse retraction and a periodic safeguard; their Riemannian proximal gradient (RPG) framework [10] gives $O(1/k)$ rates under retraction-convexity and local KL rates, but leaves the vanilla accelerated variant (V-ARPG) unanalyzed. Accelerated Riemannian proximal rates under curvature/convexity assumptions appear in Feng et al. [11]. Related Euclidean theory includes nonconvex acceleration [12,13] and adaptive restart [14]. Other Stiefel tools include Riemannian proximal Newton [15], inexact RPG [16], and recent inertial methods [17]. Overviews and orthogonality-constrained first-order frameworks are [18,19].

Penalty and constraint-handling on Stiefel manifolds. Smooth exact penalties for orthogonality constraints and ${\ell }_{2,1}$ problems [20,21,22], a penalty-free infeasible method with tangential and normal steps [23], constraint dissolving [24,25], and multiplier/parallel schemes [26,27] form alternative families; they are not drop-in replacements of ManPG tangent subproblems.

Adaptive ManPG. Chen et al. [5] already propose ManPG-Ada (their Algorithm 2). Adaptive proximal parameters are therefore not new here; we focus on transported-direction extrapolation with a monotone safeguard and on when a KL rate is preserved.

KL-based local linear convergence. The Kurdyka–Łojasiewicz (KL) inequality [28,29,30] yields linear rates when $\theta =1/2$ and a relative-error bound holds. Liu, So, and Wu [31] prove $\theta =1/2$ for smooth quadratics on $\mathrm{St}(n,r)$. Wang, Liu, and So [32] study robust ${\ell }_1$-PCA ($\max \parallel D^{\top }{Q\parallel }_1$), not the penalized variance objective. Huang and Wei [10] obtain a Riemannian KL property with unspecified exponent. Semialgebraicity alone does not prescribe $\theta =1/2$ for $-\mathrm{Tr}\left(X^{\top }{A}^{\top }AX\right)+\mu {\parallel X\parallel }_1$; see Remark 1.

1.2. Our Contributions

This paper studies transported-direction extrapolation of the ManPG search direction with monotone restart and a bounded adaptive proximal parameter. Contributions:

1.

Algorithm. ATE-ManPG (adaptive transport-extrapolated ManPG, Algorithm 1): orthogonal-projection transport (no inverse retraction), bounded ${\beta }_k\le {\beta }_{\max }$, objective-based restart, adaptive $t\in [1/L,10/L]$, and return of the work point where the residual is certified. One extra retraction and function evaluation per iteration relative to ManPG.

2.

Theory. Sufficient decrease (Lemma 1), global subsequential stationarity (Theorem 1), $O\left({\epsilon }^{-2}\right)$ iteration complexity (Theorem 2), and a conditional R-linear rate (Theorem 3) via a standard KL argument, uniform in adaptive $t\in [1/L,10/L]$. We do not claim $\theta =1/2$ for every sparse-PCA stationary point (Remark 1); relative error is verified under smoothness or sign compatibility (Proposition 1).

3.

Experiments. ATE-ManPG is compared with ManPG, ManPG-Ada, TE-ManPG, AManPG, P-ARPG, and RPG on sparse PCA from archived scalar records with an external certificate. A harmonized factorial ablation shows adaptation explains most of the total speedup; transported momentum adds $6.4\%$ mean iterations on top of adaptation (concentrated in one paired trial), with runtime overhead and occasional quality tradeoffs. Cap sensitivity covers ${\beta }_{\max }\in [0.2,1]$.

Algorithm 1 ATE-ManPG for solving (1)

Require:  $X_0\in \mathrm{St}(n,r)$, $t_0=1/L$, $t=t_0$, $t_{\max }=10/L$, ${\epsilon }_{\mathrm{stop}}>0$ (tolerance on $\parallel V_k{\parallel }_{\mathrm{F}}$), ${\beta }_{\max }>0$ (default $0.5$), optional floor ${\alpha }_{\mathrm{tol}}\ge 0$ 1: $D_{\mathrm{prev}}\leftarrow 0$,    ${\alpha }_{\mathrm{prev}}\leftarrow 1$ 2: for  $k=0,1,2,\dots$ do 3:     if $k=0$ or $D_{\mathrm{prev}}=0$ then 4:         ${\beta }_k\leftarrow 0$,    $X_{\mathrm{work}}\leftarrow X_k$ 5:     else 6:         $\tilde{D}\leftarrow {\mathrm{Proj}}_{T_{X_k}\mathrm{St}(n,r)}\left({\alpha }_{\mathrm{prev}}{D}_{\mathrm{prev}}\right)$ 7:         ${\beta }_k\leftarrow \min \left({\displaystyle \frac{k}{k+3}},{\beta }_{\max }\right)$         ▹ matches the archived implementation 8:         $X_{\mathrm{work}}\leftarrow {\mathrm{Retr}}_{X_k}\left({\beta }_k\tilde{D}\right)$ 9:         if $F\left(X_{\mathrm{work}}\right)>F\left(X_k\right)$then             ▹ monotone restart safeguard 10:            $X_{\mathrm{work}}\leftarrow X_k$,    $D_{\mathrm{prev}}\leftarrow 0$,    ${\beta }_k\leftarrow 0$ 11:         end if 12:     end if 13:     Compute $\nabla f\left(X_{\mathrm{work}}\right)$ 14:     Solve (7) at $X_{\mathrm{work}}$ to obtain $V_k$ 15:     if $\parallel V_k{\parallel }_{\mathrm{F}}\le {\epsilon }_{\mathrm{stop}}$ then return $X_{\mathrm{work}}$           ▹ certify the returned point 16:     end if 17:     $\alpha \leftarrow 1$ 18:     while $F\left({\mathrm{Retr}}_{X_{\mathrm{work}}}\left(\alpha V_k\right)\right)>F\left(X_{\mathrm{work}}\right)-{\displaystyle \frac{\alpha }{4t}}{\parallel V_k\parallel }_{\mathrm{F}}^2$ and $\alpha >{\alpha }_{\mathrm{tol}}$ do 19:         $\alpha \leftarrow 0.5\alpha$ 20:     end while 21:     if the Armijo test fails at the final $\alpha$ (evaluated once more) then return status “linesearch failed” 22:     end if 23:     $X_{k+1}\leftarrow {\mathrm{Retr}}_{X_{\mathrm{work}}}\left(\alpha V_k\right)$ 24:     $D_{\mathrm{prev}}\leftarrow V_k$,    ${\alpha }_{\mathrm{prev}}\leftarrow \alpha$ 25:                     ▹ Adaptive proximal parameter $t_k$: 26:     if $\alpha =1$ then 27:         $t\leftarrow \min (t_{\max },t·1.02)$ 28:     else if $\alpha =0.5$ then 29:         $t\leftarrow t$ 30:     else 31:         $t\leftarrow \max (t_0,t/1.1)$ 32:     end if 33: end for

2. Preliminaries

2.1. Stiefel Manifold Geometry

The Stiefel manifold $\mathrm{St}(n,r)=\{X\in {\mathbb{R}}^{n\times r}:X^{\top }X=I_r\}$ is an embedded submanifold of ${\mathbb{R}}^{n\times r}$ of dimension $nr-r(r+1)/2$, compact when $n\ge r$. The tangent space at X is

$$T_X\mathrm{St}(n,r)=\{V\in {\mathbb{R}}^{n\times r}:X^{\top }V+V^{\top }X=0\}.$$

(2)

Equipped with the Frobenius inner product $⟨A,B⟩=\mathrm{Tr}\left(A^{\top }B\right)$, the orthogonal projection onto $T_X\mathrm{St}(n,r)$ is

$${\mathrm{Proj}}_{T_X\mathrm{St}(n,r)}\left(Z\right)=Z-Xsym\left(X^{\top }Z\right),$$

(3)

where $sym\left(A\right)=(A+A^{\top })/2$. The projection is non-expansive: $\parallel {\mathrm{Proj}}_{T_X\mathrm{St}(n,r)}{\left(Z\right)\parallel }_F\le {\parallel Z\parallel }_F$.

A retraction ${\mathrm{Retr}}_X:T_X\mathrm{St}(n,r)\to \mathrm{St}(n,r)$ satisfies ${\mathrm{Retr}}_X\left(0\right)=X$, and its derivative at the origin is the identity, ${\displaystyle \frac{d}{dt}}{\mathrm{Retr}}_X\left(tV\right){|}_{t=0}=V$ for every tangent V. For the polar retraction

$${\mathrm{Retr}}_X\left(V\right)=(X+V){(I+V^{\top }V)}^{-1/2},$$

the matrix $I+V^{\top }V$ is positive definite for every tangent V, so the map is globally defined, and one has the global bounds (no radius restriction)

$$\begin{array}{cc}\hfill \parallel {\mathrm{Retr}}_X{\left(V\right)-(X+V)\parallel }_F& \le \frac{1}{2}{\parallel V\parallel }_F^2,\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill \parallel {\mathrm{Retr}}_X{\left(V\right)-X\parallel }_F& \le {\parallel V\parallel }_F.\hfill \end{array}$$

(5)

Thus we may take $M_1=1/2$ and $M_2=1$ below; see also [33,34].

For a smooth function $f:\mathrm{St}(n,r)\to \mathbb{R}$, the Riemannian gradient is $\mathrm{grad}f\left(X\right)={\mathrm{Proj}}_{T_X\mathrm{St}(n,r)}(\nabla f\left(X\right))$. A point $X^{*}\in \mathrm{St}(n,r)$ is stationary for (1) if

$$0\in \mathrm{grad}f\left(X^{*}\right)+{\mathrm{Proj}}_{T_{X^{*}}\mathrm{St}(n,r)}(\partial h\left(X^{*}\right)),$$

(6)

where $\partial h\left(X^{*}\right)$ is the limiting subdifferential of h in the ambient space [5].

2.2. The ManPG Framework

At iteration k, ManPG [5] solves the strongly convex subproblem in the tangent space:

$$V_k={argmin}_{V\in T_{X_k}\mathrm{St}(n,r)}⟨\nabla f\left(X_k\right),V⟩+{\displaystyle \frac{1}{2t}}{\parallel V\parallel }_{\mathrm{F}}^2+h(X_k+V),$$

(7)

where $t=1/L$, and updates $X_{k+1}={\mathrm{Retr}}_{X_k}\left({\alpha }_k{V}_k\right)$ with Armijo backtracking on ${\alpha }_k$.

The subproblem (7) is solved by semismooth Newton (SSN) on the dual system [5]: a symmetric multiplier $\Lambda \in {\mathbb{R}}^{r\times r}$ yields $q=r(r+1)/2$ variables; the dense code factors a $q\times q$ system ($O\left(r^6\right)$ with assembly), with conjugate gradient (CG) for larger r. For $h={\mu \parallel ·\parallel }_1$ the prox is element-wise soft-thresholding. Warm starts give a few inner iterations, but the theory assumes exact minimizers (Section 4; Remark 3 quantifies how the inexact inner solves used in practice relate to that framework).

2.3. Vector Transport via Orthogonal Projection

Parallel transport costs $O\left(n^3\right)$ on $\mathrm{St}(n,r)$; Huang and Wei [7] use inverse retraction ($O\left(r^3\right)$ via a Lyapunov equation). We use orthogonal projection as vector transport [33]:

$${\Gamma }_{X\to Y}\left(D\right):={\mathrm{Proj}}_{T_Y\mathrm{St}(n,r)}\left(D\right),\parallel {\Gamma }_{X\to Y}{\left(D\right)\parallel }_F\le {\parallel D\parallel }_F,$$

at cost $O\left(n{r}^2\right)$ (one product and symmetrization). Boundedness, not isometry, is enough for our momentum step; no linear system is required.

2.4. Kurdyka–Łojasiewicz Inequality

Let $\Phi$ be a proper lower semicontinuous function. Following the standard local form [30], $\Phi$ satisfies the KL inequality at $z^{*}\in dom\partial \Phi$ with exponent $\theta \in [0,1)$ if there exist $\eta >0$, a neighborhood $\mathcal{U}$ of $z^{*}$, and $\delta >0$ such that

$$\Phi \left(z^{*}\right)<\Phi \left(z\right)<\Phi \left(z^{*}\right)+\delta ,z\in \mathcal{U}⟹{\left(\Phi \left(z\right)-\Phi \left(z^{*}\right)\right)}^{\theta }\le \eta dist(0,\partial \Phi \left(z\right)).$$

(8)

On $\mathrm{St}(n,r)$ we work with the ambient composite $F=f+h$ and the projected stationarity measure $dist(0,{\mathrm{Proj}}_{T_X\mathrm{St}}(\partial F\left(X\right)))$, writing “proximal KL” when (8) is used with that measure. The exponent controls the local rate: $\theta =0$ finite termination; $\theta \in (0,1/2]$ linear; $\theta \in (1/2,1)$ sublinear [30]. The theoretical residual is $\parallel V_k{\parallel }_F$ (equivalently ${\min }_{i\le k}{\parallel V_i\parallel }_F$); experiments threshold the scaled squared residual $\parallel V_k{\parallel }_F^2/t^2$ against a tolerance proportional to $nr$ (Section 5)—a different quantity from the norm accuracy $\epsilon$ in Theorem 2.

Remark 1 (Scope of exponent 1/2).

We do not claim that the penalized objective

$$F\left(X\right)=-\mathrm{Tr}\left(X^{\top }{A}^{\top }AX\right)+\mu {\parallel X\parallel }_1$$

has KL exponent $1/2$ at every stationary point. Explicitly, on $\mathrm{St}(2,1)=S^1$ with $\mu =1$ and

$$B={\displaystyle \frac{1}{250}}\left(\begin{array}{cc}337& 91\\ 91& 338\end{array}\right),x_{*}={(3/5,4/5)}^{\top },w={(-4/5,3/5)}^{\top },$$

set $x\left(s\right)=x_{*}\cos s+w\sin s$. Both coordinates stay positive for small $\left|s\right|$, so $F\left(x\right)=-x^{\top }Bx+{\parallel x\parallel }_1$ is smooth on that arc; $x_{*}$ is stationary and

$$F\left(x\left(s\right)\right)-F\left(x_{*}\right)=-\frac{1}{10}{s}^3+O\left(s^4\right),|{(F\circ x)}^{\prime }\left(s\right)|=\frac{3}{10}{s}^2+{O\left(\right|s|}^3).$$

On the positive-gap side $s<0$, $\sqrt{F-F^{*}}/\parallel \mathrm{grad}F\parallel$ diverges, so no finite KL constant with exponent $1/2$ exists (local exponent $2/3$). This disproves an unrestricted stationary-point claim; it does not assert that every local minimizer has exponent $2/3$. Liu, So, and Wu [31] prove $\theta =1/2$ for smooth quadratics $\mathrm{Tr}\left(X^{\top }QX\right)$; Wang, Liu, and So [32] analyze robust ${\ell }_1$-PCA ($\max \parallel D^{\top }{Q\parallel }_1$), not the penalized variance objective; Huang and Wei [10] obtain a Riemannian KL property for semialgebraic restrictions with unspecified exponent. Semialgebraicity guarantees some KL exponent, not $1/2$. Theorem 3 is therefore conditional: it applies at limit points where exponent $1/2$ and Assumption 1 are both verified (e.g., under suitable nondegeneracy on an identified smooth stratum).

3. The ATE-ManPG Algorithm

3.1. Transport-Based Search-Direction Extrapolation

Classical FISTA forms $y_k=x_k+{\beta }_k(x_k-x_{k-1})$ [8,9]. On a manifold, $X_k-X_{k-1}$ need not lie in $T_{X_k}\mathrm{St}(n,r)$. We instead transport the previous proximal search direction  $V_{k-1}$ (already a tangent vector at the previous work point):

1.

Transport: ${\tilde{D}}_{k-1}={\mathrm{Proj}}_{T_{X_k}\mathrm{St}(n,r)}\left({\alpha }_{k-1}{V}_{k-1}\right)$.

2.

Extrapolate: $X_k^{\mathrm{ext}}={\mathrm{Retr}}_{X_k}\left({\beta }_k{\tilde{D}}_{k-1}\right)$.

3.

Solve: At $X_k^{\mathrm{ext}}$, solve the ManPG subproblem to obtain $V_k$, and retract to obtain $X_{k+1}$.

Even on a flat manifold this is not FISTA: transported-direction extrapolation behaves like $y_k=x_k+{\beta }_k(x_k-y_{k-1})$, not $x_k+{\beta }_k(x_k-x_{k-1})$.

Relative to Huang and Wei [7]: (i) no inverse retraction; (ii) bounded ${\beta }_k\le {\beta }_{\max }$ (default $0.5$) as a design choice (FISTA’s ${\beta }_k\uparrow 1$ stays finite); (iii) monotone restart if extrapolation increases F (Section 3.2). Related: [14].

3.2. Algorithm Description

Idealized iteration versus implementation. Section 4 analyzes the exact idealized iteration: exact subproblems, unrestricted backtracking (${\alpha }_{\mathrm{tol}}=0$), and no positive numerical stopping when infinite sequences are studied. A finite-floor implementation may return a linesearch-failure status; the theorems do not cover failed runs. The archived MATLAB code uses ${\alpha }_{\mathrm{tol}}={10}^{-4}$; here ${\alpha }_{\mathrm{tol}}$ is the Armijo step-size floor of the backtracking loop, not the accuracy of the inner solver. The SSN subproblems are solved to an inner tolerance tied to the outer stopping rule: $\mathrm{inner}\_\mathrm{tol}=\max ({10}^{-13},\min ({10}^{-11},{10}^{-3}ϵt^2))$ on the squared dual residual, tightened to $\max (5\times {10}^{-16},\min ({10}^{-14},{10}^{-5}ϵt^2))$ whenever the line search struggles. Here $ϵ$ is the experimental tolerance on $\parallel V_k{\parallel }_{\mathrm{F}}^2/t^2$ of Section 5 (Algorithm 1 states the idealized norm test with ${\epsilon }_{\mathrm{stop}}$). Remark 3 and (18) state in what sense this inexactness affects the theory. In that sense, the theoretical results support the implemented algorithm as follows: the safeguard, the extrapolation step, and the acceptance test are implemented verbatim; under the relative inner stopping rule just described, inexactness degrades only the constants of Lemma 1 and Theorems 1–2 (Remark 3); and every reported run is validated by an external certificate—one additional SSN solve at the returned point—whose verdict is independent of the inner accuracy used during the run, with finite-floor failures reported as failures rather than successes. The archived code accepts a run only if an external residual certificate at the returned point passes. Failure is decided by the Armijo predicate after the loop, not merely by reaching the floor.

Output certificate. $\parallel V_k{\parallel }_F$ is computed at $X_{\mathrm{work}}$, so we return $X_{\mathrm{work}}$, not $X_k$.

Armijo. Trials are compared with $F\left(X_{\mathrm{work}}\right)$ (valid after restart). Lemma 1 gives $F\left({\mathrm{Retr}}_Y\left(\alpha V\right)\right)\le F\left(Y\right)-{\displaystyle \frac{\alpha }{t}}{\parallel V\parallel }_F^2+K{\alpha }^2{\parallel V\parallel }_F^2$, so ${\displaystyle \frac{\alpha }{4t}}{\parallel V\parallel }_F^2$ is guaranteed for $\alpha \le 3/\left(4Kt\right)$; unrestricted halving accepts some ${\alpha }_k\ge \min \{1,3/\left(8Kt\right)\}$.

Restart and ${\beta }_{\max }$. If extrapolation increases F, we set $X_{\mathrm{work}}\leftarrow X_k$ and clear the momentum buffer. ${\beta }_{\max }=0.5$ is a tuning default; Lemma 1 does not depend on it (Table 1).

Table 1. Cap sensitivity for ATE-ManPG ($n=500$, $r=5$, $\mu =0.8$, $ϵ={10}^{-8}nr$, 10 trials; all certified $10/10$).

${\mathit{\beta }}_{\max }$	0.2	0.5	0.8	1.0
Iterations	$205\pm 63$	$206\pm 70$	$204\pm 69$	$205\pm 69$
Restarts/run	104	148	163	168

The adaptive update of $t_k$ uses only backtracking information already computed. The analysis of Section 4 is uniform over $t\in [1/L,10/L]$; see Remark 7. We write $t_k$ when the proximal parameter varies.

3.3. Comparison with Prior Accelerated Methods

AManPG (accelerated ManPG) [7]. Nesterov sequence $s_{k+1}=(1+\sqrt{1+4s_k^2})/2$ and inverse-retraction extrapolation $y_{k+1}={\mathrm{Retr}}_{x_{k+1}}\left({\displaystyle \frac{1-s_k}{s_{k+1}}}{R}_{x_{k+1}}^{-1}\left(x_k\right)\right)$ (moving away from the old iterate), with a periodic safeguard that has a stationarity guarantee; no accelerated rate in that paper.

ARPG/P-ARPG (accelerated RPG variants) [10]. RPG rates as above; the vanilla variant V-ARPG is left open; the practical variant P-ARPG uses restarts.

Feng et al. [11]. Accelerated rates under geodesic convexity/curvature, different subproblem and safeguards.

ATE-ManPG. Differences: (i) projected search-direction extrapolation (no inverse retraction or iterate combination); (ii) objective-based restart every iteration; (iii) conditional KL rate for the safeguarded tangent-linearized ManPG iteration. We do not claim the first accelerated rate for all Riemannian proximal methods, nor a worst-case order improvement over ManPG.

4. Convergence Analysis

We establish the theoretical properties of the exact idealized iteration of Algorithm 1: exact subproblem minimizers, unrestricted backtracking (${\alpha }_{\mathrm{tol}}=0$), and no positive numerical stopping when infinite sequences are studied. The analysis covers both a fixed parameter $t=1/L$ and the adaptive rule of Section 3.2 with $t\in [1/L,10/L]$ (Remark 7); the inexactness of the inner SSN solves used by the implementation is treated in Remark 3. It proceeds in four stages: sufficient decrease, global convergence, worst-case complexity, and improved complexity under the KL condition.

Standing assumptions and constants. f is the restriction of a smooth ambient function with L-Lipschitz gradient near $\mathrm{St}(n,r)$; h is finite-valued convex on ${\mathbb{R}}^{n\times r}$. Model and retracted points lie in a compact tube; $\mathrm{Lip}\left(h\right)$ is a Lipschitz constant of h there and $M_{\Lambda }$ bounds Karush–Kuhn–Tucker (KKT) multipliers ${\parallel \Lambda \parallel }_2$ on that tube. Let $G:={\max }_{\mathrm{St}}{\parallel \nabla f\parallel }_F$, $K:=L/2+M_1(G+\mathrm{Lip}\left(h\right))$, $M_1=1/2$, $C_{\mathrm{gap}}:=M_2{\beta }_{\max }={\beta }_{\max }$, $D_k:={\alpha }_k{V}_k$, $D_{-1}:=0$, $F_{\mathrm{cl}}:={\lim }_{k}F\left(X_k\right)$. In Theorem 2, $F^{*}$ may be any $F_{\mathrm{lb}}\le F$; in Theorem 3, $F^{*}:=F_{\mathrm{cl}}$.

4.1. Sufficient Decrease

Lemma 1 (Sufficient decrease).

Let $\left\{X_k\right\}$ be generated by Algorithm 1, and let $X_k^{\mathrm{ext}}$ denote $X_{\mathrm{work}}$ at iteration k (so $X_k^{\mathrm{ext}}=X_k$ after a restart). Then $F\left(X_k^{\mathrm{ext}}\right)\le F\left(X_k\right)$ for every k. If $t>0$ is fixed, backtracking accepts some

$${\alpha }_k\ge \overline{\alpha }:=\min \left\{1,{\displaystyle \frac{3}{8Kt}}\right\}>0$$

(9)

and

$$F\left(X_{k+1}\right)-F\left(X_k\right)\le -{\displaystyle \frac{{\alpha }_k}{4t}}\parallel V_k{\parallel }_{\mathrm{F}}^2\le -{\displaystyle \frac{\overline{\alpha }}{4t}}{\parallel V_k\parallel }_{\mathrm{F}}^2.$$

(10)

If $t_k\in [1/L,10/L]$ varies, the same holds uniformly with

$${\overline{\alpha }}_{\min }:=\min \left\{1,{\displaystyle \frac{3}{8K{t}_{\max }}}\right\},a_{\min }:={\displaystyle \frac{{\overline{\alpha }}_{\min }}{4t_{\max }}}>0,$$

(11)

so ${\alpha }_k\ge {\overline{\alpha }}_{\min }$ and $F\left(X_{k+1}\right)-F\left(X_k\right)\le -a_{\min }{\parallel V_k\parallel }_F^2$ for all k.

Proof. 

The restart safeguard yields $F\left(X_k^{\mathrm{ext}}\right)\le F\left(X_k\right)$.

Let $Y:=X_k^{\mathrm{ext}}$ and $V:=V_k$. Optimality of (7) and convexity of h give the strong model inequality

$$⟨\nabla f\left(Y\right),V⟩+h(Y+V)-h\left(Y\right)\le -{\displaystyle \frac{1}{t}}{\parallel V\parallel }_{\mathrm{F}}^2,$$

(12)

because the first-order condition at V paired with V yields $⟨\nabla f\left(Y\right),V⟩+{\displaystyle \frac{1}{t}}{\parallel V\parallel }^2+⟨g,V⟩=0$ for some $g\in \partial h(Y+V)$, and $⟨g,V⟩\ge h(Y+V)-h\left(Y\right)$ by convexity. (Testing the objective at V versus 0 gives the weaker bound with ${\displaystyle \frac{1}{2t}}{\parallel V\parallel }^2$; we use the stronger form.)

By L-Lipschitz continuity of $\nabla f$ along the polar path, the retraction bound (4) with $M_1=1/2$, and convexity of h,

$$\begin{array}{cc}\hfill f\left({\mathrm{Retr}}_Y\left(\alpha V\right)\right)& \le f\left(Y\right)+\alpha ⟨\nabla f\left(Y\right),V⟩+\frac{L}{2}{\alpha }^2{\parallel V\parallel }_{\mathrm{F}}^2+G{M}_1{\alpha }^2{\parallel V\parallel }_{\mathrm{F}}^2,\hfill \end{array}$$

(13)

$$\begin{array}{cc}\hfill h\left({\mathrm{Retr}}_Y\left(\alpha V\right)\right)& \le h(Y+\alpha V)+\mathrm{Lip}\left(h\right)M_1{\alpha }^2{\parallel V\parallel }_{\mathrm{F}}^2\hfill \end{array}$$

(14)

$$\begin{array}{cc}\hfill & \le (1-\alpha )h\left(Y\right)+\alpha h(Y+V)+\mathrm{Lip}\left(h\right)M_1{\alpha }^2{\parallel V\parallel }_{\mathrm{F}}^2.\hfill \end{array}$$

(15)

Summing and applying (12),

$$F\left({\mathrm{Retr}}_Y\left(\alpha V\right)\right)\le F\left(Y\right)-{\displaystyle \frac{\alpha }{t}}{\parallel V\parallel }_{\mathrm{F}}^2+K{\alpha }^2{\parallel V\parallel }_{\mathrm{F}}^2.$$

(16)

The Armijo target $F\left({\mathrm{Retr}}_Y\left(\alpha V\right)\right)\le F\left(Y\right)-{\displaystyle \frac{\alpha }{4t}}{\parallel V\parallel }^2$ holds whenever $K\alpha \le 3/\left(4t\right)$, i.e., for all $\alpha \le 3/\left(4Kt\right)$. Halving from $\alpha =1$ therefore accepts some ${\alpha }_k\ge \min \{1,3/\left(8Kt\right)\}=\overline{\alpha }$, and

$$F\left(X_{k+1}\right)\le F\left(X_k^{\mathrm{ext}}\right)-{\displaystyle \frac{{\alpha }_k}{4t}}\parallel V_k{\parallel }_{\mathrm{F}}^2\le F\left(X_k\right)-{\displaystyle \frac{{\alpha }_k}{4t}}{\parallel V_k\parallel }_{\mathrm{F}}^2.$$

(17)

For $t\le t_{\max }$ the uniform envelopes follow as stated. □

Remark 2.

Lemma 1 uses only the monotone restart (not decay of $\parallel V_k\parallel$), so there is no circular dependence on later theorems. When extrapolation is not monotone, momentum is discarded and the step reduces to ManPG at $X_k$.

Remark 3 (Inexact inner solves).

The SSN solver returns a direction whose subproblem optimality condition (7) holds up to a dual residual $E_k$ with $\parallel E_k{\parallel }_F=:{\delta }_k$, the quantity monitored by the SSN stopping test. Pairing the perturbed condition with $V_k$ exactly as in the proof of Lemma 1 yields, in place of (12),

$$⟨\nabla f\left(Y\right),V⟩+h(Y+V)-h\left(Y\right)\le -{\displaystyle \frac{1}{t}}{\parallel V\parallel }_{\mathrm{F}}^2+{\delta }_k{\parallel V\parallel }_{\mathrm{F}},$$

(18)

and hence $F\left({\mathrm{Retr}}_Y\left(\alpha V\right)\right)\le F\left(Y\right)-{\displaystyle \frac{\alpha }{t}}{\parallel V\parallel }_{\mathrm{F}}^2+K{\alpha }^2{\parallel V\parallel }_{\mathrm{F}}^2+\alpha {\delta }_k{\parallel V\parallel }_{\mathrm{F}}$. If the inner solver is stopped by a relative rule

$${\delta }_k\le c_0{\displaystyle \frac{\parallel V_k{\parallel }_{\mathrm{F}}}{t_k}},c_0\in (0,\frac{3}{4}),$$

(19)

then the last term is absorbed as $c_0{\displaystyle \frac{\alpha }{t_k}}{\parallel V_k\parallel }_{\mathrm{F}}^2$, the Armijo predicate holds for every α with $K\alpha \le {\displaystyle \frac{3/4-c_0}{t_k}}$, and the conclusions of Lemma 1, Theorem 1, and Theorem 2 persist verbatim with $\overline{\alpha }$, ${\overline{\alpha }}_{\min }$, and $a_{\min }$ multiplied by $1-{\displaystyle \frac{4c_0}{3}}$, all uniformly over $t\in [1/L,10/L]$. The archived code enforces a rule of the form (19): an iterate is processed only while $\parallel V_k{\parallel }_{\mathrm{F}}^2/t_k^2\ge ϵ$, and the branch $\mathrm{inner}\_\mathrm{tol}\le {10}^{-3}ϵt_k^2$ of Section 3.2 then gives ${\delta }_k\le \sqrt{{10}^{-3}}{t}_k\sqrt{ϵ}\le 0.032{\parallel V_k\parallel }_{\mathrm{F}}$, i.e., (19) with $c_0=0.032t_k\le 0.32/L\le 0.16$ on the sparse-PCA instances of Section 5, where column normalization gives $L\ge 2$. Inexactness therefore degrades constants, not the qualitative guarantees. At tolerances so tight that the absolute floor ${10}^{-13}$ dominates the ${10}^{-3}ϵt_k^2$ branch, the relative guarantee (19) ceases to bind; this is precisely the regime of the high-precision failures discussed in Section 5.3. A complete inexact analysis in the spirit of [16] is left as future work (Section 5.4).

4.2. Global Convergence and Worst-Case Complexity

Theorem 1 (Global convergence).

Every limit point of $\left\{X_k\right\}$ generated by Algorithm 1 is a stationary point of (1). Moreover $\parallel V_k{\parallel }_F\to 0$ and $\parallel X_k^{\mathrm{ext}}-X_k{\parallel }_F\to 0$.

Proof. 

From Lemma 1, $\left\{F\left(X_k\right)\right\}$ is nonincreasing. Compactness of $\mathrm{St}(n,r)$ and continuity of F imply $\left\{F\left(X_k\right)\right\}$ converges. Summing the descent inequality gives ${\sum }_{i=0}^{\infty }{\parallel V_i\parallel }_F^2<\infty$, hence $\parallel V_k{\parallel }_F\to 0$. By () and non-expansiveness of projection,

$$\parallel X_k^{\mathrm{ext}}-X_k{\parallel }_{\mathrm{F}}\le M_2{\beta }_k\parallel {\tilde{D}}_{k-1}{\parallel }_{\mathrm{F}}\le C_{\mathrm{gap}}\parallel D_{k-1}{\parallel }_{\mathrm{F}}\le C_{\mathrm{gap}}{\parallel V_{k-1}\parallel }_{\mathrm{F}}\to 0.$$

(20)

Let $X_{*}$ be a limit point with subsequence $\left\{X_{k_j}\right\}\to X_{*}$. Then also $X_{k_j}^{\mathrm{ext}}\to X_{*}$ and $X_{k_j}^{\mathrm{ext}}+V_{k_j}\to X_{*}$. The subproblem optimality condition at $Y_j:=X_{k_j}^{\mathrm{ext}}$ is

$$0\in \nabla f\left(Y_j\right)+{\displaystyle \frac{1}{t_j}}{V}_{k_j}+\partial h(Y_j+V_{k_j})+{\mathcal{A}}_{Y_j}^{*}\left({\Lambda }_j\right),$$

(21)

where $t_j\in [1/L,10/L]$, ${\mathcal{A}}_X^{*}(\Lambda )=X\Lambda$ with $\Lambda ={\Lambda }^{\top }$. Since $\parallel {\mathcal{A}}_{Y_j}^{*}\left({\Lambda }_j\right){\parallel }_F={\parallel {\Lambda }_j\parallel }_F$ and, choosing $g_j\in \partial h(Y_j+V_{k_j})$ in the inclusion,

$$\parallel {\Lambda }_j{\parallel }_{\mathrm{F}}\le \parallel \nabla f\left(Y_j\right){\parallel }_{\mathrm{F}}+{\displaystyle \frac{\parallel V_{k_j}{\parallel }_{\mathrm{F}}}{t_j}}+\parallel g_j{\parallel }_{\mathrm{F}}\le G+L{\parallel V_{k_j}\parallel }_{\mathrm{F}}+\mathrm{Lip}\left(h\right),$$

(22)

the sequence $\left\{{\Lambda }_j\right\}$ is bounded. Passing to a further subsequence with ${\Lambda }_j\to {\Lambda }_{*}$ (symmetric) and $g_j\to g_{*}$, outer semicontinuity of the limiting subdifferential of the convex function h [35] yields $g_{*}\in \partial h\left(X_{*}\right)$, and continuity of $\nabla f$ gives

$$0\in \nabla f\left(X_{*}\right)+\partial h\left(X_{*}\right)+{\mathcal{A}}_{X_{*}}^{*}\left({\Lambda }_{*}\right).$$

(23)

Projecting onto $T_{X_{*}}\mathrm{St}(n,r)$ and noting ${\mathrm{Proj}}_{T_{X_{*}}\mathrm{St}}\left({\mathcal{A}}_{X_{*}}^{*}\left({\Lambda }_{*}\right)\right)=0$ because ${\Lambda }_{*}$ is symmetric yields the stationarity condition (6). □

Theorem 2 (Worst-case complexity).

Algorithm 1 requires $O\left({\epsilon }^{-2}\right)$ iterations to achieve ${\min }_{i<k}{\parallel V_i\parallel }_F\le \epsilon$.

Proof. 

From Lemma 1, telescoping yields ${\min }_{i<k}{\parallel V_i\parallel }_F^2\le {\displaystyle \frac{F\left(X_0\right)-F_{\mathrm{lb}}}{a_{\min }}}·{\displaystyle \frac{1}{k}}$ for any lower bound $F_{\mathrm{lb}}\le F$, giving ${\min }_{i<k}{\parallel V_i\parallel }_F=O(1/\sqrt{k})$, i.e., $O\left({\epsilon }^{-2}\right)$ iterations. □

4.3. Conditional Linear Convergence Under the KL Condition

We state a conditional local rate. Its inputs are (i) proximal KL with exponent $1/2$ at the limit set and (ii) a relative-error bound linking projected stationarity at the accepted point to previous steps. Neither hypothesis is verified here for all sparse-PCA instances (Remark 1). Relative error of this type is standard for proximal algorithms [29,30], but for nonsmooth h the subproblem supplies a subgradient at the affine model point $Y+V$, while the accepted iterate is ${\mathrm{Retr}}_Y\left(\alpha V\right)$; small distance between those points does not by itself transfer subgradients across a kink.

Assumption 1 (Lagged relative error).

There exist $b_1,b_2>0$ such that for all sufficiently large k,

$$dist\left(0,{\mathrm{Proj}}_{T_{X_k}\mathrm{St}(n,r)}(\partial F\left(X_k\right))\right)\le b_1\parallel V_{k-1}{\parallel }_{\mathrm{F}}+b_2{\parallel D_{k-1}\parallel }_{\mathrm{F}}.$$

(24)

Proposition 1 (Sufficient conditions for Assumption 1).

Let Y be a work point, V the exact subproblem direction, $Z=Y+V$, and $X={\mathrm{Retr}}_Y\left(\alpha V\right)$ the accepted point with $\alpha \in (0,1]$. Suppose there exist $g_Z\in \partial h\left(Z\right)$ and $g_X\in \partial h\left(X\right)$ with $\parallel g_X-g_Z\parallel \le c_h{\parallel V\parallel }_F$, and let $M_{\Lambda }$ bound the KKT multiplier. Then

$$dist\left(0,{\mathrm{Proj}}_{T_X\mathrm{St}}(\partial F\left(X\right))\right)\le (2L+c_h+M_{\Lambda }){\parallel V\parallel }_F.$$

(25)

In particular: (a) if h is $C^1$ with $L_h$-Lipschitz gradient near $Y,Z,X$, take $c_h=2L_h$; (b) if $h={\mu \parallel ·\parallel }_1$ and $Z,X$ are sign-compatible on the nonzero coordinates of X (same signs; zeros of X admit $[-\mu ,\mu ]$), take $g_X=g_Z$ and $c_h=0$. Numeric observation of a stable support is not a proof of (b).

Proof. 

The subproblem KKT system yields a symmetric $\Lambda$ with ${\parallel \Lambda \parallel }_2\le M_{\Lambda }$ and $g_Z\in \partial h\left(Z\right)$ such that $\nabla f\left(Y\right)+t^{-1}V+g_Z+Y\Lambda =0$. Since $\alpha \le 1$ and $M_2=1$, $\parallel X-Y\parallel \le {\parallel V\parallel }_F$ and $\parallel X-Z\parallel \le {2\parallel V\parallel }_F$. Using ${\mathrm{Proj}}_{T_X}(X\Lambda )=0$, nonexpansiveness of projection, $\parallel \nabla f\left(X\right)-\nabla f\left(Y\right)\parallel \le L\parallel X-Y\parallel$, $1/t\le L$, and $\parallel g_X-g_Z\parallel \le c_h{\parallel V\parallel }_F$,

$$dist(0,{\mathrm{Proj}}_{T_X}\partial F\left(X\right))\le \parallel {\mathrm{Proj}}_{T_X}(\nabla f\left(X\right)+g_X)\parallel \le (2L+c_h+M_{\Lambda }){\parallel V\parallel }_F.$$

Under unrestricted backtracking $\alpha \ge {\overline{\alpha }}_{\min }>0$, so ${\parallel V\parallel }_F\le {\overline{\alpha }}_{\min }^{-1}{\parallel D\parallel }_F$ and Assumption 1 follows on the accepted step. □

Remark 4 (Scope of Assumption 1).

Without Proposition 1 (for example at a kink of h with incompatible signs), $\parallel V_k{\parallel }_F$ remains the primary residual in Theorem 2, and the rate below stays an abstract implication. We do not claim that Assumption 1 holds for every sparse-PCA instance.

Theorem 3 (Conditional linear convergence under KL).

Let $F^{*}:=F_{\mathrm{cl}}={\lim }_{k}F\left(X_k\right)$. Assume the proximal KL inequality holds with exponent $\theta =1/2$ on a neighborhood of the limit set of $\left\{X_k\right\}$ in the one-sided window $F^{*}<F\left(X\right)<F^{*}+\delta$, and that Assumption 1 holds. Then there exist $k_0$ and $\rho \in (0,1)$ such that for $k\ge k_0$,

$${\Delta }_{k+1}\le \rho {\Delta }_k,{\parallel V_k\parallel }_F\le C{\rho }^{(k-k_0)/2},$$

(26)

where ${\Delta }_k:=F\left(X_k\right)-F^{*}$. Consequently ${\min }_{i\le k}{\parallel V_i\parallel }_F\le \epsilon$ in $O\left(\log \right(1/\epsilon \left)\right)$ iterations.

Proof. 

Set $p_k:={\parallel V_k\parallel }_F$. By Lemma 1, ${\Delta }_k-{\Delta }_{k+1}\ge a{p}_k^2$ with $a:=a_{\min }>0$ uniform. KL with $\theta =1/2$ and Assumption 1 give, for $k\ge k_0$,

$${\Delta }_k^{1/2}\le \eta dist\left(0,{\mathrm{Proj}}_{T_{X_k}\mathrm{St}}(\partial F\left(X_k\right))\right)\le \eta (b_1{p}_{k-1}+b_2\parallel D_{k-1}{\parallel }_F)\le \eta (b_1+b_2)p_{k-1},$$

(27)

using $\parallel D_{k-1}{\parallel }_F={\alpha }_{k-1}{p}_{k-1}\le p_{k-1}$. In particular ${\Delta }_k\le c_{\mathrm{KL}}^2{p}_{k-1}^2$ with $c_{\mathrm{KL}}:=\eta (b_1+b_2)$. Shifting the index, ${\Delta }_{k+1}\le c_{\mathrm{KL}}^2{p}_k^2$, while ${\Delta }_k-{\Delta }_{k+1}\ge a{p}_k^2$ implies $p_k^2\le ({\Delta }_k-{\Delta }_{k+1})/a$, so

$${\Delta }_{k+1}\le {\displaystyle \frac{c_{\mathrm{KL}}^2}{a+c_{\mathrm{KL}}^2}}{\Delta }_k=:\rho {\Delta }_k,\rho :={\displaystyle \frac{c_{\mathrm{KL}}^2}{a+c_{\mathrm{KL}}^2}}\in (0,1).$$

(28)

This is a standard one-step KL contraction: a momentum-energy Lyapunov term is not required once (24) is in force. R-linear decay of $p_k$ follows from $a{p}_k^2\le {\Delta }_k-{\Delta }_{k+1}\le {\Delta }_k\le {\Delta }_{k_0}{\rho }^{k-k_0}$. □

Remark 5 (Uniformization and scope).

If the proximal KL inequality holds uniformly on a compact neighborhood of the compact limit set intersected with a sublevel set, then $k_0<\infty$. Under Theorem 3, $\parallel X_{k+1}-X_k\parallel \le M_2\parallel V_k\parallel +C_{\mathrm{gap}}\parallel V_{k-1}\parallel$ is summable, so $\left\{X_k\right\}$ has finite length and converges to one stationary point. For ManPG, ${\beta }_k=0$ and $X_{\mathrm{work}}=X_k$; the accepted step $D_{k-1}$ remains the natural step measure.

Remark 6 (Role of the bounded schedule).

${\beta }_{\max }=0.5$ is a tuning default. Polar bounds $M_1=1/2$, $M_2=1$ are global; FISTA’s ${\beta }_k\uparrow 1$ stays bounded; Lemma 1 has no ${\beta }_{\max }$ dependence. Restart, not the cap, gives monotonicity (Table 1). We do not claim that the cap prevents basin transitions. Restart enforces $F\left(X_{\mathrm{work}}\right)\le F\left(X_k\right)$; objective monotonicity does not rule out basin transitions in a nonconvex problem.

Remark 7 (Adaptive t).

$a_{\min }$, ${\overline{\alpha }}_{\min }$, and $1/t\le L$ are uniform on $t\in [1/L,10/L]$, so Algorithm 1’s adaptive rule preserves the results with those envelopes. Adaptation is not proved to improve the rate relative to fixed $t=1/L$.

In summary, the adaptive parameter $t_k\in [1/L,10/L]$ is theoretically harmless (uniform constants, Remark 7); the theorems assume exact solves and unrestricted backtracking (Section 3.2), and Remark 3 records that the relative inner stopping rule used by the implementation degrades only the constants of those theorems.

5. Numerical Experiments

We evaluate ATE-ManPG on the sparse-PCA model $f\left(X\right)=-\mathrm{Tr}\left(X^{\top }{A}^{\top }AX\right)$, $h\left(X\right)={\mu \parallel X\parallel }_1$, with $A\in {\mathbb{R}}^{50\times n}$ (columns normalized). Baselines:

• ManPG [5]: fixed $t=1/L$, SSN subproblem, Armijo constant $1/\left(2t\right)$.
• ManPG-Ada [5]: the adaptive proximal-parameter variant of Chen et al. (their Algorithm 2), included as the natural adaptive baseline. Our implementation adapts t from the backtracking count (increase by $1.01$ without backtracking, decrease by $1.01$ otherwise) and, for comparability with the ATE family, caps $t\in [1/L,10/L]$; we do not claim identical update constants to their pseudocode.
• TE-ManPG (transport-extrapolated ManPG): the fixed-t transported-direction variant (extrapolation and restart, no adaptation), reported in the ablation.
• AManPG [7]: Huang–Wei FISTA-style extrapolation with inverse retraction and a periodic safeguard. The supplied driver uses its own tangent semismooth subproblem routine (not the RPMWH routine of [10], the Riemannian proximal-mapping subproblem solver of Huang and Wei).
• P-ARPG and RPG [10]: practical accelerated and non-accelerated RPG, using the RPMWH subproblem model.

Certified protocol. Tables come from archived scalar records (method, dimensions, tolerance, seed, outer iterations, wall-clock time, termination status, restarts, external residual/feasibility at the returned point). Data matrices and returned iterates are not archived; endpoints require rerunning from the seed. Iteration counts are method-specific outer iterations (AManPG and P-ARPG may solve several subproblems per outer step). The external certificate is one SSN solve at returned X with $t=1/L$ and tight inner tolerance; success requires residual below tolerance and feasibility <10⁻⁶. Seeds are shared (trial t: $42+10,000t$). Random singular-value-decomposition (SVD) initial points; MATLAB R2024a, Windows 11, Core i7. Main tables: 10 trials; sweeps: 3; high-precision: 1 trial (seed 10042). Sub-second timings are implementation-specific.

5.1. Standard-Precision Comparison

Table 2 reports $ϵ={10}^{-8}$, $n=500$, $r=5$, $\mu =0.8$, 10 trials. All 60 runs shown in the table, together with the 10 additional TE-ManPG runs of the factorial ablation (Section 5.4), passed the external certificate (residual below tolerance at the returned point, feasibility <10⁻⁶); every method certified $10/10$.

Table 2. Standard-precision comparison ($n=500$, $r=5$, $\mu =0.8$, $ϵ={10}^{-8}nr$, 10 trials), generated from archived per-run records. All methods certified $10/10$. Iter. speedup is the mean iteration count relative to ManPG; Solution F is the mean final certified objective over the trials. The proposed method ATE-ManPG is typeset in bold; boldface numbers indicate the best value in each column among the methods sharing the same subproblem model (ManPG, ManPG-Ada, ATE-ManPG). Methods marked ^† use a different subproblem model and are not ranked, and the Solution F column is not ranked.

Method	Iterations	Time (s)	Iter. Speedup	Solution F
ManPG	$741\pm 296$	$0.070\pm 0.045$	$1.00\times$	$-20.062$
ManPG-Ada	$251\pm 72$	$\mathbf{0}.\mathbf{034}\pm \mathbf{0}.\mathbf{020}$	$2.95\times$	$-20.062$
ATE-ManPG	$\mathbf{206}\pm \mathbf{70}$	$0.051\pm 0.026$	$\mathbf{3}.\mathbf{59}\times$	$-19.994$
AManPG	$140\pm 25$	$0.049\pm 0.032$	$5.29\times$	−19.963 †
P-ARPG	$107\pm 23$	$0.116\pm 0.047$	$6.93\times$	−19.983 †
RPG	$658\pm 255$	$0.286\pm 0.095$	$1.13\times$	−20.091 †

ATE-ManPG shows a $3.59\times$ mean iteration reduction vs. fixed-parameter ManPG, but this is not a momentum-only effect. ManPG-Ada already gives $2.95\times$. In the harmonized ablation, adding transported momentum yields a further 6.4% mean iteration reduction ($220.4\to 206.3$), concentrated in one paired trial (seed 90042: about 106 of 141 iterations saved, with a worse objective $-18.75$ vs. $-19.43$); the other nine pairs improve by about $1.8\%$. Mean runtime is higher with momentum ($0.051$ s vs. $0.0317$ s adaptive-only; ATE slower in 9/10 pairs).

Table 2 is therefore best read jointly with the factorial ablation of Section 5.4 when attributing the improvement. Of the $3.59\times$ total mean iteration reduction relative to fixed-parameter ManPG, the pre-existing adaptive proximal parameter accounts for $2.95\times$ (ManPG-Ada vs. ManPG), and the harmonized ablation separates the two mechanisms on one implementation: adaptation alone gives $3.4\times$, transported extrapolation alone $1.6\times$, and their combination $3.6\times$, so the marginal contribution of extrapolation on top of adaptation is $6.4\%$ in mean iterations. The difference between $3.59\times$ and $2.95\times$ in Table 2 should accordingly not be read as a pure momentum effect: that table compares total methods whose adaptive constants differ ($1.02/1.1$ here against the $1.01$-rule of ManPG-Ada), and part of that gap is tuning rather than extrapolation (Section 5.4).

The main advantages of ATE-ManPG over the closest methods, as demonstrated by these examples, are as follows. Compared with fixed-parameter ManPG, it is uniformly faster across settings: $3.59\times$ fewer mean iterations here ($741\to 206$, all $10/10$ certified), $2.9$–$3.9\times$ across $n=100$–1500 (Table 3), and $1.5$–$3.6\times$ across $\mu$ (Table 4). Compared with the extrapolation-based AManPG and P-ARPG, it avoids inverse retractions (the transport is one orthogonal projection), enforces decrease of the objective at every iteration through the restart, and returns the point at which the residual is certified; on this test bed it also reaches a lower certified objective than either ($F=-19.994$ vs. $-19.963$ and $-19.983$ in Table 2) at a lower runtime than P-ARPG ($0.051$ s vs. $0.116$ s), whereas P-ARPG needs fewer outer iterations under a different subproblem model and outer-step accounting. Compared with ManPG-Ada, its advantage is modest and instance-dependent, as quantified above. At standard precision, RPG attains a marginally lower objective ($-20.091$) at the highest iteration and time cost; at high precision it is the only method certified at all four tolerances (Section 5.3), so no single method dominates. The demonstrated strength of ATE-ManPG is the combination of ManPG-level guarantees, retraction independence, and certified monotone progress at standard precision.

Table 3. Dimension sweep: iterations vs. n ($r=5$, $\mu =0.8$, $ϵ={10}^{-8}nr$, 3 trials). Parenthesized entries give certified/total runs. Boldface indicates the best value in each column.

n	100	200	500	800	1000	1500
ManPG	$487\pm 87$	$528\pm 123$	$1021\pm 178$	$1474\pm 699$	$1487\pm 638$	$1337\pm 621$
ManPG-Ada	$177\pm 15$	$190\pm 31$	$326\pm 49$	$459\pm 184$	$\mathbf{459}\pm \mathbf{196}$	$\mathbf{457}\pm \mathbf{168}$
ATE-ManPG	$\mathbf{126}\pm \mathbf{11}$	$\mathbf{166}\pm \mathbf{63}$	$\mathbf{287}\pm \mathbf{51}$	$\mathbf{415}\pm \mathbf{204}$	$516\pm 156$$(2/3)$	$462\pm 164$
ATE speedup	$3.9\times$	$3.2\times$	$3.6\times$	$3.6\times$	$2.9\times$	$2.9\times$

Table 4. Iterations vs. $\mu$ ($n=500$, $r=5$, $ϵ={10}^{-8}nr$, 3 trials; all certified $3/3$). Boldface indicates the best value in each column.

$\mathit{\mu }$	0.2	0.4	0.8	1.6	3.2
ManPG	$1122\pm 602$	$768\pm 116$	$1021\pm 178$	$82\pm 29$	$36\pm 6$
ManPG-Ada	$\mathbf{482}\pm \mathbf{230}$	$\mathbf{315}\pm \mathbf{41}$	$326\pm 49$	$60\pm 15$	$31\pm 5$
ATE-ManPG	$486\pm 234$	$324\pm 75$	$\mathbf{287}\pm \mathbf{51}$	$\mathbf{41}\pm \mathbf{8}$	$\mathbf{24}\pm \mathbf{3}$
ATE speedup	$2.3\times$	$2.4\times$	$3.6\times$	$2.0\times$	$1.5\times$

5.2. Dimension Scaling and Regularization

Table 3 and Table 4 report iteration counts (3 trials, ManPG-Ada included). All runs passed the external certificate except one ATE-ManPG trial at $n=1000$ (reported as $2/3$), whose returned residual exceeded the tolerance after its internal stopping triggered.

Iteration reductions vs. ManPG are stable in n but vary with $\mu$. These sweeps compare total methods (different tunings), not isolated momentum; at $n=1000$ ATE certifies only $2/3$ runs, so that column is not paired.

5.3. High-Precision Behavior

Table 5 reports a single trial (seed 10042) under a uniform iteration budget of 5000, with success decided only by the external certificate. All iteration counts are archived outer iterations (numbers of outer subproblem solves).

Table 5. High-precision comparison: iterations vs. scaled squared-residual tolerance ($n=500$, $r=5$, $\mu =0.8$, single trial, budget 5000). ✓: external certificate passed at the returned point. “LS fail”: explicit Armijo failure triggered before the tolerance was reached (returned residual exceeds it). “safe.”: early safeguard exit whose returned residual exceeds the tolerance. “—”: did not complete the budget within the allotted time (see text).

$\mathit{ϵ}$	ManPG	ManPG-Ada	TE-ManPG	ATE-ManPG	AManPG	P-ARPG	RPG
${10}^{-8}nr$	1054✓	327✓	725✓	297✓	171✓	125✓	824✓
${10}^{-10}nr$	1245✓	372✓	852✓	337✓	191✓	168✓	1017✓
${10}^{-12}nr$	1320 ls	412✓	885 ls	339 ls	—	168 safe.	1212✓
${10}^{-14}nr$	1320 ls	434 ls	885 ls	339 ls	—	168 safe.	1409✓

The certified picture differs from internal-stopping counts. At $ϵ\le {10}^{-12}$ the SSN family (ManPG, TE, ATE) hits explicit Armijo failure before the tolerance (operative inner floor ${10}^{-13}$ in the revised code). ManPG-Ada passes ${10}^{-12}$ and fails at ${10}^{-14}$. P-ARPG safeguards out after 168 iterations with residual near $1.3\times {10}^{-7}$. AManPG’s inner solver slows severely and did not finish the budget at $ϵ\le {10}^{-12}$ (no coded timeout in the archives). RPG is certified at all four tolerances here.

The failure entries have identifiable causes, which are numerical rather than theoretical. The Armijo failures of the SSN family at $ϵ\le {10}^{-12}nr$ reflect the interaction between the line search and the inner solver at very small tolerances: Lemma 1 guarantees acceptance of every sufficiently small $\alpha$ only for exact subproblems, whereas near the requested residual the decrease predicted by the computed direction, of order ${\displaystyle \frac{\alpha }{4t}}{\parallel V_k\parallel }_{\mathrm{F}}^2$, falls below the perturbation induced by the inner dual-residual floor ($\sqrt{{10}^{-13}}\approx 3\times {10}^{-7}$), which biases $V_k$ at a level that backtracking cannot repair—halving $\alpha$ shrinks the target linearly while the realizable decrease is already dominated by that bias. The adaptive tightening of $\mathrm{inner}\_\mathrm{tol}$ when the line search struggles (Section 3.2) targets precisely this interaction. At $ϵ={10}^{-14}nr$ the predicted per-step decrease is of order ${10}^{-12}$; the inner-residual bias then dominates the scale being verified, and the cancellation roundoff in evaluating differences of objective values of order 20 (absolute noise of order ${10}^{-14}$) is no longer negligible, so certification at that level sits at the limit of double-precision arithmetic for this family of codes. The P-ARPG entries are not numerical failures: its safeguard exit is a designed termination that returns a point with residual ≈1.3$\times {10}^{-7}$, above the tighter thresholds. AManPG’s dashes are budget failures: its inner routine ties the inner effort to the outer tolerance, the cost per outer step grows as $ϵ$ shrinks, and the run did not finish the budget within the allotted time. None of these entries contradicts the theory: Theorems 1–2 concern the exact idealized iteration, and the certified entries of ManPG-Ada and RPG at $ϵ={10}^{-12}nr$ show that the practical floor is implementation-dependent rather than universal.

These counts are empirical observations under one implementation and budget, not a verification of asymptotic orders: a worst-case $O\left({\epsilon }^{-2}\right)$ bound neither predicts individual trajectories nor is refuted by budget exhaustion, the table’s $ϵ$ thresholds a scaled squared residual (a factor-100 tighter squared tolerance is a factor-10 tighter norm tolerance), and every method can enjoy faster local behavior under additional regularity.

5.4. Factorial Ablation and Cap Sensitivity

Table 6 reports the factorial ablation requested to separate the two mechanisms: the four variants are produced by one implementation (identical SSN accuracy, stopping certificate, Armijo constant, adaptive rule, seeds), toggling only the momentum flag and the adaptive proximal parameter.

Table 6. Factorial ablation ($n=500$, $r=5$, $\mu =0.8$, $ϵ={10}^{-8}nr$, 10 trials; all certified $10/10$). “Restarts” is the mean number of monotone-restart triggers per run.

Configuration	Iterations	Solution F	Restarts/Run
fixed t, no extrapolation (= ManPG)	$741\pm 296$	$-20.062$	—
fixed t, extrapolation	$477\pm 187$	$-19.994$	35
adaptive t, no extrapolation	$220\pm 70$	$-20.062$	—
adaptive t, extrapolation (ATE)	$\mathbf{206}\pm \mathbf{70}$	$-19.994$	148

The attribution is as follows. Adaptation alone: $3.4\times$ ($741\to 220$); momentum alone: $1.6\times$ ($741\to 477$); combined: $3.6\times$ ($741\to 206$). Momentum adds 6.4% on top of the harmonized adaptive rule, concentrated in one paired trial (Section 5), with higher runtime and a worse objective there. ATE triggers restarts frequently (148 per ∼206 iterations). Our adaptive factors $1.02/1.1$ are stronger than the $1.01$-rule ManPG-Ada (220 vs. 251), so part of that gap is tuning.

The marginal effect of extrapolation is also strongly instance-dependent, and its concentration in one pair indicates a dependence on the problem instance and on the initial point. The restart statistics are consistent with this mechanism: extrapolation can help only while successive proximal directions remain correlated along a smooth arc of the objective, whereas near the ${\ell }_1$ kinks of h the directions decorrelate and the monotone restart discards the momentum—ATE triggers restarts on 148 of roughly 206 iterations—so only instances whose trajectory stays on such an arc long enough for the extrapolation to compound benefit materially; the seed-90042 trial is the one such instance among the ten. The transported momentum should accordingly be regarded as a safeguarded option that is decisive for certain instances rather than a uniform accelerator: across the $\mu$-sweep its marginal effect ranges from negligible ($\mu =0.2$: 486 vs. 482 iterations) to moderate ($\mu =1.6$: 41 vs. 60), and the worst-case theory of Section 4 claims no order improvement from it (Remark 6).

Table 1 varies the momentum cap. Mean iteration counts are flat across ${\beta }_{\max }\in \{0.2,0.5,0.8,1.0\}$ (all within one standard deviation), while the restart frequency increases mildly with the cap. This confirms that $0.5$ is a tuning default with no special theoretical role (Remark 6): the safeguard, not the numerical cap, controls the iteration.

Limitations. (i) Prove $\theta =1/2$ and Assumption 1 for nondegenerate sparse-PCA subclasses, or state general-exponent rates; (ii) harmonize inexact SSN with the analysis [16], of which Remark 3 is a constant-level first step; (iii) restart-only/cap-only studies and additional applications; (iv) explain the quality tradeoff in the concentrated trial. Penalty and constraint-dissolving alternatives [20,23,25] remain uncompared.

6. Conclusions

ATE-ManPG is a safeguarded transport-extrapolated proximal gradient method on the Stiefel manifold: projected search-direction extrapolation, monotone restart, and a bounded adaptive proximal parameter, returning the work point where the residual is certified. For the exact idealized iteration we proved global subsequential stationarity and $O\left({\epsilon }^{-2}\right)$ iteration complexity, and a conditional R-linear rate under explicit KL-$1/2$ and relative-error hypotheses (smoothness or sign compatibility). The penalized sparse-PCA objective has no universal exponent-$1/2$ property; we give an explicit counterexample. Any local order change from $O\left({\epsilon }^{-2}\right)$ to $O\left(\log \right(1/\epsilon \left)\right)$ is thus a KL effect rather than a worst-case acceleration (Theorem 3); we do not claim a better worst-case order than ManPG. Experiments under a uniform external certificate show a $3.6\times$ mean iteration reduction vs. fixed-parameter ManPG, mostly from the pre-existing adaptive parameter; transported momentum adds a small, instance-dependent $6.4\%$ with runtime overhead and occasional quality tradeoffs. The distinctive construction is search-direction transport plus objective-based restart; the theory records when a KL rate is preserved.