LDC-GAT: A Lyapunov-Stable Graph Attention Network with Dynamic Filtering and Constraint-Aware Optimization

Abstract

Graph attention networks are pivotal for modeling non-Euclidean data, yet they face dual challenges: training oscillations induced by projection-based high-dimensional constraints and gradient anomalies due to poor adaptation to heterophilic structure. To address these issues, we propose LDC-GAT (Lyapunov-Stable Graph Attention Network with Dynamic Filtering and Constraint-Aware Optimization), which jointly optimizes both forward and backward propagation processes. In the forward path, we introduce Dynamic Residual Graph Filtering, which integrates a tunable self-loop coefficient to balance neighborhood aggregation and self-feature retention. This filtering mechanism, constrained by a lower bound on Dirichlet energy, improves multi-head attention via multi-scale fusion and mitigates overfitting. In the backward path, we design the Fro-FWNAdam, a gradient descent algorithm guided by a learning-rate-aware perceptron. An explicit Frobenius norm bound on weights is derived from Lyapunov theory to form the basis of the perceptron. This stability-aware optimizer is embedded within a Frank–Wolfe framework with Nesterov acceleration, yielding a projection-free constrained optimization strategy that stabilizes training dynamics. Experiments on six benchmark datasets show that LDC-GAT outperforms GAT by 10.54% in classification accuracy, which demonstrates strong robustness on heterophilic graphs.

1. Introduction

Graph-structured data, as one mathematical representation of complex relationships in non-Euclidean spaces, has become a fundamental tool for modeling intricate systems, involving social network analysis [1] and molecular interaction prediction [2]. In recent years, graph neural networks (GNNs) have enabled end-to-end learning through spectral-domain frequency response filtering [3] and spatial-domain attention-based aggregation [4]. Notably, the adaptive spectral filtering model introduced by Defferrard et al. [5] leverages differentiable frequency response functions to overcome the heuristic tuning constraints inherent in traditional graph convolutional networks (GCNs). Nevertheless, contemporary approaches used for complex graph modeling usually face an inherent conflict between training stability and topological adaptability [6,7,8], which substantially impairs their ability to distinguish structures [9] and constrains their practical efficacy.

1.1. Related Work

Although attention-based graph neural networks have demonstrated superior performance in node classification tasks [4,10], their deployment in practical systems remains hindered by three critical bottlenecks.

Firstly, the use of fixed self-loop weights limits the model’s ability to dynamically adapt the graph filtering process to heterophilic structural patterns. As demonstrated in Bronstein et al.’s geometric deep learning framework [11], this constraint can induce signal distortion in the spectral domain. GNNs fundamentally operate through implicit modulation of graph signals [12], where traditional neighborhood aggregation acts as a low-pass filter. However, this approach inherently assumes homophily [13], while overlooking the possibility that, in heterophilic graphs, high-frequency components may carry essential discriminative cues [14]. Consequently, GATs with static filtering patterns have been proven to have excessive Dirichlet energy decay [15], lacking the capacity to modulate filtering strength in response to local structural variation.

Secondly, current research lacks rigorous theoretical characterization of the stability of iterative feature propagation. In deep GNN architectures, discretization errors may accumulate across layers, significantly degrading model robustness [16]. These highlight the need for a principled analysis of energy preservation and system dynamics in recursive graph signal processing.

Finally, the issue of unstable optimization caused by topological sensitivity remains largely unaddressed. Traditional graph attention networks, when updating multi-head weights independently, lack spectral constraints in high-dimensional parameter spaces, which may lead to gradient instability during training [9,17]. The gradient dynamics of neural networks are shaped not only by graph topology but also by the geometry of the parameter space itself [18]. While projected gradient methods incur high computational overhead due to frequent re-projections, alternatives such as SimGrad [11] often ignore the influence of graph structure on the spectral radius of weight matrices, potentially leading to unstable convergence behavior.

In summary, the limits in frequency adaptability, propagation stability, and constrained optimization [19,20,21,22,23] indicate the need for a unified framework that jointly addresses signal-level filtering and parameter-level training dynamics in GATs.

1.2. Our Approach

To address the aforementioned challenges, we present Lyapunov-Stable Graph Attention Network with Dynamic Filtering and Constraint-Aware Optimization (LDC-GAT), which systematically integrates graph signal processing theory with dynamic system stability analysis. The key innovations of our work can be summarized as follows:

• We introduce Dynamic Residual Graph Filtering (DRG-Filtering), where a tunable self-loop factor dynamically regulates the coupling between neighborhood and self-node features. Moreover, we refer to the energy conservation idea of the Hamiltonian graph dynamics model [24] to construct a DRG-Filtering that maintains the Dirichlet energy lower bound.
• We derive a Frobenius norm constraint for multi-head attention weights using Lyapunov theory, which is subsequently used to design the learning-rate-aware perceptron. This strategy dynamically gauges the compatibility between the current learning rate and local loss landscape characteristics by computing the relative relationship between the Frobenius norm of the weight matrix and a predefined critical threshold.
• We pioneer the design of a projection-free optimization algorithm FWNAdam by integrating the Nesterov momentum mechanism into the Frank–Wolfe framework. By replacing high-dimensional projection steps with linear searches, and embedding the proposed learning-rate-aware perceptron into FWNAdam, we introduce Fro-FWNAdam, which ensures feasibility under convex constraints while enabling progressively stable network optimization.

Beyond classification, LDC-GAT is well suited for combinatorial constraint-aware GNNs, which are increasingly important in solving combinatorial problems like subgraph tracing and subgraph selection, where graph constraints are critical. Recent work in this area, such as the DeepTrace framework by P. Yu et al. [25], demonstrates the power of GNNs in optimizing contact tracing in epidemic networks, a task requiring efficient handling of graph constraints. Similarly, Rumor Centrality by D. Shah et al. [26] introduces a method for handling network-based combinatorial problems. Both works emphasize the growing need for optimization techniques tailored to graph-based combinatorial tasks, aligning well with the potential applications of LDC-GAT.

2. Preliminaries

2.1. Graph Attention Networks

Graph attention networks (GATs) [4] utilize a self-attention mechanism to dynamically adjust connection weights between nodes, improving the performance of graph convolutional networks. The graph structure is denoted as $G=(V,E,H)$, where $V={\left\{v_i\right\}}_{i=1}^N$ represents the set of nodes, $E\subseteq V\times V$ represents the set of edges, and $X\in {\mathbb{R}}^{N\times F_0}$ is the initial input feature matrix, with $F_0$ being the dimensionality of the original features. The node feature matrix at the l-th layer is $H^{\left(l\right)}\in {\mathbb{R}}^{N\times F}$, where F is the corresponding feature dimension, and $H^{\left(0\right)}=X$.

The original feature vector $h_i\in {\mathbb{R}}^F$ of each node $v_i$ is first linearly transformed into a high-dimensional feature $h_i^{\prime }=W{h}_i, h_i^{\prime }\in {\mathbb{R}}^{F^{\prime }}$ using a weight matrix $W\in {\mathbb{R}}^{F\times F^{\prime }}$, where $F^{\prime }$ represents the feature dimension at the $l+1$-th layer, and the weight matrix W is a learnable parameter matrix.

The attention coefficient for a node pair $(v_i,v_j)$ is defined as $e_{ij}=a(W{h}_i,W{h}_j)$, where $a\in {\mathbb{R}}^{2F^{\prime }}$ is the attention mechanism parameter vector. The attention coefficients within the neighborhood $N_i$ of node $v_i$ are normalized into ${\alpha }_{ij}$ using LeakyReLU activation and the softmax function as following:

$${\alpha }_{ij}=\mathrm{softmax}\left(e_{ij}\right)={\displaystyle \frac{exp\left(\mathrm{LeakyReLU}\left(a^T[W{h}_i\Vert W{h}_j]\right)\right)}{{\sum }_{k\in N_i}exp\left(\mathrm{LeakyReLU}\left(a^T[W{h}_i\Vert W{h}_k]\right)\right)}}$$

(1)

To enhance the model’s fault tolerance and expressive power, GATs introduce the multi-head attention mechanism. Let there be K independent attention heads working in parallel, with the corresponding weight matrices denoted as ${\{W^{\left(k\right)}\in {\mathbb{R}}^{F\times F^{\prime }}\}}_{k=1}^K$ and the attention parameter sets as ${\{a^{\left(k\right)}\in {\mathbb{R}}^{2F^{\prime }}\}}_{k=1}^K$. After transformation by the k-th head, the output features of K independent heads are integrated through average pooling, yielding the $l+1$-th layer output feature matrix $H^{(l+1)}\in {\mathbb{R}}^{N\times F^{\prime }}$ after multi-head average fusion:

$$H^{(l+1)}=\sigma \left({\displaystyle \frac{1}{K}}\sum _{k=1}^K{\tilde{A}}^{\left(k\right)}{H}^{\left(l\right)}{W}^{\left(k\right)}\right)$$

(2)

where $A^{\left(k\right)}\in {\mathbb{R}}^{N\times N}$ is the normalized attention matrix for the k-th head.

While existing theories provide effective frameworks for GAT, three critical limitations remain: dynamic graph filtering stability [19], parameter optimization robustness [20], and computational efficiency [21]. To better understand the GAT in various structural scenarios, it is essential to introduce a spectral tool that quantifies signal variation across the graph.

2.2. Dirichlet Energy and Graph Filtering

In graph neural networks and graph signal processing, the Dirichlet energy quantifies the variation of signals across the graph structure, serving as a key measure of smoothness and expressive capacity. For Dirichlet energy H, this is defined as follows.

Definition 1. 

Given a graph signal matrix $H\in {\mathbb{R}}^{N\times d}$, the Dirichlet energy is defined as

$$\mathcal{E}\left(H\right)={\displaystyle \frac{1}{2}}\sum _{(i,j)\in \mathcal{E}}{\parallel H_i-H_j\parallel }_2^2,$$

(3)

where $H_i\in {\mathbb{R}}^d$ is the feature vector at node i, and $\mathcal{E}$ is the set of graph edges.

Maintaining a lower bound on Dirichlet energy prevents the complete suppression of high-frequency components, enabling the graph filter to preserve discriminative information while enforcing smoothness. This balance enhances model expressiveness and improves robustness to structural perturbations, contributing to better generalization on diverse graph topologies.

While Dirichlet energy helps evaluate the behavior of graph filters during forward propagation, model training stability also depends heavily on how parameters are optimized over time [27,28]. We next introduce the underlying principles of optimization dynamics.

2.3. Lyapunov Stability in Discrete Dynamical Systems

To analyze and constrain training dynamics, we adopt Lyapunov stability theory, a classical tool in control systems used to evaluate the stability of discrete-time processes. Consider a general iterative system:

$$H^{(n+1)}=H^{\left(n\right)}+\Delta t·f\left(H^{\left(n\right)}\right),$$

(4)

where $f(·)$ is a residual update function and $\Delta t$ is the step size. The system is said to be asymptotically stable if

$${\parallel H^{(n+1)}-H^{\left(n\right)}\parallel }_F\le {\gamma }_0{\parallel H^{\left(n\right)}\parallel }_F,0<{\gamma }_0<1.$$

(5)

In the context of GNNs, this criterion allows us to derive Frobenius norm bounds on weight matrices, which can be used to regulate training dynamics and ensure convergence stability. To further quantify and evaluate the long-term learning behavior of such optimizers, we turn to regret analysis [17].

2.4. Regret Bounds

In constrained optimization problems involving learnable parameters $\theta \in \mathcal{F}$, especially when $\mathcal{F}$ is a convex and compact subset of ${\mathbb{R}}^d$, the learning process is often modeled as an online learning procedure across time steps $t=1,2,\dots ,T$. At each step, the learner selects ${\theta }_t\in \mathcal{F}$, then observes a convex loss function $L_t(·)$, and incurs loss $L_t\left({\theta }_t\right)$. A key performance measure in this setting is the cumulative regret, defined as

$$R_T:=\sum _{t=1}^T\left(L_t\left({\theta }_t\right)-L_t\left({\theta }^{\ast }\right)\right),$$

(6)

where ${\theta }^{\ast }=arg{min}_{\theta \in \mathcal{F}}{\sum }_{t=1}^T{L}_t\left(\theta \right)$ is the best fixed decision in hindsight. A learning algorithm is said to have no regret if

$$\underset{T\to \infty }{lim}{\displaystyle \frac{R_T}{T}}=0,$$

(7)

i.e., the average regret vanishes asymptotically. One classic class of methods in this domain is Frank–Wolfe-type projection-free algorithms, which avoid expensive Euclidean projections by using linear optimization steps. However, existing variants such as FWAdam typically suffer from regret bounds as high as $O\left(T^{4/3}\right)$ [29]. In contrast, the following proposed Fro-FWNAdam combines adaptive learning rate control with gradient normalization under Frobenius norm constraints, and achieves an improved regret bound of $O\left(\sqrt{T}\right)$ This theoretical foundation is critical for the analysis in subsequent sections.

3. Dynamic Residual Graph Filtering

GNNs fundamentally operate through implicit response modulation of graph signals [12]. Traditional neighborhood aggregation leverages low-pass filtering to suppress high-frequency components, thereby promoting the smoothness of node representations. However, this approach heavily relies on the assumption of graph homophily [13], while high-frequency features in heterophilic networks often carry critical discriminative information through inter-node differences [14]. Consequently, existing GATs, due to their static self-loop weights and fixed filtering patterns, are prone to uncontrollable Dirichlet energy decay and lack frequency-domain adaptability [15]. As a result, they are unable to dynamically adjust filtering patterns according to graph heterophily metrics.

To address this, this paper introduces an adjustable self-loop influence factor $\beta$, which is integrated into the attention weight matrix to regulate the coupling strength between node features. The variable attention weight matrix for the k-th head, $A_{\mathrm{att}}^{\left(k\right)}$, is defined as follows:

$$A_{\mathrm{att}}^{\left(k\right)}={\tilde{A}}^{\left(k\right)}+\beta I,\beta \in [-1,1]$$

(8)

where $I\in {\mathbb{R}}^{N\times N}$ is the identity matrix. Existing research indicates that the strong coupling between graph filters and weight matrices can induce spectral degradation, limiting the model’s adaptability to diverse graph structures.

This paper projects the original features $H^{\left(0\right)}$ into the latent space through the mapping $\varphi :{\mathbb{R}}^{N\times F}\to {\mathbb{R}}^{N\times F^{\prime }}$, constructing cross-layer feature pathways:

$$H^{(l+1)}=\sigma \left({\displaystyle \frac{1}{K}}\sum _{k=1}^K{A}_{\mathrm{att}}^{\left(k\right)}{H}^{\left(l\right)}{W}^{\left(k\right)}+\alpha \varphi \left(H^{\left(0\right)}\right)\right)$$

(9)

where $\alpha \in (-1,1)$ is the learnable spectral balance coefficient that dynamically adjusts the cross-layer signal strength through a gating mechanism. This structure mitigates Dirichlet energy decay through the residual term.

Theorem 1. 

Given the residual propagation rule, assume the activation function $\sigma (·)$ satisfies the contraction property ${\parallel \sigma \left(X\right)\parallel }_2\le {\parallel X\parallel }_2$, and that the linear mapping $\varphi (·)$ satisfies ${\parallel \varphi \left(X\right)\parallel }_F\le {\parallel X\parallel }_F$.

$$H^{(l+1)}=\sigma \left({\displaystyle \frac{1}{K}}\sum _{k=1}^K{A}_{\mathit{att}}^{\left(k\right)}{H}^{\left(l\right)}{W}^{\left(k\right)}+\alpha \varphi \left(H^{\left(0\right)}\right)\right),$$

(10)

If the attention matrices $A_{\mathit{att}}^{\left(k\right)}$ are properly normalized, then the Dirichlet energy of the propagated features is bounded from below by

$$\mathcal{E}\left(H^{(l+1)}\right)\ge {\alpha }^2\mathcal{E}\left(\varphi \left(H^{\left(0\right)}\right)\right)+{(1-\alpha )}^2\mathcal{E}\left(H^{\left(l\right)}\right).$$

(11)

The Proof of Theorem 1 is given in Appendix A. Based on Definition 1, maintaining a lower bound on the Dirichlet energy ensures the coherent integration of multi-frequency signals, preventing the loss of critical feature information caused by reliance on a single filtering mode.

The residual term $\varphi \left(H^{\left(0\right)}\right)$ preserves high-frequency details, while the attention filtering term captures low-frequency topological patterns. Together, these components enable adaptive frequency band fusion through a differentiable mechanism, enhancing the overall network performance. While the residual filtering mechanism enhances forward signal propagation, uncontrolled parameter growth during training can still lead to instability. We now derive a Frobenius-norm-based constraint to ensure stability during weight updates in high-dimensional spaces and effectiveness.

4. The Frobenius Norm Constraints for Multi-Head Weights

Traditional graph attention networks, when updating multi-head weights independently, lack spectral constraints in high-dimensional parameter spaces, leading to gradient instability during training [9]. Therefore, leveraging Lyapunov stability theory, this paper establishes explicit Frobenius norm bounds for multi-head weight matrices and dynamically adjusts the learning rate based on the exceedance of the norm, feeding this information into Fro-FWNAdam for autonomous parameter space optimization.

Based on Section 2.3, the inter-layer propagation rule of the multi-head GAT is reconstructed into a discretized dynamical system with a time step $\Delta t$:

$$H^{(n+1)}=H^{\left(n\right)}+\Delta t·f\left({\displaystyle \frac{1}{K}}\sum _{k=1}^K{A}_{\mathrm{att}}^{\left(k\right)}{H}^{\left(n\right)}{W}^{\left(k\right)}+\alpha \varphi \left(H^{\left(0\right)}\right)\right)$$

(12)

where $f\left(X\right)=\sigma \left(X\right)-X$ is a linear residual mapping function, and $\sigma (·)$ is the Lipschitz-continuous activation function. According to Lyapunov stability theory, the system is asymptotically stable if the following condition holds:

$${\parallel H^{(n+1)}-H^{\left(n\right)}\parallel }_F\le {\gamma }_0{\parallel H^{\left(n\right)}\parallel }_F(0<{\gamma }_0<1).$$

(13)

Substituting into the discretized equation gives

$$\Delta t\parallel f\left({\displaystyle \frac{1}{K}}\sum _{k=1}^K{A}_{\mathrm{att}}^{\left(k\right)}{H}^{\left(n\right)}{W}^{\left(k\right)}+\alpha \varphi \left(H^{\left(0\right)}\right)\right){\parallel }_F\le {\gamma }_0{\parallel H^{\left(n\right)}\parallel }_F.$$

(14)

Assuming that the activation function $\sigma (·)$ satisfies the Lipschitz condition ${\parallel \sigma \left(X\right)\parallel }_F\le L_{\sigma }{\parallel X\parallel }_F$, the Lipschitz constant of the residual mapping $f\left(X\right)$ is given by

$$L_{\Phi }=max\left\{|1-L_{\sigma }|,|L_{\sigma }-1|\right\}.$$

(15)

For the ReLU family of functions ($L_{\sigma }=1$), $L_{\Phi }=0$, and in this case, system stability is dominated by the linear terms. By neglecting higher-order nonlinear terms ($L_{\Phi }=0$), the stability condition degenerates to

$${\displaystyle \frac{\Delta t}{K}}\sum _{k=1}^K\parallel A_{\mathrm{att}}^{\left(k\right)}{H}^{\left(n\right)}{W}^{\left(k\right)}{\parallel }_F\le {\gamma }_0{\parallel H^{\left(n\right)}\parallel }_F.$$

(16)

Using the matrix norm compatibility property ${\parallel AB\parallel }_F\le {\parallel A\parallel }_F{\parallel B\parallel }_F$, we obtain

$${\displaystyle \frac{\Delta t}{K}}\sum _{k=1}^K\parallel A_{\mathrm{att}}^{\left(k\right)}{\parallel }_F{\parallel W^{\left(k\right)}\parallel }_F\le {\gamma }_0.$$

(17)

Assuming the attention matrix is normalized (${\parallel A_{\mathrm{att}}^{\left(k\right)}\parallel }_F\le 1$), the weights for each head must satisfy

$${\parallel W^{\left(k\right)}\parallel }_F\le {\displaystyle \frac{{\gamma }_{0}K}{\Delta t}}.$$

(18)

We consider using the above weight constraint to adjust the learning rate to help the model accurately adjust parameters and accelerate model convergence. Although full-time-domain constraint adjustment mechanisms can precisely maintain parameter feasibility, its real-time computational overhead may lead to significant performance degradation in deep graph networks. Actually, once the Frobenius norm of the weight matrix enters an $ϵ$-neighborhood, the probability of constraint violation decays rapidly, so the following adjustment critical point is set to reduce computational overhead.

Lemma 1. 

Let the parameter space be a convex and compact set $\mathcal{F}=\{W\in {\mathbb{R}}^{F\times F^{\prime }}\mid \parallel W{\parallel }_F\le \tau \}$, and suppose the initial parameter satisfies ${\delta }_1:={\parallel W_1^{\left(k\right)}-\tau \parallel }_F\le C_0$. Assume that whenever ${\parallel W_t^{\left(k\right)}\parallel }_F>\tau$, the learning rate satisfies ${\eta }_t^{\left(k\right)}\ge {\displaystyle \frac{\eta \tau }{(\tau +ϵ)\sqrt{t}}}$, and that there exists a constant $G>0$ such that ${\parallel g_t\parallel }_F\le G$. Then, there exists a finite time

$$t_0=⌈{\left({\displaystyle \frac{2\eta \tau G}{(\tau +ϵ)ϵ}}\right)}^2⌉,$$

(19)

such that for all $t>t_0$, the Frobenius norm satisfies ${\parallel W_t^{\left(k\right)}-\tau \parallel }_F\le ϵ$.

The Proof of Lemma 1 is given in Appendix B.

5. Optimization Under Norm Constraints: The Fro-FWNAdam Algorithm

The gradient [17,18] dynamics of neural networks are subject to dual constraints arising from both topological sensitivity and the coupling of high-dimensional parameter spaces. To address this issue, this paper introduces Fro-FWNAdam, based on Frank–Wolfe and Nesterov momentum [30]. This approach optimizes the convergence speed of the loss surface by leveraging Nesterov momentum and replaces high-dimensional projections with the Frank–Wolfe linear search [29], making it better suited for constrained optimization scenarios.

Let the feasible domain of the multi-head weight matrices be a convex compact set $F\subset {\mathbb{R}}^d$, subject to the Frobenius norm constraint:

$$F=\{W^{\left(k\right)}\in {\mathbb{R}}^{F\times F^{\prime }}\mid \parallel W^{\left(k\right)}{\parallel }_F\le \tau \}$$

(20)

where $\tau$ is determined by asymptotic stability theory. In the online convex optimization framework, the composite loss function is defined as

$$L_t\left({\theta }_t\right)=-{\lambda }_1\sum _i{y}_{i}log\left({\widehat{y}}_i\right)+{\lambda }_2{(max({\parallel W\parallel }_F-\tau ,0\left)\right)}^2+{\lambda }_3\sum {\tilde{m}}_t$$

(21)

At the t-th iteration, the algorithm selects ${\theta }_t={\left\{W_t^{\left(k\right)}\right\}}_{k=1}^K\in F$ to minimize the cumulative regret $R_T={\sum }_{t=1}^T\left(L_t\left({\theta }_t\right)-L_t\left({\theta }^{\ast }\right)\right)$, where ${\theta }^{\ast }$ denotes the global optimum. The gradient $g_t$ of ${\theta }_t$ is computed as

$$g_t={\nabla }_{{\theta }_t}{L}_t\left({\theta }_t\right)$$

(22)

The gradient of the cross-entropy term is computed through backpropagation, and the gradient of the regularization term is

$${\nabla }_{W^{\left(k\right)}}max(\parallel W^{\left(k\right)}{\parallel }_F{-\tau ,0)}^2=2max(\parallel W^{\left(k\right)}{\parallel }_F-\tau ,0)·{\displaystyle \frac{W^{\left(k\right)}}{\parallel W^{\left(k\right)}{\parallel }_F}}$$

(23)

Introduce the decay factor ${\beta }_1^t={\beta }_1{\lambda }^{(t-1)}$ ($\lambda \in (0,1)$), and update the first-order momentum $m_t$; then calculate the Nesterov-corrected momentum ${\widehat{m}}_t$:

$$m_t={\beta }_1^t{m}_{t-1}+(1-{\beta }_1^t)g_t,{\widehat{m}}_t={\beta }_1^t{m}_t+(1-{\beta }_1^t)g_t$$

(24)

Let ${\beta }_2\in [0,1)$ be the second-order momentum coefficient. Update the second-order momentum $v_t$. Then perform further bias correction for $m_t$ and $v_t$:

$${\tilde{m}}_t={\displaystyle \frac{{\widehat{m}}_t}{1-{\beta }_1^t}},{\widehat{v}}_t={\displaystyle \frac{v_t}{1-{\beta }_2^t}}={\displaystyle \frac{{\beta }_2{v}_{t-1}+(1-{\beta }_2)g_t^2}{1-{\beta }_2^t}}$$

(25)

This paper constructs an auxiliary function to assist the Frank–Wolfe algorithm in determining the search direction:

$$F_t\left(\theta \right)={\eta }_t〈\sum _{\tau =1}^t{\tilde{m}}_{\tau },\theta 〉+{\parallel \theta -{\theta }_1\parallel }_2^2$$

(26)

where ${\eta }_t$ is the learning rate at the t-th iteration. The following linear subproblem is then solved to determine the descent direction:

$$S_t=arg\underset{\theta \in F}{min}\langle \nabla F_t\left({\theta }_t\right),\theta \rangle$$

(27)

where $\nabla F_t\left({\theta }_t\right)={\eta }_t{\tilde{m}}_t+2({\theta }_t-{\theta }_1)$. Finally, the update rule for the weight matrix parameters is as follows:

$${\theta }_{t+1}={\theta }_t+{\displaystyle \frac{{\eta }_t^{\left(k\right)}(S_t-{\theta }_t)}{\sqrt{{\widehat{v}}_t}+ϵ}}$$

(28)

where the Frobenius-based dynamic learning rate adaptation strategy introduced in Fro-FWNAdam is as follows:

$${\eta }_t^{\left(k\right)}=\left\{\begin{array}{cc}{\eta }_{t_0}^{\left(k\right)}·{\displaystyle \frac{\tau }{max(\parallel W^{\left(k\right)}{\parallel }_F,ϵ)}}·{\displaystyle \frac{1}{\sqrt{t}}},\hfill & \mathrm{if}\parallel W^{\left(k\right)}{\parallel }_F>\tau \&t<t_0,\hfill \\ {\eta }_{t_0}^{\left(k\right)}·{\displaystyle \frac{1}{\sqrt{t}}},\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(29)

Here, the norm constraint boundary is defined as $\tau ={\displaystyle \frac{{\gamma }_{0}K}{\Delta t}}$, where ${\eta }_{t_0}^{\left(k\right)}$ represents the original learning rate for the k-th head at the t-th iteration of Fro-FWNAdam, and ${\eta }_t^{\left(k\right)}$ denotes the adjusted adaptive learning rate. Moreover, building on Lemma 1, we propose a phased learning rate adjustment strategy: During the initial stage $(t<t_0)$, dense boundary monitoring is employed, while boundary constraints are relaxed once the parameters converge within the feasible region.

Based on the execution flow of the proposed Fro-FWNAdam and Section 2.4, its core innovations—namely, the integration of dynamic learning rate modulation and projection-free constrained optimization—require rigorous theoretical justification. To this end, we present the following theorem, which establishes the cumulative regret of the proposed algorithm.

Theorem 2. 

Let the parameter space $\mathcal{F}$ satisfy $\mathit{diam}\left(\mathcal{F}\right)\le D$ and $\parallel \theta -{\theta }^{\prime }{\parallel }_{\infty }\le D_{\infty }$. Assume that when ${\parallel W_t^{\left(k\right)}\parallel }_F>\tau$, the learning rate satisfies ${\eta }_t^{\left(k\right)}\ge {\displaystyle \frac{\eta \tau }{(\tau +ϵ)\sqrt{t}}},$ and there exists a finite time $t_0={\left({\displaystyle \frac{2\eta \tau G}{(\tau +ϵ)ϵ}}\right)}^2,$ such that for all $t>t_0$ we have ${\parallel W_t^{\left(k\right)}\parallel }_F\le \tau +ϵ$. Furthermore, suppose there exist constants $G,G_{\infty }>0$ such that for any $\theta \in \mathcal{F}$, the gradient satisfies $L_t\left(\theta \right)-L_t\left({\theta }^{\ast }\right)\le \langle \nabla L_t\left(\theta \right),\theta -{\theta }^{\ast }\rangle ,\parallel \nabla L_t\left(\theta \right){\parallel }_F\le G,{\parallel \nabla L_t\left(\theta \right)\parallel }_{\infty }\le G_{\infty }.$ Then, the cumulative regret $R_T={\sum }_{t=1}^T\left(L_t\left({\theta }_t\right)-L_t\left({\theta }^{\ast }\right)\right)$ is upper-bounded by

$$R_T\le {\displaystyle \frac{D^2\sqrt{T{v}_T}}{2\eta (1-{\beta }_1)}}+{\displaystyle \frac{\eta ({\beta }_1+1)G_{\infty }}{(1-{\beta }_1)\sqrt{1-{\beta }_2}}}\sum _{i=1}^d{\parallel g_{1:T,i}\parallel }_2+{\displaystyle \frac{D_{\infty }^2{G}_{\infty }\sqrt{1-{\beta }_2}}{2\eta {(1-\lambda )}^2}}.$$

(30)

The proof of Theorem 2 is given in Appendix C.

Theorem 2 implies that when the data features are sparse and the gradients are bounded, the summation term can be significantly smaller than its theoretical upper bound, ${\sum }_{i=1}^d{\parallel g_{1:T,i}\parallel }_2\ll d{G}_{\infty }\sqrt{T},$ which indicates that, similar to Adam, the Fro-FWNAdam algorithm also achieves a regret bound of $O\left(d\sqrt{T}\right)$.

Corollary 1. 

Assume the loss function $L:{\mathbb{R}}^d\to \mathbb{R}$ has bounded gradients such that, for all $\theta \in {\mathbb{R}}^d$ and $t\ge 1$, $\parallel g_t{\parallel }_2\le G,{\parallel \nabla L\left(\theta \right)\parallel }_{\infty }\le G_{\infty }.$ Then, there exists a constant $D>0$ such that the distance between any two iterates ${\theta }_m$ and ${\theta }_n$ generated by the Fro-FWNAdam algorithm is uniformly bounded: ${\parallel {\theta }_n-{\theta }_m\parallel }_2\le D,\forall m,n\in \mathbb{N},\mathrm{with}m\le n.$ Moreover, for all $T\ge 1$, the average regret of Fro-FWNAdam satisfies the sublinear bound ${\displaystyle \frac{R\left(T\right)}{T}}\le {\displaystyle \frac{C}{\sqrt{T}}},$ for some constant $C>0$. As a result, Fro-FWNAdam achieves asymptotic non-regret:

$$\underset{T\to \infty }{lim}{\displaystyle \frac{R\left(T\right)}{T}}=0.$$

(31)

Our algorithm over T iterations is bounded by $O\left(\sqrt{T}\right)$—a significant improvement over the $O\left(T^{4/3}\right)$ theoretical bound of the FWAdam optimizer.

6. LDC-GAT

To address the limitations of existing graph attention networks in frequency adaptability, propagation stability, and optimization robustness, we propose LDC-GAT (Figure 1).

LDC-GAT first linearly projects initial features and fuses them with multi-head attention aggregates. Simultaneously, it computes Frobenius norms for each attention head, monitoring boundary deviations: heads exceeding the threshold receive weight penalties with Fro-FWNAdam learning rate adaptation, while others retain their original parameters for global loss optimization by Fro-FWNAdam. The core idea of LDC-GAT is to jointly enhance the forward feature propagation and backward parameter update processes under a unified stability-aware framework.

In the forward path, we introduce the Dynamic Residual Graph Filtering, which integrates a learnable self-loop modulation factor and a residual feature pathway to enable frequency-adaptive filtering. This mechanism enforces a lower bound on the Dirichlet energy during propagation, ensuring that high-frequency signals are not excessively attenuated.

In the backward path, we develop Fro-FWNAdam, a projection-free optimizer that incorporates a Frobenius norm constraint derived from Lyapunov stability theory. This constraint ensures that the weight matrices remain within a bounded range, preventing training divergence caused by uncontrolled norm growth.

At each iteration, Fro-FWNAdam firstly computes the gradient of a composite loss function that includes both a cross-entropy and a Frobenius norm penalty when the weight exceeds a predefined threshold $\tau$. The learning rate is then dynamically adjusted: if the norm of ${\parallel W^{\left(k\right)}\parallel }_F$ exceeds $\tau$, the learning rate decays proportionally to the inverse of both the norm magnitude and the square root of the iteration count; otherwise, a standard inverse square root schedule is applied. This adaptive mechanism promotes stability in the early training phase and efficiency in later stages.

The optimizer further incorporates Nesterov momentum and bias-corrected moment estimates, following the Adam family of methods, but avoids costly Euclidean projections by adopting a Frank–Wolfe update rule. Specifically, it computes a linear minimization direction over the feasible parameter set and interpolates between the current iterate and this direction, scaled by a normalized step size. As a result, Fro-FWNAdam achieves convergence without explicit projection steps, maintaining both computational efficiency and theoretical boundedness. The details of Fro-FWNAdam are provided in Algorithm 1.

Algorithm 1 Fro-FWNAdam

Require: $H^{\left(l\right)}$, adjacency matrix A, step size $\Delta t$, initial parameters ${\theta }_1$, norm threshold $\tau$, contraction rate $\gamma =0.5$ Ensure: Updated parameters ${\theta }_{t+1}$, feature matrix $H^{(l+1)}$, learning rates ${\eta }_t^{\left(k\right)}$   1: Initialize: ${\theta }_1\in \mathcal{F}$, ${\beta }_1,{\beta }_2\in [0,1)$, $\lambda \in (0,1)$   2: Set ${\beta }_{1t}\leftarrow {\beta }_1·{\lambda }^{t-1}$   3: for $t=1$ to T do   4:     for $k=1$ to K do   5:         $H_k^{\left(l\right)}\leftarrow H^{\left(l\right)}{W}^{\left(k\right)}$   6:         $A^{\left(k\right)}\leftarrow {\left(a^{\left(k\right)}\right)}^{\top }[H_{ki}^{\left(l\right)}\mid H_{kj}^{\left(l\right)}]$   7:         ${\parallel W^{\left(k\right)}\parallel }_F\leftarrow min(\parallel W^{\left(k\right)}{\parallel }_F,\tau )$   8:         $g_t\leftarrow \nabla L_t\left({\theta }_t\right)$   9:         if ${\parallel W^{\left(k\right)}\parallel }_F>\tau$ and $t<t_0$ then 10:            ${\eta }_t^{\left(k\right)}\leftarrow {\eta }_{t_0}^{\left(k\right)}·{\displaystyle \frac{\tau }{max(\parallel W^{\left(k\right)}{\parallel }_F,ϵ)·\sqrt{t}}}$ 11:         else 12:            ${\eta }_t^{\left(k\right)}\leftarrow {\displaystyle \frac{{\eta }_{t_0}^{\left(k\right)}}{\sqrt{t}}}$ 13:         end if 14:     end for 15:     $H^{(l+1)}\leftarrow \sigma \left({\displaystyle \frac{1}{K}}{\sum }_{k=1}^K{A}_{\mathrm{att}}^{\left(k\right)}{H}^{\left(l\right)}{W}^{\left(k\right)}+\alpha ·\phi \left(H^{\left(0\right)}\right)\right)$ 16:     $m_t\leftarrow {\beta }_{1t}{m}_{t-1}+(1-{\beta }_{1t})g_t$ 17:     ${\widehat{m}}_t\leftarrow {\beta }_{1t}{m}_t+(1-{\beta }_{1t})g_t$ 18:     $v_t\leftarrow {\beta }_2{v}_{t-1}+(1-{\beta }_2)g_t^2$ 19:     ${\tilde{m}}_t\leftarrow {\displaystyle \frac{{\widehat{m}}_t}{1-{\beta }_{1t}^t}}$,    ${\widehat{v}}_t\leftarrow {\displaystyle \frac{v_t}{1-{\beta }_2^t}}$ 20:     $S_t\leftarrow arg{min}_{\theta }\langle \nabla L_t\left({\theta }_t\right),\theta \rangle$ 21:     ${\theta }_{t+1}\leftarrow {\theta }_t+{\displaystyle \frac{{\eta }_t^{\left(k\right)}·(S_t-{\theta }_t)}{\sqrt{{\widehat{v}}_t}+ϵ}}$ 22: end for

By combining structure-aware gradient control, norm-constrained learning rates, and projection-free updates, Fro-FWNAdam achieves stable and efficient training dynamics tailored to the demands of graph-structured data.

7. Simulations

This paper presents ablation experiments of LDC-GAT on publicly available homophilic graph datasets, including Cora, Citeseer, and Pubmed [31], as well as heterophilic graph datasets, such as Chameleon, Texas, and Squirrel [5]. Dataset details are shown in

Table 1. Dataset details.

Dataset	Nodes	Edges	Classes	Features	Homophily Level
Cora	2708	5429	7	1433	0.81
Citeseer	3327	4732	6	3703	0.74
PubMed	19,717	44,338	3	500	0.80
Chameleon	2277	36,101	3	2325	0.18
Texas	183	309	5	1703	0.11
Squirrel	5201	217,073	3	2089	0.018

.

It is compared with traditional neural networks like MLP, Graph SAGE [15], ChebNet [5], GAT [4], GIN [13], and GCN [8]. Our experimental setup adopts a time step $\Delta t$ of 0.01, 64-dimensional embeddings, a dropout rate of 0.5, and a self-node factor $\alpha$ of 0.5. The experiments validate the effectiveness of the research in this paper by recording the training and testing performance of each network in the semi-supervised node classification task.

In homophilic graphs, LDC-GAT enhances the geometric fidelity of node feature representations by adaptively modulating feature coupling through DGR-Filtering and accelerates model training stability and accuracy through Fro-FWNAdam. The training process is shown in Figure 2 and Figure 3.

As demonstrated in Figure 2, LDC-GAT, incorporating the Fro-FWNAdam optimizer, achieves faster convergence to higher accuracy compared to the vanilla GAT. The subfigures in Figure 2 show the performance of LDC-GAT on three different datasets: Cora, Citeseer, and PubMed. Each subfigure has been separated and enlarged in Appendix D for better clarity and interpretation.

Correspondingly, Figure 3 shows that LDC-GAT exhibits a steeper loss reduction, reaching lower asymptotic loss values more rapidly. The following figure illustrates the loss dynamics over training epochs.

Each subplot in Figure 3 provides an insight into the loss reduction pattern for Cora, Citeseer, and PubMed, with LDC-GAT showing a more rapid and consistent decrease in loss compared to GAT. Each subfigure of Figure 2 and Figure 3 has been separated and enlarged in Appendix D for better clarity and interpretation.

To comprehensively evaluate the overall effectiveness of the improvement strategy on homophilic and heterophilic graphs, this paper conducts a series of ablation experiments on LDC-GAT, followed by a comparison with GAT.Results are shown in

Table 2. The ablation experiment results of LDC-GAT on Cora.

DRG-Filtering	Weight Norm Constraints	Fro-FWNAdam	Acc	Loss
×	×	×	83.90%	1.0893
✓	×	×	83.94%	1.0725
✓	✓	×	84.28%	1.0549
✓	✓	✓	84.50%	1.0433

.

To ensure conciseness given the extensive dataset collection, this paper focuses on Cora for homophilic graph ablation studies and Chameleon for heterophilic graph experiments. Results are shown in

Table 3. The ablation experiment results of LDC-GAT on Chameleon.

DRG-Filtering	Weight Norm Constraints	Fro-FWNAdam	Acc	Loss
×	×	×	60.31%	1.2416
✓	×	×	63.14%	1.1317
✓	✓	×	65.74%	1.0288
✓	✓	✓	67.79%	0.9677

.

The proposed triple collaborative enhancement mechanism effectively captures graph structural features, leading to improved accuracy and reduced loss. To address the performance limitations of traditional optimizers under constraints [21], we introduce the FWNAdam algorithm, which efficiently corrects gradient updates. To further validate the reliability of our network, we re-ran all experiments 10 times with different random seeds, reporting the mean accuracy, standard deviation, and 95% confidence intervals.

The experimental results on homophilic graph datasets (Cora, Citeseer, and PubMed) demonstrate that LDC-GAT consistently outperforms GAT in terms of accuracy. Specifically, LDC-GAT achieves a 5.70% improvement on Citeseer. These improvements are statistically significant, with p-values of 0.011, 0.002, and 0.025, respectively. This indicates that the enhancements provided by LDC-GAT are reliable and not due to random fluctuations. The 95% confidence intervals for these results further confirm the robustness of LDC-GAT’s performance on homophilic graphs. Specific results are shown in

Table 4. Model performance on homophilic graph datasets: Mean accuracy, standard deviation, confidence interval, and statistical significance of GAT and LDC-GAT.

Metric	Cora	Citeseer	PubMed
GAT mean accuracy	83.90%	48.75%	77.70%
LDC-GAT mean accuracy	84.50%	54.45%	78.20%
Standard deviation (±)	±0.0031	±0.0042	±0.0045
Confidence interval (95%)	[84.2%, 84.8%]	[53.9%, 55.0%]	[78.0%, 78.4%]
p-value	0.011	0.002	0.025

below.

For heterophilic graph datasets such as Chameleon, Texas, and Squirrel, LDC-GAT demonstrates significant improvements over GAT, as summarized in

Table 5. Model performance on heterophilic graph datasets: Mean accuracy, standard deviation, confidence interval, and statistical significance of GAT and LDC-GAT.

Metric	Chameleon	Texas	Squirrel
GAT mean accuracy	60.31%	60.31%	42.65%
LDC-GAT mean accuracy	67.79%	81.88%	70.13%
Standard deviation (±)	±0.0050	±0.0020	±0.0050
Confidence interval (95%)	[67.4%, 68.2%]	[81.6%, 82.1%]	[69.8%, 70.5%]
p-value	0.0005	0.0001	0.0012

. In particular, LDC-GAT outperforms GAT, with improvements that are all statistically significant, with p-values of 0.0005, 0.0001, and 0.0012, respectively. These results confirm LDC-GAT’s superior adaptability in handling complex heterophilic graph structures, where nodes with diverse features or labels are involved.

To assess the superiority of LDC-GAT, we compared its performance with six classic baseline models. As shown in

Table 6. Mean accuracy (%) comparison of model performance on homophilic and heterophilic graph datasets.

	Homophilic Graph Datasets			Heterophilic Graph Datasets
Model	Cora	Citeseer	PubMed	Chameleon	Texas	Squirrel
Graph SAGE	78.00	52.19	76.00	66.67	56.76	56.20
MLP	57.00	53.35	72.90	50.44	78.38	42.56
ChebNet	73.60	53.85	69.00	58.77	81.78	39.75
GCN	83.60	50.23	77.90	47.55	37.84	69.80
GIN	76.40	47.60	77.00	66.89	56.75	52.16
GAT	83.90	48.75	77.70	61.31	60.31	42.65
LDC-GAT	84.50	54.45	78.20	67.79	81.88	70.13

, LDC-GAT consistently outperformed other models.

GCN and GAT face performance limitations on heterophilic graphs due to their neglect of cross-type feature interactions [32]. LDC-GAT addresses this by leveraging a residual-enhanced multi-scale attention propagation rule and an adaptive learning rate adjustment strategy to enhance mutual information between heterophilic nodes. Consequently, LDC-GAT achieves a 0.1% to 0.9% improvement in average test accuracy on heterophilic graph datasets such as Chameleon, Texas, and Squirrel, surpassing other top baseline models.

To comparatively assess the robustness of GAT and LDC-GAT under graph sparsity conditions, we systematically removed 10–50% of edges from benchmark datasets to simulate topological degradation and evaluated model performance (

Table 7. Mean accuracy (%) comparison of GAT and LDC-GAT when the edge removal ratio increases from 10% to 50%.

Edge Removal Ratio	Drop 10%		Drop 30%		Drop 50%
Model	GAT	LDC-GAT	GAT	LDC-GAT	GAT	LDC-GAT
Cora	83.1	83.6	78.6	81.9	76.6	79.6
Citeseer	46.17	52.79	45.38	51.89	44.13	51.42
PubMed	75.2	77.92	72.3	75.68	71.8	75.2
Chameleon	59.65	66.94	59.64	66.91	59.36	65.8
Squirrel	40.46	70.08	40.25	69.88	39.88	69.7
Texas	57.76	79.76	56.76	78.46	54.05	76.05

).

The experimental results demonstrate that as the edge removal ratio increases from 10% to 50%, GAT exhibits an average performance degradation of 6.7%, whereas LDC-GAT shows only a 4.2% reduction. Notably, on datasets including Citeseer and PubMed, LDC-GAT maintains high accuracy even at 50% edge removal. Under extreme sparsity conditions (50% edge deletion), LDC-GAT outperforms GAT by an average margin of 13.2% in accuracy. These findings substantiate the superior adaptability of our method in sparse graph scenarios.

8. Conclusions

This paper mitigates the inherent limitations of graph attention networks in dynamic topology adaptation and training stability by constructing a full-stack solution for stabilizing graph neural network training from three dimensions: theoretical modeling based on discrete ODE stability, algorithm design, and architectural innovation. We creatively propose the Lyapunov-Stable Graph Attention Network with Dynamic Filtering and Constraint-Aware Optimization (LDC-GAT) featuring residual graph filtering and Frobenius norm threshold-based dynamic learning rate adaptation. The core innovations achieved are as follows:

• In semi-supervised classification tasks, LDC-GAT demonstrates an average accuracy improvement of approximately 10.54% over vanilla GAT across six benchmark datasets (including Cora and PubMed), validating the effectiveness of multi-band signal fusion enabled by its dynamic residual filtering mechanism for cross-layer feature injection.
• For constrained optimization, LDC-GAT pioneers explicit parameter-space compression through Frobenius norm constraint boundaries on multi-head weight matrices. The proposed collaborative optimization algorithm, Fro-FWNAdam, integrates dynamic learning rate adaptation, Nesterov momentum acceleration, and Frank–Wolfe direction search. It improves convergence speed by 10.6% while ensuring constraint feasibility, making it well suited for constrained scenarios.

In futurework, the stability-aware design of LDC-GAT could be extended to dynamic or temporal graphs, where topological changes over time further challenge propagation stability. Additionally, generalizing Fro-FWNAdam to support alternative constraint types such as spectral norm or structured sparsity may enhance its adaptability to broader optimization scenarios in graph learning. LDC-GAT’s potential in combinatorial optimization for real-world applications like contact tracing and social network analysis should also be explored. The DeepTrace framework [25] and Rumor Centrality method [26] highlight the importance of graph constraint-aware optimization, with applications in epidemic modeling and rumor propagation. LDC-GAT’s dynamic filtering and optimization strategies could significantly contribute to solving graph-constrained optimization problems, advancing the field of combinatorial optimization in networked systems.