Self-Attention Mechanisms in HPC Job Scheduling: A Novel Framework Combining Gated Transformers and Enhanced PPO

Abstract

In HPC systems, job scheduling plays a critical role in determining resource allocation and task execution order. With the continuous expansion of computing scale and increasing system complexity, modern HPC scheduling faces two major challenges: a massive decision space consisting of tens of thousands of computing nodes and a huge job queue, as well as complex temporal dependencies between jobs and dynamically changing resource states.Traditional heuristic algorithms and basic reinforcement learning methods often struggle to effectively address these challenges in dynamic HPC environments. This study proposes a novel scheduling framework that combines GTrXL with PPO, achieving significant performance improvements through multiple technical innovations. The framework leverages the sequence modeling capabilities of the Transformer architecture and selectively filters relevant historical scheduling information through a dual-gate mechanism, improving long sequence modeling efficiency compared to standard Transformers. The proposed SECT module further enhances resource awareness through dynamic feature recalibration, achieving improved system utilization compared to similar attention mechanisms. Experimental results on multiple datasets (ANL-Intrepid, Alibaba, SDSC-SP2) demonstrate that the proposed components achieve significant performance improvements over baseline PPO implementations. Comprehensive evaluations on synthetic workloads and real HPC trace data show improvements in resource utilization and waiting time, particularly under high-load conditions, while maintaining good robustness across various cluster configurations.

1. Introduction

With the acceleration of digital transformation and the widespread adoption of computationally intensive applications in fields such as scientific research, industrial engineering, and business analytics [1], scheduling problems in HPC environments have become increasingly complex and face higher performance requirements [2]. In modern HPC systems, coordinating thousands of nodes and managing large-scale task submissions across diverse objectives and resource constraints presents substantial challenges [3,4]. However, traditional scheduling methods typically rely on static heuristic algorithms and fixed assumptions, making them ill suited to the inherent dynamism and uncertainty of real-world HPC workloads [5,6].

Increasing workload heterogeneity and time complexity exacerbate job scheduling difficulty, particularly due to temporal relationships between jobs and resources. [7,8]. These relationships include: (i) execution sequence constraints, where the output of one job must be used by another job, such as variant detection in genomics requiring prior sequence alignment results; (ii) runtime trigger conditions, where a task can only be scheduled after other tasks are completed or specific conditions are met, such as in multi-stage engineering simulations [9]; and (iii) resource contention, where concurrent requests for scarce resources (e.g., GPUs or bandwidth) may lead to latency spikes or idle time. Research indicates that unmanaged competition can reduce system throughput by up to 40% [10].

This challenge is further exacerbated under diverse workloads. Computing-intensive tasks require CPU/GPU throughput, while data-intensive tasks require high I/O bandwidth [11,12]. Modern architectures employ hierarchical storage and dedicated interconnect technologies, further complicating the scheduling process [13].

Traditional scheduling algorithms have obvious shortcomings: FCFS often leads to low resource utilization [14], while SJF may cause long job starvation [15]. Additionally, rule-based and priority-based schemes typically exhibit limited adaptability and high system maintenance complexity [16].

To address these challenges, we propose GTrXL-SPPO, a unified framework that combines Gated GTrXL with PPO. GTrXL captures long-term sequence patterns via multi-head attention mechanisms [17], while GRU-inspired gating mechanisms enhance the retention of essential scheduling signals [18,19]. Additionally, the SE module [20] further enhances the model’s ability to focus on critical scheduling information by adaptively reweighting input features.

In our model, we employ PPO for policy learning [21], leveraging its truncated objective function to ensure stable training in volatile HPC environments. PPO demonstrates superior sampling efficiency and convergence in high-dimensional control tasks [22], and we further incorporate adaptive learning rates to enhance responsiveness to workload fluctuations.

Evaluations on synthetic and real-world datasets (including ANL-Intrepid, Alibaba, and SDSC-SP2) show that our model consistently outperforms traditional methods and standard PPO in terms of throughput, latency, and resource utilization. Additionally, GTrXL-SPPO demonstrates strong generalization capabilities across different system architectures and workload types.

The key contributions of this paper include:

• We design a dual-channel policy network based on GTrXL to capture long-term job sequences.
• We propose a lightweight SE layer for dynamic resource reweighting in scheduling-sensitive environments.
• We developed a reinforcement learning framework based on PPO, incorporating exponential clipping and memory replay to achieve better convergence.
• We validated our model on large-scale HPC task sequences, demonstrating performance superior to existing baselines.

The remainder of this paper is organized as follows: Section 2 describes the background and related work in the field of HPC job scheduling. Section 3 introduces our proposed GTrXL-SPPO framework, including architectural design and training methods. Section 4 details the experimental setup and evaluation metrics. Section 5 discusses the experimental results and comparative analysis. Finally, Section 7 summarizes the paper and outlines potential future work.

2. Related Work

2.1. Traditional High-Performance Computing Task Scheduling

Traditional high-performance computing (HPC) job scheduling methods, such as rule-based policies and heuristic algorithms, have long been used to coordinate resources and balance workloads.These methods typically rely on fixed strategies manually designed for specific environments. However, the increasing heterogeneity of modern HPC cluster architectures and the growing diversity of workloads pose substantial challenges to these static strategies. In particular, they often struggle to manage execution time dependencies and balance critical performance objectives such as energy efficiency, latency, and throughput [23,24,25].

Table 1. Comparative analysis of cluster scheduling methods across key features.

Characteristics	Rule [26,27]	Priority [28,29]	Backfilling [30,31]	RL [32,33]	GTrXL-SPPO
Long-term Effects	×	×	×	✔	✔
Auto Policy Tuning	×	×	×	✔	✔
Dynamic Resource Priority	×	×	✔	×	✔
Temporal Relation Handling	×	×	×	×	✔

summarizes the inherent limitations of representative baseline methods, highlighting the necessity of adaptive and learning-based scheduling methods.

In addition to rule-based techniques, previous studies have explored various methods, including feedback control theory, queuing models, and evolutionary optimization algorithms (e.g., genetic algorithm-based schedulers). While these methods are effective in constrained or static environments, they typically struggle to demonstrate the required flexibility and scalability under the real-time constraints of dynamic, large-scale HPC systems.

Case Study Analysis

In earlier versions, Google’s Borg system primarily used a first-come, first-served (FCFS) policy. However, under fluctuating workloads, this policy proved insufficient to ensure the reliable restart of service tasks and batch tasks.

Similarly, traditional batch scheduling frameworks, such as Moab/Maui, LoadLeveler, LSF, PBS, and Sun Grid Engine (SGE), typically rely on space-sharing mechanisms and a two-level queue architecture. These systems sort job queues based on submission time and allocate tasks to idle nodes using static resource matching strategies. However, such static heuristic algorithms often result in poor system performance, exacerbated resource fragmentation, and reduced predictability of performance metrics [34].

For example, PBS Pro often experiences task processing delays when faced with a surge of high-priority tasks, demonstrating the rigid nature of traditional heuristic algorithms in real-time decision-making. Among these traditional schedulers, SLURM [27] is one of the most widely deployed systems in modern HPC environments, including deployments at LLNL, NERSC, and TACC. It adopts a modular, plugin-driven design, containing components such as priority, scheduler, and task submission, and primarily allocates computing nodes through a queue-based space sharing mechanism [35].

Although SLURM offers scalability and integration flexibility, it inherently relies on static heuristic logic. Under dynamic workloads, especially when rigid filling strategies conflict with actual runtime task patterns [3], this often leads to low resource utilization, task starvation, and task startup delays. These challenges highlight the urgent need for adaptive scheduling strategies.

To address this issue, deep learning-based policy learning methods (DRL) such as Approximate PPO and the GTrXL-SPPO proposed in this paper have emerged as promising alternatives. These methods achieve policy adaptation through continuous interaction with the environment and demonstrate superior performance in managing the long-term scheduling complexity of diverse and volatile workloads [36].

2.2. Deep Reinforcement Learning

Unlike continuous control tasks (such as robot control), high-performance computing (HPC) job scheduling inherently involves discrete decisions: assigning specific jobs to specific computing nodes or deferring scheduling decisions. Therefore, our GTrXL-SPPO framework adopts a discrete action space, where each action corresponds to a specific job-to-node assignment or waiting decision. The size of the action space varies with the number of available nodes plus one waiting action, as shown in our implementation on an 800-node cluster (Section 4.4).

As shown in

Table 2. Comparison of mainstream DRL algorithms for HPC scheduling.

Features	DQL [37]	PG [38]	PPO [21]	SAC [39]	A3C [40]
Sample Efficiency	×	✔	✔	✔	×
Discrete Action Support	✔	✔	✔	×	✔
Training Stability	✔	×	✔	✔	×
Tuning Ease	✔	×	×	✔	×
Parallel Support	×	×	✔	×	✔
Compute Overhead	Low	Medium	Medium	High	Medium

, PPO has become a widely adopted baseline due to its robust performance and stable training characteristics. By constraining policy updates with a clipped surrogate objective, PPO mitigates the instability issues common in other policy gradient methods.

DRL-based schedulers can eliminate the need for manually designed heuristic methods by continuously interacting with the environment to learn optimal decision strategies [3,41]. Algorithms such as PPO, A2C, and DQL have demonstrated significant effectiveness in reducing task delays and improving resource utilization [42,43] and have been extended to multi-objective optimization scenarios. A recent review paper [44] points out that modeling long-term temporal relationships and rewards can further improve DRL scheduling performance.

Scheduling problems are typically modeled as multi-state multi-objective reinforcement learning (MDP) problems, where agents observe system states, select scheduling actions, receive feedback (rewards), and update their policies accordingly [45,46]. The environment state includes task attributes and system resource metrics, while the reward function is designed to incentivize desirable behaviors, such as maximizing throughput, minimizing delay, and balancing load.

2.3. Application of Transformer Mechanisms in Scheduling

The self-attention mechanism of the Transformer architecture is widely used in scheduling tasks because it can effectively model long-term sequential dependencies [47]. Unlike RNN-based models, Transformers support parallel processing of sequence elements, significantly accelerating training speed [48]. Additionally, their global attention structure can model complex cross-task temporal relationships and provide better interpretability through attention heatmaps.

In the field of HPC scheduling, Transformer-based models offer promising solutions for adapting to dynamic and heterogeneous workloads. Their flexibility allows for encoding diverse task features and system states, while their scalability supports efficient deployment in large-scale environments. Therefore, combining Transformer mechanisms with DRL strategies is emerging as a promising direction for the next generation of intelligent schedulers.

3. Methodology

3.1. Problem Statement

In HPC task scheduling problems, the scheduler manages a dynamic task queue, where each task has specific resource requirements and execution time. The primary objectives of resource scheduling are to optimize resource allocation, minimize task waiting time, improve resource utilization, and maximize system throughput. In this study, GTrXL-SPPO models the scheduler as an intelligent agent that determines the optimal allocation of idle computing nodes to tasks in the waiting queue, as shown in Figure 1. The intelligent agent retrieves job characteristics and resource node states from the environment and concatenates this information into a unified state vector. Within the Markov Decision Process (MDP) framework, the state space and action space are systematically modeled to support intelligent scheduling decisions. Through continuous interaction with the environment, the intelligent agent learns the optimal scheduling strategy.

Environment Module (upper part): Represents a high-performance computing (HPC) multi-resource scheduling system, consisting of four core components: (1) Job Resource Information Management Module, responsible for maintaining dynamic job queues; (2) Window-based Resource Allocation Mechanism, used for resource scheduling; (3) GTrXL History Buffer, storing past scheduling decisions and system states; (4) Multi-step Feedback Mechanism, providing real-time system state updates. Agent Module (Lower Part): Implements an actor–critic architecture, where the actor network processes state information through the GTrXL layer (handling long-term dependencies) and the SE layer (enhancing features) to output scheduling strategies. The critic network evaluates state values to optimize strategies. Reward feedback evaluates scheduling effectiveness (resource utilization, waiting time), state feedback updates system conditions, and historical feedback enables experience-based learning through buffers. Arrows indicate the direction of data flow between components.

3.1.1. MDP Formulation

We model the HPC scheduling problem as an MDP: At each step, the agent observes the system state ${\mathbf{s}}_t$ and selects an action $a_t$ to assign tasks to idle nodes.

To handle resource constraints and priority requirements in HPC environments, we extend the basic MDP to a constrained Markov Decision Process (CMDP), which can be formally defined as a tuple $\langle \mathcal{S},\mathcal{A},\mathcal{P},\mathcal{R},\mathcal{C},\gamma \rangle$, where:

• $\mathcal{S}$ is the state space, representing all possible system configurations
• $\mathcal{A}$ is the action space, representing all possible job-to-node assignments
• $\mathcal{P}:\mathcal{S}\times \mathcal{A}\times \mathcal{S}\to [0,1]$ is the transition probability function
• $\mathcal{R}:\mathcal{S}\times \mathcal{A}\times \mathcal{S}\to \mathbb{R}$ is the reward function
• $\mathcal{C}:\mathcal{S}\times \mathcal{A}\times \mathcal{S}\to {\mathbb{R}}^m$ is a vector of m constraint functions
• $\gamma \in [0,1)$ is the discount factor

The optimization objective is to find a policy ${\pi }^{\ast }$ that maximizes the expected cumulative discounted reward while satisfying the constraints:

$$\begin{array}{c}\hfill {\pi }^{\ast }=arg\underset{\pi }{max}{\mathbb{E}}_{\tau \sim \pi }\left[\sum _{t=0}^{\infty }{\gamma }^t\mathcal{R}({\mathbf{s}}_t,a_t,{\mathbf{s}}_{t+1})\right]\end{array}$$

(1)

subject to:

$$\begin{array}{c}\hfill {\mathbb{E}}_{\tau \sim \pi }\left[\sum _{t=0}^{\infty }{\gamma }^t{\mathcal{C}}_i({\mathbf{s}}_t,a_t,{\mathbf{s}}_{t+1})\right]\le b_i,\forall i\in \{1,2,\dots ,m\}\end{array}$$

(2)

where $\tau =({\mathbf{s}}_0,a_0,{\mathbf{s}}_1,a_1,\dots )$ is a trajectory, and $b_i$ are threshold constants for each constraint.

3.1.2. State and Action Space

The system state ${\mathbf{s}}_t$ contains comprehensive job and system information with a hierarchical structure. At the high level, we organize state information into functional modules: ${\mathbf{s}}_t=[J_t,W_t,R_t,h_t]$ where $J_t$ represents the job feature vector including the number of nodes requested by job i (reqProc), the runtime requested by job i (reqTime), and the waiting time $w_i$ (currentTime − submitTime). $W_t$ denotes the waiting time vector across all queued jobs, $R_t$ represents the current resource state vector indicating node availability and utilization, and $h_t$ contains the historical information buffer that tracks past scheduling decisions and system performance.

At decision step t, the implementation-level state representation ${\mathit{\varphi }}_t$ reflects the real-time status of system resources and pending tasks. It contains key information such as CPU, GPU, and memory usage, as well as task-specific attributes including resource requests, estimated execution time, and waiting time. Formally, the state space can be modeled as a combination of task windows and node pools. For each waiting task i, ${\mathrm{req}}_i$ denotes the number of requested resource cores, $w_i$ denotes the waiting time, defined as $w_i=t_{\mathrm{current}}-t_{\mathrm{arrival}}^{\left(i\right)}$, $p_i$ denotes the assigned priority, and $e_i$ denotes the estimated execution time. For each compute node k, $n_k$ denotes the node’s available state, and $r_k$ denotes the remaining execution time of the current task on node k. The detailed state representation at time step t can be represented as:

$${\mathit{\varphi }}_t=\bigcup _{i=1}^n[{\mathrm{req}}_i,w_i,p_i,e_i]⨁\bigcup _{k=1}^N[n_k,r_k]$$

(3)

where n denotes the number of tasks to be processed, and N denotes the total number of resource nodes.

The action space is implemented as a discrete space whose size is equal to the window size, formally defined as $A=\{0,1,2,\dots ,\mathrm{window\_size}-1\}$. Each action $a_t\in A$ represents an index in the current waiting queue. The agent selects this index to determine the task to be scheduled with the highest priority. The action mechanism is implemented through queue reordering: when action $a_t=i$ is selected, the task with index i in the waiting queue is moved to the front of the queue, thereby obtaining the highest scheduling priority. In practical applications, when the waiting queue size is smaller than the window size, the effective action space is dynamically restricted to $A_{\mathrm{eff}}=\{0,1,2,\dots ,min(\mathrm{window\_size},\left|\mathrm{wait\_queue}\right|)-1\}$. This discrete action modeling solves the task selection problem while maintaining computational feasibility, with the size of the action space varying with the observation window size.

This hierarchical design enables the agent to comprehensively observe the characteristics of the task queue and the real-time status of the resource pool, providing sufficient information for making scheduling decisions within the CMDP framework.

3.1.3. Decision Variables and Constraints

The scheduling decision at each time step can be represented by the following decision variables:

• $x_{j,n,t}\in \{0,1\}$: Binary variable indicating whether job j is assigned to node n at time step t
• $y_{j,t}\in \{0,1\}$: Binary variable indicating whether job j starts execution at time step t

These decision variables are subject to the following constraints:

• Resource Capacity Constraint: Each node can be assigned to at most one job at any time

  $$\begin{array}{c}\hfill \sum _{j\in J}{x}_{j,n,t}\le 1,\forall n\in N,\forall t\end{array}$$

  (4)
• Job Completeness Constraint: A job must receive all its requested resources or none at all

  $$\begin{array}{c}\hfill \sum _{n\in N}{x}_{j,n,t}={\mathrm{req}}_j·y_{j,t},\forall j\in J,\forall t\end{array}$$

  (5)
• Non-preemption Constraint: Once a job starts execution, it must run to completion

  $$\begin{array}{c}\hfill y_{j,t}=1\Rightarrow \forall t^{\prime }\in [t,t+d_j),\sum _{n\in N}{x}_{j,n,t^{\prime }}={\mathrm{req}}_j\end{array}$$

  (6)
• Priority Constraint: Higher priority jobs with long waiting times should be scheduled before lower priority jobs

  $$\begin{array}{c}\hfill (p_j>p_{j^{\prime }})\wedge (w_j>w_{\mathrm{threshold}})\Rightarrow \mathrm{schedule\_priority}\left(j\right)>\mathrm{schedule\_priority}\left(j^{\prime }\right)\end{array}$$

  (7)

In practice, the resource capacity and job completeness constraints are enforced through action space design and action masking, while the non-preemption and priority constraints are encouraged through reward shaping.

3.2. Neural Architecture Design

At the upper decision layer, the agent evaluates jobs in the waiting queue and decides whether to execute or defer their scheduling, ensuring fair resource allocation through priority adjustment and reservation strategies. At the lower decision layer, the agent maximizes resource utilization by filling idle nodes with smaller jobs. This hierarchical decision structure enhances the flexibility and robustness of the scheduler, ensuring fairness for large jobs while improving overall resource efficiency under dynamic load conditions. The overall neural network architecture supporting this scheduling strategy is illustrated in Figure 2, detailing input feature processing, SE-based feature enhancement, GTrXL sequence modeling, and policy/value output branches. The system-level scheduling process (environment, job queue, reservation mechanism, agent decisions) was previously depicted in Figure 1. Here, Figure 2 focuses specifically on the internal design of the neural network responsible for decision-making.

3.3. Multi-Scale Feature Enhancement

To address the challenge of dynamically allocating weights for heterogeneous tasks and system features in HPC environments, especially in scenarios with burst workloads and resource competition, we enhance traditional reinforcement learning by introducing an adaptive feature recalibration mechanism. Specifically, the SE module [20] combines average pooling with a bottleneck multi-layer perceptron (channel → channel/4 → channel) and a Sigmoid activation function to capture global temporal relationships between feature channels, effectively suppressing irrelevant features while amplifying key features. The computational overhead is low; for example, adding SE to ResNet-50 increases parameters by only 10% but significantly improves accuracy [20].

In our architecture, the SE block is placed before the GTrXL layer in both the actor network and critic network, enabling temporal modeling to operate on the enhanced features.

To extend adaptability beyond static representations, we propose a novel SECT layer that fuses global statistical information and scheduling-aware dynamics by combining average and max aggregation Algorithm 1.

$$\begin{array}{cc}\hfill z_{\mathrm{fusion}}& ={\displaystyle \frac{1}{2}}(\mathrm{mean}\left(x\right)+\max \left(x\right))\hfill \end{array}$$

(8)

$$\begin{array}{cc}\hfill w_{\mathrm{base}}& =\sigma (W_2·\mathrm{Gelu}(W_1·z_{\mathrm{fusion}}))\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill w_{\mathrm{sched}}& =\sigma (W_4·\mathrm{Dropout}\left(\mathrm{Gelu}(W_3·\mathrm{mean}\left(x\right))\right))\hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill w_{\mathrm{final}}& ={\displaystyle \frac{w_{\mathrm{base}}+w_{\mathrm{sched}}}{2}},x_{\mathrm{out}}=x\otimes w_{\mathrm{final}}\hfill \end{array}$$

(11)

This design adapts channel weights to workload volatility, enhancing robustness without significant overhead [20].

Algorithm 1 SECT Layer

Require:  Input tensor $\mathbf{X}\in {\mathbb{R}}^{B\times L\times C}$, Channel dimension C, Reduction ratio $r=4$ Ensure:  Output tensor $\mathbf{Y}\in {\mathbb{R}}^{B\times L\times C}$   1: function SECT($\mathbf{X}$)   2:     $B,L,C\leftarrow \mathrm{shape}\left(\mathbf{X}\right)$   3:     $\mathbf{s}\leftarrow {\displaystyle \frac{1}{L}}{\sum }_{i=1}^L{\mathbf{X}}_{:,i,:}$                   ▹ Global Average Pooling   4:     ${\mathbf{z}}_1\leftarrow \mathrm{Linear}(\mathbf{s},C/r)$               ▹ First fully connected layer   5:     ${\mathbf{z}}_2\leftarrow \mathrm{ReLU}\left({\mathbf{z}}_1\right)$   6:     ${\mathbf{z}}_3\leftarrow \mathrm{Linear}({\mathbf{z}}_2,C)$               ▹ Restore channel dimension   7:     $\mathbf{a}\leftarrow \mathrm{Sigmoid}\left({\mathbf{z}}_3\right)$   8:     $\mathbf{Y}\leftarrow \mathbf{X}\odot \mathbf{a}.\mathrm{unsqueeze}\left(1\right)$   9:     return $\mathbf{Y}$ 10: end function

Mathematical Variable Definitions:

• B: Batch size representing the number of parallel scheduling instances
• L: Sequence length corresponding to the number of jobs/nodes in the current scheduling window
• C: Channel dimension representing the feature dimension of each job/node embedding
• r: Reduction ratio controlling the bottleneck dimension ($C/r$) in the excitation pathway
• $\mathbf{s}\in {\mathbb{R}}^{B\times C}$: Global statistics vector obtained through temporal average pooling across the sequence dimension
• ${\mathbf{z}}_1\in {\mathbb{R}}^{B\times C/r}$: Compressed feature representation after dimensionality reduction
• ${\mathbf{z}}_2\in {\mathbb{R}}^{B\times C/r}$: Activated features after ReLU non-linearity
• ${\mathbf{z}}_3\in {\mathbb{R}}^{B\times C}$: Restored feature dimension after the second linear transformation
• $\mathbf{a}\in {\mathbb{R}}^{B\times C}$: Channel-wise attention weights normalized to $[0,1]$ range via sigmoid activation
• $\mathbf{a}.\mathrm{unsqueeze}\left(1\right)$: Expands attention weights to shape ${\mathbb{R}}^{B\times 1\times C}$ for broadcasting across sequence dimension
• ⊙: Element-wise multiplication (Hadamard product) applying channel attention to input features

The SECT mechanism learns to emphasize important scheduling-relevant channels while suppressing less informative features through this squeeze-and-excitation operation.

3.4. Dynamic Memory Mechanism

In high-performance computing (HPC) scheduling, task queues with heterogeneous arrival times and different resource requirements can lead to complex timing patterns. This O(n²) complexity stems from the self-attention mechanism that computes pairwise interactions between all task positions [17], and the fixed context window constraint leads to performance degradation when exceeding a certain threshold [49].

3.4.1. GTrXL: Enhanced Memory Architecture

To address these empirically validated limitations, we propose GTrXL (Gated Transformer-XL), an enhanced Transformer architecture specifically designed for long-term temporal modeling in HPC scheduling scenarios. GTrXL builds upon the segment-level recursive mechanism [49] to model dependencies beyond fixed-length context without compromising temporal consistency. GTrXL supports a relaxed Markov approximation where $\pi \left(a_t\right|s_t)$ is determined by the most recent sequence of states $[s_{t-k},\dots ,s_t]$. Unlike standard Markov models that only consider immediate states, our method preserves contextual information over extended time spans [50,51]. To maintain temporal consistency between scheduled segments, GTrXL uses relative position encoding instead of absolute position encoding. This enables generalization to sequences longer than training sequences—critical for handling variable-length task queues in HPC workloads. A GRU-based gating system dynamically regulates information flow within each Transformer block, selectively preserving relevant sequential information while filtering outdated inputs as Algorithm 2.

Algorithm 2 Gated Transformer-XL Core Module in GTrXL-SPPO

Require:  State input $\mathbf{X}$, Temporal memory $\mathbf{M}$, Attention mask mask Ensure:  Output feature $\mathbf{Y}$, Updated memory ${\mathbf{M}}_{\mathrm{new}}$   1: function GTrXLBlock($\mathbf{X},\mathbf{M},\mathrm{mask}$)   2:     ${\mathbf{X}}^{\prime }\leftarrow \mathrm{LayerNorm}\left(\mathbf{X}\right)$                            ▹ Input   3:     ${\mathbf{M}}^{\prime }\leftarrow \mathrm{LayerNorm}\left(\mathbf{M}\right)$                        ▹ Memory   4:     $\mathbf{A}\leftarrow \mathrm{MultiHeadAttention}({\mathbf{X}}^{\prime },{\mathbf{M}}^{\prime },{\mathbf{M}}^{\prime },\mathrm{mask})$   5:     $\mathbf{r}\leftarrow \sigma (W_r\mathbf{A}+U_r\mathbf{X})$                  ▹ Reset gate computation   6:     $\mathbf{z}\leftarrow \sigma (W_z\mathbf{A}+U_z\mathbf{X}-b_g)$                       ▹ Update gate   7:     $\tilde{\mathbf{h}}\leftarrow tanh(W_g\mathbf{A}+U_g(\mathbf{r}\odot \mathbf{X}))$   8:     ${\mathbf{H}}_1\leftarrow (1-\mathbf{z})\odot \mathbf{X}+\mathbf{z}\odot \tilde{\mathbf{h}}$                     ▹ 1st GRU output   9:     $\mathbf{F}\leftarrow \mathrm{FeedForward}\left(\mathrm{LayerNorm}\left({\mathbf{H}}_1\right)\right)$ 10:     ${\mathbf{r}}_2\leftarrow \sigma (W_r\mathbf{F}+U_r{\mathbf{H}}_1)$                  ▹ Reset gate (2nd GRU) 11:     ${\mathbf{z}}_2\leftarrow \sigma (W_z\mathbf{F}+U_z{\mathbf{H}}_1-b_g)$               ▹ Update gate (2nd GRU) 12:     ${\tilde{\mathbf{h}}}_2\leftarrow tanh(W_g\mathbf{F}+U_g({\mathbf{r}}_2\odot {\mathbf{H}}_1))$ 13:     $\mathbf{Y}\leftarrow (1-{\mathbf{z}}_2)\odot {\mathbf{H}}_1+{\mathbf{z}}_2\odot {\tilde{\mathbf{h}}}_2$ 14:     return $\mathbf{Y}$ 15: end function

3.4.2. Temporal Dependency Modeling

GTrXL models the temporal execution order by combining multi-head self-attention with relative position encoding. Each attention head computes queries, keys, and values:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}Q=X{W}_Q\in {\mathbb{R}}^{L\times d}\hfill \\ K=X{W}_K\in {\mathbb{R}}^{L\times d}\hfill \\ V=X{W}_V\in {\mathbb{R}}^{L\times d}\hfill \end{array}\right.\end{array}$$

(12)

Relative position encodings capture execution order relationships:

$$\begin{array}{c}\hfill R_{i,j}={\displaystyle \frac{1}{\sqrt{d}}}cos((i-j)·\omega )\end{array}$$

(13)

$$\begin{array}{c}\hfill {\mathrm{AttnScore}}_{i,j}={\displaystyle \frac{Q_i{K}_j^T-Q_i{R}_{i-j}^T}{\sqrt{d_h}}}\end{array}$$

(14)

This design allows the agent to recognize execution-order correlations critical for effective scheduling decisions.

The GTrXL architecture employs a two-layer gating strategy within each Transformer block to adaptively control temporal context updates. The high-level operations are:

$$\begin{array}{c}\hfill A=\mathrm{MHA}(Q,K,V),h={\mathrm{GRU}}_1(Q,A)\end{array}$$

(15)

$$\begin{array}{c}\hfill F=\mathrm{FFN}\left(h\right),O={\mathrm{GRU}}_2(h,F)\end{array}$$

(16)

This gated structure enables selective temporal state updates, allowing rapid integration of critical scheduling signals (e.g., high-priority job arrivals) while maintaining stability under dynamic workloads. The complete memory flow and compression pipeline are illustrated in Figure 3.

Attention Weight Matrices:

• $X\in {\mathbb{R}}^{L\times d}$: Input sequence tensor where L is sequence length and d is model dimension
• $W_Q\in {\mathbb{R}}^{d\times d}$: Query projection matrix mapping input to query space
• $W_K\in {\mathbb{R}}^{d\times d}$: Key projection matrix mapping input to key space
• $W_V\in {\mathbb{R}}^{d\times d}$: Value projection matrix mapping input to value space
• $Q,K,V\in {\mathbb{R}}^{L\times d}$: Query, key, and value matrices for attention computation

These projection matrices are learned parameters that transform the input scheduling context into query, key, and value representations for multi-head attention.

Where $\omega \in \mathbb{R}$ is the frequency parameter that controls the rate of positional variation (typically initialized as $\omega =1/\mathrm{10,000}$), and $d_h=d/H$ represents the dimension per attention head, with H being the total number of attention heads and d being the model dimension. The relative position encoding $R_{i,j}$ captures the temporal distance between positions i and j in the scheduling sequence, enabling the model to understand execution order dependencies.

3.4.3. Implementation Details and Training Integration

The GRU-based gating mechanism within each GTrXL block follows standard gated recurrent unit formulations. For each gating layer, the operations are defined as:

$$\begin{array}{cc}\hfill r_t& =\sigma (W_r{K}_t+U_r{Q}_t)\hfill \end{array}$$

(17)

$$\begin{array}{cc}\hfill z_t& =\sigma (W_z{K}_t+U_z{Q}_t-b_g)\hfill \end{array}$$

(18)

$$\begin{array}{cc}\hfill {\tilde{h}}_t& =tanh(W_g{K}_t+U_g(r_t\odot Q_t))\hfill \end{array}$$

(19)

$$\begin{array}{cc}\hfill h_t& =(1-z_t)\odot Q_t+z_t\odot {\tilde{h}}_t\hfill \end{array}$$

(20)

GRU Gating Parameters:

• $r_t\in {\mathbb{R}}^d$: Reset gate vector controlling how much past information to forget
• $z_t\in {\mathbb{R}}^d$: Update gate vector determining the balance between past and new information
• ${\tilde{h}}_t\in {\mathbb{R}}^d$: Candidate hidden state representing new information
• $h_t\in {\mathbb{R}}^d$: Final output combining past and candidate states
• $Q_t,K_t\in {\mathbb{R}}^d$: Input vectors encoding scheduling context (runtime estimates, resource status)
• $W_r,W_z,W_g\in {\mathbb{R}}^{d\times d}$: Input-to-hidden weight matrices for reset, update, and candidate gates
• $U_r,U_z,U_g\in {\mathbb{R}}^{d\times d}$: Hidden-to-hidden recurrent weight matrices
• $b_g\in {\mathbb{R}}^d$: Gate bias vector (typically initialized to 0.1 for stable training)
• $\sigma (·)$: Sigmoid activation function mapping values to $(0,1)$ range
• tanh$(·)$: Hyperbolic tangent activation function mapping values to $(-1,1)$ range
• ⊙: Element-wise multiplication (Hadamard product)

The weight matrices are randomly initialized using Xavier/Glorot initialization and learned during training through backpropagation.

When critical job characteristics (e.g., high-priority arrivals) emerge, the gates adaptively adjust, facilitating fast integration of new signals (Algorithm 3) and improving responsiveness under dynamic workloads.

Algorithm 3 GTrXL-SPPO collaborative processing framework

Require:  Input state $\mathbf{X}\in {\mathbb{R}}^{B\times L\times 1\times 2}$, Memory $\mathbf{M}$, Mask mask Ensure:  Actor output $\pi \left(a\right|s)$, Critic output $V\left(s\right)$   1: function CollaborativeProcessing($\mathbf{X},\mathbf{M},\mathrm{mask}$)   2:     if $\mathbf{X}.\dim \left(\right)==4$ then   3:         $\mathbf{X}\leftarrow \mathbf{X}.\mathrm{squeeze}\left(2\right)$   4:     end if   5:     ${\mathbf{X}}_{\mathrm{flat}}\leftarrow \mathbf{X}.\mathrm{view}(-1,2)$   6:     ${\mathbf{X}}_{\mathrm{fused}}\leftarrow \mathrm{ReLU}\left({\mathrm{Linear}}_{2\to 1}\left({\mathbf{X}}_{\mathrm{flat}}\right)\right)$   7:     ${\mathbf{X}}_{\mathrm{seq}}\leftarrow {\mathbf{X}}_{\mathrm{fused}}.\mathrm{view}(1,-1)$   8:     ${\mathbf{H}}_1\leftarrow \mathrm{ReLU}\left(\mathrm{Linear}\left({\mathbf{X}}_{\mathrm{seq}}\right)\right)$   9:     $\mathbf{Y}\leftarrow \mathrm{SEC}\mathrm{T}\left({\mathbf{H}}_1\right)$                ▹ Apply Algorithm 1 10:     ${\mathbf{H}}_2\leftarrow \mathrm{ReLU}\left(\mathrm{Linear}\left(\mathbf{Y}\right)\right)$ 11:     if ${\mathbf{H}}_2.\dim \left(\right)==1$ then 12:         ${\mathbf{H}}_{\mathrm{input}}\leftarrow {\mathbf{H}}_2.\mathrm{unsqueeze}\left(0\right)$ 13:     else 14:         ${\mathbf{H}}_{\mathrm{input}}\leftarrow {\mathbf{H}}_2$ 15:     end if 16:     ${\mathbf{H}}_{\mathrm{final}}\leftarrow \mathrm{GTrXLBlock}({\mathbf{H}}_{\mathrm{input}},\mathbf{M},\mathrm{mask})$        ▹ Apply Algorithm 2 17:     ${\mathbf{H}}_{\mathrm{output}}\leftarrow {\mathbf{H}}_{\mathrm{final}}.\mathrm{squeeze}\left(0\right)$ 18:     ${\mathbf{logits}}_{\mathrm{actor}}\leftarrow {\mathrm{Linear}}_{\mathrm{action}}\left({\mathbf{H}}_{\mathrm{output}}\right)$ 19:     $\pi \left(a\right|s)\leftarrow \mathrm{Softmax}\left({\mathbf{logits}}_{\mathrm{actor}}\right)$ 20:     $V\left(s\right)\leftarrow {\mathrm{Linear}}_{\mathrm{value}}\left({\mathbf{H}}_{\mathrm{output}}\right)$ 21:     return $\pi \left(a\right|s),V(s)$ 22: end function

GTrXL-SPPO Collaborative Integration Analysis:

The collaborative framework synergistically combines SECT (Algorithm 1) and GTrXL (Algorithm 2) to maximize their complementary advantages in HPC scheduling:

Processing Pipeline:

• Input Standardization: Consolidates dual-channel HPC state information through 2→1 linear transformation
• Feature Enhancement: Applies SECT channel attention to amplify scheduling-relevant features
• Temporal Modeling: Leverages GTrXL gated Transformer blocks for long-range dependency capture
• Shared Dual Output: Both actor and critic networks utilize identical enhanced representations

GTrXL-SPPO collaboration enhances feature quality through SECT preprocessing, improves temporal modeling through GTrXL processing of refined features, achieves computational efficiency through a shared pipeline architecture, and enables memory-aware processing that preserves critical scheduling history across sessions.

3.5. Training Strategy

3.5.1. Algorithm Design

GTrXL-SPPO employs the PPO algorithm to update scheduling policies via gradient ascent, combining policy- and value-based reinforcement learning for stable convergence [21]. The enhanced GTrXL serves as the temporal backbone to capture sequential scheduling dynamics in HPC environments.

The clipped surrogate objective is:

$$\begin{array}{cc}\hfill {\mathcal{L}}^{\mathrm{clip}}\left(\theta \right)& ={\mathbb{E}}_t\left[min\left(r_t\left(\theta \right){\widehat{A}}_t,\mathrm{clip}\left(r_t\left(\theta \right),1-ϵ,1+ϵ\right){\widehat{A}}_t\right)\right]\hfill \end{array}$$

(21)

where $r_t\left(\theta \right)={\displaystyle \frac{{\pi }_{\theta }\left(a_t\right|s_t)}{{\pi }_{{\theta }_{\mathrm{old}}}\left(a_t\right|s_t)}}$, $ϵ=0.2$, and ${\widehat{A}}_t$ is computed via Generalized Advantage Estimation (GAE):

$$\begin{array}{cc}\hfill {\widehat{A}}_t& =\sum _{l=0}^{T-t}{(\gamma ·\lambda )}^l·{\delta }_{t+l}\hfill \end{array}$$

(22)

$$\begin{array}{c}\hfill {\delta }_t=r_t+\gamma ·V_{\varphi }\left(s_{t+1}\right)-V_{\varphi }\left(s_t\right)\end{array}$$

(23)

To address the long-tail reward distribution induced by dynamic job arrivals and evolving system states, we adopt an exponentially sensitive clipping strategy and introduce a time-decaying discount factor ${\gamma }_t=1-{\displaystyle \frac{t}{T}}$, reducing the weight of late-stage decisions and stabilizing learning.

3.5.2. Constraint Handling in Training

To incorporate constraints into the reinforcement learning framework, we adopt a dual approach:

• Hard constraints: Resource capacity and task completion constraints are implemented through action space design and action masking:

  $$\begin{array}{c}\hfill {\mathcal{A}}_t=\{a\in \mathcal{A}\mid a\mathrm{satisfies}\mathrm{constraints}1\mathrm{and}2\}\end{array}$$

  (24)
• Soft constraints: Non-preemption and priority constraints are encouraged through reward shaping:

  $$\begin{array}{c}\hfill R_t^{\mathrm{constraint}}=-{\mu }_1·\mathrm{preemption\_violations}-{\mu }_2·\mathrm{priority\_violations}\end{array}$$

  (25)

This hybrid approach ensures that all scheduling decisions are feasible while guiding the strategy toward developing solutions for more complex constraints. The constrained optimization problem is thus transformed into an unconstrained problem with a modified reward function:

$$\begin{array}{c}\hfill R_t^{\mathrm{modified}}=R_t+R_t^{\mathrm{constraint}}\end{array}$$

(26)

Implementation Scope Clarification: While our CMDP framework supports constraint handling as described above, the experimental validation in this work focuses on core scheduling efficiency to establish baseline performance. Full constraint integration (deadlines, energy, thermal limits) represents ongoing development work and will be addressed in future extensions of this framework.

3.5.3. Reward Function Design

We formulate the reward function as a multi-objective optimization problem. The total reward $R_{\mathrm{total}}$ in GTrXL-SPPO integrates four scheduling objectives and a queue pressure regularization term:

$$\begin{array}{cc}\hfill R_{\mathrm{total}}& =\sum _{k=1}^4{\omega }_k·r_k+{\alpha }_{\mathrm{reg}}·r_{\mathrm{pressure}}\hfill \end{array}$$

(27)

Each component reward is mathematically defined as follows:

• System Utilization Reward:

  $$\begin{array}{c}\hfill r_1=\sigma \left(10·(u_t-0.5)\right)\end{array}$$

  (28)

  where $u_t={\displaystyle \frac{{\mathrm{occupied\_nodes}}_t}{\mathrm{total\_nodes}}}$ is the system utilization at time t, and $\sigma$ is the sigmoid function.
• Waiting Time Penalty:

  $$\begin{array}{c}\hfill r_2=-\mathrm{Huber}\left(w_t-w_{\mathrm{target}}\right)\end{array}$$

  (29)

  where $w_t$ is the average waiting time of jobs in the queue at time t, $w_{\mathrm{target}}$ is the target waiting time, and Huber is the Huber loss function that reduces sensitivity to outliers.
• Throughput Reward:

  $$\begin{array}{c}\hfill r_3=\alpha ·{\mathrm{jobs\_completed}}_t\end{array}$$

  (30)

  where ${\mathrm{jobs\_completed}}_t$ is the number of jobs completed at time t, and $\alpha$ is a scaling factor.
• Priority Reward:

  $$\begin{array}{c}\hfill r_4=\beta ·\sum _{j\in {\mathrm{scheduled}}_t}{p}_j\end{array}$$

  (31)

  where ${\mathrm{scheduled}}_t$ is the set of jobs scheduled at time t, $p_j$ is the priority of job j, and $\beta$ is a scaling factor.

The queue pressure regularization term penalizes the accumulation of jobs in the waiting queue:

$$\begin{array}{c}\hfill r_{\mathrm{pressure}}=-log(1+{\mathrm{queue\_length}}_t)\end{array}$$

(32)

where ${\alpha }_{\mathrm{reg}}=0.5$ is the regularization coefficient. The adaptive temperature parameter T controls the exploration–exploitation balance in the reward weighting mechanism:

$$\begin{array}{c}\hfill {\omega }_k^{\left(T\right)}={\displaystyle \frac{exp({\omega }_k/T)}{{\sum }_{j=1}^{4}exp({\omega }_j/T)}}\end{array}$$

(33)

This adaptive design enables the agent to switch its focus between efficiency and fairness under different load conditions. As shown in our parameter sensitivity analysis (Section 5.5), this dynamic temperature adjustment is crucial for balancing multiple scheduling objectives.

When $T=1.0$ (default), the model achieves optimal performance across all metrics; during periods of low utilization, lower values ($T=0.5$) prioritize resource efficiency; and during periods of high queue pressure, higher values ($T=2.0$) promote fairness through more balanced target weight allocation. In long-tail scheduling environments [52,53], our reward scheme accelerates convergence and improves the resource-delay Pareto frontier compared to standard PPO training.

3.6. Training Acceleration Strategies

To accelerate training convergence and improve efficiency, the GTrXL-SPPO framework adopts a comprehensive optimization strategy that combines distributed parallel training, memory replay technology, and hybrid optimization methods.

(1) Distributed Training: Parallel processing significantly accelerates the training process compared to a single GPU setup [54]. We use PyTorch to implement DDP training on multiple GPUs. The training dataset is divided into multiple shards, each assigned to a GPU device for independent gradient computation:

$$\begin{array}{c}\hfill D_k=\{(s_t^i,a_t^i,r_t^i)\mid imodN=k\},k=0,1,\dots ,N-1\end{array}$$

(34)

where N denotes the number of available GPUs, and $D_k$ denotes the data subset processed by GPU k. After local gradient updates, the AllReduce operation synchronizes gradients across all devices.

(2) Memory replay mechanism: To better capture long-term temporal patterns in scheduling dynamics, we introduce a memory replay mechanism [19,49]. Specifically, historical hidden states and intermediate features across time steps are stored in a recursive memory matrix. Combined with the GRU gating structure, this design dynamically updates memory content during training, enhancing the agent’s ability to handle time-extended task sequences.

(3) Hybrid Optimization Strategy: In the policy optimization phase, we integrate the Adam optimizer with learning rate warm-up and cosine annealing strategies to ensure a smoother training process and more stable model convergence. Additionally, to reduce communication overhead in multi-GPU environments, we apply 1-bit Adam compression, where gradients are compressed before synchronization:

$$\begin{array}{c}\hfill {\tilde{g}}_t=\mathrm{sign}\left(g_t\right)\odot \sqrt{|g_t|},g_t={\nabla }_{\theta }\mathcal{L}\left(\theta \right)\end{array}$$

(35)

This strategy significantly reduces synchronization communication while maintaining model convergence performance [55].

3.7. Scalability and Deployment Considerations

Facing practical deployment issues, we analyze the ability of GTrXL-SPPO to address three key challenges: node scaling, heterogeneity, and architecture migration.

3.7.1. Node Scaling Analysis

Our sliding window mechanism (Figure 4) ensures that the memory complexity is $O\left(W\right)$ rather than $O\left(N\right)$, where W is the fixed window size (200) and N is the total number of nodes. Combined with the attention aggregation mechanism in GTrXL, this design supports scalability without triggering linear memory growth.

3.7.2. Heterogeneous Adaptation

The SECT module (Section 3.3) supports heterogeneous environments through dynamic feature recalibration. Our experiments on ANL-Intrepid (grid architecture), Alibaba (general-purpose cluster), SDSC-SP2 (homogeneous), and PIK-IPLEX (heterogeneous) show that this module can achieve effective adaptation under different cluster configurations without architecture-specific modifications (Section 5).

3.7.3. Architecture Migration

The modular design supports migration through the following mechanisms:

• Configuration-specific model loading: The framework adapts to clusters of different scales by loading pre-trained models for different system configurations.
• Distributed training support: Multi-GPU training capabilities are implemented through PyTorch’s distributed training framework, supporting scalable training on different hardware architectures.
• General scheduling pattern learning: The GTrXL backbone network learns a general scheduling pattern that can be transferred between different computing architectures without architecture-specific retraining.

Our multi-dataset validation demonstrates cross-architecture generalization capabilities, indicating that the framework can successfully adapt to different computing scales and resource configurations without fundamental algorithmic changes.

3.8. Demonstrative Iteration of the Proposed Algorithm

To further clarify the practical application of the proposed GTrXL-SPPO framework, we present a step-by-step example of a single scheduling iteration, highlighting the use of the SECT and GTrXL modules.

Example Setup: Suppose the job queue and resource state are as follows (in

Table 3. Example job queue.

Job	Req. Nodes	Req. Time	Priority	Waiting Time	Estimated Exec Time
A	1	30	2	10	30
B	2	20	1	5	20
C	1	40	3	15	40

and

Table 4. Resource state.

Node	Availability	Remaining Time
1	Free	0
2	Busy	10

):

Step 1: State Vector Construction

The state vector $s_t$ is constructed as:

$$s_t=\{[1,10,2,30],[2,5,1,20],[1,15,3,40]\}\oplus \{[1,0],[0,10]\}$$

Step 2: Feature Enhancement via SECT Layer

The input tensor $\mathbf{X}$ is processed by the SECT layer (Algorithm 1). This adaptively recalibrates channel-wise features.

Step 3: Temporal Modeling via GTrXL Block

The enhanced features are passed through the GTrXL block (Algorithm 2), which models temporal dependencies and gating.

Step 4: Action Selection

The policy network outputs action probabilities, e.g.,

$$\pi \left(a\right|s_t)=[0.2,0.5,0.3]$$

The agent selects job B (index 2) for scheduling.

Step 5: Reward Calculation

After scheduling, the reward is computed as:

$$R_{\mathrm{total}}=\sum _{k=1}^4{\omega }_k·r_k+{\alpha }_{\mathrm{reg}}·r_{\mathrm{pressure}}$$

where $r_1$ is system utilization, $r_2$ is the waiting penalty, etc.

Step 6: Policy Update

The PPO loss is calculated and the policy is updated accordingly.

This example demonstrates how the proposed GTrXL-SPPO framework processes input features, models temporal dependencies, selects actions, and updates the policy in a single scheduling iteration.

Our approach significantly improves scheduling efficiency by optimizing both resource utilization and job waiting times through the novel GTrXL-SPPO framework.

4. Experiments Setup

4.1. Experimental Configuration and Reproducibility

To ensure full reproducibility of the experiments, we provide key experimental settings and evaluation methods.

4.1.1. Dataset Configuration and Partitioning

For each dataset (ANL-Intrepid, Alibaba, SDSC-SP2, PIK-IPLEX), we adopt a time division: training period (first 80% of task submissions) and testing period (last 20% of task submissions), with no time overlap to ensure the authenticity of the evaluation. Data preprocessing includes z-score normalization and normalization to the range [0,1].

4.1.2. Model and Training Configuration

GTrXL-SPPO employs a 12-layer Transformer architecture (12 attention heads, 768-dimensional hidden layer) combined with the SECT attention module (reduction ratio 16) and the PPO reinforcement learning algorithm. Key training parameters include: Actor learning rate $1\times {10}^{-4}$, Critic learning rate $1\times {10}^{-3}$, batch size 128, window size 200, PPO clipping ratio 0.2, GAE $\lambda$ 0.95, and total training rounds 5000. All experiments are implemented on an NVIDIA V100 GPU cluster (4 × 32 GB) based on PyTorch 1.12.0, with 30 independent random seeds (42–71) to ensure statistical reliability. Complete parameter details are provided in Appendix A.

4.1.3. Evaluation Protocol

Performance evaluation was conducted every 100 training rounds, with results recorded after a 500-round warm-up period. All results are reported as the mean, standard deviation, and 95% confidence interval of 30 independent runs, with statistical significance verified via paired t-tests (p < 0.001).

4.2. Baseline Algorithms

We evaluate GTrXL-SPPO against two categories of baselines:

• Rule-based algorithms: FCFS, SJF, and EASY, which follow fixed heuristics but lack adaptability to dynamic workloads.
• Reinforcement learning methods: Among RL approaches, PPO is selected due to its proven performance in HPC scheduling [3].

4.3. Workload Trajectories

We evaluate GTrXL-SPPO using four representative real-world workload trajectories: Alibaba [56], ANL-Intrepid [57], SDSC-SP2 [58], and PIK-IPLEX [59]. These datasets span multiple application domains, including cloud services, academic research, and scientific computing, and exhibit significant differences in job characteristics and resource requirements (see

Table 5. Comparison of experimental dataset features.

Dataset	Job Size (Tasks)	Max Nodes	Max Procs	Sys Arch	Load Type
Alibaba	80.5 k	300	300	Gen Clust	Online Svc Intense
ANL-Intrepid	68.9 k	40,960	163,840	Mesh	Sci Comp Intense
SDSC-SP2	73.4 k	128	128	Homo Clust	Batch Proc Intense
PIK-IPLEX	74.3 k	320	2560	Hetero Clust	Mixed Load

1. Abbreviations: Sys Arch: System Architecture; Load Type: Load Characteristics; Gen Clust: General Cluster; Online Svc Intense: Online Service Intensive; Sci Comp Intens; Scientific Computing Intensive; Homo Clust: Homogeneous Cluster; Batch Proc Intense: Batch Processing Intensive; Hetero Clust: Heterogeneous Cluster; 2. Computational scale: Data are derived from the configuration files of each dataset, formatted as “Number of Nodes/Number of Processors.”; 3. ANL-Intrepid uses a mesh architecture with 4 cores per node, suitable for large-scale scientific computing; 4. PIK-IPLEX supports heterogeneous resource scheduling with an average of 8 cores per node; 5. The Alibaba dataset includes configurations with both 300 and 200 nodes in different versions.

). The job arrival distributions and resource request distributions for each dataset are shown in Figure 5 and Figure 6. This diversity provides a foundation for evaluating the generalization capability of our method. Detailed experimental results are presented in Section 5.

4.4. GTrXL-SPPO Training

We implement GTrXL-SPPO in CQSim, a widely used HPC scheduling simulator. Real workload traces (Section 4.2) are fed sequentially, and the agent schedules jobs using a sliding window mechanism (see Figure 4). All code is implemented in PyTorch.

Table 6. Parameter comparison of GTrXL-SPPO under multiple workloads.

Dataset	Input	Output	Actor Output Layer Parameters	Transformer Parameters	Other Parameters	Total Trainable Parameters
Alibabav2017	[2,200]	801	461,376	67,723,776	68,341,480	136,526,632
ANL-Intrepid	[2,150]	40,961	23,593,536	67,723,776	68,381,640	159,698,952
SDSC-SP2	[2,50]	129	74,304	67,723,776	68,340,808	136,138,888
PIK-IPLEX	[2,75]	321	184,896	67,723,776	68,341,000	136,249,672

Dynamic output dimension adaptation to node scale (e.g., from 100 nodes to 101).

summarizes the parameter configurations under different workloads. The model receives state inputs constructed by sliding windows (e.g., [2,200] for the Alibaba workload) and outputs discrete probabilities corresponding to node allocation decisions (e.g., 801 candidate actions representing 800 node assignments plus one wait decision). The Transformer backbone (12 layers, 576 dimensions, 12 heads) accounts for nearly half of the total parameters, with the model scaling from 136M to 160M parameters across workloads.

To further illustrate training effectiveness, Figure 7 presents the progression of total reward across all datasets. GTrXL-SPPO consistently achieves higher final reward values and faster early-stage gains compared to baselines such as FIFO, SJF, and vanilla RL. Notably, the reward curve converges more smoothly under diverse workload dynamics, confirming the model’s robustness and training stability across heterogeneous environments.

Training objectives include utilization, wait time, and throughput. We use a linear learning rate decay (initially $5\times {10}^{-5}$), curriculum-based load escalation, and sample preservation to enhance generalization. As shown in Figure 7, GTrXL-SPPO consistently surpasses baselines under all workload scenarios, demonstrating faster convergence and achieving significantly higher total rewards, while static algorithms (FIFO and SJF) exhibit fixed reward levels (represented by horizontal lines). Evaluation covers both real and synthetic traces (see Section 5.1).

4.5. Evaluation Metrics

To comprehensively assess the effectiveness of the proposed scheduling strategy, we employ a set of user-level and system-level evaluation metrics, as summarized below:

• System Utilization: The proportion of time during which system resources are actively utilized.
• Throughput: The number of jobs completed per unit time.
• Average Wait Time: The mean time a job spends waiting from submission to execution.
• Job Completion Time: The total time required from job submission to completion.

Among these, System Utilization and Throughput are system-level metrics, while Average Wait Time and Job Completion Time are user-level metrics. Together, they provide a multi-dimensional evaluation framework, enabling a comprehensive analysis of scheduling strategies in terms of system efficiency, job responsiveness, and resource optimization. This facilitates the identification and selection of strategies best suited to specific HPC workload scenarios.

5. Case Study

Recent studies have adopted a sampling-real-synthetic data paradigm to construct training workloads for reinforcement learning-based HPC schedulers [3,60]. Building on this foundation, we conduct our experiments using CQGym, a widely adopted simulation platform for job scheduling research [3,61].

5.1. System Performance Evaluation

5.1.1. System Utilization and Waiting Time

As shown in Figure 8a, system utilization frequently approaches 100%, reflecting the model’s effectiveness in maximizing resource usage under varying load conditions. Additionally, Figure 8b demonstrates that our approach significantly reduces job scheduling waiting times during both training and testing phases, further highlighting the efficiency and responsiveness of the proposed method.

5.1.2. Overall Performance Evaluation

Figure 7 shows the total reward achieved by GTrXL-SPPO and baseline methods across different workloads, demonstrating the superior learning capability and convergence speed of our approach.

5.1.3. Comparative Analysis with PPO

As shown in Figure 9 and Figure 10, GTrXL-SPPO demonstrates superior performance across all evaluation metrics. The results on simulated workloads validate the algorithm’s effectiveness, while real-world workloads confirm its excellent generalization capability: GTrXL-SPPO achieves higher resource utilization, larger cumulative rewards, and lower task waiting times, demonstrating robust scheduling performance under diverse workload patterns.

5.2. Ablation Study and Component Analysis

To systematically evaluate the contributions of each component in our proposed GTrXL-SPPO framework, we conducted comprehensive ablation experiments on multiple datasets (ANL-Intrepid, SDSC-SP2, and Alibaba).

5.2.1. Experimental Design

We designed five model variants to evaluate different components:

• GTrXL-SPPO (Full): The complete model with all components.
• GTrXL-PPO (No SE): Removes the squeeze-and-excitation modules to assess their contribution to feature enhancement.
• SPPO (No GTrXL): Replaces the Gated Transformer-XL with a standard feedforward network to assess the impact of temporal modeling.
• TrXL-SPPO (No Gating): Removes the gating mechanism from the Transformer-XL to assess the importance of selective memory updates.
• GTrXL-SPPO (Fixed Reward): Uses fixed weights in the reward function instead of dynamically temperature-controlled softmax weights.

Each variant is trained under identical conditions with the same hyperparameters, with the only difference being the specific component under evaluation. We assess performance using multiple metrics: system utilization, average task waiting time, throughput (tasks/hour), and resource fragmentation.

5.2.2. Results and Analysis

As shown in

Table 7. Ablation study results across three datasets.

Model Variant	ANL-Intrepid	SDSC-SP2	Alibaba	Average Improvement
	Util. (%)	Util. (%)	Util. (%)	over Baseline (%)
GTrXL-SPPO (Full)	87.5	85.6	83.2	-
GTrXL-PPO (No SE)	83.1	81.2	77.8	−4.7
SPPO (No GTrXL)	73.1	70.2	65.8	−15.7
TrXL-SPPO (No Gating)	84.0	81.7	78.2	−4.2
GTrXL-SPPO (Fixed Reward)	84.3	82.0	78.7	−3.8

, each component contributes significantly to the overall performance of GTrXL-SPPO:

Impact of GTrXL: Removing the GTrXL component results in the most significant performance degradation, with system utilization decreasing by 14.4%, 15.4%, and 17.4% on the three datasets, respectively. This highlights the critical importance of temporal modeling in capturing long-term job dependencies.

Contribution of SE Module: The SE module improves system utilization by 4.4%, 4.4%, and 5.4% across datasets. By dynamically recalibrating feature importance, it enables better adaptation to changing resource conditions.

Role of Gating Mechanism: The gating mechanism improves system utilization by 3.5%, 3.9%, and 5.0%, respectively, enabling selective memory updates and noise filtering.

Dynamic Reward Weighting: Temperature-controlled dynamic weights improve system utilization by 3.2%, 3.6%, and 4.5% compared to fixed weights, allowing adaptive optimization priority adjustment.

5.3. Statistical Stability Analysis

To address the inherent randomness of reinforcement learning and ensure result reliability, we conducted GTrXL-SPPO experiments using 10 different random seeds (42–51) under identical conditions. Each run processed 3000 tasks using the same Alibaba Cluster workload with a 400-node configuration (Figure 11 and

Table 8. Statistical performance metrics of GTrXL-SPPO Across 10 independent runs.

Metric	Mean	Standard Deviation	95% Confidence Interval
System Utilization (%)	84.6	2.1	[82.9, 86.3]
Average Wait Time (s)	2850	145	[2741, 2959]
Throughput (tasks/hour)	135.2	4.8	[132.1, 138.3]
Cumulative Reward	0.648	0.028	[0.632, 0.664]

).

The statistical analysis demonstrates exceptional stability with low coefficients of variation: system utilization (CV = 2.5%), average waiting time (CV = 5.1%), and throughput (CV = 3.6%). Paired t-tests show statistically significant improvements (p < 0.001) with effect sizes exceeding Cohen’s d = 0.8, confirming substantial practical significance.

5.4. Runtime Performance Evaluation

To validate practical deployability in HPC environments, we conducted comprehensive runtime performance evaluation focusing on decision latency and computational overhead using PyTorch implementation on NVIDIA GPU hardware.

GTrXL-SPPO exhibits 9.0 ms mean decision latency with low variance ($\mathsf{\sigma }$ = 0.2 ms), indicating stable performance. Through optimization strategies including 16-bit precision, gradient checkpointing, and attention pruning, we achieved 24% latency reduction and 25% memory decrease while maintaining algorithmic benefits (

Table 9. Runtime performance comparison for HPC scheduling algorithms.

Algorithm	Decision Latency (ms)	Memory Usage (MB)	Throughput (Decisions/s)	Scale Limit (Nodes)
PPO	6.2 ± 0.8	578	161	8000
GTrXL-SPPO	9.0 ± 0.2	1049	111	5000
GTrXL-SPPO (Optimized)	6.8 ± 0.3	785	147	8000

).

5.5. Parameter Sensitivity Analysis

We conducted comprehensive sensitivity analysis on two critical hyperparameters: pressure regularization coefficient ${\alpha }_{\mathrm{reg}}$ and temperature parameter T in dynamic reward weighting.

5.5.1. Pressure Regularization Coefficient (${\alpha }_{\mathrm{reg}}$)

The optimal value ${\alpha }_{\mathrm{reg}}=0.5$ provides the best balance between queue management and scheduling efficiency (

Table 10. Effect of ${\alpha }_{\mathrm{reg}}$ on system performance (ANL-Intrepid dataset).

${\mathit{\alpha }}_{\mathbf{reg}}$ Value	System Utilization (%)	Avg. Wait Time (s)	Throughput (Jobs/h)	Queue Length (Avg.)
0.0	85.1	2532	139.7	42.3
0.1	86.2	2495	141.1	35.8
0.5	87.5	2458	142.5	32.1
1.0	86.6	2483	141.8	30.5
2.0	85.0	2545	139.7	29.7

), achieving peak system utilization (87.5%) and minimum waiting time (2458 s).

5.5.2. Temperature Parameter (T)

T = 1.0 provides optimal balance between objective differentiation and integration, achieving the best overall performance across all metrics (

Table 11. Effect of temperature parameter T on system performance (ANL-Intrepid dataset).

T Value	System Utilization (%)	Avg. Wait Time (s)	Throughput (Jobs/h)	Reward Balance Index
0.1	83.1	2705	135.4	0.28
0.5	86.2	2495	141.1	0.62
1.0	87.5	2458	142.5	0.81
2.0	85.8	2520	140.4	0.93
5.0	82.3	2728	134.0	0.97

).

5.6. Comparative Analysis of Attention Mechanisms

We evaluated our SECT module against two mainstream alternatives: CBAM [62] and ECA-Net [63].

SECT consistently outperforms alternatives with 2.3–3.6% higher utilization and superior computational efficiency (

Table 12. Performance comparison of different attention mechanisms.

Attention Mechanism	ANL Util. (%)	Alibaba Util. (%)	Avg. Improvement	Training Time (h)	Efficiency Rank
SECT (Ours)	87.5	83.2	-	5.2/7.3	1
CBAM	84.9	79.6	-3.2%	6.8/9.1	3
ECA-Net	85.2	80.3	-2.8%	5.5/7.8	2

), validating its specialized design for HPC scheduling tasks.

5.7. Comparison with Optimization-Based Methods

We evaluated GTrXL-SPPO against Mixed-Integer Programming (MIP) and Constraint Programming (CP) solvers on two scales (

Table 13. Performance comparison with optimization methods.

Method	Utilization (%)	Wait Time (s)	Makespan (h)	Optimality Gap (%)	Status
Small-scale (100 jobs, 128 nodes)
MIP (Gurobi)	89.8	2398	18.1	0.0	Optimal
CP (OR-Tools)	89.7	2464	18.5	0.8	Near-optimal
GTrXL-SPPO	87.5	2458	18.7	2.0	Complete
Large-scale (3000 jobs, 600 nodes)
MIP (Gurobi)	74.9	3596	28.2	19.5	Timeout
CP (OR-Tools)	83.7	3262	26.3	10.1	Timeout
GTrXL-SPPO	85.8	2508	19.2	4.0	Complete

):

GTrXL-SPPO maintains competitive performance at small scales (within 3% of optimal) and significantly outperforms optimization methods at large scales where traditional solvers encounter timeout issues.

5.8. Computational Complexity Analysis

We conducted comprehensive complexity analysis combining theoretical bounds with empirical measurements across multiple operational scales.

Despite quadratic complexity, GTrXL-SPPO demonstrates superior per-task efficiency at large scales (7.4 ms vs. PPO’s 15.6 ms), validating its practical scalability for production HPC environments (

Table 14. Multi-scale computational complexity comparison.

Method	Time Complexity	Small Scale (100)	Medium Scale (1000)	Large Scale (5000)
		Time per Task (ms)	Time per Task (ms)	Time per Task (ms)
FCFS	$O\left(n\right)$	24.5	3.6	1.5
PPO	$O(n\times d)$	26.7	17.7	15.6
GTrXL-SPPO	$O(n^2\times d)$	71.3	18.8	7.4

).

5.9. Practical Impact Analysis

To quantify the economic significance of performance improvements, we analyzed the practical impact across different HPC scales based on our measured 25-s wait time reduction and 5.4% utilization improvement (

Table 15. Annual impact of performance improvements on different HPC scales.

HPC Environment	Time Saved (Hours/Year)	Cost Savings (USD)	ROI Multiple
Small University (100 nodes)	1267	$72,857	0.73×
Medium Research (400 nodes)	5069	$485,714	1.21×
Large Enterprise (1000 nodes)	12,674	$1,457,141	1.46×
National Supercomputer (5000 nodes)	38,021	$9,612,883	1.92×
Total Impact	57,031	$11,628,595	1.33×

).

The analysis reveals significant scale amplification effects, where modest per-task improvements accumulate to substantial annual benefits: 57,031 h of time savings equivalent to 35.7 full-time research teams, with total economic impact of USD 11.6 million annually across different HPC scales.

6. Results and Discussion

In this work, we have successfully achieved our objective of developing a more efficient and robust scheduling framework for HPC environments. The proposed GTrXL-SPPO framework effectively addresses the challenges of high-dimensional decision spaces and complex temporal dependencies in dynamic scheduling environments, as demonstrated by our comprehensive experiments across multiple datasets.

The experimental results confirm that the two core components of GTrXL-SPPO—the GTrXL backbone and the SE module—provide complementary benefits across heterogeneous HPC workloads. These advantages are evident in both scientific (ANL) and industrial (Alibaba) scenarios, demonstrating the framework’s applicability to diverse real-world environments. The GTrXL component enhances long-horizon scheduling by effectively capturing temporal execution signals, while the SE module adaptively reweights features to improve responsiveness under resource variability. Together, these modules improve throughput, reduce turnaround time, and enhance resource utilization, as validated across datasets in Section 5.2. Overall, the results demonstrate the robustness and generalizability of our design.

7. Conclusions and Future Work

7.1. Results and Discussion

The proposed GTrXL-SPPO framework successfully addresses the challenges posed by high-dimensional decision spaces and complex temporal dependencies in dynamic high-performance computing scheduling environments. Our comprehensive experiments on multiple datasets demonstrate that the two core components—the GTrXL backbone network and the SECT module—offer certain advantages under heterogeneous workloads.

The GTrXL component enhances long-time-domain scheduling performance by capturing temporal execution signals, while the SECT module improves responsiveness under resource fluctuations. These two innovations collectively achieve quantifiable improvements in throughput, turnaround time, and resource utilization, as validated in Section 5.2.

The framework demonstrates robust performance across diverse workload characteristics, ranging from scientific simulations to cloud computing tasks.

7.2. Limitations and Future Work

Although GTrXL-SPPO achieves significant improvements, we acknowledge the following limitations, which provide directions for future research:

Current Limitations

Manual configuration requirements: Our reward mechanism relies on manually tuned weights and parameters, which poses portability challenges across different HPC platforms.

Limited system-specific constraints: The framework does not directly support production environment requirements, such as task deadlines, energy budgets, or thermal constraints.

Fairness considerations: During sustained high loads, non-preemptive designs may prioritize shorter tasks, causing long-running or resource-intensive tasks to starve.

7.3. Future Research Directions

Automatic configuration: Develop meta-learning methods to automatically adjust reward weights and quickly adapt to new environments through transfer learning.

Constraint-aware framework: Integrate multi-objective optimization to handle concurrent constraints and integrate real-time monitoring systems.

Fairness mechanisms: Refer to the research by Duque et al. [64] to explore fairness reward design and adaptive intervention strategies that consider aging factors to achieve resource reallocation.

These research directions aim to enhance the framework’s practical deployment capabilities while retaining its attention-based optimization advantages.

7.4. Broader Impact

GTrXL-SPPO represents a significant advancement in applying machine learning to infrastructure optimization. The framework’s ability to handle complex temporal dependencies goes beyond high-performance computing scheduling and can be scaled to other resource allocation domains. With the proposed improvements, we expect to provide more robust and fair scheduling solutions for modern computing infrastructure.