Cloud Resource Workload Forecasting Method Based on the MST-iTransformer Model

Abstract

With the widespread adoption of cloud computing technology, modern cloud platforms have become increasingly complex and dynamic, posing significant challenges for efficient resource management. Accurate forecasting of cloud resource loads has therefore become essential for improving service quality, optimizing resource utilization, and reducing operational costs. To address the intrinsic characteristics of cloud load time series, including nonlinear fluctuations, multi-scale temporal dependencies, and redundant high-dimensional features, this paper proposes the MST-iTransformer model, which integrates multi-scale temporal encoding, sparse attention, and adaptive feature selection mechanisms. Specifically, a multi-scale temporal encoding module is developed to capture and fuse temporal dependencies across multiple periodic scales. Furthermore, an adaptive feature selection module is introduced to dynamically assign importance weights to resource features, enhancing informative variables while suppressing redundant ones. Meanwhile, a sparse attention mechanism is incorporated to reduce computational overhead while maintaining forecasting accuracy. The proposed model is evaluated on the Alibaba Cluster Trace dataset. Experimental results demonstrate that MST-iTransformer achieves MSE, RMSE, and MAE values of 0.5559, 0.7456, and 0.4968, respectively. Compared with the original iTransformer, the proposed model achieves simultaneous reductions in prediction errors and inference latency, validating the effectiveness of the multi-scale temporal encoding, sparse attention mechanism, and adaptive feature selection modules in improving forecasting accuracy and computational efficiency. These improvements provide reliable prediction support for resource scheduling and elastic scaling in cloud data centers.

1. Introduction

With the rapid development and widespread adoption of cloud computing technologies, the scale and complexity of services deployed on cloud platforms have increased significantly. The highly dynamic and bursty characteristics of user behaviors, together with complex dependencies among cloud services, cause cloud resource workloads to exhibit nonlinear variations, non-stationary patterns, and multi-scale periodic dependencies. Accurate workload forecasting is a critical prerequisite for achieving on-demand resource allocation, elastic scaling, energy efficiency optimization, and guaranteed service quality [1,2].

Traditional cloud resource forecasting approaches mainly rely on statistical models and machine learning algorithms. Autoregressive Integrated Moving Average (ARIMA) models [3] and their variants capture temporal trends based on linear assumptions. However, these methods exhibit limited capability when dealing with nonlinear and non-stationary cloud workload sequences. Although machine learning approaches, such as Support Vector Regression (SVR) [4] and Random Forests, can model nonlinear relationships to some extent, they generally require carefully designed features and have limited capability in capturing complex long-term temporal dependencies. In recent years, Recurrent Neural Networks (RNNs), particularly Long Short-Term Memory (LSTM) networks, have been widely applied in cloud resource workload forecasting. Benefiting from gating mechanisms, LSTM can alleviate the vanishing gradient problem to a certain extent. However, its sequential computation paradigm limits parallel processing efficiency, and its capability to capture long-term temporal dependencies remains constrained when processing complex workload sequences [5].

With the continuous advancement of deep learning, attention mechanisms have introduced new opportunities for time-series modeling. The Transformer architecture captures global dependencies among sequence elements through the self-attention mechanism, achieving remarkable success in machine translation and natural language processing tasks [6]. Owing to its powerful representation capability and parallel computation advantages, Transformer-based architectures have also demonstrated promising performance in long-sequence time-series forecasting tasks.

As an innovative time-series forecasting model, iTransformer achieves outstanding performance in multivariate time-series forecasting tasks by reversing the dimensional processing strategy of the original Transformer and applying the attention mechanism along the variate dimension rather than the temporal dimension [7,8]. However, when applied to cloud resource forecasting scenarios, the original iTransformer still faces three main challenges:

(1) Insufficient multi-scale temporal modeling capability.

The original temporal representation strategy lacks an explicit multi-scale temporal modeling mechanism, making it difficult to fully capture the diverse periodic patterns of cloud workloads, such as hourly, daily, and weekly fluctuations.

(2) Lack of an explicit adaptive feature selection mechanism.

Although iTransformer can capture correlations among variables through variate-wise attention, it lacks an explicit mechanism for dynamically identifying task-relevant features before feature interaction. Consequently, redundant and irrelevant information may participate in feature representation, thereby affecting forecasting accuracy.

(3) High computational cost of the full attention mechanism.

Although the attention mechanism enables effective modeling of variable interactions, the quadratic computational complexity of variate-wise full attention increases computational overhead and limits the deployment efficiency of iTransformer-based models in multi-resource cloud workload forecasting scenarios.

To address these challenges, this paper proposes the MST-iTransformer (Multi-Scale Sparse Temporal iTransformer) model for cloud resource workload forecasting. The proposed model integrates multi-scale temporal encoding, sparse attention mechanism, and multi-scale adaptive feature selection to improve forecasting accuracy while reducing computational overhead.

Compared with existing Transformer-based time series forecasting methods, the proposed MST-iTransformer is specifically designed for the characteristics of cloud workload forecasting. Unlike conventional temporal encoding approaches that mainly provide position-related temporal information, the proposed multi-scale temporal encoding module explicitly models multiple periodic patterns commonly observed in cloud workloads, such as hourly, daily, and weekly fluctuations. Different from feature weighting strategies that may not explicitly adapt to workload variations, the multi-scale adaptive feature selection module learns variable importance from the observed workload patterns, allowing informative resource variables to be emphasized while relatively redundant features are suppressed. Furthermore, instead of applying dense attention over all variables as in the original iTransformer, the proposed sparse attention mechanism introduces adaptive variable selection before attention computation, reducing redundant variable interactions while preserving important resource dependencies.

The main contributions of this study are summarized as follows:

(1) A multi-scale temporal encoding module is designed to integrate periodic information from different temporal granularities into variate representations. By capturing temporal dependencies across multiple scales, the proposed module enhances the model’s ability to characterize complex periodic fluctuations in cloud workloads.

(2) A multi-scale adaptive feature selection module is developed to introduce a dynamic feature importance weighting mechanism. This module adaptively emphasizes informative variables and suppresses redundant features, thereby improving feature representation quality.

(3) A sparse attention mechanism is proposed to perform selective variable interaction modeling along the variate dimension. By selecting the Top-k informative variables for attention computation, the proposed mechanism reduces computational overhead while maintaining forecasting performance.

2. Related Work

2.1. Cloud Resource Forecasting Methods Based on Traditional Statistical Models

Early research on cloud resource forecasting primarily relied on traditional statistical methods, such as Autoregressive (AR) models, Moving Average (MA) models, Autoregressive Moving Average (ARMA) models, and Autoregressive Integrated Moving Average (ARIMA) models. These methods forecast future workloads by establishing linear dependencies in time series, offering advantages such as simple model structures and strong interpretability.

Calheiros et al. [9] proposed a dynamic resource forecasting method based on ARIMA, achieving proactive resource allocation by predicting future resource demands, thereby improving resource utilization and service quality. Aldossar et al. established an energy-aware cost forecasting framework that integrates ARIMA time-series forecasting models with linear regression algorithms to, respectively, forecast cloud platform workloads and device power consumption, accurately calculating the overall operational costs of virtual machines throughout their service lifecycle. Additionally, Ros et al. [10] designed a new cloud resource workload forecasting algorithm based on exponential smoothing theory and developed corresponding application tools, effectively enhancing the scheduling efficiency and elastic scaling capabilities of computational resource workloads in cloud systems.

However, traditional statistical models generally assume that time series satisfy stationarity conditions, whereas actual cloud resource workloads often exhibit distinct non-stationarity and burstiness. When faced with complex long-sequence forecasting tasks, these methods struggle to capture long-term dependencies and multi-scale dynamic changes, thereby severely limiting their prediction accuracy.

2.2. Forecasting Methods Based on Machine Learning and Deep Learning

To overcome the performance bottlenecks of traditional statistical methods in nonlinear workload modeling, researchers introduced machine learning methods—such as Support Vector Machines, Random Forests, ensemble learning, and hybrid optimization models—into the cloud resource forecasting domain.

Barati et al. proposed a twin support vector regression model integrating genetic algorithms and particle swarm optimization, achieving adaptive global searches for model parameters through a hybrid optimization strategy, which effectively improved workload prediction accuracy [11]. Kim et al. proposed the CloudInsight ensemble forecasting model, which dynamically adjusts the weight coefficients of sub-predictors in real time, significantly enhancing the model’s adaptability to non-stationary workload sequences [12]. Qiu et al. further proposed an optimized Random Forest model based on oblique hyperplane partitioning, strengthening the model’s feature learning capabilities for complex data distributions [13].

While machine learning methods have improved nonlinear modeling capabilities to some extent, they still rely heavily on manual feature engineering and struggle to automatically extract deep temporal dependency features. Moreover, in long-sequence forecasting tasks, traditional machine learning models still lack the capacity to learn long-term dependencies, multi-scale change patterns, and complex periodic modes.

With the development of deep learning, methods represented by Recurrent Neural Networks have gradually become the mainstream research direction in cloud resource forecasting. Duggan et al. proposed an RNN-based CPU resource forecasting model for short-term and multi-step prediction tasks in cloud environments. Karim et al. proposed the BHyPreC model, combining Bi-LSTM, LSTM, and GRU units to enhance multi-step prediction capabilities through the joint learning of historical and future information [14]. Concurrently, Convolutional Neural Networks (CNNs) and Temporal Convolutional Networks (TCNs) have gained attention due to their excellent local feature extraction capabilities [15].

Although models like RNN, LSTM, and TCN have somewhat improved long-term dependency learning, the recurrent structure inherently suffers from vanishing gradients, low training efficiency, and insufficient parallel computing capabilities. Furthermore, single-scale feature extraction methods struggle to adequately describe the complex variation patterns of cloud workloads across different time scales.

2.3. Long-Sequence Forecasting Methods Based on Transformer Architecture

Due to their self-attention mechanism’s ability to effectively capture global dependencies, Transformer architectures have been widely applied to long time-series forecasting tasks in recent years. Compared to RNN-type models, Transformers possess stronger parallel computing capabilities and long-term dependency modeling capabilities.

The Informer model reduces the time complexity of traditional self-attention through a ProbSparse Self-Attention mechanism and utilizes a self-attention distillation mechanism to improve long-sequence forecasting efficiency [16]. Autoformer [17] and FEDformer [18] further enhance long-sequence modeling capabilities through trend-cyclical decomposition and frequency domain learning.

For cloud resource forecasting scenarios, researchers have begun targeted improvements to the Transformer structure. MSCNet proposes a multi-scale convolutional network architecture that extracts different periodic patterns through multi-scale patch partitioning and multi-scale convolutional modules to achieve long-term cloud workload forecasting [19]. Building upon Informer, SDVformer introduces Savitzky–Golay filtering, a Discrete Variation Self-Attention (DVSA) mechanism, and a Type-aware Mixture-of-Experts (T-MOE) framework to improve feature extraction capabilities and prediction accuracy in highly fluctuating cloud resource sequences [20].

The core innovation of the iTransformer lies in its fundamental reconstruction of the Transformer’s operational dimensions [21]. It embeds the entire temporal sequence of a single variable across the historical window as a variate token, allowing the attention mechanism to focus exclusively on cross-variate correlation modeling. This reversal strategy offers significant advantages in multivariate forecasting tasks.

3. Model Design and Construction

3.1. Definition and Modeling of the Cloud Workload Forecasting Problem

Cloud workload forecasting is essentially a typical multivariate non-stationary time-series forecasting problem, meaning it infers future time-series features based on historical multivariate time-series data. Cloud data center workload forecasting involves multi-dimensional variables: server CPU utilization, memory usage, disk I/O, network throughput, etc. These variables experience non-stationary fluctuations over time, accompanied by multi-level periodic patterns (e.g., hourly, daily, weekly), as well as short-term random volatility [22]. These characteristics make cloud workload forecasting a challenging and continuously evolving research problem [23].

Previous time-series models primarily modeled along the temporal dimension, making it difficult to adapt to characteristics in cloud environments such as multivariate coupling, feature redundancy, and multi-scale periodic superposition. By shifting attention to the variate dimension, iTransformer mines relationships among different resource indicators, making it more adaptable to cloud workload forecasting tasks. However, the original model still has the following flaws: single-scale time encoding cannot fit multi-level periods; undifferentiated feature learning leads to modeling interference from redundant features; and full-variable attention computation incurs high overhead. Therefore, this paper introduces modular improvements addressing these pain points sequentially, jointly forming an end-to-end forecasting model.

Assume the historical observation duration is $T$, and the cloud workload monitoring feature dimension is $N$. The input raw time-series matrix can be expressed as:

$$X=\{x_1,x_2,\dots ,x_T\},x_t\in {\mathbb{R}}^N$$

(1)

where $x_t$ represents the vector set of all cloud resource monitoring indicators at time $\mathrm{t}$. The core objective of multi-step cloud resource forecasting is to learn a nonlinear mapping function $\mathcal{F}(\cdot )$ based on continuous historical observations $X_{1:T}$ to achieve precise deduction of resource workloads for the future $\tau$ steps. The mathematical expression is:

$$\{{\widehat{x}}_{T+1},{\widehat{x}}_{T+2},\dots ,{\widehat{x}}_{T+\tau }\}=\mathcal{F}(X_{1:T};\Theta )$$

(2)

where Θ represents all learnable parameters of the model, and $\tau$ is the prediction step size.

3.2. Overall Model Architecture

To address the issues of insufficient multi-scale periodic feature mining, high-dimensional feature redundancy interference, and excessive attention computational overhead in traditional cloud environment time-series forecasting, the MST-iTransformer model proposed in this paper constructs a five-stage end-to-end deep time-series forecasting architecture: “preprocessing-encoding-purification-interaction-prediction”. The overall model is composed of four core components: a data preprocessing module, a multi-scale time encoding module, an adaptive feature selection module, and a sparse attention module, as well as a fully connected prediction layer, as shown in Figure 1.

Figure 1. Overall Model Architecture Diagram.

The overall inference logic and data flow of the model are as follows: First, raw cloud workload time-series data enters the data preprocessing module, where dimensional differences are eliminated through standardization, and it is reconstructed into a high-dimensional time-series matrix via a sliding window strategy. The preprocessed time-series matrix then enters the multi-scale time encoding module, where the model captures short-, medium-, and long-term periodic features in parallel, fusing them with original features along the time-step dimension through an adaptive weighting strategy to initialize foundational representations.

Second, to prevent high-dimensional noise features from polluting subsequent correlation computations, the fused augmented matrix is fed into the multi-scale adaptive feature selection module. Combining data-driven learnable gating weights and domain prior knowledge, this module applies soft weighting to resource variables across dimensions, automatically amplifying core variate representations while suppressing redundant noise, thus achieving preliminary feature purification across the variate dimension.

Subsequently, the purified high-quality feature matrix is sent into the sparse attention module. This module retains the interaction logic of the variate dimension and performs a Top-k sparse truncation by quantifying the global importance of the variables, accomplishing deep modeling of global dependencies under a low computational complexity of O(k²).

Finally, the time-series representations deeply mined by the multi-layer encoders undergo dimensionality reduction mapping via a feedforward neural network and a fully connected layer to output the cloud workload forecasting results for future time steps.

3.3. Data Preprocessing Module

Data detected in cloud environments suffers from inconsistent dimensions, anomalous values, missing values, and discrete numerical distributions. Feeding raw datasets directly into the model easily leads to gradient divergence and fitting bias. This paper preprocesses the raw dataset using standardization and sequential segmented sampling.

First, Z-Score standardization is applied to achieve feature normalization, eliminating dimensional differences among various resource indicators:

$$x_{t,i}^{\prime }={\displaystyle \frac{x_{t,i}-{\mu }_i}{{\sigma }_i}}.$$

(3)

where $x_{t,i}$ is the raw observation value of the $i$-th feature at time $t$; ${\mu }_i$ and ${\sigma }_i$ are the mean and standard deviation of the $i$-th dimension feature within the training set, respectively; and $x_{t,i}^{\prime }$ is the standardized feature data.

Second, a sliding window segmentation method is used to construct model input samples. A fixed window length $T$ is set as the historical input length, and time-series segments are intercepted sequentially with a fixed step size to complete the partitioning of the training, validation, and testing sets. This ensures the continuity and temporal causality of the time-series data, preventing future information leakage. The preprocessed standardized time-series matrix serves as the foundational input for subsequent time encoding and feature fusion.

3.4. Multi-Scale Time Encoding Module

Cloud computing workloads possess a distinct multi-scale periodic structure in the temporal dimension: hourly fluctuations reflect the tidal effects of intraday business activity, daily patterns demonstrate systemic differences between workdays and non-workdays, and weekly regularities correspond to medium- and long-term business rhythms such as holiday peaks. A single fixed-period time encoding cannot simultaneously capture short- and long-term temporal dependencies; therefore, this paper designs a parallel multi-scale time encoding module to more accurately predict cloud workload sequences, as shown in Figure 2.

Figure 2. Multi-Scale Time Encoding Mechanism Diagram.

First, a multi-level time scale set $S=\{s_1,s_2,\dots ,s_m\}$ is defined, corresponding, respectively, to short-, medium-, and long-term workload variation periods. For any given time step $t$, its relative periodic position at each scale is calculated:

$${pos}_{s_i}(t)={\displaystyle \frac{tmod{s}_i}{s_i}}$$

(4)

Following the classic sine-cosine encoding method, independent positional encoding vectors are generated for each time scale:

$$\{\begin{array}{l}P{E}_{s_i}(t,2k)=sin\left({\displaystyle \frac{{ pos }_{s_i}(t)}{{10000}^{2k/d_{model }}}}\right)\\ P{E}_{s_i}(t,2k+1)=cos\left({\displaystyle \frac{{ pos }_{s_i}(t)}{{10000}^{2k/d_{model }}}}\right)\end{array}$$

(5)

where $d_{model}$ is the model’s hidden layer dimension, and $k$ is the vector dimension index.

To prevent information redundancy caused by simple concatenation of multi-scale features, a scale-adaptive weighted fusion strategy is introduced. A shallow network learns the contribution weight ${\alpha }_i$ of different time scales to cloud workload forecasting, satisfying ${\sum }_{i=1}^m{\alpha }_i=1$, ultimately fusing them into a unified multi-scale temporal feature:

$$TE(t)={\sum }_{i=1}^m{\alpha }_i·P{E}_{s_i}(t)$$

(6)

The fused time encoding vectors and preprocessed workload features are concatenated time-step by time-step to obtain an augmented feature matrix containing multi-scale temporal information, providing rich periodic contextual representations for the subsequent encoders.

3.5. Multi-Scale Adaptive Feature Selection Module

In multi-indicator cloud workload forecasting scenarios, there are inherent differences in the information content that various resource dimensions contribute to a specific forecasting target. Taking CPU utilization forecasting as an example, the correlation between memory usage and task queue length is generally much higher than network packet loss rate. If the encoded representations of all variables are treated equally, high-noise variables will dilute the gradient signals of effective features, negatively impacting the model’s convergence quality and generalization capability. This paper constructs a multi-scale adaptive feature selection module to establish a dynamic correlation mechanism between features and prediction scales, as shown in Figure 3.

Figure 3. Adaptive Feature Selection Mechanism Diagram.

To achieve this, a multivariate adaptive feature selection module is designed to learn an importance weight between 0 and 1 for each resource variable. This mechanism automatically emphasizes informative resource variables while suppressing less relevant or redundant representations. Simultaneously, it provides an optional mechanism for integrating business prior knowledge. In this study, considering the objective evaluation of the model’s inherent feature selection capability, the prior weight was set to zero, and only the data-driven adaptive weighting strategy is employed.

3.5.1. Learnable Variate Gating Weights

This module first assigns an independent learnable parameter vector $\theta \in {\mathbb{R}}^N$ to the $N$ cloud resource variables. The parameter dimensions strictly align with the number of variables, achieving precise one-variable-to-one-weight regulation.

To constrain the parameters within the effective gating interval of $[0,1]$, a Sigmoid activation function is introduced for normalized mapping:

$$w_{learned}=\sigma \left(\theta \right)\in {\mathbb{R}}^N$$

(7)

where $\sigma (·)$ is the Sigmoid activation function:

$$\sigma (x)={\displaystyle \frac{1}{1+e^{-x}}}$$

(8)

The value of each element in $w_{learned}$ ranges between 0 and 1. Values closer to 1 indicate higher informational value for the corresponding variable, thus preserving and strengthening its representation; values closer to 0 indicate that the corresponding variable is redundant noise, thereby suppressing its feature output.

3.5.2. Optional External Prior Weight Fusion

In actual cloud business scenarios, importance scores for each variable can be pre-assigned based on business experience and statistical priors to construct a prior weight vector $w_{prior}\in {\mathbb{R}}^N$.

To balance the model’s data self-learning with domain prior knowledge, a weighted combination method is used to fuse the dual weights:

$$w=(1-{\alpha }_{prior})·w_{learned}+{\alpha }_{prior}·w_{prior}$$

(9)

The physical meaning of the hyperparameter ${\alpha }_{prior}\in [0,1]$ is as follows:

(1) ${\alpha }_{prior}=0$: Completely discards external priors, degrading the module into purely data-driven adaptive gating where variable importance is learned solely from training data.

(2) ${\alpha }_{prior}=1$: Completely relies on business prior weights, and the model no longer autonomously learns variable contributions.

(3) $0<{\alpha }_{prior}<1$: Balances the fusion of data distribution features and cloud business domain knowledge.

3.6. Sparse Attention Mechanism Module

The original iTransformer applies the attention mechanism to the variate dimension, enhancing multivariate time-series modeling capabilities by modeling correlations among features. However, full-variable attention computation has a complexity of $O\left(N^2\right)$, which incurs extremely high computational costs under the high-dimensional monitoring features found in cloud scenarios. Considering the differential contribution characteristics of cloud resource features, this paper constructs a variate-aware sparse attention mechanism, as shown in Figure 4.

Figure 4. Sparse Attention Mechanism Diagram.

First, based on the global temporal mean, an importance score is quantified for each single-class resource variable to measure that indicator’s sensitivity to workload changes:

$$scor{e}_i=\sigma \left(MLP\left({\displaystyle \frac{1}{T}}{\sum }_{t=1}^T{x}_{t,i}\right)\right)$$

(10)

where $\sigma$ is the Sigmoid activation function that constrains the importance score to the $[0,1]$ interval. Based on the sorted scores, variables with higher importance are retained.

To avoid manually specifying a fixed k value and achieve a better balance between information preservation and computational efficiency, an adaptive Top-k strategy is further introduced. Specifically, k is determined as: $k=max(\lceil \alpha N\rceil ,\lceil \beta ln(N+1)\rceil )$.

Where N represents the total number of input variables, α denotes the minimum retention ratio, and β controls the logarithmic growth rate. In this study, α = 0.5 and β = 2 are adopted. The minimum retention constraint ensures that sufficient variables are preserved to maintain cross-variable interactions, while the logarithmic term allows the selected variable number to increase gradually with feature dimension, thereby reducing redundant attention computation.

Within the sparse variable set, the variate dimension attention calculation logic of the iTransformer is maintained to generate the Query matrix $Q$, Key matrix $K$, and Value matrix $V$, completing localized attention interactions:

$$Attention(Q,K,V)=softmax\left({\displaystyle \frac{Q{K}^T}{\sqrt{d_k}}}\right)V$$

(11)

After sparsification, the attention complexity drops from $O(N^2)$ to $O(k^2)$, drastically reducing computational overhead in multi-variable cloud monitoring scenarios. Each layer of the encoder is paired with residual connections and layer normalization operations to alleviate gradient degradation issues in deep networks, stabilizing the model training process and accomplishing the deep mining of inter-variable dependencies.

4. Experimental Results and Discussion

4.1. Experimental Dataset and Parameter Settings

In order to verify the effectiveness of the MST-iTransformer model, experiments were conducted using Alibaba Cluster Trace v2018 dataset. We specifically used the machine_usage.csv table, which records per-machine resource utilization metrics over time. The dataset contains multi-dimensional resource usage indicators of real production clusters, covering time series monitoring data of CPU utilization, memory occupancy, disk and network resources, with typical multi-cycle superposition and sudden load characteristics.

The dataset is divided into training set, validation set and test set according to the ratio of 8:1:1, where cpu_util_percent, mem_util_percent, net_in, net_out and disk_usage_percent are selected as the main model input variates andall features are standardized by Z-score. The historical window length is L = 96, and the prediction step size is S ∈ {12,24,48}. The mean square error (MSE), root mean square error (RMSE), mean absolute error (MAE) and inference time are used as evaluation indices in the experiment. The proposed MST-iTransformer is evaluated under a multivariate-to-multivariate (M-M) forecasting setting, where multiple historical cloud resource variables are used to predict future states of all resource variables simultaneously.

To ensure fair comparison and the reproducibility of experimental results, all deep learning models were implemented using a unified PyTorch 2.1.0 framework and evaluated under the same data preprocessing procedures, training strategies, and evaluation metrics. For Transformer-based models, a consistent backbone configuration was adopted, where the number of encoder layers was set to 3, the number of decoder layers was set to 1, the hidden dimension was configured as 512, the number of attention heads was set to 16, and the dropout rate was set to 0.15.

For the recurrent neural network baseline, LSTM was implemented using a two-layer stacked architecture, with 256 hidden units and a dropout rate of 0.1. Regarding the optimization strategy, all models were trained using the Adam optimizer with a learning rate of 3 × 10⁻⁴, a batch size of 32, and a maximum of 80 training epochs. An early stopping strategy was employed to prevent overfitting. Specifically, the training process was terminated when the validation loss did not decrease for 10 consecutive epochs (patience = 10), and the model parameters corresponding to the best validation performance were retained.

To ensure the statistical robustness and reproducibility of the experimental results, experiment was independently repeated five times, and the final reported performance metrics were calculated as the mean and standard deviation over the five runs. For each repeated experiment, random seeds ranging from 1 to 5 were assigned, and the seeds were globally fixed during data loading, model parameter initialization, and all stochastic operations involved in the training process. This setting ensures that the experimental results can be reproduced under the same software and hardware environments.

In the Multi-Scale Adaptive Feature Selection Module, the prior knowledge coefficient was set to zero in all experiments. Therefore, the reported results mainly reflect the data-driven feature selection capability of the proposed method rather than external domain knowledge injection.

4.2. Comparison of Experimental Results

4.2.1. Comparison of Prediction Accuracy

Table 1 shows the prediction accuracy comparison results of each model on the Alibaba Cluster dataset (12-step prediction). The experimental results show that MST-iTransformer is superior to the comparison model in all evaluation indices, which verifies the effectiveness of the three improved modules proposed in this paper.

Table 1. Prediction accuracy comparison results.

Model	MSE	RMSE	MAE
LSTM	0.7156	0.8460	0.6203
Transformer	0.6783	0.8236	0.5823
Autoformer	0.6620	0.8136	0.6082
Informer	0.6799	0.8246	0.6469
Triformer	0.6713	0.8193	0.5303
iTransformer	0.6356	0.7973	0.5886
MST-iTransformer	0.5559 ± 0.017	0.7456 ± 0.012	0.4968 ± 0.013

To further evaluate the reliability and statistical robustness of the experimental results, each model was independently trained and tested five times with different random seeds. The final results are reported as the mean value and standard deviation of the five runs.

As shown in Table 1, MST-iTransformer achieves MSE, RMSE, and MAE values of 0.5559 ± 0.017, 0.7456 ± 0.012, and 0.4968 ± 0.013, respectively. Although slight variations exist due to random initialization and stochastic optimization, the standard deviations remain within an acceptable range, indicating stable forecasting performance.

The MST-iTransformer proposed in this paper achieves the lowest mean square error in all evaluation models, which is 12.5% higher than the suboptimal iTransformer. The traditional LSTM baseline model has the highest MSE, reflecting its lack of ability to model long-term temporal dependence. In Transformer-based variants, Autoformer and Triformer are improved compared to the original Transformer, but there is still a significant gap between them and MST-iTransformer. The above results show that the proposed multi-scale temporal fusion mechanism can effectively reduce the square magnitude of the prediction error, and verify its accuracy advantage on the benchmark dataset.

RMSE has the same physical dimension as the target variable and is more sensitive to large prediction bias. MST-iTransformer achieved the lowest RMSE, which was 6.5% lower than iTransformer and 11.9% higher than LSTM baseline. The relatively narrow gap between the baseline variants of Transformer shows that the marginal architecture improvement within the standard Transformer framework has limited RMSE improvement. In contrast, the significantly lower RMSE of MST-iTransformer confirms that the proposed model not only reduces the average prediction error, but also effectively suppresses the large prediction deviation. This feature has important practical significance in practical application scenarios where the prediction error cost is high.

The index treats all deviations linearly, so it is less affected by extreme outliers than MSE and RMSE. MST-iTransformer achieved the lowest MAE, which was 6.3% higher than the second-ranked Triformer and 19.9% higher than the LSTM baseline. A noteworthy phenomenon is that the MAE ranking of each baseline model is different from the MSE and RMSE rankings: although the MSE of Triformer is in the middle, it has achieved the second lowest MAE, indicating that its architecture is particularly effective in suppressing systemic bias, but there are occasional large deviations. Informer had the highest MAE among all baselines, indicating that it tended to produce predictions with large mean deviation, as shown in Figure 5. The consistency advantage of MST-iTransformer on three indicators shows that the improvement of its prediction accuracy is not the result of outlier suppression, but the real improvement of point-level prediction accuracy.

Figure 5. Bar and line chart comparison of prediction accuracy.

4.2.2. Comparison of Multiple Prediction Steps

In order to verify the effectiveness of the proposed MST-iTransformer model in cloud resource load forecasting tasks, this paper selects LSTM, Transformer, Informer and iTransformer as comparison models, and conducts experiments under the conditions of prediction steps of 12, 24 and 48, respectively. The mean square error (MSE) and mean absolute error (MAE) are used as the evaluation indices. The experimental results are shown in Table 2 and Table 3.

Table 2. MSE comparison of various models for multi-step prediction.

Model	12-Step	24-Step	48-Step
LSTM	0.7156	0.7181	0.7245
Transformer	0.6783	0.7276	0.7713
Informer	0.6799	0.7654	0.7389
iTransformer	0.6356	0.6712	0.6894
MST-iTransformer	0.5559	0.5891	0.6003

Table 3. MAE comparison of various models for multi-step prediction.

Model	12-Step	24-Step	48-Step
LSTM	0.6203	0.6242	0.6274
Transformer	0.5823	0.5726	0.5977
Informer	0.6469	0.5771	0.5747
iTransformer	0.5886	0.5981	0.6072
MST-iTransformer	0.4968	0.4993	0.5177

• Analysis of MSE results

It can be observed from Table 2 that with the increase in prediction step size, the prediction error of most models shows an upward trend, indicating that the long-term prediction task has higher complexity and uncertainty.

Among them, the MSE of LSTM under three prediction steps is 0.7156, 0.7181 and 0.7245, respectively; the error of Transformer in the 48-step prediction increases to 0.7713; informer reached 0.7654 in 24-step prediction, with a local peak. The performance of iTransformer is better than the above baseline model, but there is still obvious error accumulation.

In contrast, MST-iTransformer achieved the lowest MSE at all prediction steps. Compared with the original iTransformer, MST-iTransformer decreased by 12.54%, 12.23%, and 12.92% in 12, 24, and 48 steps, respectively. It shows that the introduced multi-scale adaptive feature selection module and multi-scale time feature can effectively capture the long-term trend and short-term fluctuation information in the cloud resource load, thus significantly improving the prediction accuracy.

2.

Analysis of MAE results

It can be seen from Table 3 that MST-iTransformer also achieves the best results on the MAE index. Compared with iTransformer, the 12-step reduction is 15.59%; 24 steps decreased by 16.52%; 48 steps decreased by 14.74%. The experimental results show that MST-iTransformer not only reduces the overall prediction error, but also effectively alleviates the problem of error propagation and accumulation in the long sequence prediction process.

3.

Multi-step predictive stability analysis

It can be clearly seen from the figure that the overall trend of the LSTM model is relatively flat, but the error is always maintained at a high level, indicating that it is difficult to fully exploit the long-term dependence in complex time series data. With the increase in the prediction step size, the error growth of the Transformer model is more obvious, especially in the 48-step prediction scenario, where the performance decreases greatly, indicating that the standard self-attention mechanism has the problem of information attenuation in the long-term span prediction task. Benefiting from the sparse attention mechanism, the performance of Informer is better than that of Transformer, but there are still large fluctuations in the long prediction range. The iTransformer enhances the feature expression ability through variable dimension modeling, and is superior to the traditional Transformer structure at all steps. The three prediction curves of MST-iTransformer maintain the lowest error level and the growth rate is small.

Taking MSE as an example, its ΔMSE is 0.0444, while the corresponding growth rate of Transformer is 0.0930, which is more than twice that of MST-iTransformer, as shown in Figure 6. It shows that the model has stronger long-term prediction robustness.

Figure 6. MSE and MAE comparison for multi-step prediction. (a) MSE comparison for multi-step prediction (Bar chart). (b) MSE comparison for multi-step prediction (Line chart). (c) MAE comparison for multi-step prediction (Bar chart). (d) MAE comparison for multi-step prediction (Line chart).

4.2.3. Ablation Experimental Analysis

In order to verify the independent contribution of each improved module, this paper designs an ablation experiment to gradually remove the three core modules of MST-iTransformer and observe the performance changes in the model. The results of ablation experiments are shown in Table 4.

Table 4. Analysis of ablation experimental results.

Model	Feature Selection	Time Encoding	Sparse Attention	RMSE	MAE	Inference Time (ms)
MST-iTransformer	✓	✓	✓	0.7456	0.4968	941
w/o Sparse Attention	✓	✓	✗	0.7503	0.5025	1365
w/o Feature Selection	✗	✓	✓	0.7814	0.5531	932
w/o Time Encoding	✓	✗	✓	0.7646	0.5394	925
Base (iTransformer)	✗	✗	✗	0.7973	0.5886	1336

The ablation experiment results demonstrate that the three proposed modules contribute positively to both the prediction performance and computational efficiency of MST-iTransformer.

It should be noted that the reported inference time reflects the end-to-end execution of the entire model, including data embedding, multi-scale temporal encoding, adaptive feature selection, Top-k variable selection, sparse attention computation, feed-forward transformation, and prediction generation.

• Overall performance analysis

The MST-iTransformer model outperforms all ablation variants in terms of both forecasting accuracy and inference efficiency, validating the comprehensive effectiveness of the three proposed modules.

Compared with the original iTransformer, the complete model reduces the RMSE from 0.7973 to 0.7456, achieving a reduction of approximately 6.48%. Meanwhile, the MAE decreases from 0.5886 to 0.4968, representing a reduction of approximately 15.60%. In addition, the inference time is reduced from 1336 ms to 941 ms, corresponding to a decrease of approximately 29.57%.

These results demonstrate that MST-iTransformer can simultaneously improve forecasting accuracy and reduce computational overhead, achieving an effective balance between prediction performance and computational efficiency.

2.

Quantitative analysis of the effectiveness of each core module

(1) Multi-scale adaptive feature selection module

The multi-scale adaptive feature selection module is an important component that significantly affects the forecasting performance of the model.

After removing this module, the RMSE increases from 0.7456 to 0.7814, and the MAE increases from 0.4968 to 0.5531, representing the most significant performance degradation among the three proposed modules.

This result indicates that different cloud resource variables contribute differently to workload variations. By dynamically learning the importance weights of different variables, the proposed module can emphasize informative resource indicators while suppressing the interference caused by less relevant features, thereby improving the feature representation capability of the model.

Meanwhile, after removing this module, the inference time changes only slightly from 941 ms to 932 ms. This indicates that the primary function of this module is to optimize feature representation rather than directly reduce computational complexity, and it introduces only limited computational overhead.

(2) Multi-scale temporal encoding module

The multi-scale temporal encoding module is mainly designed to enhance the model’s ability to capture multi-period workload variation patterns.

After removing this module, the RMSE increases from 0.7456 to 0.7646, and the MAE increases from 0.4968 to 0.5394, corresponding to increases of approximately 2.55% and 8.58%, respectively.

This result verifies that cloud resource workloads contain significant multi-scale temporal patterns. By integrating periodic information at different temporal granularity levels, including hourly, daily, and weekly patterns, the proposed module effectively improves the capability of the model to characterize complex temporal dependencies.

Furthermore, removing this module reduces the inference time from 941 ms to 925 ms, which represents only a minor change. This indicates that the multi-scale temporal encoding module introduces effective temporal information with limited additional computational cost.

(3) Sparse attention mechanism

The sparse attention mechanism mainly contributes to improving the computational efficiency of the proposed model.

After removing this module, the RMSE increases slightly from 0.7456 to 0.7503, and the MAE increases from 0.4968 to 0.5025, indicating only a minor degradation in forecasting accuracy.

However, the inference time increases significantly from 941 ms to 1365 ms after removing the sparse attention mechanism. This demonstrates that conventional full attention computation among all variables introduces considerable redundant interactions.

By selecting the Top-k informative variables based on importance scores and performing attention computation only among these selected variables, the sparse attention mechanism effectively reduces unnecessary computational operations. Meanwhile, it maintains comparable prediction performance while significantly improving inference efficiency.

3.

Rationality verification of model design

Overall, the ablation results demonstrate that the three core components of MST-iTransformer provide complementary contributions.

Specifically, the multi-scale temporal encoding module enhances the capability of capturing complex periodic patterns; the multi-scale adaptive feature selection module improves the representation quality of important resource variables; and the sparse attention mechanism reduces redundant variable interactions and computational overhead.

The collaborative optimization of these three modules enables MST-iTransformer to achieve a better balance between forecasting accuracy and computational efficiency.

The experimental results verify the rationality and effectiveness of the proposed MST-iTransformer architecture, demonstrating its capability to meet the dual requirements of high accuracy and high efficiency for cloud resource workload forecasting tasks.

5. Conclusions

Aiming to address the limitations of the original iTransformer model in cloud resource prediction scenarios, this paper proposes an improved model, MST-iTransformer, which integrates multi-scale temporal encoding, a sparse attention mechanism, and multi-scale adaptive feature selection. The multi-scale temporal encoding module is designed to capture the multi-periodic characteristics of cloud workloads, while the multi-scale adaptive feature selection module dynamically identifies key features. Meanwhile, the sparse attention mechanism is employed to reduce computational complexity and improve computational efficiency.

Experimental results demonstrate that the proposed model achieves a favorable balance between prediction accuracy and computational efficiency compared with the original iTransformer and other mainstream time-series forecasting models. Specifically, MST-iTransformer achieves MSE, RMSE, and MAE values of 0.5559, 0.7456, and 0.4968, respectively, on the Alibaba Cluster dataset. Furthermore, MST-iTransformer achieves the lowest MSE values among all compared models across different prediction horizons. The ablation study results further validate the rationality of the proposed model design and demonstrate its effectiveness in improving cloud resource forecasting performance. This work provides more reliable prediction support for elastic resource scheduling in cloud data centers.

The current implementation focuses on purely data-driven adaptive feature selection, where incorporating reliable domain knowledge or statistical priors may further enhance feature selection interpretability and robustness. Future work will investigate adaptive fusion strategies between learned representations and external prior knowledge. Different cloud platforms may exhibit diverse workload characteristics, resource distributions, and temporal patterns. Therefore, future work will extend the evaluation to additional public cloud workload datasets, to further validate the robustness and generalization capability of the proposed model.