LLM-Empowered Kolmogorov-Arnold Frequency Learning for Time Series Forecasting in Power Systems

Abstract

With the rapid evolution of artificial intelligence technologies in power systems, data-driven time-series forecasting has become instrumental in enhancing the stability and reliability of power systems, allowing operators to anticipate demand fluctuations and optimize energy distribution. Despite the notable progress made by current methods, they are still hindered by two major limitations: most existing models are relatively small in architecture, failing to fully leverage the potential of large-scale models, and they are based on fixed nonlinear mapping functions that cannot adequately capture complex patterns, leading to information loss. To this end, an LLM-Empowered Kolmogorov–Arnold frequency learning (LKFL) is proposed for time series forecasting in power systems, which consists of LLM-based prompt representation learning, KAN-based frequency representation learning, and entropy-oriented cross-modal fusion. Specifically, LKFL first transforms multivariable time-series data into text prompts and leverages a pre-trained LLM to extract semantic-rich prompt representations. It then applies Fast Fourier Transform to convert the time-series data into the frequency domain and employs Kolmogorov–Arnold networks (KAN) to capture multi-scale periodic structures and complex frequency characteristics. Finally, LKFL integrates the prompt and frequency representations through an entropy-oriented cross-modal fusion strategy, which minimizes the semantic gap between different modalities and ensures full integration of complementary information. This comprehensive approach enables LKFL to achieve superior forecasting performance in power systems. Extensive evaluations on five benchmarks verify that LKFL sets a new standard for time-series forecasting in power systems compared with baseline methods.

1. Introduction

In contemporary society, as a nation’s critical infrastructure, power systems are undergoing continuous expansion in scale and complexity [1,2]. Encompassing vast domains—from large-scale power stations and intricate grids to countless end-users—the extensive scope and intricate composition of these systems make time-series forecasting increasingly vital. Accurate forecasting is indispensable for the efficient operation of power systems. For daily operations, power companies rely on forecasted electricity demand to rationally schedule generator units, determining their startup/shutdown times and output levels to minimize operational costs. Grid operators depend on accurate forecasts to monitor grid status, proactively identify potential failure points, ensure safe and stable grid operation, and prevent large-scale blackouts. Simultaneously, for policymakers, long-term power time-series data forecasts enable rational planning of power resource development and allocation, facilitate the formulation of scientific energy policies, and guide the sustainable development of the power industry [3,4,5]. Historically, power system time-series forecasting primarily employed traditional statistical methods, such as autoregression (AR) and moving averages (MA). These methods, leveraging their mature theoretical foundations and relatively simple model structures, could achieve reasonable forecasting performance when handling data with pronounced linear relationships. However, as power systems evolved, data inevitably exhibited significant nonlinear characteristics—such as electricity demand being influenced by the complex interplay of factors like temperature, holidays, and industrial production cycles. Traditional statistical methods proved inadequate in capturing these nonlinear relationships [6]. Furthermore, the dimensionality of data generated by power systems continues to increase. Traditional methods often require cumbersome feature selection and data preprocessing for high-dimensional data and impose demanding smoothness requirements on the data. Any data volatility or anomalies significantly degrades prediction accuracy. These limitations substantially constrain their effectiveness in modern power system time-series forecasting [7,8,9].

In recent years, machine learning methods have emerged to address the limitations of traditional statistical approaches in power system time-series prediction. Support vector machines (SVMs) have shown effectiveness in some small-scale prediction problems by mapping data to high-dimensional spaces and finding optimal classification hyperplanes [10,11]. AdaBoost, an adaptive boosting algorithm, enhances prediction accuracy by combining weak learners and focusing on previously misclassified samples. However, these machine learning methods often have high computational complexity, making their application to large-scale power system data time-consuming and resource-intensive. With the rise of deep learning, new opportunities have arisen for power system time-series prediction. Recurrent neural networks (RNNs), long short-term memory networks (LSTMs), and gated recurrent units (GRUs) have gained attention [12,13,14,15]. RNNs model time dependencies in sequence data through their internal loops [16]. LSTMs improve upon RNNs with gating mechanisms to capture long-term dependencies [17], and GRUs further simplify the structure while retaining the ability to model long-term relationships [18]. These deep learning models excel at automatic feature extraction and efficiently handle nonlinear and high-dimensional data, bringing new breakthroughs to power system time-series prediction [19,20].

Despite the superior predictive performance of current deep learning methods in power system time-series prediction, several issues remain. First, most existing deep learning models have relatively small architectures and have not fully leveraged the potential of large-scale models. Power system data contains complex patterns and relationships that small-scale models, with limited parameters and simple neural connections, can only partially capture. This restricts further improvements in predictive accuracy. Second, current power system time-series prediction methods are predominantly based on multi-layer perceptrons (MLPs) [21,22,23]. Although MLPs can approximate complex function mappings through multi-layer neural networks with nonlinear transformations, these nonlinear transformations are often based on fixed activation functions like ReLU or sigmoid. Such fixed nonlinear mappings limit the model’s ability to dynamically adapt to the intricate and variable patterns present in power system data, resulting in suboptimal feature extraction and underfitting of the complex relationships within the data. This inflexibility makes it difficult for MLPs to fully capture the rich temporal dynamics and multi-scale characteristics inherent in power system time-series data, ultimately restricting their prediction accuracy and reliability.

To this end, a LLM-Empowered Kolmogorov–Arnold frequency learning (LKFL) is proposed for time series forecasting in power systems, which consists of LLM-based prompt representation learning, KAN-based frequency representation learning, and entropy-oriented cross-modal fusion. Specifically, LKFL constructs LLM-based prompt representation learning by transforming time-series data into text prompts and leveraging a pre-trained LLM to extract semantic-rich representations. This approach captures complex patterns and relationships within the data that are difficult to identify with traditional methods, enabling the model to better understand the underlying structure of power system data and providing a more profound contextual understanding. Moreover, LKFL builds KAN-based frequency representation learning by applying Fast Fourier Transform to convert time-series data into the frequency domain and employing Kolmogorov–Arnold networks to capture multi-scale periodic structures and complex frequency characteristics. Simultaneously, LKFL constructs entropy-oriented cross-modal fusion to bridge the semantic gap between different representation modalities. By minimizing the information entropy of frequency and prompt representations under fusion conditions, the model ensures that complementary information from both modalities is fully integrated. This fusion process, guided by information entropy minimization, enables the model to make more informed and accurate predictions by leveraging the strengths of both representations. Extensive evaluations on five benchmarks verify that LKFL sets a new standard for time-series forecasting in power systems compared with baseline methods.

The main contributions of our work are threefold:

• We introduce a pioneering framework that transforms multivariable time-series data into structured text prompts and leverages pre-trained Large Language Models (LLMs) to extract semantically rich representations. This approach captures complex, contextually nuanced patterns within power system data that are difficult to discern with conventional methods.
• We propose a novel frequency-domain learning module utilizing Kolmogorov–Arnold Networks. By applying Fast Fourier Transform and employing KANs with learnable activation functions, this component effectively captures multi-scale periodic structures and intricate frequency characteristics inherent in power system time-series data, overcoming the limitations of fixed-activation MLPs.
• We develop an entropy-minimization strategy for cross-modal fusion to bridge the semantic gap between prompt-based (semantic) and frequency-based representations. This theoretically grounded approach ensures the full integration of complementary information from both modalities, significantly enhancing the model’s ability to leverage diverse data characteristics for accurate forecasting.

2. The Proposed Method

Given a multivariable time-series data $X\in R^{M\times L}$ with L historical time steps in power systems, where M denotes the number of variables, the goal of the time series forecasting task aims to learn a mapping function $f(·)$ to predict future time-series values $Y\in R^{M\times T}$ with T time steps, i.e., $f:X\to Y$. To achieve the above objective, this paper proposes LLM-empowered Kolmogorov–Arnold frequency learning method for time series forecasting in power systems, which contains LLM-based prompt representation learning, KAN-based frequency representation learning, and entropy-oriented cross-modal fusion (see Figure 1).

2.1. LLM-Based Prompt Representation Learning

LLM-based prompt representation learning aims to leverage LLM [24,25] that has been trained on a vast amount of data, as a powerful feature extractor to learn robust representations from time-series data. To achieve this, it utilizes the prompt learning to transform time series data into the text description, and then extract latent representations via the prompt encoder with the text tokenization.

Specifically, given a multivariable time-series data $X\in R^{M\times L}$, we transform it as the prompt $P=\{p_1,p_2,\dots ,p_N\}\in R^{M\times S}$ where $p_i$ denotes the prompt corresponding the i-th variable that contains words and time series values, i.e., from $t^1$ to $t^L$, the values are $x_i^1,\dots ,x_i^L$ every 5 min, and the total trend value is ${\Delta }_T={\sum }_{t=1}^{L-1}{x}_i^{t+1}-x_i^t$. Then, we utilize a non-overlapping tokenization mechanism to reshape the prompt as a sequence of tokens, i.e., $\{g_1,g_2,\dots ,g_N\}\in R^{M\times G}$, where $g_i={\left[g_i^j\right]}_{j=1}^G$ denotes the j token of the i-th variable. Next, we input these tokens into the pre-trained LLM to obtain initial prompt representations. The computation process of the l-th layer in the pre-trained LLM is as follows:

$${\overline{E}}^l=MMSA\left(E^l\right)+E^l$$

(1)

$$E^{l+1}=FFN\left(LN\left({\overline{l}}^l\right)\right)+{\overline{E}}^l$$

(2)

where $E^0=G+E^p$. $E^p$ stands for the positional encoding. $LN(·)$ and $FFN(·)$ stand for the layer normalization and the feed-forward network, respectively. $MMSA(·)$ is defined as follows:

$$MMSA\left(E^l\right)={\rho }_o(Atten({\rho }_q{E}^l,{\rho }_k{E}^l,{\rho }_v{E}^l)$$

(3)

where ${\rho }_o$, ${\rho }_q$, ${\rho }_k$, and ${\rho }_v$ are the learnable parameters of the attention function $Atten(·)$. In the experiments, the pre-trained LLM is chosen as GPT-2 and parameters are frozen during training.

It has been empirically demonstrated that the significance of tokens in language model training is not uniform. Among all tokens, the last token in a prompt sequence emerges as a repository of the most comprehensive contextual knowledge. This is predominantly due to the inherent mechanism of masked multi-self attention in the pre-trained LLM, which facilitates the aggregation of information from preceding tokens. In other words, the representation of the last token is only affected by its previous token representation. Hence, to optimize computational efficiency, we store the last token representation $E^G$ to stand for the overall prompt E and input it into Transformer-based prompt encoder to generate final prompt representations $Z_p$.

$$Z_p=PEncoder\left(E^G\right)$$

(4)

where $PEncoder(·)$ denotes the Transformer-based prompt encoder.

2.2. KAN-Based Frequency Representation Learning

Time-series data in power systems are rich in frequency information. Current methods mainly rely on multi-layer perceptron frameworks to extract frequency-domain features, yet they cannot effectively identify multi-scale periodic structures in time-series data, restricting prediction accuracy. Kolmogorov–Arnold networks, with their excellent data fitting, flexibility, and learnable activation functions, can capture complex time-series patterns more accurately. Inspired by them, this paper introduces KAN-based frequency representation learning to better capture frequency characteristics of time-series data and enhance the prediction performance.

Specifically, given a multivariable time-series data $X\in R^{M\times L}$, we leverages the Fast Fourier Transform $FFT(·)$ to mapping it into the frequency domain:

$$X_f=FFT\left(X\right)={\int }_{-\infty }^{\infty }X\left(t\right)e^{-j2\pi wt}dt$$

(5)

where j stands for the imaginary unit. $X_f$ denotes the frequency transformation of X.

Then, we utilize KAN with learnable nonlinear mapping as a aggregator to extract fruitful the frequency information:

$${\overline{X}}_f=FKAN(X_f,W_{base},W_{spline})$$

(6)

where $W_{base}$ and $W_{spline}$ denote parameters of the base transformation and B-spline transformation, respectively, which are defined as follows:

$$X_f^b=W_{base}SiLU\left(X_f\right)$$

(7)

$$X_f^s=\sum _i{W}_{spline}^i·B_i^k\left(X_f\right)$$

(8)

where $SiLU(·)$ denotes the nonlinear activation. $W_{spline}^i$ is learnable weight. $B_i^k(·)$ is the i-th B-spline of degree k:

$$B_i^k\left(x\right)={\displaystyle \frac{x-g_i}{g_{i+k}-g_i}}{B}_i^{k-1}\left(X_f\right)+{\displaystyle \frac{g_{i+k+1}-X_f}{g_{i+k+1}-g_{i+1}}}{B}_{i+1}^{k-1}\left(X_f\right),$$

(9)

where the grid $[g_{-k},\dots ,g_0,g_1\dots ,g_{s+k}]$ that follows a uniform distribution over $[-1,1]$, determines the scale of spline interpolation. Attentively,

$$B_i^0\left(X_f\right)=\left\{\begin{array}{cc}1\hfill & \mathrm{if}{X}_f\in [g_i,g_{i+1})\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(10)

After performing the base transformation and B-spline transformation, the fusion output of KAN is as follows:

$${\overline{X}}_f=X_f^b+X_f^s$$

(11)

Finally, we leverage inverse $FF{T}^{-1}(·)$ to map ${\overline{X}}_f$ with fruitful frequency features back to the time domain:

$$Z_f=FF{T}^{-1}\left({\overline{X}}_f\right)={\int }_{-\infty }^{\infty }{\overline{X}}_f{e}^{j2\pi wt}dw$$

(12)

where $Z_f$ denotes frequency representations.

2.3. Entropy-Oriented Cross-Modal Fusion

To eliminate the cross-modal heterogeneous semantic gap between frequency and prompt representations, we propose entropy-oriented cross-modal fusion for fully integrating their complementary information.

Specifically, given prompt representations $Z_p$ and frequency representations $Z_f$, the fusion representations can be obtained via:

$$Z=Z_p⨂Z_f$$

(13)

where ⨂ denotes fusion operations such as the concatenation. To ensure full aggregation of cross-modal complementary information, we minimize the information entropy of frequency representation or prompt representation under fusion representation conditions:

$$minH\left(Z_f\right|Z)+H\left(Z_p\right|Z)=-{\mathbb{E}}_{{\mathcal{P}}_{Z_f,Z}}\left[\log \mathcal{P}\left(Z_f\right|Z)\right]-{\mathbb{E}}_{{\mathcal{P}}_{Z_p,Z}}\left[\log \mathcal{P}\left(Z_p\right|Z)\right]$$

(14)

where the smaller the $H\left(Z_f\right|Z)$ and $H\left(Z_p\right|Z)$, the better the complementary information fusion.

Taking the $H\left(Z_f\right|Z)$ as an example, computing the conditional entropy directly is challenging due to the complexity of the underlying data distributions and the high-dimensional nature of the representations. To address this challenge, we introduce a variational distribution $Q\left(Z_f\right|Z)$, which serves as an approximation to the true posterior distribution $\mathcal{P}\left(Z_f\right|Z)$. The goal then shifts to maximizing the lower bound of the expectation, as shown below:

$$\max {\mathbb{E}}_{{\mathcal{P}}_{Z_f,Z}}\left[\log \mathcal{P}\left(Z_f\right|Z)\right]⟺{\mathbb{E}}_{{\mathcal{P}}_{Z_f,Z}}\left[\log Q\left(Z_f\right|Z)\right]$$

(15)

To make this optimization more tractable, we assume that the variational distribution Q follows a Gaussian distribution, i.e., $\mathcal{N}\left(Z_f\right|G^f\left(Z\right),\sigma \mathbf{I})$. Here, $G^f(·)$ represents a learnable cross-view mapping function that transforms fusion representations back to the frequency modality, and $\sigma \mathbf{I}$ stands for a diagonal covariance matrix with variance $\sigma$. Substituting this Gaussian assumption into the expectation yields

$$\max {\mathbb{E}}_{{\mathcal{P}}_{Z_f,Z}}[\log Q\left(Z_f\right|Z)]={\mathbb{E}}_{{\mathcal{P}}_{Z_f,Z}}[\log ({\displaystyle \frac{1}{\sqrt{\sigma \mathbf{I}\sqrt{2\pi }}}}{e}^{-{\displaystyle \frac{{(Z_f-G^f\left(Z\right))}^2}{2\left(\sigma \mathbf{I}\right)}}})]$$

(16)

Expanding the logarithm of the Gaussian density function results in the following optimization problem:

$$\max {\mathbb{E}}_{{\mathcal{P}}_{Z_f,Z}}[-{\displaystyle \frac{{(Z_f-G^f\left(Z\right))}^2}{2\sigma \mathbf{I}}}+\log {\displaystyle \frac{1}{\sqrt{2\pi \sigma \mathbf{I}}}}]$$

(17)

where the term $-{\displaystyle \frac{{(Z_f-G^f\left(Z\right))}^2}{2\sigma \mathbf{I}}}$ corresponds to the negative squared Mahalanobis distance between $Z_f$ and $G^f\left(Z\right)$, which encourages the model to produce representations that are close under the cross-view mapping. The term $\log {\displaystyle \frac{1}{\sqrt{2\pi \sigma \mathbf{I}}}}$ acts as a normalization constant that ensures the Gaussian distribution is properly scaled. However, since this term is constant with respect to the model parameters, it can be safely ignored during optimization. Similarly, the scaling factor $2\sigma \mathbf{I}$ can be omitted for simplicity, leading to the prediction loss

$$\max -{\mathbb{E}}_{{\mathcal{P}}_{Z_f,Z}}{\parallel (Z_f-G^f\left(Z\right))\parallel }_2^2$$

(18)

Finally, the loss of entropy-oriented cross-modal fusion given prompt representations $Z_p$ and frequency representations $Z_f$ is defined as follows:

$$L_e={\parallel (Z_f-G^f\left(Z\right))\parallel }_2^2+{\parallel (Z_p-G^p\left(Z\right))\parallel }_2^2$$

(19)

2.4. The Loss Optimization

Current methods employ MSE for time series prediction optimization. While MSE is widely used due to its simplicity and effectiveness in penalizing large errors, it has certain limitations. For instance, MSE is highly sensitive to outliers, which can lead to suboptimal model performance when dealing with noisy data. To address these issues, this paper designs a smooth L1 Loss function to optimize the prediction performance. The Smooth L1 Loss combines the benefits of both L1 and L2 loss functions. It uses a squared term for small errors and a linear term for larger errors, which makes it less sensitive to outliers and provides more robust performance. The mathematical formulation of the smooth L1 Loss is as follows:

$$L_s=\left\{\begin{array}{cc}0.5 \times |Y-\overline{Y}{|}^2\hfill & \mathrm{if}|Y-\overline{Y}| <1\hfill \\ |Y-\overline{Y}| - 0.5\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(20)

where Y denotes true values. $\overline{Y}$ denotes prediction values generated via a linear predictor with learnable parameters $W_{pred}$:

$$\overline{Y}=W_{pred}Z$$

(21)

The benefits of using Smooth L1 Loss are multifold. Firstly, it reduces the influence of outliers by switching to a linear scaling for larger errors, which results in a more balanced overall loss. Secondly, it maintains the differentiability at zero, which helps in stable gradient descent optimization. Thirdly, it provides a good balance between the robustness of L1 loss and the stability of L2 loss during optimization.

The overall loss function is a combination of the $L_s$ and $L_e$. For the time series forecasting task, the total loss function $L_{total}$ can be expressed as

$$L_{total}=L_s+\lambda L_e$$

(22)

where $\lambda$ is a weighting parameter that balances the contributions of the different loss. For the optimization process, we adopt the Adam optimizer, which is a popular adaptive gradient-based optimization algorithm. It computes adaptive learning rates for each parameter by maintaining moving averages of the gradients and the squared gradients. This allows the model to converge more efficiently and effectively, especially in high-dimensional parameter spaces. The optimizer updates the model parameters by moving in the direction of the negative gradient of the loss function with respect to the parameters, thereby minimizing the overall loss.

3. Experimental Evaluation

3.1. Setup

Datasets. The publicly available time-series datasets from five power systems were used to validate the performance of the model [26,27]. ETTh1 and ETTh2 datasets contain 7 variables with a 1 h time interval, a total length of 17,420, and follows a 6:2:2 split ratio. ETTm1 and ETTm2 datasets comprise 7 variables recorded at a 15 min interval, with a length of 69,680 and a 6:2:2 split ratio of the training set, the validating set, and the testing set. Electricity includes 321 variables sampled at a 1 h interval, has a total length of 26,304, and adopts a 7:1:2 split ratio to conduct the training set, the validating set, and the testing set.

Evaluation Metrics. To evaluate the prediction performance, MSE and MAE are used as metrics in the experiments [27,28]. RMSE is a statistical measure used to evaluate prediction accuracy by computing the square root of the average squared differences between predicted and observed values. MAE calculates the average absolute differences between forecasted and true values, offering an intuitive understanding of the typical prediction error magnitude and serving as a robust metric less affected by outliers.

Comparison methods. Ten time-series prediction methods are used, including FS-TSF [1], Hybrid-net [3], IMC-net [4], DAE-TSF [14], SAC-ConvLSTM [16], LT-TSF [22], Timecma [24], Gpt4mts [26], Effformer [29], and TimeKAN [30]. In order to guarantee a fair comparison across all experiments, we meticulously ensured that every experiment was carried out within the same hardware and software settings. Regarding the hardware environment, all experiments were performed on identical high-performance servers equipped with Intel Xeon Gold 6248 with the BIOS version 1.30, NVIDIA Tesla V100 GPUs, and 192 GB of RAM. The software environment was also kept uniform, with the same versions of Ubuntu 20.04 LTS operating system, Python 3.8, PyTorch 1.9.0 deep learning framework, and other related libraries and tools. When it comes to hyperparameter configurations, we strictly followed the original publications of each method. We carefully examined the original papers, extracted the exact hyperparameter values, and implemented them without any alteration to ensure that the performance of each method was evaluated under conditions that aligned with the authors’ initial setups, thus providing a fair and valid basis for comparison.

Implementation details. A grid dimension of $s=2$ alongside a B-spline degree of $k=1$ is utilized to conduct KAN, with each layer within the KAN possessing a latent dimensionality of 256. Five datasets undergo standardization through application of the MinMax scaling technique. For the optimization procedure, we implement the Adam algorithm, wherein the learn rate is calibrated within the interval spanning $[{10}^{-4},{10}^{-2}]$. Meanwhile, the batch sizes are varied across the spectrum from 4 to 64 samples. Training of all architectures is executed for a maximum of 20 epochs, with premature termination of training initiated when the loss on the validation set plateaued, indicating potential overfitting.

3.2. Comparison Evaluation

In this section, a full evaluation of the proposed method and ten methods is designed on five common time series datasets in power systems in terms of MSE and MAE. In the evaluation, the historical window size L is set to =96 and future window size T is chosen from {96, 192, 336, 720} for five datasets. The evaluation results are depicted in

Table 1. The time-series prediction results on the ETTh1 dataset. Bold indicates the best result.

Dataset Metric	T = 96		T = 192		T = 336		T = 720
	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE
FS-TSF	0.121	0.099	0.145	0.108	0.169	0.127	0.189	0.143
Hybrid-net	0.098	0.076	0.106	0.080	0.110	0.089	0.115	0.094
IMC-net	0.135	0.102	0.178	0.146	0.204	0.174	0.229	0.176
DAE-TSF	0.108	0.082	0.146	0.115	0.156	0.124	0.159	0.128
SAC-ConvLSTM	0.103	0.074	0.109	0.078	0.113	0.082	0.120	0.088
LT-TSF	0.089	0.063	0.096	0.068	0.110	0.080	0.117	0.086
Timecma	0.107	0.077	0.121	0.089	0.134	0.101	0.150	0.119
Gpt4mts	0.091	0.063	0.096	0.069	0.117	0.083	0.121	0.099
Effformer	0.092	0.064	0.101	0.073	0.112	0.086	0.150	0.116
TimeKAN	0.092	0.061	0.097	0.069	0.115	0.085	0.119	0.092
Ours	0.088	0.062	0.093	0.067	0.107	0.079	0.110	0.083

,

Table 2. The time-series prediction results on the ETTh2 dataset. Bold indicates the best result.

Dataset Metric	T = 96		T = 192		T = 336		T = 720
	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE
FS-TSF	0.090	0.074	0.093	0.082	0.095	0.099	0.097	0.102
Hybrid-net	0.092	0.076	0.095	0.081	0.097	0.090	0.105	0.097
IMC-net	0.088	0.075	0.091	0.079	0.102	0.087	0.109	0.096
DAE-TSF	0.067	0.051	0.064	0.048	0.079	0.060	0.106	0.085
SAC-ConvLSTM	0.075	0.052	0.085	0.061	0.089	0.065	0.090	0.066
LT-TSF	0.076	0.052	0.084	0.059	0.093	0.068	0.094	0.069
Timecma	0.054	0.038	0.062	0.047	0.068	0.048	0.075	0.057
Gpt4mts	0.055	0.038	0.065	0.046	0.075	0.054	0.089	0.067
Effformer	0.054	0.039	0.062	0.046	0.087	0.067	0.099	0.072
TimeKAN	0.068	0.045	0.077	0.052	0.083	0.058	0.088	0.062
Ours	0.053	0.037	0.060	0.043	0.063	0.046	0.072	0.054

,

Table 3. The time-series prediction results on the ETTm1 dataset. Bold indicates the best result.

Dataset Metric	T = 96		T = 192		T = 336		T = 720
	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE
FS-TSF	0.108	0.086	0.114	0.097	0.127	0.108	0.132	0.116
Hybrid-net	0.120	0.092	0.127	0.098	0.134	0.108	0.155	0.119
IMC-net	0.117	0.084	0.132	0.101	0.157	0.123	0.185	0.133
DAE-TSF	0.102	0.070	0.100	0.071	0.104	0.072	0.110	0.089
SAC-ConvLSTM	0.096	0.071	0.101	0.073	0.110	0.085	0.122	0.096
LT-TSF	0.083	0.058	0.090	0.064	0.099	0.069	0.110	0.082
Timecma	0.088	0.061	0.801	0.610	0.756	0.504	0.770	0.551
Gpt4mts	0.084	0.062	0.090	0.069	0.102	0.077	0.112	0.093
Effformer	0.082	0.056	0.088	0.062	0.098	0.072	0.108	0.083
TimeKAN	0.082	0.055	0.085	0.063	0.099	0.074	0.109	0.082
Ours	0.079	0.054	0.080	0.060	0.095	0.066	0.106	0.077

,

Table 4. The time-series prediction results on the ETTm2 dataset. Bold indicates the best result.

Dataset Metric	T = 96		T = 192		T = 336		T = 720
	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE
FS-TSF	0.069	0.059	0.074	0.063	0.082	0.074	0.89	0.082
Hybrid-net	0.080	0.072	0.088	0.078	0.096	0.088	0.107	0.092
IMC-net	0.058	0.039	0.065	0.043	0.070	0.046	0.081	0.054
DAE-TSF	0.043	0.031	0.048	0.037	0.059	0.044	0.084	0.060
SAC-ConvLSTM	0.095	0.077	0.104	0.086	0.117	0.095	0.126	0.109
LT-TSF	0.083	0.065	0.101	0.079	0.110	0.084	0.121	0.082
Timecma	0.047	0.045	0.054	0.040	0.756	0.043	0.087	0.055
Gpt4mts	0.064	0.042	0.073	0.056	0.088	0.056	0.096	0.073
Effformer	0.044	0.033	0.052	0.036	0.070	0.047	0.085	0.056
TimeKAN	0.051	0.033	0.060	0.038	0.068	0.045	0.078	0.051
Ours	0.040	0.028	0.047	0.033	0.054	0.038	0.063	0.045

and

Table 5. The time-series prediction results on the Electricity dataset. Bold indicates the best result.

Dataset Metric	T = 96		T = 192		T = 336		T = 720
	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE
FS-TSF	0.091	0.067	0.103	0.074	0.123	0.077	0.126	0.079
Hybrid-net	0.083	0.056	0.094	0.070	0.112	0.079	0.120	0.086
IMC-net	0.077	0.058	0.081	0.063	0.087	0.069	0.097	0.075
DAE-TSF	0.085	0.071	0.097	0.068	0.107	0.075	0.153	0.092
SAC-ConvLSTM	0.088	0.068	0.096	0.065	0.106	0.075	0.135	0.077
LT-TSF	0.078	0.057	0.084	0.054	0.087	0.059	0.092	0.065
Timecma	0.082	0.059	0.087	0.057	0.079	0.056	0.096	0.071
Gpt4mts	0.074	0.045	0.072	0.044	0.073	0.047	0.086	0.058
Effformer	0.072	0.045	0.074	0.047	0.077	0.051	0.086	0.058
TimeKAN	0.075	0.048	0.079	0.052	0.080	0.055	0.093	0.063
Ours	0.069	0.041	0.071	0.043	0.072	0.045	0.084	0.056

. The proposed method obtains the optimal results on five datasets under different future window size T. This results demonstrate our method achieves a new baseline in this task. The reasons for the performance advantage are threefold. Firstly, the integration of LLM-based prompt representation learning enables the model to leverage the vast knowledge contained within pre-trained large language models. This allows for the extraction of robust and semantically rich representations from time-series data. By transforming time-series data into text descriptions and utilizingprompt encoders to extract latent representations, the model can better understand the underlying patterns and relationships in power system data. This significantly enhances the overall forecasting accuracy and provides a more profound contextual understanding compared to traditional methods. Secondly, the KAN-based frequency representation learning component effectively captures multi-scale periodic structures and complex frequency characteristics inherent in power system time-series data. Kolmogorov–Arnold networks, with their excellent data fitting capabilities and learnable activation functions, can more accurately model the intricate patterns of time-series data. This component allows the model to exploit frequency-domain features that are often overlooked by conventional approaches, thereby improving the prediction performance. Lastly, the entropy-oriented cross-modal fusion strategy bridges the semantic gap between different representation modalities. By minimizing the information entropy of frequency and prompt representations under fusion conditions, the model ensures that complementary information from both modalities is fully integrated. This fusion process, guided by information entropy minimization, enables the model to make more informed and accurate predictions by leveraging the strengths of both representations. The combination of these three components creates a powerful framework that outperforms existing methods across various future window sizes and datasets.

Meanwhile, a comparison experiment of cumulative relative error between our methods and competitive methods is designed for the model robustness in power systems. The cumulative relative error CRE is defined as follows:

$$\mathrm{CRE}={\displaystyle \frac{{\sum }_{t=1}^T({\overline{Y}}_t-Y_t)}{{\sum }_{t=1}^T{Y}_t}}$$

(23)

CRE calculates the ratio of cumulative deviation of predictions to actual values over the planning horizon, which reflects the cumulative bias of a model. The results are shown in

Table 6. The results of the cumulative relative error on the ETTm1 and Electricity datasets. Bold indicates the best result.

Dataset Metric	T = 96		T = 336		T = 720
	ETTm1	Electricity	ETTm1	Electricity	ETTm1	Electricity
IMC-net	−0.0018	−0.0048	0.0065	0.0088	0.0139	0.0166
LT-TSF	0.0019	0.0049	0.0067	0.0090	0.0137	0.0165
Timecma	0.0022	0.0053	0.0072	0.0096	0.0146	0.0168
Gpt4mts	0.0016	0.0042	0.0063	0.0084	−0.0133	−0.0154
Effformer	−0.0014	−0.0042	0.0064	0.0085	−0.0135	−0.0155
TimeKAN	0.0017	0.0044	0.0064	0.0086	0.0138	0.0159
Ours	0.0015	0.0038	0.0060	0.0080	0.0130	0.0150

. The results indicate that the ours method demonstrates superior performance by achieving the lowest cumulative relative error across all forecasting horizons and datasets. This suggests that the method possesses enhanced robustness and generalization capability. It is capable of accurately capturing the underlying patterns of the data, even when faced with complex distributions and dynamic characteristics, thereby maintaining predictive accuracy over extended forecasting periods. In contrast, other models may exhibit acceptable performance in short-term forecasting but tend to accumulate larger deviations as the forecasting horizon increases, leading to a decline in their effectiveness.

3.3. Ablation Analysis

This section performs the ablation analysis to verify the impact each component in the proposed method on the prediction performance, which contains the loss ablation and the architecture ablation.

Specifically, the loss ablation sets up two ablation variants. (1) Variant_1 utilizes vanilla MSE and $L_e$ losses to optimize the overall network. (2) Variant_2 utilizes only vanilla MSE loss to optimize the overall network. (3) Variant_3 utilizes $L_s$ loss to optimize the overall network. The ablation results on the ETTh1 dataset with four future window sizes are shown in

Table 7. Loss analysis on the ETTh1 dataset in terms of RMSE and MAE. Bold indicates the best result.

Dataset Metric	T = 96		T = 192		T = 336		T = 720
	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE
Variant_1	0.098	0.072	0.100	0.076	0.120	0.097	0.132	0.107
Variant_2	0.089	0.063	0.094	0.069	0.110	0.083	0.113	0.086
Variant_3	0.092	0.067	0.099	0.075	0.117	0.092	0.127	0.102
Ours	0.088	0.062	0.093	0.067	0.107	0.079	0.110	0.083

. Results show that variant_3 outperforms variant_2, highlighting the effectiveness of the $L_s$ loss function. Similarly, variant_1 demonstrates superior performance compared to variant_2, which underscores the positive impact of incorporating the $L_e$ loss function. Furthermore, our proposed method surpasses all three variants. This indicates that our overall loss function design is not only rational but also advanced, as it integrates the advantages of different loss components to achieve optimal results. The ablation study thus provides strong evidence for the effectiveness of each loss component and the superiority of our comprehensive loss function design.

The architecture ablation sets up two ablation variants: Ours w/o LLM denotes the erasure of the LLM-based prompt representation learning; Ours w/o KAN denotes the erasure of the KAN-based frequency representation learning; Ours w/o entropy denotes the erasure of the entropy-oriented cross-modal fusion representation learning. The ablation results on the ETTh1 dataset with four future window sizes are shown in

Table 8. Architecture analysis on the ETTh1 dataset in terms of RMSE and MAE. Bold indicates the best result.

Dataset Metric	T = 96		T = 192		T = 336		T = 720
	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE
Ours w/o LLM	0.098	0.071	0.102	0.075	0.119	0.081	0.131	0.103
Ours w/o KAN	0.091	0.065	0.095	0.070	0.110	0.085	0.117	0.096
Ours w/o Entropy	0.090	0.065	0.098	0.072	0.115	0.86	0.121	0.097
Ours	0.088	0.062	0.093	0.067	0.107	0.079	0.110	0.083

. There are three observations: (1) The variant Ours w/o LLM demonstrates inferior performance compared to the full model. This indicates that the LLM-based prompt representation learning component plays a crucial role in capturing robust and semantic-rich representations from the time-series data. By leveraging the vast knowledge contained within pre-trained LLMs, this component enables the model to better understand the underlying patterns and relationships in the power system data, thereby enhancing the overall forecasting accuracy. (2) Ours w/o KAN shows a significant performance drop compared to the complete model. This highlights the importance of the KAN-based frequency representation learning component. Power system time-series data inherently contains rich frequency information, and the KAN module is specifically designed to effectively capture multi-scale periodic structures and complex frequency characteristics. Without this component, the model’s ability to exploit frequency-domain features is compromised, leading to less accurate predictions. (3) Ours w/o Entropy also underperforms relative to the full model. This underscores the value of the entropy-oriented cross-modal fusion strategy. This component is essential for bridging the semantic gap between different representation modalities and ensuring that complementary information from both prompt and frequency representations is fully integrated. The fusion process, guided by information entropy minimization, allows the model to make more informed and accurate predictions by leveraging the strengths of both representation types.

3.4. Parameter Analysis

This section performs parameter analysis of the trade-off parameter $\lambda$, the latent representation size D, and the layer number $Ln$ of KAN on the ETTh1 dataset with four future window sizes T.

Trade-off Parameter $\lambda$: We conduct experiments with $\lambda \in \{100,10,1,0.1,0.01,0.001\}$. The results in Figure 2 indicate that the model achieves optimal performance when $\lambda$ is set within the range of 0.1 to 1. A higher $\lambda$ tends to overemphasize the frequency representation learning, while a lower $\lambda$ may underweight its significance. The optimal range effectively balances the contributions of LLM-based prompt representation learning and KAN-based frequency representation learning, ensuring comprehensive and accurate predictions.

Latent Representation Size D: Experiments are carried out with different sizes $D\in \{10,64,128,256,512,1024\}$. The findings in Figure 2 reveal that a latent representation size that is too small (e.g., $D=10$) limits the model’s capacity to capture intricate patterns in the data, resulting in underfitting. Conversely, an excessively large D (e.g., $D=1024$) may introduce noise and lead to overfitting. The optimal performance is observed when D is within the range of 128 to 256, which strikes a balance between capturing sufficient information and avoiding redundancy, thereby enhancing the model’s forecasting accuracy.

Layer Number M of KAN: We explore values of M from 1 to 5. The results in Figure 2 demonstrate that a single layer is insufficient to model the complex frequency characteristics of power system time-series data. As the number of layers increases, the model’s ability to capture multi-scale periodic structures improves. However, beyond three layers, the performance gains diminish and overfitting becomes a concern. The optimal performance is achieved with M set to 3, which effectively captures the essential frequency features while maintaining computational efficiency and generalization capability.

4. Conclusions

This study addresses the critical challenge of accurate time-series forecasting in complex power systems by proposing LLM-Empowered Kolmogorov–Arnold frequency learning (LKFL) that consists of LLM-based prompt representation learning, KAN-based frequency representation learning, and entropy-oriented cross-modal fusion. Three components work seamlessly together to capture fruitful frequency and temporal complementary patterns for time series forecasting in power systems. Meanwhile, ablation experiments demonstrate the ablation of each component yield inferior results compared with LKFL, which validates the effectiveness of each component. Future work will explore lightweight LLM distillation and extend the framework to irregularly sampled or sparse power data. Nevertheless, LKFL establishes a new paradigm for time-series forecasting in critical infrastructure, with immediate applications in grid scheduling, fault prevention, and energy policy optimization.