Skip to Content
You are currently on the new version of our website. Access the old version .
EntropyEntropy
  • Article
  • Open Access

5 January 2026

22 Pages

Perceiving Unpredictability for New Energy Power and Electricity Consumption Forecasting

,
,
,
,
and
1
Institute of Economics and Technology of State Grid Liaoning Electric Power Co., Ltd., Shenyang 110006, China
2
School of Science, Harbin Institute of Technology, Shenzhen 518055, China
*
Authors to whom correspondence should be addressed.

Abstract

Accurate prediction of sensor network data in critical domains such as electric power systems and traffic planning is a core task for ensuring grid stability and enhancing urban operational efficiency. Although deep learning models have achieved significant architectural advancements, their training strategy implicitly assumes that all future events are equally predictable, ignoring that the future evolution of sensor signals intertwines deterministic patterns with stochastic events and that prediction difficulty increases with temporal distance. Forcing a model to fit inherently unpredictable events with a uniform supervision may impair its ability to learn generalizable patterns. To address this, we introduce an Unpredictability Perception loss that dynamically computes a supervision weight. The computation of this weight unifies two assessment dimensions of the intrinsic unpredictability of the forecasting task. The first originates from a posterior analysis of the signal content’s randomness, while the second stems from an a priori consideration of temporal distance. The first dimension, through a complexity-aware weight derived from local spectral entropy, reduces supervision on random segments of the signal. The second dimension, through a temporal decay weight based on exponential decay, lessens supervision for distant future points. Applied to the advanced TimeMixer model, experimental results show that our approach achieves performance improvements across multiple public benchmark datasets. By matching the supervision strength to the intrinsic predictability of the signals, our proposed Unpredictability Perception loss function enhances the forecasting accuracy for sensor network data, providing a more reliable technical foundation for ensuring the stability of critical infrastructures like power grids and optimizing urban traffic systems.

1. Introduction

Time series forecasting is an indispensable technology in modern scientific and industrial decision-making [1,2,3]. In the domain of energy management, the precise prediction of electricity load, informed by feedback from thousands of sensor nodes within smart grids, serves as the cornerstone for ensuring grid stability, optimizing energy dispatch, and integrating volatile renewable sources such as wind and solar power [4,5,6]. In urban traffic systems, the accurate forecasting of traffic flow, derived from data collected by road sensor networks, provides critical support for intelligent traffic guidance and public resource planning, thereby effectively mitigating daily commuter congestion and responding to traffic disruptions caused by unforeseen incidents [7,8,9]. Similarly, in meteorological science, the long-term prediction of key indicators such as temperature and humidity, monitored by distributed weather sensors, is directly related to agricultural production, extreme weather event warnings, and societal well-being [10,11]. A commonality among these high-stakes applications is the urgent need for models capable of understanding and predicting the high-dimensional time series data generated by massive sensor arrays [12,13]. However, these data themselves are often interwoven with complex dynamic characteristics [14,15], which encompass both learnable, deterministic patterns governed by physical laws or societal rhythms, and are inevitably accompanied by inherently unpredictable, random events triggered by internal system perturbations or abrupt environmental changes [16]. In recent years, the rapid development of deep learning has provided powerful tools to address this challenge. From Transformer-based architectures such as PatchTST [17] and iTransformer [18], to MLP-based models renowned for their simplicity and efficiency like TimeMixer [19], the academic community has made significant progress in enhancing predictive accuracy. These state-of-the-art methods, through their sophisticated structural designs, have far surpassed traditional statistical models in capturing long-term dependencies and complex patterns.
Despite the notable achievements in architectural design, nearly all existing models adhere to a common supervision principle during training, which involves optimizing model parameters by minimizing the Mean Squared Error (MSE) [20]. This supervision principle contains a fundamental assumption that all future events are accurately predictable based on historical information. However, the evolutionary processes of the real world are naturally intertwined with both deterministic patterns and stochastic events. For instance, the electricity load data involved in our experiments contains learnable daily and weekly periodic patterns determined by social production and life rhythms, but it also inevitably includes random disturbances that are inherently unpredictable from historical data, caused by factors such as the random behavior of a large number of individual users, sudden equipment failures, or extreme weather. Likewise, traffic flow data also mixes predictable peak-hour patterns with unpredictable sharp fluctuations caused by contingent events like traffic accidents. The traditional MSE loss function, by imposing a uniform penalty on prediction errors at all future time points, effectively forces the model to fit these intrinsically random and unpredictable future events. This form of supervision may not only lead to the model overfitting to artifacts in the training data but could also impair its ability to learn generalizable underlying patterns, as it compels the model to expend valuable modeling capacity in pursuit of a theoretically unattainable goal.
Therefore, a more effective training paradigm should shift from uniform supervision to adaptive supervision, aligning the learning objective with the intrinsic predictability of future events. This is the core motivation of our research. We posit that the model training process should be redefined as a selective knowledge extraction process rather than a uniform data-fitting task. Specifically, the supervision strength applied to the model should dynamically match the predictability of the task itself. The model should be incentivized to accurately capture predictable dynamics that are close in time and exhibit high signal regularity, while for dynamics that are distant in time or have high signal randomness and thus low predictability, the supervision strength should be appropriately attenuated. Such an adaptive supervision strategy holds the promise of fundamentally optimizing the model’s learning process without adding any architectural complexity, thereby significantly enhancing its robustness and final prediction accuracy.
To this end, we propose an Unpredictability Perception (UP) loss function. This method dynamically computes a supervision weight for each time point in the prediction horizon by fusing a posterior analysis of the signal’s local complexity with a priori knowledge of temporal distance. This weight accurately reflects the intrinsic credibility of each prediction sub-task, thereby guiding the model toward more effective and robust learning. By encoding this inductive bias directly into the optimization objective, we provide a simple yet powerful functional extension to state-of-the-art time series forecasting models, enabling them to make more reliable and precise judgments in complex predictive environments.

3. Methodology

3.1. Framework Overview

Deep learning models have achieved significant success in time series forecasting, largely guided by the optimization of a loss function. The Mean Squared Error (MSE) is widely adopted for its simplicity and ease of optimization. However, a fundamental deficiency of the MSE loss is that it applies a uniform penalty weight to all points within the prediction horizon, failing to differentiate among the intrinsic difficulties of various forecasting tasks. In fact, prediction uncertainty stems from two distinct sources. The first is the inherent randomness of the signal itself, meaning that the sequence dynamics in some time segments are intrinsically harder to predict than in others. The second is the a priori difficulty of the forecasting task, where predictions for the distant future naturally contain more uncertainty than those for the near future. An advanced training paradigm should be able to recognize and adapt to both types of uncertainty, thereby intelligently allocating the model’s learning resources to patterns that are genuinely learnable in both a content and temporal sense. To this end, we propose the UP loss function, which guides the model toward more effective and robust learning by dynamically computing a supervision weight for each time point in the prediction horizon. The construction of this weight integrates a posterior analysis of the signal’s local complexity with a priori knowledge of temporal distance, as conceptually illustrated in Figure 1.
Figure 1. Overview of the Unpredictability Perception loss function. The (a) illustrates how future sensor signals contain regions of high unpredictability. The (b,c) show the two core components of our method: Content Unpredictability Perception and Time Unpredictability Perception.
To elucidate the complete computational flow and the integration of our loss function with a backbone forecasting model, we present a systematic framework diagram in Figure 2. As depicted, the input time series is first processed by the core forecasting model (e.g., TimeMixer), which utilizes its internal architecture, such as decomposition and multi-resolution mixing, to generate the prediction sequence Y ^ . The proposed UP Loss module then operates as the optimization objective. It takes both the model’s prediction Y ^ and the ground-truth sequence Y as inputs. Critically, it computes the complexity weight ( w complexity ) by analyzing the spectral entropy of the ground-truth signal itself, while concurrently calculating the temporal decay weight ( w time ). These two weights are multiplicatively combined to modulate the standard Mean Squared Error, forming the final L UP . This adaptively weighted loss is then used to compute the gradients and update the parameters of the forecasting model via backpropagation.
Figure 2. Systematic framework illustrating the integration of the UP Loss with a backbone forecasting model. The input time series is processed by the backbone model (e.g., TimeMixer using decomposition and mixing) to generate the prediction. The UP Loss module then compares the prediction to the ground truth. It computes the final L UP by weighting the standard MSE, using w complexity (derived from spectral entropy) and w time (derived from temporal distance), to guide backpropagation.

3.2. Local Complexity Quantification via Spectral Entropy

The first component of our method aims to quantify the predictability of the signal content itself. This addresses what is formally known as aleatoric uncertainty, the irreducible randomness inherent in the data generation process. In time series analysis, particularly in volatile energy systems, segments dominated by aleatoric uncertainty (e.g., sensor noise or chaotic wind power fluctuations) theoretically lack extractable patterns. Forcing a model to fit these segments leads to overfitting. Therefore, we require a metric to distinguish between deterministic patterns and stochastic noise.
Ideally, Kolmogorov complexity from algorithmic information theory offers the ultimate theoretical measure of randomness, defining the complexity of a sequence as the length of its shortest generative program [43,44]. However, since this complexity is theoretically uncomputable, it inspires the search for an effective and computable proxy. We employ Spectral Entropy (SE) [45] for this purpose. By calculating the Shannon entropy of a signal’s power spectral density, spectral entropy effectively measures the signal’s structural regularity rather than just its amplitude [46,47]. A signal segment with a simple, regular structure has its energy concentrated in a few frequencies, resulting in low spectral entropy and indicating high predictability. Conversely, a segment resembling white noise has its energy uniformly distributed across the frequency domain, yielding high spectral entropy and indicating weak predictability. This makes SE an ideal, theoretically grounded proxy for modulating supervision based on inherent signal learnability.
To capture these time-varying local dynamics, we adopt a sliding window strategy. For any time point t ∈ { 1 , … , H } within the ground-truth future sequence Y = ( y 1 , y 2 , … , y H ) , we define a local window Y t local of length P local centered at that point. To compute its spectral entropy, we first estimate the Power Spectral Density (PSD). The signal within the window is processed by the Fast Fourier Transform (FFT) to obtain its frequency representation. The periodogram, a common estimator for the PSD, is then calculated as
P t ( f ) = 1 P local ∑ n = 0 P local − 1 y t , n local e − j 2 π f n / P local 2
where y t , n local is the n-th sample in the window Y t local and f represents the discrete frequency bins. To treat the power distribution as a probability distribution, we normalize the PSD across all K frequency bins
p t ( f k ) = P t ( f k ) ∑ i = 1 K P t ( f i )
The Shannon entropy of this distribution is then computed as
H ( p t ) = − ∑ k = 1 K p t ( f k ) log 2 ( p t ( f k ) )
To ensure the entropy value is scaled between 0 and 1, facilitating its use as a weight, we normalize it by the maximum possible entropy for a K-bin distribution, which is log 2 ( K ) . This yields the final Local Spectral Entropy S E t :
S E t = H ( p t ) log 2 ( K )
From this, we define the first weight component, the complexity weight w complexity , t , which is inversely proportional to the signal’s randomness:
w complexity , t = 1 − S E t
This weight assigns a value approaching 1 to time points in regular, predictable regions and a value approaching 0 to those in noisy, random regions. This mechanism compels the model during training to focus its attention more on signal segments that exhibit clear and learnable patterns.

3.3. Temporal Weighting via Exponential Decay

The second component addresses epistemic uncertainty, which arises from the model’s lack of knowledge about the future state. A fundamental axiom in forecasting is that this uncertainty accumulates as the prediction horizon extends, often referred to as the “butterfly effect” in dynamical systems. To model this horizon-dependent uncertainty, we introduce a temporal decay weight w time , t . We specifically select an exponential decay function, w ( t ) = e − λ t , rather than a linear one, to align with the information-theoretic principle that the mutual information between the current state and the future state decays exponentially in chaotic or mixing systems. This non-linear formulation effectively penalizes errors in the near-term (where high precision is feasible) while gracefully relaxing constraints in the long-term (where uncertainty is dominant), thereby aligning the optimization objective with the theoretical limits of predictability.
w time , t = exp ( − λ · t − 1 H − 1 )
where t is the time step index in the prediction horizon, ranging from 1 to H. The term t − 1 H − 1 normalizes the time step to the interval from 0 to 1, ensuring that the decay process is independent of the specific forecast horizon and thus preserving the generalizability of the mechanism across different tasks. λ is a non-negative hyperparameter that controls the rate of decay. When λ is zero, temporal decay has no effect, and the model treats all time points equally regardless of their distance. When λ is greater than zero, more distant prediction points are assigned a stronger decay in their weights.
The final supervision weight w final , t is determined by the element-wise product of the complexity and temporal decay weights. This multiplicative fusion integrates two distinct dimensions of unpredictability assessment, one derived from a posteriori analysis of the signal and the other from a priori temporal knowledge, ensuring that a high supervision strength is assigned only to time points that are concurrently characterized by high signal regularity and temporal proximity.
w final , t = w complexity , t · w time , t = ( 1 − S E t ) · exp − λ · t − 1 H − 1
Based on this dynamic weight, we construct the final UP loss function. Given the model’s predicted sequence Y ^ = ( y ^ 1 , y ^ 2 , … , y ^ H ) , the UP loss is calculated as
L UP = 1 H ∑ t = 1 H w final , t · ( y t − y ^ t ) 2
This loss function, while operationally simple, profoundly alters the model’s learning paradigm. It guides the model to preferentially allocate its finite modeling capacity to recent and regular portions of the signal, while permitting greater error on distant or inherently noisy segments. This dual-dimension adaptive regularization aims to produce a model that is not only more accurate but also more aligned with the fundamental principles of inductive inference in its very design.

3.4. Algorithmic Implementation and Computational Flow

To ensure methodological clarity and reproducibility, we formalize the complete training procedure, which integrates the Unpredictability Perception loss with a backbone forecasting model M. To clarify the integration of our method, let D = { ( X , Y ) } denote the dataset, where X is the lookback window and Y is the forecast horizon. Let f θ ( · ) represent the backbone forecasting model (e.g., TimeMixer) parameterized by θ . Our UP loss replaces the standard loss function during the backward pass. The complete training procedure, detailing the interaction between the backbone model’s forward pass and the UP loss’s weight calculation, is formalized in Algorithm 1.
Algorithm1: Training Procedure with Unpredictability Perception (UP) Loss
Require: Training dataset D = { ( X i , Y i ) } i = 1 N , Forecasting Model f θ with parameters θ
Require: Hyperparameters: Learning rate η , Local window size P local , Temporal decay factor λ , Max epochs E
Ensure:  Optimized model parameters θ *
  1:
  Initialize θ randomly
  2:
  for epoch e = 1 to E do
  3:
      for each batch ( X , Y ) sampled from D  do
  4:
           Y ^ ← f θ ( X ) Forward pass
  5:
           H ← Length ( Y )
  6:
          Initialize weight vector W final ∈ R H
  7:
          for  t = 1 to H do
  8:
               Y t local ← ExtractLocalWindow ( Y , t , P local )
  9:
               S E t ← ComputeSpectralEntropy ( Y t local )
10:
               w complexity , t ← 1 − S E t
11:
               w time , t ← exp ( − λ · t − 1 H − 1 )
12:
               w final [ t ] ← w complexity , t · w time , t
13:
          end for
14:
           L UP ← 1 H ∑ t = 1 H w final [ t ] · ( Y [ t ] − Y ^ [ t ] ) 2
15:
           θ ← θ − η · ∇ θ L UP Update model parameters (Backpropagation)
16:
      end for
17:
end for
18:
return  θ *
The framework is designed to be model-agnostic, allowing M to be any forecasting architecture.The algorithm commences by obtaining a prediction Y ^ from the model’s forward pass. It then iterates through the prediction horizon H to compute the two core weight components for each time step t. The complexity weight w complexity , t is derived by analyzing the local spectral entropy S E t of the ground-truth signal Y, as detailed in Section 2.2. Concurrently, the temporal weight w time , t is calculated based on the exponential decay function described in Section 2.3. These two weights are multiplicatively fused to create the final weight w final , t . This final weight dynamically scales the point-wise squared error. The mean of these weighted errors constitutes the final L UP , which is then used for backpropagation to optimize the model’s parameters. Key hyperparameters in this process include the local window size P local for spectral entropy calculation and the decay rate λ for temporal weighting. We set P local = 24 and λ = 0.5 as a robust default.

3.5. Computational Complexity Analysis

A critical consideration for any novel loss function is the potential introduction of computational overhead. The standard MSE loss has a time complexity of O ( H ) per sample, where H is the prediction horizon. Our proposed UP loss introduces two additional weight calculation steps. The temporal decay weight involves a constant-time operation O ( 1 ) per time step. The complexity weight requires computing the Spectral Entropy, which relies on the Fast Fourier Transform (FFT) applied to a local window of size P local . The complexity of the FFT operation is O ( P local log P local ) . Consequently, the total complexity of the UP loss calculation for one time step is O ( P local log P local ) .
It is important to note that since the calculation of weights depends solely on the ground truth Y, which remains static during training, these weights can theoretically be pre-computed offline, reducing the online training overhead to zero. However, to evaluate the worst-case scenario, we assessed the runtime cost when computing weights on-the-fly during training. We conducted a comparative experiment between the original TimeMixer and our proposed UP-TimeMixer under identical hardware and software configurations (NVIDIA RTX 4090 GPU, Batch Size 32, Input/Output Horizon 96). As detailed in Table 1, the theoretical time complexity of the UP loss calculation for one time step is O ( P local log P local ) . In practice, this introduces a marginal overhead during the training phase. The average training time per epoch increased from 11.23 ± 0.15 s (TimeMixer) to 11.68 ± 0.18 s (UP-TimeMixer), representing a relative increase of approximately 4.0%. This slight increase is attributable to the on-the-fly FFT computation for spectral entropy. Crucially, however, the proposed UP loss functions exclusively as an optimization objective and is detached from the model architecture after training. Consequently, the inference time remains strictly identical ( 1.42 ± 0.03 ms per batch), and the model parameter count is unchanged. This confirms that our method improves predictive accuracy without imposing any additional computational burden during the deployment phase, making it highly suitable for latency-sensitive applications.
Table 1. Comparison of computational complexity and actual running time on the ETTh1 dataset (Input-96-Predict-96). Inference time is measured per batch (size = 32).

4. Experiments and Results

4.1. Datasets

To comprehensively evaluate the effectiveness and generalization capability of our proposed framework, we selected eight public benchmark datasets covering critical domains such as energy, meteorology, and traffic. These datasets exhibit significant diversity in their temporal characteristics, providing an ideal testbed for validating the core ideas of our method.
The ETT series (ETTh1, ETTh2, ETTm1, and ETTm2) [48] records key metrics of power transformers. This data is known for its clear seasonal periodicity but is also interspersed with substantial random noise introduced by grid load fluctuations and environmental factors. This coexistence of periodicity and randomness constitutes a perfect scenario for testing a model’s ability to distinguish between predictable patterns and unpredictable disturbances.
The Weather dataset contains 21 meteorological indicators. A meteorological system is inherently a complex chaotic system, with its long-term dynamics being theoretically unpredictable. In the short term, however, it exhibits discernible patterns governed by geophysical laws. This dataset offers an opportunity to test our model’s performance in a complex dynamical system with intrinsic prediction limits.
The Electricity dataset records the hourly electricity consumption of 321 clients. Its dynamics are dominated by strong periodic patterns determined by social production and life, such as daily and weekly rhythms. However, the aggregation of random behaviors from a massive number of individual users injects continuous random disturbances into this macroscopic regularity, which are difficult to model from historical data alone.
The Traffic dataset describes the occupancy rates of road sensors in the San Francisco Bay Area. Traffic flow data is a typical dual-characteristic signal; it contains predictable peak and off-peak patterns driven by commuting regularities, yet it is frequently impacted by sudden, contingent events like traffic accidents, which produce unforeseen sharp fluctuations.
The Solar-Energy dataset [49] records the power generation of 137 solar power plants. Photovoltaic power generation is directly subject to highly variable meteorological factors such as cloud cover and light intensity, causing the signal to exhibit extreme volatility and intermittency. The signal contains a large number of inherently unpredictable segments driven by the randomness of weather systems.
These datasets, sourced from various sensor networks, all exhibit an interweaving of deterministic patterns and stochastic fluctuations, providing a rigorous empirical foundation for evaluating the effectiveness of the proposed framework.

4.2. Experimental Setup

To verify the practical value of our proposed framework, we chose to integrate and validate it on TimeMixer, one of the current state-of-the-art models. As a pure MLP-based architecture, TimeMixer has achieved SOTA performance on numerous time series forecasting benchmarks, surpassing many structurally complex Transformer models. The pure MLP architecture of TimeMixer lacks sequence-specific inductive biases, making it particularly sensitive to the quality of the supervision signal. Our proposed optimization framework provides the precise guidance it needs to distinguish between deterministic patterns and random noise.
In all experiments, we evaluated all models on four progressively longer prediction tasks of 96, 192, 336, and 720 steps. To ensure a fair comparison and reproducibility, our experimental setup strictly aligns with the configurations of the baseline TimeMixer model. Specifically, all models were implemented using PyTorch 2.4.1 and trained on a server equipped with two NVIDIA RTX 4090 GPUs. Following the official implementation of TimeMixer, we utilized the Adam optimizer with a learning rate adjusted between 10 − 3 and 10 − 2 depending on the dataset characteristics, and a batch size ranging from 32 to 128. An early stopping mechanism with a patience of 10 epochs was employed to prevent overfitting. The local window size P local was set to 24 for hourly datasets. The calculation of Spectral Entropy relies on the frequency domain representation derived via the Discrete Fourier Transform (DFT). A window size of 24 provides the minimal sufficient spectral resolution (approximately 12 effective frequency bins up to the Nyquist limit) to generate a statistically stable entropy estimate. Smaller windows (e.g., 12 points) result in a sparse spectrum, leading to mathematically unstable entropy values that fail to distinguish noise from signal. Conversely, significantly larger windows risk smoothing out the local volatility that our method aims to capture. Furthermore, this setting maintains consistency with the moving average kernel size used in the backbone TimeMixer model, ensuring structural alignment. For the temporal decay factor, we set λ = 0.5 as a fixed default across all experiments. This parameter alignment strategy ensures that the performance improvements originate from the proposed method’s mechanism rather than extensive hyperparameter tuning. The source code will be made publicly available upon acceptance to promote uncertainty awareness in deep learning models.

4.3. Quantitative Comparison

Detailed experimental results are presented in Table 2 and Table 3. To thoroughly assess the performance of our method, we divided the comparison models into two groups based on their publication dates: the first group includes recent state-of-the-art models that are contemporary competitors, while the second group consists of influential and representative models from the past several years. The experimental data shows that our method achieves stable and significant performance improvements over the original TimeMixer model and sets new state-of-the-art performance records in the vast majority of test scenarios.
Table 2. Performance comparison with contemporary state-of-the-art models for time series forecasting. The best results are in bold, and the second best are underlined.
Table 3. Performance comparison with representative models from the last five years for time series forecasting. The best results are in bold, and the second best are underlined.
The performance gains achieved by our method are particularly prominent on the ETT series and Solar-Energy datasets, which are characterized by the coexistence of periodicity and noise. The signals in these datasets contain a large amount of endogenous randomness. The traditional MSE loss forces the model to fit this unlearnable noise, thereby impairing its ability to capture core periodic patterns. Our UP loss function, through its complexity-aware mechanism, effectively reduces the supervision weight on these highly random signal segments. This allows the model to focus more of its modeling capacity on learning the underlying patterns that have genuine generalization value, leading to a substantial improvement in prediction accuracy.
On the Weather, Electricity, and Traffic datasets, the results exhibit different characteristics. The signal dynamics in these datasets are dominated by energy-concentrated, low-frequency periodic components. This causes the local spectral entropy of the signal to remain at a low level for most of the time, thereby weakening the differential adjustment capability of the complexity weight w c o m p l e x i t y , t in UP-TimeMixer and making the supervision signal it provides somewhat approximate that of standard MSE. However, the prior knowledge that prediction difficulty increases with the time step still holds true in these tasks. Therefore, the temporal decay weight w t i m e , t in UP-TimeMixer plays a key regularization role. Especially in long-term forecasting tasks, by reducing the supervision strength on distant future points, it effectively prevents the model from overfitting in regions of higher uncertainty. This explains why, although the advantage of our method is less pronounced on these datasets, it still consistently outperforms others in most long-term forecasting scenarios. It also demonstrates the complementarity and robustness of the two weight components designed within the UP-TimeMixer.
Ultimately, the experimental results indicate that guiding a model’s learning process through a more refined optimization objective is an equally effective, and perhaps even more efficient, path to improving prediction performance than continually designing more complex model architectures. A simple MLP model, when equipped with our unpredictability perception framework, can systematically outperform numerous structurally complex Transformer models. This offers an inspiring new perspective for the future development of the time series forecasting field.

4.4. Visualization Comparison

A qualitative analysis of the forecast results, as visually presented in Figure 3, reveals that the uniform training objectives of conventional models result in characteristic failure modes. For instance, earlier Transformer models like Informer and Autoformer tend to produce erratic, high-amplitude oscillations decorrelated from the ground truth. This suggests that forcing a model to equally fit all future points, including inherently unpredictable ones, can impair the learning of stable long-term patterns. Models with strong smoothing biases, such as DLinear and TimesNet, exhibit another reaction to this uniform objective by systematically dampening the amplitude of sharp peaks, effectively prioritizing more predictable low-frequency trends over volatile, high-frequency details. Even robust architectures like PatchTST may occasionally produce spurious, high-amplitude patterns, a behavior suggesting that a uniform training objective may cause overfitting to locally complex yet globally insignificant features. The baseline TimeMixer successfully captures the signal’s primary dynamics and periodicities, but its prediction curve contains excessive high-frequency oscillations, indicating its uniform training objective may have misinterpreted random noise as an intrinsic signal pattern to be fitted. Our proposed UP-TimeMixer, however, visibly overcomes this specific limitation, yielding a prediction that is both structurally accurate and significantly less noisy. By encoding signal predictability and temporal distance into its loss function, our method mitigates the inherent limitations of uniform supervision. The model’s capacity to precisely approximate both low-frequency trends and high-frequency transients demonstrates the efficacy of this adaptive training paradigm. By concentrating modeling resources on learnable signal components, it effectively prevents overfitting to stochastic, unpredictable dynamics.
Figure 3. Visual comparison of different time series prediction models on a signal segment from the ETTh1 dataset. The grey shaded region denotes the lookback window (historical input), while the dashed vertical line marks the onset of the forecasting horizon. Ground truth is shown in black, with model predictions displayed in color.

4.5. Ablation Study

To evaluate the contribution of each component of the proposed UP Loss, we conduct an ablation study across all eight datasets used in our experiments. We compare the baseline model (using only the standard MSE loss) with three variants: one using only the temporal weight w time , one using only the complexity weight w complexity , and the full UP Loss that combines both components. The results, summarized in Table 4, show that the full UP-TimeMixer consistently outperforms the baseline MSE in most cases, confirming the effectiveness of re-weighting the loss function. The most significant improvements are observed in datasets such as ETTh1, ETTm2, and Solar-Energy, where the full UP Loss reduces the average MSE and MAE by a small but consistent margin.
Table 4. Ablation study of the UP Loss components on all datasets. We compare the baseline (MSE) against variants using only temporal weight ( w time ), only complexity weight ( w complexity ), and the full UP Loss. Best results are in bold.
However, the impact of the individual components of the UP Loss varies across datasets. On datasets like ETTh2 and ETTm1, where horizon-dependent uncertainty is more pronounced, the temporal weight w time contributes more to the reduction in error than the complexity weight w complexity . For instance, on ETTh2 at the 720-step horizon, using only w time yields a better MSE than using only w complexity . On the other hand, for datasets with more regular periodicity, such as Solar-Energy and ETTh1, the complexity weight w complexity proves to be more beneficial. For example, on ETTh1 at the 96-step horizon, the complexity-based model performs slightly better than the temporal-only variant. Interestingly, the full UP Loss does not always outperform the baseline on datasets with some predictable signals, such as Weather and Electricity. In these cases, the addition of re-weighting does not result in substantial gains, and the baseline MSE performs comparably or even slightly better.
To visualize the contribution of our proposed Unpredictability Perception loss, we compare the empirical distributions of mean squared errors between the baseline TimeMixer and TimeMixer+UP on the test sets. Figure 4 presents histograms overlaid with kernel density estimates (KDE) for the baseline TimeMixer and TimeMixer+UP, derived from per-instance MSE values aggregated over the prediction horizons (96, 192, 336, 720 steps). In datasets such as ETTh1, ETTh2, ETTm1, ETTm2, and Solar-Energy, where TimeMixer+UP yields lower average MSE, the distributions exhibit modes centered at reduced error levels, often accompanied by narrower spreads and diminished high-error tails, reflecting improved generalization and robustness to stochastic signal components. Overall, these visualizations affirm the efficacy of the UP loss in modulating supervision to match the intrinsic predictability of sensor signals, thereby fostering more stable and accurate forecasts in critical applications like power grid management and urban traffic optimization.
Figure 4. Histograms and kernel density estimates of the mean squared error distributions for the baseline TimeMixer model (teal) and TimeMixer with Unpredictability Perception loss (orange) across all benchmark datasets.
This ablation study demonstrates that the full UP Loss generally improves forecasting performance by effectively re-weighting the loss to prioritize more predictable segments. The individual components (temporal and complexity weights) have different impacts depending on the dataset, with the temporal weight being more effective on datasets with higher uncertainty and the complexity weight being more useful on datasets with regular patterns. These findings validate the proposed loss function and highlight its adaptability across different types of time series data.
To rigorously validate the selection of the hyperparameter λ = 0.5 , we conducted a comprehensive parameter sensitivity analysis on the representative ETTh1 dataset. We evaluated the model’s performance by varying λ across the set { 0.1 , 0.3 , 0.5 , 0.7 , 0.9 } while keeping all other parameters fixed. The results, summarized in Table 5, reveal a distinct pattern. When λ is small, the temporal weight function approximates a uniform distribution, resulting in a supervision signal similar to the standard MSE; consequently, the performance improvement over the baseline is marginal. As λ increases to 0.5, the loss function effectively down-weights the supervision for distant, high-uncertainty time steps, allowing the model to prioritize learnable short-term dynamics, which leads to the lowest average MSE of 0.436. However, as λ continues to increase beyond 0.7, the excessive decay overly relaxes the constraints on long-term predictions. This causes the model to essentially neglect the supervision for the distant future, resulting in a degradation of overall forecasting accuracy. This empirical evidence confirms that λ = 0.5 provides the optimal equilibrium between enforcing prediction accuracy and accommodating temporal uncertainty.
Table 5. Parameter sensitivity analysis of λ on the ETTh1 dataset (Avg. MSE/MAE over 4 horizons).

5. Conclusions

This study introduced an Unpredictability Perception Framework to address the inherent limitations of the mean squared error loss function in time series forecasting. Our approach redefines the model’s optimization objective through an innovative loss function that dynamically computes supervision weights based on a dual assessment of the forecasting task’s intrinsic unpredictability. This assessment unifies a posterior analysis of signal content randomness via local spectral entropy with an a priori consideration of temporal distance via exponential decay. This mechanism guides the model to prioritize learning high-certainty deterministic patterns while reducing the fitting intensity on inherently random or distant uncertain events. By applying this framework to the advanced TimeMixer model, experimental results across multiple public benchmarks demonstrate that this optimization at the loss function level systematically enhances prediction accuracy without any architectural modifications. The findings of this research reveal that matching the supervision strength to the intrinsic predictability of the forecasting task provides a simple yet effective path toward enhancing the robustness and accuracy of deep learning forecasting models, offering a new optimization perspective for sensor network data analysis.

Author Contributions

Conceptualization, Y.Z.; Methodology, Y.W.; Validation, Y.J.; Resources, R.C.; Project administration, J.D.; Funding acquisition, L.Z. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Data Availability Statement

The ETT, Electricity, Traffic, Weather, and Solar-Energy datasets can be found at the following repositories: https://github.com/zhouhaoyi/ETDataset (access on 25 November 2025) and https://github.com/thuml/Autoformer (access on 25 November 2025).

Conflicts of Interest

Lin Zhao, Jian Dong and Ruojing Chen are employees of Institute of Economics and Technology of State Grid Liaoning Electric Power Co., Ltd. The paper reflects the views of the scientists and not the company.

References

  1. Li, M.; Yang, M.; Chen, S.; Li, H.; Xing, G.; Li, S. FCP-Former: Enhancing Long-Term Multivariate Time Series Forecasting with Frequency Compensation. Sensors 2025, 25, 5646. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  2. Xia, M.; Shao, H.; Ma, X.; de Silva, C.W. A Stacked GRU-RNN-Based Approach for Predicting Renewable Energy and Electricity Load for Smart Grid Operation. IEEE Trans. Ind. Inform. 2021, 17, 7050–7059. [Google Scholar] [CrossRef] [Scilit]
  3. Qiao, W.; Li, Z.; Liu, W.; Liu, E. Fastest-growing source prediction of US electricity production based on a novel hybrid model using wavelet transform. Int. J. Energy Res. 2022, 46, 1766–1788. [Google Scholar] [CrossRef] [Scilit]
  4. Imani, M.H.; Bompard, E.; Colella, P.; Huang, T. Forecasting Electricity Price in Different Time Horizons: An Application to the Italian Electricity Market. IEEE Trans. Ind. Appl. 2021, 57, 5726–5736. [Google Scholar] [CrossRef] [Scilit]
  5. Liu, H.; Wang, H.; Yu, M.; Wang, Y.; Luo, Y. Long Short-Term Memory–Model Predictive Control Speed Prediction-Based Double Deep Q-Network Energy Management for Hybrid Electric Vehicle to Enhanced Fuel Economy. Sensors 2025, 25, 2784. [Google Scholar] [CrossRef] [Scilit]
  6. Park, M.J.; Yang, H.S. Comparative Study of Time Series Analysis Algorithms Suitable for Short-Term Forecasting in Implementing Demand Response Based on AMI. Sensors 2024, 24, 7205. [Google Scholar] [CrossRef] [Scilit]
  7. Lu, X.; Qiu, J.; Lei, G.; Zhu, J. An Interval Prediction Method for Day-Ahead Electricity Price in Wholesale Market Considering Weather Factors. IEEE Trans. Power Syst. 2024, 39, 2558–2569. [Google Scholar] [CrossRef] [Scilit]
  8. Li, F.; Wang, X.; Bao, X.; Wang, Z.; Li, R. Energy Management Strategy for Fuel Cell Vehicles Based on Online Driving Condition Recognition Using Dual-Model Predictive Control. Sensors 2024, 24, 7647. [Google Scholar] [CrossRef] [Scilit]
  9. Pourbunthidkul, S.; Pahaisuk, N.; Laon, P.; Houngkamhang, N.; Phasukkit, P. An Enhanced Cascaded Deep Learning Framework for Multi-Cell Voltage Forecasting and State of Charge Estimation in Electric Vehicle Batteries Using LSTM Networks. Sensors 2025, 25, 3788. [Google Scholar] [CrossRef] [Scilit]
  10. Sang, L.; Xu, Y.; Long, H.; Hu, Q.; Sun, H. Electricity Price Prediction for Energy Storage System Arbitrage: A Decision-Focused Approach. IEEE Trans. Smart Grid 2022, 13, 2822–2832. [Google Scholar] [CrossRef] [Scilit]
  11. Kiouvrekis, Y.; Givisis, I.; Panagiotakopoulos, T.; Tsilikas, I.; Ploussi, A.; Spyratou, E.; Efstathopoulos, E.P. A Comparative Analysis of Explainable Artificial Intelligence Models for Electric Field Strength Prediction over Eight European Cities. Sensors 2025, 25, 53. [Google Scholar] [CrossRef] [Scilit]
  12. Yan, J.; Li, H.; Bai, Y.; Liu, J.; Lv, H.; Bai, Y. Timestamp-Guided Knowledge Distillation for Robust Sensor-Based Time-Series Forecasting. Sensors 2025, 25, 4590. [Google Scholar] [CrossRef] [Scilit]
  13. Zhang, C.; Fu, Y. Probabilistic Electricity Price Forecast with Optimal Prediction Interval. IEEE Trans. Power Syst. 2024, 39, 442–452. [Google Scholar] [CrossRef] [Scilit]
  14. Song, N.; Chang-Silva, R.; Lee, K.; Park, S. Dynamic Model Selection in a Hybrid Ensemble Framework for Robust Photovoltaic Power Forecasting. Sensors 2025, 25, 4489. [Google Scholar] [CrossRef] [Scilit]
  15. Wang, C.; Wang, Y.; Ding, Z.; Zheng, T.; Hu, J.; Zhang, K. A Transformer-Based Method of Multienergy Load Forecasting in Integrated Energy System. IEEE Trans. Smart Grid 2022, 13, 2703–2714. [Google Scholar] [CrossRef] [Scilit]
  16. Wang, Y.; Zhao, Y. Optimal Transport with Mamba for Multimodal Inertial Signal Enhancement. IEEE Trans. Ind. Inform. 2025, 21, 7023–7034. [Google Scholar] [CrossRef] [Scilit]
  17. Nie, Y.; H. Nguyen, N.; Sinthong, P.; Kalagnanam, J. A Time Series is Worth 64 Words: Long-term Forecasting with Transformers. In Proceedings of the Eleventh International Conference on Learning Representations, Kigali, Rwanda, 1–5 May 2023. [Google Scholar]
  18. Liu, Y.; Hu, T.; Zhang, H.; Wu, H.; Wang, S.; Ma, L.; Long, M. iTransformer: Inverted Transformers Are Effective for Time Series Forecasting. In Proceedings of the Twelfth International Conference on Learning Representations, Vienna, Austria, 7–11 May 2024. [Google Scholar]
  19. Wang, S.; Wu, H.; Shi, X.; Hu, T.; Luo, H.; Ma, L.; Zhang, J.Y.; Zhou, J. TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting. In Proceedings of the Twelfth International Conference on Learning Representations, Vienna, Austria, 7–11 May 2024. [Google Scholar]
  20. Wang, Y.; Xu, J.; Zhao, Y. Wavelet Encoding Network for Inertial Signal Enhancement via Feature Supervision. IEEE Trans. Ind. Inform. 2024, 20, 12924–12934. [Google Scholar] [CrossRef] [Scilit]
  21. Yunus, K.; Thiringer, T.; Chen, P. ARIMA-Based Frequency-Decomposed Modeling of Wind Speed Time Series. IEEE Trans. Power Syst. 2016, 31, 2546–2556. [Google Scholar] [CrossRef] [Scilit]
  22. Chen, P.; Pedersen, T.; Bak-Jensen, B.; Chen, Z. ARIMA-Based Time Series Model of Stochastic Wind Power Generation. IEEE Trans. Power Syst. 2010, 25, 667–676. [Google Scholar] [CrossRef] [Scilit]
  23. Taylor, J.W. Short-Term Load Forecasting with Exponentially Weighted Methods. IEEE Trans. Power Syst. 2012, 27, 458–464. [Google Scholar] [CrossRef] [Scilit]
  24. Connor, J.; Martin, R.; Atlas, L. Recurrent neural networks and robust time series prediction. IEEE Trans. Neural Netw. 1994, 5, 240–254. [Google Scholar] [CrossRef] [Scilit]
  25. Han, M.; Xi, J.; Xu, S.; Yin, F.L. Prediction of chaotic time series based on the recurrent predictor neural network. IEEE Trans. Signal Process. 2004, 52, 3409–3416. [Google Scholar] [CrossRef]
  26. Yan, W. Toward Automatic Time-Series Forecasting Using Neural Networks. IEEE Trans. Neural Netw. Learn. Syst. 2012, 23, 1028–1039. [Google Scholar] [CrossRef] [Scilit]
  27. Bi, J.; Zhang, X.; Yuan, H.; Zhang, J.; Zhou, M. A Hybrid Prediction Method for Realistic Network Traffic with Temporal Convolutional Network and LSTM. IEEE Trans. Autom. Sci. Eng. 2022, 19, 1869–1879. [Google Scholar] [CrossRef] [Scilit]
  28. Liu, S.; Ji, H.; Wang, M.C. Nonpooling Convolutional Neural Network Forecasting for Seasonal Time Series with Trends. IEEE Trans. Neural Netw. Learn. Syst. 2020, 31, 2879–2888. [Google Scholar] [CrossRef] [Scilit]
  29. Liu, H.; Dong, Z.; Jiang, R.; Deng, J.; Deng, J.; Chen, Q.; Song, X. Spatio-Temporal Adaptive Embedding Makes Vanilla Transformer SOTA for Traffic Forecasting. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, New York, NY, USA, 21–25 October 2023; CIKM’23. pp. 4125–4129. [Google Scholar] [CrossRef] [Scilit]
  30. Cheng, Y.; Qv, J.; Feng, K.; Han, T. A Bayesian adversarial probsparse Transformer model for long-term remaining useful life prediction. Reliab. Eng. Syst. Saf. 2024, 248, 110188. [Google Scholar] [CrossRef] [Scilit]
  31. Wu, H.; Xu, J.; Wang, J.; Long, M. Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting. In Proceedings of the Advances in Neural Information Processing Systems, Virtual, 6–14 December 2021; Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P., Vaughan, J.W., Eds.; Curran Associates, Inc.: Red Hook, NY, USA, 2021; Volume 34, pp. 22419–22430. [Google Scholar]
  32. Zhou, T.; Ma, Z.; Wen, Q.; Wang, X.; Sun, L.; Jin, R. FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting. In Proceedings of the 39th International Conference on Machine Learning, Baltimore, MD, USA, 17–23 July 2022; Chaudhuri, K., Jegelka, S., Song, L., Szepesvari, C., Niu, G., Sabato, S., Eds.; Proceedings of Machine Learning Research. PMLR: New York, NY, USA, 2022; Volume 162, pp. 27268–27286. [Google Scholar]
  33. Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; et al. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In Proceedings of the International Conference on Learning Representations, Vienna, Austria (Virtual), May 3–7 2021. [Google Scholar]
  34. Zhang, Y.; Yan, J. Crossformer: Transformer Utilizing Cross-Dimension Dependency for Multivariate Time Series Forecasting. In Proceedings of the Eleventh International Conference on Learning Representations, Kigali, Rwanda, 1–5 May 2023. [Google Scholar]
  35. Zhou, N.; Zeng, H.; Zhou, J. DLinear-Based Prediction of the RUL of PEMFC. In Proceedings of the 2024 4th International Conference on Consumer Electronics and Computer Engineering (ICCECE), Guangzhou, China, 12–14 January 2024; pp. 221–224. [Google Scholar] [CrossRef] [Scilit]
  36. Wu, H.; Hu, T.; Liu, Y.; Zhou, H.; Wang, J.; Long, M. TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis. In Proceedings of the Eleventh International Conference on Learning Representations, Kigali, Rwanda, 1–5 May 2023. [Google Scholar]
  37. Wang, H.; Peng, J.; Huang, F.; Wang, J.; Chen, J.; Xiao, Y. MICN: Multi-scale Local and Global Context Modeling for Long-term Series Forecasting. In Proceedings of the Eleventh International Conference on Learning Representations, Kigali, Rwanda, 1–5 May 2023. [Google Scholar]
  38. Kendall, A.; Gal, Y. What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? In Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA, 4–9 December 2017; Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R., Eds.; Curran Associates, Inc.: Red Hook, NY, USA, 2017; Volume 30. [Google Scholar]
  39. Salinas, D.; Flunkert, V.; Gasthaus, J.; Januschowski, T. DeepAR: Probabilistic forecasting with autoregressive recurrent networks. Int. J. Forecast. 2020, 36, 1181–1191. [Google Scholar] [CrossRef] [Scilit]
  40. Gal, Y.; Ghahramani, Z. Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. In Proceedings of the 33rd International Conference on Machine Learning, New York, NY, USA, 20–22 June 2016; Balcan, M.F., Weinberger, K.Q., Eds.; Proceedings of Machine Learning Research. PMLR: New York, NY, USA, 2016; Volume 48, pp. 1050–1059. [Google Scholar]
  41. Runge, J.; Bathiany, S.; Bollt, E.; Camps-Valls, G.; Coumou, D.; Deyle, E.; Glymour, C.; Kretschmer, M.; Mahecha, M.D.; Muñoz-Marí, J.; et al. Inferring causation from time series in Earth system sciences. Nat. Commun. 2019, 10, 2553. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  42. Han, T.; Gu, X.; Li, D.; Chen, K.; Cong, R.G.; Zhao, L.T.; Wei, Y.M. Causal neural network for carbon prices probabilistic forecasting. Appl. Energy 2025, 397, 126343. [Google Scholar] [CrossRef] [Scilit]
  43. Wang, Y.; Zhao, Y. HEROS-GAN: Honed-Energy Regularized and Optimal Supervised GAN for Enhancing Accuracy and Range of Low-Cost Accelerometers. In Proceedings of the AAAI Conference on Artificial Intelligence, Philadelphia, PA, USA, 25 February–4 March 2025; Volume 39, pp. 21348–21356. [Google Scholar]
  44. Mendes-Santos, T.; Schmitt, M.; Angelone, A.; Rodriguez, A.; Scholl, P.; Williams, H.J.; Barredo, D.; Lahaye, T.; Browaeys, A.; Heyl, M.; et al. Wave-Function Network Description and Kolmogorov Complexity of Quantum Many-Body Systems. Phys. Rev. X 2024, 14, 021029. [Google Scholar] [CrossRef] [Scilit]
  45. Zheng, J.; Ying, W.; Tong, J.; Li, Y. Multiscale three-dimensional Holo–Hilbert spectral entropy: A novel complexity-based early fault feature representation method for rotating machinery. Nonlinear Dyn. 2023, 111, 10309–10330. [Google Scholar] [CrossRef] [Scilit]
  46. Wang, Y.; Zhao, Y. General pre-trained inertial signal feature extraction based on temporal memory fusion. Inf. Fusion 2025, 123, 103274. [Google Scholar] [CrossRef] [Scilit]
  47. Li, Y.; Kind, T.; Folz, J.S.; Vaniya, A.; Mehta, S.S.; Fiehn, O. Spectral entropy outperforms MS/MS dot product similarity for small-molecule compound identification. Nat. Methods 2021, 18, 1524–1531. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  48. Zhou, H.; Zhang, S.; Peng, J.; Zhang, S.; Li, J.; Xiong, H.; Zhang, W. Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, Virtual, 19–21 May 2021; Volume 35, pp. 11106–11115. [Google Scholar]
  49. Lai, G.; Chang, W.C.; Yang, Y.; Liu, H. Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks. In Proceedings of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, Ann Arbor, MI, USA, 8–12 July 2018; pp. 95–104. [Google Scholar]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Article Metrics

Citations

Article Access Statistics

Multiple requests from the same IP address are counted as one view.