Forecasting Aggregated Hybrid Order Curves in the Day-Ahead Electricity Market via FPCA and Gradient Boosting ^†

Abstract

Accurate modelling of price formation in electricity markets requires capturing the underlying supply and demand structure rather than predicting scalar prices alone. This structure is based on the participants’ bidding process, who submit energy volume offers in the form of block orders and hybrid curve orders. This work addresses the problem of day-ahead forecasting of aggregated hybrid supply curves in the Greek DAM at a 15 min resolution. A hybrid methodology is proposed that combines FPCA, which reduces the high-dimensional curve space to a small set of latent factors, with gradient-boosted forecasts that recursively use models driven by calendar effects, lagged latent dynamics, and exogenous variables (load, RES, and cross-border capacities). Forecasted curves are reconstructed and refined via band-specific residual correction and feasibility constraints. A baseline FPCA backed by ARIMA residual forecasting is used for comparison. The approach is validated for January–March 2026 using a strict day-ahead setup. Evaluation over selected days between January and March 2026 demonstrates that the proposed methodology generally improves forecasting accuracy and maintains stronger performance under evolving market conditions. Results demonstrate that the proposed framework effectively captures curve dynamics, while emphasising the importance of system-state information under changing conditions.

1. Introduction

1.1. Aggregated Supply Curves in the Electricity Market

Electricity markets operate through complex interactions between supply and demand, where prices emerge from the intersection of bidding curves submitted by market participants aggregated by MTU, regardless of energy-producing technology and type of demand. In the European DAM, including the Greek market, orders submitted by market participants are aggregated and cleared through a centralised algorithm that matches supply and demand to maximise social welfare, as defined within the European market coupling framework [1,2].

In this context, aggregated hybrid supply curves play a central role. Each hybrid curve represents the cumulative offered energy from all energy suppliers as a function of price and reflects the system’s instantaneous marginal cost structure, combining renewable generation, thermal units, and cross-border exchanges. These curves are inherently high-dimensional, monotonic, and time-dependent, and their evolution encapsulates the effects of system conditions such as load demand, renewable penetration, and network constraints. Furthermore, market mechanisms such as block orders introduce non-convexities, shifting the effective supply curve and directly affecting price formation.

1.2. Literature Review

Classical approaches rely on statistical and econometric models, while more recent work employs machine learning techniques. A comprehensive review by Rafał Weron [3] highlights the evolution of these methods and emphasises the inherent difficulty of capturing price spikes, regime changes, and nonlinear dependencies in electricity markets. Structural models explicitly incorporate supply and demand mechanisms, offering improved interpretability and economic consistency [4]. Despite their predictive capability, such models treat price as an isolated variable and do not explicitly account for the structural mechanism of market clearing.

To address this limitation, recent research has explored the use of FDA techniques to high-dimensional temporal forecasting problems, demonstrating that complex curve structures can be efficiently represented through a small number of latent functional components [5,6], enabling the transformation of a complex functional forecasting problem into a multivariate time-series problem. Recent work has also explored direct forecasting of supply and demand curves, such as the X-model proposed by Ziel and Steinert [7], which reconstructs market clearing through the interaction of sale and purchase curves. In particular, FPCA provides an efficient framework for representing time-varying curves in a low-dimensional latent space. Applications of FPCA in energy systems demonstrate its ability to capture dominant modes of variation while preserving the structural properties of the data [8].

However, baseline FPCA-based approaches that rely on complementary linear time-series models, such as ARIMA, for forecasting the latent factors are limited in their ability to capture nonlinear relationships and the influence of exogenous variables. Electricity markets are driven by a wide range of external factors, including load demand, renewable generation, cross-border capacity, and system constraints, which introduce nonlinear and regime-dependent behaviour. This work extends previous research [9], which indicated that incorporating exogenous variables such as load and RES significantly improves forecasting performance, while also highlighting the limitations of purely linear latent dynamics.

Recent advances in machine learning, particularly ensemble methods such as gradient boosting, offer a flexible alternative for modelling nonlinear dependencies and interactions between features [10]. These methods have been successfully applied in forecasting problems where complex relationships between predictors and target variables are present, a frequent occurrence in electricity markets [11].

To address these challenges, this paper proposes a hybrid FPCA–machine learning framework for the day-ahead forecasting of aggregated hybrid supply curves at a 15 min resolution. The methodology combines the structural advantages of FPCA with the predictive flexibility of machine learning. Specifically, FPCA is used to reduce the dimensionality of the curve space, while the resulting latent factors are forecast using gradient boosting models driven by a rich set of features, including calendar effects, lagged dynamics, and exogenous variables such as electricity load, renewable generation, and ATCs between Greece and the surrounding countries, Albania, Italy, Bulgaria, Türkiye, and N. Macedonia. Forecasting is performed in a day-ahead framework, in which the 96 MTUs for a selected delivery day are predicted using only past information, ensuring consistency with real market operations.

2. Materials and Methods

2.1. Data Repositories

For data training, past aggregated hybrid curves that are publicly available from the Greek NEMO, HEnEx S.A., were used. Day-ahead RES and load forecasts, and published ATCs between Greece and the surrounding countries from the Greek TSO, IPTO S.A., were used as features.

2.2. Curve Decomposition with FPCA

In order to forecast the aggregated hybrid supply curves in the Greek DAM at a 15 min MTU resolution, both models subject each curve to decomposition via the FPCA method. The hybrid curves are defined on a fixed price grid $\left\{P_1, \dots , P_M\right\}$ and represented as:

$$y_t=\left[y_{t,1}, \dots , y_{t,M}\right]$$

(1)

where $y_{t,j}$ is the cumulative offered volume in MW at the price level $P_j$, while t indexes time in 15 min intervals. The price grid was derived from the published files that contain the aggregated curve data, resulting in the adoption of a non-uniform, fixed price grid, ranging from −500 EUR/MWh to 4000 EUR/MWh, which are the price lower and upper bounds, according to the DAM Trading Rulebook published by HEnEx. Each curve can be stacked in a single matrix in the form:

$$Y=\left[\begin{array}{c}{y}_1\\ ⋮\\ y_T\end{array}\right]$$

(2)

Each row is a monotonic non-decreasing function of price, consisting of a set of price-volume points, as published by HEnEx. The forecasting setup is divided into a training set with a rolling window of selected days, a validation step over a certain number of days for hyperparameter selection, and a testing phase, where the forecast is compared with an actual aggregated supply curve.

Both the baseline model and the proposed one share a common process that addresses the multidimensional data, the FPCA method, where the curves are centred by subtracting from each one the mean curve μ:

$$Y_C=Y-\mu$$

(3)

And Singular Value Decomposition is applied to the centred curves:

$$Y_C=U\Sigma V^T$$

(4)

The forecasted curves are constructed in the form of

$$y_t=\mu +{\displaystyle {\sum }_{k=1}^K}{s}_{t,k}·v_k$$

(5)

With $v_k$ symbolising the basis curves and $s_{t,k}$ the time-varying scores. The process retains K significant terms and forecasts the $s_{t,k}$ components. As a condition for the determination of the K terms, the acceptance ratio, ar, is used:

$$ar\le {\displaystyle \frac{{\displaystyle {\sum }_{k=1}^K}{\sigma }_k^2}{{\displaystyle {\sum }_{k=1}^M}{\sigma }_k^2}}$$

(6)

With M symbolising the total k components, and ${\sigma }_k^2$ the squared singular values of the Σ component. The deviation between the ML model and the linear baseline model begins at this point, where the scores need to be forecast.

2.3. Linear Score Forecasting with the Baseline Model

The baseline model forecasts each score by using ridge regression for the deterministic part and ARIMA for the residuals. Ridge regression is formulated as

$$s_{t,k}^{det}=g_k\left(x_t\right)$$

(7)

where $x_t$ contains Fourier terms to capture potential periodic behaviours, calendar dummies, and exogenous features of electricity load forecasts, renewable energy generation forecasts, and ATCs with neighbouring countries, published the day before market opening time, as well as their lagged values, with 1, 2, 4, 96, 672 lags.

Residual modelling is performed by implementing ARIMA to capture remaining patterns in the residuals:

$$r_{t,k}=s_{t,k}-s_{t,k}^{det}$$

(8)

Since any trends are expected to be captured by the deterministic component, the backshift operator is neglected, and the parameter search is limited to a (p,q) grid. After this process, the forecasts are evaluated for overfitting using the AIC, while residual whiteness was evaluated using the Ljung–Box test.

2.4. Score Forecasting with Gradient Boosting

2.4.1. Model Description

The ML model is a gradient-boosted regression tree, an LSBoost regression ensemble. Each FPCA score is modelled as an ensemble of 100 weak regression trees:

$$f_k(x)={\displaystyle {\sum }_{m=1}^M}\eta ·h_{m,k}\left(x\right)$$

(9)

where M is the total number of trees, η is the learning rate, calibrated equal to 0.05, and $h_{m,k}$ is each regression tree. The model is trained sequentially, meaning that each new tree tries to correct the residual errors left by the previous tree:

$$f_k^{\left(m\right)}(x)=f_k^{(m-1)}(x)+\eta ·h_{m,k}(x)$$

(10)

2.4.2. Feature Selection

The feature vector x contains daily, semi-daily, and weekly Fourier terms, lagged features of market-state features, such as load forecasts, RES forecasts, their subtraction as net load, import/export daily ATCs, and regime-like dummy variables. The lag vector that was used contains 1, 2, 4, 96, 672, taking into account lagged values up to 1 h of the same day, values of the same MTU but previous day, and the same MTU of the week before.

Based on the available data, additional features were produced, such as the aforementioned net load:

$$N_t=L_t-R_t$$

(11)

Capturing the residual demand that must be covered by thermal generation, hydropower, and imports. To isolate instances when the system is stressed during positive net load, the subtraction’s positive values were stored as a separate feature. Ramping stress and sudden system changes were taken into account by calculating net load ramps and their absolute values:

$${\Delta N}_t^{15-min}=N_t-N_{t-1}, {\Delta N}_t^{1-hour}=N_t-N_{t-4}$$

(12)

Another produced feature is the share of RES energy:

$${RESShare}_t={\displaystyle \frac{R_t}{\left|L_t+R_t\right|}}$$

(13)

This quantifies how dominant renewable energy is at each MTU. The features of net ATC quantify cross-border flexibility:

$${NetATC}_t={ATC}_t^{imp}-{ATC}_t^{exp}$$

(14)

Therefore, calculating the import/export availability balance, and the total interconnection capacity:

$${TotalATC}_t={ATC}_t^{imp}+{ATC}_t^{exp}$$

(15)

The three-regime dummy variables that were used divide the forecasts into three states: low-price regime, scarcity regime, and neutral regime. The low-price regime is met in MTUs when the renewable energy is substantial, and the net load is relatively small. The scarcity regime is characterized by high net load or large ramp events, while the neutral regime includes any other state:

$$I=\left\{\begin{array}{c}{I}_t^{low}, {RESShare}_t>0.52 and N_t<Q_{0.55}\left(N\right)\\ I_t^{scar}, N_t>Q_{0.85}\left(N\right) or \left|{\Delta N}_t^{1-hor}\right|>Q_{0.85}\left(\left|\Delta N\right|\right)\\ I_t^{neutral}, otherwise\end{array}\right.$$

(16)

For the low-price regime, the check conditions include a share of renewables exceeding half of total demand, while net load is compared with its 55th quantile, a relatively relaxed threshold that encompasses both low-net-load and moderately low-load states. In order to register a scarcity regime, the net load needs to surpass the 85th quantile, a state in which the system requires expensive thermal units and imports, or during a sharp increase in the net load. In both cases, the 85th quantile is used to filter the “upper tail” of system stress, avoiding a stricter quantile that would be activated less often.

Instead of an ARIMA residual layer, the boosted trees learn score persistence and nonlinear autoregressive effects by using the score persistence values produced during training as forecast features. In the forecasting phase, the lagged score values 1, 2, 3, 4, 8, 12, 16, 96, 192, and 672 were used for each component k.

2.4.3. Rolling Score Statistics

For rolling windows equal to 4, 16 and 96 (1 h, 4 h and 1 day), the rolling mean is calculated:

$${\overline{s}}_{t,k}^{\left(w\right)}={\displaystyle \frac{1}{w}}{\displaystyle {\sum }_{i=1}^w}{s}_{t-i,k}$$

(17)

The rolling standard deviation is also calculated:

$${\sigma }_{t,k}^{\left(w\right)}=std\left(s_{t-w:t-1,k}\right)$$

(18)

These features assist the model in understanding whether the curve shape has recently been stable or volatile, whether the latent dynamics are trending, or if the current day resembles the previous hours or the previous day.

2.4.4. Band Correction

After the curve construction, a distinct LSBoost is deployed for band correction, aiming to correct systematic curve errors in specific price regions. The band correction process extracts values at selected price grid locations that represent the supply curve geometry. The extractions correspond to the 10th, 20th, 50th, 80th, and 90th quantiles and maximum price level. In addition, local slope indicators are constructed through differences between adjacent quantile points, such as $Y_{20}-Y_{10}$ and $Y_{90}-Y_{80}$, that serve as proxies for the steepness of the curves in the low-price and high-price regions, respectively.

For each price column j under correction, a moving average of the curves serves as the baseline curve $y_{t,j}^{base}$, which is subtracted from the original $y_{t,j}$, and then the residuals are forecasted:

$$e_{t,j}=y_{t,j}-y_{t,j}^{base}$$

(19)

Resulting in the final curve:

$${\widehat{y}}_{t,j}^{final}={\widehat{y}}_{t,j}+e_{t,j}$$

(20)

The features that were used were the same for the curve reconstruction, with lags 1, 2, 4, and 96. To ensure that the forecasting model prioritises economically relevant regions of the aggregated supply curve, model selection is performed through a weighted reconstruction loss during validation, which assigns higher importance to specific segments of the curve. For a set of observed curves $y_{t,j}$ where t = 1, …, T indexes time and j = 1, …, M indexes the price grid, the loss function is defined as

$$L=\sqrt{{\displaystyle \frac{1}{T·M}}{\displaystyle {\sum }_{t=1}^T}{\displaystyle {\sum }_{j=1}^M}{w}_j{\left(y_{t,j}-{\widehat{y}}_{t,j}\right)}^2}$$

(21)

where $w_j$ symbolises a price-dependent weight. The weight scheme is designed to emphasise the upper price tail, meaning the scarcity region, and the region around the empirical maximum marginal increment of the curve, where the market-clearing zone often occurs.

2.5. Post-Processing for Both Models

To ensure that, at the end of the process, the forecasted curves maintain their expected properties, they are subjected to certain constraints regarding their monotonicity, spike-prevention and physical feasibility. The constraint for non-negative volumes and spike-prevention is ensured by

$${\widehat{y}}_{t,j}^{final}=max\left(0, min\left({\widehat{y}}_{t+h,i}, q_{0.98}\left(y_{.,i}\right)\right)\right)$$

(22)

And the monotonicity:

$${\widehat{y}}_{t,j}^{final}=max\left({\widehat{y}}_{t+h,i}, {\widehat{y}}_{t+h,i-1}\right)$$

(23)

Figure 1 illustrates the overall methodology of the linear model:

Figure 1. Baseline model methodology.

And Figure 2, the ML methodology:

Figure 2. ML model methodology.

3. Results and Discussion

The two forecasting frameworks were evaluated on seven selected delivery days between January and March 2026. The selected evaluation days were chosen to represent different market conditions observed during the January–March 2026 period, including relatively stable operating conditions, high-renewable low-price periods, scarcity-driven conditions, and regime-transition days. The selected set intentionally includes both successful and failure cases of the proposed methodology in order to evaluate robustness under varying system states. In all cases, the forecasting horizon consisted of the full day-ahead schedule of 96 15 min MTUs, using a rolling 30-day training window split into 2208 sub-training samples and 672 validation samples.

Overall, the proposed ML model outperformed the baseline model in five out of seven test days. The strongest improvements were observed on 31 March, 1 March, 28 February, and 14 February, where the ML model reduced RMSE by approximately 9.6%, 9.7%, 7.7%, and 8.1%, respectively. The improvement was also visible in sMAPE, indicating that the gains were not only due to scale effects but also reflected better relative curve reconstruction. Table 1 summarises the findings for indicative days.

Table 1. Results for indicative days of January-March 2026.

Forecast Day	Model	MAE (MW)	RMSE (MW)	sMAPE (%)	R-Squared (%)
31 March 2026	Baseline	493.0	633.9	12.56	95.0
31 March 2026	ML	468.7	573.1	10.83	95.9
15 March 2026	Baseline	397.5	507.3	7.40	96.2
15 March 2026	ML	355.2	475.1	6.16	96.7
1 March 2026	Baseline	367.3	557.3	4.69	95.4
1 March 2026	ML	345.4	503.3	4.37	96.2
28 February 2026	Baseline	387.2	530.3	6.41	92.3
28 February 2026	ML	381.2	489.3	5.76	93.4
14 February 2026	Baseline	499.1	635.2	7.37	89.2
14 February 2026	ML	409.1	583.7	6.28	90.9
1 February 2026	Baseline	442.2	548.2	8.05	91.5
1 February 2026	ML	730.5	836.6	10.74	80.2
15 January 2026	Baseline	379.9	477.0	9.23	97.0
15 January 2026	ML	382.8	493.4	8.77	96.8

Excluding the clear failure case of 1 February, the proposed ML model achieved average reductions of approximately 7–8% in MAE, 6–7% in RMSE, and 10–11% in sMAPE (excluding the regime-shift outlier of 1 February) while increasing the average R-squared from 94.7% to 95.0%. This indicates that, under most tested conditions, the nonlinear latent-score model provides a more accurate reconstruction of the full hybrid curve surface, particularly under dynamically evolving market conditions.

The results also show that the FPCA representation remains highly efficient in both models. The baseline retained between 4 and 5 components, explaining approximately 99.55–99.63% of curve variance. The proposed ML model often selected fewer components, usually K = 2 or 3, except on 15 March, where K = 5 was selected. This suggests that the machine-learning score model can extract stronger predictive information from a more compact latent representation, especially when nonlinear exogenous and lagged-score features are informative.

The exception is 1 February 2026, where the ML model performed substantially worse than the baseline, with RMSE increasing from 548.2 MW to 836.6 MW and R-squared falling from 91.5% to 80.2%. This indicates that the ML model is more sensitive to validation misselection or abrupt regime transitions. In that case, the weighted validation loss selected a configuration that did not generalise well to the target day. The baseline model, although less flexible, was more stable because its ridge regression and ARIMA structures impose smoother linear dynamics.

The January result is also informative. On 15 January, both models performed similarly, with the baseline slightly better in MAE, RMSE, and R-squared, while the ML model achieved lower sMAPE. This suggests that when the curve dynamics are relatively regular, the added flexibility of the ML model does not necessarily provide large gains over the baseline.

Overall, the empirical results support the proposed methodology, but with an important qualification: the ML framework improves forecasting accuracy in most cases, particularly in February and March, but it is less robust under certain regime-transition days. Therefore, the proposed model is not uniformly superior but rather a more flexible and generally more accurate framework whose performance depends on stable validation selection and representative recent training data.

4. Conclusions

This paper presented a day-ahead forecasting framework for aggregated hybrid supply curves in the Greek DAM, with a 15 min MTU resolution. Instead of forecasting MCP directly, the proposed approach models the full supply curve as the underlying structural object from which price formation emerges. The methodology combines FPCA-based dimensionality reduction with machine-learning forecasting of latent curve scores, using calendar information, lagged latent dynamics, load, RES, net load, and cross-border ATC variables as inputs. A baseline ridge regression model coupled with ARIMA residual forecasting was used for comparison.

The results show that both approaches can represent hybrid curves efficiently, with a small number of FPCA components explaining more than 99.5% of curve variance. Across the selected test days, the ML framework outperformed the baseline in five out of seven cases, achieving lower MAE, RMSE, and sMAPE and slightly higher R-squared in most February and March evaluations. These findings indicate that nonlinear score forecasting and richer system-state features improve the model’s ability to capture curve dynamics beyond the linear residual structure of the baseline.

However, the results also reveal an important limitation. The proposed ML model underperformed significantly on 1 February 2026, suggesting sensitivity to regime changes, validation misselection, or insufficient representativeness of the recent training window. Therefore, while the ML approach offers improved accuracy and stronger flexibility, the baseline remains more stable under certain conditions.

Future work will focus on improving robustness through adaptive regime detection, direct multi-horizon score forecasting to reduce recursive error propagation, and inclusion of additional supply-side variables such as plant availability, fuel-specific capacity constraints, outages, and block-order forecasts.