A Hybrid 1D-CNN–LSTM–MHA Model for Short-Term Probabilistic Photovoltaic Power Forecasting

Abstract

The intermittent nature of photovoltaic power generation makes short-term forecasting critical for grid management. In this study, a hybrid model combining a one-dimensional convolutional neural network (1D-CNN), Long Short-Term Memory (LSTM), and multi-head attention (MHA) mechanism was developed and evaluated on a 300-day dataset at 15 min (H1) and 60 min (H4) forecast horizons. The model generates prediction intervals with 80% nominal coverage using a quantile regression approach trained with the pinball loss function. The hyperparameters were determined through Optuna-based Tree-structured Parzen Estimator (TPE) optimisation. The LSTM, 1D-CNN-LSTM, LSTM-MHA, and 1D-CNN-LSTM-MHA models were compared under the same experimental setting. The 1D-CNN-LSTM-MHA model achieved the best deterministic performance at both forecast horizons. At the H1 horizon, R² = 0.9370 and nRMSE = 7.13% were obtained, whereas at the H4 horizon, R² = 0.9327 and nRMSE = 7.37% were achieved. In the probabilistic evaluation, this model produced the lowest PINAW and Winkler score values. In the statistical comparison, the performance differences in the 1D-CNN-LSTM-MHA model relative to the LSTM reference model were statistically significant in the DM-MAE, DM-RMSE, Clark–West, and Fisher-Z tests at both horizons. The statistical results indicate that the contribution of the attention mechanism becomes more evident when combined with the convolutional component, whereas adding attention alone to the LSTM did not produce a statistically significant improvement. This study provides a comparative evaluation framework for short-term PV active power forecasting by combining probabilistic forecasting through quantile regression with statistical model comparison.

1. Introduction

With global warming, the depletion of fossil fuels, and rising energy demand, renewable energy sources are shaping the future of the energy transition [1]. The use of fossil fuels leads to environmental pollution and greenhouse gas emissions. Reducing dependence on non-renewable energy sources is crucial for social stability and development [2]. Among renewable energy sources, solar energy stands out for its popularity, ease of access, and low operating costs [3]. One method of solar energy production is through photovoltaic (PV) systems [4]. Over the past twenty years, PV energy has experienced rapid growth and has consistently been the fastest-growing source of electricity worldwide [5]. In 2024, PV energy accounted for approximately half of China’s total renewable energy capacity (1889 GW/887 GW) [6].

Due to the influence of solar irradiance and ambient temperature, the output power of grid-connected PV systems exhibits an intermittent, fluctuating and random pattern. This situation causes problems in the operation, scheduling and planning of the electricity system [7]. Accurate PV power forecasting is essential for the operation and energy production scheduling of PV power plants [1,4].

Statistical approaches, physical model-based techniques and data-driven methods are used in this power forecasting. In traditional statistical approaches, forecasts are made by fitting linear regression models to historical data; however, there are also non-linear relationships between power and numerous factors. Time series models, on the other hand, exploit temporal dependencies in historical data but struggle to predict outcomes from sudden weather changes. Physical model approaches account for the fundamental operating principles of PV panels and their interactions with meteorological factors. For these methods to be successful, detailed system models and comprehensive meteorological data are required [2].

Recent studies have made significant progress in deep learning-based time series forecasting and have offered new solutions for PV power forecasting. Due to the complexity and dynamic nature of PV energy production, forecasting time-series data in this field remains challenging. PV energy production is significantly influenced by temporal and spatial dependencies, as well as numerous factors such as solar irradiance, temperature, humidity and cloud cover. There are complex, interactive and non-linear relationships between these factors [8]. Deep learning has been favoured in recent years due to advancements in its ability to process complex data and computational power [9]. Various architectures, ranging from simple artificial neural networks to long short-term memory (LSTM) networks, are applied in renewable energy forecasting. LSTM networks are used as supervised learning algorithms for time-series forecasting [10].

In one study applying LSTM networks to PV power forecasting, day-ahead numerical weather prediction (NWP) data were used as input to capture solar irradiance behaviour. In that setting, forecasting performance depended on the accuracy of the NWP data. NWP data is not readily available at high resolution and low cost. In their study comparing 24 machine learning models based on NWP across 16 PV plants, [11] noted that input selection was even more decisive than model selection. They demonstrated that when using only daily-average irradiance and solar position angles without NWP data, the root-mean-square error (RMSE) increased by only 1.5% relative to the best results. These findings highlight the importance of models that do not require NWP data.

In another study utilising LSTM networks, different weights were assigned to time steps via an attention mechanism. In this architecture as well, a lack of the ability to extract complex features from variables was identified [3]. When the convolutional neural network (CNN)-LSTM hybrid architecture is used in conjunction, the CNN layers reveal local patterns and recurring structures in the raw time series data. The input representation fed to the LSTM layer is significantly simpler than the raw data. When the LSTM uses the CNN-processed feature representation as input rather than raw data, it learns temporal dependencies more effectively [12]. In their PV power forecasting study [13], the authors used a CNN layer to classify weather-related patterns and an LSTM layer to model dependencies in power generation time series based on the classification results.

PV power forecasting outputs are generally categorised into two types: deterministic and probabilistic approaches. Whilst deterministic forecasting provides single-point forecasts at each time step, the probabilistic approach treats PV output as a random variable, and the model generates quantile-based prediction intervals. Probabilistic forecasts provide additional information about uncertainty that is not available in deterministic forecasts, as they provide information about the uncertainty in future power outcomes [14]. One statistical method for obtaining probabilistic forecasts is quantile regression [15]. Quantile regression is regarded as an extension of classical linear regression, which produces a single point, to provide probabilistic outputs [16]. This approach is integrated into deep learning architectures by designing the output layer to produce multiple quantile levels and by using a quantile loss function (pinball loss) during training, which applies asymmetric penalisation rather than mean squared error (MSE). This loss function weighs the prediction error differently for each quantile according to the target probability level. In this way, the model learns outputs for different quantile levels rather than producing a single point forecast [17].

Malakouti et al. [18] divided the dataset into 10 random subgroups without regard to time sequence. In each iteration, 9 groups were used as training data and the remaining 1 as test data. It has been noted that K-fold cross-validation applied to time-series data disrupts the data’s temporal continuity and leaks future information into the model’s training phase [19].

In deep learning models, the correct determination of hyperparameters, such as the number of filters, the number of LSTM units, and the learning rate, affects the model’s ability to extract temporal patterns and its generalisation performance. The literature has shown that hyperparameter optimisation significantly improves model accuracy and robustness [20]. Optuna features an adaptive approach that utilises the Tree-Structured Parzen Estimator (TPE) to model hyperparameter configurations and dynamically refine the search space [21]. In architectures combining CNNs, LSTMs and attention mechanisms, a trial-and-error approach within the hyperparameter space, such as the number of filters, hidden layer size, number of attention heads and learning rate, can prevent the model from reaching its true potential. Song et al. [5] performed hyperparameter optimisation for the CNN–BiLSTM–Attention architecture using Optuna and noted that this approach enhances the model’s feature extraction capability and generalisation performance.

Research on PV energy forecasting is categorised into three groups based on the time horizon. Short-term forecasts range from a few minutes to a few hours. Medium-term forecasts range from one week to one year. Long-term forecasts, meanwhile, cover periods longer than one year. Short-term forecasts are particularly important for the effective integration of solar energy sources [22].

This study aims to develop a 1D-CNN-LSTM-MHA hybrid model that probabilistically forecasts active power output at 15- and 60 min forecast horizons using PV and meteorological data from the Alice Springs PV plant operated by the Desert Knowledge Australia Solar Centre [23], and to evaluate this model using deterministic, probabilistic, and statistical evaluation metrics.

The remainder of this paper is organised as follows: the data preprocessing steps, model architecture and training, evaluation metrics, and hyperparameter optimisation. The third section presents the findings and comparative evaluation results. In the fourth section, the findings are discussed in the context of the model and the literature. The fifth section presents the study’s conclusions and recommendations for future work.

2. Method

In this study, a probabilistic deep learning hybrid model combining 1D-CNN, LSTM and the Multi-Head Attention (MHA) mechanism has been designed for short-term PV active power output (AC Power) at different forecast horizons (15–60 min). The proposed hybrid model extracts local short-term patterns by applying the 1D-CNN layer to the input time window. These outputs form the inputs to the LSTM layer. The MHA mechanism simultaneously learns relationships among the hidden representations generated by the LSTM via multiple attention heads, thereby updating the weights of time steps that are significant for prediction. The architecture of the model proposed in this study is shown in Figure 1.

In this study, 300 days of data from the DKASC Alice Springs dataset covering the period 27 February 2015 to 23 December 2015 were used [23]. The model aims to generate a prediction interval using active power output and meteorological sensor data. The proposed model adopts a quantile regression approach trained using the pinball loss function. An 80% nominal coverage prediction interval is generated using lower, median and upper limit predictions corresponding to the 0.10, 0.50, and 0.90 quantiles. In this structure, the 50th percentile forecast (i.e., the median) is also used as the model’s deterministic point forecast, enabling the calculation of classical forecast accuracy metrics alongside probabilistic outputs. Dividing the data in chronological order reduced the look-ahead leakage risk caused by data splitting. The performance of the proposed model has been evaluated using deterministic metrics (R², RMSE, MAE, sMAPE, nRMSE), probabilistic metrics (PICP, PINAW, Winkler Score) and statistical comparison tests for both the 15 min (H1) and 60 min (H4) forecast horizons. In the statistical evaluation, the LSTM model was used as the comparison model. The Diebold-Mariano (DM) test based on absolute error loss, the Clark-West (CW) test, and the R² difference test based on the Fisher Z transformation were employed.

The general workflow of the proposed probabilistic 1D-CNN–LSTM–MHA model is shown in Figure 2.

2.1. Data Pre-Processing Process

Active power output measurements and meteorological sensor data were organised by timestamp and sorted chronologically. Invalid or anomalous observations were identified and removed from the dataset to ensure physical consistency. A daytime indicator was created to represent the daytime period. For the night-time period, solar radiation variables and active power output were set to zero. Sinusoidal time components were calculated to represent diurnal cyclicity. The model inputs consist of ambient temperature, global horizontal irradiance, diffuse horizontal irradiance, global tilted irradiance, relative humidity, sinusoidal time components, the daylight indicator, and four consecutive past observations for H1, and sixteen consecutive past observations for H4.

2.2. Data Splitting, Scaling and Creation of the Supervised Data Structure

To preserve the time series structure and prevent future data from leaking into the training process, the dataset was split in chronological order. Based on the timestamp, the first 65% was allocated to the training set, the next 15% to the validation set, and the final 20% to the test set. This approach was chosen to reduce the risk of data leakage that random splitting can introduce in time-series forecasting.

The input variables and the target variable (AC power) were transformed using separate min-max scaling to stabilise the model’s training. The scaling parameters were determined solely based on the training set. The same parameters were applied to the validation and test sets.

The scaled data were converted into supervised learning samples using a sliding window approach. Here, $L$ denotes the input window length, $h$ denotes the forecast horizon, and $N$ denotes the total number of observations. For any time index $t$, the input-target pair is defined as follows:

$${\mathbf{X}}_t=[x_t, x_{t+1}, \dots , x_{t+L-1}], y_t^{*}=y_{t+L+h-1}$$

In this structure, $X_t$ is a multivariate input tensor consisting of $L$ consecutive time steps. $y_t^{*}$ denotes a single target value located h steps after the last time step of the input window. The total number of valid samples is calculated as $N-L-h+1$. For the 15 min forecast horizon (H1), $L=4$ and $h=1$, whereas for the 60 min forecast horizon (H4), $L=16$ and $h=4$. The model produces three quantile predictions for both forecast horizons, defined as $Q=\{0.10,0.50,0.90\}$. In the training data loader, samples are shuffled at the beginning of each epoch, whereas the validation and test data loaders preserve the chronological order. The batch size is fixed at 64 for all data loaders.

2.3. The Proposed 1D-CNN-LSTM-MHA Model

In the proposed model architecture, the input tensor generated during the sliding-window stage,

$${\mathbf{X}}_t=[{\mathbf{x}}_t,{\mathbf{x}}_{t+1},\dots ,{\mathbf{x}}_{t+L-1}]\in {\mathbb{R}}^{L\times d}$$

is sequentially passed through a 1D-CNN layer, an LSTM layer, and an MHA layer to generate quantile predictions. Here, $d$ denotes the number of input features. The 1D-CNN layer extracts local short-term patterns by applying $F$ convolutional filters to the input tensor $X_t$ along the temporal axis. The output of the $k$-th filter at the $l$-th time step is calculated as follows:

$$z_{l,k}=\mathrm{ReLU}\hspace{0.17em}({\displaystyle \sum _{j=0}^{K-1}}{\mathbf{w}}_{k,j}^{\mathrm{\top }}{\mathbf{x}}_{l+j-p}+b_k),l=1,\dots ,L;k=1,\dots ,F$$

Here, ${\mathbf{w}}_{k,j}\in {\mathbb{R}}^d$ denotes the convolutional kernel weights, and $b_k$ denotes the bias term. To maintain the temporal length of the output at the same level as the input, the kernel size $K=3$ and the padding size $p=1$ are fixed in the convolution operation. The number of filters F is determined via Optuna hyperparameter optimisation. The output of the convolutional layer is obtained in the form $\mathbf{Z}=[{\mathbf{z}}_1,{\mathbf{z}}_2,\dots ,{\mathbf{z}}_L]\in {\mathbb{R}}^{L\times F}$. The LSTM layer models temporal dependencies by taking the convolutional output $\mathbf{Z}$ as input. At each time step $l$, the LSTM cell is updated using the following gate equations [24]:

$${\mathbf{f}}_l=\sigma \left({\mathbf{W}}_f\right[{\mathbf{h}}_{l-1};\hspace{0.17em}{\mathbf{z}}_l]+{\mathbf{b}}_f)$$

$${\mathbf{i}}_l=\sigma \hspace{0.17em}\left({\mathbf{W}}_i\right[{\mathbf{h}}_{l-1};\hspace{0.17em}{\mathbf{z}}_l]+{\mathbf{b}}_i)$$

$${\tilde{\mathbf{c}}}_l=\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{h}\left({\mathbf{W}}_c\right[{\mathbf{h}}_{l-1};\hspace{0.17em}{\mathbf{z}}_l]+{\mathbf{b}}_c)$$

$${\mathbf{c}}_l={\mathbf{f}}_l\odot {\mathbf{c}}_{l-1}+{\mathbf{i}}_l\odot {\tilde{\mathbf{c}}}_l$$

$${\mathbf{o}}_l=\sigma \left({\mathbf{W}}_o\right[{\mathbf{h}}_{l-1};\hspace{0.17em}{\mathbf{z}}_l]+{\mathbf{b}}_o)$$

$${\mathbf{h}}_l={\mathbf{o}}_l\odot \mathrm{t}\mathrm{a}\mathrm{n}\mathrm{h}\left({\mathbf{c}}_l\right)$$

The MHA mechanism operates as self-attention on the LSTM output sequence $H$. In this layer, the query, key, and value matrices are derived from the LSTM output through linear transformations [25]:

$$Q=H{W}^Q,K=H{W}^K,V=H{W}^V$$

Here, $W^Q$, $W^K$, and $W^V$ denote learnable projection matrices. These matrices are divided into $M$ attention heads, and scaled dot-product attention is computed for each head $m\in \{1,\dots ,M\}$:

$$hea{d}_m=\mathrm{s}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}\left({\displaystyle \frac{Q_m{K}_m^T}{\sqrt{d_k}}}\right)V_m,d_k={\displaystyle \frac{d_h}{M}}$$

This structure computes attention weights across time steps:

$$A=\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{c}\mathrm{a}\mathrm{t}(hea{d}_1,\dots ,hea{d}_M)W^O\in {\mathbb{R}}^{L\times d_h}$$

Here, $W^O$ denotes the output projection matrix. The output of the attention layer, $A$, is reduced to a single representation vector by averaging over all time steps:

$$r={\displaystyle \frac{1}{L}}{\displaystyle \sum _{l=1}^L}{a}_l\in {\mathbb{R}}^{d_h}$$

The output layer generates an independent prediction for each target quantile $q\in Q$ from this representation vector:

$$g=\mathrm{R}\mathrm{e}\mathrm{L}\mathrm{U}(W_{fc}r+b_{fc})$$

$${\widehat{y}}_q=w_q^{\mathrm{\top }}g+b_q$$

The output layer consists of three neurons corresponding to the quantile predictions $\left({\widehat{y}}_{0.10},{\widehat{y}}_{0.50},{\widehat{y}}_{0.90}\right)$. In this architecture, the number of filters $F$, the LSTM hidden dimension, the number of LSTM layers, the number of attention heads M, and the learning rate are jointly selected within a common search space via Optuna-based hyperparameter optimisation.

The proposed model can be summarised as follows:

$$\widehat{\mathbf{y}}=W_{out}\hspace{0.17em}\mathrm{R}\mathrm{e}\mathrm{L}\mathrm{U}\left(W_{fc}\left[{\displaystyle \frac{1}{L}}{\displaystyle {\sum }_{l=1}^L}\mathrm{M}\mathrm{H}\mathrm{A}(\mathrm{L}\mathrm{S}\mathrm{T}\mathrm{M}(\mathrm{C}\mathrm{N}\mathrm{N}(X_t)){)}_l\right]+b_{fc}\right)+b_{out}$$

where $\widehat{\mathbf{y}}=[{\widehat{y}}_{0.10},{\widehat{y}}_{0.50},{\widehat{y}}_{0.90}]$.

2.4. Loss Function and Probabilistic Output

The model was trained using pinball loss, an asymmetric loss function suitable for quantile regression. The loss function is derived from the minimisation problem proposed by [26]. Pinball loss guides the network to estimate the conditional quantile at a given quantile level by assigning asymmetric weights to positive and negative prediction errors. The loss function is defined as follows [26]:

$${\rho }_q(u)=\left\{\begin{array}{cc}q\cdot u,& u\ge 0\\ (q-1)\cdot u,& u<0\end{array}\right.$$

Here, $u=y-{\widehat{y}}_q$ denotes the prediction error, while $q\in (0,1)$ denotes the target quantile level. Since the model in this study estimates multiple quantiles simultaneously, the total loss is defined as the average pinball loss calculated across all target quantiles:

$$\mathcal{L}={\displaystyle \frac{1}{\mid Q\mid }}\sum _{q\in Q}{\displaystyle \frac{1}{N_s}}{\displaystyle \sum _{i=1}^{N_s}}{\rho }_q(y_i-{\widehat{y}}_{q,i})$$

The definition of the total loss is an adaptation of the single-quantile loss function proposed by [26] to a multi-quantile output.

In this study, the quantile set for both forecast horizons was set to $Q=\{0.10,0.50,0.90\}$. In this structure, the 0.50 quantile is used as the median forecast. The 0.10 and 0.90 quantiles form a prediction interval with an 80% nominal coverage level [16]. During the testing phase, the three quantile outputs from the model were inverse-transformed back to the original kW scale.

2.5. Hyperparameter Optimisation

The five hyperparameters in the proposed model, namely the number of convolutional filters, LSTM hidden dimension, number of LSTM layers, number of attention heads, and learning rate, were jointly optimised within an Optuna-based common search space. Optuna dynamically guides candidate hyperparameter combinations using a Tree-structured Parzen Estimator (TPE)-based sampling strategy [21]. The search space is given in

Table 1. Hyperparameter search space.

Hyperparameter	Search Type	Range/Set
Number of convolutional filters	Categorical	{32, 64, 128}
LSTM hidden dimension	Categorical	{32, 64, 128}
Number of LSTM layers	Integer	[1, 3]
Number of attention heads	Categorical	{4, 8, 16}
Learning rate	Continuous, log scale	[10−4, 5 × 10−3]

. The number of convolutional filters and the LSTM hidden dimension were selected from the categorical set $\left\{32,64,128\right\}$, the number of LSTM layers was selected from the integer range $\left[1, 3\right]$, and the number of attention heads was selected from the categorical set $\left\{4,8,16\right\}$. The learning rate was sampled on a logarithmic scale within the range $\left[{10}^{-4},5\times {10}^{-3}\right]$. In each Optuna trial, the candidate model was trained for 15 warm-up epochs, and the validation loss was reported to Optuna as an intermediate value at the end of each epoch. The hyperparameter combination yielding the lowest validation loss for each forecast horizon was selected for final model training. Table 1 presents the hyperparameter search space.

2.6. Model Training

The model variants created with the specified hyperparameters, namely LSTM, 1D-CNN-LSTM, LSTM-MHA, and 1D-CNN-LSTM-MHA, were trained under the same experimental setting. To improve reproducibility, a fixed seed value $\left(seed=42\right)$ was used in all experiments. Python 3.11, NumPy 1.26.4, PyTorch 2.6.0, and CUDA 12.4 random number generators were initialised with the same seed value. In addition, deterministic cuDNN 9.1.0 execution was enabled, and cuDNN benchmark mode was disabled. The shuffling operation in the training data loader was also controlled using a PyTorch generator initialised with the same seed.

During training, the AdamW optimiser was used, which decouples weight decay from gradient-based parameter updates [27]. The weight decay coefficient was set to $1\times {10}^{-4}$. In standard L2 regularisation, the squared norm of the weights is added to the loss function as a penalty term. Ref [27] showed that the equivalence between L2 regularisation and weight decay does not hold for adaptive gradient methods. AdamW addresses this issue by decoupling the weight decay step from gradient adaptation. The learning rate was initialised with the value determined during the Optuna search.

The ReduceLROnPlateau scheduler was applied to adaptively reduce the learning rate when validation pinball loss stopped improving. This scheduler reduced the learning rate by a factor of 0.5 when the validation loss did not improve for five consecutive epochs.

The training process was limited to 100 epochs. An early stopping mechanism was applied to reduce overfitting. At the end of each epoch, the model was evaluated on the validation set, and its weights were saved when the validation loss decreased to the lowest value recorded so far. If no improvement in validation loss was observed for 15 consecutive epochs, training was terminated. After training was completed, the weights yielding the lowest validation loss were loaded, and predictions were generated on the test set.

2.7. Evaluation Metrics

Model performance was evaluated using deterministic and probabilistic metrics. All evaluation metrics were calculated using daytime test samples.

The median quantile $\left(q=0.5\right)$ output was used as the point forecast and evaluated using the following metrics:

$$R^2=1-{\displaystyle \frac{{\displaystyle {\sum }_{i=1}^N}(y_i-{\widehat{y}}_i{)}^2}{{\displaystyle {\sum }_{i=1}^N}(y_i-\overline{y}{)}^2}}$$

$$RMSE=\sqrt{{\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}(y_i-{\widehat{y}}_i{)}^2}$$

$$MAE={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}\mid y_i-{\widehat{y}}_i\mid$$

Symmetric mean absolute percentage error (sMAPE) was calculated only for daytime observations where the true value was above 5 kW. This threshold was applied to prevent numerical instability when the denominator approaches zero during nighttime and transition periods:

$$sMAPE={\displaystyle \frac{100}{N_d}}{\displaystyle \sum _{i=1}^{N_d}}{\displaystyle \frac{2\mid y_i-{\widehat{y}}_i\mid }{\mid y_i\mid +\mid {\widehat{y}}_i\mid }}$$

Here, $N_d$ denotes the number of observations above the threshold. The normalised root mean square error (nRMSE) was calculated as the ratio of RMSE to the maximum observed power value and expressed as a percentage.

The quality of the prediction intervals was evaluated using PICP, PINAW, and the Winkler score [28]. Prediction interval coverage probability (PICP) measures the proportion of true values falling within the lower and upper quantile bounds:

$$PICP={\displaystyle \frac{100}{N}}{\displaystyle \sum _{i=1}^N}1[{\widehat{y}}_{0.10,i}\le y_i\le {\widehat{y}}_{0.90,i}]$$

Prediction interval normalized average width (PINAW) measures the average interval width normalized by the observed range of the target variable [14]:

$$PINAW={\displaystyle \frac{\frac{1}{N}{\displaystyle {\sum }_{i=1}^N}({\widehat{y}}_{0.90,i}-{\widehat{y}}_{0.10,i})}{y_{max}-y_{min}}}$$

The Winkler score combines the interval width with penalty terms for observations falling outside the prediction interval [29]:

$$W_i={\delta }_i+{\displaystyle \frac{2}{\alpha }}({\widehat{y}}_{0.10,i}-y_i)\mathbf{1}[y_i<{\widehat{y}}_{0.10,i}]+{\displaystyle \frac{2}{\alpha }}(y_i-{\widehat{y}}_{0.90,i})\mathbf{1}[y_i>{\widehat{y}}_{0.90,i}]$$

The average Winkler score was calculated as follows:

$$Winkler={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{W}_i$$

Here, ${\delta }_i={\widehat{y}}_{0.90,i}-{\widehat{y}}_{0.10,i}$ denotes the interval width, and $\alpha =0.2$ denotes the nominal significance level. The calibration error was calculated as the absolute difference between the observed coverage and the target nominal coverage: $\mid PICP-80\mid$.

The predictive accuracy of the LSTM model and the CNN-LSTM, LSTM-MHA, and CNN-LSTM-MHA models was evaluated using DM-MAE, DM-RMSE, CW, and Fisher-Z tests for the R² difference [30,31,32]. Here, $m$ denotes the model compared with the LSTM reference model, namely CNN-LSTM, LSTM-MHA, or CNN-LSTM-MHA. $N$ denotes the number of daytime observations included in the evaluation. $HLN$ denotes the Harvey-Leybourne-Newbold small-sample correction factor applied in the Diebold-Mariano test [33].

The DM-MAE statistic based on absolute error loss was calculated as follows:

$$d_{MAE,i}=\mid y_i-{\widehat{y}}_{LSTM,i}\mid -\mid y_i-{\widehat{y}}_{m,i}\mid$$

$$D{M}_{MAE}=\left[{\displaystyle \frac{{\overline{d}}_{MAE}}{\sqrt{Var\left(d_{MAE}\right)/N}}}\right]\times HLN$$

Here, $\overline{d}$ denotes the mean of the loss differential series, and $Var\left(d\right)$ denotes its sample variance. Statistical significance in the DM-MAE and DM-RMSE tests was assessed using a two-tailed p-value. A positive test statistic indicates that the compared model produced lower error than the LSTM reference model.

The DM-RMSE test was computed using the squared-error loss differential, as follows:

$$d_{RMSE,i}=(y_i-{\widehat{y}}_{LSTM,i}{)}^2-(y_i-{\widehat{y}}_{m,i}{)}^2$$

$$D{M}_{RMSE}=\left[{\displaystyle \frac{{\overline{d}}_{RMSE}}{\sqrt{Var\left(d_{RMSE}\right)/N}}}\right]\times HLN$$

The CW test was used to evaluate the adjusted squared-error difference between the compared model and the LSTM reference model. The CW statistic was calculated as follows:

$$e_{LSTM,i}=y_i-{\widehat{y}}_{LSTM,i}$$

$$e_{m,i}=y_i-{\widehat{y}}_{m,i}$$

$$C{W}_i=e_{LSTM,i}^2-\left[e_{m,i}^2-({\widehat{y}}_{LSTM,i}-{\widehat{y}}_{m,i}{)}^2\right]$$

$$CW={\displaystyle \frac{\overline{C{W}_i}}{\sqrt{Var\left(C{W}_i\right)/N}}}$$

The difference in explanatory power between the two models was assessed using a Fisher-Z-based test for the R² difference. For each model, an approximate signed correlation coefficient was obtained from $R^2$ using the sign of the Pearson correlation between the true and predicted values, and the Fisher Z transformation was then applied. The $Z_{R^2}$ statistic evaluates whether the explanatory power of the compared model differs from that of the LSTM reference model. It was calculated as follows:

$$R_{\Delta }^2=R_m^2-R_{LSTM}^2$$

$$r_{LSTM}=sign(corr(y,{\widehat{y}}_{LSTM}))\times \sqrt{R_{LSTM}^2}$$

$$r_m=sign(corr(y,{\widehat{y}}_m))\times \sqrt{R_m^2}$$

$$z_{LSTM}=0.5\mathrm{l}\mathrm{n}\left({\displaystyle \frac{1+r_{LSTM}}{1-r_{LSTM}}}\right)$$

$$z_m=0.5\mathrm{l}\mathrm{n}\left({\displaystyle \frac{1+r_m}{1-r_m}}\right)$$

$$Z_{R^2}={\displaystyle \frac{z_m-z_{LSTM}}{\sqrt{2/(N-3)}}}$$

3. Results

3.1. Results of Optuna Hyperparameter Optimisation

In total, 80 Optuna trials were conducted independently for each horizon. The hyperparameter combinations yielding the lowest validation loss were selected for model training.

Table 2. Final hyperparameter values determined by Optuna.

Hyperparameter	H1 (15 min)	H4 (60 min)
Number of convolutional filters	32	64
LSTM hidden dimension	64	64
Number of LSTM layers	1	3
Number of attention heads	8	16
Learning rate	2.99 × 10−3	9.17 × 10−4

shows the hyperparameter values determined for the two forecast horizons.

Differences were observed in the configurations determined for the two horizons. While a single-layer LSTM architecture with a hidden dimension of 64 was selected for the H1 horizon, a three-layer architecture with the same hidden dimension was chosen for the H4 horizon. The number of attention heads was set to 8 for H1 and 16 for H4. The number of convolutional filters was selected as 32 for H1 and 64 for H4. In our experiments, the longer H4 horizon was associated with an architecture having more LSTM layers, more attention heads, and a higher number of convolutional filters.

3.2. Forecast Results

The median quantile $\left(q=0.5\right)$ output of the proposed model was evaluated as a point forecast. Model performance was examined using both deterministic and probabilistic metrics for the 15 min H1 and 60 min H4 forecast horizons.

Table 3. Model Performance Results.

Model	Horizon	R2	RMSE	MAE	sMAPE (%)	nRMSE (%)	PICP (%)	PINAW	Winkler	|PICP − 80|
LSTM	H1	0.8773	21.0921	11.3593	17.5066	9.9087	84.0338	0.2108	46.8913	4.0338
1D-CNN-LSTM		0.8946	19.9712	9.0208	25.5307	9.4009	82.0768	0.0947	50.1706	2.0768
LSTM-MHA		0.8697	21.7281	12.0756	19.0444	10.2075	80.7572	0.2335	43.3030	0.7572
1D-CNN-LSTM-MHA		0.9370	15.1203	7.5344	25.7163	7.1319	81.8811	0.0818	41.0252	1.8811
LSTM	H4	0.8661	22.0285	11.8792	18.2681	10.3484	84.3081	0.2270	65.1624	4.3081
1D-CNN-LSTM		0.8763	21.1749	11.1555	17.4419	9.9476	82.1005	0.2358	62.4237	2.1005
LSTM-MHA		0.8753	21.2673	13.4283	22.7455	10.0244	82.1877	0.2421	64.6733	2.1877
1D-CNN-LSTM-MHA		0.9327	15.6790	6.9363	10.1183	7.3657	83.8625	0.1868	61.8512	3.8625

shows the results of the LSTM, 1D-CNN-LSTM, LSTM-MHA, and 1D-CNN-LSTM-MHA models for the two forecast horizons.

The highest deterministic performance in the H1 forecast horizon was observed in the proposed model, with $R^2=0.9370$, RMSE = 15.1203 kW, and MAE = 7.5344 kW. The model’s error level in short-term forecasting was lower than that of the LSTM, 1D-CNN-LSTM, and LSTM-MHA models, with an nRMSE of 7.1319%. However, the lowest sMAPE value was obtained by the LSTM model. The proposed model exhibited the lowest PINAW and the lowest Winkler score. The PINAW value of 0.0818 indicates that the model produced narrower prediction intervals. According to the Winkler score of 41.0252, the model produced higher-quality prediction intervals for this horizon. The PICP value of 81.8811% was also slightly above the nominal coverage level of 80%. This result indicates that the proposed model provided a balance between narrow prediction intervals and coverage for the H1 horizon.

Figure 3 shows a time-series comparison between the median forecasts $\left(q=0.50\right)$ produced by the proposed model at the H1 horizon and the actual values. The forecast curve largely follows the daily rise and fall profile of daytime generation periods. However, the model slightly underestimates the actual values in peak regions, while short-term deviations occur at certain daily transition points.

The scatter plot (Figure 4) shows that the predicted values are distributed along the 45° reference line with the actual values. The scatter is concentrated in the 150–200 kW range and widens in the 0–100 kW generation range. This pattern indicates that the model exhibits larger errors during transition periods and under low irradiance conditions.

As shown in Figure 5, the residual values at the H1 horizon are concentrated around zero. No clear asymmetry was observed in the distribution.

At the H4 horizon, the proposed model achieved the lowest error values in deterministic metrics, with $R^2=0.9327$, RMSE = 15.6790 kW, MAE = 6.9363 kW, and nRMSE = 7.3657%. These results indicate that, even with a 60 min forecast horizon, the combined use of CNN, LSTM, and MHA components provides higher explanatory power and lower error than the other models. According to the probabilistic results, the proposed model produced the lowest PINAW value of 0.1868 at this horizon and also achieved the lowest Winkler score. However, in terms of calibration error, the 1D-CNN-LSTM and LSTM-MHA models produced lower values. This finding indicates that although the proposed model generated narrower intervals with a lower Winkler score at the H4 horizon, its coverage level remained above the nominal 80% level. Furthermore, the H4 PICP value of 83.8625% was higher than the H1 PICP value of 81.8811%, indicating that the H4 horizon deviated more from the target coverage level than the H1 horizon. Figure 6 presents a time-series comparison of the actual and predicted values for the proposed model at the H4 horizon.

As shown in Figure 6, the median forecasts generally follow the daytime generation profile at the H4 horizon. However, compared with H1, the forecast curve shows greater deviation from actual values in the peak regions, and sudden generation drops on some days are not captured by the model immediately.

The scatter plot for the H4 horizon is shown in Figure 7. The points are generally distributed along the 45° reference line at the H4 horizon, but the scatter widens in the 50–150 kW generation range compared with H1. In the 0–50 kW range, the scatter also widens, and larger prediction deviations occur in this range compared with the H1 horizon.

According to the residual distribution generated for the H4 horizon, the residuals are concentrated around zero, and no clear asymmetry was observed. Figure 8 presents the residual distribution for the H4 horizon.

3.3. Results of the Statistical Comparison

To determine whether the performance differences observed in the model comparisons were statistically significant, the 1D-CNN-LSTM, LSTM-MHA, and 1D-CNN-LSTM-MHA models were evaluated against the LSTM reference model using DM-MAE, DM-RMSE, CW, and Fisher-Z tests for the $R^2$ difference.

Table 4. Statistical Comparison Results Relative to the LSTM Model.

Horizon	Model	DM-MAE	DM-MAE p-Value	DM-RMSE	DM-RMSE p-Value	CW	CW p-Value	(R2) Fisher-Z	(R2) Fisher-Z p-Value
H1	1D-CNN-LSTM	2.0637	0.0390	2.1702	0.0300	6.9761	0.0001	1.9223	0.0547
	LSTM-MHA	−0.4674	0.6402	1.0983	0.2721	8.0980	0.0001	0.6864	0.4925
	1D-CNN-LSTM-MHA	3.1876	0.0014	5.3462	0.0001	3.0739	0.0021	2.8785	0.0040
H4	1D-CNN-LSTM	3.4493	0.0007	0.9963	0.3191	5.9528	0.0001	2.3397	0.0193
	LSTM-MHA	−1.0924	0.2747	1.6965	0.0898	9.5504	<0.0001	1.0394	0.2986
	1D-CNN-LSTM-MHA	2.8418	0.0045	2.3460	0.0190	4.9590	0.0001	2.0304	0.0423

presents the statistical comparison results relative to the LSTM reference model.

At the H1 horizon, the prediction accuracy differences in the 1D-CNN-LSTM and 1D-CNN-LSTM-MHA models relative to the LSTM reference model were statistically significant according to both the DM-MAE and DM-RMSE statistics. The 1D-CNN-LSTM-MHA model produced stronger performance and statistical results than the 1D-CNN-LSTM model. Although the performance values of the LSTM-MHA model were worse than those of the reference model, the difference was not statistically significant (DM-MAE = −0.4674, p = 0.6402; DM-RMSE = 1.0983, p = 0.2721). According to the CW test, all three models produced statistically significant results at the H1 horizon. The LSTM-MHA model, which was not statistically significant in the DM tests, produced a significant CW statistic (8.0980, p = 0.0001). According to the Fisher-Z test results, only the 1D-CNN-LSTM-MHA model was statistically significant at the H1 horizon (2.8785, p = 0.0040). At this horizon, the $R^2$ difference was statistically significant only for the 1D-CNN-LSTM-MHA model.

At the H4 horizon, although some models showed better deterministic performance than the LSTM reference model, statistical tests did not consistently confirm these improvements across all loss criteria. The 1D-CNN-LSTM model produced a statistically significant improvement in the DM-MAE test (DM-MAE = 3.4493, p = 0.0007), but its DM-RMSE result was not statistically significant (DM-RMSE = 0.9963, p = 0.3191). This indicates that the 1D-CNN-LSTM model reduced the absolute error level relative to the LSTM reference model but did not reduce larger squared errors with the same consistency. In contrast, the LSTM-MHA model did not provide statistically significant improvement in either DM test; its DM-MAE statistic was negative (DM-MAE = −1.0924, p = 0.2747), and its DM-RMSE result was also not statistically significant (DM-RMSE = 1.6965, p = 0.0898). Therefore, despite its deterministic metric values, the LSTM-MHA model cannot be interpreted as providing a statistically supported forecasting improvement over the LSTM reference model at the H4 horizon. The 1D-CNN-LSTM-MHA model, however, produced statistically significant results in both DM-MAE and DM-RMSE tests (DM-MAE = 2.8418, p = 0.0045; DM-RMSE = 2.3460, p = 0.0190), indicating a more consistent error reduction across both absolute and squared-error loss criteria.

4. Discussion

In this study, comparing the LSTM, 1D-CNN-LSTM, LSTM-MHA, and 1D-CNN-LSTM-MHA models across the same data partitions and training regime allows the contribution of each component to be evaluated separately. Whilst the 1D-CNN-LSTM model highlights the contribution of the 1D-CNN component to local pattern extraction, the LSTM-MHA model demonstrates the effect of adding the attention mechanism to the LSTM alone. This comparative framework aims not only to reveal the performance of the proposed model but also to highlight the limitations of contributions at the component level.

According to the deterministic results, the proposed 1D-CNN-LSTM-MHA model achieved better performance than other models for short-term PV active power forecasting, as indicated by R², RMSE, MAE, and nRMSE. The LSTM-MHA model does not differ from the LSTM reference model in terms of deterministic performance. The fact that the DM-MAE statistic takes a negative value (−0.4674) at the H1 horizon indicates that the addition of the attention mechanism alone does not provide a measurable contribution compared to the LSTM in terms of absolute error. In contrast, the significant improvement in R², RMSE, MAE, and nRMSE values in the 1D-CNN-LSTM-MHA model suggests that the MHA mechanism contributes more when combined with the CNN layer’s feature representation. This finding indicates that the combined use of CNN and MHA components provides more balanced, statistically consistent performance than the reduced model variants. Ghimire et al. [12] noted that when an LSTM takes a CNN-processed feature representation as input rather than raw data, it learns temporal dependencies more effectively. The fact that the 1D-CNN-LSTM model achieves lower RMSE and MAE than the LSTM reference model at the H1 horizon, and that this difference is statistically significant in the DM-MAE and DM-RMSE tests, demonstrates that the CNN layer can reduce error through short-term local pattern extraction. However, the fact that the p-value in the Fisher-Z test remains above the 0.05 significance threshold indicates that this improvement is not validated with the same strength in terms of the difference in explanatory power. This result is consistent with the view highlighted by [12] that CNN-based feature representations can support LSTM learning; however, in the present study, this contribution varies depending on the metric type. Figure 9 visually summarises these deterministic differences across models and horizons, showing that the full 1D-CNN-LSTM-MHA model provides the most consistent improvement in R², RMSE, MAE, and nRMSE.

The 1D-CNN-LSTM model is significant in the DM-MAE test at the H4 horizon (p = 0.0007) but is not significant in the DM-RMSE test (p = 0.3191). This result indicates that the 1D-CNN-LSTM model reduces the mean absolute error at the H4 horizon compared to the LSTM reference model, but fails to reduce large prediction errors to the same extent. The LSTM-MHA model, on the other hand, was not found to be significant in the DM tests at either horizon, whilst a significant result was observed in the CW test. It demonstrates that adding the attention mechanism to the LSTM alone does not reduce the absolute error, but alters the structure of the squared error at specific prediction points. According to the Fisher-Z test, only the 1D-CNN-LSTM-MHA model showed a statistically significant R² difference at the H1 horizon. At the H4 horizon, the R² difference was statistically significant for both the 1D-CNN-LSTM and 1D-CNN-LSTM-MHA models, whereas the LSTM-MHA model remained non-significant.

According to probabilistic evaluation metrics, the 1D-CNN-LSTM-MHA model produced the lowest PINAW (0.0818) and the lowest Winkler score (41.0252) at the H1 horizon, whilst the PICP value was 81.88%, slightly above the nominal 80% target. The calibration error was 1.88%. This result indicates that the model maintains coverage balance while producing narrow prediction intervals. At the H4 horizon, the PINAW value (0.1868) and the nominal coverage deviation were observed to be higher compared to the H1 horizon. As the interval width increases with the extended horizon, it is observed that the coverage target is captured with less precision. Upadhaya et al. [34] noted that raw quantile outputs from deep models trained with pinball loss exhibit systematic deviations from nominal coverage levels. The fact that the evaluation metrics were calculated solely from daytime observations reduced the risk of artificially inflated metrics due to zero-production values at night. Figure 10 summarises the probabilistic performance differences across models and forecast horizons.

Kumari and Toshniwal [35] proposed an LSTM–CNN hybrid model for hourly GHI forecasting, in which the LSTM layer extracted temporal features and the CNN layer extracted spatial features from the correlation matrix of meteorological variables across 23 locations in California. The proposed model achieved a forecast skill score of approximately 37–45%, outperforming standalone models such as smart persistence, SVM, ANN, LSTM, and CNN. The R² value of 0.9370 obtained for the H1 forecast horizon in the present study is comparable to the results reported in the relevant literature; however, direct numerical comparison is not appropriate due to differences in the dataset, target variable (GHI versus PV active power), forecast horizon, and input variables. Gaboitaolelwe et al. [36] demonstrated that nRMSE values increase with increasing horizon length, regardless of the model type used. In the 1D-CNN-LSTM-MHA model proposed in this study, as the forecast horizon extended from H1 to H4, the R² value decreased only slightly from 0.9370 to 0.9327, the RMSE value rose from 15.1203 kW to 15.6790 kW, and the nRMSE increased from 7.1319% to 7.3657%. In contrast, the MAE value decreased from 7.5344 kW to 6.9363 kW. This indicates that the horizon extension did not result in deterioration across all error metrics. Consequently, the results indicate that, despite the prediction task at the H4 horizon involving a longer forecast horizon, the proposed model generally maintained its error level within a limited range.

In the Optuna optimisation results, different parameters emerged for the two horizons. Whilst a single-layer LSTM and 8 attention heads were selected for the H1 horizon, a three-layer LSTM and 16 attention heads were determined for the H4 horizon. This increase in LSTM depth and the number of attention heads may stem from the extended forecast horizon, which requires greater capacity for modelling temporal dependencies. The horizon-specific architectures identified by the TPE-based search are consistent with the observation of Song et al. [5] that Optuna-based hyperparameter optimisation can adapt the structural capacity of attention-based hybrid models to the forecasting task. In the present study, this adaptation manifested as an increase in depth from one to three LSTM layers and a doubling of attention heads when moving from H1 to H4, suggesting that the selected component capacities vary in a horizon-specific manner under the experimental setting of this study.

Limitations

The model proposed in this study was compared with LSTM, LSTM-MHA and 1D-CNN-LSTM models using 300 days of data from the DKASC Alice Springs PV plant. The model has not been evaluated on PV plants with different capacities or panel configurations. The proposed model was not validated on an independent second PV plant dataset; therefore, the findings should be interpreted within the scope of a single-site 300-day case study. In the study, the input set included ambient temperature, global horizontal irradiance, diffuse horizontal irradiance, global tilted irradiance, relative humidity, sinusoidal time components, a daylight indicator, and autoregressive past active power generation. The model has not been evaluated with different input types. Probabilistic evaluation was performed using only the quantile set Q = {0.10, 0.50, 0.90} and a single nominal coverage level (80%). The model behaviour at different nominal coverage levels was not evaluated.

5. Conclusions

In this study, a 1D-CNN-LSTM-MHA model was developed for short-term PV active power forecasting and evaluated on the DKASC Alice Springs dataset. Model performance was examined using deterministic, probabilistic, and statistical metrics at the H1 (15 min) and H4 (60 min) forecast horizons, comparing the LSTM, 1D-CNN-LSTM, LSTM-MHA, and 1D-CNN-LSTM-MHA variants.

The main findings can be summarised as follows:

• The 1D-CNN-LSTM-MHA model achieved the highest deterministic accuracy across R², RMSE, MAE, and nRMSE for both horizons.
• The findings revealed that the combined use of CNN and MHA components yielded more consistent performance than the reduced model variants. The LSTM-MHA model did not differ statistically from the LSTM reference model in deterministic accuracy at either horizon.
• According to the probabilistic evaluation results, the 1D-CNN-LSTM-MHA model produced the lowest PINAW and Winkler scores for both horizons. The PICP values remained slightly above the nominal 80% coverage target.
• In the statistical comparison, the performance differences in the 1D-CNN-LSTM-MHA model relative to the LSTM reference model were statistically significant in all DM-MAE, DM-RMSE, CW, and Fisher-Z tests for both horizons.
• This study combines ablation-based model comparison, probabilistic forecasting via quantile regression, and multidimensional statistical evaluation within a single experimental framework for short-term PV active power forecasting.

Future Work

It is recommended that the proposed model be tested on PV plants with different capacities and in different geographical locations to assess its generalisability. In this study, 300 days of data from a single plant were used; therefore, the effect of different panel technologies and installation configurations on model behaviour could be investigated in future studies. Comparative experiments with alternative architectures such as GRU, Transformer, and CNN-BiLSTM could position the 1D-CNN-LSTM-MHA architecture within a broader reference framework. Finally, comparative experiments conducted with input sets supported by NWP data could evaluate the contribution of external meteorological variables to forecasting accuracy.