5.2. Photovoltaic Power Forecasting Evaluation Metrics
This model ultimately outputs point forecasts and probability forecasts. The error evaluation metrics for each component are defined as follows.
For point forecasts, the Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE) serve as error evaluation metrics. Both metrics characterize the deviation between the forecast value and the actual value, with smaller values indicating better forecasting performance.
Here, represents the number of samples, denotes the predicted power at time , and indicates the actual power at time .
For probabilistic forecasting, the following metrics evaluate prediction performance: Prediction Interval Coverage Probability (PICP), Prediction Interval Normalized Root Width (PINRW), and Coverage Width-based Criterion (CWC) [
22]. PICP reflects the probability that the actual PV power output falls within the predicted interval, serving to assess prediction stability. PINRW indicates the sharpness of the prediction interval; a smaller value signifies a narrower interval and better sharpness. CWC reflects the overall prediction performance of PV power output, where a smaller value indicates better prediction quality.
Among these, and represent the upper and lower bounds of the power interval for the th sample , respectively; is a Boolean variable that equals 1 if the constraint is satisfied and 0 otherwise; denotes the difference between the maximum and minimum photovoltaic power values in the test set; and and are penalty coefficients, both set to 50.
5.3. Case Verification
This experiment was conducted in a PyTorch 3.10 environment. The input feature dimension is 2, comprising PV power output and solar irradiance. The look-back window is set to 96 time steps (one day) and the prediction horizon is also 96 time steps (one day). The training and test sets were split in an 8:2 ratio in chronological order. Three preset quantiles (0.025, 0.5, and 0.975) were defined to construct the 95% confidence interval, along with the quantile loss function, and fed into STGAT for training. All model hyperparameters were determined through grid search and validation set performance evaluation. The hidden layer dimension is 64. The Graph Attention Layer uses a LeakyReLU activation with a negative slope of and a dropout rate of 0.2. The main model employed the Adam optimizer for training, with a learning rate of 0.001, batch size of 32, and 50 training epochs. To accelerate convergence, a learning rate decay strategy was adopted: the learning rate was halved when the validation set loss failed to decrease for 10 consecutive epochs. After inference, two physical constraints were applied as post-processing steps: all predicted values were clipped to non-negative values via max(x, 0), and predicted values at nighttime time steps (steps 1–20 and steps 80–96) were set to zero. For the CWC metric, the penalty coefficient was set to and the target coverage rate to .
Considering the diurnal nature of PV output, predicting only the next 4 h at an ultra-short timescale cannot fully capture the daily PV output pattern. This study first performs short-term PV output forecasting for the next day, generating a 96-step-ahead forecast. The resulting point forecasts are shown in
Figure 6.
As shown in
Figure 6, the STGAT model accurately captures the daily variation patterns of PV power output, precisely tracking the rapid climb in the early morning, sustained peak during midday, and abrupt drop in the evening. Notably, even during periods of intense power fluctuations caused by cloud cover, the model’s prediction curve remains highly consistent with actual values. This demonstrates that the spatio-temporal attention mechanism effectively integrates collaborative information from neighboring stations, significantly enhancing adaptability to abrupt operational changes. Furthermore, the model demonstrates outstanding stability in multi-day forecasts, with peak prediction errors controlled within 2%, validating its long-term modeling capability.
To comprehensively evaluate model performance, ultra-short-term forecasts extending 4 h ahead were conducted. The error metrics RMSE and MAE for 4 h and 1-day forecasts across the entire dataset are presented in
Table 3.
The data in
Table 3 indicate that the prediction time scale significantly impacts model accuracy. For 4 h lead forecasts, the RMSE and MAE were 5.2570% and 2.3685%, respectively, while 1-day lead forecasts saw errors increase to 14.0591% and 6.1984%, representing relative error increases of approximately 8.8% and 4.3%. This discrepancy stems from the cumulative effect of meteorological uncertainty during extended forecasting periods. Nevertheless, even within a 24 h prediction window, the model maintains MAE below 7% and RMSE under 15%, fully demonstrating the QR-STGAT architecture’s robust modeling capability for complex spatio-temporal dependencies.
To validate the effectiveness of the proposed PV power forecasting method, the point forecasting model based on QR-STGAT was further compared with models based on LSTM, BiLSTM, and Transformer, as shown in
Figure 7 and
Table 4. As evident from
Figure 7, the photovoltaic power prediction curve generated by the proposed method exhibits the highest degree of alignment with actual values. Particularly during periods of intense power fluctuations (such as the cloud disturbance period from 12:00 to 14:00), the STGAT model accurately tracks sudden power changes, whereas LSTM and BiLSTM models exhibit noticeable lag and deviation. Although the Transformer model possesses parallel computing advantages, its response to short-term sudden changes is slower than that of STGAT.
Further analysis of
Table 4 reveals that under equivalent conditions, the QR-STGAT model achieves a prediction accuracy of 93.48%, with RMSE and MAE values of 14.0591 MW and 6.1984 MW, respectively. Compared to the traditional LSTM model, the RMSE improved by 11.61% and MAE by 11.99%; compared to BiLSTM, the RMSE improved by 10.38% and MAE by 10.72%; and compared to Transformer, the RMSE improved by 9.52% and MAE by 9.87%. Importantly, QR-STGAT also outperforms the graph-based spatial baseline STGCN, achieving an RMSE improvement of 20.17% and an MAE improvement of 33.07% over STGCN. This result demonstrates that the adaptive attention-based spatial aggregation in QR-STGAT captures inter-station dependencies more effectively than the fixed-weight graph convolution used in STGCN, thereby validating the specific added value of the proposed spatial modeling approach. This significant overall performance improvement stems from: (1) the graph attention mechanism dynamically capturing nonlinear correlations between sites, overcoming the limitation of fixed spatial weights in STGCN and the single-site temporal modeling constraint of LSTM-based methods; (2) the spatiotemporal joint modeling architecture simultaneously extracting spatial dependencies and temporal evolution patterns, which neither BiLSTM nor Transformer can achieve due to their lack of spatial topology integration; (3) the quantile regression loss function enhancing robustness against abnormal operating conditions. Collectively, these findings demonstrate that the proposed method exhibits superior generalization capabilities and disturbance resistance in complex spatiotemporal coupling scenarios.
To verify that the performance improvements of the proposed QR-STGAT model over the baseline models are statistically meaningful rather than attributable to random variation in training, the Diebold–Mariano (DM) test was conducted. The DM test is a standard statistical tool for comparing the predictive accuracy of two forecasting models based on their loss differentials. A negative DM statistic indicates that the forecast errors of QR-STGAT are significantly smaller than those of the corresponding baseline, and a p-value below 0.01 indicates that this difference is statistically significant at the 1% level.
As shown in
Table 5, the DM statistics for QR-STGAT against all four baseline models are negative, with values of −6.7587, −3.7650, −3.0761, and −5.9227 against LSTM, BiLSTM, Transformer, and STGCN, respectively. All
p-values are below 0.01, indicating that the improvements are statistically significant at the 1% level across all comparisons. These results confirm that the superior forecasting performance of QR-STGAT is not attributable to random variation in training, but reflects a genuine and statistically robust improvement over existing temporal and spatial baseline models.
Given the limitations of point forecasting, confidence intervals are calculated by combining quantile results with different confidence levels. The calculations indicate that the best fit between predicted and actual values occurs when the quantile is set to 0.5. When a 95% confidence interval is applied, the resulting probability forecast range is shown in
Figure 8. The figure clearly demonstrates that the prediction interval widens during periods of significant power fluctuations and narrows during stable conditions. This behaviour is consistent with the expected properties of probabilistic forecasting, where uncertainty grows with power variability. The adaptive interval width reflects the model’s robustness to meteorological disturbances, providing reliable uncertainty bounds for grid dispatch decisions.
The probability prediction evaluation metrics under different confidence intervals are shown in
Table 6. Increasing the confidence level requirement elevated the prediction interval coverage probability (PICP) from 88.78% to 93.05%, while the prediction interval normalized width (PINRW) increased from 0.0481 to 0.0561. The composite score (CWC) improved from 19.4801 to 14.6579. This pattern aligns with fundamental probability prediction characteristics: expanding the confidence interval trades a broader prediction range for higher coverage probability, thereby enhancing prediction reliability. Notably, point prediction error metrics RMSE and MAE remain largely stable across confidence levels, maintaining around 14% and 6%, respectively. This indicates that confidence interval adjustments primarily affect the precision of uncertainty quantification while having minimal impact on deterministic prediction accuracy.
To further quantify the impact of the static graph topology on model performance under varying meteorological conditions, an additional weather-stratified error analysis was conducted. The test set was partitioned into three weather categories based on daily physical characteristics, including daily maximum power output and first-order variability rate: sunny days, cloudy/abrupt-change days, and rainy/overcast days. Point prediction metrics (RMSE, MAE) are normalised by the daily maximum power output to ensure fair comparison across conditions with different absolute output scales. Probabilistic metrics (PICP, PINRW) are also reported. Results are presented in
Table 7.
As shown in
Table 7, the model performs consistently well under sunny and cloudy/abrupt-change conditions, with RMSE values of 27.22% and 27.04%, respectively, and PICP values above 93%. However, under rainy and overcast conditions, RMSE increases to 36.55% and PICP drops to 88.96%, falling below the nominal 90% coverage target. This performance degradation is attributable to the static nature of the adjacency matrix. Under heavy cloud cover and localised precipitation, the instantaneous spatial correlation structure among PV stations deviates substantially from the historical average encoded in the fixed graph topology, leading to misspecified spatial aggregation weights and reduced accuracy in both point prediction and interval coverage. This finding provides empirical support for the limitation acknowledged in the Conclusions and further motivates the adoption of a dynamic graph construction mechanism in future work.