A Parallel Prediction Model for Photovoltaic Power Using Multi-Level Attention and Similar Day Clustering

Abstract

Photovoltaic (PV) power generation is significantly impacted by environmental factors that exhibit substantial uncertainty and volatility, posing a critical challenge for accurate PV power prediction in power system management. To address this, a parallel model is proposed for PV short-term prediction utilizing a multi-level attention mechanism. Firstly, gray relation analysis (GRA) and an improved ISODATA algorithm are used to select a dataset of similar days with comparable meteorological characteristics to the forecast day. A transformer encoder layer with multi-head attention is then used to extract long-term dependency features. Concurrently, BiGRU, optimized with a Global Attention network, is used to capture global temporal features. Feature fusion is performed using Cross Attention, calculating attention weights to emphasize significant features and enhancing feature integration. Finally, high-precision predictions are achieved through a fully connected layer. Utilizing historical PV power generation data to predict power output under various weather conditions, the proposed model demonstrates superior performance across all three climate types compared to other models, achieving more reliable predictions.

1. Introduction

The emergence of low-carbon clean power systems has brought sustainable energy generation to the forefront of global interest [1]. Photovoltaic (PV) technology plays a crucial role in these new power systems. However, it is susceptible to meteorological conditions [2], which exhibit complex and fluctuating features and structures [3]. Therefore, to guarantee the secure, consistent, and cost-effective functioning of power systems, accurate prediction of PV power generation is essential to fulfill the demands of power system planning, operation, control, and scheduling [4].

Various prediction methods have been identified for PV power prediction, including mathematical–physical modeling, statistical time series, and machine learning methods [5]. The mathematical–physical modeling approach involves simplifying the output power calculation based on the physical structure of the PV power generation system through mathematical modeling, rather than relying on weather and other numerical values [6,7]. While this method offers a simple calculation process, it fails to replicate the fluctuations observed in actual operation, resulting in a lower prediction accuracy. Statistical time series methods utilize statistical models that build upon mathematical and physical models to forecast PV power output. Commonly employed statistical prediction techniques include the AutoRegressive Integrated Moving Average (ARIMA) method [8], the AutoRegressive and Moving Average (ARMA) method [9], and regression analysis [10]. These methods are effective for short-term linear time series predictions; however, their efficiency diminishes with longer prediction horizons or when faced with nonlinear challenges. Recent developments in intelligent sensing and communication technologies have facilitated the collection of vast amounts of data, such as load, weather, and geographical information. These data serve as a crucial foundation for artificial intelligence, big data analytics, and other methodologies in PV power prediction. Consequently, machine learning methods, known for their capacity to extract complex abstract features, are gaining significant attention due to their learning capabilities [11].

Machine-learning-based prediction methods can be categorized into shallow and deep learning models. Deep learning models are known to be more effective in capturing data features and patterns compared to shallow learning models [12]. They are commonly utilized in the analysis and processing of large-scale data, such as in PV prediction [13,14,15]. Rodríguez et al. [16] employed feed-forward neural networks in conjunction with spatiotemporal parameters to improve the accuracy of short-term predictions. Annalisa et al. [17] utilized a Nonlinear AutoRegressive with eXogenous inputs (NARX) network for PV prediction, which can handle nonlinear relationships and consider external input impacts, and it exhibits superior flexibility and performance compared to traditional feedforward networks. As machine learning technology advances, various hybrid models have been introduced to address the limitations of individual models and produce more accurate predictions. This has led to the development of diverse prediction combinations. Agga et al. [18] proposed a hybrid prediction method that combines a Convolutional Neural Network (CNN) with a Long Short-Term Memory (LSTM) model, resulting in improved prediction accuracy. Tang et al. [19] integrated the attention mechanism with CNN and Bidirectional Long Short-Term Memory (BiLSTM) to create a day-ahead PV prediction model. This model effectively examines the causal relationship between features and outputs through the attention mechanism, outperforming single machine learning models in terms of prediction accuracy. These studies demonstrate that combined prediction models generally yield superior prediction results compared to individual prediction models.

The results of the aforementioned model investigations exhibit suboptimal performance in forecasting long time series. This limitation predominantly stems from the incapacity of networks such as CNN and LSTM to sustain extended sequences and adequately capture long-term dependencies [20]. Transformer models integrate the attention mechanism to augment their capability of capturing long-term dependencies [21]. However, in situations with restricted computational resources and limited datasets, this strategy may become unstable, potentially resulting in information loss. To address this issue, Kim et al. [22] combined the Transformer model with LSTM, leveraging the strengths of both models to improve local and global feature extraction and improve prediction accuracy. Moon et al. [23] merged the Transformer model with CNN, offering precise, immediate forecasts and the capacity to discern long-term power generation trends. In addition to improvements in composite models, the temporal modeling and optimization of long series can be achieved by modifying the Transformer model’s structure or introducing new iterations of the attention mechanism, such as informer [24], autoformer [25], and crossformer [26]. These variations in time series prediction have demonstrated positive results, albeit with increased structural complexity.

The current focus of research on PV power prediction primarily emphasizes enhancing model accuracy, often neglecting the quality of the training dataset and the impact of various feature factors on the output. To bridge this research gap, this study integrates the strengths of multiple models and devises a deep learning framework that integrates similar day selection, data feature selection, and a multi-level attention mechanism for PV power prediction. This approach leverages the advantages of each module without significantly escalating the overall complexity. The training dataset is refined through gray relation analysis (GRA) and an improved ISODATA clustering algorithm to identify a dataset comprising days with akin meteorological characteristics to the forecast day. Furthermore, weather conditions are classified into different categories, such as sunny, cloudy, and rainy days, across different seasons to reduce the data needed for prediction. Long-term dependency features are extracted utilizing a Transformer encoder layer with multi-attention, while global temporal features are captured by a BiGRU network optimized with Global Attention. Feature fusion is facilitated through Cross Attention, enabling the model to concentrate on crucial features by computing attention weights, thereby enhancing feature integration. Ultimately, precise predictions are generated using a fully connected layer.

This paper integrates the strengths of multiple models to develop a short-term PV power prediction model that incorporates similar day selection, data processing, attention mechanisms, and deep learning. This approach effectively addresses the limitations of individual models in terms of long-term data dependence and global feature extraction. The main innovations of this paper are outlined as follows:

(1)

Improved Clustering and Data Quality: Employing similar day selection and an improved ISODATA clustering algorithm improves the clustering impact, categorizing the dataset by different seasons and climate types. This approach reduces the amount of data required for prediction and improves data quality.

(2)

Transformer Encoder Layer: Dependencies between different positions in the sequence are extracted and global contextual information is captured using a Transformer encoder layer with a multi-head attention mechanism. Additionally, a parallel structure is implemented to accelerate model training and inference.

(3)

BiGRU with Global Attention: A BiGRU network, optimized with Global Attention, captures global temporal features, enhancing the model’s ability to perceive time-domain characteristics of multi-feature sequences.

(4)

Cross Attention for Feature Fusion: Cross Attention is employed to fuse information from different sources, enhancing the representation capability of features and achieving high-accuracy predictions.

2. Meteorological Data Processing

2.1. Data Preprocessing

Historical data form the foundation of prediction, and to ensure accuracy, the data must undergo preprocessing to improve their quality. The preprocessing steps are summarized as follows:

(1)

It must be ensured that data recording intervals, feature labels, and timestamps are uniformly formatted;

(2)

For noncontinuous missing data, interpolation methods can be used for estimation. Given the strong periodicity of load data, the Lagrangian interpolation method is employed to estimate missing values and fill in the gaps [27];

(3)

Evident errors and repeatability issues, such as sudden zero values or extremely high values exceeding system capacity, should be identified and corrected. Invalid data must be discarded or replaced, as PV systems are expected to operate normally with slow, gradual changes in power. Invalid data points can be replaced by the average value of the preceding and succeeding time points.

2.2. Similar Day Selection

The performance of PV power generation systems is impacted not only by their structural aspects but also by various weather conditions. Predicting PV power generation is complex due to the impact of numerous non-critical factors, which can reduce the prediction accuracy. This study collected meteorological data such as total horizontal irradiance, scattered horizontal irradiance, temperature, relative humidity, wind speed, and wind direction. Pearson’s coefficient was utilized to identify meteorological factors that have a higher correlation with power generation. These factors were then used to select similar days [28], aiming to eliminate the impact of less significant meteorological data on the selection process. The calculation is presented as follows:

$${\mathrm{\rho }}_{xy}={\displaystyle \frac{{\displaystyle \sum _{i=1}^n\left(X_i-\overline{X}\right)\left(Y_i-\overline{Y}\right)}}{\sqrt{{\displaystyle \sum _{i=1}^n{\left(X_i-\overline{X}\right)}^2}}\sqrt{{\displaystyle \sum _{i=1}^n{\left(Y_i-\overline{Y}\right)}^2}}}}$$

(1)

where n denotes the total number of samples; $X_i$ denotes the meteorological parameter at time i; $\overline{X}$ denotes the average value of the meteorological parameter; $Y_i$ denotes the PV output power at time i; and $\overline{Y}$ denotes the average value of the PV output power. The correlation coefficients between each meteorological factor and the PV power were calculated and are outlined in

Table 1. Correlation coefficients between meteorological factors and output power.

Meteorological Factors	Pearson Coefficient
Irradiance	0.989
Temperature	0.519
Humidity	0.504
Wind speed	0.346
Wind direction	0.109
Air pressure	0.058

. The analysis revealed that irradiance, temperature, humidity, and wind speed exhibited the highest Pearson correlation coefficients with power generation, suggesting a strong correlation. Therefore, these four variables were selected for the calculation of similar days.

Irradiance has a direct impact on the power generation efficiency of PV panels, while temperature impacts the conversion efficiency of PV panels, and wind speed and humidity are significant factors impacting the heat dissipation of the system [29]. As a result, this study chose weather variables such as irradiance, temperature, wind speed, and humidity as the input vectors for prediction. This selection aimed to simplify the data complexity and improve the prediction accuracy.

However, the accuracy of power prediction for a given day can be impacted by various weather conditions. Due to the inherent variability in weather patterns, relying only on the proximity of the day for training prediction models may introduce a certain level of error. To improve the prediction accuracy, the gray correlation analysis method is employed to filter the data pertaining to different meteorological factors. This method involves calculating the gray correlation value for each sample to elucidate the relationships among meteorological variables and to evaluate their similarities. In this research, data from days exhibiting a gray correlation value exceeding 0.8 with the prediction day data were selected as the training dataset [30].

(1)

Meteorological feature vectors are developed based on the selected meteorological features M:

$$M=\left[\overline{R},\overline{T},\overline{W},\overline{H}\right]$$

(2)

where $\overline{R}$ denotes the average daily irradiance; $\overline{T}$ denotes the average daily temperature; $\overline{W}$ denotes the average daily wind speed; and $\overline{H}$ denotes the average daily humidity.

(2)

After processing each component, the meteorological feature vectors for the forecast day and d day are $x_0$ and $x_d$, respectively.

$$x_0=\left[x_{01},x_{02},x_{03},x_{04}\right]$$

(3)

$$x_d=\left[x_{d1},x_{d2},x_{d3},x_{d4}\right]$$

(4)

(3)

The correlation coefficients of $x_0$ and $x_d$ in the s (s = 1, 2, …, 4) classification, ${\mathsf{\xi }}_{ds}$, are determined as follows:

$${\mathrm{\xi }}_{ds}={\displaystyle \frac{a+r\ast b}{\left|x_{0s}-x_{ds}\right|+r\ast b}}$$

(5)

$${\displaystyle a}\underset{d}{{\displaystyle =\min }}\underset{s}{{\displaystyle \min }}\left|x_{0s}-x_{ds}\right|$$

(6)

$${\displaystyle b}\underset{d}{{\displaystyle =\max }}\underset{s}{{\displaystyle \max }}\left|x_{0s}-x_{ds}\right|$$

(7)

where r denotes the resolution coefficient, which takes a value between [0,1]. In this study, r = 0.5.

(4)

The correlation coefficient of each component is synthesized, and the similarity between $x_0$ and $x_d$ is defined as $\gamma \left(x_0,x_d\right)$.

$$\gamma \left(x_0,x_d\right)={\displaystyle \frac{1}{m}}{\displaystyle \sum _{s=1}^4{\xi }_{ds}}$$

(8)

where m denotes the total number of components of the meteorological eigenvectors.

2.3. Improved ISODATA Clustering Algorithm

The process yields a rough set of days with similar PV prediction, which can be further refined through clustering techniques to improve dataset optimization [31]. This study focuses on applying clustering methods to meteorological data to improve the prediction accuracy by minimizing intra-category variances and maximizing inter-category differences. Commonly used clustering algorithms such as K-means and ISODATA [32] are employed. While the K-means algorithm is valued for its simplicity and efficiency, challenges arise in determining the optimal k value and initial clustering centers. Furthermore, its reliance on Euclidean distance metrics may not adequately capture temporal changes in high-dimensional features, particularly in load forecasting for different time periods. The ISODATA algorithm addresses these limitations by incorporating merging and splitting operations to dynamically adjust cluster numbers based on cluster characteristics. However, the random selection of initial cluster centers may lead to slow convergence and unpredictable results. Additionally, the continued use of Euclidean distance metrics may limit its ability to capture high-dimensional features effectively. To address these issues, this paper introduces an improved ISODATA algorithm that optimizes the configuration of initial clustering centers and Euclidean distances.

2.3.1. Optimization of Initial Clustering Centers

The fundamental concept of this optimization approach is to improve the separation between initial clustering centers during their selection process. Initially, the selection of the first clustering center is random. Subsequent clustering centers are selected based on their distance from the already selected centers, with points that are farther away from the existing centers being more likely to be selected.

This optimization technique is designed to mitigate the issue of initial clustering centers being in close proximity. When the initial centers are too close, there is a risk of them being grouped into the same cluster, which would require additional iterations to separate them into different clusters. Conversely, widely dispersed initial centers are more likely to correspond to separate clusters, facilitating quicker convergence and yielding improved clustering results.

The steps in the optimization algorithm for initial clustering center selection are given as follows:

(1)

A random sample from the dataset is selected to serve as the first initial clustering center.

(2)

For each remaining sample x_i, its distance from each existing clustering center is calculated, and the shortest distance d(x_i), is recorded.

(3)

The probability of each sample x_i being selected as the next clustering center is proportional to ${\displaystyle \frac{d{\left(x_i\right)}^2}{{\displaystyle \sum _{x\in D}d}{\left(x_i\right)}^2}}$. Samples with larger d(x_i) have a higher probability of being selected.

(4)

Step 2 is repeated until k initial clustering centers are selected.

This strategy ensures that the selected initial clustering centers are widely separated, thereby enhancing the probability of them belonging to different clusters and expediting the convergence process.

2.3.2. Kernel Method

The optimization of Euclidean spatial distance is achieved through the utilization of a kernel method. Initially, the data from the original space undergo mapping to a high-dimensional feature space via nonlinear mapping, followed by clustering. This mapping improves the probability of linear separability among data points. In the high-dimensional space, the clustering algorithm computes distances between samples based on the high-dimensional features of the load curves, leading to improved clustering performance. However, computations in the high-dimensional space are complex. To mitigate this issue, the kernel function K(x,z) is introduced.

Let X denote the input space and H denote a higher-dimensional space. Let ϕ(x) be a mapping function that maps a point x in space X to a point h in space H, such that h = ϕ(x). If there is a function K(x,z) for all x,z ∈ X, such that K(x,z) = ϕ(x)·ϕ(z), then the inner product of the mappings of two points can be directly computed in the original space X using the kernel function.

The distance between two samples is typically represented by a kernel function in clustering calculations as follows:

$$\begin{array}{l}d(x_i,x_j)=\parallel \varphi (x_i)-\varphi (x_j){\parallel }^2\\ =\varphi {(x_i)}^2-2\varphi (x_i)\varphi (x_j)+\varphi {(x_j)}^2\\ =K(x_i,x_j)-2K(x_i,x_j)+K(x_i,x_j)\end{array}$$

(9)

This is calculated as the distance from each sample x_i to each clustering center μ_j. In the high-dimensional space, the distance is determined as follows:

$$\begin{array}{l}d(x_i,{\mu }_j)=\parallel \varphi (x_i)-\varphi ({\mu }_j){\parallel }^2\\ =\varphi {(x_i)}^2-2\varphi (x_i)\varphi ({\mu }_j)+\varphi {({\mu }_j)}^2\end{array}$$

(10)

The center of clustering in the high-dimensional space can be expressed as follows:

$$\varphi ({\mu }_j)={\displaystyle \frac{1}{\mid C_j\mid }}{\displaystyle \sum _{\varphi (x)\in C_j}\varphi }(x)$$

(11)

By substituting Equation (11) into Equation (10), the distance between sample x_i and clustering center μ_j can be further solved as follows:

$$\begin{array}{l}d(x_i,{\mu }_j)=\varphi {(x_i)}^2-{\displaystyle \frac{2}{\mid C_j\mid }}{\displaystyle \sum _{\varphi (x)\in C_j}\varphi }(x_i)\varphi (x)+{\displaystyle \frac{1}{\mid C_j{\mid }^2}}{\displaystyle \sum _{\varphi (x_1)\in C_j}{\displaystyle \sum _{\varphi (x_2)\in C_j}\varphi }}(x_1)\varphi (x_2)\\ =K(x_i,x_i)-{\displaystyle \frac{2}{\mid C_j\mid }}{\displaystyle \sum _{\varphi (x)\in C_j}K}(x_i,x)+{\displaystyle \frac{1}{\mid C_j{\mid }^2}}{\displaystyle \sum _{\varphi (x_1)\in C_j}{\displaystyle \sum _{\varphi (x_2)\in C_j}K}}(x_1,x_2)\end{array}$$

(12)

The equation above illustrates that the approach has the capability to bypass the mapping function and perform calculations only based on the kernel function. In this study, the Gaussian kernel function is employed as $K(x,z)=\exp (-\gamma \parallel x-z{\parallel }^2)$.

2.3.3. Evaluation Criteria

A desirable clustering result is defined by a high level of similarity within clusters and a low level of similarity between clusters. To assess clustering effectiveness based on this principle, this study employs the Davies–Bouldin Index (DBI) and Dunn Index (DI) [33].

$$DBI={\displaystyle \frac{1}{k}}{\displaystyle \sum _{i=1}^k\underset{j=1,\dots ,k,j\ne i}{\max }}({\displaystyle \frac{a\nu g(C_i)+a\nu g(C_j)}{d_{cen}({\mu }_i,{\mu }_j)}})$$

(13)

$$DI=\underset{1\le i\le k}{\min }\{\underset{j=1,\dots ,k,j\ne i}{\min }({\displaystyle \frac{d_{\min }(C_i,C_j)}{\underset{1\le l\le k}{\max }diam(C_l)}})\}$$

(14)

Let C denote the clustering result, dist(x_i,x_j) denote the distance between two samples, avg(C) denote the average distance between samples within a cluster, diam(C) denote the maximum distance between samples within a cluster, d_min(C_i,C_j) denote the minimum distance between samples in clusters C_i and C_j, and d_cen(C_i,C_j) denote the distance between the centers of clusters C_i and C_j.

The expressions of DBI and DI are examined as follows: a smaller avg(C) suggests greater similarity within clusters, while a larger d_cen(C_i,C_j) indicates lower similarity between clusters. Therefore, a lower DBI value is more desirable, as it signifies higher intra-cluster similarity and lower inter-cluster similarity. Conversely, a larger d_min(C_i,C_j) implies lower inter-cluster similarity, and a smaller diam(C) indicates higher intra-cluster similarity. Therefore, a higher DI value is preferable, as it indicates lower similarity between clusters and higher similarity within clusters.

3. Multi-Level Attention Parallel Prediction Model

3.1. Transformer–Multi-Head Attention

The data intended for loading demonstrates significant nonlinearity, time-varying characteristics, and complex seasonal periodicity. Conventional load forecasting models are insufficient in capturing these complex features effectively. Consequently, a sampling method has been devised to mirror the structure of the Transformer encoder, leveraging the multi-attention mechanism. This method capitalizes on the Transformer model’s exceptional ability to acquire remote dependencies, enabling the capture of complex external features and nonlinear relationships within PV data. By integrating these aspects into the prediction model, the accuracy of the predictions is improved. In Domhan et al. [34], the primary component within the Transformer model is the encoder layer. To streamline the model, this study focuses only on the encoder layer’s structure, which comprises the multi-head attention mechanism and feed-forward neural network. The configuration of the Transformer encoder structure is illustrated in Figure 1.

Self-attention is a mechanism that modulates the importance of individual features in the output by evaluating the relevance of features within the input sequence. The multi-attention approach amplifies the model’s capacity for representation by conducting several self-attention operations concurrently, enabling the capture of diverse characteristics of the sequence data across different subspaces.

The multi-head attention mechanism update equation can be expressed as follows:

$$\begin{array}{l}\mathrm{Attention}(Q,K,V)=\mathrm{softmax}\left({\displaystyle \frac{Q{K}^T}{\sqrt{d_k}}}\right)V\\ {\mathrm{head}}_i=\mathrm{Attention}(Q_i,K_i,V_i),\hspace{1em}i=1,\cdots ,n\\ \mathrm{MultiHead}(Q,K,V)=\mathrm{Concat}({\mathrm{head}}_1,\cdots ,{\mathrm{head}}_n)W^O\end{array}$$

(15)

where Q, K, and V denote the query, key, and value matrices, respectively; d_k denotes the dimension of the key vector; and W^O denotes the linear transformation matrix that transforms the result of the attention computation.

Subsequently, the attention mechanism extracts features that undergo additional extraction and transformation through a feed-forward neural network. This integration of feature information from diverse sources amplifies the model’s nonlinear and expressive capabilities. The equation can be expressed as follows:

$$\mathrm{FFN}(x)=\max (0,x{W}_1+b_1)W_2+b_2$$

(16)

To address the issue of gradient vanishing in the training process, a residual connection is implemented, where the output of each sublayer is added to its input. Additionally, layer normalization is incorporated to improve the stability of the training procedure.

3.2. BiGRU-Global Attention

The transformer model demonstrates improved performance in extracting information from long sequences; however, it may sacrifice many local extraction details. For PV power-related time data, there exists a bidirectional flow of information that encompasses both forward and backward temporal relationships. To delve deeper into the temporal dynamics among PV power sequences, this research leveraged the BiGRU model for PV power prediction. The BiGRU model amalgamates the strengths of GRU and bidirectional Recurrent Neural Networks (RNNs), enabling the utilization of past and future sequence information for model training [35]. This model exhibits superior nonlinear expression capabilities and improves prediction accuracy for time series data by effectively incorporating both preceding and succeeding information.

The structure of the BiGRU model (Figure 2) comprises the input, forward propagation, back propagation, and output layers. The forward and back propagations offer the historical and future time correlation information of the input PV data sequence, respectively. The detailed configuration is outlined as follows:

$$\mathrm{forward} \mathrm{GRU}: \overrightarrow{h_t}=\mathrm{GRU}(x_t,\overrightarrow{h_{t-1}})$$

(17)

$$\mathrm{backward} \mathrm{GRU}: \overleftarrow{h_t}=\mathrm{GRU}(x_t,\overleftarrow{h_{t+1}})$$

(18)

$$\mathrm{output}: h_t=[\overrightarrow{h_t};\overleftarrow{h_t}]$$

(19)

where x_t denotes the tth element of the input sequence; $\overrightarrow{h_t}$ and $\overleftarrow{h_t}$ denote the hidden states of the forward and backward GRUs, respectively; and h_t denotes the output of BiGRU.

The BiGRU structure demonstrates effectiveness in capturing short-term dependencies, but it may prove insufficient for long-range dependencies. The incorporation of the Global Attention mechanism [36] enables the model to adaptively modulate its reliance on global information at every time step via the attention weight matrix, thereby enhancing its ability to capture long-range dependencies and complex patterns within the load data. The Global Attention framework is depicted in Figure 3.

Global Attention initially computes the correlation between each time step based on Equations (20) and (21), utilizing the BiGRU output. It dynamically modifies the weights of each time step, facilitating the model to amalgamate global information in the prediction.

$$e_{tj}=\mathrm{score}(h_t,h_j)$$

(20)

$${\alpha }_{tj}={\displaystyle \frac{\exp (e_{tj})}{{\displaystyle \sum _{k=1}^T\exp }(e_{tk})}}$$

(21)

Subsequently, the context vector is calculated using Equation (22) to incorporate global information into the prediction of the current time step, thereby resulting in the final output y_t.

$$c_t={\displaystyle \sum _{j=1}^T{\alpha }_{tj}}{h}_j$$

(22)

$$y_t=\tanh (W[c_t;h_t]+b)$$

(23)

The incorporation of the Global Attention mechanism enables the utilization of global contextual information in load forecasting, particularly for PV data. Additionally, it allows for the adjustment of the impact of each time step based on the input data characteristics. This improvement increases the model’s flexibility and adaptability, thereby improving its efficiency in addressing the distinctive attributes inherent in PV data.

3.3. Cross Attention

The Cross Attention mechanism integrates the outputs of various models (Transformer and BiGRU–Global Attention) [37], allowing the model to leverage multiple features concurrently. This improves the feature representation of the model, enabling it to capture more complex patterns of PV power variations. Additionally, the Cross Attention mechanism facilitates the model in dynamically regulating the significance of different features, thereby enhancing the adaptability and resilience of the model across various PV power scenarios. The structure of the Cross Attention mechanism is illustrated in Figure 4.

The Cross Attention equation is expressed as follows:

$$\mathrm{CrossAttention}(Q,K,V)=\mathrm{softmax}\left({\displaystyle \frac{Q{K}^T}{\sqrt{d_k}}}\right)V$$

(24)

where Q denotes the output of one module, while K and V denote the output of another module. The results of Equation (25) for Transformer and Equation (26) for BiGRU–Global Attention are combined in the Cross Attention Equation (27) to yield Equations (28) and (29). Subsequently, the ultimate output Hfusion is obtained through fusion.

$$H_T=\mathrm{Transformer}(X)$$

(25)

$$H_B=\mathrm{BiGRUGlobal}(X)$$

(26)

$$H_{TB}=\mathrm{CrossAttention}(H_T,H_B,H_B)$$

(27)

$$H_{BT}=\mathrm{CrossAttention}(H_B,H_T,H_T)$$

(28)

$$H_{fusion}=\mathrm{Concat}(H_{TB},H_{BT})$$

(29)

Finally, Cross Attention features are utilized in conjunction with the average pooling layer to reshape the tensor, ultimately generating the final prediction through the fully connected layer.

3.4. Framework of This Paper

Combining the benefits of various models, a multi-level attention parallel prediction model was developed. This model leverages a parallel structure to predict targets at multiple time steps concurrently, ensuring accuracy, and accelerating the training and inference processes. The proposed model improves the extraction of long-term dependencies and improves the perception of temporal features in multi-feature sequences, leading to increased accuracy and reduced computational complexity compared to traditional ensemble models and Transformer model variations. The prediction framework is depicted in Figure 5.

4. Calculus Analysis

4.1. Data Sources

In this study, the calculation example selected PV data from a 150 kWp monocrystal-line silicon photovoltaic system located in northwest China, covering the period from 1 January 2020 to 30 December 2020, with a sampling time interval of 15 min. Input parameters for the gray correlation degree model included irradiance, temperature, humidity, wind speed, wind direction, and air pressure. Subsequently, based on the results in Table 1, irradiance, temperature, wind speed, and humidity were selected as the model inputs. The dataset was categorized into three climate types, i.e., sunny, cloudy, and rainy days, identified through a process of similar day analysis and clustering. In this paper, the PyTorch 2.0.0 framework was used to build the model, and the processor was an Intel(R) Core i5-10400 CPU @2.9 GHz with SAMSUNG 16 GB of RAM (Related equipment are purchased from Daejeon, South Korea). The prediction of real PV power was carried out using the Transformer, BiGRU, BiGRU–Global attention, CNN-LSTM, NARX, informer, Crossformer, and Transformer–BiGRU–Global Attention–Cross attention (multi-level attention model) models.

4.2. Clustering Impact Analysis

Figure 6 illustrates that the improved ISODATA algorithm demonstrated the lowest DBI value and the highest DI value, indicating superior clustering performance. In contrast, the DBI and DI values of the traditional ISODATA and K-means algorithms exhibited similarity, with the traditional ISODATA algorithm slightly surpassing K-means. This can be attributed to the fundamental resemblance between the two algorithms, with the former incorporating cluster splitting and merging procedures. As both algorithms yield the same number of clusters, their clustering results were comparable. However, the improved ISODATA algorithm transforms the data from the original feature space to a high-dimensional space using the kernel method, enhancing the linear separability of the load curve. Furthermore, the more judicious selection of initial clustering centers aids in converging towards the global optimum. Therefore, the improved ISODATA algorithm achieved the most effective clustering performance.

Based on the data in

Table 2. Clustering effect analysis.

Algorithm	DBI	DI	Time
K-means	0.156	0.020	0.670
ISODATA	0.123	0.033	0.860
Improved ISODATA	0.057	0.087	0.530

(derived from observations on cloudy days), the improved ISODATA algorithm demonstrated the shortest runtime for clustering, followed by K-means. This result can be attributed to two primary factors. Firstly, the traditional ISODATA algorithm involves split and merge operations, necessitating additional computations. These operations entail calculating distances from samples to clustering centers, distances between clustering centers, and the standard deviation of the samples, leading to longer processing times compared to the improved ISODATA algorithm. Secondly, the improved ISODATA algorithm improves the initial clustering center selection strategy, resulting in quicker convergence and reduced processing times when contrasted with the traditional ISODATA algorithm. Furthermore, the improved ISODATA algorithm utilizes kernel functions, obviating the necessity for data mapping. By leveraging pre-computed kernel function values between data points through a lookup table method, computations are further expedited, resulting in faster runtimes than those achieved by K-means. In summary, the improved ISODATA algorithm, which does not require predetermined cluster numbers, is particularly well-suited for scenarios involving clustering large datasets with high dimensionality, especially in load profile clustering. Its utilization of kernel functions facilitates mapping to high-dimensional feature spaces, thereby enhancing load profile classification. Additionally, the optimized initial clustering center selection reduces processing times, leading to improved performance. Consequently, the improved ISODATA algorithm emerges as a load profile clustering technique that excels in both clustering effectiveness and efficiency.

4.3. Comparative Analysis of Predictive Models

To assess the efficacy of the models, this study conducted a comparative analysis of the prediction results generated by each model. Figure 7 illustrates the prediction results of each model under the three weather conditions.

For model evaluation, various metrics such as the Root Mean Square Error (RMSE), Mean Squared Error (MSE), Mean Absolute Error (MAE), Integrated Absolute Error (IAE), Relative Error (RE), and Coefficient of Determination (R²) were selected to assess and validate the predictive accuracy of the model. Furthermore, to improve the assessment of model quality, a benchmark model was developed based on the optimal convex combination of climatology and persistence methods [38,39]. Subsequently, the model’s performance was evaluated using the skill score as follows:

$$\mathrm{Skill} \mathrm{Score}=1-{\displaystyle \frac{\mathrm{Prediction} \mathrm{model}’\mathrm{s} \mathrm{RMSE}}{\mathrm{Benchmark} \mathrm{model}’\mathrm{s} \mathrm{RMSE}}}$$

(30)

Figure 7 and

Table 3. Comparison of prediction results.

Weather	Model	Pavg	RMSE	MSE	MAE	R2	IAE	RE	Score
Sunny	BiGRU	46.622	3.388	11.479	2.429	0.995	218.717	40.953	0.221
	Transformer	45.867	3.512	12.335	2.934	0.995	265.230	63.827	0.193
	BiGRU–Global Attention	45.393	3.186	10.153	2.651	0.996	199.101	38.190	0.268
	CNN-LSTM	46.166	3.393	11.511	2.682	0.995	244.016	46.183	0.220
	NARX	46.312	3.307	10.936	2.639	0.995	240.162	53.984	0.240
	informer	46.078	3.324	11.046	2.806	0.995	255.299	62.639	0.236
	Crossformer	45.792	3.178	10.101	2.498	0.996	227.352	54.673	0.270
	Multi-level attention	45.565	3.154	9.945	2.053	0.996	184.917	25.730	0.275
Cloudy	BiGRU	24.297	4.221	17.821	3.184	0.975	283.404	45.191	0.037
	Transformer	25.357	4.189	17.547	3.092	0.975	289.245	53.719	0.045
	BiGRU–Global Attention	23.627	4.049	16.397	2.691	0.977	242.632	31.818	0.077
	CNN-LSTM	23.995	4.099	16.800	2.902	0.976	264.090	37.420	0.065
	NARX	24.351	4.440	19.713	3.019	0.976	293.897	44.912	−0.013
	informer	24.492	3.951	15.614	2.865	0.978	260.734	40.393	0.099
	Crossformer	24.812	3.758	14.124	2.688	0.980	244.639	38.199	0.143
	Multi-level attention	24.082	3.727	13.889	2.198	0.980	196.012	18.979	0.150
Rainy	BiGRU	13.953	3.746	14.031	2.644	0.959	232.934	41.589	0.042
	Transformer	13.536	3.703	13.709	2.646	0.959	231.802	44.003	0.053
	BiGRU–Global Attention	14.923	2.843	8.082	2.049	0.976	185.543	43.276	0.273
	CNN-LSTM	14.471	3.469	12.032	2.461	0.965	223.913	47.399	0.113
	NARX	14.008	3.941	15.528	2.708	0.954	246.430	39.696	−0.008
	informer	14.230	3.075	9.455	2.140	0.972	194.692	38.050	0.214
	Crossformer	14.403	2.822	7.965	1.974	0.977	179.616	35.327	0.278
	Multi-level attention	13.913	2.788	7.774	1.692	0.977	150.891	24.904	0.287

indicate that the PV output power curve exhibited smoother and slight fluctuations on sunny days. Therefore, each model aligned more closely with the actual value curve. However, the model introduced in this study offers different advantages over the base model.

In instances of overcast and rainy weather conditions, the power curves exhibited varying trends and reduced regularity, leading to decreased accuracy in each model. However, the Transformer model demonstrated superior extraction of global information due to its inherent attention mechanism. While the prediction accuracy for sunny days may be inferior to that of BiGRU, the Transformer model outperformed BiGRU in predicting phases with high volatility. Subsequently, the introduction of Global Attention in BiGRU confers certain advantages over both the base BiGRU and Transformer models. For instance, in the context of rainy days, the evaluation metrics of the MSE, RMSE, and MAE showed a 41.05%, 23.22%, and 22.56% improvement, respectively, in the performance of BiGRU–Global Attention compared to Transformer. This improvement underscores the beneficial impact of the global information provided by the Global Attention mechanism on enhancing prediction accuracy. Furthermore, when compared to the BiGRU–Global Attention model in this study, it exhibited improved stability and accuracy across all the weather conditions, as evidenced by the lower IAE and RE values and increased MSE, RMSE, and MAE values by 3.82%, 1.93%, and 17.41%, respectively, in rainy weather conditions. The combined model’s effectiveness is thus validated.

To further evaluate the performance of the proposed model, a comparison was made with the Crossformer model, which has demonstrated effectiveness in time series prediction in recent years. The results showed that on rainy days, the MSE, RMSE, and MAE increased by 2.40%, 1.21%, and 14.26%, respectively. This suggests that the proposed model exhibited lower error rates in predicting PV power. Particularly in cases of significant errors, the proposed model outperformed the Crossformer model. The higher R² value indicates that the model could better elucidate power fluctuations and had a stronger correlation with actual values. The low IAE and RE values signify high prediction accuracy, especially when the actual values approached zero. Furthermore, the superior skill scores provide additional evidence of the model’s efficacy. In conclusion, the model introduced in this study excels in capturing long-term dependencies and global information through a multi-level attention mechanism, enabling the simultaneous processing of multi-feature sequences. Its key strengths include robustness, adaptability, and improved accuracy and stability in predicting PV output.

5. Conclusions

To improve the ability to capture PV time series fluctuations and improve prediction accuracy, this study proposed a multi-level attention prediction model based on similar days, considering the impact of weather factors. Combined with historical PV data for simulation and analysis, the following conclusions can be drawn:

(1)

The training set data were selected through a similar day analysis, reducing the complexity of the model training set and shortening the prediction time.

(2)

An improved ISODATA algorithm was proposed to optimize the configuration of initial clustering centers and Euclidean distances, enhancing the clustering impact and speed. This further optimizes the similar day dataset and guarantees prediction accuracy.

(3)

A parallel prediction model with a multi-level attention mechanism was proposed to synthesize the advantages of networks such as Transformer, BiGRU, Global Attention, and Cross Attention. This allows the model to integrate information from different feature subspaces, achieving better prediction results under various climate types compared to the basic model, as validated experimentally.

(4)

The forecasting method in this study focused on the temporal correlation of PV series. Future research will consider mining and utilizing more hidden spatial correlation information from the historical load series to further improve forecasting accuracy.

To address PV fluctuation issues, the multi-level attention prediction model proposed in this study improved THE PV prediction accuracy and demonstrated practical applicability. This can provide a data basis for power system planning and scheduling, thereby enhancing the stability of the power system.