An Operation Mode Analysis Method for Power Systems with High-Proportion Renewable Energy Integration Based on Autoencoder Clustering

Abstract

With the integration of high-proportion renewable energy, the operation modes of the power system are becoming increasingly complex and diverse. The typical operation modes selected with manual experience cannot comprehensively represent system operating characteristics. To more accurately analyze system operating characteristics, an analysis method for power system operation modes based on autoencoder clustering is proposed. Compared to other clustering methods, the autoencoder clustering method can adapt to data of different types and structures, extract features and perform clustering in a reduced-dimensional space, and suppress noise in the data to a certain extent. First, multi-dimensional analysis metrics for power system operation modes are proposed. The metrics are used to evaluate system characteristics such as cleanliness, security, flexibility, and adequacy. The evaluation metrics for clustering are designed based on the metrics. Second, an operation mode analysis framework is constructed. The framework uses an autoencoder to extract implicit coupling relationships between system operation variables. The encoded feature vectors are used for clustering, which helps to find the internal similarities of the operation modes. Regulation resources such as pumped hydro storage are also considered in the framework. Finally, the proposed method is tested on the IEEE 39-node system. In the test, the comparison of clustering evaluation metrics and operation mode analysis errors shows that the proposed method has the best clustering performance and operation mode analysis effect compared to other clustering methods. The results prove that the proposed method can effectively extract the inner correlations and coupling relations of high-dimensional operating vectors, form consistent operation mode clusters, select typical operation modes, and accurately assess the characteristics and risks of the power system with high-proportion renewable energy integration. This paper helps to build a stronger power system that can integrate a higher proportion of renewable energy, replace fossil fuel generation, and contribute to a higher level of sustainable development.

1. Introduction

With the accelerating pace of energy transition, the energy structure on the supply side of the power system is gradually shifting from traditional fossil fuels to renewable energy sources, such as wind and solar power. Due to the significant volatility and uncertainty of wind and photovoltaic power, their large-scale integration poses severe challenges to the security of the power system [1,2]. For instance, on 28 April 2025, a major power outage occurred in Spain and Portugal, causing the Iberian Peninsula power grid to disconnect from the European mainland power grid. This outage was the most serious incident to occur in the European power system in over 20 years, with a major impact on citizens and society in Spain and Portugal [3]. It highlights the complexity and the constant security challenges of power system operation posed by high-proportion renewable energy integration, as well as the necessity of the operation mode analysis [4]. In addition to security, the power system also faces severe challenges in terms of adequacy, flexibility, and economy [5,6].

Power system operation mode analysis can select typical operation modes and evaluate their operational characteristics [7]. It can be used to provide a basis for system operation arrangement and control strategy formulation. In traditional power systems, the load and hydropower have relatively fixed variation patterns as seasons change, so seasons are often used as the basis for selecting typical operation modes, i.e., “summer peak/summer valley” and “winter peak/winter valley”. However, after a large proportion of renewable energy integration, due to the uncertainty of power generation, the operation modes are becoming increasingly complex and no longer follow fixed variation patterns with seasonal changes. In this case, the typical operation modes selected by traditional methods with manual experience have limitations in comprehensively characterizing the power system [8,9].

Due to the disadvantages of traditional methods, data-driven methods are widely applied [10,11,12]. In data-driven methods, the power system operation mode is treated as a vector composed of active power, reactive power, voltage, phase angle, and other data, converting operation mode analysis to a high-dimensional nonlinear data analysis problem [13]. For example, in [14], an analysis framework is constructed to analyze operating data, e.g., weather data, equipment status, fault information, and load data. In [15], the operating vectors include processed results from the historical operation data, equipment monitoring data, and equipment type and status data. In [16], load level and the output level of wind and solar power are used to represent power system operation modes. In [17], system operating vectors are composed of voltage magnitude, phase, generation active power, generation reactive power, load active power, and load reactive power. In existing research, clustering analysis is widely used in typical operation mode extraction and analysis due to its ability to reduce the dimension of massive operation data and extract implicit similarities within operation modes [18]. For example, a data-driven operation mode analysis framework for high-proportion renewable energy power systems based on high-dimensional power system operation data is proposed in [19] to identify the pattern of the operation modes and analyze the impact of high renewable penetration. The framework includes steps, such as scenario simulation, data preprocessing, scenario clustering based on k-means++, data dimension reduction, and scenario visualization. Several indices are introduced to quantify space dispersion, time variation, and seasonal consistency of operation modes. An automatic operation mode extraction algorithm based on k-means++ and an improved clustering validity metric is proposed in [20]. The proposed clustering method can consider multi-uncertainty scenario boundaries based on the combination of artificial and automatic algorithms. An improved semi-supervised k-means algorithm for running section similarity matching is proposed in [21]. The algorithm can perform similarity clustering on historical operation sections to obtain effective samples and reduce the data scale. Then, the proposed method is used to find the most valuable historical operation section similar to the current system operation section. The decision information corresponding to the current operation section can be obtained. Based on convolutional neural networks and self-attention mechanisms, a clustering extraction method for typical operation modes is proposed in [22]. The method can effectively explore the spatial correlation and complex combination patterns of high-dimensional operating variables, expand the operation mode sample set represented by the centroid with samples on the cluster boundaries, conduct static security assessment, and evaluate security risks of the power system. An improved fuzzy C-means (FCMs) clustering method embedded with multi-scale key feature information is proposed in [23]. The method can be used in the similarity identification of a power grid transmission section. A transmission section limit transfer capacity (TTC) calculation model considering thermal stability and transient angle stability constraints is also proposed. The cosine distance is introduced to consider the sample morphological features. A power system operation state identification method based on particle swarm optimization (PSO) and convolutional neural networks (CNNs) is proposed in [24], where different feature subsets are constructed with system operating variables based on knowledge and experience. The feature subsets are clustered into categories representing different power system operating states with the proposed method. Considering the existence of extreme imbalance in state categories in actual power systems, an algorithm is introduced to adjust the weight of loss functions for different categories of the model to improve the identification effect. In [25], an identification method for key transmission sections is proposed based on an improved Girvan–Newman algorithm, as well as an online analysis method for the maximum transfer capability of the transmission sections based on graph convolutional networks. In the proposed method, CLIQUE grid clustering is used to select typical operation modes representing common operating states, as well as minority operation modes representing special or uncommon operating states. The selected operation modes are used for transfer learning to improve the online analysis effect. Existing power system operation mode analysis methods mainly focus on system operation security. Other characteristics, such as cleanliness, flexibility, and adequacy, are seldom considered, which are also useful to comprehensively characterize the operational characteristics of power systems with high-proportion renewable energy. Moreover, most existing studies only consider generation and load conditions, such as wind power, photovoltaic, and thermal power, without considering system regulation resources, such as electrical energy storage and pumped hydro storage. As these regulation resources become increasingly widespread in power systems, their impact on power flow distribution and system characteristics should be considered.

Autoencoder clustering (AEC) is a widely used clustering method based on neural networks. Compared to other clustering methods, e.g., centroid-based, hierarchical, distribution-based, density-based, fuzzy clustering methods, and other clustering methods based on deep learning, AEC can automatically learn “informative” representations of the data, extract features, perform clustering in a reduced-dimensional space, and consider the data regeneration quality [26,27]. AEC also has merits such as adaptation to data of different types and structures and suppressing noise to a certain extent [28]. Reference [29] performs a systematic introduction to the autoencoder (AE), discussing its mathematics and fundamental concepts, as well as applications. The paper points out that the use of AE for dimensionality reduction has one main advantage over other methods, e.g., principal component analysis (PCA), from a computational point of view: it can deal with a very large amount of data efficiently since its training can be done with mini-batches. A temporal AE model based on CNN and bidirectional long short-term memory networks (Bi-LSTMs) is proposed in [30]. The method uses AEC to extract the temporal characteristics of the operation data to obtain typical microgrid scenarios. In the case study, the proposed AEC-based method is compared to other methods, i.e., k-means, hierarchical clustering, Gaussian mixture model, and PCA + k-means, and performs better clustering effects. An AEC framework for a compressed and informative representation of financial time series is introduced in [31]. By measuring the performance of cluster techniques with reduced data, the proposed AEC-based method shows superiority over other methods. The results prove that AEC can not only be used to decrease the dimensionality of intraday prices but also to reduce large datasets covering different prices from diverse instruments and markets. An AEC-based unsupervised technique is proposed in [32] to cluster power quality events into categories, enabling filtering of anomalous waveforms from recurring or normal waveforms. The test results demonstrate that the AEC-based technique can help to investigate a large number of captured events in a quick manner and filter the massive amount of recorded power quality waveforms.

Based on the AEC model, this paper proposes a power system operation mode analysis method. Main contributions include:

• Operation mode analysis metrics, i.e., renewable energy penetration rate, renewable energy utilization rate, line overload rate, and spinning reserve rate, are proposed. These metrics evaluate the multi-dimensional characteristics of the power system. Based on the analysis metrics, clustering evaluation metrics are designed.
• An AEC-based power system operation mode analysis framework is established. The framework uses AE for the dimensional reduction and transformation of system operating vectors. The obtained feature vectors are clustered, which are used to extract typical system operation modes. The proposed method considers system regulation resources such as pumped hydro storage.
• The proposed method is tested on the IEEE 39-node system. Results show that the proposed method can effectively extract the correlation and coupling relationships among high-dimensional power system operation variables, select typical system operation modes, and accurately assess characteristics and risks of power system operation.

The main structure of this paper is as follows. Multi-dimensional analysis metrics for power system operation modes are proposed in Section 2. An analysis framework for power system operation modes based on AEC is established in Section 3. The detailed algorithm of the framework is introduced in Section 4. Section 5 presents case studies to demonstrate the effectiveness of the proposed method.

2. Multi-Dimensional Analysis Metrics for Operation Modes of Power Systems with High-Proportion Renewable Energy

Compared with traditional power systems, power systems with high-proportion renewable energy operate under more constraints due to the uncertainty of renewable energy and the variety of operation modes. In traditional power systems, only security is usually considered; yet in power systems with high-proportion renewable energy, the analysis of their operation modes needs to consider multiple dimensions besides security, such as cleanliness, flexibility, and adequacy [5,33]. In this section, four analysis metrics are proposed to measure the multi-dimensional operating characteristics of the power system.

2.1. Renewable Energy Utilization Rate

The renewable energy utilization rate refers to the ratio of grid-connected power to the maximum power of wind turbines, photovoltaics, and other renewable energy sources, i.e.,

$$U{R}_m={\displaystyle \frac{{\displaystyle \sum P_m^{wa}}+{\displaystyle \sum P_m^{sa}}}{{\displaystyle \sum P_m^w}+{\displaystyle \sum P_m^s}}}$$

(1)

where $P_m^{wa}$ and $P_m^{sa}$ represent vectors composed of the grid-connected power of the wind turbine and the photovoltaic in the m-th sample, respectively, and $P_m^w$ and $P_m^s$ represent vectors of the maximum output power determined by the primary energy, i.e., wind and solar.

UR measures the integration capacity for renewable energy sources of the power system and characterizes its cleanliness. With more renewable energy integrated into the power system, UR is becoming more critical and is selected as one of the three primary risk assessment indices in [5].

2.2. Renewable Energy Penetration Rate

The renewable energy penetration rate refers to the ratio of the grid-connected power of renewable energy sources to the total generation power in the power system. The metric is used to characterize the power supply-side structure of the power system. It is one of the most commonly used metrics to characterize the cleanliness of the power system, i.e.,

$$P{R}_m={\displaystyle \frac{{\displaystyle \sum P_m^{wa}}+{\displaystyle \sum P_m^{sa}}}{{\displaystyle \sum P_m^t}+{\displaystyle \sum P_m^{wa}}+{\displaystyle \sum P_m^{sa}}}}$$

(2)

where $P_m^t$ represent the vector composed of thermal unit power.

2.3. Line Overload Rate

The line overload rate is the proportion of overloaded lines to the total number of transmission lines in the power system. The metric is used to characterize the overall safety of the operation mode, i.e.,

$$O{R}_m={\displaystyle \frac{\left|O{L}_m\right|}{\left|L_m\right|}}$$

(3)

where $L_m$ and $O{L}_m$ represent the sets composed of all operating lines and heavily loaded lines in the system, respectively.

$$O{L}_m=\left\{k|P_m^k>\alpha {\overline{P}}^k\right\}$$

(4)

where k represents a transmission line; $P_m^k$ and ${\overline{P}}^k$ represent the absolute value of the power flow of the line in the operation mode and its rated power, respectively; and $\alpha$ is the overload factor, typically set as 80% or 90% in practical applications.

2.4. Spinning Reserve Rate

The spinning reserve rate is the ratio of the spinning reserve capacity of operational thermal power units to the total load. The metric is used to characterize the flexibility and adequacy of the operation mode, namely

$$RV{R}_m={\displaystyle \frac{{\displaystyle \sum \left(s_m\odot {\overline{P}}_m^t-P_m^t\right)}}{{\displaystyle \sum P_m^l}}}$$

(5)

where ${\overline{P}}_m^t$ is the generation capacity of the thermal power unit; $s_m$ is a vector composed of the operating status of the power unit: the corresponding element is 1 when the power unit is operating, otherwise it is 0; $P_m^l$ is the load vector; and $\odot$ is the Hadamard product of the two vectors.

3. An Analysis Framework for Power System Operation Mode Based on AEC

The operation mode analysis framework for power systems with high-proportion renewable energy based on AEC is shown in Figure 1.

The framework shown in Figure 1 mainly includes four steps:

• Obtain operation mode samples of the power system. These samples may come from historical or simulated data. The historical data is obtained from the dispatching automation system, e.g., SCADA data, WAMS data, or day-ahead generation arrangement. The simulated data is generated by random production simulation for day-ahead or long-term applications, e.g., power system analysis, risk assessment, and grid planning. To ensure the quality of operation mode analysis, the criteria for data selection mainly include two rules: (a) The power grid topology should be similar. If dramatic changes occur to the power grid, the data corresponding to the scenario should not be included in the sample set. (b) The quality of data should be relatively good, i.e., no significant data omissions or noise.
• Construct operating vectors for the operation mode samples. Select the power of wind turbines, photovoltaics, thermal power units, load, and the two-way power of pumped hydro storage in the operation mode data to form the operating vector. Each operating vector corresponds to one operation mode. To prevent variables with large numerical ranges from dominating the clustering results, variables in the operating vectors are normalized, where the supply-side data and load data are normalized separately.
• Use the sample data to train an AE. The trained AE can extract spatial features from the operating vectors and generate feature vectors in a reduced-dimensional space. The feature vectors represent deep-level characteristics of power system operation. Cluster the feature vectors with the k-means clustering method. Use the clustering evaluation metrics to assess the clustering effect, and select the optimal AE training hyperparameters and clustering number to minimize the clustering evaluation metrics.
• Obtain the clusters of operating vectors with the clustering result of feature vectors. Use the operation modes of centroids to select typical operation modes of the system. Evaluate the operational characteristics of the power system using multi-dimensional analysis metrics, i.e., renewable energy penetration rate, renewable energy utilization rate, line overload rate, and spinning reserve rate. The evaluation results can serve as representatives of system operating characteristics.

4. Analysis Method of Power System Operation Mode Based on AEC

4.1. Operating Vector Formation

In the power system, there are numerous devices, including generators, transmission lines, breakers, switches, transformers, loads, buses, and pumped hydro storage. The operation mode of the system can be described by various electrical variables such as unit power, load, line flow, node voltage, and phase angle. Considering the representativeness and independence of these variables, variables such as unit power, load, and bidirectional power of pumped hydro storage are selected as characteristic variables to form the operating vector, i.e.,

$$P_m=\left\{P_m^t,P_m^w,P_m^s,P_m^l,P_m^{hin},P_m^{hout}\right\}$$

(6)

where $P_m$ is the operating vector of the m-th sample; and $P_m^w$, $P_m^s$, $P_m^{hin}$, $P_m^{hout}$ are vectors composed of maximum wind power, maximum photovoltaic power, and pumped hydro storage of the sample, respectively.

$$P_m^t=\left\{P_{m,1}^t,P_{m,2}^t,\dots ,P_{m,n_t}^t\right\}$$

(7)

$$P_m^w=\left\{P_{m,1}^w,P_{m,2}^w,\dots ,P_{m,n_w}^w\right\}$$

(8)

$$P_m^s=\left\{P_{m,1}^s,P_{m,2}^s,\dots ,P_{m,n_s}^s\right\}$$

(9)

$$P_m^l=\left\{P_{m,1}^l,P_{m,2}^l,\dots ,P_{m,n_l}^l\right\}$$

(10)

$$P_m^{hin}=\left\{P_{m,1}^{hin},P_{m,2}^{hin},\dots ,P_{m,n_h}^{hin}\right\}$$

(11)

$$P_m^{hout}=\left\{P_{m,1}^{hout},P_{m,2}^{hout},\dots ,P_{m,n_h}^{hout}\right\}$$

(12)

where $P_{m,i}^t$, $P_{m,i}^w$, $P_{m,i}^s$, $P_{m,i}^l$, $P_{m,i}^{hin}$, $P_{m,i}^{hout}$ are the power of the i-th generation unit/load/pumped hydro storage, respectively; and $n_t$, $n_w$, $n_s$, $n_l$, $n_h$ are the total numbers of the corresponding devices.

It should be noted that in the metric calculation, the grid-connected renewable energy power is used in the numerator in Equations (1) and (2), while the maximum power is used in the denominator of UR in Equation (1) and the formation of the operating vector in Equation (6). The reasons for the selection are as follows: (a) In the day-ahead or long-term operation mode analysis, the maximum renewable energy power is usually estimated by weather conditions first, and then the grid-connected power is determined by the integration capability of the power system. Using the maximum power to form an operating vector is more straightforward and practical. (b) Since the operating vector includes power generation of other power units and load, which determines the power flow of the grid, the grid-connected power of the renewable energy is implicitly inherent, and the proposed AEC-based method can extract the inherent features from the operating vector to assess the metrics. (c) From the computational perspective, if replacing the maximum power with the grid-connected power in the operating vector, it will be impossible to calculate UR due to the lack of primary renewable energy information.

After forming the operating vector in Equation (6), to prevent variables with large numerical ranges from dominating the clustering results, each variable in the vector is normalized according to its classifications, i.e., on the power supply side or load side. Two per-unit bases are selected, i.e., the maximal power generation as the per-unit base for the supply side and the maximal load power as the per-unit base for the load side. Then, on the supply side and load side, divide the actual values by the corresponding per-unit bases, obtaining the normalized values, respectively. The normalized operating vector is input into the AEC model.

4.2. AEC Model

AEC is an unsupervised learning method. The core of AEC is AE. AE is a neural network model composed of two parts: an encoder and a decoder, as shown in Figure 2.

In Figure 2, both the encoder and decoder are neural networks with multiple layers, and each layer has a nonlinear structure [23]. The encoder utilizes the nonlinear mapping capability of the neural network to map input vectors into a low-dimensional feature space, extracting deep-level features from input data to form feature vectors. The decoder reconstructs the original input with a reverse mapping of low-dimensional feature vectors with neural networks. When training the AE model, both the encoder and decoder are trained simultaneously by minimizing the mean square error between the input data and reconstructed output data, i.e.,

$$L_e\left(\theta ,\varphi \right)={‖x-g\left(f(x,\theta ),\varphi \right)‖}^2$$

(13)

where $x$ is the input vector; $f(•)$, $g(•)$, and $L_e(•)$ are the encoder, decoder, and error function, respectively; and $\theta$ and $\varphi$ are the parameters of the encoder and decoder, respectively.

In each training iteration step, after calculating the error function $L_e\left(\theta ,\varphi \right)$, its value is used to update the parameters of the encoder and decoder simultaneously, as shown in Figure 2. In the proposed AEC, the operating vector P_m is used as the input of AE. After processing by the encoder, the feature vector h_m in the low-dimensional space is obtained, namely

$$h_m=f(P_m,\theta )$$

(14)

h_m contains the implicit features of P_m. By calculating the feature similarity of h_m, it is clustered into clustering categories. Suppose the feature vectors are clustered into n_j categories, and the corresponding clustering center of h_m is c_j, 1 ≤ j ≤ n_j, then the similarity between h_m and c_j is represented by their Euclidean distance, i.e.,

$$L_c={\displaystyle \sum _m{\displaystyle \sum _{j}s(h_m,c_j){‖h_m-c_j‖}^2}}$$

(15)

where L_c is the clustering error function; $s(h_m,c_j)$ is the membership function: if h_m belongs to the cluster corresponding to c_j, $s(h_m,c_j)$ = 1; otherwise, $s(h_m,c_j)$ = 0.

In the proposed method, the k-means algorithm is used to cluster h_m. Its simplicity, intuitiveness, and efficacy have made the algorithm popular across many domains. In the clustering process, the centroids are first selected randomly. Then, the feature vectors are clustered to the neighboring centroids. Calculate the distances and use them to update the centroids. Iterate the process until the objective function shown in Equation (15) is minimized. After clustering h_m, the cluster of P_m is determined by the cluster of h_m. The centroids in the feature space are mapped to the operating vector space with the decoder, and the nearest sample is selected as the typical operation mode. This process ensures that the typical operation mode satisfies system operating constraints, e.g., power balance and load-flow limits. The elbow method is used to select the optimal clustering number k. By plotting the clustering evaluation metric values against increasing k values, look for a point where the improvement slows down. This point is called the “elbow”, and the k value corresponding to the elbow is selected as the optimal clustering number.

4.3. Clustering Evaluation Metric

Based on the analysis metrics proposed in Section 2, i.e., renewable energy penetration rate, renewable energy utilization rate, line overload rate, and spinning reserve rate, the clustering evaluation metric for operation modes is defined. The metric ${\sigma }_R$ is defined as the fluctuation rate of the analysis metrics, i.e.,

$${\sigma }_R={\displaystyle \frac{{\displaystyle {\sum }_{j=1}^{n_j}\left(R_{j,\max }-R_{j,\min }\right)}}{n_j\left(R_{\max }-R_{\min }\right)}}$$

(16)

where R represents the analysis metrics, i.e., UR, PR, OR, and RVR, respectively; R_max and R_min are the maximum and minimum values of the corresponding metric of all operation modes, respectively; R_j,max and R_j,min are the maximum and minimum values of the corresponding metric of the operation modes in the j-th cluster, respectively; and n_j is the total number of clusters.

If the operation modes within each group are more similar, i.e., $R_{j,\max }$ and $R_{j,\min }$ are closer, ${\sigma }_R$ in Equation (16) is smaller. Therefore, after the operation modes are clustered, a smaller ${\sigma }_R$ states that the internal spatial coupling characteristics of each cluster are more similar, and the consistency of the operation modes is higher, indicating better clustering results. In this case, the typical operation mode can better characterize the operating characteristics of the operation modes within the cluster. Besides ${\sigma }_R$, a more general clustering evaluation metric, i.e., the silhouette coefficient (SC), is also used to evaluate the clustering effect. The silhouette coefficient for the m-th sample is defined as:

$$S{C}_m={\displaystyle \frac{b_m-a_m}{\max \left\{a_m,b_m\right\}}}$$

(17)

where a_m and b_m are the intra-cluster distance and nearest-cluster distance of the sample, respectively. The silhouette coefficient to evaluate the clustering effect is defined as the average value of all the samples, i.e.,

$$SC={\displaystyle \frac{1}{n_m}}{\displaystyle {\sum }_{m=1}^{n_m}S{C}_m}$$

(18)

where n_m is the total number of samples.

Equations (17) and (18) indicate that SC ranges from −1 to +1. A score close to +1 means each sample fits well in its own cluster and is far from others, indicating better clustering results, while a score close to −1 means each sample is in the wrong cluster, indicating worse clustering results.

5. Case Study

5.1. Testing System Introduction

The IEEE 39-node system is used as a test system, with 10 thermal generators, one wind turbine, one photovoltaic unit, and one pumped hydro storage unit connected to the system. Each node in the system is connected to one load. The topology of the test system is shown in Figure 3.

The operation mode samples are simulation data generated by a random production simulation. The wind and solar power are generated with the Markov chain Monte Carlo method, whose parameters are identified from historical data. The historical data comes from the dispatching automation system with a period of 1 year, which is clustered with FCM to form categories with different kinds of fluctuations. Identify the characteristic parameters for each category and use them to build the Markov state transition matrix. The sampling interval is 1 h. The operation modes for the year 2024, i.e., a total of 8784 operation modes, are selected to compose the sample set. Using the operating vector formation method proposed in Section 4.1 to obtain the preprocessed system operating vector. Each vector has a dimension of 53. According to the training performance, the optimal parameter values of AE are obtained. The latent dimension of the AE model is 42, i.e., the exact layer sizes are 53 → 42 → 53. The maximum training epoch is 1000. The parameters of L2 weight regularization, sparse regularization, and sparse proportion are 0.0009, 0.9, and 0.08, respectively. The satlin function is selected as the encoder transfer function. The scaled conjugate gradient algorithm is used as the training function, which adapts its learning parameters dynamically during training. The batch size is 128. The train/validation/test split follows the rate of 70:15:15, and the early stopping criterion is the gradient of less than 1 × 10⁻⁶. The training performance of AE is shown in Figure 4.

In Figure 4, the mean squared error with L2 and sparse regularization decreases gradually in the iteration process until it converges to its best value.

5.2. Comparison of Clustering Effects

The operation mode sample set is clustered with four clustering methods, i.e., the proposed AEC, k-means, Self-Organizing Map (SOM), and PCA + k-means. After clustering, use the clustering evaluation metrics to assess the clustering performance. Under different clustering numbers, the comparison of clustering evaluation metrics is shown in

Table 1. Comparison of clustering evaluation metrics of different clustering methods.

Clustering Number	Clustering Method	${\mathit{\sigma }}_{\mathit{P}\mathit{R}}$	${\mathit{\sigma }}_{\mathit{U}\mathit{R}}$	${\mathit{\sigma }}_{\mathit{O}\mathit{R}}$	${\mathit{\sigma }}_{\mathit{R}\mathit{V}\mathit{R}}$	SC
25	k-means	0.413	0.1165	0.744	0.4054	0.2899
	SOM	0.4085	0.0987	0.72	0.4014	0.1485
	PCA + k-means	0.4188	0.081	0.7307	0.4022	0.2971
	AEC	0.3837	0.0984	0.7227	0.3978	0.5336
100	k-means	0.2763	0.022	0.5767	0.289	0.2683
	SOM	0.296	0.0235	0.5732	0.2779	0.1469
	PCA + k-means	0.3004	0.0235	0.5827	0.2812	0.2614
	AEC	0.2047	0.0224	0.5707	0.2685	0.4352
225	k-means	0.2255	0.0195	0.504	0.2271	0.1341
	SOM	0.2302	0.0187	0.4912	0.2292	0.2392
	PCA + k-means	0.2231	0.0189	0.5076	0.2328	0.238
	AEC	0.1431	0.0168	0.491	0.2165	0.4158
400	k-means	0.1769	0.0147	0.4363	0.1915	0.1262
	SOM	0.1732	0.0147	0.4362	0.1923	0.2405
	PCA + k-means	0.1758	0.0134	0.4365	0.1916	0.2476
	AEC	0.1129	0.0122	0.4325	0.1857	0.3642
484	k-means	0.1556	0.0135	0.4091	0.1784	0.1127
	SOM	0.1631	0.0125	0.4186	0.1822	0.2439
	PCA + k-means	0.1579	0.012	0.4088	0.1781	0.2453
	AEC	0.0951	0.0103	0.4088	0.176	0.3752

. In Table 1, 484 is the optimal clustering number of the SOM. It is calculated by an empirical equation: $5\sqrt{N}$, where N is the sample set size. Given N = 8784, 484 is the closest square number greater than $5\sqrt{N}$, so it is selected as the optimal clustering number of SOM. The clustering evaluation metric of renewable energy penetration rate in Table 1 is chosen as an example, whose value is compared in Figure 5.

In Table 1 and Figure 5, the ${\sigma }_R$ values of AEC are generally the smallest under different clustering numbers, and the SC values are the largest. The ${\sigma }_R$ values of k-means, SOM, and PCA + k-means are similar when compared to AEC, while PCA + k-means usually has larger SC values than k-means and SOM. The test results state that the operation mode characteristics within each cluster are more consistent under the AEC method, indicating a better clustering result. To test the stability of the proposed method, the clustering process is repeated 100 times when the clustering number is 100. The average values and standard deviation of the clustering evaluation metrics are shown in

Table 2. Stability test of clustering evaluation metrics of different clustering methods.

	${\mathit{\sigma }}_{\mathit{P}\mathit{R}}$		${\mathit{\sigma }}_{\mathit{U}\mathit{R}}$		${\mathit{\sigma }}_{\mathit{O}\mathit{R}}$		${\mathit{\sigma }}_{\mathit{R}\mathit{V}\mathit{R}}$
	Avg	Std	Avg	Std	Avg	Std	Avg	Std
k-means	0.3026	0.0088	0.0317	0.0089	0.5867	0.0044	0.2861	0.0040
SOM	0.2953	0.0039	0.0248	0.0015	0.5798	0.0052	0.2794	0.0035
PCA + k-means	0.3032	0.0078	0.0326	0.0105	0.5868	0.0047	0.2862	0.0046
AEC	0.2259	0.0166	0.0226	0.0029	0.5731	0.0048	0.2729	0.0069

.

In Table 2, the average value of each ${\sigma }_R$ size of AEC is the smallest compared to other clustering methods, which is consistent with the test results in Table 1 and Figure 5. Due to the randomness of the neural network, the standard deviation of AEC is also larger. It should be noted that the randomness only happens in the training process. After a trained clustering model with better performance is selected, there will be no randomness in the operation mode analysis process afterwards.

5.3. Optimal Clustering Number Selection

The clustering evaluation metrics of AEC under different clustering numbers are calculated, as shown in Figure 6 and Figure 7. Figure 6 displays the changes in four normalized evaluation metrics with respect to the clustering number. Figure 7 chooses the evaluation metric of renewable energy penetration rate as an example, displaying its value and the difference between the current value and the previous value varying with the clustering numbers.

In Figure 6 and Figure 7, the clustering evaluation metrics decrease as the clustering number increases, indicating that the consistency within each cluster is increasing. The evaluation metric first decreases sharply when the clustering number is relatively small, then the decrease becomes steady when the clustering number is large. The objective of clustering is to identify overall patterns from the operation modes. When the number of clusters is too large, each cluster only contains a few samples, with subtle differences between them. The differences between samples of varied clusters are also very small, leading to decreased separation between clusters. In this case, it is difficult to identify meaningful overall patterns from clustering results, making it hard to interpret and difficult to use. According to the elbow method, when the clustering number exceeds 100, the change in clustering evaluation metrics slows down significantly, indicating that the difference between operation mode samples within the clusters is small and the marginal utility starts to decline. Therefore, 100 is selected as the optimal clustering number for the selected operation mode sample set.

5.4. Clustering Result Analysis

To visually demonstrate the clustering effect of AEC on the system operation modes, two clusters with the largest number of operation modes are selected as examples when the clustering number is 100. The box plots of the distribution of supply-side components in the operating vectors of the two clusters are shown in Figure 8 and Figure 9, respectively. In the figure, Cluster 1 contains 290 operation modes, and Cluster 2 contains 286 operation modes. The thermal generators are marked with Gen1, Gen2, …, and Gen10. The wind turbine unit is marked with Wind, and the photovoltaic unit is marked with Solar. PS_in and PS_out denote the two-way power of the pumped hydro storage unit. The typical operation modes of the two clusters are shown in Figure 10.

In Figure 8, Figure 9 and Figure 10, the operation modes in Cluster 1 mainly represent scenarios where Units 1 and 2 are shut down, while Cluster 2 mainly contains operation modes where Unit 1 operates at its maximal capacity, Unit 2 is shut down, and the renewable power generation is much smaller. The power generation of the other units in Cluster 2 is also smaller than in Cluster 1. The distributions of the two-way power of the pumped hydro storage in the two clusters also show some differences in Figure 8 and Figure 9. The results demonstrate that after AEC, operation modes with similar distribution patterns of wind, solar, thermal, and pumped hydro storage power are aggregated together, while significant differences exist between operation modes of different clusters, resulting in varied power flow distribution and system characteristics.

In the clustering process, the Euclidean distance is used to measure the similarity between operating vectors and clustering centroids, as shown in Equation (15). To test the clustering performance of different distance functions, cosine and Mahalanobis distance are used in the clustering, with results compared in

Table 3. Comparison of clustering evaluation metrics under different distance functions.

	${\mathit{\sigma }}_{\mathit{P}\mathit{R}}$	${\mathit{\sigma }}_{\mathit{U}\mathit{R}}$	${\mathit{\sigma }}_{\mathit{O}\mathit{R}}$	${\mathit{\sigma }}_{\mathit{R}\mathit{V}\mathit{R}}$
Euclidean	0.2047	0.0224	0.5707	0.2685
cosine	0.2089	0.0211	0.5793	0.2702
Mahalanobis	0.3441	0.0354	0.5787	0.355

.

In Table 3, the ${\sigma }_R$ The values of Euclidean and cosine distance are very close, and the Euclidean distance has better performance for most of the metrics. The ${\sigma }_R$ values of the Mahalanobis distance are comparatively large. The test results demonstrate that the Euclidean distance is the most suitable for the proposed AEC-based method.

5.5. Operation Mode Analysis

To test the effectiveness of the proposed method on the operation mode analysis of the power system, 500 testing operation modes are randomly selected. After clustering the 500 operation modes into existing clusters determined by the operation mode sample set, the corresponding typical operation modes are obtained according to the clusters they belong to. Calculate the analysis metrics for these operation modes and their corresponding typical operation modes separately. The errors of operation mode analysis can be defined as the errors of analysis metrics of the operation mode and its typical operation mode, i.e.,

$$L_r={\displaystyle \frac{{\displaystyle {\sum }_{m=1}^{n_r}\left|R_m^{ty}-R_m\right|}}{n_r\cdot R_m}}$$

(19)

where $R_m$ and $R_m^{ty}$ are the analysis metrics of the m-th sample and its typical operation mode, respectively; and $n_r$ is the number of samples selected in the operation mode analysis.

The errors of the analysis metrics under different clustering methods are shown in Figure 11.

In Figure 11, the errors of system operation modes based on AEC are the smallest for each analysis metric, followed by the SOM clustering method, and then the k-means method. The test results demonstrate that the typical operation modes selected based on the proposed AEC method can best represent the operation modes in the operating analysis. The analysis metrics of the renewable energy penetration rate and spinning reserve rate of AEC are displayed as examples in Figure 12 and Figure 13.

In Figure 12 and Figure 13, the analysis metrics of the selected operation modes and the corresponding typical modes all converge to the line of y = x, verifying the representatives of the typical operation modes. The training and computational time for the operation mode analysis of different clustering methods are compared in

Table 4. Comparison of training and computational time of different methods.

	k-Means	SOM	AEC
Training Time (s)	0.8	21	53
Computational Time (s)	0.009	0.01	0.009

. The case is tested on a desktop computer with intel(R) Core(TM) i5-9500F CPU and 32 GB of memory.

In Table 4, the training time of AEC is the longest, followed by SOM. K-means has the shortest training time. It is because AEC has a complex neural network, and the training takes more time. The computational time of the three methods, i.e., the time consumed by operation mode analysis, is very short and close to each other. The test results prove that once the neural network is trained, the proposed method is suitable for real-time applications.

In AEC, the 500 testing operation modes are clustered into 95 groups. Two clusters with the largest number of testing operation modes are selected as examples, with their typical operation modes shown in Figure 14. In the figure, Testing Cluster 1 contains 19 testing operation modes, and Testing Cluster 2 contains 17 operation modes.

In Figure 14, the two typical operation modes show some obvious differences. In Testing Cluster 1, generation of Gen 3, Gen 4, and renewable energy is larger, indicating a heavier system load level. The power generation is much smaller in Testing Cluster 2. The pumped hydro storage is operating in pump mode in both clusters. The analysis metrics of the two typical operation modes are shown in

Table 5. Comparison of analysis metrics of Testing Clusters 1 and 2.

Testing Cluster	PR	UR	OR	RVR
Testing Cluster 1	0.025	1	0.3696	0.5149
Testing Cluster 2	0.0144	1	0.3478	1.4576

.

In Table 5, the PR of Testing Cluster 1 is larger than that of Testing Cluster 2 due to the larger renewable energy generation shown in Figure 14. UR of the two clusters is equal to 1, indicating that the renewable energy is fully utilized. Since Testing Cluster 1 has a heavier load level, its OR is larger than Testing Cluster 2, and its RVR is smaller. The comparison demonstrates that operation modes in Testing Cluster 1 have a higher risk of power balance and transmission line overloading.

From the above test results, it can be concluded that the typical operation modes selected based on the proposed AEC method can comprehensively characterize the multi-dimensional operating characteristics of the power system under various conditions, e.g., cleanliness, security, flexibility, and adequacy. By selecting typical operation modes, the proposed method can accurately evaluate the multi-dimensional characteristics of the power system, as well as assess system operating risks.

6. Conclusions and Discussion

This paper proposes an operation mode analysis method for power systems based on AEC. This method employs AE to perform dimensional reduction and transformation on system operating vectors. System regulation resources, such as pumped hydro storage, are considered in the operating vector. By clustering feature vectors, typical system operation modes are extracted, and multi-dimensional analysis metrics, i.e., renewable energy penetration rate, renewable energy utilization rate, line overload rate, and spinning reserve rate, are used to evaluate various characteristics of the power system with high-proportion renewable energy, such as cleanliness, security, flexibility, and adequacy.

In the testing cases on the IEEE 39-node system, the proposed method achieves better results compared with other commonly used clustering methods, i.e., SOM and k-means. The case study demonstrates that the proposed method can effectively extract correlations and coupling relationships among high-dimensional operating variables of power systems, select representative typical operation modes, and accurately assess system operating characteristics and risks. This method can provide strong support for subsequent system operation mode arrangement, control strategy formulation, and power grid planning.

When applying the proposed method to real-world power systems, the operation mode analysis may encounter data quality problems, e.g., data missing, noise, or time label mismatch. In this case, it is true that the data quality problems may affect the operation mode analysis results; however, the proposed AEC-based method also has some noise-suppressing capabilities to minimize the effect: (1) When extracting features to the low-dimensional space, AE can suppress some noise. The neural network can adapt to reduce noise in tasks of varying complexities by adjusting network depth and width [28,29]. (2) In real-time applications, the operation mode data is usually processed by state estimation before analysis. Most of the data errors are eliminated in this process, and the data quality is improved. Further research of this work will be focused on the application of denoising AEC-based methods for real-world applications.

Compared to the AEC-based method proposed in this paper, the deep embedded clustering (DEC) method can embed data points in a lower-dimensional feature space and perform clustering using the obtained space simultaneously, ensuring the obtained latent space has better clustering performance. But a DEC-based approach usually requires a large amount of training data to achieve stability. Further research will be focused on expanding the sample set by increasing the operating vector dimension, generating more representative operating modes, and testing the clustering performance of the DEC-based method on the operation mode samples.

Further research will also include the selection of some extreme operation modes for the power system. In the proposed method, the operation modes in centroids are selected as typical operation modes, which are used to represent the characteristics of the operation modes within the clusters. Since the feature differences between the sample points at each cluster boundary and the centroid still exist, errors are produced when using the centroid to represent the features of all samples within the cluster. These extreme operation modes of the cluster boundaries may be important for application scenarios such as robust control strategy formulation and risk assessment under extreme conditions. Therefore, a filter method to select representative extreme operation modes of the cluster boundaries is required as a supplement to the AEC-based clustering method proposed in this paper.