Power System Transient Stability Assessment Using Convolutional Neural Network and Saliency Map

Abstract

This study proposes a model for transient stability assessment, which is a convolutional neural network model combined with a saliency map (S–CNN model). The convolutional neural network model is trained on dynamic data acquired through the data measurement devices of a power system. Applying the saliency map to the acquired dynamic data visually highlights the critical aspects of transient stability assessment. This reduces data training time by eliminating unnecessary aspects during the convolutional neural network model training, thus improving training efficiency. As a result, the proposed model can achieve high performance in transient stability assessment. The dynamic data are acquired by configuring benchmark models, IEEE 39 and 118 bus systems, through MATLAB/Simulink and performing time-domain simulations. Based on the acquired dynamic data, the performance of the proposed model is verified through a confusion matrix. Furthermore, an analysis of the effects of noise interference on the performance is conducted.

1. Introduction

Power system transient stability problems can occur due to various external factors, such as facilities, equipment and transmission line failures, etc. Transient stability assessment (TSA) is one of the important parts of the stable operation of power systems. TSA determines whether a synchronous generator can provide a stable power supply during disturbances. If a synchronous generator exceeds the transient stability threshold, it results in instability in frequency and voltage, leading to load shedding or widespread blackouts. Therefore, real-time TSA is essential for maintaining a stable power system operation [1]. TSA is an area of power system analysis that has been extensively researched for a long time and has been performed in the past through numerical integration methods, such as the Runge–Kutta technique [2]. This method requires a lot of repetitive calculations and computational time because it requires calculating high-dimensional nonlinear differential equations [3]. Therefore, it was difficult to apply real-time TSA in power system operations where immediate control actions are required. However, recent advancements in artificial intelligence technology and improvements in computer performance have led to significant research in the real-time prediction of power system transient stability. Artificial neural network models based on artificial intelligence technology, constructed from power system state data, have a very simple computational structure for inference. This makes them suitable systems for real-time power system state analysis [4,5,6,7]. Having the ability to perform real-time evaluations of transient states in the power system enables more effective control actions for disturbances that occur in the power system.

The widespread deployment of phasor measurement units (PMUs) that perform high-speed sampling in the power system has enabled the real-time acquisition of dynamic data from the power system. Consequently, real-time TSA based on artificial intelligence has become possible. The dynamic data obtained through PMUs typically have an error margin of less than 1%. This margin accounts for errors in the measurement instruments and noise during data acquisition. Using PMUs with such small error margins allows for the real-time acquisition of dynamic data from the power system, and transient stability can be assessed in real time [8,9].

Using these data, various research studies have proposed artificial intelligent models, such as support vector machines (SVMs), known for their superior classification performance [10], long short-term memory (LSTM) [11], neural networks (NNs) [12], and convolutional neural networks (CNNs) [6,13], etc. Moreover, there are studies that have combined artificial intelligence techniques with conventional mathematical methods [14] or integrated two or more artificial intelligence techniques [15,16]. In recent research trends, a significant number of studies have been focusing on using CNN models that transform data into images and use them as inputs for artificial intelligence models. The CNN model demostrates outstanding performance in image analysis abilities, even though it comes with the disadvantage of longer computation times for obtaining results. However, these disadvantages have been overcome due to the enhancements in high-performance computing. As a result, CNN models are being extensively applied in various image analysis domains, and their exceptional performance is evident [17]. In the research results of power system TSA using CNN models, it can be seen that it shows higher accuracy than the results using other artificial intelligence techniques [18]. CNN models affect calculation speed and accuracy depending on how many convolution layers and optimization parameters are used. However, specific values for these aspects are typically determined through various research findings [19,20]. As the number of convolutional layers increases, the possibility of over-fitting increases, and high accuracy can only be achieved for specific input images. As such, it is somewhat unsuitable to increase accuracy with the detailed structure and parameter values of the model.

To solve these problems, this study proposes a model that combines a CNN model with a saliency map (S–CNN) model to achieve higher accuracy in TSA. A saliency map is a method that recognizes contrasting colors, brightness, etc., in the pixels that make up an image and finds and emphasizes areas with rapid changes in the pixels. It is a method that greatly increases image analysis performance and is used in computer vision, object recognition, and object detection. It has been used as a popular image analysis method in image-related fields, such as [14,15,16,17]. Recently, studies using saliency maps have been published in the field of power system load prediction [18,19]. This study is the first to apply saliency maps to TSA, as no research on such applications has been presented before. The contributions and research gaps are as follows:

(1)

Incorporating saliency maps: This study pioneers the utilization of saliency maps, a novel image processing technique, as a preprocessing step for CNN models in the context of TSA. This innovation leads to the development of more high-performing models.

(2)

Introducing saliency maps to TSA: By applying saliency maps to the field of TSA, this research highlights the significance of the method in generating CNN model input images. It emphasizes that even with the same dataset, the structure of input images significantly influences the model’s performance.

(3)

Saliency map’s advantages: The use of saliency maps emphasizes the most relevant areas in the acquired data, contributing to better training outcomes for the CNN model. This results in reduced training time and improved training accuracy.

(4)

Immediate TSA results: Since only fault-time data are used to construct the input images, this method allows for immediate TSA results after the fault is cleared.

The proposed method uses dynamic data obtained from various faults in the IEEE 39 and 118 bus systems. These data are transformed into RGB image maps, and saliency maps are generated. Then, these maps are used as inputs to a CNN model, enhancing the classification performance of TSA. The validity of the proposed model is verified by comparing the accuracy of classification performance.

The remaining sections of the paper are structured as follows: Section 2 outlines the setup for transient stability assessment using a CNN model. In Section 3, an explanation of saliency maps is provided. Section 4 describes the proposed model for performing TSA by combining saliency maps and CNN models. Section 5 validates the performance of the proposed model and compares it with previous research.

2. Application of Convolutional Neural Network in Transient Stability Assessment

2.1. The Basic Structure of Convolutional Neural Network

Figure 1 shows the basic structure of the CNN model, and a description of each structure is as follows.

(1)

Convolution layer: The process involves taking the pixel data of an image and applying a user-defined kernel (filter size: n × n) as it slides over the image. This kernel acts as a kind of filter and is used to detect various characteristics of an image, such as edges, color, texture, etc. The output of the convolution layer is called a feature map, which indicates how each portion of the input image has acquired specific features through each filter. In TSA, the role of the convolution layer is to identify information about transient stability in the input image constructed through data acquired form power system.

(2)

Pooling layer: The pooling layer is employed to reduce the dimensions of the feature map through down-sampling. This serves to decrease computational complexity and prevent overfitting in the model. Pooling methods are primarily represented by max-pooling and average-pooling techniques. Max pooling involves selecting the highest value within each section of the feature map, which helps to highlight prominent features. Average pooling computes the average value within each section of the feature map, providing a representation of the overall characteristics. In TSA, emphasizing the prominent parts in each feature map is more effective. Therefore, in this study, the max-pooling method is used.

(3)

Fully connected layer: The fully connected layer in a neural network connects all input nodes to all output nodes, typically positioned at the network’s end to aggregate features and produce the final classification output. In TSA, the aim is classification, determining if the power system is in a transient stability state, and assigning input data to specific categories. The output layer structure depends on the classification task. In TSA, the result should indicate the system’s transient stability status. This is typically achieved by attaching a single neuron with a sigmoid function to the final network layer, creating a logistic regression model. For multi-class classification (e.g., stable and unstable), a softmax function is employed.

$$y_j=\frac{e^{z_j}}{{{\displaystyle \sum }}_{j=1}^N{e}^{z_j}}$$

(1)

Softmax function can be expressed as Equation (1), where $z_j$ represents the score of the previous node, $N$ represents the total number of previous nodes (i.e., the number of categories), and $y_j$ indicates the probability of belonging to a category. To perform classification, an appropriate cost function for training is necessary. To train a classification problem, cross-entropy is used as the cost function. Here, cross-entropy quantifies the difference between the predicted probabilities and the actual labels, guiding the training process to adjust the model’s parameters for accurate classification. It can be expressed as Equation (2), where $k$ represents the class, $y$ is the neural network’s output, and t is correct label, where the output is 1 for the correct answer and 0 otherwise.

$$E=-{{\displaystyle \sum }}_{k=1}^N{t}_{k}ln{y}_k$$

(2)

2.2. Observation Window

Figure 2 represents the proposed observation window (OW) for TSA. OW refers to a specific time period or duration during which data are collected and analyzed for a particular purpose. In power systems and data analysis, the concept of an OW is commonly used to define a specific time period for data collection and observation. From the point when the simulation starts and data is acquired time ($T_S)$, to the point when it ends and data acquisition end time ($T_S)$. TSA, OW is typically defined as fault duration time that begins at the fault occurrence time in the power system ($T_F$) and until the fault clear time ($T_C$). During this defined OW, dynamic data related to the power system state, including bus voltage magnitude, generator rotor speed, and other relevant parameters, are acquired. These data are used for training for TSA.

In this study, data collected during the previously defined observation window period are utilized in TSA by providing them as input to the CNN model. Consequently, this allows for an immediate TSA result at the point of fault clear time. Furthermore, in the case of transient unstable cases, it enables the immediate execution of additional control actions. These advantages significantly contribute to enhancing the stability of the power system, thus demonstrating the effective validity of the proposed method.

2.3. Preprocessing of Dynamic Data

To apply the CNN model to TSA, various factors influencing transient stability in the power system are used as input data, which are derived from the acquired dynamic data within the power system. This study applies a CNN model, as shown in Figure 3, to TSA by effectively analyzing the physical properties of the acquired dynamic data from the power system.

The dynamic data acquired from an operating system are stored as a three-dimensional tensor in the database from the data acquisition start time ($T_S$) to the data acquisition end time ($T_E$). The observation window (OW) is defined as the period from the fault occurrence time ($T_F$) to the fault clear time ($T_C$) when storing dynamic data in the database. These data, collected during the fault period, are only used for training in TSA. Through the data acquired during this period, the state values of each dimension are normalized and assigned to the corresponding chroma red (R), green (G), or blue (B) channels, thus constructing an RGB image map. For instance, during the observation window period, the magnitude of each bus voltage magnitude is stored in the database as a 3D tensor. These values are mapped to the “R” channel. During this process, the data structured as 3D tensors are transformed into values representing attributes such as color chroma, brightness, and other characteristics that can denote colors. The constructed RGB map is used as input to the CNN model to perform training for TSA. Passing through convolution layers and pooling layers, the data are organized into vector features related to transient stability. The features are aggregated through the fully connected layer, and transiently stable and unstable states are determined through a softmax classifier [16].

3. Transient Stability Assessment Using Saliency Map

A saliency map is a fundamental concept in the field of computer vision and image processing. It serves as a visual representation that aims to identify and highlight the most significant regions within an image. The primary purpose of generating a saliency map is to simulate human visual attention and perception, emphasizing the portions of an image that are most likely to capture a person’s interest or are crucial for specific tasks, such as object recognition or scene understanding. Saliency maps are generated through computational algorithms that analyze various low-level features of an image, such as color, contrast, texture, and spatial orientation. These algorithms identify regions where these features exhibit abrupt changes or deviate significantly from their surroundings. The rationale behind this process is that humans tend to focus on areas where these features display significant differences, making these areas more likely to contain relevant objects or information. In essence, saliency maps provide the regions of higher intensity indicating the portions of the image that stand out the most in terms of visual distinctiveness. This technique finds applications in a wide range of fields, including object detection, image segmentation, gaze tracking, and even design and marketing, where understanding the areas that draw the most attention can be crucial. As PMUs become increasingly prevalent and installed in the field of power systems, real-time data acquisition from various devices and their components that make up the power system is made feasible. Using the exceptional feature of saliency maps to recognize rapidly changing data values from the acquired data, they can be applied to several power system field. These applications include the diagnosis and discrimination of component failures in the field of power device components. Additionally, the data continuously acquired can be analyzed through saliency maps for state estimation, which can be applied to forecast future states, thus making it suitable for load forecasting. Furthermore, saliency maps can be utilized to identify key factors of energy consumption, thereby finding applications in the field of energy efficiency analysis. Moreover, saliency maps can be used in an integrated power quality monitoring system to assess power quality and stability, as well as to detect power quality issues at an early stage by identifying anomalies through the use of saliency maps.

In the area of artificial intelligence and deep learning, saliency maps can be used to interpret the decisions made by the model. By analyzing which regions of an input image contributed the most to a particular output or classification decision, researchers can gain insights into how the model arrived at its conclusion. This technique is particularly useful in understanding the inner workings of complex models like CNN and improving their interpretability. Overall, saliency maps play a pivotal role in bridging the gap between human perception and computer-based image analysis.

In this study, we perform TSA using saliency maps constructed from data acquired from the power system. To train the CNN model and assess for model performance accuracy using saliency maps, it compares the results of training a CNN model with RGB maps. This comparative analysis validates the effectiveness of the proposed method.

3.1. Generation of Saliency Map

Figure 4 and Figure 5 show an outline of the procedure for generating the saliency map proposed in this study [21]. In essence, images consist of the colors R, G, and B, each with its distinct frequency values. Fourier transform (FT) is applied to analyze the image in the frequency domain. This process highlights small and irregular patterns within the image, such as small targets, while effectively suppressing repetitive patterns in the surrounding areas. Consequently, the saliency map plays a crucial role in accentuating areas in the image that demand attention [22], such as those exerting a substantial influence on deep learning model outcomes. Using a saliency map as model input helps identify critical patterns, thereby leading to an overall improvement in classification accuracy.

3.2. The Method of Combining Saliency Map with CNN Model (S–CNN Model) for TSA

Figure 6 represents the proposed model, which combines the saliency map with the CNN model (S–CNN model) for TSA. Data acquired from the power system, as shown in Figure 3, are transformed into a saliency map through the process in Figure 6, and it is used as input for the CNN model. The saliency map effectively highlights areas in the acquired data that are most relevant to TSA. This allows the CNN model to learn TSA-related patterns more effectively than using the traditional RGB image map as input. These advantages can lead to reduced training times and improved accuracy, demonstrating that the proposed model is suitable for TSA. The n is number of buses and generators.

4. Configuration of Proposed Model for Transient Stability Assessment

The structure of the proposed model (S–CNN model) uses the saliency map as input to the CNN model to perform TSA. For TSA, each saliency map must be labeled indicating its corresponding transient stable state or not. Each saliency map can be labeled using a transient stability index (TSI), which is defined by the generator rotor angle among the data stored in the database. This can be expressed as Equation (3). ${\theta }_{max}$ represents the maximum angle difference among generator rotors. The transient stability can be classified as a stable state in $\mathrm{TSI}>0$, while cases in which $\mathrm{TSI}\le 0$ can be classified as unstable.

$$\mathrm{TSI}=\frac{360°-\left|{\theta }_{max}\right|}{360°+\left|{\theta }_{max}\right|}$$

(3)

The proposed model is then trained using appropriate training datasets consisting of labeled saliency map. The training involves an iterative optimization process where the model adjusts its weights and biases to minimize the classification error. This enables the model to learn the intricate relationships between the input images and their corresponding stability states. Upon completion of the training process, the CNN model is ready for the power system TSA. The performance of the proposed model is assessed through a confusion matrix, providing a comprehensive understanding of its effectiveness in predicting transient stability. In essence, the proposed TSA model integrates advanced deep learning techniques with the unique concept of saliency map transformation. y employing this method, the image analysis capability is enhanced, providing significant assistance in determining the excessive stability of the power system.

By employing this method, the image analysis capability is enhanced, providing significant assistance in determining the transient stability of the power system.

4.1. Selection of Input Data

The selection of input data is important for TSA using the proposed model. In this study, dynamic data acquired from the power system are used to compose the images that serve as input to the CNN model. In this study, voltage phasor values from each bus and data obtainable from the generator side are used among the acquired data from the power system. To create suitable input data configurations for the proposed model, three cases for input data construction were examined to assess transient stability in the power system. To generate an appropriate input data configuration for the proposed model and find the data combination that has the most significant impact on TSA, three cases were constructed as follows, with each case consisting of three variables. Through this, the optimal input data configuration can be selected to maximize the efficiency of the proposed model in

Table 1. Input data for each case.

	Input Data
Case 1	Bus voltage, bus angle, $\Delta$bus angle
Case 2	Generator rotor speed, generator rotor theta, bus angle
Case 3	Bus voltage, bus angle, generator rotor speed

. The case 1 dataset comprises bus voltage, bus angle, and the rate of change in bus angle, all of which can be obtained through PMUs. In case 2, dataset are formed by combining PMU data and generator data. The case 2 dataset consists of generator rotor speed, generator theta, and bus angle. The case 3 dataset is composed of bus voltage, bus angle, and generator rotor speed.

4.2. Constructing an Input Image Map Using a Saliency Map

In this study, we use data collected from the power system to create an RGB image map and a saliency map, using these two as inputs for the CNN model used in TSA. These input image maps contribute to the model’s classification process and serve an important role in determining the transient stability of the power system. Through this process, the performance of the saliency map can be evaluated. The two types of images used as input images can be seen in Figure 7 and Figure 8, which represent one of the various fault scenarios in case 1.

Figure 7 and Figure 8 show input images for faults that occurred in the IEEE 39 bus system transmission line three-phase short-circuit fault (#28 to #29: fault duration time, 205 ms to 260 ms) and IEEE 118 bus system bus three-phase short-circuit fault (#19: fault duration time, 400 ms to 520 ms), respectively. In each figure, (a) represents the stable state RGB image, (b) illustrates the saliency map for the stable state, (c) displays the RGB image for the unstable state, and (d) shows the saliency map for the unstable state.

Each pixel value in the RGB image map corresponds to the value of a 3D tensor, and the color changes depending on the magnitude of this value. Analyzing the RGB pixel colors in the image reveals that the color intensity changes based on the magnitude of the values within the 3D tensor. The saliency map detects differences in color between adjacent pixels, representing areas with significant changes more brightly and areas with minor changes more darkly. Regions with significant changes in the image are areas that affect transient stability, so emphasizing these areas through the use of the saliency map, which represents them more prominently than the RGB image map, can enhance the performance of the CNN model.

4.3. Proposed Model (S–CNN Model) for TSA

Figure 9 shows the proposed method for TSA using saliency maps. When a fault occurs in a normally operating system, dynamic data are stored in the database at data sample rates until the fault is cleared. In this process, data used for each case are classified to construct an RGB image map. Additionally, a saliency map is constructed. These two images are then separately used to train and validate the CNN model for TSA.

4.4. CNN Model Constructure and Parameters

To validate the performance of the suggested method, as illustrated in Figure 9, the time-domain simulation models, IEEE 39 and 118 bus systems, were implemented using Matlab/Simulink. The IEEE 39 bus system has 39 buses, 10 generators, 19 loads, and 46 branches. The IEEE 118 bus system has 118 buses, 54 thermal units, 91 loads, and 186 branches. The IEEE 39 and 118 bus systems are versatile models widely employed in various domains of power system research, spanning from traditional power system analysis to the present challenges posed by the integration of renewable energy sources. Known for their robustness, these models have become benchmarks extensively utilized in the field of power system analysis These models are extensively employed as benchmark systems in power engineering studies. The specifics of each model, including parameters and configurations, have been established, as outlined in

Table 2. CNN model structure and simulation parameters.

CNN Model Structure		Parameters [19]
Input	RGB image map, Saliency map (3D tensor 39,5,3)	Optimizer	SGDM 1
Hidden	Convolution layer (3, 3, 64) Pooling layer	Initial learning rate	1 × 10−2
	Convolution layer (3, 3, 16) Pooling layer	Momentum	0.95
	Convolution layer (3, 3, 4) Pooling layer	Min. batch size	16
	Flatten (64)	Max. epochs	40
Output	Softmax (2)

¹ Stochastic gradient descent with momentum.

.

4.5. Simulation Conditions

The training data for the time-series analysis model are gathered by conducting time-domain simulations on various fault scenarios, including three-phase short-circuit faults in each transmission line and bus. The specifics of these fault cases are outlined in

Table 3. Simulation conditions.

Simulation Conditions
Fault type	Three-phase short-circuit fault
Fault location	Transmission line, bus
Simulation times	10 s
Fault duration times	0.2~0.6 s
Sampling rate	60 Hz
Dataset	IEEE 39 bus system: 3567 (stable: 2816, unstable: 751) IEEE 118 bus system: 5168 (stable: 4437, unstable: 737)
Dataset split ratio	70% (training), 30% (validation)

. The fault duration time can be determined from a uniform distribution ranging between 0.2 s and 0.6 s. We use a random selection of 70% of the initially acquired data for training both the proposed model and comparative models. The remaining 30% of the data was dedicated to performance validation for each model. Specifically, during the training phase with the initially acquired data, we concluded the training either upon reaching the maximum epoch condition or when the rate of change in error fell below a specified value. Subsequently, we validated the performance of each model using the unused 30% of the data. This approach enables the demonstration of model performance through performance metrics.

“Validation” refers to the process of assessing the performance of the trained model using an independent dataset not used during the model training. This process evaluates the model’s ability to generalize to unseen data, prevents overfitting, and contributes to enhancing the overall quality of the model.

5. Verification of Proposed TSA Model

In this study, the proposed TSA model with the application of saliency maps is compared to other methods proposed in different studies.

5.1. Verification Index

The verification of the proposed model was evaluated using a confusion matrix in

Table 4. Confusion matrix.

Actual Value	Prediction Value
	Stable	Unstable
Stable	True positive (TP)	False negative (FN)
Unstable	False positive (FP)	True native (TN)

. To compare the effectiveness of the proposed saliency-map-based TSA model with other existing methods, the study employed a confusion matrix for performance assessment. This allowed for a comprehensive evaluation of the proposed model’s accuracy and efficiency in comparison to alternative approaches.

The confusion matrix enables the calculation of accuracy (AC), precision (PR), false positive rate (FPR), and false discovery rate (FDR), recall (RC), and F1-Score (F1s) values, providing a means to validate the performance of the model [23]. Through these metrics, the model’s effectiveness can be rigorously assessed, allowing for a thorough evaluation of its performance.

$$\mathrm{AC}=\frac{TP+TN}{TP+FP+TN+FN}$$

(4)

$$\mathrm{PR}=\frac{TP}{TP+FP}$$

(5)

$$\mathrm{FPR}=\frac{FP}{FP+TN}$$

(6)

$$\mathrm{FDR}=\frac{FP}{TP+FP}$$

(7)

$$\mathrm{RC}=\frac{TP}{FP+TN}$$

(8)

$$\mathrm{F}1\mathrm{S}=\frac{2\times PR\times RC}{PR+RC}$$

(9)

Among these, the important performance indices for assessing the model are AC and FPR. AC represents the ratio of cases where the predicted value is positive when the actual data are positive and the predicted value is negative when the actual data are negative among all data. FPR refers to the ratio in which the predicted value is positive when the actual data are negative. When using these two indices in TSA, a higher accuracy indicates that the model is adept at accurately classifying both stable states and unstable states, while a lower FPR suggests that the model is less likely to incorrectly classify unstable states as stable ones. Therefore, a model with higher accuracy and lower FPR is considered to be more effective in TSA.

5.2. Performance Verification

To validate the performance of the proposed method, we selected SVM [24], LSTM [25], and CNN models [18], which are mainly used for TSA as comparative models. The simulation used data from the three cases listed in Table 1. Each model was executed 10 times to calculate the average results. These results are visually represented through confusion matrices, with detailed information presented in

Table 5. Results of confusion matrix for each case.

		SVM Model (Class)	LSTM Model (Class)	CNN Model (Class)	Proposed Model (Class)	Deviation from Proposed Model
						SVM Model (Class)	LSTM Model (Class)	CNN Model (Class)
		IEEE 39 bus system
Case 1	TP	824.0	827.4	834.0	841.0	17.0	13.6	7.0
	FN	20.4	17.5	9.6	3.6	−16.8	−13.9	−6.0
	FP	17.6	15.5	12.4	6.4	−11.2	−9.1	−6.0
	TN	207.0	208.6	213	218	11.0	9.4	5.0
Case 2	TP	826.0	829.1	836.4	841	15.0	11.9	4.6
	FN	15.8	15.0	5.8	1.9	−13.9	−13.1	−3.9
	FP	15.2	12.0	10.2	5.1	−10.1	−6.9	−5.1
	TN	212.0	212.9	216.6	221.0	9.0	8.1	4.4
Case 3	TP	824.6	828.0	835.2	844.5	19.9	16.5	9.3
	FN	18.2	17.0	6.2	3.2	−15.0	−13.8	−3.0
	FP	15.8	14.0	12.8	5.8	−10.0	−8.2	−7.0
	TN	210.4	209	214.8	215.5	−5.1	6.5	0.7
		IEEE 118 bus system
Case 1	TP	1315.0	1320.2	1321.4	1337.6	22.6	17.4	16.2
	FN	41.0	38.0	39	12	−29.0	−26.0	−27.0
	FP	26.0	20.0	15	13	−13.0	−7.0	−2.0
	TN	168.0	171.8	174.6	187.4	19.4	15.6	12.8
Case 2	TP	1319.0	1324.0	1321	1326.4	7.4	2.4	5.4
	FN	34.0	24.0	13	3.5	−30.5	−20.5	−9.5
	FP	18.0	16.0	16	9.5	−8.5	−6.5	−6.5
	TN	179.0	186.0	200	210.6	31.6	24.6	10.6
Case 3	TP	1317.8	1322.0	1330	1338	20.2	16.0	8.0
	FN	38.0	28.0	18	8.5	−29.5	−19.5	−9.5
	FP	23.0	20.0	19	12.5	−10.5	−7.5	−6.5
	TN	171.2	180.0	183	189	17.8	9.0	6.0

. Through the three cases, we can validate the following:

(1)

Analyze model performance through input datasets according to case configuration to determine the most suitable input datasets for TSA.

(2)

Verify the effectiveness of the proposed model by comparing the performance of the proposed model and the comparison model through the verification index.

Using the results in Table 5 and Figure 10 and Figure 11. The performance index for each case can be calculated through Equations (4)–(7) and is shown in

Table 6. The comparison of performance index for each case.

		SVM Model (%)	LSTM Model (%)	CNN Model (%)	Proposed Model (%)	Deviation from Proposed Model
						SVM Model (%)	LSTM Model (%)	CNN Model (%)
		IEEE 39 bus system
Case 1	AC	96.45	96.91	97.94	99.06	2.61	2.15	1.12
	PR	97.91	98.16	98.53	99.24	1.33	1.08	0.70
	FPR	7.84	6.92	5.50	2.85	−4.99	−4.07	−2.65
	FDR	2.09	1.84	1.47	0.76	−1.33	−1.08	−0.71
	RC	97.58	97.93	98.86	99.57	1.99	1.65	0.71
	F1S	97.75	98.04	98.70	99.41	1.66	1.36	0.71
Case 2	AC	97.10	97.47	98.50	99.40	2.30	1.93	0.90
	PR	98.19	98.57	98.80	99.39	1.20	0.82	0.59
	FPR	6.69	5.34	4.50	2.25	−4.44	−3.09	−2.25
	FDR	1.81	1.43	1.20	0.60	−1.21	−0.83	−0.60
	RC	98.12	98.22	99.31	99.77	1.65	1.55	0.46
	F1S	98.16	98.40	99.06	99.58	1.43	1.19	0.53
Case 3	AC	96.82	97.01	98.22	99.16	2.34	2.15	0.94
	PR	98.12	98.34	98.49	99.32	1.20	0.98	0.83
	FPR	6.98	6.28	5.62	2.62	−4.36	−3.66	−3.00
	FDR	1.88	1.66	1.51	0.68	−1.20	−0.98	−0.83
	RC	97.84	97.99	99.26	99.62	1.78	1.63	0.36
	F1S	97.98	98.16	98.88	99.47	1.49	1.31	0.60
		IEEE 118 bus system
Case 1	AC	95.68	96.26	96.52	98.39	2.71	2.13	1.87
	PR	98.06	98.51	98.88	99.04	0.98	0.53	0.16
	FPR	13.40	10.43	7.91	6.49	−6.91	−3.94	−1.42
	FDR	1.94	1.49	1.12	0.96	−0.98	−0.53	−0.16
	RC	96.98	97.20	97.13	99.11	2.13	1.91	1.98
	F1S	97.52	97.85	98.00	99.08	1.56	1.22	1.08
Case 2	AC	96.65	97.42	98.13	99.16	2.51	1.74	1.03
	PR	98.65	98.81	98.80	99.29	0.64	0.48	0.49
	FPR	9.14	7.92	7.41	4.32	−4.82	−3.60	−3.09
	FDR	1.35	1.19	1.20	0.71	−0.64	−0.48	−0.49
	RC	97.49	98.22	99.03	99.74	2.25	1.52	0.71
	F1S	98.07	98.51	98.91	99.51	1.45	1.00	0.60
Case 3	AC	96.06	96.90	97.61	98.64	2.58	1.74	1.03
	PR	98.28	98.51	98.59	99.07	0.79	0.56	0.48
	FPR	11.84	10.00	9.41	6.20	−5.64	−3.80	−3.21
	FDR	1.72	1.49	1.41	0.93	−0.79	−0.56	−0.48
	RC	97.20	97.93	98.66	99.37	2.17	1.44	0.70
	F1S	97.74	98.22	98.63	99.22	1.48	1.00	0.59

. It can be seen that the model proposed in the first benchmark model, the IEEE 39 bus system, showed AC performance up to 2.61% higher than the comparative model, and FPR was improved by up to 4.99%. When comparing the performance of the models for each case, case 2 showed the highest performance of each model. In the second benchmark model, the IEEE 118 bus system, the performance of the proposed model in terms of AC was up to 2.71% higher, and FPR was also improved by up to 6.91%. Similar to the first benchmark model, case 2 showed a higher performance for all models compared to other cases.

As a result of applying the proposed model and three comparative models to two benchmark models, it was confirmed that the performance of the proposed model was superior to the comparative model, and the dataset configuration of case 2 was confirmed to be the most suitable configuration for TSA, thereby verifying the effectiveness of the proposed method.

5.3. Comparison with Previous Studies

In this chapter, the effectiveness of the proposed method is verified by comparing it with previous study results.

Table 7. The comparison of accuracy with previous studies.

	Proposed Method (%)		Previous Studies (%)
			Study A	Study B	Study C	Study D
IEEE 39 bus system	Network	CNN	CNN [19]	CNN [20]	CNN [26]	SVM [27]
	AC	99.72	97.70	99.55	98.63	93.04
IEEE 118 bus system	Network	CNN	CNN [28]	MLP 1 [29]	DNN 2 [30]	SVM [31]
	AC	99.16	98.67	98.8	99.08	93.96

¹ Multi-layer perceptron, ² deep neural network.

and Figure 12 summarize the TSA accuracy based on the proposed method and various artificial intelligence techniques used in previous study methods. When comparing the first benchmark model in the IEEE 39 bus system, the proposed method shows accuracy differences of 2.02%, 0.17%, 1.09%, and 6.68%. In the case of the second benchmark model in the IEEE 118 bus system, accuracy differences are 0.49%, 0.36%, 0.08%, and 5.20%. Table 7 confirms that the proposed model can achieve higher accuracy compared to previous study, thereby validating the effectiveness of proposed model.

5.4. Robustness against Noise

Robustness verification using noise involves assessing how well a model can handle variations in input data caused by noise or external factors. This process is crucial for evaluating the model’s generalization ability and stability. The PMU at the distribution level exhibited a signal-to-noise ratio (SNR) close to 60 dB. Tests under interference revealed the need for interventions for an SNR between 40 and 60 dB. Consequently, the impact of white Gaussian noise (at SNRs of 40, 50, and 60 dB) on our proposed model was analyzed when such noise was present in the dynamic data.

Table 8. The comparison of accuracy with different noise levels.

	IEEE 39 Bus System					IEEE 118 Bus System
		SVM Model (%)	LSTM Model (%)	CNN Model (%)	Proposed Model (%)	SVM Model (%)	LSTM Model (%)	CNN Model (%)	Proposed Model (%)
Case 1 (40 db)	AC	95.16	95.96	97.42	98.81	94.94	95.34	96.40	98.03
	PR	97.83	98.09	98.42	99.14	97.50	98.47	98.47	99.04
	FPR	7.91	7.01	5.59	2.96	14.61	11.53	7.84	6.55
	FDR	2.17	1.91	1.58	0.86	2.50	1.53	1.53	0.96
Case 1 (50 db)	AC	95.45	96.08	97.56	98.86	95.15	95.97	96.49	98.12
	PR	97.85	98.11	98.44	99.19	97.87	98.51	98.24	99.10
	FPR	7.79	7.05	5.56	2.92	13.97	11.12	7.98	6.61
	FDR	2.15	1.89	1.56	0.81	2.13	1.49	1.76	0.90
Case1 (60 db)	AC	95.98	96.15	97.78	98.91	95.62	96.03	96.50	98.29
	PR	97.90	98.11	98.49	99.22	98.01	98.49	98.31	99.02
	FPR	7.86	6.99	5.51	2.89	13.45	10.46	7.70	6.53
	FDR	2.10	1.89	1.51	0.78	1.99	1.51	1.69	0.98
Case 2 (40 db)	AC	96.28	97.06	98.12	99.40	96.03	97.19	97.24	98.94
	PR	97.84	98.02	98.64	99.71	98.24	98.41	98.69	99.21
	FPR	7.67	5.79	4.66	0.59	9.72	8.69	7.61	5.53
	FDR	2.16	1.98	1.36	0.29	1.76	1.59	1.31	0.98
Case 2 (50 db)	AC	96.43	97.19	98.38	99.24	96.16	97.25	97.67	98.94
	PR	98.01	98.31	98.68	99.78	98.40	98.54	98.74	99.21
	FPR	7.53	5.36	4.61	0.56	9.69	8.64	7.59	4.37
	FDR	1.99	1.69	1.32	0.22	1.60	1.46	1.26	0.79
Case 2 (60 db)	AC	96.98	97.21	98.42	99.24	96.49	97.34	97.94	98.96
	PR	98.14	98.54	98.74	99.78	98.61	98.76	98.76	99.23
	FPR	7.02	5.39	4.58	0.54	9.19	8.16	7.56	4.34
	FDR	1.86	1.49	1.26	0.16	1.39	1.24	1.24	0.77
Case 3 (40 db)	AC	96.45	96.54	97.94	98.76	95.49	96.25	97.03	98.15
	PR	97.71	98.14	98.35	99.28	97.85	98.33	98.48	98.79
	FPR	7.76	6.61	5.98	2.40	12.83	11.06	9.51	6.72
	FDR	2.29	1.86	1.65	0.72	2.15	1.67	1.52	1.21
Case 3 (50 db)	AC	96.69	96.61	98.06	98.89	95.57	96.46	97.2	98.24
	PR	97.84	98.21	98.40	99.24	98.21	98.41	98.45	98.82
	FPR	7.69	6.49	5.91	2.36	12.59	10.98	9.45	6.75
	FDR	2.16	1.79	1.60	0.76	1.79	1.59	1.55	1.18
Case 3 (60 db)	AC	96.71	96.94	98.18	99.08	95.87	96.69	97.49	98.49
	PR	98.06	98.31	98.44	99.30	98.24	98.44	98.55	98.88
	FPR	7.19	6.34	5.88	2.59	12.03	10.26	9.43	6.60
	FDR	1.94	1.69	1.56	0.70	1.76	1.56	1.45	1.12

and Figure 13 summarize the performance metrics with increasing noise. In the case of SVM and LSTM models, which directly use data with noise as input, and the CNN model, which constructs images as input, performance significantly decreases compared to de-noising data. However, the proposed model does not significantly reflect the noise component when generating a saliency map even if the data contain noise. In other words, even if the saliency map is an image that contains noise, it is less affected by noise because the change in each pixel value is identified according to the saliency map creation algorithm and the image is displayed in shading. Essentially, the saliency map can act as a filter for noise.

Comparing the performance of the comparative model and the proposed model, it can be seen that, in the case of SVM and LSTM models that use data directly containing noise components, the performance decreases significantly as the variance increases because there is no filter component for noise. In the case of the CNN model, data containing noise are imaged and used as input, but when converted to an image, they are less affected by noise, and the rate of performance decrease is smaller than that of the SVM and LSTM models. However, the proposed model stands out in handling noisy data. It transforms noisy data into RGB images and generates a saliency map as an additional step before using it as input to the model. This process effectively removes a significant portion of noise components. As a result, the proposed model demonstrates the smallest variation when the noise level changes, thereby verifying its robustness against noise.

6. Discussion

This study primarily focused on implementing a model for TSA with higher accuracy by combining data acquired from the power system using a CNN model and a saliency map. Through a benchmark model, various dynamic data were acquired by simulating three-phase short-circuit faults at different locations and durations for each transmission line and bus. The acquired data exhibit variations depending on the location and duration of each fault, allowing for the TSA states based on the conditions of various dynamic data.

However, it is crucial to prioritize the use of data highly relevant to TSA, rather than employing all acquired data, to construct a model with higher performance. Among the data acquired through various faults, we examine data changes from the time of failure occurrence time to the clear time. It was confirmed that the values of various variables changed significantly. Among the various variables, some can be measured using actual measuring devices and applied to actual systems. The actual measured variables include the voltage magnitude and phase angle of each bus through the PMU; on the generator side, the rotor speed and rotor theta can be measured using a sensor.

In this research, data with a high correlation to TSA and data that can actually be measured are selected. The three scenario cases are created to determine the optimal combination for TSA by combining each data. For each scenario case, the acquired data were transformed into RGB images and used as inputs for the CNN model. In such cases, RGB saturation values are assigned based on the size of the data, creating the image. However, using these generated images as inputs for the CNN model makes it difficult to readily identify which parts of the pixels in the image have the most significant impact on transient stability.

In this research, saliency maps were supplemented to address this issue. Saliency maps are derived from the original RGB images with additional transformations. They emphasize areas with rapid changes by identifying the values of adjacent pixels, rendering these regions brighter, while areas with minimal changes appear darker. This allows for the identification of regions that are highly relevant to TSA, contributing to improved CNN model accuracy. This research highlights an important point by introducing image processing techniques into deep learning methods in the field of power systems. It emphasizes the significance of image processing techniques through the valuable results in the assessment of TSA. However, there are limitations and potential challenges in saliency maps for TSA:

(1)

Sensitivity to Input Characteristics: Saliency maps are highly dependent on input characteristics and may be sensitive to variations in the dataset. If the dataset does not adequately represent the diversity of possible scenarios in power systems, the saliency maps might not generalize well.

(2)

Biases in Pattern Identification: Saliency maps identify important regions based on gradients, and these might introduce biases. If certain critical patterns are not well-represented in the training data, the model may not accurately identify them, leading to potential oversights in TSA.

(3)

Interpretability Challenges: While saliency maps provide a visual indication of important features, interpreting the exact nature of these features can be challenging. Understanding the underlying reasons for a specific region being highlighted might require domain expertise.

(4)

Dynamic System Changes: Power systems are subject to dynamic changes over time. Saliency maps may not adapt well to evolving system conditions, and their performance might degrade if the model encounters scenarios significantly different from the training data.

Understanding these limitations and potential challenges is crucial for practitioners and researchers. While saliency maps offer valuable insights into feature importance, their application in TSA should be approached with caution. Combining this method with others and considering a diverse and representative dataset can enhance the robustness and generalizability of the proposed approach. Recently, various deep CNN models have been introduced. Capsule network models show excellent performance in various image classification tasks, transfer learning models can identify specific image objects using a pre-trained model when training data are insufficient. Various techniques are being studied based on CNN models [32]. In addition, active research is being conducted on model construction and inference that can be useful in real life by applying CNN models to the IoT field [33]. In the future, the proposed method in this research can be applied to deep CNN models to improve model performance, and if the proposed model is appropriately applied to the IoT field, it can be improved to a model that can be used in real life as well.

7. Conclusions

In this study, we propose a model that combines saliency maps and a CNN model, i.e., S–CNN model for TSA. This model represents a significant research outcome as it is the first to apply saliency maps to TSA and provides model performance based on the method of generating input images for the CNN model. When analyzing the performance results of the proposed method and the comparative models, it was evident that the proposed method achieved higher AC and lower FPR. These outcomes serve as evidence of the effectiveness of employing saliency maps in the assessment and management of power system stability. Furthermore, since, in this study, we constructed input images using data from the proposed OW period, the TSA results could be judged immediately after the fault was cleared. This advantage shows the fastest TSA results when compared to the results of previous studies. This provides additional time to execute control actions, leading to more stable power system operation. Finally, various data were selected to find the most suitable dataset for TSA. By comparing the data configuration of previous study results with the results of this study, it can be verified that the input dataset proposed in this study is the most suitable configuration for TSA.

Future research will focus on constructing an integrated control system based on the results of this study, which rapidly and accurately assess transient stability conditions and integrate them with emergency control in power systems. This integrated control system aims to enhance power system stability and responsiveness during emergency situations.