Power Outage Prediction on Overhead Power Lines on the Basis of Their Technical Parameters: Machine Learning Approach

Abstract

In this study, data on the characteristics of overhead power lines of high voltage was used in a classification task to predict power supply outages by means of a supervised machine learning technique. In order to choose the most optimal features for power outage prediction, an Exploratory Data Analysis on power line parameters was carried out, including statistical and correlational methods. For the given task, five classifiers were considered as machine learning algorithms: Support Vector Machine, Logistic Regression, Random Forest, and two gradient-boosting algorithms over decisive trees LightGBM Classifier and CatBoost Classifier. To automate the process of data conversion and eliminate the possibility of data leakage, Pipeline and Column Transformers (builder of heterogeneous features) were applied; data for the models was prepared using One-Hot Encoding and standardization techniques. The data were divided into training and validation samples through cross-validation with stratified separation. The hyperparameters of the classifiers were adjusted using optimization methods: randomized and exhaustive search over specified parameter values. The results of the study demonstrated the potential for predicting power failures on 110 kV overhead power lines based on data on their parameters, as can be seen from the derived quality metrics of tuned classifiers. The best quality of outage prediction was achieved by the Logistic Regression model with quality metrics ROC AUC equal to 0.78 and AUC-PR equal to 0.68. In the final phase of the research, an analysis of the influence of power line parameters on the failure probability was made using the embedded method for determining the feature importance of various models, including estimating the vector of regression coefficients. It allowed for the evaluation of the numerical impact of power line parameters on power supply outages.

1. Introduction

Electrical energy is distributed to consumers via a sophisticated network of supports, transformers, overhead and underground power lines at various voltages, linear equipment of various kinds, and other equipment. The power supply system (PSS) reliability—that is, the probability of an uninterrupted supply of electrical energy to consumers—increases with the use of new, innovative solutions, including the modern means of protection, monitoring, and the management of electrical networks [1,2]. Despite the overall high reliability of PSS, failures on power transmission lines (PTLs) are a relatively common occurrence, accounting for 35–50% of all failures in power supply systems with voltages of 35–750 kV [3]. The large territorial extent of power lines and their vulnerability to weather-related events are factors contributing to this volume of outages [4,5]. Even while adverse weather conditions are the most common causes of failure, there are other factors to consider, such as wear and tear as well as the aging of the electrical network infrastructure [6].

The analysis of data on power outages allows us to determine the factors that have the greatest impact on failure probability. There is a large number of works devoted to the statistical analysis of failures on power lines, for example, examining situations in various countries such as the Russian Federation [7,8,9], Estonia [10], Iraq [11], and Bulgaria [12]. These studies aim to determine the causes of power line outages and identify primary strategies for reducing their frequency. Statistical analysis is also used in work [13] to examine power line failures that occur as a result of icing conditions. In work [14], in addition to statistical data processing, a regression analysis of the time series of PSS failures, which made it possible to determine the primary causes of emergency outages, parameters, and quality criteria were determined for trend inverse models of the frequency of average monthly failures.

Despite the fact that some works on the statistical analysis of emergency outages have identified the causes of failures, indicating certain failed elements, they have not considered the influence of the element types on the probability of power line failures. Predictive modeling based on machine learning (ML) techniques is one of the advanced techniques to calculate the failure probability [15,16]. So, in work [17], the use of a three-dimensional Support Vector Machine (SVM) is proposed for predicting outages of power system components; in work [18], the same method is used to determine the damage location in the system by measuring the magnitude and angle of incidence voltage at the primary substation of the distribution system. Article [19] presents an approach to identifying equipment faults in distribution systems, which solves the binary classification problem and separates outages into two categories: equipment failures and failures unrelated to the equipment. Three algorithms are used as classifiers: a decision tree, Logistic Regression, and naive Bayes classifier. The application of artificial neural networks is presented in [20] for multi-class fault classification in power supply systems based on current and voltage values on all power transmission lines (PTLs). In [21], authors successfully applied a Graph Neural Networks model for predicting cascading failures in power systems evaluated on the IEEE 39-bus and 118-bus test systems. In the study [22], the authors apply a Natural Language Processing (NLP) approach to predict the time required for power supply restoration. They utilize a recurrent neural network (RNN) to analyze historical outage reports and repair logs.

A significant number of research focuses on applying machine learning methods for predicting outages in power distribution networks during adverse weather conditions [23,24], in particular, hurricanes, tropical storms, rain, and windstorms, as well as coastal floods, including databases in the USA [24], France [25], Puerto Rico [26], and China [27]. So as to determine the number of outages in the event of a typhoon, work [28] applies a Random Forest algorithm for solving a multi-class classification problem, work [29] does a gradient boosting algorithm, and work [27] does an ensemble consisting of two levels, with the last one being XGBoost gradient boosting. Furthermore, research [30] proposes an approach for predicting outages in distribution systems caused by environmental factors through the use of deep neural networks with independent blocks combined into an ensemble. In a related study, the authors of [17] utilize a three-dimensional Support Vector Machine (SVM) to predict power grid component failures based on three features: component deterioration, distance from an extreme event, and the event’s intensity. Convolutional Neural Network-Long Short-Term Memory architecture is proposed to use in research [31] to predict power outage areas during extreme weather events.

Other studies investigate specific causes of power outages, such as those related to trees and animals, by leveraging weather data. For instance, the authors in [32] employ a combination of nonlinear machine learning regression and a time series model to identify the primary causes of vegetation-related outages. Their analysis uses outage data from Duke Energy (USA), geographical information, and weather forecasts. Similarly, research [33] analyzes faults caused by falling trees using a Logistic Regression on data related to weather conditions, seasons, the time of day, and activated protective devices. In a study on outages caused by wildlife, researchers in [34] apply AdaBoost algorithms (AdaBoost.RT and AdaBoost+) to predict such events in overhead distribution systems based on weather conditions and the time of year.

Despite the extensive existing literature on power outage prediction, no studies have explored the prediction of PTL failures using data on the lines’ own technical parameters. This represents a critical and unaddressed area, demonstrating the relevance of the conducted research. The purpose of this research is to develop a machine learning model that can solve the classification problem of predicting potential outages on power transmission lines based on their technical specifications.

2. Methodology and Materials

2.1. Materials for Research

This study is based on data on power outages in electrical networks in the Orel region, Russian Federation. In the previous study [35], the data under consideration were analyzed by methods of mathematical statistics, which included processing missing values, removing duplicates, creating synthetic parameters (including a target feature), identifying outliers and anomalies, and choosing the best features for ML models. A table with 9 features, including the target feature, and 395 objects was the end product.

The features listed in the table are as follows.

(a) Three characteristics with categorical values:

-

Outage fact (target feature);

-

Conductor, type, and crosssection;

-

PTL relation to transit (it indicates whether power transmission line is in transit or not).

(b) Six characteristics with quantitative values:

-

Condition index, % (PTL Technical Condition Score: A rating assigned by maintenance personnel, ranging from 0 (poor) to 100 (excellent));

-

Overhead PTL length, km;

-

Overexploitation, d.q. (this indicator shows whether a PTL has exceeded its standard service life of 35 years. A value greater than 1 indicates that the actual service life exceeds the normative period, while a value less than 1 indicates it does not);

-

Reinforced concrete supports, % (the ratio of reinforced concrete supports to the total number of supports);

-

PTL length through the forest, % (the ratio of the total PTL length in forest areas to the overall length);

-

PTL length in populated areas, % (the ratio of the total PTL length in populated areas to the overall length).

2.2. Research Methodology

The beginning of the study consisted of checking the quality of the prepared data using exploratory data analysis methods. It includes statistical analysis of the distribution of quantitative and categorical variables in the context of the target feature, as well as studying the correlation dependence between variables using the correlation coefficient phi ϕ_k.

Since the goal of the work is to predict the probability of PTL failure based on its parameters, binary classification is the task for machine learning algorithms. Therefore, five ML models based on the following algorithms were selected as classifiers:

• Support Vector Machine (SVM);
• Logistic Regression (LR);
• Random Forest Classifier (RFC);
• Gradient-boosting algorithms over decisive trees: LightGBM Classifier and CatBoost Classifier.

The correct selection of classifier hyperparameters was carried out using the method of random parameter optimization (RandomizedSearchCV), and the best parameters found were refined by the GridSearchCV grid search algorithm. The data is divided into training and validation samples through cross-validation with stratified separation. Data preparation for ML models was carried out using the methods of One-Hot Encoding for categorical variables and data standardization (Standard Scaler) for quantitative ones. To automate the process of data conversion and model training, as well as to eliminate the possibility of data leakage, a pipeline and a heterogeneous feature builder, Column Transformer, were used. The selection of the best trained ML model was based on the ROC AUC quality metric. In addition to this metric, the quality of models was assessed using additional metrics [36], such as AUC-PR, Accuracy, Precision, Recall, and the F1-score. At the last stage, an analysis was made on the influence of power line parameters on failure probability by using the embedded method of determining the importance of the features of various models, including estimating the vector of regression coefficients.

In this research, data processing and analysis were carried out in the Python programming language (ver. 3.13.7) in the Jupyter notebook (ver. 7.0.6) development environment of the Anaconda Python software package. To work with tabular data, the Pandas library (ver. 2.1.4) was used, for mathematical processing of data arrays—NumPy (ver. 1.26.3), for data visualization—Matplotlib (ver. 3.8.0) and Seaborn (ver. 0.12.2), and for correlation analysis—Phik (ver. 0.12.4). Scikit-learn library (ver. 1.2.2) was used to build learning algorithms, transform data suitable for ML tasks, and work with the main classical machine learning models. In addition to it, gradient boosting frameworks from the LightGBM (ver. 4.1.0) and CatBoost (ver. 1.2.2) libraries were used. To work with an unbalanced data set, tools from the Imbalanced-learn library were applied.

3. Results

The development of a machine learning model capable of performing a given task involves properly selecting and preprocessing training data, selecting the most appropriate machine learning algorithm, and fine-tuning the model’s hyperparameters to ensure optimal performance through model testing [37]. Each of these stages is an important link in building a quality model in accordance with the metric being measured.

3.1. Exploratory Data Analysis

Before starting to develop machine learning models, it is necessary to study the prepared data. Figure 1 depicts histograms derived for categorical features and a distribution density graph for quantitative ones, while the latter are constructed with a kernel density estimate [38,39] as the weight function, which is the Gaussian kernel [40].

The following conclusions can be drawn from the given graphs:

• As can be seen from the histograms of categorical features, it is clear that there is a different distribution of values for power transmission lines with and without failures, which indicates their impact on the target variable. The low cardinality of categorical features (four values for the feature “Conductor, type, section” (Figure 1b) and two for “PTL relation to transit” (Figure 1c)) must also be noted. According to this, when preparing data for the direct training of ML models, the most optimal coding of categorical variables is considered to be the One-Hot Encoding technique.
• Density graphs of quantitative feature distributions between power lines, on which outages were observed, and they did not differ in their shape, so it can be assumed that there is an influence of these features on the target variable.
• There is some imbalance in the target attribute “Outage fact” (Figure 1a). There are 163 electrical lines that have failed and 232 that have not; the difference is almost 20%. Class imbalance can cause problems when training machine learning models that are non-probabilistic, such as the SVM (Support Vector Machine), or when solving multi-class classification problems.

The method of correlation analysis was applied so as to find the degree of the relationship between various variables. The correlation coefficient phi ϕ_k was used as a correlation measure, which enables the analysis of not only quantitative variables but also categorical ones [41]. The degree of correlation with this method is in the range 0…1, where 0 means the absence of a relationship between the characteristics, and 1 indicates its maximum degree [42]. The correlation matrix obtained by computing the correlation coefficient ϕ_k between the variables under investigation is graphically displayed as a heat map in Figure 2.

The correlation analysis showed that

-

The replacement of the absolute values of the number of reinforced concrete (RC) and metal supports, as well as the length of the forest and populated areas with relative values, made it possible to cope with the problem of the multicollaterality of these features (this problem was discussed at the previous stage of the study [35]), which should have a beneficial effect on the training of ML models;

-

The target attribute “Outage fact” has a not strong but sufficient correlation with the variables under consideration. The highest correlation is shown by the feature “Overhead PTL length” (0.63) and the smallest by the categorical feature “Conductor, type, section” (0.11);

-

A fairly strong correlation (0.85) was revealed between PTL service life and the fact of whether the power line is in transit or not. This issue was discussed at the previous stage of preparing data for machine learning [35], in which it was concluded that transit PTLs, according to the statistics presented, have a longer service life than non-transit lines. Finally, it was decided to leave both characteristics, since these parameters reflect completely different values.

3.2. Algorithm for Training and Tuning Hyperparameters of ML Models

The algorithm for training machine learning models with the method of random parameter optimization is shown in Figure 3. In accordance with the algorithm, a randomized set of model hyperparameters is selected from the distribution over possible parameter values (parameter grid) and initialized for training (block 4). The number of variations in model hyperparameters using the RandomizedSearchCV method was chosen to be 100 iterations (the “n_iter” value is block 2 of Figure 3). In the same block, the number of folds for cross-validation is also selected (the “cv” value is block 2 of Figure 3). The value of “scor_iter” (the value of the quality metric of the model) is set to 0 and is used later to find the best model (blocks 11, 12, and 13). As soon as the number of iterations exceeds the “n_iter” value (block 3), the process of selecting hyperparameters will stop and an ML model with hyperparameters corresponding to the best quality metric will be produced (block 14).

3.3. Feature Encoding and Scaling

A focused and thorough approach to data preparation is critical to the development and application of machine learning models, including neural networks. The success of the ML model training process largely depends on the quality and suitability of the data fed into the models. In the previous stage, [35], an important part of the processing was performed. It included the processing of missing values and duplicates, as well as the identification of suitable features for the given classification task. Now it is required to convert categorical variables to numeric representations as well as to normalize quantitative variables by scaling or centering their numeric constituents to promote convergence.

As noted above, categorical variables have low cardinality, so it was decided to use the One-Hot Encoding method to encode these variables. To normalize quantitative variables, the data standardization method was used (block 7 of Figure 3). To automate the process of data conversion and model training, as well as to eliminate the possibility of data leakage, a pipeline and a heterogeneous feature builder Column Transformer were used.

It was decided to use two methods to combat the class imbalances of the target feature: the method of weighting classes (CW) implemented inside models from the Scikit-learn library, and the synthetic minority over-sampling technique for the nominal and continuous (SMOTE-NC) method, implemented through the pipeline of the Imblearn library to automate the process and eliminate the issue of feature leakage. Every model was trained using both techniques, and the outcomes were then shown in the final table.

3.4. Splitting the Data Set into Training and Test Samples

When developing machine learning models, it is also critical to validate the model performance using independent data; that is, the data the model has not seen before. For this purpose, a test sample was taken from the source data, usually in the amount of 10–20% of the training one. This approach allows for identifying and, accordingly, eliminating the main problem with training ML models—overtraining, which is the phenomenon when trained ML models learn answers from the training set but poorly determine patterns from third-party data.

In the research, it was decided to separate the test sample in the amount of 20% and apply the cross-validation method to the training sample (in Figure 3, blocks 5–9 and 10), which implies dividing the data into several parts (folds). Each fold at its training stage should act as a validation sample, the rest as a training one. In our case, we used a 5-fold cross-validation with stratified division, guaranteeing the same class ratio in all samples, which is especially important for unbalanced data. Thus, the number of training iterations on one set of model hyperparameters depends on the number of folds (the “cv” value, block 5 of Figure 3). Then, according to the algorithm, as soon as training was carried out on all folds, the average value of the quality metric was calculated (block 10) and compared with the best metric in the condition (block 11).

3.5. ML Models and Hyperparameter Grids

3.5.1. Support Vector Machine

The Support Vector Machine (SVM) is a supervised learning algorithm used for classification and regression analysis [43]. The SVM algorithm works by transforming the input data into a high-dimensional space using a kernel function, then finding the hyperplane that best separates the data points into different classes. The main hyperparameters on which the training of an SVM model depends are the regularization coefficient C, the type of kernel used, and, accordingly, its coefficient. The grid of values for these hyperparameters is presented in

Table 1. ML model hyperparameter grid.

ML Model	Hyperparameter	Hyperparameter Definition	Hyperparameter Values
SVM	C	Regularization strength	0…10, step–0.5
	kernel	Kernel type	‘rbf’, ‘poly’, ‘sigmoid’
	gamma	Kernel coefficient	0…1, step–0.01
Logistic Regression	solver	Optimization algorithm	“saga”
	penalty	Norm of penalty	“elasticnet”
	l1_ratio	Relation between l1 and l2 regularizations	0…1, step–0.1
	C	Regularization strength	0…5, step–0.1
Random Forest Classifier	n_estimators	Number of solving trees	10…1000, step–100
	min_samples_split	Minimum number of samples required to split an internal node	2…50, step–10
	min_samples_leaf	Minimum number of samples required to be at a leaf node	2…50, step–10
	max_depth	Maximum tree depth	1…21, step–1
	criterion	Split criterion	“gini”, “entropy”, “log_loss”
LGBM Classifier	learning_rate	Boosting learning rate	0.0001, 0.001, 0.01
	max_depth	Maximum tree depth	1…21, step–1
	n_estimators	Number of solving trees	10…1000, step–10
	num_leaves	Maximum tree leaves	2…50, step–1
	boosting_type	Boosting algorithm	“gbdt”, “dart”, “goss”
	reg_alpha	L1 regularization coefficient	0…1, step–0.1
	reg_lambda	L2 regularization coefficient	0…1, step–0.1
CatBoost Classifier	depth	Maximum tree depth	1…10, step–1
	learning_rate	Boosting learning rate	0.0001, 0.001, 0.01
	iterations	Number of solving trees	10…1000, step–10
	l2_leaf_reg	L2 regularization coefficient	1…15, step–1
	max_leaves	Maximum tree leaves	2…50, step–1

.

3.5.2. Logistic Regression

Regression modeling is one of the most popular statistical approaches for determining the relationships between a target variable and a set of independent predictors. Regression models are divided into logistic and linear, but a linear regression cannot be used to determine a dichotomous (binary) variable, since the result of such a model is continuous values, including those with a negative direction [44]. Therefore, to predict the probability of a binary value, an extended version of linear regression is used—Logistic Regression (LR), which predicts the probability distribution of an event (yes/no or 1/0) through the logit link function. During the training process, unregulated hyperparameters of the optimization algorithm (“saga”) and the type of regularization (“Elastic-Net”) were selected, which allow for the use of both types of regularization (l1 and l2) simultaneously. The acceptable ratio between these two regularization types was determined by changing the hyperparameter “l1_ratio” (Table 1). The strength of C regularization was also adjusted.

3.5.3. Random Forest

A Random Forest is one of the most widely used machine learning algorithms, applied in both regression and classification problems [45]. The random forest algorithm is based on the ensemble learning method, the results of which are constructed by combining independent forecasts of the forest—many decision trees trained on samples obtained using the bootstrap method [46]. In the case of a classification problem, the final answer of the model will be based on the voting of decision trees [47]. It is known that the more decision trees there are, the higher (in most cases) the model quality is; therefore, up to 1000 trees were used when training the Random Forest. The depth of decision trees also affects the quality of the model’s prediction (the larger it is, the higher the quality is), so ML training was carried out at a depth of 1 to 21 objects. In addition to this, all splitting criteria, the limit on the number of objects in leaves, and the minimum number of objects at which splitting is performed were also examined (Table 1).

3.5.4. Gradient Boosting Algorithms LightGBM and CatBoost

The method of gradient boosting over decision trees is also used, the essence of which is to train a certain number of models (in our case, decision trees), taking into account the errors obtained on previous models. Gradient boosting allows for building an additive function in the form of a sum of decision trees iteratively by analogy with the gradient descent method [48]. Thus, this approach enables achieving a higher prediction accuracy.

As gradient boosting algorithms, the LightGBM Classifier and CatBoost Classifier were used [49]. CatBoost is an open-source gradient boosting library introduced by Yandex in 2017 for supervised machine learning, containing two innovations: ordered target statistics and ordered boosting [50]. A distinctive feature of CatBoost is the ability to work with heterogeneous data sets with different data types. CatBoost is used for regression, classification, and ranking problems. LightGBM is a gradient boosting library introduced by Microsoft DMKT, also in 2017. Due to its speed and high performance [51,52], this model is widely used in solving regression, classification, and other ML problems. Like CatBoost, LightGBM has built-in support for encoding categorical variables.

Gradient boosting algorithms are retrained quite quickly, so it is necessary to accurately select hyperparameters to obtain a high-quality model. The number of decision trees, their depth, and the number of leaves (terminal nodes) are important parameters, as they are for a Random Forest model. Nonetheless, if, for a Random Forest, a large depth of trees, in most cases, has a positive role on the prediction quality, for gradient boosting, it is recommended to use an average depth to achieve a balance between learning and generalization. Other important parameters are the learning rate, which is the degree of contribution of each tree to the model’s prediction, the type of boosting algorithm (gbdt, dart, or goss) and the degree of regularization.

3.6. ML Model Quality Assessment

The efficiency of predicting a dichotomous variable in ML models is assessed by various metrics, calculated on the basis of the confusion matrix (error matrix). The confusion matrix is a classification of model prediction results into true positive ones (TP), true negative ones (TN), false positive ones (FP), and false negative ones (FN), as shown in Figure 4.

Since there is a slight class imbalance in the data under study, the quality of the model was assessed using the ROC AUC metric, which is immune to this problem. The ROC AUC metric is the area under the ROC curve (receiver operating characteristic curve), which displays the relationship between the number of correctly and incorrectly classified answers when varying the threshold of the decision rule [53]. ROC AUC effectively captures the trade-off between the true positive rate and the false positive rate, which is critical for our task, where missing a true event (high false negative rate) is as important as avoiding excessive false alarms (high false positive rate). The metric values range from 0 to 1, with 1 indicating a high-quality model, 0.5 performing a random prediction of the model’s answers, and 0 performing a false prediction—that is, the model predictions are opposite to the true values. Correctly classified TPR (true positive rate) and incorrectly classified FPR (false positive rate) answers are found by Equations (1) and (2).

$$\mathrm{T}\mathrm{P}\mathrm{R}={\displaystyle \frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}}}$$

(1)

$$\mathrm{F}\mathrm{P}\mathrm{R}={\displaystyle \frac{\mathrm{F}\mathrm{P}}{\mathrm{F}\mathrm{P}+\mathrm{T}\mathrm{N}}}$$

(2)

In addition to the ROC AUC metric also being taken as a loss function when training models, additional quality metrics were calculated to assess the model performance [36], such as the following:

-

Accuracy—proportion of correct answers:

$$\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}={\displaystyle \frac{\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}}{\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}+\mathrm{F}\mathrm{P}+\mathrm{F}\mathrm{N}}}$$

(3)

-

Precision—proportion of true positive predictions among all positive predictions of the model:

$$\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}={\displaystyle \frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}}}$$

(4)

-

Recall—proportion of true positive predictions among all positive cases:

$$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}={\displaystyle \frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}}}$$

(5)

-

F1score (F1 мepa)—harmonic mean of Recall and Precision, taking a value from 0 to 1 and allowing for assessing the model quality in the presence of an unbalanced data set:

$$\mathrm{F}1-\mathrm{s}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{e}={\displaystyle \frac{2\times \mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}\times \mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}{\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{\ast }\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}}$$

(6)

-

AUCPR—area under the PR curve, displaying the relationship between the metrics of Precision and Recall. AUCPR, unlike ROC, is sensitive to class imbalance.

4. Discussion

4.1. Model Training Results

The results of model training are summarized in

Table 2. Quality metrics of models.

ML Model	Accuracy	Recall	Precision	F1	AUC-PR	ROC-AUC
Support Vector Machine (CW)	0.687	0.685	0.606	0.642	0.672	0.768
Support Vector Machine (SMOTE-NC)	0.687	0.67	0.61	0.637	0.671	0.761
Logistic Regression (CW)	0.69	0.685	0.611	0.644	0.683	0.779
Logistic Regression (SMOTE-NC)	0.69	0.678	0.612	0.641	0.676	0.772
Random Forest Classifier (CW)	0.687	0.693	0.607	0.644	0.661	0.772
Random Forest Classifier (SMOTE-NC)	0.671	0.67	0.592	0.626	0.67	0.761
LightGBM Classifier (CW)	0.715	0.762	0.631	0.687	0.647	0.771
LightGBM Classifier (SMOTE-NC)	0.69	0.686	0.609	0.642	0.652	0.772
CatBoost Classifier (CW)	0.693	0.708	0.618	0.657	0.655	0.776
CatBoost Classifier (SMOTE-NC)	0.712	0.692	0.641	0.664	0.64	0.771
Dummy Model	0.519	0.523	0.431	0.472	0.411	0.5

, sorted by the ROC AUC metric.

Table 3. Hyperparameters of best models.

ML Model	Hyperparameters
Support Vector Machine	kernel: ‘rbf’; gamma: 0.04; C: 9.0; class_weight: ‘balanced’
Support Vector Machine (SMOTE)	kernel: ‘sigmoid’; gamma: 0.03; C: 6.0
Logistic Regression	penalty: ‘elasticnet’; solver: ‘saga’; l1_ratio: 0.8; C: 1; class_weight: ‘balanced’
Logistic Regression (SMOTE)	penalty: ‘elasticnet’; solver: ‘saga’; l1_ratio: 0.6; C: 1
Random Forest Classifier	n_estimators: 300; min_samples_split: 12; min_samples_leaf: 32; max_depth: 12; criterion: ‘log_loss’; class_weight: ‘balanced’
Random Forest Classifier (SMOTE)	n_estimators: 100; min_samples_split: 32; min_samples_leaf: 22; max_depth: 1
LightGBM Classifier	reg_lambda: 0.2; reg_alpha: 0.6; num_leaves: 31; n_estimators: 490; max_depth: 1; learning_rate: 0.01; boosting_type: ‘gbdt’; class_weight: ‘balanced’
LightGBM Classifier (SMOTE)	reg_lambda: 0.7; reg_alpha: 0.9; num_leaves: 47; n_estimators: 865; max_depth: 15; learning_rate: 0.001; boosting_type: ‘goss’
CatBoost Classifier	max_leaves: 16; learning_rate: 0.001; l2_leaf_reg: 14; iterations: 720; depth: 4; class_weight: ‘balanced’
CatBoost Classifier (SMOTE)	max_leaves: 16; learning_rate: 0.001; l2_leaf_reg: 14; iterations: 720; depth: 4

displays the hyperparameters of the top performing models. To check for adequacy, in addition to the considered models, a constant DummyClassifier model was added with a “uniform” classification strategy that generates predictions randomly with equal probability for each class.

There were experiments combining two methods for handling class imbalance: CW and the SMOTE-NC method. For all models analyzed, this combined approach did not lead to any performance increase compared to using each method separately. Although we believe there is a high probability that combining these methods could yield more satisfactory results, we decided not to include these findings in Table 2 to avoid visual clutter.

According to Table 2, all trained ML models showed results that were an order of magnitude higher than the predictions of the constant model. The best method for dealing with unbalanced sampling turned out to be the method of weighting classes. According to the ROC AUC metric, the best model was chosen to be a Logistic Regression with the class weighting method to combat class imbalance (0.779), which is reflected in the simplicity of the model itself and its good approximation. This model also showed the best results in terms of the AUC-PR metric (0.68), which indicates successful prediction of the target feature under the conditions of an unbalanced sample. The model was tested on a test set and showed excellent results—ROC AUC, with a value of 0.84. The metric value turned out to be higher than on the validation set, which indicates the absence of overtraining the model; however, large discrepancies between the metrics on different samples indicate a problem with insufficient data (there were only 79 objects in the test set). The problem of a small sample is clearly visible in the ROC and PR curves, which have sharp “breaks” due to insufficient data (Figure 5).

Despite this, the model copes with the task, and the graphical display of the ROC curve (Figure 5a) allows us to state that, by changing the classification threshold, a high result can be achieved. For example, if it is necessary that the model’s prediction includes all 100% of the power lines on which failures are observed (a lack of type I errors), then, according to the ROC curve graph (Figure 5a), up to 60% of power lines with no failures will be assigned to this class (type II errors). Conversely, if prediction accuracy for both classes is desired, then it can be calculated that, at a classification threshold of approximately 0.4, the TPR predictions will be 0.8 while the FPR predictions are only 0.2.

The PR curve (Figure 5b) shows that Precision stabilizes at a value of 0.72…0.75 with Recall from 0.35 to 0.78 and corresponding classification thresholds of 0.45…0.58. Thus, knowledge of the metrics discussed above at certain levels of the classification threshold makes it possible to regulate the number of errors of different types. The optimal threshold for deployment must be chosen based on the specific requirements of the end application.

4.2. Feature Importance Analysis by Embedded Methods

In Section 3.1 of this paper, an analysis of the correlation between features is carried out using the correlation coefficient phi ϕ_k, which is essentially one of the statistical methods (filtering methods). During this analysis, the degree of influence of the PTL parameters on the target variable “Outage fact” was determined, and, accordingly, the features were properly processed to achieve the best correlation. At the same time, many ML models themselves determine the degree of influence of features on the final prediction result, and accordingly optimize their influence to obtain the best result. Analysis of such data allows for an even better understanding of the features’ importance in the given task.

The selection of features in accordance with model parameters is called the embedded method for assessing the importance of features [54]. Regarding regression models, such a method is the analysis of weighting coefficients for each feature (degree of regularization), as it is shown for the distribution network fault diagnosis in North Carolina (USA) as the LASSO/ALASSO feature selection method [55]. For models based on decision trees, the appropriate method is the analysis of importance indicators for each feature, calculated at the training stage using the Gini criterion. Figure 6 shows the importance of features for the best ML models trained in this article: Logistic Regression and CatBoost Classifier.

As expected, the greatest contribution to the prediction of both models is made by the feature “Length of PTL overhead sections, km”, while the influence of this feature on the CatBoost Classifier prediction reaches 48%. Thus, it is once again established that the number of failures on a power line grows with its length. For this reason, the failure flow parameter, the failure number per certain length of power lines, is frequently used to evaluate the effectiveness of energy supply organizations.

The features determining the length of power lines in forested and populated areas were converted into relative units of the total length of power lines, so their influence on the model results turned out to be not so significant. Despite this, an interesting fact is the presence of an inverse relationship between power line failures and the relative length in a populated area (the weight coefficient of the LR is −0.27, with an importance for the CatBoost Classifier of 3%); that is, the longer the PTL length in a populated area, the lower the failure probability is. This can likely be attributed to the enhanced control and maintenance of power lines in populated areas, which is due to their critical social importance and potential risks to the public. In turn, the CatBoost Classifier (Figure 1b) assigns a 3% importance to the PTL length through the forest, while the weight coefficient for the LR model is only 0.02, which is quite small and indicates that tree and shrub vegetation are well cleared in the protective zone of 110 kV power lines, and their influence on failures is not evident.

A previous study on EDA [35] found that the number of supports clearly correlates with the length of power lines, so the current research used a synthetic feature reflecting the relative number of reinforced concrete supports out of the total number of supports (RC plus metal). It was revealed that a higher content of RC supports relative to metal supports entails an increase in the probability of failures on power lines. So, the importance of this feature in the CatBoost Classifier is 15.5%; the weight coefficient of the logistic model is 0.2.

The fact that the line is of transit has a strong influence on the probability of outages on 110 kV power lines. This feature is categorical, and in the data preparation algorithm proposed in this work, this feature was processed by the One-Hot Encoding method, which resulted in the formation of two features, the fact of PTL transit and the fact of PTL non-transit, both of which influence the CatBoost prediction with an importance of almost 10%. At the same time, the weight coefficients of the Logistic Regression also confirm the strong influence of this feature on predicting the power outage fact. The fact of transit has a direct relationship with the target variable (weight coefficient is 0.49), while the fact of non-transit has an inverse relationship (weight coefficient is −0.59).

The service life expressed through the “Overexploitation” feature affects the target variable rather weakly (the feature importance is 3.3%; the weight coefficient is 0.04). While this may seem counterintuitive, it can reflect the reality of maintenance practices for this specific infrastructure. The 110 kV PTLs are critical components of the power supply network. Consequently, they receive significant and consistent maintenance, which helps mitigate the effects of aging. This rigorous maintenance regimen likely prevents any significant correlation between the age of the line and its vulnerability to outages.

The influence of the PTL condition index on the target variable is quite unobvious. Logically, it is possible to assume that the worse the condition of the line is, the more probable the failure is. However, this fact is not observed when analyzing the importance of features—only 3.8% for predicting the CatBoost Classifier, while the weighting coefficient of the LR model is minus 0.04. Such a weak influence can be explained by the artificial control of this parameter in the reporting documents of electric grid companies, depending on the need for scheduled maintenance.

Considering the importance of the type of power line conductor on the model prediction, it is necessary to note that the weighting coefficients for conductors of the AC-185, AC-120, and AC-95 types are equal to zero, which indicates the absence of their influence on power line failures. However, there is a sufficient negative dependence between the AC-150 conductor and the target variable (−0.24). This fact can be explained by a small sample and the random specifics of electrical networks in the Orel region—AC-type conductors with a cross-section of 150 and 185 are more often of transit (as a percentage in 78% and 80% of cases, respectively), while conductors with a cross-section of 150 are only in 56% of cases.

5. Conclusions

As part of this study, it was proposed to solve classification task of predicting power supply outages on 110 kV power lines based on the technical parameters of the lines themselves by means of a supervised machine learning technique. Five classifiers were considered as ML algorithms: Support Vector Machine, Logistic Regression, Random Forest, LightGBM Classifier, and CatBoost Classifier. It was proposed to prepare data for the models using One-Hot Encoding and standardization techniques for categorical and quantitative variables, respectively. To automate the process of data conversion and eliminate the possibility of data leakage, Pipeline and Column Transformers were applied. The hyperparameters of the classifiers were adjusted using a randomized and exhaustive search over specified parameter values.

According to the ROC AUC metric, the best quality was obtained by Logistic Regression with the class weighting method to combat class imbalance. The model showed a result of 0.78, which was reflected in the simplicity of the model itself and its good approximation. This model also produced the highest results in terms of the AUC-PR metric (0.68), demonstrating that, even in an unbalanced sample, the target feature could be successfully predicted. Additionally, the Logistic Regression performed exceptionally well on the test sample, as evidenced by a ROC AUC of 0.84, which supported the lack of model overtraining. However, this result indicates the impact of the data shortage issue, which was seen in a marked excess of quality metrics on the test sample relative to the validation sample.

The use of embedded methods for assessing the importance of features for a Logistic Regression and CatBoost Classifier made it possible to assess the degree of influence of PTL parameters on power outage facts. As anticipated, the feature “Length of overhead power line sections” contributes the most to the prediction of both models (the importance in the CatBoost Classifier prediction reaches up to 48%). The PTL length in populated areas shows the presence of an inverse relationship with the target variable; that is, the greater the PTL length in this area is, the less probable the failure is. At the same time, the PTL length through the forest has a rather weak correlation with the failure fact. In terms of the support types, it was revealed that an increased proportion of reinforced concrete supports in comparison to metal supports results in a higher risk of power line failures. So, the importance of this feature in the CatBoost Classifier is 15.5%, while the weight coefficient of the logistic model is 0.2. The transit fact of the PTL has a significant impact on the probability of outages—the importance of this feature in the CatBoost Classifier is 10%. The transit fact has a direct relationship with the target variable (the LR weight coefficient is 0.49), while the non-transit fact has a reverse influence (the weight coefficient is −0.59). The service life and conditions of power lines have a negligible impact on the target variable. This feature analysis can serve as a valuable starting point for guiding future power network reconstruction and maintenance efforts.

The results of this study showed the possibility of predicting power outages on 110 kV power lines based on data on the technical parameters of the lines themselves, as can be seen from the derived quality metrics of ML models. Nevertheless, due to the limited data set (395 objects), it was not possible to achieve a consistent result. Specifically, the notable discrepancy between the test and validation set metrics suggests a high degree of model overfitting. Therefore, future research directions should focus on mitigating this limitation by expanding the data set to include power lines from other regions and/or incorporating data from additional time periods. This expansion is crucial for building more robust and generalizable models and moving toward a practical, deployable solution for outage prediction.