Substation Abnormal Scene Recognition Based on Two-Stage Contrastive Learning

Abstract

Substations are an important part of the power system, and the classification of abnormal substation scenes needs to be comprehensive and reliable. The abnormal scenes include multiple workpieces such as the main transformer body, insulators, dials, box doors, etc. In this research field, the scarcity of abnormal scene data in substations poses a significant challenge. To address this, we propose a few-show learning algorithm based on two-stage contrastive learning. In the first stage of model training, global and local contrastive learning losses are introduced, and images are transformed through extensive data augmentation to build a pre-trained model. On the basis of the built pre-trained model, the model is fine-tuned based on the contrast and classification losses of image pairs to identify the abnormal scene of the substation. By collecting abnormal substation images in real scenes, we create a few-shot learning dataset for abnormal substation scenes. Experimental results on the dataset demonstrate that our proposed method outperforms State-of-the-Art few-shot learning algorithms in classification accuracy.

1. Introduction

Substations are an indispensable part of the power system, and their normal operation is crucial for the stability and safety of the power system [1]. However, substation equipment is usually exposed to the open environment and is subject to natural weather conditions, leading to issues such as rust, aging, and component detachment. Additionally, staff non-compliance with operational standards and disturbances from the natural environment, such as bird nests on transformers, are common occurrences. These abnormal situations pose potential threats to the operational safety of substations, necessitating timely detection and identification of abnormal scenarios within substations. With the continuous expansion of substation scales, traditional manual inspection methods for power inspection tasks are facing challenges, the most significant of which is the low efficiency and susceptibility to environmental influences. Moreover, manual inspections are also limited by subjective judgments [2]. The subjective judgment of inspectors can be easily influenced by various factors, such as fatigue, visual limitations, and personal biases, thereby increasing potential safety risks. Ensuring the safe inspection of substation systems has become an important issue that needs to be addressed urgently. Given the limitations of traditional manual inspection methods, there is an urgent need to introduce new technologies and methods to improve inspection efficiency and accuracy.

Currently, substation equipment can be inspected by using inspection robots to collect images on-site, followed by applying machine learning algorithms for abnormal scenario recognition. Kong et al. [3] were the first to apply pattern recognition technology for the video monitoring of unattended substations, using the AdaBoost algorithm for weighted voting to detect moving targets, successfully achieving recognition of abnormal environments. Wu et al. [4] introduced the concept of a fault detection algorithm based on improved YOLO5 that sets a hyperparameter at each layer according to the recognition situation to re-extract features for substation equipment. Gao et al. [5] combined YOLOX with higher-order gated convolution to enhance the interaction capability in feature space, improving detection accuracy and robustness. However, the number of images for abnormal scenarios in substations is relatively small, resulting in an imbalance between positive and negative samples. Researchers have proposed innovative solutions for this issue, such as augmenting the training set with virtual abnormal samples generated using generative adversarial networks [6] to achieve a numerical balance between positive and negative samples. Additionally, techniques like transfer learning and semi-supervised learning can be employed to train with a few labeled abnormal samples [7]. However, the field primarily faces two challenges: (1) a large number of abnormal scenario categories with significant differences between each scenario and (2) the difficulty in collecting abnormal scenario samples, resulting in a limited number of samples. In fact, abnormal scenario recognition in substations is a typical few-shot learning problem. The backgrounds and objects to be identified in different abnormal situations exhibit distinct differences, which leads to poor generalization capabilities of existing models and difficulties in accurately identifying new abnormal scenarios.

Numerous scholars have conducted related research to address the challenges of machine learning in substation abnormal scene recognition with small sample sizes. In the field of few-shot learning, the researchers divided the dataset into a base set and a support set. The base set is used to train a foundational model with a limited number of labeled samples, and the support set is employed to evaluate and further fine-tune the model’s performance. The support set consists of a small number of support samples and query samples; support samples are used for model evaluation, and query samples are utilized for performance measurement. Small-sample models aim to classify query samples using support samples. To cope with few-shot learning scenarios, metric learning methods [8,9] are widely used to learn the feature space of samples in the base set, which is then applied to classify query samples in the support set. However, when recognizing abnormal scenes in substations, these methods face challenges due to the large differences in sample morphology, the scarcity of samples for certain defects, and large numbers of unlabeled samples. Therefore, some researchers have adopted self-supervised learning to perform the task. Usually, they use data augmentation on the base dataset to construct a pre-trained model and then fine-tune the pre-trained model with the support set to build a classifier. Nevertheless, existing algorithms neglect the distinctiveness and diversity among samples when handling abnormal scene samples in substations, resulting in poor classification performance. The abnormal scenarios in substations significantly differ in sample morphology, with specific categories of defects encompassing very few samples, and many samples are without annotated labels. Constructing a model that correctly identifies each defective instance under small-sample conditions poses a significant challenge. Inspired by self-supervised learning, this paper proposes a novel two-stage contrastive learning (2-SCL) method for identifying abnormal scenes in substations. It combines contrastive learning with various data augmentation techniques in the first stage to construct a pre-trained model, aiming to minimize the distance between original and augmented samples, thereby enhancing sample distinctiveness and improving the model’s inter-class discrimination and generalization ability. At the same time, to prevent the distance between similar samples from becoming too far, this paper proposes supervised contrastive learning for fine-tuning to further improve performance, effectively addressing the problems existing research in this field faces.

To implement self-supervised tasks, it is necessary to perform data augmentation on the original image [10]. In fact, data augmentation of samples is crucial for improving the algorithm’s performance. In the proposed algorithm, various complex transformations are used to generate multiple transformations and utilize self-supervised objectives to learn rich representations, thereby enabling the model to have better discrimination for different category samples. Furthermore, based on the pre-trained model, the support set enables the learned model to use fewer samples and classify query samples more accurately. The main contributions of this paper are as follows:

• A novel few-shot learning method, which utilizes two-stage contrastive learning (2-SCL) to construct the learning objective, is proposed. In the first stage, a pre-trained model is constructed through self-supervised learning, thereby enhancing the model’s inter-class discrimination ability.
• Supervised contrastive learning is introduced in the second stage of the model, further fine-tuning the pre-trained model within a supervised framework to improve sample identification capability and model generalization ability.
• In the self-supervised phase, multiple data augmentation methods are proposed to enhance sample diversity, thereby obtaining more generalized feature representations.
• A large-scale dataset of substation abnormal scenarios has been constructed, which can provide better data support for subsequent research.

2. Related Work

This section introduces some existing work on small-sample learning and self-supervised learning algorithms.

2.1. Few-Shot Learning

The primary objective of few-shot learning is to train a model that can classify the samples of unseen classes with limited training samples. Many researchers have proposed methods based on metric learning to address the few-shot learning problem. Early studies utilized Siamese neural networks [11] to recognize the similarity between support and query samples through the L1 distance (The distance measures the sum of the absolute differences between the coordinates of the feature vector). MatchingNet [12] employed different networks to extract features from support and query samples and calculated their similarity using cosine distance. In recent years, attention mechanisms have also been widely applied to few-shot learning. For instance, Ref. [13] introduced a cross-attention module to improve detection performance by capturing the accurate target area, while the self-attention mechanism [14] was used to explore task-specific information. Moreover, building a pre-trained model on a large dataset without class labels and then fine-tuning it on small-sample data [15] has been proven effective in enhancing recognition performance. RelationNets [16] first uses a Relation Network (RN) that learns an embedding and a deep non-linear distance metric for comparing query and sample items through end-to-end and episodic training. In contrast, RelationNets2 [17] learns multiple non-linear distance metrics for few-shot learning and achieves State-of-the-Art results. A prototypical network [18] is proposed for few-shot and zero-shot learning, which learns a metric space by computing distances to prototypes, achieving excellent results with a simpler design. Ref. [19] uses ridge regression as the main adaptation mechanism for few-shot learning to enable deep networks to quickly adapt to novel data, achieving competitive performance.

2.2. Self-Supervised Learning

Self-supervised learning is a method that constructs a pre-trained model in a supervised manner by introducing self-supervised objectives. Early studies [20] used a set of image-augmented samples to predict categories; while ref. [21] employed the model to predict the rotation angle of images to better extract features and improve classification performance. Self-supervised learning [22] has also been used to predict the position of patches after random cropping of images, thus enhancing model performance. Building on this, some studies generate missing images by adding random noise, cropping, or removing parts of the color channels in images, and using self-supervised learning [23,24] to restore images for better image representation. Contrastive learning is another typical self-supervised technique [25] that achieves this by minimizing the embedding distance of images from the same category and maximizing the embedding distance of images from different categories. Early studies [26] introduced negative samples from different categories into image construction to learn embedded features through a triplet loss function. Meanwhile, subsequent research [9] introduced multiple-pair losses to extract optimal embedded features.

Self-supervised learning is also widely applied in few-shot learning. Some researchers [10] regard self-supervised learning as a regularization method and learn based on specific weight selection of samples; others enhance sample labels through [27] a method to learn the joint distribution of small samples and self-supervised tasks. Since self-supervised learning does not require class labels, constructing a pre-trained model with a large amount of unlabeled data and then fine-tuning it with small samples to adapt to specific tasks [28] has been proven to be effective. Based on this idea, researchers have also proposed models based on contrastive learning [29], which greatly enhances the model’s generalization ability by utilizing supervised contrastive learning from different views.

3. Materials and Methods

The classification task in few-shot learning further divides a dataset into training, validation, and test sets by splitting each set into a support set ${\mathrm{D}}_{\mathrm{s}}$ and a query set ${\mathrm{D}}_{\mathrm{q}}$ for each meta-learning task. In a standard N-way, K-shot meta-learning task, each iteration selects N classes from the dataset to construct ${\mathrm{D}}_{\mathrm{s}}$ and ${\mathrm{D}}_{\mathrm{q}}$, which includes K samples from each class for category feature extraction. The purpose is to correctly categorize the query set ${\mathrm{D}}_{\mathrm{q}}$ into N classes. Here we provide the definitions:

Assuming the meta-training set ${\mathrm{D}}_{\mathrm{t}\mathrm{r}}=\left\{\right({\mathrm{x}}_{\mathrm{i}},{\mathrm{y}}_{\mathrm{i}}\left)\right|{\mathrm{y}}_{\mathrm{i}}\in {\mathrm{C}}_{\mathrm{b}\mathrm{a}\mathrm{s}\mathrm{e}}\}$ contains samples from the base categories C_base, and the meta-test set ${\mathrm{D}}_{\mathrm{t}\mathrm{e}\mathrm{s}\mathrm{t}}=\left\{\right({\mathrm{x}}_{\mathrm{i}},{\mathrm{y}}_{\mathrm{i}}\left)\right|{\mathrm{y}}_{\mathrm{i}}\in {\mathrm{C}}_{\mathrm{n}\mathrm{e}\mathrm{w}}\}$ contains samples from new categories C_new, where y_i is the category label of the sample x_i. The goal of small-sample learning is to learn a model based on the training set D_tr and apply it to the test set D_t, where the meta-training set and the meta-test set have no intersection. In this paper, we adopt a simulated evaluation design. The specific implementation process is as follows: First, M categories are randomly sampled from the base categories for meta-training, and K instances are sampled from each category to obtain the support set: ${\mathrm{S}=\left\{\right({\mathrm{x}}_{\mathrm{i}},{\mathrm{y}}_{\mathrm{i}}\left)\right|}_{\mathrm{i}=1}^{\mathrm{M}\mathrm{*}\mathrm{K}}$. Based on this, we sample Q instances from each selected category to obtain the query set: ${{\mathrm{D}}_{\mathrm{q}}=\left\{\right({\mathrm{x}}_{\mathrm{i}},{\mathrm{y}}_{\mathrm{i}}\left)\right|}_{\mathrm{i}=1}^{\mathrm{M}\mathrm{*}\mathrm{Q}}$. The category labels y_i ∈ {1, 2, …, M}, ${\mathrm{D}}_{\mathrm{s}}$ and ${\mathrm{D}}_{\mathrm{q}}$ have no intersection. During the training process, the samples are classified into categories corresponding to the samples in the set. Figure 1 shows the distribution of samples in the feature space after contrastive learning, which can make it easier to classify the samples into the specified classes in the feature space.

In response to few-shot learning, this paper proposes a Two-stage Contrastive Learning algorithm (2-SCL), which applies contrastive learning in two phases to obtain more generalized representations. The overall framework of our proposed Two-stage Contrastive Learning algorithm (2-SCL) is illustrated in Figure 2. The proposed framework consists of two distinct yet complementary stages: the pre-training stage and the fine-tuning stage.

The first phase, the pre-training stage, is designed to build a strong foundation for the model. By applying contrastive learning without needing sample labels, we can avoid overfitting and leverage the vast amount of unlabeled data. The global and local contrastive losses work together to enhance the model’s feature representation, making it more discriminative and generalizable. This stage is shown in Figure 2a; samples are transformed into images through different data augmentation methods (data augmentation method 1, DA1, and data augmentation method 2, DA2). These augmented samples are then input into a convolutional neural network, which extracts their feature representations that are further processed through a pooling layer and a fully connected layer to output feature vectors. These feature vectors are used to calculate global contrastive loss. Simultaneously, after passing through a fully connected layer, the feature maps are input into a locally mapped layer proposed in this paper. This layer is designed to calculate the local contrastive loss, which helps capture local discriminative information that the global loss function might overlook. Through self-supervised loss, the model is guided to produce more generalized representations.

The second phase, the fine-tuning stage, is designed to use a pre-trained model for the classification task. In the fine-tuning phase of the network, as shown in Figure 2b, different data augmentations and sample pairs are input. These inputs are then encoded to extract relevant features. The extracted features are used to calculate the contrastive loss and the final classification loss. The contrastive loss is calculated by considering two images from the same category as positive pairs and two images from different categories as negative pairs. This helps the model to overcome biases from defective image samples. The classification loss is calculated based on the probability that an image belongs to a particular category. By jointly training with these two losses, the model is able to achieve good generalization performance across small-sample practical tasks. This two-stage process enables the model to better identify substation abnormal scenes compared to State-of-the-Art methods.

3.1. Global Self-Supervised Contrastive Loss

Global contrastive loss is utilized to enhance the similarity between different views of the same image while minimizing the similarity between views of different images. Assuming that two data augmentation methods are randomly applied to N samples ${\{{\mathrm{x}}_{\mathrm{i}},{\mathrm{y}}_{\mathrm{i}}\}|}_{\mathrm{i}=1}^{\mathrm{N}}$ in the meta-training set Dt, generating 2N augmented samples ${\{\widetilde{{\mathrm{x}}_{\mathrm{i}}},\widetilde{{\mathrm{y}}_{\mathrm{i}}}\}|}_{\mathrm{i}=1}^{\mathrm{N}},{\{{\tilde{\mathrm{x}}}_{\mathrm{i}}^{’},{\tilde{\mathrm{y}}}_{\mathrm{i}}^{’}\}|}_{\mathrm{i}=1}^{\mathrm{N}}$, where $\widetilde{{\mathrm{x}}_{\mathrm{i}}}$ and ${\tilde{\mathrm{x}}}_{\mathrm{i}}^{\mathrm{‘}}$ represent two different views of the original sample ${\mathrm{x}}_{\mathrm{i}}$ and are considered as positive pairs. Here, we define ${\mathrm{f}}_{\mathrm{\varnothing }}$ as the feature extractor, which can transform the samples ${\mathrm{x}}_{\mathrm{i}}$ into feature maps ${\widehat{\mathrm{x}}}_{\mathrm{i}}={\mathrm{f}}_{\mathrm{\varnothing }}\left(\widetilde{{\mathrm{x}}_{\mathrm{i}}}\right)\in {\mathrm{R}}^{\mathrm{C}\times \mathrm{H}\times \mathrm{W}}\mathrm{a}\mathrm{n}\mathrm{d}{\widehat{\mathrm{x}}}_{\mathrm{i}}^{’}={\mathrm{f}}_{\mathrm{\varnothing }}\left({\tilde{\mathrm{x}}}_{\mathrm{i}}^{’}\right)\in {\mathrm{R}}^{\mathrm{C}\times \mathrm{H}\times \mathrm{W}}$ according to the learned parameters ϕ, and further obtain global features ${\mathrm{h}}_{\mathrm{i}}\mathrm{a}\mathrm{n}\mathrm{d}{\mathrm{h}}_{\mathrm{i}}^{’}\in {\mathrm{R}}^{\mathrm{C}}$ through a pooling layer. A multi-layer perception is used to convert the global features into a vector, generating the projected vector ${\mathrm{z}}_{\mathrm{i}}=\mathrm{F}\mathrm{C}\left({\mathrm{h}}_{\mathrm{i}}\right),{\mathrm{z}}_{\mathrm{i}}^{’}=\mathrm{F}\mathrm{C}\left({\mathrm{h}}_{\mathrm{i}}^{’}\right)\in {\mathrm{R}}^{\mathrm{D}}$. Then, the global self-supervised contrastive loss can be calculated as:

$${\mathrm{L}}_{\mathrm{g}\mathrm{l}\mathrm{o}\mathrm{b}\mathrm{a}\mathrm{l}}=-\sum _{\mathrm{i}=1}^{\mathrm{N}}\mathrm{l}\mathrm{o}\mathrm{g}{\displaystyle \frac{\mathrm{e}\mathrm{x}\mathrm{p}({\mathrm{z}}_{\mathrm{i}}·{\mathrm{z}}_{\mathrm{i}}^{’}/\mathsf{\tau })}{\sum _{\mathrm{j}=1}^{\mathrm{N}}\mathrm{e}\mathrm{x}\mathrm{p}({\mathrm{z}}_{\mathrm{i}}·{\mathrm{z}}_{\mathrm{j}}^{’}/\mathsf{\tau })}}$$

(1)

In the given context, the operation “·” represents the dot product after L2 normalization (The L2 norm is the square root of the sum of squares for absolute differences between feature vector coordinates.), where N is the number of images in a batch, τ is a temperature parameter used to control the penalty on negative samples, and the positive pair ${\mathrm{z}}_{\mathrm{i}}^{’}$ and ${\mathrm{z}}_{\mathrm{i}}$ consists of feature vectors extracted from the same sample ${\mathrm{x}}_{\mathrm{i}}$ through different augmentation methods.

3.2. Local Self-Supervised Contrastive Loss

Although the global loss function based on the global feature vector h_i is beneficial for obtaining a global representation, it may overlook some local discriminative information in the feature maps, which can be more effective in small-sample learning. Therefore, this paper proposes a local self-supervised contrastive loss. Unlike previous methods, we utilize a local mapping module (referred to as Loc Map in Figure 2a) to enhance the robustness and generalization ability of the representation, with the specific module shown in Figure 3. Here, three spatial projection heads ${\mathrm{f}}_{\mathrm{q}}$, ${\mathrm{f}}_{\mathrm{k}}$, ${\mathrm{f}}_{\mathrm{v}}$ are used to project the local feature maps ${\widehat{\mathrm{x}}}_{\mathrm{i}}$ into queries ${\mathrm{q}}_{\mathrm{i}}={\mathrm{f}}_{\mathrm{q}}\left({\widehat{\mathrm{x}}}_{\mathrm{i}}\right)$, keys ${\mathrm{k}}_{\mathrm{i}}={\mathrm{f}}_{\mathrm{k}}\left({\widehat{\mathrm{x}}}_{\mathrm{i}}\right)$, and values ${\mathrm{v}}_{\mathrm{i}}={\mathrm{f}}_{\mathrm{k}}\left({\widehat{\mathrm{x}}}_{\mathrm{i}}\right)$, respectively, where ${\mathrm{q}}_{\mathrm{i}}$, ${\mathrm{k}}_{\mathrm{i}}$, ${\mathrm{v}}_{\mathrm{i}}\in {\mathrm{R}}^{\mathrm{H}\mathrm{W}\times \mathrm{D}}$. For a pair of local feature maps ${\widehat{\mathrm{x}}}_{\mathrm{a}}$ and ${\widehat{\mathrm{x}}}_{\mathrm{b}}$, we align ${\widehat{\mathrm{x}}}_{\mathrm{a}}$ and ${\widehat{\mathrm{x}}}_{\mathrm{b}}$ to obtain ${\mathrm{v}}_{\mathrm{a}|\mathrm{b}}^{’}=\mathrm{s}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}({\displaystyle \frac{{\mathrm{q}}_{\mathrm{b}}{\mathrm{k}}_{\mathrm{a}}^{\mathrm{T}}}{\sqrt{\mathrm{d}}}}){\mathrm{v}}_{\mathrm{a}}$, and align ${\widehat{\mathrm{x}}}_{\mathrm{b}}$ with ${\widehat{\mathrm{x}}}_{\mathrm{a}}$ to obtain ${\mathrm{v}}_{\mathrm{b}|\mathrm{a}}^{’}=\mathrm{s}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}({\displaystyle \frac{{\mathrm{q}}_{\mathrm{a}}{\mathrm{k}}_{\mathrm{b}}^{\mathrm{T}}}{\sqrt{\mathrm{d}}}}{)\mathrm{v}}_{\mathrm{a}}$. After L2 normalizing each position (i, j) in the aligned results, we can calculate the similarity between the two local feature maps, ${\widehat{\mathrm{x}}}_{\mathrm{a}}$ and ${\widehat{\mathrm{x}}}_{\mathrm{b}}$, defined as follows:

$$\mathrm{s}\mathrm{i}\mathrm{m}({\widehat{\mathrm{x}}}_{\mathrm{a}},{\widehat{\mathrm{x}}}_{\mathrm{b}})={\displaystyle \frac{1}{\mathrm{H}\mathrm{W}}}\sum _{\mathrm{i}=1,\mathrm{j}=1}^{\mathrm{i}=\mathrm{H},\mathrm{j}=\mathrm{W}}{\left({\mathrm{v}}_{\mathrm{a}|\mathrm{b}}^{’}\right)}_{\mathrm{i}\mathrm{j}}^{\mathrm{T}}{\left({\mathrm{v}}_{\mathrm{b}|\mathrm{a}}^{’}\right)}_{\mathrm{i}\mathrm{j}}$$

(2)

The above formula calculates the similarity between two feature maps ${\mathrm{v}}_{\mathrm{a}|\mathrm{b}}^{’}$ and ${\mathrm{v}}_{\mathrm{b}|\mathrm{a}}^{’}$ (both belonging to ${\mathrm{R}}^{\mathrm{H}\mathrm{W}\times \mathrm{D}}$). The self-supervised contrastive loss for the pair of feature maps can be computed as follows:

$${\mathrm{L}}_{\mathrm{l}\mathrm{o}\mathrm{c}\mathrm{a}\mathrm{l}}=-\sum _{\mathrm{i}=1}^{\mathrm{N}}\mathrm{l}\mathrm{o}\mathrm{g}{\displaystyle \frac{\mathrm{e}\mathrm{x}\mathrm{p}\left(\mathrm{s}\mathrm{i}\mathrm{m}\right({\widehat{\mathrm{x}}}_{\mathrm{i}}·{\widehat{\mathrm{x}}}_{\mathrm{j}}^{’})/{\mathsf{\tau }}_1)}{\sum _{\mathrm{j}=1}^{\mathrm{N}}\mathrm{e}\mathrm{x}\mathrm{p}\left(\mathrm{s}\mathrm{i}\mathrm{m}\right({\widehat{\mathrm{x}}}_{\mathrm{i}}·{\widehat{\mathrm{x}}}_{\mathrm{j}}^{’})/{\mathsf{\tau }}_1)}}$$

(3)

Here, (${\widehat{\mathrm{x}}}_{\mathrm{i}}·{\widehat{\mathrm{x}}}_{\mathrm{j}}^{’}$) is a positive pair, i ≠ j, and ${\mathsf{\tau }}_1$ represents the temperature parameter. At the same time, we employ a local module to further mine the local contrastive information between instances, as shown in Figure 3. During the pre-training phase, we minimize the following loss:

$${\mathrm{L}}_{\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{g}\mathrm{e}1}={\mathsf{\alpha }}_1{\mathrm{L}}_{\mathrm{g}\mathrm{l}\mathrm{o}\mathrm{b}\mathrm{a}\mathrm{l}}+{\mathsf{\alpha }}_2{\mathrm{L}}_{\mathrm{l}\mathrm{o}\mathrm{c}\mathrm{a}\mathrm{l}}$$

(4)

where α₁ and α₂ are weight factors that balance the global and local losses. By optimizing Lstage1, the model’s feature representation is enhanced in terms of discriminability and generalization, further preparing it for the next step of supervised fine-tuning.

3.3. Supervised Fine-Tune

In the first stage, through self-supervised learning, the model effectively utilizes a large number of unlabeled abnormal image samples. In the second stage, the model is provided with a limited number of samples, for example, only one sample or five samples per substation anomaly sample image. To better recognize substation anomaly samples, we employ a supervised contrastive learning approach and train it in conjunction with the image classification loss. We consider two images from the same category as positive pairs and two images from different categories as negative pairs for contrastive learning. The corresponding loss is as follows:

$${\mathrm{L}}_{\mathrm{c}\mathrm{o}\mathrm{n}}=-{\displaystyle \frac{1}{\mathrm{T}}}{\sum }_{\mathrm{i}=1}^{\mathrm{N}}{\sum }_{\mathrm{j}=1}^{\mathrm{N}}1{\mathrm{y}}_{\mathrm{i}}={\mathrm{y}}_{\mathrm{j}}\mathrm{l}\mathrm{o}\mathrm{g}{\displaystyle \frac{\mathrm{e}\mathrm{x}\mathrm{p}\left(\mathrm{c}\mathrm{o}\mathrm{s}\right({\mathrm{h}}_{\mathrm{i}},{\mathrm{h}}_{\mathrm{j}})/\mathsf{\tau })}{\sum _{\mathrm{k}=1}^{\mathrm{N}}\mathrm{e}\mathrm{x}\mathrm{p}\left(\mathrm{c}\mathrm{o}\mathrm{s}\right({\mathrm{h}}_{\mathrm{i}},{\mathrm{h}}_{\mathrm{k}})/\mathsf{\tau })}}$$

(5)

where T is the number of sample pairs from the same category within a batch. cos() refers to the cosine similarity between two vectors. The classification loss for the abnormal images is as follows:

$${\mathrm{L}}_{\mathrm{c}\mathrm{l}\mathrm{s}}=-{\displaystyle \frac{1}{\mathrm{N}}}{\sum }_{\mathrm{j}=1}^{\mathrm{C}}{\sum }_{\mathrm{i}=1}^{\mathrm{N}}\mathrm{l}\mathrm{o}\mathrm{g}\mathrm{P}\left({\mathrm{C}}_{\mathrm{j}}\right|{\mathrm{x}}_{\mathrm{i}})$$

(6)

where $\mathrm{P}\left({\mathrm{C}}_{\mathrm{j}}\right|{\mathrm{u}}_{\mathrm{i}})$ is the probability that the i-th image is predicted to have the j-th category. We jointly train these two losses in each batch:

$${\mathrm{L}}_{\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{g}\mathrm{e}2}={\mathsf{\beta }}_1{\mathrm{L}}_{\mathrm{c}\mathrm{o}\mathrm{n}}+{{\mathsf{\beta }}_2\mathrm{L}}_{\mathrm{c}\mathrm{l}\mathrm{s}}$$

(7)

where ${\mathsf{\beta }}_1$ and ${\mathsf{\beta }}_2$ are weight factors serving as hyperparameters.

4. Experimental Results and Discussion

4.1. Dataset of Abnormal Scenes in Substation

This study creates a dataset of abnormal scenarios encountered in substations, capturing instances of blurred dials, damaged dials, fractured insulators, ground oil contamination, closed cabinet doors, bird nests, damaged cover plates, abnormal meter readings, and discolored silicone, as illustrated in Figure 4. The other six classes include broken dial, discoloration of silica gel, broken insulator, abnormal closure of the cabinet door, suspended objects in the air, and not wearing a safety helmet. The dataset comprises a total of 3000 images, with each anomaly class containing 200 images. For the experimental setup, ten classes are chosen for the training set, and five classes are designated for the test set. No labels were used when constructing the pre-trained model.

4.2. Experimental Setup

In this experiment, Conv-4 [16] and ResNet-12 [8], widely used as backbone networks by researchers, are adopted for feature extraction. Conv-4 consists of four repeated convolutional blocks, each with a kernel size of 3, stride of 1, padding of 1 for convolutional layer, a BatchNorm layer, a ReLU function, and a size of 2 for max pooling layer. ResNet-12 has four residual blocks, each containing three convolutional layers, where “12” means a total of twelve convolutional layers. To prevent the impact of size differences on model performance, the image size in both the training and test sets is set to the same number of pixels. The network is trained using the Stochastic Gradient Descent (SGD) optimizer in the pre-training and meta-training fine-tuning stages. During pre-training (first stage) and meta-training (second stage), the weight decay and momentum are set to 4 × 10⁻⁴ and 0.8, respectively. During pre-training, the learning rate is initialized to 0.1 and adjusted using a cosine learning rate scheduler after a warm-up period. The training lasts for 20 epochs with a batch size of 64. The temperature parameters τ and τ1 are both set to 0.1. The weight factors α and β are set to 0.7 and 0.3, respectively, allowing the model to focus more on the global loss during the first pre-training stage. For different meta-learning fine-tuning tasks, the pre-trained model is fine-tuned with a 5-way 5-shot (5 training samples per class) and a 5-way 1-shot setting, with a batch size of 16 for a total of 30 epochs. The weight factors ${\mathsf{\beta }}_1$ and ${\mathsf{\beta }}_2$ are set to 0.2 and 0.8, respectively, with the loss function focusing more on the classification loss in the second stage. The entire model training is conducted on a single NVIDIA 3090 GPU with 24 GB of video memory. Data augmentation: for the constructed dataset, during the pre-training stage, various data augmentation strategies are used for contrastive learning, including image transformations such as random resized cropping, color jittering, and random horizontal flipping, as well as random grayscale conversion. In the second stage, during the fine-tuning of supervised contrastive learning, data augmentation is combined with data pairing strategies due to the limited amount of data.

4.3. Evaluation Metrics

In our experiment, the created dataset is a multi-class classification problem. The multi-class distribution of the test dataset cannot reflect the effect of the classificaton of few-shot learning. Therefore, binary classification experiments were carried out on the created dataset to make the evaluation fairer. The final classification results are divided into four states: TP (true positive), FP (false positive), TN (true negative), and FN (false negative). These are also four basic metrics of the confusion matrix. TP is the number of normal images classified in the normal scene. FP is the number of abnormal images incorrectly classified in the normal scene. TN is the number of abnormal images classified correctly. FN is the number of normal images classified in abnormal scenes. To evaluate the performance of the proposed method, four states—the accuracy, precision, detection rate, and false alarm rate—and F-measure are defined below:

$$\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}={\displaystyle \frac{TP+TN}{TP+TN+FN+FP}}$$

(8)

$$\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}={\displaystyle \frac{TP}{TP+FP}}$$

(9)

$$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}={\displaystyle \frac{TP}{TP+FN}}$$

(10)

$$\mathrm{F}1-\mathrm{S}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{e}={\displaystyle \frac{2\ast (Precision\ast Recall)}{Precision+Recall}}$$

(11)

4.4. Experimental Results

In this research, the binary classification, which has only two categories, namely normal and abnormal, is used to calculate the evaluation metrics. All evaluation metrics are reported as averaged results with 95% confidence intervals based on 30 experiments. This subsection presents the conducted meta-learning experiments, including 5-way 1-shot and 5-way 5-shot experiments, and reports the classification accuracy, precision, F1-score, and recall in percentage form. The feature extraction backbone networks are Conv-4 and ResNet-12. The detailed evaluation metric is calculated as shown in

Table 1. Results of substation anomaly detection with Conv-4 as the backbone.

Backbone	Methods	5-Way 1-Shot				5-Way 5-Shot
		Accuracy	Precision	F1-Score	Recall	Accuracy	Precision	F1-Score	Recall
Conv-4	RelationNets [16]	61.76 ± 0.48	58.32 ± 0.81	57.5 ± 0.43	56.71 ± 0.54	77.59 ± 0.73	68.27 ± 0.78	70.52 ± 0.81	72.93 ± 0.76
	MatchingNets [12]	64.52 ± 0.67	56.14 ± 0.63	59.13 ± 0.52	62.45 ± 0.7	80.63 ± 0.53	73.37 ± 0.54	72.55 ± 0.65	71.76 ± 0.64
	PrototypicalNets [18]	67.67 ± 0.58	56.21 ± 0.59	60.84 ± 0.59	66.31 ± 0.63	82.39 ± 0.42	74.15 ± 0.61	73.32 ± 0.73	72.5 ± 0.75
	R2D2 [19]	69.56 ± 0.42	62.6 ± 0.53	64.94 ± 0.72	67.47 ± 051	87.65 ± 0.63	64.86 ± 0.65	73.9 ± 0.72	85.89 ± 0.76
	RelationNets2 [17]	72.12 ± 0.82	64.9 ± 1.21	67.66 ± 0.93	70.67 ± 0.36	85.49 ± 0.59	79.5 ± 0.63	76.39 ± 0.58	73.52 ± 0.65
	2-SCL (ours)	73.76 ± 0.56	67.85 ± 0.48	69.64 ± 0.51	71.54 ± 0.46	88.76 ± 0.56	82.54 ± 0.55	84.29 ± 0.54	86.13 ± 0.51

and

Table 2. Results of substation anomaly detection with ResNet-12 as the backbone.

Backbone	Methods	5-Way 1-Shot				5-Way 5-Shot
		Accuracy	Precision	F1-Score	Recall	Accuracy	Precision	F1-Score	Recall
ResNet-12	RelationNets [16]	72.56 ± 0.63	61.67 ± 0.75	65.23 ± 0.67	69.23 ± 0.59	85.93 ± 0.49	74.75 ± 0.77	70.67 ± 0.72	67.02 ± 0.74
	MatchingNets [12]	75.91 ± 0.49	72.18 ± 0.47	71.87 ± 0.46	71.58 ± 0.53	88.37 ± 0.69	82.18 ± 0.61	80.37 ± 0.63	78.64 ± 0.64
	PrototypicalNets [18]	76.43 ± 0.52	77.12 ± 0.43	75.08 ± 0.48	73.15 ± 0.51	89.61 ± 0.83	85.12 ± 0.72	82.82 ± 0.76	80.64 ± 0.81
	TADAM [8]	81.72 ± 0.67	74.32 ± 0.68	77.19 ± 0.59	80.3 ± 0.57	90.72 ± 0.72	77.11 ± 0.67	80.62 ± 0.63	84.46 ± 0.66
	R2D2 [19]	82.61 ± 0.73	81.23 ± 0.78	79.5 ± 0.72	77.85 ± 0.71	91.59 ± 0.51	85.18 ± 0.58	82.83 ± 0.56	80.6 ± 0.54
	2-SCL (ours)	83.13 ± 0.49	84.35 ± 0.48	82.77 ± 0.46	81.25 ± 0.42	93.12 ± 0.64	86.6 ± 0.52	87.97 ± 0.5	89.39 ± 0.48

. Compared to other methods, 2-SCL significantly improves classification performance across various metrics. Since the data are balanced, in terms of accuracy, when Conv-4 and ResNet-12 are employed as the feature extraction backbone network, 2-SCL achieves a 12% and 10.57% improvement over RelationNet [16] in the 5-way 1-shot task and an 11.17% and 7.19% improvement in the 5-way 5-shot task. Under Conv-4, compared with the current top algorithm, 2-SCL improves accuracy by 1.64% in 5-way 1-shot and 1.21% in 5-way 5-shot. ResNet-12 improves accuracy by 0.52% in 5-way 1-shot and 1.53% in 5-way 5-shot. Other performance indicators also verify the same result. This indicates that 2-SCL can effectively enhance classification accuracy by utilizing the similarity of different attributes for classification. Although MatchingNets [12] and PrototypicalNets [18] are much better, their classification performance is not as good as that of 2-SCL. These results further demonstrate the proposed method’s effectiveness and superiority and the important role of two-stage training in meta-learning tasks. Additionally, as shown in Table 2, replacing the feature extraction network with a deeper network, ResNet-12, effectively improves classification accuracy. In the 5-way 5-shot task, 2-SCL can achieve an accuracy rate of 93.12%. This suggests that 2-SCL can effectively classify images to new categories under conditions of scarce samples, thus better adapting to the complex and variable environment of substations and being able to timely identify anomalies to prevent potential problems.

4.5. Ablation Experiments

We conducted ablation studies to investigate the effects of self-supervised contrastive pre-training and supervised contrastive fine-tuning.

Table 3. Testing accuracy (×100%) under different settings.

Model	5-Way 1-Shot	5-Way 5-Shot
(ours)	83.83 ± 0.57	93.12 ± 0.43
w/o Contrastive pre-training	−8.15	−6.13
w/o Supervised contrastive learning	−4.56	−3.32
w/o Contrastive pre-training + w/o Supervised contrastive learning	−9.35	−7.69

shows the test results of the proposed model on the constructed dataset. The experimental results indicate that both stages are necessary, and the model can only achieve optimal performance by fine-tuning the pre-trained model. The self-supervised contrastive pre-training conducted in the first stage is indispensable, as all datasets have a significant drop in performance. The experiments demonstrate that contrastive pre-training algorithm with unlabel data, helps the model distinguish images with similar defects. Furthermore, performance also declines if supervised contrastive learning is not included in the small-sample fine-tuning phase. Specifically, in the 1-shot and 5-shot experiments, classification performance decreased by 4.56% and 3.32%, respectively; this is because the constructed dataset includes some similar defect images, where supervised contrastive learning can clearly differentiate images that are semantically similar but have limited training samples. The experiment also attempted to train the first and second stages together; however, compared to the proposed model structure, there was almost no improvement in recognition performance. Additionally, the experiment studied whether contrastive pre-training without including the test dataset aids in defect image recognition. Specifically, we performed pre-training using the training dataset in the first stage and used the test data for small-sample learning in the second stage. Compared to the model that did not undergo contrastive pre-training in the first stage, performance improved by 8.15% and 6.13% in the 1-shot and 5-shot settings, respectively. These improvements suggest that contrastive pre-training enhances recognition performance on the test dataset.

The proposed method can be abstracted into a multi-label classifier based on supervised contrastive meta-learning in the second stage, capable of simultaneously recognizing multiple attribute features within an image. Two loss functions are employed during training to enhance the model’s generalization and adaptability: the contrastive loss and the classification loss. The contrastive loss measures the model’s performance across all meta-learning tasks by calculating the discrepancy between the model’s predictions and the true labels, allowing the model to learn common features among different meta-learning tasks, thereby improving its generalization and adaptability. The classification loss, on the other hand, measures the model’s performance on each meta-learning task, enabling the model to learn the commonalities and differences between different samples and to understand the relationships between the intrinsic attribute features of each sample. Subsequently, ablation studies were conducted to explore the roles of these two losses. The model’s accuracy on the test set was observed when the contrastive and classification losses were removed separately. As shown in

Table 4. Results of ablation experiments of different loss components.

Backbone	Contrastive Loss	Classification Loss	5-Way 1-Shot	5-Way 5-Shot
Conv-4	×	√	62.43 ± 0.96	75.32 ± 0.87
	√	×	71.29 ± 0.42	85.91 ± 0.62
	√	√	73.76 ± 0.56	88.76 ± 0.56
ResNet-12	×	√	73.89 ± 0.45	82.19 ± 0.61
	√	×	79.47 ± 0.68	88.40 ± 0.55
	√	√	83.83 ± 0.57	93.12 ± 0.43

“×” and “√” indicate that the model does not contain and contain the corresponding loss respectively.

, the removal of either loss led to a decrease in accuracy to varying degrees. Taking the Conv-4 backbone as an example, the accuracy dropped by approximately 10% when only the contrastive loss was included. However, relying solely on contrastive loss is insufficient because it overlooks the mutually exclusive attribute features between different samples, leading to decreased accuracy. Meanwhile, including only the classification loss causes the network to focus solely on the attribute feature relationships between categories within each meta-learning task, failing to learn broader feature relationships based on global features. Although this allows the model to learn the relationships between each sample’s intrinsic attribute features, improving its accuracy and sensitivity, the global features are neglected, resulting in the failure to further enhance the recognition performance.The contrastive features are shared across all meta-learning tasks, enabling the model to learn a wider range of feature relationships and thus enhancing its generalization ability. Additionally, the same pattern applies to the ResNet-12 backbone, which once again confirms the effectiveness of using both contrastive loss and classification loss during training.

4.6. Parameter Sensitivity Experiment

In this subsection, we also investigated the impact of hyperparameters in contrastive learning, specifically the temperature parameters τ and τ1, as well as the weight factors α and β. We set τ and τ1 to be within the range {0.05, 0.1, 0.3, 0.5} {0.05, 0.1, 0.3, 0.5} and ${\mathsf{\alpha }}_1$ + ${\mathsf{\alpha }}_2$ = 1. Experimental results showed that τ and τ1 significantly affect the performance during the first stage of self-supervised contrastive pre-training. Additionally, we found that a batch size greater than 32 performs well in the pre-training phase. However, in the few-shot fine-tuning phase, setting τ to a smaller value of 0.05, which strongly increases the penalty on hard negative samples, ${\mathsf{\beta }}_1$ + ${\mathsf{\beta }}_2$ = 1, ${\mathsf{\beta }}_1$ to a smaller value, and ${\mathsf{\beta }}_2$ to a larger one significantly improves the model’s classification performance. This is because it increases the weight of the supervised contrastive learning loss. Moreover, the batch size also affects performance in this stage. Therefore, when the number of training samples is limited, the supervised contrastive loss is sensitive to hyperparameters. In the fine-tuning stage of model training, ${\mathsf{\beta }}_1$ was set to 0.2, 0.4, 0.6, and 0.8 to explore its role as a hyperparameter.

Table 5. Results of different coefficient ${\mathsf{\beta }}_1$ for 2-SCL.

Backbone	${\mathsf{\beta }}_1$	5-Way 1-Shot	5-Way 5-Shot
Conv-4	0.2	72.53 ± 0.82	86.37 ± 0.63
	0.4	73.76 ± 0.56	88.76 ± 0.56
	0.6	71.61 ± 0.42	84.30 ± 0.67
	0.8	72.66 ± 0.15	85.81 ± 0.72
ResNet-12	0.2	80.18 ± 0.49	89.29 ± 0.54
	0.4	83.83 ± 0.57	93.12 ± 0.43
	0.6	83.44 ± 0.58	92.89 ± 0.50
	0.8	83.01 ± 0.60	91.94 ± 0.49

summarizes the results of the few-shot learning classification. The choice of ${\mathsf{\beta }}_1$ has a certain impact on the classification results. Taking the Conv-4 backbone as an example, the highest classification accuracy was achieved when ${\mathsf{\beta }}_1$ was set to 0.4, with the 5-way 1-shot experimental result being 73.76%. As ${\mathsf{\beta }}_1$ increases, the experimental results gradually deteriorate, which may be due to overfitting. The backbone ResNet-12 also shows the same trend, achieving the best performance when ${\mathsf{\beta }}_1$ is 0.4, followed by a decline in classification accuracy as ${\mathsf{\beta }}_1$ increases. In this study, ${\mathsf{\beta }}_1$ is set to 0.4 by default, and ${\mathsf{\beta }}_2$ is set to 0.6.

5. Conclusions

This research presents a two-stage contrastive learning model (2-SCL) for abnormal substation scene identification. We constructed a dataset covering diverse abnormal conditions to address the scarcity of abnormal scene data in substations. 5-way 1-shot and 5-way 5-shot meta-learning experiments were conducted on this dataset, and the model performance was evaluated using metrics based on the confusion matrix. Experimental results show that the proposed 2-SCL method is superior. When using Conv-4 and ResNet-12 as the backbone network, 2-SCL improves accuracy over RelationNet by 12% and 10.57% in 5-way 1-shot and 11.17% and 7.19% in 5-way 5-shot. Even when compared with State-of-the-Art algorithms, 2-SCL exhibits better classification performance across all evaluation metrics. Ablation experiments confirm the importance of both pre-training and fine-tuning stages. The experimental results show that contrastive pre-training improves recognition performance on the test dataset, and using both contrastive loss and classification loss in training enables the model to learn a broader range of feature relationships, thus enhancing its generalization ability. This research advances State-of-the-Art substation abnormal detection and provides a feasible solution for the automated detection of substation abnormal scenes. Enhancing the accuracy and reliability of abnormal scene recognition contributes to the overall safety and stability of substation operations.