A Gated Neighbor Fusion-Based Knowledge Graph for Relay Protection Operation and Maintenance

Abstract

Relay protection operation and maintenance (O&M) involves multi-source documents such as device manuals, O&M records, and industry regulations, with dense professional terminology and dispersed knowledge, making traditional retrieval methods inadequate for precise association. This paper proposes a knowledge graph construction and application method for relay protection O&M that incorporates a neighborhood gating mechanism. First, a domain schema is established covering five core entity types, primary equipment, protection functions, functional components, setting parameters, and secondary circuit signals, along with the semantic relationships between entities. Second, a MacBERT-GNF-CRF entity recognition model is proposed, which introduces a gated neighborhood feature fusion (GNF) mechanism on the basis of pre-trained semantic encoding to enhance local semantic representations near entity boundaries; experimental results show that this model achieves an F₁ score of 0.946. A MacBERT-GNF-RC relation extraction model is constructed to realize automatic discrimination of semantic associations between entity pairs. After entity disambiguation and knowledge fusion, knowledge graph storage and visualization are completed based on Neo4j. Furthermore, a knowledge graph-enhanced alarm question-answering prototype is developed and evaluated using a representative 220 kV composite alarm case and 50 historical alarm cases. Expert evaluation shows that KG augmentation significantly improves alarm interpretation quality and evidence traceability compared to direct LLM answering.

1. Introduction

The relay protection system is a critical defense line ensuring the safe and stable operation of power systems [1,2]. With the increasing integration of renewable energy sources, distributed generators, and power electronic equipment, the characteristics of grid faults, short-circuit current levels, sequence component features, and protection operating conditions have all undergone significant changes. Traditional relay protection faces new challenges in sensitivity, selectivity, reliability, and coordination settings [3,4]. Meanwhile, digital protection devices, protection information systems, and substation monitoring systems continuously generate large volumes of SOE (Sequence of Events) alarms, action reports, device records, and O&M texts during operation, which are closely related to protection function numbers, communication protocols, and substation automation standards. Therefore, relay protection O&M is no longer merely the judgment of a single device action result, but rather a comprehensive analysis process involving multi-source data, multiple device types, various protection functions, and complex secondary circuit logic.

Relay protection O&M knowledge is typically dispersed across device manuals, operating regulations, alarm texts, and historical disposal records. Such texts are characterized by dense professional terminology, numerous abbreviations, and nested action criteria with complex logic. When on-site O&M personnel analyze complex alarms, they often need to search back and forth among multiple documents, manually sorting out the logical relationships between alarm signals, protection functions, functional components, setting parameters, and primary equipment. This process is not only time-consuming but also highly dependent on personal experience, making it difficult to ensure consistency and traceability of analysis results in complex scenarios.

Knowledge graphs can organize domain knowledge in the form of “entity–relationship–entity” triples and support semantic retrieval, association analysis, and explainable reasoning through explicit relational expressions [5,6]. In recent years, artificial intelligence, graph learning, and knowledge modeling methods have received widespread attention in the field of power system fault diagnosis and intelligent O&M. For example, research on distribution network fault diagnosis has shown that artificial intelligence and signal processing methods have become important technical directions for smart grid fault analysis [7]; GraphSAGE-based power system fault diagnosis and localization methods demonstrate that graph structure modeling can effectively express power grid topology and fault association information [8]. Furthermore, knowledge graph-enhanced methods for power secondary system operation risk assessment show that structured domain knowledge helps improve risk identification and semantic understanding capabilities [9]; research on named entity recognition for power equipment maintenance work orders also indicates that entity extraction from professional O&M texts is an important foundation for equipment knowledge organization and intelligent retrieval [10].

In the field of natural language processing, pre-trained language models and sequence labeling models provide effective methods for professional text knowledge extraction. BERT, MacBERT, and other pre-trained models can enhance contextual semantic representations [11,12], while BiLSTM-CRF and its improved models can combine contextual features and label transition constraints to improve entity boundary recognition [13,14]. For relation extraction, methods such as convolutional neural networks, attention mechanisms, and bidirectional recurrent neural networks have been widely used for semantic relationship discrimination between entity pairs [15,16]. However, in relay protection texts, entity boundaries are often determined by adjacent terms collectively. For example, continuous professional phrases such as “PT disconnection blocking logic,” “zero-sequence overcurrent protection startup,” and “distance protection blocking signal” have entity boundaries that depend on local neighborhood features near the boundaries, which may still be overlooked when relying solely on global contextual representations.

With the advancement of large language models and retrieval-enhanced generation techniques, integrating external knowledge into the Q&A generation process has become a key approach to enhancing the reliability of specialized queries. Traditional RAG systems improve Q&A performance by retrieving text fragments [17]; recent studies have combined knowledge graphs with RAG, demonstrating how structured evidence enhances retrieval and reasoning in manufacturing document Q&A [18], CNC fault diagnosis [19], and the general GraphRAG framework [20], providing valuable insights for interpreting relay protection alarms.

Although previous research has made progress in knowledge modeling and text extraction for power equipment operation and maintenance, significant shortcomings remain in relay protection applications: entity names are often lengthy with boundary dependencies that rely on local context, and existing models inadequately utilize neighborhood information; current knowledge graphs struggle to simultaneously represent engineering semantic relationships among secondary circuit signals, protection functions, functional components, set parameters, and primary equipment; existing validation efforts primarily focus on visual queries, lacking an alarm interpretation process that integrates knowledge graph triples with large language model (LLM) question-answering.

To address these limitations, this paper proposes a knowledge graph construction and application framework for relay protection O&M based on multi-source technical documents and operational alarm texts. At the knowledge extraction stage, a gated neighbor fusion (GNF) mechanism is introduced to adaptively integrate global contextual representations with adjacent semantic information without relying on external lexical resources. The mechanism is applied to entity recognition and further extended to candidate entity–pair relation classification. The main contributions of this paper are as follows:

(1) A domain schema layer oriented toward relay protection O&M is constructed, defining five core entity types—primary equipment, protection functions, functional components, setting parameters, and secondary circuit signals—along with four semantic relationships: triggers, contains, depends on, and linked to. This schema provides unified constraints for subsequent knowledge extraction, triple generation, and graph queries.

(2) A MacBERT-GNF-CRF entity recognition model is proposed, in which a learnable gate adaptively balances global contextual semantics and adjacent boundary information without requiring external lexical resources. The same GNF mechanism is further extended to candidate entity–pair relation classification.

(3) A MacBERT-GNF-RC relation extraction model is constructed to distinguish four domain relations and No Relation between candidate entity pairs. After filtering unrelated pairs, the extracted triples are normalized and imported into Neo4j for structured storage and graph-based retrieval.

(4) A knowledge graph-enhanced relay protection alarm Q&A prototype is constructed and validated using a 220 kV line protection composite alarm as an example, analyzing the role of graph triple evidence in alarm cause explanation and inspection suggestion generation.

2. Construction Method of the Relay Protection O&M Knowledge Graph

2.1. System Framework and Construction Process

Relay protection operation and maintenance knowledge is dispersed across various documents such as equipment manuals, protection system records, alarm logs, and maintenance logs. To achieve structured organization of this multi-source knowledge, this paper proposes a knowledge graph construction framework comprising a data layer, a knowledge extraction layer, and a knowledge application layer, as illustrated in Figure 1.

At the data layer, relay protection-related documents and alarm records are preprocessed to generate standardized text. In the knowledge extraction layer, the MacBERT-GNF-CRF model is employed for entity recognition, while a relation extraction model identifies semantic relationships between entities, enabling entity fusion and triple construction. At the knowledge application layer, triples are imported into the Neo4j graph database to support alarm correlation analysis and large language model-enhanced question-answering.

2.2. Domain Schema Layer Construction

To constrain the knowledge extraction and graph construction process, this paper defines five core entity types based on relay protection manuals, technical regulations, and on-site alarm records: Primary Equipment, Protection Function, Functional Component, Setting Parameter, and Secondary Circuit Signal. Specifically, Primary Equipment refers to the protected object and its associated bay or circuit; Protection Function corresponds to the protection principle or functional module inside the device; Functional Component represents the key discrimination unit within a protection function; Setting Parameter covers action thresholds, time delays, enabling conditions, and other setting values; Secondary Circuit Signal denotes alarm or event information from the monitoring system, including signal name, action/reset status, level, and timestamp.

Regarding relationship types, four semantic relationships are defined: triggers, contains, depends on, and linked to. The triggers relationship describes how a secondary circuit signal activates a related protection function; contains describes the composition between a protection function and its functional components; depends on describes the dependency of a protection function on setting parameters or blocking logic; linked to describes general semantic associations, such as the correspondence between a protection function and primary equipment, and the association between alarm signals and related protection functions. Figure 2 illustrates the domain schema layer of the relay protection O&M knowledge graph.

3. Knowledge Extraction and Fusion Method

Relay protection documentation is characterized by a high concentration of technical terminology, numerous abbreviations, and complex composite naming conventions, with entity boundaries often determined by local collocation relationships between adjacent terms. Therefore, this section designs the knowledge extraction method around two requirements: first, improving boundary-aware entity representation for NER; second, converting recognized entities into reliable entity–pair relations for triple construction. Specifically, MacBERT is used to capture global contextual semantics, the gated neighbor fusion (GNF) layer is introduced to strengthen local boundary features, and the CRF layer is used to enforce global label-sequence consistency. On this basis, a relation extraction module is further constructed to generate triples for subsequent knowledge fusion and graph construction.

3.1. Named Entity Recognition Model

Named Entity Recognition (NER) is a key technology for constructing high-quality domain knowledge graphs and realizing knowledge extraction in relay protection [21]. The proposed NER model adopts a hierarchical structure of MacBERT encoding, neighborhood feature fusion, and label decoding. It consists of a MacBERT pre-trained encoding layer, a gated neighbor feature fusion layer, a linear mapping layer, and a CRF decoding layer. The overall structure is shown in Figure 3.

The processing flow is as follows. The input text is encoded by MacBERT to obtain contextual semantic representations at each position. For each position, left and right neighborhood information is introduced and fused through a gating mechanism. The fused features are mapped by a linear layer to obtain tag emission scores. Finally, the CRF layer uses label transition constraints to decode the globally optimal label sequence. Different from a conventional CRF model that relies only on label transition probabilities, the proposed model explicitly injects neighborhood contextual information before CRF decoding, so that entity boundary features are strengthened at the representation stage.

3.2. MacBERT Pre-Trained Encoding Layer

MacBERT is a pre-trained language model specifically optimized for Chinese corpora. It addresses the inconsistency between pre-training and fine-tuning phases through strategies such as full-word masking, and N-gram masking, while enhancing the semantic representation capability of multi-character Chinese words [12].

In this paper, MacBERT is used as the basic encoder. Let the input text sequence be:

$$X=\{x_1,x_2,\dots ,x_n\}$$

(1)

where $x_t$ denotes the input token at the t-th position and $n$ is the sequence length. After MacBERT encoding, the contextual representation corresponding to each position is obtained as follows:

$$h_t=MacBERT(X{)}_t, t=1,2,\dots ,n$$

(2)

where $h_t\in R^d$ and $d$ is the hidden representation dimension. MacBERT is particularly suitable for Chinese professional texts because its masking strategy and perturbation mechanism reduce the mismatch between pre-training and downstream sequence labeling tasks.

3.3. Gated Neighbor Feature Fusion Layer

A relay protection entity typically consists of multiple consecutive terms, and its boundary identification relies on the local contextual relationships between the current position and its immediate neighbors. To enhance the local semantic representation near boundaries, this paper introduces a neighborhood-gated feature fusion layer following the MacBERT encoding layer.

Compared to existing local-feature enhancement approaches, the proposed GNF mechanism has two characteristics. First, conventional CNN-based local encoders typically aggregate neighboring features using shared convolution kernels, whereas GNF employs a learnable gate at each token position to adaptively balance the global contextual representation and the neighborhood-fused representation. This enables tokens inside long professional terms and those near entity boundaries to receive different degrees of local enhancement. Second, unlike lexicon-augmented Chinese NER methods that depend on external word dictionaries, GNF requires no additional lexical resources, making it suitable for relay protection texts containing numerous manufacturer-specific terms and abbreviations.

In the following equations, $h_t\in {\mathbb{R}}^d$ denotes the global contextual representation of the $t$-th token generated by the MacBERT encoder, where $d$ is the hidden representation dimension. Accordingly, $h_{t-1}$ and $h_{t+1}$ denote the contextual representations of the immediate left and right neighboring tokens, respectively. The local neighborhood vector $c_t$ is constructed by concatenating $h_{t-1}$, $h_t$ and $h_{t+1}$. The candidate representation ${\tilde{h}}_t$ denotes the neighborhood-enhanced representation obtained through nonlinear transformation, while $g_t\in (0,1{)}^d$ denotes the learnable gate vector that adaptively controls the element-wise contribution of ${\tilde{h}}_t$ and the original contextual representation $h_t$. The resulting fused representation is denoted by $z_t$.

For the t-th position in the sequence, its left-neighborhood representation $h_{t-1}$, current representation $h_t$, and right-neighborhood representation $h_{t+1}$ are considered simultaneously, and the local neighborhood representation is constructed as:

$$c_t=[h_{t-1};h_t;h_{t+1}]$$

(3)

where $[;]$ denotes vector concatenation. For the first and last positions of the sequence, missing neighborhood vectors are padded with zero vectors.

On this basis, a candidate fused representation is first generated through nonlinear mapping:

$${\tilde{h}}_t=\mathit{tanh}(W_c{c}_t+b_c)$$

(4)

where $W_c$ and $b_c$ are trainable parameters. A gate vector is then introduced to control the weight allocation between the original representation of the current position and the neighborhood-fused representation:

$$g_t=\sigma (W_g{c}_t+b_g)$$

(5)

where $\sigma (\cdot )$ denotes the sigmoid function, and $W_g$ and $b_g$ are gate parameters. The enhanced local representation is finally defined as

$$z_t=g_t\odot {\tilde{h}}_t+(1-g_t)\odot h_t$$

(6)

where $\odot$ denotes element-wise multiplication.

To improve training stability and reduce the influence of representation distribution differences across positions, the fused representation is further normalized:

$${\widehat{z}}_t=LayerNorm\left(z_t\right)$$

(7)

After gated neighbor fusion, local structural information is directly encoded into the representation of each token without significantly increasing model complexity, which benefits subsequent tag prediction. The structure of the gated neighbor feature fusion mechanism is shown in Figure 4.

3.4. Linear Mapping Layer

The enhanced representation from neighborhood gated feature fusion already contains the current position semantics and local neighborhood structural information. To map this feature to the label space, this paper uses a linear layer to generate the label emission score at each position:

$$o_t=W_o{\widehat{z}}_t+b_o$$

(8)

where $W_o$ and $b_o$ are the linear mapping layer parameters, and $o_t$ denotes the emission score of each label corresponding to the t-th position, to be used by the subsequent CRF layer for decoding.

3.5. CRF Decoding Layer

Entity recognition requires not only correct classification at each position but also overall legality of the label sequence. For example, in BIO annotation, “I-Protection Function” cannot directly follow “B-Setting Parameter”. Therefore, this paper adds a Conditional Random Field (CRF) at the output layer to jointly constrain label transitions. Let the input sequence be $X$ and the corresponding label sequence be $Y=\{y_1,y_2,\dots ,y_n\}$; then the total score of this label sequence can be expressed as:

$$s(X,Y)={\displaystyle \sum _{t=1}^n}{o}_{t{,y}_t}+{\displaystyle \sum _{t=1}^{n-1}}{A}_{y_t,y_{t+1}}$$

(9)

where ${ o}_{t,y_t}$ denotes the emission score when the t-th position selects label $y_t$, and $A$ is the label transition matrix, describing the transition tendency between adjacent labels. The conditional probability is further defined as:

$$P(Y|X)={\displaystyle \frac{\exp (s(X,Y))}{{\sum }_{Y^{\prime }\in { \mathcal{Y}}_X}\exp (s(X,Y^{\prime }))}}$$

(10)

where ${\mathcal{Y}}_X$ denotes the set of all possible legal label sequences for the input sequence $X$, and $Y^{\prime }\in {\mathcal{Y}}_X$ denotes an arbitrary candidate label sequence. During training, the negative log-likelihood is used as the optimization objective. During prediction, the Viterbi algorithm is used to search for the label path with the highest score:

$$\widehat{Y}=\mathit{arg}\underset{Y^{\prime }\in {\mathcal{Y}}_X}{\mathit{max}}s(X,Y^{\prime })$$

(11)

Here, $\widehat{Y}$ denotes the optimal label sequence predicted by the model; and $s(X,Y^{\prime })$ is the global sequence score computed by the CRF layer. By using Viterbi decoding, the model jointly considers label emission scores and transition constraints between adjacent labels, thereby reducing boundary errors near entities.

3.6. Relation Extraction Model

On the basis of the NER model, the CRF sequence decoding layer is replaced with a relation classification module to construct an entity–pair-oriented relation extraction model. The model also uses MacBERT to encode the text context and applies the GNF mechanism to strengthen local semantic features near entities. Following entity recognition, entity pairs co-occurring within the same sentence are generated as candidate pairs. The MacBERT-GNF-RC model classifies each candidate pair into one of five labels: triggers, contains, depends on, linked to, or No Relation. The model therefore jointly determines whether a semantic relation exists between two entities and identifies the corresponding relation type when a relation is present.

Let the head entity and tail entity in a sentence be eh and et, respectively. First, the GNF-enhanced representations of the tokens corresponding to the two entities are average-pooled to obtain the head entity representation ${ h}_h$ and the tail entity representation $h_t$. The two entity representations are then concatenated to construct the joint feature vector of the entity pair:

$$v_{pair}=[h_h;h_t]$$

(12)

where $[;]$ denotes vector concatenation and $v_{pair}$ denotes the joint feature vector of the entity pair. The vector is then input to a fully connected classification layer to obtain the probability distribution over candidate relation classes:

$$p=Softmax(W_r{v}_{pair}+b_r)$$

(13)

where $W_r$ and $b_r$ are the weight matrix and bias of the relation classification layer, respectively, and $p$ denotes the probability distribution over the predefined relation categories. Specifically, $p_r$ denotes the predicted probability that the candidate entity pair $(e_h,e_t)$ belongs to relation category $r,$ and $\mathcal{R}$ denotes the predefined set of relation categories. The final predicted relation is selected as the class with the maximum probability:

$$\widehat{r}=\underset{r\in R}{argmax }{p}_r$$

(14)

Figure 5 shows the architecture of the relation extraction model. The model reuses the MacBERT encoder and GNF layer from the NER stage, so that relation classification can exploit both global contextual semantics and local features near entity boundaries. The enhanced head and tail entity representations are concatenated and input into a Softmax classifier to predict the relation label. Entity pairs predicted as No Relation are discarded, while the remaining predictions are organized as candidate triples for manual verification, knowledge fusion, and graph construction.

3.7. Knowledge Fusion and Entity Disambiguation

To minimize node redundancy caused by synonymous expressions and variations in manufacturer naming, entity names must be standardized before being stored as triples. This paper establishes a specialized terminology dictionary containing 3718 standardized relay-protection terms and an abbreviation/alias mapping table containing 974 mappings, together with entity type constraint rules based on terminology conventions in the relay protection domain. These resources provide unified mappings for synonymous, abbreviated, and manufacturer-specific expressions, such as “CT secondary circuit” and “current transformer secondary circuit”. For entities not covered by these rules, manual verification is performed considering the entity context and engineering semantics; if consistent with existing entity definitions, they are standardized into corresponding standard entities; otherwise, they are added as new entity nodes to the knowledge graph. After these steps, the processed triples are imported into the Neo4j graph database.

4. Experimental Analysis and Results

This section presents experimental validation of the proposed knowledge extraction models. Section 4.1 introduces the data sources, preprocessing pipeline, and annotation specifications. Section 4.2 evaluates the named entity recognition model performance through baseline comparisons and ablation studies. Section 4.3 further validates the effectiveness of the relation extraction model on entity–pair relation classification tasks.

4.1. Data Sources and Processing

The experimental corpus was constructed from a combination of normative technical documents and actual operation and maintenance texts in the relay protection domain. The normative sources mainly include relay protection device manuals, relay protection setting regulations, and power system operation evaluation specifications, whereas the operational sources include actual alarm records collected from relay protection O&M scenarios. Therefore, the corpus contains both standardized technical descriptions and real-world operational texts rather than relying on a single type of data source.

To improve data quality, the original texts are first uniformly preprocessed, including invalid character removal, duplicate content elimination, and correction of garbled tables and formulas. On this basis, long texts are segmented to meet the input length requirements of the pre-trained model. Subsequently, according to the entity type system constructed in this paper, the texts are manually annotated using the BIO annotation method to complete entity boundary and category identification. Some annotation examples are shown in

Table 1. Examples of entity annotation.

Sample Text	Label
bus differential protection	B-Protection Function I-Protection Function I-Protection Function
GIS busbar	B-Primary Equipment I-Primary Equipment
current threshold setting	B-Setting Parameter I-Setting Parameter I-Setting Parameter

.

A total of 5820 annotated samples containing 84,098 token-level labels were constructed, including 46,254 entity-labeled tokens and 37,844 O-labeled tokens. The samples were divided into training, validation, and test sets at an 8:1:1 ratio.

Table 2. Statistics of annotated entity types.

Label Category	Training	Validation	Test	Total
Primary Equipment	5545	693	693	6931
Protection Function	11,106	1388	1388	13,882
Functional Component	7402	925	925	9252
Setting Parameter	6661	833	832	8326
Secondary Circuit Signal	6290	786	787	7863
Outside (O)	30,276	3784	3784	37,844
Total	67,280	8409	8409	84,098

presents the label distribution, with B- and I-tags merged according to their corresponding entity categories.

Based on the annotated entities, entity pairs co-occurring within the same sentence were generated as relation candidates. Positive pairs were assigned one of four domain relation labels—triggers, contains, depends on, or linked to—while 3206 manually verified pairs without an explicit semantic relation were labeled as No Relation. The resulting dataset contained 9618 instances and was divided into 7694 training, 962 validation, and 962 test instances, as shown in

Table 3. Distribution of relation classification instances in the dataset.

Relation Label	Training	Validation	Test	Total
triggers	924	115	115	1154
contains	1283	160	160	1603
depends on	1128	141	141	1410
linked to	1795	225	225	2245
No Relation	2564	321	321	3206
Total	7694	962	962	9618

.

The No Relation class accounts for approximately 33.3% of the relation classification dataset (3206 of 9618 instances). Although it is the largest individual class, the four positive relation categories collectively account for approximately 66.7% of the dataset. The No Relation instances were retained to provide sufficient negative entity–pair examples for learning whether an explicit semantic relation exists, while avoiding an excessively negative candidate distribution. For relation classification, macro-averaged Precision, Recall, and $F_1$ score are reported to reduce the influence of class-frequency differences on the overall evaluation.

Precision ($P$), Recall ($R$), and $F_1$ score are used as evaluation metrics. Precision measures the proportion of correctly identified entities in the recognition results, recall measures the proportion of actual entities that are correctly identified, and the $F_1$ score is the harmonic mean of the two:

$$P={\displaystyle \frac{T_P}{T_P+F_P}}$$

(15)

$$R={\displaystyle \frac{T_P}{T_P+F_N}}$$

(16)

$$F_1={\displaystyle \frac{2PR}{P+R}}$$

(17)

where $T_P$, $F_P$, and $F_N$ denote the number of correctly identified entities, incorrectly identified entities, and missed entities in the confusion matrix, respectively. Entity matching adopts the strict matching criterion, i.e., it is counted as correct only when both the entity boundary and the type are correct.

4.2. Entity Extraction Experiment

To verify the effectiveness of the proposed entity extraction model, the experiments are completed in a computing environment equipped with an Intel Core i7 processor(Intel Corporation, Santa Clara, CA, USA), an NVIDIA GeForce RTX 3060 graphics card (NVIDIA Corporation, Santa Clara, CA, USA), and 32 GB of memory. The model is implemented based on Python 3.10, the deep learning framework adopts PyTorch 2.2.2, the CUDA version is 11.8, and the relevant training parameter settings are shown in

Table 4. Training parameters of the MacBERT-GNF-CRF model.

Training Parameter	Value
Epoch	20
Batch size	8
Learning rate	2 × 10−5
Dropout rate	0.1
Max length	512
Weight decay	0.01
Warmup ratio	0.1

. The model optimization adopts the AdamW optimizer, and a learning rate warmup mechanism is set to improve training stability.

To verify the effectiveness of the proposed entity recognition model, five models were compared: (1) BiLSTM-CRF; (2) MacBERT-CRF, which directly applies CRF decoding to MacBERT representations; (3) MacBERT-BiLSTM-CRF, which introduces a BiLSTM layer between MacBERT and CRF; (4) MacBERT-GlobalPointer, a span-based model that jointly identifies the start and end positions of entities [22]; and (5) the proposed MacBERT-GNF-CRF.

The experimental results are shown in

Table 5. Experimental results of different entity recognition models over five independent runs.

Model	Precision	Recall	${\mathit{F}}_1$-Score
BiLSTM-CRF	0.863 ± 0.004	0.874 ± 0.005	0.868 ± 0.004
MacBERT-CRF	0.914 ± 0.002	0.919 ± 0.003	0.916 ± 0.002
MacBERT-BiLSTM-CRF	0.923 ± 0.002	0.928 ± 0.003	0.925 ± 0.002
MacBERT-GlobalPointer	0.934 ± 0.001	0.927 ± 0.002	0.931 ± 0.001
MacBERT-GNF-CRF	0.944 ± 0.002	0.948 ± 0.002	0.946 ± 0.002

. The proposed model achieves an $F_1$-score of 0.946, improving the $F_1$-score by 7.8, 3.0, 2.1 and 1.5 percentage points compared to BiLSTM-CRF, MacBERT-CRF, MacBERT-BiLSTM-CRF, and MacBERT-GlobalPointer, respectively. These results indicate that the proposed model can effectively improve entity recognition performance for relay protection texts. Figure 6 shows the $F_1$-score curve of the entity recognition models on the validation set.

To further examine class-specific recognition errors, Figure 7 presents the entity-level confusion matrix of MacBERT-GNF-CRF on the test set for a representative run. Strict entity-level matching is adopted, whereby an entity is considered correct only when both its boundary and type are correctly predicted. In the matrix, “None” denotes unmatched entities rather than an additional entity type: the None column represents missed ground-truth entities, whereas the None row represents spurious predictions.

As shown in Figure 7, the representative run contains 1163 ground-truth entity mentions and 1168 predicted entity mentions, of which 1101 are correctly matched in both boundary and type. This corresponds to a precision of 0.943, a recall of 0.947, and an F1-score of 0.945, which are close to the five-run averages reported in Table 5. Most predictions are concentrated on the diagonal, indicating limited systematic confusion among the five entity types. The most noticeable type-level confusion occurs between Protection Function and Functional Component, with 7 Protection Function entities misclassified as Functional Component and 8 errors in the opposite direction. Unmatched predictions are also observed for several categories, reflecting the remaining difficulty in identifying entity boundaries and distinguishing semantically adjacent professional terms.

To further verify the contribution of each module in MacBERT-GNF-CRF, an ablation study was conducted. MacBERT-CRF removes the GNF module and uses only MacBERT and CRF for sequence labeling. MacBERT-Concat-CRF directly concatenates the current representation with left and right neighborhood representations without the gating mechanism. MacBERT-GNF-Softmax retains the GNF module but replaces CRF decoding with Softmax classification. MacBERT-GNF-CRF w/o LN removes the normalization operation from the GNF module. MacBERT-GNF-CRF is the complete model. Each ablation variant was independently trained three times, and the mean and standard deviation are reported.

Table 6. Ablation study of the MacBERT-GNF-CRF model for entity recognition.

Model	Precision	Recall	${\mathit{F}}_1$-Score
MacBERT-CRF	0.914 ± 0.005	0.919 ± 0.004	0.916 ± 0.004
MacBERT-Concat-CRF	0.934 ± 0.003	0.938 ± 0.004	0.936 ± 0.003
MacBERT-GNF-Softmax	0.937 ± 0.002	0.927 ± 0.002	0.932 ± 0.002
MacBERT-GNF-CRF w/o LN	0.939 ± 0.004	0.942 ± 0.003	0.940 ± 0.003
MacBERT-GNF-CRF	0.944 ± 0.001	0.948 ± 0.002	0.946 ± 0.001

presents the results of ablation experiments for the entity recognition model. MacBERT-Concat-CRF achieved a higher $F_1$ score than MacBERT-CRF, indicating that neighborhood information aids in entity boundary detection; MacBERT-GNF-Softmax performed worse than the full model, demonstrating that CRF decoding remains essential for maintaining label sequence consistency; performance declined after removing LayerNorm, confirming that normalization helps stabilize feature fusion. The full model attained the highest $F_1$ score of 0.946, validating the effectiveness of neighborhood information, gated fusion, CRF decoding, and normalization.

4.3. Relation Extraction Experiment

To verify the performance of the relation extraction model, three models are set up: (1) BiLSTM-Attention; (2) MacBERT-CLS (directly using the [CLS] vector for classification); (3) the proposed MacBERT-GNF-RC model, which introduces gated neighbor fusion and entity–pair representation for relation classification. The training parameters are shown in

Table 7. Training parameters of the relation extraction model.

Parameter	Value
Epoch	20
Batch size	32
Learning rate	3 × 10−5
Max length	128
Dropout rate	0.1
Class num	5

.

The experimental results are shown in

Table 8. Experimental results of the relation classification models over five independent runs.

Model	Precision	Recall	${\mathit{F}}_1$-Score
BiLSTM-Attention	0.872 ± 0.005	0.881 ± 0.005	0.876 ± 0.004
MacBERT-CLS	0.912 ± 0.004	0.915 ± 0.003	0.913 ± 0.003
MacBERT-GNF-RC	0.928 ± 0.002	0.931 ± 0.003	0.929 ± 0.002

. The $F_1$-score of the proposed model reaches 0.929, which is 5.3 percentage points higher than that of BiLSTM-Attention and 1.6 percentage points higher than that of MacBERT-CLS. This result indicates that transferring the GNF mechanism from the NER stage to the relation extraction task can effectively enhance local semantic aggregation around entity pairs.

Figure 8 shows the $F_1$ score variation curve of the relation extraction model on the validation set, demonstrating that our model achieves a high $F_1$ score during training.

To further investigate class-specific relation prediction errors, Figure 9 presents the confusion matrix of MacBERT-GNF-RC on the 962-instance test set for a representative run.

As shown in Figure 9, most predictions are concentrated along the diagonal, indicating good discrimination among the predefined relation labels. For this representative run, the macro-averaged precision, recall, and F1-score are 0.929, 0.932, and 0.930, respectively, which are close to the five-run averages reported in Table 8. The largest off-diagonal error occurs for No Relation instances predicted as linked to, with 16 such cases. In addition, 8 No Relation instances are classified as depends on, while 8 contains instances are predicted as No Relation. The relatively frequent confusion between linked to and No Relation is mainly attributable to the broader semantics of linked to, which represents general semantic associations and can therefore be more difficult to distinguish from weak or implicit associations between entity pairs. Overall, the remaining errors are concentrated primarily in semantically ambiguous relation boundaries rather than widespread confusion among explicit relation types.

To further analyze the influence of entity representation methods and the GNF mechanism on relation extraction performance, this paper sets up three model variants for comparison. MacBERT-CLS directly uses the [CLS] vector output by MacBERT for relation classification without explicitly introducing entity position information. MacBERT-Concat-RC average-pools the MacBERT representations at the head and tail entity positions and directly concatenates the two entity representations for classification without using the GNF mechanism. MacBERT-GNF-RC is the complete relation extraction model, which introduces the neighborhood gated feature fusion mechanism before entity representation generation and performs relation classification after concatenating the enhanced head and tail entity representations. Each relation classification variant was independently trained three times, and the results are reported as mean ± standard deviation.

Table 9. Ablation study of the MacBERT-GNF-RC model for relation extraction.

Model	Precision	Recall	${\mathit{F}}_1$-Score
MacBERT-CLS	0.912 ± 0.002	0.915 ± 0.001	0.913 ± 0.001
MacBERT-Concat-RC	0.918 ± 0.003	0.924 ± 0.002	0.921 ± 0.003
MacBERT-GNF-RC	0.928 ± 0.002	0.931 ± 0.002	0.929 ± 0.002

presents the results of ablation experiments for the relation extraction model. MacBERT-Concat-RC achieved a higher $F_1$ score than MacBERT-CLS, demonstrating that explicit combination of head and tail entity representations enhances relation discrimination. With the introduction of the GNF mechanism, MacBERT-GNF-RC’s $F_1$ score further improved to 0.929, confirming the effectiveness of integrating entity representations with neighborhood-gated features in relation extraction tasks.

5. Knowledge Graph Construction and Application

The preceding section verifies the effectiveness of the proposed entity and relation extraction models. On this basis, this section organizes the extracted entities and relations into standardized triples, imports them into Neo4j, and further applies the constructed knowledge graph to KG-augmented alarm interpretation.

5.1. Graph Construction

After model evaluation, the trained MacBERT-GNF-CRF model was applied to the relay protection corpus to identify entity mentions. Entity pairs co-occurring within the same sentence were subsequently classified by MacBERT-GNF-RC. Pairs predicted as No Relation were discarded, while the remaining candidate triples were manually reviewed to correct erroneous entity or relation labels. After terminology normalization and entity fusion, 5642 validated entity instances and 5128 relation triples were retained and imported into Neo4j using Cypher statements.

5.2. Knowledge Graph-Augmented Alarm Question-Answering Prototype

After completing the knowledge graph construction, this paper developed an enhanced alarm Q&A prototype based on the knowledge graph. The prototype utilizes the relay protection knowledge graph in Neo4j as a structured knowledge source. Upon receiving an SOE alarm record or a natural language query, the system retrieves relevant graph nodes and their adjacency relationships based on alarm signals and protection terminology, returns evidence triples related to the alarm, and feeds these into a large language model as structured context to generate potential causes and inspection recommendations.

Compared to directly calling large language models, knowledge graph-enhanced Q&A addresses queries using a model constrained by structured evidence, reducing the risk of unfounded inferences and providing operations personnel with traceable relationship paths. The workflow of the KG-augmented alarm Q&A prototype is summarized in Figure 10.

5.3. Typical Alarm Case and Comparative Analysis

A 220 kV line protection alarm event was selected as a representative case study. This incident involved multiple SOE signals—including “abnormal secondary voltage on PT,” “distance protection lockout signal,” “zero-sequence overcurrent protection activation,” and “line protection device alarm”—and encompassed information from various systems such as the secondary voltage circuit, distance protection lockout logic, and zero-sequence overcurrent protection mechanisms, making it well-suited for multi-signal correlation analysis.

From the perspective of relay protection principles, an abnormal PT secondary voltage indicates that the voltage measurement supplied to the protection device may be unreliable. When voltage-circuit supervision or PT-disconnection logic detects such an abnormality, the distance protection function may be blocked to avoid incorrect operation caused by unreliable impedance calculation. Meanwhile, the startup of zero-sequence overcurrent protection indicates the presence of a zero-sequence current component and can serve as evidence of a possible ground fault; however, this signal alone is insufficient to confirm a primary-system fault. Therefore, the main suspected causes in this composite alarm include PT secondary-circuit disconnection or abnormal voltage sampling, while the possibility of a primary ground fault should be further verified using fault waveforms and zero-sequence current records.

To ensure comparability, this study employs the DeepSeek-V3.2 model via the DeepSeek API, implementing two input approaches: direct question-answering and knowledge graph-enhanced question-answering. The former uses only alarm text as input, while the latter incorporates graph triples retrieved from Neo4j alongside the same alarm text. Both methods employ identical models, question formulations, and default API configurations, differing solely in whether graph evidence is provided. Both approaches require the model to identify the most probable cause and prioritize inspection items. This comparison represents a case study rather than a large-scale performance evaluation of question-answering systems, primarily aimed at assessing how graph triples enhance traceability in alarm explanations.

Table 10. Retrieved knowledge graph evidence for the typical alarm case.

No.	Retrieved KG Triple	Role in Reasoning
1	PT Secondary Voltage Abnormality—triggers—Voltage Circuit Supervision	Identifies the voltage circuit abnormality as the main entry point of the event.
2	Line Protection Device Alarm—linked to—Voltage Circuit Supervision	Indicates that the device alarm is associated with voltage circuit supervision.
3	Distance Protection Blocking Signal—linked to—Distance Protection	Links the blocking signal to the affected protection function.
4	Distance Protection—contains—PT Disconnection Blocking Element	Explains why PT abnormality may lead to distance protection blocking.
5	Distance Protection—depends on—PT Disconnection Blocking Logic	Supports checking the blocking logic and PT secondary circuit.
6	Zero-sequence Overcurrent Startup—triggers—Zero-sequence Overcurrent Protection	Provides auxiliary evidence for checking whether a primary grounding fault exists.

presents the key KG evidence obtained from analysis in this case. The relevant triples establish connections between alarm signals and various parameters, including voltage circuit monitoring, distance protection, PT open-circuit lockout elements, lockout logic, and zero-sequence overcurrent protection. Specifically, triples related to PT secondary voltage anomalies and distance protection support troubleshooting of the secondary circuitry and lockout logic, while the activation of zero-sequence overcurrent protection serves as an auxiliary indicator for identifying primary ground faults.

Under identical alarm input conditions, when the large language model is directly invoked, it can provide reasonable diagnoses such as PT secondary circuit faults or open circuits, and recommend checking the PT secondary fuse, secondary wiring, grounding status, and distance protection PT lockout settings. However, this response primarily relies on the model’s general domain knowledge without providing clear structured evidence support, particularly lacking traceable justification for its diagnosis of primary ground faults.

In contrast, the knowledge graph-enhanced question-answering approach first returns the graph triple evidence shown in Table 10, linking information such as PT secondary voltage anomalies, line protection device alarms, distance protection lockout, and zero-sequence overcurrent protection activation. Based on these structured pieces of evidence, the model identifies the primary suspected causes as PT secondary circuit abnormalities, secondary-circuit disconnection, or voltage sampling errors, recommending priority inspection of the PT secondary circuit and distance protection lockout status; for zero-sequence overcurrent protection activation, it serves as an auxiliary diagnostic clue, suggesting that the presence of a primary ground fault should be further determined by combining fault waveforms and zero-sequence current records rather than treating it as a definitive conclusion.

This case demonstrates that KG-enhanced Q&A improves the traceability of alarm explanations by leveraging structured evidence chains, distinguishes between primary causes and supporting clues, reduces unfounded inferences, and generates inspection recommendations more aligned with actual operational workflows.

This representative 220 kV alarm event was also included as Case 09 in the 50-case expert evaluation described in Section 5.4. For this case, the average scores assigned by the three experts to the direct LLM response were 3.67, 3.00, 3.33, and 3.33 for cause correctness, evidence traceability, inspection practicality, and overall reliability, respectively. The corresponding KG-augmented response achieved average scores of 4.67, 4.33, 4.67, and 4.33, respectively. These case-level results are consistent with the overall trend observed in the 50-case evaluation and further illustrate the benefit of incorporating structured graph evidence into alarm interpretation.

5.4. Small-Scale Expert Evaluation

To provide a preliminary quantitative evaluation of the KG-augmented Q&A prototype, 50 representative composite alarm cases were selected from historical alarm records. Three experts with relay protection backgrounds independently evaluated the outputs generated by direct LLM question-answering and KG-augmented question-answering. Each answer was scored on a five-point scale in terms of cause correctness, evidence traceability, inspection practicality, and overall reliability. The average evaluation results are shown in

Table 11. Expert evaluation of direct and KG-augmented question-answering.

Evaluation Metric	Direct LLM	KG-Augmented LLM
Cause Correctness	3.78	4.31
Evidence Traceability	2.67	4.58
Inspection Practicality	3.73	4.29
Overall Reliability	3.49	4.38

.

The KG-augmented approach achieved higher scores across all four evaluation dimensions, with the most evident improvement observed in evidence traceability. For each case and evaluation metric, the ratings of the three experts were averaged, and two-sided paired $t$-tests were then conducted across the 50 case-level mean scores. The results showed that the KG-augmented approach significantly outperformed the direct LLM approach across all four evaluation metrics (all $p$ < 0.001). To further assess the consistency of the expert ratings, inter-rater reliability was evaluated using Fleiss’ kappa separately for each evaluation metric and each answering approach. For the calculation, the five score levels were treated as discrete rating categories. Let $N$ denote the number of evaluated cases, $n$ the number of experts, $k$ the number of rating categories, and $n_{ij}$ the number of experts assigning case $i$ to category $j$. In this study, $N$ = 50, $n$ = 3, and $k$ = 5.

$$P_i={\displaystyle \frac{1}{n(n-1)}}{\sum }_{j=1}^k{n}_{ij}\left(n_{ij}-1\right), \overline{P}={\displaystyle \frac{1}{N}}{\sum }_{i=1}^N{P}_i$$

(18)

$$p_j={\displaystyle \frac{1}{Nn}}{\sum }_{i=1}^N{n}_{ij}, {\overline{P}}_e={\sum }_{j=1}^k{p}_j^2$$

(19)

$$\kappa ={\displaystyle \frac{\overline{P}-{\overline{P}}_e}{1-{\overline{P}}_e}}$$

(20)

Here, $P_i$ denotes the observed agreement for case $i$, $\overline{P}$ is the mean observed agreement across all cases, $p_j$ is the overall proportion of ratings assigned to category $j$, and ${\overline{P}}_e$ is the agreement expected by chance. Fleiss’ kappa therefore normalizes the observed agreement relative to chance agreement, with $\kappa =1$ indicating perfect agreement and $\kappa =$ 0 indicating agreement at the chance level. The resulting coefficients are shown in

Table 12. Inter-rater agreement measured by Fleiss’ kappa.

Evaluation Metric	Direct LLM	KG-Augmented LLM
Cause Correctness	0.507	0.771
Evidence Traceability	0.684	0.503
Inspection Practicality	0.683	0.668
Overall Reliability	0.638	0.796

.

The Fleiss’ kappa coefficients ranged from 0.507 to 0.684 for the direct LLM approach and from 0.503 to 0.796 for the KG-augmented approach across the four evaluation dimensions, indicating generally moderate to substantial agreement among the three experts. It should be emphasized that Fleiss’ kappa measures inter-rater agreement rather than system performance; therefore, differences in kappa between the two approaches do not directly indicate differences in answer quality. Combined with the consistently higher mean scores reported in Table 11 and the statistically significant paired comparisons, these results support the reliability of the expert evaluation and indicate that structured graph evidence improves the evidential basis of generated answers, alarm interpretation, and inspection recommendations.

6. Discussion

The experimental results indicate that the GNF mechanism is effective for extracting relay protection knowledge. Compared with MacBERT-CRF and MacBERT-BiLSTM-CRF, MacBERT-GNF-CRF achieved higher and more stable recognition performance. The ablation results further show that neighborhood information, gated fusion, CRF decoding, and LayerNorm jointly contribute to entity boundary identification. This suggests that adaptively integrating adjacent semantic units is beneficial for recognizing long and compositionally complex relay protection terms.

The improvement in MacBERT-GNF-RC demonstrates that neighborhood-enhanced entity representations are also useful for relation classification. At the application level, the knowledge graph provides explicit relation paths connecting alarm signals, protection functions, functional components, and setting information. The expert evaluation shows that KG augmentation produces the greatest improvement in evidence traceability while also improving cause correctness and inspection practicality. Therefore, the main value of the knowledge graph is to provide structured evidence constraints for alarm interpretation rather than merely support graph storage and visualization.

This study still has several limitations. First, the current relation extraction strategy generates candidate entity pairs only when the entities co-occur within the same sentence. This design helps reduce noisy candidate pairs and limits the search space, but semantic relations expressed across sentence boundaries cannot be directly captured. Future work will investigate document-level relation extraction by incorporating cross-sentence contextual representations, coreference information, and graph-based context aggregation. Second, the current corpus and relation schema cover a limited range of relay protection documents and four predefined relation types. In addition, the constructed graph mainly represents textual knowledge and does not yet incorporate fault waveforms or device event sequences. Future work will therefore expand the corpus and alarm cases and construct multimodal knowledge graphs integrating textual, waveform, and event-sequence information.

7. Conclusions

This paper proposes a method for constructing and applying a knowledge graph for relay protection operation and maintenance that incorporates a neighborhood gating mechanism. A domain model covering five types of entities and four semantic relationships was established. The proposed MacBERT-GNF-CRF and MacBERT-GNF-RC models achieved $F_1$ scores of 0.946 and 0.929, respectively, and the resulting knowledge graph provides a foundation for querying and structurally retrieving relay protection operation and maintenance knowledge.

In terms of application, this paper develops a knowledge graph-enhanced alarm Q&A prototype and analyzes it using a composite alarm from a 220 kV line protection system as an example. The results demonstrate that graph triples provide a traceable, structured evidence chain for alarm explanations generated by large language models, thereby supporting cause identification and diagnostic recommendations.

Although the preliminary expert evaluation supports the effectiveness of KG augmentation, the current evaluation scale remains limited, and the generated results are intended only to assist operational analysis. Future work will expand the corpus and authentic alarm dataset, evaluate additional large language models, and integrate fault waveforms, device event records, and textual knowledge into a multimodal knowledge graph.