Abstract
Fault diagnosis in modern power systems increasingly depends on unstructured operation and maintenance (O&M) logs, yet real-world logs are often small in scale and highly imbalanced across fault types, which degrades the generalizability of standard neural models. This paper proposes a graph-augmented diagnostic framework that integrates imbalance-aware knowledge extraction with interpretable reasoning. The framework consists of three stages: (1) domain adaptation of a BERT–BiLSTM–CRF NER model and a BERT–MLP RE model using an imbalance-aware training recipe that combines Low-Rank Adaptation (LoRA), a mixed focal–range loss, and undersampling; (2) construction of a power-system knowledge graph that organizes extracted entities and relations (e.g., fault devices, abnormal phenomena, causes, and handling measures); and (3) a graph-augmented assistant agent that reuses the NER model as a graph-aware retriever within a retrieval-augmented generation (RAG) architecture to support contextualized and interpretable diagnostic reasoning. Experiments on 3921 real-world fault-processing logs show consistent gains: NER reaches 92.0% accuracy and 71.3% Macro-F1 (vs. 80.3% and 63.2%), and RE achieves 88.0% accuracy and 70.1% F1 (vs. 82.1% and 60.4%), while reducing average training time per epoch by about 18%. These results demonstrate an efficient and practical path toward robust log-based fault diagnosis under scarce and imbalanced data.
1. Introduction
The safe and stable operation of modern power systems increasingly relies on timely and accurate fault diagnosis [1]. With the rapid growth of grid scale, the diversity of equipment, and the complexity of operating conditions, traditional model-driven or rule-based approaches are finding it difficult to cope with the volume and heterogeneity of fault information generated by real-world networks. In practical settings, a large proportion of fault knowledge is buried in unstructured operation and maintenance (O&M) logs written by frontline engineers, including alarm descriptions, on-site inspections, switching operations, and corrective actions [2]. However, these logs are usually analyzed manually or via simple keyword search, which makes it hard to reuse accumulated fault experience in a systematic and data-driven way. Recent surveys on knowledge graphs and intelligent fault diagnosis in power systems and intelligent manufacturing repeatedly emphasize the need to transform such scattered textual knowledge into structured forms that can support intelligent analysis and decision-making [3].
In parallel, knowledge graph (KG) technologies have been rapidly adopted in the power domain to integrate multi-source data and encode expert experience [4]. Existing work has constructed power-system KGs around equipment, faults, protection actions, and emergency handling, and then used graph reasoning or rule engines for fault localization and treatment recommendation [2,3]. For example, several studies design ontology schemas and rule sets to represent faults and their causal relations, and then link SCADA/PMU (supervisory control and data acquisition/phasor measurement unit) data and O&M records to the graph for assisted diagnosis [5,6]. Similar KG-driven fault-diagnosis frameworks have also been developed for rotating machinery and intelligent manufacturing systems under the Industrial Internet of Things paradigm [7]. Nevertheless, many of these systems still depend on manual template design or shallow pattern matching for information extraction, which limits their ability to capture diverse expressions in free-form logs and to scale to new equipment types and fault scenarios. In addition, the KG-based reasoning module is often decoupled from the underlying text processing pipeline, so the system cannot fully exploit the semantic capabilities of modern neural language models.
Positioning vs. cited KG fault-diagnosis studies: unlike signal- or simulation-driven fault-type classifiers, we focus on extracting structured entities and relations from real-world unstructured O&M narratives and using the resulting triplets to ground a retrieval-augmented diagnostic assistant; Section 4.7 provides a structured comparison with representative cited works.
In natural language processing (NLP), named entity recognition (NER) and relation extraction (RE) are core information extraction (IE) tasks that convert unstructured text into structured triplets [8]. The advent of large pretrained language models (PLMs) such as BERT has dramatically improved the performance of NER and RE in many domains by providing contextualized representations that can be fine-tuned on downstream tasks with relatively modest data [9]. Domain-specific PLMs (e.g., BioBERT in the biomedical field) further demonstrate that adapting general BERT models to specialized corpora significantly boosts performance on domain NER and RE [10]. Fine-tuned BERT-style encoders have achieved strong results on various span-based NER and RE architectures, and joint models such as SpERT show that shared span representations can effectively improve triplet extraction accuracy by modeling entities and relations in a unified manner [11]. These advances suggest that PLM-based IE is a promising way to mine structured fault knowledge directly from noisy O&M logs in power systems.
At the same time, knowledge graphs and graph-driven diagnosis have begun to be explored in both power systems and manufacturing [12]. KG-based fault-diagnosis frameworks have been proposed to represent equipment structures, fault modes, and maintenance procedures, and to combine data-driven models with symbolic reasoning [4]. In the power domain, KGs have been constructed for relay-protection device faults [13] and power-system fault-diagnosis procedures, and graph-based retrieval or reasoning is used to assist human operators in locating faults and recommending handling measures [14]. However, most of these methods either focus on structured data (e.g., protection signals, alarms) or rely on relatively simple entity patterns, and they seldom employ strong PLM-based IE to robustly parse long, unstandardized O&M narratives. Moreover, the IE module and the diagnostic decision module are usually loosely coupled: the KG is populated offline, while online diagnosis does not dynamically exploit newly extracted knowledge from incoming logs.
Recent progress in large language models (LLMs) and retrieval-augmented generation (RAG) offers a new paradigm for building knowledge-centric diagnostic agents. In a typical RAG framework, a retriever first selects relevant documents or knowledge items from an external store, and an LLM then conditions on these retrieved contexts to generate grounded answers or recommendations [15]. RAG has been successfully applied in knowledge-intensive NLP tasks such as open-domain question answering and multi-hop reasoning [16], and has also begun to appear in engineering domains by combining KGs with LLMs for fault analysis and decision support [17]. While these end-to-end LLM approaches can be effective for offline analysis, their direct adoption in power-system operation faces several practical constraints. First, an LLM-only output is typically free-form text, which makes it difficult to enforce domain constraints and to audit the reasoning process. In contrast, our framework explicitly converts O&M logs into structured triplets and organizes them in a power-fault knowledge graph, enabling traceable evidence chains (e.g., entity spans, retrieved triplets, and graph paths) that can be inspected by dispatchers and engineers. Second, power-system operation is safety-critical, and hallucinated facts or inconsistent suggestions are unacceptable. By grounding generation on retrieved KG triplets and by applying schema- and rule-based checks during KG updates, the proposed hybrid approach provides a more controllable mechanism to mitigate unverified statements compared with unconstrained end-to-end generation. Third, operational workloads often involve long logs and bursty inputs; running an LLM as the sole decision maker may lead to unstable latency and higher compute cost due to long-context inference. Our design confines the LLM usage mainly to information extraction and response generation conditioned on a compact retrieved subgraph, while the core reasoning is performed over the KG, which improves efficiency and stabilizes online response time. Therefore, we position the hybrid IE + KG + graph-augmented RAG approach as a pragmatic choice for real-world deployment where interpretability, controllability, and predictable latency are required.
From a modeling perspective, several practical challenges arise when applying PLM-based IE to power-system O&M logs:
- Limited and imbalanced training data. Real-world logs are typically collected over many years, but annotated corpora for NER and RE in this domain are small and highly imbalanced: a few frequent entity/relation types dominate, while critical but rare fault patterns (e.g., specific protection misoperations or abnormal switching sequences) have very few examples. The NER literature has pointed out that class imbalance and scarce supervision remain key obstacles in specialized domains [18].
- Computational cost of full fine-tuning. Fully fine-tuning a large PLM on a small domain corpus can be computationally expensive and prone to overfitting. Parameter-efficient fine-tuning methods such as Low-Rank Adaptation (LoRA) alleviate this issue by freezing the original model parameters and injecting a small number of low-rank trainable matrices into Transformer layers, drastically reducing trainable parameters while maintaining performance [19].
- Robust learning under extreme class imbalance. In the context of imbalanced classification, focal loss has been proposed to down-weight easy examples and focus training on hard, misclassified ones, and has achieved significant success in dense object detection [20]. Complementary to such reweighting strategies, range loss was originally introduced in face recognition to simultaneously reduce intra-class variance and enlarge inter-class separation, which is particularly beneficial under long-tailed distributions [21]. Combining these loss functions with sampling strategies (e.g., undersampling of majority classes) offers a principled way to stabilize training in heavily skewed datasets.
Motivated by these observations, we propose a schema-grounded, evidence-driven graph-augmented fault-diagnosis framework for power-system equipment based on O&M logs. Different from existing KG + NLP fault-analysis architectures that often use the KG merely as an auxiliary context source or a post hoc explanation layer, our framework is designed as a tightly coupled log-to-knowledge-to-decision pipeline: the KG is treated as an explicit carrier of diagnostic evidence and operational constraints, and every diagnostic recommendation is produced together with retrievable and verifiable supporting evidence (graph triples and historical cases).
Concretely, we first annotate a corpus of O&M logs with domain-specific entities (e.g., equipment, fault phenomena, protection actions, operations, and handling measures) and relations (e.g., cause-of, located-at, handled-by), and formulate a joint NER–RE task under a tailored schema. We then adapt a BERT-based encoder with task-specific decoding layers (BiLSTM-CRF for NER and MLP for RE) using LoRA, enabling parameter-efficient domain specialization while keeping the number of trainable parameters small. To address the severe long-tail distribution in both entity and relation labels—where rare but safety-critical classes can be overshadowed by frequent routine patterns—we introduce an imbalance-aware training objective that combines focal loss, range loss, and majority-class undersampling, explicitly improving learning on minority classes.
The extracted entities and relations are further canonicalized and linked to construct and continuously enrich a power-equipment fault knowledge graph. Importantly, the KG is not only used as “background knowledge”; instead, it becomes the primary evidence substrate for downstream diagnosis. At inference time, we reuse the trained NER model as a KG-aware retriever to link new incoming logs to KG entities, and we build a graph-augmented RAG diagnostic agent that retrieves relevant triples and historical cases and then conditions a generative model to produce diagnostic suggestions and handling recommendations. This yields traceable reasoning: the agent outputs not only the diagnosis/action suggestions but also the corresponding supporting evidence, facilitating operator verification and practical deployment in safety-critical power systems.
The main contributions of this work are summarized as follows:
- Evidence-driven log-to-KG-to-diagnosis framework and domain corpus. We build an annotated O&M log corpus with a tailored entity–relation schema and construct a continuously expandable power-fault KG. On this basis, we propose an end-to-end framework that transforms unstructured logs into structured, retrievable knowledge and supports evidence-grounded diagnostic reasoning, making the KG an integral part of the decision process rather than a peripheral context component.
- LoRA-based joint NER–RE with imbalance-aware optimization for long-tail fault labels. We design a PLM-based joint extraction model with LoRA for parameter-efficient adaptation, and introduce an imbalance-aware optimization scheme combining focal loss, range loss, and majority-class undersampling. This design explicitly targets rare but safety-critical entity/relation types and yields consistent improvements over standard fine-tuning and cross-entropy baselines under realistic class imbalance.
- Graph-augmented RAG diagnostic agent with traceable evidence chaining. We propose a KG-grounded retrieval-and-generation workflow that links new logs to KG entities and retrieves relevant triples and historical cases to guide a generative model. The agent produces interpretable diagnostic suggestions together with verifiable evidence (retrieved graph context and log snippets), improving the coverage and auditability of diagnostic results compared with keyword-based retrieval and non-graph-augmented RAG setups on real-world regional-grid O&M logs.
The remainder of this paper is organized as follows. Section 2 describes the imbalance-aware information extraction framework for fault logs, detailing the joint NER–RE model and the training strategy incorporating LoRA and the mixed loss function. Section 3 introduces the graph-augmented RAG diagnostic agent and its inference workflow. Section 4 presents the experimental settings, results, and a qualitative case study. Section 5 provides a discussion of the findings and outlines future research directions. Finally, Section 6 concludes the paper.
2. Imbalance-Aware Information Extraction for Fault Logs
In this section, we present the proposed imbalance-aware information extraction framework for power system fault logs. The goal is to extract entities and relations from unstructured O&M texts and transform them into structured triplets suitable for knowledge-graph construction and downstream diagnostic reasoning.
2.1. Problem Formulation
Let a fault-processing log be a token sequence:
For RE, we assume that entity spans have been identified, and we denote the set of entities in a log as
where each entity is defined by its start index , end index , and entity type . Given an entity pair , the RE model predicts a relation type (including a no-relation class).
Our information extraction module therefore consists of:
- A BERT-based shared encoder for the input sequence ;
- A BiLSTM-CRF decoder for token-level NER;
- A relation classifier operating on entity-pair representations;
- An imbalance-aware training strategy shared across NER and RE.
2.2. BERT-Based Contextual Encoding
We first obtain contextualized token embeddings using a pre-trained BERT encoder. Given the input sequence , BERT produces hidden representations
where is the hidden size and denotes the model parameters.
For completeness, the self-attention mechanism in each Transformer layer computes
where , , are the query, key and value projections of the input ; , , are learnable matrices, and is the key dimensionality.
Although BERT is originally trained with masked language modeling and next sentence prediction, in our work, we treat BERT as a contextual feature extractor and fine-tune it with task-specific losses defined in Section 2.3, Section 2.4 and Section 2.5.
2.3. NER with BiLSTM-CRF
To model sequential dependencies in entity labels, we stack a bidirectional LSTM (BiLSTM) and a conditional random field (CRF) layer on top of the BERT embeddings.
2.3.1. BiLSTM Layer
The BiLSTM processes the sequence in both forward and backward directions:
and the BiLSTM output at position is the concatenation:
We then compute emission scores for each possible tag:
where and are learnable parameters.
2.3.2. CRF Decoding
The CRF layer models the conditional probability of a label sequence given the sequence of emission scores . Let denote the transition matrix, where is the score of transitioning from tag to tag . The score of a label sequence for input is
where is a special start tag and is the emission score of tag at position .
The CRF defines the conditional probability:
Given a training set , the NER loss is the negative log likelihood:
During inference, the most probable label sequence is obtained via the Viterbi algorithm:
2.4. Relation Extraction with BERT-MLP
For relation extraction, we use the BERT encoder to obtain token representations and then build span-based representations for entity pairs.
For an entity , we compute its span embedding by mean-pooling the corresponding BERT outputs:
Given a candidate head–tail pair , we form the pair representation:
where denotes element-wise multiplication, and is the element-wise absolute value.
A multi-layer perceptron (MLP) composed of a linear layer, nonlinear activation and output layer yields logits for each relation type:
where are learnable parameters and is a nonlinear activation (e.g., ReLU).
Given a relation-annotated dataset containing triples , the basic cross-entropy loss is
2.5. Parameter-Efficient Adaptation with LoRA
Directly fine-tuning all parameters of BERT on small, imbalanced corpora is computationally expensive and prone to overfitting. We therefore adopt LoRA to achieve parameter-efficient fine-tuning.
Consider a weight matrix in a linear layer of the Transformer (typically in the self-attention projections). Instead of updating directly, LoRA introduces a low-rank decomposition:
where , , and is the adaptation rank.
During fine-tuning:=, is kept frozen; only and are updated by gradient descent. If we denote by the input to this layer, the forward computation becomes
so that the expressive capacity is enhanced with only additional parameters. We apply LoRA to selected projection matrices (e.g., query and value) in the BERT encoder for NER and RE tasks, which significantly reduces training time and GPU memory consumption while maintaining or improving task performance.
2.6. Imbalance-Aware Loss: Focal Loss and Range Loss
The annotated O&M corpus exhibits a highly skewed distribution of entity and relation labels: a few frequent classes (e.g., common device types or general “fault” descriptions) dominate the dataset, whereas rare but safety-critical classes have very few samples. To address this, we adopt an imbalance-aware optimization scheme that combines focal loss and range loss, along with undersampling of majority classes.
2.6.1. Focal Loss for Sample-Level Reweighting
For a multi-class classification problem with a ground-truth label and predicted probability for the true class, the standard cross-entropy loss is
Focal loss introduces a modulating factor to down-weight easy examples and focuses on hard or minority examples:
where is the focusing parameter, and is a class-specific balancing weight. When is large (easy samples), the factor becomes small, reducing the contribution of these samples to the total loss. We apply focal loss to both token-level NER classification (at the emission layer) and relation-type classification in RE.
2.6.2. Range Loss for Feature-Level Structuring
Focal loss operates at the sample level. To further encourage discriminative feature distributions under long-tailed label frequencies, we incorporate a range loss defined on deep embeddings. Let denote the embedding of sample (e.g., token-level or span-level representation), and let be the set of embeddings belonging to class . Within each class, we consider pairwise Euclidean distances:
For class , we select the largest intra-class distances and define the intra-class term as
where is the mean of these distances. This term encourages the largest intra-class distances to shrink toward their mean, reducing intra-class variance. We also compute a class center for each class:
Among all pairs of distinct class centers , we identify the pair with the smallest distance:
and define the inter-class term as a hinge-style penalty:
where is a margin. This pushes the closest class centers farther apart, enlarging inter-class separability. The overall range loss for a mini-batch is
where and control the relative importance of the two terms.
2.6.3. Combined Objective and Undersampling
For both NER and RE, the final training objective for a batch is a weighted combination of focal loss and range loss:
where denotes the sum of focal losses over all samples in the batch (on top of the CRF or softmax outputs), and is a hyperparameter.
In addition to loss-level reweighting, we apply simple undersampling to reduce the dominance of majority classes in mini-batch construction. For each class , let be the number of training samples. We cap the effective sample size to a maximum , and randomly subsample majority classes with such that
This yields a more balanced training distribution without discarding rare-class samples.
2.7. Overall Pipeline
The overall pipeline of the proposed imbalance-aware information extraction module is summarized as follows:
- Encoding: A fault log is tokenized and fed into the BERT encoder with LoRA-based adaptation, producing contextual representations .
- Entity recognition: The BERT embeddings are passed through a BiLSTM-CRF decoder to obtain token-level label sequences. The model is trained with a combination of CRF negative log-likelihood, focal loss, and range loss.
- Relation extraction: Detected entities are grouped into candidate pairs. For each pair, we construct span-level representations and feed them to an MLP classifier. The classifier is trained with focal loss and range loss under an undersampled mini-batch scheme.
- Triplet construction: For each log, the predicted entities and relations are converted into structured triplets , which are then used to build and update the power-system fault knowledge graph.
Figure 1 illustrates the overall architecture of the proposed knowledge extraction model, including the BERT encoder, the BiLSTM-CRF NER branch, the relation classification branch, and the imbalance-aware training components.
Figure 1.
Architecture of the proposed knowledge extraction model.
2.8. Streaming Scalability and Throughput Considerations
In real control-center operation, fault-processing logs may arrive continuously and in bursts. To avoid backlogs that could delay decision support, our framework separates (i) offline knowledge-graph construction from historical logs and (ii) online incremental inference for new logs. During online operation, the NER/RE models run in a lightweight, parameter-efficient LoRA mode, and only the retrieved top-K entities/triplets are passed to the RAG generator. This design bounds the per-request computation and prevents the extraction module from being “overfilled” by the incoming stream. In addition, we apply (a) maximum sequence length truncation/padding, (b) batch-based micro-batching for throughput, and (c) priority handling strategies (e.g., alarm-severity or protection-action keywords) to select which logs should be processed first under extreme load. These engineering controls ensure stable latency and maintain the relevance of retrieved evidence for decision-making.
3. Assistance Agent Based on Graph-Augmented RAG
In this section, we describe the proposed assistance agent that performs graph-augmented diagnostic reasoning over power system fault logs. The agent follows the RAG paradigm: given a new fault description, it first retrieves relevant knowledge from a power-fault KG and then generates diagnostic suggestions conditioned on both the input log and the retrieved knowledge.
3.1. Knowledge Graph Representation
Based on the information extraction module described in Section 2, we construct a power-fault knowledge graph:
where is the set of entities (e.g., devices, fault phenomena, causes, operations, handling measures), is the set of relation types (e.g., cause-of, located-at, handled-by), and is the set of directed labeled edges (triplets):
Each node is associated with a textual description (such as a canonical name, fault narrative snippet, or operation record), and each edge may also be associated with a brief textual annotation . We use a text encoder (e.g., the same BERT encoder as in Section 2) to obtain node and edge embeddings:
where denotes the encoder and is the embedding dimension.
These embeddings are used by the agent to measure semantic similarity between query entities and graph nodes or triplets during retrieval.
3.2. Query Analysis and Entity-Based Retrieval
Given a new fault-processing log , the assistance agent first applies the NER model from Section 2 to obtain the label sequence:
and extracts a set of query entities:
where each entity is defined by its span and type . For each query entity , we compute a span embedding using the BERT encoder:
where are the contextual token representations of as in Section 2.2.
To measure the relevance between a query entity and a KG node, we define a similarity function:
where and are learnable linear projections (or identity mappings) that map entity and node embeddings into a shared space:
For each node , we aggregate its relevance to the entire query via the maximum similarity over all query entities:
We then select the top- nodes with the highest scores:
and expand them into a local subgraph by including their -hop neighbors:
where consists of the nodes in and their neighbors up to hops, and contains all edges between nodes in .
This entity-aware retrieval process allows the agent to focus on the region of the KG most relevant to the devices, phenomena, and operations mentioned in the current fault log, rather than relying on purely lexical similarity between the entire log and node descriptions.
3.3. Construction of Textual Knowledge Context
To feed graph-based knowledge into the generative model, we linearize the retrieved subgraph into a textual context. Specifically, for each edge , we form a natural-language template:
where and are node descriptions and is expressed as a relation phrase. For example,
We then rank the triplets in by the relevance of their head and tail nodes to the query:
and select the top- triplets:
The textual knowledge context is then constructed by concatenating the templates of the selected triplets, possibly with separators:
This context captures a compact, locally consistent view of historically observed faults, causes, and handling measures that are most relevant to the current query.
3.4. BART-Based Retrieval-Augmented Generation
We adopt a sequence-to-sequence model based on Bidirectional and Auto-Regressive Transformers (BART) as the generator in our assistance agent. Let the input to the generator be the concatenation of the original fault log and the retrieved knowledge context:
The BART encoder converts into a sequence of hidden states:
The BART decoder generates an output token sequence:
which represents diagnostic suggestions and reasoning explanations (e.g., likely fault location, probable cause, recommended handling steps). The conditional probability of the output sequence is
where each term is modeled by the decoder with auto-regressive attention over previous outputs and cross-attention to . Given a training set of query–response pairs , the generator is optimized with the standard token-level cross-entropy loss:
In our framework, the BART generator and the retrieval components share the same NER model and text encoder, which ensures consistency between the way entities are extracted from queries and the way they are matched to KG nodes.
3.5. Inference Workflow and Discussion
At inference time, given a new fault log , the assistance agent performs the following steps:
- Entity extraction: Apply the NER model to obtain the entity set and their span embeddings .
- Entity-aware retrieval: Compute relevance scores for all nodes , select the top- nodes and expand them into a local subgraph .
- Triplet selection and context construction: Rank edges in by , select top- triplets, and construct the textual context .
- Retrieval-augmented generation: Form the generator input and use the BART model to generate diagnostic suggestions .
4. Experimental Evaluation
In this section, we empirically evaluate the proposed framework from three perspectives: (1) the effectiveness of the imbalance-aware training strategy for NER; (2) its impact on RE; and (3) the overall behavior of the graph-augmented assistance agent through a qualitative case study.
4.1. Experimental Setup
4.1.1. Dataset and Tasks
All experiments are conducted on an internal corpus of power system O&M logs collected from a regional grid. Each log records a real fault-processing process, including textual descriptions of the fault phenomena, protection behavior, on-site inspection, operations executed by staff, and preliminary diagnostic conclusions.
We define two supervised information extraction tasks:
- NER: The goal is to assign BIO-style labels to each token, identifying entities such as DEVICE, DEVICE_PART, PHENOMENON, PROTECTION, OPERATION, MEASURE, CAUSE, etc. This task evaluates how well the model can recognize domain-specific concepts from free-form fault logs.
- RE: Based on annotated entity spans, this task predicts semantic relations between entity pairs, such as cause-of, located-at, triggered-by, and handled-by. It measures the model’s ability to reconstruct structured triplets from unstructured descriptions.
Entity and relation annotations are created by experienced power-system engineers according to a predefined schema, and the corpus is randomly split into training, validation, and test sets. Care is taken to ensure that logs belonging to the same fault event do not appear in multiple splits.
Dataset schema and statistics (added for reproducibility): Our annotation schema contains seven entity types (DEVICE, DEVICE_PART, PHENOMENON, PROTECTION, OPERATION, MEASURE, CAUSE) and four relation types (cause-of, located-at, triggered-by, handled-by), with an additional no-relation label for negative pairs.
Log length handling: Logs are tokenized with the BERT WordPiece tokenizer. We set the maximum sequence length to 256 tokens (Table A1). For logs longer than this limit, we apply a sliding-window chunking strategy with overlap (stride 128) and merge chunk-level predictions back to the original character spans; this avoids dropping late-occurring entities/relations in long narratives.
Class distribution note: The corpus exhibits a long-tailed distribution across both entity and relation labels (a few frequent labels dominate, while safety-critical labels are much rarer). Because the raw label counts are confidential, we report schema-level counts and evaluation metrics (Macro-F1) that are robust to imbalance; future public releases will include full distributions.
4.1.2. Model Variants and Baselines
For both tasks, we evaluate a series of model variants that progressively incorporate LoRA and imbalance-aware strategies. This allows us to perform a detailed ablation study. The statement of compared models is listed in Table 1.
Table 1.
Setting statement of compared models.
4.1.3. Implementation Details and Evaluation Metrics
For all models, we use a pre-trained BERT encoder as the shared backbone. LoRA modules are inserted into the self-attention projections of selected Transformer layers. Hyperparameters such as the learning rate, batch size, focal loss focusing parameter , class weights , and the weights of the range loss terms are tuned on the validation set.
We report three metrics:
- Accuracy (%): For NER, token-level accuracy; for RE, relation-label accuracy over candidate entity pairs.
- Macro-F1 (%): For NER, Macro-F1 over all entity tags; for RE, Macro-F1 over all relation types, giving equal weight to rare and frequent classes.
- Average training time per epoch (s): Used to assess the efficiency of different training strategies and the effect of LoRA.
4.1.4. Training and Hardware Parameters
Table A1 (Appendix A) summarizes the key hyperparameters used in our experiments, including the BERT backbone, maximum sequence length, optimizer and learning-rate schedule, LoRA rank and target modules, focal-loss parameters (γ and class weights), range-loss parameters (K, margin m, and weighting coefficients), and the undersampling cap M for majority classes.
All experiments were executed on a single workstation; we now report the hardware configuration (CPU model, GPU model and memory, system RAM, and CUDA/software stack) in Table A2 (Appendix A) to improve reproducibility.
4.1.5. Metric Definitions
This section details the calculation methods of all evaluation metrics used in our case studies.
- (1)
- Accuracy
- (2)
- Macro-F1
- (3)
- Average time per epoch
Note on MAPE/RMSE: Our core tasks (NER and RE) are multi-class classification, not continuous-value regression; therefore, MAPE/RMSE are not directly applicable. We add Micro-F1 as an auxiliary IE metric and discuss metric choice in Section 4.7.
4.2. NER Results and Analysis
Table 2 reports the NER performance and efficiency of the eight model variants, and Figure 2 and Figure 3 visualize the distributions of Accuracy and Macro-F1 across 50 training epochs. The baseline BERT-BiLSTM-CRF achieves 80.3% token Accuracy and 63.2% Macro-F1 with 27.3 s per epoch. In contrast, the full configuration (LoRA + mixed focal–range loss + undersampling) reaches 92.0% Accuracy and 71.3% Macro-F1 while reducing the average time to 22.3 s per epoch (≈18% faster).
Table 2.
Comparison of proposed method and Baselines on NER task.
Without LoRA, both imbalance-mitigation components provide clear gains. Undersampling improves the baseline to 87.4% Accuracy and 67.4% Macro-F1, while the mixed focal–range loss yields 88.2% Accuracy and 67.9% Macro-F1. Combining mixed loss with undersampling further increases performance to 91.8% Accuracy and 70.9% Macro-F1, indicating that sampling and loss-level reweighting are complementary and mainly benefit long-tail entity tags.
Figure 2.
Distribution of accuracy across different models on NER task.
LoRA mainly contributes training efficiency. LoRA-only reduces the per-epoch time from 27.3 s to 22.1 s, but brings only marginal changes in Macro-F1 (63.5%). When paired with imbalance handling, the speedup is preserved while performance improves (e.g., 89.3% Accuracy with undersampling and 87.8% Accuracy/67.2% Macro-F1 with mixed loss). Overall, the full recipe achieves the best accuracy–robustness–efficiency trade-off, and the boxplots in Figure 2 and Figure 3 suggest the improvement is stable across epochs rather than driven by a few favorable runs.
Figure 3.
Distribution of Macro-F1 across different models on NER task.
4.3. RE Results and Analysis
Table 3 summarizes the RE results and average training time, and Figure 4 and Figure 5 show the performance distributions across 50 epochs. The baseline BERT-MLP achieves 82.1% Accuracy and 60.4% Macro-F1 with 25.3 s per epoch. The full configuration (LoRA + mixed focal–range loss + undersampling) improves to 88.0% Accuracy and 70.1% Macro-F1 while reducing training time to 21.2 s per epoch (≈16% faster), with the larger Macro-F1 gain indicating stronger benefits on minority relations.
Table 3.
Comparison of proposed method and Baselines on RE task.
Without LoRA, undersampling and the mixed loss each improve over the baseline: under-sampling reaches 85.4% Accuracy and 63.1% Macro-F1, while the mixed loss attains 86.9% Accuracy and 65.2% Macro-F1. Combining the two further increases performance to 87.8% Accuracy and 69.2% Macro-F1, which is important for knowledge-graph construction because rare relations (e.g., fine-grained cause-of or handled-by) are typically underrepresented.
Figure 4.
Distribution of accuracy across different models on RE task.
Figure 5.
Distribution of Macro-F1 across different models on RE task.
LoRA substantially reduces computation (about 21 s per epoch) but is not sufficient alone under severe label imbalance (81.3% Accuracy and 59.2% Macro-F1). Coupling LoRA with undersampling or the mixed loss restores and improves performance (e.g., 87.2%/63.9% with undersampling and 85.1%/64.9% with mixed loss), and the full recipe provides the best overall trade-off (88.0% Accuracy, 70.1% Macro-F1, 21.2 s per epoch). Figure 4 and Figure 5 also suggest more stable performance across epochs for the proposed configuration compared with the baseline.
4.4. Case Study: Transformer Overheating Fault
To gain qualitative insights into how the proposed framework behaves in realistic scenarios, we briefly examine one representative case from the test set, involving a transformer overheating fault.
The O&M log describes abnormal temperature rise at a main transformer, local overheating near the high-voltage bushing, the triggering of over-temperature alarms without protective tripping, on-site observations of oil seepage and discoloration, and the decision to reduce load and take the transformer out of service. The free-form narrative mixes observations, protection behavior, and operations, which can be challenging for rule-based systems.
Using the proposed NER model, the log is successfully parsed into entities corresponding to the device (“main transformer”, “high-voltage bushing”), phenomena (“abnormal temperature rise”, “local overheating”, “oil seepage”, “discoloration”), protection information (“over-temperature alarm”, “no trip”), operations/handling (“reduce load”, “take out of service”), and cause (“poor contact at the bushing”, “insulation deterioration”). The baseline model, by contrast, tends to miss the rare cause-type entity and merges some phenomena into generic labels, which weakens downstream reasoning.
On top of these entities, the RE model correctly identifies relations such as has-part (transformer–bushing), cause-of (poor contact → local overheating; insulation deterioration → fault), and handled-by (local overheating → load reduction; insulation deterioration → remove from service). These triplets populate the power-fault knowledge graph, where they can be matched against historical cases with similar patterns.
When the fault log is fed to the assistance agent, the entity-aware retriever focuses on graph regions containing transformer bushing faults with overheating and oil leakage, and the BART-based generator produces diagnostic suggestions that explicitly point to the bushing connection as the most likely root cause and recommend keeping the transformer out of service until refurbishment and testing are completed. The generated explanation aligns well with expert judgment recorded in the maintenance report, illustrating how the combination of accurate information extraction and graph-augmented reasoning leads to interpretable and practically useful diagnostic support.
4.5. What Drives Accuracy, Macro-F1, and Training Time
Based on the ablation results in Table 2 and Table 3, the largest contributors to Accuracy and Macro-F1 are the imbalance-mitigation components (mixed focal–range loss and majority-class undersampling), which consistently improve minority-class recognition and therefore raise Macro-F1. LoRA mainly reduces the number of trainable parameters in the Transformer backbone and improves training efficiency, leading to shorter per-epoch time while preserving accuracy when combined with imbalance handling. The BiLSTM-CRF decoding cost (NER) and the number of candidate entity pairs (RE) also affect runtime; therefore, controlling sequence length and pruning candidate pairs are important in practice.
4.6. How to Further Improve Training and Inference Efficiency
To further improve the average time per epoch and online latency, the following measures can be adopted: (1) reduce sequence length via domain-aware sentence/segment splitting; (2) enable mixed-precision training and inference; (3) freeze additional layers or increase LoRA sparsity to reduce backpropagation cost; (4) cache embeddings for frequent KG nodes and use approximate nearest neighbor search for retrieval; and (5) limit relation candidates by type constraints (e.g., only allow CAUSE→PHENOMENON for cause-of) to reduce the O(N2) pairing overhead.
4.7. Comparison with Cited Literature and Metric Choice
Direct numerical comparison across studies is challenging because many cited power-fault works focus on signal- or simulation-based fault-type classification, whereas our main contribution is information extraction (NER/RE) from unstructured O&M text logs and subsequent KG-grounded RAG assistance. Nevertheless, we add a structured comparison, as shown in Table 4, to position our results relative to the cited literature and to clarify metric choice.
Table 4.
Comparison with cited works.
Consideration of MAPE/RMSE: These errors are designed for continuous-value regression (e.g., forecasting) and are not well-defined for our discrete-label IE outputs. We therefore keep Accuracy and Macro-F1 as the main quality measures; Macro-F1 is particularly informative under severe label imbalance. For completeness, Appendix A (Table A4) provides the Micro-F1 definition, which is also widely used in IE and complements Macro-F1.
5. Discussion and Future Work
5.1. Discussion
5.1.1. Operational Deployment Considerations
For real-time deployment in power-system control centers, factors beyond offline accuracy must be considered, including inference latency, scalability to a continuously growing KG, integration with SCADA/EMS alarm streams, and reliability/uncertainty handling in safety-critical decision support. Our framework is designed as a bounded and evidence-grounded pipeline—LoRA-based IE → KG-aware retrieval (top-K + h-hop) → triplet linearization → BART-based RAG—which provides explicit knobs to control computation and enables predictable online behavior.
Inference latency and throughput. End-to-end inference time is composed of (i) log preprocessing/tokenization, (ii) NER/RE inference for entity–relation extraction (or reuse of the NER model for query entity detection), (iii) KG retrieval and bounded subgraph construction, and (iv) RAG conditioned on a compact set of retrieved triplets. In online operation, the major latency risks come from long logs and bursty arrivals. Our design mitigates these risks by (a) enforcing a maximum sequence length for the IE models (with chunking/overlap for long logs as described in the log-length handling strategy), (b) running IE in parameter-efficient LoRA mode, and (c) bounding retrieval by selecting top-K nodes/triplets and restricting neighborhood expansion to h hops. In addition, the streaming controls described in Section 2.8 (micro-batching and priority processing by alarm severity/keywords) can be used to maintain stable throughput under high load.
Scalability to large KGs. As triplets continuously enrich the KG, retrieval and neighborhood expansion must remain efficient. Our entity-aware retrieval is inherently scalable because it operates on embedded nodes/edges and selects only a compact subgraph for each query (top-K nodes and a bounded h-hop expansion). For large-scale deployments, the same mechanism can be further strengthened by (i) maintaining approximate nearest neighbor (ANN) indices for node embeddings, (ii) partitioning the KG by region/substation/asset type and searching within the most relevant partitions, (iii) pruning neighbors via type constraints (e.g., CAUSE→PHENOMENON for cause-of) to avoid exploding neighborhoods, and (iv) caching embeddings and frequently accessed subgraphs for recurrent equipment/fault patterns.
Integration with SCADA/EMS and alarm systems. The proposed agent is compatible with standard dispatching workflows where SCADA/EMS provides time-stamped alarms/events and O&M platforms provide free-text logs. In a practical interface, each request can be represented as {timestamp, equipment ID, alarm codes (optional), log text}. The agent outputs (i) predicted fault location/cause/handling suggestions, (ii) supporting evidence spans from the input log, and (iii) retrieved KG triplets and graph paths as an explanation trace. This structured output facilitates human verification, event reporting, and post-incident auditing, and can be surfaced as a decision-support panel in existing operation platforms without replacing established procedures.
Reliability and uncertainty handling. Because power-system operation is safety-critical, the agent must avoid overconfident recommendations. Our framework supports reliability controls at multiple stages: (i) confidence filtering for extracted entities/relations before KG insertion (Section 5.2), (ii) evidence grounding by conditioning generation on retrieved triplets and their provenance, and (iii) conservative fallback behaviors when confidence is low. In deployment, we recommend an “abstain/verify” policy: if extraction confidence or retrieval relevance is below thresholds, the agent returns a conservative suggestion (e.g., request manual verification or follow rule-based procedures) instead of a definitive diagnosis. These mechanisms improve trustworthiness and reduce operational risk while preserving the practical benefit of fast, evidence-based assistance.
5.1.2. Knowledge Graph Update, Validation, and Maintenance
Our KG is continuously enriched by triplets extracted from newly arriving O&M logs. However, because extraction is imperfect and operational knowledge evolves over time, KG updates require explicit validation, conflict handling, and maintenance to prevent error accumulation and to preserve auditability.
(1) Candidate-to-validated update with confidence gating. For each newly extracted triplet ⟨head, relation, tail⟩, we store it first as a candidate edge with metadata including: extraction confidence, source log identifier, timestamp, and (when available) evidence spans supporting head/relation/tail. A candidate is promoted to a validated edge only if it passes a validation gate, including: (i) confidence thresholding, (ii) schema/type constraints (valid entity types and relation types), and (iii) entity normalization/linking to canonical nodes (e.g., equipment IDs, synonym dictionaries). This conservative promotion strategy reduces noise from one-off extraction errors.
(2) Contradiction detection and resolution with time stamps and evidence tracking. Contradictory information may arise when different logs suggest conflicting causes or handling measures. Instead of overwriting existing knowledge, we maintain time-stamped, evidence-backed edges. Conflicts are detected when a new candidate contradicts an existing validated relation under the same context (e.g., same equipment and fault event). In such cases, we keep both relations but mark them with a conflict flag, retain their provenance (source logs and timestamps), and allow temporal reasoning (e.g., “latest evidence” vs. “historical evidence”). When multiple independent logs support the same relation, evidence accumulation (e.g., repeated observations) increases its confidence, whereas isolated contradictory edges remain in a pending/conflict state until further confirmation.
(3) Graph maintenance over time (deduplication, aging, and re-indexing). As the KG grows, maintenance is needed for both correctness and retrieval efficiency. We periodically perform: (i) synonym merging and duplicate removal through entity linking, (ii) archival/aging of obsolete identifiers or relations that are no longer relevant, while preserving version history for auditability, and (iii) re-indexing of node/edge embeddings and retrieval indices to ensure stable query performance. These safeguards keep the KG consistent and support scalable, reliable retrieval for the graph-augmented RAG agent in long-term operation.
5.2. Future Work
The proposed framework has demonstrated promising performance on real-world fault-processing logs, but several limitations remain. First, the information extraction (IE) module still depends on supervised annotations for entities and relations; for some rare but safety-critical types, labeled examples are scarce even after applying imbalance-aware training. Second, the current KG construction and retrieval strategy are relatively simple: triplets are inserted as nodes and edges, and retrieval is based mainly on text-derived embeddings and similarity scores, without explicitly modeling higher-order graph structure. Third, the BART-based generator conditions on a linearized textual context but does not provide calibrated uncertainty estimates for its diagnostic suggestions, which would be desirable in safety-critical power-system applications.
A natural next step is to better exploit the structure of the power-fault KG by integrating graph neural networks (GNNs) into the graph-augmented RAG framework, as sketched in Figure 6. Starting from initial node embeddings obtained from textual descriptions, a GNN can propagate information along the edges of via message-passing layers of the form:
producing structure-aware embeddings after layers. The entity-aware retriever described in Section 3 could then operate in this GNN-refined embedding space, selecting a more coherent subgraph and ranking triplets not only by textual similarity but also by structural context (e.g., multi-hop patterns and communities of related fault modes). We expect such GNN-enhanced retrieval to be particularly beneficial for complex faults that involve chains of devices and protection anomalies.
Figure 6.
Conceptual architecture for integrating graph neural networks (GNNs) into the graph-augmented RAG framework.
Beyond GNNs, several additional extensions are worth exploring. One is uncertainty-aware diagnosis, where Bayesian techniques or ensembles are used to attach confidence estimates to the agent’s suggestions and to flag low-confidence cases for expert review. Another is data-efficient learning, combining semi-supervised or active learning with the current IE module to better exploit large volumes of unlabeled logs and reduce annotation cost. A third direction is multimodal integration, extending the KG to include SCADA/PMU time-series and equipment condition data, and aligning these signals with textual events to support more comprehensive diagnostic reasoning. These directions build on the core components developed in this work and will be the focus of our future research.
Although Section 4 focuses on accuracy-oriented evaluation of the information extraction (IE) module, practical deployment in power-system operation requires additional considerations beyond predictive performance. We summarize key operational aspects below.
Inference latency. End-to-end inference latency consists of four stages: (i) log preprocessing and tokenization, (ii) entity and relation extraction, (iii) entity-aware KG retrieval and subgraph expansion, and (iv) retrieval-augmented generation conditioned on the retrieved knowledge context. In online operation, latency variability (e.g., under bursty log arrivals) can be as important as mean latency. Our pipeline is amenable to latency control because retrieval operates on compact embeddings and a bounded local subgraph, and the generator is conditioned on a truncated, top-K triplet context rather than the full KG. In practice, caching frequently accessed node/edge embeddings and precomputing indices for entity linking can further reduce retrieval overhead.
Scalability to large knowledge graphs. As the KG grows over time, scalability becomes a concern for retrieval and neighborhood expansion. Our retrieval procedure in Section 3 selects top-k relevant nodes and expands to a bounded h-hop subgraph, which provides an explicit knob to control computational cost. For large-scale KGs, additional strategies can be applied, including (i) graph partitioning by region/substation/asset type, (ii) time-windowed subgraphs to prioritize recent operational knowledge, (iii) top-m neighbor pruning to avoid exploding neighborhoods, and (iv) maintaining inverted indices from entity mentions to candidate canonical nodes for efficient entity linking.
Integration with SCADA/operation systems. In a real dispatching environment, the proposed agent can be integrated as a service that consumes alarms/events and O&M logs from existing operation platforms (e.g., SCADA/event management systems). The input can be standardized as a tuple of time stamp, equipment identifier, alarm codes, and free-text log description. The output can be produced in a structured form including (i) predicted fault location/cause/handling suggestions, (ii) supporting evidence spans from the log, (iii) retrieved KG triplets and their relevance scores, and (iv) an explanation trace that links the current case to historical incidents. Such a structured interface supports human-in-the-loop decision-making and facilitates post-event auditing.
Reliability and uncertainty handling. For safety-critical applications, the system should expose uncertainty and avoid overconfident suggestions. At the IE level, low-confidence entities/relations can be filtered or flagged before entering the KG. At the agent level, an “abstain” mechanism can be adopted: when retrieval confidence or generation confidence is low, the system returns a conservative recommendation (e.g., request manual verification) instead of a definitive diagnosis. Consistency checks across multiple evidence sources (e.g., repeated log mentions, alarms, or validated KG relations) can further improve reliability. These mechanisms provide a practical path for deploying the framework in operational settings where erroneous recommendations must be minimized.
6. Conclusions
This paper has presented a graph-augmented diagnostic framework for power-system fault logs, designed to cope with the practical challenges of scarce and highly imbalanced domain data. Starting from unstructured O&M records, the framework integrates an imbalance-aware information extraction module, a power-fault knowledge graph, and a graph-augmented assistance agent based on retrieval-augmented generation.
At the information-extraction level, we proposed a BERT-BiLSTM-CRF model for NER and a BERT-MLP model for RE, both adapted to the power-fault domain via LoRA-based parameter-efficient fine-tuning. To address the severe class imbalance in entity and relation labels, we introduced a unified training recipe that combines focal loss, range loss, and majority-class undersampling. Experiments on 3921 real-world fault-processing logs demonstrated that this strategy substantially improves both Accuracy and Macro-F1 over standard fine-tuning, while reducing training time per epoch. For NER, the full configuration achieved 92.0% Accuracy and 71.3% Macro-F1, improving on the baseline by 11.7 and 8.1 percentage points, respectively. For RE, the best configuration reached 88.0% Accuracy and 70.1% F1, with similar gains in performance and efficiency.
On top of the extracted entities and relations, we constructed a power-fault knowledge graph that organizes equipment, phenomena, causes, and handling measures into structured triplets. We then designed a graph-augmented assistance agent that reuses the NER model as a KG-aware retriever and employs a BART-based generator within a RAG framework. The agent links new logs to relevant regions of the knowledge graph, linearizes the retrieved triplets into textual context, and generates diagnostic suggestions grounded in historical cases. A qualitative case study on a transformer overheating fault illustrated that the agent could highlight critical rare causes, focus retrieval on semantically related incidents, and produce explanations that align well with expert judgments, thereby enhancing both the accuracy and interpretability of log-based fault diagnosis.
In future work, we plan to extend the framework in several directions. First, we will enrich the corpus with data from multiple utilities and explore semi-supervised and active learning techniques to further reduce annotation cost. Second, we intend to integrate graph neural networks into the retrieval and ranking process, leveraging structural information in the knowledge graph more effectively and exploring uncertainty-aware diagnosis. Third, we will investigate tighter multimodal integration of textual logs with SCADA/PMU time-series and equipment condition data, aiming at a unified, knowledge-centric diagnostic platform for complex power systems. We believe that the ideas presented in this work provide a practical and extensible basis for deploying intelligent, interpretable fault-diagnosis assistants in real-world grid operation environments.
Author Contributions
Conceptualization, Y.Z.; Methodology, Y.Z., Y.G. and Z.F.; Software, Y.G.; Validation, Y.Z., Q.L., S.Q., X.W. and X.F.; Formal analysis, Y.G.; Investigation, Y.Z.; Writing—original draft, Y.Z.; Writing—review and editing, Y.G. and Z.F.; Visualization, Q.L. and S.Q.; Supervision, Y.G. and X.W.; Project administration, Y.G. and Z.F. All authors have read and agreed to the published version of the manuscript.
Funding
This work was supported by the Research and Application of Power Grid Abnormal Scheduling Decision Technology Driven by Data and Knowledge Collaboration, with the project number 000005KC24010014 (Science and Technology Project number: ZDKJXM20240015).
Institutional Review Board Statement
Not applicable.
Informed Consent Statement
Not applicable.
Data Availability Statement
The original contributions presented in this study are included in the article. Further inquiries can be directed to the corresponding authors.
Conflicts of Interest
The authors declare no conflicts of interest.
Appendix A
Table A1.
Main training and model hyperparameters used in experiments.
Table A2.
Hardware and software environment.
Table A3.
Main abbreviations used in this paper.
Table A4.
Key dataset and preprocessing details added per reviewer request.
References
- Liu, F.; Peng, Y.; Liu, Q.; Wang, J.; Zhou, Y. Holistic Capture Graph Convolution Network With Ambient Signals for Line Short-Circuit Fault Diagnosis. IEEE Trans. Instrum. Meas. 2025, 74, 3541712. [Google Scholar] [CrossRef] [Scilit]
- Chen, Q.; Li, Q.; Wu, J.; Mao, C.; Peng, G.; Wang, D. Application of knowledge graph in power system fault diagnosis and disposal: A critical review and perspectives. Front. Energy Res. 2022, 10, 988280. [Google Scholar] [CrossRef] [Scilit]
- Li, C.; Wang, B. A knowledge graph method towards power system fault diagnosis and classification. Electronics 2023, 12, 4808. [Google Scholar] [CrossRef] [Scilit]
- Wang, X.; Han, H.; Gao, B. Enhancing Power Transformer Fault Diagnosis Through Dynamic Knowledge Graph Reasoning. IEEE Trans. Instrum. Meas. 2025, 74, 3502709. [Google Scholar] [CrossRef] [Scilit]
- Zhao, T.; Yue, M.; Wang, J. Structure-Informed Graph Learning of Networked Dependencies for Online Prediction of Power System Transient Dynamics. IEEE Trans. Power Syst. 2022, 37, 4885–4895. [Google Scholar] [CrossRef] [Scilit]
- Al-Qirim, N.; Majdalawieh, M.; Bani-hani, A.; Al Hamadi, H. Cyber threat intelligence for smart grids using knowledge graphs, digital twins, and hybrid machine learning in SCADA networks. Int. J. Eng. Bus. Manag. 2025, 17, 18479790251328183. [Google Scholar] [CrossRef] [Scilit]
- Peng, C.; Sheng, Y.; Gui, W.; Tang, Z.; Li, C. A Rolling Bearing Fault Diagnosis Method Based on Multimodal Knowledge Graph. IEEE Trans. Ind. Inform. 2024, 20, 13047–13057. [Google Scholar] [CrossRef] [Scilit]
- Li, J.; Sun, A.; Han, J.; Li, C. A survey on deep learning for named entity recognition. IEEE Trans. Knowl. Data Eng. 2022, 34, 50–70. [Google Scholar] [CrossRef] [Scilit]
- Devlin, J.; Chang, M.W.; Lee, K.; Toutanova, K. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies; Association for Computational Linguistics: Minneapolis, MN, USA, 2019; Volume 1, pp. 4171–4186. [Google Scholar]
- Lee, J.; Yoon, W.; Kim, S.; Kim, D.; Kim, S.; So, C.H.; Kang, J. BioBERT: A pre-trained biomedical language representation model for biomedical text mining. Bioinformatics 2020, 36, 1234–1240. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Eberts, M.; Ulges, A. Span-based joint entity and relation extraction with transformer pre-training. arXiv 2019, arXiv:1909.07755. [Google Scholar]
- Xia, L.; Zheng, P.; Li, X.; Gao, R.; Wang, L. Toward cognitive predictive maintenance: A survey of graph-based approaches. J. Manuf. Syst. 2022, 64, 107–120. [Google Scholar] [CrossRef] [Scilit]
- Zheng, S.; Liu, Z.; Yu, Y.; Dong, P.; Yang, X.; Wang, S.; Tao, C.; Xue, A. A quality evaluation method for the unstructured defect record of relay protection devices based on ontology and knowledge graph. Energy Rep. 2023, 9, 771–781. [Google Scholar] [CrossRef] [Scilit]
- Li, X.; Jiao, R.; Zhou, C.; Li, C.; Yan, B.; Li, R. Actively masked reasoning over knowledge graphs for fault diagnosis under insufficient information condition. Expert Syst. Appl. 2026, 295, 128854. [Google Scholar] [CrossRef] [Scilit]
- Lewis, P.; Perez, E.; Piktus, A.; Petroni, F.; Karpukhin, V.; Goyal, N.; Küttler, H.; Lewis, M.; Yih, W.; Rocktäschel, T.; et al. Retrieval-augmented generation for knowledge-intensive NLP tasks. Adv. Neural Inf. Process Syst. 2020, 33, 9459–9474. [Google Scholar]
- Zhang, X.; Zhao, F.; Liu, Y.; Chen, P.; Wang, Y.; Wang, X.; Ma, D.; Xu, H.; Chen, M.; Li, H. TreeQA: Enhanced LLM-RAG with logic tree reasoning for reliable and interpretable multi-hop question answering. Knowl.-Based Syst. 2025, 330, 114526. [Google Scholar] [CrossRef] [Scilit]
- Xu, J.; Xu, Z.; Jiang, Z.; Chen, Z.; Luo, H.; Wang, Y.; Gui, W. Labeling-free RAG-enhanced LLM for intelligent fault diagnosis via reinforcement learning. Adv. Eng. Inform. 2026, 69, 103864. [Google Scholar] [CrossRef] [Scilit]
- Nguyen, N.; Tan, W.; Buntine, W.; Beare, R.; Chen, C.; Du, L. AUC Maximization for Low-Resource Named Entity Recognition. arXiv 2023, arXiv:2212.04800. [Google Scholar] [CrossRef] [Scilit]
- Gu, J.; Yuan, J.; Cai, J.; Zhou, X.; Fan, L. La-LoRA: Parameter-efficient fine-tuning with layer-wise adaptive low-rank adaptation. Neural Netw. 2026, 194, 108095. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Lin, T.Y.; Goyal, P.; Girshick, R.; He, K.; Dollár, P. Focal loss for dense object detection. In Proceedings of the IEEE Transactions on Pattern Analysis and Machine Intelligence; IEEE: Piscataway, NJ, USA, 2020; Volume 42, pp. 318–327. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Zhang, X.; Fang, Z.; Wen, Y.; Li, Z.; Qiao, Y. Range loss for deep face recognition with long-tailed training data. In Proceedings of the IEEE International Conference on Computer Vision (ICCV); IEEE: Piscataway, NJ, USA, 2017; pp. 5419–5428. [Google Scholar]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.





