Abstract
With the continuous improvement of power system intelligence, multimodal data generated during distribution network maintenance have grown exponentially. However, existing power multimodal datasets commonly suffer from issues such as low sample quality, frequent factual errors, and inconsistent instruction expressions caused by regional differences.Traditional sample correction methods mainly rely on manual screening or single-feature matching, which suffer from low efficiency and limited adaptability. This paper proposes a multimodal sample correction framework based on large-model instruction enhancement and knowledge guidance, focusing on two critical modalities: temporal data and text documentation. Multimodal sample correction refers to the task of identifying and rectifying errors, inconsistencies, or quality issues in datasets containing multiple data types (temporal sequences and text), with the objective of producing corrected samples that maintain factual accuracy, temporal consistency, and domain-specific compliance. Our proposed framework employs a three-stage processing approach: first, temporal Bidirectional Encoder Representations from Transformers (BERT) models and text BERT models are used to extract and fuse device temporal features and text features, respectively; second, a knowledge-injected assessment mechanism integrated with power knowledge graphs and DeepSeek’s long-chain-of-thought (CoT) capabilities is designed to achieve precise assessment of sample credibility; third, beam search algorithms are employed to generate high-quality corrected text, significantly improving the quality and reliability of multimodal samples in power professional scenarios. Experimental results demonstrate that our method significantly outperforms baseline models across all evaluation metrics (BLEU: 0.361, ROUGE: 0.521, METEOR: 0.443, F1-Score: 0.796), achieving improvements ranging from 21.1% to 73.0% over state-of-the-art methods: specifically, a 21.1% improvement over GECToR in BLEU, 26.5% over GECToR in ROUGE, 30.3% over Deep Edit in METEOR, and 11.8% over Deep Edit in F1-Score, with a reduction of approximately 35% in hallucination rates compared to existing approaches. These improvements provide important technical support for intelligent operation and maintenance of power systems, with implications for improving data quality management, enhancing model reliability in safety-critical applications, and enabling scalable knowledge-guided correction frameworks transferable to other industrial domains requiring high data integrity.
1. Introduction
With the deepening advancement of new power system construction, distribution network maintenance work is undergoing a fundamental transformation from traditional manual inspection to intelligent and automated maintenance [1,2,3]. Modern power systems continuously generate massive volumes of multimodal data through high-resolution sensors, unmanned aerial vehicle (UAV) inspections, mobile maintenance robots, and other equipment, including text documentation, time series signals, and other forms. The advancement of deep learning techniques has significantly enhanced data processing capabilities across various domains. In vision and language applications, recent developments include multimodal fusion networks [4] for cross-modal understanding, dual-stream architectures [5] for visual–linguistic alignment, scene understanding frameworks [6], and text-guided person re-identification methods [7]. Additionally, learning-based approaches have shown promising results in detection tasks [8], knowledge-driven reasoning [9], topological structure learning [10], and intelligent system optimization [11]. However, these general-purpose methods often lack domain-specific adaptations necessary for power system applications. These data provide critical information support for equipment condition analysis, fault detection, and defect identification. However, the explosive growth in data volume also brings unprecedented challenges: data quality issues.
In this context, we formally define the multimodal sample correction task as follows: Given a multimodal input sample consisting of temporal sequences and textual documentation , the objective is to produce a corrected output that (1) maintains factual accuracy with respect to domain knowledge constraints , (2) ensures temporal consistency between time-series patterns and textual descriptions, (3) adheres to professional standards and regulatory requirements specific to power systems, and (4) eliminates hallucinations, semantic inconsistencies, and quality defects present in the original sample. The expected output includes corrected text with minimal edit distance while preserving semantic integrity, accompanied by credibility scores indicating the confidence level of corrections.
Distribution network maintenance data exhibit obvious regional characteristics with varying management requirements and instruction language expressions across different cities and regions. Traditional large models [12,13] cannot adequately meet regional maintenance needs. During data processing procedures such as OCR recognition and sample augmentation, substantial inaccurate or incomplete data are generated, leading to decreased model generalization ability. Real-time fluctuations in power system operating states further cause static data to become disconnected from actual states, exacerbating data timeliness issues.
Existing multimodal sample correction methods mainly fall into three categories: traditional rule-based correction methods, statistical learning-based correction methods, and deep learning-based correction methods [14,15]. Traditional methods offer fast computation but perform poorly when handling complex backgrounds, lighting variations, and geometric structures typical of power equipment. Statistical learning methods improve efficiency and robustness to some extent but often fail to distinguish deeper semantic differences. Deep learning models provide stronger representational capacity but require substantial computational resources and labeled datasets and still lack adequate semantic understanding of power-specific content. More critically, existing methods often fail to effectively distinguish between actual equipment state changes and superficial imaging differences, limiting their deployment in real-world power scenarios.
To address these limitations, this paper proposes a multimodal sample correction framework based on large-model instruction enhancement and knowledge guidance. The framework deeply integrates perceptual and semantic features, adopting a three-stage processing pipeline: “multimodal feature extraction and fusion → knowledge-injected assessment → sample correction,” as illustrated in Figure 1. First, temporal BERT models and text BERT models are used to extract device temporal features and text features, respectively, followed by cross-modal attention mechanisms to achieve effective fusion of features from different modalities. Second, a knowledge graph-enhanced reasoning mechanism integrated with DeepSeek’s long-chain-of-thought capabilities is designed to achieve precise assessment of sample credibility. Third, beam search algorithms are employed to generate high-quality corrected text, ensuring accuracy and consistency of correction results.
Figure 1.
Overview of the three-stage multimodal sample correction methodology. The framework consists of multimodal feature extraction and fusion (temporal and text feature extraction with cross-modal attention), knowledge-injected assessment (knowledge graph construction, chain-of-thought reasoning, and credibility assessment), and sample correction (strategy selection and beam search repair).
The main contributions of this paper include the following: (1) designing a comprehensive multimodal feature extraction framework that integrates temporal attention mechanisms and multi-head self-attention for text processing, enabling effective handling of complex power system data, achieving F1-Score improvements of 5.1 percentage points over single-modality approaches (from 0.745 to 0.796); (2) proposing a knowledge graph-enhanced reasoning system that combines entity and relation embeddings with attention-weighted knowledge retrieval, achieving precise sample credibility assessment through weighted aggregation of multiple scoring components, with knowledge graph integration contributing 3.5 percentage points improvement in F1-Score and reducing factual errors by approximately 35%; (3) developing an adaptive correction strategy selection mechanism with gating networks and beam search algorithms that incorporate knowledge graph constraints, providing targeted solutions for different types of data quality issues in power professional scenarios, demonstrating 21.1–73.0% improvements across different evaluation metrics (BLEU, ROUGE, METEOR, F1-Score) compared to state-of-the-art baseline methods.
2. Related Work
2.1. Multimodal Feature Extraction
Multimodal feature extraction is a fundamental technology for achieving cross-modal information fusion, aiming to extract semantically consistent feature representations from data of different modalities [16]. In power system applications, multimodal data mainly include two types: text documentation and time series signals. The challenge lies in bridging the semantic gap between different modalities while preserving the unique characteristics of each data type.
For text modality, early methods mainly relied on statistical features such as Bagof-Words models and TF-IDF. These approaches, while computationally efficient, fail to capture semantic relationships and contextual dependencies crucial for understanding technical documentation. Word2Vec [17] embeddings provided initial improvements by capturing semantic similarities but lacked the contextual understanding necessary for professional domains.
With the development of pre-trained language models, models like BERT [13] and GPT [12] have learned rich language representations through large-scale unsupervised pre-training. BERT’s bidirectional encoding mechanism enables better understanding of context, while GPT’s autoregressive approach excels in text generation tasks. Recent journal publications have advanced domain-specific language modeling: Xu et al. [1] reviewed multimodal learning with transformers, highlighting the effectiveness of such models for information extraction; These methods primarily focus on single-modal text processing and lack mechanisms for integrating temporal dynamics or structured knowledge, limiting their effectiveness in complex maintenance scenarios. Particularly in the power professional domain, targeted pre-trained models such as ELECTRA [18] can better understand power professional terminology and semantic relationships. Domain-specific adaptations like PowerBERT and GridBERT have been developed to handle technical jargon, regulatory language, and operational procedures specific to power systems.
For time series modality, traditional methods mainly rely on statistical features and frequency domain features such as Fourier transforms, wavelet analysis, and autoregressive models. These methods work well for stationary signals but struggle with the nonstationary, multi-scale nature of power system data. Recent advances in load forecasting [19] and graph-based behavior identification [20] have demonstrated the importance of advanced temporal modeling techniques in power systems. In recent years, sequence models such as Recurrent Neural Networks (RNNs), Long Short-Term Memory networks (LSTMs) [21], and Gated Recurrent Units (GRUs) [22] have shown excellent performance in time series feature extraction. The introduction of the Transformer architecture [23] has further improved long-sequence modeling capabilities, with temporal Transformer models achieving good results in power load forecasting and equipment condition monitoring. Recent journal studies have explored temporal feature extraction for power systems: Wang et al. [24] proposed Temporal-Attention Networks achieving 94.2% accuracy in load forecasting but limited to single-device scenarios without cross-device generalization capability; Kumar et al. [25] developed Hierarchical Temporal Convolutions reaching 91.5% fault detection accuracy yet requiring manual feature engineering for each equipment type, reducing scalability; Zhou et al. [26] introduced Multi-scale Temporal Transformers demonstrating strong performance (RMSE = 0.087) on stationary signals but degrading significantly (RMSE = 0.243) under non-stationary conditions and topology changes. These approaches lack integration with textual documentation and domain knowledge, resulting in reduced interpretability and higher false positive rates (15–23%) in real-world deployments. Recent advances in attention-based temporal models have enabled better handling of long-range dependencies and irregular sampling patterns common in power system monitoring data.
Research Limitations: Despite these advances, existing multimodal feature extraction methods exhibit several critical limitations: (1) Inadequate cross-modal fusion: current approaches typically process each modality independently and employ simple concatenation or addition for fusion, failing to capture complex semantic correlations between temporal patterns and textual descriptions; (2) Lack of domain adaptation: general-purpose pre-trained models require extensive fine-tuning (480+ GPU-hours) and large annotated datasets (100K+ samples) for power system applications, limiting practical deployment; (3) Limited interpretability: deep learning models provide little insight into which features drive predictions, hindering trust and adoption in safety-critical power grid operations; (4) Temporal-text inconsistency: existing methods cannot effectively detect and correct misalignments between time series anomalies and their textual descriptions, leading to propagation of errors in downstream applications.
2.2. Knowledge Graph and Large-Model Integration
Knowledge graphs, as an important form of structured knowledge representation, can effectively organize and manage professional domain knowledge [27]. In power systems, knowledge graphs typically contain multi-level information including equipment entities, attribute relationships, operational rules, regulatory requirements, and historical maintenance records. The complexity of power system knowledge, spanning multiple engineering disciplines and regulatory frameworks, makes knowledge graph construction particularly challenging.
Traditional knowledge graph construction methods mainly rely on expert knowledge and rule definitions, suffering from high construction costs and update difficulties. Manual curation approaches, while ensuring high quality, are time-consuming and may not scale to the rapidly evolving nature of power system technologies. Semi-automated approaches using natural language processing and information extraction techniques have shown promise but require significant domain expertise for validation and refinement.
The integration of large models and knowledge graphs is a current research hotspot with significant implications for power system applications [28,29]. On one hand, knowledge graphs can provide structured prior knowledge for large models, enhancing model reasoning capabilities in specific domains and reducing hallucination problems common in general-purpose models. On the other hand, the powerful language understanding capabilities of large models can assist in automatic construction and updating of knowledge graphs, enabling more dynamic and comprehensive knowledge representation.
Recent journal publications have investigated knowledge graph integration for industrial systems: Yao et al. [30] proposed KG-BERT to enhance knowledge graph completion by treating triples as text sequences, achieving high accuracy in link prediction tasks; Anderson et al. [31] introduced attention-based embeddings for relation prediction, capturing complex dependencies in local neighborhoods. These methods focus primarily on knowledge retrieval rather than dynamic integration within reasoning processes, resulting in limited adaptability to evolving operational requirements and regional standards.
Existing integration methods mainly include knowledge-enhanced pre-training, knowledge-guided fine-tuning, and knowledge graph embedding [32]. Knowledgeenhanced pre-training introduces knowledge graph information during the pre-training stage, enabling models to learn richer knowledge representations. Models like Know-BERT [29] and ERNIE [28] adopt this approach, achieving significant improvements in multiple natural language processing tasks. These models demonstrate improved performance in tasks requiring factual knowledge and logical reasoning.
Knowledge-guided fine-tuning introduces knowledge constraints during the finetuning stage of downstream tasks, ensuring model outputs conform to domain knowledge. This approach is particularly valuable in safety-critical applications like power systems, where adherence to established procedures and regulations is paramount. Knowledge graph embedding methods map entities and relationships in knowledge graphs to continuous vector spaces, facilitating integration with neural network models and enabling efficient similarity computation and reasoning operations.
Recent developments in neuro-symbolic approaches have shown promise in combining the flexibility of neural networks with the interpretability and reliability of symbolic knowledge representation. These hybrid approaches are particularly relevant for power system applications where explainability and compliance with established standards are crucial requirements.
Research Limitations: Current knowledge graph integration methods face significant challenges: (1) Static knowledge representation: existing knowledge graphs require manual ontology updates every 3–6 months, failing to capture the dynamic nature of power system operations where equipment states, regulations, and operational procedures evolve continuously; (2) Knowledge incompleteness: industrial knowledge graphs typically suffer from 31% average missing relations and sparse entity coverage, leading to reasoning gaps and incorrect inferences in critical decision-making scenarios; (3) Limited scalability: knowledge graph embedding methods show degraded performance beyond 50K entities, restricting their application to large-scale power grid networks with hundreds of thousands of equipment units; (4) Weak temporal reasoning: current approaches lack mechanisms for incorporating temporal validity of knowledge triplets, treating all information as equally current regardless of when it was verified, resulting in outdated recommendations in rapidly changing operational contexts.
2.3. Chain-of-Thought Reasoning
Chain-of-thought (CoT) reasoning represents an important breakthrough in largemodel reasoning capability research in recent years [33]. By guiding models to generate intermediate reasoning steps, chain-of-thought reasoning can significantly improve model performance in complex reasoning tasks. This approach is particularly valuable in technical domains like power systems, where decision-making processes must be transparent and auditable.
Traditional chain-of-thought reasoning mainly relies on manually designed reasoning templates, guiding models to generate reasoning chains through few-shot learning. However, this approach often struggles to generate high-quality reasoning steps when facing complex professional domain problems that require deep technical knowledge and multi-step logical inference. The static nature of template-based approaches limits their adaptability to novel scenarios and evolving technical requirements.
To address these limitations, researchers have proposed various improvement methods, including self-consistency reasoning, tree-based reasoning, and program-assisted reasoning [34]. Self-consistency approaches generate multiple reasoning paths and select the most consistent solution, improving reliability in uncertain scenarios. Tree-based reasoning enables exploration of multiple solution branches, particularly valuable for troubleshooting and diagnostic tasks. Program-assisted reasoning combines natural language reasoning with executable code, enabling precise calculations and logical operations. Recent journal research has advanced reasoning mechanisms: Madaan et al. [35] proposed Self-Refine, allowing models to iteratively improve their outputs through self-feedback, though this process can be computationally intensive; Zhang et al. [36] modeled hierarchical reasoning chains by linking discourse units, enhancing the interpretation of complex reading comprehension tasks; Wang et al. [37] explored self-consistency in chain-of-thought prompting, showing that sampling diverse reasoning paths significantly boosts performance on complex reasoning benchmarks. These approaches do not adequately address error propagation mitigation or logical consistency verification across extended reasoning sequences. Advanced techniques such as retrieval-augmented reasoning and tool-augmented reasoning have further enhanced the capabilities of chain-of-thought approaches [38]. Retrievalaugmented reasoning enables models to access relevant external knowledge during the reasoning process, while tool-augmented reasoning allows models to invoke specialized computational tools and databases.
Research Limitations: Existing chain-of-thought reasoning methods encounter substantial obstacles in practical deployment: (1) Computational inefficiency: advanced CoT methods incur 3.2× computational overhead compared to standard inference, making realtime applications infeasible for time-critical power grid operations; (2) Error propagation: incorrect reasoning steps in early stages cascade through the reasoning chain, with studies showing 18% hallucination rates in specialized technical domains where a single error can invalidate the entire reasoning process; (3) Lack of consistency verification: current approaches do not adequately verify logical consistency across extended reasoning sequences, leading to contradictory conclusions within the same reasoning chain (variance = 0.18 across problem types); (4) Limited domain integration: existing methods struggle to incorporate domain-specific constraints and professional knowledge systematically, requiring extensive manual prompt engineering (450+ tokens per task) and showing brittleness to terminology variations (12% accuracy drop with 5% input perturbation).
3. Methodology
3.1. Overall Framework
The proposed multimodal sample correction framework based on large-model instruction enhancement and knowledge guidance is illustrated in Figure 2. The framework adopts a three-stage processing pipeline: multimodal feature extraction and fusion, knowledge-injected assessment, and sample correction. This architectural design addresses the key challenges identified in existing methods: inadequate cross-modal fusion, lack of domain-specific knowledge integration, and insufficient reasoning verification mechanisms. The framework processes power system maintenance data containing both temporal sensor readings and textual documentation, producing corrected outputs that maintain factual accuracy, temporal consistency, and regulatory compliance.
Figure 2.
Detailed framework architecture of the multimodal sample correction system. The system integrates temporal BERT and text BERT models for feature extraction, employs knowledge graphs and chain-of-thought reasoning for assessment, and utilizes beam search algorithms for sample correction, demonstrating the complete workflow from input processing to corrected output generation.
As shown in Figure 2, the framework operates through a systematic workflow that processes multimodal inputs and generates quality-assured corrected outputs. We now elaborate on each major component:
Stage 1: Multimodal Feature Extraction and Fusion. The input layer receives two complementary data streams: (i) temporal sequences from power equipment sensors capturing operational parameters such as voltage, current, temperature, and load patterns over time; (ii) textual documentation comprising maintenance reports, operation logs, and equipment specifications. These heterogeneous modalities are processed through specialized encoder architectures. The temporal BERT encoder applies multi-head self-attention mechanisms to capture temporal dependencies and equipment-specific operational patterns, incorporating positional encodings that preserve chronological information. Simultaneously, the text BERT encoder processes linguistic content through bidirectional attention, extracting semantic features from technical terminology, operational procedures, and maintenance instructions. The extracted representations and are then integrated through a cross-modal attention module, which computes attention weights measuring the relevance between temporal patterns and textual descriptions. This fusion mechanism generates a unified multimodal representation that encapsulates both temporal dynamics and semantic content, enabling downstream modules to reason about temporal–text consistency.
Stage 2: Knowledge-Injected Assessment. The fused multimodal features enter the assessment module, which evaluates sample credibility through knowledge-guided reasoning. This stage leverages a power system knowledge graph containing 15,432 equipment entities, 87 relation types, and 38,765 validated triplets encoding domain expertise including equipment specifications, operational constraints, regulatory requirements, and historical maintenance patterns. The knowledge graph provides structured domain knowledge through entity embeddings and relation embeddings learned via TransE, augmented with temporal decay factors to prioritize recently validated information. The DeepSeek-based long-chain-of-thought reasoning module generates intermediate reasoning steps, with each step constrained by knowledge graph triplets to prevent hallucinations and ensure domain compliance. Unlike standard CoT approaches using fixed-length reasoning chains, our method dynamically adjusts reasoning depth based on sample complexity , allocating more reasoning steps to samples with detected inconsistencies. A consistency verification mechanism validates each reasoning step against prior steps and knowledge constraints through scoring function , triggering self-correction via beam search when consistency falls below threshold . The reasoning outputs feed into a hierarchical credibility assessment module that computes four complementary scores: (1) temporal consistency score measuring alignment between time series patterns and textual descriptions, (2) knowledge conformity score evaluating adherence to domain constraints, (3) semantic coherence score assessing logical consistency, and (4) regional compliance score verifying conformity to local regulations. These scores are aggregated through learned weights optimized during training, producing a final credibility score indicating sample quality.
Stage 3: Sample Correction. Based on the credibility assessment and identified error patterns, an adaptive strategy selection mechanism determines the appropriate correction approach. A gating network processes the enhanced features , credibility score, and context features (including error type distribution, sample source, and regional requirements) to select among three strategies: (i) minor editing for samples with localized errors, (ii) major restructuring for samples with structural inconsistencies, (iii) complete regeneration for samples with pervasive quality issues. The selected strategy guides a knowledge-constrained beam search decoder that generates corrected text. At each decoding step t, the decoder maintains candidate sequences, computing generation probability combined with knowledge graph conformity score weighted by . A knowledge-constrained pruning mechanism eliminates candidates violating hard constraints (e.g., incompatible equipment-operation pairs) at each step, reducing search space by 40% while preserving quality. The beam search incorporates length penalty to balance output length and quality. The final output consists of corrected text accompanied by credibility scores and identified error types, providing transparency for downstream quality control.
This three-stage architecture enables end-to-end processing of multimodal power system data, addressing the limitations of existing methods through explicit knowledge integration, dynamic reasoning depth control, and adaptive correction strategies. The modular design facilitates component-wise optimization and allows flexible adaptation to different power grid operational scenarios and regional regulatory requirements.
3.2. Multimodal Feature Extraction and Fusion
3.2.1. Temporal Feature Extraction
For temporal data of power equipment, we employ temporal BERT models for feature extraction. Consider a temporal data sequence , where represents the feature vector at the ith time step, T denotes the sequence length, and d is the feature dimension (set to 768 in our implementation).
Following the standard Transformer attention mechanism [23], the temporal attention mechanism computes attention weights for each time step:
where and are query and key vectors at positions i and j, respectively, and is the key dimension (set to 64 in our experiments). The temporal feature representation is computed as:
where is the value vector at position j, and is the positional encoding for the ith time step, computed using sinusoidal functions as in [23]. Novel contribution: Unlike standard temporal transformers, we introduce device-specific positional encodings that incorporate equipment-type information through learnable embeddings , modifying Equation (2) to , enabling better capture of device-specific temporal patterns as shown in Figure 2 (Temporal Feature Extraction module).
3.2.2. Text Feature Extraction
For power maintenance-related text data, we employ BERT models specifically pre-trained for the power domain. Consider an input text sequence , where represents the ith token, and N is the text sequence length (maximum 512 tokens in our implementation).
Following the BERT architecture [13], the multi-head self-attention mechanism computes attention scores as:
where are query, key, and value matrices, respectively. The multi-head attention combines multiple attention heads:
where , is the number of attention heads, are projection matrices, and is the output projection matrix, and the final text representation incorporates layer normalization and residual connections:
where l indexes the transformer layer (total 12 layers), and LayerNorm denotes layer normalization [23]. Novel contribution: We extend the text BERT with domain-specific token type embeddings that distinguish between equipment names, operation commands, and descriptive text, enhancing the model’s ability to capture power system documentation structure as illustrated in Figure 2 (Text Feature Extraction module).
3.2.3. Multimodal Feature Fusion
To effectively fuse features from different modalities, we designed a cross-modal attention-based fusion mechanism. This mechanism can adaptively learn correlation relationships between different modalities, generating unified multimodal representations.
3.2.4. Cross-Modal Attention Mechanism
Given feature representations from two modalities and , where and , we first map them to a unified feature space through linear projections:
where are learnable projection weight matrices, and are bias vectors. Then, following standard practice [23], cross-modal attention weights are computed:
where represents the attention weights between temporal and text features. This component is illustrated in the Cross-Modal Attention module of Figure 2.
3.2.5. Feature Fusion
Based on the computed attention weights, feature fusion is performed:
where and are learnable fusion weight parameters, initialized to 0.5 and optimized during training. Novel contribution: We introduce adaptive fusion weights computed via a gating network , where and , allowing the model to dynamically balance modality contributions based on input quality, as depicted in the Feature Fusion module of Figure 2.
3.3. Knowledge-Injected Assessment
3.3.1. Knowledge Graph Construction
Based on professional knowledge in the power maintenance domain, we construct a knowledge graph containing equipment entities, attribute relationships, and operational rules, where are head and tail entities from the entity set (15,432 entities in total), and is a relation from the relation set (87 relation types). Following TransE [32], entity embeddings are learned through:
where is the embedding for entity i, is the entity embedding matrix, is the bias vector, and is the knowledge graph embedding dimension. Relation embeddings capture semantic relationships:
where is the embedding for relation j, and is the relation embedding matrix. The knowledge graph scoring function measures triplet validity:
where lower scores indicate higher validity. Novel contribution: We extend the standard TransE model with temporal decay factors , where is the time elapsed since the triplet was last verified, and is a decay rate, allowing the knowledge graph to prioritize recently validated information. This enhancement addresses the dynamic nature of power system operations as shown in the Knowledge Graph module of Figure 2.
3.3.2. Long-Chain-of-Thought Reasoning Based on DeepSeek
Unlike standard chain-of-thought (CoT) reasoning that relies on fixed-length reasoning chains and static prompt templates, our DeepSeek-based long-CoT mechanism introduces three key innovations: (1) Dynamic reasoning length control: the reasoning chain length L is adaptively determined based on sample complexity through a learned gating function , where is a scaling parameter, and quantifies the number of conflicting knowledge triplets, temporal anomalies, and semantic inconsistencies in the input sample; (2) Logical consistency maintenance: at each reasoning step t, a consistency checker validates the generated reasoning token against previously generated tokens and knowledge graph constraints through a scoring function , where represents relevant knowledge embeddings, and is the hidden state at step t; (3) Error propagation mitigation: a self-correction mechanism is triggered when consistency scores fall below threshold , allowing the model to backtrack and regenerate alternative reasoning paths using beam search with width .
The reasoning process incorporates attention-weighted knowledge retrieval. Given query features q, relevant knowledge is retrieved through:
To ensure knowledge nodes are effectively constrained within the reasoning chain, we implement a structured integration mechanism. Each knowledge triplet is represented as , where embeddings are learned through Equations (10) and (11). During reasoning step t, knowledge constraints are enforced through a gating mechanism:
which determines the influence of each knowledge node on the current reasoning step. The knowledge-constrained reasoning state is then computed as:
where ⊙ denotes element-wise multiplication. This mechanism ensures that knowledge graph constraints are directly integrated into each reasoning step, preventing the generation of reasoning chains that violate domain knowledge.
The knowledge-enhanced representation is computed as:
where and are key and value representations of knowledge triplets, and the summation aggregates multiple relevant knowledge nodes weighted by their attention scores , enabling soft constraint enforcement that balances data-driven features with structured knowledge.
3.3.3. Sample Credibility Assessment
Credibility assessment combines multiple scoring components through weighted aggregation:
where are different scoring functions, including (1) temporal consistency score measuring alignment between temporal patterns and text descriptions, (2) knowledge conformity score evaluating adherence to knowledge graph constraints, (3) semantic coherence score assessing internal logical consistency, and (4) regional compliance score checking conformity to local regulations ( total scoring functions), where are learned weights, optimized via gradient descent with constraints and , and is the sigmoid function. Novel contribution: We introduce a hierarchical credibility assessment that first evaluates individual modalities separately then combines them with cross-modal consistency checks, as illustrated in the Credibility Assessment module of Figure 2, enabling fine-grained identification of quality issues across different data aspects.
3.4. Sample Correction
3.4.1. Correction Strategy Selection
The correction strategy is determined by a gating mechanism that considers multiple factors:
where represents context features for strategy i, including error type distribution, sample source, and regional requirements, , where K is the number of correction strategies (we use : minor editing, major restructuring, and complete regeneration), is the context feature dimension (set to 128), and the concatenation produces a combined feature vector. The final strategy selection is:
Novel contribution: Unlike fixed correction strategies, our adaptive selection mechanism dynamically chooses between different correction granularities based on error severity and type, as shown in the Strategy Selection module of Figure 2.
3.4.2. Beam Search Repair
The beam search process maintains top-B candidate sequences, where in our implementation. At each step t, the probability of generating token is:
where is the output projection matrix, is the vocabulary size (50,000 tokens), is the hidden state at step t, and is the output bias. Following standard sequence-to-sequence models [23], the beam search score combines generation probability with knowledge constraints:
where measures consistency with knowledge graph constraints, and balances the two components, set to 0.3 based on validation set performance. Novel contribution: We enhance standard beam search with knowledge-constrained pruning that eliminates candidate sequences violating hard constraints from the knowledge graph (e.g., incompatible equipment–operation pairs) at each decoding step, reducing the effective search space by 40% while maintaining output quality. This is implemented in the Beam Search Repair module shown in Figure 2.
4. Experiments
4.1. Dataset Construction
To comprehensively evaluate the effectiveness of the proposed method, we constructed a specialized multimodal sample correction dataset for the power domain. The dataset primarily focused on core business scenarios such as distribution network dispatching maintenance applications and dispatching operation tickets, covering the most critical document types in power system operation and maintenance. The construction of the dataset fully considered the professional characteristics and regional differences of the power industry, ensuring the practicality and reliability of experimental results.
The dataset consisted of two main components: First, the safety monitoring internal Q&A data, constructed based on the power safety examination question bank, containing 7946 professional question–answer pairs covering core knowledge areas such as work ticket management, safety operation procedures, and equipment maintenance standards. These data originated from actual safety training and assessment scenarios, demonstrating strong practicality. Second, the distribution network dispatching Q&A data, constructed based on actual dispatching business in Xuancheng and Ma’anshan regions of Anhui Province, including distribution network dispatching maintenance application Q&A pairs and 1498 annotated samples for generating dispatching operation tickets from maintenance applications.
To enhance the model’s professional knowledge understanding capability, we also constructed an external knowledge base using 500 complete-version-annotated samples as knowledge support. These samples were carefully annotated by professional personnel, covering various typical scenarios and operational procedures of distribution network dispatching, providing rich professional knowledge background for the Q&A system and ensuring the accuracy and professionalism of answers.
4.2. Experimental Setup and Evaluation Metrics
This study adopted a multi-dimensional evaluation system to comprehensively measure model performance. Considering the characteristics of text correction tasks, we selected widely recognized evaluation metrics in the industry, including BLEU, ROUGE, METEOR, and F1-Score, which can evaluate the quality and authenticity of text correction from different perspectives. The BLEU score mainly evaluates the n-gram overlap between generated text and reference text, the ROUGE score focuses on evaluating text summary quality and content coverage, the METEOR score considers text similarity of synonyms and stems, while the F1-Score comprehensively considers the harmonic mean of precision and recall.
Implementation Details and Reproducibility: All experiments were conducted on a computing cluster equipped with four NVIDIA A100 GPUs (80 GB memory each), Intel Xeon Platinum 8358 CPUs (2.60 GHz, 64 cores), and 512 GB of RAM. We used PyTorch 2.0.1 with CUDA 11.8 for the implementation. To ensure reproducibility, we set random seeds across all libraries: Python random seed = 42, NumPy random seed = 42, PyTorch manual seed = 42, and CUDA manual seed = 42. The deterministic mode was enabled in PyTorch (torch.backends.cudnn.deterministic = True, torch.backends.cudnn.benchmark = False). Our temporal BERT model was initialized with 12 transformer layers, a hidden size of 768, and 12 attention heads. The text BERT model employed the bert-base-chinese pre-trained weights. Training used AdamW optimizer with a learning rate of 2 × 10−5, batch size of 16, and gradient accumulation steps of 4. The knowledge graph contained 15,432 entities and 38,765 relation triplets. All models were trained for 50 epochs with early stopping (patience = 5). The beam search width was set to B = 5 with length penalty .
The experiment adopted a rigorous controlled design, selecting multiple representative baseline models for comparison, including T5 [39], BART [40], GPT-3 [12], Deep Edit [15], and GECToR [14] public models. These models represent the mainstream technical routes in the current text correction field, providing comprehensive performance benchmarks for this study. For fair comparison, all baseline models were fine-tuned on our dataset using their recommended hyperparameters with consistent early stopping criteria. The experiment conducted 10 independent validation experiments on datasets such as dispatching maintenance applications and dispatching operation tickets, ensuring statistical significance and reproducibility of results. Statistical significance was assessed using paired t-tests with a threshold, and we report mean ± standard deviation across all runs with 95% confidence intervals computed using bootstrap resampling (10,000 iterations).
4.3. Experimental Results and Analysis
Through large-scale comparative experiments conducted on dispatching maintenance application and dispatching operation ticket datasets, our method achieved significant performance improvements on all evaluation metrics. As shown in Table 1, our method significantly outperformed other comparison models on the four key metrics of BLEU, ROUGE, METEOR, and F1-Score.
Table 1.
Comparison of different methods on a multimodal sample correction task. Results shown as mean ± std across 10 runs. * indicates statistical significance (p < 0.05) compared to the best baseline (GECToR). The 95% confidence intervals for our method were as follows: BLEU [0.352, 0.369], ROUGE [0.509, 0.532], METEOR [0.436, 0.451], F1 [0.788, 0.804].
From the experimental results, we can see that compared to the best baseline model GECToR, our method improved by 6.3 percentage points on the BLEU score, 10.9 percentage points on the ROUGE score, 2.8 percentage points on the METEOR score, and 1.2 percentage points on the F1-Score. These improvements fully demonstrate the significant advantages of our method in text correction quality and authenticity.
Particularly noteworthy is that our method achieves the most significant improvement on the ROUGE score, indicating that our method has obvious advantages in maintaining text content integrity and coverage. This is mainly attributed to the multimodal feature fusion mechanism and knowledge graph-guided chain-of-thought reasoning, enabling the model to better understand the semantic structure and professional content of text, thereby generating more accurate and complete correction results.
4.4. Ablation Studies and Component Analysis
To deeply understand the contribution of each component to overall performance, we conducted comprehensive ablation experiments by systematically removing or modifying individual components of our framework. These experiments provide crucial insights into the relative importance of different architectural choices and help validate our design decisions.
The ablation study results are presented in Table 2, which shows the performance impact of removing various components from our complete framework. The experimental results demonstrate that multimodal feature fusion is a key factor for performance improvement, with each modality contributing significantly to the overall system performance.
Table 2.
Ablation study results showing the contribution of different components. Results shown as mean ± std across 10 runs. F1 indicates absolute percentage point difference from complete framework. All differences are statistically significant ().
When using only text features, the model’s F1-Score was 0.745, demonstrating the baseline performance of single-modal approaches. Using only temporal features yielded an F1-Score of 0.728, showing that temporal information alone provides valuable but limited insights. The combination of text and temporal features achieved an F1-Score of 0.782, with an increase of 3.7 percentage points compared to text-only configuration, highlighting the importance of temporal information in understanding power system operations.
The complete multimodal fusion scheme achieved an F1-Score of 0.796, improving by 5.1 percentage points compared to single text features and 6.8 percentage points compared to single temporal features. This significant improvement demonstrates the synergistic effects of combining text and temporal modalities, validating our hypothesis that multimodal approaches are essential for comprehensive sample correction in power systems.
The chain-of-thought reasoning mechanism also played a crucial role in system performance. Without chain-of-thought reasoning, the model’s comprehensive performance decreased by approximately 2.8 percentage points, indicating that DeepSeek-based longchain-of-thought reasoning can effectively improve the model’s logical reasoning ability and professional knowledge understanding depth. This component is particularly valuable for handling complex technical scenarios that require multi-step reasoning.
The introduction of knowledge graphs brought significant performance improvements, with a 3.5 percentage point decrease in F1-Score when removed. This component particularly excels in handling professional terminology and domain-specific knowledge, ensuring that corrections adhere to established power system standards and practices.
The cross-modal attention mechanism showed the largest individual impact, with its removal causing a 4.2 percentage point decrease in F1-Score. This result validates our architectural choice of sophisticated attention-based fusion over simpler concatenation approaches, demonstrating that learning complex interaction patterns between modalities is crucial for optimal performance.
Finally, the beam search repair mechanism contributed 2.3 percentage points to the overall performance, showing that sophisticated text generation approaches are necessary for producing high-quality corrections that maintain both linguistic fluency and technical accuracy.
4.5. Fine-Tuning Analysis and Model Adaptation
To comprehensively evaluate the effectiveness of fine-tuning on domain-specific data, we conducted systematic experiments comparing pre-trained baseline models with their fine-tuned counterparts. Table 3 presents the performance comparison before and after fine-tuning on our power system dataset.
Table 3.
Performance comparison of baseline models before and after fine-tuning on power system dataset. Results shown as mean ± std across 10 runs. represents absolute improvement from pre-trained to fine-tuned models.
The results demonstrate that fine-tuning significantly improved model performance across all metrics. For the GECToR baseline model, fine-tuning yielded improvements of 6.3 percentage points in the BLEU score, 6.4 percentage points in the ROUGE score, 7.4 percentage points in the METEOR score, and 6.3 percentage points in the F1-Score. Notably, our method showed comparable or superior improvement margins (7.2, 7.8, 6.5, and 6.5 percentage points, respectively), validating that the performance gains were not solely attributable to fine-tuning but resulted from the architectural innovations including multimodal fusion, knowledge graph integration, and chain-of-thought reasoning.
4.6. Case Analysis and Application Effects
Through specific case analysis, we can more intuitively see the actual effects of our method. When processing distribution network dispatching maintenance applications, traditional methods often struggle to accurately identify and correct the incorrect use of professional terminology, while our method, through integrating knowledge graphs and chain-of-thought reasoning, can accurately identify the normative differences between “10 kV distribution line” and “10 kilovolt distribution line” and provide correct correction suggestions.
In the processing of dispatching operation tickets, our method demonstrates strong regional adaptability. For dispatching terminology and operational procedures specific to Xuancheng region in Anhui, the model can provide correction results that comply with local regulations based on regionalized knowledge provided by the external knowledge base. This regional adaptability is difficult for traditional general models to achieve, fully reflecting the value of our method in practical applications.
Experimental results also show that our method is significantly effective in reducing hallucination phenomena in multimodal large models. Through introducing knowledge graph constraints and chain-of-thought reasoning verification, the correction text generated by the model significantly improves in factual accuracy, with error rates reduced by approximately 35% compared to baseline methods. This is of great significance for application domains like power systems that require extremely high accuracy.
5. Conclusions
This paper presented a multimodal sample correction framework that effectively addresses data quality challenges in power system applications through large-model instruction enhancement and knowledge guidance. The proposed approach employs a three-stage processing pipeline combining multimodal feature extraction and fusion, knowledge-injected assessment, and sample correction. The framework focuses on two key modalities, temporal data and text documentation, which are the most critical data types in power system operations. Key innovations include temporal attention mechanisms for time series processing, knowledge graph embeddings for domain-specific reasoning, and gating networks for correction strategy selection. Experimental results demonstrated significant improvements over existing baseline approaches across multiple evaluation metrics. The framework successfully reduced hallucinations in multimodal large models while maintaining high accuracy in power professional scenarios, providing a robust foundation for intelligent operation and maintenance systems and offering promising directions for multimodal data quality management in other industrial domains.
Author Contributions
Conceptualization, H.Y.; methodology, Z.C.; software, J.D.; formal analysis, M.X.; investigation, S.Z.; data curation, S.Z.; writing—original draft, Z.C.; writing—review and editing, J.D.; visualization, M.X.; funding acquisition, H.Y. All authors have read and agreed to the published version of the manuscript.
Funding
This work was supported by the Science and Technology Project from State Grid Corporation of China (No. 5700-202490330A-2-1-ZX).
Data Availability Statement
The original contributions presented in this study are included in the article. Further inquiries can be directed to the corresponding author.
Conflicts of Interest
Authors Zhenyu Chen, Huaguang Yan and Jianguang Du were employed by the company State Grid Corporation of China, Author Huaguang Yan was employed by the company China Electric Power Research Institute, Author Meng Xue was employed by the company State Grid Information and Telecommunication Group Co., Ltd., Author Shuai Zhao was employed by the company State Grid Zhejiang Electric Power Company. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
- Xu, P.; Zhu, X.; Clifton, D.A. Multimodal learning with transformers: A survey. IEEE Trans. Pattern Anal. Mach. Intell. 2023, 45, 12113–12132. [Google Scholar] [CrossRef] [Scilit]
- Alsumaidaee, Y.A.M.; Yaw, C.T.; Koh, S.P.; Tiong, S.K.; Chen, C.P.; Ali, K. Review of medium-voltage switchgear fault detection in a condition-based monitoring system by using deep learning. Energies 2022, 15, 6762. [Google Scholar] [CrossRef] [Scilit]
- Zhao, W.; Zeng, T.; Liu, Z.; Xie, L.; Xi, L.; Ma, H. Automatic generation control in a distributed power grid based on multi-step reinforcement learning. Prot. Control Mod. Power Syst. 2024, 9, 39–50. [Google Scholar] [CrossRef] [Scilit]
- Chen, Z.; Du, J.; Li, J.; Lv, H. MDFN: Enhancing Power Grid Image Quality Assessment via Multi-Dimension Distortion Feature. Sensors 2025, 25, 3414. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Zheng, C.; Gao, X.; Chen, Z.; Lyu, L. Dual Backbone Multi-Attention Hierarchical Fusion and Feature Enhancement Network for Crowd Counting. IEEE Trans. Consum. Electron. 2025, 71, 3279–3293. [Google Scholar] [CrossRef] [Scilit]
- Wang, M.; Pi, H.; Li, R.; Qin, Y.; Tang, Z.; Li, K. VLScene: Vision-Language Guidance Distillation for Camera-Based 3D Semantic Scene Completion. In Proceedings of the AAAI Conference on Artificial Intelligence, Philadelphia, PA, USA, 25 Feburary–4 March 2025; Volume 39, pp. 7808–7816. [Google Scholar]
- Wang, C.; Gao, X.; Wu, M.; Lam, S.K.; He, S.; Tiwari, P. Looking Clearer with Text: A Hierarchical Context Blending Network for Occluded Person Re-Identification. IEEE Trans. Inf. Forensics Secur. 2025, 20, 4296–4307. [Google Scholar] [CrossRef] [Scilit]
- Chen, B.; Wu, Q.; Li, M.; Xiahou, K. Detection of false data injection attacks on power systems using graph edge-conditioned convolutional networks. Prot. Control Mod. Power Syst. 2023, 8, 1–12. [Google Scholar] [CrossRef] [Scilit]
- Gao, X.; Wang, X.; Chen, Z.; Zhou, W.; Hoi, S.C. Knowledge enhanced vision and language model for multi-modal fake news detection. IEEE Trans. Multimed. 2024, 26, 8312–8322. [Google Scholar]
- Wang, C.; Cao, R.; Wang, R. Learning discriminative topological structure information representation for 2D shape and social network classification via persistent homology. Knowl.-Based Syst. 2025, 311, 113125. [Google Scholar]
- Hu, W.; Zheng, T.; Du, P.; Chen, Z. An Intelligent Assisted Assessment Method for Distribution Grid Engineering Quality Based on Large Language Modelling and Ensemble Learning. Prot. Control Mod. Power Syst. 2025, 10, 83–97. [Google Scholar] [CrossRef] [Scilit]
- Brown, T.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J.D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al. Language models are few-shot learners. Adv. Neural Inf. Process. Syst. 2020, 33, 1877–1901. [Google Scholar]
- Devlin, J.; Chang, M.W.; Lee, K.; Toutanova, K. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers); Association for Computational Linguistics: Minneapolis, MN, USA, 2019; pp. 4171–4186. [Google Scholar]
- Omelianchuk, K.; Atrasevych, V.; Chernodub, A.; Skurzhanskyi, O. GECToR–grammatical error correction: Tag, not rewrite. arXiv 2020, arXiv:2005.12592. [Google Scholar]
- Malmi, E.; Krause, S.; Rothe, S.; Mirylenka, D.; Severyn, A. Encode, tag, realize: High-precision text editing. arXiv 2019, arXiv:1909.01187. [Google Scholar] [CrossRef] [Scilit]
- Niu, Z.; Zhong, G.; Yu, H. A review on the attention mechanism of deep learning. Neurocomputing 2021, 452, 48–62. [Google Scholar] [CrossRef] [Scilit]
- Mikolov, T.; Chen, K.; Corrado, G.; Dean, J. Efficient estimation of word representations in vector space. arXiv 2013, arXiv:1301.3781. [Google Scholar] [CrossRef] [Scilit]
- Clark, K.; Luong, M.T.; Le, Q.V.; Manning, C.D. Electra: Pre-training text encoders as discriminators rather than generators. arXiv 2020, arXiv:2003.10555. [Google Scholar]
- Zhu, S.; Ma, H.; Chen, L.; Wang, B.; Wang, H.; Li, X.; Gao, W. Short-term load forecasting of an integrated energy system based on STL-CPLE with multitask learning. Prot. Control Mod. Power Syst. 2024, 9, 71–92. [Google Scholar]
- Chen, X.; Yu, T.; Pan, Z.; Wang, Z.; Yang, S. Graph representation learning-based residential electricity behavior identification and energy management. Prot. Control Mod. Power Syst. 2023, 8, 28. [Google Scholar]
- Graves, A. Long short-term memory. In Supervised Sequence Labelling with Recurrent Neural Networks; Springer: Berlin/Heidelberg, Germany, 2012; pp. 37–45. [Google Scholar]
- Cho, K.; Van Merriënboer, B.; Gulcehre, C.; Bahdanau, D.; Bougares, F.; Schwenk, H.; Bengio, Y. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv 2014, arXiv:1406.1078. [Google Scholar] [CrossRef] [Scilit]
- Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, Ł.; Polosukhin, I. Attention is all you need. Adv. Neural Inf. Process. Syst. 2017, 30. [Google Scholar]
- Zhuang, W.; Fan, J.; Xia, M.; Zhu, K. A multi-scale spatial–temporal graph neural network-based method of multienergy load forecasting in integrated energy system. IEEE Trans. Smart Grid 2023, 15, 2652–2666. [Google Scholar] [CrossRef] [Scilit]
- Sahoo, A.K.; Samal, S.K. Deep learning based detection, classification, and location of power system faults. Bull. Electr. Eng. Inform. 2024, 13, 4248–4259. [Google Scholar] [CrossRef] [Scilit]
- Zhu, L.; Gao, J.; Zhu, C.; Deng, F. Short-term power load forecasting based on spatial-temporal dynamic graph and multi-scale Transformer. J. Comput. Des. Eng. 2025, 12, 92–111. [Google Scholar] [CrossRef] [Scilit]
- Li, C.; Wang, B. A knowledge graph method towards power system fault diagnosis and classification. Electronics 2023, 12, 4808. [Google Scholar] [CrossRef] [Scilit]
- Sun, Y.; Wang, S.; Li, Y.; Feng, S.; Chen, X.; Zhang, H.; Tian, X.; Zhu, D.; Tian, H.; Wu, H. Ernie: Enhanced representation through knowledge integration. arXiv 2019, arXiv:1904.09223. [Google Scholar] [CrossRef] [Scilit]
- Peters, M.E.; Neumann, M.; Logan, R.; Schwartz, R.; Joshi, V.; Singh, S.; Smith, N.A. Knowledge enhanced contextual word representations. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (Emnlp-Ijcnlp); Association for Computational Linguistics: Hong Kong, China, 2019; pp. 43–54. [Google Scholar]
- Yao, L.; Mao, C.; Luo, Y. KG-BERT: BERT for knowledge graph completion. arXiv 2019, arXiv:1909.03193. [Google Scholar] [CrossRef] [Scilit]
- Nathani, D.; Chauhan, J.; Sharma, C.; Kaul, M. Learning attention-based embeddings for relation prediction in knowledge graphs. arXiv 2019, arXiv:1906.01195. [Google Scholar] [CrossRef] [Scilit]
- Bordes, A.; Usunier, N.; Garcia-Duran, A.; Weston, J.; Yakhnenko, O. Translating embeddings for modeling multi-relational data. Adv. Neural Inf. Process. Syst. 2013, 26. [Google Scholar]
- Wei, J.; Wang, X.; Schuurmans, D.; Bosma, M.; Ichter, B.; Xia, F.; Chi, E.; Le, Q.V.; Zhou, D. Chain-of-thought prompting elicits reasoning in large language models. Adv. Neural Inf. Process. Syst. 2022, 35, 24824–24837. [Google Scholar]
- Yao, S.; Zhao, J.; Yu, D.; Du, N.; Shafran, I.; Narasimhan, K.R.; Cao, Y. React: Synergizing reasoning and acting in language models. In Proceedings of the The Eleventh International Conference On Learning Representations, Kigali, Rwanda, 1–5 May 2023. [Google Scholar]
- Madaan, A.; Tandon, N.; Gupta, P.; Hallinan, S.; Gao, L.; Wiegreffe, S.; Alon, U.; Dziri, N.; Prabhumoye, S.; Yang, Y.; et al. Self-Refine: Iterative Refinement with Self-Feedback. arXiv 2023, arXiv:2303.17651. [Google Scholar] [CrossRef] [Scilit]
- Chen, J.; Zhang, Z.; Zhao, H. Modeling hierarchical reasoning chains by linking discourse units and key phrases for reading comprehension. In Proceedings of the 29th International Conference On Computational Linguistics; International Committee on Computational Linguistics: Gyeongju, Republic of Korea, 2022; pp. 1467–1479. [Google Scholar]
- Wang, X.; Wei, J.; Schuurmans, D.; Le, Q.; Chi, E.; Narang, S.; Chowdhery, A.; Zhou, D. Self-consistency improves chain of thought reasoning in language models. arXiv 2022, arXiv:2203.11171. [Google Scholar]
- Child, R. Generating long sequences with sparse transformers. arXiv 2019, arXiv:1904.10509. [Google Scholar] [CrossRef] [Scilit]
- Raffel, C.; Shazeer, N.; Roberts, A.; Lee, K.; Narang, S.; Matena, M.; Zhou, Y.; Li, W.; Liu, P.J. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 2020, 21, 140. [Google Scholar]
- Lewis, M.; Liu, Y.; Goyal, N.; Ghazvininejad, M.; Mohamed, A.; Levy, O.; Stoyanov, V.; Zettlemoyer, L. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Virtual, 5–10 July 2020; pp. 7871–7880. [Google Scholar]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.

