Virtual-Document-Augmented Retrieval-Augmented Generation for Power-Domain Knowledge-Base Question Answering with Noise-Enhanced Robustness

Abstract

The Retrieval-Augmented Generation (RAG) technology enables large language models (LLM) to access external knowledge bases by introducing external documents, enhancing their capability for knowledge question answering in professional domains and generating more reliable responses. It effectively addresses issues such as LLM hallucinations and knowledge obsolescence. In the electric power domain, RAG technology can be leveraged to fully utilize accumulated corporate data and resources. However, in the retrieval phase of RAG, there are significant differences in the semantic space representation between short sentences and long text documents. Additionally, when generating answers based on retrieved relevant documents, the generator prioritizes highly relevant document fragments, a strategy that may overlook sub-relevant documents containing useful information. This paper uses an LLM to generate hypothetical documents. These documents are combined with the original question to perform similarity retrieval in the corpus, followed by the first round of answer generation. Subsequently, the original question is combined with the answer generated in the first round, and this combined content is used to retrieve relevant documents. Finally, irrelevant documents are added to the context of the retrieved relevant documents to enhance the LLM’s attention to the relevant documents. Based on the above strategies, experiments are conducted on the electricity dataset. The results show that, compared with the naive RAG method, the proposed model achieves a relative improvement of 4.63% in the ROUGE-L metric and 11.32% in the BLEU-4 metric on the electricity dataset. Meanwhile, experiments are also carried out on the public CMRC dataset, and the effectiveness of the proposed method is verified.

1. Introduction

With the in-depth integration of machine learning technologies and complex system theories, Large Language Models (LLM) have demonstrated potential application value in scenarios involving intelligent analysis and decision-making for power systems. Large language models such as the GPT3 model [1] and the Llama2 model [2] have demonstrated impressive language and knowledge mastery capabilities. However, their ability to adapt to highly specialized complex systems remains significantly limited. In complex power systems characterized by knowledge intensity and dynamic changes, LLM suffer from two key drawbacks: hallucinations [3] and insufficient professional knowledge. The former stems from the models’ lack of real information support for unseen complex scenarios, which easily leads to conclusions that contradict the actual operational laws of the system. The latter, due to the scarcity and professionalism of training data in the power system domain [4], results in the models’ inability to accurately parse technical terminology, operational logic, and regulatory frameworks within the field. When faced with system-related questions beyond the scope of their training data, LLM often fabricate answers or provide ineffective responses.

The complexity of power systems is reflected in the collaborative operation of subsystems throughout the entire chain of power generation, transmission, transformation, distribution, and consumption. Meanwhile, high-value knowledge is scattered across various types of document carriers, such as power standards, operating procedures, and safety regulations. This creates a contradiction between fragmented knowledge and the full-process system [5]. Such a contradiction makes it difficult for general-purpose LLM to achieve in-depth understanding of professional contexts and accurate parsing of semantics in intelligent processing scenarios for complex power systems. This directly leads to insufficient professionalism and accuracy in the content they generate for key tasks such as equipment maintenance recommendations and business development, failing to meet the stringent requirements for knowledge reliability in complex system optimization and decision-making.

To address the adaptation challenge of LLM in the intelligent processing of complex systems, the Retrieval-Augmented Generation (RAG) [6] paradigm integrated with machine learning technologies has been proposed, emerging as a key technology bridging general-purpose models and specialized complex systems. In tasks involving intelligent question-answering and text generation for complex power systems, RAG enables the effective utilization of professional knowledge through a dual-module collaborative mechanism. The retrieval module leverages retrieval models to efficiently locate contextual information relevant to the current system-related question within the power system domain corpus. The generation module, through prompt engineering techniques, integrates the professionally retrieved knowledge with the generative capabilities of LLM, and utilizes the capabilities of LLM to achieve the rational application of knowledge. By explicitly introducing external professional knowledge sources, this paradigm fundamentally enhances the output reliability of LLM in intelligent processing tasks for complex power systems, providing technical support for machine learning-aided optimization and decision-making in power systems.

When retrieving relevant documents, RAG always retrieves the most relevant documents. However, studies have shown that semantically similar distracting information can adversely affect LLM [7]. Interestingly, in certain scenarios, introducing unrelated “noisy” documents into the context of a RAG system can improve model performance [8], as such irrelevant documents help the model better focus on useful information.

In view of the above issues, this paper proposes a fusion method. First, a large language model generates a hypothetical document to the given question, and then the question is concatenated with this document. After that, the retriever performs similarity retrieval in the corpus based on the concatenated document and submits the results to the large language model for answer generation. Next, the question is concatenated with the answer generated by the first round of RAG; this concatenated content is used to retrieve relevant documents from the corpus. Before submitting to the large language model for generation, different numbers of completely irrelevant noisy documents are added to different positions in the context of the retrieved relevant documents, so as to help the large language model focus better on the relevant documents. Figure 1 presents an example of the method.

Figure 1. An example of generating virtual documents and adding noise for iterative RAG.

In our experiments, we demonstrate the performance of the method proposed in this paper using Llama3-Chinese-8B-Instruct as the generative model on the power dataset and the CMRC2018 dataset. Overall, our main contributions can be summarized as follows:

(1) A two-stage RAG optimization framework via hypothetical answer-guided retrieval and dynamic noise augmentation: This framework breaks through the inherent paradigm of traditional RAG, which is limited to question-only retrieval and noise-free context. Leveraging hypothetical answers generated by LLM, it expands the semantic dimensions of retrieval, addressing the critical pain point in professional domains where “vague question formulation leads to retrieval bias”. Meanwhile, it innovatively designs a dynamic noise injection mechanism, clarifying the quantity gradient of noisy documents and position selection strategies, thus providing a reproducible technical solution for revealing the intrinsic mechanism of “irrelevant information enhancing model focus”.

(2) Verification of the effectiveness and universality of the noise augmentation strategy in professional complex domains: Contrast experiments are conducted for the first time on a power domain dataset and the general CMRC2018 dataset. The study systematically analyzes the influence patterns of noise quantity and injection position on model performance, verifying the effectiveness of the proposed method in improving the accuracy of professional question answering.

(3) Provision of technical references for AI-assisted intelligent question answering in power complex systems: Targeting the characteristics of fragmented knowledge and dense professional terminology in the power domain, the proposed method—through the dual guarantees of retrieval optimization and noise augmentation—reduces the hallucination rate of general LLM in power professional question answering. It thereby offers a new implementation path for the reliable utilization of knowledge in the intelligent processing of complex systems.

2. Related Works

In 2017, Google introduced the Self-Attention mechanism in its research and constructed the Transformer model [9]. This architecture broke through the processing limitations of traditional sequence models, opening up a new path for the field of natural language processing. Subsequently, based on the technical accumulation of Transformer, Google further developed the pre-trained language model BERT [10]. In subsequent research, researchers found that expanding the scale of language model parameters or data can improve the performance of the model. Therefore, some studies have begun to attempt to train pre-trained language models with larger parameter and data scales. LLM are more powerful than any models previously seen in Natural Language Processing (NLP).

The basic RAG architecture consists of a retrieval system and a generator. The retriever converts user input queries into vectors through an encoding model, then retrieves information from an external corpus based on similarity, and integrates the retrieved relevant documents with user queries into a prompt template, which is then fed to a large language model for generation. This approach allows developers to provide incremental information input to the model through external knowledge bases without retraining the entire large model for specific tasks, thereby achieving more accurate responses while keeping model parameters unchanged. This hierarchical architecture of “retrieval-prompt-generation” not only avoids the high computational costs of large-scale model fine-tuning but also provides a flexible interface for the dynamic update of professional domain knowledge, making RAG a key technology for enhancing the practicality of large language models in current knowledge-intensive tasks. With the development of RAG technology, it has gradually evolved into a modular structure. Modular RAG [11] breaks through the framework of traditional RAG by integrating various methods at different stages to expand functional modules, making RAG more flexible and diverse.

Information retrieval plays a crucial role in the RAG process. Sparse retrieval methods such as Term Frequency-Inverse Document Frequency (TF-IDF) [12] rely on precise keyword matching and demonstrate high efficiency in large-scale document retrieval. However, they often struggle to understand the semantic relationships between words. In contrast, dense retrieval transforms documents and queries into dense vectors through deep learning models, leveraging vector similarity (e.g., cosine distance) to retrieve semantically matched documents. Dense retrieval methods like DPR [13] have proven to be highly effective in handling semantic matching tasks.

A method called Generation-Augmented Retrieval (GAR) [14] optimizes the retrieval process by generating heuristic relevant context to enhance queries, without external resources for supervision. Hypothetical Document Embeddings (HyDE) [15] is used for dense retrieval in zero-shot scenarios. With the help of the power of the LLM, it can achieve efficient retrieval without the need for relevant annotation data. Instead of directly converting the query into an embedding vector, it generates a fictional document that can answer the query through the LLM and then converts the generated virtual answer into an embedding vector. The Rewrite-Retrieve-Read (RRR) [16] framework adds a query rewriting step before retrieval to bridge the gap between the input text and retrieval needs, which helps generate more precise queries. This method first uses a large language model to generate queries, then employs a web search engine to retrieve relevant documents, and then uses a small trainable language model as a rewriter, which is trained via reinforcement learning with the performance of the large language model as a reward. In this way, the rewriter can adjust queries based on feedback from the large language model.

Noise injection in LLM refers to the process of adding noise to input data during the training or inference phase. A Robust Contrastive Pre-training (RCP) [17] approach is proposed, combining multilingual noisy data augmentation with contrastive loss in the pre-training phase. Existing studies have also employed noisy data for data augmentation [18]. In recent years, researchers have focused their attention on noise injection techniques for RAG systems [19]. Some studies have classified noise into beneficial noise and detrimental noise [20,21]. Their research found that in the presence of beneficial noise, the model enhances its attention to key context and achieves better discrimination between correct and incorrect information. Furthermore, beneficial noise contributes to the formation of more standardized answer formats. When combined with beneficial noise, LLM typically exhibit lower uncertainty and greater confidence in their outputs. The underlying principle behind these phenomena is that the semantic similarity between the noise and the query is extremely low. After Softmax normalization, these noisy documents obtain attention weights close to 0. Since the sum of attention weights equals 1, the relative attention weight of the Golden Context is passively amplified.

3. Methodology

To address the semantic gap between queries and corpora in RAG systems and enhance the robustness of LLM for improved overall performance, we propose a novel framework illustrated in Figure 2. More specifically, our approach optimizes the traditional RAG pipeline across three key dimensions:

Figure 2. The Flowchart of the Methodology.

(1)

Virtual Document Substitution: Using a pre-trained LLM, we generate context-rich pseudo-documents to semantically expand the original query. This mechanism closes the representational mismatch between concise queries and lengthy corpus documents by simulating the semantic distribution of complete documents, which in turn improves retrieval accuracy.

(2)

Iterative Query-Answer Retrieval: We first integrate the retrieved documents into a prompt template to generate an initial answer. We then concatenate this intermediate answer with the original query and embed them as a refined retrieval vector; this allows for a second-stage retrieval that focuses on more relevant documents.

(3)

Noise-Enhanced Robustness: Before inputting documents into the LLM, we introduce irrelevant “noise” documents alongside the retrieved content. This training strategy forces the model to discern signal from noise, which enhances its ability to focus on informative content and improves output robustness.

We treat the process of virtual document substitution and second-stage retrieval generation as one module, and adding irrelevant documents as another module. We conducted experiments on the electricity power dataset and the CMRC dataset [22], and validated through ablation studies that both modules contribute to performance improvements in generated results. The technical details and experimental design of this approach will be elaborated in the subsequent sections.

3.1. Text Chunk Indexing

We optimize text chunking in three ways. First, we standardize text by removing special characters and fixing encoding errors, while preserving content accuracy. This improves RAG performance. Second, we use semantic chunking. Semantic chunking is employed to maximize the relevance of text chunk contents, which not only facilitates retrieval but also collects as much context as possible while reducing noise. Third, we embed metadata directly into chunks, which makes retrieval more targeted and efficient.

3.2. Virtual Document Retrieval

Dense retrieval evaluates the similarity between a query and documents by computing their inner product. For a question q and a document $d_i$ from the corpus D, two encoders $E_q$ and $E_d$ are employed to embed them respectively. Specifically, $E_d$ maps M passages into d-dimensional dense vectors. For a question q, $E_q$ maps it into a d-dimensional vector and calculates the inner product with document $d_i$. The inner product calculation formula is as follows:

$$\mathrm{sim}(q,d_i)=E_q\left(q\right)·E_d\left(d_i\right).$$

(1)

This score quantifies the relevance of each document to the query by measuring their similarity in the embedded vector space. Documents are then ranked based on these scores, and the top-k documents with the highest rankings are selected as relevant documents:

$$\mathrm{top-}k\mathrm{docs}=arg\underset{d_i\in D}{max}\left\{\mathrm{sim}(q,d_i)\right\}.$$

(2)

We introduce a LLM to generate virtual documents:

$$d_{virtual}=\mathrm{LLM}\left(q\right),$$

(3)

which replaces the query for retrieval. The retrieved relevant documents are added to a prompt template for generation, yielding an intermediate answer denoted as $q_v$. Combining $q_v$ with q, we perform a second retrieval to obtain relevant documents, which are then added to the prompt template together with irrelevant documents for generation, resulting in the final answer.

3.3. The Introduction of Irrelevant Documents

In natural language processing tasks, the introduction of irrelevant documents serves as a key means to evaluate the robustness of retrieval systems and verify their semantic filtering capabilities. By introducing a certain amount of irrelevant information interference, the model’s ability to distinguish between semantic relevance and superficial feature matching is tested. The irrelevant documents used in this paper are composed of 3000 randomly selected common Chinese characters and short sentences completely unrelated to the electric power industry. When embedding relevant documents into the prompt template, different quantities of irrelevant documents are inserted at various positions—such as before or after the relevant documents—to test the impact of relevant documents’ positions within the prompt template on the generation results.

4. Experiments

In this section, we introduce the datasets, metrics, comparative methods, and experimental results. Based on the experimental findings, we also discuss the impact of placing relevant documents in different positions of the prompt template on the results. Finally, we validate the above experiments on the public dataset CMRC.

4.1. Setup

4.1.1. Implementation

In our experiments, the bge-large-zh-v1.5 [23] (https://huggingface.co/BAAI/bge-large-zh-v1.5, accessed on 11 February 2026) model is employed for retrieval, and metadata (such as dates) is added during index construction to enhance retrieval performance. For the generative model, we use Llama3-Chinese-8B-Instruct (https://huggingface.co/hfl/llama-3-chinese-8b-instruct, accessed on 11 February 2026), an 8-billion-parameter model that incorporates Grouped Query Attention (GQA), significantly improving inference speed and efficiency. The experiments were conducted by deploying LLM on NVIDIA A800 80 G GPUs. We conducted experiments using the sequential RAG method pipeline provided by FlashRAG [24], which offers a highly modular RAG framework allowing free combination of different components.

4.1.2. Datasets

The electricity dataset used in this paper is generated by domain experts in the electricity field based on professional knowledge documents within the power sector. These questions cover aspects such as data governance, power analysis and prediction, and business mechanisms. The dataset consists of a total of 500 entries, each containing: (1) a question; (2) a reference answer strictly grounded in the corpus; (3) the source paragraph of the reference answer; and (4) the source document identifier.

To ensure the consistency and reliability of annotation quality, we strictly adhere to a standardized annotation workflow, with the specific guidelines as follows: (1) Each answer to the annotated questions must be derived from exactly one paragraph-level source, ensuring the uniqueness of answer provenance. (2) Reference answers must be strictly grounded in the knowledge provided in the corpus, with no external knowledge or subjective inferences permitted. (3) Ambiguous or vague questions shall be rewritten to ensure their standardization.

In the annotation execution phase, each sample is independently drafted by a dedicated annotator. Upon completion, the sample must be submitted to at least one senior domain expert for review and validation, to ensure the accuracy and authority of the final annotation results. Among the question types, definition-based questions account for 12.4%, procedural questions for 37.2%, and analytical questions for 50.4%.

Additionally, we conducted experiments using the public dataset CMRC to validate the effectiveness of the proposed method. CMRC is a dataset designed for Chinese machine reading comprehension tasks. It comprises a series of carefully designed questions and their corresponding texts, annotated by human experts from Wikipedia paragraphs, aiming to evaluate machines’ ability to understand and parse Chinese language corpora. We randomly selected 500 examples from this dataset for experimentation.

4.1.3. Metrics

For evaluating the experimental results, we employed ROUGE family metrics [25] and BLEU score [26] to measure the similarity between the reference answers and the generated answers. ROUGE is an indicator widely used in text summarization evaluation and machine translation. ROUGE mainly focuses on the recall rate, that is, it emphasizes the coverage and integrity of text information. Let x be the candidate text, $\{s^{\left(1\right)},\dots ,s^{\left(K\right)}\}$ be the set of reference texts, and W be the set of n-grams extracted from the set of reference texts. The calculation formula for ROUGE-N is as follows:

$$\mathrm{ROUGE}-\mathrm{N}\left(x\right)={\displaystyle \frac{{\sum }_{k=1}^K{\sum }_{w\in \mathcal{W}}min(c_w\left(x\right),c_w\left(s^{\left(k\right)}\right))}{{\sum }_{k=1}^K{\sum }_{w\in W}{c}_w\left(s^{\left(k\right)}\right)}},$$

(4)

where $c_w\left(x\right)$ is the number of occurrences of the n-gram w appears in the candidate text x, and $c_w\left(s^{\left(k\right)}\right)$ is the number of occurrences of the n-gram w in the reference text $s^{\left(k\right)}$. There is a variant of the ROUGE series, namely ROUGE-L. The “L” in ROUGE-L stands for the longest common subsequence, which measures the similarity between two sequences. ROUGE-L is calculated by the F1 score, which combines the information of precision and recall. BLEU is an evaluation metric widely adopted in the field of machine translation. It evaluates by calculating the lexical similarity between the candidate text and the reference text. BLEU mainly calculates the co-occurrence frequency of n-grams between the candidate text and the reference text. The specific calculation formula is as follows:

$$\mathrm{BLEU}=\mathrm{BP}\times exp\left(\sum _{n=1}^N{w}_n\times log{p}_n\right)$$

(5)

Among them, $w_n$ is the weight of the n-gram, which is used to adjust the influence of n-gram of different lengths on the final score. In this study, n is usually set to 4, and $w_n$ follows an equal weight distribution. BP is the penalty coefficient, which is used to penalize the differential impact caused by short sentences.

In knowledge-based question answering systems, RAGAS [27] is commonly employed to evaluate the factual consistency and faithfulness of the QA system. This paper utilizes four metrics from RAGAS: Context Relevance, Answer Relevance, Faithfulness, and Response Groundedness. Context Relevance assesses whether all content in the retrieved documents is relevant to the question. It decomposes all retrieved contexts into individual sentences using an LLM, where the total number of sentences is denoted as $|S_c|$. It then counts the number of sentences that are helpful to the question, denoted as $|V_c|$. The ratio of these two values yields the Context Relevance score:

$$\mathrm{CR}={\displaystyle \frac{|V_c|}{|S_c|}}.$$

(6)

Answer Relevance measures the alignment between the answer and the question. It uses an LLM to reverse-engineer the question from the answer, and employs an embedding model to extract the semantic vectors of the answer and the generated question. The similarity between these vectors is calculated, with the formula expressed as:

$$\mathrm{AR}={\displaystyle \frac{1}{n}}\sum _{i=1}^{n}sim(q,q_i),$$

(7)

where n is the number of generated questions, $q_i$ represents the i-th generated question, and q denotes the actual question.

Faithfulness evaluates the extent to which the answer is consistent with the retrieved contexts. Since LLMs have the ability to fabricate responses, in an ideal RAG system, the answer should be entirely inferred from the provided contexts. The method involves extracting key points from the answer using an LLM, with the total number of key points denoted as $|S_a|$. An LLM is then used to verify whether these key points can be inferred from the contexts, where the number of inferable key points is denoted as $|V_a|$. The ratio of these two values is computed as:

$$\mathrm{F}={\displaystyle \frac{|V_a|}{|S_a|}}.$$

(8)

Response Groundedness is a core metric that evaluates whether the generated content is grounded in the retrieved context. It measures the factual consistency between the model’s response and the retrieved documents, serving as a critical tool for detecting “hallucinations” in RAG systems.

4.1.4. Failure Cases Analysis

We analyze the failure cases to better understand the failure causes of the RAG pipeline in this study. Detailed cases and corresponding reasons are provided in Appendix A.

4.2. Main Result

Table 1 and Table 2 compare our method with other baselines on the electricity power and CMRC datasets.

Table 1. Experimental results of comparing different methods on electricity power dataset.

Method	ROUGE-1	ROUGE-2	ROUGE-L	BLEU-4
Naive RAG	53.67	37.30	44.94	25.08
ITER-RETGEN [28]	54.31	38.05	45.94	25.76
Sure [29]	48.45	30.09	39.89	19.39
Flare [30]	45.73	29.93	37.46	17.76
Ircot [31]	50.88	33.45	42.58	22.39
Replug [32]	40.49	20.61	31.58	10.52
Selective-context [33]	43.26	22.16	33.56	11.61
Ours	55.34	39.16	47.02	27.92

Table 2. Experimental results of comparing different methods on CMRC dataset.

Method	ROUGE-1	ROUGE-2	ROUGE-L	BLEU-4
Naive RAG	45.35	35.57	43.59	25.27
ITER-RETGEN [28]	43.88	34.13	41.97	23.94
Sure [29]	33.62	25.52	32.03	17.69
Flare [30]	29.01	20.08	26.88	13.45
Ircot [31]	42.62	32.83	40.53	22.62
Replug [32]	28.15	14.75	25.21	8.57
Selective-context [33]	27.20	13.09	24.41	7.42
Ours	47.13	36.96	45.32	26.56

Our method achieves the best results on both datasets, improving retrieval accuracy and generation quality. Analyzing these results reveals why our framework performs better:

First, virtual document generation overcomes the query-document mismatch in standard RAG. By expanding raw queries into full pseudo-documents, we capture wider semantic context and improve retrieval coverage. Second, we concatenate the initial answer with the original query for a second retrieval. This refines the search using partial results from the first stage, improving retrieval accuracy. Finally, adding noise documents during training improves robustness. The model learns to ignore distractors, maintaining output quality when irrelevant content appears.

Table 3 presents the RAGAS evaluation results on the CMRC dataset. Our method beats the naive baseline on most metrics. ITER-RETGEN performance best overall, confirming that iterative refinement works well. IRCoT match ITER-RETGEN on Answer Relevancy and Response Groundedness. On Faithfulness, ur method surpasses the naive approach and trails ITER-RETGEN, while exceeding all other comparative methods. This indicates that pseudo-document generation and secondary retrieval do not substantially introduce hallucination risks, and the factual quality of retrieved documents is preserved. But the gap with ITER-RETGEN suggests that simple query expansion is less effective than iterative self-correction in ensuring factual consistency.

Table 3. Multi-Dimensional evaluation results of different RAG methods on the CMRC dataset.

Method	Context Relevance	Answer Relevancy	Faithfulness	Response Groundedness
Naive RAG	77.00	41.40	75.55	78.50
ITER-RETGEN [28]	81.50	47.42	77.00	86.00
Sure [29]	77.50	37.39	71.95	68.00
Flare [30]	41.00	28.75	53.00	47.00
Ircot [31]	77.50	52.33	73.40	85.50
Replug [32]	77.50	42.37	56.33	70.00
Selective-context [33]	77.50	37.98	42.95	39.00
Ours	78.50	42.49	76.05	83.00

4.3. Ablation Study

To explore the roles of different modules, we conducted an ablation study, and the results are shown in Table 4. We use w/o to denote the version without a specific module. It can be observed that: the performance of the method declines when any component is removed, indicating that all components are highly effective. The results indicate that on electricity power dataset, removing the HyDE module leads to a relative decline of 1.91% in Rouge-L and 6.01% in BLEU-4. This suggests that the generated semantically expanded documents can effectively bridge the representational gap between the query and the corpus. When irrelevant documents are excluded, Rouge-L decreases by 2.19% relatively and BLEU-4 by 4.65% relatively, verifying the necessity of the noise enhancement strategy for improving the robustness of LLM.

Table 4. The ablation results under different module settings.

Module	ROUGE-1	ROUGE-2	ROUGE-L	BLEU-4
electricity power dataset
HyDE + irrelevant documents	55.34	39.16	47.02	27.92
w/o irrelevant documents	54.50	38.11	45.99	26.62
w/o HyDE	54.59	38.01	46.12	26.24
w/o HyDE+irrelevant documents	53.67	37.30	44.94	25.08
CMRC dataset
HyDE + irrelevant documents	45.35	36.96	45.32	26.56
w/o irrelevant documents	46.51	36.12	44.65	26.16
w/o HyDE	45.94	36.24	44.23	25.98
w/o HyDE+irrelevant documents	45.35	35.57	43.59	25.27

4.4. Impact of Irrelevant Documents

Table 5 shows how adding irrelevant documents affects answer quality. We let N denote the number of irrelevant documents added.

Table 5. Relevant documents are behind different numbers of irrelevant documents.

N	Naive				HyDE
	ROUGE-1	ROUGE-2	ROUGE-L	BLEU-4	ROUGE-1	ROUGE-2	ROUGE-L	BLEU-4
0	53.67	37.30	44.94	25.08	54.50	38.11	45.99	26.62
1	53.70	37.15	45.04	25.08	54.49	37.96	45.99	26.63
2	54.11	37.83	45.72	25.91	54.98	38.82	46.78	27.58
3	54.18	37.44	45.57	25.55	54.94	38.46	46.71	27.34
4	54.59	38.01	46.12	26.24	54.66	38.17	46.34	27.03
5	54.54	37.97	46.12	26.31	54.53	38.17	46.27	27.02
6	54.13	37.55	45.73	26.08	55.06	38.65	46.75	27.63
7	54.07	37.52	45.68	25.74	54.53	38.32	46.32	27.24
8	54.33	37.80	45.96	26.35	55.34	39.16	47.02	27.92
9	54.03	37.54	45.69	26.19	54.73	38.40	46.42	27.26
10	54.08	37.59	45.87	26.08	54.86	38.51	46.45	27.39

It can be observed that as the number of irrelevant documents increases, the generation quality of the large language model first rises and then declines. Likely because without noise, the model fixates on highly relevant documents and misses weaker but valid ones. Irrelevant documents provide contrast, redirecting attention to these weaker relevant documents. As more irrelevant documents are added, performance degrades and eventually falls below the baseline. This indicates that noise receives low attention scores in LLM. Moderate noise helps the LLM focus attention, while excessive noise increases its weight and diverts attention, degrading generation quality.

4.5. Impact of Relevant Documents Positioning

Figure 3 and Figure 4 show how the position of relevant documents in the prompt affects performance.

Figure 3. The ROUGE-L scores when relevant documents are located at different positions in the prompt template.

Figure 4. The BLEU-4 scores when relevant documents are located at different positions in the prompt template.

We test three arrangements: relevant documents placed after, among, or before irrelevant ones. These correspond to placing relevant information far from, near, or closest to the query. When relevant documents are placed after irrelevant ones, the model achieves better overall and best performance than the other two groups. This arrangement also outperforms the baseline across all noise levels, while the other two positions fall below baseline when noise increases.

When processing long prompts, generative LLM tend to focus more on end-sequence content. Placing relevant documents after irrelevant ones puts key information in this high-attention region, helping the model better extract relevant content from noise and reduce interference.

Table 6 demonstrates the optimal performance of relevant documents in different positions. R stands for relevant documents.

Table 6. The influence of the position of relevant documents on the results.

Model Condition	ROUGE-1	ROUGE-2	ROUGE-L	BLEU-4
Naive RAG	53.67	37.30	44.94	25.08
R are behind irrelevant ones	54.59	38.01	46.12	26.24
HyDE + R are behind irrelevant ones	55.34	39.16	47.02	27.92
R are among irrelevant ones	54.11	37.68	45.54	25.92
HyDE + R are among irrelevant ones	54.78	38.64	46.58	27.09
R are in front of irrelevant ones	54.04	37.66	45.51	25.42
HyDE + R are in front of irrelevant ones	54.72	38.77	46.43	27.46

Results in Table 6 further demonstrate that the optimal performance achieved by varying the position of relevant documents consistently outperforms the naive RAG method.This indicates that, without modifying the base retrieval and generation framework, simply optimizing the position of relevant documents within the prompt can stably improve the model robustness in complex noisy environments.Compared with naive RAG without positional optimization, properly arranging document order in prompts enables the model to focus more effectively on critical information.

4.6. Efficiency Analysis

Figure 5 illustrates the resource consumption characteristics of various methods when processing power domain datasets. The horizontal axis represents the total running time, which reflects the inference speed of each method. The vertical axis represents the peak memory footprint, indicating the computational resource overhead during execution. The bubble size is positively correlated with the answer generation quality, where the ROUGE-L score is adopted as the quantitative metric. Meanwhile, the color intensity indicates the value of the comprehensive efficiency score: darker colors represent better overall performance, while lighter colors represent poorer performance. To quantitatively evaluate the trade-off among efficiency, resource cost, and generation quality, we define the comprehensive efficiency score $E_{score}$ as a weighted sum of three normalized indicators, formulated as follows:

$$E_{score}=w_t{E}_{time}+w_m{E}_{mem}+w_q{E}_{qual}.$$

(9)

Figure 5. Three-Dimensional perspective on RAG method efficiency comparison: a collaborative evaluation of time, memory footprint, and answer quality.

Here, $E_{time}$, $E_{mem}$, and $E_{qual}$ denote the min-max normalized scores of running time, memory footprint, and ROUGE-L score, respectively. All original metrics are linearly normalized to the range [0, 1] to ensure comparability. To ensure visual interpretability, the ROUGE-L scores were first normalized to a range of [0, 1], where the minimum normalized value is 0. Despite this, the bubble size in Figure 5 never reaches 0, as we set a fixed minimum bubble size to avoid imperceptible bubbles for samples with the lowest ROUGE-L scores. In this study, the weights are empirically set to $w_t=0.4$, $w_m=0.3$, and $w_q=0.3$, placing a slightly higher emphasis on inference efficiency while balancing memory usage and answer quality.

Naive, Flare, and selective-context in the lower-left quadrant show fast inference and low GPU memory footprint. Among them, selective-context is fastest but its small bubbles show it sacrifices quality. Flare uses the least VRAM and maintains fast inference, with its dark color indicating high overall efficiency. Naive RAG balances quality and speed. It runs as fast as Flare but has much larger bubbles, making it a stable baseline.

In the lower-right quadrant, the Sure method represents the extreme case of inference latency while maintaining negligible GPU memory overhead. Its intermediate color intensity suggests moderate overall efficiency, rendering it suitable for memory-constrained environments that can tolerate prolonged latency. ITER-RETGEN, conversely, offers high quality at moderate speed—bubbles match Naive RAG but take longer.

The center of the scatter plot shows a balanced trade-off. The proposed method (Ours) is positioned centrally, with its deep red hue denoting superior composite efficiency. Compared to ITER-RETGEN, our approach achieves a 20% reduction in runtime while maintaining comparable GPU memory utilization; relative to Naive RAG, our method incurs an approximately 2× increase in computational time but offers advantages in accuracy and inference stability. Ircot occupies the same vertical segment as ITER-RETGEN but exhibits marginally smaller bubble dimensions, indicating slightly inferior generation quality.

Finally, Repluge manifests as an outlier with anomalous GPU memory consumption, utilizing approximately 1.7× the VRAM of other methods. Its pale yellow coloration signals low composite efficiency, likely attributable to full-context loading mechanisms that lack memory optimization.

5. Conclusions

This study addresses the core challenges faced by RAG technology in knowledge-base question answering scenarios within the power domain, including semantic space mismatch, the tendency to overlook sub-relevant information, and insufficient model robustness. We propose a framework that combines virtual document generation, iterative retrieval, and noise injection to address these problems. Using LLMs to generate virtual documents broadens the query semantics, and the iterative retrieval with “query + preliminary answer” helps locate information more accurately. Additionally, motivated by the long-tail effect of the attention mechanism, adding an appropriate amount of irrelevant documents can enhance the LLM attention toward relevant documents, and appropriately adjusting the position of relevant documents within the context also effectively improves their attention weights. Experiments on power-domain and general-domain (CMRC) datasets show consistent improvements.

However, this research still has several limitations that require critical reflection and subsequent improvement:

(1) Semantic chunking sometimes split critical power-domain concepts across chunks, hurting retrieval completeness.

(2) The retrieval performance degrades when facing abstract or implicit queries that lack explicit terminological matches.

(3) The model incurs increased latency and cannot well meet high real-time requirements.

Future work can address these limitations in the following directions: First, we can improve retrieval by integrating power-specific terminology and knowledge graphs to better handle abstract queries. Second, explore lightweight iterative retrieval strategies and model compression technologies to improve inference efficiency while ensuring performance, thereby promoting the practical application of the method in real-time scenarios. Finally, we will investigate how to improve the generalization ability of the method, enabling effective deployment in entirely new professional domains.