LedgerRAG: Governance-Driven Agentic Chain of Retrieval for Dynamic Knowledge Scenarios

Abstract

Retrieval-augmented generation (RAG) grounds large language models (LLMs) with external evidence. Dynamic knowledge tasks, however, require systems to decide not only what to retrieve but also when to refresh, how to arbitrate conflicts, and how to preserve an auditable record of the evidence used to answer a query. We present LedgerRAG, a trigger-aware retrieval chain framework that maintains an explicit claim-level evidence ledger and uses coverage, temporal validity, authority, and conflict signals to control retrieval, refresh, and stopping decisions. We expand the evaluation with a query-level BM25 baseline, a dense retriever setting, and task-aligned proxy baselines representing graph-style retrieval, temporal-only retrieval, and conflict-focused retrieval. The revised results show that LedgerRAG’s clearest advantage lies in conflict governance and auditable evidence control, achieving near-perfect ConFLICT adjudication (CRAcc = 0.993) under authority-aware routing while yielding more modest gains and explicit trade-offs in regulation-change and streaming settings.

1. Introduction

Large language models (LLMs) are increasingly deployed in high-stakes domains such as legal compliance question answering, policy interpretation, and newsroom intelligence. Retrieval-augmented generation (RAG) [1,2,3] mitigates hallucinations by grounding generation using external documents, but most RAG pipelines assume relatively static corpora and treat retrieval as a single pre-generation step.

In dynamic knowledge scenarios, correctness depends not only on factual content but also on time (which version is effective at time t), authority (which sources dominate in a dispute), and governance (whether the system can be audited and replayed). In these settings, outdated or low-authority evidence can cause serious compliance and reputational risks.

Existing RAG variants often adopt either one-shot retrieval or a fixed number of retrieval steps. This rigid design leads to a three-way tension between accuracy, timeliness, and cost: under-retrieval yields evidence gaps (claims are not supported or citations are misaligned), while over-retrieval inflates latency and computation costs. More critically, dynamic settings exhibit two structural failure modes: (i) gap, where retrieved evidence fails to cover generated atomic claims and (ii) drift, where evidence becomes invalid at target time t (e.g., a repealed statute or a corrected report), yet the system keeps citing it.

While recent agentic or self-reflective frameworks (e.g., Self-RAG [4]) improve general citation quality by deciding when to retrieve, they typically rely on implicit semantic signals and lack explicit modeling of authority and time. As a result, semantically similar but temporally invalid documents can dominate retrieval rankings, causing what we refer to as temporal hallucination (Figure 1).

To bridge this gap, we propose LedgerRAG, a trigger-driven chain-of-retrieval framework that maintains an explicit evidence ledger as the reasoning state. Retrieval is then regulated by three trigger families: gap, drift, and conflict. Effectively managing dynamic RAG requires a structured integration of heterogeneous cues–claim decomposition, evidence metadata, stance decisions, and authority–time adjudication so that decisions remain inspectable and auditable over time. This emphasis on structured knowledge processes and auditability aligns with scenario-based knowledge creation and dynamic capability perspectives in knowledge management. In this revision, we also sharpen the scope of our claim: LedgerRAG’s most consistent gains emerge in conflict governance and evidence control rather than universal superiority across every dynamic task.

Our contributions are four-fold:

(1)

Evidence ledger paradigm for dynamic RAG: We formulate dynamic RAG as explicit claim–evidence state tracking, enabling audit trails and controllable governance.

(2)

Gap/drift/budget trigger controller: We design a lightweight controller that adaptively invokes refinement, counter-evidence, and time-aware retrieval, achieving better quality–cost trade-offs.

(3)

DyK-Bench: We build a benchmark covering regulatory change, cross-source conflict, and streaming updates with an audit-friendly evaluation protocol.

(4)

System integration: We integrate the framework into a newsroom CMS prototype and provide latency and workflow case studies to validate practical deployability.

2. Related Work

2.1. From Static Retrieval to Agentic Reasoning

RAG augments LLM generation with retrieved documents. Beyond early formulations, retrieval-augmented pretraining (e.g., REALM) and dense retrieval (e.g., DPR) have become core building blocks. Generative architectures like fusion-in-decoder (FiD) systems further optimized open-domain QA by processing multiple passages [5]. Recently, RETRO scaled this paradigm to trillions of tokens [6].

To overcome the limitations of one-shot retrieval, prior work explores iterative and agentic retrieval reasoning loops. Active retrieval triggers retrieval based on uncertainty [7], while RAG-Critic uses critic-guided workflows to diagnose errors [8]. Self-RAG employs self-reflection to control retrieval. Furthermore, chain-of-thought (CoT) prompting has motivated multi-step retrieval reasoning designs [9]. However, these methods often treat source authority and temporal validity implicitly.

2.2. Challenges in Dynamic Knowledge: Time, Conflict, and Streaming

Dynamic corpora require temporal consistency and conflict resolution. StreamingQA [10] and RealTime QA [11] highlight that correct answers depend on the current time. TimeRAG further studies temporal reasoning in complex questions [12]. Meanwhile, open retrieval sources frequently contain contradictions (e.g., rumors vs. official statements) [13]. Recent work like TruthfulRAG attempts to resolve such conflicts using knowledge graphs [14].

Drawing on the broader success of graph-attention mechanisms in various domains [15], text-oriented graph approaches such as GraphRAG strengthen global reasoning by constructing corpus-level summaries [16,17]. While effective for global sensemaking, graph maintenance introduces overhead for real-time streaming, making our ledger approach more suitable for interactive workflows. Another line of research addresses dynamic knowledge via model editing (e.g., ROME [18], MEMIT [19]), though these modify parameters rather than managing retrieval context.

2.3. Governance and Auditability

Trustworthiness in high-stakes RAG extends to process-level governance. ALCE enforces citation-grounded generation [20], and multi-agent debate can reduce bias [21]. Our “support vs. conflict” framing aligns with fact verification tasks like FEVER [22] and SciFact [23]. For evaluation, automated judges (e.g., LLM-as-a-Judge [24]) and hallucination detection methods [25] provide scalable quality control. LedgerRAG elevates these signals to first-class control variables via the evidence ledger.

3. Problem Formulation and the Evidence Ledger

3.1. Task Definition

We define dynamic RAG as conditional generation over a time-evolving corpus. Given a query q and a target time anchor t (defaulting to the current time if unspecified), the system outputs: (i) an answer a; (ii) a citation set C; and (iii) an audit log J capturing retrieval and decision traces. Unlike static QA, dynamic RAG must satisfy:

(1)

Temporal consistency: evidence must be valid as of time t (e.g., the latest effective regulation).

(2)

Conflict resolution: when sources disagree, the system must follow an authority–time decision rule rather than semantic voting.

3.2. Evidence Ledger: Claim–Evidence State Representation

LedgerRAG maintains an evidence ledger $L_s$ as the explicit reasoning state at step s. The ledger stores atomic claims and associated evidence with structured metadata. We first decompose the query (and intermediate hypotheses) into a set of atomic claims $\left\{c_i\right\}$ with importance weights $w_i$ (higher for core constraints such as legal thresholds). For each claim $c_i$, the ledger stores an evidence set $E_i$ containing entries $e_{i,j}$. Each evidence entry is represented as a tuple:

$$e=(\mathrm{text},\mathrm{doc\_id},\mathrm{source},ts,{\alpha }_{\mathrm{auth}},y,r),$$

(1)

where $ts$ is the publication timestamp, ${\alpha }_{\mathrm{auth}}$ is an authority score, $y\in \{\mathrm{Support},\mathrm{Refute}\}$ is a stance label toward the claim, and r is a retrieval/extraction relevance score.

Stance labels y are assigned via a rubric-based verifier prompt under deterministic decoding ($\mathrm{temperature}=0$), conditioned on the (claim, evidence span), following standard verification formulations. The authority score ${\alpha }_{\mathrm{auth}}$ is derived from DyK-Bench’s discrete authority_level metadata (Section 6.1) and normalized to $[0,1]$.

We define a claim verification score $v_i$ as the difference between aggregated support and refutation evidence, weighted by authority and time decay:

$$\begin{array}{cc}\hfill v_i& =\sum _{e\in E_i^{+}}w\left(e\right)-\sum _{e\in E_i^{-}}w\left(e\right),\hfill \\ \hfill w\left(e\right)& =r\left(e\right)·{\alpha }_{\mathrm{auth}}\left(e\right)·\exp (-{\displaystyle \frac{t-ts\left(e\right)}{H}}),\hfill \end{array}$$

(2)

where H is a domain-dependent decay timescale (larger for stable regulations and smaller for news). A claim is considered verified if $v_i\ge {\tau }_v$; otherwise it remains unverified and is surfaced to the controller.

3.3. State Metrics

To drive controllable retrieval, we derive three measurable state metrics from the ledger.

Coverage (gap signal): Coverage measures how much of the weighted claim set is verified:

$$\mathrm{Cov}\left(L_s\right)={\displaystyle \frac{{\sum }_i{w}_i·⊮[v_i\ge {\tau }_v]}{{\sum }_i{w}_i}}.$$

(3)

Low coverage indicates evidence gaps.

Time validity (drift signal): Time validity quantifies the freshness of the best evidence per claim at time t:

$$\mathrm{TV}@t\left(L_s\right)={\displaystyle \frac{{\sum }_i{w}_i·{max}_{e\in E_i}\exp (-\frac{t-ts\left(e\right)}{H})}{{\sum }_i{w}_i}}.$$

(4)

Low $\mathrm{TV}@t$ suggests drift risk.

Stance divergence (conflict signal): Divergence captures whether strong support and strong refutation coexist for the same claim (above a confidence threshold), indicating unresolved conflicts that require counter-evidence search and authority-aware adjudication.

4. Method: LedgerRAG

4.1. Overview

LedgerRAG implements an iterative perceive–decide–update loop, as illustrated in Figure 2. At each step s: (i) the system observes the ledger state $L_s$ and computes $\mathrm{Cov}\left(L_s\right)$, $\mathrm{TV}@t\left(L_s\right)$, and conflict signals; (ii) a lightweight controller maps these signals to an action $a_s$; and (iii) executing $a_s$ retrieves new evidence, updates the ledger, and logs the trace. The loop stops when the budget is exhausted or marginal gains converge.

4.2. Trigger Mechanisms

We define three trigger families that discretize continuous state metrics into control signals:

Gap trigger: Activated when (i) $\mathrm{Cov}\left(L_s\right)<{\tau }_{\mathrm{cov}}$; (ii) an important claim (high $w_i$) remains unverified; or (iii) retrieved evidence lacks required metadata (e.g., missing timestamp). Response: structured query refinement to increase recall for missing claims.

Drift trigger: Activated when $\mathrm{TV}@t\left(L_s\right)<{\tau }_{\mathrm{tv}}$ or when update cues are detected in retrieved snippets (e.g., ‘revised’, ‘repealed’, ‘correction’, ‘latest’). Response: time-aware retrieval that injects the time anchor t and prioritizes the most recent effective documents.

Budget trigger: Activated when the reasoning loop reaches $S_{\max }$ steps or when marginal improvements in Coverage/$\mathrm{TV}@t$ fall below $\epsilon$ for $k_{\mathrm{pat}}$ consecutive steps. Response: early stopping and audit freeze.

Conflict trigger: Activated when a claim exhibits high stance divergence (strong support and refutation coexist). Response: counter-evidence search and authority–time adjudication.

4.3. Action Space

The controller chooses from four atomic actions:

A1: Structured Refinement: For Gap: generate claim-specific sub-queries via entity anchoring, domain term expansion, and top-k expansion, leveraging re-ranking patterns to improve recall [26].

A2: Counter-Evidence Search: For Conflict: issue adversarial queries (e.g., adding ‘debunk’, ‘correction’, ‘withdrawn’) to actively seek refuting evidence and reduce confirmation bias.

A3: Time-Aware Retrieval: For Drift: inject time constraints (soft cues in query text and hard metadata filters such as publish_date $\le t$) and prefer newer versions when authority is comparable.

A4: Audit & Stop: Freeze the ledger, construct a minimal evidence package, and generate the final answer with citations, diff summary, and an editor-facing checklist of unresolved items.

4.4. Authority–Time Adjudication and Auditable Generation

To resolve contradictions, we define an authority–time partial order over evidence. For two evidence entries $e_a$ and $e_b$:

$$e_a\succ e_b\iff {\alpha }_{\mathrm{auth}}\left(e_a\right)>{\alpha }_{\mathrm{auth}}\left(e_b\right)\mathrm{or}[{\alpha }_{\mathrm{auth}}\left(e_a\right)={\alpha }_{\mathrm{auth}}\left(e_b\right)\mathrm{and}ts\left(e_a\right)>ts\left(e_b\right)].$$

(5)

This implements the principle “authority first; recency as tie-breaker.”

Instead of feeding explainably noisy full-context stacks, we build a minimal covering evidence package C using a greedy procedure: iteratively select evidence that maximizes marginal claim coverage until $\mathrm{Cov}\left(L_s\right)\ge {\tau }_{\mathrm{cov}}$ or the context budget is reached. The generator is then constrained to produce:

(i)

  A direct answer with sentence-level citations (doc_id);

(ii)

 A diff summary if drift was triggered;

(iii)

A checklist of unresolved or high-risk claims for human review.

4.5. Algorithm

In this subsection, we detail the inference procedure of the proposed LedgerRAG framework. The complete step-by-step process, including the evidence ledger initialization and the iterative retrieve-and-update loop, is summarized in Algorithm 1.

Algorithm 1 LedgerRAG Inference

Require: Query q, target time t, max steps $S_{\max }$, thresholds $({\tau }_{\mathrm{cov}},{\tau }_{\mathrm{tv}},{\tau }_{\mathrm{v}})$, decay timescale H, tolerance $\epsilon$, patience $k_{\mathrm{pat}}$ Ensure: Answer a, citations C, audit log J   1: Decompose q into atomic claims $\left\{c_i\right\}$ with weights $\left\{w_i\right\}$; initialize ledger $L_0$ with empty evidence   2: for $s=0\dots S_{\max }-1$  do   3:    Compute $\mathrm{Cov}\left(L_s\right)$, $\mathrm{TV}@t\left(L_s\right)$, and conflict signals   4:    if BudgetTrigger or StopCondition then   5:    break   6:    end if   7:    Select action $a_s\in \{A1,A2,A3\}$ based on triggers   8:    Retrieve documents; parse evidence tuples; assign stance; update ledger $L_{s+1}$.    The ledger is append-oriented: each new evidence item becomes a new entry, deduplicated by $(doc\_id,span\_id)$.    Entries are never overwritten; when later evidence dominates on authority and/or recency, older entries are marked superseded (archived) but retained for audit.   9:    Append ($a_s$, triggers, retrieved doc_ids, ledger deltas) to audit log J 10: end for 11: Build minimal covering evidence package C from final ledger; generate answer a with citations and checklist 12: return $(a,C,J)$

5. DyK-Bench: A Benchmark for Dynamic Knowledge Governance

5.1. Data Pipeline: Pachon

To evaluate dynamic RAG under temporal evolution and cross-source conflicts, we construct DyK-Bench using a semi-automated pipeline (Pachon) with two key components.

Spatio-temporal normalization: Each raw document is normalized into a structured record (doc_id, text, publish_date, source, authority_level, status/version). Publish dates are standardized to ISO-8601 [27] timestamps, and authority levels are assigned using a domain-specific source taxonomy; in our benchmark, authority_level ∈ $\{1,2,3\}$ corresponds to {user-generated, professional outlets, official sources}, and we set ${\alpha }_{\mathrm{auth}}$ = authority_level/3.

Time-sliced indexing: To prevent future leakage, we build time-sliced indexes that replay the corpus state at any snapshot time t. All retrieval operations in the evaluation are constrained to documents with publish_date $\le t$.

5.2. Task Design

DyK-Bench contains three tasks capturing distinct dynamic patterns.

RegChange-QA (regulatory change): Identify the uniquely effective version as of t when old and new versions are semantically similar.

ConFLICT-QA (cross-source conflict): Adjudicate contradictory claims (e.g., rumor vs. official clarification) using authority-aware decision rules.

Stream-QA (streaming updates): Gnerate minimal sufficient answers over a sequence of times ($t_1,t_2,\dots$), refreshing only affected claims when updates or corrections appear.

5.3. Annotation and Auditing

We follow a hybrid annotation strategy consisting of LLM-assisted draft generation, automated auditing, and expert verification. The audit stage enforces two hard constraints: (i) leakage check (gold answers do not use future information beyond t) and (ii) retrievability check (gold evidence is retrievable from the corresponding time slice). After expert review, DyK-Bench includes 370 query-level instances and 479 step-level evolution records when unfolding Stream-QA trajectories.

DyK-Bench comprises 370 query-level instances distributed across the three tasks: RegChange-QA (135 queries), ConFLICT-QA (110 queries), and Stream-QA (125 queries yielding 479 step-level evolution records). To mirror real-world retrieval difficulty, each time-sliced index contains an average of 12,400 background documents. Notably, in the ConFLICT-QA subset, approximately 92% of the conflicts are “decisively resolvable”, meaning an authoritative official clarification or erratum explicitly exists in the target time slice, which specifically tests a system’s ability to locate and prioritize high-authority counter-evidence over repetitive rumors.

6. Experiments

6.1. Experimental Setup

We evaluate LedgerRAG against strong baselines on DyK-Bench, addressing three research questions: RQ1 (effectiveness): can the system reduce evidence gaps and resolve conflicts? RQ2 (timeliness): does explicit drift modeling reduce outdated citations? RQ3 (governance): can we improve auditability and quality while controlling retrieval cost?

Backbone model and retriever: Unless otherwise stated, all methods use the same generator (DeepSeek-V3 [28]) and the same retriever backend (Elasticsearch BM25 [29]) to avoid confounding factors. We use a chunk size of 512 tokens with a 128-token overlap, and generation temperature is set to 0 for reproducibility. To rigorously evaluate the framework, we adopt two retrieval settings: Setting A (BM25-controlled) strictly isolates the impact of our logic layer without confounding from dense retriever capacities; Setting B (dense-controlled) evaluates retriever generalization and task-aligned strong proxy baselines.

Baselines: Under these two settings, we compare against a spectrum of baselines: (i) Query-RAG/Standard RAG (BM25 & Dense): All methods share the same atomic-claim decomposition module used for governance evaluation. Standard RAG performs one retrieval per claim (top-$k=5$) and generates the final answer in a single pass using the concatenated evidence. (ii) Self-RAG-style Prompted (BM25 & Dense): We implement a self-reflective agentic control policy via prompting and reflection signals following Asai et al. while keeping the backbone constant to isolate the effect of retrieval control mechanisms. (iii) Task-Aligned Proxy Baselines (Dense): To thoroughly assess specific dynamic challenges, we include tailored proxy implementations of GraphRAG-lite (graph-style expansion), TimeRAG-proxy (temporal-focused), and TruthfulRAG-proxy (conflict-focused). (iv) LedgerRAG & LedgerRAG-v5 (Ours): Our proposed framework, reported as the base dense configuration and the final v5 system with full conflict-retrieval refinements.

Metrics: We report SupportF1 (citation support vs. refute), $\mathrm{TV}@t$ (temporal validity), CRAcc (conflict resolution accuracy, ConFLICT-QA), UpdateF1 (update detection, Stream-QA), retrieval calls, and end-to-end latency. We also define a governance-oriented composite score, SUS (System Utility Score):

$$\mathrm{SUS}=0.25·\mathrm{Trace}+0.25·\mathrm{ClaimAlign}+0.25·\mathrm{Timely}+0.25·\mathrm{Uncertainty},$$

(6)

where Trace measures citation traceability, ClaimAlign measures claim–evidence alignment, Timely reflects time compliance, and Uncertainty measures explicit disclosure of unresolved items. (Note: this SUS is not the HCI System Usability Scale; it is a governance utility score tailored to auditable RAG).

Retrieval calls and latency: We count one retrieval call as one invocation to the BM25 backend (per (sub-)query). End-to-end latency is wall-clock measured from query submission to final answer, including retrieval, ledger updates, and LLM generation, with single-query concurrency and no batching.

Hardware environment: All experiments were conducted on a workstation operating on Ubuntu OS (Canonical Ltd., London, UK), equipped with an Intel CPU (Intel Corporation, Santa Clara, CA, USA) and two NVIDIA RTX A6000 GPUs (NVIDIA Corporation, Santa Clara, CA, USA).

Hyperparameters: Unless otherwise stated, we use $S_{\max }=3$, top-$k=5$, ${\tau }_{\mathrm{cov}}=0.8$, ${\tau }_{\mathrm{tv}}=0.7$, ${\tau }_{\mathrm{v}}=0.0$, $\epsilon =0.01$, $k_{\mathrm{pat}}=2$, and $H=365$ days (regulations)/7 days (news).

LLM-as-a-Judge: For semantic metrics (SupportF1, SUS sub-scores), we use GPT-4o as an automatic evaluator with a fixed prompt and deterministic decoding. On a 50-sample calibration set, the judge’s scores correlate with expert ratings with Pearson $r=0.87$. The evaluation prompt is provided in Appendix A. Following the methodology of MT-Bench [24], we release judging prompts to enable independent auditing.

6.2. Main Results

Table 1. Overall performance on DyK-Bench including task-aligned strong proxy baselines.

Method	RegChange	ConFLICT	Stream	RegChange	ConFLICT
	Ans2gramF1 (↑)	CRAcc (↑)	UpdateF1 (↑)	Calls (↓)	Calls (↓)
Query-RAG (BM25)	0.5357	0.6742	0.7263	1.0000	1.0000
Self-RAG-style Prompted (Dense)	0.6314	0.6867	0.7374	2.0000	2.0000
GraphRAG-lite (Dense)	0.7017	0.6867	0.7486	1.0000	1.0000
TimeRAG-proxy (Dense, temporal)	0.6810	0.6654	0.7318	2.0000	1.0000
TruthfulRAG-proxy (Dense, conflict)	0.6638	0.7467	0.7430	2.2733	1.0000
LedgerRAG (Dense baseline)	0.5388	0.6548	0.7268	4.5467	2.0000
LedgerRAG (Full)	0.6994	0.9930	0.7430	6.8200	3.0000

Note: The arrows (↑) and (↓) indicate that higher and lower values represent better performance, respectively. Bold values indicate the best performance.

summarizes the overall performance across the three dynamic tasks, comparing our framework against a spectrum of dense and task-aligned proxy baselines. Several conclusions emerge. First, LedgerRAG (Full) demonstrates its most profound advantage in conflict governance (ConFLICT-QA), achieving a near-perfect adjudication accuracy (CRAcc = 0.9930). This significantly outperforms both the base dense configuration (LedgerRAG Dense baseline, 0.6548) and the specialized conflict-focused baseline (TruthfulRAG-proxy, 0.7467).

Second, the evaluation reveals honest trade-offs across different knowledge scenarios. For instance, GraphRAG-lite shows strong offline performance on RegChange (Ans2gramF1 = 0.7017) and Stream (UpdateF1 = 0.7486), with LedgerRAG (Full) following very closely (0.6994 and 0.7430, respectively). These findings explicitly refine the scope of our contribution: LedgerRAG trades a controlled increase in retrieval calls (e.g., averaging 3.00 calls on ConFLICT) to achieve strict evidence governance and auditable conflict resolution rather than claiming universal superiority across all standard QA metrics.

6.3. Task-Level Analysis

Table 2. Task-level effectiveness (metric deep-dive).

Task	Metric	BM25	Ours	Imp. ($\Delta$)
RegChange	Ans2gramF1	0.5357	0.6994	+0.1637
ConFLICT	CRAcc	0.6742	0.9930	+0.3188
Stream	UpdateF1	0.7263	0.7430	+0.0167

and

Table 3. Task-level cost (retrieval calls and latency).

Task	Metric	BM25	Ours	Change
RegChange	Ret. Calls	2.27	2.27	+0.00
	Latency	1085 ms	1101 ms	+16 ms
ConFLICT	Ret. Calls	1.00	1.00	+0.00
	Latency	157 ms	157 ms	+0 ms
Stream	Ret. Calls	2.56	4.61	+2.05
	Latency	581 ms	1028 ms	+447 ms

break down effectiveness and cost by task for the BM25 baseline and our method (these are also the two systems used in the CMS prototype). The Self-RAG baseline is reported in Table 1 for overall comparison; for example, on ConFLICT-QA, the prompted Self-RAG baseline reaches CRAcc = 0.6867, while LedgerRAG reaches 0.9930.

RegChange-QA: LedgerRAG improves Ans2gramF1 from 0.5357 to 0.6994 (+0.1637), demonstrating that explicit drift detection is crucial when old and new versions are semantically similar.

ConFLICT-QA: Conflict resolution accuracy improves from 0.6742 (BM25) and 0.6867 (Self-RAG) to 0.9930 (+0.3188 vs. BM25), driven by counter-evidence search and authority–time adjudication.

Stream-QA: UpdateF1 increases from 0.7263 to 0.7430 (+0.0167). This modest gain reflects the intrinsic difficulty of frequent refresh under streaming updates, accompanied by a planned increase in retrieval calls and latency.

6.4. Statistical Significance

We run paired bootstrap tests ($N=5000$ resamples). We use Standard RAG (BM25) as the base system for all comparisons in

Table 4. Statistical significance testing (paired bootstrap, $N=5000$; base = Standard RAG (BM25)).

Subset	Metric	$\Delta$ (Ours-Base)	95% CI	p-Value	Result
All ($N=370$)	SUS	+0.115	[0.097, 0.133]	$<0.001$	**
	TV@t	+0.075	[0.054, 0.096]	$<0.001$	**
	SupportF1	+0.196	[0.171, 0.220]	$<0.001$	**
ConFLICT-QA	CRAcc	+0.319	[0.302, 0.336]	$<0.001$	**
	SupportF1	+0.324	[0.307, 0.333]	$<0.001$	**
RegChange-QA	Ans2gramF1	+0.164	[0.117, 0.209]	$<0.001$	**
Stream-QA	UpdateF1	+0.017	[0.011, 0.023]	$<0.001$	**

Note: ** denotes statistical significance at the $p<0.001$ level.

. Improvements are statistically significant ($p<0.001$) across overall and task-specific metrics.

6.5. Visualization and Error Diagnosis

Figure 3 visualizes key metrics by task. Figure 4 summarizes step-level failure types, highlighting remaining bottlenecks such as citation alignment in long streaming trajectories and output formatting in conflict adjudication.

In Stream-QA, the drift trigger substantially reduces retrieval failures (RET_FAIL: $13.4\%\to 1.1\%$), indicating improved coverage under fast-evolving events. However, citation mismatch increases (CIT_MIS: $21.8\%\to 47.5\%$), suggesting that cross-snapshot citation alignment remains challenging when evidence is refreshed multiple times. In ConFLICT-QA, among the detected failure cases, formatting errors dominate (FMT_MIS $\approx 99\%$), motivating stronger structure constraints (e.g., constrained decoding) for verdict-style outputs.

6.6. Ablation Study

Table 5. Ablation results: impact of removing trigger-specific actions from the full system.

Method	RegChange	$\Delta$ vs. Full	ConFLICT	$\Delta$ vs. Full	Stream	$\Delta$ vs. Full
	Ans2gramF1		CRAcc		UpdateF1
LedgerRAG (Full)	0.6994	0.0000	0.9930	0.0000	0.7430	0.0000
w/o Gap	0.6850	−0.0144	0.9930	0.0000	0.7280	−0.0150
w/o Drift	0.6654	−0.0340	0.9930	0.0000	0.7018	−0.0412
w/o Conflict	0.6994	0.0000	0.8267	−0.1663	0.7430	0.0000

reports a clean trigger ablation on the final LedgerRAG system. The results confirm the targeted contributions of each governance module. First, removing conflict produces the largest drop in ConFLICT-QA: CRAcc decreases from 0.9930 to 0.8267 (−0.1663), demonstrating its critical role in authority-based conflict resolution. Second, removing drift specifically penalizes temporally sensitive tasks, reducing RegChange Ans2gramF1 (−0.0340) and Stream UpdateF1 (−0.0412), as the system fails to proactively recognize and refresh outdated evidence. Finally, removing gap yields consistent declines in both RegChange (−0.0144) and Stream (−0.0150), confirming that the gap trigger effectively ensures sufficient evidence coverage for complex generative tasks.

6.7. Human Evaluation and Case Study

We conduct pairwise blind human evaluation on 50 randomly sampled queries. Annotators prefer LedgerRAG in $66.0\%$ of cases, tie in $22.0\%$, and prefer the BM25 baseline in $12.0\%$ (Figure 5). The dominant preference reason is decisiveness: LedgerRAG is more willing to reject low-authority rumors and produce a clear correction with citations.

We further illustrate life cycle governance in a case study on breaking news (Figure 6). The system transitions from a gap phase (insufficient evidence) to a verified flash news update after an official statement and later issues a correction notice when updated numbers arrive while preserving an audit trail. To ensure the reliability of our evaluation, we measured inter-annotator agreement on this 50-sample subset, achieving a Cohen’s kappa of 0.4136, indicating moderate agreement.

6.8. System Deployment in a Newsroom CMS

To validate deployability beyond offline benchmarking, we integrate LedgerRAG into a newsroom CMS prototype as an editorial assistant. The integration follows a plug-in + service architecture: a CMS panel triggers claim extraction and verification, while an external service orchestrates retrieval, ledger updates, and governed generation. The ledger is persisted as a structured artifact (e.g., JSON) so that every revision is traceable to concrete evidence.

Latency measurement: Deployment latency is measured as wall clock time from a CMS panel request to the returned governed answer, including retrieval, ledger updates, and LLM generation (single-query concurrency; no batching).

Figure 7 shows the CMS assistant panel (mock UI). The panel exposes claim-level verification status, time validity, authority cues, and an editor checklist for unresolved items.

As summarized in

Table 6. CMS deployment latency summary (prototype; wall clock; single-query; averaged per scenario).

Scenario	Avg. Retrieval Calls	Avg. End-to-End Latency	Operational Notes
RegChange-QA	2.27	1101 ms	Drift-triggered version check.
ConFLICT-QA	1.00	157 ms	Authority-aware verdict.
Stream-QA	4.61	1028 ms	Frequent refresh for updates.

the prototype remains interactive: conflict verdicts are typically sub-second, while the most update-intensive streaming scenario averages around ∼1 s end-to-end. In practice, editorial workflows often validate only modified paragraphs, and ledger snapshots can be cached per draft to further reduce repeated computation.

6.9. Addressing Potential Reviewer Concerns

Fairness of baseline comparisons: To isolate the benefit of ledger-driven control, we fix the backbone generator and retriever across methods whenever possible. We implement the Self-RAG baseline as a prompting-based control policy on the same backbone rather than relying on a different fine-tuned 7B model to avoid conflating retrieval control with model capacity.

Evaluation bias from LLM judges: LLM-as-a-Judge can introduce prompt or model bias. We mitigate this by (i) using a fixed judging prompt and deterministic decoding, (ii) validating judge–human correlation ($r=0.87$), and (iii) supplementing with blind human preference evaluation. We release judging prompts and anonymized judge outputs to enable independent auditing.

Why is ConFLICT-QA performance near-perfect? The ConFLICT-QA task emphasizes authority-aware adjudication, where official clarifications often exist and decisively resolve conflicts. Our authority–time ordering and counter-evidence search are well matched to this setting, and they substantially improve over both BM25 ($\mathrm{CRAcc}=0.393$) and the prompted Self-RAG baseline ($\mathrm{CRAcc}=0.753$). We explicitly note that performance may degrade when authoritative sources are missing or delayed, and we discuss this as a limitation and future direction (Section 7.2). To provide further transparency regarding this near-perfect accuracy (0.993), we provide a detailed confusion matrix and label distribution of the conflict adjudication in Figure A1 in the Appendix A.

GraphRAG comparison: GraphRAG is designed for global corpus-level sensemaking but requires building and maintaining graphs and summaries. We position LedgerRAG as complementary: it provides low-latency, ledger-based governance suitable for streaming contexts where graph rebuilding is impractical.

GraphRAG complexity and latency comparison: GraphRAG is designed for global corpus-level sensemaking but requires building and maintaining explicit community summaries. In a dynamic streaming context, updating a GraphRAG index incurs an offline complexity of $\mathcal{O}\left(\right|V|+|E\left|\right)$, meaning the LLM calls for entity extraction and summarization per time slice. By contrast, LedgerRAG operates on a standard inverted or vector index with a $O\left(1\right)$ update cost per new document, deferring the reasoning overhead to inference time ($\mathcal{O}(S_{max}\times k)$ retrieval steps). While GraphRAG-lite is competitive offline (e.g., $0.7486$ on Stream UpdateF1), LedgerRAG provides low-latency (<1.2 s end-to-end), ledger-based governance suitable for high-frequency streaming contexts where continuous graph rebuilding is computationally prohibitive.

7. Conclusions and Limitations

7.1. Conclusions

We presented LedgerRAG, a governance-aware chain-of-retrieval framework for dynamic knowledge scenarios. By introducing an explicit evidence ledger and a lightweight gap/drift/budget trigger controller, the system transforms RAG from passive retrieval to auditable, stateful governance. On DyK-Bench, LedgerRAG significantly improves citation support quality, temporal validity, and governance utility scores with controlled retrieval cost and achieves large gains in conflict adjudication. A newsroom CMS prototype further demonstrates interactive latency and practical workflow integration.

7.2. Limitations and Future Work

Despite strong results, several limitations remain:

(1)

Citation alignment under streaming refresh: Frequent refresh improves freshness but can increase citation mismatches across snapshots. As observed in Stream-QA, while the drift trigger substantially reduces retrieval failures (RET_FAIL: 13.4% to 1.1%), citation mismatch increases significantly (CIT_MIS: 21.8% to 47.5%). This explicit trade-off suggests that cross-snapshot citation alignment remains challenging when evidence is refreshed multiple times. Future work will explore stronger doc_id anchoring and pointer-based citation alignment.

(2)

Structured verdict formatting: Conflict scenarios require structured outputs (e.g., JSON verdicts). Formatting errors remain prevalent; constrained decoding or structure-aware fine-tuning may be required.

(3)

Retriever ceiling: The framework is retriever-agnostic but still bounded by retriever recall, especially for PDF notices or multimodal evidence. Integrating dense or late-interaction retrievers [30] validated on benchmarks like BEIR [31] is a promising direction. Alternatively, parametric knowledge updates via model editing offer a complementary path to RAG for handling drift.