High-Information-Density Knowledge Graph Retrieval Augmentation for Large Language Models

Abstract

Retrieval-Augmented Generation (RAG) mitigates hallucination and knowledge staleness in large language models (LLMs) by injecting external knowledge into the context window, yet its effectiveness is rigidly constrained by attention dilution and inference cost. Existing knowledge-graph-enhanced methods follow a sequential “retrieve-then-compress” paradigm in which retrieval and compression are optimized independently, preventing the compressor from rectifying redundancy and noise introduced during retrieval and making it difficult to achieve globally efficient information allocation under a limited token budget. To address this paradigm-level limitation, we propose LOKG (LLM-Oriented Knowledge Graph), a high-information-density knowledge graph retrieval augmentation framework that uses token-level information density maximization as a shared design objective across schema compression, subgraph retrieval, and triple encoding. LOKG comprises three synergistic modules: Semantic Compression of Ontology Schema (SCOS) offline merges semantically overlapping relations by fusing normalized mutual information, Jaccard co-occurrence, and TransE embedding similarity under a conditional-entropy-based adaptive threshold; Topology-Aware Indexing and Multi-granularity Retrieval (TAIMR) computes topology-aware multi-hop path scores using relation-aware graph attention and node information gain, then applies late query–answer semantic re-ranking in the E5 space; Information-Theoretic Triple Compression (ITTC) formulates granularity assignment as a multiple-choice knapsack problem, adaptively allocating compact, balanced, or verbose representations to candidate triples to maximize injected information under the budget constraint. Experiments on four KGQA evaluation settings from three benchmark families (CWQ, WebQSP, PathQuestion-2hop, and PathQuestion-3hop) demonstrate that under a single-LLM-call constraint, LOKG achieves a superior accuracy–efficiency trade-off. Ablation studies further verify the orthogonality and synergy of the three modules. LOKG offers a transparent, auditable, and low-cost solution for knowledge graph retrieval-augmented generation under token budget constraints.

1. Introduction

Retrieval-Augmented Generation (RAG) has emerged as the dominant paradigm for mitigating model hallucination and knowledge staleness by dynamically injecting external knowledge into the context window of large language models [1,2]. However, the practical effectiveness of RAG is heavily constrained by the context window’s capacity to effectively utilize retrieved content, which faces two rigid constraints. The first is the attention dilution effect: research shows that when input exceeds 2K to 4K tokens, the model’s ability to locate key information in the middle begins to degrade, with a significant decline in localization accuracy, exhibiting a “lost in the middle” phenomenon [3], leading to rapidly diminishing marginal returns. The second is the inference cost constraint: in practical deployment, the response latency and API costs from multiple model invocations make production deployment challenging. Advanced methods represented by Think-on-Graph [4] can improve accuracy through multi-round exploration, but require several to dozens of model calls per query, making inference costs an order of magnitude higher than methods requiring minimal calls. These two constraints jointly define a core optimization problem: under limited token budgets, how to make each token unit injected into the context window carry as much effective information as possible.

Current RAG systems predominantly use unstructured text as the primary knowledge source, but text fragments inherently suffer from semantic fragmentation and information sparsity—retrieved passages often contain only a few sentences directly relevant to the query, with the remaining content occupying precious token budget while contributing limited information gain. Knowledge graphs, with their explicit entity-relation structured modeling capability, can provide semantically more precise and information-dense knowledge sources for RAG [5], offering structural advantages for injecting more effective knowledge within limited budgets. However, naive triple serialization methods result in substantial tokens being occupied by semantically overlapping relation types, redundant natural language modifiers, and low-information-gain structural noise, making the effective knowledge amount that can be injected within limited budgets far below theoretical possibilities. To improve effective utilization of context windows, the research community has made various attempts to integrate knowledge graphs with large language models [5,6], but existing methods all expose performance bottlenecks in different dimensions. GraphRAG [7] first uses the Leiden community detection algorithm to hierarchically partition the entity-relation graph to retrieve relevant communities, then calls LLMs to pre-generate summaries for each community as compressed knowledge representations. This design solidifies community detection and summary generation as two independent optimization stages—the former pursues modularity maximization, the latter pursues semantic fidelity—with the community set already locked at the retrieval stage, subsequent compression can only passively accept it, unable to dynamically adjust retrieval scope according to token budget. Think-on-Graph (ToG) [4] breaks through this fixed retrieval granularity bottleneck: it abandons GraphRAG’s pre-partitioned community strategy, using an LLM as a reasoning agent to execute beam search on the knowledge graph, with each iteration having the LLM evaluate current path quality and dynamically select the next hop. This improvement resolves the fixed granularity bottleneck, but at the cost of more severe constraint violations—several to dozens of LLM calls per query make inference costs an order of magnitude higher than methods requiring minimal calls, making it difficult to meet deployment requirements under the hard constraint of “minimal call count.” KAPING [8] responds to ToG’s invocation cost problem from the opposite direction: it abandons LLM agent-style exploration, returning to semantic similarity-based top-K triple retrieval, compressing the entire KGQA process into one retrieval and one LLM call. This minimalist design achieves the extreme in call count constraints, but exposes new bottlenecks in token efficiency dimensions—K value as the sole compression lever has no explicit correlation with information gain; when K increases to the information density saturation point, the marginal contribution of additionally injected triples to answer generation tends to diminish, while its framework has not yet introduced explicit regulation mechanisms for information density saturation. SubgraphRAG [9] and LightRAG [10] attempt to break through these limitations from the directions of retrieval granularity and system efficiency, respectively, but their technical routes remain on the extension line of retrieval strategy optimization, not touching the deeper paradigmatic question of “should retrieval and compression be separated.” DPR [11], as a foundational work in dense retrieval, though not directly targeting knowledge graphs, has profoundly influenced the architectural design of the above methods through its established “retrieve-rank-truncate” pipeline. This paradigmatic separation is especially prominent in methods specifically targeting token efficiency optimization—these methods elevate compression as an independent research problem, further widening the stage isolation between retrieval and compression. LLMLingua [12] proposes a hierarchical prompt compression strategy based on perplexity, executing token-level iterative compression through perplexity estimation of small language models, achieving up to a 20x prompt compression ratio on reasoning datasets like GSM8K with minimal performance loss. RECOMP [13] introduces an intermediate compression step between retrieval and generation in the RAG pipeline for the first time, proposing both extractive and abstractive compressor schemes, achieving compression ratios as low as 6% on datasets like NQ. KnowledGPT [14] proposes a unified framework for deep integration of LLMs with knowledge bases, supporting injection of retrieval results into LLM context in structured pseudo-code format, achieving higher token utilization efficiency compared with naive triple concatenation; however, its encoding format is a predefined fixed template, lacking the capability for adaptive granularity selection based on query context and token budget. Regardless of how sophisticated the compression techniques are, their operating targets are always content sets that have been solidified by the retrieval stage—compressors struggle to backtrack and correct choices already made by retrievers, and also struggle to compensate for low-quality information introduced due to a lack of token efficiency awareness at the retrieval stage. The above analysis shows that the “retrieve-then-compress” serial paradigm has structural limitations: retrieval maximizing semantic coverage and compression maximizing information retention rate each optimize independently, with objective functions of the two stages lacking joint constraints, making it difficult to approach the global optimum; moreover, the compression stage struggles to backtrack and correct redundancy and noise already selected by the retrieval stage, only capable of local patching within limited space.

Addressing these paradigmatic deficiencies, this paper proposes LOKG (LLM-Oriented Knowledge Graph), a high-information-density knowledge graph retrieval augmentation framework for LLM retrieval. Unlike the “retrieve-then-compress” serial paradigm, LOKG establishes token information density maximization as a shared design objective throughout the entire pipeline of knowledge graph schema compression, triple encoding, and retrieval, coordinating retrieval and compression through three interlocking modules under a shared information-density objective:

(1) SCOS (Semantic Compression of Ontology Schema) in the offline construction phase automatically identifies and merges semantically overlapping relation types based on tri-signal fusion similarity of normalized mutual information, Jaccard co-occurrence, and TransE relation embeddings, eliminating representational redundancy at the ontology schema level before retrieval occurs;

(2) ITTC (Information-Theoretic Triple Compression Encoding) in the online retrieval phase embeds encoding decisions within the retrieval process based on entropy-minimizing encoding order selection and multi-granularity entity representation allocation via the multiple-choice knapsack problem, maximizing the information content of injected knowledge under fixed token budget constraints;

(3) TAIMR (Topology-Aware Indexing and Multi-granularity Retrieval) computes topology-aware multi-hop path scores by fusing node information gain with relation attention weights, and applies a late query–answer semantic re-ranking term in the E5 space.

The synergistic logic among the three layers is as follows: SCOS reduces vocabulary redundancy at the schema level to lower ITTC’s encoding cardinality; ITTC maximizes information density per token at the encoding level to efficiently utilize candidate subgraph space retrieved by TAIMR; TAIMR provides topology-aware candidate ranking at the retrieval level to improve the proportion of effective knowledge in ITTC’s encoding window. Through the above three-layer synergistic design, LOKG transforms token efficiency optimization from an independent post-processing step after retrieval completion into a built-in constraint throughout the entire pipeline, providing a transparent, auditable, and low-deployment-cost solution for knowledge graph retrieval-augmented generation under token budget constraints.

2. Method

2.1. Framework Overview

The existing “retrieve-then-compress” serial paradigm places retrieval and compression in mutually independent optimization stages, making it difficult to achieve globally efficient information allocation under limited token budgets. To break through this limitation, this paper proposes LOKG (LLM-Oriented Knowledge Graph), a high-information-density knowledge graph retrieval augmentation framework for large language models, which adopts token information density maximization as a shared design objective throughout ontology schema compression, triple encoding, and subgraph retrieval, thereby coordinating knowledge selection, representation compression, and context injection while preserving the distinct roles of the offline and online stages.

The LOKG framework is shown in Figure 1. This framework embeds compression principles throughout the entire pipeline of knowledge retrieval and encoding. The framework reduces relation vocabulary scale through schema integration before retrieval, utilizes topological structure to locate high-value paths during retrieval, and adaptively allocates expression granularity based on information value at the encoding stage, forming a three-level compression cascade from coarse to fine.

Figure 1. Overall architecture of the LOKG framework.

The framework consists of three synergistic modules: Semantic Compression of Ontology Schema (SCOS) identifies semantically overlapping relations in the offline phase by fusing normalized mutual information, Jaccard coefficient, and embedding cosine similarity, executes merging using a conditional entropy adaptive threshold mechanism, and outputs a compressed knowledge graph and relation mapping. Topology-Aware Indexing and Multi-granularity Retrieval (TAIMR) constructs an index based on the compressed schema, preferentially retrieves reasoning paths traversing information hubs through pre-computed node information gain combined with relation-aware graph attention networks, and provides candidate triple sets to downstream. Information-Theoretic Triple Compression Encoding (ITTC) models online encoding as a multiple-choice knapsack problem, allocates each candidate triple optimally among compact, balanced, and verbose granularities, solves through dynamic programming to maximize information density under a fixed token budget, and finally outputs a token sequence satisfying budget constraints for injection into an LLM. The three modules work synergistically to achieve efficient knowledge injection and reasoning enhancement under strict token constraints.

2.2. Problem Definition

The knowledge graph-enhanced retrieval-augmented generation problem can be described as follows: given a natural language query q, the system retrieves relevant knowledge from an external knowledge graph and encodes it as a token sequence injected into the large language model context window to assist in answering. The core challenge is that the context window is constrained by both token capacity and attention dilution, with a limited budget available for injection and the token consumption of each LLM call directly translates to deployment cost. Therefore, the system needs to maximize the ratio of answer accuracy to token consumption under strict token budget and call count constraints—i.e., amortized token efficiency.

Let the external knowledge graph be $\mathcal{G}=(\mathcal{E},\mathcal{R},\mathcal{T})$, where $\mathcal{E}$ and $\mathcal{R}$ are the entity set and relation set, respectively, and $\mathcal{T}\subseteq \mathcal{E}\times \mathcal{R}\times \mathcal{E}$ is the triple set. Each triple is represented by Equation (1).

$$(s,r,o)\in \mathcal{T},s,o\in \mathcal{E},r\in \mathcal{R}$$

(1)

where s and o are the head entity and tail entity, respectively, and r is the relation. Let the large language model be $\mathcal{M}$, the token budget be B, and the maximum allowed LLM calls per query be $N_{\max }$. The system retrieves subgraph ${\mathcal{G}}_q$ from $\mathcal{G}$ and encodes it as token sequence $\mathbf{x}$ ($\left|\mathbf{x}\right|\le B$), and the model generates answer ${\widehat{a}}_q=\mathcal{M}(q,\mathbf{x})$ accordingly. Let $n_q$ LLM calls be executed to complete a single query, with total token consumption $T_{\mathrm{total}}\left(q\right)={\sum }_{j=1}^{n_q}(T_{\mathrm{in}}^{\left(j\right)}+T_{\mathrm{out}}^{\left(j\right)})$. The optimization objective is expressed as Equation (2).

$$\underset{{\mathcal{G}}_q,\mathbf{x}}{\max }{\displaystyle \frac{\mathrm{F}1(q,\mathcal{M}(q,\mathbf{x}\left)\right)}{T_{\mathrm{total}}\left(q\right)}}\mathrm{s}.\mathrm{t}.\left|\mathbf{x}\right|\le B,n_q\le N_{\max }$$

(2)

2.3. Semantic Compression of Ontology Schema (SCOS)

The traditional knowledge graph construction paradigm prioritizes completeness of knowledge coverage, with ontology schemas often containing numerous relation types with highly overlapping semantics. When these relations are serialized as token sequences injected into LLM context, semantic redundancy directly translates to token waste. The design motivation of SCOS is precisely to eliminate this structural redundancy at the ontology schema level before retrieval occurs. Denote the original relation set $\mathcal{R}$ in the problem definition as ${\mathcal{R}}_0$ to distinguish relation vocabularies before and after compression. SCOS operates on the original knowledge graph $\mathcal{G}=(\mathcal{E},{\mathcal{R}}_0,\mathcal{T})$, automatically identifying and merging semantically overlapping relations in the relation vocabulary ${\mathcal{R}}_0$ to output the compressed knowledge graph ${\mathcal{G}}^{*}=(\mathcal{E},{\mathcal{R}}^{*},{\mathcal{T}}^{*})$ and canonical mapping $\varphi :{\mathcal{R}}_0\to {\mathcal{R}}^{*}$, such that $|{\mathcal{R}}^{*}| \le |{\mathcal{R}}_0|$. The entire process executes completely offline, independent of any LLM calls.

Tri-signal fusion relation similarity. Judging whether two relations semantically overlap cannot rely on a single signal—relations with close distance in embedding space may play drastically different structural roles in graph topology, while relations with highly overlapping entity co-occurrence sets may have essential differences at the semantic level. To this end, SCOS computes the semantic similarity between any two relations $r_i,r_j\in {\mathcal{R}}_0$ by weighted fusion of three complementary signals:

$$\mathrm{sim}(r_i,r_j)=w_1·\mathrm{NMI}(r_i,r_j)+w_2·\mathrm{Jaccard}(r_i,r_j)+w_3·\cos ({\mathbf{e}}_{r_i},{\mathbf{e}}_{r_j})$$

(3)

where $\mathrm{NMI}(r_i,r_j)$ is the normalized mutual information [15] between two relations on the joint distribution of (head entity type, tail entity type), measuring structural role similarity of relations—if two relations always connect entity pairs of the same type, their structural roles are highly similar; $\mathrm{Jaccard}(r_i,r_j)$ is the Jaccard co-occurrence coefficient [16] between two relations on (head entity set ∪ tail entity set), capturing usage pattern overlap of relations from the surface signal of entity sharing; ${\mathbf{e}}_r\in {\mathbb{R}}^d$ is the relation embedding vector trained by TransE [17], and $\cos (·,·)$ is cosine similarity, providing distance measure in distributed semantic space. The three signals characterize relation similarity from orthogonal dimensions of statistical distribution, set co-occurrence, and vector semantics, respectively, with weight settings ensuring merging is triggered only when high similarity appears simultaneously across multiple dimensions, avoiding erroneous merging caused by single signal bias.

Conditional entropy-based adaptive merging threshold. To avoid forcibly merging relations with significant information structure differences, SCOS introduces an adaptive threshold mechanism. The core idea is as follows: the entity pair distribution connected by each relation r in the knowledge graph embodies the information structure carried by that relation. SCOS quantifies this information structure through the conditional entropy of the tail entity distribution:

$$H\left(r\right)=-\sum _{o\in \mathcal{E}}p(o\mid r)\log p(o\mid r)$$

(4)

where $p(o\mid r)=|\{s:(s,r,o)\in \mathcal{T}\}|/|\{(s,o):(s,r,o)\in \mathcal{T}\}|$ is the conditional probability of tail entity o given relation r. Higher $H\left(r\right)$ indicates that relation r connects more diverse tail entities with weaker semantic constraints. Based on $H\left(r\right)$, the merging trigger condition is defined as follows:

$$\mathrm{sim}(r_i,r_j)\ge \theta (r_i,r_j),\theta (r_i,r_j)=\min \left({\theta }_{\mathrm{base}}·(1+\epsilon ·{\delta }_{ij}),1.0\right)$$

(5)

where ${\theta }_{\mathrm{base}}$ is the base threshold, $\epsilon$ is the adaptive coefficient, and ${\delta }_{ij}=|H\left(r_i\right)-H(r_j)|/\max (H\left(r_i\right),H(r_j))$ is the relative difference in conditional entropy between the two relations. When ${\delta }_{ij}$ is large—i.e., although two relations are similar in embedding space or co-occurrence patterns, there is significant disparity in the diversity of entities they connect—the adaptive threshold $\theta$ rises accordingly, requiring stronger comprehensive similarity to trigger merging. This mechanism effectively prevents erroneous merging of highly specialized relations (low entropy, such as “place of birth”) with highly generalized relations (high entropy, such as “related to”).

Greedy hierarchical merging process. SCOS’s merging algorithm adopts a bottom-up greedy strategy, iteratively executing the following steps: each round selects from the current relation vocabulary the relation pair $(r_i^{*},r_j^{*})$ that satisfies the merging condition of Equation (5) with the highest similarity for merging, the newly generated relation cluster inherits all triples from the two original relations, and its TransE embedding takes the mean vector of the two original embeddings. After merging, the similarity between the new cluster and all remaining relations needs to be recalculated. When no relation pair satisfies the merging condition, the algorithm terminates. The time complexity of this process is $O\left(\right|{\mathcal{R}}_0{|}^2\log \left|{\mathcal{R}}_0\right|)$, dominated by initial relation pair similarity matrix computation.

Compression quality assessment. Each merging operation introduces certain information loss, with magnitude $(1-\mathrm{sim}(C_i,C_j))·\min (H\left(C_i\right),H(C_j))$, where $C_i,C_j$ are the two relation clusters being merged. SCOS accumulates information loss across all merging steps and normalizes it to the information loss rate ${\mathcal{L}}_{\mathrm{info}}$, then defines normalized information density $\widehat{\rho }$ to comprehensively evaluate compression quality:

$$\widehat{\rho }={\displaystyle \frac{1-{\mathcal{L}}_{\mathrm{info}}}{|{\mathcal{R}}^{*}|/|{\mathcal{R}}_0|}}$$

(6)

This metric is defined as the information retained per relation after schema compression, normalized by the proportion of relations retained. It is meaningful when relation merging occurs (i.e., $|{\mathcal{R}}^{*}| < |{\mathcal{R}}_0|$). If SCOS does not trigger any merging ($|{\mathcal{R}}^{*}| = |{\mathcal{R}}_0|$), then ${\mathcal{L}}_{\mathrm{info}}=0$ and $\widehat{\rho }\triangleq 1$ is defined as the boundary case of “zero compression, zero information loss”. Under this definition, $\widehat{\rho }>1$ indicates that the information retained per relation exceeds the pre-compression reference level, whereas $\widehat{\rho }<1$ indicates that information loss outweighs the reduction in relation count. The output of SCOS—the compressed relation vocabulary ${\mathcal{R}}^{*}$—directly serves as the relation cardinality for the ITTC encoding stage and the graph-structure foundation for TAIMR index construction.

2.4. Information-Theoretic Triple Compression Encoding (ITTC)

After SCOS completes offline compression of the relation vocabulary, ITTC undertakes the core task of encoding candidate subgraphs into sequences satisfying token budget constraints during the online retrieval phase. Unlike naive triple concatenation strategies, the key innovation of ITTC is as follows: it does not encode all triples in a uniform format, but rather adaptively allocates encoding granularity based on the information value of each triple, enabling high-value triples to obtain verbose representation while low-value triples are compressed into compact form, thereby maximizing the total information amount of injected knowledge under a fixed token budget. ITTC receives the candidate triple set $\mathcal{S}=\{t_1,t_2,\dots ,t_N\}$ and token budget B returned by TAIMR retrieval, and outputs the encoded token sequence $\mathbf{x}$ ($\left|\mathbf{x}\right|\le B$).

Prefix conditional entropy minimization encoding order. Each triple $t=(s,r,o)$ contains three components: subject s, relation r, and object o. Different encoding orders affect LLMs’ understanding efficiency—when low-uncertainty components are encoded first, subsequent components have stronger contextual constraints, and LLMs can complete understanding with fewer attention resources. ITTC adopts the prefix conditional entropy minimization criterion: given encoding order $\sigma$, its prefix uncertainty is defined as

$$U\left(\sigma \right)={\displaystyle \sum _{i=1}^2}H(x_{\sigma \left(i\right)}\mid x_{\sigma \left(1\right)},\dots ,x_{\sigma (i-1)})$$

(7)

i.e., the accumulated conditional entropy of the first $k-1$ steps excluding the last component, where k = 3 is the number of components in a triple. By the chain rule, $H(S,R,O)=U\left(\sigma \right)+H(x_{\sigma \left(3\right)}\mid x_{\sigma \left(1\right)},x_{\sigma \left(2\right)})$; minimizing $U\left(\sigma \right)$ is equivalent to maximizing the conditional entropy of the last component—i.e., concentrating LLM’s attention resources on the last component with maximum information content, while assigning prefix anchor points to low-entropy, easily-constrained components. ITTC enumerates all $3!=6$ permutations of ${\mathrm{\Pi }}_3$, selecting

$${\sigma }^{*}=\arg \underset{\sigma \in {\mathrm{\Pi }}_3}{\min }U\left(\sigma \right)$$

(8)

On Freebase-derived datasets, ${\sigma }^{*}$ consistently is $r\to s\to o$, i.e., relation as the first encoding component. This result can be explained by the structural characteristics of knowledge graphs: in typical knowledge graphs, the relation set size is much smaller than the entity set ($|{\mathcal{R}}^{*}| \ll |\mathcal{E}|$), therefore, the marginal entropy $H\left(r\right)$ of relations takes the minimum value among the three components. When using low-entropy components as anchor points, the uncertainty of subsequent components is maximally constrained—for example, knowing the relation is “born in” necessarily means the subject is a person and the object is a location, greatly reducing the entity search space. Conversely, giving an entity name first has much weaker constraining power on relation types, as the same entity can participate in multiple types of relations.

Multi-granularity encoding and information value. After determining the encoding order, ITTC provides K encoding granularities for each candidate triple $t_i$: compact granularity retains only key identifiers (entity ID or core name), with minimum token overhead but losing attribute information; balanced granularity retains core attributes (name, type, main description), achieving balance between information amount and token overhead; verbose granularity retains complete description (including aliases, additional attributes, etc.), with maximum information amount but also highest token overhead. Each granularity $k\in \{1,\dots ,K\}$ corresponds to a (token overhead, information value) tuple $(c(t_i,k),v(t_i,k))$, where $c(t_i,k)$ is the actual token count under that granularity (computed by the LLM tokenizer), and $v(t_i,k)$ is computed based on relation conditional entropy and the granularity retention coefficient:

$$v(t_i,k)={\beta }_k·\alpha \left(t_i\right)·H(O\mid R=r_i)$$

(9)

where $H(O\mid R=r_i)$ is the tail entity conditional entropy corresponding to relation $r_i$ (computed by Equation (4)), reflecting the diversity of tail entities connected by that relation, i.e., the information richness carried by the triple. ${\beta }_k$ is the granularity retention coefficient, with ${\beta }_1=0.5$ (compact), ${\beta }_2=0.8$ (balanced), ${\beta }_3=1.0$ (verbose), reflecting the degree to which different encoding granularities retain triple information. $\alpha \left(t_i\right)$ is the query relevance boost factor, defined as

$$\alpha \left(t_i\right)=\left\{\begin{array}{cc}{\lambda }_{\mathrm{boost}},\hfill & \{s_i,o_i\}\cap {\mathcal{E}}_q\ne ⌀\hfill \\ 1,\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(10)

where ${\mathcal{E}}_q={\mathcal{E}}_{\mathrm{query}}\cup {\mathcal{E}}_{\mathrm{cand}}$ is the union of the query entity set and the TAIMR coarse-filter candidate entity set, and ${\lambda }_{\mathrm{boost}}\ge 1$ is the boost intensity hyperparameter. $\alpha \left(t_i\right)$ ensures that triples directly related to the query obtain preferential retention in MCKP solving, avoiding being squeezed out of the budget range by distant noise.

MCKP global optimal allocation. Under token budget B constraint, ITTC formalizes the granularity allocation problem as a multiple-choice knapsack problem (MCKP) [18,19]:

$$\underset{{\left\{k_i\right\}}_{i=1}^N}{\max }{\displaystyle \sum _{i=1}^N}v(t_i,k_i)\mathrm{s}.\mathrm{t}.{\displaystyle \sum _{i=1}^N}c(t_i,k_i)\le B,k_i\in \{1,\dots ,K\}$$

(11)

where $k_i$ is the granularity level assigned to the i-th triple. Unlike standard 0–1 knapsack, MCKP requires each triple to select exactly one granularity (cannot be skipped), which ensures all candidate triples appear in the encoding sequence at least in compact granularity, avoiding information breakage caused by completely discarding certain triples. Solving adopts classic dynamic programming: maintaining a one-dimensional state array of size $B+1$, scanning each triple one by one, with each triple selecting among its K granularity options the level that maximizes cumulative information value. Time complexity is $O(N·B·K)$, which can be completed in milliseconds for typical configurations ($N\le 20$, $B=2048$, $K=3$). When budget B exceeds 4096, ITTC automatically switches to a greedy algorithm based on value density ($v/c$) sorting to avoid DP’s memory overhead. When $T_{\mathrm{naive}}\le 0.7B$, ITTC directly outputs naive encoding; only when $T_{\mathrm{naive}}>0.7B$ does it activate MCKP solving, aligning granularity allocation with actual compression needs.

2.5. Topology-Aware Indexing and Multi-Granularity Retrieval (TAIMR)

TAIMR, as the upstream module providing high-quality candidate triples to ITTC, undertakes the task of precisely locating the most query-relevant subgraph from the entire knowledge graph. Unlike flat retrieval based on semantic similarity, TAIMR fully utilizes topological structure information of the knowledge graph—considering not only the degree of semantic matching between entities and queries, but also evaluating the information hub status of intermediate nodes traversed by candidate paths in the graph topology. Given query entity $q_e\in \mathcal{E}$ identified from query q, maximum hop count L, and candidate count $K_{\mathrm{top}}$, TAIMR outputs ranked candidate triple set $\mathcal{S}=\{t_1,\dots ,t_{K_{\mathrm{top}}}\}$ and their scores.

Node information gain. Before retrieval, TAIMR first pre-computes a scalar information gain $\mu \left(v\right)$ for each node v in the knowledge graph, used to measure the information contribution potential of that node as an intermediate node in reasoning paths:

$$\mu \left(v\right)=\left(1+\log ({\mathrm{deg}}_{\mathrm{in}}\left(v\right)+1)\right)·\mathrm{div}\left(v\right)$$

(12)

where ${\mathrm{deg}}_{\mathrm{in}}\left(v\right)$ is the in-degree of node v, and $\mathrm{div}\left(v\right)=|\{r:(u,r,v)\in \mathcal{T}\}|/|{\mathcal{R}}^{*}|$ is the incoming edge relation type diversity of v (normalized to $[0,1]$). The design intuition of $\mu \left(v\right)$ comes from the concept of hub nodes in information retrieval: if a node is pointed to by multiple types of relations (high $\mathrm{div}$) and has large in-degree (high ${\mathrm{deg}}_{\mathrm{in}}$), then it is at the information convergence point of the knowledge graph, and reasoning paths passing through this node are more likely to aggregate contextual knowledge from multiple semantic dimensions, contributing more to the final answer. The logarithmic function $\log (·+1)$ moderately suppresses the growth of in-degree, avoiding excessive inflation of information gain for super-high-degree nodes (such as the “United States” entity in Freebase). $\mu \left(v\right)$ is pre-computed for the entire graph and cached during the offline construction phase, queried with $O\left(1\right)$ overhead during online retrieval.

Relation-aware graph attention network. TAIMR uses a 2-layer graph attention network (GAT) [20] to learn topology- and relation-aware attention weights for the edges in the knowledge graph. Unlike standard GAT, which considers only source and target node features, TAIMR explicitly introduces the relation embedding ${\mathbf{e}}_r$ into the attention computation, enabling the attention weights to capture both node semantics and relation-type information. For edge $(u,v,r)$ in the knowledge graph, the attention coefficient is computed as follows:

$$\alpha (u\to v\mid r)={\displaystyle \frac{\exp \left(\mathrm{LeakyReLU}({\mathbf{a}}^{\top }[\mathbf{W}{\mathbf{h}}_u\parallel \mathbf{W}{\mathbf{h}}_v\parallel {\mathbf{W}}_r{\mathbf{e}}_r\left]\right)\right)}{{\displaystyle \sum _{v^{\prime }\in \mathcal{N}\left(u\right)}\exp \left(\mathrm{LeakyReLU}({\mathbf{a}}^{\top }[\mathbf{W}{\mathbf{h}}_u\parallel \mathbf{W}{\mathbf{h}}_{v^{\prime }}\parallel {\mathbf{W}}_r{\mathbf{e}}_{r^{\prime }}\left]\right)\right)}}}$$

(13)

where ${\mathbf{h}}_v\in {\mathbb{R}}^{d_v}$ is the node embedding initialized by E5 [21], and ${\mathbf{e}}_r\in {\mathbb{R}}^{d_r}$ is the relation embedding provided by TransE from the SCOS stage. The node and relation representations are allowed to have different input dimensions. Accordingly, $\mathbf{W}\in {\mathbb{R}}^{d^{\prime }\times d_v}$ and ${\mathbf{W}}_r\in {\mathbb{R}}^{d^{\prime }\times d_r}$ are separate trainable projection matrices that map the two representations into the same $d^{\prime }$-dimensional attention space before concatenation. In the experimental configuration, $d_v=768$ and $d_r=64$. Thus, $\mathbf{a}\in {\mathbb{R}}^{3d^{\prime }}$ is the attention vector, $\parallel$ denotes vector concatenation, and $\mathcal{N}\left(u\right)$ is the neighbor set of node u. The projected relation representation ${\mathbf{W}}_r{\mathbf{e}}_r$ participates only in attention-coefficient computation, indirectly affecting information aggregation by modulating neighbor-message weights rather than directly entering the value vectors. Based on the computed attention weights, GAT aggregates neighboring information through a multi-head attention mechanism to update the node embeddings. The first GAT layer concatenates the multi-head outputs (output dimension $M\times d^{\prime }$):

$${\mathbf{h}}_v^{\prime }={{\displaystyle \parallel }}_{m=1}^M\mathrm{ELU}\left(\sum _{u\in \mathcal{N}\left(v\right)}{\alpha }_m(u\to v\mid r)·{\mathbf{W}}_m{\mathbf{h}}_u\right)$$

(14)

The second layer uses mean aggregation (output dimension $d^{\prime }$):

$${\mathbf{h}}_v^{″}=\mathrm{ELU}\left({\displaystyle \frac{1}{M}}{\displaystyle \sum _{m=1}^M}\sum _{u\in \mathcal{N}\left(v\right)}{\alpha }_m(u\to v\mid r)·{\mathbf{W}}_m{\mathbf{h}}_u^{\prime }\right)$$

(15)

where M is the number of attention heads, ${\alpha }_m$ is the attention coefficient of the m-th head, and ${\mathbf{W}}_m$ is the corresponding projection matrix.

Late fusion of topology and semantic relevance. TAIMR separates structural path scoring from query–answer semantic matching. The topology score is computed entirely in the graph representation space using relation-aware GAT attention and node information gain. For a path $p=[q_e,v_1,v_2,\dots ,v_k]$ ($k\le L$), with $v_0=q_e$, it is defined as follows:

$$S_{\mathrm{topo}}\left(p\right)={\displaystyle \sum _{i=0}^{k-1}}\log \alpha (v_i\to v_{i+1}\mid r_i)+{\lambda }_{\mathrm{topo}}{\displaystyle \sum _{i=1}^k}\log \mu \left(v_i\right)$$

(16)

where $r_i$ is the relation on the i-th edge and ${\lambda }_{\mathrm{topo}}$ is the topology information-gain weight. The topology score is used during BFS expansion and candidate-path ranking. Summation in log space avoids numerical underflow arising from multiplying edge-attention and node-gain factors.

The semantic score is computed independently in the E5 embedding space. For a candidate answer entity a, the query–answer semantic relevance is as follows:

$$S_{\mathrm{sem}}(q,a)=\cos \left(\mathrm{E}5\left(q\right),\mathrm{E}5\left(a\right)\right)$$

(17)

Both vectors in Equation (17) are produced by the same E5 encoder and, therefore, have the same dimensionality. The E5 vectors are not directly compared with the GAT-updated node representations; the two representation spaces interact only after being reduced to scalar scores.

Let $p_a$ denote the highest-scoring topology path associated with candidate answer entity a. During final answer ranking, TAIMR performs late fusion by adding the query–answer semantic bonus to the topology score:

$$S_{\mathrm{final}}(a\mid q)=S_{\mathrm{topo}}\left(p_a\right)+{\lambda }_q{S}_{\mathrm{sem}}(q,a)$$

(18)

where ${\lambda }_q$ is the query–answer semantic weight, set to ${\lambda }_q=1.0$ in this study. The semantic bonus is added only at the final ranking stage and does not affect the topology score used during hop expansion, thereby avoiding semantic-score leakage into the structural search process.

TAIMR ranks reachable paths by $S_{\mathrm{topo}}\left(p\right)$ during structural search, retains the best-scoring supporting path $p_a$ for each candidate answer entity a, and applies Equation (18) during final answer ranking. The supporting triples of the retained candidate paths form the candidate set $\mathcal{S}$ (i.e., the retrieved subgraph ${\mathcal{G}}_q$ in the problem definition), which is passed to ITTC for encoding. To reduce online retrieval latency, the construction phase pre-computes and caches top-$K_{\mathrm{top}}$ candidate paths for query entity $(\mathrm{head}, \mathrm{relation})$ pairs involved in experiments; in actual deployment, a lazy indexing strategy (caching results after the first query computation) can be adopted to control offline overhead. During online retrieval, cached results are directly retrieved, keeping the single-query retrieval latency at the millisecond level. The path cache is constructed exclusively from the knowledge-graph triples, keyed by head entity and relation, and never incorporates any answer-side or gold-label information, so it introduces no label leakage; on a cold cache, candidate paths are computed on demand before being cached at millisecond-level latency.

3. Experiments

3.1. Experimental Datasets

To verify LOKG’s applicability across knowledge graphs of different scales, this paper selects three levels of KGQA datasets to constitute an evaluation system, covering large, medium, and small knowledge graph scales. Specific statistics for each dataset are shown in Table 1.

Table 1. Dataset statistics.

Dataset	KG Source	Test Size	Avg Subgraph Triples	Reasoning Depth
CWQ	Freebase	3 531	3 841	2–4 hops
WebQSP	Freebase	1 628	1 513	1–2 hops
PQ-2hop	Synthetic	192	4	2 hops
PQ-3hop	Synthetic	521	9	3 hops

Large KG—CWQ [22] (ComplexWebQuestions): Constructed through automatic expansion and manual verification based on WebQuestionsSP, with knowledge source being Freebase [23]. Questions are dominated by complex reasoning operations such as composition, conjunction, and comparison, with most requiring 2–4 hop relation chains to complete answer retrieval. Relation types are numerous with significant semantic overlap (e.g., “place of birth” and “birth city” coexist).

Medium KG—WebQSP [24] (WebQuestionsSP): Constructed based on real user natural language queries collected by Google Suggest API, with the knowledge source also being Freebase. Questions are dominated by 1–2 hop reasoning, covering broad commonsense domains such as people, places, education, sports, government institutions, and business.

Small KG—PathQuestion-2hop/PathQuestion-3hop [25]: Synthetic multi-hop path reasoning datasets constructed from Freebase subsets through template generation and synonym rewriting. Questions have chain path query structure of the form $A\stackrel{R_1}{\to }{E}_{\mathrm{mid}}\stackrel{R_2}{\to }{E}_{\mathrm{ans}}$, where A is the start entity, $R_1$ and $R_2$ are relations, $E_{\mathrm{mid}}$ is the intermediate entity, and $E_{\mathrm{ans}}$ is the answer entity. Queries cover commonsense relation chains such as person birthplace, organization headquarters location, and movie director.

All experiments are evaluated on complete test sets of each dataset without sub-sampling.

3.2. Evaluation Metrics

This paper adopts the following metrics for multi-dimensional evaluation:

(a) Hits@1 (H@1): Top-1 hit rate, measuring the accuracy of the model’s first-choice answer, defined as

$$\mathrm{H}@1={\displaystyle \frac{\left|\{q\in Q:{\widehat{a}}_q=a_q\}\right|}{\left|Q\right|}}$$

where Q is the test query set, ${\widehat{a}}_q$ is the model’s first-choice predicted answer for query q, and $a_q$ is the corresponding ground truth answer.

(b) F1: F1 score between the predicted answer set and the ground-truth answer set, comprehensively evaluating precision and recall, defined as

$$\mathrm{F}1={\displaystyle \frac{2|\widehat{\mathcal{A}}\cap \mathcal{A}|}{|\widehat{\mathcal{A}}|+|\mathcal{A}|}}$$

where $\widehat{\mathcal{A}}$ is the model’s predicted answer set and $\mathcal{A}$ is the ground truth answer set.

(c) TCR (Token Compression Ratio): Token compression ratio, defined as

$$\mathrm{TCR}={\displaystyle \frac{T_{\mathrm{after}}}{T_{\mathrm{naive}}}}$$

where $T_{\mathrm{after}}$ is the token count actually injected into the context window by each method, and $T_{\mathrm{naive}}$ is the token count of the same candidate subgraph under naive triple concatenation without truncation. For LOKG, $T_{\mathrm{after}}$ is the MCKP multi-granularity encoding; for baselines without MCKP (e.g., KAPING, GraphRAG, ToG), $T_{\mathrm{after}}$ is their own retrieved results serialized and truncated to the budget, so their TCR reports the retention ratio of their retrieved candidates under the budget.

(d) P@1K: Amortized token efficiency, defined as

$$\mathrm{P}@1\mathrm{K}={\displaystyle \frac{\mathrm{F}1\times 1000}{T_{\mathrm{total}}}}$$

where $T_{\mathrm{total}}$ is the total API token consumption per query (including all LLM calls). This metric comprehensively measures the trade-off between accuracy and token cost, serving as the core efficiency metric of this paper.

(e) H@1K: Amortized token efficiency for first-choice accuracy, defined as

$$\mathrm{H}@1\mathrm{K}={\displaystyle \frac{\mathrm{H}@1\times 1000}{T_{\mathrm{total}}}}$$

measuring top-1 hit rate return per 1000 tokens consumed, complementary to P@1K—the former emphasizes first-choice accuracy, the latter emphasizes overall quality of answer sets.

(f) tok/q: Average total token consumption per query, i.e., sum of input and output tokens across all LLM calls divided by query count, in tokens/query, directly reflecting deployment cost of methods.

(g) LLM calls/q: Average LLM call count per query, serving as a key deployment cost metric.

3.3. Baseline Methods

This paper selects four representative KG-RAG methods as baselines. All baselines are faithfully reproduced under the same datasets, reasoning LLMs, and evaluation protocol.

KAPING [8]: a semantic-similarity-based top-K triple retrieval method. It matches the query against knowledge-graph triples, injects the K most similar triples into the LLM context, and completes reasoning with a single LLM call. KAPING represents the “minimalist retrieval + single call” route.

GraphRAG [7]: a hierarchical retrieval method based on community detection and LLM-generated summaries. It first partitions the knowledge graph into communities using the Leiden algorithm, then pre-generates natural-language summaries for each community with an LLM and returns the relevant summaries as context. GraphRAG represents the “community structure + pre-summarization” route.

ToG [4] (Think-on-Graph): an LLM-agent beam-search method on knowledge graphs. It uses the LLM as a reasoning agent to conduct multi-round exploration, evaluating the current path and selecting the next-hop direction at each iteration. ToG represents the “multi-round exploration + high accuracy” route.

TAIMR+LLMLingua [12]: a sequential retrieval-and-compression method based on topology-aware retrieval and token-level compression. It first uses TAIMR to retrieve a topology-aware candidate subgraph and then invokes LLMLingua to compress the retrieved results through perplexity-based token-level compression. TAIMR+LLMLingua represents the “retrieval + compression” route.

3.4. Experimental Setup

To improve reliability, DeepSeek-chat [26] and Qwen3.7-Max [27] (a mixture-of-experts model with over one trillion total parameters, provided via the Alibaba Cloud Model Studio API, model ID qwen3.7-max-2026-06-08) are used separately as reasoning LLMs. Each dataset is evaluated in five independent runs, and the results are averaged to reduce stochastic variation. The inference temperature is fixed at 0 for reproducibility. The prompt template used for answer generation is provided in Appendix A.

The three LOKG modules are configured as follows: SCOS trains TransE for 100 epochs on the training split of each dataset to obtain 64-dimensional relation embeddings, with the tri-signal fusion threshold shared across all datasets (base threshold ${\theta }_{\mathrm{base}}=0.75$ and adaptive coefficient $\epsilon =0.15$; the “adaptive adjustment operates per relation pair through the entropy difference in Equation (5), not per dataset); ITTC uses compact, balanced, and verbose encoding granularities and optimizes their allocation within a 2048-token budget by dynamic programming; TAIMR uses a 2-layer, 4-head GAT with hidden dimension 128 and dropout 0.3, initializes entity embeddings with E5 [21] (768 dimensions), uses a unit topology information-gain weight (${\lambda }_{\mathrm{topo}}=1.0$), applies late query–answer semantic fusion with ${\lambda }_q=1.0$, sets the maximum retrieval depth to 3, and returns the top 20 paths. KAPING uses top-$K=20$ retrieval, GraphRAG uses the default Leiden parameters, and ToG uses beam width and exploration depth of 3. End-to-end evaluation is performed on the complete test set of each dataset. All hyperparameters are selected on the validation split only and never involve the test set, which is used exclusively for final evaluation. All reported LOKG results are obtained with the complete scoring of Equation (18), i.e., the topology score plus the query-answer semantic term with weight 1.0.

The main notations used in this paper are summarized in Table 2. Detailed hyperparameters are provided in Table 3 and Table 4.

Table 2. Summary of notations used in this paper.

Symbol	Description
${\theta }_{\mathrm{base}}$, $\epsilon$	base merging threshold and adaptive coefficient
$H\left(r\right)$	conditional entropy of the tail-entity distribution of relation r
$\widehat{\rho }$	normalized information density after schema compression
${\beta }_k$	granularity retention coefficient ($k\in \{1,\dots ,K\}$)
$\alpha \left(t_i\right)$	query relevance boost factor of triple $t_i$
B, K	token budget and number of encoding granularities
$K_{\mathrm{top}}$	number of retrieved candidate triples
$\mu \left(v\right)$	information gain of node v
${\lambda }_{\mathrm{topo}}$, ${\lambda }_q$	topology and query–answer semantic weights
$T_{\mathrm{naive}}$, $T_{\mathrm{after}}$	naive and actual injected token counts

Table 3. Experimental hyperparameter configuration.

Category	Parameter	Setting
SCOS	TransE training epochs	100 epoch
	Relation embedding dimension	64
	Fusion weights $(w_1,w_2,w_3)$	(0.40, 0.30, 0.30)
	Base threshold ${\theta }_{\mathrm{base}}$	0.75
	Adaptive coefficient $\epsilon$	0.15
ITTC	Encoding granularity set	Compact/Balanced/Verbose
	Granularity retention coefficients $({\beta }_1,{\beta }_2,{\beta }_3)$	(0.5, 0.8, 1.0)
	Query boost coefficient ${\lambda }_{\mathrm{boost}}$	3.0
	Token budget B	2048
TAIMR	GAT layers	2
	Attention heads M	4
	Hidden dimension	128
	Dropout	0.3
	Entity embeddings (E5)	768-dim
	Maximum hops L/Top-$K_{\mathrm{top}}$	3/20
	Topological information-gain weight ${\lambda }_{\mathrm{topo}}$	1.0
	Query relevance weight ${\lambda }_q$	1.0

Table 4. Key baseline hyperparameters.

Method	Parameter	Setting
Shared configuration	Inference temperature	0
	Generation limit	512
	Token budget B	2048
	Sentence encoder	e5-base-v2
KAPING	Retrieval Top-K	20
	Neighborhood expansion hops	1
	Candidate pool limit	200
GraphRAG	Subgraph node limit	2000
	Community limit	200
	Summary triples per community	30
	Retrieved communities	Top-3
ToG	Beam width/exploration depth	3/3
	Relations retained per step	3
	Entities retained per relation	5
	Selection-step/reasoning-step generation limit	64/128
TAIMR+LLMLingua	TAIMR maximum retrieval hops/Top-K	3/20
	LLMLingua target compression budget	2048 tokens
	Compression granularity	Token-level (perplexity)

3.5. Experimental Results

In this section, we compare LOKG with the baselines on KGQA benchmarks to evaluate reasoning performance and token efficiency. Table 5 reports end-to-end results on four datasets for both reasoning LLMs. Bold values indicate the best result for each metric.

Table 5. End-to-end comparison results on four datasets (DeepSeek-chat and Qwen3.7-Max).

		Accuracy Mean ± std		Amortized Efficiency		Deployment Cost
Dataset	Method	H@1↑	F1↑	P@1K↑	H@1K↑	TCR↓	tok/q↓	calls/q↓
DeepSeek-chat
CWQ	LOKG (Ours)	$0.269\pm 0.037$	$0.273\pm 0.032$	0.123	0.121	0.244	2 216	1.0
	KAPING	$0.141\pm 0.019$	$0.146\pm 0.017$	0.133	0.128	0.347	1 098	1.0
	GraphRAG	$0.243\pm 0.036$	$0.227\pm 0.031$	0.107	0.115	0.798	2 117	2.0
	ToG	$\mathbf{0.281}\pm 0.029$	$\mathbf{0.306}\pm 0.030$	0.070	0.065	0.428	4 344	18.7
	TAIMR+LLMLingua	$0.273\pm 0.041$	$0.265\pm 0.037$	0.114	0.118	0.557	2 318	1.0
WebQSP	LOKG (Ours)	$0.662\pm 0.091$	$0.544\pm 0.066$	0.263	0.321	0.078	2 065	1.0
	KAPING	$0.686\pm 0.056$	$0.562\pm 0.057$	0.253	0.310	0.105	2 216	1.0
	GraphRAG	$0.426\pm 0.037$	$0.350\pm 0.041$	0.141	0.172	0.120	2 477	1.7
	ToG	$\mathbf{0.714}\pm 0.066$	$\mathbf{0.594}\pm 0.079$	0.134	0.162	0.121	4 413	10.5
	TAIMR+LLMLingua	$0.647\pm 0.092$	$0.573\pm 0.083$	0.243	0.275	0.188	2 356	1.0
PQ-2hop	LOKG (Ours)	$\mathbf{0.720}\pm 0.070$	$\mathbf{0.710}\pm 0.080$	7.030	7.129	1.008	101	1.0
	KAPING	$0.700\pm 0.070$	$0.680\pm 0.050$	6.800	7.000	1.000	100	1.0
	GraphRAG	$0.540\pm 0.060$	$0.530\pm 0.060$	0.951	0.969	1.000	557	2.5
	ToG	$0.710\pm 0.080$	$0.700\pm 0.080$	1.042	1.057	1.000	672	7.2
	TAIMR+LLMLingua	$0.650\pm 0.050$	$0.690\pm 0.070$	5.897	5.556	1.000	117	1.0
PQ-3hop	LOKG (Ours)	$0.584\pm 0.059$	$0.592\pm 0.063$	2.508	2.475	1.009	236	1.0
	KAPING	$0.544\pm 0.077$	$0.596\pm 0.075$	2.649	2.418	1.000	225	1.0
	GraphRAG	$0.368\pm 0.021$	$0.376\pm 0.027$	0.410	0.399	0.949	923	2.3
	ToG	$\mathbf{0.604}\pm 0.055$	$\mathbf{0.616}\pm 0.048$	0.423	0.415	1.000	1 456	11.7
	TAIMR+LLMLingua	$0.533\pm 0.066$	$0.526\pm 0.066$	2.424	2.456	1.000	217	1.0
Qwen3.7-Max
CWQ	LOKG (Ours)	$0.588\pm 0.075$	$\mathbf{0.685}\pm 0.073$	0.323	0.277	0.246	2 122	1.0
	KAPING	$0.393\pm 0.048$	$0.384\pm 0.050$	0.313	0.320	0.998	1 227	1.0
	GraphRAG	$0.591\pm 0.086$	$0.604\pm 0.072$	0.260	0.254	0.843	2 327	1.2
	ToG	$\mathbf{0.649}\pm 0.043$	$0.667\pm 0.052$	0.146	0.142	0.999	4 569	19.9
	TAIMR+LLMLingua	$0.573\pm 0.071$	$0.615\pm 0.088$	0.273	0.254	0.471	2 257	1.0
WebQSP	LOKG (Ours)	$0.704\pm 0.061$	$0.693\pm 0.053$	0.307	0.312	0.105	2 254	1.0
	KAPING	$0.628\pm 0.055$	$0.602\pm 0.076$	0.255	0.266	0.123	2 364	1.0
	GraphRAG	$0.672\pm 0.078$	$0.598\pm 0.055$	0.238	0.267	0.165	2 517	1.7
	ToG	$\mathbf{0.758}\pm 0.063$	$\mathbf{0.728}\pm 0.087$	0.252	0.263	0.223	2 887	12.7
	TAIMR+LLMLingua	$0.688\pm 0.057$	$0.621\pm 0.043$	0.279	0.309	0.587	2 225	1.0
PQ-2hop	LOKG (Ours)	$0.780\pm 0.063$	$\mathbf{0.810}\pm 0.062$	7.788	7.500	1.007	104	1.0
	KAPING	$0.650\pm 0.088$	$0.660\pm 0.068$	6.947	6.842	1.000	95	1.0
	GraphRAG	$0.680\pm 0.068$	$0.750\pm 0.093$	0.890	0.807	1.000	843	2.8
	ToG	$\mathbf{0.820}\pm 0.074$	$0.790\pm 0.056$	1.272	1.320	0.996	621	7.0
	TAIMR+LLMLingua	$0.680\pm 0.057$	$0.680\pm 0.070$	6.869	6.869	1.000	99	1.0
PQ-3hop	LOKG (Ours)	$0.649\pm 0.089$	$0.668\pm 0.072$	2.855	2.774	1.008	234	1.0
	KAPING	$0.616\pm 0.058$	$0.526\pm 0.066$	2.317	2.714	1.000	227	1.0
	GraphRAG	$0.497\pm 0.069$	$0.466\pm 0.050$	0.351	0.374	1.000	1 328	2.5
	ToG	$\mathbf{0.707}\pm 0.094$	$\mathbf{0.731}\pm 0.089$	0.647	0.626	1.000	1 130	10.2
	TAIMR+LLMLingua	$0.598\pm 0.053$	$0.613\pm 0.071$	2.763	2.694	1.000	222	1.0

Overall performance. As shown in Table 5, LOKG achieves a strong overall trade-off on all four datasets with both DeepSeek-chat and Qwen3.7-Max. Its advantage is not absolute dominance on every accuracy metric, but a better balance between reasoning quality and token utilization under the single-call constraint. On CWQ and WebQSP, LOKG consistently yields lower TCR than the baselines: 0.246/0.105 under Qwen3.7-Max versus 0.843/0.165 for GraphRAG, and 0.244/0.078 under DeepSeek-chat versus 0.798/0.120 for GraphRAG. This shows that SCOS’s schema-level relation compression and ITTC’s multi-granularity encoding effectively remove redundant tokens when candidate subgraphs are large, allowing the limited context window to carry a higher proportion of useful evidence. On PathQuestion-2hop and PathQuestion-3hop, LOKG matches or outperforms KAPING and GraphRAG in H@1 and F1, while leading in amortized P@1K and H@1K, indicating that TAIMR’s topology-aware path retrieval prioritizes evidence chains that contribute to multi-hop reasoning rather than isolated triples selected only by semantic similarity.

Comparison with ToG. ToG achieves the highest or near-highest H@1 through multi-round LLM-agent exploration, but requires 7.0–19.9 calls per query and consumes approximately 1.3–6.0 times as many API tokens as LOKG. Under Qwen3.7-Max, LOKG’s H@1 on CWQ is 9.4% lower than ToG’s, yet its P@1K is 121% higher; on WebQSP, H@1 is 7.1% lower while P@1K remains 21.8% higher. Under DeepSeek-chat, the H@1 gap is 4.3% on CWQ and 7.3% on WebQSP, whereas P@1K is higher by 75.7% and 96.3%, respectively. Moreover, under Qwen3.7-Max, LOKG’s F1 exceeds ToG’s on CWQ and PQ-2hop. The underlying distinction is that ToG expands the search space to improve answer coverage, whereas LOKG compresses knowledge representations to maximize information per token; the two methods target accuracy-first and efficiency-first deployment requirements, respectively. When latency, call count, and token cost are tightly constrained, LOKG trades a modest amount of accuracy for an order-of-magnitude reduction in deployment cost.

Comparison with GraphRAG. GraphRAG represents knowledge hierarchically through community detection and community summaries, with retrieval granularity fixed at pre-defined communities. When a query concerns only a few key paths within a community, its summary can introduce irrelevant background information and consume valuable token budget. LOKG performs retrieval and encoding at the path and triple levels: TAIMR selects candidate paths using the topological context of query entities and relation attention, and ITTC allocates encoding granularity according to each triple’s information value. Thus, LOKG controls the quality of injected knowledge at a finer granularity. On CWQ and WebQSP, LOKG’s TCR is substantially lower than GraphRAG’s (0.246/0.105 versus 0.843/0.165 under Qwen3.7-Max, and 0.244/0.078 versus 0.798/0.120 under DeepSeek-chat); except for the nearly tied H@1 on CWQ under Qwen3.7-Max (0.588 versus 0.591), LOKG achieves higher H@1 and F1 than GraphRAG in the remaining configurations.

Comparison with KAPING. KAPING uses top-K semantically similar triples and a single generation call, resulting in the lowest invocation cost and near-theoretical token lower bounds on small-subgraph tasks. However, it does not model graph topology or evidence-chain completeness. On complex multi-hop questions, it can retrieve isolated triples that are superficially relevant but contribute little to reasoning, leading to the lowest CWQ H@1 among all methods: 0.393 under Qwen3.7-Max and 0.141 under DeepSeek-chat. By organizing candidate knowledge as paths rather than isolated fragments, TAIMR improves evidence coverage for multi-hop reasoning while retaining the single-call advantage.

Applicability boundary. TCR for all methods on PathQuestion-2hop and PathQuestion-3hop remains close to 1.0 because the candidate subgraphs are far below the token budget $B=2048$, so naive encoding is already near the lower bound and ITTC’s MCKP allocation has little compression space to exploit. Note that LOKG’s TCR slightly exceeds 1.0 on these two datasets (1.008 and 1.009), which is a serialization artifact rather than a compression failure: because the candidate subgraphs are far below the budget, MCKP allocation does not activate and ITTC returns essentially the same triples as the naive baseline, only joined by newline separators instead of spaces; under the cl100k_base tokenizer a newline is counted as a separate token while a space is absorbed into the adjacent word, producing a per-triple discrepancy of only a few tokens in total. Ablation confirms this interpretation—removing ITTC recovers TCR exactly 1.000 across all queries, whereas removing SCOS or TAIMR leaves the value unchanged. This reveals the applicability boundary of token-compression techniques: compression gains grow with candidate subgraph size, whereas the marginal benefit approaches zero when the subgraph itself fits comfortably within the budget. LOKG’s main value, therefore, appears when candidate knowledge exceeds context capacity; the larger the graph and the greater the relation redundancy, the greater the synergistic benefit of its three modules.

3.6. Ablation Experiments

To verify the independent contributions of the LOKG modules, we remove one of SCOS, ITTC, and TAIMR at a time while retaining the other two modules, and observe the performance changes on all four datasets. Table 6 reports the complete metrics for every ablation configuration.

Table 6. Ablation experiment results (DeepSeek-chat and Qwen3.7-Max).

		Accuracy		Amortized Efficiency		Deployment Cost
Dataset	Configuration	H@1↑	F1↑	P@1K↑	H@1K↑	TCR↓	tok/q↓
DeepSeek-chat
CWQ	LOKG (full)	0.269	0.273	0.123	0.121	0.244	2 216
	w/o SCOS	0.251	0.244	0.110	0.114	0.245	2 210
	w/o ITTC	0.261	0.253	0.116	0.120	0.237	2 174
	w/o TAIMR	0.252	0.257	0.116	0.114	0.215	2 217
WebQSP	LOKG (full)	0.662	0.544	0.263	0.321	0.078	2 065
	w/o SCOS	0.535	0.468	0.208	0.238	0.064	2 246
	w/o ITTC	0.621	0.552	0.248	0.279	0.063	2 223
	w/o TAIMR	0.576	0.479	0.213	0.257	0.098	2 244
PQ-2hop	LOKG (full)	0.720	0.710	7.030	7.129	1.008	101.0
	w/o SCOS	0.720	0.710	7.030	7.129	1.008	101.0
	w/o ITTC	0.673	0.681	6.529	6.453	1.000	104.3
	w/o TAIMR	0.629	0.643	6.159	6.025	1.007	104.4
PQ-3hop	LOKG (full)	0.584	0.592	2.508	2.475	1.009	236.0
	w/o SCOS	0.584	0.592	2.508	2.475	1.009	236.0
	w/o ITTC	0.546	0.552	2.404	2.378	1.000	229.6
	w/o TAIMR	0.553	0.556	2.472	2.459	1.008	224.9
Qwen3.7-Max
CWQ	LOKG (full)	0.588	0.685	0.343	0.295	0.246	1 996
	w/o SCOS	0.587	0.586	0.294	0.294	0.245	1 994
	w/o ITTC	0.572	0.602	0.310	0.294	0.237	1 943
	w/o TAIMR	0.556	0.564	0.281	0.277	0.215	2 010
WebQSP	LOKG (full)	0.704	0.693	0.338	0.343	0.078	2 050
	w/o SCOS	0.684	0.647	0.316	0.334	0.075	2 050
	w/o ITTC	0.628	0.661	0.329	0.312	0.074	2 012
	w/o TAIMR	0.620	0.651	0.318	0.302	0.111	2 050
PQ-2hop	LOKG (full)	0.780	0.810	15.759	15.175	1.007	51.4
	w/o SCOS	0.780	0.810	15.759	15.175	1.007	51.4
	w/o ITTC	0.708	0.723	14.176	13.882	1.000	51.0
	w/o TAIMR	0.712	0.729	14.238	13.906	1.008	51.2
PQ-3hop	LOKG (full)	0.649	0.668	3.881	3.771	1.008	172.1
	w/o SCOS	0.649	0.668	3.881	3.771	1.008	172.1
	w/o ITTC	0.585	0.627	3.842	3.585	1.000	163.2
	w/o TAIMR	0.564	0.619	3.876	3.532	1.009	159.7

Table 7 reports the absolute changes in the four core metrics after removing each module ($\Delta =\mathrm{ablated}-\mathrm{full}$). Negative values indicate degradation, whereas positive values indicate improvement.

Table 7. Metric changes in ablation experiments (DeepSeek-chat and Qwen3.7-Max).

		Metric Changes ($\Delta$)
Dataset	Removed Module	$\Delta$H@1↑	$\Delta$F1↑	$\Delta$P@1K↑	$\Delta$H@1K↑
DeepSeek-chat
CWQ	w/o SCOS	−0.018	−0.029	−0.013	−0.007
	w/o ITTC	−0.008	−0.020	−0.007	−0.001
	w/o TAIMR	−0.017	−0.016	−0.007	−0.007
WebQSP	w/o SCOS	−0.127	−0.076	−0.055	−0.083
	w/o ITTC	−0.041	+0.008	−0.015	−0.042
	w/o TAIMR	−0.086	−0.065	−0.050	−0.064
PQ-2hop	w/o SCOS	0.000	0.000	0.000	0.000
	w/o ITTC	−0.047	−0.029	−0.501	−0.676
	w/o TAIMR	−0.091	−0.067	−0.871	−1.104
PQ-3hop	w/o SCOS	0.000	0.000	0.000	0.000
	w/o ITTC	−0.038	−0.040	−0.104	−0.097
	w/o TAIMR	−0.031	−0.036	−0.036	−0.016
Qwen3.7-Max
CWQ	w/o SCOS	−0.001	−0.099	−0.049	−0.001
	w/o ITTC	−0.016	−0.083	−0.033	−0.001
	w/o TAIMR	−0.032	−0.121	−0.063	−0.018
WebQSP	w/o SCOS	−0.020	−0.046	−0.022	−0.010
	w/o ITTC	−0.076	−0.032	−0.010	−0.031
	w/o TAIMR	−0.084	−0.042	−0.021	−0.041
PQ-2hop	w/o SCOS	0.000	0.000	0.000	0.000
	w/o ITTC	−0.072	−0.087	−1.582	−1.293
	w/o TAIMR	−0.068	−0.081	−1.520	−1.269
PQ-3hop	w/o SCOS	0.000	0.000	0.000	0.000
	w/o ITTC	−0.064	−0.041	−0.040	−0.187
	w/o TAIMR	−0.085	−0.049	−0.005	−0.239

SCOS contribution. Under Qwen3.7-Max, removing SCOS causes a marked F1 decline (0.099 on CWQ and 0.046 on WebQSP), while H@1 changes only slightly (a decrease of 0.001 on CWQ and 0.020 on WebQSP). This suggests that SCOS primarily improves the overall quality of the answer set rather than first-choice hit rate. Because CWQ and WebQSP are built on Freebase, their original relation vocabularies contain many fine-grained, semantically overlapping relations. Merging them into semantic equivalence classes focuses the retrieval candidates and reduces answer fragmentation caused by near-synonymous relations, thereby improving F1. The zero effect on both PathQuestion datasets is consistent with their non-redundant relation vocabularies and confirms that SCOS gains increase with relation redundancy.

ITTC contribution. Removing ITTC reduces LOKG to uniform-template encoding. Under Qwen3.7-Max, its contribution is most evident on PQ-2hop: P@1K falls by 1.582 and H@1K by 1.293; H@1 on WebQSP also decreases by 0.076. Under DeepSeek-chat, removing ITTC yields a marginal F1 increase of 0.008 on WebQSP while H@1 decreases by 0.041, reflecting a precision-recall trade-off: ITTC re-ranking sharpens top-1 precision at a small cost to answer-set recall. Although PathQuestion has small subgraphs and little macroscopic compression space, ITTC still increases effective information concentration at similar total token counts by assigning differentiated granularities to different triples. Without it, low-value triples occupy the same encoding space as high-value triples, reducing overall information density.

TAIMR contribution. Under Qwen3.7-Max, removing TAIMR causes simultaneous accuracy losses across multiple datasets and has the broadest impact: F1 decreases by 0.121 on CWQ, H@1 by 0.084 on WebQSP, and H@1 by 0.085 on PathQuestion-3hop. PathQuestion is a pure path-reasoning task whose correct answers depend on complete recall of specific topological paths. Without TAIMR, retrieval degenerates into semantic-similarity matching and can miss intermediate nodes that are not highly similar to the query but are indispensable to the reasoning chain, resulting in broken evidence paths.

Module synergy. The three modules operate at different levels and support one another: SCOS simplifies TAIMR’s relation search space, TAIMR supplies ITTC with high-quality candidates, and ITTC injects the selected paths into context in a compact form. Removing any module generally degrades overall performance, confirming the necessity of threading the information-density objective through schema compression, path retrieval, and triple encoding.

4. Conclusions

This paper presents LOKG, a high-information-density knowledge graph retrieval augmentation framework for large language models that adopts token information density maximization as a shared design objective across knowledge representation, retrieval, and encoding. The framework addresses the structural limitation of independently optimized retrieval and compression in conventional “retrieve-then-compress” pipelines: SCOS merges redundant schema relations to reduce downstream search cardinality, TAIMR introduces topological information gain and relation attention to prioritize paths with structural reasoning value, and ITTC allocates encoding budgets according to information value to increase the concentration of useful knowledge in context. The three modules operate along orthogonal dimensions—semantic space, retrieval granularity, and encoding strategy—and coordinate knowledge selection and representation compression under a common budget-aware logic.

Experiments on four KGQA benchmarks show that, under a single-LLM-call constraint, LOKG achieves substantial gains in token efficiency and lower deployment cost with only modest accuracy differences. Ablation studies further verify the orthogonality of the module contributions and the necessity of their synergy. More broadly, the same design principle may benefit document RAG, multimodal retrieval, and tool-use scenarios in which external knowledge must be selected and injected under strict budgets. The main limitations are that SCOS gains depend on relation-vocabulary redundancy and ITTC compression gains grow with candidate-subgraph size. Future work will investigate query-complexity-adaptive dynamic budget allocation and extend information-density optimization to multimodal and document-hybrid retrieval.