Repairing DNN Numerical Defects with Semantic-Driven Knowledge Graph Retrieval

Abstract

Ensuring numerical robustness in deep neural networks (DNNs) is critical, as defects like overflow or NaN can cause silent failures. However, automated repair is challenged by fragmented domain knowledge and the semantic gap for general-purpose large language models (LLMs). This work proposes NCKG, a Numerical–Conceptual Knowledge Graph-based method for retrieval-augmented repair of DNN numerical defects. NCKG introduces a unified semantic formalization that explicitly models DNN execution contexts, numerical defects, mitigation methods, and constraint knowledge, transforming dispersed defect knowledge into a consistent, machine-interpretable representation. Based on this formalization, a multi-view semantic graph index is constructed, enabling a hybrid semantic-driven retrieval mechanism that combines structure-aware graph matching with vector similarity. Retrieved, semantically aligned defect–repair knowledge is then used to guide LLMs in generating context-aware repairs. Experimental results demonstrate that NCKG significantly outperforms standard retrieval baselines and consistently improves the quality and correctness of LLM-generated fixes across different model scales. This work demonstrates that explicit semantic structuring and retrieval of domain knowledge are crucial for enabling reliable, automated numerical defect repair in DNNs.

1. Introduction

Deep neural networks (DNNs) have become ubiquitous across critical domains such as autonomous systems, healthcare, and scientific computing. However, ensuring their numerical robustness remains a significant and persistent challenge [1,2]. DNN programs are inherently susceptible to subtle numerical defects—including overflow, underflow, division-by-zero, and gradient pathologies—that can lead to silent failures (e.g., incorrect outputs), training instability (NaN/Inf values), or complete runtime crashes [3,4].

Debugging and repairing such defects is notoriously difficult, requiring deep expertise in numerical analysis, framework specifics, and the problem domain. A core underlying issue is the heterogeneous and fragmented nature of the defect knowledge. Relevant information is scattered across various sources: Framework documentation details API-specific behaviors. Mathematical textbooks describe stability conditions. Issue trackers contain empirical bug reports, and commit histories hold concrete fix examples [5,6]. This knowledge exists in disparate formats—natural language, code, and symbolic constraints—resulting in fragmented and implicit knowledge that is hard to reuse or systematize.

Existing approaches to DNN repair exhibit notable limitations. Rule-based methods [7,8] and symbolic techniques [9] are precise but struggle with the diversity of real-world defects beyond their predefined patterns. These approaches typically operate at the level of strategy recommendation or implicit mitigation, rather than producing concrete, code-level repairs. Rule-based systems usually map observed numerical symptoms to a fixed set of predefined repair strategies, while symbolic methods often provide implicit corrections or safety constraints without explicitly modifying the source code, which makes it difficult to generalize to heterogeneous code contexts or to support direct, executable fixes in real-world DNN source code.

The emergence of powerful large language models (LLMs) offers a promising avenue for addressing software bugs. LLMs trained on vast code corpora can generate plausible code fixes. However, applying general-purpose LLMs directly to DNN numerical defects yields suboptimal results. These models lack specific, actionable knowledge about numerical stability patterns, the semantic relationships between DNN components (e.g., layers and optimizers) and failure symptoms, and the nuanced strategies required for valid fixes (e.g., precision adjustment vs. algorithm replacement). Consequently, LLM-generated patches for numerical issues often fail to address the root cause or introduce new errors.

To bridge these gaps, this paper introduces NCKG (a Numerical–Conceptual Knowledge Graph-based method) for graph-indexed, retrieval-augmented repair of DNN numerical defects. This work first establishes a unified conceptual semantic framework for DNN numerical defects. The framework centers on a concept graph that defines and links core semantic entities—such as the DNN component, the observed symptom, and the applicable mitigation strategy—along with their definitive relations. This formalization achieves semantic alignment across heterogeneous knowledge sources by providing a consistent, abstract representation, transforming fragmented artifacts into a coherent, machine-interpretable structure. Based on this formalization, a multi-view graph index is constructed, where each instance is decomposed into three focused semantic subgraphs (defect, context, and mitigation). These graphs serve as a structured retrieval index, enabling hybrid search that combines symbolic graph matching with subsymbolic vector similarity. For a new defect, NCKG retrieves the most semantically related concepts and rationales, enriching the prompt with relevant examples. This knowledge-guided context is then provided to an LLM to generate a context-aware repair. This study employs NCKG to integrate structured, semantic knowledge, aiming to enhance both the retrieval of relevant fixes and the generation of subsequent repairs by LLMs.

The primary contributions of this work are as follows:

• Constructing a conceptual semantic framework for DNN numerical defects to unify and interrelate heterogeneous defect–repair knowledge. A structured conceptual semantic framework is proposed to model DNN numerical defects through explicit representations of faulty components, numerical anomaly states, repair operations, and domain constraints. By focusing on conceptual representations, this framework unifies heterogeneous defect–repair artifacts, enabling transferable defect–repair knowledge beyond individual code instances.
• Developing a multi-view semantic-knowledge-driven graph indexing and hybrid retrieval method. Building on the proposed semantic framework, an index-based GraphRAG extension for numerical defect retrieval is introduced, where multi-view semantic subgraphs are constructed to capture complementary aspects of defect knowledge, and a hybrid retrieval mechanism is designed to integrate structure-aware subgraph matching with semantic vector similarity, allowing precise yet flexible retrieval of relevant defect–repair knowledge grounded in domain semantics.
• Proposing a knowledge-guided LLM-based repair generation pipeline for DNN numerical defects. A knowledge-guided repair pipeline is developed to leverage the retrieved semantic context to guide large language models, reducing hallucinations and improving the reliability of numerical defect–repair generation.

To systematically evaluate the effectiveness of the proposed NCKG method, this study is guided by the following research questions: RQ1 assesses the fundamental retrieval capability, testing whether NCKG retrieves more semantically relevant fixes than conventional methods. RQ2 evaluates the end-to-end repair generation performance, examining whether augmenting LLMs with this retrieved knowledge yields more reliable repairs. RQ3 performs an ablation study to deconstruct the hybrid mechanism, quantifying the individual contribution of its graph-based and vector-based components to substantiate the design rationale. The specific questions guiding this investigation are as follows:

• RQ1: Comparative Evaluation of Retrieval Effectiveness. Does NCKG enable more semantically accurate retrieval of numerical defect–repair knowledge compared to existing retrieval methods, as measured by the semantic alignment between the retrieved mitigation strategies and the ground-truth strategies?
• RQ2: Impact of the Retrieved Context on Repair Generation. How does the quality of the final generated repair change when large language models are augmented with NCKG-retrieved semantic contexts across different retrieval backbones and generative models?
• RQ3: Ablation Study of the Hybrid Retrieval Mechanism. What are the individual contributions of the graph-based and vector-based retrieval components within the hybrid NCKG framework to overall repair effectiveness?

This paper is organized as follows: Section 2 reviews related work. Section 3 details the proposed methodology, which consists of four main parts: the formal definition of conceptual semantics and relations for DNN numerical defects, the construction of a graph index via semantic subgraphs, a hybrid semantic retrieval mechanism over graph indices, and the knowledge-guided repair generation pipeline. Section 4 and Section 5 present the experimental setup and the analysis of the results, respectively. Section 6 provides a discussion of the findings, and Section 7 concludes the paper.

2. Related Work

The related work is investigated from the perspectives of knowledge engineering for software tasks, retrieval-augmented generation with large language models, and automated repair of DNN numerical defects.

2.1. Knowledge Engineering for Software Tasks

Knowledge engineering, which involves the formal representation and utilization of domain knowledge, has a long history in software engineering. Traditional approaches often rely on manually or semi-automatically constructed knowledge bases, such as rule-based systems [10,11] and code property graphs [12,13]. For instance, pattern-based bug detection tools use predefined defect patterns encoded as rules [14,15,16]. In program analysis, semantic code graphs [17,18,19] (e.g., control flow graphs, data flow graphs, and program-dependent graphs) have been extensively used to capture structural and semantic relationships for tasks like clone detection, vulnerability analysis, and program understanding. More recently, the concept of “knowledge graphs” [20] has been adopted to integrate heterogeneous software artifacts (e.g., code, commits, issues, and documentation) to support tasks such as code search, API recommendation, and bug localization.

However, applying such knowledge engineering techniques to the specific domain of deep neural network (DNN) numerical defects presents distinct and significant challenges. First, the relevant knowledge is highly heterogeneous, spanning multiple dimensions: mathematical–numerical constraints, hardware-specific floating-point behavior, concrete defect–fix examples, empirical debugging heuristics, and framework-specific API specifications [5,6]. This diversity makes it difficult for traditional symbolic knowledge representations to adapt to and align with the semantics across varied textual and code-based artifacts. Second, unlike traditional software with explicit algorithmic logic, DNNs exhibit opaque, data-driven behavior [21], making it harder to define and capture their failure modes and repair strategies within a static, rule-bound schema.

In other engineering domains, similar challenges of heterogeneous information have been addressed through ontological frameworks. For example, in the maintenance domain, Zhou et al. [22] addressed the inconsistent propagation of maintenance process information (MPI) by establishing a lifecycle MPI propagation architecture to unify the semantic description of maintenance activities. Similarly, Xia et al. [23] proposed a Maintenance Knowledge Graph (MKG) ontology framework to tackle inefficient knowledge utilization and the lack of unified standards in maintenance scenarios. This framework provides a paradigm model for structuring maintenance information based on a hierarchical representation of maintenance elements.

The proposed method introduces a lightweight, domain-grounded conceptual semantic schema tailored for DNN numerical defects. Instead of attempting to construct a comprehensive, general-purpose code graph, this method departs from code-structure–centric semantic defect modeling approaches and focuses on building a concept-centric graph that distills the key entities along with their causal and logical relations, achieving semantic alignment based on the concept graph and enabling semantic associations directly related to repair.

2.2. Prompt Engineering and Retrieval-Augmented Generation for LLMs in Code Processing

The effectiveness of large language models (LLMs) on downstream tasks is critically dependent on the input context provided, a practice known as prompt engineering [24]. Retrieval-augmented generation (RAG) [25] enhances this factor by dynamically fetching relevant external knowledge to condition the LLM, mitigating issues like hallucination and improving factual accuracy. Research in prompt engineering has explored various in-context learning designs [26], such as providing few-shot examples, chain-of-thought reasoning steps, or specific instruction formats. Tools like ThinkRepair [27] retrieve “bug–Chain-of-Thought–fix” triples to explicitly guide the LLM’s reasoning process for general program repair.

The retrieval core of most existing RAG systems for code relies primarily on measuring similarity between unstructured text passages, such as code snippets or commit messages [28]. Sparse retrievers like BM25 [29] are matched based on keyword overlap, while dense retrievers like Dense Passage Retrievers (DPRs) [30] or those using code-specific sentence transformers (e.g., CodeBERT) are matched based on learned semantic embeddings. A fundamental limitation of this text-centric paradigm is semantic misalignment: textual similarity does not guarantee that the retrieved example illustrates the same type of problem or employs the same repair strategy as the query. This is particularly problematic for complex, semantically nuanced tasks like numerical defect repair, where the relevance of a fix is defined by specific conceptual relationships rather than surface-level text.

To overcome the limitations of unstructured text retrieval, graph retrieval-augmented generation (GraphRAG) [31] has emerged as a paradigm that leverages structured knowledge to better ground LLMs. Existing GraphRAG approaches can be categorized based on their utilization of the graph structure. Knowledge-based GraphRAG [32,33] uses graphs primarily as carriers of factual knowledge, where entities and relations from a pre-existing, often general-purpose, knowledge graph are retrieved to enrich the context. Index-based GraphRAG [34,35], conversely, uses a graph as a structured index over a corpus. Here, data points (e.g., bug-fix pairs) are decomposed into entities and relations to build a task-specific graph index, and retrieval is performed by matching the query’s structured representation against this index. Hybrid GraphRAG methods attempt to combine both approaches.

The NCKG method presented in this work is an instantiation of the index-based GraphRAG paradigm, and it is specifically adapted for DNN numerical defects. NCKG advances this paradigm through domain-specific semantic modeling and indexing design. In particular, a multi-view graph index is constructed based on a conceptual semantic schema, where three focused subgraph indices are built to capture complementary facets—defect diagnosis, component context, and mitigation strategy—of each defect–fix pair. Retrieval operates over structure-aware subgraph matching with semantic vector similarity, enabling precise alignment between a query’s defect scenario and the repair logic of candidate examples.

2.3. Automated Repair for DNN Numerical Defects

Automatically addressing numerical instability in DNNs is a critical and distinct sub-field. Early approaches were largely rule-based. AutoTrainer [7] defined a set of training-time symptoms and applied a series of predefined candidate fixes in sequence. DeepDiagnosis [8] extended this by using a decision tree to prune the search space of fixes based on a broader symptom set. In both cases, “repair” is operationalized as selecting from a fixed set of coarse-grained actions (e.g., adjusting the learning rate or initialization) rather than generating concrete, context-sensitive code-level patches. These methods are inherently limited by their predefined rule sets and their reliance on executable training pipelines with runtime monitoring, which restricts their applicability to real-world scenarios.

Importantly, semantics-driven approaches have also been explored. RANUM [9] frames numerical repair as a constraint optimization problem, using abstract interpretation to compute safe input intervals for operators to prevent instability (e.g., division by zero). While powerful for range-related defects, RANUM adopts a black-box fixing paradigm and produces constraint-level artifacts rather than explicit source-code repairs, and its applicability is constrained to defects expressible via interval constraints and may not cover other common issues like gradient vanishing/explosion or algorithmic inefficiency.

Very recently, researchers have begun exploring LLMs for DNN debugging and repair, typically via prompt engineering that asks the model to localize faults and suggest patches [36]. However, these initial attempts often treat the problem as generic code generation, lacking mechanisms to incorporate domain-specific knowledge about numerical stability patterns. There is a clear gap between the generic capabilities of LLMs and the specialized knowledge required for reliable numerical defect repair. This work bridges this gap by introducing a knowledge-aware retrieval framework grounded in a conceptual semantic model of DNN numerical defects. Instead of relying on unstructured text retrieval or fixed rule mappings, NCKG retrieves defect–fix examples based on explicit semantic alignment among defect symptoms, affected components, and repair strategies. This positions NCKG as a novel synthesis of concept-level knowledge modeling, graph-based retrieval, and LLM-based repair generation for a specialized, high-stakes domain.

3. Methodology

The proposed method enhances generative large language model (LLM) repair generation for DNN numerical defects through structured knowledge retrieval. The method is demonstrated in Figure 1 and comprises three phases: (1) the construction of semantic graphs, (2) a hybrid retrieval mechanism over these graphs, and (3) a guided generation process.

3.1. Unified Framework of Conceptual Semantics and Relations for DNN Numerical Defect

Numerical defects in deep neural networks are documented and discussed across a wide range of heterogeneous sources, including source code repositories, mathematical references, and informal developer discussions. These sources differ substantially in structure, abstraction level, and terminology. As a result, knowledge related to the same numerical defect or repair strategy is often fragmented and difficult to integrate, limiting systematic reasoning and reuse across cases.

A key challenge in numerical defect repair lies in unifying such heterogeneous knowledge into a consistent and machine-interpretable representation. Without explicit semantic abstraction, it becomes difficult to align defect symptoms with their causes, relate repairs across different components, or generalize repair strategies beyond individual instances. To address this challenge, a conceptual semantic framework is introduced to abstract numerical defects and their repairs into a unified representation, as shown in Figure 2. This framework decouples defect semantics from source-specific representations by identifying a small set of core semantic elements and explicitly modeling their relationships. By doing so, heterogeneous information originating from diverse sources can be normalized into a common semantic space, enabling consistent interpretation, integration, and subsequent reasoning.

3.1.1. Numerical Defect Semantic Elements Definition

The core of the framework comprises four semantic elements, as defined in

Table 1. Semantic element definitions for DNN numerical defects.

Semantic Elements	Symbol	Description	Primary Attribute
Execution Context	$C$	(WHEN and WHERE) denote the architectural or algorithmic module within a DNN pipeline.	Phase type ($\tau$), function descriptor (f), and operation descriptor ($op$)
Numerical Defect	$\mathit{ND}$	(WHAT) characterizes the problem and observable manifestation of the numerical defect.	Symptom type ($\sigma$), root cause ($\rho$), and contextual description ($\chi$)
Mitigation Method	$M$	(HOW) categorizes the approach for repairing or mitigating the defect.	Strategy type ($\psi$) and method descriptor ($\mu$)
Constraint	$K$	(WHY) documents the origin of the background knowledge.	Knowledge type ($\kappa$) and knowledge context ($\xi$)

. Detailed mathematical definitions for each semantic element are provided in Appendix A.1.

The core elements are formally aligned with two established foundational ontologies from complementary domains: FIDES [37] ontologies, which semantically annotates data, parameters, and processes across the ML’s life cycle, and the Ontology of Software Defects, Errors and Failures (OSDEF) [38], a reference model grounded in the Unified Foundational Ontology (UFO) [39] for clarifying software anomaly concepts.

The proposed execution context (C), numerical defect ($ND$), and mitigation method (M) elements directly map to the execution–executor–procedure (EEP) and result–context (RC) Ontology Design Patterns (ODPs), which form the backbone of the EEPSA ontology [40] reused by the FIDES framework. The DNN component (the executor) performs an execution within a specific C, which may implement a faulty mathematical procedure that manifests as an $ND$. Concurrently, within the RC ODP, the observed $ND$ is the result, occurring within the circumstantial context defined by C. The M then acts as a corrective procedure for this specific result–context pair, with its validity anchored by the constraint (K). This structuring allows the framework to reuse FIDES’s approach of formally linking procedures ($ND$ and M) with their outcomes and contexts.

Simultaneously, the framework’s concepts are grounded in the OSDEF ontology. Here, the $ND$ is specialized from OSDEF’s core taxonomy: a numerical defect (a bug in the code or model specification) may cause a runtime error (describe an incorrect internal state, e.g., an overflow, refers to $\rho$), which can lead to a failure (an observable deviation from expected behavior, e.g., a NaN output, refers to $\sigma$). The C precisely locates this chain within the DNN pipeline artifact, while M targets the rectification of the underlying defect or error.

By integrating the EEP/RC ODPs from FIDES/EEPSA with the conceptual taxonomy of OSDEF, this framework achieves a multifaceted foundation. It inherits OSDEF’s rigorous, UFO-based distinctions for software anomalies, ensuring clarity between defects, errors, and failures in the DNN context. Concurrently, it adopts FIDES’s reusable, pattern-based method for structurally linking the what (procedure of defect to failure), where (context), and how (procedure of fix to failure) of a DNN numerical defect. Collectively, the semantic elements C, $ND$, M, and K provide a comprehensive and structured abstraction of numerical defects and their repairs.

3.1.2. Semantic Relation Schema

The semantic richness of the framework emerges from the defined relations R that link the conceptual entities. These relations form a directed, labeled multi-graph schema, establishing causal, associative, and applicative links. The relation schema R is formally defined as a set of relation types $r_i$, each associating a source entity type $S_i$ with a target entity type $T_i$, capturing a specific semantic dependency ${\varphi }_i$.

The full relation schema, derived from domain analysis, is presented in

Table 2. Formal semantic relation schema R.

Relation Name r	Source S	Target T	Semantic Description $\mathit{\varphi }$
phase_defines	$C.\tau$	$C.f$	A component’s high-level type logically encompasses or is implemented by specific functions.
operation_implements	C.op	$C.f$	A mathematical operation (e.g., exp) is a constituent part or the core computation within a specific function.
symptom_manifests_in	$ND.\sigma$	$C.f$	A specific symptom is observed during the execution of a particular function.
symptom_indicates	$ND.\sigma$	$ND.\rho$	A manifested symptom implies or is directly caused by an underlying root cause (e.g., Inf symptom caused by “division by zero”).
context_informs_cause	$ND.\chi$	$ND.\rho$	The problem context contains information that further explain the root cause.
cause_suggests_method	$ND.\rho$	$M.\mu$	An identified root cause dictates or strongly suggests a specific mitigation method.
context_suggests_method	$ND.\chi$	$M.\mu$	The problem context dictates or strongly suggests a specific mitigation method.
strategy_generalizes	$M.\psi$	$M.\mu$	A high-level repair/mitigation strategy is concretely implemented by a specific method.
knowledge_explains_context	$K.\xi$	$ND.\chi$	External knowledge (e.g., a forum post, a numerical stability principle) provides the rationale or explanation for the observed problem context.
knowledge_motivates_strategy	$K.\xi$	$M.\psi$	External knowledge (e.g., a mathematical principle) motivates or justifies a repair strategy.
operation_constrained_by	C.op	$K.\xi$	A particular mathematical operation is associated with specific background knowledge (e.g., “numerical stability trick: subtract max(logit)”).
type_of_knowledge	$K.\kappa$	$K.\xi$	The knowledge type categorizes the nature or provenance of the knowledge context.

. Each relation is defined between specific attributes of the entities, effectively establishing paths between entity instances in the knowledge graph.

Let $\mathcal{E}=\left\{\mathrm{Execution}\mathrm{Context}\right(C),\mathrm{Numerical}\mathrm{Defect}(ND),\mathrm{Mitigation}\mathrm{Method}(M),\mathrm{Constraint}(K\left)\right\}$ be the set of entity types. A relation $r\in R$ is a triple $r=(S,T,\varphi )$ where $S,T\in \mathcal{E}$ and $\varphi$ are predicates describing the relationship.

This structured relation schema, R, transforms a collection of isolated attributes into a connected, semantic knowledge graph. A path through this graph, such as $\sigma \to \rho \to \mu \leftarrow \psi$, represents a coherent reasoning chain: from a symptom to its root cause, leading to a method that instantiates a particular strategy. This formalism enables the subgraph matching and retrieval processes detailed in the next chapter, which are essential for augmenting LLM-based repair generation with precise, context-aware exemplars.

3.2. Graph-Index Construction via Semantic Subgraphs

The core of the proposed methodology is a structured graph-indexed knowledge base that is engineered to enable efficient and semantically precise retrieval. The graph index is designed to integrate multi-source knowledge, for example, code-level defect–fix pairs and descriptive knowledge concerning numerical defects. The construction involves two main phases: (1) extracting instances of the formal semantic entities from each data point and (2) organizing these entities into three distinct, interlinked semantic subgraphs that serve as the primary indexing units. These subgraphs capture complementary facets of the defect–repair–knowledge triad, facilitating targeted retrieval based on different query aspects (e.g., symptom, component, or mitigation strategy).

Given a corpus of defect–fix pairs and related documents $D=\{d_1,d_2,\dots ,d_i\}$, each data point $d_i$ typically contains a defective code snippet, its corrected version, and accompanying textual descriptions (e.g., commit messages, issue reports, and forum discussions). For each $d_i$, a concept extraction function, $\Phi$, is applied, which combines rule-based parsing or lightweight entity recognition models to map the raw data to a set of structured entity instances:

$${\mathcal{E}}_i=\{C_i,N{D}_i,M_i,K_i\}=\Phi \left(d_i\right)$$

(1)

where $C_i$, $N{D}_i$, $M_i$, and $K_i$ correspond to instances of the execution context, numerical defect, mitigation method, and constraint entities, respectively, following the formal definitions in Section 3.1.1. For the construction of the retrieval knowledge base, the concept extraction procedure primarily follows and adopts the manual analysis results aligned with the DeepStability dataset [5]. Specifically, for high-level semantic dimensions such as the phase type, symptom type, and strategy type, the extraction is grounded in a predefined set of abstract concept categories (see the definition part in Appendix A.1), which act as canonical semantic anchors. The alignment between extracted concepts and the underlying ontologies is established at the concept definition level through systematic ontology reuse and specialization. Notice that the semantic elements of NCKG reuse and specialize core concepts and design patterns from the FIDES and OSDEF ontologies ensure semantic consistency at the definition level; these ontology-aligned concept definitions could provide explicit semantic guidance for the extraction function $\Phi$, enabling consistent instantiation across heterogeneous data sources. Further details are provided in Appendix A.2.

Further, to construct a monolithic heterogeneous graph, the extracted entity attributes are decomposed into three focused semantic subgraphs per data point. Each subgraph serves as a distinct indexing lens, capturing orthogonal dimensions of the defect–repair–knowledge relationship, as shown in Figure 3. This multi-view decomposition enables targeted retrieval operations aligned with different query intents.

Defect Semantic Index (${\mathcal{G}}^{\left(\mathit{d}\right)}$): This index models the diagnostic pathway from observable symptoms to root causes, as contextualized by background knowledge. For each data point $d_i$, the corresponding subgraph $G_i^{\left(d\right)}=(V_i^{\left(d\right)},E_i^{\left(d\right)})$ is constructed, where

$$\begin{array}{cc}\hfill V_i^{\left(d\right)}\subseteq & \{{\sigma }_i,{\rho }_i,{\chi }_i,{\xi }_i\}\hfill \\ \hfill E_i^{\left(d\right)}=& \{({\sigma }_i\stackrel{symptom\_indicates}{\to }{\rho }_i),\hfill \\ \hfill & ({\chi }_i\stackrel{context\_informs\_cause}{\to }{\rho }_i),\hfill \\ \hfill & ({\xi }_i\stackrel{knowledge\_explains\_context}{\to }{\chi }_i)\}\hfill \end{array}$$

(2)

This subgraph encodes the diagnostic reasoning chain, serving as an index focusing on error symptoms and their causation.

Context Semantic Index (${\mathcal{G}}^{\left(\mathit{c}\right)}$): This index anchors defects within the DNN computational architecture by linking component types to specific functions and mathematical operations. The per-instance subgraph $G_i^{\left(c\right)}=(V_i^{\left(c\right)},E_i^{\left(c\right)})$ is defined as

$$\begin{array}{cc}\hfill V_i^{\left(c\right)}\subseteq & \{{\tau }_i,f_i,o{p}_i,{\sigma }_i\}\hfill \\ \hfill E_i^{\left(c\right)}=& \{({\tau }_i\stackrel{phase\_defines}{\to }{f}_i),\hfill \\ \hfill & (o{p}_i\stackrel{operation\_implements}{\to }{f}_i),\hfill \\ \hfill & ({\sigma }_i\stackrel{symptom\_manifests\_in}{\to }{f}_i)\}\hfill \end{array}$$

(3)

This subgraph captures the computational context, enabling retrieval based on architectural or operational similarity.

Repair Semantic Index (${\mathcal{G}}^{\left(\mathit{m}\right)}$): This index captures the repair rationale by connecting diagnosed causes to concrete mitigation methods that are constrained by external knowledge. Each instance yields a subgraph $G_i^{\left(m\right)}=(V_i^{\left(m\right)},E_i^{\left(m\right)})$ where

$$\begin{array}{cc}\hfill V_i^{\left(m\right)}\subseteq & \{{\rho }_i,{\chi }_i,{\psi }_i,{\mu }_i,{\xi }_i\}\hfill \\ \hfill E_i^{\left(m\right)}=& \{({\rho }_i\stackrel{cause\_suggests\_method}{\to }{\mu }_i),\hfill \\ \hfill & ({\chi }_i\stackrel{context\_suggests\_method}{\to }{\mu }_i),\hfill \\ \hfill & ({\psi }_i\stackrel{strategy\_generalizes}{\to }{\mu }_i),\hfill \\ \hfill & ({\xi }_i\stackrel{knowledge\_motivates\_strategy}{\to }{\psi }_i)\}\hfill \end{array}$$

(4)

This subgraph encapsulates the repair logic, indexing the pathway from cause analysis to actionable fixes.

These multi-view graph index structures form the foundation for the hybrid semantic retrieval described in the next section.

3.3. Hybrid Semantic Retrieval over Graph Indices

The constructed multi-view graph index enables a hybrid semantic retrieval mechanism that combines symbolic graph matching with subsymbolic vector similarity. Given a query representing a new numerical defect instance, the objective is to retrieve the most relevant defect–fix information from the knowledge base to augment LLM-based repair generation. The process involves three main steps: (1) query representation, (2) subgraph-based similarity retrieval, and (3) multi-view score aggregation and hybrid fusion.

3.3.1. Query Representation

An incoming query q, which may consist of code snippets, error messages, or natural language descriptions, is first processed by the same concept extraction function $\Phi$ used during index construction. In our experiments, the $\Phi$ is implemented primarily via LLM-based analysis followed by fuzzy matching to the predefined semantic candidate set, ensuring retrieval efficiency with the knowledge graph index. This yields a set of semantic entities ${\mathcal{E}}_q=\{C_q,N{D}_q,M_q,K_q\}=\Phi \left(q\right)$, where some entities may be partially specified or entirely missing. Three query subgraphs ($G_q^{\left(d\right)}$, $G_q^{\left(c\right)}$, and $G_q^{\left(m\right)}$) are instantiated from ${\mathcal{E}}_q$, mirroring the index structure. Note that $M_q$ (the mitigation) is typically empty as the repair is unknown.

A key advantage of the graph-based representation is its ability to perform conceptual-level contextual expansion. When a query lacks certain entities (e.g., no mitigation information is present), the semantic relations in R could be used to traverse the graph indices and infer potentially relevant concepts. For example, if a query specifies a symptom ${\sigma }_q$ and a component function $f_q$, the $symptom\_indicates$ and $cause\_suggests\_method$ relations can be chained to retrieve mitigation methods associated with similar symptom–function pairs. This expansion is formally defined as a path traversal operation:

$$Expand\left(q\right)=\bigcup _{p\in \mathcal{P}}\{n|\exists \mathrm{path}p\mathrm{from}{v}_q\mathrm{to}n\mathrm{in}{\mathcal{G}}^{\left(k\right)}\mathrm{with}\mathrm{length}<L\}$$

(5)

where $\mathcal{P}$ is a set of relation sequences predefined as plausible inference chains and L is a maximum traversal depth. The expanded nodes enrich the query context without requiring explicit specification.

3.3.2. Subgraph-Based Similarity Retrieval

For each subgraph type k, the retrieval algorithm computes a similarity score between $G_q^{\left(k\right)}$ and every candidate subgraph $G_i^{\left(k\right)}$ in the corresponding index ${\mathcal{G}}^{\left(k\right)}$. The similarity function $si{m}_G^{\left(k\right)}:G_q^{\left(k\right)}\times G_i^{\left(k\right)}\to [0,1]$ is defined as a weighted combination of node overlap and edge overlap and is formalized as follows:

Let $V_q$ and $V_i$ denote the node sets of $G_q^{\left(k\right)}$ and $G_i^{\left(k\right)}$ respectively, where each node is identified by its semantic attribute value (e.g., “NaN” or “Adam”). Similarly, let $E_q$ and $E_i$ denote the edge sets, where each edge is represented as a triple $(u,v,r)$ indicating a relation r from node u to node v.

• Node Overlap Similarity: The Jaccard similarity coefficient between node sets measures the conceptual commonality:

  $$si{m}_{node}(G_q^{\left(k\right)},G_i^{\left(k\right)})={\displaystyle \frac{|V_q\cap V_i|}{|V_q\cup V_i|}}$$

  (6)

  where intersection is defined as nodes sharing identical semantic attributes, accounting for potential variations through normalization techniques such as lower casing and those stemming for textual attributes.
• Edge Overlap Similarity: Structural similarity is measured via edge set overlap:

  $$si{m}_{edge}(G_q^{\left(k\right)},G_i^{\left(k\right)})={\displaystyle \frac{|E_q\cap E_i|}{|E_q\cup E_i|}}$$

  (7)

  where two edges are considered identical if they share the same source node attribute, target node attribute, and relation type r.

Overall graph similarity is computed as a convex combination:

$$si{m}_G^{\left(k\right)}(G_q^{\left(k\right)},G_i^{\left(k\right)})=\alpha ·si{m}_{node}+(1-\alpha )·si{m}_{edge},\alpha \in [0,1]$$

(8)

Empirically, $\alpha =0.6$ is set to slightly prioritize node (conceptual) overlap while maintaining structural consistency.

For each subgraph type k, this process yields a ranked list ${\mathcal{L}}^{\left(k\right)}$ of candidate defect–fix pairs, where each pair $d_i$ is associated with a graph similarity score $s_i^{\left(k\right)}=si{m}_G^{\left(k\right)}(G_q^{\left(k\right)},G_i^{\left(k\right)})$. The candidates per subgraph are retained for subsequent fusion.

3.3.3. Multi-View Score Aggregation and Hybrid Retrieval

To produce a unified retrieval result, a two-level fusion strategy is applied. First, the scores from each view are aggregated into a single graph retrieval score $S_G\left(d_i\right)$ for each candidate $d_i$:

$$S_G\left(d_i\right)=\sum _k{\beta }_k·s_i^{\left(k\right)}$$

(9)

where ${\beta }_k$ are view-specific weights that reflect the relative importance of each semantic perspective. These weights can be tuned or set uniformly (e.g., 0.3/0.5/0.2 in our experiments). Candidates appearing in multiple lists benefit from score accumulation.

Simultaneously, a vector-based retrieval score $S_V\left(d_i\right)$ is computed using dense embeddings. The query q and each candidate $d_i$ (represented by the concatenated textual attributes) are encoded into fixed-dimensional vectors ${\mathbf{v}}_q$ and ${\mathbf{v}}_i$ using a pre-trained language model.

Then the hybrid retrieval algorithm integrates graph-based and vector-based matching through a multi-stage fusion process. Given a query q with its extracted semantic entities and subgraphs $G_q^{\left(k\right)}$, the algorithm returns the top-K most relevant defect–fix pairs. Algorithm 1 presents the pseudocode of the proposed hybrid retrieval process.

Algorithm 1 Pseudocode of the hybrid retrieval method

Input:  Query q with semantic subgraphs $G_q^{\left(d\right)},G_q^{\left(c\right)},G_q^{\left(m\right)}$; Graph indices ${\mathcal{G}}^{\left(d\right)},{\mathcal{G}}^{\left(c\right)},{\mathcal{G}}^{\left(m\right)}$; Vector index $\mathcal{V}$ with embeddings ${\left\{{\mathbf{v}}_i\right\}}_{i=1}^N$; Retrieval parameter K; Fusion weights ${\beta }_1,{\beta }_2,{\beta }_3$; Fusion parameter $\gamma$ Output:  Ranked list of top-K defect–fix pairs $\mathcal{R}$   1: Phase 1: Parallel Candidate Retrieval   2: ${\mathcal{C}}_G\leftarrow Ø$ {Graph-based candidates}   3: ${\mathcal{C}}_V\leftarrow Ø$ {Vector-based candidates}   4: for  $k\mathrm{in}\{d,c,m\}$ do   5:    Compute $s_G^{\left(k\right)}\left(d_i\right)$ for all $d_i\in \mathcal{D}$ using ${\mathrm{sim}}_G^{\left(k\right)}(G_q^{\left(k\right)},G_i^{\left(k\right)})$ {Graph retrieval per subgraph view}   6: end for   7: for each $d_i\in \mathcal{D}$ do   8:     $S_G\left(d_i\right)\leftarrow {\sum }_{k=1}^3{\beta }_k·s_G^{\left(k\right)}\left(d_i\right)$ {Aggregate graph scores}   9: end for 10: ${\mathcal{C}}_G\leftarrow {\mathrm{Top}}_{2K}(\mathcal{D},S_G)${Top $2K$ by graph score} 11: {Vector retrieval} 12: ${\mathbf{v}}_q\leftarrow \mathrm{Encoder}\left(q\right)$ {Encode query to vector} 13: for each $d_i\in \mathcal{D}$ do 14:    $S_V\left(d_i\right)\leftarrow cos({\mathbf{v}}_q,{\mathbf{v}}_i)$ {Compute vector similarity} 15: end for 16: ${\mathcal{C}}_V\leftarrow {\mathrm{Top}}_{2K}(\mathcal{D},S_V)$ {Top $2K$ by vector score} 17: Phase 2: Candidate Pool Formation 18: $\mathcal{C}\leftarrow {\mathcal{C}}_G\cup {\mathcal{C}}_V$ {Union of candidate sets} 19: Phase 3: Score Normalization and Fusion 20: for each $d_i\in \mathcal{C}$ do 21: ${\tilde{S}}_G\left(d_i\right)\leftarrow {\displaystyle \frac{S_G\left(d_i\right)-{min}_{d_j\in \mathcal{C}}{S}_G\left(d_j\right)}{{max}_{d_j\in \mathcal{C}}{S}_G\left(d_j\right)-{min}_{d_j\in \mathcal{C}}{S}_G\left(d_j\right)}}$ {Normalize graph score} 22: ${\tilde{S}}_V\left(d_i\right)\leftarrow {\displaystyle \frac{S_V\left(d_i\right)-{min}_{d_j\in \mathcal{C}}{S}_V\left(d_j\right)}{{max}_{d_j\in \mathcal{C}}{S}_V\left(d_j\right)-{min}_{d_j\in \mathcal{C}}{S}_V\left(d_j\right)}}$ {Normalize vector score} 23: $S_{\mathrm{hybrid}}\left(d_i\right)\leftarrow \gamma ·{\tilde{S}}_G\left(d_i\right)+(1-\gamma )·{\tilde{S}}_V\left(d_i\right)$ {Compute hybrid score} 24: end for 25: Phase 4: Re-ranking and Final Selection 26: $\mathcal{R}\leftarrow \mathrm{SortDescending}(\mathcal{C},S_{\mathrm{hybrid}})$ {Sort by hybrid score} 27: return $\mathcal{R}[1:K]${Return top-K results}

The parameter $\gamma$ can be adjusted based on query characteristics, such as the completeness of its semantic graph, allowing adaptive emphasis on structural or semantic features by considering candidates that excel in retrieval modality. The method mitigates the risk of missing relevant matches due to the limitations of any single approach.

3.4. Knowledge-Guided Repair Generation

The top-K retrieved defect–fix pairs and associated knowledge are then used to augment a generative large language model (LLM) for repair generation. The key idea is to construct a structured prompt that provides the LLM with relevant context and examples, guiding it to produce a correct and context-aware fix for the query defect.

3.4.1. Prompt Design for Contextual Repair Generation

The prompt is designed as a multi-context instructional template that presents the model with both in-context examples and the target problem. The template structure is formally defined as follows:

Let $\mathcal{R}=\{d_{i_1},\cdots ,d_{i_k}\}$ be the top-K retrieved defect–fix pairs. For each pair $d_{i_j}\in \mathcal{R}$, let ${\mathrm{bug}}_{i_j}$ and ${\mathrm{fix}}_{i_j}$ denote its defective and repaired code snippets, respectively. Let ${\mathrm{bug}}_q$ denote the defective code from the query q.

The prompt P is constructed as a concatenation of three components:

1.

Instruction Header: A fixed natural language instruction specifying the repair task.

2.

Retrieved Exemplars Block: A sequence of $m(m\le K)$ retrieved exemplars, formatted as shown below.

3.

Target Problem Block: The query’s defective code with a placeholder for the model to complete.

Listing 1 shows the structure of the knowledge-guided repair prompt template.

Listing 1. Knowledge-Guided Repair Prompt Template.

In practice, m is chosen based on the context window of the target model, which is typically set to 1–2 exemplars. The exemplars are selected from $\mathcal{R}$ based on their hybrid retrieval scores, ensuring the most relevant examples are provided.

3.4.2. Generation and Post-Processing

Given the prompt P constructed as shown above, the LLM generates a completion that ideally contains the fixed version of the target function. The generation probability can be formalized as

$$P_{LM}\left(\mathrm{fix}\right|P)=\prod _{t=1}^T{P}_{LM}\left(w_t\right|w_{<t},P)$$

(10)

where $w_t$ represents the t-th token in the generated fix.

Following the generative phase, a systematic post-processing pipeline extracts and validates the repair candidates from the LLM outputs. The extraction process identifies the code block immediately following the “### Fixed Function:” marker in the model’s generated text.

To enhance robustness and account for the stochastic nature of generative models, a multi-sampling strategy is employed. For each target defect, the generation process is executed independently n times using identical prompt templates and retrieval contexts. Each generated output is processed through the aforementioned extraction method. Successful repair candidates are retained, while malformed outputs are discarded.

The validated repair candidates are aggregated through frequency analysis. Let $C_{gen}=\{c_1,c_2,\cdots ,c_m\}$ represent the set of distinct, syntactically valid repair candidates generated across n trials. Each candidate $c_j$ is associated with a frequency count $f_j$, corresponding to the number of times an identical repair was generated across all successful trials. This frequency distribution provides a measure of confidence for each candidate, with higher frequencies indicating greater consistency in the model’s repair generation.

Mathematically, this process can be formalized as

$$C_{final}=\{(c_j,f_j)|c_j\in C_{gen},f_j=\sum _{i=1}^n\mathbb{I}(\mathrm{extract}\left(O_i\right)=c_j)\}$$

(11)

where $O_i$ denotes the model output from the i-th generation trial and $\mathbb{I}$ is the indicator function.

4. Experimental Setup

To evaluate the effectiveness of the proposed NCKG framework and to answer the research questions outlined in Section 1, we conduct an experimental study covering retrieval accuracy, repair generation quality, and component-level ablation analysis. The experiments are designed to reflect realistic DNN numerical defect repair scenarios across different generative models. Specifically, we evaluate NCKG on a curated dataset of real-world numerical defects, compare it against representative baseline methods, and assess performance using both retrieval-oriented and repair-oriented evaluation metrics. All experiments are implemented under controlled and reproducible settings.

4.1. Dataset

The experiments are conducted on a curated dataset combining code-level defect–fix pairs and text-level numerical knowledge. The primary source is the DeepStability dataset [5], which contains real-world numerical stability issues in deep learning frameworks (e.g., TensorFlow and PyTorch) and their fixes, which are extracted from commit histories. Each instance includes the defective code, the fixed code, commit messages, and often related issue reports. This is augmented with numerical defect knowledge documents from relevant sources, including Stack Overflow posts tagged with numerical issues and excerpts on numerical analysis [6].

In this study, we framed the task as retrieving relevant defect repair code pairs. After filtering, a curated dataset of 153 instances was retained. To rigorously evaluate the retrieval capability of NCKG and prevent data leakage, strict separation is maintained between data used for knowledge base construction and testing. The knowledge graph is built solely from instances in the training split. During testing, the model is presented only with the unseen instances in the test split. The mitigation-type proportion of the dataset is shown in Figure 4. The dataset is split into 80%/20% for training (indexing) and testing, respectively, with the distribution remaining unchanged. To mitigate potential biases introduced by a single random split, we repeat the test procedure 5 times with different random seeds for test set construction.

4.2. Baseline Methods

We compare NCKG’s hybrid retrieval capability against representative baselines from two distinct retrieval paradigms: lexical term-based retrieval and dense neural retrieval. The purpose of this comparison is to assess whether conceptual graph-based retrieval provides advantages beyond both “shallow” and dense retrieval methods.

• BM25 baseline—a classic lexical retrieval model that relies on exact keyword matching and term frequency statistics. BM25 is included as a representative lexical-based retrieval baseline and is applied to the concatenated textual fields of each defect–fix pair, including code tokens and commit messages. This baseline represents shallow lexical matching methods that do not explicitly model semantic or structural relationships.
• DPR baseline—a neural dual-encoder retrieval model following the DPR paradigm, where queries and candidate passages are independently encoded into a shared dense vector space. In our implementation, both encoders are instantiated using GraphCodeBERT, a code-aware pretrained model that captures syntactic and semantic properties of the source code. The model is fine-tuned on the training split using defective contexts as queries and corresponding fix contexts as positive passages.

The NCKG-augmented prompts are compared against prompts using contexts retrieved by BM25 and the GraphCodeBERT-based dense retriever, enabling a controlled comparison between lexical, dense neural, and knowledge-graph-guided hybrid retrieval.

For the generation stage, six state-of-the-art LLMs from the HuggingFace Transformers library are employed to evaluate the effectiveness of the proposed knowledge-guided approach across different model architectures and scales:

• GPT-Neo (125M, 1.3B, and 2.7B parameters)—autoregressive decoder-only models. Developed by EleutherAI, this family of models replicates the GPT-3 architecture using the open-source GPT-NeoX framework. These models were trained on The Pile, a diverse 825 GB English text corpus, providing strong general language-understanding capabilities.
• Phi-2 (2.7B parameters)—developed by Microsoft, this compact transformer model demonstrates remarkable reasoning abilities despite its relatively small parameter count. Phi-2 employs innovative training techniques including “textbook-quality” synthetic data generation and reinforcement learning from human feedback (RLHF), resulting in superior performance on reasoning benchmarks compared to models of a similar size.
• DeepSeekCoder (6.7B parameters)—a series of code-specialized LLMs pretrained on a corpus of 2 trillion tokens across 87 programming languages. In this study, we use the DeepSeekCoder-6.7B-Instruct version. It employs an advanced “fill-in-the-middle” training objective that enables bidirectional context awareness, making it particularly suitable for code completion and repair tasks.
• CodeLlama (7B parameters)—Meta’s code-optimized variant of the Llama 2 architecture, further pretrained on 500B tokens of code-specific data. The model demonstrates state-of-the-art performance on code generation benchmarks including HumanEval and MBPP.

For each model, we use the same prompt template and retrieval context, enabling a controlled comparison of how different architectures leverage the provided semantic knowledge. During generation, we employ nucleus sampling with $p=0.95$ and a temperature of $\tau =0.8$ to balance diversity and coherence. The maximum generation length is set to 512 tokens to accommodate typical function-level repairs.

4.3. Evaluation Metrics

4.3.1. Retrieval Metrics

Given a query defect, the retrieval system returns a ranked list of defect–fix pairs. We evaluate them based on the correctness of the mitigation strategy in the retrieved pairs, as the ultimate goal is to guide correct repairs.

1.

Exact Match@K: The proportion of queries where the top-K retrieved results contain at least one exact match of the ground-truth mitigation strategy. This measures the retrieval system’s ability to return perfectly relevant fixes within the top-K positions.

$$\mathrm{Exact}\_\mathrm{Match@K}={\displaystyle \frac{{\sum }_{i=1}^Q\mathbb{I}(\exists j\le K:S_{ij}=S_i^{*})}{Q}}$$

(12)

where Q is the total number of queries, $S_{ij}$ is the strategy of the j-th retrieved result for query i, and $S_i^{*}$ is the ground-truth strategy.

2.

Reciprocal Rank (RR) and Mean Reciprocal Rank (MRR): The reciprocal of the rank at which the first exact strategy match occurs. For a single query q, the reciprocal rank is defined as

$${\mathrm{RR}}_q={\displaystyle \frac{1}{min\{i:S_i^q=S_q^{*}\}}}$$

(13)

with ${\mathrm{RR}}_q=0$ if no exact match is found. The MRR is the average of ${\mathrm{RR}}_q$ across all queries:

$$\mathrm{MRR}={\displaystyle \frac{1}{\left|Q\right|}}\sum _{q\in Q}{\mathrm{RR}}_q$$

(14)

3.

Overall Success Rate (OSR): The proportion of queries for which at least one exact strategy match exists within the entire retrieved list. In our implementation, we retrieve up to 10 results per query, so this is equivalent to EM@10. Formally,

$$\mathrm{OSR}={\displaystyle \frac{{\sum }_{i=1}^Q\mathbb{I}(\exists i\in \{1,...,R\}:S_{ij}=S_i^{*})}{Q}}$$

(15)

where R is the total number of retrieved results per query (10 in our experiments). Note that the OSR provides an upper bound on recall given the retrieval depth constraint.

These metrics provide a multi-faceted evaluation: EM@K measures recall at different cut-offs. The MRR evaluates ranking quality, and the OSR indicates the absolute recall capability given the retrieval depth. All metrics are reported as averages over the test set.

4.3.2. Generation Metrics

For each query, the top-K retrieved contexts (K = 1 in our experiments) are used to construct a prompt for the generative LLM, which then produces a repaired code snippet. We evaluate the generated repair using a two-stage process: automated LLM-based assessment followed by human validation.

1.

Automated Assessment with an LLM: We employ a structured prompt template to instruct a Deepseek-R1-8b model to act as a software repair evaluation expert. The template asks for three analyses:

• Strategy Match: It measures whether the generated repair’s mitigation strategy aligns with the ground truth. The confidence score (match_confidence) directly serves as $SM\in [0,1]$.
• Code Similarity: It quantifies resemblance to the ground-truth fix through three sub-scores: syntactic, semantic, and textual similarity.
• Feasibility: It assesses practical viability through three sub-scores: compilation feasibility, logical correctness, and the risk of new bugs.

The output is constrained to a specific JSON format containing scores and reasoning. From this JSON, we extract numerical metrics and calculate their averages as final indicators. The rationale behind each metric’s score is also preserved to facilitate subsequent manual verification.

2.

Human Validation: To ensure reliability, a subset of the Deepseek evaluations (stratified by score ranges) is reviewed by human experts. Experts verify the correctness of the strategy classification and the plausibility of the similarity/feasibility scores, correcting any clear discrepancies. The final reported generation metrics are based on this validated set.

For final reporting, we evaluate the top-1 and top-3 generated repairs (ranking generation probabilities) using the overall score (avgOS), which is the average of the strategy match score, the code similarity score, and the feasibility score.

As discussed in Appendix B, the LLM-based evaluator represents a practical and extensible instantiation of the evaluation process. Individual dimensions—such as compilation feasibility in the feasibility dimension—can be replaced or augmented with static analysis, compilation checks, or dynamic execution when appropriate tooling and executable artifacts become available.

4.4. Implementation and Configuration

The NCKG method and all baselines are implemented in Python 3.10. For hybrid retrieval, the fusion parameter $\gamma$ is set to 0.6, balancing the contributions of graph-based and vector-based similarity. The dense retriever component uses GraphCodeBERT as the encoder backbone in a dual-encoder (DPR-style) architecture. The model is fine-tuned on the training set using a contrastive loss objective, ensuring that both the dense baseline and the dense component of NCKG share identical representation capacity.

For the baseline models, the BM25 baseline is implemented using the rank_bm25 Python package with default parameters. The dense baseline shares the same GraphCodeBERT encoder, training data, and optimization settings as NCKG, ensuring a fair and controlled comparison.

All experiments are conducted on a server with NVIDIA A100 GPUs.

5. Results

5.1. RQ1: Comparative Evaluation of Retrieval Effectiveness

Table 3. Experimental result of retrieval effectiveness.

	Exact Match @1	Exact Match @3	Exact Match @5	Mean Reciprocal Rank	Overall Success Rate
BM25	0.3330	0.2727	0.2647	0.4610	0.6777
DPR	0.3443	0.3333	0.2990	0.4540	0.6883
NCKG	0.8710	0.5520	0.4410	0.8297	0.9460

Note: Bold values indicate the best performance in each column.

presents the comprehensive retrieval performance of NCKG against two strong baselines, BM25 and DPR, across five key metrics. The results show that NCKG achieves notably higher retrieval performance than both traditional lexical retrieval and modern neural retrieval methods under the evaluated settings.

NCKG achieves the highest scores across all evaluated metrics. Most notably, it attains an Exact Match@1 score of 0.871, which is 2.6 times higher than that that for DPR (0.344) and 2.5 times higher than that for BM25 (0.333). This suggests that NCKG’s top-ranked retrieval result is more likely to employ the exact mitigation strategy required by the query, a critical feature for practical and automated repair assistance. The high Mean Reciprocal Rank (MRR) of 0.8297 further confirms that NCKG not only retrieves relevant matches more frequently but also ranks them higher in the result list.

A similar performance trend is observed across different recall depths, indicating that the advantage of NCKG is not limited to top-1 retrieval. At Exact Match@3, NCKG (0.5520) surpasses DPR (0.3333) by 65.5% and BM25 (0.2727) by 102.5%. At Exact Match@5, NCKG (0.4410) maintains a lead of 47.5% over DPR (0.2990) and 66.6% over BM25 (0.2647). The Overall Success Rate (OSR), which measures the proportion of queries for which at least one correct match is found within the retrieved set (10 in our experiment), reaches 0.9460 for NCKG. This corresponds to a high recall rate (94.6%) within the evaluated dataset, substantially exceeding those of DPR (0.6883) and BM25 (0.6777). The similar performance between BM25 and DPR suggests that pure text matching is insufficient for this task.

5.2. RQ2: Impact of Retrieved Context on Repair Generation

The generation performance, an overall score integrating strategy matching, code similarity, and feasibility, is shown in Figure 5 for both the top-1 and top-3 generation settings.

In both settings, the NCKG retrieval backbone consistently outperformed BM25 and DPR across all six generative models (GPT-Neo: 125M, 1.3B, and 2.7B; DeepSeek-Coder: 6.7B; and CodeLlama: 7B).

For top-1 generation tasks, the average overall score across all models with NCKG is 0.723, compared to 0.553 with BM25 and 0.565 with DPR, representing a relative improvement of 30.7% and 28.0%, respectively. The advantage of NCKG is most pronounced with the GPT-Neo 2.7B model, where it achieves an overall score of 0.744, surpassing BM25 (0.514) and DPR (0.561) by 44.6% and 32.6%, respectively.

Results for top-3 generation show a similar trend, with NCKG achieving an average Overall Score of 0.697 across all models, compared to 0.581 for BM25 and 0.566 for DPR. This corresponds to relative improvements of 20.0% and 23.1%, respectively. Notably, also with GPT-Neo 2.7B, NCKG reaches 0.738, which is 32.9% higher than DPR (0.555) and 29.4% higher than BM25 (0.570).

Among the models, the GPT-Neo family showed the greatest sensitivity to retrieval quality. NCKG provides a consistent performance gain under the evaluated configurations, particularly for the 125M parameter version, where it plays an important role in achieving competitive results. The performance of DeepSeekCoder is generally high and stable when using NCKG. The CodeLlama model also achieves high absolute scores overall.

5.3. RQ3: Ablation Study of the Hybrid Retrieval Mechanism

Figure 6 presents the results of the ablation study comparing the complete version of NCKG (hybrid) against its graph-only and vector-only variants in terms of both retrieval and final generation performances (using DeepSeek-Coder for its stable performance across multiple runs). The graph-only variant only uses the graph-based retrieval component ($\gamma =1.0$), while the vector-only variant only uses the vector-based retrieval component ($\gamma =0.0$), and the complete NCKG method runs with optimal fusion ($\gamma =0.6$).

The hybrid version of NCKG achieves the best overall performance with an Exact Match@1 of 0.871, an MRR of 0.552, and an OSR of 0.946. The graph-only variant follows closely, with strong Exact Match@1 (0.871) and OSR (0.935), but its MRR (0.498) is notably lower than the that of the hybrid approach. The vector-only variant performs significantly worse across all metrics, with an Exact Match@1 of only 0.366.

The generation results, as measured by match confidence, similarity, feasibility, and their composite score, mirror the retrieval trends. The hybrid NCKG yields the best generation quality (composite score: 0.706). The graph-only variant lags slightly (0.693), and the vector-only variant performs the worst (0.604). This also demonstrates a direct correlation between retrieval precision and downstream generation quality.

6. Discussion

The experimental results comprehensively validate the proposed NCKG framework and its hybrid retrieval mechanism. The key findings and their implications are discussed below.

6.1. The Necessity of Structured Semantic Retrieval (RQ1)

The observed performance advantage of NCKG over lexical-based and neural-based retrieval baselines (BM25 and DPR) in RQ1 provides empirical evidence for our core hypothesis: effective repair knowledge retrieval requires modeling the explicit semantic relationships between defect symptoms, root causes, and mitigation strategies, rather than relying on surface-level text similarity.

The similar performance between BM25 and DPR reinforces the limitation of text-only retrieval for this task. Both methods treat defect–fix pairs as unstructured text passages, missing the structured relationships that define repair relevance. For instance, a query containing a code snippet like torch.softmax(x, dim = −1) that results in NaN may lexically match a fix that modifies a similar code token softmax, but DPR could also retrieve unrelated fixes that mention NaN in entirely different contexts, such as a data-loading routine. NCKG, by contrast, ensures that retrieved pairs share not just superficial code tokens but also a coherent semantic pattern of defect and repair via relational edges (e.g., linking the NaN symptom node to the softmax function node and further to the rewrite math formula strategy node), enabling it to retrieve pairs that share analogous repair logic even if their surface descriptions differ.

The observed performance gap suggests the practical importance of the proposed semantic framework as a prerequisite for effective retrieval. While text-based methods fail to distinguish between coincidental lexical matches and semantically relevant repair patterns, NCKG’s structure ensures that retrieved cases are linked by a shared, machine-interpretable semantic logic. This capability directly enables the transfer of repair strategies across heterogeneous code instances, a task for which unstructured retrieval proves inadequate.

Also, the explicit modeling of symptom–context–fix relationships benefits the repair strategy suggestion. The high Exact Match@1 and OSR scores demonstrate that NCKG can reliably serve as a highly accurate “look-up” mechanism for known defect patterns and a robust “recall” system for identifying semantically analogous cases, which are the foundation for effective repair generation.

In summary, the results for RQ1 affirm that NCKG provides a far more effective retrieval mechanism for numerical defect repair contexts compared to baseline retrievers, successfully aligning retrieved fixes with the required repair strategies. The results validate the core hypothesis that repair strategies are transferable across semantically similar defect contexts and that explicitly modeling these contexts through a structured semantic graph within the abstract conceptual level is key to unlocking this transferability.

6.2. From Accurate Retrieval to Reliable Generation (RQ2)

The results of RQ2 establish a direct causal link between retrieval quality and the reliability of LLM-based repair generation. The consistent performance gain when using the NCKG-retrieved context, across LLMs of varying scales and architectures, confirms that providing precise, semantically structured exemplars is more effective than providing keyword-based or generally relevant text matches. The retrieval context allows the model to better understand the specific defect pattern and the appropriate repair strategy, leading to generations that are more likely to be logically correct and syntactically valid.

The performance gains are consistent across model sizes, indicating that NCKG’s retrieval augmentation benefits both small and large language models. While larger LLMs have a stronger capacity to “ignore” irrelevant parts of the context or to “hallucinate” correct fixes from minimal clues, they still achieve their best performance when guided by the precise, relation-aware examples provided by NCKG. NCKG’s context reduces the LLM’s burden of inference and strategy selection, channeling its generative capacity towards synthesizing a correct fix based on a well-understood pattern. This is particularly crucial for smaller models (e.g., GPT-Neo 125M), where NCKG’s context is essential for achieving competitive results, effectively democratizing access to high-quality automated repair.

This finding validates the knowledge-guided generation pipeline. NCKG not only enhances retrieval metrics but also directly translates this enhancement into higher-quality generated repairs, validating its role as an effective augmentative component for LLM-based program repair systems.

6.3. Synergy of the Hybrid Design (RQ3)

The ablation study (RQ3) offers critical insights into the inner workings of NCKG and justifies its hybrid design. The graph-only variant’s strong Exact Match@1 performance highlights the indispensable role of structured semantic matching for precision—it excels at finding the single most relevant fix. However, its lower MRR suggests a weakness in optimally ranking a set of relevant results, potentially due to the discrete nature of graph matching.

The vector-only variant’s poor performance confirms that unstructured semantic similarity is too coarse for this precision-demanding task. The integration of the vector-based component (at an optimal fusion weight of $\gamma$ = 0.6) addresses the ranking limitation of the graph component. It acts as a “smoothing” function, using continuous semantic embeddings to re-rank and diversify the candidate set retrieved by the graph, thereby improving the MRR and the overall quality of the context set for generation.

Therefore, the two components are complementary: the graph index provides the essential, precise semantic scaffolding, while the vector index enhances ranking robustness and semantic coverage. Their synergy is what enables NCKG to achieve state-of-the-art performance in both retrieval and the downstream generative task, demonstrating that structured semantic matching and dense semantic similarity are complementary and mutually reinforcing for the task of numerical defect repair.

6.4. Limitation and Future Work

Limitations: The primary limitation pertains to the scale of the dataset. Although carefully curated, the collection of real-world DNN numerical defect–fix pairs remains limited in size compared to broader software defect corpora, reflecting the inherent difficulty of annotating this specialized domain. This constraint may affect the generalizability and robustness of our findings. To mitigate associated threats to validity, all experiments were conducted with multiple random runs, with reported metrics representing averaged results. Given the general absence of reliable test suites for DNN numerical defects, the evaluation of generated repairs was conducted using a structured, LLM-assisted assessment template, followed by expert validation to ensure objectivity and reduce subjective bias.

Future Work: These limitations naturally lead to two main future directions. First, expanding the dataset with more diverse and extensive real-world examples is crucial for further validating and strengthening the approach. Second, and more fundamentally, the proposed unified conceptual semantic framework is designed not only for the immediate retrieval and generation task but also to serve as a foundational schema for systematically organizing knowledge in the domain of DNN numerical stability. Future work will leverage this framework to guide the consistent annotation and curation of larger datasets, enabling more comprehensive benchmarking and facilitating broader applications in DNN reliability analysis and repair.

7. Conclusions

This paper presented the Numerical–Conceptual Knowledge Graph-based (NCKG) method for the retrieval-augmented repair of numerical defects in deep neural networks. To bridge the semantic gap in unified knowledge representation and LLM-based repair, a structured semantic formalization that defines core entities and their relations is introduced, providing a unified, machine-interpretable schema for representing heterogeneous defect–repair knowledge. Based on this schema, a multi-view graph index is constructed, and a hybrid retrieval mechanism that integrates precise graph matching with semantic vector similarity is developed. This enables the retrieval of contextually relevant defect–fix pairs, which are then used within a knowledge-guided pipeline to augment LLMs for generating reliable repairs.

The experimental results indicate the effectiveness of the proposed method. NCKG outperforms the considered retrieval baselines (BM25 and DPR) on the evaluated dataset in strategy matching accuracy, validating that repair strategy transfer is governed by the structured semantic context. When used to augment various generative LLMs, the NCKG-retrieved context tends to yield higher-quality repairs, demonstrating that high-quality semantic grounding is critical for successful repair generation regardless of the scale of the generative model. Ablation studies confirm the complementary value of both graph-based and vector-based components in the hybrid design.

In summary, this work presents NCKG as an effective, semantics-driven paradigm that shows strong potential to enhance the reliability of automated numerical defect repair by bridging LLMs with structured domain knowledge. The positive results, which were obtained on a dataset at a specialized and limited scale, indicate a promising direction for DNN numerical defect repair generation.