Intelligent Q&A System for Welding Processes Based on a Symmetric KG-DB Hybrid-RAG Strategy

Abstract

This paper pioneers the use of the symmetrical Hybrid-RAG strategy in the ship welding process domain, addressing the problems of fragmented, unstructured knowledge storage, as well as the limitations of traditional Retrieval-Augmented Generation (RAG), particularly high retrieval noise and low accuracy when answering complex procedural queries. This study proposes an intelligent three-stage symmetric “Generate–Retrieve–Generate” framework for the ship welding process (SWP-Chat), supported by dual retrieval engines: a Neo4j knowledge graph for symbolic reasoning and a vector database for semantic retrieval. Unlike approaches that rely solely on LLM-based process planning, SWP-Chat uses the LLM to generate a logical form, then executes Cypher queries on Neo4j, enabling transparent traceability, precise entity–relation constraints, and deterministic retrieval. Meanwhile, the vector channel supplements unstructured or contextual welding information to enhance semantic coverage. To further improve efficiency, principal component analysis (PCA) was employed for vector dimensionality reduction, reducing average retrieval latency by 31% while retaining more than 95% variance. In addition, an explainable structural–confidence fusion formula integrates evidence from both engines to produce auditable and trustworthy industrial responses. Experimental evaluation demonstrates that the framework achieves an F1 score of 79.35%, greatly surpassing typical RAG systems.

1. Introduction

China has remained the world’s largest manufacturing economy for 15 consecutive years according to the latest statistics from the Ministry of Industry and Information Technology [1]. Intelligent modernization of manufacturing processes is essential for improving efficiency, reducing costs, and ensuring product quality in an increasingly competitive environment. The shipbuilding sector, which is a prime example of high-end equipment production, deals with highly intricate welding procedures. These present difficulties with large amounts of various data and include a variety of welding materials, climatic circumstances, welding parameters, and process specification documents.

The development of intelligent question-answering systems in manufacturing has mainly progressed through two stages aimed at improving the management and utilization of process knowledge. Knowledge engineering techniques such as knowledge graphs (KGs), which allow for accurate storage and logical reasoning for process parameters and specification clauses, were the mainstay of early systems [2,3,4]. However, these systems frequently rely on formal query languages like Cypher or manually coded rules, which limits practical human–machine interaction. In addition, they encounter difficulty integrating and using unstructured process documentation, and real-time knowledge updates and maintenance are expensive.

Due to its superior natural language understanding and unstructured text-processing capabilities, RAG technology has become the standard option in industrial Query and Answer (Q&A) as large language models have grown in popularity [5]. RAG greatly improves system interaction and knowledge coverage by embedding extensive process documentation into vector spaces, allowing for effective semantic search.

Despite these efforts, the strictly logical and parameter-coupled vertical industrial scenario of ship welding still faces three major core challenges:

(1)

Conflict between the retrieval paradigm and logical reasoning: The welding process involves complex question-answering tasks that rely on multi-parameter coupled decisions across various material performance indicators. Traditional RAG uses a “retrieve first, then generate” blind recall strategy. This method aggressively retrieves a large number of document fragments, leading to an explosion of retrieval noise. Especially in complex process Q&A that involves selecting multiple parameters based on several material performance metrics, the model’s logical chain easily breaks, and structural information is lost. This makes the output unreliable for high-risk welding decision support, where precision is critical. There is an urgent need for an innovative approach to isolate retrieval noise before the recall step.

(2)

Lightweight and real-time knowledge updating: Industry standards, classification society rules, and internal specifications change rapidly; However, existing GraphRAG methods require recalculation of the embeddings for the entire graph, resulting in prohibitively high update costs (O($N^2$)). Once a new rule is issued after system deployment, it often necessitates several hours of downtime for knowledge reconstruction [6]. This contradicts the manufacturing sector’s requirement for real-time knowledge and continuous system operation.

(3)

Lack of answer credibility and auditability: For safety-critical applications in the shipbuilding industry, every engineering decision derived from the Q&A system must be fully traceable and justifiable. Current RAG strategies only provide an opaque black-box score based on text similarity. This lack of transparent graph paths and standard clause citations prevents engineers from confidently making judgments, failing to meet crucial auditability and accountability requirements.

In summary, this paper presents SWP-Chat, the first hybrid-RAG intelligent query-answering system for shipbuilding processes based on the “Generate–Retrieve–Generate” symmetric framework. In contrast to conventional RAG systems, the SWP-Chat system uses sophisticated LLM reasoning powers to first produce logical phrases, as shown in Figure 1. This logical form generation drives cypher queries on the knowledge graph, which work jointly with vector retrieval to effectively filter retrieval noise before the recall stage. This system design makes industry-grade advances in real-time knowledge updates and answer dependability, improves retrieval accuracy for complex situations, and successfully lowers noise interference in complex process queries. In complex process contexts, it offers a feasible route for intelligent question answering and decision assistance. The following are this paper’s primary contributions:

(1)

This study builds a symmetric KG+Vector dual engine that supports 11 types of welding entities, making it the first to apply symmetrical design to ship welding technology. This architecture achieves mirrored balance between the vector channel and knowledge graph channel. This bidirectional symmetry mechanism fundamentally isolates pre-retrieval noise, significantly enhancing the accuracy of complex reasoning and the coherence of question answering, with an F1 score improvement of 40.89%.

(2)

This paper avoids global graph embedding by explicitly adopting Neo4j Cypher symbolic queries. Furthermore, by integrating PCA vector dimensionality reduction technology, we successfully shortened the average retrieval latency by 31%. This achievement enables agile, minute-level updates of industrial knowledge, thereby circumventing the lag and system interruptions caused by the high recalculation cost of GraphRAG’s graph embedding and ensuring the continuity of knowledge services.

(3)

This work suggests an explainable fusion process that strikes a balance between source weighting and structural confidence. It offers transparent, auditable reliability proof for every choice made in shipbuilding scenarios, guaranteeing complete outcome traceability and adherence to strict safety regulations.

This article’s main organizational framework is outlined as follows: Section 2 reviews the related work of process knowledge graph, industrial RAG, and Hybrid-RAG. Section 3 elaborates in detail on the core methods of the proposed Hybrid-RAG intelligent question-answering system, including the system architecture, natural language question parsing and logical generation, and the construction and retrieval mechanisms of the knowledge graph and vector database. Section 4 introduces the experimental design, dataset construction, and evaluation metrics. Section 5 presents the results of ablation experiments, comparative experiments, and subjective evaluations, as well as detailed discussions. Section 6 summarizes the full text and looks forward to future research directions.

2. Related Work

2.1. Process Knowledge Graph

In the manufacturing industry, technological processes include bridge product design and production [7]. As product requirements have diversified, process planning has become increasingly complex and variable. Knowledge graphs are well-suited for modeling complex entities and their relationships and have therefore become an important tool for managing industrial process information.

Early manufacturing knowledge graphs were primarily built using ontology-driven methods. For example, Ford Motor Company began storing production knowledge in ontologies in 2017, enabling intelligent assistance for complex assembly processes. This allowed for intelligent support for complex product assembly processes. Yahya et al. [8] introduced the Generalized Ontological Model (RGOM) ontology framework based on the Resource Description Framework/Web Ontology Language (RDF/OWL) in the manufacturing area, combining elements like workpieces, welding procedures, and welding robots into a single semantic model. To check for process compliance, they used Semantic Web Rule Language (SWRL) rules. Similarly, a top-down strategy of “defining ontologies before populating entities” is emphasized in Meckler’s [9] procedure model for building knowledge graphs for industry applications. After using capability questions to define domain boundaries, it uses OWL constraints on classes, properties, and instances to accomplish controlled evolution of process knowledge graphs. However, these approaches still depend heavily on structured data. Their ability to extract and integrate knowledge is limited when facing semi-structured models, unstructured logs, or maintenance reports. In later levels of reasoning, this increases dependence on contextual interpretation, which often leads to conceptual ambiguity and unstable reasoning outcomes.

To overcome these limitations, researchers have begun combining knowledge graphs with intelligent algorithms, such as deep learning and statistical models, to improve automation and reasoning in process knowledge applications. The BITC deep network was proposed by Guo [10] et al. to automatically extract triples from large amounts of machine-generated free text. These triples were written into Neo4j following cosine similarity-based entity alignment, allowing process knowledge graphs to be constructed from semi-structured data from beginning to end. Liu [11] et al. developed a knowledge graph-based industrial knowledge question-answering system by using a naive Bayes classification model for semantic categorization. In order to increase the efficiency of process creation, Zhang [12] et al. combined deep learning with knowledge graphs for macro-level process planning.

Building upon this foundation, subsequent research increasingly focused on knowledge reasoning and dynamic generation. For ship heterogeneity models, Dong [13] built a multi-element process knowledge graph that allows for the logical creation of production processes and the quick reuse of case knowledge. Some research has moved toward intelligent question answering and assembly process planning as application demands have increased. For example, Zhou [14] created a knowledge network about the assembly process using a hierarchical modeling technique. They produced intelligent assembly process planning by combining distributed graph embedding with a sequence convolutional network, which greatly increased assembly automation and efficiency. In order to enable intelligent inference of product assembly sequences, Shi [15] created a multidimensional integrated assembly resource knowledge graph and combined it with the BERT model.

To summarize, the use of knowledge graphs in the field of current processes has evolved from ontology-driven to deep learning-integrated and from static knowledge organization to dynamic reasoning. In addition, KGs have shown great promise in decision support and manufacturing process planning, particularly in complicated modeling and knowledge reuse, where they have seen early success. However, current knowledge graph-based industrial systems still have notable limitations that make it difficult to meet the increasing complexity of modern industrial production:

(1)

Insufficient integration of unstructured knowledge: Many industrial process documents are text-based and unstructured. This knowledge is underutilized, which results in incomplete representation within knowledge graphs.

(2)

Weak intelligent question answering and interactivity: Existing KG-based systems focus mainly on retrieval and recommendation. They lack flexible natural language interaction, making it difficult to support engineers’ daily needs for quick querying and reasoning.

Therefore, with the breakthroughs of LLMs in natural language processing and reasoning capabilities, leveraging the language understanding ability of LLMs to make up for the shortcomings of KG in unstructured knowledge fusion and natural language interaction has become the key to the development and large-scale application of industrial intelligent question-answering systems.

2.2. Industrial RAG Technology

RAG is an effective method for combining LLMs with external knowledge sources and has been widely used to improve accuracy and reliability in specialized question-answering tasks. In industrial settings, RAG strategies have been applied to tasks such as equipment diagnostics, fault prediction, and path planning [16,17,18], providing intelligent solutions for the optimization of process parameters, diagnosis of malfunctions, and forecasting of quality outcomes. Compared with fine tuning, RAG reduces training costs by retrieving relevant documents first, then generating responses. It also reduces the risk of LLM hallucination and improves adaptation to domain-specific content. For instance, Werheid [19] et al. applied RAG to a “co-pilot” system for manufacturing equipment selection, providing engineers with high-quality references to address skill shortages and resource constraints. The RAGraph framework by Jiang et al. [20] supports both static and dynamic data, maintaining high performance in manufacturing scenarios without requiring fine tuning for specific tasks. In the sPecIalized KnowledgE and Rationale Augmentation Generation (PINKE-RAG) framework, Wang et al. [21] ensure coherent understanding and accurate responses to industrial process texts through a closed-loop system integrating domain-specific knowledge retrieval and LLM reasoning.

However, while RAG enhances text retrieval and single-hop reasoning capabilities, it still faces the following limitations in manufacturing processes:

(1)

Insufficient utilization of structured knowledge: RAG focuses on document fragment retrieval, making it difficult to directly manipulate process entities and relationships, leading to imprecise understanding of parameter associations and process flows.

(2)

Limited complex logical reasoning: For complex queries involving cross-document, multi-table, or multi-parameter dependencies, RAG often misses key reasoning steps, leading to incomplete answers.

(3)

Lagging knowledge updates and evolution: RAG retrieval repositories rely on manual document imports for updates, failing to reflect real-time iterations of new materials and processes.

In this paper, our SWP-Chat system introduces the symmetrical Hybrid-RAG technology to the manufacturing process domain. By integrating the structured advantages of knowledge graphs with the semantic matching capabilities of vector retrieval, it aims to effectively address the aforementioned shortcomings in unstructured knowledge fusion, complex reasoning limitations, and knowledge update delays. This approach seeks to achieve more efficient, precise, and reliable intelligent question answering.

2.3. Hybrid-RAG Technology

Hybrid-RAG combines the semantic matching of vector databases with the structured reasoning of knowledge graphs, overcoming weaknesses of single-path RAG in complex reasoning tasks. Works like those of Barron [22] and Opoku [23] have confirmed Hybrid-RAG’s efficacy in industries like healthcare and finance. Additionally, Hybrid-RAG was used by Russell-Gilbert [24] for adaptive anomaly detection, and Sarmah [25] also suggested unique knowledge graph–vector fusion frameworks to improve the accuracy and efficiency of information extraction.

However, applying Hybrid-RAG in industrial environments remains challenging. First, the majority of the work that has already been done pertains to fields that are more static than manufacturing, such as healthcare and finance, where real-time knowledge updates are less important. There are very few examples of direct industrial application. GraphRAG [26], the current major fusion approach, integrates the complete knowledge structure with text vector databases after encoding it into a continuous vector space using graph embedding. In addition, graph embedding methods introduce a black-box problem and do not meet industrial requirements for traceability and answer auditability. Due to a black-box problem in the answer generation process caused by this graph embedding technique based on numerical similarity, it is challenging to meet industry criteria for answer auditability and reliability. Furthermore, graph embedding has a very high recalculation cost, which significantly restricts real-time knowledge updates.

Given the aforementioned challenges and existing research gaps, the SWP-Chat system proposed in this paper builds upon traditional Hybrid-RAG strategies. It explicitly adopts Neo4j Cypher symbolic queries for knowledge graph retrieval, eschewing the graph embedding technique frequently employed in industrial GraphRAG systems. The significant update costs and black-box problems that come with graph embedding are essentially fixed by this. Although several recent studies have combined knowledge graphs with vector databases for hybrid retrieval in industrial domains [27], these applications mainly target general smart manufacturing or equipment maintenance and do not address the highly domain-specific constraints of ship welding processes. In contrast, the shipbuilding industry lacks an interpretable, auditable, and standards-aligned knowledge reasoning system. The system presents a “Generate–Retrieve–Generate” symmetric framework, which builds on this foundation. It makes use of LLMs’ sophisticated reasoning powers to produce logical forms first, then accurately drive dual-engine collaborative retrieval before combining the results to produce outputs in natural language. This efficiently suppresses retrieval noise and greatly improves the accuracy of complex industrial Q&A. The system processes the vector database using PCA vector dimensionality reduction, which substantially lowers query latency and further optimizes performance. In order to achieve industrial-grade trustworthiness and auditability, we simultaneously suggest an explainable fusion formula that integrates source weighting and structural confidence to meet the demanding dependability requirements of industrial scenarios.

3. Methodology

3.1. Overall System Architecture

In addition to the structurally controllable and semantically enriched “Generate–Retrieve–Generate” framework, the SWP-Chat system adopts a symmetric dual-channel fusion design. This design substantially enhances response accuracy, efficiency, and interpretability at an industrial level in ship welding scenarios and the efficiency of the response and the interpretation of industrial grade in the context of ship welding processes, in addition to successfully solving logical confusion and structural information gaps in complex question–answer scenarios. Figure 2 shows the particular architecture.

The system comprises three core modules: the User Question Understanding Module, the Database Construction Module, and the Information Retrieval and Response Generation Module.

The User Question Understanding Module is responsible for parsing natural language questions and identifying user intent. It provides logical forms using template matching, question classification, and keyword extraction to support downstream retrieval.

The Database Construction Module includes the building of a structured knowledge graph database based on Neo4j and the construction of an unstructured vector database using semantic embedding. It also incorporates PCA algorithms for dimensionality reduction of the vector database to enhance retrieval efficiency.

Under the Retrieval and Response Generation Module, using graph retrieval, the LLM initially establishes the fundamental question path and response structure. It then incorporates additional information from the Faiss vector database to provide contextual explanation and boundary refinement. This strategy guarantees the system’s continued robust scalability and accuracy.

This study uses a “semantic parsing-logic generation-structural querying-answer generation” symmetric paradigm of “generation–retrieval–generation” in which the knowledge-graph channel and vector-retrieval channel are driven by the same logical form developed in the first stage. The two retrieval processes run in parallel: the KG channel performs Cypher-based symbolic reasoning with strict entity–relation constraints, while the vector channel retrieves semantically related unstructured knowledge. This approach produces a domain-specialized Q&A system that maintains structural control and efficient response generation while preserving broad semantic coverage.

3.2. Problem Analysis and Logic Generation

This paper employs CoT-Prompting and a Naive Bayes classifier to rapidly perform semantic analysis and retrieve the intent of user queries. Additionally, it accomplishes phrase segmentation within user queries by constructing a TF-IDF feature extractor based on specialist welding terminology and jieba word segmentation. After segmentation, the findings are sent to an LLM that has been fine-tuned by LoRA to produce logical forms. This method shows more flexibility in addressing real-world issues in industrial contexts, like vague claims, sparse data, and non-standardized vocabulary.

3.2.1. Query Parsing and Intent Recognition

User natural language queries are initially received by large language models. However, because natural language queries are diverse and ambiguous, direct processing often leads to misunderstanding and inefficiency. Therefore, the primary task is to transform them into a more structured intermediate representation. To address this, the system applies a multi-stage parsing workflow, as shown in in Figure 3. It first utilizes the jieba tokenization tool to segment user questions into phrases. This allows for initial identification of candidate entities and relations, assisted by a predefined domain lexicon for similarity matching.

Subsequently, the segmented results are vectorized using the TF-IDF feature extractor [28]. TF represents term frequency, indicating how often a term appears in user queries, while IDF denotes inverse document frequency, reflecting a term’s rarity across the entire corpus. The TF-IDF formula is expressed as follows:

$$T{F}_{i,j}={\displaystyle \frac{n_{i,j}}{{\sum }_k{n}_{k,j}}}$$

(1)

$$ID{F}_i=lg{\displaystyle \frac{\left|D\right|}{1+|j:t_i\in d_j|}}$$

(2)

$$T\text{-}{I}_{i,j}=T{F}_{i,j}\times ID{F}_i$$

(3)

Here, $n_{i,j}$ denotes the number of occurrences of word i in question j and; ${\sum }_k{n}_{k,j}$ represents the total number of words in question j; $\left|D\right|$ denotes the total number of documents in the corpus; $d_j$ represents the j-th document in the corpus; $1+|j:t_i\in d_j|$ indicates the number of questions containing word $t_i$; $T\text{-}{I}_{i,j}$ signifies the importance of word i in question j, accounting for its global significance across the entire question set (D).

Statistical analysis shows that multi-hop queries account for only 9.6% of Q&A samples in real welding data. Traditional complexity classification is not applicable to this system due to the unbalanced distribution of data. Therefore, the system classifies user intent into two categories: welding requirements and process activities. This classification offers significant semantic guidance and is more in line with industry demands.

To achieve this intent classification, the system models question texts using a naive Bayes classifier [29] combined with a constructed keyword feature dictionary. Sample features include core term frequency, contextual collocation phrases, and structural template matching. The classifier is trained on a small, manually annotated dataset of question types and rapidly categorizes user questions during the prediction phase. The naive Bayes implementation of intent classification is expressed as follows:

$$f\left(x\right)=arg\underset{y_k}{max}P\left(y_k\right)\prod _i^{n}P(x_i\mid y_k),k=1,2,$$

(4)

where $x_i$ denotes the i-th feature in query x, $y_k$ represents one of two intent categories (process operation/welding specification) present in the query, and $P\left(·\right)$ denotes probability. After completing intent classification, the system performs slot filling to obtain complete structured information. The completed structured queries are then passed to the LLM for logical form generation and answer synthesis.

Furthermore, the intent recognition module conducts a preliminary evaluation for user-posted “boundary questions” that are not covered by the system’s knowledge graph and vector database. The system automatically treats the query as having a “non-business-related intent” and uses the LLM’s own general knowledge to generate a response once it has concluded that retrieving the information from the knowledge base is not an appropriate way to answer it. To ensure auditability, the system explicitly notifies users when an answer is generated solely by the LLM rather than retrieved from vetted knowledge sources. It shows that these responses come directly from the LLM itself, not from organized knowledge sources or other publications, and are, thus, solely offered for reference. By keeping the Q&A process transparent and ensuring that the system may still provide feedback on unknown queries, this technique helps users avoid misinterpreting the origins of the answers.

3.2.2. LLM Logic Generation and LoRA Fine Tuning

To enable LLMs to convert parsed queries into machine-interpretable logical forms, the model is fine-tuned using LoRA. The core of the LoRA algorithm involves introducing a parallel low-rank branch alongside the Transformer layer, performing a dimension reduction followed by dimension expansion. The specific structure is illustrated in Figure 4.

Only the low-rank matrix $\mathbf{A}$ (down-projection) and matrix $\mathbf{B}$ (up-projection) are trained, while the pretrained weights remain frozen. The model’s input and output dimensions remain unchanged. During output, the $\mathbf{BA}$ matrix is merged with the parameters of the pre-trained model. This approach minimizes trainable parameters and GPU memory usage while preserving fine-tuning performance. The following formula can be used to calculate the neural network layer’s output vector ($\mathbf{h}$) during fine tuning:

$$\mathbf{h}=\mathbf{W}\mathbf{x}+\mathbf{B}\mathbf{A}\mathbf{x},\mathbf{B}\in {\mathbb{R}}^{d\times r},\mathbf{A}\in {\mathbb{R}}^{r\times k},$$

(5)

where $\mathbf{x}$ represents the input vector before fine tuning; $\mathbf{W}$ denotes the pre-trained weight matrix; d is the output feature dimension of $\mathbf{W}$; r is the rank of the low-rank matrix; and $r\ll min(d,k)$, where k is the input feature dimension of $\mathbf{W}$.

To enable accurate logical form generation, the DeepSeek-V3 base bodel is fine-tuned using LoRA. In this study, the rank is set to $r=8$, the scaling factor is set to $\alpha =16$, and a dropout rate of 0.1 is applied to prevent overfitting. Training is performed using AdamW with a learning rate of $2\times {10}^{-4}$ for 3 epochs, which provides stable convergence without increasing computational overhead. The full hyperparameter configuration and rationale for these choices are provided in Appendix A.

A logical form is a structured representation of a natural language question. Taking S-expressions as an example, a logical form typically consists of projections and various operators. The projection operation represents a one-hop query on the triplet $(e_1,r,e_2)$ over either $e_1$ or $e_2$, where $(?,r,e_2)$ is represented as $\left(JOINr{e}_2\right)$ and $(e_1,r,?)$ is represented as $\left(JOIN\left(Rr\right)e_1\right)$. Practical application examples are shown in

Table 1. Example of generating logical forms.

Query	Logical Forms
What are the mechanical property indicators of welding material grade 1Y?	1. (JOIN Part_of mechanical properties of welding materials) 2. (JOIN Yield strength welding material grade 1Y) 3. (JOIN Elongation welding material grade 1Y)

.

3.3. Construction of Knowledge Graphs and Vector Databases

3.3.1. Definition of Welding Process Entities

Ship welding process knowledge includes extensive tabular data, images, process resources, and testing information. Therefore, welding process entities are divided into 11 categories, each assigned a corresponding label based on technical standards and literature [30].

Table 2. Welding process knowledge entity labels.

ID	Entity Label	Entity Example
1	Welding materials	Welding rods, flux, and associated gases
2	Welding method	Shielded metal arc welding, $C{O}_2$ gas shielded welding
3	Welding equipment	Torches, automatic gravity feeders, and gas chisels
4	Welding operation	Groove preparation, arc striking, tack welding, and workpiece heating
5	Welding base metal	Steel and aluminum alloys
6	Welded structure	Welding position, groove type, and joint design
7	Welding quality	Weld formation factors, welding defects, and various tests
8	Process parameter	Current, voltage, diameter, and wire feed speed
9	Table	-
10	Image	-
11	Others	-

provides specifics.

3.3.2. Graph Construction Based on Large Language Models

LLMs now show a great deal of promise for ontology building. Domain-specific fine-tuning prompts can be used to improve the domain adaptability of these models, as well as to build and refine ontologies. Therefore, the “Materials and Welding Specifications” document published by the CCS in 2024 [31] is the source of pertinent welding process content that is extracted in this work. The text input is preprocessed before being loaded into a sizable language model that is set up with pertinent instructions to extract appropriate symmetric triples. This triplet architecture imposes fundamental structural symmetry on knowledge representation, wherein each relation serves as a bilateral link between two balanced entities. Following manual refinement and expert validation, these are imported into Neo4j to construct a complete knowledge graph, as shown in Figure 5. Ultimately, 1583 valid entities and 2088 triples were extracted from the selected document sections.

3.3.3. Document Vector Database Construction

A Faiss vector database is constructed from both structured and unstructured text sources, following a structure-first, semantics-enhanced fusion strategy. Cosine similarity is used for vector retrieval, and Z-score standardization is applied to ensure consistent scaling across embedding dimensions. The Faiss index is implemented using the IVF-Flat structure, with nprobe = 16 to balance retrieval accuracy and latency. Therefore, the vector database is built according to the pipeline shown in Figure 6.

In order to guarantee semantic coherence, this research first uses the Embedding model (bge-zh-v1.5) to divide the CCS “Materials and Welding Specifications” document into segments of 128–512 tokens at the granularity of “paragraph + figure caption”, with a 20% stride. Semantic vectors with 1024 dimensions are then created from these parts. Finally, the 1024-dimensional embeddings are reduced using PCA. This enables initial coarse retrieval with reduced vectors, followed by top-K refinement using the original 1024-dimensional embeddings, thereby improving retrieval speed while maintaining accuracy.

As a classical linear dimensionality reduction method, PCA effectively removes redundant information and noise while preserving core semantic features. In the specific implementation process, we first applied Z-score normalization to the document embedding vectors to ensure the data had a mean of 0 and a variance of 1. Subsequently, the covariance matrix of the standardized data was calculated, and the eigenvalues and eigenvectors were extracted via eigen-decomposition, as shown in Figure 7. Considering the high process performance requirements in the domain of ship welding technology and based on the cumulative contribution rate of the eigenvalues, we ultimately chose a target dimension that retains $95\%$ of the data variance for the dimensionality reduction transformation. Specifically, the original 1024-dimensional vector database was reduced to 280 dimensions. The corresponding pseudocode for this dimensionality reduction task is provided in Algorithm 1.

Algorithm 1: PCA-Based Vector Dimension Reduction

Input: Set of original high-dimensional vectors $V^h$

Output: Set of reduced-dimension vectors $V^l$

1  Compute covariance matrix: $Cov=\mathrm{covariance}\left(V^h\right)$; 2  Compute eigenvalues and eigenvectors: $E^{\mathrm{val}},E^{\mathrm{vec}}=\mathrm{eigen}\left(Cov\right)$; 3  Select principal components: choose the first k eigenvectors based on cumulative explained variance threshold; 4  Data projection: $V^l=V^h·E^{\mathrm{vec}}[:,1:k]$; 5  return $V^l$;

Analysis of the vector database after dimension reduction reveals relevant parameter metrics, as shown in Figure 8 and Figure 9.

To evaluate the response time and accuracy of retrieval before and after dimensionality reduction, this paper primarily examines metrics such as retrieval latency and Mean Absolute Error (MAE). The corresponding MAE calculation formula is expressed as follows:

$$\mathrm{MAE}={\displaystyle \frac{1}{n}}\sum _{i=1}^n|y_i-{\widehat{y}}_i|$$

(6)

The results are shown in

Table 3. Performance Comparison Before and After Dimension Reduction.

Metric	Time Performance (ms)	Accuracy Performance (MAE)
Cosine Similarity	0.5365	0.3799
Cosine Similarity-PCA	0.1692	0.3814

. After dimensionality reduction, the retrieval response time was significantly reduced, with the average retrieval latency shortened by 31% on the system’s mixed image–text dataset. Although dimensionality reduction caused some loss of information features, leading to a slight decrease in the fine-grained discrimination capability of vector representations and a corresponding increase in MAE, the increase was less than 0.01, indicating a negligible impact. Therefore, this method is considered effective and acceptable in balancing retrieval efficiency and accuracy.

3.4. Hybrid Information Retrieval and Response Generation

3.4.1. Knowledge Graph Retrieval

This study applies an unsupervised entity–relation alignment method to match query keywords with Neo4j graph nodes and relation types. Let the entities in the keywords be denoted as $e_i^q$. The set of all m entities forms $Q_E=\{e_1^q,e_2^q,\dots ,e_m^q\}$. Let the entities in the graph be denoted as $e_j^g$ and the n entities set in the graph be $G_E=\{e_1^g,e_2^g,\dots ,e_n^g\}$.

A dimension-reduced text embedding model converts each entity name into semantic representations, followed by calculation of the semantic similarity between them:

$$s_{i,j}^e=\mathrm{CosSim}\left(\mathit{v}\left(e_i^q\right),\mathit{v}\left(e_j^g\right)\right)={\displaystyle \frac{\mathit{v}\left(e_i^q\right)·\mathit{v}\left(e_j^g\right)}{∥\mathit{v}\left(e_i^q\right)∥∥\mathit{v}\left(e_j^g\right)∥}}$$

(7)

Here, $\mathit{v}$ indicates the corresponding semantic vector. The $CosSim\left(·\right)$ symbol denotes the semantic similarity function. Its range is [−1, 1], where values closer to 1 indicate greater semantic similarity.

Subsequently, all candidate entities are sorted in descending order of similarity; the top $k_e$ entities are selected to form the candidate set ($in{d}_e=\{s_{i,j}^e,j=1,2,\dots ,n\}$), and those with similarity greater than or equal to the threshold ${\tau }_E=0.7$ are selected as final candidates:

$$S_E\left(e_i^q\right)=\left\{e_j^g\in in{d}_e\mid s_{i,j}^e\ge {\tau }_E\right\}$$

(8)

Similarly, the relationships extracted from keywords are denoted as $r_i^q$. The m relationships make up the set of $Q_R=\{r_1^q,r_2^q,\dots ,r_m^q\}$. The set of $G_R=\{r_1^g,r_2^g,\dots ,r_n^g\}$ is created by matching this set to the n relationships specified in the graph. The following is the definition of the similarity function:

$$s_{i,j}^r=\mathrm{CosSim}\left(\mathit{v}\left(r_i^q\right),\mathit{v}\left(r_j^g\right)\right)={\displaystyle \frac{\mathit{v}\left(r_i^q\right)·\mathit{v}\left(r_j^g\right)}{∥\mathit{v}\left(r_i^q\right)∥∥\mathit{v}\left(r_j^g\right)∥}}$$

(9)

Candidate relationships are sorted by similarity from highest to lowest, and the top $k_r$ relationships are selected to form the candidate set ($in{d}_r=\{s_{i,j}^r,j=1,2,\dots ,n\}$). Those with scores exceeding the threshold of ${\tau }_R=0.7$ are filtered out as final candidate relation words:

$$S_R\left(r_i^q\right)=\left\{r_j^g\in in{d}_r\mid s_{i,j}^r\ge {\tau }_E\right\}$$

(10)

Based on the entity set ($S_E\left(e_i^q\right)$) and the relation candidate set ($S_R\left(r_i^q\right)$), all possible structural query sets are constructed as follows:

$$S_C=S_E\times S_R=Comb(S_E,S_R),$$

(11)

where $Comb\left(·\right)$ denotes the ordered pairs formed by pairwise combinations of elements from sets $S_E$ and $S_R$, i.e., the Cartesian product.

For each candidate combination $(e,r)$, a corresponding Cypher query statement ($p(e,r)$) is constructed and executed in both the Neo4j graph database and the vector database:

$$answe{r}_p=Execute\left(p\right)$$

(12)

If the Cpyher query (p) returns matching entities and relations, the result is accepted as a valid retrieval component.

3.4.2. Vector Database Retrieval

After intent recognition, the system converts key information into a query vector for semantic retrieval. This procedure ensures that query vectors and document vectors in the database stay inside the same vector space by using identical PCA dimensionality reduction and using the bge-zh model for text embedding. For an effective similarity search, the query vector is then entered into the Faiss vector database. This method efficiently assesses the semantic significance between the query and document fragments by using cosine similarity matching for vector retrieval:

$$S_i^{Vec}=\mathrm{CosSim}\left({\mathit{v}}^q,{\mathit{v}}_i^{\mathrm{d}}\right)={\displaystyle \frac{{\mathit{v}}^q·{\mathit{v}}_i^{\mathrm{d}}}{∥{\mathit{v}}^q∥∥{\mathit{v}}_i^{\mathrm{d}}∥}}$$

(13)

Here, ${\mathit{v}}^q$ represents the user query vector after model embedding and PCA dimensionality reduction, while ${\mathit{v}}_i^{\mathrm{d}}$ denotes the fragment vector for document i. The Faiss engine returns the top-k most relevant text fragments based on cosine similarity. The next answer generation module receives these extracted text fragments, as well as the knowledge graph’s structured query results. These fragments are provided to the LLM as supplementary context, improving answer completeness and accuracy.

3.4.3. Hybrid-RAG Fusion Ranking Mechanism

The system applies a symmetric “Generate–Retrieve–Generate” framework, combining the knowledge graph and vector retrieval channels for unified reasoning. The relevant pseudocode is outlined in Algorithm 2 as follows.

After both retrieval steps, results are standardized into a unified format and ranked using confidence-weighted fusion:

$$scor{e}_i=\alpha \times K{G}_i^{score}+\beta \times S_i^{Vec}+\gamma \times W_i$$

(14)

where $\alpha =0.6,\beta =0.3$, and $\gamma =0.1$ are fusion weights determined through cross-validation on a small-scale validation set, aiming to maximize answer accuracy and information completeness. $K{G}^{score}$ denotes structural confidence, comprising path length, entity matching, etc., while W represents priority weights assigned based on source relevance. Finally, the bge-reranker-m3 model dynamically adjusts the optimal top-k ranking based on question intent. The selected results are then organized into a structured prompt and passed to the LLM, which integrates them to generate coherent natural language responses. (Detailed calculations of weight values ($\alpha ,\beta$, and $\gamma$) can be found in Appendix B).

Algorithm 2: Dual-Channel Fusion Sorting Pseudocode

Input: Knowledge graph retrieval results: ${\mathcal{R}}_{\mathrm{KG}}$(entities, relations, attributes, etc.)

Vector database retrieval results: ${\mathcal{R}}_{\mathrm{D}}$ (relevant document fragments)

Original User Query: Q

Output: The sorted results after integration ${\mathcal{R}}_{\mathrm{Final}}$

1  Weighted Merge Sort: 2      $C_i\leftarrow {\mathcal{R}}_{\mathrm{KG}}\cup {\mathcal{R}}_{\mathrm{D}}$ 3      Calculate $scor{e}_i$ 4      Generate preliminary rankings ${\mathcal{P}}_{\mathrm{pre}-\mathrm{rank}}\leftarrow \text{s\_d}(C_i,scor{e}_i)[:K]$ 5  Rerank: 6      $reRankScor{e}_i\leftarrow \mathit{\text{ReRank-Model.predict}}(Q,\mathrm{item})$ 7  Generate the final Ranking: 8      ${\mathcal{R}}_{\mathrm{Final}}\leftarrow \text{s\_d}({\mathcal{P}}_{\text{pre-rank}},reRankScore)[:N]$ 9  return  ${\mathcal{R}}_{\mathrm{Final}}$

4. Experimental Design and Evaluation

4.1. Experimental Equipment and Parameters

In this study, BGE-zh-v1.5 was selected due to its strong performance in Chinese dense retrieval tasks and its robustness in handling technical terminology typical of welding specifications. BGE-Reranker-m3 was adopted as a lightweight cross-encoder to refine top-k results while maintaining low computational overhead, making the system deployable in industrial settings. Detailed configurations are provided in

Table 4. Equipment configuration.

Equipment	Configuration
GPU	RTX 2080Ti 12G
CUDA	11.4
Python	3.9
Neo4j	5.18.1
LLM	DeepSeek-V3 BGE-Reranker-m3 BGE-zh-v1.5

.

4.2. Dataset

The dataset was built from the 2024 CCS Materials and Welding Specifications and was cleaned, deduplicated, and restructured for Q&A tasks. To create a high-quality Q&A dataset, the most representative and high-frequency industrial questions were selected. The final dataset contains 47 welding requirement questions and 39 process operation questions.

4.3. Evaluation Metrics

Performance is evaluated using precision, recall, F1 score, and MAP@k, following standard practice in information retrieval and NLP tasks.

• Precision
  Conceptual Q&A performance is evaluated by checking whether the generated answer aligns with the key content of the reference answer. The corresponding precision rate calculation formula is expressed as follows:

  $$Precision={\displaystyle \frac{C^w}{T^w}}$$

  (15)

  where $C^w$ denotes correct words generated by the model and $T^w$ denotes total words generated by the model.
• Recall
  The model’s recall indicates its capacity to recognize and incorporate all pertinent data. The following is the precise formula used to determine recall:

  $$Recall={\displaystyle \frac{N^{cw}}{N^{tw}}}$$

  (16)

  where $N^{cw}$ denotes the number of correct words generated by the model and $N^{tw}$ os total number of words in the standard answer.
• F1 Scores
  F1-scores combine precision and recall, making them more suitable for scenarios where answers contain partial matches. The following is the precise formula used to determine F1 scores:

  $$\mathrm{MAP}@\mathrm{k}={\displaystyle \frac{1}{M}}\sum _{i=1}^{M}AP@k_i$$

  (17)
• MAP@6
  Mean average precision at k ($MAP@k$) is a metric used to evaluate the quality of a model’s predictions in information retrieval tasks. It computes the average across all samples and assesses the significance of the top k outcomes anticipated by the model.

  $$F1=2\times {\displaystyle \frac{Precision\times Recall}{Precision+Recall}}$$

  (18)

  where M represents the number of welding process issues, and $AP@k_i$ denotes the Average Precision at k for the i-th issue, calculated as follows:

  $$\mathrm{AP}@k_i=\sum _{i=1}^n{P}_i·re{l}_i$$

  (19)
  $P_i$ denotes the accuracy at the i-th position in the prediction list, calculated as follows:

  $$P_i={\displaystyle \frac{\mathrm{Number}\mathrm{of}\mathrm{relevant}\mathrm{predictions}\mathrm{in}\mathrm{top}i}{i}}$$

  (20)

  where $re{l}_i$ is an indicator function: if the i-th prediction is in the actual answer list, $re{l}_i=1$; otherwise, $re{l}_i=0$. Because graph retrieval may produce multiple keywords or text segments, MAP@6 is used for evaluation.

5. Experimental Results and Discussion

5.1. Ablation Experiment

As clearly shown in

Table 5. Ablation experiment.

Model	Precision (%)	Recall (%)	F1 Score (%)	MAP@6 (%)
SWP-Chat	74.74	84.57	79.35	81.48
SWP-Chat w/o KG	35.77	55.92	43.63	-
SWP-Chat w/o LoRA	66.20	53.24	59.02	63.30
SWP-Chat w/o KG, LoRA	34.43	43.56	38.46	-

Note: Bold = best result.

, the experimental results demonstrate that removing the knowledge graph (w/o KG) leads to a marked drop in the F1 score from 79.35% to 43.63%, underscoring that the semantic–logical symmetry established by the KG + vector dual-engine architecture is fundamental to system effectiveness.

The knowledge graph not only delivers accurate structured representations of process entities but also provides a logical constraint mechanism that helps resolve semantic ambiguity and supports complex reasoning chains.

When the LoRA module is removed (w/o LoRA), MAP@6 declines from 81.48% to 71.44%, and the F1 score drops to 69.42%. These results confirm that LoRA fine tuning considerably enhances logical form generation and improves the model’s grasp of welding-specific terminology and semantics.

While the Hybrid-RAG architecture combined with LoRA provides significant performance improvements, the overall metrics remain slightly below 90% due to inherent variability in the test data. Minor batch differences and qualitative expressions in reference answers may cause small score fluctuations, though these have a negligible impact on the practical accuracy of the system.

5.2. Comparative Experiments

To assess the system’s domain-specific performance, SWP-Chat was evaluated on the same dataset against several mainstream RAG-enhanced large language models, including Qwen, ChatGLM, and DeepSeek.

Since these baseline models do not contain domain knowledge of manufacturing or welding, a shared vector database was adopted to ensure comparability. On this basis, the proposed model integrates a structured knowledge graph and a dual-channel fusion strategy to verify the contribution of structural reasoning to answer precision and domain relevance.

As summarized in

Table 6. Effectiveness evaluation of process operation Q&A.

	Metric	Precision (%)	Recall (%)	F1 Score (%)	Processing Time (s)
Model
DeepSeek-R1		28.13	32.68	30.23	4.20
ERNIE4.5-8k		29.17	33.42	31.15	3.87
Llama3-70b		28.93	33.07	30.86	3.69
ChatGLM3-32k		28.71	33.51	30.92	3.67
Qwen3-32B		29.40	33.33	31.24	3.88
Ours		78.08	84.65	81.23	5.50

Note: Bold = best result.

, SWP-Chat achieved the highest performance across all key metrics. It outperformed existing RAG-based LLM systems in precision, recall, and F1 score, achieving an F1 score of 81.23%. This clearly demonstrates the effectiveness of the proposed Hybrid-PCA-RAG architecture and the symmetric “Generate–Retrieve–Generate” design in tackling complex process-oriented Q&A tasks.

The main advantage of SWP-Chat lies in its ability to represent and reason over multi-step procedural logic rather than relying solely on fragment-level retrieval as in traditional RAG systems. By generating structured logical forms that guide Cypher-based reasoning, the system can answer multi-stage process questions with greater coherence and accuracy.

Although its average response time is approximately 43% longer than conventional RAG systems, this trade-off is acceptable in industrial contexts where traceability and correctness take precedence over speed.

As shown in

Table 7. Effectiveness evaluation of welding specification Q&A.

	Metric	Precision (%)	Recall (%)	F1-Scores (%)	Processing Time (s)
Model
DeepSeek-R1		38.88	53.07	44.88	4.36
ERNIE4.5-8k		41.69	48.81	44.97	3.03
Llama3-70b		38.64	51.39	44.11	3.22
ChatGLM3-32k		38.36	51.24	43.87	3.16
Qwen3.5-144b		38.90	51.98	44.50	3.35
Ours		71.97	84.50	77.73	5.42

Note: Bold = best result.

, the proposed system achieves an F1 score of 77.73% in welding specification question answering, maintaining a leading position among all compared models. This demonstrates the effectiveness of the proposed approach for handling domain-specific queries with complex contextual dependencies.

Welding specifications usually contain multiple parameters, standards, and regulatory clauses stored in structured or semi-structured formats. In such contexts, the knowledge graph plays a central role—its explicit entity–relationship–entity symmetry compensates for the limitations of pure vector retrieval by accurately representing constraints and relationships within the requirements.

Comprehensive analysis reveals a consistent pattern across all models, including ours, where precision is lower than recall. This is not a model defect but, rather, an intentional optimization aligned with user needs in practical engineering scenarios. Unlike process operation queries, which rely on numerical or parameterized data (e.g., current, voltage, or welding speed), specification-related questions often involve unstructured text derived from standards or manuals. Such content is rich in complex semantics, synonyms, and context-dependent expressions, which naturally favor higher recall during retrieval.

5.3. System Usability Analysis

To evaluate the value of this system in practical industrial applications, we further conducted a comparative analysis between SWP-Chat and traditional knowledge graph-based query methods in terms of usability and interaction efficiency.

As shown in

Table 8. Model performance and remarks.

Model	Results	Remark
Only-KG	Query statement: MATCH (n:Other{name:“Hydrogen Content Grade H15”})-[r:Measurement Method]-(m); MATCH (n:Other{name:“Hydrogen Content Grade H15”})-[r:Diffusion Hydrogen Content]-(q) RETURN m,q Query Results: [$15{\mathrm{cm}}^3$]; [$10{\mathrm{cm}}^3$]; [Mercury Method]; [Thermal Conductivity Method]; [Glycerin Method]	This requires knowledge of Cypher, increasing the operational threshold and limiting usability for non-technical users; query results are returned as unordered nodes (content and method), preventing employees from directly associating specific methods with content.
SWP-Chat w/o KG, LoRA	Hydrogen content grade H15 may be determined using the mercury method, thermal conductivity method specified in ISO 3690 [31], or the glycerol method specified in 2.3.6.2. The corresponding diffusion hydrogen measurement procedure and specimen preparation quantity shall be performed according to Table 2.3.6.1 (the thermal conductivity method must meet the minimum collection time requirements at different test temperatures specified in Table 2.3.6.1). The hydrogen content grade is H15.	Without a knowledge graph structure, the system cannot accurately extract key information, leading to redundant and sometimes ambiguous responses. In addition, The system merely returns isolated strings such as “Table 2.3.6.1”, stripped of any explanatory context.
SWP-Chat w/o LoRA	The definition of hydrogen content grade H15 cannot be found in the provided information. Applicable measurement methods include the glycerol method. Diffusion hydrogen content refers to the hydrogen content collected through hydrogen measurement tests (such as thermal conductivity, gas chromatography, or carrier gas thermal extraction) under specific	Without LoRA fine tuning, the model exhibits insufficient intent recognition and logical generation capabilities, resulting in recall failures and an inability to generate correct Cypher statements. This leads to poor system reliability and incomplete information retrieval.
SWP-Chat w/o KG	Hydrogen content grade H15 can be measured using the mercury method, thermal conductivity method, or glycerol method. The corresponding diffusion hydrogen content cannot be determined from the information provided.	Vector-only retrieval introduces high semantic noise and lacks precise inference, making it difficult to determine exact hydrogen diffusion values.
SWP-Chat	The measurement methods corresponding to hydrogen content grade H15 include the mercury method, thermal conductivity method, and glycerin method. When using the glycerin method, the diffusion hydrogen content is $10{\mathrm{cm}}^3$; for other methods, the diffusion hydrogen content is $15{\mathrm{cm}}^3$.	The combination of LLM-based logic generation and automatic Cypher execution enables zero-threshold natural interaction, providing concise and accurate answers, regardless of user expertise.

, traditional KG-based queries require users to write Cypher statements, creating a high technical barrier and returning unordered results that are difficult to interpret. Pure vector retrieval (w/o KG) often yields redundant or inconsistent answers, while models without LoRA (w/o LoRA) fail to generate correct logical chains, leading to incomplete or ambiguous outputs.

SWP-Chat overcomes these limitations through its symmetric Hybrid-RAG design. By automatically generating logical forms and executing Cypher queries, it enables natural language interaction without requiring database expertise. The system’s dual-engine retrieval—semantic vector matching and symbolic reasoning—ensures precise and context-aware responses, even for complex welding queries.

Ultimately, SWP-Chat delivers concise, interpretable, and highly accurate answers in fluent natural language. This balanced input–output symmetry, combined with dual-channel retrieval, markedly enhances industrial reliability, reasoning accuracy, and user experience in ship welding process applications.

5.4. Significance Testing

To objectively validate the performance enhancement of the proposed Hybrid-PCA-RAG fusion strategy for question-answering systems, this paper compares the constructed system with a system based on traditional RAG technology. Wilcoxon signed-rank tests are employed to conduct significance analysis on the query-answering results.

5.4.1. Wilcoxon Signed-Rank Test

The Wilcoxon signed-rank test is used to compare two paired samples and determine whether the performance differences are statistically significant. For samples $X_1,X_2,\dots ,X_i$ obtained by this system and samples $Y_1,Y_2,\dots ,Y_i$ obtained by the RAG-only system, the difference ($d_i$) between each pair of data is calculated as follows:

$$d_i=X_i-Y_i$$

(21)

Subsequently, $|d_i|$ are sorted in ascending order, assign each to a rank ($R_i$), and the rank sums are computed for corresponding positive and negative differences:

$$\begin{array}{cc}\hfill & \mathrm{Rank}\mathrm{sum}\mathrm{for}\mathrm{positive}\mathrm{differences}:W^{+}=\sum R_i^{+}(d_i>0)\hfill \\ \hfill & \mathrm{Rank}\mathrm{sum}\mathrm{for}\mathrm{negative}\mathrm{differences}:W^{-}=\sum R_i^{-}(d_i<0)\hfill \end{array}$$

Finally, the relevant test statistic is calculated by taking $W=\min (W^{+},W^{-})$. For large samples in this system ($n>30$), the normal approximation is employed in this paper:

$$Z={\displaystyle \frac{W-{\mu }^W}{{\sigma }^W}},$$

(22)

where p is the effective sample size, corresponding to ${\mu }^W$, ${\sigma }^W$, and the calculation formula is expressed as follows:

$${\mu }^W={\displaystyle \frac{p(p+1)}{4}}$$

(23)

$${\sigma }^W=\sqrt{{\displaystyle \frac{n(n+1)(2n+1)}{24}}}$$

(24)

5.4.2. Results Analysis

Given the high-precision requirements for question-answering performance in practical applications, this study sets the significance level ($\alpha$) at 5%. The F1 score was calculated for each query–answer pair using both the proposed method and the baseline model, yielding 86 paired samples. The test results were $W=375, Z=-6.44, p<0.001$, indicating a statistically significant improvement. The effect size was calculated as $r=Z/\sqrt{N}$, resulting in $r=0.694$, indicating a large effect.

To control for multiple comparisons, Bonferroni correction was applied to F1, precision, and recall, yielding an adjusted threshold of ${\alpha }^{\prime }=\alpha /3=0.0167$. All three p-values remained below this level ($p<0.0167$), confirming that the observed improvement is statistically robust rather than random.

After 1000 bootstrap resamples, the 95% confidence interval for the F1-score gain over the “w/o KG, w/o LoRA” variant was [33.55, 50.35] and [13.68, 28.93] compared with “w/o LoRA”, further supporting the significant benefit of the symmetric generate–retrieve–generate framework integrated with Hybrid-RAG.

As illustrated in Figure 10, Figure 11 and Figure 12, SWP-Chat achieves the highest values for F1 score, precision, and recall among all systems. Its median lines lie well above those of the ablation variants, and both upper and lower quartiles shift rightward, indicating consistently stronger performance rather than isolated high scores.

Although a few lower outliers are observed, the higher median and narrower interquartile range demonstrate stable and reliable improvements across most queries. Together, the statistical tests and distribution visualizations confirm that the proposed Hybrid-RAG strategy with the symmetric “generate–retrieve–generate” framework delivers significant and consistent gains in welding process question answering.

6. Conclusions

This paper proposes SWP-Chat, a symmetric Hybrid-RAG question-answering system designed to support intelligent decision making in ship welding processes.The system significantly outperforms conventional RAG baselines across multiple evaluation metrics.

Unlike typical generate-then-retrieve LLM workflows, SWP-Chat first converts natural language queries into structured logical forms that guide precise Cypher queries over a Neo4j knowledge graph. This generation-first design reduces retrieval noise and prevents loss of structured information when answering complex procedural questions.

The approach fuses the logical rigor of the knowledge graph with the semantic breadth of a vector database, creating a practical semantic–logical balance. Vector retrieval provides high semantic recall, while Neo4j enforces entity–relation constraints and deterministic reasoning. Together, these parallel channels deliver richer, more accurate context for diverse question types.

To improve efficiency, PCA is applied to compress 1024-dimensional embeddings to a lower-dimensional space; this reduces average retrieval latency by 31% while retaining over 95% of variance. Ablation studies show that PCA preserves retrieval accuracy while substantially improving response speed, supporting real-world deployment.

Although the system performs well overall, several limitations remain:

(1)

Knowledge graph maintenance and coverage: Real-time updates rely on manual review, and incomplete KG coverage limits structured reasoning, even when relevant documents exist in the vector database. Expanding multi-source data integration and automated update mechanisms will improve consistency.

(2)

Segmentation granularity of the vector database: Paragraph-based segmentation can compress key procedural details into short references, causing fragments to lose critical context during retrieval. Adaptive merging strategies based on syntax or topic coherence will improve semantic integrity.

(3)

Ambiguity in user queries: Without explicit disambiguation, identical terms may have different meanings in different process contexts, producing partial or incorrect answers. Future work will add LLM-based entity disambiguation and clarification questions to reduce ambiguity.

In summary, the symmetric Hybrid-RAG framework provides interpretable, audit-ready answers suitable for industrial use. Future work will address automatic KG expansion, dynamic chunking strategies, and incremental learning to improve scalability and robustness.