A Dual-Channel Multimodal RAG System: OCR- and Semantic Description-Driven Question Answering for Industrial Robot After-Sales Service

Abstract

Industrial robot after-sales question answering often depends on multimodal evidence, such as error screenshots, interface displays, and wiring diagrams, which are difficult for conventional text-based retrieval-augmented generation (RAG) systems to exploit effectively. To address this issue, we design a dual-channel multimodal RAG system that converts image content into retrievable textual knowledge through the collaboration of optical character recognition (OCR) and structured semantic description. In the proposed system, OCR is used to extract explicit textual cues, such as error codes, parameter fields, and interface prompts, while expert-authored semantic descriptions complement implicit visual evidence, including device parts, fault phenomena, and contextual scene information. The transformed knowledge is further integrated into a hybrid retrieval pipeline that combines dense retrieval and BM25, followed by Reciprocal Rank Fusion (RRF) and Maximal Marginal Relevance (MMR) reordering to improve both relevance and contextual diversity. Experiments on a real-world industrial robot after-sales dataset show that the proposed method achieves an overall question-answering accuracy of 87.9%, outperforming the LLM-only baseline by 35.6 percentage points. For image-related questions, accuracy improves from 46.7% to 83.3%. These results indicate that the proposed framework provides a deployment-friendly and interpretable system-level alternative to end-to-end multimodal model fine-tuning for industrial after-sales question answering.

1. Introduction

Industrial robots are increasingly deployed in manufacturing, logistics, and other high-value production environments, which places growing pressure on after-sales service systems. In practical maintenance scenarios, after-sales engineers must respond to user queries involving equipment models, alarm codes, controller parameters, operation procedures, and fault symptoms under strict efficiency requirements. Unlike general customer service tasks, industrial robot after-sales question answering is characterized by dense domain terminology, long-tail fault patterns, and frequent reliance on multimodal evidence, such as error screenshots, interface prompts, wiring diagrams, and component status images. These characteristics make it difficult for conventional text-centered knowledge services to provide accurate, timely, and traceable support.

Recent advances in retrieval-augmented generation (RAG) have provided an effective paradigm for improving the factuality and domain adaptability of large language models by grounding generation in external knowledge [1,2,3,4,5]. However, standard RAG pipelines remain insufficient for industrial after-sales scenarios for two reasons. First, user queries in this domain often mix standardized technical identifiers with colloquial descriptions of failure phenomena. Dense retrieval is effective for capturing semantic similarity, but it may confuse technically distinct yet semantically related terms [6,7,8]. In contrast, sparse retrieval methods such as BM25 preserve exact term matching, but they are less robust to diverse and non-standard user expressions. Second, most existing RAG systems are designed primarily for textual corpora and therefore have limited ability to exploit visual evidence that is critical for industrial diagnosis and troubleshooting.

To address the multimodal nature of after-sales knowledge, current approaches mainly follow two directions. One direction relies on end-to-end multimodal large language models or vision–language models to jointly reason over image and text inputs [9]. Although these methods offer strong cross-modal modeling capacity, they typically incur high computational cost, limited interpretability, and reduced deployment flexibility in resource-constrained industrial environments. The other direction converts visual information into retrievable textual or structured representations before integrating it into RAG frameworks [10,11,12]. This strategy is more compatible with existing retrieval pipelines and offers better traceability, but conventional OCR-only conversion is often insufficient because industrial images contain not only explicit textual cues, such as error codes and interface prompts, but also implicit semantic information, such as damaged components, abnormal indicator states, and scene context. As a result, deployment-feasible and semantically complete multimodal knowledge transformation remains an open challenge for industrial after-sales question answering [13,14].

Semantic communication and task-oriented semantic representation have recently attracted increasing attention because they emphasize the transmission and utilization of task-relevant meaning rather than only bit-level or symbol-level information. Recent surveys summarize semantic communication as a paradigm that integrates application requirements and information meaning into communication and data-processing procedures [15]. Existing studies have explored semantic communication in various scenarios, including cooperative control, wireless communication, and multimodal perception. For example, semantic communication has been investigated for decentralized UAV swarm control, where task-relevant semantic information is used to support cooperative tracking under wireless channel constraints [16]. Reliable multimodal semantic communication has also been studied for audio-visual event localization, where semantic coding and cross-modal information fusion are used to improve the robustness of multimodal understanding [17]. These studies indicate that semantic information modeling is not limited to natural language representation, but can also serve as a bridge between raw multimodal signals and task-specific decision-making.

Although the proposed semantic description mechanism is related to the above semantic representation perspective, its objective is different from conventional semantic communication. Semantic communication mainly focuses on transmitting compact task-relevant meaning under communication constraints, whereas this work focuses on transforming industrial image evidence into retrievable textual knowledge for retrieval-augmented question answering. In this sense, the structured semantic description channel can be regarded as a task-oriented semantic representation mechanism that converts implicit visual fault evidence into explicit and traceable knowledge units.

In this work, we design and implement a domain-specific dual-channel multimodal RAG system for industrial robot after-sales question answering. The proposed framework converts image evidence into retrievable textual knowledge through the collaboration of two complementary channels: an OCR channel for extracting explicit textual content and a structured semantic description channel for capturing implicit visual semantics that OCR cannot reliably represent. Based on this transformed knowledge, we further construct a hybrid retrieval pipeline that combines dense retrieval with BM25, and introduce Reciprocal Rank Fusion (RRF) and Maximal Marginal Relevance (MMR) to improve both candidate relevance and contextual diversity. In this way, the framework aims to achieve a practical balance among retrieval accuracy, interpretability, and deployment cost in real-world industrial environments. To better illustrate the engineering differences between existing approaches and the proposed framework, a comparison is provided in

Table 1. Engineering comparison between pure LLM, conventional text-RAG, and the proposed dual-channel RAG framework.

Dimension	Pure LLM/VLM Path	Conventional Text-RAG	Proposed Dual-Channel RAG
Knowledge grounding	Relies mainly on model parameters; external evidence is not explicitly retrieved	Retrieves text chunks from external documents	Retrieves knowledge blocks constructed from OCR text, structured semantic descriptions, and metadata
Visual evidence handling	Can directly process images when using VLMs, but inference cost and traceability are less controllable	Usually discards image evidence or uses OCR-only text conversion	Converts explicit text and implicit visual evidence into retrievable textual representations
Terminology matching	May confuse similar robot models, alarm codes, or component identifiers	Depends on either keyword or semantic matching according to the retriever design	Combines BM25 for exact terms and dense retrieval for semantic variants
Evidence traceability	Generated answers are difficult to trace to specific documents or images	Can cite retrieved text blocks	Retains source image/document metadata for evidence-level traceability
Main trade-off	Simple interface but higher hallucination risk for private industrial knowledge	Easy to deploy for text corpora but weak for multimodal fault evidence	Requires expert semantic annotation but improves multimodal evidence coverage and interpretability

.

The main contributions of this work are summarized as follows. First, we propose a task-oriented dual-channel knowledge transformation mechanism for industrial after-sales images. It does not treat OCR and semantic description as two independent auxiliary tools, but organizes them into a unified representation in which OCR provides exact textual anchors and structured semantic descriptions provide expert-verifiable visual fault semantics. Second, we construct a retrieval pipeline that explicitly combines dense retrieval, BM25, RRF, and MMR to balance semantic generalization, terminology precision, and contextual diversity. The contribution of this pipeline is system-level synthesis for industrial robot after-sales question answering rather than the invention of a new ranking algorithm. Third, we provide empirical validation on a real-world industrial robot after-sales dataset and further analyze annotation overhead, fusion-weight sensitivity, retrieval ablation, and engineering trade-offs. These analyses clarify both the effectiveness and the practical limitations of the proposed framework.

It should be emphasized that the novelty of this work does not lie in redefining individual algorithms such as BM25, RRF, or MMR. Instead, the scientific and engineering contribution lies in the task-oriented organization of multimodal industrial evidence, the explicit modeling of OCR–semantic complementarity, and the systematic validation of this design in a real after-sales service scenario. Overall framework of the proposed dual-channel multimodal RAG system, as shown in Figure 1.

2. Related Work and Positioning

2.1. Semantic Communication and Task-Oriented Semantic Representation

Semantic communication has shifted part of communication-system research from symbol-level transmission to the delivery of task-relevant meaning. Recent survey work classifies semantic communication into semantic-oriented, goal-oriented, and semantic-aware communication, and highlights its applications and open challenges in future intelligent networks [15]. In this view, the value of a representation is measured not only by whether the original signal can be reconstructed, but also by whether the transmitted or retained information supports the target task. Recent studies have applied this idea to cooperative control and multimodal localization. For example, decentralized semantic communication has been investigated for UAV swarm cooperative tracking over wireless MIMO fading channels [16]. Reliable multimodal semantic communication has also been explored for audio-visual event localization, where cross-modal semantic representations are used to improve robust event understanding [17].

The semantic description channel in this work is related to the task-oriented semantic representation idea, but the technical objective is different. Instead of designing a communication coding scheme, this paper transforms implicit visual evidence in industrial robot after-sales images into explicit, retrievable, and verifiable textual knowledge. Therefore, semantic description is used as a knowledge-representation mechanism for RAG rather than as a communication-transmission protocol.

2.2. Multimodal and Hybrid Retrieval-Augmented Generation

RAG improves factuality by grounding large language model generation in external knowledge [1,6]. Standard RAG usually retrieves text chunks using sparse retrieval, dense retrieval, or a hybrid of both. Sparse retrievers such as BM25 preserve exact lexical matching and are useful for alarm codes, parameter names, and model identifiers, while dense retrievers are more robust to semantic paraphrases and colloquial user descriptions [7,18,19]. Hybrid RAG pipelines combine these advantages, and additional techniques such as HyDE can improve semantic retrieval by generating hypothetical documents before retrieval. However, these text-centered methods remain insufficient when the key evidence is stored in screenshots, wiring diagrams, or component-state images.

Recent multimodal RAG studies attempt to bridge this gap by incorporating visual documents or multimodal large language models. For example, DRAG proposes a dual-channel retrieval-augmented framework for hybrid-modal document understanding, separately preserving visual structural information and semantic information [20]. Other multimodal frameworks integrate RAG with multimodal large language models to support complex data analysis over text and visual sources [21]. Recent benchmark work further emphasizes that multimodal RAG evaluation should consider multi-modal question answering, fact verification, image captioning, and image re-ranking under consistent retrieval and generation settings [22]. Compared with these general document-oriented, multimodal-LLM-oriented, or open-domain benchmark settings, this paper focuses on a narrower but highly practical industrial after-sales scenario. The proposed system emphasizes expert-verifiable semantic descriptions, exact industrial terminology matching, source traceability, and deployment feasibility.

2.3. Position of the Proposed Framework

The proposed method should therefore be understood as an engineering-oriented multimodal RAG framework for industrial robot after-sales service rather than a new foundational retrieval algorithm. Its main distinction from conventional OCR-RAG is that it supplements visible text with structured semantic descriptions of non-textual fault evidence. Its main distinction from general hybrid RAG is that the retrieval unit is no longer a purely textual document chunk, but a multimodal knowledge block containing OCR evidence, semantic annotation, and source metadata. Its main distinction from end-to-end multimodal LLM methods is that it avoids full model fine-tuning and preserves evidence traceability through explicit retrieval.

3. Materials and Methods

3.1. Framework Overview

This study proposes a dual-channel multimodal RAG framework for industrial robot after-sales question answering. The framework is designed to improve the utilization of multimodal fault evidence, particularly images such as error screenshots, operation interfaces, and wiring diagrams, which are difficult to exploit effectively in conventional text-based RAG systems.

The overall pipeline consists of two major stages: multimodal knowledge transformation and retrieval-augmented answer generation, as shown in Figure 2. In the first stage, image content is converted into retrievable textual knowledge through two complementary channels, namely OCR-based explicit text extraction and structured semantic description of implicit visual evidence. In the second stage, the transformed knowledge is incorporated into a hybrid retrieval pipeline that combines dense retrieval and BM25, followed by rank fusion and diversity-aware reordering. Through this design, the proposed framework aims to achieve a practical balance among semantic understanding, terminology precision, contextual completeness, and deployment cost in industrial after-sales environments.

3.2. Dual-Channel Knowledge Mechanism

The key challenge addressed in this work is that industrial images contain both explicit textual information and implicit visual semantics. Explicit textual cues include alarm codes, interface prompts, parameter fields, and equipment labels, while implicit visual evidence includes abnormal indicator states, damaged components, wiring anomalies, and scene context. Relying on only one type of representation often leads to incomplete knowledge extraction. To address this issue, we design a dual-channel knowledge transformation mechanism that converts both types of information into unified retrievable knowledge blocks. This mechanism includes an OCR channel for extracting visible text and a structured semantic description channel for representing non-textual diagnostic evidence. The outputs of the two channels are then integrated with source metadata to construct the final textual representation for retrieval and generation.

3.2.1. OCR-Based Explicit Text Extraction

The first channel is designed to extract directly observable textual information from industrial images. In practical after-sales scenarios, such information often includes error codes, alarm prompts, device labels, control parameters, and interface text, all of which play an important role in fault localization and procedural guidance. To improve OCR robustness under real industrial conditions, input images are first preprocessed through grayscale conversion and adaptive binarization to enhance foreground–background contrast and suppress noise. The processed images are then passed to Tesseract OCR (v5.3) to generate textual outputs together with confidence scores. To reduce the impact of reflections, occlusions, blur, and viewpoint variation, a confidence threshold is applied to filter low-confidence recognition results before indexing. Compared with directly using image inputs in end-to-end multimodal reasoning models, this OCR-based strategy offers lower computational cost, higher interpretability, and better engineering compatibility [23]. However, OCR alone remains insufficient because it can only capture visible character-level information and cannot reliably represent non-textual fault cues, such as rapid red-light flashing, loose wiring, or cracked housings.

3.2.2. Structured Semantic Description of Implicit Visual Evidence

To complement the limitations of OCR, a second channel is introduced to represent implicit visual semantics that are diagnostically important but not explicitly expressed in text. In this channel, domain experts annotate images using a unified structured template. The template contains four dimensions: scene, position, phenomenon, and key elements. The scene field describes the operating condition or process stage of the robot; the position field indicates the physical location where the anomaly occurs; the phenomenon field summarizes observable abnormal behavior; and the key-elements field records additional visual cues that may support diagnosis. Compared with unrestricted free-text descriptions, this structured schema improves annotation consistency and retrieval stability. In addition, because the semantic evidence is explicitly written into text, the resulting knowledge representation remains traceable and easier to verify than implicit cross-modal representations learned by black-box multimodal models. As listed in

Table 2. Dimension definition and examples of manual semantic description templates.

Dimension	Definition and Function	Example
Scene	The working state or process flow of the robot	During welding operationsinitiation of self-inspection phase
Position	The exact physical location where the fault occurred	Six-axis servo motorcontrol cabinet motherboard
Phenomenon	Visually observed abnormal manifestations (OCR blind spots)	Accompanied by severe shakingred indicator flash
Key elements	Implicit non-textual key cues in images	Excessive environmental humiditydamaged sensor housing

, to improve annotation consistency, each image is described according to the same four-field template rather than unrestricted free-form text. In practice, the semantic description is first drafted by an after-sales engineer familiar with the corresponding robot model and fault scenario, and then checked by another engineer to ensure that the described phenomenon is consistent with the original image evidence and official maintenance knowledge. This process is intended to reduce subjective interpretation while preserving expert-verifiable diagnostic information.

3.2.3. Unified Knowledge Block Construction

After the two channels are completed, the system integrates OCR outputs, structured semantic descriptions, and image-related metadata into a unified textual knowledge block. The retained metadata include source image information, file path, page number, image identifier, and other descriptive fields that support retrieval traceability and answer interpretation. To construct the final representation, a weighted fusion strategy is adopted. The fusion formula is defined as follows:

$${\mathbf{z}}_i=\alpha {\widehat{\mathbf{z}}}_i^{\mathrm{ocr}}+\beta {\widehat{\mathbf{z}}}_i^{\mathrm{sem}}+\gamma {\widehat{\mathbf{z}}}_i^{\mathrm{meta}},\alpha +\beta +\gamma =1.$$

(1)

where ${\widehat{\mathbf{z}}}_i^{\mathrm{ocr}}$, ${\widehat{\mathbf{z}}}_i^{\mathrm{sem}}$, and ${\widehat{\mathbf{z}}}_i^{\mathrm{meta}}$ denote the normalized embedding representations of OCR text, semantic descriptions, and metadata, respectively. In implementation, the three text fields are encoded using the same embedding model and then normalized by L2 normalization before weighted fusion, so that the contribution of each channel is controlled mainly by the assigned weight rather than by vector magnitude. The coefficients $\alpha$, $\beta$, and $\gamma$ control the relative contribution of each channel.

As shown on Figure 3, given that critical knowledge in industrial after-sales scenarios often exists not only in explicit textual form but also in visual relations, component states, and contextual environments, this study employs a weighted representation strategy that prioritizes semantic descriptions while retaining OCR information as a supplement. The weights were selected based on validation-set performance rather than fixed arbitrarily. Specifically, we performed a grid search over feasible weight combinations under the constraint $\alpha +\beta +\gamma =1$. Since the semantic description channel captures non-textual visual evidence that cannot be extracted by OCR, a relatively larger weight is assigned to this channel. The OCR channel is retained with a moderate weight because it provides precise anchors for alarm codes, parameter names, and interface prompts. The metadata channel is assigned a smaller weight because it mainly supports source traceability rather than semantic matching. Based on validation performance, the parameters are set to $\alpha =0.3$, $\beta =0.6$, and $\gamma =0.1$. OCR data provides precise term anchors, human descriptions enhance contextual semantics, and metadata improves source traceability. This unified representation method transforms raw images into text-based knowledge blocks suitable for vectorization and indexing, enabling multimodal fault knowledge to be directly integrated into text retrieval-augmented generation processes.

3.3. Hybrid Retrieval and Reordering Mechanism

Building on the transformed knowledge base, we further design a hybrid retrieval and reordering module to address two common challenges in industrial after-sales question answering: diverse semantic expression and high-precision terminology matching. User queries in this domain may contain fault symptoms written in colloquial language, but they may also include highly structured technical identifiers such as model names, alarm codes, parameter names, and component labels. A single retrieval strategy is therefore insufficient to balance semantic robustness and term precision. To solve this problem, the proposed framework combines dense retrieval and BM25 and then applies rank fusion and diversity-aware reordering to improve candidate quality before generation, as shown on Figure 4.

3.3.1. Dense and Sparse Retrieval

The dense retrieval module maps both user queries and knowledge blocks into a shared vector space using an embedding model, thus capturing latent semantic associations between query intent and candidate evidence [7,8,19]. This is especially useful when users describe faults through informal or non-standard expressions. However, in industrial robot after-sales scenarios, dense retrieval alone may confuse semantically related but technically distinct concepts, especially when equipment models, error codes, or parameter names are involved. To preserve exact matching for such terminology, a sparse retrieval module based on BM25 is introduced [18]. BM25 is particularly effective for symbolic expressions, encoded identifiers, and short domain-specific terms. By combining dense and sparse retrieval, the system can simultaneously support semantic generalization and precise term matching, which is essential for practical industrial question answering.

$$\mathrm{BM25}(q,d)=\sum _{t\in q}\mathrm{IDF}\left(t\right){\displaystyle \frac{f(t,d)(k_1+1)}{f(t,d)+k_1\left(1-b+b\frac{\left|d\right|}{\mathrm{avgdl}}\right)}}.$$

(2)

where $f(t,d)$ is the frequency of term t in document d, $\left|d\right|$ is the document length, and avgdl is the average document length in the corpus. Compared with RAG methods that rely solely on dense retrieval, this paper retains sensitivity to precise terms. Meanwhile, unlike traditional BM25-based retrieval systems, it enhances semantic generalization capabilities through dense retrieval, thereby achieving a better balance between term precision and semantic robustness.

3.3.2. RRF-Based Rank Fusion

After dense and sparse retrieval produce their candidate lists, the system applies Reciprocal Rank Fusion (RRF) to merge the ranking results [24]. RRF integrates the ranking positions of documents returned by different retrievers and computes a unified fusion score for each candidate. Let R denote the set of retrieval strategies and k denote the smoothing constant. The RRF score of a document is computed as

$$\mathrm{RRF}\left(d\right)=\sum _{r\in R}{\displaystyle \frac{1}{k+{\mathrm{rank}}_r\left(d\right)}}.$$

(3)

where k is typically set to 60. A higher RRF score indicates that the document ranks well across multiple retrieval strategies. Compared with directly truncating the results of a single retriever, RRF improves the overall relevance and stability of the candidate set by preserving both semantically matched evidence and exact term-matched evidence.

3.3.3. MMR-Based Context Reordering

Although RRF improves candidate relevance, the fused results may still contain duplicated or highly similar content, which reduces the efficiency of downstream generation and narrows contextual coverage [25]. To alleviate this problem, we apply Maximal Marginal Relevance (MMR) as a reordering step after rank fusion. MMR selects candidates by jointly considering query relevance and redundancy with already selected documents. Its objective can be written as

$$d^{\ast }=\arg \underset{d_i\in D\setminus S}{\max }\left[\lambda \mathrm{Rel}(d_i,q)-(1-\lambda )\underset{d_j\in S}{\max }\mathrm{Red}(d_i,d_j)\right].$$

(4)

where $\mathrm{Rel}(d_i,q)$ measures the relevance between candidate $d_i$ and query q, $\mathrm{Red}(d_i,d_j)$ measures redundancy between the candidate and the selected set S, and $\lambda$ controls the trade-off between relevance and diversity. Through this mechanism, the final context becomes more complementary and less repetitive, which improves information coverage and reduces the risk of generation bias caused by redundant evidence.

4. Results

To comprehensively evaluate the proposed method in industrial robot after-sales question answering, we conduct experiments from two perspectives: overall performance comparison and module-level ablation analysis. Evaluation metrics include accuracy, Recall@5, fidelity, and average response time. Specifically, accuracy measures semantic and factual consistency between system-generated answers and standard answers; Recall@5 assesses whether the retrieval module can identify valid evidence within the top five candidate knowledge blocks; fidelity evaluates whether generated answers remain grounded in retrieved context and avoid unsupported hallucinations; and average response time measures the end-to-end latency from user query submission to final answer generation. Together, these metrics reflect answer accuracy, retrieval sufficiency, generation reliability, and system efficiency.

4.1. Experimental Setup

4.1.1. Dataset and Evaluation Metrics

The test set construction for industrial robot after-sales scenarios adheres to the principles of representativeness, completeness, and reproducibility. Test questions are sourced from three channels: first, authentic user inquiry records extracted from corporate customer service systems over the past year to ensure realistic problem distribution; second, common issues identified from official documents such as product manuals and repair guides, covering the core semantic space of knowledge bases; and third, simulated consultation scenarios by after-sales engineers to supplement edge cases.

The final test set comprises 1500 questions categorized into three types based on task objectives: fact retrieval questions (500), fault diagnosis questions (600), and operational guidance questions (400), as detailed in

Table 3. System test set composition.

Question Type	Quantity	Proportion	Sample Question
Fact retrieval	500	33.3%	What is the rated load capacity of an F1 robot?
Fault diagnosis	600	40.0%	The system suddenly stopped during runtime with E001 displayed on the screen. How should I handle this?
Operational guidance	400	26.7%	How to calibrate the robotic arm.

. This classification framework effectively addresses key requirements in robotic after-sales question answering systems. Fact retrieval questions primarily assess the system’s ability to retrieve explicit knowledge such as equipment parameters and configuration specifications. Fault diagnosis questions evaluate the system’s comprehensive understanding of fault symptoms, error messages, and multimodal clues. Operational guidance questions specifically measure the system’s capability to organize procedural knowledge and operational steps.

The standard answer annotation process employs a dual-person independent annotation system with expert review. Two after-sales engineers independently draft responses based on official documentation, which are then verified by a third-party expert. During annotation, answers must strictly adhere to document content without subjective assumptions, encompass all critical information required for the question, and uniformly adopt standardized after-sales terminology. Each answer is limited to 50–200 words to facilitate manual evaluation.

4.1.2. Experimental Design and Baseline Methods

To validate the effectiveness of the proposed method, we compare it with five baseline systems under the same test set, generator model, prompt template, and answer-evaluation protocol. LLM-only directly generates responses from the large language model without using external knowledge, and is used to evaluate the applicability boundary of parametric knowledge in industrial after-sales scenarios. BM25-RAG retrieves evidence using BM25 and feeds the top-3 results to the generator, representing a keyword-based retrieval baseline. Dense-RAG uses vector similarity to retrieve the top-3 most relevant document blocks, representing a standard dense-retrieval RAG baseline. HyDE-RAG applies Hypothetical Document Embeddings (HyDE) to generate hypothetical documents before retrieval, and is used to evaluate the effect of query expansion [19]. Hybrid-RAG combines BM25 and dense retrieval with RRF-based fusion, and is used to compare single-retriever and hybrid-retriever settings. Dual-Channel RAG extends Hybrid-RAG by incorporating OCR-based explicit text extraction, structured semantic description, and MMR-based context reordering. The comparison is intended to clarify not only whether the proposed method improves performance, but also which engineering trade-offs are introduced by dual-channel knowledge construction and diversity-aware reordering.

The core system parameters are listed in

Table 4. Core parameter values of the dual-channel RAG system.

Module	Parameter	Value	Description
Document splitting	chunk_size	500	Maximum character count per text block
Document splitting	chunk_overlap	50	Character count between adjacent blocks
Retrieval	retriever_k	5	Initial recall count
Retrieval	mmr_lambda	0.5	MMR diversity control parameter
Retrieval	similarity_threshold	0.6	Similarity filter threshold
Generation	temperature	0.1	Controls answer randomness
Generation	max_tokens	1024	Maximum output length

. Document segmentation uses a chunk size of 500 and a chunk overlap of 50; the initial retrieval top-k is set to 5; the MMR diversity control parameter is set to 0.5; and the similarity filtering threshold is set to 0.6. The embedding model is text-embedding-3-small, while the generation model is Qwen3.5-Flash (Alibaba, Beijing, China) with a temperature of 0.1 and a maximum output length of 1024. These parameters were determined based on preliminary experiments and real-time system performance requirements.

To make the parameter selection more transparent without introducing an additional large table, the empirical tuning process is summarized here. The chunk size was tested from 300 to 800 characters, and 500 was selected because it preserved procedural completeness while avoiding excessive irrelevant context. The chunk overlap was tested from 0 to 100 characters, and 50 was selected to retain cross-boundary procedural information without creating too many duplicate chunks. The initial retrieval size was tested from 3 to 10, and top-$k=5$ was selected to balance evidence coverage and latency. The MMR parameter was tested from 0.3 to 0.7, and $\lambda =0.5$ was used to balance relevance and diversity. The similarity threshold was tested from 0.5 to 0.8 and finally set to 0.6 according to the threshold analysis in Figure 5. The temperature was set to 0.1 to reduce random generation and improve answer consistency in maintenance guidance. These values should be regarded as empirically tuned settings for the present industrial after-sales dataset rather than universally optimal constants.

All quantitative comparisons were conducted on the same fixed test set. Because the main evaluation uses deterministic retrieval settings and a fixed benchmark rather than repeated random sampling, the reported values should be interpreted as benchmark results under a controlled setting. We therefore avoid claiming statistical significance beyond the reported evaluation protocol.

4.2. Overall Results

Figure 5 presents a comparative analysis of overall performance across different methods. The proposed dual-channel RAG approach demonstrates superior performance across three key metrics: accuracy, Recall@5, and fidelity, achieving values of 87.9%, 85.2%, and 0.91, respectively, with an average response time of 2.1 s. Compared to pure LLM-based question answering methods, our approach achieves a 35.6 percentage point improvement in accuracy, highlighting the critical need for enhanced frameworks that leverage external knowledge retrieval in knowledge-intensive industrial after-sales scenarios characterized by pronounced long-tail features. This underscores the limitations of relying solely on model-parametric prior knowledge for practical applications.

Compared to BM25-RAG, our method achieves a 13.7 percentage point improvement in accuracy and a 17.4 percentage point improvement in Recall@5, demonstrating that keyword matching alone struggles to effectively process users’ colloquial expressions and semantic variants. Compared to Dense-RAG, our approach further improves accuracy by 6.4 percentage points and fidelity by 0.09, indicating that the dual-channel image knowledge transformation and reordering strategy makes practical contributions to industrial multimodal question answering.

While hybrid retrieval RAG systems have achieved an effective balance between term matching and semantic recall, they still exhibit limitations in image knowledge utilization and context redundancy control. Our proposed method enhances candidate knowledge completeness and complementarity through dual-channel knowledge representation integrating OCR and manual semantic descriptions, combined with the MMR reordering mechanism. This approach significantly improves answer generation accuracy and credibility. Overall, the proposed method demonstrates substantial performance enhancement for industrial after-sales question answering systems while maintaining acceptable response times, validating its applicability in resource-constrained industrial environments.

From an engineering perspective, the proposed method differs from representative advanced RAG variants in both retrieval unit and deployment trade-off. Hybrid-RAG combines sparse and dense retrieval over text chunks, while the proposed method uses dual-channel multimodal knowledge blocks that contain OCR evidence, structured semantic descriptions, and source metadata. HyDE-RAG improves semantic retrieval by generating hypothetical documents, but this extra generation step may introduce query-expansion noise for precise alarm codes and model identifiers. End-to-end multimodal RAG or MLLM-based systems provide stronger cross-modal modeling capacity, but they usually require higher inference cost and offer weaker evidence traceability in resource-constrained deployments. In contrast, the proposed framework introduces offline expert annotation and template maintenance, but improves multimodal evidence coverage, terminology precision, and traceability for industrial after-sales QA.

4.3. Ablation Study

To further analyze the specific contributions of each core module to system performance improvement, this study conducts ablation experiments from three aspects: image processing strategies, retrieval fusion and reordering mechanisms, and similarity threshold settings.

4.3.1. Annotation Overhead and Scalability Analysis

Since the proposed semantic description channel relies on expert-authored annotations, we further analyze the annotation overhead and scalability of the knowledge construction process. A small-scale pilot annotation experiment was conducted to estimate the practical labor cost. Specifically, 50 industrial images were randomly selected from the after-sales knowledge base. For each image, the annotation time refers to the time required to complete the structured semantic description, while the review time refers to the time required by another engineer to check terminology consistency and factual correctness. The statistical results are shown in

Table 5. Annotation overhead of structured semantic descriptions.

Item	Value	Description
Number of annotated images	50	Images sampled from the industrial after-sales knowledge base
Average annotation time per image	2.6 min	Time required to complete the four-field semantic template
Average review time per image	0.8 min	Time required for consistency checking by another engineer
Average description length	58 words	Average length of structured semantic descriptions
Images requiring later update	8.0%	Images updated after new fault cases or maintenance records appeared

.

The results show that the average annotation time for one image is 2.6 min, and the average review time is 0.8 min. Therefore, the total manual processing time for one image is approximately 3.4 min. For the pilot set containing 50 images, the total annotation and review time is about 170 min, which is acceptable for the construction of an industrial after-sales knowledge base. The average description length is 58 words, indicating that the structured semantic description is concise while still retaining key diagnostic information.

The results also indicate that the annotation cost is moderate for industrial after-sales scenarios where the knowledge base is relatively stable and fault cases are accumulated gradually. Compared with fully automatic image indexing, expert-authored semantic descriptions require additional labor; however, they provide better traceability, terminology consistency, and diagnostic reliability, which are critical for industrial maintenance tasks. In addition, only 8.0% of the sampled images required later updates, suggesting that most semantic descriptions remained stable after their initial construction. In large-scale deployment, the annotation burden can be further reduced by using semi-automatic semantic description generation followed by expert verification.

4.3.2. Effect of Dual-Channel Image Knowledge Transformation

To evaluate the contribution of the proposed image knowledge transformation mechanism, we conduct an ablation study on 30 image-related questions, including 15 fault-screenshot questions, 8 wiring-diagram questions, and 7 operation-interface questions. Four settings are compared: No Image Processing, OCR Only, Semantic Description Only, and Dual-Channel Fusion.

As listed in

Table 6. Results of ablation experiments for the image processing module.

Configuration	Fault Screenshot (%)	Wiring Diagram (%)	Operation Interface (%)	Total (%)
No image processing	46.7	50.0	42.9	46.7
OCR channel only	73.3	62.5	57.1	66.7
Semantic description only	80.0	75.0	85.7	80.0
Dual-channel fusion	86.7	75.0	85.7	83.3

, the results demonstrate that without image processing, the system’s overall accuracy for image-related tasks is only 46.7%. When using the OCR channel alone, accuracy improves to 66.7%, indicating that explicit character information plays a crucial role in identifying fault codes and interface text, though support for weak-text scenarios such as wiring diagrams remains limited. Incorporating the manual description channel further boosts accuracy to 80.0%, proving that structured semantic completion effectively compensates for OCR’s shortcomings in visual semantic representation. The dual-channel fusion configuration achieves the best result of 83.3%, representing a 36.6 percentage point improvement over image-free processing. These findings confirm that the “OCR + manual semantic description” collaborative mechanism provides a more comprehensive representation of fault knowledge in industrial images, serving as a key factor in enhancing multimodal after-sales question answering performance.

4.3.3. Effect of Retrieval Fusion

To evaluate the contribution of the RRF fusion strategy to retrieval performance, we compare system performance under three configurations: dense retrieval only, BM25 retrieval only, and RRF fusion. The results are shown in

Table 7. Validation results of RRF fusion strategies.

Retrieval Mode	Accuracy (%)
Dense retrieval	81.2
BM25 retrieval	78.5
RRF fusion	86.2

.

The system achieves an accuracy of 81.2% when using dense retrieval alone and 78.5% when using BM25 retrieval alone, while RRF fusion improves accuracy to 86.2%. These findings demonstrate that RRF effectively integrates semantic matching with precise keyword matching, thereby enhancing the overall relevance and stability of the candidate knowledge set. Compared with single-path retrieval, the RRF fusion strategy is particularly suitable for industrial after-sales question answering tasks characterized by dense terminology and diverse user expressions.

4.3.4. Effect of MMR-Based Reordering

To evaluate the effectiveness of MMR-based reordering, we further compare retrieval performance between similarity-based ranking and MMR-based ranking, as summarized in

Table 8. Comparison of ablation experiments based on retrieval strategies.

Metric	Similarity Search	MMR Retrieval	Change
Accuracy (%)	85.3	87.9	+2.6
Results repetition rate (%)	23.5	7.8	$-15.7$
Information coverage rate (%)	76.4	85.2	+8.8
Response time (s)	1.8	2.1	+0.3

.

By introducing MMR after RRF fusion, the system improves accuracy from 85.3% to 87.9%, reduces result repetition from 23.5% to 7.8%, and increases information coverage from 76.4% to 85.2%, with only a 0.3 s increase in average response time. These results indicate that RRF primarily improves the relevance of the candidate pool, whereas MMR further improves contextual diversity and complementarity. Together, these components enhance the completeness and credibility of generated answers.

4.3.5. Effect of Fusion Weights

To justify the weight setting in Equation (1), we further conduct a sensitivity analysis on different combinations of OCR, semantic description, and metadata weights. In this experiment, all other retrieval and generation parameters are kept unchanged, including the RRF fusion strategy, the MMR diversity control parameter, the similarity threshold, and the generation model settings. Only the values of $\alpha$, $\beta$, and $\gamma$ in Equation (1) are adjusted, where $\alpha$ denotes the OCR channel weight, $\beta$ denotes the structured semantic description channel weight, and $\gamma$ denotes the metadata channel weight. The tested weight combinations are shown in

Table 9. Sensitivity analysis of fusion weights.

$\mathit{\alpha }$	$\mathit{\beta }$	$\mathit{\gamma }$	Accuracy (%)	Recall@5 (%)	Fidelity
0.5	0.4	0.1	84.6	82.7	0.87
0.4	0.5	0.1	86.4	84.1	0.89
0.3	0.6	0.1	87.9	85.2	0.91
0.2	0.7	0.1	86.8	84.6	0.90
0.3	0.5	0.2	86.1	83.8	0.88

.

As shown in Table 9, the system achieves the best overall performance when $\alpha =0.3$, $\beta =0.6$, and $\gamma =0.1$, with an accuracy of 87.9%, a Recall@5 of 85.2%, and a fidelity score of 0.91. When the OCR channel weight is increased to $\alpha =0.5$, the system obtains lower accuracy and recall. This is because OCR-derived information mainly captures explicit textual cues, such as alarm codes, interface prompts, and parameter names, but it is insufficient for representing non-textual visual evidence, such as component states, wiring conditions, and abnormal phenomena in industrial images.

When the weight of the semantic description channel is increased from 0.4 to 0.6, both accuracy and Recall@5 improve steadily, indicating that structured semantic descriptions play a key role in transforming implicit visual evidence into retrievable textual knowledge. However, when $\beta$ is further increased to 0.7, the performance slightly decreases. This suggests that although semantic descriptions provide important visual semantic information, excessively weakening the OCR channel may reduce the system’s ability to precisely match fault codes, interface text, and technical terms. Therefore, OCR and semantic descriptions are complementary rather than mutually replaceable.

In addition, increasing the metadata weight from 0.1 to 0.2 does not lead to further improvement. Although metadata such as document source, image type, and fault category is useful for knowledge traceability and contextual organization, it contributes less directly to semantic matching than OCR text and structured descriptions. Assigning an overly large weight to metadata may introduce additional noise into the retrieval representation. Overall, the results demonstrate that the selected setting of $\alpha =0.3$, $\beta =0.6$, and $\gamma =0.1$ provides a balanced trade-off among precise textual matching, visual semantic representation, and source-level traceability.

4.3.6. Effect of Similarity Threshold

To analyze the impact of similarity threshold on system performance, this study further compares retrieval results under different threshold values $\tau$, as shown in Figure 6.

As the threshold value increases, the system’s overall accuracy shows an upward trend, but the proportion of no-result cases also rises significantly. When $\tau =0.5$, the system achieves sufficient recall but encounters numerous low-relevance candidates, resulting in an accuracy of only 81.2%. At $\tau =0.8$, while accuracy reaches 90.1%, the no-result rate increases to 26.7%, significantly affecting system usability. After a comprehensive evaluation of accuracy, recall adequacy, and system stability, this study ultimately adopts $\tau =0.6$ as the default setting. This configuration maintains high accuracy with only 3.3% of cases returning no results, achieving a more balanced trade-off between accuracy and usability.

4.4. Representative Case Validation

To supplement the quantitative evaluation, representative cases were analyzed to examine whether the retrieved evidence and generated answers were practically useful for after-sales engineers. To avoid relying on screenshot-style illustrations without quantitative meaning, the cases are summarized in

Table 10. Representative case validation of the proposed system.

Case Type	User Query	Key Retrieved Evidence	Observed Answer Behavior
Operational guidance	EPRobot cannot emit Wi-Fi	Operation manual entries and troubleshooting records related to hotspot configuration and terminal restart	Provides a direct procedure with reference resources instead of a vague general suggestion
Text-based fault QA	Error code 5 occurred while burning the image, prompting access denied	Fault records about image-burning failure, memory-card formatting, and tool-based reburning procedures	Identifies the error condition and organizes the solution steps according to maintenance workflow
Image-assisted diagnosis	Uploaded trajectory image indicating abnormal robot movement	Semantic description of trajectory drift and configuration-file adjustment records	Links visual trajectory abnormality to localization drift and provides configuration-level corrective steps

. The table reports the question type, required evidence, retrieved cues, and observed answer behavior. This presentation is intended to highlight the engineering insight of each case rather than to use interface screenshots as experimental evidence.

These cases show that the proposed system is effective in three representative after-sales situations: Procedural guidance, text-based fault troubleshooting, and image-assisted diagnosis. The case analysis is not used as an independent statistical proof; rather, it illustrates how OCR evidence, semantic descriptions, and source metadata jointly support traceable answer generation.

5. Discussion

The ablation results provide further insight into why the proposed system improves industrial robot after-sales question answering. From a mechanism perspective, the performance gain comes from the complementarity between evidence representation and retrieval organization.

First, the OCR-only setting improves performance over the no-image-processing baseline, but its effectiveness remains limited in scenarios where key evidence is weakly textual or entirely non-textual. This is particularly evident for wiring diagrams and component-state images, where diagnostic information often depends on spatial relations, indicator states, or abnormal visual patterns rather than explicit text. As a result, OCR alone cannot provide a sufficiently complete representation of image-based fault knowledge.

Second, the semantic-description-only setting performs better than OCR-only in several image-related cases, which suggests that structured semantic annotation is effective for capturing implicit visual evidence. However, this setting still lacks the explicit textual anchors needed for precise matching of alarm codes, parameter names, and interface prompts. In industrial after-sales question answering, such identifiers often play a decisive role in fault localization and procedural guidance. Therefore, semantic description alone is also insufficient for robust retrieval and answer generation.

Third, the retrieval ablation results show that the gains of the proposed system are not only due to multimodal knowledge transformation, but also to the organization of the retrieval and reordering pipeline [24,25]. RRF improves retrieval performance by combining the strengths of dense retrieval and BM25, which is important in this task because user queries frequently contain both colloquial fault descriptions and highly structured technical terms [6]. On top of this, MMR further improves answer quality by reducing redundancy among retrieved chunks and increasing contextual diversity. This is particularly useful in industrial diagnosis scenarios, where complete answers often depend on multiple complementary pieces of evidence rather than repeated variants of the same information. The interaction between the two channels can be understood from their different retrieval roles. OCR-derived evidence provides high-precision lexical anchors, such as alarm codes, interface prompts, parameter names, and device identifiers. These anchors are especially useful for BM25-based sparse retrieval because many industrial faults are indexed by exact symbolic expressions. In contrast, structured semantic descriptions provide high-level visual semantics, such as component states, wiring conditions, abnormal indicator behavior, and scene context. These descriptions are more suitable for dense retrieval because they allow semantically similar user expressions to be matched even when the query does not contain the exact words appearing in the original document. Based on this complementarity, RRF is used to merge the candidate lists produced by sparse and dense retrieval. It helps preserve both exact term-matched evidence from OCR and semantically matched evidence from structured descriptions. MMR is then applied to reduce redundant chunks and improve evidence coverage. Therefore, RRF mainly improves candidate relevance and retrieval stability, whereas MMR improves contextual diversity and reduces repeated evidence before generation.

Overall, the results suggest that strong performance in industrial robot after-sales question answering depends on two conditions: sufficiently complete knowledge representation for multimodal evidence, and sufficiently balanced retrieval for both semantic relevance and terminology precision. The proposed framework is effective because it addresses both conditions simultaneously.

5.1. Engineering Trade-Offs and Deployment Complexity

The proposed system also introduces engineering trade-offs. Compared with LLM-only or text-only RAG systems, the dual-channel framework requires an offline knowledge-construction stage, including OCR extraction, expert semantic annotation, metadata binding, embedding generation, and index maintenance, as listed in

Table 11. Engineering trade-offs of different retrieval pipelines.

Pipeline	Latency/Complexity	Memory and Index Cost	Deployment Implication
LLM-only	Lowest retrieval cost, but depends entirely on model inference	No external index required	Simple deployment but weak for private and frequently updated industrial knowledge
Dense-RAG	Vector search cost depends on embedding index size	Requires vector embeddings for all chunks	Good semantic recall but may confuse precise technical identifiers
Hybrid-RAG	Requires both sparse and vector retrieval	Requires BM25 index and vector index	Better balance between term precision and semantic recall
Proposed dual-channel RAG	Adds offline semantic annotation and MMR reordering	Stores OCR, semantic description, metadata, and fused embeddings	Higher construction cost, but better multimodal evidence coverage and traceability

. However, this offline cost improves evidence traceability and reduces the need for end-to-end multimodal model fine-tuning. The main complexity of the retrieval stage comes from running both sparse and dense retrievers, merging candidate lists through RRF, and applying MMR for context selection. In practice, this increases response time from 1.8 s under similarity-based ranking to 2.1 s after MMR, which remains acceptable for after-sales service scenarios where answer correctness and traceability are more important than millisecond-level response.

5.2. Scalability, Generalization, and Limitations

The current evaluation is based on one industrial robot after-sales dataset. Therefore, the results demonstrate effectiveness in the selected application scenario, but they should not be overinterpreted as proof of universal cross-domain generalization. When the framework is extended to other industrial domains, such as instrument maintenance, assembly guidance, or equipment fault diagnosis, the semantic description template may need to be redesigned according to domain-specific components, fault phenomena, and operating procedures.

From a scalability perspective, the knowledge base can be expanded incrementally: Newly added images are processed by OCR, annotated according to the structured template, embedded, and inserted into the sparse and vector indices. This makes the system suitable for gradually accumulated after-sales knowledge. However, large-scale deployment across multiple product lines may increase annotation and maintenance costs. Future work should therefore investigate semi-automatic semantic description generation, active-learning-based annotation prioritization, and automated consistency checking between semantic descriptions and official maintenance documents.

6. Conclusions

This study designed and evaluated a domain-specific dual-channel multimodal RAG system for industrial robot after-sales question answering. The framework was designed to improve the use of multimodal fault knowledge in resource-constrained industrial environments and was evaluated on a real-world after-sales dataset. Experimental results showed that the proposed method achieved strong overall performance and clear improvements on image-related question answering tasks compared with pure LLM, Dense-RAG, and hybrid retrieval baselines. These findings demonstrate the practical value of deployment-friendly and interpretable multimodal knowledge grounding for industrial after-sales service.

Despite these advantages, the current framework still has several limitations. First, its effectiveness depends on the quality of image understanding, and performance may decline when images suffer from severe blur, occlusion, or viewpoint distortion. Second, the current evaluation is mainly conducted on a specific industrial robot after-sales knowledge base, so the generalization ability of the framework across different devices, enterprises, and industrial sub-scenarios still needs further validation. Third, the use of manual semantic description improves interpretability and knowledge completeness, but also introduces additional annotation and maintenance costs. Although the annotation overhead analysis shows that the cost is acceptable for a relatively stable industrial after-sales knowledge base, scalability may become challenging when the system is extended to multiple robot models, rapidly changing product versions, or large-scale cross-enterprise deployment. Finally, the present system is based on single-round question answering and therefore remains limited in handling highly ambiguous faults or multi-component coupled failures.

Future work will focus on three directions. First, semi-automatic semantic completion will be explored to reduce the cost of manual annotation while preserving knowledge quality. Second, incremental knowledge updating and multi-source knowledge integration will be strengthened to improve the timeliness and scalability of the knowledge base. Third, multi-turn interaction and diagnostic reasoning mechanisms will be introduced to better support complex fault analysis in practical after-sales scenarios. Overall, this work provides a feasible foundation for building practical and extensible multimodal question answering systems for industrial robot after-sales service.