An Enhanced RAG–LLM Framework for Decision Support in Sustainable Industrial Engineering and Management

Abstract

Retrieval-augmented generation (RAG) with large language models (LLMs) provides a new technical pathway for integrating industrial-domain knowledge with the reasoning capabilities of LLMs and has substantial application potential in sustainable industrial decision support. However, existing RAG-based decision-support methods focus mainly on enhancing semantic relevance and remain inadequate in industrial knowledge governance, the participation of retrieved evidence throughout the decision process, and the validation of decision results and feedback before implementation, thereby limiting the accuracy, traceability, and reliability of decision outcomes. To address these problems, this study proposes a RAG–LLM framework for sustainable industrial decision support. The framework generates governed knowledge units with explicit evidence boundaries from multi-source heterogeneous industrial data and combines hybrid retrieval, evidence packaging, and validation and feedback mechanisms to enable reliable industrial-knowledge retrieval and controllable decision generation. Simulated experiments were conducted in sustainable industrial engineering and management decision scenarios covering four representative tasks: equipment-fault diagnosis, quality-anomaly analysis, energy optimization, and wastewater-compliance management. The results show that the proposed method effectively improves evidence-retrieval performance and content coverage. With the hybrid retrieval strategy, mean Recall@6 increased from 0.7917 for BM25 to 1.0000, mean nDCG@6 increased from 0.8398 to 1.0000, and mean GVR@6 decreased from 0.1667 to 0. The enhanced RAG–LLM maintained high gold-standard source coverage across different LLMs, with mean evidence source-ID coverage values of 1.0000, 0.9917, and 0.9750 for the ChatGPT 5.5, Gemini 3.5 Flash, and DeepSeek V4 Flash combinations, respectively. After feedback regulation before decision implementation was introduced, the mean evidence source-ID coverage for representative task responses increased from 0.4167 to 1.0000. The mechanism that binds source-labeled evidence throughout the decision process further improved the verifiability of generated results and the traceability of decision evidence, indicating that the enhanced RAG–LLM framework can provide reliable decision support for sustainable industrial engineering and management. Future research will further explore real-time multimodal data integration, privacy-preserving deployment, and long-term validation in real industrial environments.

1. Introduction

With the rapid development of industrial digitalization and intelligent technologies, sustainable industrial engineering and management plays an important role in enabling industrial systems to achieve the Sustainable Development Goals (SDGs). Sustainable industrial decision-making involves multiple complex tasks, including production scheduling, quality control, resource allocation, and compliance management. Its implementation depends not only on the effective use of multi-source industrial data but also on the integration of multidimensional information such as enterprise knowledge, domain expertise, and decision rules. Although modern industrial systems have accumulated extensive data resources that provide a foundation for data-driven decision-making, industrial data present intertwined challenges involving multi-source integration, dynamic operating conditions, enterprise-specific processes, and cross-organizational access and collaboration [1,2,3,4]. These data are usually distributed across equipment-control systems, quality-inspection records, maintenance logs, and enterprise management platforms, forming data silos that make unified cross-system organization, associative analysis, and knowledge utilization difficult, thereby limiting the effectiveness and reliability of complex industrial decision-making.

With the support of technologies such as the Internet of Things, digital twins, and artificial intelligence, industrial systems can already support data-driven tasks such as equipment-condition monitoring, fault diagnosis, and manufacturing-system optimization [5,6,7,8]. However, such methods usually depend on structured sensor data, mathematical models, or predefined rules, and their use of the abundant unstructured knowledge resources in industrial environments, such as enterprise documents, operating specifications, and management rules, remains limited. LLMs demonstrate significant advantages in natural-language understanding, knowledge integration, and complex reasoning, providing a new technical path for the semantic analysis and utilization of unstructured industrial knowledge [9,10,11,12]. Nevertheless, the direct application of general-purpose LLMs in industrial decision-making environments still faces key challenges. On the one hand, industrial decision-making is highly dependent on enterprise-specific data, process constraints, and safety rules [13,14], whereas general-purpose LLMs usually lack an effective understanding of internal enterprise data and constraint rules. On the other hand, in the absence of external evidence constraints, general-purpose LLMs may generate unsupported content [15], making it difficult to satisfy industrial requirements for reliability, traceability, and compliance [16]. Retrieval-augmented generation (RAG) introduces an external knowledge-retrieval mechanism [17,18], providing an effective means of improving the ability of LLMs to use knowledge in specialized domains. By combining parametric models with external knowledge sources, RAG helps improve generation quality and interpretability in knowledge-intensive tasks [15,17]. However, existing RAG methods mainly focus on introducing external knowledge and enhancing retrieval performance, tend to overlook the reliability of external industrial knowledge, and lack mechanisms for domain-rule integration, evidence verification, and feedback optimization. Therefore, existing methods still have difficulty satisfying the comprehensive requirements of complex industrial environments for decision reliability, controllability, traceability, and continuous optimization. A unified framework capable of processing heterogeneous industrial knowledge, incorporating evidence constraints, and supporting closed-loop decision optimization is urgently needed.

This paper proposes a retrieval-augmented LLM framework (RAG–LLM) for sustainable industrial engineering and management. By constructing governed knowledge units, the proposed method effectively integrates multi-source industrial data, domain knowledge, and decision constraints. It further uses hybrid retrieval to introduce relevant evidence into the model’s reasoning process, thereby improving the reliability and controllability of the decision process. On this basis, this paper designs a process-oriented mechanism for industrial decision-making that integrates problem understanding, evidence retrieval, constraint-aware reasoning, answer generation, and result validation, thereby enabling controllable evidence-based generation and traceable decision support.

The remainder of this paper is organized as follows: Section 2 reviews related research. Section 3 presents the proposed RAG–LLM framework. Section 4 validates the effectiveness of the proposed method through case experiments and analyzes the experimental results. Section 5 summarizes the main findings and limitations and presents directions for future research.

2. Literature Review

2.1. RAG Design for Manufacturing

Retrieval-augmented generation (RAG) combines external knowledge retrieval with the generative capabilities of LLMs, allowing a non-parametric knowledge base to support knowledge-intensive question-answering, and thereby alleviating knowledge staleness, factual hallucination, and insufficient domain knowledge [17,18]. Early manufacturing-oriented RAG research focused primarily on knowledge-retrieval enhancement for specific tasks. Heredia Álvaro and González Barreda applied RAG to manufacturing quality control and improved the factual consistency and domain-specificity of manufacturing question-answering by retrieving quality-management documents and historical records [19]. Related studies subsequently expanded the knowledge-representation capabilities of RAG. Bahr et al. introduced a knowledge graph into RAG and used structured entity relationships to strengthen the identification and analysis of potential failures in manufacturing systems [20]. Li et al. proposed the multimodal ManuRAG framework, which integrates multiple types of manufacturing knowledge, including text, images, formulae, and tables, to improve knowledge utilization in complex tasks [21]. In response to limitations in existing RAG studies regarding domain-knowledge constraints, standards integration, and complex knowledge reasoning, researchers have begun to explore the integration of RAG with industrial knowledge representation and domain rules [22,23]. Shi et al. combined standardized industrial-equipment knowledge representation with an enhanced retrieval strategy to optimize RAG performance, allowing the model to conduct domain reasoning using equipment entities, operating states, and their relationships [24]. Xiong et al. explicitly embedded aviation digital-model design rules into retrieval and generation, making external knowledge a domain rule that constrains design generation [25]. Liu et al. further improved manufacturing-knowledge retrieval and domain-knowledge organization by constructing a scalable industrial question-answering framework through knowledge organization and query enhancement [26]. Overall, existing studies mainly focus on retrieval optimization, semantic representation, or the execution of specific rules, while paying insufficient attention to knowledge governance for industrial decision-making. Governance attributes such as knowledge content, access permission, version and validity period, authority level, and provenance identifiers have not yet been uniformly organized into a structured knowledge representation for decision support. Consequently, data units cannot readily support knowledge retrieval, evidence organization, and constraint-aware reasoning under permission, version, and validity constraints.

2.2. Sustainable Industrial Decision Support

Early applications of RAG to sustainable industrial decision support used corporate reports, technical documents, and sustainability-assessment materials as external knowledge sources to provide information for sustainability assessment. Cole et al. applied RAG to product social life-cycle assessment, extracted information on social topics from corporate reports and stakeholder materials, and determined assessment levels according to predefined criteria, thereby providing an information basis for decisions on products’ social sustainability [27]. However, their method focused mainly on a single product case and predefined social-assessment topics and did not connect the social-assessment task with other decision stages. To address this limitation, Zhang et al. combined RAG with Text-to-SQL, code execution, and report generation to implement life-cycle inventory queries, environmental-impact calculations, and assessment-report generation, thereby extending RAG from evidence retrieval to life-cycle assessment process support [28]. RAG was subsequently extended to sustainable resource management and operational decision-making. Arslan et al. used RAG to retrieve government reports, industry publications, academic studies, and news reports to provide information support for the participation of small and medium-sized enterprises in sustainable urban water-resource management [29]. Although this approach broadened the application scope and information sources of RAG in sustainable resource management, it remained essentially a question-answering information system in which evidence was mainly used to generate advisory responses and was not connected with enterprise-equipment states, production processes, action execution, or outcome feedback. To advance RAG from information consultation to operational tasks, Chen et al. introduced RAG, tool calling, and human–AI collaboration into the operation and maintenance of building-energy systems, enabling task routing among knowledge retrieval, fault analysis, tool execution, and operational-control recommendations [30]. To overcome the limitations of traditional RAG in relationship representation and cross-source knowledge association, Wang et al. combined knowledge graphs, RAG, and dynamic knowledge-orchestration mechanisms for intelligent enterprise decision support [31]. Compared with document-only RAG, knowledge-graph-enhanced RAG preserves explicit semantic relationships among entities and supports structured retrieval and multi-hop evidence organization [32]. However, existing sustainability-oriented RAG applications mainly enhance the role of evidence in a particular decision stage or specific task, and the evidence-use process remains clearly segmented by stage. Research still lacks an RAG framework for full-chain, multi-task decision-making in sustainable industrial engineering and management, in which retrieved evidence is used throughout requirement analysis, plan analysis, decision generation, human approval, action execution, and outcome recording, thereby improving the transparency, verifiability, and end-to-end traceability of industrial decisions.

2.3. Evidence Governance and Access Control in LLM Systems

Once an LLM system introduces RAG, model generation no longer depends only on internal parameters but is also based on externally retrieved evidence. System-governance concerns therefore expand from traditional output correctness to the relevance, authenticity, and traceability of evidence. Early research mainly assessed whether LLM-generated content was effectively supported by retrieved evidence. Priola combined RAG with the NMISS evaluation mechanism to distinguish unsupported generation from context-supported content, shifting LLM evaluation from surface-text overlap to evidence consistency [33]. However, this method focuses on post-generation evaluation and cannot proactively repair insufficient evidence. To address this problem, Li et al. proposed an evidence-aware self-correction mechanism that enables an LLM to actively inspect retrieved evidence, identify missing evidence, and reorganize its reasoning [34]. For conflicts between parametric and external knowledge, Xiong et al. proposed the ICR framework, which determines how knowledge should be used through conflict identification and strategy selection [35]. To improve the traceability of evidence use, LedgerRAG extends retrieval and generation into a recordable evidence chain and organizes evidence coverage, temporal validity, authority, and conflict status [36]. In addition to evidence reliability, the use of enterprise data by LLMs involves privacy protection and access authorization. He et al. proposed LPRAG, which identifies sensitive entities before text enters the retrieval process and perturbs words, numbers, and phrases through local differential privacy [37]. This method can reduce the direct exposure of sensitive information, but perturbation may alter the original meaning of evidence and affect its accuracy. Moreover, data desensitization can only control hidden content and cannot determine access rights according to user identity. Access-control research has therefore gradually shifted from content protection to identity-driven authorization mechanisms. For example, Jeong and Lee placed identity and access-management rules before retrieval and filtered candidate knowledge according to user identity, role, and resource permissions [38]. Al Masoud et al. further determined which knowledge fragments could enter the generation context according to security policies, thereby addressing prompt-leakage attacks [39]. Although existing studies can evaluate the evidence consistency of generated content, record evidence sources, and control knowledge-access permissions, evidence governance is mainly concentrated before retrieval and during generation, and it has not yet converted evidence sufficiency, claim support, conflict status, compliance with business constraints, and decision risk into unified pre-execution validation conditions. Before an industrial decision is implemented, a feedback-regulation mechanism is still needed to simultaneously check evidence sufficiency, conclusion support, compliance with business constraints, uncertainty, and potential risks.

2.4. Research Gaps and Contributions

The preceding review shows that, although existing research in sustainable industrial engineering and management has extended RAG to manufacturing-knowledge enhancement and sustainable industrial decision support, and has begun to consider the reliability, traceability, and access security of retrieved evidence, the following limitations remain:

First, manufacturing-oriented RAG still lacks a unified structured knowledge representation. Existing studies mainly focus on retrieval performance, knowledge representation, and domain rules, but equipment records, enterprise documents, domain rules, evidence sources, access permissions, versions, and validity periods are generally managed separately by different modules. A unified knowledge structure capable of simultaneously supporting retrieval, evidence organization, and constraint-aware reasoning has not yet been formed. As a result, RAG may retrieve relevant knowledge but cannot readily ensure the consistency of knowledge from different sources during retrieval and reasoning.

Second, evidence in sustainable industrial RAG has not yet fully participated in the complete decision process. Existing applications mainly target specific industrial decision tasks, and the retrieved evidence usually supports only answer generation, assessment calculations, or decision execution at the current stage. It is not continuously associated with the task context, intermediate conclusions, validation results, human approval, decision execution, and feedback updates. Consequently, related evidence cannot readily be transferred, reused, and traced across different decision stages.

Finally, evidence validation and feedback regulation before decision execution remain insufficient. Existing studies mainly focus on the consistency, traceability, and access legitimacy of retrieved evidence, while paying insufficient attention to evidence validation before decision execution. A pre-execution validation stage that integrates evidence-sufficiency checks, business-constraint validation, risk assessment, human release, and controlled feedback has not yet been established. Mechanisms for renewed retrieval, reasoning correction, human release, and controlled feedback after validation failure are also lacking.

Table 1. Overview of related research.

Representative Study	Heterogeneous Data Integration	Evidence-Governance Attributes	Access Permission/Control	Evidence Throughout the Decision Process	Pre-Execution Validation and Feedback	Sustainable-Task Decisions
Li et al. [21]	✓
Shi et al. [24]	✓
Xiong et al. [25]	✓
Cole et al. [27]						✓
Zhang et al. [28]	✓					✓
Arslan et al. [29]	✓					✓
Chen et al. [30]						✓
LedgerRAG [36]		✓		✓
He et al. [37]			✓
Jeong and Lee [38]			✓
Al Masoud et al. [39]
This study	✓	✓	✓	✓	✓	✓

summarizes the preceding research and its differences from this study.

Based on these research gaps, the application of existing RAG–LLM systems to decision support in sustainable industrial engineering and management still faces three challenges involving knowledge governance, evidence utilization, and decision validation, as shown in Figure 1. This study addresses three research questions: (1) How can a governed knowledge-representation method be constructed for multi-source heterogeneous industrial data to enable the unified organization and effective management of industrial knowledge, evidence attributes, and decision constraints? (2) How can evidence-enhancement and constraint-aware reasoning mechanisms be designed to improve the knowledge reliability, reasoning consistency, and result traceability of RAG–LLM industrial decision support? (3) How can a decision architecture integrating retrieval, reasoning, validation, human release, and controlled feedback be constructed to enable a continuously optimized decision process with human oversight?

To fill these research gaps, this paper proposes an enhanced RAG–LLM decision-support framework for sustainable industrial engineering and management. Based on governed industrial knowledge units, the framework organizes heterogeneous industrial data, domain knowledge, evidence attributes, decision constraints, and access permissions into an associated graph structure. It further uses task dependencies to organize retrieval, evidence construction, decision generation, and result validation, thereby extending the system from individual knowledge question-answering to full-chain, multi-task decision support. At the same time, response validation, human release, and controlled feedback mechanisms embed evidence-validity checks, permission control, constrained generation, and feedback updating before decision execution to improve the reliability, controllability, and end-to-end traceability of industrial decisions. The main contributions are summarized as follows:

(1)

A RAG–LLM decision-support framework for sustainable industrial engineering and management is proposed, together with an industrial knowledge-unit representation that organizes multi-source content—including equipment records, enterprise documents, domain knowledge, decision constraints, access permissions, and version information—into a unified knowledge structure, thereby providing a basis for industrial knowledge integration and reliable retrieval.

(2)

An evidence-enhanced retrieval and constraint-aware reasoning mechanism is proposed to support full-chain evidence participation. Context-aware hybrid retrieval is used to construct evidence packages with source identifiers and governance attributes, and evidence is continuously bound to user requests, task contexts, intermediate conclusions, and decision plans. Evidence therefore participates throughout request analysis, knowledge retrieval, constraint-aware reasoning, and decision generation, improving the continuity of evidence use, decision transparency, and process traceability.

(3)

A response-validation, human-release, and controlled-feedback mechanism is constructed for industrial decision execution. Before execution, it checks evidence sufficiency, conclusion support, permission compliance, business constraints, and potential risks, and triggers evidence supplementation, renewed retrieval, reasoning correction, or human review when validation fails. The proposed framework is also evaluated in terms of knowledge retrieval, compliance with governance constraints, evidence coverage, and generation reliability through an industrial corpus, representative industrial tasks, and multi-model comparisons.

3. Enhanced RAG–LLM Framework

The enhanced RAG–LLM framework effectively connects industrial-enterprise knowledge with LLM reasoning through three core modules: industrial knowledge-unit representation, evidence generation based on hybrid retrieval, and decision validation and feedback regulation, as shown in Figure 2. First, multi-source industrial-enterprise data undergo data governance and knowledge organization to form governed knowledge units with clearly defined sources, versions, permissions, and evidence boundaries, providing a reliable knowledge basis for subsequent retrieval and reasoning. Second, a user decision task is semantically parsed to generate retrieval requirements. Relevant information is selected from the knowledge units through a hybrid retrieval mechanism and is further assembled into an evidence package containing source identifiers and evidence boundaries, supporting reliable LLM reasoning under constraints. Finally, the generated decision result undergoes automated validation, human review, and controlled feedback updating to achieve quality control of decision results and continuous optimization of knowledge and reasoning processes.

The framework-structure definitions and controlled-feedback logic underlying Figure 1 and Figure 2 are documented in Supplementary Material S16.

3.1. Industrial Knowledge-Unit Generation Module

Decision support in industrial scenarios usually involves multiple types of data, including equipment-operating data, enterprise documents, maintenance records, quality information, and environmental-management documents. Through multi-source industrial-data governance, structured knowledge representation, and provenance-tracing mechanisms, the industrial knowledge-unit generation module converts raw industrial data into knowledge units with clearly defined semantic content and governance attributes, providing a basis for subsequent constrained retrieval, evidence packaging, and decision generation.

Let the set of raw data generated during industrial-enterprise operations be

$$D=\{d_1,d_2,\dots ,d_n\},$$

(1)

where $d_i$ denotes the ith raw industrial record, including equipment alarms, sensor-state summaries, maintenance work orders, clauses in equipment manuals, quality-inspection records, energy-consumption data, and environmental-compliance documents.

Unlike knowledge-base construction in general text scenarios, industrial data generally contain structured, semi-structured, and unstructured information and exhibit clear business constraints. Consequently, each raw record contains not only its principal content but also metadata describing its business context and governance status, including timestamps, task identifiers, equipment identifiers, data sources, indicator names, values and units, status information, alarm information, access roles, validity periods, and evidence categories.

During knowledge-unit generation, raw industrial data are first governed to ensure the legitimacy, consistency, and reliability of knowledge utilization. Specifically, the system verifies access permissions according to data sources, user roles, authorization scopes, and validity periods, allowing only data that satisfy access-control requirements to enter the knowledge-construction process. On this basis, differences in data formats, units of measurement, and quality among industrial systems are addressed through standardization, including time-format unification, unit conversion, missing-field processing, anomaly detection, and consistency checks. Structured processing is then performed according to the industrial-data type. Textual data are segmented according to document hierarchy, semantic integrity, and topic boundaries to ensure that knowledge fragments retain complete business meaning. For tabular data, table headers, field relationships, and row-level identifiers are retained to prevent structural information loss. For event records and time-series data, event time windows, original indices, and contextual associations are retained to support subsequent evidence localization and provenance tracing. After this processing, the system further extracts knowledge content and its equipment, task, time, source, and other contextual information to provide standardized input for constructing governed knowledge units.

To uniformly organize and efficiently utilize industrial knowledge from different sources, governed industrial data are converted into a governed knowledge unit $u_i$ as follows:

$$u_i=F\left(d_i\right),$$

(2)

where $F(·)$ is the knowledge-unit construction function, whose principal role is to extract business-meaningful knowledge content from raw industrial records and integrate the corresponding governance attributes. Knowledge-unit construction includes the following steps:

Step 1:

Extract the principal knowledge content $x_i$ from the raw record to describe core information such as equipment states, fault phenomena, process parameters, management rules, and historical experience.

Step 2:

Extract industrial–business association attributes, including the task identifier ${\tau }_i$, equipment identifier $z_i$, source type $c_i$, and record time $h_i$.

Step 3:

Construct knowledge-governance attributes, including the access role $p_i$, validity interval $v_i=[v_i^{-},v_i^{+}]$, and authority level $a_i$. Here, $p_i$ represents knowledge-use permission, $v_i$ describes the applicable time range of the knowledge, and $a_i$ measures the reliability of its source.

Step 4:

Construct the knowledge-relationship attribute $r_i$ to describe associations among knowledge units, such as the hierarchical relationship among equipment, components, fault phenomena, and maintenance measures, thereby supporting subsequent relationship-enhanced retrieval.

Step 5:

Introduce the sustainability tag $s_i$ to identify the sustainable-development attributes of the knowledge unit, including energy consumption, resource utilization, and environmental impact, allowing the industrial decision process to be further associated with sustainability objectives.

Step 6:

Associate the knowledge-unit identifier $\mathrm{source}\_{\mathrm{id}}_i$ with the raw-data identifier $\mathrm{raw}\_{\mathrm{id}}_i$ to ensure knowledge provenance traceability.

The governed knowledge unit is ultimately formalized as

$$u_i=\langle x_i,{\tau }_i,z_i,c_i,h_i,p_i,v_i,a_i,r_i,s_i,\mathrm{source}\_{\mathrm{id}}_i,\mathrm{raw}\_{\mathrm{id}}_i\rangle .$$

(3)

This representation extends traditional RAG, which focuses only on textual semantic-vector representations, by jointly modeling industrial-knowledge content with its source, permissions, temporal validity, business relationships, and sustainability attributes, thereby providing a structured basis for reliable knowledge retrieval and constraint-aware reasoning in complex industrial environments.

3.2. Hybrid-Retrieval-Based Evidence-Packaging Module

In industrial decision scenarios, user requests are generally expressed in natural language, whereas industrial knowledge units contain multidimensional information such as textual content, business attributes, permission information, and temporal constraints. The hybrid-retrieval-based evidence-packaging module follows a process of task-aware query analysis, eligibility-constraint filtering, multidimensional relevance ranking, and evidence-package assembly. It selects reliable evidence from governed knowledge units that closely matches the current decision task and satisfies governance requirements, thereby supporting subsequent constrained generation and decision validation. The process consists of the following steps:

Step 1: Decision-request task analysis. After an enterprise user submits a decision request q, the parsing function $\Psi (·)$ performs field normalization, entity recognition, and output-requirement mapping. It extracts key fields, including task type (task), associated equipment (asset), query time (time), user role (role), and expected output (output). The parsed structured query is

$$q^{\prime }=\Psi \left(q\right)=\{\mathrm{task},\mathrm{asset},\mathrm{time},\mathrm{role},\mathrm{output}\}.$$

(4)

Unlike general question-answering tasks, industrial decisions generally need to satisfy explicit output constraints. This study therefore extends the output requirements into a decision-output contract containing the diagnosis, supporting evidence, implementation measures, risk warnings, an uncertainty statement, sustainability impacts, and human-review requirements.

For example, for the decision request “Equipment P-17 has repeatedly produced vibration and bearing-temperature alarms. Diagnose the cause of the fault and provide the required measures, supporting evidence, safety risks, and sustainability impacts”, the task-analysis result can be represented as

$$\begin{array}{c}{q}^{\prime }=\{\mathrm{task}=\mathrm{equipment}\text{-}\mathrm{fault}\mathrm{diagnosis},\mathrm{asset}=\mathrm{P}\text{-}17,\\ \mathrm{time}=2026\text{-}05\text{-}21,\mathrm{role}=\mathrm{maintenance}\mathrm{engineer},\\ \mathrm{output}=[\mathrm{fault}\mathrm{diagnosis},\mathrm{evidence},\mathrm{measures},\mathrm{risks},\mathrm{uncertainty},\\ \mathrm{sustainability}\mathrm{impacts},\mathrm{human}\mathrm{review}\left]\right\}.\end{array}$$

Step 2: Evidence-eligibility gating based on permissions and temporal validity. After query analysis, eligibility gating is required according to knowledge-unit access permissions and validity periods to prevent unauthorized or invalid knowledge from entering the reasoning process. Let B denote the set of roles permitted to access the current query, and let $I[·]$ be an indicator function that equals 1 when its condition holds and 0 otherwise. For knowledge unit $u_i$, the eligibility function $g_i\left(q\right)$ is

$$g_i\left(q\right)=I[p_i\cap B\ne ⌀]·I[v_i^{-}\le t_q\le v_i^{+}],$$

(5)

where $p_i$ is the knowledge-unit access role, $v_i=[v_i^{-},v_i^{+}]$ is the knowledge-validity interval, and $t_q$ is the query time. When $g_i\left(q\right)=1$, the knowledge unit satisfies the permission and temporal-validity constraints and can enter the subsequent relevance-ranking stage; otherwise, it is filtered out. Eligibility gating prevents unauthorized, expired, and inapplicable knowledge from entering the model-reasoning process.

Step 3: Multidimensional relevance ranking based on hybrid retrieval. Traditional BM25 calculates relevance mainly from lexical matching between query text and knowledge content, and it cannot fully express task constraints, equipment associations, or knowledge-trustworthiness attributes in industrial decision scenarios. This study therefore develops a hybrid retrieval algorithm based on BM25 and introduces task matching, equipment associations, authority levels, and knowledge relationships to construct the hybrid retrieval scoring function $S_i\left(q\right)$:

$$\begin{array}{c}{S}_i\left(q\right)=g_i\left(q\right)[\mathrm{BM}25(q^{\prime },x_i)+2I({\tau }_i={\tau }_q)+I\left(z_i\in \{z_q,z_{q\text{-}\mathrm{series}}\}\right)\\ +0.12a_i+0.35|r_i\cap Z_3\left(q\right)|].\end{array}$$

(6)

Here, $\mathrm{BM}25(q^{\prime },x_i)$ denotes the lexical match between the user query and the content of a knowledge unit. The terms ${\tau }_i$ and ${\tau }_q$ denote the task types corresponding to the knowledge unit and the query request, respectively; a task-match reward is assigned when they are identical, to strengthen consistency between the retrieval results and the current decision objective. The terms $z_i$ and $z_q$ denote the equipment associated with the knowledge unit and the target equipment in the query, respectively. The set $z_{q\text{-}\mathrm{series}}$ contains associated equipment in the same equipment series as the queried equipment and captures equipment relationships in the industrial system. The term $a_i$ is the knowledge-unit authority level and measures the reliability of knowledge from different sources. The set $r_i$ contains the relationships associated with $u_i$ and describes structural associations among industrial entities. The seed set $Z_3\left(q\right)$ consists of the three highly relevant knowledge units obtained from the initial retrieval results, and $|r_i\cap Z_3\left(q\right)|$ represents the relationship match between the current knowledge unit and the highly relevant seed knowledge units.

Through this scoring mechanism, retrieval is extended from a single text-similarity-based matching approach into a multidimensional evidence-selection process that integrates semantic content, task objectives, equipment relationships, and knowledge-governance attributes, thereby improving the match between retrieval results and industrial decision tasks.

Step 4: Score-ranked evidence packaging. After the composite ranking in Step 3, the highest-ranked knowledge units are selected as candidate evidence for the current decision task. The final retrieval result is

$$E_k=TopK\left(\{u_i,S_i\left(q\right)\}\right),$$

(7)

where $E_k$ denotes the set of k candidate knowledge units with the highest composite scores, and $TopK(·)$ returns the first k knowledge units in the ranked results. The selected knowledge units are then deduplicated while necessary knowledge-conflict information is retained. Source identifiers, raw-data identifiers, version information, temporal-validity intervals, and other attributes are appended to form the ordered evidence package $E=\{E_1,E_2,\dots ,E_K\}$.

The system also retains relationships and potential conflicts among evidence units and explicitly records evidence gaps. When a key decision field lacks sufficient supporting material, the system restricts unsupported inference and passes the evidence-insufficiency condition to the subsequent validation stage, thereby improving the reliability, interpretability, and traceability of the industrial intelligent-decision process.

3.3. Decision-Validation and Feedback-Regulation Module

Although retrieval augmentation can provide external knowledge support for an LLM, the generated results may still contain insufficient evidence, contextual mismatch, omitted rules, and inadequate expressions of uncertainty. The decision-validation and feedback-loop module automatically validates the generated decision-support results, subjects them to human review, and updates the system through controlled feedback to ensure the reliability, interpretability, and traceability of the industrial decision process.

During decision validation, the system first checks whether the evidence cited by a generated result comes from the packaged evidence set E, determines whether key claims are supported by the corresponding evidence, and verifies the consistency of the equipment object, temporal context, and applicable conditions. The system further checks whether risk warnings, uncertainty statements, and mandatory safety and compliance rules are fully expressed. A generated recommendation is treated as a decision-support object awaiting validation rather than as a final industrial decision that can be executed directly. Its executability must satisfy explicit release conditions, including whether the recommendation describes specific implementation measures, specifies the applicable permissions and implementation conditions, identifies potential risks, and triggers human escalation when necessary. When validation fails, the system generates a traceable disposition instead of concealing an internal scoring process.

Table 2. Validation signals and disposition mechanisms before human release.

Validation Signal	Detection Method	System Disposition	Release Condition
Missing or malformed metadata	Check whether required source and context fields are complete	Quarantine the abnormal knowledge unit and notify the data owner	The corrected version passes validation
Unauthorized or expired evidence	Permission or validity gate is not passed	Exclude before ranking and record the source and reason	Valid authorization is obtained or the valid version is updated
Insufficient evidence	A key decision field lacks eligible supporting evidence	Rerun retrieval or escalate to human handling	Required evidence is supplemented, or the responsible person accepts the limitation
Evidence–claim inconsistency	A material claim lacks a supporting source or conflicts with source content	Perform one constrained repair and stop automated processing if it fails	Every key claim is supported by evidence or explicitly marked as a hypothesis
Omitted mandatory rule or risk	A safety, compliance, or high-impact condition check is missing	Stop automated release and submit to the responsible reviewer	The rule is supplemented and approval is obtained
Conflict among evidence sources	Eligible and authoritative records support mutually incompatible actions	Retain conflict information and block automated release	A reviewer resolves the conflict or the control rule is updated

summarizes the principal validation signals and corresponding dispositions before human release.

Based on this validation mechanism, a structured feedback loop is further constructed through the following steps:

Step 1:

Generate a Decision-Support Draft: Based on the retrieved evidence package E, the system generates a decision-support draft containing the recommended actions, corresponding evidentiary basis, applicable conditions, potential risks, and uncertainty statements. The draft is only an auxiliary decision object and can enter subsequent industrial execution only after automated validation and human review; it does not directly constitute the final decision.

Step 2:

Conduct Validation and Human Review: The decision-support draft is automatically checked according to the validation rules in Table 2, including evidence sufficiency, evidence–claim consistency, source validity, knowledge authority, retrieval noise, and safety and compliance requirements. After validation, the responsible reviewer approves, revises, rejects, or escalates the decision-support object according to the validation results. All validation-failure records and human dispositions are linked through a unified audit identifier to form a traceable feedback record.

Step 3:

Form Structured Feedback Signals: The system converts observations generated through validation and human review into structured feedback signals rather than directly using unconstrained natural language for model updating. For example, the absence of a current valid procedural document is recorded as an evidence gap, while an insufficiently relevant but highly ranked record is marked as a retrieval error. Structured feedback explicitly identifies the feedback type, affected object, and adjustment scope, thereby improving the controllability of subsequent optimization.

Step 4:

Review and Adjust Operating Parameters: After review, structured feedback signals are converted into controlled operating parameters or proposals for knowledge-governance adjustment. Adjustable items include retrieval weights, relevance thresholds, top-k parameters, and source-authority and validity weights. New or corrected evidence remains subject to permission verification, provenance tracing, version control, authority assessment, and validity checking. Feedback involving safety risks, compliance requirements, equipment shutdowns, or major process adjustments may be handled only by the responsible reviewer. This feedback mechanism updates only controlled operating parameters and knowledge-governance states and does not directly modify the parameters of the foundation LLM, thereby ensuring the stability of model capability and the decision process.

Step 5:

Optimize the Feedback-Driven Decision Process: Parameter adjustments and knowledge-governance updates approved after review are applied to subsequent decision cycles to optimize future evidence retrieval, evidence selection, response generation, and validation. The updated system remains subject to permission gating, validity checks, human review, and audit requirements. New decision-support drafts again undergo validation, human review, and feedback recording, thereby forming a closed loop of “decision-support draft generation–validation and human review–structured feedback–operating-parameter adjustment–subsequent decision cycle.”

3.4. Computational Process of the Enhanced RAG–LLM Framework

For a specific industrial decision task, the proposed RAG–LLM framework first governs industrial source data that have passed permission verification and generates knowledge units with business and governance attributes. After a user submits a decision request, the task-analysis module converts the natural-language request into a structured query containing the task type, associated equipment, query time, access role, and output requirements. Candidate knowledge units are then filtered by permission and validity-period constraints. Only knowledge units satisfying the governance requirements enter the subsequent hybrid-retrieval ranking process. The highest-ranked units are selected according to their composite relevance scores and further packaged into a structured evidence package containing source identifiers and evidence attributes.

The LLM generates a decision-support result and its rationale within the evidence boundary defined by the evidence package. The automated validation module then checks the generated result in terms of evidence consistency, source validity, contextual matching, completion of required fields, and safety and compliance constraints. For repairable problems, the system performs one constrained repair. For insufficient evidence, rule conflicts, or high-risk decisions, it stops automated processing and transfers the result to the responsible reviewer. Only decision-support results approved through human review can enter subsequent application stages.

At the same time, the system stores the validation process, records of human modifications, and confirmed operational feedback in a structured form. After review, these records are used to optimize subsequent retrieval parameters, knowledge metadata, and source states. The overall computational process is shown in Figure 3.

4. Experiments

To validate the effectiveness of the proposed enhanced RAG–LLM framework for decision support in sustainable industrial engineering and management, this study constructs representative industrial application scenarios for case experiments. Specifically, raw industrial data are first constructed according to representative industrial-business processes, and the decision-support process is completed according to the sequence of industrial knowledge-unit generation, hybrid-retrieval-based evidence packaging, and decision validation and feedback regulation. Different retrieval strategies are then combined with LLMs to form comparison experiments in which the knowledge-retrieval capability, evidentiary-support capability, and decision-generation quality of each method are compared. Ablation experiments are further used to analyze the contributions of key modules in the proposed framework to industrial decision reliability, while parameter-sensitivity analysis examines the effect of evidence-package size (Top-k) on model performance.

4.1. Experimental Data

This study constructs an experimental dataset based on simulated industrial-enterprise scenarios. Considering typical data-driven decision requirements in sustainable industrial engineering and management, four representative industrial decision scenarios are designed around four key decision dimensions in industrial-system operations: equipment reliability, product-quality control, resource-use efficiency, and environmental-compliance management. These scenarios are equipment-fault diagnosis, quality-anomaly analysis, energy optimization, and wastewater-compliance management.

The four scenarios cover representative decision requirements in industrial production and are closely associated with the objectives of sustainable industrial engineering and management. Equipment-fault diagnosis simulates the identification of abnormal equipment states and maintenance decision-making, supporting the timely detection of potential faults and the development of controlled maintenance plans to reduce unplanned downtime, secondary equipment damage, and the waste of spare parts and lubricants. Quality-anomaly analysis simulates product-defect identification, process-cause tracing, and batch disposition to reduce raw-material losses, reprocessing energy consumption, and the risk of nonconforming-product circulation. Energy optimization simulates the analysis of abnormal production-energy consumption and operational scheduling and identifies peak loads, idle equipment, and production-schedule conflicts to provide a basis for reducing energy consumption and indirect carbon emissions. Wastewater-compliance management simulates the identification of environmental-monitoring anomalies and compliance disposition and connects monitoring alarms, test records, and discharge rules to reduce the risk of noncompliant discharge.

The experimental data consist of multi-source heterogeneous industrial data generated during simulated industrial-enterprise operations, including equipment-operation records, quality-inspection data, energy-consumption data, and environmental-management documents. The integration of structured data, event records, and textual rule information simulates cross-data-type and cross-business-domain decision-support requirements in real industrial environments. Raw industrial data are organized in a flattened one-record-per-row structure. Each record contains fields such as the task identifier, raw-record identifier, equipment or object information, temporal information, data source, key indicator, status description, and raw text.

Table 3. Examples of raw industrial data.

Task/Raw Record	Equipment/Object and Time	Data Source	Raw Indicator Value	Status/Context	Raw Information
T1/Equipment-fault diagnosis/RAW-001	P-17/2026-05-21/09:15:00	Synthetic alarm log (synthetic_alarm_log/alarm_log)	Peak vibration/8.6 mm/s	Alarm/three alarms within 25 min	Press P-17 triggered three vibration alarms within 25 min; peak vibration reached 8.6 mm/s, exceeding the internal warning threshold.
T2/Quality-anomaly analysis/RAW-007	Q-042/2026-05-21/10:30:00	Synthetic inspection report (synthetic_inspection_report/inspection_report)	Pinhole-defect rate/6.8%	Out of control/normal value below 1.5%	Batch Q-042 had a pinhole-defect rate of 6.8%, whereas the normal range was below 1.5%, and its surface roughness exceeded the control limit.
T3/Energy optimization/RAW-013	Line-C/2026-05-21/14:00:00	Synthetic energy record (synthetic_energy_record/energy_record)	Compressed-air peak deviation/18%	Peak/13:00–15:00	Compressed-air demand on Line C peaked from 13:00 to 15:00, 18% higher than the normal daily peak.
T4/Wastewater-compliance management/RAW-019	WW-2/2026-05-21/10:25:00	Synthetic alarm log (synthetic_alarm_log/alarm_log)	Conductivity-alarm count/2	Alarm/10:10–10:40 after cleaning	Wastewater line WW-2 triggered two conductivity alarms between 10:10 and 10:40 after the cleaning cycle ended.

gives examples of raw industrial data in the different decision scenarios.

Following the knowledge-unit construction method in the enhanced RAG–LLM framework, raw logs, inspection reports, equipment-state information, management rules, operating procedures, and other data are further converted into governed knowledge units with source IDs, permissions, validity periods, authority levels, and relationship information. In total, 32 governed knowledge units are generated from the raw records for subsequent retrieval-augmented generation and decision-validation experiments. The complete simulated industrial records and their data-generation algorithm are provided in Supplementary Material S0, and the governed knowledge units are provided in Supplementary Material S1.

4.2. Experimental Description

Following the operating process of the enhanced RAG–LLM framework, the experiment first constructs corresponding user decision requests for the four types of industrial decision scenarios. The system extracts the task type, associated equipment, query time, access role, output requirements, and other information from each user decision request to form a structured task query. Evidence-retrieval experiments are then conducted using the governed knowledge-unit repository. Candidate knowledge units are first filtered according to permission and validity-period constraints. Eligible knowledge units are further ranked by the hybrid retrieval strategy, and the highest-ranked knowledge units form a source-labeled evidence package. Finally, the user decision request and evidence package are jointly provided to the LLM, which generates a structured decision-support result within the defined evidence boundary. The industrial-user decision requests are as follows:

(1)

Equipment-Fault Diagnosis: Equipment P-17 repeatedly exhibits vibration and bearing-temperature alarms after maintenance. How should the fault cause be diagnosed, and how should the required disposition measures, supporting evidence, safety risks, and sustainability impacts be determined?

(2)

Quality-Anomaly Analysis: Batch Q-042 exhibits abnormal pinholes and surface roughness. What are the possible process causes, how should the batch be handled, and what supporting evidence and rework or scrap impacts should be stated?

(3)

Energy Optimization: Line-C exhibits a compressed-air energy-use peak. How should the cause of the peak be explained, and what operational adjustment plan can account for energy consumption and emissions while maintaining production and safety constraints?

(4)

Wastewater-Compliance Management: Wastewater line WW-2 exhibits conductivity alarms after cleaning. How should the cause be determined, the mandatory control measures established, and the supporting evidence and water-environmental or compliance impacts described?

To ensure decision completeness, each industrial decision output contains six fields: (1) the diagnostic or decision conclusion, (2) cited evidence or explicit assumptions, (3) recommended actions, (4) risks and uncertainty, (5) sustainability-related effects, and (6) human-review trigger conditions. The current query roles, gold-standard evidence IDs, and complete task specifications are provided in Supplementary Material S2.

4.3. Evaluation Metrics

To evaluate the effectiveness of the enhanced RAG–LLM framework in industrial knowledge-retrieval tasks, metrics are selected from three aspects: retrieval coverage, ranking quality, and knowledge-governance reliability. Recall@k measures the coverage of task-related gold-standard evidence among the top-k retrieval results; Precision@k evaluates the proportion of returned results that constitute relevant gold-standard evidence; normalized discounted cumulative gain (nDCG@k) measures the position quality of relevant evidence in the ranked retrieval list; mean reciprocal rank (MRR) evaluates how promptly the first gold-standard item of evidence appears; and the governance-violation rate (GVR@k) measures the proportion of knowledge units among the top-k results that violate permission or validity-period constraints. The metrics are defined as follows:

$$\mathrm{Recall}@k={\displaystyle \frac{|G_t\cap R_{t,k}|}{\left|G_t\right|}},$$

(8)

$$\mathrm{Precision}@k={\displaystyle \frac{|G_t\cap R_{t,k}|}{k}},$$

(9)

$$\mathrm{nDCG}@k={\displaystyle \frac{\mathrm{DCG}@k}{\mathrm{IDCG}@k}},$$

(10)

$$\mathrm{MRR}={\displaystyle \frac{1}{\left|T\right|}}\sum _{t\in T}{\displaystyle \frac{1}{{\mathrm{rank}}_t}},$$

(11)

$$\mathrm{GVR}@k={\displaystyle \frac{N_{\mathrm{expired}}+N_{\mathrm{unauthorized}}}{k}}.$$

(12)

Here, $G_t$ is the set of gold-standard evidence for task t, and $R_{t,k}$ is the set of the top-k retrieval results returned by the system. The terms DCG@k and IDCG@k are the cumulative gains obtained by relevant evidence under the actual and ideal retrieval rankings, respectively; ${\mathrm{rank}}_t$ is the ranked position of the first item of gold-standard evidence for task t; and $N_{\mathrm{expired}}$ and $N_{\mathrm{unauthorized}}$ are the numbers of expired and unauthorized knowledge units, respectively, in the top-k results.

4.4. Experimental Results and Analysis

4.4.1. Comparison Experiments

To validate the effectiveness of the enhanced RAG–LLM framework in industrial knowledge-retrieval and decision-support tasks, comparison experiments are conducted using combinations of different retrieval strategies and LLMs. All experiments use the same industrial data, task requests, output constraints, and retrieval parameters to ensure that performance differences arise primarily from the retrieval and knowledge-organization mechanisms.

As shown in

Table 4. Combinations of retrieval frameworks and LLMs.

Comparison Model	Combination with an LLM
General BM25–LLM	General BM25–ChatGPT
	General BM25–Gemini
	General BM25–DeepSeek
	General BM25–Qwen
Metadata-gated BM25–LLM	Metadata-gated BM25–ChatGPT
	Metadata-gated BM25–Gemini
	Metadata-gated BM25–DeepSeek
	Metadata-gated BM25–Qwen
General RAG–LLM	General RAG–ChatGPT
	General RAG–Gemini
	General RAG–DeepSeek
	General RAG–Qwen
Enhanced RAG–LLM	Enhanced RAG–ChatGPT
	Enhanced RAG–Gemini
	Enhanced RAG–DeepSeek
	Enhanced RAG–Qwen

, the comparison frameworks include general BM25–LLM, metadata-gated BM25–LLM, general RAG–LLM, and the enhanced RAG–LLM proposed in this study. As a classic sparse-retrieval method, BM25 calculates relevance between a query and candidate text through term-frequency statistics and document-length normalization and is a widely used baseline in information retrieval [40]. Metadata-gated BM25 is a comparison method constructed in this study to analyze the role of governance constraints, such as permissions and validity periods, during retrieval. It first filters candidate knowledge units according to the current query role and knowledge-access policy, and then it applies BM25 for ranking. Its implementation logic is provided in Supplementary Materials S2 and S18. This design is consistent with the idea of pre-retrieval identity authentication and resource-permission filtering [38,39]. General RAG follows the retrieval-augmented generation framework proposed by Lewis et al. [17]; it retrieves external knowledge and supplies relevant information to the generation model to improve the ability of an LLM to answer knowledge-intensive questions. The three comparison retrieval methods represent pure lexical matching, governance-eligibility filtering, and a general retrieval-augmented generation process, respectively. Together with the governance-enhanced hybrid retrieval proposed in this study, they form a progressive comparison for analyzing the effects of different knowledge-organization and retrieval mechanisms on industrial-evidence acquisition. Specific implementations, experimental algorithms, and configuration assets are provided in Supplementary Materials S3 and S19.

The LLMs selected for the combined experiments are ChatGPT 5.5, Gemini 3.5 Flash, DeepSeek V4 flash, and Qwen3.7, allowing the adaptability of the retrieval-augmented framework across different LLMs to be analyzed. The complete experimental configuration is provided in Supplementary Material S11.

The experiment contains four industrial decision tasks, four retrieval frameworks, four LLMs, and five repeated runs, generating 320 decision responses. It records gold-standard evidence coverage in the evidence packages, source IDs cited in the answers, non-gold-standard or unknown sources, and governance violations. Complete experimental parameter configurations, response records, and run logs are provided in Supplementary Materials S10, S11, S13 and S26–S32.

The evidence-acquisition capabilities of the different retrieval frameworks are evaluated first. This process selects evidence from the governed knowledge-unit repository and forms evidence packages for LLM reasoning. Its objective is to satisfy permission and validity constraints in industrial scenarios while ensuring evidence relevance.

Table 5. Retrieval results for different evidence-acquisition configurations at $k=6$ (mean ± sample standard deviation across four tasks).

Evidence-Acquisition Configuration	Recall@6	Precision@6	nDCG@6	MRR	GVR@6
BM25	$0.7917\pm 0.0833$	$0.7917\pm 0.0833$	$0.8398\pm 0.0570$	$1.0000\pm 0.0000$	$0.1667\pm 0.0000$
Metadata-gated BM25	$0.9167\pm 0.0962$	$0.9167\pm 0.0962$	$0.9438\pm 0.0650$	$1.0000\pm 0.0000$	$0.0000\pm 0.0000$
General RAG	$0.6667\pm 0.1361$	$0.6667\pm 0.1361$	$0.7501\pm 0.1127$	$1.0000\pm 0.0000$	$0.2500\pm 0.0962$
Governance-enhanced hybrid retrieval	$1.0000\pm 0.0000$	$1.0000\pm 0.0000$	$1.0000\pm 0.0000$	$1.0000\pm 0.0000$	$0.0000\pm 0.0000$

presents the comprehensive retrieval performance of different evidence-acquisition configurations across the four tasks, and Figure 4 further compares their retrieval performance and governance-violation rates.

The retrieval-result processing algorithms supporting the comparative analysis in Figure 4 are provided in Supplementary Material S5.

General BM25 achieved mean Recall@6 and Precision@6 values of 0.7917 and an nDCG@6 of 0.8398, while exhibiting a governance-violation rate of 0.1667. This indicates that traditional lexical retrieval can identify some relevant knowledge but cannot prevent expired or unauthorized knowledge from entering the retrieval results. After permission, validity-period, and status filtering were added to BM25, metadata-gated BM25 increased mean Recall@6 and Precision@6 to 0.9167 and reduced GVR@6 to zero. This result shows that pre-retrieval governance-eligibility filtering can effectively reduce knowledge units that do not satisfy the constraints and improve evidence reliability. General RAG used character-level TF–IDF retrieval and obtained mean Recall@6 and Precision@6 values of 0.6667, an nDCG@6 of 0.7501, and a GVR@6 of 0.2500. These results show that retrieval based only on general textual relevance cannot simultaneously satisfy the requirements for evidence completeness and governance compliance in industrial decision scenarios. In comparison, the governance-enhanced hybrid retrieval used by the enhanced RAG–LLM further integrates task matching, equipment associations, knowledge authority, temporal validity, and relationship information to achieve constraint-aware evidence selection for industrial tasks. Across the four tasks, its Recall@6, Precision@6, and nDCG@6 values all reached 1.0000, while GVR@6 decreased to zero. The MRR of all four retrieval configurations reached 1.0000, indicating that the first key item of evidence could be located rapidly in every configuration.

Overall, the experimental results show that industrial knowledge retrieval cannot rely only on textual similarity and must simultaneously consider knowledge content, business associations, and governance constraints. The eligibility-gating mechanism mainly addresses whether knowledge satisfies the conditions of use, while the hybrid ranking mechanism further addresses whether eligible knowledge can accurately enter the evidence package. Task-level retrieval results and source information are provided in Supplementary Materials S23–S25. The basic retrieval algorithm is provided in Supplementary Material S3; task-level outputs, summaries, and error counts are provided in Supplementary Materials S6, S7, and S9; task-level results and summaries for the four frameworks are provided in Supplementary Materials S23–S24; and the audited data-processing algorithms supporting the manuscript tables and figures are provided in Supplementary Material S34.

To further analyze the stability of the retrieval frameworks across different industrial decision scenarios,

Table 6. Recall@6 and governance-violation counts for each task (recall/count).

Task	BM25	Metadata-Gated BM25	General RAG	Governance-Enhanced Hybrid Retrieval
Equipment-fault diagnosis	0.8333/1	0.8333/0	0.6667/1	1.0000/0
Quality-anomaly analysis	0.8333/1	1.0000/0	0.6667/2	1.0000/0
Energy optimization	0.6667/1	0.8333/0	0.8333/1	1.0000/0
Wastewater-compliance management	0.8333/1	1.0000/0	0.5000/2	1.0000/0

presents Recall@6 and the number of governance violations for the four tasks. General BM25 exhibits some evidence omission in every task and introduces one governance-violating knowledge unit in each task. Metadata-gated BM25 eliminates governance violations and achieves complete recall for the quality-anomaly and wastewater-compliance tasks, but it still omits some key evidence for the equipment-fault and energy-optimization tasks. General RAG exhibits greater performance fluctuations across tasks. Its Recall@6 is only 0.5000 for the wastewater-compliance task, in which two governance-violating knowledge units are also retrieved, indicating that a general retrieval-augmented method cannot readily process industrial decision tasks subject to strict permission and rule constraints. In comparison, the enhanced RAG–LLM achieves Recall@6 $=1.0000$ and zero governance violations across equipment-fault diagnosis, quality-anomaly analysis, energy optimization, and wastewater-compliance management, indicating that it can stably obtain the key evidence required to support industrial decisions. The complete text and source IDs of the top-six evidence items actually provided to the LLM by each framework in the four tasks are presented in Supplementary Material S25.

To further evaluate the end-to-end performance of the enhanced RAG–LLM framework in industrial decision support, the generated results are analyzed for source consistency and structural completeness on the basis of the retrieval results. Specifically, the analysis calculates gold-standard evidence coverage in the evidence packages, actual gold-standard source-ID coverage in the answers, and six-field output coverage. It further records whether an answer cites a non-gold-standard source ID within the evidence package, generates an unknown ID outside the corpus, or contains a human-review prompt. Each retrieval-framework–large-language-model combination contains four industrial tasks with five repeated runs per task, yielding 320 end-to-end decision responses. The item-level deterministic audit procedure and run-level results are provided in Supplementary Material S29, and the summary results for the 16 retrieval-framework–large-language-model combinations are provided in Supplementary Material S30.

Table 7. Evidence coverage and answer source-ID coverage for different retrieval–LLM combinations.

Retrieval Framework	LLM	Gold-Standard Evidence-Package Coverage	Answer Source-ID Coverage	Six-Field Output Coverage	Human-Review Marker Rate
BM25	ChatGPT	$0.7917\pm 0.0740$	$0.7917\pm 0.0740$	$1.0000\pm 0.0000$	1.0000
	Gemini		$0.7833\pm 0.0784$	$1.0000\pm 0.0000$	1.0000
	DeepSeek		$0.7667\pm 0.1368$	$1.0000\pm 0.0000$	1.0000
	Qwen		$0.7083\pm 0.1783$	$0.9417\pm 0.0816$	0.7000
Metadata-gated BM25	ChatGPT	$0.9167\pm 0.0855$	$0.9167\pm 0.0855$	$1.0000\pm 0.0000$	1.0000
	Gemini		$0.9167\pm 0.0855$	$1.0000\pm 0.0000$	1.0000
	DeepSeek		$0.8917\pm 0.0816$	$1.0000\pm 0.0000$	1.0000
	Qwen		$0.4250\pm 0.3991$	$0.6583\pm 0.1750$	0.1500
General RAG	ChatGPT	$0.6667\pm 0.1209$	$0.6667\pm 0.1209$	$1.0000\pm 0.0000$	1.0000
	Gemini		$0.6667\pm 0.1209$	$1.0000\pm 0.0000$	1.0000
	DeepSeek		$0.6167\pm 0.1881$	$1.0000\pm 0.0000$	1.0000
	Qwen		$0.1917\pm 0.2247$	$0.6583\pm 0.1265$	0.0000
Governance-enhanced hybrid retrieval	ChatGPT	$1.0000\pm 0.0000$	$1.0000\pm 0.0000$	$1.0000\pm 0.0000$	1.0000
	Gemini		$0.9917\pm 0.0373$	$1.0000\pm 0.0000$	1.0000
	DeepSeek		$0.9750\pm 0.0611$	$1.0000\pm 0.0000$	1.0000
	Qwen		$0.8167\pm 0.3102$	$0.8333\pm 0.1950$	0.4500

presents the evidence coverage and answer-source consistency results for the different combinations. Gold-standard coverage in the evidence package is primarily determined by the front-end retrieval framework and, therefore, remains consistent with the preceding retrieval experiment. The mean gold-standard evidence-package coverage values of BM25, metadata-gated BM25, general RAG, and governance-enhanced hybrid retrieval are 0.7917, 0.9167, 0.6667, and 1.0000, respectively.

The results in Table 7 and Figure 5 show that retrieval quality is a key factor affecting the traceability of industrial decisions, but it is not the only factor. Although traditional BM25 and general RAG can provide some relevant information to the LLM, their lack of industrial knowledge-governance and task-constraint mechanisms results in insufficient evidence coverage and increases the risk of citing non-key sources. Metadata-gated BM25 can effectively reduce governance violations but still omits key evidence. In comparison, governance-enhanced hybrid retrieval achieves nearly complete or complete gold-standard evidence coverage across the four industrial tasks and substantially improves the consistency between LLM outputs and the actual evidence sources. However, differences remain among LLMs. For example, Qwen still exhibits insufficient source citation when complete evidence is provided, indicating that retrieval augmentation alone cannot fully prevent evidence deviation during generation. Introducing automated validation and human review is therefore necessary to ensure the reliability of industrial decisions.

The preceding comparison shows that the contribution of the enhanced RAG–LLM framework to industrial sustainability is reflected in the decision chain of “anomalous facts–gold-standard evidence–constrained actions–human release.” To analyze sustainability pathways in the model outputs, this study selects one audited enhanced RAG–LLM answer from each of the four tasks and extracts its decision-action and sustainability-effect content. The original answers and item-level audit records are provided in Supplementary Materials S32 and S29.

Table 8. Decision outputs and sustainability pathways for four industrial tasks using the enhanced RAG–LLM.

Scenario Task	Decision Action Output by the Enhanced RAG–LLM	Industrial-Sustainability Effect	Corresponding SDGs
Equipment-fault diagnosis (T1)	Conduct a controlled shutdown; resume trial operation only after completing lubrication, alignment, and sensor verification.	Limit secondary equipment damage and reduce spare-part and lubricant waste, friction-related energy consumption, and unplanned downtime.	SDG 9 (Target 9.4): Industry, Innovation and Infrastructure; SDG 12 (Target 12.2): Responsible Consumption and Production.
Quality-anomaly analysis (T2)	Isolate the anomalous batch; restore drying conditions and reinspect it; perform one controlled rework only through a validated route.	Reduce whole-batch scrapping, reprocessing energy consumption, solvent emissions, and hazardous-waste generation.	SDG 12 (Targets 12.4 and 12.5): Responsible Consumption and Production; SDG 13: Climate Action.
Energy optimization (T3)	Stagger the heat-treatment startup; after confirming production and safety constraints, isolate the idle compressor branch and continue monitoring.	Reduce compressed-air peak load, unnecessary electricity use, peak-demand costs, and indirect carbon emissions.	SDG 7 (Target 7.3): Affordable and Clean Energy; SDG 12 (Target 12.2): Responsible Consumption and Production; SDG 13: Climate Action.
Wastewater-compliance management (T4)	Stop discharge and divert the wastewater to a holding tank; resample it; maintain diversion until two consecutive readings comply, and inspect valves and interlocks.	Prevent noncompliant wastewater from entering the external environment, reduce water-environmental and permit-compliance risks, and lower the burden of anomalous-event disposition.	SDG 6 (Target 6.3): Clean Water and Sanitation; SDG 12 (Target 12.4): Responsible Consumption and Production.

presents the model outputs and their sustainability effects. Across the four tasks, the enhanced RAG–LLM forms evidence-grounded action recommendations under equipment-reliability, quality-control, energy-scheduling, and environmental-compliance constraints. These recommendations correspond to reducing secondary equipment damage and material waste, reducing whole-batch scrapping and reprocessing, reducing unnecessary energy consumption and indirect carbon emissions, and reducing water-environmental and discharge-permit compliance risks, respectively, thereby reflecting the role of the model in sustainable industrial decision support.

4.4.2. Ablation Experiments

To further analyze the contributions of different functional modules in the enhanced RAG–LLM framework to industrial decision reliability, ablation experiments were conducted using ChatGPT, which achieved relatively strong overall performance, as the generation model. The experiments examined differences in decision outputs under four configurations: no external enterprise knowledge, governed knowledge units only, a source-labeled evidence package, and the complete enhanced RAG–LLM framework.

Specifically, the first configuration supplies only the user decision request and no enterprise-knowledge evidence to analyze the output capability of an LLM without domain-knowledge constraints. The second configuration supplies the content of governed knowledge units but does not provide a ranked and source-bound evidence package, thereby validating the effect of knowledge organization on industrial decision support. The third configuration further introduces a source-labeled evidence package to analyze the role of evidence packaging and provenance tracing. The fourth configuration uses the complete enhanced RAG–LLM framework and adds automated validation and feedback regulation to evidence retrieval to evaluate the improvement in decision reliability produced by the closed-loop control module. Prompt protocols for the different ablation configurations are provided in Supplementary Material S4, and the item-level response-audit results are provided in Supplementary Materials S12, S14 and S15.

Table 9. Ablation results for the components of the enhanced RAG–LLM under ChatGPT.

Model-Framework Configuration	Gold-Standard Coverage	Six-Field Output Coverage	Human-Review Marker Rate
ChatGPT (no enterprise evidence)	0.0000	1.0000	1.0000
Governed knowledge units + ChatGPT	0.0000	1.0000	1.0000
Governed knowledge units + evidence package + ChatGPT + feedback	1.0000	1.0000	1.0000

presents the experimental results for the different module configurations. When only the user question is supplied, the model can generate a structurally complete industrial decision response, but its gold-standard source-ID coverage is zero because it lacks support from enterprise evidence, and it therefore cannot satisfy the traceability requirements of industrial decisions. After governed knowledge units are introduced, the model can use enterprise-domain knowledge to produce responses that better fit the business context. However, because an explicit evidence-binding mechanism is absent, the source-ID coverage remains zero, indicating that knowledge content alone cannot guarantee evidence traceability in the decision process. After a source-ID-labeled evidence package is introduced, an explicit association is established between model outputs and gold-standard evidence, and source-ID coverage increases substantially, showing that the evidence-packaging mechanism can effectively enhance the verifiability of industrial decision results. In the complete enhanced RAG–LLM framework, the additional automated validation and feedback-regulation module identifies omitted source citations, missing fields, and insufficient human-review conditions and improves output completeness and reliability through controlled repair.

These results show that governed knowledge units mainly address the unified organization of multi-source heterogeneous industrial knowledge, the evidence-packaging mechanism further establishes an explicit association between knowledge sources and decision results, and the feedback-regulation module ensures that the generated results satisfy industrial-application requirements through validation, correction, and human-review mechanisms.

To further validate the role of the feedback-regulation module in actual operation, an end-to-end feedback experiment was conducted using the complete enhanced RAG–LLM framework. The system recorded 320 decision responses and automatically checked the generated results according to gold-standard evidence coverage, source-ID citation, output-field completeness, and human-review requirements. It adopted a tiered handling strategy for different problems. Evidence-package incompleteness was automatically returned to the retrieval stage; omissions of source citations or output fields were automatically recorded as feedback and allowed one constrained repair; and unknown sources, omitted mandatory rules, or high-risk decisions stopped automated release and were transferred to human review. This strategy allows the system to preferentially handle problems that can be identified through rules and submits only problems that cannot be automatically resolved or that involve safety or compliance responsibilities for human confirmation, thereby reducing the cost of item-by-item manual inspection. The automated validation results for the 320 answers are presented in

Table 10. Automated checks and feedback triggers for 320 end-to-end answers.

Feedback Signal	Detection Rule	All Answers ($\mathit{n}=320$)	Enhanced RAG Answers ($\mathit{n}=80$)	Automated or Subsequent Handling
Retrieval-evidence gap	Gold-standard evidence coverage $<1$ (evidence_gold_coverage<1)	200 (62.5%)	0 (0.0%)	Automatically return to retrieval; re-retrieve or check task, equipment, and metadata settings
Omitted source-ID citation	Answer decision correctness below evidence coverage (response_DC<evidence_coverage)	58 (18.1%)	13 (16.3%)	Automatically record feedback and perform one source-ID self-check; transfer to human review if still incomplete
Incomplete six-field output	Six-field output coverage $<1$ (requested_field_coverage<1)	58 (18.1%)	11 (13.8%)	Automatically check and supplement missing fields, then re-verify the six output fields
Omitted sustainability effect	No sustainability effect is provided (sustainability=0)	24 (7.5%)	4 (5.0%)	Automatically prompt supplementation of effects on energy, materials, water resources, or compliance
Omitted human-review requirement	No human-review prompt is provided (human_review=0)	54 (16.9%)	11 (13.8%)	Automatically add a human-review requirement; stop release if it remains unsatisfied
Non-gold-standard ID cited	A non-gold-standard corpus source is cited (non_gold_corpus_id_count>0)	79 (24.7%)	0 (0.0%)	Automatically check source relevance; return to ranking if necessary
Unknown ID outside the corpus	An unknown source outside the corpus appears (unknown_id_count>0)	0 (0.0%)	0 (0.0%)	Automatically stop release and transfer to human review

, and records of multiple feedback-trigger types are provided in Supplementary Material S29.

Table 10 shows that the system effectively identifies the different validation signals among all 320 answers. Evidence gaps mainly arise from insufficient evidence produced by the comparison retrieval frameworks, whereas the 80 answers generated by the complete enhanced RAG–LLM exhibit no retrieval-evidence gaps or citations of non-gold-standard sources. The remaining problems in the enhanced RAG–LLM results are concentrated in generation-stage issues, including omitted source IDs, incomplete output fields, insufficient descriptions of sustainability impacts, and missing human-review conditions.

The system performs one controlled repair through the feedback-regulation mechanism and then re-executes the validation process. Results that still do not satisfy the release requirements or involve high-risk judgments are submitted for human review. One representative response that did not pass the automated release check was further selected from each of the four industrial tasks. After the feedback was confirmed by a human, the next decision cycle was rerun, and the before-and-after audit results are presented in

Table 11. Paired audit results before and after feedback.

LLM/Task	Source-ID Coverage Before Feedback	Source-ID Coverage After Feedback	Field Coverage Before Feedback	Field Coverage After Feedback	Unknown IDs (Before/After)
Qwen/T1	0.0000	1.0000	1.0000	1.0000	0/0
Gemini/T2	0.8333	1.0000	1.0000	1.0000	0/0
Qwen/T3	0.0000	1.0000	1.0000	1.0000	0/0
Qwen/T4	0.8333	1.0000	1.0000	0.8333	0/0

.

Table 11 shows that, after feedback regulation, the mean gold-standard source-ID coverage of the four representative responses increases from 0.4167 to 1.0000, while the number of unknown IDs outside the corpus remains zero throughout. This indicates that the feedback mechanism can effectively improve source consistency and traceability in industrial decision results.

Overall, the enhanced RAG–LLM does not complete decision support through one-time retrieval and generation alone. Instead, it continuously improves decision reliability through a closed-loop process of knowledge governance, evidence retrieval, constrained generation, automated validation, and feedback regulation. Without changing the parameters of the foundation LLM, the feedback-regulation module achieves continuous optimization for industrial environments by updating retrieval strategies, knowledge states, and validation rules in a controlled manner. The feedback algorithm, feedback signals, and human-approval records are provided in Supplementary Materials S20–S22, and the complete answers and paired audit results are provided in Supplementary Materials S32 and S33.

4.4.3. Parameter-Sensitivity Analysis

To analyze the effect of evidence-package size on enhanced RAG retrieval performance, a Top-k parameter-sensitivity experiment was conducted. The experiment fixed 32 governed knowledge units, four industrial decision tasks, and the eligibility-gating and hybrid-ranking parameters, while varying only the number of evidence items k returned during retrieval. Specifically, k was increased from 3 to 8 in steps of 1, and retrieval coverage, ranking quality, and governance reliability were evaluated under the different evidence-package sizes. The results were used to determine a uniform evidence-package size for the subsequent end-to-end decision experiments, ensuring sufficient coverage of gold-standard evidence while reducing the introduction of irrelevant evidence. Detailed experimental data are provided in Supplementary Material S8, and eligibility-gating results and ranking-process traces are provided in Supplementary Material S17.

As shown in

Table 12. Top-k parameter-sensitivity analysis of enhanced RAG retrieval.

Number of Gold-Standard Items k in All Evidence Packages	Mean Recall@k	Mean Precision@k	Mean nDCG@k	Mean GVR@k
3	0.5000	1.0000	1.0000	0.0000
4	0.6667	1.0000	1.0000	0.0000
5	0.8333	1.0000	1.0000	0.0000
6	1.0000	1.0000	1.0000	0.0000
7	1.0000	0.8571	1.0000	0.0000
8	1.0000	0.7500	1.0000	0.0000

, when k increases from 3 to 6, mean Recall@k increases from 0.5000 to 0.6667, 0.8333, and 1.0000, indicating that the system gradually covers more task-related gold-standard evidence as the evidence-package size increases. Under $k\le 6$, Precision@k remains 1.0000, showing that all returned results are valid evidence relevant to the task.

When $k=7$ and 8, the system has already retrieved all gold-standard evidence, and Recall@k remains 1.0000. However, because the additional returned knowledge units are non-gold-standard candidates that satisfy the governance constraints, Precision@k decreases to 0.8571 and 0.7500, respectively. GVR@k remains zero for all values of k, indicating that the eligibility-gating mechanism effectively prevents expired or unauthorized knowledge from entering the evidence package.

Overall, at $k=6$, complete gold-standard evidence coverage is achieved, while Precision@k remains 1.0000. This setting preserves complete evidence coverage, limits the introduction of unnecessary evidence, and provides a favorable balance between retrieval precision and evidence-package size.

5. Conclusions

This study proposes an enhanced RAG–LLM framework for decision support in sustainable industrial engineering and management. By constructing governed industrial knowledge units, the framework organizes private enterprise data, domain knowledge, and decision constraints in a unified form and combines hybrid retrieval, access control, evidence enhancement, constraint-aware generation, and response validation to improve the reliability and traceability of industrial decision support. Moreover, the RAG–LLM framework not only maps multi-source industrial data to intelligent decisions but also constructs a closed-loop decision mechanism comprising knowledge construction, enhanced retrieval, constraint-aware reasoning, decision validation, and feedback updating. This closed-loop mechanism can continuously update the knowledge base and reasoning process according to decision results and user feedback, thereby improving the adaptability of industrial intelligent-decision systems in dynamic environments.

The proposed RAG–LLM framework was validated through controlled simulated cases involving four representative industrial decision tasks: equipment-fault diagnosis, quality-anomaly analysis, energy optimization, and wastewater-compliance management. The experimental results show that the framework effectively improves the accuracy of knowledge retrieval in industrial scenarios, the reliability of decision evidence, and the verifiability of generated results. Specifically, the hybrid retrieval strategy incorporating metadata constraints substantially outperforms traditional keyword-matching methods and achieves higher recall of key knowledge units in the controlled industrial corpus. By further combining evidence-source identifiers with constraint-aware generation, the framework effectively associates decision results with supporting evidence and reduces the possibility that irrelevant, outdated, or unauthorized knowledge enters the reasoning process. These results validate the effectiveness of the proposed RAG–LLM framework in integrating multi-source knowledge, enhancing decision transparency, and improving the trustworthiness of intelligent decision-making in sustainable industrial engineering and management.

Despite these advantages, the framework still has several limitations. First, compared with traditional decision methods, integrating multi-source data processing, retrieval augmentation, and LLM reasoning may introduce additional computational costs. In large-scale data environments in particular, vector-index updates, knowledge-graph maintenance, and model reasoning may affect system-response efficiency. Second, the scalability of the framework remains challenging when it processes massive heterogeneous data in very large industrial enterprises and in multi-plant, multi-business-chain collaborative environments. Third, the effectiveness of RAG–LLM depends on the availability of high-quality metadata and structured knowledge representations, which may be difficult to maintain in dynamic industrial environments. Future research will explore real-time integration with MES/ERP/SCADA systems and digital-twin platforms, multimodal industrial knowledge retrieval, secure and privacy-preserving deployment, and long-term validation in real industrial scenarios to advance the practical application of the RAG–LLM framework in sustainable industrial engineering and management.