Review Reports

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The topic of the paper is timely and interesting. In the age of automation, the domain-specific application of AI can be significantly improved with structured knowledge of the problem domain, which is the thesis of the paper. The focus on perspective analytics further enhances the potential of the work's original scientific contribution. However, the paper requires significant improvement to fulfil that potential. Primarily, greater clarity is needed in explaining how the knowledge graph is constructed and integrated with the multi-modal data pipeline to support the inference of perspective analytics. Equally critical is providing sufficient detail on how the performance of the perspective analytics platform—claimed as the work’s most significant contribution—is evaluated.

Once the recommended improvements are made, a 'methodology' for employing KG-driven RAG systems for the targeted class of applications might emerge, which can be another valuable contribution of the paper, especially if its compressed into 'algorithmic' format. 

I offer the following remarks to help improve the paper:

L36: The use-case of transferring knowledge through generations of workers is not clear and does not sufficiently motivate the work. The smart factory offers a much stronger motivational scenario, attempt to link it here.

L107: here and elsewhere (e.g. L286), you rely inappropriately on references to previously published work to complement your arguments, which disrupts the paper’s readability. The critical articulation should be self-contained within the main body of the text.

L255: Can you exemplify how the interconnections can be useful? Similarly in L264, it would be interesting to elaborate, with examples, on the “multi-hop link prediction tasks”

L306-307: the schematic diagram is excellent and brings to the fore the interesting use-case of the IoT factory.

L346-349: It is not clear how the KG nodes are built! What is the meta-information?

L363: Ontologies are domain-specific in general, so the ‘strict-structure’ normally is not be considered a limitation. Can you explain your argument? You also need to elaborate on the methodology you used to produce the ontology skeleton (T-Box).

L379: what are the corresponding nodes and how do they relate to the text embedding?

L473: The autonomous execution requires some detail of the metric of the  achieved automation. What is its impact?

Figure 8: The caption is overly crowded; it would be better to move this content into the main body of the paper, where a more detailed description of the process flow can be provided.

L506-7: How are the relationships mapped? Another example of high-level description where the detail of the methodology is required.

L546-7: What is the detail of this study and why haven’t you performed the expected performance evaluation for KG-driven RAG?

L570: The efficient embedding of time-series temporal information into the knowledge graph is an interesting and complex task. Why haven’t you provided us with the detail of your approach. 

L683: can the appendix supplementary information on the prescriptive platform SLR be useful for improving the critical evaluation section in the main body of the paper?

Some typos: L155: ?? , L236-L254: the numbering , L295-6: assemble/disassemble

 

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

This is an interesting paper, which covers an important topic. It definitely deserves to be published after the corrections proposed below are addressed. The key contribution of this paper is the proposition to use knowledge graphs (KGs) combined with RAG, LLM, IoT and other technologies to build a prescriptive recommender system for manufacturing, where the required information is available.

Unfortunately, knowledge graphs not only help to decrease hallucinations of large language models but also can bring their own hallucinations.  Many years of developing and using automatically generated knowledge graphs reveal that often knowledge graphs require manual corrections. This is extremely time-consuming and inefficient activity, which often cannot be conducted for the large knowledge graphs. Respectively, funding agencies started the programs to make editing and correction of such knowledge graphs more efficient.  I did not find in the paper a study that the proposed knowledge graphs do not suffer from this issue. In general, the efficiency of the adding knowledge graphs is not shown in the paper. This can be added in several ways. One of them would be to compare the efficiency of the system that includes KG with the system without KG but only with LLM and RAG. The current presented efficiency of the system by cutting out the time needed for the user to fix the error in the manufacturing system does not separate the contribution of the knowledge graph relative to all other components of the system.  An alternative would be to analyze the knowledge graph itself by domain experts to confirm or correct/edit it.

The detailed comments are below.

Title and abstract: Integrating Graph Retrieval-Augmented Generation into Prescriptive Analytics Platforms

Review. I feel that the words “recommender system” should be in the title to put this paper in this context. In essence the proposed system is an advanced form of the recommender system that provides prescriptive recommendation. The focus of the paper is manufacturing with sufficiently available data but the title is not capturing it creating a mismatch.

Abstract.  … large language models (LLMs) can be leveraged to improve interpretability by translating complex recommendations into clear, context-specific explanations, enabling non-experts to grasp the rationale behind the suggested actions.

Review.  While generally it is possible, however, with the black box nature of the large language models themselves, it needs to be stated with this warning.

p.8. Lines 321-327.

Review. This text is an example of text that can be shortened in paper without losing the actual content for educated readers.

p.9. Lines 338-342.

Review. This is an overly optimistic statement.  LLM easily can lose some information that is present in the image and provide an erroneous output. Please edit this text. This text is an example of several such optimistic statements in the paper.  Please add warnings about limitations. Also, it will be useful to describe possible mitigations.

p.9. 357-369. Review. This text can be shortened.

p.10. line 383. The clustering of documents containing the same label shows that their semantic similarity is accurately captured in the embedding space.

Review. This is an overoptimistic claim. Both clustering n-D data and t-SNE quite often violate it.

p.11, lines 395-396. Review. Please provide technical details about these models.

p.11, line 399-400. Review. Please describe exactly what information was provided for the zero-shot classification.

p.11. Lines 410-414. Review.  The source text is ambiguous for both a human and a classifier.  Why is the discrepancy between results from humans and the classifier so high? Other factors might cause it not ambiguity.

p.11, lines 471-420, p.14, lines 503-505. Review.  The intent is good but used texts mix concepts. Thus, an automated system and even manual process cannot resolve them. It will require adding text that is unambiguous for three classes or modify the text. Please clarify how it was resolved?  

p.12, lines 422-423, p.14, Lines 503-538, p.15, line 538 … components included are a graph viewer for visualizing the knowledge graph.

Review.  It will be natural to pick up some error cases, action and general information nodes connected to them and inspect by the domain experts. Please show the graph viewer.  It is not clear at what stage the KG visualization is created. Testing KG only after integration with RAG and LLM can be too late with two more  layers added with their own possible errors.    

P12. Lines 437- 438. Review. Is a health index the aggregated single feature?

p.15. Line 550. The results (Table A1) showcase a significant time saving in the acquisition of knowledge required to solve the problem. p16, lines 135-151. Examples of actions prescribed.

Review. How much saving time is a result of the correct action prescription or is result of other provided relevant information?  The table needs to have this information/statistics

p. 16, lines 567-568. GraphRAG is more complex than traditional RAG, it requires more maintenance work to build and maintain the knowledge graph.

Review. Please specify technically this more maintenance work.

p. 17, line 637. Future research directions. First, the recommendation could potentially be wrong.

Review.  This section lists several options for future research directions to deal with wrong recommendations. But it misses an important cause of wrong recommendations, which is the lack of necessary information to feed KG, RAG and LLM.  The paper makes an implicit assumption that available information is sufficient to build a high-quality prescriptive recommender system. It is acceptable for well-established manufacturing processes, but not for new systems that are just in the design process. Often here most critical information is only known to the domain expert and is not externalized. This requires extracting that information from them by building so-called expert mental models to feed KG, RAG and LLM [https://arxiv.org/pdf/2509.10818].  

p.5 line 155. ??  Review: Please remove ??

References. lines 806-807. The same references.

48. Beijing Academy of Artificial Intelligence. bge-small-en-v1.5, 2024.
49. Beijing Academy of Artificial Intelligence. bge-base-en-v1.5, 2024.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

The revised manuscript demonstrates a clear improvement in the quality of the submitted research, and most of the previously raised concerns have been satisfactorily addressed. However, to meet the publication standards expected by MDPI, I recommend the following amendments related to the evaluation of the analytical platform:

1) Please move Table 1, which details the improvement in efficiency, to Section 5.1. Additionally, expand the discussion on the impact of reduced analytics time on the industrial process—both within Section 5.1 and specifically in lines 719–721. This will help contextualise the technical improvement in terms of practical relevance.

2) In your response, you mention plans to conduct a [technical] performance evaluation in the future. However, this is not reflected in the manuscript’s “Future Work” section. Please include a description of how such an evaluation could further validate the platform’s practical deployment and contribute to its robustness.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

Reviewer Comment:

p.15. Line 550. The results (Table A1) showcase a significant time saving in the acquisition

of knowledge required to solve the problem. p16, lines 135-151. Examples of actions prescribed.

Review. How much saving time is a result of the correct action prescription or is result of other

provided relevant information? The table needs to have this information/statistics

Response:

The time savings result from a combination of the action provided and the corresponding

contextual information. Once the correct error has been found in the knowledge graph, the corresponding linked action and contextual information are retrieved. Therefore, it is not possible for us to determine exactly what percentage of the time savings is attributable solely to the action and what percentage is attributable to the contextual information.

Second review.

The key appeal of this paper is that adding a knowledge graph will improve the recommendation system. While it is expected intuitively, the answer shows that the design of the study did not allow to identify the actual contribution of the knowledge graph. An alternative study design could do this. I believe that the paper needs to clearly state it as a current study design limitation and as a suggestion for future work.  Knowing clearly the actual contribution of the knowledge graph will help to further improve the recommendation system.

 

 

The key appeal of this paper is that adding a knowledge graph will improve the recommendation system. While it is expected intuitively, the answer shows that the design of the study did not allow to identify the actual contribution of the knowledge graph. An alternative study design could do this. I believe that the paper needs to clearly state it as a current study design limitation and as a suggestion for future work.  Knowing clearly the actual contribution of the knowledge graph will help to further improve the recommendation system.

 

Author Response

Please see the attachment.

Author Response File: Author Response.pdf