Multi-Agent Transfer Learning Based on Evolutionary Algorithms and Dynamic Grid Structures for Industrial Applications
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsThe paper is written in correct technical English, though the authors could re-read it and eliminate some redundancies, as ,,Distributed production systems'' repeated a number of times both in the introductory part, as well as in the abstract of the paper. The same applies to ,,knowledge transfer'' seen multiple times in the paper. Some sentences ould be splitten into two, such as the one describing the first challenge. Please provide also explaination to ,,convex hull.. or ,,Delaunay triangulation.. to link it to the literature.
In the paper, the authors propose Evolutionary State-based Potential Games (with Dynamic Grid Structures and multi-agent knowledge transfer for distributed optimization in manufacturing systems. The references are cited properly and aptly selected, and despite the fact that similar evolutionary multi-agent transfer learning exists elsewhere, different mechanisms (model-based opponent prediction or submodular selection) are used therein. This work extends Yuwono/Schwung's transfer learning by adding evolutionary algorithms and DGS. It shows a decent advancement to self-optimizing manufacturing by integrating EA for population-based exploration, DGS for adaptive state-action coverage, as well as agent-to-agent transfer, outperforming baseline solutions (vanilla SbPG, gradient-based). The limits of the solution are properly addressed. However, scalability to non-lab environments is untested.
​
​In the introductory part of the paper, prior work in multi-agent reinforcement learning and game theory is cited for the field of for production systems, refering to foundations, model-based extensions, Stackelberg hybrids, as well as gradient methods. Key limitations are efined, and from this point the core novelties arise, inclyuding EA-integrated SbPG with convergence guarantees, DGS population-based state-action grids adapting via system dynamics, as well as EA-embedded transfer. The novelty is build on previous papers of the authors. Theorems prove convergence and present properties of the solution. The experimental assessment is broad with multiple indices.
From this point one could state the drawback in the form of no heterogeneous agent tests, or no attempt to show scalability. The perfect state observation is also a srtong assumption, potentially leading to problems such as partial observability or delayed measurements, common in engineering practice.
In the future papers, try to introduce some quantitative measures, such as Hausdorff distance between population convex hull and state space, or epsilon-net coverage, with proofs of adaptation speed under system dynamics. Try to replicate on standard MARL environments or on another industrial sim (as FlowShop), include also real noise, as sensor delays or actuator failures. To increase the reproduction of your papers, try to make the code available on the GitHub, and add heatmaps of DGS evolution over episodes.
Author Response
Please see the attachment.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Comments and Suggestions for AuthorsThe paper is in the area of multi-agent transfer learning.
The abstract of the paper is comprehensive, even though some more information related to the results obtained through the present work would have been helpful. Thus, if possible, please enhance this part.
The introduction provides a series of references to papers in the field. Also, the authors are summarizing the main results of the paper, which is very helpful. A roadmap of the paper is provided in this section.
Section 2 is dedicated to literature review, while section 3 captures the problem to be addressed in the paper.
Section 4 presents the framework. Figure 3 is hard to read, thus, I suggest to the author to increase the font used in this figure.
Section 5 is quite long, I suggest to the authors to provide a short description or scheme at the beginning of this section in order to better understand its purpose. For the equations used, it is not clear if they belong to the authors - if not, please add references. The same observation for section 6.
Moreover, the text in Figure 6 should be increased as well.
In section 7, it is unclear to which extent by changing the environment the results would change. Please add a discussion in this direction.
If possible, please add discussions and comparisons with other works from the field.
Limitations should be added. The concluding remarks should be more specific, thus rather than saying that "and can be adapted for various applications.", please try to provide some examples of such applications.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 3 Report
Comments and Suggestions for Authors1. The relationship between the physical meaning of Equations (2) and (3) in Definition 1 and the "evolutionary population-based learning process" is not fully articulated. Please add a brief paragraph at the end of Section 4 explaining how evolutionary operations (recombination, mutation) are integrated into the game update and clarifying the specific role of Equation (3) in the convergence proof to strengthen theoretical completeness.
2. Sections 5.1 to 5.2 describe the update of support vectors, and local and global selection, but lack an overall flowchart illustrating the complete process from state input to action output.Please add a sub-figure or process schematic alongside Figure 4 to aid understanding of the specific implementation of "dynamic adjustment" and "coverage maintenance".
3. Tables 1 and 2 present comparison results for various methods (e.g., Vanilla SbPG, GB SbPG M1/M2), but do not specify the configuration differences among these baseline methods (e.g., whether the grid is static, if gradient updates are used). Please briefly outline the settings of each comparative method at the beginning of Section 7.2 and incorporate statistical tests (e.g., t-test or Wilcoxon test) into the results analysis to enhance the credibility of the conclusions.
4. Equation (16) uses Euclidean distance to measure similarity between individuals, but does not justify the choice of this metric or its effectiveness in a dynamic, high-dimensional state space. Please supplement Section 5.3 with an explanation of normalization in similarity calculation, or a discussion on the potential impact of other metrics (e.g., cosine similarity), and to justify the threshold setting in Equation (17) where transfer occurs "only if it improves the target fitness".
5. The time series plots in Figure 8 contain many lines, and the legend labels are not very clear. Please use more distinct colors and line styles to differentiate between methods and briefly state the units of physical quantities for each subplot in the caption. In Figure 9, the "1σ" and "2σ" labels overlap with the IQR in the KDE plot; consider displaying them separately or using different fill patterns.
6. The conclusion in Section 8 is somewhat general and does not explicitly mention the specific advantages of Evo-SbPG + DGS in terms of "computational efficiency," "scalability," or "real-time capability."Please add a paragraph in the conclusion briefly summarizing the practical applicability and potential limitations of the method, and to concretize "future research" into 2–3 actionable directions (e.g., "knowledge transfer mechanisms in multi-objective games," "adaptability of dynamic grids in non-stationary environments").
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Round 2
Reviewer 3 Report
Comments and Suggestions for AuthorsThis paper can be accepted now.
Author Response
Thank you very much.
We are delighted that we were able to implement your comments so effectively.
