An Enhanced NSGA-II Driven by Deep Reinforcement Learning to Mixed Flow Assembly Workshop Scheduling System with Constraints of Continuous Processing and Mold Changing

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The paper identifies the mixed-flow assembly line scheduling problem in manufacturing, particularly with multiple product variants, overload, and mold-changing constraints. Existing literature mainly focuses on efficiency or customer satisfaction, with limited work on workload balancing, efficiency, and delivery performance in a unified scheduling model. The authors propose a scheduling optimization model addressing both overload and mold-changing constraints, aiming to minimize additional production time and delivery deviation. An enhanced NSGA-II algorithm is developed, integrating a deep reinforcement learning (DRL) guided neighborhood search for improved local search and solution quality.

Weaknesses and Areas for Improvement

1. The abstract would be strengthened by including numerical values that highlight the improvements achieved with the proposed approach. At lines 25-27, authors simply state “Simulation experiments demonstrate that the proposed algorithm outperforms existing methods in local search performance,…”
2. In the introduction, summarize the core problem in 1–2 clear sentences early on, before detailing the literature review. When discussing challenges such as overloads or mold changes, clearly link each one to the specific method proposed to address it, such as multi-objective optimization or deep reinforcement learning (DRL)-guided search. Additionally, consider incorporating a concise table that maps each identified research gap or challenge to the corresponding contribution made in the study, providing a clear and structured overview of the work’s impact.
3. The introduction currently presents a wide range of prior studies and scheduling challenges, but the transitions between topics—such as shifting from algorithmic trends to specific application constraints—can feel abrupt. To improve the flow, consider guiding the reader more deliberately: begin with broad trends like the adoption of digital technologies in manufacturing, then narrow the focus to specific challenges such as mixed-flow assembly, and finally highlight the unresolved issues. Organizing the literature into thematic clusters—such as algorithmic advancements and application-specific constraints—can also help clarify the narrative. Clearly indicate what each group of studies addresses and where gaps remain.
4. Additionally, avoid clustering multiple citations at the end of sentences; instead, briefly explain the relevance of each major work to the problem at hand. For example, instead of writing something like “Current research has explored diverse production scenarios in mixed-flow assembly lines: Wu et al. investigated..., Shen et al. and Zhang et al. emphasized..., Ning et al. focused on..., Zheng et al. also studied..., Wang et al. incorporated...”, use an approach more like “Wu et al. modeled a mixed-flow assembly system with… Ning et al. developed a decision approach for ...”
5. Toward the end of the introduction, explicitly state the research questions—such as how to simultaneously achieve workload balancing, efficiency, and delivery performance in mixed-flow line scheduling under real-world constraints. Follow this with a clear summary of the study’s goals, either in a concise paragraph or bullet list. For example: (1) develop a model that incorporates both overload and mold-changing constraints; (2) design an algorithm that adaptively selects neighborhood search operators based on population diversity and solution quality; and (3) demonstrate improvements in both efficiency and delivery reliability compared to existing methods.
6. The paper offers a clear and well-structured technical methodology, with detailed modeling and a step-by-step explanation of the enhanced NSGA-II algorithm integrated with deep reinforcement learning. It includes sufficient experimental details to support reproducibility. However, full replicability is limited by the lack of complete pseudocode, implementation specifics, and access to datasets or source code. Sharing these additional materials would significantly improve the ability of others to reproduce the results precisely.
7. The current short conclusion effectively summarizes the main contributions of the study but offers only limited insight into its real-world applicability. To strengthen this section, it is recommended to explicitly restate the key findings, including quantitative performance metrics—for example, noting that the proposed DRL-NSGA-II approach achieves a specific percentage improvement in generational distance and hyper-volume compared to benchmark methods, thereby demonstrating superior convergence and solution diversity.
8. Additionally, the conclusion should elaborate on the practical implications of the research. This includes explaining how scheduling managers could apply the proposed model to manage real production lines characterized by high product customization. The potential for adoption in industries such as automotive and electronics, where complex assembly processes are common, should also be highlighted. Finally, the conclusion could suggest application pathways, such as integrating the model into manufacturing execution systems (MES), enhancing real-time production scheduling, or serving as a foundation for future work that incorporates inventory and logistics considerations.

Comments on the Quality of English Language

The paper demonstrates a solid command of technical vocabulary and adheres to the standard structure expected in scientific writing. Most sentences are syntactically correct and effectively convey the intended meaning. However, there are areas where the English language and academic style could be improved. Some sentences are overly long or contain awkward phrasing, which affects fluency and conciseness—for instance, repetitive constructions like “To address …” appearing in close succession, or sentences overloaded with too many details. Additionally, the cohesion between paragraphs and themes could be enhanced by using transitional phrases such as “Furthermore,” “In contrast,” or “To bridge this gap,” to guide the reader more smoothly through the argument. Lexical consistency is also important; key terms like “overload constraints” should be used uniformly rather than alternating with variations like “workstation overload.”

To address these issues, a thorough language editing pass is recommended, focusing on clarity, active voice, and reducing redundancy. For submissions to this journals, professional English proofreading may also be beneficial.

Author Response

• Comments 1: The abstract would be strengthened by including numerical values that highlight the improvements achieved with the proposed approach. At lines 25-27, authors simply state “Simulation experiments demonstrate that the proposed algorithm outperforms existing methods in local search performance,…”
• Response 1: Thanks for your comments. Agree. The previous version of the manuscript did not standardize the results after calculating GD and HV. This time we have first corrected this oversight. Furthermore, we have revised the abstract based on your suggestions. Please refer to line 32 to line 34 for modifications.

 

• Comments 2: In the introduction, summarize the core problem in 1–2 clear sentences early on, before detailing the literature review. When discussing challenges such as overloads or mold changes, clearly link each one to the specific method proposed to address it, such as multi-objective optimization or deep reinforcement learning (DRL)-guided search. Additionally, consider incorporating a concise table that maps each identified research gap or challenge to the corresponding contribution made in the study, providing a clear and structured overview of the work’s impact.
• Response 2: Thanks for your comments. We have rewritten the first paragraph of the article. Based on your feedback, the latest manuscript more directly introduces the background and research question. For specific modifications, please refer to lines 39 to 58 of the manuscript. Additionally, we have added Table 1. Current and Previous Studies in line 107.

 

• Comments 3: The introduction currently presents a wide range of prior studies and scheduling challenges, but the transitions between topics—such as shifting from algorithmic trends to specific application constraints—can feel abrupt. To improve the flow, consider guiding the reader more deliberately: begin with broad trends like the adoption of digital technologies in manufacturing, then narrow the focus to specific challenges such as mixed-flow assembly, and finally highlight the unresolved issues. Organizing the literature into thematic clusters—such as algorithmic advancements and application-specific constraints—can also help clarify the narrative. Clearly indicate what each group of studies addresses and where gaps remain.
• Response 3: Thank you for your feedback. Agree. We have readjusted the structure of the literature based on your suggestions. The current structure is divided into model construction and solution methods. The model components are categorized into single-objective optimization and multi-objective optimization. The solution methods are divided into two categories based on whether reinforcement learning is used, in line with the theme of this paper. Furthermore, we have added Table 1. Current and Previous Studies to discuss the specific challenges and innovations of this paper. Please see lines X to X for specific modifications.

 

• Comments 4: Additionally, avoid clustering multiple citations at the end of sentences; instead, briefly explain the relevance of each major work to the problem at hand. For example, instead of writing something like “Current research has explored diverse production scenarios in mixed-flow assembly lines: Wu et al. investigated..., Shen et al. and Zhang et al. emphasized..., Ning et al. focused on..., Zheng et al. also studied..., Wang et al. incorporated...”, use an approach more like “Wu et al. modeled a mixed-flow assembly system with… Ning et al. developed a decision approach for ...”
• Response 4: Thank you for your feedback. Agree. We have adjusted the argument in this section. Please see lines 63 for specific modifications.

 

• Comments 5: Toward the end of the introduction, explicitly state the research questions—such as how to simultaneously achieve workload balancing, efficiency, and delivery performance in mixed-flow line scheduling under real-world constraints. Follow this with a clear summary of the study’s goals, either in a concise paragraph or bullet list. For example: (1) develop a model that incorporates both overload and mold-changing constraints; (2) design an algorithm that adaptively selects neighborhood search operators based on population diversity and solution quality; and (3) demonstrate improvements in both efficiency and delivery reliability compared to existing methods.
• Response 5: Thank you for your feedback. Agree. As shown in lines 119 to 125 of the latest manuscript, we presented the issues and key points of this article after summarizing the similarities and differences with existing results.

 

• Comments 6: The paper offers a clear and well-structured technical methodology, with detailed modeling and a step-by-step explanation of the enhanced NSGA-II algorithm integrated with deep reinforcement learning. It includes sufficient experimental details to support reproducibility. However, full replicability is limited by the lack of complete pseudocode, implementation specifics, and access to datasets or source code. Sharing these additional materials would significantly improve the ability of others to reproduce the results precisely.
• Response 6: Thank you for your suggestion. Agree. We have added overall pseudocode at the beginning of Section 3 (line 266). This pseudocode reflects the overall structure of the algorithm. This pseudocode encompasses the content of the remaining part of Section 3. The parts proposed in this paper are specifically introduced in Section 3.

 

• Comments 7: The current short conclusion effectively summarizes the main contributions of the study but offers only limited insight into its real-world applicability. To strengthen this section, it is recommended to explicitly restate the key findings, including quantitative performance metrics—for example, noting that the proposed DRL-NSGA-II approach achieves a specific percentage improvement in generational distance and hyper-volume compared to benchmark methods, thereby demonstrating superior convergence and solution diversity.
• Response 7: Thank you for your suggestion. Agree. We further provided the leading amplitude of the hypervolume. (Line 471 to 473)

 

• Comments 8: Additionally, the conclusion should elaborate on the practical implications of the research. This includes explaining how scheduling managers could apply the proposed model to manage real production lines characterized by high product customization. The potential for adoption in industries such as automotive and electronics, where complex assembly processes are common, should also be highlighted. Finally, the conclusion could suggest application pathways, such as integrating the model into manufacturing execution systems (MES), enhancing real-time production scheduling, or serving as a foundation for future work that incorporates inventory and logistics considerations.
• Response 8: Thank you very much for your suggestion. Agree. We have enriched the application value of the article in the conclusion section (line 472 to 479).

 

In addition, this article has been polished for English expression. Long sentences in the article have been reduced to make the presentation clearer. Finally, thank you for reviewing the article and for your suggestions. Wishing you all the best.

 

 

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

1. The research gap is unclear from the abstract. Why is this study vital for the discipline?
2. Line 62 says many scholars, who are those? Similar things need to be taken care of.
3. Introduction has a lot of information and it seems like jumping from one aspect to another. Be concise and to the point on finding research agendas and its introductory context.
4. Section 2 explains the problem formulation clearly, but the notation table (Table 1) would benefit from better alignment and clearer categorization of symbols (e.g., decision variables vs. parameters). What are the sources of finding parameters?
5. Section 2.3 talks about model formulation, but it is hard to understand whether such a model exists in reality, or is this the new one or an extension?
6. The objective functions are mathematically defined, but the rationale behind selecting them (especially weightings and assumptions) is not elaborated for readers less familiar with the manufacturing context.
7. Equation after equation, mathematically presented but with insufficient explanation based on the case or process, reducing readability and understandability for those with less functional knowledge.
8. Do figures 2 and 5 originate from the source or were they created? How?
9. Figure 3 needs more details; the same is true for Figure 4.
10. The simulation process needs to be better clarified with the application used, parameter used, the scenario considered, etc.
11. Lines 378-383 say, “All three experiments used the same local search operator library. Based on the 378 experimental results and related analysis, it can be concluded that under different test 379 cases, the proposed deep reinforcement learning-driven operator can guide the algorithm 380 to effectively utilize population diversity information and the objective function 381 information of search objects to select appropriate search operators, thereby improving 382 the algorithm's solving performance.” Explain the details.
12. While mold-changing and continuous processing constraints are acknowledged, their real-world motivation or industrial implications are not discussed, making the formulation appear abstract.
13. The constraints are mathematically sound, but some equations (like constraints) would benefit from a short sentence explaining their meaning to improve accessibility.
14. No illustrative example or diagram showing the scheduling context (e.g., workstation layout or job flow) would support comprehension for practical readers.
15. Grammar is mostly acceptable, though some sentences are overly long and dense, especially in the methodology and results sections.
16. The conclusion restates findings but adds little about policy implications or real-world deployment potential.

Comments on the Quality of English Language

1. Grammar is mostly acceptable, though some sentences are overly long and dense, especially in the methodology and results sections.

Author Response

• Comments 1: The research gap is unclear from the abstract. Why is this study vital for the discipline?
• Response 1: Thank you for your suggestion. Agree. We have enriched the current difficulties encountered in the mixed-flow assembly line in the abstract section (line 15 to 21) and the introduction section (line 39 to 58) to highlight the research gap.

 

• Comments 2: Line 62 says many scholars, who are those? Similar things need to be taken care of.
• Response 2: Thank you for your feedback. Agree. We have readjusted the structure of the literature based on your suggestions. The current structure is divided into model construction and solution methods. The model components are categorized into single-objective optimization and multi-objective optimization. The solution methods are divided into two categories based on whether reinforcement learning is used, in line with the theme of this paper. Furthermore, we have added Table 1. Current and Previous Studies to discuss the specific challenges and innovations of this paper. Please see lines 59 to 117 for specific modifications.

 

• Comments 3: Introduction has a lot of information and it seems like jumping from one aspect to another. Be concise and to the point on finding research agendas and its introductory context.
• Response 3: Thank you for your feedback. Agree. We rewrote the first paragraph of the article and adjusted the literature structure as mentioned in the previous response. Taking into account the opinions of other reviewers, the latest manuscript introduces the background and research question more directly. Please refer to lines 39 to 58 of the manuscript for specific modifications. Additionally, we added Table 1. Current and Previous Studies in line 107 and analyzed the focus of this study afterward (Line 109 to 117).

 

 

• Comments 4: Section 2 explains the problem formulation clearly, but the notation table (Table 1) would benefit from better alignment and clearer categorization of symbols (e.g., decision variables vs. parameters). What are the sources of finding parameters?
• Response 4: Thank you for your feedback. Agree. We have reclassified and arranged Table 2 in line 172 (Table 1 of the first version of the manuscript) according to the classification of the variables. The model parameters in this article are determined based on a certain electric vehicle mixed-flow assembly line, as stated in the experimental section of Chapter 4 (see line 394 to 396).

 

• Comments 5: Section 2.3 talks about model formulation, but it is hard to understand whether such a model exists in reality, or is this the new one or an extension?
• Response 5: Thank you for your feedback. Agree. To clarify the sources of the model used in this article, we indicated that the model presented in this paper is based on a flexible job shop scheduling optimization model (please refer to line 168 for specifics). From a practical perspective, this article has repeatedly clarified the originating scenarios of the problems discussed (please refer to lines 42, 174 to 176, and 396 to 398 for specifics).

 

• Comments 6: The objective functions are mathematically defined, but the rationale behind selecting them (especially weightings and assumptions) is not elaborated for readers less familiar with the manufacturing context.
• Response 6: Thank you for your feedback. Agree. In the latest manuscript, we have detailed the calculation methods for the two objective functions based on the first manuscript version to improve readability (please refer to line 183 to 185 and 187 to 189 for specifics).

 

• Comments 7: Equation after equation, mathematically presented but with insufficient explanation based on the case or process, reducing readability and understandability for those with less functional knowledge.
• Response 7: Thank you for your feedback. Agree. To enhance the readability of the model, we ensured that each model constraint has specific calculation ideas in the latest version of the manuscript. For details, please refer to the constraint analysis section in part 2.3 of the manuscript.

 

• Comments 8: Do figures 2 and 5 originate from the source or were they created? How?
• Response 8: Thank you for your feedback. Agree. The solution algorithm in this paper is based on improvements made to NSGA-II, hence the process depicted in Figure 2 is derived from NSGA-II. We have added content in line 259 to 262 of the manuscript to highlight the source of Figure 2. Figure 5 shows the deep reinforcement learning-driven operator designed in this paper; we have added its concept in line 348 to 356 to explain how this figure was created.

 

• Comments 9: Figure 3 needs more details; the same is true for Figure 4.
• Response 9: Thank you for your feedback. Agree. We have provided a more detailed introduction to Figure 3 and Figure 4 in the latest manuscript. For the content of Figure 3, please refer to line 281 to 285. We have further related the existing text in section 3.2 to Figure 4 to enhance its readability. (Please specifically refer to lines 301 to 302, 308, and 327 for this content.)

 

• Comments 10: The simulation process needs to be better clarified with the application used, parameter used, the scenario considered, etc.
• Response 10: Thank you for your feedback. Agree. The testing environment and parameter settings have been improved in section 4.1 of the article. (Please specifically refer to lines 388 to 394 for this content.). At the beginning of section 4.2, we added the sources of the examples used in this article and the scenarios covered. (Please specifically refer to lines 396 to 401 for this content.)

 

 

• Comments 11: Lines 378-383 say, “All three experiments used the same local search operator library. Based on the 378 experimental results and related analysis, it can be concluded that under different test 379 cases, the proposed deep reinforcement learning-driven operator can guide the algorithm 380 to effectively utilize population diversity information and the objective function 381 information of search objects to select appropriate search operators, thereby improving 382 the algorithm's solving performance.” Explain the details.
• Response 11: Thank you for your feedback. Agree. First, referring to the comments from other reviewers, we supplemented the convergence experiments of the reinforcement learning-driven operator (Please specifically refer to lines 410 to 418 for this content.). Based on the results of this convergence experiment and subsequent results related to GD and HV, this paper further explains the details of this part: the reinforcement learning-driven operator can accurately predict the state values under various conditions and search for operators based on its decisions. The other two algorithms are based on fixed rules or random decision search operators. With the same local search operator library, RLVNS-NSGA-II benefits from the reinforcement learning-driven operator and demonstrates better performance. (Please specifically refer to lines 452 to 461 for this content.)

 

• Comments 12: While mold-changing and continuous processing constraints are acknowledged, their real-world motivation or industrial implications are not discussed, making the formulation appear abstract.
• Response 12: Thank you for your feedback. Agree. In order to better highlight that the research questions of this paper genuinely exist in reality, this paper further provides examples of the sources of these issues in the Line 39 to 58 and 174 to 176. As mentioned earlier, the source of the examples in this paper has been supplemented at the beginning of section 4.2. (Please specifically refer to lines 396 to 399 for this content.)

 

• Comments 13: The constraints are mathematically sound, but some equations (like constraints) would benefit from a short sentence explaining their meaning to improve accessibility.
• Response 13: Thank you for your feedback. Agree. As stated in Response 7, we have ensured that each model constraint has a specific calculation approach in the latest version of the manuscript. For more details, please refer to the analysis of constraints in section 2.3 of the manuscript.

 

• Comments 14: No illustrative example or diagram showing the scheduling context (e.g., workstation layout or job flow) would support comprehension for practical readers.
• Response 14: Thank you for your feedback. Agree. We have supplemented the statement 'The production line layout and processing flow are known' in the problem description 2.1. (Please specifically refer to lines 137 for this content.) We further highlighted this element in Figure 1. (Please specifically refer to lines 149 for this content.)

 

• Comments 15: Grammar is mostly acceptable, though some sentences are overly long and dense, especially in the methodology and results sections.
• Response 15: Thank you for your feedback. Agree. We have polished the entire article to enhance its readability.

 

• Comments 16: The conclusion restates findings but adds little about policy implications or real-world deployment potential.
• Response 16: Thank you very much for your suggestion. Agree. We have enriched the application value of the article in the conclusion section (line 474 to 481).

 

Finally, thank you for reviewing the article and for your suggestions. Wishing you all the best.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors The study focuses on the mixed-flow assembly workshop scheduling problem, integrating continuous processing constraints and mold-changing constraints to construct a multi-objective optimization model aiming to minimize additional production time and customer waiting time. It proposes a deep reinforcement learning (DRL)-driven enhanced NSGA-II algorithm (RLVNS-NSGA-II). The research topic is closely aligned with the practical needs of manufacturing, and the idea of introducing DRL to guide the selection of neighborhood search operators, addressing the insufficient local search capability of traditional heuristic algorithms, exhibits innovation. It fills the gap in existing research where iterative optimization data are rarely used to drive scheduling decisions. Experimental results demonstrate that RLVNS-NSGA-II performs better in most test cases, particularly showing significant advantages in convergence (GD), solution diversity (IGD), and comprehensive performance (HV), validating the effectiveness of the DRL-driven strategy. However, there are aspects that can be improved:   1. The literature review in the introduction should be more focused to clearly clarify the essential differences between this study and existing applications of DRL in scheduling. 2. There is a lack of time complexity analysis of the algorithm, and its applicability in large-scale problems has not been verified. 3. The learning stability of the DRL module is not analyzed; it is recommended to present convergence curves during training iterations.

Author Response

• Comments 1: The literature review in the introduction should be more focused to clearly clarify the essential differences between this study and existing applications of DRL in scheduling.
• Response 1: Thank you for your feedback. Agree. We have readjusted the structure of the literature based on your suggestions. The current structure is divided into model construction and solution methods. The model components are categorized into single-objective optimization and multi-objective optimization. The solution methods are divided into two categories based on whether reinforcement learning is used, in line with the theme of this paper. (Please see lines 59 to 106 for specific modifications.) Furthermore, we have added Table 1. Current and Previous Studies to discuss the specific challenges and innovations of this paper. (Please see lines 107 to 117 for specific modifications.)

 

• Comments 2: There is a lack of time complexity analysis of the algorithm, and its applicability in large-scale problems has not been verified.
• Response 2: Thank you for your feedback. Agree. This article is based on the design improvement algorithm of NSGA-II, but does not alter the content of fast nondominated sorting, crowding distance calculation, and other components. Therefore, its algorithmic complexity remains . This point is emphasized in the new manuscript at line 259 to 262.

 

• Comments 3: The learning stability of the DRL module is not analyzed; it is recommended to present convergence curves during training iterations.
• Response 3: Thank you for your feedback. Agree. We added the prediction errors of the deep reinforcement learning-driven operator under different scenarios in line 410 to 418. From the results, it can be seen that the operator is able to determine that the errors gradually decrease as training progresses. This proves that the reinforcement learning-driven operator can accurately assess the current state and decide on the appropriate action.

 

In addition, this article has been polished for English expression. Long sentences in the article have been reduced to make the presentation clearer. Finally, thank you for reviewing the article and for your suggestions. Wishing you all the best.

 

Author Response File: Author Response.pdf