Generative Design by Using Exploration Approaches of Reinforcement Learning in Density-Based Structural Topology Optimization
Round 1
Reviewer 1 Report
In this work, the authors extend Bi-directional evolutionary optimization (BESO) method based on four classes of exploration algorithms of reinforcement learning (RL) to structural topology optimization (STO) problems, which proceeds the Generative Design in a new way. The four approaches are: first, ?-greedy policy to disturb the search direction; second, Upper confidence bound (UCB) to add a bonus on sensitivity; third, Thompson sampling (TS); and forth, Information-directed sampling (IDS) to direct the search. Te authors concluded that UCB is used as a bonus when calculating the sensitivities, which encourages the exploration on less-exploited elements, it’s able to create acceptable results without adding the computation complexity dramatically.
The purpose of the topology optimization is to obtain an optimized topology, which satisfies the given constrints. The reviewer think that the readers studying on topology optimization are interested in how to get an optimal topology effectively and accurately using GAN, CNN, RL or DQN etc. rather than how to get generative designs.
The reviewer thinks that this study is only exercising the four approaches well known for topology problems, and then found that USB method is recommendable approach. And there is no the authors' contribution. This study should be further proceed to get an optimal topology using the generative designs and deep learning algorithms such as GAN, CNN, RL or DQN etc. So, this paper does not have publishable novelty and quality in this journal.
Author Response
Please see the attachment.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Please extend the literature review and highlight what is new at work
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 3 Report
The generative design methods are useful to find novel solutions in structural optimization. As the authors claim, those novel solutions are dependent of the search space size and of the capability of the algorithms to exploit and explore that space. The manuscript is interesting and clear to the readers. However, some issues must be addressed:
1- Please avoid lump sum citations. Critically review each one individually in text.
2- Please consider adding a workflow of the algorithms.
3- Verify if all variables are defined. Consider adding a nomenclature table.
4- For each case study, please add information on runtime and evolving indicators for each algorithm variation. This will provide evidence of the claims presented in the results.
5- Figures should be reference in text before showing them.
6- Consider merging Figures 4 and 5; Figures 7 and 8; and Figures 10 and 11.
7- Avoid references in conclusions. The conclusion section should only focus on the conclusions of the work.
8- Do not reference manuscripts under review. Only published works should be mentioned.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 4 Report
Review comments: The manuscript presents a method of bi-directional evolutionary structural optimization (BESO) with four different exploration algorithms of reinforcement learning. A brief review of each algorithm and topology optimization is provided. Numerical examples of topology optimization with an objective function of compliance minimization and a volume constraint are illustrated to show the performance of the proposed method. Although the proposed method utilizes exploration algorithms in BESO, there is a lack of significant development and a new contribution to the optimization research field. Those should be highlighted and emphasized by comparing the proposed method (results) to existing approaches. Also, illustrated numerical examples are focused on solving ‘toy problems.” Optimization problems with more realistic engineering-related design criteria are recommended. Below are some questions and comments that have been raised during the review. They should be addressed before this manuscript can be recommended for publication.
2.2 Sensitivity analysis and filter scheme
Eq.(2) shows the sensitivity of compliance with respect to the design variable. Should it include the negative sign in front of the expression? The current expression doesn’t make any sense because adding solid element will reduce the compliance. Thus, the sensitivity expression should have a negative sign.
Because the proposed method utilizes a filtering scheme, a filtered density would be between 0 (or minimum value) and 1. Does the element stiffness matrix incorporate the filtered density? If so, a proper citation and more explanation should be provided in the manuscript for readers.
The element sensitivity is not smoothed in general when a gradient-based optimization algorithm is used. What is the main reason for the smoothing sensitivity and significance of this process in the proposed method?
2.3. Adding and removing elements
What are the criteria to determine addition (recovery) or removal of a design variable in optimization? The detailed procedure of removing and adding elements should be discussed.
It states that “the elements which need removing are those ones with the minimum compliance, just like…” This sentence is not clear for the reviewer to understand the meaning of a removal criterion. The minimum compliance is an objective function in optimization measured globally. How is the minimum compliance in terms of an element measured?
2.4. Convergence criterion
The convergence criterion or optimization termination criterion in Eq.(8) is very typical in topology optimization. What is the unique characteristic of Huang and Xie [28]? The reference should be removed if the conventional convergence criterion is considered.
2.5. Evaluation
It states that “ to guarantee the engineering performance and the novelty of results,….” How is the novelty (and engineering performance) defined in the manuscript? Numerical results provided in Section 4 are just converged topologies with an objective function of compliance. The reviewer cannot find any relevant engineering performance or novelty comparisons to support the sentence.
What about the multiple constraints problems or non-volume constraints problems? The compliance with volume constraint is a toy problem in engineering. For practical applications of the proposed method, other types (buckling, stress, dynamic properties, etc.) of objective and constraint functions might be more relevant engineering consideration.
4. Cases and discussion
The convergence histories of objective and constraint functions for a numerical example are helpful for readers to understand the overall performance of the proposed method.
What is the stacked view? Is that an image of topology at a specific iteration of optimization? What is the significance of the stacked view associated with the numerical example?
There are so many approaches and methods in topology optimization using gradient-based optimization algorithms showing efficiency and promising results. When comparing to well-known methods, what are the main advantages or superior aspects of the proposed approach?
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
The authos' reply is sufficient to provide what the reviewer's questions.
The reply is better than what I expected.
Now, the reviewer think the manuscript deserves to be accepted.
Reviewer 3 Report
The authors have addressed all comments adequately.
Reviewer 4 Report
The reviewer's comments and suggestions were addressed in the revised manuscript. The manuscript has been also improved in terms of technical contents after revision. Although numerical examples in the revised manuscript are still focusing on small toy types of problems, the feasibility of the proposed method for finding converged designs of given examples is described. The overall efficiency of the proposed approaches still remains questionable. Considering the overall scope of the study and future research as well as the current development, the reviewer recommends this manuscript be accepted for publication. There is a typo (‘as in [27]’ line 126 that needs to be removed from the manuscript).
