A Helping Hand: A Survey About AI-Driven Experimental Design for Accelerating Scientific Research

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This review discusses the application of AI-driven experimental design in accelerating scientific research in chemistry and materials science, using techniques like active learning and optimization.

 

The comments of the article are shown as below:

Comment 1：Issues in the Abstract

• Some expressions are too general or vague; for example, when “different tasks” is mentioned, it does not specify what these tasks entail, thereby reducing the precision of the conveyed information.
• The language is somewhat verbose and lacks the necessary conciseness and emphasis on key points, failing to adequately showcase the paper’s innovative aspects and research contributions.

 

Comment 2：Issues in the Introduction

• Although the introduction introduces emerging technologies such as large language models, its connection to “experimental design” is not sufficiently tight, which may leave readers questioning the actual background and significance of the research problem.
• Some sentence structures are overly long and the logical transitions are not rigorous enough, resulting in a somewhat cluttered narrative that does not establish a clear, focused discussion.
• The review of existing literature and the analysis of its shortcomings are not in-depth enough, lacking critical reflection on the issues and specific perspectives on future research directions.

 

Comment 3：Improving the rigor and comprehensiveness of the full text

• Despite listing multiple databases and formulating keyword combinations, the selection of keywords may not be comprehensive enough, missing relevant literature and thus affecting the representativeness and breadth of the included studies.
• The review mentions a current lack of deep learning-based frameworks but does not further explore the reasons behind this phenomenon or discuss how to utilize deep learning to address complex, high-dimensional data issues in experimental design, resulting in less specific guidance for future research directions.
• There is some overlap in the classification of AI technologies (such as optimization, active learning, supervised learning, reinforcement learning), and sometimes the same technology may be categorized into different classes. This blurs the boundaries between methods, affecting the depth of comparative and critical analysis of each technique's advantages and disadvantages.
• Although various methods and their applications are described in detail, there is insufficient discussion on the limitations, application conditions, and challenges in actual experiments for each technique. The suggestions for improving existing methods are relatively general, lacking specificity and depth.
• Some paragraphs contain repetitive statements and lack compact formatting, affecting the overall reading fluency and the rigor of the paper.
• The article employs filtering standards such as FC1–FC4, which may be overly subjective in defining "specific solutions" and "whether AI technology is used." This could lead to the omission of literature with exploratory or novel ideas.
• It is suggested that more details could be given on how to collect and process spatiotemporal data in actual power systems, especially on the specific selection and use of state variables and topology structures in power flow optimization. This is very important for readers to understand why GNN is used.

Comments on the Quality of English Language

It is recommended to optimize the expression  to make the information more precise and easier to read.

Author Response

Reviewer 1:

 

Comment 1：Issues in the Abstract

 

• Some expressions are too general or vague; for example, when “different tasks” is mentioned, it does not specify what these tasks entail, thereby reducing the precision of the conveyed information.

 

Answer: Thank you very much for your advice, we specified vague expressions and revised the abstract accordingly!

 

• The language is somewhat verbose and lacks the necessary conciseness and emphasis on key points, failing to adequately showcase the paper’s innovative aspects and research contributions.

 

Answer: Thank you very much for your advice, we tried to clarify the paper's innovative aspects and research contribution. We further checked the manuscript for formulations and descriptions that needed refinement or clarification.

 

 

Comment 2：Issues in the Introduction

 

• Although the introduction introduces emerging technologies such as large language models, its connection to “experimental design” is not sufficiently tight, which may leave readers questioning the actual background and significance of the research problem.

 

Answer:

Thank you very much for your hint, we revised the introduction to clarify the background and significance of the research problem!

 

 

• Some sentence structures are overly long and the logical transitions are not rigorous enough, resulting in a somewhat cluttered narrative that does not establish a clear, focused discussion.

 

Answer:

Thank you very much for your hint, we tried to shorten sentences in order to achieve a clear and focused discussion!

 

 

• The review of existing literature and the analysis of its shortcomings are not in-depth enough, lacking critical reflection on the issues and specific perspectives on future research directions.

 

Answer:

Thank you very much for your hint, we revised the review of existing literature and differentiation to existing literature!

 

 

Comment 3：Improving the rigor and comprehensiveness of the full text

 

• Despite listing multiple databases and formulating keyword combinations, the selection of keywords may not be comprehensive enough, missing relevant literature and thus affecting the representativeness and breadth of the included studies.

 

Answer:

Thank you very much for your response, the combination of terms for the search query was derived experimentally. We investigated different terms and combinations, leading to exclusion of relevant studies or to addition of non-relevant publications. The used search query includes all publications found to be relevant for this literature while it minimizes non-relvant studies. We modify the search strategy section from line 130 to make that clearer!

 

• The review mentions a current lack of deep learning-based frameworks but does not further explore the reasons behind this phenomenon or discuss how to utilize deep learning to address complex, high-dimensional data issues in experimental design, resulting in less specific guidance for future research directions.

 

Answer:

Thank you very much for your hint, we extended the exploration and discussion of the lack of deep learning-based frameworks more in the discussion section!

 

• There is some overlap in the classification of AI technologies (such as optimization, active learning, supervised learning, reinforcement learning), and sometimes the same technology may be categorized into different classes. This blurs the boundaries between methods, affecting the depth of comparative and critical analysis of each technique's advantages and disadvantages.

 

Answer:

Thank you very much for your hint, we distinguish frameworks based on functional aspects rather than explicitly distinguishing methods. We revised the categorization from line 264 in order to make that clear!

 

 

• Although various methods and their applications are described in detail, there is insufficient discussion on the limitations, application conditions, and challenges in actual experiments for each technique.

 

Answer:

Thank you very much for your hint, we added a discussion about limitations and challenges in the discussion chapter!

 

 

• The suggestions for improving existing methods are relatively general, lacking specificity and depth.

 

Answer:

Thank you very much for your advice, we tried to modify the discussion chapter to address the issue of to deepen the suggestion of possible improvements!

 

 

• Some paragraphs contain repetitive statements and lack compact formatting, affecting the overall reading fluency and the rigor of the paper.

 

 

Answer:

Thank you very much for your hint, we revised the text, to avoid the unnecessary repetition of statements!

 

 

• The article employs filtering standards such as FC1–FC4, which may be overly subjective in defining "specific solutions" and "whether AI technology is used." This could lead to the omission of literature with exploratory or novel ideas.

 

Answer:

Thank you for your hint, we clarify the definition of the filter criteria, particularly from line 189 and 200 in order to make the filtering more objective.

Reviewer 2 Report

Comments and Suggestions for Authors

1:the manuscript introduces the abbreviation "SDL" for "self-driving laboratory" upon its first mention but inconsistently reverts to the full term ("self-driving laboratories") in subsequent discussions (e.g., "...referred to as a 'self-driving laboratory (SDL)' [4]. ... Various self-driving laboratories...").

2:The term "active learning" is abbreviated as "AL" in tables and subsequent sections (e.g., Table 3, Section 4.2), but the abbreviation is not explicitly defined upon its first appearance in the abstract.

3:The search strategy's reliance on "artificial intelligence" as the primary keyword may limit the methodological breadth of the review, as critical studies using terms like "machine learning," "deep learning," or "automated experimentation" in domains such as biomedicine or environmental science could be underrepresented.

4:In Table 8, Genetic Algorithms (GA) are categorized under both "Optimization" (as a standalone method) and "Reinforcement Learning (RL)" (as a component of Multi-period RL). This creates ambiguity in distinguishing between optimization algorithms and RL techniques, blurring their conceptual boundaries.

Author Response

Reviewer 2:

 

1. the manuscript introduces the abbreviation "SDL" for "self-driving laboratory" upon its first mention but inconsistently reverts to the full term ("self-driving laboratories") in subsequent discussions (e.g., "...referred to as a 'self-driving laboratory (SDL)' [4]. ... Various self-driving laboratories...").

 

Answer:

Thank you very much for this hint, we change it!

 

2. The term "active learning" is abbreviated as "AL" in tables and subsequent sections (e.g., Table 3, Section 4.2), but the abbreviation is not explicitly defined upon its first appearance in the abstract.

 

Answer:

Thank you very much for pointing out the missing term, we fixed this issue!

 

 

3. The search strategy's reliance on "artificial intelligence" as the primary keyword may limit the methodological breadth of the review, as critical studies using terms like "machine learning," "deep learning," or "automated experimentation" in domains such as biomedicine or environmental science could be underrepresented.

 

Answer:

Thank you very much for your response, we investigated different terms and combinations for the search query and the proposed search query leads to the best trade-off of including all relevant publications found during the literature search and minimizing non-relevant. Therefore, we revised the search strategy section in order to make that clear and added a short discussion in the discussion chapter!

 

4. In Table 8, Genetic Algorithms (GA) are categorized under both "Optimization" (as a standalone method) and "Reinforcement Learning (RL)" (as a component of Multi-period RL). This creates ambiguity in distinguishing between optimization algorithms and RL techniques, blurring their conceptual boundaries.

 

Answer:

Thank you very much for your hint, we distinguish frameworks based on functional aspects rather than explicitly distinguishing methods. We revised the categorization from line 264 in order to make that clear!

Reviewer 3 Report

Comments and Suggestions for Authors

1. The final list of 22 papers is small for a full-length survey (only 10% of initially considered documents). While this is justified (niche topic) and well explained in Table 2, it should be framed more critically. This should be included in the discussion: (Is the field too narrow, or is the query too restrictive?)

2. It would be valuable for the paper to add visual meta-analytic summaries.

3. The document briefly mentions explainable AI. Thus, concepts like explainability and trust in AI could be emphasized more.

• Are any reviewed models explainable (e.g., interpretable surrogates)?

4.  Expand the discussion on the limitations of current techniques (Data sparsity, Scalability of optimization methods)

5. Add a short paragraph about ethical considerations, especially in fully autonomous experimentation.

Author Response

Reviewer 3:

 

1. The final list of 22 papers is small for a full-length survey (only 10% of initially considered documents). While this is justified (niche topic) and well explained in Table 2, it should be framed more critically. This should be included in the discussion: (Is the field too narrow, or is the query too restrictive?)

 

Answer: Thank you very much for your hint, we explained the development of the search query more in detail in and added a short discussion of the niche-restriction problematic in the discussion chapter!

 

2. It would be valuable for the paper to add visual meta-analytic summaries.

 

Answer:

Thank you very much for your hint, we extended our table-based meta-analysis through a diagram in figure 5 and 6.

 

3. The document briefly mentions explainable AI. Thus, concepts like explainability and trust in AI could be emphasized more. Are any reviewed models explainable (e.g., interpretable surrogates)?

 

Answer:

Thank you very much for the great advice, we discussed the explainability part in the discussion chapter further!

 

4. Expand the discussion on the limitations of current techniques (Data sparsity, Scalability of optimization methods)

 

 

Answer:

Thank you very much for your hint, we extended the discussion chapter by a discussion of the mentioned limitations!

 

5. Add a short paragraph about ethical considerations, especially in fully autonomous experimentation.

 

Answer:

Thank you very much for your great idea, we added a short discussion about ethical considerations in the discussion chapter!

Reviewer 4 Report

Comments and Suggestions for Authors

In this survey,  a search and filter strategy is developed and applied on appropriate databases with the objective of identifying relevant literature on the application of AI methodology in the experimental design process. The results show a predominance of applications of AI-driven experimental design in chemistry and material science.

 

This survey is generally comprehensive. There are some comments:

 

1. The abstract mentions the application advantages of AI driven experimental design in chemistry and materials science. So what about in other fields (such as biology, medicine, etc.).

 

2. Further exploration is needed on the specific definition of 'generalizability' used in Table 4.

 

3. The criteria for distinguishing "partly autonomous" and "fully autonomous" frameworks in the filtering strategy are unclear.

Author Response

Reviewer 4:

 

1. The abstract mentions the application advantages of AI driven experimental design in chemistry and materials science. So what about in other fields (such as biology, medicine, etc.).

 

Answer:

Thank you for your hint, we adjusted the abstract and the domain section from line 252 in order to clarify that the five categories of domains are based on the domains of the included contributions! Other domains are not found in the relevant contributions.

 

2. Further exploration is needed on the specific definition of 'generalizability' used in Table 4.

 

Answer:

Thank you very much for your hint, we added a definition of the term “generalizability”

 

3. The criteria for distinguishing "partly autonomous" and "fully autonomous" frameworks in the filtering strategy are unclear.

 

Answer:

Thank you very much for your feedback, we adjusted the degree of automation section, to make the differentiation between fully autonomous, partly autonomous and supportive frameworks comprehensible!

Reviewer 5 Report

Comments and Suggestions for Authors

The Introduction needs improvement. I suggest combining Sections 1 and 2 to make it more concise and clear. There are too many paragraphs, which hinders readability.

Some paragraphs in this section consist of only one sentence. Ideally, paragraphs should be longer and sentences shorter.

The main research objective is unclear. I recommend aligning it explicitly with the objective presented in the Abstract.

The manuscript also requires a minor English revision. Throughout the text, different terms refer to the same concept. For example, the study is sometimes referred to as "this work" and other times as "this survey."

Regarding the term “survey,” the authors should avoid using it unless interviews or questionnaires were actually conducted. In this case, it would be more appropriate to use “research.”

The Methodology section provides a good description of the research steps, but it is confusing, poorly written, and lacks details about the databases used for the search. I recommend three improvements:

1. Add a figure at the beginning of Section 3 that summarizes all methodological steps.

2. Revise the text to avoid single-sentence paragraphs.

3. Simplify the structure to improve readability.

The Results section seems to present all the inferences drawn from the analysis of the articles.

In the Conclusion, the first paragraph restates the methodology using different words. The following paragraphs provide only superficial inferences. I expected a clear response to the main hypothesis stated in the title — that AI can accelerate scientific research. What did the research actually conclude on this matter? How can AI support and speed up research? The conclusion must include a deeper discussion of the findings and clearly explain why this article deserves to be published. Who can benefit from it, and how?


Please see attached for more details.

Comments for author File: Comments.pdf

Author Response

Reviewer 5:

 

1. The Introduction needs improvement. I suggest combining Sections 1 and 2 to make it more concise and clear. There are too many paragraphs, which hinders readability. Some paragraphs in this section consist of only one sentence. Ideally, paragraphs should be longer and sentences shorter.

 

Answer:

Thank you very much for your hint, we combined chapter one and two and modify the text in order to make sentences shorter and paragraphs longer!

 

1. The main research objective is unclear. I recommend aligning it explicitly with the objective presented in the Abstract.

 

Answer:

Thank you very much for your advice, we tried to make our research objective clearer in the introduction and align it with the goal stated in the abstract, to achieve in total a comprehensive review!

 

2. The manuscript also requires a minor English revision. Throughout the text, different terms refer to the same concept. For example, the study is sometimes referred to as "this work" and other times as "this survey." Regarding the term “survey,” the authors should avoid using it unless interviews or questionnaires were actually conducted. In this case, it would be more appropriate to use “research.”

 

Answer:

Thank you very much for your hint, we replaced the terms “survey” and “work” with the terms “research” and “study”

 

 

3. The Methodology section provides a good description of the research steps, but it is confusing, poorly written, and lacks details about the databases used for the search. I recommend three improvements:

• Add a figure at the beginning of Section 3 that summarizes all methodological steps.
• Revise the text to avoid single-sentence paragraphs.
• Simplify the structure to improve readability.

 

Answer:

Thank you very much for your advice, we added the figure 2 to make the methodology more comprehensible as well as change the structure of the methodology chapter, by avoiding single sentence paragraphs and simplifying sentences!

4. The Results section seems to present all the inferences drawn from the analysis of the articles.

 

Answer:

Thank you very much for your feedback!

 

5. In the Conclusion, the first paragraph restates the methodology using different words. The following paragraphs provide only superficial inferences. I expected a clear response to the main hypothesis stated in the title — that AI can accelerate scientific research. What did the research actually conclude on this matter? How can AI support and speed up research? The conclusion must include a deeper discussion of the findings and clearly explain why this article deserves to be published. Who can benefit from it, and how?

 

Answer:

Thank you very much for your feedback, we revised the conclusion, focusing on the main hypothesis and explaining the findings as well as who benefits from our research and how!

Reviewer 6 Report

Comments and Suggestions for Authors

This paper presents a comprehensive survey on the use of artificial intelligence in experimental design, highlighting how AI tools are transforming the scientific research process. The topic is timely and relevant, given the increasing interest in AI-assisted scientific discovery. 

- The paper occasionally lacks a clear structure within individual sections. Consider breaking down longer paragraphs and using more subheadings to guide the reader through different themes and technologies.

- While the paper includes a good number of references, some recent advancements in AI-assisted laboratory automation and design of experiments (DoE) in 2023–2024 are missing. Including those would strengthen the state-of-the-art coverage.

- The survey reads more like a summary of existing work rather than a critical synthesis. It would benefit from a more analytical discussion on the strengths and limitations of current AI models and methods.

- The paper briefly mentions ethical issues but does not delve into them. A deeper discussion on reproducibility, bias in data, and transparency of AI systems in scientific settings would add important value.

- The conclusion is somewhat generic. Consider summarizing key insights and outlining concrete future research directions or open challenges.

Author Response

Reviewer 6:

 

1. The paper occasionally lacks a clear structure within individual sections. Consider breaking down longer paragraphs and using more subheadings to guide the reader through different themes and technologies.

 

Answer: Thank you very much for your hint, we tried to simplify sentences and revised parts of the text structure!

 

2. While the paper includes a good number of references, some recent advancements in AI-assisted laboratory automation and design of experiments (DoE) in 2023–2024 are missing. Including those would strengthen the state-of-the-art coverage.

 

Answer:

Thank you very much for your feedback, we aimed to include all literature reviews found during our systematic literature research about AI-driven experimental design as our scope lies here. Can you specify which publications are meant by that? We would like to include your mentioned references

 

 

3. The survey reads more like a summary of existing work rather than a critical synthesis. It would benefit from a more analytical discussion on the strengths and limitations of current AI models and methods.

 

Answer: Thank you very much for your advice, we made our survey more analytical by adding several aspects like limitations of current AI methods and discussed them in the discussion section!

 

4. The paper briefly mentions ethical issues but does not delve into them. A deeper discussion on reproducibility, bias in data, and transparency of AI systems in scientific settings would add important value.

 

Answer: Thank you very much for your hint, we added ethical issues in the discussion section and integrated reproducibility, bias in data and transparency!

 

5. The conclusion is somewhat generic. Consider summarizing key insights and outlining concrete future research directions or open challenges.

 

Answer: Thank you very much for your advice, we revised the conclusion chapter by summarizing key points, outlining concrete future research directions and open challenges!

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

Thank you for your careful response. After conducting a second-round review, I  found that it demonstrates a certain level of innovation and academic value. The overall quality of the paper is good. 

Reviewer 4 Report

Comments and Suggestions for Authors

There are no more comments. This version can be accepted.

Reviewer 5 Report

Comments and Suggestions for Authors

Thank you for replying to all my comments

Reviewer 6 Report

Comments and Suggestions for Authors

After the second review, I believe the authors have carefully addressed all the comments. I would strongly recommend the publication of this paper.