Next Article in Journal
Evaluation of Keratin/Bacterial Cellulose Based Scaffolds as Potential Burned Wound Dressing
Next Article in Special Issue
Towards Bio-Hybrid Energy Harvesting in the Real-World: Pushing the Boundaries of Technologies and Strategies Using Bio-Electrochemical and Bio-Mechanical Processes
Previous Article in Journal
Reduction of Pesticide Use in Fresh-Cut Salad Production through Artificial Intelligence
Previous Article in Special Issue
Controllable Height Hopping of a Parallel Legged Robot
 
 
Article
Peer-Review Record

Wankelmut: A Simple Benchmark for the Evolvability of Behavioral Complexity

Appl. Sci. 2021, 11(5), 1994; https://doi.org/10.3390/app11051994
by Thomas Schmickl 1, Payam Zahadat 2 and Heiko Hamann 3,*
Reviewer 1: Anonymous
Reviewer 2: Anonymous
Reviewer 3: Anonymous
Appl. Sci. 2021, 11(5), 1994; https://doi.org/10.3390/app11051994
Submission received: 18 November 2020 / Revised: 29 January 2021 / Accepted: 10 February 2021 / Published: 24 February 2021
(This article belongs to the Special Issue Biorobotics and Bionic Systems)

Round 1

Reviewer 1 Report

The authors in this paper propose the “Wankelmut” task as a benchmark to test algorithms for evolutionary robotics. The result shows that none of the fitness functions proposed was capable to evolve the desired Wankelmut behaviour in the robots. The authors claim that this is because the evolved neural networks lack modularity.

The idea of creating a benchmark task for evolutionary robotics algorithms is good. However, I have two major issues:

1. The paper needs proofreading and the results layout need changing.

2. The authors need to justify different important claims across the paper.

More details below:

1.

The paper has typos, styling and grammatical issues all across the paper. Here’s a shortlist of some of them:

- Line 73: suggests → suggest

- Line 138: neural networks that are evolved → evolved neural networks

- Line 140: two example hand-coded neural networks → two examples of hand-coded neural networks.

- Line 144: Figure 3b, has → Figure 3b has

- Line 151: Figure 11b, 11c → Figure 11b and 11c

- Line 157: This equation is cited a couple of times. It needs to be by itself. Also in italics.

- Line 157: “We had 11 neurons...”. This sentence is not clear

- Line 163: “In a second” → In the second

- Line 259: “We were” → We use

- Line 275: environment, like → environment like

- Line 278: in the wanted way: A post-hoc → in the wanted way. A post-hoc

- Line 306: This sentence is not clear.

- Line 322: This way → In this way.

Figures 4-10 take too much space. All the first rows from these figures (box plots) can be summarized in a table. The labels of each figure are repetitive and they do not contribute to the paper. This can be improved by summarizing the results shown in each figure with one sentence.

The paragraph in line 104 does not seem to contribute to the core message of the paper and it can be deleted.

I do not find Figure 2 necessary. This can be described in the main text instead.

I am unsure Section 4.3 is needed.

Figures 12-15 are not discussed in the main body of the paper. It is only mentioned in line 370.

2.

- Across the paper, the authors claim that the EA can create “cheap tricks”. I feel this term should not be used. While it is true that EA can generate unexpected results, it does not necessarily mean that they are “cheap tricks”. It has been proven that “cheap tricks” can outperform that design of humans. Therefore, they should not be “prevented” (line 319) and more like addressed and studied. If the behaviour produced by “cheap tricks” is incorrect then this means that there is something missing in the experimental design. It is not clear to me why this term is used.

- I am confused towards the authors’ claim that the algorithms in this paper failed to evolve the Wankelmut behaviour because they were not pre-informed. However, the structures chosen for NN and CTRNN follow the same structure for the hand-made architectures. To me, their explanation on the failure of achieving Wankelmut behaviour is unreasonable.

- In different sections of the paper (Abstract, Introduction and Conclusion), the authors claim that modular structures of neural networks are needed to solve the task. In my opinion, there is not enough evidence to support this. The authors could have used the algorithms mentioned in paragraph 417 to prove this hypothesis.

Author Response

Many thanks for this very elaborated and very helpful review. We have tried to address all raised points carefully and have tried to explain ourselves where we choose to disagree.

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

The article is interesting by approaching the evolutionary calculus, but it is necessary a comparison with other algorithms of this type for example Game of Life. Also, I found on the Internet an almost identical article, written by you, from 2016, and the results presented are identical.
What is the difference between the results obtained in 2016 and those presented in the 2020 proposal. In my opinion, this aspect must be commented.

Author Response

Many thanks for your review. We have tried to answer your comments carefully.

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

In this paper, the authors are proposing a discussion in the field of artificial intelligence, evolutionary robotics, and artificial life. It is an ongoing and exciting discussion. Even though it sounds interesting, I would like to point some observations:
1) The title of the paper is not clear about its purpose neither its abstract. However, after reading, it sounds like the authors are presenting new insights for evolutionary robotics. Why not let this clear?
2) This paper is not new. There is a 2016 version available on Researchgate (https://www.researchgate.net/publication/308646686_Sooner_than_Expected_Hitting_the_Wall_of_Complexity_in_Evolution). My question is: Why the authors are submitting the same article (or a little variation of that) and presenting nothing new after four years?
3) Argument in lines (34-36) should be supported by a reference.
4) Line 40, the expression says "most studies in literature". How many studies were analyzed to support this conclusion? Why not cite those?
5) The discussion and results are not clear. It seems there is more work to be done.
6) Line 397, "We speculate that learning the downhill behavior "destroys" the already learned network structure for performing the uphill walk behavior." Is it possible to have a scientific conclusion instead of an opinion or guessing?
7) This study should bring other works to be compared or contrasted
8) The authors argued that (lines 454, 455) "Maybe this would then be not "evolutionary computation" anymore but rather "artificial evolution" a real valid, yet still simple model of natural evolution."
In this case, it seems to me that more rigorous and objective methodologies are lacking. At this point, I suggest approaching this issue like a new theory and bring all formal and scientific elements to this.

Author Response

Many thanks for this very elaborated and very helpful review. We have tried to address all raised points carefully and have tried to explain ourselves where we choose to disagree.

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 3 Report

The authors addressed all comments and suggestions. The final manuscript, in my perception, can bring more discussion to the studied field.

Back to TopTop