Net-Net AutoML Selection of Artificial Neural Network Topology for Brain Connectome Prediction

Round 1

Reviewer 1 Report

The manuscript presents an automatic machine learning based method to assist the selection of ANN topology. Overall, this manuscript is well structured. However, the potential value of this technique is still unclear. The authors needs to add more information in the introduction section and compare with other studies in the discussion.

In the Introduction section, the authors should provide more detail descriptions on the meaningful of this work. It seems that this work only focus on a specific case of a BCN. The model uses the Shannon entropy information as an input. It seems that only 20 features are included. How to choose these features? Is it enough since the 500,470 examples are used? Could the authors show which features are more important for decision-making? For example, the heatmap. The authors used five statistical based machine learning classifiers to predict the best ANN topology. How about the well-developed support vector machine (SVM) classifier or other neural networks? It would be better to show the visualization of high-dimension dataset. For example, to use t-SNE to visualize the feature distribution. The methodology about the Net-Net AutoML model is not clear to the reviewer. Do the authors mean that AutoML is decided from the five classifier? If so, it cannot be stated in this way. It’s important to show how the AutoML model works.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 2 Report

The method presented in the paper is described terribly, only figure 1 gives some insight. First of all, how are ANN topologies generated (upper part of the figure)? Why do authors write about ANNs? If I understand well, only topologies are analyzed, so, using ANNs in the paper may be confusing. Then, we have Shannon entropies, but how they are used to train classifiers is not given. What is input to classifiers, what are their parameters, what is their output, how they are evaluated, etc. Because a reader does not know how the method works it is difficult to assess its results.

"In the first step, the connectivity matrix L was obtained by downloading public resources data or by searching  the data about the links for the nodes of the matrix. An n by n matrix was generated, with n vertices). In the next step,  a Markov matrix Π was derived from L by calculating the vertices probability (pij)." - It seems that matrix L alone is insufficient to generate Markov matrix. Let's suppose situation with a network including 3 nodes, numbered from 1 to 3, the connections are from node 1 to 2 and from 1 to 3. If the network is an ANN, a signal coming from node 3 goes simultaneously to both remaining nodes that is to node 2 and 3. In other case, for example in the case of a natural system, the signal goes either to node 2 or to node 3,  not to both nodes. In both cases, Markov matrix will be different.

Even though I am not a native speaker I noticed many language errors, for example (only from the very beginning of the paper):  "large number of links to be study", it seems that it should be "studied", "Brain Connectome Networks (BCN) results from", rather  "result from", "to predict properties complex biological systems", "of" seems to be missing, "There many examples of applications....",  where is "are" - English should be carefully improved

"nodes (neurons) and links (functions)" - in ANNs, neurons represent functions, not links

"Artificial Neural Networks (ANNs) are powerful bio-inspired algorithms " - ANN is not an algorithm and as such it cannot learn anything by itself, an external learning algorithm has to be used for that purpose

"ML algorithms; i.e., different ANN topologies" - algorithm is not a topology

Equations no. (1) and (2) - there is no explanation of applied letters, what is G, I suppose it is graph but is should be clearly given in the paper. What is more, style of presentation of both equations is different, the bottom equation is printed in italics,  whereas the upper one is not.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

No more comments

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 2 Report

1. Introduction is unclear, it includes unnecessary details which make it difficult to understand what the article is about.
1. line 31, "BNCs" instead of "BCNs"
2. line 32, the same error
3. line 56 "could be use", rather "could be used"
4. line 60 abbreviateion ML is not explained
5. line 71, "In this context, this Automated Machine Learning", second "this" is redundant
6. line 81, "was able help us", rather "was able to help us"
7. line 84, "deep fully connected neuronal network", rather "...neural network"
8. There are many other language errors in the further part of the paper, which means that it should be carefully spell-checked.
Despite the fact that I suggested spell-checking in the review, the authors did nothing with it.
9. line 129, "The dataset was used to train 10 different ANNs with STATISTICA software" -
which learning algorithm was used?, what topology of ANNs was apllied (inputs, outputs, hidden nodes, whether the topology was set manually)?
what is input to ANNs, what are their parameters, what is their output, how they are evaluated, etc. - this information is still missing
which means that my remark from review has not been taken into account!!!!
10. line 170, next remark from the review has not been taken into account - my review:
"What is more, style of presentation of both equations is different, the bottom equation is printed in italics, whereas the upper one is not."
this remark was applied to eq. 1 and 2 which again are presented differently
11. line 173, "five Machine Learning classifiers from scikit-learn (python) have been tested" - not five methods,
12. Table 1 - Why do networks have a different number of inputs?, Again, what is input/output to networks? What is LNN?

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Round 3

Reviewer 2 Report

In order for every reader to be able to understand the method and evaluate the results obtained, ANNs which are a key component of the method should be properly described, at least the inputs and outputs.