Optimizing GNN Architectures Through Nonlinear Activation Functions for Potent Molecular Property Prediction
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsAreen Rasool et al. used a new activation function, the sinusoidal linear unit (SLU), in the graph neural network (GNN) in the hope of improving its performance in molecular property prediction. Overall, the article is innovative, but it still has shortcomings in the following aspects, and further revisions are recommended:
1. The types of tasks tested are not sufficient: the article only uses two datasets, FreeSolv and QM9, to evaluate the performance of SLU, and the types are relatively single, only involving regression tasks. Whether classification tasks should also be added for better evaluation. And although SLU performs well in most experiments, in some cases (for example, the performance of the GCN model on the QM9 dataset), SLU fails to outperform traditional activation functions. Compared with FreeSolv, QM9 has a richer amount of data. Whether SLU cannot show its advantages on larger datasets.
2. At present, the article only experiments on two GNN architectures (GIN and GCN). Although these two models are relatively classic, their performance on different GNN architectures (such as GAT) has not been verified. It is recommended to further expand the experiment to explore the performance of SLU on more types of GNN architectures. And the model does not explore whether SLU can have a positive impact on the model's performance in graph network models with different layers.
3. The current article mainly compares SLU with activation functions such as ReLU and Leaky ReLU. But it is not compared with common Sigmoid, tanh, Softplus, and Maxout. It is hoped that it can be compared with more common activation functions to fully demonstrate the advantages and disadvantages of SLU.
I hope the author can supplement and improve the above issues.
Author Response
Dear Editor:
The authors are very grateful to you and the reviewers for the valuable suggestions and comments concerning our manuscript. We have studied all the comments carefully and made corrections which we hope meet with approval. The main corrections in the paper and the responds to the reviewer’s comments are as follows:
Response to Reviewer-1
- The types of tasks tested are not sufficient: the article only uses two datasets, FreeSolv and QM9, to evaluate the performance of SLU, and the types are relatively single, only involving regression tasks. Whether classification tasks should also be added for better evaluation. And although SLU performs well in most experiments, in some cases (for example, the performance of the GCN model on the QM9 dataset), SLU fails to outperform traditional activation functions. Compared with FreeSolv, QM9 has a richer amount of data. Whether SLU cannot show its advantages on larger datasets.
Our response:
Thank you for your helpful feedback. In response to your suggestion to include classification tasks, we have added two more datasets, BACE and BBBP, both of which involve classification. SLU performed well on these datasets, showing its ability to improve results across both regression and classification tasks. As for SLU’s performance on the QM9 dataset with the GCN model, we acknowledge that it did not outperform traditional activation functions in some cases. In light of our keen interest, we have acknowledged your point as an exception and see it as a valuable opportunity for further investigation as well. Thank you for your insightful feedback!
- At present, the article only experiments on two GNN architectures (GIN and GCN). Although these two models are relatively classic, their performance on different GNN architectures (such as GAT) has not been verified. It is recommended to further expand the experiment to explore the performance of SLU on more types of GNN architectures.
And the model does not explore whether SLU can have a positive impact on the model's performance in graph network models with different layers.
Our response:
Thank you for your thoughtful feedback. We focused on the GIN and GCN architectures because both are well-established in the literature and have consistently shown strong results across various graph-based tasks, including molecular property prediction. These models are widely regarded as benchmarks in the field, and their success in both regression and classification tasks provides a solid evaluation of SLU’s effectiveness. Since GIN and GCN are standard models that capture the core principles of graph networks, we believe testing SLU on these architectures is sufficient to demonstrate its potential. Expanding to other architectures is not necessary, as these models already reflect the essential design aspects common in graph neural networks.
SLU is designed to enhance the property prediction task, removing it from any layer would likely lead to a degradation in performance. Our aim was to demonstrate the effectiveness of SLU in improving molecular property prediction, and maintaining it across all layers was crucial to achieving this goal. Therefore, we did not explore the effects of varying the layer configurations in this study, as our focus was on evaluating SLU's overall impact in a controlled manner.
- The current article mainly compares SLU with activation functions such as ReLU and Leaky ReLU. But it is not compared with common Sigmoid, tanh, Softplus, and Maxout. It is hoped that it can be compared with more common activation functions to fully demonstrate the advantages and disadvantages of SLU.
Our response:
Thank you for your helpful suggestion. We compared SLU with ReLU, LeakyReLU, ELU and SELU because these activation functions are the most commonly used in graph neural networks, especially for molecular property prediction. Their proven performance makes them suitable benchmarks for evaluating SLU.
While we recognize that activation functions like Sigmoid, tanh, Softplus, and Maxout are well-known, they tend to be less effective for GNN tasks due to issues like vanishing gradients or slower convergence. Some of these functions, such as Sigmoid and tanh, are mainly used for classification tasks, which makes them less relevant for our research. That’s why we focused on ReLU and others as they are better suited for the type of tasks we are investigating.
Thank you very much
Best regards!
Dr. Quaid Iqbal
Reviewer 2 Report
Comments and Suggestions for AuthorsComments on the manuscript computation-3257660 by Areen Rasool et al. titled “Optimizing GNN Architectures Through Nonlinear Activation Functions for Potent Molecular Property Prediction”:
The authors present an interesting study on applying graphical neural networks (GNNs) to predict molecular properties, an important task in computational chemistry and drug discovery. To improve GNN performance, they propose introducing a new activation function called Sin Linear Unit (SLU). The manuscript is well structured, the introduction is informative and detailed. The authors logically interpret the data, discuss the results obtained, and draw conclusions. The manuscript does not require improvement. The manuscript quality is adequate for scientific publication in Computations.
A minor point: The list of references cited should be one and include all sources cited.
Author Response
Dear Editor:
The authors are very grateful to you and the reviewers for the valuable suggestions and comments concerning our manuscript. We have studied all the comments carefully and made corrections which we hope to meet with approval. The main corrections in the paper and the responds to the reviewer’s comments are as follows:
Response to Reviewer-2
The authors present an interesting study on applying graphical neural networks (GNNs) to predict molecular properties, an important task in computational chemistry and drug discovery. To improve GNN performance, they propose introducing a new activation function called Sin Linear Unit (SLU). The manuscript is well structured, the introduction is informative and detailed. The authors logically interpret the data, discuss the results obtained, and draw conclusions. The manuscript does not require improvement. The manuscript quality is adequate for scientific publication in Computations.
A minor point: The list of references cited should be one and include all sources cited.
Our response:
Thank you for your positive feedback and kind words about our manuscript, “Optimizing GNN Architectures Through Nonlinear Activation Functions for Potent Molecular Property Prediction.” We’re glad to hear that you found the study interesting and well-structured.
Regarding your comment about the references, we have included all relevant references and cited them appropriately in the revised manuscript.
Thank you very much
Best regards!
Dr. Quaid Iqbal
