MultiGNN: A Graph Neural Network Framework for Inferring Gene Regulatory Networks from Single-Cell Multi-Omics Data

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The authors propose a supervised Graph Neural Network (GNN) architecture for Gene Regulatory Networks (GRNs) Inference, characterised by the combination of information stemming from both gene expression and chromatin accessibility. The proposed architecture consists of two pathways characterised by two distinctive GNNs extracting features from both gene expression and chromatin accessibility inputs, and using a self-attention network to perform the combination of the features resulting from both pathways. The resulting representation is subsequently fed into a Multi-Layer Perceptron (MLP) to compute the score representing the probability of the existence of a link between the corresponding nodes/genes.

The work is well written and easy to follow. However, some points have to be improved in order to improve further the readability, as well as the reproducibility of the performed experiments:

1. To improve the manuscript's readability, the plots, tables, and graphical representations must be placed near the paragraphs where they are first referenced.
2. Many abbreviations are used throughout the manuscript without the corresponding terms being previously defined. Please first define the terms before using the corresponding abbreviations.
3. Please add some space between the word in a sentence and the following citation (e.g. DeepTFni[6] —> DeepTFni [6]). This has to be done throughout the whole manuscript.
4. Please correct X_R in line 152 (X_{R} serves as the value of the initial …) accordingly.
5. Please correct equation 6 (line 161) accordingly.
6. Please correct equation 7 (line 165) accordingly.
7. In the section Results, the authors mention: “ … The number of layers in MGCN is set to 2, with hidden layer sizes of 512 and 128, respectively, while the MLP's hidden          layer size is 256. The training epoch of the model is set to 200, and an exponentially decaying learning rate is used, with an initial rate of 0.001 and a decay of 80% every 20 training epochs. … ” Are these the results of the hyper parameter optimisation, thus representing the parameters used to perform the final inference task throughout the experiments? In that case, please state it explicitly in the manuscript in order to avoid any form of confusion.
8. Figure 2 has to be improved for the sake of readability.
9. For the results presented in Figure 2, the authors are depicting the average score after a total of 5 runs (I believe 5 runs of evaluation on the corresponding test set). What about the variance of the results? This should be interesting, since it would depict the stability of the performance of the proposed architecture (by using a box-plot for example).
10. Please improve the quality of Figure 3 (it is barely readable).
11. Furthermore, the results depicted in Figure 3 (A) do not correspond to the described ablation study performed in Sub-Section 3.3: the authors claim to perform experiments including gene expression data only, chromatin accessibility only, and the fusion of both data types. However, Figure 3 (A) depicts the performance of the multimodal model with different forms of feature aggregation (attention fusion, concatenated fusion, and added features (unweighted sum)). The reported results can therefore not be taken into consideration. This should be corrected, and the results corresponding to the ablation study should be depicted and commented.
12. In Sub-Section 3.5, are each of the mentioned parameters set to a specific value, while the target parameter is varied (e.g.: Fixed number of hidden layers and hidden layer size; Varying sizes of H_R and U_P)? What are the values of the fixed parameters in this case? Wouldn't a three-dimensional representation of the hyperparameters in combination with the inference performance be a better representation of the impact of these parameters on the performance of the approach? Please be more specific about the experiments conducted and described in this section.  

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 2 Report

Comments and Suggestions for Authors

Please find the comments attached.

Comments for author File: Comments.docx

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 3 Report

Comments and Suggestions for Authors

The authors introduce MultiGNN, a supervised framework using graph neural networks for gene regulatory network (GRN) inference. Their key innovation is the integration of both gene expression and chromatin accessibility data. This addresses the limitations of existing methods that typically use only gene expression and results in high false positive rates. They also benchmarked MultiGNN against several state-of-the-art methods across seven multi-omics datasets, demonstrating superior performance. This manuscript is suitable to publish in Computation journal. The main areas for improvement are the presentation of table, figures, and some grammar issues. Specific comments are lay out below:

• Section 3.1 Experimental setting should be moved to Method section. It would be nice to include a table for all the hyperparameters. It flows well with the model framework and activation function and other functions in the Method section.
• In Table 1, under the "Dataset" column, please consider grouping the datasets by "Human" "Mouse" and then listing the specific datasets under each group? The current formatting splits dataset names across lines, making the table difficult to read and follow.
• In Figure 2, please use a different color scale, the black number is hard to read with the black background.
• The labels in Figure 3 and Figure 5C are too tiny to read, please increase the text size or increase the whole figure size.
• In discussion section, could you please consider adding some discussions on the limitation of the random negative sampling method? Would this approach potentially introduce bias or potentially miss important non-regulatory pairs? And how might that affect the model’s performance and generalizability?
• There are some grammar issues, please correct those. For example, on line 263, it should be “Each model had its hyperparameters tuned on …”. Please keep the verb tense consistent. For example, revise the section 3.5 to past tense, so it will be consistent with the paragraphs in Section 3.6.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 4 Report

Comments and Suggestions for Authors

The authors presented an artificial neural network for inferring gene regulatory networks from combined data consisting of the single-cell gene expression data and chromatin accessibility information. Applying their model to seven published datasets, they demonstrated a better performance of the model compared to a set of existing AI instruments. The study exhibited an advantage of using multi-omics data for training new machine learning models. The model elaborated by the authors can be used in the community for direct application and further development. The manuscript is well written, the material is clear and concise. The conclusions made by the authors are well supported by the shown result. Therefore, I have only minor remarks:

 

• “Gene j” should be “gene i” in eq. (1)
• v_n should be v_N in line 125
• (10): The same N used for both the number of nodes (from line 125) and the number of TF-gene pairs.
• 3.1 should be moved to Materials and Methods.
• 3.1 title should reflect the fact that it’s about computational experiment.
• Figures 3–5: The text and labels on axes and other objects in the figures are hardly seen.

Author Response

Please see the attachment.

Author Response File: Author Response.docx