A Topological Approach to Enhancing Consistency in Machine Learning via Recurrent Neural Networks

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Many of the references cited are prior to 2022. To ensure the work reflects the latest advancements, please include more recent studies and developments in the field.

 

The citation format  is inconsistent. For example, in line 57, both “[12]” and “(2022)” are used.

 

It is unclear why the Methodology section (Section 2) is only one paragraph, while Section 3 immediately delves into the theoretical background. This creates confusion regarding where the actual method is presented. Is Equation (6) part of the theoretical background or the proposed method? 

 

The claim in lines 138–141 that no existing RNN modifications address consistency issues seems too broad. The references provided (2019–2021) are relatively dated, and there has likely been progress since then. Please review more recent literature to confirm or revise this claim.

 

GRU, introduced in 2014, is an older method. Please compare the proposed method with more recent RNN modifications.

 

Transformers are the current state-of-the-art models for processing sequences. A discussion on how the consistency issues addressed by your method compare to transformers is suggested.

 

Some writing issues:

Line 13, “Long Short Term Memory” —> "Long Short-Term Memory."

Line 18, “purposely” —> "intentionally" for formal writing

Line 87, "elaborated in detail" is redundant, —> “are elaborated."

Line 400, “Altough” —> “Although”, “is" —> "are"

Author Response

1-Many of the references cited are prior to 2022. To ensure the work reflects the latest advancements, please include more recent studies and developments in the field. 

The citation format  is inconsistent. For example, in line 57, both “[12]” and “(2022)” are used.

Dear Reviewer, 
We truly appreciate your comments regarding the citation dates and the minor formatting errors. In the revised manuscript, we have addressed these issues. Specifically, in the introduction and methodology sections, we have updated the references by selecting recent papers from leading machine learning conferences such as ICML and ICLR, as well as from reputable journals. 

2- It is unclear why the Methodology section (Section 2) is only one paragraph, while Section 3 immediately delves into the theoretical background. This creates confusion regarding where the actual method is presented. Is Equation (6) part of the theoretical background or the proposed method?  

Thank you for pointing this out. After working on the manuscript for a while, we decided to merge the theoretical background part with the methodology section, as we are explaining our solution in the theoretical background, which is part of the methodology. Additionally, we removed some well-known book knowledge, like the pure description of RNNs, and instead focused directly on the construction of our proposed approach. Equation (6) is directly related to the proposed RNN formula in the paper. We believe these changes made the paper more clear and concise.  

3- The claim in lines 138–141 that no existing RNN modifications address consistency issues seems too broad. The references provided (2019–2021) are relatively dated, and there has likely been progress since then. Please review more recent literature to confirm or revise this claim. 

In the revised manuscript, we have updated the references and included example papers that are directly related to changing the mathematical structure of RNNs, with publication dates after 2022. We also revised our claim in the manuscript to say, “very few of these works are explicitly constructed with the aim of achieving consistency in the predictions.” We provided the papers by Bonatti et al. and Wu et al. as examples of relevant research. 

4- GRU, introduced in 2014, is an older method. Please compare the proposed method with more recent RNN modifications. 

Thanks for your comment. We decided to make the relevant data and results from the different models publicly available in the revised manuscript. As for our preference for GRU, it aligns with the work of Bonatti and Mohr, who also compared their proposed RNN with GRU-based architectures. In their paper, they noted: “Most of the literature on RNNs relies on architectures called Long Short-Term Memory (LSTM) cells, and Gated Recurrent Units (GRUs).” GRUs are widely used, efficient, and effective, making them a solid benchmark for tasks involving sequential data. 

Additionally, we are planning a future study where we will conduct a literature review of recently introduced RNNs, such as the examples we provided in the paper, and test their ability to deliver consistent estimates. This will form the basis for another research paper. For this study, our primary focus was to introduce the proposed formula and compare it with well-known RNN architectures, leaving the exploration of newer RNN structures as a part of future work. 

5- Transformers are the current state-of-the-art models for processing sequences. A discussion on how consistency issues addressed by your method compare to transformers is suggested. 

In our research, we compared and evaluated 1D-CNN, GRU, LSTM, and Transformer-based NN architectures for their ability to provide consistent estimates, as well as their MSE results on the test sets. This information was already mentioned in the theoretical background section, where we stated: “Although all these sequential learning approaches were tested on the generated datasets, for reasons of brevity we only present and compare our results obtained with Consistent RNN (ConsRNN)-based NN architectures with the results from the Gated Recurrent Unit (GRU)-based NN architectures because this model outperformed others.”

We also shared the results related to Transformer-based architectures in the publicly available data. However, these architectures did not produce consistent predictions, which is directly related to their internal mathematical computational process. We believe this topic warrants further investigation and plan to explore it in a subsequent study.  

6- Some writing issues: 

Line 13, “Long Short Term Memory” —> "Long Short-Term Memory." 

Line 18, “purposely” —> "intentionally" for formal writing 

Line 87, "elaborated in detail" is redundant, —> “are elaborated." 

Line 400, “Altough” —> “Although”, “is" —> "are" 

We also appreciate the warning of these grammatical mistakes; in the revised manuscript we have fixed these mistakes. 

 

Reviewer 2 Report

Comments and Suggestions for Authors

·       The introduction section needs to be rewritten for better clarity and organization. It would be helpful to separate the research challenges, motivations, and contributions into distinct subheadings for a more structured presentation. The research challenges section should highlight the specific problems that the study aims to address. The motivations section should explain the significance of the study and why it is worth pursuing.

·       In the contributions section, the authors should clearly state what is novel about the study and its impact on the field.

·       A Related Work section should be added after the introduction to provide a literature review of recent studies in the field. The authors should critically assess recent works from reputable journals and conferences in the Related Work section. This section should identify the gaps in existing research that the current study seeks to address.

·       Citing prominent studies will help contextualize the paper within the existing body of knowledge.

·       The methodology section is not sufficiently detailed. The stages and methods used are unclear. The authors should create a main figure that outlines the overall methodology, providing a visual summary of the research process. This main figure should clearly depict the different stages of the methodology to improve the reader’s understanding.

·       The authors need to break down the methodology into clear substeps, explaining each step in detail. These improvements will help readers grasp the full scope of the research approach and how it was carried out.

·       The data used in the study is not clearly defined. The authors should specify the type of data used (e.g., experimental, simulated, or real-world data). It is important to include a detailed description of the data source, including how the data was collected and its relevance to the research. The authors should cite the data source to ensure transparency and allow readers to assess the validity of the findings.

·       A clear discussion of the data will strengthen the paper by providing context for the results and conclusions.

Comments on the Quality of English Language

Needs minor english editing. 

Author Response

1- The introduction section needs to be rewritten for better clarity and organization. It would be helpful to separate the research challenges, motivations, and contributions into distinct subheadings for a more structured presentation. The research challenges section should highlight the specific problems that the study aims to address. The motivations section should explain the significance of the study and why it is worth pursuing.

Dear Reviewer,

We truly appreciate your comments regarding the introduction. In the revised manuscript, we have rewritten the introduction to clearly outline the research challenge, the problem we aim to solve, and the motivation behind our work. Our main goal is to address the consistency problem from a broader perspective rather than keeping it application-specific, as it often is in material science and engineering-related surrogate applications. To this end, we proposed an RNN formula that can provide consistency across a wide range of applications, which we demonstrated using both synthetic and real-world datasets. Additionally, we have updated the references, including more recent works from high-level journals and pure machine learning conferences in the first paragraph.

2- In the contributions section, the authors should clearly state what is novel about the study and its impact on the field.

We also appreciate this comment and have added a summary of the paper's contributions, listed one by one, at the end of the introduction. The main novelty of our work is that we aimed to introduce an RNN approach that can provide consistent estimates in a wide range of engineering applications, rather than being specific to one application. In the revised manuscript, we have explicitly stated this in the last paragraph.

3- A Related Work section should be added after the introduction to provide a literature review of recent studies in the field. The authors should critically assess recent works from reputable journals and conferences in the Related Work section. This section should identify the gaps in existing research that the current study seeks to address.

In the revised manuscript, we added a 'Related Works' section and provided several examples of works related to 'consistency'. We have clearly outlined the current gap in existing literature and the challenge we intend to address. In particular, we focused on two significant research studies from the fields of materials science and engineering that are directly related to this problem. In order to highlight their importance, we created separate sections for each of them and reviewed them in detail, explaining the gaps in the existing research.

4- Citing prominent studies will help contextualize the paper within the existing body of knowledge.

Again, thanks for this comment. In the revised manuscript we specifically tried to be careful on this subject. We cited reputable journals, and specifically investigated two notable works from the field.

5- The methodology section is not sufficiently detailed. The stages and methods used are unclear. The authors should create a main figure that outlines the overall methodology, providing a visual summary of the research process. This main figure should clearly depict the different stages of the methodology to improve the reader’s understanding.

Thanks for this comment. In the revised manuscript, we created a figure illustrating the overall research steps, and the process we followed during the conducted study. Additionally, we merged the methodology section with the theoretical background and increased the details. We also updated the figures to improve clarity and enhance understanding of the theoretical explanations.

6- The authors need to break down the methodology into clear substeps, explaining each step in detail. These improvements will help readers grasp the full scope of the research approach and how it was carried out.

In the revised manuscript, we created subsections based on the steps we followed during the research. We organized each explanation separately, focusing on experiments, the theoretical background, and the construction of our proposed approach.

7- The data used in the study is not clearly defined. The authors should specify the type of data used (e.g., experimental, simulated, or real-world data). It is important to include a detailed description of the data source, including how the data was collected and its relevance to the research. The authors should cite the data source to ensure transparency and allow readers to assess the validity of the findings.

In the revised manuscript, we cited and provided links to the research papers that serve as sources for the real-world datasets used in this study. For the Forming Limit Curves (FLCs), we referenced our previous publication and the related data. For the stress-strain predictions, we utilized data generated in the research work of Bonatti and Mohr, and we included a link to their work as well. The generation of the synthetic dataset is explained in detail for each 1D, 2D, and 3D space in the Experiments section.

8-A clear discussion of the data will strengthen the paper by providing context for the results and conclusions.

We also appreciate this comment. For the real-world datasets used in our study, we have already shared their publication sources, so we chose to give their details briefly in the current study. For the synthetic datasets, we have already provided detailed explanations of their generation in the Experiments section.

Reviewer 3 Report

Comments and Suggestions for Authors

This study addresses the problem of ensuring consistency in recurrent neural network (RNN) models such as LSTM and GRU when discretizing continuous events into time series data for sequence prediction. It presents a novel RNN transition formula designed to provide robust and consistent estimators across domains, tested on both synthetic datasets (1D, 2D, and 3D) with high nonlinearity and complexity and real-world datasets. The method overcomes the limitations of traditional RNN architectures, enhancing their applicability to complex time series problems. Strengths of the study include its robust validation process and potential applicability across various engineering domains. However, its reliance on synthetic datasets for initial validation may limit generalizability, and further testing on real-world domain-specific datasets is needed to confirm its broader effectiveness.

My comments and recommendations:

1.       The review of existing research is poorly done. It needs to be significantly expanded and unresolved issues specified. It is also necessary to clearly formulate the research aim.

2.       The key section "Methodology" is written in one paragraph without graphic and mathematical support! This is unacceptable and incorrect! I suggest that this section either be completely rewritten or combined with another.

3.       What is the essence and novelty presented in the section "Theoretical Background"? If this section presents well-known scientific provisions, then what is the point of completely rewriting them in a scientific article? The authors need to work radically with this section.

4.       The results of the research are presented qualitatively, there is a lot of graphic material. However, it is unclear what is their novelty? Why are these results obtained by traditional methods needed?

5.       The conclusions must be completely rewritten, taking into account the specificity of the results obtained.

6.       The references must be expanded by 2 times and formatted in accordance with the MDPI requirements.

The article requires serious and careful revision and significant adjustments.

Author Response

1-The review of existing research is poorly done. It needs to be significantly expanded and unresolved issues specified. It is also necessary to clearly formulate the research aim.

Dear Reviewer,
We truly appreciate your comments regarding the review of the existing research. In the revised manuscript, we have added a related works section with a more detailed literature review, along with an updated introduction. At the end of the introduction, we clearly stated the current gap in the literature and outlined our contribution with the following sentences: “In this study, we explore consistency within RNNs, inspired by recent works of \cite{BONATTI2022104697, WU2024116881}. While these studies focus on consistency in RNNs for constitutive modeling, we believe the problem requires further investigation, particularly considering the purely mathematical nature of the problem. The aim of this work is to introduce a specialized RNN formula that provides consistent estimates across various real-world applications. Our proposed approach introduces a simple, but effective RNN formula that delivers consistent estimates across different real-world applications. We demonstrate the effectiveness of our approach through experiments on both synthetic and real-world datasets.”

2-The key section "Methodology" is written in one paragraph without graphic and mathematical support! This is unacceptable and incorrect! I suggest that this section either be completely rewritten or combined with another.

Thank you for this relevant comment. In fact, Reviewer 1 had the same comment. In the revised manuscript, we merged the theoretical background section with the methodology part, and we excluded the unnecessary parts. Additionally, we also added one diagram as Figure 1, that shows the big picture regarding the steps in our research. We believe that your comment helped us to improve the general flow of the paper and clarify the aim.  

3-What is the essence and novelty presented in the section "Theoretical Background"? If this section presents well-known scientific provisions, then what is the point of completely rewriting them in a scientific article? The authors need to work radically with this section.

Thank you for pointing this out. We acknowledge that presenting well-known scientific provisions does not add any significant contribution to our research. Therefore, we have rewritten this part completely. In the revised manuscript, we use the Bonatti, and Mohr’s closed conditional statement, Eq. (2), as a foundational model which ensures self-consistency in RNN predictions. In the remaining explanation, we have highlighted and clearly stated our proposed formula that ensures consistency in the estimates. 

4-The results of the research are presented qualitatively, there is a lot of graphic material. However, it is unclear what is their novelty? Why are these results obtained by traditional methods needed?

We included the results from traditional methods, specifically the Gated Recurrent Unit (GRU), to demonstrate its inability to provide consistent predictions. Our primary reason for presenting GRU results lies in its mathematical structure, which does not support consistent estimates. Despite its broad application in surrogate modeling within materials science, this limitation highlights the novelty and importance of our proposed approach. By contrasting our method with GRU, we aim to emphasize that our developed RNN formula delivers consistent estimates.

5-The conclusions must be completely rewritten, taking into account the specificity of the results obtained.

We appreciate this comment as well. In the revised manuscript, we have completely rewritten the conclusion. Specifically, we have interpreted and highlighted the results by carefully examining each figure and have worked to draw a stronger conclusion.

6-The references must be expanded by 2 times and formatted in accordance with the MDPI requirements.

Thank you for the suggestion. In updating the manuscript, we went through some additional literature and more recent investigations, which were also added to paper.  As a result, the number of citations increased to 40. We specifically gave citations from the most recent works in literature from high level journals, and conferences. 

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

The paper was significantly improved.

Reviewer 2 Report

Comments and Suggestions for Authors

The authors have addressed my given comments and suggestions. 

Comments on the Quality of English Language

English is fine and understandable. 

Reviewer 3 Report

Comments and Suggestions for Authors

The authors have taken into full account the reviewer's comments and criticisms and have substantially revised the article. Therefore, I recommend that the academic editor consider accepting it for publication.