Comparing Activation Functions in Machine Learning for Finite Element Simulations in Thermomechanical Forming

Round 1

Reviewer 1 Report

This paper studies on machine learning methods for thermomechanical Forming problems with an emphasis on the comparison of various activation functions. Compared with the conventional finite element methods that use complex mathematical-physical models, the proposed neural network technique can play important roles of experimental data. The paper can be considered to be accepted in the journal. However, some essential revisions need to be made as follows:

(1) A section or paragraph about the data used in the experiment is needed, which should clearly describe the data size, data format, source of data, how to create the data. Based on the data set, the readers would be able to evaluate whether the NN models developed are appropriate. 

(2) Also, the training data set and tested data set need to be described clearly. We note the equation (15) in page 9, N means both the training data and predicted data. This is not clear. In general, the whole data is divided into two groups, the training set and test set; the former is used to train the model, and the latter is used to test the generalization error of the model. Only the error of training set is not enough to evaluate the model because of great possibility of overfitting. 

(3) The network structure used in this paper is simple and shallow, but the training time is too long. The training time depends on the data and network structure. This should be explained clearly. Only the focus on the activation function is enough, and the novelty is not enough. The network depth and width need to be tested also. Moreover, the other deep learning methods, e.g., random forest, can be tested. 

Author Response

Thank you for your comments on this article proposal for Algorithms. Please find the answers to your questions in the attached PDF file. The manuscript has been updated in accordance with your requests.

Author Response File: Author Response.pdf

Reviewer 2 Report

Application of artificial neural networks (ANN) has a high potential in the computational material science. The author of the paper " Comparing Activation Functions in Machine Learning for Finite Element Simulations in Thermomechanical Forming" has developed several models for the calculation of steel resistance during the sheets’ forming. The constructed models have shown good accuracy. The authors have shown that the ANN based model with Swish activation function has the best accuracy with an average error of about 0.9 %. The obtained results show the advantage of the ANN-based model in comparison with analytical models and potential application for finite element simulation. The paper is well written and may be accepted for publication after the modification accordingly following comments:

1.                 How was chosen the number of neurons in hidden layers? It is known that number of neurons should be linked with the number of training data (10.3390/met12030447).

2.                 It is known that ANN have satisfactory described the data inside in the range of the training input parameters. However, overtrained ANN has a bad predictability out of the range. The author should demonstrate a workability of the constructed ANN at the deformation parameters that are out of the experimental data. It is recommended to provided calculated stress – strain curves for the temperature 1000 °C, strain rate 10 1/s and for the strain up to 2.

3.                 Figure 1 may be removed. The experimental stress-strain curves are presented in Figure 6.

4.                 Minor correction:

-                     The chemical composition of the P20 steel grade should be added to the manuscript.

Author Response

Thank you for your comments on this article proposal for Algorithms. Please find the answers to your questions in the attached PDF file. The manuscript has been updated in accordance with your requests.

Author Response File: Author Response.pdf

Reviewer 3 Report

1. The mathematical formulations of the ANN architecture must be explicitly described. The current manuscript solely outlines the general form of the flow law, which is already widely known, and may not necessitate an introduction. This information is also insufficient.

 2. Six activation functions are employed to assess the equivalent plastic strain. However, there is inconsistency in the numerical values defined by the colors in the legend, which can mislead readers. It is recommended to ensure uniformity in the values associated with the same color. Please revise both the plot and the legend in Figure 12 accordingly.

 3. The number of hidden layers holds a pivotal role in the architecture of artificial neural networks (ANN). It is advisable to discuss the accuracy of selecting the number of hidden layers and provide a convergence analysis for these layers.

 4. The results indicate that among the six activation functions utilized, the Swish function yields the best results in terms of solution quality, while the ReLU function performs the worst. Please offer further insights and explanations for these outcomes.

 5. This study employs finite element simulation. It is recommended to specify the number of elements required to achieve the desired level of accuracy.

 6. Figure 2 displays a two-hidden-layer ANN architecture with three input neurons. However, the current manuscript does not introduce the concept of an entry layer. Is the entry layer equivalent to the input layer? Please ensure consistency and clarify this terminology.

 7. In Figure 6, which presents a comparison between experimental data (dots) and the flow stress predicted by the ANN, ensure uniformity in the y-axis values across all subplots. Additionally, enhance the visibility of the subplot related to the activation function.

 8. Figure 7 illustrates the Python function used for computing flow stress and the derivative vector. Given that this figure contains code, it is suggested that it be relocated to the appendix for a more appropriate placement. Please make corresponding revisions to the entire manuscript.

 9. At line 536, a grammar error is present. For instance, "Can can note that" should be comprehensively corrected throughout the manuscript to "We can note that." Please execute the necessary revisions.

 10. The manuscript should incorporate a dedicated discussion section. One of the primary motivations of this study is to investigate the behavior and impact of different activation functions on the results. Provide a detailed analysis of the behavior of these activation functions and, notably, suggest a recommended activation function.

Author Response

Thank you for your comments on this article proposal for Algorithms. Please find the answers to your questions in the attached PDF file. The manuscript has been updated in accordance with your requests.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

It can be accepted.

Reviewer 2 Report

The author has answered previous comments and improved the manuscript. The paper may be accepted for publication.

Reviewer 3 Report

The authors have revised the manuscript in response to the previous comments. It is recommended that the current manuscript be accepted.