A Comparison Study of Constitutive Equation, Neural Networks, and Support Vector Regression for Modeling Hot Deformation of 316L Stainless Steel

In this research, hot deformation experiments of 316L stainless steel were carried out at a temperature range of 800–1000 °C and strain rate of 2 × 10−3–2 × 10−1. The flow stress behavior of 316L stainless steel was found to be highly dependent on the strain rate and temperature. After the experimental study, the flow stress was modeled using the Arrhenius-type constitutive equation, a neural network approach, and the support vector regression algorithm. The present research mainly focused on a comparative study of three algorithms for modeling the characteristics of hot deformation. The results indicated that the neural network approach and the support vector regression algorithm could be used to model the flow stress better than the approach of the Arrhenius-type equation. The modeling efficiency of the support vector regression algorithm was also found to be more efficient than the algorithm for neural networks.


Introduction
Stainless steel is an iron-based alloy that is widely used because it has greater wear and corrosion resistance than other alloys such as mild steels and low alloy steels [1]. Depending on the alloying contents used, stainless steel can have good strength and oxidation resistance in high temperature environments, as well as the desired properties for cryogenic environments and marine applications [1][2][3]. One of the main types of stainless steel is austenitic stainless steel, which contains Cr (12-25 wt.%), Ni (8-25 wt.%), and Mo (0-6 wt.%) and has good high temperature properties. Austenitic stainless steel has several different types and characteristics. For instance, stainless steel type 316 is a molybdenum-added alloy which has good resistance to pitting, high temperature creep, and high temperature oxidation [3]. Thus, this alloy is often applied in defence and nuclear applications [3]. Type 304 alloy contains about 18% Cr and about 8% Ni, and therefore is protected from the aggressive environment by passive films made of chromium oxides [4,5]. Therefore, because 304 stainless steel has good corrosion resistance, the alloy is widely used in the gas and the oil industries [4,5]. Meanwhile, due to its excellent oxidation and creep resistance in high temperature environments, stainless steel type 310S is used in nuclear power plants and chemical industries [6].
For the design and production of appropriate stainless-steel products, it is crucial that the deformation behaviors of the various types of stainless steel are thoroughly studied across a wide range of temperatures and strain rates, and a deformation model of the alloys should be developed. To date, many relevant studies have been conducted, including but not limited to a study on the deformation of 316 stainless steel under 0.1-100 s −1 of strain rate and 1173-1473 K temperature [7], a study on the compressive behavior of AISI 321 under the temperature range of 950-1100 • C and the strain rate of 0.01-1 s −1 [8], and a study on the compression deformation behavior of AISI 304 under the strain rate of 0.001-5 s −1 and the temperature of 800-1200 • C.

Materials and Methods
In the current study, hot tensile tests for 316-type stainless steel were done in the MTS-810 servo hydraulic material testing machine ( Figure 1). The planar specimens for the tests were prepared by wire-EDM machining. Figure 2a shows the dimensions of the specimen and Figure 2b shows the actually prepared specimen. The high-temperature tensile experiments were done at strain rates of 0.0002 s −1 , 0.002 s −1 , and 0.02 s −1 , at temperatures of 800 • C, 900 • C, and 1000 • C, for each strain rate.

Materials and Methods
In the current study, hot tensile tests for 316-type stainless steel were done in the MTS-810 servo hydraulic material testing machine ( Figure 1). The planar specimens for the tests were prepared by wire-EDM machining. Figure 2a shows the dimensions of the specimen and Figure 2b shows the actually prepared specimen. The high-temperature tensile experiments were done at strain rates of 0.0002 s −1 , 0.002 s −1 , and 0.02 s −1 , at temperatures of 800 °C, 900 °C, and 1000 °C, for each strain rate.  The measured true stress-true strain curves for temperatures of 800 °C, 900 °C, and 1000 °C, respectively, are shown in Figure 3a-c, under varying strain rates of 0.0002 s −1 , 0.002 s −1 , and 0.02 s −1 . Throughout the experiments, it is observed that flow stress is strongly dependent on the deformation strain rate and temperature. It is observed that as the temperature increases, the stress decreases; while as the stress increases, the strain rate increases.

Materials and Methods
In the current study, hot tensile tests for 316-type stainless steel were done in the MTS-810 servo hydraulic material testing machine ( Figure 1). The planar specimens for the tests were prepared by wire-EDM machining. Figure 2a shows the dimensions of the specimen and Figure 2b shows the actually prepared specimen. The high-temperature tensile experiments were done at strain rates of 0.0002 s −1 , 0.002 s −1 , and 0.02 s −1 , at temperatures of 800 °C, 900 °C, and 1000 °C, for each strain rate.  The measured true stress-true strain curves for temperatures of 800 °C, 900 °C, and 1000 °C, respectively, are shown in Figure 3a Throughout the experiments, it is observed that flow stress is strongly dependent on the deformation strain rate and temperature. It is observed that as the temperature increases, the stress decreases; while as the stress increases, the strain rate increases. The measured true stress-true strain curves for temperatures of 800 • C, 900 • C, and 1000 • C, respectively, are shown in Figure 3a Throughout the experiments, it is observed that flow stress is strongly dependent on the deformation strain rate and temperature. It is observed that as the temperature increases, the stress decreases; while as the stress increases, the strain rate increases. Materials 2020, 13, x FOR PEER REVIEW 4 of 13

Arrhenius Type Constitutive Modeling
In this study, the Arrhenius-type equation was employed to model the hot deformation of 316L stainless steel. The influences of temperature and strain rate on the flow behavior is represented by the Zener-Hollomon parameter, as shown below in Equation (1): Q represents the activation energy (J·mol −1 ); R represents the gas constant (8.314 J• mol K ); T denotes the temperature (K); and is the strain rate (s ; while A, ′, α, β, and represents material constants. First, substituting the first Equation (3) into (2) and taking logarithms of both sides gives Then, ′ can be obtained from the slopes of the lnσ − ln curves. For instance, Figure 4a shows the plot of lnσ − ln when the strain is 0.3.

Arrhenius Type Constitutive Modeling
In this study, the Arrhenius-type equation was employed to model the hot deformation of 316L stainless steel. The influences of temperature and strain rate on the flow behavior is represented by the Zener-Hollomon parameter, as shown below in Equation (1): Q represents the activation energy (J·mol −1 ); R represents the gas constant (8.314 J·mol −1 K −1 ); T denotes the temperature (K); and . ε is the strain rate (s −1 ); while A, n , α, β, and n represents material constants.
First, substituting the first Equation (3) into (2) and taking logarithms of both sides gives Then, n can be obtained from the slopes of the lnσ − ln . ε curves. For instance, Figure 4a shows the plot of lnσ − ln . ε when the strain is 0.3.  Further, substituting the second Equation (3) into (2) and taking logarithms of both sides gives: Similarly, β can be calculated from the slopes of the σ − ln curves. Figure 4b shows the σ − ln plot.
The material constant α is defined as follows: Therefore, α is obtained after calculating β and ′.
The material constant can be obtained after substituting the last Equation (3) into (2) and taking logarithms of both sides. From the equation shown below, n can be calculated using the slopes of the ln − ln sinh curves as: Figure 5a shows the plot of ln − ln sinh when the strain is 0.3. Similarly, the material constant Q can be calculated from the ln sinh − plot. Figure 5b shows the plot of ln sinh − . Using Equations (1) and (7), one can obtain the following equation: From Equation (8), the material constant A can be calculated by using the y-intercept of the lnZ − ln sinh plot, which is shown in Figure 5c. Further, substituting the second Equation (3) into (2) and taking logarithms of both sides gives: Similarly, β can be calculated from the slopes of the σ − ln .
ε curves. Figure 4b shows the σ − ln . ε plot. The material constant α is defined as follows: Therefore, α is obtained after calculating β and n . The material constant n can be obtained after substituting the last Equation (3) into (2) (1) and (7), one can obtain the following equation: From Equation (8), the material constant A can be calculated by using the y-intercept of the lnZ − ln[sinh(ασ)] plot, which is shown in Figure 5c.
After determining the material constants, one can obtain the equation describing the flow stress using Equations (1) and (2) in the form of the equation below. Table 1 shows the parameters obtained for the Arrhenius equation. After determining the material constants, one can obtain the equation describing the flow stress using Equations (1) and (2) in the form of the equation below. Table 1 shows the parameters obtained for the Arrhenius equation.  Figure 6 is a schematic diagram of the developed neural networks. Each of the neural networks has three layers, namely input, hidden, and output layers. In this study, two hidden layers were used in the developed model. Each layer has neurons. For example, the input layer has three neurons, which represent the temperature, strain rate, and strain. Neurons are processed in terms of their weight numbers. During the operation of the neural networks, the numbers received by a neuron from the neurons in the previous layer are processed according to the weight number of the current neuron. After the calculation, the calculated number is delivered to the neurons in the next layer in   Figure 6 is a schematic diagram of the developed neural networks. Each of the neural networks has three layers, namely input, hidden, and output layers. In this study, two hidden layers were used in the developed model. Each layer has neurons. For example, the input layer has three neurons, which represent the temperature, strain rate, and strain. Neurons are processed in terms of their weight numbers. During the operation of the neural networks, the numbers received by a neuron from the neurons in the previous layer are processed according to the weight number of the current neuron. After the calculation, the calculated number is delivered to the neurons in the next layer in order. In the current model, the final value is received by the output layer with one neuron, which represents the flow stress of hot deformation. The numbers of hidden layers and neurons in the hidden layers directly and significantly influence the accuracy of the prediction results. Gupta et al. [12] employed a single hidden layer with 15 hidden neurons. Li et al. [13] employed 18 hidden layers and Sabokpa et al. [14] employed 15 hidden neurons in a single hidden layer. In this research, the accuracy of the neural networks changed irregularly with the change of the numbers of hidden layers and neurons. Therefore, the ideal number of hidden layers and hidden neurons are decided after many simulations on a trial and error basis. In the current model, 250 neurons are used for each hidden layer. The activation function is the RELU (rectified linear unit) function. Training the neural network for good prediction accuracy involves obtaining the most appropriate set of weight values of neurons. For the current model, a back-propagation training algorithm with ADAM optimization was adopted. Mean square error is employed as a loss function for evaluating the performance of the neural networks during the training. Learning rate and weight decay are the parameters related to the amount being modified during the update of the weight of the neurons. In this research, 0.001 and 0.1 are chosen for learning rate and weight decay. In this research, the weight decay was set as 0.1 and the learning rate was set as 0.01. For the development of the neural networks, Keras (a Python package) was utilized. For the training data of the current model, 324 points of data were chosen randomly from nine stress-strain curves for the strain range from 0.05 to 0.45. Another 81 points of data from the same nine curves for the strain range from 0.05 to 0.45 were chosen to test the developed model. To train the neural networks, training data is normalized by the equation below: (10) and neurons. Therefore, the ideal number of hidden layers and hidden neurons are decided after many simulations on a trial and error basis. In the current model, 250 neurons are used for each hidden layer. The activation function is the RELU (rectified linear unit) function. Training the neural network for good prediction accuracy involves obtaining the most appropriate set of weight values of neurons. For the current model, a back-propagation training algorithm with ADAM optimization was adopted. Mean square error is employed as a loss function for evaluating the performance of the neural networks during the training. Learning rate and weight decay are the parameters related to the amount being modified during the update of the weight of the neurons. In this research, 0.001 and 0.1 are chosen for learning rate and weight decay. In this research, the weight decay was set as 0.1 and the learning rate was set as 0.01. For the development of the neural networks, Keras (a Python package) was utilized. For the training data of the current model, 324 points of data were chosen randomly from nine stress-strain curves for the strain range from 0.05 to 0.45. Another 81 points of data from the same nine curves for the strain range from 0.05 to 0.45 were chosen to test the developed model. To train the neural networks, training data is normalized by the equation below:

Neural Networks
Here, , is the normalized value of the ith index data; is the ith data; and and are maximum value and minimum value of data, respectively. Table 2 shows the process parameters for the developed neural network model.

SVR (Support Vector Regression)
SVR (support vector regression) [15] is an applied type of SVM (support vector machine) classification algorithm. In SVR, the data is split into groups by hyperplane. The distance from the hyperplane to the data at each group's boundary is called the margin, and the data on the marginal Here, x i, nor is the normalized value of the ith index data; x i is the ith data; and x max and x min are maximum value and minimum value of data, respectively. Table 2 shows the process parameters for the developed neural network model.

SVR (Support Vector Regression)
SVR (support vector regression) [15] is an applied type of SVM (support vector machine) classification algorithm. In SVR, the data is split into groups by hyperplane. The distance from the hyperplane to the data at each group's boundary is called the margin, and the data on the marginal boundary is called the support vector. The purpose of training the SVR is to find the optimal hyperplane with the largest margin. Figure 7 shows a schematic diagram of the support vector machine algorithm with the original linear hyperplane. For the training of the current data, the relationship of input data with the output data is nonlinear and complex. Therefore, a new hyperplane is induced to a feature space by the kernel method to help train the nonlinear dataset. A Python package (Scikit-learn) was used in the current study. The regularization parameter is chosen to be 5000, and the epsilon value is chosen to be 0.003. Table 3 lists the process parameters for the developed SVR model. machine algorithm with the original linear hyperplane. For the training of the current data, the relationship of input data with the output data is nonlinear and complex. Therefore, a new hyperplane is induced to a feature space by the kernel method to help train the nonlinear dataset. A Python package (Scikit-learn) was used in the current study. The regularization parameter is chosen to be 5000, and the epsilon value is chosen to be 0.003. Table 3 lists the process parameters for the developed SVR model.       Figure 9 shows the calculated true stress by the neural networks indicated on the measured stress for the temperatures of (a) 800 °C, (b) 900 °C, and (c) 1000 °C, when the strain rates are 0.0002 s −1 , 0.002 s −1 , and 0.02 s −1 . The RMSE of the true stress using the neural network model was found to be 2.85. The results indicate that the true stress using the developed neural network is more similar  Figure 9 shows the calculated true stress by the neural networks indicated on the measured stress for the temperatures of (a) 800 • C, (b) 900 • C, and (c) 1000 • C, when the strain rates are 0.0002 s −1 , 0.002 s −1 , and 0.02 s −1 . The RMSE of the true stress using the neural network model was found to be 2.85. The results indicate that the true stress using the developed neural network is more similar to the measured values than the stress calculated by the Arrhenius equation. However, substantial effort was needed to obtain an appropriate model with good process parameters. In addition, the neural network model was computationally more expensive.  Figure 10 shows the stress predicted using the support vector regression algorithm indicated with the measured stress at (a) 800 °C, (b) 900 °C, and (c) 1000 °C for the strain rates of 0.0002 s −1 , 0.002 s −1 , and 0.02 s −1 . The RMSE value for the stress by support vector regression was found to be 2.55. The support vector regression could predict the stress with similar accuracy as the neural network model, while being computationally less expensive.   Table 4 shows the RMSEs (root mean square error) for three algorithms. In this research, a study on relative percentage errors by three algorithms was done. Figure 11 shows the relative frequency of errors versus relative percentage error using (a) Arrhenius equation, (b) neural networks, and (c) the support vector machine algorithm. The plots (a), (b), and (c) for three algorithms are provided with mean value, and standard deviation of relative percentage errors as Equations (11) and (12):

Support Vector Regression
where μ is the mean value of the relative percentage errors and w is the standard deviation, δ is the relative percentage error, n is the number of samples, and i is the index of a sample.  Table 4 shows the RMSEs (root mean square error) for three algorithms. In this research, a study on relative percentage errors by three algorithms was done. Figure 11 shows the relative frequency of errors versus relative percentage error using (a) Arrhenius equation, (b) neural networks, and (c) the support vector machine algorithm. The plots (a), (b), and (c) for three algorithms are provided with mean value, and standard deviation of relative percentage errors as Equations (11) and (12): where µ is the mean value of the relative percentage errors and w is the standard deviation, δ is the relative percentage error, n is the number of samples, and i is the index of a sample.

3.
The support vector regression algorithm predicted the flow stress with similar RMSE as the neural network model. At the same time, developing the support vector regression model was efficient and computationally inexpensive.

4.
The neural networks and support vector regression algorithms predicted the flow stress more reliably than the Arrhenius equation. In addition, the performance of support vector regression was more reliable than the neural network. 5.
Although the neural network and the support vector regression algorithms calculated flow stress with better RMSE than the Arrhenius equation, the algorithms could not extrapolate and predict the flow stress outside the range of the training data.
Funding: This work was supported by the Soonchunhyang University Research Fund (no. 20181005).