Development of Machine Learning Models to Evaluate the Toughness of OPH Alloys

Oxide Precipitation-Hardened (OPH) alloys are a new generation of Oxide Dispersion-Strengthened (ODS) alloys recently developed by the authors. The mechanical properties of this group of alloys are significantly influenced by the chemical composition and appropriate heat treatment (HT). The main steps in producing OPH alloys consist of mechanical alloying (MA) and consolidation, followed by hot rolling. Toughness was obtained from standard tensile test results for different variants of OPH alloy to understand their mechanical properties. Three machine learning techniques were developed using experimental data to simulate different outcomes. The effectivity of the impact of each parameter on the toughness of OPH alloys is discussed. By using the experimental results performed by the authors, the composition of OPH alloys (Al, Mo, Fe, Cr, Ta, Y, and O), HT conditions, and mechanical alloying (MA) were used to train the models as inputs and toughness was set as the output. The results demonstrated that all three models are suitable for predicting the toughness of OPH alloys, and the models fulfilled all the desired requirements. However, several criteria validated the fact that the adaptive neuro-fuzzy inference systems (ANFIS) model results in better conditions and has a better ability to simulate. The mean square error (MSE) for artificial neural networks (ANN), ANFIS, and support vector regression (SVR) models was 459.22, 0.0418, and 651.68 respectively. After performing the sensitivity analysis (SA) an optimized ANFIS model was achieved with a MSE value of 0.003 and demonstrated that HT temperature is the most significant of these parameters, and this acts as a critical rule in training the data sets.


Introduction
Developing new structural alloys for industrial applications requires a shared effort in the commercial sector and a movement towards a green environment. These efforts will thrive if industries use carbon emission-free, safe, and globally available energy sources. One of the primary challenges for structural materials is improving their mechanical properties, mainly focusing on ultimate tensile strength (UTS), elongation, and toughness [1]. The new generations of Oxide Dispersion-Strengthened (ODS) alloys and Oxide Precipitation-Hardened (OPH) alloys are promising candidates for industrial applications, due to their high strength, corrosion resistance, and toughness [2][3][4][5]. Based on the importance of the

Materials and Methods
The new OPH alloy is based on metal powders using powder metallurgy [34]. The main powders (Fe and Al) and other components (Cr, Mo, Ta, Y) are mechanically alloyed in a vacuum low energy ball mill developed by the authors. After sufficient milling, the MA powder is transferred to a low-alloy rolling container with no contact to the air, evacuated, and sealed by welding. Then, it is consolidated using a hot rolling mill in three steps. As first step, the container is rolled under a temperature of 900 • C and rolled to a thickness of 7.5 mm. In the same way, it rolled to thicknesses of 5 mm and 3.2 mm in the next two steps. All the steps have a rolling speed of 0.2 m/s. Finally, the OPH sheet with approximate thickness of 2.5 mm covered on both sides by a 0.3 mm thick scale from the rolling container is produced in this way.
In the current research, different variants of OPH varying in milling time, rolling temperature, and HT were developed to check the effect of each element on the toughness of final semi-products. All variants were produced in a similar way so that the comparison could be available through they could be compared using machine learning methods.
Standard tensile samples were cut for all the variants of the OPH alloys. A waterjet cutting system was used to cut the pieces in a longitudinal direction (parallel to the rolling direction), and then the samples are ground to a final thickness of 2 mm. The authors manufactured purpose-built clamps to hold the samples on the servo-hydraulic MTS machine. All the tensile tests were carried out with a strain rate of 1 × 10 −3 s −1 . Three samples were tested for each state, and the average values of the ultimate tensile strength (UTS) and elongation to failure (A) were statistically calculated. The stress-strain curve area was also calculated as toughness using in-house software developed by the authors. The toughness is measured by calculating the area under the stress-strain curve for the OPH alloys. The toughness is calculated from the tensile graph ( Figure 1) and based on the following formula: where ε is strain, ε f is the strain upon failure, and σ is stress. The following figure shows a schematic of the stress-strain curve of an OPH alloy and the desired surface area.

Materials and Methods
The new OPH alloy is based on metal powders using powder metallurgy [34]. The main powders (Fe and Al) and other components (Cr, Mo, Ta, Y) are mechanically alloyed in a vacuum low energy ball mill developed by the authors. After sufficient milling, the MA powder is transferred to a low-alloy rolling container with no contact to the air, evacuated, and sealed by welding. Then, it is consolidated using a hot rolling mill in three steps. As first step, the container is rolled under a temperature of 900 °C and rolled to a thickness of 7.5 mm. In the same way, it rolled to thicknesses of 5 mm and 3.2 mm in the next two steps. All the steps have a rolling speed of 0.2 m/s. Finally, the OPH sheet with approximate thickness of 2.5 mm covered on both sides by a 0.3 mm thick scale from the rolling container is produced in this way.
In the current research, different variants of OPH varying in milling time, rolling temperature, and HT were developed to check the effect of each element on the toughness of final semi-products. All variants were produced in a similar way so that the comparison could be available through they could be compared using machine learning methods.
Standard tensile samples were cut for all the variants of the OPH alloys. A waterjet cutting system was used to cut the pieces in a longitudinal direction (parallel to the rolling direction), and then the samples are ground to a final thickness of 2 mm. The authors manufactured purpose-built clamps to hold the samples on the servo-hydraulic MTS machine. All the tensile tests were carried out with a strain rate of 1 × 10 −3 s −1 . Three samples were tested for each state, and the average values of the ultimate tensile strength (UTS) and elongation to failure (A) were statistically calculated. The stress-strain curve area was also calculated as toughness using in-house software developed by the authors. The toughness is measured by calculating the area under the stress-strain curve for the OPH alloys. The toughness is calculated from the tensile graph ( Figure 1) and based on the following formula: where ε is strain, εf is the strain upon failure, and σ is stress. The following figure shows a schematic of the stress-strain curve of an OPH alloy and the desired surface area.
As explained above, fourteen different OPH alloys were prepared and tested to investigate the toughness as a part of the mechanical properties. Machine learning methods (ML) were used to predict the toughness obtained by tensile testing. Systems with different chemical compositions of OPH alloys (Al, Mo, Fe, Cr, Ta, Y, and O), heat treatment conditions, and mechanical alloying conditions were considered as the model inputs and toughness was set as the output. As explained above, fourteen different OPH alloys were prepared and tested to investigate the toughness as a part of the mechanical properties. Machine learning methods (ML) were used to predict the toughness obtained by tensile testing. Systems with different chemical compositions of OPH alloys (Al, Mo, Fe, Cr, Ta, Y, and O), heat treatment conditions, and mechanical alloying conditions were considered as the model inputs and toughness was set as the output.

Machine Learning Methods Procedure
Three models (ANN, ANFIS, and SVR) were developed to predict and simulate the toughness of OPH alloys. According to the experimental data set, the models were trained, and the performance of each model was evaluated. The accuracy and implementation of the constructed models were calculated by several mathematical errors such as Mean Square Error (MSE): The root mean square error (RMSE) was calculated by Equation. (2).
Mean Absolute Error (MAE) and the absolute fraction of variance (R 2 ) are measured by In these relations, n is the data numbers in training, t is experiment data, and o is predicted data. The above statistical errors are objective functions between experimental and simulated data. The value of R 2 ranges between 0 and 1. If a model results in an R 2 value near 1, it means a slight fluctuation between the experimental and predicted data. If a model results in an R 2 value close to zero, the most significant difference between the experimental data and the constructed model is explained [35].

Artificial Neural Network (ANN)
An ANN system built several biological neural structures, and its simple parallelism helps solve complex problems that could find suitable relationship parameters [36]. The ANN structure comprises three essential layers: input, hidden, and output layer. According to the experimental tests, twelve parameters influence the toughness of OPH alloys. As shown in Figure 1, twelve parameters were selected as input parameters, and the toughness of OPH alloys was set as the output.
In the ANN structure, there are several neurons in each layer to train the model. As shown in Figure 1, the neuron receives the signals from an input, and the neuron weights each input by a specific weight index (w). The sum of the weighted inputs represents the transfer function f (Σ wixi) and the bias (b). Each neuron in a layer is connected to the other neurons, and, finally, the input layer is connected to the output layer by nonlinear mapping. The signal data transfer between each neuron is converted by an activation function or a transfer function [37]. The training process has occurred in the hidden layer, and the performance of the ANN model is highly affected by this layer. The number of neurons in each hidden layer and the number of hidden layers have a significant role in the efficiency of an ANN model. Besides, other essential factors for constructing a suitable ANN structure are the transfer function and training algorithm [33]. In this study, one hidden layer is used as a suitable framework for an ANN structure.
The number of neurons in the hidden layer significantly influences the performance and complexity of the ANN model and helps to avoid underfitting or overfitting. Several topologies were built at various neurons in the hidden layer to determine the optimum number of neurons in the hidden layer. The Levenberg-Marquardt backpropagation (LMBP) training algorithm has been used extensively by researchers in the past and is widely regarded as the best training algorithm [33]. LMBP is used to find a suitable number of neurons. A five-neuron hidden layer architecture was chosen because it resulted in lower error values ( Figure 2). The hyperbolic tangent sigmoid transfer function and Purlin transfer function were used for the output in the hidden layer. In this research, we used the ANN toolbox in MATLAB software. Before building an ANN structure, the data set was ranged in a normalized range. According to the data set, 103 pieces of data were collected from experimental tests and were used to construct the models. Typical structure for the ANN model used in this study [32].

Support Vector Regression (SVR)
The regression method is an efficient method to find the best relationship between the dependent and independent variables. The polynomial regression model may generate in different forms such as vector of random errors, response vector, parameter vector, and design matrix. Nonlinear correlations are found between each variable in polynomial regression models [41]. The support vector machine (SVM) algorithm is one of the most robust and suitable methods for simulating linear and nonlinear data sets [42]. In this research, the linear regression toolbox in MATLAB with the SVM algorithm was developed. As discussed earlier, we used twelve variables as input and toughness as an output to simulate the experimental data set.  [32].
In order to train the data set using the ANN model, the input and output values are scaled within the normalized range (before presenting the data). The normalization method improves the ability of simulation and accuracy of the training process. Therefore, all of the data values were set between 0.0 to 1.0. The normalized values (Xnorm) can be calculated by the following equation [33]: where X is the actual value, Xmin is the minimum value and Xmax is the maximum value of the data set. Table 1 presents the range of values for the variables in this study.

Adaptive Neuro-Fuzzy Inference Systems (ANIFS)
In ANFIS modeling, artificial neural networks and fuzzy system design are mixed, resulting in a robust predictive model [33]. As shown in Figure 3, the ANFIS model comprises five layers that build the fuzzy structure using the "if-then" method and employs the Takagi-Sugeno fuzzy system [38]. In this study, we used the ANFIS toolbox of MATLAB, in which 80% of the total data set was used for training, and 20% of the entire data set was applied for the testing step. Due to the higher efficiency of subtractive clustering (SC) in creating the ANFIS structure, we used this method to generate a fuzzy model [39]. Typical parameters in the SC method are range from influence (RI) and squash factor (SF), which are usually manually changed to build a suitable ANFIS model with an optimal structure [40]. The RI factor ranges from 0.1 to 1 and SF changes from 1 to 7 to find the best ANFIS structure. The number of membership functions is a function of the sub-clustering parameter values, and the number of input membership functions (MF) represents the number of rules. The input and output MF were set as gauss and linear for all structures.

Support Vector Regression (SVR)
The regression method is an efficient method to find the best relationship between the dependent and independent variables. The polynomial regression model may generate in different forms such as vector of random errors, response vector, parameter vector, and design matrix. Nonlinear correlations are found between each variable in polynomial regression models [41]. The support vector machine (SVM) algorithm is one of the most robust and suitable methods for simulating linear and nonlinear data sets [42]. In this research, the linear regression toolbox in MATLAB with the SVM algorithm was developed. As discussed earlier, we used twelve variables as input and toughness as an output to simulate the experimental data set.

Support Vector Regression (SVR)
The regression method is an efficient method to find the best relationship between the dependent and independent variables. The polynomial regression model may generate in different forms such as vector of random errors, response vector, parameter vector, and design matrix. Nonlinear correlations are found between each variable in polynomial regression models [41]. The support vector machine (SVM) algorithm is one of the most robust and suitable methods for simulating linear and nonlinear data sets [42]. In this research, the linear regression toolbox in MATLAB with the SVM algorithm was developed. As discussed earlier, we used twelve variables as input and toughness as an output to simulate the experimental data set.

The Analysis of the Constructed Models
As discussed in the modelling section, to find the suitable structure of the ANN model, five neurons in a hidden layer with the same transfer function were selected for training the data set. Table 2 shows the comparison between the ANN models with different training algorithms. The results demonstrated that the ANN-4 model had the lowest MSE, MAE (459.22, 15.75), and a higher coefficient of determination than the other models (0.86). Table 3 shows the modeling results for ANFIS constructed models. The higher the RI values, the higher the error; thus, the lower RI value is more suitable for the ANFIS model. Similarly, as the SF factor's value rises, the performance of ANFIS decreases the optimal value of the SF factor which was 1. As a result, ANFIS-SC8 provided a more suitable arrangement with the lowest RMSE value (RMSE = 0.20). Table 2. MSE, MAE, and R 2 of the five ANN models executed with five neurons in the hidden layer architecture to find the best training algorithm.

ANN Models
Training  The best model is shown in italics.
The optimum ANN structure with five neurons in a hidden layer was selected (e.g., 12-5-1). The LM training algorithm provided a more suitable performance of the ANN model. Consequently, the ANN model for predicting the toughness of OPH alloys has MSE and MAE values of 459.22 and 15.75. Figure 4a shows the correlation between the experimental and predicted values of toughness by the ANN model; the R 2 value for the ANN model is 0.86, which shows that ANN could find a suitable correlation between the variables.
For ANFIS modeling, several fuzzy clustering structures were developed, and ANFIS-SC8 resulted in higher accuracy. The values of MSE and MAE for the ANFIS model were 0.0418 and 0.0517. The comparison between experimental and simulated toughness values for the ANFIS model is shown in Figure 4b

Analysis of the Validity and Performance of the Constructed Models
In this section, various criteria were calculated to evaluate the performance and accuracy of each model. As shown in Table 4, several formulae such as R, k, k′, Ro 2 , and R′o 2 were presented based on previous research [43][44][45][46]. In these relations, hi and ti represent the observed output and predicted output. Furthermore, the permissible value for the criteria is presented. According to the results, all the models are suitable for predicting the toughness of OPH alloys, and the models fulfilled all the desired performance criteria. Several criteria validated the fact that the ANFIS model results in better conditions and better ability in simulation.

Prediction of the Toughness of OPH Alloys
Ten datasets were collected for the test to compare the models, and these ten datasets were not used to train the models. Moreover, the test datasets were randomly selected to eliminate the problem of the influence of human selection on the results. As shown in

Analysis of the Validity and Performance of the Constructed Models
In this section, various criteria were calculated to evaluate the performance and accuracy of each model. As shown in Table 4, several formulae such as R, k, k , R o 2 , and R o 2 were presented based on previous research [43][44][45][46]. In these relations, hi and ti represent the observed output and predicted output. Furthermore, the permissible value for the criteria is presented. According to the results, all the models are suitable for predicting the toughness of OPH alloys, and the models fulfilled all the desired performance criteria. Several criteria validated the fact that the ANFIS model results in better conditions and better ability in simulation.

Prediction of the Toughness of OPH Alloys
Ten datasets were collected for the test to compare the models, and these ten datasets were not used to train the models. Moreover, the test datasets were randomly selected to eliminate the problem of the influence of human selection on the results. As shown in Table 5, these ten datasets (e.g., T1 to T10) were reported. Figure 5 and Table 5 illustrate the comparison between the experimental values and the modeling results. It could be understood that the predicted values are compatible with experimental values. However, the ANFIS model exhibited a better and more reliable prediction performance than the ANN and the SVR models. Therefore, we can say that the ANFIS model is more efficient than the other models.  Table 5, these ten datasets (e.g., T1 to T10) were reported. Figure 5 and Table 5 illustrate the comparison between the experimental values and the modeling results. It could be understood that the predicted values are compatible with experimental values. However, the ANFIS model exhibited a better and more reliable prediction performance than the ANN and the SVR models. Therefore, we can say that the ANFIS model is more efficient than the other models. The models for testing data sets predict the value of the hardness of OPH alloy [32].

Sensitivity Analysis (SA) of Input Parameters
SA is a suitable technique for investigating the influence of each input on the toughness and for finding the significant input parameters [47]. The inputs were categorized into significant and non-significant parameters. It has been reported that if the non-effective parameters were removed, the accuracy and performance of the model could be enhanced [48]. As discussed in the previous section, the ANFIS model has better performance and accu-racy. Therefore, SA was performed on the ANFIS model. The R 2 , MSE, and MAE values for the SA of the ANFIS model are given in Table 6. The result demonstrates that the ANFIS model is sensitive to the input parameters, especially HT temperature. By removing the HT temperature, the R 2 decreases considerably from 0.99 to 0.54. Nevertheless, some input variables such as Fe, Mo, and Ta have a more negligible effect on the performance of the ANFIS model (according to Table 6). The HT duration, Rolling Temperature, and Milling time have a higher impact on the model accuracy, and the removal of these parameters causes a higher error of the model. Furthermore, the reduction of input parameters for Fe, Cr, and Y decreased the MSE from 0.04176 to 0.02615, 0.00171, and 0.06486 respectively. Moreover, the R 2 value increases and MAE decreases for these input parameters, which showed that removing these parameters could improve the accuracy of the ANFIS model. Therefore, it may be concluded that removing the input parameters results in an increase in MSE and MAE values for the ANFIS model. In particular, reducing HT temperature, HT duration, rolling temperature, milling time, and strain rate increases the error values (see Table 7). Thus, these five parameters are more effective in the performance of the predictive ANFIS model, and the model is more sensitive to these parameters. However, HT temperature is the most significant of these parameters, which acts as a critical rule in training the data sets. To enable better training of the ANFIS model, it can be optimized by applying the SA technique. Non-effective input parameters, including Fe, Cr, and Y, were removed. In that case, the ANFIS model was developed with nine inputs. Based on that, the predicted data sets versus the actual data obtained from the developed model are shown in Figure 6. It is obvious that the new model with the lowest input parameters achieved better performance with higher accuracy. Above that, by ignoring the non-sensitive input parameters, the overfitting of the model can be avoided and may reduce the complexity and nonlinearity of the data sets. The nature of the input data or configuration directly influences the accuracy of the constructed model [49,50]. By ignoring Fe, Cr, and Y, as non-sensitive parameters, the performance of the ANFIS model can be enhanced. Therefore, a new developed ANFIS model was formed to predict the toughness of OPH alloys with high accuracy.

Conclusions
Fourteen different OPH alloys were prepared by mechanical alloying from a mixture of powder components, consolidating, and hot rolling. A series of standard tensile tests were performed on different variants of the OPH alloys to investigate the toughness as a part of mechanical properties. Machine learning methods (ML) like ANN, ANFIS, and SVR models were used to predict the toughness obtained by tensile testing. Systems with different chemical compositions of OPH alloys (Al, Mo, Fe, Cr, Ta, Y, and O), heat treatment conditions, and mechanical alloying conditions were considered model inputs and toughness was set as output. The results showed that the proposed strategies can determine the complex behavior of the alloys with an approximate accuracy of 95% and can help the designer predict relevant uncertainties without using analytical calculations. A better understanding of chemical composition to achieve the optimum mechanical properties in a combination of the effectivity of the hybrid model proves the efficiency of the presented models. The value of MSE for ANN, ANFIS, and SVR models was 459.22, 0.0418, and 651.68. Several criteria validated the fact that the ANFIS model results in better conditions and better ability in simulation. Ten datasets were collected for testing the models and it was found that the predicted values are compatible with the experimental values. However, the ANFIS model exhibited a better and more reliable prediction performance. The outcome of SA revealed that the reduction of input parameters for Fe, Cr, and Y decreased the MSE from 0.04176 to 0.02615, 0.00171, and 0.06486, which showed that removing these parameters could improve the accuracy of the ANFIS model. Finally, an optimized ANFIS model was achieved with an MSE value of 0.003.