Application of Machine Learning to Predict Grain Boundary Embrittlement in Metals by Combining Bonding-Breaking and Atomic Size Effects

The strengthening energy or embrittling potency of an alloying element is a fundamental energetics of the grain boundary (GB) embrittlement that control the mechanical properties of metallic materials. A data-driven machine learning approach has recently been used to develop prediction models to uncover the physical mechanisms and design novel materials with enhanced properties. In this work, to accurately predict and uncover the key features in determining the strengthening energies, three machine learning methods were used to model and predict strengthening energies of solutes in different metallic GBs. In addition, 142 strengthening energies from previous density functional theory calculations served as our dataset to train three machine learning models: support vector machine (SVM) with linear kernel, SVM with radial basis function (RBF) kernel, and artificial neural network (ANN). Considering both the bond-breaking effect and atomic size effect, the nonlinear kernel based SVR model was found to perform the best with a correlation of r2 ~ 0.889. The size effect feature shows a significant improvement to prediction performance with respect to using bond-breaking effect only. Moreover, the mean impact value analysis was conducted to quantitatively explore the relative significance of each input feature for improving the effective prediction.


Introduction
Segregation-induced changes in grain boundary (GB) cohesion are often the controlling factor limiting the mechanical properties of metallic alloys. A small amount of solute atoms may significantly alter fracture toughness and corrosion of metallic alloys by orders of magnitude [1][2][3]. To evaluate the strengthening or weakening effect of segregants on GB cohesion, one prevalent approach is to calculate the so-called strengthening energy or embrittling potency, ∆E SE , of a particular segregated impurity, which is the segregation energy difference between a GB and a fracture free surface (FS) using the Rice-Wang model [4,5]. The value of ∆E SE plays a key role in the GB embrittlement or strengthening because there is a positive correlation between ∆E SE and the experimental shift of ductile-to-brittle transition temperature (DBTT) that could be used for the design of new alloys [6]. During the last few decades, based on accurate first-principle calculations, an intensive effort has been focused on quantification of the segregation-induced changes of GB cohesion and a large amount of quantitative determine the factors and predict ∆E SE of solutes in different host metals. Our purpose is to develop fast and accurate models for prediction of ∆E SE and uncover the key features.

Methods
The dataset we use for training and testing contains 142 data points by DFT calculations collected from literature [8,19,25] and first principles calculations, see Supplementary Table S1. The set contains 5 hosts which are Ni, Fe, W, Al, and Mo. To build accurate and reliable machine learning models, it is important to include relevant features that collectively capture the trends in the ∆E SE across the different metals. Based on the input parameters of the above models and by taking into account the accessibilities of the parameters, we select the difference of cohesive energies (∆C) between the host and the segregated solute atoms, the ratio of their surface energies (RS), and the difference of sublimation enthalpies (∆H) as chemical input parameters. For structural input parameters, the difference of atomic radii (∆R) between the hosts and solutes is adopted. The values of chemical and structural features are provided in Table S1 of the Supplementary Material. Note that all the chemical and structural parameters can be obtained without performing any first principles calculations. A scatter-plot of the relationship between ∆E SE and ∆C, RS, ∆H, and ∆R is shown in Figure 1. It is clear from the figure that ∆E SE is positively correlated with ∆C, RS, and ∆H, indicating that the bond-breaking effect plays an important role in GB embrittlement. In addition, the atomic size effect also plays a non-negligible role because it is also well correlated with ∆E SE .
Materials 2020, 13, 179 3 of 11 function (RBF) kernel, and artificial neural network (ANN). We use standard statistical analysis methods to determine the factors and predict ΔESE of solutes in different host metals. Our purpose is to develop fast and accurate models for prediction of ΔESE and uncover the key features.

Methods
The dataset we use for training and testing contains 142 data points by DFT calculations collected from literature [8,19,25] and first principles calculations, see Supplementary Table S1. The set contains 5 hosts which are Ni, Fe, W, Al, and Mo. To build accurate and reliable machine learning models, it is important to include relevant features that collectively capture the trends in the ΔESE across the different metals. Based on the input parameters of the above models and by taking into account the accessibilities of the parameters, we select the difference of cohesive energies (ΔC) between the host and the segregated solute atoms, the ratio of their surface energies (RS), and the difference of sublimation enthalpies (ΔH) as chemical input parameters. For structural input parameters, the difference of atomic radii (ΔR) between the hosts and solutes is adopted. The values of chemical and structural features are provided in Table S1 of the Supplementary Material. Note that all the chemical and structural parameters can be obtained without performing any first principles calculations. A scatter-plot of the relationship between ΔESE and ΔC, RS, ΔH, and ΔR is shown in Figure 1. It is clear from the figure that ΔESE is positively correlated with ΔC, RS, and ΔH, indicating that the bond-breaking effect plays an important role in GB embrittlement. In addition, the atomic size effect also plays a non-negligible role because it is also well correlated with ΔESE. As discussed above, the correlation between bond-breaking effect and size effect in the context of GB embrittlement motivate the use of machine learning models to analyze the extent to which each of these phenomena is correlated with ∆E SE , once the other is taken into account. To asses this question, and to see if other variables are associated with ∆E SE , three machine learning models are constructed including SVM with linear kernel, SVM with RBF kernel, and ANN [31,44]. For SVM, there are many types of kernel function that can affect the performance of predictions and what we use in this work are two common ones, linear and radial basis function (RBF) [31,32]. To realize the SVM modeling, we consider the ε-SVR method using the LIBSVM package in MATLAB (version 9.3.0.713579, R2017b, MathWorks, Inc., Natick, MA, USA) [45]. For ANN, the type of network model we use in this work is the common one based on the back-propagation learning algorithm with Bayesian regularization [46,47], and operated via the toolbox called "nntool" in MATLAB.
The prediction performance of these methods is evaluated by four metrics, which are generally used as the error statistical parameters, mean absolute error (MAE), root mean square error (RMSE), standard deviation of error (SDE), and square correlation coefficient (r 2 ). MAE is defined as RMSE is defined as SDE is defined as Square correlation coefficient r 2 is defined as where l is data size, y i is the value of DFT calculations, and f (x i ) is the predicted value of corresponding input. f (x) = 1 l l i=1 f (x i ) (analogously for y) and r is Pearson correlation coefficient [48]. Figure 2 shows the fitted results from each of three machine learning methods compared against the DFT calculations using the above four features. These results are from using the entire dataset as the training data for each machine learning method. For the prediction using SVM with linear kernel in Figure 2a, a grid optimization method is used to optimize C, and the values of C range from 2 −10 to 2 10 and the step size is 0.5. The SVM model of the best prediction performance was found when C = 2 3.5 . The linear regression results are given by ∆E SE = −0.238980475 + 0.024468378 ∆H + 0.033997136 RS + 0.223070248 ∆C +1.05643494 ∆R. For the prediction using SVM with RBF kernel in Figure 2b, two parameters, C and γ, can be changed to optimize the model. Similarly, the grid optimization method is adopted to optimize C and γ ranging from 2 −10 to 2 10 , respectively. The best parameters are found when C = 2 4 and γ = 2 0.5 . For the prediction using ANN in Figure 2c, we use one input layer, one hidden layer including 4 hidden units and the output layer through the pre-test. The results of RMSE and r 2 for three models are summarized in Table 1. It can be seen that the SVM model with RBF kernel and the ANN model show better performances than the SVM model with a linear kernel.  Table 1. Values of mean absolute error (MAE), root mean square error (RMSE), square correlation coefficient (SDE) and r 2 from full fit and 10-fold cross validation predictions of three machine learning models with four input features. SVM: support vector machine; RBF: radial basis function; ANN: artificial neural network. Note that a good prediction for the training dataset does not suggest that the model has good prediction ability for the unknown dataset. Thus, to increase reliability of method, the data is randomly divided into 10 groups and during 10 iterations one group is set as the testing dataset, while the remaining nine groups are set as the training dataset. This process is called the 10-fold cross validation that can eliminate the chance of over-fitting [48,49]. For the SVM models with linear kernel and RBF kernel, we follow the similar optimization of model parameters for every group predictions. The values of parameters C and γ are listed in Table 2. For the stochastic nature of ANN model, the prediction procedure of every group is repeated 30 times and an average of 30 times predictions is taken. The prediction results of three models with all four features are shown in Figure  3. A comparison of MAE, RMSE, SDE, and r 2 from the three models is also displayed in Table 1. Generally, the error of cross-validation procedure should always be higher than the error of fitted procedure and this is this case here. It can be seen from Table 1 that, for all three models, the values of MAE, RMSE, SDE, and r 2 are in the range of 0.280-0.300 eV, 0.409-0.424 eV, 0.290-0.300 eV, and 0.827-0.839, respectively. This indicates that the cross-validation errors from three methods using the four features are comparable. Moreover, to increase the accuracy of the models, the 14-fold cross validation tests are also performed, and the prediction results are shown in the Supplementary Material. Generally, the prediction results are similar to that using 10-fold cross validation, suggesting that the latter is sufficient to ensure accuracy.  Note that a good prediction for the training dataset does not suggest that the model has good prediction ability for the unknown dataset. Thus, to increase reliability of method, the data is randomly divided into 10 groups and during 10 iterations one group is set as the testing dataset, while the remaining nine groups are set as the training dataset. This process is called the 10-fold cross validation that can eliminate the chance of over-fitting [48,49]. For the SVM models with linear kernel and RBF kernel, we follow the similar optimization of model parameters for every group predictions. The values of parameters C and γ are listed in Table 2. For the stochastic nature of ANN model, the prediction procedure of every group is repeated 30 times and an average of 30 times predictions is taken. The prediction results of three models with all four features are shown in Figure 3. A comparison of MAE, RMSE, SDE, and r 2 from the three models is also displayed in Table 1. Generally, the error of cross-validation procedure should always be higher than the error of fitted procedure and this is this case here. It can be seen from Table 1 that, for all three models, the values of MAE, RMSE, SDE, and r 2 are in the range of 0.280-0.300 eV, 0.409-0.424 eV, 0.290-0.300 eV, and 0.827-0.839, respectively. This indicates that the cross-validation errors from three methods using the four features are comparable. Moreover, to increase the accuracy of the models, the 14-fold cross validation tests are also performed, and the prediction results are shown in the Supplementary Material. Generally, the prediction results are similar to that using 10-fold cross validation, suggesting that the latter is sufficient to ensure accuracy.  To analyze the effect of features on the ΔESE, the SVM models with RBF kernel are used with different combinations of input features. For each set of features, the SVM models are optimized with full fitting. The values of RMSE and r 2 of the SVM models with different input features are displayed in Figure 4. The nonlinear SVM model including ΔH, ΔC, and ΔR is found to be the best description for ΔESE, with RMSE of 0.339 eV and r 2 of 0.889, while the worst performance is obtained using ΔH and RS, with RMSE of 0.423 eV and r 2 of 0.828, respectively. The correlation among ΔH, RS, and ΔC can be inferred from the fact that the values of RMSE and r 2 do not add to one another when the regressors are combined in the same model. However, the bond breaking effects reflected by ΔH, RS, and ΔC are clearly demonstrated as significant in GB cohesion. It should be noted that, when removing the feature ΔR from the groups of ΔH + RS + ΔC + ΔR, ΔH + ΔC + ΔR, ΔH + RS + ΔR, and RS + ΔC + ΔR, the performance becomes poorer in different degrees. The analysis shows that ΔR is a significant feature to the contribution of ΔESE. Therefore, the statistical conclusion shows that bond-breaking and atomic size effects are independent and substantial contributors to GB cohesion.  To analyze the effect of features on the ∆E SE , the SVM models with RBF kernel are used with different combinations of input features. For each set of features, the SVM models are optimized with full fitting. The values of RMSE and r 2 of the SVM models with different input features are displayed in Figure 4. The nonlinear SVM model including ∆H, ∆C, and ∆R is found to be the best description for ∆E SE , with RMSE of 0.339 eV and r 2 of 0.889, while the worst performance is obtained using ∆H and RS, with RMSE of 0.423 eV and r 2 of 0.828, respectively. The correlation among ∆H, RS, and ∆C can be inferred from the fact that the values of RMSE and r 2 do not add to one another when the regressors are combined in the same model. However, the bond breaking effects reflected by ∆H, RS, and ∆C are clearly demonstrated as significant in GB cohesion. It should be noted that, when removing the feature ∆R from the groups of ∆H + RS + ∆C + ∆R, ∆H + ∆C + ∆R, ∆H + RS + ∆R, and RS + ∆C + ∆R, the performance becomes poorer in different degrees. The analysis shows that ∆R is a significant feature to the contribution of ∆E SE . Therefore, the statistical conclusion shows that bond-breaking and atomic size effects are independent and substantial contributors to GB cohesion. To quantitatively explore the relative significance of each input variable for improving the prediction performance, the mean impact value (MIV) analysis was conducted using the similar method previously by Jiang et al. [50] and Liu et al. [51]. The MIV values for each input variable on each output variable are calculated and shown in Figure 5. It can be found that the important sequence of the factors for the strengthening energies is △C > △R > △H > RS. △C and △R are the two most important factors to influence the strengthening energies. This may explain why the two-factor model is able to account for most of the variation in the strengthening energies in Mo GBs [28]. As shown in Figure 5, the features △H, △C and △R show a positive correlation with grain boundary embrittlement, while RS appears a negative correlation. According to the MIV analysis, it can be found that △H, △C, and △R have a high correlation with grain boundary embrittlement. This is consistent with the above conclusion that the SVM model with these three features yields the best results amongst all the test models. To quantitatively explore the relative significance of each input variable for improving the prediction performance, the mean impact value (MIV) analysis was conducted using the similar method previously by Jiang et al. [50] and Liu et al. [51]. The MIV values for each input variable on each output variable are calculated and shown in Figure 5. It can be found that the important sequence of the factors for the strengthening energies is C > R > H > RS. C and R are the two most important factors to influence the strengthening energies. This may explain why the two-factor model is able to account for most of the variation in the strengthening energies in Mo GBs [28]. As shown in Figure 5, the features H, C and R show a positive correlation with grain boundary embrittlement, while RS appears a negative correlation. According to the MIV analysis, it can be found that H, C, and R have a high correlation with grain boundary embrittlement. This is consistent with the above conclusion that the SVM model with these three features yields the best results amongst all the test models. The aggregation of ΔESE in different host metals from so many sources should be expected to lead to significant scatter due to the different computations such as potential selection and relaxation standard [52]. Here, the relative good performance of nonlinear SVM model using ΔH, ΔC, and ΔR is thus interpreted to be highly physically meaningful. The present model could not only be used to understand GB strengthening or embrittlement and its underlying physical origins, but also serves as a quantitative prediction of the strengthening energies by solute segregation in other systems that have not yet been studied experimentally or computationally, which is helpful for the rational design and screening of novel materials with desired mechanical properties for specific applications.

Conclusions
In this work, we have trained three separate machine learning models to infer the driving forces for GB embrittlement and predict strengthening energies for impurities in different host metals. The readily accessible features (i.e., difference of sublimation enthalpies ΔH, ratio of surface energies RS, difference of cohesive energies ΔC and difference of atomic radii ΔR) are chosen as descriptors. It was shown that the energetics of embrittlement is quantitatively described by two simple effects: bond-breaking and atomic size. A nonlinear kernel based support vector regression model with features ΔH, ΔC, and ΔR shows the best performance prediction for the aggregated set of strengthening energies, with RMSE of 0.339 eV and r 2 of 0.889. The feature of the size effect was found to exhibit considerable importance to the model prediction. Additionally, the MIV based analysis has been carried out for evaluating the features' importance. Results show that the important sequence of the factors for the GB embrittlement is: ΔC > ΔR > ΔH > RS. The methods employed in the present work, i.e., clarifying the physical mechanism and extracting the key feature quantities as descriptors by first principles calculations and then predicting material properties via machine learning, can be extended to predict other material properties.
Supplementary Materials: The data related to this article are available in the www.mdpi.com/xxx/s1. Figure S1: Comparison of ΔESE from DFT calculations and 14-fold cross validation prediction results using three machine learning models; Table S1: Statistical parameters from full fit and 14-fold cross validation predictions of three machine learning models;   The aggregation of ∆E SE in different host metals from so many sources should be expected to lead to significant scatter due to the different computations such as potential selection and relaxation standard [52]. Here, the relative good performance of nonlinear SVM model using ∆H, ∆C, and ∆R is thus interpreted to be highly physically meaningful. The present model could not only be used to understand GB strengthening or embrittlement and its underlying physical origins, but also serves as a quantitative prediction of the strengthening energies by solute segregation in other systems that have not yet been studied experimentally or computationally, which is helpful for the rational design and screening of novel materials with desired mechanical properties for specific applications.

Conclusions
In this work, we have trained three separate machine learning models to infer the driving forces for GB embrittlement and predict strengthening energies for impurities in different host metals. The readily accessible features (i.e., difference of sublimation enthalpies ∆H, ratio of surface energies RS, difference of cohesive energies ∆C and difference of atomic radii ∆R) are chosen as descriptors. It was shown that the energetics of embrittlement is quantitatively described by two simple effects: bond-breaking and atomic size. A nonlinear kernel based support vector regression model with features ∆H, ∆C, and ∆R shows the best performance prediction for the aggregated set of strengthening energies, with RMSE of 0.339 eV and r 2 of 0.889. The feature of the size effect was found to exhibit considerable importance to the model prediction. Additionally, the MIV based analysis has been carried out for evaluating the features' importance. Results show that the important sequence of the factors for the GB embrittlement is: ∆C > ∆R > ∆H > RS. The methods employed in the present work, i.e., clarifying the physical mechanism and extracting the key feature quantities as descriptors by first principles calculations and then predicting material properties via machine learning, can be extended to predict other material properties.
Supplementary Materials: The data related to this article are available in the http://www.mdpi.com/1996-1944/ 13/1/179/s1. Figure S1: Comparison of ∆E SE from DFT calculations and 14-fold cross validation prediction results using three machine learning models; Table S1: Statistical parameters from full fit and 14-fold cross validation predictions of three machine learning models; Table S2: Database on ∆E SE of solutes in different host metals for the training and test of the machine learning models; Table S3: Statistical parameters for the SVM model with RBF kernel with different features; Table S4: Statistical parameters for three machine learning models with best performance using different features.