Comparative Study of Machine Learning Approaches for Predicting Creep Behavior of Polyurethane Elastomer

The long-term mechanical properties of viscoelastic polymers are among their most important aspects. In the present research, a machine learning approach was proposed for creep properties’ prediction of polyurethane elastomer considering the effect of creep time, creep temperature, creep stress and the hardness of the material. The approaches are based on multilayer perceptron network, random forest and support vector machine regression, respectively. While the genetic algorithm and k-fold cross-validation were used to tune the hyper-parameters. The results showed that the three models all proposed excellent fitting ability for the training set. Moreover, the three models had different prediction capabilities for the testing set by focusing on various changing factors. The correlation coefficient values between the predicted and experimental strains were larger than 0.913 (mostly larger than 0.998) on the testing set when choosing the reasonable model.


Introduction
Polymers are widely used in traditional industry, agriculture and high and new technology sectors due to their extensive sources, industrial maturity, and excellent properties (light weight, high strength, good toughness, etc.). Contemporary human life, from groceries to space shuttles and rockets, is closely related to polymers. In recent years, the service life of polymers has been required to be longer, even up to several decades of applications, which makes the long-term mechanical properties of polymers a hot research subject.
A polymer is a kind of substance with a polymer chain. The multiplicity, time dependence and temperature dependence of molecular chain movement makes polymer a typical viscoelastic material. Therefore, mechanical relaxation phenomena, such as the stress relaxation, creep and recovery of polymer is shown to be significant, and its mechanical behavior strongly depends on the time of exogenic force exertion [1]. The time dependence of viscoelastic material's mechanical behavior indicates that there is characteristic time in the material [2]. The characteristic time is influenced by factors such as temperature [3], stress [4], strain [5] and physical aging [6]. The relative research shown that temperature could affect the characteristic time by changing the free volume of material [7]. For polymers, the effect of changing the temperature scale and time scale on their macro viscoelastic mechanical properties is equivalent [1]. Thus, the time-temperature superposition principle (TTSP) was presented. According to TTSP, the long-term mechanical properties of viscoelastic materials at lower temperatures can be obtained by shifting the short-term experimental curve at higher temperatures along the logarithmic time axis [1,7]. However, the time-stress superposition principle (TSSP) indicates that increasing the stress level has a similar effect [4]. These superposition principles make it possible to accelerate the characterization of the long-term mechanical properties of polymers [2].
Polyurethane elastomer (PUE) is an important type of polymer, which is made up of hard segment and soft segment, arranged alternately. Including isocyanate and chain mization swarm methods, to predict the long-term settlement creep index. The datasets of structural liquid limit, plasticity index, void ratio, clay content and creep index were collected through literature review. The RF model was established. Using the particle swarm optimization method and cross-validation method, the model was optimized. The results indicated that the prediction error of this method was significantly lower than that of existing empirical formulae. Then, the researchers developed a constitutive model for soils by using pure mathematical skills through learning from raw data using the machine learning method [23]. The research summarized the application of the machine learning algorithm in soil constitutive model development. The results showed that the long shortterm memory neural network was most suitable for developing the constitutive model of the soil.
Similarly, several kinds of research have been carried out in the polymers field using machine learning techniques in recent years. According to the machine learning algorithm, Mannodi-Kanakkithodi et al. [24] developed an on-demand property prediction model to directly realize the design of polymers with given target properties using a genetic algorithm to evolutionarily optimize polymer constituent blocks. Then, by combining computations or experiments with machine learning techniques, they utilized a first principles-generated dataset of the electronic and dielectric properties of a chemical space of polymers to test different kinds of regression algorithms. Several possibilities for the hyper-parameters have been explored, and the optimal strategies and parameters for high-fidelity polymer dielectrics property prediction have been established [25]. Doan Tran et al. [26] provided an overview of some of the critical technical aspects based on the polymer genome machine learning method, including polymer data curation, representation, learning algorithms, and prediction model usage. Furthermore, the remaining challenges and possible future directions were discussed. Zhong et al. [27] built a long-term creep behavior prediction model of PMI materials using the ANN technique. The effects of different activation functions, hidden layer structures, and other super-parameters on the prediction performance were investigated. The results suggested that the statistical value of the correlation coefficient was greater than 0.995. Rahman et al. [28] developed a surrogate machine learning model trained with molecular dynamics models of functionalized CNT-epoxy and the corresponding interfacial shear strength. Yildirim et al. [29] predicted and compared perovskite solar cells performances, based on machine learning approaches, with those developed WO3 and its composites. The results showed that the decision tree model has a 0.9656 R 2 score for the WO3-poly (3,4-ethylenedioxythiophene) and the random forest model has 0.9976, 0.9968, 0.9772 R 2 scores for the WO3-poly(N-methylaniline), WO3-poly(2-fluoroaniline), and WO3-polyfuran, respectively. Yuan et al. [30] focused on the incomplete database of The Membrane Society of Australasia, by imputing missing values in the database using the machine learning method, which extended the potential use of the database.
In this paper, the feasibility of the machine learning method to predict the compression creep deformation of PUE with highly nonlinear properties was analyzed, when changing the conditions of creep time, temperature, stress and hardness of the material. Based on the experiment results, the multilayer perceptron (MLP) network, RF and SVM algorithms of machine learning were used, combining the genetic algorithm and cross-validation, the prediction model was established. The accuracy and stability of the model were studied by comparing the performance of training set fitting and new condition prediction of the three models. This paper opens up a new way to predict the long-term mechanical properties of polymers through the machine learning method, which could reduce the number of experiment working conditions as well as shorten the experiment period, and provide an idea for the accelerated characterization of long-term mechanical properties of materials, in addition to various superposition principles.

Creep Experiment
A commercial high-performance PUE produced by the casting machine of Nanjing Jinsanli Rubber & Plastic Co., Ltd. (Nanjing, China) was considered for a compression creep experiment in this research.
Traditional PUE material can be used for a long time under 80 • C, and its short-term service temperature is a maximum of 120 • C. It can be found that such low-temperature resistance limits its application. In this study, the polyurethane elastomer synthesized by 4,4 -methylenedianilin (MDI), polyether polyols, trihydroxymethylpropane crosslinking chain extender and auxiliaries were considered, which could improve the thermal stability and mechanical properties. The hardness of PUE was controlled by adjusting the component content.
The cylindrical specimens with 12.5 mm diameter and 6.5 mm thickness were made by mold casting process, as shown in Figure 1. Each type of specimen was made of the same batch of raw materials and production, to ensure that the thermal and mechanical properties of specimens were consistent. The hyper-elastic properties of PUE were first proposed using a universal mechanical testing machine and other equipment. Three parallel experiments were conducted for each kind of hardness. The results are shown in Table 1.  The universal testing machine with calorstat was used to implement the compression creep experiment, as shown in Figure 2 where ε t refers to total strain, ε e refers to elastic strain and ε c refers to creep strain. The results were shown from Tables 2 to 4 by calculation. It could be seen that the creep strain ε c of PUE increased with the increase in creep stress and temperature at the same creep time.
In the variable region of this research, the effect of creep stress was more significant.  The factors that affect the mechanical properties of polymer materials include time, stress, strain rate, temperature, humidity, aging, and crystallinity. From the results of the compression creep experiment mentioned in Section 2.1, creep stress and temperature had more influence on the creep properties of PUE. In addition, due to the different content of each component and their varying degree of crystallinity, when preparing the PUE, the hardness variance affects the macroscopic creep behavior of the material on the microscopic level. Therefore, the creep time x ti , creep stress x st , temperature x te and hardness of PUE x h were considered as the input variables x = [x ti , x st , x te , x h ] T . The axial strain of PUE was taken as the output variable. The range of input variables chosen from the experiment conditions is shown in Table 5.

. Data Analysis
After determining the input and output variables, the correlation between the variables in the creep experiment results was analyzed, as shown in Figure 6. It could be seen that the correlation coefficients between the input variables in this research were slim to zero, which means that the input variables were independent of each other. While there were varying degrees of correlation between the input and output variables. These results indicated that the input variables considered in this research are reasonable with no redundancy, and the research on the machine learning prediction model could be carried out on this basis.

Data Normalization
The range of input variables and strain results of the creep experiment showed that the input and output variables were not in the same order of magnitude. However, multiple machine learning methods, such as ANN, require that the weights and other parameters in the model are parallel in order of magnitude. If the difference of input variables is large, the input variables with a smaller order of magnitude will be covered by those with a larger order of magnitude during the error propagation. Furthermore, the effect of each input variable on the output cannot be rendered properly. Consequently, normalizing the input and output variables is crucial before modeling.
In this research, the z-score method, which is commonly used alongside machine learning, was considered to normalize the input and output variables, so that the mean value of each variable equals 0 while the variance equals 1. The method can be expressed asx where x (n) are the original samples, n = 1, 2, · · · , N, and N is the number of samples,x is the mean value of samples, S is the variance of samples,x (n) are the normalized samples. After normalization, the input variables can be expressed asx = [x t ,x st ,x te ,x h ] T , while the output variables change toŷ.

Multilayer Perceptron Network
MLP network is a typical ANN, which is a nonlinear complex network system composed of a large number of "biological neurons". As shown in Figure 7, a mathematical model is used to describe the biological neural network structure, so that the intelligent behavior to some extent can be simulated under the guidance of the algorithm. In this research, MLP based on the backpropagation algorithm is considered to train the prediction model. It is composed of an input layer, an output layer and at least one hidden layer. The training consists of two processes: signal forward propagation and error backpropagation. During the forward propagation, the input samples are transmitted from the input layer to each hidden layer and output layer. Then, the backpropagation stage begins if the output value is not equal to the real value. The error is allocated among all neural of the hidden layer by transmitting the output error back, and the error of neurons in each layer is obtained as the basis for optimizing the weights of neurons. The above processes are repeated until the output error is acceptable or reaches the training iterations' limitation.

Input layer
Output layer Hidden layer The nonlinear properties of the PUE creep strain-time curve were considered to have initially identified the main structural parameters range of the MLP network, as shown in Table 6. The activation function can be expressed as In order to avoid unnecessarily increasing the complexity of the model, the number of the hidden layer is set to 1 or 2. The number of neurons is setting from 1 to 100. The training method is selected from L-BFGS [31], SGD [32] and Adam [33]. Subsequently, the MLP network prediction model of PUE creep properties is constructed by optimizing the hyper-parameters of the model.

Random Forest
RF is a machine learning algorithm that integrates multiple decision trees based on the idea of ensemble learning. Decision tree, the basic unit of RF, is a kind of tree-like structure with the function of data classification or regression. It is composed of an internal node, leaf node and directed edge, as shown in Figure 8. For the regression problem, the predictive value of each leaf node is the mean value of the training set elements' output, which can be expressed as The leaf nodes represent the predicted value, and mean squared error (MSE) or mean absolute error (MAE) is generally used as the criteria to feature and split.
The main hyper-parameter range of RF was initially identified to include the maximum depth of RF (max depth), the maximum number of decision trees in the RF (max DT), the minimum number of samples at the leaf node (min samples leaf), and the minimum number of samples required to split an internal node (min samples split), as shown in Table 7. Subsequently, the RF prediction model of PUE creep properties is constructed by optimizing the hyper-parameters of the model.

. Support Vector Machine Regression
As one of the most common methods in the machine learning field, support vector machine regression (SVR) has shown its unique advantages in solving the problems of small sample, nonlinear and high-dimensional pattern recognition. SVR is developed from the optimal classification surface of a linearly separable problem, using nonlinear transformation defined by the inner product function to transform the sample input space into another higher dimensional space, and then solving the generalized optimal classification hyperplane. SVR mainly solves the finite sample problem and finds the best compromise between the complexity and the learning ability of the model in order to obtain the best generalization ability, as shown in Figure 9. The SVR method successfully avoids the traditional process from induction to deduction and efficiently realizes the "transductive inference" from training data to predicted data. The basic idea of nonlinear SVR is to map the data x to the Hilbert feature space using a nonlinear mapping φ, then linear regression is carried out in this space. The kernel function k x i , x j = φ(x i ) · φ x j is used to realize the correspondence between the linear regression of high-dimensional space to the nonlinear regression of low-dimensional space. The SVR theory was widely introduced in formal research like Refs. [10,14], and a three-order RBF kernel was considered in this study. The main hyper-parameter range of SVR was initially identified to include C and gamma, as shown in Table 8.

The Adjustment and Validation of Hyper-Parameters
In this research, the k-fold cross-validation method was used to avoid overfitting. The training set was randomly divided into k folds. The training set was composed of k − 1 folds and the validation set was performed by the remaining fold. With various subsets being used as the validation set, the training process was repeated k times. The cross-validation error was obtained by averaging the MSE of k times. Then, the training accuracy of the model under the current hyper-parameters was represented. Ten-fold cross-validation was considered according to the number of experiment samples in this study, as shown in Figure 10. The genetic algorithm was considered to optimize the hyper-parameters of the three machine learning models. The genetic algorithm mainly used the law of the "survival of the fittest" in the process of biological evolution, imitating the genetic reproduction mechanism. First, binary or other systems were used to code the individuals in the solution space of the optimization problem. Then, genetic operations such as selection, crossover and mutation are being carried out. By repeatedly and properly using the operators and selection principles of genetic algorithms, the population can continuously reproduce from parental generation to filial generation, which makes the adaptability of the population to the environment increase continuously. Through the iterative method mentioned above, the results with an optimal solution or better solution will be found from the new population. The parameters of the genetic algorithm in this research are shown in Table 9.

Evaluation Index
It is a typical regression problem to predict the creep properties of PUE. The evaluation index of the machine learning model of the regression problem includes the MAE, root mean square error (RMSE), correlation coefficient R and the coefficient of determination R 2 , which can be expressed as Var(x) Var(y) (9) (10) where N is the number of samples,ŷ i is the normalization value of the ith sample, y * i is the prediction value of the ith sample, andȳ is the mean value of samples, Cov(x, y) is the covariance, Var(x) is the variance. The four indicators mentioned above were considered to evaluate the ability of the three machine learning models. PYTHON 3.7 was used to build and train the machine learning models. The building method of prediction model in this research is shown in Figure 11.

Optimization Results of Hyper-Parameters
The cross-validation method and genetic algorithm mentioned in Section 2.3.4 were used to optimize the hyper-parameters of the MLP, RF and SVR machine learning prediction models. The results are shown as: To verify the fitting and prediction performance of the model hyper-parameters above, the creep master curve data of PMI material in Ref. [27] were used for comparison. The creep curves at three temperatures were set to be the training set, and the creep curve at the other temperature was set to be the prediction set, respectively. Furthermore, 100 training groups were carried out. The results are shown in Figure 12. The lines refer to the master curve processed by the test, Ref. [27], the model of present work, while the shaded area refers to the envelope range of the prediction after 100 training groups of Ref. [27]. It can be seen that the three machine learning models with optimized hyper-parameters have good fitting and prediction performance for the creep curve of materials and can be used for the creep performance prediction. Then, the strain-time curves of PUE creep predicted by the three machine learning methods mentioned above are shown in the next subsection. The overall trend, fitting performance of the training set and prediction performance of the prediction set were analyzed in detail.

Comparison of Fitting Performance of Training Set
Six compression creep working conditions of PUE were obtained by orthogonal experimental design method, as shown in Table 10. The fitting performance of the three machine learning methods was obtained through the contrast of model and the experimental strain-time creep curve, as shown in Figure 13. The index of fitting accuracy is shown in Table 11.
The fitting curve of the MLP model was basically consistent with the experiment curve under working conditions 1, 5 and 6. The fitting accuracy under working condition 4 was slightly decreased. The coefficient of determination R 2 was 0.7443 and 0.9059 under conditions 2 and 3, respectively, which means that the fitting curve had a large deviation. The RF model had an excellent fitting performance under all six working conditions. The coefficient of determination R 2 was larger than 0.997, which showed the best accuracy among the three methods. The fitting curve of the SVR model was basically consistent with the experiment curve under working conditions 1, 4, 5 and 6. The fitting accuracy under working condition 2 was slightly decreased. The coefficient of determination R 2 was 0.5632, which is the lowest fitting accuracy of the three models. c d e f  By analyzing the fitting performance of the three models for the training set, it could be seen that the overall fitting performance of the RF model was the best. The accuracy of MLP and SVR models was sufficient in most working conditions, while numerical and trend distortion existed in some working conditions. In order to avoid the overfitting of current machine learning models, the prediction performance of PUE compression creeps under new working conditions was considered in Section 3.3.

Comparison of Prediction Performance.
Using the orthogonal experimental design method, the working conditions for machine learning prediction were designed by changing the experimental conditions in Table 10, as shown in Table 12. The changed conditions were written in bold. In this research, six prediction conditions were considered to carry out the compression creep experiment of PUE, and the prediction results of three machine learning models are shown in Figure 14. The prediction accuracy is shown in Table 13. The generalization ability of three prediction models was analyzed by changing the creep time, creep temperature, hardness of PUE and creep stress.

Creep Time
According to the prediction results of working condition 1 shown in Figure 14a, when predicting 8 h creep properties of PUE, the prediction properties of 4 h prior were much better than 4 h later, due to the 4 h creep training set of models. In the interval of 4 h-8 h: there was a cross between the strain growth of the MLP model and the experiment curve, and the strain growth rate of prediction was higher than that of the experiment; the strain growth rate of RF model prediction was closest to that of the experiment, while the overall strain value was slightly smaller; the trend of SVR prediction was quite different from the experiment.
As mentioned in working condition 1 of Table 13, the evaluation index MAE, RMSE and R 2 of the MLP model were better, while the evaluation index R of the RF model was better. The evaluation index of SVR was the worst among the three models. Therefore, the MLP and RF machine learning models have a better prediction ability than the SVR model in the working conditions of extending creep time.   According to prediction results of working conditions 2 and 3 shown in Figure 14b,c, when predicting the creep properties of PUE in the range of 20 • C-90 • C, using the three machine learning models: the prediction creep curves of the RF model were the closest to the experiment curves; the prediction curves of the MLP model under two temperatures were both higher than the experiment curves, and the overall trend was nearly the same, while the prediction accuracy was worse than that of the RF model; the prediction ability of the SVR model represented instability, and the bug contrast of experiment curves, the prediction curve was higher under 30 • C and lower under 50 • C, which had the worst overall trend among the three models.
As mentioned in working conditions 2 and 3 of Table 13, the four evaluation indexes of the RF model were the best, which means the prediction accuracy was the best. The MAE, RMSE and R 2 of the MLP model were better than that of the SVR model, while the evaluation index R of the SVR model was better. Therefore, the main priority of machine learning methods under the working conditions of the changing creep temperature of PUE was RF.

Hardness of PUE
According to prediction results of working condition 4 shown in Figure 14d, when predicting creep properties of PUE by changing the hardness: the prediction creep curves of the MLP model were the closest to the experiment curves, and the trend was nearly the same, which indicates the highest prediction accuracy; the prediction curves of the RF model had the nearest overall trend with the experiment curve, while the curve value was higher than that of the experiment, and the accuracy was worse than the MLP model. The overall trend of the SVR model was the worst among the three models, and the curve value was lower than that of the experiment.
As mentioned in working condition 4 of Table 13, the evaluation index MAE, RMSE and R 2 of the MLP model were the best, while the evaluation index R of the RF model was the best. Therefore, the main priority of the machine learning methods in the working conditions of changing the hardness of PUE was MLP.

Creep Stress
According to the prediction results of working conditions 5 and 6 shown in Figure 14e,f, when changing the creep stress of PUE: the prediction creep curves of the SVR model were the closest to the experiment curves, and MLP model came second. The overall trend of creep curves predicted by the three models was all close to the experiment curve.
As mentioned in working conditions 5 and 6 of Table 13, the evaluation index MAE, RMSE and R 2 of the SVR model were the best, while the evaluation index R of the RF model was the best. Therefore, the main priority of the machine learning methods in the working conditions of changing the creep stress was SVR.

Limitations
Although a new idea was put forward for the prediction of compression creep properties of polymer materials, the author thinks that the current research still has the following limitations: • The experimental working conditions considered were few and the total dataset was relatively small. The accuracy and reliability of the machine learning models such as MPL, RF and SVR would be enhanced under a larger dataset; • Only four creep relative variables including creep time, temperature, hardness of the material and creep stress were considered. However, the creep properties of polymer materials are more complicated. Therefore, more variables should be involved in the subsequent research to further optimize the prediction model; • Due to the inconsistency of the specimen's size, thermal properties of material and experiment error, the inherent laws of the samples under different working conditions were weakened, while the nonlinear properties were further intensified. The training difficulty of the prediction model was increased and the prediction accuracy was reduced.

Conclusions
In this research, a series of machine learning methods (based on MLP, RF and SVR) was used to predict the compressive creep deformation of PUE materials. Considering the variables of creep time, creep temperature, hardness of the material and creep stress, the compression creeps experiment of the PUE specimen was carried out. The genetic algorithm and k-fold cross-validation method were used to optimize the hyper-parameters of the model, and the fitting accuracy of the model in the training set and the prediction ability under new working conditions were verified. According to the results, the following conclusions could be obtained: • The creep properties of PUE was closely related to creep time, creep temperature, creep stress and hardness, which showed strong nonlinear characteristics; • The optimization method by combining genetic algorithm and k-fold cross-validation to the machine learning model's hyper-parameters could effectively improve the fitting accuracy in the training set; • The generalization ability of the MLP model was better when changing the creep time and hardness of the material while changing the creep temperature and creep stress was relatively poor; • The generalization ability of the RF model was better when changing the creep time, creep temperature and hardness of material, while changing the creep stress was relatively poor; • The generalization ability of the SVR model was better when changing the creep stress while changing the creep time, creep temperature and hardness of material were relatively poor; • The method described in this research was the application case of machine learning technology in the field of mechanical response analysis, which could provide a new research idea for the accelerated representation of long-term mechanical properties of polymers.