#### 3.1. Evaluation of Surrogate Models for the Rib Structure

The evaluation of the results intends to discuss anomalies in the performance metrics of the training and test data, investigate the general suitability of data-driven FEM surrogate models for the development of digital twins of automotive structures and to determine the most suitable method for the given numerical example.

At first, it should be noted that there are significant differences in the achieved coefficient of determination

${R}^{2}$ for the different methods based on the test data (cf.

Figure 5). The methods Decision Tree (

${R}^{2}$ = 0.81), Grad Boost (

${R}^{2}$ = 0.73), Random Forest (

${R}^{2}$ = 0.83) and XGBoost (

${R}^{2}$ = 0.73) reveal a relatively high goodness of fit on testing data. In contrast, the models AdaBoost (

${R}^{2}$ = −0.34) and Polynomial Regression (

${R}^{2}$ = 0.25) show a significantly weaker performance. It is also remarkable that very high scores can be achieved for a large number of FEM samples, but a number of training and test samples deviate significantly from these good results, for example, Random Forest (

${R}_{max}^{2}$ = 0.994 for samples 7 and 52 and

${R}_{min}^{2}$ = 0.33 for sample 69). Depending on the model, the number of outliers (defined threshold

${R}^{2}$ < 0.7) ranges between 21 and 24 samples for the good models, with Random Forest having the smallest amount of outliers. All samples of AdaBoost and Polynomial Regression do not reach the defined threshold. In order to find the cause for the poor prediction quality, a comparison of all outliers in contrast to the parametrization of the numerical simulation was carried out. The investigation shows that all samples that have not been well-fitted have a low organo sheet and relatively low mold temperature. For those parametrizations, the temperature at the ribs does not reach the target temperature of 163 °C during the overmolding process, which results in a huge amount of zero values for the objective variable

${t}_{m}$. To draw a better picture of the model performances for the further course of the study all those outliers with unfeasible process conditions are removed from the data set and their corresponding metrics (except for the weak models AdaBoost and Polynomial Regression to keep the approaches available for comparison).

Figure 5 shows a selection of performance metrics, that is,

${R}^{2}$ [%], MSE [%], MAX error [%], MAE [%] (

Figure 5a), and the relative training and prediction times for the six models examined (

Figure 5b). For better comparability, all metrics except for

${R}^{2}$ were normalized based on the maximum value per metric. The strong performance of the Random Forest and Decision Tree methods is striking, both for the global model quality

${R}^{2}$ and for local deviations at the point of integration (MSE, MAX error and MAE). On the other hand, the comparison of training and prediction times shows the great speed efficiency of the Decision Tree. The prediction is 101 times faster (31 ms) than with the Random Forest approach. For the average full scale simulation a CPU time of approximately 5 h has been used, which can be reduced to 30 to 60 min when computing on parallel threads.

Especially with regard to the real-time capability for an inline quality gate, short prediction times are crucial and, depending on the application, can be considered as a criterion of exclusion. To examine the local goodness of fit of the models, the Max error is used as a metric.

Figure 6 shows the minimum, mean and maximum MAX error for the test data of all methods. Comparing the four better approaches, Decision Tree, which has been very solid in the previous analysis, generates a relatively high max MAX error (5.09 s). In contrast, Random Forest clearly provides the best values for the max MAX error (3.79 s) and the lowest maximum error (min MAX error) for sample 76 (0.2 s). Even if those errors are relatively high, they only occur at some rare points of the mesh, which should not influence the decision support on part quality. A detailed discussion on this is carried out in terms of the absolute difference and relative error in the following (cf.

Figure 7 and

Figure 8).

The analysis shows that for the given numerical example Random Forest is the best approach for FEM surrogate modeling (high global and local goodness of fit, fewest issues with outliers and acceptable training and prediction times). Decision Tree, with disadvantages in local prediction quality, however, can achieve a similarly good performance, having an advantage in prediction speed. Due to this, the following aspects are discussed in terms of the Random Forest results, representing the best suitable approach for the underlying numerical example.

Among all computed samples, sample 52 shows the best coefficient of determination (

${R}^{2}$ = 0.994). In

Figure 7, a comparison of the numerical simulation (

Figure 7a) with the corresponding surrogate model (

Figure 7b) based on the Random Forest approach is shown. The contour plots are qualitatively similar and show comparable distributions of

${t}_{m}$. Only the result in the compensation volume on the right side differs noticeably. Regarding the absolute difference in

Figure 7c, it can be observed that the maximum deviation at some nodes is up to 0.91 s, which is an error of 14% compared to the maximum value of the sample. However, evaluating the difference of the whole interface domain, a difference of 0.5 s or less is observed for most of the nodes. In order to get a better impression of the approximation quality, the relative error

with

${t}_{m,pred}$ denoting the predicted results and

${t}_{m,num}$ the result of the full-scale simulation is depicted in

Figure 7d. In this plot, two phenomena can be observed. First, due to large deviations at single nodes, large errors can be observed for some singular points. Especially at the boundary of the interface region the largest error is observed. In the center of the rib the error is comparatively small. Second, the Random Forest approach predicts in some points values of

${t}_{m}>0$, where usually no contact between injected material and the part insert is present.

A further example of the surrogate model is given for sample 11. It yields the best value of

${R}^{2}$ of all samples that are not contained in the training data. Further, it shows one of the smallest min MAX error of (0.8 s) of all samples. Analogously to sample 52,

Figure 8 shows the performance of the surrogate model. The comparison of the result from a full-scale simulation in

Figure 8a with the corresponding predicted result by Random Forest in

Figure 8b shows qualitatively similar distributions. The absolute difference of this example is depicted in

Figure 8c. Similar to sample 52, the largest differences occur only for singular points. In the average the absolute difference of the predicted solution is less than 0.4 s. The relative error

$\u03f5$ for this sample is depicted in

Figure 8d. Also for this example the error at the boundary of the interface domain is greater than in the center of a rib. Compared to

Figure 7d, the number of points where a value

${t}_{m}>0$ is predicted but the full-scale simulation shows

${t}_{m}=0$ is significantly larger. These points are mainly clustered along the boundary of the rib interface. Even if these predicted values are insignificant in their absolute value (cf.

Figure 8b), geometric information about the contact surface must be included to avoid such nonphysical values and to further improve the prediction.

Another issue in data-driven surrogate modeling concerns the ability to deal with outliers that have been identified as simulated unfeasible process conditions. Those conditions are characterized by almost all target variables to be zero. The phenomena is illustrated for sample 76 in

Figure 9a where the numerical results are all zero for

${t}_{m}$, but the corresponding surrogate predicts small temperatures (

${t}_{m,pred,max}=0.2$ s) at the interface (

Figure 9b). This fact might be caused by the sampling procedure. When no or too few number of samples with unfeasible process conditions are contained in the training set, the surrogate can hardly predict a solution, where all results are zero. However, in this example, the absolute difference is very small and it represents the smallest MAX error of all samples.

#### 3.2. Experimental Results of Cross Tension Testing

With the surrogate model, we are able to predict temperature profiles within a few seconds or shorter. These models are valid within the investigated space of process variables defined in

Table 1. In the simulations, the objective variable

${t}_{m}$ describes the time, the part insert temperature is greater than the melting temperature

${T}_{PP}^{*}$ of the polymer. During the manufacturing, this time cannot be measured directly. Hence, results from simulations are needed to estimate the temperature at the interface. In order to qualify the surrogate model based on the simulation results as digital twin, it is necessary to correlate the computed objective variable with a structural quality value. Here, the interface bond strength

${\sigma}_{b}$ between organo sheet and the injected polymer is chosen as quality value. Therefore, cross tension tests are conducted using specimens manufactured in accordance with the process parameters in

Table 2. In the corresponding simulations, the mean value for

${t}_{m}$ in the test domain of each rib is computed to achieve a direct link between

${t}_{m}$ and

${\sigma}_{b}$.

In

Figure 10, the experimentally evaluated bond strength values are plotted over the computed objective value

${t}_{m}$ at the rib interface. For low

${t}_{m}$, a large variability in the resulting bond strength can be seen. For

${t}_{m}<0.4$ s, the bond strength varies between 0.66 MPa and 7.83 MPa. For

${t}_{m}>1.5$ s on the other hand, the experiments yield in every experiment a bond strength greater than 6 MPa. Also here, a relatively large variability in the experimental values is observed. The maximum bond strength reached in the experiments is 11.3 MPa. The significant difference in the experiments implies that further process parameters, such as injection and packing pressure, also influence the interdiffusion. However, despite the large variability in the results, the trendline shows a significant increase of the bond strength with increasing

${t}_{m}$. Hence, the derived FEM surrogate of thermoplastic composite overmolding is suitable to support the quality control in a digital twin application. The quality domains are divided according to

${t}_{m}$. The quality domain ‘poor’ is defined in the range

$0\le {t}_{m}<0.4$ since for such short times, bond strength values less than 4 MPa most likely appear in the experimental data. As ‘excellent’, the quality domain is chosen where the experiments yield only bond strength values greater than 6 MPa (

${t}_{m}>1.5$). According to the sampled experimental data, within the intermediate domain a bond strength of at least 4 MPa is ensured, which is denoted as ‘good’ in the diagram. By computing process dependent temperature distributions in real time, the classification into quality domains and the corresponding visualization provides an immediate visual decision support.