Online Hyperparameter Tuning in Bayesian Optimization for Material Parameter Identification: An Application in Strain-Hardening Plasticity for Automotive Structural Steel

Abstract

Effective identification of strain-hardening parameters is essential for predictive plasticity models used in automotive applications. However, the performance of Bayesian optimization depends strongly on kernel hyperparameters in the Gaussian-process surrogate, which are often kept fixed. In this work, we propose a likelihood-based online hyperparameter strategy within Bayesian optimization to identify strain-hardening parameters in plasticity. Specifically, we used the rational polynomial strain-hardening scheme for the plasticity model to fit the force vs. displacement response of automotive structural steel in tension. An in-house Bayesian optimization framework was first developed, and an online hyperparameter tuning algorithm was further incorporated to advance the optimization scheme. The optimization histories obtained from the fixed and online-tuning hyperparameters were compared. For the same number of iterations, the online hyperparameter adaptation reduced the final residual by approximately 20.4%, 24.0%, and 3.8% for Specimens 1–3, respectively. These results demonstrate that the proposed strategy can significantly improve the efficiency and quality of strain-hardening parameter identification. The results show that the online tuning scheme improved the optimization efficiency. This proposed strategy may be readily extensible to other materials and identification problems where enhancing optimization efficiency is needed.

1. Introduction

Parameter identification in plasticity is a nonlinear and computationally expensive optimization problem. In industrial practice, these parameters are often obtained through inverse finite element analyses using real component geometries, where simulations are calibrated to match measured force–displacement responses. In each iteration, the parameters are updated and require a full finite element simulation, which can be expensive in time. Bayesian optimization is attractive, as it can significantly reduce the number of expensive simulations needed to identify suitable material parameters [1,2,3]. Nevertheless, the performance of Bayesian optimization itself depends on algorithmic settings such as the choice of kernel hyperparameters in the Gaussian process surrogate.

In a recent paper by Long et al. [4], a novel strain-hardening model and a Bayesian optimization-based material parameter identification strategy were introduced for an automotive structural steel application. It has been demonstrated that the material parameters for the strain-hardening plasticity were successfully identified by Bayesian optimization using rational polynomial stress-strain relation. However, the Bayesian optimization part of that work used fixed hyperparameters in the covariance matrix. In practice, those hyperparameters strongly influence the acquisition function and the search trajectory [5]. The fixed hyperparameters do not allow the surrogate model to adapt to new data, which can lead to numerical instability, poor exploration of the parameter space, etc. Thus, hyperparameters are one of the major difficulties in optimization systems and directly affect the optimization quality.

Existing hyperparameter tuning strategies can broadly be divided into search-based methods and model-based [5,6,7,8,9]. Grid search is the most intuitive approach [7,8,10]. It is a kind of manual procedure to test each set of hyperparameters. With more hyperparameters, the number of possible configurations increases explosively (combinatorial growth). Thus, this method is practical with only a few hyperparameters, which makes this straightforward approach lose its advantages due to the high time complexity. Compared to grid search, random search usually works better when there are more hyperparameters [7,8,11], because it does not have to iterate over each single sampling point and can explore more combinations quickly. A genetic algorithm based on the evolutionary approach is also applied in hyperparameter tuning [5]. This is a bio-inspired method that conducts selection, crossover, and mutation behaviors [12]. It is gradient-free and robust to nonconvexity. Although it is a method for the global optimum, it is computationally costly and does not guarantee convergence to a global optimum. The Gaussian process also plays an important role in hyperparameter tuning. The likelihood estimation approach can be designed inside the main optimization loop to estimate the most likely hyperparameters based on the historical observations. It can tune hyperparameters in the kernel function inside Bayesian optimization based on the principle of maximizing the probability of the density function of the normal distribution. In contrast to grid, random, and evolutionary search, likelihood-based approaches exploit the surrogate structure and can update hyperparameters online using all past evaluations, which motivates the strategy adopted in this work.

Numerous studies have been reported to estimate the material model parameters in plasticity. As a foundational curve-fitting approach, linear regression has been used to calibrate the Ramberg–Osgood model [13]. The algorithm of the grid search method combined with random search was developed to identify steel stress–strain curves [14]. A pattern search algorithm was employed to find plasticity parameters for a wide range of metallic materials [15]. The gradient-based optimization was applied to the viscoplasticity problem [16]. In addition, the metaheuristic approaches such as genetic algorithms have been employed [17,18]. More recently, Bayesian optimization has been adopted for constitutive calibration for single-crystal plasticity [2] and rational-polynomial strain-hardening models [4]. However, to our knowledge, no prior work has performed hyperparameter tuning within the optimization loop for plasticity parameter identification, nor has any study employed maximum margin likelihood estimation inside Bayesian optimization to adapt kernel hyperparameters while characterizing strain-hardening behavior.

The aim of this study is to integrate hyperparameter tuning into the Bayesian optimization workflow and assess its impact on optimization performance. By adapting the Gaussian-process hyperparameters online tuning scheme, the proposed method is designed to improve sampling behavior and reduce the number of expensive finite element evaluations required for strain-hardening parameter identification in automotive structural steel. To achieve this goal, we reused the experimental data and finite element (FE) models from Long et al. [4]. An in-house Bayesian optimization code was first developed. Then, an online hyperparameter-tuning module was added into the optimization loop. To this end, the baseline version and online hyperparameter tuning version were compared for the strain-hardening material parameter characterization of automotive structure steel.

The remainder of this paper is organized as follows. Section 1 recalls the experiment, finite element model, and material model in the previous study [4], since they are the same materials to be reused. Section 2 reviews and introduces the Bayesian optimization formulation and hyperparameter tuning scheme in this work. Section 3 and Section 4 report and discuss the results, respectively. Finally, Section 5 concludes the paper and outlines future work.

2. Materials and Methods

In this section, the experiment, finite element model, and plastic material model from Long et al. [4] are first reviewed. Then, equations to build the in-house Bayesian optimization are introduced. This is followed by the formulas of hyperparameter tuning used inside the Bayesian optimization. The way to organize the content mentioned above is shown in Figure 1.

2.1. Recap of Experiment and Finite Element Setups

This study reused the experimental dataset finite element model reported in the literature [4]. Using this established setup allows us to isolate and analyze the influence of Bayesian optimization hyperparameters on convergence and accuracy, without introducing additional variability from new experiments or remodeling. As for the experiment, the automotive structural steel specimens were extracted from actual vehicle components rather than resources from standardized raw-material suppliers. Specifically, the same ASTM E8 flat sub-size tensile specimens described in the literature [19] were employed. Three specimens were tested: Specimen No.1 (Van ladder frame’s rear cross-member), Specimen No.2 (Van ladder frame’s front rail), and Specimen No.3 (body-in-white structural steel). To minimize the strain rate effect in the elastic-plastic process, the MTS 810.12 test machine was programmed to conduct the tensile test at a 0.0001 (1/s) strain rate (grip velocity with 0.1905 mm/min). As for the finite element model (unit system: mm, ms, kg, kN, and GPa), we applied prescribed displacement boundary conditions in the Z direction to the nodes within the purple and yellow rectangular regions in Figure 2d. These nodes were fixed in both X and Y to represent the grips, and the reaction force was obtained from the cross-sectional force along the red dashed line. After the mesh convergence study, the mesh (hexahedral element and single integration) size of 0.44 mm was selected. Again, the maximum displacements from experimental data we chose for the finite element simulations to fit are the points where the force begins to decrease [4]. Figure 2 summarizes the experimental and numerical setup: (a) the ASTM E8 flat sub-size specimen geometry [19], (b) the tensile test configuration and extensometer arrangement used to measure deformation, (c) the measured force–displacement curves for the three specimens as target data for parameter identification, and (d) the finite element model highlighting the applied boundary conditions and the cross-section from which the reaction force is extracted.

Figure 2. Experiment and its according finite element model [4]. (a) Standard ASTM E8 flat sub-size specimen (0.25 inch (6.35 mm) wide) [19]. (b) Demonstration of experimental measurement with hydraulic grip and extensometer. (c) Force vs. displacement responses from experiment. (d) Finite element model.

Figure 2. Experiment and its according finite element model [4]. (a) Standard ASTM E8 flat sub-size specimen (0.25 inch (6.35 mm) wide) [19]. (b) Demonstration of experimental measurement with hydraulic grip and extensometer. (c) Force vs. displacement responses from experiment. (d) Finite element model.

As mentioned above, the same rational polynomial-based material model for strain hardening described in [4] was adopted. For this material model, there are five parameters to be optimized, with three in the numerator and two in the denominator, as shown in Equation (1). Different parameter combinations of $p_1, p_2, p_3, q_1, q_2$ change the response between yield stress (${\sigma }_Y$) and effective plastic strain (${\overline{\epsilon }}^p$). This curve was written into the *MAT_024 (Young’s modulus = 210 GPa and Poisson’s ratio = 0.3 were determined previously), a piecewise plasticity model, in the LS-DYNA [20] to find the five parameters during the inverse finite element process.

$${\sigma }_Y={\displaystyle \frac{p_1{\left({\overline{\epsilon }}^p\right)}^2+p_2\left({\overline{\epsilon }}^p\right)+p_3}{{\left({\overline{\epsilon }}^p\right)}^2+q_1\left({\overline{\epsilon }}^p\right)+q_2}}$$

(1)

2.2. Bayesian Optimization

To implement hyperparameter tuning, writing our own in-house Bayesian optimization is the best option for us. In this way, it will be easier to add the hyperparameter function based on the basic version. Since the 5-term rational polynomial function was used, where $p_1$, $p_2$, $p_3$, $q_1$, and $q_2$ are the parameters to be identified in Equation (1), the vector $\mathit{\theta }=[p_1, p_2, p_3, q_1, q_2]$ represents the variable space to store the material parameters to be optimized. The Gaussian process is defined and shown in Equation (2), where $m(\mathit{\theta })$ and k are the prior mean and the covariance function taking $\mathit{\theta }$ and ${\mathit{\theta }}^{\prime }$ as inputs.

$$f\left(\mathit{\theta }\right)\sim \mathcal{N}\left(m\left(\mathit{\theta }\right),k\left(\mathit{\theta },{\mathit{\theta }}^{\prime }\right)\right)$$

(2)

For simplicity with fewer hyperparameters, the exponential kernel is selected as the kernel function k:

$$k\left(\mathit{\theta },{\mathit{\theta }}^{\prime }\right)=s^{2}exp\left[-{\displaystyle \frac{1}{2}}{\displaystyle \frac{{\left(\mathit{\theta },{\mathit{\theta }}^{\prime }\right)}^2}{{\ell }^2}}\right]$$

(3)

where s (amplitude scaling) and ℓ (length scale) are the hyperparameters [21]. The fixed hyperparameters can be first evaluated based on s = $0.5\sqrt{n}$ and ℓ = $\sqrt{m}$. n is the dimension of the parameter variables. m is the objective function order, which can be roughly estimated using several runs [22].

If we assume the prior mean to be zero (without enough information, this is a general practice [23,24,25,26]), the joint Gaussian distribution [23,27,28] vector form is expressed as follows:

$$\left[\begin{array}{c}{\mathbf{f}}_{1:t}\\ f_{t+1}\end{array}\right]\sim \mathcal{N}\left(0,\left[\begin{array}{cc}\mathbf{K}& \mathbf{k}\\ {\mathbf{k}}^T& k({\mathit{\theta }}_{t+1},{\mathit{\theta }}_{t+1})\end{array}\right]\right),$$

(4)

where

$$\mathbf{K}=\left[\begin{array}{ccc}k({\mathit{\theta }}_1,{\mathit{\theta }}_1)& \dots & k({\mathit{\theta }}_1,{\mathit{\theta }}_t)\\ ⋮& \ddots & ⋮\\ k({\mathit{\theta }}_t,{\mathit{\theta }}_1)& \dots & k({\mathit{\theta }}_t,{\mathit{\theta }}_t)\end{array}\right],$$

(5)

$$\mathbf{k}={\left[\begin{array}{cc}k({\mathit{\theta }}_{t+1},{\mathit{\theta }}_1)& k({\mathit{\theta }}_{t+1},{\mathit{\theta }}_2)\dots k({\mathit{\theta }}_{t+1},{\mathit{\theta }}_t)\end{array}\right]}^T,$$

(6)

$${\mathbf{f}}_{1:t}={\left[\begin{array}{cc}f\left({\mathit{\theta }}_1\right)& f\left({\mathit{\theta }}_2\right)\dots f\left({\mathit{\theta }}_t\right)\end{array}\right]}^T,$$

(7)

$$f_{t+1}=f_{t+1}\left({\mathit{\theta }}_{t+1}\right).$$

(8)

Following the procedure of the Sherman–Woodbury–Morrison identity, it naturally leads to the distribution of the completed parameter space [21]. Then, given any parameter space ${\mathit{\theta }}_{t+1}$ to be estimated, the corresponding posterior distributions of mean and variance are written as follows:

$$P\left(f_{t+1}|{\mathbb{D}}_{1:t},{\mathit{\theta }}_{t+1}\right)\sim \mathcal{N}\left(\mu \left({\mathit{\theta }}_{t+1}\right),{\sigma }^2\left({\mathit{\theta }}_{t+1}\right)\right),$$

(9)

where

$$u\left({\mathit{\theta }}_{t+1}\right)={\mathbf{k}}^T{\mathbf{K}}^{-1}{\mathbf{f}}_{1:t},$$

(10)

$${\sigma }^2\left({\mathit{\theta }}_{t+1}\right)=k({\mathit{\theta }}_{t+1},{\mathit{\theta }}_{t+1})-{\mathbf{k}}^T{\mathbf{K}}^{-1}\mathbf{k},$$

(11)

and ${\mathbb{D}}_{1:t}$ is the historical observations.

As more data are observed during the optimization process, the dimension of the covariance matrix, $\mathbf{K}$, increases. To compute the inverse of the covariance matrix accurately and effectively, the Cholesky decomposition [29] was applied, and the identity matrix scaled by the noise value, $1 \times {10}^{-4}$, was added to the covariance matrix in Equation (5) to maintain the numerical stability [30,31].

The lower confidence bound (LCB) in Equation (12), where $\kappa$ controls the trade-off between exploration and exploitation, was assigned with 0.5 to minimize the cost function in Equation (13) [22].

$${\alpha }_{LCB}(\mathit{\theta })=\mu \left(\mathit{\theta }\right)-\kappa \sigma \left(\mathit{\theta }\right),$$

(12)

$${\mathit{\theta }}_{t+1}=\mathrm{argmin}{\alpha }_{LCB}\left(\mu \left(\mathit{\theta }\right),\sigma \left(\mathit{\theta }\right)\right).$$

(13)

In this study, eight random datasets were generated to feed the model initially to start the optimization process. Each dataset is made of five material parameters and a residual value. The parameter space ($\mathit{\theta }$), as the input for Equation (13), was randomly generated to avoid local minima as much as possible. Same as the literature [4], root mean square error (RMSE) was again utilized to calculate the value of cost function [32].

After the global optimization by Bayesian optimization, the Nelder–Mead simplex algorithm, a local optimizer, refined the result. It inherited the best fit from Bayesian optimization as the initial guess in its algorithm. In this study, the Nelder–Mead simplex algorithm is achieved by the Python (3.13.5) package Scipy [33].

2.3. Hyperparameter Tuning

The performance of a Gaussian process (GP) depends on the hyperparameters. In this work, the hyperparameter in the covariance function in Equation (3) was designed to be optimized. A common and principal approach is to estimate these hyperparameters to maximize the marginal likelihood based on the observed data [21,34]. Given input observations, the hyperparameters, s and ℓ, in the kernel function in Equation (3) can be estimated by maximizing the log marginal likelihood function in Equation (14).

$$\mathcal{L}=logp\left({\mathbf{f}}_{1:t}\mid {\mathit{\theta }}_{1:t},\ell ,s\right)=-{\displaystyle \frac{1}{2}}{\mathbf{f}}^T{\mathbf{K}}^{-1}\mathbf{f}-{\displaystyle \frac{1}{2}}log\left|\mathbf{K}\right|-{\displaystyle \frac{n}{2}}log\left(2\pi \right).$$

(14)

In this study, the optimization method, L-BFGS-B, was used to maximize the log marginal likelihood function in Equation (14) to find the best hyperparameter s and ℓ. In other words, the maximum likelihood estimation (MLE) procedure needs to be done to find the best hyperparameter according to the current data observed. In a practical procedure, the hyperparameters were evaluated every 5 iterations in the Bayesian optimization.

3. Results

3.1. Force vs. Displacement Fit

Figure 3a, Figure 4a, and Figure 5a show the final optimization results for force-displacement responses (solid line for experiment and dashed line for simulation fit), while the yield stress vs. effective plastic strain curves are plotted in Figure 3b, Figure 4b, and Figure 5b. These strain-hardening curves correctly reflect the hardening process in plasticity, since the strain hardening means the slope is positive. The curve fit demonstrates that our online hyperparameter optimization framework can provide high-quality results. The best-fit rational polynomial parameters for each specimen are shown in

Table 1. Best fit coefficients of rational polynomial in Equation (1) from Bayesian with online hyperparameter tuning and Nelder–Mead simplex optimizations based in Table 2.

Specimen	${\mathit{p}}_1$	${\mathit{p}}_2$	${\mathit{p}}_3$	${\mathit{q}}_1$	${\mathit{q}}_2$
No.1	2292.7831	270,106.4510	7115.7027	429,882.9182	17,536.0385
No.2	3356.5040	263,855.2440	13,307.1132	441,846.1177	42,262.0418
No.3	82.6904	299,899.8547	6969.02445	300,938.0988	15,976.2444

. The lower and upper bounds of Bayesian optimization are shown in Appendix A. Since Nelder–Mead simplex method, as the local fine-tuning algorithm, is designed to be unbounded in this study, some of the final parameters after the fine-tuning in Table 1 are outside the bounds.

3.2. Convergence History

In this manuscript, since we are only interested in the performance of hyperparameter tuning in Bayesian optimization, the differences are focused on the performance between Bayesian optimization with and without hyperparameter tuning in Figure 6. Since the online hyperparameter tuning version gives better results, the follow-up local optimization by Nelder–Mead simplex algorithm is only conducted for the online hyperparameter tuning version. This way, the curves in the figure will not be too crowded.

Obvious improvements are shown in residual values for Bayesian optimization with an online hyperparameter tuning scheme. With the same initial residual value for each specimen, the residual values by online hyperparameter tuning in

Table 2. Residual values by Bayesian optimization with online hyperparameter tuning using maximum likelihood estimation (Online-MLE BO). Residual values by Nelder–Mead simplex use the Online-MLE BO results as the initial guesses.

Specimen	Online-MLE BO Initial	Online-MLE BO Final	Nelder–Mead Simplex
No.1	1.1128	0.4352	0.3783
No.2	1.0953	0.4314	0.4215
No.3	1.1586	0.2116	0.2019

decrease faster than the Bayesian baseline version in

Table 3. Residual values by baseline version Bayesian optimization (Baseline BO).

Specimen	Baseline BO Initial	Baseline BO Final
No.1	1.1128	0.5464
No.2	1.0953	0.5679
No.3	1.1586	0.2199

. The residual value of sample No.3 in Table 2 is significantly lower than the other two samples. This can be explained by the fact that the samples No.1 and No.2 have a “platform” right after the yield point. However, sample No.3 does not. For steel materials, it is common for the response to either exhibit a clear yield “platform” or have no distinct “platform”. To simulate that “platform” behavior needs a material model beyond the piecewise linear plasticity model and the rational material model that the current paper is based on.

For the same number of iterations, the final values of the baseline Bayesian optimization and online hyperparameter Bayesian optimization schemes are compared qualitatively.

Table 4. Residual values improvement by Online-MLE BO final using Baseline BO final as reference.

Specimen	No.1	No.2	No.3
Percentage (%)	20.3514	24.0359	3.7744

shows the online hyperparameter scheme reduced the final residual by approximately 20.4%, 24.0%, and 3.8% for Specimens 1–3, respectively.

3.3. Hyperparameter Tuning History

During Bayesian optimization, kernel hyperparameters (s and ℓ) were reestimated online by maximizing the marginal likelihood function in Equation (14) and the hyperparameter tuning history is shown in Figure 7. Hyperparameter updates were performed every five iterations. Between updates, the last hyperparameters by the MLE process were held fixed. In the early phase (up to roughly iteration 200), hyperparameters show substantial variability as it explored the space. Beyond 200 iterations, the hyperparameter becomes stabilized with around three for s and slightly less than 0.4 for ℓ with only minor fluctuations.

4. Discussion

Our results demonstrate that the online hyperparameter tuning obviously provides a better performance for Bayesian optimization. In this section, we first analyze the costs and benefits of performing online hyperparameter estimation by maximum likelihood. We then compare the present Bayesian optimization results with those reported in the literature [4] and discuss the specimen-to-specimen variation in improvement achieved by the online hyperparameter tuning scheme. Finally, we examine whether using the hyperparameters obtained in the final iterations as fixed values in a baseline Bayesian optimization scheme is an effective and practical strategy.

Online hyperparameter tuning improves the overall optimization performance. However, it is not free. For each of the several iterations, it requires Cholesky factorization to solve the inverse of the covariance matrix, which is then passed to the MLE function to use an optimization solver to estimate the hyperparameter. Like the Bayesian optimization baseline code, as the iteration goes up, it takes more computational cost to find the kernel function inverse, and this naturally increases the hyperparameter tuning time. However, compared with costly objectives such as finite element simulations, this is typically negligible.

Although the results by Long et al. [4] were obtained by open-source code [30] with a different software architecture (e.g., Matérn kernel), it is still valuable to make a comparison because both studies used the same material model, finite element model, experimental dataset, and Bayesian optimization iteration numbers. Compared with the final Bayesian optimization residual values, our Online-MLE BO offers slightly lower values. Online-MLE BO final residual values in the second column in Table 2 are lower than the BO final residual values reported in the literature [4] by around 9% for each specimen (No.1: 9.1441%, No.2: 8.9873%, No.3: 8.86075%), which indicates that our performance is making sense and is even better. Accordingly, the values refined by the local optimizer, the Nelder–Mead simplex method, obtain better results.

Focusing on the three specimens, different improvement levels by the online hyperparameter tuning scheme across the three specimens are shown in Table 4. The performance of Bayesian optimization, as a stochastic and statistics-based method, is also influenced by randomness in the sampling process. For specimens No.1 and No.2, the initial hyperparameters were not as suitable as specimen No.3 and thus online adaptation could significantly refine the surrogate model, leading to residual reductions of about 20–24%. For specimen 3, the fixed hyperparameters already worked reasonably well, so the online adaptation still helps but yields a smaller additional improvement of about 3.8%.

The online tuning scheme obviously helps in the optimization process. However, whether the tuned hyperparameter at the last time step (Baseline-Fixed-Final BO) ultimately performs better than the online hyperparameter (Online-MLE BO) or even baseline Bayesian optimization with fixed hyperparameters (Baseline BO) remains a question. Thus, the residual histories of the three cases are compared and shown in Figure 8. In this figure, Online-MLE BO residual values typically reduce faster and reach lower residuals in fewer iterations than Baseline BO. Baseline-Fixed-Final BO often performs better than Baseline BO, but it usually does not show as good performance as BO Online MLE in the early or middle phases. However, Baseline-Fixed-Final BO is not very practical since the Online-MLE BO needs to run first to provide the hyperparameter information.

5. Conclusions and Further Work

In this study, we incorporated online hyperparameter tuning by adopting the likelihood function into a Bayesian optimization framework for identifying strain-hardening parameters in automotive structural steel. By adaptively updating the Gaussian-process kernel hyperparameters during the sampling, this approach improved the optimization quality as shown in Table 4. These results show that the online hyperparameter tuning scheme helps the Bayesian optimization sample the parameter in a smarter way for the inverse finite element method. For researchers and engineers, the proposed strategy offers a practical way to obtain better strain-hardening parameters more efficiently.

Future work will investigate broader strategies for hyperparameter estimation (including acquisition-function and noise parameters) and extend this approach to other materials and loading conditions to further assess its robustness and applicability.