Comparison of Sensitivity-Guided and Black-Box Machine Tool Parameter Identification

Abstract

Dynamic machine tool simulation models can be used for various applications such as process simulations, design optimization, and condition monitoring. However, all these applications require that the model replicates the real system’s behavior as accurately as possible. Next to carefully building the model, the parameterization of the model, that is, determining the parameter values the model is based upon, is the most crucial step. This paper describes the application of both sensitivity-based and black-box parameter identification to a machine tool. It further provides a comparison between these two methods and the method of sequential assembly. It is shown that both methods can increase the mode shape conformity by more than 25% and significantly reduce damping deviations. However, sensitivity-based parameter identification is the most economical method, offering the chance to update a dynamic machine tool model within minutes.

1. Introduction

To maximize productivity, cutting processes have to be carefully designed. Otherwise, either the machine tool’s capability is not fully exploited or the machine tool, the cutting tool, or the workpiece are at risk of damage due to dynamic instabilities. The design of cutting processes requires a solid understanding of both the process and the machine tool’s dynamics [1].

While the former is not the subject of this publication, the latter can be identified via experimental modal analyses (EMAs), leading to a gray-box or black-box model of the machine tool dynamics. In the past, this was only conducted in laboratory conditions using impact hammers, shakers, or machine tool drives as excitation sources, which is a time-consuming process and subject to many inaccuracies [2]. Operational modal analysis was designed to overcome these problems [3]. The so-called “rapid identification” approach, in particular, uses in situ computer numerical control data but also only focuses on single-input single-output dynamics [4]. While operational modal analysis is very accurate, it is generally only valid in close vicinity to the working position at which the data were collected and cannot be interpolated and extrapolated. This limits its value for complex process simulations, which generally involve a larger portion of the machine’s working space in contrast to a limited number of working points.

Alternatively, machine tool dynamics can be calculated by means of bottom-up theoretical models. In simple cases, these models can be derived analytically starting from the underlying equations of motion [5,6,7]. More complex but also more informative models can be built using finite element analysis (FEA) methods [8,9,10,11]. In recent years, these models have become position-flexible and computationally very efficient due to intelligent substructuring and elaborate model order reduction methods [9].

Even though dimensionality reduction methods exist [12], the increasing complexity of the models also increases the number of unknown model parameters. These parameters represent the material properties of the machine tool (e.g., density and Young’s modulus), the properties of machine tool joints replaced by surrogate models (e.g., spring stiffnesses and viscous dampers), and, in some cases, the properties of additional friction models (e.g., see Rebelein [10]). To achieve high model accuracy, these parameters must be carefully identified, meaning that values for them must be found that reflect the real physical properties of the machine tool. In the literature, many works on this exist, ranging from parameter identification for simplified feed drive structures [5,13] to complete machine tool systems [11,14,15,16]. The unknown parameters are either directly determined on the assembled system [5,11,13,16] or identified on separate, carefully designed test benches [10,14]. The former method leads to a challenging optimization problem [11,16,17], while the latter may result in parameters that are only valid on the test bench but not for the overall machine tool system [18]. To overcome this problem, the method of sequential assembly has been introduced [10,14]. Here, the machine tool is incrementally built up, starting, for example, with the machine tool bed until its final shape is reached. In parallel, a model is built, always matching the machine tool’s assembly state. This way, only a few parameters have to be identified in each step, reducing the overall complexity of the parameter identification problem. However, this approach is often uneconomical and only feasible if the machine tool can be disassembled and assembled at will.

Different strategies exist for parameter identification on complete machine tools: Garitaonandia et al. [19] used a Bayesian optimization algorithm to find optimal parameters for the ball screw drive (BSD) and the mounting elements (MEs) based on three manually selected eigenmodes of a grinding machine. Hernandez-Vazquez et al. [16] identified machine tool parameters using a least squares (LS) approach and found it beneficial to consider two machine tool axes positions in the objective function rather than just one. To decrease the simulation time, Hernandez-Vazquez et al. [11] first built a response surface surrogate model of the original FEA model and, second, solved it for the unknown machine tool parameters also using a LS algorithm. In contrast, Semm et al. [14] have exploited the computational efficiency of their model and combined it with a genetic algorithm, a particle swarm optimization (PSO), and a deterministic sequential least squares programming (SLSQP) algorithm to identify unknown machine tool parameters. They found the PSO to deliver the best and most stable results. Their model was reused in Ellinger and Zaeh [17], where, in contrast to the presented approaches, the overall optimization problem was first partitioned into many smaller subproblems by means of global sensitivity analyses (GSAs). Afterward, SLSQP was used to determine the unknown model parameters. By using simulated reference data (instead of data acquired via measurements on the real-world machine tool), it was shown that optimal and globally valid parameters can be found. However, validating the identification strategy on a real-world machine tool was denoted as a field for further research.

The present paper aims to validate the approach first presented in Ellinger and Zaeh [17] on a real-world machine tool and to compare it with a state-of-the-art PSO parameter identification approach. Furthermore, it will be shown that, in contrast to other strategies, it can be automated to a high extent and thus has the potential to be economically applied in modern production environments. For that, the remainder of the paper is structured as follows: Section 2 briefly recalls the parameter identification methods applied in this work. Furthermore, model conformity measures are presented, which are important for both designing the objective functions of the underlying optimization problems and assessing the results. In Section 3, the considered machine tool structure, the corresponding model, and the measurements performed to acquire input data for parameter identification are described. Afterward, the application of the sensitivity-guided (see Section 3.2) and the black-box (see Section 3.3) parameter identification approaches is described, and the corresponding results are presented. In Section 3.4, both methods are compared regarding the resulting model accuracy and the required economic effort. The paper is concluded in Section 4, with a summary and the outlook for future research.

2. Background

This section presents background information intended to guarantee the understandability of the methods used in this paper. First, model evaluation criteria are introduced, which can be used to quantitatively assess the quality of structural dynamic models. Second, the algorithms used to identify the unknown model parameters in Section 3 are briefly explained, and the strategy of sensitivity-guided parameter identification [17] is recapitulated.

2.1. Model Evaluation Criteria

The most straightforward way to assess the structural dynamic behavior of a system is calculating frequency response functions (FRFs) based on time-domain measurements. A well-parameterized model will show the same input–output behavior as the real-world system. Thus, one way to evaluate a model’s quality is by comparing measured and calculated FRFs. This can be achieved using the frequency response assurance criterion (FRAC) [20]

$$\begin{array}{c}\hfill \mathrm{FRAC}\left({\mathit{H}}_c\right(\omega \left)\right)=\frac{{|{\mathit{H}}_c{\left(\omega \right)}^T{\mathit{H}}_m^{\ast }\left(\omega \right)|}^2}{\left({\mathit{H}}_c{\left(\omega \right)}^T{\mathit{H}}_c^{\ast }\right(\omega \left)\right)\left({\mathit{H}}_m{\left(\omega \right)}^T{\mathit{H}}_m^{\ast }\right(\omega \left)\right)},\end{array}$$

(1)

where ${\mathit{H}}_c$ and ${\mathit{H}}_m$ denote the calculated and measured FRFs, $\omega$ represents the angular frequency, and ${(•)}^T$ and ${(•)}^{\ast }$ denote the transpose and complex conjugation operations. A FRAC value of 100% indicates perfect conformity, and a value of 0% no match at all.

The modal properties of the system are closely related to the measured FRFs [7]. A good model will, in turn, match the modal behavior of the real-world system, meaning that it has the same eigenvectors, eigenfrequencies, and modal damping. The match between the measured and calculated eigenvectors can be assessed using the modal assurance criterion (MAC) [21]

$$\begin{array}{c}\hfill \mathrm{MAC}({\mathit{\phi }}_c,{\mathit{\phi }}_m)=\frac{{|{\mathit{\phi }}_c^T{\mathit{\phi }}_m^{\ast }|}^2}{\left({\mathit{\phi }}_c^T{\mathit{\phi }}_c^{\ast }\right)\left({\mathit{\phi }}_m^T{\mathit{\phi }}_m^{\ast }\right)}\end{array}$$

(2)

with ${\mathit{\phi }}_c$ and ${\mathit{\phi }}_m$ being the calculated and measured eigenvectors and, again, ${(•)}^T$ and ${(•)}^{\ast }$ denoting the transpose and complex conjugation operations. Similar to the FRAC, the MAC range is 0–100% for totally unrelated to perfectly matching eigenvectors. The model’s eigenfrequencies can, for example, be compared using the natural frequency difference defined by Imamovic [22]. Similarly, the model’s modal damping can be assessed using the natural damping difference $\left(\mathrm{NDD}\right)$ or, for the use in optimization algorithms (see Section 2.2), the squared natural damping difference $\left({\mathrm{NDD}}^2\right)$:

$$\begin{array}{c}\hfill {\mathrm{NDD}}^2({\xi }_c,{\xi }_m)={(\mathrm{NDD}({\xi }_c,{\xi }_m)}^2={\left(\frac{{\xi }_c-{\xi }_m}{{\xi }_m}\right)}^2\end{array}$$

(3)

Here ${\xi }_c$ and ${\xi }_m$ denote the calculated and measured modal damping, respectively. An ${\mathrm{NDD}}^2$ of 0% indicates perfect conformity between the model and the real-world system, and it increases with rising damping differences.

2.2. Optimization Strategies and Algorithms

Parameter identification can be seen as a constrained nonlinear optimization problem, which can be formulated as

$$\begin{array}{c}\hfill \underset{\mathit{x}\in {\mathbb{R}}^n}{min}f\left(\mathit{x}\right)\end{array}$$

(4)

subject to

$$\begin{array}{c}\hfill g_j\left(\mathit{x}\right)=0,j=1,\dots ,m_e,\end{array}$$

(5)

$$\begin{array}{c}\hfill g_j\left(\mathit{x}\right)\ge 0,j=m_e+1,\dots ,m,\end{array}$$

(6)

$$\begin{array}{c}\hfill \mathrm{and}{\mathit{x}}_l\le \mathit{x}\le {\mathit{x}}_u,\end{array}$$

(7)

with $\mathit{x}$ being a vector grouping the n unknown parameters with lower and upper bounds ${\mathit{x}}_l$ and ${\mathit{x}}_u$, the objective function f, constraint functions $g_j$, and the $m_e$ equality constraints and the m total constraints [23]. In the case of structural dynamic parameter identification, the model is involved in the calculation of the objective function f, and $\mathit{x}$ represents the unknown model parameters. For example, f can be the average FRAC value of all considered FRFs, and the calculated FRFs ${\mathit{H}}_{c,i}(\omega , \mathit{x})$ each depend on the model parameters $\mathit{x}$.

One of the most efficient methods for solving constrained nonlinear optimization problems is sequential least squares programming (SLSQP) [23]. This method generates a new estimate of the optimal parameters by solving a quadratic subproblem. Further details can be found in both Kraft [23] and Nocedal and Wright [24]. However, it is to be noted that SLSQP is a so-called local method that finds the optimum parameters $\mathit{x}$ in the vicinity of the starting point but not necessarily the global optimum in the overall parameter space defined by Equation (7).

Global optimization algorithms do not suffer from this drawback. Some of them are inspired by evolutionary processes such as the movement of a swarm of birds, which has lead to the development of particle swarm optimization (PSO) [25]. Here, a swarm of particles, each representing a parameter estimate, moves through the parameter space and, ideally, converges to the global optimum of the objective function. The movement of a particle i is defined by its velocity, which is updated in every iteration by

$$\begin{array}{c}\hfill {\mathit{v}}_{i,k+1}=u_1{\mathit{v}}_{i,k}+u_2({\mathit{p}}_i-{\mathit{x}}_{i,k})+u_3({\mathit{p}}_g-{\mathit{x}}_{i,k})\end{array}$$

(8)

where k is the iteration number, ${\mathit{v}}_i$ is the velocity of the particle, ${\mathit{p}}_i$ is the location with the particle’s individual best value of the cost function, ${\mathit{x}}_i$ is the particle’s current position in the parameter space, ${\mathit{p}}_g$ is the location of the swarm’s best value of the cost function, and $u_1$, $u_2$, and $u_3$ are user-defined weights controlling the particle’s inertial, nostalgic, and social behavior [25]. In each iteration, the particle’s position is updated as

$$\begin{array}{c}\hfill {\mathit{x}}_{i,k+1}={\mathit{x}}_{i,k}+{\mathit{v}}_{i,k}.\end{array}$$

(9)

However, global optimization in general and PSO in particular are not as efficient as local methods and may require many function evaluations (i.e., once per particle in each iteration). Additionally, these algorithms may still end up in local minima of the objective function, especially if the global minimum is not very prominent. To overcome these problems, a method called “sensitivity-guided parameter identification” was developed by Ellinger and Zaeh [17]. First, they partitioned the overall parameter identification problem comprising many unknown parameters into several smaller optimization problems, each with only a handful of unknown parameters, by means of GSA. This already reduces the risk of local minima. Second, these smaller problems are repeatedly solved using varying initial values and a local optimization algorithm such as, in this case, SLSQP. If there are still local minima, these runs still end up at different points in the parameter space. However, for each parameter, the suboptimization problem with the least variance is determined, and the final value for the parameter is calculated as the mean value of all repetitions. This way, the likelihood of ending up in local minima can be further reduced. More details can be found in Ellinger and Zaeh [17].

3. Results

In this section, the application of two parameter identification approaches to a real machine tool system is described, which was described first in Section 3.1. Additionally, information on the measurement data is given. Next, Section 3.2 shows the results of applying sensitivity-guided parameter identification to the system, while Section 3.3 does the same for a black-box method. Last, both approaches are discussed and compared to one another in Section 3.4. Additionally, the method of sequential assembly is used as a benchmark from the literature.

3.1. Machine Tool Model Description

The subject of the investigation in this paper is a DMG DMC duo Block 55H machine tool in a uniaxial configuration, as depicted in Figure 1. For four different workpiece table (WPT) positions ($z_1$ = −294 mm, $z_2$ = −60 mm, $z_3$ = 135 mm, and $z_4$ = 330 mm) evenly spread across its full motion range, acceleration data were acquired at a sampling rate of 10,240 Hz at 17 nodes all across the machine tool in response to impulse hammer excitations at node $N_8$ in all three coordinate directions. Three repetitions were performed for each measurement. This was conducted using four Kistler^® triaxial accelerometers (two times type 8762A10 and two times type 8762A50) and a National Instruments (NI)^® cDAQ-9198 rack with three type NI^®-9232 modules and one type NI^®-9234 module. The measurement nodes and the excitation node are briefly described in

Table 1. Considered model and measurement nodes. The node locations are illustrated in Figure 1b.

Node	Description
N${}_{1a,b}$, N${}_{2a,b}$, N${}_{3a,b}$, N${}_{4a,b}$	Shoe (a) and rail (b) nodes of the linear guiding system (LGS) shoes
N${}_5$, N${}_6$, N${}_7$	ME nodes
N${}_8$	Excitation node
N${}_9$, N${}_{10}$, N${}_{11}$, N${}_{12}$	Column nodes
N${}_{13}$	WPT node

and illustrated in Figure 1b. Figure 1a shows a photograph of the real machine tool, which was digitally edited to highlight the measured components. The measurements were performed in a uniaxial configuration by separating the x-axis (and the y-axis) from the machine tool bed and supporting it with a crane a short distance apart. This allows achieving the best possible separation of these components from the machine bed without disconnecting the electric cables and the hydraulic lines could be achieved. Altogether, this resulted in 153 FRFs per WPT position. Using the approach presented by Ellinger et al. [26], modal parameters (i.e., a set of modal damping, eigenvector, and eigenfrequency for each mode) were extracted at each of the four WPT positions, which served as a basis for the parameter identification methods. For each of the four positions, 26–42 modes were found in the range from 10 Hz to 500 Hz.

For this system, a simulation model already exists, which is based on FEA and has been proven to be capable of capturing the system’s dynamic behavior with high fidelity [27]. Additionally, the model has been made position-flexible and computationally efficient via an elaborate combination of model order reduction and linearization techniques [28,29]. The model was further subjected to a dimensionality reduction step [12], resulting in the 29 unknown model parameters listed in

Table 2. The 29 unknown machine tool model parameters; stiffness parameters are denoted by k, viscous and hysteretic damping parameters by b and d, respectively. x, y, and z represent translational and $rx$, $ry$, and $rz$ rotational degrees of freedom.

Element	Number	Stiffness	Damping
			Viscous	Hysteretic
MEs	3	$k_x,k_y,k_z$	$b_x,b_z$	$d_y$
BSD fixed bearing	1	$k_z$	-	-
LGS shoes	1 *	$k_x,k_y,k_{rx},k_{ry}$	-	-
BSD	1	$k_z$	-	$d_{rz}$
Coupling	1	$k_{rz}$	-	$d_{rz}$
LGS linearization parameters	1 *	$k_z$	$b_z$	-

* Same set of parameters used for all four LGS shoes.

. Based on previous model identification efforts, the initial values for these parameters are known. More information on the used model can be found in Zaeh et al. [27], Semm et al. [28], and Semm et al. [29].

The measurement data, which will also be called “reference data”, and the machine tool simulation model serve as a basis for the parameter identification described in Section 3.2 and Section 3.3.

3.2. Sensitivity-Guided Parameter Identification

In preceding work [17], sensitivity-guided parameter identification was demonstrated using simulated reference data originating from the very same model in which it was used to parameterize. Thus, on the one hand, the ground truth values for the parameters to be identified (i.e., the values used in the reference data generation) were known. On the other hand, the model could also be guaranteed to perfectly replicate the reference data (i.e., by using the ground truth model parameter values) or, in other words, no modeling errors exist. Since measured data were used in this work, both premises no longer hold. Sensitivity-guided parameter identification relies on mode tracking [17]. To ensure the approach’s operability, the reference data set was reduced to modes that, to some extent, also show up in the uncalibrated model (i.e., the model with the initial values for the parameters in Table 2). This was enforced by selecting only reference modes with a partner model mode with MAC and extended modal assurance criterion [30] conformities of at least 50% and a maximum extended modal assurance criterion conformity to all other modes of 30%. This procedure resulted in a reference data set with nine modes for positions $z_1$ and $z_3$, eight modes for position $z_2$, and seven modes for position $z_4$. The selected modes and their eigenfrequencies are displayed in

Table 3. Selected reference modes from the measured input data.

	Position ${\mathit{z}}_1$	Position ${\mathit{z}}_2$	Position ${\mathit{z}}_3$	Position ${\mathit{z}}_4$
1	Mode 3 ( 31.0 Hz)	Mode 2 ( 31.0 Hz)	Mode 1 ( 28.7 Hz)	Mode 2 ( 28.8 Hz)
2	Mode 6 ( 52.0 Hz)	Mode 4 ( 52.1 Hz)	Mode 2 ( 31.0 Hz)	Mode 3 ( 31.0 Hz)
3	Mode 7 ( 76.1 Hz)	Mode 6 ( 95.0 Hz)	Mode 4 ( 52.1 Hz)	Mode 6 ( 52.1 Hz)
4	Mode 8 ( 91.9 Hz)	Mode 7 (108.1 Hz)	Mode 6 ( 97.9 Hz)	Mode 8 ( 99.1 Hz)
5	Mode 9 ( 95.7 Hz)	Mode 8 (120.0 Hz)	Mode 7 (107.0 Hz)	Mode 9 (106.1 Hz)
6	Mode 10 (109.1 Hz)	Mode 11 (180.4 Hz)	Mode 8 (122.4 Hz)	Mode 11 (127.3 Hz)
7	Mode 12 (117.3 Hz)	Mode 14 (226.3 Hz)	Mode 9 (180.0 Hz)	Mode 13 (179.1 Hz)
8	Mode 13 (180.1 Hz)	Mode 16 (251.5 Hz)	Mode 12 (231.2 Hz)	-
9	Mode 14 (222.4 Hz)	-	Mode 15 (251.8 Hz)	-

.

Since both the GSAs for partitioning the parameter identification problem and the actual identification [17] can be highly parallelized, the method scales very well with the number of CPU cores. In total, 1,966,080 model evaluations were conducted for the GSAs, and each resulting optimization problem was executed 50 times (see Ellinger and Zaeh [17]). On a workstation with $2\times 48$ AMD^® EPYC^® 7642 cores, the whole calculation took 44 min, with 35 min for the GSAs and 9 min for the actual parameter identification. The final parameter values can be found in the appendix in

Table A1. Final parameter values found by the sensitivity-guided (SG) and black-box (BB) parameter identification approaches. The parameters are further described in Table 2.

Element	Parameter	SG	BB
BSD	$d_{rz}$ in $\mathrm{N} \mathrm{m}{\mathrm{m}}^{-1}$	$2.63\times {10}^4$	$1.84\times {10}^5$
	$k_z$ in $\mathrm{N} \mathrm{m}{\mathrm{m}}^{-1}$	$4.69\times {10}^5$	$5.02\times {10}^5$
Coupling	$d_{rz}$ in $\mathrm{N} \mathrm{m}{\mathrm{m}}^{-1}$	$8.23\times {10}^5$	$5.54\times {10}^6$
	$k_{rz}$ in $\mathrm{N} \mathrm{m}{\mathrm{m}}^{-1}$	$1.07\times {10}^5$	$1.21\times {10}^7$
Fixed bearing	$k_z$ in $\mathrm{N} \mathrm{m}{\mathrm{m}}^{-1}$	$1.54\times {10}^6$	$1.43\times {10}^6$
LGS	$b_z$ in $\mathrm{N}\mathrm{s} \mathrm{m}{\mathrm{m}}^{-1}$	$1.26\times {10}^0$	$1.48\times {10}^0$
	$k_{ry}$ in $\mathrm{N} \mathrm{m}{\mathrm{m}}^{-1}$	$1.88\times {10}^9$	$1.59\times {10}^9$
	$k_{rz}$ in $\mathrm{N} \mathrm{m}{\mathrm{m}}^{-1}$	$9.53\times {10}^8$	$6.90\times {10}^8$
	$k_x$ in $\mathrm{N} \mathrm{m}{\mathrm{m}}^{-1}$	$1.10\times {10}^6$	$1.29\times {10}^6$
	$k_y$ in $\mathrm{N} \mathrm{m}{\mathrm{m}}^{-1}$	$1.36\times {10}^6$	$1.52\times {10}^6$
	$k_z$ in $\mathrm{N} \mathrm{m}{\mathrm{m}}^{-1}$	$6.99\times {10}^3$	$7.63\times {10}^3$
ME1	$b_x$ in $\mathrm{N}\mathrm{s} \mathrm{m}{\mathrm{m}}^{-1}$	$6.63\times {10}^0$	$1.45\times {10}^1$
	$b_z$ in $\mathrm{N}\mathrm{s} \mathrm{m}{\mathrm{m}}^{-1}$	$3.34\times {10}^0$	$1.93\times {10}^1$
	$d_y$ in $\mathrm{N} \mathrm{m}{\mathrm{m}}^{-1}$	$1.10\times {10}^4$	$1.41\times {10}^3$
	$k_x$ in $\mathrm{N} \mathrm{m}{\mathrm{m}}^{-1}$	$3.77\times {10}^4$	$6.34\times {10}^4$
	$k_y$ in $\mathrm{N} \mathrm{m}{\mathrm{m}}^{-1}$	$2.31\times {10}^5$	$2.52\times {10}^5$
	$k_z$ in $\mathrm{N} \mathrm{m}{\mathrm{m}}^{-1}$	$4.73\times {10}^4$	$4.77\times {10}^4$
ME2	$b_x$ in $\mathrm{N}\mathrm{s} \mathrm{m}{\mathrm{m}}^{-1}$	$2.01\times {10}^1$	$6.84\times {10}^0$
	$b_z$ in $\mathrm{N}\mathrm{s} \mathrm{m}{\mathrm{m}}^{-1}$	$3.34\times {10}^0$	$1.67\times {10}^1$
	$d_y$ in $\mathrm{N} \mathrm{m}{\mathrm{m}}^{-1}$	$1.75\times {10}^4$	$1.04\times {10}^5$
	$k_x$ in $\mathrm{N} \mathrm{m}{\mathrm{m}}^{-1}$	$2.42\times {10}^5$	$1.59\times {10}^5$
	$k_y$ in $\mathrm{N} \mathrm{m}{\mathrm{m}}^{-1}$	$3.46\times {10}^5$	$3.00\times {10}^5$
	$k_z$ in $\mathrm{N} \mathrm{m}{\mathrm{m}}^{-1}$	$1.01\times {10}^5$	$1.17\times {10}^5$
ME3	$b_x$ in $\mathrm{N}\mathrm{s} \mathrm{m}{\mathrm{m}}^{-1}$	$3.34\times {10}^0$	$1.42\times {10}^1$
	$b_z$ in $\mathrm{N}\mathrm{s} \mathrm{m}{\mathrm{m}}^{-1}$	$3.01\times {10}^1$	$4.00\times {10}^0$
	$d_y$ in $\mathrm{N} \mathrm{m}{\mathrm{m}}^{-1}$	$1.40\times {10}^4$	$1.94\times {10}^4$
	$k_x$ in $\mathrm{N} \mathrm{m}{\mathrm{m}}^{-1}$	$6.35\times {10}^4$	$7.12\times {10}^4$
	$k_y$ in $\mathrm{N} \mathrm{m}{\mathrm{m}}^{-1}$	$3.36\times {10}^5$	$3.88\times {10}^5$
	$k_z$ in $\mathrm{N} \mathrm{m}{\mathrm{m}}^{-1}$	$1.44\times {10}^5$	$1.13\times {10}^5$

. In the case of repeated parameterization (e.g., due to changes of the real-world system), the GSAs would not have to be repeated, making the sensitivity-guided parameter identification a computationally efficient method.

Figure 2 shows an exemplary FRF from the measurement data set (“Reference”) and its simulated counterparts using the initial and found parameter values. Even though a significant improvement measured in terms of the FRAC value of roughly 15% could be reached via sensitivity-guided parameter identification, there is low overall conformity between the measured and simulated data. This is confirmed by

Table 4. FRAC value statistics for each WPT position with the initial parameters and after the sensitivity-guided (SG) and black-box (BB) parameter identification approaches.

FRAC in %		Median	Mean	Best
Position $z_1$	Initial	2.4	11.9	26.3
	SG	9.3	17.3	56.3
	BB	41.5	18.5	45.1
Position $z_2$	Initial	0.8	9.9	34.9
	SG	7.6	12.7	53.8
	BB	43.9	13.6	45.0
Position $z_3$	Initial	1.5	8.8	32.0
	SG	6.1	10.3	52.5
	BB	36.2	11.1	44.5
Position $z_4$	Initial	0.8	8.2	21.0
	SG	7.0	8.6	33.4
	BB	26.8	9.9	30.5

, which shows FRAC value statistics for simulated FRFs using the initial and found parameters with respect to their measured counterparts.

However, the low FRAC conformity between the parameterized model and the measurement data is not necessarily a drawback of the parameterization method, as other error sources exist along the way from the measurement data to the simulated FRFs: First, the modal parameter extraction leads to a modal model that can no longer perfectly replicate the original input data due to nonlinear effects of the system and errors in the modal parameter calculation. Second, modeling errors are present, meaning that, even with theoretically perfect model parameters, the model cannot entirely reproduce the behavior of the real-world structure. Last, imperfect parameter identification leads to further deviations between the simulated and measured FRFs. For the data used in this work, the modal model shows, on average, a FRAC conformity with the measurement data below 90%, with one-tenth of the FRFs being even below 82% [26]. The effect of the parameterization and modeling errors cannot be quantified. However, the latter is believed to be more significant than the former. Even though the model has shown a high FRAC conformity of 82% and higher in the past [10], this was only for one WPT position, which differs from the positions considered here. Furthermore, frequent BSD and LGS changes have been conducted since then, possibly introducing other non-modeled effects. Additionally, creating a uniaxial setup by hanging two axes on a crane did not lead to total separation, since electric wires and hydraulic lines were not disconnected. Even though these were supported as much as possible, this arrangement may have caused other nonlinearities, further reducing the model’s validity.

To more realistically evaluate the capabilities of the sensitivity-guided parameter identification approach, modal-based conformity measures can be used, eliminating fitting errors in determining the modal parameters [26].

Table 5. Comparison of MAC values between the reference modes and the simulated modes using the model with the initial parameter values and the resulting values from the sensitivity-guided (SG) and black-box (BB) parameter identification approaches.

MAC in %		1	2	3	4	5	6	7	8	9
Position $z_1$	Initial	81.8	86.6	69.1	96.0	55.1	75.6	87.0	81.1	52.8
	SG	92.8	88.1	90.5	96.6	54.9	77.7	87.4	80.9	53.1
	BB	65.3	92.2	84.1	96.8	54.6	78.8	87.3	81.2	54.7
Position $z_2$	Initial	85.0	89.3	97.8	87.9	94.5	96.5	83.3	61.5	-
	SG	92.9	90.4	97.8	91.5	94.3	96.5	83.5	61.8	-
	BB	69.1	94.5	97.3	93.0	94.0	96.6	84.3	60.9	-
Position $z_3$	Initial	58.5	86.6	83.7	96.6	82.6	94.9	94.4	82.2	59.0
	SG	84.5	93.2	84.5	98.4	89.2	94.9	94.3	82.5	59.1
	BB	89.0	73.3	86.2	98.7	91.9	94.9	94.4	82.4	58.4
Position $z_4$	Initial	80.4	88.3	89.4	83.7	75.3	89.9	61.6	-	-
	SG	85.8	92.4	90.2	92.0	88.4	90.0	61.2	-	-
	BB	69.7	73.9	93.5	94.4	92.9	90.2	61.5	-	-

shows the MAC values for the selected identification modes and all considered WPT positions (see Table 3) before and after the sensitivity-guided parameterization, that is, using the initially estimated and actually found parameter values for the simulations. It can be seen that the parameterization has led to an improvement for most modes, in some cases even resulting in more than 25% higher MAC values. In Figure 3, the selected identification modes (see Table 3) are compared with the model modes using the found parameters. Here, most measured identification modes have one distinct model counterpart with high MAC conformity and vice versa, showing the validity of the parameterization routine. Ideally, the found parameters would improve the model’s match with all measured modes, not just with the selected identification modes from Table 3. However, the MAC value statistics given in

Table 6. MAC value statistics for all measured modes after the sensitivity-guided (SG) parameter identification and their changes with respect to the initial state (Delta).

MAC in %	Position ${\mathit{z}}_1$		Position ${\mathit{z}}_2$		Position ${\mathit{z}}_3$		Position ${\mathit{z}}_4$
	SG	Delta	SG	Delta	SG	Delta	SG	Delta
Worst	0.4	−0.7	7.6	−0.2	15.1	−0.5	5.3	−2.7
25% percentile	21.7	−0.0	16.0	−0.0	36.4	−0.1	17.3	−0.0
Mean	44.5	1.8	50.7	2.7	58.3	2.2	38.2	1.6
75% percentile	62.9	1.1	86.1	0.4	87.0	0.6	54.5	0.3
Best	97.4	25.5	97.8	28.6	98.1	24.1	93.6	18.1

indicate that the other modes were largely unaffected by the parameterization with, for example, 75% of all modes at WPT position $z_3$ improved by less than 0.6% (or even deteriorated), confirming the presence of modeling errors.

Table 7. Comparison of NDD values between the reference modes and the simulated modes using the model with the initial parameter values and the resulting values from the sensitivity-guided (SG) and black-box (BB) parameter identification approaches.

NDD in %		1	2	3	4	5	6	7	8	9
Position $z_1$	Initial	330.4	24.2	144.4	36.4	38.4	31.6	16.3	42.7	44.1
	SG	41.9	51.3	17.4	58.8	10.4	49.3	25.5	16.2	45.6
	BB	26.1	2.9	6.4	46.9	15.4	9.1	3.6	32.3	47.1
Position $z_2$	Initial	282.5	38.0	41.8	80.8	45.1	26.7	44.1	4.1	-
	SG	34.6	72.4	66.4	37.2	9.8	3.4	48.3	0.2	-
	BB	20.0	6.5	53.5	13.5	25.2	17.4	47.5	2.0	-
Position $z_3$	Initial	183.2	270.2	38.5	7.1	120.5	4.5	4.5	24.9	13.2
	SG	5.6	38.0	75.5	50.3	31.1	42.8	14.6	32.2	8.1
	BB	5.7	23.3	3.9	27.1	23.9	16.2	3.3	29.5	11.3
Position $z_4$	Initial	161.3	212.8	40.1	8.2	175.0	6.4	13.0	-	-
	SG	3.1	23.4	79.9	53.0	29.0	40.3	29.1	-	-
	BB	7.7	10.5	3.9	25.7	24.5	6.3	19.7	-	-

is a comparison of the NDD before and after the sensitivity-guided parameterization, using the initially estimated and actually found machine tool model parameters for all considered WPT positions. Note that the table presents relative rather than absolute damping differences, meaning that the listed NDD value for the sensitivity-guided parameterization at position $z_1$ of 41.9% in the table indicates, for example, a modal damping of 1.419% instead of 1.0% rather than 42.9% versus 1.0%. It can be seen that the sensitivity-guided parameterization approach has generally improved—that is, reduced—the damping deviations. However, as expected by the LS algorithm used [17] in the presence of modeling errors, some modes had moderately deteriorated.

3.3. Black-Box Global Optimization

As an alternative to the sensitivity-guided identification method used in Section 3.2, the optimal model parameters (see Table 2) can be estimated using a black-box approach. For this, the same two-stage optimization strategy utilizing PSO as in Semm et al. [14] was deployed. In the first stage, the optimal stiffness parameters, which maximize the mean of the MAC overall modes, were sought, while the damping parameters were fixed to their initial values. This was conducted by assuming that the damping parameters do not influence the natural frequencies and mode shapes, which holds for lightly damped structures such as machine tools [10,14,31]. In the second stage, the stiffness parameters were fixed to their found optimal values, and the optimal damping parameters, which minimize the mean of the ${\mathrm{NDD}}^2$ overall modes, were searched for in a second PSO run.

Semm et al. [14] set the number of particles to 100 and optimized over 100 iterations. For better comparability with the sensitivity-guided approach in Section 3.2, which utilized 192 CPU threads in parallel, the number of particles was increased to 192. For the same reasons, the parameter identification was also restricted to the same selection of reference modes (see Table 3) as in Section 3.2. The values of the parameters $u_1$, $u_2$, and $u_3$ in Equation (8) were set to $1.1$, $1.49$, and $1.49$ by Semm et al. [14] and also used in this work.

The black-box approach took 54 min for calculation. Again, the final parameter values can be found in the appendix in Table A1. Even though the FRAC value of the FRF shown in Figure 2 increased from 25.7% to 34.6%, the conformity is still low. Again, the initial and found FRAC values for each considered WPT position are shown in Table 4. In particular, the median FRAC values were significantly increased compared to the initial values, while the mean and maximum values were only moderately changed. This indicates that the considered FRFs can be divided into two groups: one positively affected by the black-box optimization approach and the other that is hardly affected at all or even deteriorated.

Table 5 shows that the found MAC values mostly increased for all modes and WPT positions. However, some modes were not affected by the optimization or even deteriorated, such as, for example, mode 6 at WPT position $z_3$ or mode 2 at WPT position $z_4$. This can also be seen in

Table 8. MAC value statistics for all measured modes after the black-box (BB) parameter identification and their changes with respect to the initial state (Delta).

MAC in %	Position ${\mathit{z}}_1$		Position ${\mathit{z}}_2$		Position ${\mathit{z}}_3$		Position ${\mathit{z}}_4$
	BB	Delta	BB	Delta	BB	Delta	BB	Delta
Worst	0.0	−16.5	7.5	−15.8	16.3	−13.2	5.4	−14.3
25% percentile	22.5	−0.6	17.5	−0.5	33.5	−1.2	18.9	−0.3
Mean	44.7	2.0	51.7	3.7	58.2	2.2	38.4	1.8
75% percentile	63.9	2.3	87.8	0.6	85.8	0.8	55.6	0.8
Best	97.3	33.8	97.2	41.8	98.3	40.2	95.6	28.2

, which shows the MAC conformity of all measured modes. Here, the mean values increased across all WPT positions. However, they are generally low, possibly due to the modeling errors described in Section 3.2. The NDD values shown in Table 7 significantly improved for most modes, with only minor setbacks for modes and WPT positions with initially already low NDD values.

3.4. Discussion

The presented parameter identification methods can be compared in two ways: On the one hand, there is the economic effort related to each method, which can, for example, be quantified using the individual costs of the methods. However, since costs can vary strongly from one user to another and are generally difficult to compare, the economic effort will be assessed by the time required to conduct the parameter identification methods. In addition to the presented sensitivity-guided and black-box parameter identification, the method of sequential assembly, which is briefly described in Section 1 and presented in Niehues [31] and Schwarz [18], will be evaluated as a benchmark.

Table 9. Workload estimation for the sequential assembly and the sensitivity-guided (SG) and the black-box (BB) parameter identification methods and assembly state description for the sequential assembly method. The workload for the EMAs was estimated with respect to a skilled engineer.

		Sequential Assembly	SG	BB
EMAs	➀ Bed	4 h	-	-
	➁ Bed + MEs	4 h	-	-
	➂ Bed + MEs + LGS + WPT	4 h	-	-
	➃ Bed + MEs + LGS + WPT + BSD + BSD bearings	4 h
GSAs		-	35 min	-
Optimization		- *	9 min	54 min
Total	First time	16 h	4 h 44 min	4 h 54 min
	Repeated		4 h 9 min

* Disregarded since unknown.

summarizes the workload of the three parameter identification methods split up into workloads for the EMA measurements and computation time for the GSAs and the actual optimization routines. Since it is equally required for all methods, it is assumed that an overall machine tool model (i.e., the one matching assembly state ➃ in Table 9) already exists. For the sequential assembly method, the model also must be easily reduced to a subsystem, which can be generally assumed. Apart from the complete machine tool, Table 9 also lists three more assembly states required for the method of sequential assembly. For each state, it is estimated that a skilled engineer needs about 4 h to conduct an EMA. Even though state ➃ will require more measurement nodes than state ➀, it is assumed that a significant part of the EMA workload is consumed by preparation and clean-up tasks, leading to an EMA workload independent of the machine tool configuration. The sensitivity-guided and black-box parameter identification methods only require measurements on the overall machine tool but not on any subsystems. For these methods, Table 9 repeats the computation times reported in Section 3.2 and Section 3.3. Since it is not known if the model parameters for the sequential assembly method were tuned manually or by an optimization algorithm, it cannot be properly estimated and is thus disregarded. By studying the total workloads in Table 9, it can be seen that sensitivity-guided parameter identification is the most economical method with a total workload of 4 h and 44 min compared to 4 h and 54 min and 16 h for the other methods. This is especially true for repeated parameter identification in the case of, for example, component wear, which changes the dynamic behavior of the machine tool. For the method of sequential assembly and black-box parameter identification, all steps have to be repeated, leading to an additional workload of 16 h and 4 h and 54 min, respectively. In contrast, it is assumed that the results of the (GSAs) for sensitivity-guided parameter identification still hold for only minor changes of the machine tool. Thus, only the EMA and the computation have to be repeated, leading to an additional workload of 4 h and 9 min. In the case that the EMA measurements can also be automated by, for example, feed drive excitation and a permanent installation of acceleration sensors across the machine tool, sensitivity-guided optimization offers the chance to update a machine tool model within minutes. Note that repeating the sequential assembly method is often impossible, since the fully assembled machine tool can no longer be broken down into parts.

On the other hand, the parameter identification methods can be compared in evaluating the accuracy of the resulting model. As it was not possible to disassemble the machine tool, the method of sequential assembly could no longer be reproduced. Thus, it is excluded from further consideration. Looking at the FRAC value statistics of both the sensitivity-guided and the black-box parameter identification methods, it can be seen that both lead to very similar mean FRAC values. However, the black-box approach leads to a significantly higher median value than the sensitivity-guided method, while the latter results in higher maximum values. This indicates that only some input–output relationships and their corresponding FRFs benefit from the black-box approach, while the others are hardly affected or even deteriorated. The sensitivity-guided method, in turn, has a more uniform positive effect on all FRFs, as the median, mean, and maximum FRAC values increase for all WPT positions. Both identification methods have a comparable effect on the MAC value statistics concerning all measured modes, as indicated by Table 6 and Table 8. Considering the individual reference modes (see Table 5), the sensitivity-guided method performs better, since it does not deteriorate one of the first two modes per WPT position as with the black-box approach. However, in most cases, the latter leads to a better modal damping match, as indicated by Table 7. Few studies have been performed in the field of updating machine tool dynamic models, which can be used to benchmark the presented results. Semm et al. [14] modeled a five-axis machine tool and achieved their targeted MAC conformity of 80% for most modes. However, they deployed a combination of parameter identification on test benches, the method of sequential assembly, and model updating via PSO and did not provide initial MAC values. Garitaonandia et al. [19] examined a centerless grinding machine. They were able to improve the MAC conformity of three of four target modes by up to 0.9%. To sum up, both the sensitivity-guided and the black-box parameter identification methods outperform those works with improvements higher than 25% without the need for cumbersome tests on test benches or a sequential assembly process.

Based on the accuracy of the resulting simulation model, no clear decision on a parameter identification method can be made, as both the sensitivity-guided and the black-box approaches generally deliver similar results with specific drawbacks on each side. Additionally, the presence of modeling errors equally and significantly affects both methods, affecting the judgment between them. However, the sensitivity-guided parameter identification approach is assumed to have a significant advantage concerning the economic effort required for repeated model identification runs (see Table 9), especially with the prospect of automated EMA measurements.

4. Conclusions and Outlook

In the work leading to this paper, the parameters of a dynamic machine tool simulation model were identified in two ways: On the one hand, the sensitivity-guided parameterization approach theoretically presented in [17] was applied to real-word machine tool measurement data. On the other hand, a state-of-the-art PSO was conducted to serve as a reference. Even though both methods are fundamentally different, they resulted in very similar final model accuracies with partial improvements regarding the MAC conformity of more than 25%. However, the overall conformity of the model was found to be generally low, indicating the presence of deviations between the model and the real-word machine tool system. Both approaches were also evaluated in terms of their corresponding costs and benchmarked against the method of sequential assembly from the literature. It was shown that sensitivity-guided parameter identification is the most economical approach, with the prospect of further outperforming the other methods in the near future and updating a dynamic machine tool model within minutes.

Future research will deal with automating the EMA measurements to further highlight the potential of sensitivity-guided parameter identification. To demonstrate the transferability of the found results, a different machine tool structure will be used. This also offers the chance to repeat the conducted comparison between sensitivity-guided and black-box parameter identification with a model more capable of replicating the real-word structure’s dynamic properties.