Improved Nonlinear Dynamic Model of Helical Gears Considering Frictional Excitation and Fractal Effects in Backlash

Abstract

Surface roughness and sliding friction are pivotal in determining the dynamic meshing performance of helical gears, especially under conditions of flexible support. In addition, the meshing parameters influenced by gear vibrations exhibit time-varying characteristics under flexible support stiffness, which is disregarded by many scholars. Based on this, a nonlinear dynamic model of a helical cylindrical gear system under flexible support conditions is developed, considering the coupling effects of dynamic friction and backlash influenced by fractal surface roughness. The motion differential equations of the system are derived using the Lagrange method, and numerical solutions are obtained through the Runge–Kutta method. The effect of several control parameters (driving speed, surface roughness and fractal dimension) on the dynamic response of gear system is studied, and the proposed dynamic model is compared with the traditional model under different support stiffness to demonstrate its adaptability to highly flexible support scenarios. The results indicate that the proposed dynamic model is better suited for flexible support structures. Moreover, the coupling effects of sliding friction and fractal backlash amplify the dynamic response of the gear system and introduce complex spectrum characteristics. This study provides theoretical guidance for the optimization of vibration and noise reduction designs in helical gear systems.

1. Introduction

As a common gear transmission device, helical gears have been widely used in mechanical manufacturing, automotive transmission, wind power generation and other fields due to their high contact ratio characteristics [1,2,3,4]. Under special working conditions such as high precision, high load, and flexible support, sliding friction influenced by surface roughness have a significant impact on the dynamic performance of gear systems [5]. Therefore, to accurately analyze and evaluate the dynamic performance of gear systems and to reduce system vibration and noise, it is essential to develop an accurate dynamic model of the gear system that considers the effects of surface roughness and sliding friction.

The establishment of gear dynamic models has always been a focus in the field of gear research. Researchers first established a 1-DOF dynamic model considering speed coordination to roughly analyze the dynamic behavior of gears [6,7]. Higher demands for vibration reduction and noise control in gearboxes have been proposed in aerospace applications, necessitating the consideration of multidirectional vibrations in gear dynamic models [8,9]. In recent years, the study of multi-degree-of-freedom (MDOF) gear systems has become a research focus. Wang et al. [10] established a 5-DOF gear system to represent the dynamic response of its flex-twist coupling. Jiang et al. [11,12] analyzed $X$ and $Z$ translations based on 6-DOF helical gear models. Wan et al. [13] analyzed helical gear dynamics due to $Y$ and $Z$ translations based on 6-DOF models. Sun et al. [14] used finite element method to build a MDOF gear model and analyzed the influence of temperature and bearing type on gear system dynamics. With the demand for lightweight gear systems in aviation and military equipment, bearing stiffness structures will no longer be suitable for the future development of the helical gear system industry, and flexible support structures such as thin hollow shafts and thin-walled gearboxes will become the future development trend [15]. However, after introducing a flexible support structure, its support stiffness will be significantly lower than that of the bearing structure. This will result in greater displacement of the two gears during operation, and the original model will not be able to accurately predict the dynamic behavior of the gears. Therefore, the dynamic modeling of helical gears under flexible support stiffness remains a challenge for the future.

In addition, tooth surface friction is an important source of gear vibration and noise that affects the system dynamics in terms of external excitation [16]. Therefore, a large number of scholars have studied the impact of this factor on gear systems. Singh analyzed the dynamic behavior of a single degree of freedom gear pair, including sliding friction, by using Floquet theory [17], the harmonic balance method [18], and the numerical method [19], and extended the influence of sliding friction to a multi-free gear pair [20]. Kar et al. [21] proposed a method and algorithm for solving the torsional friction moment of a helical gear system under constant friction coefficient. Wang et al. [22] studied the vibration characteristics of the star bearing based on the dynamic response under constant friction coefficient excitation and controlled the dynamic behavior by changing the support stiffness. Chen et al. [23] calculated the meshing efficiency of gear pairs considering sliding friction losses by studying the tooth surface meshing force and relative sliding/entrainment velocity and incorporating them into the formula for sliding friction coefficient. Chen et al. [24] found that at low driving speeds, friction increases the displacement amplitude and has a significant impact on the high-frequency part of the frequency domain by considering friction and dynamic backlash in the dynamic study of the system. Building on this foundation, Kang et al. [25] introduced fractal theory to characterize rough surfaces based on their study of friction. While this approach improved the understanding of microstructural surface properties, it still lacks comprehensive exploration of its impact on system dynamics. Therefore, further studying the coupling effect between sliding friction and fractal tooth gaps is crucial for accurate gear dynamics analysis.

Focusing on the issues, this paper establishes a dynamic model of a helical gear system considering fractal backlash and sliding friction under flexible support and analyses the dynamic characteristics of the system based on this model. The rest of this article is organized as follows: In Section 2, dynamic meshing parameters and corresponding dynamic mesh force and friction forces affected by lateral vibration were presented, and a 7-degree-of-freedom nonlinear dynamic model of a helical gear system with velocity coordination, time-varying pressure angle, dynamic clearance, and sliding friction was established. And the Lagrange method was used to derive the equation. In Section 3, the proposed dynamic model is compared with the traditional model under different support stiffness to demonstrate its adaptability to highly flexible support scenarios. In the following section, Section 4, the effects of friction and Fractal Effects in Backlash on the dynamic response of the gear system are analyzed. Finally, some brief findings of this study are provided.

2. Model and Methodology

Considering the effects of input, output, and supporting bearings, the dynamic lumped parameter model of the helical gear-rotor-bearing system is shown in Figure 1. The system is simplified into a 7-degree-of-freedom (DOF) bending-torsion coupled generalized lumped parameter model, comprising the driving gear and driven gear. Among them, $k_x^{\mathrm{1,2}}$, $k_y^{\mathrm{1,2}}$ and $k_z^{\mathrm{1,2}}$ represent the support stiffness of the gear mesh in the $x$, $y$ and z directions, respectively, while $c_x^{\mathrm{1,2}}$, $c_y^{\mathrm{1,2}}$ and $c_z^{\mathrm{1,2}}$ denote the corresponding bearing damping. The gear pair is simplified as two rigid disks connected through the meshing stiffness $k_m$ and meshing damping $c_m$. The driving speed ${\dot{\theta }}_1$ is exerted on the pinion, and the output torque $T_1$ is loaded to the gear. In addition, a representation of the pressure plane is also illustrated, depicted as a two-dimensional “unwrapping band” that unwinds from one base cylinder and winds onto the other. In this figure, solid lines represent changes in the contact line, while dashed lines represent extensions of the contact line.

According to the research of Liu et al. [26], a general expression of the excitation speed of pinion for rigid-body conditions can be rewritten as:

$${\dot{\theta }}_1\left(t\right)={\dot{\theta }}_{1m}\left(1+{\displaystyle {\sum }_{i=1}^{\infty }{\lambda }_i\sin \left(i\omega t+{\phi }_2\right)}\right)$$

(1)

here, ${\dot{\theta }}_{1m}$ denotes the average driving speed, ${\lambda }_i$ and ${\phi }_i$ denote the fluctuation amplitudes of driving speed and the phase of the $i^{\mathrm{t}\mathrm{h}}$ harmonic, respectively. The excitation frequency $\omega$ mainly depends on the relevant configuration of the prime mover [27].

Based on the geometric relationship, the relative displacement of the gear system ${\delta }_L$ along the line-of-action ($LOA$) and the dynamic transmission error components ${\delta }_L$ along the $LOA$ and ${\delta }_O$ along the off-line-of-action ($OLOA$) are derived [28]. These parameters can be expressed as:

$${\delta }_L=\left(\left(x_1-x_2\right)\sin \psi +\left(y_1-y_2\right)\cos \psi +{\theta }_1{R}_1+{\theta }_2{R}_2\right)\cos \beta -\left(z_1-z_2\right)\sin \beta$$

(2)

$${\delta }_O=\left(\left(x_1-x_2\right)\cos \psi -\left(y_1-y_2\right)\sin \psi \right)\cos \beta +\left(z_1-z_2\right)\sin \beta$$

(3)

Considering the positive or negative values of the transmission error, the angle $\psi$ can be expressed as:

$$\psi =\left\{\begin{array}{cc}{a}_n-b_n& \delta \left(t\right)\ge 0\\ -a_n-b_n& \delta \left(t\right)<0\end{array}\right.$$

(4)

The lateral relative position angle of the gear pair $b_n$ can be expressed as:

$$b_n={\tan }^{-1}\left(\left(y_1-y_2+d_0\sin {\beta }_0\right)/\left(x_1-x_2+d_0\cos {\beta }_0\right)\right)$$

(5)

The transverse working pressure angle of the gear $a_n$ can be expressed as [29]:

$$a_n={\cos }^{-1}\left(\left(R_1^b+R_2^b\right)/d\right)$$

(6)

To facilitate the analysis, the transverse displacement of the gear center is magnified. As shown in Figure 2, $LOA$ represents the line of action, while $OLOA$ is perpendicular to the meshing line. $d_{\tau }$ denotes the surface center distance, $a_n$ is the surface meshing angle, and $b_n$ is the relative position angle at any given time. $O_1$ and $O_2$ represent the initial center positions of gear $p$ and gear $g$, respectively. $d_0$ refers to the initial center distance, and $a_0$ is the initial pressure angle, which is assumed to be $20°$ in this paper.

Due to the translational motion of the drive and driven wheels, the surface center distance of the gear changes from $d_0$ to $d$ [30]. The time-varying center distance d can be given as:

$$d=\sqrt{{\left(x_1-x_2+d_0\cos {\beta }_0\right)}^2+{\left(y_1-y_2+d_0\sin {\beta }_0\right)}^2}$$

(7)

where the original center distance is calculated as $d_0$ = $R_1$ + $R_2$, and $R_1$ and $R_2$ denote the radii of the indexing circle for the large and small gears, respectively.

The meshing process of helical gears involves alternating logarithmic cycles of different teeth, with the tooth width determining the instantaneous meshing number. Mo et al. [31] discussed the calculation of the meshing teeth logarithm but did not account for the z-axis vibration error affecting the tooth width. To address this, the variation in tooth width during meshing is analyzed based on the helical gear’s force angle. Using the actual center distance d from Equation (7), the total contact ratio of helical gears and the length of the line of action $Z_{act}$, which can be expressed as:

$$Z_{zct}=\sqrt{{\left(R_1^o\right)}^2-{\left(R_1^b\right)}^2}+\sqrt{{\left(R_2^o\right)}^2-{\left(R_2^b\right)}^2}-d\sin a_n$$

(8)

here, $R_1^o$ and $R_2^o$ represent the pinion and gear’s tooth top circle radii, respectively. The initial contact surface tooth width is defined as $L_{\tau }$. When considering the influence of helix angle $\beta$ on axial transmission error, the positive and negative directions of gear helix angle can be defined as:

$$\beta \left\{\begin{array}{c}>0\mathrm{if}\mathrm{gear}\mathrm{has}\mathrm{right}-\mathrm{hand}\mathrm{teeth}\\ <0\mathrm{if}\mathrm{gear}\mathrm{has}\mathrm{left}-\mathrm{hand}\mathrm{teeth}\end{array}\right.$$

(9)

According to left-hand helical gear as shown in Figure 1, the axial force of the master and slave gears can be determined based on the spiral direction. Therefore, the relative axial vibration mode of the driven gear relative to the driving gear is determined. The tooth width $L_n$ of the contact surface at this time can be expressed as:

$$L_n=L+\mathrm{sign}\left(\beta \right)\left(z_1-z_2\right)$$

(10)

here, $L$ in the above equation is the initial tooth width, set to 30 mm.

With the contact surface tooth width determined, the instantaneous number of meshing teeth $n$ can be calculated. Considering the influence of the helix angle on axial vibration, the diagonal pitch length along the line of action $L_n$ can be evaluated, alongside the number of teeth behind $N_b$ and in front $N_f$ of the current contact location. The number of contacted mesh teeth at any given moment is established as:

$$n=1+\underset{N_b}{\underbrace{\mathrm{floor}\left\{l/{P_b}^{\prime }\right\}}}+\underset{N_f}{\underbrace{\mathrm{floor}\left\{\left(Z_{act}/\sin \left(\gamma \right)-l\right)/{P_b}^{\prime }\right\}}}$$

(11)

The function rounds off the ratio ($x/y$) to the nearest inter (toward a lower value); $Z_{act}$ is represented by the equation above Equation (8); ${P_b}^{\prime }$ = $P_b\ast \mathrm{c}\mathrm{o}\mathrm{s}\beta /\mathrm{s}\mathrm{i}\mathrm{n}\left(\gamma +\beta \right)$ represents the projection of tooth width on the diagonal; ${\phi }_m$ = $Z_{act}/R_1^b$ is the theoretical meshing cycle. The $\gamma$ = ${\tan }^{-1}\left(Z_{zct}/L_n\right)$ angle is shown in the figure above as the angle formed by the diagonal $AB$ and $L_{\tau }$ of the 3-dimensional unfolded band; l is the parameter increased diagonally from point $A$ to point $B$, which can be expressed as:

$$l=\left({\phi }_{m1}\mathrm{mod}\left({\theta }_1{R}_1,Z_{act}\right)\cos {\beta }_b\right)/\sin \left(\gamma +{\beta }_b\right)$$

(12)

$$\mathrm{mod}\left(y,z\right)=y-z\mathrm{floor}\left(y/z\right)z\ne 0$$

(13)

here, ${\beta }_b$ = ${\tan }^{-1}\left(\tan \beta \times \cos a_0\right)$ is the base circular helix angle.

2.1. Time-Varying Gear Backlash Based on Fractal Theory

To analyze the dynamic performance of gears under specific parameters, researchers simplify gear contact into spring-damping models and establish corresponding dynamic equations. For example, Yi et al. [32] incorporated the calculation of time-varying clearance at the gear end face, which provides a significant advantage in analyzing the time-varying backlash of helical gears. Additionally, the true contour of the rough gear surface, characterized by a series of peaks and valleys, has been found to considerably influence the variation in backlash between meshing teeth [33]. In fractal modeling approaches for different surface characteristics, the microscopic morphology of surfaces with varying roughness can be simulated by adjusting the fractal dimension as well as other fractal parameters, such as the roughness scale [34].

As illustrated in Figure 3, the contact characteristics of helical gear pairs exhibit marked disparities between ideal and actual tooth surfaces. The dashed lines represent the ideal tooth surface, while the solid lines represent the actual tooth surface affected by manufacturing errors and roughness. Fractal theory, characterized by inherent self-affinity and self-similarity, provides a robust framework for describing the complex surface morphology of mechanical interfaces. Accordingly, the Weier-Mandelbrot (W-M) function is widely employed to simulate rough surfaces [35], and its mathematical expression is given by:

$$z\left(x\right)=G^{D-1}{\displaystyle \sum _{n=n_l}^{\infty }{\gamma }^{\left(D-2\right)n}\cos \left(2\pi {\gamma }^{n}x\right)},1<D<2,\gamma >1$$

(14)

here, ${\gamma }^n$ is the discrete frequency spectrum, $n_l$ is the low cut-off frequency of minimum frequency index.

When considering the surface roughness values of two gears. The mathematical expression for the distance between the end faces in the normal direction ${\Delta b}_f$ can be calculated by Ref. [35] as:

$$\Delta b_f=R_a\left({\displaystyle \sum _{k=nl}^{\infty }{\gamma }^{\left(D-2\right)k}\sin \left(2\pi N_s{\gamma }^k\frac{\mathrm{mod}\left({\theta }_1,{\theta }_{m1}\right)}{{\theta }_{m1}}\right)}\right)/{\overline{R}}_D$$

(15)

here, $N$ denotes the intermediate variables affecting the value of the sampled wave crest of the calculated roughness curve, $R_a$ is the arithmetic mean deviation ($\mathsf{\mu }\mathrm{m})$; The torsional angular displacement ${\theta }_{m1}$ = $m\pi /R_1^b$ for one meshing cycle; $m$ denotes the gear module; In this paper, $\gamma$ and $D$ are set as 1.5 and 1.1, respectively.

$${\overline{R}}_D=\left({\displaystyle {\int }_0^{{\theta }_{m1}}\left|{\displaystyle \sum _{k=nl}^{\infty }{\gamma }^{\left(D-2\right)k}\sin \left.\left(2\pi N_s{\gamma }^k\frac{\mathrm{mod}\left({\theta }_1,{\theta }_{m1}\right)}{{\theta }_{m1}}\right)\right|}\right.}d{\theta }_1\right)/{\theta }_{m1}$$

(16)

The total backlash, namely the normal backlash of the helical gear on the end face is expressed as:

$$b_t=b_0+\Delta b\cos \beta +\Delta b_f$$

(17)

here, $∆b$ is caused by changes in the center distance, which can be expressed as [36]:

$$\Delta b=\left(R_1^b+R_2^b\right)\left(inv\left(a_n\right)-inv\left(a_0\right)\right)$$

(18)

where $inv\left(x\right)$ is the involute function; $inv\left(x\right)$ = $\tan \left(x\right)-x$. The above derivation gives the basic theory of backlash. More detailed research of backlash can refer to Refs. [24,32].

2.2. Dynamic Model of the Gear System

The different equations of motion for the gear pair can be derived by the seven generalized coordinates, whose vector form is expressed as:

$$\left[q\right]=\left[\begin{array}{ccccccc}{x}_1& y_1& z_1& x_2& y_2& z_2& {\theta }_2\end{array}\right]$$

(19)

where $x_1$, $y_1$, $z_1$, $x_2$, $y_2$, $z_2$, ${\theta }_2$ represent the translational and angular vibration displacements of the active and driven wheels, respectively, along the $x$, $y$ and $z$-axis.

The displacement vectors for the centers of gears $p$ and $g$ in the coordinate system are subsequently derived, which can be given as:

$$r_1=x_{1}i+y_{1}j+z_{1}k,r_2=x_{2}i+y_{2}j+z_{2}k$$

(20)

where $i$ and $j$ and $k$ are unit vectors along the $x$ and $y$ and $z$, respectively.

The kinetic energy $T_E$ of the system can be expressed by mass and driving speed, can be calculated as:

$$T_E={\displaystyle \sum _{i=1}^2\left(m^i{‖{\dot{r}}_i‖}^2+J^i{{\dot{\theta }}_i}^2\right)}/2$$

(21)

The energy generated by gear pairs and bearings can be expressed as elastic potential energy $U$ and dissipated energy as follows:

$$U=\left({\displaystyle {\sum }_{i=1}^2\left(k_x^i{x}_i^2+k_y^i{y}_i^2+k_z^i{z}_i^2\right)}+{\displaystyle \sum _{j=1}^n{k}_m^j{f}^2\left({\delta }_L,b_t\right)}\right)/2$$

(22)

here, $b_t$ is helical gear end normal backlash, which can be obtained from Equation (17);

The total dissipation function $R_C$ consists of the dissipation function of radial bearing damping and gear meshing damping:

$$R_C=\left({\displaystyle {\sum }_{i=1}^2\left(c_x^i{\dot{x}}_i^2+c_y^i{\dot{y}}_i^2+k_z^i{\dot{z}}_i^2\right)}+{\displaystyle \sum _{j=1}^n{c}_m^j{\dot{f}}^2({\delta }_L,b_t)}\right)/2$$

(23)

Considering the effect of gravity, the generalized force vector $Q$ can be expressed as:

$$Q_j=\left[\begin{array}{ccccccc}0& -m_{1}g& 0& 0& -m_{2}g& 0& T_2\end{array}\right]$$

(24)

$$Q_f=\left[\begin{array}{ccccccc}{\displaystyle \sum _{j=1}^n{F}_x^{f_{1,j}}}& {\displaystyle \sum _{j=1}^n{F}_y^{f_{1,j}}}& {\displaystyle \sum _{j=1}^n{F}_z^{f_{1,j}}}& {\displaystyle \sum _{j=1}^n{F}_x^{f_{2,j}}}& {\displaystyle \sum _{j=1}^n{F}_y^{f_{2,j}}}& {\displaystyle \sum _{j=1}^n{F}_z^{f_{2,j}}}& {\displaystyle \sum _{j=1}^n{T}_{\theta }^{f_{2,j}}}\end{array}\right]$$

(25)

The differential equation of motion was derived using the Lagrange equation:

$${\displaystyle \frac{d}{dt}}\left({\displaystyle \frac{\partial T_E}{\partial {\dot{q}}^i}}\right)-{\displaystyle \frac{\partial T_E}{\partial q^i}}+{\displaystyle \frac{\partial U}{\partial q^i}}+{\displaystyle \frac{\partial R_C}{\partial {\dot{q}}^i}}=Q_j+Q_f$$

(26)

Substituting the four equations $T_E$, $R_C$, $U$, $Q_j$ in the above equation into the Lagrange Equation (26) can produced:

$$m^i{\ddot{x}}_i+c_x^i{\dot{x}}_i+k_x^i{x}_i+{\displaystyle \sum _{j=1}^n{F}_x^i}-{\displaystyle \sum _{j=1}^n{F}_x^{f_{i,j}}}=0$$

(27)

$$m^i{\ddot{y}}_i+c_x^i{\dot{y}}_i+k_x^i{y}_i+{\displaystyle \sum _{j=1}^n{F}_y^i}-{\displaystyle \sum _{j=1}^n{F}_y^{f_{i,j}}}=-m_{i}g$$

(28)

$$m^i{\ddot{z}}_i+c_x^i{\dot{z}}_i+k_x^i{z}_i+{\displaystyle \sum _{j=1}^n{F}_z^i}-{\displaystyle \sum _{j=1}^n{F}_z^{f_{i,j}}}=0$$

(29)

$$J_2{\ddot{\theta }}_2+{\displaystyle \sum _{j=1}^n{F}_{\theta }^2}-{\displaystyle \sum _{j=1}^n{T}_{\theta }^{f_{2,j}}}=T_2$$

(30)

here, superscripts (⋅) and (⋅⋅) denote the first and second order derivatives with respect to time, respectively.

The equivalent stiffness and damping coefficients are employed to simulate the flexibility of shafts and bearings. Under the condition of flexible support stiffness, the system’s support stiffness is not merely approximated as the bearing stiffness $k_{x,y,z}^{\mathrm{1,2}}$. Instead, it considers the series stiffness of the shaft stiffness $k_{x,y,z}^j$ and the bearing stiffness $k_{x,y,z}^{\mathrm{1,2}}$. In this case, the calculation utilizes the empirical Formula (26). Consequently, the total equivalent stiffness of the bearing system can be obtained as follows:

$$k_{x,y,z}^{1,2}={\displaystyle \frac{k_{x,y,z}^{b1,b2}{k}_{x,y,z}^j}{k_{x,y,z}^{b1,b2}+k_{x,y,z}^j}},c_{x,y,z}^{1,2}=2{\xi }_s\sqrt{m^{1,2}{k}_{x,y,z}^{1,2}}$$

(31)

where support damping coefficient ${\xi }_s$ is 0.025.

The $x,y$ and $z$ directions of the dynamic meshing force $F_{∆}^{\mathrm{1,2}}$ can be expressed as:

$$F_{\Delta }^{1,2}=\left(c_m^j+k_m^j\right)\left(\partial f/\partial \Delta \right),\Delta =x_i,y_i,z_i,{\theta }_i$$

(32)

where $k_m^i$ is the mesh stiffness of the helical gear.

The nonlinear function of backlash $f({\delta }_L,b_t)$ and its relative speed $f_1\left({\delta }_L,b_t\right)$ of the gear system can be given as:

$$f({\delta }_L,b_t)=\left\{\begin{array}{cc}{\delta }_L\left(t\right)-\mathrm{sign}\left({\delta }_L\right)b_t& \left|\left.{\delta }_L\right|\right.>b_t\\ 0& \left|\left.{\delta }_L\right|\right.\le b_t\end{array}\right.,f_1({\delta }_L,b_t)=\left\{\begin{array}{cc}{\dot{\delta }}_L\left(t\right)-\mathrm{sign}\left({\delta }_L\right){\dot{b}}_t& \left|\left.{\delta }_L\right|\right.>b_t\\ 0& \left|\left.{\delta }_L\right|\right.\le b_t\end{array}\right.$$

(33)

$$\partial f({\delta }_L,b_t)/\partial \Delta =\left\{\begin{array}{cc}\begin{array}{c}\partial {\delta }_L/\partial \Delta -\mathrm{sign}({\delta }_L)\left(\partial b_t/\partial \Delta \right)\\ 0\end{array}& \begin{array}{c}\left|\left.{\delta }_L\right|>\right.b_t\\ \mathrm{else}\end{array}\end{array}\right.,\Delta =x_i,y_i,z_i,{\theta }_i$$

(34)

$$\left[\begin{array}{c}\partial b_t/\partial x_1\\ \partial b_t/\partial y_1\\ \partial b_t/\partial z_1\end{array}\right]=-\left[\begin{array}{c}\partial b_t/\partial x_2\\ \partial b_t/\partial y_2\\ \partial b_t/\partial z_2\end{array}\right]=\left[\begin{array}{ccc}\sin \left(a_n\right)\cos \left(\beta \right)& 0& 0\\ 0& \sin \left(a_n\right)\cos \left(\beta \right)& 0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{c}\cos \left(b_n\right)\\ \sin \left(b_n\right)\\ 0\end{array}\right]$$

(35)

where ${\displaystyle \frac{\partial f_1\left(\dot{\delta },{\dot{b}}_t\right)}{\dot{{∆}_i}}}$ =${\displaystyle \frac{\partial f\left(\delta ,b_t\right)}{{∆}_i}}$. ${∆}_i$ =$x_i$, $y_i$, $z_i$, ${\theta }_i$.

The individual derivation components in the gear dynamics equation can be expressed as:

$${\dot{\delta }}_L\left(t\right)={\displaystyle \sum _{i=1}^2\left({\dot{x}}_i\left(\partial {\delta }_L/\partial x_i\right)+{\dot{y}}_i\left(\partial {\delta }_L/\partial y_i\right)+{\dot{z}}_i\left(\partial {\delta }_L/\partial z_i\right)+{\dot{\theta }}_i\left(\partial {\delta }_L/\partial {\theta }_i\right)\right)}$$

(36)

$$\left[\begin{array}{c}\partial {\delta }_L/\partial x_1\\ \partial {\delta }_L/\partial y_1\\ \partial {\delta }_L/\partial z_1\\ \partial {\delta }_L/\partial {\theta }_1\end{array}\right]=\left[\begin{array}{c}-\partial {\delta }_L/\partial x_2\\ -\partial {\delta }_L/\partial y_2\\ -\partial {\delta }_L/\partial z_2\\ \left(R_2/R_1\right)*\left(\partial {\delta }_L/\partial {\theta }_2\right)\end{array}\right]=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{c}\sin \psi \cos \beta +{\delta }_O^{\prime }\left(\partial \psi /\partial x_1\right)\\ \cos \psi \cos \beta +{\delta }_O^{\prime }\left(\partial \psi /\partial y_1\right)\\ \sin \beta \\ R_1^b\cos \beta \end{array}\right]$$

(37)

The frictional force components of each axis in the above formula are expressed as follows:

$$F_x^{f_{i,j}}=\left(\partial {\delta }_O/\partial x_i\right)\mathrm{sign}\left(V_s\right)\mu \left(k_m^{j}f\left({\delta }_L,b_t\right)+c_m^j{f}_1\left({\delta }_L,b_t\right)\right)$$

(38)

$$F_y^{f_{i,j}}=\left(\partial {\delta }_O/\partial y_i\right)\mathrm{sign}\left(V_s\right)\mu \left(k_m^{j}f\left({\delta }_L,b_t\right)+c_m^j{f}_1\left({\delta }_L,b_t\right)\right)$$

(39)

$$F_z^{f_{i,j}}=\left(\partial {\delta }_O/\partial z_i\right)\mathrm{sign}\left(V_s\right)\mu \left(k_m^{j}f\left({\delta }_L,b_t\right)+c_m^j{f}_1\left({\delta }_L,b_t\right)\right)$$

(40)

$$T_{\theta }^{f_{2,j}}=\mathrm{sign}\left(V_s\right)\mu R_f^{i,j}\left(k_m^{j}f\left({\delta }_L,b_t\right)+c_m^j{f}_1\left({\delta }_L,b_t\right)\right)$$

(41)

where $R_f^{i,j}$ is the friction arm of the driving and driven gear, respectively, and the detailed information of these variables are elaborately expressed in Appendix A. $\mu$ is the coefficient of friction of the system; its calculations can be obtained by empirical formulas in Ref. [37] as:

$$\mu =0.127*\left[1.27/\left(1.27-S\right)\right]{\mathrm{Log}}_{10}\left[\left(29.66*W\right)/\left(v*\left|V_s\right|*{\left(V_r\right)}^2\right)\right]$$

(42)

Regarding the calculation of the normal force $W$ on the tooth surface, He provided the formula $W$ = $T_p/(L\ast R_1\ast cos{a}_n)$ in Ref. [38]. However, this calculation method ignores the situation where the tooth surface is not in contact, which is obviously inconsistent with the modeling method in this article. Therefore, for the normal force $F$ on the tooth surface, it can be improved to $W$ = $\sqrt{{\left(\sum _{i=1}^n{F}_{x_1}^i\right)}^2+{\left(\sum _{i=1}^n{F}_{y_1}^i\right)}^2}/L_{\tau }\left(N/m\right)$; $L_{\tau }$ is the width of the contact surface (mm), given by (10), $F_{x_1}$ and $F_{y_1}$ are the meshing forces on the x-$\mathrm{a}\mathrm{x}\mathrm{i}\mathrm{s}$ and y-$\mathrm{a}\mathrm{x}\mathrm{i}\mathrm{s}$ during meshing on the pinion, respectively; $S$ is $RMS$ of surface roughness in $\mathsf{\mu }\mathrm{m}$ on Ref. [25] and can be calculated as $S$ = 1.1$R_a$.

The vibratory velocity influences the relative sliding velocity between meshing teeth, which, in turn, alters the friction force, potentially increasing gear vibrations [39]. Therefore, the sliding speed of the gear and the speed of the entrainment motion can be expressed by the following equation as:

$$V_s=v_{pi}-v_{gi}+\left(\left({\dot{x}}_1-{\dot{x}}_2\right)\cos \psi +\left({\dot{y}}_1-{\dot{y}}_2\right)\sin \psi \right)\cos \beta +\left({\dot{z}}_1-{\dot{z}}_2\right)\sin \beta$$

(43)

$$V_r=v_{pi}+v_{gi}+\left(\left({\dot{x}}_1+{\dot{x}}_2\right)\cos \psi +\left({\dot{y}}_1+{\dot{y}}_2\right)\sin \psi \right)\cos \beta +\left({\dot{z}}_1+{\dot{z}}_2\right)\sin \beta$$

(44)

where $v_{pi}$ and $v_{gi}$ are the viscosity coefficient of the gear and the tooth surface velocity of the driving and driven wheels, and can be derived from Ref. [40].

The derivative of the frictional force components of each axis in the above formula can be expressed as:

$$\left[\begin{array}{c}\partial {\delta }_O/\partial x_1\\ \partial {\delta }_O/\partial y_1\\ \partial {\delta }_O/\partial z_1\end{array}\right]=-\left[\begin{array}{c}\partial {\delta }_O/\partial x_2\\ \partial {\delta }_O/\partial y_2\\ \partial {\delta }_O/\partial z_2\end{array}\right]=\left[\begin{array}{ccc}1& 0& 0\\ 0& -1& 0\\ 0& 0& -1\end{array}\right]\left[\begin{array}{c}\cos \psi \cos \beta -{\delta }_L^{\prime }\left(\partial \psi /\partial x_1\right)\\ \sin \psi \cos \beta +{\delta }_L^{\prime }\left(\partial \psi /\partial y_1\right)\\ \sin \beta \end{array}\right]$$

(45)

The planar displacement along the $LOA$ direction and the $LOA$ in the normal direction are defined as ${\delta }_L^{\prime }$ and ${\delta }_O^{\prime }$, calculated as:

$${\delta }_L^{\prime }={\delta }_L-\left({\theta }_1{R}_1+{\theta }_2{R}_2\right)\cos \beta +\left(z_1-z_2\right)\sin \beta$$

(46)

$${\delta }_O^{\prime }={\delta }_O-\left(z_1-z_2\right)\sin \beta$$

(47)

where $\partial \psi /\partial x_1$, $\partial \psi /\partial y_1$ are partial derivatives of $\psi \left(t\right)$ with respect to the $x$ and $y$ axes, respectively, and can be given as:

$${\displaystyle \frac{\partial \psi }{\partial x_1}}=\left\{\begin{array}{cc}\cos \left(a_n-b_n\right)/d\sin a_n& {\delta }_L\ge 0\\ -\cos \left(a_n+b_n\right)/d\sin a_n& {\delta }_L<0\end{array}\right.,{\displaystyle \frac{\partial \psi }{\partial y_1}}=\left\{\begin{array}{cc}-\sin \left(a_n-b_n\right)/d\sin a_n& {\delta }_L\ge 0\\ -\sin \left(a_n+b_n\right)/d\sin a_n& {\delta }_L<0\end{array}\right.$$

(48)

The magnitude of dynamic meshing force and friction force can be expressed as:

$$F_m=k_{m}f\left(\delta ,b_t\right)+c_m\left(df\left(\delta ,b_t\right)/dt\right),F_f=\mu F_m$$

(49)

Therefore, we do not consider z-axis friction and Lang-range residual terms ${\delta }_O^{\prime }=0$ and ${\delta }_L^{\prime }=0$, The dynamic equation can be transformed into the following form Equations (27)–(30).

$$m^i{\ddot{x}}_i+c_x^i{\dot{x}}_i+k_x^i{x}_i+{\left(-1\right)}^{i-1}{F}_m\sin \left(\psi \right)\cos \beta +{\left(-1\right)}^i{F}_f\cos \left(\psi \right)\cos \beta =0$$

(50)

$$m^i{\ddot{y}}_i+c_y^i{\dot{y}}_i+k_y^i{y}_i+{\left(-1\right)}^{i-1}{F}_m\cos \left(\psi \right)\cos \beta +{\left(-1\right)}^{i+1}{F}_f\sin \left(\psi \right)\cos \beta =-m_{i}g$$

(51)

$$m^i{\ddot{z}}_i+c_z^i{\dot{z}}_i+k_z^i{z}_i+{\left(-1\right)}^i{F}_m\sin \beta +{\left(-1\right)}^i{F}_f\sin \beta =0$$

(52)

$$J_2{\ddot{\theta }}_2-F_m{R}_2\cos \left(\beta \right)-F_f{R}_f^2\sin \left(\beta \right)=T_2$$

(53)

The time-varying mesh stiffness of the system is a periodic excitation with a contact ratio of $m_p$(2 $<m_p<$3) and a mesh period of $T_m$($T_m$ = 2$\pi /{\omega }_m$). The contact ratio of the system is closely related to the variation in gear mesh stiffness, so the stiffness variation in the system is determined by the variation in the contact ratio of the system, as shown in Figure 4. For any integer $n\left(n\ge 1\right)$, the stiffness of the contact area of the three tooth pair ranges from $\left(m_p-2\right)T_m$ to $\left(m_p+n-3\right)T_m$, and the stiffness of the double tooth pair ranges from $\left(3-m_p\right)T_m$ to $n{T}_m$. In order to achieve higher accuracy in dynamic calculations, this study used the finite element method to calculate the time-varying mesh stiffness of helical gears.

3. Model Validation and Adaptability

3.1. Model Experimental Verification

To validate the accuracy of the gear dynamics model proposed in this paper, the experimental results from Feng [41] were compared with the simulation results obtained herein. The gear pair parameters used in the experimental verification are listed below in

Table 1. Geometrical and material parameters of helical gear pairs in Ref. [41].

Parameters	Pinion/Gear
$\mathrm{Number}\mathrm{of}\mathrm{teeth}{Z}_{g,p}$	30/53
$\mathrm{Helix}\mathrm{angle}\beta (°)$	30
$\mathrm{Pressure}\mathrm{angle}a$$(°)$	20
$\mathrm{Tooth}\mathrm{width}L$$\left(\mathrm{m}\mathrm{m}\right)$	20
$\mathrm{Module}m\left(\mathrm{m}\mathrm{m}\right)$	2.5
$\mathrm{Face}\mathrm{width}\left(\mathrm{m}\mathrm{m}\right)$	20
$\mathrm{Module}m\left(\mathrm{m}\mathrm{m}\right)$	2.5
$\mathrm{Base}\mathrm{circle}\mathrm{diameter}\left(\mathrm{m}\mathrm{m}\right)$	79.83/141.04
$\mathrm{Reference}\mathrm{diameter}\left(\mathrm{m}\mathrm{m}\right)$	86.60/153.00
$\mathrm{Center}\mathrm{distance}\left(\mathrm{m}\mathrm{m}\right)$	12.01
$\mathrm{Total}\mathrm{contact}\mathrm{ratio}\left(\mathrm{m}\mathrm{m}\right)$	2.65
$\mathrm{Shaft}\mathrm{length}\left(\mathrm{m}\mathrm{m}\right)$	160
$\mathrm{Shaft}\mathrm{average}\mathrm{radius}\left(\mathrm{m}\mathrm{m}\right)$	50
$\mathrm{Shaft}\mathrm{inner}\mathrm{radius}\left(\mathrm{m}\mathrm{m}\right)$	25/30
$\mathrm{Gear}\mathrm{damping}\mathrm{coefficient}{\xi }_m$	$5\times {10}^6$

.

The amplitude of the acceleration signal from the driven wheel was selected as the dependent variable, while the input torque was treated as the independent variable, resulting in Figure 5. As shown in the figure, the simulation results (Sim) are in closer agreement with the experimental data (Exp) compared to the traditional numerical method (Trad). Furthermore, the figure clearly illustrates that the simulation curve generated by the proposed model provides a more accurate reflection of the actual gear behavior compared to the reference model [41]. This improvement is attributed to the fact that the proposed modeling method projects forces onto each axis more accurately than traditional approaches, making it particularly suitable for simulating and predicting dynamic scenarios under flexible support conditions.

3.2. Universal Adaptability of Model

To explore the differences between the proposed model using the Lagrange method and traditional models under varying stiffness and driving speeds, the kinetic equation derived from the energy method in Ref. [29] is cited for comparison. The results of the two models are analyzed as follows. To ensure computational accuracy, the surface roughness coefficient is set to 0.1 $\mathsf{\mu }\mathrm{m}$, and the driving speed is set to 800 RPM. Since the primary focus of this paper is on the steady-state response of the gear system, the numerical procedure should be executed for a sufficiently extended duration until the difference between the last two cycles of the ${\mathsf{\delta }}_L$ is less than ${10}^{-8}$ m. The comparative analysis of ${\delta }_{LOA}$ and ${\delta }_{OLOA}$ is presented in the following results: The gear pair parameters used in the comparison are listed below in

Table 2. Geometrical and material parameters of helical gear pairs.

Parameters	Pinion/Gear
$\mathrm{Number}\mathrm{of}\mathrm{teeth}{Z}_{g,p}$	23/47
$\mathrm{Inertia}{I}_{g,p}(\mathrm{k}\mathrm{g}/{\mathrm{m}\mathrm{m}}^2)$	501.6/8748
$\mathrm{Gear}\mathrm{mass}m\left(\mathrm{K}\mathrm{g}\right)$	0.78/3.28
$\mathrm{Radial}\mathrm{bearing}\mathrm{stiffness}{DK}_z\left(\mathrm{N}/\mathrm{m}\right)$	$5\times {10}^8$
$\mathrm{Radial}\mathrm{axis}\mathrm{stiffness}{DK}_{x,y}\left(\mathrm{N}/\mathrm{m}\right)$	$6.17\times {10}^6$
$\mathrm{Module}m\left(\mathrm{m}\mathrm{m}\right)$	2.5
$\mathrm{Helix}\mathrm{angle}\beta (°)$	8.0
$\mathrm{Pressure}\mathrm{angle}a$$(°)$	20
$\mathrm{Tooth}\mathrm{width}L$$\left(\mathrm{m}\mathrm{m}\right)$	30
$\mathrm{Backlash}2C_b$$\left(\mathsf{\mu }\mathrm{m}\right)$	120
$\mathrm{Contact}\mathrm{surface}\mathrm{roughness}{R}_a\left(\mathsf{\mu }\mathrm{m}\right)$	0.1
$\mathrm{Tooth}\mathrm{top}\mathrm{height}\mathrm{factor}{h}_a^{*}\left(\mathrm{H}/\mathrm{a}\right)$	1.0
Damping coefficient of the shaft ${\xi }_s$	0.05
$\mathrm{Gear}\mathrm{damping}\mathrm{coefficient}{\xi }_m$	0.025

.

From Figure 6, it can be preliminarily concluded that as the radial stiffness increases, the results of the model established in this study gradually converge with those obtained using traditional methods. As shown in

Table 3. Different flexible support stiffness.

Driving Speed (RPM)	Support Stiffness (Nm)
	${\mathsf{\delta }}_{\mathbf{L}\mathbf{O}\mathbf{A}}$(1 × 106)	${\mathsf{\delta }}_{\mathbf{O}\mathbf{L}\mathbf{O}\mathbf{A}}$(1 × 106)	${\mathsf{\delta }}_{\mathbf{L}\mathbf{O}\mathbf{A}}$(6 × 106)	${\mathsf{\delta }}_{\mathbf{O}\mathbf{L}\mathbf{O}\mathbf{A}}$(6 × 106)	${\mathsf{\delta }}_{\mathbf{L}\mathbf{O}\mathbf{A}}$(1.2 × 107)	${\mathsf{\delta }}_{\mathbf{O}\mathbf{L}\mathbf{O}\mathbf{A}}$(1.2 × 107)
$800$	$25.6\mathrm{‰}$	$50.2\mathrm{‰}$	$1.8\mathrm{‰}$	$10.98\mathrm{‰}$	$0.36\mathrm{‰}$	$0.89\mathrm{‰}$
$1800$	$27.4\mathrm{‰}$	$51.7\mathrm{‰}$	$2.12\mathrm{‰}$	$10.18\mathrm{‰}$	$0.74\mathrm{‰}$	$1.68\mathrm{‰}$
$2800$	$26.5\mathrm{‰}$	$48.8\mathrm{‰}$	$2.15\mathrm{‰}$	$9.56\mathrm{‰}$	$0.66\mathrm{‰}$	$2.14\mathrm{‰}$

, at a support stiffness of ${10}^6$ $\mathrm{N}\mathrm{m}$, the difference in ${\delta }_{LOA}$ between the two models is approximately 25.6$\%$, while ${\delta }_{OLOA}$ reaches about 50.2$\%$, highlighting a significant disparity. Moreover, it is expected that as the support stiffness decreases, this divergence will further increase. This observation provides valuable insights for practical engineering applications. To quantify the system’s impact, an impact (residual) factor is introduced to represent the magnitude of the deviation. This factor is defined as $m$ = $\left(\left({DTE}_B-{DTE}_P\right)/{DTE}_P\right)$, Among them, ${DTE}_B$ is the model proposed in this article, while ${DTE}_P$ is the model proposed in Ref. [29].

To more clearly illustrate the differences between the two models under varying driving torques, Figure 7 depicts the relationship between dynamic response amplitude differences and driving torque. A total of 150,000 simulation data points were extracted to ensure the accuracy of the calculated dynamic response amplitudes. It is evident that significant deviations exist between the traditional method and the method proposed in this study, particularly when applied to scenarios involving different control parameters. Since helical gear systems are widely used under heavy load conditions, the traditional method is inadequate for accurately modeling systems with low support stiffness. Therefore, the dynamic model developed in this study is of great significance for improving the performance of helical gears in precision instruments.

4. Result and Analysis

4.1. Effect of Frictional Excitation on the Gear System

In order to explore the influence of friction on the helical gear system as a whole, based on the established lumped mass model in Section 2, a series of numerical simulations are conducted to illustrate the influence of driving speed on the dynamic characteristics of the gear system. The seven -order Runge–Kutta method with variable time steps is employed to solve the dynamic equations of the gear systems. Since the primary focus of this paper is on the steady-state response of the gear system, the numerical procedure should be executed for a sufficiently extended duration until the difference between the last two cycles of the ${\delta }_L$ is less than ${10}^{-8}$ m. The following are the changes in the maximum transmission error corresponding to different driving speeds.

From the observations in Figure 8, the gear system exhibits volatility in the low driving speed range due to the effects of friction. Specifically, at a driving speed of 1160 RPM, a significant change in transmission error occurs. This phenomenon is caused by the strong coupling effect between friction force and dynamic backlash, leading to resonance in the gear system. When the excitation frequency approaches the gear’s natural frequency (${\omega }_1$ = 444.4 Hz), resonance occurs. At low driving speeds, friction can alter the excitation frequency of the gear system. When this excitation frequency nears the output shaft’s natural frequency (${\omega }_2$ = 222 Hz), resonance peaks are also observed. This highlights the significant influence of friction-induced nonlinear dynamics on the gear system, particularly under low driving speed operating conditions. This kind of jump clearly has a significant impact on the stability of the system, so it is necessary to study the reasons for the occurrence of the “jump” at this time to maintain the stability of the system. Therefore, the first studied point ${\omega }_2$ = 575 RPM ($f_m$ = 222.2 $\mathrm{H}\mathrm{z}$). To investigate the reason for the “jump” of frictional force at this time, two resonant speeds were analyzed in the following content.

As shown in Figure 9, at a driving speed of 1160 RPM, when friction is neglected, the corresponding Poincare diagram exhibits nine distinct points, indicating that the system operates in a 9-periodic motion. The FFT analysis reveals the presence of a fourth order meshing frequency. However, when the influence of friction is considered, the phase-frequency diagram shows only two regular circles, and the corresponding Poincare diagram reduces to two discrete points. From the FFT spectrum of the system, it can also be observed that only the second order meshing frequency remains, indicating a transition of the system from 9-periodic motion to 2-periodic motion.

As illustrated in Figure 10, at a driving speed of 1160 RPM and without considering the effects of friction, the corresponding Poincare map displays five distinct points, indicating that the system operates in a 5-period motion. The FFT analysis reveals the presence of a fourth order meshing frequency. However, when the influence of friction is taken into account, the phase-frequency diagram reduces to a single regular circle, and the Poincare map consists of only one point. The FFT spectrum further confirms that only the first order meshing frequency remains, signifying a transition of the system from 5-period motion to 1-period motion.

To better observe the frequencies at points A and B in Figure 8, a three-dimensional frequency spectrum was constructed based on the driving speed range of 500–2000 RPM. As shown in Figure 11, when the system operates within the a-band (0–580 RPM), the first order meshing frequency resonates with both the output shaft and the input shaft in the subharmonic mode. In the three-dimensional spectrum, the vibration amplitude at point A is greater than that at point B. Consequently, the sudden increase in transmission error between points A and B can be attributed to the excitation of transverse resonance frequency by the frictional force present at this specific speed.

Based on the aforementioned findings, it is evident that in helical gear systems with flexible supports, particularly under low driving speed conditions, friction has a significant influence on gear dynamics. Therefore, the surface roughness coefficient $R_a$ can be used as a control parameter to further investigate the dynamic behavior of helical gear systems. This study selects a helical gear system operating under low driving speed conditions (0–740 RPM) as the research object. According to Refs. [25,42], the system exhibits complex dynamic behavior under these conditions. Therefore, the surface roughness range [0, 0.82] is chosen to analyze the bifurcation characteristics of the system. The bifurcation diagram provides a visual representation of the system’s dynamic characteristics under varying surface roughness conditions, revealing the relationships between periodic behavior and key parameters such as surface roughness. It also identifies potential critical points in the system and determines the response characteristics under different operating scenarios.

In Figure 12, the dynamic behavior of the gear system exhibits notable variations as surface roughness $R_a$ changes. At lower roughness values $R_a$ $\in$ [0.1, 0.48], the system demonstrates stable periodic motion, characterized by minimal displacement variations. During this range, the Largest Lyapunov exponents remains negative, further confirming the system’s stability and periodic behavior. As the roughness increases beyond $R_a$ = 0.48, the system undergoes period-doubling bifurcations, leading to increasingly complex dynamics. At approximately $R_a$ = 0.54, the system transitions into a chaotic state, as evidenced by the scattered points in the bifurcation diagram. This transition is also reflected in the Largest Lyapunov exponents, which shifts to positive values, signifying the onset of chaotic behavior. At higher roughness values $R_a$ $\in$ [0.63, 0.82], the system alternates between chaotic and quasi-periodic motion. In this range, the Largest Lyapunov exponents oscillates between positive and negative values, indicating the dynamic transitions between chaotic and quasi-periodic states. This observation highlights the complex interplay between surface roughness and the gear system’s nonlinear dynamics.

To more clearly demonstrate the dynamic characteristics of the helical gear system before and after the critical roughness values, time-domain responses, phase-frequency plots, Poincare maps, and FFT spectra under various critical roughness values ($R_a$ = 0.48, 0.52, 0.54, 0.63) are presented. Figure 13 shows that the motion period of the system undergoes significant changes under different critical roughness levels. Specifically, low roughness gear systems will experience continuous subharmonic motion. In this case, changes in roughness will not cause significant changes in gear dynamics, as shown in [0.1, 0.48] where the system is always in 2-period motion. As the roughness increases, the system will exhibit a period doubling phenomenon and become a stable 4-period motion; As $R_a$ continues to increase, the final Poincare map will exhibit chaotic attractors, and the corresponding Largest Lyapunov exponents will show positive values, indicating that the system is in a chaotic state. However, chaotic changes are not static. When $R_a$ $\in$ [0.63, 0.82], the motion of the system will transition from chaos to quasi periodic motion and then enter periodic motion.

Figure 14 illustrates the effect of surface roughness on the system in frequency domain. It can be clearly seen that within the roughness range [0.1, 0.52], the mesh frequency $f_m$ and its second harmonic are the dominant frequencies, with no other frequency components present. As the friction coefficient increases from 0.52 to 0.54, periodic motion is replaced by quasi-periodic motion. While $f_m$ remains the primary response, additional frequency components begin to emerge. When the friction coefficient exceeds 0.63, the gear system enters a chaotic state. The meshing frequency $f_m$ slightly decreases, while the amplitude of its subharmonic component 0.5$f_m$ increases significantly, surpassing the amplitude of $f_m$ and becoming the dominant system response. As surface roughness increases further, the system alternates between chaotic and periodic motion, and 0.5$f_m$ exhibits an initial decline followed by a sharp increase.

4.2. The Coupling Effect of Frictional Excitation and Fractal Excitation

In current studies on fractal theory, the influence of fractal dimension on the system cannot be ignored. Fractal dimension serves as a key parameter in describing the spatial distribution characteristics of surface roughness, which directly impacts the meshing behavior and dynamic performance of gear systems. To investigate the effect of fractal dimension, this paper analyzes the dynamic behavior of the system under varying surface roughness levels and fractal dimensions ($R_a$ = 0.2, 0.4, 0.6, 0.8).

From Figure 15, it can be observed that the effect of fractal dimension on the dynamic response of the gear system is not significant when the surface roughness is relatively low. As the surface roughness increases, the peak value of the dynamic response gradually intensifies. Simultaneously, the maximum dynamic transmission error shifts forward with the increase in surface roughness. This phenomenon is attributed to the alteration of the excitation frequency caused by changes in surface roughness, leading to an earlier resonance peak. When the roughness increases to $R_a$ = 0.8 $\mathsf{\mu }\mathrm{m}$, the influence of the fractal dimension under different driving speeds becomes significant. Notably, at low driving speeds (${\dot{\theta }}_1$ < 864 RPM), the peak value of the dynamic response increases, resulting in larger vibrations within the gear system. However, when the driving speed exceeds 864 RPM, the low driving speed vibration is alleviated, leading to a noticeable reduction in gear system vibrations caused by the fractal dimension. This reduction occurs because a higher fractal dimension corresponds to a smoother tooth surface, which reduces the impact of friction on the gear system.

To investigate the forward-shifting phenomenon in the dynamic response of gear systems caused by fractal effects, three-dimensional spectrum diagrams under different surface roughness $R_a$ conditions are analyzed. From Figure 16, it can be observed that when the surface roughness is relatively low ($R_a$ = 0.2 $\mathsf{\mu }\mathrm{m}$), the first order meshing frequency remains stable, with consistent amplitude. This indicates that under smooth surface conditions, the impact of fractal characteristics on the gear system has not yet become evident. However, as the surface roughness increases to $R_a$ = 0.4 $\mathsf{\mu }\mathrm{m}$, fluctuations in the dynamic response amplitude of the gear system begin to appear. Simultaneously, at a driving speed of 1437 RPM, the excitation frequency changes, leading to a shift in the resonance peak position. With further increases in roughness, the dynamic response peaks of the gear system exhibit more pronounced fluctuation characteristics, and the excitation frequency undergoes further alterations, causing significant changes in the second- and third order meshing frequencies. Finally, when the roughness reaches $R_a$ = 0.8 $\mathsf{\mu }\mathrm{m}$, the impact of roughness on the excitation frequency of the gear system reaches its maximum. At this point, higher surface roughness results in the gear system exhibiting spectrum characteristics that fluctuate with driving speed.

To more clearly investigate the influence of fractal dimension on the spectrum characteristics of gear systems, the driving speed corresponding to the maximum transmission error is analyzed. From Figure 17, it is observed that when the fractal dimension remains constant, the driving speed corresponding to the maximum transmission error decreases as the surface roughness increases. This phenomenon primarily occurs because the uneven distribution of backlash regions induces elastic deformation of the gear surface in different directions as the driving speed changes, causing a lag in the dynamic response. Consequently, the maximum transmission error shifts forward in speed. Furthermore, an analysis of the fractal dimensions (D = 1.1, 1.5, 1.9) reveals that the driving speed corresponding to the maximum transmission error increases with higher fractal dimensions. This indicates that a larger fractal dimension suppresses the excitation frequency, thereby mitigating the lag effect in the gear system.

To further investigate the impact of driving speed on the system, this study represents the maximum dynamic response based on input speed. By comparing Figure 8 and Figure 18, it is evident that when considering the coupled effects of fractal backlash and friction, the system’s primary resonance points (A and B) show no significant changes. However, new resonance points (C and D) emerge between points A and B within the two resonance regions. To explore the influence of this coupling relationship on the system in deep, a three-dimensional spectrum of the driving speed in the range of 500–2000 RPM is presented, illustrating the frequency variations.

From the spectrum shown in Figure 19, it can be observed that the amplitude variations of the resonance frequencies are minimal when coupling effects are considered. Nevertheless, as the speed increases, the number of frequency peaks grows, reflecting increasingly complex dynamic behavior. At lower driving speeds, the system spectrum response remains relatively stable, with amplitude unaffected by speed changes. However, as speed increases, the first noticeable frequency peak emerges at 0.5$f_m$. At approximately 850 RPM, a smaller amplitude frequency peak appears at $1/5f_m$. These findings demonstrate that new frequency components arise at higher driving speeds, highlighting the influence of the fractal structure on the system spectrum response.

To further investigate the impact of coupling effects on the dynamic characteristics of gears under low driving speed conditions, the influence of surface roughness on motion periodicity is analyzed. Figure 20 compares the dynamic behavior under coupling effects, considering only friction, with the results in Figure 12. By revealing the relationship between coupled periodic motion and key parameters such as surface roughness Ref. [35]. this analysis delves deeper into the periodic motion of the system under coupling effects.

From the figure, it can be seen that after considering the coupling effect, the motion period significantly increases, and the dynamic response of the system undergoes significant changes, especially in the nodes; The time-domain diagrams, Poincare maps, and FFT logarithmic amplitude diagrams of the system under different roughness conditions ($R_a$ = 0.4, $R_a$ = 0.52, $R_a$ = 0.6) were presented to analyze the nodes. As shown in Figure 21, it can be clearly observed from the figure that with the increase in roughness, compared to only considering the friction effect, the transition from 2-period to 4-period-chaotic motion is presented. Considering the coupling phenomenon, the transition from multi period chaotic multi period motion is exhibited. Specifically, when $R_a$ = 0.4, the Poincare diagram considering only the friction effect presents two isolated points, and the corresponding FFT spectrum only shows integer order meshing frequencies, indicating that the system’s motion exhibits a motion period characteristic of 1$t_c$. However, considering coupled motion, in addition to an increase in Poincare points, the spectrum shows a multiplication and division component of the meshing frequency 0.5$f_m$, which is different from the subharmonic motion characteristic indicating that the system’s motion exhibits $2t_c$; when the roughness $R_a$ = 0.52, the four isolated steady-state attractors without considering coupling effects are replaced by chaotic attractors, and the spectrum shows chaotic edge frequency phenomena, indicating that the steady-state motion of the system has been replaced by chaotic motion and that the system has transformed from multi period motion to chaotic motion. As the roughness increases, $R_a$ = 0.6, the opposite phenomenon occurs, that is, the chaotic attractor without considering coupling effects is replaced by the steady-state attractor, and the chaotic edge frequency phenomenon of the spectrum becomes a stable spectrum phenomenon dominated by 0.5$f_m$, indicating that the chaotic phenomenon of the system is suppressed.

As shown in Figure 22, with the change in motion period, the frequency of the system shows that $R_a$ $\in$ [0, 0.17] system maintains multi period motion, and no other frequency components appear in the corresponding three-dimensional spectrum. When the motion period of $R_a\in$ [0.17, 0.45] system changes, the phenomenon of increasing motion period corresponds to the multiplication and division components of meshing frequency 0.5$f_m$ appearing in the three-dimensional spectrum; when $R_a$ $\in$ [0.48, 0.53], considering the coupling effect, the system will enter a chaotic state, and the corresponding Largest Lyapunov exponents graph will show positive values, indicating that it is in a chaotic state. At this time, there are more frequency components in the three-dimensional spectrum, but the amplitude value of the meshing frequency $f_m$ is much larger than that of other frequency components; As the roughness increases, the amplitude value of the system frequency component will also increase. Unlike not considering coupling effects, in the roughness range of $R_a$ $\in$ [0.6, 0.69] at higher roughness, considering coupling effects will actually reduce the instability factor of the system, and the system will transition from chaos to periodic motion; As the roughness continues to increase, the system enters a chaotic state with an increasing number of attractors, and similarly to not considering coupling effects, 0.5$f_m$ will suddenly increase, becoming the main system frequency exceeding the first-order meshing frequency, and other frequency components will increase.

5. Conclusions

This article proposes a new 7-DOF dynamic model for helical cylindrical gear systems with dynamic meshing parameters, which focuses on the influence of sliding friction and fractal roughness on gear vibration under high flexibility support stiffness conditions. The effects of the modulated input speed and the new dynamic model on the dynamic response and frequency of the gear system are analyzed. The simulation results demonstrate that the proposed dynamic model can produce a more realistic dynamic response than the previous model under high flexibility support stiffness. The input speed has a significant influence on the dynamic characteristics of the gear system. In the low-to-medium input speed range, two resonance points,$V_1$ (575 RPM) and $V_2$ (1160 RPM), are observed. Notably, at low driving speeds, the resonance effect of the flexible shaft results in the maximum dynamic transmission error of the system.

Moreover, as surface roughness increases, friction induces transitions in the system’s motion, including double-period bifurcations, quasi-periodic to chaotic motion evolution. During chaotic periodic motion, the system’s 0.5$f_m$ vibration frequency exceeds the primary meshing frequency $f_m$, becoming the dominant vibration frequency of the system. When the fractal effect couples with friction in the gear system, the motion periods significantly increase, and the resonance points exhibit slight shifts with changes in speed. In the corresponding second-order steady-state period (c) and fifth-order steady-state period, two abrupt changes occur, as revealed in the three-dimensional frequency diagram with sudden peak values at 0.5$f_m$ and 0.2$f_m$. This indicates that the coupled effects of friction and fractals not only enhance the system’s dynamic response but also lead to more complex nonlinear spectral dynamics.

In future work, this model will be applied to analyze the dynamic characteristics of gear-shaft-bearing systems, and finite element methods will be employed to design the shafts.