Time-Series Machine Learning Techniques for Modeling and Identification of Mechatronic Systems with Friction: A Review and Real Application

Abstract

Developing accurate dynamic models for various systems is crucial for optimization, control, fault diagnosis, and prognosis. Recent advancements in information technologies and computing platforms enable the acquisition of input–output data from dynamical systems, resulting in a shift from physics-based methods to data-driven techniques in science and engineering. This review examines different data-driven modeling approaches applied to the identification of mechanical and electronic systems. The approaches encompass various neural networks (NNs), like the feedforward neural network (FNN), convolutional neural network (CNN), long short-term memory (LSTM), transformer, and emerging machine learning (ML) techniques, such as the physics-informed neural network (PINN) and sparse identification of nonlinear dynamics (SINDy). The main focus is placed on applying these techniques to real-world problems. A real application is presented to demonstrate the effectiveness of different machine learning techniques, namely, FNN, CNN, LSTM, transformer, SINDy, and PINN, in data-driven modeling and the identification of a geared DC motor. The results show that the considered ML techniques (traditional and state-of-the-art methods) perform well in predicting the behavior of such a classic dynamical system. Furthermore, SINDy and PINN models stand out for their interpretability compared to the other data-driven models examined. Our findings explicitly show the satisfactory predictive performance of six different ML models while also highlighting their pros and cons, such as interpretability and computational complexity, using a real-world case study. The developed models have various applications and potential research areas are discussed.

1. Introduction

The development of an accurate dynamic model is crucial for the purpose of optimization, control, fault diagnosis, and prognosis [1,2,3]. There are three main modeling approaches for dynamic systems: the physics-based approach, data-driven approach, and hybrid approach. In the physics-based approach, the governing equations of a system are formulated from conservation laws, and they are solved analytically or numerically, while the data-driven approach uses a combination of data and deep learning techniques.

Machine learning (ML) techniques [4], and its subset, that is deep learning (DL) methods [5] such as Neural networks (NNs), have received significant attention in the field of science and engineering due to their capability in modeling nonlinear and complex systems [6]. The modeling of a dynamic system using NNs or ML techniques can be implemented in two ways: purely data-driven or physics-informed data-driven implementation. The purely data-driven model, otherwise called a surrogate model, is developed by creating a neural network architecture that maps the input of a system to its output while the network parameters are optimized to ensure an accurate prediction. Examples of such neural networks are the multilayer perceptron (MLP) or FNN, CNN, recurrent neural network (RNN), and transformer. A recent ML technique that is used to discover parsimonious models of a system from data is called sparse identification of nonlinear dynamics (SINDy). This method has the advantage of interpretability and is subject to a tradeoff between structural complexity and the accuracy of the resulting models [7,8].

In the case of the hybrid modeling, a physics-informed neural network is a new form of NN that uses physical laws (or insights) described by ODEs or PDEs to guide the construction, training, and optimization of its underlying data-driven model [9]. When training a PINN model, the physical laws provide constraints to the network, and the advantage of this is that the issue of overfitting is prevented, less training data are required, and the model becomes robust [10,11,12].

Furthermore, there are different nonlinear phenomena in mechatronic systems and a data-driven or hybrid model can handle such complex characteristics. Friction is the most common one, and it usually affects the dynamics and overall efficiency of any system. When modeling a mechatronic system through the physics-based approach, frictional losses are integrated into the model equation to improve the model’s accuracy. The problem with this is that there is no general model for capturing frictional losses and friction models (whether static or dynamic) [13,14] introduce more unknown parameters to the model equations. Thus, it is advantageous to use a purely data-driven or hybrid approach in identifying friction [15].

This review investigates various data-driven modeling methods employed for identifying mechanical and electronic systems with a real-world classic system. There are many data-driven methods that have been proposed in the literature; however, the scope of this work is limited to these machine learning methods: FNN, CNN, LSTM, transformer, SINDy and PINN. Other promising data-driven methods such as symbolic regression [16,17], dynamic mode decomposition [18,19], and new variants of neural networks like Fourier neural networks [20] are not covered. The contributions of this work are as follows:

• Four distinct purely data-driven models were developed for a geared DC motor, namely, FNN, CNN, LSTM, and transformer models. These models demonstrated high prediction accuracies. In addition, SINDy approach was employed to directly discover various parsimonious models of the system from experimental data. Using different optimizers, three sparse regression models were generated, utilizing a library of candidate functions with a polynomial order of 2;
• PINN modelling method was employed to resolve coupled first-order differential equations governing a geared DC motor. The unidentified physical parameters of the dynamical system were successfully determined. The results demonstrate that the PINN approach effectively forecasts the dynamic response of the nonlinear system and estimates the unmeasured system state, specifically the armature current.

Our findings clearly demonstrate the satisfactory predictive capabilities of six distinct ML models. Additionally, we shed light on their strengths and weaknesses, including factors like interpretability and computational complexity, through a practical case study.

The paper’s structure is depicted in a mind map in Figure 1. Section 2 covers essential background concepts, such as system identification, forward and inverse modeling problems, online and offline parameter identification, and dynamic modeling with friction. In Section 3, we discuss the utilization of neural networks and machine learning techniques for the dynamic modeling of mechatronic systems. Additionally, Section 4 presents a case study on the dynamic modeling of a geared DC motor, employing various data-driven and hybrid approaches, along with their potential applications. Finally, in Section 5, the paper summarizes the research contributions and identifies gaps for future studies.

The results presented in this paper can be reproduced through the code available on GitHub [21].

2. Background

In order to provide a proper context to our work, the following key concepts of modeling are presented in this section: system identification, forward and inverse problems, online and offline parameter identification, and dynamic modeling with friction.

2.1. System Identification

System identification, as the name suggests, is the process of identifying a system behavior by means of its measured or observed input and output data [22]. Through different algorithms that are based on numerical optimization, the relationship between the system input and output is mapped during identification [1].

As illustrated in Figure 2, system identification involves four main steps: data collection or acquisition, model structure definition or selection, model parameters estimation, and model validation [2,23]. The first step involves getting input–output data from the physical system by designing and conducting an experiment covering the system’s operating limits/bounds. It is crucial to acknowledge that the obtained dataset could have outliers and this necessitates data prepossessing operations such as normalization, resampling, or filtering. Secondly, a model structure such as transfer function, state-space, autoregressive with exogenous (ARX), or neural network is selected depending on the available insight or prior knowledge of the system under consideration. In the third step, the model parameters are trained or estimated using different algorithms like LS or backpropagation. During estimation, the model parameters act as a driver and play the role of driving the model, thereby fitting its response to match that of the actual system. The fourth and last step is to validate the estimated model. This is performed to guarantee that the developed model will serve the intended application (e.g., control or fault detection). In a scenario where the developed model turns out to be unsatisfactory, the identification process is reworked (i.e., performed all over again) [2,4]. For this reason, system identification is regarded as an iterative process and it involves the user trying out different choices in terms of the model structure and even the estimation algorithm to arrive at the best model.

There are three main modeling approaches: physics-based, data-driven, and hybrid modeling.

• Physics-based modeling: This method utilizes knowledge of the underlying physical laws governing the system’s behavior. The equations and parameters are theoretically determined and solved [1]. However, in reality, a purely physics-based model is impractical for large-scale or complex systems [24,25]. The model developed through this approach is referred to as a mathematical one;
• Data-driven modeling: This method is suitable for complex nonlinear systems where little or no knowledge of their governing physical laws exists. The model structure and parameters are unknown, and the model architecture is not based on physical laws. As a result, the parameters often lack physical meaning or interpretation [1,26]. The model developed through this approach is known as data-driven or intelligent;
• Hybrid modeling: In this case, partial knowledge about the system is available. The model structure is either fully or partially known, and its parameters are estimated using system data [1,27]. The resulting model is called hybrid or physics-informed data-driven.

The three modeling approaches are depicted in Figure 3. The upper arrow indicates increasing complexity, while the lower arrow indicates the direction of increasing knowledge and model assumptions according to the modeling methods.

Remark 1.

Based on their level of transparency and ease of understanding, the modeling approaches are classified as follows: (i) white-box modeling, which relies on physics-based principles; (ii) black-box modeling, which is data-driven in nature, (iii) grey-box modeling, which combines physics-based and data-driven aspects [1].

2.2. Forward and Inverse Modeling Problems

The modeling problems posed by dynamical systems can be classified into forward and inverse problems. The forward modeling problem involves computing a system response or solution given its mathematical model and parameters [28]. In forward problems, ordinary differential and partial differential equations (ODEs and PDEs) are solved analytically or numerically using the Euler method, Runge–Kutta method, finite element method, finite difference method, and so on. In contrast, the inverse modeling problem involves the estimation of unknown model parameters that would give a small prediction error when compared to the output data observed from a physical system [29]. The parameters can be identified (estimated) using various algorithms such as least squares (LS) [30], recursive least squares (RLS) [31], the genetic algorithm (GA) [32], NNs [33], etc.

2.3. Online and Offline Parameter Identification

The concept of identifying the model parameters of a system is known as parameter identification, and this topic can be viewed as an inverse modeling problem. Parameter identification is performed either online or offline. In online parameter identification, model parameters are estimated in real time with the help of algorithms such as RLS [30,34] and the extended Kalman filter [35,36], as new sensor data are available from the physical system. On the other hand, offline parameter identification is conducted by acquiring input–output data from the system, storing the data, and then using an algorithm like LS or GA to estimate the unique parameters of the system. In

Table 1. Differences between online and offline model parameters identification.

Property	Online Parameter Identification	Offline Parameter Identification
Type of estimated parameter	Time-varying parameters	Constant or unique parameters
Data preprocessing	Difficult to integrate into the estimation process	Easy to integrate into the estimation process
Computational power and storage	High as powerful controllers are needed	The computational demand is moderate

, the two-parameter identification methods are compared in terms of the type of parametric results they give, the ease of data preprocessing, and the computational power and storage demands.

2.4. Dynamic Modeling with Friction

In mechatronic systems, such as pendulums, DC motors, and mass-damper setups, modeling can be accomplished using either first principles or physical laws. However, achieving accurate models often requires identifying the system’s parameters. One major challenge is accurately representing the impact of friction, a nonlinear phenomenon that affects the efficiency of these systems [37,38].

Friction models can be categorized into static and dynamic models. Static models assume no relative motion during standstill (resting friction phase) and describe the relationship between friction force and relative velocity. On the other hand, dynamic models capture the features of presliding and sliding regions, including their transition effects [39].

Static models consist of Coulomb friction, viscous friction, the Stribeck effect, Karnopp, and Armstrong. Moreover, dynamic models include Dahl, LuGre, Leuven, and bristle friction models [40,41,42,43].

Applying an accurate friction model can facilitate friction compensation, helping to prevent issues such as slow responses, tracking errors, stick-slip motion, and limit cycles [44,45]. Notably, for dynamic systems developed purely through data-driven approaches, a separate friction model is unnecessary, as frictional losses are already accounted for in the experimental data used to build the model.

3. Dynamic Modeling of Mechanical and Electronic Systems Using Neural Networks and Machine Learning Techniques

Artificial neural networks (ANNs) or simply NNs are deep learning techniques (which is a subset of machine learning) inspired by the functioning of neurons in the brain. There are three fundamental learning approaches used in machine learning: they include supervised, unsupervised, and reinforcement learning [46]. Neural networks predominantly depend on supervised learning, utilizing labeled data for predictive tasks. In contrast, unsupervised learning reveals patterns in unlabeled data, while reinforcement learning centers on trial-and-error decision-making processes.

An NN can learn complex functions by extracting relations from training data; thus, it is regarded as a universal approximator. NNs can be used in developing data-driven models for nonlinear, multivariable, and complex dynamics. A typical NN consists of interconnected neurons with weights and a bias. In addition, an activation function is also used in a neuron to filter the information that is transferred to the next neuron. Some common activation functions are the rectified linear unit (ReLU), sigmoid, hyperbolic tangent function (Tanh), and pure linear function [47,48]. The algebraic calculation that happens within a single neuron is:

$$z=\sum _{i=1}^n{w}_i{x}_i+b,$$

(1)

$$\widehat{a}=f_{act}\left(z\right),$$

(2)

where $\widehat{a}$ is the output of the neuron after activation, z denotes the output of the neuron after passing the summation unit, $f_{act}\left(z\right)$ represents the activation function, $w_i$ is the weight associated with the i-th input, b is the bias, and n is for the number of input (x).

The model of a single neuron is depicted in Figure 4, see [49].

Through supervised training, a learning algorithm is used to iteratively adjust the network parameters based on the error between the predicted output and the training data outputs. The error is computed during a forward pass while the weights are optimized from the output layer to all the other layers during the backward pass (this is called error backpropagation) [50]. Most NN models use the gradient-based method to minimize the loss J of the network, i.e.,

$$J=MSE(y,\widehat{y})=\frac{1}{m}\sum _j^m{\left(y_j-{\widehat{y}}_j\right)}^2,$$

(3)

where J is the loss between the target output and predicted network output, y the target output, $\widehat{y}$ is the predicted output by the network, and m represents the number of data points. Accordingly, the weights and bias being the trainable parameters of the neuron are updated as iteratively as follows:

$$w_i^{new}=w_i^{old}-\epsilon \frac{\partial J}{\partial w_i^{old}},b^{new}=b^{old}-\epsilon \frac{\partial J}{\partial b^{old}},$$

(4)

where $\epsilon$ is defined as the learning rate.

The manner in which gradient descent optimizes the weight parameter w is illustrated in Figure 5.

Besides gradient descent, other common training algorithms are adaptive gradient descent, momentum, adaptive momentum, Newton, quasi-Newton, and Levenberg–Marquart.

In this section, different types of NN structures, such as MLP, CNN, RNN and PINN, including ML methods such as SINDy will be discussed. Moreover, we will compare those models based on their advantages, disadvantages, and applications.

3.1. Multilayer Perceptron Network

An MLP or FNN has three layers: the input layer, hidden layers, and output layer. The MLP network illustrated in Figure 6 consists of an input, three hidden layers, and an output layer. It is possible to improve the fit of a network by using more layers; however, this can also increase the training time. The size of neurons in the hidden layer can cause a poor fit (underfitting), while many neurons can result in overfitting (a poor generalization of the network to new data). A middle way is usually sought through trial and error for a good result [50].

A nonlinear autoregressive with exogenous inputs (NARX) model is commonly used in the data-driven modeling of dynamic systems. The structure of a NARX model is

$$y_{t+1}=f\left(x_t\right),$$

(5)

where $y_{t+1}$ is the predicted output at a future time $t+1$, f is a nonlinear function, and $x_t$ is the regressor vector of the form

$$x_t=\left[y_t,y_{t-1},\dots ,y_{t-n_y+1},u_t,u_{t-1},\dots ,u_{t-n_u+1}\right],$$

(6)

where $n_u$ and $n_y$ are time delays, and they are determined based on the system order.

An FNN can be employed for estimating the nonlinear function of the NARX model structure. In the specific case study [4] presented, a NARX-based neural network was utilized to develop a data-driven model for a DC motor. This neural network mapped the nonlinear relationship between the motor’s PWM signal and its speed. Similarly, in another study [51], an NN was used to model a brushless DC motor. The NN was trained with experimental data comprising torque, angular velocity, and motor efficiency. The dataset was split into training (85%), testing (10%), and validation (5%) datasets. The trained model took torque and angular velocity as inputs and provided motor efficiency as the output. Post-training, the NN model demonstrated good performance, with a resulting mean square error of 1.5.

3.2. Convolutional Neural Network

A convolutional neural network is popularly used in the field of computer vision for classification tasks, but it is also useful for regression tasks [52]. The convolutional layer is the central part of a CNN architecture and Figure 7 shows a one-dimensional CNN architecture. Suppose we are given an $n\times 1$ time-series input data. The data are first transformed to a three-dimensional array by data sequencing using g number of data points as the sliding window. During the convolution process, a set of weights known as kernels or filters slide through the transformed input data. The dot product of the kernel weights and the input local receptive field is computed, they are summed, a bias is added, and each element of the resulting array is passed through a nonlinear activation function [52,53].

Mathematically, convolution operation is expressed as:

$$z_{ij}^l=\sum _i^{i+K-1}{W}_j^l\ast z_i^{l-1}+b_{ij}^l,$$

(7)

where $Z_{ij}^l$ and $Z_i^l$ are the output and input of the convolution, the convolution operator is denoted by ∗, i is the iteration index, which is determined by the input ($Z_i^l$) size, $W_j^l$ represents the kernel or filter of the j-th neuron in layer l, K is the kernel size, and $b_{ij}^l$ is a scaler bias of layer l.

The convolution output $z_{ij}^l$ is passed through an activation function (such as ReLU) and the final output of the layer is:

$$O_{ij}^l=f_{act}\left(z_{i,j}^l\right).$$

(8)

The outputs of the convolutional layer $O_{ij}^l$, which are called feature maps, are flattened to a feature vector at the fully connected layer. The other layers that are used in a CNN architecture for classification purposes include the pooling layer and dropout layer, which are used to reduce the computation size of the model and help to prevent overfitting. A dense layer can also be used after the fully connected layer, particularly when dealing with a regression problem [10,53].

A CNN was used in [54] to identify the model of a nonlinear system. The Weiner–Hammerstein system was used to form the dataset and the performance of a CNN and MLP in system identification was compared. The simulation result showed that the CNN performs better than MLP when a noisy data are used.

3.3. Recurrent Neural Network

An RNN is a type of NN that is suited for modeling time-series or sequential data. The reason is that RNN, unlike MLP, uses the concept of memory, allowing the network to store the state of the previous step or useful information from past inputs, and that information is used when generating the output of the next sequence [52]. An RNN architecture is shown in Figure 8.

It is worth noting that the same weights and biases are used in the network as it unfolds at each time step t. At time t, the network input is $x_t$, the state is $s_t$, the output is $y_t$, and particularly:

$$s_t=f(w_x{x}_t+w_s{s}_{t-1}+b_s),y_t=f(w_y{s}_t+b_y),$$

(9)

where $w_x$, $w_s$, and $w_y$ are the weights associated to the input, state, and output, respectively; $b_s$ and $b_y$ are the state and output biases.

An RNN uses backpropagation through the time (BPTT) algorithm to update its weight and minimize the total error of the network [55].

$$E_N=\sum _{t=1}^N{y}_{t,\mathrm{actual}}-y_t.$$

(10)

The partial derivatives/gradient of the network with respect to the weights can be mathematically calculated as follows:

$$\frac{\partial E_N}{\partial w_s}=\sum _{i=1}^N\frac{\partial E_N}{\partial Y_N}\frac{\partial Y_N}{\partial s_i}\frac{\partial s_i}{\partial w_s},\frac{\partial E_N}{\partial w_x}=\sum _{i=1}^N\frac{\partial E_N}{\partial Y_N}\frac{\partial Y_N}{\partial s_i}\frac{\partial s_i}{\partial w_x},\frac{\partial E_i}{\partial w_y}=\frac{\partial E_i}{\partial Y_i}\frac{\partial Y_i}{\partial w_y}.$$

(11)

However, due to the huge multiplication involved when the partial derivative of the error is calculated across the hidden states, the gradient may end up exploding or vanishing.

There are different RNN architectures that are used to solve various ML problems. They are bidirectional recurrent neural networks, gated recurrent units, or even LSTM [56].

3.4. Long Short-Term Memory

This is a special type of RNN that can handle long-term dependencies in data and can also eliminate the vanishing gradient problem of an RNN. An LSTM cell depicted in Figure 9 stands for a computational unit that controls the flow of information. The key parts of an LSTM unit are the cell state, forget gate, input gate, and output gate. The operations performed in an LSTM unit to process information are as follows:

• Forgetting the irrelevant part of the previous state;

  $$f_t=\sigma (w_f[h_{t-1},x_t]+b_f).$$

  (12)
• Storing the most relevant new information;

  $$i_t=\sigma (w_i[h_{t-1},x_t]+b_i).$$

  (13)
• Updating the internal cell state;

  $$C_t=f_t·C_{t-1}+i_t·{\overline{c}}_t,$$

  (14)

  where ${\overline{c}}_t=tanh(w_c[h_{t-1},x_t]+b_c)$ and · represents element-wise multiplication.
• Generating an output;

  $$h_t=tanh\left(C_t\right)·o_t,$$

  (15)

  where $o_t=\sigma (w_o[h_{t-1},x_t]+b_o)$.

When training the LSTM network, the weights and biases of the input, output, and forget gates are updated at each time step through the BPTT algorithm [52,55].

The LSTM network was used to model the rolling friction of a mechanical system in  [57] to capture the system’s friction characteristics at the presliding region. The authors proposed an initial value design for the RNN such that the final value of the last training set is used as the initial value of the next training. The effectiveness of the developed model was verified with the LuGre and Kiozumi models using an experimental setup consisting of a motor, gear, load, encoder, and a clamp to adjust the friction level.

3.5. Transformer Network

Transformers, well-known for their sequence generation capabilities, have garnered significant attention in the domain of natural language processing due to their impressive performance in generative tasks. Their success can be attributed to the attention mechanism, which enables the learning of long-term dependencies in data without the need for recurrent connections, such as RNN and LSTM [58]. Furthermore, transformers offer the advantage of supporting parallel processing of input sequences, unlike LSTM, which requires sequential processing. This feature makes transformers highly efficient for training, especially when dealing with extensive datasets.

The encoder part of the transformer network illustrated in Figure 10 can be used in time-series prediction, fault classification, anomaly detection, etc. [59,60].

The steps involved in time series-based modeling using transformer encoder networks are as follows:

• Data preparation into sequence. Time series data acquired experimentally from a mechatronic system are intrinsically ordered, thus negating the need for position encoding;
• The sequence is inputted to a transformer encoder, which comprises N multihead attention and feedforward layers [59,60,61]. The multihead attention enables the model to understand both long- and short-term dependencies or relationships among various time steps in the sequence. Moreover, the feedforward layer is responsible for capturing high-level and complex interactions and patterns in the data.

  $$Multihead(Q,K,V)=concat(Hea{d}_1,Hea{d}_2,\cdots Hea{d}_h)W^0,$$

  (16)

  $$Hea{d}_i=Attention(Q{W}_i^Q,K{W}_i^K,V{W}_i^V),$$

  (17)

  where $W_i^Q$, $W_i^K$, and $W_i^V$ are the weight matrices of the i-th attention head, $W^0$ represents the weight matrix of the multihead attention, h represents the total number of heads, and concat is a function for concatenating the outputs of the attention heads.
  The equation below describes the attention mechanism used in the network:

  $$Attention(Q,K,V)=Softmax\left(\frac{Q\times K^T}{\sqrt{d_K}}\right)\times V,$$

  (18)
  In this equation, $Q=X{W}^Q$, $K=X{W}^K$, and $V=X{W}^V$ represent the query, key, and value matrices, respectively. These matrices are obtained by multiplying the input feature matrix X with the parameter matrices $W^Q$, $W^K$, and $W^V$. Here, ’d’ denotes the dimension of Q, K, and V.
  The feedforward layer processes the output of the multihead attention layer. It consists of a fully connected layer (e.g., MLP or 1D CNN can be used), followed by a ReLU or Gaussian error linear unit (GeLU) activation function, and a dropout layer;
• The output layer involves a fully connected layer along with a linear activation function;
• Training and optimization: The transformer encoder undergoes training with input sequences, and the corresponding system output serves as the target. The model is then optimized through a fitting loss function like MSE (refer to Equation (3)) to minimize the discrepancy between predicted and actual values;
• Model evaluation: after being trained, the transformer encoder can be utilized for inference, enabling predictions to be made using novel sequences of dynamic system data.

Though transformer network is a state-of-the-art algorithm but its application in the dynamic modeling of systems is limited as reported in [62].

3.6. Physics-Informed Neural Networks

The physics-based modeling method is easily affected by a high bias arising from simplifying assumptions. Similarly, the solely data-driven approach exhibits a high variance when the experimental data fails to encompass the system’s entire operating range. To address these respective issues of bias and variance in physics-based and purely data-driven modeling, a novel data-driven modeling technique based on physics-informed neural networks (PINN) can be employed.

Recently, there has been a growing interest in the application of PINNs in science and engineering. PINNs effectively combine data and physical laws to address supervised learning tasks. This novel approach finds utility in solving forward problem-solving tasks involving ODEs and PDEs. Additionally, PINNs demonstrate proficiency in handling inverse problems, such as parameter identification in differential equations. The implementation of the algorithm delivers the continuous and discrete PINN approach. In the continuous one, the system equation is integrated into the neural network architecture’s loss function.

Consider solving the following ordinary differential equation:

$$\frac{du}{dt}=f(u\left(t\right),t,\lambda ),\mathrm{with}u\left(0\right)=u_0\mathrm{and}t\in [0,T],$$

(19)

where u represents the dependent variable, t is the independent variable (time), and $\lambda$ is a system parameter.

In the continuous PINN approach, we approximate the solution to Equation (19) using a neural network $\tilde{N}\left(t\right)$ such that $\tilde{N}\left(t\right)\approx u\left(t\right)$, with $\tilde{N}\left(0\right)\approx u_0$.

By computing the derivatives of the network output with respect to its inputs using automatic differentiation [63], we can calculate the equation’s residual in the form:

$$r\left(t\right)=\frac{d\tilde{N}\left(t\right)}{dt}-f(\tilde{N}\left(t\right),t,\lambda ).$$

(20)

This motivates the loss function to optimize the neural network:

$$L_T=L_s+L_{eq}+L_i,$$

(21)

$$L_T=\sum _i{\left(\tilde{N}\left(t_i\right)-u\left(t\right)\right)}^2+\sum _i{\left(\frac{d\tilde{N}\left(t_i\right)}{dt}-f(\tilde{N}\left(t_i\right),t_i,\lambda )\right)}^2+{\left(\tilde{N}\left(0\right)-u_0\right)}^2.$$

(22)

The model’s training objective is to minimize $L_T$ by adjusting the weights (w) and biases (b) of the network $\tilde{N}$. The physics loss ensures that the system’s governing equations are enforced. Various case studies and engineering problems where this approach has been applied can be found in [10,12,63,64,65,66,67,68]. Additionally, Figure 11 illustrates a continuous PINN architecture demonstrating the physics-guided loss formulation.

The discrete approach or multistep PINN involves utilizing multistep discretization techniques such as forward Euler and other Runge–Kutta schemes to numerically discretize the system equation over time. The unknown functions or state update vector are approximated using neural networks.

To solve the first-order Equation (19) numerically, two methods can be employed. The first one is the forward Euler method, expressed as:

$$u_{n+1}=u_n+\Delta t·f(t_n,u_n),$$

(23)

where f represents a function dependent on t and u itself. Alternatively, the same equation can be solved using the Runge–Kutta 4th order method:

$$u_{n+1}=u_n+\frac{\Delta t}{6}(k_1+2k_2+2k_3+k_4),$$

(24)

where

$$t_{n+1}=t_n+\Delta t,$$

$$k_1=f(t_n,u_n),$$

$$k_2=f\left(t_n+\frac{\Delta t}{2},u_n+\frac{k_1}{2}\right),$$

$$k_3=f\left(t_n+\frac{\Delta t}{2},u_n+\frac{k_2}{2}\right),$$

$$k_4=f\left(t_n+\Delta t,u_n+k_3\right),$$

and $\Delta t$ is the step size.

The functions present in both discretization schemes are replaced with neural networks and the networks are trained using the regular loss of typical neural networks (i.e., the MSE between the actual u and predicted $\widehat{u}$ at each time step). This approach was discussed in following papers [69,70,71] and used to model the nonconservative forces in the dynamic equation of an inverted pendulum in [11]. PINNs are usually implemented with feed-forward neural networks because they are faster to train. However, this does not rule out the possibility of using other NNs. For instance, a physics-informed ML approach was investigated in a recent study for modeling the surface roughness of a milling process [72].

3.7. Sparse Identification of Nonlinear Dynamics

SINDy is a promising ML technique that can be used for the discovery of a system’s governing equations directly from data. This technique was introduced in [7], and it can be used to discover ordinary and partial differential equations. Suppose we consider a nonlinear dynamical system with the form:

$$\dot{x}\left(t\right)=f(x\left(t\right),u),$$

(25)

where $x\left(t\right)$ represents the state of the system, u is the control input, and f denotes the vector field of the system.

The SINDy algorithm is implemented in the following steps to find the parsimonious model that will describe the system by employing sparse regression.

• Collect the time-series data of the dynamical system, X, and U and compute the derivative of system states, $\dot{X}$. If x comprises m states, then $x=[x_1,x_2,\dots ,x_m]$ and:

  $$X=\left[\begin{array}{c}x\left(t_1\right)\\ x\left(t_2\right)\\ ⋮\\ x\left(t_n\right)\end{array}\right]=\left[\begin{array}{cccc}{x}_1\left(t_1\right)& x_2\left(t_1\right)& \cdots & x_m\left(t_1\right)\\ x_1\left(t_2\right)& x_2\left(t_2\right)& \cdots & x_m\left(t_2\right)\\ ⋮& ⋮& \ddots & ⋮\\ x_1\left(t_n\right)& x_2\left(t_n\right)& \dots & x_m\left(t_n\right)\end{array}\right],$$

  (26)

  $$U={[u\left(t_1\right),u\left(t_2\right),....,u\left(t_n\right)]}^T,\dot{X}={[\dot{x}\left(t_1\right),\dot{x}\left(t_2\right),\dots ,\dot{x}\left(t_n\right)]}^T,$$

  (27)

  where $t_1,t_2,\dots ,t_n$ are the sampling points of the time-series data;
• Create a library candidate function that consists of constant, polynomial of degree k (which is user defined) and the trigonometric terms:

  $$\theta (X,U)=[1,X^{P_1},U^{P_1},X^{P_2},U^{P_2},X^{P_k},U^{P_k},XU,\dots ,sin\left(X\right)sin\left(U\right),\dots ].$$

  (28)
• Solve a sparse regression problem:

  $$\dot{X}=\theta (X,U)E,$$

  (29)

  where the matrix $E=[{\xi }_i,{\xi }_{i+1},\dots ,{\xi }_m]$ consists of sparse vectors that are matched to their corresponding active terms in the candidate function library $\theta$.
  To find ${\xi }_i$, a sparse optimization is conducted to minimize the following function:

  $${\xi }_m=\underset{{\widehat{\xi }}_m}{argmin}\frac{1}{2}{∥{\dot{X}}_m-{\widehat{\xi }}_m\theta (X,U)∥}_2^2+\lambda {∥{\widehat{\xi }}_m∥}_1.$$

  (30)
  There are different methods of solving the optimization problem. They include least absolute shrinkage and selection operator (LASSO) [73], stepwise sparse regression (SSR) [74], sequential threshold least-squares (STLSQ) [7,75], and sparse relaxed regularized regression (SR3) [76,77].

In [78], the SINDy algorithm was used to model the discrepancies between the observed data and the simplified mathematical models of some physical systems. Specifically, the algorithm was used to discover the discrepancies induced by parameter inaccuracy or model insufficiency. The examples considered to demonstrate the algorithm’s effectiveness include the Van der Pol oscillator and a double inverted pendulum on an actuated cart.

3.8. Comparison of Different ML Methods

Table 2. Advantages, disadvantages, and domain application of ML models.

Model	Advantages	Disadvantages	Domain Application
FNN	Simple architecture and quick prediction after training.	Limited for complex patterns and not interpretable/explainable.	Regression, time series prediction, and classification.
1D CNN	Excellent for image/time series data and captures spatial relationships.	May require large datasets, computationally complex, and not interpretable.	Image classification, object detection, regression, and time series prediction.
LSTM	Captures sequential dependencies and suitable for time series.	Prone to vanishing/exploding gradients, complex architecture, and not interpretable.	Time series prediction/forecasting and natural language processing.
Transformer	Captures long-range dependencies and parallelizable computations.	Sensitive to sequence length, high computational demands, and not interpretable.	Machine translation and text generation, classification, and time series prediction.
SINDy	Clear and understandable representation of a system dynamics, suitable for sparse data, and fast to train.	Requires domain knowledge in creating a library of possible functions/terms that are sparsely considered in the model formulation.	Dynamic system modeling and identifying governing equations.
PINN	Incorporates system governing equations and model is interpretable.	Requires careful selection of the parameters upper and lower bounds, and computationally demanding.	Solving ODEs and PDEs, and dynamic system identification.

shows the pros, cons, and application domain of each of the ML models discussed in the previous subsections.

4. Identification of a Direct Current Geared Motor’s Model—Real Application

In this mechatronic study, we undertook an analysis focused on the dynamic model of a geared DC motor. Our methodology encompassed three distinct approaches: a physics-based approach, a purely data-driven approach utilizing various neural network models (FNN, CNN, LSTM, and SINDy), and a hybrid approach known as PINN.

The governing equations of the physical object under examination are derived from the mechanical and electrical components’ operational concepts, utilizing Newton’s second law and Kirchhoff’s laws, respectively [79,80]. These fundamental principles lay the foundation for the mathematical system model detailed in Section 4.1.

In parallel, the purely data-driven models were directly developed from experimental data. To achieve this, two datasets were acquired, comprising input and output data from a geared 12V DC Motor (SG555123000-10K). These datasets encompass the full operating range of the motor and were designated as the training and test datasets.

The data acquisition’s control structure is depicted by the block diagram in Figure 12. This diagram provides an insightful view of the experimental setup’s configuration.

An open-loop control experiment was conducted utilizing a time-varying PWM step signal as the input to the physical system, with the motor’s angular speed or rotational velocity as the measured output, which was obtained through an encoder.

To execute the experiment, the necessary hardware and specifications can be sourced from [81]. Subsequently, the collected data were employed to train four neural network models and a machine learning model, elaborated in Section 4.2 and Section 4.3. Furthermore, the same dataset and the system’s physical laws were leveraged to formulate a PINN model in Section 4.4. The training and testing csv files are accessible at [21].

4.1. Physics-Based Model of the Motor

A geared DC motor possessing uncertain frictional resistance forces can be classified as a nonlinear system, showcasing common nonlinear phenomena such as dead zones and backlash [82]. However, to facilitate a simplistic and approximate physical representation, the system is conventionally characterized as a second-order linear system. The system’s diagrammatic representation can be observed in Figure 13.

The equation of the electrical circuit shown on the left-hand side of the schematic is:

$$L\frac{di}{dt}+Ri=V-E,$$

(31)

where $E=K_b{\dot{\theta }}_{rs}$; V and i are the armature voltage [V] and current [A], respectively; L and R are the armature inductance [H] and resistance [$\mathsf{\Omega }$], respectively, and $K_b$ is the back EMF constant [V·s/rad]; and ${\theta }_{rs}$ is the angular position of the rotor shaft [rad].

The total motor torque $T_m$ [N·m] produced by the motor is proportional to i, i.e.,

$$T_m=K_{m}i=T_{rs}+\frac{1}{K_{gr}}{T}_{os},$$

(32)

where $T_{rs}=J_{rs}{\ddot{\theta }}_{rs}+B_{rs}{\dot{\theta }}_{rs}$ and $T_{os}=J_{os}{\ddot{\theta }}_{os}+B_{os}{\dot{\theta }}_{os}$ are the torques applied to the shaft of the rotor and gear [N·m]. Comparing Equations (31) and (32), we find:

$$J_{rs}{\ddot{\theta }}_{rs}+B_{rs}{\dot{\theta }}_{rs}+\frac{1}{K_{gr}}\left[J_{os}{\ddot{\theta }}_{os}+B_{os}{\dot{\theta }}_{os}\right]=K_{m}i,$$

(33)

where $J_{rs}$ and $J_{os}$ are the shaft inertia of the rotor and gear [kg·m²]; $B_{rs}$ and $B_{os}$ are the damping coefficient of the rotor and gear [N·s/m], respectively; $K_m$ is the motor torque constant [N·m/A]; $K_{gr}$ is the gear ratio [-]; and ${\theta }_{os}$ is the shaft position [rad].

Let ${\dot{\theta }}_{rs}={\psi }_{rs}$, ${\dot{\theta }}_{os}={\psi }_{os}$, $i=I$, and $V=U$, then Equations (31) and (33) can be rewritten as two first-order coupled differential equations of the form:

$$\frac{d{\psi }_{os}}{dt}=\frac{1}{J_{os}}\left(K_{gr}{K}_{m}I-K_{gr}[J_{rs}{\dot{\psi }}_{rs}+B_{rs}{\psi }_{rs}]-B_{os}{\psi }_{os}\right),\frac{dI}{dt}=\frac{1}{L}\left(U-k{\psi }_{rs}-RI\right).$$

(34)

When the physical parameters mentioned in Equations (34) can be determined, it becomes feasible to estimate the system’s dynamic response numerically. However, a challenge arises when dealing with the unknown parameters of the DC motor, requiring careful estimation. Therefore, the focus is now on investigating the capability of utilizing the PINN modeling paradigm to unveil the motor parameters. This means the creation of a physics-guided data-driven model is explored in Section 4.4.

4.2. The FNN, CNN, LSTM, and Transformer Networks Estimating Models of the Motor

Now, we investigate four neural networks: FNN, CNN, LSTM, and the transformer, which are used to estimate the purely data-driven models of the system. The models were trained differently using 20 epochs for each model and their structures are shown in

Table 3. Purely data-driven model structure with the number of neurons/filters in each model layer, including the model computation time and losses.

Model Name	Model Structure	Computation Time (s)	Training Loss	Validation Loss
FNN	Dense (100) + Dense (50) + Dense (10) + Dense (1)	148.68	20.204	30.033
CNN	Conv1D (128) + Conv1D (64) + Flatten + Dense (5) + Dense (1)	173.28	20.367	28.693
LSTM	LSTM (50) + Dense (50) + Dense (1)	217.67	20.146	27.580
Transformer	Transformer-encoder (2) + Dense (10) + Dense (1)	387.16	23.021	23.806

.

It is also evident that, despite the noise in the acquired angular speed data from the physical system, the time history in Figure 14 depicts a close match between the model predictions and the actual system output, that is, the angular speed of the geared DC motor.

In this study, we used the Adam optimizer with a learning rate of 0.001 for training each model. The computation times are shown in Table 2. The FNN model trained faster than the CNN, LSTM, and transformer models. The transformer network performed well in terms of validation loss but required more time to train due to its complex attention mechanisms and limited dataset size. The problem’s complexity contributed to the time-consuming training process. For reproducing the results, the numerical code is available in the specified repository [21].

All four models had similar prediction errors or validation loss, indicating only slight differences in their performance. Improving the models’ accuracy would involve exploring various network hyperparameters, such as the number of layers, neurons, optimizer type, learning rate, and training epochs. To enhance the transformer model’s performance, incorporating position encoding to capture the input data’s sequential nature explicitly would likely lead to significant improvements in predictions.

Remark 2.

The models were trained offline using TensorFlow deep learning framework on a computer equipped with an Intel Core i7 1.80 GHz CPU, 16-GB RAM, and a NVIDIA GeForce MX150 4 GB GPU.

4.3. SINDy Model of the Motor

SINDy, a machine learning method, was utilized to reveal the system’s dynamic equations following the approach detailed in Section 3.7. This process involved employing the pySINDy framework introduced in [83], which allows the customization of the library candidate function and the optimizer selection, along with the option to choose a differentiation method. For numerical differentiation, the finite difference method was used. The library candidate function for representing the sparse system incorporated the geared DC motor’s speed and voltage data, utilizing a third-order polynomial as the set order.

The discovered dynamical models of the system with SSR optimizer are as follows:

$${\dot{\psi }}_{os}=-89.8381-0.484{\psi }_{os}+4.598U+0.007{{\psi }_{os}}^2-0.036{\psi }_{os}U+0.010U^2,$$

(35)

with the LASSO optimizer:

$${\dot{\psi }}_{os}=-1.734{\psi }_{os}+3.174U+0.012{{\psi }_{os}}^2-0.014{\psi }_{os}U-0.007U^2,$$

(36)

and with the FROLS optimizer:

$${\dot{\psi }}_{os}=-85.4301-0.687{\psi }_{os}+4.800U+0.006{{\psi }_{os}}^2-0.029{\psi }_{os}U.$$

(37)

As shown in Figure 15, the simulation responses of the three models indicate that SINDy is effective in identifying a nonlinear system from experimental data.

The computation times for the SINDy models are shown in

Table 4. Computation times with different SINDy models.

Model	SSR	LASSO	FRLOS
Time (s)	0.2883	0.1207	0.2623

.

The discovered model suggests a library candidate function with a polynomial of order 2 being sufficient to represent the dynamics of the system. The numerical code that was used to generate the reported results is available at the repository [21].

4.4. PINN Model of the Motor

Deep FNNs have been used in several studies to implement the PINN modeling approach because they are computationally efficient [63,69,84,85]. Hence, this work examines the use of a three-layer-deep FNN with 32 neurons in each layer in implementing the PINN architecture. The PINN algorithm is described in Appendix A and it was used in solving both direct and inverse problems.

The initial steps of the algorithm involve importing necessary packages (numpy, pandas, matlplotlib, keras, and tensorflow), initializing the system parameters, and loading training and testing datasets. Next, three feedforward neural network models with three hidden layers and 32 hidden neurons each are created. These networks use $tanh$ as the activation function, have two inputs, and produce one output. The weights and biases of these neural networks are randomly initialized and then trained.

The three neural networks, denoted as ${\tilde{N}}_1$, ${\tilde{N}}_2$, and ${\tilde{N}}_3$, represent the approximated solutions ${\psi }_{rs}$, ${\psi }_{os}$, and I corresponding to Equation (34). During the training, the unknown physics parameters $J_{rs}$, $J_{os}$, $B_{rs}$, $B_{os}$, $K_m$, L, R, and $K_b$, along with the weights and biases of the networks, are initialized to enable the first prediction for each network error computation. This prediction is done in the forward pass, where three losses are computed with the approximated solutions ${\widehat{\Psi }}_{rs}$, ${\widehat{\Psi }}_{os}$, and $\widehat{I}$: loss 1 is based on the mean squared error (MSE) from the first Equation (34); loss 2 is based on the MSE from the second Equation (34); and loss 3 is based on the MSE of the output shaft speed.

The physical parameters, and the weights and biases of each network, are updated during the backward pass using a standard backpropagation algorithm. The derivative of the total error $L_{T1}$ with respect to the weights and biases of ${\tilde{N}}_1$ is computed using the sum of loss 1 and loss 3. Similarly, the derivative of the total error $L_{T2}$ with respect to the weights and biases of ${\tilde{N}}_2$ is computed using loss 2. The Adam optimizer is used with a learning rate of 0.001 and the algorithm is trained for 20,000 epochs.

After training, the identified physics parameters are displayed, and the weights and biases are saved for future use. Additionally, the network predictions are plotted using the test dataset.

As shown in Figure 16a,b, the PINN model successfully predicted the motor’s shaft speed and armature current even with limited training data containing noisy time-series of rotational velocity.

The system’s physical parameters, including the viscous friction coefficient, were estimated during training, and the resulting values are displayed in

Table 5. Estimated physics parameters after training the PINN model.

Estimated Geared DC Motor Parameters
${\mathit{J}}_{\mathit{rs}}$	${\mathit{J}}_{\mathit{os}}$	${\mathit{B}}_{\mathit{rs}}$	${\mathit{B}}_{\mathit{os}}$	${\mathit{K}}_{\mathit{m}}$	$\mathit{L}$	$\mathit{R}$	${\mathit{K}}_{\mathit{b}}$
0.001	0.0379	1.2539	0.3532	0.6585	0.001	1.3798	0.3938

. It is essential to acknowledge that the physical parameters obtained through the PINN algorithm are not unique. However, we constrained the search space for each parameter between 0.001 and 10 to determine their values.

The computation time for the PINN algorithm was 576.26 s, and the training loss was 6.57. For verification, the numerical code is available at the repository [21].

4.5. Discussion of Results

The performance of the developed models can be compared based on the prediction error or residual of each model as shown in Figure 17, while

Table 6. Performance of different models based on three metrics.

Model	MAE	MSE	RMSE
FNN	2.5338	30.0333	5.4803
CNN	2.5482	28.6930	5.3536
LSTM	2.4327	27.5797	5.2516
Transformer	1.9206	21.2892	4.6140
SINDy-SSR	1.9558	7.7613	2.7859
SINDy-LASSO	1.8789	7.4694	2.7330
SINDy-FRLOS	1.8789	7.4694	2.7330
PINN	2.4329	17.624	4.1981

shows the performance of the models with metrics like mean absolute error (MAE), MSE, and root mean squared error (RMSE).

The plot depicts high prediction errors of the PINN model at the first three steady speeds of the DC motor. However, all models exhibit the least prediction error when the motor operates at speeds greater than 200 rpm. This high error at low speed is attributed to the motor’s significant nonlinearities, such as friction and noise. Additionally, based on the performance metrics of the six (6) ML models in Table 6, the SINDy models (trained with different optimizers) gave the least errors, followed by the PINN and transformer models.

Despite this, all models, as shown in Figure 14, Figure 15 and Figure 16, provide predictions close to the actual time-series motor output. They demonstrate good generalization with the new dataset. Furthermore, the developed SINDy models are computationally efficient compared to other methods. Although the PINN model requires a longer training time, it performs fast predictions once trained.

Moreover, the four data-driven models (FNN, CNN, LSTM, and transformer) lack interpretability, operating as black-boxes. In contrast, the SINDy and PINN models are explainable. In the case of the SINDy model, the constants and chosen candidate functions post optimization are sparse and known, while for the PINN model, the physics-based parameters that guide its adherence to physical consistency during training are identifiable. Specifically, the PINN model leverages the governing laws of the DC motor, enabling the identification of physical parameters, such as the friction coefficient. Additionally, the PINN architecture facilitates the estimation of unmeasured system states, like the armature current, a capability absent in other models. Despite the highlighted benefits of using PINN for dynamic system modeling, its applicability may be constrained when the bounds of physical parameters and governing equations remain unknown.

4.6. Potential Applications of the Developed Dynamic Models

4.6.1. Control

A dynamic model plays a vital role in the model-based design of classic controllers. It also helps simulate the designed controllers to validate their effectiveness in achieving the desired objectives set by the designer. These objectives typically involve stabilization, reference tracking, and disturbance rejection during the development of controllers.

For example, when dealing with an inverted pendulum, control objectives include stabilizing the pendulum in an upright position and rejecting external disturbances like wind. On the other hand, for a DC motor, the objective is to achieve reference speed tracking even at the presence of nonsmooth effects like stick-slip friction. Commonly employed control techniques for dynamic systems encompass the PID [86], fractional order PID [87], LQR [88], fuzzy [89], sliding mode [90], and model predictive control [81].

Additionally, in mechanical systems where friction significantly impacts performance, friction models are developed and utilized for friction compensation. However, friction models are needed for systems developed through the hybrid approach and systems built to study the effects of friction. This suggests that pure data-driven models incorporate frictional effects in their internal structure.

4.6.2. Fault Detection

Dynamic models play a crucial role in identifying potential faults within a system. This involves creating models for the system under normal operating conditions and various fault scenarios [1]. By comparing the outputs of these models with the actual system’s responses, specific types of faults, such as actuator, sensor, or plant faults, can be detected [91]. An inference system is employed for error classification. The accuracy of the fault models and distinct error characterization associated with each fault greatly influence the effectiveness of the fault detection algorithm.

4.6.3. Digital Twin

A digital twin is an updated online model of a system, incorporating either a physics-based or data-driven model. It utilizes real-time data from the actual system for continuous updates [92]. Due to the extensive use of cloud computing and the Internet-of-Things, various industries employ digital twin technology for tasks like simulation, remote monitoring, control, predictive maintenance, and estimating the remaining useful life of physical assets or machines, thereby enhancing their reliability and safety [93].

5. Conclusions

In this review paper, various modeling approaches were presented and relevant concepts were briefly discussed. These concepts encompassed forward and inverse modeling problems, online and offline parameter identification, and dynamic modeling with friction. The shift from physics-based to data-driven and hybrid approaches was investigated, and it was attributed to the challenges faced by physics-based models when dealing with highly complex systems. Formulating models based on physical laws or first principles becomes difficult in such cases. On the other hand, data-driven models lack interpretability, impeding their adoption by industrial experts who prefer models grounded in physical principles rather than black-box models. The potential of hybrid or physics-informed models to address complex systems while incorporating some level of physics was emphasized, thus providing applicability.

A real experimental station was presented to develop a model for a geared direct current motor using various modeling techniques. Four data-driven models (FNN, CNN, LSTM, and transformer) were constructed, along with the use of SINDy (a machine learning technique) to infer the system’s governing equations. Additionally, a physics-based model was employed to create a PINN model, which successfully identified the system’s physics parameters, such as the viscous friction coefficient, and predicted the armature current (not directly measurable during experimentation). The neural network models and machine learning techniques produced satisfactory results, but the SINDy and PINN models were considered superior in terms of their interpretability.

Three key applications of dynamic models were discussed: control, fault detection, and digital twin. However, there are still various open research areas in the modeling of mechatronic systems that require further exploration.

The recommendations for future work include enhancing the integration of physical laws into neural networks, especially for multibody mechatronic systems with dry friction. Utilizing physics-guided networks like physics-guided transformers would result in the creation of models that are transparent and can be easily explained. It is also suggested to explore improved versions of SINDy, such as SINDy with sensitivity analysis, to create interpretable and generalizable models for mechatronic systems. This should involve accurate, low-dimensional, and sparse machine learning models, considering noisy data. Moreover, the scarcity of experimental studies on the application of PINNs and SINDy for parameter identification and the modeling of mechatronic systems was noted. Therefore, extending the use of these techniques to different systems with more complexities (such as industrial manipulators or induction motors) and comparing their performance is proposed.