A Neural Network Based Sliding Mode Control for Tracking Performance with Parameters Variation of a 3-DOF Manipulator

Featured Application: The proposed control scheme can be applied not only for position tracking control for industrial manipulators, but also for any non-linear systems to improve its performance. Abstract: The manipulator, in most cases, works in unstructured and changeable conditions. With large external variations, the demand for stability and robustness must be ensured. This paper proposes a neural network sliding mode control (NNSMC) to cope with uncertainties and improve the behavior of the robotic manipulator in the presence of an external disturbance. The proposed method is applied to the three degrees of freedom (DOF) manipulator. Some comparisons between the proposed and the conventional algorithms are given in both simulation and experiments to prove that the designed control can achieve higher accuracy in tracking motion.

Sliding mode control (SMC) is known as one of the most feasible solutions with fast response and strong robustness to resolve perturbation such as external disturbances, or unmodeled blocks in nonlinear systems compared with traditional methods [14,15]. When precise control is considered as an important criterion, the adaptive algorithm needs to be constructed rigorously. Ahmed et al. [16] formulated an adaptive fuzzy sliding mode control (AFSMC) using a supervisory fuzzy control where a fuzzy logic system (FLS) was employed to replace switching terms and switching gain in the robust control signal of the SMC. Jun et al. [17] integrated adaptive laws to determine the parameters of the FLS and then used FLS to approximate switching gains and eliminate the chattering. The FLS, whose laws were designed based on some knowledge about the control system [18], has been successfully applied to many linear and non-linear systems [19,20]. In such studies, the FLS with adaptive laws were employed to adjust controller gains of the SMC. The adjusted gains for the controller were chosen based M q q C q q q G q F q F q τ where 3 , , q q q R ∈   are position, angular velocity, and angular acceleration vectors of each joint, respectively, 3 3 ( ) M q R × ∈ is the symmetric and positive definite matrix of inertia, 3 3 ( , ) C q q R × ∈  denotes the Coriolis and centrifugal term matrix, 3 ( ) G q R ∈ is the gravity term, τ is torque acting on joints, is a diagonal matrix of viscous and dynamic friction coefficients, and 3 s F R ∈ is the vector of unstructured friction effects such as static friction terms. In practice, the dynamics of the manipulator is difficult to determine clearly due to the mass distribution, and oscillation in the process of operation. Then the equation (1) is represented as follows: M q q C q q q G q τ τ where ˆ( ) ( , ) ( ) M q q C q q q G q + +    represents the estimated terms of ( ) ( , ) ( ) M q q C q q q G q + +    , and can be assumed to be known, and is the vector of any generalized input due to uncertainties, disturbances or un-modeled terms.
Assumption. All (.) Δ are supposed to be bounded, i.e., (.) Unlike a conventional manipulator, the last link of our proposed hydraulic manipulator employs a cylinder to load heavier objects. The torque acting on the joint is essentially induced by force generated by the cylinder; then the relationship between the rotational motion of the joint and the translational motion of the actuator will be investigated. The architecture of the last link is illustrated in Figure 2. The dynamics of the 3-DOF can be expressed as: q ∈ R 3 are position, angular velocity, and angular acceleration vectors of each joint, respectively, M(q) ∈ R 3×3 is the symmetric and positive definite matrix of inertia, C(q, . q) ∈ R 3×3 denotes the Coriolis and centrifugal term matrix, G(q) ∈ R 3 is the gravity term, τ is torque acting on joints, F d ∈ R 3×3 is a diagonal matrix of viscous and dynamic friction coefficients, and F s ∈ R 3 is the vector of unstructured friction effects such as static friction terms.
In practice, the dynamics of the manipulator is difficult to determine clearly due to the mass distribution, and oscillation in the process of operation. Then the Equation (1) is represented as follows: . q +Ĝ(q) represents the estimated terms of M(q) ..
. q + G(q), and can be assumed to be known, and τ d = ∆M(q) q ∈ R 3 is the vector of any generalized input due to uncertainties, disturbances or un-modeled terms.
Unlike a conventional manipulator, the last link of our proposed hydraulic manipulator employs a cylinder to load heavier objects. The torque acting on the joint is essentially induced by force generated by the cylinder; then the relationship between the rotational motion of the joint and the translational motion of the actuator will be investigated. The architecture of the last link is illustrated in Figure 2. The correlation between the motion of the actuator in the actuator space and the motion in joint space can be derived by: where 0 d is an initial length in case of maximum retracting; the positive direction in the actuator space is chosen for extracting the piston. To obtain the correlational velocity between the actuator and the joint motion, taking the derivation respect to time, the equation (3) is: 2  3  3  3  3  3  3  3  3  3  1  2  1 2  3 sin cos where Ja3 (q3) is a Jacobian matrix of the 3 rd joint. Thus, a torque acting on this joint can be derived as: The completely correlational motion between the actuator and the joint motion is described by: where ( ) denotes a position vector of the actuator in actuator space.

Sliding Mode Control
Define error variables such as: where 3 e R ∈ is the error between the desired position and a simulated position, 3 , , are the vector of the desired position, desired angular velocity, and desired angular acceleration of the joint, respectively, and and e e  are supposed to be bounded. Then the sliding surface s(t) in the statespace 3 R can be derived as: The correlation between the motion of the actuator in the actuator space and the motion in joint space can be derived by: where d 0 is an initial length in case of maximum retracting; the positive direction in the actuator space is chosen for extracting the piston. d 3 denotes the position of the end-effector under the motion of the cylinder, d 1 and d 2 are fixed length of the hinge joints.
To obtain the correlational velocity between the actuator and the joint motion, taking the derivation respect to time, the Equation (3) is: where J a3 (q3) is a Jacobian matrix of the 3 rd joint. Thus, a torque acting on this joint can be derived as: The completely correlational motion between the actuator and the joint motion is described by: where q 1a q 2a d 3 T ∈ R 3 denotes a position vector of the actuator in actuator space.

Sliding Mode Control
Define error variables such as: where e ∈ R 3 is the error between the desired position and a simulated position, q d , .. q d ∈ R 3 are the vector of the desired position, desired angular velocity, and desired angular acceleration of the joint, respectively, and e and . e are supposed to be bounded. Then the sliding surface s(t) in the state-space R 3 can be derived as: where λ = diag λ 1 λ 2 λ 3 , λ i (i = 1, 2, 3) is a positive constant.
In this circumstance, we choose n = 2, i.e., s = . e + λe is a function of the tracking position error and the velocity error based on the manipulator dynamics.
The derivative of the sliding surface with respect to time is derived as: Property.
q) x = 0 for any vector x ∈ R n .
Then, multiplying both sides of the equation above with M(θ), we obtain: ..
The Lyapunov function can be defined as: Taking the derivative, the Equation (11) with respect to time, we obtain: Let the equivalent state e [18]; then substituting into Equation (8), we obtain: Then the control signal function is designated as: . q s +Ĝ(q) +M(q) ..
where α = diag α 1 α 2 α 3 is a diagonal positive definite matrix, α i is positive constant, tanh(s) = tanh(s 1 ) tanh(s 2 ) tanh(s 3 ) T , and K = diag(K 1 , K 2 , K 3 ) is a positive matrix in which s T Ks > 0 ∀ s 0. The term -Ks is added to force the state to approach the switching manifolds faster and the tracking error will be converged in a finite time.
The stability of the system can be easily proved if the gains of the switching controller are bigger than the upper bounded uncertainties. Then substituting Equation (14) to (13), one can obtain: Since we supposed that all ∆(.) are bounded, i.e., d ∞ ≤ d bound , then to guarantee the stability of the system, the derivative of the Lyapunov function must be negative: where x) and f 1 and f 2 are positive constant matrices.

Neural Network Control
According to the portrait phase of the sliding mode control presented in Yuri et al. [32] and Wilfrid [33], the first stage is when the state trajectory is outside the sliding surface and needs to be driven towards the sliding surface. Then the second phase occurs when the state trajectory moves along towards the sliding surface. The term −αtanh(s) − Ks in Equation (14) is to drive the state trajectory toward the sliding surface, whereas the slope λ of the sliding surface affects the convergence of state variables. The two-terms λ and robust term −αtanh(s) − Ks, in fact, depend on each other. The correlation between the two terms is depicted in Figure 3.
Despite the robustness of the sliding mode control, the tracking performance is difficult to achieve with high precision with fixed robust gains in the presence of the external payload. Some articles have reported the algorithm for gains-tuning using the neural network method. In this part, the neural network based sliding mode control is designed to adapt with the parameter variation. The NNSMC architecture is described in Figure 4.
As can be seen in Figure 4, instead of setting the fixed value for both the slope of the sliding surface and controller gains, in the NNSMC, the slope of the sliding surface and controller gains are considered as a weighting factor to be adjusted. Tracking error e and its change rateė are considered as two nodal inputs of the neural network (NN). The first weights to be tuned are the gains for each e andė that specify the slope of the sliding surface. The hidden layer is presented by using three nodes determined from the sliding surface that is an output of the input layer. Then the controller gains of the SMC are considered as the second weights to be tuned to adapt with the system dynamics. The output of the NN is a control signal.
For simplicity, let e i and . e i be an element of e and . e (where i = 1,2,3). The input of the NN is the error and the rate of the error: The output of the first layer is: where λ 1 and λ 2 are gains corresponding to e i and . e i . Then, the sliding surface is modified as: where f i1 (y i ) is an activate function of the element ith.  Figure 3.
Despite the robustness of the sliding mode control, the tracking performance is difficult to achieve with high precision with fixed robust gains in the presence of the external payload. Some articles have reported the algorithm for gains-tuning using the neural network method. In this part, the neural network based sliding mode control is designed to adapt with the parameter variation. The NNSMC architecture is described in Figure 4.
As can be seen in Figure 4, instead of setting the fixed value for both the slope of the sliding surface and controller gains, in the NNSMC, the slope of the sliding surface and controller gains are considered as a weighting factor to be adjusted. Tracking error e and its change rate ė are considered as two nodal inputs of the neural network (NN). The first weights to be tuned are the gains for each e and ė that specify the slope of the sliding surface. The hidden layer is presented by using three nodes determined from the sliding surface that is an output of the input layer. Then the controller gains of the SMC are considered as the second weights to be tuned to adapt with the system dynamics. The output of the NN is a control signal.
For simplicity, let and The output of the first layer is: Then, the sliding surface is modified as: where ( )   The input of the hidden layer is defined by: , and Kio is an offset signal to compensate the control input signal.
The output of the hidden layer and the control input signal, respectively, are: The weighting vectors are automatically updated to minimize the cost function: where ( ) and ( ) id i q t q t represent the desired and actual trajectory, respectively, of the i th element.
Let us define the cost function constant εei, εsi. When the cost function is greater than εsi, the adjusted gains for driving the state trajectory towards the sliding surface are updated, and when the error is greater than εei, the slope of the sliding surface λ is adjusted based on the chain-rules: The change of each gain is computed using gradient descent: The input of the hidden layer is defined by: where s i1 = s i , s i2 = tanh(s i ), and K io is an offset signal to compensate the control input signal. The output of the hidden layer and the control input signal, respectively, are: The weighting vectors are automatically updated to minimize the cost function: where q id (t) and q i (t) represent the desired and actual trajectory, respectively, of the ith element. Let us define the cost function constant ε ei , ε si . When the cost function is greater than ε si , the adjusted gains for driving the state trajectory towards the sliding surface are updated, and when the error is greater than ε ei , the slope of the sliding surface λ is adjusted based on the chain-rules: where η k im (m = 1,2,3) and η λ in (n = 1,2) are the learning rate of each gain and factors of the sliding surface. Both η k im and η λ in are positive constants. The change of each gain is computed using gradient descent: After updating the controller gains, the slope of the sliding surface is then tuned due to the back-propagation process to satisfy the performance. The change of the slope is computed as: Hence:

The Constraint of the Updating Law
According to Equation (19), the sliding surface is: When the state trajectory moves towards the sliding surface: where is an equivalent slope of the sliding surface. Taking derivative Equation (41) respects to time, we obtain: Hence, to reduce the convergence time, the updating law for the slope of the sliding surface is then adjusted such that the equivalent slope increases and achieves optimal value:

Simulation
Remark 1. The control signals (voltage) generated from the controller designed are proportional to torques applied on each joint of the manipulator regarding the characteristics of the drivers.

Remark 2.
The initial parameters are chosen based on trial and error such that the highest performance can be achieved. In this study, the learning rate η k ij for updating each parameter is designed based on experiments.

Remark 3.
In the simulations, the initial values of the NNSMC are set to be the same as the conventional SMC.
In this section, some simulations were conducted on the MATLAB-Simulink for the 3-DOF manipulator presented in Figure 1. The simulations were carried out with two case studies and comparisons between the proposed algorithm with the conventional controller are discussed to verify the effectiveness in tracking performance. The system parameters are described in Table 1. The estimated inertia, Coriolis, and gravity matrix describing the manipulator dynamics are shown in the Appendix A. The payload profile is described in Figure 5. We considered the payload change to verify the effectiveness of the proposed algorithm.
The tracking performances of the three joints are depicted in Figure 6, Figure 7, and Figure 8, respectively. The updating of each controller gains of the proposed NNSMC corresponding to three joints are expressed in Figure 9, and the slope of the sliding surfaces is depicted in Figure 10. In this section, some simulations were conducted on the MATLAB-Simulink for the 3-DOF manipulator presented in Figure 1. The simulations were carried out with two case studies and comparisons between the proposed algorithm with the conventional controller are discussed to verify the effectiveness in tracking performance. The system parameters are described in Table 1. The estimated inertia, Coriolis, and gravity matrix describing the manipulator dynamics are shown in the Appendix. The payload profile is described in Figure 5. We considered the payload change to verify the effectiveness of the proposed algorithm.
The initial parameters of the three controllers are shown in The tracking performances of the three joints are depicted in Figure 6, Figure 7, and Figure 8, respectively. The updating of each controller gains of the proposed NNSMC corresponding to three joints are expressed in Figure 9, and the slope of the sliding surfaces is depicted in Figure 10.   As can be seen in Figures 6-8, with respect to the response of the three joints, when the external payload increased suddenly at the 20 th second time period, the tracking performance did not change so much, its tracking error when applying the proposed NNSMC was the smallest in comparison with that of PD and conventional SMC, and the stability of the system was maintained. The behavior of the system was adapted by the changing of the controller feedback gains shown in Figures 9 and 10. At the beginning, the initial was set as the same value as the conventional SMC (for the robust gains K and α) and the same as PD (for the ratio between error and the change of error). Regarding the updating law, the controller gains were adjusted in order to achieve a suitable value for the working conditions, thus reducing the tracking error with external payload impacts. The controller gains are described in Figure 9, whereas Figure 10 shows the adjusted slope of the sliding surfaces. Due to the errors in the second and third joints satisfying the constraints, the robust gains of these joints did not change much even when the external payload was attached at the time 20 th second However, due to the structure, the mass inertia of the first joint increased significantly, thus resulting in the time for the first joint to adapt to the change being longer. As opposed to previous works, where only the term −αtanh(s) − Ks was adjusted to adapt with the working operation, in this study, we considered the slope of the sliding surface as an important parameter that needed to be tuned to minimize the state error. Therefore, the slope of the NNSMC was adjusted depending on the error between the desired and the simulated trajectory to determine the suitable value. When the state trajectory was driven and moved along the sliding surface, the behavior of λ e and λ de only depended on the state error and its change rate . e. Consequently, the value of λ e and λ de converged and remained at the value in which the steady state was achieved.

Experimental Setup
The hardware architecture included a computer (PC) for data processing, a hydraulic supply for the manipulator, and a 3-DOF hydraulic manipulator communicating with the PC via terminal 68 LPR and a PCIE Card (PCIE-6363). The hydraulic manipulator supplied by the hydraulic system was driven by electro-hydraulic actuators equipped with incremental rotary encoders E40H6-5000-3-V-5 to measure the movement. The data read from encoders (incremental) was then processed by the MATLAB program via PCIE card. After processing, the command signal from the computer was sent to the system to force the manipulator to follow the desired trajectory via servo valve amplifier SVA-R-1. The hardware connection is depicted in Figure 11. The dash lines stand for the data obtained from the system, and the solid lines stand for the command signal from the computer. The comparison in tracking performance and its error of the proposed algorithm with the conventional controllers are depicted in Figure 12 and Figure 13, respectively. The frequency was set at 0.2 Hz to verify the workability of the proposed algorithm. As can be seen in Figure 13, as with the simulation results, the tracking error from the NNSMC was the smallest. However, the slope of the sliding surface of each element reached larger values compared with the simulations due to existing modeling errors inside the actuators. In addition, as opposed to the conventional SMC, the control signals of the NNSMC were modified by adding an offset term Ko to compensate the variations or uncertainties of the system. However, this parameter could not be set to a large value and its learning rate had to be kept at a small value because the large value could affect and degrade the performance of the system.

Remark 4.
In our experiments, the learning rate of each element was set up at a constant value. This value was determined by a trial and error method. Besides, due to the modeling error, the learning rates were scaled in comparison with the simulation.

Experimental Results
In the experiments, the 3-DOF hydraulic manipulator was used to verify the proposed algorithm. The system parameters were supposed to be well known and are presented in Table 1. The desired trajectory is described by the dot-line. The dash-dot line, the solid line, and the dash-line represent the tracking performances of the PD controller, SMC controller, and the NNSMC controller, respectively. The parameters of the PD and SMC controller were manually set and fine-tuned such that the best performance could be achieved, whereas the parameters of the NNSMC are initialized with small values to verify their adaptability. The desired trajectory was designed as: (degree) q 2d = 20 + 20 sin( 2πt /5 + π /2) (degree) q 3d = 40 + 20 sin( 2πt /5) (degree) .
The comparison in tracking performance and its error of the proposed algorithm with the conventional controllers are depicted in Figures 12 and 13, respectively. The frequency was set at 0.2 Hz to verify the workability of the proposed algorithm. As can be seen in Figure 13, as with the simulation results, the tracking error from the NNSMC was the smallest. However, the slope of the sliding surface of each element reached larger values compared with the simulations due to existing modeling errors inside the actuators. In addition, as opposed to the conventional SMC, the control signals of the NNSMC were modified by adding an offset term K o to compensate the variations or uncertainties of the system. However, this parameter could not be set to a large value and its learning rate had to be kept at a small value because the large value could affect and degrade the performance of the system.
Compared to the simulation results, the robust gains depicted in Figure 14 changed very quickly at the beginning and tended to be a large value due to the uncertainties. The slope of the sliding surface was changed by tuning the weighting of the two variables e and . e. The first value of the state trajectory could be anywhere depending on the initial value of the system. Then adjusting the slope of the sliding surface at the same time with tuning robust term could drive the system error convergence to zero (ideal case) or a boundary quickly. The adaptive slopes of the three joints are presented in Figure 15. The profiles of the adjusted parameters in the experiment result were quite similar to the simulation due to uncertainties or disturbance existing in the real mechanical structure.     The comparison in tracking performance and its error of the proposed algorithm with the conventional controllers are depicted in Figure 12 and Figure 13, respectively. The frequency was set at 0.2 Hz to verify the workability of the proposed algorithm. As can be seen in Figure 13, as with the simulation results, the tracking error from the NNSMC was the smallest. However, the slope of the sliding surface of each element reached larger values compared with the simulations due to existing modeling errors inside the actuators. In addition, as opposed to the conventional SMC, the control signals of the NNSMC were modified by adding an offset term Ko to compensate the variations or uncertainties of the system. However, this parameter could not be set to a large value and its learning rate had to be kept at a small value because the large value could affect and degrade the performance of the system.

Conclusions
This paper presented the neural network based sliding mode control for tracking control of the 3-DOF manipulator with the presence of uncertainties and external payload variation. The structure of the neural network sliding mode control as well as the correlation between the slope of the sliding surface and robust term in the control signal are derived. The conventional sliding mode controller (SMC) is described by nonlinear terms, the sliding surface and the robust term with gain K, and α via the term "-Ks-αsign(s)" to generate the tracking performance. As mentioned in the introduction and in Section 3.2, despite its strong robustness, fixed gains cannot satisfy the tracking performance with the variant uncertainties or external disturbances. Previous works proposed many control schemes for the SMC with adaptive gains by using intelligent techniques without considering the slope of the sliding surface that affects the convergence rate to zero of the system errors. Besides, suitable gains are not discussed clearly. Based on a missing in constructing an advanced algorithm on constructing an advanced algorithm, this paper employed the neural network with gradient descent method for adjusting the robust term and the slope of the sliding surface (same function as nonlinear PD control) to a suitable value such that the stability is guaranteed, and higher tracking performance can be achieved. The proposed controller is tested on the 3 DOF hydraulic manipulator to validate its effectiveness and workability. Comparisons of tracking performance between the proposed algorithm with the conventional sliding mode and traditional PD control are given in both simulation and experiments under the presence of the uncertainties and the external payload. The results indicate that the proposed algorithm can exhibit high precision in tracking performance.
However, in this study, the learning rate parameters are set manually based on several experiments as mentioned in the above remarks. These parameters should be automatically adopted with system behavior. Besides, we ignored use of the intelligent technique as compensation like other convention adaptive sliding mode controller schemes. Therefore, the neural network SMC proposed in this study is a premise to develop an adaptive learning rate and can be considered as the first step to get a suitable value for a novel adaptive control with intelligent techniques (radial basis function or adaptive control) in future works.

Conflicts of Interest:
The author declares no conflict of interest.

Appendix A
According to the manipulator structure in Figure 1, following the inward and outward iteration method, the manipulator dynamics is derived by Equation (2) with: