Optimized-Based Fault-Tolerant Control of an Electro-Hydraulic System with Disturbance Rejection

: In this article, the design and implementation of a fault-tolerant controller are proposed for an electro-hydraulic actuator (EHA) in the presence of disturbances and actuator faults. The existence of nonlinearities, uncertainties, and a bias fault (i.e., internal leakage fault) in the system dynamics signiﬁcantly decreases the desired performance. The nonlinear disturbance observers (NDO) are constructed to handle the adverse inﬂuences caused by the above disadvantages. The whole fault-tolerant control (FTC) scheme consists of two design loops: an inner force control loop and an outer position control loop. The inner loop is based on an optimized backstepping framework to achieve the optimal performance, whilst the problem of uncertainties and disturbances is dealt with using a terminal sliding mode directly designed from the position tracking error. It is shown by theoretical analysis that system stability is ensured under faulty conditions. Finally, simulation results and comparison studies are conducted to further verify the effectiveness of the proposed approach.


Introduction
Over the past few decades, electro-hydraulic systems have been extensively used in various civilian and industrial applications due to many outstanding achievements [1][2][3][4]. There exist different research studies, and many results have been introduced to obtain the force or position control requirements, including model-based feedback control [5], slidingmode control [6], backstepping control [7], adaptive robust control [8], and so on. However, in the abovementioned works, many simultaneous problems of EHAs, such as uncertainties, disturbances, faults, and optimal performance, have not been well investigated. It is noted that the uncertainties include parametric uncertainties (i.e., parameter deviation, modeling errors, etc.) and uncertain nonlinearities (i.e., nonlinear friction force, and external load) [5][6][7][8]. Meanwhile, the most popular fault in the EHA is the internal leakage fault that comes from the piston seal and/or internal flow loss [9,10]. Such problems may degrade tracking performance or even cause instability of the whole EHA system if they are not tackled in a timely manner. Thus, the fault-tolerant capability is vital for guaranteeing the reliable control system and obtaining optimal performance of the EHA system.
To overcome the adverse effects of disturbance and uncertainty, the actuator fault issue, many effective FTC methods have been suggested to ensure system stability at an acceptable level [11]. Most notably, observers or an intelligent approximator-based FTC approach are widely adopted to react to faults, i.e., disturbance observer (DO) [12], state-dependent delay (SDD) [13], fixed-time observer [14], extended state observer [15,16], neural networks (NN) [17,18], fuzzy logic system (FLS) [19,20], and so on. It is noted that the effect of fault is significantly larger than lumped disturbance; many works merged the disturbance and bias fault into lumped disturbance. To deal with this issue, a designed compensator is then presented in Refs. [12][13][14][15][16][17][18][19][20]. In Ref. [21], by using the time delay estimation-based FTC control approach, the effects of disturbances and an internal leakage fault are compensated 1.
Compared with Refs. [34,35], this work firstly investigates the FTC problem for the EHA system to simultaneously solve the lumped disturbances, bias fault, so that the force control of the EHA system is optimized.

2.
To effectively attenuate the influences of the disturbances and the fault in the EHA system, the combination of DOs, a nonsingular terminal sliding-mode (NTSMC) scheme for the position tracking control loop, and an optimized force control method is developed. 3.
The system stability and asymptotic tracking error convergence is warranted by Lyapunov theory, and the effectiveness and feasibility of the suggested methodology are verified based on the simulation results.
The rest of this article is arranged as follows: In Section 2, the mathematical model with the actuator fault model of the EHA is described. The proposed cascade FTC comprises of the two loops (position and force controls) and two DOs, which is designed in Section 3. Section 4 presents the numerical simulation and comparison studies, respectively. Finally, the brief conclusions and future works are drawn in Section 5.

System Descriptions and Preliminaries
The schematic diagram of the EHA with actuator faults is shown in Figure 1. The internal leakage fault of the hydraulic driven actuator is studied.
The schematic diagram of the EHA with actuator faults is shown in Figure 1. The internal leakage fault of the hydraulic driven actuator is studied.
The dynamics of the actuator's moving part can be derived by Newton's second law where z is the piston's displacement; ( ) Δ • represents the disturbance terms, i.e., load's force, friction; is the load pressure, with 1 2 , P P being the pressure inside the extend and retract chambers; and m and b k describe the mass of the moving part and the equivalent viscous damping coefficient. We suppose that the valve's voltage input is related to the spool valve displacement by the directly proportional. Then, the load flow rate L Q is related to the control signal u by ( ) where Ps is the supply pressure, s k is a servo-valve's proportional gain, and sign(•) is the standard signum function. The fluid flow distribution into the two chambers of the cylinder is given by Refs. [8,36]: where , e A β are the Bulk modulus, a piston effective area, and l Q is the internal leakage. c V is a fixed control volume of the cylinder. Le q denotes the time-varying disturbances (i.e., external flow leakage, lumped modeling error).
Considering the system suffering from an internal leakage fault, the l Q can be represented as follows: where 0 C is the nominal coefficient, and the time-varying bias fault signal υ is leakage at the piston seal or internal flow loss. It can be formulated as follows in Refs. [37,38]  The dynamics of the actuator's moving part can be derived by Newton's second law where z is the piston's displacement; ∆(•) represents the disturbance terms, i.e., load's force, friction; P L = P 1 -P 2 is the load pressure, with P 1 , P 2 being the pressure inside the extend and retract chambers; and m and k b describe the mass of the moving part and the equivalent viscous damping coefficient. We suppose that the valve's voltage input is related to the spool valve displacement by the directly proportional. Then, the load flow rate Q L is related to the control signal u by where P s is the supply pressure, k s is a servo-valve's proportional gain, and sign(•) is the standard signum function. The fluid flow distribution into the two chambers of the cylinder is given by Refs. [8,36]: where β e , A are the Bulk modulus, a piston effective area, and Q l is the internal leakage. V c is a fixed control volume of the cylinder. q Le denotes the time-varying disturbances (i.e., external flow leakage, lumped modeling error).
Considering the system suffering from an internal leakage fault, the Q l can be represented as follows: where C 0 is the nominal coefficient, and the time-varying bias fault signal υ is leakage at the piston seal or internal flow loss. It can be formulated as follows in Refs. [37,38] where the time-varying internal leakage fault C f is considered with time profiles modeled by where T 0 , α, C f 0 denote, in turn, the unknown time that a fault occurs, the unknown fault evolution rate, and the leakage fault coefficient.
z, AP L ] T as the state variables, from (3)-(6), the system with actuator faults modeled can then be written as: . where

Assumption 1.
a. The tracking reference signal ζ 1d , its derivative . ζ 1d , and all states are bounded. b.
Under normal working conditions, the pressures P 1 , P 2 , and |P L | are sufficiently smaller than P s . c.

Remark 1.
For assumptions 1a and c: These are popular assumptions that have been used in previous works, for instance Refs. [1,7,38], to facilitate validating the stability problem of the closedloop system. For assumption 1b: The pressures P 1 , P 2 originate from P s , whilst P L = P 1 -P 2 also does not exceed P s . Therefore, Assumption 1 is reasonable and feasible.
where a 0 and b 0 are two positive constants, then it can be implied that L 1 (t) is bounded.

Remark 2.
The control objective is to synthesize a control signal u such that the output position ζ 1 tracks the desired trajectory ζ 1d to a bounded compact set, in the face of the lumped disturbances and actuator faults. Inspired by the combination of NTSMC, NDO, and optimal force control, the proposed FTC is developed to obtain the system stability and have an optimal performance for the nonlinear system (7).

Proposed Control Scheme
The control structure for the EHA system is illustrated in Figure 2. The cascade FTC control scheme comprises of outer-loop position tracking control and inner-loop force tracking control designs. By considering the lumped disturbances and the bias fault as lumped uncertainties, the two DOs are designed to handle the impacts of them. To obtain a high-accuracy tracking performance, a NTSMC scheme is elaborated for the position tracking control loop. Meanwhile, an optimal force controller is firstly constructed to guarantee the actual torque signals tracking the desired virtual torque signals. Optimization force control is implemented based on the RL strategy of actor-critic architecture. The suggested FTC strategy is proposed by combining the DOs, cascade control scheme including position control and optimal force control, which can not only minimize the selected cost functions to achieve the good tracking performance, but also ensure the boundness of all signal states in the face of the actuator fault and disturbances. tracking control loop. Meanwhile, an optimal force controller is firstly constructed to guarantee the actual torque signals tracking the desired virtual torque signals. Optimization force control is implemented based on the RL strategy of actor-critic architecture. The suggested FTC strategy is proposed by combining the DOs, cascade control scheme including position control and optimal force control, which can not only minimize the selected cost functions to achieve the good tracking performance, but also ensure the boundness of all signal states in the face of the actuator fault and disturbances.

Disturbance Observer-Based NTSMC Design
The nonlinear system (7) with lumped disturbances and the bias fault is first considered. Consequently, the position control design procedure for the suggested FTC is presented in this subsection.
The error states are defined as follows: where 1d ζ is the desired signal.
The derivative of e1 can be expressed as A terminal sliding manifold is given by 1 1 s me e λ η = +  (11) where , m η are positive terms, The time derivative of (11) is computed as

Disturbance Observer-Based NTSMC Design
The nonlinear system (7) with lumped disturbances and the bias fault is first considered. Consequently, the position control design procedure for the suggested FTC is presented in this subsection.
The error states are defined as follows: where ζ 1d is the desired signal. The derivative of e 1 can be expressed as A terminal sliding manifold is given by where m, η are positive terms, λ = λ 1 /λ 2 , λ 1 and λ 2 are positive odd integers satisfying The time derivative of (11) is computed as Let us denote the estimations of the disturbancesσ 1 ,σ 2 . The DO for mismatched disturbance σ 1 is elaborated aŝ where χ 1 is internal states, κ 1 denotes the observer gain of the mismatched DO. Define the mismatched disturbance error σ 1 = σ 1 −σ 1 . Taking the derivative of σ 1 , one yields .
The exponential approach rate is used. Hence, the desired force for the mechanical subsystem is constructed as follows: where γ 1 , γ 2 , k 1 are control gains and positive constants, and 0 < ϑ < 1. Let the torque error be determined as follows: The Lyapunov function candidate V 1 is adopted as Taking the derivative of V 1 and noting (12), (14), we obtain: .
Differentiating e 3 with respect to time yields that .
The matched DO is constructed to approximate both matched disturbance and the bias fault σ 2 asσ where χ 2 is internal states, κ 2 denotes the observer gain of the matched DO. The matched disturbance error is calculated by σ 2 = σ 2 −σ 2 . Taking the derivative of σ 2 , one obtains .
Remark 3. The singular problem in (15) can be overcome by using the modified terminal sliding manifold as Ref. [40]: where , η s is a positive scalar, and s = m . e 1 + ηe λ 1 .

Optimal Force Control Design
In this subsection, the final control law u is constructed to ensure the optimized solutions of inner-loop force tracking control designs. The system stability and acceptable performance of the whole closed-loop system can be preserved. The control design procedure is presented as follows: Let us introduce the following cost function as The HJB equation is elaborated as where J e 3 = ∂J ∂e 3 denotes the gradient of J. The optimal control u * can be calculated if the derivative of the HJB function with respect to u * obtains: The J * e 3 function can be presented as where J 0 , k 3 is a positive scalar. By using NN approximation to construct the critic network, an unknown continuous vector function J 0 e 3 can be estimated; it follows that: where φ * is the ideal weight of the NN, ς is the error of NN approximation, h(ζ 1 , ζ 2 , e 3 ) is the basis function. Throughout this paper, for the simplicity of expression, we abbreviate h(ζ 1 , ζ 2 , e 3 ) by h. In general, the most popularly used Gaussian radial basis functions are employed as follows: where ζ in = ζ 1 ζ 2 e 3 T is the NN inputs vector, and µ i , b i denote the center of the neural, the Gaussian function's width, respectively.
Substituting (27) and (28) to (26), one obtains Due to the unavailable ideal weight vector φ * , the optimal controller (30) cannot be implemented. Hence, the RL algorithm is introduced by utilizing the following both critic and actor NNs. The critic NN is used to evaluate the controlling performance, whilst the actor NN generates the final control law. The critic NN for evaluating the control performance is further derived aŝ whereφ c is the estimation of NN optimal weight φ * . The critic NN weight is updated by the following law as where β c is a positive constant. The final control law of the optimal force controller can be constructed as follows: whereφ a is the estimation of actor NN optimal weight φ * , and the adaptive law of the actor NN weight is determined as Ref. [34]: .φ where β a is a positive constant. Applyingû * andĴ * e 3 to (25), the approximation of the HJB equation yields Defining the residual error between the HJB and its approximation, we have and considering the optimized solutionû * is expected to satisfy, the following equality is derived: Under the premise that (37) is warranted, the RL updating laws are given by the establishment of a non-negative function, as follows: With the differentiation of (38) with respect to time, one obtains From (38), (39), it is concluded that Q = 0 can be obtained by using the updating laws (32) and (34), and then (37) can be satisfied as well.
Choose the overall Lyapunov function candidate for the system as where φ a =φ a − φ * , φ c =φ c − φ * are the NN weight errors. The time derivative of V is derived by Using Young's inequality, one obtains Applying Young's inequality and according to the equivalent transformation (i.e., φ T a hh T φ a + 2 φ T a hh T φ * + φ * hh T φ * =φ T a hh Tφ a ), one obtains Combined with (43) and (44), we obtain If the inequality 1 2 < β a < 1 2 β c , holds, there exist the upper bounds of the nonlinear function g 3 , f 3 , [25,35], and σ 2 ≤ d 2m , . σ 1 ≤ D 1m under Assumption 1. Thus, the time derivative of V can be given as (7), if the two DOs are designed as (13), (20), the position control law is presented in (15), the optimal force controller is indicated by (33). The adaptive law of the critic and actor NN weights is determined by (32) and (34). Then, all closed-loop signals, i.e., tracking errors, disturbance errors, and critic and actor NN weight errors are uniformly ultimately bounded (UUB) in the face of actuator failures and lumped disturbances.

Simulation Setup
To verify the improved performance of the proposed FTC, the simulation results are conducted in the EHA system by using MATLAB/Simulink. Two desired trajectories, i.e., the variable frequency sinusoidal (VFS) signals and a ramp signal, are described as ζ 1d = 10 + 10(sin(πt − π/2) + cos(0.5πt))(mm) The values of the EHA system parameters are given on the basis of data reported in Ref. [21], as listed in Table 1. For comparison, other control strategies are given to validate the advantages of the proposed FTC method in the simulation.
(1) DBC: This is the direct backstepping controller without two DOs. The control law is designed by Ref. [41]: where k 1 , k 2 , and k 3 represent the positive gain parameters.
(2) BSMCDO: This is the traditional backstepping sliding-mode controller with two DOs [42]. The control law is described by: where k i , i = 1, 2, 3 are the positive constant.
(3) Proposed controller: This strategy is a nonsingular terminal sliding-mode optimized backstepping control (NTSMOBC) with two DOs.
The following control gains are carefully regulated and tuned via the trial and error method and chosen as: k 1 = 50; γ 1 = 20; γ 2 = 1.5; λ 1 = 3; λ 2 = 5; η = 10. Meanwhile, the parameters of the optimal force controller are chosen as k 3 = 25; β a = 5.5; β c = 23. There exist two RBF NNs in the control EHA system. The critic NN part contains 65 nodes with a width equal to 0. 5  In fair comparison, the control parameters of DBC, BSMCDO controller are the same as the corresponding parameters in the proposed controller, e.g., same value of k 1 ; k 2 = γ 1 ; k 3 ; η; κ 1 ; κ 2 .
To assess the qualification of the proposed control algorithm in the face of the actuator fault, the various operational conditions are provided during 30 s. In the first condition, the system operates in normal condition with lumped disturbances. In the second condition, the internal leakage fault occurs after 10 s.

Evaluation Result
Two case studies were executed to appraise the effectiveness of the suggested FTC scheme. Case study 1 considered only the lumped uncertainty, while both the internal leakage fault and lumped uncertainty were tested in Case study 2. The simulation results are described as follows.
Case study 1: In this case study, the electro-hydraulic system operates in normal condition with inherent lumped disturbances. Figure 3 depicts the simulation results with a ramp and a sinewave response. From Figure 3a,b, the output position of three control algorithms exhibits good tracking capability. Nevertheless, the peak value of the position tracking error using the proposed controller and BSMCDO is smaller than that of usually applied DBC, as seen in Figure 3c,d, because the DOs effectively suppress lumped disturbances, which can be achieved by Figure 4. It can be known that the combination of NTSMC and the optimized-based BC scheme can not only obtain the purpose of optimization, but also guarantee the high-accuracy tracking performance in hydraulic servosystems; then, the suggested controller outperforms the BSMCDO controllers in terms of transient and final tracking errors. To ensure the ORC's optimal performance, the critic and actor NN weight estimationsφ c ,φ a of the ORC converge to the corresponding steady-state value, as shown in Figure 5. The effectiveness of comparative controllers is assessed through three performance indices and expressed in Table 2. From Table 2, the root mean square error (e RMS ), average tracking error (e a ), and maximum error (e max ) [7,43] of the proposed controller are, in turn, 0.0351, 0.0298, and 0.1954 mm with the desired sinewave signal, and 0.0278, 0.0201, and 0.0980 mm with the desired ramp signal, which is the smallest in comparison with other controllers. This reconfirms the perfect control performance of the proposed FTC strategy. In fair comparison, the control parameters of DBC, BSMCDO controller are the same as the corresponding parameters in the proposed controller, e.g., same value of k1; k2 = γ1; k3; η; κ1; κ2.
To assess the qualification of the proposed control algorithm in the face of the actuator fault, the various operational conditions are provided during 30 s. In the first condition, the system operates in normal condition with lumped disturbances. In the second condition, the internal leakage fault occurs after 10 s.

Evaluation Result
Two case studies were executed to appraise the effectiveness of the suggested FTC scheme. Case study 1 considered only the lumped uncertainty, while both the internal leakage fault and lumped uncertainty were tested in Case study 2. The simulation results are described as follows.
Case study 1: In this case study, the electro-hydraulic system operates in normal condition with inherent lumped disturbances. Figure 3 depicts the simulation results with a ramp and a sinewave response. From Figure 3a,b, the output position of three control algorithms exhibits good tracking capability. Nevertheless, the peak value of the position tracking error using the proposed controller and BSMCDO is smaller than that of usually applied DBC, as seen in Figure 3c,d, because the DOs effectively suppress lumped disturbances, which can be achieved by Figure 4. It can be known that the combination of NTSMC and the optimized-based BC scheme can not only obtain the purpose of optimization, but also guarantee the high-accuracy tracking performance in hydraulic servosystems; then, the suggested controller outperforms the BSMCDO controllers in terms of transient and final tracking errors. To ensure the ORC's optimal performance, the critic and actor NN weight estimations ˆ, c a φ φ of the ORC converge to the corresponding steady-state value, as shown in Figure 5. The effectiveness of comparative controllers is assessed through three performance indices and expressed in Table 2. From Table 2, the root mean square error (eRMS), average tracking error (ea), and maximum error (emax) [7,43] of the proposed controller are, in turn, 0.0351, 0.0298, and 0.1954 mm with the desired sinewave signal, and 0.0278, 0.0201, and 0.0980 mm with the desired ramp signal, which is the smallest in comparison with other controllers. This reconfirms the perfect control performance of the proposed FTC strategy.          Case study 2: In this scenario, the internal leakage fault in simulation is generated after T 0 = 10s with internal leakage coefficient C f 0 = 3.5 × 10 −8 m 4 / N 1/2 s , and the fault evolution rate α = 10. The simulation results are displayed in Figure 6, which indicates that the position errors of the DBC, BSMCDO, and the FTC proposed method are affected by the appearance of leakage fault. However, in comparison with DBC, the tracking error of the other methods is smaller due to a disturbance compensation system of two DOs. The accurate estimation of the matched, mismatched disturbances, and the bias fault using two Dos, are shown in Figure 7. Meanwhile, compared with the BSMCDO control method, the proposed controller combines the robustness properties of the NTSMC and the optimal performance of the optimized force controller. Therefore, the FTC proposed approach obtained the better performance and stronger robustness even though the bias fault appears. Figure 8 illustrates the norms of critic and actor NN weight estimations, respectively, and theφ c ,φ a converge eventually. Table 3 presents the results of three performance indices of three controllers. It indicates that the e RMS of the proposed controller is reduced by 73.9%, and 48.4%; the e a of the proposed controller is reduced by 79.8%, and 60.8%, in comparison with DBC, BSMCDO for VFS demand, respectively. The performance indices for ramp demand also reflect similar results in Table 3. This clearly validates the effectiveness of the proposed method.
two Dos, are shown in Figure 7. Meanwhile, compared with the BSMCDO control method, the proposed controller combines the robustness properties of the NTSMC and the optimal performance of the optimized force controller. Therefore, the FTC proposed approach obtained the better performance and stronger robustness even though the bias fault appears. Figure 8 illustrates the norms of critic and actor NN weight estimations, respectively, and the ˆ, c a φ φ converge eventually. Table 3 presents the results of three performance indices of three controllers. It indicates that the eRMS of the proposed controller is reduced by 73.9%, and 48.4%; the ea of the proposed controller is reduced by 79.8%, and 60.8%, in comparison with DBC, BSMCDO for VFS demand, respectively. The performance indices for ramp demand also reflect similar results in Table 3. This clearly validates the effectiveness of the proposed method.      According to the comparative simulation studies in Figures 3−8 above, it is verified that the proposed FTC can obtain the desired control goal. Moreover, it provides the excellent tracking performance in the face of the matched, mismatched disturbances, the internal leakage fault, among three controllers.

Conclusions
The optimal FTC strategy was successfully applied to the EHA system with lumped disturbances and internal leakage faults via the cascade control using NTMSC and a simplified RL algorithm based on the optimized technology. The lumped uncertainties including matched, mismatched disturbances and the internal leakage fault are catered for simultaneously. Two DOs were deployed for estimations of them. A novel controllerbased NTSMC, and an optimal force tracking control approach is further constructed for the EHA system, which can not only obtain the purpose of optimization, but also guarantee the high-accuracy tracking performance in hydraulic servosystems. The simulation  According to the comparative simulation studies in Figure 3 above, it is verified that the proposed FTC can obtain the desired control goal. Moreover, it provides the excellent tracking performance in the face of the matched, mismatched disturbances, the internal leakage fault, among three controllers.

Conclusions
The optimal FTC strategy was successfully applied to the EHA system with lumped disturbances and internal leakage faults via the cascade control using NTMSC and a simplified RL algorithm based on the optimized technology. The lumped uncertainties including matched, mismatched disturbances and the internal leakage fault are catered for simultaneously. Two DOs were deployed for estimations of them. A novel controller-based NTSMC, and an optimal force tracking control approach is further constructed for the EHA system, which can not only obtain the purpose of optimization, but also guarantee the high-accuracy tracking performance in hydraulic servosystems. The simulation results were implemented to verify the superior performance and system stability of the suggested strategy, even in faulty conditions. Nevertheless, the presented strategy still has some disadvantages. First, the loss of effectiveness fault was ignored in this work. Second, verifying the proposed algorithm in actual experiments has not yet been performed. In future works, we will tend to develop the FTC scheme for practical EHAs with some types of actuator faults, including bias fault and loss of effectiveness fault.
Author Contributions: K.K.A. was the supervisor providing funding and administrating the project, and he reviewed and edited the manuscript. V.D.P. carried out the investigation, methodology, analysis, validation, made the MATLAB simulation, and wrote the original manuscript. All authors have read and agreed to the published version of the manuscript.