Impedance Learning-Based Hybrid Adaptive Control of Upper Limb Rehabilitation Robots

: This paper presents a hybrid adaptive control strategy for upper limb rehabilitation robots using impedance learning. The hybrid adaptation consists of a differential updating mechanism for the estimation of robotic modeling uncertainties and periodic adaptations for the online learning of time-varying impedance. The proposed hybrid adaptive controller guarantees asymptotical control stability and achieves variable impedance regulation for robots without interaction force measurements. According to Lyapunov’s theory, we proved that the proposed impedance learning controller guarantees the convergence of tracking errors and ensures the boundedness of the estimation errors of robotic uncertainties and impedance profiles. Simulations and experiments conducted on a parallel robot validated the effectiveness and the superiority of the proposed impedance learning controller in robot-assisted rehabilitation. The proposed hybrid adaptive control has potential applications in rehabilitation, exoskeletons, and some other repetitive interactive tasks.


Introduction
Comfort and safety are of significant value in robot-assisted rehabilitation and can be improved through robot impedance regulation.As a well-known compliance control approach, impedance control has shown its powerful abilities in robot impedance regulation and has been extensively applied in service robots and industrial robots [1][2][3][4].However, robot modeling uncertainties and human impedance variation bring in difficulties for impedance control implementation in rehabilitation training.
Robot modeling uncertainties affect the control stability and the control robustness of impedance control.Up to now, considerable research attention has been paid to robust impedance control [5], adaptive impedance control [6,7], neural impedance control [8][9][10][11], and iterative learning impedance control [12].These impedance control methods improve the stability and robustness of closed-loop control systems; however, these methods mainly focus on constant impedance regulation.
In robot-assisted rehabilitation, human impedance varies along with the variation in interaction forces and tasks.Time-varying impedance control [13] has better robustness and compliance in rehabilitation training through robotic variable impedance regulation, in comparison with constant impedance control [14][15][16].Although categories of variable impedance control methods were designed for robot-assisted rehabilitation [17], many of these results neglected the stability of the desired variable impedance dynamics (DVID) which is of significant value for human-robot cooperative motion.Actually, the DVID are described by linear time-varying systems and may be violated by the variation in impedance profiles.In [18], the stability region of impedance parameters was evaluated experimentally for the DVID with zero stiffness.In [19], an adaptive variable impedance controller was designed for force tracking in interaction with an unknown environment with constant impedance.In [20][21][22], novel constraints on time-varying impedance profiles were proposed to guarantee the stability of the DVID, but how to design impedance profiles in robot-assisted rehabilitation was not given in these results.
Designing appropriate impedance profiles to balance between the desired control stability and the desirable interaction performance is of significant value for robot-assisted rehabilitation.For impedance profile construction, the algorithms of Learning from Demonstration and sEMG-based Dynamic Movement Primitives (DMPs) were applied for timevarying impedance control in physical HRI.In [23][24][25], two-loop-based optimal admittance controllers were proposed for robot-assisted rehabilitation.An inner-control loop makes robots behave in a prescribed impedance model, and an outer-loop optimal control aims at finding the optimal parameters of the prescribed impedance model by minimizing a performance function of interaction forces and tracking errors.Almost all impedance/admittance controllers mentioned above were designed using interactive forces; however, the force measurements may become impossible without related force sensors or ineffective due to measurement delay or noises.
From the above analysis, the variable impedance regulation of robots can improve human-robot interactive performances, but impedance variation brings in control stability problems.Recently, model-based impedance learning controllers were proposed in [26][27][28] for human-robot interaction with a control stability guarantee and without the requirement of interaction forces.Specifically, human impedance uncertainties were estimated by using the periodicity characteristics of task reference trajectory and human impedance profiles in [26,27], and an adaptive impedance learning controller was proposed in [28], where robot modeling uncertainties and human impedance uncertainties were estimated by using traditional differential updating laws.However, the results in [26][27][28] have the following deficiencies: (i) these adaptive impedance learning controllers only guarantee the closedloop control systems be uniformly ultimately bounded (UUB) but not be asymptotically stable; (ii) precise knowledge about robotic dynamic parameters is required in [26,27]; and (iii) the result in [28] requires the unknown impedance profiles be slowly varying due to the weakness of differential adaptation in estimating time-varying signals.
In this paper, we propose an impedance learning-based hybrid adaptive control strategy with potential applications in repetitive rehabilitation training.In comparison with the related literature, the innovations and contributions of this paper are stated as follows: (1) A differential updating mechanism is designed for estimating robotic parameter uncertainties, while periodic adaptive learning laws are designed for the learning of human time-varying impedance profiles.
(2) The projection function used in the adaptive law and the saturation functions in the periodic adaptive laws avoid the possible drift of estimators.
(3) The proposed hybrid adaptive robot controller guarantees asymptotical control stability and achieves variable impedance regulation for rehabilitation robots without interaction force measurements.
(4) The asymptotical convergence of tracking errors and the finiteness of the estimation errors of the robot parameters and the human impedance profiles are validated through a Lyapunov-like analysis.
Compared with the constant impedance control, the proposed impedance learningbased controller can provide time-varying impedance regulation for rehabilitation robots.In comparison with the impedance learning controllers in [13,[18][19][20][21][22], the proposed modelbased impedance learning controller can guarantee the control stability without using interaction force signals.Compared with the model-based impedance learning controllers in [26][27][28], the proposed controller guarantees the asymptotical convergence of tracking errors.In addition, robotic parameter uncertainties are considered and estimated in this paper in comparison with [26,27], and the impedance profiles are not required to be slowly varying in comparison with [28].A summary of currently used adaptive controllers with impedance regulation and the gap where this paper is located are listed in Table 1.
Table 1.A summary of the currently used adaptive controllers with impedance regulation in rehabilitation.

Index
Currently Used Methods

Innovations or Contributions of This Paper 1
The constant impedance control [14][15][16] Time-varying impedance regulation for rehabilitation robots are provided

2
The impedance learning controllers [13,[18][19][20][21][22] The control stability without using interaction force signals is guaranteed The model-based impedance learning controllers [26][27][28] (a) The asymptotical convergence of tracking errors is guaranteed (b) Robotic parameter uncertainties are considered and estimated (c) The impedance profiles are not required to be slowly varying

Problem Formulation
Consider the robot dynamics with the following form
where q ∈ R n denotes the vector of joint angles, M(q) the inertial matrix, C q, .q the Coriolis and centrifugal matrix, G(q) the gravity torque, and F .q the viscous force with F being a constant matrix.In (1), τh and τ are the interaction force vector in joints and the system control input, respectively.
Denote x as the end-effector Cartesian position.The robot has the following kinematic transformations .
x = J(q) . q. (3) In (3), J(q) = d∆(q)/dq is the Jacobian matrix from the joint space to the Cartesian coordinates.Based on (2) and (3), one can convert the joint space dynamics in (1) into the following Cartesian dynamics where Property 1. M x (x) is a symmetric positive definite matrix and satisfies where σ 1 , σ 2 > 0, and σ 1 and σ 2 can be regarded as the minimum and maximum eigenvalues of M x (x), respectively.Property 2. .
x is skew symmetric, i.e., Property 3. The left part of (4) can be parameterized in the following form x θ (7) where x is a regression matrix, and θ is a constant vector that contains unknown parameters and satisfies ||θ||≤ θ c .
x , G x (x), and their partial derivatives with respect to their respective arguments are bounded.If x , and its partial derivatives with respect to its arguments are bounded.Assumption 1. Suppose the desired trajectory x r and its first and second time derivatives are bounded for the considered robot in the HRI.
Define the trajectory tracking error e as e = x r − x As proven in [26], the interaction force can be expanded as where f f (t) denotes the feedforward force of the human, and K h (t) and B h (t) denote the stiffness term and the damping term of the interaction force, respectively.The interaction force f h can be further expressed as where .
x r denotes a modified feedforward force.In repetitive rehabilitation training, the impedance profiles f f (t), K h (t), and B h (t) can be assumed to be periodic with T, i.e., It is assumed that f f (t), K h (t) and B h (t) are continuous functions and satisfy where f * , K * , and B * are constant matrices; '≤' in the last equation means each element of the left part of '≤' is no more than the related element of the right part of '≤'.
Remark 1.In [25], the feedforward force, the stiffness term, and the damping term are assumed to be slowly varying and are estimated by using traditional adaptive laws.In this paper, f f (t), K h (t), and B h (t) are only required to be continuous and bounded functions that will be estimated in Section 3 using periodic adaptation.

Hybrid Adaptive Control via Impedance Learning
Define the auxiliary error ϵ as ϵ = .
e + k 1 e where k 1 > 0. From (4), ϵ has the following dynamics It is convenient to design the following control law e, .
x θ − fh (16) where θ and fh are estimators of θ and f h , respectively; fh is designed as with fh , Kh , and Bh being estimators of f f , K h , and B h , respectively.Substituting ( 11) and ( 12) into (10) yields where

and
In (15), θ is a constant vector that can be estimated by the sing traditional differential updating mechanism, while the impedance profiles f f (t), K h (t), and B h (t) are time- varying signals that may not be slowly varying and cannot be effectively estimated using traditional adaptive laws.By exploiting their periodicity characteristics, the impedance profiles can be estimated by using periodic adaptations.Therefore, we propose a hybrid adaptive law for estimating θ and the impedance profiles.
For θ, we design the following adaptive update law e, .
x ϵ (19) where α 0 is the learning rate, and proj θ (•) is a projection function defined as with θ c being defined in Property 3.
To estimate the time-varying terms f f (t), K h (t), and B h (t), we design the following periodic learning laws where f f (t) = 0, Kh (t) = 0, and Bh (t) = 0 for t < 0; α 1 , α 2 , and α 3 are positive gains; α 10 (t), α 20 (t), and α 30 (t) are a continuous and strictly increasing function that satisfy and sat K * Kh (t − T) , sat B * Kh (t − T) are defined similarly to the definition of sat f * ( f f (t − T)).

and
∼ B h are uniformly bounded, and the tracking error e converges to zero.
V is negative-definite outside the following region Furthermore, ∼ θ is bounded from the projection modification (20).Therefore, V is bounded on [T 1 , T].
ii. Convergence of e For t ≥ T, the time derivatives of V 1 and .
from which one can obtain V(t) ∈ L ∞ for t ≥ T and From the above analysis, Therefore, the righthand side of ( 18) is bounded and then .ϵ ∈ L ∞ which shows that ϵ is uniformly continuous in [T, ∞).Babalat's Lemma can be conducted to show that lim Remark 3. The proposed impedance learning control method requires that the human impedance profiles are temporally periodic, and the robotic dynamics can be linearly parameterized.If any of these requirements are not satisfied, the proposed control method cannot be implemented.

Remark 4.
The hybrid adaptive impedance learning control can provide variable impedance regulation for repetitive robot-assisted rehabilitation without the requirement of the interactive force sensing.Remark 5.In applications, control parameters can be selected step by step.Firstly, PD control gains k 1 and k 2 are adjusted to make the closed-loop control system stable.Secondly, the adaptive learning rate α 0 is designed to make the robot follow the reference trajectory with satisfactory tracking accuracy in the case of no human-robot interaction.Next, periodic learning parameters for time-varying impedance are set.Finally, the control parameters are further fine-tuned to achieve satisfactory interactive performances.Under the proposed hybrid adaptive control strategy, the human-robot system follows the reference trajectory cooperatively and repetitively.

Simulations
The planar five-bar parallel mechanism depicted by Figure 1 with the structure in Figure 2 is considered as the mechanism for some upper limb rehabilitation robots and is adopted here for simulation validation.To show the effectiveness of the proposed hybrid adaptive control (HAC) and its advantages in comparison with the adaptive control (AC) in [28] and the biomimetic adaptive control (BAC) in [26,27], simulations are conducted on a planar parallel robot depicted by the render graph in with a five-bar parallel.
Actuators 2024, 13, x FOR PEER REVIEW 9 of 18 adopted here for simulation validation.To show the effectiveness of the proposed hybrid adaptive control (HAC) and its advantages in comparison with the adaptive control (AC) in [28] and the biomimetic adaptive control (BAC) in [26,27], simulations are conducted on a planar parallel robot depicted by the render graph in with a five-bar parallel.In Figure 2,   ,  = 0, 1, 2, 3, 4 are the five joint points, where  0 is the HRI point, and  2  4 ,  4  0 ,  0  3 ,  3  1 denote four robotic links whose masses, lengths, and inertial moments are given in Table 2.The distances between the joints and respective centers of mass are all 0.25 m, i.e.,   = 0.25 m for  = 1, 2, 3, 4, and the length of  1  2 is 0.15 m.In addition to its advantages over a series mechanism, a parallel mechanism results in a more complex model for the robot.In robot-assisted rehabilitation, the reference trajectory is   = [0.075+ 0.15(/2), 0.55 + 0.15(/2)]  .
Table 2.The parameters of the parallel robot.In Figure 2, A i , i = 0, 1, 2, 3, 4 are the five joint points, where A 0 is the HRI point, and A 2 A 4 , A 4 A 0 , A 0 A 3 , A 3 A 1 denote four robotic links whose masses, lengths, and inertial moments are given in Table 2.The distances between the joints and respective centers of mass are all 0.25 m, i.e., l ci = 0.25 m for i = 1, 2, 3, 4, and the length of A 1 A 2 is 0.15 m.In addition to its advantages over a series mechanism, a parallel mechanism results in a more complex model for the robot.In robot-assisted rehabilitation, the reference trajectory is x r = [0.075+ 0.15sin(πt/2), 0.55 + 0.15cos(πt/2)] T .In Figure 2,   ,  = 0, 1, 2, 3, 4 are the five joint points, where  0 is the HRI point, and  2  4 ,  4  0 ,  0  3 , 3  1 denote four robotic links whose masses, lengths, and inertial moments are given in Table 2.The distances between the joints and respective centers of mass are all 0.25 m, i.e.,   = 0.25 m for  = 1, 2, 3, 4, and the length of  1  2 is 0.15 m.In addition to its advantages over a series mechanism, a parallel mechanism results in a more complex model for the robot.In robot-assisted rehabilitation, the reference trajectory is   = [0.075+ 0.15(/2), 0.55 + 0.15(/2)]  .The feedforward force, the stiffness matrix, and the damping matrix are The feedforward force, the stiffness matrix, and the damping matrix are

Inertial
For the proposed HAC, we choose the following control parameters: In the PD control, the proportional control parameters are determined according to the requirements for the response speed and the static speed error based on the transfer function of the robot model, and the derivative control parameters are designed based on the regulation time and overshooting.Then, the hybrid adaptive learning parameters are determined by trial and evaluation to obtain satisfactory interactive performance.
Figure 3a shows the tracking performance by the proposed HAC, the AC in [28], and the BAC in [26,27] with locally enlarged parts shown in the right part of the figure.From Figure 3a, we can see that the actual trajectory (AT) and the desired trajectory (DT) are almost coincident after several periodic motions using the proposed method, while the AT consistently deviates from the DT using the AC in [28] and the BAC in [26,27].To better display the effectiveness and advantages of the proposed control method, comparisons of tracking errors among the three methods are illustrated in Figure 3b.It can be seen that the HAC makes the tracking error converge to zero, while the AC in [28] and the BAC in [26,27] only make the tracking error be bounded.The not satisfactory tracking performance by the AC is mainly due to the limitations of traditional differential adaptation in estimating time-varying signals.The better tracking performance validates the superiority of the HAC in comparison to the BAC in [28].
It should be noted that the estimated force cannot be very close to the actual interactive force (see Figure 4a) due to the strong coupling between robot modeling uncertainties and human impedance uncertainties.Such derivation is allowable and does not affect control performance shown in Figure 3.In addition, the stiffness and damping variation is finite, which is illustrated by Figure 4b.
isons of tracking errors among the three methods are illustrated in Figure 3b.It can be seen that the HAC makes the tracking error converge to zero, while the AC in [28] and the BAC in [26,27] only make the tracking error be bounded.The not satisfactory tracking performance by the AC is mainly due to the limitations of traditional differential adaptation in estimating time-varying signals.The better tracking performance validates the superiority of the HAC in comparison to the BAC in [28].It should be noted that the estimated force cannot be very close to the actual interactive force (see Figure 4a) due to the strong coupling between robot modeling uncertainties and human impedance uncertainties.Such derivation is allowable and does not affect control performance shown in Figure 3.In addition, the stiffness and damping variation is finite, which is illustrated by Figure 4b.

Experiments
To further verify the effectiveness and validity of the proposed control method, experiments are implemented on the self-designed planar five-bar parallel mechanism (see Figure 5) which was driven by rotary actuators (erob80, ZeroErrInc., Beijing, China).

Experiments
To further verify the effectiveness and validity of the proposed control method, experiments are implemented on the self-designed planar five-bar parallel mechanism (see Figure 5) which was driven by rotary actuators (erob80, ZeroErrInc., Beijing, China).Twincat 3.0 eXtended automation engineering running on a PC acts as the fieldbus master in the EtherCAT network.The control algorithm is implemented by the TwinCAT Component Object Model (TcCOM) programed in structured Text (ST) languages.Both the control frequency and the sampling period for the controller are 1 ms.The rotary actuator is operated in torque mode.The planar parallel robot used in our study is an end-effector rehabilitation robot.The trajectory of the robot is usually predefined by therapists during rehabilitation training.During the rehabilitation process, the patient's upper limbs are trained with the help of a rehabilitation robot, as in Figure 5.The emerging interactive forces between subjects and the robot may give rise to trajectory deviation for the robot.The proposed impedance learning control can make the human robot system cooperatively follow the desired trajectory.In the experiment, the robot and a subject repetitively track an ellipse trajectory.A typical set of experimental results of the proposed method is illustrated in Figure 6 which presents the tracking performance from the 1st task completion to the 40th completion.It can be inferred from Figure 6 that the tracking error is satisfactory after 30 task completions, i.e., after 120 s.The tracking error in the experiment will not converge to zero due to measurement noises, device inherent deficiencies, and so on.The proposed HAC provides finite stiffness variation for robot-environment interaction (see Figure 7).Although there exists some chattering in the experiment results by the proposed HAC due to measurement noises and the difference form of the periodic adaptation in (21)-( 23), the chattering is very limited and is allowable.The experimental results by the AC [28] are shown in Figure 8.It is obvious that the tracking performance and the tracking error are unsatisfactory in the whole interactive process.The reason comes from the inherent deficiencies of the traditional differential adaptation in estimating time-varying signals.Taking Figures 6 and 8, we can conclude the superiority of the proposed HAC in comparison with the AC [28].

Conclusions
This paper presents an impedance learning-based HAC strategy for HRI with robotic modeling parameter uncertainties and human impedance uncertainties.The robotic parameter vector is constant and is estimated by a differential adaptive law with projection modification, while the time-varying impedance profile including the feedforward force, the stiffness, and the damping is estimated by periodic adaptations in (21)- (23).In Theorem 1, we prove the convergence of tracking errors and the boundedness of estimation errors using Babalat's Lemma.Though the illustrative example on a parallel robot, we validate the following: (i) the HAC achieves time-varying impedance regulation for HRI with a stability guarantee; (ii) the asymptotic convergence of the tracking error is achieved by the HAC as opposed to the uniformly ultimately bounded results of the tracking errors

Conclusions
This paper presents an impedance learning-based HAC strategy for HRI with robotic modeling parameter uncertainties and human impedance uncertainties.The robotic parameter vector is constant and is estimated by a differential adaptive law with projection modification, while the time-varying impedance profile including the feedforward force, the stiffness, and the damping is estimated by periodic adaptations in (21)- (23).In Theorem 1, we prove the convergence of tracking errors and the boundedness of estimation errors using Babalat's Lemma.Though the illustrative example on a parallel robot, we validate the following: (i) the HAC achieves time-varying impedance regulation for HRI with a stability guarantee; (ii) the asymptotic convergence of the tracking error is achieved by the HAC as opposed to the uniformly ultimately bounded results of the tracking errors in [25]; and (iii) the periodic adaptations have better performances in the estimation of time-varying impedance profiles than the differential adaptation in [25] by using the periodicity.
The subjects in this study are healthy individuals.Next, we will apply the device to patients with spasticity in upper limbs for assisted rehabilitation to study its robustness, as well as its capability in assistive movement with low tracking error in the presence of spasticity.

Figure 1 .
Figure 1.The render graph of a planar parallel robot.

Figure 1 .
Figure 1.The render graph of a planar parallel robot.

A 3 Figure 1 .
Figure 1.The render graph of a planar parallel robot.

Figure 3 .
Figure 3. Simulation performances by the proposed HAC, the AC in [28], and the BAC in [26,27].(a) The tracking performance, (b) The tracking error, (c) The control input.Actuators 2024, 13, x FOR PEER REVIEW 12 of 18

Figure 4 .
Figure 4.The estimation of the interactive force and the estimated stiffness and damping by the proposed HAC.(a) Interactive force estimation, (b) Stiffness and damping.

Figure 4 .
Figure 4.The estimation of the interactive force and the estimated stiffness and damping by the proposed HAC.(a) Interactive force estimation, (b) Stiffness and damping.

Figure 6 .Figure 6 .
Figure 6.The experimental results by the proposed HAC.(a) The tracking performance, (b) The tracking error, (c) The control input.

Figure 7 .
Figure 7.The estimated stiffness by the proposed HAC.

Figure 7 .
Figure 7.The estimated stiffness by the proposed HAC.Actuators 2024, 13, x FOR PEER REVIEW 16 of 18

Figure 8 .
Figure 8.The experimental results by the AC in [29].(a) The tracking performance, (b) The tracking error, (c) The control input.

Figure 8 .
Figure 8.The experimental results by the AC in [29].(a) The tracking performance, (b) The tracking error, (c) The control input.

Table 2 .
The parameters of the parallel robot.

Table 2 .
The parameters of the parallel robot.