Shared Driving Assistance Design Considering Human Error Protection for Intelligent Electric Wheelchairs

Abstract

To effectively provide the handicapped with mobility aids, studies on the shared autonomy of robotic systems have been widely cultivated. This study proposes an adaptive shared control strategy to realize reliable and safe driving assistance on an intelligent electric wheelchair with protection against human errors. The theoretical framework of the system is analyzed by the linearized reference wheelchair model and stable characteristics of obstacle avoidance behavior can be subsequently derived according to the Lyapunov analysis and Liénard-Chipart criterion. Based on the convex analysis, the relationships between human input and robot control are investigated to determine shared control weights. As such, safety and reliability can be guaranteed. To verify the performances of the proposed approach, human errors including skill-based errors, decision errors, and violations are considered in the experiments. The experimental results based on a comprehensive study show that the proposed method is capable of enhancing driving safety and reducing operation burden in terms of the designed criteria with fluency, smoothness, and time efficiency while protecting the user from human manual errors.

1. Introduction

Mobility inconvenience usually hinders elders or physically inconveniences people to complete activities of daily livings and decreases their will to go outside. The raising problems of negative emotions as well as psychological diseases thus easily occur. One possible solution to this problem is to provide a useful and suitable robotic system to assist this group of people with their daily activity livings. For example, an electric mobile robot can provide a reliable ability to avoid obstacles [1] along with techniques such as path-following, speed planning, and so on. Given the large need for assistance from a rapidly aging population and people with various disabilities, assistive robotic systems have shown great potential in the research community. However, even though an assistive robot can help solve limited mobility, sensory, and cognitive level problems and improve safety for the users, according to [2], it should provide help only when it is needed. In other words, when a fully autonomous robot takes overall control authority, it is reported in [3] that the user would feel unsafe and would try to reclaim control of the system. Hence, a fully autonomous system is not an appropriate solution.

In order to simultaneously address the user’s control authority and the autonomous control of the robot, the idea of shared control is increasingly focused on. A shared control system has features that combine an autonomous agent and a human user by taking advantage of both human intelligence and the agent to aid each other. Thereafter, it has been shown by many contributors that shared control can be applied in many areas, such as walking assistant robots in [4] and semi-autonomous controlled robots in [5]. Involving shared control in a robotic wheelchair can also be commonly seen from the literature, as it is out in [6], that with shared control ability, the robotic wheelchair not only provides self-dominance for the user but also ensures overall security. According to the needs of the wheelchair user, shared control can be categorized into two classes: task-level shared control and servo or execution-level control. The former refers to macro-level control, where control inputs are issued by broad motion commands from the user such as moving along the corridor, moving through an open door, etc. Then, as the robotic wheelchair starts to perform the general tasks according to the commands, the control of the wheelchair is mostly taken over by the robotic system. Macro-level control is widely shown in teleoperation applications. For example, Uratsuji et al. proposed dynamic shared control to increase the comfort for the user and then included an annotated map to adjust control weights of turning velocities [7]. Though task-level control only passively provides autonomous service when a command is requested by the user, it cannot actively handle hazardous situations when unreliable manual control or instantaneous control behavior is given by the user.

As being a cooperative way that is highly accepted and extensively used in human-robot interaction systems, servo-level shared control is regarded as micro-control. The feasible control input is generated with the combination of the robot itself and the user at any sample point. As shown in (1), $u_s$ is the controlling of the overall system, while $u_r$ and $u_h$ are the inputs from the robot and the human user, respectively. ${\alpha }_s$, which ranges from 0 to 1, is responsible for adjusting the ratio of $u_r$ and $u_h$, as illustrated by

$$u_s={\alpha }_s{u}_r+\left(1-{\alpha }_s\right)u_h$$

(1)

Based on the design of the allocation weight ${\alpha }_s$, there are two categories of servo-level shared control. One is the fixed shared control, which means that the weight is invariant to time, and the other is a self-adaptive shared control, where the weight is adaptively changed according to given rules to adjust the proportion of the two inputs. The fixed share control is preferable if input conditions are known in advance, such as dual-user haptic training for medical surgery [8,9], where the control allocation can be chosen according to the expertise of the trainee and the trainer. Self-adaptive shared control, on the other hand, dynamically allocates control authority to the human user and autonomy. It is used in scenarios when inputs may bring uncertain results or the environment may change. The rules of the allocation can be based on human performances [10], safety [11], interaction time duration [12], environmental information [13], comfort and obedience [14], and so on.

Because self-adaptive shared control harmoniously brings efforts from both the human and robot, it is capable of counteracting dangerous maneuvers in terms of preventing collisions and falls due to a failure in judgment or conflicting requests from an unsuspecting operator [15]. Therefore, successful demonstrations can be commonly seen from many contributions. In [16], a harmonic potential field-based non-linear sliding mode controller was developed to obtain both the control effort exerted by the human and the autonomy control for obstacle avoidance. Through the Lyapunov-based stability analysis, the proposed semi-autonomous wheelchair is able to navigate in an environment with obstacles safely and reliably. However, even though it generates different safe and collision-free reference trajectories, the motion constraints are still belated due to the instantaneous control behavior or imperfect trajectory tracking performance which may result in hazards or damage [17]. Research such as [18,19] proposed semi-autonomous wheelchair navigation in the form of intuitive obstacle avoidance. By merging the user control coming from a wheelchair controller with a set of constraints deduced from sensors, the shared control law is developed based on distinct areas of the wheelchair velocity domain. Different from the idea of linear blending method as described in (1), a probabilistic framework was proposed in [11,20] which designed the probabilistic shared control (PSC) by modeling the interaction between the user’s intention and the wheelchair’s path planner as a joint probability distribution. According to their experiments, PSC yielded a greater reduction in collisions than the linear blending method without compromising on distance traveled and task duration time. Nevertheless, the above-mentioned approaches presume that their system requires good driving skills by the user [21], and thus any incorrect user’s decision could make the system fail to provide safe and stable assistance.

Apart from the weight allocation methods, another shared control community uses haptic control. Interesting papers such as [22,23] involved learning from demonstrations in wheelchair assistance that were presented to learn shared control policies from demonstrations offered by a human assistant. The general concept behind the utility of a haptic controller is to circumvent the unnatural aspects of robotic autonomy and instead adjust a driver’s manual steering input onto a safer path via experts’ demonstrations. However, since haptic feedback on wheelchairs uses haptic-enabled wheelchair controllers, i.e., kinesthetic joysticks, it is pointed out in [24,25] that such a method could lead to instability issues. For example, a third-person perspective on a task is a transformed frame of reference that could result in misguided assistance, and the demonstrator might not agree with the plan originating from the primary user. Recently, developing a shared control framework based on brain-machine interaction (BMI) is also attractive to some researchers. In [26], a shared control strategy was proposed by combining brain-machine control mode and autonomous control mode. The former is a steady-state visual evoked potential (SSVEP)-based brain-computer interface employed to control a wheelchair moving in different directions on a plane. The latter, on the other hand, is achieved by a Kinect RGBD sensor used for simultaneous localization and mapping. Although [26] provides astonishing effectiveness, according to [27], most of the SSVEP-based BCI systems do not provide a user-friendly interactive interface, producing more time for the user to pay attention to the interface. Moreover, to move the robot to a certain destination, the operator had to change the moving directions of the robot frequently in the trajectory, which costs much time and energy. Still, another shared control is designed by means of employing a machine learning model. In [28], a model was proposed that correlates objective performance metrics and subjective evaluations of autonomous wheelchair control paradigms. By doing so, the model can predict the most preferred shared-control method according to metrics including safety, effort, performance, etc. Further presented in [29], the assistance could come at the expense of user satisfaction as the users often feel that they are fighting for autonomy. Thus, the trade-off between task success and autonomous intervention is not consistently accepted by the users.

From the viewpoint of control theory, analysis, and verification of a controller are crucial for optimizing the effectiveness of the overall system. Indeed, as mentioned in [30], stability analysis of shared control systems is essential to ensure performance within safety limits. In [4,30], fuzzy logic control and model predictive control are, respectively, used in obstacle avoidance of shared control systems. Unfortunately, they did not provide rigorous stability guarantees. Jiang et al. [31,32] showed fixed control allocation of a shared controller, where Lyapunov-based stability analysis was presented to guarantee finite-time stability for its obstacle avoidance controller working in collaboration with the human agent. The use of convex feasible set stability analysis can also be seen in the literature. In [33,34,35], convex feasible sets are used and analyzed for designing their shared control strategy. Although those proposed adaptive weighting methods can be proved in a stable manner and have some margin to guarantee stability, their design of shared control strategy is lack of operation safety consideration. According to their experiments, when the wheelchair is driven for a long distance at a high speed, multiple collisions have easily occurred, especially existing human operation errors.

This study is motivated by the human-robot collaborative driving task in the context of human error protection. Especially for seniors and disabled people suffering from limited sensory capability and operation ability during real-time manual driving for a standard electric wheelchair, the adaptation of the shared control approach should be able to make a significant shift from the traditional human-machine cooperative control to the flexible mobility aid and further the protection from human errors. To avoid the drawbacks of previous work as mentioned above, this study concerns manual errors, as specified in [36], that include control delay (skill-based error), miscarriage of orientation (decision error), and over-speeding (violation). The developed shared control strategy for the electric wheelchair can assist the user to move freely and smoothly in an environment with collision avoidance regarding obstacles or dangerous areas. Additionally, the stability of the proposed adaptive shared controller can be guaranteed to ensure the safe operation of the electric wheelchair, while compensating the unsafe operation under human control. The conducted experiments demonstrate the advantages of the proposed shared control strategy via a set of test in a realistic human-robot interaction and ensure the safe mobile assistance of the developed intelligent electric wheelchair (iE-Wheelchair). The remainder of this paper is organized as follows. Section 2 describes an overview of our developed iE-Wheelchair. Section 3 provides the obstacle avoidance controller with the stability analysis using the Liénard-Chipart criterion. Section 4 discusses the design of the shared controller and its stability analysis using convex sets. The experiments are presented in Section 5, and the conclusion is given in Section 6.

2. System Overview

The iE-Wheelchair developed in this study is shown in Figure 1. The GUI is a laptop, which is connected to a joystick, and is provided for the user to send control signals to the electric wheelchair. Because laser measurements provide more reliable and precise distance measurements than sonar sensors, a 2D laser range finder (SICK LMS100) is mounted to the front base of the wheelchair and used in the obstacle avoidance control. The R-net motor module is responsible for driving the wheelchair motion and allocating power to the left and right motors. An embedded controller (NI CompactRIO) is utilized for controlling the wheelchair motors, and all of the required electricity is provided by a 24 V DC battery pack. An onboard computational unit, which is another laptop installed to process and integrate sensory information, manages the proposed shared control system, and runs higher-level navigation tasks.

3. Design of the Obstacle Avoidance Controller

Before designing the obstacle avoidance controller, the incomplete constraints of the robot and the user-controlled inputs for the iE-Wheelchair are investigated. As to the feasibly shared control used in human-robot interaction systems, Figure 2 depicts the proposed obstacle avoidance controller as the reference model control while considering the human control inputs. The motion controlling from the user is turning accelerations ${\dot{\omega }}_u$ and accelerations ${\dot{v}}_u$ with respect to the center of the iE-Wheelchair. On the other hand, the controlling of the robot considers the measurements d_ri and d_li, representing, respectively the distances observed from different angles by the laser range finder. The subscripts r and l denote the sensing distance from the right- and left-half sides of the iE-Wheelchair, respectively, as shown in Figure 3.

Based on the dynamics of a two-wheeled mobile robot model, a simple illustration of iE-Wheelchair employed in our study is shown in Figure 3, where the system model in terms of turning acceleration $\ddot{\varphi }$ and translational acceleration $\dot{v}$ with respect to the laser range finder attached to the front of the iE-Wheelchair, without considering the wheel-slippage, are given as follows:

$$I\ddot{\varphi }+C_{\varphi }\dot{\varphi }=\underset{human control inputs}{\underbrace{I\ddot{{\varphi }_u}}}+\underset{robot control inputs}{\underbrace{{\displaystyle \sum }_{i=1}^m\frac{{\alpha }_i}{\sqrt[n]{d_{ri}}}-{\displaystyle \sum }_{i=1}^m\frac{{\alpha }_i}{\sqrt[n]{d_{li}}}}}=\underset{human control inputs}{\underbrace{I\ddot{{\varphi }_h}+C_{\varphi }\dot{{\varphi }_h}}}+\underset{robot control inputs}{\underbrace{I\ddot{{\varphi }_r}+C_{\varphi }\dot{{\varphi }_r}}}$$

(2)

$$M\dot{v}+C_{v}v=\underset{human control inputs}{\underbrace{M\dot{v_u}}}-\underset{robot control inputs}{\underbrace{{\displaystyle \sum }_{i=1}^m\frac{{\beta }_i}{\sqrt[n]{d_{ri}}}-{\displaystyle \sum }_{i=1}^m\frac{{\beta }_i}{\sqrt[n]{d_{li}}}}}=\underset{human control inputs}{\underbrace{M\dot{v_h}+C_v{v}_h}}+\underset{robot control inputs}{\underbrace{M\dot{v_r}+C_v{v}_r}}$$

(3)

where I and M are angular inertial and mass of the iE-Wheelchair, and the viscous friction coefficients of angular velocity and velocity are $C_{\varphi }$ and $C_v$, respectively. Since there may exist viscous frictions when the wheelchair moves, they are concerned about the system in order to increase the overall stability. In Equation (2), the last two terms on the right-hand side depict the steering effect from the robot control that depends on the sensed distance magnitude $d_{ri}$ and $d_{li}$. Similarly, the last two terms on the right-hand side Equation (3) provide the braking effect that depends on the sensed distance magnitude $d_{ri}$ and $d_{li}$. Note that the terms $I\ddot{{\varphi }_u}$ and $M\dot{v_u}$ present the desired driving force from the human control inputs where $\ddot{{\varphi }_u}={\dot{\omega }}_u$. The terms $I\ddot{{\varphi }_r}+C_{\varphi }\dot{{\varphi }_r}$ and $M\dot{v_r}+C_v{v}_r$ are the ones yielded from the robot to be able to perform the obstacle-avoidance task. The parameters $\dot{{\varphi }_h}$ and $v_h$ derived from human control inputs as well as $\dot{{\varphi }_r}$ and $v_r$ derived from robot control inputs are both used for the design of the shared controller, as introduced in Section 4. In the summation calculation of Equations (2) and (3), m represents half the number of distance observations, while ${\alpha }_i$ and ${\beta }_i$ are constant parameters that are used to change turning and braking conditions. Finally, the value friction of the term $1/\sqrt[n]{d_{ji}}$ (j = r and l) is designed to represent the extent due to the influences from obstacles to turning and braking; that is, the lower value adopted for n raises the higher impact of the relative distances from obstacles to the wheelchair.

Based on the linear model of the control system in Equations (2) and (3), the stability analysis of the obstacle avoidance controller can be accordingly developed. To begin with, this study investigates the motion dynamics of the wheelchair in the moving environment as shown in Figure 4, which consists of curves of walls on the two sides of the path. As for the case in which straight walls appeared on the two sides of the path, this has been analyzed and proposed in [37]. By assuming that the two curves belong to a simple concentric circle with radii $R_c+r$ and $R_c-r$, respectively, the wheelchair moves along the central path with velocity and angular velocity can be expressed as $v=v_0+v_S$ and $\dot{\varphi }={\dot{\varphi }}_0+{\dot{\varphi }}_S$, where $v_0$ and ${\dot{\varphi }}_0$ are two expected constants while $v_S$ and ${\dot{\varphi }}_S$ are small variants. According to Equations (2) and (3), it is desired to assume that human control inputs satisfy $I\ddot{{\varphi }_u}=C_{\varphi }{\dot{\varphi }}_0$ and $M\dot{v_u}=C_v{v}_0$. Additionally, we assume that the distance between the wheelchair and the center of two curves is $r_s$, which is a small variance, ${\psi }_i$ is the angle between each sensory observation i and the center of the wheels, and the distance between the center of the wheelchair, as well as the laser range finder, is R. As a result, a distance between every sensory observation and the walls can be provided by,

$$d_{ri}=-z_{ri}+\sqrt{z_{ri}{}^2-r_s\left(2R_c+r_s\right)+r\left(2R_c+r\right)}-R$$

(4)

$$d_{li}=z_{li}-\sqrt{z_{li}{}^2-r_s\left(2R_c+r_s\right)+r\left(-2R_c+r\right)}-R$$

(5)

$$z_{ri}=\left(R_c+r_s\right)cos\left({\psi }_i+{\varphi }_S\right)$$

(6)

$$z_{li}=\left(R_c+r_s\right)cos\left({\psi }_i-{\varphi }_S\right)$$

(7)

Because $r_s$ and ${\varphi }_S$ are small variants, the zero and first derivative of Equations (4) and (5) with respect to $r_s$ and ${\varphi }_S$ are linear. Thus, by using Taylor expansion, we can linearize $d_{ri}$ and $d_{li}$ as,

$$d_{ri}\approx -a_i+b_i-R-\left(c_i+d_i\right)r_s+\left(e_i-f_i\right){\varphi }_S$$

(8)

$$d_{li}\approx a_i-\widehat{b_i}-R+\left(c_i+{\widehat{d}}_i\right)r_s+\left(e_i-{\widehat{f}}_i\right){\varphi }_S$$

(9)

where

$$a_i=R_{c}cos{\psi }_i$$

(10)

$$b_i=\sqrt{a_i{}^2+2r{R}_c+r^2}$$

(11)

$$c_i=cos{\psi }_i$$

(12)

$$d_i=R_c\left(1-{cos}^2{\psi }_i\right)/b_i$$

(13)

$$e_i=R_{c}sin{\psi }_i$$

(14)

$$f_i=R_c{}^{2}sin{\psi }_{i}cos\psi /b_i$$

(15)

$${\widehat{b}}_i=\sqrt{a_i{}^2-2r{R}_c+r^2}$$

(16)

$${\widehat{d}}_i=R_c\left(1-{cos}^2{\psi }_i\right)/{\widehat{b}}_i$$

(17)

$${\widehat{f}}_i=R_c{}^{2}sin{\psi }_{i}cos\psi /{\widehat{b}}_i$$

(18)

Since the linearization of $1/\sqrt[n]{d_{ji}}$ can also be approximated by using the Taylor series expansion, i.e.,

$$1/\sqrt[n]{d_{ji}}=-p_i{d}_{ji}+q_i$$

(19)

where j = l, r and $p_i$ as well as $q_i$ are two positive constants. From (8), (9) and (19), Equation (2) considering only the robot control inputs can be rewritten as

$$I\ddot{\varphi }+C_{\varphi }\dot{\varphi }={\displaystyle \sum }_{i=1}^m{\alpha }_i{p}_i\left\{\left(2a_i-b_i-{\widehat{b}}_i\right)+\left(2c_i+d_i+{\widehat{d}}_i\right)r_s-\left(f_i+{\widehat{f}}_i\right){\varphi }_S\right\}$$

(20)

Similarly, the linearization of Equation (3) considering only the robot control inputs is given as follows:

$$M\dot{v_S}+C_v{v}_S={\displaystyle \sum }_{i=1}^m{\beta }_i\left\{p_i\left[\left(b_i-{\widehat{b}}_i\right)+\left({\widehat{d}}_i-d_i\right)r_s+\left(2e_i+f_i-{\widehat{f}}_i\right){\varphi }_S-2R\right]-2q_i\right\}$$

(21)

By assuming that ν_s and ϕ_S are small, the derivative of r_s is accordingly linearized as shown below.

$$\dot{r_s}=-v\sin {\psi }_S\simeq -\left(v_0+v_s\right){\varphi }_S\simeq -v_0{\varphi }_S$$

(22)

From Equations (20)–(22), the state-space of the system dynamic equation $\dot{\mathit{X}}=\mathit{A}\mathit{X}+\mathit{\Phi }$ can therefore be found as follows.

$$\left[\begin{array}{c}\dot{{\varphi }_s}\\ \ddot{{\varphi }_s}\\ \dot{r_s}\\ \dot{v_s}\end{array}\right]=\dot{\mathit{X}}=\mathit{A}\mathit{X}+\mathit{\Phi }=\mathit{A}\left[\begin{array}{c}{\varphi }_s\\ \dot{{\varphi }_s}\\ r_s\\ v_s\end{array}\right]+\mathit{\Phi }$$

(23)

where A and $\mathit{\Phi }$ are shown below.

$$\mathit{A}=\left[\begin{array}{cccc}0& 1& 0& 0\\ -\frac{1}{I}{\mathsf{\Sigma }}_{i=0}^m{\alpha }_i{p}_i\left(f_i+{\widehat{f}}_i\right)& \frac{-C_{\varphi }}{I}& \frac{1}{I}{\mathsf{\Sigma }}_{i=0}^m{\alpha }_i{p}_i\left(2c_i+d_i+{\widehat{d}}_i\right)& 0\\ -v_0& 0& 0& 0\\ \frac{1}{M}{\mathsf{\Sigma }}_{i=0}^m{\beta }_i{p}_i\left(2e_i+f_i+{\widehat{f}}_i\right)& 0& \frac{1}{M}{\mathsf{\Sigma }}_{i=0}^m{\beta }_i{p}_i\left(-d_i+{\widehat{d}}_i\right)& \frac{-C_v{v}_0}{M}\end{array}\right]$$

(24)

$$\mathit{\Phi }=\left[\begin{array}{c}0\\ \frac{1}{I}{\mathsf{\Sigma }}_{i=0}^m{\alpha }_i{p}_i\left(2a_i+b_i+{\widehat{b}}_i\right)\\ 0\\ \frac{1}{M}{\mathsf{\Sigma }}_{i=0}^m{\beta }_i(p_i\left(b_i-{\widehat{b}}_i-2R\right)-2q_i)\end{array}\right]$$

(25)

Remark 1. 

Substituting the parameters of (10)–(18) to (25), a constant matrix $\mathit{\Phi }$ can be achieved. To investigate the stability of the state X, a Lyapunov candidate function $V\left(X\right)=\frac{1}{2}{\mathit{X}}^T\mathit{X}$ can be chosen. To satisfy the stability in the sense of Lyapunov theory $\dot{V}\left(X\right)<0$, the state X of the linear system $\dot{\mathit{X}}=\mathit{A}\mathit{X}+\mathit{\Phi }$ is uniformly ultimately bounded (UUB) stable if the matrix A is Hurwitz and $\mathit{X}\le \mathit{\Phi }$. It is worth mentioning that the bounded value of $\Vert \mathit{\Phi }\Vert$ can represent the error bound for the relative distance between the wheelchair and the obstacle. As a result, the nonzero value of $\Vert \mathit{\Phi }\Vert$ also provides a buffer zone around the obstacles with regarding to the chosen parameters.

Consequently, to investigate whether the matrix A is Hurwitz, the characteristic real polynomial can be derived as follows:

$$P\left(s\right)=\left|s\mathit{I}-\mathit{A}\right|=s^4+s^3{h}_3+s^2{h}_2+s{h}_1+h_0$$

(26)

where the coefficients h₀, h₁, h₂, and h₃ are

$$h_0=\frac{C_v{v}_0}{IM}{{\displaystyle \sum }}_{i=0}^m{\alpha }_i{p}_i\left(2c_i+d_i+{\widehat{d}}_i\right)$$

(27)

$$h_1=\frac{C_v}{IM}{{\displaystyle \sum }}_{i=0}^m{\alpha }_i{p}_i\left(f_i+{\widehat{f}}_i\right)+\frac{h_0\mathrm{M}}{C_v}$$

(28)

$$h_2=\frac{C_{\varphi }{C}_v}{IM}+\frac{1}{I}{{\displaystyle \sum }}_{i=0}^m{\alpha }_i{p}_i\left(f_i+{\widehat{f}}_i\right)$$

(29)

$$h_3=\frac{C_{\varphi }}{I}+\frac{C_v}{M}$$

(30)

By using the Liénard-Chipart criterion [38], as long as all of the coefficients of Equation (26) with respect to the Hurwitz matrix of P are positive, then it is regarded as stable. Hence, to achieve the safe operation with the stability guarantees for the obstacle avoidance controller, the following equation must be satisfied:

$$D=\left|\begin{array}{ccc}{h}_3& h_1& 0\\ 1& h_2& h_0\\ 0& h_3& h_1\end{array}\right|=\left(h_3{h}_2-h_1\right)h_1-h_3^2{h}_0>0$$

(31)

Substituting the parameters of (10)–(18) to (31), we can obtain

$$C_{\varphi }\left(f_i+{\widehat{f}}_i\right)-I{v}_0\left(2c_i+d_i+{\widehat{d}}_i\right)>0$$

(32)

Then, after substituting the parameters of (10)–(18) to (32), as long as $C_{\varphi }$ satisfies the above inequality, the system stability of the dynamic equation in (23) can be guaranteed.

4. Design of Adaptive Shared Controller

This section presents the design of the self-adaptive shared controller based on the convex analysis similar to [34,35]. Before introducing the convex set stability analysis, let us define several necessary parameters: $u_r=[v_r$, ${\omega }_r]$ and $u_h=[v_h$, ${\omega }_h]$ are the control inputs from the robot and the human user, respectively; $u_s=[v_s$, ${\omega }_s]$ is the final controlling of the shared controller based on the designed weight ${\alpha }_s$ to tune the ratio between $u_r$ as well as $u_h$.

4.1. Convex Set Stability Analysis

Assume that the valid input set from the user is defined as $U_h$, where $U_h=\{u_h: \dot{V_h}\left(u_h\right)<0\}$. The set is then regarded as a convex set because $u_h\in U_h$ and the range of the stability are continuous during the operation. Likewise, the valid input set from the robot is defined as $U_r=\{u_r: \dot{V_r}\left(u_r\right)<0\}$ where $u_r\in U_r$. The control weight of $u_s$ is the point that goes through the closed line between $u_r$ and $u_h$, where the closed line can be defined as,

$$U_s=\{u_s: u_s={\alpha }_s{u}_r+\left(1-{\alpha }_s\right)u_h |u_s\in U_s, 0<{\alpha }_s<1\}$$

(33)

Accordingly, there are four conditions between $u_r$ and $u_h$ that are preferable for designing and developing the shared controller:

• As shown in Figure 5a, the state is stable if $U_h\subset U_r$ because any arbitrary control applied on the iE-Wheelchair is independent of causing collision with obstacles. Hence, the system is reliable and safe. Under such a scenario, ${\alpha }_s$ is assigned to be 0, implying that the system is under is controlled completely by the user.
• As shown in Figure 5b, the condition where $U_r\subset U_h$ refers to the situation is when the valid input set from the robot involves the inputs from the user. Therefore, some inputs can be in both sets simultaneously, while others cannot. In actual operations, such a scenario implies that there exist obstacles in the environments and thus the iE-Wheelchair requires manual control to avoid collisions by tuning the value of control weight ${\alpha }_s$. From the viewpoint of the control theory, $U_s=U_r\cap U_{sl}$, and the proper value of ${\alpha }_s$ lies within the inequality ${\alpha }_{sl}$ < ${\alpha }_s$ < 1, where ${\alpha }_{sl}$ is the minimum of ${\alpha }_s$.
• When the valid input sets of the user and the robot intersect with each other, i.e., $U_r\cap U_h\ne \varnothing$ as well as $U_r\not\subset U_h$, as shown in Figure 5c, this scenario is similar to the second condition, but it is more complicated in real environments. The stability range of ${\alpha }_s$ is ${\alpha }_{sl}<{\alpha }_s<{\alpha }_{su}$, where ${\alpha }_{su}$ is the maximum of ${\alpha }_s$.
• The instability condition occurs if $U_r\cap U_h=\varnothing$ since no intersection appears in both sets, as shown in Figure 5d. In other words, there does not exist a valid control weight such that the iE-Wheelchair can be controlled properly. Such a scenario happens when the wheelchair moves very close to obstacles; therefore, the robot will immediately stop the wheelchair from moving further.

4.2. Algorithm of Deriving the Weight Adaptation

According to the stability analysis of the self-adaptive shared controller, it is required to develop an algorithm that is in charge of computing the proper control weight. The control weights for the user and the robot input (namely, ${\alpha }_h$ and ${\alpha }_r$, respectively) are determined in accordance with the analysis of the convex model. The designed algorithm is detailed as follows:

• Control weight of user-controlled input ${\alpha }_h$: the most stable status of the iE-Wheelchair is set to stationary without any control from the user. That is, when the velocity for moving straightforward $v_h$ and the angular velocity ${\omega }_h$ are zero, ${\alpha }_h$ is assigned as 1. The higher the velocity, the more unsafe the system is. Therefore, as soon as the maximum value of $v_h$ and ${\omega }_h$ are reached, this control weight has to be set as zero. As illustrated in Figure 6a, the control weight is inversely and linearly proportional to the velocity. As a result, ${\alpha }_h$ is defined as follows:

  $${\alpha }_h=1-\frac{\sqrt{{\nu }_h{}^2+{\omega }_h{}^2}}{\sqrt{{\left(v_h{}_{max}\sin {\mathsf{\theta }}_h\right)}^2+{\left({\omega }_h{}_{max}\cos {\mathsf{\theta }}_h\right)}^2}}$$

  (34)

  where

  $${\mathsf{\theta }}_h={\tan }^{-1}\left(\raisebox{1ex}{$v_h$}\!\left/ \!\raisebox{-1ex}{${\omega }_h$}\right.\right)$$

  (35)
• Control weight of the robot control input ${\alpha }_r$: The most stable state occurs when the velocity of moving straightforward $v_r$ and the angular velocity ${\omega }_r$ are zero, which implies that no obstacles appear in the surrounding environment of the iE-Wheelchair. In such a scenario, the weight control of the robot is assigned to be zero. On the other hand, when the maximum values of $v_r$ and ${\omega }_r$ are reached, implying that the obstacle is very closed to the robot, ${\alpha }_r$ has to be 1. Similar to the weight control in the user input, Figure 6b presents the relationship between ${\alpha }_r$ and the velocity as well as the angular velocity, where ${\alpha }_r$ is linearly proportional to ${\omega }_r$ while inversely and linearly proportional to $v_r$, respectively. As a result, ${\alpha }_r$ is defined as follows:

  $${\alpha }_r=\frac{\sqrt{{\left(v_r{}_{max}-{\nu }_r\right)}^2+{\omega }_r{}^2}}{\sqrt{{\left(v_r{}_{max}\sin {\mathsf{\theta }}_r\right)}^2+{\left({\omega }_r{}_{max}\cos {\mathsf{\theta }}_r\right)}^2}}$$

  (36)

  where

  $${\mathsf{\theta }}_r={\tan }^{-1}\frac{v_r{}_{max}-{\nu }_r}{{\omega }_r}$$

  (37)

Algorithm 1 shows the pseudo-code of the proposed algorithm for deriving the weights adaptation. Steps 3 to 11 present the auto-tuning scheme for the weight ${\alpha }_s$, where the dichotomy method is used when the valid input set from the robot is included in the one from the user. As for the case when an unstable condition happens, the shared weight yielded to take the input set from the robot more than the one from the user into account. Note that since there is no sensor in the back of the wheeled chair, our system currently is only allowed to move forward, stop, or turn either left or right.

Algorithm 1: Weight adaptation for the Human and the Robot Inputs

Input: ${\omega }_h$: the rotational velocity of human control. $v_h$: the translational velocity of human control. ${\omega }_r$: the rotational velocity of robot control. $v_r$: the translational velocity of robotic control. $v_{max}=\left[v_h{}_{{}_{\max }},v_{r_{\max }}\right]$: the robot maximum translational velocity. ${\omega }_{max}=\left[{\omega }_h{}_{{}_{\max }},{\omega }_{r_{\max }}\right]$: the robot maximum rotational velocity. ${\alpha }_h$: the weight of human control given by (35). ${\alpha }_r$: the weight of robot control given by (37). Output: $u_s:\left[v_s,{\omega }_s\right]$: $v_s$ and ${\omega }_s$ are the translational and rotational velocity of the robot Optimal ${\alpha }_s$: the weight of shared control. Main: 1.  for each step do 2.    $$\begin{gathered} {\alpha }_h=1-\frac{\sqrt{{\nu }_h{}^2+{\omega }_h{}^2}}{\sqrt{{\left(v_h{}_{max}\sin {\mathsf{\theta }}_h\right)}^2+{\left({\omega }_h{}_{max}\cos {\mathsf{\theta }}_h\right)}^2}} \\ {\mathsf{\theta }}_h={\tan }^{-1}\left(\raisebox{1ex}{$v_h$}\!\left/ \!\raisebox{-1ex}{${\omega }_h$}\right.\right) \\ {\alpha }_r=\frac{\sqrt{{\left(v_r{}_{max}-{\nu }_r\right)}^2+{\omega }_r{}^2}}{\sqrt{{\left(v_r{}_{max}\sin {\mathsf{\theta }}_r\right)}^2+{\left({\omega }_r{}_{max}\cos {\mathsf{\theta }}_r\right)}^2}} \\ {\mathsf{\theta }}_r={\tan }^{-1}\frac{v_r{}_{max}-{\nu }_r}{{\omega }_r} \end{gathered}$$ 3.  if ($v_h<0\left|v_r<0\right.$) then 4.      $v_s\leftarrow 0$; ${\omega }_s\leftarrow 0$; 5.  else if ($v_h<v_r$) & (${\omega }_h<{\omega }_r$) & (${\omega }_h{\omega }_r>0$) then 6.      ${\alpha }_s\leftarrow 0$ 7.  else if ($v_h>v_r$) & (${\omega }_h>{\omega }_r$) & (${\omega }_h{\omega }_r>0$) then 8.      ${\alpha }_s\leftarrow \frac{1+{\alpha }_r}{2}$ 9.  else 10.      ${\alpha }_s\leftarrow \frac{{\alpha }_r}{{\alpha }_h+{\alpha }_r}$ 11.   end if 12.  end for

4.3. Adaptive Shared Controller Design

To provide a dynamic model of our developed iE-Wheelchair, we integrate the motion model with linear stability as mentioned in Section 3, and the algorithm of auto-tuning the control weight based on the convex analysis as mentioned in Section 4.1. With the extension from Equations (2) and (3), the dynamic model with weight adaptation scheme can be developed as follows:

$$I\ddot{\varphi }+C_{\varphi }\dot{\varphi }=\underset{human control inputs}{\underbrace{\left(1-{\alpha }_s\right)\left(I{\ddot{\varphi }}_u\right)}}+\underset{robot control inputs}{\underbrace{{\alpha }_s\left({\displaystyle \sum }_{i=1}^m\frac{{\alpha }_i}{\sqrt[n]{d_{ri}}}-{\displaystyle \sum }_{i=1}^m\frac{{\alpha }_i}{\sqrt[n]{d_{li}}}\right)}}$$

(38)

$$M\dot{v}+C_{v}v=\underset{human control inputs}{\underbrace{\left(1-{\alpha }_s\right)\left(M{\dot{v}}_u\right)}}+\underset{robot control inputs}{\underbrace{{\alpha }_s\left(-{\displaystyle \sum }_{i=1}^m\frac{{\beta }_i}{\sqrt[n]{d_{ri}}}-{\displaystyle \sum }_{i=1}^m\frac{{\beta }_i}{\sqrt[n]{d_{li}}}\right)}}$$

(39)

The block diagram of the proposed self-adaptive shared control system is shown in Figure 7. The reference model takes human and robot inputs to provide expected angular velocity ${\dot{\varphi }}_h$, ${\dot{\varphi }}_r$ and the moving velocity $v_h, v_r$ of human and robot control, respectively. Meanwhile, the control weight α_s in the adaptive shared controller is adjusted based on the proposed algorithm in Algorithm 1. Subsequently, the adaptive shared controller can generate the referenced angular velocity ${\dot{\varphi }}_{cmd}$ and referenced moving velocity $v_{cmd}$ for driving the iE-Wheelchair with obstacle-avoidance capability while incorporating human user control authority.

5. Experiments

To verify the proposed approach, several experimental environments are designed based on various human errors. Additionally, how the parameters are designed in the proposed approach are addressed. The experimental results are verified by several criteria such as smoothness and fluency as well as required completion time.

5.1. Design of Controller Parameters

Table 1. Parameters designed in the proposed iE-Wheelchair system.

M	165 [kg]	I	49.91 [kg∙m2]
ν0	1.67 [m/s]	R	0.55 [m]
Rc	0.75 [m]	r	0.6 [m]
αi	2000	βi	160
Cν	450 [Nm/s]	Cϕ	[Nm/(rad/s)]

shows the parameters used in the proposed system, including the mass M, angular inertial I, the distance between the center of the wheelchair and the laser range finder R, halfwidth of the pavement r, expected velocity v₀, the radius of the curve R_c, and the viscous frictions coefficient of angular velocity $C_{\varphi }$. Notice that $C_{\varphi }$ needs to be carefully designed to ensure the stability criterion as mentioned in (32). As for constant parameters such as α_i, β_i, and viscous frictions coefficient of velocity $C_v$, they are determined to meet the aforementioned stability and performance requirements to the developed iE-Wheelchair system.

There are four different control modes in the experiments: Auto, Manual, Fixed Share, and Adaptive Share. Specifically, except for the adaptive shared control proposed in the conducted experiments, the control weights α_s of the other three control modes are designed as 1, 0, and 0.5, respectively. Detailed descriptions of each mode are provided as follows.

• Auto mode: The whole system is performed autonomously, without any control by the user. Therefore, the iE-Wheelchair only moves forward and makes either left or right turns until the user terminates the system.
• Manual mode: No obstacle avoidance is provided by the system. Only when the iE-Wheelchair approaches the obstacles too closely does the shared control system immediately activate the emergency braking.
• Fixed Shared mode: The value of α_s is set as the constant 0.5 in the experiments.
• Adaptive Shared mode: The values of weights are adaptively adjusted according to the environment.

5.2. Experimental Setup

To investigate the performances of the proposed system in scenarios when human errors occur during the control of the iE-Wheelchair, three experimental environments of human errors are designed based on skill-based error, decision error, and speed error.

• Skill-based error: skill-based error refers to the scenario in which the user is distracting when controlling the iE-Wheelchair. Thus, a delay of the operation control is likely to happen. In our experiments, the time delay is set to be 0.5 s for the manual operation of the user.
• Decision error: the miscarriage of orientation, regarded as the decision error, could be happened when the user operates the wheelchair towards walls or obstacles. Therefore, decision error is represented as the iE-Wheelchair is controlled by the user to approach the wall or obstacles.
• Over-speeding error: if the wheelchair is moving at a high speed, the risk of danger increases while causing injuries both for the user and the robot. Thus, it is essential to concern such a scenario in our experiments.

The environments for the three scenarios can be seen from Figure 8a–c, respectively. In Figure 8a, an S-shape path is designed for the iE-Wheelchair to avoid colliding with obstacles that appeared on the corridor. As such, the delay of manipulations by the user can be investigated. As for Figure 8b, the iE-Wheelchair is approaching the left walls to investigate the decision error from the user. Finally, the over-speed iE-Wheelchair is moving along the inverted L-shape corridor shown in Figure 8c.

5.3. Evaluation Criterion

Based on the criterion of [6], smoothness C_s, fluency C_f, and the required time for completing the experiment, C_t are used in the experiments, where C_s and C_f are, respectively, expressed as follows:

$$C_s={\left(\frac{1}{T}{\displaystyle \sum }_{t=1}^T\frac{\left|\theta \left(t\right)-\theta \left(t-1\right)\right|}{l\left(t-1\right)+1}\right)}^{-1}$$

(40)

$$C_f={\left(\frac{1}{T}{\displaystyle \sum }_{t=1}^T\left|l\left(t\right)-l\left(t-1\right)\right|\right)}^{-1}$$

(41)

where T is the total amount of time steps, l(t) represents the forward distance from t to t + 1, and θ(t) is the orientation at t. If the orientation is frequently changed, the value of C_s is small. Similarly, if the velocity is frequently changed, the value of C_f is also small. Therefore, the larger the values of C_s and C_f with less required time, C_t, to complete the specific task, the better the control system is.

5.4. Test Results and Analysis

(1) Operation of the iE-Wheelchair with Skill-Based Errors: In this experiment, three boxes are arbitrarily placed on the corridor, and the iE-Wheelchair is only allowed to move in an S-shape path to avoid colliding with those boxes. Four different control modes are investigated when the iE-Wheelchair approaches these obstacles. The experimental results using the four control modes are shown in Figure 9a, in which four colors of trajectories correspond to four control modes, respectively. The Auto mode can provide the smooth moving trajectory while the Manual mode does not control the iE-Wheelchair smoothly. This is because the user has to employ his/her experiences and pay more attention to controlling the iE-Wheelchair to avoid any collision, while the shared control with either fixed or adaptive weights allows the iE-Wheelchair to promptly adjust its own orientation before it gets too close to the obstacles. Accordingly, both the shared control modes are superior to the Manual mode in terms of the smoothness of the trajectory. The changes of weights of adaptive shared control mode are shown in Figure 9b. When the iE-Wheelchair approaches the obstacles (or moves into narrow areas), the weight values of Adaptive Shared mode become higher, implying that the driving action is about to be unsafe, and thus the weights are increased to prevent human mistakes.

It can be observed that the values of the resulting weights in the Adaptive Shared mode are higher than 0.5 as in the Fixed Shared mode, because there exists the time delay 0.5 s from human operation. Therefore, it is obvious to distinguish the results conducted by the shared control with adaptive weights and the fixed weights. Moreover, to compare the performances of the four control modes in terms of the criterion used in [6], the performances of the four control modes in terms of the three criteria are shown in Figure 10a–c, respectively. From the observation in Figure 9a, the moving trajectory driven from the Adaptive Shared mode is similar to the Auto mode. Furthermore, we can see that the Adaptive Shared mode performs superior to others in terms of smoothness C_s and fluency C_f. As for the required time for task completion, C_t, it is also similar to the Auto mode. Hence, when a skill-based error occurs in human operations, the proposed Adaptive Shared mode reveals the most favorable mobility assistance.

(2) Operation of the iE-Wheelchair with Decision Errors: If the user miscarriages the orientation during wheelchair turning, the collision to obstacles is more likely to happen. Thus, in this experiment, the iE-Wheelchair is allowed to move towards the side-wall. The experimental results using the four control modes are shown in Figure 11a, and the changes of weights of the adaptive shared control mode are shown in Figure 11b. Auto mode controls the iE-Wheelchair to move on the central trajectory, independent of the user control since it does not consider the user control input. The Manual mode yields the most shaking trajectory, and as the iE-Wheelchair approaches the side-wall, collision is likely to have occurred. On the other hand, the shared control with either fixed or adaptive weights allows the iE-Wheelchair to move smoothly and safely. From the obvious evidence shown in Figure 11b, as the iE-Wheelchair approaches the side-wall, the weights then increase, which implies that the shared controller dominates and is more controlling than the user to prevent any collision to wall or obstacles. After that, when the iE-Wheelchair moves away from the side-wall, the weights decreases to small due to less risk of collision.

To further investigate the performances of the shared control with fixed weights and adaptive weights, we can see from Figure 12a–c, in which the Adaptive Shared mode yields better smoothness C_s and fluency C_f. The required completion time C_t is just slightly worse than the shared control with fixed weights. It is noticeable that the Auto mode in such a case can be regarded as the standard because it does not consider human errors. Thus, concerning both the human control inputs as well as being controlled by the robot inputs, the Adaptive Shared mode reveals the most favorable operation.

(3) Operation of the iE-Wheelchair with Over-Speeding Errors: In this experiment, the over-speeding wheelchair driving is investigated in an inverted L-shape corridor, as shown in Figure 13a, where the trajectories provided by the four modes are presented. We can see that the Auto mode and the shared control (Fixed and Adaptive Shared mode) perform smooth trajectories, while Manual mode provides the worst result. Besides, Figure 13b is similar to Figure 11b where it can be seen that as the iE-Wheelchair approaches obstacles, the weight of shared control increases. As to the performance evaluation of the four modes according to the three criteria, i.e., smoothness C_s, fluency C_f, and completion time C_t, it can be seen from Figure 14a–c that, apart from the Auto mode, the Adaptive Shared mode provides the better results. However, shared control with a fixed weight, on the other hand, cannot cope with the over-speeding scenario because the weight is constant as 0.5. Therefore, it is only better than the Manual mode. When the iE-Wheelchair is over-speeding, shared control with adaptive weight can balance the safety of the system and the errors that occurred in the manual control. By concerning human errors, the Adaptive Shared mode thus performs a favorable result.

Aside from solely the over-speeding, the final experiment further investigates the over-speeding error together with skill-based error (0.5 s time delay for manual operation). The resultant trajectories of the four control modes are shown in Figure 15a, and the changes of weight in the adaptive shared control are presented in Figure 15b. Note that when skill-based error and over-speeding error happens at the same time, it causes high risk for the iE-Wheelchair to collide with obstacles, leading itself to hazardous situations. Thus, the values of weights in the adaptive shared control vary dramatically so that smoothness and safety in controlling performance can be achieved simultaneously, as can be seen from Figure 15b. The performance evaluations of the four control modes in such a scenario can be found in Figure 16a–c. Similar to the over-speeding error experiment, the shared control with fixed weight cannot immediately react to the environments such as moving through the narrow corridor. As a result, it does not perform the same satisfactory results as the Adaptive Shared mode, which provides excellent performances in terms of the yielded three criteria, i.e., smoothness C_s, fluency C_f, and completion time C_t.

According to the above experiments, we can summarize the performances of the four modes as follows:

• Shared control with fixed weights gives better results over adaptive shared control when the iE-Wheelchair is controlled normally.
• However, when considering human errors, the proposed method has shown the effectiveness in terms of improving smoothness, fluency, and completion time, and also reducing user driving load while ensuring the stable and safe operation of the iE-Wheelchair.

6. Conclusions

In this study, an adaptive shared control strategy is proposed and implemented into an intelligent wheelchair system while considering human error protection. To respect the control ability of the user while keeping the stable and safe human–robot interaction, the domination over the wheelchair mobility is adjusted according to the sensory observation of a laser range finder. The safety operation of the shared control system is guaranteed by the stability analysis using the linearized reference model and the Liénard-Chipart criterion. As such, the essential criteria for the design of an obstacle avoidance controller can be developed. Subsequently, based on convex analysis and set theory, the stability and performance of the adaptive shared controller can be analyzed to find the adaptive weight for suitable human–robot cooperation in real-time navigation. To verify the proposed system, three sets of human error experiments are used, including skill-based error, decision error, and speed error. In terms of the three criteria including smoothness, fluency, and required completion time of specific tasks, the experimental results show that the proposed iE-Wheelchair not only effectively guarantees the safe operation for the user, but also provides stable control behavior with smoothness and fluency.

Finally, it is worth mentioning that the distance measurements play important roles to determine the shared controlling effectiveness of our proposed approach. The distance information can also be provided from other distance measuring sensors such as ultrasonic sensors. However, to provide the same detection range in a horizontal field, we may need several ultrasonic sensors located around the iE-Wheelchair. The different effect to our approach while considering distance sensor characteristics can be left for future works.