Detecting and Controlling Slip through Estimation and Control of the Sliding Velocity

Abstract

Slipping detection and avoidance are key issues in dexterous robotic manipulation. The capability of robots to grasp and manipulate objects of common use can be greatly enhanced by endowing these robots with force/tactile sensors on their fingertips. Object slipping can be caused by both tangential and torsional loads when the grip force is too low. Contact force and moment measurements are required to counteract such loads and avoid slippage by controlling the grip force. In this paper, we use the SUNTouch force/tactile sensor, which provides the robotic control system with reliable measurements of both normal and tangential contact force components together with the torsional moment. By exploiting the limit surface concept and the LuGre friction model, we build a model of the object/fingertip planar sliding. This model is the basis of a nonlinear observer that estimates the sliding velocity and the friction state variable from the measured contact force and torsional moment. The slipping control system uses the estimated friction state to detect the slipping event and the estimated sliding velocity to control the grasp force. The control modality is twofold: the first one is aimed at avoiding object slip, while the second one allows the object to perform a controlled pivoting about the grasping axis. Experiments show that the robot is able to safely manipulate objects that require grasping forces in a large range, from $0.2$ N to $10$ N. This level of manipulation autonomy is attained by a suitably identified dynamic model that overcomes the limited generalization capability of existing learning-based approaches in the general roto-translational slip control.

1. Introduction

Humans can grasp and manipulate a variety of objects with dexterity and safety without knowing a priori their physical properties by perceiving the motion of the object upon contact with the fingertips by means of the sense of touch [1]. On the basis of tactile perception, they control the magnitude of the friction force and torque by acting on the grasp force, which the fingertips exert on the object, thereby avoiding grasp failure. Thus, tactile sensing is of paramount importance for humans to interact with the environment. In the last two decades, many designs of tactile sensors have been proposed in the scientific literature by robotics researchers with the aim of mimicking the human sense of touch. Comprehensive reviews of tactile sensing systems for dexterous robotic hands can be found in [2,3]. The proposed solutions are based on two alternative concepts: sensors based on an array of independent contacts or tactile receptors that easily allow for the identification of contact location and sensors based on a continuous deformable medium between the contact and the receptors, which allows a few receptors to interpolate the information among them. The latter are more frequently used to measure contact force intensity or magnitude. Both sensor principles may be supported by different transduction technologies, e.g., resistive, capacitive and magnetic, which can be selected depending on their costs, weights, integration levels, and dimensions. The perception of the torsional moment is a key requirement to select the appropriate tactile sensor to securely grasp an object when it is subject not only to force but also to moment loads. Only a few sensors are capable of measuring the full 6-D wrench; moreover, sensors that are also able to measure torque, e.g., the 6-axis version by Optoforce, use an array of 3-D sensors coupled with a rigid surface that limits the frictional torque that the fingertip can apply to the grasped object. Recently, the GelSight visuotactile sensor, which uses a vision system to translate the deformation of the gel layer caused by the contact with the environment into tactile information [4,5] has been proposed to estimate shear force and contact geometry. A review of visuotactile sensors with an emphasis on GelSight can be found in [6]. In this paper, we use the SUNTouch force/tactile sensor [7,8] developed in our robotics lab. The SUNTouch force/tactile sensor has been demonstrated to be able to provide the grasp controller with accurate measurements of the 6-D contact wrench [9].

Mimicking human ability, slip detection, slipping avoidance and controlled sliding are key facets of making robots capable of grasping and manipulating objects. This means that the robot should be able to detect the incipient grasp failure and control the grasp force to avoid it without damaging the grasped object, or, to change the object/fingertip relative position, to execute controlled sliding of the fingertips without losing contact with the object. Several methods of slip detection have been proposed in the last decade [10]. The approaches can be classified as model-based and model-free approaches. In [11], using the BioTac sensor and resorting to the friction cone concept, the authors compared various slip detection methods. However, slipping detection algorithms based on the friction cone concept fail when a torsional load acts on the grasped object. In [9], the authors presented a model-based approach to slipping detection and avoidance in the presence of both shear and torsional load by resorting to the concept of the limit surface [12]. Model-free approaches, the prevailing trend in recent years, refer to learning-based slip detection methods, mostly based on the use of visuotactile sensors such as GelSight and TacTip. In [13,14], the authors proposed a slip classification framework by resorting to a convolutional long short-term memory network. In [15], the authors, by endowing a three-fingered hand with the TacTip integrated optical tactile sensor, present a method to perform slip detection based on a support-vector machine (SVM).

The decision to adopt a model-based approach here is related to the difficulty of current data-driven methods for robotic slipping detection and avoidance to generalize to manipulated objects made of different types of materials and surface conditions. A re-training of the learning-based strategy is still required for such a class of solutions, especially when surface conditions change due to environmental variability, e.g., moisture or temperature. On the contrary, the superior generalization capabilities of the model-based approach appear to be still attractive to attain a high level of autonomy of the robotic system. Moreover, in all the model-free approaches, after detecting a slip, the reaction designed to stabilize the grasp is of a heuristic type, which means increasing the grasp force until the object stops sliding; the selection of an increasing force of the grasp force should be a trade-off between the desired speed to stop slipping and the caution needed to not damage the object. Differently from the method used in [16] to compute the grasp force to avoid slip, here, the control algorithm, owing to the nonlinear observer, distinguishes the static friction contribution from the viscous one, thus leading to the true minimum amount of grasp force necessary to compensate for the external load and avoid slippage. A recent work proposing this sort of approach is [17], where the slip event was predicted by a classifier based on random forests and the proposed grasp force controller does not compensate for rotational slips.

In this paper, we do not follow the learning-based prevailing trend, but following the approach already presented in [9,16], we present a revisited model-based approach to slip detection/avoidance and controlled sliding. General in-hand manipulation tasks involve both translational and rotational motion; thus, it is necessary to model the relationship between the friction force and torque and the sliding motion. The approach adopted to describe such a relationship, assuming that the contact is planar, is the limit surface concept (LS) [12,18], which relates the sliding motion, described as a rotational motion about the instantaneous center of rotation (CoR), to the maximum friction force and torque as functions of the normal force. The LS concept is a generalization of the Coulomb friction cone concept when friction torsional torque is involved in the soft contact. In order to perceive the object’s motion upon contact with the fingertips using only a force/tactile sensor, it is necessary to estimate the relative velocity between the object and the fingertips. In [19], we introduced a nonlinear observer based on the celebrated LuGre dynamic friction model [20,21] extended to the case of rotational sliding about the CoR. Such a model describes the friction as a pure friction torque with respect to the CoR. The maximum friction torque is provided by the limit surface method. The dynamic model requires the knowledge of the CoR position estimated by using the measurement of the dry friction provided by the force/tactile sensor at the fingertips. The nonlinear observer in [19] has been used in [22] to estimate the sliding velocity with the aim of avoiding object slipping and in [16], where a generalization of the LS method has been proposed by considering not only the effects of the dry friction but also the effects of the viscous one. An additional advantage of the adoption of nonlinear dynamics to model the transition from a no-slip to a slip condition is its numerical stability compared to traditional threshold-based approaches also adopted in tribology, such as in [23].

With the objective of controlling a robot that should be able to grasp and manipulate objects of unknown weight and made of different materials, in this paper, we propose a revisited version of the nonlinear observer to estimate the relative velocity, which also considers the viscous damping depending on the motion of the micro-asperities in contact during the sliding. Introducing this viscous term avoids physically inconsistent oscillations of the internal friction state variable. Although such oscillations do not macroscopically affect the sliding velocity dynamics, as the dynamics of the friction state variable are very fast, they can affect the controlled grasp force if this variable is involved in the control algorithm. We use the friction state variable estimated by the nonlinear observer to detect the slip by means of an indicator, which involves such a state variable and the maximum friction torque provided by the LS with the estimated COR. Then, we design a control scheme, which regulates the estimated sliding velocity to zero if the slipping avoidance modality is selected; alternatively, it allows controlled sliding, which can be rotational, e.g., object pivoting, or linear depending on the estimated COR position. Several experiments will be illustrated in the experiment section to show the effectiveness of the proposed planar sliding model and the proposed grasp control strategy.

2. Contact Modeling

In this paper, the manipulated object is modeled as a planar slider. With reference to Figure 1, a planar slider is a rigid body that can rotate and translate in a 2-D space. The robotic finger is modeled as a hemispherical soft pad that makes contact with the slider with a non-zero contact area. This way, the fingertip exchanges both friction forces and torques with the slider. The friction can be indirectly controlled by acting on the normal force that the fingertip exerts on the body, which in turn changes the maximum friction force and torque that the contact can sustain.

Any 2-D roto-translational motion can be instantaneously described as a pure rotation about the CoR. Based on our previous friction modeling [19] and inspired by the revisited LuGre friction model [21], in this paper, we propose a novel planar slider dynamic model that considers the micro-damping between the micro-asperities in contact [21]. The slider motion about the CoR is modeled as

$$\begin{array}{cc}\hfill \dot{z}& =\omega -\frac{{\sigma }_0}{g(f_n,c)}z\left|w\right|\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill J\dot{\omega }& =-{\sigma }_1(f_n,c)\omega -{\sigma }_{0}z-{\sigma }_z\dot{z}+{\tau }_e,\hfill \end{array}$$

(2)

where ${\tau }_e$ is the external torque acting on the slider, $\omega$ is the slider’s rotational velocity about the CoR, z is the friction state variable, which represents the deflection of the micro asperities, ${\sigma }_0$ and ${\sigma }_z$ are the so-called stiffness and damping of the micro asperities, respectively, J is the inertia moment of the slider about the CoR; $g(f_n,c)$ is the maximum rotational friction torque about the CoR depending on the normal force $f_n$ and the CoR position c, and finally, ${\sigma }_{0}z$ and ${\sigma }_1(f_n,c)\omega$ are the dry and viscous friction torques, respectively. The relevant difference of the model in (1) and (2) with respect to the classical revisited LuGre friction model is the extension to the roto-translational case. This generalization is achieved by incorporating the LS method into the model and, specifically, in the computation of the maximum friction torque $g(f_n,c)$, which requires the estimation of the instantaneous CoR.

The only available control input is the normal force $f_n$ that is nonlinearly related to the maximum dry friction and the viscous one. The greater the value of $f_n$, the greater the fingertip deformation (and the contact area), and the greater the available friction. This relationship can be found by resorting to the limit surface concept (LS) [18]. The LS is a closed surface defined in the space of the friction force and torque (Figure 2) and represents the maximum dry friction that a contact can sustain without sliding. With reference to Figure 1, we define a contact frame located in the center of pressure (CoP) of the contact area, with the y-axis along the direction of the tangential force $f_t$ and the z-axis normal to the contact surface and aligned to the normal force $f_n$. It is well known that, assuming an axisymmetric pressure distribution [24], the limit surface is axisymmetric as well, and we can represent it using any radial cross-section. Moreover, the CoR position is always orthogonal to the tangential force, and, thus, it lies on the x-axis [18]. Consequently, it is possible to represent the CoR position with the scalar c. Finally, taking into account the load-motion inequality [12], the friction torque $\tau$ and c have opposite signs.

Given one particular CoR position, it is possible to generate a point on the LS. Two values are of paramount importance: the point corresponding to a pure translation ($c\to \infty$) and a pure rotation ($c=0$), which represent the maximum possible values for the translational and torsional friction, respectively [24], i.e.,

$$\begin{array}{cc}\hfill {f_t}_{max}& =\mu f_n\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill {\tau }_{max}& =\mu \xi \delta f_n^{\gamma +1},\hfill \end{array}$$

(4)

where $\mu$ is the Coulomb friction coefficient, $\xi$ is a parameter that weakly depends on the particular pressure distribution and varies between $\frac{3\pi }{16}$ (Hertzian) and $\frac{2}{3}$ (uniform) [18], and finally, $\delta$ and $\gamma$ are additional parameters that relate the radius $\rho$ of the contact area to the normal force [24] according to the relationship $\rho =\delta f_n^{\gamma }$.

The point on the LS (${f_t}_{\mathrm{LS}}$, ${\tau }_{LS}$) can be computed as a function of the CoR position c [16]

$$\begin{array}{cc}\hfill {f_t}_{\mathrm{LS}}& =-sign\left(\omega \right){\tilde{f}}_{t\mathrm{LS}}^{*}\left(\tilde{c}\right){f_t}_{max}\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill {\tau }_{LS}& =-sign\left(\omega \right){\tilde{\tau }}_{\mathrm{LS}}^{*}\left(\tilde{c}\right){\tau }_{max}\hfill \end{array}$$

(6)

where $\tilde{c}=\frac{c}{\rho }$ is the normalized CoR position, and ${\tilde{f}}_{t\mathrm{LS}}^{*}\left(\tilde{c}\right)$ and ${\tilde{\tau }}_{\mathrm{LS}}^{*}\left(\tilde{c}\right)$ are known functions whose shape weakly depends on the parameter $\xi$ [16] and are shown in Figure 3. Given their very weak variation with respect to this parameter, in the following, we will consider the intermediate value $\xi =0.6354$ already experimentally identified in [16].

The LS theory was originally conceived as an extension of the Coulomb dry friction for the rototranslational case. During the slippage, viscous friction should also be taken into account to describe the motion. Such a component is modeled by the term ${\sigma }_1(f_n,c)\omega$ in (2). In our previous work [16], we extended the limit surface theory to also take into account the viscous friction, which corresponds to points in the force/torque space outside the LS (Figure 2). Thus, the viscous friction as a function of the CoR position can be expressed as

$$\begin{array}{cc}\hfill {f_t}_v& =\pi {\beta }_A{\delta }^2{f}_n^{2\gamma }\omega c\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill {\tau }_v& =-\frac{\pi }{2}{\beta }_A{\delta }^4{f}_n^{4\gamma }\omega .\hfill \end{array}$$

(8)

The dynamic model (1) and (2) describes the motion as a pure rotation about the CoR axis, and thus, it is important to estimate the CoR position. The LS theory is typically used to solve the forward problem given the CoR position (i.e., the instantaneous motion) by computing the friction force and torque. We are interested in the inverse problem, i.e., estimating the CoR position c given the measured friction force and torque. Moreover, when there is no motion ($\omega =0$), the CoR is meaningless, but the dynamic model also needs the CoR to be defined in this case. To solve this issue, it is possible to define a virtual CoR [16], i.e., when the velocity is zero, the virtual CoR is the CoR that would result if we lowered the normal force down to the point that a sliding motion occurs. In the following, we will use the acronym CoR to refer to the CoR and the virtual CoR indistinctly.

To estimate the CoR from the measurements, it is useful to represent the LS in a space normalized with respect to the maximum friction force and torque (3) and (4). Applying such normalization, the complete expression of the normalized friction force ${\tilde{f}}_t$ and torque $\tilde{\tau }$ (i.e., the superposition of the dry (5) and (6) and viscous (7) and (8) components) can be written as

$$\begin{array}{cc}\hfill {\tilde{f}}_t& =-sign\left(\omega \right){\tilde{f}}_{t\mathrm{LS}}^{*}\left(\tilde{c}\right)+\frac{\pi {\delta }^3{\beta }_A{f}_n^{3\gamma -1}}{\mu }\omega \tilde{c}\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill \tilde{\tau }& =-sign\left(\omega \right){\tilde{\tau }}_{\mathrm{LS}}^{*}\left(\tilde{c}\right)-\frac{1}{2\xi }\frac{\pi {\delta }^3{\beta }_A{f}_n^{3\gamma -1}}{\mu }\omega .\hfill \end{array}$$

(10)

Such expressions relate the friction forces and torques with the CoR position c only outside and on the LS, but not inside it. To also define the CoR when the forces are inside the LS, we have to resort to the virtual CoR definition and consider what happens when the normal force varies. Inside the LS, there is no viscous friction; hence, the friction force and torque are simply given by the normalization formula

$$\begin{array}{c}\hfill {\tilde{f}}_t=\frac{f_t}{{f_t}_{max}}\end{array}$$

(11)

$$\begin{array}{c}\hfill \tilde{\tau }=\frac{\tau }{{\tau }_{max}},\end{array}$$

(12)

where the dependence on the normal force is in the definition of the maximum friction force and torque ${f_t}_{max}$ (3) and ${\tau }_{max}$ (4). The (virtual) CoR in such a case can be computed by considering that the sliding happens when $f_n$ is such that the point (${\tilde{f}}_t$, $\tilde{\tau }$) coincides with the normalized LS point (${\tilde{f}}_{t\mathrm{LS}}$, ${\tilde{\tau }}_{LS}$).

Finally, by combining (11) and (12) when the forces are inside the LS and (9) and (10) when they are outside it, the CoR position c can be estimated with the following algorithm:

$$\begin{array}{c}\tilde{c}:\begin{array}{cc}\hfill & \left|\sigma \right|=\frac{|{\tilde{f}}_{t\mathrm{LS}}^{*}\left(\tilde{c}\right){|}^{\gamma +1}}{{\tilde{\tau }}_{\mathrm{LS}}^{*}\left(\tilde{c}\right)}\hfill \\ & \mathrm{s}.\mathrm{t}.sign\left(\tilde{c}\right)=-sign\left(f_t\right)sign\left(\tau \right)\hfill \end{array},\begin{array}{cc}\hfill & if(f_t,\tau )\in {\mathcal{V}}_L\hfill \\ & \mathrm{and}\tau \ne 0\hfill \end{array}\\ \tilde{c}:\begin{array}{cc}{\tilde{f}}_t=\hfill & sign\left(\tau \right){\tilde{f}}_{t\mathrm{LS}}^{*}\left(\tilde{c}\right)+\hfill \\ & -2\xi \tilde{c}(\begin{array}{c}\hfill \tau \end{array}-sign\left(\tau \right){\tilde{\tau }}_{\mathrm{LS}}^{*}\left(\tilde{c}\right))\hfill \end{array},\begin{array}{cc}\hfill & if(f_t,\tau )\notin {\mathcal{V}}_L\hfill \\ & \mathrm{and}\tau \ne 0\hfill \end{array}\end{array}$$

(13)

where

$$\sigma =\frac{\xi \delta }{{\mu }^{\gamma }}\frac{|f_t{|}^{\gamma +1}}{\tau },$$

(14)

and ${\mathcal{V}}_L$ is the limit volume, i.e., the set of points inside the LS. The conditions in (13) can be easily checked by resorting to the convexity of the LS, i.e.,

$$(f_t,\tau )\in {\mathcal{V}}_L\iff {\tilde{f}}_t^2+{\tilde{\tau }}^2<{\tilde{f}}_{t\mathrm{LS}}^{*}{\left(\tilde{c}\right)}^2+{\tilde{\tau }}_{\mathrm{LS}}^{*}{\left(\tilde{c}\right)}^2.$$

(15)

Given the CoR position, it is possible to write the expression of the torques about the CoR axes needed by the dynamic system (1) and (2). The friction torque about the CoR axis ${\tau }_c$ can be obtained by transforming the wrench in the contact frame to a pure torque about the CoR axis as

$${\tau }_c=\tau -c{f}_t.$$

(16)

By following the same arguments, the viscous friction torque in the dynamic equation can be computed with the same transformation, i.e.,

$$\begin{array}{cc}\hfill -{\sigma }_1(f_n,c)\omega & ={{\tau }_c}_v={\tau }_v-c{f_t}_v\hfill \\ & =-\pi {\rho }^4{\beta }_A\left({\tilde{c}}^2+\frac{1}{2}\right)\omega ,\hfill \end{array}$$

(17)

from which the expression of ${\sigma }_1$ is straightforward. The maximum rotational friction $g(f_n,c)$ can be computed in the same way. It is the LS point (${f_t}_{\mathrm{LS}}$, ${\tau }_{LS}$) transformed as a pure torque about the CoR axis. It is important to underline that $g(f_n,c)$ is always positive; it is the magnitude of the maximum friction, and ${f_t}_{\mathrm{LS}}$ and ${\tau }_{LS}$ independently contribute to $g(f_n,c)$, i.e.,

$$\begin{array}{cc}\hfill g(f_n,c)& =|{\tau }_{LS}|+|c{f_t}_{\mathrm{LS}}|\hfill \\ & ={\tilde{\tau }}_{\mathrm{LS}}^{*}\left(\tilde{c}\right){\tau }_{max}\left(f_n\right)+\left|c{\tilde{f}}_{t\mathrm{LS}}^{*}\left(\tilde{c}\right)\right|{f_t}_{max}\left(f_n\right).\hfill \end{array}$$

(18)

where ${\tau }_{max}$ and ${f_t}_{max}$ are the same variables from (3) and (4) explicitly written as functions of the normal load $f_n$.

3. Velocity Observer

The control law proposed in Section 5 needs the state of the system (1) and (2), which is not directly measurable. We need to estimate the state and, in particular, the sliding velocity of the planar slider by resorting to only force and torque measurement at the fingertips. To this aim, in addition to the state Equations (1) and (2), we also consider the following output equation:

$$y=h(z,\omega )={\sigma }_{0}z+{\sigma }_1(f_n,c)\omega ,$$

(19)

which is the superposition of the dry and viscous friction torque about the CoR axis. The measurable output y can be obtained by the measured force and torque similarly to (16) as

$$y=\tau -c{f}_t$$

(20)

where $\tau$ and $f_t$ are the components of the measured wrench at the fingertip.

It can be proved that the system (1), (2), and (19) is locally weakly observable [25], and, thus, it is possible to use the following nonlinear observer to estimate the velocity $\omega$:

$$\begin{array}{cc}\hfill \dot{\widehat{z}}& =\widehat{\omega }-\frac{{\sigma }_0}{g}\widehat{z}\left|\widehat{\omega }\right|\hfill \end{array}$$

(21)

$$\begin{array}{cc}\hfill \dot{\widehat{\omega }}& =l\left(-{\sigma }_0\widehat{z}-{\sigma }_1\widehat{\omega }-{\sigma }_z\dot{\widehat{z}}+y\right),l>0\hfill \end{array}$$

(22)

$$\begin{array}{cc}\hfill \widehat{y}& ={\sigma }_0\widehat{z}+{\sigma }_1{\widehat{\omega }}_z.\hfill \end{array}$$

(23)

The observer structure is the same as the original dynamic system (1)–(2) with the measured friction y playing the same role of the external torque ${\tau }_e$. The inertia moment is substituted by the observer gain l; the higher the gain, the faster the observer convergence. The local observability is proved by the following proposition.

Proposition 1 

(Observability). Let $\mathcal{M}=\left\{(z,\omega )\in {\mathbb{R}}^2:\omega >0\right\}$. Then, the system (1), (2), and (19) is locally weakly observable [25] at any initial state $\left(z\right(0),\omega (0\left)\right)\in \mathcal{M}$. The same result holds when considering the domain ${\mathcal{M}}^{\prime }=\left\{(z,\omega )\in {\mathbb{R}}^2:\omega <0\right\}$.

Proof. 

The nonlinear system in (1), (2), and (19) can be written in the more compact form

$$\begin{array}{cc}\hfill \dot{\mathit{x}}& =\mathit{f}(\mathit{x},{\tau }_e)\hfill \end{array}$$

(24)

$$\begin{array}{cc}\hfill y& =\mathit{h}\left(\mathit{x}\right)=\mathit{H}\mathit{x},\hfill \end{array}$$

(25)

where $\mathit{x}={[z,\omega ]}^{\top }$ is the state vector, $\mathit{f}(\mathit{x},\tau )$ is a vector function whose components are the second members of the state-space Equations (1) and (2), and the output matrix is

$$\mathit{H}=\left[\begin{array}{cc}{\sigma }_0& {\sigma }_1\end{array}\right]$$

(26)

According to [26], the local observability of the system (24) and (25) is ensured at any initial state $\left(z\right(0),\omega (0\left)\right)\in \mathcal{M}$ if the following matrix has rank 2 $\forall (z,\omega )\in \mathcal{M}$:

$$\mathsf{\Theta }\left(\mathit{x}\right)=\left[\begin{array}{c}\mathrm{d}\mathit{h}\\ {\mathcal{L}}_{\mathbf{f}}^1\mathrm{d}\mathit{h}\end{array}\right],$$

(27)

where $\mathrm{d}\mathit{h}$ represents the Jacobian of the vector function $\mathit{h}$, and ${\mathcal{L}}_{\mathbf{f}}^1$ represents the Lie derivative operator along $\mathbf{f}$ of order 1.

For the system (24) and (25), $\mathsf{\Theta }\left(\mathit{x}\right)$ can be written as

$$\left[\begin{array}{c}{\sigma }_0\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}{\sigma }_1\\ {\sigma }_0{\sigma }_1-\frac{{\sigma }_0{\sigma }_1}{J}-\frac{{\sigma }_0}{g}({\sigma }_0-\frac{{\sigma }_1{\sigma }_z}{J})\left|\omega \right|{\sigma }_0-\frac{{\sigma }_1({\sigma }_1+{\sigma }_z)}{J}-\frac{{\sigma }_0}{g}({\sigma }_0-\frac{{\sigma }_1{\sigma }_z}{J})zsign\left(\omega \right)\end{array}\right]$$

(28)

with $sign(·)$ being a version of the signum function regularized around zero, which means considering an absolute value function regularized around zero as well to make it differentiable. The rank of the matrix above would be lower than 2 if and only if

$$1-\frac{{\sigma }_1{\sigma }_z}{{\sigma }_{0}J}-\frac{{\sigma }_0}{g}\left(1-\frac{{\sigma }_1{\sigma }_z}{{\sigma }_{0}J}\right)zsign\left(\omega \right)+\frac{{\sigma }_1}{g}\left(1-\frac{{\sigma }_1{\sigma }_z}{{\sigma }_{0}J}\right)\left|\omega \right|=0.$$

(29)

This equation has no solutions, nor does it in $\mathcal{M}$ and in ${\mathcal{M}}^{\prime }$. In fact, for $(z,\omega )\in \mathcal{M}$, it is equivalent to

$$z=\frac{{\sigma }_1}{{\sigma }_0}\omega +\frac{g}{{\sigma }_0}$$

(30)

which has no solutions because the LuGre friction model ensures that $\left|z\right|\le g/{\sigma }_0$ [21]. The same result holds for $\omega <0$ in the domain ${\mathcal{M}}^{\prime }$, yielding to the non-solvable equation

$$z=\frac{{\sigma }_1}{{\sigma }_0}\omega -\frac{g}{{\sigma }_0}.$$

(31)

□

4. Data Collection and Parameter Estimation

The sliding velocity observer relies on the knowledge of the system physical parameters, which needs to be estimated. In particular, friction coefficients such as $\mu$ and ${\beta }_A$ are of paramount importance to detect the slip and estimate the slip velocity. For this reason, in this work, we conducted an extensive data collection campaign to validate the ability of the observer to estimate the sliding velocity.

The experimental setup used to this aim is depicted in Figure 4. The SUNTouch finger was mounted on a reference 6-axis force/torque sensor, the ATI Nano43, by ATI Industrial Automation, USA, which was able to accurately measure the friction forces exchanged between the fingertip and the slider. The planar slider was a flat object rigidly mounted on a Meca500 robot by Mecademic Robotics, CA. The robot was commanded to push the slider against the fingertip and apply various combinations of forces and torques, resulting in sliding motions. The ground-truth velocity $\omega$ was measured as the robot end-effector velocity obtained via joint encoders and forward differential kinematics. The data were collected with a ROS network and MATLAB.

A cardboard layer was attached to the slider. If the slider was rigid, the deformation of the fingertip depended only on the soft fingertip material and not on the slider [24]; thus, the only parameters that depended on the slider were the friction parameters $\mu$ and ${\beta }_A$. The remaining parameters can be estimated once and also used for other rigid planar sliders.

The parameter estimation problem can be formalized as the following optimization algorithm:

$$\begin{array}{cc}\hfill & \underset{\mathcal{P}}{min}\frac{1}{N}\sum _{k=1}^N{({\widehat{\omega }}_k-{\omega }_k)}^2\hfill \end{array}$$

(32)

$$\begin{array}{cc}\hfill & \mathrm{s}.\mathrm{t}.\left[\begin{array}{c}{\widehat{z}}_k\\ {\widehat{\omega }}_k\end{array}\right]={\mathbf{f}}_d({\widehat{\omega }}_{k-1},y_{k-1};\mathcal{P})\hfill \end{array}$$

(33)

where ${\mathit{f}}_d$ is a discrete-time version of the observer (21) and (22), k is the sample-time instant, and N is the total number of samples. The vector $\mathcal{P}$ contains the model parameters to be identified, i.e., $\mu$, ${\beta }_A$, ${\sigma }_1$, ${\sigma }_z$, $\delta$, and $\gamma$. The time discretization of the observer equations is not trivial. The dynamics of the internal friction state variable z is very fast, yielding a very stiff equation. This means that simple numerical methods for solving the equation are numerically unstable unless the step size is extremely small. We found that, to discretize the observer, it was not possible to adopt the classical Euler discretization, thus, we adopted the fourth-order Runge–Kutta method (RK4).

The results of the parameter estimation problem (32) are reported in Figure 5, and the corresponding parameters are in

Table 1. Model parameters estimated on a rigid cardboard slider.

$\mathit{\mu }$	${\mathit{\beta }}_{\mathit{A}}$	${\mathit{\sigma }}_0$	${\mathit{\sigma }}_{\mathit{z}}$	$\mathit{\delta }$	${\gamma }$
$0.47$	$4.9·{10}^9$ Ns/m	$16.66$ Nm/rad	2 Nms/rad	$0.0018$ m/N ${}^{\gamma }$	$0.254$

. The figure shows 1342 sliding maneuvers applied with the setup in Figure 4; the robot pushes the slider against the fingertip with increasing forces and torques, resulting in increasing sliding velocity. Finally, the ground-truth sliding velocity $\omega$, together with the tangential force $f_t$ and torque $\tau$ measured by the F/T sensor, were used as input to the optimization problem (32) solved with the sequential quadratic programming (SQP) algorithm. To facilitate the convergence, it was useful to bound the parameter space. Some parameters could be theoretically bounded (e.g., $0<\gamma <1/3$ [24]), and others could be roughly estimated with a single sliding maneuver. Thus, it was also possible to provide a reasonable initial guess for the optimization algorithm. The resulting parameters (Table 1) yielded the estimated velocity $\widehat{\omega }$ in Figure 5 (red line). The results clearly show that the velocity observer is able to correctly estimate the ground-truth sliding velocity and catches all the sliding events with a root-mean-square error (RMSE) of $0.0222$ rad/s.

5. Sliding Controller

This section presents a novel slipping control strategy based on the observer presented in the previous section. The basic idea is similar to the one presented in our previous work [16], but with a substantial difference: the control law involves the full state feedback estimated by the observer.

The control scheme works in two possible control modes, “slipping avoidance” and “pivoting”. With reference to Figure 6a, in slipping avoidance mode, the objective of the sliding controller is to regulate the estimated slipping velocity $\widehat{\omega }$ to zero by modulating the grasping force $f_n$. The pivoting mode allows an in-hand manipulation; it consists of letting a manipulated object rotate in hand subject to the gravity so as to change the relative orientation between the gripper and the object. In this modality, the controller allows rotational sliding between the fingertip and the slider. The control strategies for the two control modalities are very similar. We will describe the slipping avoidance mode first, in which all the control components are active.

The control law is composed of two components (Figure 6b):

$$f_n=f_{ns}+f_{nd}.$$

(34)

where $f_{ns}$ is the static contribution and represents the normal force that is able to avoid the slippage by bringing the maximum torsional friction around the CoR $g(f_n,c)$ above the actual dry friction ${\sigma }_{0}z$ in (1) and (2). It is computed by exclusively using the estimated dry friction and not the viscous friction, yielding lower grasping forces with respect to the strategy in [16]. The static contribution is able to avoid slippage only in static or quasi-static conditions. In fact, it is well-known that when the load is time-varying, the maximum friction that the contact between the slider and the fingertip can sustain decreases as the rate of variation of the load increases [27]. Thus, the dynamic contribution $f_{nd}$ is introduced, and it is synthesized by means of the estimated velocity $\widehat{\omega }$ as

$$f_{nd}=\left|C_d\widehat{\omega }\right|,$$

(35)

where $C_d$ is a suitable linear differential operator, and the absolute value operator ensures that $f_{nd}>0$ both for positive and negative slipping velocities. $C_d$ can be represented with a transfer function

$$C_d=k_{cd}\frac{s+z_{cd}}{s+p_{cd}},$$

(36)

where the real zero and pole ($-z_{cd}$ and $-p_{cd}$, respectively) were selected to reduce the high-frequency gain ($z_{cd}>p_{cd}>0$) so as to reduce the control sensitivity to the high-frequency noise. The gain $k_{cd}>0$ was selected to obtain a quick reaction to any relative velocity $\widehat{\omega }$. The parameters of the controller were tuned to the values $k_{cd}=10$, $p_{cd}=7$, $z_{cd}=250$.

The static contribution was designed by resorting to the feedback of the estimated friction state variable $\widehat{z}$. The main idea was the definition of the following slippage indicator

$$\mathcal{S}=\frac{{\sigma }_0\left|z\right|}{g(f_n,c)}.$$

(37)

It is well-known that the LuGre state z is bounded by the maximum friction [21], and it always holds that ${\sigma }_0\left|z\right|\le g(f_n,c)$. In particular, when ${\sigma }_0\left|z\right|=g(f_n,c)$, the dry friction has reached its maximum, and slippage can take place. This implies that the slippage indicator $\mathcal{S}\le 1$, and, when $\mathcal{S}=1$, the slider can slip with respect to the finger. Moreover, note that the greater the grasp force $f_n$, the greater the maximum friction $g(f_n,c)$; thus, $\mathcal{S}$ decreases as $f_n$ increases. Given these considerations, the objective of the static contribution $f_{ns}$ is to regulate the slippage indicator $\mathcal{S}$ to the value ${\overline{\mathcal{S}}}_a<1$ designed, taking into account the desired degree of slipping avoidance robustness. The lower the value of ${\overline{\mathcal{S}}}_a$, the greater the resulting grasp force. In the experiment section, we will select ${\overline{\mathcal{S}}}_a=0.8$. The regulation of $\overline{\mathcal{S}}$ is carried out by a classical PI controller with the proportional gain $k_p=4$ and the integral gain $k_i=10$ tuned so as to obtain the fastest possible response time with the hardware used in the experiments (Section 6).

In pivoting mode, which is the controlled sliding maneuver considered here, the control scheme is almost the same as the slipping avoidance mode, as depicted in Figure 6b. This time, the objective is to allow slippage; $\mathcal{S}$ has to be regulated to 1, and the grasping force has to be continuously decreased as long as the sliding motion is complete. To this aim, it is sufficient to activate the switches in Figure 6b. The dynamic contribution is deactivated ($f_{nd}=0$) because the sliding velocity does not have to be controlled to zero, and the reference slippage indicator is set to the value $\overline{\mathcal{S}}={\overline{\mathcal{S}}}_p>1$. Additionally, the static control law is switched to a first-order filter with a time constant ${\tau }_p$. This way, the grasp force will decrease down to the point for which $\mathcal{S}=1$ reaches its maximum; then, the constant error ${\overline{\mathcal{S}}}_p-1$ will cause an exponential decrease in the static contribution $f_{ns}$ governed by the time constant ${\tau }_p$. As soon as the pivoting motion has been completed, the slipping avoidance mode is reactivated.

At the beginning of the pivoting, the object is grasped far from its center of gravity (CoG), and the friction sustains the gravitational torque. Decreasing the grasp force would cause the object to rotate about the grasp axis in a pendulum-like motion until its equilibrium point, where the gravitational torque is zero and the object is in a “vertical” configuration (with the CoG just below the grasp axis). In the beginning, the object is more prone to rotationally slide subject to the torque; at the end, the object is more prone to translate. These two phases can be discriminated by the estimated CoR position (13); low values of $\tilde{c}$ mean that the object will rotate, and high values mean that the object will translate. This can be graphically visualized in Figure 7. The figure shows the normalized LS parametrized with respect to $\tilde{c}$. Low values of $\tilde{c}$ correspond to high-friction torque values and vice versa. Thus, $\tilde{c}$ can be used in the pivoting mode to detect the end of the pivoting maneuver and stop the decreasing of the grasping force. In the experiments, as soon as the normalized CoR position is such that $\tilde{c}>1$, the pivoting maneuver is considered completed.

6. Experiments

This section presents a series of experiments performed on a real system to evaluate the control strategy presented in Section 5. The experiments were carried out in a lab-scale in-store logistic scenario [28,29]. The experimental setup for the manipulation experiments is depicted in Figure 8.

The grasping device was a commercial 1DOF WSG50 parallel gripper by Weiss Robotics, DE, equipped with the SUNTouch force/tactile sensor [8] designed and produced in our laboratory. The sensor has an accuracy of $0.1$ N and $0.002$ Nm in the measurement of forces and torsional moments, respectively. This end effector was mounted on a LBR iiwa robot by Kuka, DE, and the whole system was controlled via a ROS network. The gripper was commanded at 50 Hz, the fingertips provided the tactile data at 500 Hz, and the iiwa robot was controlled via the FRI interface at 1 kHz.

Four objects were selected (Figure 9): two resin blocks (a big and a small one), a plastic bottle, and an aluminum can. We considered the objects rigid so that all the parameters, except $\mu$ and ${\beta }_A$, estimated in Section 4 could also be used with these objects. Obviously, not all the objects in Figure 9 were perfectly rigid, but we could approximate them as rigid compared to the sensor soft pad. The two remaining friction coefficients $\mu$ and ${\beta }_A$ could be roughly estimated online by rubbing the fingertips on the object and comparing the measured force with the robot velocity with a procedure similar to the one described in [30]. The object-dependent parameters are reported in

Table 2. Friction parameters estimated for the manipulated objects.

	Big Resin Block	Small Resin Block	Plastic Bottle	Aluminum Can
$\mu$	$0.85$	$0.7$	$0.85$	$0.75$
${\beta }_A$ (Ns/m)	$1.24·{10}^7$	$1.825·{10}^7$	$1.963·{10}^7$	$2.121·{10}^7$

.

The first experiment involved the big resin block. At the beginning, the block rested vertically on the picking desk, and the robot was commanded to grasp it above its CoG (Figure 10a) and lift it. As soon as the gripper grasped the object, the slipping avoidance algorithm was activated. The results are shown in Figure 11. At $t=1$ s, the lift began, the sudden increment of the external load due to the object weight caused the velocity observer to detect a sliding velocity. Thanks to the dynamic contribution, the controller responded by increasing the grasp force up to 5 N. Since the object was grasped above the CoG and almost no gravitational torque was applied to the fingertip, after the lift (Figure 10b), the estimated normalized CoR position $\tilde{c}$ was high (middle plot red line). The CoR should be theoretically infinity in this condition; however, to avoid numerical issues, we saturated the normalized CoR to an upper limit of $1.5$. When the lift was completed and all the dynamic effects vanished, the static contribution stabilizes our slippage indicator $\mathcal{S}$ (middle plot, blue line) to the reference value ${\overline{\mathcal{S}}}_a=0.8$. To test and stress our algorithm, at $t=14$ s, a human operator applied various disturbance forces by hands on the resin block (Figure 10c,d). The observer detected such events as slippage velocity peaks that were counteracted by the dynamic contribution. At the same time, as soon as the disturbances were applied, the slippage indicator increased, and the static controller also helped to regulate the force to higher values.

The second experiment was carried out with the lighter small resin block. This time, the block rested horizontally on the picking desk (Figure 12a), and the robot did not grasp it in its CoG so that a gravitational torque was applied on the fingertips. The results are shown in Figure 13. Once again, as soon as the lift began, the observer estimated a slippage velocity (bottom plot), and the controller responded by increasing the grasping force (top plot, blue line). Even if the small black was lighter than the big one, after the lift (Figure 12b), the grasping force reached similar values (5 N). This was because this time, the torsional moment (top plot, magenta line) also had to be sustained. The effects of the torsional load were also evident in the estimated CoR position $\tilde{c}$. In the previous experiment, it was saturated to $1.5$, but now, the CoR estimation algorithm computed a much lower value, i.e., $\tilde{c}=0.4$. This means that, if the grasping force decreased, the slider would be more prone to a rotation than a translation. Additionally, in this experiment, the control algorithm was stressed at $t=17$ s by applying disturbances (Figure 12b). Even in this different configuration, the results were analogous to the previous experiment. As soon as the disturbances were applied, the observer detected slippage velocity peaks, and the controller responded by increasing the grasping force to avoid the slippage.

The third experiment shows how the manipulation ability developed in this work can be used in a collaborative scenario. The robot had to grasp a plastic bottle that was knocked over the picking desk (Figure 14a) and had to hand it over to a human partner in a vertical orientation. The results are shown in Figure 15. The first phase was similar to the previous experiment. The robot grasped the bottle far from its CoG so that the slippage controller had to also counteract the gravitational torque. This time, after the lift ($t=6$ s, Figure 14b), the measured torque (top plot, magenta line) increased up to 15 Nmm, and the slippage controller automatically modulated the grasping force to 16 N (top plot, blue line). In this configuration, the estimated CoR position was very low ($\tilde{c}=0.3$, bottom plot). Thanks to this information, we can tell in advance that, if the grasping force is decreased, the manipulated object would start to rotationally slip about the grasp axis, and the translational slippage would be negligible. This means that it is possible to activate the pivoting control modality to reorient the object in hand to the vertical configuration depicted in Figure 14c. At $t=11.5$ s the pivoting began, the grasp force exponentially decreased, and the bottle started to rotate. Since the object was moving, the viscous friction caused an initial increment of the torsional torque, but, as the object rotated toward the vertical configuration, the gravitational torque decreased. While the bottle rotated, the CoR increased, and as soon as $\tilde{c}>1$ (at $t=16$ s), the pivoting maneuver was considered completed and the grasp force stopped its descent since the object had reached the configuration in Figure 14c and can be handed over to the human partner. The final handover phase (Figure 14d) and the control of the forces exchanged between the robot and the human was carried out with the algorithm in [31].

The fourth experiment demonstrates the slipping avoidance algorithm’s ability to automatically choose the grasp force when handling objects of very different weights. The objective of this experiment was to hand the aluminum can over to a human partner. The task was repeated two times with a full and an empty can, respectively. The can was grasped above its CoG, as shown in Figure 16a. The results are shown in Figure 17. The top plot shows the grasp force synthesized by the slipping controller as well as the measured tangential and torsional loads. After the lift ($t=3$ s, Figure 16a), the forces reached very different values in the two task repetitions. When the can was full, the tangential load was 4 N, while the torsional load was 6 Nmm; thus, the controller applied a grasping force of 8 N to avoid slippage. When the can was empty, it was much lighter: the tangential force was $0.24$ N, and the torsional moment was almost zero. Thus, the controller needed a very low grasp force of $0.32$ N to safely hold the aluminum can without damaging it.

A video of all the experiments is available as Supplementary Material (Video S1).

7. Discussion

The experimental results presented so far give evidence of the ability of the robot to handle different types of objects made of different materials (resin, plastic, and metal) and with large differences in their weight and friction properties. The robot is able to grasp and lift the objects without any knowledge of their weight by applying the minimum grasping force to hold them without any translational or rotational slippage. This is shown especially in the lifting of the empty aluminium can, which requires a grasping force as low as $0.2$ N, while lifting the full can requires a 40 times larger force. The manipulation ability of the robot is not limited to safely grasping any object with no slipping; the grasp force can be suitably controlled to allow an intentional slip, which might be useful to change object orientation in hand even with a simple parallel gripper. The selection of a model-based manipulation method is motivated by the objective of allowing the robot to manipulate different types of objects without requiring an excessive learning and training burden. Indeed, the off-line learning phase is limited to the experimental identification of a few parameters of a friction dynamic model that is the basis of the manipulation control strategy. The few remaining parameters that depend on the specific object to be handled are estimated at run time by a simple and quick exploration phase (see the accompanying video). The main limitations of the method, at the present development stage, are related to the limited accuracy of the force/tactile sensor when it interacts with largely deformable objects and with rigid objects characterized by a curvature radius smaller than the one of the soft pad. Moreover, the model learning phase, even though robotized, is still quite lengthy.

8. Conclusions

This paper presented a model-based control strategy for slipping avoidance and pivoting using a simple parallel gripper. The control system is able to automatically regulate the grasping force without knowing the object’s weight, instead using the measurement of the friction force and torque between the fingertip and the manipulated object. The revisited model presented in this paper is used to build a slippage observer that has been validated with more than a thousand sliding manoeuvres at different velocities and forces. By using the full-state feedback of the observer, we proposed a novel sliding control scheme that considers the internal friction state by defining a novel slippage indicator to be suitably controlled to the desired value. The experimental tests evaluated the proposed control scheme in a lab-scale real-world environment in which the robot manipulated everyday objects that differed in weight, material, and grasp configuration. Future research efforts will be devoted to exploring sensor calibration methods more suitable for deformable objects and to generalize the manipulation control to multi-fingered hands, where not only the magnitude but also the direction of the grasping forces can be exploited to control the object grasp and its in-hand manipulation.