Sustainable Multi-modal Sensing by a Single Sensor Utilizing the Passivity of an Elastic Actuator

When a robot equipped with compliant joints driven by elastic actuators contacts an object and its joints are deformed, multi-modal information, including the magnitude and direction of the applied force and the deformation of the joint, is used to enhance the performance of the robot such as dexterous manipulation. In conventional approaches, some types of sensors used to obtain the multi-modal information are attached to the point of contact where the force is applied and at the joint. However, this approach is not sustainable for daily use in robots, i.e., not durable or robust, because the sensors can undergo damage due to the application of excessive force and wear due to repeated contacts. Further, multiple types of sensors are required to measure such physical values, which add to the complexity of the device system of the robot. In our approach, a single type of sensor is used and it is located at a point distant from the contact point and the joint, and the information is obtained indirectly by the measurement of certain physical parameters that are influenced by the applied force and the joint deformation. In this study, we employ the McKibben pneumatic actuator whose inner pressure changes passively when a force is applied to the actuator. We derive the relationships between information and the pressures of a two-degrees-of-freedom (2-DoF) joint mechanism driven by four pneumatic actuators. Experimental results show that the multi-modal information can be obtained by using the set of pressures measured before and after the force is applied. Further, we apply our principle to obtain the stiffness values of certain contacting objects that can subsequently be categorized by using the aforementioned relationships.


Introduction
A compliant joint is an important mechanism that provides increased dexterity and dynamic behavior when compared with the performance of a joint driven by an electric servo motor with a higher reduction gear.A compliant joint can enable a robot hand to passively and adaptively grasp objects of many different shapes [1][2][3], and it can also effectively absorb the landing impact of a legged robot [4][5][6].In order to enhance the dexterity of a robot hand and provide more dynamic locomotion for a legged robot, it is very important to obtain the contact information regarding the object being grasped or landing impact information, including multi-modal information such as the magnitude and direction of the applied force and the joint angle formed by the passive displacement of the joint due to the applied force.In many studies, the force sensors such as strain gauges [7], resistance [8], and capacitance [9] are attached to the contact point at which the force is applied.Further, the joint angle is measured by an angle sensor attached to the joint.However, this approach is not suitable for use in daily-use robot, which must be durable and robust, because the sensors and connecting wires can undergo damage owing to iterative contacts or excessive applied force.This approach also requires multiple types of sensors to measure such physical values, which add to the complexity of the device system of the robot.Because failure of the robot tends to follow the complicated system, it is preferable to avoid adopting multiple types of sensors.
Some previous researches have adopted approaches in which an end effector force is estimated or calculated without force sensor [10][11][12][13].Others have reported that joint stiffness that cannot be measured directly is estimated or calculated [14][15][16].In [12], a proposed external force estimator is run parallel to compliant robot manipulation.In [13], a proposed method is verified using a robot driven by elastic Pleated Pneumatic Artificial Muscles.In [14][15][16], the joint mechanism itself is made compliant by inserting elastic materials between a motor and link.In many researches including [10][11][12][13][14][15][16], it is assumed that the joint angle, velocity and torque in some studies, can be measured.Therefore, the sensors are attached on the joint in order to measure the joint information directly.However, similar with aforementioned problem, the wire can undergo damage owing to iterative contacts.Furthermore, especially for small robot such as human-sized robot hand, a space for the joint sensor is a large restriction for designing the structure, and the sensor is exposed to noise by the actuator.Therefore, it is required that the sensing information is obtained at a point distant from the joint as well as contact point.
The authors have adopted McKibben pneumatic actuators [17,18] for realizing the compliant joint, and they have proposed a method to obtain an end effector force and deformed angle by applied force without a force sensor and joint sensor by utilized the passivity of the actuator, wherein the inner pressure of the actuator changes passively according to the deformation of the actuator owing to the applied force [19].In [19], Takuma et al. studied a one-degree-of-freedom joint mechanism driven by two pneumatic actuators, and they derived the relationship between the magnitude of the force applied perpendicularly to the link and the pressures before and after the force was applied.They also derived the relationship between the joint angle and the pressures.Utilizing this approach, the multi-modal information can be obtained by using only one type of sensor as a pressure sensor, and because the sensor is located at a point distant from the contact point and the joint, this method can be sustainable for daily-use in robots that must be durable and robust.However, the direction of the force in [19] is limited to the perpendicular direction because a force parallel to the link does not rotate the joint, and therefore, this force does not contribute to the deformation of the actuator.Therefore, in order to obtain the magnitude of the applied force and the consequent displacement of the joint angle, the direction of the force should be known or measured.
In this work, we employ a two-degrees-of-freedom (2-DoF) joint mechanism driven by four pneumatic actuators.For 2-DoF joint mechanism, the joints (at least one joint) are rotated passively even though the direction of the force is parallel to the top link.We derive the relationship between the contact information, including the magnitude of the force perpendicular and parallel to the link and the angles of the joint, and pressures of the four pneumatic actuators.In this paper, conditions relative to 2-DoF joint mechanism driven by pneumatic actuators, such as geometrical constraints, equilibrium condition of the applied force and the restoring force of the actuator, typical properties of a McKibben pneumatic actuator, and Boyle's law are adopted.Subsequently, we explain how we utilize such conditions to derive the relationship between the contact information and the actuator pressures.For the evaluation of the derived equations, a physical 2-DoF joint mechanism is developed, and the accuracy of the derived relationship is verified by comparing the calculated and observed forces and angles.We also demonstrate an application in which various kinds of materials are categorized based on their stiffness upon bringing the materials in contact with the joint mechanism from arbitrary direction.

Passivity of the Pneumatic Actuator
Figure 1a shows the McKibben pneumatic actuator.The actuator is constructed using a rubber balloon covered by a nylon sleeve.Upon supplying compressed air to the balloon, the actuator expands radially and contracts longitudinally.Figure 1b-d shows the physical characteristics of a certain pneumatic actuator used in the experiments.Although the characteristics of the actuators used in the experiments are different from each other, such differences can be compensated by using a stochastic method explained in Section 3. Figure 1b shows the relationship between the pressure and the natural length, which corresponds to the length of the actuator without payload.Each length is measured three times at each pressure value.As shown in the figure, the natural length depends on the pressure, and the results are similar for the same pressure value.Since the actuator has the property of elasticity, the actuator extends passively according to the magnitude of the external force.Compared with an air cylinder made from metallic components, the McKibben pneumatic actuator is very lightweight, making it a powerful candidate for the compliant joint mechanism in the robot.Although a pneumatic actuator, including the McKibben pneumatic actuator, is expected to be used for a wide range of application [20,21], it is difficult to obtain a precise relationship between the contact information and the pressure because of the friction of the nylon sleeve [22].For example, in the case of the air cylinder, the relationship between the force f and the pressure P can be expressed as f = PS where S is the cross-sectional area of the cylinder and is constant value.However, both length and cross section of the McKibben pneumatic actuator change simultaneously when it is extended or compressed.Therefore, it is more difficult to obtain the relationship between the length of the McKibben actuator and the force when it is extended than the case of the air cylinder.There are some nonlinear models concerned with a relationship between the tension force and the extended length of the McKibben pneumatic actuator ( [17,23] for example).Because the range of the extended length in this work is small, this paper attempts to adopt a simple linear model.Figure 1c shows the relationship between the tension force (vertical axis) and the extended length (horizontal axis) for an inner pressure or 0.33 MPa.For each value of the force, the extended length is measured three times.As shown in the figure, the relationship between the length and the force can be approximated as a linear spring model because of small range of extended length.Therefore, after modifying the second-order nonlinear model in [23], we adopt the simple linear model as where k is a constant value, P is the pressure, and Δl is the extended length.Note that the coefficient of the spring model contains the inner pressure of the actuator, P.
As explained above, the McKibben pneumatic actuator is extended by the tension force.The actuator has an interesting characteristic that the inner pressure changes without more air being supplied to or removed from it when an external force is applied.Figure 1d shows the relationship between the magnitude of the external force and the inner pressure.Before applying the force, compressed air is supplied to the actuator, and forces of various magnitudes are subsequently applied.Consequently, the length of the actuator changes according to the external force.Assuming that the temperature of the actuator does not change before or after the force is applied, the pressure is expected to change in accordance with Boyle's law.This means that the external force and the length can be estimated based on the inner pressure.

Contact Information Using Actuator Pressures
We adopt the 2-DoF joint model driven by four antagonistic actuators in our study.Figure 2 shows the joint positions before and after the force is applied.Using the 2-DoF model, the relationship between the contact information, including the external force components F x and F y and the joint angles θ 1 or θ 2 , and the pressures is determined by utilizing conditions such as geometrical constraints, equilibrium conditions of the applied force and restoring force of the actuator, and typical characteristics of the McKibben pneumatic actuator as follows.Firstly, the relationships between the angles and the extended length of the actuator are derived in addition to the relationships between the applied force and the extended length.Secondly, after deriving the relationship between the extended length and the pressure, the relationship between the contact information including the angles and forces and the pressure is derived.

Joint Angle Using Extended Length
As shown in the left panel in Figure 2, each actuator pressure is denoted as P ij0 .The subscript i denotes the left or right actuator, where i = L for the left-side actuator, and i = R for the right-side actuator.The subscript j denotes the left or right actuator at the side position, where j = 1 for the outside actuator whose tendon wire is attached at a bottom moment arm, and j = 2 for the inside actuator whose tendon wire is attached at an upper moment arm through a bottom moment arm.The parameters h 1 , h 2 , and h 3 denote the lengths from the baseline to the bottom joint, from the bottom joint to the upper joint, and from the upper joint to the contact point, respectively.Before the force is applied, the lengths of the actuators are expressed as where l sij is the tendon wire length, l nij is the natural length, and Δl ij0 is the extended length such that total length of l sij , l nij , and Δl ij0 is equals to h 1 or h 1 + h 2 .The tendon wires are made of non-extensible material, and so they have a higher back-drivability that transmits the force to the actuator.After the force is applied as shown in the right panel of Figure 2, the lengths of the actuators change by Δl ij .The geometrical conditions of the actuators are expressed as where r 1 and r 2 denote the length of the upper or bottom moment arm between each of the joints and the attachment point of the outside actuator and inside actuator, respectively.Δl ij is the extended length of the actuator after the force is applied (see Figure 3).Δl Lθ1 and Δl Rθ1 are the extended lengths corresponding to the joint angle θ 1 .By subtracting Equation (3) from Equation (2), we obtain the joint angle θ 1 as Because Δl i1 2 is negligible compared with Δl i1 , Equation ( 8) is approximated as By subtracting Equation (5) from Equation ( 4), θ 1 is also expressed using Δl Rθ1 and Δl Lθ1 as From Equations ( 9) and (10), By subtracting Equation ( 7) from Equation ( 6), the joint angle θ 2 is obtained as considering that (Δl ij -Δl iθj ) 2 is negligible compared with (Δl ij -Δl iθj ).From Equations ( 11) and ( 12), the rotated angle θ 2 is then expressed as

Applied Force Using Extended Length
From the principle of virtual work, the external force F is expressed as follows: 1 where F denotes the applied force given by F = (F x ,F y ) T , and τ denotes the joint torques given by τ = (τ 1 ,τ 2 ) T as shown in Figure 2. The matrix J denotes the Jacobian matrix for the 2-DoF joint.
Considering that the position of the contact point (x,y) is expressed as (x,y) T = (−h 2 sinθ 1 −sin(θ 1 +θ 2 ), h 1 + h 2 cos(θ 1 +θ 2 )) T , the Jacobian matrix is given by: As shown in Figure 2, the bottom joint, whose angle is denoted as θ 1 , is driven by four actuators.The top joint, whose angle is denoted as θ 2 , is driven by two actuators.In order to derive the joint torques τ 1 and τ 2 , torques by a single actuator at each joint are expressed first.Considering that the forces f L2 and f R2 , as shown in Figure 2, are transferred via pulley at a bottom moment arm whose angle is denoted as θ 1 , the joint torques by a single force f L1 or f L2 (see Figure 4) is expressed as: Note that the second term on right side of Equation ( 16) is a reaction force of τ 2_L2 .Similarly, the torques τ 1_R1 , τ 1_R2 , and τ 2_R2 are expressed by the single forces f R1 and f R2 .From these equations, considering that the directions of the torque by the force f R1 and f R2 are opposite to the torques by f L1 and f L2 , joint torques τ 1 = τ 1_L1 + τ 1_L2 + τ 1_R1 + τ 2_R2 and τ 2 = τ 2_L2 + τ 2_R2 are expressed as: where From Equation (1), the restoring forces of the actuators are expressed as: where k ij is a positive constant.From the left panel of Figure 2, the equilibrium equation before the force is applied is expressed as follows: Using Equations ( 14), ( 18) and ( 20), the force is expressed as:

Extended Length Using Pressures
In order to derive the extended length Δl ij , the condition of McKibben actuator is adopted.From Boyle's law, the relationship between the initial pressure P ij0 and the final pressure P ij is expressed as The cross-sectional areas of the actuators before and after the force is applied (S ij0 and S ij ) are expressed as: As shown in Figure 5, the diameters of the actuator are subsequently expressed as: where n ij is the number of turns of the fiber.By substituting Equations (24-27) into Equation ( 23), it is rewritten as: Because Δl ij 2 and Δl ij 3 are much smaller than Δl ij , Δl ij is derived from Equation (28) as: where Note that β ij is determined by the characteristics of the actuator, and does not depend on the pressure.
Figure 5. Fiber of the nylon sleeve.

Contact Information Using Pressures
Using Equation (29), the joint angles θ 1 and θ 2 and forces F x and F y can be expressed by the pressures.From Equations ( 9) and (29), the equilibrium angle θ 1 is expressed as: From Equations ( 13) and ( 29), the equilibrium angle θ 2 is expressed as: where These equations indicate that the angles can be expressed in terms of the ratio of the pressures P ij0 and P ij .Further, the β θij coefficients account for the length of the fiber of the actuator and the length of the actuator before the force is applied.
Using Equations ( 21), ( 22) and ( 29), the force can be expressed by the pressures as: where and These equations indicate that the magnitude of the applied force F can be expressed in terms of the product of the triangular function derived from the Jacobian matrix, determined from the difference between the directions of the force and each moment arm, and the difference in the pressure, P ij0 -P ij , before and after the force was applied.Note that the coefficients β F1 -β F6 and β θij are not correlated with the pressure, but rather with characteristics of the actuator, such as the spring coefficients k ij and the length of the actuator before the force is applied.However, it is difficult to obtain these physical parameters, and therefore, it is difficult to obtain the coefficients.

Developed 2-DoF Joint Mechanism
For the evaluation of the relationship between the contact information including the magnitudes of the force components F x , F y and the joint angles θ 1 , θ 2 , we developed a physical 2-DoF joint mechanism.We confirmed the accuracy of the derived equations by comparing the calculated and observed contact information.Figure 6a shows the developed 2-DoF test bed with a height of 0.590 m.The length of the upper and bottom moment arms r 1 and r 2 are 0.075 m and 0.035 m, respectively.The heights h 1 , h 2 , and h 3 are 0.355, 0.130, and 0.065 m, respectively.As a single type of sensing information, the pressure is measured by pressure sensors (SMC PSE530) that are located between the actuator and the air valve, and the sensing signals are sent to a PC (OS:Ubuntu) via a controller (iXs Research iMCs03).Note that the pressure sensors need not be attached to the contact point or the joint; they can be attached at arbitrary positions between the actuator and the valve.In order to evaluate the accuracy of the joint angle calculations, the actual joint angles are measured using a potentiometer (COPAL ELECTRONICS JC22E2k).

Experimental Setup
The forces and the angles can be calculated using Equations (30-33).However, as explained in the previous section, it is difficult to obtain the coefficients β θij and β F1-β F6 in these equations.In addition, physical parameters such as k ij in Equations ( 32) and (33) are different from each other as mentioned in Section 1 because the elements of the actuator, such as length, are different.Therefore, we first estimate the coefficients using the method of least squares.
The procedure for obtaining the contact information and the corresponding pressure data is as follows: • Supplying compressed air.Compressed air is supplied to the actuators.The pressures P ij0 are measured by the pressure sensor; • Applying an external force.The force F is applied to the contact point.In order to apply forces of various magnitudes and directions, a weight is suspended from a movable pulley, as shown in Figure 6b.The direction of the force is varied by moving the pulley; • Measuring contact information and pressure; After a certain interval of time subsequent to applying a force, the joint mechanism adopts an equilibrium posture.The angles θ 1 , θ 2 , the forces F x , F y , and the pressures P ij are subsequently recorded.
In order to obtain the coefficients, 90 sets of contact information including θ 1 , θ 2 , F x , F y , P ij0 , and P ij are measured.Because of construction issues in the experimental setup, the ranges of the measured angles for θ 1 and θ 2 and the measured forces F x and F y are different.Because the coefficients are largely influenced by the measured ranges for the method of least squares, we set different coefficients for each angle and force, i.e., β θ1L1 and β θ1R1 for θ 1 ; β θ2L1 , β θ2R1 , β θ2L2 , and β θ2R2 for θ 2 ; β Fx1 -β Fx6 for F x ; and β Fy1 -β Fy6 for F y from Equations ( 30)-(33) as After estimating the coefficients, another 18 sets of observed data are recorded to evaluate the accuracy of the obtained multiple regression equations.

Result
By using the method of least squares, the coefficients are estimated as:  7.In Figure 7a,b, the horizontal axis represents the observed angle values, and the vertical axis represents the calculated angle using Equations (34) and (35).In Figure 7c,d, the horizontal axis represents the observed force, and the vertical axis represents the calculated force using Equations ( 36) and (37).The diagonal line in these figures indicates the set of points for which the observed data match the calculated data.As shown in the figures, most of the data points are observed near the diagonal line, and we can see that the contact information including angles and forces can be accurately calculated by using the inner pressures of the pneumatic actuators.

Application
As an application to demonstrate the accuracy of the estimated contact information, this paper shows that the compliance of some types of materials can be categorized by pushing the object as shown in Figure 8a.In the first half of this section, we show that the stiffness of a given object can be calculated even if the direction of push relative to the joint mechanism is unknown.In this study, we used cylindrically shaped objects such as polyethylene terephthalate (PET) bottles and drinking cans.Figure 8b shows the position of the joint mechanism before and after the object is pushed.As shown in the figure, we adopt a simple 2-D cylindrical model that has a radial arrangement of springs.Before the object is pushed, it is set such that it just makes contact with the contact point.In this situation, the coordinates of the center of the object are given by O 1 (R,h 2 +h 3 ) T , where R denotes the radius of the object.When the object is pushed, the center of the object is moved from O 1 to O 2 (x o ,y o ) T , and the direction of the force owing to the spring corresponds with the vector from O 2 to the contact point C(x c , y c ) T .Consequently, the stiffness of the object k o is defined as where |F| denotes the magnitude of the force given by |F| =(F x2 +F y2 ) −1/2 , and Δx denotes the distance of deformation of the object (see Figure 8b).When the object is pushed along any arbitrary direction φ for a known distance d, the coordinates of the center of the object O 2 can be expressed as: Note that the coordinates of C can be estimated using pressures P L2 , P R2 , P L1 , and P R1 .
Let the vector e be the normal vector of the link at which the force is applied such that e = (−cos (θ 1 +θ 2 ), −sin(θ 1 +θ 2 )) T , and therefore, the angle θ F between the vector e and the direction of the force C O 2 can be expressed as The angle θ F is also acquired by using the estimated force F x and F y as From Equations ( 40) and (41), the direction φ can be numerically calculated.By using φ, the coordinates of O 2 can be calculated using Equation (40).Therefore, the distance of deformation of the object Δx can be calculated as By using Equations ( 38) and (42), and the estimated force F, the stiffness of the object can be calculated.In the experiment, six objects made of three materials are chosen (three plastic bottles, two aluminum cans, and one steel can).In this experiment, we let the position of the object, i.e., O 2 , be known.The stiffness is subsequently calculated from Equations ( 38) and (42).From the inner pressures of the actuator before and after the object has been pushed, the stiffness of the object is obtained.Each of the selected objects is pushed, the stiffness in each case is calculated several times, and the variation in the calculated stiffness is observed.Figure 9 shows a plot of the stiffness calculated for each sample.The horizontal axis indicates the type of sample object, and the vertical axis represents the calculated softness of the object.For each object, contact information is measured ten times.The stiffness values in the figure indicate that the stiffness of an object can be categorized based on the measured pressures.

Conclusions
In this study, in order to acquire varied contact information by using a single type of sensor for a durable joint mechanism used in robots, we focused on the passivity of a pneumatic actuator wherein the pressure of the actuator changes passively owing to the applied force.We employed a 2-DoF joint mechanism driven by McKibben pneumatic actuators, and we aimed at obtaining contact information including the magnitude and direction of the applied force and the joint angles by using the pressures in the actuators.Because the sensors used to measure the pressures need not to be set at the contact point and the joint, it is possible to obtain sensor durability, thereby avoiding sensor damage due to excessive magnitude of force or iterative contact, which is the case in conventional joint mechanisms.Because only one type of sensor as pressure sensor is required to obtain the contact information, such multi-modal information can be obtained by a simple device system.In order to obtain the relationship between the contact information and the pressures, we constructed a 2-DoF joint model and imposed certain conditions.Using the model and conditions, we found that the joint angles can be expressed via multiple regression equations involving the ratio of the pressures before and after the force is applied.We also found that the applied forces can be expressed via multiple regression equations involving the product of the triangular function derived from the Jacobian matrix, which was determined from the difference between the directions of the force and each moment arm, and difference in the pressure before and after the force is applied.
In order to evaluate the regression models, we performed experiments using a physical 2-DoF joint mechanism.Before conducing the experiments, we estimated the coefficients in the regression equations mentioned above by using the least squares method, because it is very difficult to obtain certain physical values represented by the coefficients.By using the estimated coefficients, we found that the angles of two joints and the forces perpendicular and parallel to the link could be calculated accurately.In terms of application, three types of materials were categorized by pushing objects made of these materials with the 2-DoF joint mechanism through a certain distance in an arbitrary direction.
There are some applications on this joint mechanism.One of the remarkable characteristics of the McKibben pneumatic actuator is that the viscoelasticity can be changed by tuning the inner pressure.The joint mechanism in this paper is driven by antagonistic pneumatic actuators, and then the stiffness of the joint can be changed by tuning the inner pressure of the actuators.For example, in [5], the bipedal robot driven by antagonistic pneumatic actuators obtained multiple types of locomotion such as walking and jumping by tuning the inner pressure of the actuators.The joint mechanism in this paper will be able to estimate a wider range of the softness of the object as explained in Section 4 by tuning its joint stiffness.For example, when the joint mechanism touches a soft material, the joint stiffness is set lower in response to the softness of the object in order to avoid the damage of the object while pushing.

Figure 1 .
Figure 1.McKibben pneumatic actuator.(a) McKibben pneumatic actuators with air (upper) and without air (bottom); (b) relationship between pressure and natural length; (c) relationship between extended length and magnitude of the force; (d) relationship between the external force and the inner pressure.

Figure 2 . 2 -
Figure 2. 2-DoF joint model before and after the force is applied.(a) before the external force is applied; (b) after the external force is applied.

Figure 3 .
Figure 3. Natural and extended length of the actuator.

Figure 4 .
Figure 4. Joint torques by the single forces f L1 and f L2 .
F x and F y are expressed by using pressures such as

Figure 8 .
Figure 8. Position of the object and the joint mechanism before/after the object is pushed.(a) posture change before and after the object is pushed; (b) geometrical position of the object and joint mechanism.
The coordinates of the contact point C are given by sin(θ θ ) cos θ cos(θ θ )

Figure 9 .
Figure 9. Acquired softness of the sample objects.