Influence of Errors in Tactile Sensors on Some High Level Parameters Used for Manipulation with Robotic Hands

Tactile sensors suffer from many types of interference and errors like crosstalk, non-linearity, drift or hysteresis, therefore calibration should be carried out to compensate for these deviations. However, this procedure is difficult in sensors mounted on artificial hands for robots or prosthetics for instance, where the sensor usually bends to cover a curved surface. Moreover, the calibration procedure should be repeated often because the correction parameters are easily altered by time and surrounding conditions. Furthermore, this intensive and complex calibration could be less determinant, or at least simpler. This is because manipulation algorithms do not commonly use the whole data set from the tactile image, but only a few parameters such as the moments of the tactile image. These parameters could be changed less by common errors and interferences, or at least their variations could be in the order of those caused by accepted limitations, like reduced spatial resolution. This paper shows results from experiments to support this idea. The experiments are carried out with a high performance commercial sensor as well as with a low-cost error-prone sensor built with a common procedure in robotics.

the grasp forces accordingly. The information regarding the shape of the contact area, the friction and direction of fingertip forces begins to shape fingertip force output within about 70 ms in human manipulation as stated in [15]. Therefore, taking into account the other data sources and control tasks that should be attended, pre-processing of raw data must be done to reduce the information to communicate and the processing load of the central unit.
This paper shows results from experiments focused on the evaluation of the influence of tactile sensor limitations on the above mentioned reduced set of parameters. Specifically, on the tactile image moments or the equivalent variables that define the associated ellipsoid. An error prone piezoresistive tactile sensor is used to see the impact of hysteresis and drift (see Figure 1c). This sensor is made by placing a sheet of sensitive material on an array of electrodes, a quite common procedure in sensors for robotic hands [10,16]. Another commercial sensor is also used, mainly to see the effect of mismatching and limited resolution (see Figure 1d [17]). This commercial sensor has good linearity and low hysteresis and also a high resolution. This good performance allows the comparison of the influence of different parameters isolated to others such as non-linearity or hysteresis. Specifically, the resolution can be changed by software and the variation of the target parameters estimated. Mismatching or differences in the response of different taxels can also be seen to be quite isolated from other interferences using commercial equipment to calibrate the sensor and equalize the output of all taxels. Although the subject is complex and this paper presents results from a limited number of experiments, it is enough to provide clear indications about the weight of these non-idealities in the sensor performance for the above mentioned applications. The rest of the paper is structured as follows: in Section 2, the experimental set up and materials used to carry out the experiments are described; Section 3 provides performance data from the two tactile sensors used in the tests; Section 4 presents the methodology of the subsequent experiments and Section 5 shows the obtained results and associated discussion. Finally, Section 6 summarizes the main conclusions derived from the results.

Experimental Setup
Two setups have been used to carry out the experiments and characterize the sensors. One of them obtains readings of the whole tactile matrix under the same uniform pressure. It is based on a pneumatic commercial equilibration/calibration device (Tekscan PB100E [17]; see Figure 1a). The sensor is laced in a slot where one side is rigid and the other is a flexible wall of a chamber. When the chamber is pressurized the wall exerts an even pressure on the sensor. Since the device is intended to be used to study the hysteresis, it has been customized to perform loading-unloading cycles as defined from the computer interface. It has been achieved by adding an electro-valve that allows the flow from a compressor until the pressure reaches the desired value. The other way is also possible by opening the electro-valve to reduce the pressure in the chamber. The electro-valve is the Pneumax 171E2N.T.D.0009 [18] and it is controlled by an application on a computer.
Moreover, a few metal pieces with different shapes were used to achieve different pressure patterns on the tactile sensors. Figure 2a shows them and their dimensions. A motorized stage was built to exert a force on the top of these objects once they are placed on the tactile sensor (see Figure 1b). The stage has a T-NA08A50 linear actuator to move along the z axis and two T-LA60A actuators for displacements along x and y axes (all of them from Zaber Technologies [19]). A piston with a spring was also added in the z axis to enlarge the dynamic range. The sensor Nano17 from ATI Industrial Automation [20] was also added at the tip of the motor in the vertical direction, to register the actual force that is exerted on the objects and then on the sensor. A hole was made at the center of mass of the objects (see Figure 2b) and the tip of the force sensor probe is inserted into it. In this way a pressure equal to the force registered by the sensor divided by the contact area between the object and the sensor is applied to the sensor. Regarding the electronics used to acquire the tactile sensor output, the I-Scan commercial handle [21] is employed with the sensor from Tekscan. With respect to the other custom sensor, the readings of the whole tactile array are registered by means of well-known interface electronics [22]. Since the goal is not the dynamic evaluation, the electronics was designed to achieve a good static performance, so electro-mechanic relays were used to implement the switches to select the rows as the array is scanned. In this way the errors due to the non-zero on resistance of the switches are minimized. The data acquisition is done with the NI-USB 6259 device from National Instruments. Sixteen analog inputs are multiplexed to scan up to 16 × 16 taxels in our testing platform.

Tactile Sensors
The results of this paper are obtained from tactile sensors based on a piezo-resistive principle. One of them is built with a common procedure that consists in placing a layer of sensitive material on an array of electrodes, usually made on a flexible printed circuit board (see Figure 1c). The sensitive electroactive material is a conducting polymer in the case of the sensor of this paper [22]. The resistance between two electrodes associated to each taxel changes when the force against the taxel increases.
The second sensor is the 5051/P1/3056T1/20 from Tekscan (see Figure 1d), it is also piezoresistive but fabricated with a screen printing technology. It has a symmetrical structure in the z axis, where two identical layers, with conductive tracks plus sensitive ink on them, are put in contact in a column and row fashion, and each cross of the array is a taxel.
In the following section, some performance data of both sensors are provided. These data are obtained with the set-up shown in Figure 1a that exerts a uniform pressure on the sensor surface so it allows the output from all taxels to be read, in the sensor under similar conditions. The results in this section are not actually a complete characterization of the sensors since that is not the goal of this paper. The only purpose it serves here is to gain a better knowledge of them, which will help to understand the readings obtained later in the experiments.

Hysteresis
Tactile sensors suffer from hysteresis, i.e., their response under a certain pressure actually depends on previous pressure values. To quantify the hysteresis, the pressure on the sensor is increased and then decreased, and the maximum deviation between the obtained responses in both directions is taken as the hysteresis error. Figure 3a,b show averaging of the output of all taxels of the sensors in Figure 1c,d respectively, when the pneumatic equilibration/calibration device in Figure 1a is controlled from the computer to carry out six consecutive loading-unloading cycles. For the PCB sensor they are as follows: 0 psi → 60 psi → 0 psi → 50 psi → 0 psi → 40 psi → 0 psi → 30 psi → 0 psi → 20 psi → 0 psi → 10 psi → 0 psi in increments of 2 psi. The time interval between the new pressure level being exerted and the voltage output being registered by the acquisition board is 5 s. The previous procedure is done 10 times and the average is computed and displayed in Figure 3a (the standard deviation is also shown at the bottom of the same figure). A few captions for intermediate values of the outer cycle are also shown in Table 1. Note also the slight difference between the real value of the pressure for frames that are presumably under the same conditions. This is because the pressure changes must always be made in the same sense, increasing or decreasing its value depending on the path in the cycle, and it is also due to the limited control capability of the system. Also note the mismatching or variation of the response of different taxels.  The same previously described procedure is carried out with the sensor 5051/P1/3056T1/20 from Tekscan in Figure 1d, but now the cycles are: 0 psi → 20 psi → 0 psi → 15 psi → 0 psi → 10 psi → 0 psi → 5 psi → 0 psi, because the pressure rating of the sensor is 20 psi. The raw data are not equilibrated nor is calibration taken to obtain the curves in Figure 3b. These data are provided by the I-Scan acquisition system [21] that encodes the pressure in a scale of 256 levels (8 bits).
Note that the low cost sensor based on a flexible printed circuit board presents a hysteresis error much larger than that observed in the commercial one in Figure 1d. Moreover, the linearity is also much higher in the latter than in the first sensor. This behavior is not due only to the PCB technology, but also to other factors like size and geometry of the electrodes; mechanical properties of the sensitive layer atop; and adherence and roughness of the interface between both [22,23]. Nevertheless, since the goal of this paper is to compare the impact of the hysteresis with that from other error sources, it may be appropriate to take this simple sensor despite its limited performance from the taxel point of view. We will discuss if this poor performance is translated at system level in a tolerable amount or not.

Drift
Another common source of errors in tactile sensors is drift, or change, of the sensor output over time when the pressure exerted on it does not change. A test was carried out with the calibration/equilibration set-up in Figure 1a described in Section 2 to estimate the drift of the custom sensor Figure 1c. The pressure is changed in large increments and remains stable (as long as the regulation system achieves stability) for a long period before changing again to the next value. Pressure transitions are increments or decrements, specifically the results in Table 2 correspond to the sequence 10 psi → 30 psi → 50 psi → 30 psi → 10 psi. There are 255 samples taken every 5 s and the reading takes 1 s, therefore the time elapsed to estimate the drift is around 25 min. A rather large drift is observed, and it is not the same for all taxels, as can be seen in the difference as well as in the histograms displayed in Table 2. The drift is larger for small pressure values as Figure 4 depicts, and the mismatching between the drift of different taxels is also larger for these pressures. Nevertheless, the regulation of the pressure is also more difficult in these cases and this causes the slight knee points of the curves. Drift is also lower for pressure decrements, and dispersion between taxels is also smaller. Similar experiments were carried out with the commercial sensor in Figure 1d and the observed drift was less than that observed for the custom PCB based sensor, so again the latter is a good choice to observe the impact of drift in the above mentioned high level application parameters.

Mismatching
Since they are matrix sensors, a noticeable variation, or mismatching, between the response of different taxels of the same tactile sensor is observed. Therefore, calibration should be made to equalize the sensor output. This can be achieved with the pneumatic device in Figure 1a. It is worth highlighting however that this calibration has quite limited duration and can be altered if the shape of the sensor changes. A simple experiment can demonstrate this fact. Table 3(a) displays the output of the Tekscan 5051/P1/3056T1/20 sensor when a pressure of 15 psi is exerted on its surface, while Table 3(b) shows the same output once the sensor is equilibrated using a procedure recommended by the vendor. The sensor is then removed from the slot and it is waved in the air so it bends softly, then it is inserted again into the device. Now the response in Table 3(c) is not as uniform as that in Table 3(b) after the equilibration. If the sensor is placed on a rigid surface and different objects are put on it (like a star in the case of Table 3(d)), the response is again somewhat altered as observed in Table 3(e) when it is inserted in the calibration device. Taking into account that tactile sensors in robotics are usually mounted on curved surfaces, and therefore not commonly calibrated after fabrication, and are covered with flexible protective layers that suffer from wear, it can be assumed that a quite large mismatching remains in the sensor output and it is accepted in practical applications.

Spatial Resolution
The PCB based sensor in Figure 1c has 16 × 16 taxels and the distance between centers in the row and column directions, or spatial resolution, is 2.54 mm. The commercial sensor in Figure 1d has 44 × 44 taxels and a spatial resolution of 1.27 mm. The high resolution of the latter will allow to assess the influence of limited resolution to be assessed, since we can merge taxels to change the resolution and still have one comparable to those reported by many tactile sensors for robotic hands, for instance 3.4 mm for the DSA 9205 sensor from Weiss Robotics [13]. Figure 5 shows a set of parameters that provide useful information for manipulative, recognition or control tasks in general, as mentioned above. They are related to the ellipsoid location, shape and orientation and are obtained from the moments of the tactile image computed as:

Parameters for Performance Assessment
is the output of the taxel with coordinates ) , ( y x , and N and M are the number of rows and columns of the sensor array, respectively. The coordinates of the centroid (X, Y) in Figure 5 are obtained from these moments (see Table 4) and then it is possible to express the moments referred to this centroid, and therefore independently of the ellipsoid location, in the so called central moments as: Table 4. Parameters obtained from the processing of the tactile image and used commonly in control tasks.

Parameter Equation
Variation from a to b (%)  Table 4 shows the expressions used to compute the parameters labeled in Figure 5 besides of the Area and Aggregate Output from Equations (1) and (2). Moreover, I1 and I2 in Table 4

Major Axis
and b(x,y) is a thresholding function to obtain a binary image from the original one. Its value is ( , ) 1 b x y  if ( , ) 0 f x y  and 0 otherwise. The column on the right of Table 4 shows the variation of the parameters as computed to obtain the results of this paper. Most of them are expressed as relative changes with respect to the full scale.
However, the full scale value is not obvious and has to allow comparison between results obtained with different sensors and objects as well as to suit its meaning as maximum reference value. Since the objects in Figure 2a were built to fit in the sensor of Figure 1c while taking advantage of its spatial resolution, i.e., making them large considering the size of a taxel, the length of the square size L = 40.64 mm of this sensor was chosen as full scale reference for the parameters related to size or translation, while 180° was chosen as full scale value for the parameter Angle.
Finally, note that the variation of the parameter Aggregate Output is computed as relative to the expression (value (a) + value (b))/2 instead of a full scale figure. The reason is that the outputs of the sensors in Figure 1c,d have different units, as can be seen in Figure 3. Moreover, the experiments do not always have the same force range because the output of some taxels is saturated depending on contact area. Therefore, the best way to provide meaningful results valid to make comparisons is that displayed in Table 4. Nevertheless, note that this figure is large for small values of (value (a) + value (b))/2, and this should to be taken into account in the analysis of the results of the experiments.

Procedure
The setups in Figures 1a,b and the objects in Figure 2a are used, as mentioned in Section 2, to exert different pressure patterns on the tactile sensors. A piece of fabric has been added between the objects and the sensors to improve the contact. The goal is to evaluate the change of the parameters in Table 4 due to hysteresis, drift, mismatching and limited resolution. To assess the influence of hysteresis, the set up in Figure 1b has been used to exert an increasing and then decreasing force on the objects in Figure 2, once placed atop of the sensor. The ellipsoid parameters in Section 4.1 are then computed for every tactile image captured along the cycle and the results from frames corresponding to similar forces on the objects are compared. The variation of these parameters for the same load on the sensor is found in a loading-unloading cycle.
To estimate the impact of drift, the change of the parameters in Table 4 is registered when the load is kept in time. The set up in Figure 1b was used to exert a force atop of each object and the sequence 10 N → 20 N → 30 N → 40 N → 30 N → 20 N → 10 N (in the case of the bar the maximum possible force was 38 N) is followed. 255 samples were taken in 25 min. Similar experiments were carried out with the 5051/P1/3056T1/20 from Tekscan, where 900 samples were taken in 30 min. Although the drift of the latter is smaller than that of the former, it is not negligible and it is worth considering its effect on this study.
Regarding mismatching, since it is not possible to see its effects experimentally, we can use a mixed or indirect method to evaluate it. This consists in taking the output for ten uniform pressures, 2.1, 4.3, 6.2, 8.3, 9.7, 12.3, 13.7, 16.2, 17.7 and 20.1 psi, with the pneumatic set-up in Figure 1a. For each pressure, the output is equilibrated and the result is also saved. In this way we have ten raw plus ten equilibrated tactile images of a uniform pressure against the sensor surface. To obtain the data corresponding to different objects or geometries, a binary mask is applied to the previous tactile images. This mask is obtained from readings of the different objects on the sensor.
Finally, the last experiments were carried out to see the effect of the limited resolution of the sensor. Again, we try to see the consequence of this limitation on the parameters of Table 4. To do so, we use the commercial sensor to register the sensor output under different forces exerted with the setup in Figure 1b. Table 5. Tactile images and ellipse for the hysteretic cycle with the objects "ring", "star", "opener" and "bar" on the PCB based sensor. Then the resolution is lowered before computing the parameters in Table 4 by applying a filter to the image so that four adjacent taxels are merged and substituted by a single one whose output is the average of the outputs from the original taxels. This is done for a decrease of 50% and 25% of the spatial resolution.

Hysteresis
Since the hysteresis and non-linearity errors of the commercial sensor are very low, this section will show the results of the experiments carried out with the other sensor. Table 5 shows a few selected tactile images out of the whole set registered for the different objects in the above described experiments, as well as their corresponding ellipse. In addition, Figure 6 shows the results of such tests taking the whole set of collected data. The variations of the parameters in Table 4 are displayed in two types of chart for every parameter in the table. The graph at the top shows the variation (as expressed in Table 4) of the parameters along the hysteretic cycle for different objects in Figure 2a. The chart at the bottom summarizes the information in five significant values: maximum, minimum, mean, percentile 80 and percentile 60. Some features can be observed in Figure 6. Firstly, the variations are larger for small force values. This is mainly because the shape of the object footprint is not yet well defined, as can be seen in Table 5. The largest changes in the centroid location are for low forces, but they are small. The worst behavior is that of the opener in vertical position, worse than the opener in diagonal orientation, so we see that the location of the object in the sensor has influence. This is mainly due to the mismatching between different taxels. Regarding the Aggregate Output, note that it is computed in a different way and it is related to the reading and not to a kind of full scale reference. Therefore Figure 6c indicates a quite uniform behavior for changing forces. The area is little affected except for low forces. The variations of the object shape, i.e., the size of the axis, are slightly larger for the bar or the opener-like object, which have an oblong shape. The objects with radial symmetry like the ring and the star show the largest errors in orientation, as indicated by the parameter Angle.

Drift
The results for the drift are shown in Tables 6 and 7 for the object labeled as "opener" and for the PCB based sensor and the Tekscan sensor respectively. It is observed again that the drift is larger in the PCB based sensor and for pressure increments larger than for decrements. It can also be observed that the histograms are more symmetric in Table 6 when compared with those in Table 7.  Actually, positive changes at output are registered in Table 6 for almost all taxels, while negative differences are also detected in many taxels of Table 7. This is also noticeable when the absolute distance of both cases in Tables 6 and 7   Note that this can mask a larger effect of drift when an average is computed, as done in Figure 7 with respect to Figure 9, or in Figure 8c with respect to Figure 10c, but it can also be seen as a positive compensation effect when the parameters of Table 4 are obtained. This can explain the different behavior observed in Figures 8 and 10.   Generally speaking, a quite clear law is followed in Figure 8 in the sense that the differences decrease for higher pressures and also for decrements, while the curves are more uniform along x axis in Figure 10. An exception is found in the objects in diagonal position whose results are worse, i.e., the orientation of the object in the sensor also affects the result. In both cases a larger difference in the parameter Angle is observed for the ring and the star, objects with radial symmetry.

Mismatching
Regarding mismatching, the result for the opener is shown in Table 8. The same is done with the other objects and the difference of the parameters in Table 4 is computed as in the previous sections and is displayed in Figure 11. Besides the error magnitude, which will be used later to compare this source of error with the others, no other clear conclusions are extracted from Figure 9. Finally, since the purpose of this paper is to explore the impact of different error sources and not to compare the sensors themselves, the commercial sensor was used for the results of this section because of its very good performance in terms of linearity, hysteresis or drift allowing a certain isolation of the influence of mismatching.  Figure 11. Variations of the system level parameters caused by the commercial sensor mismatching.

Limited Resolution
The results of the experiment described in the final paragraph of Section 4.2 are illustrated in Table 9 for the star object, and the variations of the parameters in Table 4 are calculated and displayed in Figures 12 and 13 for all the objects and two resolution decrements. A quite uniform behavior of the error for different forces is observed in these figures, though it is higher for very low forces (the profile of the object is not well defined yet). Quite large errors are observed for the centroid location, Aggregate Output and Area, while the error is smaller in the shape estimation (ellipse axes) and very small in the orientation (Angle), except in the case of the star. For the latter, a significant impact of the spatial resolution decrement is observed.

Summary
As mentioned in the introduction, the goal of this paper is to see the influence of common non-idealities in certain high level parameters. Besides some partial conclusions previously provided, this section aims to compare the effect of the different sources of error. For this purpose, some data is summarized and rearranged in Tables 10 and 11. Moreover, the same information is shown graphically in Figures 14 and 15. These tables and figures display the relative variations caused by hysteresis, drift, mismatching, and limited spatial resolution on the parameters in Table 4 computed for the objects in Figure 2a. Table 11 and Figure 15 are obtained from the full set of data, while Table 10 and Figure 14 show the results from the percentile 80 of the data. Note from previous sections that large variations were found for small forces. Tactile images show that the profile of the objects was not well defined yet for these small forces, so the conclusions cannot be reliable enough. Therefore the discussion focuses mainly of the results obtained for the percentile 80 of the data to eliminate this effect and the outliers. In the following, every parameter in Table 4 is discussed.     Table 4 (maximum deviation).
Centroid (contact location): The largest variation is due to the limited spatial resolution. Note that the 50% lowering of resolution results in a sensor whose spatial resolution is still higher than that of the sensor based on PCB. The change in the centroid location for this reason is quite large in comparison with the other sources of error. The latter affect the result to a similar degree, generally speaking, except the drift of the commercial sensor that has hardly any influence.
Aggregate output (contact force): The hysteresis and drift of the PCB based sensor have a large effect, so the estimation of this parameter has a significant error with this sensor. The variation caused by the mismatching and drift of the commercial sensor are much smaller, but quite significant when compared to the effect on other parameters. It makes no sense to observe the effect of the limited spatial resolution here, because it introduces only a scale factor in all cases. Area (contact size): The largest variation is produced by the limited spatial resolution, and it is quite high. The drift of the PCB based sensor has a moderate influence, and that of the hysteresis is slightly smaller, while the drift of the commercial sensor has a small effect on this parameter. Moreover, mismatching has no effect at all if percentile 80 is considered because once the force is high enough, the object is well defined in the tactile image and the area does not change. If we consider the whole set of data it has a small influence.
Major and minor axes (contact shape and size): Here the largest variation is due to hysteresis, but the change due to mismatching is close, generally speaking. The error caused by the drift of the PCB based sensor is also quite similar but smaller, while the drift of the commercial sensor has a small impact. It is also worth highlighting that limited spatial resolution changes these results very little.
Angle (contact orientation): As mentioned in previous sections, the effect of the considered sources on the orientation is larger in objects with radial symmetry than that observed in objects with axial symmetry. The hysteresis has quite a high influence on this parameter but the effect of mismatching is even larger. The impact of drift is smaller but not negligible and it is quite close in both the commercial and the PCB based sensor. The resolution has a similar influence but the change is larger in the star than in the ring, which seems logical.

Conclusions
As stated above, the goal of this paper is to find out to what extent some significant parameters in manipulation with robotic hands, obtained from tactile sensors, are affected by the main limitations and errors of these sensors. An error-prone, low cost PCB sensor was used in the experiments, as well as a commercial one. In summary, the hysteresis and drift of the simple low-cost PCB based sensor cause error mainly in the estimation of the contact force and also of the contact shape (major and minor axes). However, the influence of the mismatching of the commercial sensor is similar in the estimation of the contact shape. On the other hand, the error in the orientation of objects with axial symmetry is small in all cases, while it can be large if the object has radial symmetry (though it can be less important in this case). In addition, the impact of the limited spatial resolution is far from being negligible when the contact location (centroid) and size (area) are estimated. We can conclude that there are sources of error which are difficult to reduce in the case of robotic hands such as the mismatching and others accepted such as limited spatial resolution, that have a similar impact on the high level parameters used in robotics to other limitations like hysteresis or drift. It seems that simple low cost sensors are good enough to provide information related to spatial distribution like orientation, shape or contact location, though are not good at providing other information like contact force, which is just the aggregation of the output of all taxels.