Artificial tactile sensors which aim to mimic human discrimination capabilities should encode information correlated with the stimulus spatial features, with its motion dynamics as well as with contact mechanics. Roughness is a fundamental feature for texture perception [1
], which has been associated with the spatial modulation of the used stimuli (i.e. “surface coarseness”) [4
]. In experiments on human perception of tactile roughness the type of used surfaces often have patterns (gratings or rising dots) with features that can be independently varied in size and spacing [5
]. This way, unlike for natural surfaces in which the spatial pattern features vary randomly, the physical characteristics of the explored surface, on which roughness perception is based, can be studied and identified.
The physical determinant of perceived roughness is not yet fully understood [2
] and there is a varied set of spatial features that should be taken into account for studies on roughness perception (e.g., using ridged stimuli: groove width, ridge width, ridge orientation, ridge height, material compliance, surface lubrication and fine finishing, etc.). In human psychophysical experiments, for example, some groups highlighted the presence of a relatively narrow region where the sense of roughness increases together with the groove width of ridged stimuli, followed by a flattened perception in case of very coarse gratings (up to 8.5 mm of groove width) [7
]. In parallel to this, using embossed dots, some researchers presented monotonic functions of roughness and dots spacing [5
], while an inverse “U” shape was shown in [8
Considering dynamic exploration of extremely fine textures, various researchers showed that humans can detect even up to microtextures [9
], highlighting the role of fingerprint ridges as vibration promoters [10
] and considering the Pacinian Corpuscles as vibration detectors [11
]. Some groups joined the Katz's duplex theory
considering vibrations useful for revealing fine forms, and a spatial mechanism (i.e. the static image of the contact between the texture and the finger) as the basis for coarse surfaces roughness perception [12
]. Importantly, in the last decade, other groups proposed and gave evidence to a unified peripheral neural mechanism highlighting the role of SA1 afferents with respect to the other mechanoreceptors [13
The understanding of the neural mechanisms underlying roughness encoding is in progress, however evidence was given to the fact that temporal frequency changes of tactile information play a major role in roughness perception in humans [5
]. Finite element analyses using human finger model during dynamic touch showed that spatial information of the textured surface are related to temporal frequency changes at the position of tactile receptors [14
]. In touch activities, if humans have the ability to estimate somehow the relative hand velocity v
between the textured surface and the exploring finger, the spatial period Δp
of the surface can be perceived by detecting the temporal frequency of the vibration [15
], such that:
The findings and debate of researchers on human touch are directly linked with the development of artificial tactile sensors, which is one of the chief challenges in robotics. Many technologies have been investigated and can be analysed in comprehensive reviews on the topic [16
]. For the above reasons, with regards to the many reported efforts to reproduce human capability to detect texture, the developed sensors were mainly based on the analysis of the vibration gathered during dynamic exploration or on the contact imaging [18
] by means of static indentation. An approach was to develop a finger-like multilayered texture sensor integrating five strain gauges for identifying the difference in roughness, softness and frictional properties of various materials [19
]. Employing such device, the texture information of a surface was quantitatively detected by estimating the vibrational frequency excited by indenting and sliding a periodic stimulus with spatial wavelength in the millimiters range [20
]. A similar method was previously shown in [21
] for finer surfaces. Another noticeable solution was presented in [22
], where a spatial filter function was used adaptively depending on sensor-stimulus relative motion parameters, thus pointing out the centrality of a spatio-temporal approach in tactile sensing. Hosoda and colleagues developed a soft fingertip with randomly distributed strain gauges and PVDF films at different depths [23
], allowing for discrimination of five different types of materials. Other recent biomimetic fingertips focused on the transduction properties, which could be either acoustic [24
] or electrical [25
], of the packaging materials for converting the surface features of the explored textures into recorded vibrations. Finally, one of the most recent developments is represented by a high-resolution thin film sensor built by Maheshwari and Saraf [26
] by means of a layer-by-layer self-assembly technique, that responds to an applied force either with electroluminescent emissions or with a change in current density. A charge-coupled device (CCD) camera was used to capture the electroluminescent emissions from the sensor providing imaging stress distribution with spatial resolution of about 40 μm.
In this work, the investigated artificial tactile sensor integrates a MEMS array having a number of sensing elements (16 channels in about 20 mm2
, i.e. 0.8 channels/mm2
) similar to the innervation density of Slowly Adapting type 1
(SA1) mechanoreceptors in the hand (about 1 unit/mm2
]. The technological approach is based on a 3D MEMS core unit [28
] with a soft and compliant packaging. As previously demonstrated, the microsensor can be integrated with a packaging architecture resulting in a robust and compliant tactile sensor for application in artificial hands, while sensitive enough to detect slip events, showing that silicon based tactile sensors can go beyond laboratory practice [29
In the long term, the presented artificial approach aims, on one side, at developing a device capable of mimicking the texture discrimination properties of the human hand and which can be integrated in an anthropomorphic artificial hand, while on the other it is intended as an artificial model to be used as a test bench for neuroscientific hypotheses describing the mechanisms of roughness perception. This long term objective gets inspiration from the above mentioned work of Yoshioka and colleagues [13
], in which it was shown that spatial variation in the firing rates of SA1 units-only can account for roughness perception even when the explored texture is finer than the SA1 innervation density.
In order to go in such direction, the specific objective of the current work was to gather the vibrations which are supposed to be the basis for the encoding of roughness in dynamic touch, as well as to perform static imaging of the contact with the same array of tactile sensors. The present experimental analysis evaluates whether there is a substantial processing advantage in using more than one output of the array for finding out the common principal frequency produced during dynamic stimulus presentation. This way, by merging the estimation of the common frequency detected by more than one sensor unit together with the knowledge of the sliding velocity of the applied stimulus, texture related features could be extracted. The suitability of the sensor for both static contact imaging and vibration detection was evaluated by means of an experimental protocol containing both motionless and dynamic contact phases involving forces and velocities in the range of those used by humans in discriminative touch.
The paper is organized as follows. In Section 2 the design of the sensor array is shown, describing the elementary MEMS unit, the packaging and the readout electronics. In Section 3 the experimental protocol and the used data analysis methods are presented. Section 4 shows the experimentation with the array prototype, which has been carried out with ridged stimuli sliding at constant velocity and regulated normal force after and before a static indentation phase. Finally, results are discussed in Section 5 and future work insights are given in the Conclusions.
The experimental results shown for dynamic artificial touch with medium-coarse periodic gratings demonstrated the perfect coherence between the principal frequency commonly revealed by the packaged MEMS sensor units and the expected one, as shown in Table 3
and Figure 7
, as well as the consistency between the surface geometry and the static image of the stimulus-sensor interface.
Looking at the background of neurophysiological and psychophysical touch studies briefly reported in the Introduction, the technological and the signal processing outcomes of this work may be classified as a successful preliminary attempt to artificially achieve roughness encoding in case of medium-coarse patterning, i.e. a deterministic link (see Table 3
) was obtained between the “spatial coarseness” of the presented stimuli and the features extracted from the sensor outputs.
These results pointed out the better processing quality guaranteed by using structured information from different units of a tactile sensor array instead of naïve Fourier analysis separately on each channel, overcoming frequency discretization limitations. These limitations are shown in Figures 4
, which differ both in the time window length used for FFT and in the grating periodicity. The latter is the reason for the 1.4 Hz difference between the nominal frequencies, which could not be detected with FFT due to the low resolution of the FFT in the relevant frequency range (width of bars ∼0.5 Hz in Figure 4 vs.
∼2.5 Hz in Figure 5
). As a consequence, using the naïve FFT approach to retrieve the frequency in a 0.35 s time window for both data series, would result in the two gratings being not distinguished, as shown in the respective plots of Figure 7
. On the contrary, with the least squares fitting procedure the separation was well feasible, and the common frequency expected when indenting and sliding at constant speed periodic ridged surfaces across an array of sensors was accurately estimated. The technological approach together with the proposed frequency estimation method guaranteed an error from 1.7% down to 0.5% over the range of spatial frequencies considered, independently of the combination of MEMS sensor units used (see Table 3
). Moreover, as shown in Figure 7
, limiting the evaluation to fixed size time windows reduced the accuracy somewhat, but the method stayed stable down to 0.4 s window size, making it potentially suitable for most near real-time settings. The applied method revealed to be robust even if, in addition to the observable principal frequency shift associated to the combination of the used grating and stimulus sliding velocity, the signal power had overtones (the first three or four harmonics of the fundamental frequency) introduced by both the non-linear packaging and the sharp edges of the periodic ridged surfaces. On the contrary, the fitting based on Fourier analysis resulted in an oscillatory behavior of the error respect to the observation window length. Further stability and precision with the gradient descent fitting method could be gained by taking into account all four MEMS sensors and tuning the sampling rate according to the target application.
As depicted in Figure 8
and confirmed by the calculated average Pearson cross-correlation coefficients, the gathered data had high repeatability across different runs of the same experimental conditions. Furthermore, as seen in Figure 8
already, MEMS sensor 1 produced higher voltage amplitudes, leading to a better Signal-to-Noise (S/N) ratio, which in turn caused the higher correlation between runs with respect to MEMS sensor 2. This effect may be associated to the shape of the compliant packaging, which could induce higher stresses in piezoresistors of the sensor unit located at the leading edge. Reversing the scan direction (not shown) exchanged the roles of MEMS sensors 1 and 2 in this regard. Moreover, it is noticeable to observe the excellent similarity between the plots shown in Figure 8
and the plot of Figure 9 (a)
. These graphs only differ for the stimulus translational speed and thus result in a compression in the time scale during phase C.
The modulation of the principal frequency due to the variation of the stimulus can be appreciated in the time domain plots of Figure 9
. Moreover, as detailed in Figure 10
, couples of piezoresistors of a sensor unit which are located one in front to the other along the direction of motion [piezoresistors 1 and 3 in Figure 1(b)
] responded with opposite sign to the stimulus. Therefore, even with packaged silicon sensors and dynamic stimulations, the symmetries of the static calibration matrix observed in [30
] for the bare MEMS sensor were still present in this work.
Finally, as regards the consistency between the surface geometry and the static artificial touch representation, it is remarkable to observe the output signals variations relatively to the steps between phases A (starting of data acquisition) and B (sensor loading) and between phases D (steady state after stimulus sliding) and E (sensor unloading). Figures 3
point out that the step heights varied between different runs depending on the used grating (but not on the velocity). This was due to the fact that a variation of the grating periodicity modified the portion of the ridge under each MEMS unit, being the initial and final position of the stimulus carrier always the same for all runs during phase C.
6. Conclusions and Future Work
The experimental analyses performed in this work demonstrated the suitability of the developed tactile sensor for revealing medium-coarse spatial features of the explored surface, both with dynamic and static stimulation. Future work will focus on performing similar experiments incorporating the developed technology in a bio-inspired mechatronic finger. Given that the feasibility of using a polyimide thin protective layer has been shown in this investigation, the actuated finger may use a polyimide glove (mimicking the human stratum corneum) for preventing the sensor to be worn or damaged by water or grit. Moreover, because of the possibility to integrate the readings from the array with proprioception information during active touch tasks, the combination of information regarding the estimated common frequency and the velocity of the finger could solve Equation (1)
and provide quantitative measurements revealing texture properties of the explored stimuli. Investigations will also be performed in implementing processing strategies to separate the velocity and periodicity information contained in Equation (1)
directly from the measurements of the array, thus avoiding the need to use the knowledge of the stimulus sliding velocity (in case of passive touch experiments) or proprioception information from an actuated finger (in case of active touch ones). Experiments will be performed with other stimuli, addressing not only a medium-coarse spatial periodic pattering, but also more general fine textures (e.g. sandpapers, gratings with oblique or aperiodic ridges or 2D patterning, …) and the frequency content due to the kind of material. In that case, the focus could move from principal frequency analysis to spectral analysis over the full frequency range, or to wavelet transform if the frequency content is supposed to change with respect to time and/or stimulus-sensor relative positioning. Moreover, the fact that the MEMS sensor is triaxial may be exploited in future work with stimuli having 2D patterning: in this paper, the raw sensor outputs were directly analyzed guaranteeing great accuracy in principal frequency retrieval without encoding the force vector at MEMS-packaging interface or at packaging-stimulus interface.
These planned experiments will require some modifications to the packaging design (e.g. lower thickness, material with different hardness or viscosity, introduction of fingerprints, etc.) in order to achieve even a higher sensitivity and selectivity for each MEMS unit [31
] and a reduction of the low-pass spatial filtering effect introduced by the materials embedding the sensor [35
] while still providing robustness for application in artificial hands dexterously interacting with the environment [29
Finally, future investigations will experiment with artificial tactile sensors the unified paradigm proposed by Yoshioka and colleagues [13
] for the perception of fine and coarse textured surfaces, in order to go towards a common theory for human and robot mediated coding and decoding of tactile stimuli.