- freely available
Sensors 2009, 9(05), 3161-3183; doi:10.3390/s90503161
Published: 27 April 2009
Abstract: A compliant 2×2 tactile sensor array was developed and investigated for roughness encoding. State of the art cross shape 3D MEMS sensors were integrated with polymeric packaging providing in total 16 sensitive elements to external mechanical stimuli in an area of about 20 mm2, similarly to the SA1 innervation density in humans. Experimental analysis of the bio-inspired tactile sensor array was performed by using ridged surfaces, with spatial periods from 2.6 mm to 4.1 mm, which were indented with regulated 1N normal force and stroked at constant sliding velocity from 15 mm/s to 48 mm/s. A repeatable and expected frequency shift of the sensor outputs depending on the applied stimulus and on its scanning velocity was observed between 3.66 Hz and 18.46 Hz with an overall maximum error of 1.7%. The tactile sensor could also perform contact imaging during static stimulus indentation. The experiments demonstrated the suitability of this approach for the design of a roughness encoding tactile sensor for an artificial fingerpad.
Artificial tactile sensors which aim to mimic human discrimination capabilities should encode information correlated with the stimulus spatial features, with its motion dynamics as well as with contact mechanics. Roughness is a fundamental feature for texture perception [1,2,3], which has been associated with the spatial modulation of the used stimuli (i.e. “surface coarseness”) . In experiments on human perception of tactile roughness the type of used surfaces often have patterns (gratings or rising dots) with features that can be independently varied in size and spacing [5,6]. This way, unlike for natural surfaces in which the spatial pattern features vary randomly, the physical characteristics of the explored surface, on which roughness perception is based, can be studied and identified.
The physical determinant of perceived roughness is not yet fully understood [2,4] and there is a varied set of spatial features that should be taken into account for studies on roughness perception (e.g., using ridged stimuli: groove width, ridge width, ridge orientation, ridge height, material compliance, surface lubrication and fine finishing, etc.). In human psychophysical experiments, for example, some groups highlighted the presence of a relatively narrow region where the sense of roughness increases together with the groove width of ridged stimuli, followed by a flattened perception in case of very coarse gratings (up to 8.5 mm of groove width) . In parallel to this, using embossed dots, some researchers presented monotonic functions of roughness and dots spacing , while an inverse “U” shape was shown in .
Considering dynamic exploration of extremely fine textures, various researchers showed that humans can detect even up to microtextures , highlighting the role of fingerprint ridges as vibration promoters  and considering the Pacinian Corpuscles as vibration detectors . Some groups joined the Katz's duplex theory considering vibrations useful for revealing fine forms, and a spatial mechanism (i.e. the static image of the contact between the texture and the finger) as the basis for coarse surfaces roughness perception . Importantly, in the last decade, other groups proposed and gave evidence to a unified peripheral neural mechanism highlighting the role of SA1 afferents with respect to the other mechanoreceptors [13,1].
The understanding of the neural mechanisms underlying roughness encoding is in progress, however evidence was given to the fact that temporal frequency changes of tactile information play a major role in roughness perception in humans . Finite element analyses using human finger model during dynamic touch showed that spatial information of the textured surface are related to temporal frequency changes at the position of tactile receptors . In touch activities, if humans have the ability to estimate somehow the relative hand velocity v between the textured surface and the exploring finger, the spatial period Δp of the surface can be perceived by detecting the temporal frequency of the vibration , such that:
The findings and debate of researchers on human touch are directly linked with the development of artificial tactile sensors, which is one of the chief challenges in robotics. Many technologies have been investigated and can be analysed in comprehensive reviews on the topic [16,17]. For the above reasons, with regards to the many reported efforts to reproduce human capability to detect texture, the developed sensors were mainly based on the analysis of the vibration gathered during dynamic exploration or on the contact imaging  by means of static indentation. An approach was to develop a finger-like multilayered texture sensor integrating five strain gauges for identifying the difference in roughness, softness and frictional properties of various materials . Employing such device, the texture information of a surface was quantitatively detected by estimating the vibrational frequency excited by indenting and sliding a periodic stimulus with spatial wavelength in the millimiters range . A similar method was previously shown in  for finer surfaces. Another noticeable solution was presented in , where a spatial filter function was used adaptively depending on sensor-stimulus relative motion parameters, thus pointing out the centrality of a spatio-temporal approach in tactile sensing. Hosoda and colleagues developed a soft fingertip with randomly distributed strain gauges and PVDF films at different depths , allowing for discrimination of five different types of materials. Other recent biomimetic fingertips focused on the transduction properties, which could be either acoustic  or electrical , of the packaging materials for converting the surface features of the explored textures into recorded vibrations. Finally, one of the most recent developments is represented by a high-resolution thin film sensor built by Maheshwari and Saraf  by means of a layer-by-layer self-assembly technique, that responds to an applied force either with electroluminescent emissions or with a change in current density. A charge-coupled device (CCD) camera was used to capture the electroluminescent emissions from the sensor providing imaging stress distribution with spatial resolution of about 40 μm.
In this work, the investigated artificial tactile sensor integrates a MEMS array having a number of sensing elements (16 channels in about 20 mm2, i.e. 0.8 channels/mm2) similar to the innervation density of Slowly Adapting type 1 (SA1) mechanoreceptors in the hand (about 1 unit/mm2) . The technological approach is based on a 3D MEMS core unit  with a soft and compliant packaging. As previously demonstrated, the microsensor can be integrated with a packaging architecture resulting in a robust and compliant tactile sensor for application in artificial hands, while sensitive enough to detect slip events, showing that silicon based tactile sensors can go beyond laboratory practice .
In the long term, the presented artificial approach aims, on one side, at developing a device capable of mimicking the texture discrimination properties of the human hand and which can be integrated in an anthropomorphic artificial hand, while on the other it is intended as an artificial model to be used as a test bench for neuroscientific hypotheses describing the mechanisms of roughness perception. This long term objective gets inspiration from the above mentioned work of Yoshioka and colleagues , in which it was shown that spatial variation in the firing rates of SA1 units-only can account for roughness perception even when the explored texture is finer than the SA1 innervation density.
In order to go in such direction, the specific objective of the current work was to gather the vibrations which are supposed to be the basis for the encoding of roughness in dynamic touch, as well as to perform static imaging of the contact with the same array of tactile sensors. The present experimental analysis evaluates whether there is a substantial processing advantage in using more than one output of the array for finding out the common principal frequency produced during dynamic stimulus presentation. This way, by merging the estimation of the common frequency detected by more than one sensor unit together with the knowledge of the sliding velocity of the applied stimulus, texture related features could be extracted. The suitability of the sensor for both static contact imaging and vibration detection was evaluated by means of an experimental protocol containing both motionless and dynamic contact phases involving forces and velocities in the range of those used by humans in discriminative touch.
The paper is organized as follows. In Section 2 the design of the sensor array is shown, describing the elementary MEMS unit, the packaging and the readout electronics. In Section 3 the experimental protocol and the used data analysis methods are presented. Section 4 shows the experimentation with the array prototype, which has been carried out with ridged stimuli sliding at constant velocity and regulated normal force after and before a static indentation phase. Finally, results are discussed in Section 5 and future work insights are given in the Conclusions.
2.1. MEMS Sensor Array
The elementary cell of the array was the 3D MEMS sensor described in , shown in Figure 1, which has a high aspect ratio 3D structure (1.5 mm×1.5 mm×625 μm). In the bare configuration of the sensor, the cylindrical mesa, located at the center of the cross-shape tethers, transmits an externally applied force to the sensor inducing stresses in the four tethers where four p-type piezoresistors are implanted. The fractional change in resistance ΔR/R of each piezoresistor of the microsensor is proportional to the longitudinal and the transversal stress components, while the design of the sensor is such that the transversal stress component in the implanted piezoresistors is neglectable with respect to the longitudinal one. In the current experimentation four microsensors were bonded on a silicon carrier chip connecting the 9 NiAu pads of each microsensor by means of a micro-soldering paste by using flip-chip bonding method. As stated in the Introduction in this study attention was paid in developing an array with a density of sensing elements that could be compared to the innervation density of Slowly Adapting type 1 (SA1) mechanoreceptors in the hand (≈ 1 unit/mm2) . The tactile sensor array, depicted in Figure 1, had 16 channels as total tactile sensor outputs. It had a pitch of 2.3 mm (indicated by ΔX in Figure 1) for technological reasons, i.e. mainly because of the operation room needed for the flip-chip bonding method and layout of the carrier chip. The resulting area of the sensing array was of 21.16 mm2 inscribing each MEMS unit inside a square of area 5.29 mm2. The silicon carrier chip was wire bonded by means of 25 μm Al wires to a Printed Circuit Board (PCB) in order to connect the array to the external instrumentation. The perimeter of the array was secured with a two component epoxy glue in order to protect the wire bonding and to improve the stability of the silicon carrier chip.
The MEMS tactile array was packaged with a synthetic material (as explained in the next section) that mechanically filters the external applied load and creates a distribution of stresses in the new configuration of microsensor and packaging, with respect to the externally applied stimulus. For the investigation reported in the present work, the outputs of the piezoresistors (i.e. the sensing elements) have been analyzed directly for their dynamic behaviour, whilst they have not been used to extract the three components of an applied force. This avoided to address the calibration of each MEMS before packaging, as done with this device in , or after packaging as for example performed in  with a different sensor together with the introduction of an analytical model for point contact loads.
The packaging of the bare silicon sensors array was developed so that the resulting tactile sensor could have compliance and softness characteristics inspired to those of the human fingerpad. Previous investigations for the application in an anthropomorphic artificial hand were considered, in which it was demonstrated that it is possible to integrate the silicon microsensor in a soft and compliant, but robust packaging . In particular, the round shape of the packaging of the array was chosen based on the anthropomorphic features of the distal phalanx of the cybernetic hand CyberHand [32,33]. In parallel, a suitable curved geometry was identified in order to increase the portion of load gathered by the sensors in case of contact with a planar textured surface, as pointed out in . As shown in Figure 1, the dimensioning parameters for the packaging where r0 and d, which were set to 8 mm and 1.3 mm, respectively, for obtaining adequate sensitivity as well as partially overlapping sensing ranges between nearest-neighbour MEMS units and acceptable low-pass spatial filtering effect  with respect to the used stimuli.
The 2×2 array was packaged with polyurethane (Poly 74-40, PolyTek, USA) and an outer thin protective layer of polyimide having thickness of 0.05 mm and shore A 82 hardness (ST1882, Stevens Urethanes, USA) in order to prevent the inner packaging from wearing. In fact, preliminary trials demonstrated that the ridged stimuli can damage the packaging of the array. In the present study, a type of polyurethane with shore A 40 hardness (instead of the previous shore A 45 ) was used attempting a step forward in human finger mimicry. Moulds hosting the array were built with rapid prototyping resin using a 3D printer, and the liquid part A and part B were poured immediately after being mixed and degassed. The polyimide layer was applied after polyurethane curing and showed excellent adhesion provided that the air between the cover and the thin sheet was removed. Moreover, the protective layer was secured by means of a frame also built in rapid prototyping resin.
2.3. Readout Electronics
Each piezoresistor was connected in series to a surface mount resistor (R1 … R16) located on the designed PCB, as shown in Figure 2(a). The values of R1 … R16 were all set to 820 Ω, which is close to the mean resistance of the piezoresistors of the 2×2 array, thus almost achieving sensitivity maximization from the quarter bridge voltage divider. The used quarter bridge topology produces a variation of the acquired voltage proportional to the fractional change in resistance of each piezoresistor. Capacitors (C1 … C16, all having capacitance of 1 μF) were placed in parallel to each completing resistor, resulting in a low-pass single pole filter at about 390 Hz (i.e. ) for reducing the noise level at frequencies outside the band of interest. The piezoresistor-resistor arms were supplied by means of a 5V DC regulated voltage, and the node between each piezoresistor and the completing resistor was directly acquired without pre-amplification by means of a 16-channel 24-bit Analog to Digital Converter (ADS1258, Texas Instruments). Each channel was sampled at a frequency of 241 Hz, which could be varied via software up to 24.7 kHz selecting a subset of channels and changing the conversion options of the used ADC. Considering the chosen sampling frequency and the cut-off of the RC low-pass filter, the fulfillment of the Nyquist theorem for aliasing avoidance mainly relied on the expected baseband properties of the gathered signals (refer to Figures 4 and 5 in the following for a qualitative validation of such assumption). Digital data transfer between the ADC and the acquisition system was performed by means of SPI protocol. The data acquisition system was based on Field Programmable Gate Array (FPGA) technology (CycloneII, Altera) and had a 64 bit hardware timer running at 50 MHz, so that the acquisition of each channel had a time reference with resolution of 20 ns and practically unlimited length. Acquired data was buffered by a soft-core processor (NiosII, Altera) instantiated onboard the FPGA and transmitted at the end of each session to a Personal Computer with JTAG UART protocol, as shown in Figure 2(b). The storage of data was allowed within the Nios II Integrated Development Environment by enabling the option “Filing System to open files on the PC” in the Altera Host Based File System.
3.1. Experimental Protocol
The packaged array was mounted on a mechatronic tactile stimulator capable of indenting the sensor with force feedback control and stroking a stimulus over it with precise position control. The configuration of the array/stimulus interface and the experimental protocol are reported in Figure 1(d) while Figure 2(b) shows a diagram of the overall experimental set-up.
Four types of stimuli were built with rapid prototyping resin material, with spatial periods Δp varying from a minimum of 2.6 mm to a maximum of 4.1 mm, as detailed in Table 1.
In order to evaluate whether the sensor outputs could be processed for automatically recognizing the instant of contact, data acquisition started prior to the phase during which the stimulus contacted the sensor array (phase A). In a second phase of the experiment, the stimulator was commanded to contact the tactile sensor (phase B). The sensor array was loaded setting at 1 N the reference of the normal force feedback controller given that such value is in the middle of the force range used by humans in fine forms discrimination during active dynamic touch experience .
The loading resulted in a contact spike in the signals gathered from the MEMS array. The target force level was held for 1 s. After that, the sliding of stimulus started (phase C) along the x-axis [piezo1-piezo3 direction of Figure 1(b)] while maintaining enabled the force feedback controller, thus obtaining a stimulation with normal force held at 1 N and tangential force depending on the contact mechanics and on the motion dynamics.
Three different translational velocities (15 mm/s, 30 mm/s and 48 mm/s) of the stimulus were chosen for overlapping with the range commonly used in related neurophysiologic studies . The direction of motion (along the x-axis as shown in Figure 1) was always the same, as well as the sign of velocity and the starting absolute position. The sliding was applied for 60 mm, providing dynamic stimulations of 4 s, 2 s or 1.25 s depending on the applied velocity. At the end of the sliding motion there was a steady state of 1 s at 1 N (phase D) and, finally, the tactile sensor array was unloaded (phase E). The initial and final static phases of the protocol were performed with repeatable conditions in order to enable analyses on static imaging capabilities of the sensor in addition to the dynamic behaviour investigation.
3.2. Common Frequency Detection
During the sliding of the periodic ridged stimulus over the packaged sensor array (phase C; see Figure 1), the output signal mi,j from the i-th piezoresistor of the j-th MEMS unit of the 2×2 array clearly showed a principal frequency component f, while the contact and the unloading operations could be revealed by the first spike and the last step in the outputs, as shown in Figure 3.
Defining as v the sliding velocity of the grating, the relationship reported in Equation (1) is expected for f.
In order to be able to detect the common frequency between all the fitting curves of the output signals, attention was paid in respecting the Nyquist condition for the sampling frequency with a safety factor, such that at least 13 samples per period were guaranteed even in the worst case stimulation conditions (i.e. minimum grating periodicity Δpk and maximum speed of the stimulus, as shown in Table 2).
To ensure data quality, a simple procedure was implemented to remove prior to processing data that was not useful for the dynamic analysis of the recorded signals. The redundancy in the system was used by jointly observing the outputs of two piezoresistors from different MEMS sensors. Since the contact spike mentioned in Section 3.1 was less pronounced for some piezoresistors than for others, the best defined spike was extracted from either one of the two time series.
To this end:selects the sensor s whose data was used for initial spike detection. Here, tinit = 0.8s is a time threshold before which the spike is expected to appear. The location of the spike tspike is then detected by: and used for both time series. According to the measurement protocol, the movement starts at t = 1.0s after the spike, and ends after 60mm of stimulus have been traversed. The effective sample length was set to Leff = 55mm for pre-processing operations, in order to avoid introducing invalid data in case of inherent timing variations. The start and the end of the valid range thus were: with ν denoting the sliding velocity of the grating.
Ten different combinations of sliding velocity and grating periodicity were formed, as detailed in Section 4, Table 3. Four measurement runs of the sensor array were carried out for each combination, yielding a total of 40 runs. In the following, these data series are referred by number, with run 1 to 4 belonging to combination one, run 5 to 8 to combination 2 and so forth. Figure 3 shows typical outputs of the procedure, which performed flawlessly on all 40 data sets.
An optional pre-processing step consisted in chopping the time series into time windows of size w. In most practical applications with gradually or abruptly changing surface characteristics, a trade-off will have to be found between the response delay given by the finite window size, and the accuracy of retrieval. Here this is investigated with non-overlapping windows to minimize redundancy, while in practice one could probably use strongly overlapping windows and thus higher update rates, if enough computation power is available.
Fast Fourier Transform
The most important prerequisite for advanced use of the developed sensor array was to establish a robust retrieval procedure for the fundamental spatio-temporal frequency of the system. As a first step, the two selected piezoresistors voltage time series underwent a Fast Fourier Transform (FFT) separately, to find a first guess for the fundamental frequency f, namely at the maximum of the periodogram. Note that by selecting the maximum peak as the fundamental frequency using naïve Fourier analysis, a discretization error of up to:occurred, where fC = 241 Hz was the channel sampling frequency and N the number of data points in the time window. The first guess magnitude of the fundamental oscillation was also difficult to read from the spectrum, because it would have to include contributions from the slopes surrounding the central peak. The solutions implemented in the following overcame this inconvenience. While it would certainly improve accuracy to average the contributions surrounding the fundamental frequency peak and/or to take into account overtones that often can be seen in the spectra, this procedure would involve several heuristic decisions about thresholds and boundaries.
Least squares fitting
In order to overcome to the discretization problem mentioned above, during dynamic stimulation each sensor output was fitted with a sine wave by using Equation (7):where:
mi,j is the signal obtained from the i-th piezoresistor of the j-th MEMS unit of the 2×2 array;
Bi,j and Ai,j are the offset and the amplitude of the sine waves used for fitting each mi,j;
– ti,j is an offset time which well fits the sine waves with data acquired during the exploration phase;
f is the common principal frequency coming out from the output signals using the analysis described below; observe that f is expected to be the same for all the outputs of the sensor array.
Therefore, a simpler second step was chosen, where a function of the form:was defined to be fitted to each channel's time domain data (see also Equation (7)). This was done by performing a gradient descent on the error function (considering piezoresistor 1 of MEMS sensors 1 and 2) overall the runs of a same combination of grating and velocity: with k running over all data points in the chosen time window, tk the sampling instants and mi,j(tk) the signal obtained from piezoresistor i of unit j of the array. Thus there were seven fitting parameters: A1,1, A1,2, B1,1, B1,2, t1,1, t1,2, and f. The purpose of this procedure was to both remove the discretization errors of the FFT, and introduce a priori information, because all the sensors were dragged over the same physical surface at the same speed, and then the same fundamental frequency was expected. It is possible to extend this method to include all valid piezoresistor readings from all sensors, if additional accuracy is required. To test this retrieval procedure, data from all measurement series was processed, averaging over the four measurements of each configuration of grating width and velocity. The data window width w was varied from 0.2 s to 1.0 s. For each w, the start of the time window was stepped through from tstart to tend-w in steps of 50 ms.
3.3. Error Parameters and Repeatability
The RMS error between the estimated frequency and the nominal one was used as a quality index for comparing the FFT results with the fitting procedure described above, i.e.:where the subscript method may be FFT or LSq depending on the usage of Naïve Fourier analysis or time domain least squares fit, respectively, for estimating the principal frequency. Moreover, n loops over all time windows in a measurement run, and q over all four runs belonging to parameter combination C and window size w. Also, a relative error parameter was used by dividing the RMS error of Equation (10) by the nominal frequency fnominal(C) and expressing the result as a percentage.
Repeatability was checked by pairwise cross-correlation of the measurement timeseries of piezoresistor i of MEMS unit j during the stimulus sliding phase (cf. Figure 3) of the four runs sharing one parameter combination C, and averaging the results. Hence, with l and p denoting two of those four runs:is the average Pearson cross-correlation coefficient. Measurements m had to be shifted by up to half a cycle relative to each other to account for phase differences due to the lack of synchronization between the starting of data saving and the starting of stimulation between different runs. Therefore, the inner sum runs over the remaining overlap region ta to ta+k, for which the mean signal m̄ and the standard deviation σ are calculated.
4. Experimental Results
This Section reports the experimental results obtained indenting and sliding the used ridged stimuli according to the parameters given in Table 2. The first part reports the preliminary naïve Fourier analysis which was performed in the process for establishing a robust retrieval procedure for the principal frequency induced by the grating spatial periodicity and sliding speed. Those preliminary results, as expected, were affected by considerable and oscillatory discretization errors depending on the chosen observation window. The second part shows the results with the proposed least squares fitting procedure, which guaranteed very high accuracy and quite fast error convergence increasing the observation window. Furthermore, the results with the least squares fitting are compared with the Fast Fourier Transform ones. Qualitative and quantitative evidence of data repeatability is given in the third part. Finally, the last subsection concerns results on the static imaging capabilities as another major feature of the designed sensor.
Fast Fourier Transform
The preliminary naïve Fourier analysis showed a considerable discretization error, according to Equation (6). This error can be quite significant for small windows, since the fundamental frequencies considered lie in the order of 1 to 10 Hz in this experiment. Figure 4 shows a typical Fourier analysis covering the full range of a sliding measurement (phase C, depicted in Figure 1). As a rather extreme example, the results using FFT with a 0.35 s time window analysis can be seen in Figure 5.
Least Squares Fitting
Figure 6 shows a graphical representation of the fitting procedure using the maximum allowed time window for the considered run. Such figure clearly shows the retrieval of the principal frequency coming out from the used combination of stimulation parameters. The frequency estimate errors, defined in Section 3.3 for comparing the FFT results with the fitting procedure described at the end of Section 3.2, are shown in Figure 7 and Table 3. Results from all windows and experiments were averaged for each point in the graphs.
As expected, errors in the initial guess fFFT ranged from about to almost zero, as the pattern of FFT-frequencies moved over the nominal frequency for each setup. However, the second estimation step using the least squares fit was very stable and converged for almost all windows larger than 0.4s. Using the entire available time series for each experiment, about 1.5s to 5s, leaded to the average errors shown in Table 3. To check consistency, results using piezoresistor 1 from MEMS sensors 1 and 2 in Equation (9) were compared to those using piezoresistor 1 from MEMS sensors 2 and 4. The errors obtained seem to agree very well, as shown in Table 3.
Figure 8 shows plots of the same channels within different runs having the same experimental conditions. Moreover, the cross-correlation coefficients defined in Section 3.3 confirmed a high degree of repeatability within one set of parameters C.
As an example, for piezoresistor 1 of MEMS sensor 1 the average Pearson correlation coefficients ranged from 0.87 to 0.97, while for piezoresistor 1 of MEMS sensor 2 their values went from 0.80 to 0.89 depending on the chosen parameters combination C. Moreover, all the coefficients for the channels close to the leading edge (e.g. MEMS sensor 1) of the stimulus during the sliding motion (phase C) were always higher than the ones for the channels at the falling edge (e.g. MEMS sensor 2) of the stimulus; this phenomenon is discussed in Section 5. The frequency modulation due to the variation of the stimulus can be appreciated in Figure 9, while a comparison between Figure 8 and Figure 9(a) points out the effect of stimulus velocity variation.
In parallel to the analysis of the frequency shift due to the variation of dynamic stimulation conditions, another major experimental result concerned the static imaging capabilities of the developed tactile sensor array. This further outcome was possible by choosing a proper experimental protocol, which included static phases in the initial and final parts of stimulation with repeatable conditions overall the runs. Figures 3, 8 and 9 show such results.
The experimental results shown for dynamic artificial touch with medium-coarse periodic gratings demonstrated the perfect coherence between the principal frequency commonly revealed by the packaged MEMS sensor units and the expected one, as shown in Table 3 and Figure 7, as well as the consistency between the surface geometry and the static image of the stimulus-sensor interface.
Looking at the background of neurophysiological and psychophysical touch studies briefly reported in the Introduction, the technological and the signal processing outcomes of this work may be classified as a successful preliminary attempt to artificially achieve roughness encoding in case of medium-coarse patterning, i.e. a deterministic link (see Table 3) was obtained between the “spatial coarseness” of the presented stimuli and the features extracted from the sensor outputs.
These results pointed out the better processing quality guaranteed by using structured information from different units of a tactile sensor array instead of naïve Fourier analysis separately on each channel, overcoming frequency discretization limitations. These limitations are shown in Figures 4 and 5, which differ both in the time window length used for FFT and in the grating periodicity. The latter is the reason for the 1.4 Hz difference between the nominal frequencies, which could not be detected with FFT due to the low resolution of the FFT in the relevant frequency range (width of bars ∼0.5 Hz in Figure 4 vs. ∼2.5 Hz in Figure 5). As a consequence, using the naïve FFT approach to retrieve the frequency in a 0.35 s time window for both data series, would result in the two gratings being not distinguished, as shown in the respective plots of Figure 7. On the contrary, with the least squares fitting procedure the separation was well feasible, and the common frequency expected when indenting and sliding at constant speed periodic ridged surfaces across an array of sensors was accurately estimated. The technological approach together with the proposed frequency estimation method guaranteed an error from 1.7% down to 0.5% over the range of spatial frequencies considered, independently of the combination of MEMS sensor units used (see Table 3). Moreover, as shown in Figure 7, limiting the evaluation to fixed size time windows reduced the accuracy somewhat, but the method stayed stable down to 0.4 s window size, making it potentially suitable for most near real-time settings. The applied method revealed to be robust even if, in addition to the observable principal frequency shift associated to the combination of the used grating and stimulus sliding velocity, the signal power had overtones (the first three or four harmonics of the fundamental frequency) introduced by both the non-linear packaging and the sharp edges of the periodic ridged surfaces. On the contrary, the fitting based on Fourier analysis resulted in an oscillatory behavior of the error respect to the observation window length. Further stability and precision with the gradient descent fitting method could be gained by taking into account all four MEMS sensors and tuning the sampling rate according to the target application.
As depicted in Figure 8 and confirmed by the calculated average Pearson cross-correlation coefficients, the gathered data had high repeatability across different runs of the same experimental conditions. Furthermore, as seen in Figure 8 already, MEMS sensor 1 produced higher voltage amplitudes, leading to a better Signal-to-Noise (S/N) ratio, which in turn caused the higher correlation between runs with respect to MEMS sensor 2. This effect may be associated to the shape of the compliant packaging, which could induce higher stresses in piezoresistors of the sensor unit located at the leading edge. Reversing the scan direction (not shown) exchanged the roles of MEMS sensors 1 and 2 in this regard. Moreover, it is noticeable to observe the excellent similarity between the plots shown in Figure 8 and the plot of Figure 9 (a). These graphs only differ for the stimulus translational speed and thus result in a compression in the time scale during phase C.
The modulation of the principal frequency due to the variation of the stimulus can be appreciated in the time domain plots of Figure 9. Moreover, as detailed in Figure 10, couples of piezoresistors of a sensor unit which are located one in front to the other along the direction of motion [piezoresistors 1 and 3 in Figure 1(b)] responded with opposite sign to the stimulus. Therefore, even with packaged silicon sensors and dynamic stimulations, the symmetries of the static calibration matrix observed in  for the bare MEMS sensor were still present in this work.
Finally, as regards the consistency between the surface geometry and the static artificial touch representation, it is remarkable to observe the output signals variations relatively to the steps between phases A (starting of data acquisition) and B (sensor loading) and between phases D (steady state after stimulus sliding) and E (sensor unloading). Figures 3, 8 and 9 point out that the step heights varied between different runs depending on the used grating (but not on the velocity). This was due to the fact that a variation of the grating periodicity modified the portion of the ridge under each MEMS unit, being the initial and final position of the stimulus carrier always the same for all runs during phase C.
6. Conclusions and Future Work
The experimental analyses performed in this work demonstrated the suitability of the developed tactile sensor for revealing medium-coarse spatial features of the explored surface, both with dynamic and static stimulation. Future work will focus on performing similar experiments incorporating the developed technology in a bio-inspired mechatronic finger. Given that the feasibility of using a polyimide thin protective layer has been shown in this investigation, the actuated finger may use a polyimide glove (mimicking the human stratum corneum) for preventing the sensor to be worn or damaged by water or grit. Moreover, because of the possibility to integrate the readings from the array with proprioception information during active touch tasks, the combination of information regarding the estimated common frequency and the velocity of the finger could solve Equation (1) and provide quantitative measurements revealing texture properties of the explored stimuli. Investigations will also be performed in implementing processing strategies to separate the velocity and periodicity information contained in Equation (1) directly from the measurements of the array, thus avoiding the need to use the knowledge of the stimulus sliding velocity (in case of passive touch experiments) or proprioception information from an actuated finger (in case of active touch ones). Experiments will be performed with other stimuli, addressing not only a medium-coarse spatial periodic pattering, but also more general fine textures (e.g. sandpapers, gratings with oblique or aperiodic ridges or 2D patterning, …) and the frequency content due to the kind of material. In that case, the focus could move from principal frequency analysis to spectral analysis over the full frequency range, or to wavelet transform if the frequency content is supposed to change with respect to time and/or stimulus-sensor relative positioning. Moreover, the fact that the MEMS sensor is triaxial may be exploited in future work with stimuli having 2D patterning: in this paper, the raw sensor outputs were directly analyzed guaranteeing great accuracy in principal frequency retrieval without encoding the force vector at MEMS-packaging interface or at packaging-stimulus interface.
These planned experiments will require some modifications to the packaging design (e.g. lower thickness, material with different hardness or viscosity, introduction of fingerprints, etc.) in order to achieve even a higher sensitivity and selectivity for each MEMS unit [31,34] and a reduction of the low-pass spatial filtering effect introduced by the materials embedding the sensor  while still providing robustness for application in artificial hands dexterously interacting with the environment [29,32].
Finally, future investigations will experiment with artificial tactile sensors the unified paradigm proposed by Yoshioka and colleagues  for the perception of fine and coarse textured surfaces, in order to go towards a common theory for human and robot mediated coding and decoding of tactile stimuli.
This study was funded by the NANOBIOTACT project (EU-FP6-NMP-033287). The authors would like to thank Nicola Vitiello from Scuola Superiore Sant'Anna (Pisa, Italy) for his collaboration in the development of the mechatronic stimulator, Mark Mitulla and Stefan Schmitt from the IMM (Institute of Microtechnology of Mainz, Germany) for their collaboration in the sensor fabrication. Finally, the authors express their thanks to the anonymous reviewers for the precious suggestions.
References and Notes
- Johnson, K.O.; Yoshioka, T. Neural mechanisms of tactile form and texture perception. In The Somatosensory System: Deciphering the Brain's Own Body Image; Nelson, RJ, Ed.; CRC Press LLC: Boca Raton, FL, USA, 2001; pp. 73–101.
- Jones, L.A.; Lederman, S.J. Tactile Sensing. In Human Hand Function; Jones, LA, Lederman, SJ, Eds.; Oxford University Press: New York, NY, USA, 2006; pp. 44–74.
- Hollins, M.; Faldowski, R.; Rao, S.; Young, F. Perceptual dimensions of tactile surface texture: a multidimensional scaling analysis. Percept. Psychophys. 1993, 54, 697–705.
- Yoshioka, T.; Bensmaïa, S.J.; Craig, J.C.; Hsiao, S.S. Texture perception through direct and indirect touch: An analysis of perceptual space for tactile textures in two modes of exploration. Somatos. Mot. Res. 2007, 24, 53–70.
- Meftah, E.M.; Belingard, L.; Chapman, C.E. Relative effects of the spatial and temporal characteristics of scanned surfaces on human perception of tactile roughness using passive touch. Exp. Brain. Res. 2000, 132, 351–361.
- Smith, M.; Chapman, C.E.; Deslandes, M.; Langlais, J.S.; Thibodeau, M.P. Role of Friction and Tangential Force Variation in Subjective Scaling of Tactile Roughness. Exp. Brain. Res. 2002, 144, 211–223.
- Lawrence, M.A.; Kitada, R.; Klatzky, R.L.; Lederman, S.J. Haptic roughness perception of linear gratings via bare finger or rigid probe. Perception 2007, 36, 547–557.
- Klatzky, R.L.; Lederman, S.J.; Hamilton, C.; Grindley, M.; Swendsen, R.H. Feeling textures through a probe: Effects of probe and surface geometry and exploratory factors. Percept. Psychophys. 2003, 65, 613–631.
- LaMotte, R.H.; Srinivasan, MA. Surface microgeometry: Neural encoding and perception. Information Processing in the Somatosensory System. In Wenner-Gren Intl. Symposium Series; Franzen, O, Westman, J, Eds.; Macmillan Press: New York, NY, USA, 1991; pp. 49–58.
- Scheibert, J.; Leurent, S.; Prevost, A.; Debrégas, G. The role of fingerprints in the coding of tactile information probed with a biomimetic sensor. Science 2009, 323, 1503–1506.
- Srinivasan, M.A.; Whitehouse, JM; LaMotte, RH. Tactile detection of slip: Surface microgeometry and peripheral neural codes. J. Neurophysiol. 1990, 63, 1323–1332.
- Hollins, M.; Risner, S.R. Evidence for the duplex theory of tactile texture perception. Percept. Psychophys. 2000, 62, 695–705.
- Yoshioka, T.; Gibb, B.; Dorsch, A.K.; Hsiao, S.S.; Johnson, K.O. Neural coding mechanisms underlying perceived roughness of finely textured surfaces. J. Neurosci. 2001, 21, 6905–6916.
- Konyo, M.; Maeno, T.; Yoshida, A.; Tadokoro, S. Roughness sense display representing temporal frequency changes of tactile information in response to hand movements. Proceedings of the First Joint Eurohaptics Conference and Symposium on Haptic Interface for Virtual Environment and Teleoperator Systems 2005, 609–610.
- Konyo, M.; Yoshida, A.; Tadokoro, S.; Saiwaki, N. A tactile synthesis method using multiple frequency vibrations for representing virtual touch. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems 2005, Edmonton, Canada, 2-6 August, 2005; pp. 3965–3971.
- Lee, M.H.; Nicholls, H.R. Tactile sensing for mechatronics: A state of the art survey. Mechatronics 1999, 9, 1–31.
- Maheshwari, V.; Saraf, R.F. Tactile devices to sense touch on a par with a human finger. Angew Chem. Int. Edit. 2008, 47, 7808–7826.
- Kim, S.H.; Engel, J.; Liu, C.; Jones, D.L. Texture classification using a polymer-based MEMS tactile sensor. J. Micromech. Microeng. 2005, 15, 912–920.
- Mukaibo, Y.; Shirado, H.; Konyo, M; Maeno, T. Development of a texture sensor emulating the tissue structure and perceptual mechanism of human fingers. Proceedings of the IEEE International Conference on Robotics and Automation 2005, Barcelona, Spain, 18-22 April 2005; pp. 2565–2570.
- Okamoto, S.; Konyo, M.; Mukaibo, Y.; Maeno, T.; Tadokoro, S. Real-time estimation of touch feeling factors using human finger mimetic tactile sensors. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems 2006, Beijing, China, October 2006; pp. 3581–3586.
- Scheibert, J.; Prevost, A.; Debrégas, G.; Rousier, R.; Rey, P. A Novel Biomimetic Haptic Sensor to study the Physics of Touch. Proceedings of Méchanotransduction, 2004.
- Shimojo, M.; Ishikawa, M. An active touch sensing method using a spatial filtering tactile sensor. Proceedings of the IEEE International Conference on Robotics and Automation 1993, Seoul, Korea, May 1993; pp. 948–954.
- Hosoda, K.; Tada, Y.; Asada, M. Anthropomorphic robotic soft fingertip with randomly distributed receptors. Robot. Auton. Systems 2006, 54, 104–109.
- Fishel, J.A.; Santos, V.J.; Loeb, G.E. A Robust Micro-Vibration Sensor for Biomimetic Fingertips. Proceedings of the IEEE 2nd Biennial IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics 2008, Scottsdale, AZ, USA, 19-22 October, 2008; pp. 659–663.
- Wettels, N.; Santos, V.J.; Johansson, R.S.; Loeb, G.E. Biomimetic Tactile Sensor Array. Adv. Robot. 2008, 22, 829–849.
- Maheshwari, V.; Saraf, R.F. High-resolution thin-film device to sense texture by touch. Science 2006, 312, 1501–1504.
- Johansson, R.S.; Vallbo, Å.B. Tactile sensibility in the human hand: relative and absolute densities of four types of mechanoreceptive units in glabrous skin. J. Physiol. (Lond.) 1979, 286, 283–300.
- Beccai, L.; Roccella, S.; Arena, A.; Valvo, F.; Valdastri, P.; Menciassi, A.; Carrozza, M.C. Design and fabrication of a hybrid silicon three-axial force sensor for biomechanical applications. Sens. Actuator A-Phys. 2005, 120, 370–382.
- Beccai, L.; Roccella, S.; Ascari, L.; Valdastri, P.; Sieber, A.; Carrozza, M.C.; Dario, P. Development and Experimental Analysis of a Soft Compliant Tactile Microsensor for Anthropomorphic Artificial Hand. IEEE-ASME Trans. Mechatron. 2008, 13, 158–168.
- Oddo, C.M.; Valdastri, P.; Beccai, L.; Roccella, S.; Carrozza, M.C.; Dario, P. Investigation on calibration methods for multi-axis, linear and redundant force sensors. Meas. Sci. Technol. 2007, 18, 623–631.
- Vásárhelyi, G.; Fodor, B.; Roska, T. Tactile sensing-processing: interface-cover geometry and the inverse-elastic problem. Sens. Actuat. A-Phys. 2007, 140, 8–18.
- Carrozza, M.C.; Cappiello, G.; Micera, S.; Edin, B.B.; Beccai, L.; Cipriani, C. Design of a cybernetic hand for perception and action. Biol. Cybern. 2006, 95, 629–644.
- Beccai, L.; Roccella, S.; Oddo, C.; Carrozza, M.C. Sensing fingertip for bio-inspired tactile encoding. In Proceedings of the First Italian Conference on Bioengineering; Burattini, R, Contro, R, Dario, P, Landini, L, Eds.; Pàtron editore: Bologna, Italy, 2008; p. 787.
- Vásárhelyi, G.; Ádám, M.; Vázsonyi, É; Bársony, I; Dücső, C. Effects of the elastic cover on tactile sensor arrays. Sens. Actuat. A-Phys. 2006, 132, 245–251.
- Shimojo, M. Mechanical filtering effect of elastic cover for tactile sensor. IEEE Trans. Rob. Autom. 1997, 13, 128–132.
- Lederman, S.J. Tactile roughness of grooved surfaces: the touching process and effects of macro- and microsurface structure. Percept. Psychophys. 1974, 16, 385–395.
- Darian-Smith, I.; Oke, L.E. Peripheral neural representation of the spatial frequency of a grating moving across the monkey's finger pad. J. Physiol.-London 1980, 309, 117–133.
|Table 1. Grating groove width (gw) and spatial period (Δp) with respect to the sample type. Ridge width (rw) was fixed to 0.6 mm for all types.|
|Table 2. Expected principal frequency from sensor outputs depending on the spatial periodicity (Δp) and on the sliding velocity (v) of the applied grating.|
|Expected frequency vs. Δp and v||Δp = 4.1 mm||Δp = 3.6 mm||Δp = 3.1 mm||Δp = 2.6 mm|
|v = 15 mm/s||3.66 Hz||4.17 Hz||4.84 Hz||5.77 Hz|
|v = 30 mm/s||7.32 Hz||8.33 Hz||9.68 Hz||11.54 Hz|
|v = 48 mm/s||11.71 Hz (not tested)||13.33 Hz (not tested)||15.48 Hz||18.46 Hz|
|Table 3. Average RMS errors obtained by FFT (εFFT) and least square fit estimation (εLSq) using the full data range. The top row indicates the expected frequency, fnominal, depending on the measurement run. The combination C of spatial period Δp and velocity v associated to each measurement run is also indicated.|
|piezo1,1 vs. piezo2,1|
|piezo4,1 vs. piezo2,1|
© 2009 by the authors; licensee Molecular Diversity Preservation International, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).