Next Article in Journal
A Second-Order Cone Programming Reformulation of the Economic Dispatch Problem of BESS for Apparent Power Compensation in AC Distribution Networks
Next Article in Special Issue
Objective Assessment of Walking Impairments in Myotonic Dystrophy by Means of a Wearable Technology and a Novel Severity Index
Previous Article in Journal
Survey of Automatic Spelling Correction
Previous Article in Special Issue
Smartphone-Based Evaluation of Postural Stability in Parkinson’s Disease Patients During Quiet Stance
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Assessment of Intuitiveness and Comfort of Wearable Haptic Feedback Strategies for Assisting Level and Stair Walking

by
Ilaria Cesini
1,2,*,†,
Giacomo Spigler
3,†,
Sahana Prasanna
1,2,†,
Jessica D’Abbraccio
1,2,
Daniela De Luca
1,4,
Filippo Dell’Agnello
1,2,
Simona Crea
1,2,5,
Nicola Vitiello
1,2,5,
Alberto Mazzoni
1,2,‡ and
Calogero Maria Oddo
1,2,*,‡
1
The BioRobotics Institute, Scuola Superiore Sant’Anna, Viale Rinaldo Piaggio 34, 56025 Pontedera, Italy
2
Department of Excellence in Robotics & AI, Scuola Superiore Sant’Anna, Piazza Martiri della Libertà 33, 56127 Pisa, Italy
3
Department of Cognitive Science and Artificial Intelligence, Tilburg University, Warandelaan 2, 5037 AB Tilburg, The Netherlands
4
Department of Information Engineering, University of Pisa, Via Caruso 16, 56122 Pisa, Italy
5
IRCCS Fondazione Don Carlo Gnocchi, 50143 Florence, Italy
*
Authors to whom correspondence should be addressed.
The authors share the first authorship based on equal contribution.
The authors share the senior authorship based on equal contribution.
Electronics 2020, 9(10), 1676; https://doi.org/10.3390/electronics9101676
Submission received: 21 September 2020 / Revised: 4 October 2020 / Accepted: 7 October 2020 / Published: 14 October 2020
(This article belongs to the Special Issue Wearable Electronics for Assessing Human Motor (dis)Abilities)

Abstract

:
Nowadays, lower-limb prostheses are reaching real-world usability especially on ground-level walking. However, some key tasks such as stair walking are still quite demanding. Providing haptic feedback about the foot placement on the steps might reduce the cognitive load of the task, compensating for increased dependency on vision and lessen the risk of falling. Experiments on intact subjects can be useful to define the feedback strategies prior to clinical trials, but effective methods to assess the efficacy of the strategies are few and usually rely on the emulation of the disability condition. The present study reports on the design and testing of a wearable haptic feedback system in a protocol involving intact subjects to assess candidate strategies to be adopted in clinical trials. The system integrated a sensorized insole wirelessly connected to a textile waist belt equipped with three vibrating motors. Three stimulation strategies for mapping the insole pressure data to vibrotactile feedback were implemented and compared in terms of intuitiveness and comfort perceived during level and stair walking. The strategies were ranked using a relative rating approach, which highlighted the differences between them and suggested guidelines for their improvement. The feedback evaluation procedure proposed could facilitate the selection and improvement of haptic feedback strategies prior to clinical testing.

1. Introduction

Walking and climbing stairs represent almost automatic tasks for intact subjects, requiring little or no cognitive effort. During locomotion, the mechanoreceptors on the foot and the proprioceptors in the muscles gather force and position information, which are delivered to the central nervous system through the afferent path [1]. Lower-limb amputation affects the sensorimotor system, causing significant functional and sensory impairments. In particular for lower-limb amputees, ascending and descending stairs are difficult and potentially dangerous activities. Foot placement on the steps is a dynamic task, requiring the individual to know the amount of toe overhang to adjust the movement of the rest of the body [2]. The absence of awareness and control over foot placement increases the risk and fear of falling and the visual dependency in amputees.
Recent years have witnessed striking advancements in the field of robotics, with the development of sensing and actuation technologies and control strategies aimed to facilitate the actions and interactions of robots and humans with the environment [3,4] and to promote human–robot cooperation [5,6]. In particular, the use of robotic systems such as augmented sensory feedback devices as part of rehabilitation programs of lower limb amputees has the potential to complement the action of clinical therapists and to continue the long-term therapy at home after hospitalization.
Several studies reported that the introduction of augmented sensory feedback systems in prostheses can improve the mobility function of amputees by providing information on the foot–ground contact or other gait parameters, and thus restoring part of the missing sensory input [7,8,9,10,11,12,13,14,15,16,17,18]. Among all the possible sensory feedback modalities, haptic feedback is widely employed to convey information, due to the possibility to tune the stimulation parameters (i.e., frequency, duration, amplitude) and to implement different stimulation strategies without adding extra loads to the visual and auditory channels [19].
When defining the stimulation strategy, it is crucial to identify the minimum amount of information needed for the achievement of a specific task, in order to limit the cognitive burden associated with the use of the feedback system [20]. The stimulation should be intuitive, i.e., the amputee should easily understand the information being transmitted (e.g., he/she should correlate the stimulation pattern to the foot/ground contact) and comfortable, i.e., the stimuli should not annoy the user. Most of the devices found in literature implement the so-called corrective feedback strategies, consisting of “simplified” activation patterns providing stimuli in the direction of error, denoting deviations of specific variables or performance parameters with respect to a given physiological range or a target [9,13,17,21]. When the strategy delivers direct feedback on the evolution of one or more measured variables [1,2,9,18,22,23,24,25,26,27,28], the so-called concurrent feedback, the complexity of the feedback pattern depends on the number of events and parameters encoded by the device. For example, the timing of the foot heel strike requires a single event-based activation, while providing information on the location of the center of pressure (CoP) and the intensity of the ground reaction force under the foot calls for a more complex activation pattern. Depending on the variables or the instructions mapped through the feedback pattern, stimulation strategies that are particularly informative in a specific locomotion task may be ineffective in other ones. For example, some feedback strategies designed for level walking [18,25,29] depend on specific sequences of gait-phase transitions, which are not preserved while climbing stairs or walking on uneven terrains.
The present study reports on a new wearable haptic feedback system providing concurrent augmented sensory information on the foot–ground interactions and designed to assist lower-limb amputees in different locomotion scenarios—i.e., ground-level walking, ascending/descending stairs—while enabling smooth locomotory transitions. Multiple stimulation strategies have been implemented in the feedback system, translating the spatiotemporal evolution of the CoP under the foot into vibrations delivered onto the user’s waist, and in some cases providing also information on the ground reaction forces. Along with these stimulation patterns already presented in a preliminary study [23], a strategy that mimics the temporal characteristics of skin mechanoreceptors [30,31,32,33,34] was included. All these strategies have been conceived with the idea to guide the amputees in the execution of level and, especially, stair walking tasks, for which information on the prosthetic foot placement and the toe overhang on the steps may prevent the risk and incidence of falls, favor the embodiment of the prosthesis and reduce the visual workload. However, complex, poorly intuitive, and bothersome stimulation strategies may prevent the effectiveness and the usability of the feedback system.
When it is not possible to establish the most suitable strategy for sensory feedback a priori, multiple candidate stimulation patterns should be tested on amputees. Nonetheless, the introduction of two or more strategies significantly complicates the experiment and lengthens the time needed to complete the trials, with the risk to increase the physical and cognitive effort of the subjects. Furthermore, restrictive inclusion criteria and difficulties in enrolling amputees for the clinical studies requires pre-selecting a proper stimulation strategy to prevent wasted trials and time. Indeed, studies addressing the testing of augmented feedback systems on lower-limb amputees often involve a small pool of subjects, with a maximum of three patients recruited for the experiments [10,17,18,28], and a reduced number of trials with a single feedback strategy, due to the limited availability of the subjects for the experiments. In this context, it is useful to define an assessment protocol to benchmark the proposed strategies on intact subjects. In order to guarantee that the results obtained on intact subjects can be transferred to amputees, the protocol should be designed to simulate the disability [2,9,35,36] or should adopt a metric that is valid for both intact subjects and amputees.
Here, we introduce a novel assessment protocol designed to pre-select the stimulation strategy to be used in clinical trials, through experiments involving intact subjects. The different strategies implemented in the haptic feedback system were ranked in terms of intuitiveness and comfort perceived by the subjects, during level and stair walking using a relative rating approach. Specifically, the subjects were asked to compare couples of consecutive strategies (i.e., the last strategy is better, same, or worse than the previous one) and the final ranking of the strategies was obtained by applying the Elo rating algorithm [37,38]. The Elo rating is a statistical system based on a logistic distribution, which was originally developed to assess the skill levels of chess players [38]. Before starting a match, each player is assigned an initial score. Then, if a player wins or loses the match, his/her score is updated based on the formula reported in [38] and detailed hereafter. Following the same approach, in the present paper, each feedback strategy was treated as a player and each comparison between couple of strategies (in terms of intuitiveness and comfort) was regarded as a match. The relative comparisons were introduced to highlight the differences among the strategies, avoiding the possible biases towards intermediate scores given by a rating performed on an absolute scale.
The energy cost of ascending stairs is considerably higher than descending stairs and walking overground [39,40,41,42,43] and requires a greater demand of attentional resources, even in unimpaired subjects [44]. The different physical and cognitive effort associated with the specific motor task performed might affect the perceived intuitiveness and comfort of the haptic feedback, biasing the rating of the stimulation strategies assessed by the subjects. Thus, the effect of the specific task of ascending or descending stairs on the ratings reported by the subjects was also examined.
The paper is organized as follows. Section 2 presents (i) an overview of the wearable haptic feedback system architecture, (ii) the feedback strategies implemented, (iii) the experimental protocol for their comparison, and (iv) the Elo rating system used for data analysis. Then, the results of the comparisons of the feedback strategies are shown (Section 3) and discussed (Section 4). Lastly, general conclusions and future applications of the device and the assessment protocol described in the study are reported (Section 5).

2. Materials and Methods

2.1. Wearable Haptic Feedback System

The haptic feedback system designed for this study and shown in Figure 1a is a redesigned version of the device presented in [45]. The system is a stand-alone, wireless device comprising a sensing module and a wearable haptic interface, integrating a set of vibrotactile (VT) units and a haptic control unit for data processing and motor control.
The sensing module consists of a pressure-sensitive insole lodged in one shoe (size 43 EU) integrating an array of optoelectronic sensors for the measurement of the plantar pressure distribution [46]. The shoe is equipped with onboard electronics for data acquisition and wireless transmission to the haptic control unit.
The wearable haptic interface is a textile belt equipped with three VT units (Pico Vibe 304-116, Precision Microdrives). The VT units are equally spaced at 9 cm apart on one side of the waist to provide ipsilateral feedback (Figure 1b, top). The VT units have been encapsulated in a polymeric matrix of PDMS (Sylgard® 184 Silicone Elastomer, Dow Corning) to guarantee comfortable contact with the skin.
The belt further integrates a haptic control unit, which is enclosed in a 3D-printed box, physically attached to the belt. The board wirelessly acquires pressure data from the insole at 100 Hz and calculates the coordinates of the CoP and the vertical ground reaction force (vGRF) from the raw sensor signals (Figure 2). The commands for the activation of the VT units are generated according to multiple stimulation strategies implemented in the control unit and described in Section 2.2.

2.2. Stimulation Strategies

Four stimulation strategies providing information about the step dynamics through stimulation of the user’s waist were implemented and compared. Figure 1c shows the activation patterns of the VT units during level walking for each stimulation strategy.
The CoP mapping strategy (CoP) was designed to convey information about the CoP progression under the foot. Specifically, the insole was virtually segmented in three regions (heel [0 cm, 5 cm], sole [5 cm, 18 cm], toe [18 cm, 25 cm]), and each region was associated to a specific VT unit (Figure 1b, middle). Every time the CoP entered one region, the corresponding VT unit was activated with 100% pulse width modulation (PWM) duty cycle (maximum activation). Figure 1c (top) shows the sequential activation of the VT units at the set intensity and following the CoP progression as described above, during four level-walking strides. The CoP shuffled (Shuffled) strategy was introduced as a control for the assessment of the efficacy of the feedback patterns. In this case, the mapping between the CoP segments and the VT units was shuffled every time the CoP moved to a different region (Figure 1c, second plot from top).
The CoP mapping + vGRF (CoP + vGRF) strategy adopted the same VT activation pattern as for the CoP but included an intensity modulation. Such a vibration amplitude modulation was obtained by setting the duty cycle of a PWM (carrier at 1 kHz) to be a linear function of the vGRF, with 0% activation at 1 N activation threshold and 100% duty cycle with forces above 60 N. The stimulation pattern is displayed in Figure 1c (third plot from top), with VT units activation plots denoting amplitude modulation. This strategy was designed to enrich the sensory information, by providing synchronous information about the foot position and the forces under the foot.
Along with the CoP-based activation patterns, a feedback strategy based on a neuronal network mapping (Neuro) was defined and tested. Specifically, sensors data, x n , generated sequences of neuronal spikes after being mapped into inputs to three Izhikevich artificial neurons tuned to regular spiking dynamics [47,48]. The three neurons were defined as heel-strike, flat-foot, and toe-off neurons, and associated with VT1, VT2, and VT3 respectively (Figure 1b, bottom). Each neuron received an input current, I V T i n , computed as in Equation (1)
I V T i n = g V T i max ( w V T i T x n , 0 ) ,
where   g V T i   is the gain and w V T i T   corresponds to the afferent receptive field weights, which can be positive (excitatory) or negative (inhibitory) in order to identify the region of the insole where the CoP lies and, in turn, activate the corresponding VT unit. In the experiment, the signs of the weights were chosen to associate the active region of the foot to the respective VT unit, i.e., [w13 = −1, w12 = −0.1, w11 = 1] for the heel-strike neuron, [−0.3, 1, −0.1] for the flat-foot neuron, and [1, −0.8, −1] for the toe-off neuron. Gains of 10, 60, and 25, were used for heel-strike, flat-foot, and toe-off neuron, respectively. The artificial neurons were updated at a frequency of 16 kHz in a Zynq-7020 FPGA embedded in a system on a module (sbRIO-9651, National Instruments, Austin, TX, USA) [48,49]. Every neuronal spike produced a 15 ms constant activation of the corresponding VT unit. Based on this activation mechanism, the Neuro strategy provided both the spatial mapping of the foot–ground contact and the contact pressure, achieved through the modulation of the vibration intensity given by changes in the firing rate (Figure 1c, bottom).

2.3. Experimental Protocol

Six intact subjects (three females and three males, age 24 ± 2, weight 62 ± 10 kg, height 170 ± 6 cm, foot size 41 ± 2 EU) were recruited as volunteers for the experiments. The experiments were conducted in accordance to the ethical guidelines of Scuola Superiore Sant’Anna.
The subjects wore the haptic belt and the shoes with the sensorized insole lodged in the right one (Figure 1a). VT feedback amplitude was predefined and might be modulated according to the feedback strategies (Figure 1c). Before starting the trial, the subjects walked for 2–3 min to familiarize with the vibration intensities. The volunteers were not informed about the stimulation strategies and they were instructed to focus on the feedback, trying to understand the vibration pattern. They were asked to maintain a uniform pace and walk on a specific path that included 11 steps on a stair and 2 short overground walks. A single trial consisted of ascending the path or descending the same track while receiving one of the pseudo-randomized feedback patterns described in Section 2.2.
In order to simplify the measurement of the time taken in each trial, subjects were instructed to start each trial with the sensorized foot (right foot), and to stop upon reaching the end of the path. This ensured that the duration of the task was unbiased, computing it as the difference between the last time the sensorized foot touched the ground and the time it was raised for the first time. Figure 3 shows an example of the sequence of stimulation strategies delivered to Subject 1 during the experiment.
At the end of two consecutive trials, the subjects were asked to compare the last strategy with respect to the previous one in terms of both intuitiveness and comfort, as better, same, or worse. The order of the stimulation strategies provided in each trial was pseudo-randomized for each subject such that all the possible pairs of consecutive strategies were repeated three times. Pairs (1, 2) and (2, 1) were considered as different, to account for possible bias in the order of presentation.

2.4. Elo-Based Data Analysis

Data analysis focused on assessing a ranking of the stimulation strategies based on the subjects’ rating of intuitiveness and comfort of each strategy. The subjects were asked to perform relative comparisons between pairs of consecutive stimulation strategies. The final rating of each strategy was derived using the Elo rating algorithm, which was originally developed to rank chess players from the outcomes of their games [38]. The Elo rating is a system whereby the performance of each player is not measured absolutely but depends on the rating of his/her adversary and the result scored in the match against him/her. The difference in ratings of two players (here stimulation strategies) predicts the likely outcome of their comparison. Specifically, given ratings R A and R B of the strategies A and B, the probability of winning the comparison (expected score) is E A for A, and E B for B, as in Equation (2)
E A   =   1 1 + 10 R B R A 400 ;                           E B   =   1 1 + 10 R A R B 400 .
At the end of each trial (comparison), the ratings are iteratively updated considering the reported outcome ‘y’ of each comparison (y = 1 if A wins, y = 0 if B wins, y = 0.5 if it is a draw) and the expected score associated to each strategy ( E A and E B ) such that more accurate estimates of the predicted outcome will be made in successive comparisons, as in Equation (3)
R B   =   R B + K ( ( 1 y ) E B ) ; R A   =   R A + K ( y   E A ) .
The constant K can be set arbitrarily and corresponds to the maximum Elo points a feedback strategy can gain or lose in one comparison [37]. In the present work, K value was set to 30. All the strategies started with an initial rating value of 0. The initial value only defines a common offset that does not affect the final score.
Additionally, the time taken by the subjects to ascend and descend the pre-specified path was analyzed to estimate the possible interference between haptic feedback and movement.
Collected data will be made available on request.

3. Results

The different feedback strategies were compared in terms of intuitiveness and comfort during level and stair walking. The scores produced with the Elo rating algorithm and extracted from the comparisons made by the subjects on consecutive trials were analyzed. Figure 4 shows the subject-wise results, while Table 1 and Figure 5 report the statistics over the whole population.
The intuitiveness scores were found to be significantly different among stimulation strategies (χ2 (3) = 13.61, p = 0.0035, Kruskal–Wallis H test). A further post-hoc comparison using a Wilcoxon rank sum test indicated that the mean score of Shuffled was significantly lower than CoP (p = 0.0152) and CoP + vGRF (p = 0.0411). Also the Neuro strategy resulted to be significantly less intuitive than both CoP (p = 0.0022) and CoP + vGRF (p = 0.0043). Thus, two levels of intuitiveness were found, with CoP-based strategies showing a similar high intuitiveness, while Neuro resulted to be as intuitive as the non-informative stimulations provided by Shuffled.
Likewise, significant differences were found between the comfort scores of the different stimulation strategies (χ2 (3) = 14.35, p = 0.0025). A post-hoc Wilcoxon rank sum test found that both Neuro strategy and CoP + vGRF were significantly more comfortable than Shuffled (p = 0.0043 and p = 0.0022 respectively). CoP resulted to be significantly less comfortable than both CoP + vGRF (p = 0.026) and Neuro (p = 0.0152). Amplitude modulated stimulations were then found to be more comfortable independently from the temporal pattern of the stimulation.
To verify whether subjects’ rankings were biased by the varying physical and cognitive efforts associated to ascending vs. descending stairs, Elo scores were computed aggregating trials going up or down (Figure 6). Though the Elo scores showed slightly higher intuitiveness and comfort for the descending trials compared to the ascending trials, the difference was not statistically significant (χ2 (1) = 1.64, p = 0.2 for intuitiveness and χ2 (1) = 0.1, p = 0.75 for comfort, Kruskal-Wallis H test).
Successively, the different stimulation strategies were compared with respect to the possible decrease in the walking speed due to the cognitive load associated to the concurrent provision of the feedback. This feature was evaluated by measuring the median time the subjects took to walk through the path while receiving the feedback (Section 2.3). The analysis was performed for climbing, descending, and the combination of the two tasks. A fixed effects analysis was performed to look for statistically significant differences in the two paths (going up vs. going down) and then between individual stimulation strategies within each of them. The results are shown in Figure 7.
A Wilcoxon rank sum test indicated that the median time taken by the subjects when going up (M = 22.74 s, IQR = [21.52, 24.03] s) was statistically significantly higher than when going down (M = 20.96 s, IQR = [19.85, 23.52] s), with Z = 4.67, p < 0.001. However, no significant difference was found in the median times between the stimulation strategies (p > 0.05, Kruskal–Wallis H test), neither when going up (χ2 (3) = 1.48, p = 0.69), nor when going down (χ2 (3) = 7.28, p = 0.06), nor combining the two (χ2 (3) = 3.49, p = 0.32).

4. Discussion

This work introduces a fully portable wearable haptic feedback system designed to convey augmented sensory information to lower limb amputees during level and stair walking tasks, and a novel assessment protocol to benchmark multiple feedback strategies, prior to clinical trials.
Different feedback strategies have been implemented to relocate the sensory information from the foot on the users’ waist. The strategies were compared in terms of intuitiveness and comfort using a relative rating system, namely the Elo rating approach, in order to pre-select the most suitable stimulation strategy to be used in clinical trials. A key issue in the design of the assessment protocol was to guarantee that the results collected on intact subjects could be extended also to amputees. Only a few studies in literature reported on experimental trials involving intact subjects to compare different feedback strategies. For example, in the assessment protocol proposed by Plauché et al. [9], the applicability of the results on amputees was guaranteed by simulating the disability. Specifically, the subjects were asked to wear a bypass adapter to walk on an above-knee prosthesis, simulating novice prosthetic users. A different approach was adopted by Afzal et al. [35] who asked the participants to walk asymmetrically targeting a specific symmetry ratio to assess a suitable strategy to improve the gait symmetry of post-stroke patients. Both the studies proposed assessment protocols involving intact subjects mimicking the disability. In the present study, the comparison of the stimulation strategies was based on a set of parameters that are independent from the disability, such as the intuitiveness of the stimulation pattern and the comfort of the vibration. Intuitiveness and comfort have been considered as valuable metrics for the pre-selection of a suitable stimulation strategy for future clinical trials with amputees. Indeed, a counterintuitive and bothersome stimulation might increase the cognitive effort and limit the long-term use of the feedback device, resulting in a slow learning curve and poor gait performance. An advantage of the assessment protocol is that it can be used with intact subjects, potentially shortening the development time and associated costs of clinical trials. The final assessment of the effectiveness of the feedback in enhancing gait performance of amputees can however be achieved only through clinical trials.
The use of the Elo rating approach highlighted the differences between the stimulation strategies implemented in terms of intuitiveness and comfort. An initial pilot run using absolute scores (1–10) and involving two subjects (subsequently excluded from the main experiment) showed a strong bias towards intermediate values and small effect sizes (Figure 8). This evidence suggested that, with respect to absolute ratings, the relating comparisons provide a clearer and more reliable differentiation between the feedback strategies.
In the specific case of the three strategies proposed in the present study, CoP and CoP + vGRF were the most intuitive. As for comfort, CoP + vGRF and Neuro were found to be the best performing strategies (Figure 5).
Several studies involving young intact subjects, such as the ones recruited for the present work, reported that the subjects experienced an increased energy expenditure when ascending the stairs with respect to descending phases [39,40,41,42,43]. Furthermore, as stair walking is more cognitively demanding than level walking [44], also the specific task of ascending or descending the stairs may require different attentional resources. Differences in physical and cognitive effort related to specific motor tasks may affect the perceived intuitiveness and comfort of the stimulation strategies, thus biasing the results of the comparison. In order to verify such possible bias, Elo scores were computed separately for going up and down the stairs (Figure 6), demonstrating no significant influence of the specific motor task on the perceived comfort and intuitiveness of each stimulation strategy.
The outcome of the experiment can be used on one hand to pre-select the strategy to be adopted in clinical trials and, on the other hand, it can provide interesting insights on the users’ perception that might guide the improvement of the stimulation strategy. For example, the lower comfort attributed to CoP suggested that the stimulation intensity was too high. Indeed, in CoP the VT units were activated at the maximum amplitude, as opposed to the other two strategies that provided either modulated activation (CoP + vGRF) or intermittent stimulation (Neuro), thus producing a softer vibration. The Neuro strategy was introduced because of promising studies performed on the upper limb [50,51], however for the specific application of stair walking the CoP strategy performed better than the other ones in terms of intuitiveness. This is presumably due to the richer information content delivered by the Neuro strategy, which was not fully appreciated in the specific gait task. Indeed, a more information-rich (and comfortable) feedback strategy such as Neuro may be more suitable for integrating the own sensorimotor scheme with information on the characteristics of terrains, and this is currently being investigated.
In order to compare the perception of the subjects against objective data, the time to complete a trial was analyzed. In the experimental protocol, the participants were asked to focus on the stimulation strategy, trying to understand the correlation between the vibration pattern and their gait. With this considered, it can be assumed that an intuitive strategy would require a lower cognitive effort to understand the stimulation signal, resulting in faster walking and thus in shorter time to complete the trial. However, no significant differences were found in the median times between the stimulation strategies and therefore the idea of using time as an objective metric for the intuitiveness must be rejected.

5. Conclusions

In conclusion, the assessment protocol presented in this study proved to be a viable way to compare stimulation strategies.
The augmented feedback strategies evaluated were capable of achieving smooth transitions among different locomotion scenarios such as ground-level walking, and ascending and descending stairs, thus showing potential for a future deployment in gait rehabilitation programs for lower limb amputees, but also in daily life use. Future studies will address the translatability of the haptic feedback system to a broader range of applications, targeting other sensory impaired subjects such as blind walkers, peripheral neuropathy, Parkinson’s and post-stroke patients, or in telepresence and virtual reality applications.
In a final remark, the adoption of a protocol grounded on a relative rating system emphasized the differences among the strategies, preventing biases towards intermediate values typically observed with absolute rating methods, and it also has the potential to generalize to other quality indicators besides intuitiveness and comfort.
Future activities will involve the validation of these results with lower limb amputees to test the effectiveness of the wearable haptic feedback system and the CoP-based strategies in improving users’ gait performance (e.g., temporal symmetry, speed, cognitive workload), during ground and stair walking tasks. The neuromorphic strategy will be tested on different terrains to verify the possibility of conveying terrain features, with the final goal of improving amputees’ contextual awareness.

Author Contributions

I.C., G.S., and S.P. equally contributed to the work; A.M. and C.M.O. equally contributed to the supervision of the work; I.C., G.S., S.P., A.M., and C.M.O. conceived the study; I.C. designed the wearable haptic interface; I.C. and G.S. conceived the CoP-based stimulation strategies; G.S. implemented the Elo rating system; S.P. conceived the neuromorphic feedback strategy and implemented the stimulation strategies, I.C., G.S., and S.P. performed the experiments; I.C., G.S., and S.P. analyzed the data; J.D. fabricated the silicone cover of the pressure sensitive insole; F.D. designed the electronics of the wearable haptic feedback system; D.D.L. contributed to the execution of the experiments; S.C. and N.V. provided the sensorized shoes; S.C., N.V., and C.M.O. contributed to funding acquisition and project administration; I.C., G.S., S.P., A.M., and C.M.O. contributed to the analysis and the discussion of the results. I.C., G.S., S.P., and C.M.O. wrote the manuscript. All authors have read and agree to the published version of the manuscript.

Funding

This work was supported in part by the EU within the CYBERLEGs Plus Plus project (H2020-ICT-2016-1 grant agreement no. 731931) and in part by the Italian National Institute for Insurance against Accidents at Work (INAIL Centro Protesi, Budrio) within the MOTU project.

Conflicts of Interest

N.V. and S.C. have commercial interests in the spin-off company IUVO Srl which is the exclusive licensee of the foot pressure sensors technology.

References

  1. Fan, R.E.; Culjat, M.O.; King, C.-H.; Franco, M.L.; Boryk, R.; Bisley, J.W.; Dutson, E.; Grundfest, W.S. A Haptic Feedback System for Lower-Limb Prostheses. IEEE Trans. Neural Syst. Rehabil. Eng. 2008, 16, 270–277. [Google Scholar] [CrossRef] [PubMed]
  2. Sie, A.; Boe, D.; Rombokas, E. Design and Evaluation of a Wearable Haptic Feedback System for Lower Limb Prostheses During Stair Descent. In Proceedings of the 2018 7th IEEE International Conference on Biomedical Robotics and Biomechatronics (Biorob), Enschede, The Netherlands, 26–29 August 2018; pp. 219–224. [Google Scholar]
  3. Foumani, M.; Smith-Miles, K.; Gunawan, I. Scheduling of two-machine robotic rework cells: In-process, post-process and in-line inspection scenarios. Rob. Auton. Syst. 2017. [Google Scholar] [CrossRef]
  4. Aboseria, M.; Clemente, F.; Engels, L.F.; Cipriani, C. Discrete Vibro-Tactile Feedback Prevents Object Slippage in Hand Prostheses More Intuitively Than Other Modalities. IEEE Trans. Neural Syst. Rehabil. Eng. 2018. [Google Scholar] [CrossRef] [PubMed]
  5. Grazi, L.; Crea, S.; Parri, A.; Molino Lova, R.; Micera, S.; Vitiello, N. Gastrocnemius Myoelectric Control of a Robotic Hip Exoskeleton Can Reduce the User’s Lower-Limb Muscle Activities at Push Off. Front. Neurosci. 2018, 12, 71. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  6. Saracino, A.; Deguet, A.; Staderini, F.; Boushaki, M.N.; Cianchi, F.; Menciassi, A.; Sinibaldi, E. Haptic feedback in the da Vinci Research Kit (dVRK): A user study based on grasping, palpation and incision tasks. Int. J. Med. Robot. Comput. Assist. Surg. 2019, e1999. [Google Scholar] [CrossRef]
  7. Shull, P.B.; Damian, D.D. Haptic wearables as sensory replacement, sensory augmentation and trainer—A review. J. Neuroeng. Rehabil. 2015, 12, 1–13. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  8. Rusaw, D.; Hagberg, K.; Nolan, L.; Ramstrand, N. Can vibratory feedback be used to improve postural stability in persons with transtibial limb loss? J. Rehabil. Res. Dev. 2012, 49, 1239. [Google Scholar] [CrossRef]
  9. Plauche, A.; Villarreal, D.; Gregg, R.D. A Haptic Feedback System for Phase-Based Sensory Restoration in Above-Knee Prosthetic Leg Users. IEEE Trans. Haptics 2016, 9, 421–426. [Google Scholar] [CrossRef] [Green Version]
  10. Pagel, A.; Arieta, A.H.; Riener, R.; Vallery, H. Effects of sensory augmentation on postural control and gait symmetry of transfemoral amputees: A case description. Med. Biol. Eng. Comput. 2016, 54, 1579–1589. [Google Scholar] [CrossRef]
  11. Peterka, R.J.; Wall, C., III; Kentala, E. Determining the effectiveness of a vibrotactile balance prosthesis. J. Vestib. Res. Equilib. Orientat. 2006, 16, 45–56. [Google Scholar]
  12. Duclos, C.; Roll, R.; Kavounoudias, A.; Roll, J.-P.; Forget, R. Vibration-induced post-effects: A means to improve postural asymmetry in lower leg amputees? Gait Posture 2007, 26, 595–602. [Google Scholar] [CrossRef]
  13. Zambarbieri, D.; Schmid, M.; Verni, G. Sensory feedback for lower limb prostheses. In Proceedings of the Intelligent Systems and Technologies in Rehabilitation Engineering; CRC Press, Inc.: Boca Raton, FL, USA, 2001; pp. 129–151. [Google Scholar]
  14. Lauretti, C.; Pinzari, G.; Ciancio, A.L.; Davalli, A.; Sacchetti, R.; Sterzi, S.; Guglielmelli, E.; Zollo, L. A vibrotactile stimulation system for improving postural control and knee joint proprioception in lower-limb amputees. In Proceedings of the RO-MAN 2017—26th IEEE International Symposium on Robot and Human Interactive Communication, Lisbon, Portugal, 28 August–1 September 2017; pp. 88–93. [Google Scholar]
  15. Sienko, K.H.; Balkwill, M.; Oddsson, L.I.E.; Wall, C. The effect of vibrotactile feedback on postural sway during locomotor activities. J. Neuroeng. Rehabil. 2013, 10, 93. [Google Scholar] [CrossRef] [Green Version]
  16. Sabolich, J.A.; Ortega, G.M. Sense of Feel for Lower-Limb Amputees: A Phase-One Study. JPO J. Prosthet. Orthot. 1994, 6, 36–41. [Google Scholar] [CrossRef]
  17. Yang, L.; Dyer, P.S.; Carson, R.J.; Webster, J.B.; Bo Foreman, K.; Bamberg, S.J.M. Utilization of a lower extremity ambulatory feedback system to reduce gait asymmetry in transtibial amputation gait. Gait Posture 2012, 36, 631–634. [Google Scholar] [CrossRef]
  18. Crea, S.; Edin, B.B.; Knaepen, K.; Meeusen, R.; Vitiello, N. Time-Discrete Vibrotactile Feedback Contributes to Improved Gait Symmetry in Patients With Lower Limb Amputations: Case Series. Phys. Ther. 2017, 97, 198–207. [Google Scholar] [CrossRef]
  19. Jones, L.A.; Sarter, N.B. Tactile displays: Guidance for their design and application. Hum. Factors J. Hum. Factors Ergon. Soc. 2008, 50, 90–111. [Google Scholar] [CrossRef] [PubMed]
  20. Tucker, M.R.; Olivier, J.; Pagel, A.; Bleuler, H.; Bouri, M.; Lambercy, O.; Del Millán, J.R.; Riener, R.; Vallery, H.; Gassert, R. Control strategies for active lower extremity prosthetics and orthotics: A review. J. Neuroeng. Rehabil. 2015, 12, 1. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  21. Canino, J.M.; Fite, K.B. Haptic feedback in lower-limb prosthesis: Combined haptic feedback and EMG control of a powered prosthesis. In Proceedings of the 2016 IEEE EMBS International Student Conference (ISC), Ottawa, ON, Canada, 29–31 May 2016; pp. 1–4. [Google Scholar]
  22. Husman, M.A.B.; Maqbool, H.F.; Awad, M.I.; Dehghani-Sanij, A.A. Portable haptic device for lower limb amputee gait feedback: Assessing static and dynamic perceptibility. In Proceedings of the 2017 International Conference on Rehabilitation Robotics (ICORR), London, UK, 17–21 July 2017; pp. 1562–1566. [Google Scholar]
  23. Cesini, I.; Spigler, G.; Prasanna, S.; Taxis, D.; Dell’Agnello, F.; Martini, E.; Crea, S.; Vitiello, N.; Mazzoni, A.; Oddo, C.M. A wearable haptic feedback system for assisting lower-limb amputees in multiple locomotion tasks. In Biosystems and Biorobotics; Springer: Cham, Switzerland, 2019; Volume 22, pp. 115–119. [Google Scholar]
  24. Leong, J.; Parzer, P.; Perteneder, F.; Babic, T.; Rendl, C.; Vogl, A.; Egger, H.; Olwal, A.; Haller, M. proCover: Sensory Augmentation of Prosthetic Limbs Using Smart Textile Covers. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology—UIST ’16, Tokyo, Japan, 16–19 October 2016; ACM Press: New York, NY, USA, 2016; pp. 335–346. [Google Scholar]
  25. Filosa, M.; Cesini, I.; Martini, E.; Spigler, G.; Vitiello, N.; Oddo, C.; Crea, S. A new sensory feedback system for lower-limb amputees: Assessment of discrete vibrotactile stimuli perception during walking. In Biosystems and Biorobotics; Springer: Cham, Switzerland, 2019; Volume 22, pp. 105–109. [Google Scholar]
  26. Sie, A.; Realmuto, J.; Rombokas, E. A Lower Limb Prosthesias Haptic Feedback System for Stir Descent. In Proceedings of the 2017 Design of Medical Devices Conference, Minneapolis, MN, USA, 10–13 April 2017; ASME: New York, NY, USA; p. V001T05A004. [Google Scholar]
  27. Crea, S.; Cipriani, C.; Donati, M.; Carrozza, M.C.; Vitiello, N. Providing Time-Discrete Gait Information by Wearable Feedback Apparatus for Lower-Limb Amputees: Usability and Functional Validation. IEEE Trans. Neural Syst. Rehabil. Eng. 2015, 23, 250–257. [Google Scholar] [CrossRef] [PubMed]
  28. Fan, R.E.; Wottawa, C.; Mulgaonkar, A.; Boryk, R.J.; Sander, T.C.; Wyatt, M.P.; Dutson, E.; Grundfest, W.S.; Culjat, M.O. Pilot testing of a haptic feedback rehabilitation system on a lower-limb amputee. In Proceedings of the 2009 ICME International Conference on Complex Medical Engineering, Tempe, AZ, USA, 9–11 April 2009; pp. 1–4. [Google Scholar]
  29. Husman, M.A.B. A Haptic Feedback System for Lower Limb Amputees Based on Gait Event Detection. Ph.D. Thesis, University of Leeds, Leeds, UK, 2017. [Google Scholar]
  30. Rongala, U.B.; Mazzoni, A.; Oddo, C.M. Neuromorphic Artificial Touch for Categorization of Naturalistic Textures. IEEE Trans. Neural Netw. Learn. Syst. 2015, 1–11. [Google Scholar] [CrossRef]
  31. Joucla, S.; Ambroise, M.; Levi, T.; Lafon, T.; Chauvet, P.; Saïghi, S.; Bornat, Y.; Lewis, N.; Renaud, S.; Yvert, B. Generation of Locomotor-Like Activity in the Isolated Rat Spinal Cord Using Intraspinal Electrical Microstimulation Driven by a Digital Neuromorphic CPG. Front. Neurosci. 2016, 10, 67. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  32. Oddo, C.M.; Raspopovic, S.; Artoni, F.; Mazzoni, A.; Spigler, G.; Petrini, F.; Giambattistelli, F.; Vecchio, F.; Miraglia, F.; Zollo, L.; et al. Intraneural stimulation elicits discrimination of textural features by artificial fingertip in intact and amputee humans. eLife 2016, 1–27. [Google Scholar] [CrossRef] [PubMed]
  33. Spigler, G.; Oddo, C.M.; Carrozza, M.C. Soft-neuromorphic artificial touch for applications in neuro-robotics. In Proceedings of the IEEE RAS EMBS International conference on biomedical robotics and biomechatronics (BioRob), Rome, Italy, 24 June 2012; pp. 1913–1918. [Google Scholar] [CrossRef]
  34. Oddo, C.M.; Mazzoni, A.; Spanne, A.; Enander, J.M.D.; Mogensen, H.; Bengtsson, F.; Camboni, D.; Micera, S.; Jörntell, H. Artificial spatiotemporal touch inputs reveal complementary decoding in neocortical neurons. Sci. Rep. 2017, 8. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  35. Afzal, M.R.; Oh, M.K.; Lee, C.H.; Park, Y.S.; Yoon, J. A Portable Gait Asymmetry Rehabilitation System for Individuals with Stroke Using a Vibrotactile Feedback. Biomed. Res. Int. 2015, 2015. [Google Scholar] [CrossRef] [PubMed]
  36. Rokhmanova, N.; Rombokas, E. Vibrotactile Feedback Improves Foot Placement Perception on Stairs for Lower-Limb Prosthesis Users. In Proceedings of the 2019 IEEE 16th International Conference on Rehabilitation Robotics (ICORR) 2019, Toronto, ON, Canada, 24 June 2019; pp. 1215–1220. [Google Scholar]
  37. Tsang, C.S.C.; Ngan, H.Y.T.; Pang, G.K.H. Fabric inspection based on the Elo rating method. Pattern Recognit. 2016. [Google Scholar] [CrossRef] [Green Version]
  38. Elo, A.E. The Rating of Chess Players, Past & Present; Arco Pub.: New York, NY, USA, 1978; ISBN 9780923891275. [Google Scholar]
  39. Teh, K.C.; Aziz, A.R. Heart rate, oxygen uptake, and energy cost of ascending and descending the stairs. Med. Sci. Sports Exerc. 2002. [Google Scholar] [CrossRef]
  40. Riener, R.; Rabuffetti, M.; Frigo, C. Stair ascent and descent at different inclinations. Gait Posture 2002. [Google Scholar] [CrossRef]
  41. Al Kandari, J.R.; Mohammad, S.; Al-Hashem, R.; Telahoun, G.; Barac-Nieto, M. Practical Use of Stairs to Assess Fitness, Prescribe and Perform Physical Activity Training. Health 2016. [Google Scholar] [CrossRef]
  42. Ainsworth, B.E.; Haskell, W.L.; Whitt, M.C.; Irwin, M.L.; Swartz, A.M.; Strath, S.J.; O’brien, W.L.; Bassett, D.R.; Schmitz, K.H.; Emplaincourt, P.O.; et al. Compendium of Physical Activities: An update of activity codes and MET intensities. Med. Sci. Sports Exerc. 2000, 32, S498–S504. [Google Scholar] [CrossRef] [Green Version]
  43. Bassett, D.R.; Vachon, J.A.; Kirkland, A.O.; Howley, E.T.; Duncan, G.E.; Johnson, K.R. Energy cost of stair climbing and descending on the college alumnus questionnaire. Med. Sci. Sports Exerc. 1997, 29, 1250–1254. [Google Scholar] [CrossRef]
  44. Vallabhajosula, S.; Tan, C.W.; Mukherjee, M.; Davidson, A.J.; Stergiou, N. Biomechanical analyses of stair-climbing while dual-tasking. J. Biomech. 2015. [Google Scholar] [CrossRef] [Green Version]
  45. Cesini, I.; Martini, E.; Filosa, M.; Spigler, G.; Sabatini, A.M.; Vitiello, N.; Oddo, C.M.; Crea, S. Perception of time-discrete haptic feedback on the waist is invariant with gait events. IEEE Trans. Neural Syst. Rehabil. Eng. 2020. [Google Scholar] [CrossRef] [PubMed]
  46. Martini, E.; Fiumalbi, T.; Dell’agnello, F.; Ivanić, Z.; Munih, M.; Vitiello, N.; Crea, S. Pressure-sensitive insoles for real-time gait-related applications. Sensors 2020, 20, 1448. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  47. Izhikevich, E.M. Simple model of spiking neurons. IEEE Trans. Neural Networks 2003, 14, 1569–1572. [Google Scholar] [CrossRef] [Green Version]
  48. Prasanna, S.; Massari, L.; Sinibaldi, E.; Detry, R.J.; Bowkett, J.; Carpenter, K.C.; Oddo, C.M. Neuromorphic tactile sensor array based on fiber Bragg gratings to encode object qualities. Opt. Photonics Inf. Process. XIII 2019, 6. [Google Scholar] [CrossRef]
  49. Gunasekaran, H.; Spigler, G.; Mazzoni, A.; Cataldo, E.; Oddo, C.M. Convergence of regular spiking and intrinsically bursting Izhikevich neuron models as a function of discretization time with Euler method. Neurocomputing 2019. [Google Scholar] [CrossRef]
  50. Sorgini, F.; Massari, L.; D’Abbraccio, J.; Palermo, E.; Menciassi, A.; Petrovic, P.; Mazzoni, A.; Carrozza, M.; Newell, F.; Oddo, C.; et al. Neuromorphic Vibrotactile Stimulation of Fingertips for Encoding Object Stiffness in Telepresence Sensory Substitution and Augmentation Applications. Sensors 2018, 18, 261. [Google Scholar] [CrossRef] [Green Version]
  51. D’Abbraccio, J.; Massari, L.; Prasanna, S.; Baldini, L.; Sorgini, F.; Airò Farulla, G.; Bulletti, A.; Mazzoni, M.; Capineri, L.; Menciassi, A.; et al. Haptic Glove and Platform with Gestural Control For Neuromorphic Tactile Sensory Feedback In Medical Telepresence . Sensors 2019, 19, 641. [Google Scholar] [CrossRef] [Green Version]
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Figure 1. Experimental set-up: (a) overview of the haptic feedback system: the sensorized insole (bottom left) and the belt integrating three vibrotactile (VT) units and the 3D-printed box (top left); the sketch on the right shows qualitatively the wearability of the two components; (b) spatial distribution of the VT units around the user’s waist (top), and schematics of the activation patterns based on the center of pressure (CoP), with segmentation of the insole in three regions activating corresponding VT units along the axis of increasing CoPy (middle) and neuronal network mapping, with activation of sensors at the heel and inhibition of other regions at the heel-strike, as the input to the neuron model that activated VT1; (c) CoP progression with time (x-axis) for four steps of ground-level walking showing the activation of the three VT units (y-axis) for each stimulation strategy.
Figure 1. Experimental set-up: (a) overview of the haptic feedback system: the sensorized insole (bottom left) and the belt integrating three vibrotactile (VT) units and the 3D-printed box (top left); the sketch on the right shows qualitatively the wearability of the two components; (b) spatial distribution of the VT units around the user’s waist (top), and schematics of the activation patterns based on the center of pressure (CoP), with segmentation of the insole in three regions activating corresponding VT units along the axis of increasing CoPy (middle) and neuronal network mapping, with activation of sensors at the heel and inhibition of other regions at the heel-strike, as the input to the neuron model that activated VT1; (c) CoP progression with time (x-axis) for four steps of ground-level walking showing the activation of the three VT units (y-axis) for each stimulation strategy.
Electronics 09 01676 g001
Figure 2. Schematic representation of the electronic systems integrated in the sensing module and wearable haptic interface.
Figure 2. Schematic representation of the electronic systems integrated in the sensing module and wearable haptic interface.
Electronics 09 01676 g002
Figure 3. Example of sequence of stimulation strategies (CoP, Shuffled, CoP+vGRF, and Neuro) delivered to Subject 1. VT1 is depicted in red; VT2 in green and VT3 in blue. Stairs ascending steps are represented in black and descending in grey.
Figure 3. Example of sequence of stimulation strategies (CoP, Shuffled, CoP+vGRF, and Neuro) delivered to Subject 1. VT1 is depicted in red; VT2 in green and VT3 in blue. Stairs ascending steps are represented in black and descending in grey.
Electronics 09 01676 g003
Figure 4. Results on Elo rating of intuitiveness and comfort of stimulation strategies, for each subject.
Figure 4. Results on Elo rating of intuitiveness and comfort of stimulation strategies, for each subject.
Electronics 09 01676 g004
Figure 5. Median values of Elo scores. Pink circles correspond to data points. Bars indicate the upper and lower quartiles. Asterisks denote statistically significant differences.
Figure 5. Median values of Elo scores. Pink circles correspond to data points. Bars indicate the upper and lower quartiles. Asterisks denote statistically significant differences.
Electronics 09 01676 g005
Figure 6. Median values of Elo scores computed clustering trials of going up and down. Pink circles denote data points. No significant difference was found between going up and down, for both intuitiveness and comfort. Bars denote the upper and lower quartiles.
Figure 6. Median values of Elo scores computed clustering trials of going up and down. Pink circles denote data points. No significant difference was found between going up and down, for both intuitiveness and comfort. Bars denote the upper and lower quartiles.
Electronics 09 01676 g006
Figure 7. Comparison of the median time required by the subjects to walk along the indicated path, when going up, going down and combining all the trials together, under stimulation using the four stimulation strategies CoP, CoP + vGRF, Shuffled and Neuro. No significant differences were found between the stimulation strategies. Data were aggregated across all subjects’ trials, with bars indicating the upper and lower quartiles.
Figure 7. Comparison of the median time required by the subjects to walk along the indicated path, when going up, going down and combining all the trials together, under stimulation using the four stimulation strategies CoP, CoP + vGRF, Shuffled and Neuro. No significant differences were found between the stimulation strategies. Data were aggregated across all subjects’ trials, with bars indicating the upper and lower quartiles.
Electronics 09 01676 g007
Figure 8. Results of the pilot experiments. Scores of intuitiveness and comfort of the stimulation strategies on absolute scale (0–10) computed for two subjects.
Figure 8. Results of the pilot experiments. Scores of intuitiveness and comfort of the stimulation strategies on absolute scale (0–10) computed for two subjects.
Electronics 09 01676 g008
Table 1. Median (M) and interquartile range (IQR [Q1, Q3]) of the Elo scores for intuitiveness and comfort of each feedback strategy.
Table 1. Median (M) and interquartile range (IQR [Q1, Q3]) of the Elo scores for intuitiveness and comfort of each feedback strategy.
FeedbackIntuitiveness M [Q1, Q3]Comfort M [Q1, Q3]
CoP68.23 [38.98, 140.52]−56.24 [−81.39, 14.23]
CoP + vGRF72.9 [10.76, 149.16]56.08 [−16.49, 110.4]
Shuffled−51.91 [−173.8, 62.79]−71.74 [−137.01, −30.14]
Neuro−111.82 [−168.34, 22.27]52.48 [−33.43, 179.86]

Share and Cite

MDPI and ACS Style

Cesini, I.; Spigler, G.; Prasanna, S.; D’Abbraccio, J.; De Luca, D.; Dell’Agnello, F.; Crea, S.; Vitiello, N.; Mazzoni, A.; Oddo, C.M. Assessment of Intuitiveness and Comfort of Wearable Haptic Feedback Strategies for Assisting Level and Stair Walking. Electronics 2020, 9, 1676. https://doi.org/10.3390/electronics9101676

AMA Style

Cesini I, Spigler G, Prasanna S, D’Abbraccio J, De Luca D, Dell’Agnello F, Crea S, Vitiello N, Mazzoni A, Oddo CM. Assessment of Intuitiveness and Comfort of Wearable Haptic Feedback Strategies for Assisting Level and Stair Walking. Electronics. 2020; 9(10):1676. https://doi.org/10.3390/electronics9101676

Chicago/Turabian Style

Cesini, Ilaria, Giacomo Spigler, Sahana Prasanna, Jessica D’Abbraccio, Daniela De Luca, Filippo Dell’Agnello, Simona Crea, Nicola Vitiello, Alberto Mazzoni, and Calogero Maria Oddo. 2020. "Assessment of Intuitiveness and Comfort of Wearable Haptic Feedback Strategies for Assisting Level and Stair Walking" Electronics 9, no. 10: 1676. https://doi.org/10.3390/electronics9101676

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop