Design and Evaluation of SONIS, a Wearable Biofeedback System for Gait Retraining

Abstract

Herein, we introduce SONIS, a wearable system to support gait rehabilitation training after a lower extremity trauma, which combines a sensing sock with a smartphone application. SONIS provides interactive, corrective, real-time feedback combining visual and auditory cues. We report the design of SONIS and its evaluation by patients and therapists, which indicates acceptance by targeted users, credibility as a rehabilitation tool, and a positive user experience. SONIS demonstrates how to successfully combine a number of feedback strategies and modalities: graphical, verbal, and music feedback on gait quality during training (knowledge of performance) and verbal and vibrotactile feedback on gait tracking (knowledge of results).

1. Introduction

Fractures of weight-bearing bones can cause an abnormal gait, which leads to severe temporary disabilities and loss in quality of life [1,2]. Trauma patients with fractures of the pelvis or the lower extremities follow an intensive rehabilitation program aiming to restore their pre-injury health status and gait [3]. Digital biofeedback systems for physical rehabilitation have been proposed as useful means for improving motor learning, as well as engagement with and adherence to rehabilitation programs [4]. Such systems can help patients to execute training exercises correctly by providing objective measurements which portray patient progress and can help them set quantifiable training goals. For example, they can help provide a detailed gait analysis, tracking gait in real time and offering assistive feedback to the patient. Prior to the emergence of biofeedback technologies, such objective and precise information was unavailable, and patients would need to rely on intrinsic feedback mechanisms (their senses) and the therapist advising them.

In comparison to other sensing technologies which support gait analysis, such as computer-vision-based systems, wearable sensing technologies offer potential advantages with respect to flexibility and costs. Wearable gait biofeedback systems can potentially extend the benefits from technology-supported rehabilitation to a broader audience who do not have sufficient access to gait retraining facilities, thus increasing training intensity. However, home rehabilitation programs have delivered mixed results due to low-intensity training and poor patient compliance [5]. Key to the success of biofeedback gait retraining solutions are their comfort, wearability, and the extent to which feedback is perceived as meaningful and motivating by the patient [6]. This observation suggests the need to involve users in testing biofeedback concepts and to extend such evaluation to cover experiential aspects of the interaction and the attitudes patients hold about the designed system as a therapeutic aid.

In rehabilitation training, biofeedback can serve two important purposes [7,8]:

• Guidance to steer patients on how to develop the skill, supporting knowledge of performance. For gait retraining, it could consist of kinematic information about the performed movement, e.g., “Turn your foot more to the right.”
• Correction and encouragement to strive towards a goal, supporting knowledge of results. In the case of gait retraining, it could consist of information about the outcome of an applied skill or performance, e.g., “You missed the target by 20 cm.”

Knowledge of performance is further differentiated into [7] descriptive feedback, which describes the nature of a performance error made, and prescriptive feedback, which describes the error and offers a suggestion as to how to correct a performance error. While negative feedback accelerates learning, positive feedback increases retention of the motor memory [9], so both are needed. Earlier research has argued for the need to combine positive and negative feedback [10].

A great variety of wearable biofeedback devices for gait rehabilitation have been designed, some of which are summarized in

Table 1. An overview of biofeedback applications for gait retraining and the feedback modalities they use, and a comparison with SONIS.

Reference	Sensor	Wearable	Haptic Feedback	Visual Feedback	Audio Feedback
					Sonification	Voice
	Resistive textile pressure sensor	SONIS (sock)	X	X	X	X
Baram, Y. [11]	Inertial Measurement Unit (IMU)	Body-mounted casing		X	X
Casamissima, F., et al. [12]	IMU	On-shoe wearable				X
SensiStep [13]	Force-sensitive pressure sensor	SensiStep (custom-made sandal)		X	X
Grosshauser, T., et al. [14]	IMU, pressure switches, force resistive sensors, goniometer	Insole and knee attachment			X
Fitness Gaming [15]	Resistive textile pressure sensor	Sensoria (sock)		X		X
Horsak, B., et al. [16]	IMU, force-sensitive resistor	SONIGait (insole)		X	X
Lurie, K., et al. [17]	Vicon 3-D motion tracking	Reflective markers	X
Mazilu, S., et al. [18]	IMU	Ankle-mounted casing			X
Maulucci, R.A., et al. [19]	Pressure switches, electromagnetic tracking device	Shoe			X
Redd, C.B. et al. [20]	Force-sensitive resistors	Insole	X	X	X
Winfree, K., et al. [21]	Force-sensitive resistors	PDShoe (shoe)	X
Wu, X. et al. [22]	IMU, resistive textile pressure sensor array	Insole		X
Zanotto, D., et al. [23]	IMU, piezo-resistive sensors	Sandal	X

. In such applications, audio feedback is quite commonly used to present temporal and spatial information which can help guide and motivate users in performing repetitive and hard tasks. However, auditory feedback is challenging to design, as it can be perceived as monotonous or unpleasant, which may produce the opposite effect to that desired.

The success of an audio-based gait feedback system depends on the design and integration of the auditory display into the patient’s rehabilitation context. Where these solutions have been evaluated empirically, the results are overall positive regarding the feasibility of influencing the gait of the user. Of the different modalities that could be used to provide feedback during gait rehabilitation, audio offers several advantages. For example, a recent literature survey of voice-based feedback for gait rehabilitation provided moderate evidence regarding its effectiveness [4], while calling for more research on biofeedback mechanisms. In this research we explore non-speech audio as a gait biofeedback modality. Current research on providing gait feedback has focused primarily on developing unobtrusive sensing devices, integrating them into wearables, and enhancing their performance. Less attention has been paid to interaction and user experience design perspectives. These, however, are key for ensuring that patients will actually use a gait feedback system and appreciate it as a rehabilitation aid.

We present the design and preliminary evaluation of SONIS, a wearable gait biofeedback system to support gait retraining after a fracture of the lower extremities. SONIS was designed as a smart sock. A gait retraining system ideally needs to monitor several clinical aspects of a weight-bearing bone fracture to ensure effective treatment, e.g., the amount of edema in the foot and the foot temperature to detect possible inflammation, when patients train in their home environment. With these future extensions in mind we designed a biofeedback system based on a smart sock, which offers the additional practical advantage that patients can wear it with different shoes and clothes.

This paper contributes the following:

• A wearable (smart sock) solution for monitoring gait quality for rehabilitation after a fracture of the lower extremities (see Figure 1);
• Insights into the challenges of feedback design for gait rehabilitation;
• Assessment of the usability and credibility of SONIS as a rehabilitation technology.

2. Wearables for Gait Quality Monitoring

An essential aspect of a biofeedback system is the sensing technology. There is already a substantial body of research on wearable devices for gait quality monitoring. Most of these rely on force-sensing resistive sensors in combination with other sensors (e.g., accelerometers) to assess walking rhythm and the unwinding of the foot [1,22,24,25].

Investigations in this direction have focused primarily on establishing the technical feasibility of assessing gait quality via a wearable sensor-based system and the efficacy of the biofeedback mechanism. Less attention has been paid to aspects like wearability, comfort, and aesthetics. However, in recent years, low-cost textile-based sensors have emerged as an alternative to hard sensors for gait monitoring with potential benefits regarding the latter considerations [25,26,27]. These developments offer the opportunity to integrate textile sensors into “smart” socks, which offers several advantages over other wearables, like shoes or in-soles: close contact to the joints that have to be monitored, sensors fitted all around the foot, and the ability to wear them with different shoes.

3. Design and Realization of SONIS

SONIS was designed in a patient-centered and therapist-centered iterative design approach. An initial set of interviews and observations of training sessions examined the needs of patients and therapists relating to biofeedback systems. This initial exploration informed an iterative design process, with different design iterations exploring different aspects of the system. Prototypes at each iteration were user tested with a small number of patients and therapists (2–3 of each depending on the iteration). Figure 2 illustrates the evolution of the prototype along three major phases in its development.

SONIS consists of in-sock embedded textile pressure sensors to detect gait patterns, an attachable wireless unit to collect data from the sensors and transmit them to a smartphone, and a smartphone application providing real-time audio feedback (see Figure 3). SONIS is designed to achieve the following:

• Capture pressure data simultaneously for both feet during walking;
• Provide real-time feedback;
• Use low-cost materials;
• Be unobtrusive.

SONIS emphasizes spatiotemporal aspects of gait correction: unwinding of the foot and walking rhythm.

Pressure sensors are mounted under the anterior and posterior heel, first metatarsal, fourth metatarsal, and great toe. The pressure sensor division was designed to provide a general picture of heel-strike and toe-off timing (i.e., walking rhythm) and unwinding of the foot [28]. The symmetry and regularity of walking are two important aspects in gait analysis. An asymmetry of gait between the left and right feet indicates low walking ability, as does low similarity [29,30]. Thus, SONIS calculates and compares the walking rhythm, as well as the unwinding of both feet. The same approach as in the study by Gonzalez et al. [31] was applied to detect gait phases. Further, SONIS identifies one additional gait phase: “full foot contact”. The walking rhythm and unwinding of the trauma-affected leg target the walking rhythm and unwinding of the “good” leg in order to pursue the goal of symmetry and regularity in walking.

In order to align the sensors at the front of the sock while walking, a hallux valgus sock was used as a basis (Ihle Strumpf GmbH 55.000.001.14). The sock’s form (two separate compartments for big and small toes) enables a good fit at the front of the foot and ensures correct sensor alignment. EeonTex textiles were used to fabricate our own unobtrusive, integrated, and wearable pressure sensors due to the textiles’ piezoresistive characteristics. From EeonTex (Eeonyx, EeonTex LLT-SLPA 20k Ohms—EET081), five textile sensors with a size of 3 cm × 2 cm were cut. Using silver-plated thread 234/34-4ply (SparkFun Electronics DEV-08549), a stitched circuit was made and embedded onto the sock. Two opposing hook-shaped circuits were stitched onto the EeonTex sensor to increase the sensitivity of the sensor surface (see Figure 2). The pressure sensors were connected in series with a 56 kΩ resistor to create a voltage divider. An Arduino Pro Mini 3.3V (ATmega328, 8 MHz processor; SparkFun Electronics DEV-11114) was used as a microcontroller. The ADC (analog to digital converter) of the microcontroller sampled at a 10-bit resolution. The data were transmitted using an HC-05 zs-040 version Bluetooth V2.0 + EDR (enhanced data rate) module with a baud rate of 115,200 bps.

The EeonTex sensor sensitivity differs for each textile pressure sensor, possibly due to handicraft- and location-related deviations in the textile fabric, while the hysteresis was negligible during tests. The measured output voltage for each voltage divider differed for the non-actuated and pressurized states, meaning that each pressure sensor differs in terms of non-actuated resistance and sensitivity range. These differences in sensor characteristics were handled software-wise by manually specifying three static parameters in the software for each sensor:

• A non-actuated voltage output;
• A pressurized voltage output, both of which define the measuring range;
• A threshold voltage output, manually given for each sensor to define when a small amount of pressure is applied.

The static parameters (1) and (2) were used to visualize the amount of pressure applied, as shown in the left-hand visualization in Figure 3. Static parameter (3) was used to define whether a sensor is effectively pressurized; see the right-hand visualization in Figure 3. This approach resulted in a feedback system which could be perceived as real-time, truthful, and accurate by users.

4. Feedback Design

In the early stages of therapy, the patient and therapist both need a feedback strategy which is clear and continuous. The SONIS system is envisioned to support gait training after these early stages of therapy, i.e., when several sessions have already taken place between patient and therapist, with feedback that adapts to and interacts with the performance of the user. Here, patient and therapist are expected to spend at least one or two sessions together to familiarize the patient with the biofeedback system before starting the home rehabilitation program.

A set of guidelines for feedback design for technology-supported rehabilitation training [32] provided the starting point for designing the gait feedback. The SONIS feedback combines multiple modalities, stepwise guidance, contextualization of information, prevention of information overflow, and customization. SONIS supports two feedback modes: a gait tracking mode which monitors the gait through the day, and a gait training mode which is activated on demand. To prevent patients becoming dependent on feedback, rather than learning to work based on their own proprioception, the feedback is not delivered continuously [33]. Rather, a layered feedback schedule is implemented, combining three strategies: summary feedback (after a number of trials have been completed), fading feedback (frequency of feedback is high during initial stages and faded towards the end of acquisition), and bandwidth feedback (feedback only if the performance errors are outside the range of correctness, which gets more precise as the patient improves).

To increase the trustworthiness of the system and visualize the gait quality, a graphical interface visualizes real-time pressure data mapped to the sensor locations of both feet (see Figure 3). The pressure value is mapped to circle size and color in a visualization that has been used in the context of gamified rehabilitation for stroke survivors [34]. The unwinding of the foot is visualized through creating an active/non-active gait subphase pattern over time, similar to the approach used in [31]. The orange line represents the trauma-affected leg and the blue line represents the “good” leg. A symmetrical and rhythmic graphical pattern indicates a good gait [29,30].

In the gait tracking mode (which, in principle, could be used throughout the day), a simple notification is provided after a number of consecutive wrong steps have been detected. The patient can choose to be informed through vibration, audio, or a combination thereof. This feedback presents information about the outcome of the skill performance (knowledge of results feedback [8]).

Gait training can be provided through voice instruction, subtle music changes, or a combination of both. This feedback mode is intended for training sessions, e.g., for 30 min a day, in order to intensively train the user’s gait.

Voice instructions inform patients about their performance in terms of the selected gait parameters (knowledge of performance), e.g., “A difference in the unwinding of both feet has been detected. Land on the heel, and roll through to your toes.” Prescriptive verbal feedback (describing the errors and suggesting how to correct them) has been argued [7] to be more effective than descriptive feedback (describing the errors made). When a good performance is detected, this specific gait parameter is mentioned and the user is praised. To avoid repetitiveness, a variety of voice messages have been pre-recorded for each feedback possibility, and one of them is played randomly. Voice instructions were generated using the IVONA Text-To-Speech synthesizer, speaker: Dutch, Ruben [35], and recorded using Audacity^®. In order to prevent cognitive overload, voice instructions were slowed down to a suitable speaking rate.

Contrary to voice instructions, subtle music changes that are dynamic, non-linguistic, deformed, and vague provide background and peripheral cues based on a user-selected music source. Consecutive bad steps decrease the cut-off frequency and resonance of a low-pass Moog VCF (voltage-controlled filter), resulting in deformed music, while good steps correct this. This feedback mode is intended to support longer training activities, e.g., while walking a certain distance.

At first, the incoming data are captured over the serial port and processed by a Moving Average Filter (n = 5) to smooth the data set. For both the walking rhythm and unwinding of the foot, the sensor values are compared to a pressure threshold to detect whether a sensor was pressed or not.

To detect the walking rhythm of each foot, the stance time is calculated using the heel contact and toe-off timing. The timing difference between the feet needs to stay within a specified range, because otherwise a step is categorized as bad. It is possible to detect which foot has a slower walking rhythm by subtracting the stance times of the left and right feet from each other.

Based on work by [31], the idea arose to detect different gait sub phases; swing, heel-strike, mid-stance, terminal, stance, pre-swing, and flat foot phase. By comparing the sub phases detected within 12 successive steps, the system can decide whether to consider the separate unwinding of both feet as identical.

5. User Evaluation

The system was evaluated with patients to assess how they perceive it as a rehabilitation aid and the suitability of the feedback design using a functional prototype.

5.1. Method

A mixed-method triangulated study design was chosen [36] wherein quantitative and qualitative data are collected using indirect observation and self-reporting. Participants provided informed consent and were put to no risk. The burden of the study was minimal, pertaining to trying on the sock for a few minutes and answering questionnaire and interview questions. After introducing the participants to SONIS, we asked them to put it on and try its different feedback modes for about 10 min. At the end, the designer (the first author) conducted semi-structured interviews regarding the participant’s experience and preference and administered a short questionnaire. The interviews were semi-structured, aiming to gather qualitative information relating to user acceptance of SONIS [37]. The questionnaire was aimed at assessing the user experience of participants and their attitudes towards the device as an aid for rehabilitation. The questionnaire was based on two validated and widely used instruments: the “User Experience Questionnaire” and the “Credibility and Expectancy Questionnaire”.

The User Experience Questionnaire [38] supports a holistic assessment of the user experience. It includes 26 semantic differentials of 7 levels, organized into 6 subscales that respectively measure the experienced attractiveness, perspicuity, efficiency, dependability, stimulation, and novelty.

The Credibility and Expectancy Questionnaire [39] includes two subscales which assess how much respondents believe the system can help improve their condition, measured by the credibility subscale, and to what extent they feel it can lead to improvements of it, measured by the expectancy subscale. The credibility and expectancy subscales include three items each, with each item being a 9-point Likert scale (1: no agreement, 9: full agreement). This instrument emphasizes that what people rationally assess as the potential benefit of the method is different to what they feel this benefit might be [40].

In total, 15 participants (age 44.7 ± 18.8 years) volunteered to evaluate the SONIS system. Five healthy participants with no severe abnormalities of gait evaluated the user experience of the system in their home environment. Ten patients with deficient gait quality participated in the study (three in-patients and seven out-patients). These patients were receiving or had recently received physiotherapy to improve their gait quality.

5.2. Results

During all evaluations with patients, the researcher observed an increase in the patient’s self-awareness about walking quality after audio feedback was given. In one case, a patient started off walking with an asymmetrical walking rhythm and unwinding of the foot. After receiving several voice instructions concerning the walking rhythm and unwinding of the foot, the patient could maintain a symmetrical gait and responded enthusiastically to this result. SONIS could help this patient to correct his gait without the intervention of a therapist. Participants often paid more attention to the visual feedback at the start of the session compared to later, by which time their focus would shift towards the audio feedback. The focus often shifted back to the visual feedback when the patient was not able to correct a certain deviation in the gait symmetry.

Interviews were transcribed and subjected to a conventional content analysis [41] by the first author. Most patients thought that the different gait training modes would be useful for different purposes and usage in different contexts, e.g., voice instructions would be more or less equivalent to a therapy session, and subtle music changes would be used as background feedback when the patient walks for longer time periods, such as in the forest or to the supermarket. Some patients praised the self-sufficiency they experienced with this system to monitor and independently correct/train their gait, as opposed to relying on the supervision and feedback of a therapist. In the words of one participant (translated from Dutch), “It is convenient that you can do your own therapy. Because making an appointment with the therapist is not always convenient, and sometimes it does not suit him. And now you do the therapy when it suits you. (…) If I would have this product at this moment, I would certainly use it. (outpatient)”. Or, as another stated, “I would be more comfortable if I could train alone compared to with a therapist, who is constantly standing beside me. So, if the system has been developed to such a level that you can set a program which allows me to train alone, I think this is perfect”. Contrary to those remarks, all patients emphasized the value of the potential symbiosis of therapy with a regular physiotherapist and therapy with the SONIS system to support the rehabilitation. One of the outpatients stated, “I think it is important to do this together with a physiotherapist. At first you get an explanation on how you need to walk, and where to pay attention to. And then you have this system to indicate when it is going good and when it does not go so well.” None of the participants stated having worries or fears about using SONIS. Next to this, patients stated that the SONIS system does not interfere with other walking aids they use.

To the question “What should change about the current system?” one patient expressed the wish for direct feedback after each step to maximize the learning effect. Another participant expressed the wish for a graphical overview showing the gait performance over time as a post-training summary. No further suggestions for improvement were offered.

Participants also found SONIS easy to use and the feedback clear; only the visualization concerning the unwinding of the foot was perceived by many participants as difficult to understand.

The findings from the User Experience Questionnaire (UEQ) showed a positive evaluation for all scales; see Figure 4 for the UEQ results. The 95% confidence intervals for scores along each of the UEQ dimensions were above neutral. The UEQ offers a benchmark (which is continuously updated) based on earlier product assessments using the UEQ. Schrepp et al. [38] developed the benchmark from testing 452 products with a total of 20,190 participants in all evaluations. SONIS was benchmarked as “excellent”, scoring in the range of the top 10% best results.

The three items comprising the credibility and expectancy subscales were added up with a potential minimum score of 3 and a potential maximum score of 27 for each subscale. The patients who participated in the evaluation rated SONIS as highly credible (M = 22.5 in a scale from 3 to 27, SD = 3.14) and had a moderate to high expectancy (M = 18.3 in a scale from 3 to 27, SD = 3.49) that the system could improve the quality and speed of gait rehabilitation.

The informal interviews, completed questionnaires, and observations confirmed the design rationale for the SONIS system. The different audio feedback modes were thought to provide added value for rehabilitation both by patients and by physiotherapists. With regards to the question “Would you like to use a SONIS-like product during your current rehabilitation?”, the overall answer was “Yes, if it contributes to my rehabilitation.”

6. Discussion

SONIS is a wearable system providing multimodal biofeedback for gait retraining. It supports complementary auditory and visual feedback displays to help rehabilitating patients to correct their gait, emphasizing symmetry in their walking rhythm. The system was iteratively developed to fit the sensing technology into a sock and to provide feedback through a handheld device (tablet or smartphone). Rehabilitation patients appreciated the different auditory displays, each of which finds applications in various contexts throughout a patient’s day. Voice instruction and subtle music change feedback were perceived by patients as stimulating, pleasant, and suitable for providing information efficiently to the patient to correct walking. This finding is in line with results from Singh et al., who argued that different sonification strategies can facilitate different needs in separate rehabilitation phases [42,43].

During this evaluation, the prototype was only available for foot size EUR43–46 (US 10–13) which could not ensure correct sensor placement for all participants. Three patients indicated that their gait has a permanent deviation in symmetry due to trauma, consisting of a pronation of the foot and a higher toe-out angle. SONIS was not able to adapt to this situation, which resulted in continuous incorrect feedback, which suggests the need for designing a more flexible calibration/personalization procedure.

While the evaluation was conducted in the context where rehabilitation training normally takes place, it was still conducted under relatively controlled conditions. More confidence in the efficacy of SONIS as a rehabilitation technology can be gained in future studies by quantitatively demonstrating the efficacy of the solution using observational data, by extending the testing period to at least a number of days, and by conducting such tests in the homes of participants without researchers present to assess its suitability in a tele-rehabilitation context. The scheduling aspects [42] of the feedback need to be carefully investigated in order to support the integration of biofeedback for gait retraining into daily life. Too much or too little feedback could be detrimental to performance and the user experience [44,45]. Allowing the user to select their own music for the “subtle music change” feedback enhances the user experience.

SONIS is the first smart sock to be developed to support gait quality monitoring for therapy purposes. Gait parameter sonification has also been demonstrated in the SoniGait insole [12], though it only provides continuous feedback as opposed to the variety of feedbacks combined in SONIS to support the rehabilitation process.

A recent laboratory experiment [46] on using prescriptive feedback on gait rhythm showed that combined haptic and audio was the easiest for their (healthy) participants to synchronize to. This result represents an interesting possibility for future development of SONIS, though caution should be exercised against creating a dependency on continuous rhythm feedback.

Efforts are still needed to determine the most optimal sensor and hardware design with respect to sensitivity, specificity, hysteresis, durability (wear and tear), and textile design. The comfort and robustness of the sock can be improved further by using soft, embedded electronics, while the precision of the placement of sensors can be improved by using digital fabrication techniques. The current sock design is received positively by patients and therapists, though quantitative results about gait detection and feedback results are still needed to prove the capabilities of SONIS from a clinical perspective.

The design of the hallux valgus sock contributes to the sensor alignment and garment positioning, yet possible misalignment due to slip has not yet been addressed in this design. Garment drift during daily usage is an important challenge for e-textile-based sensor systems [47]. High-friction elements, for example, elastomer beads, on the inner surface of the sock may decrease slip, but caution is needed to avoid compromising comfort and damaging the skin. Investigating different textile sensor shapes to solve issues related to misalignment and garment drift thus seems a more promising direction.

Hygiene is a key factor to successfully embed a sensor sock system in daily life. Future research should examine the effects of washing on the piezo resistive sensors, hardware, and shedding of conducting material. A combination of thermoplastic polyurethane protection film for the conductive threads (and flexible printed circuit board) and a latex-based barrier for the rigid printed circuit board and electronic components seems a promising direction for washable, durable, and reliable textile-based electronics [48]. Incorporating a battery into such washable systems is still a point of research.

7. Conclusions

Herein, we presented SONIS, a novel smart sock device for gait rehabilitation that addresses problems of sensor placement and overall comfort through the use of textile sensors. Drawing on theory and earlier works, this paper provides a resource for designers interested in feedback design for rehabilitation and motor learning more generally. The design of SONIS emphasizes the importance of combining multiple modalities for notifications (vibrotactile, voice, music, and graphics) to display feedback to help patients improve their gait and to help them track their progress. Its evaluation with patients in a rehabilitation clinic provided encouraging evidence regarding the attitudes of patients and therapists towards the device as a rehabilitation aid and the overall user experience. Future research should aim to provide clinical evidence of the efficacy of the device in supporting patient rehabilitation.