Designing Ecological Auditory Feedback on Lower Limb Kinematics for Hemiparetic Gait Training
Abstract
:1. Introduction
2. Materials and Methods
2.1. User-Centered Design Methodology
2.2. Feedback Algorithm Design Software
- 1.
- Real-time inertial data reception and recordings from up to five inertial sensors—M5Stack Gray (manufactured by M5Stack, sourced from Copenhagen, Denmark) devices equipped with MPU 9250 9-axis inertial units (https://shop.m5stack.com/products/grey-development-core, accessed on 28 February 2023). The MPU-9250 has, in comparison with optical systems, been shown to exhibit sufficient validity and reliability for gait analysis purposes in healthy [43,44] and patient populations [45,46]. These sensors transmitted data ( = 100 Hz) to the software over WiFi using the UDP protocol.
- 2.
- Real-time sonification of movement features computed from the raw inertial data.
- 3.
- Dedicated interface for real-time manipulation of the sonification algorithm (depicted in Figure 1).
- 4.
- Storage and recall of sonification algorithms.
- 5.
- Gait visualization from recorded and real-time data using stick figure representations.
- 6.
- Sonification of the recorded inertial data at a user-specified rate to simulate real-time motion input.
- 7.
- A range of inbuilt audio synthesizers implemented in FAUST (https://faust.grame.fr/, accessed on 28 February 2023) as well as the ability to send mapping data to third-party audio software using UDP.
2.3. Feedback Rationale
2.4. Feedback Requirements
- 1.
- Swing phase focus: It must concurrently convey segment angular velocity information during the forward rotation of the thigh and shank in the swing phase.
- 2.
- 3.
- Wireless wearable sensing and feedback actuation: All motion sensing must be carried out using wireless wearable inertial measurement units (IMUs), and the feedback should be delivered wirelessly to the patient using a suitable device (headphones or loudspeakers, depending on the training paradigm) [10].
- 4.
- Clinically compatible: The technology and its application must be compatible with existing gait training protocols for hemiparetic patients, and necessitate little or no restructuring of current clinical processes. It must be versatile in order to suit the inherent diversity of motor and cognitive impairment in patients [1,8,10,38], in addition to being inexpensive and portable.
- 5.
- Feedback design: The feedback must satisfy the following requirements for the target group:
- It must be capable of supporting motor learning without introducing cognitive barriers related to feedback interpretation:
- −
- −
- −
2.5. Motion Data Collection and Analysis
- Knee angular velocities: The angular velocities of knee rotation were calculated as the instantaneous difference between the smoothed thigh and shank SAVs on either side. Signal polarity was adjusted to achieve a common sign convention (positive values = knee extension); we refer to these angular velocities as .
- Step modulo: This was a cyclically varying measure that kept track of the number of steps completed by the user. Steps were simply tracked from the local minima and maxima of the thigh SAV exceeding an adjustable threshold. This was calculated as:
2.6. Developed Feedback Algorithms
- The values were normalized between 0 and 1, such that the rest state and backward segment rotations (negative initial values) corresponded to 0, the configurable patient-specific max for that segment corresponded to 1 and, thus, the forward rotations were in the (0, 1) range. These normalized signals (, , etc.) then underwent a series of preparatory operations such as smoothing, summing, nonlinear transformation, and linear gain to yield a parallel set of control signals –, which were bounded between 0 and 1 and then scaled to an appropriate value range for audio synthesizers and processors. All operations were defined using an analysis-by-synthesis approach. These operations took place at the sensor sampling rate of 100 Hz, and are shown on the left side of Figure 3, Figure 4 and Figure 5.
- The real-time audio output had a sampling rate of 48 kHz, a resolution of 24 bits, and a channel count of 2 (stereo).
- At any instant, immobile or backward-rotating limb segments were silent, whilst forward-rotating segments generated sounds governed by their corresponding values. It was expected that this mapping would keep the instantaneous focus of the user on the segment(s) that were in forward rotation. This could help avoid cluttering the auditory channel with non-informative sounds related to the stance phase limb. Moreover, L-R differences in thigh and shank SAV trajectories that were inherent to asymmetric gait patterns were readily reflected in the discrepant energetic properties of the resulting sounds.
2.6.1. Feedback Algorithm 1 (FA1)—Wading
2.6.2. Feedback Algorithm 2 (FA2)—Abstract Waveform
2.6.3. Feedback Algorithm 3 (FA3)—Synthesized Music
3. Hands-on Testing by Physiotherapists
3.1. Participants
3.2. Setup
3.3. Procedure
3.4. Data Analysis
3.5. Key Takeaways
With (wading) sound feedback I think we can address tempo and asymmetry issues. Of course the sounds have to be tweaked to be more concrete. [T3, Wrap-up Discussion]
I wonder if it can be used for self training where the patient had headphones on, someone comes and clicks play and the tech gives feedback instead of us giving verbal feedback during the process. [T2, Wrap-up Discussion]
...there was a relationship between speed and volume. Here you could get the feedback to come up in tempo. It goes hand in hand with the ideal that we have to have higher intensity during training. [T4, Wrap-up Discussion]
Too much information so the patient is stuck in their head and forget their body, so cannot use it. [T2, Synthesized Music]
It is hard for us that don’t have brain damage to hear whether it is the octaves going around or the timbre that is changing. [T4, Abstract Waveform]
One also needs to be at a cognitive level to be able to distinguish between the feedback and the music that gets added later. [T2, Wading]
I think one could well add something complex onto the feedback but not immediately, maybe once the brain gets used to the sound. [T5, Wrap-up Discussion]
If the sound clearly signals what you should and shouldn’t do, then there is a clearer goal in terms of what feedback is being given on. [T3, Abstract Waveform]
Talking about motivation, one should take care that one doesn’t provide a ’buzzer’ sound every time one makes a mistake. [T4, Wrap-up Discussion]
Something cool should happen when one walks normally. Right now not enough of a difference between right and wrong. [T5, Synthesized Music]
Would be nice to have different sound effects depending on what it is that we want to train. [T2, Wrap-up Discussion]
It would be good to have an option—difficulty level 1 2 3 [T1, Wading]
Compared to the first one (abstract), I felt it was clearer in this example when I hit the correct swing with the shank. Because of that, I also felt more like playing with it and seeing what would happen when I took slow and high steps. [T3, Wading]
It is personal but I would have a hard time walking with this electronic sound for half an hour. [T2, Abstract Waveform]
That’s because the tone is held, when I walk fast the notes blend together. [T3, Synthesized Music]
The ambient music doesn’t go with the intention of walking and moving, I feel more like sitting on a beach and relaxing. [T2, Wading]
4. Redesign Specifics
- 1.
- The musical pitch changes controlled by the step modulo, which interfered with the sound properties representing the SAVs.
- 2.
- The artificial and potentially annoying nature of their respective tonalities.
- Positive reinforcement (splash): Here, the goal was to reward the patient with a clear impulsive splash sound each time their shank angular velocity crossed a configurable target value. The goal of the user would be to achieve the splashing sound during every step. The difficulty level could be adjusted by altering the target SAV value.
- Negative reinforcement (urinating): The principle was to provide the user with a water-based sound with a clear negative connotation if their thigh and shank angular velocities were too small in magnitude (configurable) during the swing phase. For this, we chose the sound of a person urinating in a toilet. This paradigm worked in such a way that the urinating sound was dominant when the swing velocity was low, and masked by the loud, broadband wading sounds when the velocity was high enough. The goal of the user would be to swing their lower limb segments fast enough to prevent the urinating sound from being audible during any step.
- Envelope following: This operation was applied so as to make the falling edge of the amplitude envelope of the wading sounds more gradual and natural-sounding, while keeping the rising edge intact to ensure immediate responsiveness. We credit this approach to [62], and implemented the envelope follower as described in [58] (Chapter 12). The time constants used for the thigh and shank were different (370 ms and 670 ms, respectively).
- Rising edge impulsification: In order to randomly trigger one of three splashing sound samples when the thresholds were crossed, we converted the rising edges of the summed ON/OFF signals from the shanks to impulses of random height (see the branch in Figure 6).
5. Feasibility Study
5.1. Participants
5.2. Setup
- IKEA Eneby loudspeaker: This allowed everybody present at the training session to hear the feedback heard by the patient. The purpose of this was to help us adjust the system to the needs of the patient as well as allow the physiotherapist to monitor the feedback.
- Thomson radio frequency headphones (https://tinyurl.com/ynbmvvfc, accessed on 28 February 2023): These closed-back wireless headphones were provided to the patient in order to ensure that the feedback was provided with consistent sound quality, irrespective of the patient’s distance from the loudspeaker.
5.3. Procedure
- Whether the patient enjoyed training with the feedback.
- Whether the patient felt that the feedback correlated well with their movements.
- Whether the patient noticed a change in their gait when using the feedback.
- Whether the patient found the wading sounds natural.
- Whether the therapist observed any change in the patient’s gait when using the different feedback variants.
- Whether the therapist had any suggestions for the improvement of the feedback.
- Any miscellaneous comments.
5.4. Data Analysis
5.5. Results
5.5.1. Patient and Therapist Experiences
I think it (splash sound) could help some people, and could be an OK extra action that tells people if you do this or that then you get this sound. [P8]
I think I would have a sharpened awareness if I trained with this....it could be good when I get used to synchronizing my sound to the movement, I could imagine it being a help. [P1]
It is the best thing I have tried so far, I felt completely safe and it was like I used my ears to hear the rhythm in how I walked. [P5]
I see the trick in having an rhythmic indicator or a scale for how consistent gait is, which gives good value. [P7]
The sound and movement went fine together. [P1]
In terms of sound-movement connection, they seemed synchronized and simultaneous. [P7]
We have all walked in water, and the sound that is heard when one moves their leg is something that I can relate to. [P8]
The reproduction is completely like listening to one trying to walk through water. [P1]
I actually think it faithfully sounded like it does when one walks through water. [P3]
In terms of naturalness, I think it was OK, I don’t think it can be more natural. [P8]
Moving and also understanding the sound at the same time is hard at the start, but it comes rather quickly. [P6]
It could be even clearer when the patient goes completely against what they should be doing, like a really unpleasant sound, so they realize it’s all wrong. [T1]
I think it was a bit hard to differentiate when one does it right, but with the splash, I had an aha-moment when I could hear it. [T6]
I had a bit of difficulty hearing the pissing sound when you shifted over to it. [T4]
One could not hear the difference between the sounds, they were too close to each other, especially with the water and pissing sound (which sounds artificial). [T8]
I think that it was the focused sound in the beginning that helped (plain) as opposed to having to achieve something—going fast (splash) or NOT doing something (urinating), because sometimes it is easier not having to think too much while walking. [T3]
It needs to be very usable, so having both the water sound and some other sound, which might be too many sounds. [T9]
I would go after the sound maybe in the long run, right now it was short enough that I would still have too much focus on walking correctly. [P1]
I was focused on trying to get them (left and right) to sound identical. [P2]
In terms of changing gait, it was the sound that I went after more than actually walking. It was much more intuitive in the sound in terms of whether I did it right. [P7]
It could get irritating at some point, I do not know, but I would easily be able to walk (with it) for an hour. [P5]
I won’t have a problem listening to this for a long time except the pissing sound. [P7]
Maybe it’ll be irritating to listen to for an hour. Max 30 min would be fine, but in an hour I would probably be stressed. [P4]
In the beginning one can be more motivated to get them to sound identical, but maybe later on, when one gets tired, it can be a bit unpleasant to hear that one cannot achieve the same result that one did when one started. I think that could irritate me a bit. [P2]
The feedback could be adjusted even better to the patient so that they get that clear feedback, so that it can also be used with the better patients. [T1]
It could be nice to give feedback on how wide your steps are so as to challenge a patient to remain within a certain amount. [T8]
It could also be nice to try with cueing rather than feedback where you have to follow a rhythm rather than generating it yourself. [T8]
It was fun, something special, made sense for me, one could hear what each leg did. [P2]
It was fun to have the sound in my ears and hear myself walk. [P4]
My movement became more natural and we laughed “Can you hear me splashing?” [P5]
I don’t know how it was to walk with this sound. [P9]
I didn’t feel that it was me that made the sound, it came from outside. [P9]
The sound was artificial. [P9]
5.5.2. Kinematic Characterizations
That was clearly the best I have seen you (patient) walk, both in terms of quality and tempo, don’t think I have seen anything comparable. [T2]
It was fun, something special...I don’t think I have walked so fast before. [P2]
You did change your movement in the first one (plain), I could see in terms of the left leg (most affected) as well as the rhythm you walked in. I was actually surprised by what the first sound did. [T3]
I don’t know whether I changed my movement. [P3]
I felt like there were some improvements when you got the feedback. [T4]
I didn’t notice myself doing anything differently with my body. [P4]
... it (feedback) got you to lift your leg more than you otherwise have. [T9]
I don’t think the sound got me to walk differently. [P9]
I think that the patient adapted his walking to the sound so that he homed in on a correct walking pattern. [T1]
I think with the splash sound something happened, that we haven’t seen over the floor before, maybe only with treadmill. [T6]
I noticed that you got closer to being completely rhythmic, and also at one point your feet were closer to each other (less step width). [T8]
6. General Discussion
6.1. Angular Velocity Measurement, Clinical Compatibility, and Feedback Actuation
6.2. Feedback Design
6.3. Kinematic Changes
- The differences in sound generated by the least- and most-affected sides enhanced their awareness of their walking asymmetry, which prompted them to modify their movements to make the two sides sound more identical.
- The synchronous and repetitive rise and fall of the wading sound envelope heightened the patients’ awareness of any rhythmic irregularities in their gait, prompting them to modify their gait rhythm to make it more periodic.
6.4. Limitations and Future Work
7. Conclusions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Abbreviations
IMU | inertial measurement unit |
FA | feedback algorithm |
SAV | segment angular velocity |
T | thigh |
S | shank |
L | left |
R | right |
EQ | equalization |
FAC | functional ambulation category |
FIM | functional independence measure |
References
- Olney, S.J.; Richards, C. Hemiparetic Gait Following Stroke. Part I: Characteristics. Gait Posture 1996, 4, 136–148. [Google Scholar] [CrossRef]
- Hara, Y. Brain Plasticity and Rehabilitation in Stroke Patients. J. Nippon. Med. Sch. 2015, 82, 4–13. [Google Scholar] [CrossRef] [Green Version]
- Frost, R.; Skidmore, J.; Santello, M.; Artemiadis, P. Sensorimotor Control of Gait: A Novel Approach for the Study of the Interplay of Visual and Proprioceptive Feedback. Front. Hum. Neurosci. 2015, 9, 14. [Google Scholar] [CrossRef] [Green Version]
- Tuthill, J.C.; Azim, E. Proprioception. Curr. Biol. 2018, 28, 194–203. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Mandouit, L.; Hattie, J. Revisiting “The Power of Feedback” from the Perspective of the Learner. Learn. Instr. 2023, 84, 101718. [Google Scholar] [CrossRef]
- Sigrist, R.; Rauter, G.; Riener, R.; Wolf, P. Augmented Visual, Auditory, Haptic, and Multimodal Feedback in Motor Learning: A Review. Psychon. Bull. Rev. 2012, 20. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Brennan, L.; Zubiete, E.D.; Caulfield, B. Feedback Design in Targeted Exercise Digital Biofeedback Systems for Home Rehabilitation: A Scoping Review. Sensors 2019, 20, 181. [Google Scholar] [CrossRef] [Green Version]
- Spencer, J.; Wolf, S.L.; Kesar, T.M. Biofeedback for Post-Stroke Gait Retraining: A Review of Current Evidence and Future Research Directions in the Context of Emerging Technologies. Front. Neurol. 2021, 12, 419. [Google Scholar] [CrossRef]
- Guerra, J.; Smith, L.; Vicinanza, D.; Stubbs, B.; Veronese, N.; Williams, G. The Use of Sonification for Physiotherapy in Human Movement Tasks: A Scoping Review. Sci. Sport. 2020, 35, 119–129. [Google Scholar] [CrossRef]
- Linnhoff, D.; Alizadeh, S.; Schaffert, N.; Mattes, K. Use of Acoustic Feedback to Change Gait Patterns: Implementation and Transfer to Motor Learning Theory—A Scoping Review. J. Mot. Learn. Dev. 2020, 8, 598–618. [Google Scholar] [CrossRef]
- Nown, T.H.; Upadhyay, P.; Kerr, A.; Andonovic, I.; Tachtatzis, C.; Grealy, M.A. A Mapping Review of Real-Time Movement Sonification Systems for Movement Rehabilitation. IEEE Rev. Biomed. Eng. 2022, 16, 672–686. [Google Scholar] [CrossRef] [PubMed]
- Hunt, A.; Hermann, T. Interactive Sonification. In The Sonification Handbook; Hermann, T., Hunt, A., Neuhoff, J.G., Eds.; Logos: Berlin, Germany, 2011. [Google Scholar]
- Eddins, D.A.; Green, D.M. Temporal Integration and Temporal Resolution. Hearing 1995, 1028, 1022–1029. [Google Scholar]
- Lahav, A.; Saltzman, E.; Schlaug, G. Action Representation of Sound: Audiomotor Recognition Network while Listening to Newly Acquired Actions. J. Neurosci. 2007, 27, 308–314. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Westerman, G.; Miranda, E.R. Modelling the Development of Mirror Neurons for Auditory-Motor Integration. J. New Music Res. 2002, 31, 367–375. [Google Scholar] [CrossRef]
- Young, W.; Rodger, M.; Craig, C.M. Perceiving and Reenacting Spatiotemporal Characteristics of Walking Sounds. J. Exp. Psychol. Hum. Percept. Perform. 2013, 39, 464. [Google Scholar] [CrossRef] [PubMed]
- Dyer, J.F.; Stapleton, P.; Rodger, M. Mapping Sonification for Perception and Action in Motor Skill Learning. Front. Neurosci. 2017, 11, 463. [Google Scholar] [CrossRef] [Green Version]
- Laurienti, P.J.; Kraft, R.A.; Maldjian, J.A.; Burdette, J.H.; Wallace, M.T. Semantic Congruence is a Critical Factor in Multisensory Behavioral Performance. Exp. Brain Res. 2004, 158, 405–414. [Google Scholar] [CrossRef]
- Stein, B.E.; Stanford, T.R. Multisensory Integration: Current Issues from the Perspective of the Single Neuron. Nat. Rev. Neurosci. 2008, 9, 255–266. [Google Scholar] [CrossRef]
- Shams, L.; Seitz, A.R. Benefits of Multisensory Learning. Trends Cogn. Sci. 2008, 12, 411–417. [Google Scholar] [CrossRef]
- Linnhoff, D.; Ploigt, R.; Mattes, K. Sofigait—A Wireless Inertial Sensor-Based Gait Sonification System. Sensors 2022, 22, 8782. [Google Scholar] [CrossRef]
- Pang, T.Y.; Feltham, F. Effect of Continuous Auditory Feedback (CAF) on Human Movements and Motion Awareness. Med. Eng. Phys. 2022, 109, 103902. [Google Scholar] [CrossRef] [PubMed]
- Skvortsov, D.V.; Kaurkin, S.N.; Ivanova, G.E. A Study of Biofeedback Gait Training in Cerebral Stroke Patients in the Early Recovery Phase with Stance Phase as Target Parameter. Sensors 2021, 21, 7217. [Google Scholar] [CrossRef] [PubMed]
- Rodger, M.W.; Young, W.R.; Craig, C.M. Synthesis of Walking Sounds for Alleviating Gait Disturbances in Parkinson’s Disease. IEEE Trans. Neural Syst. Rehabil. Eng. 2013, 22, 543–548. [Google Scholar] [CrossRef] [PubMed]
- Giraldo-Pedroza, A.; Lee, W.C.C.; Lam, W.K.; Coman, R.; Alici, G. A Wearable Biofeedback Device to Increase Gait Swing Time Could Have Positive Effects on Gait among Older Adults. Sensors 2022, 22, 102. [Google Scholar] [CrossRef] [PubMed]
- Chia, F.S.; Kuys, S.; Low Choy, N. Sensory Retraining of the Leg after Stroke: Systematic Review and Meta-Analysis. Clin. Rehabil. 2019, 33, 964–979. [Google Scholar] [CrossRef] [Green Version]
- Lam, T.; Pearson, K.G. The Role of Proprioceptive Feedback in the Regulation and Adaptation of Locomotor Activity. In Sensorimotor Control of Movement and Posture; Springer: Berlin/Heidelberg, Germany, 2002; pp. 343–355. [Google Scholar]
- Rossignol, S.; Dubuc, R.; Gossard, J.P. Dynamic Sensorimotor Interactions in Locomotion. Physiol. Rev. 2006, 86, 89–154. [Google Scholar] [CrossRef]
- Giggins, O.M.; Persson, U.M.; Caulfield, B. Biofeedback in Rehabilitation. J. Neuroeng. Rehabil. 2013, 10, 1–11. [Google Scholar] [CrossRef] [Green Version]
- Bowman, T.; Gervasoni, E.; Arienti, C.; Lazzarini, S.G.; Negrini, S.; Crea, S.; Cattaneo, D.; Carrozza, M.C. Wearable Devices for Biofeedback Rehabilitation: A Systematic Review and Meta-Analysis to Design Application Rules and Estimate the Effectiveness on Balance and Gait Outcomes in Neurological Diseases. Sensors 2021, 21, 3444. [Google Scholar] [CrossRef]
- van Gelder, L.M.; Barnes, A.; Wheat, J.S.; Heller, B.W. The Use of Biofeedback for Gait Retraining: A Mapping Review. Clin. Biomech. 2018, 59, 159–166. [Google Scholar] [CrossRef] [Green Version]
- Neuhoff, J.G. Is Sonification Doomed to Fail? In Proceedings of the 25th International Conference on Auditory Display (ICAD 2019), Newcastle upon Tyne, UK, 23–27 June 2019; pp. 327–330. [Google Scholar] [CrossRef] [Green Version]
- Thomas, J.P.; Shiffrar, M. I Can See You Better if I Can Hear You Coming: Action-Consistent Sounds Facilitate the Visual Detection of Human Gait. J. Vis. 2010, 10, 14. [Google Scholar] [CrossRef] [Green Version]
- Murgia, M.; Santoro, I.; Tamburini, G.; Prpic, V.; Sors, F.; Galmonte, A.; Agostini, T. Ecological Sounds Affect Breath Duration More Than Artificial Sounds. Psychol. Res. 2016, 80, 76–81. [Google Scholar] [CrossRef] [PubMed]
- Pizzera, A.; Hohmann, T. Acoustic Information During Motor Control and Action Perception: A Review. Open Psychol. J. 2015, 8. [Google Scholar] [CrossRef]
- Dabbs, A.D.V.; Myers, B.A.; Mc Curry, K.R.; Dunbar-Jacob, J.; Hawkins, R.P.; Begey, A.; Dew, M.A. User-Centered Design and Interactive Health Technologies for Patients. Comput. Inform. Nurs. CIN 2009, 27, 175. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Craig, P.; Dieppe, P.; Macintyre, S.; Michie, S.; Nazareth, I.; Petticrew, M. Developing and Evaluating Complex Interventions: The New Medical Research Council Guidance. Br. Med. J. 2008, 337. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Lesaffre, M. Investigating Embodied Music Cognition for Health and Well-Being. In Springer Handbook of Systematic Musicology; Springer: Berlin/Heidelberg, Germany, 2018; pp. 779–791. [Google Scholar]
- Abras, C.; Maloney-Krichmar, D.; Preece, J. User-Centered Design. Bainbridge W. Encycl. Hum.-Comput. Interaction. Thousand Oaks Sage Publ. 2004, 37, 445–456. [Google Scholar]
- Lee, S.H. Usability Testing for Developing Effective Interactive Multimedia Software: Concepts, Dimensions, and Procedures. J. Educ. Technol. Soc. 1999, 2. [Google Scholar]
- Nielsen, J. Usability Engineering; Morgan Kaufmann: Burlington, MA, USA, 1994. [Google Scholar]
- Kersten-van Dijk, E.T.; Westerink, J.H.; Beute, F.; IJsselsteijn, W.A. Personal Informatics, Self-Insight, and Behavior Change: A Critical Review of Current Literature. Hum.-Interact. 2017, 32, 268–296. [Google Scholar] [CrossRef] [Green Version]
- Cho, Y.S.; Jang, S.H.; Cho, J.S.; Kim, M.J.; Lee, H.D.; Lee, S.Y.; Moon, S.B. Evaluation of validity and reliability of inertial measurement unit-based gait analysis systems. Ann. Rehabil. Med. 2018, 42, 872–883. [Google Scholar] [CrossRef] [Green Version]
- Li, J.; Liu, X.; Wang, Z.; Zhao, H.; Zhang, T.; Qiu, S.; Zhou, X.; Cai, H.; Ni, R.; Cangelosi, A. Real-time human motion capture based on wearable inertial sensor networks. IEEE Internet Things J. 2021, 9, 8953–8966. [Google Scholar] [CrossRef]
- Haji Hassani, R.; Willi, R.; Rauter, G.; Bolliger, M.; Seel, T. Validation of Non-Restrictive Inertial Gait Analysis of Individuals with Incomplete Spinal Cord Injury in Clinical Settings. Sensors 2022, 22, 4237. [Google Scholar] [CrossRef]
- Bidabadi, S.S.; Murray, I.; Lee, G.Y.F. Validation of foot pitch angle estimation using inertial measurement unit against marker-based optical 3D motion capture system. Biomed. Eng. Lett. 2018, 8, 283–290. [Google Scholar] [CrossRef] [PubMed]
- Kos, A.; Umek, A. Biomechanical Biofeedback Systems and Applications; Springer: Berlin/Heidelberg, Germany, 2018. [Google Scholar]
- Todorov, E.; Shadmehr, R.; Bizzi, E. Augmented Feedback Presented in a Virtual Environment Accelerates Learning of a Difficult Motor Task. J. Mot. Behav. 1997, 29, 147–158. [Google Scholar] [CrossRef] [PubMed]
- Schmitz, G.; Mohammadi, B.; Hammer, A.; Heldmann, M.; Samii, A.; Münte, T.F.; Effenberg, A.O. Observation of Sonified Movements Engages a Basal Ganglia Frontocortical Network. BMC Neurosci. 2013, 14, 1–11. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Thoret, E.; Aramaki, M.; Kronland-Martinet, R.; Velay, J.L. Sonifying Drawings: Characterization of Perceptual Attributes of Sounds Produced by Human Gestures. In Proceedings of the Acoustics 2012, Nantes Conference 11th Congrès Français d’Acoustique, Nantes, France, 23–27 April 2012. [Google Scholar]
- Roddy, S.; Bridges, B. Addressing the Mapping Problem in Sonic Information Design through Embodied Image Schemata, Conceptual Metaphors, and Conceptual Blending. J. Sonic Stud. 2018, in press. [Google Scholar]
- Dyer, F.J.; Stapleton, P.; Rodger, W.M.M. Sonification as Concurrent Augmented Feedback for Motor Skill Learning and the Importance of Mapping Design. Open Psychol. J. 2015, 8, 192–202. [Google Scholar] [CrossRef] [Green Version]
- Vinken, P.M.; Kröger, D.; Fehse, U.; Schmitz, G.; Brock, H.; Effenberg, A.O. Auditory Coding of Human Movement Kinematics. Multisensory Res. 2013, 26, 533–552. [Google Scholar] [CrossRef] [Green Version]
- Effenberg, A.O.; Fehse, U.; Schmitz, G.; Krueger, B.; Mechling, H. Movement Sonification: Effects on Motor Learning Beyond Rhythmic Adjustments. Front. Neurosci. 2016, 10, 219. [Google Scholar] [CrossRef] [Green Version]
- Thoret, E.; Aramaki, M.; Kronland-Martinet, R.; Velay, J.L.; Ystad, S. From Sound to Shape: Auditory Perception of Drawing Movements. J. Exp. Psychol. Hum. Percept. Perform. 2014, 40, 983. [Google Scholar] [CrossRef] [Green Version]
- Kannape, O.; Blanke, O. Agency, Gait and Self-Consciousness. Int. J. Psychophysiol. 2012, 83, 191–199. [Google Scholar] [CrossRef]
- Kantan, P.R.; Dahl, S.; Jørgensen, H.R.M.; Khadye, C.; Spaich, E.G. Designing Sonified Feedback on Knee Kinematics in Hemiparetic Gait Based on Inertial Sensor Data. In Proceedings of the SoniHED Conference on Sonification of Health and Environmental Data, virtual event, 27–28 October 2022. [Google Scholar]
- Pirkle, W. Designing Audio Effect Plug-Ins in C++: With Digital Audio Signal Processing Theory; Routledge: Abingdon-on-Thames, UK, 2012. [Google Scholar]
- Maes, P.J.; Buhmann, J.; Leman, M. 3Mo: A Model for Music-Based Biofeedback. Front. Neurosci. 2016, 10, 548. [Google Scholar] [CrossRef] [Green Version]
- Bergstrom, I.; Seinfeld, S.; Arroyo-Palacios, J.; Slater, M.; Sanchez-Vives, M.V. Using Music as a Signal for Biofeedback. Int. J. Psychophysiol. 2014, 93, 140–149. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Smith, J.O. Physical Modeling using Digital Waveguides. Comput. Music J. 1992, 16, 74–91. [Google Scholar] [CrossRef]
- Groß-Vogt, K.; Svoronos-Kanavas, I.; Weger, M.; Amon, C. The Augmented Floor-Assessing Auditory Augmentation. In Proceedings of the 17th International Audio Mostly Conference, St. Pölten, Austria, 6–9 September 2022; pp. 7–14. [Google Scholar]
- Madgwick, S. An Efficient Orientation Filter for Inertial and inertial/magnetic sensor arrays. Rep. X-Io Univ. Bristol (UK) 2010, 25, 113–118. [Google Scholar]
- Kantan, P.; Spaich, E.G.; Dahl, S. A Technical Framework for Musical Biofeedback in Stroke Rehabilitation. IEEE Trans. Hum.-Mach. Syst. 2022, 52, 220–231. [Google Scholar] [CrossRef]
- Umek, A.; Kos, A. Validation of smartphone gyroscopes for mobile biofeedback applications. Pers. Ubiquitous Comput. 2016, 20, 657–666. [Google Scholar] [CrossRef] [Green Version]
- Boothroyd, A. Room Acoustics and Speech Perception. In Proceedings of the Seminars in Hearing; Thieme Medical Publishers, Inc.: New York, NY, USA, 2004; Volume 25, pp. 155–166. [Google Scholar]
- Kjellberg, A. Effects of Reverberation Time on the Cognitive Load in Speech Communication: Theoretical Considerations. Noise Health 2004, 7, 11. [Google Scholar]
- Kantan, P.; Spaich, E.G.; Dahl, S. An Embodied Sonification Model for Sit-to-Stand Transfers. Front. Psychol. 2022, 13, 806861. [Google Scholar] [CrossRef]
- Neuhoff, J.G.; Wayand, J.; Kramer, G. Pitch and Loudness Interact in Auditory Displays: Can the Data Set Lost in the Map? J. Exp. Psychol. Appl. 2002, 8, 17. [Google Scholar] [CrossRef]
- Geerars, M.; Minnaar-van der Feen, N.; Huisstede, B.M. Treatment of Knee Hyperextension in Post-Stroke Gait. A Systematic Review. Gait Posture 2022, 91, 137–148. [Google Scholar] [CrossRef]
- Iyendo, T.O. Sound as a Supportive Design Intervention for Improving Health Care Experience in the Clinical Ecosystem: A Qualitative Study. Complement. Ther. Clin. Pract. 2017, 29, 58–96. [Google Scholar] [CrossRef]
- Kantan, P.R.; Dahl, S.; Serafin, S.; Spaich, E.G. Sonifying Gait Kinematics Using the Sound of Wading: A Study on Ecological Movement Representations. In Proceedings of the 28th International Conference on Auditory Display (ICAD 2023), Norrköping, Sweden, 26–30 June 2023. under review. [Google Scholar]
- Grond, F.; Berger, J. Parameter Mapping Sonification. In The Sonification Handbook; Hermann, T., Hunt, A., Neuhoff, J.G., Eds.; Logos Verlag: Berlin, Germany, 2011. [Google Scholar]
- Dyer, J.; Stapleton, P.; Rodger, M. Advantages of Melodic over Rhythmic Movement Sonification in Bimanual Motor Skill Learning. Exp. Brain Res. 2017, 235, 3129–3140. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Rocchesso, D.; Andolina, S.; Ilardo, G.; Palumbo, S.D.; Galluzzo, Y.; Randazzo, M. A Perceptual Sound Space for Auditory Displays Based on Sung-Vowel Synthesis. Sci. Rep. 2022, 12, 19370. [Google Scholar] [CrossRef] [PubMed]
ID | Gender | Age | FAC | FIM-Cog | Location of Stroke/Injury | Walking Aids |
---|---|---|---|---|---|---|
P1 | M | 62 | 4 | 29 | Left middle cerebral artery | None |
P2 | F | 43 | 2 | 25 | Cerebellum | High rollator walker |
P3 | F | 78 | 1 | 26 | Right middle cerebral artery | Therapist support, Walking stick |
P4 | M | 25 | 5 | n/a | Left thalamus | None |
P5 | F | 68 | 1 | 20 | Right thalamus | Support bench |
P6 | M | 52 | 5 | 23 | Middle cerebral artery | None |
P7 | M | 53 | 5 | 31 | Right medulla oblongata | None |
P8 | M | 51 | 4 | 31 | Right subdural, Bilateral subarachnoid | None |
P9 | F | 67 | 1 | 21 | Right basal ganglia | None |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Kantan, P.R.; Dahl, S.; Jørgensen, H.R.; Khadye, C.; Spaich, E.G. Designing Ecological Auditory Feedback on Lower Limb Kinematics for Hemiparetic Gait Training. Sensors 2023, 23, 3964. https://doi.org/10.3390/s23083964
Kantan PR, Dahl S, Jørgensen HR, Khadye C, Spaich EG. Designing Ecological Auditory Feedback on Lower Limb Kinematics for Hemiparetic Gait Training. Sensors. 2023; 23(8):3964. https://doi.org/10.3390/s23083964
Chicago/Turabian StyleKantan, Prithvi Ravi, Sofia Dahl, Helle Rovsing Jørgensen, Chetali Khadye, and Erika G. Spaich. 2023. "Designing Ecological Auditory Feedback on Lower Limb Kinematics for Hemiparetic Gait Training" Sensors 23, no. 8: 3964. https://doi.org/10.3390/s23083964
APA StyleKantan, P. R., Dahl, S., Jørgensen, H. R., Khadye, C., & Spaich, E. G. (2023). Designing Ecological Auditory Feedback on Lower Limb Kinematics for Hemiparetic Gait Training. Sensors, 23(8), 3964. https://doi.org/10.3390/s23083964