Next Article in Journal
Design and Development of Large-Band Dual-MSFA Sensor Camera for Precision Agriculture
Next Article in Special Issue
Unobtrusive Cognitive Assessment in Smart-Homes: Leveraging Visual Encoding and Synthetic Movement Traces Data Mining
Previous Article in Journal
Experimental Study of Fully Passive, Fully Active, and Active–Passive Upper-Limb Exoskeleton Efficiency: An Assessment of Lifting Tasks
Previous Article in Special Issue
A Hybrid Protection Scheme for the Gait Analysis in Early Dementia Recognition
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sensors
Volume 24
Issue 1
10.3390/s24010065
sensors-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Sonification for Personalised Gait Intervention

by
Conor Wall
1,
Peter McMeekin
2,
Richard Walker
3,
Victoria Hetherington
4,
Lisa Graham
5 and
Alan Godfrey
1,*
1
Department of Computer and Information Sciences, Northumbria University, Newcastle upon Tyne NE1 8ST, UK
2
Department of Nursing, Midwifery and Health, Northumbria University, Newcastle upon Tyne NE1 8ST, UK
3
Northumbria Healthcare NHS Foundation Trust, North Shields NE29 8NH, UK
4
Cumbria, Northumberland Tyne and Wear NHS Foundation Trust, Wolfson Research Centre, Campus for Ageing and Vitality, Newcastle upon Tyne NE4 9AS, UK
5
Department of Sport, Exercise and Rehabilitation, Northumbria University, Newcastle upon Tyne NE1 8ST, UK
*
Author to whom correspondence should be addressed.
Sensors 2024, 24(1), 65; https://doi.org/10.3390/s24010065
Submission received: 10 November 2023 / Revised: 19 December 2023 / Accepted: 20 December 2023 / Published: 22 December 2023
(This article belongs to the Special Issue Human Activity Recognition in Smart Sensing Environment)
Browse Figures
Versions Notes

Abstract

:
Mobility challenges threaten physical independence and good quality of life. Often, mobility can be improved through gait rehabilitation and specifically the use of cueing through prescribed auditory, visual, and/or tactile cues. Each has shown use to rectify abnormal gait patterns, improving mobility. Yet, a limitation remains, i.e., long-term engagement with cueing modalities. A paradigm shift towards personalised cueing approaches, considering an individual’s unique physiological condition, may bring a contemporary approach to ensure longitudinal and continuous engagement. Sonification could be a useful auditory cueing technique when integrated within personalised approaches to gait rehabilitation systems. Previously, sonification demonstrated encouraging results, notably in reducing freezing-of-gait, mitigating spatial variability, and bolstering gait consistency in people with Parkinson’s disease (PD). Specifically, sonification through the manipulation of acoustic features paired with the application of advanced audio processing techniques (e.g., time-stretching) enable auditory cueing interventions to be tailored and enhanced. These methods used in conjunction optimize gait characteristics and subsequently improve mobility, enhancing the effectiveness of the intervention. The aim of this narrative review is to further understand and unlock the potential of sonification as a pivotal tool in auditory cueing for gait rehabilitation, while highlighting that continued clinical research is needed to ensure comfort and desirability of use.

1. Introduction

Globally, the proportion of the population > 65 years (older adults) is increasing. In the UK, projections suggest that by 2046, that group will represent 25% of the population [1]. Typically, older adults face challenges due to the potential decline in gait-related mobility. That decline impacts crucial aspects of personal independence and the overall quality of life [2]. Conditions such as Parkinson’s disease (PD), stroke, and arthritis can significantly curtail an individual’s ability to freely ambulate and perform routine daily tasks unaided [3]. Functional challenges are compounded by mobility difficulties resulting in falls, which constitute the leading cause of injury-related deaths for those > 65 years [4]. Moreover, a harmful cycle due to a lack of confidence in walking due to a fear of falling can further exacerbate physical decline, highlighting a necessity for public health initiatives aimed at mitigating this escalating silent epidemic [5]. This pervasive issue underscores the critical need for practical interventions aimed at alleviating gait-related mobility issues [6].
Rehabilitation research has examined cueing, leveraging auditory, visual, and tactile cues to normalize abnormal gait patterns and improve mobility [7]. However, auditory cueing has been found to both be the most effective at improving gait and the most practical to apply in outdoor settings [8,9]. There is also an appreciation regarding cueing paradigms that one size does not fit all, i.e., there is a need to ensure personalised approaches to ensure interventions are tailored to the individual and their functional limitations at a moment in time. Furthermore, the long-term effectiveness of cueing mechanisms, such as metronomes, ultimately suffer from their lack of engagement [10]. This highlights the existing gap in personalised gait-cueing interventions that not only cater to an individual’s specific physical needs but also offer an engaging experience to promote long-term adherence.
Sonification is the methodology of using non-speech audio to communicate information [11]. Within medicine, heart rate and electrocardiograph monitors use a humble representation of sonification to audibly represent a patient’s heartbeat, alerting doctors to any potential irregularities in their cardiac cycle [12,13]. This is especially important in critical care situations, where transforming patient vital signs into audio cues allows caregivers to quickly detect and respond to any changes. Other examples showcase how sonification can be used to provide valuable enhancements in medical diagnostics and monitoring [14,15].
Applying sonification to audibly represent the rhythmic properties of a patient’s vital signs underscores its potential application in gait rehabilitation where the goal is to establish rhythmic, distinct, and harmonious walking patterns. This potential has been recognized where several implementations of sonification in gait rehabilitation have been effective in enhancing mobility, particularly for older adults who face challenges with various mobility disorders [16,17,18]. A recently conducted review emphasizes the technical and design aspects of real-time movement sonification systems relating to gait. This includes the dimensions of movement to sonify, what anatomy to track, and what motion capture methodologies to use [19]. However, while the findings of that review are valuable and insightful, it notably omits a discussion on how acoustic variables influence the valence of such rehabilitation systems. Understanding these acoustic factors is crucial, as they can significantly affect patient engagement and treatment outcomes. Additionally, the review does not delve into the potential for manipulating these acoustic variables to provide more personalised interventions to further enhance the valence and overall usability of these systems. Understanding these gaps could lead to more effective and user-friendly rehabilitation solutions, tailored to the unique needs and preferences of individuals undergoing neurological rehabilitation.
Here, we explore the concept of personalised interventions, focusing on sonification and its potential in gait rehabilitation. This review discusses ways sonification can enhance gait-related mobility, how it can be manipulated, and how it can be used to improve the landscape of gait rehabilitation.

2. Assessing Gait

Measuring the gait of those with physical mobility challenges is an initial step to providing effective and long-lasting rehabilitation strategies. By identifying which aspects of an individual’s gait are abnormal, pragmatic approaches can be devised [20]. Digital technologies are playing an important role within instrumented rehabilitation, objectifying gait to provide high-resolution data. For example, instrumented walkways have integrated pressure sensors that record footfall data as a person walks across the surface, generating a comprehensive representation of an individual’s spatial and temporal gait pattern. Gait characteristics such as step length and step time can be recorded at millimetre and millisecond resolution, respectively [21]. Alternatively, wearable inertial measurement units (IMUs) are finding more routine use due to their greater affordability (compared to instrumented walkways) and ability to be used in a range of environments. IMUs provide equally precise, high-resolution data on a person’s gait with the capacity to provide many more characteristics when fused as part of a sensing array [22]. Accordingly, wearable IMUs are a more contemporary tool for performing accurate gait analysis [23].
Often, IMUs have routine connectivity capabilities through the Internet of Things (IoT), where they can routinely integrate with other systems, offering a streamlined and more rounded experience for clinicians and patients alike [24]. IMUs are also capable of real-time sensing. That is useful by providing users with precise gait data with the immediate delivery of some feedback mechanism(s) during a rehabilitation process. Real-time sensing offers clinicians and patients valuable insights into the efficacy of rehabilitation strategies which could then be immediately adjusted to optimize gait [25]. In short, IMU-based real-time monitoring with immediate feedback describes a process of biofeedback, enriching the therapeutic landscape and enhancing patient outcomes further.

3. Personalising Interventions

3.1. Biofeedback

Biofeedback has gained prominence as an effective method to enhance biomechanical and physiological aspects of movement and, in particular, the gait of those with mobility issues [23]. Biofeedback offers a real-time response to various physiological signs such as body movement and cardiovascular parameters, allowing patients to gain a better understanding of their physical condition, which is beneficial for those who find it difficult to recognize their physical limitations [26].
Moreover, biofeedback plays a significant role in neural plasticity, activating neural circuits vital for motor control and learning [27]. This can be transformative for individuals with mobility issues, enabling them to achieve meaningful improvements in overall function. Ultimately, biofeedback aids immediate rehabilitation which can contribute to long-term independence and improved quality of life for patients [28].
Real-time biofeedback suggests personalised interventions, i.e., tailored to an individual’s physiological condition. This can enable a more nuanced approach to address the physical and cognitive aspects of specific mobility challenges, rather than adopting a one-size-fits-all approach. In contrast, contemporary rehabilitation lies in the personalised approach to interventions, where a prominent example is in the implementation of biofeedback through personalised cueing [29].

3.2. Personalised Cueing

Cueing techniques play a crucial role in the context of reducing falls by assisting individuals to improve their step-by-step gait patterns and therefore their overall walking stability. Generally, cueing involves providing external stimuli or cues to guide and enhance motor control and coordination during gait [30]. In gait rehabilitation, standardised cueing techniques have been used. For auditory stimuli, these include metronome cues or the use of music, where individuals synchronize their steps to a rhythmic beat, or visual cues such as lines or markers on the floor to encourage specific stride lengths or step widths [7]. These standardized cues provide a consistent framework for gait retraining and can be effective in promoting immediate improvements in gait characteristics [8].
To optimise the efficiency of cueing, researchers have examined personalised cueing mechanisms by incorporating biofeedback. That approach includes customizing the cues to harmonise with individuals’ distinct gait and mobility traits, thereby amplifying their efficacy. By adopting a personalised approach, not only can patients’ comfort be enhanced, but also intervention effectiveness. Various studies have underscored the substantial benefits of tailoring the delivery of cueing mechanisms to align with individual gait characteristics, resulting in a marked improvement in the effectiveness of each cueing mechanism [31,32].
To enable personalised cueing, the fundamental requirement is the accurate and near real-time recording of gait, which can be performed with IMUs [33]. By leveraging specific gait characteristics, such as cadence, it becomes possible to modify cueing mechanisms to align with the physiological condition of individuals. For example, gait abnormalities have been reduced by the use of a personalised auditory cue (a 10% increase in the number of beats per minute on an individual’s natural cadence) [34]. To further refine personalised gait rehabilitation, sonification is proposed as a dynamic and versatile methodology.

4. Sound and Sonification Theory

Fundamentally, sonification is a method that uses sound, i.e., non-speech audio, to convey information or data. This involves processing non-auditory data and converting it into an auditory format, thereby creating an alternative data perception channel [35]. The technique can be useful when visual channels are overloaded or less efficient, or not accessible [36,37]. Several acoustic variables are typically used to represent qualitative, quantitative, and categorical aspects of information, which include pitch, timbre, amplitude, tempo, duration, and spatialization [11,38].
A wide range of sonification techniques has been explored to represent data, including sonic landscapes, auditory icons, ambient soundscapes, tonal music, ad hoc pitch mappings, simple pitch mapping, speech and non-speech audio, earcons, and musical motifs [39]. Like personalised cueing, applying sonification to gait rehabilitation involves utilising gait monitoring technology such as IMUs to provide real-time auditory biofeedback. For example, gait characteristics can be transformed into discernible sounds [40]. Nown et al. [19] have comprehensively explored the various ways sonification has been mapped to specific aspects of movement, including position, velocity, orientation, and acceleration, with auditory feedback ranging from pitch, amplitude, and timbre to more complex melodies and spatial cues. However, there is a lack of focus on how acoustic variables can influence the overall effectiveness of interventions. Thus, there is also a lack of focus on how such acoustic variables can be manipulated to further enhance treatment; this requires further investigation.
However, before exploring the various applications of sonification and auditory biofeedback, it is crucial to have a solid understanding of sound theory. This ensures optimal approaches for comprehending and implementing sonification and its core concepts into gait rehabilitation.

4.1. Acoustic Variables

4.1.1. Frequency

Frequency is a fundamental property of sound and refers to the number of cycles of a sound wave that occur in one second, measured in hertz (Hz). The frequency of a sound wave ultimately determines its pitch, where shorter frequency wavelengths are associated with higher-pitched sounds and longer frequency wavelengths are associated with lower-pitched sounds. In sonification, the frequency of auditory stimuli can be used to represent various data attributes or dimensions [41]. For example, different data values/variables can be mapped to specific frequency ranges, allowing users to perceive variations in pitch and identify patterns or trends in the data [42]. By assigning frequencies to data parameters, sonification designers can create intuitive auditory representations to enhance the understanding of complex information [43]. Figure 1 illustrates the difference in waveforms with varying frequencies.

4.1.2. Amplitude

Amplitude represents the strength or intensity of a sound wave and is typically measured in decibels (dB). Amplitude directly affects the loudness or volume, with larger amplitudes producing louder sounds and smaller amplitudes resulting in softer sounds [44]. In sonification, the amplitude can be used to convey the magnitude or intensity of data values, whereby mapping data attributes to specific amplitudes, users can perceive variations in loudness and quickly identify data extremes or significant changes [45]. Figure 2 typifies waveform difference between three amplitudes.

4.1.3. Timbre

Timbre refers to the unique quality or tone of a sound, enabling a distinction between musical instruments or voices producing the same pitch and amplitude [46]. Timbre is influenced by factors such as harmonics, envelope, and spectral content. Harmonics are additional frequencies accompanying the fundamental frequency, giving instruments or voices their characteristic timbre [47]. Envelope represents the dynamic changes in a sound’s amplitude over time, including attack, decay, sustain, and release, and finally, spectral content refers to the distribution of energy across frequencies, contributing to the timbre [48]. For sonification, timbre can represent distinct data categories or attributes, aiding the differentiation of information through auditory characteristics and enhancing the identification of data dimensions [43]. Figure 3 as an example shows the difference in waveforms between a human voice, piano, and tuning fork.
With these acoustic variables and their potential use within sonification being established, it is essential to understand how they can be used to enhance auditory rehabilitation.

4.2. Manipulating Acoustic Variables

4.2.1. Time Stretching

Several methods of acoustic manipulation have been explored. One prominent method is the application of a phase vocoder to perform time stretching, which has opened possibilities in gait training [49]. The phase vocoder provides the ability to modify the tempo of music while preserving its original pitch, paving the way for a new level of personalised auditory experiences. Time stretching through the use of the phase vocoder algorithm employs a multi-step process [50] where, first, the original audio signal is divided into short segments using an analysis window, such as a Hanning Windows, and Fourier transforms are subsequently applied to obtain the short-time Fourier transform (STFT) representation. In the modification step, the amplitudes of the sinusoidal components in the STFT are adjusted according to a modification factor, while the phases are calculated using phase unwrapping and a phase-propagation formula to ensure coherence.
Finally, in the resynthesis stage, the modified STFT is inverse Fourier transformed to obtain short-time signals, which are then combined to create the time-scaled output signal. Although the output signal may not precisely match the original signal’s STFT representation due to the modifications, it retains the local spectral characteristics, which in this case is pitch. Figure 4 below demonstrates the various steps as part of the phase vocoder algorithm for both time stretching and pitch shifting.
One particularly prudent application of a time-stretching algorithm is the application of personalizing auditory stimuli to match an individual’s real-time gait cadence. For example, as an individual walks, the tempo of the music can adjust dynamically, synchronizing with the person’s stride in real time [49]. However, providing the ability to adjust music tempo without changing the pitch of the music could therefore negate any effect changing the tempo of music can have on gait initiation, motivation, and processing of information.

4.2.2. Pitch Shifting

Conversely, as it has been found that a higher pitch can have a positive impact on the effectiveness of auditory stimuli, it may also be beneficial to modify the pitch of music while maintaining the original tempo. The phase vocoder algorithm can also be used for this purpose, where the algorithm applies a similar framework for pitch shifting as with time stretching, but the difference lies in the processing step after the STFT is applied. Rather than altering frame spacing, the algorithm shifts the frequency components of each frame, which is typically achieved by multiplying each frequency bin by a pitch ratio. However, directly shifting frequencies could lead to phase inconsistencies. Therefore, the phase vocoder again must adjust the phase to ensure smooth transitions between frames [51].
Applying pitch shifting to personalised music cues could enable the creation of personalised auditory stimuli tailored to individual needs or preferences. For instance, the pitch of a music piece could be subtly modulated to enhance emotional engagement with the method of rehabilitation, with the goal being to improve its effectiveness.

4.2.3. Dynamic Range Compression (DRC)

DRC is another audio processing technique that modulates the dynamic range, the disparity between the loudest and softest parts, of an audio signal. By subtly diminishing the amplitude contrast, this technique encourages sonic harmony and balance, ensuring that every element of the sound can be heard clearly and resonantly [52]. The DRC algorithm is carried out in several steps, where first it calculates the amplitude of the incoming audio signal. Then, based on a set threshold and ratio, it determines how much the signal’s amplitude exceeds the threshold and applies gain reduction proportional to this excess. The ratio determines the degree of compression, i.e., a higher ratio results in stronger compression. For instance, with a ratio of 4:1, for every 4 dB the input level increases above the threshold, the output level will only increase by 1 dB.
The threshold level, ratio, attack time, which is how quickly the compressor starts to work, and release time, which is how quickly the compressor stops working after the signal drops below the threshold, are adjustable parameters in the DRC algorithm [53]. This technique could be beneficial for use within auditory biofeedback for several key reasons. First, if the rhythmic aspect of the biofeedback is less distinct, with the use of DRC, percussive components like drumbeats become more pronounced and discernible.
This improvement could enable patients to synchronize their movements more effectively with the rhythm, facilitating a more immersive and engaging therapeutic experience. Furthermore, its ability to minimize sudden and drastic changes in volume could limit the level of disorientation or distress for individuals undergoing therapy. By applying DRC, such jarring shifts in volume could be reduced, resulting in a more predictable and comfortable listening experience for patients. Figure 5 below illustrates the DRC algorithm being used to increase/decrease the level of gain using a threshold.

4.3. Influence of Acoustic Variables on Auditory Rehabilitation

4.3.1. Pitch

To investigate the influence of pitch, a study evaluated the effects of pitch on participants’ emotional response to audio during movement-based tasks [54]. The study uncovered that higher pitch frequencies not only elevated participants’ sense of capability but also boosted their motivation and comfort during physical movement. In contrast, lower pitch frequencies generally produced the opposite effects. The study found also that higher pitches positively influenced valence—the emotional response to pleasantness in external stimuli. This is a significant discovery, given that valence plays a crucial role in initiating gait [55]. Additional research has also discovered that pitch variations also play a role in affecting the perceptual dimension of human cognition, where it critically influences factors such as motivation and the processing of information [51].

4.3.2. Amplitude

To assess the role that levels of amplitude have on auditory gait rehabilitation, a review was conducted on a total of 13 studies that were found through an extensive search [56]. These studies revealed that high-intensity auditory cues significantly enhance gait performance in PD patients, improving metrics like speed, stability, and muscle activation. One study in particular involved 13 PD participants where they were instructed to perform a quick walk initiated with their right leg, taking three consecutive steps in response to three distinct types of stimuli: a visual cue (a white square shown on a black computer screen), a combination of visual and low-intensity auditory cues, and a combination of visual and high-intensity auditory cues [57]. Each auditory cue consisted of 750-Hz tone bursts lasting 30 ms with an amplitude of 80 dB for low-intensity stimuli and 130 dB for high-intensity stimuli. The results of this study revealed that in all PD participants, tibialis anterior and rectus femoris muscle activation was faster with high-intensity auditory stimuli than with low-intensity auditory stimuli. These results suggest that for an auditory stimulus to be effective, a higher level of amplitude may be necessary.

4.3.3. Timbre

Focusing specifically on the effect of timbre, a study involving 13 young adults with diplegic cerebral palsy examined the effect of simple vs. complex chords as auditory cues on gait [58]. Participants were divided into two groups based on the complexity of chords and melodies, where both groups showed significant improvements in walking speed, stride length, and cadence. It was also found that complex chords led to better ankle movement, suggesting that timbral qualities could enhance the effectiveness of auditory cueing in improving mobility for neurological conditions. It is hypothesized in this paper that this is also due to the increased valence in more complex auditory stimuli, demonstrating how timbre can also impact valence and effectiveness.
The evidence indicates that acoustic variables play a crucial role in the success of auditory rehabilitation. By carefully choosing and manipulating auditory cues, it is possible to substantially improve the emotional experience associated with movement. This, in turn, could pave the way for more effective strategies in sonification-based gait rehabilitation, with the sonification of gait characteristics being a key example.

5. Sonification as a Gait Rehabilitation Tool

This section explores the various applications of sonification in gait rehabilitation, covering the use of sonified cues generated from various aspects of gait including characteristics, cycles, and joint kinematics. Figure 6 provides a visual summary, covering both the previously discussed topics and the applications of sonification in gait rehabilitation that we will explore in this section.

5.1. Sonification of Gait Characteristics

5.1.1. Cadence Sonification

The application of cadence as a parameter for biofeedback extends beyond personalised cueing, as it has also been explored within the realm of sonification [46]. To explore the potential of utilizing cadence with sonification as a gait retraining approach, a comprehensive study was conducted involving the design and evaluation of a music-sonification system [17]. The first part of the study explores the selection of these feedback signals and the establishment of a perception curve, where the feedback signals tested include white noise, pink noise, amplitude-modulated noise, downsampling, and volume decrease. The tests were conducted on a group of 10 participants who were asked to indicate the perceived intensity level of the feedback signal and asked to fill in a questionnaire to evaluate the clarity, valence, and annoyance of the feedback signals. A statistical evaluation of perception tests involved fitting different curves (linear, second order, exponential) to the data. The accuracy of mapping from objectively measured intensity to subjective perception determined auditory input performance. It was found that pink noise with an exponential fit emerged as the optimal choice, delivering the most perceivable signal and effective mapping, while also having high scores for clarity and pleasantness, solidifying its selection for subsequent experiments.
For the second part of the study, the chosen pink noise was tested under running conditions to determine the minimum perceived sonification level and the just-noticeable difference between different intensity levels. The participants were informed to run while music with superimposed randomized pink noise intensity levels played; the experiment resulted in an interval size of 20% being chosen for further experiments due to the high probability of the detection of a difference in pink noise intensity level.
The study’s final part aimed to confirm the effectiveness of the sonification system in modifying the running cadence of trained athletes. Four conditions were examined: baseline, verbal instruction, fixed target noise feedback, and changing target noise feedback. The baseline measured the average runner cadence post-warm-up, while in the verbal instruction, runners were told to increase their cadence by 15% from the baseline. For the pink noise feedback, runners were asked to adjust their cadence to improve the sound quality of the music they were listening to, which was distorted according to their cadence. Real-time data from two tri-axial accelerometers, one being placed on each leg, were processed to dynamically adjust the music being played to match the runner’s cadence using a phase vocoder time-stretching algorithm. Before the task, runners chose songs from music genres like pop, rock, and dance, with tempos between 140 and 190 BPM, and during the running, if the cadence mismatched the BPM by over 4% for 8 s, a different song played. In the fixed target noise feedback, cadence was tracked and mapped from 95% to 115% of the initial cadence, converted into an exponential curve, and segmented into 20% intervals. The resulting value determined the pink noise level added to the music, with 0% being no noise and 100% being as loud as the music. The changing target phase used similar noise mapping but with fluctuating cadence ranges: 95–115%, 120–85%, and 80–100%. The results, illustrated in Figure 7, found that the pink noise feedback strategy was found to perform significantly better than verbal instructions, suggesting potential for its use in self-training and gait retraining programs, without requiring external assistance. Furthermore, the pink noise feedback was effective for both increasing and decreasing cadence, where feedback from questionnaires also indicated that the combination of minimized noise and synchronized music had a rewarding effect.

5.1.2. Velocity Sonification

Velocity is another gait characteristic that has been explored for gait rehabilitation purposes specifically concerning ameliorating freezing-of-gait (FoG) in PD patients [59]. An experimental protocol, Action Observation plus Sonification (AOS), was devised and applied in a study involving 37 PD patients with bouts of FoG, who were split into two distinct groups: experimental and control. The core experiment involved exposure to eight videos, each demonstrating an actor executing eight unique motor gestures. Participants across both groups were required to mimic these motor gestures synchronously while viewing the videos. However, the marked differentiation between the two groups lay in the sensory aids incorporated into their respective videos. For the experimental cohort, the visual stimuli were supplemented with enhanced auditory signals, which involved the sonification of the walking velocity of the actors. These signals were recorded using a seven-camera motion-capture Qualisys system (120 Hz) and were transformed into audio pitch variations [60]. In contrast, the control group was provided with the same motor gestures, but their videos were complemented by strategic visual (stripes on the floor) and auditory (metronome) cues. For the experimental cohort, a specialized sonification process was employed to generate the enhanced auditory signals from the walking velocity data.
This process first involved the application of a median filter that incorporates a window size of three frames, which suppresses any sensor noise that may potentially originate from the kinematic data. Following this noise suppression, the data stream is adjusted, undergoing a linear scaling process that normalizes it within an interval from 0 to 1. Following this, the generation of the pitch sound itself is executed and subsequently and carefully mapped to the audio channels, distributing it to either the left or the right channel.
The results, illustrated in Figure 8, from the experiment suggest the AOS treatment significantly outperformed the standard cueing protocol, where reduced severity and duration of FoG were observed, not just post-treatment, but also during subsequent follow-ups. AOS also enhanced motor function, reduced daily activity discomfort, and maintained these improvements over time. Additionally, it substantially ameliorated daily living issues and somewhat improved motor balance, with enduring benefits observed three months post-treatment. These results overall showcase the transformative power of the sonification of gait characteristics, underscoring its superiority over traditional metronome cueing in terms of sustained clinical effectiveness. However, while the sonification of gait characteristics has demonstrated effectiveness, the sonification of gait cycles has also been explored as a clinical method of regulating gait.

5.2. Sonification of Gait Cycles

A gait cycle, comprising roughly 60% stance phase and 40% swing phase, represents the movements in a single step. The stance phase involves weight-bearing, starting with heel contact (HC) and progressing through loading, midstance, terminal stance, and pre-swing stages. The swing phase, on the other hand, includes the foot’s off-ground movement in stages of the initial, mid-, and terminal swing [61].
The application of sonification for the swing phase of the gait cycle has been investigated to evaluate the efficacy of the approach for delivering effective gait rehabilitation to individuals with PD [18]. Each of the nine PD participants was guided to undertake two full walks, both up and down, along a designated 12-m pathway, where a total of five reflective markers (200 Hz) were attached to the shoes of each participant. Positional (X and Y axes) data were streamed from the markers during each walk, with the sonification of each swing phase being calculated and subsequently played in real time. This sonification process first involves each X and Y displacement alteration consolidated into a singular vector for each heel marker, where the absolute variance in displacement between two distinct samples is calculated to provide an approximate measure of velocity. The commencement of the swing phase of the step is defined when this calculated velocity exceeds a predetermined threshold of 0.5 ms−1, which then subsequently triggers the onset of the sonification synthesis process. This process involves a four-part sine tone being played, starting with a fundamental frequency of 261.63 Hz.
As the distance from the initial stride increased in 0.1 m increments, each subsequent tone was played, with the pitch rising by 1 MIDI number per interval, which creates an auditory sliding effect for each stride. The sonification of each swing phase for each respective foot was subsequently played in the corresponding ear of the individual. The results, illustrated in Figure 9, from the study showcase that when listening to the sonified swing phases while walking, the coefficient of variation (CoV) for step length decreased to 7.08% from 9.65%, while the CoV for step duration decreased slightly to 5.32% from 5.66%. The decrease in CoV for step length was statistically significant (p = 0.019), indicating a meaningful improvement compared to baseline measurements. However, the change in CoV for step duration was not significant (p = 0.675). These findings suggest that the real-time sonification of gait cycles, i.e., the swing phase, can significantly reduce spatial variability in the gait of PD patients, potentially improving gait consistency.
The promising outcomes of sonifying gait characteristics and cycles highlight the potential of sonification within gait rehabilitation. Building upon these findings, the next section delves into another aspect of sonification in the field of biomechanics, focusing on the sonification of joint kinematics during ambulation.

5.3. Sonification of Joint Kinematics

Joint kinematics refers to the study of joint movements and the measurement of angles at specific joints during different activities, such as walking. One specific area where joint kinematics and rehabilitation intersect is the rehabilitation of knee flexion, where the use of sonification has been explored for this purpose [62]. The development of a novel prototype system, namely Sofigait, was conducted to both address the necessity of quantifying knee flexion, as well as to provide sonification to evaluate the perception of sonification feedback in participants with artificially created gait asymmetry. This system, equipped with IMUs and a sonification module, was validated against a Vicon motion capture system in a controlled experiment involving a cohort of 24 healthy individuals.
For the first phase of testing, participants walked on a treadmill at a speed of 4 km/h while their gait was recorded by Sofigait and Vicon. Ten gait cycles captured by both systems were compared for measurement system analysis. For the second phase of testing, participants wore a knee brace on their right leg to create an artificial gait asymmetry and walked five times consecutively on the treadmill at a reduced speed of 3.5 km/h to compensate for the gait restriction caused by the knee brace. They received sonification through wireless headphones from the Sofigait system, which was randomized into four different versions for trials 2–5, where trial 1 served as an acclimatization trial. Throughout each of the trials, the sagittal knee angle is systematically measured and sonified with the position of an extended leg serving as the reference point, denoted as 0°. Subsequently, as flexion increases, the knee angle progressively rises, eventually reaching 180°, which signifies the angle formed between the thigh and lower leg segments.
Four distinct versions of sonification, Soni 1.1, 1.2, 2.1, and 2.2, were implemented as part of the sonification module. Soni 1.1 utilizes a sine continuous tone during the swing phase, ranging from 35°. The knee angle is logarithmically mapped from −45° to +90°, resulting in frequencies spanning from 220 Hz to 1760 Hz. Notably, at 0°, the knee angle corresponds to the concert pitch of 440 Hz, effectively modulating a sine oscillator. Additionally, the knee angle is mapped to the volume, gradually fading out linearly below 35° until it reaches 0°.
Soni 1.2 employs the same principles as Soni 1.1 but with a lower pitch, half an octave below. For Soni 2.1, a continuous sine tone is utilized during the stance phase, ranging up to 35°. Like the previous versions, the knee angle is mapped to the volume, but in this case, the continuous tone gradually decreases linearly from 0° to 35°. As a result, an alternating left/right sound is generated, with emphasis on the smaller angles. Lastly, Soni 2.2 shares the same principles as Soni 2.1 but with a half-octave lower pitch. Table 1 below summarizes the principles of each Soni version.
The results of the study showed that changes in pitch significantly affected the perceived pleasantness of the sonification feedback, with participants indicating that a lower pitch was more pleasant. However, the accentuation of the feedback did not appear to influence participants’ perception of their gait asymmetry. Based on the findings, the swing-accentuated and lower-pitched version of the sonification (Soni 1.2) received the highest ratings on both the asymmetry perception and pleasantness and is, therefore, the recommended approach from the four sonification techniques for patients with asymmetric gait from restricted knee movement.
While these results are limited regarding their effectiveness in reducing asymmetry, they further indicate the potential impact that acoustic variables such as pitch can have on the pleasantness of sonification as a clinical intervention. This finding also emphasizes the requirement for advanced audio processing techniques to effectively address any potential negative impact that acoustic variables, such as pitch, timbre, and amplitude, may have on sonification and personalised cueing in general. Table 2 provides a summary of methods and results regarding gait sonification.

5.4. The Unmet Clinical Need

As evidenced in a review by Nown et al. [19] and in this paper, sonification has rehabilitation potential in movement and gait. However, there is a clear lack of focus within the literature on the sonification of clinically relevant gait characteristics that comprise five gait domains: pace, rhythm, variability, asymmetry, and postural control. This is important as these domains are associated with specific cognitive domains: attention, working memory, visual memory, executive function, visuospatial function, and global cognition [63,64]. Those identified gait domains are not uniform but rather involve different neural mechanisms, where, for example, attention has been linked with gait velocity and with grey matter volumes in certain brain regions [65]. Such gait domains become even more distinct in the context of pathological gait as they specifically each have connections to neurodegenerative diseases (such as PD) and general cognitive decline [66,67].
Accordingly, there is an unmet need for harnessing the use of sonification within clinically defined gait. There is a need for targeted and personalised sonification within specific gait domains and associated characteristics. Subsequently, future research could better enable clinicians to focus on specific aspects of gait to target underlying neural mechanisms and this should be the approach taken going forward.

6. Conclusions

Personalised cueing with sonification may have added value in gait rehabilitation. Sonification has demonstrated a myriad of applications in gait rehabilitation, to improve spatial and temporal gait characteristics, gait cycles, and joint kinematics. Research has demonstrated extremely promising results, with sonification-based interventions demonstrating significant improvements in reducing freezing-of-gait, spatial variability, and motor balance, and enhancing gait consistency in individuals with mobility disorders such as PD.
Moreover, our comprehensive exploration of sound theory and manipulation of acoustic features, including aspects such as pitch, timbre, and amplitude, has highlighted the significant potential to augment the effectiveness of auditory-based gait interventions. Through detailed analysis, we have discussed and shown how these techniques can profoundly impact factors such as valence, motivation, and information processing. This understanding not only underpins the physical effectiveness of these interventions on improving gait but may also demonstrates their practical application, which is imperative for long-term engagement. The insights gained from this discussion reinforce the importance of these auditory elements, showing how they enhance the overall efficacy of the interventions, thereby offering promising avenues for future research and application in neurological rehabilitation.
Overall, the application of sonification and the manipulation of acoustic biofeedback in gait rehabilitation has the potential to improve personalised interventions and improve the effectiveness of therapy to reduce falls. By providing real-time auditory feedback and tailored auditory experiences mapped to gait domains and clinically relevant characteristics, sonification-based interventions could enhance motor learning and promote neural plasticity. Such an approach could ultimately contribute to improving mobility and reducing falls, restoring independence and improving the quality of life.

Author Contributions

Conceptualization, C.W. and A.G.; methodology, C.W.; writing—original draft preparation, C.W. and A.G.; writing—review and editing, C.W., P.M., R.W., V.H., L.G. and A.G.; visualization, C.W. and A.G.; supervision, A.G., P.M. and R.W. All authors have read and agreed to the published version of the manuscript.

Funding

This research is co-funded by a grant from the National Institute of Health Research (NIHR) Applied Research Collaboration (ARC) North East and North Cumbria (NENC). This research is also co-funded by the Faculty of Environment and Engineering at Northumbria University.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

No new data were created or analysed in this study. Data sharing is not applicable to this article.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Graham, A.; Budd, L.; Ison, S.; Timmis, A. Airports and Ageing Passengers: A Study of the UK. Res. Transp. Bus. Manag. 2019, 30, 100380. [Google Scholar] [CrossRef]
  2. Frank, J. Balance and Mobility Challenges in Older adultsImplications for Preserving Community Mobility. Am. J. Prev. Med. 2003, 25, 157–163. [Google Scholar] [CrossRef] [PubMed]
  3. Horak, F.B.; Mancini, M. Objective Biomarkers of Balance and Gait for Parkinson’s Disease Using Body-Worn Sensors: Balance and Gait Biomarkers. Mov. Disord. 2013, 28, 1544–1551. [Google Scholar] [CrossRef] [PubMed]
  4. Baker, S.P.; Harvey, A.H. Fall Injuries in the Elderly. Clin. Geriatr. Med. 1985, 1, 501–512. [Google Scholar] [CrossRef] [PubMed]
  5. Trombini-Souza, F.; Nogueira, R.T.d.S.A.; Serafim, A.C.B.; Lima, T.M.M.d.; Xavier, M.K.A.; Perracini, M.R.; de Araújo, R.C.; Sacco, I.C.; Nascimento, M.d.M. Concern About Falling, Confidence in Balance, Quality of Life, and Depression Symptoms in Community-Dwelling Older Adults After a 24-Week Dual-Task Training With Variable and Fixed Priority: A Randomized Controlled Trial. Res. Aging 2022, 44, 658–668. [Google Scholar] [CrossRef]
  6. Rantanen, T. Promoting Mobility in Older People. J. Prev. Med. Public Health 2013, 46, S50–S54. [Google Scholar] [CrossRef]
  7. Palacios-Navarro, G.; Albiol-Pérez, S.; García-Magariño García, I. Effects of Sensory Cueing in Virtual Motor Rehabilitation. A Review. J. Biomed. Inform. 2016, 60, 49–57. [Google Scholar] [CrossRef]
  8. Spaulding, S.J.; Barber, B.; Colby, M.; Cormack, B.; Mick, T.; Jenkins, M.E. Cueing and Gait Improvement Among People With Parkinson’s Disease: A Meta-Analysis. Arch. Phys. Med. Rehabil. 2013, 94, 562–570. [Google Scholar] [CrossRef]
  9. Muthukrishnan, N.; Abbas, J.J.; Shill, H.A.; Krishnamurthi, N. Cueing Paradigms to Improve Gait and Posture in Parkinson’s Disease: A Narrative Review. Sensors 2019, 19, 5468. [Google Scholar] [CrossRef]
  10. Fiorente, N.; Calabrò, R.S. Music in Parkinson’s Disease Rehabilitation: Are We Heading in the Right Direction? Innov. Clin. Neurosci. 2023, 20, 11–13. [Google Scholar]
  11. Hermann, T.; Hunt, A.; Neuhoff, J.G. (Eds.) The Sonification Handbook; Logos Verlag: Berlin, Germany, 2011; ISBN 978-3-8325-2819-5. [Google Scholar]
  12. Ballora, M.; Pennycook, B.; Ivanov, P.C.; Glass, L.; Goldberger, A.L. Heart Rate Sonification: A New Approach to Medical Diagnosis. Leonardo 2004, 37, 41–46. [Google Scholar] [CrossRef]
  13. Blanco, A.L.A.; Grautoff, S.; Hermann, T. CardioSounds: A Portable System to Sonify ECG Rhythm Disturbances in Real-Time. In Proceedings of the 24th International Conference on Auditory Display—ICAD 2018, Houghton, MI, USA, 10–15 June 2018; The International Community for Auditory Display: Houghton, MI, USA, 2018; pp. 20–27. [Google Scholar]
  14. Sanderson, P.; Shek, V.; Watson, M. The Effect of Music on Monitoring a Simulated Anaesthetised Patient with Sonfication. In Proceedings of the 2004 Annual Conference of the Computer-Human Interaction Special Interest Group of the Human Factors and Ergonomics Society of Australia (OzCHI 2004), Wollongong, Australia, 22–24 November 2004. [Google Scholar]
  15. Özcan, E.; Chou, C.; Bogers, K.; van der Helm, A. Caretunes: 18th Sound and Music Computing Conference, SMC 2021. In Proceedings of the SMC 2021—18th Sound and Music Computing Conference, Virtual, 29 June–1 July 2021; pp. 276–283. [Google Scholar] [CrossRef]
  16. Särkämö, T.; Altenmüller, E.; Rodríguez-Fornells, A.; Peretz, I. Editorial: Music, Brain, and Rehabilitation: Emerging Therapeutic Applications and Potential Neural Mechanisms. Front. Hum. Neurosci. 2016, 10, 103. [Google Scholar] [CrossRef] [PubMed]
  17. Lorenzoni, V.; Maes, P.-J.; Van Den Berghe, P.; De Clercq, D.; De Bie, T.; Leman, M. A Biofeedback Music-Sonification System for Gait Retraining. In Proceedings of the 5th International Conference on Movement and Computing, Genoa, Italy, 28–30 June 2018; pp. 1–5. [Google Scholar]
  18. Rodger, M.W.M.; Young, W.R.; Craig, C.M. Synthesis of Walking Sounds for Alleviating Gait Disturbances in Parkinson’s Disease. IEEE Trans. Neural Syst. Rehabil. Eng. 2014, 22, 543–548. [Google Scholar] [CrossRef] [PubMed]
  19. Nown, T.H.; Upadhyay, P.; Kerr, A.; Andonovic, I.; Tachtatzis, C.; Grealy, M.A. A Mapping Review of Real-Time Movement Sonification Systems for Movement Rehabilitation. IEEE Rev. Biomed. Eng. 2023, 16, 672–686. [Google Scholar] [CrossRef] [PubMed]
  20. Van der Marck, M.A.; Klok, M.P.C.; Okun, M.S.; Giladi, N.; Munneke, M.; Bloem, B.R. Consensus-Based Clinical Practice Recommendations for the Examination and Management of Falls in Patients with Parkinson’s Disease. Park. Relat. Disord. 2014, 20, 360–369. [Google Scholar] [CrossRef]
  21. Van Uden, C.J.; Besser, M.P. Test-Retest Reliability of Temporal and Spatial Gait Characteristics Measured with an Instrumented Walkway System (GAITRite®). BMC Musculoskelet. Disord. 2004, 5, 13. [Google Scholar] [CrossRef]
  22. Celik, Y.; Stuart, S.; Woo, W.L.; Godfrey, A. Gait Analysis in Neurological Populations: Progression in the Use of Wearables. Med. Eng. Phys. 2021, 87, 9–29. [Google Scholar] [CrossRef]
  23. Porciuncula, F.; Roto, A.V.; Kumar, D.; Davis, I.; Roy, S.; Walsh, C.J.; Awad, L.N. Wearable Movement Sensors for Rehabilitation: A Focused Review of Technological and Clinical Advances. PM&r 2018, 10, S220–S232. [Google Scholar] [CrossRef]
  24. Young, F.; Mason, R.; Morris, R.E.; Stuart, S.; Godfrey, A. IoT-Enabled Gait Assessment: The Next Step for Habitual Monitoring. Sensors 2023, 23, 4100. [Google Scholar] [CrossRef]
  25. Prasanth, H.; Caban, M.; Keller, U.; Courtine, G.; Ijspeert, A.; Vallery, H.; von Zitzewitz, J. Wearable Sensor-Based Real-Time Gait Detection: A Systematic Review. Sensors 2021, 21, 2727. [Google Scholar] [CrossRef]
  26. Bowman, T.; Gervasoni, E.; Arienti, C.; Lazzarini, S.; Negrini, S.; Crea, S.; Cattaneo, D.; Carrozza, M. Wearable Devices for Biofeedback Rehabilitation: A Systematic Review and Meta-Analysis to Design Application Rules and Estimate the Effectiveness on Balance and Gait Outcomes in Neurological Diseases. Sensors 2021, 21, 3444. [Google Scholar] [CrossRef] [PubMed]
  27. Maier, M.; Ballester, B.R.; Verschure, P.F.M.J. Principles of Neurorehabilitation After Stroke Based on Motor Learning and Brain Plasticity Mechanisms. Front. Syst. Neurosci. 2019, 13, 74. [Google Scholar] [CrossRef] [PubMed]
  28. Belkacem, A.N.; Jamil, N.; Palmer, J.A.; Ouhbi, S.; Chen, C. Brain Computer Interfaces for Improving the Quality of Life of Older Adults and Elderly Patients. Front. Neurosci. 2020, 14, 692. [Google Scholar] [CrossRef] [PubMed]
  29. Giggins, O.M.; Persson, U.; Caulfield, B. Biofeedback in Rehabilitation. J. NeuroEng. Rehabil. 2013, 10, 60. [Google Scholar] [CrossRef]
  30. Lim, I.; van Wegen, E.; de Goede, C.; Deutekom, M.; Nieuwboer, A.; Willems, A.; Jones, D.; Rochester, L.; Kwakkel, G. Effects of External Rhythmical Cueing on Gait in Patients with Parkinson’s Disease: A Systematic Review. Clin. Rehabil. 2005, 19, 695–713. [Google Scholar] [CrossRef]
  31. Bella, S.D.; Dotov, D.; Bardy, B.; de Cock, V.C. Individualization of Music-Based Rhythmic Auditory Cueing in Parkinson’s Disease: Rhythmic Auditory Cueing in Parkinson’s Disease. Ann. N. Y. Acad. Sci. 2018, 1423, 308–317. [Google Scholar] [CrossRef]
  32. Ivkovic, V.; Fisher, S.; Paloski, W.H. Smartphone-Based Tactile Cueing Improves Motor Performance in Parkinson’s Disease. Park. Relat. Disord. 2016, 22, 42–47. [Google Scholar] [CrossRef]
  33. Barrios, J.A.; Crossley, K.M.; Davis, I.S. Gait Retraining to Reduce the Knee Adduction Moment through Real-Time Visual Feedback of Dynamic Knee Alignment. J. Biomech. 2010, 43, 2208–2213. [Google Scholar] [CrossRef]
  34. Hausdorff, J.M.; Lowenthal, J.; Herman, T.; Gruendlinger, L.; Peretz, C.; Giladi, N. Rhythmic Auditory Stimulation Modulates Gait Variability in Parkinson’s Disease: Effects of RAS on Gait Variability in PD. Eur. J. Neurosci. 2007, 26, 2369–2375. [Google Scholar] [CrossRef]
  35. Nees, M.; Walker, B. Theory of Sonification. In The Sonification Handbook; Logos Publishing House: Berlin, Germany, 2012; pp. 9–39. ISBN 978-3-8325-2819-5. [Google Scholar]
  36. Sawe, N.; Chafe, C.; Treviño, J. Using Data Sonification to Overcome Science Literacy, Numeracy, and Visualization Barriers in Science Communication. Front. Commun. 2020, 5, 46. [Google Scholar] [CrossRef]
  37. Vickers, P.; Alty, J.L. Using Music to Communicate Computing Information. Interact. Comput. 2002, 14, 435–456. [Google Scholar] [CrossRef]
  38. Franklin, K.M.; Roberts, J.C. Pie Chart Sonification. In Proceedings of the Seventh International Conference on Information Visualization, London, UK, 18 July 2003; pp. 4–9. [Google Scholar]
  39. Vickers, P. Sonification for Process Monitoring. In The Sonification Handbook; Logos Verlag: Berlin, Germany, 2011. [Google Scholar]
  40. Torres, A.V.; Kluckner, V.; Franinovic, K. Development of a Sonification Method to Enhance Gait Rehabilitation. In Proceedings of the ISon 2013, Erlangen, Germany, 10 December 2013; pp. 37–43. [Google Scholar]
  41. Gray PhD, L. Properties of Sound. J. Perinatol. 2000, 20, S6–S11. [Google Scholar] [CrossRef] [PubMed]
  42. Vickers, P. Sonification and Music, Music and Sonification. In The Routledge Companion to Sounding Art; Routledge: London, UK, 2016. [Google Scholar]
  43. Dubus, G.; Bresin, R. A Systematic Review of Mapping Strategies for the Sonification of Physical Quantities. PLoS ONE 2013, 8, e82491. [Google Scholar] [CrossRef] [PubMed]
  44. Zollinger, S.A.; Podos, J.; Nemeth, E.; Goller, F.; Brumm, H. On the Relationship between, and Measurement of, Amplitude and Frequency in Birdsong. Anim. Behav. 2012, 84, e1–e9. [Google Scholar] [CrossRef]
  45. Shojaeiarani, J.; Bajwa, D.; Holt, G. Sonication Amplitude and Processing Time Influence the Cellulose Nanocrystals Morphology and Dispersion. Nanocomposites 2020, 6, 41–46. [Google Scholar] [CrossRef]
  46. Siedenburg, K.; Saitis, C.; McAdams, S. The Present, Past, and Future of Timbre Research. In Timbre: Acoustics, Perception, and Cognition; Siedenburg, K., Saitis, C., McAdams, S., Popper, A.N., Fay, R.R., Eds.; Springer Handbook of Auditory Research; Springer International Publishing: Cham, Switzerland, 2019; Volume 69, pp. 1–19. ISBN 978-3-030-14831-7. [Google Scholar]
  47. Vickers, P.; Hogg, B. Sonification Abstraite/Sonification Concr\`ete: An “Aesthetic Perspective Space” for Classifying Auditory Displays in the Ars Musica Domain. arXiv 2013, arXiv:1311.5426. [Google Scholar]
  48. Wessel, D. Timbre and the Perceptual Organization of Musical Patterns. J. Acoust. Soc. Am. 1989, 86, S58–S59. [Google Scholar] [CrossRef]
  49. Cochen De Cock, V.; Dotov, D.; Damm, L.; Lacombe, S.; Ihalainen, P.; Picot, M.C.; Galtier, F.; Lebrun, C.; Giordano, A.; Driss, V.; et al. BeatWalk: Personalized Music-Based Gait Rehabilitation in Parkinson’s Disease. Front. Psychol. 2021, 12, 655121. [Google Scholar] [CrossRef]
  50. Laroche, J.; Dolson, M. Improved Phase Vocoder Time-Scale Modification of Audio. IEEE Trans. Speech Audio Process. 1999, 7, 323–332. [Google Scholar] [CrossRef]
  51. Sklar, A.G. A Wavelet-Based Pitch-Shifting Method. Unpublished. 2006. Available online: https://www.semanticscholar.org (accessed on 1 June 2023).
  52. Hjortkjær, J.; Walther-Hansen, M. Perceptual Effects of Dynamic Range Compression in Popular Music Recordings. J. Audio Eng. Soc. 2014, 62, 37–41. [Google Scholar] [CrossRef]
  53. Schmidt, J.C.; Rutledge, J.C. Multichannel Dynamic Range Compression for Music Signals. In Proceedings of the 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings, Atlanta, GA, USA, 9 May 1996; Volume 2, pp. 1013–1016. [Google Scholar]
  54. Ley-Flores, J.; Alshami, E.; Singh, A.; Bevilacqua, F.; Bianchi-Berthouze, N.; Deroy, O.; Tajadura-Jiménez, A. Effects of Pitch and Musical Sounds on Body-Representations When Moving with Sound. Sci. Rep. 2022, 12, 2676. [Google Scholar] [CrossRef] [PubMed]
  55. Avanzino, L.; Lagravinese, G.; Abbruzzese, G.; Pelosin, E. Relationships between Gait and Emotion in Parkinson’s Disease: A Narrative Review. Gait Posture 2018, 65, 57–64. [Google Scholar] [CrossRef] [PubMed]
  56. Gómez-González, J.; Martín-Casas, P.; Cano-de-la-Cuerda, R. Effects of Auditory Cues on Gait Initiation and Turning in Patients with Parkinson’s Disease. Neurología 2019, 34, 396–407. [Google Scholar] [CrossRef] [PubMed]
  57. Fernández-Del-Olmo, M.; Bello, O.; Lopez-Alonso, V.; Andrés Sanchez, J.; Santos-García, D.; Valls-Solé, J. The Effects of Auditory Startle and Nonstartle Stimuli on Step Initiation in Parkinson’s Disease. Mov. Disord. 2012, 27, 1570–1573. [Google Scholar] [CrossRef] [PubMed]
  58. Kim, S.J.; Yoo, G.E.; Shin, Y.; Cho, S. Gait Training for Adults with Cerebral Palsy Following Harmonic Modification in Rhythmic Auditory Stimulation. Ann. N. Y. Acad. Sci. 2020, 1473, 11–19. [Google Scholar] [CrossRef] [PubMed]
  59. Mezzarobba, S.; Grassi, M.; Pellegrini, L.; Catalan, M.; Kruger, B.; Furlanis, G.; Manganotti, P.; Bernardis, P. Action Observation Plus Sonification. A Novel Therapeutic Protocol for Parkinson’s Patient with Freezing of Gait. Front. Neurol. 2018, 8, 723. [Google Scholar] [CrossRef]
  60. Motion Capture—Mocap. Available online: https://www.qualisys.com/ (accessed on 2 June 2023).
  61. Kharb, A.; Saini, V.; Jain, Y.K.; Dhiman, S. A Review of Gait Cycle and Its Parameters. IJCEM Int. J. Comput. Eng. Manag. 2011, 13, 78–83. [Google Scholar]
  62. Linnhoff, D.; Ploigt, R.; Mattes, K. Sofigait—A Wireless Inertial Sensor-Based Gait Sonification System. Sensors 2022, 22, 8782. [Google Scholar] [CrossRef]
  63. Morris, R.; Hickey, A.; Del Din, S.; Godfrey, A.; Lord, S.; Rochester, L. A Model of Free-Living Gait: A Factor Analysis in Parkinson’s Disease. Gait Posture 2017, 52, 68–71. [Google Scholar] [CrossRef]
  64. Lord, S.; Galna, B.; Coleman, S.; Yarnall, A.; Burn, D.; Rochester, L. Cognition and Gait Show a Selective Pattern of Association Dominated by Phenotype in Incident Parkinson’s Disease. Front. Aging Neurosci. 2014, 6, 249. [Google Scholar] [CrossRef]
  65. Lord, S.; Galna, B.; Verghese, J.; Coleman, S.; Burn, D.; Rochester, L. Independent Domains of Gait in Older Adults and Associated Motor and Nonmotor Attributes: Validation of a Factor Analysis Approach. J. Gerontol. Ser. A 2013, 68, 820–827. [Google Scholar] [CrossRef] [PubMed]
  66. Verghese, J.; Wang, C.; Lipton, R.B.; Holtzer, R.; Xue, X. Quantitative Gait Dysfunction and Risk of Cognitive Decline and Dementia. J. Neurol. Neurosurg. Psychiatry 2007, 78, 929–935. [Google Scholar] [CrossRef] [PubMed]
  67. Morris, R.; Lord, S.; Lawson, R.A.; Coleman, S.; Galna, B.; Duncan, G.W.; Khoo, T.K.; Yarnall, A.J.; Burn, D.J.; Rochester, L. Gait Rather Than Cognition Predicts Decline in Specific Cognitive Domains in Early Parkinson’s Disease. J. Gerontol. Ser. A 2017, 72, 1656–1662. [Google Scholar] [CrossRef] [PubMed]
Sensors 24 00065 g001
Figure 1. This graph displays the waveforms of musical notes A1 and A5 sampled at a rate of 44.1 kHz. The waveform of the A1 note has a lower frequency of 110 Hz, while the green curve represents the A5 note with a higher frequency of 880 Hz, demonstrating that the ‘closer’ the waves, the higher the pitch.
Figure 1. This graph displays the waveforms of musical notes A1 and A5 sampled at a rate of 44.1 kHz. The waveform of the A1 note has a lower frequency of 110 Hz, while the green curve represents the A5 note with a higher frequency of 880 Hz, demonstrating that the ‘closer’ the waves, the higher the pitch.
Sensors 24 00065 g001
Sensors 24 00065 g002
Figure 2. The graph displays three sine waves, each generated at 440 Hz but with different amplitudes of 10 dB, 30 dB, and 50 dB, demonstrating that a higher waveform results in a higher amplitude.
Figure 2. The graph displays three sine waves, each generated at 440 Hz but with different amplitudes of 10 dB, 30 dB, and 50 dB, demonstrating that a higher waveform results in a higher amplitude.
Sensors 24 00065 g002
Sensors 24 00065 g003
Figure 3. Displays waveforms of 3 audio signals: human voice, piano, and tuning fork, all centred around 440 Hz fundamental frequency. Timbre varies in all due to the difference in unique harmonic structure, seen in each waveform.
Figure 3. Displays waveforms of 3 audio signals: human voice, piano, and tuning fork, all centred around 440 Hz fundamental frequency. Timbre varies in all due to the difference in unique harmonic structure, seen in each waveform.
Sensors 24 00065 g003
Sensors 24 00065 g004
Figure 4. A diagram outlining the steps commonly taken during the phase vocoder algorithm for pitch shifting and time-stretching purposes.
Figure 4. A diagram outlining the steps commonly taken during the phase vocoder algorithm for pitch shifting and time-stretching purposes.
Sensors 24 00065 g004
Sensors 24 00065 g005
Figure 5. A diagram illustrating the DRC algorithm and how it is used to increase and decrease gain on audio input based on a set threshold.
Figure 5. A diagram illustrating the DRC algorithm and how it is used to increase and decrease gain on audio input based on a set threshold.
Sensors 24 00065 g005
Sensors 24 00065 g006
Figure 6. This diagram provides examples, referred to in text, of how sonification and auditory manipulation techniques could be applied to gait rehabilitation, illustrating the extraction of various features from inertial measurement units (IMUs) on the feet. The IMU-based gait characteristics could undergo sonification to generate biofeedback acoustic variables through specialized auditory techniques, thereby changing the person’s gait.
Figure 6. This diagram provides examples, referred to in text, of how sonification and auditory manipulation techniques could be applied to gait rehabilitation, illustrating the extraction of various features from inertial measurement units (IMUs) on the feet. The IMU-based gait characteristics could undergo sonification to generate biofeedback acoustic variables through specialized auditory techniques, thereby changing the person’s gait.
Sensors 24 00065 g006
Sensors 24 00065 g007
Figure 7. This box plot illustrates the distribution of mean/initial steps per minute (SPM) ratios for two conditions: verbal instruction and noise feedback—fixed target. The blue lines indicate the SPM ratio (1) and the target SPM ratio (1.5).
Figure 7. This box plot illustrates the distribution of mean/initial steps per minute (SPM) ratios for two conditions: verbal instruction and noise feedback—fixed target. The blue lines indicate the SPM ratio (1) and the target SPM ratio (1.5).
Sensors 24 00065 g007
Sensors 24 00065 g008
Figure 8. Eta Squared (η2G%) values for the top five outcome measures. The bar chart represents the η2G% for the New Freezing of Gait Questionnaire (NFOGQ), Parkinson’s Disease Questionnaire-39 mobility domain (PDQ39 mobility), Unified Parkinson’s Disease Rating Scale Part III (UPDRSIII), PDQ39 bodily discomfort, and PDQ39 total. Notably, NFOGQ exhibits the highest η2G% value.
Figure 8. Eta Squared (η2G%) values for the top five outcome measures. The bar chart represents the η2G% for the New Freezing of Gait Questionnaire (NFOGQ), Parkinson’s Disease Questionnaire-39 mobility domain (PDQ39 mobility), Unified Parkinson’s Disease Rating Scale Part III (UPDRSIII), PDQ39 bodily discomfort, and PDQ39 total. Notably, NFOGQ exhibits the highest η2G% value.
Sensors 24 00065 g008
Sensors 24 00065 g009
Figure 9. A bar chart illustrating the results of the experiment, demonstrating the significant effectiveness of sonification in increasing step length.
Figure 9. A bar chart illustrating the results of the experiment, demonstrating the significant effectiveness of sonification in increasing step length.
Sensors 24 00065 g009
Table 1. Summary of Soni version principles.
Table 1. Summary of Soni version principles.
AttributeSoni 1.1Soni 1.2Soni 2.1Soni 2.2
Phase of ActionSwing PhaseSwing PhaseStance PhaseStance Phase
Tone TypeContinuous SineContinuous SineContinuous SineContinuous Sine
Angle Range35° to −45° to +90°35° to −45° to +90°Up to 35°Up to 35°
Frequency Range220 Hz to 1760 HzLower by half an octave (165 Hz to 1320 Hz)220 Hz to 1760 HzLower by half an octave (165 Hz to 1320 Hz)
Table 2. Summary of gait sonification methods and outcomes.
Table 2. Summary of gait sonification methods and outcomes.
ApproachMethodPopulationMain Outcomes
Cadence Sonification [17].A music-sonification system with pink noise, runs under various conditions. Real-time adjustment based on the runner’s cadence.10 healthy participantsPink noise feedback significantly better than verbal instruction for altering cadence. Effective for both increasing and decreasing cadence.
Velocity Sonification [59].Action Observation plus Sonification (AOS) using visual and enhanced auditory cues.37 PD patientsReduced severity and duration of FoG. Improved motor function and daily activity comfort. Lasting benefits observed.
Sonification of Swing Phase [18].Real-time sonification of the swing phase of the gait cycle with specific pitch tones.9 PD patientsStatistically significant reduction in the CoV for step length. Improved gait consistency.
Sonification of Knee Flexion [62].Prototype system Sofigait for sonification during walking. Different versions of sonification techniques.24 healthy individualsLower pitch (Soni 1.2) was found to be more pleasant. Limited data on the effectiveness in reducing gait asymmetry.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Wall, C.; McMeekin, P.; Walker, R.; Hetherington, V.; Graham, L.; Godfrey, A. Sonification for Personalised Gait Intervention. Sensors 2024, 24, 65. https://doi.org/10.3390/s24010065

AMA Style

Wall C, McMeekin P, Walker R, Hetherington V, Graham L, Godfrey A. Sonification for Personalised Gait Intervention. Sensors. 2024; 24(1):65. https://doi.org/10.3390/s24010065

Chicago/Turabian Style

Wall, Conor, Peter McMeekin, Richard Walker, Victoria Hetherington, Lisa Graham, and Alan Godfrey. 2024. "Sonification for Personalised Gait Intervention" Sensors 24, no. 1: 65. https://doi.org/10.3390/s24010065

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop