Search Results (17)

People use their hands for intricate tasks like playing musical instruments, employing myriad touch sensations to inform motor control. In contrast, current prosthetic hands lack comprehensive haptic feedback and exhibit rudimentary multitasking functionality. Limited research has explored the potential of upper limb amputees to feel, perceive, and respond to multiple channels of simultaneously activated haptic feedback to concurrently control the individual fingers of dexterous prosthetic hands. This study introduces a novel control architecture for three amputees and nine additional subjects to concurrently control individual fingers of an artificial hand using two channels of context-specific haptic feedback. Artificial neural networks (ANNs) recognize subjects’ electromyogram (EMG) patterns governing the artificial hand controller. ANNs also classify the directions objects slip across tactile sensors on the robotic fingertips, which are encoded via the vibration frequency of wearable vibrotactile actuators. Subjects implement control strategies with each finger simultaneously to prevent or permit slip as desired, achieving a 94.49% ± 8.79% overall success rate. Although no statistically significant difference exists between amputees’ and non-amputees’ success rates, amputees require more time to respond to simultaneous haptic feedback signals, suggesting a higher cognitive load. Nevertheless, amputees can accurately interpret multiple channels of nuanced haptic feedback to concurrently control individual robotic fingers, addressing the challenge of multitasking with dexterous prosthetic hands. Full article

Haptic feedback holds the potential to enhance the engagement and expressivity of future digital and electric musical instruments. This study investigates the impact of artificial vibration on the perceived quality of a silent electric cello. We developed a haptic cello prototype capable of rendering vibration signals of varying degree of congruence with the produced sound. Experienced cellists participated in an experiment comparing setups with and without vibrotactile feedback, rating them on preference, perceived power, liveliness, and feel. Results show nuanced effects, with added vibrations moderately enhancing feel and liveliness, and significantly increasing perceived power when using vibrations obtained from the pickup at the cello’s bridge. High uncertainty in our statistical model parameters underscores substantial individual differences in the participants responses, as commonly found in qualitative assessments, and highlights the importance of consistent feedback in the vibrotactile and auditory channels. Our findings contribute valuable insights to the intersection of haptics and music technology, paving the way for creating richer and more engaging experiences with future musical instruments. Full article

The metaphor of being touched by music is widespread and almost universal. The tactile experience, moreover, has received growing interest in recent years. There is, however, a need to go beyond a mere metaphorical use of the term, by positioning the tactile experience within the broader frame of embodied cognition and the experiential turn in cognitive science. This article explores the possible contribution of a science of touch by defining music as a vibrational phenomenon that affects the body and the senses. It takes as a starting point the clinical findings on the psychological and physiological value of tender touch with a special focus on the method of kangaroo mother care, which is a method for holding the baby against the chest of the mother, skin-to-skin. It is seen as one of the most basic affiliative bondings with stimuli that elicit reward. Via an extensive review of the research literature, it is questioned as to what extent this rationale can be translated to the realm of music. There are, in fact, many analogies, but a comprehensive theoretical framework is still lacking. This article aims at providing at least some preparatory groundwork to fuel more theorizing about listening and its relation to the sense of touch. Full article

This article describes the design, implementation, and evaluation of vibrotactile concert furniture, aiming to improve the live music experience of people with hearing loss using hearing technology such as cochlear implants (CI). The system was the result of a series of participatory design sessions involving CI users with different hearing assistive setups (bi-implant, bimodal, and monoimplant), and it was evaluated in a concert scenario (drums, bass, and female vocals) at the Royal Danish Academy of Music. The project aimed to improve the music appreciation for CI users by providing a multisensory concert designed with CI challenges in mind, but not excluding normal-hearing individuals or individuals with other forms of hearing aids from participating in the event. The evaluation was based on (video-recorded) observations and postexperience semistructured interviews; the data were analyzed using event analysis and meaning condensation. The results indicate that tactile augmentation provides a pleasant experience for CI users. However, concertgoers with residual hearing reported being overwhelmed if the tactile stimulation amplitude exceeds a certain threshold. Furthermore, devices that highlight instrument segregation are preferred over ones that present a tactile mixdown of multiple auditory streams. Full article

A growing body of work on musical haptics focuses on vibrotactile feedback, while musical applications of force feedback, though more than four decades old, are sparser. This paper reviews related work combining music and haptics, focusing on force feedback. We then discuss the limitations of these works and elicit the main challenges in current applications of force feedback and music (FF&M), which are as follows: modularity; replicability; affordability; and usability. We call for the following opportunities in future research works on FF&M: embedding audio and haptic software into hardware modules, networking multiple modules with distributed control, and authoring with audio-inspired and audio-coupled tools. We illustrate our review with recent efforts to develop an affordable, open-source and self-contained 1-Degree-of-Freedom (DoF) rotary force-feedback device for musical applications, i.e., the TorqueTuner, and to embed audio and haptic processing and authoring in module firmware, with ForceHost, and examine their advantages and drawbacks in light of the opportunities presented in the text. Full article

Today, some of the most widely attended concerts are in virtual reality (VR). For example, the videogame Fortnite recently attracted 12.3 million viewers sitting in homes all over the world to a VR Travis Scott rap concert. As such VR concerts become increasingly ubiquitous, we are presented with an opportunity to design more immersive virtual experiences by augmenting VR with other multisensory technologies. Given that sound is a multi-modal phenomenon that can be experienced sonically and vibrationally, we investigated the importance of haptic feedback to musical experiences using a combination of qualitative and empirical methodologies. Study 1 was a qualitative study demonstrating that, unlike their live counterparts, current VR concerts make it harder for audiences to form a connection with artists and their music. Furthermore, VR concerts lack multisensory feedback and are perceived as less authentic than live concert experiences. Participants also identified a variety of different kinds of touch that they receive at live concerts and suggested that ideal VR concerts would replicate physical touch and thermal feedback from the audience, emotional touch, and vibrations from the music. Specifically, users advocated for the use of haptic devices to increase the immersiveness of VR concert experiences. Study 2 isolated the role of touch in the music listening experience and empirically investigated the impact of haptic music players (HMPs) on the audio-only listening experience. An empirical, between-subjects study was run with participants either receiving vibrotactile feedback via an HMP (haptics condition) or no vibrotactile feedback (control) while listening to music. Results indicated that listening to music while receiving vibrotactile feedback increased participants’ sense of empathy, parasocial bond, and loyalty towards the artist, while also decreasing participants’ feelings of loneliness. The connection between haptics condition and these dependent variables was mediated by the feeling of social presence. Study 2 thus provides initial evidence that HMPs may be used to meet people’s need for connection, multisensory immersion, and complex forms of touch in VR concerts as identified in Study 1. Full article

The physics-based design and realization of a digital musical interface asks for the modeling and implementation of the contact-point interaction with the performer. Musical instruments always include a resonator that converts the input energy into sound, meanwhile feeding part of it back to the performer through the same point. Specifically during plucking or bowing interactions, musicians receive a handful of information from the force feedback and vibrations coming from the contact points. This paper focuses on the design and realization of digital music interfaces realizing two physical interactions along with a musically unconventional one, rubbing, rarely encountered in assimilable forms across the centuries on a few instruments. Therefore, it aims to highlight the significance of haptic rendering in improving quality during a musical experience as opposed to interfaces provided with a passive contact point. Current challenges are posed by the specific requirements of the haptic device, as well as the computational effort needed for realizing such interactions without occurrence during the performance of typical digital artifacts such as latency and model instability. Both are however seemingly transitory due to the constant evolution of computer systems for virtual reality and the progressive popularization of haptic interfaces in the sonic interaction design community. In summary, our results speak in favor of adopting nowadays haptic technologies as an essential component for digital musical interfaces affording point-wise contact interactions in the personal performance space. Full article

Touch has a pivotal importance in determining the expressivity of musical performance for a number of musical instruments. However, most digital musical devices provide no interactive force and/or vibratory feedback to the performer, thus depriving the somatosensory channel of a number of cues. Is the lack of haptic feedback only an aesthetic issue, or does it remove cues essential for digital instrument playing? If so, at which level is the interaction objectively impoverished? What are the effects on musical performance? In this survey article we illustrate our recent research about the use of vibrotactile feedback in three digital instrument interfaces, using tools that we developed over several years and made available to the community in open-source form. These interfaces span a wide range of familiarity and gestural opportunities, enabling us to explore the impact of haptic feedback on different types of digital instruments. We conducted experiments with professional musicians to assess the impact of vibratory cues on both the perceived quality of the instrument and the playing experience, as well as on musical performance. Particular attention was paid to scientific rigor and repeatability of the results, so as to serve as a reference for researchers and practitioners of the musical haptics community. Our results suggest a significant role of vibrotactile feedback in shaping the perception of digital musical instruments, although the effects on musical performance varied depending on the interfaces tested. Full article

Despite technological and accessibility advances, the performing arts and their cultural offerings remain inaccessible to many people. By using vibrotactile stimulation as an alternative channel, we explored a different way to enhance emotional processes produced while watching audiovisual media and, thus, elicit a greater emotional reaction in hearing-impaired people. We recorded the brain activity of 35 participants with normal hearing and 8 participants with severe and total hearing loss. The results showed activation in the same areas both in participants with normal hearing while watching a video, and in hearing-impaired participants while watching the same video with synchronized soft vibrotactile stimulation in both hands, based on a proprietary stimulation glove. These brain areas (bilateral middle frontal orbitofrontal, bilateral superior frontal gyrus, and left cingulum) have been reported as emotional and attentional areas. We conclude that vibrotactile stimulation can elicit the appropriate cortex activation while watching audiovisual media. Full article

Tactile rendering has been implemented in digital musical instruments (DMIs) to offer the musician haptic feedback that enhances his/her music playing experience. Recently, this implementation has expanded to the development of sensory substitution systems known as haptic music players (HMPs) to give the opportunity of experiencing music through touch to the hearing impaired. These devices may also be conceived as vibrotactile music players to enrich music listening activities. In this review, technology and methods to render musical information by means of vibrotactile stimuli are systematically studied. The methodology used to find out relevant literature is first outlined, and a preliminary classification of musical haptics is proposed. A comparison between different technologies and methods for vibrotactile rendering is performed to later organize the information according to the type of HMP. Limitations and advantages are highlighted to find out opportunities for future research. Likewise, methods for music audio-tactile rendering (ATR) are analyzed and, finally, strategies to compose for the sense of touch are summarized. This review is intended for researchers in the fields of haptics, assistive technologies, music, psychology, and human–computer interaction as well as artists that may make use of it as a reference to develop upcoming research on HMPs and ATR. Full article

We present and evaluate the concept of FeelMusic and evaluate an implementation of it. It is an augmentation of music through the haptic translation of core musical elements. Music and touch are intrinsic modes of affective communication that are physically sensed. By projecting musical features such as rhythm and melody into the haptic domain, we can explore and enrich this embodied sensation; hence, we investigated audio-tactile mappings that successfully render emotive qualities. We began by investigating the affective qualities of vibrotactile stimuli through a psychophysical study with 20 participants using the circumplex model of affect. We found positive correlations between vibration frequency and arousal across participants, but correlations with valence were specific to the individual. We then developed novel FeelMusic mappings by translating key features of music samples and implementing them with “Pump-and-Vibe”, a wearable interface utilising fluidic actuation and vibration to generate dynamic haptic sensations. We conducted a preliminary investigation to evaluate the FeelMusic mappings by gathering 20 participants’ responses to the musical, tactile and combined stimuli, using valence ratings and descriptive words from Hevner’s adjective circle to measure affect. These mappings, and new tactile compositions, validated that FeelMusic interfaces have the potential to enrich musical experiences and be a means of affective communication in their own right. FeelMusic is a tangible realisation of the expression “feel the music”, enriching our musical experiences. Full article

Using functional magnetic resonance imaging, here we monitored the brain activity in 12 early blind subjects and 12 blindfolded control subjects, matched for age, gender and musical experience, during a beat detection task. Subjects were required to discriminate regular (“beat”) from irregular (“no beat”) rhythmic sequences composed of sounds or vibrotactile stimulations. In both sensory modalities, the brain activity differences between the two groups involved heteromodal brain regions including parietal and frontal cortical areas and occipital brain areas, that were recruited in the early blind group only. Accordingly, early blindness induced brain plasticity changes in the cerebral pathways involved in rhythm perception, with a participation of the visually deprived occipital brain areas whatever the sensory modality for input. We conclude that the visually deprived cortex switches its input modality from vision to audition and vibrotactile sense to perform this temporal processing task, supporting the concept of a metamodal, multisensory organization of this cortex. Full article

Herein, we introduce SONIS, a wearable system to support gait rehabilitation training after a lower extremity trauma, which combines a sensing sock with a smartphone application. SONIS provides interactive, corrective, real-time feedback combining visual and auditory cues. We report the design of SONIS and its evaluation by patients and therapists, which indicates acceptance by targeted users, credibility as a rehabilitation tool, and a positive user experience. SONIS demonstrates how to successfully combine a number of feedback strategies and modalities: graphical, verbal, and music feedback on gait quality during training (knowledge of performance) and verbal and vibrotactile feedback on gait tracking (knowledge of results). Full article

Current advancements in music technology enable the creation of customized Digital Musical Instruments (DMIs). This paper presents a systematic review of Accessible Digital Musical Instruments (ADMIs) in inclusive music practice. History of research concerned with facilitating inclusion in music-making is outlined, and current state of developments and trends in the field are discussed. Although the use of music technology in music therapy contexts has attracted more attention in recent years, the topic has been relatively unexplored in Computer Music literature. This review investigates a total of 113 publications focusing on ADMIs. Based on the 83 instruments in this dataset, ten control interface types were identified: tangible controllers, touchless controllers, Brain–Computer Music Interfaces (BCMIs), adapted instruments, wearable controllers or prosthetic devices, mouth-operated controllers, audio controllers, gaze controllers, touchscreen controllers and mouse-controlled interfaces. The majority of the AMDIs were tangible or physical controllers. Although the haptic modality could potentially play an important role in musical interaction for many user groups, relatively few of the ADMIs (14.5%) incorporated vibrotactile feedback. Aspects judged to be important for successful ADMI design were instrument adaptability and customization, user participation, iterative prototyping, and interdisciplinary development teams. Full article

In this study, the influence of piano key vibration levels on players’ personal judgment of the instrument quality and on the dynamics and timing of the players’ performance of a music piece excerpt is examined. In an experiment four vibration levels were presented to eleven pianists playing on a digital grand piano with grand piano-like key action. By evaluating the players’ judgment of the instrument quality, strong integration effects of auditory and tactile information were observed. Differences in the sound of the instrument were perceived by the players, when the vibration level in the keys was changed and the results indicate a sound-dependent optimum of the vibration levels. By analyzing the influence of the vibration levels on the timing and dynamics accuracy of the pianists’ musical performances, we could not observe systematic differences that depend on the vibration level. Full article