The collection and analysis of the data was carried out by one of the authors, who also participated in the workshop as an organiser and assistant. Although she is hearing and sighted, she can be considered a “native” sensory ethnographer in this context. She has many years of experience of working with deafblind people and through marriage she is also sharing a deafblind persons’ everyday life. As an interpreter of the deafblind perspective, she is as well-suited as possible for anyone not sharing this very particular condition. In her interpretation of this data she is therefore able to use her experiential and long-term personal knowledge in her role as researcher. The participants’ verbalised statements of their experiences provide us with only a tiny part of all the dimensions of the event, but through the interpretation of the researcher these are reflected through the theoretical lens as well as through an overall understanding of the multisensory experience and multimodal communication of the participants.
As a frame for the text-based description analysis, we used the model by Lederman and Klatzky [35
] concerning hand movements during haptic object recognition. This model includes the notions of sensing of pressure, vibration, temperature, postures and movements [35
] (p. 121) but has also been extended by Akner-Koler and Ranjbar [10
] (p. 4). In this article, we further developed the model to include the notion of amount
. For the vocalisation and sounds, we utilised a qualitative content analysis based on the sounds produced
, means of sound production
, type of description
, pitch and volume
3.1. Text Description
The independent deafblind person who gave us the text descriptions is blind and not using any hearing devices, which means there is no useful hearing available. The person has many decades of experience of receiving different kinds of environmental descriptions by interpreters, personal assistants and friends. In everyday life, the person mainly uses the tactile sense which constitutes the main channel for receiving information. The person has thus developed a haptic aesthetic sensitivity for tactile differentiations and was able to utilise this expertise in the written production of a haptic description. The person is also very verbally talented and has an analytical disposition.
The text descriptions included material and texture, their shape, size or thickness, temperature, and weight as elaborated in a previous version of this study [36
]. These features have been recognised by Akner-Koler and Ranjbar [10
] and Klatzky and Lederman [37
]. However, in addition to these familiar aspects in haptic research, we also detected the properties of amount, orientation and mental image in the text descriptions (see Table 2
). As an example, we here show the fourth text description made by the deafblind person:
‘This fragile sculpture is like a metal wrapping that swirls upwards. The outside surface of the wrapping is smooth in a rough way but the inside is spiky. The wrapping is shaped like a tube that fits well inside a hand. Some parts of the sculpture extend above the tube- shaped wrapping, and above it there is a nob or a head’.
This text shows similarities to the sound description made by participant 4 (see Table 3
Both informants described the different materials, shapes/forms and surface structure of all the sculptures. The thickness, amount and weight of the objects were also expressed. In the written text, there were also descriptions of different parts of the sculpture and their relationships, that is, their orientation to each other (swirls upwards).
3.2. Vocal and Sound Descriptions
A text description might be repeated similarly using similar words by other people later, but a sound reproduction is individual. The sound description captured a momentary experiential notion, experimenting with the sounds the participants could produce. In comparison, the text description took a much longer time to produce and allowed the participant to read and correct it over time. In addition, mental images were created by analogy in both text and sound descriptions where the participants had had sight and thus visual memory.
Participants 1 and 4 did not have any visual image of a subject matter of the sculptures, in other words they have been visually impaired since early childhood. Participant 1 produced sounds by using their own voice and hands, sometimes at the same time describing the sculpture’s material and surface texture and quantity. Participant 4 mentioned that the sculpture did not evoke any predefined real image when exploring it, but the whooshing sound was influenced by the participant’s perceived mental image of a hollow pipe. The sounds produced included whispering sounds, building up into a storm that turned into a howling wind sound. The howl (storm) sounds like a meditative humming that is building up into a loud dramatic climax, “a deathly cry”. These sounds were produced by their own voices, blowing in and out, starting at a low pitch and then moving through a middle- to a higher-level pitch (Table 3
One of the participants, participant 4, has formal training and education in music therapy. The participant also has a long background as a composer, song writer and singer, with concerts in arenas such as the Finlandia Hall in Helsinki. The participant described the artistic process of making the sound afterwards as follows;
‘Since I was not able to see the sculpture, I used my hands as a basis to explore and to create improvised sounds using my voice. Working from the base level of the sculpture, my hands moved around the shape upwards towards the top. As I was doing this, I also created my own improvised sounds through blowing and breathing heavily like a rushing wind sensation. At the same time as my hands explored the hollow windpipe-like structure, I started to create a vocalised variable humming sound which got louder and louder and increased in pitch level at the same time. This crescendo built up to a deathly cry at the end which was influenced by the sharp jagged shape of the sculpture at the top’.
As participant 4 described the shape of the sculpture, he synchronised his voice with the movements of his hands. The shape influenced the vocalising of the pitch level from low to high with a dramatic ending as the hands found the sharp jagged shape of the sculpture at the top. The participant did a creative, improvised soundscape, which was more varying and longer compared to some of the others.
Participants 2, 3 and 5 had a clear and predefined mental image of the themes of the sculptures, which are of a dog, a meditating person and a camp fire, as they had vision before becoming blind. Participant 2 produced a dog sound using their own voice and was able to relatively easily produce this because they were familiar sounds (they had experience of hearing), therefore imitating it was possible. The dog-shaped sculpture was thus described by a dog-like barking and exploring through sniffing (sniff) and growling (grr).
Participant 3 had an image of a person sitting down holding a ball in his hands. This is described by a “thinking, humming, meditating” sound. The meditative sound was relatively easy to produce in a simple and musical manner, producing two pitch levels. Participant 5 had a predefined image of a campfire and had had personal experience of being at a campfire before. The participant was touching the sculpture using hands while wearing a ring on the finger, thus creating a rustling and clicking sound effect. Wood as a material is very responsive and you can feel the vibration. The participant was creating different sounds through the mouth, such as clicking and a wind sound effect (blowing in and out). No vocalisation sounds were produced in this description.
In general, the sound descriptions were combinations of different creative hand and vocalised sounds which had different levels of volume. Three participants of the five (2, 3, 5) had a clear idea of the subject, that is, a clear mental image of the subject matter based on their haptic exploration and previous experience. Their sound descriptions thus mimicked the sound of the subject. However, there is no agreed vocabulary for sound descriptions. Non-vocalised sound descriptions are individual, personal experiences based on touch and hand movements of the art work. When compared to the description in written format, the words used have a certain learnt meaning and are based on linguistic grammar. A similar type of text could be produced by another person, because haptically we would pick up the same kinds of elements, such as material, temperature, size, etc. (see [10
We analysed how the sound descriptions represented inner experiences of haptic exploration (the mental image of sculpture) and were re-interpreted by each participant’s own voice and body sounds, sometimes in combination with sounds made by touching the art work. These sounds describe the participants’ instantaneous experiences as their hands touch one point of the object, moving to the next point, discovering the material and size. They did not have a unified agreed tone symbol in use. The sound descriptions can be described as short, experimental and playful sound effects that these participants tried out for the first time in their life. We experience these sound descriptions as personal artistic productions by the participants and unique in themselves but that may be related to on a general level.
The sound descriptions were instructed by a born-blind music teacher, educated at the Sibelius Academy in Helsinki, Finland; they were designed and thought through by the participants, rehearsed and performed sincerely. The participants also reported being empowered by the experience of overcoming their shyness and trusting their ability to create new expressions by utilising their voices in new ways. This led to the idea that the creation of sound descriptions could be also utilised in adult CI rehabilitation, where speech and voice are used. Participant 4 said:
‘The only rehab I got was how to pronounce sounds, but I did not get any therapy where I could listen to my own sounds through CI or any experience of using my own voice or using my breath in the right way. Many deaf people are afraid to use their voices and levels of dynamics as they are not able to judge the intensities or volumes.’
Many of the participants extended their voice production to include adapted sounds, also using hand clapping and drumming and tapping on the sculptures. In this way, they started a communicative process with the sculpture, one in which the sculpture was also brought into the act of sound making. The sounds were not only human but also came from the sculpture itself. If we understand the aesthetic experience as wider than only the distant visual or auditive experience, this way of experiencing may even enhance the aesthetic experience of an object. These participants have a bodily communication with the object that allows them to take it into their physical realm and “play it as an instrument” through a haptic aesthetic exploration. This sense of play also triggered humorous aspects and joy. Further, it provided the opportunity to dwell on the art piece and to take time with it, to indulge in it and to participate in it.
This haptic aspect of the aesthetic experience is seldom generally available. This is also apparent in the poor vocabulary for tactile experiences [13
]. The are many colour descriptions of various shades of red, for example, but few shades of softness or hardness. In the sound descriptions, the participants were able to “visualise” their experiences with intonation and volume as well as multiple simultaneous sounds giving a kind of three-dimensional view of the sculpture. In this sense, the sound descriptions were more multi-faceted than mere words, linking to the theory of multimodal communication becoming enriched by multiple modes of expression. The sound descriptions were describing the landscapes of the sculpture surfaces, giving time-based narratives of the journey along the surface. Participant 4 described the recording as helping to create new memories of the sculpture “While making the sound description I created my own story that helped me to remember the sculpture’s shape”. Naturally, these embodied memories are important for people who cannot outsource their memory to photographs.
When participant 4 was asked about what mental image he had of the sculpture one and a half year later, the participant described the hollow pipe with hand movements swirling up and the top of the sculpture, that earlier was mentioned as the “deadly cry” as a cry for “help”. The experience was re-lived and the participant expressed the same severity and despair.