Sound Descriptions of Haptic Experiences of Art Work by Deafblind Cochlear Implant Users
Abstract
:1. Introduction
1.1. Deafblindness and Perceived CI Sounds
1.2. Talking Devices and Computer-Assisted Technology
1.3. Multimodality
2. Methods
2.1. Research Setting
2.2. Sound Descriptions of Haptic–Aesthetic Experiences
3. Results and Discussion
3.1. Text Description
‘This fragile sculpture is like a metal wrapping that swirls upwards. The outside surface of the wrapping is smooth in a rough way but the inside is spiky. The wrapping is shaped like a tube that fits well inside a hand. Some parts of the sculpture extend above the tube- shaped wrapping, and above it there is a nob or a head’.
3.2. Vocal and Sound Descriptions
‘Since I was not able to see the sculpture, I used my hands as a basis to explore and to create improvised sounds using my voice. Working from the base level of the sculpture, my hands moved around the shape upwards towards the top. As I was doing this, I also created my own improvised sounds through blowing and breathing heavily like a rushing wind sensation. At the same time as my hands explored the hollow windpipe-like structure, I started to create a vocalised variable humming sound which got louder and louder and increased in pitch level at the same time. This crescendo built up to a deathly cry at the end which was influenced by the sharp jagged shape of the sculpture at the top’.
‘The only rehab I got was how to pronounce sounds, but I did not get any therapy where I could listen to my own sounds through CI or any experience of using my own voice or using my breath in the right way. Many deaf people are afraid to use their voices and levels of dynamics as they are not able to judge the intensities or volumes.’
4. Conclusions
Benefits of this Research
Supplementary Materials
Author Contributions
Acknowledgments
Conflicts of Interest
References
- Pallasmaa, J. The Eyes of the Skin; John Wiley & Sons: Chichester, UK, 2005. [Google Scholar]
- Pallasmaa, J. The Thinking Hand; John Wiley & Sons: Chichester, UK, 2009. [Google Scholar]
- Vermeersch, P.; Nijs, G.; Heylighen, A. Mediating artifacts in architectural design: A non-visual exploration. In Proceedings of the CAAD Futures: Designing Together, Liege, Belgium, 4–8 July 2011; Les editions de l’Universite de Liege: Liege, Belgium, 2011; pp. 721–734. [Google Scholar]
- Heylighen, A.; Nijs, G. Designing in the absence of sight: Design cognition rearticulated. Des. Stud. 2014, 35, 113–132. [Google Scholar] [CrossRef]
- Herssens, J.; Heylighen, A. Challenging architects to include haptics in design: Sensory paradox between content and representation. In Designing Together—CAAD Futures 2011; Leclercq, P., Ed.; Les Editions de L’Universite’ de Liege: Liege, Belgium, 2011; pp. 685–700. [Google Scholar]
- Heylighen, A.; Bouwen, J.; Neuckermans, H. Walking on a thin line. Des. Stud. 1999, 20, 211–235. [Google Scholar] [CrossRef]
- Shinohara, K.; Tenenberg, J. A blind person’s communication with technology. Commun. ACM 2009, 52, 58–66. [Google Scholar] [CrossRef]
- Groth, C.; Mäkelä, M.; Seitamaa-Hakkarainen, P. Making Sense—What can we learn from experts of tactile knowledge? FORMakadem. J. 2013, 6, 1–12. [Google Scholar] [CrossRef]
- Groth, C. Making Sense through Hands: Design and Craft Practice Analysed as Embodied Cognition. Doctoral Thesis, Aalto Arts Books, Helsinki, Finland, 2017. [Google Scholar]
- Akner-Koler, S.; Ranjbar, P. Integrating Sensitizing Labs in an Educational Design Process for Haptic interaction. FORMakadem. J. 2016, 9, 1–25. [Google Scholar] [CrossRef]
- Merleau-Ponty, M. Phenomenology of Perception; Routledge: London, UK, 2010; (Original English Version Published in 1945). [Google Scholar]
- Ingold, T. Culture on the ground. J. Mater. Cult. 2004, 9, 315–340. [Google Scholar] [CrossRef]
- Mcpherson, H. Articulating blind touch: Thinking through the feet. Senses Soc. 2009, 4, 179–194. [Google Scholar] [CrossRef]
- Harrison, P. Making sense: Embodiment and the sensibilities of the everyday. Environ. Plan. D Soc. Space 2000, 18, 497–517. [Google Scholar] [CrossRef]
- Paterson, M. Haptic geographies: Ethnography, haptic knowledges and sensuous dispositions. Prog. Hum. Geogr. 2009, 33, 766–788. [Google Scholar] [CrossRef]
- Gallace, A. Living with touch. Psychol. Br. Psychol. Soc. 2012, 25, 896–899. [Google Scholar]
- Gibson, J.J. The Senses Considered as Perceptual Systems; Greenwood Press: Westport, CT, USA, 1983; (Original Work Published in 1966). [Google Scholar]
- Torppa, R. Pitch-Related Auditory Skills in Children with Cochlear Implants: The Role of Auditory Working Memory, Attention and Music; Cognitive Brain Research Unit, Cognitive Science, Institute of Behavioral Sciences University of Helsinki: Helsinki, Finland, 2015. [Google Scholar]
- Zatorre, R.J.; Baum, S.H. Musical Melody and Speech Intonation: Singing a Different Tune? PLoS Biol. 2012, 10, e1001372. [Google Scholar] [CrossRef] [PubMed]
- Maarefvand, M.; Marozeau, J.; Blamey, P.J. A cochlear implant user with exceptional musical hearing ability. Int. J. Audiol. 2013, 52, 424–432. [Google Scholar] [CrossRef] [PubMed]
- Prause-Weber, M.; Schraer-Joiner, L. Cochlear implants can help patients enjoy listening to and making music. Hear. J. 2007, 60, 60–63. [Google Scholar] [CrossRef]
- Lahtinen, R. Haptices and Haptemes—A Case Study of Developmental Process in Social-Haptic Communication of Acquired Deafblind People. Ph.D. Thesis, University of Helsinki, Helsinki, Finland. Cityoffset Oy: Tampere, Finland, 2008.
- Palmer, R.; Ojala, S. Feeling music vibrations—A vibrosensoric experience. In Proceedings of the BNAM2016, Stockholm, Sweden, 20–22 June 2016. Paper 53. [Google Scholar]
- Everett, J.; Gibert, W. Art and touch: A conceptual approach. Br. J. Vis. Impair. 1992, 9, 87–89. [Google Scholar] [CrossRef]
- Nilsson, M.; Schenkman, B. Blind people are more sensitive than sighted people to binaural sound-location cues, particularly inter-aural level differences. Hear. Res. 2016, 332, 223–232. [Google Scholar] [CrossRef] [PubMed]
- Tuominen, H.T.; Palmer, R.; Korhonen, I.; Ojala, S. Evidence-based study on performance environment for people with and without cochlear implants (CI). In Proceedings of the BNAM2014 Conference, Tallinn, Estonia, 2–4 June 2014; ISBN 978-87-995400-1-3. [Google Scholar]
- Lahtinen, R.; Palmer, R.; Lahtinen, M. Environmental Description for Visually and Dual Sensory Impaired People; Art-Print Oy: Helsinki, Finland, 2010. [Google Scholar]
- Jewitt, C.; Bezemer, J.; O’Halloran, K. Introducing Multimodality; Routledge: London, UK, 2016. [Google Scholar]
- Pink, P. A multisensory approach to visual methods. In The SAGE Handbook Visual Research Methods; Margolis, E., Pauwels, L., Eds.; SAGE Publications: London, UK, 2011; pp. 601–615. [Google Scholar]
- Gallace, A.; Spence, C. The science of interpersonal touch: An overview. Neurosci. Biobehav. Rev. 2010, 34, 246–259. [Google Scholar] [CrossRef] [PubMed]
- Shifferstein, H.; Wastiels, L. Sensing materials: Exploring the building blocks for experiential design. In Materials Experience: Fundamentals of Materials and Design; Karana, E., Pedgley, O., Rognoli, V., Eds.; Elsevier: Oxford, UK, 2014. [Google Scholar]
- Pink, S. Doing Sensory Ethnography; SAGE Publications: London, UK, 2009. [Google Scholar]
- Keller, H. The World I Live in; Hodder & Stoughton: London, UK, 1908. [Google Scholar]
- Nicholas, J. From Active Touch to Tactile Communication: What’s Tactile Cognition Got to Do with It? The Danish Resource Centre on Congenital Deafblindness: Aalborg, Denmark, 2010. [Google Scholar]
- Lederman, S.; Klatzky, R. Hand movements: A window into haptic object recognition. Cogn. Psychol. 1987, 3, 342–368. [Google Scholar] [CrossRef]
- Lahtinen, R.; Groth, C.; Palmer, R. Haptic Art Experiences Described as Vocals, Sounds and Written Words by Deafblind. In Proceedings of the Art of Research Conference, Helsinki, Finland, 28–29 November 2017. [Google Scholar]
- Klatzky, R.L.; Lederman, S.J. Toward a computational model of constraint-driven exploration and haptic object identification. Perception 1993, 22, 603–604. [Google Scholar] [CrossRef] [PubMed]
Participant 1 | Participant 2 | Participant 3 | Participant 4 | Participant 5 | |
---|---|---|---|---|---|
Gender | Male | Male | Male | Male | Female |
Age | 58 | 54 | 34 | 22 | 41 |
VI | Blind | VI | VI | VI | VI |
HD 2016 | 2 CI (2004, 2010) | 2 CI (2011, 2012) | CI (2016, 2017) | 2 HABE (2015) | 2 CI (2012, 2014) |
First HD | 4.5 years old | 3 years old | 4 years old | 7 years old | 7 years old |
Properties of Interpreted Haptic Experiences | Elements of Text Description of Sculpture 4 |
Material | Metal |
Shape | Swirl, nob, head |
Size, thickness | Fits inside hand |
Amount | |
Temperature | |
Weight | |
Texture | Smooth, rough way, spiky |
Orientation | Swirls upwards, outside surface, above |
Other, such as mental images | Fragile, shaped like a tube, wrapping |
Analysis of vocal and sound description | Participant 4, Elements of Sound Description of Sculpture 4 |
Produced sound | Whim…woosh howl…AAAH |
Means of sound production | Voice, blowing |
Type of description | Shape |
Other such as pitch and volume | Low, middle, higher pitch level |
© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Lahtinen, R.; Groth, C.; Palmer, R. Sound Descriptions of Haptic Experiences of Art Work by Deafblind Cochlear Implant Users. Multimodal Technol. Interact. 2018, 2, 24. https://doi.org/10.3390/mti2020024
Lahtinen R, Groth C, Palmer R. Sound Descriptions of Haptic Experiences of Art Work by Deafblind Cochlear Implant Users. Multimodal Technologies and Interaction. 2018; 2(2):24. https://doi.org/10.3390/mti2020024
Chicago/Turabian StyleLahtinen, Riitta, Camilla Groth, and Russ Palmer. 2018. "Sound Descriptions of Haptic Experiences of Art Work by Deafblind Cochlear Implant Users" Multimodal Technologies and Interaction 2, no. 2: 24. https://doi.org/10.3390/mti2020024