Creative Methods and Tools for Multimodal Technologies

A special issue of Multimodal Technologies and Interaction (ISSN 2414-4088).

Deadline for manuscript submissions: closed (10 December 2021) | Viewed by 9261

Special Issue Editors

Institute of Education, University College London, London WC1E 6BT, UK
Interests: emerging technologies; digital touch; sensory ethnography; design futures
Institute of Education, University College London, London WC1E 6BT, UK
Interests: embodiment; technology and society; sensory ethnography
Institute of Education, University College London, London WC1E 6BT, UK
Interests: emerging technologies; embodiment; multisensory interaction; learning; cognition
Special Issues, Collections and Topics in MDPI journals
Institute of Education, University College London, London WC1E 6BT, UK
Interests: multimodality; interdisciplinary methods and methods innovation; digital interaction and communication
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleague,

This Special Issue aims to take stock of creative methods and tools for multimodal technologies, to assess their nature, value, and significance in a range of research and design contexts, and to look to their future. By creative methods, we mean arts/design-based and innovative forms of making and expression that help to represent, explore, and understand bodily-sensorial experience as well as meaning making through interaction. These might involve and extend audiovisual or literary forms, crafts, and performance. Although often working under different premises, they share many commonalities with, and indeed sometimes double up as, tools for the design of multimodal technologies—technologies that engage multiple senses beyond audiovisual. In the context of design processes, creative methods find parallels in such tools as rapid prototyping, mapping, design fictions, gaming, and cultural probes. 

Creative methods have a long history across social sciences, media studies, HCI, and design, often favouring interdisciplinary collaboration to evoke new ways of thinking and feeling, to reframe problems, embrace serendipity and improvisation, and make room for the unexpected. They have, over the last three decades or so, accompanied and complemented an increased focus on the body and the senses as sites for research into lived experience and social interaction. Their very position at the intersection of disciplines and in contexts of rapid technological innovation means that their trajectories can be difficult to trace and critically evaluate. 

This Special Issue invites researchers and designers to submit original articles which apply creative methods and/or tools in the context of empirical research on—or the design process of—multimodal technologies. It asks: How are creative methods and tools being brought to the development of multimodal technologies? What emphasis do these approaches place on, or beyond, individual senses and modes (touch, smell, vision, sound, taste) and their interaction? How do they engage with the body and wider sensorium in relation to the technologies they seek to understand and advance? What forms of knowledge do they produce and, critically, (why) does this matter?

The call is open to submissions that focus on creative and innovate approaches within a broad range of digital technologies (e.g., biosensing, haptics, interactive skin, robotics, VR and augmented environments, affective technologies), which actively seek to engage with the body and senses in novel ways beyond audiovisual (e.g., haptic, olfactory, gustatory, kinaesthetic). Creative methods and/or tools can be brought to a range of topics, including but not limited to:

  • Research focusing on the bodily-sensorial experience, interactional contexts, or design of multimodal technologies;
  • Design processes for multimodal technologies;
  • Studies that transcend boundaries between creative research methods and design tools;  
  • Cross-context or multi-sited research and design;
  • Studies which have explored or refined creative approaches over time;
  • Arts-inspired research methods, including audiovisual, literary, performance- or crafts-based forms of expression and representation (e.g., video, photography, painting, crafts, poetry, choreography/dance, re-enactments, installations);
  • Design-led methods and creative resources for design (e.g., design fiction, mapping, cultural and disruptive probes, rapid prototyping, digital prompts, card-based tools, gaming).

Dr. Kerstin Leder Mackley
Dr. Ned Barker
Prof. Dr. Sara Price
Prof. Dr. Carey Jewitt
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Multimodal Technologies and Interaction is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Published Papers (3 papers)

Order results
Result details
Select all
Export citation of selected articles as:

Research

16 pages, 2127 KiB  
Article
Employing a Dance-Somatic Methodological Approach to VR to Investigate the Sensorial Body across Physical-Virtual Terrains
by Lisa May Thomas
Multimodal Technol. Interact. 2022, 6(4), 25; https://doi.org/10.3390/mti6040025 - 07 Apr 2022
Cited by 1 | Viewed by 2235
Abstract
This article takes a somatic dance standpoint to investigate bodily sensorial encounters with virtual reality technology (VR). Using somatic dance practices as sensory tuning methods, this article investigates the virtuality and corporeality of human and nonhuman bodies across virtual and physical environments and [...] Read more.
This article takes a somatic dance standpoint to investigate bodily sensorial encounters with virtual reality technology (VR). Using somatic dance practices as sensory tuning methods, this article investigates the virtuality and corporeality of human and nonhuman bodies across virtual and physical environments and expanded sensory modes of seeing and feeling, using a multiperson interactive VR framework. Predicated on a dance-based sensibility and mode of engagement with VR, workshops crafted for groups of dancer and nondancer participants offer methods for the design and facilitation of VR encounters that expand ideas and expectations of both body and technology. Using autoethnographic writing and participant testimonies, alongside conceptual thinking around participation in VR, I present this methodological approach and demonstrate its potential to address new questions about VR experiences that are centred on the sensorial body. Full article
(This article belongs to the Special Issue Creative Methods and Tools for Multimodal Technologies)
Show Figures

Figure 1

22 pages, 3485 KiB  
Article
The Compositor Tool: Investigating Consumer Experiences in the Circular Economy
by Bruna Petreca, Sharon Baurley, Katrine Hesseldahl, Alexa Pollmann and Marianna Obrist
Multimodal Technol. Interact. 2022, 6(4), 24; https://doi.org/10.3390/mti6040024 - 06 Apr 2022
Cited by 3 | Viewed by 3545
Abstract
Humanity is living through a crisis that sees our way of life exhausting the resources of the earth and ourselves. The fashion sector shows the negative impacts of conspicuous consumption on our socioenvironmental wellbeing. Despite citizens’ growing awareness of their responsibility within consumption [...] Read more.
Humanity is living through a crisis that sees our way of life exhausting the resources of the earth and ourselves. The fashion sector shows the negative impacts of conspicuous consumption on our socioenvironmental wellbeing. Despite citizens’ growing awareness of their responsibility within consumption cycles, they reveal concerns about their lack of understanding and the support required for them to become agents of responsible consumption. The Circular Economy flourishes as a conceptual approach to help society transition to a more sustainable existence. This paper explores how emerging creative technology and interaction design might support a shift in the role of citizens in the Circular Economy. We performed a design inquiry that investigated the moment of acquisition via configuration of products, storytelling, and multimodal interaction techniques for the creation of experiences that could catalyse citizen-consumers to become custodians of materials. We developed a retail-based concept tool—The Compositor Tool—with which we ran a user study to investigate new experiential ways that consumers can participate in materials’ circularity. The study highlighted how experience design and new interaction techniques can introduce circularity as part of consumer experience by forging deeper connections between people and products/materials and enabling consumers to have more creative and informative material engagement. Full article
(This article belongs to the Special Issue Creative Methods and Tools for Multimodal Technologies)
Show Figures

Figure 1

20 pages, 2695 KiB  
Article
Creative Probes, Proxy Feelers, and Speculations on Interactive Skin
by Carey Jewitt, Ned Barker and Lili Golmohammadi
Multimodal Technol. Interact. 2022, 6(4), 22; https://doi.org/10.3390/mti6040022 - 26 Mar 2022
Cited by 4 | Viewed by 2671
Abstract
This paper critically discusses the combination of creative and social research methods to generate a novel approach to explore the multimodal technoscape. This paper draws on an interdisciplinary exploratory case study on interactive skin—an emergent technology that augments and/or interacts with the skin. [...] Read more.
This paper critically discusses the combination of creative and social research methods to generate a novel approach to explore the multimodal technoscape. This paper draws on an interdisciplinary exploratory case study on interactive skin—an emergent technology that augments and/or interacts with the skin. This paper shows how concepts from skin studies and the HCI literature can be used to draw on creative methods to think about and with the body. We describe the use of an online probe pack, a speculative research workshop and sensory research interviews using ‘proxy feelers’ to agitate the design space of interactive skin futures. We show how combining these methods provoked and expanded the scope of interactive skin from the technological to the sensory and the social. We discuss the opportunities and challenges of the research dialogues that this approach facilitated, make the case for creative methodological improvisation and exploration of emergent technologies and show how creative and social research methods can be combined to explore the interconnection between technology, society and design. Full article
(This article belongs to the Special Issue Creative Methods and Tools for Multimodal Technologies)
Show Figures

Figure 1

Back to TopTop