An Embodied Approach to Designing Meaningful Experiences with Ambient Media
2. Theoretical Foundations
2.1. Embodied Cognition and Embodied Interaction
2.2. Embodiment in Cognitive Linguistics and Design Research
2.3. Experience with Ambient Media
3.1. Case Analyses
3.2. Empirical Study on the Piano Staircase
4. Designing Meaningful Experience with Ambient Media
4.1. Pilot Study
- Incorporating the installation into an everyday public environment.
- Eliciting unexpected experience by designing input–output relations.
- Coupling the audience’s bodily actions with sensory feedbacks.
- Creating embodied metaphorical mappings and considering the congruency between spatial structures (embodied schemas) and everyday bodily actions.
4.2. Design Ideation
4.3. Ideation Analyses
4.3.1. Analysis Framework
4.3.2. Overall Analysis Results
5.1. Understanding and Designing Embodied Experience
5.2. Embodied Interaction with Ambient Media
5.3. Implications for Design
Conflicts of Interest
- Gibson, J.J. The Ecological Approach to Visual Perception; Houghton Mifflin: Boston, MA, USA, 1979. [Google Scholar]
- Wilson, M. Six views of embodied cognition. Psychon. Bull. Rev. 2002, 9, 625–636. [Google Scholar] [CrossRef] [PubMed]
- Gibbs, R.W.; Wilson, N.L. Bodily action and metaphorical meaning. Style 2002, 36, 524–540. [Google Scholar]
- Antle, A.N.; Corness, G.; Bevans, A. Springboard: Designing image schema based embodied interaction for an abstract domain. In Whole Body Interaction; Karat, J., Vanderdonckt, J., Eds.; Springer: London, UK, 2011; pp. 7–18. [Google Scholar]
- Bilda, Z.; Edmonds, E.; Candy, L. Designing for creative engagement. Des. Stud. 2008, 29, 525–540. [Google Scholar] [CrossRef]
- Chow, K.K.N.; Harrell, D.F.; Wong, K.Y.; Kedia, A. Provoking Imagination and Emotion through a Lively Mobile Phone: A User Experience Study. Interact. Comput. 2016, 28, 451–461. [Google Scholar] [CrossRef]
- Hurtienne, J. How Cognitive Linguistics Inspires HCI: Image Schemas and Image-Schematic Metaphors. Int. J. Hum.-Comput. Interact. 2017, 33, 1–20. [Google Scholar] [CrossRef]
- Van Rompay, T.; Hekkert, P.; Saakes, D.; Russo, B. Grounding abstract object characteristics in embodied interactions. Acta Psychol. 2005, 119, 315–351. [Google Scholar] [CrossRef] [PubMed]
- Loke, L.; Robertson, T. Moving and making strange: An embodied approach to movement-based interaction design. ACM Trans. Comput.-Hum. Interact. 2013, 20, 7. [Google Scholar] [CrossRef]
- Hassenzahl, M.; Diefenbach, S.; Göritz, A. Needs, affect, and interactive products—Facets of user experience. Interact. Comput. 2010, 22, 353–362. [Google Scholar] [CrossRef]
- Hulten, B. Sensory marketing: The multi-sensory brand-experience concept. Eur. Bus. Rev. 2011, 23, 256–273. [Google Scholar] [CrossRef]
- Pucillo, F.; Cascini, G. A framework for user experience, needs and affordances. Des. Stud. 2014, 35, 160–179. [Google Scholar] [CrossRef]
- Dreyfus, H.L. The current relevance of Merleau-Ponty’s phenomenology of embodiment. Electron. J. Anal. Philos. 1996, 4, 1–16. [Google Scholar]
- Varela, F.; Thompson, E.; Rosch, E. The Embodied Mind: Cognitive Science and Human Experience; MIT Press: Cambridge, MA, USA, 1991. [Google Scholar]
- Merleau-Ponty, M. Phenomenology of Perception; Routledge & Kegan Paul: London, UK, 1962. [Google Scholar]
- Dourish, P. Where the Action Is: The Foundations of Embodied Interaction; MIT Press: Cambridge, MA, USA, 2001. [Google Scholar]
- Bakker, S.; Antle, A.N.; Van Den Hoven, E. Embodied metaphors in tangible interaction design. Pers. Ubiquitous Comput. 2012, 16, 433–449. [Google Scholar] [CrossRef]
- Garde-Perik, E.V.D.; Offermans, S.; Boerdonk, K.V.; Lenssen, K.M.; Elise, V.D.H. An analysis of input-output relations in interaction with smart tangible objects. ACM Trans. Comput.-Hum. Interact. 2013, 20, 9. [Google Scholar] [CrossRef]
- Svanæs, D. Interaction design for and with the lived body: Some implications of merleau-ponty’s phenomenology. ACM Trans. Comput.-Hum. Interact. 2013, 20, 1–30. [Google Scholar] [CrossRef]
- Hornecker, E.; Marshall, P.; Hurtienne, J. Locating Theories of Embodiment along Three Axes: 1st–3d Person, Body-Context, Practice-Cognition. In Proceedings of the Workshop Position Paper for CHI 2017 Workshop on Soma-Based Design Theory, Denver, CO, USA, 6–11 May 2017. [Google Scholar]
- Johnson, M. The Body in the Mind: The Bodily Basis of Meaning, Imagination, and Reason; University of Chicago Press: Chicago, IL, USA, 1987. [Google Scholar]
- Lakoff, G.; Johnson, M. Metaphors We Live By; University of Chicago Press: Chicago, IL, USA, 1980. [Google Scholar]
- Talmy, L. Force dynamics in language and cognition. Cognit. Sci. 1988, 12, 49–100. [Google Scholar] [CrossRef]
- Talmy, L. Toward a Cognitive Semantics; MIT Press: Cambridge, MA, USA, 2000; Volume 2. [Google Scholar]
- Evans, V.; Green, M. Cognitive Linguistics: An Introduction; Edinburgh University Press: Edinburgh, UK, 2006. [Google Scholar]
- Lakoff, G. Explaining Embodied Cognition Results. Top. Cognit. Sci. 2012, 4, 773–785. [Google Scholar] [CrossRef] [PubMed]
- Chow, K.K.N. Investigating User Interpretation of Dynamic Metaphorical Interfaces. In Proceedings of the International Conference of Design, User Experience, and Usability, Vancouver, BC, Canada, 9–14 July 2017. [Google Scholar]
- Hurtienne, J.; Israel, J.H. Image Schemas and Their Metaphorical Extensions—Intuitive Patterns for Tangible Interaction. In Proceedings of the 1st international conference on Tangible and embedded interaction, TEI ′07, Baton Rouge, LA, USA, 15–17 February 2007. [Google Scholar]
- Tan, L.; Chow, K.K.N. An embodied interaction framework for facilitating audience experience with ambient media. In Proceedings of the Fourth International Conference on Design Creativity (4th ICDC), Atlanta, GA, USA, 2–4 November 2016. [Google Scholar]
- Myachykov, A.; Scheepers, C.; Fischer, M.H.; Kessler, K. TEST: A tropic, embodied, and situated theory of cognition. Top. Cognit. Sci. 2014, 6, 442–460. [Google Scholar] [CrossRef] [PubMed]
- Shapiro, L. Embodied Cognition; Routledge: New York, NY, USA, 2010. [Google Scholar]
- Oxford-Dictionaries. Ambient. 2017. Available online: https://en.oxforddictionaries.com (accessed on 21 November 2017).
- Tan, L.; Chow, K.K.N. Facilitating Meaningful Experience with Ambient Media: An Embodied Engagement Model. In Proceedings of the Fifth International Symposium of Chinese CHI, Guangzhou, China, 8–9 June 2017. [Google Scholar]
- Lugmayr, A. Connecting the real world with the digital overlay with smart ambient media—Applying Peirce’s categories in the context of ambient media. Multimedia Tools Appl. 2012, 58, 385–398. [Google Scholar] [CrossRef]
- Vatavu, R.D. Enhancing human-human interactions through emotional responsive ambient media. In Proceedings of the 2nd Workshop on Semantic Ambient Media Experience—SAME, Salzburg, Austria, 31–31 October 2009. [Google Scholar]
- Krueger, M.W. Responsive environments. In Proceedings of the National Computer Conference, Dallas, TX, USA, 13–16 June 1977. [Google Scholar]
- Alves Lino, J.; Salem, B.; Rauterberg, M. Responsive environments: User experiences for ambient intelligence. J. Ambient Intell. Smart Environ. 2010, 2, 347–367. [Google Scholar]
- Heft, H. Affordances and the body: An intentional analysis of Gibson’s ecological approach to visual perception. J. Theory Soc. Behave. 1989, 19, 1–30. [Google Scholar] [CrossRef]
- Norman, D. The Design of Everyday Things; Basic Books: New York, NY, USA, 1988. [Google Scholar]
- Bianchi-Berthouze, N. Understanding the role of body movement in player engagement. Hum.-Comput. Interact. 2013, 28, 40–75. [Google Scholar]
- Desmet, P.; Hekkert, P. The basis of product emotions. In Pleasure with Products, beyond Usability; Green, W., Jordan, P., Eds.; Taylor & Francis: London, UK, 2002; pp. 60–68. [Google Scholar]
- Van Rompay, T.; Ludden, G. Types of Embodiment in Design: The Embodied Foundations of Meaning and Affect in Product Design. Int. J. Des. 2015, 9, 1–11. [Google Scholar]
- Brakus, J.J.; Schmitt, B.H.; Zarantonello, L. Brand experience: What is it? How is it measured? Does it affect loyalty? J. Market. 2009, 73, 52–68. [Google Scholar] [CrossRef]
- Hespanhol, L.; Tomitsch, M. Strategies for Intuitive Interaction in Public Urban Spaces. Interact. Comput. 2015, 27, 311–326. [Google Scholar] [CrossRef]
- Wouters, N.; Downs, J.; Harrop, M.; Cox, T.; Oliveira, E.; Webber, S.; Vande Moere, A. Uncovering the honeypot effect: How audiences engage with public interactive systems. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems, Brisbane, Australia, 4–8 June 2016. [Google Scholar]
- Groenewald, T. A phenomenological research design illustrated. Int. J. Qual. Methods 2004, 3, 42–55. [Google Scholar] [CrossRef]
- Creswell, J.W. Qualitative Inquiry and Research Design: Choosing among Five Approaches; Sage Publications: Thousand Oaks, CA, USA, 2007. [Google Scholar]
- Dalsgaard, P. Pragmatism and Design Thinking. Int. J. Des. 2014, 8, 143–155. [Google Scholar]
- Jonas, W. Research through design is more than just a new form of disseminating design outcomes. Constr. Found. 2015, 11, 32–36. [Google Scholar]
- Zimmerman, J.; Stolterman, E.; Forlizzi, J. An analysis and critique of research through design: Towards a formalization of a research approach. In Proceedings of the 8th ACM Conference on Designing Interactive Systems, Aarhus, Denmark, 16–20 August 2010. [Google Scholar]
- Gaver, W.; Höök, K. What Makes a Good CHI Design Paper? Interactions 2017, 24, 20–21. [Google Scholar] [CrossRef]
- Guest, G.; MacQueen, K.M.; Namey, E.E. Applied Thematic Analysis; Sage: Los Angeles, CA, USA, 2012. [Google Scholar]
- Tan, L.; Chow, K.K.N. Piano Staircase: Exploring Movement-based Meaning Making in Interacting with Ambient Media. In Proceedings of the IFIP Conference on Human-Computer Interaction, Mumbai, India, 25–29 September 2017. [Google Scholar]
- DDB. Piano Staircase. 2009. Available online: http://www.thefuntheory.com/piano-staircase (accessed on 28 April 2017).
- Hartson, R. Cognitive, physical, sensory, and functional affordances in interaction design. Behav. Inf. Technol. 2003, 22, 315–338. [Google Scholar] [CrossRef]
- Zimmerman, J.; Forlizzi, J.; Evenson, S. Research through design as a method for interaction design research in HCI. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, San Jose, CA, USA, 30 April–3 May 2007. [Google Scholar]
- Johnson, M. The philosophical significance of image schemas. In From Perception to Meaning: Image Schemas in Cognitive Linguistics; Hampe, B., Grady, J.E., Eds.; Walter de Gruyter: Berlin, Germany, 2005; pp. 15–33. [Google Scholar]
- UN. The Global Goals. 2015. Available online: www.globalgoals.org (accessed on 20 September 2017).
- Ju, W.; Takayama, L. Approachability: How people interpret automatic door movement as gesture. Int. J. Des. 2009, 3, 1–10. [Google Scholar]
- Hey, J.; Linsey, J.; Agogino, A.M.; Wood, K.L. Analogies and metaphors in creative design. Int. J. Eng. Educ. 2008, 24, 283–294. [Google Scholar]
- Madsen, K.H. A guide to metaphorical design. Commun. ACM 1994, 37, 57–62. [Google Scholar] [CrossRef]
- Hutter, K. Unusual location and unexpected execution in advertising: A content analysis and test of effectiveness in ambient advertisements. J. Market. Commun. 2015, 21, 33–47. [Google Scholar] [CrossRef]
- Liang, R.H. Designing for unexpected encounters with digital products: Case studies of serendipity as felt experience. Int. J. Des. 2012, 6, 41–58. [Google Scholar]
|Embodied Schemas and Spatial Features||Container||Path||Orientation |
(e.g., Blockage, Gravitation)
|Criteria||Rubric for Analysis||Level|
|Metaphorical mapping||Level 3: Embodied metaphor and visual metaphor are both constructed in the interaction process. |
Level 2: Embodied metaphor is constructed in the interaction process.
Level 1: Visual metaphor is constructed in the interaction process.
|Bodily engagement||Level 3: Participating in motor interaction through the whole body. |
Level 2: Participating in motor interaction through parts of the body (e.g., arms, hands).
Level 1: Observing a situation by standing or sitting.
|Unexpectedness||Level 3: All strategies are involved. |
Level 2: Two of the strategies are involved.
Level 1: One of the strategies is involved.
Three strategies for triggering unexpectedness: (1) Contrasting input/output (I/O) relations: When the audience performs an action, the output does not match the audience’s expectation (e.g., twisting a shoe results in the distortion of a building). (2) Contrasting physical and virtual elements: Digital media is embedded in the physical space, which leads to a mismatch between tangible reality and virtual reality (e.g., after a physical door is opened, a digital image emerges to show the scene from another city. (3) Visual elements mismatching the content or context (e.g., stairs resembling the shape of piano keys elicit a surprise.)
|Public space||Level 3: The work is situated in an outdoor public space. |
Level 2: The work is situated in an indoor public space.
Level 1: The work is situated in an indoor semi-public space.
|Idea 2: Reading Door|
|Metaphorical mapping||The action of opening the door simultaneously triggers a sound like flipping book pages, and the sensory-motor interaction as a source domain is metaphorically mapped onto reading a book (target domain), which constructs a metaphor that opening a physical space is like entering a knowledge space. The door is decorated like a book cover, which is a visual metaphor.||3|
|Bodily engagement||Whole body is engaged in the interaction with the Reading Door.||3|
|Unexpectedness||Two strategies are used: The door resembles the visual features of a book cover and a hidden speaker plays a sound of opening book pages as feedback.||2|
|Public space||The entrance of the library is an outdoor public space.||3|
© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Tan, L.; Chow, K.K.N. An Embodied Approach to Designing Meaningful Experiences with Ambient Media. Multimodal Technol. Interact. 2018, 2, 13. https://doi.org/10.3390/mti2020013
Tan L, Chow KKN. An Embodied Approach to Designing Meaningful Experiences with Ambient Media. Multimodal Technologies and Interaction. 2018; 2(2):13. https://doi.org/10.3390/mti2020013Chicago/Turabian Style
Tan, Liang, and Kenny K. N. Chow. 2018. "An Embodied Approach to Designing Meaningful Experiences with Ambient Media" Multimodal Technologies and Interaction 2, no. 2: 13. https://doi.org/10.3390/mti2020013