Next Article in Journal
A Study on the Use of Eye Tracking to Adapt Gameplay and Procedural Content Generation in First-Person Shooter Games
Previous Article in Journal
A Review of Heritage Building Information Modeling (H-BIM)
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

An Experience-Centered Framework for Designing Non-Task-Oriented Embodied Interaction Environments

Laura Malinverni
1,* and
Marie-Monique Schaper
Department of Visual Arts and Design, Faculty of Fine Arts, University of Barcelona, 08007 Barcelona, Spain
Department of Information and Communication Technologies, Universitat Pompeu Fabra, 08002 Barcelona, Spain
Author to whom correspondence should be addressed.
Multimodal Technol. Interact. 2018, 2(2), 22;
Submission received: 15 March 2018 / Revised: 25 April 2018 / Accepted: 3 May 2018 / Published: 6 May 2018


Embodied Interaction faces designers with the challenge of thinking about users and interaction from different viewpoints with respect to traditional technologies. This task is even more complex when designing non-task oriented systems. We propose a framework to guide researchers in thinking and designing non-task-oriented Embodied Interaction Environments or, in other words, embodied experiences that users can enjoy for its own sake and not as means for accomplishing a task or achieving an extrinsic goal. The framework is grounded on experience-centered design approaches and will present four qualities ((1) Spatial, Corporeal and Material Consistency, (2) Contingent Enhancement, (3) Mindful Embodied Engagement and (4) Situated Reflexivity) aimed at providing critical lenses, strategies and techniques to guide the design and research processes. Finally, we will discuss how designers can implement the proposed framework in different stages of the design process and paths for future research.

1. Introduction

The interest in including a wider range of human abilities in the interaction with computers [1] and the availability of novel technological solutions have strongly diversified the panorama of Human-Computer Interaction. In this context, the pivotal work of Dourish [2] defined the concept of Embodied Interaction as “the use of the physical world to interact with digital technologies”. This innovative approach set a research agenda aimed at gradually moving away from the dominant paradigm of separating computation and physicality to merge digital technology with sociomaterial culture.
Building on this framework, several novel interaction modalities have been proposed, such as: Tangible Interaction [3], Gesture-Based Interaction [4], Full-Body Interaction [5], etc. These approaches have gained an increasing relevance in the design community thanks to their potential benefits such as: enabling conditions for thinking and learning by doing [6,7], allowing offloading cognition in the environment [6], facilitating collaboration [8] and involving users at different levels (i.e., sensorimotor experience, cognitive aspects, affective factors [9]). As a consequence, Embodied Interaction Environments (EIEs) have met fertile terrains of application in different fields such as health, education, computer-supported cooperative work, digital heritage, somatic awareness, performance, etc.
Nonetheless, even if this conceptual and technological shift is gradually reshaping the panorama of Human Computer Interaction (HCI), designing for Embodied Interaction entails its own set of challenges. Designers need to think about users and interaction from different viewpoints with respect to traditional technologies and can no longer rely on the models, tools and methods that guided the design of desktop-based systems and Graphical User Interfaces (GUI).
To help designers in this process, several researchers [6,7,10,11,12] have proposed specific frameworks to design for Embodied Interaction. This research has generally been oriented toward identifying key themes and areas of application for Embodied Interaction (e.g., learning, collaboration, etc.), hence focusing on a clearly goal-oriented and task-oriented approach. Instead, even if an increasing number of researchers [13,14] are starting to explore the possibilities of Embodied Interaction for non-task-oriented systems, existing design frameworks tend to overlook this research area. As Harrison et al. [15] pointed out, non-task oriented systems tend to poorly fit with approaches that are formalized in terms of tasks, goals and efficiency, hence requiring for alternative frames of reference [16].
Starting from this perspective, we propose a framework to guide researchers in thinking and designing non-task-oriented EIEs. The proposed framework is mainly grounded on experience-centered design approaches [17] and aims at specifying this knowledge in the context of non-task-oriented EIEs. Its goal is not to offer prescriptive guidelines, but to provide designers with instruments and critical lenses to guide their design and research processes. To this end, we structured the framework as a set of four qualities ((1) Spatial, Corporeal and Material Consistency, (2) Contingent Enhancement, (3) Mindful Embodied Engagement and (4) Situated Reflexivity) aimed at shedding light on different aspects of designing for non-task-oriented EIEs.
In the next sections, we will first overview the existing frameworks for designing for Embodied Interaction. Subsequently, we will present the conceptual underpinnings of the proposed framework and its constituting set of qualities. Finally, we will discuss how designers and researchers can implement the proposed framework in different stages of the design process and paths for future research.

2. Frameworks to Design for Embodied Interaction

Designing for Embodied Interaction faces designers with the need for thinking about users, technologies and interaction from different viewpoints with respect to traditional technologies (e.g., GUI or WIMP based system). To support this process, several researchers have proposed specific design frameworks and guidelines. These frameworks pursue different goals such as: suggesting possible uses of Embodied Interaction; making the affordances of these novel technologies visible; allowing designers to focus on relevant aspects; guiding design decisions, etc.
For instance, Levisohn and Schiphorst [18] and Fogtmann et al. [12] propose taxonomies to distinguish the approaches and purposes that Embodied Interaction environments may have. Specifically, Levisohn and Schiphorst [18] distinguish between two complementary research agenda in Embodied Interaction. The first one, labelled as “movements for interaction”, aims at investigating how to use bodily interactions to facilitate certain tasks or to support meaning construction processes. The second one, defined as “movement as an aesthetic and felt-experience”, aims at designing systems that promote somatic awareness or expressive movements.
Similarly, Fogtmann et al. [12] identify a set of key themes in Embodied Interaction. Specifically, they distinguish between: (1) systems aimed at using Embodied Interaction to improve bodily skills (e.g., exertion interfaces); (2) systems that use Embodied Interaction as a mean for something else (e.g., supporting learning) and (3) systems that use Embodied Interaction to challenge our senses and generate novel experiences. These taxonomies, even if from different viewpoints, indicate that Embodied Interaction can either be employed to facilitate an extrinsic task (e.g., support cooperative work) or to design environments where the experience itself is the end goal.
Within this distinction, several scholars [6,7,11,19] have proposed frameworks that analyze how Embodied Interaction can be used to facilitate specific tasks or to achieve extrinsic goals. For instance, Klemmer et al. [7] discuss the affordances that Embodied Interaction can offer to support thinking and learning by doing, performing specific tasks or collaborative activities, and risk-taking. Hornecker and Buur [11], further specify these reflections and propose a framework that focuses on analyzing how Tangible Interfaces can support cooperative work and collaborative learning. Specifically, the authors point out that central themes such as tangible manipulation, spatial interaction, embodied facilitation and expressive representation can facilitate co-construction of knowledge and collaboration. Similarly, Antle [6] contextualizes this knowledge in the field of designing for children and overviews the affordances that Embodied Interaction offers for learning (e.g., allowing offloading cognition in the world; using movements to help children thinking), Finally, Abrahamson et al. [19] focus on proposing a framework to use Embodied Interaction to support learning of mathematical concepts.
From a slightly different perspective, Jacob et al. [10] describe the design implications that can be extrapolated by exploring how people use their body. The Reality-based interaction framework [10] focuses on the role of bodily-knowledge and points out that our bodily experiences can be useful to support the understanding of certain concepts (e.g., naïve physics understanding of the physical world, the awareness of one’s own body and physical skills, social understanding, etc.). Starting from this analysis, they hence derive possible fields of application for Embodied Interaction technologies.
Finally, from a strongly applied perspective, Deng et al. [20] and Melcer et al. [21] propose design frameworks oriented toward highlighting relevant concepts that designers need to take into account when designing EIEs for learning. Deng et al. [20] elaborate a collection of cards (Tango Cards) to design tangible learning games. Specifically, they describe a series of key concepts about learning and interaction design (e.g., multiple modalities, coherent mapping, well-ordered problems, intrinsic rewards, etc.). Melcer et al. [21], instead, propose a taxonomy that focuses on interaction design (e.g., definition of the physical interaction, of the mapping strategy and of the multiuser modality) to guide researchers in taking design decisions to create embodied learning games.
On the other hand, the raising of novel approaches to HCI (e.g., third-wave HCI [15]) promoted an emerging interest toward exploring the research on movement as an aesthetic and felt-experience instead of concentrating on its task-oriented utility [18]. For instance, Svanaes [22] uses Merleau-Ponty’s analysis of perception and felt experience to derive possible design implications. Specifically, the author points out how embodied perception (i.e., human-artifacts interactions whose nature is perceptual and bodily) and kinesthetic creativity (i.e., use the body to think about future solutions) can constitute two crucial resources to inform design. Within this context, two possible research lines can be identified. On the one hand, EIEs which focus on non-task oriented computing (e.g., ambient interfaces) and do not aim at specific tasks or goals but at designing meaningful users experiences [13,14]. On the other, research aimed at exploring movement-based practices (e.g., somatic and dance) as methods for investigating and designing EIEs [23,24,25,26]. Nonetheless, as Levisohn and Schiphorst [18] pointed out, this research is still in its early stage, hence requiring further explorations.

3. A Framework for Non-Task-Oriented Embodied Interaction Environments

Embodied Interaction offers a rich set of affordances which make it suitable both for designing task-oriented environments as well as for non-task-oriented computing [15]. While most design frameworks for Embodied Interaction have focused on task-oriented domains, our purpose is to delineate a framework to guide researchers in thinking and designing non-task-oriented EIEs.
The proposed framework derives from: our previous research on Embodied Interaction and Participatory Design [27,28,29,30]; our teaching experiences in HCI; and our works as multimedia artists. The variety of scopes and contexts of this background allowed us to methodologically situate our approach and to identify relevant considerations and concepts to guide designers in this field.
On the one hand, in previous studies, we focused on researching children’s understandings and behavior in learning environments based on Embodied Interaction. This research showed us the primacy of sensorimotor explorations as means to make sense and enjoy the experience. Furthermore, the critical analysis of own works [31,32] showed us the tensions that can emerge between pursuing a specific learning goal and designing rich embodied experiences (e.g., focusing excessively on the learning contents and relegate embodied experience to a secondary role). These considerations motivated our interest in designing experiences that are sensorially interesting per se and not as means for something else. This standpoint brought us closer to a view of Embodied Interaction that shares some affinities with the approaches that research movement as an aesthetic and felt-experience [18]. Specifically, as for research on aesthetic and felt-experience, our approach does not focus on looking at the utility of movements-based interactions but, instead, it aims at helping designers in offering embodied experiences that users can enjoy for its own sake.
Second, both our research on EIE as well as our works as artists brought us closer to a view akin to experience-centered design approaches [17]. Specifically, our framework builds on a holistic perspective on experience, considering it as an aesthetically, sensual, emotional, intellectual and relational significant fact that includes both the experienced and the experiencing and emerges from “complete interpenetration of the self and the world of objects and events” [33].
Finally, our experience in leading Participatory Design workshops and in teaching HCI, showed us the importance of putting “experience-first” in the design process. Specifically, we noticed that both adults and children came out with better design ideas for Embodied Interaction when they use methods and techniques oriented toward investigating the phenomenological and sensual dimensions of experience instead of starting the design from abstract concepts [28,34]. These observations, again, strengthen our position in favor of phenomenological approaches to EIEs.
However, our standpoint also present subtle differences with respect to the aforementioned approaches. On the one hand, while sharing the conceptual underpinnings of experience-centered design approaches [17], we aim at specifying this knowledge in the context of non-task-oriented EIEs. On the other, while most research on the aesthetic and felt dimensions of embodied interaction has generally concentrated on the phenomenological aspects of bodily movements, our goal is to broaden this scope by focusing on experience as the relational entanglement and interpenetration between the self and the world of objects and events. This idea derives from the relational onto-epistemologies proposed by Overton [35] and Barad [36]. Specifically, Overton [35] propose a relational notion of embodiment that overcomes the idea of embodiment as simply related to the physical structures of our bodies (and their cognitive roles) and instead includes the body as a lived experience engaged with the world of sociocultural objects, where the identity of objects and events derives from the relational context in which they are embedded [36]. From a slightly different perspective, also Barad [36] defends the idea that objects and subjects are ontologically inseparable and that the properties and boundaries of an entity do not pre-exist to a phenomenon but emerge from the intra-actions that take place within the phenomenon. Furthermore, the author, by taking a post-humanist standpoint, defends the role of materiality as having an agency and being an active actor in the world.
These views, in the context of Embodied Interaction, require, first to overcome the idea of users as actors separated from the materiality of the EIE, second to acknowledge the agency of materiality and, third, to consider experiences in EIE as phenomenon were the users and the environment codefine themselves. Starting from this perspective, we define Non-task-oriented Embodied Interaction Environments (NOEIEs) as environments that aim at using Embodied Interaction to allow embodied experiences of entanglement with the physical-digital setting that are aesthetically, sensually, emotionally and intellectually significant and complete for its own sake.
In order to translate this viewpoint into an applied instrument, we propose a set of qualities that we consider relevant to guide the design of NOEIEs. The goal of the proposed qualities is not to offer a prescriptive approach to design or engineer user experience [17]. Instead, they aim at functioning as a generative tool to provide designers with concepts to reflect on their own practice, making distinctions and asking relevant questions in the design of environments where significant users’ experiences can take place.
We elaborated these qualities using an interdisciplinary approach that merges knowledge proceeding from different fields (developmental psychology, cognitive science, HCI, semiotic, art, theatre and somatic practices). In the paper, we will present the following qualities: (1) Spatial, Corporeal and Material Consistency, (2) Contingent Enhancement, (3) Mindful Embodied Engagement and (4) Situated Reflexivity. The first two qualities address the design of the environment where the experience will take place and are mainly concentrated on the sensory and bodily engagement with it. The third and the fourth qualities, instead, focus on aspects that emerge in the relation between the users and the environment and on the way in which users can connect, interpret and reflect through and with the environment.
We will frame each quality according to a strongly applied perspective. Specifically, we will first introduce its conceptual definition and provide explanatory examples. Subsequently, we will discuss eventual risks and strategies to explore this quality in the design process and provide case studies of their use in different stages of our design practice.

3.1. Spatial, Corporeal and Material Consistency

Often, in the interaction with digital technologies, we may end up considering the physical interface and the interaction modality only for their instrumental and functional roles (e.g., when typing a text, we do not really focus on our relationship with the keyboard). Within our framework, we consider that the design of NOEIEs should start from the experience that we can experiment by physically, emotionally and intellectually relating to a specific setting or object. This requires designers to put their embodied experiences at stake in the design process and to consider the spatial configuration, the materials and the bodily movements as creative resources that hold the potential to give access to certain experiences, convey meanings and evoke feelings. Starting from this perspective, we propose the quality of spatial, corporeal and material consistency as the degree to which the design of the spatial, corporeal and material features of a NOEIE offers a coherent fit with the experience that we want to support.

3.1.1. Designing for Spatial, Corporeal and Material Consistency

Designing for spatial, corporeal and material consistency deals with making concrete design decisions about the material features of the EIE (e.g., How is the space? What are the employed materials? How do users interact with them? etc.). This implies that both the physical interface and the ways in which users can interact should have the appropriate aesthetic qualities to evoke the experiences that the designed environment aims at supporting.
For instance, it is not the same to play a video game using a classical joystick or using a game controller based on a wearable soft jacket that requires massaging another person to play [37]. At the same time, it is not the same to play a shooter game with a gaming gun or with an interface such as SweetPads [38] that requires players to slowly and gently touch a spherical interface to kill. Similarly, users can have radically different experiences if we change the scale an interactive environment (e.g., be in a huge interactive space or in a small interactive cocoon), its materials (e.g., manipulating a hard plastic prop or a soft textile prop) or their ways of moving through it (e.g., requiring fast-paced actions or slow and small movements).
These examples, by keeping the digital content unaltered while changing our ways of physically interacting with it, exemplify the power of the spatial, corporeal and material features in making accessible some nuances of meaning that are deeply grounded in our felt and embodied experience. Furthermore, they shed light on the performativity of the material features of the physical interface, or in other words, on their capacity to act as agents that construct a specific experience or discourse.
HCI researcher have tackled the analysis of the performative roles of spatiality and materiality from different perspectives. From a theoretical perspective, Larssen et al. [39] had pointed out the importance of the feel dimension and of the kinesthetic perception that is established in the dialogue between users and objects while interacting with tangible interfaces. Similarly, Jacucci & Wagner [40] showed that physical materials and spaces, thanks to their properties (e.g., texture, geometry, material, energy, etc.) offer a series of communicative, referential, evocative and engaging opportunities, whose richness is impossible to be replicated in virtual environments. Complementary, emerging research on novel interfaces such as shape-changing interfaces [41] and ephemeral interfaces [42] are opening rich opportunities for material, spatial and corporeal experimentation in HCI.

3.1.2. Material, Corporeal and Spatial Consistency in the Design Process: Risks and Opportunities

To take advantage of materiality, corporeality and spatiality, we need to address some considerations about the design process. On the one hand, it may be the case that budget and/or requirements constraints can limit the possibilities for exploring different materials and spatial configurations. On the other, often the habits of interaction design and the way in which the design process sometimes unfolds may end up playing against the possibility of more carefully taking into account spatial, material and corporeal features.
For many years, interaction design tended to privilege the digital and the visual rather than the physical [43]. Traditional training programs for interaction designers generally tended to focus on graphic design, information architecture, wireframe elaboration, etc. while the physicality of the interface was considered as something given (e.g., screen-based interaction). At the same time, the design process often follows a top-down approach that starts with intellectual conceptualizations to later find a specific solution. Furthermore, this process is generally built around logo-centric and/or graphical modes of expression, which rarely involves the body and the sensual experience. As a consequence, the design of NOEIEs may face several risks.
First, the tendency to privilege the digital over the physical may lead to approaches that, instead of taking advantage of the richness of embodied experiences, end up employing bodily interactions only as strategies to sugarcoat certain tasks (e.g., the user has to run or jump to answer some questions) or as replicas of traditional interaction paradigms (e.g., using Embodied Interaction to emulate the mouse-cursor dyad). Second, when defining the physical interface, often designers end up employing well-known configurations (e.g., vertical screen or floor projection in Full-body Interaction [5], tangible tabletops in Tangible User Interfaces) without further or more deeply exploring different design possibilities. Third, top-down design approaches may end up focusing mainly on intellectual knowledge while neglecting the knowledge that arises and is part of the embodied experiences. Finally, the prominence of a logo-centric modus operandi may run the risk of falling in a sort of representationalist trap where designers try to use Embodied Interaction to translate words into experience and experience into words (e.g., establishing a strictly codified correspondence between meaning and form, where the action X means Y). This last risk is particularly critical in the context of NOEIEs since it may end up missing the very nature of embodied experience. As Hickey-Moody et al. [44] point out, embodied experiences are inherently meaningful and do not need to be translated into words to make sense or to be understood. As a consequence, when designing for NOEIEs, practitioners should not try to use embodiment to speak about something that is out of the experience but, instead, they should start from experience to find ways to make it even more relevant or powerful.
From a design perspective, a powerful way to tackle these risks is to put “experience first” in the design process. This means to use materiality, spatiality and corporeality both as the media to realize our thoughts and as the ends of the design process. Research in this field has indicated the effectiveness of using our own bodies to think about design choices. Within our framework, we suggest three possible paths to embrace spatiality, materiality and corporeality in the design process. Namely: legitimating kinesthetic creativity [22], exploring felt-experience [45] and learning from art practices.
Svanaes [22] defines kinesthetic creativity as the use of the body to think about future and novel creative solutions. Conceptually, this notion legitimates the body as a skilled agent in the process of constructing knowledge and supporting creative processes. From an applied perspective, it implies activating and engaging our bodies (and their relation to the environment) in the design process. We can find examples of this approach in practices such as body-storming and embodied elicitation methods [23,25]. Body-storming methods are based on enacting the experiences we want to design for. They generally start by defining a specific scenario and require the design team to role-play this situation. In a similar way, embodied elicitation methods use bodily experiences to stimulate ideas about novel design concepts. For instance, Tomico et al. [46] asked participants to experience different materials on their bodies to think about novel designs. Similarly, Wilde [47] created a series of wearable props to experience and think about future technologies. These techniques, by kick-starting the design process from our embodied experiences can allow both changing the way in which we see and feel our bodies (and, hence, design for them) as well as giving access to concepts or experiences that otherwise may remain opaque.
In a similar way, design processes grounded on exploring felt-experience propose to use the knowledge that arises from bodily practices both to become more aware of our bodies as well as to extrapolate ideas that can be applied to design. Höök et al. [26,38] have carried out a great amount of research in this direction. On the one hand, they explored the use of body-mind practices such as Feldenkrais to approach design from a bodily-centered perspective [45]. On the other hand, Höök used her autoethnographic account about horse riding to derive possible experiential qualities that can be extended to the design of ITC systems [26].
Finally, designers can learn from art practices to design for spatiality, corporeality and materiality. This learning can take different forms. On the one hand, designers can learn from existing artistic strategies. This approach is beautifully described by Munari [48], who took examples from artworks and art movements to suggest possible approaches to support creativity and creative solutions (e.g., changing color, dimensions and materials of familiar objects, researching visual affinities, etc.). In the context of HCI, we can find examples of this approach in the work by Wilde et al. [25], who use the surrealist slogan of “making the ordinary extraordinary” to inspire the idea of using estrangement as an embodied design method. On the other hand, designers can research relevant references from artistic and architectural works to find inspiration about how these disciplines have employed spaces and bodies to create meanings and generate specific feelings (i.e., the scale and richness of thrones or cathedrals to convey the idea of power, the bodily exploration of performative arts).

3.1.3. Case study: Exploring Material, Corporeal and Spatial Consistency through Embodied Design Ideation Techniques

To explore the idea of starting from embodied experience to design for material, corporeal and spatial consistency, we carried out a workshop based on Embodied Design Ideation Techniques [25]. The workshop took place in the house of one of the researchers. We began the activity using a revised version of the technique proposed by Tomico [46]. Each of us picked up one object of the house and explored interesting ways of moving and relating to it. One of us chose a duvet cover and started exploring different ways of covering her body with it to generate different tactile and light conditions, another chose a small trampoline and started experiencing different ways of walking on it, the third chose a pot with a cactus and explored the idea of having “a cactus hand”. The experiences were later experimented by the others to discuss and annotate relevant concepts and ideas. Subsequently, following a simplified version of the autoethnographic account proposed by Höök [26], each of us thought about the qualities of an interesting sensorimotor experience she had have in the last weeks. One of us described the tension between being mindful and patient that she experienced in an exercise where she was asked to eat a grape in 10 minutes; another described the playfulness of walking in a muddy trail and the third reflected on the feeling of protection that she and her baby have when using the baby carrier. These ideas, again, were transcribed on cardboards.
Using the ideas annotated on the cardboards, we discussed the feelings, sensations and qualities we want to embed in the to-be-designed NOEIE. We, therefore, focused on the idea of designing “a uterine space” that can convey feelings related to caring, protection and containment. This idea was particularly inspired by the experience with the duvet cover and by the reflections on the baby carrier. Each of us, individually, drew a sketch of her proposal, detailing the features of the space, the materials and the way in which users could relate to it. All proposals were describing a cocoon-like space (similar to a bathtub), with rounded shapes, made of soft and warm materials. To further define the idea, we started to physically explore available materials that have these properties (e.g., pillow, yoga balls, water and air balloons, plastic bags full of air, quilt, etc.). Within them, we considered that water balloons were offering some fundamental qualities for the to-be-designed experience: the pressure that they exert, their malleability and their possibilities of changing temperature. We, hence, started to physically explore the kind of embodied interactions that they afford. From these experimentations, we elaborated a first design concept (Figure 1) which propose a soft space, where users can lay down and interact with smart water-balloons-like objects (covered with different materials) which slowly respond to users’ interactions through temperature change, color change and sound.

3.2. Contingent Enhancement

Often, a well-designed space, an interesting corporeal movement or a rich material can be subjectively meaningful enough per se. In this case, we would not need any technology to make it a relevant experience capable of being sensually, emotionally and intellectually significant. However, in the context of Embodied Interaction, the mediation offered by digital technologies, in some cases, can serve as a resource to give an added value to these experiences. Within our framework, we consider that digital technologies should allow enhancing and making richer the embodied experience that users can live in a specific environment. Or, in other words, the digital augmentation should be contingent to the feature of the embodied experience we want to support and of the space we are designing for. To this end, we propose the quality of contingent enhancement as the degree to which digital technologies can make some aspects of the embodied experience richer and more interesting.

3.2.1. Design for Contingent Enhancement

The importance of a meaningful coupling between the digital and the physical has been widely addressed in HCI [11,41,42]. Nonetheless, in most cases, research in this field mainly aimed at facilitating the comprehension of the interaction and the usability of the system. Building on this premise, in our framework, we consider that the coupling between the physical and the digital should not only be seen as a strategy to facilitate interaction but should work as a way to use interactive technologies for adding value to the embodied experience that we may live in and through a specific environment.
This perspective requires, on the one hand, to critically analyze whether the use of technology really allows making the embodied experience richer, more interesting or more relevant. On the other hand, it implies taking decisions about the mode through which interactive technologies will respond to the users (e.g., visual, auditory, haptic, etc.), the kind of content that will be offered and the mapping that will connect the users’ actions with the system’s responses. For instance, it can be radically different to just move our legs or to experience their movements with a system such as the one proposed by Françoise et al. [49] that transforms our muscular micro-movements into an auditory feedback. Similarly, we would have completely different experiences if we move in a system that responds to our small movements with subtle sounds or with loud noises. Finally, we would relate to the system in different ways if it responds to gross bodily movements or to small changes in our breath.
The variety of possibilities available in the use of interactive technologies points out that interactive augmentation, together with the spatiality, corporeality and materiality of the environment, acts as a performative agent in constructing specific discourses and experiences. Specifically, it signals relevant design possibilities related to using interactive technologies to shape the ways users enter in contact and understand their bodies, relate to the systems and/or with other users.

3.2.2. Contingent Enhancement in the Design Process: Risks and Opportunities

Designing system’s responses that make the embodied experience richer, more interesting or more relevant is not a trivial task. Often, the legacy of design methods derived from GUI ends up privileging the content over the experience, and the feedback over the intertwined relation that can be established between spatiality, corporeality, materiality and augmentation. As a consequence, designers may fall into the trap of creating environments where the digital augmentation is merely juxtaposed to the material features of the experience (e.g., the digital content do not really exploit the spatial/material features of the environment nor the sensorimotor explorations that can be experienced).
Hence, to design for contingent enhancement, designers should start with an in-depth physical and material understanding of the embodied experience that they want to support. Subsequently, they need to carefully explore the way in which technology can offer novel entry paths to this experience. Interesting possibilities can be found in using interactive technologies as media that allow experiencing embodiment in a different way, either by making visible some features of the experience that are not otherwise accessible or by promoting novel ways of entering in contact with our bodies and our surroundings. Starting from this perspective, we suggest three possible paths for exploring the possibilities of contingent enhancement. We labelled them: making the invisible visible, supporting embodied explorations and generating embodied dissonance.
Making the invisible visible refers to the possibilities of using digital augmentation to play with the features of the embodied experience and make perceptible some of its specificities, meanings or underlying phenomena that are not otherwise evident. This means that the digital augmentation should go along with analogous experience, analyzing what kinds of qualities are embedded in spatiality, corporeality and materiality and providing “what space, materials and sensorimotor experience want”. This exploration can either address the specific formal features of the experience or its underlying connotations. We can find interesting applications of the former case in the digital mapping used to play with the architectonic features of specific buildings or in projects such as the Augmented Reality Sandbox [50]. This project plays with the sensorimotor experience of touching and manipulating the sand to create an augmented digital topographic representation of the mountains and valleys. On the other hand, projects as “Kobito-Virtual Brownies” [51] or “Perfect woman” [52] use digital technology to play with socio-cultural constructions embedded in specific objects or experiences. Specifically, Kobito [51] plays with the fantasy of invisible little creatures to augment autonomous objects’ movements. Instead, “Perfect woman” [52] starts from the relationship between our bodies and our subjectivity to map and project stereotypes related to beauty and gender on the bodies of women.
Another possible path to explore contingent augmentation is using digital technologies as a way to support embodied exploration. This means to use technology to support users exploring novel ways of moving or experiencing their bodies and environment. An analogic example of this opportunity can be found in Jacques Lecoq methods for theatrical training [53], where performers wear portable structures to experiment the relations that can be built with their bodies, objects and spaces. Similarly, other relevant examples can be found in systems aimed at promoting kinesthetic awareness [49] or expanding our sensing capacities. For instance, Schiphorst et al. [54] design the system “Whispers” to promote users paying attention to their physiological state. Similarly, in the artistic installation “When Infinity comes to life”, Cantoni & Crescenti [55] use a system of moving mirrors and platforms to generate novel ways of perceiving and experiencing space and motion. On the other hand, projects such “Improvised Empathetic Device” [56] or “Animal SuperPowers” [57] use interactive technologies to augment our sensing capabilities either by allowing users to physically feel the death of each soldier on their bodies or by providing children with the sensory skills of different animals.
Finally, from a slightly different perspective, designers can explore the possibilities of generating embodied dissonance to enrich technology-mediated embodied experiences. The concept of dissonance derives from the estrangement that a person may feel when she holds two or more contradictory beliefs or ideas [58]. Similarly to the notion of kinesthetic disorder proposed by Fogtmann et al. [12], it suggests the possibilities of using Embodied Interaction to challenge senses and, more specifically, it deals with designing experiences that use digital augmentation to create a certain contradiction between our embodied experience and our digital experience. We can find examples of this strategy in the “Very long Arm Virtual Reality illusion” [59] where users can experiment having an incredibly long arm and in projects such as “You are not here now: New York/Baghdad” [60] where players have to move around New York City using an augmented reality map of Baghdad.

3.2.3. Case study: Using Contingent Enhancement as a (Participatory) Design Concept

During the last years, we have explored the possibility of using the quality of contingent enhancement as a Participatory Design strategy. Specifically, we focused on using digital technology to contingently enhance the features, qualities and meanings of everyday places. We took inspiration from the work of Bruno Munari “Da lontano era un’isola” [61], where he uses the formal features (e.g., lines, shapes, etc.) of stones found in the beach to overlay them with drawings and create short stories. Starting from this concept, we were interested in encouraging participants to look at their everyday places “through the lens of imagination” and use digital technology to play with the material and symbolic features of these spaces.
To this end, we developed a basic solution that combines a pico-projector and an app to create contents and project them on different surfaces. Participants can choose a place, observe its features and create contents that add a “layer of imagination” to the physical world and make visible the invisible life of this place.
To experiment with the possibilities of this proposal, we first created a series of contents to project in the streets of the city in a performative manner. Subsequently, we ran a study in a school and a participatory art project. In the study [62], we asked children to imagine that some fantastic creatures were inhabiting different places of their school. Children had to choose a place that they found particularly interesting, create a fictional character for this space and enact its story using the pico-projector. Similarly, in the participatory art project, we collaborated with people from a neighborhood association to reimagine and redesign different places of their neighborhood. Participants were invited to choose different places they wanted to change and created audiovisual content that could be projected in these places to resignify them.
The children’s and neighbors’ works showed us that the concepts of contingent enhancement and making the invisible visible promoted different ways of looking at everyday places to transform them through digital technologies. For instance, several children played with the fantasy of the invisible creatures and the features of the space to create site-specific narratives that explain or subvert different places of the school (e.g., a group of children animated a monster that messed-up the kitchen’s utensils and for this reason the food was so bad). Similarly, the group of neighbors used digital augmentation to create different layers of temporality in their neighborhood (e.g., overlaying existing places with contents related to their historical memory or projecting future possibilities to change this place). Nonetheless, due the technical limitations of the system, almost all digital augmentations where oriented toward contingently enhance the features of the space, while the sensorimotor experiences were not taken into account directly.

3.3. Mindful Embodied Engagement

The material, spatial, sensorimotor and interactive features characterize an EIE as a complex system where multiple stimuli can potentially attract (or not) the attention and interests of the users. As a consequence, users may get engaged with different aspects of the experience (e.g., focus mainly on exploring the visual contents or the specific tactile properties of a certain object or bodily movements, etc.).
These diverse ways of getting engaged may serve the different goals that EIEs can have (e.g., skill acquisition, somatic awareness, collaborative work, etc.). Nonetheless, for the purpose of our framework, we consider that NOEIEs should allow users to focus and get engaged with the feelings, knowledge and sensations that emerge from their entanglement with the physical-digital environment. Hence, we propose the quality of mindful embodied engagement as the degree to which the system allows getting engaged and focused on exploring our entanglement with the technology-mediated embodied experience.
From a theoretical perspective, the notion of mindful embodied engagement builds on the concept of mindfulness described by Langer et al. [63], the notion of flow by Csikszentmihalyi [64] and the idea of entanglement by Barad [36]. According to Langer et al. [63], mindfulness constitutes a flexible state of mind in which we are actively engaged in the present, sensitive to one’s environment, open to notice new things and to create new categories. Hence, it designates a mindset through which we enter and take advantage of an experience by being intellectually, emotionally and sensorially attentive to it. Similarly, Csikszentmihalyi [64] describes the flow as the state of being completely involved in an activity for its own sake. Finally, Barad [36] explains the idea of entanglement as an ontological perspective according to which we only exist in the relation to our environment and we are inseparable from it.
Building on these constructs, the idea of mindful embodied engagement acquires its specificity by focusing on defining the mindful engagement that users can feel by experiencing their entanglement with the physical-digital EIE. This quality shares some affinities with concepts such as somatic and kinesthetic awareness [18]. Nonetheless, while the former constructs mainly address users engagement with their bodies, in our case, we slightly extend the scope of the concept. Specifically, we suggest that the system should facilitate users to experiment their inseparability with the physical-digital environment and explore how they are mutually constructing each other through relationality.

3.3.1. Design for Mindful Embodied Engagement

Designing for mindful embodied engagement deals with modelling the relations that can be physically, emotionally and intellectually experienced within a NOEIE. As a consequence, it is not specifically concerned with making concrete design decisions on the different elements of the experience (e.g., physical interface, digital augmentation, interaction, etc.) but, instead, it addresses the ways of engaging and relating that can emerge from the network of human and non-human actors that constitute the experience (e.g., users, context, materials, digital augmentation, etc.).
On the one hand, assuming this relational standpoint requires to focus on the links and relations that can be created within this network instead than on the nodes that constitute it. On the other hand, it implies to critically analyze whether and how these relations can help users to experiment and get engaged with their inseparability to the physical-digital environment.
To ground this idea, a useful example can be found in the first author’s experience as a beginner climber on natural rocks or on artificial walls: “I started climbing on natural rocks to follow my passion for hiking and experiencing a new way of living the mountain. Subsequently, I enrolled in a climbing gym to improve my skills since I felt that I was not improving technically. When climbing on natural rocks, if I want to reach the top of the cliff, I need to be constantly focused on the present moment, on the few next steps and on the relationship between myself and the environment. I need a deep embodied understanding of the relation between the rock and my body in order to be sensitive to the subtle differences in the conformation of the environment and adapting my movements consequently. At the same time, I need to be aware of my emotional, physical and intellectual state. If that day I easily get distracted, I know that is not a good day to climb. Similarly, if something provokes me an emotional distress I suddenly get blocked. Conversely, in the second case, climbing on artificial walls does not require me an exclusive concentration of this activity. I can eventually let my thoughts wander and I do not need to focus so deeply on the relationship between my body and the environment. The artificial grips for hands and feet explicitly suggest me what I should grab. I can eventually experiment with different possibilities of bodily movements or I can just enter in a sort of “automatic pilot mode” to train my strength or a certain technique.”
This vignette illustrates the different relationships that are established between the climber and the environment and how these different ways of relating end up either promoting a mindset of mindful embodied engagement (natural rocks) or not (artificial rocks). Furthermore, it points out the fundamental role of subjectivity, personal interests and values as drivers of this quality. As a consequence, designing for mindful embodied engagement inevitably deals with facing complexities and uncertainties that surround and constitute (an) experience.

3.3.2. Mindful Embodied Engagement in the Design Process: Risks and Opportunities

Support a mindful embodied engagement faces practitioners with several challenges, risks and intricacies. First, allowing users to enter in a state of mindful engagement is not something that we achieve only through design. Instead, it also depends on the much richer network of elements and meanings that constitute an experience (e.g., the context where the experience takes place, the interests and predisposition of the users, their specific mindset at that moment, etc.). As a consequence, offering concrete guidelines to design for mindful embodied engagement can be difficult since it may fall into the trap of trivializing complexity in the effort of engineering experiences. Second, sometimes, certain design patterns of interactive technologies may push toward a direction that actually ends up hindering a mindful state of mind. On the one hand, in the broad panorama of ICT, the technology-induced pressure for velocity and multitasking, the need for being always connected and the overload of information have been often associate with the risks of being endlessly distracted, mentally elsewhere [65] and with a reduced ability to sustain focused attention [66]. On the other hand, the hype of gamified solutions may encourage designs where the goal of making something “entertaining” (e.g., making game points, including some amusing element, etc.) runs the risk of adding unnecessary distractors which may divert the attention from the embodied experience per se.
This complex situation characterizes design for mindful embodied engagement as a complex endeavor. Nonetheless, to guide designers in this task, we suggest a set of concepts that can be used as possible paths to explore in one’s own design practice. Namely, we suggest: exploring embodied significance and facilitating a beginner mind.
A first undeniable aspect that makes an experience engaging is whether we consider it is worthy of attention or not. In other words, we do not get engaged or focused on something we do not care about. In this context, we consider embodied significance as the extent to which users consider that an EIE is worth to be experienced. This worthiness may arise from different factors: the curiosity that the technology-mediated experience may trigger, the sensual pleasure that it can provoke, the sensorimotor challenges that it can offer, the playfulness that it can evoke, etc. As designers, a possible way to understand embodied significance resides in exploring what kind of embodied experiences can be experientially, emotionally or sensorially interesting for users, so to design experiences where embodiment matters for them. We can find possible methodological approaches in the use of embodied elicitation techniques [25] to investigate sensorial and relational aspects that are worth their exploration or in the observation of users’ interactions with the system to analyze their ways of engaging with it [32]. Insights derived from this research can be used both to define the specific formal features of the environments, as well as aspects related to its context of use, its purpose etc.
Parallel to exploring embodied significance, a complementary path to support a mindful embodied engagement is the design effort to facilitate a beginner mind in the user that is approaching the experience. The idea of beginner mind derives from Zen Buddhism and describes the attitude of openness that a beginner has in front of a novel experience [67]. From the perspective of designing NOEIEs, this mindset can be facilitated by proposing experiences that do not allow to rely on already acquired assumptions or bodily gestalts but require to “re-learn” how to look at the world and de-familiarize us from our usual habits [25]. Within this context, Loke [23] and Wilde [25] have proposed interesting explorations of the design potential offered by unusual ways of moving or experiencing one’s own body. Similarly, performing art have widely explored these possibilities and in the context of HCI, projects such as “CHILDHOOD: Wearable Suit for Augmented Child Experience” [68] challenges our usual way of moving and seeing the world but putting the users in the body of a young child through a virtual reality system.

3.3.3. Case study: Using Mindful Embodied Engagement to Critically Examining Existing Designs

We used the quality of mindful embodied engagement as a reflexive lens to critically analyze existing design solutions and define possible improvements. In particular, we employed this notion in the study reported by Malinverni et al. [32], where we analyzed how children engage and relate to an interactive environment based on Full-Body Interaction.
The analyzed environment was based on an inflatable slide, on which we were projecting an interactive game (Figure 2). The game aimed at supporting the learning of buoyancy and Archimedes principle: children have to help a fish and a cat to reach their goals by raising the level of a water and building a bridge. To this end, children had to slide over virtual objects that were scrolling over the top of the environment.
The observations of children’s behaviors showed that this design offered poor affordances to support a mindful embodied engagement with the EIE. For instance, some children mainly engaged with the sensorimotor experience of sliding down, without paying attention to the digital contents. Instead, other children mainly focused on the goals of the game and did not explore the sensorimotor experiences offered by the space. Finally, a small group of children mainly engaged with trying to hit the digital cat by throwing objects on it.
These different ways of engaging with the system showed that this EIE failed in building a meaningful relationship between the users’ interests, the embodied experience and the digital augmentation, hence not allowing to fully explore and engage with their intertwined relationships. This failure can be explained by several factors. First, the design process behind this EIE mainly focused on the learning contents, without properly exploring the embodied experience or the children’s interests. Second, the proposed digital augmentation did not really build on the quality of the sensorimotor experience of sliding down nor on the features of the physical interface (i.e., it offers a poor contingent enhancement of the embodied experience). Third, the EIE proposes a goal that is extrinsic to the embodied experience itself (i.e., solving the game), hence reducing the relevance of the sensorimotor exploration and the attention that users devoted to it. Finally, the behavior of the cat (who complain when an object hits him) acted as an unnecessary distractor which diverted the attention from the overall experience.
These shortcomings indicated how using the quality of mindful embodied engagement as a critical lens helped us in identifying relevant improvements to be addressed both in the design process and in the design solution (e.g., kick-starting the design from the experience that can be lived through the slide; using digital technology to augment the experience of sliding down by making some of its hidden qualities visible; avoiding focusing the experience on extrinsic goals; etc.).

3.4. Situated Reflexivity

Significant embodied experiences may offer rich opportunities to materially embody and understand concepts, feelings and sensations without recourse to a linguistic medium or to any framework of preconceived cultural assumptions [44]. Nonetheless, often, to let this process occur, it may not be enough to just live the experience but we need to be capable of transforming it into an object to think with. Starting from this perspective, we suggest the quality of situated reflexivity as the degree to which the system enables conditions for an on-going reflection on the experience we are living.

3.4.1. Design for Situated Reflexivity

As Dewey [33] pointed out, we do not learn from experience, but we learn from reflecting on it. Hence, embodied experiences per se may run the risk of remaining blind and meaningless if they are just acted out without a reflection on their implications [69]. As a consequence, in order to fully profit of embodied experiences, people need to be capable of taking advantage of the interplay between acting and perceiving to reconsider their acts and decide whether to continue, discontinue or modify their actions [69].
The idea that concrete experiences need to be transformed to generate knowledge has been widely analyzed in research on embodied cognition and experiential learning [62,63,64]. Nonetheless, often, designers (and educators) have considered the reflective aspects of the experience as a separated moment, where experience precedes reflection and reflection becomes a disembodied thinking task oriented toward articulating the accumulated experience (as in [70]). Examples of this standpoint can be easily identified in approaches that create a structured distinction between “the moment for doing” and “the moment for sit down and thinking” [32].
Even if this strategy can also be effective in supporting reflective moments, within our framework, we consider that reflexivity should not be considered as a moment separated from the embodied experience but, instead, it should be leveraged in the situated and “lived” experience. As a consequence, designing for reflexivity deals with supporting conditions for which users can reflect in the situated interaction with the NOEIE, through and within the embodied experience that they are living.
Schön [71] describes a powerful example of this form of situated reflexivity through the concept of reflection-in-action. Specifically, by describing how practitioners think in action, he highlights the reflective conversation that unfolds between the designer and the materials of a situation, where the situation “talks-back” to the designer and he uses this knowledge to reflect-in-action on the possible problems, strategies and solutions. In the context of NOEIEs, this notion suggests the importance of creating conditions to support on-going and situated reflexivity with the materials and situations of the technology-mediated embodied experience.

3.4.2. Situated Reflexivity in the Design Process: Risks and Opportunities

Designing to support situated reflexivity is a complex endeavor since, as in the case of mindful embodied engagement, is not something that only deals with making the right design choices but, instead, it depends on the overall network of conditions that surround the experience (e.g., context, users’ predispositions, etc.). As a consequence, it may not be enough to design conditions to support reflexivity and expect that this will automatically occur. To guide designers in this task, we identified possible factors that may hinder situated reflexivity and eventual paths that can be explored to support it.
On the one hand, we can identify possible hindering factors in the risk of designing experiences that require a constant immersion, fast-paced actions or an excessive physical workload (e.g., time-based games, where the user needs to be constantly attentive and focused on repetitive tasks), hence not allowing moments to reflect-in-action. On the other hand, designers may run the risk of falling into the dualistic trap of adopting a prescriptive approach where they define a specific modulation that clearly separates time/space for thinking and time/spaces for doing.
To tackle these risks, researchers can explore different possible paths, formerly: allowing distancing, inhabiting otherness and identifying moments of situated reflexivity. The first two paths are mainly concerned with providing conditions to shift the user’s perspective to allow looking at the experience through different lenses and from different vantage points. The third one, instead, addresses the observation and interpretation of users’ behaviors during the interaction.
Allowing distancing focuses on creating conditions to allow users distancing themselves from the experience to re-frame it. Using a theatrical metaphor, it may correspond to the notion of “distancing effect” proposed by Bertolt Brecht as a form of breaking the theatrical illusion and hinder the audience from simply identifying themselves with the characters in the play [72]. Faced with the quality of mindful embodied engagement, this notion represents one of its complementary dimensions. In other words, we should provide users with conditions to allow them dwelling between getting immersed in the experience and re-emerging from immersion by living or looking at the experience from different perspectives. In the context of NOEIEs, we can find relevant opportunities to allow distancing in the features of the space and in the nature of the sensorimotor experience, e.g., providing physical vantage points to look at the experience from the distance. Possible research directions can explore how the bodily interactions and the physical configuration of the environment can allow users to physically inhabit or enacting different standpoints during the interaction with the system. Examples of them can be found in allowing users to go in an “observer position” or to see the experience at a different scale. A beautiful example of this latter idea is materialized in the project “Sandbox” by Lozano-Hemmer [73] where a video-streaming telepresence system between the beach and a sandbox allow users to either act as giant “gods” projected on a large scale of the beach or as tiny creatures projected in the small playable sandbox.
From a similar perspective, the notion of inhabiting otherness focuses on offering possibilities to put oneself in other people’s shoes. This means to assume the standpoint of something or somebody else and using their embodied experiences as a way for reflecting on our own. This approach is widely used in embodied practices such as psychodrama or art-therapy, where therapists use mutual sharing and role-play to support taking advantage of the experiences of the others as tools to rethink and re-conceptualize one’s own experience. In the context of NOEIEs, we can find relevant opportunities in grounding the practice of inhabiting otherness in its embodied and relational nature. For instance, research in social cognition has shown that even a simple exercise such as making children stand in the location previously taken by somebody else can facilitate their understandings of the other person’s standpoint [74]. Similarly, approaches that enable users to exchange their roles, living the experience through the eyes and body of others or just observing other people ways of doing can constitute useful strategies. An example of it can be found in the project of Nemirosky et al. [75], where, to explore graphs of motion, children can alternate between an egocentric and an exocentric perspective, either using their bodies or a remote device to create graphs. From a different perspective, instead, the installation “the machine to be another” [76], allows users to explore the experience of living in the body of somebody else through a Virtual Reality system.
Finally, a last strategy relies on observing, identifying and interpreting the moments of situated reflexivity that users may live while interacting with the system. This means to adopt an observational approach to spot out indicators of reflection-in-action and use the knowledge gained from this research to improve design. A relevant example of this strategy can be found in the work by Malinverni et al. [32] where the authors analyze the different ways through which children transformed the embodied experience of playing with an education Full-Body interaction environment into an ‘object–to-think-with’ and construct novel knowledge.

3.4.3. Case Study: Experiencing Situated Reflexivity

The “machine to be another” [76] is an Art investigation project on embodiment by BeAnotherLab. The system uses Virtual Reality to allow individuals to experience the world through the eyes and body of another person and offers different possible experiential scenarios (e.g., live performance, pre-recorded materials etc.). The first author had the opportunity of experiencing it during a public presentation of the project, where she tried a live performance scenario. In this configuration, the user wears a VR head-mounted display and earphones. A performer wears a headset with a camera that records his movements and transmits them to the head-mounted display of the user (Figure 3). From the account of the first author’s experience: “After sitting on a seat, they asked me to wear the head-mounted display, the earphones, put my hands on my knee and close the eyes to wait for the calibration. Subsequently, when the system was ready, they told me to open my eyes and to look at my hands. I looked down and, for few seconds, it seemed to me that everything was completely normal and familiar. They told me to move a hand. I rotated my hand and I saw a tattoo on the arm that was not mine. I observed more carefully and I realized that the body that I was seeing was not mine: the arms were bigger and furrier than mine, the legs were longer, the clothes were different. I was extremely surprised by fact that I needed to see the tattoo to realize that this was obviously not my body but I was inhabiting the body of a tall man (I’m a short and tiny woman). It made me think about up to which point I take my body for granted and do not pay attention to it. I started experimenting what does it means to have the body of a tall man: I moved “my” arms and legs, touched objects, stood up, etc. The performer was emulating all my movements, so to generate the illusion that my new body was actually responding to my intentions. It was puzzling. It was at the same time my body and not my body. I was dwelling between the confusion of do not know exactly what to do and the willingness of “testing” this new body and the feelings that this displacement was generating in me. All everyday banal actions seemed interesting and worth to be endlessly explored (e.g., opening and closing a hand, touching my belly, grab something, etc.). I was feeling like a toddler who discovers something new and cannot stop to repeat that action. When the experience ended and I removed the head-mounted display, I saw the performer. I felt a strange and visceral feeling of intimacy that was actually quite embarrassing for me and I needed to go and hug him. I kept thinking about the experience all day, imagining how it could be, for instance, to try it for a longer period (e.g., a whole day) or with a body that was even more different from mine”.

4. Discussion

The presented framework allowed defining a set of qualities that we considered relevant for thinking and designing NOEIEs ((1) Spatial, Corporeal and Material Consistency, (2) Contingent Enhancement, (3) Mindful Embodied Engagement and (4) Situated Reflexivity). At the same time, it offered referents, inspirations, concepts and potential methodological approaches and case studies to guide practitioners in this task (For a summary see Table 1). With respect to other design frameworks for Embodied Interaction [6,7,10,11,12], our approach made the effort of addressing the challenges of non-task-oriented Embodied Interaction Environments and focused on the intertwined relationship that is established between users and digital-physical environments.
The presented qualities and their related strategies do not aim at working as prescriptive guidelines but as flexible tools-to-think-with and lenses through which researchers can look at their design proposals. Specifically, we suggest different possible contexts of use. On the one hand, researchers and designers can use them as generative tools during the design process. On the other hand, the qualities can be used as instruments to support the critical analysis of one’s own work or others design solutions. Finally, they can be used as a pedagogical tool in students’ training.
However, the proposed qualities to not aim at being an exhaustive and complete panorama, but, as all taxonomies, they offer a view that is always partial and incomplete [77]. Hence, further research may expand them either be including other relevant concepts or by proposing carefully crafted methods to incorporate them in the design process.

4.1. Inspiring the Design Process

Designers and researchers can use the proposed qualities in different stages of the design process. For instance, designers can use them in the initial ideation stage as inspirational, evocative and reflective materials to explore novel design possibilities. In particular, we consider that the proposed qualities can expand our ways of thinking about design, hence, hopefully, helping in spotting out possible paths for future research or design of solutions that do not exist yet [25]. Similarly, the qualities can be used in the elaboration of preliminary prototypes as a sort of “soft” heuristic to critically revise our work and enable other standpoints to look at our designs. Finally, they can be employed as analytical lenses during research with users in iterative design cycles. In particular, we consider that their features make them particularly suitable to analyze users’ interactions from a standpoint that goes beyond the focus on usability, task-completion and effectiveness.
In our practice, we started to use the proposed qualities to guide our design and research. However, we consider that further research is still needed to make them accessible to a broader public. First, until now we did not fully extend the use of the different qualities to Participatory Design processes. Second, we are currently researching on the possibilities of proposing the qualities in a format suitable to be a “ready-to-use” prompt during the design sessions. We made some initial attempts in this direction and we created a set of design cards describing the qualities and tried to use them both with our design team as well as with students. Nonetheless, the conceptual, logo-centric and visual format of the cards was not particularly adequate or useful for the purpose and contents of the framework. Hence, further research is still needed to explore more appropriate “ready-to-use” formats (e.g., evocative objects that can embody the concepts described by the proposed qualities).

4.2. Critically Analyzing Design

A second context of use for the proposed qualities is critical analysis. While critique represents a common and widespread practice in a vast range of cultural productions, it is still partially underestimated in the field of HCI. Nonetheless, we consider that a critical analysis of our own works or of others designers’ proposals can be useful to foster self-reflective practice and move design forward. Within our framework, we suggest that practitioners can use the proposed qualities to analyze and evaluate different design solutions by helping in asking relevant questions and tracing important distinctions.

4.3. Students’ Training

Finally, the proposed qualities can serve as a useful instrument in students’ training. As Levisohn et al. [18] pointed out often the traditional interaction design pedagogy tends to neglect training students on skills needed to design for Embodied Interaction. Hence, we need instruments and approaches to help students thinking about embodiment for design.
In our case, we did a preliminary attempt to use an early version of proposed qualities in a short workshop with master students. The main goal of the workshop was to observe whether these concepts were understandable and whether they can support an informed design process. To this end, we initially presented the theoretical underpinning of each quality and relevant references. Subsequently, we proposed a mini-design challenge for each of them. Even if the workshop was not formally evaluated, the proposed approach showed the usefulness of the qualities in providing a critical perspective to design for Embodied Interaction. However, due to the limited purpose of the workshop, we observed the use of the qualities only in the ideation of early prototypes. Hence, further research is still needed to evaluate their usefulness in more longitudinal and complex design processes.

5. Conclusions

We presented a framework for guiding researchers in thinking, designing and critically analyzing environments that aim at using Embodied Interaction to allow embodied experiences of entanglement with the physical-digital setting that are aesthetically, sensually, emotionally and intellectually significant and complete for its own sake. The framework is grounded on perspectives akin to experience-centered design approaches [17], research on movement as an aesthetic and felt-experience [18] and relational epistemologies [35,36]. It proposes set of qualities that we considered relevant to guide design. Specifically, we discuss the qualities of (1) Spatial, Corporeal and Material Consistency, (2) Contingent Enhancement, (3) Mindful Embodied Engagement and (4) Situated Reflexivity and offer strategies and examples to incorporate them in the design process. Despite further research is still needed to transform these qualities into “ready-to-use” tools during design processes with people that are not familiar with them, we consider that they can offer useful reflective lenses for researchers interested in non-task-oriented Embodied Interaction Environments.

Author Contributions

Both authors contributed to the conceptualization of the framework. Laura Malinverni wrote the main corpus of text and Marie-Monique Schaper revised and improved it.

Conflicts of Interest

The authors declare no conflict of interest.


  1. Buxton, W. There’s more to interaction than meets the eye: Some issues in manual input. In User Centered System Design: New Perspectives on Human-Computer Interaction; Norman, D.A., Draper, S.W., Eds.; Lawrence Erlbaum Associates, Hil: Mahwah, NJ, USA, 1986. [Google Scholar]
  2. Dourish, P. Where the Action Is: The Foundations of Embodied Interaction; The MIT Press: Cambridge, MA, USA, 2004. [Google Scholar]
  3. Ishii, H.; Ullmer, B. Tangible bits: Towards seamless interfaces between people, bits and atoms. In Proceedings of the Conference on Human Factors in Computing Systems, CHI’97, Atlanta, GA, USA, 22–27 March 1997. [Google Scholar]
  4. Wilson, A.D. TouchLight: An imaging touch screen and display for gesture-based interaction. In Proceedings of the 6th International Conference on Multimodal Interfaces, State College, PA, USA, 14–15 October 2004; pp. 69–76. [Google Scholar]
  5. Malinverni, L.; Pares, N. Learning of Abstract Concepts through Full-Body Interaction: A Systematic. Educ. Technol. Soc. 2014, 17, 100–116. [Google Scholar]
  6. Antle, A.N. Research Opportunities: Embodied Child–computer Interaction. Int. J. Child Comput. Interact. 2013, 1, 30–36. [Google Scholar] [CrossRef]
  7. Klemmer, S.R.; Hartmann, B.; Takayama, L. How bodies matter: Five themes for interaction design. In Proceedings of the 6th Conference on Designing Interactive Systems, University Park, PA, USA, 26–28 June 2006; pp. 140–149. [Google Scholar]
  8. Dillenbourg, P.; Evans, M. Interactive Tabletops in Education. Int. J. Comput. Collab. Learn. 2011, 6, 491–514. [Google Scholar] [CrossRef]
  9. Revelle, G. Applying developmental theory and research to the creation of educational games. In Digital Games: A Context for Cognitive Development. New Directions for Child and Adolescent Development; John Wiley & Sons: Hoboken, NJ, USA, 2013; pp. 31–40. [Google Scholar]
  10. Jacob, R.J.; Girouard, A.; Hirshfield, L.M.; Horn, M.S.; Shaer, O.; Solovey, E.T.; Zigelbaum, J. Reality-based interaction: A framework for post-WIMP interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Florence, Italy, 5–10 April 2008; pp. 201–210. [Google Scholar]
  11. Hornecker, E.; Buur, J. Getting a grip on tangible interaction: A framework on physical space and social interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Montreal, QC, Canada, 22–27 April 2006; pp. 437–446. [Google Scholar]
  12. Fogtmann, M.H.; Fritsch, J.; Kortbek, K.J. Kinesthetic interaction—Revealing the bodily potential in interaction design. In Proceedings of the 20th Australasian Conference on Computer-Human Interaction: Designing for Habitus and Habitat, OZCHI’08, Cairns, Australia, 8–12 December 2008; pp. 89–96. [Google Scholar]
  13. Alaoui, S.F.; Caramiaux, B.; Serrano, M. From dance to touch: Movement qualities for interaction design. In Proceedings of the CHI’11 Extended Abstracts on Human Factors in Computing Systems, Vancouver, BC, Canada, 7–12 May 2011; pp. 1465–1470. [Google Scholar]
  14. Caramiaux, B.; Altavilla, A.; Pobiner, S.G.; Tanaka, A. Form follows sound: Designing interactions from sonic memories. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, Seoul, Korea, 18–23 April 2015; pp. 3943–3952. [Google Scholar]
  15. Harrison, S.; Sengers, P.; Tatar, D. Making Epistemological Trouble: Third-Paradigm HCI as Successor Science. Interact. Comput. 2011, 23, 385–392. [Google Scholar] [CrossRef]
  16. Petersen, M.G.; Iversen, O.S.; Krogh, P.G.; Ludvigsen, M. Aesthetic interaction. In Proceedings of the 2004 Conference on Designing Interactive Systems Processes, Practices, Methods, and Techniques, DIS’04, Cambridge, MA, USA, 1–4 August 2004; p. 269. [Google Scholar]
  17. Wright, P.; Wallace, J.; McCarthy, J. Aesthetics and Experience-Centered Design. ACM Trans. Comput. Interact. 2008, 15, 1–21. [Google Scholar] [CrossRef]
  18. Levisohn, A.; Schiphorst, T. Embodied Engagement: Supporting Movement Awareness in Ubiquitous Computing Systems. Ubiquitous Learn. Int. J. 2011, 3, 97–111. [Google Scholar]
  19. Abrahamson, D.; Trninic, D. Toward an embodied interaction design framework for mathematical concepts. In Proceedings of the 10th International Conference on Interaction Design and Children (IDC’11), Ann Arbor, MI, USA, 20–23 June 2011. [Google Scholar]
  20. Deng, Y.; Antle, A.N.; Neustaedter, C. Tango cards: A card-based design tool for informing the design of tangible learning games. In Proceedings of the 2014 Conference on Designing Interactive Systems, Vancouver, BC, Canada, 21–25 June 2014; pp. 695–704. [Google Scholar]
  21. Melcer, E. Bridging the physical learning divides: A design framework for embodied learning games and simulations. In Proceedings of the First International Joint Conference of DiGRA and FDG; DIGRA: Dundee, UK, 2016; pp. 2225–2233. [Google Scholar]
  22. Svanæs, D. Interaction Design for and with the Lived Body: Some Implications of Merleau-Ponty’s Phenomenology. ACM Trans. Comput. Interact. 2013, 20, 8:1–8:30. [Google Scholar] [CrossRef]
  23. Loke, L.; Robertson, T. Moving and Making Strange: An Embodied Approach to Movement-Based Interaction Design. ACM Trans. Comput. Interact. 2013, 20, 7:1–7:25. [Google Scholar] [CrossRef]
  24. Schiphorst, T. From the Inside Out: Design Methodologies of the Self; Simon Fraser University: Surrey, BC, Canada, 2001. [Google Scholar]
  25. Wilde, D.; Vallgårda, A.; Tomico, O. Embodied design ideation methods. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems–CHI’17, Denver, CO, USA, 6–11 May 2017; pp. 5158–5170. [Google Scholar]
  26. Höök, K. Transferring qualities from horseback riding to design. In Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries (NordiCHI’10), Reykjavik, Iceland, 16–20 October 2010; pp. 226–235. [Google Scholar]
  27. Malinverni, L.; Mora-Guiard, J.; Pares, N. Towards Methods for Evaluating and Communicating Participatory Design: A Multimodal Approach. Int. J. Hum. Comput. Stud. 2016, 94, 53–63. [Google Scholar] [CrossRef]
  28. Schaper, M.-M.; Malinverni, L.; Pares, N. Sketching through the body: Child-generated gestures in full-body interaction design. In Proceedings of the 14th International Conference on Interaction Design and Children (IDC’15), Medford, MA, USA, 21–25 June 2015; pp. 255–258. [Google Scholar]
  29. Malinverni, L.; Schaper, M.; Pares, N. An Evaluation-Driven Design Approach to Develop Learning Environments Based on Full-Body Interaction. Educ. Technol. Res. Dev. 2016, 64, 1337–1360. [Google Scholar] [CrossRef]
  30. Schaper, M.M.; Santos, M.; Malinverni, L.; Zerbini Berro, J.; Pares, N. Learning about the Past through Situatedness, Embodied Exploration and Digital Augmentation of Cultural Heritage Sites. Int. J. Hum. Comput. Stud. 2018, 114, 36–50. [Google Scholar] [CrossRef]
  31. Malinverni, L.; Pares, N. Learning from failures in designing and evaluating full-body interaction learning environments. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems, CHI EA’17, Denver, CO, USA, 6–11 May 2017; pp. 1065–1074. [Google Scholar]
  32. Malinverni, L.; Ackermann, E.; Pares, N. Experience as an object to think with: From sensing-in-action to making-sense of action in full-body interaction learning environments. In Proceedings of the TEI’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction, TEI’16, Eindhoven, The Netherlands, 14–17 February 2016; pp. 332–339. [Google Scholar]
  33. Dewey, J. Art as Experience; Penguin: London, UK, 2005. [Google Scholar]
  34. Schaper, M.-M.; Malinverni, L.; Pares, N. Participatory design methods to define educational goals for full-body interaction. In Proceedings of the 11th Conference on Advances in Computer Entertainment Technology (ACE’14), Funchal, Portugal, 11–14 November 2014; p. 4. [Google Scholar]
  35. Overton, W.F. Embodied development: Ending the nativsim-empiricism debate. In Nature and Nurture: The Complex Interplay of Genetic and Environmental Influences on Human Behavior and Development; Garcia Coll, C., Bearer, E., Lerner, R., Eds.; Lawrence Erlbaum Associates: Mahwah, NJ, USA, 2004; pp. 201–223. [Google Scholar]
  36. Barad, K. Posthumanist Performativity: Toward an Understanding of How Matter Comes to Matter. Signs J. Women Cult. Soc. 2003, 28, 801–831. [Google Scholar] [CrossRef]
  37. Massage Me. Available online: (accessed on 13 March 2018).
  38. SweetPad. 2004. Available online: (accessed on 13 March 2018).
  39. Larssen, A.T.; Robertson, T.; Edwards, J. The feel dimension of technology interaction. In Proceedings of the 1st international conference on Tangible and embedded interaction, TEI’07, Baton Rouge, LA, USA, 15–17 February 2007; p. 271. [Google Scholar]
  40. Jacucci, G.; Wagner, I. Performative roles of materiality for collective creativity. In Proceedings of the 6th ACM SIGCHI conference on Creativity & cognition, C&C’07, Washington, DC, USA, 13–15 June 2007; p. 73. [Google Scholar]
  41. Ishii, H.; Lakatos, D.; Bonanni, L.; Labrune, J.B. Radical Atoms: Beyond Tangible Bits, toward Transformable Materials. Interactions 2012, 19, 38–51. [Google Scholar] [CrossRef]
  42. Döring, T.; Sylvester, A.; Schmidt, A. A design space for ephemeral user interfaces. In Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction, Barcelona, Spain, 10–13 February 2013; pp. 75–82. [Google Scholar]
  43. Djajadiningrat, T.; Matthews, B.; Stienstra, M. Easy Doesn’t Do It: Skill and expression in tangible aesthetics. In Personal and Ubiquitous Computing; Springer: Berlin, Germany, 2007; Volume 11, pp. 657–676. [Google Scholar]
  44. Hickey-Moody, A.; Palmer, H.; Sayers, E. Diffractive Pedagogies: Dancing across New Materialist Imaginaries. Gend. Educ. 2016, 28, 213–229. [Google Scholar] [CrossRef]
  45. Höök, K.; Jonsson, M.P.; Ståhl, A.; Mercurio, J. Somaesthetic appreciation design. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, CHI’16, San Jose, CA, USA, 7–12 May 2016; pp. 3131–3142. [Google Scholar]
  46. Tomico, O.; Wilde, D. Soft, Embodied, Situated & Connected: Enriching Interactions with Soft Wearables. mUX J. Mob. User Exp. 2016, 5, 3. [Google Scholar] [CrossRef] [Green Version]
  47. Wilde, D. Embodying material ideation. In Proceedings of the 4th Participatory Innovation, The Hague, The Netherlands, 18–20 May 2015; p. 386. [Google Scholar]
  48. Munari, B. Fantasia; Editori Gius. Laterza & Figli: Bari, Italy, 1977. [Google Scholar]
  49. Françoise, J.; Candau, Y.; Fdili Alaoui, S.; Schiphorst, T. Designing for kinesthetic awareness: Revealing user experiences through second-person inquiry. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA, 6–11 May 2017; pp. 5171–5183. [Google Scholar]
  50. Reed, S.E.; Kreylos, O.; Hsi, S.; Kellogg, L.H.; Schladow, G.; Yikilmaz, M.B.; Segale, H.; Silverman, J.; Yalowitz, S.; Sato, E. Shaping Watersheds Exhibit: An Interactive, Augmented Reality Sandbox for Advancing Earth Science Education. In AGU Fall Meeting Abstracts; AGU: Washington, DC, USA, 2014; Volume 1, p. 1. [Google Scholar]
  51. Aoki, T. Kobito-virtual brownies-art and science. In Proceedings of the ACM SIGGRAPH Emerging Technologies, Los Angeles, CA, USA, 31 July–4 August 2005. [Google Scholar]
  52. Full Body Projection Mapping. Available online: (accessed on 14 March 2018).
  53. Scheffler, I. A Brief History of the LEM. In The Routledge Companion to Jacques Lecoq; Taylor & Francis: Oxford, UK, 2016. [Google Scholar]
  54. Schiphorst, T.; Andersen, K. Between bodies: Using experience modeling to create gestural protocols for physiological data transfer. In ACMCHI 04; CHI Fringe: Vienna, Austria, 2004; pp. 1–8. [Google Scholar]
  55. When Infinity Comes to Life–YouTube. Available online: (accessed on 14 March 2018).
  56. I.E.D. Available online: (accessed on 14 March 2018).
  57. Animal Superpowers—YouTube. Available online: (accessed on 14 March 2018).
  58. Festinger, L. A Theory of Cognitive Dissonance; Stanford University Press: St. Redwood, CA, USA, 1962. [Google Scholar]
  59. Kilteni, K.; Normand, J.M.; Sanchez-Vives, M.V.; Slater, M. Extending Body Space in Immersive Virtual Reality: A Very Long Arm Illusion. PLoS ONE 2012, 7, e40867. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  60. You Are Not Here A Dislocative Tourism Agency. Available online: (accessed on 14 March 2018).
  61. Munari, B. Da Lontano Era Un’isola; Corraini Edizioni: Mantova, Italy, 2006. [Google Scholar]
  62. Malinverni, L.; Maya, J.; Schaper, M.-M.; Pares, N. The world-as-support: Embodied exploration, understanding and meaning-making of the augmented world. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI’17), Denver, CO, USA, 6–11 May 2017. [Google Scholar]
  63. Langer, E.J.; Moldoveanu, M. The Construct of Mindfulness. J. Soc. Issues 2000, 56, 1–9. [Google Scholar] [CrossRef]
  64. Csikszentmihalyi, M. Finding Flow: The Psychology of Engagement with Everyday Life; Basic Books: New York, NY, USA, 1997. [Google Scholar]
  65. Turkle, S. Alone Together: Why We Expect More from Technology and Less from Each Other; Hachette Book Group: New York, NY, USA, 2012. [Google Scholar]
  66. Jackson, M. Distracted: The Erosion of Attention and the Coming Dark Age; Prometheus Books: Westminster, MD, USA, 2008. [Google Scholar]
  67. Suzuki, S. Zen Mind, Beginner’s Mind: Informal Talks on Zen Meditation and Practice; Shambhala Publications: Boulder, CO, USA, 2010. [Google Scholar]
  68. Nishida, J.; Takatori, H.; Sato, K.; Suzuki, K. Childhood. In Proceedings of the ACM SIGGRAPH 2015 Emerging Technologies SIGGRAPH’15, Los Angeles, CA, USA, 9–13 August 2015. [Google Scholar]
  69. Ackermann, E. Tools for Constructive Learning: Rethinking Interactivity; Epistemology and Learning Group, MIT Media Laboratory: Cambridge, MA, USA, 1993. [Google Scholar]
  70. Di Stefano, G.; Gino, F.; Pisano, G.; Staats, B. Learning by Thinking: Overcoming the Bias for Action through Reflection; Harvard Business School: Cambridge, MA, USA, 2014. [Google Scholar]
  71. Schon, D. The Reflective Practitioner; Basic Books, Inc.: New York, NY, USA, 1983. [Google Scholar]
  72. Willett, J. Brecht on Theatre; Hill and Wang: New York, NY, USA, 1964. [Google Scholar]
  73. Rafael Lozano-Hemmer. Project Sandbox. Available online: (accessed on 14 March 2018).
  74. Perinat, A.; Lalueza, J.L. Psicología del Desarrollo: Un Enfoque Sistémico; Editorial UOC: Rambla del Poblenou, Barcelona, Spain, 2007. [Google Scholar]
  75. Nemirovsky, R.; Tierney, C.; Wright, T. Body Motion and Graphing. Cogn. Instr. 1998, 16, 119–172. [Google Scholar] [CrossRef]
  76. The Machine to Be Another: Art Investigation Using Embodiment and Performances. Available online: (accessed on 14 March 2018).
  77. Van Leeuwen, T. Introducing Social Semiotics: An Introductory Textbook; Routledge: London, UK, 2004. [Google Scholar]
Figure 1. Design concept for the installation.
Figure 1. Design concept for the installation.
Mti 02 00022 g001
Figure 2. Children playing with the Archimedes game.
Figure 2. Children playing with the Archimedes game.
Mti 02 00022 g002
Figure 3. The experience of “the machine to be another”.
Figure 3. The experience of “the machine to be another”.
Mti 02 00022 g003
Table 1. Summary of the proposed qualities and strategies.
Table 1. Summary of the proposed qualities and strategies.
Spatial, Corporeal and Material ConsistencyThe degree to which the design of the spatial, corporeal and material features of a NOEIE offers a coherent fit with the experience that we want to support
  • Legitimating kinesthetic creativity
  • Exploring felt-experience
  • Learning from art practices
Contingent EnhancementThe degree to which digital technologies can make some aspects of the embodied experience richer and more interesting
  • Making the invisible visible
  • Supporting embodied explorations
  • Generating embodied dissonance
Mindful Embodied EngagementThe degree to which the system allows getting engaged and focused on exploring our entanglement with the technology-mediated embodied experience
  • Exploring embodied significance
  • Facilitating a beginner mind
Situated ReflexivityThe degree to which the system enables conditions for an on-going reflection on the experience we are living
  • Allowing distancing
  • Inhabiting otherness
  • Identifying moments of situated reflexivity

Share and Cite

MDPI and ACS Style

Malinverni, L.; Schaper, M.-M. An Experience-Centered Framework for Designing Non-Task-Oriented Embodied Interaction Environments. Multimodal Technol. Interact. 2018, 2, 22.

AMA Style

Malinverni L, Schaper M-M. An Experience-Centered Framework for Designing Non-Task-Oriented Embodied Interaction Environments. Multimodal Technologies and Interaction. 2018; 2(2):22.

Chicago/Turabian Style

Malinverni, Laura, and Marie-Monique Schaper. 2018. "An Experience-Centered Framework for Designing Non-Task-Oriented Embodied Interaction Environments" Multimodal Technologies and Interaction 2, no. 2: 22.

Article Metrics

Back to TopTop