An Exploratory Study of the Uses of a Multisensory Map—With Visually Impaired Children

Abstract

This paper reports an empirical study of a multisensory map used by visually impaired primary school pupils, to study human habitats and differences between urban, suburban and rural areas using a local example. Using multimodal analysis, we propose to examine how the use of smell and taste shape pupils’ engagement and the development of a non-visual knowledge of geography. Our research questions include: How do pupils try to make sense of this unusual material, in conjunction with the tactile, audio and tangible material used in this lesson? How does the special education teacher support the development of these interpretations? Multisensory material has the potential to support experiential and embodied learning: were these promises achieved? Our findings show how this multisensory map reconfigures spatial occupation and interaction dynamics, and that it has the potential to make the classroom more pervasive to pupils’ social, spatial and emotional lives. In doing so, it provides opportunities for the teacher to develop citizenship education. The paper provides concrete examples of uses of smell and taste in learning activities to support engagement, and has implications for pedagogical design beyond special education.

1. Introduction

To engage pupils in learning geography, many scholars recommend building on children’s experiences of space (e.g., [1,2,3]), for instance by organizing field trips [4,5,6]. Moreover, some geographers have long challenged vision as the primary or only way to learn and experience space, and to favor embodied and multisensory approaches instead [7,8,9]. In fact, we could argue this concern dates from the end of the 19th century, as it is present in popular education initiatives for an emancipatory geography curriculum [10]. On a didactic level, it means geographic learning starts with a given place and a given number of traits, towards more general concepts. One way to do so is to compare two places, their similarities and differences. Another is to zoom out: showing how a neighborhood fits into a city, county, region, etc. [3]—the purpose being to accompany children so that they can understand the macro dynamics that shape their lives and act upon it [11].

Everyday spatial experiences or field trips are multisensory by design. However, teachers often remain focused on visual cues, on the seen landscape, to introduce geographic concepts. This is due to the lack of examples available to harness other types of cues for formal geographic teaching, let alone guidelines [12,13]. Olfaction and taste in particular have been studied very little, whether inside or outside the classroom. When they are, it often is for investigating the so-called Proust’s madeleine effect, i.e., in the hope that associating an information with a specific smell will enable to better recall it (e.g., [14]). Although many studies suggest so, others contest this fact (for a review, see [15]). In addition, others argue that smell provides a sense of security, well-being, and pleasure [16], hence that their usefulness resides in creating pleasant atmospheres improving focus.

In other words, smell and taste in education are more often considered as useful to process unrelated information, rather than meaningful on their own right. This might be because their perception and interpretation is a highly subjective process: smells and flavors are considered difficult to describe [17]. Thibaud, for instance, argues that “olfaction tends to produce overly diffuse and volatile phenomena” ([18], p. 12), hence not necessarily adequate for studying and teaching geography. However, this can also be seen positively. For instance, McBride and Nolan [19] argue, citing [20], that olfactive components may enable learning experiences beyond language, in line with the desired embodied and experiential approaches to learning. Moreover it might open, for visually impaired children, more possibilities to participate and collaborate.

Conducted as part of a larger participatory research project on technologies to teach geography to visually impaired children, this study also finds its roots in existing special education practices. The special education teacher involved in the research project wanted to better understand how some of her hunches about pedagogy worked. One such hunch was to use food, e.g., to bring traditional food from all continents in a lesson dedicated to learn to recognize continents and their features. She argued it improved memorization for some of her pupils, which she thought could be explained by the theory of learning styles, i.e., that children have a sense or a combination of senses of preference for learning. However, this theory is controversial [21] and lacks empirical basis [22]. Furthermore, it does not take into account smell or taste.

We thus decided to further investigate the possible uses of smell and taste for teaching geography at a primary school level, and how it can be articulated with traditional supports for geography teaching: maps, hereby tactile. How do pupils try to make sense of this still unusual material, in conjunction with the better-known map? How does the special education teacher support the development of these interpretations? Multisensory material in the classroom has the potential to support experiential and embodied learning: does it change the kind of geographic knowledge discussed? To answer these questions, we take a case-study approach. We observed a lesson using Mapsense, an interactive multisensory map mediating the use of smell and taste using tangible interaction, designed with our participants.

Using multimodal analysis, we discuss the meanings attributed to smellable and tastable cues by pupils, and the ways the prototype itself shapes the possible classroom interactions. It provides concrete examples of uses of smell and taste in learning to support pupils’ engagement with the geography curriculum. More specifically, the findings suggest that the wide range of meanings that can be attributed to these cues, their open-endedness, can open new spaces for discussion—in particular, discussions on social (dis)advantages through stories about their social, spatial and emotional lives, which they can then better situate in larger (spatial) dynamics. In doing so, it afforded the teacher opportunities to further her pupils’ civic and citizenship education. This endeavor has implications for pedagogical design beyond special education—our overarching goal being to draw opportunities for a more inclusive curriculum and classroom, to invite all children in developing an emancipatory and non-visual knowledge of geography.

2. Design of the Prototype

MapSense is a multisensory map that relies on tangible interaction to mediate the use of smell and taste (thereby also named smellables and tastables). This section describes the specific instance of Mapsense used in the study below. The timeline in Appendix A gives an overall idea of the continuous re-assemblages and transfers of design properties [23] that resulted in the instance of the prototype presented here. It is also described in more detail in [24,25].

2.1. Process

In the design of Mapsense, children were considered as informers [26]. This means we took the design decisions in agreement with the teacher, but these were inspired by multiple previous observations and occasions for the pupils involved to express preferences or devices they would like to build.

2.2. General Setup

MapSense was based on an interactive audio-tactile map illustrated in Figure 1, and initially designed by [27]. Its main components are a raised-line map overlay placed over a 27-inch touch screen using capacitive projected technology, connected to a computer and speakers. Double tapping on the screen provides audio information about map elements. In addition to the main display and exploration area, the prototype includes a menu to access different types of information (e.g., points of interests, or general directions, see Figure 2). It also enables the use of tangibles made of conductive material, which can contain smellable or tastable material, as illustrated in Figure 1c.

2.3. Interaction Principles

Interaction was limited to specific interactive elements on the map, which were defined by their coordinates, category (e.g., type of information, such as cities), and content (e.g., audio cues, such as city names or sounds recorded in a given city). To trigger audio cues, users had to double tap on interactive elements, which were indicated by a tactile symbol. Double-taps were used in order to prevent unexpected audio cues during manual exploration. A double tap with a conductive tangible had the same effect as a finger double-tap.

The different types of verbal information delivered by the map are selected via a two-level menu (see Figure 2). For the second sequence, for instance, pupils were invited to select points of interest (e.g., city halls, private services, public services), general directions (e.g., roads, rivers, map orientation), and audio discovery (e.g., playful cues for each interactive area). Pupils browsed the categories by swiping, and selected a category by double tapping. Once one type of information was selected (e.g., points of interests), all related interactive areas could be triggered by a double tap. The audio discovery category provided non-verbal audio cues that had been recorded by the children themselves (e.g., the sound of water, the sound of their white cane on the ground, or noises in school). Furthermore, users could browse the names of all interactive elements available within each category by using a second menu. A double tap on one of these elements activated verbal indications (left/right/top/bottom) that guided the hand towards the target location.

The design of the menu was based on the design of keys or legends for regular tactile maps. Legends were structured around the different types of geographical information. In general, when discovering a new tactile map, visually impaired people first read the legends [27]. Similarly, in the prototype, students can go through the first menu, and then read the content within each category (second menu). The guiding option allows pupils to find a point of interest by themselves and thus to take full advantage of the interactive system.

2.4. Map

We designed a colored raised-line map with the help of the teacher and a tactile document maker. It illustrated the itinerary that the pupils followed during a class trip. It depicts one of France’s major cities and the surrounding countryside, a small sized city and a village (see Figure 3a). Audio cues recorded during that trip were associated with interactive elements. Thus, children could associate their subjective experience during the trip with a geographic representation of the trip. The map was produced on swell paper in A3 format, which is a format frequently used in the context of maps for visually impaired people. A smaller (A4) non-interactive version of the map with written legend (either in braille or in large characters) could be printed for each child to take home.

Tangibles

Our set of tangible objects illustrates literally [28] various objects that children had encountered during a previous field trip and represents different kinds of urban settings. Their design follows the recommendations and principles derived from previous uses of the same prototype (see [25]). They could be used to trigger audio interaction on the map, or independently from the map. They were designed in collaboration with the teacher, using feedback from the five children involved in the study and represented:

• housing: a farm, a building, two medium- and three small-sized houses;
• shops: a grocery store, a bank, a restaurant and a supermarket;
• services: two post offices and two schools of different sizes;
• infrastructure: an airport, two planes;
• green spaces: fields, parks and trees;
• historical points of interest: churches, tombs, war memorial;

2.5. Olfactory and Flavor Cues

We used six different smellables and tastables, related to different concepts the teacher wanted the pupils to elaborate on. The teacher first identified several possible associations for each concept. The associations that she proposed were guided by three principles:

• Similarity to something children had smelled during a previous field trip;
• Familiarity i.e., cues that all children are expected to have experienced in their daily lives;
• Or unknown cues that could introduce new experiences and meaning. Some cues were direct associations (e.g., colza oil for colza fields), others were metaphors (e.g., goat cheese for the farm).

The teacher finally chose six cues that were easily available: (1) Pastries were used to illustrate a bakery near the service provider’s building and the tertiary economic sector. (2) A sample of rubber was used to represent the traffic, and thus the transportation infrastructure. (3) Fresh grass was used to illustrate both green spaces in urban settings and on the countryside. (4) Strawberries were used to illustrate raw products (in contrast with pastries). (5) Colza oil and goat cheese represented the region’s farming and livestock. These were direct references to the field-trip undertaken earlier in the pedagogical unit (see Appendix C). (6) Licking envelopes were used as a representation of the postal infrastructure and services.

The olfactory and flavor cues were contained in small, conductive 3D printed bowls in which children could pick up with a small spoon (see Figure 4b), or distributed when the related topic was discussed (e.g., envelope would not fit in the bowls). Using the bowls has several advantages: pupils could easily manipulate them and place them on or around the map; they could contain any kind of Do-It-Yourself scents and tastes (in comparison with a scent diffuser, which only provides a limited number of synthetic scents); finally, pupils associated it with dinnerware.

3. Methods: Study Design

3.1. Context

The study was conducted in the main building of a service provider for visually impaired children situated in Toulouse, France (Centre d’Education Spécialisé pour Déficients Visuels; Institut des Jeunes Aveugles (CESDV-IJA)). It provides education to children with visual impairments and assistance to their families. It ensures adequate care (e.g., psychological assistance, psycho-motor and low-vision skills, orientation and mobility training, etc.). It helps with the inclusion of visually impaired children in mainstream schools. For that purpose, specialized teachers create accessible material that can be used in mainstream classrooms. For example, they adapt graphical representations and books (e.g., production of enlarged or braille documents, and tactile diagrams and maps). In addition, the center provides professional training and autonomy skills to adults undergoing sight loss.

Ethics

When this research project began, the home institution of the field researcher did not have an ethics committee. This was still common for social sciences research in France at the time. The research team thus defined an ethical framework, with ongoing discussions to address dilemmas and improve practices (following the recommendations of [29]). We followed the UNICEF guidelines on research involving children [30], which are based on: (1) the respect of children’s cultures; (2) doing research that benefits children and does not harm them; and (3) addressing issues of justice, by recognizing children’s rights, encouraging them in formulating their views and aiming at improving their living conditions. All prototypes used in the classroom were devised with the teacher according to her pedagogical needs and the existing school culture (and were designed with the children as much as possible). We made suggestions and presented possibilities, but the teacher always made the final decisions, with the explicit goal of improving her teaching practices. Thus, the current study was entirely embedded in children’s usual activities, did not change the content of the lessons and did not add a burden on the children. The field researcher specifically asked for children’s authorization to work with them, and reminded them regularly that they could refuse to participate (which they sometimes did). Children received several gifts throughout the study (such as design probes which they could keep). The researcher informed them about the research results and asked for their comments and inputs. Children expressed their satisfaction regarding participation in the study and using the resulting instructional technologies. They also reported that they felt like the field researcher was listening to them and cared for their opinion. We presented the research goals and methods to parents of the children and they gave informed consent. All teachers and caregivers (e.g., transcription specialists) who participated in the project were compensated for the extra time they put in the project, according to the legal hourly rate.

3.2. Research Questions

This prototype and study were exploratory, as discussed previously. The research questions synthesize the interests of the researcher and the special education teacher involved.

• How can children best be supported in making sense of multisensory material and how can it be integrated more often? What can we learn about children’s meaning-making processes when using smell and taste in the classroom for geography courses?
• Does multisensory material allow for more connections to be drawn between children’s lives, geography field-trips, and the classroom?
• How are these activities valued by pupils and teacher? Can this prototype be used to support collaboration in children with diverse sensory perceptions and diverse background?

3.3. Activity Design

The activity was designed with the special education teacher, as part of her progression in the skills base to be acquired by all French pupils (Domain 5: understanding space and time, human organizations and developing citizenship [31]). The overall design of the pedagogical units and its goals can be found in Appendix C. This session focused specifically on geographic skills (e.g., reading and constructing a map) and revisiting the themes explored in earlier lessons a month before. The main goal was to support a pupil-driven synthesis about the differences between human habitats.

The first activity consisted of collaboratively building a physical layout of different types of human habitats (e.g., a major city, a small city, a village) using the tangibles they had discovered in a previous lesson. Children worked in pairs (C4–C5; C3–C1; C2 made his alone). Children chose the tangibles they wanted to include, and then organized them on the table (see Figure 5). They were to discuss it. The teacher regularly asked the pupils to orally describe what they were doing and why. In the C3–C1 pair, C3 led the activity. In the C4–C5 pair, the special education teacher assisted at every step, as this pair did not manipulate the tangibles much. Both C2 and C3 tried multiple layouts and encompassing or not the wood shape for support. C1 manipulated the tangibles a great deal, but rather as he was using them to tell stories about the elements represented. We will not examine this part of the lesson, but it contributes to explain existing classroom dynamics and particularly to show the difficulties of collaboration.

The second activity was built around Mapsense. The teacher had a list of themes (see Appendix C and Appendix D), which she introduced by a cue to be smelled or tasted. She first asked what they thought it was, and what it represented. Each cue was associated with a step of the field trip undertaken some time earlier. The pupils had time to discuss freely or to use the map to find what the cue is associated with. She then asked a question (e.g., why do we build cities?) to which she indicated they could answer using the cue and what they had just discussed. Note that only one child at a time could interact with MapSense (although they disregarded this rule several times), and the others were helping by providing suggestions or making hypotheses.

3.4. Participants

Five pupils participated in this study. Their demographic and other information are reported in the

Table 1. Information about the children who participated in the design of the probe and the study.

Id	Gender	Age	Grade	Impairments	SES
C1	M	10	3rd grade	Blindness with light perception, motor, memory difficulties and psychological impairment	Disadvantaged
C2	M	11	5th grade	Blindness with light perception	Disadvantaged
C3	M	11	5th grade	Severe visual impairment and dyslexia	Middle class
C4	M	10	4th grade	Severe visual and hearing impairment	Advantaged
C5	M	11	4th grade	Blindness with light perception, multiple learning difficulties	Disadvantaged

. They attend different mainstream schools part-time (i.e., 3–4/5 days of the week) and the same special education classroom the rest of the week. Their time in this special education classroom is dedicated to developing specific skills (e.g., reading braille) and attending rehabilitation sessions (e.g., mobility and orientation). They are either perceived as being able to perform well academically but lacking motivation and resilience, or as unable to follow the same curriculum as their peers. The socio-economic status (SES) reported follows the nomenclature used in French education statistics. It includes four categories: disadvantaged, middle class, advantaged and highly advantaged, which is defined by the household referent person’s occupational status.

We chose to work with these children for two reasons. The first was pragmatic: they were the students of the special education teacher involved in the research project and would attend the same classroom for two years. This meant being able to build a relationship with them and to observe changes on the long run. The second was that it was a very diverse group on many aspects; except gender, which is partly due to a gendered imbalance in the prevalence of visual impairments and partly to gender differences in experiences of schooling and modalities of schooling thus proposed by carers. This diversity poses additional challenges in pupils’ collaboration, which is an interesting way to test the limits of a given intervention.

A special education teacher with seven years of experience was involved in the study. She designed the activity led, asking for the field-researcher’s inputs as to how to best make use of the map. She tasked her with sustaining children’s engagement by asking questions or answering theirs.

3.5. Procedure

The children were familiar with the interaction principles. The audio-tactile map had been in use for over a year [24] and we had organized a similar activity 10 months earlier [25]. The teacher reintroduced the map and described its functioning (particularly the menu) before beginning the activity. We assisted children when they encountered issues with the prototype (mostly unrecognized taps) and took over the class discussion when the teacher focused on explaining a concept to one child in particular. Immediately after the lesson and a few days later, we interviewed to get her analysis on how the activity unfolded and on perceived learning outcomes. We also interviewed each participating pupils a few days after the lesson.

3.6. Analysis

The data on which the analysis was conducted consisted of:

• 1:41 h of video from the third lesson of the sequence (see Appendix C);
• Three interviews with the teacher: one before the three lessons to define the content, two after the lesson presented here. These last two interviews investigated among other things her assessment of learning outcomes;
• Semi-guided interviews with the children, conducted after this study. We discussed the session, what they remembered from it and the goals of learning (this last topic was useful for understanding how they perceived this unit compared to their other school experiences).

To analyze the activity, we relied on multimodal analysis [32]. It provides concepts and tools to study interactions and the construction of meaning. We first transcribed the video data to identify:

• Broad phases in the lesson: subtasks, moments of switching from a theme to another;
• Modes of interaction: overall classroom setting including the types of interactions with the map, speaker, utterance, gestures, gaze, olfactory representation, gustatory representation, and tangibles. Here, gaze is mainly denoted by body orientation, which suggests efforts to listen, focus on manipulation, etc.

An excerpt of the transcription can be found in Appendix B.

This type of transcription enables seeing similar interactions in different events, which gave shape to the themes outlined in results. Our iterative analysis focused on semiotic chains, i.e., the diverse association of meanings across different representations and modes. For example, what kind of associations do the different types of representations elicit? How does it inform us about the potential uses of smell and taste in the classroom? We also focused greatly on the spatial occupation of the classroom: the different representations and artifacts may have intrinsic qualities supporting a specific emergence of meaning, but they are also spatially distributed in the classroom, which makes them individual or group resources, enabling certain kinds of social interactions but not others. In the analysis below, we sometimes use observations from the earlier ethnographic study, to contrast or complement the observations made in this lesson.

4. Findings and Discussion

All pupils reported having enjoyed the activity. They all engaged with each of the resources brought to the classroom. We structured the findings around four key themes to understand how Mapsense reshapes learning: smell and taste in geographic reasoning; children’s efforts to reclaim the space of the classroom; personal and emotional engagement; and a more general discussion of the implications for citizenship education.

4.1. Smell and Taste: (in)Congruities

In this section, we look at how olfactory and gustatory representations were interpreted, and how this interpretation was intertwined with the other representations of the same concept (e.g., tangible objects, tactile map). We discussed the Proust effect above, and we propose here to be attentive to the congruities identified by children across multiple situations when interpreting olfactory and gustatory cues, to how they can be part of “the re-representation process incorporat(ing) the interaction and context of preceding activities and project(ing) them into subsequent activities” [33].

For instance, during the first lesson of the unit (see Appendix C), the children visited a farm dedicated to the raising of livestock, which they identified primarily through smell. During the second lesson, they manipulated a tangible representing a farm building. In the third, the farm was represented by the model, the sounds of farm animals played on the map and the goat cheese. During the classroom discussion, we aimed at getting children to formulate that farming was an economic sector that encompassed multiple activities related to the production of food. Note that in French, their names are not related: ferme for farm, and agriculture or agriculture et élevage for farming. The teacher asked the group if they knew other places that smelled like a farm. C1 answered that he knew a similar place that smelled of olives. This was a recall of a class trip that occurred nine months prior, during which they visited an olive farm. The smell of an olive farm is extremely different, so the convergence between the two was not easy to perceive—except for the fact both are farming activities—a concept that children were then able to manipulate to apply to another cue (strawberries) and to a different question (“Why do we build cities?”). However, the convergence might actually be found elsewhere: this might be an especially strong association because visiting farming spaces often includes a time dedicated to tasting the produce of the farm. We discuss this below, in connection to children’s spatial, emotional and social lives.

However, tastable and smellable representations seem useful to open up a breadth of meanings—or, in the words of C3, to encourage “answers that sound silly”. An example of these types of answers: To the question “what feels different in the city and the countryside?,” C2 answered “the lack of space”. However, upon the presentation of rubber from a tire, he gave a different answer (“pollution!”) and went on explaining and discussing with C5 why cities feel more polluted:

C5: “it smells like cars”

C2: “because there are more cars, because there are more humans”,

or, in other words, less space for individuals. C2’s initial answer was a reference to a bodily and personal experience, but it is the smell of rubber that makes it fit into a larger pattern Let’s note the pollution due to agricultural activities was not discussed. It creates a false dichotomies between cities and countryside, or culture and nature. However, this should be understood as a first step to understand the link between human activities and pollution, which harnesses children’s beliefs about the difference between the countryside and cities, enabling appropriation. Here, the congruity between the experiences and the representation in the classroom is very visible: C2 discussed the lack of space as affecting the ability to breathe—When manipulating the rubber C5 turned his head away to signify disgust, and C1 agitated his hands under his nose as if he was dissipating the odor. This is a particularly evocative representation of pollution. It is also an unpleasant one, which might be a limit of this approach (see Section 5).

We had indeed included rubber in the lesson to suggest urban pollution and more specifically to discuss the roles of urban centers in the development of the industrial sector. Its interpretation as a proof of car pollution and population density is interesting because it connects different aspects of the same issue i.e., the spatial distribution of jobs that affects their family lives. In other words, the potential uses of smell and taste in education go far beyond their use as a memorization technique or as a tool to lessen learner’s anxiety. They enable a situated interpretation and learning process, and grounds learning in pupils’ everyday lives.

These two examples inform us on olfaction and taste as modes too. There can not be a single interpretation of these cues. They are dependent on the learning context in which they are introduced to a higher degree than, say, a map. The map has a legend and many indications about what it represents and how it fits with other representations of the same kind (e.g., indications of scale and orientation). Here, this context is provided by the themes of the unit and the other representations available in the classroom.

4.2. Reconfiguring Space

In this section, we turn to the spatial interactions enabled by the apparatus (see Figure 3 for a visual depiction of the classroom setting). We address this at the level of the classroom and at the level of the map. MapSense shaped what was allowed in the classroom, especially moving around. In addition, the fact that only one child at a time could interact with the prototype offered new pedagogical opportunities.

First, a few notes on the roles undertaken by C1 in the classroom. This pupil usually is not eager to participate in group activities. However, his known sensitivity to olfactory stimuli became part of a collaborative construction of knowledge. C3 particularly marked his interest to support C1 and C5’s participation in the follow-up interview (“they feel good, they dare to say things”). Although he was usually withdrawn in the classroom, we noted a very different pattern of engagement during this lesson: he turned towards his peers and made tangibles fly to nearby desks…

As Mapsense is currently implemented, it is preferable that only one child at a time interact directly with the tactile map. When several pupils interact, the audio indications are played at the same time, rendering them inaudible. In this activity, the pupil interacting directly with the map was verbally guided by the others and was in charge of verifying or generating hypothesis by interrogating the map. During this time, the other pupils were free to manipulate the unused tangibles scattered on their five desks. The tangibles were especially used to compare scales and quantities—for example, comparing the tower’s height with that of the countryside house, or the number of buildings in the city. However, to do so, they had to be exchanged or manipulated, which requires moving around the classroom during discussion. Furthermore, not only does Mapsense require taking a turn, but if a pupil thinks he knows how to find the needed information from the map, he is invited to guide the hand of the current user, or bring him the related tangible. Space was thus constantly renegotiated, as a result of Mapsense’s integration in the classroom. This is in stark contrast with the previous lessons observed. Fixed seating is generally considered an asset because it enables finding needed material (e.g., pens, documents) quickly. Additionally, it provides new perspectives from classroom-size geographical representations.

Let’s discuss this adaptation process. When it was initially introduced in the classroom, MapSense was used by the teacher as a tool to discover the general content of the map. One student interacted with the prototype, while the teacher described the current interaction, with all other pupils surrounding them. The rest of the lesson relied on individual paper maps distributed to each pupil. Later on, the teacher developed a more collaborative pedagogical scenario: the student exploring the map described his actions and perceptions, while the others reconstructed the exploration on their own paper map. Finally, during this study, MapSense was used as a tool for collaborative inquiry, which the teacher described as a step towards understanding its full pedagogical potential. This changed the way the map was explored.

On a tactile paper map, or on an interactive map without tangible interactions, these pupils are taught to read linearly (e.g., starting from left to right and top to bottom). Carers often argued it enables developing a survey knowledge of space, providing more precise and generalizable representations. With MapSense, pupils were rather encouraged to focus on finding a point of interest or to discover the map by following routes that interest them, i.e., to develop a route-based knowledge. Previous studies have suggested that those two kinds of knowledge are complementary and build upon each other [34].

What a route-based exploration of the map enables is the grounding of personal stories in a global representation. As planned, the pupils first tried to follow the previous class trip, which started from the special education center on the map. C4 proceeded to find the nearby bakery, from which the pastries they had eaten came from and is a popular place for them. We had made a point of interest for this bakery, as we know their attachment to this place.

4.3. Making the Classroom Pervasive to Children’s Lives

As discussed above, Mapsense is inspired by a model of education situated in children’s experiences and everyday lives. Hence, the decision to use children’s audio recordings, and the route-based reading of the map. Additionally, with the teacher, we had hypothesized that using smellable and tastable material participates in fostering engagement with the content of the course. It also facilitated the introduction of children’s narratives about their spatial, emotional and social lives.

We can identify three types of experiences to build on: joint experiences, that the group has had together; shared experiences, such as experiences of similar places, which they compare and talk about; and individual experiences, which they do not necessarily like to share. For instance, C5 did not share his wish of traveling as an adult during the lesson. From here on, we will try to identify implications for socio-constructivist pedagogies, demonstrating that different forms of learning are at work. To do so, we follow the typology outlined by [35]: “Knowledge construction recognizes that individuals create meaning for themselves rather than just receiving it preformed from others; collaborative knowledge construction more specifically locates this meaning-making in a group context; intersubjective learning further specifies that the process of meaning-making is itself constituted of social interactions; and knowledge building requires that this group-based meaning-making is being done intentionally.”

4.3.1. Children’s Spatial Lives

This case study was on regional geography and we had designed the map to reflect what we knew of pupils’ joint experiences of the neighborhood (e.g., we represented the postbox, grocery store and bakery they like), so it is not surprising that children had related stories. The lesson apparatus afforded a way of grounding their stories in social and spatial structures—but also to reflect on their own lives using geographic tools.

We discussed above the interpretation of rubber as a sign for pollution and how it opened up a space for discussing familial struggles with spatial mobility. The stories told by children often focus on the spaces they know best: their home and the journey to the service provider’s building in which this lesson unfolds; the direct neighborhood of this organization (the park, the bakery, the canal, the post box, the museum and its mummy); the mainstream school they attend; previous field-trips; family members’ homes. They can all be situated on the map, or evoked in relation with the tangibles (e.g., C5 “Me, I live in a building”) and the smellables and tastables (e.g., C1 “We get pastries when we pass by the bakery!”). Those are shared and joint experiences that allow for intersubjective learning and knowledge building.

Intersubjective learning is not restricted to the group of pupils or the classroom C5, in a follow-up interview: “Eating, and the sounds, it helps me understand, and think. Maybe it helps me remember. But that I don’t know. It helps me think about things I can do when I grow up in a long time.” We discussed cognitive theories, arguing that smell or taste might foster memorization when associated to a chunk of information. We argue this is different: here, learning is “a process that constructs personal identity, entwining individual learning with group practices that themselves can change” ([35], p. 4). C5 memorizes what he finds useful for the future, that coincides with what he thinks adults do, which will enable him to participate in society.

4.3.2. Children’s Emotional and Social Lives

In C5’s case, the use of food is especially relevant. He is overweight and often experiences negative remarks when eating, from peers and carers. During a previous discussion, he expressed his frustration: “people say it’s my fault if I’m fat, but it’s not!”. This was in contrast with the situation in the classroom where tasting food was a valid way to contribute to the activity—we also had chosen the pastries as tastables to provide an enjoyable lesson experience. C5’s bodily, emotional and social lives are here intertwined. Food access and health are strong markers of one’s socio-economic context [36,37]. It is not only his weight that sparks remarks, but also the fact that carers attribute it to faulty parental food practices that need to be corrected for C5’s good. Using tastable material necessitates taking stock and engaging with these highly social and personal experiences of food—which can become a subject of geography.

There is ample work on mobile and multimodal technologies to scaffold learning and reflection in the wild. However, less attention is given to bodily experiences and emotions in classroom-based activities. This was the case of the earlier example on rubber, pollution and bodily experiences of the city. In the follow-up discussions, the group discussed spatial inequality. C2 talked about feeling betrayed, after being promised that life would be better in the city but having to live in cramped high rise towers: “Cities are for stockpiling people,” he said. For those living in the countryside (C3 and C5 especially), the main concern is the lack of potential jobs. This apparatus provided an outlet for vivid social experiences related to the differences of human habitats. We would argue this opens up an opportunity for them to understand the impact of policies, but also to transform individual issues into community ones, i.e., to support knowledge building communities. We note however that this was not an aim of the teacher, hence that it should here be considered as intersubjective learning.

Spatial experiences are imbued with emotions [38]; and emotions are both personal and intra-personal processes [39]. Let us clarify: to identify a given bodily experience as an emotion, one must have the ability to name it, borrowing from a set of available cultural resources. For example, pastries are not only appreciated because of the sugar they contain but also because getting them is a social event; when they find cities suffocating, the pupils refer to familial discourses and academic discourses. Cities are this way because they have “too much CO${}_2$ [carbon dioxyde]” (C2), but also because their parents say so (C1). The interest of this activity is that, by asking explicitly to discuss experiences, it lays bare the many associations of meanings, the value judgments, made by children in their everyday lives. It confirms our previous observation about this apparatus as the support and the tool for intersubjective learning.

4.4. Opportunities for Citizenship Education

Therefore, it affords many opportunities for an emancipatory curriculum and the development of knowledge building communities, or at least for an education oriented towards the development of citizenship. This includes fostering integration in society by raising societal and cultural questions. Another example of this was related to gender equality... and strawberries. The strawberries and the pastries were there to discuss the three economical sectors. However, C2 refused to eat them, protesting they were for girls only. When asked by the teacher why he thought that, he could not answer. It transformed into an unforeseen opportunity to address gender biases for the teacher. She asked all the other pupils whether they liked strawberries, and the group concluded that boys like strawberries too. Previous research has demonstrated that boys are actively discouraged from expressing and reflecting on their emotions, which creates behavioral issues [40]. Creating a space where expressing emotional experiences is safe thus can impact different aspects of gender inequalities.

The multisensory approach facilitated discussions about situated social issues needing to be addressed in this context, but go beyond the geography curriculum. However, it is not without difficulties. Addressing any of these issues in the classroom is not necessarily well-perceived. For instance, a book made by the French government to address gender inequalities in all classrooms was censored in 2014 [41]. Pragmatically, teachers might prefer to prepare and control debates on certain subjects. The breadth of meanings emerging from the use of smellables and tastables is not necessarily considered desirable.

Then, how might this generalize to material pupils have never encountered before? Do these modes depend on previous life experiences? Even the licked envelopes, although they had never been used with the children, were not completely unknown: they have gone to the post-office or the post-box before. As far as can be known from this study, this type of material for not-yet-experienced geographic topics and spaces opens up potential futures as much as it supports engagement with past and present issues.

5. Perspectives and Future Work

In this section, we propose to outline implications for practitioners, the design of multimodal, multisensory, systems as well as a few perspectives for future research. Let’s begin with implications for deployment in other contexts. For those who would want to implement these types of approaches, there are a number of difficulties and questions to address, especially regarding larger groups of pupils. Is it possible to use this type of system with a larger group, or should it be divided into smaller groups? Would a mixed-gender group react the same way? How can all pupils be made to feel comfortable to talk about their spatial experiences? Based on our preliminary experiences, forming small groups is preferable in order to make pupils comfortable, but further research is needed on the design of adequate learning activities.

Moreover, the use of smell and taste raises specific issues of trust. Some are practical and can be addressed during the implementation. For instance, if we were to use a device to simulate taste, we would need to prove that the device is perfectly clean before using it or the parts used would need to be disposable (which was the case of MapSense and its spoons). Other issues are more difficult to address. For instance, how do we handle likes and dislikes in terms of flavors? Do we have to only choose eatables that children are likely to appreciate? How do we introduce unknown tastes in the classroom?—hence the issues of trust, and its entanglement with pleasantness. The smell of the rubber was highly unpleasant. However, C2 notes that, in this context, trust having been acquired, this unpleasantness is acceptable (C2: “that’s at school so it’s okay to try”). Another line of inquiry is the use of multisensory interactions for creating pleasant learning experiences. It is consistent with the installation of multisensory rooms in special education institutions, which we previously discussed in [42]. In that case, there would be a need to research and accommodate group preferences as well as their potential evolutions. Moreover, given the scent-free policies are implemented in public spaces in different countries, the use of smell to create an atmosphere might be a difficult endeavor. Finally, we make a note on the choice of tastables and smellables, in order to complement the description of our process in Section 2. Here, the choices were simple because the lesson was based on a field trip, during which we had observed with the teacher the kind of cues pupils had noticed. However, the activity could be to ask pupils to bring something to eat or something to smell that they think is related to the theme of the next lesson. This study focuses on reviewing and synthesizing previously studied concepts, but a previous deployment was for introducing a new topic. Those are the two types of lessons for which this apparatus seems best suited, at this stage.

Regarding the design of multimodal systems, we hope this case study convincingly demonstrated that these sensory experiences should be considered as culturally shaped, and potentially useful modes of communication. There is still much work to be done on how smellables and tastables can become useful representations, especially in other disciplines. These modes may be primarily suited for teaching social and human sciences. Although baking can be used to teach some areas of mathematics [43] or physics or chemistry [44], it does not exploit the serendipity of this type of material that makes them so useful in our case study. Those are both design and didactic questions that should be developed in future research. Another approach would be to focus on skills. For example, Metatla et al. [45] has proposed the use of multisensory systems to support storytelling and social skills.

6. Conclusions

Our study shows how the use of ‘smellables’ and ‘tastables’ as representations in geography can lead to grounding geography concepts in personal experiences—and this can enable children to reflect on how their lives are shaped by larger spatialized dynamics. They are never interpreted alone: their interpretation emerges from the creation and association of meanings across the map, the small scale models and the collective discussion. They seem to elicit more personal stories than any of these other modalities. We can hypothesize that welcoming the use of smell and taste in the classroom legitimates these recollections—maybe because they are deeply rooted in everyday experiences.

These findings can inform the design of learning activities or of learning technologies. Smell and taste are not commonly used in technologies given the technical difficulties they still pose. However, as has been shown, they can be integrated into learning activities. With the growing interest in embodied learning experiences in many curriculum areas, this study can inform future research and practices from two aspects.

Firstly, it shows how to shape the meanings associated with a given cue. Whereas the interactive tactile map acts as a frame of reference, olfaction and taste are invitations to deepen a specific theme through discussions. In contrast, audio cues made by children and used on the map seem to support a different type of engagement [25]—one that is more playful and imaginative. Though both hold promises of bridging experiences and classroom-based learning, they do so in very different ways.

Secondly, smell and taste are not necessarily useful for all curriculum areas. However, their use in the classroom blurs the difference between in situ and classroom-based learning, which can open up new perspectives at a time when there is a push towards less costly and easier to manage virtual field trips. Instead of going for primarily visual experiences of landscapes, this is an invitation to envision a different path for experiential learning in the classroom, one that would take into account sensory diversity in learners.