Flow, Staging, Wayfinding, Personalization: Evaluating User Experience with Mobile Museum Narratives
Abstract
:1. Introduction
2. Related Work and Issues Identified
2.1. Content Delivery
2.2. Screen vs. Physical Space
2.3. Location-Awareness and Navigation
2.4. Interface, Interaction and Usability
2.5. Social Dimension of the Visit
2.6. Personalization and Adaptivity
3. Overview of the Experiences
- Collecting all kinds of data: Museum visitation numbers, demographics, museum visitor habits, styles of visit, context of a visit, web statistics, etc.
- Studying the research literature on visiting styles (and finally adopting the ant, fish, butterfly, and grasshopper model by Véron and Levasseur [36]).
- Carrying out (by an experienced ethnographer) fieldwork that included the exploration of sensitizing concepts and the conduct of in-depth interviews with explainers, curators, educators, receptionists, etc. [37].
- Observing visitors’ habits and behavior in the specific museum setting [37].
- Organizing authoring workshops with interdisciplinary teams of researchers and practitioners (a total of more than 80 professionals engaged in the course of three years), using participatory ideation techniques and “bodystorming” situated in the museum space [38].
4. Evaluation Methodology
4.1. Participants
4.2. Data Collection-Materials and Methods
- A general, pre-experience demographics questionnaire, administered verbally in the form of an interview;
- video and audio recording and note-taking of the observation of visitors' behavior throughout their visit, i.e., from reception and first contact with the system (which begins with the CHESS Visitor Quiz), through to a complete experience of the story in the Museum’s galleries, and the handing back of the mobile device (Figure 2);
- two semi-structured post-experience interviews per visitor/group, conducted immediately after the end of the visit. The interviews were delivered in a conversational tone in order to draw visitors out on what they experienced. The questions that served as guides (see Supplementary Documents S2) were adapted versions of instruments to measure UX and visitors’ experiences via technology in museums [12,40];
- system logs which made time stamped records (time stamps synchronized across devices) and measured such things as interaction session start and end times and total time spent on experience, number of story branches traversed, etc.
4.3. Procedure
4.4. Data Analysis
5. Findings
- Story Plot and Flow: Was it clear, interesting, exciting, and immersive? Were the audio-visual assets appropriate?
- Digital vs. Physical Space: What did the visitors focus on and for how long?
- Wayfinding and Navigation: Did visitors have difficulty locating exhibits or finding where to go?
- User Interface, Interaction and Usability: Was the interface (device) easy to use and interact with, were there external issues hindering the experience?
- Pre- and Post-visit Activities: Did visitors wish to experience content at a later time? Are people interested in preparing their visit or re-visiting later?
- Social Aspects: How was the experience with other people organized around the device?
- Personalization and Adaptivity: Which aspects of the experience related to the user model, profiling, menus and preferences, and other features of each story version were appreciated by the visitor? Did the visitor appreciate (i.e., understand) the personalized experience overall?
5.1. Story Plot and Flow
“I really don't care about "little statues half broken". And extracting information from the labels next to them is hell for me. But this was the very first time I did care about them, because of the story.”[B.u07m]
“I want my museum experience to be fun.”[A.u19f]
“This was exciting. It got to me.”[B.u27m]
“I loved it! No comparison with roaming around the museum alone.”[A.u05f]
“I liked the humor and the fact that it was talking to me, it was like a dialogue.”[B.u08f]
“It was narrated in a way that made things interesting.”[B.u14f, a 10 y. old girl]
“Not all the activities are related to horses but it is not a problem. Mythology is interesting.”[A.u02m]
“Oh no, my time is up!”[A.u14f, a 12 y. old girl]
“I loved the character, the narration, the voice. It was clear, not an academic text.”[B.u15f]
“Very interesting because I have been here before and there were things I knew nothing about and were highlighted and I saw them in a different way. It was immersive in a way, because it was so interesting.”[A.u08f]
“I have been here many times but had not noticed Hippalektryon (the horse-rooster statue). It is a big success that I now noticed the exhibit because I can realize its importance and rarity.”[A.u06f]
“I think it was bits and pieces, a narration with different flows.”[B.u01m]
“I expected that the things said should be related to the plot, the frame of helping horses, but not all were.”[A.u03m]
“There was a theme rather than a plot, with subplots.”[A.u07m]
“It kept asking me to go on with her "story" but I was wandering what the "story" is all about.”[B.u04m]
“With this guide, I experienced only a selection of the museum exhibits. 99% of the exhibits were missing from the story; it felt like cheating the museum.”[B.u18f]
“The story was nice and what you had to do here was nice. […] But I would've liked it more if you could feel “the rush”. Maybe more sound effects would help to make you feel that you are more part of it.”[A.u09f]
“The story was interesting but just standing there listening, made me feel awkward like something was missing.”[B.u01m]
“Moving is interesting, standing is not.”[A.u15m]
“I would have liked some more movement, particularly at the last part of the visit.”[A.u06f]
5.2. Digital vs. Physical Space
“While standing there I was constantly looking at the screen. I was afraid that I would be shown something and miss it.”[A.u20f]
“I felt like I had to look at the screen even though I did not want to, even though my eyes were forced to look around.”[A.u04f]
“I sensed my eyes drifting to the exhibits. I wanted more interaction with the exhibits.”[A.u06f]
“Too little time with the exhibits because I was looking at the exhibit only when there was an image of the exhibit on the screen.”[A.u03m]
“I need clear directions where to look because otherwise I am absorbed by the screen.”[A.u06f]
“It's difficult to have 3 eyes, one on the tablet, one on the statue, one on its label. I ignored the screen.”[A.u07m]
“At some points I felt awkward just having to look at the screen all the time. I wanted to know specifically where to look.”[A.u04f]
5.3. Wayfinding
5.4. User Interface, Interaction and Usability
5.5. Pre- and Post-Visits
5.6. Social Aspects
5.7. Personalization
6. Discussion
6.1. Story Plot and Flow (Story Perspective)
6.2. The All-Consuming Screen (Staging Perspective)
6.3. Moving and Searching through Exploration
6.4. Varying Levels of Control According to Interaction Style (Interaction Perspective)
6.5. Before, During, and After Continuity and Sense of Ownership
6.6. Experiencing with Others (Sharing Perspective)
6.7. What Type of Visit Am I in the Mood for Today?
6.8. Limitations of the Evaluation
7. Conclusions
Supplementary Materials
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
Appendix A
Appendix A.1. Horse Story—Brief Description and Excerpts
Selected Excerpts from the Story Follow
Appendix A.2. Theseus’ Story—Brief Description and Excerpts
Selected Excerpts from the Story Follow
Appendix A.3. Melesso’s Story—Brief Description and Excerpts
Selected Excerpts from the Story Follow
Appendix B
Connect (to Plot) | Approach | Engage | Experience | Disengage | (Re-) Connect (to Plot) |
---|---|---|---|---|---|
Transition between the overarching story and the exhibit-based information | A teaser to draw the visitor in + Instructions on navigation to the point of interest | Practical: A shift of focus from the overarching story to being involved in the exhibit-based information-telling the system you are there Conceptual: Drawing initial interest in the story | Draw attention to details of physical exhibits (or digital assets) + Deliver main story content | Closure of the exhibit-based information segment. Generic phrases that do not affect the plot but give some sense of story e.g., “But I’m getting carried away” | Transition between the overarching story and the exhibit-based information |
References
- Pine, B.J., II; Gilmore, J.H. The Experience Economy: Work Is Theater & Every Business a Stage; Harvard Business School Press: Boston, MA, USA, 1999. [Google Scholar]
- Hassenzahl, M. User Experience and Experience Design. In Encyclopedia of Interaction Design, 2nd ed.; Interaction Design Foundation: Aarhus, Denmark, 2012. [Google Scholar]
- Law, E.L.-C. The Measurability and Predictability of User Experience. In Proceedings of the 3rd ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS), Pisa, Italy, 13–16 June 2011; pp. 1–9. [Google Scholar]
- Wright, A. Storytelling and Interpretation: Examining Best Practice at Heritage Sites; University of York: York, UK, 2017. [Google Scholar]
- Perry, S.; Roussou, M.; Economou, M.; Young, H.; Pujol, L. Moving Beyond the Virtual Museum: Engaging Visitors Emotionally. In Proceedings of the 23rd International Conference on Virtual Systems and Multimedia-VSMM, Dublin, Ireland, 31 October 2017. [Google Scholar]
- Johnsson, E. Telling Tales. A Guide to Developing Effective Storytelling Programmes for Museums; Adler, C., Ed.; London Museums Hub, Museum of London: London, UK, 2006. [Google Scholar]
- Tallon, L.; Walker, K. (Eds.) Digital Technologies and the Museum Experience: Handheld Guides and Other Media; AltaMira Press: Lanham, MD, USA, 2008. [Google Scholar]
- American Alliance of Museums, Museums Association, and Fusion Research + Analytics. Mobile in Museums Study—2012: A Survey of American Alliance of Museums (US) and Museums Association (UK) Members; AAM Press: Washington, DC, USA, 2012. [Google Scholar]
- Victoria & Albert Museum, Fusion Research & Analytics LLC, and Frankly, Green + Webb Ltd. Understanding the Mobile V&A Visitor: Autumn 2012; V & A: London, UK, 2012. [Google Scholar]
- Natural History Museum and Fusion Research & Analytics LLC. Natural History Museum: Understanding the Mobile Visitor; NHM: London, UK, 2013; Available online: http://www.nhm.ac.uk/resources-rx/files/nhm-mobile-report-july-2013-127910.pdf (accessed on 9 June 2018).
- Proctor, N. Mobile Apps for Museums: The AAM Guide to Planning and Strategy, 2nd ed.; AAM Press: Washington, DC, USA, 2011. [Google Scholar]
- Othman, M.K. Measuring Visitors’ Experiences with Mobile Guide Technology in Cultural Spaces; The University of York: York, UK, 2012. [Google Scholar]
- Smith, L.F.; Smith, J.K.; Tinio, P.P.L. Time spent viewing art and reading labels. Psychol. Aesthet. Creat. Arts 2017, 11, 77–85. [Google Scholar] [CrossRef]
- Kuflik, T.; Stock, O.; Zancanaro, M.; Gorfinkel, A.; Jbara, S.; Kats, S.; Sheidin, J.; Kashtan, N. A visitor’s guide in an active museum. ACM J. Comput. Cult. Heritage 2011, 3, 1–25. [Google Scholar] [CrossRef]
- Damala, A. Interaction Design and Evaluation of Mobile Guides for the Museum Visit: A Case Study in Multimedia and Mobile Augmented Reality Synthesis; Conservatoire National des Arts et de Metiers: Paris, France, 2009. [Google Scholar]
- Pujol, L.; Roussou, M.; Poulou, S.; Balet, O.; Vayanou, M.; Ioannidis, Y. Personalizing interactive digital storytelling in archaeological museums: The CHESS project. In Archaeology in the Digital Era, Proceedings of the 40th Annual Conference of Computer Applications and Quantitative Methods in Archaeology (CAA), Southampton, UK, 26–29 March 2012; Earl, G., Sly, T., Chrysanthi, A., Murrieta-Flores, P., Papadopoulos, C., Romanowska, I., Wheatley, D., Eds.; Amsterdam University Press: Amsterdam, The Netherlands, 2013; pp. 77–90. [Google Scholar]
- Vayanou, M.; Karvounis, M.; Kyriakidi, M.; Katifori, A.; Manola, N.; Roussou, M.; Ioannidis, Y. Towards Personalized Storytelling for Museum Visits. In Proceedings of the 6th International Workshop on Personalized Access, Profile Management, and Context Awareness in Databases (PersDB 2012), Istanbul, Turkey, 31 August 2012. [Google Scholar]
- Rennick-Egglestone, S.J.; Roussou, M.; Brundell, P.; Chaffardon, C.; Kourtis, V.; Koleva, B.; Benford, S. Indoors and outdoors: Designing mobile experiences for Cité de l’Espace. In Proceedings of the 6th NODEM (Network of Design and Digital Heritage) 2013 Conference: Beyond Control—The Collaborative Museum and its Challenges, Stockholm, Sweden, 1–3 December 2013; pp. 89–97. [Google Scholar]
- Rennick-Egglestone, S.J.; Brundell, P.; Koleva, B.; Benford, S.; Roussou, M.; Chaffardon, C. Families and mobile devices in museums: Designing for integrated experiences. ACM J. Comput. Cult. Heritage 2016, 9, 11. [Google Scholar] [CrossRef]
- Wyman, B.; Smith, S.; Meyers, D.; Godfrey, M. Digital Storytelling in Museums: Observations and Best Practices. Curator Mus. J. 2011, 54, 461–468. [Google Scholar] [CrossRef]
- Falk, J.H.; Dierking, L.D. Enhancing Visitor Interaction and Learning with Mobile Technologies. In Digital Technologies and the Museum Experience: Handheld Guides and Other Media; Tallon, L., Walker, K., Eds.; AltaMira Press: Lanham, MD, USA, 2008; pp. 19–33. [Google Scholar]
- Vom Lehn, D.; Heath, C. Displacing the object: Mobile technologies and interpretive resources. In Proceedings of the ICHIM03, International Cultural Heritage Informatics Meeting, Paris, France, 8–12 September 2003. [Google Scholar]
- Grinter, R.E.; Aoki, P.M.; Hurst, A.; Szymanski, M.H.; Thornton, J.D.; Woodruff, A. Revisiting the Visit: Understanding How Technology Can Shape the Museum Visit. In Proceedings of the Computer Supported Collaborative Work (CSCW), New Orleans, LA, USA, 16–20 November 2002; ACM Press: New Orleans, LA, USA, 2002; pp. 146–155. [Google Scholar]
- Lombardo, V.; Damiano, R. Storytelling on mobile devices for cultural heritage. New Rev. Hypermedia Multimedia 2012, 18, 11–35. [Google Scholar] [CrossRef] [Green Version]
- Falk, J.H.; Dierking, L.D. The Museum Experience Revisited; Routledge: London, UK; New York, NY, USA, 2016. [Google Scholar]
- Massung, E. Visitor Reception to Location-based Interpretation at Archaeological and Heritage Sites. In Thinking beyond the Tool: Archaeological Computing and the Interpretive Process; BAR International Series; Archaeopress: Oxford, UK, 2012. [Google Scholar]
- Danks, M.; Goodchild, M.; Rodriguez-Echavarria, K.; Arnold, D.; Griffiths, R. Interactive Storytelling and Gaming Environments for Museums: The Interactive Storytelling Exhibition Project. In Proceedings of the 2nd International Conference on Technologies for E-Learning and Digital Entertainment, LNCS 4469, Hong Kong, China, 11–13 June 2007; Springer: Berlin/Heidelberg, Germany, 2007; pp. 104–115. [Google Scholar]
- Hsi, S. A study of user experiences mediated by nomadic web content in a museum. J. Comput. Assist. Learn. 2003, 19, 308–319. [Google Scholar] [CrossRef]
- Woodruff, A.; Aoki, P.M.; Hurst, A.; Szymanski, M.H. Electronic Guidebooks and Visitor Attention. In Proceedings of the 6th International Cultural Heritage Informatics Meeting (ICHIM), Milan, Italy, 3–7 September 2001; Archives and Museum Informatics: Milan, Italy, 2001; pp. 437–454. [Google Scholar]
- Damala, A.; Cubaud, P.; Bationo, A.; Houlier, P.; Marchal, I. Bridging the Gap between the Digital and the Physical: Design and Evaluation of a Mobile Augmented Reality Guide for the Museum Visit. In Proceedings of the 3rd International Conference on Digital Interactive Media in Entertainment and Arts—DIMEA ’08, Athens, Greece, 10–12 September 2008; ACM Press: New York, NY, USA, 2008; pp. 120–127. [Google Scholar]
- Moussouri, T.; Roussos, G. Conducting Visitor Studies Using Smartphone-Based Location Sensing. ACM J. Comput. Cult. Heritage 2015, 8, 1–16. [Google Scholar] [CrossRef] [Green Version]
- Woodruff, A.; Szymanski, M.H.; Aoki, P.M.; Hurst, A. The Conversational Role of Electronic Guidebooks. In Proceedings of the 3rd International Conference on Ubiquitous Computing, Atlanta, GA, USA, 30 September–2 October 2001; Abowd, G.D., Shafer, S., Brumitt, B., Eds.; Springer: Berlin, Germany, 2001; pp. 187–208. [Google Scholar]
- Zurita, G.; Nussbaum, M. A constructivist mobile learning environment supported by a wireless handheld network. J. Comput. Assist. Learn. 2004, 20, 235–243. [Google Scholar] [CrossRef]
- Ardissono, L.; Kuflik, T.; Petrelli, D. Personalization in cultural heritage: The road travelled and the one ahead. User Model. User Adapt. Interact. 2011, 22, 73–99. [Google Scholar] [CrossRef]
- Damiano, R.; Gena, C.; Lombardo, V.; Nunnari, F.; Pizzo, A. A stroll with Carletto: Adaptation in drama-based tours with virtual characters. User Model. User Adapt. Interact. 2008, 18, 417–453. [Google Scholar] [CrossRef]
- Roussou, M.; Katifori, A.; Pujol, L.; Vayanou, M.; Rennick-Egglestone, S.J. A Life of Their Own: Museum Visitor Personas Penetrating the Design Lifecycle of a Mobile Experience. In Proceedings of the CHI 2013 Extended Abstracts on Human Factors in Computing Systems—CHIEA’13, Paris, France, 27 April–2 May 2013; ACM Press: New York, NY, USA, 2013; pp. 547–552. [Google Scholar]
- Tolmie, P.; Benford, S.; Greenhalgh, C.; Rodden, T.; Reeves, S. Supporting group interactions in museum visiting. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing—CSCW’14, Baltimore, MD, USA, 15–19 February 2014; ACM Press: New York, NY, USA, 2014; pp. 1049–1059. [Google Scholar]
- Roussou, M.; Pujol, L.; Katifori, A.; Chrysanthi, A.; Perry, S.; Vayanou, M. The museum as digital storyteller: Collaborative participatory creation of interactive digital experiences. In Proceedings of the 19th annual Museums and the Web Conference, Chicago, IL, USA, 8–11 April 2015. [Google Scholar]
- Benford, S.; Giannachi, G.; Koleva, B.; Rodden, T. From Interaction to Trajectories: Designing Coherent Journeys through User Experiences. In Proceedings of the CHI ’09: SIGCHI Conference on Human Factors in Computing Systems, Boston, MA, USA, 4–9 April 2009; ACM Press: New York, NY, USA, 2009; pp. 709–718. [Google Scholar]
- Roto, V.; Vermeeren, A.P.O.S.; Väänänen-Vainio-Mattila, K.; Law, E.L.-C. User Experience Evaluation–Which Method to Choose? In Proceedings of the 13th IFIP TC 13 International Conference, Human-Computer Interaction—INTERACT 2011, Lisbon, Portugal, 5–9 September 2011; Springer: Lisbon, Portugal, 2011; pp. 714–715. [Google Scholar]
- Antoniou, A.; Katifori, A.; Roussou, M.; Vayanou, M.; Karvounis, M.; Kyriakidi, M.; Pujol-Tost, L. Capturing the Visitor Profile for a Personalized Mobile Museum Experience: An Indirect Approach. In Proceedings of the 24th ACM Conference on User Modeling, Adaptation and Personalisation (UMAP 2016), Halifax, NS, Canada, 13–17 July 2016; ACM Press: New York, NY, USA, 2016. [Google Scholar]
- Fosh, L.; Benford, S.; Reeves, S.; Koleva, B.; Brundell, P. ‘See Me, Feel Me, Touch Me, Hear Me’: Trajectories and Interpretation in a Sculpture Garden. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems—CHI’13, Paris, France, 27 April–2 May 2013; ACM Press: New York, NY, USA, 2013; pp. 149–158. [Google Scholar]
- Katifori, A.; Perry, S.; Vayanou, M.; Pujol, L.; Chrysanthi, A.; Ioannidis, Y. Cultivating mobile-mediated social interaction in the museum: Towards group-based digital storytelling experiences. In Proceedings of the MW2016: Museums and the Web 2016, Los Angeles, CA, USA, 6–9 April 2016. [Google Scholar]
- Keil, J.; Pujol, L.; Roussou, M.; Engelke, T.; Schmitt, M.; Bockholt, U.; Eleftheratou, S. A digital look at physical museum exhibits: Designing personalized stories with handheld Augmented Reality in museums. In Proceedings of the Digital Heritage International Congress (DigitalHeritage), Marseille, France, 28 October–1 November 2013; IEEE: Marseille, France, 2013; Volume 2, pp. 685–688. [Google Scholar] [CrossRef]
- Lanir, J.; Kuflik, T.; Sheidin, J.; Yavin, N.; Leiderman, K.; Segal, M. Visualizing museum visitors’ behavior: Where do they go and what do they do there? Pers. Ubiquitous Comput. 2017, 21, 313–326. [Google Scholar] [CrossRef]
Study A (12/12) | Study B (01/14) | |||
---|---|---|---|---|
Age Group | Female | Male | Female | Male |
−11 | 2 | 1 | 1 | 2 |
12–17 | 2 | 2 | 2 | 1 |
18–29 | 2 | 3 | 3 | 8 |
30–49 | 5 | 3 | 5 | 3 |
50+ | 3 | 2 | 2 | 1 |
14 | 11 | 13 | 15 | |
Total | 25 | 28 |
Where the Action is | Key Aspects |
---|---|
Digital & physical space | Story Plot & Flow (1); Staging, i.e., instantiating a particular plot to create a specific user experience, as an interplay of screen and museum space (2); Social Aspects (6) |
Physical space | Wayfinding (3) |
Digital space | User Interface, Interaction and Usability (4); Pre- and Post-Activities (5); Personalization and Adaptivity (7) |
Story Perspective (Plot & Flow) | |
---|---|
Positives | Narrative constructs such as humor, links to everyday contemporary life, an informal tone, and unconventional characters. |
Drawbacks | Story too informational and “static”, even after improvement. |
Implications for design |
|
Staging Perspective (Digital vs. Physical Space) | |
---|---|
Positives | In some cases, visitors felt that the visuals augmented the experience by bringing forth exhibit details that were not visible otherwise or related informational content and artifacts. |
Drawbacks | The all-encompassing screen. |
Implications for design |
|
Navigation Perspective (Wayfinding & Navigation) | |
---|---|
Positives |
|
Drawbacks |
|
Implications for design |
|
Interaction Perspective (Control) | |
---|---|
Positives | Some visitors liked to be guided by the storytelling experience. |
Drawbacks |
|
Implications for design |
|
Temporal Interaction (Pre- and Post-Visit Activities) Perspective | |
---|---|
Positives | Visitors liked the pre-quiz. |
Drawbacks | There was no option for “take away” content or a digital souvenir. |
Implications for design |
|
Social Interaction Perspective | |
---|---|
Positives | The stories were perceived by some visitors, who visited the museum alone, as companions. |
Drawbacks | The stories were individual experiences, not designed for social interaction. |
Implications for design |
|
Personalization & Adaptivity Perspective | |
---|---|
Positives | The visitors liked the idea that the story is personalized. |
Drawbacks | It has not been proven that personalization really made a difference. |
Implications for design |
|
© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Roussou, M.; Katifori, A. Flow, Staging, Wayfinding, Personalization: Evaluating User Experience with Mobile Museum Narratives. Multimodal Technol. Interact. 2018, 2, 32. https://doi.org/10.3390/mti2020032
Roussou M, Katifori A. Flow, Staging, Wayfinding, Personalization: Evaluating User Experience with Mobile Museum Narratives. Multimodal Technologies and Interaction. 2018; 2(2):32. https://doi.org/10.3390/mti2020032
Chicago/Turabian StyleRoussou, Maria, and Akrivi Katifori. 2018. "Flow, Staging, Wayfinding, Personalization: Evaluating User Experience with Mobile Museum Narratives" Multimodal Technologies and Interaction 2, no. 2: 32. https://doi.org/10.3390/mti2020032
APA StyleRoussou, M., & Katifori, A. (2018). Flow, Staging, Wayfinding, Personalization: Evaluating User Experience with Mobile Museum Narratives. Multimodal Technologies and Interaction, 2(2), 32. https://doi.org/10.3390/mti2020032