User Experience in Mobile Augmented Reality: Emotions, Challenges, Opportunities and Best Practices

Abstract

Mobile Augmented Reality (MAR) is gaining a strong momentum to become a major interactive technology that can be applied across domains and purposes. The rapid proliferation of MAR applications in global mobile application markets has been fueled by a range of freely-available MAR software development kits and content development tools, some of which enable the creation of MAR applications even without programming skills. Despite the recent advances of MAR technology and tools, there are still many challenges associated with MAR from the User Experience (UX) design perspective. In this study, we first define UX as the emotions that the user encounters while using a service, a product or an application and then explore the recent research on the topic. We present two case studies, a commercial MAR experience and our own Virtual Campus Tour MAR application, and evaluate them from the UX perspective, with a focus on emotions. Next, we synthesize the findings from previous research and the results of the case study evaluations to form sets of challenges, opportunities and best practices related to UX design of MAR applications. Based on the identified best practices, we finally present an updated version of the Virtual Campus Tour. The results can be used for improving UX design of future MAR applications, thus making them emotionally engaging.

1. Introduction

The rapid evolution of technology has brought new opportunities for developing new applications and for interacting with them. Augmented Reality (AR) has become an alternative way to engage users with applications more efficiently [1]. Kipper and Rampolla [2] define Augmented Reality (AR) as a blending of real-world and digital information. This combination of the real world and the virtual world has been harnessed for various purposes, such as marketing, education, entertainment and health care. For example, AR has been used for educational and training purposes across subject areas both in formal and informal settings [3,4,5,6,7,8], and it has been shown to possess a multitude of affordances for learning and training scenarios [7,8,9,10,11]. Despite these success stories, AR (and mobile AR in particular) has not yet fulfilled a previously made prediction of becoming the eighth mass media [12]. There is a significant body of research on AR capabilities in different sectors [13], and the challenges and limitations of AR have been discussed [7,14].

Aligned with the popularity of smart phones and tablets, Mobile Augmented Reality (MAR) has emerged as a mainstream technology and is currently a hot topic among researchers across fields. As in our earlier work [3], MAR is defined here as “a type of AR where a mobile device (smartphone or tablet) is used to display and interact with virtual content that are overlaid on top of a real-time camera feed of the real world”. Virtual content in MAR applications can be context-sensitive and take various forms including, but not limited to three-dimensional (3D) models, animations, annotations and videos. Recently, popular mobile applications with AR features, such as Pokémon Go and Snapchat, have changed the developers’ and users’ perceptions of AR. These applications brought MAR to people’s smartphones in earnest, especially targeting kids and youth. Moreover, the increase in the number of toolkits for creating MAR applications has enabled even non-technical individuals to create their own MAR content [3], thus further strengthening the position of MAR as a mainstream interaction technology.

Although Pokémon Go has been associated with many benefits, such as making people exercise more [15] and helping those who have withdrawn from social life [16], little attention has been put on investigation of User Experience (UX) factors that contribute to positive reception of MAR applications. Following our earlier work [17], UX is defined here as the emotions that the user encounters while using a service, a product or an application. An example of this is the negative emotions experienced by some Pokémon Go players when the AR feature of the game drains the smartphone battery and makes their smartphones slow [18,19].

Although UX of MAR applications have received some attention from researchers (e.g., [20,21]), there is a need for a study that focuses on MAR UX from the perspective of emotions. To address this need, the main objectives of this study are to assess the UX of two MAR applications and to draw conclusions as to what challenges, opportunities and best practices are associated with MAR application development from the UX perspective.

The rest of the paper is organized as follows. We start with an overview of previous research on mobile user experience and its relation to mobile augmented reality. Then, after presenting the research design, we describe two MAR applications—a commercial AR experience and our own Virtual Campus Tour application—that are used as case studies in this paper. In the Results Section, we present the results of mixed-method evaluations of the two case studies and then describe challenges, opportunities and best practices associated with mobile augmented reality user experience. The identified best practices are then utilized to improve the second case study. Before concluding the study, we provide discussion on the findings.

2. Background

2.1. Defining Mobile User Experience

Despite a more than a decade’s worth of research on UX, the concept still suffers from vague and broad definitions, which is reflected in the 27 definitions of UX reported as the result of the Dagstuhl Seminar of Demarcating User Experience [22]. UX is a multidimensional phenomenon [23] in which many factors influence success, and it has been around for many years [17]. Carlos et al. [24] identify users, context, interaction and artifacts as the key elements of UX. Moreover, Hassenzahl [25] refers to UX as the “quality of interactive technology”, focusing on the human and not on the product. Hassenzahl and Tractinsky [26] believe that with the advancement of technology, interactive products and services become not only useful and usable, but also trendy and fashionable. In this study, UX is defined as the emotions that the user encounters while using a service, a product or an application [17]. This definition is a consequence of the recent development of design thinking, which recognizes emotional engagement as a key factor in enabling sustainable use of mobile applications.

In mobile applications, UX requires special attention due to limitations of mobile devices (e.g., screen size) and the context-awareness afforded by the embedded sensors. Various attempts have been made to promote mobile UX through evaluation metrics and design approaches in recent years. Yong [27] recommends UX evaluation methods for mobile devices and proposes that these methods should measure the emotional responses of users on visceral, behavioral and reflective dimensions. Väätäjä and Roto [28] recommend mobile questionnaires for mobile UX evaluation, but point out that in the design of the questionnaires, one must pay special attention to the clarity and simplicity of the structure including questions, icons, items and scales. Pelet and Taieb [29] focus on mobile application design to promote UX, with recommendations on how color contrast can be used to impact the positive effects on behavioral intentions. Furthermore, Dirin and Nieminen [30] propose a usability and UX framework for mobile learning applications. The framework has been proven to promote UX in a mobile-based application by making the application adjustable, delightful and reliable, all of which are factors contributing to the emotional dimension of UX.

The existing mobile UX measurement and design approaches may also be to some extent applied to MAR applications. However, MAR UX design is not the same as mobile UX design. Although emotional engagement, personalization and reliability are important in all applications and services, in MAR UX, users are engaged both mentally and physically in the application. The mental engagement in this context means that users fully concentrate on the application, and short- and long-term memory are engaged in the application. Physical engagement means that users often need both hands, eyes, ears and even the rest of their bodies to be involved with the application.

2.2. Mobile Augmented Reality and User Experience

Previous studies on MAR from the UX perspective are vague, and there is certainly room for further research [31]. The main reasons for the lack of MAR UX research and design guidelines is the recentness of MAR as a mainstream technology. Major advancements in the development of MAR happened during the last decade [32]. The main breakthrough occurred with the emergence of smartphones equipped with advanced technologies [33]. Yet, there are some researchers who have investigated the area of MAR UX, as we briefly explain in the following.

Irshad et al. [31] conducted a review on MAR UX research published in 2005–2014 and presented a categorization of the MAR UX domain, including aspects such as UX as a phenomenon, UX as a field of study and UX as a practice. Furthermore, they promoted User-Centered Design (UCD) as a recommended method for MAR application development. When we reviewed previous research on MAR UX, we discovered that MAR applications, where UX was specifically considered, were developed and researched either through case study evaluations or through MAR application development frameworks. In both cases, the user of the application is typically at the center of the design and development, thus confirming the results of Irshad et al. For example, Rashidi et al. [34] proposed a framework based on UCD for developing MAR tourism applications. In another example, Olsson et al. [21] and Olsson and Väänänen-Vainio-Mattila [35] elaborated through a case study MAR application and came to the conclusion that MAR applications’ acceptance by potential users requires that designers learn about users and their essential needs in advance. This is not unlike most UCD frameworks that involve end-users early in the design process.

Furthermore, usability (i.e., effectiveness, efficiency, satisfaction), which is an important factor in positive UX, has traditionally been associated with mobile application design and development. Aligned with this, Ritsos et al. [20] advocated the importance of usability as a key factor for the adoption of MAR applications by users, whereas Jung et al. [36] focused on the satisfaction attribute of usability while conducting a study on a marker-based MAR application for tourists. The findings of Jung et al. indicated that content, personalized service and system quality affect users’ satisfaction.

To promote MAR usability and UX, some researchers have focused on framework development. For example, Kim [37] proposed a framework for context immersion in MAR applications. The framework is constructed of three contexts: time- and location-based context, object-based context and user-based context. Understanding users’ needs in these contexts is a crucial part of UX design in MAR applications. Aligned with this, Irshad and Awang Rambli [38] proposed a multi-layered MAR framework for achieving positive UX. Their framework is divided into various dimensions of a MAR application, such as product, time, specific context and UX.

Our literature review revealed that MAR UX is attracting interest among researchers, with most of the related publications having been published in recent years. Previous studies indicated that the focus has been on UX and usability evaluation of MAR applications, but we still lack information on how users perceive MAR applications emotionally and what are the best practices for MAR UX designers. Important questions to answer include: “What kinds of emotions arise among users of MAR applications?” and “What concerns do users have when they deal with MAR applications?” The aim of this paper is to answer these questions through evaluations of two MAR case studies.

3. Research Design

The following sections describe the research questions, participants and methods used in the evaluation of the two case study applications, which are described in Section 4.

3.1. Research Questions

It is evident that UX of a mobile application is different from that of a MAR application [39]. To investigate the role of UX in the development and evaluation of MAR applications, this study aims to answer the following research questions through two case studies (see Section 4):

• How do users perceive two MAR case study applications emotionally?
• What are the major opportunities and challenges associated with MAR UX?
• What are the best practices for MAR UX?

We chose to analyze the case studies by a mixed-method approach comprising qualitative and quantitative methods. We explain these methods in the following sections after describing the study participants.

3.2. Participants

The case study evaluations were conducted at the Haaga-Helia University of Applied Sciences in March 2018.

Table 1. Participants of the two case studies.

Case Study	Participants	N	Males	Females	Age (Average)
Case Study 1	Haaga-Helia students	22	13	9	18–31 (23)
Case Study 2	Haaga-Helia students (4) and other students (2)	6	2	4	20–34 (25)

summarizes the demographics of the participants for both case studies. The participants were recruited on a voluntary basis from a user experience design course at Haaga-Helia. No rewards were given to the participants. All collected data were anonymized so that the participants cannot be identified from the results presented in this study.

The participants were experts in using information technology, thus no technology training was provided to them before the experiments. Many participants used Android smartphones, and a few owned iPhones. The experiments were conducted in English because the participants were from different countries.

It is noteworthy that although the sample sizes are relatively small, and we therefore cannot draw statistically-significant conclusions, the quantitative and qualitative results together indicate useful information on the UX of the two case studies. The number of recommended test users for usability experiments varies between three and five. For example, Nielsen [40] states that with five test users, one almost gets close to the maximum benefit-cost ratio in usability testing. Nielsen [40] further promotes qualitative user research for collecting insights to drive the design, not the number of involved people.

3.3. Research Methods: Case Study 1

3.3.1. Data Collection

In Case Study 1, we asked the participants to experiment with a commercial MAR application featuring an interactive cat (described in Section 4.1) and then administered a questionnaire based on [41] about emotions that the participants felt during the experiment. The questionnaire consisted of basic information, such as gender, age and previous experiences with MAR. Additionally, the questionnaire had Likert-scale questions about the participant’s emotions during and after the experiment. Figure 1 presents samples of the Likert statements used for collecting data on the participants’ emotions.

We also conducted short interviews with selected participants about their experiences with the application. We randomly selected nine participants (n = 9, females: 5, males: 4) for interviews among those who finished the experiment in 3–5 min (n = 4), in 6–10 min (n = 3) and in more than 10 min (n = 2). The interviews were recorded as audio and transcribed later. The following questions were asked in the interview:

• Question #1: What was your first impression when you saw the cat?
• Question #2: Did you find it difficult to test the application and bring the cat to the screen?
• Question #3: Would you like to have this type of application in your daily life?
• Question #4: How would you change this application?

The experiment was conducted in a classroom with the following procedure. The participants were given smartphones (Samsung Galaxy S6) and were instructed to download the Arilyn application from Google Play. The participants were then asked to scan the cat picture on a milk package (we had two empty packages of milk in the classroom) or through a cat picture, which, for the sake of accessibility, was presented to the participants through a PowerPoint slide. The duration of the experiment varied between 3 and 15 min among the participants. The participants were given freedom to explore the functionality of the MAR application; therefore, no specific instructions or tasks were given. The researchers observed the participants during the experiment and took notes accordingly. After the participants finished the experiment, they were asked to fill in a questionnaire that was delivered as a hardcopy. After the experiment, a researcher conducted interviews with the selected participants.

3.3.2. Data Analysis

In Case Study 1, we applied statistical tools to the collected quantitative data to assess how the participants felt about the MAR application. The collected questionnaire data on emotions was based on a Likert-scale; thus, a simple proportional analysis of the participants’ answers was considered to be an appropriate approach to represent the users’ emotional states towards the tested MAR application. To analyze the questionnaire data, we used the ToolPak add-in for Microsoft Excel, which provides various data analysis tools for statistical and engineering analysis.

After transcribing the interview data, the participants’ answers and researchers’ observations were analyzed to learn more about the participants’ perceptions and reactions towards the MAR application. The analysis was conducted by applying the method of coding [42] in a word processor whereby keywords were assigned to the qualitative data excerpts. These keywords were, for example, emotional words that the participants mentioned or the emotional expressions that were extracted by observing. The frequencies of the keywords are not reported in this study due to the relatively small sample size.

3.4. Research Methods: Case Study 2

3.4.1. Data Collection

In Case Study 2, the participants used a Virtual Campus Tour application that was designed and developed at the Haaga-Helia University of Applied Sciences (see more details in Section 4.2). We conducted a usability and UX evaluation in a usability lab at the Haaga-Helia University of Applied Sciences. The usability evaluation focused on the performance of a set of predefined tasks to measure the efficiency of the application. The UX evaluation, which was based on observations and post-experiment interviews of the participants, focused on how the participants felt about the MAR application. The aim of this case study was to investigate the emotions that the participants felt about the application and to find out whether the participants found the application easy and efficient to use and whether it fulfilled their essential needs.

Table 2. Case Study 2 test plan.

Plan category	Details
Equipment:	Mobile phone (Samsung Galaxy A8, Android), test instructions
Observations:	Notes of the user’s actions and the time it takes for the user to complete the tasks.
Pre-test:	Introduction and application orientation
Tasks:	Go to Riitta’s room on the 6th floor. Proceed to the library and be welcomed by the guide Go to the 1st floor lobby entrance/ exit door
Test Method:	Facilitator instructs the users of the tasks.
Data Collection:	Video and audio recording, written notes, think-aloud, interviews
Interview questions:	Comments about the hardware Comments about the app. What would they change? Negative and positive aspects? Comments about how users felt while accomplishing the tasks. Emotions? Is it fun to use, interesting, anything surprising? Other comments
Introduction:	5 min
Test Tasks:	4–15 min (Usability and UX)
Debriefing:	5–10 min
Reporting	120 min
Total:	15–30 min per user 1.5–3 h for 6 users

presents the details of the test plan.

In the usability assessment, which was video-recorded, we applied the think-aloud method to learn about the participants’ ways of performing the given tasks. In the think-aloud method, which is rooted in psychology [43], the participants are advised to say what they think while performing a given task. In our experiment, the objective of using the think-aloud method was to give us feedback on whether the interface is logically designed and whether the participants find the components in expected places.

There were two people conducting the Case Study 2 experiment: the facilitator and the observer. The facilitator briefly introduced the application prototype and the purpose of the test to the participants and answered any questions that the participants had about the experiment. During the experiment, the facilitator assisted the participants if they had problems, but without guiding them to solve the tasks independently. Finally, the facilitator interviewed the participants. The observer silently recorded the times of the tasks and observed the verbal language and facial expressions of the participants. The observer also took notes on the participants’ actions, comments, procedural errors and problems and assisted the facilitator in writing down the participants’ interview answers.

3.4.2. Data Analysis

To analyze the data collected in Case Study 2, we applied usability evaluation metrics, such as the time spent on performing the predefined tasks and the number of tasks performed in a given time, to measure the application’s efficiency. Furthermore, the qualitative dataset comprised transcribed think-aloud videos, observations and interview recordings. As in Case Study 1, we applied coding [42] in a word processor to identify emotional reactions and usability issues raised by the participants during the experiment. The frequencies of the keywords are not reported in this study due to the relatively small sample size.

4. Case Studies

4.1. Case Study 1: Aamu Cat

The first case study is a commercial advertisement MAR application by a Scandinavian dairy corporation (Arla), which was in the market during a promotional period (in 2018) in Finland. The customers were advised to download the application (Arilyn) to their phones and scan a picture of a cat embedded in a milk package, thereby activating the AR content. The AR content features a cat, called Aamu (morning in Finnish), that jumps to the phone screen out of the milk package, thus granting the user the opportunity to play and interact with the cat (e.g., stroking). In this application, the Aamu cat first wanders around, and by swiping the screen, the user can trigger the cat to perform different tricks, such as jumping. Figure 2 presents screenshots of the Arilyn application with Aamu. A detailed description of the application and the integrated AR solution are available at https://www.arla.fi/aamukissa/.

4.2. Case Study 2: Virtual Campus Tour

The second case study was designed and developed by two students at the Haaga-Helia University of Applied Sciences. The aim of this MAR application is to provide a virtual tour of the Pasila campus to those students who received admission to the Business Information Technology (BITE) degree program for the fall 2018 semester. The concept design and implementation processes are out of the scope of this paper.

Figure 3 presents the screenshots of the Virtual Campus Tour application. In this MAR application, an avatar of the BITE study advisor, Riitta, orients new students to the Pasila campus, including information on facilities, lecture rooms, the library and key people. The application helps new students to navigate in the first, third and sixth floors where the classrooms, library and teachers’ and advisors’ offices are located. The avatar helps the student select the correct floor based on the place where they want to visit. For the implementation, we used Unity 3D, 3D videos and Vuforia AR. The Android-based application scans a marker, thus causing the avatar to appear. The student starts to move with the smartphone to get to the door of the elevator where more virtual content is shown. The avatar appears in the front of the elevator next to the buttons, and at the same time, a virtual button panel is displayed. The student is then advised to pick the floor where they want to visit.

5. Results

In the following, we present the results of two case study evaluations based on the methods described in Section 3. Then, we collectively summarize the emotions that were identified from the case studies.

5.1. Case Study 1: Aamu Cat

5.1.1. Questionnaire Results

The demographics section of the questionnaire indicated that 15 out of 22 participants had previous experience with MAR, whereas seven participants were inexperienced with MAR, thus suggesting that MAR as a technology has reached a relatively high degree of awareness among young adults.

Figure 4 shows what types of emotions the participants felt towards the Aamu Cat MAR application. The results indicate that positive emotions (relaxing, inspiring, encouraging, exciting) were experienced by most of the participants. Among all participants, a majority (59%) agreed that the MAR application made them relaxed, whereas 14% did not find the MAR application relaxing. Similarly, a majority experienced also other positive emotions, as 73%, 87% and 73% of the participants agreed that the MAR application was inspiring, encouraging and exciting, respectively, with very few negative responses.

Compared to positive emotions, negative emotions gained much less agreements. Among all participants, 87%, 86% and 77% disagreed that the MAR application was unpleasant, depressing and boring, respectively. Only 9–14% of the participants felt neutral about the negative emotion statements. Although low in proportions, it is noteworthy that 5% and 9% of the participants felt that the MAR experiment was unpleasant and boring, respectively.

5.1.2. Interview Results

In the following, we present the results related to the four interview questions that were asked from the selected participants (9).

Question #1: What was your first impression when you saw the cat?

The participants considered their MAR experiment with the milk package and the Aamu Cat to be interesting and expected to see similar applications of MAR more in public places, for example:

“I really like its simple interactions. Cannot wait for it to kick off in everyday public places.”

(Outi, 18, Female)

Some participants identified the MAR application to be intuitive, thus giving more control to the user on how to utilize the interactions in the application, for example:

“The interaction was engaging and with fun, I almost knew how I should play with the cat.”

(Outi, 18, Female)

Several participants considered that the application engaged them emotionally, which, as per our definition, is the essence of positive user experiences:

“It was emotionally interesting and delightful, and having a real connection with the object, through getting it working took a while.”

(Timo, 27, Male)

“Getting the cat out of the box for the first time was cool […] and its fun for a short time.”

(Anastasia, 28, Female)

“The cat jumping out of the box was surprising.”

(Joan, 21, Female)

The participants’ emotional engagement in the application was also visible in the following terms that frequently appeared in the participants’ comments as their first impressions: unexpected, surprised, liked it, fun, friendly and interactive. In addition to emotional engagement, some participants considered that they became involved with AR and the objects within:

“AR is more interactive, and I feel involved.”

(Mia, 26, Female)

“AR is about feeling the objects.”

(Timo, 27, Male)

Question #2: Did you find it difficult to test the application and bring the cat to the screen?

All students managed to perform the experiment successfully despite no detailed instructions given by researchers. Therefore, all students answered “no” to this question. The main concerns that emerged during the experiment were getting the application to work and that interacting with the cat was time consuming.

Question #3: Would you like to have this type of application in your daily life?

Some participants considered that MAR is suited well to advertisements, buying and selling. The applicability of MAR as a learning technology, however, received more skepticism, for example:

“I’ve tried AR in a language learning application, but I dropped the language course [where the AR learning application was used], so I cannot say whether it is helpful. I prefer the normal two-dimensional (2D) view when it comes to language learning. MAR drains battery way faster.”

(Janne, 24, Male)

The following comment from a participant shows that the perceived benefit for the invested time may depend on the user’s age:

“Maybe kids want to play with cats for minutes if not hours, I do not like to spend time with a virtual pet in my phone.”

(Alexey, 21, Male)

Question #4: How would you change this application?

The changes that almost all participants recommended were having more functionalities in the application, such as: communication with the cat, performing more tricks with the cat and selecting different avatars (e.g., a dog). Furthermore, some participants suggested that the cat’s avatar should be more versatile and realistic and that users should have a choice of personalizing the avatar, as indicated in these comments:

“Choose the figure.”

(Sara, 21, Female)

“Different animal, more colorful surroundings.”

(Mia, 26, Female)

“More entertainment actions.”

(Outi, 18, Female)

“I guess some alert that it would popup is good to have.”

(Timo, 27, Male)

“Feel like this is a beta version, I want to have a more realistic figure. Seems that the cat moves very slow.”

(Ivan, 21, Male)

Furthermore, we discovered that the cat avatar promoted positive UX and may even compensate some factors that contributed to negative UX:

“My first AR experience wasn’t surprising because the app lagged a bit but the cat on class today was really nice.”

(Sara, 21, Female)

5.2. Case Study 2: Virtual Campus Tour

In the usability evaluation, we measured the efficiency of the Virtual Campus Tour application usage, i.e., how well and fast the participants find the locations defined in the tasks with the Virtual Campus Tour application. The loading time of the application was around 0.5–3 s from launching until the participant could start interacting with it. It took on average around 10 min (minimum: 5 min, maximum: 15 min) for the participants to complete the predefined tasks. The participants were generally satisfied with the application’s user interface and the way of interacting with the application.

When prompted in the interviews what the participants would like to change in the application, all of them said that the application should have more instructions of where they should go, the names and locations of places/areas and special places, such as laboratory and computer rooms. Moreover, some participants suggested that the application should also have a map or a list of places in the beginning, so users would know what they can learn from the navigation and where they are in relation to the whole school map. Finally, the navigation could be more playful and interesting by applying a game concept, such as missions to find different areas in the school, and including more interactable objects, as well as background music, to make the tour more relaxing and exciting.

The participants’ main objections concerned the application’s graphics and insufficient control over the application’s performance. For example, the augmented elevator panel did not work properly, as the participants needed to touch a specific area on the floor number to activate it. Although non-active keys were dimmed, some participants still attempted to click them. Moreover, it was not possible to cancel a selected key.

In the UX part, we assessed how the participants felt during and after the experiment. In general, the participants found the application concept to be appealing, especially due to the simple and straightforward interaction and intuitive and obvious movement. The emotions experienced by the participants are listed in Section 5.3.

5.3. Positive Emotions Experienced in the Case Studies

Table 3. Positive emotions that the participants experienced in the case studies.

Emotional Impact	Case Studies
Relaxing	The cat made the participants feel relaxed while they interacted with the application. Aligned with this, Riitta’s avatar in Case Study 2 provided a friendly and relaxing environment to the participants.
Inspiring	The cat avatar and the augmented objects inspired the participants to explore more the functionalities of the application. This is one reason why it is recommended to use avatars to engage users emotionally.
Encouraging	The participants were encouraged to try and explore further interaction with the cat or wait to see what the cat avatar does next. Similarly, in the second case study, the avatar helped the participants feel that somebody was there to help, thus encouraging them to explore further and not being afraid of making mistakes or being lost.
Exciting	The participants found the MAR applications exciting. The excitement in the first case study was on the user expectation as to what the cat avatar might do next. The excitement in the second case study was based on the avatar’s behavior and interaction with users.
Delightful	The evaluation results demonstrate that a well-designed interactive avatar and virtual objects made the participants delighted.
Fun	In the first case study, almost all participants (except one) pointed out the fact that they had fun working with the application because of the cat.
Surprising	The participants were surprised by MAR and the virtual objects (e.g., Aamu cat).

summarizes the positive emotions that the participants experienced in our case studies. These emotions were extracted from the data collected by observations, questionnaire (Case Study 1), interviews and the think-aloud method (Case Study 2). It is noteworthy that the diversity of emotions that MAR applications can evoke in users is far beyond those that emerged in this study.

6. Challenges, Opportunities and Best Practices for MAR UX

The following sections present the challenges, the opportunities and the best practices for MAR UX that emerged from our collective analysis and interpretation of literature and case study findings, as well as from our experiences in developing the Virtual Campus Tour application. Then, we apply some of the identified best practices to create an updated version of the Virtual Campus Tour.

6.1. Challenges of MAR UX

There are many challenges associated with UX design that are specific to MAR applications.

Table 4. Challenges associated with user experience of MAR applications.

Challenges	Description
Physical	Often, almost the whole body needs to be involved while interacting with a MAR application. Users need to interact and communicate with the application by clicking, tapping, moving, gesturing or giving voice commands. In non-MAR applications, tapping or gestures are typically used.
Mental	New MAR applications often require that the user installs them first and possibly require special glasses. For example, in the first case study, the participants had to install Arilyn before they were able to scan the cat. Installing new and possibly unknown applications requires extra effort and trust from users. The user should be convinced of the necessity of installing the application. Another mental challenge is the cognitive load caused by the bridging of the virtual and the real world via rich multimedia content. Moreover, the new way of thinking about content and interacting with it can also be mentally exhausting.
Prototyping	Technologies for supporting MAR application design are still in an early phase despite recent developments. For example, we still lack rapid prototyping tools to design low- and high-fidelity prototypes.
Technical	There still are many technical challenges associated with MAR UX that may affect the emotions experienced by the user, including, but not limited to battery drain, required processing power (i.e., how to make it available on low-end devices), screen size and the type of tracking (e.g., marker-based, markerless).
User interface	Designers have already managed to construct user interface and interaction metaphors for non-AR mobile applications. All mobile application design metaphors are not necessarily applicable to MAR applications; hence, new interface and interaction metaphors specific for MAR applications are needed.
Development	To develop a robust MAR application, application developers must utilize multiple integrated development environments. For example, in the second case study, we worked with Unity 3D, Vuforia AR and Visual Studio.
UX design	The mindset change from mobile application to MAR concept design is associated with some challenges. Designers are required to adjust their design approaches, which are based on 2D sketching of non-AR objects using a pen and paper or mock-up tools, towards sketching of 3D scenes and objects that are viewed through a 2D perspective (i.e., the camera). Connected to this, as mentioned above, MAR UX designers lack robust prototyping tools to create prototypes for MAR objects and interaction.
Timing	The experiment demonstrated that the participants get frustrated if MAR objects take too much of their time, especially when the application’s purpose is to entertain the user. The optimal presentation time needs to be carefully assigned depending on the context and the target user group.

summarizes some of the challenges that we learned in this study.

6.2. Opportunities of MAR UX

Despite many challenges, there are significant opportunities that MAR applications bring to UX design. Some of these opportunities, as identified through the case studies, are listed in

Table 5. Opportunities associated with user experience of MAR applications.

Opportunity	Description
Marketing Potential	MAR creates a unique opportunity for practitioners to promote products and services. Applying MAR in product advertisements helps users perceive the product better. Furthermore, it helps users become attached to and involved more closely with the product. For example, the cat avatar aims to establish emotional engagement towards the milk brand. Traditionally, this type of emotional engagement with tangible products has been feasible to achieve through video advertisements. MAR can enable this emotional engagement through richer, two-directional interaction (e.g., eyes, ears, hands).
Emotional Engagement	Emotional engagement, as mentioned above, is a key deliverable that MAR can bring to the user. Emotional engagement was also evident in our evaluation results. For example, some participants in the first case study were observed to be holding an empty milk package with smiles on their faces.
User Interface	MAR provides unique opportunities to create novel user interfaces. Voice commands and touching MAR objects (e.g., virtual buttons) are examples of novel UI components that can be associated with MAR applications.
Interaction	AR enables unique interaction experiences for mobile applications that previously were not feasible, such as voice commands, natural communication with an avatar and rich visualization of UI instead of having text-only labels and 2D graphics to guide the user.
Experience Development	MAR enables designers to create context-sensitive illusions that ease the cognitive process for complex tasks, such as when an interior designer is fitting furniture in a room or when an architect is illustrating architectural models on top of a real-world view of a vacant plot.

.

6.3. Best Practices for MAR UX Design

As MAR applications seek to create a bridge between the real world and a virtual world, UX designers need to pay attention to the special requirements that arise from the interplay between the two worlds. To alleviate this challenge, we have devised a set of best UX design practices for MAR applications based on the analysis of the case studies. These practices are listed in

Table 6. Best MAR UX design practices.

Best Practice	Description	Examples
Spatial correspondence	MAR object dimensions and location should match the intended target used on top of the real world.	In Case Study 2, the MAR panel dimensions and locations were inaccurate, thus causing inconvenience to the participants.
Tolerance to movement	MAR objects should be tolerant to camera movement. For example, if the marker goes out of the camera view, the MAR object should remain visible for a period of time.	In Case Study 1, the cat avatar was sensitive to movement, as it disappeared from the screen when the participants slightly moved their smartphones.
Object detail	The details of MAR objects should be sufficiently high (within the limits of the target hardware) to make them recognizable and appealing.	The avatar in Case Study 2 was a 2D cartoon object. The cat avatar in Case Study 1 occupied the screen properly, which makes the impression to the participants that the cat is the only interactive object in the screen.
Object correspondence	Establishing a correspondence between a MAR object and a real-world object creates a metaphorical link that increases familiarity and might also lower the learning curve.	The avatar in Case Study 2 had very little resemblance to Riitta, the academic advisor at Haaga-Helia.
Natural interaction	Allow users to interact with MAR content by multiple natural methods (e.g., speech, touch, gestures), thus promoting realism.	The interaction in the first and second case studies was based on a touch-based user interface, which is a typical approach in MAR applications [3].
Personalized experience	Making the MAR experience personal to each user increases the likelihood for emotional engagement. This can be done, for example, by allowing the user to customize the application or automatically adapting the application based on the user’s context and preferences.	Both case studies provided a one-size-fits-all experience to all users. The participants were not able to change preferences or otherwise customize the content. In Case Study 1, some participants suggested to add a personalization feature, such as choosing a dog avatar instead of a cat avatar.
Emotion-evoking avatar	It is recommended to include an avatar in MAR applications to make interaction with the application more natural. Moreover, the avatar should have emotion-evoking properties, such as pleasing appearance and body language that signal how the avatar feels.	The cat in the first case study was found to be appealing and was able to evoke emotions among the participants. In Case Study 2, the Riitta avatar did not achieve this, possibly due to the avatar design and lack of interaction.

, together with descriptions and examples. By following these practices, UX designers can increase the likelihood of users becoming emotionally attached to the target MAR application. It is noteworthy that this list is intended to complement, not replace, existing mobile UX design guidelines.

6.4. Applying Best Practices to Virtual Campus Tour

Figure 5 presents an updated version of the Case Study 2 application (Virtual Campus Tour) based on some of the best practices presented above. First, we designed a new avatar for Riitta, whom the freshmen meet when they arrive at the university for the first time. We designed the avatar to be as realistic as possible, thus providing a true reality-virtuality object correspondence and object detail to new students. It is not merely a design transformation from a 2D to 3D object; the MAR object actually represents a real-world entity, in this case a person. Hence, we used an existing and expected metaphor for MAR content representation.

Furthermore, to make MAR objects more realistic, all of them were resized to match their real sizes. To further improve spatial correspondence, we ensured that the locations of MAR objects were aligned with their real-world counterparts, where applicable. This is unlike in the original Virtual Campus Tour, where the virtual panel size and location were far from those of the actual panel.

Furthermore, to engage users emotionally with the MAR application through natural interaction, we enabled voice commands. The student can start to communicate with the avatar by saying “Hello” as soon as the avatar appears. The avatar responds, and then the student may ask, for example: “Who are you?” The avatar first introduces herself, and then, she explains about the different floors, classrooms and offices. Finally, after the tour ends, the avatar wishes the student all the best. A future study is needed to measure whether the added voice commands and other changes to the avatar make it more emotion-evoking and natural.

7. Discussion

With the advancement of smart device technologies and the popularity of smartphones, users’ expectations have exceeded mere usability of mobile applications. In contemporary application design and development, designers must not only expect that the application is easy and efficient to use while satisfying the user’s essential needs, but that it also makes the user emotionally engaged. Emotional engagement is associated with the experience that the user constructs while interacting with the application. Yet, UX is a complex concept, as it deals with many different factors. The focus of this paper was on the emotional aspects of UX in the context of MAR applications. Furthermore, MAR UX design, development and evaluation are somehow different than those of traditional mobile applications. From the MAR concept design perspective, UCD frameworks, such as mLUX (m-Learning User eXperience) [30], provide relevant concept development methods that ensure appropriate usability together with UX factors, including, but not limited to delightfulness, reliability and adjustability.

From the development perspective, various integrated development environments, such as Unity 3D and Visual Studio, and AR toolkits, such as Vuforia AR, provide great assistance to developers to come up with MAR applications. In fact, there even exist AR toolkits (e.g., ARIS, Augment, Aurasma, Blibbar and Layar) that enable the creation of full MAR experiences without programming skills [3]. Consequently, in the wake of the success of Pokémon Go, the number of published MAR applications is soaring, but research is needed to determine to what extent they engage users emotionally.

Unlike the MAR concept design and MAR application development, MAR UX design and assessment are associated with many challenges (Table 4) that require further research and tool development. This has also been identified previously by other researchers who have investigated users’ overall impressions of MAR applications. For example, some Pokémon Go players believe that the AR feature of the game drains the smartphone, which may lead them to disable the AR feature [18,19]. Furthermore, MAR applications often require bodily involvement beyond mere touching. For example, users may need to walk to the correct location to launch a location-aware MAR experience, orient the device appropriately in relation to the real world to see the target MAR scene, use hand gestures to control MAR objects in the scene and possibly use voice commands or movement gestures to further interact with the application. If done poorly, this rich interaction can make the application unfriendly, difficult to use and therefore emotionally disengaging.

The interaction with MAR applications in public spaces requires special design considerations. It is not enough to make the application easy to use, which is a de facto design goal for all mobile applications, including MAR applications. Designers of mobile applications should acknowledge that, for safety reasons, the application should not disrupt users’ awareness of their surroundings. This is even more important in the case of MAR applications that often require deeper interaction than ordinary mobile applications. Moreover, Carmigniani et al. [44] believe that the main problem regarding social acceptance comes from the level of disruption caused by mobile devices (in other words: mobile applications) in public places and during conversations.

The acceptance of MAR applications from the UX perspective requires that the application follows the user’s mental models. For example, in Case Study 1, an interviewed participant mentioned that she prefers dogs over cats. Consequently, she did not find the cat as appealing as some other participants did. This indicates that designers must put attention on the correspondence of selected metaphors and mental models of the target users. The best way to achieve this is to utilize a UCD method that involves end-users as co-designers, thus revealing their mental models in the design process.

The acceptance of MAR also depends on whether users are sufficiently skilled to perform the necessary application setup and configuration tasks, which should be minimal. For example, the participants of Case Study 1 had to install the Arilyn application before interacting with the cat. Moreover, the results demonstrated that the participants, who were university students, were ready to use MAR applications if and only if they find an added-value in having the application installed in their smartphones. This conclusion is only valid in the context where the experiment was conducted; in other contexts, and with other users, the results could be different.

There are several limitations in this study that are noteworthy. Firstly, the sample sizes in case studies was relatively small; thus, we cannot draw statistically-significant conclusions. Statistical analysis of a large quantity of interview data could, for example, be used in a future study to identify emotions that are particularly prominent among MAR application users. Secondly, because the evaluation experiments were conducted in a single session, long-term effects of UX design remain to be investigated. For example, it would be important to investigate how long a MAR application succeeds in supporting emotional engagement of its users, as it directly relates to sustainable use of the application. Thirdly, the interviews were fully structured, thus leaving little room for drilling deeper into specific topics of interest. Considering these shortcomings and the results of this study, we aim to develop a richer MAR application and investigate its UX with a larger sample size and versatile data collection instruments over a long period of time in the future.

8. Conclusions

The aims of this study were to find out how users perceive MAR applications emotionally, to identify challenges and opportunities associated with MAR applications and to recommend best practices from the perspective of MAR UX design. These aims were reached through evaluations of two case studies: a commercial advertisement application featuring an animated AR cat and our Virtual Campus Tour application designed to orient new students at a university campus. The analysis of the evaluation results, together with the conducted literature review and the experiences acquired during the development of the Virtual Campus Tour application, were the basis for identifying several challenges, opportunities and best practices for MAR UX. The identified best practices were then used to update the Virtual Campus Tour application, thus increasing its potential to engage users emotionally in the future. Whether this potential is realized or not remains to be evaluated in a future study.

The evaluations of the two case studies demonstrated that the participants value MAR applications based on their functionalities, usefulness and ease of use. A major reason as to why designers should utilize AR in mobile applications is to provide positive experiences to users by engaging them emotionally with the application. Indeed, as we have shown in this study, emotional engagement is closely associated with the term user experience. We further demonstrated that having AR objects embedded in a mobile application requires proper design considerations, such as identifying users’ needs, providing natural interaction and engaging users with the application through a realistic avatar.

The results of this study may help UX designers come up with more user-friendly and robust MAR applications. Yet, there remain many research gaps to be investigated, such as developing an acceptance model for MAR UX, modeling MAR content interaction, personalizing MAR experiences and sustainable use through appropriation of MAR experience lengths. We also recommend further studies on MAR applications in various contexts to draw universal MAR UX guidelines based on the best practices presented here.