Impact of Location, Gender and Previous Experience on User Evaluation of Augmented Reality in Cultural Heritage: The Mjällby Crucifix Case Study

Abstract

In recent decades, a growing number of museums have adopted digital media, both as an enhancement of exhibitions of real artifacts and as an alternative to traditional display methods. The digital acquisition of artifacts generates accurate 3D replicas that can be displayed via different digital media. With an increase in immersive technologies in the cultural heritage (CH) domain, it is common to see digital artifacts presented in Virtual Reality (VR) and Augmented Reality (AR). This paper presents two user studies conducted in different locations evaluating the use of an AR application in the portrayal of the Mjällby Crucifix artifact. This paper presents the overall results from both user studies evaluating and discussing the AR application on a number of different aspects on a 7-point Likert scale: (1) understanding the artifact, (2) ease of use, (3) object feeling part of reality, (4) perceived visual quality of the object, (5) overall satisfaction experience, and (6) willingness to download the AR application. The results have been compared between genders, age groups, and previous experience with AR. Potential benefits and disadvantages of AR experiences in the context of a museum exhibition were also gathered in free text from the visitors.

1. Introduction

Digital visual media are commonly used in the Cultural Heritage (CH) context, and more and more museums are adopting the use of digital representations of artifacts and CH sites and displaying them to the public using different media [1]. Moreover, the use in museums of immersive technologies such as Augmented Reality (AR) and Virtual Reality (VR) has increased in the last decades [2]. AR and VR provide an added value to the visitor experience from several different aspects, e.g., interactivity or sense of presence [2]. Furthermore, AR and VR broaden the physical boundaries of the museums in a two-way direction. These technologies can give the possibility to the visitors of the museum to virtually travel to far or inaccessible locations [3] or show the visitors reconstructions of historical sites which no longer exist or ancient ruins [4]. On the other hand, AR and VR applications can “bring the museum to the visitor” by easily providing access from different geographical locations to digital replicas of artifacts located in the museum. Creating a digital replica of CH artifacts is not only useful for inventory purposes or to broaden their availability in terms of locations. Sometimes it could be the only way to show some artifacts to the general public. For example, some CH artifacts are too fragile to be displayed and are instead kept in storage under specific conditions for durability and safety. Recent works also studied the use of digital visual media and immersive technologies for the conservation of Intangible Cultural Heritage (ICH) [5]. In terms of possibilities that these technologies can offer, while VR can fully immerse the user in a virtual environment allowing the user to travel to far locations, AR combines the virtual elements with reality, e.g., by showing digital artifacts to the original location where they were recovered and adding a new dimension to the visitor experience. Several studies showed how VR and AR could be useful in presenting many different artifacts or historical buildings and ancient sites [2]. In this paper, we present the continuation of the work introduced in [6] with the aim of exploring the use of AR technology to display the Mjällby Crucifix, a large medieval crucifix found in Mjällby, located in the Blekinge region in Sweden. The crucifix is currently stored at the Blekinge Museum warehouse. The original setup of this artifact also includes a wooden base located in the Mjällby church. The digitization of these two artifacts allows us to combine them, and with AR, we can display them together with their original composition in context with the surroundings of a real scene.

In [6], we started our work by analyzing the possibilities and challenges of AR in the CH context. We conducted an exploratory study involving CH experts from the Blekinge Museum, asking them through an online questionnaire and a group interview their thoughts about possibilities and challenges with the use of AR technology in digital CH.

The main findings presented in [6] supported some positive features of AR technology already discussed in previous work [2] such as exhibition enhancement and the support for exploration and educational activities. Some examples of exhibition enhancement were the possibility of the display of artifacts with additional information and metadata, showing reconstructed CH areas in their original locations or artifacts in the original context, and displaying inaccessible or fragile artifacts. Further discussions also mentioned the possibility of showing the artifacts not only in their current conditions but reconstructing their original appearance, as well as the possibility of expanding the geographic area of impact of the museum by displaying some of the museum’s artifacts in several different locations of the region. This could be used to engage and interest visitors to the museum as a sort of preview of the type of artifacts available on site. Some experts also mentioned the possibility of reaching new target audiences, e.g., younger visitors, and the support for a more interactive experience for the visitors. Additional benefits that were discussed included displaying small artifacts on a larger scale, allowing easier interaction and examination for a better understanding of the use of artifacts.

As the main challenges related to the users, the main aspects discussed were usability and accessibility. An AR application must work properly even when used by people who are not familiar with the technology, it should be easy to use by a broad audience, and the instructions should be very simple and clear. An AR experience should also be accessible to people with disabilities (e.g., position of the markers, physical area of interaction). During this exploratory study, the experts also discussed aspects related more to the perspective of the museum, for example, the cost of acquiring and maintaining the devices and the digitization process, as well as training for the museum staff. Moreover, in light of the COVID-19 situation, health and safety aspects were mentioned since the setup of the exhibition could include sharing the devices among visitors. As a solution for the cost of the devices, the experts mentioned the Bring Your Own Device (BYOD) policy, however, with this approach other challenges arise such as multiplatform implementation and maintenance of the AR application as well as its necessity to run also on devices with limited hardware capabilities. Some technical limitations must also be addressed, e.g., the necessity of a robust tracking algorithm capable of working in several different lighting conditions.

In this paper, we focus on the museum visitors’ perspective, i.e., on the user evaluation of an AR application displaying the Mjällby Crucifix artifact in its original setup. We created a digital replica of the two artifacts (crucifix and base), implemented an AR application that runs on an Android tablet showing the combined artifacts in their original scale, and performed two user studies. Our evaluation takes into consideration several aspects: the easiness of use and user satisfaction of the AR application, how much it helps the understanding of the artifact, the user’s willingness to download the AR application on their own smartphone, and the realism and visual quality of the displayed digital replica of the artifact. We also asked the participants to describe what they think are the main benefits and disadvantages of the use of AR in museums and CH context to gather their perspectives. We were interested to analyze the impact of the location in which the AR application, i.e., if the different appearance of the surroundings might influence the user evaluation of the aforementioned aspects, hence we run two identical user studies in two different sites. The first user study was conducted during a museum exhibition in the interior of the Sölvesborgs Slott estate, a 19th-century building located nearby the ruins of Sölvesborg’s Castle. The second user study was run in an empty lab room at the Blekinge Institute of Technology. Moreover, we analyzed the gathered data comparing gender, age groups of the participants, and different previous experiences with AR.

The remainder of the paper is organized as follows. The latest related work in the area of AR applied to a CH context is presented in Section 2. Section 3 introduces the Mjällby Crucifix history to contextualize the artifact, while in Section 4 we describe the pipeline for creating the digital replica of the crucifix and its base as well as the AR application, and we report the details and setup of the two user studies that were conducted. The results of the two user studies are presented in Section 5 and discussed in Section 6. We summarize our work in Section 7 including also future works.

2. Related Work

Several works in the last decades investigated the use of AR technology applied to the CH context. The survey by Bekele et al. [2] proposed a comprehensive analysis of the latest studies, including AR, VR, and Mixed Reality (MR) in CH, also identifying their main application areas, i.e., virtual museums, exhibition enhancement, education, exploration and reconstruction, some of them already mentioned in Section 1.

Older research includes the design and implementation of ad hoc AR systems; for example, systems working on mobile devices augmenting the captured images with additional drawings created from CH experts [7], or tracking natural features of specific landmark buildings and superimposing aligned 3D models [8]. Examples of outdoor systems were proposed in [9,10]. In [9], the authors implemented a head-mounted system capable of enhancing the users’ view with additional information for specific points of interest, while the work in [10] proposed a mobile AR application that allows the display of a reconstruction of Sheffield’s medieval castle in its original location. In the context of restoration, the authors of [11] proposed an AR system that displays the different phases of artifacts restoration.

AR frameworks such as Google ARCore [12], Apple ARKit [13], and Vuforia [14] integrating with game engines (e.g., Unity and Unreal) substantially improved the availability of AR technology, and the focus of the proposed studies shifted to content creation and user evaluation. In [15], the authors proposed a series of AR serious games supporting learning activities during a tour guide that were evaluated in terms of some usability aspects, visual appeal, and usefulness.

Proposed AR applications had also been evaluated in terms of usability, e.g., refs. [16,17,18]. Another aspect commonly evaluated is the user experience of AR applications, e.g., in [18]. A further example can be found in [19] the authors developed an outdoor AR game-based experience showing virtual characters from 1800’s town and their habits and evaluated visitors’ experience.

In [20,21], the role of aesthetics of AR content has also been investigated, revealing a strong influence on user satisfaction. Visitors’ satisfaction was the main focus in [22]. In this work, the authors examined the perceived added value of the use of an AR application in CH, analyzing different age groups. Increasing the value of the visit and willingness to pay for the AR app were some of the evaluated aspects, and the results showed a significant difference between age groups regarding the willingness to pay, with the older age group more inclined to pay for the app. An analysis of the perceived value of AR from the point of view of internal stakeholders, including experts in the CH domain, was presented in [23].

An extensive evaluation of visitors’ experience can be found in [24]. The authors designed an AR application for HMDs (HoloLens) focused on the storytelling and the historical narrative of an ancient Egypt battle scene. The aspects that had been rated were visitors’ enjoyment, easiness of use, usefulness, and willingness for future use. In [25], the authors highlighted the need to use interactive storytelling as a way to engage further the visitors compared to static visualization. Furthermore, they discussed different interaction techniques suitable for VR and AR digital storytelling applications.

The social aspect of user interaction has also been explored by several works, e.g., refs. [26,27]. In [28] the authors examined the interaction between two users using a hybrid setup, i.e., one user using an HMD VR device and the other using a mobile AR solution, and evaluated the acceptance of this setup with positive findings.

In [29], Huang et al. presented an interesting application of AR in CH by enhancing a pop-up book with animated 3D elements. The evaluation showed positive responses from the visitors in terms of engagement, experience satisfaction, and usefulness.

Recent works also show the positive effect on learning within the CH context of AR application [30], used in synergy with more traditional frontal lectures [31], or either outperforming frontal lessons [32].

The importance of realism has also been studied. In [33], the authors discuss a workflow for the implementation of realistic virtual human characters involved in interactive storytelling.

Finally, it is worth mentioning the impact of multisensory AR experiences on enhancing the visitors’ experience. A recent survey by Marto et al. [34] discusses several different AR and VR applications in CH involving other senses, from the most common audio integration to haptics and smells.

Our work contributes to the area with an analysis of several different aspects of an AR application that runs on mobile devices, i.e., the easiness of use and user satisfaction, the understanding of the artifact, the willingness to download the AR application, the realism, and the visual quality of the digital artifact. In particular, we have evaluated the impact of the location in which the AR application was run, and we analyzed and compared the data according to gender, age group, and previous experience with AR. We also have gathered input from the visitors regarding the benefits and disadvantages of the use of this type of AR application in museums and CH exhibitions.

3. The Mjällby Crucifix

The Mjällby Crucifix is a large late-medieval triumphal cross (310 cm high and 260 cm wide) from the Mjällby church located in the western part of Blekinge. It is not known where the crucifix sculpture was made. Triumphal crosses are usually placed at the triumphal arch of the church, above the entrance to the chancel. The triumph refers to the resurrected Jesus Christ’s victory over death [35]. According to local legends, the crucifix has been perceived as strange-looking, and this is partly due to the crudely cut face of the depiction of Christ [36]. It was believed, according to some legends, that the crucifix could cause pregnant women to miscarriage and that the cross was once swept up on the shore in Hellevik (Hällevik = holy bay), near Mjällby [36].

During the 18th and 19th centuries, many of the medieval parish churches in Blekinge were demolished as they were too small to accommodate the growing population. The old medieval church in Mjällby, possibly dating from the 12th century, was replaced by a new church that was consecrated in 1791 [35]. The large triumphal cross was then moved to the new church. In the middle of the 19th century, it was taken down and stored in the attic of the church. It was removed since it obscured the view from the altar, but as mentioned earlier, some thought it was too ugly or crude to be shown in the church room.

The wood material of the crucifix is oak. Today, the crucifix is painted gray, but there are still remnants of the color red representing blood where the crown of the thorns once was [35]. The torso and legs of the sculpture are cut in one piece, and the head and arms are mounted [35]. There is a square cut on the chest of the sculpture. Here, a small wooden fragment relic was kept along with a piece of parchment dated 1492. According to tradition, the relic came from the “true cross”, i.e., the cross on which Jesus was crucified. The reliquary is today preserved at Lund University’s historical museum. It is quite a rare object since many relics disappeared during confiscations by the Swedish King after the reformation of the church in the early modern time.

The crucifix is currently kept at the Blekinge Museum warehouse and, on occasions, shown to the public during guided tours. It is now displayed in a horizontal position on a low platform with other surrounding objects next to it, as shown in Figure 1a. Due to its positioning on top of the platform with other objects nearby, it is not possible for visitors to explore it closely.

4. Materials and Methods

In this section, we report the digitization process of the artifact through 3D scanning and the details regarding the development of the AR application. Moreover, we present a description of the two user studies conducted in the two different sites together with the details regarding the questionnaire filled by the participants.

4.1. Digitization of the Mjällby Crucifix and AR Application

To obtain a digital replica of the Mjällby Crucifix, we performed a 3D scanning using an Artec Leo 3D scanner. The Leo scanner is a structured light 3D scanner with a high 3D resolution that can reach 0.2 mm. With the Leo scanner, we captured both geometry and texture information of the crucifix. During the acquisition process, the scanner can automatically execute an initial alignment of several different scans. This feature helps to improve the acquisition phase both in terms of time and quality as we could easily check during the process if specific parts of the artifact had not been captured correctly or were still missing. During the post-processing phase, we use the Artec Studio Projects software to obtain the final 3D model. A cleaning step removed all the parts of the surrounding scene that had been captured but were not part of the crucifix object. The software then performed a global registration converting all scans to a single coordinate system. The fusion of all the relevant scans generated a highly detailed 3D mesh that had been simplified obtaining a 3D model of 400,000 faces in OBJ format. We performed a similar procedure for the digitization of the base of the crucifix currently located in the Mjällby church (Figure 1b), obtaining a 3D model of 50,000 faces. We combined the two models and exported them as a unique 3D model in GLB format, binary GL Transmission Format (glTF).

We developed an Android marker-based AR application based on the Google AR platform ARCore [12] and the 3D framework Sceneform [37,38]. As a marker image, we used an A1 format picture of a blueprint dated 1788 of the Mjällby church. The ARCore application detects horizontal and vertical planes. In our setup, the marker image is positioned on the floor, and when the camera view of the AR application detects and recognizes the blueprint, it displays the 3D model of the crucifix and the base on top of the marker, as shown in Figure 2. Automatic lighting estimation is supported by the 3D framework Sceneform, and this results in the rendering of the 3D model with a lighting setting consistent with the current real scene providing a coherent display of the 3D model and a more realistic effect. The AR application also provides textual information in a separate view describing the history of the Mjällby Crucifix. Before starting the AR experience, the application shows the user a list of instructions on how to visualize the 3D model. Due to the large size of the artifact, the application has its best fruition with a tablet device, giving the possibility for the user to visualize the details of the artifact on a large screen. The application has been tested on two Android tablets, i.e., Samsung Galaxy Tab S7 and Tab S7+. It runs on both devices at 30 frames per second. The Tab S7 (${11}^{″}$ display) has been used for both user studies. Figure 3 shows two screenshots of the AR application. Although the tracking accuracy of the application was not specifically evaluated, small-scale setups using ARCore have reported mean absolute errors of around 6 cm with static users and around 12 cm in case of users moving [39]. The application was perceived smooth during the initial tests and hence it was deemed sufficient in quality to be used in the user studies.

4.2. User Studies and Questionnaire

We conducted two user studies in different locations, one in the interior of the Sölvesborgs Slott estate during the museum exhibition and another one later in an empty lab room at the Blekinge Institute of Technology. The idea was to compare the results from sites different in nature, one presenting a historical setting and the other one being a simple empty room with no additional elements. The same procedure was performed for both user studies as well as the same set of questions was provided to the participants.

Before taking part in the user study, the potential participants were informed with flyers or verbally that their participation in the study was voluntary and the collected information was anonymous. They were also informed that they needed to be over 18 years old to participate. The participant’s task was to open the AR application, target the marker with the camera view to display the 3D model, and then look at the 3D model freely. The participants could look at the 3D model for as long as they wanted. After the completion of the task, the participants were asked to answer a questionnaire.

The participants were first asked to fill in their ages. They were then asked to specify their gender with the options: male, female, other/prefer not to say. In case they were unfamiliar with the concept of AR, a short text in the questionnaire first explained that it is a technique that shows virtual objects in combination with the real world. They were asked next if they had used AR before, with a yes/no option to answer.

Table 1. Questions asked in the surveys. For the Likert scale rating questions (Q1-6) their corresponding lower and higher extreme values of the Likert scale have been included.

Rating Questions	Likert Scale Extreme Values
	1	7
Q1: To what degree do you think the AR application helps you get an understanding about the Mjällby Crucifix?	Not at all	To a very high degree
Q2: How easy did you think it was to use the AR application?	Extremely difficult	Extremely easy
Q3: To what degree did you think that the AR object (the crucifix) felt like part of the reality?	Not at all	To a very high degree
Q4: How would you rate the perceived visual quality of the AR object (the crucifix)?	Extremely poor	Excellent
Q5: How satisfied are you with the overall experience of the AR application?	Extremely unsatisfied	Extremely satisfied
Q6: How likely is it that you would like to download and use the AR application (with more objects from the museum) for your own phone/tablet?	Extremely unlikely	Extremely likely
Free Text Questions
Q7: As a visitor, what do you think are the benefits of AR-experiences in the context of an exhibition?
Q8: As a visitor, what do you think are the disadvantages of AR-experiences in the context of an exhibition?

lists the questions related to the AR experience. The first six questions (Q1-Q6) were rating questions using a 7-point Likert scale included to evaluate different aspects of the AR application. In detail, Q1 asked to what degree the AR application gave an understanding of the artifact, Q2 assessed the easiness of use of the AR application, and Q3 and Q4 focused on the 3D model of the crucifix visualized in the AR application rating its degree of being part of the reality (Q3) and its visual quality (Q4). The overall satisfaction with the AR experience was assessed with question Q5, while Q6 rated the willingness to download the AR application on their own phone. The final two free text questions asked the participants to briefly summarize their thoughts about the benefits (Q7) and disadvantages (Q8) of the use of AR experiences in the context of a museum exhibition.

5. Results

In this section, we present the results from the two user studies. First, we report the results from the rating scale questions (Q1-6) of the museum exhibition (Section 5.1) and the user study conducted in an empty lab room (Section 5.2). The comparison between these results from the two locations is presented in Section 5.3. The results from the free text questions (Q7 and Q8) are reported in aggregated form in Section 5.4.

5.1. User Study 1: Museum Exhibition Results

During the museum exhibition, we gathered data from $N_e=49$ participants (23 female, 22 male, and four n.a.) from 18 to 85 years old ($M=51.11$, $SD=16.77$). Only eight participants ($16.32\%$) indicated they had already previous experience with AR while 28 ($57.14\%$) expressed no previous experience and the remaining 13 did not indicate any preference. Overall the results reveal a very positive evaluation of the AR application for all the six questions (Q1-Q6), as shown in Figure 4. The mean values for Q1-5 are all above 6.0, the only aspect that has been evaluated slightly below the 6.0 value is the willingness to download the app (Q6 M = 5.82).

Gender Analysis. Since the participants’ population was quite evenly spread in terms of gender (23 female, 22 male, and four n.a.), we could compare the results between genders from quite balanced groups. Analyzing the descriptive statistics values in

Table 2. Museum exhibition results grouped by gender. Results include median values (Mdn), mean values (M), standard deviation values (SD), and p-values obtained from the Mann-Whitney U (MWU) test.

	Female			Male			MWU Test
	Mdn	M	SD	Md	M	SD	p-Value
Q1	7	6.56	0.66	7	6.50	0.80	0.978
Q2	7	6.41	0.91	6	5.91	1.11	0.118
Q3	7	6.43	0.84	6	6.24	0.70	0.224
Q4	7	6.61	0.66	6	6.27	0.63	0.051
Q5	7	6.48	0.66	6	6.04	0.90	0.096
Q6	6	6.00	1.13	6	5.45	1.40	0.182

and Figure 5, we can see that both mean and median values of all questions are slightly higher for the female group. However, the results from the Mann-Whitney U test with applied continuity correction show no statistical difference between the two groups, as shown by all p-values $>0.05$ in Table 2. We applied a non-parametric test due to the small size of the groups, and the Mann-Whitney U test was chosen since we are comparing two independent groups.

Age Groups Analysis. We analyzed the answers by comparing participants’ age groups. From the initial set of participants, five people did not include their specific age and have been removed from this analysis. The remaining 44 participants have been divided into three age groups: 12 adults (18–39), 16 middle-age adults (40–59), and 16 older adults (60+). Figure 6 shows the box plots for each group age divided by question, the descriptive statistics values are indicated in

Table 3. Museum exhibition results divided by age groups. Results include median values (Mdn), mean values (M), standard deviation values (SD), and p-values obtained from the Kruskal-Wallis (KW) test.

	Adults			Middle-Age Adults			Older Adults			KW Test
	Mdn	M	SD	Md	M	SD	Md	M	SD	p-Value
Q1	6.0	6.25	0.75	7.0	6.31	0.87	7.0	6.81	0.40	0.075
Q2	6.5	6.50	0.52	6.0	5.93	1.22	6.0	6.06	1.06	0.571
Q3	6.0	5.75	0.87	6.0	6.19	0.91	7.0	6.67	0.61	0.014
Q4	6.0	6.17	0.83	7.0	6.44	0.73	6.5	6.44	0.63	0.611
Q5	6.0	6.17	0.72	6.5	6.37	0.72	6.0	6.19	0.91	0.721
Q6	5.5	5.33	1.37	6.0	6.00	1.21	6.0	5.75	1.34	0.400

. Interesting results can be seen for Q1 (understanding) and Q3 (realism) for which we can see an increase in the rating values with older age groups. The evaluation of the statistical significance has been performed using a non-parametric test due to the limited size of the groups. Since the comparison, in this case, is done among three independent groups we choose the Kruskal-Wallis (KW) test. As shown in Table 3, only the p-value for Q3 (which inquires to what degree the 3D Model of the crucifix felt like part of the reality) reveals a statistically significant difference among the age groups. A follow-up analysis with pairwise comparisons using the MWU test (with continuity correction) shows a statistical significant difference between the adults group (18–39) and the older adults (60+) group ($p=0.004$). No statistical significance has been found for adults/middle-age adults ($p=0.167$) and middle-age adults/older adults ($p=0.098$).

Previous Experience Analysis. Results from the comparison between participants having previous experience (eight people) and participants with no experience (28 people) show no significant difference for any question, as shown in

Table 4. Museum exhibition results compared between participants having previous experience with AR and participants with no experience. Results include median values (Mdn), mean values (M), standard deviation values (SD), and p-values obtained from the Mann-Whitney U (MWU) test.

	Previous Experience			No Experience			MWU Test
	Mdn	M	SD	Md	M	SD	p-Value
Q1	7.0	6.50	0.75	7.0	6.53	0.74	0.855
Q2	6.5	6.37	0.74	6.0	5.96	1.19	0.527
Q3	6.0	6.12	0.64	6.0	6.26	0.86	0.500
Q4	6.0	6.25	0.70	7.0	6.46	0.64	0.420
Q5	6.5	6.37	0.74	6.5	6.28	0.85	0.901
Q6	6.0	5.75	1.49	6.0	5.86	1.32	0.841

and Figure 7.

5.2. User Study 2: Lab Room Results

A second user study has been conducted at the Blekinge Institute of Technology in an empty lab room as shown in Figure 2b. The participants have been asked to perform the same task as the user study at the museum exhibition and to fill in the same questionnaire, the altered condition is the nature of the site in which the user study has been conducted. $N_l=39$ participants took part in the user study (nine female, 30 male) from 21 to 59 years old ($M=38.66$, $SD=10.98$). For this user study, the number of participants that indicated they had already previous experience with AR is much higher compared to the participants from the museum exhibition 32 participants ($82.05\%$), while six participants ($15,38\%$) expressed no previous experience. Only one participant did not reply to the previous experience question.

The results reveal a very positive evaluation of the AR application also in the lab room site for all the six questions (Q1-Q6), as shown in Figure 8. However, the median and mean values are slightly lower compared to the museum exhibition, with mean values for Q1, Q2, Q3, and Q6 slightly below 6.0 (Q1 M = 5.85, Q3 M = 5.46, Q4 M = 5.92, and Q6 M = 5.58).

Gender Analysis. A comparison between genders shows results similar to the museum exhibition, with slightly higher mean and median values for the female group compared to the male group (Figure 9), but no significant difference has been found performing the MWU test with applied continuity correction as shown in

Table 5. Lab room responses grouped by gender. Results include median values (Mdn), mean values (M), standard deviation values (SD), and p-values obtained from the Mann-Whitney U (MWU) test.

	Female			Male			MWU Test
	Mdn	M	SD	Md	M	SD	p-Value
Q1	6.0	6.11	1.05	6.0	5.77	0.93	0.294
Q2	7.0	6.55	0.73	6.5	6.37	0.72	0.446
Q3	6.0	5.78	1.20	5.5	5.37	1.30	0.324
Q4	6.0	6.44	0.53	6.0	5.77	1.07	0.102
Q5	7.0	6.44	0.73	6.0	6.03	0.67	0.112
Q6	7.0	5.57	1.94	6.0	5.55	1.57	0.543

. It is worth noticing that, in this case, the sample size of the two groups is quite unbalanced (9 females, 30 males).

Age Groups Analysis. Analyzing the responses comparing participants’ age groups, in the lab room population we find 16 participants which belong to the adults group and 20 to the middle-age adults group. In this population the older adults group is not represented hence it does not appear in the analysis. Three people did not include their specific age in the answer and they have been removed from the analysis. Figure 10 shows the box plots for each age group divided by the question. Similar to the results from the museum exhibition, Q3 (realism) was rated with a higher value by the older age group, as well as Q4 (visual quality), however, no significant difference has been found for any of the six questions, as shown in

Table 6. Lab room responses divided by age group. Results include median values (Mdn), mean values (M), standard deviation values (SD), and p-values obtained from the Mann-Whitney U (MWU) test.

	Adults			Middle-Age Adults			MWU Test
	Mdn	M	SD	Md	M	SD	p-Value
Q1	6.0	5.80	1.00	6.0	5.81	0.98	0.960
Q2	7.0	6.45	0.69	7.0	6.50	0.63	0.885
Q3	5.0	5.15	1.39	6.0	5.69	1.14	0.242
Q4	5.5	5.60	1.14	6.0	6.25	0.77	0.085
Q5	6.0	6.15	0.74	6.0	6.06	0.68	0.702
Q6	6.0	5.35	1.84	6.0	5.73	1.44	0.678

.

Previous Experience Analysis. The population for the lab room user study in terms of previous experience differs from the first user study: 32 people indicated having previous experience with AR while only six replied having no experience. The results in

Table 7. Lab room results compared between participants having previous experience with AR and participants with no experience. Results include median values (Mdn), mean values (M), standard deviation values (SD), and p-values obtained from the Mann-Whitney U (MWU) test.

	Previous Experience			No Experience			MWU Test
	Mdn	M	SD	Md	M	SD	p-Value
Q1	6.0	5.75	0.98	6.5	6.33	0.82	0.187
Q2	7.0	6.41	0.71	7.0	6.67	0.52	0.457
Q3	6.0	5.47	1.24	5.5	5.33	1.63	0.934
Q4	6.0	5.94	1.01	6.0	5.83	1.17	0.867
Q5	6.0	6.03	0.69	7.0	6.67	0.52	0.042
Q6	6.0	5.51	1.73	6.5	6.00	1.26	0.621

and Figure 11 show that for Q5 (experience satisfaction) there is a significant difference between the two groups, with people indicating no experience showing higher mean and median values.

5.3. Comparison Results between Sites

Figure 12 shows higher mean and median values for all questions apart from Q5 (easiness of use) with statistical significance for Q1, Q3, and Q4 indicated by the results of the MWU test with applied continuity correction (

Table 8. Responses of the participants of both user studies grouped by site, i.e., the museum exhibition and the lab room. Results include median values (Mdn), mean values (M), standard deviation values (SD), and p-values obtained from the Mann-Whitney U (MWU) test.

	Museum Exhibition			Lab Room			MWU Test
	Mdn	M	SD	Md	M	SD	p-Value
Q1	7.0	6.51	0.71	6.0	5.85	0.96	0.00058
Q2	6.0	6.13	1.01	7.0	6.41	0.71	0.28537
Q3	6.0	6.27	0.84	6.0	5.46	1.27	0.00124
Q4	7.0	6.39	0.70	6.0	5.92	1.01	0.03187
Q5	6.0	6.28	0.79	6.0	6.13	0.69	0.22459
Q6	6.0	5.82	1.27	6.0	5.58	1.64	0.70388

).

To check if the different age groups could have influenced these results, we performed a further test by filtering out the data related to the age group older adults and comparing the data of the two locations. The new population of the museum exhibition consists of 28 participants compared to the 39 participants in the lab room. The results in Figure 13 reveals similar patterns than the unfiltered data (Figure 12), however the results of the MWU test with applied continuity correction show no significance difference (

Table 9. Responses of the participants of both user studies grouped by site excluding the data from the older adults age group. Results include median values (Mdn), mean values (M), standard deviation values (SD), and p-values obtained from the Mann-Whitney U (MWU) test.

	Museum Exhibition			Lab Room			MWU Test
	Mdn	M	SD	Md	M	SD	p-Value
Q1	6.5	6.28	0.81	6.0	5.85	0.96	0.065
Q2	6.0	6.18	1.00	7.0	6.41	0.71	0.500
Q3	6.0	6.00	0.90	6.0	5.46	1.27	0.090
Q4	6.5	6.33	0.77	6.0	5.92	1.01	0.120
Q5	6.0	6.28	0.71	6.0	6.13	0.69	0.349
Q6	6.0	5.71	1.30	6.0	5.58	1.64	0.995

).

5.4. Results from the Free Text Questions

From the analysis of the answers gathered from the free text questions (Q7 and Q8), the three most mentioned benefits of the AR technology applied to the CH context were the possibility to look at digital artifacts from home or several different locations, the accessibility of some artifacts that might not be available, and the opportunity to look at details of the artifact from a near distance. Some other benefits mentioned were a good understanding of the scale of the real object since the digital replica is shown in context with reality, the realism of the 3D model, the possibility to add features enhancing the real artifact, and the fact that there is no risk to damage the real artifact.

The most cited disadvantage was the fact that an AR experience cannot replicate the same feeling while looking at the real artifact, e.g., missing touch or smell features. It has also been mentioned a potential negative effect of reducing the availability of exhibitions of real artifacts in case of excessive use of digital media. From a technical point of view, some participants reported that the use of an AR application could be complicated, especially for the elderly. The cost of the devices has also been mentioned, and the fact that, with the use of dedicated devices, their limited availability could induce a long waiting time for the visitors or crowding around the marker. Despite the fact that we opted for the use of a tablet device with a larger screen compared to a smartphone, some participants reported that the size of the screen was too small. This feedback might also be due to the large size of the artifact displayed in this case study.

6. Discussion

The results of both user studies show high ratings for all the assessed aspects confirming the positive trends of previous evaluations of the use of AR in the CH context. When comparing the gathered data between female and male populations, we found slightly higher rates from the female group for both user studies, however, having no significant difference.

In terms of previous experience with AR, the data shows different results between the two sites. For example, the lab room results reported significantly higher experience satisfaction (Q5) for the population with no experience than the participants with previous experience. This result might be due to the “wow effect”, the positive astonishment generated by the perception of a new experience. However, the data from the museum exhibition shows no significant difference in this aspect. No significant differences were found for the other rated aspects on both sites. However, it is worth mentioning that in both user studies, the two populations for this particular comparison (previous experience vs. no experience) are quite unbalanced; hence results might need to be confirmed by further studies.

Interesting results were derived from the age group analysis. The data from the museum exhibition shows that the older adults group rated significantly higher the level of realism (Q3) of the 3D model of the crucifix compared to the results from the adults group. This could be because younger people are more exposed to digital 3D content, for example, video games and movies with special effects, and might be more trained to look at the details of the 3D models.

The results of the comparison between the two locations show a higher rating for the museum exhibition in all aspects except for the easiness of use (Q2), with a significant difference in understanding (Q1), realism (Q3), and visual quality (Q4). However, these results might not be influenced solely by the location since the populations from the two sites differ in terms of age groups. An analysis of the data derived only from the two age groups that were present in both sites showed the same pattern but no significant differences.

The answers to the free text questions confirmed to a great extent the possibilities and challenges mentioned by the CH expert in our initial study [6]. Some further limitations gathered from the participants were the potential establishment of long waiting lines for using the museum devices or crowd around the AR target marker. Moreover, some participants highlighted the limited size of the screen even if they tested the app on an ${11}^{″}$ display tablet. This feedback suggests that AR with handheld devices might be suitable for displaying objects with a limited size, while AR head-mounted devices could be more convenient for visualizing large artifacts.

7. Conclusions

This paper has presented a case study evaluating the visitors’ experience of a mobile AR application displaying the Mjällby crucifix. We digitized the crucifix artifact and its base and combined them to reconstruct the original setup of the crucifix. The digital replica of the combined artifacts has been shown through an AR application in its original size in context with reality. Two user studies conducted in two distinct locations showed high rating results for the evaluated aspects, i.e., the realism and the visual quality of the digital artifact, the easiness of use of the application, the experience satisfaction, the understanding of the artifact, and the willingness to download the AR application. Older adults (60+) rated significantly higher on the level of realism of the digital artifact w.r.t. than younger adults (18–39). However, the impact of the location seems not to induce significant differences in the evaluation of the same age groups. In future work, it could be interesting to conduct further user studies focusing more on the comparison of age groups, with a deeper analysis of the elements influencing their differences in the evaluation. A further possible evaluation is an analysis of the impact of the size of artifacts as well as the size of the device on the visitors’ experience.