A Review of Augmented Reality Applications for History Education and Heritage Visualisation

Abstract

Augmented reality is a field with a versatile range of applications used in many fields including recreation and education. Continually developing technology spanning the last decade has drastically improved the viability for augmented reality projects now that most of the population possesses a mobile device capable of supporting the graphic rendering systems required for them. Education in particular has benefited from these technological advances as there are now many fields of research branching into how augmented reality can be used in schools. For the purposes of Holocaust education however, there has been remarkable little research into how Augmented Reality can be used to enhance its delivery or impact. The purpose of this study is to speculate regarding the following questions: How is augmented reality currently being used to enhance history education? Does the usage of augmented reality assist in developing long-term memories? Is augmented reality capable of conveying the emotional weight of historical events? Will augmented reality be appropriate for teaching a complex field such as the Holocaust? To address these, multiple studies have been analysed for their research methodologies and how their findings may assist with the development of Holocaust education.

1. Introduction

The British Computer Society defines augmented reality as “combining the digital world with the physical one and therefore augmenting the real-world experience” [1]. This technology has rapidly become more viable for commercial and research projects in the last decade due to the prevalence of head mounted devices (HMDs) and smart devices such as phones, tablets and handheld games consoles that are now intrinsically woven into daily life. This has reduced the major challenge of deploying an augmented reality application because specialized hardware is no longer required to use the technology, instead users are able to operate the system from their own devices. With the hardware barrier reduced, augmented reality has begun seeing use for areas such as entertainment, simulations, education and training scenarios along with a variety of other applications. Regarding these educational applications however, research has been primarily focused on school education with little study on alternative audiences or environments, including the potential of augmented reality to enhance learning for an emotionally complex area such as the Holocaust. Information retention is only one element of history education, another purpose to learning history is to understand it so that the mistakes of the past will not be repeated. In their study on empathy in history, Berti et al. [2] discussed various historical perspectives that would be difficult to understand the ability to empathize with the people of the time, such as why ordeal was considered an acceptable means of trial in the middle ages or what would cause an average German citizen to support the Nazi party during the period leading up to world war two. The challenge for educators is getting students to empathize and take into account historical perspectives [3], so they may understand not just what happened, but how it was able to happen and the consequences of those events after they had occurred. This leads to questions as to whether augmented reality has the potential to be used within an educational capacity to teach difficult topics of history such as the Holocaust but before those questions may be asked, it must first be determined how effective the technology for education? Does usage of augmented reality correlate to higher understanding of a historical topic? How effective has augmented reality been within classroom environments? Are there any detriments to the use of augmented reality and if so, how might they be mitigated or overcome? Have there been any studies performed within a museum environment for augmented reality technology and if so, did the results differ to the studies performed within a classroom? These are the questions this literature review seeks to answer with the intention of informing the design and development of an augmented reality application for a Holocaust Memorial.

2. Augmented Reality in Education

This section will explore the current uses of augmented reality within the field of education to identify how it is currently being used and analyse which uses of augmented reality would be appropriate for history education and the Holocaust in particular.

2.1. Augmented Reality in the Classroom

Augmented reality has been used for studies within classroom environments to research multiple areas of the technology and how well they assist with learning. An example of this would be Billinghurst and Duenser’s research [4] using augmented books, in which schoolchildren between the ages of ten and fourteen were shown a storybook that was enhanced with augmented reality. The students would read the book with the augmented reality system projecting scenes over the physical pages that were enhanced with audio effects and a voiced narrative by the author of the book. Following the study, a report created by the teachers of the class concluded that the students enjoyed the workshop and were motivated by the technology as they had found it exciting. This study also tested learning using an augmented reality system for physics education, using enhanced imagery to display concepts with spatial elements such as electromagnetic fields. Performance of two groups were compared, one with augmented reality and one without. The group that used the augmented reality achieved better results in the test, with the mean score being 72 percent compared to the control group who scored a mean result of 60 percent. Four weeks after the initial test, a retention test was performed with the focus group scoring higher once again, with a mean result of 55 percent against the control groups 45 percent. This is significant because the students who interacted with the system were able to retain more information than those who were taught in a traditional manner.

Within a classroom environment, augmented reality has called the role of the teacher into question with its strong implementation of active learning rather than traditional passive learning. In 1949, Ralph W. Tyler established a curriculum model [5] in which the teacher is considered the expert who guides all learning by following their own trajectory. This was challenged in 1975 when a contrasting model was constructed by Lawrence Stenhouse that instead establishes the teacher as a facilitator of knowledge [6]. The teacher as a facilitator is part of a constructivist learning theory, which encourages learner autonomy and has an emphasis on both interaction and engagement. This was the theory acknowledged by Wojciechowski and Cellary in their study on augmented reality environments [7]. This study involved using an augmented reality system to allow students to safely conduct chemistry experiments, after which the student’s observations, experiences and opinions of the system were collected as data for analysis. The research found that the students perceived the technology as being useful but had a much bigger impact on their perceived enjoyment, indicating a positive impact on motivation. This was attributed to the conflict of active learning versus passive learning, with students engaging more with the learning due to their interaction with the augmented reality experiments.

Augmented reality learning environments have been proven effective for active learning method due to their ability to commit taught content to long term memory. This was explained by Santos et al.in their study on augmented reality learning environments [8], in which it was hypothesized based on cognitive theory how these environments would assist with education. Because an augmented reality learning environment has environmental inputs required to use the system, the act of the student interacting with them would cause their minds to register the sensory information associated with those inputs, whether they be visual, auditory or activated via touch. All information is first held in short term memory but is usually not stored, however via the active engagement required by an augmented reality learning environment, these engagements can utilize the sensory register to store what is learned into the long-term memory. An example of this theory outside of technological domains would be when a student rehearses the information they want to recall or commits to it an acronym/abbreviation. The study also gave recommendations for three elements of augmented reality that should be implemented to assist in committing to long term memory. These elements are;

• Real World Annotation: Juxtaposing real world objects with virtual text or symbols to explain the content to the user
• Real Object Centred: The real-world object becomes the central point of learning with the augmented elements enhancing it for the purpose of learning
• Multimedia Learning theory [9]: Utilizing multimedia content to assist in teaching the student. This can include videos, audio files or interactive elements that would not be possible or accessible without augmented reality.

Educational environments have requirements for learning, and for the purposes of augmented reality, these were explored by Bujak et al. [10] in their research on the psychological aspect of the technology within a classroom. By analysing the psychological elements of using an augmented reality system, the study found that the technology providing more autonomy when learning as it was a way to bridge abstract concepts and physical ones, giving the advantage that a user of the system can have both individual perspective and control over their experience. It also allows for collaboration in a shared space using the same educational content; users can work together simultaneously without anyone having to await their turn interacting with the content available.

Despite the strengths of using augmented reality for educational environments, there are potential problems that must be addressed. Two of these were brought to attention in a study by Hsin-Kai Wu et al. [11] on the challenges of using augmented reality for education. Whilst Augmented Reality has the potential to bring new possibilities to a classroom, there remains the concern of causing cognitive and/or emotional overload in the students. When a student is using an Augmented Reality system for the first time, they are required to learn how to operate the system and become familiar with its interface, gestures and conventions, which can be problematic if the student is also required to perform complex tasks as part of the systems design [12]. Another hazard contributing to cognitive overload are gameplay elements, as the learner’s attention may be concentrated on gameplay instead of the content [13], especially if the gameplay elements are complicated or difficult. As well as cognitive overload, the second concern is that if too many gameplay elements are present, students may become too fixated on playing, which is not only destructive for productivity but may also pose a physical danger to them if they become too distracted by the augmented elements to pay heed to hazards in the real world [12].

Additional challenges exist for the creator of an augmented reality system besides the health and safety elements, the system must be able to engage with the student rather than simply exist and hope for results. Hsin-Kai Wu et al. [11] also addressed the requirements of an augmented reality system within the context of education to allow for active learning to happen. These criteria are:

• Fun—The student must enjoy using the system
• Challenge—The student must have some sort of objective to complete with the system
• Curiosity—The system must stimulate the student to further explore the topic

Providing these three criteria are met, the student will be able to engage with the system to autonomously develop their own learning of the subject in a meaningful manner. This was the case on a study of an augmented reality study by Rosenbaum et al. on a simulated outbreak of influenza [14]. Students were given the roles are various health officials or medical professionals in a simulated scenario in which an infection would spread among the users, with the augmented reality tracking who had interacted with one another and which virtual equipment had been obtained. The purpose was not to eliminate the outbreak but rather to contain it as effectively as possible. The challenge of filling a role to assist in containing an outbreak was reported to be a fun experience, with students praising the authenticity and remarking that they felt like real medical staff. Students had reported misconceptions about how systems worked but playing the game had stimulated their curiosity and enabled them to learn more about how viruses affect one’s health, how to treat an infected person and to prevent the spread of influenza.

Medical Education has also benefitted from the usage of augmented reality technology. Kamphuis et al. [15] explored the usage of augmented reality within medical education for three areas. These included having CT scan data projected over a living person and visualizing a set of three-dimensional lungs over a manikin within an operating theatre. The main body of the study focused on an Augmented System with haptic feedback to train the user in performing a laparoscopy, a procedure which requires anatomical knowledge and practiced motor skills to perform. The study noted that no empirical studies had been performed on augmented reality within medical education but did note that the technology would be beneficial for the learning process due to the immersive nature and sense of presence that the learner would have inside the environment.

2.2. Augmented Reality for History Education

The challenge of adapting augmented reality for history education was addressed in a study performed by Blanco-Fernández et al. [16] on immersive learning. The authors expressed frustration over the way in which films, video games and comic books have influenced history education by adapting complex military and geopolitical events into a simple good against evil narrative when these events were much greyer in terms of morality. The purpose of this study was to create an immersive learning experience in which the users participated in three stages. During the first stage, they would use an augmented reality application to participate in the battle of Thermopylae as a major historical figure and make decisions over the battle. In the second stage, the participants were taken to a projection room to analyse the battle from the outside and see how their decisions compared to the real events, collaborated by an expert who could explain facts and the significance of these decisions. The final stage was a debate between the participants and the expert who would then brainstorm with the audience and explain the long reaching consequences of the battle and how history may have changed had things occurred differently. Results collected from this research were to assess the quality of the experience itself rather than measure the learning or emotional response of the audience but did find via participant questionnaires that the users felt that the experience would have high educational perspective for children and adults, but slightly less so for teenagers. These types of educational experiences that allow the student to participate in a historical event with broader context given about their ramifications and consequences are important not only to teach history, but to give an understanding about the way that the current world was shaped by a series of both major and minor decisions, therefore providing a sense of empathy and/or sympathy with historical figures.

Augmented reality for history education has an additional benefit: environmental immersion. Being physically present at the site provides the user with the sense of historical empathy that cannot be achieved from a classroom with a textbook. In their study on augmented reality for Heritage Places, Chang et al. [17] referred to this as a Sense of Place, which they defined as “the combination of feelings of attachment, dependence, concern, identity, and belonging that people develop regarding a place”. This study experimented using an augmented reality guided tour of historical locations within the Tamsui District of New Taipei City, Taiwan. The participants were university students who were given different versions of the application; one group had the full augmented reality setup that could identify buildings and provide information, one had an audio only configuration to provide pre-recorded information and the control group had no guidance at all. These students were given written tests and surveys to record their knowledge and emotional understanding prior to the testing. Results found that test scores increased the most drastically for the group with the Augmented System and least drastically for the audio only group. A similar result was found when testing for the Sense of Place criteria; the augmented reality enhanced group had a more emotional response to the locations they had visited than the others, with the control group recording similar results to the audio group. The results of this study are interesting because the audio only group reported much poorer results than the group with augmented reality which would suggest that audio-based enhancement alone is insufficient to correctly stimulate the audience’s interest and properly immerse them within the environment.

This was further explored in the research by Harley et al. [18] in a study on location-based augmented reality learning. This study involved a mobile phone-based augmented reality application being used at the historical site of the Roddick Gates in Montreal, with the enhanced components being used to compare the current state of the site with how it appeared in the 19^th century. The participants were then challenged to identify the differences in the state of the site, with eye tracking technology monitoring the areas being looked at. The students were able to identify most of the differences between the old and current site, but the emotional response was also recorded and found that the students enjoyed the test, with a minority expressing boredom at either the location, the topic or the tour guide. Student responses were informally recorded, but one student remarked that the experience had given them a different perspective of a landmark that they had previously given no heed to. The results of this study demonstrate when enhanced with augmented reality, students had a more profound understanding of historical landmarks which would indicate that the same response could be possible at any historical site provided there was enough content.

Mobile device-based studies have been recurring throughout research on augmented reality due to the portable and accessible nature of the technology. Research performed by Choudary et al. [19] examined the use of mobile devices for augmented reality on their study on using the technology for cultural heritage. This study involved using a 2007 Nokia N95 smartphone. The purpose of the research was to augment cave paintings with interpreted drawings from subject experts to explain the imagery to the audience. Unfortunately, this study was proposed for a conference and did not publish quantitative results but does discuss the technical logistics of deploying an augmented reality application to a mobile device and the optimizations it would need. The primary concern addressed by the authors is the rendering speed of the Nokia N95, which was only able to perform at 14 frames per second on that device and rendered at a resolution of 320 × 240 pixels. With the development of modern smartphones, the graphical rendering capabilities are far superior in newer phones and would be able to deliver higher framerates on larger resolutions. An example of this would be the popular augmented reality game Pokémon Go [20], which is able to render a resolution of 720 × 1280 pixels at a speed of 30 frames per second.

Another example of a mobile device-based study would be the research performed by Herbst et al. [21] on creating a mixed reality mobile game. The purpose of this research was to create a mobile game called Timewarp, which would allow the user to explore the city of Cologne with a team of virtual elves and interact with different areas to experience visualizations of historical structures that no longer exist. This study also features a heavy focus on gameplay elements, with the players learning through a narrative and having to locate items or time travel gates to proceed through game, solving challenges and interacting with different time periods of the city as they go. This study did not publish its quantitative results, but rather discussed their findings informally. Of these findings, the authors only discussed the technical issues that users had experienced when playing the game, such as virtual objects moving around on screen due to GPS instability or strong sunshine interfering with the device’s ability to render the required objects. The study concluded with ideas about how the application could be improved, suggesting features such as having a simpler interaction scheme, including real objects as part of the experience and ensuring that gameplay objectives are in safe locations, away from dangerous areas such as roads.

The history of architecture is another topic that has explored for educational purposes using mobile augmented reality. This topic was explored by Keil et al. [22] on their research regarding augmented reality tours for architectural history. The focus of this study was on renovated buildings which had lost some of their original design, such as the house of the architect Joseph Maria Olbrich which was destroyed and later rebuilt. The application created by Keil et al. allowed the user to take a photograph of the house in its current state, and have it augmented with a 3D model constructed from the original drawings. This model was also augmented with buttons that would provide the user with information when pressed, giving them details about the corresponding area of the house. This study did not publish nor discuss any results, with the focus being on the technical elements of the application construction.

2.3. Augmented Reality for Holocaust Education

Using an augmented reality learning environment for Holocaust Education is not a new concept; it has been explored by a limited number of studies and experiments thus far. One of these studies on augmented reality for the Holocaust was presented by Stapleton and Davies [23]. The study was performed in collaboration with the Maitland Holocaust Museum and sought to use an augmented reality system for an exhibit that would make the topic relevant to a diverse range of visitors. The resulting system produced depicted a Holocaust story from the perspective of a teenager, giving witness to the rise of the fascist regime through stories told by diaries. Although the study did not publish quantitative or qualitive results, the authors did discuss that audiences were given a storytelling experience that was so emotional that some were moved to tears and questioned how the Holocaust had been allowed to happen. While this would have benefited from having a measurable result, it does stand testament to the emotional impact that an augmented reality enhanced narrative can have.

Narrative as a tool in augmented reality is a powerful way to convey the emotional significance of the event to students and learners. Although the previous study told stories from the perspective of a victim, another study looked to preserve the testimony of a survivor and relay it through the format of augmented reality. This researched performed by Ma et al. [24] worked in collaboration with the UK National Holocaust Centre and Museum and a survivor to record his stories, experiences and opinions in stereoscopic 3D videos, which were then integrated with an animated 3d model of the survivor in a mixed reality learning environment. Museum visitors can have a natural conversation with the virtual survivor and ask him questions, to which the virtual survivor would respond depending on natural language processing and predetermined answers. The purpose of the study was to create and evaluate this kind of narrative system from a technical perspective rather than evaluate the results of effect on users. However, creating experiences like these are important whilst it is still possible; it has been almost seventy-four years since the Holocaust ended and survivors with meaningful memories of the incident are becoming fewer with each passing year. Soon any recollections of this time-period will be lost to history and so preserving these experiences are essential for future generations.

Beyond the realm of academics, other institutions have explored the usage of augmented reality for the purposes of teaching the Holocaust. The U.S. Holocaust Museum in Washington D.C. has an exhibit called the Tower of Faces, a three-story room depicting images of the villagers of Eisiskes, a town with a large Jewish community who were executed by German soldiers in 1941. In 2018, the Museum had this exhibit enhanced with an augmented reality application [25] to describe the lives of the townspeople prior to their executions. This system was documented in news articles but was not linked to any academic studies, indicating that it was commissioned as a feature of the museum rather than to be researched for a quantitative or qualitative metric, which unfortunate as there is potential for studies to be made using this system.

Another example of an augmented reality application being used to preserve a piece of history is the Bergen-Belsen Memorial, a former Holocaust camp that was destroyed following the end of world war two. The area mostly resembles a forest with no traces of the camp remaining, but the application uses geo-localization data to display a reconstruction of the area and display what it looked like prior to its destruction along with documentation and historical records. This project was created in the research performed by Pacheco et al. [26] when studying spatial interaction with historical datasets. The purpose of this research was purely technical, the authors were investigating spatial interactions systems to further the digital heritage field. Participants in this research were divided into two groups, one freely exploring with the application and the other taking a guided tour with it. Both groups were tested afterward for spatial memory, with the free exploration group scoring higher. The authors concluded by suggesting that the data gathered will be used to continue researching the use of the application for other purposes, such as adding narratives.

2.4. Augmented Reality in Museums

Museums as an augmented reality learning environment is a concept that has been explored by other researches for various means, from educational benefit to usability. A study performed by Kyriakou and Hermon [27] analysed the usage of an augmented reality system for a museum environment. This study was performed to test user reactions when using an Augmented Reality system to examine artefacts that would otherwise be off-limits to physical interactions due to their fragility and the risk associated with allowing members of the public to touch things, such as potential damage or additional wear over time. This study utilized a mobile device with a Google Carboard head mounted display for the hardware and Unity as the engine, building a system that would allow for interaction via hand gestures. Users were tasked to perform simple operations such as grabbing an object or rotating it. The results found that an overwhelming majority of users enjoyed their experience and expressed interest in the application, with no one strongly disagreeing and only a small percentage (>10%) being neutral on the matter. There was also mention of the learning curve required to adjust to the system but also found that users with experience playing video games were able to adjust more quickly.

The research does give brief mention to the demographics of the system users but does not analyse them further which is a topic that could be explored further; is there a correlation between age/gender group and enjoyment of using an augmented reality system?

Usability of augmented reality devices for museums and cultural heritage sites is another area that also must be considered, as the effectiveness of the technology will be impeded if the visitor is unable to reliably control the system. Hammady and Ma [28] researched this in their study on the creation of a virtual museum guide, specifically within the context of using the Microsoft HoloLens as their hardware. The purpose of this study was to build and test an augmented reality application on a small group of nine users to determine how they were able to use the system along with how comfortable it was. A majority of the group claimed the HoloLens was comfortable to wear, although over a third of the participants were either neutral or disagreed that they were able to look around their environment comfortably when wearing the headset. Responses to the control elements of the hardware were varied, with a majority agreeing or strongly agreeing that that they could perform the hand controls for the HoloLens whilst a few disagreed or were neutral. The results would indicate that whilst most users were able to use the system controls, there are still concerns about comfortably wearing an HMD and being able to interact with its control system, something which the authors attributed to the bulky size of the HoloLens and its limited field of view. These concerns may possibly be alleviated with the pending release of the HoloLens 2, or potentially the use of a different HMD.

Augmented reality usage for museums stems beyond user enjoyment, there are many elements to the use of such a system with more factors to consider. Some of these factors were identified and examined in the study performed by He et al. [29] on their research into enhancing museum experiences. This research acknowledged how augmenting an exhibit is not limited to using 3D models, there is potential for usage of Heads-Up-Display (HUD) information similar to that in a video game. The experimental part of this study had users look at Vincent van Gogh’s Starry Night painting with different augmented reality displays, some using visual elements such as making the stars glimmer or the water reflect, whilst others augmenting the scene with floating text (specified as Verbal elements), giving descriptions of the painting instead. The testers were given questionnaires in the aftermath which found that the verbal elements had a greater impact on the users and their willingness to pay for similar experiences. This shows that simply adding visual effects to a scene is insufficient for an augmented reality system to engage with a user, the content itself must be contextually relevant and informative.

This point was preceded in another museum-based study performed by Keil et al. [30] on designing personalized stories with handheld augmented reality devices. The intention of this study was to utilize handheld devices such as smartphones and tablets to add augmented reality to exhibits in the Acropolis Museum in Athens, Greece. The use of the augmented reality was to provide interactive storytelling experiences that were related to the exhibits and included features such as videos, games, audio narration, imagery and digital reconstructions. This study was documented as part of a conference paper and did not publish any results; however, the discussion gave mention to an evaluation of one of the stories and discussed how the visitors wanted to use the augmented reality to obtain more information about the exhibits. Without access to the data to confirm these findings, this may not be a reliable result to obtain from this study, but this point does correlate with the data obtained in the study by He et al. [29]

Science Museums have been a target of research for augmented reality research as science is an area where abstract concepts may be difficult to explain [10] or the topic of the exhibit is not directly observable within the museum environment. This was further explored in the research of Yoon and Wang [31] of visualizing invisible concepts in science museums with augmented reality technology. The purpose of this study was to use augmented reality to visualize magnetic fields for teaching science to children between the ages of nine and twelve. The results found that the students in the augmented reality group were displaying more cognitive behaviours than those in the control group, including additional team working and problem identifying skills. The study also recorded the amount of time that each group spent with the exhibit and found that the students with the augmented reality system were interacting with the exhibit for longer by an average of seventy-three seconds, which would indicate that the system was not only beneficial for cognitive development and learning, but also for keeping the audience’s interest.

The future of augmented, mixed and virtual reality research was discussed in a paper by Bekele et al. [32] in their survey of these technologies for usage in cultural heritage. The survey is a very comprehensive technical analysis of the three technologies along with an examination of the hardware and software that enables them and their uses. The authors also discussed areas of the technology that research can expand upon in future studies. These were the examples provided;

• Robust Tracking: Use camera-based tracking instead of sensor-based tracking as it’s less prone to errors and has had more development.
• Standardization: Define or adopt a standard for the creation of the system, including the mark-up language, documentation, data structures, metadata and model format. The authors claim a community standard for project creation would assist in development of future projects.
• User-Driven Semantics: Allow the user to focus on points of interest to avoid cluttering them with information in environments with multiple areas of interest.
• Tangible Augmented Reality: utilize tangible user interfaces, a type of user interface interfacing that allows the user to interact with and directly manipulate the information provided by a system by interacting with physical objects in the real world.
• Fully Immersive Virtual Reality: Whilst not applicable to this paper, the concept involves creating a fully immersive simulation for the user to enhance their experience. The authors did comment that this type of content is very expensive to manufacture.
• Multimodal Interfaces: Using two or more natural interaction modes. These rely on a combination of sensing devices to stimulate natural interaction from the users. This initially seems at odds with the suggestion for Tangible User Interfaces, but the two can co-exist providing there are multiple interaction methods.

The authors claim that whilst each project has its own requirements, being able to expand upon these points would be beneficial to the research community. It may not be possible for a single project to expand upon all six points, but these are topics to consider for future projects to assist with expanding the medium and the understanding of it.

2.5. The Impact of Augmented Reality on Learner Motivation

In their study on motivation, Ryan and Deci [33] defined a three-category model for criteria of motivational attitudes. These criteria are as follows:

• Intrinsically motivated: The person is genuinely passionate about their task/subject and will endeavour to succeed regardless of difficulty or setbacks.
• Extrinsically motivated: The person is not fully dedicated to their task/subject but will fulfil it to gain a reward or avoid a punishment.
• A-motivational: The person has no interest in the task/subject and cannot be compelled to engage with it.

These categories can be used to define a person’s attitude toward any task or project and will affect both their motivation and drive to succeed at it. However, in another study on motivational profiles, Vansteenkiste et al. [34] introduced another motivational category: Autonomous Motivation. Autonomous Motivation occupies the space between intrinsic and extrinsic motivation in which a person will remain motivated because they are in control of their task. Within the context of education, Roy and Zaman [35] argue that gamified learning motivates learners in a qualitive manner providing that it satisfies the following psychological needs:

• The need for autonomy
• The need for competence
• The need for relatedness

As augmented reality can be used as a tool for gamified learning, it has the potential to assist as a motivational asset providing it meets those three criteria. Utilizing augmented reality for an educational task provides the user with total autonomy as they can progress at their own speed and investigate the elements that interest them, therefore having the potential to motivate them beyond what a traditional learning would be capable of.

Theory was put into practice during a study performed by Di Serio et al. [36] on how augmented reality could be used to motivate middle-school students. Students who were due to begin studying renaissance era Italian art were shown traditional paintings that were enhanced with augmented reality to include text, audio, video and 3d model files. The students were then given autonomy to free explore the augmented reality elements of their class using the hardware provided, however the results collected from this study are not fully dependable as it was unable to divide the students into control or focus groups. Sixty-nine students participated but fourteen of them were deleted from the sample due to missing data or outliers. The initial testing focused on four categories; Attention, relevance, confidence and satisfaction, each measured on a scale of one to five. In the first round of tests, these categories scored below 3.5, but almost all rose above that during the second round to 3.76, 3.48, 3.63 and 3.51 respectively. The relevance score only increased by a mean value of 0.17 between tests, which would indicate that the usage of augmented reality does not increase a student’s interest in a topic.

Unfortunately, due to the lack of a control group, it is not possible to derive from this study whether the usage of augmented reality is beneficial or detrimental to motivation when compared to traditional teaching methods.

3. Guidance for Heritage Visualization in Augmented Reality

Technology as a tool for Holocaust education has a challenging requirement; the use of any type of system must be done tastefully and in an objective manner. This applies to any complex period of history and not just the Holocaust, but nonetheless any indication of bias or misinterpretation can have consequences for the learner. This was discussed by Ethan Black in his thesis on virtual reality for history education [37], in which he discussed the ethical challenges of what can be ethically shown to a learner. Whilst this was within the context of virtual reality, the ramifications are potentially identical for any technology that intends to immerse the learner. The thesis discusses various emotional historical topics and the dangers of exposing learners to visceral content, such as experiencing the D-Day Normandy landing as a soldier or living in Auschwitz as a prisoner.

This usage of technology to visualize topics such as the Holocaust have been scrutinized for the potential ethical implications behind exposing users to disturbing imagery. This was addressed by the website Alphr in their article on a virtual reality simulation of Auschwitz [38]. The article covered an announced virtual reality simulation called Experience Auschwitz, a project created by Studio 101 intended to demonstrate the daily life of prisoners within the camp. The article took issue with the project because of its refusal to depict any violent or distressing content, which it considered to be a misrepresentation of history. However, other institutions have warned against the visualization of violent or distressing imagery, such as the Virtual Reality Society [39] who warned that users are at risk of becoming desensitized to violence and may not be affected by it in the future, both within virtual reality simulations and the real world. The United States Holocaust Memorial Museum [40] have also expressed concerns about any simulation exercises, claiming that they are pedagogically unsound and that graphic visuals should only be used when necessary to achieve the lesson objectives.

This was further explored by Bathrick et al. [41] in their book on visualising the Holocaust. The book discusses a problem that can occur during historical perspective taking; how the viewer can disregard the intentions of the photographer (or visualisation creator) and attempt to insert either themselves or their feelings on the subject into the scene as their way of trying to understand it. This process is known as empathetic substitution and the United States Holocaust Memorial Museum warns against it [40] as viewers should not be left with the impression that they have experienced or can relate to the suffering endured by the victims of the Holocaust. To this end, virtual heritage visualisations of the Holocaust should not attempt to be made as simulations or portrayed in a manner that would allow the viewer to manifest themselves into the scene as these would be disrespectful to the victims. By visualising the Holocaust, audiences are given a new means through which history can be interpreted but this is not the same as experiencing or participating in these events and should not be implied as such.

Heritage Virtualisation can be affected by the difficulties of obtaining accurate historical documentation for the creation of content. Without detailed photography or records of historical sites, any project created is built from the data has been extrapolated from incomplete or imprecise documentation. This creates the additional challenge that the project authors must communicate to audiences that the visualisation is an approximate representation based upon the historical evidence but may not be fully accurate. This was demonstrated by Kerti [42] in their work on a virtual reconstruction of the Lager Sylt concentration camp, in which an effective means to communicate the potential errors was utilised. Due to incomplete data available, the buildings depicted in this reconstruction were made as simplistic shapes and displayed in either grey or white; grey buildings represented buildings that once stood but the exact measurements or locations were unknown, white buildings were constructed to the historically accurate scale and placed in the accurate location. These buildings were not created to have complex architecture and had no textures due to the photography available; it was not possible to determine the materials used to construct them, nor the colour of them because of the black and white photographs. These decisions are incredibly effective because there is transparency from the creator about their visualisation and how it may be incorrect, which is always a possibility but the audience can understand that if there are mistakes then they have not intentionally been misinformed.

Guidelines for the development of virtual heritage have been introduced by academic institutions for establishing core principles when developing educational visualizations. An example of these would be the Seville principles [43] that seek to assist in the creation of a visualization project regardless of technological medium. These principles include the following;

• Interdisciplinarity: Having a team of experts from different areas of knowledge to collaborate and exchange both ideas and views.
• Purpose: Ensuring the project can improve aspects of research and/or conservation of architectural heritage.
• Complementarity: Any computer visualization should be used to enhance archaeological heritage and not to replace existing methods.
• Authenticity: Working to make the visualization as accurate as possible with the current understanding of the time-period.
• Historical Rigor: Showing as much of the time-period as possible, including the peak points of the topic along with their decline.
• Efficiency: The visualization should aim to use fewer resources to achieve more than traditional methods.
• Scientific Transparency: The visualization must be testable by other researchers.
• Training and evaluation: having appropriate means of evaluating the visualization.

These principles, whilst not exclusive to any medium, would be an idealized set of guidelines to follow for an augmented reality research project, especially one with the Holocaust as the theme. Interdisciplinarity would accompany authenticity; having a team of experts would allow for multiple aspects of the subject to be suitably considered and developed to ensure an accurate representation of the time-period. The purpose of such a project should always be to improve understanding of the Holocaust and the complementarity aspect should be to work alongside a historical site or museum, rather than seek to replace it. Historical rigor may be slightly more difficult depending on the area of the Holocaust being covered, but depictions of the site over time should be displayed if possible. In terms of efficiency, only the initial setup costs should be required with the possibility of maintenance or upgrade costs existing for future updates, however these should be established prior to the project commencing. Scientific transparency would mean that a different set of researchers should be able to make use of the augmented reality system and either perform new research with it or attempt to replicate existing results. Lastly, the created system would require adequate testing and evaluation to ensure that it functions as intended, this would include studies on both visitors and professionals to measure the desired outcome, whether that be functionality, educational value or another criterion. Providing these principles are followed, the created system would have educational value not just for the heritage site, but for the academic community too.

Augmented reality for virtual heritage is an expanding field; there has been a great deal of research on developing the technology and not all of it is related to educational matters. However, some of these studies have been able to analyse the core elements of augmented reality systems to identify methodologies for their creation along with their benefit. Frydenberg and Andone [44] proposed the Substitution, Augmentation, Modification and Redefinition framework (SAMR) to evaluate the adoption of augmented reality technology in an educational context and provided guidelines for creating projects to assess how effective they have been for student learning. This is how each letter of the framework is used for evaluation:

• Substitution: Using technology to accomplish tasks that could have been performed without it.
• Augmentation: Occurs when the use of technology provides an improvement or is a more effective tool for completing the same task without technology.
• Modification: Did the technology change the way learning may have taken place previously?
• Redefinition: Occurs when learners participate in activities that would not have been possible without technology.

This theory was applied to a practical study, with seventy student participants using augmented reality authoring tools to create artefacts. 92.3% of the students expressed interest in the technology and its potential applications, with 7.7% proclaiming that they would not use augmented reality again after the test. As a means of measuring the student perception of the experience, a set of forty buzzwords were selected for the students to pick from and rate on a scale of one to five. The most commonly selected words in order of popularity were Collaborative (46.25%), Accessible (40%), Attractive (33.75%), New and Fun (both 32.5%). Of the negative words, the most popular were Stressful (33.75%), Confusing (23.75%), Frustrating (21.25%), Time-Consuming (16.25%) and Inconsistent (12.5%).

The study concluded that the students who engaged with technology guided by the SAMR model were able to develop critical and computational thinking skills along with rapidly learning new technologies when they engage with the designed learning activities. However, these points were not quantified in the results as they were not tested for. The framework suggested by the study provided a benchmark set of questions to evaluate an augmented reality system with, yet this was not represented with the methodology for collecting quantifiable results. The goal of the study was to evaluate the adoption of augmented reality for education using the SAMR framework, but the results were collected using an unrelated method, leaving the framework untested as an evaluation tool. Future research would be required on this framework with a focus on evaluating against each of the four points of SAMR to test its effectiveness as an evaluation methodology.

4. Results

A summary of the referenced augmented reality systems is listed in

Table 1. List of augmented reality systems for education.

Study	Project Name	AR Design Features	Subject Area	Accuracy and Authenticity	Photorealism	Impact on Learning	Emotional Impact
Berti et al. [2]	N/A	N/A	History education	N/A	N/A	Historical perspective taking/empathizing with historical figures.	N/A
Huijgen et al. [3]	N/A	N/A	History education	N/A	N/A	Historical perspective taking/empathizing with historical figures.	N/A
Billinghurst & Duenser [4]	MagicBook	Augmented Books, Mobile Devices	Storytelling/physics education	High	N/A	Learning retention. Students were able to retain more of the lesson content when taught with AR than traditional means.	N/A
Wojciechowski & Cellary [7]	ARIES	Authoring tools Mobile devices	Chemistry education	High	Low	Usability/enjoyment testing. Students found using the system easy and enjoyed using it.	Students happier for having used the system.
Santos et al. [8]	N/A	Augmented Reality Learning Environments	Education (Various disciplines)	N/A	N/A	Committing educational content to long term memory.	N/A
Mayer [9]	N/A	N/A	Multimedia Learning Theory	N/A	N/A	Increased motivation from students when correctly applied.	N/A
Bujak et al. [10]	N/A	N/A	Mathematics education	N/A	N/A	Educational Autonomy.	N/A
Wu et al. [11]	N/A	N/A	General Education	N/A	N/A	Technical considerations and concerns of gameplay distractions or cognitive overload.	N/A
Dunleavy et al. [12]	Alient Contact!	Handheld devices, GPS markerless Augmented Reality	Mathematics, language arts and scientific literacy	N/A	Cannot be seen, no screenshots provided in study.	Motivation/student engagement. Dangers of cognitive overload.	Potential implications to cause stress to the user if they are overloaded.
Iten & Petko [13]	AWWWARE learning game	N/A	Computer education	N/A	N/A	Attitudes toward serious games and how enjoyment influenced motivation to learn.	N/A
Rosenbaum et al. [14]	Outbreak @ The Institute	Handheld devices, markerless augmented reality	Pathology/medical education	High	N/A, no visuals	Location-based studying and its influence on learning.	N/A
Kamphuis et al. [15]	Miracle	Projected CT Scan data, simulated training environments	Medical education	High	High	Immersive training environments, simulated realism.	N/A
Blanco-Fernández et al. [16]	REENACT	Handheld devices, marker-based augmented reality	History education	High	Medium	Historical perspective taking/empathizing with historical figures. Collaborating with experts to understand the consequences of historical decisions.	N/A
Chang et al. [17]	N/A	Handheld devices, digital tours	History education	High	N/A	Sense of place within a learning environment and the benefits of an augmented reality system within them	N/A
Harley et al. [18]	MetaGuide	Mobile devices, object recognition	History education	High	N/A	Location-based learning, historical differences and changes over time.	N/A
Choudary et al. [19]	N/A	Mobile Devices	History Education	High	N/A	N/A (No results on learning impact)	N/A
Herbst et al. [21]	TimeWarp	Mobile Devices	History Education	High	Medium	N/A (No results on learning impact)	N/A
Keil et al. [22]	House of Olbrich	Mobile Devices	History Education	High	Low	N/A (No results on learning impact)	N/A
Stapleton & Davies [23]	MemoryScape	N/A (Proposed project for markerless augmented reality)	Holocaust/history education	N/A	N/A (No screenshots provided, only sketches)	Location-based learning, historical empathy.	Emotionally challenging content and the difficulties approaching it tastefully.
Ma et al. [24]	Forever (a.k.a Interact)	Mixed reality, natural language processing	Holocaust/history education	High	High	Natural language interactions and historical empathy, a preservation of	Interacting with a virtual survivor and
Takahashi, Dean [25]	Tower of Faces	Mobile devices, object recognition	Holocaust/history education	High	N/A (Augmented information, no visuals)	History of Holocaust victims.	Emotionally challenging content, exposure to stories of Holocaust victims.
Pacheco et al. [26]	Bergen-Belsen Memorial AR	Mobile devices, GPS markerless augmented reality.	Holocaust/history education	High	Low	Historical empathy, sense of place, location-based learning.	N/A
Kyriakou & Herman [27]	CHESS	HMD’s, interactable content	History education	High	Medium	Enjoyment of interactions with an augmented reality enhanced museum exhibit.	N/A
Ramy & Ma [28]	MuseumEye	HMD’s	History education	High	Medium	User comfort when wearing a HoloLens HMD and their ability to interact with its control set.	N/A
He et al. [29]	N/A	HMD’s	Art and history education	N/A	N/A	Different types of enhanced exhibits and how verbal enhancement is more beneficial for educational purposes than visual enhancement.	N/A
Yoon & Wang [31]	N/A	Object recognition	Science education	High	N/A	Cognitive behavioural development and effect on motivation	N/A
Bekele et al. [32]	N/A	Mobile Devices, HMD’s, Interfaces, tracking types.	Augmented, Virtual and Mixed Reality	N/A	N/A	Creation of Augmented, Virtual or Mixed Reality projects for Cultural Heritage, along with different technologies and their uses.	N/A
Deci and Ryan [33]	N/A	N/A	General education	N/A	N/A	Different motivational categories and how they impact learning.	N/A
Vansteenkiste et al. [34]	N/A	N/A	General education	N/A	N/A	The impact of motivation on learning.	N/A
Roy & Zaman [35]	N/A	N/A	General education	N/A	N/A	Gamified learning and how to implement it within a classroom environment.	N/A
Di Serio et al. [36]	N/A	Mobile devices	Art and history education	High	N/A	Effect of augmented reality on student motivation.	N/A
Black, Ethan [37]	N/A	N/A	Holocaust/history education	N/A	N/A	Historical empathy	Emotional challenges of exposure to visceral content and ethical implications.
Gardner, Elliott [38]	Witness: Auschwitz	N/A	Holocaust/history education	Medium (Deliberately avoids certain content)	High	Immersive simulation of life within the Auschwitz Holocaust camp. Historical empathy.	Emotional challenges of exposure to visceral content and ethical implications.
Kerti [42]	Explore Lager Sylt	N/A	Holocaust/history education	High - potential inaccuracies communicated to the audience.	Low	Virtual representation of a destroyed Holocaust Site designed to give an accurate depiction based on historical evidence.	N/A – no emotional content.
Frydenberg & And one [44]	N/A	Mobile devices, authoring tools	Computer education/authoring augmented reality tools	N/A	N/A	Evaluating an educational augmented reality system upon for effectiveness.	N/A

.

Referring to the questions asked in the introduction, these are our findings from the literature.

4.1. Does Usage of Augmented Reality Correlate to Higher Understanding of a Historical Topic?

Augmented reality has been found to positively impact upon student performance. Students who used an augmented reality system as part of their education scored higher on tests [4], indicating higher learning retention. Studies have found that test scores or student understanding have improved after exposure to an augmented reality learning system, but test scores alone do not represent a higher understanding of a topic. Emotional understanding and empathy are also key factors in history education, which have also been found to be improved with the usage of an augmented reality system [14,17]. The other factors to consider are long term versus short term memory; taking a test after engaging with a system will yield results from short-term memory, but does augmented reality assist in committing these experiences to the long-term memory? The study by Santos et al. [8] supports that it will, providing the system contains the correct usage criteria. The results of these tests strongly suggests that augmented reality usage can correlate to higher understanding, but only providing that the system is able to meet the cognitive and emotional requirements to correctly stimulate the user.

4.2. How Effective Is Augmented Reality in Classroom Environments?

Augmented reality within a classroom has provided the benefit of autonomy within education; students can take control of their learning and progress at their own speed [7,10,14,34,36]. This is a positive impact upon education as it provides an individual catering to each student, rather than making assumptions about what speed a class is comfortable with. The criteria to achieve this level of autonomy are fun, challenge and curiosity 11] and must satisfy the users’ needs for autonomy, competence and relatedness [35].

4.3. Are There Any Detriments to the Use of Augmented Reality and If So, How Might They Be Mitigated or Overcome?

There are complications with using an augmented reality system for educational purposes. The first of which is the potential for cognitive overload: the student gets overwhelmed by either the complexity of the system or the amount of information on screen [11,12]. This can be mitigated in development by making an accessible design for users and by limiting the augmented content to avoid overloading the user’s senses. Another risk is user distraction [12] as the user could potentially become injured if they are too focused upon the application but depending on the environment this may be an issue that would have to be addressed by teaching staff. Gameplay elements are also a source of contention as a player having fun does not mean that they are learning [13] meaning that the focus of an augmented reality system should be on the content rather than the entertainment.

4.4. Have There Been Any Studies Performed within a Museum Environment for Augmented Reality Technology and If So, Did the Results Differ to the Studies Performed within a Classroom?

Augmented reality has been tested within museums before and has found that in terms of usability, users have found that the technology is incredibly enjoyable with expressed interest in wanting to use it more [27]. When tested for purpose within a museum, the results found that users preferred exhibits that were enhanced by information rather than visual effects [29]. Exhibits that are enhanced with information via augmented reality also increased the amount of time each user spent at an exhibit and caused users to demonstrate behaviours which indicated additional cognitive development [31]. Results did not contrast with classroom environment studies as the focus of the tests was on different criteria.

5. Conclusions and Future Work

The results from previous research are highly encouraging for future studies and projects as they indicate that usage of augmented reality has been able to stimulate learning engagement in bold new ways that have previously been unavailable. Many of these studies have noted a lack of empirical research performed within the area, signifying that there is still a great deal of potential study still to be done. The results found thus far have indicated very strongly that as a technology, augmented reality has the potential to revolutionize the way that certain subjects are taught, along with being able to bring new experiences or revitalize old ones at museums, heritage sites and other areas of historical value. Historically significant events that have occurred within living memory have the rare and unique opportunity to be recorded within the confines of modern technology, such as augmented or virtual reality so that the first-hand accounts can be told or experienced directly by current and future generations. Documentaries and interview recordings existed prior to this technology emerging, but they lacked the capacity to actively engage with the delivered content.

Future research projects into the development of augmented reality applications for educational purposes should be aware of the requirements of the users to engage with a system, along with the potential hazards that they will face when interacting with it. The studies show that technology is effective for motivating learners [35,36], stimulating interest and building short term memory [4]. We seek to utilize the knowledge gained from this review in our upcoming collaboration with the Neuengamme Holocaust Memorial Site, with whom we are collaborating to develop an augmented reality application for educational purposes.

We feel that augmented reality is the ideal choice of technology because it can be used to recount experiences that require the user to be physically present at the site to truly comprehend. An example of this would be the interior of the brick factory itself, a very large and open room that no longer has any original furnishings aside from a structural support and markings on the floors to represent where the factory components once stood. The building itself once housed kilns for firing the bricks, but these later repurposed and used as beds by prisoners of war when the camp was converted into a prison for the former SS soldiers. augmented reality technology could assist with displaying and preserving these parts of history which may be difficult for a visitor to imagine or no longer exist as they once did, such as the kilns which were removed long ago. Visitors to the museum are given tours in groups, therefore allowing a collaborative experience [10] in which all visitors experience the exhibits together. This gives visitors the ability to communicate and learn as a group, thereby giving more opportunities to actively interact with their experience rather than passively learn by observing [7].

Although virtual reality would also be able to visualize the interior of the building, augmented reality is more suitable a medium because of the physical immersion within the environment. Virtual reality can create an immersive atmosphere but lacks stimulation of certain senses, such as being able to smell the environment. There are also the ethical implications of exposing visitors to a virtual reality Holocaust experience [38,39] and the complications of structuring it. A museum tour with augmented reality elements does not require a narrative and has the benefit of the museum guide who can answer any question asked. A virtual guide could be programmed to answer questions, but these would be limited to pre-recorded responses rather than the potentially unlimited responses of a real guide. In terms of resources, a virtual reality experience would also require a longer development time and cost more to both deploy and maintain. Augmented reality has the benefit of being deployable to Android or iPhone devices through a Wi-Fi internet connection; specialised hardware would not be required by the museum providing visitors brought their phones with them.

The research focus of this study will be to expand upon study on the effect of augmented reality on developing long term memories. Whilst the theory of this has been covered in this review [8], there is a lack of empirical study on how effective augmented reality is for creating long term memories. The testing of this project will educate users on the Neuengamme Holocaust Memorial site using two groups; a focus group and a control group. The focus group will have their experience enhanced with an augmented reality system and the control group will be given a more traditional class on the subject. At intervals following these classes, the groups will be given online tests to determine how much information they have retained about this part of the Holocaust and have the results compared to see which group was able to remember the most. This will demonstrate the impact of augmented reality upon long term memory creation memory retention.

This goal is particularly important due to the nature of the Holocaust; crimes against humanity were committed under the banner of nationalism and without preserving these stories in a neutral and educational manner, the conditions that lead to them occurring may be repeated in the future. This was an event that should never be forgotten, and if an augmented reality application has the potential for creating long term memories that will assist in preserving this area of history then it is a medium that must be explored for both educational purposes, and to ensure that visitors never forget what happened during the Holocaust.