Augmented Reality in Seismic Risk Management: A Contribution to the Reduction of Non-Structural Damage

Abstract

To increase seismic resilience is one of the challenges the developers of new technologies face to reduce seismic risk. We set up an augmented reality (AR) exhibition with which users’ curiosity was confronted with the opportunity to have a wealth of information on damaging earthquakes that could be a multimedia add-on to the plain “single-layer exhibit”. AR is an emergent technology developed to “augment” reality through various devices; it combines the real world with virtual items, such as images and videos. Our AR exhibition aims to: (i) show the effects of earthquakes even in cases of moderate magnitude; and (ii) promote preventive actions to reduce non-structural damage. It can be customized for different seismic scenarios. In addition, it offers a holistic approach to communicate problems and solutions—with the cost and degree of ease of execution for each solution—to reduce non-structural damage at home, school, and office. Our AR exhibition can do more than just a plain text or a preconceived video: it can trigger fruitful interaction between the presenters, or even the stand-alone poster, and the public. Such interactivity offers an easy engagement to people of all ages and cultural backgrounds. AR is, indeed, extremely flexible in raising recipients’ interest; moreover, it is an appealing tool for the digital native generations. The positive feedback received led us to conclude that this is an effective way to raise awareness and individual preparedness to seismic risk.

1. Introduction

Virtual reality (VR), machine learning (ML), and internet of things (IoT) belong to a field of emergent technologies that offer solutions to seismic risk management; they have a rapidly increasing number of applications (e.g., [1]), such as serious games for building earthquake preparedness (e.g., [2]), evaluations of behavioral response to an earthquake (e.g., [3,4]), supervised classification of seismic vulnerability using photographs of buildings [5], and earthquake early warning systems [6]. The digital world offers a wealth of information that can reach us through several channels; however, how much information are we really able to retain? From the first pioneer equipment for aviation realized in the late 1960s to games such as Pokémon Go in 2016, augmented reality (henceforth AR) is a technique that has proved to be effective for learning (e.g., [7]) and memory encoding [8]. In particular, neuronal studies document almost three times the level of activity of the human brain using AR in subjects exposed to specific tests [9].

The name of the technique was coined in the early 1990s by Caudell and Mizell [10]. AR overlays digital data in various formats (numerical, graphical, video, etc.) on the real world, providing various kinds of information that “augment” our knowledge. The way this is done is unique: it is based on a personal choice of exploration and, in other words, relies on curiosity and on the chance to enhance it; thus, the user is not only passively watching. The implementation of AR applications (apps) currently requires the use of mobile devices, such as smartphones and tablets. Toolkits for app developers have allowed spreading AR in many fields, from military to medical, to entertainment (e.g., [11]). Moreover, the computer rendered overlay is effective in enhancing understanding, engagement, and attentiveness. Exploring the literature on science education, Swensen [12] describes how children, from the use of AR, can achieve a positive outcome in cognitive effort, motivation, situated learning, and inquiry-based learning. Performance improvement in all types of training is also reported by Cöltekin et al. [13]. Taken together, these features gave us a glimpse of the great potential of AR—among emergent technologies—in disaster risk management; in particular, in damage mitigation, a commitment in which every person is involved.

In this paper, we describe our AR exploitation using the “talking poster”; this is a display interactive stand for custom AR apps. Focusing on seismic risk communication, the talking poster allows AR users of all ages and cultural backgrounds to engage on topics related to earthquakes; raise awareness on the need and effectiveness of prevention; and build the capacity to reduce damage starting from simple actions that every single citizen can undertake. The proposed exhibition is modular and set up in such a way as to form a variable number of thematic sections according to the target audience and the characteristics of seismic activity of the geographic areas for the showcase. Using AR technology, we highlight that preventive reduction of damage caused by earthquakes can effectively provide safe evacuation of buildings, continuation of operations, and a faster resumption of commercial activity.

2. The Challenge: Increase Resiliency

A more resilient world to the impact of natural disasters requires awareness and preparedness of each person, no one excluded. Looking at the numbers of victims for earthquakes, the death toll is high: the Emergency Events Database (EM-DAT) reports 26,931 deaths on record in the world between 2009 and 2018 [14]. Furthermore, in the European Union alone, the natural disasters that occurred between 1980 and 2020 caused damage amounting to € 24 billion (the World Bank, https://www.worldbank.org/en/news/feature/2021/06/04/economics-for-disaster-prevention-and-preparedness-in-europe (accessed on 20 September 2021). Seismic prone areas show great differences in the magnitude of earthquakes and in the impact produced. It is worth noting that even small magnitude earthquakes (M ≤ 5) cause costly disasters. Indeed, building safety depends not only on structural parts, but also on the non-structural components (NSC) whose damage is the source of severe economic losses (e.g., [15]; Figure 1). Along with architectural components (such as parapets, chimneys, and decorations) and building utilities (pipes and systems for heating, electricity, gas, and water), NSC are furnishing equipment that can fall and topple inside homes, offices, and schools due to the ground shaking. For example, the NSC in an office make up, on average, 82% of its overall content [16]. Therefore, even small earthquakes can disrupt local communities from the social and economic viewpoint by damaging NSC. This mostly occurs because worldwide codes are not fully compliant with the NSC of buildings, and citizen education on preventing damage to non-structural elements is rarely pursued.

The two-year European project named KnowRISK (Know your city, Reduce seISmic risK through non-structural elements; grant agreement ECHO/SUB/2015/718655/PREV28) ended in 2018. Moving on from the assessment of seismic activity in a few European countries (Portugal, Italy, and Iceland), it had a special focus on non-structural damage caused by earthquakes; it explored the causes of disruption and proposed measures to reduce it (e.g., [17,18]). The KnowRISK Practical Guide [19] (https://knowriskproject.com/practical-guide/ (accessed on 16 November 2021)) and Portfolio of Solutions [19] (https://knowriskproject.com/portfolio/ (accessed on 16 November 2021)) provide useful suggestions and information for a wide audience, from youths to professionals. Benefiting from the multidisciplinary background of the partners, who were engineers, geophysicists, architects, and psychologists, the project relied on risk communication as a preventative strategy, organizing many dissemination activities [20]. Within this framework, we designed our AR apps, leveraging them in our commitment to increase seismic resilience.

3. The Talking Poster

AR is known for its interactivity, simplicity, and efficacy [21]. On this basis, we have chosen AR to raise awareness on seismic risk and disseminate information by using the most common mobile devices: smartphones and tablets. This way of communication is also well accepted by the so-called digital native generation, and can create useful links with the educational world at different levels (primary and secondary school, and university).

We designed a display interactive stand in which “augmented” posters could tell the AR user their own content (Figure 2). We named it “the talking poster”, a prototype of which was described by Reitano et al. [22]. Our talking posters contained 2D images that worked as virtual switches for a tablet or smartphone provided with image tracking (Figure 2 and Figure 3).

The camera of the mobile device tracked each target by means of the object identification system of the AR app. The digital content associated with the target appeared on the tablet or smartphone display and, if opportune, was transferred to a large multimedia monitor.

The use of a large display has been very effective for AR demonstrations to groups of people at festivals and scientific meetings (Figure 4). A sound system (speakers) completed the exhibition to reproduce the sounds (Figure 5).

The targets with the static content of the talking poster were set up to form thematic sections (Figure 5). Based on modularity, the number of sections was changed; further, the overall content of each one was calibrated according to the audience (youths, laypersons, professionals) and the context of the showcase (Falsaperla et al. [20] provide a complete list of the events).

Since implementing preventive safety measures is of paramount importance, the talking poster gave not only information on the problem (earthquake hazard and risk), but also practical solutions to prevent damage. The static and digital contents encompassed a selected knowledge production coming from KnowRISK and other studies aimed at consolidating the science–practice interface. Examples of these contents are reported in the following.

The sections on seismic hazard and risk contained maps of Europe (e.g., [23]) or of single countries; seismograms with the associated “sound” of earthquakes (speeding up the frequency of the seismic record) were particularly impressive in the case of seismic swarms with tens or even hundreds of events. These included those of Etna (Italy); the distribution of the epicenters (dynamically deployed onto a satellite map of the zone in which the AR demonstration took place); and diagrams depicting the percentage of structural and non-structural components in hospitals, hotels, and offices (from FEMA [16]). Effects of the potential impact of earthquakes—such as the fall of plaster blocking escape routes in a school—were discussed with the AR users; the users were asked to identify NSC before these were highlighted by the AR applications on snapshots of outdoor and indoor places. Objects of the discussion were also a few shaking table tests, such as the videos inside and outside a two-story house realized by UC Berkeley (2000; http://seismo.berkeley.edu/~rallen/seishaz/buildings/CUREEtest/ (accessed on 21 January 2022)). The indoor footage concerned a living room and a bedroom; by watching these videos, we invited the AR users to reflect on the potential toppling of hanging objects and furniture in their homes in the case of earthquakes.

In the sections for the prevention and reduction of damage, AR disclosed images and videos inspired by three key points: move, protect, and secure; these points are at the core of our communication strategy. The talking poster depicted snapshots with the effects of ground shaking (for example, inside a supermarket) and their possible solutions (tilted shelves) to prevent non-structural damage. In these sections was also the KnowRISK spot, with other simple protection measures (https://knowriskproject.com/move-protect-secure/ (accessed on 16 November 2021)). An additional video encouraged the fixing of various problems at home, school, and office; for this, no technical expertise is required, promoting the use of the KnowRISK Practical Guide (see Video SI1). By giving examples of damages and solutions, every person (including young people) can reflect on the advisability of adopting preventive measures, even at zero cost (e.g., moving heavy objects from high to low shelves).

4. Communication and Related Perceptions of Seismic Hazard and Risk

Italy, Portugal, and Iceland are three different contexts in terms of seismic hazard and risk not only for magnitude, but also for the occurrence frequency of earthquakes, time span since the last damaging seismic event, type of damage, level of implementation of protection measures, cultural attitudes towards prevention, and actions undertaken at the local level to increase resilience. The KnowRISK communication campaign was based on the idea that it is possible to trigger best practices by appropriately modifying people’s knowledge and attitude [24,25,26]. This concept is summarized by the Knowledge-Attitude-Practice (KAP) [27], where Knowledge refers to the understanding of earthquakes and associated risks; Attitude is linked to preconceived feelings and ideas towards hazard and risk; and Practice is how communities demonstrate their knowledge and attitudes through the actions they undertake. As part of the KnowRISK communication campaign, our talking posters were prepared with KAP in mind and responding to local needs.

People living in Catania (Italy) perceive earthquakes as probable, dangerous, and high-risk [28]. While still underestimating the seismic hazard for the area they live in, they are aware of it. Mostly, they needed their knowledge to be enhanced and a demonstration that easy-to-implement, low-cost actions can significantly reduce seismic risk.

On the other hand, people living in Lisbon (Portugal) may never have felt an earthquake and perceive seismic risk as a low priority [29]. Information on seismic hazard exposure should act as a warning, leading individuals to ask themselves: “Is there a real threat to be reckoned with?” However, knowledge and awareness of the hazard do not necessarily lead to the immediate adoption of protective action. For this reason, the talking poster is not just a presentation of information, but acts as a stimulus for discussions and questions on the relevance of the risk (i.e.: “Is the seismic risk really relevant where I live?); and foreseeing a possible situation (i.e.: “What would happen to my town if an earthquake occurred?”); before once again indicating easy and low-cost interventions capable of significantly reducing non-structural damage.

Finally, the South Iceland seismic zone is where the earthquakes of June 2000 and the Ölfus earthquake of May 2008 caused mostly non-structural damage (see, e.g., [30]); while most parts of the residential buildings suffered significant ground shaking without major structural damage. In this context, we set up the talking poster to provide advices for keeping calm and maintaining balance during an earthquake; the poster also provided tips on what can be done to easily reduce non-structural damage.

5. Method: How AR Works in Our Apps

Since the proposed experience is multipurpose, we decided not to use a single generic app. Each talking poster had its targeted audience and referred to specific contents, which were related to a specific issue (a geographical point of view, recent or past earthquakes, etc.). Targets and related output were the basis of the app deployment. In particular, we referred to the Wikitude™ framework provided by Wikitude GmbH (www.wikitude.com (accessed on 31 January 2018)), under Android OS version 4+. Wikitude offers a software development kit (SDK) for Android, exploiting web technology such as HTML. ARchitectView is a custom Wikitude class that we applied to our platform to exploit AR. By using the Wikitude™ framework, target images are first proposed to a custom tool; the tool provides a score for each single image. Once the targets are validated by the system, they are integrated in the app during the code-building phase. Consequently, there is no need for an external server that hosts the images; as a result, this technique also offers the possibility to use the app if an internet connection is not available. The same image can be reused for different targets; this is because each app is compiled for a specific goal and its results depend on the version used.

A brief description of the hardware used in our AR exhibition is the following (see also Figure 5). Our talking poster required:

• A mobile device (smartphone, tablet) with a camera: the front-end device and run by the presenter of the exhibition or end users;
• An object identification system (for target identification): in the talking poster, images were typically tracked as main targets; they provided details, videos, and sounds;
• A display (a digital interactive whiteboard, DIW, or large monitor for groups of people): to let the audience be an active part of the presentation, the monitor or DIW showed in real time what the smartphone or tablet was capturing and revealing by adding the scene with AR media outputs;
• A Chromecast (optional, an access point, TP-Link™ router wireless, for Android Wi-Fi streaming): the Google Chromecast with its software enabled the connection between devices and duplicated the information streamed by the smartphone;
• A sound system: speakers completed the hardware in order to share the sounds with the whole audience.

The AR apps were installed in the tablets and smartphones that the visitors of the exhibition were invited to use. For small groups, a few sets of dedicated devices were available to the participants; while in the case of large groups, the exhibition had a single user (the presenter) and the “augmented” content of the mobile device was also displayed on a large, digital interactive whiteboard (Figure 4). The mobile device was connected through a Chromecast to ensure the Wi-Fi streaming of the digital content.

6. Discussion

The “talking poster” was customized for different seismic scenarios in the three countries involved in KnowRISK, namely Italy, Iceland, and Portugal; and within various contexts, from national open-door events (e.g., Settimana del Pianeta Terra and Week of Planet Earth [31]) to international scientific meetings (e.g., [32]; the AGU Fall Meeting in 2019). The AR display stand in Figure 4 is an example. It was designed for a meeting in Catania, Italy, in which we presented the seismic activity in Sicily with a particular focus on the Etna volcano (Figure 1). Etna is a basaltic volcano with frequent eruptions [33]. Here, many earthquakes with a magnitude of up to ~5 have shallow hypocenters (within a few kilometers from the ground surface), and cause severe non-structural damage [34]]. In addition, the seismic occurrence rate is high, with 2236 earthquakes (Ml ≥ 1) in 2021 alone [35]. As this context is responsible for a first-hand experience of earthquakes even among children, the thematic sections of the talking poster emphasized preparedness and response above all.

The taking poster for Iceland discussed damaging earthquakes that resulted in the spreading affecting the island due to the divergent movements of the North American and the Eurasian tectonic plates. Icelandic people have a high awareness of seismic risk; however, the implementation of good practices to reduce non-structural damage remains insufficient [36].

A completely different seismic scenario is to be found in Portugal and the “talking poster” was customized accordingly. For example, Lisbon has a low seismic occurrence rate, and the capacity of people to absorb the idea of preparedness finds more resistance here [36]. Nevertheless, the historical record encompasses relatively recent damaging earthquakes; for example, that of M7.9 in 1969, which struck the SW of Portugal [17]. It was pointed out that non-structural damage was apparently only a minor issue. The talking poster featured the damage caused by the 1998 Azores earthquake and images of the many non-structural elements on the streets of Lisbon. The idea was to help laypeople connect what happened in the past with what might happen in their neighborhoods. People have not only been presented with the problem, but it was shown that the solutions are worthwhile as they can cost relatively little effort and money.

Despite the differences among the aforementioned targeted audiences, empowering our building resilience requires the commitment of each person; it becomes more and more a societal issue of paramount importance. Our talking poster offered a holistic approach—made of sounds, images, and information—that made our way as geoscientists to communicate problems and solutions effective; we proposed preventive safety measures to reduce non-structural damage at home, school, and office. The answers to questions on specific problems (e.g., how to fix objects) could be found in the AR exhibition itself, along with the cost and degree of ease of execution for each solution (e.g., through the KnowRISK portfolio). This can effectively help to eradicate the idea that damage cannot be avoided. Furthermore, emotional involvement facilitates communication; in turn, this results in positive leverage on AR users’ attentiveness. Emotional and cognitive aspects are beyond the scope of this paper, even though they play an important role in effective communication; we address readers interested in these topics to Cöltekin et al. ([13] and references therein).

7. Conclusions

News content in our media world mostly focuses on earthquakes through highlighting basic data (locations affected, magnitude, deaths, and injury counts); however, little or no information is provided on preventive measures. Devès et al. [37] document such biases in earthquake media coverage on seismic risk; they analyzed a dataset of ~7000 earthquakes with M > 4.5 in 2015, finding a meagre 5.6% of the related international news items on recovery, restoration, and reconstruction, and none focused on preparedness. In our view, disaster risk management, in general, and seismic risk, in particular, could overcome this drawback by relying on new and emergent techniques to catch the attention of the general public, convey information that stands on solid science, and promote good practices that enhance resilience. In this respect, AR can: (i) offer valuable hands-on learning experience in the education of disaster management; (ii) enable effective engagement on seismic risk reduction with society at large (i.e., even non-professionals); and (iii) allow users to retain information that often requires an emotional engagement. Effective engagement is supported by the fact that the public is not passively watching, but can choose what is worthy of being “augmented”.

Our AR apps were developed for mobile devices with widespread use, such as tablets and smartphones. AR glasses may be another effective portable device. At the beginning of 2015, there were high expectations for a large commercial uptake of AR glasses by the end of 2020 [38] (https://www.augmentedreality.org/smart-glasses-report (accessed on 23 September 2021). This result has not yet been achieved; however, huge developments of various forms of mixed reality (digital plus physical) have been accomplished (e.g., Microsoft’s HoloLens) and further forthcoming progress can be envisaged.