Studies from all over the world have shown that history and cultural heritage are often presented at schools, at museums and in the media in strongly-politicized ways, that nurture attachment to the idea of homogeneous nations that transcend class divisions and erase local specificity [1
]. The reinforcement of such siloed and one-sided structures of collective memory is seen as one of the most pressing contemporary challenges in a world marked by boundaries, tensions and polarization [5
]. Accordingly, many scholars and institutions have advocated for cultural diversity and promotion of transnational understandings of history, focusing on pedagogy to embed democratic principles of inclusion, inquiry, integration of multiple interpretations and freedom of expression [6
Knowing that textbooks have fixed structures, museum exhibitions are quasi-static, and most of the mass media have well-defined editorial policies (not to say ideological bias), the active usage of internet-enabled mobile devices and online social networks could be one key to realizing a shift to these new models, helping people develop awareness of their own personal and national identities through an understanding of their own and others’ histories and cultures [8
In our view, the way historical and cultural heritage knowledge is experienced and interpreted can be significantly improved by the recent advances of artificial intelligence (AI). The formal, explicit and machine-readable specifications than underpin semantic web technologies can be employed for delivering appropriate content to the users of mobile devices, in a way that encourages curiosity about intercultural and cross-border aspects, provokes thought and leaves a long-lasting impact in their minds. In particular, we argue that AI should be aimed at discovering historical/cultural symmetries, like the following one posted on Quora in November 2018 in reply to the question “How do Spanish people view Turks and Turkey?”:
(We view them) like a very distant, exotic culture with almost no ties to our own country or whatsoever, kebabs aside. Most Spanish people have outright forgotten about our shared history and military conflicts with the Ottomans. …
I personally think that Turkey … is a “Spain in reverse”, a sort of weird twin of us. Both are countries located in rugged peninsulas, both had a Muslim/Christian extensive past, yet they were conquered by Christians/Muslims and started their own Christian/Muslim empires. [Retrieved on 22 November 2018, via qr.ae/TUh64C]
This comment conveys powerful images and associations, easy to retain and with the potential to leave the reader wondering about historical clashes or alliances between the two sides, and about common or opposing cultural aspects. If the symmetries are selected according to the interests and preferences of the users, as well as their context (e.g., their current location, the subjects they are studying at school, or whichever topics resound in their social networks at the moment), then the positive impact can be amplified.
Following the notable investments in digitization during the early 2000s [10
], nowadays there are vast numbers of digital resources available that could be trawled through searching for historical and cultural symmetries. Obviously, this is too great a task for a lone humanities expert, and that is precisely where AI is called to provide assistance. AI has the ability to process the enormous archives to discover the pieces of information that could be related in order to identify relevant associations and thereby deliver meaningful messages or stories.
In this paper, we discuss how advances in semantic web and deep learning technologies can be used to identify historical and cultural symmetries between different characters, locations, events, venues, …, and how these can be harnessed to develop new strategies to improve the pedagogy of history and cultural heritage. These strategies—grounded on the state-of-the-art in the areas of semantic reasoning [11
], personalization [12
] and digital storytelling [13
]—have been put to the test in the context of the European project CrossCult (www.crosscult.eu
). We look at the potential to provoke curiosity in cross-border topics and consider how such novel applications can support learning processes.
The paper is organized as follows. In Section 2
, we summarize the main ideas of how semantic web technologies can be used to create a repository of machine-analyzable knowledge that interconnects the history and cultural heritage of different countries. Then, in Section 3
we present the key approaches followed in CrossCult to discover/crowdsource symmetries between different characters, locations, events or venues, and thereby develop transverse readings of history and cultural heritage. Then, in Section 4
, we illustrate each approach with examples drawn from pilot experiences that we conducted within the project since March 2016. Final conclusions are given in Section 5
2. The Semantic Web of History and Cultural Heritage
For many years, museums all around the world have been using disparate and largely incompatible databases to gather and manage the information about their collections, although there are a few cities or regions that have common databases for their tangible or intangible heritage [10
]. In the meantime web sites have flourished, but they constituted information silos with little possibilities for exchange and reuse. What is more, the information offered to the public often contained raw data and with little substance or detailed metadata (e.g., for a picture we would find its size, author, year and a brief synopsis) so the interested user had to seek supplementary information through searches in external resources such as Wikipedia.
Interestingly, Wikipedia showed a new way to go in making the knowledge contained in its articles repurposable, thanks to the development of the DBpedia database (wiki.dbpedia.org
) and Wikidata (www.wikidata.org
). These projects aim to extract structured content (i.e., machine-readable knowledge) from Wikipedia, and to provide means to query relationships and properties of any resources in it. DBpedia is arguably one of the most popular resources of the semantic web, having fostered an ecosystem of projects that use the structured content for different purposes in many areas of application.
The key idea is to break down the knowledge into triples, which are sets of three entities that codify individual statements about data in the form of subject-predicate-object expressions (e.g., statements to be read as “Bob is 35”, or “Bob knows John”, though with a formal syntax). DBpedia now contains more than three billion such triples, connecting more than 4.58 million entities, including 1.5 million persons, 735,000 places, 120,000 historical events, 550,000 artworks, plus 26 million links to images and 30 million links to external web pages. This is an invaluable asset per se for uses in history and cultural heritage area, but the DBpedia syntax does not reach as far as to capture all the relevant pieces of information and their relationships. This is where domain-specific Semantic Web resources are needed. Fortunately, many big museums and other institutions worldwide have recently agreed on a common format for representing information about their collections, as reflected in successive versions of the CIDOC conceptual reference model (CRM) [16
], an international standard (ISO 21127:2006) for cultural heritage documentation.
shows an excerpt of semantic information about one painting from the National Gallery London collection. The basic CIDOC-CRM constructs capture the information that this painting is titled “Alexander and his Doctor”, it was created by Eustache Le Sueur at some point between 1646 and 1651, it is currently located at Room 09, it has a certain size and the artist used certain materials (oil painting on canvas) for its production.
Given some plain text (e.g., a description of a cultural heritage item or a narrative), we can apply named entity recognition and linking techniques to further enrich the textual descriptions with links to semantic web resources. Using DBpedia Spotlight [17
], for example, we can relate items to DBpedia entities, thus allowing associations to be identified that go through facts from general culture. In turn, other resources from the Linked Data initiative (linkeddata.org
) may enable more fine-grained reasoning about geographical features around the world, sports events, yearly celebrations relevant to specific regions, facts of nature, health or etymology, etc.
The knowledge base created in the CrossCult project (hereafter, the CrossCult knowledge base or CCKB [18
]) supplements the CIDOC-CRM constructs with the semantics of SKOS (Simple Knowledge Organization System (www.w3.org/2004/02/skos/
)), which are used for representing thesauri, classification schemes, subject-heading systems, or any other type of structured controlled vocabulary. We use the Dublin Core schema (dublincore.org/documents/dcmi-terms/
) as standard vocabulary for describing digital resources, and FOAF (Friend of a Friend (xmlns.com/foaf/spec/
)) ontology to capture user-related entities and their interests. Within the framework provided by these norms, we have defined a set of new antagonistic properties and relationships to capture useful knowledge for the purposes of managing historical and cultural symmetries, such as the fact that one character may be seen both as a “hero” by some people and as a “villain” by others, that the same military event may be both perceived as a “victory” and a “defeat”, etc. Likewise, we can model periods of alliance or enmity, and such relationships as supporter-hater, supplier-provider, etc.
The CCKB comes with additional resources for the purposes of reflection-provoking storytelling. The central notion is the reflective topic, referring to what a particular set of semantic entities (historical events or characters, exhibits, etc.) can trigger reflection about. A reflective topic is characterized by a title (“Medicine and healing in art” in Figure 1
) and related terms from a vocabulary scheme of broader and narrower concepts (“medical treatment”, “health” and “therapeutics”). This vocabulary aggregates terminology from standard thesauri resources such as the Arts and Architecture Thesaurus (AAT) of Getty, the EU’s multilingual thesaurus (EuroVoc), the UNESCO thesaurus and the Library of Congress (LC) subject authorities vocabulary. A reflective topic can also be linked to pieces of text and multimedia materials, which (although not shown in Figure 1
) can be arranged into different sequences: the reflective narratives.
Narratives are the cornerstone of digital storytelling [13
]. Their use is already widespread in museums in various forms, aiming “to present inclusive and nuanced history, to make big ideas less overwhelming and abstract, and to create frames of experience that encourage deep and satisfying engagement for visitors and online users” [21
]. Notable developments have allowed location-specific narratives to be presented to visitors, through systems that use spatial information, maps and synchronized content [13
]. The design of the CCKB in relation to the modeling of narratives was informed by related work on narrative structures [24
], event ontologies and relations [16
4. Experiments and Results
Next, we shall explain how we have applied the strategies described in Section 3
in the pilot experiments of the CrossCult project, after incorporating in the CCKB the knowledge about the collaborating venues and cities. The experiments vary in scope and methods, even if they share the same technological foundations. We designed them so that we would have the opportunity not only to obtain some quantitative results, but also to closely observe participants and their interactions with the venues and the devices, focusing on identifying traces of reflection, curiosity and other cognitive phenomena. The following subsections touch all the strategies to exploit associations and symmetries, except for the case of connections via trending topics, since we do not have significant results for it yet. (Further information and newer developments will be kept in the sections of official deliverables and publications on the CrossCult web site (www.crosscult.eu
4.1. Associations between Four Venues around Europe
Our first experiment with associations and symmetries via common or opposing features (Section 3.1
) involved four European venues, namely the Roman healing spa of Lugo (Spain), the Roman healing spa of Chaves (Portugal), the archaeological site of Montegrotto Terme (Italy) and the sanctuary of Epidaurus (Greece). Specifically, we created a mobile app to highlight cultural similarities and differences between these sites before, during and after the Roman Empire. The app displayed interactive graphs of associations like the one shown in Figure 9
in order to visualize common and differing aspects of the four venues in relation to selected reflective topics: “when and where”, “human settlements and water”, “health and cult”, “pilgrimage and communication routes” and “material culture”. The graphs were created by experts with the aid of the association and symmetry discovery algorithms, following the process described in [42
]; the users could examine them along with a connecting story, tailored to the points of view of each venue.
During the second half of 2018, we carried out one experiment with 14 volunteers following a Think Aloud protocol [43
], recorded their interactions with the app and also the audio of a live conversation with the experimenters about the app contents—the experimenters would intervene as little as possible, prompting only to encourage the users to share their thoughts. The analysis of the interactions and the audio transcripts (fully documented in [44
]) showed that the app can stimulate different emotional responses. Specifically, we observed 56 explicitly expressed emotions: 44 were positive (78%), 12 were negative (22%). Positive emotions were triggered mostly by correct answers when interacting with the graph, discovery of new content and reading the stories. Negative emotions, in turn, reflected usability issues and frustration during game play with incorrect answers. These reactions play a role in the intellectual journey of reflection and provide insight into the users’ satisfaction with the application.
4.2. Computing Personalized Selections of Paintings in a Big Museum
Taking a different approach on the strategy of Section 3.1
, we assessed the value of the rich semantic annotations of the CCKB (including links to DBpedia and other linked data resources) in a pilot experiment that took place in the National Gallery, London, aiming to illustrate the connections between paintings, artists, places and events across European history.
In this case, we created a mobile app that allows the users to indicate thematic preferences in art, via a quick profiling interface where users could like (swipe right) or dislike (swipe left) paintings from the Gallery’s collection. From the information so gathered, the app provided personalized itineraries through the venue, where users could see pictures that have some similarities, and thus symmetries, with the liked ones. Thus, the users could reflect on the information presented and the diverse works in the collection based on their own knowledge, choices and experience, rather than being forced along a single choreographed route or a random route. Figure 10
shows snapshots of the mobile app, displaying the locations of sets of pictures in the Gallery (e.g., connected by the appearance of costumes or imagery related to death) plus corresponding snippets of information.
For the evaluation, we recruited a sample of 35 users from among undergraduate and master’s students with an interest in history and art, and we equipped them with pairs of eye-camera glasses in order to record the use of the app. From among the results—again, fully documented in [44
]—we can highlight the fact that 80% of the users slightly/strongly agreed with the statement that “each painting in the recommended group was according to my visit objectives”, whereas 69% of them slightly/strongly agreed with “I think I reached my visit objectives following the app recommendations”, even if the recommendation logic intentionally left aside the most popular masterpieces.
We sought deeper insight into reflection phenomena by following the Remind protocol [45
], i.e., by making the users re-live the experience of using the app thanks to the recordings made by the glasses. In this regard, we observed that the results of the personalization algorithms supported the participants in a process of contemplation about their choices during the profiling process, and encouraged them to think about the art collections from a different viewpoint. We took the results as evidence that symmetries inferred from quick profiling can be used to deliver explicit/implicit narratives linked to selections of cultural heritage items from large collections.
4.3. Enriching the Visits to a Small Museum
In order to assess the strategy of finding symmetries via universal themes (Section 3.2
), we ran another pilot experiment at the Archaeological Museum of Tripoli, in Greece. This is a largely unknown venue with a small but valuable collection of items from antiquity, placed in different rooms according to criteria that are not apparent for the visitor. Each item in the museum is accompanied by a short information card that includes shallow and unconnected pieces of text, written on metal plaques next to each exhibit.
The experiment consisted in the creation of seven sets of narratives (in two languages) that connected the exhibits through different themes, exploring symmetries between the past represented by Ancient Greece and the present reality of Western society. We created another mobile app (see the snapshot of Figure 11
), that examined the user’s preferences to sort out those narratives according to an estimated level of appeal. The user can nonetheless select any of the narratives, and following that he/she can browse the contents, check the location of each exhibit on a map of the venue, and provide his/her own reflections. Often, the successive parts of the story finish with a question, for instance “was this older woman dressed like this only because she was married or did it also have something to do with her social status?”. These questions were asked to the user as a starting point for triggering reflection, seeking to stimulate an emotional response by comparing ancient social behaviours with modern ones.
This pilot was evaluated also by means of a Think Aloud protocol, during October 2018, with a sample of seven volunteer users recruited from among museum visitors. Their ages ranged between 40 and 76, and none of them had a background in history or archaeology. They were instructed to move freely inside the museum, access any feature on the app they liked and try to behave as they would do outside an experimental setting. Since the app structures content per exhibit, the participants were asked to tell us their thoughts after they completed the interaction with each exhibit, expressing any thoughts that might come to their minds. The user comments were audio recorded, and transcribed for subsequent analysis.
The report of [44
] shows that the users went through an intellectual journey of different characteristics, based on their past experiences, their personalities and the knowledge gained from the app. The following are just three comments (out of more than three hundred) containing traces of reflection, re-interpretation, meta-understanding and questioning of stereotypes, etc.,
I did not understand why she is nicely dressed and he is not. Oh, it must be because she never went outside her home!
Aphrodite symbolises beauty, why is she the only one to be naked … was this older woman dressed like this only because she was married or did it also have something to do with her social status?
This is closer to the heroes of the last two decades than the heroes of the popular media up until 2000 at least. Heroes nowadays, starting with Nolan’s Batman series, have brought this more ancient possibility… like Hercules, he was not all that great! This is in absolute contrast with American films of the 1950s, 1960s and 1970s, where heroes are all-great people.
… that’s why the ancient gods and heroes are so special to me, so I believe in them more than I believe in a perfect god.
Overall, the stories and questions allowed the users to articulate and express their opinions and feelings, with extensive observations of shame, sadness, fear as well as happiness, pride and satisfaction. Furthermore, besides enjoying a guidance that was not available before, the users realized that it could be worth visiting the museum more than once, since a new visit would take them through different narratives.
4.4. Tagging Personalized Connections to the Current Date
During the first months of 2018, we made one experiment with calendar- and location-based associations using the knowledge available in DBpedia as well as in the YAGO ontology (YAGO (https://github.com/yago-naga/yago3
) is a large semantic knowledge base, derived from Wikipedia, WordNet, WikiData, GeoNames, and other data sources.), with the aid of 30 volunteers recruited from among researchers and students of the University of Vigo (Spain) and Universidad Politécnica Salesiana (Ecuador). We created a very simple mobile app that would raise up to three notifications a day saying “Did you know that today is
?” or “Did you know that this place was
?”. Each notification also referred the user to a selected Wikipedia article. For the selection of connections and articles, the app relied on the initialization of a user profile by means of a questionnaire about historical interests and cultural preferences, and the estimation of the relevance of any association/symmetry to every user was computed as indicated in [37
We asked the users to rate the daily highlights by using the quick feedback interface shown in Figure 12
. Table 1
summarizes the ratings we gathered, comparing the personalized approach against random selections of articles from Wikipedia’s front page. The figures do not add up to 100% because it was not mandatory to provide any rating. The results show that exploiting any symmetries with the users’ profiles leads to greater interest, greater reflection, greater novelty and less indifference.
Following this experiment, we made a second one using DBpedia Spotlight and some scripts of our own to automatically feed the CCKB with thousands of annotations excerpted from the online almanacs of On This Day and National Today. In this case, we asked a group of four humanities experts and 81 regular users to tag the associations and symmetries discovered between relevant dates and the narratives elaborated for the Archaeological Museum of Tripolis (Section 4.3
). The participants were of a broad age range, and potentially interested in visiting the venue.
The connections were automatically formulated in a way that presented the date, museum item, reflective narrative and associated events. To begin with, the participants were asked to assign any of the following tags (possibly none, possibly several) to the findings and the linked narratives:
Informative: the connection/narrative provides new knowledge.
Thought-provoking: the connection/narrative makes me reflect on the association itself.
Memorable: the connection/narrative is likely to be remembered.
Curious: the connections triggers curiosity to make the narrative attractive.
Funny: the connection/narrative can be perceived in a humorous way.
In addition, they had to choose one of the following mutually-exclusive tags for each association:
Notable: the connection is close to museum items or its narrative.
Indirect: the connection has several degrees of separation to the museum items.
Irrelevant: the connection is either purely circumstantial, uninteresting, misleading (because of incorrect interpretation of the meaning of a term) or unclear.
The final tag counts are shown in Table 2
. The presence of the “memorable”, “thought-provoking” and “curious” tags reinforces the intended value of our approach in terms of promoting curiosity, retention, reflection and, in the end, understanding. The number of “informative” tags provides a measure of interest, whereas the number of connections tagged as “funny” was a secondary aspect. Notably, the regular users were more prone to use such tags than the experts, owing to their presumably shallower knowledge about the topics discussed in the narratives of the museum. We view this as a positive finding, because the app is targeted at general museum visitors. Likewise, it was positive that only a small percentage of the proposed connections were deemed irrelevant (again, the experts were more demanding). Among the relevant ones, the predominance of “indirect” connections, compared to “notable” ones, can be seen as a case of promoting serendipity.
4.5. Crowdsourcing in Urban Exploration
The last experiment we are presenting led us to create a mobile app to share and gather stories from the users as they walk around a city. Figure 13
shows some snapshots of the app, which adopts principles of gamification to add playfulness and encourage motivation, and is coupled to a web front end that can be used to moderate the users’ input, perform entity recognition, add new points of interest (POIs), etc.
The app challenged users to reconsider (even if only for a short time) the local environment and the everyday lived experience. The POIs provided navigational clues that directed them to new locations where new stories (official or user-provided ones) could be activated. Then, the users could rate their perception of the stories, or answer expert-curated questions. They could also provide new stories at any time and location as long as they were outside. Comments on questions were not made publicly available until they had been moderated via the aforementioned front end. Users earned points for all the interactions and discoveries they made and a personal avatar won items as they proceed through the different phases of the game. Stories were connected together through the CCKB so players could choose to follow stories about particular topics.
In order to test the app, we recruited 35 volunteers in the campus of Belval, in Luxembourg. Two thirds were aged between 20 and 34 and almost a quarter were aged between 35 and 54, with the remaining being 55 and over. There was an even distribution between Luxembourg natives and non-natives, in line with population statistics for the country. We analyzed their interactions with the app and their impressions by using a combination of methods: questionnaires, post evaluation debriefs, classification of open-ended questions, Think Aloud and Remind. The complete report of the experiment can be found in [44
In terms of individual reflection, we could confirm that location-based stories can promote an openness to express personal experiences, together with an articulation of the users’ sense of comfort/uncomfortableness with the places. In terms of social reflection, the publication of the stories and comments provided by the users resulted in a feeling of connectedness to the POIs. This allowed users to reflect on the experiences of others and stimulated the identification of commonalities and differences, as advocated the seminal works of Bandura [46
] and Zeichner and Liston [47
]. From a quantitative perspective, we classified 70% of the reflection types that were evidenced in our experiment transcripts as users either discovering information, sharing personal memories and feelings or connecting to their prior knowledge. In the process, the users created valuable knowledge in the shape of new associations in the CCKB and new stories for the city, too. Whilst these stories were by no means long as the expert-created ones, they were further evidence that the process of content creation triggers a deeper thoughtful reflection.
4.6. Overview of the Experiments
With cohort sizes of just a few tens—and not always fully representative of the populations targeted by the different apps and venues—all the counts and percentages given in the preceding sections must be interpreted with due caution, as more tests are needed in order to draw statistically significant results. However, on the qualitative side, it was precisely the reduced samples what allowed us to focus on analyses that provided detailed information about the users’ intellectual and cognitive journeys. Thus, we believe that the observed traces of reflection, interpretation, curiosity, serendipity and other phenomena confirm that the many types of associations and symmetries advocated in this paper can be meaningful per se. Still, it became clear that the positive effects are maximized if there is some sort of supervision or guidance by experts, who curate the contents and narratives beforehand or assist during the museum visits.
The CrossCult project has faced the questions of how to facilitate the creation of narratives so as to grasp the large number of symmetries between historical events/characters or heritage items and the interests, preferences and contexts of the target audiences. We see this as one way to overcome the predominant narratives implicitly enforced by the static content of textbooks, by the unique arrangement of exhibits into different rooms of a cultural venue (according to certain classification criteria: chronology, authorship, painting style, …), by the navigational structure of a cultural web site, by the editorial bias of mainstream media, or by the fixed layout of the tangible heritage spread around a city.
To our understanding, the work we have done in CrossCult has revealed some of the many possibilities of artificial intelligence to support more effective storytelling in digital humanities. The experience gathered through the different pilots and experiments has provided evidence that diverse AI techniques—from semantic web and graph theory, to entity recognition and paragraph vectors—can provide significant aids to uncover interesting associations and symmetries between digitized resources, user profiles and any features of the current context. Those connections may be used directly to draw the attention of mobile technology users to selected pieces of content, either from the repositories of a given museum or from publicly-available sources such as Wikipedia or Europeana. Alternatively, they could become the seed for humanities experts to create more focused narratives, with precise, reliable and targeted content. Actually, even if the connections are strong, cultural heritage apps may be only as good as the content they serve and the lasting messages they deliver. This fact highlights the importance of embedding AI inside proper authoring tools, with features to support collaboration, integration of venue archives and Linked Data resources, collaborative creation of plots for storytelling, association/symmetry visualization and curation, etc. The tools should seek involvement from experts while trying to make the most of their time and effort, and also resort to the wisdom of the crowds, following suits on pioneering experiences of what is known as crowd computing or human computation in the cultural heritage area [48
]. We believe this is a niche market that still needs to be developed, with plenty of space for innovation and notable market potential.
All in all, we believe that having new, unconventional narratives that focus on intercultural and cross-border aspects can make a difference in the appraisal of history and cultural heritage as a shared experience, as we have repeatedly found—in different international settings, with different app concepts and with diverse target groups—that the historical and cultural symmetries can support curiosity, reflection and understanding.