2.1. Short Discussion about the Cultural Heritage Digitization Procedures
These procedures have now a long tradition of work that goes back to the sixties. They were limited, initially, to some elementary descriptions of the Cultural Heritage items that could be likened, in practice, to standard annotations created making use of keywords (metadata) extracted from thesauri as the Art and Architecture Thesaurus (AAT,
http://www.getty.edu/research/tools/vocabularies/aat/index.html), ICONCLASS (
http://www.iconclass.nl) or the Union List of Artist Names, ULAN (
http://www.getty.edu/research/tools/vocabularies/ulan/index.html). They were typically focused, then, on the utilization—mainly for documentary/information retrieval purposes—of the “physical” aspects of the Cultural Heritage Digital Twins introduced above. The use of (relatively simple) data models very popular in the eighties-nineties as the original Dublin Core (
http://dublincore.org/) proposal—defined as “pidgin metadata” in [
5]—or of more sophisticated models like VRA (Visual Resources Association) Core 4 XML Schema (
https://www.loc.gov/standards/vracore/schemas.html) has contributed to improve the efficacy and generality of these annotation procedures. The iconographic items descriptions obtained thanks to this kind of processes correspond then with reference, e.g., to the VRA model—to the use of unstructured sequences of binary entities of the property-value type (like “name-Peter Paul Rubens” or “culture-Flemish”) linked to simple XML attributes. The conversion of several of these models into RDF-compatible tools under the influence of the Semantic Web initiative—see the DCMI (Dublin Core Metadata Initiative) Abstract Model [
6] or the RDF(S) version of the CIDOC CRM tool (
http://www.cidoc-crm.org/rdfs/5.0.4/cidoc-crm)—has further increased their interoperability and standardization potential. CIDOC CRM (Conceptual Reference Model)—see [
7] for the last version (6.2.6)—is a well-known, powerful and outstanding tool that aims at providing definitions and a formal, ontological-oriented structure for describing the main implicit and explicit concepts (99 in the last version) and relationships (191) used in the Cultural Heritage documentation. Since 9/12/2006, it is the official standard ISO 21127:2006.
If the adoption of a Semantic Web (RDF) perspective has indubitably produced important beneficial effects with respect to the digital encoding of the “physical” properties of the iconographic items, it does not seem to have introduced real progresses in the search for concrete solutions for representing the second, “immaterial” size of the Cultural Heritage Digital Twins. A canonical and well-known example in this context is represented by the RDF-based description of Claude Monet’s Garden at Sainte-Addressee painting included in the Image Annotation on the Semantic Web W3C Incubator Group Report [
8]. This description includes an impressive amount of documentary/bibliographic/descriptive/ “physical” details. However, the description of the proper “content/deep meaning” of the painting and of the “message” it could transmit to the observer is reduced to three extremely sketchy statements—e.g., <vra:subject>Adolphe Monet (artist’s father)</vra:subject>—relating the presence on the picture of three persons (three Claude Monet’s relatives). No information is given about the real semantic content of the painting, for example, the mode of sitting of the personages in front of the see, their mutual relationships, their attitude about the peaceful and bright landscape, the impressive number of flowers, etc. The Incubator Group Report goes back to 2007, and we could suppose that the state of the art about the formal description of iconographic items described in this report is now largely outdated. This is not always the case. We can see for example, in the context of the very complex and convoluted formalization of the Mona Lisa painting carried out in 2013 in an Europeana context [
9] making use of the RDF-oriented European Data Model (EDM), that the description of the semantic content of this painting seems to be several times reduced to flat statements in the style of “the subject of the painting is a woman” [
9], p. 19.
A number of proposals concerning the implementation of specialised systems for the description/management of iconographic items that are based on extensions of CIDOC CRM have been advanced these last years. In [
10] for example, the authors suggest to expand CIDOC CRM making use of the Situations&Descriptions (S&D) module of the DOLCE Semantic Web tool. In this context, CIDOC CRM should be used to represent the properties and the basic relationships of these items, while the S&D module should supply an in-depth formal description of their “informative content”; the iconographic domain of application concerns some temple scenes and reliefs pertaining to the Meroitic civilization in Sudan. From the paper, it appears clearly through some concrete examples how CIDOC CRM, in association with other ontologies, could be employed to formalise the relationships between temples, scenes and reliefs, highlighting in particular some spatial relationships. On the contrary, the contribution of S&D to the formalization of the informative content is not concretely illustrated in the paper and seems, therefore, to belong more to the domain of intent than to that of tangible results—even if the authors mention, in the “Conclusion”, the existence of some undefined “successful tests”. Another recent proposal of extension of CIDOC CRM concerns the implementation of an ontological tool called VIR (Visual Representation) [
11]—see also
https://ncarboni.github.io/vir/. VIR expands the key entities and properties of CIDOC CRM by introducing seven new classes (e.g., “Iconographical Atom”) and 20 new relationships (e.g., “portray/is portrayed”) corresponding to the needs of the visual and art historical community. According to the authors, the resulting model is able to provide a clear distinction between “denotation” and “signification” of a Cultural Heritage element, permitting, in particular, the definition of diverse denotative criteria for the same representation. Taking as an example the representation of a widely known iconographical character, Saint George, the VIR-supported ontological/graphical representations allow us to follow the evolution in time of the set of attributes illustrating this character. These are simply reduced to “horse” and “spear” in the frescos of the Panagia Phorbiotissa church in Cyprus, to which are added “Castle, Princess, Lake and Dragon” when the “Saint George killing the dragon” painting by Vittore Carpaccio is taken into consideration. This work is of course interesting, but it contributes scarcely to the representation of the “deep meaning” of the iconographic items examined. For example, the modelling of the Carpaccio’s picture is reduced to a simple “binary” representation where an individual, Saint George, is denoted by a set of static properties, horse, spear, castle etc., and the highly complex, dramatic and dynamic illustration of the fight with the dragon is totally ignored.
We can conclude this short discussion about the present state of the art with respect to the modelling of the “inner meaning” of Cultural Items by mentioning a recent paper [
12], which concerns the project Cultural-ON (Cultural Ontology) resulting from a collaboration between the Italian Ministry for Cultural Heritage and the Italian National Council for Research. The project has been recently restructured (see
http://dati.beniculturali.it/cultural_on/to) to adapt it to other “ontological” initiatives of the Ministry, but its basic approach did not change. In this paper we find, e.g., the formal description of a Giambologna’s sculpture denoting the kidnapping of a Sabine woman by a Roman warrior, on display at the Capodimonte museum in Naples in the framework of the so-called “Collezione Farnese”. The description of this sculpture is restricted to purely documentary/information retrieval-oriented statements like cis: isMemberOf collection: Collezione_Farnese or cis: isInSite: Sede_del_Museo_di_Capodimonte, without any attempt to model the frightened woman, the kidnapper, the man who tries to prevent the kidnapping, the reciprocal attitudes of the different protagonists of the scene, etc.
Note, eventually, that finding a solution to the above difficulties is not facilitated by the choice of systematically choosing RDF as implementation support, because of the well-known “lack of expressiveness” problem that bothers all the Semantic Web (SW) tools from the beginning. This problem is linked to the choice of making use only of the quite limited “binary” knowledge representation model. In this, in fact, a concept is simply defined through a set of properties/attributes; when the concept is instantiated into a concrete individual, each associated property can only link this individual to another individual or a value, individual1-property-individual2/value. Such an approach renders then particularly difficult the setting up of complete and effective formal descriptions of real-world, complex information structures like spatio-temporal data, contexts, reified situations, human intentions and behaviours, etc. The evident solution to this problem is that of making use, instead of binary models, of n-ary ones where a given predicate can be associated with multiple arguments—for example, the n-ary “purchase” relation/property concerns events where at least a seller, a buyer, a good, a price, and a timestamp are involved. Unfortunately, it is not so evident that an n-ary solution should be used, given the associated implementation difficulties. To give only an example, a common misunderstanding consists in asserting that the use of specific n-ary knowledge representation structures is not necessary given that any n-ary relationship can be simply reduced to a set of binary relationships. This is in principle true, and this sort of decomposition can be also useful for, e.g., dealing with very practical problems like storing efficiently n-ary relationships into standard databases. However, binary and n-ary relationships are conceptually irreconcilable, and an n-ary relation cannot be reduced to the simple addition of binary elements without losing its “deep meaning”—it is impossible to reason about, e.g., the possible reasons and consequences, the context, etc., of a purchase without considering the purchase event in its whole conceptual entirety, i.e., by taking all its arguments simultaneously into account.
2.2. Using NKRL in the Context of the Cultural Heritage Domain
NKRL is both a “structured n-ary” (see below) knowledge representation language and a fully implemented [
4], computer science environment built up thanks to several European projects. It has been successfully used in many different narrative-based domains like terrorism news, analysis of industrial incidents, sentiment analysis, conceptual IoT, etc.
The most important NKRL features are highlighted below.
From an ontological point of view, the most striking characteristic of the language concerns the addition of an “ontology of elementary events” to the usual ontology of concepts. In NKRL this last—called “HClass”, hierarchy of classes—presents some interesting aspects, with respect in particular to the modelling of difficult notions like colour and substance [
4], pp. 123–137. However, its architecture is a traditional one, and the HClass concepts are represented according to the usual “binary” model.
The nodes of the ontology of events are represented, on the contrary, by n-ary knowledge patterns—called “templates” in an NKRL context—that denote formally general classes of elementary events/states/situations/actions/episodes/experiences. Examples of these general classes can “be present in a place”, “move a physical object”, “have a specific attitude towards someone/something”, “send/receive messages”, “be characterized by a given property”, etc. The instances of these templates—called “predicative occurrences” in an NKRL context—describe then formally the “meaning/inner content" of specific elementary events/states/situations, etc., pertaining to one of these classes.
A “conceptual predicate” denotes the main structuring element of a specific template or predicative occurrence; it defines the general semantic class to which the template/occurrence pertains. Seven conceptual predicates are used in NKRL: BEHAVE, EXIST, EXPERIENCE, MOVE, OWN, PRODUCE, RECEIVE; see [
4], pp. 57–59 for the reasons of this reduced choice.
The different entities (human being, physical objects, but also properties, locations, events, situations etc.) involved in the original elementary events correspond, within templates/occurrences, to the n-ary “arguments of the predicate”. These arguments are created by using concepts or instances of concepts (individuals) proper to the HClass hierarchy; the two NKRL ontologies are then formally/functionally different, but strictly integrated from an operational point of view.
“Structured arguments (expansions)” under the form of lists of HClass terms can be created making use of the four operators of the so-called AECS sub-language; see [
4], pp. 68–70. AECS includes the disjunctive operator ALTERN(ative) = A, the distributive operator ENUM(eration) = E, the collective operator COORD(ination) = C and the attributive operator SPECIF(ication)—their interweaving within a structured argument is controlled by a “priority rule” that forbids, e.g., to use a list of the ALTERN type within the scope of a list COORD.
To specify exactly the “function/role” of each argument in the context of a specific template/occurrence, these arguments are introduced by the following functional roles: SUBJ(ect), OBJ(ect), SOURCE, BEN(e)F(iciary), MODAL(ity), TOPIC, CONTEXT [
13]. They individuate then, e.g., the main protagonist (SUBJ(ect)) of an elementary event, the beneficiary (BEN(e)F(iciary)) or the origin (SOURCE) of a given action, etc.
The whole n-ary expression, template or occurrence, is then unequivocally identified making use of a “symbolic label” that, through special reification mechanisms, allows this template/occurrence to be associated to other templates/occurrences in order to allow, in the end, the representation of complex events/scripts/scenarios, etc. These reification mechanisms are implemented according to two main techniques, the “binding occurrences” (see
Table 1) and the “completive construction” (see the NKRL clause denoted as gio3.c13 in Table 3).
“Determiners” can be added to templates or predicative occurrences to introduce further details about the basic core of their formal representation, see Equation (1) below. For example, determiners of the “location” type can be associated through the colon operator, “:”, with the arguments of the predicate introduced by the SUBJ, OBJ, SOURCE and BENF functional roles. Another important category of determiners associated, in this case, to a full template or predicative occurrence concerns some constants called “modulators”, for example, the temporal modulators (begin, end, obs(erve)). These last, associated with the determiners date-1 and date-2, are of a particular importance in the framework of the temporal representation system of NKRL; a recent paper on this subject is [
14].
Eventually, “querying/reasoning” in NKRL ranges from the direct questioning of NKRL knowledge bases making use of “search patterns” (NKRL equivalents of natural language queries) to high-level inference procedures [
15]. In this last context, e.g., the “transformation rules” try to “adapt” the original query/queries (search patterns) that failed to the real contents of the KB. This means using rules to automatically “transform” the original query into one or more different queries that are not strictly “equivalent” but only “semantically close” to the original one making use of a sort of “analogical” reasoning. “Hypothesis rules” allow us to build up realistic answers according to a number of reasoning schemata, e.g., “causal” ones.
Equation (1) denotes the “core” of the formal representation of an NKRL template/predicative occurrence:
In Equation (1), Li is the symbolic label identifying (reifying) the whole template/occurrence, Pj is the conceptual predicate, Rk is a generic functional role and ak is a (simple or structured) predicate argument introduced by the role Rk.