You are currently viewing a new version of our website. To view the old version click .
Future Internet
  • Article
  • Open Access

28 February 2022

Graphol: A Graphical Language for Ontology Modeling Equivalent to OWL 2

,
,
and
1
Department of Computer, Control and Management Engineering, Sapienza Università di Roma, Via Ariosto 25, 00185 Roma, Italy
2
OBDA Systems S.R.L., Via di Casal Boccone, 00137 Roma, Italy
3
Department of Management, Information and Production Engineering (DIGIP), Università degli Studi di Bergamo, Via A. Einstein 2, 24044 Dalmine, Italy
*
Authors to whom correspondence should be addressed.
This article belongs to the Special Issue Modern Trends in Multi-Agent Systems

Abstract

In this paper we study Graphol, a fully graphical language inspired by standard formalisms for conceptual modeling, similar to the UML class diagram and the ER model, but equipped with formal semantics. We formally prove that Graphol is equivalent to OWL 2, i.e., it can capture every OWL 2 ontology and vice versa. We also present some usability studies indicating that Graphol is suitable for quick adoption by conceptual modelers that are familiar with UML and ER. This is further testified by the adoption of Graphol for ontology representation in several industrial projects.

1. Introduction

There is a long tradition in many areas of computer science of conceptualizing domains of interest in terms of classes and relationships using a graphical or diagrammatic model. Consider, for example, ER (entity–relationship) diagrams [], ubiquitously used in databases, or UML class diagrams [], the de facto standard in software engineering for information modeling (when used as conceptual models rather than to represent software components). While often such diagrams are used in a semi-formal way to help communication, it is well-recognized that having precise semantics is actually needed to avoid ambiguities in design.
Interestingly, the very first conceptual languages developed in AI were also graphical, most prominently semantic networks [,]. However, most work on knowledge representation in AI has focused more on automated reasoning, and has gradually abandoned the graphical conceptual languages in favor of logical languages. This process has started with the famous paper “What’s in a link” [], which questioned the inherent ambiguity of graphical conceptual languages of the time, and has continued with the work on KL-ONE [], then followed by the introduction of modern description logics (DLs) []. Nevertheless, by the early 1990s, a research program started to emerge: not to disregard, but to try to logically reconstruct graphical conceptual models used in many fields, such as software development and information systems, in order to enable automated reasoning on them [,,]. This program has actually been one of the thrusts towards more and more expressive DLs [] that ultimately led to the development of OWL and OWL 2 [].
In this paper we bring about a novel contribution to this program: we study a graphical formalism, called Graphol, which resembles ER and UML class diagrams, but has inherent formal semantics based on DLs and is able to fully capture the ontology language OWL 2. Our proposal comes after a few years of experience in ontology modeling in IT organizations that are knowledgeable on information systems and software engineering, so are familiar with UML and ER, but have only a technological view of ontology languages such as OWL 2 (e.g., [,,]). In these contexts, people often struggle to effectively use the logical formalisms through which ontologies are typically specified, thus slowing down the adoption of semantic technologies. Graphol mitigates this problem, since it provides IT people with a formalism for specifying and reading ontologies rooted in conceptual modeling languages they are used to. Indeed, its usage has helped substantially in taking up semantic technologies in the industrial use cases [,,,,,].
We point out that we have also developed tools for drawing Graphol diagrams and translating them into standard OWL 2 ontology format. These tools, however, are not treated in depth in this paper, and we refer the reader to [,] and to the Github repository of the Eddy ontology editor (https://github.com/obdasystems/eddy, accessed on 17 January 2022) for the details. Here, instead, we study Graphol as a language, and focus on its formal properties. Specifically, we give its mathematical semantics, based on DLs, and show that Graphol diagrams can be translated into OWL 2 ontologies and vice versa. Furthermore, we describe a user evaluation study we carried out to verify the usability of our language. A preliminary version of some of the contributions given in this paper can be found in [].

1.1. Introducing the Graphol Language

To obtain an idea of Graphol and its relation with UML and ER, in Figure 1 we model a simple situation about students and courses they attend, using a UML class diagram, a Graphol diagram, and a set of DL axioms, all expressible in OWL 2. We assume that the reader is familiar with UML class diagrams [], DLs [], and OWL 2 []. Furthermore, since we adopt the DL notation throughout the paper to express logical axioms, we use the DL terminology for ontology predicates, e.g., we use “concept” to denote a set of objects (i.e., a “class” in OWL parlance), “role” to denote a binary relationship between concepts (i.e., an “ObjectProperty” in OWL parlance), and “attribute” to denote a binary relationship between a concept and a domain of values (i.e., a “DataProperty” relating objects to datatypes in OWL parlance). In all the three versions of the model, we use the same alphabet for predicates. The model states that a student must attend at least a course, that university students are students, that graduate courses are courses, and that university students can only attend graduate courses. One can see that, in Graphol, concepts are represented through rectangles, analogously to UML, while, as in ER, diamonds are used for roles. Furthermore, solid directed arrows represent inclusions, as in UML and ER (cf. the inclusion between UniversityStudent and Student ). However, differently from UML and ER, in Graphol they do not need to involve only named concepts. The use of a diamond to represent a role, i.e., a “node” in the diagram, allows us indeed to depict, in a simple graphical way, concept expressions over the domain and the range of a role, i.e., over its first or second component, by connecting the role to possibly labeled blank and solid boxes, respectively. For example, in Figure 1, the blank box labeled with “forall” and linked to the role attends denotes the set of individuals that attend only graduate courses. The inclusion drawn between UniversityStudent and this concept expression specifies that a university student can only attend graduate courses, which corresponds to the DL inclusion axiom UniversityStudent attends . GraduateCourse . We remark that this property is not directly expressible in a graphical way in UML, where we need to specify it as an external constraint (cf. the note in the diagram), possibly expressed in a logical language such as OCL.
Figure 1. Introductory example.
We notice that the idea of extending or adapting UML or ER to capture OWL is indeed not new. However, Graphol is distinguished from the other proposals with precise semantics, such as [,,,,] by its ability to capture any OWL 2 ontology in a completely graphical way. Indeed, previous UML-inspired approaches typically require to annotate diagrams with formulas corresponding to complex OWL expressions. Clearly, this hinders both the diagrammatic representation of the ontology and its intuitive understanding. A more in depth discussion on related work is given in Section 2.

1.2. Paper Organization and Contributions

The rest of the paper is organized as follows. As stated, we first provide an overview of related work on graphical languages for ontology design and visualization. Then, we give some preliminaries on DLs, which we use as formal tool for defining the semantics of Graphol constructs and for establishing the correspondence between Graphol and OWL 2, which also has a DL counterpart. Next, in Section 4, we give the formal syntax of the language (Section 4.1), its semantics (Section 4.2), and show its equivalence with OWL 2 (Section 4.4), i.e., we prove that every Graphol diagram corresponds to an OWL 2 ontology, and, conversely, that every OWL 2 ontology corresponds to a Graphol diagram. Then, in Section 5, we discuss the relationship between Graphol and UML, and finally, in Section 6, we provide some user evaluations of our language. Our study shows that Graphol can be adopted by non-expert modelers introducing only a minimal overhead with respect to the use of standard conceptual languages such as UML and ER, and that this overhead pays off when the UML/ER diagrams need logical annotation to fully capture the ontology of interest. More interestingly, the evaluations we carried out show that Graphol can be adopted by expert conceptual modelers with ease, greatly facilitating the adoption of a full-fledged ontology language, such as OWL 2, as a formal conceptual modeling language. We conclude the paper in Section 7.

3. Preliminaries

Description logics (DLs) [] are formalisms used to model a domain of interest in a formal way. They are portions of first-order logic that allow for decidable reasoning. In DLs, the domain of interest is represented in terms of objects (i.e., individuals), concepts (i.e., abstractions for sets of objects), and roles (i.e., binary relationships between concepts). Moreover, some DLs consider also value-domains, denoting sets of values, and attributes, denoting binary relationships between concepts and value-domains.
We assume to have an alphabet Γ partitioned into Γ P and Γ C . The former is in turn partitioned into sets of symbols for atomic concepts, atomic roles, atomic attributes, and atomic value-domains. Γ C instead contains symbols for constants, and is further partitioned into the sets Γ O , which is the set of constants denoting objects, and Γ V , which is the set of values.
We now provide syntax and semantics of the DL language of interest in this work. We notice that the DL we provide in this section captures the DL S R O I Q ( D ) [,], i.e., the DL underlying OWL 2. In Section 4.4, we will discuss which syntactic restrictions to impose to make this language equivalent to S R O I Q ( D ) (and so to OWL 2). Concepts, roles, attributes, and value-domains expressions in such DL are defined by the following rules:
C A ¬ C C C C C R R . C R . C n R . C n R . C R . S e l f V V . F V . F n V . F n V . F { c 1 , , c n } C C F T V ¬ F F F F F D D { w 1 , , w n } R P P ¬ R R R R R V U ¬ V A A
where A denotes an atomic concept, P an atomic role, U an atomic attribute, T an atomic value-domain, C (resp., R , A , D ) the universal concept (resp., role, attribute, value-domain), and C (resp., R , A , D ) the empty concept (resp., role, attribute, value-domain).
We call C, R, V, and F a general concept, role, attribute, and value-domain, respectively. In our treatment, such symbols can be used with subscripts. P denotes the inverse of an atomic role, while ¬ C , ¬ R , ¬ V , ¬ F denote the negation of C, R, V, and F, respectively. The expression R (resp. V ) denotes the domain of a role R (resp. of an attribute V). Instead, the qualified concept existential restriction  R . C indicates the domain of R restricted to the class C, i.e., it is an abstraction for the set of objects that R relates to some instance of C. Similarly, V . F denotes the qualified domain of V with respect to a value-domain F, i.e., the set of objects that V relates to some value in F. The concept R . C , also called value restriction, denotes the set of objects that are associated by R only to objects that are instances of C. Similarly, V . F denotes the set of objects that are associated by V only to values in F. The concept R . S e l f is used to express local reflexivity to a role R. V denotes the range of an attribute V. Similarly, R denotes the range of a role R, which corresponds to the domain of the inverse of R. The set { c 1 , , c n } denotes the concept whose instances are denoted by c 1 , , c n , and, similarly, { w 1 , , w n } denotes the value-domain whose instances are denoted by w 1 , , w n . The symbols ⊓ and ⊔ are the usual A N D and O R logical connectives. n and n indicate number restrictions, respectively, at-most restriction and at-least restriction, where n ranges over the nonnegative integers. Finally, R R denotes a role chain.
The semantics of a DL KB is given in terms of interpretations. An interpretation I = ( Δ I , · I ) consists of:
-
A nonempty interpretation domain Δ I = Δ O I Γ V , where Δ O I is the domain of objects, and Γ V is the set of values previously introduced (indeed, in every interpretation each value is intepreted by itself);
-
An interpretation function · I that assigns an element of Δ O I to each constant in Γ O , and interprets each DL expression as shown in Table 1.
Table 1. The DL constructs with their semantics.
In the table, c 1 I , c n I denote the interpretation of the constants c 1 , c n from the alphabet Γ O .
A DL KB K is a pair T , A , where T is a finite set of intensional assertions, called TBox, and A is a finite set of extensional assertions, called ABox. In this paper we focus on the modeling of intensional knowledge, i.e., the TBox, so we will not detail further the form of the ABox (which is the component of the knowledge base maintaining the data).
The TBox assertions we focus on in this work are as follows:
C 1 C 2 ( c o n c e p t i n c l u s i o n ) ; R 1 R 1 ( r o l e i n c l u s i o n ) ; V 1 V 2 ( a t t r i b u t e i n c l u s i o n ) ; F 1 F 2 ( v a l u e - d o m a i n i n c l u s i o n ) .
Concept inclusion assertions state that all instances of one concept are also instances of another concept, analogously for role, attribute, and value-domain inclusion assertions.
An interpretation I satisfies a TBox T if it satisfies all inclusions in T , where the notion of satisfaction of an inclusion is as follows:
C 1 C 2 if C 1 I C 2 I ; R 1 R 2 if R 1 I R 2 I ; V 1 V 2 if V 1 I V 1 I ; F 1 F 2 if F 1 I F 2 I .

4. The Graphol Language

In this section we introduce the graphical elements and features of the Graphol language for ontologies. We will use the popular Pizza (http://protegewiki.stanford.edu/wiki/Pr4_UG_ex_Pizza, accessed on 17 January 2022) ontology as a running example to illustrate how different expressions and assertions of an ontology are represented in Graphol, and will use the DL syntax for logical symbols presented in Section 3 to define the correspondence between Graphol shapes and their logical meaning.

4.1. Graphol Syntax

The basic principle that guides Graphol in representing ontologies is that a Graphol ontology is a graph whose nodes and edges assume the forms described in Figure 2.
Figure 2. Nodes and edges in a Graphol ontology.
Nodes can be of two kinds: predicate nodes and operator nodes. Predicate nodes model the terms in the ontology alphabet, i.e., atomic concepts, rendered as rectangles, atomic roles, which are depicted as diamonds, atomic attributes, drawn as circles, atomic value-domains, represented as rounded rectangles, and constants (i.e., individuals and values), denoted through octagons. Each predicate node is associated to a label, which is a name from the ontology alphabet (to distinguish between individual nodes and value nodes, the labels of value nodes are in inverted commas). We note that the shapes used to depict concepts, roles, and attributes are the same as those used in ER diagrams. This choice was made to take advantage of the familiarity that many of the potential users of Graphol may have with this well-known language for conceptual modeling.
The special labels, “Top” and “Bottom”, are reserved for representing universal and empty predicates (i.e, concept, role, attribute, or value-domain), respectively. For example, a diamond with label “Top” represents the universal role ( R ), while a diamond with label “Bottom” represents the empty role ( R ), analogously for concepts, attributes, and value-domains.
Operator nodes are instead used to graphically construct complex expressions. Two shapes for operators are adopted, i.e., the box (which can be either blank or solid), and the hexagon. A blank (resp. solid) box, called domain (resp. range) restriction node, is used to represent restrictions on roles or attributes (resp. their inverses). A box is labeled with one of the following keywords: “exists”, for existential restriction, “forall”, for universal restriction, “self”, for self-restriction, and “ ( x , ) ” or “ ( , y ) ”, with x and y positive integers, for min and max cardinality restrictions, respectively. When the label is omitted we intend “exists”. The other operator nodes are denoted by a hexagon and can assume one of the following labels: “or” (union node), “and” (intersection node), “not” (complement node), “inv” (inverse node), “oneOf” (one-of node), and “chain” (chain node).
We observe that Graphol uses three visual variables [] to encode the predicate and constructor nodes. These are (i) the shape of the nodes, which is used to distinguish between predicate nodes, restriction nodes, and operator nodes, and among the different kinds of predicate nodes; (ii) the size of the nodes, which allows to clearly discriminate between the quadrilaterals used for concept nodes, value-domain nodes, and restriction nodes; and (iii) brightness, which is used to distinguish the domain restriction node from the range restriction node.
To avoid encumbering the user in learning the Graphol syntax, we have chosen to limit the number of different graphical symbols used to depict Graphol nodes, and to maintain it to around seven, which is the commonly recognized ideal upper bound for software engineering graphical languages []. Indeed, experimental studies demonstrate that a high number of different symbols in a language for software engineering increases the learning difficulty by non-expert users [].
Graphol provides two types of edges: inclusion edges, which are solid directed arrow edges (whose target end is denoted by the arrow), used to represent inclusion assertions, and the input edges, which are dashed directed diamond edges (whose target end is denoted by the diamond), used to construct ontology expressions.
An expression in Graphol is a directed acyclic graph, where the nodes can be both operator and predicate nodes and the edges are input edges only. In every Graphol expression there is a single node without outgoing edges, called sink. We give below a formal definition:
Definition 1.
AGrapholexpression can be of four types: concept, role, attribute, or value-domain, defined inductively as follows:
1
A concept expression can be:
-
A concept node (in this case the sink is the node itself);
-
A domain or range restriction node, with label “exists”, “forall”, “ ( x , ) ”, or “ ( , y ) ”, taking as input a role expression and a concept expression (in this case the sink is the domain or range restriction node);
-
A domain or range restriction node, with label “self”, taking as input a role expression (in this case, the sink is the domain or range restriction node);
-
A domain restriction node, with label “exists”, “forall”, “ ( x , ) ”, or “ ( , y ) ”, taking as input an attribute expression and a value-domain expression (in this case, the sink is the domain restriction node);
-
A union or intersection node taking as input at least two concept expressions (in this case, the sink is the union or intersection node);
-
A complement node taking as input a concept expression (in this case, the sink is the complement node);
-
A one-of node taking as input at least an individual node (in this case, the sink is the one-of node).
2
A role expression can be:
-
A role node (in this case, the sink is the node itself);
-
An inverse node taking as input a role expression (in this case, the sink is the inverse node);
-
A complement node taking as input a role expression (in this case, the sink is the complement node);
-
A chain node taking as input n role expressions, with n 2 , each associated to a label 1 i n and such that there are no two input edges with the same label (in this case, the sink is the chain node).
3
An attribute expression can be:
-
An attribute node (in this case, the sink is the node itself);
-
A complement node taking as input an attribute expression (in this case, the sink is the complement node).
4
A value-domain expression can be:
-
A value-domain node (in this case, the sink is the node itself);
-
A range restriction node with label “exists” taking as input an attribute expression (in this case, the sink is the range restriction node);
-
A union or intersection node taking as input at least two value-domain expressions (in this case, the sink is the union or intersection node);
-
A complement node taking as input a value-domain expression (in this case, the sink is the complement node);
-
A one-off node taking as input at least a value node (in this case, the sink is the one-off node).
Intensional assertions in Graphol, as well as in OWL 2 and DLs, are specified as inclusions. Thus, an (inclusion) assertion in Graphol is specified via an inclusion edge from the (sink of the) expression that is included to the (sink of the) expression that it includes. For instance, a concept inclusion assertion between the two concepts C 1 and C 2 is obtained by linking through an inclusion edge the sink of the Graphol expression of C 1 to the sink of the Graphol expression of C 2 . Of course, Graphol does not allow to specify inclusion edges between expressions of different types (e.g., a role expression with a concept one).
We finally define a Graphol ontology as a set of Graphol inclusion assertions.
To simplify ontology design and to keep the diagram easier to read, we allow in a Graphol ontology to have multiple occurrences of the same predicate, obviously all labeled with the same label (which is the name of the predicate). This is particularly useful in those cases in which a predicate occurs in many assertions of the ontology, and so representing such predicate with a single node would lead to having a plethora of incoming or outgoing edges from that node, likely leading to layout issues. We notice that the tool Eddy [], which offers an environment for the graphical specification of Graphol ontologies, provides functionalities for the refactoring of the ontology in case a predicate node is modified, e.g., changes on the predicate node label are automatically propagated to all the replicas of such predicate in the ontology.

4.2. Graphol Semantics

An important desiderata of visual modeling languages is to have clear and precise semantics [,]. To this aim, here we provide the semantics of Graphol expressions and assertions by giving their one-to-one correspondence with DL expressions and assertions, which in turn have formal semantics, as discussed in Section 3.
We start with Graphol expressions, and define a function Λ which takes as input a Graphol expression E G and computes a DL expression that encodes it.
To formalize Λ , we use sk ( E G ) to denote the sink of E G , and ar ( E G ) the set of Graphol expressions linked to sk ( E G ) through input edges (of course, this set can be empty). We formally define Λ as follows:
-
If sk ( E G ) is a concept, role, attribute, value-domain, or individual/value node with label S, then Λ ( E G ) = S (we are considering concept, role, attribute, value-domain, individual, and value alphabets as pairwise disjoint. Moreover, if S = “Top”, we assume that Λ returns the corresponding DL universal predicate, which depends on the form of sk ( E G ) . Analogously if S = “Bottom”);
-
If sk ( E G ) is a domain restriction node with label “exists” (resp., “forall”, “ ( x , ) ”, “ ( , y ) ”), and ar ( E G ) = { ϵ R A , ϵ C V } , where either ϵ R A is a Graphol role expression and ϵ C V is a Graphol concept expression or ϵ R A is a Graphol attribute expression and ϵ C V is a Graphol value-domain expression, then Λ ( E G ) = Λ ( ϵ R A ) . Λ ( ϵ C V ) (resp., Λ ( E G ) = Λ ( ϵ R A ) . Λ ( ϵ C V ) , Λ ( E G ) = x Λ ( ϵ R A ) . Λ ( ϵ C V ) , Λ ( E G ) = y Λ ( ϵ R A ) . Λ ( ϵ C V ) );
-
if sk ( E G ) is a range restriction node with label “exists” (resp. “forall”, “ ( x , ) ”, “ ( , y ) ”), and ar ( E G ) = { ϵ R , ϵ C } , where ϵ R is a Graphol role expression and ϵ C is a Graphol concept expression, then Λ ( E G ) = ( Λ ( ϵ R ) ) . Λ ( ϵ C ) (resp. Λ ( E G ) = ( Λ ( ϵ R ) ) . Λ ( ϵ C ) , Λ ( E G ) = x ( Λ ( ϵ R ) ) . Λ ( ϵ C ) , Λ ( E G ) = y ( Λ ( ϵ R ) ) . Λ ( ϵ C ) );
-
If sk ( E G ) is a domain (resp., range) restriction node with label “self”, and ar ( E G ) = { ϵ R A } , where ϵ R A is a Graphol role expression, then Λ ( E G ) = Λ ( ϵ R A ) . S e l f (resp., Λ ( E G ) = ( Λ ( ϵ R A ) ) . S e l f );
-
If sk ( E G ) is a range restriction node with label “exists” and ar ( E G ) = { ϵ A } , where ϵ A is a Graphol attribute expression, then Λ ( E G ) = ( Λ ( ϵ A ) ) ;
-
If sk ( E G ) is a union (resp. intersection or a one-of ) node and ar ( E G ) = { ϵ 1 , , ϵ n } , then Λ ( E G ) = i = 1 n Λ ( ϵ i ) (resp. Λ ( E G ) = i = 1 n Λ ( ϵ i ) , Λ ( E G ) = { Λ ( ϵ 1 ) , , Λ ( ϵ n ) } );
-
if sk ( E G ) is a complement node and ar ( E G ) = { ϵ } , then Λ ( E G ) = ¬ Λ ( ϵ ) ;
-
If sk ( E G ) is an inverse node and ar ( E G ) = { ϵ R } , then Λ ( E G ) = ( Λ ( ϵ R ) ) ;
-
If sk ( E G ) is a chain node and ar ( E G ) = { ϵ R 1 , ϵ R n } , where every ϵ R i , where 1 i n , is a Graphol role expression that is connected to sk ( E G ) through an input edge with label i, then Λ ( E G ) = Λ ( ϵ R 1 ) Λ ( ϵ R 2 ) Λ ( ϵ R n ) .
Analogously, to define the semantics of Graphol inclusion assertions, we use a function Ψ which takes as input one such inclusion α G and returns its DL encoding. We use source ( α G ) to denote the Graphol expression having as sink the node from which the inclusion edge starts in α G , and target ( α G ) to denote the Graphol expression, having as sink the node to which the inclusion edge arrives in α G . We thus define Ψ as follows: Ψ ( α G ) = Λ ( source ( α G ) ) Λ ( target ( α G ) ) . Letting G be a Graphol ontology, we transform G in a DL ontology D L by executing Ψ for every assertion in G . The semantics of G thus are given by the semantics of D L , which we have defined in Section 3.
Table 2 and Table 3 give examples of application of Λ to Graphol expressions of “depth” 0 or 1, i.e., expressions with either only predicate nodes or a constructor node with only predicate nodes as input.
Table 2. Correspondence between Graphol and DL for concept and role expressions of depth 0 or 1. C, C 1 , and C 2 denote atomic concepts, and R, R 1 , and R 2 denote atomic roles.
Table 3. Correspondence between Graphol and DL for attribute and value-domain expressions of depth 0 or 1. V denotes an atomic attribute, and F, F 1 , and F 2 denote an atomic value-domain.

4.3. Shortcuts

We have defined some shortcuts to help the designer in specifying a Graphol ontology. Through such shortcuts it is possible to use a more compact representation of some expressions and assertions that often occur in ontologies. Below we describe them, and provide examples of their use.
The disjoint union node is a new type of node, having the shape of a black hexagon, used to represent a disjoint union expression, i.e., a union that at the same time states the disjointness between its arguments. Therefore, through a disjoint union node we also represent negative inclusions over its inputs. This shortcut is particularly useful for defining disjoint concept hierarchies. This is inspired by a similar construct used in the ER model and in UML class diagrams. In Figure 3, we give an example of one such hierarchy in the two versions, with and without the use of the shortcut.
Figure 3. Example of a disjoint concept hierarchy represented in Graphol with (left-hand side figure) and without (right-hand side figure) the disjoint union node.
The second shortcut we introduce is a compact notation for the definition of the existential domain or range restriction on a role (resp. attribute) taking as input the universal concept (resp. universal value-domain), which is equivalent to an unqualified existential restriction. Since this is among the most recurring restrictions occurring in ontologies, we allow to omit the universal concept (resp. value-domain) (notice also that the OWL 2 syntax includes redundant constructs such as owl:ObjectPropertyDomain and owl:ObjectProperyRange with a similar aim). In other words, to express the unqualified existential restriction on a role or attribute, one can simply link a role or attribute sub-graph to an existential restriction node, as shown in Figure 4.
Figure 4. Example of an existential restriction in Graphol with (left-hand side figure) and without (right-hand side figure) the compact notation.
Another commonly used assertion in ontologies is the one which specifies that a role R (resp., an attribute V) is globally functional, i.e., C 1 R . C (resp., C 1 V . D ). Global functionality can be obviously expressed in Graphol through a graphical inclusion assertion. However, in order to provide a more compact representation, we allow to use a blank double-bordered role node (resp., attribute) for a functional role, a solid double-bordered role node for an inverse functional role, and a double-bordered role half blank and half solid for a role that is both functional and inverse functional. Notice that OWL 2 also allows for the use of a compact syntax to specify global functionality (cf. owl:FunctionalObjectProperty).
As an example, in the left-hand side of Figure 5 we show the standard Graphol assertion that defines the functionality of a role, and on the right-hand side, the compact notation for a functional role (top left), an inverse functional role (top right), a role that is both functional and inverse functional (bottom left), and, finally, a functional attribute (bottom right).
Figure 5. Example of globally functional role represented without the compact notation (left-hand side), and examples (right-hand side) of globally functional role, inverse functional role, functional and inverse functional role, and functional attribute (resp., top left, top right, bottom left, and bottom right) represented with the compact notation.
We further notice that two domain (resp. range) restriction nodes with labels ( x , ) and ( , y ) , respectively, and the same input expressions, can be substituted by a single domain (resp. range) restriction node with label ( x , y ) and the same inputs. In other terms, m i n and m a x cardinality restrictions can be drawn together. Lastly, if one wishes to define equivalence between two expressions of the same kind, it is possible to use a single inclusion edge with an arrow both on the source end and on the target end instead of two inclusion edges (with the arrow only on the target end) in opposite directions.

4.4. Graphol and OWL 2

In this section we study the relationship between Graphol and OWL 2 [], the W3C standard ontology language. We first study whether Graphol ontologies can be entirely expressed in OWL 2, and then consider the other way around. In fact, for a formal treatment we identify OWL 2 with its underlying DL SROIQ ( D ) DL [,] (some OWL 2 features not in SROIQ ( D ) , such as data type restrictions or key axioms, can be modeled in Graphol by using some additional graphical elements, as reported in the Eddy documentation https://github.com/obdasystems/eddy (accessed on 17 January 2022). These aspects are not described here for the sake of simplicity).
According to the Graphol syntax given earlier, Graphol expressiveness goes slightly beyond that of OWL 2. This allowed us to maintain the formal definition of the syntax of our language simple, without burdening it with too many syntactic categories. Of course, due to this choice, reasoning in full Graphol is undecidable []. However, by suitably restricting the way in which Graphol expressions can be combined, we easily obtain decidable languages. In particular, we can limit Graphol in such a way that it becomes translatable in OWL 2. To precisely describe this restriction, we need to define basic role expressions, which are expressions constituted by either a role node or the inverse node with a role node as input. Below we give a proviso needed to our aims.
Proviso. Role expressions given as input to domain or range restriction nodes, to self nodes, inverse nodes, complement nodes, or chain nodes can be only basic role expressions. Attribute expressions given as input to domain restriction nodes can be only attribute nodes. Role and attribute expressions having the complement node as sink cannot be the source of any inclusion edge. Role expressions having the chain node as sink cannot be the target of any inclusion edge and can be included only in basic role expressions. Attribute expressions in input to range restriction nodes can be only attribute nodes. Graphol inclusion assertions between value-domain expressions must involve at least a value-domain node (i.e., the source or the target of the assertion must be an atomic data type). Finally, value-domains and values are as in OWL 2, and the ontology must obey the same global restrictions imposed on OWL 2 (e.g., only regular role hierarchies are allowed and only simple roles can occur in cardinality restrictions) [,].
Theorem 1.
EveryGrapholontology constructed under the above proviso is correctly translatable into an OWL 2 TBox in linear time.
Proof. 
The function Ψ given in Section 4.2 is obviously applicable to a Graphol ontology restricted according to the above proviso. It is then easy to verify that each DL inclusion assertion returned by Ψ is an OWL 2 axiom. Since Ψ returns one DL inclusion assertion for each Graphol inclusion assertion, the cost of the transformation is linear. □
The following theorem considers instead transformation of OWL 2 TBoxes in Graphol.
Theorem 2.
Every OWL 2 TBox is correctly translatable into aGrapholontology in linear time.
Proof. 
We first observe that any OWL 2 ontology can be written as a set of inclusion assertions. Below, we provide some well-known correspondences useful for this aim (Q denotes a role or its inverse, whereas C denotes the universal concept ).
( funct Q ) C 1 Q . C ( transitive Q ) Q Q Q ( symmetric Q ) Q Q ( asymmetric Q ) Q ¬ Q ( reflexive Q ) C Q . S e l f ( irreflexive Q ) C ¬ Q . S e l f ( disjoint Q 1 Q 2 ) Q 1 ¬ Q 2
It is then possible to define a function that translates such a normalized TBox in Graphol. Intuitively, this function inverts the function Θ introduced in Section 4.2. More precisely, we first define a function Λ 1 by induction on the structure of OWL 2 formulas, which can be seen as the inverse of the function Λ introduced in Section 4.2. For the base case, letting S be an atomic concept (resp., role, attribute, or value-domain), Λ 1 ( S ) returns a Graphol concept node (resp. role, attribute, or value-domain node) labeled with S. Letting then P . C (resp. P . C ) an OWL 2 expression, with P an atomic role and C a concept, Λ 1 ( P . C ) (resp. Λ 1 ( P . C ) ) returns the Graphol expression whose sink node is a domain (resp. range) restriction node labeled “exists” taking as input Λ 1 ( C ) and Λ 1 ( P ) . Other inductive cases can be defined analogously. Then, letting α = α α r be an OWL 2 inclusion assertion, Λ 1 ( α ) = α G , where α G is the Graphol inclusion assertion whose sink node is Λ 1 ( α ) and target node is Λ 1 ( α r ) . It is easy to see that the cost of this transformation is linear. □

4.5. Example

In Figure 6, we provide the Graphol specification of a portion of the Pizza ontology we have used in some previous examples. We also present below a logically equivalent representation of such ontology given in terms of DL assertions.
P i z z a P i z z a T o p p i n g F o o d P i z z a ¬ P i z z a T o p p i n g M e a t T o p p i n g ¬ C h e e s e T o p p i n g M e a t y P i z z a ¬ V e g e t a r i a n P i z z a h a s I n g r e d i e n t F o o d h a s I n g r e d i e n t F o o d h a s T o p p i n g h a s I n g r e d i e n t h a s T o p p i n g P i z z a T o p p i n g c a l o r i e s F o o d c a l o r i e s xsd : integer ( f u n c t c a l o r i e s ) ( f u n c t h a s T o p p i n g ) M e a t T o p p i n g C h e e s e T o p p i n g P i z z a T o p p i n g C h e e s e y P i z z a M e a t y P i z z a V e g e t a r i a n P i z z a P i z z a P i z z a ¬ h a s T o p p i n g . M e a t T o p p i n g V e g e t a r i a n P i z z a
Figure 6. Graphol ontology example: an excerpt of the Pizza ontology.

5. Comparison with UML Class Diagrams

In this section, we discuss more in depth the relation between Graphol and UML class diagrams. This section is mainly addressed to people knowledgeable with UML and willing to approach ontology modeling. The aim here is to highlight the differences between the two formalisms, so that it can be easier to understand the potential of ontology design and the shift in the modeling tools that is required to move from conceptual models of limited expressiveness to full-fledged powerful ontology languages. The content of this section should also further help understanding the syntax of Graphol starting from that of UML class diagrams. In Figure 7, we show how the main UML constructs are expressible in Graphol and for each construct we specify the corresponding OWL 2 axioms, expressed in DL syntax (UML n-ary relationships are not considered, since they are not directly expressible in OWL 2 and can always be captured using only roles, i.e., binary relationships, through reification, similarly for role attributes. We also note that all domain/range restriction nodes without labels in Figure 7 have to be read as if they were labeled with “exists”, as said in Section 4). In the first four rows of the figure we show how to represent in Graphol a UML relationship R between two concepts, with different cardinality restrictions. In the first row, where there are no such restrictions, we simply type R by requiring that its domain belongs to concept C1 and its range to concept C2. This is performed through two Graphol inclusion assertions, corresponding to the DL assertions R C 1 and R C 2 . In the second row, we add a mandatory participation of C1 to R (indicated in UML with a min cardinality restriction 1 . . * ). In Graphol, this is reflected by adding an inclusion edge from the concept node C1 to the domain of the role node R. In the third row, the UML diagram specifies that the participation of C1 to R is both mandatory and functional (notice the cardinality restriction 1 . . 1 ). In Graphol this can be easily specified by using the shortcut for global functionality introduced in Section 4.3. Indeed, in UML both the domain and range of a role are always typed on a named concept, and thus every local cardinality restriction is actually global, in the sense that every object participating in the domain or range obeys it. In the next row, the participation of C1 to R is constrained by the cardinality restriction ( x , y ) . This is represented in Graphol through the use of a second domain restriction node labeled with “ ( x , y ) ”, besides the one used to type the domain of R. The following four rows show the main cases of concept hierarchies. Here, through the union node (the hexagon labeled “or”), we represent the union expression between the two concepts C2 and C3, while we use the black hexagon (a shortcut introduced in Section 4.3) to represent their disjoint union expression. Notice that completeness of a hierarchy is specified through a double-headed arrow. The next two rows describe how attributes and restrictions over them are depicted. Note that in Graphol these are treated analogously as roles (we recall that in UML, in the absence of a cardinality restriction on attributes, they are considered mandatory and functional). Finally, in the last row, we have an example of role hierarchy, depicted in Graphol as two role nodes linked by an inclusion edge.
Figure 7. UML constructs, Graphol corresponding diagrams, and related DL assertions.

6. User Evaluation Study

In this section, we present the results and setup of the user evaluation tests we carried out on the Graphol language. The goal of the tests was to evaluate the effectiveness of Graphol for ontology design and understanding. Both these aspects were tested independently and in comparison with other ontology languages similar, in spirit, to Graphol. Users with different backgrounds and levels of experience in ontology and conceptual modeling participated in the tests.
We conducted two different studies on different models and in which users were asked to perform a variety of editing tasks.

6.1. Setup of the Study

Before defining the definitive setup of the user study, we conducted two test runs with ontology and conceptual modeling experts who were already familiar with the Graphol language, which allowed us to iteratively improve the setup of the experiments. Our primary goals for these test runs were to verify that the tasks which we asked to perform were clear, and that their difficulty was adequate for the expertise of the final test participants. Examples of modifications that we made were the removal of overcomplicated tasks, the modification of several questions to avoid ambiguity in what the questions were asking, the refinement of the cheat sheets which were handed out with the questionnaires, and the definition of the time limits for each part of the experiments.
Because we conducted two different studies, with two groups of users and on different dates, we also took advantage of the experience of the first test to refine the second one. For instance, we made some modifications to the tutorial slides which were presented prior to the second test, in light of some doubts that were expressed by the users during the presentation of these slides during the first test.

6.2. Objectives of the Study

The design of our user evaluation study was geared towards the achievement of two main objectives.
  • Evaluate the difficulty of using the Graphol language for ontology comprehension and editing by users very experienced in conceptual modeling and (in some cases) with basic skills in logics and ontologies.
  • Evaluate the difficulty of approaching and learning the Graphol language for users with only basic knowledge of conceptual modeling and little or no experience with ontologies, both in isolation, and in comparison with another graphical ontology language that is heavily based on a formalism that is already familiar to them.
In accordance to these two objectives, we identified two groups of test participants, and defined two different types of tasks for each of these groups. Further details of the test participants, of the required tasks, and of the structure of the two tests are given in the remainder of this section. Here, we give a brief sketch of these two tests: the first one included a series of comprehension tasks and of editing tasks on two ontology models represented in the Graphol language; the second one, involving users with limited knowledge of conceptual modeling and ontologies, instead included two sets of comprehension tasks on two ontologies represented in Graphol and in OWLGrEd [,].
The reason for limiting the comparative evaluation of Graphol with OWLGrEd to the comprehension test with less-skilled users is that the purpose of the test was to verify that the Graphol language could be learned and used by these users with no more effort and difficulty than that needed for another language which is, by its very nature, more recognizable, due to its strong relation to UML class diagrams. Therefore, it was not in our interest to gauge the effectiveness of Graphol against OWLGrEd, or any other similar visual language, among more expert users. Instead, our test with these users was specifically designed towards measuring their perception of Graphol as a viable candidate for future use in the design of ontologies in real life.
Furthermore, we have not carried out a comparative evaluation of our language with ontology editors which do not provide solutions for graphic editing and which are based on formal non-graphic languages, such as Protégé, given that our main goal is to deal with users who are not necessarily experts in formal languages and ontologies.

6.3. Participants

Participants were recruited among computer science master’s and Ph.D. students, postdocs, and researchers. Eighteen participants took part in the test: ten with only basic knowledge of conceptual modeling and limited or no experience with ontologies (beginners), and eight with advanced skills in conceptual modeling and basic knowledge of ontology design and logic (experts). Table 4 recaps some descriptive statistics about the age of the users, their education degree, the number of years of experience with ontologies, and their knowledge in ontologies and conceptual modeling. Note that, as expected, the knowledge of conceptual modeling among experts is in general very high (4.2 out of 5 average, with a low standard deviation of 0.7), and is fairly high among beginners (3.3 out of 5, again with a low standard deviation of 0.7). Furthermore, the average knowledge of ontologies is lower than that of conceptual modeling for both experts and beginners.
Table 4. Statistics of the participants (“Beg.” indicates statistics for beginners, “Exp.” indicates statistics for experts): for Education, 1 = Bachelor’s degree, 2 = Master’s degree, 3 = Ph.D; conceptual modeling and ontology knowledge are on a scale from 1 to 5, with 1 indicating no knowledge.

6.4. Ontology Models

We chose three different ontologies for the study, the Pizza ontology (http://130.88.198.11/co-ode-files/ontologies/pizza.owl, accessed on 17 January 2022), the Lehigh University Benchmark (LUBM) ontology (http://swat.cse.lehigh.edu/projects/lubm/, accessed on 17 January 2022), and the Family ontology (http://rpc295.cs.man.ac.uk:8080/repository/download?ontology=http://www.mindswap.org/ontologies/family.owl&format=RDF/XML, accessed on 17 January 2022), and modeled excerpts of the Pizza and LUBM ontologies in both Graphol and OWLGrEd, and of the Family ontology in Graphol. These ontologies were chosen for their popularity among the Semantic Web community, and due to the fact that the simple and widely-understood nature of the domain of these ontologies guarantees that the results of the test would not be altered by misinterpretation of the meanings of the terms in the ontologies.
The ontology models can be found in File S1 of the Supplementary Material.

6.5. Language for Comparison

Among the available language candidates for the comparative test, we chose OWLGrEd [,] which allows for designing OWL 2 ontologies through a graphical notation based on UML class diagrams. OWLGrEd was chosen because its goal and its expressive power are akin to Graphol’s, while its visual representation and UML-based design principles are rather different from those of Graphol, but are accessible, at least without big efforts, to a user with knowledge in UML class diagrams. As stated earlier, among our goals was to evaluate the difficulty of approaching the Graphol language for a non-expert user, in comparison to that of learning a language based on a formalism with which the user is familiar with, and OWLGrEd is an ideal fit for this task.
The comparative aspect of the study was limited to these two languages in order to avoid encumbering the users with an excessive amount of new information to process during the tests.
Here, we give some further details about OWLGrEd, and in Figure 8 and Figure 9, provide a very simple example of a model represented respectively in Graphol and OWLGrEd. For a complete presentation of OWLGrEd, we refer the reader to [,,].
Figure 8. Example of a small ontology in Graphol.
Figure 9. Example of a small ontology in OWLGrEd.
The OWLGrEd notation is based on UML class diagrams. Specifically,
-
OWLGrEd concepts are represented as UML classes, without operations;
-
OWLGrEd attributes are represented as UML class attributes, but with different default cardinalities;
-
OWLGrEd roles are represented as UML binary associations with the arrow indicating the direction, from the domain to the range. OWLGrEd roles are thus typed in both the domain and the range;
-
OWLGrEd cardinalities on roles are represented as UML cardinalities on roles, with the possibility of further refining the cardinality;
-
OWLGrEd cardinality restrictions on attributes are represented as UML cardinalities on attributes;
-
OWLGrEd inclusions between concepts are represented as UML ISAs between classes;
-
OWLGrEd generalizations are represented as UML generalizations, using a special graphical symbol;
-
OWLGrEd uses the OWL 2 Manchester syntax to specify expressions which denote complex concepts;
-
OWLGrEd role restrictions are represented as red arrows from the concept which is included in the restriction to the concept that qualifies the restriction, labeled with the name of the restricted role.

6.6. Tasks

We designed a series of model comprehension tasks for both the Pizza and LUBM ontology models, and of model modification tasks for the Family ontology model. The tasks were designed to present a varying degree of difficulty to the user: those for beginners were limited to more basic aspects of conceptual modeling, while those presented to experts also focused on slightly more advanced aspects.
Each comprehension task consisted of answering a question regarding the domain represented by the given model. Question types vary, from open format questions to closed format (or multiple choice) questions, to yes or no questions. Each modification task instead requested the user to modify the given Graphol model of the Family ontology by modeling one or more assertions, provided in natural language. The complete set of tests is provided in the Supplemental Material.
For each task, the participant was asked to measure the time in minutes in which he completed the task. We also asked each participant to indicate, on a scale from 0 to 4, the clarity and the easiness of each task. To understand the difference between these two response variables, consider that the participant may think he has clearly understood what he must do for a certain task, but may not be able to easily place it into practice, or vice versa. In different words, the first variable is a measure of the quality of the questionnaire, while the second is of the tools the user is provided with to carry out the tasks. Examples of the two question types are provided below.
-
Was the question clear?
Not clear at all Very clear
-
Were you able to easily answer this question?
Not at all Absolutely

6.7. Structure of the Study

Here we provide the details for the evaluation studies of beginners and experts, which were conducted separately, on different dates.
Beginners:
  • Introduction and briefGrapholtutorial (15 min): a general introduction to the purpose of the experiment, and a brief tutorial on ontologies and on the Graphol language.
  • Brief OWLGrEd tutorial (15 min): a brief tutorial on the OWLGrEd language.
  • Brief user background questionnaire (5 min): participants had to answer a brief questionnaire in which they were asked to provide some personal background information, as well as to rate their knowledge of conceptual modeling and ontologies on a scale from 1 to 5 (with 1 indicating extremely low and 5 extremely high expertise), to indicate how many years they had of experience with ontologies (if any), whether they were familiar with some of the more popular ontology editors and knowledge representation and conceptual modeling formalisms, and whether they had any experience with ontologies in real-life scenarios or in manually creating or editing ontologies.
  • LUBM comprehension tasks (40 min): each user was asked to answer ten questions on the LUBM model they were provided. Half of the users were provided a Graphol version of the LUBM model, and half an OWLGrEd version.
  • Pizza comprehension tasks (40 min): each user was asked to answer ten questions on the Pizza model they were provided. Half of the users (those which were provided the OWLGrEd version of the LUBM model) were provided a Graphol version of the Pizza model, and half an OWLGrEd version.
  • Ex-post survey (10 min): after all the comprehension tasks were completed, we asked the participants to compile a short survey regarding their experience. The survey required the users to rate, on a scale from 0 to 4, the general difficulty of the comprehension tasks, the difficulty of learning the Graphol and OWLGrEd symbols, the difficulty of using Graphol and OWLGrEd to read ontologies, and, optionally, to indicate aspects of Graphol and OWLGrEd which they particularly liked, or that they would like to see improved.
Experts:
  • Introduction and briefGrapholtutorial (30 min): participants were given the same introductory tutorial on ontologies and on the Graphol language as the beginners, with the addition of some more complex features on the Graphol language which were featured in the expert questionnaire but not the beginner questionnaire.
  • Brief user background questionnaire (5 min): we asked the participants to fill out the same background questionnaire given to the beginners.
  • Grapholcomprehension tasks (35 min): after completing the introductory part on Graphol, each user was asked to answer ten questions on the Graphol model of the Pizza ontology they were provided.
  • Grapholediting tasks (35 min): we asked each participant to perform ten editing tasks on the Graphol model of the Family ontology they were provided.
  • Ex-post survey (5 min): after carrying out both the comprehension and editing tasks, the users were asked to fill out a brief survey, analogous to the one given to the beginners.
We now discuss some more detailed aspects of the study.
-
All participants, in support of their tasks, were provided with documentation regarding the languages in play for that specific task. Specifically, the questionnaire included some cheat sheets which recapped the symbols of the Graphol and the OWLGrEd language (the latter only for tests carried out by beginners) and their meaning, along with some examples of the representation of some of the most common ontology expressions and assertions in the two formalisms. Additionally, users were provided with a printout of the slides of the introductory tutorials.
-
The order in which the tasks were presented in the questionnaires was intentionally random, i.e., not linked to the expected difficulty of each task. This choice was made in order to compensate for a potential bias given by the learning curve of familiarizing with Graphol or OWLGrEd during the course of the tasks. In other words, we wanted to avoid facilitating the participants by allowing them to face easier questions at the beginning of each task, and more difficult ones at the end, when they would probably have gained familiarity with the language.
-
The experimental design method we chose for the comparative study between Graphol and OWLGrEd is the within-subjects method, common in HCI []. This choice, as opposed to the between-subjects technique, was made mainly due to the limited number of participants to the experiment. Therefore, each user was asked to complete the comprehension tasks both for the Graphol language and for the OWLGrEd language. In order to avoid the transfer of learning effects between tasks, we split the ten users into two groups of five, and asked the first group to first carry out the comprehension tasks on the Graphol version of the LUBM model, and then the comprehension tasks on the OWLGrEd version of the Pizza model, and the second group to do the opposite.
As already stated, all questionnaires we used in our tests can be found in the Supplemental Materials.

6.8. Study Results

Figure 10 summarizes some of the results of the tests. Each box plot shows the full range of variation, from minimum to maximum, indicated by the whiskers, the likely range of variation, indicated by the two boxes, and the median value. The overall correctness results for the comparative test, shown in boxplot (a), indicate a good comprehension level by the novice users of the Graphol language, that can be considered comparable to the one obtained for OWLGrEd, which is based on UML class diagram, which is a formalism which the users were familiar with. Boxplot (b) summarizes the correctness results of the comparative test limited to five questions that deal with complex modeling aspects that go beyond UML, i.e., questions 1, 3, 5, 9, and 10 on the LUBM questionnaires, and questions 2, 4, 5, 7, and 9 on the Pizza questionnaires. These results confirm that users were able to more easily understand the completely graphical representation provided by Graphol than the formulas or non-UML constructs adopted in OWLGrEd. Finally, boxplot (c) shows the correctness results of the non-comparative test by the more expert users. The very high scores indicate that such users quickly learned how to read Graphol diagrams and how to use the language for modeling.
Figure 10. Correctness results for user tests. In boxplots (a,b) the scale is 0–20 (sum of correctness scores for five students on questions graded from 0 to 4); in boxplot (c) the scale is 0–32 (sum of correctness scores for eight students on questions graded from 0 to 4).

6.9. Post-Questionnaire Analysis

Finally, we discuss the results of the post-test questionnaires, which were presented to all participants. We recall that the goal of the post-test questionnaires was to measure the perceived general difficulty of the tasks required, of learning of test language symbols and of using Graphol to read ontologies. The average results of the questionnaires are shown in Figure 11.
Figure 11. Post-questionnare results (scale: 1–5, with 1 being the best value).
As one can see, the feedback essentially confirms the positive impression gained from the analysis of the test results. Users felt about as comfortable reading ontologies modeled through the novel Graphol language as they were with those in OWLGrEd (almost identical average scores for “Reading Difficulty”), even though, as expected, they felt that learning Graphol’s symbols was slightly more difficult than learning OWLGrEd’s UML-based ones (“Symbol Difficulty” scores). Indeed, we recall that the participants were knowledgeable in conceptual modeling languages such as UML class diagrams and entity–relationship diagrams (ER). While OWLGrEd strictly adheres to the former (at the cost of recurring to expressions in logical languages when needed), Graphol is rooted in ER, but it adds several graphical elements in order to completely cover OWL 2 without using non-graphical formulas.
The average values in the “Task Difficulty” column are more a reflection of the difficulty of the tests rather than the languages, but in both cases users seem to feel that the tests were not excessively difficult (average difficulty score of 2.4 and 2.2 out of a maximum of 5).
We finally point out that the results of the tests brought to light the need to enhance the usability of the language through a dedicated ontology tool. We used many suggestions we collected during the evaluation study to push forward the development of the Eddy editor tool for Graphol [], which is also equipped with some specific functionalities to facilitate the specification of Graphol ontologies (e.g., through commands for the automatic construction of some recurrent modeling patterns, such as concept hierarchies or role typing axioms).

7. Conclusions

In this article, we studied the graphical language for ontologies Graphol, which is as expressive as OWL 2. As we have illustrated, the key features of Graphol are its precise semantics, its expressive power, and its completely graphical representation of ontologies, inspired by popular conceptual modeling languages, such as ER. This combination sets Graphol apart from other proposals.
We remark, once again, that an editor, called Eddy [], specifically tailored to support the specification of Graphol ontologies, is available. This tool provides advanced functionalities for drawing syntactically correct Graphol diagrams, for documenting them, and for translating them into standard OWL 2 syntax. Eddy is currently developed by ODBA Systems (http://obdasystems.com/, accessed on 17 January 2022), a Sapienza startup company, and is available as open-source software (https://github.com/obdasystems/eddy, accessed on 17 January 2022).
Currently, our work is mainly focused on devising Graphol-like mechanisms for the visual specification of SPARQL queries over OWL 2 ontologies. Our idea is to automatically produce a query in SPARQL syntax on the basis of the selections of predicates performed by a user on a Graphol ontology, which naturally allows to trace the basic graph pattern of the query, plus additional conditions imposed on the query variables through user-friendly visual mechanisms [].
We are also working to add additional features to the language itself, to allow an even broader scope of modeling possibilities. In particular, we are looking into the addition of metamodeling features such as metaconcepts, which are concepts whose instances can be concepts themselves, and metaproperties, which are relationships between metaconcepts. Metaconcept representation could be useful, for instance, for the representation in Graphol of formal ontologies, where specific metaproperties, such as, for instance, rigidity, can be exploited for expressing key aspects of the intended meaning of predicates in ontologies.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/fi14030078/s1, File S1: Graphol: A graphical ontology language Survey and User Evaluation Study.

Author Contributions

Conceptualization, D.L., D.F.S., V.S. and G.D.G.; Formal analysis, D.L., D.F.S. and V.S.; Methodology, D.L. and D.F.S.; Supervision, D.L.; Validation, D.L., D.F.S. and V.S.; Writing—original draft, D.L., D.F.S., V.S. and G.D.G. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Data Availability Statement

Not applicable.

Acknowledgments

This work was partly supported by the EU within the H2020 Programme under the grant agreement 834228 (ERC Advanced Grant WhiteMec) and the grant agreement 825333 (MOSAICrOWN), by Regione Lombardia within the Call Hub Ricerca e Innovazione under the grant agreement 1175328 (WATCHMAN), and by the Italian MUR (Ministero dell’Università e della Ricerca) through the PRIN project HOPE (prot. 2017MMJJRE), by Sapienza (project CQEinOBDM), and by the J.P.Morgan AI Faculty Research Award 2021 “Resilience-based Generalized Planning and Strategic~Reasoning”.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Chen, P.P. The Entity-Relationship Model: Toward a Unified View of Data. ACM Trans. Database Syst. 1976, 1, 9–36. [Google Scholar] [CrossRef]
  2. Unified Modeling Language (UML) Superstructure, Version 2.0. 2005. Available online: http://www.uml.org/ (accessed on 17 January 2022).
  3. Sowa, J.F. Conceptual Structures: Information Processing in Mind and Machine; Addison Wesley Publ. Co.: Boston, MA, USA, 1984. [Google Scholar]
  4. Sowa, J.F. (Ed.) Principles of Semantic Networks: Explorations in the Representation of Knowledge; Morgan Kaufmann: Burlington, MA, USA, 1991. [Google Scholar]
  5. Woods, W.A. What’s in a Link: Foundations for Semantic Networks. In Representation and Understanding: Studies in Cognitive Science; Bobrow, D.G., Collins, A.M., Eds.; Academic Press: Cambridge, MA, USA, 1975; pp. 35–82. [Google Scholar]
  6. Woods, W.A.; Schmolze, J.G. The KL-ONE Family. In Semantic Networks in Artificial Intelligence; Lehmann, F.W., Ed.; Pergamon Press: Oxford, UK, 1992; pp. 133–178. [Google Scholar]
  7. Brachman, R.J.; Levesque, H.J. The Tractability of Subsumption in Frame-Based Description Languages. In Proceedings of the AAAI-84: Fourth National Conference on Artificial Intelligence, Austin, TX, USA, 6–10 August 1984; pp. 34–37. [Google Scholar]
  8. Borgida, A.; Brachman, R.J. Conceptual Modeling with Description LogicsIn. Chapter 10. pp. 349–372. Available online: http://www.cs.toronto.edu/~jm/2507S/Readings/DLTutorial.pdf (accessed on 17 January 2022).
  9. Berardi, D.; Calvanese, D.; De Giacomo, G. Reasoning on UML Class Diagrams. Artif. Intell. 2005, 168, 70–118. [Google Scholar] [CrossRef]
  10. Catarci, T.; Lenzerini, M. Representing and using Interschema Knowledge in Cooperative Information Systems. J. Intell. Coop. Inf. Syst. 1993, 2, 375–398. [Google Scholar] [CrossRef]
  11. Baader, F.; Calvanese, D.; McGuinness, D.; Nardi, D.; Patel-Schneider, P.F. (Eds.) The Description Logic Handbook: Theory, Implementation and Applications, 2nd ed.; Cambridge University Press: Cambridge, UK, 2007. [Google Scholar]
  12. Bao, J. OWL 2 Web Ontology Language Document Overview (Second Edition). W3C Recommendation, World Wide Web Consortium. 2012. Available online: http://www.w3.org/TR/owl2-overview/ (accessed on 17 January 2022).
  13. Amoroso, A.; Esposito, G.; Lembo, D.; Urbano, P.; Vertucci, R. Ontology-based Data Integration with Mastro-i for Configuration and Data Management at SELEX Sistemi Integrati. In Proceedings of the 16th Italian Conference on Database Systems (SEBD), Mondello, Italy, 22–25 June 2008; pp. 81–92. [Google Scholar]
  14. Savo, D.F.; Lembo, D.; Lenzerini, M.; Poggi, A.; Rodríguez-Muro, M.; Romagnoli, V.; Ruzzi, M.; Stella, G. Mastro at Work: Experiences on Ontology-Based Data Access. Proc. DL 2010, 573, 20–31. [Google Scholar]
  15. Calvanese, D.; De Giacomo, G.; Lembo, D.; Lenzerini, M.; Poggi, A.; Rodriguez-Muro, M.; Rosati, R.; Ruzzi, M.; Savo, D.F. The Mastro System for Ontology-based Data Access. Semant. Web J. 2011, 2, 43–53. [Google Scholar] [CrossRef]
  16. Civili, C.; Console, M.; De Giacomo, G.; Lembo, D.; Lenzerini, M.; Lepore, L.; Mancini, R.; Poggi, A.; Rosati, R.; Ruzzi, M.; et al. MASTRO STUDIO: Managing Ontology-Based Data Access Applications. Proc. VLDB Endow. 2013, 6, 1314–1317. [Google Scholar] [CrossRef]
  17. Kharlamov, E.; Giese, M.; Jiménez-Ruiz, E.; Skjaeveland, M.G.; Soylu, A.; Bagosi, T.; Console, M.; Haase, P.; Horrocks, I.; Horrocks, I.; et al. Optique 1.0: Semantic Access to Big Data. The Case of Norwegian Petroleum Directorate’s FactPages. In Proceedings of the 12th International Semantic Web Conference (ISWC), Sydney, NSW, Australia, 21–25 October 2013; pp. 65–68. [Google Scholar]
  18. Antonioli, N.; Castanò, F.; Coletta, S.; Grossi, S.; Lembo, D.; Lenzerini, M.; Poggi, A.; Virardi, E.; Castracane, P. Ontology-based Data Management for the Italian Public Debt. In Proceedings of the 8th International Conference on Formal Ontology in Information Systems (FOIS), Rio de Janeiro, Brazil, 22–25 September 2014; pp. 372–385. [Google Scholar]
  19. Aracri, R.M.; Radini, R.; Scannapieco, M.; Tosco, L. Using Ontologies for Official Statistics: The ISTAT Experience. In Current Trends in Web Engineering; Springer: Berlin/Heidelberg, Germany, 2018; pp. 166–172. [Google Scholar]
  20. Santarelli, V.; Lembo, D.; Ruzzi, M.; Ronconi, G.; Bouquet, P.; Molinari, A.; Pompermaier, F.; Caltabiano, D.; Catoni, E.; Fabrizi, A.; et al. Semantic Technologies for the Production and Publication of Open Data in ACI-Automobile Club d’Italia. In CEUR Workshop Proceedings, Proceedings of the ISWC 2019 Satellite Tracks (Posters &amp Demonstrations, Industry, and Outrageous 57 Ideas), Auckland, New Zealand, 26–30 October 2019; Volume 2456, pp. 307–308. [Google Scholar]
  21. Lembo, D.; Li, Y.; Popa, L.; Scafoglieri, F.M. Ontology mediated information extraction in financial domain with Mastro System-T. In Proceedings of the 6th International ACM Workshop on Data Science for Macro-Modeling (DSMM 2020), Portland, OR, USA, 14 June 2020; ACM Press: New York, NY, USA, 2020; pp. 3:1–3:6. [Google Scholar]
  22. Lembo, D.; Pantaleone, D.; Santarelli, V.; Savo, D.F. Drawing OWL 2 ontologies with Eddy the editor. AI Commun. 2018, 31, 97–113. [Google Scholar] [CrossRef]
  23. Lembo, D.; Pantaleone, D.; Santarelli, V.; Savo, D.F. Easy OWL Drawing with the Graphol Visual Ontology Language. In Proceedings of the 15th International Conference on the Principles of Knowledge Representation and Reasoning (KR), Cape Town, South Africa, 25–29 April 2016; pp. 573–576. [Google Scholar]
  24. Console, M.; Lembo, D.; Santarelli, V.; Savo, D.F. GRAPHOL: Ontology Representation Through Diagrams. In Proceedings of the 27th International Workshop on Description Logic (DL), Vienna, Austria, 17–20 July 2014; Volume 1193, pp. 483–495. [Google Scholar]
  25. Brockmans, S.; Volz, R.; Eberhart, A.; Löffler, P. Visual modeling of OWL DL ontologies using UML. In Proceedings of the 3rd International Semantic Web Conference (ISWC), Hiroshima, Japan, 7–11 November 2004; Lecture Notes in Computer Science. Springer: Berlin/Heidelberg, Germany, 2004; Volume 3298, pp. 198–213. [Google Scholar]
  26. Djuric, D.; Gasevic, D.; Devedzic, V.; Damjanovic, V. A UML Profile for OWL Ontologies. In Proceedings of the 2003/2004 European Workshop on Model Driven Architecture (MDAFA), Twente, The Netherlands, 26–27 June 2003; Revised Selected Papers. 2004; pp. 204–219. [Google Scholar]
  27. Guizzardi, G. Ontological Foundations for Structural Conceptual Models; Centre for Telematics and Information Technology (CTIT): Enschede, The Netherlands, 2005. [Google Scholar]
  28. Falco, R.; Gangemi, A.; Peroni, S.; Shotton, D.M.; Vitali, F. Modelling OWL Ontologies with Graffoo. In The Semantic Web: ESWC 2014 Satellite Events; Revised Selected Papers; Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2014; Volume 8798, pp. 320–325. [Google Scholar]
  29. Cerans, K.; Ovcinnikova, J.; Liepins, R.; Grasmanis, M. Extensible Visualizations of Ontologies in OWLGrEd. In The Semantic Web: ESWC 2019 Satellite Events; Revised Selected Papers; Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2019; Volume 11762, pp. 191–196. [Google Scholar]
  30. Evans, A.S. Reasoning with UML Class Diagrams. In Proceedings of the 2nd IEEE Workshop on Industrial Strength Formal Specification Techniques (WIFT), Boca Raton, FL, USA, 23 October 1998; IEEE Computer Society Press: Piscataway, NJ, USA, 1998. [Google Scholar]
  31. Artale, A.; Franconi, E. Temporal ER Modeling with Description Logics. In Proceedings of the 18th International Conference on Conceptual Modeling (ER), Paris, France, 15–18 November 1999; Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 1999; Volume 1728, pp. 81–95. [Google Scholar]
  32. Simmonds, J.; Bastarrica, M.C.; Hitschfeld-Kahler, N.; Rivas, S. A Tool Based on DL for UML Model Consistency Checking. Int. J. Softw. Eng. Knowl. Eng. 2008, 18, 713–735. [Google Scholar] [CrossRef]
  33. Chein, M.; Mugnier, M.L. Graph-Based Knowledge Representation: Computational Foundations of Conceptual Graphs; Springer: Berlin/Heidelberg, Germany, 2008. [Google Scholar]
  34. Kaneiwa, K.; Satoh, K. On the Complexities of Consistency Checking for Restricted UML Class Diagrams. Theor. Comput. Sci. 2010, 411, 301–323. [Google Scholar] [CrossRef]
  35. Queralt, A.; Artale, A.; Calvanese, D.; Teniente, E. OCL-Lite: Finite Reasoning on UML/OCL Conceptual Schemas. Data Knowl. Eng. 2012, 73, 1–22. [Google Scholar] [CrossRef]
  36. Franconi, E.; Mosca, A.; Oriol, X.; Rull, G.; Teniente, E. Logic Foundations of the OCL Modelling Language. In Proceedings of the 14th European Conference on Logics in Artificial Intelligence (JELIA), Funchal, Portugal, 24–26 September 2014; pp. 657–664. [Google Scholar]
  37. Oriol, X.; Teniente, E.; Tort, A. Computing repairs for constraint violations in UML/OCL conceptual schemas. Data Knowl. Eng. 2015, 99, 39–58. [Google Scholar] [CrossRef][Green Version]
  38. Zhang, F.; Ma, Z.M.; Cheng, J. Enhanced entity-relationship modeling with description logic. Knowl. Based Syst. 2016, 93, 12–32. [Google Scholar] [CrossRef]
  39. Halpin, T.A. Object-Role Modeling: Principles and Benefits. Int. J. Inform. Syst. Model. Des. (IJISMD) 2010, 1, 33–57. [Google Scholar] [CrossRef][Green Version]
  40. Halpin, T.A. Formalization of ORM Revisited. In OTM Workshops; Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2012; Volume 7567, pp. 348–357. [Google Scholar]
  41. Franconi, E.; Mosca, A.; Solomakhin, D. ORM2: Formalisation and Encoding in OWL2. In On the Move to Meaningful Internet Systems: OTM 2012 Workshops; Springer: Berlin/Heidelberg, Germany, 2012; pp. 368–378. [Google Scholar]
  42. Keet, C.M. Mapping the Object-Role Modeling language ORM2 into description logic language DLRidf. arXiv 2007, arXiv:cs/0702089. [Google Scholar]
  43. Wagih, H.M.; ElZanfaly, D.S.; Kouta, M.M. Mapping object role modeling 2 schemes to OWL2 ontologies. In Proceedings of the 3rd International Conference on Computer Research and Development, Shanghai, China, 11–13 March 2011; Volume 3, pp. 126–132. [Google Scholar]
  44. Sportelli, F.; Franconi, E. A Formalisation and a Computational Characterisation of ORM Derivation Rules. In On the Move to Meaningful Internet Systems: OTM 2019 Conferences-Confederated International Conferences: CoopIS, ODBASE, C&TC; Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2019; Volume 11877, pp. 678–694. [Google Scholar]
  45. Brockmans, S.; Haase, P.; Hitzler, P.; Studer, R. A metamodel and UML profile for rule-extended OWL DL ontologies. In The Semantic Web: Research and Applications; Springer: Berlin/Heidelberg, Germany, 2006; pp. 303–316. [Google Scholar]
  46. Object Management Group. Ontology Definition Metamodel. Technical Report formal/2009-05-01, OMG. 2009. Available online: http://www.omg.org/spec/ODM/1.0 (accessed on 17 January 2022).
  47. Bechhofer, S.; van Harmelen, F.; Hendler, J.; Horrocks, I.; McGuinness, D.L.; Patel-Schneider, P.F.; Stein, L.A. OWL Web Ontology Language Reference. W3C Recommendation, World Wide Web Consortium. 2004. Available online: http://www.w3.org/TR/owl-ref/ (accessed on 17 January 2022).
  48. Benevides, A.B.; Guizzardi, G. A model-based tool for conceptual modeling and domain ontology engineering in OntoUML. In Enterprise Information Systems; Springer: Berlin/Heidelberg, Germany, 2009; pp. 528–538. [Google Scholar]
  49. Barzdins, J.; Barzdins, G.; Cerans, K.; Liepins, R.; Sprogis, A. UML Style Graphical Notation and Editor for OWL 2; BIR, Forbrig, P., Günther, H., Eds.; Lecture Notes in Business Information Processing; Springer: Berlin/Heidelberg, Germany, 2010; Volume 64, pp. 102–114. [Google Scholar]
  50. Gaines, B.R. Designing visual languages for description logics. J. Logic Lang. Inf. 2009, 18, 217–250. [Google Scholar] [CrossRef][Green Version]
  51. Dau, F.; Eklund, P. A diagrammatic reasoning system for the description logic ALC. J. Vis. Lang. Comput. 2008, 19, 539–573. [Google Scholar] [CrossRef]
  52. Krivov, S.; Williams, R.; Villa, F. GrOWL: A tool for visualization and editing of OWL ontologies. J. Web Sem. 2007, 5, 54–57. [Google Scholar] [CrossRef]
  53. Lohmann, S.; Negru, S.; Haag, F.; Ertl, T. Visualizing ontologies with VOWL. Semant. Web J. 2016, 7, 399–419. [Google Scholar] [CrossRef]
  54. Dudáš, M.; Lohmann, S.; Svátek, V.; Pavlov, D. Ontology visualization methods and tools: A survey of the state of the art. Knowl. Eng. Rev. 2018, 33, E10. [Google Scholar] [CrossRef]
  55. Wiens, V.; Lohmann, S.; Auer, S. WebVOWL Editor: Device-Independent Visual Ontology Modeling. In Proceedings of the International Semantic Web Conference (ISWC 2018) Posters & Demonstrations, Industry and Blue Sky Ideas Tracks, Monterey, CA, USA, 8–12 August 2018; Volume 2180. [Google Scholar]
  56. da Silva, I.; Santucci, G.; del Sasso Freitas, C. Ontology visualization: One size does not fit all. In Proceedings of theEuroVA 2012: International Workshop on Visual Analytics, Vienna, Austria, 4–5 June 2012; pp. 91–95. [Google Scholar]
  57. Shneiderman, B. Tree visualization with tree-maps: 2-d space-filling approach. ACM Trans. Graph. (TOG) 1992, 11, 92–99. [Google Scholar] [CrossRef]
  58. de Souza, K.X.; dos Santos, A.D.; Evangelista, S.R. Visualization of ontologies through hypertrees. In Proceedings of the Latin American Conference on Human-Computer Interaction, Rio de Janeiro, Brazil, 17–20 August 2003; ACM Press: New York, NY, USA, 2003; pp. 251–255. [Google Scholar]
  59. Wang Baldonado, M.Q.; Woodruff, A.; Kuchinsky, A. Guidelines for using multiple views in information visualization. In Proceedings of the Working Conference on Advanced Visual Interfaces, Palermo, Italy, 23–26 May 2000; ACM Press: New York, NY, USA, 2000; pp. 110–119. [Google Scholar]
  60. Lanzenberger, M.; Sampson, J.; Rester, M. Visualization in Ontology Tools. In Proceedings of the 2009 International Conference on Complex, Intelligent and Software Intensive Systems, (CISIS), Fukuoka, Japan, 16–19 March 2009; IEEE Computer Society: Washington, DC, USA, 2009; pp. 705–711. [Google Scholar]
  61. Katifori, A.; Halatsis, C.; Lepouras, G.; Vassilakis, C.; Giannopoulou, E.G. Ontology visualization methods—A survey. ACM Comput. Surv. 2007, 39, 10. [Google Scholar] [CrossRef]
  62. Gennari, J.H.; Musen, M.A.; Fergerson, R.W.; Grosso, W.E.; Crubézy, M.; Eriksson, H.; Noy, N.F.; Tu, S.W. The evolution of Protégé: An environment for knowledge-based systems development. Int. J. Hum.-Comput. Stud. 2003, 58, 89–123. [Google Scholar] [CrossRef]
  63. Haase, P.; Lewen, H.; Studer, R.; Tran, D.T.; Erdmann, M.; d’Aquin, M.; Motta, E. The NeOn ontology engineering toolkit. In Proceedings of the 17th International World Wide Web Conference (WWW), Beijing, China, 21–25 April 2008. [Google Scholar]
  64. Adamou, A.; Palma, R.; Haase, P.; Montiel-Ponsoda, E.; Aguado de Cea, G.; Gómez-Pérez, A.; Peters, W.; Gangemi, A. The NeOn Ontology Models. In Ontology Engineering in a Networked World; Springer: Berlin/Heidelberg, Germany, 2012; pp. 65–90. [Google Scholar]
  65. Weiten, M. OntoSTUDIO® as a Ontology Engineering Environment. In Semantic Knowledge Management; Springer: Berlin/Heidelberg, Germany, 2009; pp. 51–60. [Google Scholar]
  66. Horrocks, I.; Kutz, O.; Sattler, U. The Even More Irresistible SROIQ. In Proceedings of the 10th International Conference on the Principles of Knowledge Representation and Reasoning (KR), Lake District, UK, 2–5 June 2006; pp. 57–67. [Google Scholar]
  67. Horrocks, I.; Sattler, U. Ontology Reasoning in the SHOQ(D) Description Logic. In Proceedings of the 17th International Joint Conference on Artificial Intelligence (IJCAI), Seattle, WA, USA, 4–10 August 2001; pp. 199–204. [Google Scholar]
  68. Bertin, J. Semiology of Graphics: Diagrams, Networks, Maps; University of Wisconsin Press: Madison, WI, USA, 1983. [Google Scholar]
  69. Miller, G.A. The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychol. Rev. 1956, 63, 81. [Google Scholar] [CrossRef]
  70. Nordbotten, J.C.; Crosby, M.E. The effect of graphic style on data model interpretation. Inf. Syst. 1999, 9, 139–155. [Google Scholar] [CrossRef]
  71. Goodman, N. Languages of Art: An Approach to a Theory of Symbols; Hackett Publishing: Indianapolis, IN, USA, 1976. [Google Scholar]
  72. Moody, D.L. The “physics” of notations: Toward a scientific basis for constructing visual notations in software engineering. Softw. Eng. IEEE Trans. 2009, 35, 756–779. [Google Scholar] [CrossRef]
  73. Motik, B.; Parsia, B.; Patel-Schneider, P.F. OWL 2 Web Ontology Language Structural Specification and Functional-Style Syntax. W3C Recommendation, World Wide Web Consortium. 2012. Available online: http://www.w3.org/TR/owl2-syntax/ (accessed on 17 January 2022).
  74. Barzdins, J.; Cerans, K.; Liepins, R.; Sprogis, A. Advanced Ontology Visualization with OWLGrEd. In CEUR Workshop Proceedings; Dumontier, M., Courtot, M., Eds.; OWLED. Available online: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.365.4375&rep=rep1&type=pdf (accessed on 17 January 2022).
  75. Dix, A. Human-Computer Interaction; Springer: Berlin/Heidelberg, Germany, 2009. [Google Scholar]
  76. Di Bartolomeo, S.; Pepe, G.; Savo, D.F.; Santarelli, V. Sparqling: Painlessly Drawing SPARQL Queries over Graphol Ontologies. In Proceedings of the CEUR Electronic Workshop Proceedings, Fourth International Workshop on Visualization and Interaction for Ontologies and Linked Data (VOILA@ISWC 2018), Monterey, CA, USA, 8 October 2018; Volume 2187, pp. 64–69. [Google Scholar]
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Article Metrics

Citations

Article Access Statistics

Multiple requests from the same IP address are counted as one view.