Graphol: A Graphical Language for Ontology Modeling Equivalent to OWL 2

Round 1

Reviewer 1 Report

This paper presents a language named Graphol for ontology modelling, which in turn is equivalent to OWL 2 (the standard language for the Semantic Web). The paper describes Graphol, its syntax and semantics. Also, it gives linear-time mappings to OWL 2 and vice-versa. Finally, it presents a user evaluation by comparing Graphol against OWLGrEd. 

For sure, Graphol tackles a relevant and interesting problem in ontology engineering. It has a theoretical background and as the authors remarked it has been successfully used in some industrial use cases. It has been also studied for more than 6/7 years. Moreover, Graphol along with Eddy are part of a suite of tools developed by a startup of Sapienza University of Rome. 

However, I have a major comment about this paper. I find it very similar to one of the previous papers published by some of the authors here. In particular: 

Marco Console, Domenico Lembo, Valerio Santarelli, Domenico Fabio Savo "Graphol: Ontology Representation through Diagrams". Description Logics 2014.

In general, my reason for the review decision is that it is not clear to me which is the research addition shown in this paper in comparison with that previous publication.

There, the authors presented the Graphol language, its syntax and semantics and provided a very similar but shorter user evaluation than the presented one in this current work. Such a DL paper is not referenced either.
Then, the only "novelty" is in section 4.4 "Graphol and OWL 2". With reference to the user evaluation, the authors do provide a more detailed specification of the experiments: objectives, languages and participants. However, the results of both experiments also seem to be similar ones (Fig 10 in the current paper vs Fig 5 in the previous work). For instance: Comprehension and Editing results or "...we also observed that Graphol has been perceived as easier than OWLGrEd for more complex expressions, which in OWLGrED require the use of Manchester syntax formulas..."

More comments:
1) The authors include section 5 "Comparison with UML class diagrams" where they compare UML structures against Graphol diagrams. However, this section does not draw any interesting conclusion about the comparison. What do you intend to show there?

2) In section 6.2, "Moreover, we are convinced that none of the graphical solutions for ontology representation discussed in Section 2 has succeeded in providing expert ontology users or designers with an effective tool to replace the more widely used non-graphical editors such as Protégé, and the fact that, to the best of our knowledge, these solutions have very rarely been adopted in real-life scenarios seems to corroborate this conviction"

I could agree with these sentences but how do you support them?

3) About the study results (sections 6.8/6.9). Have you identified any possible enhancements from the participants about Graphol symbols or modelling tasks? I mean, Graphol symbols are more difficult to read but why? How this study could help to enhance Graphol? For instance: the additional features described in the conclusions (metamodelling) have been derived from this study?

4) section 4.3. line 438-449. How do you know that such restrictions are the most recurring ones? Maybe some references should be provided.

Minor comment:

5) Have you considered evaluating Graphol by using Moody's principles?
 
Conclusion:

Overall it is clear that Graphol is relevant for the ontology engineering community. The paper is well structured and written. However, I urge the authors to look into improving the paper so that it clearly shows new technical/engineering improvements when compared against your previous DL work.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

GRAPHOL is language similar to OWLgrEd language, except that it is fully graphical rather than partially textual. Its syntax is that of a basic GRAPH drawing language (i.e., it is based on the drawing syntax of the UML class diagram modeler) extended with a few extra graphical symbols. The semantics of GRAPHOL is that of OWL 2.

The paper features a set of transformation rules that allow to translate the GRAPHOL syntax into standard OWL 2 syntax.

Author Response

We thank the reviewer for her\his evaluation of our submission 

Reviewer 3 Report

This paper proposes and evaluated a graphical language for OWL 2 ontologies. It is an interesting paper, with a significant contribution, useful for many applications and well compared with the related work. However, presentation must be improved to guarantee that the text is technically correct.

I agree that it is possible to identify a subset of Graphol which is equivalent to OWL 2, but I am not sure that the current presentation is correct. In particular, I have doubts about Theorem 2. I am not sure that the fact that "any OWL 2 ontology can be written as a set of inclusion axioms" is correct: the authors should (i) provide a reference to a proof, and (ii) include not some but all correspondences. Indeed, in the paper "The Even More Irresistible SROIQ" (Proceedings of KR 2006, pp. 57-67), the authors (Ian Horrocks, Oliver Kutz, Ulrike Sattler) state that "None of reflexivity, irreflexivity, antisymmetry or disjointness of roles can be enforced by role inclusion axioms", which seems to contradict your claim. For example, it seems that the proposed correspondence for irreflexive axioms cannot be expressed in OWL 2, where the negation of a role can only appear in a negated role assertion.

Minor comments:

• Line 45. Rather than saying that Graphol diagrams can be traslated into OWL 2 TBoxes and viceversa, I think you should say OWL 2 ontologies, to include ABoxes and RBoxes.
• Line 88. This section could mention existing works translating from UML or ER to DLs and viceversa, even if the scope of such papers is different.
• Line 182. "e set" -> "a set".
• Line 245. Most of the readers would be more familiar with OWL 2 than with DLs, so you could use or at least mention the common terminology used in OWL 2 (object properties, data properties, datatypes...)
• Line 256. I am aware that you are trying to a give a presentation of the DL language that is easy to understand, but the language here does not correspond to SROIQ(D). Actually, I have never seen this language before, and I am not sure that defining a new language or using a nonexistent DL language makes sense.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 4 Report

The article describes a visual language for developing OWL ontologies. 

The topic is of interest and well described. It would be better to have some justification of the design rationale. Another weakness concerns the number of participants. Since the article proposes a comparative study rather than a pure usability test, to have sensible results on expects appropriate number of participants.

The article is of value, although the evaluation could be considered anecdotal.

Author Response

The article describes a visual language for developing OWL ontologies. 

The topic is of interest and well described. It would be better to have some justification of the design rationale. Another weakness concerns the number of participants. Since the article proposes a comparative study rather than a pure usability test, to have sensible results one expects appropriate number of participants.

The article is of value, although the evaluation could be considered anecdotal.

We thank the reviewer for the comment. As for the number of participants to our tests, we would like to point out that it does not seem too small to produce significant results considering that the number of participants to our evaluation study is in line with guidelines on usability testing provided for instance in “Dix, A., Finlay, J.E., Abowd, G.D., Beale, R.: Human-Computer Interaction (3rd Edition). Prentice Hall, 3 edn. (Dec 2003)”. Having said this, we agree that increasing the number of participants would not hurt the results of our study, but finding a significantly larger number of participants is unfortunately not practically easy to do, especially in the case of conceptual modeling experts.

Round 2

Reviewer 1 Report

I would like to thank the authors for considering my comments, which have been addressed too.

Now, I think that the paper is ok for publishing in the journal.