Semantic Similarity Based on Taxonomies

Abstract

The evaluation of the semantic similarity of concepts organized according to taxonomies is a long-standing problem in computer science and has attracted great attention from researchers over the decades. In this regard, the notion of information content plays a key role, and semantic similarity measures based on it are still on the rise. In this review, we address the methods for evaluating the semantic similarity between either concepts or sets of concepts belonging to a taxonomy that, often, in the literature, adopt different notations and formalisms. The results of this systematic literature review provide researchers and academics with insight into the notions that the methods discussed have in common through the use of the same notation, as well as their differences, overlaps, and dependencies, and, in particular, the role of the notion of information content in the evaluation of semantic similarity. Furthermore, in this review, a comparative analysis of the methods for evaluating the semantic similarity between sets of concepts is provided.

1. Introduction

In the literature, we are assisting a growing interest in the problem of evaluating the semantic similarity between concepts, words, digital resources, etc., not only in computer science but also in the social sciences, medicine, biology, etc. [1]. Semantic similarity, i.e., the identification of different entities that are semantically close, is used in many research areas, such as bioinformatics [2,3], natural language processing [4,5], semantic web search [6,7], geographic information systems [8,9], and business process management [10,11], often by using different notations, overlapping definitions, etc., and, currently, it is still a challenge.

In the literature, there are at least two dimensions along which semantic similarity methods are organized. They are the type of resources that are used to accomplish this task and the type of entities that are compared. On the basis of the type of used resources, most of the approaches fall into one of the following categories: corpus-based methods, that use large corpora of natural language texts and leverage co-occurrences of words [12]; knowledge-based methods that rely on structured resources [13]; and hybrid methods that are a mixture of both the mentioned approaches [14].

In the last ten years, many methods have used machine learning and deep learning techniques to compute semantic similarity by encoding the available resources as numerical vectors. When these resources are in the form of textual documents, this step is referred to as word embedding, whereas, when dealing with graph-shaped knowledge, such as graph embedding (e.g., [15]). Therefore, even if in some cases they are presented as a further category (e.g., [1]), depending on the type of used resources, they fit in one of the three above categories.

This work focuses on the second category, i.e., on knowledge-based methods, and in particular on methods that exploit a taxonomy, which is a set of concepts organized according to the well-known is-a relationship [16]. In the framework of taxonomy-based similarity methods, the types of entities to be compared are usually concepts or sets of concepts. In particular, semantic annotations of real-world resources, for instance, commercial products and scientific papers, are in general sets of concepts. Furthermore, in many cases, resources are tagged with sets of concepts belonging to a taxonomy. The approach of evaluating the similarity of resources by using semantic annotations is particularly useful in the case the content of the resources is not available. For instance, many scientific journals require categorizing articles with keywords from a classification system organized according to a taxonomy like the Association for Computing Machinery (ACM) Computing Classification System (https://dl.acm.org/ccs, accessed on 29 October 2023), or the Physics Subject Headings classification system (https://physh.org/browse, accessed on 29 October 2023). But while the keywords associated with the articles and classification systems are publicly available, the articles’ contents are not. Then, in order to search for an article addressing given topics, taxonomy-based semantic similarity is a significant opportunity.

Therefore, the objective of this paper is to provide an overview of taxonomy-based methods for computing the semantic similarity between either concepts or sets of concepts. Also, it has shown that taxonomy-based methods for computing the semantic similarity between sets of concepts rely on the ones to compute the similarity between concepts. Furthermore, since many of these methods use an information theoretic approach, the notion of information content for deriving the informativeness of a concept in a taxonomy, and methods for computing it are also introduced. The results of this review aim to provide researchers and academics with an insight into the notions that the methods proposed in the literature have in common, using the same notation, the differences of these methods, their overlaps and dependencies, and in particular the role of the notion of information content in assessing semantic similarity.

The work is organized as follows. Section 2 introduces the reader to the notion of information content and the methods for computing its value. Section 3 and Section 4 show the taxonomy-based methods for computing the similarity between concepts and sets of concepts, respectively. Section 5 focuses on one of the methods for evaluating the semantic similarity between sets of concepts, which, according to the current literature, outperforms the other mentioned approaches. Section 6 provides a discussion about the illustrated methods and also about the results of a recent experiment. Section 7 presents the conclusions and future directions.

2. Methods for Computing the Information Content of a Concept in a Taxonomy

The notion of the information content (IC) of a concept to compute semantic similarity was introduced for the first time by Resnik [17]. Based on the standard argumentation of information theory, it is defined as the negative log of the likelihood of the concept (see Equation (A1) in Appendix A). This is an extensional, or corpus-based, method as the likelihood of a concept is computed as the relative frequency of that concept in a corpus of documents. Another extensional method is annotation frequency IC [18], which exploits the inverse document frequency (IDF) of a concept in a corpus of textual resources [19]. In particular, in [18], this method is applied to annotation vectors, i.e., sets of concepts of a taxonomy that are used to semantically annotate digital resources.

However, most of the methods in the literature for computing IC are intensional or intrinsic. In fact, they compute it by considering only the structure of the taxonomy and do not need external resources. Indeed, extensional methods require the analysis of documents whose dimensions are statistically significant. And this amount of textual resources is not always available, especially in the case of very specific application domains. Therefore, intensional methods aim at overcoming these limits and avoiding additional efforts by exploiting the sole topology of the taxonomy. In particular, intensional methods leverage the following features of the taxonomy (see

Table 1. Basic definitions.

Notation	Description
Taxonomy T	$T=(C,E)$, where C is a set of nodes or concepts, and E is a set of edges,
	i.e., concept pairs, $(c_i,c_j)$, such that $c_i$, $c_j$ ∈ C, and $c_i$ is-a $c_j$ holds
hyper(c)	The set of the hypernyms (or subsumers) of the concept c
hypo(c)	The set of the hyponyms (or subsumes) of the concept c
directHyper(c)	The set of hypernyms of c directly linked to it, i.e., the set of concepts
	$\left\{c_i\right\}$, such that $(c,c_i)\in E$
directHypo(c)	The set of hyponyms of c directly linked to it, i.e., the set of concepts
	$\left\{c_i\right\}$ such that $(c_i,c)\in E$
lcs(c${}_i$, c${}_j$)	The least common subsumer, i.e., one of the most specific common hypernyms
	of $c_i$ and $c_j$ (that, in a tree-shaped taxonomy, is unique)
len(c${}_i$, c${}_j$)	The shortest path length between $c_i$ and $c_j$, i.e, the length of the path
	with the minimum number of edges connecting $c_i$, $c_j$
depth(c)	The shortest path length between c and the root of the taxonomy
height(T)	The maximum depth that a concept can have in T
leaves(T)	The set of all the leaves in T, i.e., the concepts without hyponyms
leaves(c)	The set of leaves having c as an hypernym
siblings(c)	The set of concepts $\left\{c_i\right\}$ such that $directHyper\left(c_i\right)\cap directHyper\left(c\right)\ne \varnothing$

, where the fundamental definitions are recalled):

• The number of hyponyms of the concept, where the greater it is, the less the IC. This is due to the assumption that the more general a concept, the less its informativeness. This feature is used by Seco et al. IC [20], Zhou et al. IC [21], and Sanchez and Batet 2 IC [22]. Taieb et al. IC [23] considers the hyponyms of the hypernyms of the concept, but the basic assumption continues to be applied.
• The depth of the concept, where the greater it is, the greater the IC. Again, this is in line with the general assumption regarding the specificity of a concept and its informativeness. This feature is used by Zhou et al. IC [21] and Meng et al. IC [24], who also consider the depth of the hyponyms, and Yuan et al. IC [25] and Taieb et al. IC [23], who also consider the depth of the hypernyms.
• The number of the hypernyms of the concept, which again take into consideration the assumption above. In fact, the greater the number of the hypernyms, the greater the IC. This feature is used by Yuan et al. IC [25] and Taieb et al. IC [23], who consider the hypernyms of the hypernyms, and Sanchez and Batet 1 IC [26].
• The number of the leaves of the concept, where the greater it is, the less the IC. This is again in accordance with the general assumption that correlates the specificity of a concept with its informativeness. This feature is used by Sanchez and Batet 2 IC [22] and Yuan et al. IC [25].
• The number of siblings of the concept, where the greater it is, the greater the IC. Here, the underlying assumption is that the greater the number of siblings of a concept, the greater its peculiarity and its informativeness, too. This assumption is exploited in the top down IC found in [27] and by Sebti and Barfroush IC [28]. However, in the latter, the siblings of hypernyms are also considered.

In general, the number of hyponyms and the number of hypernyms of a concept are normalized by the total number of the concepts in the taxonomy, whereas the depth of the concept is normalized by the height of the whole taxonomy.

The main features of the above recalled methods are summarized in Table 2.

Additional methods for computing the IC of concepts are, for instance, the ones proposed in [29], which exploits not only the is-a relationship but also the synonymy and polysemy contained in semantic structures such as WordNet, and in [30], which computes the IC of events in a process model. However, they are not further detailed in the present work since they are not based solely on a taxonomy.

Table 2. Methods for computing the IC of a concept c in a taxonomy.

Method	Year	Features
Resnik IC [17]	1995	Frequency of c in a corpus of documents
Seco et al. IC [20]	2004	Hyponyms of c
Zhou et al. IC [21]	2008	Hyponyms and depth of c, and height of T, and tuning parameter
Sebti and Barfroush IC [28]	2008	Siblings of c and of its hypernyms
Meng et al. IC [31]	2012	Depth of c and of its hyponyms, and height of T
Sanchez and Batet 1 IC [26]	2012	Hypernyms of c and of its leaves
Sanchez and Batet 2 IC [22]	2013	Hyponyms and leaves of c, and leaves of T
Top Down IC [27]	2013	Direct hypernyms and siblings of c
Yuan et al. IC [25]	2013	Hypernyms, depth, and leaves of c, and height and leaves of T
Taieb et al. IC [23]	2014	Hypernyms and depth of c, hyponyms, and depth of the hypernyms
Adhikari et al. IC [32]	2015	Depth, leaves and hypernyms of c, depth of hyponyms of c, height and leaves of T
Zhang et al. IC [33]	2018	Hypernyms and hyponyms of c, siblings of hypernyms of c
Annotation Frequency IC [18]	2023	$IDF$ of c in a corpus of documents

Table 2. Methods for computing the IC of a concept c in a taxonomy.

Method	Year	Features
Resnik IC [17]	1995	Frequency of c in a corpus of documents
Seco et al. IC [20]	2004	Hyponyms of c
Zhou et al. IC [21]	2008	Hyponyms and depth of c, and height of T, and tuning parameter
Sebti and Barfroush IC [28]	2008	Siblings of c and of its hypernyms
Meng et al. IC [31]	2012	Depth of c and of its hyponyms, and height of T
Sanchez and Batet 1 IC [26]	2012	Hypernyms of c and of its leaves
Sanchez and Batet 2 IC [22]	2013	Hyponyms and leaves of c, and leaves of T
Top Down IC [27]	2013	Direct hypernyms and siblings of c
Yuan et al. IC [25]	2013	Hypernyms, depth, and leaves of c, and height and leaves of T
Taieb et al. IC [23]	2014	Hypernyms and depth of c, hyponyms, and depth of the hypernyms
Adhikari et al. IC [32]	2015	Depth, leaves and hypernyms of c, depth of hyponyms of c, height and leaves of T
Zhang et al. IC [33]	2018	Hypernyms and hyponyms of c, siblings of hypernyms of c
Annotation Frequency IC [18]	2023	$IDF$ of c in a corpus of documents

3. Methods for Computing Semantic Similarity between Concepts

This section is dedicated to presenting the methods for evaluating a similarity degree between concepts in a taxonomy. They are organized into two groups, which distinguish between methods that are based on the IC of the concepts (Section 3.1) and methods that are not (Section 3.2). It is worth mentioning that IC-based semantic similarity has been extensively experimented in the literature, relying both on statistical information from a large-scale corpus (the Resnik’s approach [17]) or on the intrinsic knowledge contained in the hierarchical structure of the taxonomy (Seco’s [20] or Zhang 1 et al.’s [33] formulations), and the experimental results overall show a higher correlation with human judgment than non-IC-based approaches [33].

3.1. Information Content-Based Methods

To compute the similarity between concepts in a taxonomy, a substantial group of methods relies on the IC of concepts. They are briefly described in the following.

• Resnik similarity ($si{m}_{res}$) [17] (see Equation (1)), which assumes that the more information two concepts share, the more similar they are. Then, the information shared by two concepts is provided by the IC of the concepts that subsume them in the taxonomy:

  $$si{m}_{res}(c_1,c_2)=ma{x}_{c_i\in hyper\left(c_1\right)\cap hyper\left(c_2\right)}ic\left(c_i\right)$$

  (1)

  where the maximum value is obtained for $c_i$ is one of the $lcs(c_1,c_2)$.
• Jiang and Conrath similarity ($si{m}_{j\&c}$) [34] (see Equation (2)), which depends on the IC of three nodes in the taxonomy, i.e, the two compared concepts, and their least common subsumer:

  $$si{m}_{j\&c}(c_1,c_2)=\frac{1}{ic\left(c_1\right)+ic\left(c_2\right)-2\ast ic\left(lcs(c_1,c_2)\right)}.$$

  (2)
• Lin similarity ($si{m}_{lin}$) [35] (see Equation (3)), which, analogously to [34], is based on the ICs of the two compared concepts and their lcs:

  $$si{m}_{lin}(c_1,c_2)=\frac{2\ast ic\left(lcs(c_1,c_2)\right)}{ic\left(c_1\right)+ic\left(c_2\right)}.$$

  (3)
• P&S similarity ($si{m}_{P\&S}$) [36], inspired by Tversky’s set-theoretic formulation of similarity [37], which is based on both the ICs of the two compared concepts and the one of their lcs (see Equation (4)):

  $$si{m}_{p\&s}(c_1,c_2)=3\ast ic\left(lcs(c_1,c_2)\right)-ic\left(c_1\right)-ic\left(c_2\right).$$

  (4)
• Meng and Zhou similarity [24], based on the IC and shortest path length, as shown in Equation (5):

  $$si{m}_{m\&z}(c_1,c_2)={\left(\frac{2\ast ic\left(lcs(c_1,c_2)\right)}{ic\left(c_1\right)+ic\left(c_2\right)}\right)}^{\left(\frac{1-e^{-k\ast len(c_1,c_2)}}{e^{-k\ast len(c_1,c_2)}}\right)}$$

  (5)

  where k is a factor, which is manually computed to improve the performance of the method.
• wpath similarity ($si{m}_{wpath}$) [38], defined in terms of the shortest path distance between the compared concepts and the ICs of their lcs (see Equation (6)):

  $$si{m}_{wpath}(c_1,c_2)=\frac{1}{1+len(c_i,c_j)\ast k^{ic\left(lcs\right(c\left)\right)}}$$

  (6)

  where k is the parameter that weighs the contribution of the IC of the lcs.
• Zhang 1 similarity ($si{m}_{zhang\_1}$) [33] (see Equation (7)), which uses the Lin similarity [35], as expressed by the argument of the logarithm function. However, differently from Lin, Zhang’s similarity, in its original version, it adopts the work of Zhang et al. IC [33] to compute the IC of a concept:

  $$si{m}_{zhang\_1}(c_1,c_2)=1-log\left(2-\frac{2\ast ic\left(lcs(c_1,c_2)\right)}{ic\left(c_1\right)+ic\left(c_2\right)}\right).$$

  (7)
• D${}_k$ similarity, proposed in [39] for IC-based similarity measures, which addresses the concept intended senses in a given context $D_k$ (or application domain). In particular, the semantic similarity of the concepts $c_1,c_2$, indicated as $si{m}_{D_k}(c_1,c_2)$, is defined by Equation (8):

  $$si{m}_{D_k}(c_1,c_2)=sim(c_1,c_2)\ast (1-{\omega }_k)+sim(S_{D_k}\left(c_1\right),S_{D_k}\left(c_2\right)))\ast {\omega }_k$$

  (8)

  where $sim$, in the original proposal, is any similarity measure between concepts based on IC, ${\omega }_k$ is a weight, $0\le {\omega }_k\le 1$, defined by the domain expert according to $D_k$, and $S_{D_k}$ is a function, referred to as the intended sense function, associating a concept with its meaning according to $D_k$ (which can be another concept or itself).
• Hierarchical semantic similarity ($si{m}_{HSS}$) [40], which was originally conceived for computing the similarity between words. Due to polysemy, one word can refer to different concepts, each representing a sense of that word. Then, if we assume that a word can refer to only one concept in the taxonomy, the complexity of the method decreases. However, we present it according to its original formulation, in which $w_1$ and $w_2$ are the compared words (see Equation (9)):

  $$si{m}_{HSS}(w_1,w_2)=\sum _{l\in \mathcal{L}}\left(\frac{S_{<w_1,w_2>\in l}}{{\left|hypo\left(l\right)\right|}^2}\ast \frac{\left|hypo\right(l)+1|}{\left|C\right|}\right)\ast \frac{1}{\frac{{\sum }_{k\in \mathcal{L}}{S}_{<w_1,w_2>\in k}}{{\left|hypo\left(k\right)\right|}^2}}$$

  (9)

  where $S_{<w_1,w_2>\in c_i}$ is the number of pairs of senses of $w_1$ and $w_2$, i.e., concepts in the taxonomy having $c_i$ as the lcs, and $\mathcal{L}$ is the set of the lcs of pairs of concepts representing senses of $w_1$ and $w_2$, respectively.

3.2. Non-Information Content-Based Methods

• Rada distance. In the literature, many proposals compute semantic similarity by using the notion of conceptual distance between concepts in a taxonomy as defined by Rada [41], as reported in Equation (10)

  $$D_{rada}(c_1,c_2)=len(c_1,c_2)$$

  (10)

  i.e., the minimum number of edges separating $c_1$ and $c_2$ in the taxonomy. In particular, the smaller the distance between concepts, the more similar the concepts are (conceptual distance is a decreasing function of similarity). In the mentioned paper, the authors show that conceptual distance satisfies the properties of a metric:
  - $len(c_1,c_1)$ = 0 (zero property);
  - $len(c_1,c_2)$ = $len(c_2,c_1)$ (symmetric property);
  - $len(c_1,c_2)$ ≥ 0 (positive property);
  - $len(c_1,c_2)$ + $len(c_2,c_3)$ ≥ $len(c_1,c_3)$ (triangular inequality).
  However, in general, symmetry and triangular inequality are not satisfied by semantic similarity.
• Wu and Palmer similarity [42] is based on the depth of the least common subsumer of the concepts and the conceptual distance of the concepts from it, as defined by Equation (11):

  $$si{m}_{w\&p}(c_1,c_2)=\frac{2\ast depth\left(lcs(c_1,c_2)\right)}{len(c_1,lcs(c_1,c_2))+len(c_2,lcs(c_1,c_2))+2\ast depth\left(lcs(c_1,c_2)\right)}.$$

  (11)
• Leacock and Chodorow similarity [43] is based on the shortest path between the two concepts. It also considers the maximum depth of the taxonomy T, i.e., its height, as in Equation (12)

  $$si{m}_{l\&c}(c_1,c_2)=-log\left(\frac{len(c_1,c_2)}{2\ast height\left(T\right)}\right).$$

  (12)
• Li similarity ($si{m}_{li}$) [44] is based on the shortest path distance between the compared concepts and the depth of their $lcs$, according to Equation (13)

  $$si{m}_{li}(c_1,c_2)=e^{\alpha \ast len(c_1,c_2)}\ast \frac{e^{\beta \ast depth\left(lcs(c_1,c_2)\right)}-e^{-\beta \ast depth\left(lcs(c_1,c_2)\right)}}{e^{\beta \ast depth\left(lcs(c_1,c_2)\right)}+e^{-\beta \ast depth\left(lcs(c_1,c_2)\right)}}$$

  (13)

  where e is Euler’s number, and $\alpha$, $\beta$ are parameters that contribute to the path length and depth, respectively. According to the experiment in [44], the empirical optimal parameters are $\alpha =0.2$ and $\beta =0.6$.
• Al-Mubaid similarity [45] (see Equation (14)) is based on the height of the taxonomy, the depth of the lcs, and the shortest path length:

  $$si{m}_{almubaid}(c_1,c_2)=log({(len(c_1,c_2)-1)}^{\alpha }\ast (CSpec{(c_1,c_2)}^{\beta }+k)$$

  (14)

  where $CSpec(c_1,c_2)=height\left(T\right)-depth\left(lcs(c_1,c_2)\right)$, $\alpha$, $\beta$ > 0, and k ≥ 0 (in their experiments k = 1).
• Rezaei and Fränti similarity [46] is based on the depth of the compared concepts and of their lcs, as shown in Equation (15):

  $$si{m}_{r\&f}(c_1,c_2)=\frac{2\ast depth\left(lcs(c_1,c_2)\right)}{depth\left(c_1\right)+depth\left(c_2\right)}$$

  (15)

  which, in a tree-shaped taxonomy coincides, with the Wu and Palmer similarity (see Equation (11)).
• Zhang 2 similarity [47] is based on the hyponyms of the concepts and their $lcs$, i.e., the $desc$ function according to the notation above. In particular, such a measure is computed according to Equation (16):

  $$si{m}_{zhang\_2}(c_1,c_2)=\frac{2\ast log\left(\right|C\left|\right)-log\left(\right|hypo\left(lcs(c_1,c_2)\right)|+1)}{2\ast log\left(\right|C\left|\right)-log\left(\right|hypo\left(c_1\right)|+1)-log\left(\right|hypo\left(c_2\right)|+1)}.$$

  (16)

Furthermore, the method proposed in [48] relies on the Wikipedia Category Graph (WCG), a hierarchical knowledge structure used to categorize Wikipedia articles. However, the WCG is not a proper taxonomy as it is built by volunteers who link the categories without explicitly specifying the nature of the relation. For this reason, the method is not considered in this review.

Table 3. Methods for computing the semantic similarity between concepts in a taxonomy.

		Depth	Height	len	lcs	Hypo	Hyper	IC	Further Info
$si{m}_{rada}$ [41]	1989			X					distance
$si{m}_{w\&p}$ [42]	1994	X		X	X
$si{m}_{res}$ [17]	1995				X		X	Resnik [17]
$si{m}_{j\&c}$ [34]	1997				X			Resnik [17]
$si{m}_{l\&c}$ [43]	1998		X	X
$si{m}_{lin}$ [35]	1998				X			Resnik [17]
$si{m}_{li}$ [49]	2006	X		X	X
$si{m}_{almubaid}$ [45]	2006	X	X	X	X
$si{m}_{p\&s}$ [36]	2009				X			Seco et al. [20]
$si{m}_{m\&z}$ [24]	2012			X	X			Seco et al. [20]	tuning param.
$si{m}_{r\&f}$ [46]	2014	X			X
$si{m}_{wpath}$ [38]	2017			X	X			Resnik [17]	tuning param.
$si{m}_{zhang\_1}$ [33]	2018				X			Zhang 1 et al. [33]
$si{m}_{zhang\_2}$ [47]	2018				X	X
$si{m}_{D_k}$ [39]	2021							any	tuning param. words senses
$si{m}_{HSS}$ [40]	2022				X	X		Resnik [17]	words senses

recaps the main features of the recalled proposals and the methods they use for computing the IC that are described in detail in Section 6.

4. Methods for Computing Semantic Similarity between Sets of Concepts

In the literature, traditionally, the semantic similarity between sets of concepts is evaluated according to the well-known Tversky model [37], such as Dice [50], Jaccard [51], and Sigmoid [52], just to mention a few. However, these are set-theoretic methods that are not addressed in this review as they do not rely on a given taxonomy of concepts. With regard to the taxonomy-based similarity measures, in this section, we recall five methods: the WNSim similarity [53], the measures introduced by Rezaei and Fränti [46], Haase et al. [54], and Wang et al. [55], and the SemSim${}^p$ similarity [18].

Let an annotation vector, $av$, be a set of concepts from the taxonomy $T=(C,E)$, defined as $av=(c_1,..,c_n),c_i\in C,i=1,..,n.$

In the following, let $a{v}_1$, $a{v}_2$ be two annotation vectors from the taxonomy $T=(C,E)$.

• WNSim similarity [53] is a method for computing semantic similarity between sets of concepts representing sentences in documents, which leverages the Leacock and Chodorow similarity $si{m}_{l\&c}$ [43] defined in the previous section (see Equation (17)):

  $$SI{M}_{WN}(a{v}_1,a{v}_2)=\frac{{\displaystyle \sum _{c_i\in a{v}_1}ma{x}_{c_j\in a{v}_2}\left(si{m}_{l\&c}(c_i,c_j)\right)\ast IDF\left(c_i\right)}}{{\displaystyle \sum _{c_i\in a{v}_1}IDF\left(c_i\right)}}.$$

  (17)
• Rezaei and Fränti similarity [46] is a similarity measure between sets of concepts based on matching the individual concepts of the sets by applying the $si{m}_{r\&f}$ defined above (see Equation (15)):

  $$SI{M}_{R\&F}(a{v}_1,a{v}_2)=\frac{{\displaystyle \sum _{c_i\in a{v}_1}si{m}_{r\&f}(c_i,c_j)}}{|a{v}_1|}$$

  (18)

  where $|a{v}_1|$ is the cardinality of the set $a{v}_1$.
• Haase et al. similarity [54] computes the similarity of pairs of concepts belonging to different sets by using the method proposed by Li et al. $si{m}_{li}$ recalled above, which combines the shortest path length between the concepts and the depths of their subsumers in the taxonomy non-linearly (see Equation (19)):

  $$SI{M}_{Haase}(a{v}_1,a{v}_2)=\frac{1}{|a{v}_1|}\sum _{c_i\in a{v}_1}ma{x}_{c_j\in a{v}_2}S(c_i,c_j)$$

  (19)

  where $|a{v}_1|$ is the cardinality of the set $a{v}_1$, and

  $$S(c_i,c_j)=\left\{\begin{array}{ccc}si{m}_{li}(c_i,c_j)\hfill & & \mathrm{if}{c}_i\ne c_j\hfill \\ 1\hfill & & \mathrm{otherwise}.\hfill \end{array}\right.$$
• Wang et al. similarity [55] computes the similarity between two sets of concepts $a{v}_1$ and $a{v}_2$ by considering, for each pair of concepts, one from $a{v}_1$ and one from $a{v}_2$, the IC of their lcs (see Equation (20)):

  $$SI{M}_{Wang}(a{v}_1,a{v}_2)=\frac{1}{|a{v}_1|\ast |a{v}_2|}\sum _{c_i\in a{v}_1}\sum _{c_j\in a{v}_2}ic\left(lcs(c_i,c_j)\right)$$

  (20)

  where the IC is computed according to the Resnik IC (see Section 2).
• SemSim${}^p$ similarity [18] is derived from the SemSim method [27], which has been conceived for evaluating the semantic similarity of resources (i.e., real-world entities) annotated by sets of concepts taken from a taxonomy. It is a parametric measure depending on a weight associated with the concepts of the taxonomy, and a normalization factor is used when the two compared annotation vectors have different cardinalities.

Our latest experimentation, which is shown in Section 6, confirms the results of the comparative assessment presented in [18], i.e., SemSim${}^p$, which, when configured according to a specific selection of parameters, improves the performance of the four methods mentioned above. For this reason, it is recalled in the next section in detail.

5. The Parametric Semantic Similarity Method SemSim${}^{\mathit{p}}$

Given two annotation vectors, the SemSim${}^p$ method [18] allows the evaluation of their semantic similarity degree by relying on the method $SemSim$ [27]. SemSim${}^p$ is based on two parametric functions, $si{m}_{lin,h}$ and $semsi{m}_{h,\mu }$, the former used to compute the similarity of pairs of concepts, whereas the latter was conceived to evaluate the similarity of pairs of annotation vectors, as formally defined below.

Let $si{m}_{lin,h}$ be the parametric Lin similarity defined by Equation (3) (see Equation (21))

$${\displaystyle si{m}_{lin,h}(c_1,c_2)=\frac{2\ast i{c}_h\left(lcs(c_1,c_2)\right)}{i{c}_h\left(c_1\right)+i{c}_h\left(c_2\right)}}$$

(21)

where $i{c}_h$ is the IC computed according to one among the approaches of Resnik, Annotation Frequency, Top Down, and Seco, as recalled in Section 2, which are formally defined in Appendix A (in particular h = {resnik, af, td, seco}, Formulas (A1), (A9), (A14) and (A2), respectively).

Consider now two annotation vectors, say $a{v}_1=(c_{11},\dots ,c_{1n})$ and $a{v}_2=(c_{21},\dots ,c_{2m})$ and the Cartesian product of $a{v}_1$, and $a{v}_2$, say S = $a{v}_1\times a{v}_2$. We borrow the matching approach from the graph theory in line with the maximum weighted matching problem in bipartite graphs [56]. Accordingly, $\mathcal{P}$$(a{v}_1,a{v}_2)$ is the set of sets of pairs, defined as follows:

$$\begin{array}{c}\mathcal{P}(a{v}_1,a{v}_2)=\{P\subset S|\forall ({\mathit{c}}_{1i},{\mathit{c}}_{2j}),({\mathit{c}}_{1q},{\mathit{c}}_{2k})\in P,{\mathit{c}}_{1i}\ne {\mathit{c}}_{1q},{\mathit{c}}_{2j}\ne {\mathit{c}}_{2k},\left|P\right|=min\{n,m\}\}.\end{array}$$

(22)

Formally, the $semsi{m}_{h,\mu }$ function identifies the set of pairs of concepts of $a{v}_1$ and $a{v}_2$ that maximizes the sum of the $consi{m}_h$ values, as follows:

$$\begin{array}{c}{\displaystyle semsi{m}_{h,\mu }(a{v}_1,a{v}_2)=\frac{\underset{P\in \mathcal{P}(a{v}_1,a{v}_2)}{max}\left\{{\displaystyle \sum _{({\mathit{c}}_{1i},{\mathit{c}}_{2j})\in P}}si{m}_{lin,h}({\mathit{c}}_{1i},{\mathit{c}}_{2j})\right\}}{\mu (n,m)}}\hfill \end{array}$$

(23)

where $\mu$, named the similarity normalization factor, is defined below:

$$\mu (n,m)=\left\{\begin{array}{c}\mathit{max}(\mathit{n},\mathit{m})\hfill \\ \mathit{min}(\mathit{n},\mathit{m})\hfill \\ \mathit{ave}(\mathit{n},\mathit{m})=\frac{n+m}{2}\hfill & \left(\mathit{arithmetic}\mathit{aver}.\right)\hfill \\ \mathit{gav}(\mathit{n},\mathit{m})=\sqrt{nm}\hfill & \left(\mathit{geometric}\mathit{aver}.\right)\hfill \end{array}\right.$$

(24)

In the following, the rationale for the choice of similarity normalization factor is briefly explained.

When calculating the degree of similarity of the two sets $a{v}_1$ and $a{v}_2$, composed of $n_1$ and $n_2$ concepts, respectively, two cases can be distinguished: either the two annotation vectors have the same cardinality or different cardinalities.

In the former case, i.e., $n_1$ = $n_2$, each concept in $a{v}_1$ can be matched with one concept in $a{v}_2$ and vice-versa. Hence, the four options lead to the same normalization factor, and the degree of similarity is calculated by considering the entire semantic description of both resources. In the latter case, assuming for instance $n_1>n_2$, part of the information about $a{v}_1$ (i.e., $n_1-n_2$ concepts) is ignored when computing the similarity value. The effects of the four proposed normalization factors are illustrated below when $n_1\ne n_2$, for instance with $n_1>n_2$. In the case the normalization factor is chosen as the maximum between $n_1$ and $n_2$, that is $n_1$, we intend to favor richer annotations, and thus the “missing information” in $a{v}_2$ weakens the similarity between the resources. If the normalization factor is chosen as the minimum between $n_1$ and $n_2$, i.e., $n_2$, we assume that a more “compact” annotation vector contains the essence of the resource $r_1$ and that the remaining concepts are redundant.

In particular, the choice of the normalization factor as the maximum considers the missing information (in the shorter annotation) as a deficiency. Conversely, the choice of the normalization factor as the minimum deems the additional information (in the longer annotation) as a redundancy. Hence, the choice of maximum or minimum emphasizes the differences and commonalities between the compared annotation vectors, respectively.

The choice of the normalization factor as the arithmetic mean implies a compromise between the two previous cases. This case takes missing and redundant information into account to some extent when comparing resources in the same way. Finally, the geometric mean behaves essentially like the arithmetic mean but is more sensitive to small values.

In accordance with the computational complexity of the Hungarian algorithm, the SemSim${}^p$ method is polynomial in the size of n, specifically $O\left(n^3\right)$, where n is the cardinality of the larger of $a{v}_1$ and $a{v}_2$.

6. Discussion

In Section 3, the methods for computing the semantic similarity between concepts in a taxonomy have been introduced and summarized in Table 3. They have been partitioned into methods relying on the IC and methods that do not use the IC.

All the methods based on the IC for computing similarity between concepts consider the least common subsumer (lcs) of the compared concepts. In particular, they consider the IC of the lcs, i.e., the amount of informativeness shared by the two concepts. In fact, in $si{m}_{res}$, the contributing hypernym is also the lcs. In the $D_k$ similarity, the $lcs$ does not appear explicitly in the formula, but since the method uses an IC-based function, it exploits this feature, indirectly. Therefore, the IC of the lcs can be recognized as the characteristic typical of these methods.

Besides the Meng and Zhou similarity [24] and the wpath similarity [38], which address the len of the compared concepts, and the hierarchical semantic similarity [40], which considers hyponyms, the other IC-based methods do not use further features. This can be explained because, indeed, the properties of the taxonomy are used in the computation of the IC.

Furthermore, it is worth mentioning that the Lin similarity [35] has inspired at least two other methods, which are the Meng and Zhou similarity [24] and the Zhang 1 similarity [33]. In fact, the former, according to Equation (5), is Lin’s formulation multiplied by an expression that is inversely proportional to the len between the compared concepts, whereas the latter, according to Equation (7) is Lin’s formulation re-scaled by the logarithm function.

Analogously, the methods that are not based on the IC notion mainly consider the lcs between the compared concepts, even if in different manners. In particular, the Wu and Palmer similarity [42], Li similarity [49], Al-Mubaid similarity [45], and Rezaei and Fränti similarity [46] consider the depth of the lcs, whereas the Zhang 2 similarity [47] considers the number of hyponyms of the lcs.

The Rezaei and Fränti similarity can be considered as the non-IC version of the Lin similarity. In fact, the former resembles the latter but considers the depth instead of the IC. Furthermore, in the case the taxonomy is a tree, this similarity metric is equivalent to the Wu and Palmer similarity. In fact, in a tree-shaped taxonomy, given two concepts $c_1$ and $c_2$, $depth\left(c_1\right)$ and $depth\left(c_2\right)$ are equal to $depth\left(lcs(c_1,c_2)\right)+len(c_1,lcs(c_1,c_2))$ and $depth\left(lcs(c_1,c_2)\right)+len(c_2,lcs(c_1,c_2))$, respectively.

In Section 4, some of the most representative methods for computing taxonomy-based semantic similarity between sets of concepts were illustrated. In particular, WNSim [53], the method defined by Rezaei and Fränti [46], the one introduced by Haase et al. [54], the measure proposed by Wang et al. [55], and SemSim${}^p$ [18].

Figure 1 shows the dependencies among the recalled methods. In the following, the experimentation presented in [18] is briefly recalled and updated by including the latest results, as explained below.

The evaluation has been performed by carrying out two experiments. They are based on a taxonomy derived from the ACM Computing Classification System (CCS) and a large dataset of about $1K$ articles from the ACM Digital Library, each associated with an annotation vector of concepts from such a taxonomy.

In general, the semantic similarity between concepts is evaluated by asking a group of people to express similarity values between pairs of concepts belonging to golden datasets (such as for instance M&C [57], R122 [58], and R&G [59]), which represent benchmarks in the evaluation and comparison of different similarity methods. However, there is no golden dataset that includes similarity scores for any possible pair of concepts defined in the ACM domain, and it is unrealistic to ask a group of people to check thousands of annotation vectors in pairs, producing millions of similarity scores. Therefore, the key idea of the experimentation was to use some special issues of the ACM as a benchmark because they contain articles whose semantic similarity, on average, should be greater than that of a randomly selected set of papers. In fact, such articles are gathered by the editor according to the research topic indicated in the call for papers. Hence, on the one hand, we performed traditional experimentation based on expert evaluation, in which we computed Pearson’s correlation between the compared methods and human judgment (HJ). On the other hand, we performed a statistical analysis in which we computed the degree of confidence in detecting the semantic cohesion (i.e., the average mutual similarity) of papers belonging to some special issues [18]. In particular, SemSim${}^p$ has been assessed by contrasting it against six of the most popular similarity methods for comparing sets of concepts. These methods were organized according to two groups. The first group relies on set-theoretic methods, i.e., which allow the similarity scores to be derived by applying set-theoretic operations on the annotation vectors, and contains Dice [50], Jaccard [51], and Sigmoid [52]. The second group includes the taxonomy-based methods recalled in the previous sections, namely WNSim [53], Rezaei and Fränti [46], and Haase et al. [54]. According to the scope of the present manuscript, here we considered the second group enriched with the recent Wang et al. similarity method [55], which was not considered in [18].

The results of the experiments are presented in

Table 4. Methods for computing semantic similarity between sets of concepts: experimental results.

	Correlation with HJ	Degree of Confidence
WNSim similarity	0.61	84.33%
Rezaei and Fränti similarity	0.46	74.73%
Haase et al. similarity	0.69	77.89%
Wang et al. similarity	0.04	74.09%
SemSim${}^p$ similarity	0.69	85.67%

where, for each method, the column labeled correlation with HJ refers to the average correlation between the method and human judgment, whereas the column labeled degree of confidence refers to the results of the statistical experiment. In line with [18], the results of the latest experiment show that SemSim${}^p$, when the parameter $h=af$ (i.e., in the case of the annotation frequency IC) and $\mu =gav$ (i.e., the geometric average similarity normalization factor), exhibits the best performance with respect to the other above-mentioned methods for evaluating the semantic similarity between sets of concepts.

7. Conclusions and Future Directions

This review shows the methods for computing the semantic similarity between either concepts or sets of concepts belonging to a taxonomy and the crucial role of the notion of IC in this activity. The main goals of this contribution are: (i) presenting all the selected methods in a homogeneous manner with the help of basic definitions shared by all the methods; (ii) identifying the main features used by each method; (iii) showing commonalities and differences among the methods; (iv) identifying dependencies among the methods, in particular among those focusing on sets of concepts and those focusing on pairs of concepts, and among the latter and the methods for computing the IC. Furthermore, concerning the methods working on sets of concepts, the previous experiment presented in [18] has been enriched by including a recent additional method (Wang et al. similarity method [55]), in order to provide an updated comparative assessment. The new experiment shows that, among the methods for computing semantic similarity between sets of concepts, SemSim${}^p$ outperforms the other approaches, in line with the previous results [18].

One interesting future direction is represented by the machine learning techniques dealing with graph-shaped knowledge. However, in this research area, computational efficiency is still an open problem because the computation of embedding is time-consuming, and experiments, even with small Resource Description Framework (RDF) datasets, do not terminate in a reasonable number of days or run out of memory.