An Inductive Logical Model with Exceptional Information for Error Detection and Correction in Large Knowledge Bases

Abstract

Some knowledge bases (KBs) extracted from Wikipedia articles can achieve very high average precision values (over 95% in DBpedia). However, subtle mistakes including inconsistencies, outliers, and erroneous relations are usually ignored in the construction of KBs by extraction rules. Automatic detection and correction of these subtle errors is important for improving the quality of KBs. In this paper, an inductive logic programming with exceptional information (EILP) is proposed to automatically detect errors in large knowledge bases (KBs). EILP leverages the exceptional information problems that are ignored in conventional rule-learning algorithms such as inductive logic programming (ILP). Furthermore, an inductive logical correction method with exceptional features (EILC) is proposed to automatically correct these mistakes by learning a set of correction rules with exceptional features, in which respective metrics are provided to validate the revised triples. The experimental results demonstrate the effectiveness of EILP and EILC in detecting and repairing large knowledge bases, respectively.

1. Introduction

The application of domain-oriented knowledge bases (KBs) is a prevalent research topic in Artificial Intelligence. KBs serve various downstream tasks, such as hierarchical reasoning [1], question answering [2], and information retrieval [3]. Many semantic KBs, for example, YAGO [4], DBpedia [5], Wikidata [6], or the Google Knowledge Graph [7], have been constructed from the knowledge in the Web and have wide applications. In the real world, KBs are multirelational, heterogeneous, semistructured, and full of uncertainty and incompleteness [8,9], which leads to inconsistencies, outliers, and erroneous relations. Hence, many algorithms have been proposed to detect errors in KBs [10,11,12]. Among them, rule learning algorithms are commonly used due to their strong interpretability of data [13,14,15,16]. A popular algorithm that falls into this category is inductive logic programming (ILP) [17,18], which learns the hypothesis with background knowledge to cover all positive examples [19]. A deficiency of ILP is that it neglects two important factors in error detection processes: (a) negative statements and (b) abnormal information. To utilize negative statements to improve the accuracy of error detection in KBs, Wu et al. [20] proposed guided inductive logic programming (GILP), which detects systematic extraction of mistaken problems by positive and negative feedback with maximum consideration of the characteristics of misinformation. In terms of utilizing abnormal information, Mohamed et al. [21] introduced the exception-enriched rule learning model and improved the quality of a set of rules by repairing Horn rules by adding negative atoms to bodies. In the present paper, we propose a new error detection algorithm, the inductive logic programming algorithm with exceptional information (EILP), which considers both negative statements and abnormal information in error detection, combining the advantages of the two methods mentioned above.

After detecting errors in KBs, a correction algorithm is needed to improve the quality of the knowledge bases [22]. Many efforts have been made in this area. For example, Martin et al. [23] employed nonembedding methods to resolve temporal conflicts in inconsistent Resource Description Framework (RDF) knowledge bases. Mohamed Yakout [24] proposed a Scalable Automatic Repairing (SCARE) system to correct the error of the data according to the statistical likelihood. Theodoros et al. [25] introduced a HoloClean framework for holistic data repairs driven by probabilistic inference. Lertvittayakumjorn [26] proposed the keyword method to correct range violation errors of DBpedia. Ortona designed the system of rule discovery in knowledge bases (RuDiK) [27] to find declarative rules over KBs, including inequality rules to expand associated rules with measures. Mohammad Mahdavi [28] proposed a Baran system to fix data errors with respect to their value, vicinity, and domain contexts. Furthermore, Abedini [29] proposed a new Correction Tower to analyze popular embedding methods. The authors of the present paper proposed the guided inductive logic learning model (GILLearn) in previous work [30,31], which corrects the range errors of the triples by building correction rules after cleaning the KB. Nevertheless, all of the above-mentioned correction algorithms heavily rely on pre-configured paradigms and complete set of rules, leading to high computational cost and limits in the range of applications. To overcome this difficulty, as an extension of our previous work [30,31], we propose an inductive logic correction algorithm with exceptional information, coined “EILC”, to correct negative knowledge based on exception features in large KBs, in which a new search space and a new rule refining algorithm are applied to mine rules and repair knowledge. In addition, cross-semantic similarity metrics based on semantic similarity and source-related contents are applied to annotate and refine triples in KBs. However, several quality issues persist in knowledge bases (KBs), such as anomalous triples, type inconsistencies, and abnormal relation patterns within RDF knowledge bases. Our model is specifically designed to detect and correct these identified error patterns.

Hence, the novelty of our work contains the following two aspects:

• The inductive logical programming algorithm with exceptional information (EILP) is proposed to detect errors in KBs by considering both negative statements and abnormal information;
• The inductive logical correction method with exceptional features (EILC) is proposed to correct errors, in which a new rule refining algorithm is applied to revise correction rules.

The remainder of this paper is organized as follows. In Section 2, some preliminary concepts are introduced. Section 3 highlights the EILP and EILC algorithms. Section 4 reports our main experimental results and the analysis of our model. The conclusion is presented in Section 5.

2. Preliminaries

For an understanding of our proposed methods, some basic notation is explained in this section. Here, three types of erroneous content are explained in the problem statement.

2.1. Problem Statement

Knowledge bases often exhibit quality issues, deriving from sincere mistakes or errors in crowd-sourced content [32]. Zaveri et al. [33] found that 12% of DBpedia triples have some type of problem. In our model, problem statements suffer from three types of erroneous content.

Inconsistencies of Type. A major error is inconsistency of knowledge. An erroneous tail of a triple is common. Correcting incorrect tails requires the range constraint of a given relation. For example, triples (<$e_i$, nationality, $e_j$>) have wrong range types, such as EthnicGroup, Language, and Agent. Most work has used traditional rule extraction metrics [34] to achieve high confidence in inconsistency problems, where triples breaking constraints with the meaning of “unusual” are viewed as incorrect in the data cleaning system. The common sense behind these learning methods is that “unusual means errors”. Therefore, the method of “unusual equals incorrect” cannot be executed within subdomains of inconsistencies in KB, such as functional dependencies [15], denial constraints, and Horn clauses [35]. For these problems, common rule learning approaches perform only the work of data cleaning; these methods are rarely used in knowledge base correction algorithms.

Triple outliers in associated KBs. The process of knowledge creation results in errors in information extraction, and there are subtle discrepancies in the same core of knowledge sources, like Wikidata, DBpedia, and YAGO. In this study, triples corrected for similar entities are investigated in structures with cooccurence facts. The property (owl: sameAs, owl: http://www.w3.org/2002/07/owl, accessed on 1 May 2022) is leveraged to enhance the precision of original KBs. The interrelations of KBs have domain or range restrictions, and the candidates violating them are not estimated in the system. Here, exceptional triples are those facts in a KB that deviate from the entities’ cooccurrence in associated KBs. For this condition, there are common connections between the entities involved with the same background information. The quality of extracted information reaches a new stage where integration into KBs should be achieved with filtering or correcting in further work.

Vicinity of erroneous relations. There are two main reasons for the erroneous relations in KBs. First, extracted resources, such as Wikipedia articles, may contain falsified relations. Extracting knowledge from unstructured or semistructured KBs is error prone in the vast amount of linked data available. Second, some erroneous relations are added to KBs after knowledge construction. The neighborhood of erroneous relations has some implicit information for the purpose of data correction. Implicit knowledge has been recovered by internal correlations of triples [26].

For example, the logic relations of France + Berlin − Germany = Paris and France + Germans − Germany = French in a KB have the same confidence scores. In nonembedding approaches, direct logic relations can be utilized in process refinement and enrichment, ignoring the condition of samples. For false triples <?a, dbr:nationality, Germans>, the right result is probed in the second equation. Traditional algorithms do not fix the erroneous value Germans in the type $Country$ with the entity value. The item itself has the right constraint on its type of $EthnicGroup$. When checking its neighboring triple attributes with the same type, Germany as the type of $Country$ is regarded the tail of the triple. Finally, mistaken triples can be revised with their vicinity in the KB. For brevity, the system corrects problems that ABox property assertions of triple with mistaken type. Here, entity assertions with erroneous tails are mined by rules and presented with simple canonicalization in our model. In particular, some simple canonicalizations are acquired in the examples; e.g., the mistaken assertion <dbr:Rodrigo_Kirchner, dbo:nationality, dbr:German> (dbr: http://dbpedia.org/resource/, accessed on 1 May 2025); (dbo: http://dbpedia.org/ontology/, accessed on 1 May 2025) should be corrected to <dbr:Rodrigo_Kirchner, dbo:nationality, dbr:Germany>. The dbpedia knowledge base is open source and available online. We can query it through the URL https://dbpedia.org/sparql and it can be accessed until 28 May 2025.

Triple: RDF is chosen as a data model to transform web facts [36]. Knowledge bases are made up of a series of RDF information called triples. For example, the sentence (Beijing is capital of China) is translated as <Beijing, isCapitalOf, China>. A three-tuple consists of $\{<sub,pre,obj>\parallel sub,obj\in \xi ,pre\in \mathcal{K}\}$, where $sub$ and $obj$ present entities of the subject and object, respectively, and $pre$ is the predicate name in the knowledge base $\mathcal{K}$.

First order logic rules [37]: We focus on first-order rules with a single atom of head and two predicates of body. The number of variables contained in the closed rule is even, and the variables of the head exist in the body of rules. The most ideal rule satisfies the condition $pr{e}_1(?a,?b),pr{e}_2(?b,?c)->pr{e}_3(?a,?c)$.

2.2. Search Space of Feedback

The background schema KB and TBox is then incorporated into the constraints taxonomy, where a set of tuples is regarded a knowledge base $\mathcal{K}$. A triple is denoted by $R\left(\overline{t}\right)\in \mathcal{K}$ from the whole $\mathcal{K}(R,\overline{t})$. Some training examples are selected to filter feedback $\mathcal{F}$ with positive/negative information based on expert knowledge or external features. Feedback $\mathcal{F}$ is either positive or negative for each triple $R\left(\overline{t}\right)$. Therefore, the two groups ${\mathcal{F}}^{+}$ or ${\mathcal{F}}^{-}\subseteq \mathcal{K}$ involve all triples receiving correct or incorrect $\mathcal{F}$, respectively. The other two sets, ${\mathcal{T}}^{+}$ and ${\mathcal{T}}^{-}\subseteq \mathcal{K}$, consist of all facts, which are detected to distinguish positive or negative features by consistency constraints, explained in rules. The main target of assignments for extracting $\mathcal{F}$ is to learn Horn clauses with exception by small samples of labels ${\mathcal{F}}^{+}$ or ${\mathcal{F}}^{-}$ while developing ${\mathcal{T}}^{+}$ or ${\mathcal{T}}^{-}$ over a swarm of triples in $\mathcal{K}$. Four conditions are kept in the KBs: ${\mathcal{F}}^{+}\cap {\mathcal{F}}^{-}=Ø$, ${\mathcal{F}}^{+}\cup {\mathcal{F}}^{-}=\mathcal{F}$, ${\mathcal{T}}^{+}\cap {\mathcal{T}}^{-}=Ø$, and ${\mathcal{T}}^{+}\cup {\mathcal{T}}^{-}=\mathcal{T}$.

A new search space is proposed to profile first-order rules for the following steps. $\mathcal{K}$, ${\mathcal{F}}^{+}$, ${\mathcal{F}}^{-}$, ${\mathcal{T}}^{+}$, and ${\mathcal{T}}^{-}$ are expressed as set predicates, and ${\mathcal{T}}^{+}\left(\overline{t}\right)\wedge R\left(\overline{t}\right)$ are served as ${\mathcal{T}}^{+}(R,\overline{t})$. Other predicates follow a similar structure. Inspired by the Markov logic network [38], each triple expands all linked information with six conditions of paths. A complete closed clique is constructed, and the weights are defined by numbers of logic formulations. For the relationship between x, y, and z, we can use different colors and dotted and solid arrows to represent the six logical relationships between them. Arrows of the same color represent one logical relationship, the solid line represents the outer circle logical relationship between x, y, and z, and the dotted line represents the inner circle logic. The feedback is similar to that of the original atomic formula, as shown in Figure 1. The new search space deletes some redundant information and concentrates on fixed predicate names in the knowledge base.

2.3. Quality Measures

Cross-similarity measures are used to validate repairs of erroneous triples in KBs. After our model operations, some mistaken assertions are matched with multiple values in the repairs process. Here, a new cross-similarity measure is proposed to analyze the final revised assertions of triples in KBs, aiming to discover common features between the original entities and the repairs after correction. In Equation (2), the distance Jaro–Winkler [39] is suitable to calculate the similarity between short strings such as names, where $d_j$ is the similarity of the Jaro–Winkler strings between $e_0$ and $e_i$, m is the number of matched strings, and t is the number of transpositions. Then, $sim\_external(,)$ analyzes the external similarity probability, matching cooccurrence Wikipages in the ($wikiPageWikiLink$) property. $s(e_0,e_i)$ is a pair of compared objects. A new cross-function, $f_{cross}$, in Equation (3) is the harmonic mean of distance and external similarity, which is designed to cover all correlations of assertions and candidate repairs.

$$sim\_external(e_0,e_i)=(P_{e_0}\cap P_{e_i})/P_{e_0}$$

(1)

$$d_j={\displaystyle \frac{1}{3}}\left({\displaystyle \frac{m}{\parallel e_0\parallel }}+{\displaystyle \frac{m}{\parallel e_i\parallel }}+{\displaystyle \frac{m-t}{m}}\right)$$

(2)

$$f_{cross}={\displaystyle \frac{2\times d_j\times sim\_external(e_0,e_i)}{d_j+sim\_external(e_0,e_i)}}$$

(3)

3. Proposed Methods

In this section, an inductive logical model with exceptional information is proposed to detect errors in large knowledge bases. For large KBs, our approaches extract some rules to detect these problems. Here, a new search space is applied to increase the speed of mining rules, described in Section 2.2. After finding erroneous knowledge, an inductive logical correction method is leveraged to correct these abnormal triples to refine KBs. Finally, some quality metrics are applied to validate the revised triples, illustrated in Section 2.3. On the whole, the core of the Inductive Logical model focuses on the algorithms of EILP and EILC. The main focus of our proposed method is the use with exceptional features of feedback and negative statements.

3.1. Overview of Inductive Logical Model

Figure 2 shows the architecture of the inductive logical model with exception, including EILP detection and the EILC algorithm. The core of the framework focuses on the modules of error detection and knowledge correction. The guidance of exceptional features is offered to update the quality of the rules in our model. The EILP algorithm acquires a higher quality of rules in order to detect more mistakes in the knowledge base. Subsequently, the EILC correction algorithm is implemented using correction rules to refine the large knowledge base, which pulls some corrected facts for the KB cleaning. The top k candidate assertions are filtered from EILC to repair the wrong features in triples. Repairs are generated from EILC, and new measures filter the best corrected triples to clean the knowledge base.

Repair similarity can be validated through various approaches [40], and multiple algorithms are available. Here, the new methods of cross-similarity of candidates containing similar words are renovated to determine the optimal correction result, as shown in Section 2.3. The incorrect range assertions of triples can be discovered automatically by EILP. A set of triples are viewed as negative by their range type in the TBox property. For a triple $<sub,pre,obj>$ in the community, the proposed correction module notices an atom from K as an object substitution. The revised entity e is semantically correlated with $obj$, and the triple $<sub,pre,e>$ is repaired. A new candidate entity assertion remains instead of erroneous objects in open KBs. For example, the fact nationality(Einstein, German) is an incorrect triple, and the erroneous German is switched with the revised atom Germany. Here, nationality can serve as a predicate name in the DBpedia training sets.

3.2. EILP

The detection part includes nonmonotonic inductive logic programming (NILP) and inductive logic programming with exceptional information (EILP), as described in the following. We wish to learn rules with high quality to explain consistency constraints in KBs. Constraints are mined by positive and negative feedback from relational first-order literals or rules. In the initial step, feedback extracted from a given predicate name is utilized to mine seed rules where, to increase the confidence of rules, those with exceptions are proposed to cover more feedback in the process of knowledge extraction.

Nonmonotonic Inductive Logic Programming. We consider traditional logic programs in definition [41] under knowledge semantics. NILP uses a set of rules,

$$\forall x_1\dots \forall x_k\left(\phi \left(\overline{X}\right),not\_E\to \psi \left(\overline{X}\right)\right),$$

(4)

where $\psi \left(\overline{X}\right)$ signifies the first-order atom of $\overline{X}$, acting as $head\left(r\right)$. $\phi \left(\overline{X}\right)$ is a conjunction of $\phi \left(\overline{x_1}\right),\dots ,\phi \left(\overline{x_k}\right)$ with positive atoms, called $bod{y}^{+}\left(r\right)$. Then, not_E shows a conjunction of not_$\phi \left(\overline{x_{k+1}}\right),\dots ,not\_\phi \left(\overline{x_n}\right)$, where $not\_$ symbolizes negation as failure (NAF) or default negation [42]. Consequently, $bod{y}^{-}\left(r\right)$ is denoted as the negative part of the bodies. When $bod{y}^{-}\left(r\right)=Ø$, the rule r without negative examples is positive.

EILP is inspired by GILP [20] and NILP. With the exception of guidance, GILP is developed and applied in the learning of consistency constraints. A relational atom is $\mathcal{Q}(P,\overline{X})$, where $\mathcal{Q}$ is one of the set predicates$\mathcal{K}$, ${\mathcal{F}}^{+}$, ${\mathcal{F}}^{-}$, ${\mathcal{T}}^{+}$, or ${\mathcal{T}}^{-}$, and $\overline{X}$ is a vector of constants or variables. P is a given relation in KBs. So, ${\mathcal{T}}^{+}(P,\overline{X})$ is an atom, meaning that all constants in $\mathit{Var}\left(\overline{X}\right)$ are deduced to be correct from a rule, while ${\mathcal{T}}^{-}(P,\overline{X})$ expresses the collection of vandalized facts.

An EILP, indicated by $\varphi$, is a Horn clause of the formula

$$\begin{array}{c}\forall x_1\dots \forall x_k\left(\phi \left(\overline{X}\right)\to not\_E\wedge \psi \left(\overline{X}\right)\right)\hfill \\ \forall x_1\dots \forall x_k\left(E\wedge \phi \left(\overline{X}\right)\to \psi \left(\overline{X}\right)\right),\hfill \end{array}$$

(5)

where $\psi \left(\overline{X}\right)$ is a single atom referred to the concept of logic rule in Section 2.1 (First order logic rules [37]); $\phi \left(\overline{X}\right)$ is a conjunction of atoms; $x_1,\dots ,x_k$ are variables in $\mathit{Var}\left(\overline{X}\right)$; and E is the exception of the body.

In addition, by covering the correlational predicates in five set predicates, rule atoms are applied to discriminate facts. Some address facts and feedback from positive or negative facts in a KB can be clearly separated, and some deduced facts are deemed correct or incorrect by relational predicates. Because $\mathcal{F}$ is either correct or incorrect in our framework, there are two kinds of EILP rules.

Exclusive exception rules have the formula

$$\begin{array}{c}\forall x_1\dots \forall x_k\left({\phi }^{-}\left(\overline{X}\right)\wedge E^{-}\to {\mathcal{T}}^{-}(R,\overline{X})\right)\hfill \\ or:\forall x_1\dots \forall x_k\left({\phi }^{+}\left(\overline{X}\right)\to not\_E^{+}\wedge {\mathcal{T}}^{-}(R,\overline{X})\right)\hfill \end{array}$$

(6)

Inclusive exception rules have the formula

$$\begin{array}{c}\forall x_1\dots \forall x_k\left({\phi }^{+}\left(\overline{X}\right)\wedge E^{+}\to {\mathcal{T}}^{+}(R,\overline{X})\right)\hfill \\ or:\forall x_1\dots \forall x_k\left({\phi }^{-}\left(\overline{X}\right)\to not\_E^{-}\wedge {\mathcal{T}}^{+}(R,\overline{X})\right)\hfill \end{array}$$

(7)

where $x_1,\dots ,x_k$ are variables in $\mathit{Var}\left(\overline{X}\right)$; $E^{+}$ and $E^{-}$ are exceptions of positive and negative features, respectively; and $not$ indicates that rules do not contain these properties.

Example 1.

The following exclusive exception rule mentions that “The family names of all humans with USA nationality are correct”.

• $\mathcal{K}(\mathit{hasFamilyName},\langle x_1,y_1\rangle )\wedge$$\mathcal{K}(\mathit{hasNationality},\langle x_1,USA\rangle )$
• $\to {\mathcal{T}}^{+}(\mathit{hasFamilyName},\langle x_1,y_1\rangle )$
• $\mathcal{K}(\mathit{hasFamilyName},\langle x_1,y_1\rangle )\wedge$$\mathcal{K}(\mathit{hasNationality},\langle x_1,y_1\rangle )$
• $\to {\mathcal{T}}^{-}(\mathit{hasFamilyName},\langle x_1,y_1\rangle )$$\wedge not\_USA\left(y_1\right)$

Conversely, the below inclusive exception rule mentions that “The family names of all humans with Asian nationality (like China) are incorrect”.

• $\mathcal{K}(\mathit{hasFamilyName},\langle x_1,y_1\rangle )\wedge$$\mathcal{K}(\mathit{hasNationality},\langle x_1,y_1\rangle )$
• $\to {\mathcal{T}}^{+}(\mathit{hasFamilyName},\langle x_1,y_1\rangle )$$\wedge not\_China\left(y_1\right)$
• $\mathcal{K}(\mathit{hasFamilyName},\langle x_1,y_1\rangle )\wedge$$\mathcal{K}(\mathit{hasNationality},\langle x_1,China\rangle )$
• $\to {\mathcal{T}}^{-}(\mathit{hasFamilyName},\langle x_1,y_1\rangle )$

Example 1 shows that two rules contain positive information in the feedback of samples. For the whole knowledge base, some outliers, or exceptions, exist in any two rules. For a single first order Horn rule, it is impossible to contain all triples that meet the conditions of the body and head. The rule learning framework extracts sparsity exception information from feedback. In the GILP model [20], these positive or negative rules filter the same triples repeatedly in different rules, leading to redundancy of knowledge. Horn rules have better explanations for human understanding and great potential for the exploration of interesting negative statements in KBs. Compared to cooccurrence entities in other knowledge bases, some labeled properties are searched to distinguish negative statements in feedback. To automatically mine rule learning, some labels attached to samples are utilized to replace feedback in the process of pruning the search space.

Algorithm 1 describes EILP. Rule augmentation updates rule generation, adding negative atoms (exception) to bodies. A set of seedRules ${\Phi }_0$ are mined from initial knowledge feedback, and ${\mathcal{F}}_0$ is generated. The fact $\mathcal{T}$ is predicated by accepted rules ${\Phi }_{acc}$, and $\mathcal{T}$ is initially empty. Similarly, the iteratively merged global rule set $\Phi$ and $\mathcal{F}$ are extended from ${\Phi }_0$ and ${\mathcal{F}}_0$, respectively.

Algorithm 1 EILP

Require: ${\Phi }_{acc}$:= Ø; $\mathcal{T}$:= Ø; $\Phi$:= ${\Phi }_0$; $\mathcal{F}$:= ${\mathcal{F}}_0$ Ensure: ${\Phi }_{acc}\wedge E$   1: ${\Phi }_0:=$ generateSeedRules(${\mathcal{F}}_0$)   2: while $\Phi \ne Ø\mathrm{or}\mathcal{T}changed$ do   3:       ${\Phi }_i:=$ all rules in $\Phi$ which are accepted based on $Property$ in schema-level or cooccurring similar property in similar knowledge base   4:       ${\Phi }_{acc}\wedge E^{+}:=$ top-k Exceptional Rules from ${\Phi }_i$ by Exception   5:       ${\Phi }_{rej}\wedge E^{-}:=$ all rules in $\Phi$ which can be rejected by Exception   6:       ${\mathcal{T}}^{+}/{\mathcal{T}}^{-}:=$ positive/negative facts predicted by ${\Phi }_{acc}$ or ${\Phi }_{rej}$   7:       if each $\varphi \in \Phi$ then   8:             $\Phi :=\Phi \wedge E\cup$ ILP($\mathcal{F}$, $\mathcal{K}$)   9:             $\mathcal{F}:=\mathcal{F}\cup$ pullPrediction($\mathcal{T}$) 10:       end if 11: end while

At each rule learning iteration, all rules in $\Phi$ which are accepted on the basis of $TBox$ property in schema-level or cooccurring similar property in similar KB. Next, ${\Phi }_{acc}$ is filtered from the current $\Phi$ by exception in the $TBox$ property based on the top 10 exceptional rules. Here, the rank of confidence is leveraged to discover exceptional rules with higher confidence. Subsequently, the mean quantity of accepted rule sizes between iterations is utilized to settle the selection of K. Referring to embedding methods, K equals 10. Similarly, predicated facts $\mathcal{T}$ are developed by ${\Phi }_{acc}$ at the current loop. The EILP model wishes to learn by two mutually exclusive sets of feedback facts, $\mathcal{F}$ and $\mathcal{K}$. pullPrediction($\mathcal{T}$) fetches the next round of knowledge feedback as a randomly chosen subset of the currently predicted facts, $\mathcal{T}$. The rule learning loops are terminated either when no $\Phi$ can be created, or when $\mathcal{T}$ remains unchanged between two rounds.

The $TBox$ and $ABox$ properties are combined to perform feedback automatically in EILP. Then, the EILP model is expanded to iteratively learn consistency constraints from feedback. Here, exceptional information is added to the extraction process of rules. So, $\Phi$ with higher precision is mined for rule augmentation with Exception. The linked property is leveraged to replace the work that experts perform for the feedback. The rules with exceptions are constructed from automated feedback. As such, the module enhances the effectiveness of rule mining. In the correction module, the previous operation mines as much negative feedback as possible and improves a large number of correct features in positive rules. The highlight of EILP is leveraged to automatically select exceptional features from feedback. Some rough features or fuzzy rules are discarded after the application of exceptions or negative features.

3.3. EILC

To devise a comprehensive correction model, all triples are divided into negative and positive feedback. To repair a group of erroneous triples from a negative rule, an incorrect range constraint of triples is applied. EILC is shown as Algorithm 2.

Algorithm 2 EILC

Require: $({\varphi }_E^{+}(x,y),{\varphi }_E^{-}(x,y)):=$ EILP(${\mathcal{F}}_0,\mathcal{K}$) Ensure: $Cor{r}_n$ 1: $Cor{r}_0:=Ø$; 2: ${\mathcal{T}}^{-}:=$ incorrect facts predicted by ${\varphi }_n^{-}$ 3: $\left[\gamma \left(x\right)\right]:={\varphi }_E^{-}(x,y)->\xi (x,y,z)$; 4: while $\xi (x,y,z)\ne Ø$ do 5:       ${\varphi }_{Refine}:=({\varphi }_E^{+}(x,y),{\varphi }_E^{-}(x,y))->{\varphi }_{refine}$ 6:       $\mathcal{T}:=$ all incorrect or correct predictions in ${\varphi }_{refine}$ which are accepted by $Property$ in TBox 7:       $Cor{r}_i:=pullCorrections\left(filterRepairs\left(\mathcal{T}\right)\right)$ 8:       $\xi (x,y,z)$ remove ${\mathcal{T}}^{-}$ 9: end while

Here, Ø is empty in the initial step. Some negative facts are extracted from ${\varphi }_n^{-}$. Erroneous triples are generated by ${\varphi }^{-}(x,y)$ and kept in $\gamma \left(x\right)$. For errors in large KBs, new rules $({\varphi }_E^{+}(x,y),{\varphi }_E^{-}(x,y))$ are revised to ${\varphi }_{refine}$ to process negative knowledge. While negative triples are nonempty, all predictions in ${\varphi }_{refine}$ are accepted by $Property$ in $TBox$. The similarity measures are leveraged in the function of $filterRepairs$ (Section 2.3). $pullCorrections$ calculates the final repair similarity to obtain the final corrections. Iterations are terminated when no triples in $\xi (x,y,z)$ can initially be generated.

Rewriting the consistency constraints in Algorithm 3 as first-order rules over set predicates, all of the above classes of constraints are unified in a single rule. Some generated ${\varphi }_{refine}$ are utilized in EILC. The Refine Rules algorithm leverages their cooccurring variables to revise negative rules after adding positive features of positive rules to produce a set of possible repairs for erroneous knowledge in KBs. In Algorithm 3, $mapVariable$ is leveraged to form a union of conjunctive rewriting queries as correction rules ${\varphi }_{refine}$.

Algorithm 3 Refine Rules

Require: ${\varphi }^{+}(x,z),{\varphi }^{-}(x,y)$(positive/negative rules), x is variable; y, z are variables or constants; Ensure: ${\varphi }_{refine}(x,y,z)$: union of conjunctive rewriting queries;   1: ${\varphi }_{refine}:=Ø$;   2: for each rule $r^{-}\in {\varphi }^{-}(x,y)$ do   3:       $r^{-}\to hea{d}^{+}(subjec{t}^{+},objec{t}^{+})$;   4:       for each atom p(sub,obj) ∈ body of $r^{+}$ do   5:             if sub==$subjec{t}^{+}$ then   6:                    mapVariable(p(sub, obj), $subjec{t}^{+}$, $subjec{t}^{-}$) = $p^{\prime }$($subjec{t}^{-}$, obj);   7:             else   8:                    mapVariable(p(sub, obj), $subjec{t}^{+}$, $subjec{t}^{-}$) = $p^{\prime }(su{b}^{\prime },obj)$;   9:             end if 10:             if obj==$subjec{t}^{+}$ then 11:                    mapVariable(p(sub, obj), $subjec{t}^{+}$,$subjec{t}^{-}$) = $p^{\prime }(sub,subjec{t}^{-})$; 12:             else 13:                    mapVariable(p(sub, obj), $subjec{t}^{+}$,$subjec{t}^{-}$) = $p^{\prime }(sub,ob{j}^{\prime })$; 14:             end if 15:             $r_{refine}=r_{refine}\cap$ mapVariable(p(sub, obj), $subjec{t}^{+}$, $subjec{t}^{-}$); 16:       end for 17:       ${\varphi }_{refine}:={\varphi }_{refine}\cup r_{refine}$; 18: end for

$mapVariable$ reconciles incorrect or correct knowledge in KBs. In a rewriting step, one input order (a negative first-order clause) is reformed as a conjunctive clause (adding a relational positive query) from the sample database. Inspired by Horn description logics (DL) [43], an example of query ${\varphi }^{*}$ is illustrated below, where $q1$ is a revised clause with respect to the property of nationality. The query is evaluated over KB, and corrected item (Germany) replaces erroneous entity (German_people) in the predicate of nationality,

$$\begin{array}{c}{\varphi }^{*}:stateOfOrigin(y,German\_people),relative(x,y),\hfill \\ nationality(x,German\_people)\wedge birthPlace(x,z),country(z,f)\hfill \\ \to nationalit{y}^{*}(x,f)\hfill \end{array}.$$

(8)

For low-quality noise, for example, one triple is <dbr:Max_Amann, dbo:nationality, dbr:Germans> in DBpedia. As this triple is with error range type of relation, these facts with low-quality noise are detected. Then, the correction rules are leveraged to correct these errors. For high-quality noise, an example is <Yao_Ming, hasGivenName, Yao> in YAGO, where the rules guarantee correct constraints of types and schema relations.

Approaches for logic-based link prediction are closed to our research. Extending previous ILP work on relational association rule mining, e.g., AMIE+ [35], mines only first-order rules, and NILPs are mined with type property in KBs. Different body and head are focused on rules. Inspired by rules with exceptions, we focus on more detailed positive rules to reform revised rules from negative statements:

$$\begin{array}{c}{\varphi }_1^{+}:birthPlace(a,b)\wedge type(b,Country)\to nationalit{y}^{+}(a,b)\hfill \\ {\varphi }_1^{-}:birthPlace(a,b)\wedge not\_Country\left(b\right)\to nationalit{y}^{-}(a,b)\hfill \\ {\varphi }_1^{*}:birthPlace(a,b)\wedge not\_Country\left(b\right)\wedge P_1(b,f)\wedge type(f,Country)\hfill \\ \to nationalit{y}^{*}(a,f)\hfill \end{array}.$$

(9)

From Equation (9), one predicate is added to ${\varphi }_1^{-}$ to create a revised rule. Thus, a new query is created to find $P_1$:select ?p where{ ?a birthPlace ?b. ?b ?p ?f. ?f a country. filter not exists{?b a Country.}}. If some entities without nationality relations satisfy query results, these new triples are acquired to complete KBs. The most appropriate rewriting forms are detected to illustrate the semantics of exception rules by revising negative rules with positive features. Finally, the revised triples are extracted from the correction rules to complete knowledge bases.

3.4. Complexity Analysis

EILP was implemented in Java 11 with the SPARQL protocol and RDF query language (SPARQL) (https://www.w3.org/TR/sparql11-query/, accessed on 1 May 2022). Based on small samples, the model mines positive/negative rules with high precision. A website query engine and a local RDF3X-Engine were utilized to validate the results. These architectural choices were motivated by the availability of large knowledge bases. The model is compliant with SPARQL. The computational complexity of EILP, excluding constraints, is linear with respect to the properties linked in the knowledge bases. SPARQL patterns are PSPACE complete. As such, the time complexity relies on the SPARQL engine that computes the queries. The number of graph patterns in queries grows linearly with the size of the rule body. Because our queries contain only AND and FILTER operators, they can be calculated with time complexity $O(\parallel \overrightarrow{B}\parallel ·\parallel N\parallel )$, where $\overrightarrow{B}$ denotes body sizes and N is the total number of triples. EILP and EILC provide pruning optimization in large knowledge bases. As queries for adjacencies might be repeated, a new validation measure is introduced to avoid unnecessary queries and time consumption for large knowledge bases. Moreover, since the rules are sorted by confidence value in descending order, the algorithm can easily prune search spaces with new measures. The model implements a rule-learning algorithm that executes each type (negative/positive rules) with high optimization.

The computational complexity of EILP can be analyzed in detail as follows, incorporating both theoretical bounds and practical optimizations:

The model of EILP complexity is linear with respect to the number of linked exceptional types in KBs. This implies a baseline complexity of $O\left(L\right)$, where L is the number of linked types. Rule generation involves iterating over graph patterns, which grow linearly with the size of the rule body (denoted as $\overrightarrow{B}$). However, since EILP restricts queries to AND and FILTER operators (excluding UNION or OPTIONAL), the complexity reduces to $O(\parallel \overrightarrow{B}\parallel ·\parallel \overrightarrow{N}\parallel )$, where $\overrightarrow{B}$ is the size of the rule body (number of graph patterns), and $\overrightarrow{N}$ is the total number of RDF triples in the KB.

This simplification avoids exponential blowup, as AND/FILTER operations are tractable via join ordering optimizations in SPARQL engines like RDF3X. Repeated adjacency queries are avoided through a novel validation measure, reducing redundant computations. Rules are sorted by confidence (descending), enabling early termination of search when lower-confidence rules are encountered. This reduces the effective search space in practice. Pruning eliminates most redundant queries, and high confidence rules are found early. Complexity approaches O(L + $\parallel \overrightarrow{B}\parallel ·\parallel \overrightarrow{N}\parallel$) with a low constant factor. No pruning occurs (e.g., no rules meet confidence thresholds), and all graph patterns are evaluated. Complexity remains O(L + $\parallel \overrightarrow{B}\parallel ·\parallel \overrightarrow{N}\parallel$) but with higher constants due to exhaustive SPARQL evaluations shown in

Table 1. Comparative Analysis of Complexity.

Aspect	EILP	Generic SPARQL Based Models
Time Complexity	$O(\parallel \overrightarrow{B}\parallel ·\parallel \overrightarrow{N}\parallel )$	PSPACE complete (unrestricted)
Space Complexity	$O(R·\parallel \overrightarrow{B}\parallel +\parallel \overrightarrow{N}\parallel )$	$O(\parallel \overrightarrow{N}{\parallel }^2)$ (intermediate results)
Scalability	Linear in $\parallel \overrightarrow{N}\parallel$ (optimized)	Limited by PSPACE constraints

.

SPARQL engines like RDF3X use indexing and compression, limiting memory overhead. Intermediate results for AND/FILTER operations scale with the number of matched triples, bounded by $O\overrightarrow{N}$. Storing positive/negative rules require space proportional to the number of rules (R) and their body sizes $O\overrightarrow{B}$, leading to $O(R·\parallel \overrightarrow{N}\parallel$). Here, pruning reduces the number of rules retained in memory, lowering effective space usage.

The use of RDF3X Engine leverages efficient indexing, reducing the hidden constants in $O(\parallel \overrightarrow{B}\parallel ·\parallel \overrightarrow{N}\parallel )$. For large KBs (e.g., Wikidata), pruning and sorted rule evaluation are critical to maintain tractability. The reported linear complexity excludes constraints; incorporating them could reintroduce PSPACE dependencies shown in Table 1. By explicitly addressing these dimensions, the analysis clarifies EILP’s efficiency trade-offs and underscores its suitability for large-scale knowledge bases with sparse constraints.

In our model, we prune search spaces with measures to short time complexity. Here, retrieving more triples related to a rule $\varphi$ is performed to obtain a better value of $CP\left(\varphi \right)$. For example, suppose that we require the confidence to be 95%, and the lower bound of $CP\left(\varphi \right)$ to be higher than 0.9; we need the users to comment on at least 34 triples (we assume here all users give correct responses) according to Equation (10).

Finally, we need an extra requirement with respect to user feedback, called correctness probability (CP in short). In terms of statistics, the more tuples covered by a rule $\varphi$, the greater confidence we have about the correctness of $\varphi$. Ideally, we want to obtain comments for all tuples inside $\mathcal{K}$ which are covered by $\varphi$. Unfortunately, in practice, we can only obtain a limited amount of feedbacks. Suppose we obtain a set of feedbacks $\mathcal{F}$, according to the Wilson interval. The probability that $\varphi$ is correct, denoted by $CP\left(\varphi \right)$, is located within the following range with quantile z:

$$CP\left(\varphi \right){\in }_{\alpha }{\displaystyle \frac{1}{1+z^2/n}}(\widehat{p}+{\displaystyle \frac{z^2}{2n}}\pm z\sqrt{{\displaystyle \frac{\widehat{p}(1-\widehat{p})}{n}}+{\displaystyle \frac{z^2}{4\times n^2}}})$$

(10)

Here, $\widehat{p}$ is the observed average, which is the base of consistent instantiations of $\varphi$ with respect to $\mathcal{F}$; n is the number of instances covered by $\varphi$, and z is the $1-{\displaystyle \frac{1}{2}}\alpha$ quantile of a standard normal distribution (that is, for z equals 1.96 when $\alpha =0.95$). Generally speaking, we prefer to rules which are probably correct (i.e., $p\left(\varphi \right)$ is larger than $\theta$) with high confidence (i.e., z is large enough).

4. Experiments

4.1. General Setup

In experiments, Wikidata (https://query.wikidata.org, accessed on 1 May 2023) and two versions of DBpedia (2016, 2020) (https://wiki.dbpedia.org/develop/datasets, accessed on 1 May 2020) were selected as training sets. Two open and available query editors, DBpedia SPARQL (https://dbpedia.org/sparql, accessed on 1 May 2024) and Wikidata query (https://query.wikidata.org, accessed on 1 May 2023), were used in our models. We chose DBpedia as the baseline and Wikidata as a control group with cooccurring similar triples. The EILLearn model was implemented in Java 1.8.0 on a notebook computer with an Intel i7 CPU at 1.80 GHz with 16 GB memory.

4.2. Error Detection

Our error detection architecture depends on some tuning parameters. The quantity of feedback was 100 in the initial step and the maximum size of the rule was 3. The other rule learning parameters can be seen in [44], and the confidence in the selection of rules was retained at 0.01. In each iteration of the EILP, the confidence of the partial completeness assumption (PCA) was considered a measure of the quality of the rules. EILP was leveraged to complete the error detection task.

In the GILP model, negative feedback is generated by experts. Here, the TBox property was leveraged to replace experts to provide the feedback. For predicate name $nationality$, some example rules were learned by GILP, as shown in

Table 2. Example rules learned by GILP.

Positive rules: ${\mathit{\varphi }}^{+}$ head: nationality

$1.birthPlace(a,f)\wedge country(f,b)\to head(a,b)$

$2.deathPlace(a,f)\wedge country(f,b)\to head(a,b)$

$3.birthPlace(a,d)\wedge rdfType(d,dbo:Country)\to head(a,d)$

$4.successor(a,f)\wedge leader(b,f)\to head(a,b)$

$5.deathPlace(a,f)\wedge capital(b,f)\to head(a,b)$

$6.deathPlace(a,f)\wedge largestCity(b,f)\to head(a,b)$

Negative rules: ${\varphi }^{-}$

$1.populationPlace(b,f)\wedge deathPlace(a,f)\to head(a,b)$

$2.deathPlace(a,f)\wedge ethnicGroup(f,b)\to head(a,b)$

$3.birthPlace(a,f)\wedge populationPlace(b,f)\to head(a,b)$

$4.knownFor(a,f)\wedge dbo:ethnicGroup(f,b)\to head(a,b)$

$5.residence(a,f)\wedge populationPlace(b,f)\to head(a,b)$

$6.ethnicity(a,f)\wedge related(b,f)\to head(a,b)$

. Figure 3 depicts the number of positive and negative GILP rules learned over multiple iterations in different PCA intervals. Among the 10 confidence intervals of the positive rules, the quantity of rules changes fastest in interval [0.9, 1.0], while there are minor differences in other intervals, and their numbers are less than 100. DBpedia can achieve very high average precision values (over 95%) (the estimated precision of those statements is 95% in the Data Set 3.9 of DBpedia), so the amount of negative information mined is very small, and there are also very small gaps in other intervals except [0.9, 1.0]. It can be seen that the quantity of mined positive/negative rules is concentrated in the interval [0.9, 1.0], and rules in this interval can be preserved to generate predictions. Finally, the accuracy threshold of candidate rules is accepted if it is more than 0.9.

Figure 4 describes the number of positive and negative facts predicted over multiple iterations in the GILP model. After about 20 iterations, the experimental results of this cycle reached a stable state, and the number of positive predictions in the PCA confidence interval exceeded the number of negative predictions. The results suggest that negative rules acquire fewer predictions than positive rules. If predictions of positive or negative rules are considered only in interval [0.9, 1.0], some implicit predictions are missing in interval [0, 0.2]. In each loop, the numbers of positive predictions are close in the two intervals. For predicted negative facts, the number in [0, 0.2] is much higher than in [0.9, 1.0]. It can be seen that some implicit information is worth investigating in the interval [0, 0.2]. If all final rules are considered, there is a large number of invalid rules in [0.0–0.2]. Here, top 10 rules are selected as the effective results. In the process of rule extraction, the top 10 rules are filtered by numbers of predictions in the final iteration.

Due to space constraints in experiments, different from expert feedback, our model mines positive/negative rules based on the TBox property of KBs. Here, the largest difference can be observed from numbers and predictions of rules in intervals [0, 0.2] and [0.9, 1.0], while the number of predictions between two intervals in PCA confidence are close, as shown in Figure 5. It can be seen that rules in the interval [0–0.2] hide a large number of new predictions, which are worthy of exploration. Thus, EILP is proposed to explore features to improve the quality of rules.

To obtain implicit information with the confidence interval in the range of [0, 0.2], we mine abnormal rules from the perspective of abnormal information. Here, some example rules are learned by NILP, as shown in

Table 3. Example rules learned by NILP.

Positive rules: ${\mathit{\varphi }}^{+}$ head: nationality(a, b)

$1.academicAdvisor(a,c)\wedge birthPlace(c,b)\wedge not\_residence(a,b)\to head(a,b)$

$2.associate(a,c)\wedge country(c,b)\wedge not\_deathPlace(a,b)\to head(a,b)$

$3.birthPlace(a,c)\wedge country(c,b)\wedge not\_education(a,b)\to head(a,b)$

$4.birthPlace(a,c)\wedge ethnicGroup(c,b)\wedge not\_genre(a,b)\to head(a,b)$

$5.child(a,c)\wedge nationality(c,b)\wedge not\_citizenship(a,b)\to head(a,b)$

$6.child(a,c)\wedge stateOfOrigin(c,b)\wedge not\_citizenship(a,b)\to head(a,b)$

Negative rules: ${\varphi }^{-}$

$1.birthPlace(a,c)\wedge ethnicGroup(c,b)\wedge not\_genre(a,b)\to head(a,b)$

$2.birthPlace(a,c)\wedge owl:differentFrom(c,b)\wedge not\_citizenship(a,b)\to head(a,b)$

$3.child(a,c)\wedge nationality(c,b)\wedge not\_birthPlace(a,b)\to head(a,b)$

$4.child(a,c)\wedge stateOfOrigin(c,b)\wedge not\_citizenship(a,b)\to head(a,b)$

$5.citizenship(a,c)\wedge ethnicGroup(c,b)\wedge not\_ethnicity(a,b)\to head(a,b)$

$6.country(a,c)\wedge ethnicGroup(c,b)\wedge not\_citizenship(a,b)\to head(a,b)$

. For positive or negative rules, they have some different exceptions in training data. The disadvantage of NILP is that it requires an ideal knowledge base containing all the information to find outliers for the training set. In ${\varphi }_5^{+}$, the abnormal information is $citizenship$, and it has a similar property with $nationality$. In training sets, the confidence of ${\varphi }_5^{+}$ is over 0.9. However, some predictions can be regarded as negative statements following ${\varphi }_5^{+}$. Inspired by GILP and NILP, exceptional information and negative statements are both considered in EILP.

Some rules are learned by EILP, as shown in

Table 4. Rules learned by EILP.

Positive rules: ${\mathit{\varphi }}^{+}$ head: nationality

$1.populationPlace(b,f)\wedge deathPlace(a,f)\wedge type(b,wikidata:Q6256)\to head(a,b)$

$2.deathPlace(a,f)\wedge ethnicGroup(f,b)\wedge type(b,wikidata:Q6256)\to head(a,b)$

$3.birthPlace(a,f)\wedge populationPlace(b,f)\wedge type(b,wikidata:Q6256)\to head(a,b)$

$4.birthPlace(a,b)\wedge residence(a,b)\wedge not\_type(b,wikidata:Q41710)\to head(a,b)$

$5.country(c,b)\wedge successor(c,a)\wedge not\_type(b,wikidata:Q41710)\to head(a,b)$

$6.birthPlace(a,b)\wedge spouse(c,a)\wedge not\_type(b,wikidata:Q41710)\to head(a,b)$

Negative rules: ${\varphi }^{-}$

$1.birthPlace(a,f)\wedge country(f,b)\wedge not\_type(b,wikidata:Q6256)\to head(a,b)$

$2.deathPlace(a,f)\wedge country(f,b)\wedge not\_type(b,wikidata:Q6256)\to head(a,b)$

$3.birthPlace(a,d)\wedge type(d,Country)\wedge not\_type(d,wikidata:Q6256)\to head(a,d)$

$4.birthPlace(a,c)\wedge populationPlace(b,c)\wedge type(b,wikidata:Q41710)\to head(a,b)$

$5.birthPlace(a,c)\wedge type(b,dbo:EthicGroup)\to head(a,b)$

$6.knownFor(a,b)\wedge type(b,dbo:language)\to head(a,b)$

. Here, some exceptional information is extracted from feedback or the type of property knowledge. In the examples, $nationality$ selects range type property $wikidata$: Q6256 (https://www.wikidata.org/wiki/Q6256, accessed on 1 May 2023) (dbo: Country) (https://dbpedia.org/ontology/, accessed on 1 May 2023) to filter positive feedback. Then, negative feedback is acquired from the property of $wikidata$: Q41710 (dbo: EthnicGroup) or other exceptional information, e.g., $dbo$: $language$, $dbo$: $organization$, $SportsTeam$.

We compare GILP and EILP rules to find exception intervals for 20 loops, as shown in Figure 6. Here, a reverse thinking algorithm is leveraged to obtain the number of rules to reduce time complexity. When positive/negative rules with exceptions satisfy $x=0$, the original rules have 100% precision with positive/negative feedback. Through reversing validation methods, exception intervals are found when the experimental results remain stable after 20 iterations. Coverage of rules still guides our assessment of rule quality: good positive rules should cover many positive/negative example pairs in KB, while negative rules cover as few pairs of outliers as possible in our model. From Figure 6, the outliers covered by positive rules are stabilized after five iterations, while negative rules are balanced after almost 18 iterations. In each iteration, rules cover fewer than 10 exceptional messages, as shown in the interval [0, 10]. These can be defined as rough rules, and there is much effective information that is worthy of exploration. A negative rule contains less negative example information, and the maximum number is 20. A positive rule contains a large amount of valid information, and more than 500 rules have 100% precision. Therefore, positive and negative feedback can be analyzed, and more effective results can be mined for subsequent corrections.

In our experiments, rule learning algorithms are amended to automatically filter rules through external information of the knowledge base. The search space is pruned with a Markov logic network, and the depth of expansion feedback is 3. Combining positive and negative rules, logical queries are devised to correct negative feedback. A single negative rule is acquired randomly to filter the negative knowledge in batches. Four rule learning algorithms, including association rule mining under incomplete evidence (AMIE), GILP, NILP, and EILP, are used to mine rules. Associated rules are considered in our framework and language bias follows the measures in the GILP model. In Figure 7, GILP mines the most positive rules, and EILP has better effects in terms of the quantity of negative feedback. In response to this phenomenon, GILP has stricter constraints for errors in KBs. Negative feedback provides little associated information to enact rules. Further pruning rule learning, an exception is added to the original rules for the selection of the best rules. The goal of rules is to mine positive examples without covering any negative example information.

In five loops, GILP mines the largest number of positive rules, and AMIE mines the most negative rules. The number of rules does not determine their accuracy. Because GILP and AMIE do not consider abnormal information, there are many fuzzy rules, containing both positive and negative information, and the correctness of their predictions cannot be determined. Therefore, exception information is added to GILP and AMIE, and new models EILP and NILP compare regular and exception rules. It can be seen that the performance of EILP is close to that of GILP, and better than that of AMIE on positive rules. In terms of negative feedback, EILP obtains the fewest rules, removes redundant rules, and obtains the best effects. Comparing two cases, it can be found that EILP only contains rules with 100% accuracy, and invalid rules are excluded. Our goal is to obtain the smallest number of rules and the most accurate rules possible. In the feedback of positive and negative examples, EILP is leveraged to achieve this.

We analyze three types of rules (GILP, NILP, EILP) and show those with confidence greater than 0.9 in Table 2, Table 3 and Table 4. The final confidence interval of rules is explained in Figure 8. Because GILP contains no outliers, the GILP model extends EILP and NILP, and the overall effects are better than those of GILP. Therefore, we analyze only the difference between EILP and NILP. The results of the last set of loops are analyzed. For small samples, two exception algorithms are compared and the results explain that EILP has better effects, with a confidence of 1.0. For positive feedback, the rules obtained by EILP achieve a correction precision of 100%, and the number of rules is more than 300. The number of rules obtained by NILP was less than 100, and their confidence satisfied precision > 0.9. Due to the small amount of negative information, the number of rules obtained is small, so it is easy to fall into local optima. In the case of small samples of negative feedback, the rules obtained by EILP have a significant effect when precision is greater than 0.9. Almost no rules satisfy the conditions in NILP. From this, it can be seen that the selection of EILP has a better effect in the rule learning algorithm with exception. Here, all errors can be detected by negative rules, as shown in Table 2, Table 3 and Table 4. Finally, all negative predictions can be generated by EILP and are leveraged in the task of knowledge correction.

4.3. Knowledge Correction

In the error detection part, EILP generates erroneous triples as training data for the task of knowledge correction. We select $nationality$ as the predicate name to generate erroneous triples in two versions of DBpedia (2016, 2020). Depending on different subjects and objects, the training data are divided into four groups: subject (2016), subject (2020), object (2016), object (2020).

In the EILC model, errors are generated by negative rules in the EILP model. To build correction rules in EILC, the best positive rule is chosen to rebuild the logical queries in EILP. For example, rule ${\varphi }_3^{-}$ is picked as the negative sample in Table 2. Analyzing the search space in ${\varphi }_3^{-}$, the correction model finds a positive rule ${\varphi }_3^{+}$ to build a logical query for correction. Then, a new refined query ($\varphi \_Revised$) is generated: $birthPlace(a,f)\wedge populationPlace(b,f)\wedge nationality(a,b)\wedge type(d,Country)\to nationalit{y}^{*}(a,f)$. The constants on f serve as repairs from the refined clause $\varphi \_Revised$. Typically, the corrected targets $nationality$ are multivalued. As such, the top 10 repairs are taken to filter the final corrections by similarity measures.

EILC focuses on generating correction rules with exceptions, which have a higher quality to obtain predictions. Here, positive rules with ${\varphi }_i^{+}$ revise false statements from negative rules based on the features of subjects. EILC corrects incorrect information on triples with different subjects using Algorithm 3.

Given an error fact, <Adélard_Turgeon, nationality, Canadians>, we extract features in ${\varphi }_4^{+}$ of the GILP model: <Adélard_Turgeon, birthPlace, Canada>, <Canada, type, Country>→<Adélard_Turgeon, nationality, Canadians>. In the EILP model, the error triple can also be extracted by ${\varphi }_1^{+}$. Following Equation (9), the exception correction rule is generated to obtain repairs <Canadians, Canada> for the subject <Adélard_Turgeon>. Then, the revised fact is <Adélard_Turgeon, nationality, Canada>. In the EILC algorithm, a single rule only corrects a limited number of error triples. This model corrects the error information of a target predicate name by multiple correction rules so as to achieve the correction of a large knowledge base. Here, some EILC correction rules are shown in

Table 5. Correction rules learned by EILC.

1 $birthPlace(a,c)\wedge country(c,b)\wedge not\_type(b,wikidata:Q6256)\wedge nationalit{y}^{-}(a,b)\bowtie P_1(b,d)\bowtie$

$populationPlace(d,c)\wedge type(d,wikidata:Q6256)\to nationalit{y}^{+}(a,d)$

2 $deathPlace(a,c)\wedge country(c,b)\wedge not\_type(b,wikidata:Q6256)\wedge isCitizenO{f}^{-}(a,b)\bowtie P_1(b,d)\bowtie$

$ethnicGroup(c,d)\wedge type(d,wikidata:Q6256)\to isCitizenO{f}^{+}(a,d)$

3 $birthPlace(a,b)\wedge type(b,Country)\wedge not\_type(b,wikidata:Q6256)\wedge locationCountr{y}^{-}(a,b)\bowtie P_1(b,d)\bowtie$

$residence(a,d)\wedge not\_type(d,wikidata:Q41710)\to locationCountr{y}^{+}(a,d)$

4 $birthPlace(a,c)\wedge populationPlace(b,c)\wedge type(b,wikidata:Q41710)\wedge nationalit{y}^{-}(a,b)\bowtie P_1(b,d)\bowtie$

$type(d,wikidata:Q6256)\to nationalit{y}^{+}(a,d)$

5 $birthPlace(a,c)\wedge type(b,dbo:EthicGroup)\wedge nationalit{y}^{-}(a,b)\bowtie P_1(b,d)\bowtie$

$type(d,wikidata:Q6256)\to nationalit{y}^{+}(a,d)$

6 $knownFor(a,b)\wedge type(b,dbo:language)\wedge nationalit{y}^{-}(a,b)\bowtie P_1(b,d)\bowtie$

$type(d,wikidata:Q6256)\to nationalit{y}^{+}(a,d)$

7 $birthPlace(a,c)\wedge ethnicGroup(c,b)\wedge not\_genre(a,b)\wedge nationalit{y}^{-}(a,b)$

$\bowtie country(c,d)\wedge not\_education(a,d)\to nationalit{y}^{+}(a,d)$

8 $birthPlace(a,c)\wedge owl:differentFrom(c,b)\wedge not\_citizenship(a,b)\wedge nationalit{y}^{-}(a,b)$

$\bowtie ethnicGroup(c,d)\wedge not\_genre(a,d)\to nationalit{y}^{+}(a,d)$

9 $child(a,c)\wedge nationality(c,b)\wedge not\_birthPlace(a,b)\wedge nationalit{y}^{-}(a,b)$

$\bowtie stateOfOrigin(c,d)\wedge not\_citizenship(a,d)\to nationalit{y}^{+}(a,d)$

10 $child(a,c)\wedge stateOfOrigin(c,b)\wedge not\_citizenship(a,b)\wedge nationalit{y}^{-}(a,b)$

$\bowtie country(c,d)\wedge not\_education(a,d)\to nationalit{y}^{+}(a,d)$

11 $academicAdvisor(a,c)\wedge ethnicGroup(c,b)\wedge not\_ethnicity(a,b)\wedge nationalit{y}^{-}(a,b)$

$\bowtie birthPlace(c,d)\wedge not\_residence(a,d)\to nationalit{y}^{+}(a,d)$

12 $associate(a,c)\wedge ethnicGroup(c,b)\wedge not\_citizenship(a,b)\wedge nationalit{y}^{-}(a,b)$

$\bowtie country(c,d)\wedge not\_deathPlace(a,d)\to nationalit{y}^{+}(a,d)$

. One symbol ⋈ is utilized to connect negative and positive rules with common attributes. Like Example 1, atoms ($birthPlace(a,c)\wedge country(c,b)$) exist in negative and positive rules. So, the final atom $nationalit{y}^{+}(a,d)$ is applied to acquire repairs for negative rules in batches.

The corresponding subject or object in Wikidata is found by property owl:sameAs in DBpedia, which is utilized as the repair of erroneous triples in triple correction assessment (TCA), which can only be used to verify triples with attribute owl:sameAs. If Wikidata lacks facts existing in DBpedia, then TCA cannot perform verification. Therefore, the verification result of TCA is used to measure the correction effect of EILC. The more error triples exist in KBs, the more time it takes to perform corrections in TCA. We match the error triples in the external knowledge base (Wikidata) and find the target entity to replace the old one as the correction. For small errors, the method is easy and quick. For large KBs, many errors and rules including features of errors are presented in Table 2. A single rule proposes two correlated predicates to the targets. Here, the top k rules are leveraged to perform the refining queries and count the confidence of the top k refined logic rules, saved in the group of correction rules. Then, the fact nationality(Donald_Heins, Canadians) is a vandalized triple, and erroneous Canadians is repaired by Canada in the TCA model.

Unlike EILC, TCA aims to construct a model that enables a machine to query a single fact, and its repair has a high probability of semantic correlation. The tail $German\_people$ is false in relation to $nationality$, and the correction entity is $Germany$. For most triples <$?,nationality,German\_people$>, the repair triple is $<?,nationality,Germany$>. The strength of TCA depends on facts in associated KBs that contain information about the target facts. If entities have no additional information, TCA cannot correct them. However, EILC avoids this problem. Figure 9 shows that more than 70% of the facts have a corresponding repair in Wikidata. Here, fewer than 15% of triples cannot obtain correction values by EILC in $subject\left(2016\right)$. Compared to TCA, EILC is applied to revise incomplete knowledge in the open-world assumption. Two methods receive repair rates based on all entities after the final iterations are finished, containing more than 80% of entities in the given predicate. Analyzing the final results, EILC is advisable for large KB correction. In addition, TCA only applies to the correction of Wikidata related databases, and the corrected precision has little effect on the subject and object of incorrect knowledge. Two methods are leveraged to correct the negative statements in the DBpedia of two versions, since they contain a detailed TBox level to filter the wrong triples.

Compared to TCA and training data, our experiments illustrate the effectiveness of EILC and the improvements in precision for the correction of KB, as shown in Figure 9. The $Base$ stands for the precision of training data. There are obvious improvements in TCA, and our model is gaining in performance. Two models have near 90% precision after refining the whole KB. The results show that EILC has a more significant effect, and we propose directed logic rules with exception to explanation. All results of empty and correction rates are shown in Figure 10. All DBpedia training data sets have correction rates greater than 0.8. In addition, EILC has an empty rate below 15%. The corrected rates for all facts are received in the final exception rules, which contain more than 80% atoms in the given predicates from DBpedia. Accordingly, EILC is favorable to correct large knowledge bases. In addition, TCA can be utilized to validate triples and supplement EILC.

After acquiring EILC repairs, 14 similarity measures are leveraged to compare these repairs to TCA, as shown in Figure 11. The mistaken entities have single values as the final correction. For multiple values as repairs, cross-similarity is proposed to discover final corrections. Distance similarity measures are leveraged to validate repairs, such as longest common subsequence (LCS), optimal string alignment (OSA), and normalized Levenshtein distance (NLD). Compared to DBpedia, the similarity of repairs in Wikidata is based on word similarity. For a single erroneous triple, the Jaro–Winkler similarity is utilized to validate repairs, and the revised correction has an interval with high precision. In the experiment, 2000 negative entities are randomly selected to verify in the TCA model. The best cross-similarity performance is shown in Figure 11 and Equation (3). So, cross-similarity is leveraged to filter final repairs in the EILC model. The final pairs of errors and corrections exhibit unique characteristics, which have a high degree of word similarity. Here, multiple repairs represent that some examples have an over 90% similarity probability, that is, Jaro–Winkler similarity.

In practice, the precisions of most rules are poloarized, i.e., close to one or close to zero. The following discussions are based on this observation. The left graph of Figure 12 illustrates the lower and upper bounds of the Wilson intervals of a positive rule (correctness probability higher than 0.9) with different numbers of predictions. We can ensure that the rule’s CP is higher than 0.8 with high confidence. Figure 12 shows the result for negative rules from negative feedback. From this figure, we can find that it is higher than 0.8 with high confidence when pulling over 400 predictions. Figure 12 provides a detailed analysis of the Wilson intervals (at $\gamma =0.95$). An estimation of the precision (CP) of the learned rules is obtained in ${\mathcal{T}}^{+}$ (positive predictions) and ${\mathcal{T}}^{-}$ (negative predictions) in each iteration ($|{\mathcal{F}}_0|=40$). Analyzing through experiments, the intervals become tighter not only due to the increased amount of evaluation but also due to the increased amount of feedback collected at each iteration. They show that the accuracy of the predictions in subsequent iterations is greatly improved. After three to eight iterations, the accuracy interval mean of positive predictions converges to about 0.91, and even converges to 0.94 for negative facts, which confirms that our iterative guided rule learning model is particularly good for pruning negative facts.

5. Conclusions

In this paper, we studied the problem of detecting and correcting errors in large KBs. The core of the detection and correction framework was learning some rules, which, with exceptions, helped reveal inconsistencies in constraints that occurred during the construction of a knowledge base in a targeted predicate name. We highlighted the challenges of this problem, especially the exceptional information and the negative statements added to the rules. An iterative EILP algorithm was proposed to detect these mistakes; the EILC algorithm was proposed to correct these errors in the large KB, and appropriate metrics were proposed to assess the quality of the revised facts. The robustness and effectiveness of the proposed framework were verified by experiments in the DBpedia Knowledge Base.

In the future, we will study other kinds of nonmonotonic rules and update the ILP algorithm. Our next research will look into the extraction of conflicting feedback for or against exceptions from literals and the pruning of the search space with clustering algorithms to refine the KB. In addition, embedding methods can be incorporated in rule learning to improve the interpretability of knowledge.