Object Identity Reloaded—A Comprehensive Reference for an Efficient and Effective Framework for Logic-Based Machine Learning

Abstract

Sub-symbolic Machine Learning (ML) techniques, and specifically Neural Network-based ones, recently took over the research landscape, thanks to their efficiency and impressive effectiveness. On the other hand, the recent debate on ethics and AI and the first regulations on AI are progressively calling for anthropocentricity, which in turn requires explicit, human-understandable, and explainable approaches and representations that allow humans to be active parts in the loop. In these cases, logic-based approaches are more suitable. The Inductive Logic Programming (ILP) branch of research in ML provides an anwer to this need and a uniform and unifying framework for three relevant industrial and research concerns: management of databases, implementation of software systems, and modeling of human-like reasoning strategies. A particular ILP framework based on the Object Identity (OI) assumption was proposed in the 1990s, for which desirable theoretical and pratical properties were demonstrated and working tools and systems that successfully approached real-world and classical problems in AI were developed. In an age when mainstream research and media seem to reduce AI and ML to just deep learning, this paper celebrates the 30th anniversary of OI by providing for the first time a comprehensive overview of the framework to be used as a reference for researchers still interested in investigating the ILP approach to ML.

1. Introduction

Research in Artificial Intelligence (AI) has traditionally investigated two kinds of approaches. The sub-symbolic one, based on numeric/probabilistic techniques, reproduces the brain’s physical structure and behavior, and it is most suited to replicate perception. The symbolic one, exploiting formal logics, aims at reproducing high-level mental functions and representations in humans, and it is most appropriate to reproduce mind. The former is extremely efficient, but unsuitable for environments where complex objects with many interacting relationships must be expressed and described. The latter allows humans to trace, understand, and check the AI outcomes, thanks to the possibility of associating logic formulas with a ‘semantics’, i.e., an intuitive meaning that links them to our world and thoughts, and the chance of reproducing the inference strategies that humans use in conscious reasoning. This comes at the cost of increased computational complexity, which pushed the research towards weaker frameworks showing a good tradeoff between costs and benefits. Logic Programming (LP) [1,2] is the most famous. It deserves attention in the current Computer Science landscape since it provides a uniform and unifying framework for three major industrial and research concerns: implementation of software systems, management of information stored in databases (DBs), and modeling of human-like reasoning strategies to be applied in less formalized and understood environments and situations, where algorithms for carrying out the requested tasks are lacking.

Research in AI quickly experienced the so-called ‘knowledge acquisition bottleneck’ [3], i.e., the enormous difficulty in filling up the systems’ knowledge base with respect to using it for reasoning. This is due to the difficulty of domain experts in eliciting and properly formalizing their competence and sometimes their (more or less unconscious) refusal to provide their own “secrets” to other people (let alone a computer system). This gave rise to another research branch, Machine Learning (ML), aimed at automatically acquiring the knowledge directly from observations. Specifically, incremental ML can be important (for improving efficiency, as a closer simulation of human behavior, and for supporting the continuous changes and adaptation in the knowledge). Again, both sub-symbolic and symbolic approaches were attempted. The former, and specifically Neural Network-based ones, recently took over the research landscape thanks to their efficiency and impressive effectiveness. On the other hand, the recent hot debate on ethics and AI and the first regulations on AI [4] are progressively calling for anthropocentricity, which in turn requires explicit, transparent, human-understandable, and explainable approaches and representations that allow humans to be active parts in the loop. When these are unnegligible requirements (e.g., in medical and other critical applications, such as those concerning bioinformatics and chemistry —see the works by Muggleton, Srinivasan, and Ross, including [5] on carcinogenesis, [6,7] on mutagenesis), the latter is the appropriate choice [8]. The symbolic approach to ML is investigated by Inductive Logic Programming (ILP) [9] based on LP. A particular ILP framework based on the Object Identity (OI) assumption was proposed in the 1990s, aimed at supporting all of the above. We stress here the fact that this paper refers to the notion of Object Identity in ILP, which has nothing to do with the notion with the same name used in the Database and Object Oriented community [10] For this framework, desirable theoretical and practical properties were demonstrated, especially in the incremental ML setting, and working tools and systems that successfully apprached real-world and classical problems in AI were developed. In an age when mainstream research and media seem to reduce AI and ML to just deep learning, this paper celebrates the 30th anniversary of OI by providing for the first time a comprehensive overview of the framework to be used as a reference for researchers still interested in investigating the ILP approach to ML. Note that no new theoretical result is proven in this paper; all the proposed proofs were already published in papers or introduced in thesis works. Rather, we organize here the whole corpus of results in a more coordinated and balanced way.

Concerning the methodology for carrying out the review, given the objective to stress the need to re-discover symbolic, especially FOL-based, approaches, as alternative and complementary to subsymbolic ones, a first decision was to exclude all works concerning attempts to combine neural and symbolic learning (see [11] for an overview on NeSy). Of course, we totally endorse the possibility of mixing the two kinds of approaches in balanced solutions that can take the best of both and let them make each for the limitations of the other; put simply, this is not in the scope of this paper. Then, we conducted a systematic review of all the papers that introduced, expanded, or used the OI framework. Since this work is specifically interested in the theoretical results, we then identified the subset of papers that included theoretical contributions and determined the minimum set that could cover all the relevant ones, preferring journal papers over conference ones. Since the results and proofs collected in this way sometimes used different notation, we rewrote them using the same style and formalism, also removing some redundancy. Finally, we turned to the subset of papers reporting on practical applications of the OI framework (e.g., describing OI-based systems or experiments using these systems). Being this a secondary objective of this work, we only selected a few most prominent papers that could showcase, without an ambition to exhaustiveness, a sufficiently varied range of algorithms, systems, and applications demonstrating the practical use and advantages brought by the OI framework.

The paper is organized as follows. Section 2 introduces the basics of logics, LP and ILP. Then, Section 3, Section 4 and Section 5 provide an overview of logics under OI, of how it restructures the space of formulas, and on how it can be used in incremental ML, respectively. Section 6 and Section 7 discuss the framework and its uses, before concluding with Section 8.

2. Basics

First-Order Logic (FOL for short) has gained predominance in symbolic AI research as a means for formally representing knowledge and implementing automated reasoning, since it reaches a good tradeoff between expressive power and computational tractability. Indeed, at this level the (informal) notion of semantic consequence between formulas (⊧) is equivalent to syntactic derivation (⊢), which is formal and algorithmic. In FOL:

• a term, representing and object, is a variable, a constant, or an n-ary function symbol applied to n terms as arguments;
• an n-ary predicate p, denoted as $p/n$, is a kind of claim involving n objects;
• an atom is an n-ary predicate applied to n terms as arguments, representing a claim that can be true or false; a formula is an atom or a suitable composition of atoms using logic functions (such as negation ¬ and disjunction ∨); a theory is a set of formulas.

$\mathit{vars}\left(E\right)$, $\mathit{consts}\left(E\right)$, and $\mathit{terms}\left(E\right)$ denote the sets of variables, constants, and terms occurring in a formula E, respectively. A formula or a term is ground if it does not contain variables.

The clausal fragment of FOL is based on the following additional definitions:

• A literal is an atom (positive literal) or its negation (negative literal); $\overline{l}$ denotes the opposite of a literal l.
• A clause is a universal formula consisting of a finite disjunction of literals; the length of a clause C, denoted by $\mid C\mid$, is the number of its literals; the empty clause □, representing a contradiction, does not contain any literal.
• Two clauses are standardized apart if they do not share variables; they are variants if they differ only in the names of variables (renaming variables in a formula does not change its meaning); in the following, clauses are always standardized apart.

A clause is usually denoted as a set of literals or in Prolog style:

$$\{A_1,\dots ,A_m,\neg B_1,\dots ,\neg B_n\}\equiv A_1,\dots ,A_m:-B_1,\dots ,B_n$$

where the $A_i$ and $B_j$ are atoms. It corresponds to the implication “If all $B_j$s are true, then at least an $A_i$ is true”, where $\left\{A_i\right\}$ are the conclusions and $\left\{B_j\right\}$ are the premises. The clausal language is the set of all clauses expressed in a given alphabet of constant, variable, function, and predicate symbols. It has the same expressive power as general FOL, since each FOL formula can be expressed as a set of clauses by casting it in normal conjunctive form and eliminating existential quantifiers by transforming them into functions of the universally quantified ones (skolemization).

A substitution is a finite mapping from variables to terms, represented as a set of bindings $\theta =\{t_1/x_1,\dots ,t_n/x_n\}$, where all variables $x_i$ are distinct and $\forall i=1,\dots ,n$: $x_i\ne t_i$. The empty substitution $ϵ=\left\{\right\}$ contains no bindings. A substitution is ground if all $t_i$ are ground; it is a renaming if all terms $t_i$ are distinct variables. Applying substitution $\theta$ to a formula E (denoted by $E\theta$) consists in replacing in E each occurrence of $x_i$ by $t_i$. The resulting formula is an instance of E. If $\theta$ is a renaming, $E\theta$ is called a variant of E. A generality ordering can be defined upon substitutions: a substitution $\theta$ is more general than another $\eta$ if $\exists \gamma$ substitution such that $\eta =\theta \gamma$ (where $\theta \gamma$ means applying $\gamma$ to the result obtained by applying $\theta$). Two formulas $E,F$ are unifiable if $\exists \theta$ substitution such that $E\theta =F\theta$. Such a $\theta$ is a unifier of $E,F$, and $E\theta$ is their unification. $\theta$ is a most general unifier (MGU) of $E,F$, denoted $\mathit{mgu}(E,F)$, if, for any other unifier $\sigma$ of $E,F$, $\theta$ is more general than $\sigma$. Unification is decidable in FOL (not in higher-order logics), where the MGU is also unique (except for renamings).

Logic formulas can be associated to a semantics, i.e., a meaning that is intuitive and that can mirror human experience. The classical declarative semantics used in logics is based on the following notions. Given a theory T,

• the Herbrand Universe of T, $U_T$ is the set of all terms that can be expressed in the language;
• the Herbrand Base of T, $B_T$, is the set of all ground atoms that can be expressed in the language;
• a Herbrand Interpretation for T associates each atom in the Hebrand Universe to a truth value (true or false); it can be seen as defining the set of true atoms in $B_T$.

A Herbrand interpretation is a model for T if it makes all formulas in T true. If a theory has at least one model, it is satisfiable, otherwise it is unsatisfiable. The intersection of Herbrand models is a Herbrand model, and the intersection of all Herbrand models of T is the least Herbrand model, denoted $M_T$.

Some ordering relationship ≤ can be applied to clauses to define their generality, where $C\le D$ means “C is more general than D” (or “D is more specific than C”); $C<D$ means that $C\le D$ and $D\neg \le C$ (C is strictly more general than D, D is strictly more specific than C), and $C\sim D$ means that $C\le D$ and $D\le C$ (C and D are equivalent).

The classical generality ordering used in logic is implication, according to which a formula is more general than another if, whenever the former is true, the latter is, too:

Definition 1 

(Implication). Given C and D formulas, C implies D ($C\Rightarrow D$ or $D{\le }_{I}C$) iff any model for C is a model for D ($C\vDash D$). Equivalence is denoted by ⇔.

Implication is a quasi-ordering; not being anti-symmetric, two equivalent clauses under implication might not be variants. Since it is undecidable (given C and D clauses, there exists no procedure to tell if $C\Rightarrow D$) [12], various attempts to obtain weaker but computable orderings take place, the most successful of which is ($\theta$) subsumption.

Definition 2 

($\theta$ subsumption). Given C and D clauses, C $\theta$ subsumes D ($C{\le }_{\theta }D$) iff $\exists \theta$ substitution such that $C\theta \subseteq D$. Equivalence is denoted by ${\sim }_{\theta }$.

$\theta$-subsumption is a quasi-ordering upon clauses, strictly weaker than implication (if $C{\le }_{\theta }D$ then $C{\le }_{I}D$, but not vice versa). Since it is not anti-symmetric, two clauses can be equivalent without being variants. $\theta$-subsumption is decidable, but NP-complete [13].

A calculus allows to reason on knowledge represented in logics using operations called inference rules, each being a tautology involving a finite number of premises and a consequence: if all premises are true, the consequence is true. A derivation, denoted by ⊢, is a sequence of applications of inference rules. An inference rule is sound if it allows to derive only logical consequences of the premises; it is complete if it allows to derive all logical consequences of a set of formulas. A very simple, but very powerful, inference rule is resolution:

Definition 3 

(Resolution). Let $C=C^{\prime }\vee l_C$, $D=D^{\prime }\vee \overline{l_D}$ be two formulas with $l_C,l_D$ literals such that $\exists \theta =\mathit{mgu}(l_C,l_D)$. A resolvent of C and D, where the parents C and D were resolved upon $l_C$ and $l_D$, is a formula $R=\mathcal{R}(C,D)=(C^{\prime }\vee D^{\prime })\theta$.

$\mathcal{R}$ is sound and is a closed operation for clauses (the resolvent of two clauses is again a clause). Together with a finite set of clauses (called axioms) T, it defines a clausal theory. The resolution closure of a set of clauses T, denoted by ${\mathcal{R}}^{*}\left(T\right)$, is defined inductively:

• ${\mathcal{R}}^0\left(T\right)=T$;
• ${\mathcal{R}}^n\left(T\right)={\mathcal{R}}_{OI}^{n-1}\left(T\right)\cup \{R=\mathcal{R}\left(\{C,D\}\right)|C,D\in {\mathcal{R}}^{n-1}\left(T\right)\}$, ($n>0$);
• ${\mathcal{R}}^{*}\left(T\right)={\mathcal{R}}^0\left(T\right)\cup {\mathcal{R}}_{OI}^1\left(T\right)\cup \cdots \cup {\mathcal{R}}^n\left(T\right)\cup \cdots$.

In a clausal theory, for any clause C, $T⊢C$ iff $C\in {\mathcal{R}}^{*}\left(T\right)$ [14,15]. $\mathcal{R}$ is incomplete by deduction because it cannot instantiate variables or add literals (what subsumption does). There exists a deduction of C from T ($T{⊢}_{d}C$) if C is a tautology, or $\exists D$ such that $T⊢D\wedge D{\le }_{\theta }C$. Derivation of the empty clause from T, called a refutation, happens iff T is unsatisfiable (i.e., contradictory). This is the basis of the ‘Proof by refutation’ strategy: if the negation of the thesis, along with the axioms, derives the empty clause, then the theorem is true.

Theorem 1 

(Subsumption theorem [16,17]). Let T be a set of clauses and C a clause. If $T\vDash C$, then $T{⊢}_{d}C$.

Corollary 1 

(Refutation completeness [16]). If T is an unsatisfiable set of clauses, then $T⊢\square$.

We focus on linear resolution ($\mathcal{L}$), where a new resolution step in the derivation must involve the result of the latest resolution step. Albeit being a restriction of $\mathcal{R}$, a subsumption theorem holds as a consequence of its completeness with respect to refutation [9].

2.1. Logic Programming

A particular restriction of clausal theories is based on Horn clauses that include at most one positive literal. In a Horn clause, the premises (if any) are the body; the conclusion (if any) is the head. A definite program clause has both head and body; a goal clause has no head; a unit clause has no body. Horn clauses are interesting since, once the premises are verified, the true consequence is univocally determined. A clause is

• domain restricted iff all variables occurring in its body also occur in its head;
• range restricted iff all variables occurring in its head also occur in its body;
• linked if all of its literals are; a literal is linked if at least one of its arguments is; an argument of a literal is linked if the literal is the head of the clause or if another argument of the same literal is linked [18].

Thus, a domain-restricted clause is also linked.

Example 1. 

$C=\left\{P\right(x),\neg Q(x,y),\neg Q(y,z\left)\right\}$ is linked; $D=C\setminus \{\neg Q(x,y\left)\right\}$ and $F=C\cup \{\neg R(v,w\left)\right\}$ are not linked.

Thanks to the procedural interpretation of Horn clauses [19], they can be used as a programming language or as a DB manipulation language based on the correspondences in

Table 1. Procedural interpretation of clauses.

Logics	Programming	Databases
predicate	procedure name	relation (table)
term	data structure	(atomic) value
clause	procedure declaration
clause head	procedure heading
clause body	procedure body
atom in clause body	statement
definite clause	sub-program	view
goal	execution	query
fact		n-tuple

. A logic program is a set of definite Horn clauses, interpreted as their conjunction. The clauses in a logic program can be divided into facts (ground unit clauses representing true claims) and rules (definite clauses allowing to deduce true claims from other true claims).

General logic programming [1] admits negated literals in the body (it borrows the terminology from LP, adding adjective general).

Execution of logic programs works by refutation using a special case of resolution, SLD resolution (Selection rule-driven Linear resolution for Definite clauses): given a logic program P and a goal $:-l_1,\dots ,l_n$, the negation of the goal is $N=\exists (l_1\wedge \dots \wedge l_n)$, which starts the sequential execution of procedures $l_1,\dots ,l_n$. If ${\theta }_0\dots {\theta }_m$ is the sequence of substitutions performed during refutation, the refutation can be seen as a proof of formula $(l_1\wedge \dots \wedge l_n){\theta }_0\dots {\theta }_m$, and the bindings associated to the variables in N in the composed substitution $\theta ={\theta }_0\dots {\theta }_m$, called computed answer substitution for $P\cup \left\{N\right\}$, are the output of the program.

The classical declarative semantics obviously holds also for logic programs. Interestingly, the least Herbrand model $M_P$ of a definite program P is exactly the set of all its logical consequences, so one may always work on $M_P$ without loss of generality:

$$M_P=\{a\in B_P\mid a \mathrm{is} \mathrm{a} \mathrm{logic} \mathrm{consequence} \mathrm{of} P\}$$

Other approaches to semantics are specific to logic programs. Denotational semantics [1] relies on the fact that the set $2^{B_P}$ of all Herbrand interpretations of a definite program P is a complete lattice with respect to the partial ordering defined by set inclusion ⊆. The immediate consequence operator $T_P:2^{B_P}\to 2^{B_P}$ such that, given a Herbrand interpretation I,

$$T_P=\{a\in B_P\mid a:-a_1,\dots ,a_n \mathrm{is} \mathrm{a} \mathrm{ground} \mathrm{instance} \mathrm{of} \mathrm{a} \mathrm{clause} \mathrm{in} P \mathrm{and} \{a_1,\dots ,a_n\}\subseteq I\}$$

is clearly a monotonic mapping, with least fixpoint $T_P\uparrow \omega$. Procedural semantics focuses on the success set $S{S}_P$ of a definite program P, i.e., the set of all the (ground atoms) $a\in B_P$ such that there exists a SLD-refutation for $P\cup \{:-a\}$. All these approaches are equivalent:

$$M_P=T_P\uparrow \omega =S{S}_P$$

2.2. Datalog

Datalog (acronym for Database Logic) [20,21] is an approach to relational DBs based on LP. Syntactically, it is the function-free fragment of General LP [1]. Allowing only variables or constants as terms, it ensures that all elements in n-tuples are atomic (and hence the resulting DB is in First Normal Form). To avoid that from a Datalog program P infinite facts can be derived, two safety conditions must be imposed: all facts in P must be ground, and all rules in P must be range restricted. While in General LP all the knowledge is expressed in the program, Datalog splits the knowledge into two sets: the Extensional DB (EDB) contains all facts, stored in a relational DB; the Intensional DB (IDB) contains the rules, i.e., the very program P that defines the views. Accordingly, predicates are also distinguished in extensional ones if they are in the EDB, and intensional ones if they are defined by the heads of rules in P. A Datalog program can be viewed as a query on the EDB.

Some extensions of Datalog may use built-in predicates, expressed by special symbols (such as <, >, ≤, ≥, =, ≠), that have a pre-defined meaning. Although they can formally be regarded as EDB predicates, they are implemented as procedures to be evaluated at runtime. Since they often correspond to infinite relations, further safety conditions are required: any variable occurring as an argument of a built-in predicate in the head of a rule must occur also as an argument of an ordinary predicate in the body of the same rule or be bound by a (chain of) equality predicate(s) to a variable occurring in an ordinary predicate or to a constant. Adding the inequality built-in ≠ yields the Datalog^≠ extension.

Interestingly, Datalog has the same expressive power as unconstrained clausal logic, thanks to a representational change called flattening [22] that transforms each n-ary function symbol f into a fresh $(n+1)$-ary predicate $f_p$ whose first n arguments are the function’s arguments and whose last argument is the function result, defined as

$$f_p(t_1,\dots ,t_n,X):-X=f(t_1,\dots ,t_n)$$

A clause is flattened by replacing in it each occurrence of a term $f(t_1,\dots ,t_n)$ with a fresh variable X, and adding to its body $f_p(t_1,\dots ,t_n,X)$. All occurrences of the same term are replaced by the same variable. Flattening must be applied recursively on terms including nested functions, up to the complete elimination of function symbols. An inverse procedure, called unflattening, goes back from a flattened program to the original one. Given a clause (or logic program) C, $\mathit{flat}\left(C\right)$ and $\mathit{unflat}\left(C\right)$ denote its flattened and unflattened versions, respectively. $\mathit{Flat}\_\mathit{Defs}\left(P\right)$ denotes the set of all definitions of flattening predicates in a program P.

Theorem 2. 

$P\vDash A$ iff $\mathit{flat}\left(P\right),\mathit{Flat}\_\mathit{Defs}\left(P\right)\vDash A$.

Corollary 2. 

Given two definite clauses C and $C^{\prime }$,

1. 

if C is a clause with function symbols, $\mathit{unflat}\left(\mathit{flat}\right(C\left)\right)=C$;

2. 

if C is a flat clause, $\mathit{flat}\left(\mathit{unflat}\right(C\left)\right)=C$;

3. 

$C{\le }_{\theta }{C}^{\prime }$ iff $\mathit{flat}\left(C\right){\le }_{\theta }\mathit{flat}\left(C^{\prime }\right)$.

Datalog performs inference using unification. The elementary production principle (EPP) states that, given a rule $R=l_0:-l_1,\dots ,l_n$ and a set of ground facts $f_1,\dots ,f_n$, if $\exists \theta$ substitution is such that $\forall i=1,\dots ,n:l_i\theta =f_i$, then from R and $f_1,\dots ,f_n$ it is possible to infer in one step the fact that $l_0\theta$. Then, a ground fact f can be deduced from a set of clauses S (in symbols $S⊢f$) if $f\in S$ or if it can be deduced by EPP from a rule $R\in S$ and from ground facts previously deduced. Such an inference is called proof of f from S.

In regard to semantics, we determine that deduction is complete:

Theorem 3. 

If S is a set of Datalog clauses and f is a ground fact, $S\vDash f\iff S⊢f$.

The set $\mathit{cons}\left(S\right)$ of all facts that are logic consequences of a finite set of clauses S can be computed using Algorithm 1. Since S is finite, it contains a finite set of predicates and constants. EPP does not introduce new ones. The arity of facts derived is bounded by the greatest arity of heads of clauses in S. Thus, the algorithm terminates and the computed set $\mathit{cons}\left(S\right)$ is finite.

Algorithm 1: Algorithm for computing the set of logical consequences of a Datalog program.

function Infer (S: finite set of Datalog clauses) : $\mathit{cons}\left(S\right)$; begin W := S while EPP can be applied to some rule and fact of W and some fact $F\notin W$ is produced do    W := $W\cup \left\{F\right\}$ return $(W\cap HB)$ /∗ all facts in W, not rules ∗/ end.

    Applying model theory, it coincides with the least Herbrand model of S, and hence with the least fixpoint of the immediate consequence operator of S ($T_S$):  

$$\mathit{cons}\left(S\right)=\{f\in HB\mid S\vDash f\}=\bigcap \{I\mid I\mathrm{is}\mathrm{a}\mathrm{Herbrand}\mathrm{model}\mathrm{of}S\}=M_S=T_S\uparrow \omega$$

In pure Datalog, negation cannot be used. Negative facts can be deduced using the Closed World Assumption (CWA): “If a fact does not logically follow from a set of Datalog clauses, then assume its negation to be true”. Datalog^¬, allowing negated literals in clause body, can infer additional information from these facts at the cost of a further safety condition: each variable occurring in a negated literal must also occur in a positive literal of the body.

In regard to semantics, we introduce the following definitions, where $\mid f\mid$ denotes the positive counterpart of a negative fact f:

Definition 4. 

Let I be a Herbrand interpretation, f a ground fact, and $R=l_0:-l_1,\dots ,l_n$ a rule.

f is satisfied in I iff f is a positive fact and $f\in I$, or f is a negative fact and $\mid f\mid \notin I$.

R is satisfied in I iff, for all ground substitutions θ for R, we have that $\forall i=1,\dots ,n:l_i\theta$ satisfied in $I\Rightarrow l_0\theta$ is satisfied in I ($\iff l_0\theta \in I$, since $l_0$ is positive).

Declarative semantics can still be applied, but the existence of one least model is not guaranteed: e.g., the clause $h:-\neg b\equiv \neg b\Rightarrow h\equiv h\vee \neg (\neg b)\equiv h\vee b$ has two incomparable minimal models, $\left\{h\right\}$ and $\left\{b\right\}$. Different strategies are used to select one reference Herbrand model when evaluating a program P on an EDB E, considering that the evaluation of a literal in a rule may cause evaluation of other rules that in turn might contain negated literals, and so on. Stratified Datalog^¬ partitions P into n sets $P^i$ (called layers) such that

• all the rules that define the same IDB predicate in P are in the same layer;
• $P^1$ contains only clauses without negated literals or whose negated literals correspond to EDB predicates;
• each $P^i$ contains only clauses whose negated literals are completely defined in lower level layers (i.e., layers $P^j$ with $j<i$).

Such a partition is called stratification, and P is stratified. A stratified program is evaluated by growing layers, applying to each one CWA locally to the EDB made up by the original EDB and by all predicates obtained by the previous layers evaluation. The evaluation of a stratified program produces a minimal Herbrand model, called perfect model, that can be characterized in a purely semantic way. In the example, first b is evaluated, obtaining an empty answer set (since no fact of the form b is derivable from it); then, by CWA, we obtain $\neg b$, hence (by the rule) h, which is the perfect Herbrand model. While the stratification for a program is not unique, all are equivalent in regard to the evaluation result. Moreover, not all programs are stratified. A Datalog^¬ program P is stratified if its extended dependence graph $EDG\left(P\right)$ contains no cycles involving edges marked with ¬, where $EDG\left(P\right)$ is a directed graph whose nodes represent IDB predicates of P, there is an edge $\langle p,q\rangle$ if q occurs in the body of a rule defining p, and the edge is labeled with ¬ if there exists at least one rule with p in the head and in whose body occurs q negated.

Another approach, applicable to all Datalog^¬ programs, is the inflationary evaluation. It is applied iteratively: each step processes in parallel all rules in P, applying them to both the EDB and the facts computed so far, and then “temporarily” applies the CWA to evaluate the body of the rules (assuming that the negation of all facts not yet derived is true). It corresponds to the computation of a least fixpoint and returns a (generally non-minimal) Herbrand model. It allows to specify goals. Inflationary Datalog^¬ is strictly more expressive than stratified Datalog^¬ that in turn is strictly more expressive than pure Datalog.

2.3. Inductive Logic Programming

A theory can be viewed as a set of conditions that are necessary and sufficient to account for some observations (knowledge that is perceived from the world) in a given environment, useful to explain past events or to predict future situations in that environment. Inductive learning aims at extracting a model of a concept, given a set of examples (tags concerning the concepts to be learned associated to observations) for that concept. Examples can be positive, representing instances of the concept, or negative, representing instances that do not belong to the concept. ILP joins the objectives of Inductive Learning and LP: it aims at learning (inducing) concept descriptions as logic programs or integrity constraints valid in a given database, starting from a set of literals expressing (positive and negative) examples of the concepts to be learned. We use direct relevance [23] in the form of a Horn clause with the example in the head and the relevant observation in the body (e.g., $uncle(sam,bob)-parent(tim,bob),brother(sam,tim)$ is a positive example for concept uncle/2; a negative example for uncle/2 is: $\neg uncle(mary,bob)-parent(tim,bob),female\left(mary\right)$.

In indirect relevance [24], the observation that is relevant to an example must be obtained from the available knowledge. The examples above would be represented as just $uncle(sam,bob)$ and $\neg uncle(mary,bob)$).

Definition 5 

(Inductive Learning paradigm). A theory T is a set of hypotheses. A hypothesis H is a set of program clauses with the same predicate in their head, i.e., defining the same concept. An example for a hypothesis H has the same predicate in the head as H. It is negative for H if negated, positive otherwise.

Given:

• a set of examples $E=E^{+}\cup E^{-}$, where $E^{+}$ are the positive ones and $E^{-}$ are the negative ones;
• a (possibly empty) background knowledge (or BK) B.

Find a theory (logic program, model) T such that (Knowing if a solution exists is the Finite Axiomatizability problem (FA-problem): finding a set of axioms that explains a potentially infinite set of observations).

• $T\cup B⊢E^{+}$                                                     (completeness or sufficiency)
• $T\cup B\cup E^{-}̸⊢\square$                               ((prior or strong ) consistency) (If the calculus were complete, we could write $T\cup B̸⊢E^{-}$).

Moreover, the following properties are fulfilled:

• $B̸⊢E^{+}$                                                         ((prior) necessity)
• $B\cup E^{-}̸⊢\square$                                                      (prior consistency)
• $T\cup B̸⊢\square$                                                        (weak consistency).

A theory that explains a negative example is inconsistent (or too strong , i.e., too general); it makes a commission error and needs to be specialized. A theory that does not explain a positive example is incomplete (or too weak , i.e., too specific); it makes an omission error and needs to be generalized. In both cases, the theory is incorrect.

Since the examples are the only source of knowledge available for learning, the learning process has to be progressive. Concept definitions inferred starting from limited observations cannot be guaranteed to be universally correct: new observations can reveal that the current formulation of a concept is wrong. Most research in ILP focuses on batch approaches that take all the available examples and return a theory that explains them. When the theory fails on new evidence, the whole process must be restarted from scratch, taking no advantage of the previous theory. On the other hand, incremental approaches can adjust the existing theory without completely rejecting it, but taking the previously generated hypotheses as the starting point of a search process whose goal is a new theory that explains both old and new observations. Incremental synthesis of theories is necessary in several cases, like changing world, sloppy modeling, and selection bias [25]. The two approaches have complementary advantages and drawbacks: the former usually yields more compact and elegant theories, but wastes computational resources for recomputing the theory by starting from scratch; in the latter, the system can just revise an existing hypothesis on the ground of new evidence, but the resulting theory might not be very elegant. So, a cooperation between the two is desirable.

The incremental approach is more complex since it involves two steps: an abstract diagnosis to detect the source of incorrectness in the theory and a debugging of the theory in order to fix it and restore correctness. Note that for commission errors the diagnosis can detect a single inconsistent clause while for omission errors it can just detect the hypothesis that does not account for the example.

A generality ordering ≤ on formulas can be interpreted from the induction perspective:

• $C\le D$ : C is a generalization of D, D is a specialization of C;
• $C<D$ : C is a proper generalization of D, D is a proper specialization of C;

or, from a theory refinement viewpoint,

• $C\le D$ : C is an upward refinement of D, D is a downward refinement of C;
• $C<D$ : C is a proper upward refinement of D, D is a proper downward refinement of C.

As to the debugging process, commission errors can be solved by a downward refinement operator; dually, upward refinement operators can cope with omission errors:

Definition 6 

(Refinement operators). Let $(S,\le )$ be a quasi-ordered set.

A downward refinement operator ρ is a mapping such that $\forall C\in S:\rho \left(C\right)\subseteq \{D\in S\mid C\le D\}$, i.e., it computes a subset of all specializations of C.

An upward refinement operator δ is a mapping such that $\forall C\in S:\delta \left(C\right)\subseteq \{D\in S\mid D\le C\}$, i.e., it computes a subset of all generalizations of C.

The debugging process aims at finding a minimal refinement of a theory [26]. In the deductive process, the fundamental operation is unification, allowing to find a common instance of two clauses (if any). Its dual, that plays an analogous role in induction, is generalization, that allows to find (if any) a clause of which two given clauses are both instances. So, just like one looks for the most general unifier in order not to miss deduction steps, for the same reason one looks for the least general generalization (lgg) in induction as the only correct generalization (if it covers a negative example, there is no more specific generalization not to cover it).

Definition 7 

(Least general generalization under $\theta$ subsumption).

A clause C is a least general generalization under $\theta$ subsumption ($\mathit{lgg}$) of two clauses $C_1$ and $C_2$ iff it is more specific of (or equivalent to) any other generalization of $C_1$ and $C_2$. Formally,

$$\mathit{lgg}(C_1,C_2)=\{C\mid C{\le }_{\theta }{C}_i,i=1,2\wedge \forall D\mathrm{s}.\mathrm{t}.D{\le }_{\theta }{C}_i,i=1,2:D{\le }_{\theta }C\}$$

This definition can be extended to any finite set of clauses S: being the space of clauses ordered by $\theta$ subsumption, a lattice [27] $\mathit{lgg}\left(S\right)$ exists and is unique. The empty clause, being a subset of any other clause, subsumes them all, but (obviously) is not equivalent to any of them. The algorithm for computing the lgg between two any clauses (not necessarily Horn ones) was given by Plotkin [27]. An $\mathit{lgg}$ of a set S of n clauses, whose maximum length is m, has at most length $m^n$.

Definition 8 

(Ideality). Let $(S,\le )$ be a quasi-ordered set, ρ a downward refinement operator, and δ an upward refinement operator.

• ρ (respectively, δ) is locally finite iff $\forall C\in S$: $\rho \left(C\right)$ (respectively, $\delta \left(C\right)$) is finite and computable.
• ρ is proper iff $\forall C\in S$: $\rho \left(C\right)\subseteq \{D\in S\mid C<D\}$
  δ is proper iff $\forall C\in S$: $\delta \left(C\right)\subseteq \{D\in S\mid D<C\}$.
• ρ is complete iff $\forall C,D\in S$: $C<D\Rightarrow \exists E$ such that $E\in {\rho }^{*}\left(C\right)$ and $E\sim D$
  δ is complete iff $\forall C,D\in S$: $D<C\Rightarrow \exists E$ such that $E\in {\delta }^{*}\left(C\right)$ and $E\sim D$.
• ρ (respectively, δ) is ideal iff it fulfills all the above three properties.

Ideality plays a key role for efficiency and effectiveness. Unfortunately, there exist no ideal refinement operators when using full Horn clause logic as representation language and either $\theta$ subsumption or implication as the generalization model because of the existence of infinite unbound strictly ascending/descending chains [28,29,30].

3. Object Identity

Object identity (OI) was first defined in [31,32] for a FOL that is both function-free and constant-free, and later generalized to a full FOL [33,34]. It is based on the following assumption:

Within a clause, terms denoted by different symbols must be distinct.

In Datalog, the adoption of the OI assumption can be viewed as a method for building an equational theory into both the ordering and the inference rules of the calculus [35] by adding one rewrite rule to the axioms of Clark’s Equality Theory (CET) [1]:

$$t\ne s\in body\left(C\right)\forall C \mathrm{clause} \mathrm{in} L \mathrm{and} \forall (t,s) \mathrm{pair} \mathrm{of} \mathrm{distinct} \mathrm{terms} \mathrm{in} \mathrm{terms}\left(C\right)\left(\mathbf{OI}\right)$$

where L denotes the language of all the Datalog clauses built from a finite number of predicates. It can be viewed as an extension of both Reiter’s unique-names assumption [36] and Axioms (7), (8), (9) of CET to the variables of the language. The resulting language, Datalog^OI (a sublanguage of Datalog^≠), is an instance of Constraint LP (which extends LP by allowing constraints in the body of clauses) [37]. Under OI, any Datalog clause $C=l_0:-l_1,\dots ,l_n$ generates a Datalog^≠ clause $C_{OI}=l_0:-l_1,\cdots ,l_n\Vert I$ consisting of two components:

• $core\left(C_{OI}\right)=C$;
• $constraints\left(C_{OI}\right)=I=\{t\ne s\mid t,s\in terms\left(C\right)\wedge t,s$ distinct}.

Example 2. 

The Datalog clause $C=p\left(X\right):-q(X,X),q(Y,a)$, where X, Y are variables and a is a constant, corresponds to the Datalog^OI clause $C_{OI}=p\left(X\right):-q(X,X),q(Y,a)\Vert [X\ne Y],[X\ne a],[Y\ne a]$.

Nevertheless, Datalog^OI has the same expressive power as Datalog, since any Datalog clause/program admits a set of Datalog^OI clauses equivalent to it:

Proposition 1 

([38]). $\forall C\in Datalog\exists C^{\prime }\subseteq Datalo{g}^{OI}:T_C\uparrow \omega =T_{C^{\prime }}\uparrow \omega$

Proof. 

See Algorithm 2.    □

Algorithm 2: Algorithm to compute the OI equivalent of a Datalog clause

function GenerateDatalogOIClauses (C: DatalogClause): DatalogOIClause begin $C^{\prime }$ := {}; for $k=0,1,\dots ,\mid vars\left(C\right)\mid$ do    foreach combination of k variables out of $\mid vars\left(C\right)\mid$ do       begin       Define some ordering between the k variables;       foreach permutation with replacement of k constants out of $\mid consts\left(C\right)\mid$ do          for $h=1,2,\dots ,\mid vars\left(C\right)\mid -k$ do             begin             $C_h^{\prime }$ := {};             foreach partition ${\left(V_i\right)}_{i=1,2,\dots ,h}$ of the remaining variables such that             $\forall i,j=1,2,\dots ,h,i<j:\mid V_i\mid =r_i,r_i\le r_j$ do                begin                $C_h^{\prime }$ := {};                Build a clause D by replacing the l-th ($l=1,2,\dots ,k$) variable                of the combination with the l-th constant of the permutation                and $\forall i=1,2,\dots ,h$ all the $r_i$ variables belonging to $V_i$ with                one new variable;                if $\forall i,j=1,2,\dots ,k,i\ne j:r_i\ne r_j$ then Insert D in $C_h^{\prime }$                elsif there exists no renaming of D in $C_h^{\prime }$ then Insert D in $C_h^{\prime }$                end;             $C^{\prime }$ := $C^{\prime }\cup C_h^{\prime }$             end;       end; return $C^{\prime }$ end;

Corollary 3 

([38]). $\forall P\subseteq Datalog$$\exists P^{\prime }\subseteq Datalo{g}^{OI}:T_P\uparrow \omega =T_{P^{\prime }}\uparrow \omega$.

Proof. 

$P^{\prime }$ is the program obtained by translating each clause $C\in P$.    □

Lemma 1 

([38]). Let C be a Datalog clause such that $\mid vars\left(C\right)\mid =n$ and $\mid consts\left(C\right)\mid =m$. The number of Datalog^OI clauses generated by Algorithm 2 (that include renamings) is

$$\sum _{k=0,n}\left(\begin{array}{c}n\\ k\end{array}\right)m^k\sum _{h=1,n-k}\sum _{\begin{array}{c}{r}_1\le \dots \le r_h\\ r_1+\dots +r_h=n-k\end{array}}\left(\begin{array}{c}n-k\\ r_1,\dots ,r_h\end{array}\right)$$

The number of distinct clauses is obtained by dividing each multinomial coefficient by $n_1!\cdots n_l!$, where $n_i$ is the number of $r_j$s in that coefficient that are equal to i and l is the greatest value of $r_j$s in that coefficient.

Proof. 

Each step of the outer loop in Algorithm 2 fixes a different k ($0\le k\le n$). Then, all possible combinations of k variables are generated (samples of k objects out of n without replacement and without taking into account the ordering that are a binomial coefficient), and to each of them one of the possible dispositions with replacement of that many constants in C (samples of k objects out of m with replacement and taking into account the ordering that are $m^k$) is associated.

The remaining $n-k$ variables are handled by the inner loop as follows. Each step fixes a different h ($1\le h\le n-k$) and generates all possible partitions of $vars\left(C\right)$ made up of h elements sorted by increasing cardinality (samples of dimension $n-k$, without replacement, taking into account their ordering—apart from the equivalent ones, i.e., those belonging to the same partition element—that are a multinomial coefficient). By the addition principle, this yields the inner summation of the formula.

Applying the multiplication principle to this sequence of operations yields the formula to be proved. The division to obtain the number of distinct clauses comes from the fact that partitions are considered distinct even if they were obtained by permutation of elements with the same cardinality. Thus, for each group of $n_i$ elements having the same cardinality i, it suffices dividing by the number of their permutations that are $n_i!$    □

Lemma 1 guarantees termination of Algorithm 2 since there is a finite number of Datalog^OI clauses corresponding to a given Datalog one.

The cases in which many arguments of a predicate must have the same value is solved under OI by introducing different variations of the predicate, associated to the possible cases of unification among arguments. E.g., given predicate $p/3$, if the first and third arguments can take the same value, we must introduce a new predicate $p_{3,1,3}/2$ representing these cases, defined as $p_{3,1,3}(X,Y)\equiv p(X,Y,X)$. Of course, to consider all possible cases of unification among arguments, a combinatorial number of new predicates is required. This technique also applies to the flattening predicates in order to deal with the case of functions where different arguments can take the same value.

These are only theoretical results to guarantee that there is no loss in expressive power from Datalog to Datalog^OI The significant growth in the number of clauses and predicates according to the procedures above is not a limitation in practice, since only the variations that are needed must be included in the theory. E.g., in ILP, a bound to the number of clauses in a theory that must explain a number of examples is the very number of examples. On the other hand, the OI assumption brings several advantages from theoretical, practical, and intuitive viewpoints (see Section 5 and Section 6).

All the FOL concepts introduced in Section 2 can be applied to the OI setting. In the following, they are redefined only when under OI they have a different behavior or specification. From now on, the OI subscript denotes the application of OI to the standard FOL concepts.

4. Generalization Models

The OI assumption changes the classical ordering relations on the representation language, restructuring the corresponding search spaces.

4.1. ${\theta }_{OI}$ Subsumption

Definition 9 

(${\theta }_{OI}$ subsumption [33,34]). Let C, D be Datalog clauses. C $\theta$-subsumes D under OI (C ${\theta }_{OI}$-subsumes D, in symbols $C{\le }_{OI}D$) iff $\exists \sigma$ substitution is such that $C_{OI}.\sigma \subseteq D_{OI}$.

Example 3. 

 

• $C:p\left(Y\right)\leftarrow q(Y,Z).$                    $D:p\left(X\right)\leftarrow q(X,a).$                    $D^{\prime }:p\left(X\right)\leftarrow q(X,a),r\left(b\right).$
  $C{<}_{OI}D:\exists \sigma =\{X/Y,a/Z\}\mathrm{s}.\mathrm{t}.C_{OI}.\sigma =D_{OI}\wedge \neg \exists \theta \mathrm{s}.\mathrm{t}.D_{OI}.\theta \subseteq C_{OI}$
  $C{<}_{OI}{D}^{\prime }:\exists \sigma =\{X/Y,a/Z\}\mathrm{s}.\mathrm{t}.C_{OI}.\sigma \subset D_{OI}^{\prime }\wedge \neg \exists \theta \mathrm{s}.\mathrm{t}.D_{OI}^{\prime }\theta \subseteq C_{OI}$
• $C:p\left(X\right)\leftarrow q\left(Y\right).$                        $D:p\left(W\right)\leftarrow q\left(Z\right).$                         $C{\sim }_{OI}D$
  $\exists \theta =\{W/X,Z/Y\}:C_{OI}\theta \subseteq D_{OI}$, $\exists \sigma =\{X/W,Y/Z\}:D_{OI}\sigma \subseteq C_{OI}$
• $C:p\left(X\right)\leftarrow q(X,a).$                      $D:p\left(Y\right)\leftarrow q(Y,Z),r\left(a\right).$                   are not comparable
  $\neg \exists \sigma \mathrm{s}.\mathrm{t}.C_{OI}\sigma \subseteq D_{OI},\neg \exists \theta \mathrm{s}.\mathrm{t}.D_{OI}\theta \subseteq C_{OI}$

Like $\theta$ subsumption, ${\theta }_{OI}$ subsumption induces a quasi-ordering upon the space of Datalog clauses, as stated by the following result.

Proposition 2. 

Let C, D, E be Datalog clauses. Then

1. 

$C{\le }_{OI}C$;                                                            (reflexivity)

2. 

$C{\le }_{OI}D$ and $D{\le }_{OI}E\Rightarrow C{\le }_{OI}E$.                                                  (transitivity)

Proof. 

1.  $C\subseteq C\Rightarrow C_{OI}.\left\{\right\}\subseteq C_{OI}\Rightarrow C{\le }_{OI}C$.

2.  If $C{\le }_{OI}D$ and $D{\le }_{OI}E$, then $\exists \sigma ,\theta$ substitutions are such that $C_{OI}\sigma \subseteq D_{OI}\wedge D_{OI}\theta \subseteq E_{OI}$. Thus, $C_{OI}\theta \sigma \subseteq D_{OI}\sigma \subseteq E_{OI}$. This proves $C{\le }_{OI}E$.

   □

A computational characterization of ${\theta }_{OI}$ subsumption is

Proposition 3. 

Let C, D be Datalog clauses,

$C{\le }_{OI}D\iff \exists \sigma \mathrm{s}.\mathrm{t}.core\left(C_{OI}\right)\sigma \subseteq core\left(D_{OI}\right)\wedge constraints\left(C_{OI}\right)\sigma \subseteq constraints\left(C_{OI}\right)$.

Proof. (⇐)Trivial.

(⇒)

From Definition 9, $C{\le }_{OI}D\iff \exists \sigma$ substitution is such that $C_{OI}.\sigma \subseteq D_{OI}\iff$

$(core\left(C_{OI}\right)\cup constraints\left(C_{OI}\right))\sigma \subseteq core\left(D_{OI}\right)\cup constraints\left(D_{OI}\right)\iff$

$(core\left(C_{OI}\right)\sigma \cup constraints\left(C_{OI}\right)\sigma )\subseteq core\left(D_{OI}\right)\cup constraints\left(D_{OI}\right)$.

Since inequalities cannot occur in a Datalog clause, $\mathit{core}(·)$ and $\mathit{constraints}(·)$ are always disjoint. So, $core\left(C_{OI}\right)\sigma \subseteq core\left(D_{OI}\right)\wedge constraints\left(C_{OI}\right)\sigma \subseteq constraints\left(D_{OI}\right)$.

   □

${\theta }_{OI}$ subsumption is weaker than $\theta$ subsumption:

Proposition 4. 

Let C, D be Datalog clauses. $C{\le }_{OI}D\Rightarrow C{\le }_{\theta }D$.

Proof. 

In Proposition 3, one further property ($constraints\left(C_{OI}\right)\sigma \subseteq constraints\left(D_{OI}\right)$) is required in addition to $\theta$ subsumption ($core\left(C_{OI}\right)\sigma \subseteq core\left(D_{OI}\right)$).    □

Proposition 5. 

Let C, D be Datalog clauses.

1. 

$C{\le }_{OI}D\Rightarrow \forall \sigma$ such that $C_{OI}\sigma \subseteq D_{OI}:$

        σ injective $\wedge \sigma :vars\left(C\right)\to vars\left(D\right)\cup \left(consts\right(D)\setminus consts(C\left)\right)$.

2. 

$C{\le }_{OI}D\Rightarrow \left|C\right|\le \left|D\right|$.

3. 

$C{\sim }_{OI}D$ iff they are variants.

Proof. 

(sketch) The constraints require substitutions to be injective and map variables in D onto terms in C that in Datalog are just variables (in C) or constants (not already occurring in D). If $consts\left(D\right)=\varnothing$, it suffices for the substitution to be injective; if there are constants, injectivity alone would not fulfill transitivity of ${\le }_{OI}$, so it would not be an ordering.

Requiring that terms are distinct means that literals cannot unify among each other; hence, ${\theta }_{OI}$ subsumption maps each literal of the subsuming clause onto a single, different literal in the subsumed one. So, equivalent clauses under ${\le }_{OI}$ must have the same number of literals; hence, the only way to be equivalent is to be variants (this does not hold under implication and $\theta$ subsumption).    □

Thus, in a search space ordered by ${\theta }_{OI}$ subsumption, there is no subset of a clause that is equivalent to the clause itself under OI. This yields smaller equivalence classes than those in a space ordered by $\theta$ subsumption.

The sets $core(·)$ and $constraints(·)$ (and hence ${\theta }_{OI}$ subsumption) can be straightforwardly extended from Datalog^OI to generic clauses, leveraging the fact that substitutions are mappings [39], and thus can be required to be injective:

Definition 10 

(OI-substitution). Let T be a set of terms and σ a substitution. σ is an OI-substitution with respect to T iff $\forall t_1,t_2\in T:t_1\ne t_2\Rightarrow t_1\sigma \ne t_2\sigma$.

(Non-injective substitutions yield contradictions when applied to constraints: e.g., $[x\ne y].\{x/a,y/a\}=[a\ne a]$). This yields another characterization for ${\theta }_{OI}$ subsumption.

Proposition 6. 

Let $C,D$ be clauses. $C{\le }_{OI}D$ iff $\exists \sigma$ OI substitution with respect to $terms\left(C\right)$ such that $C.\sigma \subseteq D$.

Proof. (⇐)Trivial.

(⇒)

If $C{\le }_{OI}D$, then $core\left(C_{OI}\right)\sigma \subseteq core\left(D_{OI}\right)\wedge constraints\left(C_{OI}\right)\sigma \subseteq constraints\left(D_{OI}\right)$

(by Proposition 3 extended to generic clauses). By contradiction, let $t_1,t_2\in \mathit{terms}\left(C_{OI}\right)$, $t_1\ne t_2$ such that $t_1\sigma =t_2\sigma =t$. By definition of $C_{OI}$,

$t_1,t_2\in \mathit{terms}\left(C_{OI}\right)\wedge t_1\ne t_2\Rightarrow t_1\ne t_2\in \mathit{constraints}\left(C_{OI}\right)\Rightarrow$

$[t_1\ne t_2]\sigma =[t_1\sigma \ne t_2\sigma ]=[t\ne t]\in \mathit{constraints}\left(D_{OI}\right)$,

which is a contradiction by definition of $\mathit{constraints}(·)$.

   □

Example 4. 

${\theta }_{OI}$-subsumption in unrestricted space

$C:p\left(f\right(X\left)\right):-p\left(f\right(X),Y).$                    $D:p\left(f^2\left(a\right)\right):-q(f^2\left(a\right),f^2\left(a\right)).$

$C{\le }_{\theta }D$ using substitution $\theta =\{f\left(a\right)/X,f^2\left(a\right)/Y\}$, but the terms $f\left(X\right)$ and Y ($f\left(X\right)\ne Y$) are both mapped to $f^2\left(a\right)$ by θ; thus, $C\neg {\le }_{OI}D$.

As a consequence, Properties 2 and 3 in Proposition 5 hold also for unrestricted clauses, since injectivity of the substitutions involved is the only required property of the Datalog^OI language: under OI, clauses are non-redundant and cannot be further reduced. Hence, both definitions of resolution are, by themselves, complete by refutation.

Note that, since $\theta$ subsumption is decidable, ${\theta }_{OI}$ subsumption, as a restriction allowing only OI-substitutions, must also be decidable.

Definition 11 

(${\mathit{lgg}}_{OI}$). A least general generalization under ${\theta }_{OI}$ subsumption of two clauses is a generalization which is not more general than (i.e., it is either more specific than or not comparable to) any other such generalization. Formally, given two Datalog clauses $C_1$ and $C_2$,

$lg{g}_{OI}(C_1,C_2)=\{C\mid C{\le }_{OI}{C}_i,i=1,2\wedge \forall D$ such that $D{\le }_{OI}{C}_i,i=1,2:C\neg {<}_{OI}D\}$.

The ${\mathit{lgg}}_{OI}$ is not unique since the space of Datalog clauses ordered by ${\le }_{OI}$ is not a lattice [32,40]. It can be computed using the algorithm in Algorithm 3, a straightforward extension to Datalog^OI of Plotkin’s algorithm [27], that takes into account the (OI) rule in order to determine a partition of the literals in $G_{core}$ that allows to find the set of all the possible lggs under OI rather than the unique lgg under $\theta$ subsumption. An equivalence test is needed, after the algorithm, to eliminate duplicate lggs. The reduced lggs of two clauses are in finite number and specifically

$$\mid lg{g}_{OI}(C_1,C_2)\mid <2^{n}n=min(\mid C_1\mid ,\mid C_2\mid )$$

since, as a consequence of the OI constraint, the literals in the generalization must be mapped one to one onto literals in the generalized clauses.

Algorithm 3: Algorithm for computing the lggOI

Let us preliminarily define a selection under OI of two DatalogOI clauses $C_1,C_2$ as a pair of literals $(c_i\in core\left(C_1\right),d_j\in core\left(C_2\right))$ that have the same predicate symbol, sign, and arity. Given two DatalogOI clauses, ${\mathit{lgg}}_{OI}$$(C_1,C_2)$ is a set of DatalogOI such that for each clause $G_{OI}$, core $\left(G_{OI}\right)=\{g\mid g=lgg(c_i,d_j)$, $(c_i,d_j)$ selection under OI of $C_1,C_2$ and ${\phi }_k\ne {\phi }_k$, $k=1,2\}$ where $lgg(c_i,d_j)$ is computed using Plotkin’s algorithm and ${\phi }_k,k=1,2$, are the substitutions such that core $\left(G_{OI}\right){\phi }_k$⊆$core\left(C_k\right)$, $k=1,2$. More precisely, given the mapping $$\mathrm{\Phi }:\mathit{terms}\left(C_1\right)\times \mathit{terms}\left(C_2\right)⟶\mathit{N}\cup (\mathit{consts}\left(C_1\right)\cap \mathit{const}\left(C_2\right))\mathrm{s}.\mathrm{t}.$$ $$\mathrm{\Phi }(t_i,s_i)=\left\{\begin{array}{cc}{t}_i& \mathit{if}{t}_i=s_i\\ X\in N& \mathit{otherwise}\end{array}\right.$$ where $(P(t_1,t_2,\dots ,t_n),P(s_1,s_2,\dots ,s_n))$ is a selection under OI of $C_1$ and $C_2$, N denotes a set of new variables, and ${\phi }_1,{\phi }_2$ are the projections of the inverse function of $\Phi$ onto $terms\left(C_1\right)$ and $terms\left(C_2\right)$, respectively. constraints $\left(G_{OI}\right)=\{x\ne y\mid {\phi }_k\left(x\right)\ne {\phi }_k\left(y\right),k=1,2\}$

Example 5. 

 

$C:bicycle\left(X\right):-wheel(X,b),wheel(X,X^{\prime }),red\left(c\right)$;

$D:bicycle\left(Y\right):-wheel(a,Y^{\prime }),stripes\left(d\right)$.

The selections along with the unification functions are

$$\begin{array}{cccc}\langle \mathit{bicycle}\left(X\right),\mathit{bicycle}\left(Y\right)\rangle \hfill & \Phi (X,Y)=W_1\hfill & \underline{{\phi }_1\left(W_1\right)=X}\hfill & {\phi }_2\left(W_1\right)=Y\hfill \\ \langle \mathit{wheel}(X,b),\mathit{wheel}(a,Y^{\prime })\rangle \hfill & {\Phi }_1(X,a)=W_2\hfill & \underline{{\phi }_1\left(W_2\right)=X}\hfill & {\phi }_2\left(W_2\right)=a\hfill \\ & {\Phi }_1(b,Y^{\prime })=W_3\hfill & {\phi }_1\left(W_3\right)=b\hfill & \underline{{\phi }_2\left(W_3\right)=Y^{\prime }}\hfill \\ \langle \mathit{wheel}(X,X^{\prime }),\mathit{wheel}(a,Y^{\prime })\rangle \hfill & {\Phi }_1(X,a)=W_2\hfill & \underline{{\phi }_1\left(W_2\right)=X}\hfill & {\phi }_2\left(W_2\right)=a\hfill \\ & {\Phi }_2(X^{\prime },Y^{\prime })=W_4\hfill & {\phi }_1\left(W_4\right)=X^{\prime }\hfill & \underline{{\phi }_2\left(W_4\right)=Y^{\prime }}\hfill \end{array}$$

Since ${\phi }_1$ and ${\phi }_2$ are not injective (see the underlined bindings), a partition among variables is determined (and hence among literals containing them) such that they cannot occur in the same generalization ($W_1$ is incompatible with $W_2$, $W_3$ is incompatible with $W_4$). So ${\mathit{lgg}}_{OI}$$(C,D)=$

$\{\mathit{bicycle}\left(W_1\right):-,$$:-\mathit{wheel}(W_2,W_3)\Vert W_2\ne W_3,$$:-\mathit{wheel}(W_2,W_4)\Vert W_2\ne W_4\}$

The last two clauses are equivalent (i.e., variants), so one of them must be removed.

4.2. OI Implication

An OI-compliant form of implication can be defined.

4.2.1. Model-Theoretic Definition

OI interpretations are always normal interpretations, while of course the converse is not true. An OI model may not be a model. Under OI, for variable bindings, the mapping is required to be one to one:

Definition 12 

(Variable binding under OI). Let I be an interpretation of L with domain D. A variable binding under OI with respect to L is a one-to-one mapping $V:\mathit{vars}\left(L\right)\to D$.

Example 6.

(OI model). The OI interpretation $I=\left\{p\right(a,b\left)\right\}$ on the domain $D=\{a,b\}$ is an OI model for $C=\left\{p\right(a,X)\vee q(a\left)\right\}$. I is not a model for C in general. Indeed, since $q\left(a\right)$ is false, the truth of the clause depends on $p(a,X)$, where variable X is universally quantified. Under OI, it is sufficient that $p(a,b)$ be true for $\forall Xp(a,X)$ to hold, while in general both $p(a,a)$ and $p(a,b)$ should be true.

Definition 13 

(OI implication). A formula ϕ is a logical consequence under OI of a set of formulas Σ ($\mathrm{\Sigma }{\vDash }_{\mathit{O}\mathit{I}}\mathit{\varphi }$) iff all OI models of Σ are also OI models of ϕ.

Given two formulas C and D, C OI implies D ($C{\Rightarrow }_{OI}D$) iff any OI model for C is also an OI model for D ($C{\vDash }_{OI}D$). Equivalence is denoted by ${\iff }_{OI}$.

Useful results hold also in this semantics:

Theorem 4 

(Deduction Theorem). Let Σ be a set of formulas and ϕ and ψ be formulas. $\mathrm{\Sigma }\cup \left\{\mathit{\varphi }\right\}{\vDash }_{\mathit{O}\mathit{I}}\mathit{\psi }$ iff $\mathrm{\Sigma }{\vDash }_{\mathit{O}\mathit{I}}(\mathit{\varphi }\Rightarrow \mathit{\psi })$.

Proof. 

$\mathrm{\Sigma }\cup \left\{\varphi \right\}{\vDash }_{OI}\psi$ iff all OI models of $\mathrm{\Sigma }\cup \left\{\varphi \right\}$ are OI models of $\psi$ iff

all OI models of $\mathrm{\Sigma }$ are OI models of $\neg \varphi$ or of $\psi$ iff

all OI models of $\mathrm{\Sigma }$ are OI models of $\neg \varphi \vee \psi$ iff $\mathrm{\Sigma }{\vDash }_{OI}(\varphi \Rightarrow \psi )$.    □

As a consequence, it is possible to also prove this result:

Theorem 5. 

Given a set of formulas Σ and a formula ϕ, $\mathrm{\Sigma }{\vDash }_{\mathit{O}\mathit{I}}\mathit{\varphi }$ iff $\mathrm{\Sigma }\cup \{\neg \mathit{\varphi }\}$ is unsatisfiable.

Proof. 

$\mathrm{\Sigma }{\vDash }_{OI}\varphi$ iff (by Theorem 4) $\mathrm{\Sigma }\cup \{\neg \varphi \}{\vDash }_{OI}\square$ iff

all OI models of $\mathrm{\Sigma }\cup \{\neg \varphi \}$ are OI models of □ iff $\mathrm{\Sigma }\cup \{\neg \varphi \}$ is unsatisfiable under OI.    □

Herbrand (OI) models only can be considered, like in normal logic.

Proposition 7. 

Let Σ be a set of clauses in a first order language L. Σ has an OI model iff Σ has a Herbrand OI model.

Proof. (⇐)Trivial.

(⇒)

Given an OI model I for $\Sigma$, consider the Herbrand OI interpretation $\overline{I}$ having $U_{\Sigma }$ as the domain. For every n ary predicate P in $\Sigma$, the interpretation ${\overline{I}}_P(t_1,\dots ,t_n)$ is true if $P(t_1,\dots ,t_n)$ is true in I, or false otherwise.

   □

4.2.2. Proof-Theoretic Definition

Applying the OI assumption to the subsumption theorem for implication yields a constructive definition of OI implication. OI resolution ${\mathcal{R}}_{OI}$ is defined just like resolution $\mathcal{R}$, except that ${\mathit{mgu}}_{OI}$s only are allowed. Moreover, equivalent clauses of a resolvent in ${\mathcal{L}}_{OI}^{*}\left(\left\{C\right\}\right)$ are not considered, since under OI they are merely variants. Like resolution, OI resolution is sound and complete by refutation.

Lemma 2. 

Let $C_1$ and $C_2$ be clauses. If $\exists R={\mathcal{R}}_{OI}(C_1,C_2)$ then $\{C_1,C_2\}{\vDash }_{OI}R$.

Proof. 

Suppose R is an OI resolvent of $C_1$ and $C_2$ upon a and b by $\theta ={\mathit{mgu}}_{OI}(a,\overline{b})$, and I is an OI model (with domain D) of $\{C_1,C_2\}$. Let $X_1,\dots ,X_n=\mathit{vars}\left(C_1\right)\cup \mathit{vars}\left(C_2\right)$ and V be a variable binding under OI. Then, if $a\theta$ is false under I and $V(X_1/d_1)\cdots (X_n/d_n)$, at least one of the other literals in $C_1$ is true under I and $V(X_1/d_1)\cdots (X_n/d_n)$, $d_i\in D$. The case is similar for $\overline{b}$.

R is made up of all literals in $C_1\theta$ and $C_2\theta$ except for $M\theta$ and $N\theta$. Since either $M\theta$ or $N\theta$ is false under I and $V(X_1/d_1)\cdots (X_n/d_n)$, at least one of the literals in R is true under I and $V(X_1/d_1)\cdots (X_n/d_n)$; hence, I is an OI model of R. Therefore, $\{C_1,C_2\}{\vDash }_{OI}R$.    □

Extending this result to OI derivations, soundness of OI resolution is proven:

Theorem 6 

(OI derivation Soundness). Let Σ be a set of clauses, and C be a clause. If $\Sigma {⊢}_{OI}C$, then $\Sigma {\vDash }_{OI}C$.

Proof. 

$\Sigma {⊢}_{OI}C\Rightarrow \exists \langle R_1,\dots ,R_k=C\rangle$ OI derivation from $\Sigma$. By induction on k,

$(k=1)$

$R_1=C\in \Sigma$; thus, obviously, $\Sigma {\vDash }_{OI}C$.

$(k>1)$

Suppose the thesis holds for $k\le m$. Let $\langle R_1,\dots ,R_{m+1}=C\rangle$ be an OI derivation of C from $\Sigma$. If $R_{m+1}\in \Sigma$; then, the theorem is obvious. Otherwise, $R_{m+1}$ is an OI resolvent of some $R_i$ and $R_j$ ($i,j\le m$). By induction hypothesis, $\Sigma {\vDash }_{OI}{R}_i$ and $\Sigma {\vDash }_{OI}{R}_j$. From Lemma 2, it follows that $\{R_i,R_j\}{\vDash }_{OI}{R}_{m+1}=C$.

   □

It is now possible to define

Definition 14 

(OI implication). Let C, D be clauses. C implies D under OI (or C OI implies D, $C{\Rightarrow }_{OI}D$) iff D is a tautology or $\exists E\in {\mathcal{R}}_{OI}^{*}\left(\left\{C\right\}\right)$ such that $E{\le }_{OI}D$.

Example 7. 

$C:p\left(a\right):-q(a,b),p\left(b\right).$    $D:p\left(c\right):-q(c,d),q(d,e),p\left(e\right).$

$E:p\left(X\right):-q(X,Y),q(Z,W),p\left(W\right).$     $F:p\left(U\right):-q(U,V),p\left(V\right).$

$E{\le }_{\theta }C$ and $E{\le }_{\theta }D$. $F{\le }_{I}C$ and $F{\le }_{I}D$. $E{<}_{\theta }F$.

Under OI, $E\neg {\le }_{OI}C$ (since E has more literals) and $E\neg {\le }_{OI}D$ (since terms Y and Z cannot be mapped onto the same term d); $F{\le }_{OI}C$ (trivial) and $F{\Rightarrow }_{OI}D$ (since, by self-resolving F, we have: ${\mathcal{R}}_{OI}\left(\left\{F\right\}\right)=\{p\left(X\right):-q(X,Y),q(Y,Z),p\left(Z\right)\}{\le }_{OI}D$.

Concerning the ordering relationships, ${\theta }_{OI}$ subsumption is weaker than OI implication:

Proposition 8. 

Let C, D be Datalog clauses.

•

If $C{\Rightarrow }_{OI}D$ then $C\Rightarrow D$;

•

If $C{\le }_{OI}D$ then $C{\Rightarrow }_{OI}D$.

Figure 1 summarizes the relationships between the various generalization models.

The soundness of OI implication bridges the gap between its model-theoretic and proof-theoretic definitions. A completeness result bridges the opposite gap.

4.2.3. Subsumption Theorem

The simplest case occurs when both $\Sigma$ and C are ground:

Lemma 3. 

Let Σ be a ground logic program and C a ground clause. If $\Sigma {\Rightarrow }_{OI}C$, then $\exists D$ such that $\Sigma {⊢}_{OI}D\wedge D\subseteq C$.

Proof. 

As for implication [16] since, being the clauses ground, no variable assignment is made.    □

In order to prove the refutation completeness when only C is ground, we need to prove two theorems that are valid for standard notions of unsatisfiability (but hold also under OI) and to lift an OI resolution step from the case of ground parent clauses to that of unrestricted ones.

Theorem 7 

(Herbrand). A set of clauses Σ is unsatisfiable (under OI) iff $\exists \Gamma$ finite unsatisfiable set of ground instances of clauses from Σ.

Proof. 

Like in [9], but using Proposition 7.    □

Theorem 8 

(Deduction Theorem). Let $\Sigma \ne \varnothing$ be a set of clauses and C be a ground clause. Then $\Sigma {\Rightarrow }_{OI}C$ iff $\exists \Gamma$ finite set of ground instances of clauses from Σ such that $\Gamma {\Rightarrow }_{OI}C$.

Proof. 

Like in [9], but exploiting Theorems 4 and 7.    □

Lemma 4 

(Resolution Lifting). Let $C_1$, $C_2$ be clauses and $C_1^{\prime }$, $C_2^{\prime }$, respectively, two instances thereof. If $R^{\prime }$ is an OI resolvent of $C_1^{\prime }$ and $C_2^{\prime }$, then $\exists R$ OI resolvent of $C_1$ and $C_2$ such that $R^{\prime }$ is an instance of R.

Proof. 

By hypothesis:

• $\exists {\sigma }_1,{\sigma }_2$ OI substitutions are such that $C_1^{\prime }=C_1{\sigma }_1,C_2^{\prime }=C_2{\sigma }_2$.
• $\exists n_1^{\prime }\in C_1^{\prime },n_2^{\prime }\in C_2^{\prime },\mu ={\mathit{mgu}}_{OI}(n_1^{\prime },\overline{n_2^{\prime }})$ such that $R^{\prime }=((C_1^{\prime }\setminus \left\{n_1^{\prime }\right\})\cup (C_2^{\prime }\setminus \left\{n_2^{\prime }\right\}))\mu$.

Let $D_1^{\prime }=C_1^{\prime }\setminus \left\{n_1^{\prime }\right\}$ and $D_2^{\prime }=C_2^{\prime }\setminus \left\{n_2^{\prime }\right\}$: $\exists \left\{n_1\right\},D_1\subseteq C_1,\exists \left\{n_2\right\},D_2\subseteq C_2$ such that

$n_1^{\prime }=n_1{\sigma }_1,n_2^{\prime }=n_2{\sigma }_2$ and $D_1{\sigma }_1=D_1^{\prime },D_2{\sigma }_2=D_2^{\prime }$. We have that

$R^{\prime }=(D_1^{\prime }\cup D_2^{\prime })\mu =(D_1{\sigma }_1\cup D_2{\sigma }_2)\mu =$ ($C_1$, $C_2$, $C_1^{\prime }$, $C_2^{\prime }$ standardized apart) $(D_1\cup D_2){\sigma }_1{\sigma }_2\mu$.

Now, $\exists {\sigma }_1{\sigma }_2\mu$ OI unifier of $n_1$ and $\overline{n_2}\Rightarrow \exists \theta ={\mathit{mgu}}_{OI}(n_1,\overline{n_2})\Rightarrow$

$\exists \delta$ OI substitution such that ${\sigma }_1{\sigma }_2\mu =\theta \delta$.

Thus, $R=((C_1\setminus \left\{n_1\right\})\cup (C_1\setminus \left\{n_2\right\}))\theta =(D_1\cup D_2)\theta \Rightarrow$

$R\delta =(D_1\cup D_2)\theta \delta =(D_1\cup D_2){\sigma }_1{\sigma }_2\mu =R^{\prime }$.    □

The lifting technique can be generalized to many OI-resolution steps.

Lemma 5 

(Derivation Lifting). Let Σ be a (finite) set of clauses and ${\Sigma }^{\prime }$ a set of instances of clauses in Σ. If $\langle R_1^{\prime },\dots ,R_k^{\prime }\rangle$ is an OI derivation of clause $R_k^{\prime }$ from ${\Sigma }^{\prime }$, then $\exists \langle R_1,\dots ,R_k\rangle$ OI derivation of clause $R_k$ from Σ such that $\forall i:R_i^{\prime }$ is an instance of $R_i$.

Proof. 

By induction on the length of the derivation k.

($k=1$)

$R_1^{\prime }\in {\Sigma }^{\prime }\Rightarrow \exists R_1\in \Sigma$ such that $R_1^{\prime }$ is an instance of $R_1$.

($k>1$)

Let $\langle R_1^{\prime },\dots ,R_k^{\prime }\rangle$ be an OI derivation of $R_k^{\prime }$ from ${\Sigma }^{\prime }$; thus, $R_k^{\prime }$ is the resolvent of two clauses in ${\Sigma }^{\prime }\cup \{R_1^{\prime },\dots ,R_{k-1}^{\prime }\}$. By induction hypothesis, $\exists \langle R_1,\dots ,R_{k-1}\rangle$ OI derivation of $R_{k-1}$ such that $\forall i=1,\dots ,k-1:R_i^{\prime }$ is an instance of $R_i$. Hence, by Lemma 4, $\exists R_k$ such that $R_k^{\prime }$ is its instance.

   □

It is now possible to prove the following results:

Lemma 6. 

Let Σ be a logic program and C be a ground clause. If $\Sigma {\Rightarrow }_{OI}C$, then $\exists D$ such that $\Sigma {⊢}_{OI}D\wedge D{\le }_{OI}C$.

Proof. 

Assume C is not a tautology. By Theorem 8, if $\Sigma {\Rightarrow }_{OI}C$, then $\exists \Gamma$ finite set of ground instances of clauses in $\Sigma$ such that $\Gamma {\Rightarrow }_{OI}C$. Then, by Lemma 3$\exists D^{\prime }$ clause such that $\Gamma {⊢}_{OI}{D}^{\prime }\wedge D^{\prime }\subseteq C$. Given $\langle R_1^{\prime },\dots ,R_k^{\prime }=D^{\prime }\rangle$ OI derivation of $D^{\prime }$ from $\Gamma$, it can be lifted by Lemma 5 to an OI derivation $\langle R_1,\dots ,R_k\rangle$ of $R_k$ from $\Sigma$, where $D^{\prime }=R_k^{\prime }$ is an instance of $R_k$. Taken $D=R_k$: $\Sigma {⊢}_{OI}D\wedge D{\le }_{OI}C$ (since $D^{\prime }{\le }_{OI}C$).    □

Theorem 9 

(Subsumption Theorem). Let Σ be a logic program and C be a clause. $\Sigma {\Rightarrow }_{OI}C$ iff $\exists D$ such that $\Sigma {⊢}_{OI}D\wedge D{\le }_{OI}C$.

Proof. (⇐)For the soundness of the derivation (and ${\theta }_{OI}$ subsumption).

(⇒)

Assume C is not a tautology. Given $\theta$ OI substitution that maps $\mathit{vars}\left(C\right)$ to new constants that do not occur in $\Sigma \cup C$, $C\theta$ is a non-tautological ground clause and $\Sigma {\Rightarrow }_{OI}C\theta$. Thus, by Lemma 6 $\exists D$ such that $\Sigma {⊢}_{OI}D\wedge D{\le }_{OI}C\theta$. $\theta$ maps variables of C to constants that are not in D because, being derived from $\Sigma$, D cannot contain constants in $\theta$. Let $\delta$ be a substitution such that $D\delta \subseteq C\theta$ and $\sigma$ be the substitution obtained from $\delta$ by replacing in each binding $x_i/t_i$ the term $t_i$ with $a_i$. Then, $\delta =\sigma \theta$. Since $\theta$ only replaces the variables $x_i$ by $a_i$$(1\le i\le n)$, it follows that $D\sigma \subseteq C$, i.e., $D{\le }_{OI}C$.

   □

So, OI resolution plus ${\theta }_{OI}$ subsumption can derive the same conclusions as model-theoretic OI implication. As a consequence, some results in [41] for the unrestricted case can be proved. First, let us show when OI implication and ${\theta }_{OI}$ subsumption coincide:

Proposition 9. 

Let $C,D$ be clauses. If C is not self-resolvent and D is not tautological, then $C{\Rightarrow }_{OI}D$ iff $C{\le }_{OI}D$.

Proof. (⇐)Trivial.

(⇒)

If $C{\Rightarrow }_{OI}D$, by Theorem 9 $\exists E\in {\mathcal{R}}_{OI}^{*}\left(\left\{C\right\}\right)$ such that $E{\le }_{OI}D$. Since C is not self-resolvent ${\mathcal{R}}_{OI}^{*}\left(\left\{C\right\}\right)=\left\{C\right\}$. Hence the thesis.

   □

Proposition 10. 

Let C and D be clauses. If $C{\Rightarrow }_{OI}D$, then $C^{+}{\le }_{OI}{D}^{+}$ and $C^{-}{\le }_{OI}{D}^{-}$

where $C^{+}$ and $C^{-}$ denote the positive and negative literals of a clause C, respectively.

Proof. 

$C^{+}{\le }_{OI}C$, so $C^{+}{\Rightarrow }_{OI}C$. By hypothesis, $C{\Rightarrow }_{OI}D$; thus, $C^{+}{\Rightarrow }_{OI}D$. $C^{+}$ and $D^{+}$ contain only positive literals, so they cannot be self-OI-resolved, and $D^{+}$ is not a tautology. By Proposition 9, it must hold $C^{+}{\le }_{OI}D$. But $C^{+}$ is made up of positive literals only, so $C^{+}{\le }_{OI}{D}^{+}$. Analogously, $C^{-}{\le }_{OI}{D}^{-}$.    □

Finally, we obtain a sufficient condition for the equivalence between OI implication and ${\theta }_{OI}$ subsumption:

Proposition 11. 

Let C, D be clauses. If D is not ambivalent, then $C{\Rightarrow }_{OI}D$ iff $C{\le }_{OI}D$.

Proof. 

Assume $C{\Rightarrow }_{OI}D$. As D is not ambivalent, it cannot be a tautology. By Proposition 10 $\exists \theta ,\delta$ OI substitutions such that $C^{+}\theta \subseteq D^{+}$ and $C^{-}\delta \subseteq D^{-}$. Now, if C were self-resolving, then there would be a literal $l=l{\sigma }_1=l{\sigma }_2$ such that $l{\sigma }_1\in C^{+}$ and $\neg l{\sigma }_2\in C^{-}$, with ${\sigma }_1$ and ${\sigma }_2$ OI unifiers. But then $l{\sigma }_1\theta \in D^{+}$ and $\neg l{\sigma }_2\delta \in D^{-}$, which does not hold for the non-ambivalence of D. Thus, C is not self-resolving and D is not ambivalent. By Proposition 9, $C{\le }_{OI}D$.    □

Example 8. 

Consider the domain $\mathcal{D}=\{a,b\}$ and the following cases.

1. 

$C=p\left(X\right)\vee q\left(Y\right)$ not self-resolvent (nor ambivalent), $D=p\left(a\right)\vee q\left(b\right)$ not tautological. $\left\{p\right(a),p(b\left)\right\}$, $\left\{p\right(a),q(a\left)\right\}$, $\left\{p\right(b),q(b\left)\right\}$, $\left\{q\right(a),q(b\left)\right\}$ and their supersets are the OI models for C, all of which are also OI models for D, so $C{\Rightarrow }_{OI}D$; also, $C{\le }_{OI}D$ because $\exists \theta =\{a/X,b/Y\}:C\theta =D$. Note that under OI, $p\left(a\right)\vee q\left(a\right)$ and $p\left(b\right)\vee q\left(b\right)$ are not to be verified by the interpretations, since they would bind both X and Y onto the same constant (a or b).

2. 

$C=p\left(X\right)\vee \neg q\left(X\right)$ not self-resolvent (nor ambivalent), $D=p\left(a\right)\vee \neg q\left(a\right)\vee \neg r\left(a\right)$ not tautological. Interpretations $\left\{\right\}$, $\left\{p\right(b\left)\right\}$, $\left\{p\right(b),q(a\left)\right\}$, $\left\{p\right(a\left)\right\}$, $\left\{p\right(a),q(b\left)\right\}$, $\left\{p\right(a),p(b\left)\right\}$, $\left\{p\right(a),p(b),q(b\left)\right\}$, $\left\{p\right(a),p(b),q(a\left)\right\}$, $\left\{p\right(a),p(b),q(a),q(b\left)\right\}$ are the OI models for C, and they are also OI models for D, so $C{\Rightarrow }_{OI}D$; also, $C{\le }_{OI}D$ because $\exists \theta =\{a/X\}:C\theta \subset D$. Moreover, for $\theta =\{a/X\}$: $C^{+}=p\left(X\right){\le }_{OI}p\left(a\right)=D^{+}$

and $C^{-}=\neg q\left(X\right){\le }_{OI}\neg q\left(a\right)\vee \neg r\left(a\right)=D^{-}$.

3. 

$C=p\left(X\right)\vee \neg q\left(Y\right)$ not self-resolvent (nor ambivalent), $D=p\left(a\right)\vee \neg q\left(b\right)\vee \neg r\left(a\right)$ not tautological. $\left\{\right\}$, $\left\{p\right(b\left)\right\}$, $\left\{p\right(b),q(b\left)\right\}$, $\left\{p\right(a\left)\right\}$, $\left\{p\right(a),q(a\left)\right\}$, $\left\{p\right(a),p(b\left)\right\}$, $\left\{p\right(a),p(b),q(b\left)\right\}$, $\left\{p\right(a),p(b),q(a\left)\right\}$, $\left\{p\right(a),p(b),q(a),q(b\left)\right\}$ are the OI models for C, and they are also OI models for D, so $C{\Rightarrow }_{OI}D$; also, $C{\le }_{OI}D$ because $\exists \theta =\{a/X,b/Y\}:C\theta \subset D$. Note that interpretations $\left\{p\right(a),p(b),q(a\left)\right\}$, $\left\{p\right(a),p(b),q(b\left)\right\}$, $\left\{q\right(a),q(b),p(a\left)\right\}$, $\left\{q\right(a),q(b),p(b\left)\right\}$, $\left\{p\right(a),p(b),q(a),q(b\left)\right\}$ are not OI interpretations since they would bind both X and Y onto the same constant (a or b).

Moreover, $C^{+}=p\left(X\right)\vee q\left(Y\right){\le }_{OI}p\left(a\right)\vee q\left(b\right)=D^{+}$ for $\theta =\{a/X,b/Y\}$

and $C^{-}=\square {\le }_{OI}\neg r\left(a\right)=D^{-}$ because the empty clause subsumes everything.

4.2.4. Refutation Completeness and Compactness

Like normal resolution, OI resolution alone is not deduction complete, yet it is complete with respect to an unsatisfiable set of clauses.

Theorem 10 

(Refutation Completeness). A finite set of clauses Σ is unsatisfiable under OI iff $\Sigma {⊢}_{OI}\square$.

Proof. (⇐)For the soundness of the OI derivation (Theorem 6).

(⇒)

If $\Sigma {\Rightarrow }_{OI}\square$, for Theorem 9$\exists D$ such that $\Sigma {⊢}_{OI}D\wedge D{\le }_{OI}\square$. So, $D=\square$ and thus $\Sigma {⊢}_{OI}\square$.

   □

All the results in this section can be extended to the case of $\Sigma$ infinite, owing to a Compactness Theorem (see [42] for the unrestricted case). A proof for our notion of satifiability is proposed here by lifting a demonstration given for the propositional calculus. Two preliminary results on binary trees must be recalled:

Lemma 7 

(König’s Lemma). A binary tree with arbitrarily long branches has an infinite branch.

Lemma 8. 

Let Σ be a set of clauses. Then, $\exists {\Sigma }^{\prime }\subseteq \Sigma$ such that Σ has an OI model iff ${\Sigma }^{\prime }$ has an OI model.

Now, the Compactness Theorem is proven in two forms:

Theorem 11 

(Compactness—First Form). Given an infinite set of clauses Σ, $\forall {\Sigma }^{\prime }\subset \Sigma$ finite: Σ has an OI model iff ${\Sigma }^{\prime }$ has an OI model.

Proof. (⇒)Trivial using Lemma 8.

(⇐)

Assume $\forall {\mathrm{\Sigma }}_0\subset \mathrm{\Sigma }$ finite: ${\mathrm{\Sigma }}_0$ has an OI model. Then, a Herbrand OI model can be built based on Proposition 7. The Herbrand base $H_{\Sigma }$ is in general infinite but countable: denote its elements with the sequence $A_1,\dots ,A_n,\dots$.

Now, let ${\Sigma }_n$ be a finite set of clauses whose truth depends on $A_1,\dots ,A_n$. Consider a binary tree built by taking node r as the root and scuh that the successive nodes stand for the truth values of $A_1,\dots ,A_n,\dots$, respectively; r has two edges towards two nodes standing for the possible truth values of $A_1$; then, from each node, two edges depart towards truth values of $A_2$, and so on. Clearly, a Herbrand interpretation is given by assigning the truth values encountered traversing a path in such a tree.

Let $T_{\Sigma }$ be the subtree obtained by taking all finite paths that represent an OI modelfor C, $\forall C\in {\Sigma }_n$. Since every ${\Sigma }_n$ has an OI model, $T_{\Sigma }$ must have arbitrarily long branches; hence, by Lemma 7, it has an infinite branch, representing an OI model for $\Sigma$.

   □

Theorem 12 

(Compactness—Second Form). Let Σ be an infinite set of formulas having no OI model. Then, $\exists {\Sigma }^{\prime }\subset \Sigma$ is finite that has no OI model.

Proof. 

By contradiction, if all the subsets ${\Sigma }^{\prime }$ have an OI model, then $\Sigma$ should have an OI model by Theorem 11.    □

4.3. Decidability

Let us start by providing some definitions and preliminary results. Let $\Sigma =\{C_1,\dots ,C_m\}$ be a set of clauses, and C a clause with $vars\left(C\right)=\{x_1,\dots ,x_n\}$.

• Given $a_1,\dots ,a_n$ distinct constants not occurring in $\Sigma$ or C, $\sigma =\{a_1/x_1,\dots ,a_n/x_n\}$ is a Skolem substitution for C with respect to $\Sigma$.
  The term set of $\Sigma$ by $\sigma$ is the set of all terms occurring in $\Sigma \sigma$.
• Given a set of terms T, the instance set of C with respect to T is
  $$\mathcal{I}(C,T)=\left\{C\theta \right|\theta =\{t_i/x_i|\forall i=1,\dots ,n:t_i\in T\}OIsubstitution\}$$
  The instance set of $\Sigma$ with respect to T is $\mathcal{I}(\Sigma ,T)=\mathcal{I}(C_1,T)\cup \cdots \cup \mathcal{I}(C_m,T)$.

Lemma 9. 

Let Σ be a set of clauses, C a clause, and σ a Skolem OI substitution for C with respect to Σ. $\Sigma {\vDash }_{OI}C$ iff $\Sigma {\vDash }_{OI}C\sigma$.

Proof. (⇒)Trivial.

(⇐)

Suppose C is not a tautology and $\sigma =\{a_i/x_i|x_i\in \mathit{vars}\left(C\right)\}$. If $\Sigma {\vDash }_{OI}C\sigma$, then, by Theorem 9, $\exists D$ clause such that $\Sigma {⊢}_{OI}D\wedge D{\le }_{OI}C\sigma$. Hence, since all constants in D must occur also in clauses in $\Sigma$, $\sigma$ can be regarded as a Skolem substitution for C with respect to D. Then, by Lemma 6, $D{\le }_{OI}C$. So, we can conclude that $\Sigma {\vDash }_{OI}C$.

   □

In the following, some ideas introduced in [43] for proving the decidability of T-implication are borrowed (T-implication [43] was an attempt to weaken implication to obtain a decidable relationship, but (being not transitive) does not induce a quasi-ordering, and hence it is not useful for defining search spaces and refinement operators on them). Like implication, OI implication is also easily proven decidable when only ground clauses are involved.

Lemma 10. 

Let Σ be a set of ground clauses and C a ground clause. $\Sigma {\vDash }_{OI}C$ is decidable.

Proof. 

Let S be the (finite) set of all ground atoms occurring in $\Sigma$ and C. Then

$\Sigma {\vDash }_{OI}C$ iff (by Theorem 5) $\Sigma \cup \neg C$ has no OI model iff (by Proposition 7);

$\Sigma \cup \neg C$ has no Herbrand OI model iff no subset of S is a Herbrand OI model of $\Sigma \cup \neg C$.

Since S is finite, this relationship is decidable.    □

The following result is based on the observation that if a term does not appear in the conclusion of an OI derivation between ground clauses, then it must have been resolved during an OI resolution step. Hence, by replacing this term with a new term in the premises, a new derivation of the same conclusion can be obtained.

Lemma 11. 

Let Σ be a set of clauses, C a clause, σ a Skolem OI-substitution for Σ with respect to $\left\{C\right\}$, and T the term set of C by σ. Then, $\Sigma {\vDash }_{OI}C$ iff $\mathcal{I}(\Sigma ,T){\vDash }_{OI}C\sigma$.

Proof. 

(⇒)

Trivial when C is a tautology. If C is not a tautology, then neither is $C\sigma$. Since $\Sigma {\vDash }_{OI}C\sigma$ by Lemma 9, from Theorem 8, it follows that $\exists \mathrm{\Gamma }$ is a finite set of instances of clauses in $\Sigma$ such that $\mathrm{\Gamma }{\vDash }_{OI}C\sigma$. For Theorem 9, there exists a derivation from $\mathrm{\Gamma }$ of a clause E, such that $E{\le }_{OI}C\sigma$. Since $\mathrm{\Gamma }$ is made up of ground clauses, E must be ground, too; hence, $E\subseteq C\sigma$. Therefore, E contains only terms from $C\sigma$.

Now, consider the ground OI substitutions ${\gamma }_i,i=1,\dots ,n$ that yield the clauses in $\mathrm{\Gamma }$ from those in $\Sigma$. $\forall i=1,\dots ,n:\exists {\gamma }_i^{\prime }$ new OI substitution such that

$$\begin{gathered} t_j/x_j\in {\gamma }_i^{\prime } \mathrm{if} t_j/x_j\in {\gamma }_i \mathrm{and} t_j\in T; \\ {s}_j/x_j\in {\gamma }_i^{\prime } \mathrm{if} t_j/x_j\in {\gamma }_i \mathrm{and} t_j\notin T, \end{gathered}$$

where $s_j\in T$ is a choice of a new term from T. In this manner, every term $\{t_1,\dots ,t_k\}$ in clauses in $\mathrm{\Gamma }$ which is not in T is replaced with a distinct term in $\{s_1,\dots ,s_k\}\subseteq T$. Since $\mathrm{\Gamma }$ contains ground instances of clauses in $\Sigma$, this replacement yields a new set of clauses ${\mathrm{\Gamma }}^{\prime }$ where each clause is a ground instance of clauses in $\Sigma$. The term replacement involves the term set T by $\sigma$; then, the existence of a derivation of E from $\mathrm{\Gamma }$ yields a derivation of E from ${\mathrm{\Gamma }}^{\prime }$. Indeed, each OI resolution step in the derivation from $\mathrm{\Gamma }$ can be transposed into an OI resolution step from ${\mathrm{\Gamma }}^{\prime }$, since the same terms in $\mathrm{\Gamma }$ are replaced by the same terms in ${\mathrm{\Gamma }}^{\prime }$. In addition, the terms in $\mathrm{\Gamma }$ that are not in T, replaced by $\{s_1,\dots ,s_k\}\subseteq T$, do not appear in the conclusion E of the derivation.

Then, a derivation of E from ${\mathrm{\Gamma }}^{\prime }$ exists, so we can write ${\mathrm{\Gamma }}^{\prime }{\vDash }_{OI}E$ and hence ${\mathrm{\Gamma }}^{\prime }{\vDash }_{OI}C\sigma$. Now, ${\mathrm{\Gamma }}^{\prime }$ is a set of ground instances of clauses in $\Sigma$ and all terms in ${\mathrm{\Gamma }}^{\prime }$ are also in T; then, ${\mathrm{\Gamma }}^{\prime }\subseteq \mathcal{I}(\Sigma ,T)$. Thus, $\mathcal{I}(\Sigma ,T){\vDash }_{OI}C\sigma$.

(⇐)

$\Sigma {\vDash }_{OI}\mathcal{I}(\Sigma ,T)$ holds, being the instance set made up of instances of clauses in $\Sigma$. By hypothesis, $\mathcal{I}(\Sigma ,T){\vDash }_{OI}C\sigma$, so $\Sigma {\vDash }_{OI}C\sigma$ and, by Lemma 9, $\Sigma {\vDash }_{OI}C$.

   □

The same does not hold for unconstrained implication, since in this case the terms replaced cannot be tracked back. For OI implication, since different terms cannot be mapped onto the same term, a ground instance of a clause C in $\Sigma$ can be transformed into another instance of C by means of an OI anti-substitution composed with an OI substitution.

Theorem 13 

(Decidability—General Case). Let Σ be a set of clauses and C a clause. The problem whether $\Sigma {\vDash }_{OI}C$ is decidable.

Proof. 

By Lemma 11, $\mathcal{I}(\Sigma ,T){\vDash }_{OI}C\sigma$ is equivalent to $\Sigma {\vDash }_{OI}C$. But $\mathcal{I}(\Sigma ,T)$ and $C\sigma$ are both ground, and OI implication between ground clauses is decidable (by Lemma 10).    □

5. Incremental Inductive Synthesis

Using the OI framework for the incremental inductive synthesis of theories brings several advantages. The process of abstract diagnosis and debugging of an incorrect theory involves a search in a space whose algebraic structure makes easy the definition of algorithms that meet several desirable properties from the viewpoint of theoretical computer science, and specifically of the mathematical theory of computation (e.g., non-termination) [32], as well as of the theory of computational complexity [33,34]. Such algorithms embody two ideal refinement operators, one for generalizing incomplete clauses and the other for specializing inconsistent clauses.

In ILP, a standard practice to restrict the search space is imposing biases on it [44]. In the following, we are concerned exclusively with logic theories expressed as hierarchical programs, that is, as (non-recursive) programs made up of Datalog linked clauses. Under OI, we can give the following definitions, where $E^{-}$ and $E^{+}$ denote the sets of all the negative and positive examples (respectively).

Definition 15 

(Inconsistency).

• A theory T is inconsistent iff $\exists H\in T$, $\exists N\in E^{-}:H$ is inconsistent with respect to N.
• A hypothesis H is inconsistent with respect to N iff $\exists C\in H$: C is inconsistent with respect to N.
• A clause C is inconsistent with respect to N iff $\exists \sigma$:

  1. 

  body(C)$\sigma \subseteq$ body(N)

  2. 

  ¬ head(C)σ = head(N)

  3. 

  constraints($C_{OI}$)σ⊆ constraints($N_{OI}$)

  where $body\left(\phi \right)$ and $head\left(\phi \right)$ denote, respectively, the body and the head of a clause φ.

Note that the ${\le }_{OI}$ notation cannot be used, since the same σ must satisfy Conditions 1, 2, and 3. If at least one of such conditions is not met, C is consistent with respect to N.

Definition 16 

(Incompleteness).

• A theory T is incomplete iff $\exists H\in T$, $\exists P\in E^{+}$: H is incomplete with respect to P.
• A hypothesis H is incomplete with respect to P iff $\forall C\in H:\neg (P{\le }_{OI}C$).

Otherwise it is complete with respect to P.

When the aim is to incrementally develop a logic program, that should be correct with respect to its intended model at the end of the development process, it becomes relevant to define operators that allow an incremental refinement of too weak or too strong programs [45].

5.1. Refinement Operators

In ILP, the definition of ideal refinement operators [34] is crucial. Such operators do exist in the space of Datalog clauses ordered by ${\theta }_{OI}$ subsumption, where infinite unbound strictly ascending/descending chains do not exist, since equivalence among clauses coincides with variance [33,34].

Definition 17 

(Refinement operators under OI). Let C be a Datalog clause.

• $D\in {\rho }_{OI}\left(C\right)$ when exactly one of the following conditions holds:

  1. 

  $D\in C\theta$, where $\theta =\{a/x\},a\notin consts\left(C\right),x\in vars\left(C\right)$;

  2. 

  $D=C\cup \{\neg l\}$, where l is an atom such that $\neg l\notin C$.

• $D\in {\delta }_{OI}\left(C\right)$ when exactly one of the following conditions holds:

  1. 

  $D\in C\gamma$, where $\gamma =\{x/a\},a\in consts\left(C\right),x\notin vars\left(C\right)$;

  2. 

  $D=C\setminus \{\neg l\}$, where l is an atom such that $\neg l\in C$.

In other words, a specialization of a clause can be obtained by substituting a variable with a (new) constant or by adding a literal to its body; a generalization of a clause can be obtained by substituting a constant with a (new) variable or by removing a literal from its body.

By 2 in the definition of ${\delta }_{OI}$, all possible generalizations of a clause must be (non-empty) subsets of its literals that are $2^{\mid C\mid }-1$. Applying 1 to each of them, a clause containing n constants can yield ${\sum }_{k=0,n}\left(\begin{array}{c}n\\ k\end{array}\right)$ generalizations. Excluding the clause itself, the total number of proper generalizations $PGE{N}_{OI}\left(C\right)$ can be bound as follows:

$$\begin{array}{ccc}\hfill \mid PGE{N}_{OI}\left(C\right)\mid & \le & (2^{\mid C\mid }-1)\sum _{k=0}^{\mid consts\left(C\right)\mid }\left(\begin{array}{c}\mid consts\left(C\right)\mid \\ k\end{array}\right)\hfill \\ & =& 2^{\mid C\mid +\mid consts\left(C\right)\mid }-2^{\mid consts\left(C\right)\mid }\hfill \end{array}$$

Proposition 12 

([34]). The refinement operators in Definition 17 are ideal for Datalog clauses ordered by ${\theta }_{OI}$ subsumption.

It is possible to define upper bounds to the number of refinement steps required to reach a clause D by starting from a clause C:

• $D\in {\rho }_{OI}^{n+t}\left(C\right)$, where $n=\mid consts\left(D\right)-consts\left(C\right)\mid \ge 0,t=\mid D\mid -\mid C\mid$
• $D\in {\delta }_{OI}^{n+t}\left(C\right)$, where $n=\mid consts\left(D\right)-consts\left(C\right)\mid \ge 0,t=\mid C\mid -\mid D\mid$

Some redundancy is present in the defined refinement operators, since in Item 2 they can add (or remove) literals containing new constants (or constants not occurring elsewhere in the clause) which is in charge of Item 1. This can be avoided, obtaining less sparse search graphs at the cost of some additional refinement item, by modifying Item 2 as follows, obtaining the modified operators ${\rho }_{OI}^{\prime }$ and ${\delta }_{OI}^{\prime }$, respectively:

${\rho }_{OI}^{\prime }:2^{\prime }.$

$D=C\cup \{\neg l\}$ where l is an atom such that $\neg l\notin C$ and $consts\left(l\right)\subseteq consts\left(C\right)$

${\delta }_{OI}^{\prime }:2^{\prime }.$

$D=C\setminus \{\neg l\}$ where l is an atom such that $\neg l\in C$ and $consts\left(l\right)\subseteq consts(C\setminus \{\neg l\left\}\right)$

Inspired by a previous concept first given by Reynolds and extended to clauses by Shapiro, we have a measure for the complexity of a clause:

Definition 18 

(size_OI). The size of a clause C under OI is $siz{e}_{OI}\left(C\right)=\mid C\mid +\mid consts\left(C\right)\mid$.

i.e., the number of literals in C plus the number of distinct constants occurring in C. It allows to predict the exact number of steps required to perform a refinement, based only on the syntactic structure of the clauses involved (that could be known or bounded a priori). Given two clauses C and D, if $C{<}_{OI}D$, then

$$C\in {\delta }_{OI}^{\prime k}\left(D\right),D\in {\rho }_{OI}^{\prime k}\left(C\right),k=siz{e}_{OI}\left(D\right)-siz{e}_{OI}\left(C\right)$$

Example 9. 

$C:p\left(X\right):-q(X,Y)$$D:p\left(X\right):-q(X,Y),q(a,b)$

Since $siz{e}_{OI}\left(C\right)=2$ and $siz{e}_{OI}\left(D\right)=3+2=5$, The number of steps required to refine C in D (or vice versa) is $siz{e}_{OI}\left(D\right)-siz{e}_{OI}\left(C\right)=3$, i.e., $D\in {\rho }_{OI}^{\prime 3}\left(C\right)$ (respectively, $C\in {\delta }_{OI}^{\prime 3}\left(D\right)$):

$$\begin{array}{ccc}\hfill C=p\left(X\right):-q(X,Y).& \stackrel{2.}{\leftrightarrow }& p\left(X\right):-q(X,Y),q(W,Z).\hfill \\ & \stackrel{1.}{\leftrightarrow }& p\left(X\right):-q(X,Y),q(a,Z).\hfill \\ & \stackrel{1.}{\leftrightarrow }& p\left(X\right):-q(X,Y),q(a,b).=D\hfill \end{array}$$

5.2. Refinement in Unrestricted Search Spaces

Through OI substitutions, the previous refinement operators can be extended to the case of unrestricted search spaces ordered by ${\theta }_{OI}$ subsumption (with functions).

Definition 19 

(Refinement operators in unrestricted space). Let $C,D$ be clauses.

•

$D\in {\rho }_{OI}\left(C\right)$ when exactly one of the conditions in Definition 17 and the following one holds:

3. 

$D=C\theta$, where $\theta =\{f(Y_1,\dots ,Y_n)/x\}$, f function symbol ($n>0$) and $x\in vars\left(C\right)$;

•

$D\in {\delta }_{OI}\left(C\right)$ when exactly one of the conditions in Definition 17 and the following one holds:

3. 

$D=C\gamma$, where $\gamma =\{x/f(Y_1,\dots ,Y_n)\}$, f function symbol ($n>0$) such that $f(Y_1,\dots ,Y_n)\in terms\left(C\right)$ and $x\notin vars\left(C\right)$.

Again, these operators are ideal for this search space. The proof requires two lemmas:

Lemma 12. 

$\forall C,D$ clauses such that $C{<}_{OI}D:C_{OI}\theta =D_{OI}\Rightarrow \exists$ a ${\rho }_{OI}$-chain from C to D.

Proof. 

Let $C_{OI}=C_0,\dots ,C_n=D_{OI}$ be a chain of clauses such that $C_i=C_{i-1}{\theta }_i,i=1,\dots ,n$, where every substitution ${\theta }_i$ is defined by 1 and 3 in the definition of ${\rho }_{OI}$ (here renaming substitutions are left out for brevity). Thus, $C_i\in {\rho }_{OI}\left(C_{i-1}\right),i=1,\dots ,n$.    □

Lemma 13. 

$\forall C,D$ clauses such that $C\subseteq D:\exists$ a ${\rho }_{OI}$-chain from C to D.

Proof. 

By induction on the number n of literals in $D\setminus C$.

(base)

If $n=0$, then $C=D$; thus, the empty chain fulfills the lemma.

(step)

Assume that for some k, $0\le k<n$, there is a ${\rho }_{OI}$-chain from C to a $C_k$, where $C_k$ is obtained from C by adding k literals contained in D. Now, let $l\in D\setminus C_k$ and $C_{k+1}=C_k\cup \left\{l\right\}$. Since $l\in D\setminus C_k$, then $l\notin C_k$; then, it can be used to refine $C_k$, according to 2 in the definition of ${\rho }_{OI}$. Hence, we obtain $C_{k+1}\in {\rho }_{OI}\left(C_k\right)$. By inductive hypothesis, there is also a chain from C to $C_k$. Thus, the lemma is satisfied for $k+1$.

   □

Theorem 14 

(Ideality of ${\rho }_{OI},{\delta }_{OI}$). In an unrestricted clausal search space, ${\rho }_{OI}$ and ${\delta }_{OI}$ are ideal refinement operators.

Proof. 

(local finiteness)

Trivial, since, by definition, a term dominated by an n-ary function ($n\ge 0$) is turned into a variable (or vice versa) or a single literal is added (or removed).

(properness)

If $D\in {\rho }_{OI}\left(C\right)$, by definition of downward refinement operator, $C{\le }_{OI}D$. If also $D{\le }_{OI}C$, then $C{\sim }_{OI}D$. Hence, by 3 in Property 5 in the unrestricted space, the clauses would be variants, which is false for the construction of the operator (either C has a new term with respect to D or it is longer than D). The case is analogous for the other operator.

(completeness)

Let C and D be two clauses such that $C{\le }_{OI}D$. We have to prove the existence of a chain from C to D (clauses equivalent to D are omitted here since they are variants). Hence, $\exists \theta :C\theta \subseteq D$. Let $E=C\theta$. For Lemma 12, there is a ${\rho }_{OI}$-chain from C to E. By definition of E, it also holds $E\subseteq D$, and by Lemma 13, there is a ${\rho }_{OI}$-chain from E to D. Thus, we have a chain from C to D, which proves that ${\rho }_{OI}$ is complete. An analogous proof holds for the completeness of ${\delta }_{OI}$ (Lemmas similar to 12 and 13 are needed).

   □

5.3. Refinement Operators for OI Implication

When dealing with non ambivalent clauses, generalization and specialization under implication (or OI implication) correspond to the cases considered for $\theta$ subsumption (or ${\theta }_{OI}$ subsumption). Thus, an operator computing the resolution and inverse resolution steps for ambivalent clauses is needed. Powers and roots of a clause [46] (operations in which a clause is resolved with itself) can be regarded as theoretical refinement operators.

Definition 20 

(Clause Power/Root). Let $C,D$ be clauses. D is an $n^{th}$ power of C (or C is an $n^{th}$ root of D) iff D is a variant of a clause in ${\mathcal{L}}^{n-1}\left(C\right),n\ge 1$.

Example 10. 

Given $C:p\left(X\right):-p\left(f\right(X\left)\right),q(X,Y)$,

• $p\left(X\right):-p\left(f^2\left(X\right)\right),q(X,Y),q(f\left(X\right),Z)$ is a second power of C;
• $p\left(X\right):-p\left(f^3\left(X\right)\right),q(X,Y),q(f\left(X\right),Z),q(f^2\left(X\right),W)$ is a third power of C.

By the subsumption theorem, downward refinements of clause C in a search space ordered by OI implication can be obtained by self-resolving Cn times ($n^{th}$ power) or by applying a downward refinement operator for ${\theta }_{OI}$ subsumption. Conversely, upward refinements require to compute an $n^{th}$ root of C or to exploit an upward refinement operator for ${\theta }_{OI}$ subsumption. Both cases are not practically useful, since the n to stop at in the process of self-resolution is not known a priori. Moreover, while it is clear how to compute $n^{th}$ powers by using linear resolution, the dual is a more complex task since it involves the inversion of resolution steps. We now tackle this latter problem.

5.3.1. Inverting OI Resolution

We adapt to OI implication the “or-introduction” technique developed in [43] to build parent clauses given the resolvent and generalize it to build OI ancestors. The resolvent of two clauses C and D is $R=((C\setminus \left\{l_C\right\})\cup (D\setminus \left\{l_D\right\}))\theta$, where $\theta =mg{u}_{OI}(l_C,\overline{l_D})$. The most specific parent clauses are those that inherit all literals from the OI resolvent. So, $\theta$ must be an $mg{u}_{OI}$ also for $\{C\setminus \left\{l_C\right\},D\setminus \left\{l_D\right\}\}$:

$$(C\setminus \left\{l_C\right\})\theta =(D\setminus \left\{l_D\right\})\theta =R$$

Thus, the two ambivalent parent clauses are obtained by introducing a new literal as

$$C=R\cup \left\{l\right\}\mathrm{and}D=R\cup \left\{\overline{l}\right\}\mathrm{where}l=l_C=\overline{l_D}$$

Proposition 13. 

Let R be a clause and l a literal. $\left\{R\right\}{\iff }_{OI}(R\cup \left\{l\right\},R\cup \left\{\overline{l}\right\})$.

Proof. 

See ([43], page 39). The “${\Rightarrow }_{OI}$” proof is based on the empty substitution which is injective. The opposite holds since OI resolution is sound, being a special case of resolution (cf. Theorem 6).    □

Another useful result can be transposed straight forwardly from the following implication.

Proposition 14. 

Let $C,D$ be clauses and R an OI resolvent of C and D. Then, $\exists l$ is literal such that $C{\le }_{OI}R\cup \left\{l\right\}$ and $D{\le }_{OI}R\cup \left\{\overline{l}\right\}$.

Proof. 

Similar to ([43], page 39), but considering OI substitutions: by definition of OI resolution, we have $R=\left(\right(C\setminus \left\{a\right\})\cup (D\setminus \left\{b\right\}\left)\right)\theta$ where $\theta =mg{u}_{OI}(a,\overline{b})$. Now, let $l=a\theta =\overline{b}\theta$ (with $\theta$ injective to be eliminated), then

$R\cup \left\{l\right\}=(C\cup (D\setminus \left\{b\right\}\left)\right)\theta \supseteq C\theta$ and $R\cup \left\{\overline{l}\right\}=((C\setminus \left\{a\right\})\cup D)\theta \supseteq D\theta$.    □

This technique can be iterated to compute clauses from which R follows in many steps, progressively introducing new literals: given $\{R\cup \left\{l\right\},R\cup \left\{\overline{l}\right\}\}$, either of the two parent clauses can be inverted; applying or-introduction to the former, we have $\{R\cup \left\{l\right\}\cup \left\{m\right\},R\cup \left\{l\right\}\cup \left\{\overline{m}\right\},R\cup \left\{\overline{l}\right\}\}.$ Generalizing, we obtain the following.

Definition 21 

(Or-introduced Clause [43]). Let C be a clause and Ω a sequence of literals. Then, a set of clauses S is or-introduced from C by Ω iff

(a) 

$S=\left\{C\right\}$ and $\Omega =\langle \rangle$ or

(b) 

$S=(S^{\prime }\setminus D)\cup \{D\cup \left\{l\right\},D\cup \left\{\overline{l}\right\}\}$ and $\Omega =\langle l_1,\dots ,l_n,l\rangle$, where $S^{\prime }$ is a set of clauses or-introduced from C by $\langle l_1,\dots ,l_n\rangle$ and $D\in S^{\prime }$.

Logical equivalence still holds through this step of inversion.

Theorem 15. 

Let S be a set of clauses or-introduced from clause C. $S{\iff }_{OI}\left\{C\right\}$.

Proof. 

See ([43], page 41). It follows by induction on the number of or-introduced literals, using Proposition 13.    □

A sequence of resolution steps can be inverted by applying or-introduction to a sequence of literals.

Lemma 14. 

Let $C,D,E$ be clauses and $\{D_1,\dots ,D_n\}$ a set of clauses or-introduced from D such that $D{\le }_{OI}E\wedge \forall i=1,\dots ,n:C{\le }_{OI}{D}_i$. Then, $\exists \{E_1,\dots ,E_n\}$ is a set of clauses or-introduced from E such that $\forall i=1,\dots ,n:C{\le }_{OI}{E}_i$.

Proof. 

Similar to ([43], page 41). Since the $D_i$s are or-introduced from D, $D_i=D\cup {\Lambda }_i$. Now, $\forall i\in \{1,\dots ,n\}:C{\le }_{OI}{D}_i\Rightarrow \exists {\theta }_i$ OI substitution such that $C{\theta }_i\subseteq D_i=D\cup {\Lambda }_i$ and $D{\le }_{OI}E\Rightarrow \exists \sigma$ OI substitution such that $D\sigma \subseteq E$. Hence, $C{\theta }_i\sigma \subseteq (D\cup {\Lambda }_i)\sigma \subseteq (E\cup {\Lambda }_i\sigma )$. Let $E_i=E\cup {\Lambda }_i\sigma$, then $\forall i\in \{1,\dots ,n\}:C{\le }_{OI}{E}_i$.    □

Theorem 16 

(Inversion of OI Resolution). Let T be a set of clauses, $D\in {\mathcal{L}}_{OI}^n\left(T\right)$. Then, $\exists S$ is a set of clauses or-introduced from $D,\exists C\in T$ such that $\forall E\in S:C{\le }_{OI}E$.

Proof. 

See ([43], pages 41–42). Let $S_n$ be or-introduced from $D_n$ by ${\mathrm{\Omega }}_n=\langle l_1,\dots ,l_{n-1}\rangle$. By induction on n,

(base)

For $n=1$, $S_1=\left\{D_1\right\}$ is or-introduced from ${\mathrm{\Omega }}_1=\langle \rangle$. Then, of course, $D_1{\le }_{OI}{D}_1$, $D_1\in T$;

(step)

By inductive hypothesis $\exists {\Omega }_k=\langle l_1,\dots ,l_{k-1}\rangle$ sequence of literals, $S_k$ set of clauses or-introduced from $D_k$ by ${\Omega }_k$ such that $\forall E_k\in S_k\exists C\in T:C{\le }_{OI}{E}_k$.

By definition of linear OI resolution, $D_{k+1}$ is an OI resolvent of $D_k$ and some $C\in T$. Then, by Proposition 14, $\exists l$ literal such that $C{\le }_{OI}{D}_{k+1}\cup \left\{l\right\}$ and $D_k{\le }_{OI}{D}_{k+1}\cup \left\{\overline{l}\right\}$.

By the inductive hypothesis and by Lemma 14, $\exists S_k^{\prime }$ set of clauses or-introduced from $D_{k+1}\cup \left\{\overline{l}\right\}$ by ${\Omega }_k^{\prime }=\langle l_1^{\prime },\dots ,l_{k-1}^{\prime }\rangle$ such that $\forall E_k^{\prime }\in S_k^{\prime }\exists C\in T:C{\le }_{OI}{E}_k^{\prime }$.

Thus, $S_{k+1}=(D_{k+1}\cup \left\{l\right\})\cup S_k^{\prime }$ is a set of clauses or-introduced from $D_{k+1}$ by ${\Omega }_{k+1}=\langle l,l_1^{\prime },\dots ,l_{k-1}^{\prime }\rangle$ such that $\forall E_{k+1}\in S_{k+1}\exists C\in T:C{\le }_{OI}{E}_{k+1}$.

   □

Restricting to the case of a single clause:

Corollary 4. 

Let C be a clause and $D\in {\mathcal{L}}_{OI}^n\left(\left\{C\right\}\right)$. $\exists S$ set of clauses or-introduced from D such that $S{\iff }_{OI}\left\{D\right\}\wedge \forall E\in S:C{\le }_{OI}E$.

5.3.2. Expansions

Section 5.3.1 makes an effort to reduce resolution to subsumption mechanisms. Thus, we come to the actual computation of upward refinements of clauses by using the following notion that can be regarded as the inverse reduction with respect to logical implication.

Definition 22 

(Expansion under OI). Let C be a clause and Ω a sequence of literals. A clause E is an expansion under OI (or OI expansion ) of C by Ω iff E is in the lgg_OI of a set of clauses or-introduced from C by Ω.

This expansion technique for unrestricted clausal spaces is practically infeasible, since it leads to an exponential growth of the computed expansion [43]: if n is the number of literals or-introduced to compute an expansion E of a clause C such that $\mid C\mid =m$, then the maximal cardinality of E is ${(m+n)}^{n+1}$. This cannot happen under OI:

Theorem 17 

(Maximal Cardinality of OI expansions). Let C be a clause, $\mid C\mid =m$, S a set of clauses or-introduced from C by $[l_1,\dots ,l_n]$, and E in the lgg_OI of S. The maximum length of E is $m+\lceil {log}_2\left(n\right)\rceil$.

Proof. 

Any $lg{g}_{OI}$ of a set of clauses, including E, cannot be longer than the shortest clause in the set. Each or-introduction replaces a clause of length l with two clauses of length $l+1$, thus yielding a binary tree. In particular, the clauses in a set $S_i$ are the leaves of the tree obtained after i or-introductions, and such clauses have length equal to m plus their depth in the tree. As a consequence, the longest $lg{g}_{OI}$ is possible when the or-introductions coming from the given sequence are performed in the most uniform way possible, i.e., by or-introducing each time one of the shortest clauses so far obtained. In fact, the order according to which or-introductions are performed depends on the goal of the search process. If this leads to a non-balanced tree, this means that some clause has a depth shorter than ${log}_2\left(n\right)$, and that would be the new bound for the $lg{g}_{OI}$.

Taking it to the extreme, if each time the longest clause in the set is choosen for or-introduction, the final set S will contain a clause of length $m+n$ and another one of length $m+1$ (resulting from the first or-introduction), and hence the resulting $lg{g}_{OI}$ will be long at most $m+1$. After the last (the nth) or-introduction, each leaf (i.e., each clause in S) will have depth at most $\lceil {log}_2\left(n\right)\rceil$, and hence E, that is, the $lg{g}_{OI}$ of all clauses in S, will be long at most $\lceil m+{log}_2\left(n\right)\rceil$.    □

It is noticeable that expansions of a clause are logically equivalent to it.

Theorem 18 

(Equivalence Preservation). Let C be a clause and E its OI expansion by some sequence Ω. Then, $C{\iff }_{OI}E$.

Proof. 

Similar to ([43], page 44). By definition of OI expansion, $\exists S$ is a set of clauses or-introduced from C by $\Omega$ such that $E\in lg{g}_{OI}\left(S\right)$.

$(\Rightarrow )$

$\forall F\in S:E{\le }_{OI}F$ and, by Proposition 8, $E{\Rightarrow }_{OI}F$. So, $\left\{E\right\}\Rightarrow S$. By Theorem 15, $S{\Rightarrow }_{OI}\left\{C\right\}$; hence, $E{\Rightarrow }_{OI}C$.

$(\Leftarrow )$

By Theorem 15, $\left\{C\right\}{\Rightarrow }_{OI}S$, i.e., $\forall F\in S:C{\Rightarrow }_{OI}F$, and so $\exists G$ such that $C{⊢}_{OI}G\wedge G{\le }_{OI}F$. For the properties of ${\mathit{lgg}}_{OI}$, $G{\le }_{OI}E$. Summing up, $C{\Rightarrow }_{OI}G{\le }_{OI}E$, and by Proposition 8, $C{\Rightarrow }_{OI}G{\Rightarrow }_{OI}E$, so $C{\Rightarrow }_{OI}E$.

   □

Theorem 19 

(OI implication reduced to ${\theta }_{OI}$ subsumption). Let C be a clause and D be a non-tautological clause such that $C{\Rightarrow }_{OI}D$. Then, $\exists E$ is the expansion of D such that C ${\le }_{OI}E$.

Proof. 

By definition of OI implication $\exists D_n\in {\mathcal{L}}_{OI}^n\left(\left\{C\right\}\right)$ is a clause such that $D_n{\le }_{OI}D$. So, by Theorem 16 $\exists S_n$ is a set of clauses or-introduced from $D_n$ such that $\forall F_n\in S_n:C{\le }_{OI}{F}_n$. Then, by Lemma 14$\exists S$ is a set of clauses or-introduced from D such that $\forall F\in S:C{\le }_{OI}F$. By definition of $lg{g}_{OI}$, $\forall E\in lg{g}_{OI}\left(S\right):C{\le }_{OI}E$.    □

This theoretical apparatus leads to the definition of the following operators [47].

Definition 23 

(${\rho }_{OI}^{\prime \prime }$, ${\delta }_{OI}^{\prime \prime }$). Given clause C,

•

$D\in {\delta }_{OI}^{\prime \prime }\left(C\right)$ iff $\exists E$ expansion of C such that $D\in {\delta }_{OI}\left(E\right)$;

•

$D\in {\rho }_{OI}^{\prime \prime }\left(C\right)$ iff $E\in {\mathcal{L}}_{OI}^n\left(\left\{C\right\}\right)$ for some n, such that $D\in {\rho }_{OI}\left(E\right)$.

${\rho }_{OI}^{\prime \prime }$ is proper and complete for the properness and completeness of ${\rho }_{OI}$, but not locally finite, because, as said, computing nth powers of a clause in the definition of ${\rho }_{OI}^{\prime \prime }$ is a merely theoretical issue, since it is not known a priori which n to stop at.

In regard to ${\delta }_{OI}^{\prime \prime }$, Theorem 17 proves that there is a limit for the length of the computed expansions. Moreover, in the modified refinement operators for ${\theta }_{OI}$ subsumption ${\delta }_{OI}^{\prime },{\rho }_{OI}^{\prime }$, the number of steps necessary to the refinement can be computed from the syntax of the clauses involved. Redefining the operators under OI implication to exploit ${\delta }_{OI}^{\prime }$ and ${\rho }_{OI}^{\prime }$ gives more control over the complexity of the refinement process.

Theorem 20 

(Ideality of ${\delta }_{OI}^{\prime \prime }$ and ${\rho }_{OI}^{\prime \prime }$). In a clausal space ordered by OI implication, ${\delta }_{OI}^{\prime \prime }$ and ${\rho }_{OI}^{\prime \prime }$ are ideal refinement operators.

Proof. 

${\delta }_{OI}^{\prime \prime }$ and ${\rho }_{OI}^{\prime \prime }$ are defined in terms of ${\delta }_{OI}$ and ${\rho }_{OI}$, respectively.

(${\delta }_{OI}^{\prime \prime }$—local finiteness)

Holds by definition of ${\delta }_{OI}^{\prime }$ and Theorem 19 ensuring the existence of an expansion, as an $lg{g}_{OI}$ of a set of or-introduced clauses (a singleton, in this case);

(${\delta }_{OI}^{\prime \prime }$—properness)

Follows from the properness of ${\delta }_{OI}$ (cf. Theorem 14);

(${\delta }_{OI}^{\prime \prime }$—completeness)

As for local finiteness, it comes from Theorem 19 ensuring the existence of an expansion and Proposition 12.

(${\rho }_{OI}^{\prime \prime }$—properness)

Like for ${\delta }_{OI}^{\prime \prime }$, using again Theorem 14;

(${\rho }_{OI}^{\prime \prime }$—local finiteness and completeness)

The level saturation procedure [9] that computes the linear self-resolution steps is needed. Call D the specialization to be computed.

For $n=1:{\mathcal{L}}_{OI}^1\left(\left\{C\right\}\right)=\left\{C\right\}$. Only a ${\theta }_{OI}$ subsumption step is needed, and the ideal operator ${\rho }_{OI}$ can be used.

For $n>1$, suppose that D has not been computed up to step $n-1$ and consider the case for n. A further linear OI resolution step can be computed as follows:

${\mathcal{L}}_{OI}^n\left(\left\{C\right\}\right)=\{{\mathcal{R}}_{OI}(E,F)\mid E\in {\mathcal{L}}_{OI}^{n-1}\left(\left\{C\right\}\right),F\in {\mathcal{L}}_{OI}^k\left(\left\{C\right\}\right),k<n\}$

and the ideal operator ${\rho }_{OI}$ can be used to compute specializations of clauses in ${\mathcal{L}}_{OI}^n\left(\left\{C\right\}\right)$. The computation is non-deterministic in the choice of F.

By construction, this procedure certainly finds specialization under OI implication when it exists. The computation terminates using as a halting point the condition $\forall R\in {\mathcal{L}}_{OI}^n\left(\left\{C\right\}\right):\left|R\right|>\left|D\right|$. In fact, the cardinality of the clauses increases monotonically, both by resolution ($|{\mathcal{R}}_{OI}(E_1,E_2)|>|E_i|,i=1,2$ except when $|E_1|\le 2$ or $|E_2|\le 2$, in which case the OI resolvent can be reached with a simple OI subsumption step) and by OI subsumption (if $C_1{\le }_{OI}{C}_2$, then $|C_1|\le |C_2|$).

   □

6. Discussion

While this paper focused on providing a comprehensive account of the OI framework from the foundational and theoretical viewpoint, it is also important to discuss its practical consequences. This section provides motivations for investigating and using the OI framework from the viewpoints of intuition and efficiency.

6.1. Intuition

Many clues support the belief that the OI assumption is built in the human way of thinking. For instance, saying that “X and Y are brothers if they share a common parent”, one implicitly means (and everybody understands) that X and Y must be two different persons. And, defining a bicycle as “an object made up of, among other components, a wheel X and a wheel Y”, nobody would think of a mono-cycle, even if X and Y could be associated to one wheel. In this respect, logic is counterintuitive. E.g., in the space of FOL clauses, $p(X,X)\vee p(X,Y)\vee p(Y,Z)$ clearly involves more objects than $p(X,X)$; nevertheless, they are equivalent under $\theta$ subsumption, since the latter is clearly a subset of the former and the former may become a subset of (in fact, equal to) the latter through substitution $\{Y/X,Z/X\}$.

Let us show a more practical example. The two structures in a block world shown in Figure 2 can be represented as follows:

$E_1$:

blocks(obj1) :− part_of(obj1,p1), part_of(obj1,p2), on(p1,p2), cube(p1), cube(p2), small(p1),big(p2), black(p1), stripes(p2).

$E_2$:

blocks(obj2) :− part_of(obj2,p3),part_of(obj2,p4), on(p3,p4), cube(p3), cube(p4), small(p3), big(p4), black(p4), stripes(p3).

Now, the lgg under $\theta$ subsumption between $E_1$ and $E_2$ is [27]

G:

blocks(X) :− part_of(X,X1), part_of(X,X2), part_of(X,X3), part_of(X,X4), cube(X4), on(X1,X2), cube(X1), cube(X2), cube(X3), small(X1), big(X2), black(X3), stripes(X4).

i.e., the most specific generalization of two structures, involving two objects each, involves four objects, which is quite counterintuitive! A person asked to tell what the two structures have in common would give two alternative answers: “a small cube on a big cube or a black cube and a stripes cube”, each capturing only the portion of correspondences that are consistent with each other. This is exactly what the ${\mathit{lgg}}_{OI}$ [40] yields:

$G_1$:

blocks(X) :− part_of(X,X1), part_of(X,X2), on(X1,X2), cube(X1), cube(X2), small(X1), big(X2).

$G_2$:

blocks(X) :− part_of(X,X1), part_of(X,X2), cube(X1), cube(X2), black(X1), stripes(X2).

6.2. Notes on Computational Complexity and Efficiency

Due to lack of space, a detailed account of computational complexity of the OI-related algorithms proposed in this paper cannot be provided here. Generally speaking, the complexity of processing each example is related to the degree of indeterminacy in the relational descriptions and depends on the ILP system (e.g., see [48] for InTheLEx). Still, we may elaborate on several aspects that show why the OI paradigm promises more efficiency than the standard approach. Crucial in this direction is the fact that equivalent formulas must be variants under ${\theta }_{OI}$ subsumption, as explained in the following.

A first consequence of this has to do with the mentioned existence of infinite unbounded strictly ascending/descending chains in the classical space of clauses ordered by $\theta$ subsumption. This is a much more relevant problem than computational complexity, since these chains may prevent termination of the refinement operators used in ML systems, and thus the very computability of generalizations/specializations [32]. On the contrary, the fact that equivalent formulas must be variants under ${\theta }_{OI}$ subsumption means that any proper generalization of a clause must be a proper subset of the clause (or, from the opposite perspective, that any proper superset of a clause is a proper specialization thereof). So, all possible generalizations of a clause are in finite number, and the number of literals in the examples represents a bound on the size of the generalizations to be searched by the ML systems. This is a guarantee of termination, because whenever we add or remove a literal we are actually moving to a different semantic item in the space, and sooner or later we will have to stop (in generalization because we reach reach an incomplete clause or, ultimately, the empty set; in specialization because we reach an inconsistent clause).

A second consequence is that, while under $\theta$ subsumption (possibly infinitely many) syntactically different clauses might be equivalent, under OI, every equivalence class includes only one clause (modulo variable renaming). In other words, the space of clauses under OI is more scattered than under $\theta$ subsumption; i.e., representing it as a graph, the number of nodes (clauses) and arcs (immediate generalization/specialization relationships) under OI is much larger. However, the meaning of every node and arc is much sharper than under $\theta$ subsumption, helping the refinement operators to better focus their search for a correct hypothesis. In regard to arcs specifically, this is good news for the purposes of ML, because learning is guided by the examples, and so only the generalizations/specializations that are compatible to the examples are to be explored, with the guarantee that every refinement step will lead us closer to the goal (the correct target concept). In regard to nodes, Lemma 1 states how many nodes are necessary under OI to represent one node in the unconstrained case. While this theoretical number may be large (due to the factorial), again one must consider that for ML purposes, no explicit expansion or translation is required: only the clauses under OI that are compliant with the examples are generated under the guidance of the examples, which are limited by the same cosiderations made above.

7. Operability: Systems and Applications

The OI framework is not just a theoretical matter. It also has an impact on effectiveness and efficiency of the ML systems, resulting in practical value. A study on the learning systems FOIL and FOCL found that, because of their not using the OI-compliant space, they cannot solve some learning tasks [31]. More in general, the traditional ILP systems proposed in the literature implicitly use the OI assumption when learning, but not when using the learned theory, resulting in an inconsistent behavior. In recent years, works on OI have focused more on the development of systems and on their application to several problems.

7.1. OI-Based Systems

Algorithms for implementing different versions of the OI-based refinement operators and experiments showing their performance can be found in [48,49,50,51]. The OI assumption was adopted in developing a series of ML systems, starting with the pair INDUBI/H [52] + INCR/H [53], where the former could learn from scratch, adopting a batch approach, a first version of the theory to be subsequently refined by the latter, adopting an incremental approach, when new examples became available. Subsequently, it was embedded in the incremental ML system InTheLEx [54] that combined different inference strategies (deductive, inductive, abductive, and similarity-based), and implemented the ideal operators for ${\theta }_{OI}$ subsumption in the space of Datalog clauses described in this paper. InTheLEx is the only existing fully and inherently incremental FOL ML system in the literature. An OI-compliant similarity measure for clauses, to be used for case-based reasoning, k-NN classification, clustering, and for guiding the search of the refinement operators, was presented in [48] and evaluated with several experiments, showing the effectiveness of the proposed operators and their efficiency (30% to 70% runtime savings). More recently, the OI framework was adopted in the state-of-the-art Process Mining and Management system WoMan [55,56] and in the automated MultiStrategy Reasoning engine GEAR [57]. WoMan is a declarative system that covers the whole range of tasks related to process management: discovery, supervision, analysis, simulation, prediction. It is the only existing fully incremental process mining system in the literature, and it was successfully applied to several real-world problems where other state-of-the-art systems in the literature failed. It embeds InTheLEx to learn pre- and post-conditions for process components, and it could learn successful models for several tough application domains, including ambient intelligence, education, and chess. GEAR aims at bringing to cooperation in a single inference engine several strategies that are typically investigated in isolation in the literature: induction, deduction, abduction, abstraction, ontological reasoning, similarity-based reasoning, uncertain/probabilistic reasoning, argumentation, and analogy.

The operability of the OI framework is also proven by its successful application to various real-world tasks and domains, of which we discuss some representative samples in the next subsections. In all these cases, the OI-based systems obtain state-of-the-art or better performance, often in less time than their competitors.

7.2. Mutagenesis

Prediction of mutagenicity of chemical compounds is an interesting testbed for the application of ML techniques because it is a relevant problem for biologists, but laboratory tests are costly. The problem consists in distinguishing molecules into “active” and “non-active” ones with respect to mutagenicity. The research in [6] found that for some molecules, classical regression-based techniques are not able to learn useful theories for mutagenicity, and domain experts were asked to define a (now classical) benchmark dataset of molecules suffering from this problem. The 188 molecules in the dataset are described with a total of 25917 atoms, for an average of about 138 atoms per observation. We applied InTheLEx and the OI-based k-NN classifier to this dataset, running a 10-fold cross-validation. InTheLEx endowed with the basic operators obtained 86% accuracy, while using the similarity-guided version of the lgg_OI operator it reached a slightly better predictive accuracy (87%) saving 70% runtime, indicating that it could quickly identify the correct correspondence of sub-parts of the compound descriptions. The k-NN approach using the same OI-based similarity measure for $k=13$ (square root of 188) obtained an average predictive accuracy of 87.22%.

We also compared the OI-based approach to that of other systems in the literature. In the original paper on mutagenesis [6], Progol obtained 88% accuracy, comparable to our results. Concerning systems based on k-NN, the performance of RIBL was just above 70% in the original paper [58] (compared to about 62% of FOIL 6.2). In a subsequent paper [59], RIBL was endowed with different kernels and compared to other systems. In the case of RIBL endowed with kNN, the performance was 77%, while the best performing algorithm in that comparison was SMD (Sum of Minimum Distances algorithm) with about 84%. On an extended dataset of 205 molecules, the k-RNN system [60] was reported to have different performance for 32 different parameter settings ($k=1\dots 20$ and the length of the saturated clause $l=2\dots 5$). In only three cases, their accuracy was slightly better than ours (89.31% for $l=4$ and $k=3$ or 4 and 88.25% for $l=3$ and $k=2$), and in all these cases k was always very low with respect to the classical square-root setting; for $k=15$, the best performance reached by k-RNN was 85.72%.

7.3. Document Image Processing

The problem of Document Image Classification consists in identifying the type of document based only on its layout, i.e., without reading its content. It is important for libraries, archives, administrations, and other kinds of document repositories, because documents can be selected, organized, and forwarded to appropriate processing steps while saving computational resources. It is challenging because the document layout can be very variable and its descriptions are characterized by significant indeterminacy (the problem, in FOL-based ML, of identifying the correct mapping between objects in different descriptions). InTheLEx was applied to several document image classification datasets. We report here about two of them, in two very different but challenging settings: contemporary documents (namely scientific papers) and historical documents.

7.3.1. Scientific Papers

Experiments on scientific papers were run on a real-world dataset (available at http://lacam.di.uniba.it/systems/inthelex/index.htm#datasets, consulted on 28 March 2025) containing 353 descriptions of scientific papers belonging to four different classes, some of which had very similar layout styles: 52 from Elsevier journals, 75 from the Springer–Verlag Lecture Notes series (SVLN), 95 from the Journal of Machine Learning Research (JMLR), and 131 from the Machine Learning Journal (MLJ) [48]. Only the first page of each paper was described as the most relevant to identify the paper’s type. The 353 documents were described with a total of 67920 literals, for an average of more than 192 literals per description (some included more than 400 literals). Effectiveness was evaluated using a stratified 10-fold cross-validation (i.e., we ensured that the proportion of examples from the different classes in each fols was the same as for the overall dataset).

First, we ran experiments on model-based concept learning, using InTheLEx under two different settings: with the basic generalization operator (B) and with the similarity-guided generalization operator (S). In addition to Predictive Accuracy, we also used F-measure (with parameter 1 in order to equally weight Precision and Recall) as performance measures to check effectiveness on both positive and negative examples (indeed, the dataset is not balanced: since each positive example for a class is a negative example for the other classes, the negative examples for a class are three times the positive ones).

Table 2. Classification results for Document Image Classification on scientific papers.

		Cl	Accuracy	F1-Measure
	S	1.9	0.98	0.97
JMLR	B	1.9	0.98	0.97
	k	—	0.90	—
	S	1	1.00	1.00
Elsevier	B	2.1	0.99	0.97
	k	—	1.00	—
	S	4.7	0.96	0.94
MLJ	B	5.2	0.93	0.91
	k	—	1.00	—
	S	2.6	0.98	0.94
SVLN	B	3.3	0.97	0.93
	k	—	0.90	—
	S	2.55	0.98	0.96
Average	B	3.125	0.97	0.94
	k	—	0.94	—

summarizes the outcomes, averaged on the 10 folds, reporting in Column Cl the number of clauses in the learned theories. The similarity-driven version outperformed the basic one for both measures; overall, in the 40 runs it saved 1.15 h, resulting in a 98% average accuracy (+1% with respect to the basic version) and 96% average F-measure (+2% with respect to the basic version).

Then, we turned to the k-NN approach. k was set, as recommended by the literature, to the square root of the number of learning instances, i.e., 17. Note that the classification was a multi-class one, so ties are possible. Rows labeled k in Table 2 report the average accuracy. The overall accuracy of 94% means that the OI-based similarity approach is very effective. A deeper analysis revealed that very often in the correct cases almost all of the nearest neighbors were from the same class. Classes Elsevier and MLJ had 100% accuracy (also in every single fold); however, also on SVLN and JMLR, high accuracy rates were reached. An analysis of the errors revealed that MLJ is quite distinct from the other classes, while Elsevier, although well recognizable in itself, is somehow in between JMLR and SVLN, which are also close to each other.

We also ran experiments on Conceptual Clustering, based on the classical K-means algorithm, using medoids as cluster representatives (centroids cannot be used because FOL formulas do not induce an euclidean space), asking for four clusters and stopping the procedure when a new iteration returned a partition already seen in previous iterations. Computing each similarity took on average 2.27 s, a reasonable time considering the complexity of the descriptions. Since the correct class of each document in the dataset is known, we associated each cluster to the class with maximum overlapping. Results are reported in

Table 3. Experimental results for Document Clustering on scientific papers.

Cluster	Size	Class	P (%)	R (%)
1	65	Elsevier	80	100
2	65	SVLN	98.46	85.33
3	105	JMLR	90.48	100
4	118	MLJ	97.46	87.79
average			91.60	93.28

: for each dataset size, it reports the number of instances in each cluster (column “Size”) and the corresponding class. Precision (P) and Recall (R) values are all above 80%, showing great effectiveness of the approach. A qualitative evaluation of the clustering outcomes revealed that errors were made on very ambiguous documents. Compound statistics are reported in the last row for an overall purity (the clustering counterpart of accuracy) equal to 92.35, a number comparable to results of supervised learning. Preliminary comparisons with other classical measures (Jaccard’s, Tverski’s and Dice’s) report an improvement of up to +5.48% for Precision, up to +8.05% for Recall, and up to +2.83% for Purity.

7.3.2. Historical Documents

Experiments on historical documents were run on a dataset coming from the EU IST-1999-20882 project COLLATE (“Collaboratory for Annotation, Indexing and Retrieval of Digitised Historical Archive Materials”, 2000–2002), consisting of 114 digitized historical documents related to film censorship and produced between the two World Wars. The aim was to train models for recognizing two document classes: registration cards from the Filmarchiv Austria (FAA) and censorship decision reports from the Deutsches Filminstitut (DIF). Specifically, the dataset included 34 FAA registration cards, 19 DIF censorship decisions, and 61 reject documents, obtained from newspaper articles and DIF registration cards. Samples of first and last pages of these documents are reported in Figure 3, where the complexity of the documents can be appreciated (they carry stamps, signatures, typed text, pencil signs, etc.): FAA registration cards are on the top-left, DIF censorship decisions on the right, and DIF censorship decisions on the bottom-left. Note that some reject documents (DIF censorship decisions) are very similar to FAA registration cards, making the task even more challenging. When expressed in FOL, the description length in literals ranged from 40 to 379 atoms per observation/example [61].

The results of a 10-fold cross-validation evaluation are shown in

Table 4. Classification Results for COLLATE historical documents.

	Clauses	Length	Runtime	Accuracy	Pos	Neg
faa-registration-card	1	12.5	90.13	1.0	1.0	1.0
dif-censorship-decision	1.1	25.8	5761.8	0.963	0.789	1.0

. Columns “Clauses” and “Length” report the average number of clauses for the 10 folds in the generated theories and the average number of literals in the clauses, respectively. We note that basically the characterizing and discriminating features of the complex layout models for these documents could be captured in just one clause for FAA registration cards and about one clause for DIF censorship decisions. The model length ranged between one and two dozen literals, which ensures human readability and understandability of the models. The average runtime for learning the models, reported in the third column (in sec), shows that efficiency is sufficient for practical real-world applications. Finally, the last three columns report average accuracy of the learned models (overall and for positive and negative examples only), which is perfect for FAA documents and nearly perfect for DIF ones. Also, note that only a few dozen documents per class were needed to learn such high-quality models.

7.4. Process Mining and Management

Process Management is relevant because one can automatically check correctness of process executions based on models of the intended process. Process Mining is relevant because such models can be learned automatically from observation of cases of process executions, saving money with respect to manual development and obtaining models which directly reflect the actual practice. These models can also be useful to analyze the current practice in order to identify problems and improve it, provided that the models are human-understandable. To this purpose, declarative approaches, adopting logic-like representations, are more suitable. A source of complexity in the process mining task is that only positive examples are available to learn the models, and so the safeguard role played by negative examples in ML is missing.

While process management is typically applied to industrial or administrative processes that are quite stable and strictly defined, real life is pervaded by processes with much higher complexity and variability. Most systems available in the literature are unable to represent or learn complex models in real-life situations, which motivated us to develop the WoMan framework and system, declarative and based on OI. WoMan’s models consist of two main kinds of components: “tasks”, specifying allowed types of activities, and “transitions”, expressing permitted task combinations. We report in the following subsections a few experiments in different domains.

7.4.1. Complex Artificial Process Models

To test WoMan under tough conditions, we ran controlled experiments that allowed to tune the amount and kind of complexity in the models. A total of 11 very complex artificial workflow models were built and, following [62], for each model, 1000 training cases were generated at random and used as training examples to learn the models from scratch. Very interestingly, in all training problems, just 50 training cases were sufficient for WoMan to learn a complete and correct model, confirming the usefulness of the incremental approach and its ability to quickly learn accurate models from very few examples.

We compared WoMan to two state-of-the-art systems with very different characteristics: a more traditional one, Little Thumb [62], and one based on genetic algorithms [63].

Table 5. Experimental outcomes on artificial workflow models.

Id	Little Thumb	Genetic	Time (Genetic)
1	y	n	1′
2	y	e	>3′
3	y	e	>1′
4	n	y	>1′
5	n	n	>25′
6	n	n	>4′
7	n	e	>30′
8	n	e	37″
9	y	n	31″
10	y	y	21″
11	y	e	30″

summarizes the learning outcomes of the two approaches (where y = correct model; n = incorrect model; e = formally wrong model). Reference [62] was unable to learn 5 of the 11 models, and [63] performed even worse: it learned the correct model only in 2 cases, and in 5 of the remaining cases it learned a formally wrong one. Its runtimes, reported in the last column, confirm its very low efficiency: it never took less than 21 s, and in one case it took more than half an hour. On the other hand, WoMan never took more than 10 s in all models.

7.4.2. Daily Routines

Learning models of people’s behavior in daily routines is interesting for using these models in personalized domotic and ambient intelligence. In this real-world context, we ran several experiments aimed at assessing the practical applicability of WoMan.

One concerned the “Aruba” dataset taken from the CASAS benchmark repository (available at https://casas.wsu.edu/datasets/, accessed on 29 March 2025). This dataset includes continuous recordings of home activities, involving different combinations of 10 tasks along 220 days, yielding 220 cases of the the home inhabitant’s daily behavior. Overall, it consists of 13 788 events, referred to 6674 activities, yielding an average of 62.67 events per case. Since the “correct” model for Aruba is not available, the learned model’s accuracy was evaluated by an “inverse” 10-fold cross-validation, in which each fold used just 10% cases for learning and the remaining 90% for testing. While [62,63] returned formally wrong models, WoMan processed the whole dataset in 0.1 s (including translation from the case logs to the FOL representation, which proves that the approach is suitable for real-world applications) and always returned syntactically valid models, yielding an average accuracy of 92% (consider that each missed case costs nearly 5% accuracy and that in real life there are always deviations from the routine that prevent reaching 100% accuracy). Being in a positive-examples-only setting, precision is always 100% for both tasks and transitions. The Recall and F1-measure results reported in

Table 6. Learning performance on the Aruba dataset workflow models.

	Tasks	Transitions
Recall	91.36%	55.49%
F1-measure	95.48%	71.37%

indicate a success, even more considering that just 22 training cases were used.

The Aruba log also reports the status of several sensors installed in the home, from which 5976 training examples (27.16 per case on average) for task pre- and post-conditions were obtained and processed by InTheLEx. The number of atoms in the example descriptions ranged from 740 to 2570 for preconditions and from 12,882 to 175,168 for postconditions, which are really huge numbers in ILP. Still, InTheLEx took on average just 0.12 s to process each example, which again ensures on-line applicability to real-world tasks. Notwithstanding the positive-examples-only setting, the lgg_OI operator in InTheLEx returned a theory consisting of 12 clauses that preserve all relevant information. The following sample,

sleeping(A)	:-	true.
eating(A)	:-	next(_,A).
meal_preparation(A)	:-	next(_,A).
bed_to_toilet(A)	:-	next(_,A), sensor_m004(A,B), status_on(B),

demonstrates human readability of the model that can be interpreted as follows: “sleeping” has no precondition (i.e., it may happen at any time); “eating”, “meal_preparation”, and “bed_to_toilet” are always preceded by another activity; “bed_to_toilet” takes place only when the movement sensor on the bathroom door (m004) is on. A classical 10-fold cross-validation with random sampling obtained 99.86% accuracy for pre-conditions and 99.75% accuracy for post-conditions. Even using the “inverted” 10-fold cross-validation (where each fold only included training examples from 22 consecutive days), WoMan still obtained 98.27% accuracy for pre-conditions and 98.42% accuracy for post-conditions.

7.4.3. Process-Related Prediction

WoMan can also predict the next activity in a process execution or the kind of process being executed among a number of candidate models, returning a ranking of candidates and abstaining when not sufficiently certain. This is important in real-world applications, where there is much more variability and subjectivity in behavior. To evaluate these features, we used the following datasets (for which

Table 7. Process prediction dataset statistics.

	Cases	Events		Tasks
		Overall	Avg	Overall
Aruba	220	13,788	62.67	10
GPItaly	253	185,844	369.47	8
White	158	36,768	232.71	681
Black	87	21,142	243.01	663
Draw	155	32,422	209.17	658

reports some statistics):

Aruba 

described in the previous section.

GPItaly 

reports the movements of an elderly person in the rooms of her home [64] along 253 days. Each day was a case of the process representing the movement routine.

Chess 

consists of 400 reports of top-level matches downloaded from https://www.federscacchi.com/fsi/index.php/punteggi/archivio-partite (consulted on 29 March 2025). A case was a chess match.

The chess dataset is characterized by very high parallelism: each game starts with 32 concurrent activities (well beyond the reach of many current process mining systems).

A k-fold cross-validation procedure was used (see column Folds in

Table 8. Activity prediction performance.

	Folds	Pred	Recall	Rank	(Tasks)
Aruba	3	0.85	0.97	0.92	6.06
GPItaly	3	0.99	0.97	0.96	8.02
chess	5	0.54	0.98	1.0	11.34

for the values of k). After learning the models, each event in the test cases updated the project status and called for a prediction about the next expected activity. Table 8 reports performance averages concerning the different processes: in how many cases a prediction was returned (“Pred”); how many of such predictions include the correct activity (“Recall”), how high it is in the ranking (“Rank”: 1.0 means it is the top one, 0.0 means it is the last one); and the average length of the ranking (“Tasks”). Most of the times WoMan makes a prediction, and in 97-98% of the times the correct activity is in the ranking, and always in the top 10% positions (always at the very top in the chess processes). WoMan is more cautions on predictions in the chess domain but still covers more than half of the match, and correctness is at the same level or better than the state-of-the-art Deep Learning and Neural Network-based approaches.

The process prediction task was evaluated on the chess dataset because it provided three different kinds of processes based on the same domain, corresponding to the possible match outcomes: white wins, black wins, or draw. The aim was predicting the match outcome as long as match events happened. We used the same folds and models as for activity prediction, but calling the process prediction function (that, in this case, always returns a prediction).

Table 9. Process prediction performance.

	Folds	Pos (%)	C	A	U	W
black	5	0.47	0.20	0.00	0.15	0.66
white	5	0.70	0.44	0.00	0.15	0.40
draw	5	0.61	0.29	0.01	0.18	0.52

summarizes the performance: Column “Pos” reports the average position of the correct prediction in the ranking (normalized to $[0,1]$, where 1 represents the top of the ranking and 0 its bottom). The last columns report, on average, for what percentage of the case duration the prediction was correct (C: the correct process was alone at the top of the ranking), approximately correct (A: the correct process shared the top of the ranking with other, but not all, processes), undefined (U: all processes were ranked equal), or wrong (W: the correct process was not at the top of the ranking). In the middle phase of the match, which is the core part for some respects, the percentage of correct predictions is much higher than what is reported in Table 9.

8. Conclusions and Future Work

FOL can be the optimal paradigm to bring together systems programming, DB management, and knowledge-based applications. The LP fragment specifically is a general-purpose technique for building systems suitable for handling relational DBs and transparent and explainable by design. This latter feature is especially important in the current technological landscape, where the use of AI in critical applications requires trustworthiness and understandability by humans. In fact, “ILP systems have discovered new knowledge that has been refereed and published in journals of the relevant subject area” [6]. In this field, the OI framework brings several advantages to ML systems, especially incremental ones, in terms of closer resemblance to human knowledge representation, efficiency, and effectiveness. This paper collected and presented in a uniform way, for the first time, the whole theoretical corpus underlying OI. It can serve as a reference for researchers interested in developing systems based on this framework, as a starting point to further expand the framework, and as a reminder that symbolic AI approaches should be still investigated, in spite of the current hype for subsymbolic (especially neural network-based) ones, in order to ensure trustworthiness, explainability, and anthropocentricity of AI solutions (as also required by recent EU norms). To show the practical usefulness of the OI framework beyond pure theory, the paper also discussed the theoretical bases for its efficiency and provided examples of real OI-based systems that were successfully applied to real-world tasks.

There are several directions for future work on the OI framework. On the theoretical side, a better assessment of computational complexity could be carried out, and the consequences of applying OI to functions could be investigated in more depth. On the implementation side, to the best of our knowledge, no algorithm or system has been developed to date for unconstrained clausal logic or for OI implication. On the application side, more tasks and domains could be explored, and experiments could be run to investigate applicability, effectiveness, and efficiency of OI-based ML under different conditions. Last but not least, the relationships and possible contributions of the OI framework to the current debate on ethics in AI should be investigated. The framework was born before the pervasive spread of AI in everyday life that happened a few years ago and posed the problem of ensuring that the uses of AI are ethical and human-compatible, especially in critical applications where privacy, security, and wellness of people are at stake. Logic-based approaches, such as the OI framework, are understandable and explainable by design, providing real causal explanations rather than post hoc ones or simply interpretable outcomes; as shown, compared to standard FOL-based approaches, OI bears additional resemblance to the human way of representing and expressing knowledge. Also, incremental approaches to ML are naturally suited to interactive behavior, fostering anthropocentricity. Logic-based approaches are also more amenable to learning from very few, selected examples, which may be related to the sustainability problem, not requiring huge computational resources which have an impact on the environment.