 freely available
 reusable
Entropy 2015, 17(2), 852865; doi:10.3390/e17020852
Abstract
: For conditional probabilistic knowledge bases with conditionals based on propositional logic, the principle of maximum entropy (ME) is wellestablished, determining a unique model inductively completing the explicitly given knowledge. On the other hand, there is no general agreement on how to extend the ME principle to relational conditionals containing free variables. In this paper, we focus on two approaches to ME semantics that have been developed for firstorder knowledge bases: aggregating semantics and a grounding semantics. Since they use different variants of conditionals, we define the logic PCI, which covers both approaches as special cases and provides a framework where the effects of both approaches can be studied in detail. While the ME models under PCIgrounding and PCIaggregating semantics are different in general, we point out that parametric uniformity of a knowledge base ensures that both semantics coincide. Using some concrete knowledge bases, we illustrate the differences and common features of both approaches, looking in particular at the ground instances of the given conditionals.1. Introduction
Probabilistic conditional knowledge bases containing conditionals of the form (BA)[d] with the reading “if A, then B with probability d” are a powerful means for knowledge representation and reasoning when uncertainty is involved [1,2]. If A and B are propositional formulas over a propositional alphabet Σ, possible worlds correspond to elementary conjunctions over Σ, where an elementary conjunction is a conjunction containing every element of Σ exactly once, either in nonnegated or in negated form. A possible worlds semantics is given by probability distributions over the set of possible worlds, and a probability distribution P satisfies (BA)[d] if for the conditional probability $P(BA)=\frac{P(A\wedge B)}{P(A)}$ the relation P (BA) = d holds. For a knowledge base $\mathcal{R}$ consisting of a set of propositional conditionals, P is a model of $\mathcal{R}$ if P satisfies each conditional in $\mathcal{R}$. The principle of maximum entropy (ME principle) is a wellestablished concept for choosing the uniquely determined model of $\mathcal{R}$ having maximum entropy. This model is the most unbiased model of $\mathcal{R}$ in the sense that it completes the knowledge given by $\mathcal{R}$ inductively but adds as little additional information as possible [3–9].
While for a set of propositional conditionals there is a general agreement about its ME model, the situation changes when the conditionals are built over a relational firstorder language. As an illustration, consider the following example.
Example 1 (Elephant Keeper). The elephant keeper example, adapted from [10,11], models the relationships among elephants in a zoo and their keepers. Elephants usually like their keepers, except for keeper Fred. However, elephant Clyde gets along with everyone, and therefore he also likes Fred. The knowledge base $\mathcal{R}$_{EK} consists of the following conditionals:
Conditional ek_{1} models statistical knowledge about the general relationship between elephants and their keepers, whereas conditional ek_{2} represents knowledge about the exceptional keeper Fred and his relationship to elephants in general. Conditional ek_{3} models the subjective belief about the relationship between the elephant Clyde and keeper Fred. From a common sense point of view, the knowledge base $\mathcal{R}$_{EK} makes perfect sense: conditional ek_{2} is an exception from ek_{1}, and ek_{3} is an exception from ek_{2}.
When trying to extend the ME principle from the propositional case to such a relational setting, a central question is how to interpret the free variables occurring in a conditional. For instance, note that a straightforward complete grounding of $\mathcal{R}$_{EK} yields a grounded knowledge base that can be viewed as a propositional knowledge base. However, this grounded knowledge is inconsistent since it contains both (likes(clyde, fred)elephant(clyde), keeper(fred))[0.9] and (likes(clyde, fred)elephant(clyde), keeper(fred))[0.05], and no probability distribution P can satisfy both P (likes(clyde, fred)elephant(clyde), keeper(fred)) = 0.9 and P (likes(clyde, fred)elephant(clyde), keeper(fred)) = 0.05.
Thus, when extending the ME principle to the relational case with free variables as in $\mathcal{R}$_{EK}, the exact role of the variables has to be specified. There are various approaches dealing with a combination of probabilities with a firstorder language (e.g., [12,13]); a comparison and evaluation of some approaches is given [14]). In the following, we focus on two semantics that both employ the principle of maximum entropy for probabilistic relational conditionals, the aggregation semantics [15] proposed by KernIsberner and the logic FOPCL [16] elaborated by Fisseler. While both approaches are related in the sense that they refer to a set of constants when interpreting the variables in the conditionals, there is also a major difference. FOPCL requires all groundings of a conditional to have the same probability d given in the conditional, and in general, FOPCL needs to restrict the possible instantiations for the variables occurring in a conditional by providing constraint formulas like U ≠ V or U ≠ a in order to avoid inconsistencies. On the other hand, under aggregation semantics the grounded instances may have distinct probabilities as long as they aggregate to the given probability d, and aggregation semantics is defined only for conditionals without constraint formulas.
In this paper, a logical framework PCI extending aggregation semantics to conditionals with instantiation restrictions and also providing a grounding semantics is proposed. From a knowledge representation point of view, this provides greater flexibility, e.g., when expressing knowledge about individuals known to be exceptional with respect to some relationship. We show that both the aggregation semantics of [15] and the semantics of FOPCL [16] come out as special cases of PCI, thereby also helping to clarify the relationship between the two approaches. Moreover, we investigate the ME models under PCIgrounding and PCIaggregating semantics, which are different in general, and we give a condition on knowledge bases ensuring that both ME semantics coincide.
This paper is a revised and extended version of [17] and is organized as follows. In Section 2, we very briefly recall the background of FOPCL and aggregation semantics. In Section 3, the logic framework PCI is developed and two alternative satisfaction relations for grounding and aggregating semantics are defined for PCI by extending the corresponding notions of [15,16]. In Section 4, the maximum entropy principle is employed with respect to these satisfaction relations; we show that the resulting semantics coincide for knowledge bases that are parametrically uniform [11,16]. In Section 5, we present and discuss ME distributions for some concrete knowledge bases both under PCIgrounding and PCIaggregating semantics, and point out their differences and common features, covering in particular the groundings of the given conditionals. Finally, in Section 6 we conclude and point out further work.
2. Background: FOPCL and Aggregation Semantics
As already pointed out in Section 1, simply grounding a relational knowledge base $\mathcal{R}$ easily leads to inconsistency. Therefore, the logic FOPCL [11,16] employs instantiation restrictions for the free variables of a conditional. An FOPCL conditional has additionally a constraint formula determining the admissible instantiations of free variables, and the grounding semantics of FOPCL requires that all admissible ground instances of a conditional c must have the probability given by c.
Example 2 (Elephant Keeper with instantiation restrictions). In FOPCL, adding K ≠ fred to conditional ek_{1} and E ≠ clyde to conditional ek_{2} in $\mathcal{R}$_{EK} yields the knowledge base $\mathcal{R}$′_{EK} with:
Note that, e.g., the ground instance (likes(clyde, fred)elephant(clyde), keeper(fred))[0.05] of conditional ek′_{2} is not admissible, and that the set of admissible ground instances of $\mathcal{R}$′_{EK} is indeed consistent under probabilistic semantics for propositional knowledge bases as considered, e.g., in [8,18].
Thus, under FOPCL semantics, $\mathcal{R}$′_{EK} is consistent, where a probability distribution P satisfies an FOPCL conditional r, denoted by $P{=}_{\text{fopcl}}r$, iff all admissible ground instances of r have the probability specified by r.
In contrast, the aggregation semantics, as given in [15], does not consider instantiation restrictions, since its satisfaction relation (in this paper denoted by ${=}_{\odot}^{noir}$ to indicate no instantiation restriction), is less strict with respect to probabilities of ground instances: $P{=}_{\odot}^{\mathit{\text{noir}}}$(BA)[d] iff the quotient of the sum of all probabilities P (B_{i} ∧ A_{i}) and the sum of P (A_{i}) is d, where (B_{1}A_{1}),…, (B_{n}A_{n}) are the ground instances of (BA). In this way, the aggregation semantics is capable of balancing the probabilities of ground instances, resulting in greater flexibility and higher tolerance with respect to consistency issues. Provided that there are enough individuals so that the corresponding aggregation over all probabilities is possible, the knowledge base $\mathcal{R}$_{EK} that is inconsistent under FOPCL semantics is consistent under aggregation semantics.
3. PCI Logic
The logical framework PCI (probabilistic conditionals with instantiation restrictions) uses probabilistic conditionals with and without instantiation restrictions and provides different options for a satisfaction relation. The syntax of PCI given in [19] uses the syntax of FOPCL [11,16]. In the following, we will precisely state the formal relationship among ${=}_{\odot}^{\mathit{\text{noir}}}$, ${=}_{\text{fopcl}}$, and the satisfaction relations offered by PCI.
As FOPCL, PCI uses functionfree, sorted signatures of the form Σ = ( $\mathcal{S}$, mathvariant='script' $\mathcal{D}$, Pred). In a PCIsignature Σ = ( $\mathcal{S}$, $\mathcal{D}$, Pred), $\mathcal{S}$ = {s_{1},…, s_{k}} is a set of sort names or just sorts. The set $\mathcal{D}$ is a finite set of constants symbols where each d ∈$\mathcal{D}$ has a unique sort s ∈$\mathcal{S}$. With ${\mathcal{D}}^{(s)}$ we denote the set of all constants having sort s; thus $\mathcal{D}={\cup}_{s\in \mathcal{S}}{\mathcal{D}}^{(s)}$ is a set being the union of (disjoint) sets of sorted constant symbols. Pred is a set of predicate symbols, each having a particular number of arguments. If p ∈ Pred is a predicate taking n arguments, each argument position i must be filled with a constant or variable of a specific sort s_{i}. Thus, each p ∈ Pred comes with an arity of the form s_{1} ×…× s_{n} ∈${\mathcal{S}}^{n}$ indicating the required sorts for the arguments. Variables $\mathcal{V}$ also have a unique sort, and all formulas and variable substitutions must obey the obvious sort restrictions. In the following, we will adopt the unique names assumption, i.e., different constants denote different elements. The set of all terms is defined as $Ter{m}_{\sum}:=\mathcal{V}\cup \mathcal{D}$. Let $\mathcal{L}$_{Σ} be the set of quantifierfree firstorder formulas defined over Σ and $\mathcal{V}$ in the usual way.
Definition 1 (Instantiation Restriction). An instantiation restriction is a conjunction of inequality atoms of the form t_{1} ≠ t_{2} with t_{1}, t_{2} ∈ Term_{Σ}. The set of all instantiation restriction is denoted by C_{Σ}.
Since an instantiation restriction may be a conjunction of inequality atoms, we can express that a conditional has multiple restrictions, e.g., by stating E ≠ clyde ∧ K ≠ fred.
Definition 2 (q, p, rConditional). Let A, B ∈ $\mathcal{L}$_{Σ} be quantifierfree firstorder formulas over Σ and$\mathcal{V}$.
(BA) is called a qualitative conditional (or just qconditional). Note that A is the antecedence and B the consequence of the qualitative conditional. The set of all qualitative conditionals over $\mathcal{L}$_{Σ} is denoted by ($\mathcal{L}$_{Σ}$\mathcal{L}$_{Σ}).
Let (BA) ∈ ($\mathcal{L}$_{Σ}$\mathcal{L}$_{Σ}) be a qualitative conditional and let d ∈ [0, 1] be a real value. Here (BA)[d] is called a probabilistic conditional (or just pconditional) with probability d. The set of all probabilistic conditionals over $\mathcal{L}$_{Σ} is denoted by ($\mathcal{L}$_{Σ}$\mathcal{L}$_{Σ})^{prob}.
Let (BA)[d] ∈ ($\mathcal{L}$_{Σ}$\mathcal{L}$_{Σ})^{prob} be a probabilistic conditional and let C ∈ C_{Σ} be an instantiation restriction. In addition, 〈(BA)[d], C〉 is called an instantiation restricted conditional (or just rconditional). The set of all instantiation restricted conditionals over $\mathcal{L}$_{Σ} is denoted by${({\mathcal{L}}_{\mathrm{\Sigma}}{\mathcal{L}}_{\mathrm{\Sigma}})}_{{C}_{\mathrm{\Sigma}}}^{prob}$.
Instantiation restricted qualitative conditionals are defined analogously. If it is clear from the context, we may omit qualitative, probabilistic, and instantiation restricted and just use the term conditional.
Definition 3 (PCI knowledge base). A pair (Σ, $\mathcal{R}$) consisting of a PCI signature$\mathrm{\Sigma}=(S,D,\mathit{\text{Pred}})$ and a set of instantiation restricted conditionals $\mathcal{R}$ = {r_{1},…, r_{m}} with r_{i} ∈ ${({\mathcal{L}}_{\mathrm{\Sigma}}{\mathcal{L}}_{\mathrm{\Sigma}})}_{{C}_{\mathrm{\Sigma}}}^{prob}$ is called a PCI knowledge base.
For an instantiation restricted conditional r = 〈(BA)[d], Ci, Θ_{Σ}(r) denotes the set of all ground substitutions with respect to the variables in r. A ground substitution θ ∈ Θ_{Σ}(r) is applied to the formulas A, B and C in the usual way, i.e., each variable is replaced by a certain constant according to the mapping θ = {v_{1}/c_{1},…, v_{l}/c_{l}} with v_{i} ∈ $\mathcal{V}$, c_{i} ∈ $\mathcal{D}$, 1 ≤ i ≤ l. Therefore, θ(A), θ(B), and θ(C) are ground formulas and we have θ((BA)) := (θ(B)θ(A)).
Given a ground substitution θ over the variables occurring in an instantiation restriction C ∈${\mathcal{C}}_{\mathrm{\Sigma}}$, the evaluation of C under θ, denoted by ${\u301aC\u301b}_{{}^{\theta}}$, yields true iff θ(t_{1}) and θ(t_{2}) are different constants for all t_{1} ≠ t_{2} ∈ C.
Definition 4 (Admissible Ground Substitutions and Instances). Let Σ = ( $\mathcal{S}$, $\mathcal{D}$, Pred) be a manysorted signature and let r = 〈(BA)[d], C〉 ∈ ${({\mathcal{L}}_{\mathrm{\Sigma}}{\mathcal{L}}_{\mathrm{\Sigma}})}_{{C}_{\mathrm{\Sigma}}}^{prob}$ be an instantiation restricted conditional. The set of admissible ground substitutions of r is defined as
The set of admissible ground instances of r is defined as
In the following, when we talk about the ground instances of a conditional, we will always refer to its admissible ground instances.
As for an FOPCL knowledge base [11], for a PCI knowledge base (Σ, $\mathcal{R}$) we define the Herbrand base $\mathscr{H}$($\mathcal{R}$) as the set of all ground atoms in all gnd_{Σ}(r_{i}) with r_{i} ∈ $\mathcal{R}$. Every subset ω ⊆ $\mathscr{H}$($\mathcal{R}$) is a Herbrand interpretation, defining a logical semantics for $\mathcal{R}$. The set Ω_{Σ} := {ω  ω ⊆ $\mathscr{H}$($\mathcal{R}$)} denotes the set of all Herbrand interpretations. Herbrand interpretations are also called possible worlds.
Definition 5 (PCI Interpretation). The probabilistic semantics of (Σ,$\mathcal{R}$) is a possible worlds semantics [12] where the ground atoms in $\mathscr{H}$($\mathcal{R}$) are binary random variables. A PCI interpretation P of a knowledge base (Σ, $\mathcal{R}$) is thus a probability distribution P : Ω_{Σ} → [0, 1]. The set of all probability distributions over Ω_{Σ} is denoted by${\mathcal{P}}_{{\mathrm{\Omega}}_{\mathrm{\Sigma}}}$ or just by${\mathcal{P}}_{\mathrm{\Omega}}$.
The PCI framework offers two different satisfaction relations: ${=}_{\u25b3}^{\mathrm{pci}}$ is based on grounding as in FOPCL, and ${=}_{\u229b}^{\mathrm{pci}}$ extends aggregation semantics to rconditionals.
Definition 6 (PCI Satisfaction Relations). Let P ∈${\mathcal{P}}_{\mathrm{\Omega}}$ and let 〈(BA)[d], C〉 ∈ ${({\mathcal{L}}_{\mathrm{\Sigma}}{\mathcal{L}}_{\mathrm{\Sigma}})}_{{C}_{\mathrm{\Sigma}}}^{prob}$ be an rconditional with${\sum}_{\theta \in \mathrm{\Theta}\underset{\mathrm{\Sigma}}{adm(\langle (BA)[d]},C\rangle )}P(\theta (A))>0$. The two PCI satisfaction relations${=}_{\u25b3}^{\mathrm{pci}}$ and${=}_{\u229b}^{\mathrm{pci}}$are defined by:
As usual, the satisfaction relations ${=}_{\u2605}^{\mathrm{pci}}$ with ★ ∈ {△, ⊛} are extended to a set of conditionals $\mathcal{R}$ by defining
The following proposition states that PCI properly captures both the instantiationbased semantics =_{fopcl} of FOPCL [11] and the aggregation semantics ${=}_{\odot}^{\mathit{\text{noir}}}$ of [15] (cf. Section 2).
Proposition 1 (PCI captures FOPCL and aggregation semantics [19]). Let 〈(BA)[d];C〉 be an rconditional and let (BA)[d] be a pconditional, respectively. Then the following holds:
4. PCI Logic and Maximum Entropy Semantics
If a knowledge base $\mathcal{R}$ is consistent, there are usually many different models satisfying $\mathcal{R}$. The principle of maximum entropy chooses the unique distribution that has maximum entropy among all distributions satisfying a knowledge base $\mathcal{R}$ [5,8]. Applying this principle to the PCI satisfaction relations ${=}_{\u25b3}^{\mathrm{pci}}$ and ${=}_{\u229b}^{\mathrm{pci}}$ yields
Example 3 (Misanthrope). The knowledge base $\mathcal{R}$_{MI} = {R_{1}, R_{2}}, adapted from [11], models friendship relations within a group of people with one exceptional member, a misanthrope. In general, if a person V likes another person U, then it is very likely that U likes V, too. However, there is one person, the misanthrope, who generally does not like other people:
Under ME_{△}($\mathcal{R}$_{MI}), all six ground conditionals emerging from R_{1} have probability 0.9, for instance, ME_{△}($\mathcal{R}$_{MI})(likes(a, b)  likes(b, a)) = 0.9.
On the other hand, for the distribution ME_{⊛}($\mathcal{R}$_{MI}), we have ME_{⊛}($\mathcal{R}$_{MI}) (likes(a, b)  likes(b, a)) = 0.46016768 and ME_{⊛}($\mathcal{R}$_{MI})(likes(a, c)  likes(c, a)) = 0.46016768, while the other four ground conditionals resulting from R_{1} have probability 0.96674480.
Example 3 shows that in general the ME model under PCIgrounding semantics of a knowledge base $\mathcal{R}$ differs from its ME model under PCIaggregation semantics. However, if $\mathcal{R}$ is parametrically uniform [11,16], the situation changes. Parametric uniformity of a knowledge base $\mathcal{R}$ is introduced in [11] and refers to the fact that the ME distribution under FOPCL (or PCIgrounding) semantics satisfying a set of m ground conditionals can be represented by a set of just m optimization parameters. A relational knowledge base $\mathcal{R}$ is parametrically uniform iff for every conditional r ∈ $\mathcal{R}$, all ground instances of r have the same optimization parameter (see [11,16] for details). For instance, the knowledge base $\mathcal{R}$′_{EK} from Example 2 is parametrically uniform, while the knowledge base $\mathcal{R}$_{MI} from Example 3 is not parametrically uniform. Thus, if $\mathcal{R}$ is parametrically uniform, just one optimization parameter for each conditional r ∈ $\mathcal{R}$ instead of one optimization parameter for each ground instance of r has to be computed; this can be exploited when computing the ME distribution [17]. In [20], a set of transformation rules is developed that transforms any consistent knowledge base $\mathcal{R}$ into a knowledge base $\mathcal{R}$′ such that $\mathcal{R}$ and $\mathcal{R}$′ have the same ME model under grounding semantics and $\mathcal{R}$′ is parametrically uniform.
Using the PCI framework providing both grounding and aggregating semantics for conditionals with instantiation restrictions, the ME models for PCIgrounding and PCIaggregation semantics coincide if $\mathcal{R}$ is parametrically uniform.
Proposition 2 ([19]). Let $\mathcal{R}$ be a PCI knowledge base. If $\mathcal{R}$ is parametrically uniform, then ME△($\mathcal{R}$) = ME_{⊛}($\mathcal{R}$).
Thus, while in general ME△($\mathcal{R}$) ≠ ME_{⊛}(R), parametric uniformity of $\mathcal{R}$ ensures that ME△($\mathcal{R}$) = ME_{⊛}($\mathcal{R}$).
5. Computation and Comparison of Maximum Entropy Distributions
In Example 3 we already presented some concrete probability values for ME distributions. We will now look into more details of the ME distributions obtained from both PCIgrounding and PCIaggregation semantics. In particular, we will illustrate how the ME distribution for PCIgrounding and PCIaggregation semantics evolve when transforming a knowledge base that is not parametrically uniform into a knowledge base that is parametrically uniform.
5.1. Achieving Parametric Uniformity
While transforming a knowledge base into one that is parametrically uniform [11] does not change its ME model under (FOPCL or PCI) grounding semantics, it allows for a simpler ME model computation [17]. In [20], a set of transformation rules $\mathcal{PU}$ is presented allowing to transform any consistent knowledge base $\mathcal{R}$ into a parametrically uniform knowledge base $\mathcal{PU}\left(\mathcal{R}\right)$ with the same maximum entropy model under grounding semantics. An implementation of $\mathcal{PU}$ [21] is available within the KReator environment (KReator can be found at http://kreatoride.sourceforge.net/), an integrated development environment for relational probabilistic logic [22]. The CSPU (Conditional Structures and Parametric Uniformity) component [23] of KReator generates $\mathcal{PU}$ transformation protocols, and a part the protocol for the misanthrope knowledge base $\mathcal{R}$_{MI} from Example 3 is shown in Figure 1. For details of the $\mathcal{PU}$ transformation rules we refer to [20]; we just remark here that $\mathcal{PU}$ stepwise removes all interactions among the conditionals where an interaction in a knowledge $\mathcal{R}$ base indicates that $\mathcal{R}$ is not parametrically uniform [20]. In each $\mathcal{PU}$ transformation step, one conditional $\mathcal{R}$ is replaced by two conditionals R_{1}, R_{2} originating from R. Table 1 illustrates how $\mathcal{R}$_{MI} evolves from $\mathcal{R}$_{MI} = $\mathcal{R}$_{1} to $\mathcal{R}$_{2} and from $\mathcal{R}$_{2} to ${\mathcal{R}}_{3}=\mathcal{PU}\left({\mathcal{R}}_{MI}\right)$.
5.2. Maximum Entropy Distributions for Grounding and Aggregation Semantics
Using KREATOR we computed the ME distributions for the three knowledge bases $\mathcal{R}$_{1}, $\mathcal{R}$_{2}, and $\mathcal{R}$_{3} involved in the $\mathcal{PU}$ transformation of $\mathcal{R}$_{MI} for both PCIgrounding and PCIaggregation semantics. For all admissible ground instances of the conditionals occurring in $\mathcal{R}$_{1}, $\mathcal{R}$_{2} and $\mathcal{R}$_{3}, we computed their probability under the ME distributions for PCIgrounding and PCIaggregation semantics. The results are shown in Table 2, using the abbreviation l(x, y) for likes(x, y).
There are three pairwise different ME distributions (i.e., ME_{⊛}($\mathcal{R}$_{1}), ME_{⊛}($\mathcal{R}$_{2}), ME_{⊛}($\mathcal{R}$_{3})) under PCIaggregation semantics for the three pairwise different knowledge bases $\mathcal{R}$_{1}, $\mathcal{R}$_{2}, $\mathcal{R}$_{3}. On the other hand ME_{△}($\mathcal{R}$_{1}) = ME_{△}($\mathcal{R}$_{2}) = ME_{△}($\mathcal{R}$_{3}) = ME_{⊛}($\mathcal{R}$ _{3}) holds since the $\mathcal{PU}$ transformation process does not change the maximum entropy model under PCIgrounding semantics and because $\mathcal{R}$_{3} is parametrically uniform.
It is interesting to note that for the ground instances originating from R_{1} there are two distinct probabilities under ME_{⊛}($\mathcal{R}$_{1}), three probabilities under ME_{⊛}($\mathcal{R}$_{2}), and as implied by Proposition 2 one probability under ME_{⊛}($\mathcal{R}$_{3}). In all cases, PCIaggregation semantics ensures that the distinct probabilities aggregate to the probability stated in the corresponding conditionals.
For the comparison of PCIgrounding and PCIaggregation, it is also interesting to compare their ME behavior with respect to queries that are not instances of a conditional given in the knowledge base. For example, for likes(b, c) we observe
6. Conclusions and Further Work
In this paper, we considered maximum entropy based semantics for relational probabilistic conditionals. FOPCL [16] employs a grounding semantics and uses instantiation restrictions for the free variables occurring in a conditional, requiring all admissible instances of a conditional to have the given probability. Aggregating semantics [15] defines probabilistic satisfaction by interpreting the intended probability of a conditional with free variables only as a guideline for the probabilities of its instances that aggregate to the conditional’s given probability, while the actual probabilities for grounded instances may differ.
While the original definition of aggregation semantics [15] considered only conditionals without constraints representing instantiation restrictions, we developed the framework PCI extending aggregation semantics so that instantiation restrictions can also be taken into account, but without giving up the flexibility of aggregating over distinct probabilities. In comparison with [15], under PCIaggregation semantics one can restrict the set of groundings of a conditional over which aggregating with respect to a conditional takes place by providing a corresponding constraint formula for the conditional. From a knowledge representation point of view, this can be useful in various situations, for instance when we talk about a particular relationship among individuals while already knowing that a specific individual like Clyde is an exception with respect to the given relationship.
Note that PCI captures both grounding semantics and aggregating semantics without instantiation restrictions as special cases. For the case that a knowledge base is parametrically uniform, PCIgrounding and PCIaggregation semantics coincide when employing the maximum entropy principle, while for a knowledge base that is not parametrically uniform the two ME semantics induce different models in general. We illustrated the differences and common features of both semantics on a concrete knowledge base, using the KREATOR environment for computing the ME models and answering queries with respect to these distributions. We expect that observations of this kind will support the discussion of both formal and common sense properties of probabilistic firstorder inference in general and inference according to the principle of maximum entropy in a firstorder setting in particular.
Acknowledgments
We would like to thank the anonymous referees of this paper for their helpful comments and valuable suggestions.
Author Contributions
This paper is joint work by the authors.
Conflicts of Interest
The authors declare no conflict of interest.
References
 Pearl, J. Probabilistic Reasoning in Intelligent Systems; Morgan Kaufmann: San Mateo, CA, USA, 1988. [Google Scholar]
 Cowell, R.; Dawid, A.; Lauritzen, S.; Spiegelhalter, D. Probabilistic Networks and Expert Systems; Springer: New York, NY, USA; Berlin/Heidelberg, Germany, 1999. [Google Scholar]
 Jaynes, E. Papers on Probability, Statistics and Statistical Physics; D. Reidel Publishing Company: Dordrecht, The Netherlands, 1983. [Google Scholar]
 Shore, J.; Johnson, R. Axiomatic derivation of the principle of maximum entropy and the principle of minimum crossentropy. IEEE Trans. Inf. Theory 1980, IT26, 26–37. [Google Scholar]
 Paris, J. The Uncertain Reasoner’s Companion—A Mathematical Perspective; Cambridge University Press: Cambridge, UK, 1994. [Google Scholar]
 Paris, J.; Vencovska, A. In defence of the maximum entropy inference process. Int. J. Approx. Reason. 1997, 17, 77–103. [Google Scholar]
 Paris, J. Common Sense and Maximum Entropy. Synthese 1999, 117, 75–93. [Google Scholar]
 KernIsberner, G. Conditionals in Nonmonotonic Reasoning and Belief Revision; Lecture Notes in Artificial Intelligence LNAI 2087; Springer: Berlin/Heidelberg, Germany, 2001. [Google Scholar]
 Paris, J.B.; What, You; See, Is. What You Get. Entropy 2014, 16, 6186–6194. [Google Scholar]
 Delgrande, J. On firstorder conditional logics. Artif. Intell. 1998, 105, 105–137. [Google Scholar]
 Fisseler, J. Learning and Modeling with Probabilistic Conditional Logic; Dissertations in Artificial Intelligence; IOS Press: Amsterdam, The Netherlands, 2010; Volume 328. [Google Scholar]
 Halpern, J. Reasoning about Uncertainty; MIT Press: Cambridge, MA, USA, 2005. [Google Scholar]
 Getoor, L.; Taskar, B. Introduction to Statistical Relational Learning; Getoor, L., Taskar, B., Eds.; MIT Press: Cambridge, MA, USA, 2007. [Google Scholar]
 KernIsberner, G.; Beierle, C.; Finthammer, M.; Thimm, M. Comparing and Evaluating Approaches to Probabilistic Reasoning: Theory, Implementation, and Applications. In Transactions on LargeScale Data and KnowledgeCentered Systems VI; Hameurlain, A., Küng, J., Wagner, R., Liddle, S.W., Schewe, K.D., Zhou, X., Eds.; Volume 7600, Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2012; pp. 31–75. [Google Scholar]
 KernIsberner, G.; Thimm, M. Novel Semantical Approaches to Relational Probabilistic Conditionals. Proceedings of the Twelfth International Conference on the Principles of Knowledge Representation and Reasoning, KR’2010, Toronto, ON, Canada, 9–13 May 2010; Lin, F., Sattler, U., Truszczynski, M., Eds.; AAAI Press: Menlo Park, CA, USA, 2010; pp. 382–391. [Google Scholar]
 Fisseler, J. Firstorder probabilistic conditional logic and maximum entropy. Log. J. IGPL 2012, 20, 796–830. [Google Scholar]
 Finthammer, M.; Beierle, C. How to Exploit Parametric Uniformity for Maximum Entropy Reasoning in a Relational Probabilistic Logic. In Logics in Artificial Intelligence, Proceedings of 13th European Conference, JELIA 2012, Toulouse, France, 26–28 September 2012; Fariñas del Cerro, L., Herzig, A., Mengin, J., Eds.; Springer: Berlin/Heidelberg, Germany, 2012; 7519, pp. 189–201. [Google Scholar]
 Rödder, W.; Reucher, E.; Kulmann, F. Features of the ExpertSystemShell SPIRIT. Log. J. IGPL 2006, 14, 483–500. [Google Scholar]
 Finthammer, M.; Beierle, C. Instantiation Restrictions for Relational Probabilistic Conditionals. In Scalable Uncertainty Management, Proceedings the 6th International Conference on Scalable Uncertainty Management, Marburg, Germany, 17–19 September 2012; Hüllermeier, E., Link, S., Fober, T., Seeger, B., Eds.; Springer: Berlin/Heidelberg, Germany, 2012; 7520, pp. 598–605. [Google Scholar]
 Beierle, C.; Krämer, A. Achieving Parametric Uniformity for Knowledge Bases in a Relational Probabilistic Conditional Logic with Maximum Entropy Semantics. Ann. Math. Artif. Intell. 2015, 73, 5–45. [Google Scholar]
 Beierle, C.; Höhnerbach, M.; Marto, M. Implementation of a Transformation System for Relational Probabilistic Knowledge Bases Simplifying the Maximum Entropy Model Computation. Proceedings of the TwentySeventh International Florida Artificial Intelligence Research Society Conference (FLAIRS 2014), Pensacola Beach, FL, USA, 21–23 May 2014; AAAI Press: Menlo Park, CA, USA, 2014; pp. 486–489. [Google Scholar]
 Finthammer, M.; Thimm, M. An Integrated Development Environment for Probabilistic Relational Reasoning. Log. J. IGPL 2012, 20, 831–871. [Google Scholar]
 Beierle, C.; Kuche, S.; Finthammer, M.; KernIsberner, G. A SoftwareSystem for the Computation, Visualization, and Comparison of Conditional Structures for Relational Probabilistic Knowledge Bases. Proceedings of the TwentyEighth International Florida Artificial Intelligence Research Society Conference (FLAIRS 2015), Hollywood, FL, USA, 18–20 May 2015; AAAI Press: Menlo Park, CA, USA, 2015; In press. [Google Scholar]
$\mathcal{R}$_{1}  $\mathcal{R}$_{2}  $\mathcal{R}$_{3} 

R_{1·1}: 〈(l(U,a)l(a,U)))[0.9],U ≠ a〉
 
R_{1} : 〈(l(U,V)l(V,U)))[0.9],U ≠ V〉  R_{1·2} : 〈(l(U,V)l(V,U)))[0.9],U ≠ V, V ≠ a〉  R_{1·2·1} : 〈(l(a, V)l(V, a)))[0.9], V ≠ a〉

R_{1·2·2·}: 〈(l(U,V)l(V,U)))[0.9],U ≠ V, V ≠ a, U ≠ a 〉  
R_{2}: 〈(l(a,V)⊤)[0.05], V ≠ a〉 
$\mathcal{R}$_{1}  $\mathcal{R}$_{2}  $\mathcal{R}$_{3}  ground instance g  ME_{⊛}($\mathcal{R}$_{1})(g)  ME_{⊛}($\mathcal{R}$_{2})(g)  ME_{⊛}($\mathcal{R}$_{3})(g) 

R_{1·1}  (l(b, a)l(a, b))  0.96674480  0.90000000  0.90000000  
(l(c, a)l(a, c))  0.96674480  0.90000000  0.90000000  
R_{1}  R_{1·2·1}  (l(a, b)l(b, a))  0.46016768  0.45380549  0.89999999  
(l(a, c)l(c, a))  0.46016768  0.45380549  0.89999999  
R_{1·2}  
R_{1·2·2}  (l(b, c)l(c, b))  0.96674480  0.96860780  0.90000000  
(l(c, b)l(b, c))  0.96674480  0.96860780  0.90000000  
R_{2}  l(a, b)  0.0500000  0.05000000  0.05000000  
l(a, c)  0.0500000  0.05000000  0.05000000 
© 2015 by the authors; licensee MDPI, Basel, Switzerland This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/).