Simplifying Implications with Positive and Negative Attributes: A Logic-Based Approach

Abstract

Concepts and implications are two facets of the knowledge contained within a binary relation between objects and attributes. Simplification logic (SL) has proved to be valuable for the study of attribute implications in a concept lattice, a topic of interest in the more general framework of formal concept analysis (FCA). Specifically, SL has become the kernel of automated methods to remove redundancy or obtain different types of bases of implications. Although originally FCA used only the positive information contained in the dataset, negative information (explicitly stating that an attribute does not hold) has been proposed by several authors, but without an adequate set of equivalence-preserving rules for simplification. In this work, we propose a mixed simplification logic and a method to automatically remove redundancy in implications, which will serve as a foundational standpoint for the automated reasoning methods for this extended framework.

1. Introduction

Since the 1980s, formal concept analysis (FCA) has been a solid framework to analyze data and extract hidden knowledge, comparable to other well-known techniques in terms of cost. Given a binary table, called formal context, FCA can build ontologies similar to AI-based knowledge representation methods [1] but with a solid algebraic structure, in which order theory and logic are the main tools: given a formal context, FCA builds a hierarchical structure of concepts, the so-called concept lattice (indeed a complete lattice), which encompasses all the information in the formal context. Moreover, in the same process, FCA returns sets of implications and/or association rules (well-known in other areas such as data mining, machine learning, and rough set theories) with a rich algebraic framework in which we can also compute closed sets and their minimal generators, pseudointents, different types of bases, etc. This knowledge can reveal interesting patterns and solve significant problems in modern areas as social network analysis [2,3] or recommender systems [4,5].

In the classical framework of FCA, the formal context is a binary relation between the elements of two sets, that is, a relationship between a set of objects and a set of attributes, establishing the properties that each object does satisfy (namely, positive information). Notwithstanding, sometimes the properties that are not satisfied for each object (negative information) are also relevant. For instance, in a table in which objects are birds, the object “ostrich” shall not have the property (attribute) “fly”. That is, for “ostrich”, not only the positive information is relevant (“large”, “heavy”, “fast”); the negative information (“does not fly”) is also relevant.

The management of negative attributes in association rules already appeared in [6,7]. Mining concise sets of association rules (also called bases) is of particular importance in the machine learning community, and, to this end, several works have been proposed that, on top of the classical minimum support–minimum confidence strategy, impose other measures of interestingness [8] or informativeness [9] based on statistical parameters, which help in the pruning of frequent item sets when determining a representative set of association rules.

In our work, we focus on exact association rules (also called implications); hence, we resort to the framework of FCA, where the first occurrences of negative information appear in Missaoui et al. [10,11], in which the authors computed mixed implications (with positive and negative attributes) from a double context formed by the initial context together with its opposite. This last approach can generate a huge number of redundant rules and algorithms executed with increased execution time. Moreover, and more importantly, the relationship between positive and negative information (mixed attributes) is not considered.

In [12], Rodríguez et al. proposed a generalization of classical FCA to consider both positive and negative attributes together with the relationship between them. New derivation operators and a Galois connection was defined to extend the classical framework. In addition, algorithms to compute the mixed concept lattice and mixed implications were proposed; furthermore, an axiomatic system based on the simplification paradigm was proposed to manage mixed implications [13].

In different areas, such as artificial intelligence, database theory, data mining, and machine learning, one of the main problems is the huge degree of redundancy contained in the rules extracted from a dataset. The focus of this work is the study of redundancy elimination in implications with positive and negative attributes. We must recall that several authors have worked to remove redundancy in association rules in data mining. Zaki [14] used the notion of closed itemsets to reduce the set of rules, and also stated “the number of redundant rules is exponential in the length of the longest frequent itemsets”. Other works study the relaxation of the notion of closure and closed itemsets [15] to describe a compact set of association rules which is approximately informative in the sense that the support and confidence of the remaining association rules can be derived from this compact set with high accuracy. Very recent works [16,17] prove that this is still an open problem.

When dealing with implications, our team has worked on this problem, providing the axiomatic system of simplification logic [18], on which it is possible to develop automated methods to remove redundancy in formal concept analysis which constitute the core to obtain bases of implications [19]. However, all these works consider the positive information in the dataset. To the best of our knowledge, no methods have been developed to eliminate redundancy in the rules when considering positive and negative attributes within the rules.

In this paper, we develop an automated logic-based method to remove redundancy in a set of mixed implications, that is, implications relating positive and negative attributes. For this, we propose the notion of a simplified mixed implicational system and an axiomatic system equivalent to that given in [12], but with rules better suited for implementation. The main idea is a set of logical equivalences oriented to detect redundancy and contradictions between positive and negative attributes, and therefore to simplify the set of implications, by removing attributes inside a single implication or even removing the whole implication.

The rest of this work is structured as follows: in Section 2, we present some preliminary notions about FCA and implicational systems with positive and mixed attributes. In Section 3, the idea of a simplified mixed implicational system is presented together with its motivation. In Section 4, we present logical equivalence rules especially suited for implications with mixed attributes, with the purpose of simplifying the set of implications. The algorithm to build simplified systems of implications is presented in Section 5, and a thorough experimental evaluation of the simplification achieved by the proposed algorithm is given in Section 6, as well as a discussion of the obtained results. The conclusions and future research lines are presented in Section 7. For the sake of readability, the proofs of the technical results of this work are collected in Appendix A.

2. Preliminary Notions and Results

Formal concept analysis (FCA) [20,21] is a mathematical theory based on lattice theory which analyses the information given in a formal context, i.e., a relationship between a set of objects and a set of attributes stored in a table. Usually, a formal context is given by a triple $\mathbb{K}=(G,M,I)$, where G is the set with all the objects, M is the set with all the attributes and I is called the incidence and is defined by $I:G\times M\to \{0,1\}$ and $I(g,m)=1$ if the object g has the attribute m and $I(g,m)=0$ otherwise.

FCA extracts the information stored in a formal context by using the derivation operators, namely, the pair of mappings ${(}^{\uparrow }{,}^{\downarrow })$ defined as follows: given $X\subseteq G$, we have that $X^{\uparrow }=\{m\in M\mid I(g,m)=1\mathrm{for}\mathrm{all}g\in X\}$, that is, the attributes shared by all the objects in X, and, given $Y\subseteq M$, we have $Y^{\downarrow }=\{g\in G\mid I(g,m)=1\mathrm{for}\mathrm{all}m\in Y\}$, that is, all the objects that share all the attributes in Y. Using the derivation operators, a formal concept is defined as a pair $(X,Y)$ satisfying $X^{\uparrow }=Y$ and $X=Y^{\downarrow }$; the set of formal concepts can be ordered by inclusion in the first component, and this ordering provides the structure of a complete lattice.

Another type of information provided by a formal context is given by the so-called attribute implications, which are any expression of the form $A\to B$ where A and B are sets of attributes, i.e., $A,B\subseteq M$ for a given a formal context $\mathbb{K}=(G,M,I)$. We are interested in implications that hold in the formal context $\mathbb{K}$: an implication $A\to B$ holds in $\mathbb{K}$ if $A^{\downarrow }\subseteq B^{\downarrow }$, i.e., if all the objects that have all the attributes from A satisfy the attributes from B as well. We denote by ${\mathcal{L}}_M$ the set of all the implications in M.

Simplification logic ($\mathcal{S}$) [18] was introduced as a means to remove redundancies in a given set of implications that hold in a formal context. The axiom system for $\mathcal{S}$ consists of the axiom schema [Ref] and the three inference rules [Frag], [Comp], [Simp] given below for all sets $A,B,C\subseteq M$:

[Ref]

Reflexivity: ${⊢}_{\mathcal{S}}A\to A$.

[Frag]

Fragmentation: (We will follow the usual convention in this research area of omitting the symbol ∪ whenever necessary. For instance, in this rule $BC$ means $B\cup C$.) $A\to BC{⊢}_{\mathcal{S}}A\to B$.

[Comp]

Composition: $A\to B,C\to D{⊢}_{\mathcal{S}}AC\to BD.$

[Simp]

Simplification: $A\to B,C\to D{⊢}_{\mathcal{S}}A(C\setminus B)\to D$.

The notion of inference in $\mathcal{S}$ is defined as usual: let $\varphi$ be an implication in ${\mathcal{L}}_M$ and let $\Sigma$ be a set of implications in ${\mathcal{L}}_M$. We say that ϕ is a syntactic consequence of Σ in $\mathcal{S}$ (and we denote it by $\Sigma {⊢}_{\mathcal{S}}\varphi$, if there exists a sequence of implications ${\varphi }_1,\cdots ,{\varphi }_n$ such that ${\varphi }_n=\varphi$) and, for all ${\varphi }_i$ with $1\le i\le n$ we have either ${\varphi }_i\in \Sigma$ or we can obtain ${\varphi }_i$ by applying one of the rules of $\mathcal{S}$ to the implications in the set $\{{\varphi }_j\mid j<i\}$.

We recall below some derived rules from simplification logic, which we will use in this paper. The proof is straightforward using the definition of derivation. Given a set of attributes M, the following inference rules hold for all $A,B,C,D\in M$:

[GenRef]

Generalized reflexivity: ${⊢}_{\mathcal{S}}A\to C$ if $C\subseteq A$.

[Augm]

Augmentation: $A\to B{⊢}_{\mathcal{S}}AC\to BC$.

It is worth mentioning that some rules of simplification logic are in fact logical equivalences. The main equivalence rules that we have in the $\mathcal{S}$ are the following:

[FragEq]

$\{A\to B\}\equiv \{A\to B\setminus A\}$.

[UnEq]

$\{A\to B,A\to C\}\equiv \{A\to BC\}$.

[GenEq]

$\{A\to B,C\to D\}\equiv \{A\to BD\}$ when $A\subseteq C\subseteq AB$.

[⌀-Eq]

$\{A\to ⌀\}\equiv ⌀$.

[SimpEq]

$\{A\to B,C\to D\}\equiv \{A\to B,C\setminus B\to D\setminus B\}$ when $A\subseteq C\setminus B$.

Thus far, we have used the information explicitly given by the formal context, but have said nothing about pairs satisfying $I(g,m)=0$. The point is that, in principle, $I(g,m)=0$ does not mean that object g does not have the attribute m; it means that we do not have any evidence either in favour or to the contrary. If $I(g,m)=0$, then we affirm that the object g does not have the attribute m. Some authors [11,12,22,23] assume that $I(g,m)=0$ means that the object g does not have the attribute m. To the best of our knowledge, the paper [11] was the first of these approaches which, given $\mathbb{K}=(G,M,I)$, built a new formal context $(\mathbb{K}\mid \overline{\mathbb{K}})=(G,M\cup \overline{M},I^{*})$ where $I^{*}(g,m)=I(g,m)$ for all $m\in M$ and $I^{*}(g,\overline{m})=min(1,1-I(g,m))$; that is, $I^{*}(g,\overline{m})=0$ if $I(g,m)=1$ and $I^{*}(g,\overline{m})=1$ otherwise. In this approach, the attributes in M are called positive and those in $\overline{M}$ are called negative. Note that, with this view, we duplicate the number of columns, and, as a consequence, the method is not the most efficient possible.

In this work, we adopt the approach considered in [12] in which $I(g,m)=0$ means that the object g does not have the attribute m and, instead of duplicating the number of columns, the derivation operators are changed without changing the formal context.

The extended operators considered here are ${}^{\Uparrow }:2^G\to 2^{M\cup \overline{M}}$ and ${}^{\Downarrow }:2^{M\cup \overline{M}}\to 2^G$, and are defined as follows:

$$X^{\Uparrow }=\{m\in M\mid (g,m)\in I\forall g\in X\}\cup \{\overline{m}\in \overline{M}\mid (g,m)\notin I\forall g\in X\}$$

$$Y^{\Downarrow }=\{g\in G\mid (g,m)\in I\forall m\in Y\}\cap \{g\in G\mid (g,m)\notin I\forall \overline{m}\in Y\}$$

When considering both positive and negative attributes, we will say that we are using a mixed context; when working with $\mathbb{K}$ (resp. $\overline{\mathbb{K}}$) with the classical view, we say that we are working with a positive (resp. negative) context.

As in the classical case, we can define when an implication holds in a mixed context in terms of ⇑ and ⇓ as follows: $A\to B$is valid in a mixed context (denoted by $\mathbb{K}\vDash A\to B)$ if and only if $A^{\Downarrow }\subseteq B^{\Downarrow }$.

The axiom system of simplification logic was extended to this new approach with mixed attributes in [12,24].

The new axiomatic system contains one axiom schema [Ref] and four inference rules, [Simp], [Key], [InKey] and [Red]:

[Ref]

Reflexivity: ${⊢}_{\mathcal{S}}A\to A$.

[Simp]

Simplification: $A\to B,C\to D{⊢}_{\mathcal{S}}A(C\setminus B)\to D$.

[Key]

Key: (Following the convention, hereonwards b represents the singleton $\left\{b\right\}$, and $A\overline{b}$ means $A\cup \left\{\overline{b}\right\}$). $A\to b{⊢}_{\mathcal{S}}A\overline{b}\to M\overline{M}$.

[InKey]

Inverse key: $Ab\to M\overline{M}{⊢}_{\mathcal{S}}A\to \overline{b}$.

[Red]

Reduction: $Ab\to C,A\overline{b}\to C{⊢}_{\mathcal{S}}A\to C$.

This new system is a proper extension of that of simplification logic in that rules [Frag] and [Comp] can be derived from the new axioms [12]. Our set of derived rules can be further extended by a version of the ex contradictione quodlibet and the contraposition rule:

[Cont]

Contradiction: $⊢a\overline{a}\to M\overline{M}$.

[Rft]

Reflection: $Aa\to b⊢A\overline{b}\to \overline{a}$.

Notice that [Key] is in fact the converse of [InKey], and they provide an equivalence between the implications $Ab\to M\overline{M}$ and $A\to \overline{b}$, which reflects the fact that whenever the set $A\cup \left\{b\right\}$ is inconsistent, then $A\to \overline{b}$ should hold. Moreover, a version of the well-known cut rule arises as the equivalence between $A\to C$ and the set $\{Ab\to C,A\overline{b}\to C\}$.

3. Simplified Mixed Implicational Systems

In order to formally define the notion of asimplified mixed implicational system, we will use the following notation:

Notation 1.

Hereafter, we will use $\overline{X}:=\{\overline{x}:x\in X\}$ and notice that $\overline{\overline{x}}=x$ for any $x\in M\overline{M}$. Thus, $x\in \overline{X}$ if, and only if, $\overline{x}\in X$. For any $A,B\subseteq M\overline{M}$, we have $\overline{AB}=\overline{A}\overline{B}$ and also $\overline{A\cap B}=\overline{A}\cap \overline{B}$.

Definition 1.

Let M be a set of attributes and Σ be an implicational system with attributes in $M\overline{M}$. Σ is said to be a simplified mixed implicational system (or sm-implicational systemin short) if the following conditions hold for all $A\to B,C\to D\in \Sigma$:

 (i) 

$B\ne ⌀$ and $A\cap B=⌀$;

 (ii) 

$A=C$ implies $B=D$;

 (iii) 

$A⊊C$ implies $C\cap B=⌀=D\cap B$;

 (iv) 

$B\ne M\overline{M}$ and $A\cap \overline{A}=⌀$;

 (v) 

If $x\in A\cap \overline{C}$ and $A\setminus x=C\setminus \overline{x}$, then $D\neg \subseteq B$.

The first three properties are inherited from the definition of a simplified implicational system by [19] and express the idea that the size of the problem cannot be reduced by applying the equivalence rules for positive attributes. We comment below on the ideas underlying conditions (iv) and (v), which are specific to the case of mixed attributes.

For (iv), the condition $B\ne M\overline{M}$ refers to the fact that, on the one hand, $⌀\to M\overline{M}$ is not admissible and, on the other hand, if $A\ne ⌀$, we would have that $A\to M\overline{M}$ is equivalent to $A\setminus x\to \overline{x}$ for any $x\in A$ (hence, the system would be reduced by considering the latter). Furthermore, by the derived rule [Cont], any implication of the form $A\to B$ with a contradiction in the antecedent ($A\cap \overline{A}\ne ⌀$) is valid; therefore, it can be safely removed from the implicational system.

Finally, (v) expresses syntactically a situation in which rule [Red] would apply. If $A\setminus x=C\setminus \overline{x}$, there exists a set of attributes S such that $A=Sx$ and $C=S\overline{x}$. In addition, if $D\subseteq B$, we would have

$$\begin{array}{cc}\hfill \{A\to B,C\to D\}& =\{Sx\to B,S\overline{x}\to D\}\hfill \\ \hfill & \equiv \{Sx\to B\setminus D,Sx\to D,S\overline{x}\to D\}\hfill & \hfill \left[\mathrm{FragEq}\right]\\ \hfill & \equiv \{Sx\to B\setminus D,S\to D\}\hfill & \hfill \left[\mathrm{Red}\right];\end{array}$$

hence, the size of the system can be reduced.

The next theorem provides a set of equivalence rules oriented to obtaining an automated method to build an sm-implicational system from a given one. For the sake of readability, the proofs of all the technical results are collected in Appendix A.

Theorem 1.

Let $\mathbb{K}=(G,M,I)$ be a formal context and $A,B,C,D\subseteq M\overline{M}$. Then, the following equivalences hold:

[KeyEq]

If $A\subseteq C$ and $B\cap \overline{C}\ne ⌀$, then

$$\{A\to B,C\to D\}\equiv \{A\to B\}.$$

[InKeyEq]

If there exists $x\in A\setminus C$, such that $A\setminus x\subseteq C$, then

$$\{A\to M\overline{M},C\to D\}\equiv \{A\to M\overline{M},C\to D\overline{x}\}.$$

[RedEq]

If $D\subseteq B$ and there exists $x\in A\cap \overline{C}$, with $A\setminus x\subseteq C\setminus \overline{x}$, then

$$\{A\to B,C\to D\}\equiv \{A\to B,C\setminus \overline{x}\to D\}.$$

By using the previous theorem, we propose the following inference rules, which are more convenient to be implemented algorithmically.

[Ref]

Reflexivity: $⊢A\to A$.

[Simp]

Simplification: $A\to B,C\to D⊢A(C\setminus B)\to D$.

[Key${}^{\prime }$]

Key: $A\to B,C\to D⊢C\to M\overline{M}$ if $A\subseteq C$ and $B\cap \overline{C}\ne ⌀$.

[InKey${}^{\prime }$]

Inverse key: $A\to M\overline{M},C\to D⊢C\to D\overline{x}$ if there exists $x\in A\setminus C$ and $A\setminus x\subseteq C$.

[Red${}^{\prime }$]

Reduction: $A\to B,C\to D⊢C\setminus \overline{x}\to D$ if $D\subseteq B$ and there exists $x\in A\cap \overline{C}$ with $A\setminus x\subseteq C\setminus \overline{x}$.

Of course, this new axiomatic system is equivalent to the initial one, as shown in the theorem below.

Theorem 2.

The system formed by rules[Ref],[Simp],[Key${}^{\prime }$],[InKey${}^{\prime }$]and[Red${}^{\prime }$]is equivalent to that formed by[Ref],[Simp],[Key],[InKey]and[Red].

4. Simplification via Equivalence Rules

In this section, we introduce some other equivalence rules that will be useful for removing redundant attributes. The starting point will use the following result:

Lemma 1.

For all $A,B,C,D\subseteq M\overline{M}$, we have:

[ContEq]

If $A\cap \overline{A}\ne ⌀$, then $\{A\to B\}\equiv ⌀$.

[ContEq${}^{\prime }$]

If $A\ne ⌀$ and $AB\cap \overline{B}\ne ⌀$, then $\{A\to B\}\equiv \{A\to M\overline{M}\}\equiv \{A\setminus x\to \overline{x}\}$ for any $x\in A$.

[ContEq${}^{″}$]

If $C\ne ⌀$, $A\subseteq CD$ and $B\cap \overline{CD}\ne ⌀$, then, for any $x\in C$

$$\{A\to B,C\to D\}\equiv \{A\to B,C\to M\overline{M}\}\equiv \{A\to B,C\setminus x\to \overline{x}\}.$$

The equivalences above allow us to detect contradictions and, hence, reduce the size of the set of implications. Below, we propose some other equivalence rules that take into account the possible relationship between different implications in order to reduce their size.

Theorem 3.

Consider $A,B,C,D\subseteq M\overline{M}$:

[KeyEq${}^{\prime }$]

If there exist $x\in A\cap D$, $y\in B\cap \overline{C}$ with $A\setminus x=C\setminus \overline{y}$, then

$$\{A\to B,C\to D\}\equiv \{A\to B\setminus y,C\setminus \overline{y}\to y\}\equiv \{A\to B\setminus y,C\to M\overline{M}\}.$$

[KeyEq${}^{″}$]

If $A\subseteq C\ne ⌀$ and $B\cap \overline{D}\ne ⌀$, for any $x\in C$ we have that then

$$\{A\to B,C\to D\}\equiv \{A\to B,C\setminus x\to \overline{x}\}.$$

[RedEq${}^{\prime }$]

If $D\subseteq B$ and there exists $x\in A\cap \overline{C}$ such that $A\setminus x=C\setminus \overline{x}$, then

$$\{A\to B,C\to D\}\equiv \{A\to B\setminus D,C\setminus \overline{x}\to D\}.$$

[RftEq]

If there exist $x\in A$, $y\in B\cap \overline{C}$ and $A\setminus x=C\setminus \overline{y}$, then

$$\{A\to B,C\to D\}\equiv \{A\to B\setminus y,C\to D\overline{x}\}.$$

[RftEq${}^{\prime }$]

If there exist $x\in A\cap \overline{D}$, $y\in B\cap \overline{C}$ and $A\setminus x\subseteq C\setminus \overline{y}$, then

$$\{A\to B,C\to D\}\equiv \{A\to B,C\to D\setminus \overline{x}\}.$$

[MixUnEq]

If there exist $x\in A$, $y\in C$ such that $A\setminus x=C\setminus y$ and $b\in D$, then

$$\{A\to b,C\to D\}\equiv \{(A\setminus x)\overline{b}\to \overline{x}\overline{y},C\to D\setminus b\}.$$

Among all the rules above, there are two which, in principle, do not help to reduce the size of the system([RftEq] and [MixUnEq]), but guarantee to keep the number of attributes fixed. However, there is a particular case where the size is reduced as a result of removing implications: the application of either [RftEq] or [MixUnUnitEq] when one of the implications is a unit implication (its right-hand side has only one element) allows to remove that implication. The result is formally stated in the following corollary:

Corollary 1.

For all $A,C,D\subseteq M\overline{M}$ and $b,y\in M\overline{M}$, the following equivalences hold:

[RftUnitEq]

If $y\in \overline{C}$ and there exist $x\in A$ such that $A\setminus x=C\setminus \overline{y}$, then

$$\{A\to y,C\to D\}\equiv \{C\to D\overline{x}\}.$$

[MixUnUnitEq]

If there exist $x\in A$, $y\in C$ such that $A\setminus x=C\setminus y$, then

$$\{A\to b,C\to b\}\equiv \{(A\setminus x)\overline{b}\to \overline{x}\overline{y}\}.$$

5. Automatic Computation of sm-Implicational Systems

In this section, we propose an algorithm to simplify a system of implications with positive and negative attributes. The algorithm is based on the equivalence rules obtained in Section 3 and Section 4. For the sake of readability, the main algorithm is decomposed into subroutines that check the conditions required to apply certain equivalence rules.

Following the same strategy as in Section 4, we start by defining the algorithms to simplify the implications that contain contradictions. In Algorithm 1, we translate [ContEq${}^{\prime }$] into pseudo-code, whereas in Algorithm 2 we do the same with [ContEq${}^{″}$].

In Algorithm 2 below, we work with two implications, $A\to B$ and $C\to D$, and check the conditions of [ContEq${}^{″}$] in both orders by changing the roles of $A\to B$ and $C\to D$ in the equivalence rule, so we can simplify both implications in a single execution of Simplify-Cont2. We will make use of this strategy in the rest of the algorithms.

The equivalence rules [KeyEq] and [KeyEq${}^{\prime }$] are presented in Algorithm 3. Observe that the conditions of those equivalence rules are nested since they had some common requirements so the algorithm can be written more compactly. Additionally, notice that when an implication $X\to Y$ can be removed from the implicational system due to the use of an equivalence rule, in the algorithms, it is indicated as $X,Y:=⌀$.

Algorithm 4 presents the pseudocode for [KeyEq${}^{″}$]. It has not been incorporated into the same algorithm as [KeyEq] and [KeyEq${}^{\prime }$] for the sake of readability. The equivalence rules derived from the inference rule [Red]are condensed in Algorithm 5. Then, in Algorithm 6, we present the equivalence rules [RftEq${}^{\prime }$] and [RftUnitEq].

Algorithm 7 presents the code for the equivalence rule [MixUnUnitEq]. It is a simple translation of the conditions into pseudocode. Note that, in this case, there is no need to include the converse conditions due to the equality required.

The complete method is presented in Algorithm 8, which incorporates the other algorithms in order to simplify the implications with all the studied equivalence rules. We can explain its procedure as follows: firstly, the set ${\Sigma }^{\prime }$ of proper implications (i.e., the antecedent and consequent are disjoint) without contradictions is built from $\Sigma$. We initialize ${\Sigma }_s:={\Sigma }^{\prime }$ and reset ${\Sigma }^{\prime }$. We will use ${\Sigma }^{\prime }$ as a list to store the simplified implications in each iteration.

For each implication $A\to B\in {\Sigma }_s$, we try to simplify it with the other implications already stored in ${\Sigma }^{\prime }$. If the result after simplification is a proper implication (nonempty consequent), it is added to ${\Sigma }^{\prime }$. By this procedure, all implications are compared with each other, and both $A\to B\in {\Sigma }_s$ and all $C\to D\in {\Sigma }^{\prime }$ are simplified in a single step. The algorithm checks whether the implications have any contradiction; in that case, rules [ContEq] and [ContEq${}^{\prime }$] are used (lines 5 and 8). Lines 9–10 express the conditions to use [GenEq], that is, where $A\to B$ is more general than $C\to D$ or vice versa, keeping only the more general. If it is not the case, then the algorithm proceeds to check the conditions of the simplification rules described in Section 3 and Section 4. The first simplification to be considered is [SimpEq] (lines 12–15), since it allows us to remove attributes from both the left-hand and right-hand sides of the implications, whenever applicable. Later, the algorithm checks all the simplifications that are specific for mixed attributes and that have already been described in Algorithms 2 to 7. Line 22 is used to add the implications $C\to D$ that have not been removed after all the simplification steps. Lines 23–26 add the implication $A\to B$ (if it has not been removed) to ${\Sigma }^{\prime }$.

The theorem below proves the termination and correctness of Algorithm 8 in the sense that the ouput is an implicational system equivalent to the input and, in addition, is an sm-implicational system. The notion of size and the following notations will be useful for the proofs of the results hereafter.

The size of an implicational system $\Sigma$ is defined as $\parallel \Sigma \parallel ={\sum }_{A\to B\in \Sigma }\left(\left|A\right|+\left|B\right|\right)$, where $\left|X\right|$ represents the cardinality of the set X. Thus, $\parallel \Sigma \parallel$ is the amount of attributes that appear in the antecedents and consequents in the implications in $\Sigma$.

Theorem 4.

The functionSimplify-Mixed given in Algorithm 8 reaches a fixed point ${\Sigma }^{\prime }$ given any finite implicational system Σ. In addition, ${\Sigma }^{\prime }\equiv \Sigma$ and ${\Sigma }^{\prime }$ is an sm-implicational system.

Proof. 

First, note that, after the comparison of a pair of implications (lines 7–22), the size of $\{A\to B,C\to D\}$ does not increase, since there are only three possibilities: (1) no simplification is made; (2) one or more attributes are removed from the left-hand side or the right-hand side of any of the two implications; and (3) one implication is removed at the cost of adding, at most, one attribute to the other implication (this happens in [RftUnitEq] and [MixUnUnitEq]). Since the minimum size of a proper implication is one, in all these three cases, we can guarantee that the size of the system does not increase.

Moreover, if no simplification is made in an iteration over all implications $A\to B\in {\Sigma }_s$, i.e., the system is not modified in that iteration, then the algorithm stops. Thus, in an iteration of the repeat loop in lines 2–27, the only two possibilities are (1) the system is reduced by removing implications or by removing attributes, and (2) the system is not modified in the iteration. There cannot be infinitely many iterations since the size of the system is finite and decreases in each iteration. Hence, the algorithm will stop after finitely many steps.

Let $\Sigma$ be an implicational system and let ${\Sigma }^{\prime }$ be the output of Algorithm 8 on $\Sigma$. Since ${\Sigma }^{\prime }$ is built iteratively by applying equivalence rules, it is obvious that ${\Sigma }^{\prime }\equiv \Sigma$. Let us now prove that ${\Sigma }^{\prime }$ is an sm-implicational system. Assume $A\to B,C\to D\in {\Sigma }^{\prime }$:

 (i) 

$B\ne ⌀$ and $A\cap B=⌀$ follow from the construction of ${\Sigma }^{\prime }$ in lines 1 and 23–26;

 (ii) 

Let us prove that in ${\Sigma }^{\prime }$ there cannot be two different implications $A\to B$ and $C\to D$ with $A=C$. At a given iteration of the algorithm, if two implications verify that their left-hand sides are equal, they would meet the conditions in line 9, so they will be merged into a single implication, and, therefore, the algorithm would not reach a fixed point at that iteration. Hence, in the fixed point ${\Sigma }^{\prime }$, there cannot be such duplicity since the implications would have been previously merged;

 (iii) 

$A⊊C$ implies $C\cap B=⌀=D\cap B$: it is a consequence of the application of [SimpEq]in lines 12–15;

 (iv) 

$B\ne M\overline{M}$ and $A\cap \overline{A}=⌀$ due to rules [ContEq] and [ContEq${}^{\prime }$];

 (v) 

If $x\in A\cap \overline{C}$ and $A\setminus x=C\setminus \overline{x}$, then $D\neg \subseteq B$. Otherwise, [RedEq] could be applied, and therefore, ${\Sigma }^{\prime }$ would not be the fixed point.

Hence, ${\Sigma }^{\prime }$ is an sm-implicational system.    □

Note that the only equivalence rules for mixed attributes needed to obtain an sm-implicational system are [ContEq], [ContEq${}^{\prime }$] and [RedEq]. The purpose of the rest of equivalence rules is to further minimize and get an even simpler implicational system.

Let us now show a brief example of the application of Algorithm 8, omitting minor details for an easier reading.

Example 1.

Given $\Sigma =\{ad\overline{b}\to c,\overline{b}\overline{c}\to ad,ac\overline{b}\to d,a\overline{c}\to bd\}$, the application ofSimplify-Mixed$(\Sigma )$ is as follows:

• Firstly, the pair $\{ad\overline{b}\to c,\overline{b}\overline{c}\to ad\}$ is studied. Using[ContEq${}^{\prime \prime }$], we simplify $\overline{b}\overline{c}\to ad$ into $\overline{c}\to b$. Thus,

  $$\{ad\overline{b}\to c,\overline{b}\overline{c}\to ad,ac\overline{b}\to d,a\overline{c}\to bd\}\equiv \{ad\overline{b}\to c,\overline{c}\to b,ac\overline{b}\to d,a\overline{c}\to bd\};$$
• Later, $\{ad\overline{b}\to c,a\overline{c}\to bd\}$ satisfies the conditions of[RftUnitEq], so $ad\overline{b}\to c$ is removed.

  $$\{ad\overline{b}\to c,\overline{c}\to b,ac\overline{b}\to d,a\overline{c}\to bd\}\equiv \{\overline{c}\to b,ac\overline{b}\to d,a\overline{c}\to bd\};$$
• For $\{\overline{c}\to b,a\overline{c}\to bd\}$,[SimpEq]can be applied, changing $a\overline{c}\to bd$ into $a\overline{c}\to d$:

  $$\{\overline{c}\to b,ac\overline{b}\to d,a\overline{c}\to bd\}\equiv \{\overline{c}\to b,ac\overline{b}\to d,a\overline{c}\to d\};$$
• When comparing $\{ac\overline{b}\to d,a\overline{c}\to d\}$,[RedEq]can be applied, transforming $ac\overline{b}\to d$ into $a\overline{b}\to d$:

  $$\{\overline{c}\to b,ac\overline{b}\to d,a\overline{c}\to d\}\equiv \{\overline{c}\to b,a\overline{b}\to d,a\overline{c}\to d\};$$
• Now, for $\{a\overline{b}\to d,a\overline{c}\to d\}$,[MixUnUnitEq]can also be applied, changing $a\overline{c}\to d$ to $a\overline{d}\to bc$ and removing $a\overline{b}\to d$:

  $$\{\overline{c}\to b,a\overline{b}\to d,a\overline{c}\to d\}\equiv \{\overline{c}\to b,a\overline{d}\to bc\};$$
• At this point, the algorithm has reached a fixed point and returns ${\Sigma }^{\prime }=\{\overline{c}\to b,a\overline{d}\to bc\}$.

We can check that the size of Σ is 16, and the size of the sm-implicational system obtained by Algorithm 8 is 6; that is, we have obtained a size reduction of ≈62.5%.

This example shows that the algorithm can be highly effective in reducing the size of a system with mixed attributes. In the next section, we will present a thorough experimental evaluation of the size reduction achieved. We conclude this section by presenting a worst-case complexity analysis of the algorithm, showing that it has polynomial complexity.

Theorem 5.

Let Σ be an implicational system with mixed attributes. The worst-case time complexity of Algorithm 8 on Σ is $\mathcal{O}\left(\right|\Sigma {|}^2\parallel \Sigma \parallel )$.

Proof. 

In every iteration of the algorithm, we remove at least one attribute or one implication. Therefore, the maximum number of iterations of the repeat loop is $\parallel \Sigma \parallel =max\left\{\right|\Sigma |,\parallel \Sigma \parallel \}$, since $\parallel \Sigma \parallel \ge |\Sigma |$, for we are considering only proper implications. Furthermore, in each iteration, all possible pairs of implications are studied in order to apply the equivalence rules, so a maximum of $\mathcal{O}\left(\right|\Sigma {|}^2)$ steps are made in every iteration. In aggregate, the maximum number of steps is $\mathcal{O}\left(\right|\Sigma {|}^2\parallel \Sigma \parallel )$.    □

It is worth noting that, as $\parallel \Sigma \parallel \ge |\Sigma |$, the complexity of Algorithm 8 is polynomial, $\mathcal{O}(\parallel \Sigma {\parallel }^3)$, in the size of the input.

As a related issue, it is important to note that the complexity of the construction of the canonical basis of implications, as well as of the enumeration of the rules belonging to this basis, has been studied before by other authors [25,26,27]. However, such a problem is different from the one we study in the present work, where we consider a previously computed system of implications as the input for our method, which, with polynomial complexity, returns an equivalent simplified system. Moreover, the input system in our proposal does not have to represent a basis associated with a given formal context.

6. Experimental Results

In order to evaluate the capability of Algorithm 8 for simplifying an implicational system, a number of random mixed contexts have been generated with different numbers of attributes in M and different densities (proportion of non-zero elements in the table of the relation I), and their Duquenne–Guigues bases of implications [28] has been computed. As these bases are built without using the logic of mixed attributes, it makes sense to use them as a benchmark for the proposed algorithm.

Contexts were constructed for $\left|M\right|\in \{4,5,\dots ,10\}$ (that is, $|M\overline{M}|\in \{8,10,\cdots ,20\}$) and density $\delta \in \{0.1,0.25\}$. This choice of values for $\delta$ is due to two main reasons: on the one hand, (purely positive) formal contexts in real situations are very sparse, with a very low proportion of non-zero elements in the table of the relation I; on the other hand, the apposition of a context $\mathbb{K}$ and its associated negative context, $\overline{\mathbb{K}}$, by construction, always has density 0.5. Therefore, the choice of $\delta$ has no relevance to the complexity of the problem since, as we have seen in Theorem 5, the main indicators of the complexity of the problem are the size and the cardinality of the implication basis. Therefore, particular values of $\delta$ typically representative of low densities have been selected. For each combination of these parameters, 50 contexts have been randomly generated. All experiments have been performed using the R programming language and the library fcaR [29], particularly for the generation of the datasets as well as for the computation of the implication bases.

As stated in Section 5, in order to obtain an sm-implicational system, it is enough to consider [ContEq], [ContEq${}^{\prime }$]and [RedEq]. For this reason, in the experiment, we have compared two different versions of the algorithm: (v1) where only the simplifications related to the positive attributes, [GenEq]and [SimpEq], as well as the aforementioned [ContEq], [ContEq${}^{\prime }$] and [RedEq], are performed; (v2) the simplification algorithm as described in Algorithm 8. Thus, (v1) is obtained by removing lines 6 and 10 from Algorithm 5 and lines 17–21 from Algorithm 8. The idea behind this comparison is to check the improvement obtained by adding the rest of the equivalence rules.

Hence, for each experiment carried out, the size and the cardinal of the sm-implicational system returned by the simplification algorithm have been measured, both for version (v1) and (v2). The number of iterations performed by each of the two versions before reaching the fixed point was also taken into account, as well as the number of simplifications performed and the calculation time.

In

Table 1. Data from the experimental evaluation. Columns 3 to 10 express the average value obtained in the experiments for each of the studied parameters (cardinality, size and execution time).

				(v1)			(v2)
$\left|M\right|$	$\delta$	$|\Sigma |$	$\parallel \Sigma \parallel$	$|{\Sigma }^{\prime }|$	$\parallel {\Sigma }^{\prime }\parallel$	t	$|{\Sigma }^{\prime }|$	$\parallel {\Sigma }^{\prime }\parallel$	t
4	0.1	8.34	45.68	4.40	11.66	0.105	2.68	6.64	0.089
	0.25	11.48	55.92	7.52	21.44	0.261	2.96	8.94	0.121
5	0.1	11.50	75.92	6.68	21.86	0.186	3.92	12.14	0.167
	0.25	22.86	120.38	17.90	57.02	1.694	6.16	21.08	0.852
6	0.1	15.60	117.22	9.78	34.78	0.435	5.66	19.16	0.335
	0.25	38.48	220.18	32.50	111.58	6.299	10.78	40.94	2.771
7	0.1	20.40	171.04	13.64	51.88	0.956	7.36	26.24	0.563
	0.25	61.96	379.92	54.96	203.40	20.356	18.68	75.16	9.439
8	0.1	28.04	254.36	20.32	78.56	2.388	10.04	37.56	1.642
	0.25	102.84	652.44	94.92	368.56	67.209	33.68	146.44	34.22
9	0.1	35.80	351.10	27.20	108.80	3.319	12.90	51.20	2.596
	0.25	149.10	992.20	140.10	571.40	144.114	50.50	232.00	74.970
10	0.1	46.21	486.50	36.40	149.90	7.921	17.60	70.10	5.220
	0.25	218.92	1494.20	208.90	891.10	393.850	81.40	393.10	188.461

, we present the averaged results of the execution of the two versions on the problems generated according to their number of attributes and density. As expected, whereas the reduction produced by the algorithm version (v1) to obtain sm-implicational systems ranges from 19% to 74% with respect to the system size, and 2% to 47% with respect tocardinality, version (v2) provides much more reduced outputs, with a range of reduction between 73% and 85% with respect size, and between 61% to 79% with respect to cardinality. Furthermore, it is worth mentioning that, despite the fact that (v2) performs a greater number of checks, the computation time is, in many occasions, smaller than that of (v1); the reason is that in (v2), it is possible to perform many more simplifications in each iteration. This shows that the extra cost of applying the extra equivalence rules pays off, since we obtain a notably smaller version of the implicational system with shorter computation times.

The study of the relationship between the size and cardinality of the input system and the size and cardinality of the simplified systems by versions (v1) and (v2) provides the following results: in Figure 1, we can see the practically linear relationship in the reduction of both factors. Again, we observe that (v2) presents a lower slope, corroborating the trend indicated in Table 1. A linear regression has been performed on the data, obtaining

(v1)

$\parallel {\Sigma }^{\prime }\parallel \approx 0.629\parallel \Sigma \parallel$ (with $R^2=0.956$), and $|{\Sigma }^{\prime }|\approx 0.975|\Sigma |$ (with $R^2=0.9992$).

(v2)

$\parallel {\Sigma }^{\prime }\parallel \approx 0.249\parallel \Sigma \parallel$ (with $R^2=0.967$), and $|{\Sigma }^{\prime }|\approx 0.331|\Sigma |$ (with $R^2=0.961$).

Finally, in Figure 2, we show the execution time of the algorithm for each of the problems, depending on the size of the input. On the scatterplot, the cubic fit to the data has been plotted using $\parallel \Sigma \parallel$ as the predictor. It can be seen how the theoretical polynomial fit mentioned in Theorem 5 is also reflected in the experimental evaluation.

Discussion

The topic of attribute reduction and simplification of implicational systems is important insofar as it can be considered a preliminary step to other algorithms (computation of logical closure, generation of direct bases) whose computational complexity depends directly on the size of the implicational system; however, to the best of our knowledge, there is no previous algorithm for simplifying implications with mixed attributes and, thus, our proposal is the first one in this extended framework.

We have presented experimental evidence of the algorithm’s ability to reduce the size of an implicational system with mixed attributes. Specifically, we have presented two versions of the algorithm: the first one (v1) simply guarantees the obtainment of an sm-implicational system, and the most complete one (v2) obtains more simplified results, achieving an average reduction of about 75% of the initial system size (see Figure 1), even with a lower execution time (see Figure 2).

7. Conclusions and Future Work

We have proposed a novel logic-based method able to face the problem of simplifying implicational systems consisting of positive and negative attributes. This method is based on the simplification logic for mixed attributes, which provides a sound and complete framework to deal with this type of formal context. Among the contributions of this paper, we have found a set of logical equivalence rules for mixed attribute implications suitable for computational implementation and an algorithm to simplify an implicational system with a polynomial time-complexity. The experimental results show very promising results with respect to the reduction ratio of the algorithm.

Concerning future work, a thorough study of the minimality properties of sm-implicational systems is needed in order to pursue definitions similar to the different types of bases for classical implicational systems [19]. This will certainly be a first step on which to build automated reasoning methods based on simplification logic.

Another interesting direction for future work is the extension of these results with unknown information, which may be due to the existence of partial information in the data, or because the context has been obtained by compacting the data, as proposed in [30]. In this line, we will follow the research line initiated in [31] and consider the formal contexts as trivalued relations in which a third value representing the unknown is added apart from the standard positive and negative values.

Extending the work presented here to the case of association rules will require adaptation of the axiomatic system and equivalence properties in order to ensure the correctness of the scheme and the preservation of the informativeness and completeness properties of a basis of rules, and this task will be undertaken in future work.

Finally, in this work, we have focused on rules with conjunctive antecedents and consequents, where both positive and negative attributes appear. However, this is not the only point of view through which to model the problem of negation. In particular, the use of disjunctive rules, such as in [32], is of particular interest because of their generality. As part of our future work, we will explore the relationship of the axiomatic system and the equivalence rules described under this construction.