Classification with Fuzzification Optimization Combining Fuzzy Information Systems and Type-2 Fuzzy Inference

Abstract

In this research, we introduce a classification procedure based on rule induction and fuzzy reasoning. The classifier generalizes attribute information to handle uncertainty, which often occurs in real data. To induce fuzzy rules, we define the corresponding fuzzy information system. A transformation of the derived rules into interval type-2 fuzzy rules is provided as well. The fuzzification applied is optimized with respect to the footprint of uncertainty of the corresponding type-2 fuzzy sets. The classification process is related to a Mamdani type fuzzy inference. The method proposed was evaluated by the F-score measure on benchmark data.

1. Introduction

Nowadays, machine learning and its applications are growing rapidly. Classification is one of the major machine learning problems. As there are various kinds of data, the need for robust techniques has become essential, especially in real-world applications [1,2]. What is more, often the data are not well-defined, and are vague or imbalanced, which is an additional obstacle. Therefore, fuzzy techniques became the right solution. We may divide these techniques into two primary groups: those related to the classical type-1 fuzzy sets, introduced by Zadeh [3] and those related to new concepts concerning type-2 fuzzy sets [4,5]. In practice, the interval type-2 fuzzy sets [6], a particular case of type-2 fuzzy sets, are commonly used for their reduced computational cost and easy implementation. What is more, researchers have proved that interval type-2 fuzzy concepts are better able to handle uncertainties than type-1 fuzzy approaches [6,7,8,9,10,11]. Interval type-2 fuzzy sets are very effective, as they provide better generalisations when it is difficult to determine the exact membership functions of the fuzzy sets applied.

On the other hand, to deal with information issues, data discovery techniques, assumed as the computational process of discovering patterns in data, are widely used to induce knowledge. Information systems and rough sets, introduced by Pawlak [12,13,14], are applied to represent knowledge. Therefore, it seems that the combination of fuzzy techniques and data discovery is very appropriate for the analysis of complex data such as medical data [15,16].

That is how the concept of fuzzy information systems came out. If each attribute of an information system is related to a fuzzy set, a fuzzy information system is defined [17]. Fuzzy information systems find their applications in the field of decision-making [18,19]. In recent research, a novel multi-criteria approach with application in investments was proposed [20]. Many other applications also can be found concerning rule extraction [19,21], feature selection and classification [22,23,24,25]. Mathematical properties of fuzzy information systems are under consideration as well [17,26]. The concept of rough sets was extended by analyzing the properties of lower and upper approximations of fuzzy sets [27,28,29,30,31] with corresponding practicable applications. For example, in the significant field of machine learning [32] or even medical image analyses, as an image segmentation approach [33].

In our research, we introduce a fuzzy information system which induces type-1 fuzzy rules and transforms them into interval type-2 fuzzy rules. We assume the information function values to be fuzzy sets, but still applying the original definitions of indistinguishability relation and rough sets defined by Pawlak. Our research experiments consist of investigating different fuzzification procedures, which determine the optimal fuzzy classifier.

The rest of the paper is organized as follows: in Section 2, the benchmark data used and the classification methodology proposed are explained, in Section 3 the classification evaluation is presented, and finally, Section 4 and Section 5 draw discussion and conclusions.

2. Materials and Methods

2.1. Materials

In our research, we have applied our classification proposal on well-known benchmark data [34] for binary classification. The benchmark details are given in

Table 1. The data sets used.

Benchmark	Number of Attributes	Number of Samples	Class Proportions
Blood Transfusion [35]	7	748	76%/24%
Breast Cancer Data	4	569	63%/37%
Breast Cancer Wisconsin [36]	9	683	65%/35%
Data Banknote Authentication	4	1372	56%/44%
Haberman	3	305	73%/27%
Heart	13	303	46%/54%
HTRU 2 [37]	8	17898	9%/91%
Immunotherapy [38]	7	90	79%/21%
Indian Liver Patient	10	583	71%/29%
Ionosphere	34	351	64%/36%
Parkinson [39]	22	187	74%/26%
Pima Indians Diabetes	8	768	65%/35%
Vertebral	6	310	68%/32%

.

2.2. Methods

This section explains the preliminaries of the type-2 Mamdani fuzzy model [4,40] and information systems rule induction procedure [41,42]. Next, we introduce our proposal of combination information systems with type-2 fuzzy inference to introduce a classification procedure.

2.2.1. Fuzzy Sets and Interval Type-2 Fuzzy Sets

A fuzzy set or type-1 fuzzy set F consists of a domain X of real numbers together with a function ${\mathsf{\mu }}_{\mathrm{F}}$: X → [0, 1], [3] i.e.,:

$${\mathrm{F}=}_{\mathrm{df}}{{\displaystyle \int }}_{\mathrm{X}}^{}{\mathsf{\mu }}_{\mathrm{F}}(\mathrm{x})/\mathrm{x},\mathrm{x}\in \mathrm{X}$$

(1)

Here, the integral denotes the collection of all points x ∈ X with associated membership grade ${\mathsf{\mu }}_{\mathrm{F}}$ (x) ∈ [0, 1]. The function ${\mathsf{\mu }}_{\mathrm{F}}$ is also known as the membership function of the fuzzy set F, as its value represents the grade of membership of the elements of X to the fuzzy set F. The idea is to use membership functions to describe imprecise or vague information.

By expanding this concept with the assumption that membership function values can be fuzzified themselves, the idea of type-2 fuzzy sets was introduced. A type-2 fuzzy set, denoted as $\tilde{\mathrm{F}}$, is defined as follows [4]:

$${\tilde{\mathrm{F}}=}_{\mathrm{df}}{{\displaystyle \int }}_{\mathrm{x}\in \mathrm{X}}^{}{{\displaystyle \int }}_{\mathrm{u}\in {\mathrm{J}}_{\mathrm{x}}}^{}{\mathsf{\mu }}_{\tilde{\mathrm{F}}}{(\mathrm{x},\mathrm{u})/(\mathrm{x},\mathrm{u}),\mathrm{J}}_{\mathrm{x}}\subseteq [0,1]$$

(2)

where $\iint$ denotes union over all admissible x and u. Interval type-2 (IT2) fuzzy sets [4], a special case of type-2 fuzzy sets, are the most widely used because of their acceptable computational complexity and easy interpretation. Uncertainty about $\tilde{\mathrm{F}}$ is conveyed by the so-called footprint of uncertainty (FOU) of $\tilde{\mathrm{F}}$:

$$\mathrm{FOU}(\tilde{\mathrm{F}}){=}_{\mathrm{df}}\underset{\mathrm{x}\in \mathrm{X}}{\overset{}{{\displaystyle \cup }}}{\mathrm{J}}_{\mathrm{x}}{,\mathrm{J}}_{\mathrm{x}}\subseteq [0,1]$$

(3)

The size of an FOU (the corresponding surface) is directly related to the uncertainty that is conveyed by an interval type-2 fuzzy set, and what follows, an FOU with more area is more uncertain than the one with less area. The upper membership function and lower membership function of $\tilde{\mathrm{F}}$ are two type-1 membership functions $\underset{\_}{\mathrm{F}}$ and $\overline{\mathrm{F}}$ that bound the FOU, which might be used to describe J_x, i.e.,:

$${\mathrm{J}}_{\mathrm{x}}={}_{\mathrm{df}}[{\mathsf{\mu }}_{\underset{\_}{\mathrm{F}}}(\mathrm{x}),{\mathsf{\mu }}_{\overline{\mathrm{F}}}(\mathrm{x})]$$

(4)

which leads to the following:

$$\mathrm{FOU}(\tilde{\mathrm{F}}){=}_{\mathrm{df}}\underset{\mathrm{x}\in \mathrm{X}}{\overset{}{{\displaystyle \cup }}}{[\mathsf{\mu }}_{\underset{\_}{\mathrm{F}}}(\mathrm{x}){,\mathsf{\mu }}_{\overline{\mathrm{F}}}(\mathrm{x})]$$

(5)

Figure 1 clearly illustrates the above definitions.

2.2.2. Mamdani Type-2 Fuzzy System

In Figure 2, we show the information flow within an IT2 fuzzy system. It is very similar to its type-1 analogue. The major difference is that because of the IT2 fuzzy sets used in the rulebase, the outputs of the inference engine are IT2 fuzzy sets too. Therefore, a type reducer [6,43] must be applied to convert them into type-1 fuzzy sets to enable the defuzzification procedure.

Below, we give a brief description of the basic steps of the computations in an IT2 fuzzy system [40]. Let consider the rule base of an IT2 fuzzy system consisting of N rules taking the following form:

$${\tilde{\mathrm{R}}}^{\mathrm{n}}{:}_{\mathrm{df}}\mathrm{IF}({\mathrm{x}}_1{\mathrm{is}\tilde{\mathrm{X}}}_1^{\mathrm{n}})\mathrm{o}\dots \mathrm{o}({\mathrm{x}}_{\mathrm{i}}{\mathrm{is}\tilde{\mathrm{X}}}_{\mathrm{i}}^{\mathrm{n}}){\mathrm{THEN}\mathrm{y}\mathrm{is}\tilde{\mathrm{Y}}}_{\mathrm{j}}^{\mathrm{n}}$$

where ${\tilde{\mathrm{X}}}_{\mathrm{i}}^{\mathrm{n}}$ (i = 1, …, I; n = 1, …, N) are IT2 fuzzy sets defined over corresponding domains, and ${\tilde{\mathrm{Y}}}_{\mathrm{j}}^{\mathrm{n}}$(j = 1, …, M) is an IT2 fuzzy set, assuming a Mamdani type-2 system, which represents the corresponding rule conclusion. The logical operator ‘o’ is defined as a fuzzy conjunction or disjunction, i.e., o ∈ {⊗, ⊕}, where ⊗ and ⊕ are binary operators (t-norm and s-norm respectively) defined over [0, 1] (⊗, ⊕: [0, 1]² → [0, 1]). The t-norm operator provides the characterization of the AND fuzzy operator, while the s-norm provides the characterization of the OR fuzzy operator [44]. In our research, we applied the Zadeh’s t- and s-norms, which correspond to the min and max operators, respectively.

Assuming an input vector $\overline{\mathbf{x}}=\{{\mathrm{x}}_1^{\prime },{\mathrm{x}}_2^{\prime },{\mathrm{x}}_3^{\prime },\dots ,{\mathrm{x}}_{\mathrm{i}}^{\prime }\}$, typical computations of an IT2 fuzzy system consist of the following steps:

1)

Compute the membership intervals of ${\mathrm{x}}_{\mathrm{i}}^{\prime }$ for each ${\tilde{\mathrm{X}}}_{\mathrm{i}}^{\mathrm{n}}$, ${[\mathsf{\mu }}_{{\underset{\_}{\mathrm{X}}}_{\mathrm{i}}^{\mathrm{n}}}({\mathrm{x}}_{\mathrm{i}}^{\prime }){,\mathsf{\mu }}_{{\overline{\mathrm{X}}}_{\mathrm{i}}^{\mathrm{n}}}({\mathrm{x}}_{\mathrm{i}}^{\prime })]$, i = 1, …, I; n = 1, …, N

2)

Compute the firing interval of the n-th rule:

$${\mathrm{F}}^{\mathrm{n}}{(\overline{\mathbf{x}})=}_{\mathrm{df}}[{\mathsf{\mu }}_{{\underset{\_}{\mathrm{X}}}_1^{\mathrm{n}}}({\mathrm{x}}_1^{\prime }){\mathrm{o}\dots \mathrm{o}\mathsf{\mu }}_{{\underset{\_}{\mathrm{X}}}_{\mathrm{i}}^{\mathrm{n}}}({\mathrm{x}}_{\mathrm{i}}^{\prime }){,\mathsf{\mu }}_{{\overline{\mathrm{X}}}_1^{\mathrm{n}}}({\mathrm{x}}_1^{\prime }){\mathrm{o}\dots \mathrm{o}\mathsf{\mu }}_{{\overline{\mathrm{X}}}_{\mathrm{i}}^{\mathrm{n}}}({\mathrm{x}}_{\mathrm{i}}^{\prime })]\equiv {[\underset{\_}{\mathrm{f}}}^{\mathrm{n}}{,\overline{\mathrm{f}}}^{\mathrm{n}}]$$

3)

Apply type reduction to combine ${\mathrm{F}}^{\mathrm{n}}(\overline{\mathbf{x}})$ with corresponding rule consequents.

There are some methods of type reduction [6,45,46], but the most commonly used one is the center-of-set (COS) type reducer [6] using the Karnik–Mendel algorithms [6,43] or their variants [47,48]. As the whole procedure concerning Mamdani type-2 fuzzy system is well described in the literature, we will omit further details. For more clarity, in Figure 3 and Figure 4, we show the idea of the introduced calculations.

2.2.3. Rule Induction with Information Systems

An information system [12] is defined by the elements: (U, A, V, f), where U is a universe, A is a set of attributes, V represents attributes domains: ${\mathrm{V}=}_{\mathrm{df}}{\cup }_{\mathrm{a}}{\mathrm{V}}_{\mathrm{a}}$, with nonempty domain V_a of the a-th attribute (a ∈ A) and f is the so called information function f: U × A → V, ${\forall }_{\mathrm{x}\in \mathrm{U},\mathrm{a}\in \mathrm{A}}\mathrm{f}(\mathrm{u},\mathrm{a})\in {\mathrm{V}}_{\mathrm{a}}$. Important role in information systems plays the indiscernibility binary relation, which is an equivalence relation, defined over U: IND(B) = _df {(x, y) ∈ U × U: ${\forall }_{\mathrm{a}\in \mathrm{B}}\mathrm{f}(\mathrm{x},\mathrm{a})=\mathrm{f}(\mathrm{y},\mathrm{a})$}, under which the lower and upper approximations of any subset of U can be defined, respectively:

B↓X = _df {x ∈ U: [x]_IND ⊆ X},

(6)

B↑X = _df {x ∈ U: [x]_IND ∩ X ≠ ∅}, B ⊆ A and X ⊆ U.

(7)

The above approximations define a rough set [13,14]. We can consider an information system as a decision table if a decision attribute is introduced. With this assumption, a decision making approach was introduced by A. Skowron and Z. Suraj [41,42], presenting a rule induction process with respect to all considered decisions. The procedure consists of the following steps:

• Introduce an information system with decision attribute,
• Eliminate information system inconsistency, i.e., objects with the same information function values, but different decision values, applying lower or upper approximation precision analysis,
• Provide attribute reduct,
• Apply rule induction and define rules for all considered decisions, which cover the decision problem.

Below, we explain the main algorithms applied in the procedure above with examples. The reduct generation is omitted, as any other attribute selection method can be applied.

Inconsistency Elimination

Let consider the following information system with decision attribute: (U, A ∪ {a*}, V ∪ V_a*, f). If inconsistency is found, then Algorithm 1, which uses the lower approximation precision as a decisive factor, should be applied. The lower approximation precision is defined as follows:

$${\mathsf{\gamma }}_{\mathrm{B}}(\mathrm{X}){=}_{\mathrm{df}}\frac{|\mathrm{B}\downarrow \mathrm{X}|}{|\mathrm{U}|},\mathrm{B}\subseteq \mathrm{A}$$

(8)

Algorithm 1 (inconsistency elimination algorithm):

Input:Inconsistent information system Output:Consistent information system Let xi and xj are casing inconsistency (xi, xj ∈ U) and let $\mathrm{f}({\mathrm{x}}_{\mathrm{i}}{,\mathrm{a}}^{*}){=\mathrm{d}}_1,\mathrm{f}({\mathrm{x}}_{\mathrm{j}}{,\mathrm{a}}^{*}){=\mathrm{d}}_2$, then: 1) Define $\begin{array}{l}{\mathrm{X}}_1{=}_{ \mathrm{df}} \{\mathrm{x}\in \mathrm{U}:\mathrm{f}({\mathrm{x},\mathrm{a}}^{*}){=\mathrm{d}}_1\},\\ {\mathrm{X}}_2{=}_{ \mathrm{df}}\{\mathrm{x}\in \mathrm{U}:\mathrm{f}({\mathrm{x},\mathrm{a}}^{*}){=\mathrm{d}}_2\}{,(\mathrm{X}}_1, {\mathrm{X}}_2)\subseteq \mathrm{U})\end{array}$ 2) $\mathrm{Calculate}{\mathsf{\gamma }}_{\mathrm{B}}({\mathrm{X}}_1)\mathrm{and}{\mathsf{\gamma }}_{\mathrm{B}}({\mathrm{X}}_2) (B\subseteq A)..$ $\mathrm{If}{\mathsf{\gamma }}_{\mathrm{B}}({\mathrm{X}}_1){<\mathsf{\gamma }}_{\mathrm{B}}({\mathrm{X}}_2){\mathrm{Then}\mathrm{remove}\mathrm{from}\mathrm{U}\mathrm{x}}_{\mathrm{i}}{\mathrm{Else}\mathrm{remove}\mathrm{from}\mathrm{U}\mathrm{x}}_{\mathrm{j}}$

Example 1.

Let U = {x₁, x₂, x₃, x₄, x₅, x₆}, A = {a₁, a₂, a₃}, B = {a₁, a₃} ⊆ A, $V_{a_1}, V_{a_2},V_{a_3},V_{a_{*}}$ = {0, 1, 2, 3, 4} with information function values represented in the matrix below.

In the above matrix, we can discover inconsistency as objects x₂ and x₄ have the same information function values with respect to B, but different decision values. Therefore, we can apply Algorithm 1:

1)

X₁ = {x₁, x₃, x₄, x₆}; X₂ = {x₂, x₅},

2)

B↓X₁= {x₁, x₃, x₆},${\gamma }_B(X_1)$= 3/6 = 0.5; B↓X₂ = {x₅}, ${\gamma }_B(X_2)$ = 1/6 ≈ 0.17, which implies U = U\{x₂}.

Of course, it is only our assumption that different decision values for the same attribute values cause inconsistency. In the decision making process introduced, such decision values could be considered as a set of acceptable values. If that is the case, then the$d_{a^{*}}^B$sets defined below wouldn’t be singletons.

Rule Induction

The aim of this step is to induce rules for each decision regarding Algorithm 2 below.

Algorithm 2 (rule induction algorithm):

Input:Information system with decision attribute Output:Corresponding decision rules Step 1: Generate Mk, k = 1, 2, …, |U| matrixes using the so-called discernibility matrix of the information system, denoted below as M(IS) and defined as: $$\mathrm{M}(\mathrm{IS}){=}_{\mathrm{df}}{{[\mathrm{m}}_{\mathrm{i},\mathrm{j}}]}_{\mathrm{i},\mathrm{j}=1,\cdots ,\mathrm{n}}=\{\mathrm{a}\in \mathrm{A}:\mathrm{f}({\mathrm{x}}_{\mathrm{i}},\mathrm{a})\ne \mathrm{f}({\mathrm{x}}_{\mathrm{j}},\mathrm{a}),\mathrm{n}=|\mathrm{U}|\}$$ (9) Step 2: Define the object implicants, Step 3: Define the set of decision rules.

Step 1:

Let c_ij are the elements of M(IS), ${\widehat{\mathrm{c}}}_{\mathrm{ij}}$ are the elements of M_k (with respect to k, k = 1, ..., n; n = |U|) and a* is the decision attribute, then:

For each k = 1, ..., n:

1)

If i ≠ k then ${\widehat{\mathrm{c}}}_{\mathrm{ij}}=\varnothing ,$

2)

If${(\mathrm{c}}_{\mathrm{kj}}\ne \varnothing )$and$({\mathrm{d}}_{{\mathrm{a}}^{*}}^{\mathrm{B}}({\mathrm{x}}_{\mathrm{j}})\ne {\mathrm{d}}_{{\mathrm{a}}^{*}}^{\mathrm{B}}({\mathrm{x}}_{\mathrm{i}}))$then${\widehat{\mathrm{c}}}_{\mathrm{ij}} {=}_{ \mathrm{df}}{\mathrm{c}}_{\mathrm{kj}}\cap {\mathrm{B}\mathrm{else}\widehat{\mathrm{c}}}_{\mathrm{ij}}{=}_{\mathrm{df}}\varnothing ,$

where B ⊆ A and

$${\mathrm{d}}_{{\mathrm{a}}^{*}}^{\mathrm{B}}({\mathrm{x}}_{\mathrm{k}}) {=}_{\mathrm{df}}\{\mathrm{v}\in {\mathrm{V}}_{{\mathrm{a}}^{*}}:{\exists }_{\mathrm{y}\in \mathrm{U}}({\mathrm{x}}_{\mathrm{k}}\mathrm{IND}(\mathrm{B})\mathrm{y})\wedge \mathrm{f}({\mathrm{y},\mathrm{a}}^{*})=\mathrm{v}\}$$

(10)

For simplicity, let consider the example below that illustrates the generation of matrix M₁

Example 2.

Let assume an information system with decision attribute extended by the corresponding $d_{a^{*}}^B$ values (U = {x₁, x₂, x₃, x₄, x₅}, A = B = {a, b, c}, V_a = V_b = V_c = V_a* = {0, 1, 2}):

The corresponding discernibility matrix takes the form:

The matrix M(IS) is a symmetric matrix. Next, we can apply Algorithm 2—step 1 and therefore, define the matrix M₁ (k = 1):

because:$d_{a*}^B(x_2){,d}_{a*}^B(x_4)\ne \{0\}$, therefore${\widehat{c}}_{ij}{=c}_{1j}\cap B(j=2,4)$, and so:${\widehat{c}}_{12}=\{a,c\}\cap B=\{a,c\}$,${\widehat{c}}_{14}=\{a\}\cap B=\{a\}$.

Step 2:

The object implicants indicate which attributes and objects are strongly related. The first implicant is obtained from M₁ applying the corresponding Boolean algebra reduction rules: Implicant₁ (M₁): x₁ ⇒ a. The rest of the implicants are presented below:

• Implicant₂ (M₂): x₂ ⇒ c,
• Implicant₃ (M₃): x₃ ⇒ a,
• Implicant₄ (M₄): x₄ ⇒ a ∧ c,
• Implicant₅ (M₅): x₅ ⇒ a ∨ b.

Step 3:

Finally, using the above implicants, we can generate the target set of decision rules regarding the decision attribute values. Each rule represents one decision and it is obtained as a sum of the object implicants related to that decision, i.e., concerning decision attribute value ‘0’, we have: f(x₁,a*) = f(x₃,a*) = f(x₅,a*) = 0, and therefore: Rule₁ = _df. f(x₁,a) ∨ f(x₃,a) ∨ (f(x₅,a) ∨ f(x₅,b)) ⇒ (decision: 0)

by analogy:

• Rule₂ = _df. f(x₂,c) ⇒ (decision: 1),
• Rule₃ = _df. f(x₄,a) ∧ f(x₄,c) ⇒ (decision: 2).

The main disadvantage of the above decision making approach, in terms of vague or imprecise data, is that the induced rules are crisp, i.e., Rule₂ has the following interpretation: if an object of x₂ type has an information function value for the attribute ‘c’ exactly equal to ‘0’ then make the decision ‘1’. We solve this disadvantage by extending information systems to our assumption of a type-1 fuzzy information system.

2.2.4. Type-1 Fuzzy Information System

The major assumption of our research is the interpretation of a fuzzy information system. First, we define the values of the information function as type-1 fuzzy sets. This assumption solves the problem of crisp values by providing a generalisation. Therefore, we propose better data granulation and thus more robustness to vague or imprecise data. Next, by assuming Gaussian distribution for each attribute, we are able to define any information function value as one of the very basic fuzzy sets: low, medium, and high. Therefore, the value of any pair object-attribute is generalised by a fuzzy set. Below in

Table 2. A type-1 fuzzy information system.

	Attribute1	Attribute2	Attribute3	…	Decision Attribute
Object1	{low, medium, high}	{low, medium, high}	{low, medium, high}	…	D1
Object2	{low, medium, high}	{low, medium, high}	{low, medium, high}	…	D2
Object3	{low, medium, high}	{low, medium, high}	{low, medium, high}	…	D3
…	…	…	…	…	…

, we give an illustration of a type-1 fuzzy information system with a decision attribute, according to our assumptions.

Where:

$${\mathsf{\mu }}_{\mathit{medium}}(\mathrm{a}){=}_{\mathrm{df}}{\mathrm{e}}^{\frac{-{({\mathrm{a}-\mathrm{a}}_0)}^2}{{2\mathsf{\sigma }}^2}},\mathrm{a}\in {\mathrm{V}}_{\mathrm{a}}\mathrm{with}\mathrm{expected}\mathrm{value}({\mathrm{a}}_0)\mathrm{and}\mathrm{standard}\mathrm{deviation}(\mathsf{\sigma })$$

(11)

The fuzzy sets low and high attribute values are easy to define as well (see Figure 5):

$$\begin{array}{l}{\mathsf{\mu }}_{\mathrm{low}}(\mathrm{a}){=}_{\mathrm{df}}\left\{\begin{array}{c}{1-\mathrm{e}}^{\frac{-{({\mathrm{a}-\mathrm{a}}_0)}^2}{{2\mathsf{\sigma }}^2}}\\ 0:\mathrm{a}\ge {\mathrm{a}}_0\end{array}{:\mathrm{a}<\mathrm{a}}_0\right\},\\ {\mathsf{\mu }}_{\mathrm{high}}(\mathrm{a}){=}_{\mathrm{df}}\left\{\begin{array}{c}{0:\mathrm{a}<\mathrm{a}}_0\\ {1-\mathrm{e}}^{\frac{-{({\mathrm{a}-\mathrm{a}}_0)}^2}{{2\mathsf{\sigma }}^2}}:\mathrm{a}\ge {\mathrm{a}}_0\end{array}\right\}\end{array}$$

(12)

If a pair object_i-attribute_j has a value defined as low, for example, it means that:

$${\mathsf{\mu }}_{\mathrm{low}}({\mathrm{f}(\mathrm{object}}_{\mathrm{i}}{,\mathrm{attribute}}_{\mathrm{j}}))\ge \max \{{\mathsf{\mu }}_{\mathrm{medium}}(\mathrm{f}({\mathrm{object}}_{\mathrm{i}}{,\mathrm{attribute}}_{\mathrm{j}})){,\mathsf{\mu }}_{\mathrm{high}}(\mathrm{f}({\mathrm{object}}_{\mathrm{i}}{,\mathrm{attribute}}_{\mathrm{j}}))\}.$$

It is important to note that our assumption was also the direct application of the original Skowron and Suraj algorithm [41,42] for rule induction. As a result, we did not have to modify the introduced indistinguishability relation (IND) to define fuzzy rough sets or modify the quality of the approximation introduced by Equation (8). We directly used the fuzzified values of the information function as if they were crisp values. This means that object_i is related to object_j with respect to IND in accordance with our model if the values of the corresponding attributes define the same fuzzy sets. For example, below we show the partition (P) of the set of objects {Object₁, …, Object₅} implied by IND for a sample information system given in

Table 3. Sample information system with five objects and two attributes.

	Attribute1	Attribute2
Object1	low	high
Object2	medium	low
Object3	medium	low
Object4	high	high
Object5	low	high

.

The corresponding partition with respect to IND has three equivalence classes:

P/_{IND({Attribute1, Attribute2})} = _df. {{Object₁, Object₅}, {Object₂, Object₃}, {Object₄}}.

The above assumption made possible the use of classical algorithms introduced in information systems theory, while at the same time generalizing the information given by the attributes. The use of membership degrees inferred from corresponding membership functions is performed after the induction of rules and their transformation into fuzzy rules, in accordance with the assumptions of the fuzzy model used. Of course, such an assumption may be too strong a generalization. We solve this problem by introducing various possibilities of fuzzification. Defining additional fuzzy sets and transforming them into interval type-2 fuzzy sets with the corresponding optimization, depending on the examined dataset (see

Table 4. The IT2 fuzzy sets used for a sample attribute of the blood transfusion data set. For all attributes, we normalized the corresponding domain values.

<3 Gausses, Equal, Center>	<3 Gausses, Equal, Mean>
<3 Gausses, Progressive, Center>	<3 Gausses, Progressive, Mean>
<5 Gausses, Equal, Center>	<5 Gausses, Equal, Mean>
<5 Gausses, Progressive, Center>	<5 Gausses, Progressive, Mean>
<7 Gausses, Equal, Center>	<7 Gausses, Equal, Mean>
<7 Gausses, Progressive, Center>	<7 Gausses, Progressive, Mean>
<9 Gausses, Equal, Center>	<9 Gausses, Equal, Mean>
<9 Gausses, Progressive, Center>	<9 Gausses, Progressive, Mean>
<11 Gausses, Equal, Center>	<11 Gausses, Equal, Mean>
<11 Gausses, Progressive, Center>	<11 Gausses, Progressive, Mean>

below).

Next, we define the decision attribute values as fuzzy sets, which affect the degree of decision. Any induced rules from such an information system can be easily transformed into fuzzy rules for a type-1 Mamdani fuzzy system. For example, if we have pairs: (object₁, attribute₁): low, (object₂, attribute₁): high, (object₃, attribute₂): medium and a rule which defines decision D as:

• (f(object₁, attribute₁) ∧ f(object₂, attribute₁)) ∨ f(object₃, attribute₂),

then we can transform it into the fuzzy rule:

• If ((f(object₁, attribute₁) is low) ⊗ (f(object₂, attribute₁) is high)) ⊕ (f(object₃, attribute₂) is medium) Then D.

2.2.5. Transformation into Type-2 Fuzzy Sets

To apply type-2 fuzzy sets in the fuzzy rule induction procedure described, we propose a simple extension. After induction of type-1 fuzzy rules, we modify the membership functions applied by changing the standard deviation of the ‘medium’ fuzzy set. By doing so, we can define the bounds of the FOU for the corresponding type-2 fuzzy set $\widetilde{\mathit{medium}}$. This also makes the possibility to define $\widetilde{\mathit{low}}$ and $\widetilde{\mathit{high}}$ type-2 fuzzy sets as well. The idea is given in Figure 5.

The above extension gives us the possibility to apply the type-2 Mamdani fuzzy inference procedure. In our research, we have applied the procedure proposed to solve classification problems.

2.2.6. Fuzzy Rules Optimization

In our experiments, the basic assumption was to analyse the influence of the shape and the FOU of the type-2 fuzzy sets used in the rule premises, derived from the corresponding fuzzy information system, on the classification accuracy. For this purpose, we have used the type-2 fuzzy sets shown in Table 4. Each of these type-2 membership functions was derived for every attribute of all benchmark data analyzed. Below, we explain the descriptions used in our research:

• The number of Gaussian functions for fuzzification: {3, 5, 7, 9, 11}. The fuzzification procedure is as follows: generate the medium membership function and, next, the rest of the membership functions based on it. All Gaussians are transformed into type-2 fuzzy sets by changing the standard deviation value (called sigma_offset parameter). The sigma_offset defines the FOU of the corresponding IT2 fuzzy sets. The same value was applied for each Gaussian function.
• If the standard deviations applied to the Gaussians are the same or not: {equal, progressive}—progressive assumes the concentration of membership functions around the mean or central values. For 3 Gaussians, equal and progressive, give the same fuzzification result.
• If the mean of the medium basic function is derived directly from the corresponding data set or if a fixed value is used: {mean, center}—center assumes Gaussian distribution with fixed mean 0.5.

For example, <3 Gausses, Equal, Mean> is considered as 3 Gaussian membership functions with equal standard deviation values used to define 3 type-2 fuzzy sets, and the medium function is defined by its corresponding mean value derived from the considered data. The <5 Gausses, Equal, Center> is considered as 5 Gaussian membership functions and the medium function is defined by a fixed mean value 0.5. The <7 Gausses, Progressive, Mean> is considered as 7 Gaussian membership functions with different standard deviation values, assuming concentration of membership functions around the mean value of the medium function.

For the type-2 sets used in the conclusions, we define s-shaped functions representing each class in the classification problem considered. We optimize the FOU of each function by changing the cross-points. We called the corresponding parameter under optimization center_offset—see Figure 6. The final classification decision value was fixed = 0.5, assuming normalized domain [0, 1]. Changes of the classification threshold did not show a positive effect on the experiment.

For each benchmark data, we optimized the sigma_offset and center_offset parameters for the type-2 fuzzy sets considered, using the grid search optimization algorithm applied in k-folded cross-validation process with evaluation on a held-out validation set. The sigma_offset takes values in the range [0.01, 0.03] with step 0.005 and the center_offset takes values in the range [0.01, 0.21] with step 0.02. The k value was chosen for any data set assuming a fair number of samples in the corresponding validation sets, presented in

Table 5. Number of folds applied in the cross-validation process.

Data Set	k-Value
Blood Transfusion	10
Breast Cancer Data	10
Breast Cancer Wisconsin	10
Data Banknote Authentication	10
Haberman	6
Heart	5
HTRU 2	10
Immunotherapy	4
Indian Liver Patient	10
Ionosphere	6
Parkinson	4
Pima Indians Diabetes	10
Vertebral	5

. Additionally, for any data partition, we have used stratified sampling to ensure balanced validation and test sets. For better clarity, we show the experiment information flow in Figure 7.

3. Results

In

Table 6. Evaluation of the method proposed.

Data Set	Classification Accuracy	F-Score	Classifier
Blood Transfusion	0.742	0.541	<11 Gausses, Equal, Mean>
Breast Cancer Data	0.918	0.912	<7 Gausses, Progressive, Mean>
Breast Cancer Wisconsin	0.961	0.957	<3 Gausses, Equal, Center> or <3 Gausses, Progressive, Center>
Data Banknote Authentication	0.99	0.993	<9 Gausses, Progressive, Mean>
Haberman	0.783	0.655	<9 Gausses, Progressive, Mean>
Heart	0.802	0.802	<5 Gausses, Equal, Mean>
HTRU 2	0.968	0.89	<9 Gausses, Progressive, Center>
Immunotherapy	0.889	0.8	<9 Gausses, Progressive, Mean>
Indian Liver Patient	0.686	0.592	<7 Gausses, Progressive, Mean>
Ionosphere	0.896	0.888	<7 Gausses, Equal, Mean>
Parkinson	0.881	0.847	<7 Gausses, Progressive, Center>
Pima Indians Diabetes	0.732	0.677	<11 Gausses, Progressive, Center>
Vertebral	0.85	0.821	<9 Gausses, Equal, Center>

we present the results achieved in our experiments. The number of rules generated for each data set is constant and equal to the cardinality of the decision attribute. In the case of binary classification, two fuzzy rules will be generated, but with very extensive and complex rule premises.

We achieved quite high classification results for seven data sets, which proves the purposefulness of the methodology used.

Below in

Table 7. Comparison regarding other classifiers with respect to the F-score measure.

Data Set	Our Approach	Other Classifiers
Breast Cancer Data	0.912	Immune-inspired semi-supervised Algorithm, introduced in [49]: 0.973
Breast Cancer Wisconsin	0.957	Extreme Learning Machine Neural Networks, introduced in [50]: 0.978 Immune-inspired semi-supervised algorithm, introduced in [49]: 0.965 Support vector machines combined with Feature Selection, introduced in [51]: 0.997
Data Banknote Authentication	0.993	Deep Neural Network with PCA and LDA, introduced in [52]: 0.99 Decision tree approach, introduced in [53]: 0.994 Random Forest approach, introduced in [54]: 0.948 Neural Network-Genetic Algorithm, introduced in [55]: 1
HTRU 2	0.89	Classical classifiers: C4.5: 0.74; MLP: 0.752; NB: 0.692; SVM: 0.789 GH-VFDT Algorithm, introduced in [37]: 0.862 A hybrid ensemble method, introduced in [56]: 0.918 (with respect to a voting threshold parameter)
Immunotherapy	0.8	A hybrid classification approach. introduced in [57]: 0.851 Considering evaluation of machine learning techniques [58]: 0.91

, we show a comparison regarding high performing classifiers for some of the data sets with respect to the F-score measure.

As we can see above, we achieved quite good results for the datasets data banknote authentication and HTRU 2, despite the very general fuzzy approach proposed.

Classification accuracy comparison with fuzzy classification techniques for chosen data sets is shown in

Table 8. Classification accuracy comparison with fuzzy classifiers.

Data Set	Our Approach	Fuzzy Classifiers
Breast Cancer Wisconsin	0.961	EM-PCA-CART-Fuzzy Rule-Based classification, introduced in [59]: 0.932
HTRU 2	0.968	Fuzzy neural network models: SORAF model [60]: 0.9597; models introduced in [61,62]: 0.961 and 0.952 respectively.
Immunotherapy	0.889	Fuzzy neural network approach, introduced in [63]: 84.32

as well.

4. Discussion

We distinguish several important conclusions and advantages of our research. First of all, it is possible to aggregate information by interpreting attributes describing objects as fuzzy sets, using the advantages of fuzzy logic for flexible modelling of real data. This enables a large generalization of information, which positively affects unbalanced or inaccurately defined data. Additionally, the introduced interpretation of fuzzy information systems allows direct work on fuzzy sets and the induction of rules that can be easily changed into type-2 fuzzy rules. By appropriate fuzzification, the values of the information function can be derived directly from the data. Removal of inconsistencies, which may appear because of the information generalization applied, is performed by appropriate use of rough sets, i.e., by analyzing the quality of the corresponding approximations. The fuzzification applied is very general. It allows even better adjustment to special cases by optimization of the fuzzy sets in the premises and the conclusions of the rules applied. In this research, we have used only one of the many possible fuzzifications of the data presented. Applying fuzzification adapted to a specific problem, after an in-depth analysis, we should be able to adjust the fuzzy sets in the premises and in the conclusions of the rules to achieve the best possible result. The methodology we introduced in our work is a general approach, aiming to show the advantage of fuzzy inference using a simple and homogeneous model. Despite that our algorithm was not strictly adjusted to any data set, we managed to obtain good results for some of them. It shows practical potential and creates the basis for generating robust classifiers.

5. Conclusions

In this research, we proposed a new fuzzy classification method. Our concept assumes the induction of fuzzy rules from the corresponding fuzzy information system. The applied fuzzification of data is done in a very simple manner by Gaussian type membership functions. Moreover, we are able to extend the induced rules into interval type-2 fuzzy rules. The experiments provided with benchmark data proved the method usefulness. Additionally, by further optimization of the interval type-2 fuzzy sets applied, we were able to improve our classification results. More suitable fuzzification, adjusted to a specific data set, is possible as well. Our concept should be considered as a general approach for unbalanced or inaccurately defined data.