On Granular Rough Computing: Handling Missing Values by Means of Homogeneous Granulation ^†

Abstract

This paper is a continuation of works based on a previously developed new granulation method—homogeneous granulation. The most important new feature of this method compared to our previous ones is that there is no need to estimate optimal parameters. Approximation parameters are selected dynamically depending on the degree of homogeneity of decision classes. This makes the method fast and simple, which is an undoubted advantage despite the fact that it gives a slightly lower level of approximation to our other techniques. In this particular article, we are presenting its performance in the process of missing values absorption. We test selected strategies on synthetically damaged data from the UCI repository. The added value is to investigate the specific performance of our new granulation technique in absorbing missing values. The effectiveness of their absorption in the granulation process has been confirmed in our experiments.

1. Introduction

Granular computing is a paradigm, dedicated to computing, based on objects similar to each other on the basis of selected similarity measure. The idea was proposed by Lotfi Zadeh [1,2]. Granulation is a part of the fuzzy theory by the very definition of fuzzy set, where inverse values of fuzzy membership functions are the basic forms of granules. Shortly after Lotfi Zadeh proposed the idea of granular computing, the granules were introduced in terms of rough set theory by T.Y. Lin, L. Polkowski, and A. Skowron. In rough set theory, granules are defined as classes of indiscernibility relations. Interesting research on more flexible granules based on blocks was conducted by (Grzymala–Busse), and templates by (H.S. Nguyen). The granules based on rough inclusions was introduced by (Polkowski and Skowron [3]), based on tolerance or similarity relations, and, more generally, binary relations by (T.Y. Lin [4], Y. Y. Yao [5,6,7]). Being in the context of rough mereology was proposed by L. Polkowski and A.Skowron, approximation spaces by A. Skowron and J. Stepaniuk [8,9], and logic for approximate reasoning by L.Polkowski and M. Semeniuk-Polkowska [10], and Qing Liu [11]. Examples of interesting studies from recent years can be found in [12,13,14,15,16,17,18].

This is a work about using granular rough computing techniques to absorb missing values [19]. The exact theoretical introduction to the family of approximation methods to which our methods belong to can be found in [20,21,22]. Of course, to understand the body of the algorithmic work, we have included all the relevant details in the following sections.

Our recently developed homogeneous granulation method is described in [23]. The main difference in our granulation algorithm, to the previously developed ones, is that there is no need to estimate the granulation radius. This parameter is being set automatically depending on the indiscernibility level of the decision classes. The degree of indiscernibility is the percentage of attributes for which objects are identical. Homogeneous granulation was also implemented in the epsilon variant for numerical data, which was described in [24,25,26] as well as being a part of a novel ensemble model—Ensemble of granular reflections—described in detail in [27]. The main motivation to carry out the tests were our previous research results in the context of absorption of unknown values, which gave very interesting results. The creation of a new technique naturally caused scientific curiosity to examine its performance in the same context.

This paper presents some preliminary experiments of using the homogeneous granulation as a missing values absorption technique. We have taken into consideration four absorption strategies, which we have named A, B, C, and D. Results of using those strategies with other granulation methods are available in Polkowski and Artiemjew [28,29].

Below is a detailed description of the strategies used.

1.1. Selected Key Variants

The strategies to consider are as follows:

• Variant A: in granulation process *=each value, when fixing the unadsorbed values *, *=each value.
• Variant B: in granulation process *=each value, when fixing the unadsorbed values *, $*=*$.
• Variant C: in granulation process$*=*$, when fixing the unadsorbed values *, *=each value.
• Variant D: in granulation process$*=*$, when fixing the unadsorbed values *, $*=*$.

In the granulation process, taking A and B strategies into consideration, stars evaluation of the similarity to any other value is always positive. For C and D variants, stars are treated as stars—so they evaluate as positive only when comparing with other stars. Those strategies bring up the following granulation definition:

In the following variants, we see the process of granule formation, where we use two basic options when comparing descriptors $*=eachvalue$ and $*=*$. Obviously, in the $*=eachvalue$ variant, the granules increase their size, i.e., after approximation they absorb potentially more damage values.

Considering $o{b}_1$ as the center of the granule, $o{b}_2$ as the compared object of the training system, r as the indiscernibility degree of the descriptors, $IND$ as the indiscernibility relation, and d as the decision attribute, we have considered the following options of internal processes for repairing unknown values. For readability, we have placed the legend of the applied markings in

Table 1. The legend of the applied markings.

Name	Description
$TRAI{N}_i$	i-th training decision system, used in cross validation process
$o{b}_i$	i-th object of selected decision system
$gran$	granule
$radius$	granulation radius
$Attr$	set of conditional attributes
$IND$	indiscernibility relation
d	decision attribute
$MaVot$	Majority Voting procedure
$conc\_dep$	concept-dependent variant of granulation
$\left|set\right|$	cardinality of the set

.

1.1.1. For Variant $*=eachValue$, the Granulation Process of the i-th Training Set $TRAI{N}_i$ Looks as Follows (A, B Variants)

$$gra{n}_{radius}^{conc\_dep,*=eachvalue}\left(o{b}_1\right)=\{o{b}_2\in TRAI{N}_i:\frac{|IN{D}^{*=eachvalue}(o{b}_1,o{b}_2)|}{\left|Attr\right|}\le radius\&d\left(o{b}_1\right)=d\left(o{b}_2\right)\},$$

for $IND$ defined as

$$IN{D}^{*=eachvalue}(o{b}_1,o{b}_2)=\{a\in Attr:a\left(o{b}_1\right)=a\left(o{b}_2\right)\parallel a\left(o{b}_1\right)=*\parallel a\left(o{b}_2\right)=*\}.$$

where $\&$means AND, $\parallel$ means OR.

1.1.2. For Variant $*=*$, The Granulation Process of the Set $TRAI{N}_i$ Looks as Follows (C, D Variants)

$$gra{n}_{radius}^{conc\_dep,*=*}\left(o{b}_1\right)=\{o{b}_2\in TRAI{N}_i:\frac{|IN{D}^{*=*}(o{b}_1,o{b}_2)|}{\left|Attr\right|}\le radius\&d\left(o{b}_1\right)=d\left(o{b}_2\right)\},$$

for $IND$ defined as

$$IN{D}^{*=*}(o{b}_1,o{b}_2)=\{a\in Attr:a\left(o{b}_1\right)=a\left(o{b}_2\right)\}.$$

1.1.3. For Variant $*=eachValue$, the Way We are Fixing the Unadsorbed Values of $TRAI{N}_i$ Looks as Follows (A, C Variants)

In this variant, the saved stars are replaced with the most frequently appearing attribute value of the training system.

For variant A, granule surrounding the defective sample $MaVot\left(gra{n}_{radius}^{conc\_dep,*=eachvalue}\left(o{b}_1\right)\right)$ (further mark as $temp$) looks like the below:

$$\mathrm{if}{a}_j\left(temp\right)=*,$$

The repairing process looks as follows:

$$gra{n}_{radius,a_j}^{conc\_dep,*=eachvalue}\left(temp\right)=\{o{b}_2\in TRAI{N}_i:\frac{|IN{D}_{a_j}^{*=eachvalue}(temp,o{b}_2)|}{\left|Attr\right|}\le radius\&d\left(temp\right)=d\left(o{b}_2\right)\},$$

$IND$ is defined as

$$IN{D}_{a_j}^{*=eachvalue}(temp,o{b}_2)=\{a\in Attr:(a\left(temp\right)=a\left(o{b}_2\right)\parallel a\left(temp\right)=*\parallel a\left(o{b}_2\right)=*)\&a_j\left(o{b}_2\right)!=*\}.$$

For variant C, granule surrounding the defective sample $MaVot\left(gra{n}_{radius}^{conc\_dep,*=*}\left(o{b}_1\right)\right)$ (further mark as $temp2$) looks like the below:

$$\mathrm{if}{a}_j\left(temp2\right)=*,$$

The repairing process looks as follows:

$$gra{n}_{radius,a_j}^{conc\_dep,*=eachvalue}\left(temp2\right)=\{o{b}_2\in TRAI{N}_i:\frac{|IN{D}_{a_j}^{*=eachvalue}(temp2,o{b}_2)|}{\left|Attr\right|}\le radius\&d\left(temp2\right)=d\left(o{b}_2\right)\},$$

$IND$ is defined as

$$IN{D}_{a_j}^{*=eachvalue}(temp2,o{b}_2)=\{a\in Attr:(a\left(temp2\right)=a\left(o{b}_2\right)\parallel a\left(temp2\right)=*\parallel a\left(o{b}_2\right)=*)\&a_j\left(o{b}_2\right)!=*\}.$$

1.1.4. For Variant $*=*$, the Way We Fix the Unabsorbed Values of $TRAI{N}_i$ Looks as Follows (B, D Variants)

In this variant, the saved stars are completed with the most frequently appearing attribute value of the training system.

For variant B, granule surrounding the defective sample $MaVot\left(gra{n}_{radius}^{conc\_dep,*=eachvalue}\left(o{b}_1\right)\right)$ (further mark as $temp3$) can be defined as follows:

$$gra{n}_{radius,a_j}^{conc\_dep,*=*}\left(temp3\right)=\{o{b}_2\in TRAI{N}_i:\frac{|IN{D}_{a_j}^{*=*}(temp3,o{b}_2)|}{\left|Attr\right|}\le radius\&d\left(temp3\right)=d\left(o{b}_2\right)\},$$

$IND$ is defined as

$$IN{D}_{a_j}^{*=*}(temp3,o{b}_2)=\{a\in Attr:a\left(temp3\right)=a\left(o{b}_2\right)\&a_j\left(o{b}_2\right)!=*\}.$$

For variant D, the granule surrounding the defective sample $MaVot\left(gra{n}_{radius}^{conc\_dep,*=*}\left(o{b}_1\right)\right)$ (further mark as $temp4$) looks like that shown below:

$$gra{n}_{radius,a_j}^{conc\_dep,*=*}\left(temp4\right)=\{o{b}_2\in TRAI{N}_i:\frac{|IN{D}_{a_j}^{*=*}(temp4,o{b}_2)|}{\left|Attr\right|}\le radius\&d\left(temp4\right)=d\left(o{b}_2\right)\},$$

$IND$ is defined as

$$IN{D}_{a_j}^{*=*}(temp4,o{b}_2)=\{a\in A:a\left(temp4\right)=a\left(o{b}_2\right)\&a_j\left(o{b}_2\right)!=*\}.$$

2. Homogenous Granulation in $*=*$ and $*=\mathit{each}\mathit{value}$ Variant

For $IN{D}^{*=eachvalue}(o{b}_1,o{b}_2)$, and variant $*=eachvalue$, we create the granule,

$$gra{n}_{r_u}^{homogenous,*=eachvalue}=\{o{b}_2\in U:|gra{n}_{r_u}^{conc\_dep,*=eachvalue}|-|gra{n}_{r_u}^{*=eachvalue}|==0,$$

$$forminimal{r}_{u}fulfillstheequation\}$$

$$gra{n}_{r_u}^{conc\_dep,*=eachvalue}=\{o{b}_2\in U:\frac{IN{D}^{*=eachvalue}(o{b}_1,o{b}_2)}{\left|Attr\right|}\le r_u\&d\left(o{b}_1\right)==d\left(o{b}_2\right)\}$$

$$gra{n}_{r_u}^{*=eachvalue}=\{o{b}_2\in U:\frac{IN{D}^{*=eachvalue}(o{b}_1,o{b}_2)}{\left|Attr\right|}\le r_u\}$$

$$r_u=\{\frac{i}{\left|Attr\right|},wherei=0.,1.,\dots ,\left|Attr\right|\}$$

in case of $*=*$ variant and earlier defined $IN{D}^{*=*}(o{b}_1,o{b}_2)$, the granule is as follows:

$$gra{n}_{r_u}^{homogenous,*=*}=\{o{b}_2\in U:|gra{n}_{r_u}^{conc\_dep,*=*}|-|gra{n}_{r_u}^{*=*}|==0,$$

$$forminimal{r}_{u}fulfillstheequation\}$$

$$gra{n}_{r_u}^{conc\_,*=*}=\{o{b}_2\in U:\frac{IN{D}^{*=*}(o{b}_1,o{b}_2)}{\left|Attr\right|}\le r_u\&d\left(o{b}_1\right)==d\left(o{b}_2\right)\}$$

$$gra{n}_{r_u}^{*=*}=\{o{b}_2\in U:\frac{IN{D}^{*=*}(o{b}_1,o{b}_2)}{\left|Attr\right|}\le r_u\}$$

$$r_u=\{\frac{i}{\left|Attr\right|},wherei=0.,1.,\dots ,\left|Attr\right|\}$$

3. Testing Session

This section is describing the experimental part followed by presentation of the results. The effectiveness was calculated on an artificially damaged datasets (10 percent of the data has been replaced with stars) chosen from UCI Repository [30].

3.1. The Steps of the Procedure

(i)

Selected dataset was uploaded,

(ii)

The data have been prepared for the Cross Validation 5 model,

(iii)

The $TRAI{N}_i^{complete}$ was granulated using a proper variant,

(iv)

The $TES{T}_i$ was classified based on $TRAI{N}_i^{complete}$ using kNN (the nil case),

(v)

$TRAI{N}_i^{complete}$ was filled with a fixed percentage of random stars;

(vi)

$TRAI{N}_i$ was fixed by granulation based on the chosen variant—$A,B,C$, or D

(vii)

classification of $TES{T}_i$ was performed based on a fixed training system using kNN,

(viii)

the average result of the classification was calculated from all of the folds,

We perform the procedure five times, receiving the mean value from each test $5Cross\_V5$).

3.2. Verification of Results Stability

We have computed an additional parameter to show the bias of accuracy, defined as follows:

$$Bias\_Acc=\frac{{\sum }_{i=1}^5(max(accurac{y}_1^{Cross\_V5},accurac{y}_2^{Cross\_V5},\dots ,accurac{y}_5^{Cross\_V5})-accurac{y}_i^{Cross\_V5})}{5},$$

(1)

for

$$accuracy=\frac{{\sum }_{i=1}^{5}accurac{y}_i^{Cross\_V5}}{5}.$$

The classifier used for our experiments is a classical kNN, where the smallest summary distance of k-nearest objects indicates the decision parameter value. k parameters are estimated with the $Cross\_V5$ method on a sample of data, which resulted in $k=5$ for Australian Credit. $k=3$ for Pima Indians Diabetes, $k=19$ for Heart Disease, $k=3$ for Hepatitis, and $k=18$ for the German Credit data set. We have selected the kNN classifier for testing due to the fact that, in past tests, testing other granulation variants to absorb unknown values, we used the same classification variant as the base classifier. Our performance tests, NB classifier, kNN, SVM, and deep neural networks, showed that kNN is fully comparable with the best classifiers in the context of granular reflection based classification.

3.3. Overview of the Testing Results

The results of missing values absorption using concept dependent granulation are shown in Table 2, Table 3, Table 4, Table 5 and Table 6. For homogeneous granulation, please refer to Table 7, Table 8, Table 9, Table 10 and Table 11. As a conclusion of the research presented in [22], we can say that granulation is an effective technique of absorbing some degree of missing values placed in the dataset. Our observations were proved by comparable classification results with the non-missing values data case. We need to point out that granulation brings another important benefit—it can significantly (up to 80 percent) reduce the number of objects used for classification. As shown in [22], this behavior strictly depends on the diversity of used datasets. Using strategies A and B for lower values of granulation radius, the approximation is faster because the $*=eachvalue$ variant causes a higher number of objects in the granules. In case of $*=*$, stars can increase diversity of the data and consequently a higher number of granules containing fewer number of objects than in the $*=eachvalue$ case.

Comparing those results to the homogeneous granulation as a missing values absorption method, those gave the following findings. This technique is increasing the number of granules in the coverings—see Table 7, Table 8, Table 9, Table 10 and Table 11—and the indiscernability, in the context of decision classes, is lowering. This gives a higher probability of finding an object which breaks the homogeneity of the formed granule. Despite the fact that strategies A and B are returning smaller granules than in case C or D, the final granular reflection systems are bigger.

For given parameters, our methods work in a stable way, and the results are comparable to the $nil$ case. A single run which is performed during the homogeneous granulation process is its biggest advantage, which might be the decisive factor when looking for the most robust method.

The results, showing our techniques using the strategy of completing unknown values with the most common values [31], can be found in

Table 12. Missing values handling using the most common value strategy; 5 × $Cross\_V5$; we consider repair options when $*=$each value, $*=*$ and nil case (classification based on original, undamaged training system) $synthetic10\%damage$; $Bias\_Acc$ = defined in Equation (1); $Trn\_Size$ = Average number of training objects, $d_1$ = Australian Credit, $d_2$ = Pima Indians Diabetes, $d_3$ = Heart Disease, $d_4$ = Hepatitis, $d_5$ = German Credit.

	Accuracy			Bias_Acc			Trn_Size
Data Set	$\mathit{nil}$	* = Each Value	* = *	$\mathit{nil}$	* = Each Value	* = *	$\mathit{nil}$	* = Each Value	* = *
$d_1$	$0.862$	$0.849$	$0.849$	$0.012$	$0.008$	$0.008$	552	$550.56$	$550.56$
$d_2$	$0.651$	$0.636$	$0.636$	$0.006$	$0.014$	$0.014$	$614.4$	$613.48$	$613.48$
$d_3$	$0.829$	$0.83$	$0.83$	$0.008$	$0.007$	$0.007$	216	$215.64$	$215.64$
$d_4$	$0.892$	$0.884$	$0.884$	$0.005$	$0.019$	$0.019$	124	$123.36$	$123.36$
$d_5$	$0.727$	$0.715$	$0.715$	$0.012$	$0.004$	$0.004$	800	$799.36$	$799.36$

. As we can see, they are equivalent to the results for the radius 1, in our strategies, where there is no approximation of training systems. Additionally, in

Table 13. The degree of homogeneity—in the sense of homogeneous granulation—of the examined systems.

Name	$\mathit{Radii}\_\mathit{Range}$
$Australian-credit$	$r_u\ge 0.5$
$Diabetes$	$r_u\ge 0.25$
$Heartdisease$	$r_u\ge 0.461$
$Hepatitis$	$r_u\ge 0.579$
$German-credit$	$r_u\ge 0.6$

, we have included degrees of homogeneity of the examined systems, i.e., the range of radii that appears during the homogeneous granulation process.

4. Conclusions

Comparing concept dependent and homogeneous granulation as a missing values absorption technique, we can point to the following conclusions.

The $*=eachvalue$ variant used with concept dependent granulation generates more approximate datasets (diversity reduction) while the $*=*$ case may increase the diversity. The granules are smaller for C and D strategies compared to the strategies A and B. Granulation of systems containing missing values reduces its size to a much higher degree than the granulation of undamaged datasets.

We can observe specific results when using homogeneous granulation as a missing values absorption technique. When comparing the results to the nil case—granulation of the undamaged dataset—granules in A and B strategies are smaller than those from C and D. It is happening because the $*=eachvalue$ case is breaking the homogeneity of the decision classes to a higher degree than the $*=*$ case. The approximation level is decreasing for damaged datasets.

Granulation techniques are absorbing missing values in an effective way as confirmed by the classification results of the $Cross\_V$ model. The most missing values are repaired during the granulation process no matter which technique is being used.

In our research, we are going to choose the most effective technique among known classifiers for specific types of data. We also plan to implement and check effectiveness of homogeneous granulation in the context of classification based on deep neural networks.