FP-Conv-CM: Fuzzy Probabilistic Convolution C-Means

Abstract

Soft computing models based on fuzzy or probabilistic approaches provide decision system makers with the necessary capabilities to deal with imprecise and incomplete information. Hybrid systems based on different soft computing approaches with complementary qualities and principles have also become popular. On the one hand, fuzzy logic makes its decisions on the basis of the degree of membership but gives no information on the frequency of an event; on the other hand, the probability informs us of the frequency of the event but gives no information on the degree of membership to a set. In this work, we propose a new measure that implements both fuzzy and probabilistic notions (i.e., the degree of membership and the frequency) while exploiting the ability of the convolution operator to combine functions on continuous intervals. This measure evaluates both the degree of membership and the frequency of objects/events in the design of decision support systems. We show, using concrete examples, the drawbacks of fuzzy logic and probability-based approaches taken separately, and we then show how a fuzzy probabilistic convolution measure allows the correction of these drawbacks. Based on this measure, we introduce a new clustering method named Fuzzy-Probabilistic-Convolution-C-Means (FP-Conv-CM). Fuzzy C-Means (FCM), Probabilistic K-Means (PKM), and FP-Conv-CM were tested on multiple datasets and compared on the basis of two performance measures based on the Silhouette metric and the Dunn’s Index. FP-Conv-CM was shown to improve on both metrics. In addition, FCM, PKM, and FP-Conv-CM were used for multiple image compression tasks and were compared based on three performance measures: Mean Square Error (MSE), Peak Signal-to-Noise Ratio (PSNR), and Structural SImilarity Index (SSIM). The proposed FP-Conv-CM method shows improvements in all these three measures as well.

1. Introduction

Thanks to their ability to simulate human reasoning, fuzzy and probabilistic models allow the construction of automatic decision makers capable of handling very complex phenomena using limited knowledge, in particular for clustering, which is an unsupervised learning task consisting of grouping a collection of elements into subgroups so that the elements in the same group are more similar (according to a certain measure) compared with the objects in other groups. Clustering plays a critical role in data mining and as a standard method for statistical data analysis. It is widely employed in various tasks in machine learning, pattern recognition, image analysis, information retrieval, bioinformatics, data compression, and computer graphics. In this work, we introduce a new clustering method named Convolution-Fuzzy-Probabilistic-C-Means (FP-Conv-CM) that implements a new sub-measure, and which is a hybrid of two soft computing approaches: fuzzy logic and probabilistic methods.

The fact that the term “cluster” is not precisely defined is one reason behind the existence of numerous clustering methods. One thing they all have in common is the grouping of data objects. Nevertheless, many researchers have developed and employed various cluster models, and, again, several methods may be provided for each of these cluster models. Typical cluster models include connectivity models [1], centroid models [2], distribution models, density-based clustering [3], subspace models [4,5], graph-based models [6], and neural clustering models [7].

Clustering methods can be roughly distinguished into hard clustering or soft clustering methods. There are more specific distinctions that can be discussed, including (a) hierarchical clustering [8] (a child cluster’s objects are likewise a part of the parent cluster); the hierarchy can be built either by dividing or merging recursively [9,10], (b) strict partitioning clustering with and without outliers (objects might belong to none, which is referred as outliers); the partition-based clustering technique divides the data points of a dataset into different partitions [11,12,13], (c) overlapping clustering techniques, where an object may belong to one or more clusters; partitioning algorithms that overlap include the commonly used Fuzzy K-means and its variations [14,15], (d) subspace clustering (i.e. while an overlapping clustering, within an individually defined subspace, clusters are not expected to overlap) [16,17]. Fuzzy clustering is one of the most established clustering methods that partition the dataset into overlapping clusters, i.e., each object belongs to several clusters with different degrees of membership [18]. The notion of membership function embodies the concept of fuzzy sets and fuzzy logic. The effectiveness of soft clustering has been confirmed when dealing effectively with the discrimination of datasets in n-dimensional spaces and is more useful for unlabeled data and datasets with outliers.

Dunn presented the widely used Fuzzy C-mean algorithm (FCM) [18], and over the years, Bezdek [19] has suggested an improved version of FCM; this algorithm being the soft centroid-based model that allows one piece of data to belong to two or more clusters. Due to the poor performance of FCM against noise and intensity inhomogeneity, many variants of FCM have been developed. These FCM variations are covered in [20]. FCM fails to discriminate between data points and noise or outliers, thus centroids could be stuck in the outliers instead of in the centers. To overcome this drawback, ref.[21] proposed a possibilistic C-means algorithm, where PCM is a distribution-based model which assumes that the observations of the learning set are the realizations of a random variable whose density function is a mixture of normal distributions [22] where the membership is the degree of belonging that illustrates how the datum is compatible with the class typicality. This makes it possible to improve performance when dealing with noise and outliers. Despite its capability, the PCM is sensitive to initializations and choices of parameters, it can generate coincident clusters, and also suffers from the degenerescence problem, i.e., when some group is empty, the standard deviation tends toward 0 [23]. To correct that, Pal et al. [24] combined FCM and PCM while considering the need for both relative and absolute typicalities in clustering. Another weakness of the possibilistic C-means is the negligence of membership values even with the observation of the typicality values. Further variants of the PCM method have been enhanced by changing the objective function to address some of the PCM’s shortcomings [25,26,27].

Furthermore, there are many attempts to benefit from the combination of FCM and the possibilistic approach to ameliorate the results, also recognizing the requirement for applying membership and typicality values simultaneously [24,28,29]. Once more, Pal et al. [30] presented a new impressive model named Possibilistic Fuzzy C-means (PFCM), which simultaneously created, for each cluster, a membership and typicity values with the standard prototypes or cluster centers. This algorithm has resolved the primary overlap and coincidence cluster issues that plagued PCM, in addition to removing noise sensitivity issues in FCM. Unfortunately, all these algorithms did not give satisfactory results, thus it was better to use a new approach [31] that enhanced the PFCM model to precisely detect the cluster centers. The authors introduced the Modified Possibilistic Fuzzy C-Means (MPFCM) with norm functions, including the covariance norm, thus making it superior and flexible at correcting the most complicated problems that other variants find challenging—mainly noise issues and outlier points.

Several other clustering methods have been created in order to prove the possibility that a data point might belong to multiple sub-clusters at once, and some of these algorithms were based on neutrosophic logic. In [32], a Neutrosophic C-means algorithm (NCM) was presented to derive such an approach by introducing a new objective function handling two different classes of rejection—the ambiguity rejection and the distance rejection—to correct the FCM method’s weakness in identifying noise and outlier data points. NCM simultaneously calculates the degrees of belonging to the determinate and indeterminate clusters for each of the data points. By using a new iterative process, Guo and Sengur [33] updated the membership degrees to the ambiguity and outlier class of data points, and as a result, the membership functions are more immune to noise. The study [34] suggested a new Kernel NCM clustering algorithm (KNCM) to improve the idea of the NCM approach for nonlinear-shaped data clustering by combining the kernel function with the neutrosophic logic. In addition, they produced a novel cost function to solve for noise and outliers using a robust parameter estimation, which led to generation of a new membership function and prototype update equations. Along the same lines, [35] proposed a new kernel-based fuzzy clustering method to deal with the issue of arbitrary cluster shapes. A novel utilization of the kernel method was proposed to extract more expressive features implicitly and may be used to reduce the struggle during fuzzy clustering of high dimensional data by discovering good new representations from the original data. The kernel approach in [36] has emerged as an interesting and quite viable alternative to fuzzy clustering. In addition, [37] proposed multiple kernel fuzzy clustering (MKFC), which expands the fuzzy c-means algorithm with the multiple kernel-learning setting. MKFC has more excellent resistance to useless features and kernels by employing multiple kernels and an automated adjustment of the kernel weights. A multiple kernel fuzzy c-means (MKFCM) approach was also presented by [38], who developed a novel algorithm that uses a linear composite of multiple kernels and automatically determines the updating of the linear coefficients of the combined kernel.

Fuzzy clustering plays an important role in the field of data mining. Since the membership function offers a powerful tool for the identification of changing class structures, ref.[39] has suggested a technique for dynamic data mining that focuses on fuzzy c-means. This approach appears to afford a convenient method for the detection of changing class structures. Unfortunately, it suffers from several weaknesses, and [40] proposed some adjustments that led to the creation of the modified dynamic fuzzy c-means (MDFCM) algorithm. This allows for more flexibility in membership function estimation and averts the use of extremely complicated equations [41]. In a recent paper, we developed a clustering model based on a recurrent neural network, an optimization model that treats various samples equally, and the Euler-Cauchy technique with a fixed time step [42]. However, outlier samples make it impossible to identify the true groups and cause a very long and incorrect convergence trajectory. We investigated, in [43], a constrained optimization method that de-assigns memberships from centers to reduce the impact of outlier samples. To benefit from the capacity of dynamic systems to memorize prior groupings and the understanding of the features of the data by neural networks, we have introduced, in our previous work [44], an original clustering method that implements fuzzy logic and a recurrent neural network, namely the Recurrent Neural Network Fuzzy C-means. Other recent versions of Fuzzy C-means were introduced. In [45], authors proposed a gradient descent algorithm based on possibilistic fuzzy c-means for clustering noisy data. In [46], authors suggested the Optimization of a Fuzzy C-Means Clustering Algorithm with a Combination of Minkowski and Chebyshev Distance Using Principal Component Analysis. Wang et al. introduced an improved index for clustering validation based on the Silhouette index and the Calinski–Harabasz index [47]. A hybrid fuzzy c-means clustering algorithm was suggested in [48] and was essentially focused on big data problems. The Deep Possibilistic C-means Clustering Algorithm was recently introduced by Gu Y et al. and was used on medical datasets [49].

In this work, we propose a new sub-measure that implements both notions (the degree of membership and the frequency of “pattern p belongs to A with degree of membership (A,p)”, event denoted with an E, while exploiting the ability of the convolution operator to combine functions on continuous intervals. This measure evaluates both the degree of membership and the frequency of E in the design of decisional systems. We show, using concrete examples, the disadvantages of fuzzy logic and probability-based approaches taken separately, and then we show how a convolution probabilistic measure allows the correction of these disadvantages. Based on this measure, we introduce a new clustering method named Fuzzy-Probabilistic-Convolution-C-Means (FP-Conv-CM). FCM, PKM, and FP-Conv-CM were comparatively tested on several datasets to analyze the clustering task and on several images to analyze the compression task. FCM, PKM, and FP-Conv-CM were compared based on several performance measures: Silhouette and Dunn’s indexes, mean square error, structural similarity (SSIM), and peak signal-to-noise ratio (PSNR). Considering the results obtained using FCM and PKM, FP-Conv-CM was able to improve the Silhouette and Dunn’s Indexes. In addition, FP-Conv-CM improved MSE, PSNR, and SSIM values.

The rest of the paper is organized as follows: In Section 2, we present the methodology adopted in this work to achieve the stated goals. In Section 3, we discuss the disadvantages of fuzzy and Probabilistic K-means using concrete data. In Section 4, the proposed fuzzy probability measure is described and used for clustering tasks. In Section 5, we present experimental results, and Section 6 concludes the paper.

2. Methodology Overview

Let $\mathcal{B}=\left\{z_1,\dots ,z_N\right\}\subset {ℝ}^n$ represent the space of unlabeled observations. The clustering methods seek to reduce the information contained in $\mathcal{B}$:

• By summing it up as a set $W=\left\{w_1,\dots ,w_k\right\}\subset {ℝ}^n$, where $k\ll N$; these vectors will be called referents (centers) throughout the rest of the article;
• By defining an assignment function, χ, which is an application of $\mathcal{B}$ in the set of indices $\left\{1, \dots , k\right\}$, this function makes it possible to realize a partition $P=\left\{G_1,\dots ,G_c,\dots ,G_k\right\}$ of $\mathcal{B}$ in $k$ subsets, $G_c=\left\{z\in \mathcal{B}/\chi \left(z\right)=c\right\}$.

It should be noted that the clustering problem is NP difficult. Indeed, if $k$ (the number of groups) and $d$ (the dimension), are fixed and $n$ is the number of items to be clustered, then the problem can be solved exactly in $\left(2^{nk+1}\right)$ time [50].

Definitions

• Data$=\left\{{\mathrm{z}}_1,\dots ,{\mathrm{z}}_{\mathrm{N}}\right\}\subset {ℝ}^{\mathrm{n}}$ is the dataset under study;
• Centers$=\left\{{\mathrm{w}}_1,\dots ,{\mathrm{w}}_{\mathrm{k}}\right\}\subset {ℝ}^{\mathrm{n}}$ is the set of the centers to be determined;
• $\mathsf{\chi }$ is the allocation function of data to the groups represented by the centers ${\mathrm{w}}_1,\dots ,{\mathrm{w}}_{\mathrm{k}}$;
• ${\mathrm{G}}_{\mathrm{j}}=\left\{\mathrm{z}\in \mathrm{Data}/\mathsf{\chi }\left(\mathrm{z}\right)=\arg \left({\mathrm{w}}_{\mathrm{j}}\right)\right\}$, $\forall \mathrm{j}=1,\dots ,\mathrm{k}$ are the groups determined based on $\mathsf{\chi }$;
• ${\mathsf{\mu }}_{\mathrm{j}}^{\mathrm{m}}\left(\mathrm{z}\right)=\frac{1}{{\sum }_{c=1}^k{\frac{\Vert \mathrm{z}-{\mathrm{w}}_{\mathrm{j}}\Vert }{\Vert \mathrm{z}-{\mathrm{w}}_{\mathrm{c}}\Vert }}^{2/\mathrm{m}-1}}$ is the membership function of group ${\mathrm{G}}_{\mathrm{j}}$ where m is a real number strictly greater than 1;
• ${\mathrm{p}}_{\mathrm{j}}\left(\mathrm{x}\right)=\frac{\exp \left(-\frac{1}{2}{\left(\mathrm{x}-{\mathrm{w}}_{\mathrm{j}}\right)}^{\prime }{\mathrm{cov}}_{\mathrm{j}}{}^{-1}\left(\mathrm{x}-{\mathrm{w}}_{\mathrm{j}}\right)\right)}{\sqrt{{\left(2\mathsf{\pi }\right)}^{\mathrm{n}}\det \left({\mathrm{cov}}_{\mathrm{j}}\right)}}$is the probability of x being in group ${\mathrm{G}}_{\mathrm{j}}$, where ${\mathrm{cov}}_{\mathrm{j}}$ is the covariance of the component $\mathrm{j}$.

The quantity ${\mathrm{p}}_j\left(\mathrm{x}\right)$ measures the frequency that $\mathrm{x}$ is taken from ${\mathrm{G}}_{\mathrm{j}}$; whereas ${\mathsf{\mu }}_j\left(\mathrm{x}\right)$ measures the degree of belonging of $\mathrm{x}$ to ${\mathrm{G}}_{\mathrm{j}}$. Our idea is to define a new borelian measure that measures the degree of belonging of $\mathrm{x}$ to ${\mathrm{G}}_{\mathrm{j}}$ with frequency${ \mathrm{p}}_j\left(\mathrm{x}\right)$.

Methods

In this work, we introduce a new measure that measures the degree of belonging of $\mathrm{x}$ to ${\mathrm{G}}_{\mathrm{j}}$ with frequency${ \mathrm{p}}_j\left(\mathrm{x}\right)$. This measure is called fuzzy probability convolution measure and implements the following membership-density functions:

For a continuous case:

$${\mathsf{\phi }}_{\mathrm{j}}^{\mathrm{m}}\left(\mathrm{z}\right)=\left({\mathsf{\mu }}_{\mathrm{j}}{\ast \mathrm{p}}_{\mathrm{j}}\right)\left(\mathrm{z}\right)=\begin{array}{c}\int {\mathsf{\mu }}_{\mathrm{j}}^{\mathrm{m}}\left(\mathsf{\tau }\right){\mathrm{p}}_{\mathrm{j}}\left(\mathrm{z}-\mathsf{\tau }\right)\mathrm{d}\mathsf{\tau }\\ {ℝ}^{\mathrm{n}}\end{array}$$

For a discrete case:

$${\mathsf{\phi }}_{\mathrm{j}}^{\mathrm{m}}\left(\mathrm{z}\right)=\left({\mathsf{\mu }}_{\mathrm{j}}{\ast \mathrm{p}}_{\mathrm{j}}\right)\left(\mathrm{z}\right)={\displaystyle \sum }_{\mathrm{i}=1}^{+\infty }{\mathsf{\mu }}_{\mathrm{j}}^{\mathrm{m}}\left({\mathsf{\tau }}_{\mathrm{i}}\right){\mathrm{p}}_{\mathrm{j}}\left(\mathrm{z}-{\mathsf{\tau }}_{\mathrm{i}}\right)$$

Based on the proposed fuzzy probability convolution measure, we introduce a new clustering method that we named Fuzzy Probabilistic Convolution C-Means (FP-Conv-CM). This method estimates the vectors ${\mathrm{w}}_{\mathrm{c}}$, the common standard deviation $\mathsf{\sigma }$, and the membership coefficients. Based on FP-Conv, the model implementing these parameters must approximate the true density that generated the data $\left\{{\mathrm{z}}_1,\dots ,{\mathrm{z}}_{\mathrm{N}}\right\}$. In this regard, our method involves maximizing the fuzzy probability of the observations:

$$\overline{\mathrm{V}}\left({\mathrm{w}}_1,\dots ,{\mathrm{w}}_{\mathrm{k}},\mathsf{\sigma },{\mathsf{\mu }}^{\mathrm{m}},\mathsf{\chi }\right)={\displaystyle \prod }_{\mathrm{i}=1}^{\mathrm{N}}\begin{array}{c}\int {\mathsf{\mu }}_{\mathsf{\chi }\left({\mathrm{z}}_{\mathrm{i}}\right)}^{\mathrm{m}}\left(\mathsf{\tau }\right){\mathrm{p}}_{\mathsf{\chi }\left({\mathrm{z}}_{\mathrm{i}}\right)}\left({\mathrm{z}}_{\mathrm{i}}-\mathsf{\tau }\right)\mathrm{d}\mathsf{\tau }\\ {ℝ}^{\mathrm{n}}\end{array}$$

where $\overline{\mathrm{V}}$ represents the likelihood measuring the capacity of the model to reproduce the data $\left\{{\mathrm{z}}_1,\dots ,{\mathrm{z}}_{\mathrm{N}}\right\}$, ${\mathsf{\mu }}^{\mathrm{m}}$ is the matrix of the membership coefficients (which represents the degree of membership of each data ${\mathrm{z}}_{\mathrm{i}}$ to each group $\mathrm{c}$), and $\mathsf{\chi }\left({\mathrm{z}}_{\mathrm{i}}\right)$ is the group that won the sample ${\mathrm{z}}_{\mathrm{i}}$ according to FP-Conv.

Metrics

One of the biggest challenges facing researchers is how to evaluate the clustering. The homogeneity of the created classes/clusters and the separation between them are calculated in order to assess the efficiency of the clustering method. These qualities may be evaluated using a variety of indices:

• Silhouette index:
• Let N be the number of patterns. The Silhouette index [43] finds the optimal clustering effect using the difference between the average distance within the cluster and the minimum distance between the clusters, i.e., the silhouette coefficient is given as follows:
• $S\left(i\right)=\frac{1}{N}{\sum }_{i=1}^N \left(\frac{b\left(i\right)-a\left(i\right)}{max\left\{a\left(i\right),b\left(i\right)\right\}}\right)$
• $a\left(i\right)$ represents the average distance from sample $i$ to other samples in the cluster;
• $b\left(i\right)$ represents the minimum distance from sample $i$ to the other clusters.
• The silhouette coefficient ranges from −1 to 1, where −1 denotes that the data point is not assigned to the relevant cluster, 0 denotes that the clusters are overlapping, and 1 denotes that the cluster is dense and well-separated. This metric is one of the most popular measurements in clustering. It can distinguish between objects that were placed wisely within their cluster and those that are in the outlier zone between clusters.
• Dunn index: The Dunn index [40,44] is defined as
• $DI=\frac{d_{\min }}{d_{\max }}$

If $c_i$ and $c_j$ are different clusters then $d_{\min }={\displaystyle \underset{i\ne j}{\min }}\{{\displaystyle \underset{\forall x_k\in c_i,\forall x_{k^{\prime }}\in c_j}{\min }}\Vert x_k^{\left(i\right)}-x_{k^{{}^{\prime }}}^{\left(j\right)}{\Vert }_2\}$ is the minimal distance between samples in different clusters and $d_{\max }=\underset{r=1,\dots ,c}{\max }\{\underset{k\ne k\prime }{\max }\Vert x_k^{\left(r\right)}-x_{k^{\prime }}^{\left(r\right)}{\Vert }_2\}$ is the largest within-cluster distance. Note that large inter-cluster distances (better separation) and smaller cluster sizes (more compact clusters) lead to higher DI values. A higher DI value implies better clustering [40]. It assumes that better clustering means that clusters are compact and well-separated from other clusters.

The performance measures used to evaluate different image compressions are:

(a)

Mean Squared Error (MSE), which calculates the error between the initial image and the compressed image. It is in fact the distance between two matrices that represent the images to be compared.

(b)

Peak Signal-to-Noise Ratio (PSNR), which implements the following equation:

$$PSNR=10log10\left(peakval/MSE\right)$$

where peakval is taken from the range of the image datatype.

(c)

Structural SIMilarity (SSIM) index, which is calculated on various windows of an image. The measure between two windows x and y of common size $a\times a$ is: $SSIM\left(x,y\right)=\frac{\left(2\overline{x}\overline{y}+a_1\right)\left(2cov\left(x,y\right)+a_2\right)}{\left({\overline{x}}^2+{\overline{y}}^2+a_1\right)\left(var{\left(x\right)}^2+var{\left(y\right)}^2+a_2\right)}$

Where $mean\left(x\right)=\overline{x}$, $var\left(x\right)$ is the variance of $x$, and $cov\left(x,y\right)$ is the covariance of $x$ and $y$; $a_1$ and $a_2$ are two constants estimated from the image [51].

Experimental validation

FCM, PKM, and FP-Conv-CM were tested on multiple datasets and compared on the basis of two performance measures, i.e., the Silhouette metric and Dunn’s Index (see Section 5.2). FP-Conv-CM was able to improve the Silhouette value by 1000 and Dunn’s Index by 0.024.

FCM, PKM, and FP-Conv-CM were used for multiple image compression tasks and were compared based on three performance measures: mean squared error (MSE), peak signal-to-noise ratio (PSNR), and structural similarity index (SSIM) (see Section 5.3). FP-Conv-CM improved MSE by 3000, PSNR by 11, and SSIM by 0.32.

3. Drawbacks of Fuzzy and Probabilistic Approaches

3.1. K-Means

The K-means method, which is the most well-known vector quantization method, determines the set of reference vectors $W$ and the assignment function $\chi$, by minimizing the cost function:

$$J\left(W,\chi \right)={\displaystyle \sum }_{z_i\in \mathcal{B}}\Vert z_i-w_{\chi \left(z_i\right)}{\Vert }^2$$

The time complexity of K-means is $\mathrm{O}\left(N x k x n x i\right)$ [52], where $i$ is the number of iterations needed until convergence. The k-means method suffers from several drawbacks due to it being a hard clustering method. To overcome these issues, there exist many extensions, including FCM and PKM [43].

3.2. Probabilistic K-Means

To obtain the probabilistic version of k-means, it is assumed that the observations of the learning set $B$ are the realizations of a random variable whose density function is a mixture of $k$ normal distributions [44,53]:

$$p\left(z\right)={\displaystyle \sum }_{c=1}^k{\alpha }_c{f}_c\left(z\right)$$

where ${\sum }_{c=1}^k{\alpha }_c=1$; and $f_c$ is the normal density function.

In addition to this formalism, the shift to the probabilistic interpretation of the K-means algorithm requires us to introduce additional assumptions:

• The prior probabilities ${\alpha }_c$ are all equal to $1/k$;
• The k normal functions $f_c$ have identical variance–covariance matrices equal to ${\sigma }^{2}I$, where $I$ represents the unit matrix and $\sigma$ is the standard deviation considered constant for all these normal distributions.
• In that case, the density function has the following expression:
• $f_c\left(z\right)=\frac{1}{{\left(2\pi \right)}^{\frac{n}{2}}{\sigma }^n}\exp \left(-\frac{\Vert z-w_c{\Vert }^2}{2{\sigma }^2}\right)$
• The k-means probabilistic version involves estimating the vectors $w_c$ and the typical standard deviation $\sigma$ trying to realize the sample as much as possible. This method, known as the maximum likelihood method, involves maximizing the probability $p \left(z_1, \dots , z_N\right)$ of these observations.
• Maximizing the classifying likelihood amounts to minimizing:
• $V\left(W,\sigma ,\chi \right)=\frac{1}{2{\sigma }^2}J\left(W,\chi \right)+N\times n\times log\left(\sigma \right)+CONST$
• Probabilistic k-means has a running time of $O\left(N\times k\times n\times ITER\right)$ [52], where $N$ is the number of $n$-dimensional vectors, $k$ is the number of clusters, and $ITER$ is the number of iterations required to reach convergence; CONST is a constant that depends only on the data.

3.3. Fuzzy C-Means (FCM)

The fuzzy mean square clustering algorithm known as Fuzzy C-means (FCM) allows one sample of data to belong to each cluster with different degrees of membership. This method is frequently used in pattern recognition [42]. The objective function of the FCM method is minimizing the following error:

$$J_m=\sum _{i=1}^N \sum _{j=1}^k u_{ij}^m{‖x_i-w_j‖}^2,1\le m<\infty$$

where the real value $m$ determines the fuzziness of the generated clusters (m > 1), $N$ is the dataset size, $k$is the number of clusters, $u_{ij}^m$ is the degree of membership of sample $x_i$ to the cluster $j$, $w_j$is the $n$-dimensional center of the cluster $j$, and ‖.‖ represents any norm denoting the similarity between each measured data and the center.

An iterative optimization of the objective function presented above is used to perform a fuzzy partitioning, with the update of membership $u_{ij}^m$ and the cluster centers $w_j$ by:

$$u_{ij}^m=\frac{1}{{\sum }_{c=1}^k {\left(\frac{‖x_i-w_j‖}{‖x_i-w_c‖}\right)}^{\frac{2}{m-1}}};c_j=\frac{{\sum }_{i=1}^N u_{ij}^m.x_i}{{\sum }_{i=1}^N u_{ij}^m}$$

This iteration will stop when $\underset{ij}{\max }\left\{\left|u_{ij}^{\mathrm{m},\left(iter+1\right)}-u_{ij}^{\mathrm{m},\left(iter\right)}\right|\right\}<\epsilon$, where $\epsilon$ is a termination criterion between 0 and 1, whereas $iter$ are the iteration steps. This procedure converges to a local minimum or a saddle point of $J_m$.

3.4. Fuzzy Reasoning and Probabilistic Reasoning Are Complementary

Let $E$ be a set from ${ℝ}^n$. The probability$, P_E\left(x\right)$, measures the probability that $x$ is taken from $E$, whereas ${\mu }_E\left(x\right)$measures the degree of belonging of $x$ to $E$.

Probabilistic reasoning corrects the weakness in fuzzy reasoning:

Consider a dataset whose elements are distributed according to two normal distributions, $N_1\left(c_1=-0.75, {\sigma }_1=0.25\right)$ and $N_2\left(c_2=0.25, {\sigma }_2=0.75\right)$, presented in Figure 1.

These two distributions form two clusters (or classes), namely C₁ and C₂. In this subsection, we will use probabilistic and fuzzy models to recognize the classes of some critical data. When we generated the data, we noticed that some samples have a very high frequency and are not close enough to the centers of the two densities. As we will see later, depending on the case study, this type of data misleads both models.

Example 1.

We consider the element $s=-0.26$, of high frequency, from class (or cluster) C₂.

(a)

The fuzzy model based on the membership functions ${\mathsf{\mu }}_1$ and ${\mathsf{\mu }}_2$, presented in Figure 1, predicts $\mathrm{s}$ as an element of C₁.

$${\mu }_1\left(x\right)=\frac{\frac{1}{{\left|x+0.75\right|}^2}}{\frac{1}{{\left|x+0.75\right|}^2}+\frac{1}{{\left|x-0.25\right|}^2}} and {\mu }_2\left(x\right)=\frac{\frac{1}{{\left|x-0.25\right|}^2}}{\frac{1}{{\left|x+0.75\right|}^2}+\frac{1}{{\left|x-0.25\right|}^2}}$$

In fact, we have ${\left|\mathrm{s}+0.75\right|}^2=0.2401$ and ${\left|\mathrm{s}-0.25\right|}^2=0.2601,$ then ${\mathsf{\mu }}_1\left(\mathrm{s}\right)=\frac{\frac{1}{{\left|\mathrm{s}+0.75\right|}^2}}{\frac{1}{{\left|\mathrm{s}+0.75\right|}^2}+\frac{1}{{\left|\mathrm{s}-0.25\right|}^2}}=0.52$and ${\mathsf{\mu }}_2\left(\mathrm{s}\right)=\frac{\frac{1}{{\left|\mathrm{s}-0.25\right|}^2}}{\frac{1}{{\left|\mathrm{s}+0.75\right|}^2}+\frac{1}{{\left|\mathrm{s}-0.25\right|}^2}}=0.48,$thus, the fuzzy model predicts that the sample $s=-0.26$ is from class 1 with center ${\mathrm{c}}_1=-0.75$, which is false.

(b)

The probabilistic model based on the densities $p_1$ and $p_2$, presented in Figure 1, predicts $s$ as an element of C₂

$$p_1\left(x\right)=\frac{1}{0.25\sqrt{2\pi }}exp(-\frac{{\left|x+0.75\right|}^2}{2\times 0.0625} ) and p_2\left(x\right)=\frac{1}{0.75\sqrt{2\pi }}exp(-\frac{{\left|x-0.25\right|}^2}{2\times 0.5625})$$

In fact, we have ${\mathrm{p}}_1\left(\mathrm{s}\right)=\frac{1}{0.25\sqrt{2\mathsf{\pi }}}\exp \left(-\frac{{\left|\mathrm{s}+0.75\right|}^2}{2\times 0.0625}\right)=\frac{1}{0.25\sqrt{2\mathsf{\pi }}}\exp \left(-\frac{{0.49}^2}{2\times 0.0625}\right)\approx 0.2338$ and ${\mathrm{p}}_2\left(\mathrm{s}\right)=\frac{1}{0.75\sqrt{2\mathsf{\pi }}}\exp \left(-\frac{{\left|\mathrm{s}-0.25\right|}^2}{2\times 0.5625}\right)\approx \frac{1}{0.75\sqrt{2\mathsf{\pi }}}\exp \left(-\frac{{0.51}^2}{1.12}\right)\approx 0.4221$.

Thus, the probability model predicts that s$=-0.26$ is from class 2 with center ${\mathrm{c}}_2=0.25$, which is true.

Fuzzy reasoning corrects the weakness in probabilistic reasoning:

Consider a dataset whose elements are distributed according to two normal distributions, $N_1\left({\mathrm{c}}_1=0,{\mathsf{\sigma }}_1=\sqrt{5}\right)$ and $N_2\left({\mathrm{c}}_2=1,{\mathsf{\sigma }}_2=1\right)$. These two distributions form two clusters (or classes), namely C₁ and C₂. In this subsection, we will use probabilistic and fuzzy models to recognize the classes of some critical data. When we generated the data, we noticed that some samples are close to the mean of the two densities and do not have a very high frequency. As we will see later, this perturbs the decision of the models.

Example 2.

We consider the element $s=-0.2$, close to the center 0, of the class (or cluster) C₁.

(a)

The fuzzy model based on the membership functions ${\mathsf{\mu }}_1$ and ${\mathsf{\mu }}_2$ given below predicts $\mathrm{s}$ as an element of C₁.

$${\mathsf{\mu }}_1\left(\mathrm{x}\right)=\frac{\frac{1}{{\mathrm{x}}^2}}{\frac{1}{{\mathrm{x}}^2}+\frac{1}{{\left|\mathrm{x}-1\right|}^2}} \mathrm{and}{ \mathsf{\mu }}_2\left(\mathrm{x}\right)=\frac{\frac{1}{{\left|\mathrm{x}-1\right|}^2}}{\frac{1}{{\mathrm{x}}^2}+\frac{1}{{\left|\mathrm{x}-1\right|}^2}}$$

In fact, we have ${\mathsf{\mu }}_1\left(s\right)=\frac{\frac{1}{s^2}}{\frac{1}{s^2}+\frac{1}{{\left|s-1\right|}^2}}=0.9730$ and ${\mathsf{\mu }}_2\left(s\right)=\frac{\frac{1}{{\left|s-1\right|}^2}}{\frac{1}{s^2}+\frac{1}{{\left|s-1\right|}^2}}=0.0270$, thus, the fuzzy logic predicts that the sample $s=-0.2$ is from class 1 with center $c_1=0$, which is true.

(b)

The probabilistic model based on the membership functions ${\mathsf{\mu }}_1$ and ${\mathsf{\mu }}_2$ given below predicts $\mathrm{s}$ as an element of C₂.

In fact, we have ${\mathrm{p}}_1\left(s\right)=\frac{1}{\sqrt{10\pi }}\exp \left(-\frac{s^2}{10}\right)\approx 0.1777$and ${\mathrm{p}}_2\left(s\right)=\frac{1}{\sqrt{2\pi }}\exp \left(-\frac{{\left|s-1\right|}^2}{2}\right)\approx 0.1942$.

Thus, the probability approach predicts that $s=-0.2$ is from class 2 of center ${\mathrm{c}}_2=1$, which is false.

Drawbacks of fuzzy and probabilistic approaches on real data: image compression case study:

To show the limits of fuzzy and probabilistic approaches on real data, we have used these methods to compress images and we have conducted a deep analysis of the obtained results. For this, we have used the image of the great scientist Max Planck (see Figure 2a). The histogram of this image is given in Figure 2b, while Figure 2c highlights an ambiguous pixel.

The image obtained after decompression is shown in Figure 3a (after compression using 2-GMM). The histograms of group 1 (center = 58.95, std = 15.98) and group 2 (center = 58.95, std = 15.98), obtained using 2-GMM, are given in Figure 3b,c, respectively.

The image obtained after decompression is shown in Figure 4a (following compression using 2-FCM). The histograms of group 1 (center 1 = 63.75) and group 2 (center 2 = 186.53), obtained using 2-FCM, are given in Figure 4b,c, respectively.

In the first phase of this example, we focus on the pixels with a gray level 125, which represent 29.05% of the image; for example, the pixel located at position [30 264], which is called p. For pedagogical reasons, we have highlighted this pixel on the different histograms and the different images of Max Planck—original and those obtained by decompression.

This kind of pixel causes ambiguities for 2-FCM, but 2-GMM assigns them correctly thanks to amplification of the small quantities (thanks to the averages and standard deviations) by the exponential function of each Gaussian component.

Indeed, the degrees of membership of p to each of these groups, formed by 2-FCM, are given by ${\mu }_1\left(p\right)=0.5022$ et ${\mu }_2\left(p\right)=0.4978$; thus ${\mu }_1\left(p\right)\approx {\mu }_2\left(p\right)$. The probabilities of p belonging to each of the two groups formed using 2-GMM are given by $P_1\left(p\right)=0.0024$ and $P_2\left(p\right)=0.9976$; hence, the clarity of the probabilistic decision with respect to the suitable group of the pixel p.

In the second phase of this example, we focus on the pixels with a gray level of 92, which represent 45.8% of the image; for example, the pixel located at position [198 279], which is called q. This kind of pixel causes ambiguities for 2-GMM, but 2-FCM assigns them correctly thanks to the integration of all centers in the formulas of all membership functions.

Indeed, the probabilities of q belonging to each of the two groups are $P_1\left(q\right)=0.5126$ et $P_2\left(q\right)=0.4873$; thus $P_1\left(q\right)\approx P_2\left(q\right)$. The degree of membership of q in each of these groups formed by 2-FCM are given by ${\mu }_1\left(q\right)=0.9179$ and ${\mu }_2\left(q\right)=0.0820$; hence the clarity of the fuzzy decision with respect to the suitable group of pixel q.

In the next section, in order to overcome these shortcomings, we will introduce a hybrid sub-measure that implements both fuzzy and probabilistic concepts.

4. Proposed Approach

4.1. Fuzzy Probability Convolution Measure

Our idea is to define a new hybrid measure that implements the fuzzy and the probability reasonings at the same time and corrects the shortcoming of the classical reasoning. Thus, this new measure computes the frequency of taking $x$ from $E$ and the degree of belonging of $x$to $E$ at the same time; we call this measure fuzzy probability convolution.

Definition 1.

(Convolution-Fuzzy-probability): The Fuzzy Probability Convolution (FP-Conv) measure is defined based on the convolution of the membership function ${\mu }_E$ and on the density (probability) $p_E$ as given by:

For a continuous case:

$${\phi }_E\left(z\right)=\left({\mu }_E\ast p_E\right)\left(z\right)=\underset{-\infty }{\overset{+\infty }{{\displaystyle \iiint }}}{\mu }_E\left(\tau \right)p_E\left(z-\tau \right)d\tau$$

For a discrete case:

$${\phi }_E\left(z\right)=\left({\mu }_E\ast p_E\right)\left(z\right)={\displaystyle \sum }_{i=1}^{+\infty }{\mu }_E\left({\tau }_i\right)p_E\left(z-{\tau }_i\right)d\tau$$

The fuzzy probability convolution corrects the incorrect decisions of the fuzzy logic.

Consider the following membership functions:

$${\mu }_1\left(x\right)=\frac{\frac{1}{{\left|x+0.75\right|}^2}}{\frac{1}{{\left|x+0.75\right|}^2}+\frac{1}{{\left|x-0.25\right|}^2}} \mathrm{and} {\mu }_2\left(x\right)=\frac{\frac{1}{{\left|x-0.25\right|}^2}}{\frac{1}{{\left|x+0.75\right|}^2}+\frac{1}{{\left|x-0.25\right|}^2}}$$

and the following normal density functions:

$$p_1\left(x\right)=\frac{1}{0.25\sqrt{2\pi }}\exp \left(-\frac{{\left|x+0.75\right|}^2}{2\times 0.0625}\right) \mathrm{and} p_2\left(x\right)=\frac{1}{0.75\sqrt{2\pi }}\exp \left(-\frac{{\left|x-0.25\right|}^2}{2\times 0.5625}\right).$$

Proposition 1.

Based on the decision model implementing the following FP-Conv densities:

$${\phi }_1\left(z\right)=\left({\mu }_1\ast p_1\right)\left(z\right)={{\displaystyle \int }}_{-\infty }^{+\infty }{\mu }_1\left(\tau \right)p_1\left(z-\tau \right)d\tau$$

and ${\phi }_2\left(z\right)=\left({\mu }_2\ast p_2\right)\left(z\right)={\int }_{-\infty }^{+\infty }{\mu }_2\left(\tau \right)p_2\left(z-\tau \right)d\tau$, the sample $s=-0.26$is predicted as the element of class 2, which is true.

Proof. 

We have${\phi }_1\left(s\right)=\left({\mu }_1\ast p_1\right)\left(s\right)={\int }_{-\infty }^{+\infty }{\mu }_1\left(x\right)p_1\left(s-x\right)dt\approx$ 0.055585, and ${\phi }_2\left(s\right)=\left({\mu }_2\ast p_2\right)\left(s\right)={\int }_{-\infty }^{+\infty }{\mu }_2\left(x\right)p_2\left(s-x\right)dt\approx 0.393632$; thus, ${\phi }_1\left(s\right)<{\phi }_2\left(s\right)$.

Thus, $s$ is considered as an element of the class with center $c_2=0.25$, which is true.

Note: 

It should be noted that$p_1\left(s\right)\approx 1.8054 p_1\left(s\right)$and ${\phi }_1\left(s\right)\approx 7.0816 {\phi }_2\left(s\right)$, which means that the decision is clearer. □

The convolution fuzzy probability corrects the incorrect decisions of the probability reasoning.

Consider the membership functions ${\mu }_1\left(x\right)=\frac{\frac{1}{x^2}}{\frac{1}{x^2}+\frac{1}{{\left|x-1\right|}^2}}$and${\mu }_2\left(x\right)=\frac{\frac{1}{{\left|x-1\right|}^2}}{\frac{1}{x^2}+\frac{1}{{\left|x-1\right|}^2}}$, and the normal densities functions $p_1\left(x\right)=\frac{1}{\sqrt{10\pi }}\exp (-\frac{x^2}{10})$ and $p_2\left(x\right)=\frac{1}{\sqrt{2\pi }}\exp (-\frac{{\left|x-1\right|}^2}{2})$.

Proposition 2.

Based on the fuzzy density measures ${\phi }_1\left(z\right)=\left({\mu }_1\ast p_1\right)\left(z\right)={\int }_{-\infty }^{+\infty }{\mu }_1\left(\tau \right)p_1\left(z-\tau \right)d\tau$and ${\phi }_2\left(z\right)=\left({\mu }_2\ast p_2\right)\left(z\right)={\int }_{-\infty }^{+\infty }{\mu }_2\left(\tau \right)p_2\left(z-\tau \right)d\tau$, the sample $s=-0.2$is predicted as an element of class 1, which is true.

Proof. 

We have ${\phi }_1\left(s\right)=\left({\mu }_1\ast p_1\right)\left(s\right)={\int }_{-\infty }^{+\infty }{\mu }_1\left(t\right)p_1\left(s-t\right)dt\approx 1.14554$ and${\phi }_2\left(s\right)=\left({\mu }_2\ast p_2\right)\left(s\right)={\int }_{-\infty }^{+\infty }{\mu }_2\left(x\right)p_2\left(s-x\right)dt\approx 0.246882$; thus ${\phi }_1\left(s\right)>{\phi }_2\left(s\right)$.

Thus, $s=-0.2$ is predicted as an element of the class with center $c_1=0$, which is true.

Note: 

It should be noted that ${\mu }_1\left(s\right)\approx 36.0370{\mu }_1\left(s\right)$and ${\phi }_1\left(s\right)\approx 4.64{\phi }_2\left(s\right)$. □

4.2. Fuzzy Probabilistic Convolution C-Means

The quantity ${\mathrm{P}}_j\left(\mathrm{x}\right)$ measures the frequency that $\mathrm{x}$ is taken from ${\mathrm{G}}_{\mathrm{j}}$; whereas, ${\mathsf{\mu }}_j\left(\mathrm{x}\right)$measures the degree of belonging of $\mathrm{x}$ to ${\mathrm{G}}_{\mathrm{j}}$. Our idea is to define a new borelian measure that measures the degree of belonging of $\mathrm{x}$ to ${\mathrm{G}}_{\mathrm{j}}$ with frequency${ \mathrm{P}}_j\left(\mathrm{x}\right)$. This new measure calculates the frequency of taking $\mathrm{x}$ from ${\mathrm{G}}_{\mathrm{j}}$ and the degree of belonging of $\mathrm{x}$ to ${\mathrm{G}}_{\mathrm{j}}$ at the same time.

Based on the concepts introduced in Section 2, the fuzzy probability convolution clustering method implements the following membership density functions:

Continuous case:

$${\mathsf{\phi }}_{\mathrm{j}}^{\mathrm{m}}\left(\mathrm{z}\right)=\left({\mathsf{\mu }}_{\mathrm{j}}{\ast \mathrm{p}}_{\mathrm{j}}\right)\left(\mathrm{z}\right)=\begin{array}{c}\int {\mathsf{\mu }}_{\mathrm{j}}^{\mathrm{m}}\left(\mathsf{\tau }\right){\mathrm{p}}_{\mathrm{j}}\left(\mathrm{z}-\mathsf{\tau }\right)\mathrm{d}\mathsf{\tau }\\ {ℝ}^{\mathrm{n}}\end{array}$$

Discrete case:

$${\mathsf{\phi }}_{\mathrm{j}}^{\mathrm{m}}\left(\mathrm{z}\right)=\left({\mathsf{\mu }}_{\mathrm{j}}{\ast \mathrm{p}}_{\mathrm{j}}\right)\left(\mathrm{z}\right)={\displaystyle \sum }_{\mathrm{i}=1}^{+\infty }{\mathsf{\mu }}_{\mathrm{j}}^{\mathrm{m}}\left({\mathsf{\tau }}_{\mathrm{i}}\right){\mathrm{p}}_{\mathrm{j}}\left(\mathrm{z}-{\mathsf{\tau }}_{\mathrm{i}}\right)$$

The following example shows, geometrically, how the new membership density function makes the decision increasingly clearer and easier.

Example 3.

Let us consider that $\mathrm{Data}=\left\{{\mathrm{z}}_1=-0.5,{ \mathrm{z}}_2=0.4,{ \mathrm{z}}_3=0.6\right\}$, $\mathrm{Centers}=\left\{{\mathrm{w}}_1=0,{ \mathrm{w}}_2=0.5\right\}$,

The samples allocated to each group: ${\mathrm{G}}_1=\left\{{\mathrm{z}}_1=-0.5\right\}$and${ \mathrm{G}}_2=\left\{{\mathrm{z}}_2=0.4,{ \mathrm{z}}_3=0.6\right\}$,

We set $\mathrm{m}=2$ ; then,

${\mathsf{\mu }}_1^2\left(\mathrm{x}\right)=\frac{1}{1+\frac{{\mathrm{x}}^2}{{\left(\mathrm{x}-0.5\right)}^2}}=\frac{{\left(\mathrm{x}-0.5\right)}^2}{{\left(\mathrm{x}-0.5\right)}^2+{\mathrm{x}}^2}$and ${\mathsf{\mu }}_2^2\left(\mathrm{x}\right)=\frac{1}{1+\frac{{\left(\mathrm{x}-0.5\right)}^2}{{\mathrm{x}}^2}}=\frac{{\mathrm{x}}^2}{{\left(\mathrm{x}-0.5\right)}^2+{\mathrm{x}}^2}$ (see Figure 2) and set $\mathsf{\sigma }=0.2$ then, ${\mathrm{p}}_1\left(\mathrm{x}\right)=\frac{1}{0.4\sqrt{2\mathsf{\pi }}}\exp \left(-\frac{{\mathrm{x}}^2}{0.32}\right)$ and ${\mathrm{p}}_2\left(\mathrm{x}\right)=\frac{1}{0.4\sqrt{2\mathsf{\pi }}}\exp \left(-\frac{{\left(\mathrm{x}-0.5\right)}^2}{0.32}\right)$ (see Figure 2).

Therefore, ${\mathsf{\phi }}_1^2 \left(\mathrm{z}\right)=\frac{1}{0.4\sqrt{2\mathsf{\pi }}} \int \frac{{\left(\mathsf{\tau }-0.5\right)}^2}{{\left(\mathsf{\tau }-0.5\right)}^2+{\mathsf{\tau }}^2}\exp \left(-\frac{{\left(\mathrm{z}-\mathsf{\tau }\right)}^2}{0.32}\right)\mathrm{d}\mathsf{\tau }$ and

${\mathsf{\phi }}_2^2 \left(\mathrm{z}\right)=\frac{1}{0.4\sqrt{2\mathsf{\pi }}}\int \frac{{\mathsf{\tau }}^2}{{\left(\mathsf{\tau }-0.5\right)}^2+{\mathsf{\tau }}^2}\exp \left(-\frac{{\left(\mathrm{z}-\mathsf{\tau }-0.5\right)}^2}{0.32}\right)\mathrm{d}\mathsf{\tau }$.

Figure 5 represents ${\mathsf{\mu }}_1^2$, ${\mathsf{\mu }}_2^2$, ${\mathrm{p}}_1$, ${\mathrm{p}}_2$, ${\mathsf{\phi }}_1^2$, and ${\mathsf{\phi }}_2^2$. The curves of the first four functions clearly underline the source of the ambiguity in the probabilistic and fuzzy models: the curves representing the two groups are very close to each other and the data located on the edges have almost the same probabilities of belonging to different groups. The introduced measure created a very safe separation zone and made a very large difference between the degrees of membership of the data at the edge to the different groups. This will enable the FCM-Conv clustering method to make decisions very comfortably.

4.3. Fuzzy Probability Convolution for the Clustering Task

Following the same principle of probabilistic models presented in Section 2, the proposed clustering method, named Fuzzy Probabilistic Convolution C-Means (FP-Conv-CM), involves estimating the vectors ${\mathrm{w}}_{\mathrm{c}}$, the common standard deviation $\mathsf{\sigma }$, and the membership coefficients, trying to make the realization of the sample of as much as possible in the sense of FP-Conv measure. In this sense, our method involves maximizing the fuzzy probability of the observations:

$$\begin{array}{c}\overline{\mathrm{V}}\left({\mathrm{w}}_1,\dots ,{\mathrm{w}}_{\mathrm{k}},\mathsf{\sigma },{\mathsf{\mu }}^{\mathrm{m}},\mathsf{\chi }\right)={\overline{\mathsf{\mu }\mathrm{p}}}^{\mathrm{m}}\left({\mathrm{z}}_1,\dots ,{\mathrm{z}}_{\mathrm{N}}/\mathsf{\chi }\right)\hfill \\ =\left({\overline{\mathsf{\mu }\mathrm{p}}}_{\mathsf{\chi }\left({\mathrm{z}}_1\right)}^{\mathrm{m}}\otimes {\overline{\mathsf{\mu }\mathrm{p}}}_{\mathsf{\chi }\left({\mathrm{z}}_2\right)}^{\mathrm{m}}\otimes \dots \otimes {\overline{\mathsf{\mu }\mathrm{p}}}_{\mathsf{\chi }\left({\mathrm{z}}_{\mathrm{N}}\right)}^{\mathrm{m}}\right)\left({\mathrm{z}}_1,\dots ,{\mathrm{z}}_{\mathrm{N}}\right)\hfill \\ \stackrel{\mathrm{def}}{=}{\displaystyle \prod }_{\mathrm{i}=1}^{\mathrm{N}}{\mathsf{\phi }}_{\mathsf{\chi }\left({\mathrm{z}}_{\mathrm{i}}\right)}^m\left({\mathrm{z}}_{\mathrm{i}}\right)\hfill \\ ={\displaystyle \prod }_{\mathrm{i}=1}^{\mathrm{N}}\begin{array}{c}\int {\mathsf{\mu }}_{\mathsf{\chi }\left({\mathrm{z}}_{\mathrm{i}}\right)}^{\mathrm{m}}\left(\mathsf{\tau }\right){\mathrm{p}}_{\mathsf{\chi }\left({\mathrm{z}}_{\mathrm{i}}\right)}\left({\mathrm{z}}_{\mathrm{i}}-\mathsf{\tau }\right)\mathrm{d}\mathsf{\tau }\\ {ℝ}^n\end{array}\hfill \end{array}$$

The parameters ${\mathrm{w}}_1,\dots ,{\mathrm{w}}_{\mathrm{k}},\mathsf{\sigma },{\mathsf{\mu }}^{\mathrm{m}},\mathrm{and} \mathsf{\chi }$ must be chosen such that the likelihood $\overline{\mathrm{V}}$ is maximum, meaning that the log-likelihood $\mathrm{L}\left({\mathrm{w}}_1,\dots ,{\mathrm{w}}_{\mathrm{k}},\mathsf{\sigma },{\mathsf{\mu }}^{\mathrm{m}},\mathsf{\chi }\right)=\ln \left(\overline{\mathrm{V}}\right)=\sum _{\mathrm{i}=1}^{\mathrm{N}}{\mathsf{\phi }}_{\mathsf{\chi }\left({\mathrm{z}}_{\mathrm{i}}\right)}\left({\mathrm{z}}_{\mathrm{i}}\right)$ is maximum.

To maximize $\mathrm{L}$, we use the partial gradient descent. We present the sample ${\mathrm{z}}_{\mathrm{i}}$to the fuzzy probability system, and then we set ${\mathrm{g}}_{\mathrm{i}}=\underset{\mathrm{c}=1,\dots ,\mathrm{k}}{\mathrm{argmax}}{\mathsf{\phi }}_{\mathrm{c}}\left({\mathrm{z}}_{\mathrm{i}}\right)$ $\left(the group containing that sample i\right)$.

At the time $\mathrm{t}$, we propose that ${\mathrm{w}}_1^{\mathrm{t}},\dots ,{\mathrm{w}}_{\mathrm{k}}^{\mathrm{t}},{\mathsf{\sigma }}_{\mathrm{t}}\left(tandard deviation\right),$ ${\mathrm{and} \mathsf{\chi }}^t\left(\mathrm{allocation} \mathrm{function}\right)$are known. In addition, thanks to the FP-Conv-CM, we propose that ${\mathsf{\phi }}_{{\mathrm{g}}_{\mathrm{i}}}\left({\mathrm{z}}_{\mathrm{i}}\right)\gg {\mathsf{\phi }}_{\mathrm{c}}\left({\mathrm{z}}_{\mathrm{i}}\right); \forall \mathrm{c}\ne {\mathrm{g}}_{\mathrm{i}}$, then we update ${\mathrm{w}}_{{\mathrm{g}}_{\mathrm{i}}}$ via the gradient descent method represented by the following equation:

$${\mathrm{w}}_{{\mathrm{g}}_{\mathrm{i}}}^{\mathrm{t}+1}={\mathrm{w}}_{{\mathrm{g}}_{\mathrm{i}}}^{\mathrm{t}}+{\mathrm{s}}_{\mathrm{t}}{\nabla }_{{\mathrm{w}}_{{\mathrm{g}}_{\mathrm{i}}}}{\mathsf{\phi }}_{{\mathrm{g}}_{\mathrm{i}}}\left({\mathrm{z}}_{\mathrm{i}}\right){/}_{{\mathrm{w}}_{{\mathrm{g}}_{\mathrm{i}}}={\mathrm{w}}_{{\mathrm{g}}_{\mathrm{i}}}^{\mathrm{t}}}$$

Theorem 1.

We have ${\nabla }_{w_{g_i}}{\phi }_{g_i}\left(z_i\right){/}_{w_{g_i}=w_{g_i}^t}=\left({\theta }_t^i\ast p_{g_i,t}\right)\left(z_i\right)+\left({\mu }_{g_i,t}^m\ast {\rho }_t^i\right)\left(z_i\right)$such that ${\theta }_t^i$ and ${\rho }_t^i$ are the gradients of ${\mu }_{g_i,t}^m$ and $p_{g_i,t}$, respectively, and ${\mathrm{s}}_{\mathrm{t}}$ is the step of the algorithm following the current direction at time t.

Proof. 

We set ${\mathrm{A}}_{\mathrm{i},\mathrm{t}}=\sum _{\mathrm{c}\ne {\mathrm{g}}_{\mathrm{i}}}1/\Vert \mathsf{\tau }-{\mathrm{w}}_{\mathrm{c}}^{\mathrm{t}}{\Vert }^{2/\left(\mathrm{m}-1\right)}$

Then, we calculate the gradient of the memberships ${\mathsf{\mu }}_{{\mathrm{g}}_{\mathrm{i}},\mathrm{t}}^{\mathrm{m}}$ and ${\mathrm{p}}_{{\mathrm{g}}_{\mathrm{i}},\mathrm{t}}$ and we obtain:

$${\nabla }_{{\mathrm{w}}_{{\mathrm{g}}_{\mathrm{i}}}}{\mathsf{\mu }}_{{\mathrm{g}}_{\mathrm{i}}}^{\mathrm{m}}\left(\mathsf{\tau }\right){/}_{{\mathrm{w}}_{{\mathrm{g}}_{\mathrm{i}}}={\mathrm{w}}_{{\mathrm{g}}_{\mathrm{i}}}^{\mathrm{t}}}=2/\left(\mathrm{m}-1\right)\frac{{\mathrm{A}}_{\mathrm{i},\mathrm{t}}\left(\mathsf{\tau }-{\mathrm{w}}_{{\mathrm{g}}_{\mathrm{i}}}^{\mathrm{t}}\right)\Vert \mathsf{\tau }-{\mathrm{w}}_{{\mathrm{g}}_{\mathrm{i}}}^{\mathrm{t}}{\Vert }^{\left(3-\mathrm{m}\right)/\left(\mathrm{m}-1\right)}}{{\left(1+{\mathrm{A}}_{\mathrm{i},\mathrm{t}}\Vert \mathsf{\tau }-{\mathrm{w}}_{{\mathrm{g}}_{\mathrm{i}}}^{\mathrm{t}}{\Vert }^{2/\left(\mathrm{m}-1\right)}\right)}^2}={\mathsf{\theta }}_{\mathrm{t}}^{\mathrm{i}}\left(\mathsf{\tau }\right)$$

and, ${\nabla }_{{\mathrm{w}}_{{\mathrm{g}}_{\mathrm{i}}}}{\mathrm{p}}_{{\mathrm{g}}_{\mathrm{i}}}\left(\mathsf{\tau }\right){/}_{{\mathrm{w}}_{{\mathrm{g}}_{\mathrm{i}}}={\mathrm{w}}_{{\mathrm{g}}_{\mathrm{i}}}^{\mathrm{t}}}=\frac{1}{\sqrt[\mathrm{n}]{2\mathsf{\pi }}{\mathsf{\sigma }}_{\mathrm{t}}^{\mathrm{n}+2}}\left({\mathrm{z}}_{\mathrm{i}}-\mathsf{\tau }-{\mathrm{w}}_{{\mathrm{g}}_{\mathrm{i}}}^{\mathrm{t}}\right)\exp \left(-\frac{\Vert {\mathrm{z}}_{\mathrm{i}}-\mathsf{\tau }-{\mathrm{w}}_{{\mathrm{g}}_{\mathrm{i}}}^{\mathrm{t}}{\Vert }^2}{2{\mathsf{\sigma }}_{\mathrm{t}}^2}\right)={\mathsf{\rho }}_{\mathrm{t}}^{\mathrm{i}}\left({\mathrm{z}}_{\mathrm{i}}-\mathsf{\tau }\right)$. Finally, ${\nabla }_{{\mathrm{w}}_{{\mathrm{g}}_{\mathrm{i}}}}{\mathsf{\phi }}_{{\mathrm{g}}_{\mathrm{i}}}\left({\mathrm{z}}_{\mathrm{i}}\right){/}_{{\mathrm{w}}_{{\mathrm{g}}_{\mathrm{i}}}={\mathrm{w}}_{{\mathrm{g}}_{\mathrm{i}}}^{\mathrm{t}}}=\left({\mathsf{\theta }}_{\mathrm{t}}^{\mathrm{i}}{\ast \mathrm{p}}_{{\mathrm{g}}_{\mathrm{i}},\mathrm{t}}\right)\left({\mathrm{z}}_{\mathrm{i}}\right)+\left({\mathsf{\mu }}_{{\mathrm{g}}_{\mathrm{i}},\mathrm{t}}^{\mathrm{m}}{\ast \mathsf{\rho }}_{\mathrm{t}}^{\mathrm{i}}\right)\left({\mathrm{z}}_{\mathrm{i}}\right)$.

Where ${\mathrm{p}}_{{\mathrm{g}}_{\mathrm{i}},\mathrm{t}}\left(\mathrm{z}\right)=\frac{1}{\sqrt[\mathrm{n}]{2\mathsf{\pi }}{\mathsf{\sigma }}_{\mathrm{t}}^{\mathrm{n}}}\exp \left(-\frac{\Vert \mathrm{z}-{\mathrm{w}}_{{\mathrm{g}}_{\mathrm{i}}}^{\mathrm{t}}{\Vert }^2}{2{\mathsf{\sigma }}_{\mathrm{t}}^2}\right)$ and ${\mathsf{\mu }}_{{\mathrm{g}}_{\mathrm{i}},\mathrm{t}}^{\mathrm{m}}\left(\mathrm{z}\right)=\frac{1}{{\sum }_{\mathrm{c}}{\frac{\Vert \mathrm{z}-{\mathrm{w}}_{{\mathrm{g}}_{\mathrm{i}}}^{\mathrm{t}}\Vert }{\Vert \mathrm{z}-{\mathrm{w}}_{\mathrm{c}}^{\mathrm{t}}\Vert }}^{2/\mathrm{m}-1}}.$ □

To update ${\mathsf{\sigma }}_{\mathrm{t}}$, we use the gradient descent method represented by the following equation:

$${\sigma }_{t+1}={\sigma }_t+a_t\frac{\partial {\phi }_{g_i}\left(z_i\right)}{\partial \sigma }{/}_{\sigma ={\sigma }_t}$$

where is the step of the Algorithm 1 following the current direction at time $t$.

Theorem 2.

We have $\frac{\partial {\phi }_{g_i}\left(z_i\right)}{\partial \sigma }{/}_{\sigma ={\sigma }_t}=\left({\mu }_{g_i,t}^m\ast \frac{\partial p_{g_i,t}}{\partial \sigma }\right)\left(z_i\right)$.

Proof. 

In fact, we have $\frac{\partial {\mathsf{\phi }}_{{\mathrm{g}}_{\mathrm{i}}}\left({\mathrm{z}}_{\mathrm{i}}\right)}{\partial \mathsf{\sigma }}{/}_{\mathsf{\sigma }={\mathsf{\sigma }}_{\mathrm{t}}}=\int {\mathsf{\mu }}_{{\mathrm{g}}_{\mathrm{i}},\mathrm{t}}^{\mathrm{m}}\left(\mathsf{\tau }\right)\frac{\partial {\mathrm{p}}_{{\mathrm{g}}_{\mathrm{i}},\mathrm{t}}\left({\mathrm{z}}_{\mathrm{i}}-\mathsf{\tau }\right)}{\partial \mathsf{\sigma }}{/}_{\mathsf{\sigma }={\mathsf{\sigma }}_{\mathrm{t}}}\mathrm{d}\mathsf{\tau }$

and

$$\frac{\partial {\mathrm{p}}_{{\mathrm{g}}_{\mathrm{i}},\mathrm{t}}}{\partial \mathsf{\sigma }}\left(\mathrm{z}\right)=\left(-\frac{{\mathsf{\sigma }}_{\mathrm{t}}^{\mathrm{n}-1}}{{\sqrt[\mathrm{n}]{2\mathsf{\pi }}}^2{\mathsf{\sigma }}_{\mathrm{t}}^{2\mathrm{n}}}+\frac{\Vert \mathrm{z}-{\mathrm{w}}_{{\mathrm{g}}_{\mathrm{i}}}^{\mathrm{t}}{\Vert }^2}{\sqrt[\mathrm{n}]{2\mathsf{\pi }}{\mathsf{\sigma }}_{\mathrm{t}}^{\mathrm{n}}}\times \frac{{\mathsf{\sigma }}_{\mathrm{t}}}{{\mathsf{\sigma }}_{\mathrm{t}}^4}\right)\exp \left(-\frac{\Vert \mathrm{z}-{\mathrm{w}}_{{\mathrm{g}}_{\mathrm{i}}}^{\mathrm{t}}{\Vert }^2}{2{\mathsf{\sigma }}_{\mathrm{t}}^2}\right)$$

then

$$\frac{\partial {\mathrm{p}}_{{\mathrm{g}}_{\mathrm{i}},\mathrm{t}}}{\partial \mathsf{\sigma }}\left(\mathrm{z}\right)=\frac{1}{\sqrt[\mathrm{n}]{2\mathsf{\pi }}{\mathsf{\sigma }}_{\mathrm{t}}^{\mathrm{n}+1}}\left(-\frac{1}{\sqrt[\mathrm{n}]{2\mathsf{\pi }}}+\frac{\Vert \mathrm{z}-{\mathrm{w}}_{{\mathrm{g}}_{\mathrm{i}}}^{\mathrm{t}}{\Vert }^2}{{\mathsf{\sigma }}_{\mathrm{t}}^2}\right)\exp \left(-\frac{\Vert \mathrm{z}-{\mathrm{w}}_{{\mathrm{g}}_{\mathrm{i}}}^{\mathrm{t}}{\Vert }^2}{2{\mathsf{\sigma }}_{\mathrm{t}}^2}\right)$$

Then we substitute $\mathrm{z}$ with ${\mathrm{z}}_{\mathrm{i}}-\mathsf{\tau }.$

In the following section, we give the version of the proposed system that implements the full-gradient of the loss function:

Algorithm 1. Fuzzy-Probabilistic-Convolution-C-Means.

Requires: Data =$\left\{{\mathrm{z}}_1,\dots ,{\mathrm{z}}_{\mathrm{N}}\right\}$, $\mathrm{k}$ (number of groups), $s (W time step)$, $a \left(\sigma time step\right)$, m (memberships parameter), b (mini-batch), ITER (maximum number of iterations).

Ensure: centers matrix, memberships matrix of ${\mathrm{z}}_1,\dots ,{\mathrm{z}}_{\mathrm{N}}$ to the groups.

Initialization: t = 0, ${\mathrm{W}}^0$, ${\sigma }^0$ are randomly chosen;   For all t = 1, …, ITER Do    For all j = 1, …, k Do     For all d = 1, …, N Do

$BNN\leftarrow b-Nearst Neighbors of z_i$

${\phi }_j^m\left(z_i\right)={\displaystyle {\displaystyle \sum }_{\tau \in BNN}}\left(1/b\right){\mu }_j^{t,m}\left(\tau \right)p_j^t\left(z_i-\tau \right)$

$w_j^{t+1}=w_j^t+s{\nabla }_{w_j}{\phi }_j\left(z_i\right){/}_{w_j=w_j^t}$

${\sigma }_{t+1}=\max \left({\sigma }_t+a\frac{\partial {\phi }_j\left(z_i\right)}{\partial \sigma }{/}_{\sigma ={\sigma }_t},0\right)$

End For   End For  End For

END

In this algorithm, the full convolution was approximated by discrete values, which implements a mini-batch of size b: the b-nearest samples of the current sample are used to estimate the continuous convolution.

In the experimentation section, we use the b nearest neighbors of each sample. It is possible to use genetic algorithm to estimate Ω (V)(support of ${ \mathsf{\phi }}_{{\mathrm{g}}_{\mathrm{i}}}$), Ω such that: $\forall \mathrm{i}\in \left\{1,\dots ,\mathrm{N}\right\},{ \mathsf{\phi }}_{{\mathrm{g}}_{\mathrm{i}}}\left(\mathrm{z}\right)=\left({\mathsf{\mu }}_{{\mathrm{g}}_{\mathrm{i}}}{\ast \mathrm{p}}_{{\mathrm{g}}_{\mathrm{i}}}\right)\left(\mathrm{z}\right)$

$$=\begin{array}{c}\int {\mathsf{\mu }}_{{\mathrm{g}}_{\mathrm{i}}}^{\mathrm{m}}\left(\mathsf{\tau }\right){\mathrm{p}}_{{\mathrm{g}}_{\mathrm{i}}}\left(\mathrm{z}-\mathsf{\tau }\right)\mathrm{d}\mathsf{\tau }\\ {ℝ}^{\mathrm{n}} \end{array} \approx \begin{array}{c}\int {\mathsf{\mu }}_{{\mathrm{g}}_{\mathrm{i}}}^{\mathrm{m}}\left(\mathsf{\tau }\right){\mathrm{p}}_{{\mathrm{g}}_{\mathrm{i}}}\left(\mathrm{z}-\mathsf{\tau }\right)\mathrm{d}\mathsf{\tau }\\ \mathsf{\Omega } \end{array}$$

Given a support Ω, it is assumed that there exists an expression for the mathematical expectation of the function $\tau \to {\mathsf{\mu }}_{{\mathrm{g}}_{\mathrm{i}}}^{\mathrm{m}}\left(\mathsf{\tau }\right){\mathrm{p}}_{{\mathrm{g}}_{\mathrm{i}}}\left(\mathrm{z}-\mathsf{\tau }\right)$ of the random variable X, of density ${\mathrm{f}}_{\mathrm{X}}=\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$\left|\mathsf{\Omega }\right|$}\right.$ of support Ω, resulting from the transfer theorem, based on which ${ \mathsf{\phi }}_{{\mathrm{g}}_{\mathrm{i}}}\left(\mathrm{z}\right)\approx \begin{array}{c}\int {\mathsf{\mu }}_{{\mathrm{g}}_{\mathrm{i}}}^{\mathrm{m}}\left(\mathsf{\tau }\right){\mathrm{p}}_{{\mathrm{g}}_{\mathrm{i}}}\left(\mathrm{z}-\mathsf{\tau }\right){\mathrm{f}}_{\mathrm{X}}\left(\mathsf{\tau }\right)\mathrm{d}\mathsf{\tau }\\ \mathsf{\Omega } \end{array}$. This can be extended to discrete probabilities by summing with a discrete Dirac-like measure. We use Monte Carlo simulation while producing a sample (x₁,x₂,...,x_E) of the random variable X on the support Ω, then we calculate an estimation of ${ \mathsf{\phi }}_{{\mathrm{g}}_{\mathrm{i}}}\left(\mathrm{z}\right)$ based on this sample [54]. Based on the law of large numbers, the empirical mean is a very good estimator. Since the probability densities and membership functions are designed so that they cover, as much as possible, the different data in the set $\left\{{\mathrm{z}}_1,\dots ,{\mathrm{z}}_{\mathrm{N}}\right\}$, then the supports of these functions are centered around these data. Thus, it is natural to choose the Monte Carlo samples from these data. In our case, the parameter b is calculated using the formula $N/k$, where $k$ is the number of clusters, because each pair (${\mathsf{\mu }}_{{\mathrm{g}}_{\mathrm{i}}}^{\mathrm{m}},{\mathrm{p}}_{{\mathrm{g}}_{\mathrm{i}}}$) is supported by the data gained by the group ${\mathrm{g}}_{\mathrm{i}}$.

5. Experimental Results

5.1. Datasets

To evaluate the performance of FP-Conv-CM, we used six datasets from the UCI [55] repository.

Table 1. Description of the experimental datasets.

Dataset	Features	Samples
Iris	4	150
Pima	8	768
Foods	19	177
Abalone	8	326
MAGIC. Gamma. Telescope	3	306
Cloud	4	625

describes the features of the selected datasets.

5.2. Clustering Results

Table 2. Dunn’s index and Silhouette of FCM, GMM, and FP-Conv-CM on different datasets.

Data Set	FCM		GMM		FP-Conv-CM
	Silhouette	Dunn’s Index	Silhouette	Dunn’s Index	Silhouette	Dunn’s Index
Iris	120.883	0.0468	117.402	0.041	117.872	0.065
Foods	131.175	0.012	−113.774	0.002	131.175	0.012
Abalone	6.07 × 103	0.0025	6.078 × 103	0.0038	6.054 × 103	0.0042
Pima	284.783	0.0120	50.691	0.0111	278.28	0.0142
MAGIC.Gamma.Telescope	4.05 × 103	6.46 × 10-4	3.52 × 103	5.59 × 10-4	3.60 × 103	0.0011
Cloud	1.3 × 103	0.0022	366.8076	0.0027	1.3 × 103	0.0052

gives the values of Silhouette and Dunn’s indexes obtained by applying FCM, GMM, and FP-Conv-CM to iris, foods, abalone, pima, MAGIC Gamma Telescope, and cloud datasets for a clustering task. Figure 6, Figure 7, Figure 8, Figure 9, Figure 10, Figure 11, Figure 12, Figure 13, Figure 14, Figure 15, Figure 16, Figure 17, Figure 18, Figure 19, Figure 20, Figure 21, Figure 22 and Figure 23 give the silhouettes obtained by applying FCM, GMM, and FP-Conv-CM to these datasets. The forms of these silhouettes show that the groups produced by FCM and FP-Conv-CM are more compact than those produced by GMM. In fact, columns two, four, and six of Table 2 show that FCM and FP-Conv-CM have the largest silhouette values, with those of FP-Conv-CM being slightly higher. However, FP-Conv-CM clearly improves the silhouette of the ones obtained by GMM.

Considering the six datasets, FCM, GMM, and FP-Conv-CM produce clusters with Dunn’s indexes inferior to 0.065 because of the natural overlapping between different classes of these datasets, which minimizes $d_{\min }$ (the minimal distance between samples in different clusters). In addition, and combining silhouette criterion with Dunn’s index, $d_{\max }\left(\mathrm{the} \mathrm{largest} \mathrm{within}-\mathrm{cluster} \mathrm{distance}\right)$ is very large, especially for FCM and GMM, because some samples are not similar but are assigned to the same groups (see the silhouettes of the groups produced by FCM and GMM using different datasets). However, FP-Conv-CM produces clusters with high Dunn’s values compared with FCM and GMM. This amelioration is achieved thanks to the combination of fuzzy and probabilistic reasonings.

To improve the performance of FP-Conv-CM, we used the attribute selection method named Correlation features Subset Evaluation (CSE) [56].

Table 3. Dunn’s index, Silhouette, and CPU time of FP-Conv-CM for different datasets with full and reduced features using the SCE method.

Attributes Selection Evaluates the Worth of a Subset of Attributes		FP-Conv-CM Full Features			FP-Conv-CM Reduced Features
Data Set	Selected Attributes	Silhouette	Dunn’s Index	Cpu Time(s)	Silhouette	Dunn’s Index	Cpu Time(s)
Iris	Petal length, petal width	117.87	0.065	0.321	123.995	0.0936		0.453
Foods	VA, VC, VE, VB6, V12, Iron, Proteins, Carbohydrates, Lipids (Tf)	131.18	0.012	0.386	120.1	33.4 × 10−3		0.482
Abalone	Shell weight, Diameter, Height, Length, Whole weight, Viscera weight	6.05 × 103	4.2 × 10−3	9.433	1.493 × 103	2.4 × 10−3		6.205
Pima	Plas, mass, pedi, age	278.28	14.2 × 10−3	0.540	208.77	16.3 × 10−3		2.291
MAGIC	fLength	3.60 × 103	1.1 × 10−3	40.031	1.215 × 104	1.0 × 10−3		48.401
Gamma	fWidth
Telescope	fAlpha
Cloud	North Control Area, South Control Area	1.3 × 103	5.2 × 10−3	1.628	1.0 × 103	5.0 × 10−3	0.697
Mean	4.3	1,913.56	0.017	8.72	2,516.13	0.025	9.76

gives Dunn’s index, Silhouette, and CPU time of FP-Conv-CM for different datasets with full and reduced features using the CSE method. Analysis of the six datasets showed that CSE permits the reduction of almost 77% of the attributes in an average CPU time of 3.99 seconds. This resulted in a 24% improvement of the average Silhouette, a 32% improvement of the Dunn’s index, and a 10.7% improvement in CPU time.

5.3. Image Compression

We evaluated the performance of FP-Conv-CM on an image compression task using several images (see Figure 24, Figure 25 and Figure 26). To avoid cluttering the document with several figures, we give here the results obtained for three images only.

5.4. Compression Results

Figure 27, Figure 28, Figure 29, Figure 30, Figure 31, Figure 32, Figure 33, Figure 34 and Figure 35 give the images obtained after decompression of the compression realized using the algorithms based on GMM, FCM, and FP-Conv-CM.

Table 4. MSE, PEAKSNR, and SSIM values associated with the images compressed by FCM, GMM, and FP-Conv-CM.

Method	MSE	PEAKSR	SSIM
	Cameraman
GMM	1.163	17.4748	0.6507
FCM	597.6039	20.3667	0.7325
FP-Conv-CM	597.4229	20.3680	0.7308
	Khwarizmi
GMM	3.0 × 10+3	13.3192	0.2574
FCM	1.2 × 10+3	17.2057	0.6609
FP-Conv-CM	1.2 × 10+3	17.1000	0.6846
	Archimedes
GMM	5.262 × 10+3	10.92	0.46
FCM	455.66	21.54	0.88
FP-Conv-CM	456.13	22.00	0.90

gives the values of the performance measures MSE, PSNR, and SSIM of different compressions realized with the algorithms that implement GMM, FCM, and FP-Conv-CM.

Analysis of MSE and PSNR shows that our method permits small improvements in the compression. Analysis of SSIM shows that our method provides an important improvement in compression quality.

6. Conclusions and Future Perspectives

Fuzzy logic makes decisions on the basis of the degree of membership without giving any information about the frequency of events, whereas probability informs us about the frequency of events but gives no information about the degree of membership to a set or class. This paper proposed a convolution fuzzy probabilistic measure that measures the membership degree and the frequency at the same time. Using concrete examples, we proved that the new measure corrects the shortcoming of both the probability measure and the fuzzy logic-based measure. Based on this measure, this paper introduced a new clustering method, named Fuzzy-Probabilistic-Convolution C-Means (FP-Conv-CM). FCM, PKM, and FP-Conv-CM were tested on multiple datasets and compared on the basis of two performance measures: the Silhouette metric and Dunn’s Index. FP-Conv-CM was able to improve the Silhouette value by 1000 and Dunn’s Index by 0.024. In addition, FCM, PKM, and FP-Conv-CM were used for multiple image compressions and were compared based on three performance measures: mean squared error (MSE), peak signal-to-noise ratio (PSNR), and structural similarity index (SSIM). FP-Conv-CM improved MSE by 3000, PSNR by 11, and SSIM by 0.32.

Given the performance of FP-Conv-CM in food grouping, in future we will use it to personalize diets for people with diabetes [57,58]. Also, given the performance of FP-Conv-CM in grouping diabetics, we will use it in the automatic grouping of a population of diabetics to determine the different components of the control models proposed in [59]. Recently, we used GMM and FCM for localization in stochastic environments to improve the results obtained in [60], and we will use FP-Conv-CM to summarize the information from LiDAR sensor, which will overcome the localization limitations caused by both GMM and FCM methods.

Unfortunately, FP-Conv-CM inherits some limitations of classical fuzzy logic, which does not take into account the degree of non-membership to different classes. Moreover, we encountered some difficulties in the selection of an optimal support, and a heuristic method is needed to make a good choice. In future, we will use evolutionary algorithms to choose the patterns that best cover the support of the membership functions and the Gaussians implemented by FC-Conv-CM. In addition, we will introduce the Fuzzy Intuitionist Convolution C-means version to take advantage of the ability of the intuitionist logic to quantify the degree of non-appartenance of patterns to different classes.