An Improved Similarity-Based Clustering Algorithm for Multi-Database Mining

Abstract

Clustering algorithms for multi-database mining (MDM) rely on computing $(n^2-n)/2$ pairwise similarities between n multiple databases to generate and evaluate $m\in [1,(n^2-n)/2]$ candidate clusterings in order to select the ideal partitioning that optimizes a predefined goodness measure. However, when these pairwise similarities are distributed around the mean value, the clustering algorithm becomes indecisive when choosing what database pairs are considered eligible to be grouped together. Consequently, a trivial result is produced by putting all the n databases in one cluster or by returning n singleton clusters. To tackle the latter problem, we propose a learning algorithm to reduce the fuzziness of the similarity matrix by minimizing a weighted binary entropy loss function via gradient descent and back-propagation. As a result, the learned model will improve the certainty of the clustering algorithm by correctly identifying the optimal database clusters. Additionally, in contrast to gradient-based clustering algorithms, which are sensitive to the choice of the learning rate and require more iterations to converge, we propose a learning-rate-free algorithm to assess the candidate clusterings generated on the fly in fewer upper-bounded iterations. To achieve our goal, we use coordinate descent (CD) and back-propagation to search for the optimal clustering of the n multiple database in a way that minimizes a convex clustering quality measure $\mathcal{L}\left(\theta \right)$ in less than $(n^2-n)/2$ iterations. By using a max-heap data structure within our CD algorithm, we optimally choose the largest weight variable ${\theta }_{p,q}^{\left(i\right)}$ at each iteration i such that taking the partial derivative of $\mathcal{L}\left(\theta \right)$ with respect to ${\theta }_{p,q}^{\left(i\right)}$ allows us to attain the next steepest descent minimizing $\mathcal{L}\left(\theta \right)$ without using a learning rate. Through a series of experiments on multiple database samples, we show that our algorithm outperforms the existing clustering algorithms for MDM.

1. Introduction

Large multi-branch companies need to analyze multiple databases to discover useful patterns for the decision-making process. To make global decisions for the entire company, the traditional approach suggests to merge and integrate the local branch-databases into a huge data warehouse, and then one can apply data mining algorithms [1] to the accumulated dataset to mine the global patterns useful for all the branches of the company. However, there are some limitations associated with this approach. For instance, the cost of moving the data over the network, and integrating and storing potentially heterogeneous databases could be expensive. Moreover, some branches may not accept sharing their raw data due to the underlying privacy issues. More crucially, integrating a large amount of irrelevant data can easily disguise some essential patterns hidden in multiple databases. To tackle the latter problems, it is suggested to keep the transactional data stored locally and only forward the local patterns mined at each branch database to a central site where they will be clustered into disjoint cohesive pattern-base groups for knowledge discovery. In fact, analyzing the local patterns present in each individual cluster of the multiple databases (MDB) enhances the quality of aggregating novel relevant patterns, and also facilitates the parallel maintenance of the obtained database clusters.Various clustering algorithms and models have been introduced in the literature, namely spectral-based models [2], hierarchical [3], partitioning [4], competitive learning-based models [5,6,7] and artificial neural networks (ANNs) based clustering [8,9,10]. Additionally, clustering could be applied in many domains [11,12] including community discovery in social networks [13,14], image segmentation [15,16] and recommendation systems [17,18,19]. In this article, we focus on exploring similarity-based clustering models for multi-database mining [20,21,22,23], due to their stability, simplicity [24] and robustness in partitioning graphs of n multiple databases into k connected components consisting of similar database objects. Nevertheless, the existing clustering quality measures in [20,21,22,23] are non-convex objectives suffering from the existence of local optima. Consequently, identifying the optimal clustering may be a difficult task, as it requires evaluating all the candidate clusterings generated at all the local optima in order to find the ideal clustering.

To address the issues associated with clustroid initialization, preselection of a suitable number of clusters and non-convexity of the clustering quality objectives, we proposed in [25,26] an algorithm named GDMDBClustering, which minimizes a quasi-convex loss function quantifying the quality of the multi-database clustering, without a priori assumptions about which number of clusters should be chosen. Therefore, in contrast to the clustering models proposed in [20,21,22,23], GDMDBClustering [25] does not require us to produce and assess all the possible candidate classifications in order to find the optimal partitioning. Alternatively, each partitioning is assessed on the fly as it is generated and the clustering algorithm terminates right after attaining the global minimum of the objective function. However, the existing gradient-based clustering algorithms [25,26] are strongly dependent on the choice of the learning rate $\eta$, which influences the number of learning cycles required to find the optimal partitioning. In fact, selecting a larger $\eta$ value may cause global minimum overshooting and setting a smaller $\eta$ value may necessitate many learning iterations for the algorithm to converge.

In this paper, we improve upon previous work [25,26] and propose a learning-rate-free (i.e., independent of the learning rate $\eta$) algorithm requiring fewer upper-bounded iterations (i.e., the maximum number of iterations is at most $(n^2-n)/2$) to minimize a clustering convex loss function $\mathcal{L}\left(\theta \right)$ using coordinate descent (CD) and back-propagation. Precisely, our proposed algorithm minimizes a quadratic hinge-based loss $\mathcal{L}\left(\theta \right)$ over the first largest coordinate variable ${\theta }_{p,q}$ while keeping the rest of the $\left(\frac{n}{2}\right)-1$ variables fixed. Then, it minimizes $\mathcal{L}\left(\theta \right)$ over the second largest coordinate variable while keeping the rest of the $\left(\frac{n}{2}\right)-1$ variables fixed, and so on until convergence or until cycling through all the $\left(\frac{n}{2}\right)$ coordinate variables. Consequently, our algorithm becomes faster than GDMDBClustering [25] which is dependent on a learning rate and also requires us to minimize the cost over a large set of variables at each iteration. This can be a very challenging problem in contrast to minimizing the loss over one single variable at a time while keeping all the other dimensions fixed.

On the other hand, existing clustering algorithms for multi-database mining (MDM) [20,21,22,23,25,26] proceed by computing $(n^2-n)/2$ pairwise similarities $sim({\mathcal{D}}_p,{\mathcal{D}}_q)\in [0,1]$ between n multiple databases, and then use these values to generate and evaluate $m\in [1,(n^2-n)/2]$ candidate clusterings in order to select the ideal partitioning optimizing a given goodness measure. However, when $sim{({\mathcal{D}}_p,{\mathcal{D}}_q)}_{n\times n}$ ($p=0,\dots ,n-2,q=p+1,\dots ,n-1$) are distributed around the mean value $\mu =0.5$, the fuzziness index of the similarity matrix increases and the clustering algorithm becomes uncertain when choosing what database pairs are considered similar and hence eligible to be put into the same cluster. Consequently, a trivial result is produced, i.e., putting all the n databases in one cluster or returning n singleton clusters. To tackle the latter problem, we propose a learning algorithm to reduce the fuzziness in the pairwise similarities by minimizing a weighted binary entropy loss function $\mathcal{H}(·)$ via gradient descent and back-propagation. Precisely, the learned model will force the similarity values above 0.5 to go closer to their maximum value (≈1), and let those below 0.5 go closer to their minimum value (≈0) in a way that minimizes $\mathcal{H}(·)$. This will significantly reduce the associated fuzziness and improve the certainty of the clustering algorithm to correctly identify the optimal database clusters. The main contributions of this article are listed as follows:

• Unlike the existing algorithms proposed in [20,21,22,23,25,26] where one-class trivial clusterings are produced when the similarity values are centered around the mean value, we have added a preprocessing layer prior to clustering where the pairwise similarities are adjusted to reduce the associated fuzziness and hence improve the quality of the produced clustering. Our experimental results show that reducing the fuzziness of the similarity matrix helps generating meaningful relevant clusters that are different to the one-class trivial clusterings.
• Unlike the multi-database clustering algorithms proposed in [20,21,22,23], our approach uses a convex objective function $\mathcal{L}\left(\theta \right)$ to assess the quality of the produced clustering. This allows our algorithm to terminate just after attaining the global minimum of the objective function (i.e., after exploring fewer similarity levels). Consequently, this avoids generating unnecessary candidate clusterings, and hence reduces the CPU overhead. On the other hand, the clustering algorithms in [20,21,22,23] use non-convex objectives (i.e., they suffer from the existence of local optima due to the use of more than two monotonic functions), and therefore require generating and evaluating all the $(n^2-n)/2$ local candidate clustering solutions in order to find the clustering located at the global optimum.
• Furthermore, unlike the previous gradient-based clustering algorithms [25,26], our proposed algorithm is leaning-rate-free (i.e., independent of the learning rate), and needs at most (in the worst case) $(n^2-n)/2$ iterations to converge. That is why our proposed algorithm is faster than GDMDBClustering [25], which is strongly dependent on the learning step size $\eta$ and its decay rate.
• Additionally, unlike the similarity measure proposed in [20], which assumes that the same threshold was used to mine the local patterns from the n transactional databases, our proposed similarity measure takes into account the existence of n different local thresholds, which are then combined to calculate a new threshold for each cluster. Afterward, using the new thresholds, our similarity measure accurately estimates the valid patterns post-mined from each cluster in order to compute the $(n^2-n)/2$ pairwise similarities.
• The experiments carried out on real, synthetic and randomly generated datasets show that the proposed clustering algorithm outperforms the compared clustering models in [20,21,22,23,25,26], as it has the shorted average running time and the lowest average clustering error.

The remainder of this paper is organized as follows: Section 2 presents an example motivating the importance of clustering for multi-database mining (MDM) and also reviews traditional clustering algorithms for MDM. Section 3 defines the main concepts related to similarity-based clustering and then introduces the proposed approach and its main components. Section 4 presents and analyzes the experimental results. Finally, Section 5 draws conclusions and highlights potential future work.

2. Motivation and Related Work

2.1. Motivating Example

Prior to mining the multiple databases (MDB) of a multi-branch enterprise, it is essential to cluster these MDB into disjoint and cohesive pattern-base groups sharing an important number of local patterns in common. Then, using local pattern analysis and pattern synthesizing techniques [27,28,29,30], one can examine the local patterns in each individual cluster to discover novel patterns, including the exceptional patterns [31] and the high-vote patterns [32], which are extremely useful when it comes to making special targeted decisions regarding each cluster of branches of the same corporation. In the following example, we show the impact of clustering the multi-databases of a multi-branch corporation prior to multi-database mining. Consider the six transactional databases $\mathcal{D}={\cup }_{p=1}^6\left\{{\mathcal{D}}_p\right\}$ shown in

Table 1. Six transactional databases ${\mathcal{D}}_p$, for $p=1,\dots ,6$.

Transactional Database $\left({\mathcal{D}}_{\mathit{p}}\right)$	Transactions/Rows
${\mathcal{D}}_1$	$(A,C),(A,B,C),(B,C),(A,B,C,D)$
${\mathcal{D}}_2$	$(A,B,C),(B,C),(A,B),(A,C),(A,B,D)$
${\mathcal{D}}_3$	$(B,C),(A,D),(B,C,D),(A,B,C)$
${\mathcal{D}}_4$	$(E,F,H),(F,H),(F,G,H,I,J)$
${\mathcal{D}}_5$	$(E,J),(F,H,J),(E,F,H,J),(F,H)$
${\mathcal{D}}_6$	$(E,I),(E,F,H),(F,H,I,J),(E,H,J)$

, where each database ${\mathcal{D}}_p$ records a set of transactions enclosed in parentheses and each transaction contains a set of items separated by commas. Consider a minimum support threshold $\alpha =0.5$. The local frequent itemsets, denoted by $FIS({\mathcal{D}}_p,\alpha )$, and discovered from each database ${\mathcal{D}}_p$ are shown in

Table 2. The frequent itemsets (FIs) discovered from each transactional database in Table 1 under a threshold $\alpha =0.5$.

Transactional Database $\left({\mathcal{D}}_{\mathit{p}}\right)$	Frequent Itemsets $\mathit{FIS}({\mathcal{D}}_{\mathit{p}},\mathit{\alpha })$
${\mathcal{D}}_1$	$\langle AC,0.75\rangle ,\langle AB,0.5\rangle ,\langle ABC,0.5\rangle ,\langle BC,0.75\rangle ,\langle C,1.0\rangle ,\langle B,0.75\rangle ,\langle A,0.75\rangle$
${\mathcal{D}}_2$	$\langle AB,0.6\rangle ,\langle C,0.6\rangle ,\langle B,0.8\rangle ,\langle A,0.8\rangle$
${\mathcal{D}}_3$	$\langle BC,0.75\rangle ,\langle D,0.5\rangle ,\langle C,0.75\rangle ,\langle B,0.75\rangle ,\langle A,0.5\rangle$
${\mathcal{D}}_4$	$\langle H,1.0\rangle ,\langle F,1.0\rangle ,\langle FH,1.0\rangle$
${\mathcal{D}}_5$	$\langle E,0.5\rangle ,\langle EJ,0.5\rangle ,\langle J,0.75\rangle ,\langle HJ,0.5\rangle ,\langle FHJ,0.5\rangle ,\langle FJ,0.5\rangle ,\langle H,0.75\rangle ,\langle FH,0.75\rangle ,\langle F,0.75\rangle$
${\mathcal{D}}_6$	$\langle I,0.5\rangle ,\langle J,0.5\rangle ,\langle HJ,0.5\rangle ,\langle F,0.5\rangle ,\langle FH,0.5\rangle ,\langle E,0.75\rangle ,\langle EH,0.5\rangle ,\langle H,0.75\rangle$

, such that $I_k$ in each tuple $\langle I_k,supp(I_k,{\mathcal{D}}_p)\rangle$ of $FIS({\mathcal{D}}_p,\alpha )$ is the frequent itemset name and $supp(I_k,{\mathcal{D}}_p)$, named $support$, is the ratio of the number of transactions in ${\mathcal{D}}_p$ containing $I_k$ to the total number of transactions in ${\mathcal{D}}_p$.

Now, the global support of each itemset $I_k\in {\cup }_{p=1}^6\left\{FIS({\mathcal{D}}_p,0.5)\right\}$ is calculated via the synthesizing equation [33] defined as follows:

$$\begin{array}{cc}\hfill supp(I_k,\mathcal{D})& =\frac{{\sum }_{p=1}^n\left|{\mathcal{D}}_p\right|\times supp(I_k,{\mathcal{D}}_p)}{{\sum }_{p=1}^n\left|{\mathcal{D}}_p\right|}\hfill \end{array}$$

(1)

where $n=6$ is the total number of databases in $\mathcal{D}$ and $|{\mathcal{D}}_p|$ is the number of transactions in ${\mathcal{D}}_p$. For instance, we can calculate the global support of the itemset A as follows:

$$\begin{array}{cc}\hfill supp(A,\mathcal{D})& =\frac{0.75\times 4+0.8\times 5+0.5\times 4+0\times 3+0\times 4+0\times 4}{4+5+4+3+4+4}\hfill \\ & =0.375<\alpha \hfill \end{array}$$

After computing the global supports of the rest of the itemsets using (1), no single novel pattern has been found, i.e., $\forall I_k\in {\cup }_{p=1}^6\left\{FIS({\mathcal{D}}_p,0.5)\right\},supp(I_k,\mathcal{D})<0.5$). The reason is that irrelevant patterns were involved in the synthesizing procedure. Now, if we examine the frequent itemsets in Table 2, we observe that some databases share many patterns in common. Precisely, the six databases seem to form two clusters, $C_1$ = $\{{\mathcal{D}}_1,{\mathcal{D}}_2,{\mathcal{D}}_3\}$ and $C_2$ = $\{{\mathcal{D}}_4,{\mathcal{D}}_5,{\mathcal{D}}_6\}$, where each cluster of databases tend to share similar frequent itemsets.

Next, let us use the synthesizing Equation (1) on the frequent itemsets coming from every single cluster $C_i$, such that $4\le p\le 6=n$ for cluster $C_2$ and $1\le p\le 3=n$ for cluster $C_1$. This time, new valid frequent itemsets having a support value above the minimum threshold $\alpha$ are discovered in the two clusters. In fact, $FIS(C_2,0.5)$ = $\{\langle FH,0.727\rangle ,\langle F,0.727\rangle ,\langle H,0.818\rangle \}$ and $FIS(C_1,0.5)$ = $\{\langle C,0.769\rangle ,\langle B,0.769\rangle ,\langle A,0.692\rangle \}$. The obtained patterns show that a percentage of more than $69\%$ of the total transactions in the cluster $C_1$ include the itemsets C, B and A. More than $72\%$ of the total transactions in the cluster $C_2$ include $FH$, F and H. Moreover, some associations between itemsets could be derived as well, such that the itemset $\langle FH,0.727\rangle \in FIS(C_2,0.5)$ suggests that on average, if a customer collects the item H at one of the branches in $C_2$, they are likely to also buy the item F with a $\frac{supp(FH,C_2)}{supp(H,C_2)}=88.87\%$ confidence.

The above example demonstrates the importance of clustering the multi-databases into disjoint cohesive clusters before synthesizing the global patterns. In fact, when the local patterns mined from the six databases were analyzed all together, no global pattern could be synthesized. On the other hand, when the six databases were divided into two different clusters and then each cluster was analyzed individually, useful and novel patterns (knowledge) were discovered. Actually, from the discovered knowledge, decision makers and stakeholders are going to have a clear vision about the branches that exhibit similar purchasing behaviors, and hence take useful decision accordingly. In fact, appropriate business decisions may be taken regarding each group of similar branches in order to predict potential purchasing patterns, increase the customer retention rate and convince customers to purchase more services in the future. Consequently, exploring and examining individual clusters of similar local patterns is going to help the discovery of new and relevant patterns capable of improving the decision-making quality.

2.2. Prior Work

The authors in [34] have adopted a divide and conquer mono-database mining approach to accelerate mining global frequent itemsets (FIs) in large transactional databases. In [35,36], the authors have proposed similar work where big transactional databases are divided into k disjoint transaction partitions whose sizes are small enough to be read and loaded to the random access memory. Then, the frequent itemsets (FIs) mined from all the k partitions are synthesized into global FIs using an aggregation function such as the one suggested by the authors in [33]. It is worth noting that for mono-database mining applications, we usually have direct access to the raw data stored in big transactional databases. On the other hand, for multi-database mining (MDM) applications, it is suggested to keep the transactional data stored locally and only forward the local patterns mined at each branch database to a central site where they will be clustered into disjoint cohesive pattern-base groups for knowledge discovery. As a result, the confidential raw data are kept safe, and also the cost associated with transmitting a large amount of data over the network is cut off. Hence, in contrast to clustering the transactional data stored in a single data warehouse, our approach consists of clustering the local patterns mined and forwarded from multi-databases without requiring the number of clusters to be set a priori. Our purpose is to identify the group of databases that share similar patterns, such as the high-vote patterns [32] and the exceptional patterns [31,37,38] that can be used to make specific decisions regarding their corresponding branches. In the traditional clustering approach [34,35,36] applied for mono-database mining, we can only mine the global patterns that are supported by the whole multi-branch company.

The existing clustering algorithms for multi-database [20,21,23,39,40] are based on an agglomerative process that generates hierarchical partitionings at different levels of similarity, where each cluster in a given candidate partitioning is included in another cluster of a partitioning produced at the next similarity level. Regardless of the latter observation, each candidate partitioning is produced without taking into account the use of the clusters generated at the previous similarity levels. As a result, the clustering algorithms in [20,21,23,39,40] unnecessarily reconstruct clusters that have been built at the previous similarity levels. The latter limitation inspired the authors in [22] to design a graph-based algorithm, which maintains the classes produced at prior similarity levels in order to produce new subsequent classes out of them. Despite the fact that the experiments done in [22] showed promising results against the prior work [20,21,23,39,40], these algorithms are based on non-convex functions to evaluate the quality of the produced candidate clusterings. Consequently, finding the ideal clustering for which a non-convex function is optimal may be a difficult problem to solve in a short time.

To face the latter problem, the authors in [26] have transformed the clustering problem into a quasi-convex optimization problem solvable via gradient descent and back-propagation. Consequently, an early stopping of the clustering process occurs right after converging to the global minimum. Hence, by avoiding generating and evaluating unnecessary candidate clusterings, we can significantly reduce the CPU execution time. Even though traditional clustering algorithm such as k-means [4,41] are intuitive, popular and not hard to implement, they remain sensitive to clustroid initialization, preselection of a suitable number of clusters and non-convexity of the clustering quality objective [42]. The silhouette plot [43] could be used to find an appropriate number of clusters, but this requires executing k-means multiple times with different number of clusters in order to find the ideal partitioning maximizing the silhouette objective. As a result, the time performance will be influenced in the case of clustering big high-dimensional datasets. Slightly different, hierarchical-based clustering algorithms [3] build nested hierarchical levels to visualize the relationships between different objects in the form of dendrograms. Then, it is up to the domain expert or to some non-convex metrics to determine at which level the tree diagram should be cut.

Conversely, the optimization problem formulated in [25,26] is quasi-convex. Therefore, convergence to the global optimum is independent of the initial settings. Furthermore, the proposed gradient-based clustering GDMDBClustering [25] does not need to have the number of clusters as a parameter. Alternatively, the number of clusters becomes a parametric function in the main objective. However, GDMDBClustering is based on the choice of a suitable learning rate, i.e., choosing a small learning rate $\eta$ may increase the number of iterations and slow down learning the optimal weights, whereas a large $\eta$ may let the algorithm overshoot the global minimum. To overcome the latter limitation, we propose in this paper a learning-rate-free clustering algorithm, named CDClustering, which minimizes a convex objective function quantifying the clustering quality. For this purpose, we use coordinate descent (CD) and back-propagation to search for the optimal clustering of n multiple database in less than $(n^2-n)/2$ iterations and without using a learning rate. This makes our algorithm faster than the previous gradient-based clustering algorithm [25,26] which remains dependent on a learning rate defined based on some prior knowledge of the properties of the loss function. On the other hand, due to the fuzziness of the similarity matrix, which increases when the pairwise similarities are distributed around the mean value, the clustering algorithm becomes indecisive when grouping similar databases together. To face this problem, we design a learning algorithm to adjust the pairwise similarities between n multiple databases, in a way which minimizes a binary entropy loss function quantifying the fuzziness associated with the similarity matrix. Thus, the proposed algorithm becomes crisp in discriminating between the different database clusters.

3. Materials and Methods

In this section, we present our fuzziness reduction model applied to the pairwise similarities between n multiple databases and describe in details our coordinate descent-based clustering approach. Some definitions and notions relevant to this work need to be presented first.

3.1. Background and Relevant Concepts

In this subsection, we define the similarity measure between two transaction databases and present the process of generating and evaluating a given candidate clustering. We also define four clustering validity functions used to evaluate the clustering quality.

3.1.1. Similarity Measure

Each transactional database ${\mathcal{D}}_p$ is encoded as a hash-table to be defined as follows:

$$FIS({\mathcal{D}}_p,{\alpha }_p)=\left\{{\cup }_{k=1}^m\langle I_k,supp(I_k,{\mathcal{D}}_p)\rangle \right|supp(I_k,{\mathcal{D}}_p)\ge {\alpha }_p\}$$

(2)

where $p=0,\dots ,n-1$, n is the number of transactional databases, m is the number of frequent itemsets in ${\mathcal{D}}_p$, $I_k$ is the name of the k-th frequent itemset, $supp(I_k,{\mathcal{D}}_p)\in [0,1]$ is the support of $I_k$, which is the ratio of the number of rows in ${\mathcal{D}}_p$ containing $I_k$ to the total number of rows in ${\mathcal{D}}_p$, and ${\alpha }_p\in [0,1]$ is the minimum support threshold corresponding to ${\mathcal{D}}_p$, such that $supp(I_k,{\mathcal{D}}_p)\ge {\alpha }_p$. In this paper, FP-Growth [1] algorithm is used to mine the frequent itemsets in each database ${\mathcal{D}}_p$ as it only requires two passes over the whole database. Our proposed similarity measure is based on maximizing the number of global frequent itemsets (FIs) synthesized from the local FIs in each cluster. Precisely, to measure the similarity between two transactional databases ${\mathcal{D}}_q$ and ${\mathcal{D}}_p$, for $p=0,\dots ,n-2, q=p+1,\dots ,n-1$, we define the following function:

$$sim({\mathcal{D}}_p,{\mathcal{D}}_q)=\frac{\genfrac{}{}{0.0pt}{}{\sum \Psi (I_k,\{{\mathcal{D}}_p,{\mathcal{D}}_q\})}{k,I_k\in \{FIS({\mathcal{D}}_p,{\alpha }_p)\cap FIS({\mathcal{D}}_q,{\alpha }_q)\}}}{|FIS({\mathcal{D}}_p,{\alpha }_p)\cup FIS({\mathcal{D}}_q,{\alpha }_q)|}$$

(3)

where

$$\begin{array}{c}\hfill \Psi (I_k,\{{\mathcal{D}}_p,{\mathcal{D}}_q\})=\left\{\begin{array}{c}1,\mathrm{if}supp(I_k,\{{\mathcal{D}}_p,{\mathcal{D}}_q\})\ge {\alpha }_{p,q}\hfill \\ 0,\mathrm{otherwise}\hfill \end{array}\right.\end{array}$$

(4)

such that

$$\begin{array}{c}\hfill supp(I_k,\{{\mathcal{D}}_p,{\mathcal{D}}_q\})=\frac{supp(I_k,{\mathcal{D}}_p)\times |{\mathcal{D}}_p|+supp(I_k,{\mathcal{D}}_q)\times \left|{\mathcal{D}}_q\right|}{|{\mathcal{D}}_p|+|{\mathcal{D}}_q|}\end{array}$$

(5)

and

$$\begin{array}{c}\hfill {\alpha }_{p,q}=\frac{{\alpha }_p\times |{\mathcal{D}}_p|+{\alpha }_q\times \left|{\mathcal{D}}_q\right|}{|{\mathcal{D}}_p|+|{\mathcal{D}}_q|}\end{array}$$

(6)

We note that the operator $|·|$ is the cardinality of the set passed in as argument. Multiplying ${\alpha }_p$ by $|{\mathcal{D}}_p|$ returns the minimum number of transactions in which a frequent itemset $I_k$ should occur in ${\mathcal{D}}_p$. Therefore, ${\alpha }_{p,q}$ is the minimum percentage of transactions from the cluster $C_{p,q}=\{{\mathcal{D}}_p,{\mathcal{D}}_q\}$ containing the itemset $I_k$, i.e., $supp(I_k,C_{p,q})\ge {\alpha }_{p,q}$. In fact, the similarity measure $sim$ in Formula (3) takes into account the local minimum support threshold at each database to calculate a new threshold for each cluster. In this paper, instead of writing ’the similarity measure $sim$ in Formula (3)’, we often write $sim$ (3).

3.1.2. Clustering Generation and Evaluation

Let $\mathcal{C}(\mathcal{D},{\delta }_i)$ = $\{C_1,C_2,\dots ,C_k\}$ be a candidate clustering of $\mathcal{D}$ = $\{{\mathcal{D}}_0,{\mathcal{D}}_1,\dots ,{\mathcal{D}}_{n-1}\}$ produced at a given level of similarity ${\delta }_i\in [0,1]$, such that ${\cap }_{j=1}^k\left\{C_j\right\}=\varnothing$ and ${\cup }_{j=1}^k\left\{C_j\right\}=\mathcal{D}$. From a graph-theoretic perspective, each cluster $C_j$ represents a connected component in a similarity graph $G=(\mathcal{D},E)$, and an edge $({\mathcal{D}}_p,{\mathcal{D}}_q)$ is added to the list of edges E if and only if $sim({\mathcal{D}}_p,{\mathcal{D}}_q)\ge {\delta }_i$, where $p=0,\dots ,n-2,q=p+1,\dots ,n-1$.

Initially, $G=(\mathcal{D},E)$ has no edge, i.e., $E=\varnothing$. Then, at a given similarity level ${\delta }_i\in [0,1]$, edges $({\mathcal{D}}_p,{\mathcal{D}}_q)$ satisfying $sim({\mathcal{D}}_p,{\mathcal{D}}_q)\ge {\delta }_i$, are added to E. The level of similarity ${\delta }_i$ ($i=1,\dots ,m$) is chosen from the list of the m unique sorted pairwise similarities $sim({\mathcal{D}}_p,{\mathcal{D}}_q)$ computed between the n transaction databases, such that ${\delta }_1>{\delta }_2>\dots {\delta }_{i-1}>{\delta }_i>{\delta }_{i+1}>\dots >{\delta }_m$ and $m\le (n^2-n)/2$. After adding all the edges $({\mathcal{D}}_p,{\mathcal{D}}_q)$ at ${\delta }_i$, each graph component $C_j$$(j=1,\dots ,k)$ will be representing one database cluster in our candidate partitioning $\mathcal{C}(\mathcal{D},{\delta }_i)$. One can then use one of the clustering goodness measures shown in

Table 3. A summary of the clustering quality measures mentioned in this paper.

Clustering Quality [Reference]	Function (Equation)	Optimal Value
[20]	$\mathit{goodness}\left(\mathcal{D}\right)=\mathit{B}\left(\mathcal{D}\right)+\mathit{W}\left(\mathcal{D}\right)-\mathit{f}\left(\mathcal{D}\right)$ $\left\{\begin{array}{c}B\left(\mathcal{D}\right)={\sum }_{C_t,C_v\in C;t<v}{\sum }_{{\mathcal{D}}_p\in C_t,{\mathcal{D}}_q\in C_v;p<q}(1-sim({\mathcal{D}}_p,{\mathcal{D}}_q))\hfill \\ W\left(\mathcal{D}\right)={\sum }_{C_t\in C}{\sum }_{{\mathcal{D}}_p,{\mathcal{D}}_q\in C_t;p<q}sim({\mathcal{D}}_p,{\mathcal{D}}_q)\times 𝟙\{({\mathcal{D}}_p,{\mathcal{D}}_q)\in E\}\hfill \\ f\left(\mathcal{D}\right):\mathrm{number}\mathrm{of}\mathrm{clusters}.\hfill \end{array}\right.$	$$\begin{array}{c}\hfill maxgoodness\left(\mathcal{D}\right)\end{array}$$
[23]	${\mathit{goodness}}^{\mathbf{2}}\left(\mathcal{D}\right)=\frac{\mathit{sum}-\mathit{dist}\left(\mathcal{D}\right)}{({\mathit{n}}^{\mathbf{2}}-\mathit{n})/\mathbf{2}}+\frac{\mathit{coupling}\left(\mathcal{D}\right)}{({\mathit{n}}^{\mathbf{2}}-\mathit{n})/\mathbf{2}}+\frac{\mathit{f}\left(\mathcal{D}\right)-\mathbf{1}}{\mathit{n}-\mathbf{1}}$ $\left\{\begin{array}{c}sum-dist\left(\mathcal{D}\right)={\sum }_{C_t\in C}{\sum }_{{\mathcal{D}}_p,{\mathcal{D}}_q\in C_t;p<q}(1-sim({\mathcal{D}}_p,{\mathcal{D}}_q))\times 𝟙\{({\mathcal{D}}_p,{\mathcal{D}}_q)\in E\}\hfill \\ coupling\left(\mathcal{D}\right)={\sum }_{C_t,C_v\in C;t<v}{\sum }_{{\mathcal{D}}_p\in C_t,{\mathcal{D}}_q\in C_v;p<q}sim({\mathcal{D}}_p,{\mathcal{D}}_q)\hfill \end{array}\right.$	$$\begin{array}{c}\hfill mingoodnes{s}^2\left(\mathcal{D}\right)\end{array}$$
[21]	${\mathit{goodness}}^{\mathbf{3}}\left(\mathcal{D}\right)=\frac{\mathit{intra}-\mathit{sim}\left(\mathcal{D}\right)+\mathit{inter}-\mathit{dist}\left(\mathcal{D}\right)}{\mathit{f}\left(\mathcal{D}\right)}$ $\left\{\begin{array}{c}intra-sim\left(\mathcal{D}\right)=\frac{1}{f\left(\mathcal{D}\right)}{\sum }_{C_t\in C}\left\{\begin{array}{c}1,|C_t|=1\hfill \\ \frac{{\sum }_{{\mathcal{D}}_p,{\mathcal{D}}_q\in C_t}sim({\mathcal{D}}_p,{\mathcal{D}}_q)\times 𝟙\{({\mathcal{D}}_p,{\mathcal{D}}_q)\in E\}}{\left(\right|C_t{|}^2-\left|C_t\right|)/2},\left|C_t\right|>1\hfill \end{array}\right.\hfill \\ inter-dist\left(\mathcal{D}\right)=\left\{\begin{array}{c}0,f\left(\mathcal{D}\right)=1\hfill \\ {\sum }_{C_t,C_v\in C}\frac{2\times {\sum }_{{\mathcal{D}}_p\in C_t,{\mathcal{D}}_q\in C_v;p<q}(1-sim({\mathcal{D}}_p,{\mathcal{D}}_q))}{|C_t|\times |C_v|\times (f{\left(\mathcal{D}\right)}^2-f\left(\mathcal{D}\right))},f\left(\mathcal{D}\right)>1\hfill \end{array}\right.\hfill \end{array}\right.$	$$\begin{array}{c}\hfill maxgoodnes{s}^3\left(\mathcal{D}\right)\end{array}$$
[43,44]	$\mathit{SC}\left(\mathcal{D}\right)=\frac{\mathbf{1}}{\mathit{n}}{\sum }_{\mathit{p}=\mathbf{0}}^{\mathit{n}-\mathbf{1}}\mathit{s}\left({\mathcal{D}}_{\mathit{p}}\right)$ $\left\{\begin{array}{c}s\left({\mathcal{D}}_p\right)=\left\{\begin{array}{c}\frac{b\left({\mathcal{D}}_p\right)-a\left({\mathcal{D}}_p\right)}{max\{a\left({\mathcal{D}}_p\right),b\left({\mathcal{D}}_p\right)\}},\left|C_p\right|>1;\hfill \\ \\ 0,if|C_p|=1\hfill \end{array}\right.\hfill \\ \left\{\begin{array}{c}a\left({\mathcal{D}}_p\right)=\frac{{\sum }_{{\mathcal{D}}_p,{\mathcal{D}}_q\in C_p,p<q}(1-sim({\mathcal{D}}_p,{\mathcal{D}}_q))\times 𝟙\{({\mathcal{D}}_p,{\mathcal{D}}_q)\in E\}}{|C_p-1|}\hfill \\ b\left({\mathcal{D}}_p\right)={min}_{{\mathcal{D}}_p\notin C_q}\frac{1}{|C_q|}{\sum }_{{\mathcal{D}}_q\in C_q}(1-sim({\mathcal{D}}_p,{\mathcal{D}}_q))\hfill \end{array}\right.\hfill \end{array}\right.$	$$\begin{array}{c}\hfill maxSC\left(\mathcal{D}\right)\end{array}$$

to assess the quality of $\mathcal{C}(\mathcal{D},{\delta }_i)$.

Once we generate and evaluate all the $m\le (n^2-n)/2$ candidate clusterings, we report the global optimum (minimum or maximum) of the goodness measure and compare its corresponding clustering with the ground truth if it is known or with the clustering generated at the maximum point of the silhouette coefficient when the ground truth is unknown. In fact, the silhouette coefficient $SC\left(\mathcal{D}\right)\in [-1,1]$ proposed in [43,44] (See the last row in Table 3) could be used to verify the correctness of the cluster labels assigned to the n transactional databases. Precisely, a value $SC\left(\mathcal{D}\right)\approx 1$ suggests that the n transactional databases are highly matched to their own clusters and loosely matched to their neighboring clusters.

We should note that each clustering goodness measure in Table 3 depends on more than two monotonic functions. For instance, the quality measure $goodness$ (see the first row in Table 3) proposed in [20] is based on maximizing both the intra-cluster similarity $W\left(\mathcal{D}\right)$ (which is a non-decreasing function on the interval [0,1]) and the inter-cluster distance $B\left(\mathcal{D}\right)$ (which is a non-increasing function on the interval [0,1]), while minimizing the number of clusters $f\left(\mathcal{D}\right)$ (which is a non-increasing function on the interval [0,1]). Consequently, as it was shown via the experiments done in [25,26], most of the time, the graphs of the objectives functions in Table 3 show a non-convex behavior, which makes identifying the ideal partitioning a hard problem to solve without generating and evaluating all the candidate clusterings generated at the local optima.

3.2. Similarity Matrix Fuzziness Reduction

In this subsection, we present our fuzziness reduction model applied to the pairwise similarities between n multiple databases. Let $z_{p,q}={\theta }_{p,q}\times x_{p,q}$ be a weighted similarity, such that $x_{p,q}=sim({\mathcal{D}}_p,{\mathcal{D}}_q)$ is the similarity value between ${\mathcal{D}}_p$ and ${\mathcal{D}}_q$ using Formula (3) and ${\theta }_{p,q}$ is the weight value associated with $x_{p,q}$ where $p=0,\dots ,n-2,q=p+1,\dots ,n-1$. Let $g:\mathbb{R}\to ]0,1[$ be a continuous piecewise linear activation function and $\partial g$ be its partial derivative defined as follows:

$$\begin{array}{c}\hfill g(z_{p,q},ϵ)=max\left(z_{p,q},ϵ\right)-\frac{sgn(z_{p,q}-1+ϵ)+1}{2}\left(z_{p,q}-1+ϵ\right)\end{array}$$

(7)

$$\begin{array}{c}\hfill \frac{\partial g(z_{p,q},ϵ)}{\partial z_{p,q}}=\frac{sgn(z_{p,q}-ϵ)+1}{2}-\frac{sgn(z_{p,q}-1+ϵ)+1}{2}\end{array}$$

(8)

The graph plots of $g(z_{p,q},ϵ)$ and $\frac{\partial g(z_{p,q},ϵ)}{\partial z_{p,q}}$ with respect to $z_{p,q}$ are depicted in Figure 1a. The parameter $ϵ$ ensures that each value $z_{p,q}$ is within the range $[ϵ,1-ϵ]$ such that $ϵ$ is a very small number (e.g., $ϵ=1\mathrm{e}$–7) forcing $g(z_{p,q},ϵ)$ to be always above 0 and below 1, so that it can be plugged into our log-based loss function defined in (10).

3.2.1. Fuzziness Index

The fuzziness index of the pairwise similarity vector $X^T$ = $[sim({\mathcal{D}}_0,{\mathcal{D}}_1),sim({\mathcal{D}}_0,{\mathcal{D}}_2),\dots ,sim({\mathcal{D}}_{n-2},{\mathcal{D}}_{n-1})]$, also known as the entropy of the fuzzy set $X^T$ [45], and defined from ${\mathbb{R}}^{\left(\genfrac{}{}{0pt}{}{n}{2}\right)}$ to $[0,1]$, is given as follows:

$$\begin{array}{cc}\hfill Fuzziness\left(X\right)& =\frac{-2}{n^2-n}\left(X^T·{log}_2\left(X\right)+(1-X^T)·{log}_2(1-X)\right)\hfill \\ & =\frac{-2}{n^2-n}\sum _{p=0}^{n-2}\sum _{q=p+1}^{n-1}\left(sim({\mathcal{D}}_p,{\mathcal{D}}_q){log}_2\left(sim({\mathcal{D}}_p,{\mathcal{D}}_q)\right)+(1-sim({\mathcal{D}}_p,{\mathcal{D}}_q)){log}_2(1-sim({\mathcal{D}}_p,{\mathcal{D}}_q))\right)\hfill \end{array}$$

(9)

The smaller the value of $Fuzziness\left(X\right)$, the better the clustering performance, and vice-versa. In fact, reducing the fuzziness of the pairwise similarities will lead to a more crisp decision making when it comes to finding the optimal partitioning of the n multiple databases. Particularly, the fuzziness of the similarity matrix increases when the pairwise values are centered around $0.5$, resulting in more confusion when we need to decide whether two databases should be in the same cluster or not.

3.2.2. Proposed Model and Algorithm

To reduce the fuzziness associated with the $(n^2-n)/2$ pairwise similarities between the n transaction databases $\mathcal{D}$ = $\{{\mathcal{D}}_0,{\mathcal{D}}_1,\dots ,{\mathcal{D}}_{n-1}\}$, we need to make the similarity values that are above the mean value $\mu =0.5$ go closer to 1, and adjust the similarity values that are below $\mu =0.5$ to go closer to 0. To do so, we consider the minimization of the sum of the binary entropy loss functions over the $(n^2-n)/2$ weighed similarity values $z_{p,q}={\theta }_{p,q}\times x_{p,q}$ as follows:

$$\begin{array}{cc}\hfill \mathbf{arg}\underset{\mathbf{\theta }}{\mathbf{min}}\mathcal{H}(\theta ,ϵ)& = \mathbf{arg}\underset{\mathbf{\theta }}{\mathbf{min}}\frac{2}{n^2-n}\sum _{p = 0}^{n - 2}\sum _{q = p + 1}^{n - 1}\mathbf{H}\left(g(z_{p,q},ϵ)\right)\hfill \\ & = \mathbf{arg}\underset{\mathbf{\theta }}{\mathbf{min}}\frac{-2}{n^2-n}\sum _{p = 0}^{n - 2}\sum _{q = p + 1}^{n - 1}(g(z_{p,q},ϵ){log}_2\left(g(z_{p,q},ϵ)\right)+(1 - g(z_{p,q},ϵ)){log}_2(1 - g(z_{p,q},ϵ)))\hfill \\ & = \mathbf{arg}\underset{\mathbf{\theta }}{\mathbf{min}}\frac{-2}{n^2-n}\left(g({\theta }^T\odot X^T,ϵ)·{log}_2\left(g(\theta \odot X,ϵ)\right)+(1 - g({\theta }^T\odot X^T,ϵ))·{log}_2(1 - g(\theta \odot X,ϵ))\right)\hfill \end{array}$$

(10)

such that n is the number of databases, ${\theta }^T=[{\theta }_{0,1},{\theta }_{0,2},\dots ,{\theta }_{n-2,n-1}]$ represents the model weight vector, $z_{p,q}$ represents the weighted similarity ${\theta }_{p,q}\times sim({\mathcal{D}}_p,{\mathcal{D}}_q)$ and $g(z_{p,q},ϵ)$ is the activation function defined in (7). The graph plots of $\mathbf{H}\left(g(z_{p,q},ϵ)\right)$ and $\frac{\partial \mathbf{H}\left(g(z_{p,q},ϵ)\right)}{\partial \mathit{g}(z_{p,q},ϵ)}$ with respect to $\mathit{g}(z_{p,q},ϵ)$ are depicted in Figure 1b. Since the fuzziness of the similarity matrix is influenced by the weights associated with the pairwise similarities, the degree to which a pair of databases $({\mathcal{D}}_p,{\mathcal{D}}_q)$ belongs to the same cluster could be changed by adjusting the corresponding weight ${\theta }_{p,q}$, which is learned by minimizing (10) via gradient descent and back-propagation. The training equations are derived as follows:

$${\theta }_{p,q}={\theta }_{p,q}-\eta \frac{\partial \mathcal{H}(\theta ,ϵ)}{\partial {\theta }_{p,q}}$$

(11)

where

$$\begin{array}{cc}\hfill \frac{\partial \mathcal{H}(\theta ,ϵ)}{\partial {\theta }_{p,q}}& =\frac{-2}{n^2-n}\frac{\partial g(z_{p,q},ϵ)}{\partial z_{p,q}}\frac{\partial z_{p,q}}{\partial {\theta }_{p,q}}{log}_2\left(\frac{g(z_{p,q},ϵ)}{1-g(z_{p,q},ϵ)}\right)\hfill \\ \hfill & =\frac{-2}{n^2-n}\frac{\partial g(z_{p,q},ϵ)}{\partial z_{p,q}}{x}_{p,q}{log}_2\left(\frac{g(z_{p,q},ϵ)}{1-g(z_{p,q},ϵ)}\right)\hfill \end{array}$$

(12)

Let ${\eta }_0$ and $epochs$ be the initial learning rate and the maximum number of learning iterations, respectively. At each epoch i, the current learning rate $\eta$ decreases as follows:

$$\begin{array}{c}\hfill \eta ={\eta }_0\times (1-i/epochs)\end{array}$$

(13)

We note that selecting a large learning rate value may cause global minimum overshooting, whereas choosing a small learning rate may necessitate many iterations for the algorithm to converge. Hence, it is reasonable to let the learning rate decrease over time as the algorithm converges to the global minimum. In Figure 2 and Algorithm 1, we present in detail, the framework and the algorithm of the proposed fuzziness reduction model. The proposed learning Algorithm 1: SimFuzzinessReduction keeps adjusting the weight vector $\theta$ by moving in the opposite direction to the gradient of the loss function $\mathcal{H}(\theta ,ϵ)$ until it reaches the maximum number of iteration $epochs$ or until the magnitude of the gradient vector becomes below the minimum value $ϵ$. After convergence, we can feed the new similarity values $[g({\theta }_0\times sim({\mathcal{D}}_0,{\mathcal{D}}_1),ϵ),g({\theta }_0\times sim({\mathcal{D}}_0,{\mathcal{D}}_2),ϵ),\dots ,g({\theta }_0\times sim({\mathcal{D}}_{n-2},{\mathcal{D}}_{n-1}),ϵ)]$ to any similarity-based clustering algorithm in order to improve the quality of the produced clustering when the latter is trivial or irrelevant.

Algorithm 1: SimFuzzinessReduction

3.3. Proposed Coordinate Descent-Based Clustering

In this subsection, we present and discuss our proposed loss function and our coordinate descent-based clustering approach in detail. Unlike the gradient-based clustering in [25,26], our algorithm is learning-rate-free and needs to run at most $(n^2-n)/2$ learning cycles to converge to the global minimum, such that n is the number of transaction databases. In fact, at each iteration, the largest coordinate variable ${\theta }_{p,q}$ is selected and popped from a max-heap data structure (initially built by pushing the $(n^2-n)/2$ pairwise similarities onto the heap). Then, we minimize our quadratic convex hinge-based loss $\mathcal{L}\left(\theta \right)$ over ${\theta }_{p,q}$ which is then adjusted by moving in the opposite direction to the gradient of $\mathcal{L}\left(\theta \right)$. This process continues until satisfying a convergence test, which will be defined later in this subsection. Each bloc of selected coordinate variables ${\theta }_{p,q}$ that have the same value will form a set of edges to be added to our graph $G=(\mathcal{D},E)$. Determining the disjoint connected components in G after convergence will allow us to discover the optimal database clusters maximizing the intra-cluster similarity and the inter-cluster distance.

3.3.1. Proposed Loss Function and Algorithm

In order to implement our coordinate descent-based clustering, we propose a quadratic version of the hinge loss $\mathcal{L}\left(\theta \right):{\mathbb{R}}^{\left(\genfrac{}{}{0pt}{}{n}{2}\right)}\to [0,\frac{n^2-n}{4}]$, which is a convex function (see proof of Theorem 1) whose minimization problem is formulated as follows:

$$\begin{array}{c}\hfill \mathbf{arg}\underset{{\theta }^{\left(\mathit{i}\right)}}{\mathbf{min}}\mathcal{L}\left({\theta }^{\left(i\right)}\right)=\mathbf{arg}\underset{\begin{array}{c}{\theta }^{\left(\mathit{i}\right)}\end{array}}{\mathbf{min}}\sum _{r=0}^{n-2}\sum _{s=r+1}^{n-1}\frac{1}{2}max{(0,1-g\left({\theta }_{r,s}^{\left(i\right)}\right))}^2\end{array}$$

(14)

A simplified 3D graph plot of $\mathcal{L}\left(\theta \right)$ is depicted in Figure 3.

Initially, the weight vector ${\theta }^T$ is set to the $\left(\genfrac{}{}{0pt}{}{n}{2}\right)$ pairwise similarities $X^T=[sim({\mathcal{D}}_0,{\mathcal{D}}_1),sim({\mathcal{D}}_0,{\mathcal{D}}_2),\dots ,sim({\mathcal{D}}_{n-2},{\mathcal{D}}_{n-1})]$, and then each weight component of ${\theta }^T$ is pushed onto a max-heap data structure. At each iteration $i=1,\dots ,\left(\genfrac{}{}{0pt}{}{n}{2}\right)$, the weight ${\theta }_{p,q}^{\left(i\right)}$ ($p=0,\dots ,n-2$, $q=p+1,\dots ,n-1$) associated with the current largest similarity value $sim({\mathcal{D}}_p,{\mathcal{D}}_q)$ is popped from the max-heap and is updated as follows:

$$\begin{array}{cc}\hfill {\theta }_{p,q}^{\left(i\right)}& ={\theta }_{p,q}^{(i-1)}-\eta \frac{\partial \mathcal{L}\left({\theta }^{(i-1)}\right)}{\partial {\theta }_{p,q}^{(i-1)}}\hfill \\ \hfill & ={\theta }_{p,q}^{(i-1)}+\eta (1-g\left({\theta }_{p,q}^{(i-1)}\right))\frac{\partial g\left({\theta }_{p,q}^{(i-1)}\right)}{\partial {\theta }_{p,q}^{(i-1)}}\hfill \end{array}$$

(15)

Such that $g:\mathbb{R}\to [0,1]$ is a differentiable activation function defined as follows:

$$\begin{array}{c}\hfill g\left({\theta }_{p,q}\right)=max({\theta }_{p,q},0)-\frac{sgn({\theta }_{p,q}-1)+1}{2}\times ({\theta }_{p,q}-1)\end{array}$$

(16)

and its partial derivative with respect to the weight ${\theta }_{p,q}$ is:

$$\begin{array}{c}\hfill \frac{\partial g\left({\theta }_{p,q}\right)}{\partial {\theta }_{p,q}}=\frac{sgn\left({\theta }_{p,q}\right)+1}{2}-\frac{sgn({\theta }_{p,q}-1)+1}{2}\end{array}$$

(17)

We note that $sgn:\mathbb{R}\to \{-1,1\}$ is the signum function. The usage of $g(·)$ ensures that each weight ${\theta }_{p,q}$ is within the range $[0,1]$. As there is no learning rate and schedule to choose for our coordinate descent-based algorithm, we set $\eta$ to 1.

Theorem 1.

$\mathcal{L}\left(\theta \right)$ (14) is convex satisfying the following inequality [46]:

$$\begin{array}{cc}\hfill \mathcal{L}((1-\epsilon ){\theta }^{(i+1)}+\epsilon {\theta }^{\left(i\right)})& \le (1-\epsilon )\mathcal{L}\left({\theta }^{(i+1)}\right)+\epsilon \mathcal{L}\left({\theta }^{\left(i\right)}\right)\hfill \\ \hfill \mathrm{for}\mathrm{all}{\theta }^{(i+1)},{\theta }^{\left(i\right)}& \in {\mathbb{R}}^{\left(\genfrac{}{}{0pt}{}{n}{2}\right)}\mathrm{with}\epsilon \in [0,1]\hfill \end{array}$$

(18)

Proof. 

To prove the convexity of $\mathcal{L}\left(\theta \right)$, we can show that its Hessian matrix $\mathbf{H}\mathcal{L}$ is positive semi-definite as follows:

$$\begin{array}{c}\hfill \mathbf{H}\mathcal{L}=\frac{{\partial }^2\mathcal{L}}{\partial {\theta }_{p,q}\partial {\theta }_{r,s}}=\left[\begin{array}{c}\frac{{\partial }^2\mathcal{L}\left(\theta \right)}{\partial {{\theta }^2}_{0,1}}=1,\frac{{\partial }^2\mathcal{L}\left(\theta \right)}{\partial {\theta }_{0,1}\partial {\theta }_{0,2}}=0,\dots ,\frac{{\partial }^2\mathcal{L}\left(\theta \right)}{\partial {\theta }_{0,1}\partial {\theta }_{n-2,n-1}}=0\\ \frac{{\partial }^2\mathcal{L}\left(\theta \right)}{\partial {\theta }_{0,2}\partial {\theta }_{0,1}}=0,\frac{{\partial }^2\mathcal{L}\left(\theta \right)}{\partial {{\theta }^2}_{0,2}}=1,\dots ,\frac{{\partial }^2\mathcal{L}\left(\theta \right)}{\partial {\theta }_{0,2}\partial {\theta }_{n-2,n-1}}=0\\ ⋮⋮\ddots ⋮\\ \frac{{\partial }^2\mathcal{L}\left(\theta \right)}{\partial {\theta }_{n-2,n-1}\partial {\theta }_{0,1}}=0,\frac{{\partial }^2\mathcal{L}\left(\theta \right)}{\partial {\theta }_{n-2,n-1}\partial {\theta }_{0,2}}=0,\dots ,\frac{{\partial }^2\mathcal{L}\left(\theta \right)}{{\partial }^2{\theta }_{n-2,n-1}}=1\end{array}\right]\end{array}$$

Since $\mathbf{H}$ is positive semi-definite satisfying ${\mathbf{x}}^{\mathbf{T}}\mathbf{H}\mathbf{x}\ge 0$ for all $\mathbf{x}\in {\mathbb{R}}^{\left(\genfrac{}{}{0pt}{}{n}{2}\right)}$, $\mathcal{L}\left(\theta \right)$ is convex, and therefore guarantees convergence to the global minimum. □

In order to reach the global minimum of $\mathcal{L}\left(\theta \right)$ (i.e., $\mathbf{min}\mathcal{L}\left(\theta \right)=0$), our learning algorithm needs to set the weight vector $\mathit{\theta }$ to $\overrightarrow{\mathbf{1}}$ (i.e., the unit vector). Consequently, the intra-cluster similarity will reach its maximum value and all the n databases will be put into the same cluster resulting in a meaningless partitioning. Therefore, in order to prevent this scenario from occurring, we need to assess the clustering quality after popping all the coordinate variables that have the same weight ${\theta }_{p,q}$ (i.e., a block of weights having the same value) from the max-heap. This corresponds to generating one candidate clustering by adding the list of edges $({\mathcal{D}}_p,{\mathcal{D}}_q)$ satisfying $sim({\mathcal{D}}_p,{\mathcal{D}}_q)\ge {\theta }_{p,q}$ to the graph $G=(\mathcal{D},E)$. Afterward, we need a stopping condition to terminate our algorithm if the current candidate clustering quality is judged to be the optimal one in terms of the similarity-intra cluster $W_{{\theta }^{\left(i\right)}}\left(\mathcal{D}\right)$ and the number of clusters $f_{{\theta }^{\left(i\right)}}\left(\mathcal{D}\right)$. For this purpose, we define the following quasi-convex loss function evaluated at the i-th iteration:

$$\begin{array}{cc}\hfill L\left({\theta }^{\left(i\right)}\right)& =\frac{1}{2}{(f_{{\theta }^{\left(i\right)}}\left(\mathcal{D}\right)-W_{{\theta }^{\left(i\right)}}\left(\mathcal{D}\right))}^2\hfill \\ \hfill & =\frac{1}{2}{(f_{{\theta }^{\left(i\right)}}\left(\mathcal{D}\right)-\phi \left({{\theta }^{\left(i\right)}}^T\right)·X)}^2\hfill \\ \hfill & =\frac{1}{2}{(f_{{\theta }^{\left(i\right)}}\left(\mathcal{D}\right)-\sum _{p=0}^{n-2}\sum _{q=p+1}^{n-1}sim({\mathcal{D}}_p,{\mathcal{D}}_q)\times \phi \left({\theta }_{p,q}^{\left(i\right)}\right))}^2\hfill \end{array}$$

(19)

where $\phi :{\mathbb{R}}^{\left(\genfrac{}{}{0pt}{}{n}{2}\right)}\to {\{0,1\}}^{\left(\genfrac{}{}{0pt}{}{n}{2}\right)}$, $\phi \left(\theta \right)=\frac{sgn(\theta -1)+1}{2}$.

Our algorithm terminates right after it reaches the global minimum of $L(·)$. In other words, if $L\left({\theta }^{\left(i\right)}\right)\le L\left({\theta }^{(i-1)}\right)$, then we continue updating the weight vector, the clustering labels and save the optimal partitioning found so far. Otherwise, the algorithm terminates as it has reached the global minimum $L\left({\theta }^{(i-1)}\right)$, and therefore, the optimal partitioning saved so far is returned as the ideal clustering of the n transactional databases. This stopping condition is only possible due to the quasi-convexity of $L(·)$.

Theorem 2.

$L\left(\theta \right)$ (19) is quasi-convex satisfying the following inequality [46]:

$$\begin{array}{cc}\hfill L((1-\epsilon ){\theta }^{(i+1)}+\epsilon {\theta }^{\left(i\right)})& \le \max \{L\left({\theta }^{(i+1)}\right),L\left({\theta }^{\left(i\right)}\right)\}\hfill \\ \hfill \mathrm{for}\mathrm{all}{\theta }^{(i+1)},{\theta }^{\left(i\right)}& {\in \mathbb{R}}^{\left(\genfrac{}{}{0pt}{}{n}{2}\right)}\mathrm{with}\epsilon \in [0,1]\hfill \end{array}$$

(20)

Proof. 

To prove the quasi-convexity of $L\left(\theta \right)$, we need to demonstrate the validity of (20). First, since $f_{\theta }\left(D\right)$ is a decreasing function on the range [0,1], it is then both quasi-concave and quasi-convex satisfying the following: $f((1-\epsilon ){\theta }^{(i+1)}+\epsilon {\theta }^{\left(i\right)})\le \max \{f\left({\theta }^{(i+1)}\right),f\left({\theta }^{\left(i\right)}\right)\}$ for all ${\theta }^{(i+1)},{\theta }^{\left(i\right)}{\in \mathbb{R}}^{\left(\genfrac{}{}{0pt}{}{n}{2}\right)}$ with $\epsilon \in [0,1]$. Since $W_{\theta }\left(D\right)$ is an increasing function on the range [0,1], it is then both quasi-concave and quasi-convex satisfying the following: $W((1-\epsilon ){\theta }^{(i+1)}+\epsilon {\theta }^{\left(i\right)})\ge \min \{W\left({\theta }^{(i+1)}\right),W\left({\theta }^{\left(i\right)}\right)\}$ for all ${\theta }^{(i+1)},{\theta }^{\left(i\right)}{\in \mathbb{R}}^{\left(\genfrac{}{}{0pt}{}{n}{2}\right)}$ with $\epsilon \in [0,1]$. By subtracting the two last inequalities, we obtain: $(f((1-\epsilon ){\theta }^{(i+1)}+\epsilon {\theta }^{\left(i\right)})-W\left((1-\epsilon ){\theta }^{(i+1)}\right)+\epsilon {\theta }^{\left(i\right)})\le (\max \{f\left({\theta }^{(i+1)}\right),f\left({\theta }^{\left(i\right)}\right)\}-\min \{W\left({\theta }^{(i+1)}\right),W\left({\theta }^{\left(i\right)}\right)\})$. Since $f\left({\theta }^{(i+1)}\right)\le f\left({\theta }^{\left(i\right)}\right)$ and $W\left({\theta }^{(i+1)}\right)\ge W\left({\theta }^{\left(i\right)}\right)$, the right side of the resulting inequality is equal to $f\left({\theta }^{\left(i\right)}\right)-W\left({\theta }^{\left(i\right)}\right)$, which could be set equal to $\max \{f\left({\theta }^{(i+1)}\right)-W\left({\theta }^{(i+1)}\right),f\left({\theta }^{\left(i\right)}\right)-W\left({\theta }^{\left(i\right)}\right)\}$. Finally, by squaring and dividing both sides of the inequality by 2, we get a variation on the Jensen inequality for quasi-convex functions [46] as defined in (20). Hence, $L\left(\theta \right)$ is quasi-convex. □

3.3.2. Time Complexity Analysis

In this subsection, we analyze the time complexity of our coordinate descent-based clustering algorithm presented in Algorithm 2, named CDClustering, which depends on the two subroutines presented in Algorithm 3: $union$ and Algorithm 4: $cluster$. We note that the superscript i enclosed in round brackets, i.e., ${\theta }_{p,q}^{\left(i\right)}$, is used to indicate the iteration number at which a given variable ${\theta }_{p,q}$ has been assigned a value. The proposed algorithm takes as argument the $\left(\genfrac{}{}{0pt}{}{n}{2}\right)$ pairwise similarities $X^T=[sim({\mathcal{D}}_0,{\mathcal{D}}_1),sim({\mathcal{D}}_0,{\mathcal{D}}_2),\dots ,sim({\mathcal{D}}_{n-2},{\mathcal{D}}_{n-1})]$ and outputs the optimal clustering minimizing our proposed loss function $\mathcal{L}\left(\theta \right)$ (14). First, the weight vector ${\theta }^T$ is initially set equal to $X^T$. Afterward, coordinate descent and back-propagation are used to search for the optimal weight vector ${\theta }^T$, which minimizes our hinge-based objective $\mathcal{L}\left(\theta \right)$. Through each learning cycle i, one coordinate variable ${\theta }_{p,q}$ is popped from a max-heap. Then, ${\theta }_{p,q}$ is updated by making the optimal step in the opposite direction to the gradient of $\mathcal{L}\left(\theta \right)$. The weights ${\theta }_{p,q}$ ($p=0,\dots ,n-2,q=p+1,\dots ,n-1$) attaining the maximum value of 1 will have their corresponding database pairs $({\mathcal{D}}_p,{\mathcal{D}}_p)$ put into the same cluster. By using a max-heap data structure within our coordinate descent algorithm, we optimally choose the current largest variable ${\theta }_{p,q}^{\left(i\right)}$ at each iteration i such that taking the partial derivative of our loss $\mathcal{L}\left(\theta \right)$ with respect to ${\theta }_{p,q}$ allows us to attain the next steepest descent minimizing $\mathcal{L}\left(\theta \right)$ without using a learning rate. This way, the maximum number of iterations required for our algorithm to converge is less than or equal to $(n^2-n)/2$, i.e., the number of the pairwise similarities. Initially, the number of clusters $f_{\theta }\left(\mathcal{D}\right)$ is set equal to the number of transactional databases n. Then, in order to keep track of the database clusters, their number $f_{\theta }\left(\mathcal{D}\right)$ and their sizes, we implement a disjoint-set data structure [47], which consists of an array A[$0,\dots ,n-1$] of n integers managed by two main operations: cluster and union. Each cluster $C_p$ is represented by a tree whose root index p satisfies $A\left[p\right]=-1$, and a database ${\mathcal{D}}_q$ belonging to the cluster $C_p$ satisfies $A\left[q\right]=p$. Therefore, the cluster function is called recursively to find the label assigned to the database index p (passed in as argument) by moving down the tree towards the root (i.e., $A\left[p\right]=-1$). On the other hand, the union procedure links two disjoint clusters $C_p$ and $C_q$ by making the root of the smaller tree point to the root of the larger one in A[$0,\dots ,n-1$]. The algorithms corresponding to union and cluster are presented in Algorithm 3 and Algorithm 4, respectively. Let $s=\left(\genfrac{}{}{0pt}{}{n}{2}\right)$ be the size of the weight vector ${\theta }^T$. The time complexity of building the max-heap is $O\left(s\right)$ and the time complexity of the proposed Algorithm 2: CDClustering is $O(s+h{log}_2\left(n\right))$, such that $h\in [1,s]$ is the number of learning cycles run until global minimum convergence, and $O\left({log}_2\left(n\right)\right)$ is the time complexity of one pop operation from the heap. The proposed model is also illustrated in Figure 4. Since it is meaningless to return a single cluster consisting of all the n databases, if the clustering obtained at step (10a) is trivial (i.e., all the n databases are put together in one class or each single database stands alone in its own cluster), then we first need to run the model proposed in Figure 2 on the pairwise similarities to reduce the associated intrinsic fuzziness measured in (9). Afterward, we can apply the proposed model Figure 4 on the new adjusted similarity values to obtain more relevant results.

Algorithm 2: CDClustering

Algorithm 3: union

Algorithm 4: cluster

4. Performance Evaluation

To assess the performance of the proposed clustering algorithm, we carried out numerous experiments on real and synthetic datasets, including Zoo [48], Iris [48], Mushroom [48] and T10I4D100K [49]. To simulate a multi-database environment, we have partitioned each dataset horizontally into n partitions ${\mathcal{D}}_1,{\mathcal{D}}_2,\dots ,{\mathcal{D}}_n$, such that $n\in \{12,10,6,4\}$. Afterward, given a minimum support threshold $\alpha \in \{0.5,0.2,0.03\}$, we run FP-Growth [1] on each partition ${\mathcal{D}}_i$ ($i=1,\dots n$) to discover the local frequent itemsets (FIs) corresponding to each partition. All the details related to the partition sizes and their corresponding FIs are shown in

Table A1. Description of the datasets used in our experiments with their database partitions and the number frequent itemsets (FIs) mined from each database partition ${\mathcal{D}}_i$ and the number of FIs mined from each cluster $C_j$ under a threshold $\alpha$.

Dataset Name/Ref	Number of Rows	Number of Rows in Partition ${\mathcal{D}}_{\mathit{i}}$	Number of $\mathit{FIS}({\mathcal{D}}_{\mathit{i}},\mathit{\alpha })$ from Partition ${\mathcal{D}}_{\mathit{i}}$	Number of $\mathit{FIS}(\mathcal{D},\mathit{\alpha })$ from Dataset $\mathcal{D}$	Ground Truth Clustering	Number of $\mathit{FIS}({\mathit{C}}_{\mathit{j}},\mathit{\alpha })$ from Cluster ${\mathit{C}}_{\mathit{j}}$
Mushroom [48] (2 classes)	8124	$|{\mathcal{D}}_1|=3916$ ($C_1$) $|{\mathcal{D}}_2|=1402$ ($C_2$) $|{\mathcal{D}}_3|=1402$ ($C_2$) $|{\mathcal{D}}_4|=1404$ ($C_2$)	$\left|FIS\right({\mathcal{D}}_1,0.5\left)\right|=375$ $\left|FIS\right({\mathcal{D}}_2,0.5\left)\right|=2063$ $\left|FIS\right({\mathcal{D}}_3,0.5\left)\right|=\mathrm{32,911}$ $\left|FIS\right({\mathcal{D}}_4,0.5\left)\right|=807$	$\left|FIS\right(\mathcal{D},0.5\left)\right|=151$	${\mathit{C}}_{\mathbf{1}}=\left\{{\mathcal{D}}_{\mathbf{1}}\right\},$ ${\mathit{C}}_{\mathbf{2}}=\{{\mathcal{D}}_{\mathbf{4}},{\mathcal{D}}_{\mathbf{3}},{\mathcal{D}}_{\mathbf{2}}\}$	$\left|FIS\right(C_1,0.5\left)\right|=375$ $\left|FIS\right(C_2,0.5\left)\right|=1441$
Zoo [48] (7 classes)	101	$|{\mathcal{D}}_1|=20$ ($C_1$) $|{\mathcal{D}}_2|=21$ ($C_1$) $|{\mathcal{D}}_3|=10$ ($C_2$) $|{\mathcal{D}}_4|=10$ ($C_2$) $|{\mathcal{D}}_5|=5$ ($C_3$) $|{\mathcal{D}}_6|=6$ ($C_4$) $|{\mathcal{D}}_7|=7$ ($C_4$) $|{\mathcal{D}}_8|=2$ ($C_5$) $|{\mathcal{D}}_9|=2$ ($C_5$) $|{\mathcal{D}}_{10}|=4$ ($C_6$) $|{\mathcal{D}}_{11}|=4$ ($C_6$) $|{\mathcal{D}}_{12}|=10$ ($C_7$)	$\left|FIS\right({\mathcal{D}}_1,0.5\left)\right|=\mathrm{24,383}$ $\left|FIS\right({\mathcal{D}}_2,0.5|=\mathrm{30,975}$ $\left|FIS\right({\mathcal{D}}_3,0.5\left)\right|=\mathrm{30,719}$ $\left|FIS\right({\mathcal{D}}_4,0.5\left)\right|=\mathrm{32,767}$ $\left|FIS\right({\mathcal{D}}_5,0.5\left)\right|=\mathrm{20,479}$ $\left|FIS\right({\mathcal{D}}_6,0.5|=\mathrm{65,535}$ $\left|FIS\right({\mathcal{D}}_7,0.5\left)\right|=\mathrm{65,535}$ $\left|FIS\right({\mathcal{D}}_8,0.5\left)\right|=\mathrm{114,687}$ $\left|FIS\right({\mathcal{D}}_9,0.5\left)\right|=\mathrm{98,303}$ $\left|FIS\right({\mathcal{D}}_{10},0.5\left)\right|=\mathrm{53,247}$ $\left|FIS\right({\mathcal{D}}_{11},0.5\left)\right|=\mathrm{57,343}$ $\left|FIS\right({\mathcal{D}}_{12},0.5\left)\right|=\mathrm{28,671}$	$\left|FIS\right(\mathcal{D},0.5\left)\right|=0$	${\mathit{C}}_{\mathbf{1}}=\{{\mathcal{D}}_{\mathbf{2}},{\mathcal{D}}_{\mathbf{1}}\},$ ${\mathit{C}}_{\mathbf{2}}=\{{\mathcal{D}}_{\mathbf{4}},{\mathcal{D}}_{\mathbf{3}}\},$ ${\mathit{C}}_{\mathbf{3}}=\left\{{\mathcal{D}}_{\mathbf{5}}\right\},$ ${\mathit{C}}_{\mathbf{4}}=\{{\mathcal{D}}_{\mathbf{7}},{\mathcal{D}}_{\mathbf{6}}\},$ ${\mathit{C}}_{\mathbf{5}}=\{{\mathcal{D}}_{\mathbf{9}},{\mathcal{D}}_{\mathbf{8}}\},$ ${\mathit{C}}_{\mathbf{6}}=\{{\mathcal{D}}_{\mathbf{11}},{\mathcal{D}}_{\mathbf{10}}\},$ ${\mathit{C}}_{\mathbf{7}}=\left\{{\mathcal{D}}_{\mathbf{12}}\right\},$	$\left|FIS\right(C_1,0.5\left)\right|=\mathrm{25,087}$ $\left|FIS\right(C_2,0.5\left)\right|=\mathrm{28,671}$ $\left|FIS\right(C_3,0.5\left)\right|=2479$ $\left|FIS\right(C_4,0.5\left)\right|=\mathrm{49,151}$ $\left|FIS\right(C_5,0.5\left)\right|=\mathrm{57,343}$ $\left|FIS\right(C_6,0.5\left)\right|=\mathrm{45,055}$ $\left|FIS\right(C_7,0.5\left)\right|=\mathrm{28,671}$
Iris [48] (3 classes)	150	$|{\mathcal{D}}_1|=25$ ($C_1$) $|{\mathcal{D}}_2|=25$ ($C_1$) $|{\mathcal{D}}_3|=25$ ($C_2$) $|{\mathcal{D}}_4|=25$ ($C_2$) $|{\mathcal{D}}_5|=25$ ($C_3$) $|{\mathcal{D}}_6|=25$ ($C_3$)	$\left|FIS\right({\mathcal{D}}_1,0.2\left)\right|=5$ $\left|FIS\right({\mathcal{D}}_2,0.2\left)\right|=6$ $\left|FIS\right({\mathcal{D}}_3,0.2\left)\right|=2$ $\left|FIS\right({\mathcal{D}}_4,0.2\left)\right|=2$ $\left|FIS\right({\mathcal{D}}_5,0.2\left)\right|=2$ $\left|FIS\right({\mathcal{D}}_6,0.2\left)\right|=5$	$\left|FIS\right(\mathcal{D},0.2\left)\right|=0$	${\mathit{C}}_{\mathbf{1}}=\{{\mathcal{D}}_{\mathbf{2}},{\mathcal{D}}_{\mathbf{1}}\},$ ${\mathit{C}}_{\mathbf{2}}=\{{\mathcal{D}}_{\mathbf{4}},{\mathcal{D}}_{\mathbf{3}}\},$ ${\mathit{C}}_{\mathbf{3}}=\{{\mathcal{D}}_{\mathbf{6}},{\mathcal{D}}_{\mathbf{5}}\}$	$\left|FIS\right(C_1,0.2\left)\right|=3$ $\left|FIS\right(C_2,0.2\left)\right|=1$ $\left|FIS\right(C_3,0.2\left)\right|=2$
T10I4D100K [49] (unknown classes)	100,000	$|D_i|=\mathrm{10,000}$ rows, $i=1\dots 10$	$\left|FIS\right({\mathcal{D}}_1,0.03\left)\right|=58$ $\left|FIS\right({\mathcal{D}}_2,0.03\left)\right|=58$ $\left|FIS\right({\mathcal{D}}_3,0.03\left)\right|=62$ $\left|FIS\right({\mathcal{D}}_4,0.03\left)\right|=57$ $\left|FIS\right({\mathcal{D}}_5,0.03\left)\right|=62$ $\left|FIS\right({\mathcal{D}}_6,0.03\left)\right|=63$ $\left|FIS\right({\mathcal{D}}_7,0.03\left)\right|=63$ $\left|FIS\right({\mathcal{D}}_8,0.03\left)\right|=59$ $\left|FIS\right({\mathcal{D}}_9,0.03\left)\right|=61$ $\left|FIS\right({\mathcal{D}}_{10},0.03\left)\right|=62$	$\left|FIS\right(\mathcal{D},0.03\left)\right|=50$	Seven clusters found via the silhouette coefficient [43] ${\mathit{C}}_{\mathbf{1}}=\left\{{\mathcal{D}}_{\mathbf{1}}\right\},$ ${\mathit{C}}_{\mathbf{2}}=\left\{{\mathcal{D}}_{\mathbf{2}}\right\},$ ${\mathit{C}}_{\mathbf{3}}=\left\{{\mathcal{D}}_{\mathbf{3}}\right\},$ ${\mathit{C}}_{\mathbf{4}}=\{{\mathcal{D}}_{\mathbf{5}},{\mathcal{D}}_{\mathbf{4}}\},$ ${\mathit{C}}_{\mathbf{5}}=\left\{{\mathcal{D}}_{\mathbf{6}}\right\},$ ${\mathit{C}}_{\mathbf{6}}=\left\{{\mathcal{D}}_{\mathbf{7}}\right\},$ ${\mathit{C}}_{\mathbf{7}}=\{{\mathcal{D}}_{\mathbf{10}},{\mathcal{D}}_{\mathbf{9}},{\mathcal{D}}_{\mathbf{8}}\}$	$\left|FIS\right(C_1,0.03\left)\right|=58$ $\left|FIS\right(C_2,0.03\left)\right|=58$ $\left|FIS\right(C_3,0.03\left)\right|=62$ $\left|FIS\right(C_4,0.03\left)\right|=59$ $\left|FIS\right(C_5,0.03\left)\right|=63$ $\left|FIS\right(C_6,0.03\left)\right|=59$ $\left|FIS\right(C_7,0.03\left)\right|=61$
Figure A1 [20] (unknown classes)	24	$|{\mathcal{D}}_1|=3$ $|{\mathcal{D}}_2|=3$ $|{\mathcal{D}}_3|=3$ $|{\mathcal{D}}_4|=4$ $|{\mathcal{D}}_5|=4$ $|{\mathcal{D}}_6|=3$ $|{\mathcal{D}}_7|=4$	$\left|FIS\right({\mathcal{D}}_1,0.42\left)\right|=3$ $\left|FIS\right({\mathcal{D}}_2,0.42\left)\right|=3$ $\left|FIS\right({\mathcal{D}}_3,0.42\left)\right|=5$ $\left|FIS\right({\mathcal{D}}_4,0.42\left)\right|=7$ $\left|FIS\right({\mathcal{D}}_5,0.42\left)\right|=7$ $\left|FIS\right({\mathcal{D}}_6,0.42\left)\right|=5$ $\left|FIS\right({\mathcal{D}}_7,0.42\left)\right|=3$	$\left|FIS\right(\mathcal{D},0.42\left)\right|=0$	Three clusters found via the silhouette coefficient [43] ${\mathit{C}}_{\mathbf{1}}=\{{\mathcal{D}}_{\mathbf{3}},{\mathcal{D}}_{\mathbf{2}},{\mathcal{D}}_{\mathbf{1}}\}$ ${\mathit{C}}_{\mathbf{2}}=\left\{{\mathcal{D}}_{\mathbf{4}}\right\}$ ${\mathit{C}}_{\mathbf{3}}=\{{\mathcal{D}}_{\mathbf{7}},{\mathcal{D}}_{\mathbf{6}},{\mathcal{D}}_{\mathbf{5}}\}$	$\left|FIS\right(C_1,0.42\left)\right|=3$ $\left|FIS\right(C_2,0.42\left)\right|=7$ $\left|FIS\right(C_3,0.42\left)\right|=3$

. We note that the fifth column of Table A1 reports the number of FIs discovered in the entire dataset, whereas the most right column of the same table reports the number of FIs aggregated from the local FIs mined from the partitions in each cluster.

The proposed similarity measure $sim$ (3) is called on the $(n^2-n)/2$ pairs of FIs to compute the $n\times n$ similarity matrices shown in Figure A1a, Figure A2a, Figure A3a, Figure A4a, Figure A5a, Figure A6a, and Figure A7a. Next, using the obtained pairwise similarities, candidate clusterings are produced via the process described in Section 3.1.2, and then evaluated using the clustering quality measures defined in Table 3, including $SC\left(\mathcal{D}\right)$ [43], $goodnes{s}^3\left(\mathcal{D}\right)$ [21], $goodnes{s}^2\left(\mathcal{D}\right)$ [23], $goodness\left(\mathcal{D}\right)$ [20] and our proposed loss function $\mathcal{L}\left(\theta \right)$ (14). The graphs corresponding to the studied goodness measures are shown in Figure A1b, Figure A2b, Figure A3b, Figure A4b, Figure A5b, Figure A6b, and Figure A7b, where the optimal point (maximum or minimum) of each objective function is depicted as a black dot on its corresponding graph, except that for the graph of our loss function $\mathcal{L}\left(\theta \right)$, there is a red dot representing the value $\mathcal{L}(arg\underset{\begin{array}{c}\theta \end{array}}{min}L\left(\theta \right))$ (i.e., the optimal point at which our algorithm terminates). It is worth mentioning that due to scale differences, we sometimes multiply or divide our loss function $\mathcal{L}\left(\theta \right)$, $goodnes{s}^3\left(\mathcal{D}\right)$ [21] and $goodnes{s}^2\left(\mathcal{D}\right)$ [23] by a scaling number to stretch or shrink their graphs in the direction of the y-axis. The experimental results depicted in Figure A1, Figure A2, Figure A3, Figure A4, Figure A5, Figure A6, and Figure A7 are summarized in

Table A2. Clustering results illustrated in Figure A1, Figure A2, Figure A3, Figure A4, Figure A5, Figure A6, and Figure A7 after using the clustering quality measures ${\mathit{goodness}}^{\mathbf{3}}\left(\mathcal{D}\right)$ [21], ${\mathit{goodness}}^{\mathbf{2}}\left(\mathcal{D}\right)$ [23], the silhouette coefficient $\mathit{S}\mathit{C}\left(\mathcal{D}\right)$ [43], $\mathit{goodness}\left(\mathcal{D}\right)$ [20] and our proposed objective function $\mathcal{L}\left(\mathbf{\theta }\right)$, where $\delta$ is the level of similarity at which each clustering evaluation/loss function reaches its optimal value.

Dataset Name/Ref	Silhouette Coefficient	Clustering Result Using Proposed Objective		Clustering Result/ Optimal Value		Clustering Result/ Optimal Value		Clustering Result/ Optimal Value
	$\mathit{max}$ $\mathit{SC}\left(\mathcal{D}\right)$	Clusters	$\mathcal{L}(\mathbf{arg}\underset{\begin{array}{c}\mathit{\theta }\end{array}}{\mathbf{min}}\mathit{L}\left(\mathit{\theta }\right))$	Clusters	$\mathit{max}$ $\mathit{goodness}\left(\mathcal{D}\right)$	Clusters	$\mathit{min}$ ${\mathit{goodness}}^2\left(\mathcal{D}\right)$	Clusters	$\mathit{max}$ ${\mathit{goodness}}^3\left(\mathcal{D}\right)$
Figure A1 $7\times 7$ [20]	0.46 at $\delta =0.444$	${\mathit{C}}_{\mathbf{1}}=\{{\mathcal{D}}_{\mathbf{3}},{\mathcal{D}}_{\mathbf{2}},{\mathcal{D}}_{\mathbf{1}}\},$ ${\mathit{C}}_{\mathbf{2}}=\left\{{\mathcal{D}}_{\mathbf{4}}\right\},$ ${\mathit{C}}_{\mathbf{3}}=\{{\mathcal{D}}_{\mathbf{7}},{\mathcal{D}}_{\mathbf{6}},{\mathcal{D}}_{\mathbf{5}}\}$	2.004 at $\delta =0.444$	${\mathit{C}}_{\mathbf{1}}=\{{\mathcal{D}}_{\mathbf{3}},{\mathcal{D}}_{\mathbf{2}},{\mathcal{D}}_{\mathbf{1}}\},$ ${\mathit{C}}_{\mathbf{2}}=\left\{{\mathcal{D}}_{\mathbf{4}}\right\},$ ${\mathit{C}}_{\mathbf{3}}=\{{\mathcal{D}}_{\mathbf{7}},{\mathcal{D}}_{\mathbf{6}},{\mathcal{D}}_{\mathbf{5}}\}$	15.407 at $\delta =0.444$	${\mathit{C}}_{\mathbf{1}}=\{{\mathcal{D}}_{\mathbf{7}},\dots ,{\mathcal{D}}_{\mathbf{1}}\}$	0.259 at $\delta =0.065$	${\mathit{C}}_{\mathbf{1}}=\{{\mathcal{D}}_{\mathbf{3}},{\mathcal{D}}_{\mathbf{2}},{\mathcal{D}}_{\mathbf{1}}\},$ ${\mathit{C}}_{\mathbf{2}}=\{{\mathcal{D}}_{\mathbf{7}},\dots ,{\mathcal{D}}_{\mathbf{4}}\}$	0.728 at $\delta =0.086$
Figure A2 $12\times 12$ $Zoo$ [48]	0.41 at $\delta =0.559$	${\mathit{C}}_{\mathbf{1}}=\{{\mathcal{D}}_{\mathbf{2}},{\mathcal{D}}_{\mathbf{1}}\},$ ${\mathit{C}}_{\mathbf{2}}=\{{\mathcal{D}}_{\mathbf{4}},{\mathcal{D}}_{\mathbf{3}}\},$ ${\mathit{C}}_{\mathbf{3}}=\left\{{\mathcal{D}}_{\mathbf{5}}\right\},$ ${\mathit{C}}_{\mathbf{4}}=\{{\mathcal{D}}_{\mathbf{7}},{\mathcal{D}}_{\mathbf{6}}\},$ ${\mathit{C}}_{\mathbf{5}}=\{{\mathcal{D}}_{\mathbf{9}},{\mathcal{D}}_{\mathbf{8}}\},$ ${\mathit{C}}_{\mathbf{6}}=\{{\mathcal{D}}_{\mathbf{11}},{\mathcal{D}}_{\mathbf{10}}\},$ ${\mathit{C}}_{\mathbf{7}}=\left\{{\mathcal{D}}_{\mathbf{12}}\right\}$	7.71 at $\delta =0.559$	${\mathit{C}}_{\mathbf{1}}=\{{\mathcal{D}}_{\mathbf{2}},{\mathcal{D}}_{\mathbf{1}}\},$ ${\mathit{C}}_{\mathbf{2}}=\{{\mathcal{D}}_{\mathbf{4}},{\mathcal{D}}_{\mathbf{3}}\},$ ${\mathit{C}}_{\mathbf{3}}=\left\{{\mathcal{D}}_{\mathbf{5}}\right\},$ ${\mathit{C}}_{\mathbf{4}}=\{{\mathcal{D}}_{\mathbf{7}},{\mathcal{D}}_{\mathbf{6}}\},$ ${\mathit{C}}_{\mathbf{5}}=\{{\mathcal{D}}_{\mathbf{9}},{\mathcal{D}}_{\mathbf{8}}\},$ ${\mathit{C}}_{\mathbf{6}}=\{{\mathcal{D}}_{\mathbf{11}},{\mathcal{D}}_{\mathbf{10}}\},$ ${\mathit{C}}_{\mathbf{7}}=\left\{{\mathcal{D}}_{\mathbf{12}}\right\}$	32.98 at $\delta =0.559$	${\mathit{C}}_{\mathbf{1}}=\{{\mathcal{D}}_{\mathbf{12}},\dots ,{\mathcal{D}}_{\mathbf{1}}\}$	0.57 at $\delta =0.348$	${\mathit{C}}_{\mathbf{1}}=\{{\mathcal{D}}_{\mathbf{12}},\dots ,{\mathcal{D}}_{\mathbf{1}}\}$	0.42 at $\delta =0.348$
Figure A3 $4\times 4$ $Mushroom$ [48]	0.08 at $\delta =0.41$	${\mathit{C}}_{\mathbf{1}}=\left\{{\mathcal{D}}_{\mathbf{1}}\right\},$ ${\mathit{C}}_{\mathbf{2}}=\{{\mathcal{D}}_{\mathbf{4}},{\mathcal{D}}_{\mathbf{3}},{\mathcal{D}}_{\mathbf{2}}\}$	0.43 at $\delta =0.41$	${\mathit{C}}_{\mathbf{1}}=\{{\mathcal{D}}_{\mathbf{4}},\dots ,{\mathcal{D}}_{\mathbf{1}}\}$	1.672 at $\delta =0.365$	${\mathit{C}}_{\mathbf{1}}=\{{\mathcal{D}}_{\mathbf{4}},\dots ,{\mathcal{D}}_{\mathbf{1}}\}$	0.55 at $\delta =0.365$	${\mathit{C}}_{\mathbf{1}}=\left\{{\mathcal{D}}_{\mathbf{1}}\right\},$ ${\mathit{C}}_{\mathbf{2}}=\{{\mathcal{D}}_{\mathbf{4}},{\mathcal{D}}_{\mathbf{3}},{\mathcal{D}}_{\mathbf{2}}\}$	0.68 at $\delta =0.41$
Figure A4 $6\times 6$ $Iris$ [48]	0.304 at $\delta =0.3$	${\mathit{C}}_{\mathbf{1}}=\{{\mathcal{D}}_{\mathbf{2}},{\mathcal{D}}_{\mathbf{1}}\},$ ${\mathit{C}}_{\mathbf{2}}=\{{\mathcal{D}}_{\mathbf{4}},{\mathcal{D}}_{\mathbf{3}}\},$ ${\mathit{C}}_{\mathbf{3}}=\{{\mathcal{D}}_{\mathbf{6}},{\mathcal{D}}_{\mathbf{5}}\}$	1.10 at $\delta =0.3$	${\mathit{C}}_{\mathbf{1}}=\{{\mathcal{D}}_{\mathbf{2}},{\mathcal{D}}_{\mathbf{1}}\},$ ${\mathit{C}}_{\mathbf{2}}=\{{\mathcal{D}}_{\mathbf{4}},{\mathcal{D}}_{\mathbf{3}}\},$ ${\mathit{C}}_{\mathbf{3}}=\{{\mathcal{D}}_{\mathbf{6}},{\mathcal{D}}_{\mathbf{5}}\}$	9.64 at $\delta =0.3$	${\mathit{C}}_{\mathbf{1}}=\{{\mathcal{D}}_{\mathbf{2}},{\mathcal{D}}_{\mathbf{1}}\},$ ${\mathit{C}}_{\mathbf{2}}=\{{\mathcal{D}}_{\mathbf{4}},{\mathcal{D}}_{\mathbf{3}}\},$ ${\mathit{C}}_{\mathbf{3}}=\{{\mathcal{D}}_{\mathbf{6}},{\mathcal{D}}_{\mathbf{5}}\}$	0.55 at $\delta =0.3$	${\mathit{C}}_{\mathbf{1}}=\{{\mathcal{D}}_{\mathbf{2}},{\mathcal{D}}_{\mathbf{1}}\},$ ${\mathit{C}}_{\mathbf{2}}=\{{\mathcal{D}}_{\mathbf{4}},{\mathcal{D}}_{\mathbf{3}}\},$ ${\mathit{C}}_{\mathbf{3}}=\{{\mathcal{D}}_{\mathbf{6}},{\mathcal{D}}_{\mathbf{5}}\}$	0.44 at $\delta =0.3$
Figure A5 $6\times 6$ $Zoo$ & $Mushroom$ [48]	0.63 at $\delta =0.384$	${\mathit{C}}_{\mathbf{1}}=\{{\mathcal{D}}_{\mathbf{2}},{\mathcal{D}}_{\mathbf{1}}\},$ ${\mathit{C}}_{\mathbf{2}}=\{{\mathcal{D}}_{\mathbf{6}},\dots ,{\mathcal{D}}_{\mathbf{3}}\}$	0.5 at $\delta =0.384$	${\mathit{C}}_{\mathbf{1}}=\{{\mathcal{D}}_{\mathbf{2}},{\mathcal{D}}_{\mathbf{1}}\},$ ${\mathit{C}}_{\mathbf{2}}=\{{\mathcal{D}}_{\mathbf{6}},\dots ,{\mathcal{D}}_{\mathbf{3}}\}$	9.96 at $\delta =0.384$	${\mathit{C}}_{\mathbf{1}}=\{{\mathcal{D}}_{\mathbf{2}},{\mathcal{D}}_{\mathbf{1}}\},$ ${\mathit{C}}_{\mathbf{2}}=\{{\mathcal{D}}_{\mathbf{6}},\dots ,{\mathcal{D}}_{\mathbf{3}}\}$	0.40 at $\delta =0.384$	${\mathit{C}}_{\mathbf{1}}=\{{\mathcal{D}}_{\mathbf{2}},{\mathcal{D}}_{\mathbf{1}}\},$ ${\mathit{C}}_{\mathbf{2}}=\{{\mathcal{D}}_{\mathbf{6}},\dots ,{\mathcal{D}}_{\mathbf{3}}\}$	0.85 at $\delta =0.384$
Figure A6 $4\times 4$ [39]	0.34 at $\delta =0.429$	${\mathit{C}}_{\mathbf{1}}=\{{\mathcal{D}}_{\mathbf{3}},{\mathcal{D}}_{\mathbf{2}},{\mathcal{D}}_{\mathbf{1}}\},$ ${\mathit{C}}_{\mathbf{2}}=\left\{{\mathcal{D}}_{\mathbf{4}}\right\}$	1.12 at $\delta =0.429$	${\mathit{C}}_{\mathbf{1}}=\{{\mathcal{D}}_{\mathbf{4}},\dots ,{\mathcal{D}}_{\mathbf{1}}\}$	2.708 at $\delta =0.25$	${\mathit{C}}_{\mathbf{1}}=\{{\mathcal{D}}_{\mathbf{4}},\dots ,{\mathcal{D}}_{\mathbf{1}}\}$	0.38 at $\delta =0.25$	${\mathit{C}}_{\mathbf{1}}=\{{\mathcal{D}}_{\mathbf{3}},{\mathcal{D}}_{\mathbf{2}},{\mathcal{D}}_{\mathbf{1}}\},$ ${\mathit{C}}_{\mathbf{2}}=\left\{{\mathcal{D}}_{\mathbf{4}}\right\}$	0.81 at $\delta =0.429$
Figure A7 $10\times 10$ T10I4D100K [49]	0.115 at $\delta =0.846$	${\mathit{C}}_{\mathbf{1}}=\left\{{\mathcal{D}}_{\mathbf{1}}\right\},$ ${\mathit{C}}_{\mathbf{2}}=\left\{{\mathcal{D}}_{\mathbf{2}}\right\},$ ${\mathit{C}}_{\mathbf{3}}=\left\{{\mathcal{D}}_{\mathbf{3}}\right\},$ ${\mathit{C}}_{\mathbf{4}}=\{{\mathcal{D}}_{\mathbf{5}},{\mathcal{D}}_{\mathbf{4}}\},$ ${\mathit{C}}_{\mathbf{5}}=\left\{{\mathcal{D}}_{\mathbf{6}}\right\},$ ${\mathit{C}}_{\mathbf{6}}=\left\{{\mathcal{D}}_{\mathbf{7}}\right\},$ ${\mathit{C}}_{\mathbf{7}}=\{{\mathcal{D}}_{\mathbf{10}},{\mathcal{D}}_{\mathbf{9}},{\mathcal{D}}_{\mathbf{8}}\}$	0.71 at $\delta =0.846$	${\mathit{C}}_{\mathbf{1}}=\{{\mathcal{D}}_{\mathbf{10}},\dots ,{\mathcal{D}}_{\mathbf{1}}\}$	35.275 at $\delta =0.737$	${\mathit{C}}_{\mathbf{1}}=\{{\mathcal{D}}_{\mathbf{10}},\dots ,{\mathcal{D}}_{\mathbf{1}}\}$	0.193 at $\delta =0.737$	${\mathit{C}}_{\mathbf{1}}=\{{\mathcal{D}}_{\mathbf{10}},\dots ,{\mathcal{D}}_{\mathbf{1}}\}$	0.806 at $\delta =0.737$

, such that $\delta \in [0,1]$ is the ideal similarity threshold for which a goodness measure attains its optimal point. Python version 3.9.2 was used to implement all the algorithms, and the codes were run on a Ubuntu-20.04 server equipped with an Intel(R) Xeon(R) CPU clocked at 2.30 GHz with 50 GB available Disk capacity and 12 GB of available RAM.

4.1. Similarity Accuracy Analysis

To demonstrate the efficiency of $sim$ (3), let us have three transaction databases, $|{\mathcal{D}}_1|$ = 200, $|{\mathcal{D}}_2|$ = 300 and $|{\mathcal{D}}_3|$ = 200 with their corresponding local frequent itemsets: $FIS({\mathcal{D}}_1,0.2)$ = $\{\langle C,0.2\rangle ,\langle B,0.2\rangle ,\langle A,0.2\rangle \}$, $FIS({\mathcal{D}}_2,0.15)$ = $\{\langle E,0.9\rangle ,\langle C,0.2\rangle ,\langle B,0.2\rangle ,\langle A,0.2\rangle \}$ and $FIS({\mathcal{D}}_3,0.25)$ = $\{\langle E,0.9\rangle \}$ mined at different minimum support threshold values ${\alpha }_1=0.2$, ${\alpha }_2=0.15$ and ${\alpha }_3=0.25$, respectively. Now, clustering the three databases using the algorithm BestDatabaseClustering [22] equipped with two different similarity measures, $simi$ proposed in [20] and our proposed similarity measure $sim$ (3), shows the results reported in

Table 4. Clustering the three databases ${\mathcal{D}}_1$, ${\mathcal{D}}_2$ and ${\mathcal{D}}_3$ under the similarity measure $simi$ [20] against our proposed measure $sim$ (3).

Output	Clustering 1 under simi [20]	Clustering 2 under sim (3)
clusters	$\left\{{\mathcal{D}}_1\right\},\{{\mathcal{D}}_2,{\mathcal{D}}_3\}$	$\{{\mathcal{D}}_1,{\mathcal{D}}_2\},\left\{D_3\right\}$
Similarity intra-cluster	0.6	0.75
Distance inter-cluster	1.6	1.75
Measure goodness [20]	0.2	0.5

. We note that $goodness$ [20] is a clustering quality measure, such that the higher the value of $goodness$ for a given candidate clustering $\mathcal{C}$, the better the quality of $\mathcal{C}$.

From Table 4, we notice that using our similarity measure $sim$ (3), we have obtained a larger intra-cluster similarity, a larger inter-cluster distance and a larger $goodness$ [20]. Now, let us synthesize the global frequent itemsets from the clusters containing more than one database, i.e., $C_{2,3}=\{{\mathcal{D}}_2,{\mathcal{D}}_3\}$ and $C_{1,2}=\{{\mathcal{D}}_1,{\mathcal{D}}_2\}$. The obtained results are shown in

Table 5. Itemsets synthesized from $C_{2,3}=\{{\mathcal{D}}_2,{\mathcal{D}}_3\}$ discovered under $simi$ [20] against the itemsets synthesized from $C_{1,2}=\{{\mathcal{D}}_1,{\mathcal{D}}_2\}$ discovered under $sim$ (3).

Synthesized Itemsets ${\mathit{I}}_{\mathit{k}}$	$\mathit{supp} ({\mathit{I}}_{\mathit{k}},{\mathit{C}}_{2,3})$ under simi [20]	$\mathit{supp} ({\mathit{I}}_{\mathit{k}},{\mathit{C}}_{1,2})$ under sim (3)
A	$0.12<{\alpha }_{2,3}=0.19$	$0.2>{\alpha }_{1,2}=0.17$
B	$0.12<{\alpha }_{2,3}=0.19$	$0.2>{\alpha }_{1,2}=0.17$
C	$0.12<{\alpha }_{2,3}=0.19$	$0.2>{\alpha }_{1,2}=0.17$
E	$0.9>{\alpha }_{2,3}=0.19$	$0.54>{\alpha }_{1,2}=0.17$

, such that ${\alpha }_{2,3}=\frac{300\times 0.15+200\times 0.25}{300+200}=0.19$ and ${\alpha }_{1,2}=\frac{200\times 0.2+300\times 0.15}{200+300}=0.17$ are the minimum support thresholds corresponding to $C_{2,3}$ and $C_{2,3}$ respectively. As we can see, the similarity measure $simi$ [20] captures only high frequency itemsets ($supp\approx 1$), such as E, and neglects low support frequent itemsets (i.e., whose supports are immediately above the minimum threshold $\alpha$ with $supp\in [\alpha ,\alpha +ϵ]$ and $ϵ$ is a very small number), such as A, B and C. This characteristic gives a high similarity value to database pairs sharing only one or very few high frequency itemsets. On the other hand, database pairs sharing many frequent itemsets with a low support will be assigned a lower similarity. However, once the clustering is done, we will be interested in the patterns discovered from each cluster individually, such as the high-vote patterns [32] and the exceptional patterns [31]. That is why our similarity measure estimates the patterns post-mined from each cluster $C_{p,q}=\{{\mathcal{D}}_p,{\mathcal{D}}_q\}$ in order to compute $sim({\mathcal{D}}_p,{\mathcal{D}}_q)$. Since our similarity measure focuses on maximizing the number of frequent itemsets synthesized from each cluster $C_{p,q}\subseteq \mathcal{D}$, only relevant clusters will be assigned a large similarity value.

4.2. Fuzziness Reduction Analysis

To demonstrate the importance of reducing the fuzziness associated with a similarity matrix, we run the clustering algorithm BestDatabaseClustering [22] on two similarity matrices in Figure 5a and Figure 6a. The obtained results in terms of the optimal clustering, $maxgoodness\left(\mathcal{D}\right)$ [20], the optimal similarity level ${\delta }_{opt}$ (i.e., the similarity level at $maxgoodness\left(\mathcal{D}\right)$) and the silhouette coefficient $SC\left(\mathcal{D}\right)$ [43] at ${\delta }_{opt}$ are shown in Figure 5b,c and Figure 6b,c corresponding to rows 1 and 2 of

Table 6. A summary of the results obtained in Figure 5, Figure 6, Figure 7 and Figure 8. We note that ${\mathit{\delta }}_{\mathit{opt}}$ is the optimal similarity level at which $\mathit{goodness}\left(\mathcal{D}\right)$ [20] attains its maximum value, and ${\theta }^T$ is the optimal weight vector learned after a number of epochs.

Similarity Matrix	Fuzziness Index (9)	${\mathit{\theta }}^{\mathit{T}},\mathit{epochs},\mathit{\eta }$	max $\mathit{goodness}\left(\mathcal{D}\right)$ [20]	${\mathit{\delta }}_{\mathit{opt}}$	$\mathit{SC}\left(\mathcal{D}\right)$ [43,44] at ${\mathit{\delta }}_{\mathit{opt}}$	Optimal Clustering at ${\mathit{\delta }}_{\mathit{opt}}$
Figure 5	0.97	${\theta }^T=[1,1,\dots ,1,1]$ (Without fuzziness reduction)	4.19	0.46	−1	$\{{\mathcal{D}}_1,{\mathcal{D}}_2,{\mathcal{D}}_3,{\mathcal{D}}_4,{\mathcal{D}}_5\}$
Figure 5	0.95	${\theta }^T=[1,1,\dots ,1,1]$ (Without fuzziness reduction)	1.29	0.313	−1	$\{{\mathcal{D}}_1,{\mathcal{D}}_2,{\mathcal{D}}_3,{\mathcal{D}}_4\}$
Figure 6	0.74	${\theta }^T$ = [1.30,0.52,0.71,0.71,0.52, 0.71,0.71,0.52,0.52,1.44], $epochs=300$, $\eta =0.1$	4.54	0.95	0.73	$\{{\mathcal{D}}_1,{\mathcal{D}}_2\},\left\{{\mathcal{D}}_3\right\},\{{\mathcal{D}}_4,{\mathcal{D}}_5\}$
Figure 6	0.81	${\theta }^T=[0.63,0.638,0.591,0.712,0.712,0.77]$, $epochs=100$, $\eta =0.1$	1.27	0.292	0.08	$\{{\mathcal{D}}_4,{\mathcal{D}}_3,{\mathcal{D}}_2\},\left\{{\mathcal{D}}_1\right\}$

, respectively. From the obtained results, we can clearly see that when the similarity matrices are centered around the mean value 0.5, the fuzziness index becomes larger and closer to 1, and BestDatabaseClustering [22] could not return a meaningful clustering since it has put all the n databases into the same cluster.

Now, let us run our fuzziness reduction model on the previous similarity matrices and depict the adjusted similarity matrices in Figure 7a and Figure 8a, respectively. Afterward, we run BestDatabaseClustering [22] on the new similarity matrices and show the clustering results in Figure 7b,c and Figure 8b,c corresponding to rows 3 and 4 of Table 6, respectively. As we can see, after reducing the fuzziness index associated with the previous similarity matrices in Figure 5a and Figure 6a, the algorithm BestDatabaseClustering [22] was able to produce meaningful non-trivial clusterings with an increase in the silhouette coefficient $SC\left(\mathcal{D}\right)$ [43] for both similarity matrices in Figure 7a and Figure 8a.

4.3. Convexity and Clustering Analysis

In this part of our experiments, we analyze the convex behavior of the proposed clustering quality functions $L\left(\theta \right)$ (19) and $\mathcal{L}\left(\theta \right)$ (14), and we also examine the non-convexity of the existing goodness measures in [20,21,23,43]. Additionally, we compare the clustering produced by our algorithm and the ones generated at the optimal points of the previous compared goodness measures (i.e., at $maxgoodness\left(\mathcal{D}\right)$ [20], $mingoodnes{s}^2\left(\mathcal{D}\right)$ [23] and $maxgoodnes{s}^3\left(\mathcal{D}\right)$ [21]) with the underlying ground-truth cluster labels. When the actual clustering is unknown, we replace it with the partitioning obtained at the maximum value of the silhouette coefficient [43], that is, at $maxSC\left(\mathcal{D}\right)$. All the graphs corresponding to our loss functions and the compared goodness measures in Table 3 are plotted in Figure A1b, Figure A2b, Figure A3b, Figure A4b, Figure A5b, Figure A6b, and Figure A7b, where the x-axis represents the similarity levels $\delta$ at which multiple candidate clusterings are generated and evaluated.

Consider the $7\times 7$ similarity matrix shown in Figure A1a. From the graphs plotted in Figure A1b and according to the results shown in the first row of Table A2, we can see that using our loss function $\mathcal{L}\left(\theta \right)$ and $goodness\left(\mathcal{D}\right)$ [20], we were able to find the optimal clustering $\{C_1=\{{\mathcal{D}}_3,{\mathcal{D}}_2,{\mathcal{D}}_1\},C_2=\left\{{\mathcal{D}}_4\right\},C_3=\{{\mathcal{D}}_7,{\mathcal{D}}_6,{\mathcal{D}}_5\}\}$ at a similarity level $\delta =0.44$ where the silhouette coefficient reaches its maximum value $SC\left(\mathcal{D}\right)=0.46$. On the other hand, $goodnes{s}^3\left(\mathcal{D}\right)$ [21] and $goodnes{s}^2\left(\mathcal{D}\right)$ [23] did not successfully discover the partitioning maximizing the silhouette coefficient. Additionally, we observe that the proposed convergence test function $L\left(\theta \right)$ has a quasi-convex behavior (see proof of Theorem 2). This allows us to terminate the clustering process right after reaching the global minimum. Conversely, the graphs corresponding to $goodnes{s}^2\left(\mathcal{D}\right)$ [23] and $goodness\left(\mathcal{D}\right)$ [20] have local optima. Consequently, it is required to explore about $(n^2-n)/2$ similarity levels in order to generate and evaluate all the candidate clusterings possible.

Now, let us examine the results of some experiments that we have conducted on the synthetic and real-world datasets shown in Table A1. From Figure A2b and Figure A7b (the last and second rows of Table A2), we observe that $goodnes{s}^3\left(\mathcal{D}\right)$ [21] and $goodnes{s}^2\left(\mathcal{D}\right)$ [23] attain their optimal values when all the partition databases are clustered together in one class. The same phenomenon is observed in Figure A3b, Figure A6b and Figure A7b (the last, the sixth and the third rows of Table A2), where both $goodnes{s}^2\left(\mathcal{D}\right)$ [23] and $goodness\left(\mathcal{D}\right)$ [20] have put all the databases into one cluster.

In contrast, the proposed loss function $\mathcal{L}\left(\theta \right)$ has successfully identified the clustering for which the silhouette coefficient $SC$ is maximum. Precisely, in Figure A7b, which corresponds to the last row of Table A2), $\mathcal{L}\left(\theta \right)$ was the only clustering quality measure which has properly identified the ideal 7-class clustering at $\delta =0.846$.

From the obtained graphs in Figure A1, Figure A2, Figure A3, Figure A4, Figure A5, Figure A6, and Figure A7, we notice that $goodnes{s}^3\left(\mathcal{D}\right)$ [21], $goodnes{s}^2\left(\mathcal{D}\right)$ [23] and $goodness\left(\mathcal{D}\right)$ [20] are neither quasi-concave nor quasi-convex on the domain $[0,1]$. As a result, we observe the existence of local optimum points on their corresponding graphs, which makes the search of the global optimum a difficult problem to solve without exploring all the local solutions.

Conversely, we observe that our loss function $\mathcal{L}\left(\theta \right)$ (14) is monotonically decreasing all the time and $\mathcal{L}\left(\widehat{\theta }\right)=0$ at $\widehat{\theta }=arg\underset{\begin{array}{c}\theta \end{array}}{min}\mathcal{L}\left(\theta \right)=\overrightarrow{\mathbf{1}}$. This corresponds to the similarity level $\delta =0$ where all the n databases are put into the same single cluster. To avoid this case from occurring, we used the quasi-convex function $L\left(\theta \right)$ (19) as a convergence test function to terminate our algorithm at the point $\mathcal{L}(arg\underset{\begin{array}{c}\theta \end{array}}{min}L\left(\theta \right))$ corresponding to the red dot on the graph of our loss function $\mathcal{L}\left(\theta \right)$. Moreover, it is worth noting that for every two real $\left(\genfrac{}{}{0pt}{}{n}{2}\right)$-dimensional vectors ${\theta }^{\left(i\right)}$ and ${\theta }^{(i+1)}$, where $L\left({\theta }^{(i+1)}\right)\le L\left({\theta }^{\left(i\right)}\right)$, the line that joins the points (${\theta }^{(i+1)},L\left({\theta }^{\left(i\right)}\right)$) and (${\theta }^{\left(i\right)},L\left({\theta }^{\left(i\right)}\right)$) remains above $L\left(\theta \right)$, which is observed in Figure A1, Figure A2, Figure A3, Figure A4, Figure A5, Figure A6, and Figure A7. Therefore, using the proposed loss function $\mathcal{L}\left(\theta \right)$ (14) along with $L\left(\theta \right)$ (19) guarantees global minimum convergence.

In the fifth and the most right columns of Table A1, we compare the number of frequent itemsets (FIs) mined from all the partitions of a given dataset $\mathcal{D}$ with the FIs mined from each single cluster $C_j$ consisting of similar partitions from the same dataset, where ${\cap }_{j=1}^k\left\{C_j\right\}=\varnothing$ and ${\cup }_{j=1}^k\left\{C_j\right\}=\mathcal{D}$. We notice that mining all the partitions from datasets Iris [48] and Zoo [48] did not result in discovering any valid frequent itemset. Whereas, mining each individual cluster of partitions from the datasets Iris and Zoo has led to the discovery of new patterns in each cluster $C_j$.

In

Table A3. The similarity levels ${\delta }_{opt}$ at which the clustering evaluation measures ${\mathit{goodness}}^{\mathbf{3}}\left(\mathcal{D}\right)$ [21], ${\mathit{goodness}}^{\mathbf{2}}\left(\mathcal{D}\right)$ [23], the silhouette coefficient $\mathit{S}\mathit{C}\left(\mathcal{D}\right)$ [43], $\mathit{goodness}\left(\mathcal{D}\right)$ [20] and our proposed objective function $\mathcal{L}\left(\mathit{\theta }\right)$ attain their optimal values in Figure A1, Figure A2, Figure A3, Figure A4, Figure A5, Figure A6, and Figure A7. The fraction $\frac{\left|\right\{{\mathit{\delta }}_{\mathbf{1}},\mathbf{\dots },{\mathit{\delta }}_{\mathit{stop}}\left\}\right|}{\left|\right\{{\mathit{\delta }}_{\mathbf{1}},\mathbf{\dots },{\mathit{\delta }}_{\mathit{m}}\left\}\right|}$ represents the number of similarity levels required to test the convergence and terminate divided by the number of total similarity levels m.

Dataset $\mathcal{D}$	Silhouette Coefficient $\underset{\begin{array}{c}\mathit{\theta }\end{array}}{\mathbf{max}}\mathit{S}\mathit{C}\left(\mathcal{D}\right)$		Proposed Loss Function $\mathcal{L}(\mathbf{arg}\underset{\begin{array}{c}\mathit{\theta }\end{array}}{\mathbf{min}}\mathcal{L}\left(\mathit{\theta }\right))$		Goodness Measure $\underset{\mathit{\theta }}{\mathbf{max}}\mathit{goodness}\left(\mathcal{D}\right)$		Goodness Measure $\underset{\mathit{\theta }}{\mathbf{min}}{\mathit{goodness}}^2\left(\mathcal{D}\right)$		Goodness Measure $\underset{\mathit{\theta }}{\mathbf{max}}{\mathit{goodness}}^3\left(\mathcal{D}\right)$
	${\mathit{\delta }}_{\mathit{opt}}$	$\frac{\left|\right\{{\mathit{\delta }}_1,\dots ,{\mathit{\delta }}_{\mathit{stop}}\left\}\right|}{\left|\right\{{\mathit{\delta }}_1,\dots ,{\mathit{\delta }}_{\mathit{m}}\left\}\right|}$	${\mathit{\delta }}_{\mathit{opt}}$	$\frac{\left|\right\{{\mathit{\delta }}_1,\dots ,{\mathit{\delta }}_{\mathit{stop}}\left\}\right|}{\left|\right\{{\mathit{\delta }}_1,\dots ,{\mathit{\delta }}_{\mathit{m}}\left\}\right|}$	${\mathit{\delta }}_{\mathit{opt}}$	$\frac{\left|\right\{{\mathit{\delta }}_1,\dots ,{\mathit{\delta }}_{\mathit{stop}}\left\}\right|}{\left|\right\{{\mathit{\delta }}_1,\dots ,{\mathit{\delta }}_{\mathit{m}}\left\}\right|}$	${\mathit{\delta }}_{\mathit{opt}}$	$\frac{\left|\right\{{\mathit{\delta }}_1,\dots ,{\mathit{\delta }}_{\mathit{stop}}\left\}\right|}{\left|\right\{{\mathit{\delta }}_1,\dots ,{\mathit{\delta }}_{\mathit{m}}\left\}\right|}$	${\mathit{\delta }}_{\mathit{opt}}$	$\frac{\left|\right\{{\mathit{\delta }}_1,\dots ,{\mathit{\delta }}_{\mathit{stop}}\left\}\right|}{\left|\right\{{\mathit{\delta }}_1,\dots ,{\mathit{\delta }}_{\mathit{m}}\left\}\right|}$
Figure A1 $7\times 7$ [20]	0.444	$\frac{10}{10}$	0.444	$\frac{5}{10}$	0.444	$\frac{10}{10}$	0.065	$\frac{10}{10}$	0.086	$\frac{10}{10}$
Figure A2 $12\times 12$ $Zoo$ [48]	0.559	$\frac{48}{48}$	0.559	$\frac{5}{48}$	0.559	$\frac{48}{48}$	0.348	$\frac{48}{48}$	0.348	$\frac{48}{48}$
Figure A3 $4\times 4$ $Mushroom$ [48]	0.41	$\frac{4}{4}$	0.41	$\frac{2}{4}$	0.365	$\frac{4}{4}$	0.365	$\frac{4}{4}$	0.41	$\frac{4}{4}$
Figure A4 $6\times 6$ $Iris$ [48]	0.3	$\frac{6}{6}$	0.3	$\frac{3}{6}$	0.3	$\frac{6}{6}$	0.3	$\frac{6}{6}$	0.3	$\frac{6}{6}$
Figure A5 $6\times 6$ $Zoo$ & $Mushroom$ [48]	0.384	$\frac{8}{8}$	0.384	$\frac{7}{8}$	0.384	$\frac{8}{8}$	0.384	$\frac{8}{8}$	0.384	$\frac{8}{8}$
Figure A6 $4\times 4$ [39]	0.429	$\frac{6}{6}$	0.429	$\frac{3}{6}$	0.25	$\frac{6}{6}$	0.25	$\frac{6}{6}$	0.429	$\frac{6}{6}$
Figure A7 $10\times 10$ T10I4D100K [49]	0.846	$\frac{31}{31}$	0.846	$\frac{4}{31}$	0.737	$\frac{31}{31}$	0.737	$\frac{31}{31}$	0.737	$\frac{31}{31}$

, we report the similarity levels ${\delta }_{opt}$ at which the clustering evaluation measures $goodness\left(\mathcal{D}\right)$ [20], $goodnes{s}^2\left(\mathcal{D}\right)$ [23], $goodnes{s}^3\left(\mathcal{D}\right)$ [21], the silhouette coefficient $SC$ [43] and our proposed loss function $\mathcal{L}\left(\theta \right)$ attain their optimal values in Figure A1, Figure A2, Figure A3, Figure A4, Figure A5, Figure A6, and Figure A7. We note that, the fraction $\frac{\left|\right\{{\delta }_1,\dots ,{\delta }_{stop}\left\}\right|}{\left|\right\{{\delta }_1,\dots ,{\delta }_m\left\}\right|}$ in Table A3 represents the number of similarity levels required to test the convergence and terminate divided by the number of all similarity levels m. We note that $opt$ is the index of the optimal similarity level according to a given clustering quality measure. Since our proposed algorithm is based on a convex loss function, we notice that $stop=opt<m$. On the other hand, as for the compared algorithms, which are based on non-convex objectives, we notice that $stop=m$. Therefore, our algorithm requires the least number of similarity levels ($opt$ out of m) in order to converge and terminate, which makes our algorithm faster than the compared algorithms in [21,22,23], requiring to generate and evaluate all the m candidate clusterings in order to return the optimal one.

All the previous results confirm that using our loss function $\mathcal{L}\left(\theta \right)$ (14) along with $L\left(\theta \right)$ (19), we have identified the ideal clustering for which the silhouette coefficient $SC$ [43] is maximum and we have also improved the quality of the frequent itemsets (FIs) mined from the multiple databases partitioned from the datasets in Table A1.

4.4. Clustering Error and Running Time Analysis

In this experimental part, we compare the running time of the proposed clustering algorithm with the execution times of two clustering algorithms for multi-database mining (MDM), namely GDMDBClustering [25] and BestDatabaseClustering [22], all run on the same random data samples. We also calculate how the clusterings produced by our algorithm and the compared models are different from the ground-truth clustering. For this purpose, we propose an error function in (21), which measures the difference between two given clusterings $\mathcal{P}$ and $\mathcal{Q}$.

First, we generated n = 30 to N = 120 isotropic Gaussian blobs using the scikit-learn generator [50], such that the number of features for each n blobs is set to $random.randint(2,10)$, while the number of clusters is set to $\lfloor \frac{n}{2}\rfloor$. In

Table 7. A brief summary of the random blobs generated via scikit-learn [50].

Number of Random Blobs $\mathbf{\left(}\mathit{n}\mathbf{\right)}$	Number of Centers $\mathbf{\lfloor }\frac{\mathit{n}}{\mathbf{2}}\mathbf{\rfloor }$	Number of Attributes $\mathbf{\left(}\mathit{m}\mathbf{\right)}$
30	15	random.randint(2, 10)
⋮	⋮	⋮
60	30	random.randint(2, 10)
⋮	⋮	⋮
120	60	random.randint(2, 10)

, we present a brief summary of the random blobs generated via scikit-learn [50].

Afterward, we use the min-max scaling [51] to normalize each feature (out of the m features) into the interval [0,1]. Then, for each n blobs, every pair of m-dimensional blobs is passed as arguments to the function $sim$ (3) in order to compute the $(n^2-n)/2$ pairwise similarities between the n blobs. We then run the proposed algorithm, GDMDBClustering [25] (with three different learning rate values) and BestDatabaseClustering [22] on each of the $(n^2-n)/2$ pairwise similarities ($n=30,\dots ,120$) and plot their running time graphs in Figure A8a, Figure A9a and Figure A10a, and then plot the clustering error graphs in Figure A8b, Figure A9b and Figure A10b.

Without loss of generality, assume $\mathcal{Q}$ is the ground-truth clustering (i.e., the actual clusters) of the current n blobs $\mathcal{D}=\{{\mathcal{D}}_1,{\mathcal{D}}_2,\dots ,{\mathcal{D}}_n\}$ generated via scikit-learn [50], and assume $\mathcal{P}$ is the partitioning of $\mathcal{D}$ produced by a given clustering algorithm. To measure how far is $\mathcal{P}$ from $\mathcal{Q}$, we propose the error function ${\mathcal{E}}_n(\mathcal{P},\mathcal{Q})\in [0,1]$ to be defined as follows:

$${\mathcal{E}}_n(\mathcal{P},\mathcal{Q})=\frac{|Pair{s}_{\mathcal{Q}}\backslash Pair{s}_{\mathcal{P}}|+|Pair{s}_{\mathcal{P}}\backslash Pair{s}_{\mathcal{Q}}|}{|Pair{s}_{\mathcal{Q}}|+|Pair{s}_{\mathcal{P}}|}$$

(21)

where $|Pair{s}_{\mathcal{P}}|$ is the number of all the database pairs obtained from every cluster in $\mathcal{P}$ and $|Pair{s}_{\mathcal{P}}\backslash Pair{s}_{\mathcal{Q}}|$ is the number of all the database pairs that only exist in $Pair{s}_{\mathcal{P}}$ and that cannot be found in $Pair{s}_{\mathcal{Q}}$. We note that ${\mathcal{E}}_n(\mathcal{P},\mathcal{Q})$ approaches the maximum value of 1 (i.e., ${\mathcal{E}}_n(\mathcal{P},\mathcal{Q})\approx 1$) when $\mathcal{P}$ and $\mathcal{Q}$ are too different and do not share many database pairs in common (i.e., $|Pair{s}_{\mathcal{P}}\cap Pair{s}_{\mathcal{Q}}|\approx 0$). Conversely, ${\mathcal{E}}_n(\mathcal{P},\mathcal{Q})\approx 0$ when the clustering $\mathcal{P}$ and $\mathcal{Q}$ are too similar, i.e., they share the maximum number of pairs $({\mathcal{D}}_p,{\mathcal{D}}_q)$.

We also define the average of the $N-n+1$ clustering errors, which could also be seen as the mean absolute clustering error:

$$\overline{\mathcal{E}(\mathcal{P},\mathcal{Q})}=\frac{{\sum }_n^N{\mathcal{E}}_n(\mathcal{P},\mathcal{Q})}{N-n+1}$$

(22)

From the obtained results in Figure A8a, Figure A9a and Figure A10a, we observe a rapid increase in the running time of BestDatabaseClustering [22] as the number of generated blobs (n) increases linearly. This is due to the fact that BestDatabaseClustering needs to generate and evaluate approximately $(n^2-n)/2$ candidate clusterings in order to find the optimal clustering for which the non-convex function $goodness\left(\mathcal{D}\right)$ [20] is maximum. In fact, $goodness\left(\mathcal{D}\right)$ suffers from the existence of local maxima, which requires exploring all the local candidate solutions in order to find the global maximum. On the other hand, using the proposed convex loss function $\mathcal{L}\left(\theta \right)$ and the quasi-convex convergence test function $L\left(\theta \right)$ allows us to stop the clustering process at $\mathcal{L}(arg\underset{\begin{array}{c}\theta \end{array}}{min}L\left(\theta \right))$. Consequently, this avoids generating unnecessary candidate clusterings, and hence reduces the CPU overhead. Since our algorithm is independent of the learning rate $\eta$, the running time of our algorithm is the same in all Figure A8a, Figure A9a and Figure A10a. Whereas, the running time of GDMDBClustering [25] increases for smaller learning rate values (e.g., Figure A10) and decreases when we set larger learning rate values (e.g., Figure A9), but this comes at the cost of having an increased clustering error.

Next, by examining the three clustering error graphs in Figure A8b, Figure A9b and Figure A10b, we observe that BestDatabaseClustering [22] has the largest clustering error among the three algorithms with a clustering average error $\overline{\mathcal{E}(\mathcal{P},\mathcal{Q})}=0.936$. In fact, on average, BestDatabaseClustering [22] tends to group all the current n blobs ($n=30,\dots ,120$) in one single cluster. On the other hand, our proposed algorithm and GDMDBClustering [25] produce clusterings that are close to the ground-truth clustering predetermined by the scikit-learn generator [50]. In fact, the average clustering error due to our algorithm is $\overline{\mathcal{E}(\mathcal{P},\mathcal{Q})}=0.285$. For GDMDBClustering [25], we get $\overline{\mathcal{E}(\mathcal{P},\mathcal{Q})}=0.285$ when the learning rate $\eta =0.0005$ or $\eta =0.001$, and the error increases to $\overline{\mathcal{E}(\mathcal{P},\mathcal{Q})}=0.29$ when $\eta =0.002$. The average running times and clustering errors of our algorithm, GDMDBClustering [25] and BestDatabaseClustering [22] are summarized in

Table A4. Summary of the average running times and the average clustering errors $\overline{\mathcal{E}(\mathcal{P},\mathcal{Q})}$ (22) for the proposed algorithm, BestDatabaseClustering [22] and GDMDBClustering [25] (with three different values for the learning rate $\eta$) on the random samples described in Table 7.

Experiment (Figure)	Proposed Algo		BestDatabaseClustering [22]		GDMDBClustering [25]
	Average Running Time	Average Clustering Error	Average Running Time	Average Clustering Error	Average Running Time	Average Clustering Error
Figure A8 $\eta =0.001$	6.367	0.285	47.208	0.936	14.825	0.285
Figure A9 $\eta =0.002$	6.367	0.285	47.208	0.936	7.305	0.290
Figure A10 $\eta =0.0005$	6.367	0.285	47.208	0.936	28.479	0.285

.

Our algorithm and GDMDBClustering [25] terminate once we reach the global minimum of the convergence test function $L\left(\theta \right)$. Consequently, the running times of our algorithm and GDMDBClustering [25] are most of the time shorter than that of BestDatabaseClustering [22]. Overall, the running time of GDMDBClustering [25] stays relatively steady with respect to n. However, GDMDBClustering depends strongly on the learning step size $\eta$ and its decay rate. On the other hand, our algorithm is learning-rate-free and needs at most (in the worst case) $(n^2-n)/2$ iterations to converge. Consequently, our proposed algorithm is faster than both BestDatabaseClustering [22] and GDMDBClustering [25].

To illustrate the statistically significant superiority of the proposed clustering model in terms of running time and clustering accuracy, we have applied the Friedman test [52] (under a significance level $\alpha =0.05$) on the measurements (execution times and clustering errors depicted in Figure A8, Figure A9 and Figure A10) obtained by our algorithm, BestDatabaseClustering [22] and GDMDBClustering [25] (with three different values for the learning rate $\eta$) considering all the random samples in Table 7.

After conducting the Friedman test [52], we obtained the results shown in

Table A5. Statistical Friedman test results [52] for the measurements obtained by all the compared clustering algorithms in Figure A8.

Algorithm	Measurements
	Running Time					Clustering Error
	Average	SD	Var	Stat	p-Value	Average	SD	Var	Stat	p-Value
Proposed Algo	6.367	3.018	9.107			0.285	0.080	0.006
BestDatabaseClustering [22]	47.208	27.537	758.313	135.707	3.40$\mathrm{e}$–30	0.936	0.066	0.004	150	$2.67\mathrm{e}$–33
GDMDBClustering [25] ($\eta =0.001$)	14.825	1.743	3.037			0.285	0.080	0.006

,

Table A6. Statistical Friedman test results [52] for the measurements obtained by all the compared clustering algorithms in Figure A9.

Algorithm	Measurements
	Running Time					Clustering Error
	Average	SD	Var	Stat	p-Value	Average	SD	Var	Stat	p-Value
Proposed Algo	6.367	3.018	9.107			0.285	0.080	0.006
BestDatabaseClustering [22]	47.208	27.537	758.313	121.62	3.88$\mathrm{e}$–27	0.936	0.066	0.004	131	$3.56\mathrm{e}$–29
GDMDBClustering [25] ($\eta =0.002$)	7.305	1.766	3.118			0.290	0.086	0.007

and

Table A7. Statistical Friedman test results [52] for the measurements obtained by all the compared clustering algorithms in Figure A10.

Algorithm	Measurements
	Running Time					Clustering Error
	Average	SD	Var	Stat	p-Value	Average	SD	Var	Stat	p-Value
Proposed Algo	6.367	3.018	9.107			0.285	0.080	0.006
BestDatabaseClustering [22]	47.208	27.537	758.313	118.90	$1.51\mathrm{e}$–26	0.936	0.066	0.004	150	2.67$\mathrm{e}$–33
GDMDBClustering [25] ($\eta =0.0005$)	28.479	4.655	21.669			0.285	0.080	0.006

, namely the average running time, the average clustering error $\overline{\mathcal{E}(\mathcal{P},\mathcal{Q})}$ (22), the standard deviation (SD), the variance (Var), the critical value (stat) and its p-value for all the tested clustering algorithms, considering all 91 random samples generated via scikit-learn [50].

We notice that all the obtained results in Table A5, Table A6 and Table A7 show p-values that are below the significance level $\alpha =0.05$. Consequently, the test suggests a rejection of the null hypothesis, stating that the compared clustering models have a similar performance. In fact, the proposed clustering algorithm significantly outperforms the other compared models, as it has the shortest average running time (6.367 milliseconds) and the lowest average clustering error ($\overline{\mathcal{E}(\mathcal{P},\mathcal{Q})}=0.285$) among all the compared models.

4.5. Clustering Comparison and Assessment

In the third part of our experiments, we are interested in using some information retrieval measures to compare the clusterings produced by our algorithm and some other clustering algorithms with the ground-truth data.

Let $\mathcal{D}=\{{\mathcal{D}}_1,{\mathcal{D}}_2,\dots ,{\mathcal{D}}_n\}$ be n transactional databases. Let $\mathcal{P}=\{P_1,P_2,\dots ,P_k\}$ be a k-class clustering of $\mathcal{D}$ produced by any given clustering algorithm, and let $\mathcal{Q}=\{Q_1,Q_2,\dots ,Q_l\}$ be the ground-truth clustering of the databases in $\mathcal{D}$, such that ${\cup }_{i=1}^k\left\{P_i\right\}={\cup }_{i=1}^l\left\{Q_i\right\}=\mathcal{D}$ and ${\cap }_{i=1}^k\left\{P_i\right\}={\cap }_{i=1}^l\left\{Q_i\right\}=\varnothing$. Let us define $Pair{s}_{\mathcal{P}}$ and $Pair{s}_{\mathcal{Q}}$ as the set of database pairs obtained from each cluster of the same clustering. That is, $Pair{s}_{\mathcal{P}}={\cup }_{P_t\in \mathcal{P}}{\cup }_{{\mathcal{D}}_r,{\mathcal{D}}_s\in P_t;r<s}\left\{({\mathcal{D}}_r,{\mathcal{D}}_s)\right\}$ and $Pair{s}_{\mathcal{Q}}={\cup }_{Q_t\in \mathcal{Q}}{\cup }_{{\mathcal{D}}_r,{\mathcal{D}}_s\in Q_t;r<s}\left\{({\mathcal{D}}_r,{\mathcal{D}}_s)\right\}$. To compare the two clusterings $\mathcal{P}$ with $\mathcal{Q}$, few methods method [53,54,55] could be used. In this paper, we use a pair counting [56,57,58,59] to calculate some information retrieval measures [60,61], including precision, recall, F-measure (i.e., harmonic mean of recall and precision), Rand index [62] and Jaccard index [63] over pairs of databases being clustered together in $\mathcal{P}$ and/or $\mathcal{Q}$. This will allow us to assess whether the predicted database pairs from $\mathcal{P}$ cluster together in $\mathcal{Q}$, i.e., are the discovered database pairs in $Pair{s}_{\mathcal{P}}$ correct with respect to the underlying true pairs in $Pair{s}_{\mathcal{Q}}$ from the ground-truth clustering $\mathcal{Q}$.

In

Table A9. Contingency matrix showing the categories in pairing clustered databases.

Clustering	Predicted Clusters
	Pairs in $\mathcal{P}$	Pairs Not in $\mathcal{P}$
Actual clusters
Pairs in$\mathcal{Q}$	$a:=|Pair{s}_{\mathcal{Q}}\cap Pair{s}_{\mathcal{P}}|$ (True Positive)	$b:=|Pair{s}_{\mathcal{Q}}\backslash Pair{s}_{\mathcal{P}}|$ (False Negative)
Pairs not in$\mathcal{Q}$	$c:=|Pair{s}_{\mathcal{P}}\backslash Pair{s}_{\mathcal{Q}}|$ (False Positive)	$d:=$ Pairs in none (True Negative)

, we show the categories of database pairs which represent the working set of all pair counting measures cited in

Table A10. Pair counting measures used for clustering assessment and comparison.

Precision [60,61]	Recall [60,61]	F-Measure [60,61]	Rand [62]	Jaccard [63]
$\frac{\left(a\right)}{\left(a\right)+\left(c\right)}$	$\frac{\left(a\right)}{\left(a\right)+\left(b\right)}$	$\frac{2\left(a\right)}{2\left(a\right)+\left(b\right)+\left(c\right)}$	$\frac{\left(a\right)+\left(d\right)}{\left(a\right)+\left(b\right)+\left(c\right)+\left(d\right)}$	$\frac{\left(a\right)}{\left(a\right)+\left(b\right)+\left(c\right)}$

. Precisely, a: represents the number of pairs that exist in both clusterings $\mathcal{Q}$ and $\mathcal{P}$, d: represents the number of pairs that do not exist in either clustering, b: is the number of pairs present only in clustering $\mathcal{Q}$, and c: is the number of pairs present only in clustering $\mathcal{P}$. By counting the pairs in each category, we get an indicator for agreement and disagreement of the two clusterings being compared. The following example illustrates how to compute the measures defined in Table A10 for two given clusterings $\mathcal{P}=\{\{{\mathcal{D}}_1,{\mathcal{D}}_2,{\mathcal{D}}_3\},\{{\mathcal{D}}_4,{\mathcal{D}}_5,{\mathcal{D}}_6,{\mathcal{D}}_7\}\}$ and the ground-truth partitioning $\mathcal{Q}=\{\{{\mathcal{D}}_1,{\mathcal{D}}_2,{\mathcal{D}}_3\},\left\{{\mathcal{D}}_4\right\},\{{\mathcal{D}}_5,{\mathcal{D}}_6,{\mathcal{D}}_7\}\}$ of seven transaction databases $\mathcal{D}={\cup }_{i=1}^7\left\{{\mathcal{D}}_i\right\}$. First, let us calculate the following pairing categories:

$$\begin{array}{cc}\hfill Pair{s}_{\mathcal{D}}=& {\cup }_{r=1}^6{\cup }_{s=r+1}^7\left\{({\mathcal{D}}_r,{\mathcal{D}}_s)\right\},\hfill \\ \hfill Pair{s}_{\mathcal{Q}}=& \{({\mathcal{D}}_6,{\mathcal{D}}_7),({\mathcal{D}}_5,{\mathcal{D}}_7),({\mathcal{D}}_5,{\mathcal{D}}_6),({\mathcal{D}}_2,{\mathcal{D}}_3),({\mathcal{D}}_1,{\mathcal{D}}_3),({\mathcal{D}}_1,{\mathcal{D}}_2)\},\hfill \\ \hfill Pair{s}_{\mathcal{P}}=& \{({\mathcal{D}}_6,{\mathcal{D}}_7),({\mathcal{D}}_5,{\mathcal{D}}_7),({\mathcal{D}}_5,{\mathcal{D}}_6),({\mathcal{D}}_4,{\mathcal{D}}_7),({\mathcal{D}}_4,{\mathcal{D}}_6),({\mathcal{D}}_4,{\mathcal{D}}_5),({\mathcal{D}}_2,{\mathcal{D}}_3),({\mathcal{D}}_1,{\mathcal{D}}_3),\hfill \\ & ({\mathcal{D}}_1,{\mathcal{D}}_2)\}.\hfill \end{array}$$

Then, $a=|Pair{s}_{\mathcal{Q}}\cap Pair{s}_{\mathcal{P}}|=6$, $b=|Pair{s}_{\mathcal{Q}}\backslash Pair{s}_{\mathcal{P}}|=0$, $c=|Pair{s}_{\mathcal{P}}\backslash Pair{s}_{\mathcal{Q}}|=3$, $d=|Pair{s}_{\mathcal{D}}\backslash (Pair{s}_{\mathcal{Q}}\cup Pair{s}_{\mathcal{P}})|=12$. Therefore, we get the following measures: $F-measure=0.8$, $precision=0.66$, $recall=1.0$, $Rand=0.857$, $Jaccard=0.66$. We note that the higher the values of the evaluation measures given in Table A10, the better the matching of the clustering $\mathcal{P}$ to its corresponding ground-truth clustering $\mathcal{Q}$.

In

Table A8. F-measure [60,61], precision [60,61] and recall [60,61] reached by the compared clustering algorithms in [21,22,23], and our proposed algorithm on the datasets shown in Figure A1, Figure A2, Figure A3, Figure A4, Figure A5, Figure A6, and Figure A7. Notice that our algorithm gets the best scores for all the datasets.

Dataset	F-Measure [60,61]				Precision [60,61]				Recall [60,61]
	Proposed Algo	Algo [22]	Algo [23]	Algo [21]	Proposed Algo	Algo [22]	Algo [23]	Algo [21]	Proposed Algo	Algo [22]	Algo [23]	Algo [21]
Figure A1 7 × 7 [20]	1	1	0.44	0.8	1	1	0.28	0.66	1	1	1	1
Figure A2 12 × 12 $Zoo$ [48]	1	1	0.14	0.14	1	1	0.075	0.075	1	1	1	1
Figure A3 4 × 4 $Mushroom$ [48]	1	0.66	0.66	1	1	0.5	0.5	1	1	1	1	1
Figure A4 6 × 6 $Iris$ [48]	1	1	1	1	1	1	1	1	1	1	1	1
Figure A5 6 × 6 $Zoo$ & $Mushroom$ [48]	1	1	1	1	1	1	1	1	1	1	1	1
Figure A6 4 × 4 [39]	1	0.66	0.66	1	1	0.5	0.5	1	1	1	1	1
Figure A7 10 × 10 T10I4D100K [49]	1	0.16	0.16	0.16	1	0.088	0.088	0.088	1	1	1	1

and

Table A11. Rand index [62] and Jaccard index [63] reached by the clustering algorithms in [21,22,23] and our proposed algorithm on the datasets shown in Figure A1, Figure A2, Figure A3, Figure A4, Figure A5, Figure A6, and Figure A7. Notice that our algorithm gets the best scores for all the datasets.

Dataset	Rand [62]				Jaccard [63]
	Proposed Algo	Algo [22]	Algo [23]	Algo [21]	Proposed Algo	Algo [22]	Algo [23]	Algo [21]
Figure A1 $7\times 7$ [20]	1	1	0.28	0.85	1	1	0.28	0.66
Figure A2 $12\times 12$ $Zoo$ [48]	1	1	0.075	0.075	1	1	0.075	0.075
Figure A3 $4\times 4$ $Mushroom$ [48]	1	0.5	0.5	1	1	0.5	0.5	1
Figure A4 $6\times 6$ $Iris$ [48]	1	1	1	1	1	1	1	1
Figure A5 $6\times 6$ $Zoo$ & $Mushroom$ [48]	1	1	1	1	1	1	1	1
Figure A6 $4\times 4$ [39]	1	0.5	0.5	1	1	0.5	0.5	1
Figure A7 $10\times 10$ T10I4D100K [49]	1	0.088	0.088	0.088	1	0.088	0.088	0.088

, we report the F-measure [60,61], precision [60,61], recall [60,61], Rand [62] and Jaccard [63] reached by the clustering algorithms in [21,22,23], and our proposed algorithm on the datasets shown in Figure A1, Figure A2, Figure A3, Figure A4, Figure A5, Figure A6, and Figure A7. From Table A8 and Table A11, we notice that our algorithm achieves the best scores against the compared clustering algorithms, considering all the experiments in Figure A1, Figure A2, Figure A3, Figure A4, Figure A5, Figure A6, and Figure A7.

5. Conclusions

An improved similarity-based clustering algorithm for multi-database mining was proposed in this paper. Unlike the previous works, our algorithm requires fewer upper-bounded iterations to minimize a convex clustering quality measure. In addition, we have proposed a preprocessing layer prior to clustering where the pairwise similarities between multiple databases are first adjusted to reduce their fuzziness. This will help the clustering process to be more precise and less confused in discriminating between the different database clusters. To assess the performance of our algorithm, we conducted several experiments on real and synthetic datasets. Compared to the existing clustering algorithms for multi-database mining, our algorithm achieved the best performance in terms of accuracy and running time. In this paper, we have used the most frequent itemsets mined from each transaction database as feature sets to compute the pairwise similarities between the multiple databases. However, when the sizes of these input vectors become large, building the similarity matrix will increase the CPU overhead drastically. Moreover, the existence of some noisy frequent itemsets (FIs) may largely influence how databases are clustered together. In future work, we will investigate the impact of compressing the size of the FIs into a latent variable represented in a lower dimensional space with discriminative features. Practically, reconstituting the input vectors from the embedding space using deep auto-encoders and non-linear dimensionality reduction techniques, such as T-SNE (t-distributed stochastic neighbor embedding) and UMAP (uniform manifold approximation and projection), will force the removal of the noisy features present in the input data while keeping only the meaningful discriminative ones. Consequently, this may help improve the accuracy and running time of the clustering algorithm. Additionally, we are interested in exploring new ways to reduce the computational time used to calculate the similarity matrix via locality sensitive hashing (LSH) techniques, such as BagMinHash for weighted sets. These methods aim to encode the feature-set vectors into hash-code signatures in order to efficiently estimate the Jaccard similarity between the local transactional databases. Last but not least, in order to design a parallel version of the proposed algorithm, we will study and explore some high-performance computing tools, such as MapReduce and Spark, as an attempt to improve the clustering performance for multi-database mining.