In-Network Processing of an Iceberg Join Query in Wireless Sensor Networks Based on 2-Way Fragment Semijoins

Abstract

We investigate the in-network processing of an iceberg join query in wireless sensor networks (WSNs). An iceberg join is a special type of join where only those joined tuples whose cardinality exceeds a certain threshold (called iceberg threshold) are qualified for the result. Processing such a join involves the value matching for the join predicate as well as the checking of the cardinality constraint for the iceberg threshold. In the previous scheme, the value matching is carried out as the main task for filtering non-joinable tuples while the iceberg threshold is treated as an additional constraint. We take an alternative approach, meeting the cardinality constraint first and matching values next. In this approach, with a logical fragmentation of the join operand relations on the aggregate counts of the joining attribute values, the optimal sequence of 2-way fragment semijoins is generated, where each fragment semijoin employs a Bloom filter as a synopsis of the joining attribute values. This sequence filters non-joinable tuples in an energy-efficient way in WSNs. Through implementation and a set of detailed experiments, we show that our alternative approach considerably outperforms the previous one.

2. Problem Statement

An iceberg join query Q for two virtual relations ${{𝓡}}_0$ and ${{𝓡}}_1$ on attribute A in WSNs is assumed to be submitted to the base station as a continuous query modeled as a sliding window join [9]. Each evaluation of the query is conducted against a window of ${{𝓡}}_0$ and that of ${{𝓡}}_1$, and the query result is returned to the base station. Initially, the base station forwards Q to three sensor nodes ${\tilde{n}}_0$, ${\tilde{n}}_1$, and $\tilde{m}$. ${\tilde{n}}_i$ is the coordinator node of the region for ${{𝓡}}_i$, i = 0, 1. $\tilde{m}$ is the node located at the midpoint between ${\tilde{n}}_0$ and ${\tilde{n}}_1$, which is to take part in in-network query optimization and processing. In each region, a routing tree whose root is ${\tilde{n}}_i$ is constructed with the standard routing tree construction algorithm of [1]. ${\tilde{n}}_i$disseminates Q to all the sensor nodes in the region. In each region, preprocessing is carried out to collect the aggregate count of each joining attribute value that is sampled in the window. Each sensor node in a region generates a node histogram, which is a set of (value, count) pairs in the node. Then, it sends the node histogram to its parent node in the routing tree. Eventually, ${\tilde{n}}_i$ obtains the region histogram. It is a binary relation with the joining attribute A and the count attribute. In [6], this relation is called the base histogram. Let us denote the base histogram of region ${{𝓡}}_0$ and ${{𝓡}}_1$ as $R_0$ and $R_1$, respectively. Now the problem is to fully reduce $R_0$ and $R_1$ such that only those tuples of them that are qualified for the iceberg join remain. Let $R{\prime }_0$ and $R{\prime }_1$ respectively denote the full reduction of $R_0$ and $R_1$. Then, the final result of Q can be obtained by two semijoins followed by a final join: (${{𝓡}}_0$ ${⋉}_{A=A}$ $R{\prime }_0$) ${\bowtie }_{A=A}$ (${{𝓡}}_1{⋉}_{A=A}$ $R{\prime }_1$). Once $R{\prime }_0$ and $R{\prime }_1$ are obtained, the operations to produce the final result of Q are straightforward. Thus, in this paper, we deal only with the problem of optimally obtaining $R{\prime }_0$ and $R{\prime }_1$.

3. Background

Our scheme employs the Bloom filter [10] and a variation of the 2-way semijoin [11,12]. In Section 3.1, semijoin and 2-way semijoin operations are briefly described. In Section 3.2, an overview of Bloom filter and its theory are given. In Section 3.3, the technique of SRJA [6] is described.

3.1. Semijoin and 2-Way Semijoin

The semijoin was proposed in [13] as a means to reduce join operand relations in distributed databases. Given a join $R_0$ ${\bowtie }_{A=A}$ $R_1$, semijoin $R_0{⋊}_{A=A}{R}_1$ reduces $R_1$ such that only those tuples of $R_1$ joinable with $R_0$ remain. Assuming that $R_0$ and $R_1$ reside at different sites in distributed databases, the semijoin $R_0{⋊}_{A=A}{R}_1$ is executed by sending $R_0$ [A] to $R_1$ and joining the two. That is, $R_0{⋊}_{A=A}{R}_1$ = $R_0\left[\text{A}\right]{\bowtie }_{A=A}{R}_1$. $R_1$ is said to be fully reduced by the semijoin. Let us call $R_0$ the reducer relation, and $R_1$ the reduced relation. The size of the result of semijoin $R_0{⋊}_{A=A}{R}_1$ is $\left|R_1\right|\cdot s$, where s is called the semijoin selectivity and it is estimated as |R[A]|/|D_A|, where $D_A$ is the domain of A [14].

The 2-way semijoin was investigated in [11,12,15] as an extension of the semijoin. It includes the backward reduction phase in addition to the forward reduction phase of the original semijoin such that both of the two join operand relations are fully reduced. Suppose semijoin $R_0{⋊}_{A=A}{R}_1$ reduces $R_1$ to $R{\prime }_1$. In the backward reduction phase of 2-way semijoin $R_0⋊{⋉}_{A=A}{R}_1$, $R{\prime }_1$[A] or its complement (i.e., $R_1$[A]$-R{\prime }_1$[A]) or a bit vector indicating which value of $R_1$[A] is joinable and which is not is sent back. In distributed query processing, this backward reduction is often effective and can be applied to a pipelined n-way join [12]. In our scheme proposed in this paper, the backward reduction is efficiently merged with the handling of the false positives inherent with the Bloom filter employed in implementing a semijoin.

3.2. Bloom Filter

A k-transform Bloom filter is a bit vector of length m that probabilistically represents a set S with k hash functions h₁(), $\cdots$, h_k() for k $\ge$ 1 [10]. Initially, all the m bits are set to 0. For each value x in S, h₁(x)-th, ⋯ , h_k(x)-th bits of the Bloom filter are set. For a given value y, y is not in S if any of the h₁(y)-th, ⋯, h_k(y)-th bits of the Bloom filter is not 1, whereas y is probably in S if all those k bits are 1. In the latter case, a false positive is possible due to the possibility of the collisions in hashing. However, it is guaranteed that false negatives are not possible.

For two relations $R_0$ and $R_1$ that reside at different locations, a Bloom filter can be employed in implementing semijoin $R_0{⋊}_{A=A}{R}_1$ with possible errors. First, $R_0$ is scanned to construct a Bloom filter $bf$ {$R_0,$A} that represents $R_0$ [A] $bf${$R_0,$A} is sent to $R_1$. Then, for each value v of $R_1$.A, the membership test of v $\in bf${$R_0,$A} is done to filter non-joinable tuples of $R_1$. None of the joinable tuples of $R_1$ is filtered out but some of the non-joinable ones could survive because of the false positives (i.e., $R_0{⋊}_{A=A}{R}_1\subseteq$ $R_0{⋊}_{A=A}^{bf}{R}_1\subseteq$ $R_1$, where ${⋊}_{A=A}^{bf}$ denotes the semijoin implemented using a Bloom filter). Since $bf${$R_0,$A} is a bit vector, it is usually much smaller than $R_0$[A]. In WSNs, it would be more energy-efficient to send $bf${$R_0,$A} than to send $R_0$[A], provided that the false positives could be properly handled.

When the length of a k-transform Bloom filter is m, and the number of values in S is n, the probability of a false positive is approximately:

$${\left(1-{\left(1-\frac{1}{m}\right)}^{kn}\right)}^k\approx {\left(1-e^{-\frac{kn}{m}}\right)}^k$$

(1)

and it is minimized to $1/2^k$ when $k=(m/n)\cdot \ln 2$ [16,17]. In this case, the number of bits used to represent a value in S is $m/n= k/\ln 2$.

3.3. SRJA

Given an iceberg join of ${{𝓡}}_0$ and ${{𝓡}}_1$ on attribute A in WSNs, SRJA works as follows [6]: The preprocessing as described in Section 2 is carried out to obtain $R_0$ and $R_1.$At ${\tilde{n}}_0$ and ${\tilde{n}}_1$, the values in $R_0\left[\text{A}\right]$ and $R_1\left[\text{A}\right]$ is respectively divided into a sequence of value ranges with the information on the count attribute associated with each range. A range is defined as a 4-tuple (minval, maxval, mincount, maxcount). minval and maxval are respectively the minimum and maximum value of A in the range, and the mincount and maxcount are respectively the minimum and maximum value of count among all the counts associated with the values of A in the range. The sequence of these ranges constitutes the synopsis of the values of A and count in $R_i$ (i = 0, 1). A value range represented as an interval would be much smaller than the list of all the values in the range. In WSNs, it is a common practice to send a value range (i.e., an interval [minval, maxval]) instead of the full list of values in the range to reduce data transmission cost though accuracy is compromised [18].

Figure 2. Protocol Executed by ${\tilde{n}}_0$ for SRJA.

${\tilde{n}}_0$ and ${\tilde{n}}_1$ send the synopses of $R_0$ and $R_1$ to the sensor node $\tilde{m}$ located at the midpoint between ${\tilde{n}}_0$ and ${\tilde{n}}_1$. At $\tilde{m}$, the value range matching is conducted first with the minval and maxval of the ranges in the synopses. In this matching process, the original synopses are modified. Some ranges of a synopsis are deleted because there is no matched counterpart in the other synopsis or further divided into subranges so that the two synopses have exactly the same set of ranges. Then, for each pair of matched ranges of $R_0$ and $R_1$, the mincount’s and maxcount’s are checked, and the ranges are tagged as PRUNE, JOIN, or DIVIDE. The pair for which maxcount($R_0$)$\times$maxcount($R_1$)$<\alpha$ is tagged as PRUNE. The pair for which mincount($R_0$)$\times$mincount($R_1$) $\ge \text{α}$ is tagged as JOIN. The remaining pairs are tagged as DIVIDE. The tagged synopses are sent back to ${\tilde{n}}_0$ and ${\tilde{n}}_1$. The tuples in a PRUNE range are deleted from $R_i$ (i = 0, 1). The tuples in a JOIN range not qualified for the query are filtered out by 2-way semijoins, and the qualified ones are finally joined for the result. A DIVIDE range is further divided into subranges. The synopsis of $R_i$ is reconstructed with them. This process is repeated until the final query result is obtained. The optimization techniques for SRJA are as follows [6]:

• Optimization 1: Before the above process begins, the tuples whose value of count exceeds α is sent separately for checking their joinability because they are likely to be qualified for the query.
• Optimization 2: The tuples whose value of A belongs to a sparse range are also separately sent. Eliminating a sparse range would make the synopsis more selective.
• Optimization 3: For a range to be tagged as DIVIDE, the maxcount of the opposite range (opp_maxcount) is also sent when the tagged synopses are sent back. When the range is divided into subranges, the tuples which turn out not to be qualified for the query (count$\times opp\_$maxcount$<\alpha$) are deleted.

The skeleton of the protocol executed by ${\tilde{n}}_0$ for SRJA is described in Figure 2. The one for ${\tilde{n}}_1$ is symmetrical. In [6], it was shown that SRJA significantly outperformed the following baseline schemes:

• NAÏVE: The external join where all the tuples of ${{𝓡}}_0$ and ${{𝓡}}_1$ are sent to the base station.
• SIJ: The synopsis join of [2] extended for iceberg joins where $R_0$ and $R_1$ are sent to $\tilde{m}$ and fully reduced there.

4. Overview of Our Approach

Given an iceberg join of ${{𝓡}}_0$ and ${{𝓡}}_1$ on attribute A in WSNs, suppose the preprocessing described in Section 2 has been carried out to obtain $R_0$ and $R_1.$In this section, the main components of our scheme in fully reducing $R_0$ and $R_1$ are described. They include 2-way fragment semijoin, Low Count Cut, Highest Count First strategy, and the sequence of 2-way fragment semijoins. The issue of optimization is dealt with in Section 5. The notations used in this section and Section 5 are summarized in Table 1.

Table 1. Notations.

Notation	Description
$R_i$, $R_{1-i}$	$R_0$ and $R_1$ when i = 0; $R_1$ and $R_0$ when i = 1
α	Iceberg threshold
${\tilde{n}}_i$	The coordinator sensor node in the region for $R_i$ (i = 0, 1)
$\tilde{m}$	The sensor node located at the midpoint between ${\tilde{n}}_0$ and ${\tilde{n}}_1$.
A	The joining attribute
$D_A$	The domain of A
||A||	The size of a value of A
$\overline{⋉}$, $\overline{⋊}$	2-way fragment semijoin operator
$R_i^n$	A logical fragment of $R_i$ defined as ${\text{σ}}_{count =n}{R}_i$ (i = 0, 1)
$M_i$	The initial maximum value of the attribute count in $R_i$ (i = 0, 1)
$H_i$	The current highest value of the attribute count in $R_i$ (i = 0, 1)
$L_i$	The current lowest value of the attribute count in $R_i$ (i = 0, 1)
$R_i\left(H_i,H_{1-i}\right)$	The current state of $R_i$ after reduced by a sequence of fragment semijoins (i = 0, 1)
$R_i^n\left(H_i,H_{1-i}\right)$	A logical fragment of $R_i\left(H_i,H_{1-i}\right)$ defined as ${\text{σ}}_{count =n}{R}_i\left(H_i,H_{1-i}\right)$ (i = 0, 1)
${\overline{⋉}}_i\left(H_i,H_{1-i}\right)$	A fragment semijoin for which $R_i^{H_i}\left(H_i,H_{1-i}\right)$ is the reducer relation
$S_{\overline{⋉}\text{*}}^i\left(H_i,H_{1-i}\right)$	Given $R_i\left(H_i,H_{1-i}\right)$ and $R_{1-i}\left(H_{1-i},H_i\right)$, the optimal sequence of fragment semijoins that fully reduces $R_0$ and $R_1$ provided that the first semijoin is ${\overline{⋉}}_i\left(H_i,H_{1-i}\right)$ (i = 0, 1)
$C_{\overline{⋉}\text{*}}^i\left(H_i,H_{1-i}\right)$	The cost of $S_{\overline{⋉}\text{*}}^i\left(H_i,H_{1-i}\right)$
${\widehat{c}}_{\overline{⋉}}^i\left(H_i,H_{1-i}\right)$	The cost of ${\overline{⋉}}_i\left(H_i,H_{1-i}\right)$
$n_{\overline{⋉}\text{*}}^i\left(H_i,H_{1-i}\right)$	The value of attribute count in $R_i$ with which (Equation (2)) is minimized (i = 0, 1). Let n = $n_{\overline{⋉}\text{*}}^i\left(H_i,H_{1-i}\right)$. Then, the following sequence of fragment semijoins is the prefix of $S_{\overline{⋉}\text{*}}^i\left(H_i,H_{1-i}\right)$: ${\overline{⋉}}_i\left(H_i,H_{1-i}\right)\cdot {\overline{⋉}}_i\left(H_i-1,H_{1-i}\right)\cdot {\overline{⋉}}_i\left(H_i-2,H_{1-i}\right)\cdot \cdot \cdot {\overline{⋉}}_i\left(n,H_{1-i}\right)$.
$B{F}_i\left(H_i,H_{1-i}\right)$	The Bloom filter sent for ${\overline{⋉}}_i\left(H_i,H_{1-i}\right)$
$\left|\right|B{F}_i\left(H_i,H_{1-i}\right)\left|\right|$	The size of $B{F}_i\left(H_i,H_{1-i}\right)$
$k_i\left(H_i,H_{1-i}\right)$	The optimal number of hash functions used for $B{F}_i\left(H_i,H_{1-i}\right)$
$C_{fp}^i\left(H_i,H_{1-i}\right)$	The cost of handling false positives with $B{F}_i\left(H_i,H_{1-i}\right)$ in executing ${\overline{⋉}}_i\left(H_i,H_{1-i}\right)$

Table 1. Notations.

Notation	Description
$R_i$, $R_{1-i}$	$R_0$ and $R_1$ when i = 0; $R_1$ and $R_0$ when i = 1
α	Iceberg threshold
${\tilde{n}}_i$	The coordinator sensor node in the region for $R_i$ (i = 0, 1)
$\tilde{m}$	The sensor node located at the midpoint between ${\tilde{n}}_0$ and ${\tilde{n}}_1$.
A	The joining attribute
$D_A$	The domain of A
||A||	The size of a value of A
$\overline{⋉}$, $\overline{⋊}$	2-way fragment semijoin operator
$R_i^n$	A logical fragment of $R_i$ defined as ${\text{σ}}_{count =n}{R}_i$ (i = 0, 1)
$M_i$	The initial maximum value of the attribute count in $R_i$ (i = 0, 1)
$H_i$	The current highest value of the attribute count in $R_i$ (i = 0, 1)
$L_i$	The current lowest value of the attribute count in $R_i$ (i = 0, 1)
$R_i\left(H_i,H_{1-i}\right)$	The current state of $R_i$ after reduced by a sequence of fragment semijoins (i = 0, 1)
$R_i^n\left(H_i,H_{1-i}\right)$	A logical fragment of $R_i\left(H_i,H_{1-i}\right)$ defined as ${\text{σ}}_{count =n}{R}_i\left(H_i,H_{1-i}\right)$ (i = 0, 1)
${\overline{⋉}}_i\left(H_i,H_{1-i}\right)$	A fragment semijoin for which $R_i^{H_i}\left(H_i,H_{1-i}\right)$ is the reducer relation
$S_{\overline{⋉}\text{*}}^i\left(H_i,H_{1-i}\right)$	Given $R_i\left(H_i,H_{1-i}\right)$ and $R_{1-i}\left(H_{1-i},H_i\right)$, the optimal sequence of fragment semijoins that fully reduces $R_0$ and $R_1$ provided that the first semijoin is ${\overline{⋉}}_i\left(H_i,H_{1-i}\right)$ (i = 0, 1)
$C_{\overline{⋉}\text{*}}^i\left(H_i,H_{1-i}\right)$	The cost of $S_{\overline{⋉}\text{*}}^i\left(H_i,H_{1-i}\right)$
${\widehat{c}}_{\overline{⋉}}^i\left(H_i,H_{1-i}\right)$	The cost of ${\overline{⋉}}_i\left(H_i,H_{1-i}\right)$
$n_{\overline{⋉}\text{*}}^i\left(H_i,H_{1-i}\right)$	The value of attribute count in $R_i$ with which (Equation (2)) is minimized (i = 0, 1). Let n = $n_{\overline{⋉}\text{*}}^i\left(H_i,H_{1-i}\right)$. Then, the following sequence of fragment semijoins is the prefix of $S_{\overline{⋉}\text{*}}^i\left(H_i,H_{1-i}\right)$: ${\overline{⋉}}_i\left(H_i,H_{1-i}\right)\cdot {\overline{⋉}}_i\left(H_i-1,H_{1-i}\right)\cdot {\overline{⋉}}_i\left(H_i-2,H_{1-i}\right)\cdot \cdot \cdot {\overline{⋉}}_i\left(n,H_{1-i}\right)$.
$B{F}_i\left(H_i,H_{1-i}\right)$	The Bloom filter sent for ${\overline{⋉}}_i\left(H_i,H_{1-i}\right)$
$\left|\right|B{F}_i\left(H_i,H_{1-i}\right)\left|\right|$	The size of $B{F}_i\left(H_i,H_{1-i}\right)$
$k_i\left(H_i,H_{1-i}\right)$	The optimal number of hash functions used for $B{F}_i\left(H_i,H_{1-i}\right)$
$C_{fp}^i\left(H_i,H_{1-i}\right)$	The cost of handling false positives with $B{F}_i\left(H_i,H_{1-i}\right)$ in executing ${\overline{⋉}}_i\left(H_i,H_{1-i}\right)$

4.1. 2-Way Fragment Semijoin

As described in Section 2, $R_0$ and $R_1$are the base histograms of ${{𝓡}}_0$ and ${{𝓡}}_1$ with attribute count. For relation $R_i$ (i = 0, 1), let $R_i^n$ denote the result of ${\text{σ}}_{count =n}{R}_i$. $R_i^n$ is a horizontal fragment of $R_i$ on count, and thus, a horizontal subset of the base histogram of ${{𝓡}}_i$. For example, given an iceberg join between ${{𝓡}}_0$ and ${{𝓡}}_1$ in Figure 1 on attribute A, $R_0=\text{{}\left(\text{A},\text{ count}\right) \text{|}\left(1,1\right), \left(2,1\right), \left(4,2\right)\}$, and $R_1=\text{{}\left(\text{A},\text{ count}\right) \text{|}\left(1,2\right), \left(2,1\right), \left(3,1\right)\}$. Thus, $R_0^1=\left\{\left(1,1\right), \left(2,1\right)\right\}$, $R_0^2=\left\{\left(4,2\right)\right\}$, $R_1^1=\left\{\left(2,1\right), \left(3,1\right)\right\}$, $R_1^2=\left\{\left(1,2\right)\right\}$.

Let us consider a logical fragmentation of $R_0$ and $R_1$on count. Suppose the lowest and highest value of count in $R_0$ is 1 and 7, respectively. Then, $R_0$ is logically partitioned into 7 fragments: $R_0^1$, $R_0^2$, …, $R_0^7$. Similarly, suppose $R_1$ is logically fragmented into $R_1^1$, …, $R_1^8$. Now let us consider an iceberg join with $\text{α}=$ 30. Since $R_0$ and $R_1$ are fragmented, the semijoin where a fragment is the reducer relation can be used. For example, semijoin $R_0^7$ ${⋊}_{A=A}$ $R_1$ is executed in the following way: $R_0^7\left[\text{A}\right]$ is sent to $R_1$. Since $\lceil \text{α}/7\rceil =5$, only those tuples of $R_1$ that belong to the fragments $R_1^j$ where $5\le j\le$8 could be joinable with the tuples in $R_0^7$. Thus, only ${{\displaystyle \cup }}_{j=5}^8{R}_1^j$ could be considered as the reduced relation. The tuples in those fragments not joinable with $R_0^7\left[\text{A}\right]$ are deleted.

Now let us define a new type of operation called fragment semijoin by modifying the semijoin. In the semijoin, the joinable tuples of the reduced relation remain whereas the non-joinable ones are deleted. In our fragment semijoin, the tuple filtering is done in the other way around with a side-effect. The joinable ones are deleted whereas the non-joinable ones remain. In other words, the result of a fragment semijoin is the complement of the conventional semijoin (in some commercial DBMSs, this variation of the semijoin is called an anti join; in fact, the term fragment anti join might be more exact one than fragment semijoin, however, we keep the term semijoin with a modifier “fragment” because it is widely known). Let us denote a fragment semijoin operator as $\overline{⋊}$. With $\text{α}=$ 30, a fragment semijoin $R_0^7$ ${\overline{⋊}}_{A=A}$ $R_1$ is executed in the following way: $R_0^7\left[\text{A}\right]$ is sent to $R_1$. The tuples in ${{\displaystyle \cup }}_{j=5}^8{R}_1^j$ not joinable with $R_0^7\left[\text{A}\right]$ are intact (i.e., not deleted). Instead, the joinable ones are deleted and inserted to a separate relation $R{\prime }_1.$ The reason why the unmatched tuples remain is that they might be joined with the tuples in other fragments of $R_0$. The insertion of matched tuples to $R{\prime }_1$ is a necessary side-effect of a fragment semijoin. Initially, $R{\prime }_1$ is empty. Every time a fragment semijoin to reduce $R_1$ is executed, the matched tuples, if any, are inserted to $R{\prime }_1$. When all the tuples of $R_1$ that are joinable with $R_0$ have been inserted to $R{\prime }_1$, $R_1$is said to be fully reduced to $R{\prime }_1$. Until then, $R_1$is said to be reduced to $R\prime {\prime }_1$, which keeps the tuples of $R_1$ yet to be checked for joinability with other fragments of $R_0$. Note that $R{\prime }_1{{\displaystyle \cup }}^{\text{}}R\prime {\prime }_1\subseteq R_1$. The difference, $R_1-\mathrm{\left(}R{\prime }_1{{\displaystyle \cup }}^{\text{}}R\prime {\prime }_1\right)$, is the set of tuples of $R_1$ that have been finally confirmed not joinable with $R_0$. How this difference is computed will be explained in the next two subsections. In a 2-way fragment semijoin, the backward reduction phase is added where the matched joining attribute values are sent back.

So far, we have assumed that the joining attribute values are sent in the forward reduction phase. In our scheme, we employ the Bloom filter as a synopsis of the joining attribute values in executing a fragment semijoin to reduce the amount of data transmission in WSNs. The fragment semijoin with a Bloom filter is the same as above except (1) the Bloom filter constructed from the joining attribute values are sent; and (2) the false positives need to be handled. With $\text{α}=$ 30, the fragment semijoin $R_0^7$ ${\overline{⋊}}_{A=A}$ $R_1$ using a Bloom filter is executed in the following way: The values in $R_0^7.\text{A}$ are represented in a Bloom filter, $bf$ {$R_0^7,$A}, which is sent to $R_1$. For each value of A in ${{\displaystyle \cup }}_{j=5}^8{R}_1^j$, the membership test is done with $bf$ {$R_0^7,$A}. The matched tuples (including those due to false positives) are moved to $R{\prime }_1$. Their values of A are sent back to $R_0$. The ones that turn out to have been sent due to false positives are sent back to $R_1$, and their corresponding tuples are moved from $R{\prime }_1$ back to their original fragments. Note that the backward reduction phase is mandatory with the fragment semijoin using a Bloom filter to sort out false positives. In the rest of this paper, what we mean by a fragment semijoin denoted with $\overline{⋊}$ is a 2-way fragment semijoin using a Bloom filter unless stated otherwise.

5. Optimal Sequence of Fragment Semijoins

In our approach, a sequence of fragment semijoins are executed to fully reduce $R_0$ and $R_1$, and the Bloom filter is employed as a synopsis of the joining attribute values. Data transmission for a fragment semijoin occurs to send the Bloom filter and to handle the false positives. According to the Bloom filter theory presented in Section 3.2, the length of a Bloom filter and the number of transformations used could be optimally set to minimize the probability of false positives. In this section, we present an algorithm that generates the optimal sequence of 2-way fragment semijoins with the HCF strategy whereby the total amount of data transmission in fully reducing $R_0$ and $R_1$ is minimized.

5.1. Formulation of Optimization Problem

We develop a dynamic programming algorithm to generate the optimal sequence of fragment semijoins that fully reduces $R_0$ and $R_1$. In describing the algorithm, it is convenient to denote $R_0$ and $R_1$ as $R_i$ and $R_{1-i}$ (i = 0, 1). For example, if we need to mention both $R_0$ $⋊$ $R_1$ and $R_1$ $⋊$ $R_0$ to state something that is applied to both (Note that $R_0$ $⋊$ $R_1$ $\ne$ $R_1$ $⋊$ $R_0$ because the semijoin operation is not commutative.), $R_i$ $⋊$ $R_{1-i}$ (i = 0, 1) will do. In the rest of this paper, the subscripts i and $1-i$ (e.g., $R_i$, $R_{1-i}$) are used with (i = 0, 1) omitted if they are related to $R_0$ or $R_1$ and the context is clear. Other notations used in this section are summarized in Table 1.

The current state of $R_i$ after a certain sequence of fragment semijoins has been executed can be represented with the two current highest counts in the remaining fragments of $R_i$ and $R_{1-i}$. Let $H_i$ denote the current highest count of $R_i$ as depicted in Figure 5. The following two statements hold:

• The current lowest count of $R_i$ is $\text{α}/H_{1-i}$ (Figure 5a).
• If the initial maximum count of $R_i$ is $M_i$, each fragment in ${{\displaystyle \cup }}_{j=H_i+1}^{M_i}{R}_i^j$ has been selected as the reducer relation for the fragment semijoins executed thus far before or after reduced by some fragments of $R_{1-i}$ (Figure 5b).

Let $R_i\left(H_i,H_{1-i}\right)$ denote $R_i$ reduced thus far, and $R_i^n\left(H_i,H_{1-i}\right)$ denote the fragment of $R_i\left(H_i,H_{1-i}\right)$ defined by ${\text{σ}}_{count =n}{R}_i\left(H_i,H_{1-i}\right)$. Given $R_i\left(H_i,H_{1-i}\right)$ and $R_{1-i}\left(H_{1-i}, H_i\right)$, let ${\overline{⋉}}_i\left(H_i,H_{1-i}\right)$ denote the fragment semijoin $R_i^{H_i}\left(H_i,H_{1-i}\right)\overline{⋊}$ $R_{1-i}\left(H_{1-i}, H_i\right)$, and $S_{\overline{⋉}\text{*}}^i\left(H_i,H_{1-i}\right)$ denote the optimal sequence of fragment semijoins provided that the first one is ${\overline{⋉}}_i\left(H_i,H_{1-i}\right).$ Let $C_{\overline{⋉}\text{*}}^i\left(H_i,H_{1-i}\right)$ denote the cost of $S_{\overline{⋉}\text{*}}^i\left(H_i,H_{1-i}\right)$, where the cost is defined to be the total amount of data transmission in bits. $C_{\overline{⋉}\text{*}}^i\left(H_i,H_{1-i}\right)$ is given in the following recurrence relation:

$$C_{\overline{⋉}\text{*}}^i\left(H_i,H_{1-i}\right)=\underset{\left|\frac{\text{α}}{H_{1-i}}\right|\le n\le H_i}{\min }\left\{{\displaystyle \sum }_{j=H_i}^n{\widehat{c}}_{\overline{⋉}}^i\left(j,H_{1-i}\right)+C_{\overline{⋉}\text{*}}^{1-i}\left(H_{1-i},n-1\right)\right\},i=0,1$$

(2)

${\widehat{c}}_{\overline{⋉}}^i\left(j,H_{1-i}\right)$ denotes the cost of ${\overline{⋉}}_i\left(j,H_{1-i}\right)$. Thus, the term ${\displaystyle \sum }_{j=H_i}^n{\widehat{c}}_{\overline{⋉}}^i\left(j,H_{1-i}\right)$ is the total cost of executing the following sequence of fragment semijoins: ${\overline{⋉}}_i\left(H_i,H_{1-i}\right)\cdot {\overline{⋉}}_i\left(H_i-1,H_{1-i}\right)\cdot {\overline{⋉}}_i\left(H_i-2,H_{1-i}\right)\cdot \cdot \cdot {\overline{⋉}}_i\left(n,H_{1-i}\right)$. With this sequence, the fragments $R_i^j\left(H_i,H_{1-i}\right)$ where $j=H_i$, $H_i-1,H_i-2,\cdots ,n$ are to be selected as the reducer relations in the descending order of j according to the HCF strategy.

Figure 5. State of $R_i$ and $R_{1-i}$ after a certain sequence of fragment semijoins has been executed. (a) The current lowest count of $R_i$ is ⌈$\text{α}/H_{1-i}$⌉, where $H_i$ denotes the current highest count of $R_i;$; (b) Let the initial maximum count of $R_i$ be $M_i$. Then, each fragment in ${{\displaystyle \cup }}_{j=H_i+1}^{M_i}{R}_i^j$ has been selected as the reducer relation for the fragment semijoins executed thus far before or after reduced by some fragments of $R_{1-i}$.

The termination condition of this recurrence relation is ⌈$\text{α}/H_{1-i}\rceil <H_i \text{or} \lceil \text{α}/H_i\rceil <H_{1-i}$, which means either one of the two relations gets empty. In such a case, no further fragment semijoin is needed. Thus:

$$C_{\overline{⋉}\text{*}}^i\left(H_i,H_{1-i}\right)=0\text{ if }\lceil \frac{\text{α}}{H_{1-i}}\rceil <H_i \text{or} \lceil \frac{\text{α}}{H_i}\rceil <H_{1-i}$$

(3)

Let $n_{\overline{⋉}\text{*}}^i\left(H_i,H_{1-i}\right)$ denote the value n for which $C_{\overline{⋉}\text{*}}^i\left(H_i,H_{1-i}\right)$ in (Equation (2)) is minimized. That is,

$$n_{⋉\text{*}}^i\left(H_i,H_{1-i}\right)=\underset{\left|\frac{\text{α}}{H_{1-i}}\right|\le n\le H_i}{minarg}\left\{{\displaystyle \sum }_{j=H_i}^n{\widehat{c}}_{\overline{⋉}}^i\left(j,H_{1-i}\right)+C_{\overline{⋉}\text{*}}^{1-i}\left(H_{1-i},n-1\right)\right\},i=0,1$$

(4)

Let $M_i$ be the maximum count in $R_i$. Then, the optimal sequence of fragment semijoins for $R_0$ and $R_1$ is either $S_{\overline{⋉}\text{*}}^0\left(M_0,M_1\right)$ or $S_{\overline{⋉}\text{*}}^1\left(M_1,M_0\right)$. Its cost is $\min \left\{C_{\overline{⋉}\text{*}}^0\left(M_0,M_1\right),C_{\overline{⋉}\text{*}}^1\left(M_1,M_0\right)\right\}$or $\underset{i\in \left\{0,1\right\}}{\min }\left\{C_{\overline{⋉}\text{*}}^i\left(M_i,M_{1-i}\right)\right\}$. Let $j=\underset{i\in \left\{0,1\right\}}{minarg}\left\{C_{\overline{⋉}\text{*}}^i\left(M_i,M_{1-i}\right)\right\}$. The value of j, which is 0 or 1, indicates that the first fragment semijoin in the optimal sequence is ${\overline{⋉}}_j\left(M_j,M_{1-j}\right)$ where $R_j^{M_j}$ is the reducer relation. Thus, the optimal sequence can be represented as i ($i\in \left\{0,1\right\})$followed by a sequence of counts in $R_i$ and $R_{1-i}$ as follows:

i; $n_i^1, n_{1-i}^1, n_i^2, n_{1-i}^2, \cdots , n_i^k, n_{1-i}^k, \cdots ,$

where

$$n_i^1=n_{\overline{⋉}*}^i\left(M_i, M_{1-i}\right)$$

$$n_{1-i}^1=n_{\overline{⋉}*}^{1-i}\left(M_{1-i}, n_i^1-1\right)$$

$$n_i^2=n_{\overline{⋉}*}^i\left(n_i^1-1, n_{1-i}^1-1\right)$$

$$n_{1-i}^2=n_{\overline{⋉}*}^{1-i}\left(n_{1-i}^1-1, n_i^2-1\right)$$

For example, if the sequence of fragment semijoins in Figure 4 is the optimal one, then it is represented as 1, 9, 7, 6, 5, 5.

5.2. Cost of 2-Way Fragment Semijoin

The cost of a fragment semijoin ${\widehat{c}}_{\overline{⋉}}^i\left(x,y\right)$ where i = 0, 1 is defined as the total amount of data transmission in bits, and given as follows:

$${\widehat{c}}_{\overline{⋉}}^i\left(x,y\right)=\left|\right|B{F}_i\left(x,y\right)\left|\right|+C_{fp}^i\left(x,y\right)$$

(5)

The first term denotes the size of the Bloom filter, and the second is for the cost of handling false positives. Since the total cost of sending all the joinable attribute values (excluding those from false positives) in the backward reduction phases is the same for all the possible sequences of fragment semijoins, it is omitted. The multiplication of the number of hops between the two regions of $R_0$ and $R_1$ is also omitted because it is the same for all the sequences.

5.2.1. Size of Bloom Filter

||$B{F}_i\left(x,y\right)$|| denotes the size of the Bloom filter $B{F}_i\left(x,y\right)$, which represents the set of joining attribute values in $R_i^x\left(x,y\right)$ to be sent in executing ${\overline{⋉}}_i\left(x,y\right)$. When this fragment semijoin is to be executed, the reduced state of $R_i$ and $R_{1-i}$ are respectively $R_i\left(x,y\right)$ and $R_{1-i}\left(y,x\right)$. That is, $H_i$= x, $H_{1-i}$= y, and $B{F}_i\left(x,y\right)$ is to represent the set of joining attribute values in $R_i^x\left(x,y\right)$. The joining attribute values in $R_i$ are unique among all the tuples. Thus, the number of distinct values in the joining attribute A of $R_i^x\left(x,y\right)$ is $\left|R_i^x\left(x,y\right)\right|$. This number can be estimated as follows using the semijoin selectivity estimation described in Section 3.1:

$$\left|R_i^x\left(x,y\right)\right|=\left|R_i^x\right|\cdot \left(1-\frac{\left|{{\displaystyle \cup }}_{j=y+1}^{M_{1-i}}{R}_{1-i}^j\right|}{\left|D_A\right|}\right)$$

(6)

$R_i$ and $R_{1-i}$ have been reduced by each other with a fragment semijoin sequence until ($H_i,H_{1-i}$) becomes (x, y). However, the above estimation is valid since the joining attribute values are unique in $R_i$, that is, $R_i^p{{\displaystyle \cap }}^{\text{}}{R}_i^q$ = $\varnothing$ (p$\ne$q), and thus, the following holds: $R_i^p$ $⋉R_{1-i}=$ $R_i^p$ $⋉$ $R_{1-i}$ $⋉R_i^q$) where p$\ne$q.

Now $\left|\right|B{F}_i\left(x,y\right)\left|\right|$. can be estimated as follows according to the Bloom filter theory described in Section 3.2:

$$\left|\right|B{F}_i\left(x,y\right)\left|\right|=\frac{\left|R_i^x\left(x,y\right)\right|\cdot k_i\left(x,y\right)}{\ln 2}$$

(7)

where $k_i\left(x,y\right)$ denotes the optimal number of hash functions for $B{F}_i\left(x,y\right)$, which will be explained shortly. In some cases, the number of tuples of $R_i^x\left(x,y\right)$ is so small that the size of the list of joining attribute values might be smaller than $\left|\right|B{F}_i\left(x,y\right)\left|\right|$. In such a case, the list is sent instead of $B{F}_i\left(x,y\right).$ To cover such exceptions, we modify the above equation as follows:

$$||B{F}_i\left(x,y\right)||=min\left\{\frac{\left|R_i^x\left(x,y\right)\right|\cdot k_i\left(x,y\right)}{\ln 2},\left|R_i^x\left(x,y\right)\right|\cdot ||A||\right\}+1$$

(8)

where $\left|\right|A\left|\right|$ denotes the number of bits to represent a value in the joining attribute A, and the one added is for a flag bit (for distinguishing a Bloom filter from a value list).

5.2.2. Cost of Handling False Positives

The cost of handling the false positives is estimated as follows: Let D, E, and F respectively denote the number of joining attribute values in $R_{1-i}$ for which the membership tests with $B{F}_i\left(x,y\right)$ are to be done, the number of non-joinable tuples among those D values, and the number of false positives out of those E values. Then:

$$D=\left|{{\displaystyle \cup }}_{j=\left|\frac{\text{α}}{x}\right|}^y{R}_{1-i}^j\left(y,x\right)\right|, E=D\cdot \left(1-\frac{\left|{\text{R}}_i^x\left(x,y\right)\right|}{\left|D_A\right|}\right), \text{and} F=E\cdot \frac{1}{2^{k_i\left(x,y\right)}}$$

(9)

The estimation of E is according to the semijoin selectivity estimation as described in Section 3.1. In the equation of F, $1/2^{k_i\left(x,y\right)}$ is the probability of a false positive as summarized in Section 3.2. The values of the false positives are sent back to $R_i$ in the backward reduction phase, and then sent back to $R_{1-i}$ after the false positives are confirmed. Thus, the cost of handling the false positives for a fragment semijoin is 2$\cdot$F$\cdot \left|\right|A\left|\right|$. That is:

$$C_{fp}^i\left(x,y\right)=\frac{1}{2^{k_i\left(x,y\right)-1}}\cdot \left|{{\displaystyle \cup }}_{j=\left|\frac{\text{α}}{x}\right|}^y{R}_{1-i}^j\left(y,x\right)\right|\cdot \left(1-\frac{\left|{\text{R}}_i^x\left(x,y\right)\right|}{\left|D_A\right|}\right)\cdot ||A||$$

(10)

In case that the list of joining attribute values is sent instead of $B{F}_i\left(x,y\right)$, $C_{fp}^i\left(x,y\right)=$0.

5.2.3. Optimal Number of Hash Functions.

$k_i\left(x,y\right)$, the optimal number of hash functions for $B{F}_i\left(x,y\right)$, is determined as follows: as summarized in Section 3.2, the probability of false positives is affected by the number of hash functions. As the number of hash functions increases, the length of the Bloom filter increases with the probability of false positives decreased. In (Equation (5)), the cost of a fragment semijoin, ${\widehat{c}}_{\overline{⋉}}^i\left(x,y\right)$, is given as a sum of two terms, $\left|\right|B{F}_i\left(x,y\right)\left|\right|$ and $C_{fp}^i\left(x,y\right)$. There exists a tradeoff between the two terms, and thus, $k_i\left(x,y\right)$ should be determined to be a positive integer such that ${\widehat{c}}_{\overline{⋉}}^i\left(x,y\right)$ is minimized. From Equations (8) and (10), the cost of a fragment semijoin ${\overline{⋉}}_i\left(x,y\right)$ with k hash functions is given as follows:

$$\frac{\left|R_i^x\left(x,y\right)\right|\cdot k}{\ln 2}+\frac{1}{2^{k-1}}\cdot \left|{{\displaystyle \cup }}_{j=\left|\frac{\text{α}}{x}\right|}^y{R}_{1-i}^j\left(y,x\right)\right|\cdot \left(1-\frac{\left|{\text{R}}_i^x\left(x,y\right)\right|}{\left|D_A\right|}\right)\cdot ||A||+1$$

(11)

This equation can be given as a function of $k$: $f\left(k\right)=\text{a}\cdot k+b/2^k+c$, where a, b, and c are constants. The differentiation of $f\left(k\right)$ with respect to $k$ reveals that $k_i\left(x,y\right)=\lceil {\log }_2\left(\left(b/a\right)\cdot \ln 2\right)\rceil$, where a$=\left|R_i^x\left(x,y\right)\right|/\ln 2$, b$=2\cdot \left|{{\displaystyle \cup }}_{j=\text{α}/x}^y{R}_{1-i}^j\left(y,x\right)\right|\cdot \left(1-\left(\left|R_i^x\left(x,y\right)\right|/\left|D_A\right|\right)\right)\cdot \left|\right|A\left|\right|$. If this value is less than 1, $k_i\left(x,y\right)$ is set to 1.

5.3. Dynamic Programming Algorithm

$C_{\overline{⋉}\text{*}}^{\text{i}}$ ($x,y$) and $n_{⋉\text{*}}^i$ ($x,y$) can be obtained with a dynamic programming algorithm. From (Equation (2)) through (Equation (4)), $C_{\overline{⋉}\text{*}}^{\text{i}}$ ($x,y$) and $n_{⋉\text{*}}^i$ ($x,y$) are initialized as follows: If there is no fragment remaining in either one of $R_0$ and $R_1$, that is, if $x<\lceil \text{α}/y\rceil$ or $y<\lceil \text{α}/x\rceil$, then $C_{\overline{⋉}\text{*}}^i$ ($x,y$) is set to 0, and $n_{⋉\text{*}}^i$ ($x,y$) is undefined. If none of $R_0$ and $R_1$ is empty but there remains only one fragment in $R_i$ (i.e., $x=\lceil \text{α}/y\rceil$), the only possible fragment semijoin is the one where that fragment is the reducer relation. Thus, $C_{\overline{⋉}\text{*}}^i$($x,y$)$={\widehat{c}}_{\overline{⋉}}^i\left(x,y\right)$ and $n_{⋉\text{*}}^i$ ($x,y$)$=x$. Starting from these initializations, the optimal solution for the cases where the number of remaining fragments of $R_0$ and $R_1$ is greater than 1 can be obtained. For example, suppose there are two fragments remaining in $R_0$ and one in $R_1$ (Figure 6a). $S_{\overline{⋉}\text{*}}^0\left(H_0,H_1\right)$ can be obtained as follows: Because of the HCF strategy, the fragment with the highest count is the reducer relation in the first semijoin (Figure 6b). The remaining fragment of $R_0$ might be the reducer relation of the next semijoin (Figure 6c). That is, Figure 6b,c show all the possible cases. Figure 6d,e respectively show the subsequent fragment semijoin where the fragment of $R_1$ is the reducer relation after the semijoins in Figure 6b,c assuming that no LCC has occurred. The optimal solutions (i.e., $S_{\overline{⋉}\text{*}}^1\left(H_1,H_0\right))$in Figure 6d,e are already known, because the number of remaining fragment in $R_0$ is 1 (Figure 6d) or 0 (Figure 6e), and that in $R_1$ is 1. The optimal solutions here are already given from the initialization. In this way, $S_{\overline{⋉}\text{*}}^i\left(H_i,H_{1-i}\right)$ can be obtained when there are 3, 4,…, $M_i-\lceil \text{α}/M_{1-i}\rceil + 1$fragments remaining in $R_i$ (i = 0, 1). Figure 7 describes the algorithm that carries out this process.

In the algorithm, procedure opt(i, $L_0$, $H_0$, $L_1$, $H_1$) is invoked. As shown in Figure 8a, the arguments $L_0$ and $H_0$ are respectively the current lowest and highest count in $R_0$. $L_1$ and $H_1$ are those in $R_1$. Procedure opt(i, $L_0$;, $H_0$, $L_1$, $H_1$) vacuously returns unless $L_0=\lceil \text{α}/H_1\rceil$ and $L_1=\lceil \text{α}/H_0\rceil$. If the arguments are valid ones, it finds $S_{\overline{⋉}\text{*}}^i\left(H_i,H_{1-i}\right)$. When i = 0, opt(0, $L_0$, $H_0$, $L_1$, $H_1$) finds out a value n in the range [$L_0$, $H_0$] that minimizes ${\displaystyle \sum }_{j=H_0}^n{\widehat{c}}_{\overline{⋉}}^0\left(j,H_1\right)+C_{\overline{⋉}\text{*}}^1\left(H_1,n-1\right)$. (Figure 8a corresponds to the first term, and Figure 8b to the second one.) In doing so, $C_{\overline{⋉}\text{*}}^0$($H_0$, $H_1$) and $n_{⋉\text{*}}^0$($H_0$, $H_1$) are obtained. Similarly, when i = 1, opt(1, $L_0$, $H_0$, $L_1$, $H_1$) finds out a value n in the range [$L_1$, $H_1$] that minimizes ${\displaystyle \sum }_{j=H_1}^n{\widehat{c}}_{\overline{⋉}}^1\left(j,H_0\right)+C_{\overline{⋉}\text{*}}^0\left(H_0,n-1\right)$. $C_{\overline{⋉}\text{*}}^1$($H_1$, $H_0$) and $n_{⋉\text{*}}^1$($H_1$, $H_0$) are obtained.

Figure 6. Optimal Solution from Initialization. (a) Two fragments remaining in $R_0$ and one in $R_1$. There are only two possible cases for $S_{\overline{⋉}\text{*}}^0\left(H_0,H_1\right)$, (b) One case, (c) The other case, (d) The optimal solution (i.e., $S_{\overline{⋉}\text{*}}^1\left(H_1,H_0\right)$) for the case of (b) is given from the initialization, (e) The same for the case of (c).

Figure 7. Algorithm for Generating Optimal Sequence of Fragment Semijoins.

Figure 8. Optimal Solution Obtained with Procedure opt(i, $L_0$, $H_0$, $L_1$, $H_1$), where $L_i$ and $H_i$ are respectively the current lowest and highest count in $R_i$. (a) The optimal value of n for $S_{\overline{⋉}\text{*}}^i\left(H_i,H_{1-i}\right)$ is obtained; (b) using the optimal solution, $S_{\overline{⋉}\text{*}}^{1-i}\left(H_{1-i},n-1\right)$, for each value of n.

The variable num_frag denotes the number of remaining fragments in $R_i$, and opp_num_frag denotes that in $R_{1-i}$. In the two outermost loops, the values of these variables increase. In the two innermost loops, pairings of $H_0$ and $H_1$ are provided such that the difference between $H_i$ and $L_i$ is set by num_frag and opp_num_frag, i = 0, 1. The invocations of opt() in Line 8 and 9 find the optimal solution for the case where the number of fragments in the reducer relation is greater than that in the reduced relation. Those in Line 11 and 12 find the optimal solution for the inverse case.

For example, suppose four and three fragments remain in $R_0$ and $R_1$, respectively (Figure 9a). Let us consider $S_{\overline{⋉}\text{*}}^0\left(H_0,H_1\right)$ for them. Figure 9b–e show that there could be four cases possible with the HCF strategy. After one, $\cdots$, four fragments of $R_0$ are used as the reducer relations in succession, the remaining fragments are shown at the bottom of Figure 9b–e, respectively. To find out which case of the four would lead to $S_{\overline{⋉}\text{*}}^0\left(H_0,H_1\right)$, $S_{\overline{⋉}\text{*}}^1\left(H_1,H_0\right)$ for the fragments in each of the bottom of Figure 9b–e should be already known (see (Equation (2))). In other words, before $S_{\overline{⋉}\text{*}}^0\left(H_0,H_1\right)$ for the fragments in Figure 9a is obtained, $S_{\overline{⋉}\text{*}}^1\left(H_1,H_0\right)$ for the fragments in each of the bottom of Figure 9b–e should have been obtained. The algorithm in Figure 7 guarantees this by calling procedure opt() in proper order through the nested loops.

Figure 9. An Example of Optimization with Procedure opt(). (a) 4 and 3 fragments remaining in $R_0$ and $R_1$, respectively, (b–e) For the remaining fragments in (a), 4 cases that could result in the optimal solution, $S_{\overline{⋉}\text{*}}^0\left(H_0,H_1\right)$, are shown at the top. Determining which case leads to the optimal solution is possible, because the optimal solution, $S_{\overline{⋉}\text{*}}^1\left(H_1,H_0\right)$, for the remaining fragments in each of the bottom of (b–e) is already known.

The complexity of the algorithm inFigure 7 is $\text{Ο}\left(f_0\cdot f_1\cdot \left(\frac{f\left(f+1\right)}{2}-1\right)\right)$ where $f_i=M_{\text{i}}-\lceil \alpha /M_{1-\text{i}}\rceil +1$, which is the number of fragments in $R_i$ remaining after initial LCCs (i = 0, 1), and f = max($f_0$, $f_1$). The innermost two for loops (line 6 and 7 in Figure 7) combined are repeated $f_0\cdot f_1$ times. The second outermost for loop (line 5) is repeated i times when num_frag = i at the outermost for loop. Since num_frag at the outermost for loop varies from 2 to f, the computation in the nested for loops (lines 8–13) would be repeated $f_0\cdot f_1\cdot \left(2+\dots +f\right)$ times. The complexity would be $\text{Ο}\left(f^4\right)$ when $f_0=f_1=f$. Due to the characteristics of iceberg queries which are supposed to produce very selective result, $M_i$’s and $\alpha$ would grow or shrink together. Otherwise, the query would not be so selective. It means that the number of fragments remaining after initial LCCs would not be high. When $M_0=M_1=15$, and $\alpha =150$, for example, $f_0=f_1=f$ = 6. When $M_0=M_1=15$, and $\alpha =200$, $f_0=f_1=f$ = 2.

5.4. Post Optimization

So far, we have assumed that the backward reduction phase is carried out for each fragment semijoin in the optimal sequence. As described in Section 4.1, the values for which false positives are confirmed need to be sent back again and their corresponding tuples need to be inserted back to the reduced relation. In the backward reduction, if the matched joining attribute values are sent back with their counts, re-insertion would not be necessary at all. In doing so, the concern is the overhead of sending the count for each individual matched value. An efficient way of handling this problem is to delay the backward reduction of each fragment semijoin until the execution of all the fragment semijoins in the optimal sequence is completed, and to conduct the backward reductions all at once. For this, $R_i$ sends $R_{1-i}$ a message of the form: $n_j$, $n_{j+1}$, …, $n_{M_i}$, $V_j$, $V_{j+1}$, …, $V_{M_i}$ where $j=\lceil \text{α}/M_{1-i}\rceil$, $V_j$ is the set of matched joining attribute values (possibly including those from false positives) whose count is equal to j, and $n_k=\left|V_k\right|(k=j,j+1,\cdots$, $M_i$).

5.5. Query Optimization and Processing

In-network optimization of an iceberg join query Q is carried out in the sensor node $\tilde{m}$ located at the midpoint between ${\tilde{n}}_0$ and ${\tilde{n}}_1$. Thus, it is required for ${\tilde{n}}_0$ and ${\tilde{n}}_1$ to send the information necessary for the optimization to $\tilde{m}$. First, ${\tilde{n}}_i$ sends the count information of $R_i$ (i.e., $\left|R_i^1\right|$, $\cdots$, $\left|R_i^{M_i}\right|$) to $\tilde{m}$. $\left|$ and $\left|D_A\right|$ are assumed to have already been sent to $\tilde{m}$ when Q was initially forwarded to ${\tilde{n}}_0$, ${\tilde{n}}_1$, and $\tilde{m}$. All the equations inSection 5 and Section 3.2 can be evaluated if the aforementioned information is available. The node $\tilde{m}$ generates the optimal sequence of fragment semijoins and sends it to ${\tilde{n}}_i$ with the count information of $R_{1-i}$ (i.e., $\left|R_{1-i}^1\right|$, $\cdots$, $\left|R_{1-i}^{M_{1-i}}\right|$). The count information of $R_i$ as well as the optimal sequence as described in Section 5.1 are represented as a short sequence of integers. Thus, the communication overhead for optimization could be very small. Now ${\tilde{n}}_0$ and ${\tilde{n}}_1$ are ready to execute the optimal sequence to fully reduce $R_0$ and $R_1$.The skeleton of the protocol executed by ${\tilde{n}}_0$ for our scheme is described in Figure 10. The one for ${\tilde{n}}_1$ is symmetrical.

Figure 10. Protocol Executed by ${\tilde{n}}_0$ for Our Scheme.

6. Performance Evaluation of Our Scheme and SRJA

In this section, we compare the performance of our scheme and that of SRJA. We have implemented both of our scheme and SRJA, measuring the total number of packets transmitted among the sensor nodes and the total number of transmissions among the sensor nodes while the two schemes are executed. These two performance metrics are employed to compare the energy-efficiency of the two schemes. The number of packets transmitted is measured assuming that the network is IEEE 802.15.4-compliant. We also measured the ratio of joinable values transmitted over all the values transmitted. This metric is to compare the effectiveness of data filtering as well as the energy-efficiency of the two schemes. Finally, we present an analytical comparison of the total number of transmissions in the two schemes. All the procedures for the experiments were implemented in C and the experiments were conducted in a system of Windows 7 with an AMD Phenom II X4 945 Processor (3.0 GHz) and 4 GB memory.

6.1. Parameters

The network and query parameters in the experiments are summarized inTable 2. We have considered WSNs where sensor nodes are uniformly deployed. Each of the two regions for an inter-region iceberg join where a join operand relation resides is assumed to be a square consisting of n × n nodes. The distance between the coordinator nodes ${\tilde{n}}_0$ and ${\tilde{n}}_1$ of the two regions is set to 30 hops.

Table 2. Parameter Settings in the Experiments.

Parameter	Value
Size of a region (n$\times$n nodes)	n = 5, 10, 15
The distance between ${\tilde{n}}_0$ and ${\tilde{n}}_1$	30 hops
The range of joining attribute values	1..10,000
The range of count	1..15
Epoch (i.e., sampling interval)	10, 20, 30, 40, 50, 60 s(default: 30 s)
Query window size	3 h
Iceberg threshold ($\text{α}$)	50, 100, 150, 200

Table 2. Parameter Settings in the Experiments.

Parameter	Value
Size of a region (n$\times$n nodes)	n = 5, 10, 15
The distance between ${\tilde{n}}_0$ and ${\tilde{n}}_1$	30 hops
The range of joining attribute values	1..10,000
The range of count	1..15
Epoch (i.e., sampling interval)	10, 20, 30, 40, 50, 60 s(default: 30 s)
Query window size	3 h
Iceberg threshold ($\text{α}$)	50, 100, 150, 200

The default epoch (i.e., sampling interval) of every sensor node is set to 30 s, and the size of the sliding window of the join query is set to 3 h as in the experiments with SRJA in [5]. After setting the join selectivity between $R_0$ and $R_1$, the values of the joining attribute in the iceberg join operand relations are randomly generated as an integer in the range [1,10,000] with a random distribution of their counts under the constraint that the highest count for a value could be 15. The iceberg thresholds considered are 50, 100, 150 and 200.

6.2. Experimental Results

Each of the reported measurements in this subsection is the average out of 50 runs against different data values. The experimental results reveal that our scheme considerably outperforms SRJA. Figure 11a,b respectively compare the total number of packets transmitted and the total number of transmissions of the two schemes as the size of each region varies from 5 × 5 nodes to 15 × 15 nodes while $\text{α}$ is set to 150. Figure 11c,d compare the same while $\text{α}$ is set to 200. As the size of each region gets bigger, more data is collected at the sensor nodes and more tuples need to be processed in both schemes. Thus, more packets are transmitted. As for the number of transmissions, it also increases in SRJA. In our scheme, it is not so sensitive to the data volume because for a given $\text{α}$, the optimal sequence of fragment semijoins generated could include the similar number of semijoins. For the total number of packets transmitted, the average performance improvement with our scheme over SRJA is 63.58% ($\text{α}=$ 150) and 71.97% ($\text{α}=$ 200). For the total number of transmissions, the average performance improvement with our scheme over SRJA is 59.66% ($\text{α}=$ 150) and 57.27% ($\text{α}=$ 200).

Figure 12a,b respectively compare the total number of packets transmitted and the total number of transmissions of the two schemes as $\text{α}$ varies from 50 to 200 while the size of each region is set to 10 × 10 nodes. Figure 12c,d compare the same while the size of each region is set to 15 × 15 nodes. As $\text{α}$ increases, the iceberg join query gets more selective. Thus, in both schemes, more tuples are filtered and the number of packets transmitted decreases. The number of transmissions turns out not so sensitive to the increase of $\text{α}$ except for the case of $\text{α}=200$. In SRJA, the effect of subrange pruning results in the decrease of transmissions for larger $\text{α}$’s. In our scheme, on the other hand, the number of transmissions slightly increases from $\text{α}=50$ to 150, then decreases for $\text{α}=200$. These changes depend on the optimal sequence of fragment semijoins generated. The more semijoins are to be executed in the optimal sequence, the more transmissions would occur. For the total number of packets transmitted, the average performance improvement with our scheme over SRJA is 59.71% (regions of 10 × 10 nodes) and 64.07% (regions of 15 × 15 nodes). For the total number of transmissions, the average performance improvement with our scheme over SRJA is 63.98% (regions of 10 × 10 nodes) and 66.13% (regions of 15 × 15 nodes).

Figure 11. Performance Comparison with respect to Varying Sizes of a Region. (a) Total number of packets transmitted ($\text{α}=150$); (b) Total number of transmissions ($\text{α}=150$); (c) Total number of packets transmitted ($\text{α}=200$); (d) Total number of transmissions ($\text{α}=200$).

Figure 12. Performance Comparison with respect to Varying Iceberg Thresholds. (a) Total number of packets transmitted (10 × 10 nodes); (b) Total number of transmissions (10 × 10 nodes); (c) Total number of packets transmitted (15 × 15 nodes); (d) Total number of transmissions (15 × 15 nodes).

Figure 13a,b respectively compare the total number of packets transmitted and the total number of transmissions of the two schemes as the epoch (i.e., sampling interval) at each node varies from 10 s to 60 s while the size of each region is set to 10 × 10 nodes and $\text{α}$ is set to 150. As the sampling rate gets higher, more data is collected at the sensor nodes and both schemes are supposed to transmit more packets. The number of transmissions also increases. However, SRJA turns out to suffer much more with higher sampling rates. For the total number of packets transmitted, the average performance improvement with our scheme over SRJA is 62.97%. For the total number of transmissions, the average performance improvement with our scheme over SRJA is 58.66%.

Figure 14a compares the ratio of joinable values transmitted in the two schemes as the size of each region varies from 5 × 5 nodes to 15 × 15 nodes while $\text{α}$ is set to 150. This ratio is defined as:

$$\frac{\text{Total number of joinable attribute values }}{\text{Total number of join attribute values transmitted}}$$

(12)

Figure 14b compares the same as $\text{α}$ varies from 50 to 200 while the size of each region is set to 10 × 10 nodes. This ratio gets lower as the query gets more selective or as more non-joinable values are transmitted. As the size of each region gets bigger, the number of tuples collected at the sensor nodes increases and the size of the iceberg join result also increases with a given $\text{α}$. Thus, this ratio increases in both schemes (Figure 14a). As $\text{α}$ increases, on the other hand, this ratio decreases in both schemes because the iceberg join query gets more selective (Figure 14b). In Figure 14, this ratio in our scheme turns out to be significantly higher than that in SRJA. This means that the effectiveness of filtering out non-joinable values and the energy-efficiency in our scheme is much higher than that in SRJA.

The major reasons for the improvements are two-fold:

• In SRJA, the histogram-based value ranges are sent as a synopsis of the joining attribute values. In our scheme, a Bloom filter constructed from a count-based fragment is sent as a synopsis of the joining attribute values. The Bloom filter is more compact than the value ranges. Besides, the false positives are efficiently handled in the backward reduction phase of the 2-way fragment semijoins in our scheme.

SRJA is centered around checking the join predicate with the cardinality constraint as an additional condition. For each pair of matched ranges, if there exists at least one pair of tuples t₀ $\in R_0$ and t₁ $\in R_1$such that t₀.count $\times$ t₁.count $\ge$ $\text{α}$, the non-joinable tuples in either range cannot be filtered out, and recursive divisions of each of the two ranges into subranges are required. In contrast, our scheme is centered around the cardinality constraint with the join predicate as the secondary condition. Only the fragment semijoins between the fragments satisfying the cardinality constraint are carried out to filter non-joinable tuples. Besides, the reductions from the LCCs that result with the HCF strategy are effective.

Figure 13. Performance Comparison with respect to Varying Epochs. (10 × 10 nodes, $\text{α}=150$). (a) Total number of packets transmitted; (b) Total number of transmissions.

Figure 14. Comparison of the Ratio of Joinable Values Transmitted with respect to (a) varying sizes of a region ($\text{α}=150$); (b) varying iceberg thresholds (10 × 10 nodes).

6.3. Analytical Comparison of the Total Number of Transmissions

In this subsection, we present an analytical comparison of the total number of transmissions among the sensor nodes throughout the execution of SRJA and our scheme. In Figure 2 and Figure 10, the protocols executed by the coordinator node of a region for SRJA and our scheme are described. In each of the two schemes, let $X_{i,1-i}$ be the number of messages that ${\tilde{n}}_i$ sends to ${\tilde{n}}_{1-i}$ plus the number of messages that ${\tilde{n}}_i$ receives from ${\tilde{n}}_{1-i}$. Let $Y_i$ be the number of messages that ${\tilde{n}}_i$ sends to $\tilde{m}$ plus the number of messages that ${\tilde{n}}_i$ receives from $\tilde{m}$. Let $M_x=$ $X_{01}+X_{10}$ and $M_y=$ $Y_0+Y_1$. Then, the total number of transmissions in each scheme is $d\cdot M_x+\frac{1}{2}d\cdot M_y$ where d is the distance between ${\tilde{n}}_0$ and ${\tilde{n}}_1$ in hops.

6.3.1. SRJA

Considering the protocol in Figure 2, we have:

$$X_{i,1-i}=H_{i,1-i}+S_{i,1-i}+{{\displaystyle \sum }}_{k=1}^{N_r}\left(f_k^J\cdot J_{i,1-i}+f_k^D\cdot S_{i,1-i}\right)$$

(13)

$H_{i,1-i}$ denotes the number of messages ${\tilde{n}}_i$ sends to ${\tilde{n}}_{1-i}$ plus the number of messages ${\tilde{n}}_i$ receives from ${\tilde{n}}_{1-i}$ in processing the values with a high count (line 2 in Figure 2), $S_{i,1-i}$ denotes that in processing sparse subranges in the synopsis (line 4 or line 16), and $J_{i,1-i}$ denotes that in processing JOIN-tagged subranges (line 13). $f_k^J$ is 1 if the tagged synopsis received from $\tilde{m}$ includes at least one JOIN-tagged subrange. It is 0 otherwise. $f_k^D$ is 1 if the tagged synopsis includes at least one DIVIDE-tagged subrange. It is 0 otherwise. The sparse subranges could be generated after the synopsis is initialized (line 3) or after the DIVIDE-tagged subranges are divided (line 15). Finally, $N_r$ denotes the number of rounds needed until SRJA is terminated. It means how many times the loop (line 5 through line 18) is repeated either fully or partially (up to line 9 for breaking the loop). Thus, $N_r\ge 1$. Now $M_x$ is:

$$\left(H_{01}+H_{10}\right)+(S_{01}+S_{10})+{{\displaystyle \sum }}_{k=1}^{N_r}\left(f_k^J\cdot (J_{01}+J_{10})+f_k^D\cdot (S_{01}+S_{10})\right)$$

(14)

Normally, $H_{01}= 2$ because a 2-way semijoin is executed. ${\tilde{n}}_0$ first sends the values with a high count with their count information to ${\tilde{n}}_1$, and then receives from ${\tilde{n}}_1$ the list of joinable values as a result (line 2). ${\tilde{n}}_1$ does the same symmetrically. If these two symmetrical processes are conducted as an asymmetrical one, one message can be saved, and thus, the number of transmissions could be significantly reduced when the distance between ${\tilde{n}}_0$ and ${\tilde{n}}_1$ is long. We can let ${\tilde{n}}_0$ start the process by sending its data to ${\tilde{n}}_1$. When ${\tilde{n}}_1$ returns the result, it can send its data as well. Then, ${\tilde{n}}_0$ finally returns the result. In this way, $H_{01}+H_{10}=3$. Similarly, $S_{01}+S_{10}=3$. Meanwhile, $J_{01}+J_{10}= 2$ because one 2-way semijoin in either direction but not the symmetrical two is enough in processing the JOIN-tagged subranges. Thus:

$$M_x=6+{{\displaystyle \sum }}_{k=1}^{N_r}\left(2f_k^J+3f_k^D\right)$$

(15)

Meanwhile, $Y_i= 2N_r$ because ${\tilde{n}}_i$ is to send the synopsis to $\tilde{m}$ (line 7) and receive the tagged synopsis from $\tilde{m}$ (line 8) in each round. Thus, $M_y= 4N_r$. Let $T_S$ be the total number of transmissions in SRJA. Then:

$$T_S=d\cdot (6+{{\displaystyle \sum }}_{k=1}^{N_r}\left(2f_k^J+3f_k^D\right)+2N_r)$$

(16)

6.3.2. Our Scheme

Considering the protocol in Figure 10, we have $X_{i,1-i}=$ $F_{i,1-i}+ B_{i,1-i}$, where $F_{i,1-i}$ denotes the number of messages ${\tilde{n}}_i$ sends to ${\tilde{n}}_{1-i}$in the forward reduction phase (line 9 through line 15 in Figure 10), while $B_{i,1-i}$ denotes the number of messages ${\tilde{n}}_i$ sends to ${\tilde{n}}_{1-i}$ plus the number of messages ${\tilde{n}}_i$ receives from ${\tilde{n}}_{1-i}$ in the backward reduction phase (line 17 through line 20). Now $M_x$ is $\left(F_{01}+F_{10}\right)+(B_{01}+B_{10})$.

In executing the optimal sequence of fragment semijoins, $R_0$ and $R_1$ are supposed to take turns to play the role of reducer relation. Thus, the number of messages including the Bloom filters for a subsequence of fragment semijoins in the forward reduction phase is equal to the number of this turn overs denoted as $N_t$. For example, in the sequence of fragment semijoins in Figure 4, there are 5 turn overs with $R_1$ being the reducer relation at first in Figure 4b. Thus, $F_{01}+F_{10}= N_t$. With the same argument for why $H_{01}+H_{10}=3$in SRJA, $B_{01}+B_{10}=3$. Thus, $M_x=$ $N_t+3$.

Meanwhile, $Y_i=$ 2 because ${\tilde{n}}_i$ sends the count information of $R_i$ to $\tilde{m}$ once (line 2) and receive from $\tilde{m}$ the optimal sequence and the count information of $R_{1-i}$ once (line 3). Let $T_O$ be the total number of transmissions in our scheme. Then:

$$T_O=d\cdot \left(N_t+5\right)$$

(17)

From (Equations (16) and (17)), $T_O<$ $T_S$ if $N_t<$ ${\displaystyle \sum }_{k=1}^{N_r}\left(2f_k^J +3f_k^D\right)+2N_r+1$. For the comparison, let us not consider the restricted case of $N_r=1$. This case happens in SRJA only when at least one of $R_0$ and $R_1$ has very small number of tuples and their join attribute values are sparsely distributed. Such a case is not of much interest. Now since $N_r \ge$ 2, we have ${\displaystyle \sum }_{k=1}^{N_r}\left(2f_k^J +3f_k^D\right)\cong 5(N_r-1)$. The reason for this is as follows: in Figure 10, we note that $f_{N_r}^J=f_{N_r}^D=0$ for the final round. In each of the interim rounds, we note that $f_k^D=1$ because without any DIVIDE-tagged subrange the next round except the final one would not be necessary. In each of the interim rounds, it is not necessarily that $f_k^J=1$. Assuming that normally it holds, $T_O<$ $T_S$ iff $N_t<$ 7$N_r-4$. The values of $N_t$ and $N_r$ depend on several parameters such as the number of tuples, distribution of joining attribute values, their count distribution, iceberg threshold, effectiveness of non-joinable value filtering, and so on. In the experiments in the previous subsection, it turns out that $N_t<$ 3 while $N_r \ge$ 2 on average. For such values, $T_O<$ $T_S$ as shown by the measurements in the experiments.

7. Related Work

In-network processing of joins in WSNs has received much attention [3]. The state-of-the-art techniques deal with various types of join queries with different types of spatio-temporal characteristics. Spatially, the type of join that has received so far the most attention is the inter-region join where each of the join operand relation is stored at a region of WSNs [2,6,19,20,21,22,23,24]. The pair-wise joins where pairing of sensor nodes for join could be determined by predicates are investigated in [25,26]. Temporally, one-shot join against a static join operand relations are dealt with in [2,23,27], while the join submitted as a continuous query against the streaming join operand relations are dealt with in [19,20,21,24,25,26,28,29]. As for the type of join predicates, equijoin queries are considered in [2,19,20], while theta-joins are considered in [20,21,22,23,24,25,26,27,28,29,30].

These techniques mostly adapt the conventional join implementations to WSNs (nested-loop join [20,21,22,23], hash join [2,19,21], and the sort-merge join [19]). The semijoins and 2-way semijoins are also employed and adapted to WSNs for filtering of non-joinable tuples [2,23,24,27,29]. In [28,30], on the other hand, a filtering approach is proposed rather than adapting the conventional join algorithms to WSNs. The cost-based optimizations are investigated in [21,22,23,24,25,26,28].

In [21], a technique called Distribute-Broadcast Join adapted from the nested-loop join is proposed. The main contribution of this work is the cost-based selection of optimal join region in WSNs. In [22,23], Mediated Join also adapted from the nested-loop join is proposed, where cost-based selection of inner and outer relation for nested-loop join is investigated. In [20], distributed algorithms for indexed nested-loop join and hash join are proposed. In the former, a technique of dynamically creating and using a distributed B+ tree in WSNs is developed. In the latter, a technique of partitioning and joining tuples with geographic hashing is investigated. In [25,26], a pair-wise join between any two sensor nodes is investigated. Multiple routing trees are employed and cost-based join initiation for long-running join query is proposed. Also, the issue of adaptive and cost-based re-optimization against the changes of sampled data is dealt with. In [2], Synopsis Join adapted from hash join as well as semijoin is proposed. It employs geographic hashing for partitioning and filtering of the joining attribute values and optimally determines the nodes of the final joins for each matched value. In [29], Two-Phase Self Join where one join operand relation is fully reduced with a semijoin is proposed. To process a join query, it employs a query decomposition technique assuming that the selection predicate on one relation is highly selective. In [27], SENS-Join based on a 2-way semijoin is proposed. It handles a general type of join predicates on multiple attributes, using a quad-tree as a multi-dimensional join filter based on Z-ordering. In [19], PEJA adapted from hash join as well as sort-merge join is proposed. In WSNs, physical sorting of the tuples distributed over the sensor nodes is infeasible. Thus, it conducts logical sort of tuples through division of joining attribute value range, partitioning and filtering tuples with geographic hashing. In [28,30], algorithms using the join filters are proposed for continuous queries on multiple attributes. At each sensor node involved in the join query processing, a join filter is installed for each joining attribute and only those tuples whose join attribute values pass the relevant filters are sent to the base station.

Other issues addressed by the state-of-the-art includes routing protocols, query dissemination, join initiation, involvement of the base station in in-network processing, collection of metadata for continuous joins [3].

In [18,24,31], in-network join processing where the Bloom filter is employed in WSNs are addressed. In [18], for a join between an external relation and the virtual relation in WSNs, a technique where a Bloom filter constructed with the joining attribute values of the external relation is injected into the network is investigated. In [24], a technique where the Bloom filter is transmitted instead of the joining attribute values in in-network computation of the two semijoins $R_0$ $⋉R_1$ and $R_0$ $⋊$ $R_1$ to answer a join $R_0$ $\bowtie$ $R_1$, is proposed. In this technique, when the Bloom filter constructed for $R_0$ is disseminated through the routing tree of $R_1$, it is selectively forwarded to the subtrees of the routing tree such that the cost of sending it is always compensated in reduction. The optimal solution in such a selective forwarding is proposed. In [31], an extension of the Bloom filter called Window Bloom filter (WBF) is devised to support a general join query with a time window. The proposed technique represents join attribute values in a compact way with the WBF, and saves the energy consumption by not sending the redundant join attribute values.

As described in Section 3.3, in SRJA [6], the value range instead of the joining attribute values is sent as a histogram-based synopsis. Transmitting the value ranges instead of the joining attribute values is considered for in-network join processing [18]. In [2], a technique using a histogram-based synopsis of the join operand relations coupled with geographic hashing is proposed. However, the considered join in these techniques is not an iceberg join.

The iceberg query was first introduced in [32]. It is defined as a query that retrieves aggregate values above some specified threshold in the applications such as data warehousing, data mining, information retrieval, and so on. Iceberg query processing over distributed data was also investigated [33,34,35]. However, the join query was not dealt with in these work. In [7], an iceberg distant join in spatial databases was investigated. In this work, the type of the join query is different from the one we consider in this paper in that the cardinality constraint is only on one join operand relation. That is, when the iceberg threshold is $\text{α}$, the tuple of one relation that is joined with more than $\text{α}$ tuples of the other relation is qualified for the query.

8. Conclusions

In this paper, we investigated an alternative approach to processing an iceberg join in WSNs, and described an optimized scheme. In the previous approach, the join predicate is checked first, and the cardinality constraint is checked next. In our approach, the order is reversed. Our scheme refers to the aggregate count of each of the joining attribute values, logically fragmenting the relations on the counts. Based on the fragmentation, it generates the optimal sequence of 2-way fragment semijoins using a Bloom filter as a synopsis of joining attribute values in filtering non-joinable tuples. A detailed set of experiments showed that our approach is substantially superior to the previous one.

As a future work, we plan to extend our scheme for more complicated cases. First, the two regions producing the virtual relations to be joined could partially or fully overlap with each other. Secondly, more than two regions are involved in multiple joins. Thirdly, the correlations among the sensor readings along the neighboring n regions are to be monitored with an n-way join.