Most of the previous work on data streams assume that items in a data stream are synchronous in the sense that the order of their arrivals is the same as the order of their creations. This synchronous model is however not suitable to applications that are distributed in nature. For example, in a sensor network, the sink collects data transmitted from sensors over a large area, and the data transmitted from different sensors would suffer different delay. It is possible that an item created at time t at a certain sensor may arrive at the sink later than an item created after t at another sensor. From the sink's viewpoint, items in the data stream are out of order with respect to their creation times. Yet the statistics to be computed are usually based on the creation times. More specifically, an asynchronous data stream (a.k.a. out-of-order data stream) [1,10,11] can be considered as a sequence (a₁, t₁), (a₂, t₂), (a₃, t₃), …, where a_i is the name of a data item chosen from a fixed universe U, and t_i is an integer timestamp recording the creation time of this item. Items arriving at the data stream are in arbitrary order regarding their timestamps, and it is possible that more than one data item has the same timestamp.

Consider a data stream and, in particular, those data items whose timestamps fall into the last W time units (W is the size of the sliding window). An item (or precisely, an item name) is said to be a frequent item if its count (i.e., the number of occurrences) exceeds a certain required threshold of the total item count. Arasu and Manku [6] were the first to study approximating frequent items over a sliding window under the synchronous model, in which data items arrive in non-decreasing order of timestamps. The space complexity of their data structure is O ( 1 ∊ ( log 1 ∊ ) 2 log ( ∊ B ) ), where ∊ is a user-specified error bound and B is the maximum number of items with timestamps falling into the same sliding window. Their work was later improved by Lee and Ting [7] to O ( 1 ∊ log ( ∊ B ) ) space. Recently, Cormode et al. [1] initiated the study of frequent items under the asynchronous model, and gave a solution with space complexity O ( 1 ∊ log W log ( ∊ B log W ) min { log W , 1 ∊ } log | U | ), where U is the set of possible item names. Later, Cormode et al. [12] gave a hashing-based randomized solution using O ( 1 ∊ 2 log | U | ) space. The space complexity is quadratic in 1 ∊, which is less preferred, but that is a general solution that can solve other problems like finding the sum and quantiles.

The earlier work on asynchronous data stream focused on a relatively simpler problem called ∊-approximate basic counting [10,11]. Cormode et al. [1] improved the space complexity of basic counting to. O ( 1 ∊ log W log ( ∊ B log W ) ) Notice that under the synchronous model, the best data structure requires O ( 1 ∊ log ( ∊ B ) ) space [9]. It is believed that there is roughly a gap of logW between the synchronous model to the asynchronous model. Yet, for frequent items, the asynchronous result of Cormode et al. [1] has space complexity way bigger than that of the best synchronous result, which is O ( 1 ∊ log ( ∊ B ) ) [7]. This motivates us to study more space-efficient solutions for approximating frequent items in the asynchronous model.

For any time interval I and any data item a, let f_a(I) denote the frequency of item a in interval I, i.e., the number of arrived items named a with timestamps falling into I. Define f_*(I) = Σ_a∈U f_a(I) to be the total number of all arrived items with timestamps within I.

Given a user-specified error bound ∊ and a window size W, we want to maintain a data structure to answer any ∊-approximate frequent item set query for any sub-window (specified at query time), which is in the form (ϕ, W′) where ϕ ∈ [∊, 1] is the required threshold and W′ ≤ W is the sub-window size. Suppose that τ_cur is the current time. The answer to such a query is a set S of item names satisfying the following two conditions:

(C1) S contains every item a whose frequency in interval I = [τ_cur − W′ + 1, τ_cur] is at least ϕf_*(I), i.e., f_a(I) ≥ ϕf_*(I).

The set S is also called an ∊-approximate ϕ-frequent item set. For example, assume ∊ = 1%, then the query (10%, 10, 000) would return all items whose frequencies in the last 10, 000 time units are each at least 10% of the total item count, plus possibly some other items with frequency at least 9% of the total count.

This paper gives a more space-efficient data structure for answering any ∊-approximate frequent item set query. Our data structure uses O ( 1 ∊ log W log ( ∊ B log W ) log log W ) words, which is significantly smaller than the one given by Cormode et al. [1] (see Table 1). Furthermore, this space complexity is larger than the best synchronous solution by only a factor of O(logW log logW), which is close to the expected gap of O(logW). Similar to existing data structures for this problem, it takes time linear to the data structure's size to answer an ∊-approximate frequent item set query. Furthermore, it takes O ( log ( ∊ B log W ) ( log 1 ∊ + log log W ) ) time to modify the data structure for a new data item. Occasionally, we might need to clean up some old data items that are no longer significant to the approximation; in the worst case, this takes time linear to the size of the data structure, and thus is no bigger than the query time. As a remark, the solution of Cormode et al. [1] requires O ( log ( ∊ B log W ) log W log log | U | ) time for an update.

In the asynchronous model, if a data item has a delay more than W time units, it can be discarded immediately when it arrives. In many applications, the delay is usually small. This motivates us to extend the asynchronous model to consider data items that have a bounded delay. We say that an asynchronous data stream has tardiness d_max if a data item created at time t must arrive at the stream no later than time t + d_max. If we set d_max = 0, the model becomes the synchronous model. If we allow d_max ≥ W, this is in essence the asynchronous model studied above. We adapt our data structure to take advantage of small tardiness such that when d_max is small, it uses smaller space (see Table 1) and support faster update time (which is O ( log ( ∊ B log d max ) ( log 1 ∊ + log log d max ) ) ) In particular, when d_max = Θ(1), the size and update time of our data structure match those of the best data structure for synchronous data stream.

To solve the frequent item set problem, we need to estimate the frequency of any item with relative error ∊f_*(I) where I = [τ_cur − W + 1, τ_cur] is the interval covered by the sliding window. To this end, we first propose a simple data structure for estimating the frequency of a fixed item over the sliding window. Then, we adapt a technique of Misra and Gries [14] to extend our data structure to handle any item. The result is an O(f_*(I))/λ)-space data structure that allows us to obtain an estimate for any item with an error bound of about λ logW. Here λ is a design parameter. To ensure λ logW to be no greater than ∊f_*(I), we should set λ ≤ ∊f_*(I)/logW. Since f*(I) can be as small as Θ ( 1 ∊ log W ) (the case for smaller f_*(I) can be handled by brute-force), we need to be conservative and set λ to some constant. But then the size of the data structure can be Θ(B) because f_*(I) can be as large as B. To reduce space, we introduce a multi-resolution approach. Instead of using one single data structure, we maintain a collection of O(logB) copies of our data structure, each uses a distinct, carefully chosen parameter λ so that it could estimate the frequent item set with sufficient accuracy when f_*(I) is in a particular range. The resulting data structure uses O ( 1 ∊ log W log B ) space.

Unfortunately, a careful analysis of our data structure reveals that in the worst case, it can only guarantee estimates with an error bound of ∊f_*(H ∪ I) where H = [τ_cur − 2W + 1, τ_cur − W], not the required ∊f_*(I). The reason is that the error of its estimates over I depend on the number of updates made during I, and unlike synchronous data stream, this number for asynchronous data stream can be significantly larger than f_*(I). For example, at time τ_cur − W + 1, there may still be many new items (a, u) with timestamps u ∈ H, for which we must update our data structure to get good estimates when the sliding window is at earlier positions. Indeed, the number of updates during I can be as large as f_*(H ∪ I), and this gives an error bound of ∊f_*(H ∪ I).

To reduce the error bound to ∊f_*(I), we introduce a novel algorithm to split the data structure into independent smaller ones at appropriate times. For example, at time τ_cur − W + 1, we can split our data structure into two smaller ones D_H and D_I, and we will only update D_H for items (a, u) with u ∈ H and update D_I for those with u ∈ I. Then, when we need to find an estimate on I at time τ_cur, we only need to consult D_I, and the number of updates made to it is f_*(I). In this paper, we develop sophisticated procedures to decide when and how to split the data structure so as to enable us to get good enough estimates when sliding window moves continuously. The resulting data structure has size O ( 1 ∊ ( log W ) 2 log ( ∊ B log W ) ) Then, we further make the data structure adaptive to the input size, allowing us to reduce the space to O ( 1 ∊ ( log log W ) log W log ( ∊ B log W ) ).

Let ∊₀ = ∊/4. Given _∊o and _∊o, we can answer any ∊-approximate frequent item set query. The total space required is O ( 1 ∊ log W log ( ∊ B log W ) log log W ).

The data structure _∊ solves the ∊-approximate counting problem. The space usage is O ( 1 ∊ log W log ( ∊ B log W ) log log W ) and it supports O ( log ( ∊ B log W ) ⋅ ( log 1 ∊ + log log W ) ) update time.

Any two interesting intervals J and H do not cross, although one can contain another, i.e., either J ⊂ H, or H ⊂ J, or J ∩ H = ∅. Furthermore, any interesting interval has at most logW ancestors.

For any node N, let (N) be the set of intervals that have been monitored by N so far. The following fact can be verified from the update procedure.

Consider a node N in Q I , λ a, where i(N) = J.

We say that N covers or is to the right of t if the intervals in (N) cover or are to the right of t; otherwise, N is to the left of t. For any queue Q I , λ a, let alive ( Q I , λ a ) be the set of nodes currently in Q I , λ a, dead ( Q I , λ a ) be those nodes of Q I , λ a that have already been deleted (because of Line 5 of the procedure Process( )), and node ( Q I , λ a ) = alive ( Q I , λ a ) ∪ dead ( Q I , λ a ). Note that the estimate f̂_a([t, r_i]) is the value sum of the nodes in alive ( Q I , λ a ) that cover or are to the right of t. For simplicity, we need to express it more succinctly. Let alive ( C I , λ , κ ) = ∪ { alive ( Q I , λ a ) ∣ Q I , λ a ∈ C I , λ , κ }be the set of nodes currently in _I,λ,κ. Define dead( _I,λ,κ) and node( _I,λ,κ) similarly. For any item a and any subset X ⊆ node( _I,λ,κ), let X^a be the set of nodes in X that are monitoring a (and thus are the nodes from Q I , λ a). For any t ∈ I, let X_≥t denote the set of nodes in X that cover or are to the right of t. Define v(X) = Σ_N∈X v(N) and d(X) = Σ_N∈X d(N). Then, f̂_a([t, r_I]) can be expressed as follows: f ^ a ( [ t , r I ] ) = v ( alive ( Q I , λ a ) ≥ t ) = v ( alive ( C I , λ , κ ) ≥ t a )

The following theorem analyzes its accuracy, as well as gives the size of _I,λ,κ.

For any t ∈ I, f_a([t, r_I]) − 1 κf_*(I) ≤ f̂_a([t, r_I]) ≤ f_a([t, r_I]) + λ logW. Furthermore, _I,λ,κ has size O(f_*(I)/λ + κ) words.

Let J be an interesting interval, and L = 〈 _J0,λ,κ, …, _Jm,λ,κ〉 be the list returned by LeftRefine( _J,λ,κ). Then, the list 〈 _J1,λ,κ, …, _Jm,λ,κ 〉 (i.e., the list obtained by throwing away the head _J0,λ,κ of L) is an interesting-partition covering [p + 1, q].

For example, if [1, 8] is an interesting interval, then the list 〈 _{[2,2],λ,κ [3,4],λ,κ [5,8],λ,κ}〉 obtained by throwing away the first element _[1,1],λ,κ from LeftRefine( _[1,8],λ,κ) is an interesting-partition covering [2, 8].

We now give details of D I , ∊ Y. Initially, it is the interesting-partition 〈C_I,λ,κ 〉 covering the whole interval I = [ℓ_I, r_I]. Throughout the execution, we maintain the following invariant:

(**) D I , ∊ Y is an interesting-partition covering some [p, r_I] ⊆ I.

When D I , ∊ Y = 〈 C J 1 , λ , κ , … , C J m , λ , κ 〉 is covering [p, r_I], it only guarantees good estimates of f_a([t, r_I]) for t ∈ [p, r_I], and this estimate is obtained by f ^ a ( [ t , r I ] ) = v ( alive ( C J h , λ , κ ) ≥ t a ) + ∑ h + 1 ≤ i ≤ m v ( alive ( C J i , λ , κ ) a )(or equivalently, f ^ a ( [ t , r I ] ) = v ( alive ( Q J h , λ a ) ≥ t ) + ∑ h + 1 ≤ i ≤ m v ( alive ( Q J i , λ a ) ), where J_h is the interval in {J₁, J₂, …, J_m} that covers t. When an item (a, u) with u ∈ [p, r_I] arrives, we find the unique _Ji,λ,κ in D I , ∊ Y where u ∈ J_i, update it by calling _Ji,λ,κ. Process((a, u)). Note that this update has no effect on the other _J,λ,κ in D I , ∊ Y.

During execution, we also keep track of the largest timestamp p_max ∈ I such that the estimate f̂_*(p_max,r_I]) given by _I,∊ is greater than (1 + ∊)Y (which implies f_*([p_max,r_I]) > Y because of Equation (7)). As soon as p_max falls in the interval covered by D I , ∊ Y, we use the following procedure to trim D I , ∊ Y to cover the smaller interval [p_max + 1, r_I].

Suppose that L = 〈 _J1,λ,κ, …, _Ji,λ,κ) is an interesting-partition covering [p, r_I], and t ∈ [p, r_I]. Trim(L, t) constructs an interesting-partition covering [t + 1, r_I] recursively as follows.

For example, Figure 4 shows that when D I , ∊ Y = 〈 C [ 2 , 2 ] , λ , κ , C [ 3 , 4 ] , λ , κ , C [ 5 , 8 ] , λ , κ 〉, Trim ( D I , ∊ Y , 3 ) return 〈 _[4,4],λ,κ, _[5,8],λ,κ 〉. Based on Fact 3, it can be verified inductively that after D I , ∊ Y ← Trim ( D I , ∊ Y , p max ), the new D I , ∊ Y is an interesting-partition covering [p_max + 1, r_I]; Invariant (**) is preserved. In the rest of this section, we analyze the size of D I , ∊ Y and the accuracy of its estimates.

Let All be the set of all _J,λ,κ's that ever exist, i.e., if _J,λ,κ ∈ All, then either (i) it is currently in D I , ∊ Y, or (ii) it has been in D I , ∊ Y some time earlier in the execution, but is thrown away during some trimming of D I , ∊ Y. For any p ∈ I, define ALL ≥ p = { C J , λ , κ ∣ C J , λ , κ ∈ ALL , and J covers or is to the right of p }

Let v_add( _J,λ,κ) be the total value added to the nodes of _J,λ,κ during its lifespan. We now derive an upper bound on Σ _{J,λ,κ ∈ All≥p} v_add( _J,λ,κ), which is crucial for getting a tight error bound on the accuracy of D I , ∊ Y's estimates.

Recall that initially D I , ∊ Y = 〈 C I , λ , κ 〉 and thus _I,λ,κ ∈ All. For any other _J,λ,κ ∈ All, _J,λ,κ must be a child of some _H,λ,κ ∈ All (i.e., _J,λ,κ is obtained from Split( _H,λ,κ))- Given _J,λ,κ and _H,λ,κ, we say that _J,λ,κ is a descendant of _H,λ,κ, and _H,λ,κ is an ancestor of _J,λ,κ, if either (i) _J,λ,κ is a child of _H,λ,κ, or (ii) it is a child of some of _H,λ,κ's descendants. Note that the original _I,λ,κ is an ancestor of every _J,λ,κ ∈ All, and in general, any _H,λ,κ ∈ All is an ancestor of every _J,λ,κ ∈ All with J ⊂ H. We have the following lemma. (Note that we are abusing the notation here and regard D I , ∊ Y as a set.)

Suppose that D I , ∊ Y = 〈 C J 1 , λ , κ , … , C J m , λ , κ 〉 is covering [p, r_I]. Let anc ( D I , ∊ Y ) = anc ( 〈 C J 1 , λ , κ , … , C J m , λ , κ 〉 ) be the set { C H , λ , κ ∣ C H , λ , κ is an ancestor of some C J i , λ , κ ∈ D I , ∊ Y }. Then,

ALL ≥ p ⊆ D I , ∊ Y ∪ anc ( D I , ∊ Y ),

Therefore, Σ{v_add( _J,λ,κ) | _J,λ,κ ∈ All_≥p} ≤ 2(1 + ∊)Y logW.

Suppose that D I , ∊ Y is covering [p, r_I]. For any item a and any t ∈ [p, r_I], the estimate f̂_a([t, r_I]) of f_a([t, r_I]) obtained by D I , ∊ Y satisfies |f̂_a([t, r_I]) − f_a([t, r_I])| ≤ ∊Y. Furthermore, D I , ∊ Y uses O ( 1 ∊ ( log W ) 2 ) space.