- freely available
- re-usable
Algorithms 2011, 4(4), 285-306; doi:10.3390/a4040285
Published: 31 October 2011
Abstract
: We propose a framework for the exact probabilistic analysis of window-based pattern matching algorithms, such as Boyer–Moore, Horspool, Backward DAWG Matching, Backward Oracle Matching, and more. In particular, we develop an algorithm that efficiently computes the distribution of a pattern matching algorithm's running time cost (such as the number of text character accesses) for any given pattern in a random text model. Text models range from simple uniform models to higher-order Markov models or hidden Markov models (HMMs). Furthermore, we provide an algorithm to compute the exact distribution of differences in running time cost of two pattern matching algorithms. Methodologically, we use extensions of finite automata which we call deterministic arithmetic automata (DAAs) and probabilistic arithmetic automata (PAAs) [1]. Given an algorithm, a pattern, and a text model, a PAA is constructed from which the sought distributions can be derived using dynamic programming. To our knowledge, this is the first time that substring- or suffix-based pattern matching algorithms are analyzed exactly by computing the whole distribution of running time cost. Experimentally, we compare Horspool's algorithm, Backward DAWG Matching, and Backward Oracle Matching on prototypical patterns of short length and provide statistics on the size of minimal DAAs for these computations.1. Introduction
The basic pattern matching problem is to find all occurrences of a pattern string in a (long) text string, with few character accesses, where a character access is the act of retrieving one character of the input string from memory. For many pattern matching algorithms, this is equivalent to speaking of character comparisons, as every accessed character is immediately compared to a character in the pattern. However, for some algorithms (e.g., the Knuth–Morris–Pratt algorithm [2]), each character access triggers a table lookup rather than a comparison. Thus, we discuss character accesses rather than character comparisons in the remainder of this article.
Let n be the text length and m be the pattern length. The well-known Knuth–Morris–Pratt algorithm [2] reads each text character exactly once from left to right and hence needs exactly n character accesses for any text of length n, after preprocessing the pattern in Θ(m) time. In contrast, the Boyer–Moore [3], Horspool [4], Sunday [5], Backward DAWG Matching (BDM, [6]) and Backward Oracle Matching (BOM, [7]) algorithms move a length-m search window across the text and first compare its last character to the last character of the pattern. This often allows to move the search window by more than one position (at best, by m positions if the last window character does not occur in the pattern at all), for a best case of n/m, but a worst case of mn character accesses. The worst case can often be improved to Θ(m + n), but this makes the code more complicated and seldom provides a speed-up in practice. For practical pattern matching applications, the most important algorithms are Horspool, BDM (often implemented as Backward Nondeterministic DAWG Matching, BNDM, via a non-deterministic automaton that is simulated in a bit-parallel fashion), and BOM, depending on alphabet size, text length and pattern length; see [8] for an experimental map.
A question that has apparently so far not been investigated is about the exact probability distribution of the number of required character accesses ${\mathit{\text{X}}}_{n}^{p}$ when matching a given pattern p against a random text of finite length n (non-asymptotic case), even though related questions have been answered in the literature. For example, [9,10] analyze the expected value of ${\mathit{\text{X}}}_{n}^{p}$ for the Horspool algorithm. In [11] it is further shown that for the Horspool algorithm, ${\mathit{\text{X}}}_{n}^{p}$ is asymptotically normally distributed for random texts with independent and identically distributed (i.i.d.) characters, and [12] extends this result to Markovian text models. In [13], a method to compute mean and variance of these distributions is given.
In contrast to these results that are special to the Horspool algorithm, we use a general framework called probabilistic arithmetic automata (PAAs), introduced at CPM'08 [1], to compute the exact distribution of ${\mathit{\text{X}}}_{n}^{p}$ for any window-based pattern matching algorithm. In [1], PAAs were introduced in order to compute the distribution of occurrence counts of patterns, a purpose for which multiple other researchers have also proposed to combine finite automata with probabilistic text models [14-17]. Especially the early approach of Nicodéme et al. [14] has shown how to derive generating functions and perform asymptotic analysis of occurrence distributions.
Here, we show that a similar idea can be applied to the analysis of pattern matching algorithms by constructing an automaton that encodes the behavior of such an algorithm and then combining it with a text model. The PAA framework allows doing this in a natural way, which further highlights its utility The random text model can be quite general, from simple i.i.d. uniform models to high-order Markov models or HMMs. The approach is applied to the following pattern matching algorithms in the non-asymptotic regime (short patterns, medium-length texts): Horspool, B(N)DM, BOM. We do not treat BDM and BNDM separately as, in terms of text character accesses, they are indistinguishable (see Section 2.2).
This paper is organized as follows. In the next section, we give a brief review of the Horspool, B(N)DM and BOM algorithms. In Section 3, we define deterministic arithmetic automata (DAAs). In Section 4, we present a simple general DAA construction for the analysis of window-based pattern matching algorithms. We also show that the state space of the DAA can be considerably reduced by adapting DFA minimization to DAAs. In Section 5, we summarize the PAA framework with its generic algorithms, define finite-memory text models and connect DAAs to PAAs. Given a pattern p, an algorithm, and a random text model, this framework allows constructing a PAA that mimics the algorithms' behavior. By applying dynamic programming to this PAA we obtain an algorithm to compute the distribution of ${\mathit{\text{X}}}_{n}^{p}$ for any finite text length n. Section 6 introduces difference DAAs by a product construction that allows to compare two algorithms on a given pattern. Results on the comparison of several algorithms for example patterns can be found in Section 7. There, we also provide statistics on automata sizes for different algorithms and pattern lengths. Section 8 contains a concluding discussion.
An extended abstract of this work has been presented at LATA'10 [18] with two alternative DAA constructions. In contrast to that version, the DAA construction in the present paper can be seen as a combination of both of those, and is much simpler. Additionally, the DAA minimization introduced in the present paper allows the analysis of much longer patterns in practice. While [18] was focused on Horspool's and Sunday's algorithms, here, we give a general construction scheme applicable to any window-based pattern matching algorithm and discuss the most relevant algorithms, namely Horspool, BOM, and B(N)DM, as examples.
Notation
Throughout this paper, Σ denotes a finite alphabet, p ∈ Σ* is an arbitrary but fixed pattern, and s ∈ Σ* is the text to be searched for p. Furthermore, m ≔ |p| and n ≔ |s|. Indexing generally starts at zero, that is, s = s[0] … s[|s| − 1] for all s ∈ Σ*. The prefix, suffix, and substring of a string s are written s[‥i] ≔ s[0] … s[i], s[i‥] ≔ s[i] … s[|s| − 1], and s[i … j] ≔ s[i] … s[j], respectively. By p⃖, we denote the reverse pattern p[m − 1] … p[0]. For a random variable X, its distribution (law) is denoted by ℒ(X). Iverson brackets are written ⟦·⟧, i.e., ⟦A⟧ = 1 if the statement A is true and ⟦A⟧ = 0 otherwise.
2. Algorithms
In the following, we summarize the Horspool, B(N)DM and BOM algorithms; algorithmic details can be found in [8].
We do not discuss the Knuth–Morris–Pratt algorithm because its number of text character accesses is constant: Each character of the text is looked at exactly once. Therefore, $\mathcal{L}({\mathit{\text{X}}}_{n}^{p})$ is the Dirac distribution on n, i.e., $\mathbb{P}({\mathit{\text{X}}}_{n}^{p}=n)=1$.
We also do not discuss the Boyer–Moore algorithm, since it is never the best one in practice because of its complicated code to achieve optimal asymptotic running time. In contrast to our earlier paper [18], we also skip the Sunday algorithm, as it is almost always inferior to Horspool's. Instead, we focus on those algorithms that are fastest in practice according to [8].
The Horspool, B(N)DM and BOM algorithms have the following properties in common: They maintain a search window w of length m = |p| that initially starts at position 0 in the text s, such that its rightmost character is at position t = m − 1. The right window position t grows in the course of the algorithm; we always have w = s[(t − m + 1) … t]. The two properties of an algorithm that influence our analysis are the following: For a pattern p ∈ Σ^{m}, each window w ∈ Σ^{m} determines
its cost ξ^{p}(w), e.g., the number of text character accesses required to analyze this window,
its shift shift^{p}(w), which is the number of characters the window is advanced after it has been examined.
2.1. Horspool
First, the rightmost characters of window and pattern are compared; that means, a ≔ w[m − 1] = s[t] is compared with p[m − 1]. If they match, the remaining m − 1 characters are compared until either the first mismatch is found or an entire match has been verified. This comparison can happen right-to-left, left-to-right, or in an arbitrary order that may depend on p. In our analysis, we focus on the right-to-left case for concreteness, but the modifications for the other cases are straightforward. Therefore, the cost of window w is
In any case, the rightmost window character a is used to determine how far the window can be shifted for the next iteration. The shift function ensures that no match can be missed by moving the window such that a becomes aligned to the rightmost a in p (not considering the last position). If a does not occur in p (or only at the last position), it is safe to shift by m positions. Formally, we define
For concreteness, we state Horspool's algorithm and how we count text character accesses as pseudocode in Algorithm 1. Note that after a shift, even when we know that a now matches its corresponding pattern character, the corresponding position is compared again and counts as a text access. Otherwise the additional bookkeeping would make the algorithm more complicated; this is not worth the effort in practice. The lookup in the shift-table does not count as an additional access, since we can remember shift[a] as soon as the last window character has been read.
The main advantage of the Horspool algorithm is its simplicity. Especially, a window's shift value depends only on its last character, and its cost is easily computed from the number of consecutive matching characters at its right end. The Horspool algorithm does not require any advanced data structure and can be implemented in a few lines of code.
Algorithm 1 Horspool-with-Cost | |
Input: text s ∈ Σ*, pattern p ∈ Σ^{m} | |
Output: pair (number occ of occurrences of p in s, number cost of accesses to s) | |
1: | pre-compute table shift[a] for all a ∈ Σ |
2: | (occ, cost) ← (0, 0) |
3: | t ← m − 1 |
4: | while t < |s| do |
5: | i ← 0 |
6: | while i < m do |
7: | cost ← cost + 1 |
8: | if s[t − i] = p[(m − 1) − i] then break |
9: | i ← i + 1 |
10: | if i = m then occ ← occ + 1 |
11: | t ← t + shift[s[t]] |
12: | return (occ, cost) |
2.2. Backward (Nondeterministic) DAWG Matching, B(N)DM
The main idea of the BDM algorithm is to build a deterministic finite automaton (in this case, a suffix automaton, which is a directed acyclic word graph or DAWG) that recognizes all substrings of the reversed pattern, accepts all suffixes of the reversed pattern (including the empty suffix), and enters a FAIL state if a string has been read that is not a substring of the reversed pattern.
The suffix automaton processes the window right-to-left. As long as the FAIL state has not been reached, we have read a substring of the reversed pattern. If we are in an accepting state, we have even found a suffix of the reversed pattern (i.e., a prefix of p). Whenever this happens before we have read m characters, the last such event marks the next potential window start that may contain a match with p, and hence determines the shift. When we are in an accepting state after reading m characters, we have found a match, but this does not influence the shift.
So, ξ^{p}(w) is the number of characters read when entering FAIL (including the FAIL-inducing character), or m if p = w. Let I^{p}(w) ⊆ {0, …, m − 1} be the set defined by i ∈ I^{p}(w) if and only if the suffix automaton of p⃖ is in an accepting state after reading i characters of w. Then
Note that I^{p}(w) is never empty as the suffix automaton accepts the empty string and, thus, 0 ∈ I^{p}(w) for all windows w.
The advantage of BDM is that it makes long shifts, but its main disadvantage is the necessary construction of the suffix automaton, which is possible in O(m) time via the suffix tree of the reversed pattern, but too expensive in practice to compete with other algorithms unless the search text is extremely long.
Constructing a nondeterministic finite automaton (NFA) instead of the deterministic suffix automaton is much simpler. However, processing a text character then does not take constant, but O(m) time. However, the NFA can be efficiently simulated with bit-parallel operations such that processing a text character takes O(m/W) time, where W is the machine word size. For many patterns in practice, this is as good as O(1). The resulting algorithm is then called BNDM.
From the “text character accesses” analysis point of view, BDM and BNDM are equivalent, as they have the same shift and cost functions.
2.3. Backward Oracle Matching, BOM
BOM is similar to B(N)DM, but the suffix automaton of the reversed pattern is replaced by a simpler deterministic automaton, the factor oracle [8]. The factor oracle of a string x (which corresponds to the reversed pattern p⃖) of length m has the following properties.
If y is a factor (substring) of x, then there exists a path spelling y from the start state to some state which is not the FAIL state; we say that y is recognized.
The only string of length m recognized is x.
It has the minimal number of states (m + 1) necessary for recognizing x (omitting the FAIL state).
It has between m and 2m − 1 transitions (omitting those into the FAIL state).
It may recognize more strings than the substrings of x (although in practice not many more), but is easier to construct. It still guarantees that, once the FAIL state is reached, the sequence of read characters is not a substring of x. We refer to [8] for the construction details and further properties of the oracle; an example is shown in Figure 1.
The cost function ξ^{p}(w) is the number of characters read when entering FAIL (including the FAIL-inducing character), or m if p = w. The shift function is based on the principle that the window can be safely shifted beyond the FAILed substring; so shift^{p}(w) is defined as m minus the number of successfully read characters in w if w ≠ p, and shift^{p}(p) ≔ 1 (although this special case for w = p can be improved by examining the pattern).
By construction, BOM never gives longer shifts than B(N)DM. The main advantage of BOM over BDM is reduced space usage and preprocessing time; the factor oracle only has m + 1 states and can be constructed faster than a suffix automaton.
3. Deterministic Arithmetic Automata
In this section, we introduce deterministic arithmetic automata (DAAs). They extend ordinary deterministic finite automata (DFAs) by performing a computation while one moves from state to state. Even though DAAs can be shown to be formally equivalent to families of DFAs on an appropriately defined larger state space, they are a useful concept before introducing probabilistic arithmetic automata (PAAs) and allow us to construct PAAs for the analysis of pattern matching algorithms in a simpler way. By using the PAA framework, we emphasize the connection between the problems discussed in the present article and those solved before using the same formalism: Other applications in biological sequence analysis include the exact computation of clump size distributions and p-values of sequence motifs [19], and the determination of seed sensitivity for pairwise sequence alignment algorithms based on filtering [20].
Definition 1 (Deterministic Arithmetic Automaton, DAA)
A deterministic arithmetic automaton is a tuple
Informally, a DAA starts with the state-value pair (q_{0}, v_{0}) and reads a sequence of symbols from Σ. Being in state q with value v, upon reading σ ∈ Σ, the DAA performs a state transition to q′ ≔ δ(q, σ) and updates the value to v′ ≔ θ_{q′}(v, η_{q′}) using the operation and emission of the new state q′.
Further, we define the associated joint transition function
As usual, we extend the definition of δ̂ inductively from Σ to Σ* in its second argument by δ̂((q, v), ε) ≔ (q, v) for the empty string ε and δ̂((q, v), xσ) ≔ δ(δ̂((q, v), x),σ) for all x ∈ Σ* and σ ∈ Σ.
When δ̂((q_{0}, v_{0}), s) = (q, v) for some q ∈ and s ∈ Σ*, we say that computes value v for input s and define value_{ }(s) ≔ v.
For each state q, the emission η_{q} is fixed and could be dropped from the definition of DAAs. In fact, one could also dispense with values and operations entirely and define a DFA over state space × , performing the same operations as a DAA. However, we intentionally include values, operations, and emissions to emphasize the connection to PAAs (which are defined in Section 5).
As a simple example for a DAA, take a standard DFA ( , q_{0}, Σ, δ, F) with F ⊂ being a set of final (or accepting) states. To obtain a DAA that counts how many times the DFA visits an accepting state when reading s ∈ Σ*, let ℰ ≔ {0, 1} and define η_{q} ≔ 1 if q ∈ F, and η_{q} ≔ 0 otherwise. Further define = ℕ with v_{0} ≔ 0, and let the operation in each state be the usual addition: θ_{q}(v, e) ≔ v + e for all q. Then value_{ }(s) is the desired count.
4. Constructing DAAs for Pattern Matching Analysis
For a given algorithm and pattern p ∈ Σ^{m} with known shift and cost functions, shift^{p} : Σ^{m} → {1, …, m}, w ↦ shift^{p}(w) and ξ^{p} : Σ^{m} → ℕ, w ↦ ξ^{p}(w), we construct a DAA that upon reading a text s ∈ Σ* computes the total cost, defined as the sum of costs over all examined windows. (Which windows are examined depends of course on the shift values of previously examined windows.) Slightly abusing notation, we write ξ^{p}(s) for the total cost incurred on s.
While different constructions are possible (see also [18]), the construction presented here has the advantage that it is simple to describe and implement and processes only one text character at a time. This property allows the construction of a product DAA that directly compares two algorithms as detailed in Section 6.
Definition 2 (DAA encoding a pattern matching algorithm)
Given a window-based pattern matching algorithm A, a pattern p ∈ Σ^{m}, and the associated shift and cost functions, shift^{p} : Σ^{m} → {1, …, m} and ξ^{p} : Σ^{m} → ℕ, the DAA encoding algorithm A is defined by
≔ Σ^{m} × {0, …,m},
q_{0} ≔ (p, m),
The remaining components are defined as
≔ ℕ,
v_{0} ≔ 0,
ℰ ≔ {1, …, m},
${\eta}_{(w,x)}:=\{\begin{array}{ll}0\hfill & \mathit{\text{if}}\phantom{\rule{0.3em}{0ex}}x>0,\hfill \\ {\xi}^{p}(w)\hfill & \mathit{\text{if}}\phantom{\rule{0.3em}{0ex}}x=0,\hfill \end{array}$
θ_{q} : (v, e) ↦ v + e for all q ∈ (addition),
$\delta :((w,x),\sigma )\mapsto \{\begin{array}{ll}(w\prime \sigma ,x-1)\hfill & \mathit{\text{if}}\phantom{\rule{0.3em}{0ex}}x>0,\hfill \\ (w\prime \sigma ,{\mathit{\text{shift}}}^{p}(w)-1)\hfill & \mathit{\text{if}}\phantom{\rule{0.3em}{0ex}}x=0,\hfill \end{array}$
Figure 2 shows an example of how a DAA for Horspool's algorithm moves from state to state. The value accumulates the cost of examined windows. Therefore, the operation is a simple addition in each state, and the emission of state (w, x) specifies the cost to add. Consequently, the emission is zero if the state does not correspond to an examined window (x > 0), and the emission equals the window cost ξ^{p}(w) if x = 0. The transition function δ specifies how to move from one state to the next when reading the next text character σ ∈ Σ: In any case, the window content is updated by forgetting the first character and appending the read σ. If the end of the current window has not been reached (x > 0), the counter x is decremented. Otherwise, the window's shift value is used to compute the number of characters till the next window aligns.
Theorem 1
Let be a DAA as given by Definition 2. Then, value_{ }(s) = ξ^{p}(s) for all s ∈ Σ*.
Proof
The total cost ξ^{p}(s) can be written as the sum of costs of all processed windows: ξ^{p}(s) = Σ_{i∈Is} ξ^{p}(s[i − m + 1 … i]), where I_{s} is the set of indices giving the processed windows, i.e., I_{s} ⊂ {m − 1, …, |s| − 1} such that
We have to prove that the DAA computes this value for s ∈ Σ*.
Let (w_{i}, x_{i}) be the DAA state active after reading s[‥i]. Observe that the transition function δ ensures that the w_{i}-component of (w_{i}, x_{i}) reflects the rightmost length-m window of s[‥i], which can immediately be verified inductively. Thus, the emission on reading the last character s[i] of s[‥i] with i ≥ m − 1 is, by definition of η(w_{i}, x_{i}), either ξ^{p}(s[i − m + 1 … i]) or zero, depending on the second component of (w_{i}, x_{i}). As the operation is an addition for all states, ${\mathit{\text{value}}}_{\mathcal{D}}(s)={\sum}_{i\in {I}_{s}^{\prime}}{\xi}^{p}(s[i-m+1\dots i])$ for
It remains to show that ${I}_{s}={I}_{s}^{\prime}$. To this end, note that by δ, we have x_{i}_{+1} = x_{i} − 1 if x_{i}_{+1} > 0 and x_{i}_{+1} = shift^{p}(w_{i}) − 1 if x_{i}_{+1} = 0. As q_{0} = (p, m), it follows that $m-1\in {I}_{s}^{\prime}$. Using w_{i} = s[i − m + 1 … i] for i ≥ m − 1, we conclude that whenever x_{i} = 0, it follows that x_{j} = 0 for j = i+shift^{p}(s[i − m + 1 … i]) and that x_{j}_{′} > 0 for i < j′ < j. Hence we obtain that $i\in {I}_{s}^{\prime}$ implies that $i+{\text{shift}}^{p}(s[i-m+1\dots i])\in {I}_{s}^{\prime}$ and $i+k\notin {I}_{s}^{\prime}$ for 0 < k < shift^{p}(s[i − m + 1 … i]), which completes the proof.
DAA Minimization
The size of the constructed DAA's state space is (m + 1)|Σ|^{m} and grows exponentially with the pattern length, making the application for long patterns infeasible in practice. However, depending on the particular circumstances (i.e., algorithm and pattern analyzed), the constructed DAA can often be substantially reduced by state space minimization [21]. For example, for B(N)DM, both cost and shift of an examined window depend only on the longest factor of p that is a suffix of the window. Since there are only O(m^{2}) different factors, it is reasonable that |Σ|^{m} can be replaced by O(m^{2}), for a total state space of size O(m^{3}). Therefore, for each algorithm, a specialized construction may exist that directly constructs the minimal state space whose size may only grow polynomially with m. For the Horspool algorithm, it is known that the state space has a size of only O(m^{2}), as the construction of Tsai [13] can be adapted to construct a DAA according to our definition. However, we have been unable to provide a direct construction of the minimal DAA applicable to all window-based pattern matching algorithms.
Hopcroft's algorithm [21] minimizes a DFA in O(| | log | |) time by iteratively refining a partition of the state set. In the beginning, all states are partitioned into two distinct sets: one containing the accepting states, and the other containing the non-accepting states. This partition is iteratively refined whenever a reason for non-equivalence of two states in the same set is found. Upon termination, the states are partitioned into sets of equivalent states. Refer to [22] for an in-depth explanation of Hopcroft's algorithm.
The algorithm can straightforwardly be adapted to minimize DAAs by choosing the initial state set partition appropriately. In our case, each DAA state is associated with the same operation. The only differences in state's behavior thus stem from different emissions. Therefore, Hopcroft's algorithm can be initialized by the partition induced by the emissions and then continued as usual.
As we exemplify in Section 7, this leads to a considerable reduction of the number of states.
5. Probabilistic Arithmetic Automata
This section introduces finite-memory random text models and explains how to construct a probabilistic arithmetic automaton (PAA) from a (minimized) DAA and a random text model. PAAs were introduced in [1], where they are used to compute pattern occurrence count distributions. Further examples for the utility of PAAs are discussed in [19] and [20].
5.1. Random Text Models
Given an alphabet Σ, a random text is a stochastic process (S_{t})_{t∈ℕ0}, where each S_{t} takes values in Σ. A text model ℙ is a probability measure assigning probabilities to (sets of) strings. It is given by (consistently) specifying the probabilities ℙ(S_{0} … S_{|s|−1} = s) for all s ∈ Σ*. We only consider finite-memory models in this article which are formalized in the following definition.
Definition 3 (Finite-memory text model)
A finite-memory text model is a tuple ( , c_{0}, Σ, φ), where is a finite state space (called context space), c_{0} ∈ a start context, Σ an alphabet, and φ : × Σ × → [0, 1] a transition function with Σ_{σ∈Σ,c′∈ } φ(c, σ, c′) = 1 for all c ∈ . The random variable giving the text model state after t steps is denoted C_{t} with C_{0} :≡ c_{0}. A probability measure is now induced by stipulating
The idea is that the model given by ( , c_{0}, Σ, φ) generates a random text by moving from context to context and emitting a character at each transition, where φ(c, σ, c′) is the probability of moving from context c to context c′ and thereby generating the letter σ.
Note that the probability ℙ(S_{0} … S_{|s|−1} = s) is obtained by marginalization over all context sequences that generate s. This can be efficiently done, using the decomposition of the following lemma.
Lemma 1
Let ( , c_{0}, Σ, φ) be a finite-memory text model. Then,
Proof
We have
Renaming c_{n} to c′ yields the claimed result.
Similar text models are used in [23], where they are called probability transducers. In the following, we refer to a finite-memory text model ( , c_{0}, Σ, φ) simply as text model, as all text models considered in this article are special cases of Definition 3.
For an i.i.d. model, we set = {ε} and φ(ε, σ, ε) = p_{σ} for each σ ∈ Σ, where p_{σ} is the occurrence probability of letter σ (and ε may be interpreted as an empty context). For a Markovian text model of order r, the distribution of the next character depends on the r preceding characters (fewer at the beginning); thus $\mathcal{C}:={\cup}_{i=0}^{r}{\sum}^{i}$. This notion of text models also covers variable order Markov chains as introduced in [24], which can be converted into equivalent models of fixed order. Text models as defined above have the same expressive power as character-emitting HMMs, that means, they allow to construct the same probability distributions.
5.2. Basic PAA Concepts
Probabilistic arithmetic automata (PAAs), as introduced in [1], are a generic concept useful to model probabilistic chains of operations. In this section, we sum up the definition and basic recurrences needed in this article.
Definition 4 (Probabilistic Arithmetic Automaton, PAA)
A probabilistic arithmetic automaton is a tuple = ( , q_{0}, T, , v_{0}, ℰ, μ = (μ_{q})_{q∈ }, θ = (θ_{q})_{q∈ }), where , q_{0}, , v_{0}, ℰ and θ have the same meaning as for a DAA, each μ_{q} is a state-specific probability distribution on the emissions ℰ, and T : × → [0, 1] is a transition function, such that T(q, q′) gives the probability of a transition from state q to state q′, i.e., (T(q, q′))_{q, q′∈ } is a stochastic matrix.
A PAA induces three stochastic processes: (1) the state process (Q_{t})_{t∈ℕ} with values in , (2) the emission process (E_{t})_{t∈ℕ} with values in ℰ, and (3) the value process (V_{t})_{t∈ℕ} with values in such that _{0} :≡ v_{0} and _{t} ≔ θ_{Qt} (V_{t−1}, E_{t}).
We now restate the PAA recurrences from [1] to compute the state-value distribution after t steps. For the sake of a shorter notation, we define f_{t}(q, v) ≔ ℙ(Q_{t} = q, V_{t} = v). Since we are generally only interested in the value distribution, note that it can be obtained by marginalization: ℙ(V_{t} = v) = Σ_{q∈ } f_{t}(q, v).
Lemma 2 (State-value recurrence, [1])
The state-value distribution is given by f_{0}(q, v) = 1 if q = q_{0} and v = v_{0}, and f_{0}(q, v) = 0 otherwise. For t ≥ 0,
The recurrence in Lemma 2 resembles the Forward recurrences known from HMMs.
Note that the range of V_{t} is finite for each t, even when is infinite, as V_{t} is a function of the states and emissions up to time t, and state set and emission set ℰ are finite. We define _{t} ≔ range V_{t} and ϑ_{n} ≔ max_{0≤t≤n} | _{t}|. Clearly ϑ_{n} ≤ (| | · |ℰ|)^{n}. Therefore all actual computations are on finite sets. When analyzing the number of character accesses of a pattern matching algorithm, we have _{t} ⊂ {0, …, m(n − m + 1)}, as at most (n − m + 1) search windows are processed, each causing at most m character accesses. Thus, ϑ_{n} ∈ O(n · m).
5.3. Constructing a PAA from a DAA and a Text Model
We now formally state how to combine a DAA and a text model into a PAA that allows us to compute the distribution of values produced by the DAA when processing a random text.
Definition 5 (PAA induced by DAA and text model)
Let a text model M = ( , c_{0}, Σ, φ) and a DAA $\mathcal{D}=({\mathcal{Q}}^{\mathcal{D}},{q}_{0}^{\mathcal{D}},\sum ,\delta ,\mathcal{V},{v}_{0},\mathcal{E},{({\eta}_{q})}_{q\in {\mathcal{Q}}^{\mathcal{D}}},{({\theta}_{q}^{\mathcal{D}})}_{q\in {\mathcal{Q}}^{\mathcal{D}}})$ over the same alphabet Σ be given. Then, we define the PAA induced by and M by giving
a state space ≔ ^{ } × ,
a start state ${q}_{0}\u2254({q}_{0}^{\mathcal{D}},{c}_{0})$,
transition probabilities
$T((q,c),(q\prime ,c\prime ))\u2254\sum _{\sigma \in \sum :\delta (q,\sigma )=q\prime}\phi (c,\sigma ,c\prime ),$(deterministic) emission probability vectors ${\mu}_{(q,c)}(e)\u2254\u27e6e={\eta}_{q}\u27e7\phantom{\rule{0.3em}{0ex}}\text{for all}\phantom{\rule{0.3em}{0ex}}(q,c)\in \mathcal{Q}$,
operations ${\theta}_{(q,c)}(v,e)\u2254{\theta}_{q}^{\mathcal{D}}(v,e)\phantom{\rule{0.3em}{0ex}}\text{for all}\phantom{\rule{0.3em}{0ex}}(q,c)\in \mathcal{Q}$.
Note that states having zero probability of being reached from q_{0} may be omitted from and T without changing the PAA's state, emission or value process. The next lemma states that the PAA given by Definition 3 indeed reflects the probabilistic behavior of the input DAA acting on a random text generated by the text model. Furthermore, it gives the runtime required to compute the distribution of DAA values via dynamic programming.
Lemma 3 (Properties of PAA induced by DAA and text model)
Let a text model M = ( , c_{0}, Σ, φ) and a DAA $\mathcal{D}=({\mathcal{Q}}^{\mathcal{D}},{q}_{0}^{\mathcal{D}},\sum ,\delta ,\mathcal{V},{v}_{0},\mathcal{E},{({\eta}_{q})}_{q\in {\mathcal{Q}}^{\mathcal{D}}},{({\theta}_{q}^{\mathcal{D}})}_{q\in {\mathcal{Q}}^{\mathcal{D}}})$be given and let = ( , q_{0}, T, , v_{0}, ℰ, μ = (μ_{q})_{q∈ }, θ = (θ_{q})_{q∈ }) be the PAA given by Definition 5. Then,
ℒ(V_{t}) = ℒ(value_{ }(S_{0} … S_{t−1})) for all t ∈ ℕ_{0}, where S is a random text according to the text model M,
the value distribution ℒ(V_{n}) can be computed with O(n · | ^{ }| · | |^{2} · |Σ| · ϑ_{n}) operations using O(| ^{ }| · | | · ϑ_{n}) space, and
if for all c ∈ and σ ∈ Σ, there exists at most one c′ ∈ such that φ(c, σ, c′) > 0, then the runtime is bounded by O(n · | ^{ }| · | | · |Σ| · ϑ_{n}).
Proof
As in Section 5.2, we define f_{t}(q, v) ≔ ℙ(Q_{t} = q, V_{t} = v). To prove 1, we show that
In the above derivation, step (4)→(5) follows from (1). Step (5)→(6) follows from the definitions of θ_{q} and μ_{q}. Step (6)→(7) uses the definitions of T and in Lemma 3. Step (7)→(8) uses the induction assumption. Step (8)→(9) uses Lemma 1. The final step (9)→(10) follows by combining the four Iverson brackets summed over q′^{ } and (v′, e) into a single Iverson bracket.
To compute the table f_{n} containing f_{n}(q, v) for all q ∈ and v ∈ , we start with f_{0} and perform n update steps. The runtime bounds given in 2. and 3. can be verified by considering a “push” algorithm: When computing f_{t}_{+1}, we initialize the table with zeros and iterate over all q ∈ , v ∈ and q′ ∈ {q″ ∈ : T(q, q″) > 0}; for each combination of q, v, and q′ with T(q, q′) > 0, we add f_{t}(q, v) · T(q, q′) to f_{t+1}(q′, θ_{q′}(v, η_{q′})).
As a direct consequence of the above lemma and of the DAA construction from Section 4, we arrive at our main theorem.
Theorem 2
Let a finite-memory text model ( , c_{0}, Σ, φ), a window-based pattern matching algorithm A, a pattern p with |p| = m, and the functions shift^{A,p} and ξ^{A,p} be given. Then, the cost distribution $\mathcal{L}({X}_{n}^{A,p})$ can be computed using O(n^{2} · m · | ^{ }| · | |^{2} · |Σ|) time and O(| ^{ }| · | | · n · m) space. If for all c ∈ and σ ∈ Σ, there exists at most one c′ ∈ such that φ(c, σ, c′) > 0, a factor of | | can be dropped from the runtime bounds.
Using optimal algorithm-dependent DAA constructions schemes (e.g., the O(m^{2}) construction for the Horspool algorithm by Tsai [13]) allows to replace | ^{ }| by a polynomial in m, instead of O(m|Σ|^{m}).
6. Comparing Algorithms with Difference DAAs
Computing the cost distribution for two algorithms allows us to compare their performance characteristics. One natural question, however, cannot be answered by comparing these two (one-dimensional) distributions: What is the probability that algorithm A needs more text accesses than algorithm B to scan the same random text? The answer will depend on the correlation of algorithm performances: Do the same instances lead to long runtimes for both algorithms or are there instances that are easy for one algorithm but difficult for the other? This section answers these questions by constructing a PAA to compute the distribution of cost differences of two algorithms. That means, we calculate the probability that algorithm A needs v text accesses more than algorithm B for all v ∈ ℤ.
We start by giving a general construction of a DAA that computes the difference of the sum of emission of two given DAAs.
Definition 6 (Difference DAA)
Let a finite alphabet Σ and two DAAs ${\mathcal{D}}^{1}=({\mathcal{Q}}^{1},{q}_{0}^{1},\sum ,{\delta}^{1},{\mathcal{V}}^{1},{v}_{0}^{1},{\mathcal{E}}^{1},{({\eta}_{q}^{1})}_{q\in {\mathcal{Q}}^{1}},{({\theta}_{q}^{1})}_{q\in {\mathcal{Q}}^{1}})$ and ${\mathcal{D}}^{2}=({\mathcal{Q}}^{2},{q}_{0}^{2},\sum ,{\delta}^{2},{\mathcal{V}}^{2},{v}_{0}^{2},{\mathcal{E}}^{2},{({\eta}_{q}^{2})}_{q\in {\mathcal{Q}}^{2}},{({\theta}_{q}^{2})}_{q\in {\mathcal{Q}}^{2}})$ be given with ${\mathcal{V}}^{1}={\mathcal{V}}^{2}=\mathbb{N},{v}_{0}^{1}={v}_{0}^{2}=0,{\mathcal{E}}^{1},{\mathcal{E}}^{2}\subset \mathbb{N}$, and all operations are additions of previous value and current emission. The difference DAA of ^{1} and ^{2} is defined as
≔ ^{1} × ^{2} and ${q}_{0}\u2254({q}_{0}^{1},{q}_{0}^{2})$,
≔ ℤ and v_{0} ≔ 0,
ℰ ≔ ℰ^{1} × ℰ^{2} and ${\eta}_{({q}^{1},{q}^{2})}\u2254\left({\eta}_{{q}^{1}}^{1},{\eta}_{{q}^{2}}^{2}\right)$,
δ : ((q^{1}, q^{2}), σ) ↦ (δ^{1}(q^{1}, σ), δ^{2}(q^{2}, σ)),
θ_{q} : (v, (e^{1}, e^{2})) ↦ v + e^{1} − e^{2}.
Lemma 4
Let ^{1} and ^{2} be DAAs meeting the criteria given in Definition 6 and ≔ DiffDAA( ^{1}, ^{2}). Then, value_{ }(s) = value_{ 1}(s) − value_{ 2}(s) for all s ∈ Σ*.
Proof
Follows directly from Definition 6.
Lemma 4 can now be applied to the DAAs constructed for the analysis of two algorithms as described in Section 4. Since the above construction builds the product of both state spaces, it is advisable to minimize both DAAs before generating the product. Furthermore, in an implementation, only reachable states of the product automaton need to be constructed. Before being used to build a PAA (by applying Lemma 3), the product DAA should again be minimized.
As discussed in Section 5.2, at most m(n − m + 1) character accesses can result from scanning a text of length n for a pattern of length m. Thus, the difference of costs for two algorithms lies in the range {−m(n − m + 1), …, m(n − m + 1)} and, hence, ϑ_{n} ∈ O(n · m).
7. Case Studies
In Section 2, we considered three practically relevant algorithms, namely Horspool's algorithm, backward oracle matching (BOM), and backward (non)-deterministic DAWG matching (B(N)DM). Now, we compare the distributions of running time costs of these algorithms for several patterns over the DNA alphabet { A, C, G, T}. Figure 3 shows these distributions for the patterns ATATAT and ACGTAC for text lengths 100 and 500 under a second order Markovian text model estimated from the human genome. For text length 500, the distributions for Horspool and B(N)DM resemble the shape of normal distributions. In fact, for Horspool's algorithm it has been proven that the distribution is asymptotically normal [12]. For smaller text lengths (e.g., 100, as shown in left column of Figure 3), the distributions are less smooth than for longer texts.
It is remarkable that for BOM we find zero probabilities with a fixed period. The period equals m + 1 which is 7 in the shown examples. This behavior is caused by the factor-based nature of BOM; when a suffix of the search window has been recognized as not being a factor (substring) of the pattern, the window is just moved far enough to exclude this substring, creating the relation shift^{p}(w) = m − ξ^{p}(w) + 1 between cost and shift of a window w. As the following lemma shows, this property is a sufficient condition for the observed zero probabilities.
Lemma 5
Let a window-based pattern matching algorithm A, a pattern p with |p| = m, and the functions shift^{A,p} and ξ^{A,p} be given such that shift^{A,p}(w) = m − ξ^{A,p}(w) + 1 for all w ∈ Σ^{m}. Then,
Proof
Let w_{i}, …, w_{k} be the sequence of windows examined by algorithm A when processing the text s ∈ Σ^{n}. In the beginning, the rightmost position of the current window is at position m − 1 and is moved by shift^{A,p}(w_{i}) after processing window w_{i}. After processing all windows it is beyond the end of the text. Formally,
By using the assumption that shift^{A,p}(w) = m − ξ^{A,p}(w) + 1, we obtain
The probability that one pattern matching algorithm is faster than another depends on the pattern. Using the technique introduced in Section 6, we can quantify the strength of this effect. Figure 4 shows distributions of cost differences for different patterns and algorithms. That means, the probability that the first algorithm is faster is represented by the area under the curve left of zero. For the pattern ACCCCC and a random text of length 100, for example, there is a 53.9% probability that Horspool's algorithm needs fewer character accesses than B(N)DM (for the same second order Markovian model as used before), while for ACGTAC, the probability is only 0.0016%.
Worth noting and perhaps surprising is the fact that there is a non-zero probability of BOM being faster than B(N)DM although, shift^{B(N)DM,p}(w) ≥ shift^{BOM,p}(w) for all window contents w. The explanation, of course, is that a shorter (say, first) shift for BOM leads to a different window content than for B(N)DM for the second window, which may have a larger shift value. This effect depends on the pattern: For the pattern ACCCCC, there is a 52.4% probability that BOM is at least as fast as B(N)DM (in terms of character accesses), while it is 4.9% for ACGTAC, again on texts of length 100. An example text where BOM is faster than B(N)DM while searching for the pattern ACCCCC is shown in Figure 5. Both algorithms read two characters of the first window but prescribe different shifts. The first window ends on TC. BOM recognizes that TC is not a substring of the pattern an shifts the window by five positions, just far enough to exclude this substring. In contrast, B(N)DM determines that neither C nor TC are prefixes of the pattern and shifts the window by six positions. It turns out, however, that a shorter shift of five positions was beneficial in this case as BOM can process the second window with one character access, while B(N)DM uses two character accesses.
To assess the effect of DAA minimization before constructing PAAs, we constructed minimized DAAs for all 21840 patterns of lengths 2 to 7 over Σ = { A, C, G, T}. The minimum, average, and maximum state counts are shown in Table 1. For length 6, Figure 6 contains a detailed histogram. These statistics show that construction and minimization as given in this article lead to smaller automata (and thus better runtimes) than the constructions given in the conference version of this article [18]. It may be conjectured that the worst-case size of the minimal state space grows only polynomially with m for all of these algorithms, as has been previously proven for the Horspool algorithm [13].
The algorithms were implemented in JAVA and are available as part of the MoSDi software package available at http://mosdi.googlecode.com. They were run on an Intel Core 2 Dual CPU at 2.1 GHz. Computing the distributions shown in Figure 3 took 0.5 to 1.3 seconds for each distribution. Distributions of differences as in Figure 4 were computed in 56 to 97 seconds.
8. Discussion
Using PAAs, we have shown how the exact distribution of the number of character accesses for window-based pattern matching algorithms can be computed algorithmically. The framework admits general finite-memory text models, including i.i.d. models, Markov models of arbitrary order, and character-emitting hidden Markov models. The given construction results in an asymptotic runtime of O(n^{2} · m · | ^{ }| · | |^{2} · |Σ|). The number of DAA states | ^{ }| can be as large as O(m · Σ^{m}), but we conjecture that for each reasonable algorithm, the necessary minimal state set ${\mathcal{Q}}_{min}^{\mathcal{D}}$ grows only polynomially with m. In particular, we conjecture O(m^{3}) sizes for B(N)DM and BOM; this is consistent with the numbers in Table 1. For Horspool, a specialized O(m^{2}) construction is known [13]. Otherwise, in practice, the DAA size can be reduced by DAA minimization, but it remains open if there exists an algorithm to construct the minimal automaton directly in general, i.e., using only $O(|{\mathcal{Q}}_{min}^{\mathcal{D}}|)$ time. A proof that this is the case for a broad class of pattern matching algorithms would be an important insight into the nature of these algorithms and therefore certainly warrants further research.
The behavior of BOM deserves further attention: first, periodic zero probabilities are found in its distribution of text character accesses; and second, it may (unexpectedly) need fewer text accesses than B(N)DM on some patterns, although BOM's shift values are never better than B(N)DM's.
We focused on algorithms for single patterns, but the presented techniques also apply to algorithms to search for multiple patterns like the Wu-Manber algorithm [25] or “set backward oracle matching” and “multiple BNDM”, as described in [8]. A comparison of the resulting distributions could yield new insights into these algorithms as well.
Other metrics than text character accesses might be of interest and could be easily substituted; for example, just counting the number of windows by defining ξ^{p}(w) = 1 for all w ∈ Σ^{m}.
The given constructions allow us to analyze an algorithm's performance for each pattern individually. While this is desirable for detailed analysis, the cost distribution resulting from randomly choosing text and pattern would also be of interest.
The results of this paper were obtained while Tobias Marschall was a PhD student with Sven Rahmann and TU Dortmund. The thesis is available at http://hdl.handle.net/2003/27760.
Table 1. Comparison of DAA sizes for all patterns of length m over Σ = { A, C, G, T}. |
m | States unminimized | States minimized (min./avg./max.) | ||
---|---|---|---|---|
|Σ|^{m} · (m + 1) | Horspool | BOM | B(N)DM | |
2 | 48 | 4 / 4.8 / 5 | 4 / 4.0 / 4 | 4 / 4.8 / 5 |
3 | 256 | 7 / 8.3 / 9 | 7 / 8.3 / 9 | 7 / 9.6 / 10 |
4 | 1280 | 11 / 14.3 / 15 | 11 / 15.6 / 18 | 11 / 17.0 / 19 |
5 | 6144 | 16 / 23.6 / 25 | 16 / 26.5 / 30 | 16 / 27.9 / 31 |
6 | 28672 | 22 / 37.0 / 39 | 22 / 41.8 / 47 | 22 / 42.8 / 48 |
7 | 131072 | 29 / 55.2 / 58 | 29 / 62.4 / 70 | 29 / 62.6 / 70 |
Most work was carried out while both authors were affiliated with TU Dortmund.
References
- Marschall, T.; Rahmann, S. Probabilistic Arithmetic Automata and Their Application to Pattern Matching Statistics. Proceedings of the 19th Annual Symposium on Combinatorial Pattern Matching, CPM '08, Pisa, Italy, 18–20 June 2008; Ferragina, P., Landau, G.M., Eds.; Springer: Berlin, Germany, 2008. Volume 5029; pp. 95–106.
- Knuth, D.E.; Morris, J.; Pratt, V.R. Fast pattern matching in strings. SIAM J. Comput. 1977, 6, 323–350.
- Boyer, R.S.; Moore, J.S. A fast string searching algorithm. Commun. ACM 1977, 20, 762–772.
- Horspool, R.N. Practical fast searching in strings. Softw.-Pract. Exp. 1980, 10, 501–506.
- Sunday, D.M. A very fast substring search algorithm. Commun. ACM 1990, 33, 132–142.
- Crochemore, M.; Czumaj, A.; Gasieniec, L.; Jarominek, S.; Lecroq, T.; Plandowski, W.; Rytter, W. Speeding up two string-matching algorithms. Algorithmica 1994, 12, 247–267.
- Allauzen, C.; Crochemore, M.; Raffinot, M. Efficient Experimental String Matching by Weak Factor Recognition. Proceedings of the 12th Annual Symposium on Combinatorial Pattern Matching, CPM '01, Jerusalem, Israel, 1–4 July 2001; Goos, G., Hartmanis, J., van Leeuwen, J., Eds.;. Volume 2089pp. 51–72.
- Navarro, G.; Raffinot, M. Flexible Pattern Matching in Strings; Cambridge University Press: Cambridge, UK, 2002.
- Baeza-Yates, R.A.; Gonnet, G.H.; Régnier, M. Analysis of Boyer-Moore-Type String Searching Algorithms. Proceedings of the 1st Annual ACM- SIAM Symposium on Discrete Algorithms, SODA '90, San Francisco, CA, USA, 22–24 January 1990; pp. 328–343.
- Baeza-Yates, R.A.; Régnier, M. Average running time of the boyer-moore-horspool algorithm. Theor. Comput. Sci. 1992, 92, 19–31.
- Mahmoud, H.M.; Smythe, R.T.; Régnier, M. Analysis of Boyer-Moore-Horspool string-matching heuristic. Random Struct. Algorithms 1997, 10, 169–186.
- Smythe, R.T. The Boyer-Moore-Horspool heuristic with Markovian input. Random Struct. Algorithms 2001, 18, 153–163.
- Tsai, T. Average case analysis of the Boyer-Moore algorithm. Random Struct. Algorithms 2006, 28, 481–498.
- Nicodème, P.; Salvy, B.; Flajolet, P. Motif statistics. Theor. Comput. Sci. 2002, 287, 593–617.
- Nicodème, P. Regexpcount, a symbolic package for counting problems on regular expressions and words. Fundam. Inform. 2002, 56, 71–88.
- Nuel, G. Pattern Markov chains: Optimal Markov chain embedding through deterministic finite automata. J. Appl. Probab. 2008, 45, 226–243.
- Lladser, M.; Betterton, M.D.; Knight, R. Multiple pattern matching: A Markov chain approach. J. Math. Biol. 2008, 56, 51–92.
- Marschall, T.; Rahmann, S. Exact Analysis of Horspool's and Sunday's Pattern Matching Algorithms with Probabilistic Arithmetic Automata. Proceedings of the 4th International Conference on Language and Automata Theory and Applications, LATA '10, Trier, Germany, 24–28 May 2010; Dediu, A.H., Fernau, H., Martín-Vide, C., Eds.;. Volume 6031pp. 439–450.
- Marschall, T.; Rahmann, S. Efficient exact motif discovery. Bioinformatics 2009, 25, i356–i364.
- Herms, I.; Rahmann, S. Computing Alignment Seed Sensitivity with Probabilistic Arithmetic Automata. Proceedings of the 8th International Workshop Algorithms in Bioinformatics, WABI '08, Karlsruhe, Germany, 15–19 September 2008; Crandall, K., Lagergren, J., Eds.; Springer: Berlin, Germany, 2008. Volume 5251; pp. 318–329.
- Hopcroft, J. An n log n Algorithm for Minimizing the States in a Finite Automaton. In The Theory of Machines and Computations; Kohavi, Z., Paz, A., Eds.; Academic Press: New York, NY, USA, 1971; pp. 189–196.
- Knuutila, T. Re-describing an algorithm by Hopcroft. Theor. Comput. Sci. 2001, 250, 333–363.
- Kucherov, G.; Noé, L.; Roytberg, M. A unifying framework for seed sensitivity and its application to subset seeds. J. Bioinform. Comput. Biol. 2006, 4, 553–569.
- Schulz, M.; Weese, D.; Rausch, T.; Döring, A.; Reinert, K.; Vingron, M. Fast and Adaptive Variable Order Markov Chain Construction. Proceedings of the 8th International Workshop Algorithms in Bioinformatics, WABI '08, Karlsruhe, Germany, 15–19 September 2008; Crandall, K.A., Lagergren, J., Eds.; Springer: Berlin, Germany, 2008. Volume 5251; pp. 306–317.
- Wu, S.; Manber, U. A Fast Algorithm for Multi-Pattern Searching. Technical report; University of Arizona: Tucson, AZ, USA, 1994.
© 2011 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/.)