A Survey on Shortest Unique Substring Queries

Abstract

The shortest unique substring (SUS) problem is an active line of research in the field of string algorithms and has several applications in bioinformatics and information retrieval. The initial version of the problem was proposed by Pei et al. [ICDE’13]. Over the years, many variants and extensions have been pursued, which include positional-SUS, interval-SUS, approximate-SUS, palindromic-SUS, range-SUS, etc. In this article, we highlight some of the key results and summarize the recent developments in this area.

1. Introduction

Let $\mathcal{S}$ be a string of length n and $\mathcal{S}[i,j]$ be the substring which starts at position i and ends at position j of $\mathcal{S}$. The substring $\mathcal{S}[i,j]$ is a repeat if it occurs more than once in $\mathcal{S}$; otherwise, it is a unique substring of $\mathcal{S}$. Since finding the shortest unique substrings is a non-trivial problem that has several applications for different purposes, variants of this problem have been studied. Table 1 shows the variants of the Shortest Unique Substring ($\mathsf{SUS}$) problem that we focused on in this survey.

Table 1. This table categorizes the main papers which are reviewed and discussed in this survey.

Variant $\mathsf{SUS}$ Queries
Position-$\mathsf{SUS}$	Interval-$\mathsf{SUS}$	Approximate-$\mathsf{SUS}$	Palindromic-$\mathsf{SUS}$	Range-$\mathsf{SUS}$
Pei et al. 2013 [1]	Hu et al. 2014 [2]	Hon et al. 2017 [3]	Inoue et al. 2018 [4]	Abedin et al. 2019 [5]
Ileri et al. 2014, 2015 [6,7]	Mieno et al. 2016 [8]	Allen et al. 2018, 2020 [9,10]	Wantabe et al. 2019, 2020 [11,12]
Tsuruta et al. 2014 [13]	Mieno et al. 2019 [14]	Schultz et al. 2018, 2020 [15,16]
Hon et al. 2015, 2017 [3,17]
Ganguly et al. 2016 [18]
Mieno et al. 2019 [14]

In 2005, Haubold et al. [19] explained how the shortest unique substring is a useful construct for alignment-free genome comparison. Unique substrings can help to determine the distinctness and difference between a group of closely related organisms [1,19]. In addition, an algorithm for finding a unique substring can be helpful to build a unique genetic fingerprint from a DNA sample or can help designing polymerase chain reaction (PCR) primer technique in molecule biology [1,20]. In 2015, Adas et al. [21] investigated the usage of shortest unique substrings for alignment and compression of DNA sequences. In addition to the applications in bioinformatics, the shortest unique substrings can be used in information retrieval for document search. The position based shortest unique substring (position-$\mathsf{SUS}$) queries was first proposed by Pei et al. [1]. Given a string $\mathcal{S}$ of length n and a query point p in the string, the problem is to find a shortest unique substring covering p. They presented an algorithm which costs $\mathcal{O}\left(n^2\right)$ time and $\mathcal{O}\left(n\right)$ space. In 2014, several publications have revisited this problem. İleri et al. [6] and Tsuruta et al. [13] proposed an optimal $\mathcal{O}\left(n\right)$ time and space algorithm for solving this problem. In addition, Hon et al. [3] and Ganguly et al. [18] presented compact data structures that can answer a position-$\mathsf{SUS}$ query using less than $2n$ words of space, where a word size is $\mathsf{\Omega }(logn)$. Several variations of the original problem have been proposed in the following years.

In 2014, Hu et al. [2] generalized the position-$\mathsf{SUS}$ problem by considering the positions in an interval rather than a single position in the string. For the interval-$\mathsf{SUS}$ problem, they preprocess the input string in $\mathcal{O}\left(n\right)$ time and space to answer any interval $\mathsf{SUS}$ query in constant time and return all $\mathsf{SUS}$s in $\mathcal{O}\left(occ\right)$ time, where $occ$ is the number of outputs. In 2019, Mieno et al. [14] improved the space complexity of this problem by providing a compact data structure.

In 2016, Mieno et al. [8] considered $\mathsf{SUS}$ queries on run-length encoded(RLE) strings. Their motivation was to reduce space and time complexity of processing the input string. They showed how to construct a data structure of size $\mathcal{O}(m+{\pi }_s(N,m))$ in $\mathcal{O}(mlogm+{\pi }_c(N,m))$ time that can answer interval-$\mathsf{SUS}$ queries in $\mathcal{O}({\pi }_q(N,m)+occ)$ time, where $occ$ is the output size, and ${\pi }_s(N,m)$, ${\pi }_c(N,m)$ and ${\pi }_q(N,m)$ are the size, construction time and the query time for a predecessor/successor query on m elements from a universe of $[1,N]$.

In 2017, Hon et al. [3] proposed the approximate version of $\mathsf{SUS}$ queries, where mismatches are allowed. This version can be applied in computational biology, where factors such as genetic mutation and experimental error make approximate string matching necessary [9]. They presented an in place algorithm for both exact and approximate versions of the problem. Afterwards, different trade-offs have been presented for k-mismatch $\mathsf{SUS}$ problem [9,10,15,16].

In 2018, Inoue et al. [4] proposed a palindromic variant of interval $\mathsf{SUS}$ problem. A palindromic string or substring is an important structure in DNA, RNA or protein sequence analysis [22]. In biology data, palindromic structures show the ability of molecules to fold and form double-stranded stems [23]. Given a string $\mathcal{S}$, a shortest unique palindromic substring ($\mathsf{SUPS}$) for an interval $[s,t]$ of $\mathcal{S}$ is the shortest substring that is palindromic and unique in $\mathcal{S}$ which contains $[s,t]$. For solving the interval $\mathsf{SUPS}$ problem, they preprocess $\mathcal{S}$ in $\mathcal{O}\left(n\right)$ time and space to output $\mathsf{SUPS}$s in $\mathcal{O}(occ+1)$ time. There are other works on $\mathsf{SUPS}$ queries by Watanabe et al. [11,12] based on the $RLE$ which are space economical solutions.

In 2019, Abedin et al. [5] focused on the range version of $\mathsf{SUS}$ queries (Range-$\mathsf{SUS}$) and generalized the problem. Range queries are a classic data structure topic, which has a great motivation in string processing problems [5,24,25,26]. Given a range $[\alpha ,\beta ]$, the problem is to return a shortest substring with exactly one occurrence in $[\alpha ,\beta ]$. They presented an $\mathcal{O}(nlogn)$-word data structure which answers $\mathsf{rSUS}$ queries in $\mathcal{O}\left({log}_{w}n\right)$ time per query in the word RAM model, where $w=\mathsf{\Omega }(logn)$ is the word size [5].

In this survey, we are going to discuss all approaches mentioned above on $\mathsf{SUS}$ queries. The focus of this work is on techniques applicable to $\mathsf{SUS}$ queries and is to compare all the main results in terms of complexities, restrictions, problem definitions, motivations, and applications. The main papers that we are going to focus on have been categorized in Table 1. In the last section, we discuss the related open questions on the variant topics related to $\mathsf{SUS}$ queries.

2. Preliminaries

2.1. Definitions

Let $\mathcal{S}[1,n]$ be a string of length n (i.e., $\left|\mathcal{S}\right|=n$) over an alphabet set $\mathsf{\Sigma }$ and $\mathcal{S}\left[i\right]$ is the ith character of $\mathcal{S}$. The substring of $\mathcal{S}$ which starts at position i and ends at position j of $\mathcal{S}$ is denoted by $\mathcal{S}[i,j]$. We have $1\le i\le j\le n$. If $j<i$, then $\mathcal{S}[i,j]$ is an empty string. A prefix of $\mathcal{S}$ is a substring $\mathcal{S}[1,i]$ of $\mathcal{S}$ for some $1\le i\le n$. $\mathcal{S}[1,i]$ is a proper prefix if $i\ne n$. A suffix of $\mathcal{S}$ is a substring $[j,n]$ for some $1\le j\le n$. It is a proper suffix if $j\ne 1$. We say $\mathcal{S}[i,j]$ covers position p, if $i\le p\le j$. $\mathcal{S}[i^{\prime },j^{\prime }]$ is a proper substring of $\mathcal{S}[i,j]$ if $i\le i^{\prime }\le j^{\prime }\le j$ and $\left|\mathcal{S}[i,j]\right|>|\mathcal{S}[i^{\prime },j^{\prime }]|$. A substring $\mathcal{S}[i,j]$ is a unique substring if there is no other substring $\mathcal{S}[i^{\prime },j^{\prime }]$ such that $\mathcal{S}[i,j]=\mathcal{S}[i^{\prime },j^{\prime }]$, where = indicates the identicality of two strings. The shortest unique substring ($\mathsf{SUS}$) covering position p is a unique substring of $\mathcal{S}$ that contains $\mathcal{S}\left[p\right]$, and there is no other unique substring with shorter length containing $\mathcal{S}\left[p\right]$. Note that there might exist more than one $\mathsf{SUS}$ covering position p. $\mathcal{S}[i,j]$ is a minimal unique substring ($\mathsf{MUS}$) if it is a unique substring and there is no proper substring of $\mathcal{S}[i,j]$ that is also unique. $\mathcal{S}[p,j]$ is called the left-bounded$\mathsf{SUS}$ for position p, $\mathsf{LSUS}\left(p\right)$, if $\mathcal{S}[p,j]$ is unique and there is no other substring $\mathcal{S}[p,j^{\prime }]$ which is also unique for $p\le j^{\prime }<j$. Symmetrically, $\mathcal{S}[j,p]$ is right-bounded$\mathsf{SUS}$ for position p, $\mathsf{RSUS}\left(p\right)$, if it is a unique substring and there is no other unique substring $\mathcal{S}[j^{\prime },p]$, where $j<j^{\prime }\le p$.

$H({\mathcal{S}}_1,{\mathcal{S}}_2)$ denotes the Hamming distance between two strings ${\mathcal{S}}_1$ and ${\mathcal{S}}_2$ of equal length, defined as the number of string positions where characters differ. $\mathcal{S}[i,j]$ is a k-mismatch unique substring if there is no substring $\mathcal{S}[i^{\prime },j^{\prime }]$ such that $i^{\prime }\ne i,j-i=j^{\prime }-i^{\prime }$ and $H(\mathcal{S}[i,j],\mathcal{S}[i^{\prime },j^{\prime }])\le k$. The k-mismatch left-bounded shortest unique substring $\mathsf{LSUS}$ starting at position p, denoted as ${\mathsf{LSUS}}_p$, is a k-mismatch unique substring $\mathcal{S}[p,j]$, such that either $p=j$ or any proper prefix of $\mathcal{S}[p,j]$ is not k-mismatch unique. k-mismatch $\mathsf{RSUS}$ and k-mismatch $\mathsf{MUS}$ can be defined similarly.

The longest common prefix of two suffixes $\mathcal{S}[p,n]$ and $\mathcal{S}[q,n]$ denoted by $\mathsf{LCP}\left(\mathcal{S}\right[p,n],\mathcal{S}[q,n\left]\right)$, is the longest common prefix between $\mathcal{S}[p,n]$ and $\mathcal{S}[q,n]$. The k-mismatch longest common prefix of $\mathcal{S}[p,n]$ and $\mathcal{S}[q,n]$ denoted as ${\mathsf{LCP}}^k(\mathcal{S}[p,n],\mathcal{S}[q,n])$ is the longest prefix which has Hamming distance ≤k between two suffixes.

2.2. Data Structures

The suffix tree data structure of string $\mathcal{S}[1,n]$ is a compact trie of the n suffixes of $\mathcal{S}$ appended with a letter $\$\notin \mathsf{\Sigma }$ [27]. This suffix tree consists of n leaves (one for each suffix of $\mathcal{S}$) and at most $n-1$ internal nodes. The edges are labeled with substrings of $\mathcal{S}$. We denote the suffix tree of string $\mathcal{S}$ with $\mathsf{ST}\left(s\right)$. The Suffix Array of string $\mathcal{S}$ of length n is denoted by $\mathsf{SA}$, which is a permutation of $\{1,\cdots ,n\}$, such that $\mathsf{SA}\left[i\right]=j$ if $\mathcal{S}[j,n]$ is the ith lexicographically smallest suffix of $\mathcal{S}$. The Inverse Suffix Array of string $\mathcal{S}$ of length n, is a permutation of $\{1,\cdots ,n\}$, such that ${\mathsf{SA}}^{-1}\left[\mathsf{SA}\left[i\right]\right]=i$. $\mathsf{SA}$ of $\mathcal{S}$ can be constructed in linear time and space [28,29]. The longest common prefix ($\mathsf{lcp}$) array of a string $\mathcal{S}$ of length n is an integer array of length n such that $\mathsf{lcp}\left[1\right]=0$ and $\mathsf{lcp}\left[i\right]$ stores the length of the longest common prefix between $\mathcal{S}\left[\mathsf{SA}\right[i-1],n]$ and $\mathcal{S}\left[\mathsf{SA}\right[i],n]$. Given the suffix array of $\mathcal{S}$, $\mathsf{lcp}$ array can be constructed in $\mathcal{O}\left(n\right)$ time. Let $A[1,n]$ be an array of length n. A range minimum query (RMQ) with an input range $[i,j]$ asks to report $RMQ(i,j)={argmin}_k\{A\left[k\right]\mid k\in [i,j]\}$. By maintaining a data structure of size $2n+o\left(n\right)$ bits, any RMQ on A can be answered in $O\left(1\right)$ time [30] (even without accessing A).

Consider s as a subset of $\{1,2,\cdots ,n\}$. Then, s can be preprocessed into an $O\left(\right|s\left|\right)$ space data structure, such that for any query p, we can return $\mathsf{pred}(p,s)$ and $\mathsf{succ}(p,s)$ in $O(loglogn)$ time [31], where

$$\mathsf{pred}(p,s)=max\{i\mid i\le p\mathrm{and}i\in s\cup \{-\infty \left\}\right\}$$

$$\mathsf{succ}(p,s)=min\{i\mid i\ge p\mathrm{and}i\in s\cup \{\infty \left\}\right\}$$

The eertree of a string $\mathcal{S}$, is a pair of two rooted trees $T_{odd}$ and $T_{even}$ that represent all distinct palindromic substrings of $\mathcal{S}$[32]. $T_{odd}$ and $T_{even}$ store the palindromic substrings of odd and even length, respectively. There is a directed edge $(r,a,v)$ from root r of $T_{odd}$ if v represents a single character $a\in \mathsf{\Sigma }$. For any non-root node u in either $T_{odd}$ or $T_{even}$, there is a labeled directed edge $(u,a,v)$ from u to v with character label a if $aua=v$. There are no two out-going edges from a node with the same label.

3. Position-SUS Queries

3.1. Motivation

Consider the procedure a search engine performs. Once a search query is given into a search engine by users, all the related pages should be identified and ranked properly. An indexing process is needed to organize information before each search query. There are algorithms such as inverted indexing to keep track of the documents with the pointers to text elements. The modern search engines may use a snippet, which is a short summarized content of a whole website and is shown in the search results. Finding a proper length for a snippet is critical. Either too short length and too long length would be problematic for making the text elements distinguishable and not overwhelming for users [1]. If the snippet for each result of a search is the shortest possible text including the query term and different from all other snippets, the search would be optimized. Thus, providing a fast algorithm for finding a shortest snippet is crucial. In addition to information retrieval purposes, there are some motivations in bioinformatics. In 2005, Haubold et al. [19] explained how the shortest unique substring is a useful construct for alignment-free genome comparison. Unique substrings can help to determine the distinctness and difference between a group of closely related organisms [1,19]. Another application is in event analysis when one wants to understand how an event differs from other events of the same type in a long sequence of historical events while extracting the context of the event. The shortest unique substring of the selected event may be helpful to proceed with the event analysis [1].

Regarding these motivations, Pei et al. [1] introduced the following problem.

Problem 1.

Position-$\mathsf{SUS}$ Queries

Input: String $\mathcal{S}[1,n]$ and a position $p\in \{1\cdots n\}$.

Output: $\mathsf{SUS}$ covering p: Substring $\mathcal{S}[i,j]$ containing position p, i.e., $i\le p\le j$, such that $\mathcal{S}[i,j]$ is unique and as short as possible

Example 1.

Given$\mathcal{S}=\stackrel{1}{\mathbf{c}}\stackrel{2}{\mathbf{a}}\stackrel{3}{\mathbf{a}}\stackrel{4}{\mathbf{b}}\stackrel{5}{\mathbf{a}}\stackrel{6}{\mathbf{a}}\stackrel{7}{\mathbf{d}}\stackrel{8}{\mathbf{d}}\stackrel{9}{\mathbf{a}}\stackrel{10}{\mathbf{a}}\stackrel{11}{\mathbf{c}}\stackrel{12}{\mathbf{a}}\stackrel{13}{\mathbf{d}}\stackrel{14}{\mathbf{d}}\stackrel{15}{\mathbf{a}}\stackrel{16}{\mathbf{a}}\stackrel{17}{\mathbf{a}}\stackrel{18}{\mathbf{a}}\stackrel{19}{\mathbf{b}}\stackrel{20}{\mathbf{a}}\stackrel{21}{\mathbf{c}}$and a query position $p=5$, we need to find a shortest substring $\mathcal{S}[i,j]$, with exactly one occurrence in $\mathcal{S}$ such that $i\le 5\le j$. One possible output is $\mathcal{S}[4,6]=baa$. Note that we may have multiple answers for Problem 1. $\mathcal{S}[5,7]=aad$ is another output.

3.2. Suffix Trees Based Approach

Pei et al. [1] stated Problem 1 and presented an algorithm to answer this problem in $\mathcal{O}\left(n\right)$ time and space. They construct $\mathsf{ST}\left(\mathcal{S}\right)$ in $\mathcal{O}\left(n\right)$ space and time [33]. Then, they use $\mathsf{ST}\left(\mathcal{S}\right)$ to get $\mathsf{LSUS}\left(p\right)$ in constant time by the following steps:

• Find the leaf node corresponding to the suffix $\mathcal{S}[p,n]$
• If the label of the leaf edge is $, it means that $\mathsf{LSUS}\left(p\right)$ does not exist and we return null; Otherwise, we continue.
• Let l be the length of the label of the leaf edge (excluding $).
• $\mathcal{S}[p,n-l+1]$ is $\mathsf{LSUS}\left(p\right).$

All of the above steps can be completed in $\mathcal{O}\left(1\right)$ time using the suffix tree properties. They make clever use of Lemma 1 and, consequently, they can find a $\mathsf{SUS}$ covering p by the following Lemma.

Lemma 1

([1,6]). Every $\mathsf{SUS}$ is either an $\mathsf{LSUS}$ or an extension of an $\mathsf{LSUS}$.

They start with assuming that $\mathsf{LSUS}\left(p\right)=\mathcal{S}[i,j]$ is a candidate answer, then they look for a $\mathsf{LSUS}\left(k\right)$ where $k<p$ and $k\ge j-i$ with the shortest length. Fore each $\mathsf{LSUS}\left(k\right)=\mathcal{S}[k,e]$, if $e<p$, instead of $\mathcal{S}[k,e]$, $\mathcal{S}[k,p]$ should be considered as a candidate answer since it should cover p. This is called an extension of an $\mathsf{LSUS}$. Thus, they always make sure that a new candidate covers p. At the end, since there may exist more than one answer, they output the leftmost $\mathsf{SUS}$ containing p.

In addition, they show how to preprocess $\mathcal{S}$ in $\mathcal{O}\left(n^2\right)$ time and $\mathcal{O}\left(n\right)$ space to compute the $\mathsf{SUS}$ corresponding to every position in the string. By doing so, $\mathsf{SUS}$ queries for any position can be answered in constant time. Their technique is based on the fact that each $\mathsf{SUS}$ should fall into one of $\mathsf{MUS}$, $\mathsf{LSUS}$, or $\mathsf{RSUS}$. By this observation, $\mathsf{SUS}$ corresponding to each position can be precomputed using their propagation procedure [1].

From the space complexity point of view, corresponding to each position p, their algorithm keeps track of a currently shortest $\mathsf{MUS}$ that covers p. The total space needed to store this information for all positions is $\mathcal{O}\left(n\right)$. At the end, they apply their algorithm on real data sets to show the effectiveness of their algorithm. Theorem 1 summarizes their result.

Theorem 1.

The position- $\mathsf{SUS}$ problem can be answered in $\mathcal{O}\left(n^2\right)$ time and $\mathcal{O}\left(n\right)$ space for every location of a string of length n.

3.3. Linear Time Approaches

Ileri et al. [6,7] and Tsuruta et al. [13] independently improved the time complexity of the Theorem 1 from $\mathcal{O}\left(n^2\right)$ to $\mathcal{O}\left(n\right)$ time. They showed that, by preprocessing the string $\mathcal{S}$ of length n in $\mathcal{O}\left(n\right)$ time, Problem 1 can be answered in constant time, which concludes a linear total time complexity for all positions in the string [6,7,13]. First, we discuss Ileri et al.’s approach.

3.3.1. Ileri et al.’s Framework

Their framework includes two cases; $\mathsf{SUS}$ finding for one position, and $\mathsf{SUS}$ finding for every position. In the first case, they present an algorithm for finding all the $\mathsf{SUS}$s covering a specific location p in $\mathcal{O}\left(n\right)$ time. Similar to the approach explained in Section 3.2, they make this observation that each $\mathsf{SUS}$ is an extension of an $\mathsf{LSUS}$. Instead of using suffix tree structure to find $\mathsf{LSUS}$s, they use inverse suffix array and $\mathsf{lcp}$ array to compute $\mathsf{LSUS}\left(i\right)$ for $i=1,2,\cdots n$ as follows:

$$\mathsf{LSUS}\left(i\right)=\left\{\begin{array}{cc}\mathcal{S}[i,i+l_i]\hfill & ifi+l_i\le n\hfill \\ null\hfill & otherwise\hfill \end{array}\right.,$$

where $l_i=max\{\mathsf{lcp}\left[{\mathsf{SA}}^{-1}\left[i\right]\right],\mathsf{lcp}[{\mathsf{SA}}^{-1}\left[i\right]+1]\}$ and $null$ means $\mathsf{LSUS}$ does not exist. Using the above equation, for each string position p, they simply compute $\mathsf{LSUS}\left(1\right)\cdots ,\mathsf{LSUS}\left(p\right)$ in $\mathcal{O}\left(p\right)$ time and maintain the shortest one for position p. Note that, if for some $k<p$, $\mathsf{LSUS}\left(k\right)$ does not cover p, we can extend it up to position p. In case of multiple shortest answers, they keep the leftmost candidate. Then, by Lemma 1, they prove their first theorem as follows:

Theorem 2

([6]). For any location p in the string $\mathcal{S}$, they can find $\mathsf{SUS}$ covering p using $\mathcal{O}\left(n\right)$ time and space. If multiple answers exist, the leftmost one is returned.

In the second case, they extend their algorithm to find all the $\mathsf{SUS}$s for every location $p\in \{1,2\cdots n\}$. Instead of iteratively running the algorithm for finding a $\mathsf{SUS}$ of a specific position n times, they use the following lemma to reduce the time complexity to the amortized cost for finding each $\mathsf{SUS}$ in $\mathcal{O}\left(1\right)$ time.

Lemma 2

([6]). For any $k\in \{2,3,\cdots ,n\}$, if $\mathsf{SUS}$ for position k is an extension of an $\mathsf{LSUS}$, then (1) $\mathsf{SUS}$ for position $k-1$ must be a substring whose right boundary is the character $\mathcal{S}[k-1]$, and (2) $\mathsf{SUS}$ for position k is the substring $\mathsf{SUS}$ for position $k-1$ appended by the character $\mathcal{S}\left[k\right]$.

For finding $\mathsf{SUS}$ of every position, they begin with $\mathsf{SUS}$ of the first position which is $\mathsf{LSUS}$ of that position; then, by Lemma 2, they compute $\mathsf{SUS}$ of position k using the already calculated $\mathsf{SUS}$ of position $k-1$. Their algorithm costs $\mathcal{O}\left(occ\right)$ for reporting the $\mathsf{SUS}$s covering a particular location. By providing an efficient constant time algorithm for computing the shortest $\mathsf{LSUS}$ covering each string position, they prove the following theorem.

Theorem 3.

All $\mathsf{SUS}$s corresponding to all the positions of string $\mathcal{S}[1,n]$ can be computed in $\mathcal{O}\left(n\right)$ time and space.

For the implementation, they use libdvsufsort library for implementing the suffix array and lcp array. They compare their results with Tsuruta et al.’s work and shows that, in terms of time complexity, both algorithms have almost the same processing time; however, their space usage is at least four times less for finding a single $\mathsf{SUS}$ and two times less for finding all $\mathsf{SUS}$s.

3.3.2. Tsuruta et al.’s Framework

Tsuruta et al. [13] improved Pei et al.’s (ICDE 2013) upper bounds for the shortest unique substring problem as well. They consider two types of $\mathsf{SUS}$ queries. The first one computes a $\mathsf{SUS}$ for any position in the string $\mathcal{S}$ in constant time after preprocessing $\mathcal{S}$ in $\mathcal{O}\left(n\right)$ time. Thus, reporting a $\mathsf{SUS}$ for all positions in $\mathcal{S}$ takes $\mathcal{O}\left(n\right)$ time in total. Their next algorithm outputs all the $\mathsf{SUS}$s covering a query position. Note that Pei et al.’s algorithm reports only one $\mathsf{SUS}$ for a query position. By preprocessing $\mathcal{S}$ in $\mathcal{O}\left(n\right)$ time, Tsuruta et al. provided an algorithm which reports all $\mathsf{SUS}$s containing a query position in $\mathcal{O}\left(occ\right)$ time, where $occ$ is the number of output.

Although their algorithm follows a similar technique to Ileri et al. [6], their work was independent. Their main idea is finding $\mathsf{SUS}$s from $\mathsf{MUS}$s using the following lemma.

Lemma 3

([13]). Every position-$\mathsf{SUS}$ contains exactly one $\mathsf{MUS}$.

They use $\mathsf{lcp}$ array data structure to compute all $\mathsf{MUS}$s of $\mathcal{S}$ in linear time. We summarize their results in the following lemma.

Lemma 4.

All the $\mathsf{LSUS}$, $\mathsf{RSUS}$, and $\mathsf{MUS}$ can be computed from $\mathcal{S}$ in $\mathcal{O}\left(n\right)$ time.

In order to compute $\mathsf{SUS}$s from $\mathsf{MUS}$s, they define the concept of meaningful and meaningless$\mathsf{MUS}$s, where meaningful $\mathsf{MUS}$s drive $\mathsf{SUS}$s corresponding to some positions. Then, they present an algorithm to collect all the $\mathsf{SUS}$s from the meaningful $\mathsf{MUS}$s in linear time. Theorem 4 summarizes their results.

Theorem 4.

A string S of length n can be preprocessed in $\mathcal{O}\left(n\right)$ time and space so that the shortest unique substring queries can be answered in $\mathcal{O}\left(occ\right)$ time, where occ is the number of shortest substrings returned. Notably, outputting a single $\mathsf{SUS}$ can be done in $\mathcal{O}\left(1\right)$ time.

3.4. In Place and Compact Data Structures’ Approaches

The suffix tree of a string $\mathcal{S}$ of length n occupies space almost 20 times larger than the space needed to store $\mathcal{S}$, which is $nlog\left|\mathsf{\Sigma }\right|$ bits. All aforementioned data structures in Section 3.3 require $\mathsf{\Theta }\left(n\right)$ words, where n is the length of the string $\mathcal{S}$. When n is large, the memory usage would be problematic. In order to avoid this issue, Hon et al. [3] proposed a data structure including the input string and two integer arrays for storing the starting positions and ending positions of $\mathsf{SUS}$s. The preprocessing time for their algorithm is linear and takes $\mathcal{O}(logn)$ bits of additional working space. If we want to consider the space needed to store all $\mathsf{SUS}$s for all positions, their algorithm takes the least amount of space to do that, which is $2n$ words to maintain the starting and ending positions of $\mathsf{SUS}$s and another n bytes to store the input string. Previous works need $\mathcal{O}\left(n\right)$ space while the hidden constant is large; however, that overhead does not exist in Hon et al.’s work. They use suffix arrays instead of suffix trees in their construction. Moreover, they handle all computations in the place of two integer arrays. As a result, their algorithm can find the position-$\mathsf{SUS}$ for every string position in $\mathcal{O}\left(n\right)$ time.

There was still a question of whether we can solve $\mathsf{SUS}$ problem in sub-linear space. To answer this question, Ganguly et al. [18] presented the first time-space trade off algorithm which uses $\mathcal{O}(n/\tau )$ words of additional space. Given a position $p\in \{1\cdots n\}$, their algorithm answers Problem 1 in $\mathcal{O}(n{\tau }^{2}log\frac{n}{\tau })$ time, where parameter $\tau \ge 1$. Another query is reporting $\mathsf{SUS}$s for all positions of $\mathcal{S}$. For this type of query, they present an $\mathcal{O}(n{\tau }^{2}logn)$ algorithm using $\mathcal{O}(n/\tau )$ words and $4n+o\left(n\right)$ bits of additional working space. In addition to these deterministic algorithms, they present a randomized algorithm in which the time complexity is $\mathcal{O}\left(n\tau {log}^{c+1}n\right)$ by using additional $n/{log}^{c}n$ words, where $c\ge 0$ is an arbitrary constant. There is a chance of at most $n^{-\mathcal{O}\left(1\right)}$ that the reported substring is unique and covers the query position but not the shortest one.

In their deterministic algorithms, their intuition is that each $\mathsf{SUS}$ for a position k is definitely the shortest unique prefix of $\mathcal{S}[i,n]$ for some $i\le k$, or the smallest right extension till position k of such prefix. For each suffix $\mathcal{S}[i,n]$, they define ${\mathsf{LS}}_i$ as the shortest unique prefix of $\mathcal{S}[i,n]$. In order to reduce the space complexity, instead of storing ${\mathsf{LS}}_i$s for all $\mathcal{S}[i,n]$, $i\in \{1\cdots n\}$, they choose a set of $\mathcal{O}(n/\tau )$ suffixes and compute the corresponding $\mathsf{LS}$s. After computing all the $\mathsf{LS}$s of the suffixes in the chosen set, using a brute force approach, for any suffix $\mathcal{S}[j,n]$, they can compute ${\mathsf{LS}}_j$. Their results are summarized in Theorem 5.

Theorem 5

([18]). For any given string position $p\in \{1\cdots n\}$, an $\mathsf{SUS}$ covering p can be computed in $\mathcal{O}(n{\tau }^{2}log\frac{n}{\tau })$ time and additional $\mathcal{O}(n/\tau )$ working space. In addition, computing any $\mathsf{SUS}$ for every position in the string can be reported in $\mathcal{O}(n{\tau }^{2}logn)$ time and additional $\mathcal{O}(n/\tau )$ words and $4n+o\left(n\right)$ bits of additional working space.

Ganguly et al.’s algorithm reports only one $\mathsf{SUS}$ for each given query. In 2019, Mieno et al. [14] presented a data structure of size $\lceil ({log}_{2}3+1)n)\rceil +o\left(n\right)$ bits, which can answer Problem 1 in $\mathcal{O}\left(occ\right)$ time, where $occ$ is the number of $\mathsf{SUS}$s for the given query point. The main intuition of their algorithm is based on Lemma 3. Their data structure includes two bit arrays ${\mathsf{MB}}_{\mathcal{S}}$ and ${\mathsf{ME}}_{\mathcal{S}}$, each of length n, to keep track of the starting and ending positions of the $\mathsf{MUS}$s. As a result, they prove that there exists a data structure of size $\lceil ({log}_{2}3+1)n)\rceil +o\left(n\right)$ bits that can answer $\mathsf{SUS}$s in $\mathcal{O}\left(occ\right)$ time. Given the bit arrays ${\mathsf{MB}}_{\mathcal{S}}$ and ${\mathsf{ME}}_{\mathcal{S}}$, the data structure can be constructed in $\mathcal{O}\left(n\right)$ time using $3n+\lceil n{log}_{2}3\rceil +o\left(n\right)$ bits of total working space.

4. Interval-SUS Queries

In this section, we discuss a generalization of position-$\mathsf{SUS}$ problem which is defined as follows:

Problem 2.

Interval $\mathsf{SUS}$ Queries

Input: String $\mathcal{S}[1,n]$ and a query interval $[s,t]\in [1,n]$.

Output: All $\mathsf{SUS}$s of $\mathcal{S}$ containing $[s,t]$

Example 2.

Given$\mathcal{S}=\stackrel{1}{\mathbf{c}}\stackrel{2}{\mathbf{a}}\stackrel{3}{\mathbf{a}}\stackrel{4}{\mathbf{b}}\stackrel{5}{\mathbf{a}}\stackrel{6}{\mathbf{a}}\stackrel{7}{\mathbf{d}}\stackrel{8}{\mathbf{d}}\stackrel{9}{\mathbf{a}}\stackrel{10}{\mathbf{a}}\stackrel{11}{\mathbf{c}}\stackrel{12}{\mathbf{a}}\stackrel{13}{\mathbf{d}}\stackrel{14}{\mathbf{d}}\stackrel{15}{\mathbf{a}}\stackrel{16}{\mathbf{a}}\stackrel{17}{\mathbf{a}}\stackrel{18}{\mathbf{a}}\stackrel{19}{\mathbf{b}}\stackrel{20}{\mathbf{a}}\stackrel{21}{\mathbf{c}}$and an interval query position $[14,16]$, we need to find a shortest substring $\mathcal{S}[i,j]$, with exactly one occurrence in $\mathcal{S}$ such that $i\le 14<16\le j$. The output here is $\mathcal{S}[14,17]=daaa$ which includes $\mathcal{S}[14,16]=daa$ and it does not have any other occurrence in $\mathcal{S}$.

4.1. Linear Time Approaches

Hu et al. [2] generalized Problem 1 to Problem 2 and presented a linear time and space data structure which can solve Problem 2 in $\mathcal{O}\left(occ\right)$ time, where $occ$ is the number of all $\mathsf{SUS}$s containing the query interval. The interval $\mathsf{SUS}$ problem is more difficult to be solved in $\mathcal{O}\left(occ\right)$ time, since there exist $\mathsf{\Theta }\left(n^2\right)$ intervals and we cannot store the candidate answer for every possible interval. In order to deal with this issue, Hu et al. use the concept of $\mathsf{LSUS}$, $\mathsf{RSUS}$, $\mathsf{MUS}$, and Corollary 1 to prove Lemma 5.

Corollary 1

([34]). There exists a data structure of $\mathcal{O}\left(n\right)$ size which can be constructed in $\mathcal{O}\left(n\right)$ time that can check whether a given substring of $\mathcal{S}$ is unique in $\mathcal{O}\left(1\right)$ time.

Lemma 5

([34]). The answer of the interval-$\mathsf{SUS}$ problem with the query interval $[x,y]$ must be the shortest of the following candidates:

1. 

$\mathcal{S}[x,y]$ if it is unique. This can be checked in constant time and linear space.

2. 

$\mathsf{LSUS}\left(x\right)$: This can be computed in constant time using linear space by Lemma 4.

3. 

$\mathsf{RSUS}\left(y\right)$: This can be computed in constant time using linear space by Lemma 4.

4. 

The shortest $\mathsf{MUS}$ containing $[x,y]$: It remains to show their structure of computing this candidate.

Let M be the set of all $\mathsf{MUS}$s in $\mathcal{S}$. For presenting their data structure, they define a set of intervals denoted by I, such that $[i,j]\in I$ iff $\mathcal{S}[i,j]$ is in M. Then, they reduce the interval-$\mathsf{SUS}$ problem to Containment Min. Given an interval $[x,y]$, a containment min query returns the shortest interval in I which contains $[x,y]$. Hu et al. [2] proposed a data structure of size $\mathcal{O}\left(n\right)$ in which RMQ is performed to answer a containment min query in $\mathcal{O}\left(1\right)$ time.

In addition, they show how to report all $\mathsf{SUS}$s in the case that interval $\mathsf{SUS}$ has more than one answer. For this purpose, they use an auxiliary problem which is defined below.

Position Constraint Query. Given a substring $\mathcal{S}[x,y]$ and two ranges $rang{e}_{start}=[s_1,s_2]$ and $rang{e}_{end}=[e_1,e_2]$ both in $[1,n]$, the problem is to return a unique substring $\mathcal{S}[i,j]$ with the minimum length that contains $\mathcal{S}[x,y]$ such that $i\in [s_1,s_2]$ and $j\in [e_1,e_2]$.

This problem is a constraint version of the interval $\mathsf{SUS}$ problem where the starting and ending positions of the answer substring should be contained in the two intervals specified in the problem. They start with the normal interval-$\mathsf{SUS}$ query. Given the input interval $[x,y]$, let $\mathcal{S}[i,j]$ be the answer of the interval-$\mathsf{SUS}$. They consider two intervals $[1,i-1]$ and $[i+1,x]$. Now, using two position constraint queries, the next answers can be found. First, they run a position constrained query with $\mathcal{S}[x,y]$, $rang{e}_{start}=[1,i-1]$ and $rang{e}_{end}=[y,n]$ to return the other possible answers. The other query corresponding to $[i+1,x]$ is symmetrical. In total, their algorithm takes $\mathcal{O}\left(occ\right)$ time to report all the $\mathsf{SUS}$s.

The mentioned algorithms are in the RAM model. They also consider solving Problem 2 in the standard external memory model [35]. They follow a similar technique. Assume that $\mathcal{O}\left(SORT\right(n\left)\right)$ is the number of I/Os needed to sort n elements and B is the number of words in a block of a disk. They pre-compute an index structure from $\mathcal{S}$ in $\mathcal{O}\left(SORT\right(n\left)\right)$ I/Os in external memory that occupies $\mathcal{O}(n/B)$ blocks and can answer any shortest unique substring query in $\mathcal{O}\left(1\right)$ I/Os [2].

4.2. RLE-Based Approaches

The Run Length Encoding (RLE) of a string is a compressed representation in which each maximal run of a character c of length ℓ is encoded as $c^{\ell }$. For instance, the $RLE$ of string $aaaaaaabbbbaaac$ is $a^7{b}^4{a}^3{c}^1$. Mieno et al. [8] considered solving Problem 2 in the case where the input string is given in $RLE$ representation. Their motivation was to reduce space and time complexity of processing the input string. They presented a data structure of size $\mathcal{O}(m+{\pi }_s(N,m))$ that can be constructed in $\mathcal{O}(mlogm+{\pi }_c(N,m))$ time to answer interval-$\mathsf{SUS}$ queries in $\mathcal{O}({\pi }_q(N,m)+occ)$ time, where $occ$ is the output size, and ${\pi }_s(N,m)$, ${\pi }_c(N,m)$ and ${\pi }_q(N,m)$ are the size, construction time and the query time for a predecessor/successor query on m elements from a universe of $[1,N]$. In their approach, they use combinatorial properties on $\mathsf{MUS}$s and $RLE$ strings. Let m be the length of the string $\mathcal{S}$ in $RLE$ representation. We denote $RLE$ representation of $\mathcal{S}$ by $RLE\left(\mathcal{S}\right)$. They show that the number of $\mathsf{MUS}$s in $RLE\left(\mathcal{S}\right)$ is $2m-1$. Thus, instead of dealing with $\mathsf{\Theta }\left(n\right)$ $\mathsf{MUS}$s like the previous related results, a considerable amount of space and time would be reduced if there exist runs in the string. By doing so, they were able to build an $\mathcal{O}\left(m\right)$ size data structure for the $RLE$ version of $\mathsf{SUS}$ problem, which is formally defined as follows:

Problem 3.

RLE-$\mathsf{SUS}$ Queries

Preprocess: $RLE\left(\mathcal{S}\right)=c_1^{{\ell }_1}{c}_2^{{\ell }_2}\cdots c_m^{{\ell }_m}$

Query: An interval $[s,t]\in [1,n]$

Return: All $\mathsf{SUS}$ of $\mathcal{S}$ containing the query interval $[s,t]$

For solving Problem 3, they show how to precompute all the $\mathsf{MUS}$s using a specific type of suffix arrays for $RLE$ strings [36]. Before discussing their techniques, we bring some definitions. Let $bpos\left(i\right)$, $epos\left(i\right)$ and $exp\left(i\right)$ be the beginning position, ending position, and exponent of the ith run of $RLE\left(\mathcal{S}\right)$. Let P be be any subset of positions of $\mathcal{S}$. The sparse suffix array of $\mathcal{S}$ w.r.t P denoted by ${\mathsf{SSA}}_P$ is an array of size $\left|P\right|$ such that ${\mathsf{SSA}}_P\left[i\right]\in P$ for all $1\le i\le \left|P\right|$ and also like a normal suffix array ${\mathsf{SSA}}_P\left[i\right]$ is lexicographically smaller than ${\mathsf{SSA}}_P[i+1]$. They use trancated $RLE$ suffix array for $RLE\left(\mathcal{S}\right)$ denoted by $tRLESA$, in which P is the set of $epos\left(i\right)$ for $1\le i\le m$. In addition, let $EXP$ be an array of length m such that $EXP\left[i\right]=exp\left(k\right)$, where $tRLESA\left[i\right]=epos\left(k\right)$. Given $RLE\left(s\right)$, all the defined arrays can be computed in $\mathcal{O}(mlogm)$ time with $\mathcal{O}\left(m\right)$ working space [14].

In order to compute $\mathsf{MUS}$s from $RLE\left(\mathcal{S}\right)$, they consider three disjoint partitions of $\mathsf{MUS}$, $M_1,M_2,M_3$. The first partition consists of $\mathsf{MUS}$s which are contained in runs. The second one is $\mathsf{MUS}$s that start at the last characters of runs, and all the other $\mathsf{MUS}$s considered to be in the third partitions. By the fact that each $\mathsf{MUS}$ cannot be a proper substring of another $\mathsf{MUS}$ and discussing the size of each partition separately, they prove that $\left|M\right|\le 2m-1$. Then, they show how to find $\mathsf{MUS}$ in $\mathcal{O}(mlogm)$ time and $\mathcal{O}\left(m\right)$ space, by computing the partitions.

In order to compute $M_1$, for each character used in $\mathcal{S}$, they check whether there exists a run of that character with a unique maximum exponent, since the number of distinct character used in $\mathcal{S}$ is $\le m$, this procedure can be done in $\mathcal{O}(mlogm)$ time and $\mathcal{O}\left(m\right)$ space using a sorting algorithm. For computing $M_2$ and $M_3$, they use $tRLESA$, $tRLES{A}^{-1}$ (inverse of $tRLESA$), and $EXP$ arrays. Similar to $M_1$, $M_2$ and $M_3$ can be computed in $\mathcal{O}(mlogm)$ time and $\mathcal{O}\left(m\right)$ space. After the preprocessing step, they could answer Problem 3 using RMQ, predecessor and successor queries on the array ${\mathsf{MUS}}_{len}$ that stores the lengths of the $\mathsf{MUS}$s. Their results were summarized in the following theorem.

Theorem 6

([8]). Given $RLE\left(\mathcal{S}\right)$ of size m, there is a data structure of size $\mathcal{O}(m+{\pi }_s(N,m))$ that can be computed in $\mathcal{O}(mlogm+{\pi }_c(N,m))$ time which can answer interval $\mathsf{SUS}$ queries in $\mathcal{O}({\pi }_q(N,m)+occ)$ time, where $occ$ is the output size, and ${\pi }_s(N,m)$, ${\pi }_c(N,m)$, and ${\pi }_q(N,m)$ are the size, construction time and the query time for a predecessor/successor query on m elements from a universe of $[1,N]$.

4.3. Compact Data Structures’ Approaches

In Section 3.4, we have discussed Mieno et al’s work [8] for solving Problem 1. They also presented a compact data structure for Problem 2 [14]. Their data structure has the size of $2n+2m+o\left(n\right)$ bits and outputs an interval $\mathsf{SUS}$ query in $\mathcal{O}\left(occ\right)$ time, where m is the number of minimal unique substrings ($\mathsf{MUS}$s) of the input string which is at most n. Their technique is based on Mieno et al.’s work [8], which is discussed in Section 4.2. Similar to the technique for solving position-$\mathsf{RSUS}$, their structure is based on the two bit arrays $\mathsf{MB}$ and $\mathsf{ME}$. In addition, they use array ${\mathsf{MUS}}_{len}$ which has been used in Mieno et al.’s work for storing the length of the $\mathsf{MUS}$s [8]. On the top of $\mathsf{MB}$ and $\mathsf{ME}$, a successor and a predecessor data structures are maintained. ${\mathsf{MUS}}_{len}$ is also endowed with RMQ data structure. Now, once an interval query $[s,t]$ comes, first we can find $g={\mathsf{pred}}_{\mathsf{ME}}\left(y\right)$ and $r={\mathsf{succ}}_{\mathsf{MB}}\left(s\right)$, then we can find the range of $\mathsf{SUS}$ covering $[s,t]$. Consequently, it is sufficient to answer RMQ queries on the corresponding range on ${\mathsf{MUS}}_{len}$. The space needed for $\mathsf{MB}$ and $\mathsf{ME}$ and the predecessor and successor data structure is $2n+o\left(n\right)$ bits. The RMQ data structure on ${\mathsf{MUS}}_{len}$ takes $2m+o\left(m\right)$ bits of space. Thus, using the constant query time of the RMQ data structure (see Section 2.2), their results can be summarized as follows:

Theorem 7.

There exists a data structure of size $2n+2m+o\left(n\right)$ bits that can answer interval $\mathsf{SUS}$ problem in $\mathcal{O}\left(occ\right)$ time, where $occ$ is the number of the answers corresponding to the given interval.

5. Approximate-SUS Queries

5.1. Motivation

In molecular biology, shortest unique substrings found in DNA sequences can be used to compare similar organisms and determine unique patterns. It also helps to design polymer chain reaction (PCR) [1,15]. If we just consider exact shortest unique substrings while comparing distinct organisms, possible patterns might be disregarded due to errors or mutations. In this section, we discuss the approximate version of $\mathsf{SUS}$ problem which is proposed by Hon et al. [3], in which the uniqueness constraint is more strict. In this variant of $\mathsf{SUS}$ problem, the unique substrings are allowed for up to k mismatches. This version can be applied in computational biology, where factors such as genetic mutation and experimental error make approximate string matching necessary [9]. Another useful application of this approximate version is in computing average common substring, which has been considered as an approach to phylogenomic reconstruction [37,38]. In order to estimate the evolutionary distance between pairs of primate genomes, Thankachan et al. [39] showed that adding a similar k-mismatch parameter to average common substring finding equation leads to better results [9]. A k-mismatch shortest unique substring covering a position p, denoted by ${\mathsf{SUS}}_p^k$, is a k-mismatch unique substring $\mathcal{S}[i,j]$ such that satisfies the condition $i\le p\le j$ and there is no other k-mismatch unique substring with shorter length which satisfies the condition. Note that, similar to the definition of k-mismatch $\mathsf{SUS}$ in Section 2.1, we also consider the Hamming distance for the k-mismatch $\mathsf{SUS}$ problem. The problem is formally defined as follows:

Problem 4.

k-mismatch $\mathsf{SUS}$ Queries

Input: String $\mathcal{S}[1,n]$ and integer k

Output: Two integer arrays A and B s.t $\mathcal{S}\left[A\right[i]\cdots B[i\left]\right]$ is the rightmost ${\mathsf{SUS}}_i^k$ for every index i

Example 3.

If$\mathcal{S}=\stackrel{1}{\mathbf{c}}\stackrel{2}{\mathbf{a}}\stackrel{3}{\mathbf{a}}\stackrel{4}{\mathbf{b}}\stackrel{5}{\mathbf{a}}\stackrel{6}{\mathbf{a}}\stackrel{7}{\mathbf{d}}\stackrel{8}{\mathbf{d}}\stackrel{9}{\mathbf{a}}\stackrel{10}{\mathbf{a}}\stackrel{11}{\mathbf{c}}\stackrel{12}{\mathbf{a}}\stackrel{13}{\mathbf{d}}\stackrel{14}{\mathbf{d}}\stackrel{15}{\mathbf{a}}\stackrel{16}{\mathbf{a}}\stackrel{17}{\mathbf{a}}\stackrel{18}{\mathbf{a}}\stackrel{19}{\mathbf{b}}\stackrel{20}{\mathbf{a}}\stackrel{21}{\mathbf{c}}$, ${\mathsf{SUS}}_5^1=\mathcal{S}[4,7]=baad$. Note that ${\mathsf{SUS}}_5^0=\mathcal{S}[4,6]=baa$ as we saw in Example 1. Since there exists a substring $caa$ such that $H(baa,caa)=1$, $\mathcal{S}[4,6]=baa$ cannot be a ${\mathsf{SUS}}_5^1$.

5.2. Hon et al.’s Framework

Hon, Thankachan, and Xu considered solving Problem 4 in the place of S, A, and B, where S is for storing string $\mathcal{S}$, A and B are two empty arrays for storing the starting and ending positions of the rightmost k-mismatch $\mathsf{SUS}$ of each position. They prove the k-mismatch version of Lemma 2. Here, we explain the high level of their in-place framework for solving both Problems 1 and 4 which has three stages.

Let ${\mathsf{LSUS}}_p^k$ be the k-mismatch left bounded shortest unique substring starting at position p. In the first stage, they compute ${\mathsf{LSUS}}_i^k$ for all is, in place of A and B. At the end of this step, each $B\left[i\right]$ stores the ending position of ${\mathsf{LSUS}}_i^k$. This procedure takes linear time when $k=0$ and $\mathcal{O}\left(n^2\right)$ time for $k\ge 1$ because of their dynamic programming approach. Let ${\mathsf{SLS}}_p^k$ be the shortest k-mismatch $\mathsf{LSUS}$ covering position p. In the second stage, they use array B, computed from the previous stage, to find the rightmost ${\mathsf{SLS}}_i^k$ for all i in place of A and B. Each $A\left[i\right]$ stores the largest j such that ${\mathsf{LSUS}}_j^k$ is an ${\mathsf{SLS}}_j^k$. Thus, if ${\mathsf{SLS}}_i^k$ exists, it is equal to $\mathcal{S}\left[A\right[i]\cdots B[i\left]\right]$. This takes $\mathcal{O}\left(n\right)$ time for all $k\ge 0$. At the last stage, they use both A and B to compute ${\mathsf{SUS}}_i^k$ for all i, in the place of A and B. At the end of the stage, $A\left[i\right]$ and $B\left[i\right]$ store the starting and ending positions of the rightmost ${\mathsf{SUS}}_i^k$. This step also takes linear time for any $k\ge 0$. Note that only one stage requires the quadratic time. They implemented their algorithm in C using $libdivsufsort$ library for the suffix array construction. Their contribution can be summarized as follows:

Theorem 8.

Using an additional $2n$ words each of size $\lceil {log}_2\left(n\right)\rceil$ bits, and n bytes of space for storing the input string, Hon et al. [3] provided a theoretically in-place framework to solve both exact and the approximate position $\mathsf{SUS}$ using a total of $\mathcal{O}\left(n\right)$ and $\mathcal{O}\left(n^2\right)$ time, respectively.

5.3. Allen et al.’s Framework

In order to reduce the quadratic time complexity for solving k-mismatch $\mathsf{SUS}$ queries, Allen et al. [9] presented an average time complexity of $\mathcal{O}\left(n{log}^{k}n\right)$ using $\mathcal{O}\left(kn\right)$ space. They follow the technique provided by Thankachan et al. [39] for solving k-mismatch average common substring.

Let the k-mismatch left-bounded longest repeat starting at position i denoted by ${\mathsf{LLR}}_i^k$ be a k-mismatch repeat substring $\mathcal{S}[i,j]$ such that either $j=n$ or $\mathcal{S}[i,j+1]$ is a k-mismatch unique. Consider L as an array of length n where $L\left[i\right]=|{\mathsf{LLR}}_i^k|$. Now, the algorithm discussed in Section 5.2 can be applied to compute ${\mathsf{SUS}}_i^k$ for each position i in $\mathcal{S}$ in linear time and space. Computing array L can be a substitution for stage one of the Hon et al.’s work [3]. To avoid $\mathcal{O}\left(n^2\right)$ time, instead of comparing every pair of suffixes of $\mathcal{S}$, they presented an expected $\mathcal{O}\left(m{log}^{k}m\right)$ time for calculating each entry of L using the technique in [39], where m is the total length of two suffixes used in computing $L\left[i\right]$ as follows:

$$L\left[i\right]=max\left\{\right|{\mathsf{LCP}}^k({\mathcal{S}}_i,{\mathcal{S}}_j)|j\ne i\}$$

The total time complexity to compute every entry of L would be expected $\mathcal{O}\left(n{log}^{k}n\right)$.

In 2020, Allen et al. [10] published a new version of their work which provides algorithmic bounds for k-mismatch $\mathsf{SUS}$ problem. They presented an algorithm which can solve Problem 4 using $\mathcal{O}\left(n{log}^{k}n\right)$ time and $\mathcal{O}\left(n\right)$ space. This worst-case bound is asymptotically much better than the practical algorithm. In comparison to Hon et al’s experimental results [3], their new implementation shows that the practical algorithm is easy to implement and it takes less time when k is small relative to n.

5.4. GPU Based Approach

All the works that have been discussed on k-mismatch $\mathsf{SUS}$ query were focusing on improving the time and space complexity for $\mathsf{SUS}$ computation in the sequential CPU model. In 2018, Schultz and Xu [15] presented the first parallel approach for k-mismatch $\mathsf{SUS}$ problem in the shared-memory model, particularly leveraging on the massive multi-threading GPU technology. Obviously, this approach is experimentally faster than the CPU solutions. Their experimental results on a mid-end GPU show that the GPU approach is at least six times faster than the CPU approach in the case of $k=0$ (exact match), and at least 23 times faster in the general case (k > 0). This is essential when the input string is massively long as genomic sequences are studied. In terms of memory usage, this approach is almost the same as the sequential CPU approaches. Their algorithms totally happen on GPU except a transferring input string to GPU, which is done by the CPU host. When there are no mismatches ($k=0$), they implicitly use the efficient data structures like suffix array and $\mathsf{lcp}$ array. However, when $k>0$, using these data structures would be problematic since they do not have enough information such as k-mismatch $\mathsf{LCP}$s. To solve this issue, Schultz and Xu [15] designed a method that can be parallelized in the GPU architecture.

In 2019, they published an extension of their paper [16], providing more experimental results on exact $\mathsf{SUS}$ and approximate $\mathsf{SUS}$ computation. They show the speedup gained by the GPU-based approach against the sequential solution, including or excluding suffix array construction. It is important to observe the difference of speedup in case of whether the suffix array is given or not. Corresponding to their results, almost $50\%$ of the time of the sequential approach is to spend on suffix array construction. Thus, if the suffix array is given, it can be copied to the GPU memory directly to achieve a better speedup.

6. SUPS Queries

6.1. Motivation

We call a substring $\mathcal{S}[i,j]$ is a palindrome if it is identical with its reverse. There have been many studies on palindromic strings and their combinatorial properties [32,40,41]. A palindromic string or substring is an important structure in DNA, RNA, or protein sequence analysis [22]. In biology data, palindromic structures show the ability of molecules to fold and form double-stranded stems [23]. In addition, by the similar palindromic structures of the protein strands, we can guess their similar secondary structures. Another application of palindromic substructures is in gene editing and gene regulation in species [22,42]. In this section, we are going to discuss a new version of $\mathsf{SUS}$ queries named shortest unique palindromic substring($\mathsf{SUPS}$) problem. A shortest unique palindromic substring $\mathsf{SUPS}$ for an interval $[s,t]$ is the shortest substring $\mathcal{S}[i,j]$ such that it is unique in $\mathcal{S}$ and contains $[s,t]$, and any other palindromic substring of $\mathcal{S}$, which contains $[s,t]$ and is shorter than $\mathcal{S}[i,j]$, which occurs more than once in $\mathcal{S}$. The $\mathsf{SUPS}$ problem is formally defined as follows:

Problem 5.

$\mathsf{SUPS}$ Queries

Preprocess: $\mathcal{S}[1,n]$

Query: An interval $[s,t]\in [1,n]$

Return: All $\mathsf{SUPS}$ of $\mathcal{S}$ containing the query interval $[s,t]$

Example 4.

If$\mathcal{S}=\stackrel{1}{\mathbf{c}}\stackrel{2}{\mathbf{c}}\stackrel{3}{\mathbf{a}}\stackrel{4}{\mathbf{b}}\stackrel{5}{\mathbf{a}}\stackrel{6}{\mathbf{a}}\stackrel{7}{\mathbf{d}}\stackrel{8}{\mathbf{d}}\stackrel{9}{\mathbf{a}}\stackrel{10}{\mathbf{a}}\stackrel{11}{\mathbf{c}}\stackrel{12}{\mathbf{a}}\stackrel{13}{\mathbf{d}}\stackrel{14}{\mathbf{d}}\stackrel{15}{\mathbf{a}}\stackrel{16}{\mathbf{a}}\stackrel{17}{\mathbf{a}}\stackrel{18}{\mathbf{a}}\stackrel{19}{\mathbf{b}}\stackrel{20}{\mathbf{a}}\stackrel{21}{\mathbf{c}}$, given the query interval $[8,9]$, $\mathsf{SUPS}$ problem outputs $\mathcal{S}[6,9]=adda$, which is the shortest palindrome containing $\mathcal{S}[8,9]=da$.

6.2. Optimal Approaches

In 2018, Inoue et al. [4] introduced Problem 5, which is a version of $\mathsf{SUS}$ queries focusing on palindromes. They propose an optimal algorithm which takes $\mathcal{O}\left(n\right)$ time to preprocess the string $\mathcal{S}$ and can return all $\mathsf{SUPS}$ for any interval query in $\mathcal{O}(occ+1)$ time, where $occ$ is the number of outputs. For any interval $[i,j]$, checking whether $\mathcal{S}[i,j]$ is a palindromic substring or not takes $\mathcal{O}\left(n\right)$ preprocessing time and constant query time using $\mathcal{O}\left(n\right)$ space. Similar to the approach for solving position/interval $\mathsf{SUS}$, they define Minimal Unique Palindromic substring(MUPS). $MUPS$ $\mathcal{S}[i,j]$ is a unique palindrome substring in $\mathcal{S}$ such that $\mathcal{S}[i+1,j-1]$ is a repeat in $\mathcal{S}$ or $1\le \left|\mathcal{S}\right[i,j\left]\right|\le 2$. Similar to $\mathsf{MUS}$s, $MUSPs$ cannot contain each others. In addition, for each $\mathsf{SUPS}$ $\mathcal{S}[i,j]$ for some interval, there exists exactly one $MUPS$ that is contained in $[i,j]$ with the same center as $\mathcal{S}[i,j]$ $\left(\frac{i+j}{2}\right)$. Using this fact, they could preprocess $\mathcal{S}$ in linear time to report all $\mathsf{SUPS}$s.

In addition, they provided some bounds on the number of $\mathsf{SUPS}$s. They showed the maximum number of point and interval $\mathsf{SUPS}$ in $\mathcal{S}$, which is a string from an arbitrary size alphabet, is at most n. For a binary alphabet, they gave a lower bound for the number of point $\mathsf{SUPS}$. Let $\mathcal{P}{\mathcal{S}}_{\mathcal{S}}$ and $\mathcal{I}{\mathcal{S}}_{\mathcal{S}}$ be the set of substrings of $\mathcal{S}$ where each of the substrings in $\mathcal{P}{\mathcal{S}}_{\mathcal{S}}$ is an $\mathsf{SUPS}$ for some point and each of the substrings in $\mathcal{I}{\mathcal{S}}_{\mathcal{S}}$ is an $\mathsf{SUPS}$ for some interval.

Theorem 9

([4]). There exists a binary string ${\mathcal{S}}_k$ of length $6k+8$ such that $\mathcal{P}{\mathcal{S}}_{{\mathcal{S}}_k}=\mathcal{I}{\mathcal{S}}_{{\mathcal{S}}_k}=4k+6$ for any $k\ge 0$. Thus, ${\mathcal{S}}_k$ contains at least $\frac{2}{3}n$ point and interval $\mathsf{SUPS}$s where $n=|{\mathcal{S}}_k|$

In the end, they discuss the number of point and interval $\mathsf{SUPS}$s on palindromic rich strings. A string is a palindromic rich if it has $n+1$ distinct palindromic substring including an empty string. They consider the palindromic rich string $R_k=a^1{b}^1\cdots a^k{b}^k$ and prove two following theorems.

Theorem 10

([4]). There exists a binary string of length n such that the number of interval $\mathsf{SUPS}$s is $n-\sqrt{1+4n}+3$.

Theorem 11

([4]). There exists a binary string $R_k$ of size $k(k+1)$ s.t $\mathcal{I}{\mathcal{S}}_{R_k}=k(k-1)+2$ for any $k\ge 1$. Thus, $R_k$ contains $n-\sqrt{1+4|R_k|}+3$ interval $\mathsf{SUPS}$.

6.3. RLE-Based Approaches

Watanabe et al. [11] considered the shortest unique palindromic substring when the input string is given in its $RLE$ representation. They showed how to preprocess a given $RLE\left(S\right)$ of length m in $\mathcal{O}\left(m\right)$ space and $\mathcal{O}(mlog{\sigma }_{RL{E}_{\mathcal{S}}}+m\sqrt{logm/loglogm})$ time, where ${\sigma }_{RL{E}_s}$ is the umber of distinct runs of $RL{E}_s$. Their work is the first space-economial for $\mathsf{SUPS}$ problem and can answer queries in $\mathcal{O}(\sqrt{logm/loglogm}+occ)$ time.

Similar to the technique for solving the normal $\mathsf{SUPS}$ problem which is discussed in the previous subsection, they compute $MUPS$s of S in the preprocessing step. Their idea is based on the fact that the number of $MUPS$s of any string $\mathcal{S}$ is at most m[11]. They first compute the palindromic substrings whose center is the same as the center for some run in the string. These palindromic substrings are called run-centered palindromes. In order to compute these substrings, they utilize Manacher’s algorithm [43]. This algorithm scans the string and constructs an array $MaxPal$ of length $2n+1$ in which the ith entry is the length of the maximal palindrome with center i. Using this technique, they compute all the $RLE$-maximal palindromes of $\mathcal{S}$ in $\mathcal{O}\left(m\right)$ time and space.

In order to improve the space complexity, they build a data structure which is a modified eertree of the input string to deal with the run-length encoded string instead of the original one. They prove that this new data structure, RLE-eertree ($e^{2}rtr{e}^2$), has $2m+1$ nodes which is linear to the length of the $RLE\left(\mathcal{S}\right)$. Thus, it takes less space complexity compared with the original eertree. The construction time for $e^{2}rtr{e}^2$ is $\mathcal{O}(mlog{\sigma }_{RLE})$. They used this tree to compute all the $MUSPS$s of $\mathcal{S}$. Using the list of $MUSPS$s, they present their algorithm for $\mathsf{SUPS}$ queries. Their technique is similar to Inoue et al.’s work [4] with the difference that the space complexity should be $\mathcal{O}\left(m\right)$.

In 2020, Watanabe et al. [12] published an extension of their work by considering a variant of $\mathsf{SUPS}$ problem where a query interval is also given in a run-length encoded representation. Their technique is similar to their previous approach [11]. They used combinatorial properties of maximal palindromes and presented an $\mathcal{O}\left(m\right)$ space data structure to answer queries in $\mathcal{O}(loglogm+occ)$ time [12].

7. Range-SUS

Range queries are a classic data structure topic, which has a great motivation in string processing problems [5,24,25,26]. In 2019, Abedin et al. [5] studied the local shortest unique substring which starts in a specific region or range of the string. Given a range $[\alpha ,\beta ]$, the problem is to return a shortest substring $\mathcal{S}[k,k+h-1]$ of $\mathcal{S}$ with exactly one occurrence in $[\alpha ,\beta ]$; i.e., $k\in [\alpha ,\beta ]$, there is no $k^{\prime }\in [\alpha ,\beta ]$ such that $\mathcal{S}[k,k+h-1]=\mathcal{S}[k^{\prime },k^{\prime }+h-1]$, and h is minimal. Abedin et al. presented an $\mathcal{O}(nlogn)$-word data structure which answers $\mathsf{rSUS}$ queries in $\mathcal{O}\left({log}_{w}n\right)$ time per query in the word RAM model, where $w=\mathsf{\Omega }(logn)$ is the word size [5]. The Range Shortest Unique Substring problem is formally defined as follows:

Problem 6.

$\mathsf{rSUS}$ Queries

Preprocess: String $\mathcal{S}[1,n]$.

Query: Range $[\alpha ,\beta ]$, where $1\le \alpha \le \beta \le n$.

Output: $(p,\ell )$ such that $\mathcal{S}[p,p+\ell -1]$ is a shortest string with exactly one occurrence in $[\alpha ,\beta ]$.

If $\alpha =\beta$, the answer $(\alpha ,1)$ is trivial. Thus, in the rest, we assume that $\alpha <\beta$.

Example 5

([5]). Given $\mathcal{S}=\stackrel{1}{\mathbf{c}}\stackrel{2}{\mathbf{a}}\stackrel{3}{\mathbf{a}}\stackrel{4}{\mathbf{b}}\stackrel{5}{\mathbf{c}}\stackrel{6}{\mathbf{a}}\stackrel{7}{\mathbf{d}}\stackrel{8}{\mathbf{d}}\stackrel{9}{\mathbf{a}}\stackrel{10}{\mathbf{a}}\stackrel{11}{\mathbf{c}}\stackrel{12}{\mathbf{a}}\stackrel{13}{\mathbf{d}}\stackrel{14}{\mathbf{d}}\stackrel{15}{\mathbf{a}}\stackrel{16}{\mathbf{a}}\stackrel{17}{\mathbf{a}}\stackrel{18}{\mathbf{a}}\stackrel{19}{\mathbf{b}}\stackrel{20}{\mathbf{a}}\stackrel{21}{\mathbf{c}}$ and a query $[\alpha ,\beta ]=[5,16]$, we need to find a shortest substring of $\mathcal{S}$ with exactly one occurrence in $[5,16]$. The output here is $(p,\ell )=(10,2)$ since $\mathcal{S}[10,11]=\mathbf{ac}$ is the shortest substring of $\mathcal{S}$ with exactly one occurrence in $[5,16]$.

For each position $k\in [1,n]$, Abedin et al. [5] provide a data structure to keep track of the last and next occurrence of substring $\mathcal{S}[k,k+h-1]$, denoted by $\mathsf{Prev}(k,h)$ and $\mathsf{Next}(k,h)$, respectively. They define $\lambda (a,b,k)$ and $C_k$ as follows:

$$\lambda (a,b,k)=min\{h\mid \mathsf{Prev}(k,h)<a\mathbf{and}\mathsf{Next}(k,h)>b\}.$$

$$C_k=\{h\mid (\mathsf{Next}(k,h),\mathsf{Prev}(k,h))\ne (\mathsf{Next}(k,h-1),\mathsf{Prev}(k,h-1))\}.$$

Their main result is providing an upper bound on the size of all $C_k$s, which is given in the following lemma.

Lemma 6

([5]). ${\sum }_k\left|C_k\right|=\mathcal{O}(nlogn)$

By the definition, $\lambda (a,b,k)$ is the length of the shortest substring that starts at position k, and $C_k$ is the set of candidate length for the $\mathsf{RSUS}$ answer, which starts at position k.

Abedin et al. solved $\mathsf{rSUS}$ queries by reducing the problem to an top-1 rectangle stabbing query on a set of rectangles with input point $(\alpha ,\beta )$. Assume that the answer for Problem 6 is $(p,\ell )$. Given a query range $[\alpha ,\beta ]$, the answer $(p,\ell )$ we are looking for is the pair $(k,h)$ with the minimum h under the following conditions: $k\in [\alpha ,\beta ]$, $h\in C_k$, $\mathsf{Prev}(k,h)<\alpha$ and $\mathsf{Next}(k,h)>\beta$. Equivalently, $(p,\ell )$ is the pair $(k,h)$ with the minimum h, such that $h\in C_k$, $\alpha \in \left(\mathsf{Prev}\right(k,h),k]$, and $\beta \in [k,\mathsf{Next}(k,h\left)\right)$.

In the preprocessing step, they map each $h\in C_k$ into a weighted rectangle $R_{k,h}$ with weight h and defined as follows:

$$R_{k,h}=[\mathsf{Prev}(k,h)+1,k]\times [k,\mathsf{Next}(k,h)-1].$$

After query $[\alpha ,\beta ]$ comes, the lowest weighted rectangle which stabbed by the point $(\alpha ,\beta )$ is $R_{p,\ell }$. By combining the optimal data structure for top-1 rectangle stabbing presented by Chan et al. [44] and the bound on the number of candidate lengths in Lemma 6, they prove the following result.

Theorem 12.

There exists an $\mathcal{O}(nlogn)$-word data structure which answers $\mathsf{rSUS}$ queries in $\mathcal{O}\left({log}_{w}n\right)$ time per query in the word RAM model, where $w=\mathsf{\Omega }(logn)$ is the word size.

8. Discussion and Future Work

In this paper, we reviewed several types of shortest unique substring queries and their corresponding solutions. All the discussed problems in this manuscript are highly motivated topics in string processing and computational biology research areas. Although we discussed more than 10 approaches for variants of $\mathsf{SUS}$ queries, there still exist related topics which have not been studied, or there is no efficient algorithm to solve them. In this section, we discuss some of such topics and open questions for future work:

• We discussed all the solutions to solve approximate $\mathsf{SUS}$ queries in Section 5. However, there is no efficient in-place algorithm which can find $\mathsf{LSUS}$s to get $\mathsf{SUS}$s afterward. Another technique that can be applied to solve approximate $\mathsf{SUS}$ queries is considering the $RLE$ representation of the input string. Section 4.2 shows this technique for solving interval-$\mathsf{SUS}$ queries. To our knowledge, an $RLE$ based approach for solving approximate $\mathsf{SUS}$ queries has not been studied. In addition, there is no work considering the standard external memory model for solving an approximate $\mathsf{SUS}$ problem. As the I/O-efficient construction of the suffix array and $\mathsf{lcp}$ array exist [45,46,47,48], it seems to be possible to change the RAM model algorithm for the construction of these arrays to the external memory model.
• In Section 4.2, the ${\pi }_q(N,m)$ in the query time of Theorem 6 is $\sqrt{logm/loglogm}$, which is actually the time for performing dynamic predecessor/successor queries using $\mathcal{O}\left(\right|RLE\left(\mathcal{S}\right))$ space [8]. In order to make the query time faster using the same space, the question is if there exists a data structure of size $\mathcal{O}\left(\right|RLE\left(\mathcal{S}\right)\left|\right)$ that can efficiently answer Problem 3 without using predecessor/successor.
• As we discussed in Section 6, palindromic substrings have great motivations in computational biology. All the reviewed works are on finding the exact $\mathsf{SUPS}$s. Similar to the approximate $\mathsf{SUS}$ problem, approximate $\mathsf{SUPS}$ query is also important to be studied for considering errors and mutations. Besides the definition of Problem 5, the following definition has a great motivation in bioinformatics. A nucleotide sequence is considered as a palindrome if the reverse of its complementary strand is equal to the original sequence [49]. The question is if the methods discussed in Section 6 can be applied to efficiently solve this problem.
• The last topic that we discussed was the $\mathsf{rSUS}$ problem. According to Theorem 12, $\mathsf{rSUS}$ queries can be solved in $\mathcal{O}\left({log}_{\omega }n\right)$ time using a data structure of size $\mathcal{O}(nlogn)$ word. The question is whether we can design an efficient $\mathcal{O}\left(n\right)$-word data structure for the $\mathsf{rSUS}$ problem. In addition, the approximate version of $\mathsf{rSUS}$ queries has not been studied. It is possible to combine the technique discussed in Section 7 and the framework of Thankachan et al. [50] to provide an efficient algorithm for approximate $\mathsf{rSUS}$ problem.
• Besides shortest unique substrings, Maximal Unique Matches is an important concept in computational biology for aligning two long genome sequences [51]. Ganguly et al. [18] applied a similar technique discussed in Section 3.4 to find maximal unique matches of two strings. As far as we are aware, the dynamic version (when mismatches are allowed) of this problem has not been studied yet. We believe that, by modifying the techniques on the dynamic longest common substring problem ($\mathsf{LCS}$ after k mismatches) [52,53,54], the approximate Maximal Unique Matches problem can be solved in subquadratic time.