Computing Maximal Lyndon Substrings of a String

Abstract

There are two reasons to have an efficient algorithm for identifying all right-maximal Lyndon substrings of a string: firstly, Bannai et al. introduced in 2015 a linear algorithm to compute all runs of a string that relies on knowing all right-maximal Lyndon substrings of the input string, and secondly, Franek et al. showed in 2017 a linear equivalence of sorting suffixes and sorting right-maximal Lyndon substrings of a string, inspired by a novel suffix sorting algorithm of Baier. In 2016, Franek et al. presented a brief overview of algorithms for computing the Lyndon array that encodes the knowledge of right-maximal Lyndon substrings of the input string. Among those presented were two well-known algorithms for computing the Lyndon array: a quadratic in-place algorithm based on the iterated Duval algorithm for Lyndon factorization and a linear algorithmic scheme based on linear suffix sorting, computing the inverse suffix array, and applying to it the next smaller value algorithm. Duval’s algorithm works for strings over any ordered alphabet, while for linear suffix sorting, a constant or an integer alphabet is required. The authors at that time were not aware of Baier’s algorithm. In 2017, our research group proposed a novel algorithm for the Lyndon array. Though the proposed algorithm is linear in the average case and has $O(nlog(n\left)\right)$ worst-case complexity, it is interesting as it emulates the fast Fourier algorithm’s recursive approach and introduces $\tau$-reduction, which might be of independent interest. In 2018, we presented a linear algorithm to compute the Lyndon array of a string inspired by Phase I of Baier’s algorithm for suffix sorting. This paper presents the theoretical analysis of these two algorithms and provides empirical comparisons of both of their C++ implementations with respect to the iterated Duval algorithm.

1. Introduction

In combinatorics on words, Lyndon words play a very important role. Lyndon words, a special case of Hall words, were named after Roger Lyndon, who was looking for a suitable description of the generators of free Lie algebras [1]. Despite their humble beginnings, to date, Lyndon words have facilitated many applications in mathematics and computer science, some of which are: constructing de Bruijin sequences, constructing bases in free Lie algebras, finding the lexicographically smallest or largest substring in a string, and succinct prefix matching of highly periodic strings; see Marcus et al. [2], and the informative paper by Berstel and Perrin [3] and the references therein.

The pioneering work on Lyndon decomposition was already introduced by Chen, Fox, and Lyndon in [4]. The Lyndon decomposition theorem is not explicitly stated there; nevertheless, it follows from the work presented there.

Theorem 1

(Lyndon decomposition theorem, Chen+Fox+Lyndon, 1958). For any word $\mathit{x}$, there are unique Lyndon words $u_1,\dots ,u_k$ so that $u_{i+1}\prec u_i$ for any $1\le i<k$, and $x=u_1{u}_2\cdots u_k$, where ≺ denotes the lexicographic ordering.

As there exists a bijection between Lyndon words over an alphabet of cardinality k and irreducible polynomials over ${\mathit{F}}_{\mathit{k}}$ [5], many results are known about this factorization: the average number of factors, the average length of the longest factor [6] and of the shortest [7]. Several algorithms deal with Lyndon factorization. Duval gave in [8] an elegant algorithm that computes, in linear time and in-place, the factorization of a word into Lyndon words. More about its implementation can be found in [9]. In [10], Fredricksen and Maiorana presented an algorithm generating all Lyndon words up to a given length in lexicographical order. This algorithm runs in a constant average time.

Two of the latest applications of Lyndon words are due to Bannai et al. In [11], they employed Lyndon roots of runs to prove the runs conjecture that the number of runs in a string is bounded by the length of the string. In the same paper, they presented an algorithm to compute all the runs in a string in linear time that requires the knowledge of all right-maximal Lyndon substrings of the input string with respect to an order of the alphabet and its inverse. The latter result was the major reason for our interest in computing the right-maximal Lyndon substrings of a given string. Though the terms word and string are interchangeable, and so are the terms subword and substring, in the following, we prefer to use exclusively the terms string and substring to avoid confusing the reader.

There are at least two reasons for having an efficient algorithm for identifying all right-maximal Lyndon substrings of a string: firstly, Bannai et al. published in 2017 [11] a linear algorithm to compute all runs in a string that depends on knowing all right-maximal Lyndon substrings of the input string, and secondly, in 2017, Franek et al. in [12] showed a linear equivalence of sorting suffixes and sorting right-maximal Lyndon substrings, inspired by Phase II of a suffix sorting algorithm introduced by Baier in 2015 (Master’s thesis [13]) and published in 2016 [14].

The most significant feature of the runs algorithm presented in [11] is that it relies on knowing the right-maximal Lyndon substrings of the input string for some order of the alphabet and for the inverse of that order, while all other linear algorithms for runs rely on Lempel–Ziv factorization of the input string. It also raised the issue about which approach may be more efficient: to compute the Lempel–Ziv factorization or to compute all right-maximal Lyndon substrings. There are several efficient linear algorithms for Lempel–Ziv factorization (e.g., see [15,16] and the references therein).

Interestingly, Kosolobov [17] showed that for a general alphabet, in the decision tree model, the runs problem is easier than the Lempel–Ziv decomposition. His result supports the conjecture that there must be a linear random access memory model algorithm finding all runs.

Baier introduced in [13], and published in [14], a new algorithm for suffix sorting. Though Lyndon strings were never mentioned in [13,14], it was noticed by Cristoph Diegelmann in a personal communication [18] that Phase I of Baier’s suffix sort identifies and sorts all right-maximal Lyndon substrings.

The right-maximal Lyndon substrings of a string $x=x[1..n]$ can be best encoded in the so-called Lyndon array, introduced in [19] and closely related to the Lyndon tree of [20]: an integer array $\mathcal{L}[1..n]$ so that for any $i\in 1..n$, $\mathcal{L}\left[i\right]=$ the length of the right-maximal Lyndon substring starting at the position i.

In an overview [19], Franek et al. discussed an algorithm based on an iterative application of Duval’s Lyndon factorization algorithm [8], which we refer to here as IDLA, and an algorithmic scheme based on Hohlweg and Reutenauer’s work [20], which we refer to as SSLA. The authors were not aware of Baier’s algorithm at that time. Two additional algorithms were presented there, a quadratic recursive application of Duval’s algorithm and an algorithm NSV* with possibly $\mathcal{O}(nlog(n\left)\right)$ worst-case complexity based on ranges that can be compared in constant time for constant alphabets. The correctness of NSV* and its complexity were discussed there just informally.

The algorithm IDLA (see Figure 1) is simple and in-place, so no additional space is required except for the storage for the string and the Lyndon array. It is completely independent of the alphabet of the string and does not require the alphabet to be sorted; all it requires is that the alphabet be ordered, i.e., only pairwise comparisons of the alphabet symbols are needed. Its weakness is its quadratic worst-case complexity, which becomes a problem for longer strings with long right-maximal Lyndon substrings, as one of our experiments showed (see Figure 11 in Section 7).

In our empirical work, we used IDLA as a control for comparison and as a verifier of the results. Note that the reason the procedure MaxLyn of Figure 1 really computes the longest Lyndon prefix is not obvious and is based on the properties of periods of prefixes; see [8] or Observation 6 and Lemma 11 in [19].

Lemma 1, below, characterizes right-maximal Lyndon substrings in terms of the relationships of the suffixes and follows from the work of Hohlweg and Reutenauer [20]. Though the definition of the proto-Lyndon substring is formally given in Section 2 below, it suffices to say the it means that it is a prefix of a Lyndon substring of the string. The definition of the lexicographic ordering ≺ is also given Section 2. The proof of this lemma is delayed to the end of Section 2, where all the technical terms needed are defined.

Lemma 1.

Consider a string $x[1..n]$ over an alphabet ordered by ≺.

A substring $x[i..j]$ is proto-Lyndon if and only if:

(a) 

$x[i..n]\prec x[k..n]$ for any $i<k\le j$.

A substring $x[i..j]$ is right-maximal Lyndon if and only if:

(b) 

$x[i..n]$ is proto-Lyndon and

(c) 

either $j=n$ or $x[j+1..n]\prec x[i..n]$.

Thus, the Lyndon array is an NSV (Next Smaller Value) array of the inverse suffix array. Consequently, the Lyndon array can be computed by sorting the suffixes, i.e., computing the suffix array, then computing the inverse suffix array, and then applying NSV to it; see [19]. Computing the inverse suffix array and applying NSV are “naturally” linear, and computing the suffix array can be implemented to be linear; see [19,21] and the references therein. The execution and space characteristics are dominated by those of the first step, i.e., computation of the suffix array. We refer to this scheme as SSLA.

In 2018, a linear algorithm to compute the Lyndon array from a given Burrows–Wheeler transform was presented [22]. Since the Burrows–Wheeler transform is computed in linear time from the suffix array, it is yet another scheme of how to obtain the Lyndon array via suffix sorting: compute the suffix array; from the suffix array, compute the Burrows–Wheeler transform, then compute the Lyndon array during the inversion of the Burrows–Wheeler transform. We refer to this scheme as BWLA.

The introduction of Baier’s suffix sort in 2015 and the consequent realization of the connection to right-maximal Lyndon substrings brought up the realization that there was an elementary (not relying on a pre-processed global data structure such as a suffix array or a Burrows–Wheeler transform) algorithm to compute the Lyndon array, and that, despite its original clumsiness, could be eventually refined to outperform any SSLA or BWLA implementation: any implementation of a suffix sorting-based scheme requires a full suffix sort and then some additional processing, while Baier’s approach is “just” a partial suffix sort; see [23].

In this work, we present two additional algorithms for the Lyndon array not discussed in [19]. The C++ source code of the three implementations IDLA, TRLA, and BSLA is available; see [24]. Note that the procedure IDLA is in the lynarr.hpp file.

The first algorithm presented here is TRLA. TRLA is a $\tau$-reduction based Lyndon array algorithm that follows Farach’s approach used in his remarkable linear algorithm for suffix tree construction [25] and reproduced very successfully in all linear algorithms for suffix sorting (e.g., see [21,26] and the references therein). Farach’s approach follows the Cooley–Tukey algorithm for the fast Fourier transform relying on recursion to lower the quadratic complexity to $O(nlog(n\left)\right)$ complexity; see [27]. TRLA was first introduced by the authors in 2019 (see [28,29]) and presented as a part of Liut’s Ph.D. thesis [30].

The second algorithm, BSLA, is a Baier’s sort-based Lyndon array algorithm. BSLA is based on the idea of Phase I of Baier’s suffix sort, though our implementation necessarily differs from Baier’s. BSLA was first introduced at the Prague Stringology Conference 2018 [23] and also presented as a part of Liut’s Ph.D. thesis [30] in 2019; here, we present a complete and refined theoretical analysis of the algorithm and a more efficient implementation than that initially introduced.

The paper is structured as follows: In Section 2, the basic notions and terminology are presented. In Section 3, the TRLA algorithm is presented and analysed. In Section 4, the BSLA algorithm is presented and analysed. In Section 5, the datasets with random strings of various lengths and over various alphabets and other datasets used in the empirical tests are described. In Section 6, the conclusion of the research and the future work are presented. The results of the empirical measurements of the performance of IDLA, TRLA, and BSLA on those datasets are presented in Section 7 in both tabular and graphical forms.

2. Basic Notation and Terminology

Most of the fundamental notions, definitions, facts, and string algorithms can be found in [31,32,33,34]. For the ease of access, this section includes those that are directly related to the work herein.

The set of integers is denoted by $\mathbb{Z}$. For two integers $i\le j$, the range$i..j=\{k\in \mathbb{Z}|i\le k\le j\}$. An alphabet is a finite or infinite set of symbols, or equivalently called letters. We always assume that the sentinel symbol $ is not in the alphabet and is always assumed to be lexicographically the smallest. A string over an alphabet $\mathcal{A}$ is a finite sequence of symbols from $\mathcal{A}$. A $-terminated string over $\mathcal{A}$ is a string over $\mathcal{A}$ terminated by $. We use the array notation indexing from 1 for strings; thus, $x[1..n]$ indicates a string of length n; the first symbol is the symbol with index 1, i.e., $x\left[1\right]$; the second symbol is the symbol with index 2, i.e., $x\left[2\right]$, etc. Thus, $x[1..n]=x\left[1\right]x\left[2\right]\dots x\left[n\right]$. For a $-terminated string $\mathit{x}$ of length n, $x[n+1]=\$$. The alphabet of string $\mathit{x}$, denoted as ${\mathcal{A}}_x$, is the set of all distinct alphabet symbols occurring in $\mathit{x}$.

We use the term strings over a constant alphabet if the alphabet is a fixed finite alphabet. The integer alphabet is the infinite alphabet $\mathcal{A}=\{0,1,2,\dots \}$. We use the term strings over integer alphabet for the strings over the alphabet $\{0,1,2,\dots \}$ with an additional constraint that all letters occurring in the string are all smaller than the length of the string, i.e., in this paper, $x[1..n]$ is a string over the integer alphabet if it is a string over the alphabet $\{0,1,\dots n-1\}$. Many authors use a more general definition; for instance, Burkhardt and Kärkkäinen [35] defined it as any set of integers of size $n^{o\left(1\right)}$; however, our results can easily be adapted to such more general definitions without changing their essence.

We use a bold font to denote strings; thus, x denotes a string, while x denotes some other mathematical entity such as an integer. The empty string is denoted by $\mathit{\epsilon }$ and has length zero. The length or size of string $x=x[1..n]$ is n. The length of a string $\mathit{x}$ is denoted by $\left|x\right|$. For two strings $x=x[1..n]$ and $y=y[1..m]$, the concatenation $xy$ is a string $\mathit{u}$

where $u\left[i\right]=\left\{\begin{array}{c}x\left[i\right]fori\le n,\hfill \\ y[i-n]forn<i\le n+m.\hfill \end{array}\right.$

If $x=uvw$, then $\mathit{u}$ is a prefix, $\mathit{v}$ a substring, and $\mathit{w}$ a suffix of $\mathit{x}$. If $\mathit{u}$ (respectively, $\mathit{v}$, $\mathit{w}$) is empty, then it is called an empty prefix (respectively, empty substring, empty suffix); if $\left|u\right|<\left|x\right|$ (respectively, $\left|v\right|<\left|x\right|$, $\left|w\right|<\left|x\right|$), then it is called a proper prefix (respectively, proper substring, proper suffix). If $x=uv$, then $\mathit{vu}$ is called a rotation or a conjugate of $\mathit{x}$; if either $u=\mathit{\epsilon }$ or $v=\mathit{\epsilon }$, then the rotation is called trivial. A non-empty string $\mathit{x}$ is primitive if there is no string $\mathit{y}$ and no integer $k\ge 2$ so that $x=y^k=\underset{ktimes}{\underbrace{yy\cdots y}}$.

A non-empty string $\mathit{x}$ has a non-trivial border $\mathit{u}$ if $\mathit{u}$ is both a non-empty proper prefix and a non-empty proper suffix of $\mathit{x}$. Thus, both $\mathit{\epsilon }$ and $\mathit{x}$ are trivial borders of $\mathit{x}$. A string without a non-trivial border is call unbordered.

Let ≺ be a total order of an alphabet $\mathcal{A}$. The order is extended to all finite strings over the alphabet $\mathcal{A}$: for $x=x[1..n]$ and $y=y[1..n]$, $x\prec y$ if either $\mathit{x}$ is a proper prefix of $\mathit{y}$ or there is a $j\le min\{n,m\}$ so that $x\left[1\right]=y\left[1\right]$, ..., $x[j-1]=y[j-1]$ and $x\left[j\right]\prec y\left[j\right]$. This total order induced by the order of the alphabet is called a lexicographic order of all non-empty strings over $\mathcal{A}$. We denote by $x⪯y$ if either $x\prec y$ or $x=y$. A string $\mathit{x}$ over $\mathcal{A}$ is Lyndon for a given order ≺ of $\mathcal{A}$ if $\mathit{x}$ is strictly lexicographically smaller than any non-trivial rotation of $\mathit{x}$. In particular:

x is Lyndon ⇒x is unbordered ⇒ x is primitive

Note that the reverse implications do not hold: $aba$ is primitive but neither unbordered, nor Lyndon, while $acaab$ is unbordered, but not Lyndon. A substring $x[i..j]$ of $x[1..n]$, $1\le i\le j\le n$ is a right-maximal Lyndon substring of $\mathit{x}$ if it is Lyndon and either $j=n$ or for any $k>j$, $x[i..k]$ is not Lyndon.

A substring $x[i..j]$ of a string $x[1..n]$ is proto-Lyndon if there is a $j\le k\le n$ so that $x[i..k]$ is Lyndon. The Lyndon array of a string $x=x[1..n]$ is an integer array $\mathcal{L}[1..n]$ so that $\mathcal{L}\left[i\right]=j$ where $j\le n-i$ is a maximal integer such that $x[i..i+j-1]$ is Lyndon. Alternatively, we can define it as an integer array ${\mathcal{L}}^{\prime }[1..n]$ so that ${\mathcal{L}}^{\prime }\left[i\right]=j$ where j is the last position of the right-maximal Lyndon substring starting at the position i. The relationship between those two definitions is straightforward: ${\mathcal{L}}^{\prime }\left[i\right]=\mathcal{L}\left[i\right]+i-1$ or $\mathcal{L}\left[i\right]={\mathcal{L}}^{\prime }\left[i\right]-i+1$.

Proof ofLemma 1.

We first prove the following claim:

Claim. Substring $x[i..j]$ is right-maximal Lyndon if and only if:

$\left(b^{\prime }\right)$

$x[i..n]\prec x[k..n]$ for any $i<k\le j$ and

$\left(c\right)$

either $j=n$ or $x[j+1..n]\prec x[i..n]$.

Let $x[i..j]$ be right-maximal Lyndon. We need to show that $\left(b^{\prime }\right)$ and $\left(c\right)$ hold.

Let $i<k\le j$. Since it is Lyndon, $x[i..j]\prec x[k..j]$. Thus, there is $0\le r$ so that $i+r\le j$ and $j+r\le j$ and $x[i+\ell ]=x[j+\ell ]$ for any $0\le \ell <r$ and $x[i+r]\prec x[j+r]$. It follows that $x[i..n]\prec x[j..n]$, and $\left(b^{\prime }\right)$ is satisfied.

If $j=n$, then $\left(c\right)$ holds. Therefore, let us assume that $j<n$.

(1)

If $x\left[i\right]\prec x[j+1]$, then $x[i..j+1]\prec x[j+1..j+1]$; together with $x[i..j]\prec x[k..j]$ for any $i<k\le j$, it shows that $x[i..j+1]$ is Lyndon, contradicting the right-maximality of $x[i..j]$.

(2)

If $x\left[i\right]\succ x[j+1]$, then $x[i..n]\succ x[j+1..n]$, and $\left(c\right)$ holds.

(3)

If $x\left[i\right]=x[j+1]$, then there are a prefix $\mathit{u}$, strings $\mathit{v}$ and $\mathit{w}$, and an integer $r\ge 1$ so that $uv=x[i..j]$, $x[i..n]=uv{u}^{r}w$, and $x[j+1..n]=u^{r}w$. Let us take a maximal such $\mathit{u}$ and a maximal such r.

(3a)

Let $r\ge 2$. Then, since $uv=x[i..j]$ is Lyndon, $u\prec v$, and so, $uu\prec uv$; hence, $u^{r}w\prec uv{u}^{r}w$, and $\left(c\right)$ holds.

(3b)

Let $r=1$, i.e., $x[i..n]=uvuw$ and $x[j+1..n]=uw$. There is no common prefix of $\mathit{v}$ and $\mathit{w}$ as it would contradict the maximality of $\mathit{u}$. There are thus two mutually exclusive cases, either $v\left[1\right]\prec w\left[1\right]$ or $v\left[1\right]\succ w\left[1\right]$. Let $v\left[1\right]=c_1$ and $w\left[1\right]=c_2$. If $c_1\prec c_2$, then $u{c}_1\prec u{c}_2$, and so, $uv\prec u{c}_2$. For any $u=u_1{u}_2$, $uv\prec u_{2}v$ as $\mathit{uv}$ is Lyndon, so $uv\prec u_2{c}_2$, giving $uv{c}_2$ to be Lyndon, a contradiction with the right-maximality of $uv$. Therefore, $c_1\succ c_2$; thus, $uw\prec uvuw$, and $\left(c\right)$ holds.

Now, we go in the opposite direction. Assuming $\left(b^{\prime }\right)$ and $\left(c\right)$, we need to show that $x[i..j]$ is right-maximal Lyndon.

Consider the right-maximal substring of $\mathit{x}$ starting at the position i; it is $x[i..\ell ]$ for some $i\le \ell \le n$.

(1)

If $\ell <j$, then by the first part of this proof, $x[i..n]\succ x[\ell +1..n]$, contradicting the assumption $\left(b^{\prime }\right)$ as $\ell +1\le j$.

(2)

If $j<\ell$, then $j+1\le \ell$, and by the first part of this proof, $x[i..n]\prec x[j+1..n]$, contradicting the assumption $\left(c\right)$ .

(3)

If $\ell =j$, then $x[i..j]$ is right-maximal Lyndon.

Now, we can prove $\left(a\right)$. Let $x[i..j]$ be a proto-Lyndon substring of $\mathit{x}$. By definition, it is a prefix of a Lyndon substring of $\mathit{x}$ and, hence, a prefix of a right-maximal Lyndon substring of $\mathit{x}$, say $x[i..\ell ]$ for some $j\le \ell \le n$. It follows from the claim that $x[i..n]\prec x[k..n]$ for any $i<k\le \ell$. Since $j\le \ell$, $\left(a\right)$ holds. For the opposite direction, if $\left(a\right)$ holds, then there are two possibilities: either there is $j<\ell \le n$ so that $x[i..n]\succ x[\ell ..n]$ or $x[i..n]\prec x[k..n]$ for any $i<k\le n$. By the claim, in the former case, $x[i..\ell -1]$ is a right-maximal Lyndon substring of $\mathit{x}$, while in the latter case, $x[i..n]$ is a right-maximal Lyndon substring of $\mathit{x}$ . Thus, in both cases, $x[i..j]$ is a prefix of a Lyndon substring of $\mathit{x}$.

With $\left(a\right)$ proven, we can now replace $\left(b^{\prime }\right)$ in the claim with $\left(b\right)$, completing the proof. □

3. $\mathit{\tau }$-Reduction Algorithm (TRLA)

The purpose of this section is to introduce a recursive algorithm TRLA for computing the Lyndon array of a string. As will be shown below, the most significant aspect is the so-called $\tau$-reduction of a string and how the Lyndon array of the $\tau$-reduced string can be expanded to a partially filled Lyndon array for the whole string, as well as how to compute the missing values. This section thus provides the mathematical justification for the algorithm and, in so doing, proves the correctness of the algorithm. The mathematical understanding of the algorithm provides the bases for the bounding of its worst-case complexity by $O(nlog(n\left)\right)$ and determining the linearity of the average-case complexity.

The first idea of the algorithm was proposed in Paracha’s 2017 Ph.D. thesis [36]. It follows Farach’s approach [25]:

(1)

reduce the input string $\mathit{x}$ to $\mathit{y}$;

(2)

by recursion, compute the Lyndon array of $\mathit{y}$; and

(3)

from the Lyndon array of $\mathit{y}$, compute the Lyndon array of $\mathit{x}$.

The input strings for the algorithm are $-terminated strings over an integer alphabet. The reduction computed in (1) is important. All linear algorithms for suffix array computations use the proximity property of suffixes: comparing $x[i..n]$ and $x[j..n]$ can be done by comparing $x\left[i\right]$ and $x\left[j\right]$ and, if they are the same, comparing $x[i+1..n]$ with $x[j+1..n]$. For instance, in the first linear algorithm for the suffix array by Kärkkäinen and Sanders [37], obtaining the sorted suffixes for positions $i\equiv 0\left(mod3\right)$ and $i\equiv 1\left(mod3\right)$ via the recursive call is sufficient to determine the order of suffixes for the $i\equiv 2\left(mod3\right)$ positions, then merging both lists together. However, there is no such proximity property for right-maximal Lyndon substrings, so the reduction itself must have a property that helps determine some of the values of the Lyndon array of $\mathit{x}$ from the Lyndon array of $\mathit{y}$ and computing the rest.

In our algorithm, we use a special reduction, which we call $\tau$-reduction, defined in Section 3.2, that reduces the original string to at most $\frac{1}{2}$ and at least $\frac{2}{3}$ of its length. The algorithm computes $\mathit{y}$ as a $\tau$-reduction of the input string $\mathit{x}$ in Step (1) in linear time. In Step (3), it expands the Lyndon array of the reduced string computed by Step (2) to an incomplete Lyndon array of the original string also in linear time. The incomplete Lyndon array computed in (3) is about $\frac{1}{2}$ to $\frac{2}{3}$ full, and for every position i with an unknown value, the values at positions $i-1$ and $i+1$ are known. In particular, the values at Position 1 and position n are both known. Therefore, much information is provided by the recursive Step (2). For instance, for 00011001, via the recursive call, we would identify the right-maximal Lyndon substrings that are underlined in $\underline{00\underline{01\underline{1}}\underline{00\underline{1}}}$ and would need to compute the missing right-maximal Lyndon substrings that are underlined in $0\underline{00\underline{1}1}0\underline{01}$.

However, computing the missing values of the incomplete Lyndon array takes at most $O(nlog(n\left)\right)$ steps, as we will show, resulting in the overall worst-case complexity of $O(nlog(n\left)\right)$. When the input string is such that the missing values of the incomplete Lyndon array of the input string can be computed in linear time, the overall execution of the algorithm is linear as well, and thus, the average case complexity will be shown to be linear in the length of the input string.

In the following subsections, we describe the $\tau$-reduction in several steps: first, the $\tau$-pairing, then choosing the $\tau$-alphabet, and finally, the computation of the $\tau \left(x\right)$. The $\tau$-reduction may be of some general interest as it preserves (see Lemma 6) some right-maximal Lyndon substrings of the original string.

3.1. $\mathit{\tau }$-Pairing

Consider a $-terminated string $x=x[1..n]$ whose alphabet ${\mathcal{A}}_x$ is ordered by ≺ where $x[n+1]=\$$ and $\$\prec a$ for any $a\in {\mathcal{A}}_x$. A $\mathit{\tau }$-pair consists of a pair of adjacent positions from the range $1..n+1$. The $\tau$-pairs are computed by induction:

• the initial $\tau$-pair is $(1,2)$;
• if $(i-1,i)$ is the last $\tau$-pair computed, then:
     if $i=n-1$ then
        the next $\tau$-pair is set to $(n,n+1)$
        stop
     elseif $i\ge n$ then
        stop
     elseif $x[i-1]\succ x\left[i\right]$ and $x\left[i\right]⪯x[i+1]$ then
        the next $\tau$-pair is set to $(i,i+1)$; repeat 2.
     else
        the next $\tau$-pair is set to $(i+1,i+2)$; repeat 2.

Every position of the input string that occurs in some $\tau$-pair as the first element is labelled black; all others are labelled white. Note that Position 1 is always black, while the last position n can be either black or white; however, the positions $n-1$ and n cannot be simultaneously both black. Note also that most of the $\tau$-pairs do not overlap; if two $\tau$-pairs overlap, they overlap in a position i such that $1<i<n$ and $x[i-1]\succ x\left[i\right]$ and $x\left[i\right]⪯x[i+1]$. The first position and the last position never figure in an overlap of $\tau$-pairs. Moreover, a $\tau$-pair can be involved in at most one overlap; for an illustration, see Figure 2; for the formal proof see Lemma 2.

Lemma 2.

Let $(i_1,i_1+1)\dots (i_k,i_k+1)$ be the τpairs of a string $x=x[1..n]$. Then, for any $j,\ell \in 1..k$:

(1) 

$if|(i_j,i_j+1)\cap (i_{\ell },i_{\ell }+1)|=1,thenforanym\ne j,\ell ,|(i_j,i_j+1)\cap (i_m,i_m+1)|=0$,

(2) 

$|(i_j,i_j+1)\cap (i_{\ell },i_{\ell }+1)|\le 1$.

Proof. 

This is by induction; trivially true for $\left|x\right|=1$ as $(1,2)$ is the only $\tau$-pair. Assume it is true for $\left|x\right|\le n-1$.

• Case $(i_k,i_k+1)=(n,n+1)$:
  Then, $(i_{k-1},i_{k-1}+1)=(n-2,n-1)$, and so, $(i_1,i_1+1)\dots (i_{k-1},i_{k-1}+1)$ are $\tau$-pairs of $x[1..n-1]$; thus, they satisfy (1) and (2) by the induction hypothesis. However, $(n,n+1)\cap (i_{\ell },i_{\ell +1})=\varnothing$ for $1\le \ell <k$, so (1) and (2) hold for $(i_1,i_1+1)\dots (i_k,i_k+1)$.
• Cases $(i_k,i_k+1)=(n-1,n)$ and $(i_{k-1},i_{k-1}+1)=(n-2,n-1)$:
  Therefore, $(i_1,i_1+1)\dots (i_{k-1},i_{k-1}+1)$ are $\tau$-pairs of $x[1..n-1]$, and thus, they satisfy (1) and (2) by the induction hypothesis. However, $(i_k,i_k+1)\cap (i_{\ell },i_{\ell }+1)=\varnothing$ for $1\le \ell <k-1$, and $(i_k,i_k+1)\cap (i_{k-1},i_{k-1}+1)=\left\{i_{k-1}\right\}=n-1$; so, $|(i_k,i_k+1)\cap (i_{k-1},i_{k-1}+1)|\le 1$, and so, (1) and (2) hold for $(i_1,i_1+1)\dots (i_k,i_k+1)$.
• Cases $(i_k,i_k+1)=(n-1,n)$ and $(i_{k-1},i_{k-1}+1)=(n-3,n-2)$:
  Then, $(i_1,i_1+1)\dots (i_{k-1},i_{k-1}+1)$ are $\tau$-pairs of $x[1..n-2]$, so they satisfy (1) and (2) by the induction hypothesis. However, $(i_k,i_k+1)\cap (i_{\ell },i_{\ell }+1)=\varnothing$ for $1\le \ell <k$, so (1) and (2) hold for $(i_1,i_1+1)\dots (i_k,i_k+1)$.

□

3.2. $\mathit{\tau }$-Reduction

For each $\tau$-pair $(i,i+1)$, we consider the pair of alphabet symbols $\left(x\right[i],x[i+1\left]\right)$. We call them symbol$\mathit{\tau }$-pairs. They are in a total order ⊲ induced by ≺ : $(x\left[i_j\right],x[i_j+1])⊲(x\left[i_{\ell }\right],x[i_{\ell }+1])$ if either $x\left[i_j\right]\prec x\left[i_{\ell }\right]$, or $x\left[i_j\right]=x\left[i_{\ell }\right]$ and $x[i_j+1]\prec x[i_{\ell }+1]$. They are sorted using the radix sort with a key of size two and assigned letters from a chosen $\tau$-alphabet that is a subset of $\{0,1,\dots ,|\tau \left(x\right)\left|\right\}$ so that the assignment preserves the order. Since the input string is over an integer alphabet, the radix sort is linear.

In the example (Figure 2), the $\tau$-pairs are $(1,2)(3,4)(4,5)(6,7)(7,8)(9,10)$, and so, the symbol $\tau$-pairs are $(0,1)(1,0)(0,2)(3,1)(1,2)(2,\$)$. The sorted symbol $\tau$-pairs are $(0,1)(0,2)(1,0)$$(1,2)(2,\$)(3,1)$. Thus, we chose as our $\tau$-alphabet $\{0,1,2,3,4,5\}$, and so, the symbol $\tau$-pairs are assigned these letters: $(0,1)\to 0$, $(0,2)\to 1$, $(1,0)\to 2$, $(1,2)\to 3$, $(2,\$)\to 4$, and $(3,1)\to 5$. Note that the assignments respect the order ⊲ of the symbols $\tau$-pairs and the natural order < of $\{0,1,2,3,4,5\}$.

The $\tau$-letters are substituted for the symbol $\tau$-pairs, and the resulting string is terminated with $. This string is called the $\mathit{\tau }$-reduction of $\mathit{x}$ and denoted $\mathit{\tau }\left(\mathit{x}\right)$, and it is a $-terminated string over an integer alphabet. For our running example from Figure 2, $\tau \left(x\right)=021534$. The next lemma justifies calling the above transformation a reduction.

Lemma 3.

For any string $\mathit{x}$, $\frac{1}{2}\left|x\right|\le \left|\tau \left(x\right)\right|\le \frac{2}{3}\left|x\right|$.

Proof. 

There are two extreme cases; the first is when all the $\tau$-pairs do not overlap at all, then $\left|\tau \left(x\right)\right|=\frac{1}{2}\left|x\right|$; and the second is when all the $\tau$-pairs overlap, then $\left|\tau \left(x\right)\right|=\frac{2}{3}\left|x\right|$. Any other case must be in between. □

Let $\mathcal{B}\left(x\right)$ denote the set of all black positions of $\mathit{x}$. For any $i\in 1..\left|\tau \right(x\left)\right|$, $b\left(i\right)=j$ where j is a black position in $\mathit{x}$ of the $\tau$-pair corresponding to the new symbol in $\tau \left(x\right)$ at position i, while $t\left(j\right)$ assigns each black position of $\mathit{x}$ the position in $\tau \left(x\right)$ where the corresponding new symbol is, i.e., $b\left(t\right(j\left)\right)=j$ and $t\left(b\right(i\left)\right)=i$. Thus,

$${\displaystyle 1..\left|\tau \right(x\left)\right|\underset{\mathrm{t}}{\overset{\mathrm{b}}{\rightleftarrows }}\mathcal{B}\left(x\right)}$$

In addition, we define p as the mapping of the $\tau$-pairs to the $\tau$-alphabet.

In our running example from Figure 2, $t\left(1\right)=1$, $t\left(3\right)=2$, $t\left(4\right)=3$, $t\left(6\right)=4$, $t\left(7\right)=5$, and $t\left(9\right)=6$, while $b\left(1\right)=1$, $b\left(2\right)=3$, $b\left(3\right)=4$, $b\left(4\right)=6$, $b\left(5\right)=7$, and $b\left(6\right)=9$. For the letter mapping, we get $p(1,2)=0$, $p(3,4)=2$, $p(4,5)=1$, $p(6,7)=5$, $p(7,8)=3$, and $p(9,10)=4$.

3.3. Properties Preserved by $\mathit{\tau }$-Reduction

The most important property of $\tau$-reduction is a preservation of right-maximal Lyndon substrings of $\mathit{x}$ that start at black positions. This means there is a closed formula that gives, for every right-maximal Lyndon substring of $\tau \left(x\right)$, a corresponding right-maximal Lyndon substring of $\mathit{x}$. Moreover, the formula for any black position can be computed in constant time. It is simpler to present the following results using ${\mathcal{L}}^{\prime }$, the alternative form of the Lyndon array, the one where the end positions of right-maximal Lyndon substrings are stored rather than their lengths. More formally:

Theorem 2.

Let $x=x[1..n]$; let ${\mathcal{L}}_{\tau \left(x\right)}^{\prime }[1..m]$ be the Lyndon array of $\tau \left(x\right)$; and let ${\mathcal{L}}_x^{\prime }[1..n]$ be the Lyndon array of $\mathit{x}$.

Then, for any black $i\in 1..n$, ${\mathcal{L}}_x^{\prime }\left[i\right]=\left\{\begin{array}{cc}b\left(r\right)\hfill & \mathrm{if}b\left(r\right)=n\mathrm{or}x\left[b\right(r)+1]⪯x\left[i\right]\hfill \\ b\left(r\right)+1\hfill & \mathit{otherwise},\hfill \end{array}\right.$where $r={\mathcal{L}}_{\tau \left(x\right)}^{\prime }\left[t\left(i\right)\right]$.

The proof of the theorem requires a series of lemmas that are presented below. First, we show that $\tau$-reduction preserves the relationships of certain suffixes of $\mathit{x}$.

Lemma 4.

Let $x=x[1..n]$, and let $\tau \left(x\right)=\tau \left(x\right)[1..m]$. Let $i\ne j$ and $1\le i,j\le n$. If i and j are both black positions, then $x[i..n]\prec x[j..n]$ implies $\tau \left(x\right)\left[t\right(i)..m]\prec \tau \left(x\right)\left[t\right(j)..m]$.

Proof. 

Since i and j are both black positions, both $t\left(i\right)$ and $t\left(j\right)$ are defined, and $t\left(i\right)\ne t\left(j\right)$. Let us assume that $x[i..n]\prec x[j..n]$. The proof is argued in several cases determined by the nature of the relationship $x[i..n]\prec x[j..n]$.

(1)

Case: $x[i..n]$ is a proper prefix of $x[j..n]$.

Then, $\left|x\right[i..n\left]\right|=n-i+1<\left|x\right[j..n\left]\right|=n-j+1$, and so, $j<i$. It follows that $x[j..j+n-i]=x[i..n]$, and thus, $x[i..n]$ is a border of $x[j..n]$.

(1a)

Case: $j+n-i$ is black.

Since n may be either black or white, we need to discuss two cases.

(1a$\alpha$)

Case: n is white.

Since n is white, the last $\tau$-pair of $\mathit{x}$ must be $(n-1,n)$. The $\tau$-pairs of $x[j..j+n-i]$ must be the same as the $\tau$-pairs of $x[i..n]$; the last $\tau$-pair of $x[j..j+n-i]$ must be $(j+n-i-1,j+n-i)$. Since $j+n-i$ is black by our assumption (1a), the next $\tau$-pair of $\mathit{x}$ must be $(j+n-i,j+n-i+1)$, as indicated in the following diagram:

Thus, $\tau \left(x\right)\left[t\right(j)..t(j+n-i-1\left)\right]=\tau \left(x\right)\left[t\right(i)..t(n-1\left)\right]$. Since $t(n-1)=m$, we have $\tau \left(x\right)\left[t\right(j)..t(j+n-i-1\left)\right]=\tau \left(x\right)\left[t\right(i)..m]$, and so, $\tau \left(x\right)\left[t\right(i)..m]$ is a proper prefix of $\tau \left(x\right)\left[t\right(j)..m]$ giving $\tau \left(x\right)\left[t\right(i)..m]\prec \tau \left(x\right)\left[t\right(j)..m]$.

(1a$\beta$)

Case: n is black.

Then, the last $\tau$-pair of $\mathit{x}$ must be $(n,n+1)$, and hence, the last $\tau$-pair of $x[j..j+n-i]$, so the next $\tau$-pair is $(j+n-i,j+n-i+1)$; since $n-1$ cannot be black when n is, the situation is as indicated in the following diagram:

Thus, $\tau \left(x\right)\left[t\right(i)..t(n-2\left)\right]=\tau \left(x\right)\left[t\right(j)..t(j+n-i-2\left)\right]$. Since $x[j+n-i]=x\left[n\right]$ and $\left(x\right[n],x[n+1\left]\right)=\left(x\right[n],\$)$, we have $\left(x\right[j+n-i],x[j+n-i+1\left]\right)⊲\left(x\right[n],x[n+1\left]\right)$, and so, $\tau \left(x\right)\left[t\right(j+n-i\left)\right]\prec \tau \left(x\right)\left[t\right(n\left)\right]$, giving $\tau \left(x\right)\left[t\right(j)..t(n\left)\right]\prec \tau \left(x\right)\left[t\right(i)..t(n\left)\right]$. Since $t\left(n\right)=m$, we have $\tau \left(x\right)\left[t\right(j)..m]\prec \tau \left(x\right)\left[t\right(i)..m]$.

(1b)

Case: $j+n-i$ is white.

Then, $j+n-i-1$ is black; hence, $n-1$ is black; so, n must also be white, and thus, $\tau \left(x\right)\left[t\right(j)..t(j+n-i-1\left)\right]=\tau \left(x\right)\left[t\right(i)..t(n-1\left)\right]$, as indicated by the following diagram:

Since $t(n-1)=m$, we have $\tau \left(x\right)\left[t\right(j)..t(j+n-i-1\left)\right]=\tau \left(x\right)\left[t\right(i)..m]$, and so, $\tau \left(x\right)\left[t\right(i)..m]$ is a proper prefix of $\tau \left(x\right)\left[t\right(j)..m]$, giving $\tau \left(x\right)\left[t\right(i)..m]\prec \tau \left(x\right)\left[t\right(j)..m]$.

(2)

Case: $x\left[i\right]\prec x\left[j\right]$ or ($x\left[i\right]=x\left[j\right]$ and $x[i+1]\prec x[j+1]$).

Then, $\left(x\right[i],x[i+1\left]\right)⊲\left(x\right[j],x[j+1\left]\right)$, and so, $\tau \left(x\right)\left[t\right(i\left)\right]\prec \tau \left(x\right)\left[t\right(j\left)\right]$, and thus, $\tau \left(x\right)\left[t\right(i)..m]\prec \tau \left(x\right)\left[t\right(j)..m]$.

(3)

Case: for some $\ell \ge 3$, $x[i..i+\ell -1]=x[j..j+\ell -1]$, while $x[i+\ell ]\prec x[j+\ell ]$.

First note that $i+\ell -2$ and $j+\ell -2$ are either both black, or both are white:

• If $i+\ell -2$ is white, then the $\tau$-pairs $\left(\right(i,i+1),\dots ,(i+\ell -3,i+\ell -2\left)\right)$ of $x[i..n]$ correspond one-to-one to the $\tau$-pairs $\left(\right(j,j+1),\dots ,(j+\ell -3,j+\ell -2\left)\right)$ of $x[j..n]$. To determine what follows $(i+\ell -3,i+\ell -2)$, we need to know the relationship between the values $x[i+\ell -3]$, $x[i+\ell -2]$, and $x[i+\ell -1]$. Since $x[i+\ell -3]=x[j+\ell -3]$, $x[i+\ell -2]=x[j+\ell -2]$, and $x[i+\ell -1]=x[j+\ell -1]$, the values $x[j+\ell -3]$, $x[j+\ell -2]$, and $x[j+\ell -1]$ have the same relationship, and thus, the $\tau$-pair following $(j+\ell -3,j+\ell -2)$ will be the “same” as the $\tau$-pair following $(i+\ell -3,i+\ell -2)$. Since $i+\ell -2$ is white, the $\tau$-pair following $(i+\ell -3,i+\ell -2)$ is $(i+\ell -1,i+\ell )$, and so, the $\tau$-pair following $(j+\ell -3,j+\ell -2)$ is $(j+\ell -1,j+\ell )$, making $j+\ell -2$ white as well.
• If $i+\ell -2$ is black, then the $\tau$-pairs $\left(\right(i,i+1),\dots ,(i+\ell -2,i+\ell -1\left)\right)$ of $x[i..n]$ correspond one-to-one to the $\tau$-pairs $\left(\right(j,j+1),\dots ,(j+\ell -2,j+\ell -1\left)\right)$ of $x[j..n]$. It follows that $j+\ell -2$ is black as well.

We proceed by discussing these two cases for the colours of $i+\ell -2$ and $j+\ell -2$.

(3a)

Case when $i+\ell -2$ and $j+\ell -2$ are both white.

Therefore, we have the $\tau$-pairs $\left(\right(i,i+1),\dots ,(i+\ell -3,i+\ell -2),(i+\ell -1,i+\ell \left)\right)$ for $x[i..n]$ that correspond one-to-one to the $\tau$-pairs $\left(\right(j,j+1),\dots ,(j+\ell -3,j+\ell -2),(j+\ell -1,j+\ell \left)\right)$ for $x[j..n]$. It follows that $\tau \left(x\right)\left[t\right(i)..t(i+\ell -3\left)\right]=\tau \left(x\right)\left[t\right(j)..t(j+\ell -3\left)\right]$. $\tau \left(x\right)\left[t\right(i+\ell -1\left)\right]=p(i+\ell -1,i+\ell )$ and $\tau \left(x\right)\left[t\right(j+\ell -1\left)\right]=p(j+\ell -1,j+\ell )$. Since $x[i+\ell -1]=x[j+\ell -1]$ and, by our assumption (3), $x[i+\ell ]\prec x[j+\ell ]$, it follows that $(i+\ell -1,i+\ell )⊲(j+\ell -1,j+\ell )$, giving $p(i+\ell -1,i+\ell )\prec p(j+\ell -1,j+\ell )$, and so, $\tau \left(x\right)\left[t\right(i+\ell -1\left)\right]\prec \tau \left(x\right)\left[t\right(j+\ell -1\left)\right]$. Since $t(i+\ell -3)+1=t(i+\ell -1)$ and $t(j+\ell -3)+1=t(j+\ell -1)$, we have $\tau \left(x\right)\left[t\right(i)..m]\prec \tau \left(x\right)\left[t\right(j)..m]$.

(3b)

Case when $i+\ell -2$ and $j+\ell -2$ are both black.

Therefore, we have the $\tau$-pairs $\left(\right(i,i+1),\dots ,(i+\ell -2,i+\ell -1\left)\right)$ for $x[i..n]$ that correspond one-to-one to the $\tau$-pairs $\left(\right(j,j+1),\dots ,(j+\ell -2,j+\ell -1\left)\right)$ for $x[j..n]$. It follows that $\tau \left(x\right)\left[t\right(i)..t(i+\ell -2\left)\right]=\tau \left(x\right)\left[t\right(j)..t(j+\ell -2\left)\right]$.

We need to discuss the four cases based on the colours of $i+\ell -1$ and $j+\ell -1$.

(3b$\alpha$)

Both $i+\ell -1$ and $j+\ell -1$ are black.

It follows that the next $\tau$-pair for $x[i..n]$ is $(i+\ell -1,i+\ell )$, and the next $\tau$-pair for $x[j..n]$ is $(j+\ell -1,j+\ell )$. It follows that $t(i+\ell -2)+1=t(i+\ell -1)$ and $t(j+\ell -2)+1=t(j+\ell -1)$. Hence, $\tau \left(x\right)\left[t\right(i+\ell -2)+1]=p(i+\ell -1,i+\ell )$ and $\tau \left(x\right)\left[t\right(j+\ell -2)+1]=p(j+\ell -1,j+\ell )$. Since $x[i+\ell -1]=x[j+\ell -1]$ and, by Assumption (3), $x[i+\ell ]\prec x[j+\ell ]$, we have $\left(x\right[i+\ell -1],x[i+\ell \left]\right)⊲\left(x\right[j+\ell -1],x[j+\ell \left]\right)$, and so, $p\left(x\right[i+\ell -1],x[i+\ell \left]\right)\prec p\left(x\right[j+\ell -1],x[j+\ell \left]\right)$, giving us $\tau \left(x\right)\left[t\right(i+\ell -2)+1]\prec \tau \left(x\right)\left[t\right(j+\ell -2)+1]$. It follows that $\tau \left(x\right)\left[t\right(i)..m]\prec \tau \left(x\right)\left[t\right(j)..m]$.

(3b$\beta$)

$i+\ell -1$ is white, and $j+\ell -1$ is black.

It follows that the next $\tau$-pair for $x[i..n]$ is $(i+\ell ,i+\ell +1)$, and the next $\tau$-pair for $x[j..n]$ is $(j+\ell -1,j+\ell )$. It follows that $t(i+\ell -2)+1=t(i+\ell )$, while $t(j+\ell -2)+1=t(j+\ell -1)$. Thus, $\tau \left(x\right)\left[t\right(i+\ell -2)+1]=p(i+\ell ,i+\ell +1)$ and $\tau \left(x\right)\left[t\right(j+\ell -2)+1]=p(j+\ell -1,j+\ell )$. Since $j+\ell -1$ is black, we know that $x[j-\ell -2]\succ x[j-\ell -1]⪯x[j+\ell ]$. Since $x[i+\ell -2]=x[j+\ell -2]$ and $x[i+\ell -1]=x[j+\ell -1]$, we have $x[i-\ell -2]\succ x[i-\ell -1]$, and so, $x[i-\ell -1]\succ x[i+\ell ]$, as otherwise, $i-\ell -1$ would be black. This gives us $x[j+\ell -1]\succ x[i+\ell ]$. Thus, $\left(x\right[i+\ell ],x[i+\ell +1\left]\right)⊲\left(x\right[j+\ell -1],x[j+\ell \left]\right)$, giving $p(i+\ell ,i+\ell +1)\prec p(j+\ell -1,j+\ell )$ and, ultimately, $\tau \left(x\right)\left[t\right(i+\ell \left)\right]\prec \tau \left(x\right)\left[t\right(j+\ell -1\left)\right]$. The last step is to realize that $t(i+\ell -2)+1=t(i+\ell )$ and $t(j+\ell -2)+1=t(j+\ell -2)$, which gives us $\tau \left(x\right)\left[t\right(i+\ell -2)+1]\prec \tau \left(x\right)\left[t\right(j+\ell -2)+1]$. It follows that $\tau \left(x\right)\left[t\right(i)..m]\prec \tau \left(x\right)\left[t\right(j)..m]$.

(3b$\gamma$)

$i+\ell -1$ is black, and $j+\ell -1$ is white.

It follows that the next $\tau$-pair for $x[i..n]$ is $(i+\ell -1,i+\ell )$, and the next $\tau$-pair for $x[j..n]$ is $(j+\ell ,j+\ell +1)$. It follows that $t(i+\ell -2)+1=t(i+\ell -1)$, while $t(j+\ell -2)+1=t(j+\ell )$. Thus, $\tau \left(x\right)\left[t\right(i+\ell -2)+1]=p(i+\ell -1,i+\ell )$ and $\tau \left(x\right)\left[t\right(j+\ell -2)+1]=p(j+\ell ,j+\ell +1)$. Since $i+\ell -1$ is black, we know that $x[i+\ell -2]\succ x[i+\ell -1]⪯x[i+\ell ]\prec x[j+\ell ]$, where the last inequality is our Assumption (3). Therefore, $x[j+\ell -1]=x[i+\ell -1]\prec x[j+\ell ]$. Thus, $\left(x\right[i+\ell -1],x[i+\ell \left]\right)⊲\left(x\right[j+\ell ],x[j+\ell +1\left]\right)$, giving $p(i+\ell -1,i+\ell )\prec p(j+\ell ,j+\ell +1)$, $\tau \left(x\right)\left[t\right(i+\ell -1\left)\right]\prec \tau \left(x\right)\left[t\right(j+\ell \left)\right]$, and ultimately, $\tau \left(x\right)\left[t\right(i+\ell -2)+1]=\tau \left(x\right)\left[t\right(i+\ell -1\left)\right]\prec \tau \left(x\right)\left[t\right(j+\ell \left)\right]=\tau \left(x\right)\left[t\right(j+\ell -2)+1]$. It follows that $\tau \left(x\right)\left[t\right(i)..m]\prec \tau \left(x\right)\left[t\right(j)..m]$.

(3b$\delta$)

Both $i+\ell -1$ and $j+\ell -1$ are white.

Then, the next $\tau$-pair for $x[i..n]$ is $(i+\ell ,i+\ell +1)$, and the next $\tau$-pair for $x[j..n]$ is $(j+\ell ,j+\ell +1)$. It follows that $t(i+\ell -2)+1=t(i+\ell )$, while $t(j+\ell -2)+1=t(j+\ell )$. Thus, $\tau \left(x\right)\left[t\right(i+\ell -2)+1]=p(i+\ell ,i+\ell +1)$ and $\tau \left(x\right)\left[t\right(j+\ell -2)+1]=p(j+\ell ,j+\ell +1)$. Since $x[i+\ell -1]=x[j+\ell -1]$ and, by our Assumption (3), $x[i+\ell ]\prec x[j+\ell ]$, $\left(x\right[i+\ell ],x[i+\ell -1\left]\right)\prec \left(x\right[j+\ell ],x[j+\ell -1\left]\right)$, giving $p(i+\ell ,i+\ell +1)\prec p(j+\ell ,j+\ell +1)$, $\tau \left(x\right)\left[t\right(i+\ell \left)\right]\prec \tau \left(x\right)\left[t\right(j+\ell \left)\right]$, and ultimately, $\tau \left(x\right)\left[t\right(i+\ell -2)+1]=\tau \left(x\right)\left[t\right(i+\ell \left)\right]\prec \tau \left(x\right)\left[t\right(j+\ell \left)\right]=\tau \left(x\right)\left[t\right(j+\ell -2)+1]$. It follows that $\tau \left(x\right)\left[t\right(i)..m]\prec \tau \left(x\right)\left[t\right(j)..m]$.

□

Lemma 5 shows that $\tau$-reduction preserves the proto-Lyndon property of certain proto-Lyndon substrings of $\mathit{x}$.

Lemma 5.

Let $x=x[1..n]$, and let $\tau \left(x\right)=\tau \left(x\right)[1..m]$. Let $1\le i<j\le n$. Let $x[i..j]$ be a proto-Lyndon substring of $\mathit{x}$, and let i be a black position.

Then, $\left\{\begin{array}{cc}\tau \left(x\right)\left[t\right(i)..t(j\left)\right]\mathit{is}\mathit{proto}-\mathit{Lyndon}\hfill & \mathit{if}j\mathit{is}\mathit{black}\hfill \\ \tau \left(x\right)\left[t\right(i)..t(j-1\left)\right]\mathit{is}\mathit{proto}-\mathit{Lyndon}\hfill & \mathit{if}j\mathit{is}\mathit{white}.\hfill \end{array}\right.$

Proof. 

Let us first assume that j is black.

Since both i and j are black, $t\left(i\right)$ and $t\left(j\right)$ are defined. Let $i_1=t\left(i\right)$, $j_1=t\left(j\right)$, and consider $k_1$, so that $i_1<k_1\le j_1$. Let $k=b\left(k_1\right)$. Then, $t\left(k\right)=k_1$ and $i<k\le j$, and so, $x[i..n]\prec x[k..n]$ by Lemma 1 as $x[i..j]$ is proto-Lyndon. It follows that $\tau \left(x\right)\left[t\right(i)..m]\prec \tau \left(x\right)\left[t\right(k)..m]$ by Lemma 4. Thus, $\tau \left(x\right)[i_1..m]\prec \tau \left(x\right)[k_1..m]$ for any $i_1<k_1\le j_1$, and so, $\tau \left(x\right)[i_1..j_1]$ is proto-Lyndon by Lemma 1.

Now, let us assume that j is white.

Then, $j-1$ is black, and $x[i..j-1]$ is proto-Lyndon, so as in the previous case, $\tau \left(x\right)\left[t\right(i)..t(j-1\left)\right]$ is proto-Lyndon. □

Now, we can show that $\tau$-reduction preserves some right-maximal Lyndon substrings.

Lemma 6.

Let $x=x[1..n]$, and let $\tau \left(x\right)=\tau \left(x\right)[1..m]$. Let $1\le i<j\le n$. Let $x[i..j]$ be a right-maximal Lyndon substring, and let i be a black position.

Then, $\left\{\begin{array}{cc}\tau \left(x\right)\left[t\right(i)..t(j\left)\right]\mathit{is}\mathit{a}\mathit{right}-\mathit{maximal}\mathit{Lyndon}\mathit{substring}\hfill & \mathit{if}j\mathit{is}\mathit{black}\hfill \\ \tau \left(x\right)\left[t\right(i)..t(j-1\left)\right]\mathit{is}\mathit{a}\mathit{right}-\mathit{maximal}\mathit{Lyndon}\mathit{substring}\hfill & \mathit{if}j\mathit{is}\mathit{white}.\hfill \end{array}\right.$

Proof. 

Since $x[i..j]$ is Lyndon and hence proto-Lyndon, by Lemma 5, we know that $\tau \left(x\right)\left[t\right(i)..t(j\left)\right]$ is proto-Lyndon for j black, while for white j, $\tau \left(x\right)\left[t\right(i)..t(j-1\left)\right]$ is proto-Lyndon. Thus, in order to conclude that the respective strings are right-maximal Lyndon substrings, we only need to prove that the property $\left(c\right)$ of Lemma 1 holds in both cases.

Since $x[i..j]$ is right-maximal Lyndon, either $j=n$ or $x[j+1..n]\prec x[i..n]$ by Lemma 1, giving $j=n$ or $x[j+1]⪯x\left[i\right]$. Since $x[i..j]$ is Lyndon and hence unbordered, $x\left[i\right]\prec x\left[j\right]$. Thus, either $j=n$ or $x[j+1]⪯x\left[i\right]\prec x\left[j\right]$.

If $j=n$, then there are two simple cases. If n is white, $n-1$ is black and $m=t(n-1)$, so $t(j-1)=m$, giving us (c) of Lemma 1 for $\tau \left(x\right)\left[t\right(i)..t(j-1\left)\right]$. On the other hand, if n is black, then $m=t\left(n\right)$, and so, $m=t\left(j\right)$ giving us (c) of Lemma 1 for $\tau \left(x\right)\left[t\right(i)..t(j\left)\right]$.

Thus, in the following, we can assume that $j<n$ and that $x[j+1]⪯x\left[i\right]\prec x\left[j\right]$. We will proceed by discussing two possible cases, one where j is black and the other where j is white.

(1)

Case: j is black.

We need to show that either $t\left(j\right)=m$ or $\tau \left(x\right)\left[t\right(i)..m]\succ \tau \left(x\right)\left[t\right(j)+1..m]$.

If $j=n$, then $t\left(j\right)=m$, and we are done. Thus, we can assume that $j<n$. We must show that $\tau \left(x\right)\left[t\right(j)+1..m]\prec \tau \left(x\right)\left[t\right(i)..m]$.

(1a)

Case: $x[j+1]⪯x[j+2]$.

Then, $x\left[j\right]\succ x[j+1]$ and $x[j+1]⪯x[j+2]$, and so, $j+1$ is black. It follows that $t\left(j\right)+1=t(j+1)$. By Lemma 4, $\tau \left(x\right)\left[t\right(j+1)..m]\prec \tau \left(x\right)\left[t\right(i)..m]$ because $x[j+1..n]\prec x[i..n]$, thus $\tau \left(x\right)\left[t\right(j)+1..m]\prec \tau \left(x\right)\left[t\right(i)..m]$.

(1b)

Case: $x[j+1]\succ x[j+2]$.

Then, $x\left[j\right]\succ x\left[i\right]⪰x[j+1]\succ x[j+2]$. It follows that the $\tau$-pair $(j,j+1)$ is followed by a $\tau$-pair $(j+2,j+3)$, and thus, $t\left(j\right)+1=t(j+2)$. Thus,

$\left(x\right[j+2],x[j+3\left]\right)⊲\left(x\right[i],x[i+1\left]\right)⊲\left(x\right[j],x[j+1\left]\right)$; hence,

$p(j+2,j+3)\prec p(i,i+1)\prec p(j,j+1)$. Since $\tau \left(x\right)\left[t\right(j)+1]=p(j+2,j+3)$, $\tau \left(x\right)\left[t\right(i\left)\right]=p(j,i+1)$, and $\tau \left(x\right)\left[t\right(j\left)\right]=p(j,j+1)$, it follows that $\tau \left(x\right)\left[t\right(j)+1]\prec \tau \left(x\right)\left[t\right(i\left)\right]$, and so, $\tau \left(x\right)\left[t\right(j)+1..m]\prec \tau \left(x\right)\left[t\right(i)..m]$.

(2)

Case: j is white.

We need to prove that $\tau \left(x\right)\left[t\right(j-1)+1..m]\prec \tau \left(x\right)\left[t\right(i)..m]$. Since j is white, necessarily both $j-1$ and $j+1$ are black and $t(j-1)+1=t(j+1)$. By Lemma 5, $\tau \left(x\right)\left[t\right(i)..t(j-1\left)\right]$ is proto-Lyndon as both i and $j-1$ are black and $x[i..j-1]$ is proto-Lyndon. Since $x[i..n]\succ x[j+1..n]$ and both i and $j+1$ are black, by Lemma 4, we get $\tau \left(x\right)\left[t\right(i)..m]\succ \tau \left(x\right)\left[t\right(j+1)..m]=\tau \left(x\right)\left[t\right(j-1)+1..m]$.

□

Now, we are ready to tackle the proof of Theorem 2.

Proof of Theorem 2.

Let ${\mathcal{L}}_x^{\prime }\left[i\right]=j$ where i is black. Then, $t\left(i\right)$ is defined, and $x[i..j]$ is a right-maximal Lyndon substring of $\mathit{x}$. We proceed by analysis of the two possible cases of the label for the position j. Let (*) denote the condition from the theorem, i.e.,

(*)

$b\left({\mathcal{L}}_{\tau \left(x\right)}^{\prime }\left[t\left(i\right)\right]\right)=n$or$x\left[b\right({\mathcal{L}}_{\tau \left(x\right)}^{\prime }\left[t\left(i\right)\right])+1]⪯x\left[i\right]$

(1)

Case: j is black.

Then, by Lemma 6, $\tau \left(x\right)\left[t\right(i)..t(j\left)\right]$ is a right-maximal Lyndon substring of $\tau \left(x\right)$; hence, ${\mathcal{L}}_{\tau \left(x\right)}^{\prime }\left[t\left(i\right)\right]=t\left(j\right)$. Therefore, $b\left({\mathcal{L}}_{\tau \left(x\right)}^{\prime }\left[t\left(i\right)\right]\right)=b\left(t\left(j\right)\right)=j={\mathcal{L}}_x^{\prime }\left[i\right]$. We have to also prove that the condition (*) holds.

If $j=n$, then the condition (*) holds. Therefore, assume that $j<n$. Since $x[i..j]$ is right-maximal, by Lemma 1, $x[j+1..n]\prec x[i..n]$, and so, $x[j+1]⪯x\left[i\right]$. Then, $x\left[b\right({\mathcal{L}}_{\tau \left(x\right)}^{\prime }\left[t\left(i\right)\right])+1]=x\left[b\right(t\left(j\right))+1]=x[j+1]⪯x\left[i\right]$.

(2)

Case: j is white.

Then, $j-1$ is black, and $\tau \left(x\right)\left[t\right(j-1\left)\right]=p(j-1,j)$. By Lemma 6, $\tau \left(x\right)\left[t\right(i)..t(j-1\left)\right]$ is a right-maximal Lyndon substring of $\tau \left(x\right)$; hence, ${\mathcal{L}}_{\tau \left(x\right)}^{\prime }\left[t\left(i\right)\right]=t(j-1)$, so $b\left({\mathcal{L}}_{\tau \left(x\right)}^{\prime }\left[t\left(i\right)\right]\right)=$$b\left(t\right(j-1\left)\right)=j-1$, giving $b({\mathcal{L}}_{\tau \left(x\right)}^{\prime }\left[t\left(i\right)\right]+1)=j$.

We want to show that the condition (*) does not hold.

If $b\left({\mathcal{L}}_{\tau \left(x\right)}^{\prime }\left[t\left(i\right)\right]\right)=n$, then $j-1=n$, which is impossible as $j\le n$. Since $x[i..j]$ is Lyndon, $x\left[i\right]\prec x\left[j\right]$, and so, $x\left[i\right]\prec x\left[b\right({\mathcal{L}}_{\tau \left(x\right)}^{\prime }\left[t\left(i\right)\right]+1\left)\right]$. Thus, Condition (*) does not hold.

□

3.4. Computing ${\mathcal{L}}_x^{\prime }$ from ${\mathcal{L}}_{\tau \left(x\right)}^{\prime }$

Theorem 2 indicates how to compute the partial ${\mathcal{L}}_x^{\prime }$ from ${\mathcal{L}}_{\tau \left(x\right)}^{\prime }$. The procedure is given in Figure 3.

To compute the missing values, the partial array is processed from right to left. When a missing value at position i is encountered (note that it is recognized by ${\mathcal{L}}_x^{\prime }\left[i\right]=nil$), the Lyndon array ${\mathcal{L}}_x^{\prime }[i+1..n]$ is completely filled, and also, ${\mathcal{L}}_x^{\prime }[i-1]$ is known. Recall that ${\mathcal{L}}_x^{\prime }[i+1]$ is the ending position of the right-maximal Lyndon substring starting at the position $i+1$. In several cases, we can determine the value of ${\mathcal{L}}_x^{\prime }\left[i\right]$ in constant time:

(1)

if $i=n$, then ${\mathcal{L}}_x^{\prime }\left[i\right]=i$.

(2)

if $x\left[i\right]\succ x[i+1]$, then ${\mathcal{L}}_x^{\prime }\left[i\right]=i$.

(3)

if $x\left[i\right]=x[i+1]$ and ${\mathcal{L}}_x^{\prime }[i+1]=i+1$ and either $i+1=n$ or $i+1={\mathcal{L}}_x^{\prime }[i-1]$, then ${\mathcal{L}}_x^{\prime }\left[i\right]=i$.

(4)

if $x\left[i\right]\prec x[i+1]$ and ${\mathcal{L}}_x^{\prime }[i+1]=i+1$ and either $i+1=n$ or $i+1={\mathcal{L}}_x^{\prime }[i-1]$, then ${\mathcal{L}}_x^{\prime }\left[i\right]=i+1$.

(5)

if $x\left[i\right]⪯x[i+1]$ and ${\mathcal{L}}_x^{\prime }[i+1]>i+1$ and either ${\mathcal{L}}_x^{\prime }[i+1]=n$ or ${\mathcal{L}}_x^{\prime }[i+1]={\mathcal{L}}_x^{\prime }[i-1]$, then ${\mathcal{L}}_x^{\prime }\left[i\right]={\mathcal{L}}_x^{\prime }[i+1]$.

We call such points easy. All others will be referred to as hard. For a hard point i, it means that $x\left[i\right]$ is followed by at least two consecutive right-maximal Lyndon substrings before reaching either ${\mathcal{L}}_x^{\prime }[i-1]$ or n, and we might need to traverse them all.

The while loop, seen in Figure 4’s procedure, is the likely cause of the $\mathcal{O}(nlog(n\left)\right)$ complexity. At first glance, it may seem that the complexity might be $\mathcal{O}\left(n^2\right)$; however, the doubling of the length of the string when a hard point is introduced actually trims it down to an $\mathcal{O}(nlog(n\left)\right)$ worst-case complexity. See Section 3.5 for more details and Section 7 for the measurements and graphs.

Consider our running example from Figure 2. Since $\tau \left(x\right)=021534$, we have ${\mathcal{L}}_{\tau \left(x\right)}^{\prime }[1..6]=6,2,6,4,6,6$ giving ${\mathcal{L}}_x^{\prime }[1..9]=9,•,3,9,•,6,9,•,9$. Computing ${\mathcal{L}}_x^{\prime }\left[8\right]$ is easy as $x\left[8\right]=x\left[9\right]$, and so, ${\mathcal{L}}_x^{\prime }\left[8\right]=8$. ${\mathcal{L}}_x^{\prime }\left[5\right]$ is more complicated and an example of a hard point: we can extend the right-maximal Lyndon substring from ${\mathcal{L}}_x^{\prime }\left[6\right]$ to the left to 23, but no more, so ${\mathcal{L}}_x^{\prime }\left[5\right]=6$. Computing ${\mathcal{L}}_x^{\prime }\left[2\right]$ is again easy as $x\left[2\right]=x\left[3\right]$, and so, ${\mathcal{L}}_x^{\prime }\left[2\right]=2$. Thus, ${\mathcal{L}}_x^{\prime }[1..9]=9,2,3,9,6,6,9,8,9$.

3.5. The Complexity of TRLA

To determine the complexity of the algorithm, we attach to each position i a counter $red\left[i\right]$ initialized to zero. Imagine a hard point j indicated by the following diagram:

$A_1$ represents the right-maximal Lyndon substring starting at the position $j+1$; $A_2$ represents the right-maximal Lyndon substring following immediately $A_1$ and so forth. To make j a hard point, $r\ge 2$ and $x\left[j\right]⪯x[j+1]$. The value of $stop$ is determined by:    

               $stop=\left\{\begin{array}{cc}{\mathcal{L}}_x^{\prime }[j-1]\hfill & if{\mathcal{L}}_x^{\prime }[j-1]>j-1\hfill \\ n\hfill & otherwise\hfill \end{array}\right.$.    

To determine the right-maximal Lyndon substring starting at the hard position j, we need first to check if $A_1$ can be left-extended by $x\left[j\right]$ to make $j{A}_1$ Lyndon; we are using abbreviated notation $j{A}_1$ for the substring $x[j..k]$ where $A_1=x[j+1..k]$; in simple words, $j{A}_1$ represents the left-extension of $A_1$ by one position. If $j{A}_1$ is proto-Lyndon, we have to check whether $A_2$ can be left-extended by $j{A}_1$ to a Lyndon substring. If $j{A}_1{A}_2$ is Lyndon, we must continue until we check whether $j{A}_1{A}_2\dots A_{r-1}$ is Lyndon. If so, we must check whether $j{A}_1\dots A_r$ is Lyndon. We need not go beyond $stop$.

How do we check if $j{A}_1\dots A_k$ can left-extend $A_{k+1}$ to a Lyndon substring? If $j{A}_1\dots A_k⪰A_{k+1}$, we can stop, and $j{A}_1\dots A_k$ is the right-maximal Lyndon substring starting at position j. If $j{A}_1\dots A_k\prec A_{k+1}$, we need to continue. Since $stop$ is the last position of the right-maximal Lyndon substring at the position $j-1$ or n, we are assured to stop there. When comparing the substring $j{A}_1\dots A_k$ with $A_{k+1}$, we increment the counter $red\left[i\right]$ at every position of $A_{k+1}$ used in the comparison. When done with the whole array, the value of $red\left[i\right]$ represents how many times i was used in various comparisons, for any position i.

Consider a position i that was used k times for $k\ge 4$, i.e., $red\left[i\right]=k$. In the next four diagrams and related text, the upper indices of A and C do not represent powers; they are just indices. The next diagram indicates the configuration when the counter $red\left[i\right]$ was incremented for the first time in the comparison of $j_1{A}_1^1\dots A_{r_1-1}^1$ and $A_{r_1}^1$ during the computation of the missing value ${\mathcal{L}}_x^{\prime }\left[j_1\right]$ where:

$$sto{p}_1=\left\{\begin{array}{cc}{\mathcal{L}}_x^{\prime }[j_1-1]\hfill & if{\mathcal{L}}_x^{\prime }[j_1-1]>j_1-1\hfill \\ n\hfill & otherwise\hfill \end{array}\right.$$

The next diagram indicates the configuration when the counter $red\left[i\right]$ was incremented for the second time in the comparison of $j_2{A}_1^2\dots A_{r_2-1}^2$ and $A_{r_2}^2$ during the computation of the missing value ${\mathcal{L}}_x^{\prime }\left[j_2\right]$ where:

$$sto{p}_2=\left\{\begin{array}{cc}{\mathcal{L}}_x^{\prime }[j_2-1]\hfill & if{\mathcal{L}}_x^{\prime }[j_2-1]>j_2-1\hfill \\ n\hfill & otherwise\hfill \end{array}\right.$$

The next diagram indicates the configuration when the counter $red\left[i\right]$ was incremented for the third time in the comparison of $j_3{A}_1^3\dots A_{r_3-1}^3$ and $A_{r_3}^3$ during the computation of the missing value ${\mathcal{L}}_x^{\prime }\left[j_3\right]$ where:

$$sto{p}_3=\left\{\begin{array}{cc}{\mathcal{L}}_x^{\prime }[j_3-1]\hfill & if{\mathcal{L}}_x^{\prime }[j_3-1]>j_3-1\hfill \\ n\hfill & otherwise\hfill \end{array}\right.$$

The next diagram indicates the configuration when the counter $red\left[i\right]$ was incremented for the fourth time in the comparison of $j_4{A}_1^4\dots A_{r_4-1}^4$ and $A_{r_4}^4$ during the computation of the missing value ${\mathcal{L}}_x^{\prime }\left[j_4\right]$ where:

$$sto{p}_4=\left\{\begin{array}{cc}{\mathcal{L}}_x^{\prime }[j_4-1]\hfill & if{\mathcal{L}}_x^{\prime }[j_4-1]>j_4-1\hfill \\ n\hfill & otherwise\hfill \end{array}\right.$$

and so forth until the k-th increment of $red\left[i\right]$. Thus, if $red\left[i\right]=k$, then $n\ge 2^{k-1}(n_1+1)\ge 2^k$ as $n_1+1\ge 2$. Thus, $n\ge 2^k$, and so, $k\le log\left(n\right)$. Thus, either $k<4$ or $k\le log\left(n\right)$. Therefore, the overall complexity is $\mathcal{O}(nlog(n\left)\right)$.

To show that the average case complexity is linear, we first recall that the overall complexity of TRLA is determined by the procedure filling the missing values. We showed above that there are at most $log\left(n\right)$ missing values (hard positions) that cannot be determined in constant time. We overestimate the number of strings of length n over an alphabet of size $\mathsf{\Sigma }$, $2\le \mathsf{\Sigma }\le n$, which will force a non-linear computation, by assuming that every possible $log\left(n\right)$ subset of indices with any possible letter assignment forces the worst performance. Thus, there are ${\mathsf{\Sigma }}^n-\left(\genfrac{}{}{0pt}{}{n}{log\left(n\right)}\right){\mathsf{\Sigma }}^{log\left(n\right)}$ strings that are processed in linear time, say with a constant $K_1$, and there are $\left(\genfrac{}{}{0pt}{}{n}{log\left(n\right)}\right){\mathsf{\Sigma }}^{log\left(n\right)}$ strings that are processed in the worst time, with a constant $K_2$. Let $K=max(K_1,K_2)$. Then, the average time is bounded by:

$$\frac{({\mathsf{\Sigma }}^n-\left(\genfrac{}{}{0pt}{}{n}{log\left(n\right)}\right){\mathsf{\Sigma }}^{log\left(n\right)})Kn+\left(\genfrac{}{}{0pt}{}{n}{log\left(n\right)}\right){\mathsf{\Sigma }}^{log\left(n\right)}Knlog\left(n\right)}{{\mathsf{\Sigma }}^n}=$$

$$Kn+Kn\frac{\left(\genfrac{}{}{0pt}{}{n}{log\left(n\right)}\right){\mathsf{\Sigma }}^{log\left(n\right)}}{{\mathsf{\Sigma }}^n}(log\left(n\right)-1)\le$$

$$Kn+Kn\frac{\left(\genfrac{}{}{0pt}{}{n}{log\left(n\right)}\right){\mathsf{\Sigma }}^{log\left(n\right)}}{{\mathsf{\Sigma }}^n}log\left(n\right)\le$$

$$Kn+Kn\frac{n^{log\left(n\right)}{\mathsf{\Sigma }}^{log\left(n\right)}}{log\left(n\right)!{\mathsf{\Sigma }}^n}log\left(n\right)\le$$

$$Kn+Kn\frac{n^{2log\left(n\right)}}{2^n}\le$$

$$Kn+Kn=2Kn$$

for $n\ge 2^7$. The last step follows from the fact that $n^{2log\left(n\right)}\le 2^n$ for any $n\ge 2^7$.

The combinatorics of the processing is too complicated to ascertain whether the worst-case complexity is linear or not. We tried to generate strings that might give the worst performance. We used three different formulas to generate the strings, nesting the white indices that might require non-constant computation: the dataset extreme_trla of binary strings is created using the recursive formula $u_{k+1}=$00$u_k$0$u_k$, using the first 100 shortest binary Lyndon strings as the start $u_0$. The moment the size $u_k$ exceeds the required length of the string, the recursion stops, and the string is trimmed to the required length. For the extreme_trla1 dataset, we used the same approach with the formula $u_{k+1}=$ 000$u_k$00$u_k$, and for the extreme_trla2 dataset, we used the formula $u_{k+1}=$ 0000$u_k$00$u_k$.

The space complexity of our C++ implementation is bounded by $9n$ integers. This upper bound is derived from the fact that a Tau object (see Tau.hpp [24]) requires $3n$ integers of space for a string of length n. Therefore, the first call to TRLA requires $3n$, the next recursive call at most $3\frac{2}{3}n$, the next recursive call at most $3{\left(\frac{2}{3}\right)}^{2}n$, ...; thus, $3n+3\frac{2}{3}n+3{\left(\frac{2}{3}\right)}^{2}n+3{\left(\frac{2}{3}\right)}^{3}n+\dots =3n(1+\frac{2}{3}+{\left(\frac{2}{3}\right)}^2+{\left(\frac{2}{3}\right)}^3+{\left(\frac{2}{3}\right)}^4+\dots )=3n\frac{1}{1-\frac{2}{3}}=9n$. However, it should be possible to bring it down to $6n$ integers.

4. The Algorithm BSLA

The purpose of this section is to present a linear algorithm BSLA for computing the Lyndon array of a string over an integer alphabet. The algorithm is based on a series of refinements of a list of groups of indices of the input string. The refinement is driven by a group that is already complete, and the refinement process makes the immediately preceding group also complete. In turn, this newly completed group is used as the driver of the next round of the refinement. In this fashion, the refinement proceeds from right to left until all the groups in the list are complete. The initial list of groups consists of the groups of indices with the same alphabet symbol. The section contains proper definitions of all these terms—group, complete group, and refinement. In the process of refinement, each newly created group is assigned a specific substring of the input string referred to as the context of the group. Throughout the process, the list of the groups is maintained in an increasing lexicographic order by their contexts. Moreover, at every stage, the contexts of all the groups are Lyndon substrings of $\mathit{x}$ with an additional property that the contexts of the complete groups are right-maximal Lyndon substrings. Hence, when the refinement is completed, the contexts of all the groups in the list represent all the right-maximal Lyndon substrings of $\mathit{x}$. The mathematics of the process of refinement is necessary in order to ascertain its correctness and completeness and to determine the worst-case complexity of the algorithm.

4.1. Notation and Basic Notions of BSLA

For the sake of simplicity, we fix a string $x=x[1..n]$ for the whole Section 4.1; all the definitions and the observations apply and refer to this $\mathit{x}$.

A group $\mathit{G}$ is a non-empty set of indices of $\mathit{x}$. The group G is assigned a context, i.e., a substring $\mathit{con}\left(\mathit{G}\right)$ of $\mathit{x}$ with the property that for any $i\in G$, $x[i..i+|con\left(G\right)|-1]=con\left(G\right)$. If $i\in G$, then $\mathcal{C}\left(\mathit{i}\right)$ denotes the occurrence of the context of G at the position i, i.e., the substring $\mathcal{C}\left(i\right)=x[i..i+|con\left(G\right)|-1]$. We say that a group $G^{\prime }$ is smaller than or precedes a group $G^{″}$ if $con\left(G^{\prime }\right)\prec con\left(G^{″}\right)$.

Definition 1.

An ordered list of groups $\langle G_k,G_{k-1},\dots ,G_2,G_1\rangle$ is agroup configurationif:

$\left(C_1\right)$

$G_k\cup G_{k-1}\cup \dots \cup G_2\cup G_1=1..n$;

$\left(C_2\right)$

$G_j\cap G_{\ell }=\varnothing$ for any $1\le \ell <j\le k$;

$\left(C_3\right)$

$con\left(G_k\right)\prec con\left(G_{k-1}\right)\prec \dots \prec con\left(G_2\right)\prec con\left(G_1\right)$;

$\left(C_4\right)$

For any $j\in 1..k$, $con\left(G_j\right)$ is a Lyndon substring of $\mathit{x}$.

Note that ($C_1$) and ($C_2$) guarantee that $\langle G_k,G_{k-1},\dots ,G_2,G_1\rangle$ is a disjoint partitioning of $1..n$. For $i\in \{1,..,n\}$, $\mathit{gr}\left(\mathit{i}\right)$ denotes the unique group to which i belongs, i.e., if $i\in G_t$, then $gr\left(i\right)=G_t$. Note that using this notation, $\mathcal{C}\left(i\right)=x[i..i+|con\left(gr\right(i\left)\right)|-1]$.

The mapping $\mathit{prev}$ is defined by $prev\left(i\right)=max\{j<i|con\left(gr\right(j\left)\right)\prec con\left(gr\right(i\left)\right)\}$ if such j exists, otherwise $prev\left(i\right)=nil$.

For a group G from a group configuration, we define an equivalence ∼ on G as follows: $i\sim j$ iff $gr\left(prev\right(i\left)\right)=gr\left(prev\right(j\left)\right)$ or $prev\left(i\right)=prev\left(j\right)=nil$. The symbol ${\left[\mathit{i}\right]}_{{}_{\sim }}$ denotes the class of equivalence ∼ that contains i, i.e., ${\left[i\right]}_{{}_{\sim }}=\{j\in G|j\sim i\}$. If $prev\left(i\right)=nil$, then the class ${\left[i\right]}_{{}_{\sim }}$ is called trivial. An interesting observation states that if G is viewed as an ordered set of indices, then a non-trivial ${\left[i\right]}_{{}_{\sim }}$ is an interval:

Observation 7.

Let G be a group from a group configuration for $\mathit{x}$. Consider an $i\in G$ such that $prev\left(i\right)\ne nil$. Let $j_1=min{\left[i\right]}_{{}_{\sim }}$ and $j_2=max{\left[i\right]}_{{}_{\sim }}$. Then, ${\left[i\right]}_{{}_{\sim }}=\{j\in G|j_1\le j\le j_2\}$.

Proof. 

Since $prev\left(j_1\right)$ is a candidate to be $prev\left(j\right)$, $prev\left(j\right)\ne nil$ and $prev\left(j_1\right)\le prev\left(j\right)\le prev\left(j_2\right)=prev\left(j_1\right)$, so $prev\left(j\right)=prev\left(j_1\right)=prev\left(j_2\right)$. □

On each non-trivial class of ∼, we define a relation ≈ as follows: $i\approx j$ iff $|j-i|=\left|con\right(G\left)\right|$; in simple terms, it means that the occurrence $\mathcal{C}\left(i\right)$ of $con\left(G\right)$ is immediately followed by the occurrence $\mathcal{C}\left(j\right)$ of $con\left(G\right)$. The transitive closure of ≈ is a relation of equivalence, which we also denote by ≈. The symbol ${\left[\mathit{i}\right]}_{{}_{\approx }}$ denotes the class of equivalence ≈ containing i, i.e., ${\left[i\right]}_{{}_{\approx }}=\{j\in {\left[i\right]}_{{}_{\sim }}|j\approx i\}$.

For each j from a non-trivial ${\left[i\right]}_{{}_{\sim }}$, we define the valence by ${val\left(j\right)=|\left[i\right]}_{{}_{\approx }}|$. In simple terms, $val\left(i\right)$ is the number of elements from ${\left[i\right]}_{{}_{\sim }}$ that are $\approx i$. Thus, $1\le val\left(i\right)\le \left|G\right|$.

Interestingly, if G is viewed as an ordered set of indices, then ${\left[i\right]}_{{}_{\approx }}$ is a subinterval of the interval ${\left[i\right]}_{{}_{\sim }}$:

Observation 8.

Let G be a group from a group configuration for $\mathit{x}$. Consider an $i\in G$ such that $prev\left(i\right)\ne nil$. Let $j_1=min{\left[i\right]}_{{}_{\approx }}$ and $j_2=max{\left[i\right]}_{{}_{\approx }}$. Then, ${\left[i\right]}_{{}_{\approx }}=\{j\in {\left[i\right]}_{{}_{\sim }}|j_1\le j\le j_2\}$.

Proof. 

We argue by contradiction. Assume that there is an $j\in {\left[i\right]}_{{}_{\sim }}$ so that $j_1<j<j_2$ and $j\notin {\left[i\right]}_{{}_{\approx }}$. Take the minimal such j. Consider $j^{\prime }=j-\left|con\left(G\right)\right|$. Then, $j^{\prime }\in {\left[i\right]}_{{}_{\sim }}$, and since, $j^{\prime }<j$, $j^{\prime }\in {\left[i\right]}_{{}_{\approx }}$ due to the minimality of j. Therefore, $i\approx j^{\prime }\approx j$, and so, $j\approx i$, a contradiction. □

Definition 2.

A group G iscompleteif for any $i\in G$, the occurrence $\mathcal{C}\left(i\right)$ of $con\left(G\right)$ is a right-maximal Lyndon substring of $\mathit{x}$.

A group configuration $\langle G_k,G_{k-1},\dots ,G_2,G_1\rangle$ ist-complete, $1\le t\le k$, if

$\left(C_5\right)$

the groups $G_t,\dots ,G_1$ are complete;

$\left(C_6\right)$

the mapping $prev$ isproperon $G_t$:

for any $i\in G_t$, if $prev\left(i\right)\ne nil$ and $v=val\left(i\right)$, then there are $i_1,\dots ,i_v\in G_t$, $i\in \{i_1,\dots ,i_v\}$, $prev\left(i\right)=prev\left(i_1\right)=\dots =prev\left(i_v\right)$, and so that $\mathcal{C}\left(prev\left(i\right)\right)\mathcal{C}\left(i_1\right)\dots \mathcal{C}\left(i_v\right)$ is a prefix of $x[j..n]$;

$\left(C_7\right)$

the family $\left\{\mathcal{C}\right(i\left)\right|i\in 1..n\}$ isproper:

$\left(a\right)$ if $\mathcal{C}\left(j\right)$ is a proper substring of $\mathcal{C}\left(i\right)$, i.e., $\mathcal{C}\left(j\right)⊊\mathcal{C}\left(i\right)$, then $con\left(G_t\right)\prec con\left(gr\left(j\right)\right)$,

$\left(b\right)$ if $\mathcal{C}\left(i\right)$ is followed immediately by $\mathcal{C}\left(j\right)$, i.e., when $i+\left|con\right(gr\left(i\right)\left)\right|=j$, and $\mathcal{C}\left(i\right)\prec \mathcal{C}\left(j\right)$, then $con\left(gr\left(j\right)\right)⪯con\left(G_t\right)$;

$\left(C_8\right)$

the family $\left\{\mathcal{C}\right(i\left)\right|i\in 1..n\}$ has theMongeproperty, i.e., if $\mathcal{C}\left(i\right)\cap \mathcal{C}\left(j\right)\ne \varnothing$, then $\mathcal{C}\left(i\right)\subseteq \mathcal{C}\left(j\right)$ or $\mathcal{C}\left(j\right)\subseteq \mathcal{C}\left(i\right)$.

The condition $\left(C_6\right)$ is all-important for carrying out the refinement process (see $\left(R_3\right)$ below). The conditions $\left(C_7\right)$ and $\left(C_8\right)$ are necessary for asserting that the condition $\left(C_6\right)$ is preserved during the refinement process.

4.2. The Refinement

For the sake of simplicity, we fix a string $x=x[1..n]$ for the whole Section 4.2; all the definitions, lemmas, and theorems apply and refer to this $\mathit{x}$.

Lemma 9.

Let ${\mathcal{A}}_x=\{a_1,\dots ,a_k\}$ and $a_1\prec a_2\prec \dots \prec a_k$. For $1\le \ell \le k$, define $G_{\ell }=\{i\in 1..n|x\left[i\right]=a_{k+1-\ell }\}$ with context $a_{k+1-\ell }$. Then, $\langle G_k,\dots ,G_1\rangle$ is a one-complete group configuration.

Proof. 

($C_1$), ($C_2$), ($C_3$), and ($C_4$) are straightforward to verify. To verify ($C_5$), we need to show that $G_1$ is complete. Any occurrence of $a_k$ in $\mathit{x}$ is a right-maximal Lyndon substring, so $G_1$ is complete.

To verify ($C_6$), consider $j=prev\left(i\right)$ and $val\left(i\right)=v$ for $i\in G_1$. Consider any r such that $j<r<i$. If $x\left[r\right]\ne a_k$, then $prev\left(i\right)<r$, which contradicts the definition of $prev$. Hence, $x\left[r\right]=a_k$, and so, $x[j+1]=\dots =x\left[i\right]=\dots x[j+v+1]=a_k$, while $x\left[j\right]=a_q$ for some $q<k$. It follows that $x[j..n]$ has $a_q{\left(a_k\right)}^v$ as a prefix.

The condition $\left(C_7\left(a\right)\right)$ is trivially satisfied as no $\mathcal{C}\left(i\right)$ can have a proper substring. If $\mathcal{C}\left(i\right)$ is immediately followed by $\mathcal{C}\left(j\right)$ and $\mathcal{C}\left(i\right)\prec \mathcal{C}\left(j\right)$, then $\mathcal{C}\left(i\right)=x\left[i\right]$, $j=i+1$, $\mathcal{C}\left(j\right)=x[i+1]$, and $x\left[i\right]\prec x[i+1]$. Then, $con\left(\mathcal{C}\left(j\right)\right)=x[i+1]⪯a_k=con\left(G_1\right)$, so $\left(C_7\left(b\right)\right)$ is also satisfied.

To verify ($C_8$), consider $\mathcal{C}\left(i\right)\cap \mathcal{C}\left(j\right)\ne \varnothing$. Then, $\mathcal{C}\left(i\right)=x\left[i\right]=x\left[j\right]=\mathcal{C}\left(j\right)$. □

Let $\langle G_k,\dots ,G_t,\dots ,G_1\rangle$ by a t-complete group configuration. The refinement is driven by the group $G_t$, and it might only partition the groups that precede it, i.e., the groups $G_k,\dots ,G_{t+1}$, while the groups $G_t,\dots ,G_1$ remain unchanged.

$\left(R_1\right)$

Partition $G_t$ into classes of the equivalence ∼.

$G_t={\left[i_1\right]}_{{}_{\sim }}\cup {\left[i_2\right]}_{{}_{\sim }}\cup \dots \cup {\left[i_p\right]}_{{}_{\sim }}\cup X$ where $X=\{i\in G_t|prev\left(i\right)=nil\}$ may be possibly empty and $i_1<i_2<\dots <i_p$.

$\left(R_2\right)$

Partition every class ${\left[i_{\ell }\right]}_{{}_{\sim }}$, $1\le \ell \le p$, into classes of the equivalence ≈.

${\left[i_{\ell }\right]}_{{}_{\sim }}={\left[j_{\ell ,1}\right]}_{{}_{\approx }}\cup {\left[j_{\ell ,2}\right]}_{{}_{\approx }}\cup \dots \cup {\left[j_{\ell ,m_{\ell }}\right]}_{{}_{\approx }}$ where $val\left(j_{\ell ,1}\right)<val\left(j_{\ell ,2}\right)<\dots <val\left(j_{\ell ,m_{\ell }}\right)$.

$\left(R_3\right)$

Therefore, we have a list of classes in this order: ${\left[j_{1,1}\right]}_{{}_{\approx }}$, ${\left[j_{1,2}\right]}_{{}_{\approx }}$, ... ${\left[j_{1,m_1}\right]}_{{}_{\approx }}$, ${\left[j_{2,1}\right]}_{{}_{\approx }}$, ${\left[j_{2,2}\right]}_{{}_{\approx }}$, ... ${\left[j_{2,m_2}\right]}_{{}_{\approx }}$, ..., ${\left[j_{p,1}\right]}_{{}_{\approx }}$, ${\left[j_{p,2}\right]}_{{}_{\approx }}$, ... ${\left[j_{p,m_p}\right]}_{{}_{\approx }}$. This list is processed from left to right. Note that for each $i\in {\left[j_{\ell ,𝓀}\right]}_{{}_{\approx }}$, $prev\left(i\right)\in gr\left(j_{\ell ,𝓀}\right)$, and $val\left(i\right)=val\left(j_{\ell ,𝓀}\right)$.

For each $j_{\ell ,𝓀}$, move all elements $\left\{prev\left(i\right)\right|i\in {\left[j_{\ell ,𝓀}\right]}_{{}_{\approx }}\}$ from the group $gr\left(prev\left(j_{\ell ,𝓀}\right)\right)$ into a new group H, place H in the list of groups right after the group $gr\left(prev\left(j_{\ell ,𝓀}\right)\right)$, and set its context to $con\left(gr\left(prev\left(j_{\ell ,𝓀}\right)\right)\right)con{\left(gr\left(j_{\ell ,𝓀}\right)\right)}^{val\left(j_{\ell ,𝓀}\right)}$. (Note, that this “doubling of the contexts” is possible due to $\left(C_6\right))$. Then, update $prev$:

• All values of $prev$ are correct except possibly the values of $prev$ for indices from H. It may be the case that for $i\in H$, there is $i^{\prime }\in gr\left(j_{\ell ,𝓀}\right)$, so that $prev\left(i\right)<i^{\prime }$, so $prev\left(i\right)$ must be reset to the maximal such $i^{\prime }$. (Note that before the removal of H from $gr\left(j_{\ell ,𝓀}\right)$, the index $i^{\prime }$ was not eligible to be considered for $prev\left(i\right)$ as i and $i^{\prime }$ were both from the same group.)

Theorem 3 shows that having a t-complete group configuration $\langle G_k,\dots ,$$G_{t+1},G_t,\dots ,G_1\rangle$ and refining it by $G_t$, then the resulting system of groups is a $(t+1)$-complete group configuration. This allows carrying out the refinement in an iterative fashion.

Theorem 3.

Let $\mathit{Conf}=\langle G_k,\dots ,G_{t+1},G_t,\dots ,G_1\rangle$ be a t-complete group configuration, $1\le t$. After performing the refinement of Conf by group $G_t$, the resulting system of groups denoted as $\mathit{Conf}{}^{\prime }$ is a $(t+1)$-complete group configuration.

Proof. 

We carry the proof in a series of claims. The symbols $gr\left(\right)$, $con\left(\right)$, $\mathcal{C}\left(\right)$, $prev\left(\right)$, and $val\left(\right)$ denote the functions for Conf, while $g{r}^{\prime }\left(\right)$, $co{n}^{\prime }\left(\right)$, ${\mathcal{C}}^{\prime }\left(\right)$, $pre{v}^{\prime }\left(\right)$, and $va{l}^{\prime }\left(\right)$ denote the functions for $\mathit{Conf}{}^{\prime }$.

When a group $G_{t+1}$ is partitioned, a part of it is moved as the next group in the list, and we call it $H_{t+1}$; thus, $G_{t+1}\prec H_{t+1}\prec G_t$. For details, please see ($R_3$) above.

Claim 1.

$\mathit{Conf}{}^{\prime }$is a group configuration, i.e., $\left(C_1\right)$, $\left(C_2\right)$, $\left(C_3\right)$, and $\left(C_4\right)$ for $\mathit{Conf}{}^{\prime }$ hold.

Proof of Claim 1.

($C_1$) and ($C_2$) follow from the fact that the process is a refinement, i.e., a group is either preserved as is or is partitioned into two or more groups. The doubling of the contexts in Step $\left(R_3\right)$ guarantees that the increasing order of the contexts is preserved, i.e., ($C_3$) holds. For any $j\in G_t$ so that $j=prev\left(i\right)\ne nil$, $con\left(gr\right(prev\left(j\right)\left)\right)$ is Lyndon, and $con\left(gr\right(j\left)\right)$ is also Lyndon, while $con\left(gr\right(prev\left(j\right)\left)\right)\prec con\left(gr\right(j\left)\right)$, so $con\left(gr\left(prev\left(j\right)\right)\right)con{\left(gr\left(j\right)\right)}^{val\left(j\right)}$ is Lyndon as well; thus, $\left(C_4\right)$ holds.

To illustrate the concatenation: let us call $con\left(gr\right(prev\left(j\right)\left)\right)$ as A and $con\left(gr\right(j\left)\right)$ as B, and let $val\left(j\right)=m$, then we know that A is Lyndon and B is Lyndon and $A\prec B$; so, $A{B}^m$ is clearly Lyndon as if A and B were letters. □

This concludes the proof of Claim 1.

Claim 2.

$\left\{{\mathcal{C}}^{\prime }\left(\mathit{i}\right)\right|\mathit{i}\in \mathbf{1}..\mathit{n}\}$is proper and has the Monge property, i.e., $\left(C_7\right)$ and $\left(C_8\right)$ for $\mathit{Conf}{}^{\prime }$ hold.

Proof of Claim 2.

Consider ${\mathcal{C}}^{\prime }\left(i\right)$ for some $i\in 1..n$. There are two possibilities:

• ${\mathcal{C}}^{\prime }\left(i\right)=\mathcal{C}\left(i\right)$ or
• ${\mathcal{C}}^{\prime }\left(i\right)=\mathcal{C}\left(i\right)\mathcal{C}\left(i_1\right)\dots \mathcal{C}\left(i_v\right)$, for some $i_1,i_2,\dots ,i_v\in G_t$, so that for any $1\le \ell \le v$, $i=prev\left(i_{\ell }\right)$, $\mathcal{C}\left(i_{\ell }\right)=con\left(G_t\right)$, $v=val\left(i_{\ell }\right)$ and for any $1\le \ell <k$ and $i_{\ell +1}=i_{\ell }+\left|con\left(G_t\right)\right|$. Note that $con\left(gr\left(i\right)\right)\prec con\left(G_t\right)$.

Consider ${\mathcal{C}}^{\prime }\left(i\right)$ and ${\mathcal{C}}^{\prime }\left(j\right)$ for some $1\le i<j\le n$.

• Case ${\mathcal{C}}^{\prime }\left(i\right)=\mathcal{C}\left(i\right)$ and ${\mathcal{C}}^{\prime }\left(j\right)=\mathcal{C}\left(j\right)$.

  (a)

  Show that $\left(C_7\left(a\right)\right)$ holds.

  If ${\mathcal{C}}^{\prime }\left(j\right)⊊{\mathcal{C}}^{\prime }\left(i\right)$, then $\mathcal{C}\left(j\right)⊊\mathcal{C}\left(i\right)$, and so, by $\left(C_7\left(a\right)\right)$ for Conf, $con\left(G_t\right)\prec con\left(gr\left(j\right)\right)$, and thus, $co{n}^{\prime }\left(H_{t+1}\right)\prec con\left(G_t\right)\prec con\left(gr\left(j\right)\right)=co{n}^{\prime }\left(g{r}^{\prime }\left(j\right)\right)$. Therefore, $\left(C_7\left(a\right)\right)$ for $\mathit{Conf}{}^{\prime }$ holds.

  (b)

  Show that $\left(C_8\right)$ holds. If ${\mathcal{C}}^{\prime }\left(i\right)\cap {\mathcal{C}}^{\prime }\left(j\right)\ne \varnothing$, then $\mathcal{C}\left(i\right)\cap \mathcal{C}\left(j\right)\ne \varnothing$, so $\mathcal{C}\left(j\right)\subseteq \mathcal{C}\left(i\right)$, and so, ${\mathcal{C}}^{\prime }\left(j\right)\subseteq {\mathcal{C}}^{\prime }\left(i\right)$; so, $\left(C_8\right)$ for $\mathit{Conf}{}^{\prime }$ holds.

• Case ${\mathcal{C}}^{\prime }\left(i\right)=\mathcal{C}\left(i\right)$ and ${\mathcal{C}}^{\prime }\left(j\right)=\mathcal{C}\left(j\right)\mathcal{C}\left(j_1\right)..\mathcal{C}\left(j_w\right)$,
  where $w=val\left(j_1\right)$, $\mathcal{C}\left(j_1\right)=\dots =\mathcal{C}\left(j_w\right)=con\left(G_t\right)$, and $j_1\approx \dots \approx j_w$.

  (a)

  Show that $\left(C_7\left(a\right)\right)$ holds.

  If ${\mathcal{C}}^{\prime }\left(j\right)⊊{\mathcal{C}}^{\prime }\left(i\right)$, then $\mathcal{C}\left(j\right)\mathcal{C}\left(j_1\right)..\mathcal{C}\left(j_w\right)⊊\mathcal{C}\left(i\right)$; hence, $\mathcal{C}\left(j\right)⊊\mathcal{C}\left(i\right)$, and so, by $\left(C_7\left(a\right)\right)$ for Conf, $con\left(G_t\right)\prec con\left(gr\left(j\right)\right)$. By the t-completeness of Conf, $\mathcal{C}\left(j\right)$ is a right-maximal Lyndon substring, a contradiction with $\mathcal{C}\left(j\right)\mathcal{C}\left(j_1\right)..,\mathcal{C}\left(j_w\right)$ being Lyndon. This is an impossible case.

  (b)

  Show that $\left(C_8\right)$ holds.

  If ${\mathcal{C}}^{\prime }\left(i\right)\cap {\mathcal{C}}^{\prime }\left(j\right)\ne \varnothing$, then $\mathcal{C}\left(j\right)\subseteq \mathcal{C}\left(i\right)$ by $\left(C_8\right)$ for Conf. By $\left(C_7\left(a\right)\right)$ for Conf, $\mathcal{C}\left(j\right)$ cannot be a suffix of $\mathcal{C}\left(i\right)$ as $con\left(gr\left(j\right)\right)\prec con\left(G_t\right)$. Hence, $\mathcal{C}\left(i\right)\cap \mathcal{C}\left(j_1\right)\ne \varnothing$, and so, $\mathcal{C}\left(j\right)\mathcal{C}\left(j_1\right)\subseteq \mathcal{C}\left(i\right)$; and since $\mathcal{C}\left(j_1\right)$ cannot be a suffix of $\mathcal{C}\left(i\right)$ as $gr\left(j_1\right)=G_t$, it follows that $\mathcal{C}\left(i\right)\cap \mathcal{C}\left(j_2\right)\ne \varnothing$, ..., ultimately giving $\mathcal{C}\left(j\right)\mathcal{C}\left(j_1\right)\dots \mathcal{C}\left(j_w\right)\subseteq \mathcal{C}\left(i\right)$. Therefore, $\left(C_8\right)$ for $\mathit{Conf}{}^{\prime }$ holds.

• Case ${\mathcal{C}}^{\prime }\left(i\right)=\mathcal{C}\left(i\right)\mathcal{C}\left(i_1\right)..\mathcal{C}\left(i_v\right)$ and ${\mathcal{C}}^{\prime }\left(j\right)=\mathcal{C}\left(j\right)$,
  where $v=val\left(i_1\right)$, $\mathcal{C}\left(i_1\right)=\dots =\mathcal{C}\left(i_v\right)=con\left(G_t\right)$, and $i_1\approx \dots \approx i_v$.

  (a)

  Show that $\left(C_7\left(a\right)\right)$ holds.

  If ${\mathcal{C}}^{\prime }\left(j\right)⊊{\mathcal{C}}^{\prime }\left(i\right)$, then either $\mathcal{C}\left(j\right)⊊\mathcal{C}\left(i\right)$, which implies by $\left(C_7\left(a\right)\right)$ for Conf that $con\left(G_t\right)\prec con\left(gr\left(j\right)\right)$, giving $co{n}^{\prime }\left(H_{t+1}\right)\prec co{n}^{\prime }\left(G_t\right)=con\left(G_t\right)\prec con\left(gr\left(j\right)\right)=co{n}^{\prime }\left(g{r}^{\prime }\left(j\right)\right)$, or $\mathcal{C}\left(j\right)\subseteq \mathcal{C}\left(i_{\ell }\right)$ for some $1\le \ell \le v$. If $\mathcal{C}\left(j\right)=\mathcal{C}\left(i_{\ell }\right)$, then $gr\left(j\right)=gr\left(i_{\ell }\right)=G_t$, giving $co{n}^{\prime }\left(H_{t+1}\right)\prec con\left(G_t\right)=con\left(gr\left(j\right)\right)$. Therefore $\left(C_7\left(a\right)\right)$ for $\mathit{Conf}{}^{\prime }$ holds.

  (b)

  Show that $\left(C_8\right)$ holds.

  Let ${\mathcal{C}}^{\prime }\left(i\right)\cap {\mathcal{C}}^{\prime }\left(j\right)\ne \varnothing$. Consider $\mathcal{D}=\left\{i_{\ell }\right|1\le \ell \le vand\mathcal{C}\left(j\right)\cap \mathcal{C}\left(i_{\ell }\right)\ne \varnothing \}$.

  Assume that $\mathcal{D}\ne \varnothing$:

  • By $\left(C_8\right)$ for Conf, either $\mathcal{C}\left(j\right)\subseteq {\bigcup }_{i_{\ell }\in \mathcal{D}}\mathcal{C}\left(i_{\ell }\right)\subseteq {\mathcal{C}}^{\prime }\left(i\right)$, and we are done, or ${\bigcup }_{i_{\ell }\in \mathcal{D}}\mathcal{C}\left(i_{\ell }\right)\subseteq \mathcal{C}\left(j\right)$. Let $i_{𝓀}$ be the smallest element of $\mathcal{D}$. Since $\mathcal{C}\left(i_{𝓀}\right)$ cannot be the prefix of $\mathcal{C}\left(j\right)$, it means that $i_{𝓀}=i_1$. Since $\mathcal{C}\left(i_1\right)$ cannot be a prefix of $\mathcal{C}\left(j\right)$, it means that $\mathcal{C}\left(i\right)\cap \mathcal{C}\left(j\right)\ne \varnothing$, and so, $\mathcal{C}\left(j\right)\subseteq \mathcal{C}\left(i\right)$, which contradicts the fact that $\mathcal{C}\left(j\right)\subseteq {\bigcup }_{i_{\ell }\in \mathcal{D}}\mathcal{C}\left(i_{\ell }\right)\subseteq {\mathcal{C}}^{\prime }\left(i\right)$.

  Assume that $\mathcal{D}=\varnothing$:

  • Then, $\mathcal{C}\left(i\right)\cap \mathcal{C}\left(j\right)\ne \varnothing$, and so, by $\left(C_8\right)$ for Conf, $\mathcal{C}\left(j\right)\subseteq \mathcal{C}\left(i\right)\subseteq {\mathcal{C}}^{\prime }\left(i\right)$ as $i<j$.

• Case ${\mathcal{C}}^{\prime }\left(i\right)=\mathcal{C}\left(i\right)\mathcal{C}\left(i_1\right)..\mathcal{C}\left(i_v\right)$ and ${\mathcal{C}}^{\prime }\left(j\right)=\mathcal{C}\left(j\right)\mathcal{C}\left(j_1\right)\dots \mathcal{C}\left(j_w\right)$,
  where $v=val\left(i_1\right)$, $\mathcal{C}\left(i_1\right)=\dots =\mathcal{C}\left(i_v\right)=con\left(G_t\right)$, and $i_1\approx \dots \approx i_v$ and where $v=val\left(j_1\right)$, $\mathcal{C}\left(j_1\right)=\dots =\mathcal{C}\left(j_w\right)=con\left(G_t\right)$, and $j_1\approx \dots \approx j_w$.

  (a)

  Show that $\left(C_7\left(a\right)\right)$ holds.

  Let ${\mathcal{C}}^{\prime }\left(j\right)⊊{\mathcal{C}}^{\prime }\left(i\right)$. Then, either $\mathcal{C}\left(j\right)\subseteq \mathcal{C}\left(i\right)$, and so, $con\left(G_t\right)\prec con\left(gr\left(j\right)\right)$, implying that $\mathcal{C}\left(j\right)$ is maximal contradicting $\mathcal{C}\left(j\right)\mathcal{C}\left(j_1\right)\dots \mathcal{C}\left(j_w\right)$ being Lyndon. Thus, $\mathcal{C}\left(j\right)⊊\mathcal{C}\left(i_{\ell }\right)$ for some $1\le \ell \le v$. However, then, $con\left(G_t\right)\prec con\left(gr\left(j\right)\right)$, implying that $\mathcal{C}\left(j\right)$ is maximal, again a contradiction. This is an impossible case.

  (b)

  Show that $\left(C_8\right)$ holds.

  Let ${\mathcal{C}}^{\prime }\left(i\right)\cap {\mathcal{C}}^{\prime }\left(j\right)\ne \varnothing$. Let us first assume that $\mathcal{C}\left(i\right)\cap \mathcal{C}\left(j\right)\ne \varnothing$. Then, $\mathcal{C}\left(j\right)\subseteq \mathcal{C}\left(i\right)$. Since $\mathcal{C}\left(j\right)$ cannot be a suffix of $\mathcal{C}\left(i\right)$, it follows that $\mathcal{C}\left(i\right)\cap \mathcal{C}\left(j_1\right)\ne \varnothing$. Therefore, $\mathcal{C}\left(j\right)\mathcal{C}\left(j_1\right)\subseteq \mathcal{C}\left(i\right)$. Repeating this argument leads to $\mathcal{C}\left(j\right)\mathcal{C}\left(j_1\right)\dots \mathcal{C}\left(j_w\right)\subseteq \mathcal{C}\left(i\right)$, and we are done.

  Therefore, assume that $\mathcal{C}\left(i\right)\cap \mathcal{C}\left(j\right)=\varnothing$. Let $1\le \ell \le v$ be the smallest such that $\mathcal{C}\left(i_{\ell }\right)\cap \mathcal{C}\left(j\right)\ne \varnothing$. Such an ℓ must exist. Then, $i_{\ell }\le j$. If $i_{\ell }=j$, then either $\mathcal{C}\left(i_{\ell }\right)$ is a prefix of $\mathcal{C}\left(j\right)$ or vice versa, both impossibilities; hence, $i_{\ell }<j$. Repeating the same arguments as for i, we get that $\mathcal{C}\left(j\right)\mathcal{C}\left(j_1\right)..\mathcal{C}\left(j_w\right)\subseteq \mathcal{C}\left(i_{\ell }\right)$, and so, we are done.

It remains to show that $\left(C_7\left(b\right)\right)$ for $\mathit{Conf}{}^{\prime }$ holds.

Consider ${\mathcal{C}}^{\prime }\left(i\right)$ immediately followed by ${\mathcal{C}}^{\prime }\left(j\right)$ with ${\mathcal{C}}^{\prime }\left(i\right)\prec {\mathcal{C}}^{\prime }\left(j\right)$.

• Assume that $g{r}^{\prime }\left(j\right)\in \{G_{t-1},\dots ,G_1\}$.
  Then, $con\left(G_t\right)=co{n}^{\prime }\left(G_t\right)$, $gr\left(j\right)=g{r}^{\prime }\left(j\right)$, and $con\left(gr\left(j\right)\right)=co{n}^{\prime }\left(g{r}^{\prime }\left(j\right)\right)$. If ${\mathcal{C}}^{\prime }\left(i\right)=\mathcal{C}\left(i\right)$, then $\mathcal{C}\left(i\right)\prec \mathcal{C}\left(j\right)$, and $\mathcal{C}\left(i\right)$ is immediately followed by $\mathcal{C}\left(j\right)$, so by $\left(C_7\left(b\right)\right)$ for Conf, we have a contradiction. Thus, ${\mathcal{C}}^{\prime }\left(i\right)=\mathcal{C}\left(i\right)\mathcal{C}\left(i_1\right)\dots \mathcal{C}\left(i_v\right)$ for $v=val\left(i\right)$ and $con\left(gr\left(i_v\right)\right)=con\left(G_t\right)\prec con\left(gr\left(j\right)\right)$, and $\mathcal{C}\left(i_v\right)$ is immediately followed by $\mathcal{C}\left(j\right)$, a contradiction by $\left(C_7\left(b\right)\right)$ for Conf.
• Assume that $g{r}^{\prime }\left(j\right)=G_t$.
  Then, the group $gr\left(i\right)$ is partitioned when refining by $G_t$, and so, ${\mathcal{C}}^{\prime }\left(i\right)=co{n}^{\prime }\left(g{r}^{\prime }\left(i\right)\right)=con\left(gr\left(i\right)\right)\mathcal{C}{\left(j\right)}^v$ for $v=val\left(j\right)$. Since ${\mathcal{C}}^{\prime }\left(i\right)$ is immediately followed by ${\mathcal{C}}^{\prime }\left(j\right)=con\left(G_t\right)$, we have again a contradiction, as it implies that $val\left(j\right)=v+1$.

□

This concludes the proof of Claim 2.

Claim 3.

The function${\mathit{prev}}^{\prime }$is proper on${\mathit{H}}_{\mathit{t}+\mathbf{1}}$, i.e., $\left(C_6\right)$ for $\mathit{Conf}{}^{\prime }$ holds.

Proof of Claim 3.

Let $j=pre{v}^{\prime }\left(i\right)$ and $i\in H_{t+1}$ with $va{l}^{\prime }\left(i\right)=v$. Then, ${|\left[i\right]}_{{}_{\approx }}|=v$, and so, ${\left[i\right]}_{{}_{\approx }}=\{i_1,\dots ,i_v\}$, where $i_1<i_2<\dots <i_v$. Hence, $i_1,\dots ,i_v\in H_{t+1}$, ${\mathcal{C}}^{\prime }\left(i_1\right)=\dots ={\mathcal{C}}^{\prime }\left(i_v\right)=co{n}^{\prime }\left(H_{t+1}\right)$, and $j=pre{v}^{\prime }\left(i\right)=pre{v}^{\prime }\left(i_1\right)=\dots =pre{v}^{\prime }\left(i_v\right)$, and so, $j<i_1$. It remains to show that ${\mathcal{C}}^{\prime }\left(j\right){\mathcal{C}}^{\prime }\left(i_1\right)\dots {\mathcal{C}}^{\prime }\left(i_v\right)$ is a prefix of $x[j..n]$. It suffices to show that ${\mathcal{C}}^{\prime }\left(j\right)$ is immediately followed by ${\mathcal{C}}^{\prime }\left(i_1\right)$.

If ${\mathcal{C}}^{\prime }\left(j\right)\cap {\mathcal{C}}^{\prime }\left(i_1\right)\ne \varnothing$, then by the Monge property $\left(C_8\right)$, ${\mathcal{C}}^{\prime }\left(i_1\right)\subseteq {\mathcal{C}}^{\prime }\left(j\right)$ as $j<i_1$, and so, by $\left(C_7\left(a\right)\right)$, $co{n}^{\prime }\left(H_{t+1}\right)\prec co{n}^{\prime }\left(g{r}^{\prime }\left(i_1\right)\right)=co{n}^{\prime }\left(H_{t+1}\right)$, a contradiction.

Thus, ${\mathcal{C}}^{\prime }\left(j\right)\cap {\mathcal{C}}^{\prime }\left(i_1\right)=\varnothing$. Set $j_1=j+\left|co{n}^{\prime }\left(g{r}^{\prime }\left(j\right)\right)\right|$. It follows that $j_1\le i_1$. Assume that $j_1<i_1$. Since $j=pre{v}^{\prime }\left(i_1\right)$ and $j<i_1$, $co{n}^{\prime }\left(g{r}^{\prime }\left(j_1\right)\right)⪰co{n}^{\prime }\left(g{r}^{\prime }\left(i_1\right)\right)=co{n}^{\prime }\left(H_{t+1}\right)$. Since $j_1\notin H_{t+1}$, $co{n}^{\prime }\left(g{r}^{\prime }\left(j_1\right)\right)\succ co{n}^{\prime }\left(H_{t+1}\right)$. Consider ${\mathcal{C}}^{\prime }\left(j_1\right)$. If ${\mathcal{C}}^{\prime }\left(j_1\right)\cap {\mathcal{C}}^{\prime }\left(i_1\right)\ne \varnothing$, then by $\left(C_8\right)$, ${\mathcal{C}}^{\prime }\left(i_1\right)\subseteq {\mathcal{C}}^{\prime }\left(j_1\right)$, and so, by $\left(C_7\left(a\right)\right)$, $co{n}^{\prime }\left(H_{t+1}\right)\prec co{n}^{\prime }\left(g{r}^{\prime }\left(i_1\right)\right)=co{n}^{\prime }\left(H_{t+1}\right)$, a contradiction. Thus, ${\mathcal{C}}^{\prime }\left(j_1\right)\cap {\mathcal{C}}^{\prime }\left(i_1\right)=\varnothing$. Since ${\mathcal{C}}^{\prime }\left(j_1\right)$ immediately follows ${\mathcal{C}}^{\prime }\left(j\right)$, by $\left(C_7\left(b\right)\right)$, $co{n}^{\prime }\left(g{r}^{\prime }\left(j_1\right)\right)⪯co{n}^{\prime }\left(H_{t+1}\right)$, a contradiction. Therefore, $j_1=i_1$, and so, $pre{v}^{\prime }$ is proper on $H_{t+1}$. □

This concludes the proof of Claim 3.

Claim 4.

${\mathit{H}}_{\mathit{t}+\mathbf{1}}$is a complete group, i.e., $\left(C_5\right)$ for $\mathit{Conf}{}^{\prime }$ holds.

Proof of Claim 4.

Assume that there is $i\in H_{t+1}$ so that ${\mathcal{C}}^{\prime }\left(i\right)$ is not maximal, i.e., for some $k\ge i+|co{n}^{\prime }\left(H_{t+1}\right)|$, $x[i..k]$ is a right-maximal Lyndon substring of $\mathit{x}$.

Either $k=n$ and so $co{n}^{\prime }\left(g{r}^{\prime }\left(k\right)\right)=x\left[k\right]$, and so, ${\mathcal{C}}^{\prime }\left(k\right)$ is a suffix of $x[i..k]$, or $k<n$, and then, $x[k+1]\prec x\left[k\right]$, since $x[k+1]⪯x\left[k\right]$ implies that $x[i..k+1]$ is Lyndon, a contradiction with the right-maximality of $x[i..k]$. Consider ${\mathcal{C}}^{\prime }\left(k\right)$, then ${\mathcal{C}}^{\prime }\left(k\right)\subseteq x[i..k]$, and so, ${\mathcal{C}}^{\prime }\left(k\right)=x\left[k\right]$.

Therefore, there is $j_1$ so that $i+|co{n}^{\prime }\left(H_{t+1}\right)|\le j_1\le k$, and ${\mathcal{C}}^{\prime }\left(j_1\right)$ is a suffix of $x[i..k]$. Take the smallest such $j_1$. If $j_1=i+\left|co{n}^{\prime }\left(H_{t+1}\right)\right|$, then ${\mathcal{C}}^{\prime }\left(i\right)\prec {\mathcal{C}}^{\prime }\left(j_1\right)$ as $x[i..k]={\mathcal{C}}^{\prime }\left(i\right){\mathcal{C}}^{\prime }\left(j_1\right)$ is Lyndon. By $\left(C_7\left(b\right)\right)$, ${\mathcal{C}}^{\prime }\left(j_1\right)⪯co{n}^{\prime }\left(H_{t+1}\right)$, so we have $co{n}^{\prime }\left(H_{t+1}\right)={\mathcal{C}}^{\prime }\left(i\right)\prec {\mathcal{C}}^{\prime }\left(j_1\right)⪯co{n}^{\prime }\left(H_{t+1}\right)$, a contradiction.

Therefore, $j_1>i+\left|co{n}^{\prime }\left(H_{t+1}\right)\right|$. Consider $x[j_1-1]$. If $x[j_1-1]⪯x\left[j_1\right]$, $x[j_1-1..k]$ is Lyndon, and since $x[j_1..k]={\mathcal{C}}^{\prime }\left(j_1\right)$, $x[j_1-1..k]$ would be a context of $g{r}^{\prime }(j_1-1)$, this contradicts the fact $j_1$ was chosen to be the smallest such one. Therefore, $x[j_1-1]\succ x\left[j_1\right]$, and so, $co{n}^{\prime }\left(g{r}^{\prime }(j_1-1)\right)=x[j_1-1]$. Thus, there is $j_2$, $i+|co{n}^{\prime }\left(H_{t+1}\right)|\le j_2<j_1\le k$, and ${\mathcal{C}}^{\prime }\left(j_2\right)$ is a suffix of $x[i..j_1-1]$. Take the smallest such $j_2$. If ${\mathcal{C}}^{\prime }\left(j_2\right)\prec {\mathcal{C}}^{\prime }\left(j_1\right)$, then by $\left(C_7\left(b\right)\right)$, ${\mathcal{C}}^{\prime }\left(j_1\right)⪯co{n}^{\prime }\left(H_{t+1}\right)$, a contradiction. Hence, ${\mathcal{C}}^{\prime }\left(j_2\right)⪰{\mathcal{C}}^{\prime }\left(j_1\right)$. If $j_2=i+i+\left|co{n}^{\prime }\left(H_{t+1}\right)\right|$, then $x[i..k]={\mathcal{C}}^{\prime }\left(i\right){\mathcal{C}}^{\prime }\left(j_2\right){\mathcal{C}}^{\prime }\left(j_1\right)$, and so, by $\left(C_7\left(b\right)\right)$, ${\mathcal{C}}^{\prime }\left(j_2\right)⪯co{n}^{\prime }\left(H_{t+1}\right)$, a contradiction. Hence, $i+|co{n}^{\prime }\left(H_{t+1}\right)|<j_2$.

The same argument done for $j_2$ can now be done for $j_3$. We end up with $i+|co{n}^{\prime }\left(H_{t+1}\right)|\le j_3<j_2<j_1\le k$ and with ${\mathcal{C}}^{\prime }\left(j_3\right)⪰{\mathcal{C}}^{\prime }\left(j_2\right)⪰{\mathcal{C}}^{\prime }\left(j_1\right)\succ co{n}^{\prime }\left(H_{t+1}\right)$. If $i+|co{n}^{\prime }\left(H_{t+1}\right)|=j_3$, then we have a contradiction, so $i+|co{n}^{\prime }\left(H_{t+1}\right)|<j_3$. These arguments can be repeated only finitely many times, and we obtain $i+|co{n}^{\prime }\left(H_{t+1}\right)|=j_{\ell }<j_{\ell -1}<\dots <j_2<j_1\le k$ so that $x[i..k]={\mathcal{C}}^{\prime }\left(i\right){\mathcal{C}}^{\prime }\left(j_{\ell }\right){\mathcal{C}}^{\prime }(j_{\ell -1}\dots {\mathcal{C}}^{\prime }\left(j_2\right){\mathcal{C}}^{\prime }\left(j_1\right)$, which is a contradiction.

Therefore, our initial assumption that ${\mathcal{C}}^{\prime }\left(i\right)$ is not maximal always leads to a contradiction. □

This concludes the proof of Claim 4.

The four claims show that all the conditions $\left(C_1\right)$ ... $\left(C_8\right)$ are satisfied for $\mathit{Conf}{}^{\prime }$, and that proves Theorem 3.

As the last step, we show that when the process of refinement is completed, all right-maximal Lyndon substrings of $\mathit{x}$ are identified and sorted via the contexts of the groups of the final configuration.

Theorem 4.

Let ${\mathit{Conf}}_1=\langle G_{k_1}^1,G_{k_1-1}^1,\dots ,G_2^1,G_1^1\rangle$ with $g{r}_1\left(\right)$, $co{n}_1\left(\right)$, $C_1\left(\right)$, $pre{v}_1\left(\right)$, and $va{l}_1\left(\right)$ be the initial 1complete group configuration from Lemma 9.

Let ${\mathit{Conf}}_2=\langle G_{k_2}^2,G_{k_2-1}^2,\dots ,G_2^2,G_1^2\rangle$ with $g{r}_2\left(\right)$, $co{n}_2\left(\right)$, $C_2\left(\right)$, $pre{v}_2\left(\right)$, and $va{l}_2\left(\right)$ be the 2complete group configuration obtained from ${\mathit{Conf}}_1$ through the refinement by the group $G_1^1$.

Let ${\mathit{Conf}}_3=\langle G_{k_3}^3,G_{k_3-1}^3,\dots ,G_2^3,G_1^3\rangle$ with $g{r}_3\left(\right)$, $co{n}_3\left(\right)$, $C_3\left(\right)$, $pre{v}_3\left(\right)$, and $va{l}_3\left(\right)$ be the 3complete group configuration obtained from ${\mathit{Conf}}_2$ through the refinement by the group $G_2^2$.

...

Let ${\mathit{Conf}}_r=\langle G_{k_r}^r,G_{k_r-1}^r,\dots ,G_2^r,G_1^r\rangle$ with $g{r}_r\left(\right)$, $co{n}_r\left(\right)$, $C_r\left(\right)$, $pre{v}_r\left(\right)$, and $va{l}_r\left(\right)$ be the rcomplete group configuration obtained from ${\mathit{Conf}}_{r-1}$ through the refinement by the group $G_{r-1}^{r-1}$. Let ${\mathit{Conf}}_r$ be the final configuration after the refinement runs out.

Then, $x[i..k]$ is a right-maximal Lyndon substring of $\mathit{x}$ iff $x[i..k]=C_r\left(i\right)=co{n}_r\left(g{r}_r\left(i\right)\right)$.

Proof. 

That all the groups of ${\mathit{Conf}}_r$ are complete follows from Theorem 3, and hence, every $C_r\left(i\right)$ is a right-maximal Lyndon string. Let $x[i..k]$ be a right-maximal Lyndon substring of $\mathit{x}$. Consider $C_r\left(i\right)$; since it is maximal, it must be equal to $x[i..k]$. □

4.3. Motivation for the Refinement

The process of refinement is in fact a process of the gradual revealing of the Lyndon substrings, which we call the water draining method:

(a)

lower the water level by one;

(b)

extend the existing Lyndon substrings; the revealed letters are used to extend the existing Lyndon substrings where possible, or became Lyndon substrings of length one otherwise;

(c)

consolidate the new Lyndon substrings; processed from the right, if several Lyndon substrings are adjacent and can be joined to a longer Lyndon substring, they are joined.

The diagram in Figure 5 and the description that follows it illustrate the method for a string 011023122. The input string is visualized as a curve, and the height at each point is the value of the letter at that position.

In Figure 5, we illustrate the process:

(1)

We start with the string 011023122 and a full tank of water.

(2)

We drain one level; only 3 is revealed; there is nothing to extend, nothing to consolidate.

(3)

We drain one more level, and three 2’s are revealed; the first 2 extends 3 to 23, and the remaining two 2’s form Lyndon substrings 2 of length one; there is nothing to consolidate.

(4)

We drain one more level, and three 1’s are revealed; the first two 1’s form Lyndon substrings 1 of length one; the third 1 extends 22 to 122; there is nothing to consolidate.

(5)

We drain one more level, and two 0’s are revealed; the first 0 extends 11 to 011; the second 0 extends 23 to 023; in the consolidation phase, 023 is joined with 122 to form a Lyndon substring 023122, and then, 011 is joined with 023122 to form a Lyndon substring 011023122.

Therefore, during the process, the following right-maximal Lyndon substrings were identified: 3 at Position 6, 23 at Position 5, 2 at Positions 8 and 9, 1 at Positions 2 and 3, 122 at Position 7, 023 at Position 4, and finally, 011023122 at Position 1. Note that all positions are accounted for; we really have all right-maximal Lyndon substrings of the string 011023122.

In Figure 6, we present an illustrative example for the string 011023122, where the arrows represent the $prev$ mapping shown only on the group used for the refinement. The groups used for the refinement are indicated by the bold font.

4.4. The Complexity of BSLA

The computation of the initial configuration can be done in linear time. To compute the initial value of $\mathit{prev}$ in linear time, a stack-based approach similar to the NSV algorithm is used. Since all groups are non-empty, there can never be more groups than n. Theorem 3 is at the heart of the algorithm. The refinement by the last completed group is linear in the size of the group, including the update of $\mathit{prev}$. Therefore, the overall worst-case complexity of BSLA is linear in the length of the input string.

5. Data and Measurements

Initially, computations were performed on the Department of Computing and Software’s moore server; memory: 32 GB (DDR4 @ 2400 MHz), CPU: 8 × Intel Xeon E5-2687W v4 @ 3.00 GHz, OS: Linux Version 2.6.18-419.el5 (gcc Version 4.1.2 and Red Hat Version 4.1.2-55). To verify correctness, new randomized data were produced and computed independently on the University of Toronto Mississauga’s octolab cluster; memory: 8 × 32 GB (DDR4 @ 3200 MHz), CPU: 8 × AMD Ryzen Threadripper 1920X (12-Core) @ 4.00 GHz, OS: Ubuntu 16.04.6 LTS (gcc Version 5.4.0). The results of both were extremely similar, and those reported herein are those generated using the moore server. All the programs were compiled without any additional level of optimization (i.e., neither -O1, nor -O2, nor -O3 flags were specified for the compilation). The CPU time was measured in clock ticks with 1,000,000 clock ticks per second. Since the execution time was negligible for short strings, the processing of the same string was repeated several times (the repeat factor varied from ${10}^6$, for strings of length 10, to one, for strings of length $5\times {10}^6$), resulting in a higher precision. Thus, for graphing, the logarithmic scale was used for both, x-axis representing the length of the strings and y-axis representing the time. We used four categories of randomly generated datasets:

(1)

bin

random strings over an integer alphabet with exactly two distinct letters (kind of binary strings).

(2)

dna

random strings over an integer alphabet with exactly four distinct letters (kind of random DNA strings).

(3)

eng

random strings over an integer alphabet with exactly 26 distinct letters (kind of random English).

(4)

int

random strings over an integer alphabet (i.e., over the alphabet $\{0,\dots ,n-1\}$).

Each dataset contains 100 randomly generated strings of the same length. For each category, there were datasets for length s 10, 50, ${10}^2$, $5\times {10}^2$, ..., ${10}^5$, $5\times {10}^5$, ${10}^6$, and $5\times {10}^6$. The minimum, average, and maximum times for each dataset were computed. Since the variance for each dataset was minimal, the results for minimum times and the results for maximum times completely mimicked the results for the average times, so we only present the averages here.

Tables 1–4 and the graphs in Figures 7–10 from Section 7 clearly indicate that the performance of the three algorithms is linear and virtually indistinguishable. We expected IDLA and TRLA to exhibit linear behaviour on random strings as such strings tend to have many, but short right-maximal Lyndon substrings. However, we did not expect the results to be so close.

Despite the fact that IDLA performed in linear time on the random strings, it is relatively easy to force it into its worst quadratic performance. The dataset extreme_idla contains individual strings $0123\dots n-1$ of the required lengths. Table 5 and the graph in Figure 11 from Section 7 show this clearly.

In Section 3.5, we describe how the three datasets, extreme_trla, extreme_trla1, and extreme_trla2, are generated and why. The results of experimenting with these datasets do not suggest that the worst-case complexity for TRLA is $\mathcal{O}(nlog(n\left)\right)$. Yet again, the performances of the three algorithms are linear and virtually indistinguishable; see Tables 6–8 and the graphs in Figures 12–14 in Section 7.

6. Conclusions and Future Work

We present two novel algorithms for computing right-maximal Lyndon substrings. The first one, TRLA, has a simple implementation with a complicated theory behind it. Its average time complexity is linear in the length of the input string, and its worst-case complexity is no worse than $\mathcal{O}(nlog(n\left)\right)$. The $\tau$-reduction used in the algorithm is an interesting reduction preserving right-maximal Lyndon substrings, a fact used significantly in the design of the algorithm. Interestingly, it seem to slightly outperform BSLA, at least on the datasets used for our experimentations. BSLA, the second algorithm, is linear and elementary in the sense that it does not require a pre-processed global data structure. Being linear and elementary, BSLA is more interesting, and it is possible that its performance could be more streamlined. However, both the theory and implementation of BSLA are rather complex.

On random strings, none of the two algorithms were significantly better than the simple IDLA, whose implementation is just a few lines. However, its quadratic worst-case complexity is an obstacle, as our experiments indicated.

Additional effort needs to go into proving TRLA’s worst-case complexity. The experiments performed did not indicate that it is not linear even in the worst case. Both algorithms need to be compared to some efficient implementation of SSLA and BWLA.

7. Results

This section contains the measurements of the average times for the datasets discussed in the previous section. For better understanding of the data, we present them in

Table 1. Average times for dataset bin (${10}^6$ clock ticks per second).

String Length	Time in Ticks	Time in Ticks	Time in Ticks
	IDLA	TRLA	BSLA
10	$3.651\times {10}^{-1}$	$1.582$	$1.054$
50	$4.082$	$1.050\times 10$	$6.372$
100	$1.101\times 10$	$2.140\times 10$	$1.277\times 10$
500	$8.655\times 10$	$1.127\times {10}^2$	$6.786\times 10$
1000	$1.975\times {10}^2$	$2.335\times {10}^2$	$1.484\times {10}^2$
5000	$1.278\times {10}^3$	$1.218\times {10}^3$	$8.595\times {10}^2$
10,000	$2.765\times {10}^3$	$2.423\times {10}^3$	$1.820\times {10}^3$
50,000	$1.665\times {10}^4$	$1.272\times {10}^4$	$1.018\times {10}^4$
100,000	$3.606\times {10}^4$	$2.523\times {10}^4$	$2.113\times {10}^4$
500,000	$2.071\times {10}^5$	$1.338\times {10}^5$	$1.493\times {10}^5$
1,000,000	$4.387\times {10}^5$	$2.717\times {10}^5$	$4.080\times {10}^5$
5,000,000	$2.483\times {10}^6$	$1.561\times {10}^6$	$3.098\times {10}^6$

,

Table 2. Average times for dataset dna (${10}^6$ clock ticks per second).

String Length	Time in Ticks	Time in Ticks	Time in Ticks
	IDLA	TRLA	BSLA
10	$3.699\times {10}^{-1}$	$1.579$	$1.080$
50	$3.509$	$1.037\times 10$	$6.627$
100	$8.898$	$2.109\times 10$	$1.403\times 10$
500	$6.403\times 10$	$1.123\times {10}^2$	$7.228\times 10$
1000	$1.431\times {10}^2$	$2.332\times {10}^2$	$1.544\times {10}^2$
5000	$8.749\times {10}^2$	$1.207\times {10}^3$	$9.039\times {10}^2$
10,000	$1.912\times {10}^3$	$2.460\times {10}^3$	$1.935\times {10}^3$
50,000	$1.134\times {10}^4$	$1.280\times {10}^4$	$1.110\times {10}^4$
100,000	$2.431\times {10}^4$	$2.588\times {10}^4$	$2.316\times {10}^4$
500,000	$1.383\times {10}^5$	$1.390\times {10}^5$	$1.781\times {10}^5$
1,000,000	$2.916\times {10}^5$	$2.865\times {10}^5$	$4.994\times {10}^5$
5,000,000	$1.643\times {10}^6$	$1.968\times {10}^6$	$3.752\times {10}^6$

,

Table 3. Average times for dataset eng (${10}^6$ clock ticks per second).

String Length	Time in Ticks	Time in Ticks	Time in Ticks
	IDLA	TRLA	BSLA
10	$3.526\times {10}^{-1}$	$1.584$	$9.865\times {10}^{-1}$
50	$3.162$	$1.006\times 10$	$5.960$
100	$7.315$	$2.057\times 10$	$1.317\times 10$
500	$4.996\times 10$	$1.117\times {10}^2$	$7.245\times 10$
1000	$1.112\times {10}^2$	$2.354\times {10}^2$	$1.542\times {10}^2$
5000	$6.722\times {10}^2$	$1.210\times {10}^3$	$9.087\times {10}^2$
10,000	$1.452\times {10}^3$	$2.427\times {10}^3$	$2.042\times {10}^3$
50,000	$8.505\times {10}^3$	$1.306\times {10}^4$	$1.301\times {10}^4$
100,000	$1.802\times {10}^4$	$2.688\times {10}^4$	$2.768\times {10}^4$
500,000	$1.025\times {10}^5$	$1.428\times {10}^5$	$2.381\times {10}^5$
1,000,000	$2.171\times {10}^5$	$3.253\times {10}^5$	$7.236\times {10}^5$
5,000,000	$1.206\times {10}^6$	$2.599\times {10}^6$	$6.092\times {10}^6$

,

Table 4. Average times for dataset int (${10}^6$ clock ticks per second).

String Length	Time in Ticks	Time in Ticks	Time in Ticks
	IDLA	TRLA	BSLA
10	$3.547\times {10}^{-1}$	$1.645$	$9.794\times {10}^{-1}$
50	$3.032$	$9.992$	$5.609$
100	$7.279$	$2.032\times 10$	$1.153\times 10$
500	$4.845\times 10$	$1.136\times {10}^2$	$6.184\times 10$
1000	$1.057\times {10}^2$	$2.376\times {10}^2$	$1.294\times {10}^2$
5000	$6.428\times {10}^2$	$1.218\times {10}^3$	$7.753\times {10}^2$
10,000	$1.388\times {10}^3$	$2.544\times {10}^3$	$1.796\times {10}^3$
50,000	$8.055\times {10}^3$	$1.448\times {10}^4$	$1.088\times {10}^4$
100,000	$1.710\times {10}^4$	$2.943\times {10}^4$	$2.379\times {10}^4$
500,000	$9.829\times {10}^4$	$1.825\times {10}^5$	$2.740\times {10}^5$
1,000,000	$2.071\times {10}^5$	$4.827\times {10}^5$	$7.989\times {10}^5$
5,000,000	$1.162\times {10}^6$	$5.143\times {10}^6$	$6.635\times {10}^6$

,

Table 5. Average times for dataset extreme_idla (${10}^6$ clock ticks per second).

String Length	Time in Ticks	Time in Ticks	Time in Ticks
	IDLA	TRLA	BSLA
10	$7.900\times {10}^{-1}$	$1.440$	$7.000\times {10}^{-1}$
50	$1.830\times 10$	$8.200$	$3.600$
100	$7.190\times 10$	$1.590\times 10$	$6.800$
500	$1.778\times {10}^3$	$7.300\times 10$	$3.550\times 10$
1000	$7.105\times {10}^3$	$1.430\times {10}^2$	$6.900\times 10$
5000	$1.776\times {10}^5$	$7.100\times {10}^2$	$3.400\times {10}^2$
10,000	$7.111\times {10}^5$	$1.550\times {10}^3$	$6.800\times {10}^2$
50,000	$1.784\times {10}^7$	$8.050\times {10}^3$	$3.400\times {10}^3$
100,000	$7.130\times {10}^7$	$1.600\times {10}^4$	$6.800\times {10}^3$
500,000	$1.783\times {10}^9$	$8.200\times {10}^4$	$3.700\times {10}^4$
1,000,000	$7.137\times {10}^9$	$1.660\times {10}^5$	$7.800\times {10}^4$
5,000,000	$1.813\times {10}^{11}$	$8.800\times {10}^5$	$4.950\times {10}^5$

,

Table 6. Average times for dataset extreme_trla (${10}^6$ clock ticks per second).

String Length	Time in Ticks	Time in Ticks	Time in Ticks
	IDLA	TRLA	BSLA
10	$4.588\times {10}^{-1}$	$1.628$	$1.126$
50	$4.987$	$1.039\times 10$	$7.112$
100	$1.275\times 10$	$2.179\times 10$	$1.439\times 10$
500	$9.033\times 10$	$1.101\times {10}^2$	$6.914\times 10$
1000	$2.060\times {10}^2$	$2.222\times {10}^2$	$1.392\times {10}^2$
5000	$1.319\times {10}^3$	$1.171\times {10}^3$	$7.699\times {10}^2$
10,000	$2.896\times {10}^3$	$2.394\times {10}^3$	$1.652\times {10}^3$
50,000	$2.209\times {10}^4$	$1.263\times {10}^4$	$8.992\times {10}^3$
100,000	$3.965\times {10}^4$	$2.567\times {10}^4$	$1.862\times {10}^4$
500,000	$2.233\times {10}^5$	$1.349\times {10}^5$	$1.091\times {10}^5$
1,000,000	$4.734\times {10}^5$	$2.759\times {10}^5$	$3.104\times {10}^5$
5,000,000	$2.632\times {10}^6$	$1.458\times {10}^6$	$2.298\times {10}^6$

,

Table 7. Average times for dataset extreme_trla1 (${10}^6$ clock ticks per second).

String Length	Time in Ticks	Time in Ticks	Time in Ticks
	IDLA	TRLA	BSLA
10	$5.040\times {10}^{-1}$	$1.600$	$1.117$
50	$5.910$	$1.042\times 10$	$7.290$
100	$1.460\times 10$	$2.145\times 10$	$1.446\times 10$
500	$1.146\times {10}^2$	$1.126\times {10}^2$	$6.979\times 10$
1000	$2.662\times {10}^2$	$2.284\times {10}^2$	$1.379\times {10}^2$
5000	$1.694\times {10}^3$	$1.205\times {10}^3$	$7.853\times {10}^2$
10,000	$3.734\times {10}^3$	$2.477\times {10}^3$	$1.739\times {10}^3$
50,000	$2.276\times {10}^4$	$1.310\times {10}^4$	$9.683\times {10}^3$
100,000	$4.901\times {10}^4$	$2.796\times {10}^4$	$2.009\times {10}^4$
500,000	$2.928\times {10}^5$	$1.465\times {10}^5$	$1.238\times {10}^5$
1,000,000	$6.199\times {10}^5$	$3.000\times {10}^5$	$3.622\times {10}^5$
5,000,000	$3.432\times {10}^6$	$1.642\times {10}^6$	$2.778\times {10}^6$

and

Table 8. Average times for dataset extreme_trla2 (${10}^6$ clock ticks per second).

String Length	Time in Ticks	Time in Ticks	Time in Ticks
	IDLA	TRLA	BSLA
10	$5.041\times {10}^{-1}$	$1.683$	$1.121$
50	$6.160$	$1.020\times 10$	$7.257$
100	$1.526\times 10$	$2.090\times 10$	$1.441\times 10$
500	$1.367\times {10}^2$	$1.074\times {10}^2$	$7.117\times 10$
1000	$3.202\times {10}^2$	$2.135\times {10}^2$	$1.390\times {10}^2$
5000	$2.024\times {10}^3$	$1.145\times {10}^3$	$7.966\times {10}^2$
10,000	$4.500\times {10}^3$	$2.257\times {10}^3$	$1.762\times {10}^3$
50,000	$2.728\times {10}^4$	$1.172\times {10}^4$	$1.012\times {10}^4$
100,000	$5.941\times {10}^4$	$2.362\times {10}^4$	$2.115\times {10}^4$
500,000	$3.639\times {10}^5$	$1.262\times {10}^5$	$1.351\times {10}^5$
1,000,000	$7.719\times {10}^5$	$2.571\times {10}^5$	$3.915\times {10}^5$
5,000,000	$4.263\times {10}^6$	$1.323\times {10}^6$	$3.118\times {10}^6$

and Figure 7, Figure 8, Figure 9, Figure 10, Figure 11, Figure 12, Figure 13 and Figure 14. All the graphs include the curve $x=y$ for reference.