Static Analysis for ECMAScript String Manipulation Programs

Abstract

In recent years, dynamic languages, such as JavaScript or Python, have been increasingly used in a wide range of fields and applications. Their tricky and misunderstood behaviors pose a great challenge for static analysis of these languages. A key aspect of any dynamic language program is the multiple usage of strings, since they can be implicitly converted to another type value, transformed by string-to-code primitives or used to access an object-property. Unfortunately, string analyses for dynamic languages still lack precision and do not take into account some important string features. In this scenario, more precise string analyses become a necessity. The goal of this paper is to place a first step for precisely handling dynamic language string features. In particular, we propose a new abstract domain approximating strings as finite state automata and an abstract interpretation-based static analysis for the most common string manipulating operations provided by the ECMAScript specification. The proposed analysis comes with a prototype static analyzer implementation for an imperative string manipulating language, allowing us to show and evaluate the improved precision of the proposed analysis.

1. Introduction

Dynamic languages, for instance JavaScript or Python, have seen an important growth in a very wide range of fields and applications. Common features in these languages are dynamic typing (typing occurs during program execution, at run-time) and implicit type conversion [1], which lighten the development phase and allow programs not to block execution in the presence of unexpected or unpredictable situations. Moreover, one important aspect of dynamic languages is the way strings may be used. In JavaScript, for example, strings can be either used to access property objects or transformed into executable code by using the global function eval. In this way, dynamic languages provide multiple string features that simplify the writing of programs, allowing, at the same time, statically unpredictable executions which might make them harder to understand [1]. For this reason, string obfuscation (e.g., string splitting) is becoming one of the most common obfuscation techniques in JavaScript malwares [2], making it hard to statically analyze code. Consider, for example, the JavaScript program fragment in Figure 1 where strings are manipulated, de-obfuscated, combined together into the variable d and finally transformed into executable code, the statement ws = new ActiveXObject(WScript.Shell). This command, in Internet Explorer, opens a shell which may execute malicious commands. The command is not hard-coded in the fragment but it is built at run-time and the initial values of $\mathtt{i}$,$\mathtt{j}$ and $\mathtt{k}$ are unknown, as is the number of iterations of the loops.

All these observations suggest that, in order to statically understand statements which are dynamically generated and executed, it may be extremely useful to statically analyze the string value of d. Unfortunately existing static analyzers for dynamic languages [3,4,5,6], might fail to precisely analyze strings in dynamic contexts. for instance, in the example above, TAJS [3], JSAI [4] and SAFE [5], lose precision on the eval input value and any information gathered so far about it. Namely, the issue of analyzing dynamic languages, even if tackled by sophisticated tools as the cited ones, still lacks formal approaches for handling the dynamic features of string manipulation, such as dynamic typing, implicit type conversion and dynamic code generation. Instead, in [7], a new approach for dynamic language analysis is proposed based on finite state automata for abstracting strings, coming with both a precise string abstraction able to infer string properties in general and a sound abstract interpreter for dynamically-generated code.

Contributions

In this paper (This is an extended and revised version of [8] integrated with a more complete range of string operations, detailed proofs of the results presented (proofs are reported in Appendix A) and an improved implementation that will be discussed in Section 6.), we focus on the characterization of an abstract interpretation-based [9] formal framework, capable of handling dynamic typing and implicit type conversion, by defining an abstract semantics able to (precisely, when possible) capture the previously mentioned dynamic features. Even if we do not tackle the problem of analyzing dynamically generated code (meaning that we do not analyze its behavior), as highlighted in [7], such semantics is a necessary step towards a sufficiently precise analysis for it, since it is able to reason about a class of string manipulation programs (as far as string values are concerned) that state-of-art static analyzers would fail to precisely analyze. Indeed the domain we propose allows us to collect (and potentially approximate) the set of all possible string values that a variable may receive during computation (at each program point). It should be clear that, in order to analyze what an eval statement might execute, we surely need to (over-)approximate the set of precise string values of its input. Hence we propose an approach defining a collecting semantics for strings. With this task in mind, we will first discuss how to combine abstract domains of primitive types (strings, integers and booleans) in order to capture dynamic typing. Once we have such an abstract domain, we will define on it an abstract semantics for a $\mu \mathsf{JS}$ language, augmented with implicit type conversion, dynamic typing and several interesting string operations taken from the official ECMAScript language specification [10], namely the JavaScript language specification, whose concrete semantics is inspired by the JavaScript one. In particular, for each one of these operations we will provide the algorithm computing its abstract semantics and we will discuss their soundness and completeness.

Paper structure

In Section 2 we recall relevant notions on finite state automata and the core language adopted for this paper is established in Section 3. In Section 4.1 we define the finite state automata domain, highlighting some important operations and theoretical results. In Section 4 we discuss and present two ways of combining abstract domains (for primitive types) suitable for dynamic languages. Then, In Section 5, we present the new abstract semantics for string manipulating operations. In Section 6 we examine and evaluate the precision of the string static analyzer based on the above semantics. Finally, in Section 7, we discuss and compare this paper to the most related works and we draw our conclusions.

2. Background

In this section, we recall some basic notations and notions that will be used in the rest of the paper.

2.1. String Notation

We denote by $\Sigma$ a finite non-empty alphabet of symbols, its Kleene-closure by ${\Sigma }^{*}$ and a string element by $\sigma \in {\Sigma }^{*}$. If $\sigma ={\sigma }_0{\sigma }_1\cdots {\sigma }_n$, the length of $\sigma$ is $\left|\sigma \right|=n+1$ and the element in the i-th position is ${\sigma }_i$. Given two strings $\sigma ,{\sigma }^{\prime }\in {\Sigma }^{*}$, $\sigma ·{\sigma }^{\prime }$ is their concatenation. A language is a set of strings, i.e., $\mathtt{L}\in \wp \left({\Sigma }^{*}\right)$. We use the following notations: ${\Sigma }^i\stackrel{\mathrm{def}}{=}\{\sigma \in {\Sigma }^{*}\left|\right|\sigma |=i\}$ and ${\Sigma }^{<i}\stackrel{\mathrm{def}}{=}{\bigcup }_{j<i}{\Sigma }^j$. Given $\sigma \in {\Sigma }^{*}$, $i,j\in \mathbb{N}$ ($i\le j\le \left|\sigma \right|$) the substring between i and j of $\sigma$ is the string ${\sigma }_i\cdots {\sigma }_{j-1}$. We denote by ${\Sigma }_{\mathbb{Z}}\stackrel{\mathrm{def}}{=}\{+,-,ϵ\}·{\{0,1,\cdots ,9\}}^{+}$ the set of numeric strings, i.e., strings corresponding to integers. $\mathcal{I}:{\Sigma }_{\mathbb{Z}}\to \mathbb{Z}$ maps numeric strings to the corresponding integers. Dually, we define the function $\mathcal{S}:\mathbb{Z}\to {\Sigma }_{\mathbb{Z}}$ that maps each integer to its numeric string representation (e.g., 1 is mapped to the string "1", and not "+1"). Given $\sigma \in {\Sigma }^{*}$ and $n\in \mathbb{N}$, we denote with ${\sigma }^n$ the n-times concatenation of $\sigma$. Given a symbol $c\in \Sigma$ we denote with $\mathsf{toLowerCase}\left(c\right)$ its corresponding lower-case symbol, if it is a capital letter, otherwise c is returned. We abuse notation denoting by $\mathsf{toLowerCase}\left(\sigma \right)$ the string $\sigma$ where at each position any upper-case symbol is replaced with the corresponding lower-case symbol.

2.2. Regular Languages and Finite State Automata

We follow [11] for automata notation. A finite state automaton (FA) is a tuple $\mathtt{A}=(Q,\Sigma ,\delta ,q_0,F)$ where Q is a finite non-empty set of states, $q_0\in Q$ is the initial state, $\Sigma$ is a finite alphabet, $\delta \subseteq Q\times \Sigma \times Q$ is the transition relation and $F\subseteq Q$ is the set of final states. In particular, if $\delta :Q\times \Sigma \to Q$ is a function then $\mathtt{A}$ is called deterministic FA (DFA). We consider DFA also those FA which are not complete, namely such that a transition for each pair $(q,a)$ ($q\in Q$, $a\in \Sigma$) does not exists. They can be easily transformed in a DFA by adding a sink state receiving all the missing transitions. The class of languages recognized by FA is the class of regular languages. We denote the set of all DFA as Dfa. Given an automaton $\mathtt{A}$, we denote the language accepted by $\mathtt{A}$ as $\mathcal{L}\left(\mathtt{A}\right)$. A language $\mathtt{L}$ is regular iff there exists a FA $\mathtt{A}$ such that $\mathtt{L}=\mathcal{L}\left(\mathtt{A}\right)$. From the Myhill-Nerode theorem [12], for each regular language uniquely exists a minimum automaton, i.e., with the minimum number of states, recognizing the language. Given a regular language $\mathtt{L}$, we denote by $\mathsf{Min}\left(\mathtt{L}\right)$ the minimum DFA $\mathtt{A}$ s.t. $\mathtt{L}=\mathcal{L}\left(\mathtt{A}\right)$. Given an automaton $\mathtt{A}$, we denote by $\mathsf{Kleene}\left(\mathtt{A}\right)$ the automaton that recognizes the language corresponding to the Kleene-closure of $\mathcal{L}\left(\mathtt{A}\right)$, namely the automaton ${\mathtt{A}}^{\prime }$ s.t. $\mathcal{L}\left({\mathtt{A}}^{\prime }\right)=\mathcal{L}\left(\mathsf{Kleene}\left(\mathtt{A}\right)\right))=\left\{{\sigma }^n\right|\sigma \in \mathcal{L}\left(\mathtt{A}\right),n\in \mathbb{N}\}$. Moreover, given an automaton $\mathtt{A}$, we rely on the predicate $\mathsf{hasCycle}\left(\mathtt{A}\right)$ that checks whether $\mathtt{A}$ is cyclic.

2.3. Abstract Interpretation

Abstract interpretation establishes a correspondence between a concrete semantics and an approximated one called abstract semantics [9,13]. In a Galois Connection framework, if C and A are complete lattices, a pair of monotone functions $\alpha :C\to A$ and $\gamma :A\to C$ forms a Galois Connection (GC for short) between C and A if for every $x\in C$ and $y\in A$ we have $\alpha \left(x\right){\le }_{A}y\iff x{\le }_C\gamma \left(y\right)$. $\alpha$ and $\gamma$ are called abstraction function and concretization function, respectively.

Let L be a complete lattice. $X\subseteq L$ is a Moore family of L if $X=\mathcal{M}\left(X\right)\stackrel{▵}{=}\{\bigwedge S|S\subseteq X\}$ and $\top \left(\mathrm{top}\mathrm{element}\right)\in \mathcal{M}\left(X\right)$. If any concrete object in C has a best abstraction in the abstract domain A implies that A is a Moore family of C and so there exists a Galois connection between C and A.

Weaker forms of correspondence are possible, e.g., when A is not a complete lattice or when only $\gamma$ exists [14]. In all cases, relative precision in A is given by comparing the meaning of abstract objects in C, i.e., $x_1{\le }_A{x}_2$ if $\gamma \left(x_1\right){\le }_C\gamma \left(x_2\right)$. If $f:C\to C$ is a continuous function and A is an abstraction of C by means of the GC $\langle \alpha ,\gamma \rangle$, then f always has a best correct approximation in A, $f^A:A\to A$, defined as $f^A\stackrel{▵}{=}\alpha \circ f\circ \gamma$. Any approximation $f^{♯}:A\to A$ of f in A is sound if $f^A⊑f^{♯}$.

In abstract interpretation, there exist two notions of completeness: backward completeness and forward completeness. The former is the best known form of completeness and focuses on complete abstractions of the inputs, while the latter is forward completeness [15,16,17] and it focuses on complete abstractions of the outputs, both w.r.t. an operation of interest. When we do not have a GC, namely when only the concretization $\gamma$ exists, we need to focus only on forward completeness, as we will do in this paper. Given a GC $\langle \alpha ,\gamma \rangle$, a concrete function $f:C\to C$ and an abstract function $f^{♯}:A\to A$, $f^{♯}$ is forward complete w.r.t. f if $\forall a\in A.f\left(\gamma \left(a\right)\right)=\gamma \left(f^{♯}\left(a\right)\right)$.

A satisfies the ascending chain condition (ACC) if all ascending chains are finite. When A is not ACC convergence to the limit of the fix-point iterations can be ensured through widening operators. A widening operator $\nabla :A\times A\to A$ approximates the least upper bounds, i.e., $\forall x,y\in A.x,y{\le }_A\left(x\nabla y\right)$ and it is such that for any increasing chain $x_1\le x_2\le \cdots \le x_n\le \cdots$ the increasing chain $w_0=\perp$ and $w_{i+1}=w_i\nabla xi$ is finite.

3. The Core Language

In this paper, we consider a JavaScript core language, reported in Figure 2, that we call $\mu \mathsf{JS}$, containing several representative string operations taken from the set of methods offered by the JavaScript built-in class String, detailed in the ECMAScript language specification [10]. Even though we have decided to focus on a core of the operations, note that the missing methods (e.g., indexOf or endsWith) can be easily modeled as composition of our chosen string methods or as particular cases of them. Nevertheless, as we will discuss in Section 6, these operations have been implemented and tested.

$\mu \mathsf{JS}$ Semantics

In $\mu \mathsf{JS}$ the primitive values are $\mathbb{V}=\mathbb{S}\cup \mathbb{Z}\cup \mathbb{B}\cup \left\{\mathtt{NaN}\right\}$ with $\mathbb{S}\stackrel{\mathrm{def}}{=}{\Sigma }^{*}$ (strings on the alphabet $\Sigma$), $\mathbb{B}\stackrel{\mathrm{def}}{=}\{\mathtt{true},\mathtt{false}\}$ and $\mathtt{NaN}$ a special value denoting not-a-number.

Program states are partial maps from identifiers to primitive values, i.e., $\mathrm{S}\mathrm{TATES}:\mathrm{I}\mathrm{D}\to \mathbb{V}$. The concrete big-step semantics $〚·〛:\mathrm{S}\mathrm{TMT}\times \mathrm{S}\mathrm{TATES}\to \mathrm{S}\mathrm{TATES}$ is standard and follows [18], and it includes dynamic typing and implicit type conversion. In addition, the expression semantics, $\left(\right|·\left|\right):\mathrm{E}\mathrm{XP}\times \mathrm{S}\mathrm{TATES}\to \mathbb{V}$, is standard and follows [18]; we only provide the formal and precise semantics of the $\mu \mathsf{JS}$ string operations. Let $\sigma ,{\sigma }^{\prime }\in \mathbb{S}$ and $i,j\in \mathbb{Z}$ (values which are not strings or numbers respectively, are converted by the implicit type conversion primitives, moreover, negative values are treated as zero).

substring:

It extracts the substring between two indexes from a string. The semantics is defined by the function Ss$:\mathbb{S}\times \mathbb{Z}\times \mathbb{Z}\to \mathbb{S}$ as:

$$\mathrm{S}\mathrm{S}(\sigma ,i,j)\stackrel{\mathrm{def}}{=}\left\{\begin{array}{cc}\mathrm{S}\mathrm{S}(\sigma ,j,i)\hfill & j<i\hfill \\ {\sigma }_i\cdots {\sigma }_j\hfill & j<\left|\sigma \right|\wedge i\le j\hfill \\ {\sigma }_i\cdots {\sigma }_n\hfill & j\ge n=\left|\sigma \right|\wedge i\le j\hfill \end{array}\right.$$

charAt:

It returns the character, i.e., the string of unitary length, at a specified index in a string $\sigma$. The semantics is the function Ca$:\mathbb{S}\times \mathbb{Z}\to \mathbb{S}$ defined as follows:

$$\mathrm{C}\mathrm{A}(\sigma ,i)\stackrel{\mathrm{def}}{=}\left\{\begin{array}{cc}{\sigma }_i\hfill & 0\le i<\left|\sigma \right|\hfill \\ ϵ\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

length:

It returns the length of a string $\sigma \in \mathbb{S}$. Its semantics is the function Le$:\mathbb{S}\to \mathbb{Z}$ defined as $\mathrm{L}\mathrm{E}\left(\sigma \right)\stackrel{\mathrm{def}}{=}\left|\sigma \right|$.

concat:

It returns the concatenation between two strings and its concrete semantics Cc : $\mathbb{S}\times \mathbb{S}\to \mathbb{S}$ relies on the concatenation operator reported in Section 2.

$$\mathrm{C}\mathrm{C}(\sigma ,{\sigma }^{\prime })=\sigma ·{\sigma }^{\prime }$$

startsWith:

It determines whether a specified string $\sigma$ starts with ${\sigma }^{\prime }$. The semantics is the function Sw : $\mathbb{S}\times \mathbb{S}\to \mathbb{B}$ defined as:

$$\mathrm{S}\mathrm{W}(\sigma ,{\sigma }^{\prime })\stackrel{\mathrm{def}}{=}\left\{\begin{array}{cc}\mathtt{true}\hfill & \exists {\sigma }^{\prime \prime }\in {\Sigma }^{*}.\sigma ={\sigma }^{\prime }·{\sigma }^{\prime \prime }\hfill \\ \mathtt{false}\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

repeat:

It returns the given string repeated n times. The semantics is the function Rt$:\mathbb{S}\times \mathbb{Z}\to \mathbb{S}$ defined as $\mathrm{R}\mathrm{T}(\sigma ,n)\stackrel{\mathrm{def}}{=}{\sigma }^n$.

includes:

It determines whether a string ${\sigma }^{\prime }$ is a substring of $\sigma$. The semantics is the function In$:\mathbb{S}\times \mathbb{S}\to \mathbb{B}$ defined as:

$$\mathrm{I}\mathrm{N}(\sigma ,{\sigma }^{\prime })\stackrel{\mathrm{def}}{=}\left\{\begin{array}{cc}\mathtt{true}\hfill & \exists \varphi ,\psi \in {\Sigma }^{*}.\sigma =\varphi ·{\sigma }^{\prime }·\psi \hfill \\ \mathtt{false}\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

toLowerCase:

It returns the given string in all lowercase letters. The semantics is the function Lc $:\mathbb{S}\to \mathbb{S}$ defined as $\mathrm{L}\mathrm{C}\left(\sigma \right)\stackrel{\mathrm{def}}{=}\mathsf{toLowerCase}\left(\sigma \right)$.

trimLeft:

It removes all the white-spaces at the beginning of a string. The semantics is the function Tl $:\mathbb{S}\to \mathbb{S}$ defined as:

$$\mathrm{T}\mathrm{L}\left(\sigma \right)\stackrel{\mathrm{def}}{=}{\sigma }^{\prime }\mathrm{where}\psi =\mathsf{max}\{{\psi }^{\prime }\in {\left(␣\right)}^{*}|\sigma ={\psi }^{\prime }·{\sigma }^{\prime }\}\wedge \sigma =\psi ·{\sigma }^{\prime }$$

trimRight:

It removes all the white-spaces at the end of a string. The semantics is the function Tr $:\mathbb{S}\to \mathbb{S}$ defined as:

$$\mathrm{T}\mathrm{R}\left(\sigma \right)\stackrel{\mathrm{def}}{=}{\sigma }^{\prime }\mathrm{where}\psi =\mathsf{max}\{{\psi }^{\prime }\in {\left(␣\right)}^{*}|\sigma ={\sigma }^{\prime }·{\psi }^{\prime }\}\wedge \sigma ={\sigma }^{\prime }·\psi$$

trim:

It removes all the white-spaces at the end and beginning of a string. The semantics is the function Tm $:\mathbb{S}\to \mathbb{S}$ defined as: $\mathrm{T}\mathrm{M}\left(\sigma \right)\stackrel{\mathrm{def}}{=}\mathrm{T}\mathrm{R}\left(\mathrm{T}\mathrm{L}\left(\sigma \right)\right)$.

Implicit Type Conversion

In order to properly capture the semantics of the language $\mu \mathsf{JS}$, inspired by the JavaScript semantics, we need to deal with implicit type conversion [18]. For each primitive value, we define an auxiliary function converting it to other primitive values (Figure 3). Note that all the functions behave like identity when applied to values not needing conversion, e.g., $\mathtt{toInt}$ on integers. Then, $\mathtt{toString}:\mathbb{V}\to \mathbb{S}$ maps any input value to its string representation; $\mathtt{toInt}:\mathbb{V}\to \mathbb{Z}\cup \left\{\mathtt{NaN}\right\}$ returns the integer corresponding to a value, when it is possible: for $\mathtt{true}$ and $\mathtt{false}$ it returns respectively 1 and 0, for strings in ${\Sigma }_{\mathbb{Z}}$ it returns the corresponding integer, while all the other values are converted to $\mathtt{NaN}$. For instance, $\mathtt{toInt}(‘‘42")=42$, $\mathtt{toInt}(‘‘42hello")=\mathtt{NaN}$. Finally, $\mathtt{toBool}:\mathbb{V}\to \mathbb{B}$ returns $\mathtt{false}$ when the input is 0, and $\mathtt{true}$ for all the other non boolean primitive values. It is worth noting that the auxiliary functions defined in Figure 3 do not correspond to explicit casting but they model the implicit type conversion implemented by JavaScript. In particular, these functions cannot be directly called by a programmer since they are exclusively used internally (indeed implicitly) by the semantics when a type value of an expression operand is required.

4. An Abstract Domain for String Manipulation

4.1. The Finite State Automata Abstract Domain for Strings

In this section, we describe the finite state automata abstract domain for strings [19,20,21], namely the domain of regular languages over $\wp \left({\Sigma }^{*}\right)$. In particular our goal is to exploit automata, and therefore regular languages, for approximating string values collected during analysis. The idea is to approximate strings as regular languages represented by the minimum DFA [12] recognizing them. In general, we have more DFA than regular languages, hence the domain of automata is indeed the quotient $\mathrm{D}{\mathrm{FA}}_{/\equiv }$ w.r.t. the equivalence relation induced by language equality: $\forall {\mathtt{A}}_1,{\mathtt{A}}_2\in \mathrm{D}{\mathrm{FA}}_{/\equiv }.{\mathtt{A}}_1\equiv {\mathtt{A}}_2\iff \mathcal{L}\left({\mathtt{A}}_1\right)=\mathcal{L}\left({\mathtt{A}}_2\right)$. Therefore any equivalence class is composed by automata that recognize the same regular language. We abuse notation by representing these classes in the domain $\mathrm{D}{\mathrm{FA}}_{/\equiv }$ w.r.t. ≡ using one of its automata (usually the minimum), i.e., when we write $\mathtt{A}\in \mathrm{D}{\mathrm{FA}}_{/\equiv }$ we mean ${\left[\mathtt{A}\right]}_{\equiv }$.

The partial order ${⊑}_{\mathrm{D}\mathrm{FA}}$ is induced by language inclusion, i.e., $\forall {\mathtt{A}}_1,{\mathtt{A}}_2\in \mathrm{D}{\mathrm{FA}}_{/\equiv }.{\mathtt{A}}_1{⊑}_{\mathrm{D}\mathrm{FA}}{\mathtt{A}}_2\iff \mathcal{L}\left({\mathtt{A}}_1\right)\subseteq \mathcal{L}\left({\mathtt{A}}_2\right)$, which is well defined since automata in the same ≡-equivalence class recognize the same language.

The corresponding least upper bound, ${\bigsqcup }_{\mathrm{D}\mathrm{FA}}:\mathrm{D}{\mathrm{FA}}_{/\equiv }\times \mathrm{D}{\mathrm{FA}}_{/\equiv }\to \mathrm{D}{\mathrm{FA}}_{/\equiv }$ on the domain $\mathrm{D}{\mathrm{FA}}_{/\equiv }$, is the standard union between automata: $\forall {\mathtt{A}}_1,{\mathtt{A}}_2\in \mathrm{D}{\mathrm{FA}}_{/\equiv }.{\mathtt{A}}_1{\bigsqcup }_{\mathrm{D}\mathrm{FA}}{\mathtt{A}}_2\stackrel{\mathrm{def}}{=}\mathsf{Min}(\mathcal{L}\left({\mathtt{A}}_1\right)\cup \mathcal{L}\left({\mathtt{A}}_2\right))$. It is the minimum automaton recognizing the union of the languages $\mathcal{L}\left({\mathtt{A}}_1\right)$ and $\mathcal{L}\left({\mathtt{A}}_2\right)$. This is a well-defined notion since regular languages are closed under union. As example consider Figure 4, where the automaton in Figure 4c is the least upper bound of ${\mathtt{A}}_1$ and ${\mathtt{A}}_2$ given in Figure 4a,b, respectively.

The (finite) greatest lower bound ${\sqcap }_{\mathrm{D}\mathrm{FA}}:\mathrm{D}{\mathrm{FA}}_{/\equiv }\times \mathrm{D}{\mathrm{FA}}_{/\equiv }\to \mathrm{D}{\mathrm{FA}}_{/\equiv }$ corresponds to automata intersection (since regular languages are closed under finite intersection): $\forall {\mathtt{A}}_1,{\mathtt{A}}_2\in \mathrm{D}{\mathrm{FA}}_{/\equiv }.{\mathtt{A}}_1{\sqcap }_{\mathrm{D}\mathrm{FA}}{\mathtt{A}}_2\stackrel{\mathrm{def}}{=}\mathsf{Min}(\mathcal{L}\left({\mathtt{A}}_1\right)\cap \mathcal{L}\left({\mathtt{A}}_2\right)).$

Theorem 1.

$\langle \mathrm{D}{\mathrm{FA}}_{/\equiv },{⊑}_{\mathrm{D}\mathrm{FA}},{\bigsqcup }_{\mathrm{D}\mathrm{FA}},{\sqcap }_{\mathrm{D}\mathrm{FA}},\mathsf{Min}\left(\mathsf{⌀}\right),\mathsf{Min}\left({\Sigma }^{*}\right)\rangle$ is a sub-lattice but not a complete meet-sub-semilattice of $\wp \left({\Sigma }^{*}\right)$.

In other words, it cannot exists a Galois connection between $\mathrm{D}{\mathrm{FA}}_{/\equiv }$ and $\wp \left({\Sigma }^{*}\right)$, i.e., there may be no minimal automaton abstracting a language. Note that some works [22,23,24] have studied automatic procedures to compute, given an input language L, the regular cover of L [23] (i.e., an automaton containing the language L). Some of them [22,23] studied regular covers guaranteeing that the automaton obtained is the best w.r.t. a minimal relation (but not minimum). However this is not a concern since the relation between concrete semantics and abstract semantics can be weakened still ensuring soundness [14]. A well known example is the convex polyhedra domain [25].

Widening

The domain $\mathrm{D}{\mathrm{FA}}_{/\equiv }$ is an infinite domain, and it is not ACC, i.e., it contains infinite ascending chains. for instance, consider the set of languages $L_i=\left\{a^j{b}^j\right|0\le j\le i\}\subseteq \wp \left({\Sigma }^{*}\right)$, indexed by a constant natural $i\in \mathbb{N}$, forming an infinite ascending chain of finite regular languages. The set of the corresponding minimal automata trivially forms an ascending chain on $\mathrm{D}{\mathrm{FA}}_{/\equiv }$. This clearly implies that any computation on $\mathrm{D}{\mathrm{FA}}_{/\equiv }$ may lose convergence [14] (Most of the proposed abstract domains for strings [3,4,5,26] trivially satisfy ACC being finite, but they may lose precision during the abstract computation [27].).

As far as automata are concerned, existing widenings are defined in terms of a state equivalence relation merging states that recognize the same language, up to a fixed length n (set as parameter for tuning the widening precision) [28,29]. We denote this parametric widening with ${\nabla }_n:\mathrm{D}{\mathrm{FA}}_{/\equiv }\times \mathrm{D}{\mathrm{FA}}_{/\equiv }\to \mathrm{D}{\mathrm{FA}}_{/\equiv }$, with $n\in \mathbb{N}$ [28] and it is defined in the following.

Let $\mathtt{A}=(Q,\Sigma ,\delta ,q_0,F)$ and ${\mathtt{A}}^{\prime }=(Q^{\prime },\Sigma ,{\delta }^{\prime },q_0^{\prime },F^{\prime })$ be two finite state automata such that $\mathcal{L}\left(\mathtt{A}\right)\subseteq \mathcal{L}\left({\mathtt{A}}^{\prime }\right)$: the widening between $\mathtt{A}$ and ${\mathtt{A}}^{\prime }$ is formalized in terms of a relation $R\subseteq Q\times Q^{\prime }$ between the sets of states of the two automata. The relation R is used to define an equivalence relation ${\equiv }_R\subseteq Q^{\prime }\times Q^{\prime }$ over the states of ${\mathtt{A}}^{\prime }$, such that ${\equiv }_R=R\circ R^{-1}$. The widening between $\mathtt{A}$ and ${\mathtt{A}}^{\prime }$ is then given by the quotient automaton of ${\mathtt{A}}^{\prime }$ w.r.t. the partition induced by ${\equiv }_R$: ${\mathtt{A}}^{\prime }{\nabla }_R{\mathtt{A}}^{\prime }={\mathtt{A}}_{{\equiv }_R}^{\prime }$ (Given $\mathtt{A}\in \mathrm{D}{\mathrm{FA}}_{/\equiv }$ and a partition $\pi$ over its states, we denote as ${\mathtt{A}}_{\pi }=(Q^{\prime },{\delta }^{\prime },q_0^{\prime },F^{\prime },\Sigma )$ the quotient automaton [12].). Thus, the widening operator merges the states of ${\mathtt{A}}^{\prime }$ that are equivalent by the relation ${\equiv }_R$. By changing the relation R, we obtain different widening operators [28]. It has been proved that convergence is guaranteed when the relation $R_n\subseteq Q\times Q^{\prime }$ is such that $(q,q^{\prime })\in R_n$ iff q and $q^{\prime }$ recognize the same language of strings of length at most n [28]. The parameter n therefore tunes the length of strings determining the equivalence of states used for merging them in the widening. It is worth noting that the smaller is n, the more information will be lost by widening.

In the following, given $\mathtt{A},{\mathtt{A}}^{\prime }\in \mathrm{D}{\mathrm{FA}}_{/\equiv }$ (without any constraints on the languages they recognize), we define the widening operator on $\mathrm{D}{\mathrm{FA}}_{/\equiv }$ parametric on $n\in \mathbb{N}$ as follows.

$$\mathtt{A}{\nabla }_n{\mathtt{A}}^{\prime }\stackrel{\mathrm{def}}{=}\mathtt{A}{\nabla }_{R_n}\left(\mathtt{A}{\bigsqcup }_{\mathrm{D}\mathrm{FA}}{\mathtt{A}}^{\prime }\right)$$

In order to show how the defined widening operator works, let us discuss the following example.

Example 1.

Consider the following $\mu \mathsf{JS}$ fragment

str = “ ”; while (x < 100) { str = str + “a”; x = x + 1; }

The value of the variablexis unknown and so is the number of iterations of thewhile-loop. In these cases, in order to guarantee soundness and termination, we apply the widening operator.

In Figure 5a we report the abstract value of the variablestrat the beginning of the second iteration of the loop, while in Figure 5b the abstract value of the variablestrat the end of the second iteration. Before starting a new iteration, in the example, we apply ${\nabla }_1$ between the two automata, specifically we merge all the states having the same outgoing character. The minimization of the so obtained automaton is reported in Figure 5c. The next iteration will reach the fix-point, guaranteeing termination.

4.2. An Abstract Domain for $\mu \mathsf{JS}$

By definition, string operations in our language also involve other primitive values, such as booleans or integers, hence we need an abstract domain able to observe any possible concrete value. This is additionally necessary for dealing with implicit type conversion as we will later observe.

We therefore have to design an abstract domain for string manipulation dealing with other primitive types, namely being able to combine different abstractions of various types. In particular, an abstract domain for string analysis equipped with dynamic typing must include all the possible primitive values, i.e., the whole $\mathbb{V}=\mathbb{Z}\cup \mathbb{B}\cup \mathbb{S}\cup \left\{\mathtt{NaN}\right\}$. The idea is to consider an abstract domain for each type of primitive value and to combine them in a unique abstract domain for $\mathbb{V}$. Consider, for each value $\mathbb{D}$, an abstract domain ${\mathbb{D}}^{♯}$ (we denote ${\mathbb{D}}_{\cancel{\perp }}^{♯}$ the domain ${\mathbb{D}}^{♯}$ without bottom), equipped with an abstraction ${\alpha }_{\mathbb{D}}:\mathbb{D}\to {\mathbb{D}}^{♯}$ and a concretization ${\gamma }_{\mathbb{D}}:{\mathbb{D}}^{♯}\to \mathbb{D}$ forming a Galois insertion [9].

4.2.1. Coalesced Sum

One way to merge domains is the coalesced sum [30]. The resulting domain contains all the non-bottom elements of the input domains, with a new top and a new bottom.

Definition 1

(Coalesced sum domain [31]). Let $\langle A,{\le }_A,{\bigsqcup }_A,{\sqcap }_A,{\perp }_A,{\top }_A\rangle$ and $\langle B,{\le }_B,{\bigsqcup }_B,{\sqcap }_B,{\perp }_B,{\top }_B\rangle$ be two lattices abstracting the posets $\langle C,{\le }_C\rangle$ and $\langle D,{\le }_D\rangle$ with abstraction functions ${\alpha }_A:A\to C$ and ${\alpha }_B:B\to D$, respectively. The coalesced sum domain $A\oplus B$ is defined as:

$$A\oplus B\stackrel{\mathrm{def}}{=}\left\{{\perp }_{A\oplus B}\right\}\cup \left\{a\right|a\in A_{\cancel{\perp }}\}\cup \left\{b\right|b\in B_{\cancel{\perp }}\}\cup \left\{{\top }_{A\oplus B}\right\}$$

such that the partial order is defined as $x{\le }_{A\oplus B}y\iff x{\le }_{A}y(x,y\in A)\vee x{\le }_{B}y(x,y\in B)$ and $\forall x\in A\oplus B.{\perp }_{A\oplus B}{\le }_{A\oplus B}x{\le }_{A\oplus B}{\top }_{A\oplus B}$, its least upper bound is defined as:

$$x{\bigsqcup }_{A\oplus B}y\stackrel{\mathit{def}}{=}\left\{\begin{array}{cc}x{\bigsqcup }_{A}y\hfill & ifx,y\in A_{\cancel{\perp }}\hfill \\ x{\bigsqcup }_{B}y\hfill & ifx,y\in B_{\cancel{\perp }}\hfill \\ x\hfill & ify={\perp }_{A\oplus B}\hfill \\ y\hfill & ifx={\perp }_{A\oplus B}\hfill \\ {\top }_{A\oplus B}\hfill & \mathit{otherwise}\hfill \end{array}\right.$$

and its greatest lower bound ${\sqcap }_{A\oplus B}$ can be dually defined. The abstraction functions ${\alpha }_{A\oplus B}:C\cup D\to A\oplus B$ is defined as:

$${\alpha }_{A\oplus B}\left(x\right)\stackrel{\mathit{def}}{=}\left\{\begin{array}{cc}{\alpha }_A\left(x\right)\hfill & ifx\in C\hfill \\ {\alpha }_B\left(x\right)\hfill & ifx\in D\hfill \\ {\top }_{A\oplus B}\hfill & \mathit{otherwise}\hfill \end{array}\right.$$

In our case, if we consider the abstract domains ${\mathbb{Z}}^{♯}$, ${\mathbb{S}}^{♯}$ and ${\mathbb{B}}^{♯}$, the coalesced sum is the abstraction of $\wp \left(\mathbb{V}\right)$ depicted in Figure 6.

This is the simplest choice but unfortunately this is not suitable for dynamic languages, in particular for dealing with dynamic typing and implicit type conversion. The problem is that the type of variables is inferred at run-time and may change during execution. For example, consider the $\mu \mathsf{JS}$ fragment $\mathtt{if}(\mathtt{y}<\mathtt{5})\{\mathtt{x}=‘‘\mathtt{42}";\}\mathtt{else}\{\mathtt{x}=\mathtt{true};\}$. The value of the variable $\mathtt{y}$ is statically unknown hence, in order to guarantee soundness, we must take into account both the branches, meaning that x may be both a string and a boolean value, after the if statement. On the coalesced sum domain, the analysis would lose any precision w.r.t. collecting semantics by returning ${\alpha }_{\mathbb{S}}(‘‘\mathtt{42}")\bigsqcup {\alpha }_{\mathbb{B}}\left(\mathtt{true}\right)=\top$.

4.2.2. Cartesian Product

In order to catch union types, without losing too much precision, we need to complete [15,16,32] the above domain in order to observe collections of values of different types. In order to define this combination, we rely on the Cartesian product, following [33]. The complete abstract domain w.r.t. dynamic typing and implicit type conversion is: ${\mathbb{Z}}^{♯}\times {\mathbb{B}}^{♯}\times {\mathbb{S}}^{♯}\times \wp \left(\left\{\mathtt{NaN}\right\}\right)$, abstraction of $\wp \left(\mathbb{V}\right)$. In this combined abstract domain, the value of x after the if-execution is precisely $(\perp ,{\alpha }_{\mathbb{B}}\left(\mathtt{true}\right),{\alpha }_{\mathbb{S}}(‘‘\mathtt{42}"),\perp )$, now an element of the domain, inferring that the value of x can be ${\alpha }_{\mathbb{B}}\left(\mathtt{true}\right)$ or ${\alpha }_{\mathbb{S}}(‘‘\mathtt{42}")$ but surely not an abstract integer of $\mathtt{NaN}$.

In the following, we consider the abstract domain ${\mathbb{V}}^{♯}$ for string analysis obtained as Cartesian product of the following abstractions: ${\mathbb{B}}^{♯}=\wp \left(\{\mathtt{true},\mathtt{false}\}\right)$, ${\mathbb{Z}}^{♯}=\mathsf{Const}\stackrel{\mathrm{def}}{=}\{{\perp }_{\mathsf{Const}},{\top }_{\mathsf{Const}}\}\cup \left\{\left\{z\right\}\right|z\in \mathbb{Z}\}$ (the abstract domain of constant integers) and ${\mathbb{S}}^{♯}=\mathrm{D}{\mathrm{FA}}_{/\equiv },$.

5. Abstract Semantics of ECMAScript String Operations

In this section, we define the abstract semantics of the language $\mu \mathsf{JS}$ over the abstract domain ${\mathbb{V}}^{♯}$. In particular, we have to define the expressions abstract semantics $〚·{〛}^{♯}:\mathrm{E}\mathrm{XP}\times \mathrm{S}\mathrm{TATES}\to {\mathbb{V}}^{♯}$, abstracting the collecting semantics (The string collecting semantics (fully reported in Appendix A) is defined lifting to $\wp \left(\mathbb{V}\right)$ the concrete one reported in Section 3. For example, the collecting semantics of substring is, abusing notation, $\mathrm{S}\mathrm{S}:\wp \left({\Sigma }^{*}\right)\times \wp \left(\mathbb{Z}\right)\times \wp \left(\mathbb{Z}\right)\to \wp \left({\Sigma }^{*}\right)$ defined as $\mathrm{S}\mathrm{S}(\mathtt{L},I,J)=\left\{\mathrm{S}\mathrm{S}\right(\sigma ,i,j\left)\right|\sigma ,\in \mathtt{L},i\in I,j\in J\}$.), which is standard except for the string operations that will be explicitly provided by describing the algorithms for computing them. Let us first recall some important notions on regular languages, useful for the algorithms we will provide.

Definition 2

(Suffixes and prefixes [12]). Let $\mathtt{L}\in \wp \left({\Sigma }^{*}\right)$ be a regular language. The suffixes of $\mathtt{L}$ are $\mathrm{S}\mathrm{U}\left(\mathtt{L}\right)\stackrel{\mathit{def}}{=}\{y\in {\Sigma }^{*}|\exists x\in {\Sigma }^{*}.x·y\in \mathtt{L}\}$, and the prefixes of $\mathtt{L}$ are $\mathrm{P}\mathrm{R}\left(\mathtt{L}\right)\stackrel{\mathit{def}}{=}\{x\in {\Sigma }^{*}|\exists y\in {\Sigma }^{*}.xy\in \mathtt{L}\}$.

We can define the suffixes from a position, namely given $i\in \mathbb{N}$, the set of suffixes from i is $\mathrm{S}\mathrm{U}(\mathtt{L},i)\stackrel{\mathrm{def}}{=}\{y\in {\Sigma }^{*}|\exists x\in {\Sigma }^{*}.x·y\in \mathtt{L},\left|x\right|=i\}$. for instance, let $\mathtt{L}=\{abc,hello\}$, then $\mathrm{S}\mathrm{U}(\mathtt{L},2)=\{c,llo\}$.

Definition 3

(Right quotient [12]). Let ${\mathtt{L}}_1,{\mathtt{L}}_2\in {\Sigma }^{*}$ be regular languages. The right quotient of ${\mathtt{L}}_1$ w.r.t ${\mathtt{L}}_2$ is $\mathrm{R}\mathrm{Q}({\mathtt{L}}_1,{\mathtt{L}}_2)\stackrel{\mathit{def}}{=}\{x\in {\Sigma }^{*}|\exists y\in {\mathtt{L}}_2.x·y\in {\mathtt{L}}_1\}$.

For example, let ${\mathtt{L}}_3=\{xab,yab\}$ and ${\mathtt{L}}_4=\{b,ab\}$. The right quotient of ${\mathtt{L}}_3$ w.r.t ${\mathtt{L}}_4$ is $\mathrm{R}\mathrm{Q}({\mathtt{L}}_3,{\mathtt{L}}_4)=\{xa,ya,x,y\}$.

Definition 4

(Substrings/Factors [34]). Let $\mathtt{L}\in \wp \left({\Sigma }^{*}\right)$ be a regular language. The set of its substrings/factors is $\mathrm{F}\mathrm{A}\left(\mathtt{L}\right)\stackrel{\mathit{def}}{=}\{y\in {\Sigma }^{*}|\exists x,z\in {\Sigma }^{*}.x·y·z\in \mathtt{L}\}$.

These operations are all defined as transformations of regular languages. In [12], the corresponding algorithms on FA are provided. In particular, let $\mathtt{A},{\mathtt{A}}_1\in \mathrm{D}{\mathrm{FA}}_{/\equiv }$ and $i\in \mathbb{N}$, then $\mathsf{SU}\left(\mathtt{A}\right)$, $\mathsf{PR}\left(\mathtt{A}\right)$, $\mathsf{SU}(\mathtt{A},i)$, $\mathsf{FA}\left(\mathtt{A}\right)$ and $\mathsf{RQ}(\mathtt{A},{\mathtt{A}}_1)$ are the algorithms corresponding to the transformations $\mathrm{S}\mathrm{U}\left(\mathtt{L}\right(\mathtt{A}\left)\right)$, $\mathrm{P}\mathrm{R}\left(\mathtt{L}\right(\mathtt{A}\left)\right)$, $\mathrm{S}\mathrm{U}\left(\mathcal{L}\right(\mathtt{A}),i)$, $Fa\left(\mathcal{L}\right(\mathtt{A}\left)\right)$ and $\mathrm{R}\mathrm{Q}(\mathcal{L}\left(\mathtt{A}\right),\mathcal{L}\left({\mathtt{A}}_1\right))$, respectively. Namely, $\forall \mathtt{A},{\mathtt{A}}_1\in \mathrm{D}{\mathrm{FA}}_{/\equiv }$, $i\in \mathbb{N}$, the following facts holds:

$$\begin{array}{c}\mathrm{S}\mathrm{U}\left(\mathcal{L}\right(\mathtt{A}\left)\right)=\mathcal{L}\left(\mathsf{SU}\right(\mathtt{A}\left)\right)\mathrm{P}\mathrm{R}\left(\mathcal{L}\right(\mathtt{A}\left)\right)=\mathcal{L}\left(\mathsf{PR}\right(\mathtt{A}\left)\right)Fa\left(\mathcal{L}\right(\mathtt{A}\left)\right)=\mathcal{L}\left(\mathsf{FA}\right(\mathtt{A}\left)\right)\hfill \\ \mathrm{R}\mathrm{Q}(\mathcal{L}\left(\mathtt{A}\right),\mathcal{L}\left({\mathtt{A}}_1\right))=\mathcal{L}\left(\mathsf{RQ}(\mathtt{A},{\mathtt{A}}_1)\right)\mathrm{S}\mathrm{U}(\mathcal{L}\left(\mathtt{A}\right),i)=\mathcal{L}\left(\mathsf{SU}(\mathtt{A},i)\right)\hfill \end{array}$$

5.1. Abstract Semantics of Substring

In this section we define the abstract semantics of substring. In particular, we define the operator SS${}^{♯}:\mathrm{D}{\mathrm{FA}}_{/\equiv }\times \mathsf{Const}\times \mathsf{Const}\to \mathrm{D}{\mathrm{FA}}_{/\equiv }$, that takes as input an automaton and two constant integer indexes $i,j\in \mathsf{Const}$, and computes the automaton recognizing the set of all substrings of the input automata language between the two provided integer indexes. Since the abstract semantics has to take into account the swaps when the initial index is greater than the final one, several cases arise when one of the two integer parameters is unknown, namely when it is equal to ${\top }_{\mathsf{Const}}$. Indeed, the abstract semantics SS${}^{♯}$ is divided in four cases that are reported in

Table 1. Definition of ${\mathsf{SS}}^{♯}$.

${\mathsf{SS}}^{♯}(\mathtt{A},\mathbf{i},\mathbf{j})$	$\mathbf{j}\in \mathbb{Z}$$(\mathbf{j}\ne {\mathbf{\top }}_{\mathsf{Const}})$	$\mathbf{j}={\mathbf{\top }}_{\mathsf{Const}}$
$\mathbf{i}\in \mathbb{Z}$$(\mathbf{i}\ne {\top }_{\mathsf{Const}})$	$\mathsf{SS}(\mathtt{A},\mathsf{min}(i,j),\mathsf{max}(i,j\left)\right)$	${⨆}_{a\in [0,i]}\mathsf{SS}(\mathtt{A},a,i)$ ⊔ ${\mathsf{SS}}^{\leftrightarrow }(\mathtt{A},i)$
$\mathbf{i}={\top }_{\mathsf{Const}}$	${⨆}_{a\in [0,j]}\mathsf{SS}(\mathtt{A},a,j)$ ⊔ ${\mathsf{SS}}^{\leftrightarrow }(\mathtt{A},j)$	$\mathsf{FA}\left(\mathtt{A}\right)$

. Consider $\mathtt{A}\in \mathrm{D}{\mathrm{FA}}_{/\equiv }$, $i,j\in \mathsf{Const}$ (for the sake of readability we denote by ⊔ the automata lub ${\bigsqcup }_{\mathrm{D}\mathrm{FA}}$, and by ⊓ the glb ${\sqcap }_{\mathrm{D}\mathrm{FA}}$). As in the concrete semantics of $\mathtt{substring}$, negative integer values are treated as zero.

• If $i,j\in \mathbb{Z}$ (second row, second column of Table 1) we have to compute the language of all the substrings between the initial index i and a final index in j, i.e., $Ss\left(\mathcal{L}\right(\mathtt{A}),i,j)$. For example, let $\mathtt{L}={\left\{a\right\}}^{*}\cup \{hello,bc\}$, the set of its substrings from 1 to 3 is $Ss(\mathtt{L},1,3)=\{ϵ,a,aa,el,c\}$. When $i<j$, as in the example, the automaton accepting this language is computed by the operator

  $$\begin{array}{c}\mathsf{SS}(\mathtt{A},i,j)\stackrel{\mathrm{def}}{=}(\mathsf{RQ}(\mathsf{SU}(\mathtt{A},i),\mathsf{SU}(\mathtt{A},j))\sqcap \mathsf{Min}\left({\Sigma }^{j-i}\right))\bigsqcup (\mathsf{SU}(\mathtt{A},i)\sqcap \mathsf{Min}\left({\Sigma }^{<j-i}\right))\hfill \end{array}$$
  If $j>i$, the integer arguments are simply swapped, as in the Table 1.
• When both integer parameters correspond to ${\top }_{\mathsf{Const}}$, the result is the automaton of all possible factors of $\mathtt{A}$ (third row, third column), i.e., $\mathsf{FA}\left(\mathtt{A}\right)$.
• When i is defined and $j={\top }_{\mathsf{Const}}$ (second row, third column), we have to compute the automaton recognizing all the substrings of $\mathcal{L}\left(\mathtt{A}\right)$ from 0 to i and any substring starting from i. For example, let us consider ${\mathsf{SS}}^{♯}(\mathsf{Min}\left(\left\{helloworld\right\}\right),5,{\top }_{\mathsf{Const}})$. Due to the semantics of substring reported in Section 3, we need to compute the substring from $a\in [0,5]$ to 5 and then any substring with initial index equal to 5. The automata recognizing any substring starting at a specific index l is defined as ${\mathsf{SS}}^{\leftrightarrow }(\mathtt{A},l)\stackrel{\mathrm{def}}{=}\mathsf{FA}\left(\mathsf{SU}(\mathtt{A},l)\right)$. The abstract semantics returns the least upper bound of all the automata of substrings from a in $[0,i]$ to the automata recognizing any substring with initial index equals to i.
• Similarly to the previous case, when j is defined and $i={\top }_{\mathsf{Const}}$ (third row, second column), we have to compute the automaton recognizing all the substring of $\mathcal{L}\left(\mathtt{A}\right)$ from 0 to j and any substring starting from j. Let us consider ${\mathsf{SS}}^{♯}(\mathsf{Min}\left(\left\{helloworld\right\}\right),{\top }_{\mathsf{Const}},5)$. Similarly to the previous case, we compute the substrings from $a\in [0,5]$ to 5 and then any substring with initial index equal to 5. The abstract semantics therefore returns the least upper bound of all the automata of substrings from a in $[0,j]$ to the automata recognizing any substring with initial index equal to j.

In Figure 7 we report an example obtained applying the rules in the table.

Theorem 2.

${\mathsf{SS}}^{♯}$ is sound and complete. Formally,

$$\forall \mathtt{A}\in \mathrm{D}{\mathrm{FA}}_{/\equiv },i,j\in \mathsf{Const}.\mathrm{S}\mathrm{S}(\mathcal{L}\left(\mathtt{A}\right),\gamma \left(i\right),\gamma \left(j\right))=\mathcal{L}\left({\mathsf{SS}}^{♯}(\mathtt{A},i,j)\right)$$

From here on, when we say completeness we mean forward completeness. As highlighted in Section 2, this is the only form of completeness we can ensure in absence of a Galois connection. In particular, when an abstract operation (e.g., ${\mathsf{SS}}^{♯}$) is forward complete for a concrete operation (e.g., Ss) means that the computation on the abstract domain (e.g., $\mathrm{D}{\mathrm{FA}}_{/\equiv }$) does not lose information due to the necessary computation only on abstract elements.

5.2. Abstract Semantics of charAt

The abstract semantics of charAt should return an automaton accepting the language of the characters at position i in the strings accepted by the given automaton. Since charAt is a particular case of substring, its abstract semantics, determined by ${\mathsf{CA}}^{♯}:\mathrm{D}{\mathrm{FA}}_{/\equiv }\times \mathsf{Const}\to \mathrm{D}{\mathrm{FA}}_{/\equiv }$, relies on the abstract semantic of substring previously defined. In particular,

$${\mathsf{CA}}^{♯}(\mathtt{A},i)\stackrel{\mathrm{def}}{=}\left\{\begin{array}{cc}{\mathsf{SS}}^{♯}(\mathtt{A},i,i+1)\hfill & i\ne {\top }_{\mathsf{Const}}\hfill \\ \mathsf{Min}\left(\mathtt{chars}\right(\mathtt{A}\left)\right)\bigsqcup \mathsf{Min}\left(\right\{ϵ\left\}\right)\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

We call ${\mathsf{SS}}^{♯}$ (defined before) when the index i corresponds to a determinate integer value otherwise we use the function $\mathtt{chars}:\mathrm{D}{\mathrm{FA}}_{/\equiv }\to \wp (\Sigma )$, returning the set of characters read in any transition of an automaton, together with $\mathsf{Min}\left(\right\{ϵ\left\}\right)$.

Theorem 3.

${\mathsf{CA}}^{♯}$ is sound and complete. Formally,

$$\forall \mathtt{A}\in \mathrm{D}{\mathrm{FA}}_{/\equiv },i\in \mathsf{Const}.\mathrm{C}\mathrm{A}(\mathcal{L}\left(\mathtt{A}\right),\gamma \left(i\right))=\mathcal{L}\left({\mathsf{CA}}^{♯}(\mathtt{A},i)\right)$$

5.3. Abstract Semantics of length

The abstract semantics of length should return a value, of the integer domain $\mathsf{Const}$, that, in a sound way, approximates the length of all the possible strings of an automaton. The abstract semantics of length is defined by the function ${\mathsf{LE}}^{♯}:\mathrm{D}{\mathrm{FA}}_{/\equiv }\to \mathsf{Const}$, computed by Algorithm 1, where $\mathsf{Paths}:\mathrm{D}{\mathrm{FA}}_{/\equiv }\to \wp \left(\wp \left(Q\right)\right)$ returns the set of the paths from the initial state to any final state of $\mathtt{A}$ [35]. Given a path $\mathsf{p}\in \mathsf{Paths}\left(\mathtt{A}\right)$, we denote by $\left|\mathsf{p}\right|$ the length of $\mathsf{p}$.

Algorithm 1:${\mathsf{LE}}^{♯}:\mathrm{D}{\mathrm{FA}}_{/\equiv }\to \mathsf{Const}$ algorithm

If the input automaton has cycles, ${\mathsf{LE}}^{♯}$ returns ${\top }_{\mathsf{Const}}$ otherwise it checks that any path of the automaton $\mathtt{A}$ has the same length (lines 5–8). Whenever the algorithm finds that there exists two paths in the automaton that have different lengths, ${\top }_{\mathsf{Const}}$ is returned (lines 8–10). Due to the constant integers domain, the abstract semantics of length can give a precise answer only when any string of the automaton has precisely the same length. More accurate results can be obtained by using more precise integer abstract domains, e.g., intervals, as we will discuss in Section 6. For example, consider the automata $\mathtt{A}$ and ${\mathtt{A}}^{\prime }$ in Figure 8a,b, respectively. ${\mathsf{LE}}^{♯}\left(\mathtt{A}\right)$ precisely returns 5, since all the strings recognized by $\mathtt{A}$ have the same length, while ${\mathsf{LE}}^{♯}\left({\mathtt{A}}^{\prime }\right)$ returns ${\top }_{\mathsf{Const}}$.

Theorem 4.

${\mathsf{LE}}^{♯}$ is sound and complete. Formally,

$$\forall \mathtt{A}\in \mathrm{D}{\mathrm{FA}}_{/\equiv }.\mathrm{L}\mathrm{E}\left(\mathcal{L}\left(\mathtt{A}\right)\right)=\gamma \left({\mathsf{LE}}^{♯}\left(\mathtt{A}\right)\right)$$

5.4. Abstract Semantics of Concat

The abstract semantics of string concatenation is ${\mathsf{CC}}^{♯}:\mathrm{D}{\mathrm{FA}}_{/\equiv }\times \mathrm{D}{\mathrm{FA}}_{/\equiv }\to \mathrm{D}{\mathrm{FA}}_{/\equiv }$ and returns the concatenation between the input automata. Since regular languages are closed under the concatenation operation, so are finite state automata. Hence, ${\mathsf{CC}}^{♯}$ exactly implements the standard concatenation operation between automata. Given the closure property on automata, the following result holds.

Theorem 5.

The function ${\mathsf{CC}}^{♯}$ is sound and complete. Formally, $\forall \mathtt{A},{\mathtt{A}}^{\prime }\in \mathrm{D}{\mathrm{FA}}_{/\equiv }.$

$$\mathrm{C}\mathrm{C}(\mathcal{L}\left(\mathtt{A}\right),\mathcal{L}\left({\mathtt{A}}^{\prime }\right))=\mathcal{L}\left({\mathsf{CC}}^{♯}(\mathtt{A},{\mathtt{A}}^{\prime })\right)$$

As we have already mentioned before, completeness holds thanks to the closure properties of regular languages (and in turn of finite state automata).

5.5. Abstract Semantics of StartsWith

The abstract semantics of startsWith takes as input two automata and checks whether a string of the language of the first automaton starts with a string of the language of the second one. The abstract semantics of startsWith is captured by the function ${\mathsf{SW}}^{♯}:\mathrm{D}{\mathrm{FA}}_{/\equiv }\times \mathrm{D}{\mathrm{FA}}_{/\equiv }\to {\mathbb{B}}^{♯}$, computed by Algorithm 2, where $\mathsf{maxString}:\mathrm{D}{\mathrm{FA}}_{/\equiv }\to \mathrm{D}{\mathrm{FA}}_{/\equiv }$ returns the (minimal) automaton recognizing the longest string of the automaton given as input and $\mathsf{isSinglePath}:\mathrm{D}{\mathrm{FA}}_{/\equiv }\to \{\mathtt{true},\mathtt{false}\}$ checks whether the input automata $\mathtt{A}=(Q,\Sigma ,\delta ,q_0,F)$ respect the following condition: $\delta ={\bigcup }_{i\in [0,|Q\left|\right]}(q_i,q_{i+1},c)$. Informally, a single-path automaton is an automaton where, if we sort the strings of its language from the shortest to the longest, each string is a prefix of the next one. An example of a single-path automaton is reported in Figure 9b where it is graphically clear that each state, excluding the initial and last one, have one incoming and one outgoing transition. Since the longest string in a single-path automaton has, as prefix, all the others of the language, it is sufficient to check, for an automaton $\mathtt{A}$, if it starts with only the former. For example, let $\mathcal{L}\left(\mathtt{A}\right)=\left\{softer\right\}$ and $\mathcal{L}\left({\mathtt{A}}^{\prime }\right)=\{s,so,soft\}$. The string s is prefix of $so$, which is in turn prefix of $soft$ so ${\mathtt{A}}^{\prime }$ is a single-path automaton. Therefore, in this case, it is sufficient to check if $softer$ starts with only $soft$ (the longest string of $\mathcal{L}\left({\mathtt{A}}^{\prime }\right)$) since, being ${\mathtt{A}}^{\prime }$ single-path, the other strings (s and $so$) are consequently prefix of $softer$. Instead, consider $\mathcal{L}\left({\mathtt{A}}^{\prime }\right)=\{s,no\}$. It would be impossible for a string to start with both of them since there is no prefix relation between them.

Algorithm 2:${\mathsf{SW}}^{♯}:\mathrm{D}{\mathrm{FA}}_{/\equiv }\times \mathrm{D}{\mathrm{FA}}_{/\equiv }\to {\mathbb{B}}^{♯}$ algorithm

Algorithm 2 takes as input two automata denoted by $\mathtt{A}$ and ${\mathtt{A}}^{\prime }$. Lines 1-9 handle some corner cases. If $\mathcal{L}\left({\mathtt{A}}^{\prime }\right)=\left\{\epsilon \right\}$, $\left\{\mathtt{true}\right\}$ is returned, since any string starts with $\epsilon$ (lines 1-3). If none of the prefixes of $\mathtt{A}$ is recognized by ${\mathtt{A}}^{\prime }$, meaning that none of the strings recognized by $\mathtt{A}$ start with a string of ${\mathtt{A}}^{\prime }$, we can safely return $\left\{\mathtt{false}\right\}$ (lines 4-6). Finally, if at least one of the input automata have cycles, we return $\{\mathtt{true},\mathtt{false}\}$ (lines 7-9). Lines 10-17 determine if any string of ${\mathtt{A}}^{\prime }$ is the beginning of any string of $\mathtt{A}$, otherwise ${\top }_{\mathsf{Bool}}$ is returned.

In order to explain our approach in lines 10-17, consider the automata $\mathtt{A}$ and ${\mathtt{A}}^{\prime }$ reported in Figure 9. To be sure that any string recognized by ${\mathtt{A}}^{\prime }$ is the beginning of any string recognized by $\mathtt{A}$ we need to check two conditions: (1) any string recognized by ${\mathtt{A}}^{\prime }$ is prefix of its longest recognized string $\sigma$ and (2) each string in $\mathtt{A}$ starts with $\sigma$ (all strings must have a common prefix). Only if both conditions occur we can safely return $\left\{\mathtt{true}\right\}$ otherwise we return ${\top }_{\mathsf{Bool}}$. In particular, (1) is checked by the function $\mathsf{isSinglePath}$ at line 10 and (2) is checked at lines 11-15. It is worth noting that if an automaton is single-path, then the longest string is unique (line 11).

In our example, both the strings p and $pan$ in $\mathcal{L}\left({\mathtt{A}}^{\prime }\right)$ are prefixes of $pan$, which is the longest string recognized by ${\mathtt{A}}^{\prime }$, so we build B, which is the (minimal) automaton that recognizes $pan$ and C, $\mathcal{L}\left(\mathtt{C}\right)=\{pan,koa\}$, and compare them (line 13). We return $\left\{\mathtt{true}\right\}$ if B and C recognize the same language otherwise we return ${\top }_{\mathsf{Bool}}$. In the other cases, as already mentioned, we return $\{\mathtt{true},\mathtt{false}\}$. For example, in Figure 9, $\{\mathtt{true},\mathtt{false}\}$ is returned because, although ${\mathtt{A}}^{\prime }$ is a single-path automaton, only the string $panda\in \mathcal{L}\left(\mathtt{A}\right)$ begins with $pan$, namely the longest string of $\mathcal{L}\left({\mathtt{A}}^{\prime }\right)$.

Example 2.

Consider for example $\mathtt{A}$ s.t. $\mathcal{L}\left(\mathtt{A}\right)=\{panda,panem\}$, and ${\mathtt{A}}^{\prime }$ s.t. $\mathcal{L}\left({\mathtt{A}}^{\prime }\right)=\{p,pan\}$. ${\mathsf{SW}}^{♯}(\mathtt{A},{\mathtt{A}}^{\prime })$ returns $\left\{\mathtt{true}\right\}$ since ${\mathtt{A}}^{\prime }$ is a single-path automaton and both strings of $\mathtt{A}$ start with the longest string in ${\mathtt{A}}^{\prime }$, the string $pan$. Consider instead the automata $\mathtt{A},\mathcal{L}\left(\mathtt{A}\right)=\{panda,koala\}$, and $\mathtt{A},\mathcal{L}\left(\mathtt{A}\right)=\{p,k\}$. In this case, ${\mathsf{SW}}^{♯}(\mathtt{A},{\mathtt{A}}^{\prime })$ returns $\{\mathtt{true},\mathtt{false}\}$ since ${\mathtt{A}}^{\prime }$ is not a single-path automaton. Indeed, we can easily check that even if the string $panda\in \mathcal{L}\left(\mathtt{A}\right)$ starts with $p\in \mathcal{L}\left({\mathtt{A}}^{\prime }\right)$, the string $koala\in \mathcal{L}\left(\mathtt{A}\right)$ does not.

Theorem 6.

${\mathsf{SW}}^{♯}$ is sound but not complete. Formally,

$$\forall \mathtt{A},{\mathtt{A}}^{\prime }\in \mathrm{D}{\mathrm{FA}}_{/\equiv }.\mathrm{S}\mathrm{W}(\mathcal{L}\left(\mathtt{A}\right),\mathcal{L}\left({\mathtt{A}}^{\prime }\right))⊊{\mathsf{SW}}^{♯}(\mathtt{A},{\mathtt{A}}^{\prime })$$

As a counterexample to completeness, consider the automata $\mathtt{A}$ s.t. $\mathcal{L}\left(\mathtt{A}\right)=\mathtt{A}=\left\{a^n\right|n>1\}$ and ${\mathtt{A}}^{\prime }$ s.t. $\mathcal{L}\left({\mathtt{A}}^{\prime }\right)=\left\{a\right\}$. The completeness condition is not met, indeed,

$${\mathsf{SW}}^{♯}(\mathtt{A},{\mathtt{A}}^{\prime })=\{\mathtt{true},\mathtt{false}\}\cancel{\subset }\mathrm{S}\mathrm{W}(\mathcal{L}\left(\mathtt{A}\right),\mathcal{L}\left({\mathtt{A}}^{\prime }\right))=\left\{\mathtt{true}\right\}$$

5.6. Abstract Semantics of ToLowerCase

The abstract semantics of toLowerCase is defined by the function ${\mathsf{LC}}^{♯}:\mathrm{D}{\mathrm{FA}}_{/\equiv }\to \mathrm{D}{\mathrm{FA}}_{/\equiv }$ which returns as result an automaton that recognizes the same strings of the input automaton, where any upper-case symbol is replaced with the corresponding lower-case symbol. ${\mathsf{LC}}^{♯}$ is computed by Algorithm 3.

Algorithm 3:${\mathsf{LC}}^{♯}:\mathrm{D}{\mathrm{FA}}_{/\equiv }\to \mathrm{D}{\mathrm{FA}}_{/\equiv }$ algorithm

Starting from an input automaton $\mathtt{A}$, the idea is to return as result the automaton ${\mathtt{A}}^{\prime }$, that is a copy of $\mathtt{A}$ with the exception that any upper-case symbol read by a transition is replaced by its corresponding lower-case symbol. Transitions that already read lower-case or special symbols are unaltered. An example is reported in Figure 10.

Theorem 7.

${\mathsf{LC}}^{♯}$ is both sound and complete. Formally,

$$\forall \mathtt{A}\in \mathrm{D}{\mathrm{FA}}_{/\equiv }.\mathrm{L}\mathrm{C}\left(\mathcal{L}\left(\mathtt{A}\right)\right)=\mathcal{L}\left({\mathsf{LC}}^{♯}\left(\mathtt{A}\right)\right)$$

5.7. Abstract Semantics of Includes

The abstract semantics of $\mathtt{includes}$ is defined by the function ${\mathsf{IN}}^{♯}:\mathrm{D}{\mathrm{FA}}_{/\equiv }\times \mathrm{D}{\mathrm{FA}}_{/\equiv }\to {\mathbb{B}}^{♯}$. It takes as input two automata $\mathtt{A}$ and ${\mathtt{A}}^{\prime }$ and checks whether a string recognized by ${\mathtt{A}}^{\prime }$ is a substring of a string recognized by $\mathtt{A}$. The function ${\mathsf{IN}}^{♯}$ is computed by Algorithm 4, where, given a path $\mathsf{p}$ of an automaton $\mathtt{A}$, we abuse notation denoting by $\mathsf{Min}\left(\mathsf{p}\right)$ the automaton that recognizes the string encoded by the path $\mathsf{p}$ (lines 11–12). The algorithm first checks some corner cases: if ${\mathtt{A}}^{\prime }$ only recognizes the empty string, {true} is returned, since the empty string is always a substring of a non-empty automaton (lines 2–4), if none of the substring of $\mathtt{A}$ is contained in ${\mathtt{A}}^{\prime }$, $\left\{\mathtt{false}\right\}$ is returned (lines 5–7) and if one of the input automata is cyclic, it returns ${\top }_{\mathsf{Bool}}$ (lines 8–10). When these corner cases are excluded, we check each string recognized by $\mathtt{A}$. If the algorithm finds at least one string ${\sigma }^{\prime }$ in $\mathcal{L}\left({\mathtt{A}}^{\prime }\right)$ that is not a substring of a string $\sigma$ of $\mathtt{A}$, ${\top }_{\mathsf{Bool}}$ is returned otherwise $\left\{\mathtt{true}\right\}$. This is done in lines 10–14 where, for each path $\mathsf{p}$ of $\mathtt{A}$ we create $\mathsf{Min}\left(\mathsf{p}\right)$ and check if its factorization with ${\mathtt{A}}^{\prime }$ equals ${\mathtt{A}}^{\prime }$, i.e., we check if it contains any string of ${\mathtt{A}}^{\prime }$.

Algorithm 4:${\mathsf{IN}}^{♯}:\mathrm{D}{\mathrm{FA}}_{/\equiv }\times \mathrm{D}{\mathrm{FA}}_{/\equiv }\to {\mathbb{B}}^{♯}$ algorithm

For example, consider the automata $\mathtt{A}$ and ${\mathtt{A}}^{\prime }$ reported in Figure 11. The algorithm returns ${\top }_{\mathsf{Bool}}$ since the string $fg\in \mathcal{L}\left({\mathtt{A}}^{\prime }\right)$ is not a substring of $abc\in \mathtt{A}$. Another example is reported in Figure 12. The result of ${\mathsf{IN}}^{♯}(\mathtt{A},{\mathtt{A}}^{\prime })$ returns $\left\{\mathtt{true}\right\}$ since $\forall \sigma \in \mathcal{L}\left(\mathtt{A}\right),\forall {\sigma }^{\prime }\in \mathcal{L}\left({\mathtt{A}}^{\prime }\right).{\sigma }^{\prime }$ is a substring of $\sigma$.

Theorem 8.

${\mathsf{IN}}^{♯}$ is sound but not complete. Formally,

$$\forall \mathtt{A},{\mathtt{A}}^{\prime }\in \mathrm{D}{\mathrm{FA}}_{/\equiv }.\mathrm{I}\mathrm{N}(\mathcal{L}\left(\mathtt{A}\right),\mathcal{L}\left({\mathtt{A}}^{\prime }\right))⊊{\mathsf{IN}}^{♯}(\mathtt{A},{\mathtt{A}}^{\prime }).$$

As a counterexample to completeness, consider the automaton $\mathtt{A}$ s.t. $\mathcal{L}\left(\mathtt{A}\right)=\left\{a^n\right|n>1\}$ and the automaton ${\mathtt{A}}^{\prime }$ s.t. $\mathcal{L}\left({\mathtt{A}}^{\prime }\right)=\left\{a\right\}$. The completeness condition is not met, indeed

$$\mathrm{I}\mathrm{N}(\mathcal{L}\left(\mathtt{A}\right),\mathcal{L}\left({\mathtt{A}}^{\prime }\right))=\left\{\mathtt{true}\right\}\ne {\mathsf{IN}}^{♯}(\mathtt{A},{\mathtt{A}}^{\prime })=\{\mathtt{true},\mathtt{false}\}$$

since when one of the input automata is cyclic, Algorithm 4 returns ${\top }_{\mathsf{Bool}}$.

5.8. Abstract Semantics of Repeat

The abstract semantics of repeat is defined by the function ${\mathsf{RT}}^{♯}:\mathrm{D}{\mathrm{FA}}_{/\equiv }\times \mathsf{Const}\to \mathrm{D}{\mathrm{FA}}_{/\equiv }$ that, given as input an automaton $\mathtt{A}$ and a constant integer value i, returns an automaton that recognizes any string of $\mathcal{L}\left(\mathtt{A}\right)$ repeated i times. ${\mathsf{RT}}^{♯}$ is computed by Algorithm 5 and we suppose that the abstract integer value i is positive or zero. Any non-positive value is treated as zero. The algorithm first checks some corner cases. If $i=0$ or the input automaton only recognizes the empty string, then $\mathsf{Min}\left(ϵ\right)$ is returned (lines 1–3). If the automaton has a cycle or $i={\top }_{\mathsf{Const}}$, it returns the Kleene-closure of the input automaton (lines 4–6). If none of these corner cases is detected then, for each string in $\mathcal{L}\left(\mathtt{A}\right)$, we concatenate it with itself $(i-1)$-times using the already defined ${\mathsf{CC}}^{♯}$. The result is the union of all the concatenated automata.

Algorithm 5:${\mathsf{RT}}^{♯}:\mathrm{D}{\mathrm{FA}}_{/\equiv }\times \mathsf{Const}\to \mathrm{D}{\mathrm{FA}}_{/\equiv }$ algorithm

Let us consider the automaton $\mathtt{A}$ reported in Figure 13a and suppose to call ${\mathsf{RT}}^{♯}(\mathtt{A},2)$. The resulting automaton, applying Algorithm 5, is reported in Figure 13b. Let us suppose to call ${\mathsf{RT}}^{♯}(\mathtt{A},{\top }_{\mathsf{Const}})$. In this case, since the input integer value is not determinate, Algorithm 5 returns the Kleene-star automaton of $\mathtt{A}$ and the result is reported in Figure 13c.

Theorem 9.

${\mathsf{RT}}^{♯}$ is sound but not complete. Formally,

$$\forall \mathtt{A}\in \mathrm{D}{\mathrm{FA}}_{/\equiv },\forall i\in \mathsf{Const}.\mathrm{R}\mathrm{T}(\mathcal{L}\left(\mathtt{A}\right),\gamma \left(i\right))⊊\mathcal{L}\left({\mathsf{RT}}^{♯}(\mathtt{A},i)\right)$$

As a counterexample to completeness, consider the automaton $\mathtt{A}$ s.t. $\mathcal{L}\left(\mathtt{A}\right)=\left\{a{b}^n\right|n\in \mathbb{N}\}$. The completeness condition is not met, indeed

$$\mathrm{R}\mathrm{T}(\mathcal{L}\left(\mathtt{A}\right),2)=\left\{a{b}^{n}a{b}^n\right|n\in \mathbb{N}\}\ne {\mathsf{RP}}^{♯}(\mathtt{A},2)=\left\{{\left(a{b}^n\right)}^m\right|n,m\in \mathbb{N}\}$$

since when the input automaton is cyclic, Algorithm 5 returns the Kleene closure of the input automaton.

5.9. Abstract Semantics of TrimLeft, TrimRight and Trim

In this section, we will show the abstract semantics of trimLeft, trimRight and trim operations. The abstract semantics of trimLeft is defined by the function ${\mathsf{TL}}^{♯}:\mathrm{D}{\mathrm{FA}}_{/\equiv }\to \mathrm{D}{\mathrm{FA}}_{/\equiv }$. In particular, it takes as input an automaton $\mathtt{A}$ and returns an automaton accepting the same strings of $\mathtt{A}$ removing, at the beginning of each string, consecutive white spaces, if present. In the following, we denote a white-space as ␣. The function is computed by Algorithm 6. The idea of algorithm is to iteratively replace white-space transitions from the initial state with $ϵ$-transition (lines 5–7), while leaving the other transitions unaltered (lines 7–9). At each iteration, the resulting automaton is minimized, and hence determinized (line 11). This operation is repeated until the initial state has no white-space transitions, checking the condition that white-space is not a prefix of the automaton (line 3). In Figure 14 is depicted an example of application of our algorithm.

Algorithm 6:${\mathsf{TL}}^{♯}:\mathrm{D}{\mathrm{FA}}_{/\equiv }\to \mathrm{D}{\mathrm{FA}}_{/\equiv }$ algorithm

Theorem 10.

${\mathsf{TL}}^{♯}$ is sound and complete. Formally, $\forall \mathtt{A}\in \mathrm{D}{\mathrm{FA}}_{/\equiv }$,

$$\mathrm{T}\mathrm{L}\left(\mathcal{L}\left(\mathtt{A}\right)\right)=\mathcal{L}\left({\mathsf{TL}}^{♯}\left(\mathtt{A}\right)\right)$$

The abstract semantics of trimRight can be defined in function of the already defined function ${\mathsf{TL}}^{♯}$. Indeed, the abstract semantics ${\mathsf{TR}}^{♯}:\mathrm{D}{\mathrm{FA}}_{/\equiv }\to \mathrm{D}{\mathrm{FA}}_{/\equiv }$ reserves the input automaton, applies ${\mathsf{TL}}^{♯}$ and finally reverses again the so obtained automaton. Formally,

$${\mathsf{TR}}^{♯}\stackrel{\mathrm{def}}{=}\mathsf{reverse}({\mathsf{TL}}^{♯}\left(\mathsf{reverse}\left(\mathtt{A}\right)\right)$$

Similarly, the abstract semantics of trim applies both the abstract semantics of trimLeft and trimRight. Thus, the abstract semantics of trim is captured by the function ${\mathsf{TM}}^{♯}:\mathrm{D}{\mathrm{FA}}_{/\equiv }\to \mathrm{D}{\mathrm{FA}}_{/\equiv }$ and it is defined as

$${\mathsf{TM}}^{♯}\left(\mathtt{A}\right)\stackrel{\mathrm{def}}{=}{\mathsf{TR}}^{♯}\left({\mathsf{TL}}^{♯}\left(\mathtt{A}\right)\right)$$

Theorem 11.

${\mathsf{TR}}^{♯}$ and ${\mathsf{TM}}^{♯}$ are sound and complete. Formally, $\forall \mathtt{A}\in \mathrm{D}{\mathrm{FA}}_{/\equiv }$

$$\mathrm{T}\mathrm{R}\left(\mathcal{L}\left(\mathtt{A}\right)\right)=\mathcal{L}\left({\mathsf{TR}}^{♯}\left(\mathtt{A}\right)\right)\mathrm{T}\mathrm{M}\left(\mathcal{L}\left(\mathtt{A}\right)\right)=\mathcal{L}\left({\mathsf{TM}}^{♯}\left(\mathtt{A}\right)\right)$$

Proof. 

The proof of ${\mathsf{TR}}^{♯}$ follows from the completeness of ${\mathsf{TL}}^{♯}$ and $\mathsf{reverse}$ operations, while the proof of ${\mathsf{TM}}^{♯}$ follows from the completeness of ${\mathsf{TL}}^{♯}$ and ${\mathsf{TR}}^{♯}$. □

5.10. Concerning Abstract Implicit Type Conversion

In this section, we discuss the abstraction of implicit type conversion functions. Here we will focus only on the conversion of automata into other values, since conversions concerning booleans, not-a-number and integers are standard. Let ${\mathtt{toBool}}^{♯}:{\mathbb{V}}^{♯}\to {\mathbb{B}}^{♯}$ be applied to $\mathtt{A}\in \mathrm{D}{\mathrm{FA}}_{/\equiv }$: If $\mathtt{A}\sqcap \mathsf{Min}\left(\right\{ϵ\left\}\right)=\mathsf{Min}\left(\mathsf{⌀}\right)$, it returns $\left\{\mathtt{true}\right\}$, when $\mathtt{A}=\mathsf{Min}\left(\right\{ϵ\left\}\right)$ the function returns $\left\{\mathtt{false}\right\}$, otherwise the function return ${\top }_{\mathsf{Bool}}$. Implicit type conversion to $\mathrm{D}{\mathrm{FA}}_{/\equiv }$ is handled by the function ${\mathtt{toStr}}^{♯}:{\mathbb{V}}^{♯}\to \mathrm{D}{\mathrm{FA}}_{/\equiv }$. As far as non numeric strings are concerned, ${\mathtt{toStr}}^{♯}$ returns $\mathsf{Min}\left(\right\{\mathtt{NaN}\left\}\right)$. If the input is the boolean value $\mathtt{true}$ [$\mathtt{false}$] it returns $\mathsf{Min}\left(\right\{\mathtt{true}\left\}\right)$ [$\mathsf{Min}\left(\right\{\mathtt{false}\left\}\right)$], otherwise it returns $\mathsf{Min}\left(\right\{\mathtt{true}\left\}\right)\bigsqcup \mathsf{Min}\left(\right\{\mathtt{false}\left\}\right)$. Regarding abstract integers, if $i\in \mathbb{Z}$, then the automaton recognizing the string $\mathcal{S}\left(i\right)$ is returned (We recall that the function $\mathcal{S}\left(i\right)$ maps an integer i to its numeric string representation.), otherwise, hence when $i={\top }_{\mathsf{Const}}$, the automaton recognizing any possible integer is returned and reported in Figure 15. Finally, ${\mathtt{toInt}}^{♯}:{\mathbb{V}}^{♯}\to \mathsf{Const}\cup \left\{\mathtt{NaN}\right\}$ handles conversion to constant integers. Given an automaton $\mathtt{A}$, if $\mathtt{A}\sqcap \mathsf{Min}\left({\Sigma }_{\mathbb{Z}}\right)=\mathsf{Min}\left(\mathsf{⌀}\right)$, the automaton is precisely converted to $\mathtt{NaN}$, since $\mathtt{A}$ does not recognize any numerical string. Otherwise, if $\mathtt{A}{⊑}_{\mathrm{D}\mathrm{FA}}\mathsf{Min}\left({\Sigma }_{\mathbb{Z}}\right)$ it means that $\mathcal{L}\left(\mathtt{A}\right)$ contains only numeric strings. In particular, if $\mathtt{A}$ recognizes only one numerical string, the corresponding integer is returned, otherwise ${\top }_{\mathsf{Const}}$ is returned.

6. $\mathbf{\mu }\mathrm{F}\mathrm{ASA}$ Implementation

In this section we present $\mu \mathsf{JS}$ Finite-state Automata String Analyzer ($\mu \mathrm{F}\mathrm{ASA}$), the string static analyzer integrating the finite state automata abstract domain, and the corresponding abstract semantics, presented in the previous sections.

6.1. Theoretical Concerns

It is worth noting that, as reported in Theorem 1, $\wp \left({\Sigma }^{*}\right)$ (string concrete domain) and $\mathrm{D}{\mathrm{FA}}_{/\equiv }$ (abstract string domain) do not form a Galois connection, however this is not a concern. We have shown, for the core language we adopted, that the abstract semantics we have defined for string operations guarantee soundness hence, if the abstract interpreter starts from regular initial conditions (i.e., constraints expressible as finite state automata) it will always compute regular invariants.

When implementing, an important issue is computational complexity. The abstract semantics reported in this paper often relies on minimization of finite state automata in order to keep the automata, which arise during abstract computations, determinized and minimized. In the worst case, minimization has exponential complexity but this is not a problem. Even if our library relies on the Brzozowski’s algorithm, which theoretically has exponential complexity in worst-case scenario, in practice it is extremely fast on average and consistently outperforms other minimization algorithms (e.g., Hopcroft’s algorithm, having average-case complexity $O(nlogn)$, where n is the number of states), as reported in [36]. Moreover, the minimization is only applied when the input automaton is not-deterministic.

6.2. Implementation

$\mu \mathrm{F}\mathrm{ASA}$ is a string static analyzer for extended $\mu \mathsf{JS}$ inter-procedural programs and it is built upon the finite state automata abstract domain described above. (Available at www.github.com/SPY-Lab/mufasa) It analyzes string variables and is also able to express associative arrays. The finite state automata abstract domain has been implemented as an external library (Available at www.github.com/SPY-Lab/java-fsm-library), offering a suitable and easy way to plug the domain into existing static analyzers, such as [3,4,5,37]. The library includes the implementation of all the algorithms concerning the finite state automata domain and provides well-known operations on automata such as suffix, right quotient, abstract domain-related operations, such as ${\bigsqcup }_{\mathrm{D}\mathrm{FA}}$, ${\sqcap }_{\mathrm{D}\mathrm{FA}}$, and the parametric widening for tuning precision and forcing convergence.

In addition to the string operations (and the corresponding automata-based abstract semantics) introduced in this paper, $\mu \mathrm{F}\mathrm{ASA}$ also analyzes functions that can be defined as composition of the ones presented here (e.g., endsWith w.r.t. startsWith, slice w.r.t. substring). The full list of implemented string operations is reported in

Table 2. $\mu \mathsf{JS}$ Finite-state Automata String Analyzer ($\mu \mathrm{F}\mathrm{ASA}$) string operations.

String Operation	Soundness	Completeness	Average Complexity
substring	✓	✓	$O(nlogn)$
charAt	✓	✓	$O(nlogn)$
length	✓	✓	$O(n+m)$
concat	✓	✓	$O(nlogn+n+m)$
startsWith endsWith	✓	✗	$O(nlogn+n+m)$
toLowerCase toUpperCase	✓	✓	$O\left(m\right)$
includes	✓	✗	$O(nlogn+n+m)$
repeat	✓	✗	$O(nlogn+n+m)$
replace	✓	✗	$O\left(\right(n+m\left)nlogn\right)$
indexOf	✓	✗	$O\left(n(nlogn)\left(n^{2}m\right)\right)$
slice	✓	✗	$O\left(\right(n+m\left)\right(nlogn\left)\right)$

, also summarizing for which operations holds soundness and completeness and the average complexity of their algorithms (w.r.t. the constant integer abstract domain).

6.3. Extension to Interval Abstract Domain

For the presentation of this paper, in Section 4.1, we have chosen to abstract integer values to the constant integer abstract domain. Of course this affects the abstract semantics of those string operations that involve them, namely substring, charAt, repeat and length, while the other methods only use strings or booleans. Nevertheless, $\mu \mathrm{F}\mathrm{ASA}$ abstracts integer values to the more precise interval abstract domain [9], i.e., to the set $\mathsf{Intervals}$.

$$\mathsf{Intervals}\stackrel{\mathrm{def}}{=}\left\{[a,b]\right|a,b\in \mathbb{Z}\cup \{-\infty ,+\infty \},a\le b\}\cup \left\{\perp \right\}$$

The choice of presenting the automata-based abstract semantics with constant integers, rather than intervals, was driven by the will to not burden the definition of the abstract semantics of the string operations involving integers. Let us consider the interval-based substring abstract semantics. Since intervals can be unbounded (e.g., $[5,+\infty ]$), more than 20 different cases have been identified in its abstract semantics [8]. Given $\mathtt{substr}(\mathtt{A},[a,b],[c,d])$, for some $\mathtt{A}\in \mathrm{D}{\mathrm{FA}}_{/\equiv }$, many of these cases include $b=+\infty$ and d definite value, b definite and $d=+\infty$, $b,d=+\infty$ and $a,c$ definite values and $a\le c$, only to cite few. Moreover, the interval-based abstract semantics does not add any further important technical detail to our contribution since the cases cited above, met with an interval-based analysis, were handled in an ad hoc manner and would have made this paper harder to follow. In particular, being the constant integer abstract domain strictly contained into the intervals one, restricting the presentation to constant integers permitted us to report only the meaningful cases (from a technical point of view), avoiding the others (related to intervals) handled in specific ways (and relevant for the implementation).

Nevertheless, $\mu \mathrm{F}\mathrm{ASA}$ implements intervals (which include constant integers) and, accordingly, the abstract semantics based on them of substring, charAt, length and repeat, as reported in [8]. The abstract semantics of the other string operations remain unaffected by the change. Just as an example, in the following we report the abstract semantics of $\mathtt{length}$ on intervals.

length Abstract Semantics with Intervals

The constant integer domain leads to a big loss in precision in the abstract semantics of length, reported in Section 5.3. The idea behind the algorithm capturing its abstract semantics is to check if any string recognized by the input automata have the same length $l\in \mathbb{N}$. If so, l is returned as result, otherwise ${\top }_{\mathsf{Const}}$ is returned. Clearly, this is a forced choice given by the fact that the constant integer abstract domain is only able to track a single integer value. In this sense, the abstract semantics of $\mathtt{length}$ can be improved, from a precision point of view, when we deal with intervals rather than constant integers. Algorithm 7 reports the abstract semantics of $\mathtt{length}$ using the former abstract domain. We compute the minimum and the maximum path reaching each final state in the automaton and then we abstract the set of lengths obtained so far into intervals. Problems arise when the automaton contains cycles. In that case, we return the undefined interval starting from the minimum path, to a final state, to $+\infty$.

Clearly, using the interval abstract domain produces more precise results for certain operations, but it complicates the abstract semantics of others.

6.4. Qualitative Evaluation of $\mu \mathrm{F}\mathrm{ASA}$

In this section, we evaluate the precision of $\mu \mathrm{F}\mathrm{ASA}$ and, in turn, of the finite state automata abstract domain. In particular, we comment and discuss two string manipulation programs. The first is the one already introduced in Section 1, namely an obfuscated malware manipulating strings and transforming them into code by using eval, while the second is a benevolent function taken from a real-world string manipulation program. In both cases, we will show that important string information can be obtained by $\mu \mathrm{F}\mathrm{ASA}$.

Algorithm 7:${\mathsf{LE}}^{♯}:\mathrm{D}{\mathrm{FA}}_{/\equiv }\to \mathsf{Intervals}$ Algorithm

6.4.1. Obfuscated Malware

Consider the fragment reported in Figure 1 in the introduction. By analyzing it with $\mu \mathrm{F}\mathrm{ASA}$, we obtain that the abstract value of d, at the eval call, is the automaton ${\mathtt{A}}_d$ in Figure 16. The cycles are caused by the widening application in while computations.

From this automaton we are able to retrieve some important and non-trivial information. For example, we are able to answer the following question: May ${\mathtt{A}}_d$ contain a string corresponding to an assignment to an ActiveXObject? We can answer by checking the predicate ${\mathtt{A}}_d\sqcap \mathsf{Min}(\mathbf{Id}·\{newActiveXObject\left(\right\}·{\Sigma }^{*}·\left\{\right)\})\ne \mathsf{⌀}$, controlling whether ${\mathtt{A}}_d$ recognizes strings that are concatenations of any identifier with the string $newActiveXObject$, followed by any possible string. In the example, the predicate returns $\mathtt{true}$. Another interesting information could be: May ${\mathtt{A}}_d$ contain eval string? We can also answer that by checking if ${\mathtt{A}}_d\sqcap \mathsf{Min}\left(\left\{eval\right\}\right)\ne \mathsf{Min}\left(\mathsf{⌀}\right)$. In this case it returns false and enforces the idea that any explicit call to eval cannot occur.

This analysis may lose precision during fix-point computations, causing the cycles in the automaton in Figure 16, due to the widening application. Nevertheless, it is worth noting that this result is obtained without any precision improvement on fix-point computations, such as loop unrolling, narrowing or widening with thresholds, that can surely be implemented in the future development of $\mu \mathrm{F}\mathrm{ASA}$.

6.4.2. String Manipulation Program

In order to evaluate the precision of $\mu \mathrm{F}\mathrm{ASA}$, we decided to create a benchmark of tests taken from real-world programs. We therefore selected string manipulating functions from popular modern frameworks (such as Mozilla useful methods, RXJava, Mockito) whose code can be easily found on GitHub. (The selected string manipulation functions are available at www.github.com/SPY-Lab/mufasa/src/test/resources and it is possible here to go back to where they were selected.) Among this set of methods, we will focus our attention to the precision of the function fixStations reported in Figure 17, taken from [38]. The function takes as input an object stations containing information about train stations (each item contains the three-letter station code, followed by some machine-readable data, followed by a semicolon, followed by the human-readable station name) and extracts the station code (in capital letters) and the station name. for instance, given the input $\mathtt{stations}$ = $\{\mathtt{st}\mathtt{1}:"\mathtt{MANay}\mathtt{781};\mathtt{Manchester}",\mathtt{st}\mathtt{2}:"\mathtt{gNfbx}\mathtt{420};\mathtt{Greenfield}"\}$, the function returns the object $\{\mathtt{st}\mathtt{1}:"\mathtt{MAN}:\mathtt{Manchester}",\mathtt{st}\mathtt{2}:"\mathtt{GNF}:\mathtt{Greenfield}"\}$.

Thus, given an object containing strings following the station information pattern previously described, the function fixStations returns another object containing strings following the pattern of three capital letters concatenated with a colon concatenated with a string. The goal of our analyzer is to exactly preserve this information on the variable $\mathtt{result}$. Let us consider a statically unknown value of $\mathtt{stations}$, namely where $\mathtt{stations}=\{\mathtt{st}\mathtt{1}:{\sigma }_1,\cdots ,\mathtt{stn}:{\sigma }_n\}$, $n\in \mathbb{N}$ and ${\sigma }_i$ follows the station information pattern, for each $i\in [1,n]$. While other static analyzers, such as TAJS, which has a finite height string abstract domain, lose any information about the returned string, $\mu \mathrm{F}\mathrm{ASA}$ is able to infer, for the variable $\mathtt{result}$, the object $\{\mathtt{st}\mathtt{1}:{\mathtt{p}}_1,\cdots ,{\mathtt{st}}_n:{\mathtt{p}}_n\}$, where each ${\mathtt{p}}_i$ is a string abstract value, namely a finite state automaton, following the desired pattern

$${\sigma }_1·{\sigma }_2·{\sigma }_3·:·\sigma \mathrm{where}{\sigma }_i\mathrm{is}\mathrm{capital},i\in [1,3],\sigma \in {\Sigma }^{*}.$$

We are therefore able to preserve the string pattern that the function returns. As we have already highlighted, the result is obtained without implementing ad-hoc improvements regarding loop computations and we believe that even more precise results can be obtained integrating such techniques. We believe the integration of these analyses will drastically decrease false positives of the proposed string analysis (will address this topic in future works section).

7. Discussion and Related Work

In this paper we have proposed an abstract semantics for a toy imperative language $\mu \mathsf{JS}$, augmented with string manipulation operations, expressive enough to handle dynamic typing and implicit type conversion. In our abstract semantics we have combined the DFA domain with abstract domains for the other primitive types, necessary to deal with static analysis of programs with dynamic typing. The proposed formal framework allows us to formally prove soundness and to study the precision of the abstract semantics of each string operation: depending on the property of interest, one can tune the degree of precision, namely the completeness of any string operation.

7.1. Analysis vs. Verification

Even if several solutions, also involving finite state machines, have been proposed for string solving and verification [21,39,40], it should be noted that our approach is placed instead in the context of string static analysis. Over the years, there has always been the intuition that program analysis was harder than verification: given a program, the aim of the former is to derive invariants for each program point, the one of the latter is instead to check whether a certain property holds for the given input program. Recently, this concept has been formalized from a computability point of view [41], confirming this belief. Therefore, our approach, placed in the context of static analysis of string manipulation programs, has goals that are hardly comparable with the solutions proposed in the context of verification, such as those cited above.

7.2. Main Related Works

The issue of analyzing strings is a widely studied problem and it has been tackled in literature from different points of view. Before discussing the most related works, we can observe what makes our approach original w.r.t. existing literature: (1) We provide a modular parametric abstract domain on the abstractions of the different primitive types, this allows us to obtain both a tunable semantics precision and to handle dynamic typing for operations having both integer and string parameters, such as substring; (2) our focus is on the characterization of a formal abstract interpretation-based framework where it is possible to prove soundness and to analyze the completeness of string operations, in order to understand where it is possible to tune precision versus efficiency. The main feature we have in common with existing works is the use of DFA (regular expressions) for abstracting strings. In [21], the authors propose symbolic string verifier for PHP based on finite state automata represented by a particular form of binary decision diagrams, the MBDD. Even if it could be interesting to understand whether this representation of DFAs may be used also for improving our algorithms, their work only considers operations exclusively involving strings (not also integers such as substring) and therefore it provides a solution for different string manipulations. In [20], the authors propose an abstract interpretation-based string analyzer approximating strings into a subset of regular languages, called regular strings and define the abstract semantics of four string operations of interest equipped with a widening. This is the most related work, but our approach is strictly more general, since we do not introduce any restriction to regular languages. In [19], the authors propose a scalable static analysis for jQuery that relies on a novel abstract domain of regular expressions. The abstract domain in [19] contains the finite state automata one but pursues a different task and does not provide semantics for string operations. Surely it may be interesting to integrate our library for string manipulation operators into SAFE. Finally, [42] proposes a lattice-based generalization of regular expression, formally illustrating a parametric abstract domain of regular expressions starting from a complete lattice of reference. However, this work does not tackle the problem of analyzing string manipulations, since it instantiates the parametric abstract domain in the network communication environment, analyzing the exchanged messages as regular expressions.

Finite state machines (transducer and automata) have also found a critical application in model checking both for enforcing string constraints and for modeling infinite transition systems [43]. For example, the authors of [44] define a sound decision procedure for a regular language-based logic for verification of string properties. The authors of [45] propose an automata abstraction in the context of regular model checking to tackle the well-known problem of state space explosion. Moreover other formal systems, similar to DFA, have been proposed in the context of string analysis [46,47,48]. As future work, it can be interesting to study the relation between standard DFA and the other existing formal models, such as logics or other forms of FA.

In the context of JavaScript several static analyzers have been proposed, pushed by the wide range of applications and security issues related to the language [3,4,5,37]. TAJS [3] is a static analyzer based on abstract interpretation for JavaScript. The authors focus on allocation site abstraction, plugging in the static analyzer the recency abstraction [49], decreasing the number of false positives when objects are accessed. Upon TAJS, a sound way to statically analyze a large range of non-trivial eval patterns has been defined in [50]. In [37], it is defined the Loop-Sensitive Analysis (LSA) that distinguishes loop iterations using loop strings in the same way call strings distinguish function calls from different call sites in k-CFA [51]. The authors have implemented LSA into SAFE [5], a JavaScript web applications static analyzer. As future work, it may be intriguing to combine LSA with our abstract semantics for decreasing the occurrences of false positives introduced by the widening operator during fix-point computations.

7.3. Future Ideas

In this paper we have proposed static string program analysis for a set of relevant JavaScript string manipulation operations, whose semantics is inspired by the official ECMAScript specification [10]. The first goal is to involve our abstract semantics into a static analyzer for JavaScript that uses finite state automata to approximate strings. in order to decrease the number of false positives in our string approximation in presence of loops, several techniques can be involved, such as loop unrolling and LSA [37]. The domain described in this paper has been equipped only with a widening, to enforce termination in fix-point computations, which may lead to a big loss in precision. A narrowing will be studied and integrated in our static analyzer in order to retrieve some of the precision lost when the widening is applied.

We conclude by observing, as already highlighted in [7], the important application of finite state automata for string-to-code primitives analysis. Consider, for instance, in JavaScript programs, the eval function, transforming strings into code. Our semantics is sound and precise enough to answer some non-trivial properties of interest. Indeed, in [7], the finite state automata domain and the corresponding abstract semantics for strings turned out to be the basis for a sound and precise enough analysis of eval.