Robust Satisfaction of Metric Interval Temporal Logic Objectives in Adversarial Environments

Abstract

This paper studies the synthesis of controllers for cyber-physical systems (CPSs) that are required to carry out complex time-sensitive tasks in the presence of an adversary. The time-sensitive task is specified as a formula in the metric interval temporal logic (MITL). CPSs that operate in adversarial environments have typically been abstracted as stochastic games (SGs); however, because traditional SG models do not incorporate a notion of time, they cannot be used in a setting where the objective is time-sensitive. To address this, we introduce durational stochastic games (DSGs). DSGs generalize SGs to incorporate a notion of time and model the adversary’s abilities to tamper with the control input (actuator attack) and manipulate the timing information that is perceived by the CPS (timing attack). We define notions of spatial, temporal, and spatio-temporal robustness to quantify the amounts by which system trajectories under the synthesized policy can be perturbed in space and time without affecting satisfaction of the MITL objective. In the case of an actuator attack, we design computational procedures to synthesize controllers that will satisfy the MITL task along with a guarantee of its robustness. In the presence of a timing attack, we relax the robustness constraint to develop a value iteration-based procedure to compute the CPS policy as a finite-state controller to maximize the probability of satisfying the MITL task. A numerical evaluation of our approach is presented on a signalized traffic network to illustrate our results.

1. Introduction

Cyber-physical systems (CPSs) are playing increasingly important roles in multiple applications, including autonomous vehicles, robotics, and advanced manufacturing [1]. In many of these applications, the CPS is expected to satisfy complex, time-critical objectives in dynamic environments with autonomy. An example is a scenario where a drone has to periodically surveil a target region in its environment. One way to specify requirements on the CPS behavior is through a temporal logic framework [2] such as metric interval temporal logic (MITL) or signal temporal logic (STL). The verification of satisfaction of the temporal logic objective can then be achieved by applying principles from model checking [2,3] to a finite transition system that abstracts the CPS [4,5,6,7]. Solution techniques to verify such an objective usually return a ‘yes/no’ output, which indicates if the behavior of the CPS will satisfy the desired task and if it is possible to synthesize a control policy to satisfy this objective.

However, such binary-valued verification results may not be adequate when an adversary can inject inputs that affect the behavior of the CPS. Small perturbations can result in significantly large changes in the output of a CPS and can lead to violations of the desired task. The authors of [8,9] defined a notion of robustness degree to quantify the extent to which a CPS could tolerate deviations from its nominal behavior without resulting in violation of the desired specification.

For time-critical CPSs, an adversary could launch attacks on clocks of the system (by timing attack) and the inputs to the system (by actuator attack). In the latter case, stochastic games (SGs) have been used to model the interaction between the CPS and the adversary [10]. However, SGs do not include information about the time taken for a transition between two states. To bridge this gap, we introduce durational stochastic games (DSGs). In addition to transition probabilities between states under given actions of the CPS and adversary, a DSG encodes the time taken for the transition as a probability mass function. Although DSGs present a modeling formalism for time-critical objectives, they introduce an additional attack surface that can be exploited by an adversary.

In this paper, we synthesize controllers to satisfy an MITL specification that can be represented by a deterministic timed Büchi automaton with a desired robustness guarantee. The robustness guarantee quantifies how sensitive the synthesized policy (that satisfies the MITL task) will be to disturbances and adversarial inputs. The adversary is assumed to have the following abilities: it can tamper with the input to the defender through an actuator attack [11], and it can affect the time index observed by the CPS by effecting a timing attack [12]. An actuator attack could steer the DSG away from a target set of states, while a timing attack will prevent it from satisfying the objective within the specified time interval.

To address perturbations originating from different attack surfaces (timing information and system inputs), we develop three notions of robustness, namely spatial, temporal, and spatio-temporal robustness. Spatial robustness is defined over discrete timed words and quantifies the maximum perturbation that can be tolerated by timed words so that the desired tasks can still be satisfied in the absence of timing attacks. The temporal robustness characterizes the maximum timing perturbation that can be tolerated by a CPS such that the given MITL objective will not be violated. We introduce a notion of spatio-temporal robustness that unifies the concepts of spatial and temporal robustness. Using these three notions of robustness, we develop algorithms to estimate them and compute controllers for CPSs to guarantee that the given MITL objective can be satisfied with the desired robustness guarantee. This paper makes the following contributions:

• We introduce durational stochastic games (DSGs) to model the interaction between the CPS that has to satisfy a time-critical objective and an adversary who can initiate actuator and timing attacks.
• We define notions of spatial, temporal, and spatio-temporal robustness, which quantify the robustness of system trajectories to spatial, temporal, and spatio-temporal perturbations, respectively, and present computational procedures to estimate them. We design an algorithm to compute a policy for the CPS (defender) with a robustness guarantee when the adversary is limited to effecting only actuator attacks.
• We demonstrate that the defender cannot correctly estimate the spatio-temporal robustness when the adversary can initiate both actuator and timing attacks. We relax the robustness constraints in such cases and present a value iteration-based procedure to compute the defender’s policy, represented as a finite-state controller, to maximize the probability of satisfying the MITL objective.
• We evaluate our approach on a signalized traffic network. We compare our approach with two baselines and show that it outperforms both baselines.

The remainder of this paper is organized as follows. Section 2 discusses related work. Section 3 provides background on MITL and deterministic timed Büchi automata. We define the DSG and notions of robustness in Section 4 and formally state the problem of interest. Section 5 and Section 6 present our results when the adversary is limited to initiating only actuator attacks and when it can effect both actuator and timing attacks, respectively. The experimental results are presented in Section 7. Section 8 concludes the paper.

2. Related Work

For a single agent, semi-Markov decision processes (SMDPs) [13] can be used to model Markovian dynamics, where the time taken for transitions between states is a random variable. SMDPs have been used in production scheduling [14] and the optimization of queues [15].

Stochastic games (SGs) generalize MDPs when there is more than one agent taking an action [16]. SGs have been widely adopted to model strategic interactions between CPSs and adversaries. For example, a zero-sum SG was formulated in  [17] to allocate resources to protect power systems against malicious attacks. Two SGs were developed in [18] to detect intrusions to achieve secret and reliable communications. The satisfaction of complex objectives modeled by linear temporal logic (LTL) formulae for zero-sum two-player SGs was presented in [10], where the authors synthesized controllers to maximize the probability of satisfying the LTL formula. However, this approach will not apply when the system has to satisfy a time-critical specification and the adversary can launch a timing attack.

Timed automata (TA) [3] finitely attach many clock constraints to each state. A transition between any two states will be influenced by the satisfaction of the clock constraints in the respective states. There has been significant work performed in the formulation of timed temporal logic frameworks, a detailed survey of which is presented in [19]. Metric interval temporal logic (MITL) [20] is one such fragment that allows for the specification of formulae that explicitly depend on time. Moreover, an MITL formula can be represented as a TA [20,21] that will have a feasible path in it if and only if the MITL formula is true.

Control synthesis under metric temporal logic constraints was studied for motion planning applications in [6,7,22,23]. The authors of [22] considered a vehicle routing problem to meet MTL specifications by solving a mixed integer linear program. Timed automaton-based control synthesis under a subclass of MITL specifications was studied in [6,7]. Cooperative task planning of a multi-agent system under MITL specifications was studied in [24]. In comparison, we consider the actions of an adversarial player, whose objective is opposite to that of the defender. This leads to a modeling of the interaction between the adversary and defender as an SG. Moreover, the previous works have limited their focus to a certain fragment of the MITL, whereas this paper offers a generalized treatment to arbitrary MITL formulae.

Finite-state controllers (FSCs) were used to simplify the policy iteration procedure for POMDPs in [25]. The satisfaction of an LTL formula of a POMDP was presented in [26]. This was extended to the case with an adversary who also only had partial observation of the environment and whose goal was to prevent the defender from satisfying the LTL formula in [27,28]. These treatments, however, did not account for the presence of timing constraints on the satisfaction of a temporal logic formula.

Control synthesis for control systems under disturbances with robustness guarantees has been extensively studied [29,30,31,32]. Such robustness guarantees can be categorized as a notion of spatial robustness. Robust satisfaction of temporal logic tasks have been studied for signal monitoring and property verification. A notion of robustness degree for continuous signals was defined in [8] by computing a distance between the given timed behavior and the set of behaviors that satisfy a property expressed in temporal logic. Our notion of spatial robustness is defined over discrete timed words using the Levenshtein distance, which distinguishes our approach from [8]. The robustness degree between two LTL formulae was introduced in [33]. The authors of [34] adopted a different approach and used the weighted edit distance to quantify a measure of robustness. The notion of temporal robustness was also investigated in [9]. There are three differences between our definition of temporal robustness and that found in [9]. First, the temporal robustness in [9] is defined for a specific trace. In our framework, as the DSG is not deterministic, there could be multiple traces that satisfy the MITL objective under the defender and adversary policies. Therefore, we define temporal robustness with respect to the policies of the defender and adversary and the MITL specification. Second, the temporal robustness of a real-valued signal is computed as the maximum amount of time units by which we can shift on the rising/falling edge of a ‘characteristic function’ in [9]. In comparison, we work with discrete timed words. Finally, our work considers the presence of an adversary, while [9] assumes a single agent. Robust control under signal temporal logic (STL) formulae has been studied based on notions of space robustness [35,36] and temporal robustness [37,38]. These works did not consider the presence of an adversary.

A preliminary version of this paper [39] synthesized policies to satisfy MITL objectives under actuator and timing attacks without robustness guarantees. In this paper, we define three robustness degrees and develop algorithms to compute these quantities. We show that any defender policy that provides a positive robustness degree is an almost-sure satisfaction policy, which is stronger than the quantitative satisfaction policies synthesized in [39].

3. MITL and Timed Automata

We introduce the syntax and semantics of metric interval temporal logic and its equivalent representation as a timed automaton. We use $\mathbb{R},{\mathbb{R}}_{\ge 0},\mathbb{N},$ and ${\mathbb{Q}}_{\ge 0}$ to denote the sets of real numbers, non-negative reals, positive integers, and non-negative rationals, respectively. Vectors are represented by bold symbols. The comparison between vectors ${\mathbf{v}}_1$ and ${\mathbf{v}}_2$ is element-wise, and $\mathbf{v}\left(i\right)$ denotes the i-th element of $\mathbf{v}$. Given a set of atomic propositions $\mathrm{\Pi }$, a metric interval temporal logic (MITL) formula is inductively defined as

$$\phi :=\top \left|\pi \right|\neg \phi |{\phi }_1\wedge {\phi }_2|{\phi }_1{\mathcal{U}}_I{\phi }_2,$$

where $\pi \in \mathrm{\Pi }$ is an atomic proposition and I is a non-singular time interval with integer end-points. MITL admits derived operators such as ‘constrained eventually’ (${\diamond }_I\phi :=\top {\mathcal{U}}_I\phi$) and ‘constrained always’ (${\square }_I\phi :=\neg ({\diamond }_I\neg \phi )$). Throughout this paper, we assume that I is bounded. We further rewrite the given MITL formula in the negation normal form so that negations only appear in front of atomic propositions. We augment the atomic proposition set $\mathrm{\Pi }$ so that any atomic proposition $\pi$ and its negation $\neg \pi$ are both included in $\mathrm{\Pi }$.

We focus on the pointwise MITL semantics [40]. A timed word is an infinite sequence $\rho =(a_0,t_0)(a_1,t_1)\dots$, where $a_i\in 2^{\mathrm{\Pi }}$; $t_i\in {\mathbb{R}}_{\ge 0}$ is the time index with $t_{i+1}>t_i\forall i\ge 0$. We denote $a_0,a_1,\cdots$ as a word over $\mathrm{\Pi }$ and $t_0,t_1,\cdots$ as a time sequence. With $\rho \left(i\right)=(a_i,t_i)$, we define: $\mathrm{UNTIME}\left(\rho \right):=a_0,a_1,\cdots ,$ and $val\left(\rho \right):=t_0,t_1,\cdots .$

We interpret MITL formulae over timed words as follows.

Definition 1 

(MITL Semantics). Given a timed word ρ and an MITL formula φ, the satisfaction of φ at position j, denoted as $(\rho ,j)\vDash \phi$, is inductively defined as follows:

1.

$(\rho ,j)\vDash \top$ if and only if (iff) $(\rho ,j)$ is true;

2.

$(\rho ,j)\vDash \pi$ iff $\pi \in a_j$;

3.

$(\rho ,j)\vDash \neg \phi$ iff $(\rho ,j)$ does not satisfy φ;

4.

$(\rho ,j)\vDash {\phi }_1\wedge {\phi }_2$ iff $(\rho ,j)\vDash {\phi }_1$ and $(\rho ,j)\vDash {\phi }_2$;

5.

$(\rho ,j)\vDash {\phi }_1{\mathcal{U}}_I{\phi }_2$ iff $\exists k\ge j$ such that $(\rho ,k)\vDash {\phi }_2$, $t_k-t_j\in I$ and $(\rho ,m)\vDash {\phi }_1$ holds for all $j\le m<k$.

We denote $\rho \vDash \phi$ if $(\rho ,0)\vDash \phi$. The satisfaction of an MITL formula can be equivalently associated with accepting words of a timed Büchi automaton (TBA) [20]. Let $C=\{c_1,\cdots ,c_M\}$ be a finite set of clocks. Define a set of clock constraints $\mathrm{\Phi }\left(C\right)$ over C as $\xi =\top |\perp |c\bowtie \delta |{\xi }_1\wedge {\xi }_2,$ where $\bowtie \in \{\le ,\ge ,<,>\}$, $c,c^{\prime }\in C$ are clocks, and $\delta \in \mathbb{Q}$ is a non-negative rational number. In this paper, we focus on a subclass of MITL formulae that can be equivalently represented as deterministic timed Büchi automaton, which are defined as follows.

Definition 2 

(Deterministic Timed Büchi Automaton [3]). A deterministic timed Büchi automaton (DTBA) is a tuple $\mathcal{A}=(Q,2^{\mathrm{\Pi }},q_0,C,\mathrm{\Phi }\left(C\right),E,F)$, where Q is a finite set of states, $2^{\mathrm{\Pi }}$ is an alphabet over atomic propositions in $\mathrm{\Pi }$ is the initial state, $E\subseteq Q\times Q\times 2^{\mathrm{\Pi }}\times 2^C\times \mathrm{\Phi }\left(C\right)$ is the set of transitions, and $F\subseteq Q$ is the set of accepting states. A transition $<q,q^{\prime },a,C^{\prime },\varphi >\in E$ if $\mathcal{A}$ enables the transition from q to $q^{\prime }$ when a subset of atomic propositions $a\in 2^{\mathrm{\Pi }}$ and clock constraints $\varphi \in \mathrm{\Phi }\left(C\right)$ evaluate to true. The clocks in $C^{\prime }\subseteq C$ are reset to zero after the transition.

We present the DTBA representing MITL formula $\phi ={\diamond }_{[2,3]}\pi$ as an example in Figure 1. In this figure, the states Q and transitions E are represented by circles and arrows, respectively. Here, the initial state is $q_0$. The set of accepting states is $F=\left\{q_2\right\}$. Consider the transition from initial state $q_0$ to state $q_2$. The transition $<q_0,q_2,\pi ,c,\varphi >$ can take place if atomic proposition $\pi$ is evaluated to be true and clock constraint $\varphi \left(c\right)$ defined on clock c satisfies $2\le c\le 3$. Furthermore, the clock c is reset to zero after the transition.

Figure 1. The deterministic timed Büchi automaton (DTBA) representing a metric interval temporal logic formula $\phi ={\diamond }_{[2,3]}\pi$. The states and transitions of the DTBA are represented by circles and arrows, respectively. The initial state of this DTBA is $q_0$ and the accepting state is $q_2$. The formula $\phi$ can be satisfied if the DTBA reaches state $q_2$.

Given the set of clocks C, $\mathbf{v}:C\mapsto V$ is the valuation of C, where $V\subseteq {\mathbb{Q}}^{\left|C\right|}$. Let $\mathbf{v}\left(c\right)$ be the valuation of clock $c\in C$. We say $\mathbf{v}=\mathbf{0}$ if $\mathbf{v}\left(c\right)=0$ for all $c\in C$. Given $\delta \in {\mathbb{R}}_{\ge 0}$, we let $\mathbf{v}+\delta :={[\mathbf{v}\left(1\right)+\delta ,\cdots ,\mathbf{v}(\left|C\right|)+\delta ]}^T$. A configuration of $\mathcal{A}$ is a pair $(q,\mathbf{v})$, where $q\in Q$ is a state of $\mathcal{A}$. Suppose a transition $<q,q^{\prime },a,C^{\prime },\xi >$ is taken after $\delta$ time units. Then, the DTBA is transited from configuration $(q,\mathbf{v})$ to $(q^{\prime },\mathbf{v}+\delta )$ such that $\mathbf{v}+\delta \vDash \xi$, ${\mathbf{v}}^{\prime }\left(c\right)=\mathbf{v}\left(c\right)+\delta$ for all $c\notin C^{\prime }$ and ${\mathbf{v}}^{\prime }\left(c\right)=0$ for all $c\in C^{\prime }$. We denote the transition between these configurations as $(q,\mathbf{v})\stackrel{a,\delta }{\to }(q^{\prime },\mathbf{v}+\delta )$. A run of $\mathcal{A}$ is a sequence of such transitions between configurations $\beta :=(q_0,{\mathbf{v}}_0)\stackrel{a_0,{\delta }_0}{\to }(q_1,{\mathbf{v}}_1)\cdots$. A feasible run $\beta$ on $\mathcal{A}$ is accepting iff it intersects with F infinitely often.

4. Problem Setup and Formulation

In this section, we propose durational stochastic games that generalize stochastic games and present the defender and adversary models in terms of the information available to them. We then define three robustness degrees and state the problem of interest.

4.1. Environment, Defender, and Adversary Models

We introduce durational stochastic games as a generalization of stochastic games [10]. Different from SGs, DSGs model (i) the timing information for transitions between states and (ii) an attack surface resulting from the timing information.

An SG is defined as follows:

Definition 3 

(Stochastic game). A (labeled) stochastic game $\mathcal{SG}$ is a tuple $\mathcal{SG}=(S,U_C,U_A,Pr,\mathrm{\Pi },\mathcal{L})$, where S is a finite set of states, $U_C$ is a finite set of actions of the defender, $U_A$ is a finite set of actions of an adversary, and $Pr:S\times U_C\times U_A\times S\to [0,1]$ is a transition function where $Pr(s,u_C,u_A,s^{\prime })$ is the probability of a transition from state s to state $s^{\prime }$ when the defender takes action $u_C$ and the adversary takes action $u_A$. $\mathrm{\Pi }$ is a set of atomic propositions. $L:S\to 2^{\mathrm{\Pi }}$ is a labeling function mapping each state to a subset of propositions in $\mathrm{\Pi }$.

The SG in Definition 3 cannot be used to verify satisfaction of an MITL objective as it does not include a notion of time. We define durational stochastic games to bridge this gap. DSGs incorporate a notion of time taken for a transition between states and also models the ability of an adversary to modify this timing information.

Definition 4 

(Durational stochastic game). A (labeled) durational stochastic game (DSG) is a tuple $\mathcal{G}=(S_{\mathcal{G}},s_{\mathcal{G},0},U_C,U_A,In{f}_{\mathcal{G},C},In{f}_{\mathcal{G},A},P{r}_{\mathcal{G}},T_{\mathcal{G}},\mathrm{\Pi },L,Cl)$. $S_{\mathcal{G}}$ is a finite set of states, $s_{\mathcal{G},0}$ is the initial state, and $U_C$, $U_A$ are finite sets of actions. $In{f}_{\mathcal{G},C}:S_{\mathcal{G}}\times {\mathbb{R}}_{\ge 0}\mapsto {(S_{\mathcal{G}}\times {\mathbb{R}}_{\ge 0})}^{*}$ and $In{f}_{\mathcal{G},A}:S_{\mathcal{G}}\times {\mathbb{R}}_{\ge 0}\mapsto {(S_{\mathcal{G}}\times {\mathbb{R}}_{\ge 0}\times U_C)}^{*}$ are information sets of the defender and adversary, respectively, where ${(·)}^{*}$ is the Kleene operator. $P{r}_{\mathcal{G}}:S_{\mathcal{G}}\times U_C\times U_A\times S_{\mathcal{G}}\mapsto [0,1]$ encodes $P{r}_{\mathcal{G}}\left(s_{\mathcal{G}}^{\prime }\right|s_{\mathcal{G}},u_C,u_A)$, the transition probability from state $s_{\mathcal{G}}$ to $s_{\mathcal{G}}^{\prime }$ when the controller and adversary take actions $u_C$ and $u_A$. $T_{\mathcal{G}}:S_{\mathcal{G}}\times U_C\times U_A\times S_{\mathcal{G}}\times \Delta \mapsto [0,1]$ is a probability mass function. $T_{\mathcal{G}}\left(\delta \right|s_{\mathcal{G}},u_C,u_A,s_{\mathcal{G}}^{\prime })$ denotes the probability that a transition from $s_{\mathcal{G}}$ to $s_{\mathcal{G}}^{\prime }$ under actions $u_C$ and $u_A$ takes $\delta \in \Delta$ time units, where $\Delta$ is a finite set of time units that each transition of DSG can possibly take to complete. $\mathrm{\Pi }$ is a set of atomic propositions. $L:S_{\mathcal{G}}\mapsto 2^{\mathrm{\Pi }}$ is a labeling function that maps each state to atomic propositions in $\mathrm{\Pi }$ that are true in that state, and $Cl$ is a finite set of clocks.

The set of admissible actions that can be taken by the defender (adversary) in a state $s\in S_{\mathcal{G}}$ is denoted as $U_C\left(s\right)$ ($U_A\left(s\right)$). A path on $\mathcal{G}$ is a sequence of states $w:=s_0\underset{{\delta }_0}{\overset{u_{C,0},u_{A,0}}{\to }}{s}_1\dots \underset{{\delta }_i}{\overset{u_{C,i},u_{A,i}}{\to }}{s}_{i+1}\dots$ such that $s_0=s_{\mathcal{G},0}$, $P{r}_{\mathcal{G}}\left(s_{i+1}\right|s_i,u_{C,i},u_{A,i})>0$ and $T_{\mathcal{G}}\left(\delta \right|s_i,u_{C,i},u_{A,i},s_{i+1})>0$ for some $u_{C,i}\in U_C\left(s_i\right)$, $u_{A,i}\in U_A\left(s_i\right)$, and $\delta \in \Delta$ for all $i\ge 0$. Consider the DSG with $S_{\mathcal{G}}=\{s_{\mathcal{G},0},s_1,s_2,s_3\}$, $U_C=\left\{u_C\right\}$, and $U_A=\left\{u_A\right\}$ presented in Figure 2 as an example. We have that $w=s_0\underset{1}{\overset{u_C,u_A}{\to }}{s}_2\underset{1}{\overset{u_C,u_A}{\to }}{s}_3$ is a finite path. We denote the set of finite (infinite) paths by ${(S_{\mathcal{G}}\times {\mathbb{R}}_{\ge 0})}^{*}$ (${(S_{\mathcal{G}}\times {\mathbb{R}}_{\ge 0})}^{\omega }$). Given a path w, $L\left(w\right):=L\left(s_{\mathcal{G},0}\right),L\left(s_1\right),\dots ,$ is the sequence of atomic propositions corresponding to states in w. The sequence of state-time tuples in w is obtained as $(s_0,k_0),(s_1,k_1),\dots$, where $k_i+{\delta }_i=k_{i+1}$, $i=0,1,\dots$.

Figure 2. This figure presents an example of a DSG consisting of 4 states, denoted as $S_{\mathcal{G}}=\{s_{\mathcal{G},0},s_1,s_2,s_3\}$. The transition probabilities $P{r}_{\mathcal{G}}$ and probability mass function $T_{\mathcal{G}}$ for some transitions are given in the figure. The labeling function L for state $s_1$ is given as $L\left(s_1\right)=\{{\pi }_1,{\pi }_2\}$.

For the defender, a deterministic policy $\mu :{(S_{\mathcal{G}}\times {\mathbb{R}}_{\ge 0})}^{*}\mapsto U_C$ is a map from the set of finite paths to its actions. A randomized policy $\mu :{(S_{\mathcal{G}}\times {\mathbb{R}}_{\ge 0})}^{*}\mapsto \mathcal{D}\left(U_C\right)$ maps the set of finite paths to a probability distribution over its actions. A policy is memoryless if it only depends on the the most recent state.

Consider a path w in $\mathcal{G}$. At a state s, the information set of the defender is $In{f}_{\mathcal{G},C}^w(s,k_C):=\{(s_{\mathcal{G},0},0),\dots ,(s,k_C)\}$, where $k_C$ is the time perceived by the defender when it reaches s along w. For example, given the finite path $w=s_0\underset{1}{\overset{u_C,u_A}{\to }}{s}_2\underset{1}{\overset{u_C,u_A}{\to }}{s}_3$ for the DSG presented in Figure 2, information set $In{f}_{\mathcal{G},C}^w(s_2,k_C)=\{(s_{\mathcal{G},0},0),(s_1,1)\}$. For the adversary, $In{f}_{\mathcal{G},A}(s,k_A):=\{(s_{\mathcal{G},0},0),\dots ,(s,k_A)\}\cup \left\{\mu \right\}$, where $k_A$ is the time observed by the adversary at s, and $\mu$ is the defender’s policy. Information sets of the defender and adversary are given by $In{f}_{\mathcal{G},C}(s,k_C):={\displaystyle \bigcup _w}In{f}_{\mathcal{G},C}^w(s,k_C)$ and $In{f}_{\mathcal{G},A}(s,k_A):={\displaystyle \bigcup _w}In{f}_{\mathcal{G},A}^w(s,k_A)$.

We assume that the initial time is 0, and this is known to both agents. The adversary having knowledge of the policy $\mu$ committed to by the defender introduces an asymmetry between the information sets of the two agents. We note that although the adversary is aware of the defender’s randomized policy, it does not know the exact action $u_C$. This is also known as the Stackelberg setting in game theory. We assume a concurrent Stackelberg setting in that both the defender and adversary take their actions at each state simultaneously.

The solution concept to a Stackelberg game is a Stackelberg equilibrium, which is defined as follows.

Definition 5 

(Stackelberg equilibrium [16]). A tuple $(\mu ,(\tau ,\gamma \left)\right)$ is a Stackelberg equilibrium if $\mu =arg{max}_{{\mu }^{\prime }}{Q}_C({\mu }^{\prime },BR\left({\mu }^{\prime }\right))$, where $Q_C(\mu ,(\tau ,\gamma ))$ and $Q_A(\mu ,(\tau ,\gamma ))$ are the expected utilities of the defender and adversary under policies μ and $(\tau ,\gamma )$, respectively, and $BR\left({\mu }^{\prime }\right)=\{(\tau ,\gamma ):(\tau ,\gamma )=\underset{({\tau }^{\prime },{\gamma }^{\prime })}{argmax}{Q}_A({\mu }^{\prime },({\tau }^{\prime },{\gamma }^{\prime }))\}$.

If $BR\left({\mu }^{\prime }\right)$ contains multiple adversary policies, the defender will arbitrarily pick one. During an actuator attack, the adversary can manipulate state transitions in $\mathcal{G}$ as its actions $u_A$ will influence the transition probabilities $P{r}_{\mathcal{G}}$. The adversary could also exploit the attack surface that will be introduced as a consequence of including timing information. We term this a timing attack. In this paper, we consider the worst-case scenario and assume that the adversary knows the correct time index at each time k. However, it can manipulate the timing information perceived by the defender through $T_{\mathcal{G}}$. Thus, the time index $k_C$ perceived by the defender need not be the same as that known to the adversary, $k_A$.

The adversary launches actuator and timing attacks through attack policies. An actuator attack policy $\tau :{(S_{\mathcal{G}}\times {\mathbb{R}}_{\ge 0})}^{*}\mapsto U_A$ specifies the action taken by the adversary given the set of finite paths. A timing attack policy$\gamma :V\times V\mapsto [0,1]$ takes as its input the correct clock valuation and yields a probability distribution over clock valuations. This models the ability of the adversary to manipulate clock valuations. For an intelligent adversary, it should launch the timing attack such that the resulting sequence of clock valuations is monotone when the clocks are not reset. The reason is such non-monotone clock valuations informs the defender of the presence of a timing attack; thus, the defender can simply ignore the perceived clock valuations.

4.2. Definitions of Robustness Degree

In this subsection, we define three robustness degrees defined with respect to policies on the DSG $\mathcal{G}$.

4.2.1. Spatial Robustness

The spatial robustness, denoted as ${\chi }_s^{\phi }(\mu ,\tau ,\gamma )$, represents the minimum distance between any accepting (resp. non-accepting) path on the DSG induced by policies $\mu$ and $(\tau ,\gamma )$ and the language of the MITL specification without regard to the timing information. We define the spatial robustness using the Levenshtein distance, which is used to measure the distance between strings [41].

Definition 6 

(Levenshtein distance [41]). The Levenshtein distance between sequences of symbols $w_1$ and $w_2$, denoted $d_L(w_1,w_2)$, is the minimum number of edit operations (insertions, substitutions, or deletions) that can be applied to $w_1$ so that $w_1$ can be converted to $w_2$.

Consider timed words $w_1=(q_0,0)(q_1,1)(q_2,2)\dots$ and $w_2=(q_0,0)(q_1^{\prime },1)(q_2,2)\dots$ that differ at position 1, where $q_1\ne q_1^{\prime }$. Then, $d_L(w_1,w_2)=1$, as $w_1$ can be converted to $w_2$ by substituting $q_1$ with $q_1^{\prime }$. Relying on the Levenshtein distance in Definition 6, we define the spatial robustness ${\chi }_s^{\phi }(\mu ,\tau ,\gamma )$ for policies $\mu$ and $(\tau ,\gamma )$ on a DSG $\mathcal{G}$ with respect to the MITL formula $\phi$ as:

$${\chi }_s^{\phi }(\mu ,\tau ,\gamma )=\left\{\begin{array}{c}\underset{w_1\in {\mathcal{B}}_{\mathcal{G}}^{\mu \tau \gamma },w_2\notin \mathcal{L}}{min}{d}_L(w_1,w_2),\mathrm{if}{\mathcal{B}}_{\mathcal{G}}^{\mu \tau \gamma }\subseteq \mathcal{L};\hfill \\ -\underset{w_1\in {\mathcal{B}}_{\mathcal{G}}^{\mu \tau \gamma },w_2\in \mathcal{L}}{min}{d}_L(w_1,w_2),\mathrm{othersise}.\hfill \end{array}\right.$$

(1)

In Equation (1), ${\mathcal{B}}_{\mathcal{G}}^{\mu \tau \gamma }$ is the set of paths enabled on $\mathcal{G}$ under policies $\mu$ and $(\tau ,\gamma )$, and $\mathcal{L}$ contains the set of paths on $\mathcal{G}$ that satisfy $\phi$. We note that as $d_L(·,·)\ge 0$, any path $w\in {\mathcal{B}}_{\mathcal{G}}^{\mu \tau \gamma }$ synthesized under policies $\mu$ and $\tau$ that satisfies $\phi$ will result in ${\chi }_s^{\phi }(\mu ,\tau ,\gamma )>0$. If, for some $w\in {\mathcal{B}}_{\mathcal{G}}^{\mu \tau \gamma }$, $w\notin \mathcal{L}$, then ${\chi }_s^{\phi }(\mu ,\tau ,\gamma )\le 0$.

4.2.2. Temporal Robustness

The temporal robustness ${\chi }_t^{\phi }(\mu ,\tau ,\gamma )$ captures the maximum time units by which any accepting path synthesized under policies $\mu$ and $(\tau ,\gamma )$ can be temporally perturbed so that the MITL formula $\phi$ is not violated.

Given an accepting run w and $k\in \mathbb{Q}$, we let $\mathrm{VAL}\left(w\right)+k:={\mathbf{v}}_0+k,{\mathbf{v}}_1+k,\dots$ We define the left temporal robustness ${\chi }_t^{\phi ,-}(\mu ,\tau ,\gamma )$ and right temporal robustness ${\chi }_t^{\phi ,+}(\mu ,\tau ,\gamma )$ as:

$$\begin{array}{cc}\hfill & {\chi }_t^{\phi ,-}(\mu ,\tau ,\gamma )=max\bigcap _{w\in {\mathcal{B}}_{\mathcal{G}}^{\mu \tau \gamma }}\left\{k\right|w^{\prime }\vDash \phi \forall w^{\prime }\mathrm{s}.\mathrm{t}.0\le \mathrm{VAL}\left(w\right)-\mathrm{VAL}\left(w^{\prime }\right)\le k\in \mathbb{Q}\},\hfill \end{array}$$

(2)

$$\begin{array}{cc}\hfill & {\chi }_t^{\phi ,+}(\mu ,\tau ,\gamma )=max\bigcap _{w\in {\mathcal{B}}_{\mathcal{G}}^{\mu \tau \gamma }}\left\{k\right|w^{\prime }\vDash \phi \forall w^{\prime }\mathrm{s}.\mathrm{t}.0\le \mathrm{VAL}\left(w^{\prime }\right)-\mathrm{VAL}\left(w\right)\le k\in \mathbb{Q}\}.\hfill \end{array}$$

(3)

The left (right) temporal robustness ${\chi }_t^{\phi ,-}(\mu ,\tau ,\gamma )$ (${\chi }_t^{\phi ,+}(\mu ,\tau ,\gamma )$) indicates that an accepting run w induced by $\mu$ and $(\tau ,\gamma )$ can be perturbed up to k time units to the left (right) without violating $\phi$. These definitions also ensure that any perturbation smaller than ${\chi }_t^{\phi ,-}(\mu ,\tau ,\gamma )$ or ${\chi }_t^{\phi ,+}(\mu ,\tau ,\gamma )$ will not violate $\phi$. The temporal robustness is then:

$${\chi }_t^{\phi }(\mu ,\tau ,\gamma )=\left\{\begin{array}{c}min\{{\chi }_t^{\phi ,-}(\mu ,\tau ,\gamma ),{\chi }_t^{\phi ,+}(\mu ,\tau ,\gamma )\},\mathrm{if}{\mathcal{B}}_{\mathcal{G}}^{\mu \tau \gamma }\subseteq \mathcal{L}\hfill \\ \Lambda ,\mathrm{otherwise}\hfill \end{array}\right.,$$

(4)

where $\Lambda$ is a symbol indicating that policies $\mu$ and $(\tau ,\gamma )$ can lead to non-accepting runs.

4.2.3. Spatio-Temporal Robustness

We define the spatio-temporal robustness ${\chi }^{\phi }(\mu ,\tau ,\gamma )$ to unify notions of spatial and temporal robustness as:

$${\chi }^{\phi }(\mu ,\tau ,\gamma )=I({\chi }_s^{\phi }(\mu ,\tau ,\gamma )\ge {ϵ}_s){\chi }_t^{\phi }(\mu ,\tau ,\gamma ),$$

(5)

where $I({\chi }_s^{\phi }(\mu ,\tau ,\gamma )\ge {ϵ}_s)$ is an indicator function that equals to 1 if ${\chi }_s^{\phi }(\mu ,\tau ,\gamma )\ge {ϵ}_s$ and $-1$ otherwise. In other words, the spatio-temporal robustness ${\chi }^{\phi }(\mu ,\tau ,\gamma )$ captures the maximum time units by which any accepting run can be perturbed without violating the MITL specification $\phi$, given a desired spatial robustness ${ϵ}_s$, under policies $\mu$ and $(\tau ,\gamma )$. Note that when the spatio-temporal robustness is $-\Lambda$, we have that policies $\mu$ and $(\tau ,\gamma )$ lead to non-accepting runs.

4.2.4. Robust MITL Semantics

Given the spatio-temporal robustness in Equation (5), we can use a real-valued function ${\zeta }^{\phi }(\rho ,j)$ to reason about the satisfaction of $\phi$ such that $(\rho ,j)\vDash \phi \equiv {\zeta }^{\phi }(\rho ,j)>0$.

Definition 7 

(Robust MITL Semantics). Let ρ be a timed word. We define a real-valued function ${\zeta }^{\phi }(\rho ,j)$ such that the satisfaction of an MITL formula φ at position j by a timed word ρ, written $(\rho ,j)\vDash \phi :={\zeta }^{\phi }(\rho ,j)>0$, can be recursively defined as:

1.

${\zeta }^{\phi }(\rho ,j)=f(\rho ,j)$;

2.

${\zeta }^{{\phi }_1\wedge {\phi }_2}(\rho ,j)=min\{{\zeta }^{{\phi }_1}(\rho ,j),{\zeta }^{{\phi }_2}(\rho ,j)\}$;

3.

${\zeta }^{{\phi }_1\vee {\phi }_2}(\rho ,j)=max\{{\zeta }^{{\phi }_1}(\rho ,j),{\zeta }^{{\phi }_2}(\rho ,j)\}$;

4.

${\zeta }^{{\phi }_1{\mathcal{U}}_{[a,b]}{\phi }_2}(\rho ,j)={max}_{t^{\prime }\in [j+a,j+b]}\{min\{{\zeta }^{{\phi }_2}(\rho ,t^{\prime }),{min}_{t^{″}\in [j,t^{\prime }]}{\zeta }^{{\phi }_1}(\rho ,t^{″})\}\}$.

where $f(\rho ,j)=I\left({min}_{w\notin \mathcal{L}}{d}_L(\rho ,w)\ge {ϵ}_s\right)\overline{k}$ and $\overline{k}=max\left\{k\right|({\rho }^{\prime },j)\vDash \phi \forall {\rho }^{\prime }s.t.0\le |\mathrm{VAL}\left(\rho \right)-\mathrm{VAL}\left({\rho }^{\prime }\right)|\le k\}$.

4.3. Problem Statement

Before formally stating the problem of interest, we prove a result which shows that a defender’s policy that provides positive spatio-temporal robustness satisfies the MITL objective $\phi$ with probability one.

Proposition 1. 

Given an MITL objective φ and policies μ and $(\tau ,\gamma )$, the spatio-temporal robustness ${\chi }^{\phi }(\mu ,\tau ,\gamma )>0$ implies almost-sure satisfaction of φ under the agent policies when there is no timing attack.

Proof. 

The proof of this result is deferred to Appendix B.    □

Given Proposition 1, we formally state our problem:

Problem 1 

(Robust policy synthesis for defender). Given a DSG $\mathcal{G}$ and an MITL formula φ, compute an almost-sure defender policy. That is, compute μ such that ${\chi }^{\phi }(\mu ,\tau ,\gamma )\ge {ϵ}_t$, where $(\tau ,\gamma )\in BR\left(\mu \right)$.

5. Solution: Only Actuator Attack

We present a solution to robust policy synthesis for the defender as described in Problem  1, assuming that the adversary only launches an actuator attack. We construct a product DSG $\mathcal{P}$ from DSG $\mathcal{G}$ and DTBA $\mathcal{A}$. We present procedures to evaluate the spatio-temporal robustness and compute an optimal policy for the defender on $\mathcal{P}$.

5.1. Product DSG

In the following, we provide the definition of product DSG.

Definition 8 

(Product durational stochastic game). A PDSG $\mathcal{P}$ constructed from a DSG $\mathcal{G}$, DTBA $\mathcal{A}$, and clock valuation set V is a tuple $\mathcal{P}=(S,s_0,U_C,U_A,In{f}_C,In{f}_A,Pr,Acc)$. $S=S_{\mathcal{G}}\times Q\times V$ is a finite set of states, $s_0=(s_{\mathcal{G},0},q_0,\mathbf{0})$ is the initial state, and $U_C$, $U_A$ are finite sets of actions. $In{f}_C$, $In{f}_A$ are information sets of the defender and adversary. $Pr:S\times U_C\times U_A\mapsto \mathcal{S}$ encodes $Pr\left((s^{\prime },q^{\prime },{\mathbf{v}}^{\prime })|(s,q,\mathbf{v}),u_C,u_A\right)$, the probability of a transition from state $(s,q,\mathbf{v})$ to $(s^{\prime },q^{\prime },{\mathbf{v}}^{\prime })$ when the defender and adversary take actions $u_C$ and $u_A$. The probability

$$Pr\left((s^{\prime },q^{\prime },{\mathbf{v}}^{\prime })|(s,q,\mathbf{v}),u_C,u_A\right):=T_{\mathcal{G}}\left(\delta \right|s,u_C,u_A,s^{\prime })P{r}_{\mathcal{G}}\left(s^{\prime }\right|s,u_C,u_A)$$

(6)

if and only if $(q,\mathbf{v})\stackrel{L\left(s^{\prime }\right),\delta }{\to }(q^{\prime },{\mathbf{v}}^{\prime })$, zero otherwise. $Acc=S_{\mathcal{G}}\times F\times V$ is a finite set of accepting states.

The following result shows that the transition probability of $\mathcal{P}$ is well defined.

Proposition 2. 

The function $Pr(·)$ satisfies $Pr\left((s^{\prime },q^{\prime },{\mathbf{v}}^{\prime })|(s,q,\mathbf{v}),u_C,u_A\right)\in [0,1]$ and

$$\sum _{(s^{\prime },q^{\prime },{\mathbf{v}}^{\prime })}Pr\left((s^{\prime },q^{\prime },{\mathbf{v}}^{\prime })|(s,q,\mathbf{v}),u_C,u_A\right)=1.$$

(7)

Proof. 

The proof is presented in Appendix B.    □

We write $\mathfrak{s}$ to represent a state $(s,q,\mathbf{v})$ in PDSG $\mathcal{P}$. We denote the clock valuation of $\mathfrak{s}$ by $Time\left(\mathfrak{s}\right)$. In the sequel, we compute a set of states called generalized accepting maximal end components (GAMECs) of $\mathcal{P}$. Any state $\mathfrak{s}$ in GAMECs satisfies that the successor state ${\mathfrak{s}}^{\prime }$ also belongs to GAMECs under any policy committed by the defender, regardless of the actions taken by the adversary. Therefore, for a path that stays within GAMECs, it is guaranteed that the path corresponds to a run that intersects with F infinitely many times, and thus, the path satisfies specification $\phi$. We can thus translate the problem of satisfying $\phi$ to the problem of reaching GAMECs under any adversary action. The set $\mathcal{C}=\left\{\mathfrak{s}\right|\mathfrak{s}\mathrm{belongs}\mathrm{to}\mathrm{some}\mathrm{GAMEC}\}$ can be computed using the procedure Compute_GAMEC($\mathcal{P}$) in Algorithm 1. The idea is that at each state, we prune the defender’s admissible action set by retaining only those actions that ensure state transitions in $\mathcal{P}$ will remain within GAMECs under any adversary action.

Algorithm 1 Computing the set of GAMECs $\mathcal{C}$.

1: procedure Compute_GAMEC($\mathcal{P}$)   2:     Input: PDSG $\mathcal{P}$   3:     Output: Set of GAMECs $\mathcal{C}$   4:     Initialization: $D\left(\mathfrak{s}\right)\leftarrow U_C\left(\mathfrak{s}\right)\forall \mathfrak{s}$; $\mathcal{C}\leftarrow \varnothing$; ${\mathcal{C}}_{temp}\leftarrow \left\{S\right\}$   5:     repeat   6:         $\mathcal{C}\leftarrow {\mathcal{C}}_{temp}$, ${\mathcal{C}}_{temp}\leftarrow \varnothing$   7:         for $N\in \mathcal{C}$ do   8:            $R\leftarrow \varnothing$   9:            Let $SC{C}_1,\cdots ,SC{C}_n$ be the set of strongly connected components of underlying digraph $G_{(N,D)}$ 10:            for $i=1,\cdots ,n$ do 11:                for each state $\mathfrak{s}\in SC{C}_i$ do 12:                    $D\left(\mathfrak{s}\right)\leftarrow \{u_C\in U_C\left(\mathfrak{s}\right)|{\mathfrak{s}}^{\prime }\in N,Pr\left({\mathfrak{s}}^{\prime }\right|\mathfrak{s},u_C,u_A)>0,\forall u_A\in U_A\left(\mathfrak{s}\right)\}$ 13:                    if $D\left(s\right)=\varnothing$ then 14:                        $R\leftarrow R\cup \left\{\mathfrak{s}\right\}$ 15:                    end if 16:                end for 17:            end for 18:            while $R\ne \varnothing$ do 19:                dequeue $\mathfrak{s}\in R$ from R and N 20:                if $\exists {\mathfrak{s}}^{\prime }\in N$ and $u_C\in U_C\left({\mathfrak{s}}^{\prime }\right)$ such that $Pr\left(\mathfrak{s}\right|{\mathfrak{s}}^{\prime },u_C,u_A)>0$ for some $u_A\in U_A\left({\mathfrak{s}}^{\prime }\right)$ then 21:                    $D\left({\mathfrak{s}}^{\prime }\right)\leftarrow D\left({\mathfrak{s}}^{\prime }\right)\backslash \left\{u_C\right\}$ 22:                    if $D\left({\mathfrak{s}}^{\prime }\right)=\varnothing$ then 23:                        $R\leftarrow R\cup \left\{{\mathfrak{s}}^{\prime }\right\}$ 24:                    end if 25:                end if 26:            end while 27:            for $i=1,\cdots ,n$ do 28:                if $N\cap SC{C}_i\ne \varnothing$ then 29:                    ${\mathcal{C}}_{temp}\leftarrow {\mathcal{C}}_{temp}\cup \{N\cap SC{C}_i\}$ 30:                end if 31:            end for 32:         end for 33:     until $\mathcal{C}={\mathcal{C}}_{temp}$ 34:     for $N\in \mathcal{C}$ do 35:         if $Ac{c}_{\mathcal{G}}\cap N=\varnothing$ then 36:            $\mathcal{C}\leftarrow \mathcal{C}\backslash N$ 37:         end if 38:     end for 39:     return $\mathcal{C}$ 40: end procedure

The procedure Compute_GAMEC($\mathcal{P}$) presented in Algorithm 1 takes the product DSG $\mathcal{P}$ as its input and returns set $\mathcal{C}$. The algorithm iteratively updates $\mathcal{C}$ by removing a set of states R. R includes any state $\mathfrak{s}$ that is in some strongly connected component (SCC) and has an empty admissible defender action set (Line 13). R also includes states ${\mathfrak{s}}^{\prime }$ from which $\mathcal{P}$ can be steered to R under some adversary action (Line 20). Lines 35–37 verify accepting conditions defined by the DTBA. The termination of Algorithm 1 is given by the following proposition.

Proposition 3. 

Algorithm 1 terminates in a finite number of iterations.

Proof. 

The proof of this proposition is given in Appendix B.    □

5.2. Evaluating Spatial Robustness

From Equation (1), evaluating the spatial robustness is equivalent to computing the Levenshtein distance between paths on the DSG synthesized under policies $\mu$ and $(\tau ,\gamma )$ and $\mathcal{L}$. This is equivalent to computing the Levenshtein distance between two automata, where the first automaton ${\mathcal{P}}^{\mu \tau \gamma }$ is the PDSG induced by policies $\mu$ and $(\tau ,\gamma )$. The second automaton is $\overline{\mathcal{A}}$, the DTBA representing $\neg \phi$. We adopt the approach proposed in [42] to compute the Levenshtein distance between ${\mathcal{P}}^{\mu \tau \gamma }$ and $\overline{\mathcal{A}}$.

We first construct a DSG ${\mathcal{G}}^{\mu \tau \gamma }$ from the original DSG $\mathcal{G}$. Given policies $\mu$ and $(\tau ,\gamma )$, we retain only those transitions such that $P{r}_{\mathcal{G}}\left(s^{\prime }\right|s,u_C,u_A)>0$, $T_{\mathcal{G}}\left(\delta \right|s,u_C,u_A,s^{\prime })>0$ for some $\delta$, $\mu (s,u_C)>0$, and $\tau (s,u_A)>0$, and we remove all other transitions. We augment the alphabet of DTBA $\mathcal{A}$ as $2^{\mathrm{\Pi }}\cup \left\{null\right\}$, where $null$ is a symbol that will be used to indicate deletion and insertion operations. The alphabet of $\overline{\mathcal{A}}$ is also augmented to include $null$. The PDSG ${\mathcal{P}}^{\mu \tau \gamma }$ in Definition 8 can be constructed from ${\mathcal{G}}^{\mu \tau \gamma }$ and $\mathcal{A}$. Given ${\mathcal{P}}^{\mu \tau \gamma }$ and $\overline{\mathcal{A}}$, we construct $\hat{\mathcal{P}}:={\mathcal{P}}^{\mu \tau \gamma }\times \overline{\mathcal{A}}$. Following [42], we construct a weighted transducer to capture the cost associated to each edit operation (assumed $=1$). We assign a cost $c((s,q,\mathbf{v},\overline{q}),(s^{\prime },q^{\prime },{\mathbf{v}}^{\prime },{\overline{q}}^{\prime }))$ to each transition from state $(s,q,\mathbf{v},\overline{q})$ to $(s^{\prime },q^{\prime },{\mathbf{v}}^{\prime },{\overline{q}}^{\prime })$ in $\hat{\mathcal{P}}$. In particular, $c((s,q,\mathbf{v},\overline{q}),(s^{\prime },q^{\prime },{\mathbf{v}}^{\prime },{\overline{q}}^{\prime }))=1$ if $L\left(s^{\prime }\right)$ is not the same as the label of the transition from $\overline{q}$ to ${\overline{q}}^{\prime }$ in $\overline{\mathcal{A}}$. We can then apply a shortest path algorithm on $\hat{\mathcal{P}}$ from the initial state $(s_0,q_0,\mathbf{0},{\overline{q}}_0)$ to the union of the GAMECs of $\hat{\mathcal{P}}$ to compute the minimum Levenshtein distance. The correctness of this approach follows from [42] [Theorem 2].

The computational complexity of calculating the spatial robustness for any given policies $\mu$ and $(\tau ,\gamma )$ is $O\left(\right(|2^{\mathrm{\Pi }}{|+1)}^2|{\mathcal{P}}^{\mu \tau \gamma }\left|\right|\overline{\mathcal{A}}\left|\right)$, where $|{\mathcal{P}}^{\mu \tau \gamma }|$ and $|\overline{\mathcal{A}}|$ are the sizes of ${\mathcal{P}}^{\mu \tau \gamma }$ and $\overline{\mathcal{A}}$, respectively [42].

5.3. Evaluating Temporal Robustness

In this subsection, we present a procedure to evaluate the temporal robustness. We introduce some notation. For a time interval I, we use $\underline{I}$ and $\overline{I}$ to represent its lower and upper bounds. The upper bound of the clock valuation set is denoted as $\overline{V}$. The indicator function $M\left(\mathfrak{s}\right)$ takes value 1 if $\mathfrak{s}$ is in GAMEC and 0 otherwise. A state ${\mathfrak{s}}^{\prime }$ is said to be a neighboring state of $\mathfrak{s}$ if $Pr\left({\mathfrak{s}}^{\prime }\right|\mathfrak{s},u_C,u_A)>0$ for some $u_C$ and $u_A$ such that $\mu (\mathfrak{s},u_C)>0$ and $\tau (\mathfrak{s},u_A)>0$. Given the policies of the defender and adversary, we define

$$b^{\mu \tau \gamma }(\mathfrak{s},{\mathfrak{s}}^{\prime }):=\left\{\begin{array}{cc}1\hfill & \mathrm{if}{\mathfrak{s}}^{\prime }\mathrm{is} \mathrm{a} \mathrm{neighboring} \mathrm{state} \mathrm{of} \mathfrak{s}\hfill \\ -\infty \hfill & \mathrm{otherwise}\hfill \end{array}\right..$$

The procedure Temporal($\phi ,\mathfrak{s},\delta ,M\left({\mathfrak{s}}^{\prime }\right)$) presented in Algorithm 2 computes the left and right temporal robustness with respect to the MITL objective $\phi$. The left and right temporal robustness of $\pi$ can be computed by searching over a directed graph representation of the product DSG. The algorithm determines the temporal robustness of $\phi$ following the robust MITL semantics (Definition 7) by simple algebraic computations over the temporal robustness of all atomic propositions in $\phi$.

Algorithm 2 Evaluate temporal robustness.

1: procedure Temporal($\phi ,\mathfrak{s},\delta ,M\left({\mathfrak{s}}^{\prime }\right)$)   2:     Input: MITL formula $\phi$, current state $\mathfrak{s}$, time duration $\delta$, indicator function $M\left({\mathfrak{s}}^{\prime }\right)$   3:     Output: Temporal robustness ${\chi }_t^{\phi }(\mu ,\tau ,\gamma )$   4:     if $\phi =\pi$ then   5:         $left\_temp\leftarrow min{\displaystyle \bigcup _{{\mathfrak{s}}^{″}}}\{Time\left({\mathfrak{s}}^{″}\right)-Time\left(\mathfrak{s}\right)\}$, where ${\mathfrak{s}}^{″}$ is reachable from $\mathfrak{s}$   6:         $right\_temp\leftarrow min{\displaystyle \bigcup _{{\mathfrak{s}}^{″}}}\{\overline{V}-Time\left({\mathfrak{s}}^{″}\right)\}$, where ${\mathfrak{s}}^{″}$ is reachable from $\mathfrak{s}$   7:         return $min\{left\_temp,right\_temp\}$   8:     else if $\phi ={\varphi }_1\wedge {\varphi }_2$ then   9:         $r_1\leftarrow$ Temporal $({\varphi }_1,\mathfrak{s},\delta ,M\left({\mathfrak{s}}^{\prime }\right))$ 10:         $r_2\leftarrow$ Temporal $({\varphi }_2,\mathfrak{s},\delta ,M\left({\mathfrak{s}}^{\prime }\right))$ 11:         return $min\{r_1,r_2\}$ 12:     else if $\phi ={\varphi }_1\vee {\varphi }_2$ then 13:         $r_1\leftarrow$ Temporal $({\varphi }_1,\mathfrak{s},\delta ,M\left({\mathfrak{s}}^{\prime }\right))$ 14:         $r_2\leftarrow$ Temporal $({\varphi }_2,\mathfrak{s},\delta ,M\left({\mathfrak{s}}^{\prime }\right))$ 15:         return $max\{r_1,r_2\}$ 16:     else if $\phi ={\varphi }_1{\mathcal{U}}_I{\varphi }_2$ then 17:         if $M\left({\mathfrak{s}}^{\prime }\right)=0$ then 18:            $r_1\leftarrow min{\displaystyle \bigcup _{{\mathfrak{s}}^{\prime },\delta ,{\delta }^{\prime }}}\left\{\mathrm{TEMPORAL}({\varphi }_1,\mathfrak{s},\delta ,M\left({\mathfrak{s}}^{\prime }\right)),b^{\mu \tau \gamma }(\mathfrak{s},{\mathfrak{s}}^{\prime })\mathrm{TEMPORAL}({\varphi }_1{\mathcal{U}}_{I-\delta }{\varphi }_2,{\mathfrak{s}}^{\prime },{\delta }^{\prime },M\left({\mathfrak{s}}^{\prime \prime }\right)\right\}$ 19:         else 20:            $r_1\leftarrow min{\displaystyle \bigcup _{{\mathfrak{s}}^{\prime }}}\left\{(Time\left({\mathfrak{s}}^{\prime }\right)-\underline{I}),(\overline{I}-Time\left({\mathfrak{s}}^{\prime }\right))\right\}$ 21:         end if 22:         if $0\in I$ then 23:            $r_2\leftarrow \mathrm{TEMPORAL}({\varphi }_2,\mathfrak{s},\delta ,M\left({\mathfrak{s}}^{\prime }\right))$ 24:         else 25:            $r_2\leftarrow -\infty$ 26:         end if 27:         return $max\{r_1,r_2\}$ 28:     end if 29: end procedure

We detail the workings of Algorithm 2, which is a recursive procedure that is used to compute the temporal robustness. It takes an MITL formula $\phi$, current state $\mathfrak{s}$, time duration $\delta$, and indicator function $M\left({\mathfrak{s}}^{\prime }\right)$ as its inputs. If $\phi =\pi$, then Algorithm 2 computes the minimum left temporal robustness (Line 5) and right temporal robustness (Line 6), respectively. The minimum of these quantities is returned as the temporal robustness. From the robust MITL semantics, Algorithm 2 returns the minimum (maximum) temporal robustness when $\phi$ is a conjunction (disjunction). When $\phi ={\varphi }_1{\mathcal{U}}_I{\varphi }_2$, the robustness is computed following Lines 16–27. Here, $I-t:=\{t^{\prime }-t|t^{\prime }\in I\}$. Because we focus on the worst-case robustness, we compute the minimum value over times $\delta$ and neighboring states ${\mathfrak{s}}^{\prime }$ in Line 18. We establish the correctness of Algorithm 2 as follows.

Theorem 1. 

Given a PDSG with initial state ${\mathfrak{s}}_0$, MITL formula φ, and policies μ and τ, suppose Algorithm 2 returns $ϵ\ge 0$. Then, any run on the PDSG synthesized under policies μ and τ can be temporally perturbed by $\widehat{ϵ}\in [0,ϵ]$ without violating φ.

Proof. 

The proof is presented in Appendix B.    □

The complexity of Algorithm 2 is $O\left(\right|cl\left(\phi \right)\left|\right(\left|S\right|+\left|Pr\right|\left)\right)$, where $\left|cl\right(\phi \left)\right|$ is the size of the closure of formula $\phi$ and $\left|Pr\right|$ is the number of nonzero elements in matrix $Pr$.

5.4. Evaluating Spatio-Temporal Robustness

We use the results of the previous two subsections to compute the spatio-temporal robustness using the procedure Robust($\phi ,\mathfrak{s},\delta ,M\left({\mathfrak{s}}^{\prime }\right),{ϵ}_s$) presented in Algorithm 3. From Equation (5), when the spatial robustness is above ${ϵ}_s$, Algorithm 3 returns the temporal robustness. Otherwise, it returns the negative value of the temporal robustness. The complexity of Algorithm 3 is $O\left(\right|cl\left(\phi \right)\left|\right(\left|S\right|+\left|Pr\right|)+\left(\right|2^{\mathrm{\Pi }}{|+1)}^2|{\mathcal{P}}^{\mu \tau \gamma }\left|\right|\overline{\mathcal{A}}\left|\right)$. Table 1 summarizes the computational complexities of evaluating the spatial and temporal robustness.

Algorithm 3 Evaluate spatio-temporal robustness.

1: procedure Robust($\phi ,\mathfrak{s},\delta ,M\left({\mathfrak{s}}^{\prime }\right),{ϵ}_s$)   2:     Input: MITL formula $\phi$, current state s, time duration $\delta$, indicator function $M\left({\mathfrak{s}}^{\prime }\right)$   3:     Output: Spatio-temporal robustness ${\chi }^{\phi }(\mu ,\tau ,\gamma )$   4:     if $\phi =\top$ then   5:         return ∞   6:     else if $\phi =\perp$ then   7:         return $-\infty$   8:     else   9:         if $\mathrm{SPATIAL}(\phi ,s)\ge {ϵ}_s$ then 10:            return $\mathrm{TEMPORAL}(\phi ,\mathfrak{s},\delta ,M\left({\mathfrak{s}}^{\prime }\right))$ 11:         else 12:            return $-\mathrm{TEMPORAL}(\phi ,\mathfrak{s},\delta ,M\left({\mathfrak{s}}^{\prime }\right))$ 13:         end if 14:     end if 15: end procedure

Table 1. Computational complexities of evaluating the spatial and temporal robustness when policies are given. $|{\mathcal{P}}^{\mu \tau \gamma }|$ is the size of product DSG ${\mathcal{P}}^{\mu \tau \gamma }$ induced by policies $\mu$ and $(\tau ,\gamma )$. $|\overline{\mathcal{A}}|$ is the size of the timed Büchi automaton of MITL specification $\neg \phi$. $\left|cl\right(\phi \left)\right|$ denotes the size of the closure of $\phi$, and $\left|Pr\right|$ is the number of nonzero elements in matrix $Pr$. The complexity of Algorithm 3 is $\left(S\right)+\left(T\right)$.

Robustness	Complexity
Spatial (S)	$O\left(\right(|2^{\mathrm{\Pi }}{|+1)}^2|{\mathcal{P}}^{\mu \tau \gamma }\left|\right|\overline{\mathcal{A}}\left|\right)$
Temporal (T)	$O\left(\right|cl\left(\phi \right)\left|\right(\left|S\right|+\left|Pr\right|\left)\right)$

5.5. Control Policy Synthesis

In this subsection, we compute a control policy that solves the robust policy synthesis for the defender in Problem 1 when there is no timing attack. From Proposition 1, solving the robust policy synthesis for the defender in Problem 1 is equivalent to finding a defender policy so that the spatio-temporal robustness exceeds a desired threshold. This procedure is named as Policy_Synthesis($\mathcal{P},\phi$) and is presented in Algorithm 4. We initialize a policy ${\mu }^k$, $k=1$ (Line 4). We also define sets of states ${\mathcal{E}}_t$ and ${\mathcal{E}}_s$ that will indicate states/transitions that lead to violations of temporal and spatial robustness. We then compute the best response to ${\mu }^k$ as $({\tau }^k,{\gamma }^k)$ and evaluate the spatio-temporal robustness ${\chi }^{\phi }({\mu }^k,{\tau }^k,{\gamma }^k)$. If ${\chi }^{\phi }({\mu }^k,{\tau }^k,{\gamma }^k)\ge {ϵ}_t$, we then synthesize the policy ${\mu }^k$ returned in Line 6. If $0\le {\chi }^{\phi }({\mu }^k,{\tau }^k,{\gamma }^k)<{ϵ}_t$, then the spatial robustness exceeds ${ϵ}_s$ but the temporal robustness is below ${ϵ}_t$. In this case, we eliminate defender actions $u_C$ that steer the PDSG into states $\mathfrak{s}$ in ${\mathcal{E}}_t$ with the positive probability thereby causing a violation of the temporal robustness constraint. If ${\chi }^{\phi }({\mu }^k,{\tau }^k,{\gamma }^k)<0$ (Line 17), then the spatial robustness constraint is violated. In this case, we eliminate defender actions that steer the system into states in ${\mathcal{E}}_s$. If no state in GAMEC is reachable from the initial state ${\mathfrak{s}}_0$ of the product DSG $\mathcal{P}$, then the procedure Policy_Synthesis($\mathcal{P},\phi$) presented in Algorithm 4 reports failure, indicating that no solution is found for robust policy synthesis for defender in Problem 1, and terminates. We establish the converge of Algorithm 4 as follows.

Algorithm 4 Robust control policy synthesis for defender.

1: procedure Policy_Synthesis($\mathcal{P}$, $\phi$)   2:     Input: Product DSG $\mathcal{P}$, MITL formula $\phi$   3:     Output: Control policy $\mu$   4:     Initialization: Iteration index $k\leftarrow 1$. Initialize ${\mu }^k(\mathfrak{s},u_C)\leftarrow \frac{1}{|U_C\left(\mathfrak{s}\right)|}$ for all $\mathfrak{s}$ and $u_C\in U_C\left(\mathfrak{s}\right)$, and compute adversary policy $({\tau }^k,{\gamma }^k)\in \mathcal{BR}\left({\mu }^k\right)$. Let ${\mathcal{E}}_s,{\mathcal{E}}_t\leftarrow \varnothing$.   5:     while true do   6:         Compute spatio-temporal robustness ${\chi }^{\phi }({\mu }^k,{\tau }^k,{\gamma }^k)=\mathrm{ROBUST}(\phi ,{\mathfrak{s}}_0,\delta ,M\left({\mathfrak{s}}^{\prime }\right))$.   7:         if ${\chi }^{\phi }({\mu }^k,{\tau }^k,{\gamma }^k)\ge {ϵ}_t$ then   8:            return ${\mu }^k$   9:         else if $0\le {\chi }^{\phi }({\mu }^k,{\tau }^k,{\gamma }^k)<{ϵ}_t$ then 10:            ${\mathcal{E}}_t\leftarrow {\mathcal{E}}_t\cup \{\mathfrak{s}:\mathrm{ROBUST}(\phi ,\mathfrak{s},\delta ,M\left({\mathfrak{s}}^{\prime }\right))<{ϵ}_t\}$ 11:            for $\mathfrak{s}\in {\mathcal{E}}_t$ do 12:                Let $U_C\left({\mathfrak{s}}^{\prime }\right)\leftarrow U_C\left({\mathfrak{s}}^{\prime }\right)\backslash \{u_C:{\mu }^k({\mathfrak{s}}^{\prime },u_C)>0,P{r}^{{\mu }^k{\tau }^k}({\mathfrak{s}}^{\prime },\mathfrak{s})>0\}$ for all ${\mathfrak{s}}^{\prime }\notin {\mathcal{E}}_t\cup {\mathcal{E}}_s$ 13:                if $U_C\left({\mathfrak{s}}^{\prime }\right)=\varnothing$ then 14:                    ${\mathcal{E}}_t\leftarrow {\mathcal{E}}_t\cup \left\{{\mathfrak{s}}^{\prime }\right\}$ 15:                end if 16:            end for 17:         else 18:            ${\mathcal{E}}_s\leftarrow {\mathcal{E}}_s\cup \{\mathfrak{s}:\mathrm{ROBUST}(\phi ,\mathfrak{s},\delta ,M\left({\mathfrak{s}}^{\prime }\right))<0\}$ 19:            for $\mathfrak{s}\in {\mathcal{E}}_s$ do 20:                Let $U_C\left({\mathfrak{s}}^{\prime }\right)\leftarrow \left\{u_C\right|{\mu }^k({\mathfrak{s}}^{\prime },u_C)>0,P{r}^{{\mu }^k{\tau }^k}({\mathfrak{s}}^{\prime },\mathfrak{s})>0\}$ 21:                if $U_C\left({\mathfrak{s}}^{\prime }\right)=\varnothing$ then 22:                    ${\mathcal{E}}_s\leftarrow {\mathcal{E}}_s\cup \left\{{\mathfrak{s}}^{\prime }\right\}$ 23:                end if 24:            end for 25:            Update defender’s policy ${\mu }^{k+1}({\mathfrak{s}}^{\prime },u_C)\leftarrow \frac{1}{|U_C\left({\mathfrak{s}}^{\prime }\right)|}$ for all ${\mathfrak{s}}^{\prime }$ and $u_C\in U_C\left({\mathfrak{s}}^{\prime }\right)$ 26:            if GAMEC is not reachable from initial state ${\mathfrak{s}}_0$ then 27:                return message “failure” indicating no solution is found 28:                Break 29:            end if 30:         end if 31:         Let $k\leftarrow k+1$. 32:     end while 33: end procedure

Theorem 2. 

Algorithm 4 terminates within a finite number of iterations.

Proof. 

The proof of this theorem is presented in Appendix B.    □

In the worst case, we have that Algorithm 4 updates ${\hat{U}}_C=\varnothing$ with at most $\left|S\right|\times |U_C|$ number of iterations. Thus, the complexity of Algorithm 4 is $O\left(\right|S|\times |U_C\left|\right)$. We further present the optimality of the policy found by Algorithm 4 in the following theorem:

Theorem 3. 

If Algorithm 4 returns a defender’s policy, denoted as ${\mu }^{*}$, then the problem of robust policy synthesis for the defender in Problem 1 is feasible. Moreover, the defender’s policy ${\mu }^{*}$ is an optimal solution to Problem 1.

Proof. 

The proof is presented in Appendix B.    □

The soundness of Algorithm 4 is given below:

Corollary 1. 

Algorithm 4 is sound but not complete. That is, any control policy returned by Algorithm 4 guarantees probability one of satisfying the given MITL specification, but we cannot conclude that there exists no solution to the problem if Algorithm 4 returns no solution.

6. Solution: Actuator and Timing Attacks

In this section, we present a solution under both actuator attack and timing attacks.

Compared with the case where there is no timing attack, we make the following observations. The evaluation of spatial robustness remains unchanged when the adversary can initiate both actuator and timing attacks. Second, the evaluation of temporal robustness can become inaccurate during a timing attack. This is because timing information perceived by the defender can be arbitrarily manipulated by the adversary. As a result, the defender will not be able to evaluate the temporal robustness and hence the spatio-temporal robustness during a timing attack. Finally, as the defender cannot accurately evaluate the temporal robustness, Proposition 1 will not hold during a timing attack. In the following, we relax the problem of robust synthesis for the defender in Problem 1 and try to compute a defender policy such that the probability of satisfying the $\phi$ is maximized in the presence of actuator and timing attacks. The reason the defender can evaluate the probability of satisfying $\phi$ is that it knows the transition probability $P{r}_{\mathcal{G}}$ and probability mass function $T_{\mathcal{G}}$. Thus, it can determine the expected probability and time of reaching each state, given the policies of the defender and adversary. The relaxed problem is:

Problem 2 

(Policy synthesis for defender). Given a DSG $\mathcal{G}$ and an MITL objective φ, compute a defender’s policy such that the probability of satisfying φ is maximized and adversary policy $(\tau ,\gamma )$ is the best response to control policy μ. That is, ${max}_{\mu }{\mathbb{P}}^{\phi }(\mu ,\tau ,\gamma )$, where $(\tau ,\gamma )\in BR\left(\mu \right)$.

Because the timing information perceived by the defender has been manipulated by the adversary, the defender has limited knowledge of the current time. Even in this case, it can still detect unreasonable time sequences, e.g., a time sequence that is not monotonic. To recover from the deficit of timing information, we represent the defender’s policy using a finite-state controller, which enables the defender to track the estimated time.

Definition 9 

(Finite-state controller [25]). A finite-state controller (FSC) is a tuple $\mathcal{F}=(Y,y_0,\mu )$, where $Y=\Lambda \times \{0,1\}$ is a finite set of internal states, $\Lambda$ is a set of estimates of clock valuations, and the set $\{0,1\}$ indicates if a timing attack has been detected (1) or not (0). $y_0$ is the initial internal state. μ is the defender policy, given by:

$$\mu =\left\{\begin{array}{c}{\mu }_0:Y\times S\times Y\times U_C\mapsto [0,1],\mathit{if}{\mathcal{H}}_0\mathit{holds};\hfill \\ {\mu }_1:Y\times S_{\mathcal{G}}\times Q\times Y\times U_C\mapsto [0,1],\mathit{if}{\mathcal{H}}_1\mathit{holds}.\hfill \end{array}\right.$$

where ${\mu }_0$ and ${\mu }_1$ denote the control policies that will be executed when hypothesis ${\mathcal{H}}_0$ or ${\mathcal{H}}_1$ holds, respectively.

For an FSC as given in Definition 9, hypothesis ${\mathcal{H}}_0$ represents the scenario where no timing attack is detected by the defender, while ${\mathcal{H}}_1$ represents the scenario where a timing attack is detected. In the FSC, the defender’s policy specifies the probability of reaching the next internal state by taking an action $u_C$ given the current state of DSG, detection result of the timing attack, and state of DTBA.

To capture the state evolutions of DSG, DTBA, and FSC, we construct a global DSG.

Definition 10 

(Global DSG (GDSG)). A GDSG is a tuple $\mathcal{Z}=(S_{\mathcal{Z}},s_{\mathcal{Z},0},U_C,U_A,In{f}_{\mathcal{Z},C},In{f}_{\mathcal{Z},A},P{r}_{\mathcal{Z}},Ac{c}_{\mathcal{Z}})$, where $S_{\mathcal{Z}}=S\times Y$ is a finite set of states and $s_{\mathcal{Z},0}=(s_0,q_0,\mathbf{0},y_0)$ is the initial state. $U_C$ and $U_A$ are finite sets of actions and $In{f}_{\mathcal{Z},C}$ and $In{f}_{\mathcal{Z},A}$ are the information sets of the defender and adversary, respectively. $P{r}_{\mathcal{Z}}:S_{\mathcal{Z}}\times U_C\times U_A\times S_{\mathcal{Z}}\mapsto [0,1]$ is a transition function where $P{r}_{\mathcal{Z}}\left((s^{\prime },q^{\prime },{\mathbf{v}}^{\prime },y^{\prime })|(s,q,\mathbf{v},y),u_C,u_A\right)$ is the probability of a transition from state $(s,q,\mathbf{v},y)$ to $(s^{\prime },q^{\prime },{\mathbf{v}}^{\prime },y)$ when the defender and adversary take actions $u_C$ and $u_A$, respectively. The transition probability is given by

$$\begin{array}{c}P{r}_{\mathcal{Z}}\left((s^{\prime },q^{\prime },{\mathbf{v}}^{\prime },y^{\prime })|(s,q,\mathbf{v},y),u_C,u_A\right)\hfill \\ \hfill =\left\{\begin{array}{cc}\sum _{{\mathbf{v}}^{″}}\gamma \left({\mathbf{v}}^{″}\right|\mathbf{v}){\mu }_0(y^{\prime },u_C|s,q,{\mathbf{v}}^{″},y)Pr\left((s^{\prime },q^{\prime },{\mathbf{v}}^{\prime })|(s,q,\mathbf{v}),u_C,u_A\right),\hfill & \mathit{if}{\mathcal{H}}_0\mathit{holds};\hfill \\ {\mu }_1(y^{\prime },u_C|s,q,y)T_{\mathcal{G}}\left(\delta \right|s,u_C,u_A,s^{\prime })P{r}_{\mathcal{G}}\left(s^{\prime }\right|s,u_C,u_A),\hfill & \mathit{if}{\mathcal{H}}_1\mathit{holds};\hfill \end{array}\right.\end{array}$$

$Ac{c}_{\mathcal{Z}}=Acc\times Y$ is the set of accepting states.

Consider the global DSG. Let $\mathbf{Q}\in {\mathbb{R}}^{|S_{\mathcal{Z}}|}$ be the probability of satisfying $\phi$. Then, $\mathbf{Q}$ can be computed from Proposition 4. A proof is presented in [39].

Proposition 4. 

Let $\mathbf{Q}:=\underset{\mu }{max}\underset{\tau ,\gamma }{min}\mathbb{P}\left(\phi \right)$ be the probability of satisfying φ. Then,

$$\begin{array}{c}\hfill \mathbf{Q}\left((\mathfrak{s},y)\right)=\underset{\mu }{max}\underset{\tau ,\gamma }{min}\sum _{u_C}\sum _{u_A}\sum _{({\mathfrak{s}}^{\prime },y)}\tau ((\mathfrak{s},y),u_A)\mathbf{Q}\left(({\mathfrak{s}}^{\prime },y^{\prime })\right)·P{r}_{\mathcal{Z}}\left(({\mathfrak{s}}^{\prime },y^{\prime })|(\mathfrak{s},y),u_C,u_A\right),\forall (\mathfrak{s},y).\end{array}$$

Moreover, the value vector is unique.

We use the procedure Control_Synthesis($\mathcal{Z}$) presented in Algorithm 5 to compute the policy $\mu$. Guarantees on its termination is presented in [39]. We finally remark on the complexity of Algorithm 5. We first make the following relaxation to Line 5 of Algorithm 5 so that ${\mathbf{Q}}^{k+1}\left((\mathfrak{s},y)\right)$ is updated if the following holds:

$$\underset{\mu }{max}\underset{\tau ,\gamma }{min}\sum _{u_C}\sum _{u_A}\sum _{({\mathfrak{s}}^{\prime },y)}\tau ((\mathfrak{s},y),u_A)\mathbf{Q}\left(({\mathfrak{s}}^{\prime },y^{\prime })\right)·P{r}_{\mathcal{Z}}\left(({\mathfrak{s}}^{\prime },y^{\prime })|(\mathfrak{s},y),u_C,u_A\right)\ge (1+ϵ){\mathbf{Q}}^k\left((\mathfrak{s},y)\right).$$

Then, Algorithm 5 converges to some ${\mathbf{Q}}^{k+1}(\mathfrak{s},y)$ satisfying $\parallel {\mathbf{Q}}^{k+1}(\mathfrak{s},y)-{\mathbf{Q}}^k{(\mathfrak{s},y)\parallel }_{\infty }<ϵ$ within $|S_{\mathcal{Z}}|{max}_{(\mathfrak{s},y)}\{log(1/{\mathbf{Q}}^0\left((\mathfrak{s},y)\right))/log(1+ϵ)\}$ iterations, where parameter ${\mathbf{Q}}^0\left((\mathfrak{s},y)\right))$ is the smallest value of ${\mathbf{Q}}^k\left((\mathfrak{s},y)\right))$ for $k=0,1,\dots$ Furthermore, Line 8 of Algorithm 5 can be solved using a linear program in polynomial time, denoted as f. Combining these arguments, the complexity of Algorithm 5 is $|S_{\mathcal{Z}}|f{max}_{(\mathfrak{s},y)}\{log(1/{\mathbf{Q}}^0\left((\mathfrak{s},y)\right))/log(1+ϵ)\}$.

Algorithm 5 Computing an optimal control policy.

1: procedure Control_Synthesis($\mathcal{Z}$)   2:     Input: Global DSG $\mathcal{P}$   3:     Output: value vector $\mathbf{Q}$   4:     Initialization: ${\mathbf{Q}}^0\leftarrow \mathbf{0}$, ${\mathbf{Q}}^1\left(\mathfrak{s}\right)\leftarrow 1$ for $\mathfrak{s}\in Acc$, ${\mathbf{Q}}^1\left(\mathfrak{s}\right)\leftarrow 0$ otherwise, $k\leftarrow 0$   5:     while $max\left\{\right|{\mathbf{Q}}^{k+1}\left(\mathfrak{s}\right)-{\mathbf{Q}}^k\left(\mathfrak{s}\right)|:\mathfrak{s}\in S\}>ϵ$ do   6:         $k\leftarrow k+1$   7:         for $\mathfrak{s}\notin Acc$ do   8:            $$\begin{gathered} {\mathbf{Q}}^{k+1}\left(\mathfrak{s}\right)\leftarrow \underset{\mu }{max}\underset{\tau ,\gamma }{min}\left\{{\displaystyle \sum _{u_C}}{\displaystyle \sum _{u_A}}{\displaystyle \sum _{({\mathfrak{s}}^{\prime },y)}}\tau ((\mathfrak{s},y),u_A)\gamma ({\mathbf{v}}^{″},\mathbf{v})\mathbf{Q}\left(({\mathfrak{s}}^{\prime },y^{\prime })\right) \\ ·P{r}_{\mathcal{Z}}\left(({\mathfrak{s}}^{\prime },y^{\prime })|(\mathfrak{s},y),u_C,u_A\right)\right\} \end{gathered}$$   9:         end for 10:     end while 11:     return ${\mathbf{Q}}^k$ 12: end procedure

7. Case Study

In this section, we present a numerical case study on a signalized traffic network. The case study was implemented using MATLAB on a Macbook Pro with a 2.6 GHz Intel Core i5 CPU and 8 GB of RAM.

7.1. Signalized Traffic Network Model

We consider a signalized traffic network [43] consisting of five intersections and twelve links under the remote control of a transportation management center (TMC). A representation of the signalized traffic network is shown in Figure 3.

Figure 3. Representation of a signalized traffic network consisting of five intersections and twelve links.

We briefly explain how a DSG from Definition 4 can model the network. Each DSG state models the total number of vehicles on a link in the network. Transitions between the states in the DSG models the flow of vehicles. Because the vehicle capacity of a link is finite, the number of states in the DSG will be finite.

The defender’s action set represents that the TMC can actuate a link by issuing a ‘green signal’ on outgoing intersections of that link. Conversely, the TMC can block a link by issuing a ‘red signal’.

The TMC is assumed to control the traffic network over an unreliable wireless channel. Thus, an intelligent adversary can launch man-in-the-middle attacks to tamper with the traffic signal issued by the TMC or manipulate observations of the TMC. In particular, the adversary can initiate an actuator attack to change the traffic signal and a timing attack to manipulate the time-stamped measurement (number of vehicles at each link along with the time index) perceived by the TMC.

The TMC is given one of the following objectives: (i) number of vehicles at link 4 is eventually below 10 before deadline $d=6$: ${\phi }_1={\diamond }_{[0,6]}(x_4\le 10)$; (ii) number of vehicles at links 3 and 4 are eventually below 10 before $d=6$: ${\phi }_2={\diamond }_{[0,6]}\left((x_3\le 10)\wedge (x_4\le 10)\right)$; or (iii) number of vehicles at links 3, 4, and 5 are eventually below 10 before $d=6$: ${\phi }_3={\diamond }_{[0,6]}\left((x_3\le 10)\wedge (x_4\le 10)\wedge (x_5\le 10)\right)$. Spatial and temporal robustness thresholds are set to ${ϵ}_s=1$ and ${ϵ}_t=1$. We compare our approach with two baselines. In Baseline 1, the TMC periodically issues green signals. In Baseline 2, the TMC always issues green signals for links $3,4,$ and 5 to greedily minimize the number of vehicles on these links.

7.2. Numerical Results

In the following, we present the numerical results using our proposed approach and the two baselines.

We first report the results when the adversary only launches an actuator attack and the TMC is given specification ${\phi }_1$. We compute a control policy using Algorithm 4. A sample sequence of traffic signals is presented in Table 2. Using Proposition 1 and Corollary 1, the MITL specification ${\phi }_1$ is satisfied with probability one.

Table 2. Sample sequence of traffic lights realized at each intersection for the MITL specification ${\phi }_1={\diamond }_{[0,6]}(x_4\le 10)$. The letters ‘R’ and ‘G’ represent ‘red’ and ‘green’ signals, respectively.

	Intersection
Time	1	2	3	4	5
1	G	R	R	G	R
2	R	R	G	G	R
3	R	G	G	G	R
4	R	R	R	R	G
5	R	G	G	G	R
6	G	G	G	R	G

We then consider an adversary that launches both actuator and timing attacks. Suppose that the TMC is equipped with an FSC with five states. We show the results of our approach using Algorithm 5 in Figure 4. In this example, ${\phi }_3$ is violated as the number of vehicles on link 5 exceeds the threshold of 10. We also give the probabilities of satisfying each MITL specification using Algorithm 5. Specifications ${\phi }_1$, ${\phi }_2$, and ${\phi }_3$ are satisfied with the probabilities $0.7000$, $0.6857$, and $0.4390$, respectively.

Figure 4. A sample of the number of vehicles on links 3, 4, and 5 over time using our proposed approach. In this realization, the number of links on link 5 is above the threshold.

We assume that the TMC commits to deterministic policies in both baselines. In Baseline 1, the adversary launches actuator attacks when the TMC issues a green signal and does not attack when it issues a red signal. In Baseline 2, the adversary always launches an actuator attack. In both baselines, the adversary launches a timing attack at each time instant to delay the TMC’s observation. As a consequence, both baselines have zero probability of satisfying ${\phi }_1,{\phi }_2$, or ${\phi }_3$.

The DSG in our experiments had 232 states. For ${\phi }_1$, the GAMEC of the product DSG had 400 states. For ${\phi }_2$ and ${\phi }_3$, the GAMEC had 160 and 80, states respectively. The computation time of Algorithm 4 for ${\phi }_1$ was 264 s. Algorithm 5 took 720 s.

8. Conclusions and Future Work

In this paper, we proposed methods to synthesize controllers for cyber-physical systems to satisfy metric interval temporal logic (MITL) tasks in the presence of an adversary while additionally providing robustness guarantees. We considered the fragment of MITL formulae that can be represented by deterministic timed Büchi automata. The adversary could initiate actuator and timing attacks. We modeled the interaction between the defender and adversary using a durational stochastic game (DSG). We introduced three notions of robustness degree—spatial robustness, temporal robustness, and spatio-temporal robustness—and presented procedures to estimate these quantities, given the defender and adversary’s policies and current state of the DSG. We further presented a computational procedure to synthesize the defender’s policy that provided a robustness guarantee when the adversary could only initiate an actuator attack. A value iteration-based procedure was given to compute a defender’s policy to maximize the probability of satisfying the MITL goal. A case study using a signalized traffic network illustrated our approach.

DSGs can be adopted to model interactions between a defender and adversary across various application domains with time-sensitive constraints. Examples include the time-sensitive motion planning of drones, product scheduling of industrial control systems, and time-sensitive message transmissions in wireless communications in the presence of adversaries. For future work, we will generalize our definition of the DSG to broaden its applications. We will generalize DSGs to address partial observations by the CPS and adversary. We will additionally investigate the scenarios where the adversary is nonrational and may not perform its best response to the strategies committed by defender.