Automatic Addition of Fault-Tolerance in Presence of Unchangeable Environment Actions ^†

Abstract

We focus on the problem of adding fault-tolerance to an existing concurrent protocol in the presence of unchangeable environment actions. Such unchangeable actions occur in cases where a subset of components/processes cannot be modified since they represent third-party components or are constrained by physical laws. These actions differ from faults in that they are (1) simultaneously collaborative and disruptive, (2) essential for satisfying the specification and (3) possibly non-terminating. Hence, if these actions are modeled as faults while adding fault-tolerance, it causes existing model repair algorithms to declare failure to add fault-tolerance. We present a set of algorithms for adding stabilization and fault-tolerance for programs that run in the presence of environment actions. We prove the soundness, completeness and the complexity of our algorithms. We have implemented all of our algorithms using symbolic techniques in Java. The experimental results of our algorithms for various examples are also provided.

1. Introduction

Model repair is the problem of revising an existing model/program so that it satisfies new properties while preserving existing properties. It is desirable in several contexts such as when an existing program needs to be deployed in a new setting or to repair bugs. Model repair for fault-tolerance enables one to separate the fault-tolerance and functionality so that the designer can focus on the functionality of the program and utilize automated techniques for adding fault-tolerance. It can also be used to add fault-tolerance against a newly discovered fault.

This paper focuses on performing such repair when some actions cannot be removed from the model. We refer to such transitions as unchangeable environment actions. There are several possible reasons for actions being unchangeable. Examples include scenarios where the system consists of several components –some of which are developed in-house and can be repaired and some of which are third-party and cannot be changed. They are also useful in systems such as Cyber-Physical Systems (CPS) where modifying physical components may be very expensive or even impossible.

The environment actions differ from fault actions considered in existing work such as [1]. Fault actions are assumed to be temporary in nature and all the previously proposed algorithms to add fault-tolerance in [1] work only with this important assumption that faults finally stop occurring. Unlike fault actions, environment actions can keep occurring. Environment actions also differ from adversary actions considered in [2] or in the context of security intrusions. In particular, the adversary intends to cause harm to the system. By contrast, environment actions can be collaborative as well. In other words, environment actions are simultaneously collaborative and disruptive. The goal of this work is to identify whether it is possible for the program to be repaired so that it can utilize the assistance provided by the environment while overcoming its disruption. To give an intuition of the role of the environment and the difference between program, environment and fault actions, we present the following examples.

Intuitive examples to illustrate the role of the environment. The first intuitive example is motivated by a simple pressure cooker (see Figure 1). The environment (heat source) causes the pressure to increase. In the subsequent discussion, we analyze this pressure cooker when the heat source is always on. There are two mechanisms to decrease the pressure, a vent and an overpressure valve. For the sake of presentation, assume that pressure below 4 is normal. If the pressure increases to 4 or 5, the vent mechanism reduces the pressure by 1 in each step. However, the vent may fail (e.g., if something gets stuck in the vent pipe) and its pressure reduction mechanism becomes disabled. If the pressure reaches 6, the overpressure valve mechanism causes the valve to open resulting in an immediate drop in pressure to be less than 4. We denote the state where the pressure is x by $s_x$ when the vent is working and by $f{s}_x$ when the vent has failed.

Our goal in the subsequent discussion is to model the pressure cooker as a program and identify an approach for the role of the environment and its interaction with the program so that we can conclude this requirement: starting from any state identified above, the system reaches a state where the pressure is less than 4.

Next, we argue that how the role of the environment differs from the role of fault and program actions. In turn, this prevents us from using existing approaches such as Reference [1]. Specifically,

• Treating the environment as a fault does not work. Faults are assumed to be exceptional events in the system that are expected to stop after some time. By contrast, heat is an essential part of the system that is needed for the system to work. In addition, if we treat the environment as a fault, then none of the environment transitions including transitions from state $f{s}_4$ to $f{s}_5$ and from $f{s}_5$ to $f{s}_6$ are required to occur. If these actions do not occur, the overpressure valve is never activated. Hence, neither the valve nor the vent mechanism reduces the pressure to be less than 4.
• Treating the environment transitions similar to program transitions is also not acceptable. To illustrate this, consider the case where we want to make changes to the program in Figure 1. For instance, if the overpressure valve is removed, then this would correspond to removing the transition from $s_6$ (respectively $f{s}_6$) to where the pressure is less than 4. Also, if we add another safety mechanism, it would correspond to adding new transitions. However, we cannot do the same with environment actions that capture the changes made by the heat source. For example, we cannot add new transitions (e.g., from $f{s}_4$ to $s_4$) to the environment and we cannot remove transitions (e.g., from $s_4$ to $s_5$). In other words, even if we make any changes to the model in Figure 1 by adding or removing safety mechanisms, the transitions marked as environment actions remain unchanged. We cannot introduce new environment transitions and we cannot remove existing environment transitions. This is what we mean by the environment being unchangeable.
• Treating the environment to be collaborative without some special fairness to the program does not work either. In particular, without some special fairness for the program, the system can cycle through states $s_4,s_5,s_4,s_5\cdots$.
• Treating the environment to be simultaneously collaborative as well as adversarial where the program has some special fairness enables one to ensure that this program achieves its desired goals. In particular, we need the environment to be collaborative, that is, if it reaches a state where only environment actions can execute then one of them execute This is necessary to ensure that system can transition from state $f{s}_4$ to $f{s}_5$ and from $f{s}_5$ to $f{s}_6$ which is essential for recovery to a state where the pressure is less than 4. We also need the program to have special fairness to require that it executes faster than the environment so that it does not execute in a cycle through states $s_4,s_5,s_4,\cdots$ (we will precisely define the notion of faster in Section 2.1).

As another example, consider the shepherding problem. In this problem, a shepherd and his dog want to steer a sheep to a specific location (cf. Figure 2). They cannot carry the sheep; the only way to move the sheep is the movement of the sheep by itself. The sheep always tries to increase its distance from the shepherd and the dog. In this example, the program captures the behavior of the shepherd and the dog and the environment captures the behaviors of the sheep.

Like the pressure cooker example, the environment is both assistive and disruptive. On one hand, the environment is assistive, because without the environment actions it is impossible to reach the desired state. On the other hand, the environment is disruptive, because the environment actions can make a (poorly designed) program go into an infinite loop thereby never reaching the desired state. A good program, however, uses the assistance of the sheep (i.e., the environment) while overcomes its disruption to steer the sheep to the desired location.

Goal of the paper. Based on the above examples, our goal in this paper is to evaluate how such simultaneously collaborative and adversarial environment can be used in adding stabilization and fault-tolerance to a given program.

A preliminary version of this work appeared in [3]. In addition to the results presented in [3], in this paper, we provide a sound and complete algorithm for adding masking fault-tolerance. We also introduce our JavaRepair package that includes implementation of our algorithms in Java using symbolic techniques to support larger state spaces. We provide experimental results of using our proposed algorithms to solve different versions of the shepherding problem. To obtain these results, we have used our JavaRepair package. The main results of this work are as follows:

• We formalize definitions and problems statements for the addition of stabilization and fault-tolerance in presence of unchangeable environment actions.
• We present an algorithm for the addition of stabilization to an existing program. This algorithm is sound and complete, that is, the program found by it is guaranteed to be stabilizing and if it declares failure then it implies that adding stabilization to that program is impossible.
• We present a sound and complete algorithm for the addition of fail-safe fault-tolerance.
• We present a sound and complete algorithm for the addition of masking fault-tolerance.
• We show that the complexity of all algorithms presented in this paper is polynomial (in the state space of the program).
• We present our JavaRepair package that includes the implementation of the algorithms presented in this paper available at [4].
• We present experimental results of applying our algorithms for different examples.

Organization of the paper. This paper is organized as follows: in Section 2, we provide the definitions of a program, environment, specification, fault, fault-tolerance and stabilization. In Section 3 we define the problem of adding stabilization and propose an algorithm to solve that problem. In Section 4, as a case study, we illustrate how the proposed algorithm can be used for a smart grid controller. In Section 5, we define the problem of adding fault-tolerance and propose algorithms to add failsafe and masking fault-tolerance. In Section 6, we present our JavaRepir package and provide experimental results for several examples. In Section 7, we show how our proposed algorithms can be extended to solve related problems. In Section 8, we discuss related work. Finally, we make concluding remarks in Section 9.

2. Preliminaries

In this section, we define the notion of program, environment, specification, fault and fault-tolerance. The definition of the specification is based on the definition by Alpern and Schneider [5]. The definitions of fault and fault-tolerance are adapted from those by Arora and Gouda [6].

2.1. Program Design Model

We define a program in terms of its states and transitions. Intuitively, the state space of a program represents the set of all possible states that a program can be in. On the other hand, transitions specify how the program moves from one state to another.

Definition 1 (Program).

A program p is a tuple $\langle S_p,{\delta }_p\rangle$ where $S_p$ is the state space of program p and ${\delta }_p\subseteq S_p\times S_p$.

In addition to program transitions, the state of a program may change due to the environment where the program is running. Instead of modeling the environment in terms of concepts such as variables that are written by program and variables that are written by the environment, we use a more general approach which models it as a set of transitions that is a subset of $S_p\times S_p$.

Definition 2 (Environment).

An environment ${\delta }_e$ for program p, is defined as a subset of $S_p\times S_p$.

Our algorithms assume that from any state in the state space of a program p in an environment ${\delta }_e$, there is at least one transition in ${\delta }_p\cup {\delta }_e$. If there is no transition from state $s_0$ in ${\delta }_p\cup {\delta }_e$ of the given program, we add the self-loop transition $(s_0,s_0)$ to ${\delta }_p$. We note that this assumption is not restrictive. Instead, it is made to simplify subsequent definitions, since we do not need to concern with terminating computations separately.

We refer to a subset of states of a program by a state predicate. Thus, we have

Definition 3 (State Predicate).

A state predicate of p is any subset of $S_p$.

Note that we can represent a state predicate by either a subset of states or a predicate. Thus, there is a correspondence between a state predicate and subset of states where the state predicate is true. For example, state predicate $True$ is a subset that includes all states of a program and $False$ is an empty set. Similarly, the negation of a state predicate is the set of states where the given state predicate is false. We say a state predicate is closed in a set of transitions if no transition in the set leaves the state predicate.

Definition 4 (Closure).

A state predicate S is closed in a set of transitions δ iff $(\forall (s_0,s_1):(s_0,s_1)\in \delta :(s_0\in S\Rightarrow s_1\in S))$. (We use $Qx:D\left(x\right):P\left(x\right)$ to specify a formula where $Qx$ is a quantifier that is instantiated to be $\forall x$ or $\exists x$, $D\left(x\right)$ is a domain where x can come from and $P\left(x\right)$ is predicate over x. When there is no restriction on the domain, we use $Q::P$. For example, ‘$\exists x:$ x is odd: x is prime’ states that there exists an x in $\{1,3,5,\cdots \}$ that is a prime.)

Definition 5 (Projection).

The projection of a set of transition δ on state predicate S, denoted as $\delta |S$, is set $\{(s_0,s_1):(s_0,s_1)\in \delta \wedge s_0,s_1\in S\}\rangle$. In other words, $\delta |S$ consists of transitions of δ that start at S and end in S.

A computation of a program in an environment is a sequence of states that starts in an initial state and in each state executes either a program transition or an environment transition. Moreover, after an environment transition executes, the program is given a chance to execute in the next $k-1$ steps. However, whenever in a state no program transition is available, an environment transition can execute.

Definition 6 (p[]_kδ_e computation).

Let p be a program with state space $S_p$ and transitions ${\delta }_p$. Let ${\delta }_e$ be an environment for program p and k be an integer greater than 1. We say that a sequence $\sigma =\langle s_0,s_1,s_2,\dots \rangle$ is a $p{\left[\right]}_k{\delta }_e$ computation iff

• $\forall i:i\ge 0:s_i\in S_p$, and
• $\forall i:i\ge 0:(s_i,s_{i+1})\in {\delta }_p\cup {\delta }_e$, and
• $\forall i:i\ge 0:((s_i,s_{i+1})\in {\delta }_e)\Rightarrow$
  $(\forall l:i<l<i+k:(\exists s_l^{\prime }::(s_l,s_l^{\prime })\in {\delta }_p)\Rightarrow (s_l,s_{l+1})\in {\delta }_p))$.

2.2. Specification

Following Alpern and Schneider [5], we let the specification of the program consist of a safety specification and a liveness specification. The safety specification identifies bad things that the program should not do. We define a safety specification as a set of (bad) transitions. Specifically,

Definition 7 (Safety).

A safety property is specified in terms of a set of bad transitions, ${\delta }_b$, that the program is not allowed to execute. Thus, a sequence $\sigma =\langle s_0,s_1,\dots \rangle$ satisfies the safety property ${\delta }_b$ iff $\forall j:j>0:(s_j,s_{j+1})\notin {\delta }_b$.

Liveness, on the other hand, identifies good things that the program should do. We define a liveness property in terms of a set of leads-to properties. A leads-to property is of the form $L⇝T$ where L and T are two state predicates and the leads-to property requires that if the program reaches a state in L it eventually reaches a state in T. Specifically,

Definition 8 (Leads-to).

A leads-to property is specified in terms of $L⇝T$, where both L and T are state predicates. A sequence $\sigma =\langle s_0,s_1,\dots \rangle$ satisfies the leads-to property iff $\forall j:s_j\in L:(\exists k:k\ge j:s_k\in T)$.

Finally, a specification consists of a safety property and a liveness specification (in terms of a set of leads-to properties).

Definition 9 (Specification).

A specification $spec$, is a tuple $\langle Sf,Lv\rangle$, where $Sf$ is a safety property and $Lv$ is a set of leads-to properties. A sequence σ satisfies $spec$ iff it satisfies $Sf$ and $Lv$.

Definition 10 (Satisfaction).

Program p k-satisfies specification $spec$ from S in environment ${\delta }_e$ iff the following conditions hold:

• S is closed in ${\delta }_p\cup {\delta }_e$, and
• Every $p{\left[\right]}_k{\delta }_e$ computation that starts from a state in S satisfies $spec$.

We note that from the above definition, it follows that starting from a state in S, execution of either a program transition or an environment transition results in a state in S. Transitions that start from a state in S and reach a state outside S will be modeled as faults (cf. Definition 12). We define an invariant for a program as a subset of states such that if the program starts from that subset, it satisfies its required specification.

Definition 11 (Invariant).

State predicate S is an invariant of p for specification $spec$ iff

• $S\ne \varnothing$, and
• p k-satisfies $spec$ from S.

2.3. Faults and Fault-Tolerance

Like environment transitions, we define a class of faults for a program as a set of transitions that may change the state of the program:

Definition 12 (Fault).

A class of faults f for $p(=\langle S_p,{\delta }_p\rangle )$ is a subset of $S_p\times S_p$.

Now, we extend Definition 6 to one that captures the computation of a program in presence of fault transitions:

Definition 13 (p[]_kδ_e[]f computation).

Let p be a program with state space $S_p$ and transitions ${\delta }_p$. Let ${\delta }_e$ be an environment for program p, k be an integer greater than 1 and f be the set of faults for program p. We say that a sequence $\sigma =\langle s_0,s_1,s_2,\dots \rangle$ is a $p{\left[\right]}_k{\delta }_e\left[\right]f$ computation iff

• $\forall i:i\ge 0:s_i\in S_p$, and
• $\forall i:i\ge 0:(s_i,s_{i+1})\in {\delta }_p\cup {\delta }_e\cup f$, and
• $\forall i:i\ge 0:(s_i,s_{i+1})\in {\delta }_e\Rightarrow$
  $\forall l:i<l<i+k:(\exists s_l^{\prime }::(s_l,s_l^{\prime })\in {\delta }_p\Rightarrow (s_l,s_{l+1})\in ({\delta }_p\cup f))$, and
• $\exists n:n\ge 0:(\forall j:j>n:(s_{j-1},s_j)\in ({\delta }_p\cup {\delta }_e))$.

Note that in the above definition, we require that fault transitions finally stop occurring. This captures the transient nature of faults. On the other hand, environment transitions keep occurring forever. Fault transitions can perturb the program by arbitrarily changing its state. The definition of fault-span captures the boundary up to which the program could be perturbed by faults. Thus,

Definition 14 (Fault-span).

The state predicate T is a k-f-span of p from S in environment ${\delta }_e$ iff

• $S\Rightarrow T$, and
• for every $p{\left[\right]}_k{\delta }_e\left[\right]f$ computation $\langle s_0,s_1,s_2,\dots \rangle$, where $s_0\in S$, $\forall i:s_i\in T$.

Next, we define the notion of fault-tolerance. We consider two different types of fault-tolerance namely failsafe and masking fault-tolerance. A failsafe fault-tolerance ensures that the safety property of the desired specification (cf. Definition 9) is not violated even if faults occur. In other words, we have

Definition 15 (Failsafe fault-tolerance).

Program p is failsafe k-f-tolerant to specification $spec$ (=$\langle Sf,Lv\rangle$) from S in environment ${\delta }_e$ iff the following two conditions hold:

• p k-satisfies $spec$ from S in environment ${\delta }_e$ and
• any prefix of any $p{\left[\right]}_k{\delta }_e\left[\right]f$ computation that starts from S satisfies $Sf$.

Masking fault-tolerance is stronger than failsafe fault-tolerance, as in addition to satisfying the safety property, masking fault-tolerance requires recovery to the invariant. Specifically,

Definition 16 (Masking fault-tolerance).

p is masking k-f-tolerant to specification $spec$ from S in environment ${\delta }_e$ iff the following two conditions hold:

• p is failsafe k-f-tolerant to $spec$ (=$\langle Sf,Lv\rangle$) from S in environment ${\delta }_e$ and
• there exists T such that (1) T is an k-f-span of p from S in environment ${\delta }_e$ and (2) for every $p{\left[\right]}_k{\delta }_e\left[\right]f$ computation $\sigma (=\langle s_0,s_1,s_2,\dots \rangle )$ that starts from a state in S, for any i such that $s_i\in T$, then there exists $j\ge i$ such that $s_j\in S$.

The second constraint in Definition 16 simply means that in any computation that starts at T must go to S.

We also define the notion of stabilizing programs. We extend the definition from References [7] and [8] for a program in the presence of environment actions.

Definition 17 (Stabilization).

Program p is k-stabilizing for invariant S in environment ${\delta }_e$, iff following conditions hold:

• S is closed in ${\delta }_p\cup {\delta }_e$, and
• for any $p{\left[\right]}_k{\delta }_e$ computation $\langle s_0,s_1,s_2,\dots ..\rangle$ there exists l such that $s_l\in S$.

3. Addition of Stabilization

In this section, we present our algorithm for adding stabilization to an existing program. In Section 3.1, we identify the problem statement and in Section 3.2, we present an algorithm to add stabilization. Finally, in Section 3.3, we provide proofs of soundness, completeness and the complexity results of the proposed algorithm.

3.1. Problem Definition

The problem of adding of stabilization begins with a program $p=\langle S_p,{\delta }_p\rangle$, its invariant S and an environment ${\delta }_e$. The goal is to change the set of program transitions so that starting from an arbitrary state, the program recovers to S. In addition, we do not want to change the behavior of the program inside its invariant. Thus, during the addition, we are not allowed to change the set of program transitions in the invariant (i.e., ${\delta }_p|S$). Also, we introduce parameter ${\delta }_r$ that identifies additional restrictions on program transitions. As an example, consider the case where a program cannot change the value of a sensor, that is, it can only read it. However, the environment can change the value of the sensor. In this case, transitions that change the value of the sensor are disallowed as program transitions but they are acceptable as environment transitions. Thus, the problem statement is as follows:

Given program p with state space $S_p$ and transitions ${\delta }_p$, invariant S, environment ${\delta }_e$, set of program restriction ${\delta }_r$ and $k>1$, identify $p^{\prime }$ with state space $S_p$ such that: C1  $p^{\prime }|S=p|S$ C2  $p^{\prime }$ is k-stabilizing for invariant S in the environment ${\delta }_e$. C3  ${\delta }_p^{\prime }\cap {\delta }_r=\varnothing$

3.2. Algorithm to Add Stabilization

The algorithm proposed here adds stabilization for the case where $k=2$. When $k=2$, environment transitions can execute immediately after any program transition. By contrast, for larger k, the environment transitions may have to wait until the program has executed $k-1$ transitions.

The procedure for adding stabilization is as shown in Algorithm 1. In this algorithm, ${\delta }_p^{\prime }$ is the set of transitions of the final stabilizing program. Inside the invariant, the transitions must be equal to the original program (Constraint C1). Therefore, in the first line, we set ${\delta }_p^{\prime }$ to ${\delta }_p|S$. Next, the algorithm expands set R that includes states from which all computations reach a state in S. Initially, R is set to S at (Line 2).

State predicate $R_p$ is the set of states that can reach a state in R using an unrestricted program transition, that is, a transition not in ${\delta }_r$. In each iteration, $R_p^{\prime }$ is the set of new states that we add to $R_p$. In Line 9, we add program transitions from states in $R_p^{\prime }$ to states in R to ${\delta }_p^{\prime }$.

Algorithm 1 Addition of stabilization

Input:  $S_p,{\delta }_p,{\delta }_e,S,$ and ${\delta }_r$ Output:  ${\delta }_p^{\prime }$ or Not-Possible 1: ${\delta }_p^{\prime }:=\left({\delta }_p\right|S);$ 2: $R=S;$ 3: $R_p=\left\{\right\}$ 4: repeat 5:  $R^{\prime }=R;$ 6:  $R_p^{\prime }=\left\{s_0\right|s_0\notin (R\cup R_p)\wedge \exists s_1:s_1\in R:(s_0,s_1)\notin {\delta }_r\};$ 7:  $R_p=R_p\cup R_p^{\prime }$; 8:  for $each{s}_0\in R_p^{\prime }$ do 9:   ${\delta }_p^{\prime }={\delta }_p^{\prime }\cup \left\{(s_0,s_1)\right|(s_0,s_1)\notin {\delta }_r\wedge s_1\in R\};$ 10:  end for 11:  for $$\begin{gathered} each{s}_0\notin R:\nexists s_2\in \neg (R\cup R_p):(s_0,s_2)\in {\delta }_e\wedge \\ (\exists s_1:s_1\in (R\cup R_p):(s_0,s_1)\in {\delta }_e\vee s_0\in R_p) \end{gathered}$$ do 12:   $R=R\cup s_0;$ 13:  end for 14: until$(R^{\prime }=R);$ 15: if$\exists s_0\notin R$then 16:  return Not-Possible; 17: else 18:  return ${\delta }_p^{\prime };$ 19: end if

In the loop on Lines 11–13, we add more states to R. We add $s_0$ to R (Line 12), whenever every computation starting from $s_0$ has a state in S. A state $s_0$ can be added to R only when there is no environment transition starting from $s_0$ going to a state outside $R\cup R_p$. In addition to this condition, there must be at least one transition from $s_0$ that reaches R. The loop on Lines 4–14 terminates if no state is added to R in an iteration. Upon termination of the loop, the algorithm declares failure to add stabilization if there exists a state outside R. Otherwise, it returns ${\delta }_p^{\prime }$ as the set of transitions of the stabilizing program.

We use Figure 3 as an example to illustrate Algorithm 1. Figure 3 depicts the status of the state space in a hypothetical ith iteration of the loop on Lines 4–14. In this iteration, state A is added to R. This is due to the fact that (1) there is at least one transition from A (namely $(\mathbf{A},\mathbf{F})$) that reaches R and (2) there is no environment transition from A that reaches outside $R\cup R_p$. Likewise, state C is also added to R. State B is not added to R due to environment transition $(\mathbf{B},\mathbf{E})$. Likewise, state D is not added to R. State E is not added to R since there is no transition from E to a state in R.

In the next, that is, $(i+1)$th iteration, E is added to R due to transition $(\mathbf{E},\mathbf{A})$ that goes to A that was added to R in the ith iteration. Continuing this, B and D will be added in the $(i+2)$th iteration.

3.3. Soundness, Completeness and Complexity Results

In this Section, we show that Algorithm 1 is sound and complete and its time complexity is polynomial in the size of the state space.

Soundness: First, we show that Algorithm 1 is sound, that is, if it finds a solution, the solution satisfies the problem statement for adding stabilization provided in Section 3.1.

Based on the notion of fairness for program transitions, we introduce the notion of whether an environment transition can execute in a given computation prefix. An environment transition can execute in a computation prefix if an environment transition exists in the last state of the prefix and either (1) there are no program transitions starting from the last state of the prefix or (2) the last $k-1$ transitions of the prefix are program transitions.

Definition 18 (Environment-enabled).

In any prefix of any $p{\left[\right]}_k{\delta }_e$ computation $\sigma =\langle s_0,s_1,\dots ,s_i\rangle$, $s_i$ is an environment-enabled state iff

$$\begin{gathered} (\exists s::(s_i,s)\in {\delta }_e)\wedge \\ \left(\left(\nexists (s_i,s^{\prime })::(s_i,s^{\prime })\in {\delta }_p\right)\vee \\ \left(\nexists j:j>i-k:(s_j,s_{j+1})\in {\delta }_e\right)\right) \end{gathered}$$.

The following lemma focuses on the recovery to S from states in R:

Lemma 1.

Any $p^{\prime }{\left[\right]}_k{\delta }_e$ computation that starts from a state in R, contains a state in S.

Proof. 

We prove this by induction.

Base case: $R=S$. The statement is satisfied trivially.

Induction hypothesis: Theorem holds for the current set of R.

Induction step: We show when we add a state to R, the theorem still hold for R. A state $s_0$ is added to R in two cases:

Case 1$(\nexists s_2:s_2\in \neg (R\cup R_p):(s_0,s_2)\in {\delta }_e)\wedge (\exists s_1:s_1\in (R\cup R_p):(s_0,s_1)\in {\delta }_e)$

Since there is no $s_2$ in $\neg (R\cup R_p)$ such that $(s_0,s_2)\in {\delta }_e$, for every $(s_0,s_1)\in {\delta }_e$, $s_1$ is in $R\cup R_p$. In addition, we know that there is at least one $s_1$ in $R\cup R_p$ such that $(s_0,s_1)\in {\delta }_e$. By the construction of $R_p$, we know that if $s_1$ is in $R_p$, there is a program transition from $s_1$ to a state in R. Since $(s_0,s_1)\in {\delta }_e$ and because of the fairness assumption, the program can occur and reach R. Thus, every computation starting from $s_0$ has a state in R. Since we do not change the set of transitions of states in R of the previous iteration, the set of computations starting from any state in R is unchanged. Thus, based on the induction hypothesis, every computation starting from $s_0$ has a state in S.

Case 2$\nexists s_2:s_2\in \neg (R\cup R_p):(s_0,s_2)\in {\delta }_e\wedge s_0\in R_p$

Since there is no $s_2$ in $\neg (R\cup R_p)$ such that $(s_0,s_2)\in {\delta }_e$, for every $(s_0,s_1)\in {\delta }_e$, $s_1$ is either in R or $R_p$. In addition, we know that there is at least one state $s_3$ in R such that $(s_0,s_3)\in {\delta }_p$. In any $p^{\prime }{\left[\right]}_k{\delta }_e$ computation starting from $s_0$ if $(s_0,s_1)\in {\delta }_p^{\prime }$ then $s_1\in R$. If $(s_0,s_1)\in {\delta }_e$ then $s_1\in R\cup R_p$. By construction of $R_p$, we know that if $s_1$ is in $R_p$, there is a program transition from $s_1$ to a state in R. Since $(s_0,s_1)\in {\delta }_e$ and because of the fairness assumption, program can reach R. Thus, every computation starting from $s_0$ has a state in R. Since we do not change the set of transitions of states in R of the previous iteration, the set of computations starting from any state in R is unchanged. Thus, based on the induction hypothesis, every computation starting from $s_0$ has a state in S. □

Theorem 1.

Algorithm 1 is sound.

Proof. 

We need to show that the constraints of the problem definition are satisfied. At the beginning of the algorithm ${\delta }_p^{\prime }={\delta }_p|S$ and all other transitions added to ${\delta }_p^{\prime }$ in the rest of the algorithm starts outside S. Thus, $p^{\prime }|S=p|S$ that satisfies the first constraint. The second constraint is satisfied based on Lemma 1 and the fact that R includes all states. The third constraint is satisfied, as we do not add any transitions in ${\delta }_r$ to the program. □

Completeness: Next, we focus on showing that Algorithm 1 is complete, that is, if there is a solution that satisfies the problem statement for adding stabilization, Algorithm 1 finds one. The proof of completeness is based on the analysis of states that are not in R upon termination.

Any state that is not in R at the end of Algorithm 1, either does not have any environment transition, or it has an environment transition that goes to $\neg (R\cup R_p)$. Thus, we note the following observation:

Observation 1.

For any $s_0$ such that $s_0\notin R$ and $\exists s_1::(s_0,s_1)\in {\delta }_e$, we have $\exists s_2:s_2\in \neg (R\cup R_p):(s_0,s_2)\in {\delta }_e$.

Also, if a state is in $R_p$ but it is not in R, it has an environment transition to $\neg (R\cup R_p)$.

Observation 2.

For any $s_0\in \neg R\cap R_p$, $\exists s_1:s_1\in \neg (R\cup R_p):(s_0,s_1)\in {\delta }_e$.

Now, for the rest of our discussion in this section, we assume that Algorithm 1 has returned failure. The following lemma focuses on the situation where a given revision $p^{″}$ reaches a state marked as $\neg (R\cup R_p)$ by Algorithm 1.

Lemma 2.

Let ${\delta }_p^{″}$ be any program such that ${\delta }_p^{″}\cap {\delta }_r=\varnothing$. Let $s_j$ be any state in $\neg (R\cup R_p)$ at the end of Algorithm 1. Then, for every $p^{″}{\left[\right]}_k{\delta }_e$ prefix $\alpha =\langle \dots ,s_{j-1},s_j\rangle$, there exists suffix $\beta =\langle s_{j+1},s_{j+2},\dots \rangle$, such that $\alpha \beta$ is a $p^{″}{\left[\right]}_k{\delta }_e$ computation and one of two conditions below is correct:

1. 

$s_{j+1}\in \neg (R\cup R_p)$

2. 

$s_{j+1}\in (R_p-R)\wedge s_{j+2}\in \neg (R\cup R_p)$

Proof. 

There are two cases for $s_j$:

Case 1 If $s_j$ is environment-enabled in prefix $\alpha =\langle ...,s_{j-1},s_j\rangle$

According to the Observation 1, since $s_j\in \neg (R\cup R_p)$, there exists $s^{″}\in \neg (R\cup R_p)$ such that $(s_j,s^{″})\in {\delta }_e$. We set $s_{j+1}=s^{″}$.

Case 2 If $s_j$ is not environment-enabled in $\alpha =\langle ...,s_{j-1},s_j\rangle$

In this case $(s_j,s_{j+1})\in {\delta }_p$ and as $s_j\in \neg (R\cup R_p)$, $s_{j+1}\in \neg R$ (otherwise, $s_j$ would be included in $R_p$, as we can reach state $s_{j+1}\in R$ with a transition which is not in ${\delta }_r$). There are two sub-cases for this case:

Case 2.1$s_{j+1}\in \neg R_p$

In this case $s_{j+1}\in \neg (R\cup R_p)$.

Case 2.2$s_{j+1}\in R_p$

As $s_{j+1}\in \neg R\cap R_p$, according to Observation 2, we have $\exists s_2:s_2\in \neg (R\cup R_p):(s_{j+1},s_2)\in {\delta }_e$. As $(s_j,s_{j+1})\in {\delta }_p$, even with fairness $(s_{j+1},s_2)$ can occur. Therefore we set $s_{j+2}=s_2$, that is, $s_{j+2}\in \neg (R\cup R_p)$. □

From this lemma, we conclude that for any given program $p^{″}$, if a computation ever reaches a state in $\neg (R\cup R_p)$, there is a computation from that state that never reaches R. Specifically, we have the following corollary:

Corollary 1.

Let ${\delta }_p^{″}$ be any program such that ${\delta }_p^{″}\cap {\delta }_r=\varnothing$. Let $s_j$ be any state in $\neg (R\cup R_p)$ at the end of Algorithm 1. Then for every $p^{″}{\left[\right]}_k{\delta }_e$ prefix $\alpha =\langle \dots ,s_{j-1},s_j\rangle$, there exists suffix $\beta =\langle s_{j+1},s_{j+2},\dots \rangle$ (possibly $\langle \rangle$), such that $\alpha \beta$ is a $p^{″}{\left[\right]}_k{\delta }_e$ computation and $\forall i:i\ge j:s_i\in \neg R$ (i.e., $\neg S$).

Theorem 2.

Algorithm 1 is complete.

Proof. 

Suppose program $p^{″}$ solve the addition problem. Algorithm 1 returns Not-possible only when, at the end of loop on Lines 4–14 there exists a state $s_0$ such that $s_0\notin R$. When $s_0\notin R$, we have two cases as follows:

Case 1$\exists s_2:s_2\in \neg (R\cup R_p):(s_0,s_2)\in {\delta }_e$

As there exists an environment action to state $s_2$ in $\neg (R\cup R_p)$, starting from $s_0$ there is a computation such that the next state after $s_0$ is in $\neg (R\cup R_p)$. Note that, when a computation starts from $s_0$, even with the fairness assumption $(s_0,s_2)\in {\delta }_e$ can occur. Based on Corollary 1, for every ${\delta }_p^{″}$ such that ${\delta }_p^{″}\cap {\delta }_r=\varnothing$, starting from $s_0$, there is a computation such that all of its states are outside R (i.e., outside s).

Case 2$\nexists s_1:s_1\in (R\cup R_p):(s_0,s_1)\in {\delta }_e\wedge s_0\notin R_p$

Based on Corollary 1, starting from $s_0\in \neg (R\cup R_p)$, there is a computation such that every state is in $\neg R$. Therefore, for every ${\delta }_p^{″}$ such that ${\delta }_p^{″}\cap {\delta }_r=\varnothing$, starting from $s_0$, there is a computation such that all of its states are outside R (i.e., outside s).

Thus in both cases, $p^{″}$ has a computation that never reaches the invariant (contradiction). □

Time complexity: Finally, regarding the time complexity of Algorithm 1 we have the following theorem:

Theorem 3.

Time complexity of Algorithm 1 is polynomial (in the size of state space of p).

Proof. 

The proof follows from the fact that each statement in Algorithm 1 is executed in polynomial time and the number of iterations is also polynomial, as in each iteration at least one state is added to R. □

4. Case Study: Stabilization of a Smart Grid

In this section, we illustrate how Algorithm 1 can be used for adding stabilization to a controller program of a smart grid. We consider an abstract version of the smart grid described in [9] (see Figure 4). In this example, the system consists of a generator G and two loads $Z_1$ and $Z_2$. There are three sensors in the system. Sensor G shows the power generated by the generator and sensors 1 and 2 show the demand of load $Z_1$ and $Z_2$, respectively. The goal is to ensure that proper load shading is used if the load is too high (respectively, generating capacity is too low).

The control center is shown by a dashed circle in Figure 4. It can read the values of the sensors and turn on/off switches connected to the loads. The program of the control center has to control switches in a manner that all the conditions below are satisfied:

• Both switches must be turned on if the overall sensed load is less than or equal to the generation capacity.
• If sensor values reveal that neither load can individually be served by G then both are shed.
• If only one load can be served then the smaller load is shed assuming the larger load can be served by G.
• If only one can be served and the larger load exceeds the generation capacity, the smaller load is served.

4.1. Program Model

We model the program of the smart grid shown in Figure 4 by program p which has five variables as follows:

• $V_G:$ The value of sensor G.
• $V_1:$ The value of sensor 1.
• $V_2:$ The value of sensor 2.
• $w_1:$ The status of switch 1.
• $w_2:$ The status of switch 2.

The value of each sensor is an integer in the range $[0,max]$. The status of each switch is a Boolean. The invariant S for this program includes all the states which are legitimate according to the conditions 1-4 mentioned above. Therefore, S is the union of state predicates $I_1$ to $I_6$ as follows (We need to add $0\le V_1,V_2,V_g\le max$ to all conditions. For brevity, we keep these implicit.):

• $I_1=(V_1+V_1\le V_G)\wedge (w_1\wedge w_2)$
• $I_2=V_1\le V_G\wedge V_2>V_G)\wedge (w_1\wedge \neg w_2)$
• $I_3=(V_1>V_G\wedge V_2\le V_G)\wedge (\neg w_1\wedge w_2)$
• $I_4=(V_1>V_G\wedge V_2>V_G)\wedge (\neg w_1\wedge \neg w_2)$
• $I_5=(V_1+V_2>V_G\wedge V_1\le V_G\wedge V_2\le V_G\wedge V_1\le V_2)\wedge (\neg w_1\wedge w_2)$
• $I_6=(V_1+V_2>V_G\wedge V_1\le V_G\wedge V_2\le V_G\wedge V_1>V_2)\wedge (w_1\wedge \neg w_2)$

We note the following observation about states in S:

Observation 3.

For any value of $V_1$, $V_2$ and $V_G$, there exists an assignment to $w_1$ and $w_2$ that changes the state to a state is in S.

The environment can change the values of sensors 1 and 2. In addition, environment can keep the current value of a sensor by self-loop environment transitions. However, environment cannot change the status of switches, change the generated load, or leave the invariant. Thus, the set of environment transitions ${\delta }_e$ is equal to $\left\{(s_0,s_1)\right|\left(w_1\left(s_0\right)=w_1\left(s_1\right)\right)\wedge \left(w_2\left(s_0\right)=w_2\left(s_1\right)\right)\wedge \left(V_G\left(s_0\right)=V_G\left(s_1\right)\right)\wedge \left({\cap }_{i=1}^6{I}_i\left(s_0\right)\Rightarrow {\cap }_{i=1}^6{I}_i\left(s_1\right)\right)\}$, where $v\left(s_j\right)$ shows the value of the variable or predicate v in state $s_j$.

Program cannot change the value of any sensor. Thus, set of program restrictions for this program is ${\delta }_r=\left\{(s_0,s_1)\right|V_G\left(s_0\right)\ne V_G\left(s_1\right)\vee V_1\left(s_0\right)\ne V_1\left(s_1\right)\vee V_2\left(s_0\right)\ne V_2\left(s_1\right)\}$.

For the sake of presentation, we also consider the case where that program cannot change the status of more than one switch in one transition. For this case, we add more transitions to the set of program restrictions. We call the set of program restrictions for this case ${\delta }_{r_2}$ and it is equal to $\left\{(s_0,s_1)\right|V_G\left(s_0\right)\ne V_G\left(s_1\right)\vee V_1\left(s_0\right)\ne V_1\left(s_1\right)\vee V_2\left(s_0\right)\ne V_2\left(s_1\right)\vee (w_1\left(s_0\right)\ne w_1\left(s_1\right)\wedge w_2\left(s_0\right)\ne w_2\left(s_1\right))\}$.

4.2. Addition of Stabilization

Here, we apply Algorithm 1 to add stabilization to program p defined in Section 4.1. We illustrate the result of applying Algorithm 1 for two sets of program restrictions, ${\delta }_{r_1}$ and ${\delta }_{r_2}$.

4.2.1. Adding Stabilization for ${\delta }_{r_1}$

At the beginning of Algorithm 1, R is initialized with S. In the first iteration of loop on Lines 4–14, $R_p$ is the set of states outside S that can reach a state in S with only one program transition. A program transition cannot change the value of any sensor. According to Observation 3, from each state in $\neg S$ it is possible to reach a state in S with changing the status of switches. Therefore, the following set of transitions are added to ${\delta }_p^{\prime }$ by Line 9:

$$\left\{(s_0,s_1)\right|V_1\left(s_0\right)=V_1\left(s_1\right)\wedge V_2\left(s_0\right)=V_2\left(s_1\right)\wedge V_G\left(s_0\right)=V_G\left(s_1\right)\wedge s_0\notin {\cup }_{i=1}^6{I}_i\wedge s_1\in {\cup }_{i=1}^6{I}_i\}$$

Since every state in $\neg S$ ($\neg R$) is in $R_p$, there does not exist any environment transition starting from any state to a state in $\neg (R\cup R_p)$. Therefore, all the states in $\neg R$ are added to R by Line 12. In the second iteration, no more states are added to R. Thus, loop on Lines 4–14 terminates. Since there is no state in $\neg R$, the algorithm returns ${\delta }_p^{\prime }$.

4.2.2. Adding Stabilization for ${\delta }_{r_2}$

At the beginning of Algorithm 1, R is initialized with S. In the first iteration of loop on Lines 4–14, $R_p$ is the set of states outside S that can reach a state in S with only one program transition. A program transition cannot change the value of any sensor. In addition, according to ${\delta }_{r_2}$, it cannot change the status of both switches simultaneously. Therefore, state predicate $R_p$ is the union of state predicates $R_{p_1}$ to $R_{p_6}$ as follows (⊕ denotes the xor operation):

• $R_{p_1}=(V_1+V_1\le V_G)\wedge (w_1\oplus w_2)$
• $R_{p_2}=(V_1\le V_G\wedge V_2>V_G)\wedge (w_1\oplus \neg w_2)$
• $R_{p_3}=(V_1>V_G\wedge V_2\le V_G)\wedge (\neg w_1\oplus w_2)$
• $R_{p_4}=(V_1>V_G\wedge V_2>V_G)\wedge (\neg w_1\oplus \neg w_2)$
• $R_{p_5}=(V_1+V_2>V_G\wedge V_1\le V_G\wedge V_2\le V_G\wedge V_1\le V_2)\wedge (\neg w_1\oplus w_2)$
• $R_{p_6}=(V_1+V_2>V_G\wedge V_1\le V_G\wedge V_2\le V_G\wedge V_1>V_2)\wedge (w_1\oplus \neg w_2)$

Similarly, $\neg (R\cup R_p)$ includes every state that is outside S and more than one step are needed to reach a state in S. Therefore, state predicate $\neg (R\cup R_p)$ is the union of state predicates $R_{p_1}^{\prime }$ to $R_{p_6}^{\prime }$ as follows:

• $R_{p_1}^{\prime }=(V_1+V_1\le V_G)\wedge (\neg w_1\wedge \neg w_2)$
• $R_{p_2}^{\prime }=(V_1\le V_G\wedge V_2>V_G)\wedge (\neg w_1\wedge w_2)$
• $R_{p_3}^{\prime }=(V_1>V_G\wedge V_2\le V_G)\wedge (w_1\wedge \neg w_2)$
• $R_{p_4}^{\prime }=(V_1>V_G\wedge V_2>V_G)\wedge (w_1\wedge w_2)$
• $R_{p_5}^{\prime }=(V_1+V_2>V_G\wedge V_1\le V_G\wedge V_2\le V_G\wedge V_1\le V_2)\wedge (w_1\wedge \neg w_2)$
• $R_{p_6}^{\prime }=(V_1+V_2>V_G\wedge V_1\le V_G\wedge V_2\le V_G\wedge V_1>V_2)\wedge (\neg w_1\wedge w_2)$

Now, observe that from any state in $R_p$, it is possible to reach a state in $\neg (R\cup R_p)$ by an environment transition. Therefore, no state is added to R in the first iteration and loop on Lines 4–14 terminate in the first iteration. Since all the states outside S remain in $\neg R$, the algorithm declares that there is no solution to the addition problem. Therefore, according to the completeness of Algorithm 1, there does not exist a 2-stabilizing program for the smart grid described in this section when the set of program restriction is ${\delta }_{r_2}$.

5. Addition of Fault-Tolerance

In this section, we present our algorithm for adding failsafe and masking fault-tolerance. In Section 5.1, we identify the problem statement for adding fault-tolerance. In Section 5.2, we present an algorithm for adding failsafe fault-tolerance. The proofs of the soundness and completeness and the complexity results for this algorithm are provided in Appendix A. In Section 5.3, we present an algorithm for adding masking fault-tolerance. The proofs of the soundness and completeness and the complexity results for this algorithm are provided in Appendix B.

5.1. Problem Definition

The problem statement for the addition of fault-tolerance is as follows:

Given p, ${\delta }_e$, S, $spec$, set of program restrictions ${\delta }_r$, $k>1$ and f such that p k-satisfies $spec$ from S in environment ${\delta }_e$ and ${\delta }_p\cap {\delta }_r=\varnothing$, identify $p^{\prime }$ and invariant $S^{\prime }$ such that: C1  every computation of $p^{\prime }{\left[\right]}_k{\delta }_e$ that starts in a state in $S^{\prime }$ is a computation of $p{\left[\right]}_k{\delta }_e$ that starts in S and C2  $p^{\prime }$ is (failsafe or masking) k-f-tolerant to $spec$ from $S^{\prime }$ in environment ${\delta }_e$ and C3  ${\delta }_p^{\prime }\cap {\delta }_r=\varnothing$

The problem statement requires that the program does not introduce new behaviors in the absence of faults (Constraint C1), provides desired fault-tolerance (Constraint C2) and does not include any transition in ${\delta }_r$ (Constraint C3).

5.2. Algorithm to Add Failsafe Fault-Tolerance

The procedure for adding failsafe fault-tolerance for $k=2$ is shown in Algorithm 2. In this algorithm, set $m{s}_1$ is the set of states from which there exists a computation suffix that violates the safety of $spec$. $m{s}_2$ is the set of states such that, if they are reached by a program or fault transition, or starting from them, there exists a computation suffix that violates safety. Note that $m{s}_2$ always includes $m{s}_1$. First, $m{s}_1$ is initialized to $\left\{s_0\right|(s_0,s_1)\in f\cap {\delta }_b\}$ and $m{s}_2$ is initialized to $m{s}_1\cup \left\{s_0\right|\exists s_1::(s_0,s_1)\in {\delta }_e\cap {\delta }_b\}$ by Lines 1 and 2, respectively. Set $mt$ is the set of transitions that the final program cannot have, as they are in ${\delta }_b\cup {\delta }_r$, or reach a state in $m{s}_2$.

Algorithm 2 Adding Failsafe Fault-Tolerance

Input:  $S_p,{\delta }_p,{\delta }_e,S,{\delta }_b$, ${\delta }_r$ and f Output:  $(S^{\prime },{\delta }_p^{\prime })$ or Not-Possible 1: $m{s}_1=\left\{s_0\right|\exists s_1::(s_0,s_1)\in f\cap {\delta }_b\};$ 2: $m{s}_2=m{s}_1\cup \left\{s_0\right|\exists s_1::(s_0,s_1)\in {\delta }_e\cap {\delta }_b\};$ 3: $mt=\left\{(s_0,s_1)\right|(s_0,s_1)\in ({\delta }_b\cup {\delta }_r)\vee s_1\in m{s}_2\};$ 4: repeat 5:  $m{s}_1^{\prime }=m{s}_1;$ 6:  $m{s}_2^{\prime }=m{s}_2;$ 7:  $$\begin{gathered} m{s}_1=m{s}_1\cup \left\{s_0\right|\exists s_1:s_1\in m{s}_2:(s_0,s_1)\in f\}\cup \left\{s_0\right|(\exists s_1::(s_1\in m{s}_1\wedge (s_0,s_1)\in {\delta }_e)\vee (s_0,s_1)\in \\ {\delta }_e\cap {\delta }_b))\wedge (\forall s_2::(s_0,s_2)\in mt)\}; \end{gathered}$$ 8:  $m{s}_2=m{s}_2\cup m{s}_1\cup \left\{s_0\right|\exists s_1:s_1\in m{s}_1:(s_0,s_1)\in {\delta }_e)\};$ 9:  $mt=\left\{(s_0,s_1)\right|(s_0,s_1)\in ({\delta }_b\cup {\delta }_r)\vee s_1\in m{s}_2\};$ 10: until$(m{s}_1^{\prime }=m{s}_1\wedge m{s}_2^{\prime }=m{s}_2)$ 11: ${\delta }_p^{\prime }={\delta }_p|S-mt;$ 12: $S^{\prime },{\delta }_p^{\prime }=ClosureAndDeadlocks(S-m{s}_2,{\delta }_p^{\prime },{\delta }_e);$ 13: repeat 14:  if $S^{\prime }=\varnothing$ then 15:   return Not-Possible; 16:  end if 17:  $S^{″}=S^{\prime };$ 18:  $m{s}_3=\left\{s_0\right|\left(\exists s_1,s_2::(s_0,s_1)\in {\delta }_e\wedge (s_0,s_2)\in {\delta }_p\right)\wedge$$\left(\nexists s_3::(s_0,s_3)\in {\delta }_p^{\prime }\right)\};$ 19:  $m{s}_4=\left\{s_0\right|\exists s_1::(s_1\in m{s}_3\wedge (s_0,s_1)\in {\delta }_e)\}$ 20:  $S^{\prime },{\delta }_p^{\prime }=ClosureAndDeadlocks(S-m{s}_4,{\delta }_p^{\prime },{\delta }_e);$ 21: until ($S^{″}=S^{\prime }$) 22: ${\delta }_p^{\prime }=\left({\delta }_p^{\prime }\cup \left((S_p-S^{\prime })\times S_p)\right)\right)-mt;$ 23: return$(S^{\prime },{\delta }_p^{\prime })$;   24: $ClosureAndDeadlocks(S,{\delta }_p,{\delta }_e)$ 25:  repeat 26:   $S^{\prime }=S;$ 27:   ${\delta }_p^{\prime }={\delta }_p;$ 28:   $S=S-\left\{s_0\right|(\forall s_1:s_1\in S:(s_0,s_1)\notin {\delta }_p\cup {\delta }_e)\};$ 29:   $S=S-\left\{s_0\right|\exists s_1::(s_0,s_1)\in {\delta }_e\wedge s_0\in S\wedge s_1\notin S\};$ 30:   ${\delta }_p={\delta }_p-\{(s_0,s_1)::s_0\in S\wedge s_1\notin S\};$ 31:  until $(S^{\prime }=S)\wedge ({\delta }_p^{\prime }={\delta }_p)$ 32:  return $S,{\delta }_p;$

In the loop on Lines 4–10, more states are added to $m{s}_1$ and $m{s}_2$. Consequently, we update $mt$. Any state $s_0$ is added to $m{s}_1$ by Line 7 in two cases: (1) if there exists a fault transition starting from $s_0$ that reaches a state in $m{s}_2$, or (2) there exists an environment transition $(s_0,s_1)$ such that $(s_0,s_1)$ is a bad transition or $s_1\in m{s}_1$ and any other transition starting from $m{s}_1$ reaches a state in $m{s}_2$ (i.e., any transition $(s_0,s_2)\in mt$).

A state is added to $m{s}_2$ by Line 8 if it is added to $m{s}_1$ or if there exists an environment transition to a state in $m{s}_1$. We update $mt$ by Line 9 to include transitions to new states added to $m{s}_2$. The loop on Lines 4–10 terminates if no state is added to $m{s}_1$ or $m{s}_2$ in an iteration.

Then, we focus on creating new invariant, $S^{\prime }$, for the revised program. $S^{\prime }$ cannot include any transition in $m{s}_2$, as starting from any state in $m{s}_2$, there is a computation which violates the safety. In addition, the set of program transitions of the revised program, ${\delta }_p^{\prime }$, cannot include any transition in $mt$, as by any transition in $mt$ a state in $m{s}_2$ is reached. Thus, we initialized ${\delta }_p^{\prime }$ with ${\delta }_p|S-mt$. Note that $S^{\prime }$ must be closed in $p^{\prime }\cup {\delta }_e$ and it cannot include any deadlock state. (Note, as we said in Section 2, for original deadlock states we have added self-loops. Thus, any deadlock state at this point is created because of removing transitions. Having these deadlock state in the invariant create new behavior inside invariant which is not acceptable.)

Thus, whenever we remove a state from $S^{\prime }$ we ensure that the $S^{\prime }$ is closed in $p^{\prime }\cup {\delta }_e$ and does not include any deadlock state by Line 12.

To satisfy condition C1, in the loop on Lines 13–21, we remove certain states that cause new computations for $p^{\prime }$ that are not computations of p. Suppose starting from $s_0$ there exists environment transition $(s_0,s_1)$. In addition, there exists a program transition $(s_0,s_2)$ in the set of program transitions of the original program, ${\delta }_p$. Set $m{s}_3$ includes any state like $s_0$. If $s_0$ is reached by environment transition $(s_3,s_0)$, in the original program, according to the fairness assumption, $(s_0,s_1)$ cannot occur. Thus, sequence $\langle s_3,s_0,s_1\rangle$ cannot be in any $p{\left[\right]}_2{\delta }_e$ computation. However, if we remove program transition $(s_0,s_2)$ in the revised program, $\langle s_3,s_0,s_1\rangle$ can be in a $p^{\prime }{\left[\right]}_2{\delta }_e$ computation. Therefore, we have to remove any state like $s_3$ (i.e., all states in $m{s}_4$ from the invariant (Line 20)).

After creating invariant $S^{\prime }$, we add program transitions outside it to ${\delta }_p^{\prime }$. Note that outside $S^{\prime }$, any program transition which is not in $mt$ is allowed to be in the final program. In Line 15, the algorithm declares that no solution to the addition problem exists, if $S^{\prime }$ is empty. Otherwise, at the end of the algorithm, it returns $({\delta }_p^{\prime },S^{\prime })$ as the solution to the addition problem.

5.3. Algorithm to Add Masking Fault-Tolerance

In this section, we present an algorithm for adding masking fault-tolerance in the presence of unchangeable environment actions. The intuition behind this algorithm is as follows: first, we utilize the ideas from adding stabilization. Intuitively, in Algorithm 1, we constructed the set R from where recovery to the invariant (S) was possible. In the case of stabilization, we wanted to ensure that R includes all states. However, for masking fault-tolerance recovery from all states is not necessary. We also need to ensure that recovery from R is not prevented by faults. This may require us to prevent the program from reaching some states in R. Hence, this process needs to be repeated to identify a set R such that recovery to S is provided and faults do not cause the program to reach a state outside R. In addition, in masking fault-tolerance, like failsafe fault-tolerance, the program must satisfy safety of specification even in presence of faults. Thus, the details of Algorithm 3 are as follows.

We start with an invariant equal to the original invariant (i.e., $S^{\prime }=S$). Like Algorithm 1, we set ${\delta }_p^{\prime }$ to ${\delta }_p|S$ which is the set of all program transitions inside the invariant. Like in the case of Algorithm 2, set $m{s}_1$ includes all states such that no matter how they are reached, there is a computation that is not desirable. In the case of failsafe fault-tolerance, such computation violates safety. On the other hand, here in case of masking fault-tolerant, such computation either violates safety or never reaches the invariant. $m{s}_2$ is the set of states such that, if they are reached by a program or fault transition, or starting from them, there exists a computation suffix which either violates safety or never reaches the invariant. Same as Algorithm 2, set $mt$ always includes all transitions to states in $m{s}_2$. We initialize sets $m{s}_1$, $m{s}_2$ and $mt$ as done in Algorithm 2 and expand them in the loop on Lines 6–43. In this loop, we use Algorithm 1 and Algorithm 2 with some modification.

Algorithm 3 Adding Masking Fault-Tolerance

Input:  $S_p,{\delta }_p,{\delta }_e,S,{\delta }_b$, ${\delta }_r$, and f Output:  $(S^{\prime },{\delta }_p^{\prime })$ or Not-Possible 1: $S^{\prime }=S;$ 2: ${\delta }_p^{\prime }=\left({\delta }_p\right|S);$ 3: $m{s}_1=\left\{s_0\right|(s_0,s_1)\in f\cap {\delta }_b\};$ 4: $m{s}_2=m{s}_1\cup \left\{s_0\right|\exists s_1:(s_0,s_1)\in {\delta }_e\cap {\delta }_b\};$ 5: $mt=\left\{(s_0,s_1)\right|(s_0,s_1)\in ({\delta }_b\cup {\delta }_r)\vee s_1\in m{s}_2\};$ 6: repeat 7:  $m{s}_1^{\prime }=m{s}_1;$ 8:  $m{s}_2^{\prime }=m{s}_2;$ 9:  $R=S^{\prime };$ 10:  $S^{″}=S^{\prime };$ 11:  $R_p=\left\{\right\};$ 12:  repeat 13:   $R^{\prime }=R;$ 14:   $R_p^{\prime }=\left\{s_0\right|s_0\notin R\wedge \exists s_1:s_1\in R:(s_0,s_1)\notin mt\};$ 15:   $R_p=R_p^{\prime }\cup Rp;$ 16:   for $each{s}_0\in R_p^{\prime }$ do 17:    ${\delta }_p^{\prime }={\delta }_p^{\prime }\cup \left\{(s_0,s_1)\right|(s_0,s_1)\notin mt\wedge s_1\in R\};$ 18:   end for 19:   for $each{s}_0\notin R:$ $\left(\nexists s_2:s_2\in (\neg (R\cup R_p)\cup m{s}_1):(s_0,s_2)\in {\delta }_e\right)\wedge$ $$\begin{gathered} \left(\right(\exists s_1:s_1\in (R\cup R_p)-m{s}_1: \\ (s_0,s_1)\in {\delta }_e)\wedge (\nexists s_2::(s_0,s_2)\in ({\delta }_e\cap {\delta }_b)))\vee s_0\in R_p) \end{gathered}$$ do 20:    $R=R\cup s_0;$ 21:   end for 22:  until $(R^{\prime }=R);$ 23:  $m{s}_1=m{s}_1\cup \neg (R\cup R_p);$ 24:  $m{s}_2=m{s}_2\cup \neg R;$ 25:  repeat 26:   $m{s}_1^{″}=m{s}_1;$ 27:   $m{s}_2^{″}=m{s}_2;$ 28:   $$\begin{gathered} m{s}_1=m{s}_1\cup \left\{s_0\right|\exists s_1:s_1\in m{s}_2:(s_0,s_1)\in f\}\cup \left\{s_0\right|\left(\exists s_1:s_1\in m{s}_1:(s_0,s_1)\in {\delta }_e)\vee (s_0,s_1)\in \\ ({\delta }_e\cap {\delta }_b)\right)\wedge \left(\nexists s_2::(s_0,s_2)\in {\delta }_p^{\prime }\right)\}; \end{gathered}$$ 29:   $m{s}_2=m{s}_2\cup m{s}_1\cup \left\{s_0\right|\exists s_1:s_1\in m{s}_1:(s_0,s_1)\in {\delta }_e)\};$ 30:   $mt=\left\{(s_0,s_1)\right|(s_0,s_1)\in ({\delta }_b\cup {\delta }_r)\vee s_1\in m{s}_2\};$ 31:  until $m{s}_1^{″}=m{s}_1\wedge m{s}_2^{″}=m{s}_2$ 32:  ${\delta }_p^{\prime }={\delta }_p^{\prime }-mt;$ 33:  $S^{\prime },{\delta }_p^{\prime }=ClosureAndDeadlocks(S^{\prime }-m{s}_2,{\delta }_p^{\prime },{\delta }_e);$ 34:  repeat 35:   if $S^{\prime }=\varphi$ then 36:    return Not-Possible; 37:   end if 38:   $S^{‴}=S^{\prime };$ 39:   $m{s}_3=\left\{s_0\right|\left(\exists s_1,s_2::(s_0,s_1)\in {\delta }_e\wedge (s_0,s_2)\in {\delta }_p\right)\wedge$ $\left(\nexists s_3:(s_0,s_3)\in {\delta }_p^{\prime }\right)\};$ 40:   $m{s}_4=\left\{s_0\right|\exists s_1::(s_1\in m{s}_3\wedge (s_0,s_1)\in {\delta }_e)\}$ 41:   $S^{\prime },{\delta }_p^{\prime }=ClosureAndDeadlocks(S-m{s}_4,{\delta }_p^{\prime },{\delta }_e);$ 42:  until ($S^{‴}=S^{\prime }$) 43: until$(S^{″}=S^{\prime }\wedge m{s}_1^{\prime }=m{s}_1\wedge m{s}_2^{\prime }=m{s}_2)$ 44: return$(S^{\prime },{\delta }_p^{\prime })$;

In the loop on Lines 12–22 we build set R which includes all states from which all computations reach a state in $S^{\prime }$. In addition, all required program transitions are added to ${\delta }_p^{\prime }$ by Line 17. When loop on Lines 12–22 terminates, we add all states in $\neg (R\cup R_p)$ to $m{s}_1$. We also set $m{s}_2$ to $\neg R$. Then, in the loop on Lines 25–31 we expand $m{s}_1$ and $m{s}_2$ with the same procedure used in Algorithm 2. In Line 32, we remove any transition in $mt$ from ${\delta }_p^{\prime }$. In Line 33, we remove all states in $m{s}_2$ from $S^{\prime }$ and ensure closure and deadlock freedom.

In the loop on Lines 34–42, we remove some states from $S^{\prime }$ to avoid new behavior inside the invariant just like what we did in Algorithm 2. If any state $s_0$ is removed from $S^{\prime }$ in Lines 33 or 41, we need to repeat the loop on Lines 6–43, because it is possible that a state in R was dependent on $s_0$ to reach $S^{\prime }$ but $s_0$ is not in $S^{\prime }$ anymore. We also need to repeat this loop if the set of $m{s}_1$ or $m_2$ has changed. In Line 36, the algorithm declares that there does not exist a solution if $S^{\prime }$ is empty. Otherwise, when loop on Lines 6–43 terminates, the algorithm returns $({\delta }_p^{\prime },S^{\prime })$ as the solution to the addition problem.

We note that the addition of nonmasking fault-tolerance considered [6] is also possible with Algorithm 3. In particular, in this case, we need to set ${\delta }_b$ to be the empty set. In principle, Algorithm 3 could also be used to add stabilization. However, we presented Algorithm 1 separately since it is a much simpler algorithm and forms the basis of Algorithm 3.

6. Experimental Results

In this section, we provide some of our experimental results of applying algorithms provided in previous sections. We have implemented the algorithms proposed in this paper in Java in a package called JavaRepair. JavaRepair and the code for examples of this section can be downloaded from [4]. JavaRepair uses JavaBDD [4] for symbolic repair using Binary Decision Diagrams (BDDs). All experiments are done on a 64-bit Windows 8.1 machine with AMD A8-6410 APU 2.00 GHz CPU and 8 GB RAM. We first provide our results for the addition of stabilization in Section 6.1, next we focus on the addition of fault-tolerance in Section 6.2.

6.1. Addition of Stabilization

In this section we use the JavaRepair package to repair/synthesis stabilizing programs for two variations of the shepherding problem [10].

6.1.1. One-Dimensional Stabilizing Shepherding Problem

In this section, we focus on the shepherding problem introduced in Section 1. We first consider the one-dimensional version of the problem where both farmer and the sheep move in a line. Thus, we have a row of n cells that the sheep/farmer can be in. Thus, we have two variables $c_f$ and $c_s$ which represent the location of the farmer and the sheep, respectively (see Figure 5). The farmer and the sheep can change their location at most by one cell at a time.

The rightmost cell is marked as the desired location. The farmer wants to steer the sheep to this cell. Thus, the invariant for the problem is $\left\{s\right|c_s\left(s\right)=n-1\}$, where n is the number of cells in the row. The sheep always tries to increase its distance with the farmer. When the farmer and the sheep are in the same location, the sheep non-deterministically decides to go to the right or left, if it has space in its both left and right. When the sheep is in the desired location, it stops moving further. With this explanation, the set of environment transitions is

$$\begin{array}{c}{\delta }_e=\{(s_0,s_1)|c_s\left(s_0\right)\ne n-1\wedge c_f\left(s_0\right)=c_f\left(s_1\right)\wedge \hfill \\ (\left(c_f\left(s_0\right)<c_s\left(s_0\right)\wedge c_s\left(s_0\right)<n-1\wedge c_s\left(s_1\right)=c_s\left(s_0\right)+1\right)\vee \hfill \\ \left(c_f\left(s_0\right)>c_s\left(s_0\right)\wedge c_s\left(s_0\right)>0\wedge c_s\left(s_1\right)=c_s\left(s_0\right)-1\right)\vee \hfill \\ \left(c_f\left(s_0\right)=c_s\left(s_0\right)\wedge c_s\left(s_0\right)<n-1\wedge c_s\left(s_1\right)=c_s\left(s_0\right)+1\right)\vee \hfill \\ \left(c_f\left(s_0\right)=c_s\left(s_0\right)\wedge c_s\left(s_0\right)>0\wedge c_s\left(s_1\right)=c_s\left(s_0\right)-1\right)\left)\right\}\hfill \end{array}$$

Also, since the farmer cannot move more than one cell in one step, the set of program restrictions is

$${\delta }_r=\{(s_0,s_1)\left|\right|c_f\left(s_0\right)-c_f\left(s_1\right)|>1\}.$$

Because of the non-determinism in the movement of the sheep, a program for the farmer can go into an infinite loop such that it never reaches the goal state. To understand how that can happen, consider this scenario: suppose both farmer and sheep are in the cell 0. Since the left side of the sheep is closed, it can only move to the right. Thus, it moves to the second cell. Now, suppose the farmer also decides to go to the second cell. Thus, again both of them are in the same cell. However, this time, since the left side of the sheep is open, it may decide to go there. Suppose sheep decides to go the left cell. Then, the farmer also goes back to the first cell. Once again, they are both in the first cell and the same scenario can repeat forever. A correct program for the farmer utilizes the behavior of the sheep, while avoiding going into an infinite loop and steers the sheep to the desired location.

We modeled this problem using JavaRepair for different sizes. We used a program with an empty set of transitions as the input program. Algorithm 1 successfully adds stabilization to the given program.

Table 1. Average Time of Addition of Stabilization for Shepherding Problem.

Size	Addition Time (s)
	1D Shepherding	2D Shepherding
4	0.008	0.025
5	0.023	0.119
6	0.007	0.231
7	0.007	0.164
8	0.007	0.115
9	0.014	0.656
10	0.038	1.141
11	0.013	1.310
12	0.011	0.802
13	0.010	1.952
14	0.012	2.776
15	0.012	2.514
16	0.010	0.680
17	0.024	7.636
18	0.029	16.826
19	0.026	16.361
20	0.020	9.655
30	0.128	42.137
40	0.119	46.789

shows the repair time for different sizes.

6.1.2. Two-Dimensional Stabilizing Shepherding Problem

In this section, we consider the shepherding problem described in Section 6.1.1 in a two-dimensional space. Thus, instead of a single row of cells, we have a grid of cells where the sheep and the farmer can move. Thus, we have four variables $r_f$, $c_f$, $r_s$ and $c_s$ that represent the row and column of the farmer and the sheep, respectively (see Figure 6).

The upper right corner of the grid is the desired location. Thus, the invariant for this problem is $\left\{s\right|r_s\left(s\right)=0\wedge c_s\left(s\right)=n-1\}$, where again n is the number of cells in a row. The sheep again tries to increases its distance with the farmer. The sheep and the farmer can only change either their row or column by at most one at each step. As in Section 6.1.1, sheep stops moving once it is in the desired location. With this explanation, the set of environment transitions is

$$\begin{array}{c}{\delta }_e=\{(s_0,s_1)|\neg (r_s\left(s_0\right)=0\wedge c_s\left(s_0\right)=n-1)\wedge \hfill \\ (r_f\left(s_0\right)=r_f\left(s_1\right)\wedge c_f\left(s_0\right)=c_f\left(s_1\right))\wedge \hfill \\ (c_s\left(s_0\right)=c_s\left(s_1\right)\vee r_s\left(s_0\right)=r_s\left(s_1\right))\wedge \hfill \\ (\left(c_f\left(s_0\right)<c_s\left(s_0\right)\wedge c_s\left(s_0\right)<n-1\wedge c_s\left(s_1\right)=c_s\left(s_0\right)+1\right)\vee \hfill \\ \left(c_f\left(s_0\right)>c_s\left(s_0\right)\wedge c_s\left(s_0\right)>0\wedge c_s\left(s_1\right)=c_s\left(s_0\right)-1\right)\vee \hfill \\ \left(c_f\left(s_0\right)=c_s\left(s_0\right)\wedge c_s\left(s_0\right)<n-1\wedge c_s\left(s_1\right)=c_s\left(s_0\right)+1\right)\vee \hfill \\ \left(c_f\left(s_0\right)=c_s\left(s_0\right)\wedge c_s\left(s_0\right)>0\wedge c_s\left(s_1\right)=c_s\left(s_0\right)-1\right)\vee \hfill \\ \left(r_f\left(s_0\right)<r_s\left(s_0\right)\wedge r_s\left(s_0\right)<n-1\wedge r_s\left(s_1\right)=r_s\left(s_0\right)+1\right)\vee \hfill \\ \left(r_f\left(s_0\right)>r_s\left(s_0\right)\wedge r_s\left(s_0\right)>0\wedge r_s\left(s_1\right)=r_s\left(s_0\right)-1\right)\vee \hfill \\ \left(r_f\left(s_0\right)=r_s\left(s_0\right)\wedge r_s\left(s_0\right)<n-1\wedge r_s\left(s_1\right)=r_s\left(s_0\right)+1\right)\vee \hfill \\ \left(r_f\left(s_0\right)=r_s\left(s_0\right)\wedge r_s\left(s_0\right)>0\wedge r_s\left(s_1\right)=r_s\left(s_0\right)-1\right)\left)\right\}\hfill \end{array}$$

Also, the set of program restrictions is

$${\delta }_r=\{(s_0,s_1)\left|\right|c_f\left(s_0\right)-c_f\left(s_1\right)|+|r_f\left(s_0\right)-r_f\left(s_1\right)|>1\}.$$

Like one-dimensional case discussed in Section 6.1.1, with a poorly designed farmer, here is also possible to chase the sheep forever and never reach the desired state. Table 1 shows the repair time using JavaRepair for this problem for different sizes.

6.2. Addition of Fault-Tolerance

In this section, we use the JavaRepair package to repair/synthesis fault-tolerant programs for two other variations of the shepherding problem. We first create a failsafe fault-tolerant program in Section 6.2.1. Next, we create a masking fault-tolerant program in Section 6.2.2.

6.2.1. Failsafe Fault-Tolerant Shepherding Problem

For this variation, we change the two-dimensional version described in Section 6.1.2 as follows: there is no requirement of steering the sheep to the desired location. Instead, we require that the farmer must be always close to the sheep. Specifically, we require that the farmer and sheep can be far apart at most by one row or one cell. We model this requirement as a safety property with the set of bad transitions

$$\begin{array}{c}{\delta }_b=\{(s_0,s_1)|\hfill \\ \neg (\left(r_s\left(s_1\right)=r_f\left(s_1\right)\wedge c_s\left(s_1\right)=c_f\left(s_1\right)\right)\vee \hfill \\ \left(|r_s\left(s_1\right)-r_f\left(s_1\right)|=1\wedge c_s\left(s_1\right)=c_f\left(s_1\right)\right)\vee \hfill \\ \left(r_s\left(s_1\right)=r_f\left(s_1\right)\wedge |c_s\left(s_1\right)-c_f\left(s_1\right)|=1\right)\left)\right\}.\hfill \end{array}$$

The invariant, the set of environment transitions ${\delta }_e$ and the set of program restrictions ${\delta }_r$ are the same as those for two-dimensional case explained in Section 6.1.2. For addition of fault-tolerance we have a set of faults as the input. We consider two sets of faults and try to repair the program with each of them. The first set captures faults that can change the state of the program arbitrary, that is, for any state the program may change to any other state. We denote this set by $f_{arbitrary}$. The second set of fault transitions that we consider captures faults that cause the system transition to a state where both farmer and sheep co-locate in one of the cells except the desired cell. We model this transition as a set of fault transitions

$$\begin{array}{c}{f}_{same-location}=\{(s_0,s_1)|\hfill \\ r_s\left(s_0\right)=0\wedge c_s\left(s_0\right)=n-1\wedge \hfill \\ r_s\left(s_1\right)=r_f\left(s_1\right)\wedge c_s\left(s_1\right)=c_f\left(s_1\right)\wedge \hfill \\ \neg (r_s\left(s_1\right)=0\wedge c_s\left(s_1\right)=n-1)\}\hfill \end{array}$$

We modeled this problem using JavaRepair for different sizes. The Algorithm 2 does not find any solution for the case of arbitrary faults, $f_{arbitrary}$. From the completeness of Algorithm 2, we know that there is no solution for this case. On the other hand, the algorithm finds a solution for $f_{same-location}$.

Table 2. Average Time of Addition of Fault-tolerance for Shepherding Problem.

Size	Addition Time (s)
	Failsafe		Masking
	Arbitrary Fault	Same-Location Fault	Arbitrary Fault	Same-Location Fault
4	0.006	0.009	0.009	0.057
5	0.019	0.027	0.043	0.176
6	0.006	0.018	0.023	0.129
7	0.007	0.017	0.016	0.197
8	0.004	0.012	0.013	0.181
9	0.019	0.027	0.025	0.340
10	0.014	0.029	0.020	0.424
11	0.007	0.024	0.018	0.547
12	0.004	0.033	0.015	0.493
13	0.005	0.035	0.016	0.698
14	0.005	0.021	0.018	0.807
15	0.006	0.019	0.025	0.928
16	0.004	0.015	0.014	0.735
17	0.032	0.081	0.074	1.142
18	0.017	0.281	0.116	1.273
19	0.019	0.284	0.036	1.416
20	0.014	0.049	0.099	1.245
30	0.886	1.137	0.973	4.995
40	1.204	1.287	1.246	5.232

shows the repair time for different sizes.

6.2.2. Masking Fault-Tolerant Shepherding Problem

In this section, we add the requirement of steering the sheep to the desired location to the failsafe version explained in Section 6.2.1. Note that this requirement is weaker than the one specified in the stabilization version explained in Section 6.1.2 where we required that the farmer must steer the sheep to the desired location starting from any arbitrary state. On the other hand, in this section, we require that the farmer should steer the sheep to the goal state once the program starts in states reached by the fault transitions (i.e., fault-span). The invariant, the set of environment transitions and the set of program restrictions are the same as those defined in Section 6.2.1. Like Section 6.2.1, we consider two sets of fault transitions $f_{arbitrary}$ and $f_{same-location}$. Like failsafe case, there is no solution for arbitrary faults but for same-location faults, Algorithm 3 finds a solution. Table 2 shows the addition time for this problem.

7. Discussion and Extensions of Algorithms

In this section, we consider problems related to those addressed in Section 3 and Section 5. Our first variation focuses on Definition 6. In this definition, we assumed that the environment is fair. Specifically, at least $k-1$ actions execute between any two environment actions. We consider variations where (1) this property is satisfied eventually. In other words, for some initial computation, environment actions may prevent the program from executing. However, eventually, fairness is provided to program actions and (2) program actions are given even with reduced fairness. Specifically, we consider the case where several environment actions can execute in a row but program actions execute infinitely often.

Our second variation is related to the invariant of the revised program, $S^{\prime }$ and the invariant of the original program, S. In the case of adding stabilization, we considered $S^{\prime }=S$ whereas, in the case of adding fault-tolerance, we considered $S^{\prime }\subseteq S$.

Changes for adding stabilization and fault-tolerance with an eventually fair environment. No changes are required to Algorithm 1 even if the environment is eventually fair. This is due to the fact that this algorithm constructs programs that provide recovery from any state, that is, it will provide recovery from the state reached after the point when fairness is restored. For Algorithm 2 and Algorithm 3, we should change the input f to include ${\delta }_e\cup {\delta }_f$. The resulting algorithm will ensure that the generated program will allow unfair execution of the program in initial states. However, fault-tolerance will be provided when the fairness is restored.

Changes for adding stabilization and fault-tolerance with multiple consecutive environment actions. If environment actions can execute consecutively, we can change input ${\delta }_e$ to be its transitive closure. In other words, if $(s_0,s_1)$ and $(s_1,s_2)$ are transitions in ${\delta }_e$, we add $(s_0,s_2)$ to ${\delta }_e$. With this change, the constructed program will provide stabilization or fault-tolerance even if environment transitions can execute consecutively.

Changes for adding stabilization and fault-tolerance based on relation between $S^{\prime }$ (invariant of the fault-tolerant program) and S (invariant of the fault-intolerant program). No changes are required to Algorithm 1 even if we change the problem statement to allow $S^{\prime }\subseteq S$ without affecting soundness or completeness. Regarding soundness, observe that the program generated by this algorithm ensures $S^{\prime }=S$. Hence, it trivially satisfies $S^{\prime }\subseteq S$. Regarding completeness, the intuition is that if it was impossible to recover to states in S then it is impossible to recover to states that are a subset of S. Regarding Algorithm 2 and Algorithm 3, if $S^{\prime }$ is required to be equal to S then they need to be modified as follows: in these algorithms if any state S is removed (due to it being in $m{s}_2$, deadlocks, etc.) then they should declare failure.

Other applications of our algorithms. In this paper, we considered the shepherding application as a way to illustrate the role of environment actions. The shepherding program is an instance of several other programs. For example, consider a smart cruise controller. In this example, we have two cars, A and B where car A is the car in front and car B is trying to match the speed of car A and maintain a safe distance. Now, considering from the perspective of car B, actions of car A are environment actions that are unchangeable. These actions are essential (without these actions, car B will not be able to satisfy its objectives) and disruptive. In other words, car A plays the same role as the sheep and car B plays the same role as the farmer.

It is also applicable in an example such as merging on a highway. In this example (when viewed from the perspective of the car that is merging), actions of other cars are the same as the role played by sheep in the shepherding problem. More generally, if we have a system with multiple components/processes where only some components are changeable, then these components would be modeled as farmer actions whereas unchangeable actions play the role of the sheep.

8. Related Work

This paper focuses on the addition of fault-tolerance properties in the presence of unchangeable environment actions. This problem is an instance of model repair where some existing model/program is repaired to add new properties such as safety, liveness, fault-tolerance and so forth. Model repair with respect to CTL properties was first considered in [11] and abstraction techniques for the same are presented in [12]. Previously, Bonakdarpour et al. [13] has considered the problem of model repair for UNITY specifications [14]. These results identify complexity results for adding properties such as invariant properties, leads-to properties and so forth. Repair of probabilistic algorithms has also been considered in the literature [15]. Roohitavaf and Kulkarni [16] provide repair algorithms for cases where the new desired properties are in conflict with existing properties.

The problem of adding fault-tolerance to an existing program has been discussed in the absence of environment actions. This work includes work on controller synthesis [17,18,19,20]. A tool for automated addition of fault-tolerance to distributed programs is presented in [1]. This work utilizes BDD based techniques to enable the synthesis of programs with state space exceeding ${10}^{100}$. However, this work does not include the notion of environment actions that cannot be removed. Hence, applying it in contexts where some processes/components cannot be changed will result in unacceptable solutions. Model repair for distributed programs using a two-phase lazy approach is proposed in [21].

The work on game theory [22,23,24] has focused on the problem of repair with the 2-player game where the actions of the second player are not changed. However, this work does not address the issue of fault-tolerance. Also, the role of the environment in our work is more general than that in [22,23,24]. Specifically, in the work on game theory, it is assumed that the players play in an alternating manner. By contrast, we consider more general interaction with the environment. In [25], authors have presented an algorithm for adding recovery to component-based models. They consider the problem where we cannot add to the interface of a physical component. However, it does not consider the issue of unchangeable actions considered in our work.

Hajisheykhi et al. [26,27] focus on the notion of auditable events which captures special events for which the program must first transition to a special state called auditable state where every process is aware of the auditable events, before returning to the normal states. Thus, compared with fault-span considered in this paper (cf. Definition 14), the set of states that a program may reach other than its states includes states that can be reached by fault transitions and those that can be reached by auditable transitions. Considering the problem of auditable restoration in presence of environment actions is an interesting future work.

9. Conclusions

In this paper, we focused on the problem of adding fault-tolerance to an existing program which consists of some actions that are unchangeable. These unchangeable actions arise due to interaction with the environment, inability to change parts of the existing program, constraints on physical components in a cyber-physical system and so on. We presented algorithms for adding stabilization and fault-tolerance. These algorithms are sound and complete and run in polynomial time (in the state space).

We considered the cases where (1) all fault-free behaviors are preserved in the fault-tolerant program, or (2) only a nonempty subset of fault-free behaviors are preserved in the fault-tolerant program. We also considered the cases where (1) environment actions can execute with any frequency for an initial duration and (2) environment actions can execute more frequently than programs. In all these cases, we demonstrated that our algorithm can be extended while preserving soundness and completeness. We provided a Java package including the implementation of our algorithm using BDDs. We presented some of our experimental results of adding stabilization and fault-tolerance using JavaRepair.

There are several future extensions to this work. One of the extensions has to deal with partial observability in distributed or embedded systems. Algorithms in this paper need to be revised by taking into account scenarios where, due to partial observability, one transition added or removed corresponds to a group of transitions [28]. Another extension is to extend the JavaRepair package so that it can be used efficiently and with a reduced learning curve. Yet another extension is to deal with the issue of time in embedded and cyber-physical systems. Specifically, in a CPS, we expect physical components to execute at a certain speed. In other words, an action of the physical component can be thought of to take some time between ${ϵ}_1$ and ${ϵ}_2$. In turn, this affects the number of steps a computational component can execute in the middle. Note that this maps to the value of k used in our algorithms. Also, our definition of computation requires that the environment cannot execute too frequently. We also intend to extend to cases where a similar restriction also applies to computational components.