1. Introduction
In [
1], Larsen and Skou proposed the notion of probabilistic bisimulation. Although described for deterministic transition systems, the same notion is also very suitable for probabilistic transition systems with nondeterminism [
2,
3], i.e. socalled PLTSs. It expresses that two states are equivalent exactly when the following condition holds: if one state can perform an action ending up in a set of states, each with a certain probability, and then the other state can do the same step ending up in an equivalent set of states with the same distribution of probabilities. Two characteristic nondeterministic transition systems of which the initial states are probabilistically bisimilar are given in
Figure 1.
In [
4], Baier et al. gave an algorithm for probabilistic bisimulation for PLTSs, thus dealing both with probabilistic and nondeterministic choice, of time complexity
$\mathit{O}\phantom{\rule{0.55542pt}{0ex}}\left(\right)open="("\; close=")">mn(logm+logn)$ and space complexity
$\mathit{O}\phantom{\rule{0.55542pt}{0ex}}\left(\right)open="("\; close=")">mn$, where
n is the number of states and
m is the number of transitions (from states to distributions over states; there is no separate measure for the size of the distributions). As far as we know, it is the only practical algorithm for bisimulation à la LarsenSkou for PLTSs. In essence, other algorithms for probabilistic systems typically target Markov chains without nondeterminism. The algorithm in [
4] performs an iterative refinement of a partition of states and a partition of transitions per action label. The crucial point is splitting the groups of states based on probabilities. For this, a specific data structure is used, called augmented ordered balanced trees, to support efficient storage, retrieval and ordering of states indexed by probabilities.
In this paper, we provide a new algorithm for probabilistic bisimulation for PLTSs of time complexity
$\mathit{O}\phantom{\rule{0.55542pt}{0ex}}\left(\right)open="("\; close=")">({m}_{a}+{m}_{p})log{n}_{p}+{m}_{p}log{n}_{a})$ and space complexity
$\mathit{O}\phantom{\rule{0.55542pt}{0ex}}\left(\right)open="("\; close=")">{m}_{a}+{m}_{p}$, where
${n}_{a}$ is the number of states,
${m}_{a}$ the number of transitions labelled with actions,
${n}_{p}$ the number of distributions and
${m}_{p}$ the cumulative support of the distributions. Our
${n}_{a}$ coincides with the
n of Baier et al. We prefer to use
${m}_{a}$,
${n}_{p}$, and
${m}_{p}$ over
m as the former support a more refined analysis. A detailed comparison between the algorithms reveals that, if the distributions have a positive probability for all states, the complexities of the algorithms are similar. However, when distributions only touch a limited number of states, as is often the common situation, the implementation of our algorithm outperforms our implementation of the algorithm in [
4], both in time as well as in space complexity.
Similar to the algorithm of Baier et al., our algorithm keeps track of a partition of states and of distributions (referred to as action states and probabilistic states below) but in line with the classical Paige–Tarjan approach [
5] it also maintains a courser partition of socalled constellations. The treatment of distributions in our algorithm is strongly inspired by the work for Markov Chain lumping by Valmari and Franceschinis, but our algorithm applies to the richer setting of nondeterministic labelled probabilistic transition systems. Using a brilliant, yet simple argument, taken from [
6], the number of times a probabilistic transition is sorted can be limited by the fanout of the source state of the transition. This leads to the observation that we can use straightforward sorting without the need of any tailored data structure such as augmented ordered balanced trees or similar as in [
4,
7]. Actually, our algorithm uses a simplification of the algorithm in [
6] since the calculation of socalled
majority candidates can be avoided, too.
We implemented both the new algorithm and the algorithm from [
4]. We spent quite some effort to establish that both implementations are free from programming flaws. To this end, we ran them sidebyside and compared the outcomes on a vast amount of randomly generated probabilistic transition systems (in the order of millions). Furthermore, we took a number of examples from the field, among others from the
Prism toolset [
8], and ran both implementations on the probabilistic transition systems that were obtained in this way. Timewise, all benchmarks indicated better results for our algorithm compared to the algorithm from [
4]. Even for rather small transition systems of about 100,000 states, performance gains of a factor 10,000 can be achieved. Memorywise the implementation of our algorithm also outperforms the implementation in [
4] when the sizes of the probabilistic state space are larger. Both findings are in line with the theoretical complexity analyses of both algorithms. Both implementations have been incorporated in the open source mCRL2 toolset [
9,
10].
1.1. Related Work
Probabilistic bisimulation preserves logic equivalence for PCTL [
11]. In [
12], Katoen c.s. reported up to logarithmic state space reduction obtained by probabilistic bisimulation minimisation for DTMCs. Quotienting modulo probabilistic bisimulation is based on the algorithm in [
7]. In the same vein, Dehnert et al. proposed symbolic probabilistic bisimulation minimisation to reduce computation time for model checking PCTL in a setting for DTMCs [
13], where an SMT solver is exploited to do the splitting of blocks. Partition reduction modulo probabilistic bisimulation is also used as an ingredient in a counterexample guided abstraction refinement approach (CEGAR) for model checking for PCTL by Lei Song et al. in [
14].
For CTMCs, Hillston et al. proposed the notion of contextual lumpability based on lumpable bisimulation in [
15]. Their reduction technique uses the Valmari–Franceschinis algorithm for Markov chain lumping mentioned earlier. Crafa and Renzato [
16] characterised probabilistic bisimulation of PLTSs as a partition shell in the setting of abstract interpretation. The algorithm for probabilistic bisimulation that comes with such a characterisation turns out to coincide with that in [
4]. A similar result applies to the coalgebraic approach to partition refinement in [
17] that yields a general bisimulation decision procedure, which can be instantiated with probabilistic system types.
Probabilistic simulation for PLTSs has been treated in [
4], too. In [
18], maximum flow techniques are proposed to improve the complexity. Zhang and Jansen [
19] presented a spaceefficient algorithm based on partition refinement for simulation between probabilistic automata, which improves upon the algorithm for simulation by Crafa and Renzato [
16] for concrete experiments taken from the PRISM benchmark suite. A polynomial algorithm, essentially cubic, for deciding weak and branching probabilistic bisimulation by Turrini and Hermanns, recasting the algorithm in [
20], is presented in [
21].
1.2. Synopsis
The structure of this article is as follows. In
Section 2, we provide the notions of a probabilistic transition system as well as that of probabilistic bisimulation. In
Section 3, the outline of our algorithm is provided and it is proven that it correctly calculates probabilistic bisimulation. This section ends with an elaborate example. In
Section 4 we provide a detailed version the algorithm with a focus on the implementation details necessary to achieve the complexity. In
Section 5, we provide some benchmarking results and a few concluding remarks are made in
Section 6.
2. Preliminaries
Let S be a finite set. A distribution f over S is a function $f:S\to [0,1]$ such that ${\sum}_{\phantom{\rule{0.166667em}{0ex}}s\in S}\phantom{\rule{0.166667em}{0ex}}f\left(s\right)=1$. For each distribution f, its support is the set $\{s\in S\mid f(s)>0\}$. The size of f is defined as the number of elements in its support, written as $\leftf\right$. The set of all distributions over a set S is denoted by $\mathcal{D}\left(S\right)$. Distributions are lifted to act on subsets $T\subseteq S$ by $f\left[T\right]={\sum}_{\phantom{\rule{0.166667em}{0ex}}s\in T}\phantom{\rule{0.166667em}{0ex}}f\left(s\right)$.
For an equivalence relation R on S, we use $S/R$ to denote the set of equivalence classes of R. We define $s\phantom{\rule{0.55542pt}{0ex}}/R=\{\phantom{\rule{0.166667em}{0ex}}t\in S\mid sRt\phantom{\rule{0.166667em}{0ex}}\}$ and, for a subset T of S, we define $T/R=\{\phantom{\rule{0.166667em}{0ex}}s\in S\mid \exists \phantom{\rule{0.55542pt}{0ex}}t\in T:sRt\phantom{\rule{0.166667em}{0ex}}\}$. A partition $\pi =\{\phantom{\rule{0.166667em}{0ex}}{B}_{i}\subseteq S\mid i\in I\phantom{\rule{0.166667em}{0ex}}\}$ is a set of nonempty subsets such that ${B}_{i}\cap {B}_{j}=\varnothing $ for all $i,j\in I$ and ${\bigcup}_{i\in I}\phantom{\rule{0.166667em}{0ex}}{B}_{i}=S$. Each ${B}_{i}$ is called a block of the partition. Slightly ambiguously, we use $S/R$ to denote the set of equivalence classes of R with respect to S. Clearly, the set of equivalence classes of R forms a partition of S. Reversely, a partition $\pi $ of S induces an equivalence relation ${R}_{\pi}$ on S, by $s{R}_{\pi}t$ iff $s,t\in B$ for some block B of $\pi $. A partition $\pi $ is called a refinement of a partition $\varrho $ iff each block of $\pi $ is a subset of a block of $\varrho $. Hence, each block in $\varrho $ is a disjoint union of blocks from $\pi $.
We use probabilistic labeled transition systems as the canonical way to represent the behaviour of systems.
Definition 1. (Probabilistic Labeled Transition System). A probabilistic labeled transition system (PLTS) for a set of actions $\mathit{Act}$ is a pair $\mathcal{A}=(\phantom{\rule{0.166667em}{0ex}}S,\phantom{\rule{0.166667em}{0ex}}\stackrel{}{\to}\phantom{\rule{1.111pt}{0ex}}\phantom{\rule{0.166667em}{0ex}})$ where
S is a finite set of states, and
$\stackrel{}{\to}\phantom{\rule{1.111pt}{0ex}}\subseteq S\times \mathit{Act}\times \mathcal{D}\left(S\right)$ is a finite transition relation relating states and actions to distributions.
It is common to write $s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}f$ for $\langle \phantom{\rule{0.166667em}{0ex}}s,\phantom{\rule{0.166667em}{0ex}}a,\phantom{\rule{0.166667em}{0ex}}f\phantom{\rule{0.166667em}{0ex}}\rangle \in \stackrel{}{\to}\phantom{\rule{1.111pt}{0ex}}\phantom{\rule{1.111pt}{0ex}}$. For $s\in S$, $a\in \mathit{Act}$, and a set $F\subseteq \mathcal{D}\left(S\right)$ of distributions, we write $s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}F$ if $s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}f$ for some $f\in F$. Similarly, we write $\stackrel{a}{\nrightarrow}F$ if there is no distribution $f\in F$ such that $s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}f$. For the presentation below, we associate a socalled probabilistic state ${u}_{f}$ with each distribution f provided there is some transition $s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}f$ of $\mathcal{A}$. We write U for $\{\phantom{\rule{0.222222em}{0ex}}{u}_{f}\mid \exists \phantom{\rule{0.55542pt}{0ex}}s\in S,a\in \mathit{Act}:s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}f\phantom{\rule{0.222222em}{0ex}}\}$, with typical element u. Note that, since $\stackrel{}{\to}\phantom{\rule{1.111pt}{0ex}}$ is finite, U is also finite. We also use the notation $s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}{u}_{f}$ if $s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}f$ for some $f\in \mathcal{D}\left(S\right)$. As a matter of notation, we write ${u}_{f}\left[T\right]$ for $f\left[T\right]$ if probabilistic state ${u}_{f}$ corresponds to the distribution f. We sometimes use a socalled probabilistic transition ${u}_{f}{\mapsto}_{\phantom{\rule{1.111pt}{0ex}}p}s$ for $0<p\u2a7d1$ and $s\in S$ iff ${u}_{f}\left(s\right)=p$. To stress $S\cap U=\varnothing $, we refer to states $s\in S$ as action states.
Below, in particular in the complexity analysis, we use ${n}_{a}=\leftS\right$ as the number of action states, ${n}_{p}=\leftU\right$ as the number of probabilistic states, ${m}_{a}=\to $ as the number of action transitions and ${m}_{p}={\sum}_{\phantom{\rule{0.166667em}{0ex}}{u}_{f}\in U}\phantom{\rule{0.222222em}{0ex}}\leftf\right$ as the cumulative size of the support of the distributions corresponding to all probabilistic states. Note that ${m}_{p}\u2a7e{n}_{p}$ as every distribution has support of at least size 1.
The following definition for probabilistic bisimulation stems from [
1].
Definition 2. (Probabilistic Bisimulation). Consider a PLTS $\mathcal{A}=(\phantom{\rule{0.166667em}{0ex}}S,\phantom{\rule{0.166667em}{0ex}}\stackrel{}{\to}\phantom{\rule{1.111pt}{0ex}}\phantom{\rule{0.166667em}{0ex}})$. An equivalence relation $R\subseteq S\times S$ is called aprobabilistic bisimulationfor $\mathcal{A}$ iff for all states $s,t\in S$ such that $s\phantom{\rule{0.55542pt}{0ex}}R\phantom{\rule{1.111pt}{0ex}}t$ and $s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}f$, for some action $a\in \mathit{Act}$ and distribution $f\in \mathcal{D}\left(S\right)$, it holds that $t\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}g$ for some distribution $g\in \mathcal{D}\left(S\right)$, and $f\left[B\right]=g\left[B\right]$ for each $B\in S/R$.
Two states $s,t\in S$ are probabilistically bisimilar iff a probabilistic bisimulation R for $\mathcal{A}$ exists such that $s\phantom{\rule{0.55542pt}{0ex}}R\phantom{\rule{1.111pt}{0ex}}t$, which we write as $s{\simeq}_{p}t$. Two distributions $f,g\in \mathcal{D}\left(S\right)$, and similarly two probabilistic states ${u}_{f},{u}_{g}\in U$, areprobabilistically bisimilariff for all $B\in S/{\simeq}_{p}$ it holds that $f\left[B\right]=g\left[B\right]$, which we also denote by $f{\simeq}_{p}g$ and ${u}_{f}{\simeq}_{p}{u}_{g}$, respectively.
By definition, probabilistic bisimilarity is the union of all probabilistic bisimulations. To be able to speak of probabilistically bisimilar distributions (or of probabilistically bisimilar probabilistic states), probabilistic bisimilarity needs to be an equivalence relation. In fact, probabilistic bisimilarity is a probabilistic bisimulation. See [
22] for a proof.
3. A Partition Refinement Algorithm for Probabilistic Bisimulation (Outline)
Many efficient algorithms for standard bisimulation calculate partitions of states [
5,
23,
24]. Here, we consider the construction of a partition
$\mathcal{B}$ of the sets of action states
S and of probabilistic states
U for some fixed PLTS
$\mathcal{A}$ over a set of actions
$\mathit{Act}$. Below blocks of the partition always contain either action states or probabilistic states.
3.1. Stability of Blocks and Partitions
An important notion underlying the algorithm introduced below is that of the stability of a block of a partition. If a block is not stable, it contains states that are not bisimilar. These states either have different transitions or different distributions. We first define the notion of stability more generically on sets instead of on blocks. Then, we lift it to partitions.
Definition 3. (Stable Sets and Partitions).  1.
A set of action states $B\subseteq S$ is called stable under a set of probabilistic states $C\subseteq U$ with respect to an action $a\in \mathit{Act}$ iff $s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}C$ whenever $t\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}C$ and vice versa for all $s,t\in B$. The set B is called stable under C iff B is stable under C with respect to all actions $a\in \mathit{Act}$.
 2.
A set of probabilistic states $B\subseteq U$ is called stable under a set of action states $C\subseteq S$ iff $u\left[C\right]=v\left[C\right]$ for all $u,v\in B$.
 3.
A set of states B with $B\subseteq S$, respectively $B\subseteq U$, is called stable under a partition $\mathcal{C}$ of $S\cup U$, with $C\subseteq S$ or $C\subseteq U$ for all $C\in \mathcal{C}$, iff B is stable under each $C\in \mathcal{C}$ with $C\subseteq U$, respectively $C\subseteq S$.
 4.
A partition $\mathcal{B}$ is called stable under a partition $\mathcal{C}$ iff all blocks B of $\mathcal{B}$ are stable under $\mathcal{C}$.
There are two simple but important properties stating that stability is preserved when splitting sets. The first one says that subsets of stable sets are also stable.
Lemma 1. Let $B\subseteq S$ be a set of action states and $C\subseteq U$ a set of probabilistic states. If B is stable under C, then any ${B}^{\prime}\subseteq B$ is also stable under C. Similarly, if C is stable under B, then any ${C}^{\prime}\subseteq C$ is also stable under B.
Proof. We only prove the first part as the argument for the second part is essentially the same. If $s,t\in {B}^{\prime}$, then also $s,t\in B$. As B is stable under C, it holds that for every action $a\in \mathit{Act}$ either both satisfy $s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}C$ and $t\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}C$, or neither does. Thus, ${B}^{\prime}$ is stable under C. ☐
The second property says that splitting a set in two parts can only influence the stability of an other set if there is a transition or a positive probability from this other set to one of the parts of the split set.
Lemma 2. Let $B\subseteq S$ be a set of action states and $C\subseteq U$ a set of probabilistic states.
 1.
Suppose B is stable under C with respect to an action a, ${C}^{\prime}\subseteq C$, and there is no $s\in B$ such that $s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}{C}^{\prime}$. Then, B is stable under ${C}^{\prime}$ and $C\setminus {C}^{\prime}$ with respect to a.
 2.
Suppose C is stable under B, ${B}^{\prime}\subseteq B$, and $u\left[{B}^{\prime}\right]=0$ for all $u\in C$. Then, C is stable under ${B}^{\prime}$ and $B\setminus {B}^{\prime}$.
Proof. We only provide the proof for the first part of this lemma. If $s,t\in B$, then both $s\stackrel{a}{\nrightarrow}{C}^{\prime}$ and $t\stackrel{a}{\nrightarrow}{C}^{\prime}$ by assumption. Thus, B is stable under ${C}^{\prime}$ with respect to a. Furthermore, B is stable under $C\setminus {C}^{\prime}$: Suppose $s,t\in B$ and $s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}C\setminus {C}^{\prime}$. Thus, $s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}C$. As B is stable under C, $t\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}C$, and by assumption $t\stackrel{a}{\nrightarrow}{C}^{\prime}$. Therefore, $t\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}C\setminus {C}^{\prime}$. Suppose $s\stackrel{a}{\nrightarrow}C\setminus {C}^{\prime}$. Then, also $s\stackrel{a}{\nrightarrow}C$. As B is stable under C, $t\stackrel{a}{\nrightarrow}C$ and hence, $t\stackrel{a}{\nrightarrow}C\setminus {C}^{\prime}$. ☐
The following property, called the stability property, says that a partition stable under itself induces a probabilistic bisimulation. In general, partition based algorithms for bisimulation search for such a stable partition.
Lemma 3. Stability Property. Let $\mathcal{A}=(\phantom{\rule{0.166667em}{0ex}}S,\phantom{\rule{0.166667em}{0ex}}\stackrel{}{\to}\phantom{\rule{1.111pt}{0ex}}\phantom{\rule{0.166667em}{0ex}})$ be a PLTS. If a partition $\mathcal{B}$ for $\mathcal{A}$ is stable under itself, then the corresponding equivalence relation $\mathcal{B}$ on S is a probabilistic bisimulation.
Proof. By the first condition of Definition 3 and stability of all blocks in $\mathcal{B}$ we have that either $B\subseteq S$ or $B\subseteq U$, for each block $B\in \mathcal{B}$. We write $s\mathcal{B}t$ iff $s,t\in B$ for some $B\in \mathcal{B}$. Note that used in this way $\mathcal{B}$ is an equivalence relation on S.
Suppose $s\mathcal{B}t$ for some $s,t\in S$ and $s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}f$. Let $u\in U$ correspond to f. Say $s,t\in B$ and $u\in {B}^{\prime}$ for some blocks $B,{B}^{\prime}\in \mathcal{B}$. Then, $s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}{B}^{\prime}$. By stability of B for ${B}^{\prime}$, it follows that $t\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}{B}^{\prime}$. Hence, $v\in {B}^{\prime}$ and $g\in \mathcal{D}\left(S\right)$ exist such that v corresponds to g and $s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}g$. Therefore, for any block ${B}^{\prime \prime}\in \mathcal{B}$ we have $f\left[{B}^{\prime \prime}\right]=u\left[{B}^{\prime \prime}\right]=v\left[{B}^{\prime \prime}\right]=g\left[{B}^{\prime \prime}\right]$ since the block ${B}^{\prime}$ of u and v is stable under each block ${B}^{\prime \prime}$ of $\mathcal{B}$.
Thus, the stable partition $\mathcal{B}$ induces an equivalence relation that satisfies the conditions for a probabilistic bisimulation of Definition 2, as was to be shown. ☐
3.2. Outline of the Algorithm
We present our algorithm in two stages. An abstract description of the algorithm is presented as Algorithm 1; the detailed algorithm is provided as Algorithm 2. The setup of Algorithm 1 is a fairly standard, iterative refinement of a partition
$\mathcal{B}$, in this particular case containing both action states and probabilistic states, which are treated differently. In addition, following the approach of Paige and Tarjan [
5], we maintain a coarser partition
$\mathcal{C}$, which we call the set of
constellations. Each constellation in partition
$\mathcal{C}$ is a union of one or more blocks of
$\mathcal{B}$, thus
$\mathcal{B}$ is a refinement of
$\mathcal{C}$. A constellation
$C\in \mathcal{C}$ that consists of exactly one block in
$\mathcal{B}$ is called
trivial. We refine partitions
$\mathcal{B}$ and
$\mathcal{C}$ until
$\mathcal{C}$ only contains trivial constellations (see Line 5 of Algorithm 1).
Algorithm 1 Abstract Partition Refinement Algorithm for Probabilistic Bisimulation. 
 1:
functionpartitionrefinement  2:
$\mathcal{C}:=\{\phantom{\rule{0.166667em}{0ex}}S,\phantom{\rule{0.166667em}{0ex}}U\phantom{\rule{0.166667em}{0ex}}\}$  3:
$\mathcal{B}:=\left\{\phantom{\rule{0.166667em}{0ex}}U\phantom{\rule{0.166667em}{0ex}}\right\}\cup \{\phantom{\rule{0.166667em}{0ex}}{S}_{A}\mid A\subseteq \mathit{Act}\phantom{\rule{0.166667em}{0ex}}\}$  4:
where ${S}_{A}=\{\phantom{\rule{0.166667em}{0ex}}s\in S\mid \forall \phantom{\rule{0.55542pt}{0ex}}a\in A\phantom{\rule{0.166667em}{0ex}}\exists \phantom{\rule{0.55542pt}{0ex}}u\in U:s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}u\phantom{\rule{0.166667em}{0ex}}\}$  5:
while$\mathcal{C}$ contains a nontrivial constellation C do  6:
choose block ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ from $\mathcal{B}$ in C  7:
replace in $\mathcal{C}$ constellation C by ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ and $C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}$  8:
if C contains probabilistic states then  9:
for all blocks B of action states in $\mathcal{B}$ unstable under ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ or $C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ do  10:
refine $\mathcal{B}$ by splitting B into blocks of states with the same actions into ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ and $C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}$  11:
end for  12:
else  13:
for all blocks B of probabilistic states in $\mathcal{B}$ unstable under ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ do  14:
refine $\mathcal{B}$ by splitting B into blocks of states with equal probabilities into ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$  15:
end for  16:
end if  17:
end while  18:
return$\mathcal{B}$

Algorithm 2 Partition Refinement Algorithm for Probabilistic Bisimulation 

Among others, we preserve the invariant that the blocks in partition $\mathcal{B}$ are always stable under partition $\mathcal{C}$. If all constellations in $\mathcal{C}$ are trivial, then the partitions $\mathcal{B}$ and $\mathcal{C}$ coincide. Hence, the blocks in $\mathcal{B}$ are stable under itself, and according to Lemma 3 we have found a probabilistic bisimulation. Our algorithm works by iteratively refining the set of constellations $\mathcal{C}$. When refining $\mathcal{C}$, we must also refine $\mathcal{B}$ to preserve the above mentioned invariant.
Since the set of states of a PLTS is finite (cf. Definition 1) refinement of the partitions $\mathcal{B}$ and $\mathcal{C}$ cannot be repeated indefinitely. Thus, termination of the algorithm is guaranteed. The partition consisting of singletons of action states and of probabilistic states is the finest that can be obtained, but this is only possible if all states are not bisimilar. In practice, the main loop of the algorithm stops well before reaching that point.
The algorithm maintains the following three invariants:
 Invariant 1.
Probabilistic bisimilarity ${\simeq}_{p}$ is a refinement of $\mathcal{B}$.
 Invariant 2.
Partition $\mathcal{B}$ is a refinement of partition $\mathcal{C}$.
 Invariant 3.
Partition $\mathcal{B}$ is stable under the set of constellations $\mathcal{C}$ (mentioned already above).
Invariant 1 states that if two action states or two probabilistic states are probabilistically bisimilar, then they are in the same block of partition $\mathcal{B}$. Thus, the partitionrefinement algorithm will not separate states if they are bisimilar. By Invariant 2, we have that, at the end and at the start of each iteration, each constellation in $\mathcal{C}$ is a union of blocks in $\mathcal{B}$. Invariant 3 says that blocks in partition $\mathcal{B}$ cannot be split by blocks in constellation $\mathcal{C}$.
In Lines 2 and 3 of Algorithm 1, the set of constellation and the initial partition are set such that the invariants hold. All probabilistic states are put in one block, and all action states with exactly the same actions labelling outgoing transitions are also put together in blocks. (Note the universal quantification over all actions a in A for the set comprehension at Line 4 to ensure that only maximal blocks are included in $\mathcal{B}$ for it being a partition indeed.) The set of constellations contains two constellations namely one with all action states, and one with all probabilistic states. It is straightforward to see that Invariants 1 and 2 hold. Invariant 3 is valid because all transitions from action states go to probabilistic states and vice versa.
Invariants 1–3 guarantee correctness of Algorithm 1. That is, from the invariants, it follows that, upon termination, when all constellations have become trivial, the computed partition $\mathcal{B}$ identifies probabilistically bisimilar action states and probabilistically bisimilar probabilistic states.
Theorem 1. Consider the partition $\mathcal{B}$ resulting from Algorithm 1. We find that (i) two action states are in the same block of $\mathcal{B}$ iff they are probabilistically bisimilar, and (ii) two probabilistic states are in the same block of $\mathcal{B}$ iff they are probabilistically bisimilar.
Proof. Upon termination, because of the while loop of Algorithm 1, all constellations of $\mathcal{C}$ are trivial, i.e. each constellation in $\mathcal{C}$ consists of exactly one block of $\mathcal{B}$. Hence, by Invariant 2, the partitions $\mathcal{B}$ and $\mathcal{C}$ coincide. Thus, by Invariant 3, each block of $\mathcal{B}$ is stable under each block in $\mathcal{B}$. In other words, partition $\mathcal{B}$ is stable under itself.
By the Stability Property of Lemma 3, we have that $\mathcal{B}$ is a probabilistic bisimulation on S. It follows that two action states in the same block of $\mathcal{B}$ are probabilistically bisimilar. Reversely, by Invariant 1, probabilistically bisimilar action states are in the same block of $\mathcal{B}$. Thus, ${\simeq}_{p}$ and $\mathcal{B}$ coincide on S. In other words, two action states are in the same block of $\mathcal{B}$ iff they are probabilistically bisimilar.
To compare ${\simeq}_{p}$ and the relation $\mathcal{B}$ on U, choose probabilistic states $u,v\in U$ such that $u\mathcal{B}v$. Thus, u and v are in the same block of $\mathcal{B}$. By stability of block B for $\mathcal{B}$ it follows that $u\left[{B}^{\prime}\right]=v\left[{B}^{\prime}\right]$, for each block ${B}^{\prime}\subseteq S$. Since ${\simeq}_{p}$ and $\mathcal{B}$ coincide on S this implies $u\left[{B}^{\prime}\right]=v\left[{B}^{\prime}\right]$ for all ${B}^{\prime}\in S/{\simeq}_{p}$. Thus, we have $u{\simeq}_{p}v$. Reversely, if $u{\simeq}_{p}v$, we have $u,v\in B$ for some block B of $\mathcal{B}$ by Invariant 1. Thus, two probabilistic states are in the same block of $\mathcal{B}$ iff they are probabilistically bisimilar. ☐
It is worth noting that in Line 5 of Algorithm 1 an arbitrary nontrivial constellation is chosen and in Line 6 an arbitrary block
${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ is selected from
C (we later put a constraint on the choice of
${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$). In general, there are many possible choices and this influences the way the final partition is calculated. The previous theorem indicates that the final partition is not affected by this choice, neither is the complexity upperbound, see
Section 4.6. However, it is conceivable that practical runtimes can be improved by choosing the nontrivial constellation
C and the block
${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ optimally.
3.3. Refining the Set of Constellations and Restoring the Invariants
As we see from the highlevel description of the partition refinement Algorithm 1, a nontrivial constellation C and a constituent block ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ are chosen (Lines 5 and 6) and C is replaced in $\mathcal{C}$ by the smaller constellations ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ and $C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ (Line 7). This preserves Invariants 1 and 2, but Invariant 3 may be violated as stability under ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ or $C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ (or both) may be lost: On the one hand, it may be the case that two actions states s and t both have an atransition into C, but s may have one to ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ but t to $C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ only or vice versa. On the other hand, it may be the case that two probabilistic states u and v yield the same value for C as a whole, i.e. $u\left[C\right]=v\left[C\right]$, but by no means this needs to hold for ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ or $C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}$, i.e. $u\left[{B}_{\phantom{\rule{0.55542pt}{0ex}}C}\right]\ne v\left[{B}_{\phantom{\rule{0.55542pt}{0ex}}C}\right]$ and $u[C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}]\ne v[C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}]$. Therefore, in the remainder of the body of Algorithm 1, the blocks that are unstable under ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ and $C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ are split such that Invariant 3 is restored, both for blocks of actions states (Lines 9 and 10) and for blocks of probabilistic states (Lines 13 and 14). In the next section, the detailed Algorithm 2 describes how this is done precisely.
The general situation when splitting a block
B for a constellation
C containing a block
${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ is depicted in
Figure 2, at the left where
B contains action states and at the right where
B consists of probabilistic states. We first consider the case at the left.
In this case, block $B\subseteq S$ is stable under constellation $C\subseteq U$ and C is nontrivial. Thus, C properly contains a block ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ of $\mathcal{B}$, and we distinguish two nonempty subsets of C, the block ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ on its own and the remaining blocks together in $C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}$. As B is stable under C, the block B can only be unstable under ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ or $C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ if there is an action $a\in \mathit{Act}$ and a state $s\in B$ such that $s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}{B}_{C}$ (Lemma 2.1). Thus, we only investigate and split blocks, for which such a transition $s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}{B}_{C}$ exists.
We can restore stability by splitting
B into the following three subsets:
Note that the remaining set $\{\phantom{\rule{0.166667em}{0ex}}s\in B\mid s\stackrel{a}{\nrightarrow}{B}_{\phantom{\rule{0.55542pt}{0ex}}C}\wedge s\stackrel{a}{\nrightarrow}C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}\phantom{\rule{0.166667em}{0ex}}\}$ must be empty; if not, this would imply that there is some action state t such that $t\stackrel{a}{\nrightarrow}C$. However, due to the existence of state s such that $s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}{B}_{C}$, this would mean that block B is unstable under C, contradicting Invariant 3.
Checking that the sets ${\mathit{left}}_{a}\left(B\right)$, ${\mathit{mid}}_{a}\left(B\right)$, ${\mathit{right}}_{a}\left(B\right)$ are stable under C is immediate. As subsets of stable sets are also stable (Lemma 1) and B is stable all other configurations of $\mathcal{C}$, the sets ${\mathit{left}}_{a}\left(B\right)$, ${\mathit{mid}}_{a}\left(B\right)$, and ${\mathit{right}}_{a}\left(B\right)$ are stable under all other configurations of $\mathcal{C}$ too.
Note that, due to the existence of state s with $s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}{B}_{C}$, it is not possible that both ${\mathit{left}}_{a}\left(B\right)$ and ${\mathit{mid}}_{a}\left(B\right)$ are equal to the empty set. It is however possible that ${\mathit{left}}_{a}\left(B\right)=B$ or ${\mathit{mid}}_{a}\left(B\right)=B$, leaving the other two sets empty.
Lines 9 and 10 can now be read as follows. For all $a\in \mathit{Act}$, investigate all blocks B such that there is an action state $s\in B$ with $s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}{B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ as these blocks are the only candidates to be unstable. Replace each such block B in $\mathcal{B}$ by $\{{\mathit{left}}_{a}\left(B\right),\phantom{\rule{0.166667em}{0ex}}{\mathit{mid}}_{a}\left(B\right),\phantom{\rule{0.166667em}{0ex}}{\mathit{right}}_{a}\left(B\right)\}\setminus \varnothing $ to restore stability under ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ and $C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}$.
Invariants 1 and 2 are preserved by splitting B. For Invariant 2, this is trivial by construction. For Invariant 1, note that the states in different blocks among ${\mathit{left}}_{a}\left(B\right),\phantom{\rule{0.166667em}{0ex}}{\mathit{mid}}_{a}\left(B\right),\phantom{\rule{0.166667em}{0ex}}{\mathit{right}}_{a}\left(B\right)$ cannot be probabilistically bisimilar as they have unique transitions to states ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ and $C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ and these target states cannot be bisimilar by Invariant 1. Thus, if two states of B are probabilistically bisimilar then both are in the same subset ${\mathit{left}}_{a}\left(B\right)$, ${\mathit{mid}}_{a}\left(B\right)$, or ${\mathit{right}}_{a}\left(B\right)$ of B.
We next turn to the case of a set of probabilistic states
B, see the rightside of
Figure 2. Again, we assume that the nontrivial constellation
C is replaced by its two nonempty subsets
${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ and
$C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}$. As in the previous case, although the block
B is stable under the constellation
C, this may not be the case under the subsets
${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ and
$C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}$.
To restore stability, we now consider for all
q,
$0\u2a7dq\u2a7d1$, the sets
Note that, for finitely many $q\in [0,1]$, we have ${B}_{q}\ne \varnothing $. Observe that each set ${B}_{q}$ is stable under ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ as by construction $u\left[{B}_{\phantom{\rule{0.55542pt}{0ex}}C}\right]=v\left[{B}_{\phantom{\rule{0.55542pt}{0ex}}C}\right]=q$ for any $u,v\in {B}_{q}$. The set ${B}_{q}$ is also stable under $C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}$. To see this consider two states $u,v\in {B}_{q}$. As block $B\subseteq U$ is stable under constellation $C\subseteq S$, $u\left[C\right]=v\left[C\right]$. Hence, $u[C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}]=u\left[C\right]u\left[{B}_{\phantom{\rule{0.55542pt}{0ex}}C}\right]=v\left[C\right]v\left[{B}_{\phantom{\rule{0.55542pt}{0ex}}C}\right]=v[C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}]$. By Lemma 1, the new blocks ${B}_{q}$ are also stable under the other constellations in $\mathcal{C}$.
According to Lemma 2.2, only those blocks B that contain a probabilistic state $u\in B$ such that $u\left[{B}_{\phantom{\rule{0.55542pt}{0ex}}C}\right]>0$ can be unstable under ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ and $C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}$. Thus, at Line 13 of Algorithm 1 we consider all those blocks B and replace each of them by the nonempty subsets ${B}_{q}$, $0\u2a7dq\u2a7d1$ at Line 14 in $\mathcal{B}$. This makes the partition stable again under all constellations in $\mathcal{C}$, in particular under the new constellations ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ and $C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}$.
Again, it is straightforward to see that Invariants 1 and 2 are not violated by replacing the block B by the blocks ${B}_{q}$. For Invariant 1, if states are probabilistically bisimilar in B, they remain in the same block ${B}_{q}$. For Invariant 2, as B is refined, partition $\mathcal{B}$ remains a refinement of partition $\mathcal{C}$.
For the detailed algorithm in
Section 4, it is required to group the sets
${B}_{q}$ as follows:
${\mathit{left}}_{p}\left(B\right):={B}_{0}$,
${\mathit{right}}_{p}\left(B\right):={B}_{1}$, and
${\mathit{mid}}_{p}\left(B\right)=\{\phantom{\rule{0.166667em}{0ex}}{B}_{q}\mid 0<q<1\phantom{\rule{0.166667em}{0ex}}\}$. This does not play a role here, but
${\mathit{left}}_{p}\left(B\right)$,
${\mathit{mid}}_{p}\left(B\right)$, and
${\mathit{right}}_{p}\left(B\right)$ are already indicated in
Figure 2, in particular
${\mathit{mid}}_{p}\left(B\right)=\{{B}_{\frac{1}{4}},{B}_{\frac{1}{2}},{B}_{\frac{3}{4}}\}$.
3.4. An Example
We provide an example to illustrate how Algorithm 1 calculates partitions.
Example 1. Consider the PLTS given in Figure 3. We provide a detailed account of the partitions that are obtained when calculating probabilistic bisimulation. The obtained partitions are listed in Table 1. In the lower table, nine partitions together with their constellations are listed that are generated for a run of Algorithm 1. In the upper table the blocks that occur in these partitions are defined. Observe that we put the blocks and constellations with action states and probabilistic states in different columns. This is only for clarity, as in the current partition and the current set of constellations they are joined. Algorithm 1 starts with four blocks of action states,
${S}_{0}$ to
${S}_{3}$, which contain the action states with no outgoing transitions and those with an outgoing transition labelled with
a, with
b, and with
c, respectively. In the algorithm, all probabilistic states are initially collected in block
${U}_{0}$. There are two constellations, viz.
${S}_{0}\cup {S}_{1}\cup {S}_{2}\cup {S}_{3}$ and
${U}_{0}$. These initial partitions are listed in R0w 0 of the lower part of
Table 1.
Since the constellation with action states is nontrivial we split it, rather arbitrarily, in
${S}_{0}$ and
${S}_{1}\cup {S}_{2}\cup {S}_{3}$. The block
${U}_{0}$ is not stable under
${S}_{0}$ and
${S}_{1}\cup {S}_{2}\cup {S}_{3}$ and is split in
${U}_{1}=\left\{{u}_{1},{u}_{3},{v}_{15}\right\}$,
${U}_{2}=\left\{{u}_{2},{u}_{4}\right\}$ and
${U}_{3}=\left\{{u}_{5},{u}_{6}\right\}$. This is because we have
$u\left[{S}_{0}\right]=1$ for
u equal to
${u}_{1}$,
${u}_{3}$, and
${v}_{1}$ to
${v}_{5}$; we have
$u\left[{S}_{0}\right]=\frac{1}{2}$ for
u equal to
${u}_{2}$ and
${u}_{4}$; we have
${u}_{5}\left[{S}_{0}\right]=0$ and
${u}_{6}\left[{S}_{0}\right]=0$. The resulting partitions are listed at Row 1 in
Table 1.
For the second iteration, we consider the nontrivial constellation ${S}_{1}\cup {S}_{2}\cup {S}_{3}$ and split it into ${S}_{1}$ and ${S}_{2}\cup {S}_{3}$. Note, the action states ${s}_{1}$ to ${s}_{4}$ in ${S}_{1}$ do not have incoming transitions. Consequently, for all $u\in {U}_{1}$, we have $u\left[{S}_{1}\right]=0$; for all $u\in {U}_{2}$ we have $u\left[{S}_{1}\right]=0$; for all $u\in {U}_{3}$ we have $u\left[{S}_{1}\right]=0$. Thus, all blocks of probabilistic states are stable under ${S}_{1}$ and ${S}_{2}\cup {S}_{3}$. Hence, no block is split.
In the third iteration, we split the nontrivial constellation ${S}_{2}\cup {S}_{3}$ into ${S}_{2}$ and ${S}_{3}$. For all, $u\in {U}_{1}$ we have $u\left[{S}_{2}\right]=0$. Thus, ${U}_{1}$ is stable under ${S}_{2}$ and ${S}_{3}$. For ${U}_{2}$, the probabilistic states ${u}_{2}$ and ${u}_{4}$ agree on the value $\frac{1}{2}$ for ${S}_{2}$, hence for ${S}_{3}$ too. Thus, ${U}_{2}$ is stable as well. However, for ${u}_{5}$ and ${u}_{6}$ in ${U}_{3}$ we have ${u}_{5}\left[{S}_{2}\right]=1$ and ${u}_{6}\left[{S}_{2}\right]=\frac{1}{3}$. Therefore, ${U}_{1}$ needs to be split in ${U}_{4}=\left\{{u}_{5}\right\}$ and ${U}_{5}=\left\{{u}_{6}\right\}$.
At this point, all constellations with actions states are trivial, so at iteration 4 we turn to the nontrivial constellation of probabilistic states ${U}_{1}\cup {U}_{2}\cup {U}_{4}\cup {U}_{5}$ and split it into ${U}_{1}$ and ${U}_{2}\cup {U}_{4}\cup {U}_{5}$. Block ${S}_{0}$ is stable since each of its states has no transitions at all. Block ${S}_{1}$ is not stable: ${s}_{1},{s}_{2}\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}{U}_{1}$ and ${s}_{1},{s}_{2}\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}{U}_{2}\cup {U}_{4}\cup {U}_{5}$, but ${s}_{3},{s}_{4}\stackrel{a}{\nrightarrow}{U}_{1}$ and ${s}_{3},{s}_{4}\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}{U}_{2}\cup {U}_{4}\cup {U}_{5}$. Thus, ${S}_{1}$ needs to be split into ${S}_{4}=\left\{{s}_{1},{s}_{2}\right\}$ and ${S}_{5}=\left\{{s}_{3},{s}_{4}\right\}$. Block ${S}_{2}$ is stable since its states have only btransitions into ${U}_{1}$. Block ${S}_{3}$ is a singleton and therefore cannot be split.
The following iteration, Iteration 5, sets ${U}_{2}$ and ${U}_{4}\cup {U}_{5}$ apart as constellations. Again, in absence of transitions, block ${S}_{0}$ is stable under ${U}_{2}$ and ${U}_{4}\cup {U}_{5}$. The same holds for ${S}_{2}$ that has only btransitions into ${U}_{0}$. Block ${S}_{3}$ can be ignored. For ${S}_{4}$, both ${s}_{1}$ and ${s}_{2}$ have an atransition into ${U}_{2}$ as their only transition. Hence, block ${S}_{4}$ is stable. Similarly, ${S}_{5}$ is stable, as its states ${s}_{3}$ and ${s}_{4}$ both have an atransition into ${U}_{4}\cup {U}_{5}$ and no other transitions. Overall, in this iteration, no blocks require splitting to restore Invariant 3.
Next, at Iteration 6, we split nontrivial constellation ${U}_{4}\cup {U}_{5}$ into ${U}_{4}$ and ${U}_{5}$. For ${S}_{0}$, ${S}_{2}$, ${S}_{3}$ and ${S}_{4}$ we conclude stability in the same way as in the previous iteration. However, now we have for ${s}_{3},{s}_{4}\in {S}_{5}$ on the one hand ${s}_{3}\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}{U}_{4}$ and ${s}_{3}\stackrel{a}{\nrightarrow}{U}_{5}$, but on the other hand ${s}_{4}\stackrel{a}{\nrightarrow}{U}_{4}$ and ${s}_{4}\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}{U}_{5}$. Hence, ${S}_{5}$ needs to be split, yielding the singletons ${S}_{6}=\left\{{s}_{3}\right\}$ and ${S}_{7}=\left\{{s}_{4}\right\}$.
Returning to constellations of actions states, at Iteration 7, we split ${S}_{4}\cup {S}_{6}\cup {S}_{7}$ over ${S}_{4}$ and ${S}_{6}\cup {S}_{7}$. All probabilistic states have value 0 for both ${S}_{4}$ and ${S}_{6}\cup {S}_{7}$, hence no split of probabilistic blocks is needed.
This is similar in Iteration 8, where the nontrivial constellation ${S}_{6}\cup {S}_{7}$ is split, and none of the blocks become unstable. Now, all constellations are trivial and the algorithm terminates. According to the Stability Property, Lemma 3, the corresponding equivalence relation is a probabilistic bisimulation. Thus, the final partition is $\left\{{S}_{0},{S}_{2},{S}_{3},{S}_{4},{S}_{6},{S}_{7},{U}_{1},\phantom{\rule{0.166667em}{0ex}}{U}_{2},\phantom{\rule{0.166667em}{0ex}}{U}_{4},\phantom{\rule{0.166667em}{0ex}}{U}_{5}\right\}$. Moreover, the deadlock states ${t}_{1},{t}_{3},{t}_{4},{t}_{6},{t}_{7}$ and ${r}_{1}$ to ${r}_{5}$ are probabilistically bisimilar, the states ${t}_{2},{t}_{5},{t}_{8},{t}_{9}$ that have only a btransition into a Dirac distribution to deadlock are probabilistically bisimilar, the states ${s}_{1}$ and ${s}_{2}$ are probabilistically bisimilar (which is clear when identifying states ${t}_{7}$ and ${t}_{8}$), whereas the remaining action states ${s}_{3}$, ${s}_{4}$ and ${t}_{10}$ have no probabilistically bisimilar counterpart. For the probabilistic states the states ${u}_{1}$, ${u}_{3}$ and ${v}_{1}$ to ${v}_{5}$ are identified by probabilistic bisimulation. This also holds for the probabilistic states ${u}_{2}$ and ${u}_{4}$. Probabilistic states ${u}_{5}$ and ${u}_{6}$ each have no probabilistically bisimilar counterpart.
4. A PartitionRefinement Algorithm for Probabilistic Bisimulation (Detailed)
Algorithm 1 gives an outline but leaves many details implicit. The detailed refinementpartition algorithm is presented in this section as Algorithm 2. It has the same structure as Algorithm 1, but in this section we focus on how to efficiently calculate whether and how blocks must be split, and how this split is actually carried out. We first explain grouping of action transitions per action, next we introduce various data structures that are used by the algorithm, subsequently we explain how the algorithm is working linebyline, and finally we give an account of its complexity.
4.1. Grouping Action Transitions per Action Label
To obtain the complexity bound of our algorithm, it is essential that we can group action transitions by actions linearly in the number of transitions. Grouping means that the action transitions with the same action occur consecutively in this ordering. It is not necessary that the transitions are ordered according to some overall ordering.
We assume that $\left\mathit{Act}\right\u2a7d{m}_{a}$ and that the actions in $\mathit{Act}$ are consecutively numbered. Recall, ${m}_{a}$ denotes the number of transitions $s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}u$. These assumptions are easily satisfied, by removing those actions in $\mathit{Act}$ that are not used in transitions and by sorting and numbering the remaining action labels. Sorting these actions adds a negligible $O\left(\right\mathit{Act}log\mathit{Act}\left\right)\u2a7dO({m}_{a}log{m}_{a})$.
Grouping transitions is performed by an array of buckets indexed with actions. All transitions are put in the appropriate bucket in constant time exploiting actions being numbered. Furthermore, all buckets that contain transitions are linked together. When all transitions are in the buckets, a straightforward traversal of all linked buckets provides the transitions in a grouped order. This requires time linear in the number of considered action transitions. Note that the number of buckets is equal to $\left\mathit{Act}\right\u2a7d{m}_{a}$ and, therefore, the buckets do not require more than linear memory.
4.2. Data Structures
We give a concise overview of the concrete data structures in the algorithm for states, transitions, blocks, and constellations. We list the names of the fields in these data structures in a programming vein to keep a close link with the actual implementation.
The chosen data structures are not particularly optimised. Exploiting ideas from [
6,
24,
25] to store states, blocks, and constellations, usage of time and memory can be further reduced. All data structures come in two flavours, one related to actions and the other related to probabilities. We treat them simultaneously and only mention their differences when appropriate.
4.2.1. Global
In the detailed algorithm, there are arrays containing transitions, actions, blocks and constellations. There is a stack of nontrivial constellations to identify in constant time which constellation must be investigated in the main loop. Furthermore, there is an array containing the variables $\mathit{state}\_\mathit{to}\_\mathit{constellation}\_\mathit{cnt}$, which are explained below.
For all action transitions
$s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}u$, it is maintained how many action transitions there are labelled with the same action
a, and that go from
s to the constellation
C containing
u. This value is called
$\mathit{state}\_\mathit{to}\_\mathit{constellation}\_\mathit{cnt}$ for this transition. The value is required to efficiently split probabilistic blocks (the idea of using such variables stems from [
5]). For each state
s, constellation
C, and action
a there is one instance of
$\mathit{state}\_\mathit{to}\_\mathit{constellation}\_\mathit{cnt}$ stored in a global array. Each transition
$s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}u$ contains a reference called
$\mathit{state}\_\mathit{to}\_\mathit{constellation}\_\mathit{cnt}\_\mathit{ptr}$ to the appropriate value in this array. See
Figure 3 for a graphical illustration with a constellation
C of probabilistic states and blocks
${B}_{1}$ and
${B}_{2}$ of action states. The purpose of this construction is that
$\mathit{state}\_\mathit{to}\_\mathit{constellation}\_\mathit{cnt}$ can be changed by one operation for all transitions from the same state with the same action to the same constellation, simultaneously.
4.2.2. Transition
Each transition consists of the fields $\mathit{from}$, $\mathit{label}$ and $\mathit{to}$. Here, $\mathit{from}$ and $\mathit{to}$ refer to an action/probabilistic state, and $\mathit{label}$ is the action label or probabilistic label of the transition. The action labels are consecutive numbers; the probabilistic labels are exact fractions. Action transitions also contain a reference $\mathit{state}\_\mathit{to}\_\mathit{constellation}\_\mathit{cnt}\_\mathit{ptr}$ to the variable $\mathit{state}\_\mathit{to}\_\mathit{constellation}\_\mathit{cnt}$ as indicated above.
4.2.3. State
Each action state and probabilistic state contains a list of incoming transitions and a reference to the block in which the state resides. For intermediate calculations, each state contains a boolean $\mathit{mark}\_\mathit{state}$ which is used to indicate that a state has been marked. Each action state also contains two more variables for temporary use. When deciding whether blocks need to be split, the variable $\mathit{residual}\_\mathit{transition}\_\mathit{cnt}$ indicates how many residual transitions there are to blocks $C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ when splitting takes place by a block ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$. The variable $\mathit{transition}\_\mathit{cnt}\_\mathit{ptr}$ is used to let the variable $\mathit{state}\_\mathit{to}\_\mathit{constellation}\_\mathit{cnt}\_\mathit{ptr}$ for an action transition point to a new instance of $\mathit{state}\_\mathit{to}\_\mathit{constellation}\_\mathit{cnt}$ when this transitions is moved to a new block. In probabilistic states, there is the temporary variable $\mathit{cumulative}\_\mathit{prob}$ used to calculate the total probability to reach a block under splitting.
4.2.4. Block
Blocks contain an indication of the constellation in which it occurs, a list of the states contained in the block including the size of this list, and a list of transitions ending in this block. For blocks of action states, this list of transitions is grouped by action label, i.e., transitions with the same action label are a consecutive sublist. For temporary use, there is also a variable to indicate that the block is marked. This marking contains exactly the information that the functions $\mathit{aMark}$ and $\mathit{pMark}$, discussed below, provide for blocks of action states and blocks of probabilistic states, respectively.
4.2.5. Constellation
Finally, constellations contain a list of the blocks in the constellation as well as the cumulative number of states contained in all blocks in this constellation.
4.3. Explanation of the Detailed Algorithm
Algorithm 1 focuses on how, by refining partitions and sets of constellations, probabilistic bisimulation can be calculated. In Algorithm 2, we stress the details of carrying out concrete refinement steps to realise the required time bound. As already indicated, the overall structure of both algorithms is the same.
The initial Lines 2 and 3 of Algorithm 2 are the same as those of Algorithm 1. In Line 3, the partition
$\mathcal{B}$ is set to contain one block with all probabilistic states and a number of blocks of action states, grouped per common outgoing action labels. Thus, two action states are in the same block initially if their menu, i.e., the set of actions for which there is a transition, is identical. This initial partition
$\mathcal{B}$ is calculated using a simple partition refinement algorithm on outgoing transitions of states. This operation is linear in the number of outgoing action transitions when using grouping of transitions as explained in
Section 4.1.
At Line 4, the incoming transitions are ordered on actions as indicated in
Section 4.1. At Line 5, an array with one instance of
$\mathit{state}\_\mathit{to}\_\mathit{constellation}\_\mathit{cnt}$ for each action label is made where each instance contains the number of action transitions that contain that action label. The reference
$\mathit{state}\_\mathit{to}\_\mathit{constellation}\_\mathit{cnt}$ for each action transition is set to refer to the appropriate instance in this array. This is done by simply traversing all transitions
$s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}u$ grouped by action labels and incrementing the appropriate entry in the array containing all
$\mathit{state}\_\mathit{to}\_\mathit{constellation}\_\mathit{cnt}$ variables. The appropriate entry can be found using the temporary variable
$\mathit{transition}\_\mathit{cnt}\_\mathit{ptr}$ associated to state
s. If no entry for
$\mathit{state}\_\mathit{to}\_\mathit{constellation}\_\mathit{cnt}$ exists yet, the variable
$\mathit{transition}\_\mathit{cnt}\_\mathit{ptr}$ belonging to
s is
null and an appropriate entry must be created.
In Line 6, selecting a nontrivial constellation is straightforward, as a stack of nontrivial constellations is maintained. Initially, this stack contains $\mathcal{C}=\{\mathcal{S},\mathcal{U}\}$. To obtain the required time complexity, we select ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ such that ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}\u2a7d\frac{1}{2}\leftC\right$ in Line 8. This is done in constant time as we know the number of states in C. Hence, either the first or second block B of constellation C satisfies that $\leftB\right\u2a7d\frac{1}{2}\leftC\right$ (for if the first block contains more than half the states the second one cannot). We replace the constellation C by ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ and $C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ in $\mathcal{C}$, see Line 8, and put the constellation $C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ on the stack of nontrivial constellations if it is nontrivial.
From Line 9 to Line 19, the partition $\mathcal{B}$ is refined to restore the invariants, especially Invariant 3. This is done by first marking the blocks (Line 11 and Line 16) such that it is clear how they must be split, and by subsequently splitting the blocks (Lines 12–14, and Lines 17–19). Both operations are described in the next two subsections.
4.4. Marking
Given a constellation C that contains a block ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ and in the case of an action transition, an action a, we need to know which blocks need to be split in what way. This is calculated using the functions $\mathit{aMark}(\mathcal{B},C,{B}_{\phantom{\rule{0.55542pt}{0ex}}C},a)$ and $\mathit{pMark}(\mathcal{B},C,{B}_{\phantom{\rule{0.55542pt}{0ex}}C})$. The first one is for marking blocks with respect to action transitions, the second for marking blocks with respect to probabilities.
Both functions yield a fivetuple $\langle \mathit{B},\mathit{left},\mathit{mid},\mathit{right},and\phantom{\rule{3.33333pt}{0ex}}\mathit{large}\rangle $. Here, $\mathit{B}\subseteq \mathcal{B}$ is a set of blocks that may have to be split and $\mathit{left}$, $\mathit{mid}$, and $\mathit{right}$ are functions that together for each block $B\in \mathit{B}$ provide the sets into which B must be partitioned. The set $\mathit{large}\left(B\right)$ is the largest set among them. For every set ${B}^{\prime}$ in which B must be partitioned, except for $\mathit{large}\left(B\right)$, it holds that ${B}^{\prime}\u2a7d\frac{1}{2}\leftB\right$. To obtain the complexity bound, we only move such small blocks out of B, i.e., those blocks not equal to $\mathit{large}\left(B\right)$.
We note that sets in $\mathit{left}\left(B\right)$, $\mathit{mid}\left(B\right)$ and $\mathit{right}\left(B\right)$ can be empty. Such sets can be ignored. It is also possible that there is only one nonempty set being equal to B itself. In this case, B is stable under ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ and $C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}$. Furthermore, it is equal to $\mathit{large}\left(B\right)$ and therefore B is kept intact.
We now concentrate on the function
$\mathit{aMark}(\mathcal{B},C,{B}_{\phantom{\rule{0.55542pt}{0ex}}C},a)$ with a partition
$\mathcal{B}$, a constellation
C, a block
${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ contained in
C, and an action
a. In this situation,
C is a nontrivial constellation of probabilistic states. Since
C contains probabilistic states only, incoming transitions for states in
${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ are action transitions. The situation is depicted in
Figure 2, at the left. The call
$\mathit{aMark}(\mathcal{B},C,{B}_{\phantom{\rule{0.55542pt}{0ex}}C},a)$ returns the tuple
$\langle {\mathit{B}}_{a},\phantom{\rule{0.166667em}{0ex}}{\mathit{left}}_{a},\phantom{\rule{0.166667em}{0ex}}{\mathit{mid}}_{a},\phantom{\rule{0.166667em}{0ex}}{\mathit{right}}_{a},\phantom{\rule{0.166667em}{0ex}}{\mathit{large}}_{a}\rangle $ defined as follows.
We calculate ${\mathit{B}}_{a}$ by traversing the list of all transitions with action a going into ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ and adding each block containing any source state of these transitions to ${\mathit{B}}_{a}$. The blocks in ${\mathit{B}}_{a}$ are the only blocks that may be unstable under ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ and $C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ with respect to a (Lemma 2).
The for loop at Line 10 iterates over all actions. As the incoming transitions into block ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ are grouped per action, all incoming transitions with the same action can easily be processed together, while the total processing time is linear in the number of incoming transitions. However, note that calculating ${\mathit{B}}_{a}$ is based on partition $\mathcal{B}$, while $\mathcal{B}$ is refined at Line 14. Thus, the calculation of ${\mathit{B}}_{a}$ for different actions a can be based on repeatedly refined partitions $\mathcal{B}$.
Next, we discuss how to construct the blocks ${\mathit{left}}_{a}\left(B\right)$, ${\mathit{mid}}_{a}\left(B\right)$, and ${\mathit{right}}_{a}\left(B\right)$. While traversing alabelled transitions into ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$, all action states in a block B with an atransition into ${B}_{C}$ are marked and (temporarily) moved into ${\mathit{left}}_{a}\left(B\right)$. The remaining states in block B form the subset ${\mathit{right}}_{a}\left(B\right)$. We keep track of the number of states in a block. Thus, we can easily maintain the size of ${\mathit{right}}_{a}\left(B\right)$.
To find out which states now in ${\mathit{left}}_{a}\left(B\right)$ must be transferred to ${\mathit{mid}}_{a}\left(B\right)$, the variables $\mathit{state}\_\mathit{to}\_\mathit{constellation}\_\mathit{cnt}$ are used. Recall that these variables record for each transition $s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}u$, with $u\in S$, how many transitions $s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}v$ there are to states $v\in C$. These variables are initialised in Line 5 of Algorithm 2. When the first state is moved to ${\mathit{left}}_{a}\left(B\right)$, we copy the value of $\mathit{state}\_\mathit{to}\_\mathit{constellation}\_\mathit{cnt}$ of transition $s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}u$ to the variable $\mathit{residual}\_\mathit{transition}\_\mathit{cnt}$ belonging to state s of the transition, subtracted by one. The number $\mathit{residual}\_\mathit{transition}\_\mathit{cnt}$ indicates how many unvisited atransitions are left from the state s into C. Every time an atransition is visited of which the source state is already in ${\mathit{left}}_{a}\left(B\right)$, we decrease $\mathit{residual}\_\mathit{transition}\_\mathit{cnt}$ of the source state by one again. If all atransitions into ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ have been visited, the number $\mathit{residual}\_\mathit{transition}\_\mathit{cnt}$ of a state s indicates how many transitions labelled a go from s into $C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}$.
Subsequently, we traverse the states in ${\mathit{left}}_{a}\left(B\right)$. If a state s has a nonzero $\mathit{residual}\_\mathit{transition}\_\mathit{cnt}$, we know that there are atransitions from s to both ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ and $C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}$. Therefore, we move state s into ${\mathit{mid}}_{a}\left(B\right)$. Otherwise, all transitions from s with action a go to ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ and s must remain in ${\mathit{left}}_{a}\left(B\right)$.
While moving states into ${\mathit{left}}_{a}\left(B\right)$ and ${\mathit{mid}}_{a}\left(B\right)$, we also keep track of the sizes of these sets. Hence, it is easy to indicate in ${\mathit{large}}_{a}\left(B\right)$ which set is the largest.
We calculate
$\mathit{pMark}(\mathcal{B},C,{B}_{\phantom{\rule{0.55542pt}{0ex}}C})$ in a slightly different manner than
$\mathit{aMark}$. In particular, we have
${\mathit{mid}}_{p}:\mathit{B}\to {2}^{\phantom{\rule{0.55542pt}{0ex}}{2}^{U}}$, i.e.,
${\mathit{mid}}_{p}\left(B\right)$ is a set of blocks. This indicates that the block
B can be partitioned in many sets, contrary to the situation with action blocks where
B could be split in at most three blocks. The situation is depicted in
Figure 2 at the right. The fivetuple that
$\mathit{pMark}$ returns has the following components:
The above is obtained by traversing through all incoming probabilistic transitions in
${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$. Whenever there is a state
u in a block
B such that
$u{\mapsto}_{\phantom{\rule{1.111pt}{0ex}}p}{B}_{\phantom{\rule{0.55542pt}{0ex}}C}$, one of the following cases applies:
If B is not in ${\mathit{B}}_{p}$ yet, it is added now. The variable $\mathit{cumulative}\_\mathit{prob}$ in state u is set to p, and u is (temporarily) moved from B to ${\mathit{left}}_{p}\left(B\right)$.
If B is already in ${\mathit{B}}_{p}$, then the probability p is added to $\mathit{cumulative}\_\mathit{prob}$ of state u.
After the traversal of all incoming probabilistic transitions into ${B}_{C}$, the variable $\mathit{cumulative}\_\mathit{prob}$ of u contains $u\left[{B}_{\phantom{\rule{0.55542pt}{0ex}}C}\right]$, i.e., the probability to reach ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ from the state u.
Those states that are left in
B form the set
${\mathit{right}}_{p}\left(B\right)$. We know the number of states in
${\mathit{right}}_{p}\left(B\right)$ by keeping track how many states were moved to
${\mathit{left}}_{p}\left(B\right)$. Next, the states temporarily stored in
${\mathit{left}}_{p}\left(B\right)$ must be distributed over
${\mathit{left}}_{p}\left(B\right)$ and
${\mathit{mid}}_{p}\left(B\right)$. First, all states with
$\mathit{cumulative}\_\mathit{prob}<1$ are moved into some set
M such that
${\mathit{left}}_{p}\left(B\right)$ contains exactly the states with
$\mathit{cumulative}\_\mathit{prob}=1$. Then, the states in
M are sorted on their value for
$\mathit{cumulative}\_\mathit{prob}$ such that it is easy to move all states with the same
$\mathit{cumulative}\_\mathit{prob}$ into separate sets in
${\mathit{mid}}_{p}\left(B\right)$. In
Figure 2, at the right, the set
${\mathit{mid}}_{p}\left(B\right)$ consists of three sets, corresponding to the probabilities
$q=\frac{1}{4}$,
$q=\frac{1}{2}$ and
$q=\frac{3}{4}$ to reach
${B}_{C}$. Note that all processing steps mentioned require time proportional to the number of incoming probabilistic transitions in
${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$, except for the time to sort. In the complexity analysis below, it is explained that the cumulative sorting time is bounded by
$\mathit{O}\phantom{\rule{0.55542pt}{0ex}}\left(\right)open="("\; close=")">{m}_{p}log{n}_{p}$.
By traversing the sets of states in ${\mathit{left}}_{p}\left(B\right)$ and ${\mathit{mid}}_{p}\left(B\right)$ once more, we can determine which set among ${\mathit{left}}_{p}\left(B\right)$, ${\mathit{right}}_{p}\left(B\right)$, and the set of sets ${\mathit{mid}}_{p}\left(B\right)$ contains the largest number of probabilistic states. This set is reported in ${\mathit{large}}_{p}\left(B\right)$.
4.5. Splitting
In Lines 14 and 19 of Algorithm 2, a block ${B}^{\prime}$ is moved out of the existing block B. By the marking procedure, either $\mathit{aMark}$ or $\mathit{pMark}$, the states involved are already put in separate lists and are moved in constant time to the new block B’.
Blocks contain lists of incoming transitions. When moving the states to a new block, the incoming transitions are moved by traversing the incoming transitions of each moved state, removing them from the list of incoming transitions of the old block and inserting them in the same list for the new block. There is a complication, namely that incoming action transitions must be grouped by action labels. This is done separately for the transitions moved to
${B}^{\prime}$ as explained in
Section 4.1 and this is linear in the number of transitions being moved. When removing incoming action transitions from the old block
B, the ordering of the transitions is maintained. Thus, the grouping of incoming action transitions into
B remains intact without requiring extra work.
When moving action states to a new block we also need to adapt the variable $\mathit{state}\_\mathit{to}\_\mathit{constellation}\_\mathit{cnt}$ for each action transition $s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}C$ with state $s\in B$. Observe that this only needs to be done if there are some atransitions to ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ and some to $C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}$, which means that $s\in {\mathit{mid}}_{a}\left(B\right)$. In that case $\mathit{residual}\_\mathit{transition}\_\mathit{cnt}$ for state s is larger than 0.
This is accomplished by traversing all incoming transitions $s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}u$ into ${B}_{C}$ one extra time. If $\mathit{residual}\_\mathit{transition}\_\mathit{cnt}$ for s is larger than 0 we need to replace the $\mathit{state}\_\mathit{to}\_\mathit{constellation}\_\mathit{cnt}$ for this transition $s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}u$ by the value of $\mathit{state}\_\mathit{to}\_\mathit{constellation}\_\mathit{cnt}\mathit{residual}\_\mathit{transition}\_\mathit{cnt}$ of s. For all nonvisited transitions $s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}{u}^{\prime}$ where ${u}^{\prime}\in C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}$, the value of $\mathit{state}\_\mathit{to}\_\mathit{constellation}\_\mathit{cnt}$ must be set to $\mathit{residual}\_\mathit{transition}\_\mathit{cnt}$ of s.
This is where we use that
$\mathit{state}\_\mathit{to}\_\mathit{constellation}\_\mathit{cnt}$ is actually referred to by the pointer
$\mathit{state}\_\mathit{to}\_\mathit{constellation}\_\mathit{cnt}\_\mathit{ptr}$ (see
Figure 3). When traversing the first transition of the form
$s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}u$ with
$u\in {B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ such that
$\mathit{residual}\_\mathit{transition}\_\mathit{cnt}$ for
s is larger than 0, a new entry in the array containing the variables
$\mathit{state}\_\mathit{to}\_\mathit{constellation}\_\mathit{cnt}$ is constructed containing the value
$\mathit{state}\_\mathit{to}\_\mathit{constellation}\_\mathit{cnt}\mathit{residual}\_\mathit{transition}\_\mathit{cnt}$ and the auxiliary variable
$\mathit{transition}\_\mathit{cnt}\_\mathit{ptr}$ is used to point to this entry. At the same time, the value in old entry in this array for
$\mathit{state}\_\mathit{to}\_\mathit{constellation}\_\mathit{cnt}$ is replaced by the value
$\mathit{residual}\_\mathit{transition}\_\mathit{cnt}$ of state
s. In this way, the values of
$\mathit{state}\_\mathit{to}\_\mathit{constellation}\_\mathit{cnt}$ of all transitions labelled with
a from
s to
$C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ are updated in constant time, i.e., without visiting the transitions that are not moved. For all transitions
$s\stackrel{a}{\to}\phantom{\rule{1.111pt}{0ex}}{u}^{\prime}$ with
${u}^{\prime}\in {B}_{\phantom{\rule{0.55542pt}{0ex}}C}$, the variable
$\mathit{state}\_\mathit{to}\_\mathit{constellation}\_\mathit{cnt}\_\mathit{ptr}$ is made to refer the new entry in the array.
4.6. Complexity Analysis
The complexity of the algorithm is determined below. Recall that ${n}_{a}$ and ${n}_{p}$ are the number of action states and probabilistic states, respectively, while ${m}_{a}$ is the number of action transitions and ${m}_{p}$ is the cumulative size of the supports of the distributions.
Theorem 2. The total time complexity of the algorithm is $\mathit{O}\phantom{\rule{0.55542pt}{0ex}}\left(\right)open="("\; close=")">({m}_{a}+{m}_{p})log{n}_{p}+({m}_{p}+{n}_{a})log{n}_{a}$ and the space complexity is $\mathit{O}\phantom{\rule{0.55542pt}{0ex}}\left(\right)open="("\; close=")">{m}_{a}+{m}_{p}+{n}_{a}$.
Proof. In Algorithm 2, the cost of each computation step is indicated. The initialisation of the algorithm at Lines 2–5 is linear in
${n}_{a}$,
${n}_{p}$ and
${m}_{a}$. At Line 3, calculating
$\{{S}_{A}\mid A\subseteq \mathit{Act}\}$ can be done by iteratively splitting
S using the outgoing transitions grouped per action label. This is linear in the number of action transitions. At Line 4, grouping the incoming transitions per action is also linear as argued in
Section 4.1.
The while loop at Line 6 is executed for each ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}\subseteq C$ where ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}\u2a7d\frac{1}{2}\leftC\right$. As ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ becomes a constellation itself, each state can only be part of this splitting step ${log}_{2}\left({n}_{a}\right)$ times and ${log}_{2}\left({n}_{p}\right)$ times, respectively. The steps in Lines 10–13 and Lines 16–18 require steps proportional to the number of incoming action transitions and probabilistic transitions, respectively, in ${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$, apart from a sorting penalty which we treat separately below. The cumulative complexity of this part is therefore $\mathit{O}\phantom{\rule{0.55542pt}{0ex}}\left(\right)open="("\; close=")">{m}_{a}log{n}_{p}+{m}_{p}log{n}_{a}$.
At Lines 14 and 19, the states in ${B}^{\prime}$ are moved to a new block. This requires to group the incoming action transitions in a block ${B}^{\prime}$ per action, which can be done in time linear in the number of these transitions. Block ${B}^{\prime}$ is not the largest block of B considered and therefore ${B}^{\prime}\u2a7d\frac{1}{2}\leftB\right$. Hence, each state can only be ${log}_{2}\left({n}_{p}\right)$ or ${log}_{2}\left({n}_{a}\right)$ times be involved in the operation to move to a new block. Hence, the total time to be attributed to moving is $\mathit{O}\phantom{\rule{0.55542pt}{0ex}}\left(\right)open="("\; close=")">({m}_{a}+{n}_{p})log{n}_{p}+({m}_{p}+{n}_{a})log{n}_{a}$.
While marking, probabilistic states in
${\mathit{mid}}_{p}\left(B\right)$ need to be sorted. An ingenious argument by Valmari and Franceschinis [
6] shows that this will at most contribute
$\mathit{O}\phantom{\rule{0.55542pt}{0ex}}\left(\right)open="("\; close=")">{m}_{p}log{n}_{p}$ to the total complexity: Let
K be the total number of times sorting takes place. Assume, for
$1\u2a7di\u2a7dK$, that the total number of distributions in
${\mathit{mid}}_{p}\left(B\right)$ when sorting it for the
ith time is
${k}_{i}$. Clearly,
${k}_{i}\u2a7d{n}_{p}$. Each time a distribution in
${\mathit{mid}}_{p}\left(B\right)$ is involved in sorting, the number of reachable constellations with nonzero probability from this distribution is increased by one. Before sorting it could reach
C, and after sorting it can reach both new constellations
${B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ and
$C\setminus {B}_{\phantom{\rule{0.55542pt}{0ex}}C}$ with nonzero probability. Note that this does not hold for the states in
${\mathit{left}}_{p}\left(B\right)$ and
${\mathit{right}}_{p}\left(B\right)$, and this is the reason why we have to treat them separately. In particular, to obtain complexity
$O({m}_{p}log{n}_{p})$, it is not allowed to involve the states in
${\mathit{left}}_{p}\left(B\right)$ and
${\mathit{right}}_{p}\left(B\right)$ in the sorting process as shown by an example in [
6]. Due to the increased number of reachable constellations, the total number of times a probabilistic state can be involved in sorting is bounded by the size of the distribution. In other words,
${\sum}_{\phantom{\rule{0.166667em}{0ex}}i=1}^{K}\phantom{\rule{0.166667em}{0ex}}{k}_{i}\u2a7d{m}_{p}$. Hence, the total time that is required by sorting is bounded as follows:
Adding up the complexities leads to the conclusion that the total complexity of the algorithm is
$\mathit{O}\phantom{\rule{0.55542pt}{0ex}}\left(\right)open="("\; close=")">({m}_{a}+{m}_{p}+{n}_{p})log{n}_{p}+({m}_{p}+{n}_{a})log{n}_{a}$. As
${m}_{p}\u2a7e{n}_{p}$, the stated time complexity in the theorem follows.
The space complexity follows as all data structures are linear in the number of transitions and states. As ${n}_{p}\u2a7d{m}_{p}$, this complexity can be stated as $\mathit{O}\phantom{\rule{0.55542pt}{0ex}}\left(\right)open="("\; close=")">{m}_{a}+{m}_{p}+{n}_{a}$. ☐
Note that it is reasonable that the number of probabilistic transitions ${m}_{p}$ is at least equal to the number of action states ${n}_{a}1$ as otherwise there are unreachable action states. This allows formulating our complexity more compactly.
Corollary 1. Algorithm 2 has time complexity $\mathit{O}\phantom{\rule{0.55542pt}{0ex}}\left(\right)open="("\; close=")">({m}_{a}+{m}_{p})log{n}_{p}+{m}_{p}log{n}_{a})$ and space complexity $\mathit{O}\phantom{\rule{0.55542pt}{0ex}}\left(\right)open="("\; close=")">{m}_{a}+{m}_{p}$ if all action states are reachable.
The only other algorithm to determine probabilistic bisimilarity for PLTS is by Baier, Engelen and MajsterCederbaum [
4]. The algorithm uses extended ordered binary trees and is claimed to have a complexity of
$\mathit{O}\phantom{\rule{0.55542pt}{0ex}}\left(\right)open="("\; close=")">mn(logm+logn)$ where
m is the number of transitions (including distributions) and
n the number of action states. For a fair comparison, we reconstructed their complexity in terms of
${n}_{a}$,
${n}_{p}$,
${m}_{a}$ and
${m}_{p}$. Their space complexity is
$\mathit{O}\phantom{\rule{0.55542pt}{0ex}}\left(\right)open="("\; close=")">{n}_{a}{n}_{p}\left\mathit{Act}\right$ and the time complexity is
$\mathit{O}\phantom{\rule{0.55542pt}{0ex}}\left(\right)open="("\; close=")">{m}_{a}{n}_{a}log{n}_{a}+{n}_{a}{n}_{p}log{n}_{p}+{n}_{a}^{2}{n}_{p}$. The last part
${n}_{a}^{2}{n}_{p}$ is not mentioned in the analysis in [
4]. It is due to taking the time into account for “inserting
$\mathit{Pre}(\alpha ,{\mu}_{i})$ into
$v.\mathit{states}$” (see page 208 of [
4]) for the version of ordered balanced trees used, and we believe it to be forgotten [
26].
This complexity is not easily comparable to ours. We make two reasonable assumptions to facilitate comparison. The first assumption is that the number of action transitions is equal to the number of distributions: ${m}_{a}={n}_{p}$. As second assumption, we use that $log{n}_{p}$ and $log{n}_{a}$ only differ by a constant.
In the rare case that the support of distributions is large, i.e., if all or nearly all action states have a positive probability in each distribution, then
${m}_{p}$ is equal or close to
${n}_{a}{n}_{p}$. In this case our space complexity becomes
$\mathit{O}\phantom{\rule{0.55542pt}{0ex}}\left(\right)open="("\; close=")">{n}_{a}{n}_{p}$ and our time complexity is
$\mathit{O}\phantom{\rule{0.55542pt}{0ex}}\left(\right)open="("\; close=")">{n}_{a}{n}_{p}log{n}_{p}$, which is comparable
mutatis mutandis to the complexity in [
4]. However, in the more common case where the support of distributions is limited by some constant
c, i.e.,
${m}_{p}\u2a7dc{n}_{p}$, we can simplify the space and time complexities to those in the following
Table 2.
In the table the underlined part stems from the extra time needed for insertions. It is clear tha,t if the assumptions mentioned are satisfied, the complexity of the present algorithm stands out well. This is confirmed in the next section where we report on the performance on a number of benchmarks of implementations of both algorithms.
5. Benchmarks
Both our algorithm, below referred to as GRV, and the reference algorithm by Baier, Engelen and MajsterCederbaum [
4], for which we use the abbreviation BEM, have been implemented in C++ as part of the mCRL2 toolset [
9,
10] (
www.mcrl2.org). This toolset is available under a Boost license which means that the source code is open and available without restriction to be inspected or used. In the implementation of BEM, some of the operations are not carried out exactly as prescribed in [
4] for reasons of practicality.
We have extensively tested the correctness of the implementation of the new algorithm by applying it to millions of randomly generated PLTSs, and comparing the results to those of the implementation of the BEM algorithm. This is not done because we doubt the correctness of the algorithm, but because we want to be sure that all the details of our implementation are right.
We experimentally compared the performance of both implementations. All experiments have been performed on a relatively dated machine running Fedora 12 with INTEL XEON E5520 2.27 GHz CPUs and 1TB RAM. For the probabilities exact rational number arithmetic is used which is much more time consuming than floating point arithmetic. The reported runtimes do not include the time to read the input PLTS and write the output.
Our first experimental question regards the growth of the practical complexity of the BEM and GRV algorithm when concrete probabilistic transition systems grow in size. To get an impression of this, we considered the socalled “ant on a grid” puzzle published in the New York Times [
27,
28]. In this puzzle, an ant sits on a square grid. When it reaches the leftmost or rightmost position on the grid it dies. When it reaches the upper or lower position of the grid it is free and lives happily ever after. On any remaining position, the ant chooses with equal probability to go to a neighbouring position on the grid. The question is what the probabilities for the ant are to die and stay alive, given an initial position on the grid.
The specification in probabilistic mCRL2 of the antonagrid is given in
Figure 4, where the dimensions of the grid are
${\mathit{max}}_{x}$ and
${\mathit{max}}_{y}$, and the initial position is given by
${i}_{x}$ and
${i}_{y}$.
The actions $\mathit{dead}$, $\mathit{live}$ and $\mathit{step}$ indicate that the ant is dead, stays alive and makes a step. The process expression $p\xb7q$ stands for sequential composition and $p+q$ represents the choice in behaviour. The notations $c\to p$ and $c\to p\diamond q$ are the ifthen and ifthenelse of mCRL2. The curly equal sign (≈) in conditions stands for equality applied to data expressions. The expression $\mathbf{dist}\phantom{\rule{3.33333pt}{0ex}}d:\mathit{Direction}[1/4]$ means that each direction d is chosen with probability $\frac{1}{4}$. From this description, PLTSs are generated that are used as input for the probabilistic bisimulation reduction tools.
Figure 5 depicts the runtime results of a set of experiments when increasing the total number of states of the ant on the grid model. At the left are the results when running the BEM algorithm, whereas the results for the GRV algorithm are shown at the right. Note that the
xaxis only depicts the number of action states. This figure indicates that the practical running times of both algorithms are pretty much in line with the theoretical complexity. This is in agreement with our findings on other examples as well. Furthermore, it should be noted that the difference in performance is dramatic. The largest example that our implementation of the BEM algorithm can handle within a timeout of 5 h requires approximately 10,000 s compared to 2 s for GRV. The particular example regards a PLTS of
$6.4\times {10}^{5}$ action states. The graphs clearly indicate that the difference grows when the probabilistic transition systems get larger.
To further understand the practical usability of the GRV algorithm, we applied it to a number of benchmarks taken from the PRISM Benchmark Suite (
www.prismmodelchecker.org/benchmarks/) and the mCRL2 toolset (
www.mcrl2.org/). The tests taken from PRISM were first translated into mCRL2 code to generate the corresponding PLTSs.
Table 3 collects the results for the experiments conducted. The
ant_N_M_grid examples refer to the antonagrid puzzle for an
N by
M grid with the ant initially placed at the approximate center of the grid. The models
airplane_N are instances of an airplane ticket problem using
N seats. In the airplane ticket problem,
N passengers enter a plane. The first passenger lost his boarding pass and therefore takes a random seat. Each subsequent passenger will take his own seat unless it is already taken, in which case he randomly selects an empty seat as well. The intriguing question is to determine the probability that the last passenger will have his or her own seat (see [
28] for a more detailed account).
The following three benchmarks stem from PRISM: The brp_N_MAX models are instances of the bounded retransmission protocol when transmitting N packages and bounding the number of retransmissions to $\mathit{MAX}$. The self_stab_N and shared_coin_N_K are extensions of the self stabilisation protocol and the shared coin protocol, respectively. For the self stabilisation protocol, N processes are involved in the protocol, each holding a token initially. The shared coin protocol is modelled using N processes and setting the threshold to decide head or tail to K.
Finally, the random_N tests are randomly generated PLTSs with N action states. All the models are available in the mCRL2 toolset.
At the left of
Table 3, the characteristics for each PLTS are given: the number of action states (
${n}_{a}$), the number of action transitions (
${m}_{a}$), the number of distributions (
${n}_{p}$), and the cumulative support of the distributions (
${m}_{p}$). The symbol “K” is an indicator for 1000 states. The same characteristics for the minimised PLTS are also provided. Furthermore, the runtime for minimising the probabilistic transition system in seconds as well as the required memory in megabytes are indicated for both algorithms. As mentioned earlier, we limited the runtime to 5 h.
The experiments show that the GRV algorithm outperforms the reference algorithm quite substantially in all studied cases. In the case of “
random_100” the difference is four orders of magnitude, despite the fact that this state space has only 100 K action states. The second last column of
Table 3 lists the relative speedup, i.e. the quotient of the time needed by BEM over the time needed by GRV, when applicable. Memory usage is comparable for both algorithms for small cases, whereas for larger examples the BEM algorithm requires up to one order of magnitude more memory than the GRV algorithm. The rightmost column of
Table 3 contains the relative efficiency in memory, i.e. the quotient of the memory used by BEM over the memory used by GRV, for the cases where BEM terminated before the deadline.