Probabilistic Plan Recognition for Multi-Agent Systems under Temporal Logic Tasks

Abstract

This paper studies the plan recognition problem of multi-agent systems with temporal logic tasks. The high-level temporal tasks are represented as linear temporal logic (LTL). We present a probabilistic plan recognition algorithm to predict the future goals and identify the temporal logic tasks of the agent based on the observations of their states and actions. We subsequently build a plan library composed of Nondeterministic B$\ddot{\mathrm{u}}$chi Automation to model the temporal logic tasks. We also propose a Boolean matrix generation algorithm to map the plan library to multi-agent trajectories and a task recognition algorithm to parse the Boolean matrix. Then, the probability calculation formula is proposed to calculate the posterior goal probability distribution, and the cold start situation of the plan recognition is solved using the Bayes formula. Finally, we validate the proposed algorithm via extensive comparative simulations.

1. Introduction

Mulit-agent plan recognition (MAPR) is the problem of predicting the goal or recognizing the task of multi-agent systems based on observations of their states and actions [1,2]. This paper considers the multi-agent plan recognition problem concerning attack and defense tasks in real-time strategy (RTS) games [3]. In RTS games, the opponents’ tasks possess certain temporal logic properties, e.g., an enemy can only attack the base after the bunker is destroyed. In order to improve the dynamic adaptability of the RTS game’s intelligence, it is necessary to predict the future goal and identify the opponent’s current task under temporal logic constraints.

A wide range of works have tackled the MAPR problem based on a plan library, including the Bayesian model [4,5,6,7], hidden Markov algorithms [8,9], specialized procedures [10], and parsing algorithms [11,12]. They all required a suitable plan library or a set of rules to establish their model. Banerjee et al. [2] proposed the formulation of the MAPR problem, proving it to be NP-complete. The plan library was modeled by a matrix, and Knuth’s Algorithm X was used to build their model. Based on the Abstract Hidden Markov Model, Saria et al. [13] provided an MAPR model consisting of a hierarchical dynamic Bayes network, which required a lot of manual drawing in advance. Synnaeve et al. [4,5] proposed a simple Bayesian model for predicting the goal of the opponent and used machine learning algorithms to calculate the parameters of the model by replaying the data multiple times. The proposed Bayesian model’s prediction accuracy is closely related to the size of the dataset, which is hard to obtain for confrontation scenarios such as wars.

Other researchers have tackled the MAPR problem based on domain theories. Ramirez et al. [14,15] used LPG-d and top-k planners to transform the plan recognition problem into a planning problem and calculated the posterior probability distribution of goals with an action cost. Sohrabi et al. [16] proposed the relaxation of the plan-recognition-as-planning formulation and compared the planned trajectory with a wide range of high-quality plans. Zhuo et al. [17] constructed the domain using an action model and solved soft (likelihood of various activities) and hard (observed or causal) constraints based on the MAX-SAT. Nevertheless, these algorithms cannot solve the plan recognition problem under temporal logic tasks due to the lack of a plan library.

To perform collaborative tasks in a multi-agent system, the dynamic team structures of the system need to be identified with an MAPR solution. In the simplest case, when all the agents in the multi-agent system perform the same task, the MAPR can be treated as a single agent plan recognition problem by matching the observations to the plan library [18]. However, this condition is rarely met in realistic applications since all the agents of a multi-agent system may dynamically join or leave their current team during the entire operation period. Kaminka et al. [19] identified the team structures and plans by considering the static social structures (such as YOYO). Zilberbrand et al. [20] built the plan library for a single agent instead of multiple agents and identified the dynamic team structures, assuming that agents in a team performed the same task.

Hence, we propose a probabilistic plan recognition algorithm while considering the complexity and temporal logic constraints of actual tasks. Different from [19,20], in order to deal with the fact that multiple ways of completing the same task exist when multiple agents cooperate, we solve the plan recognition problem using the probabilistic notion. Different from the algorithms in [14,15], we establish a plan library based on the theory of LTL in order to model the temporal logic tasks. Compared with [2], our model is more adaptable to complex multi-agent tasks [21].

The main contributions of this paper are as follows:

• For task recognition, we propose an algorithm to map the planning library and the planned multi-agent trajectories into a Boolean matrix. The complexity of the matrix is reduced by introducing the automata graph model. We optimize the calculation of the task completion degree and identify the multi-agent organization structure dynamics by analyzing the Boolean matrix.
• For goal prediction, we propose a probability calculation algorithm based on the Bayesian formula, considering information such as task constraints, agent trajectories, and potential collaborative relationships. Combined with the Bayesian inference theory, the probability calculation algorithm overcomes the problem of multi-agent plan recognition under cold start conditions.

The remainder of the paper is structured as follows. We introduce the linear temporal logic and workspace model and present the definition of the multi-agent plan recognition problem in Section 2. A multi-agent probabilistic plan recognition algorithm is proposed to solve the plan recognition problem in Section 3. We provide the comparative simulation to illustrate the effectiveness of the algorithm and analyze the advantages of the proposed algorithm in Section 4. Finally, we conclude the paper in Section 5.

2. Preliminaries and Problem Formulation

2.1. Linear Temporal Logic

Linear temporal logic (LTL) [22,23,24] provides a concise and formal way of defining propositions and temporal constraints for the robot task, which consists of atomic propositions (APs), boolean operators, and temporal operators. LTL formulas can be formed based on the following syntax definition:

$$\phi ::=True\left|a\right|{\phi }_1\vee {\phi }_2|{\phi }_1\wedge {\phi }_2|\neg \phi |\square \phi |\diamond \phi |◯\phi |{\phi }_1\mathit{U}{\phi }_2$$

(1)

where $a\in AP$, $\vee \left(or\right)$, $\wedge \left(and\right)$, and $\neg \left(not\right)$ are boolean operators, while $◯ \left(next\right)$, U (until), $\square \left(always\right)$, and $\diamond \left(eventually\right)$ are temporal operators. The Boolean operators $\neg ,\vee ,\mathrm{and}\wedge$ are used to denote negation, disjunction, and conjunction, respectively. U is the Until operator, and ${\phi }_1\mathit{U}{\phi }_2$ indicates that ${\phi }_1$ is true until ${\phi }_2$ becomes true. $F\phi :=\top U\phi$ and $G\phi :=\neg F\neg \phi$ represent Eventually and Always, respectively. These operators are called future operators. An LTL formula can be transformed into a corresponding Nondeterministic B$\ddot{\mathrm{u}}$chi Automation (NBA) [25]:

Definition 1.

(NBA): The NBA over $2^{AP}$ is a tuple $A_{\phi }=(Q,2^{AP},\delta ,Q_0,F)$, where Q is the finite set of states, $2^{AP}$ indicates the alphabet, $\delta :Q\times 2^{AP}\to 2^Q$ indicates the transition relation, $Q_0\subseteq Q$ is the set of initial states, and $F\subseteq Q$ is the set of accepting states.

We define the accepting run of the NBA as an infinite sequence that follows the transition relation from the initial state, i.e., $r=q_0{q}_1{q}_2\dots$, where $q_0\in Q_0$, $q_{k+1}\in \delta (q_k,S)$, and $Inf\left(r\right)\cap F\ne \varnothing$, while $Inf\left(r\right)$ denotes the set of states that infinitely appear in r.

2.2. Workspace Model

The workspace can be represented by a weighted Finite Transition System (FTS), defined as follows.

Definition 2.

(FTS) [25]: An FTS is a tuple $T_c=\left(\Pi ,AP,{\to }_c,W_c\right)$, where $\Pi =\left\{{\pi }_1,{\pi }_2,\dots ,{\pi }_N\right\}$ is the set of workspace regions that have a special sign. ${\to }_c\subseteq \Pi \times \Pi$ is the transition relation, $AP$ is the set of APs, $L_c:\Pi \to 2^{AP}$ is the labeling function, i.e., the AP that the region ${\pi }_i$ has, and $W_c:\Pi \times \Pi$ is the weighted function, i.e., the transition cost of ${\to }_c$.

In order to model the robot motion and task, we compose Definitions 1 and 2 to the weighted product B$\ddot{\mathrm{u}}$chi automation (PBA), defined as follows.

Definition 3.

(PBA) [25]: A PBA is a tuple $A_p=T_c⨂A_{\phi }=\{Q^{\prime },Q_0^{\prime },{\delta }^{\prime },F^{\prime },W_p\}$, $Q^{\prime }=\Pi \times Q=\{〈\pi ,q〉\in Q^{\prime }|\forall \pi \in \Pi ,\forall q\in Q\}$, while ${\delta }^{\prime }:Q^{\prime }\to 2^{Q^{\prime }}$. $Q_0^{\prime }=\left\{〈\pi ,q〉\right|\forall \pi \in {\Pi }_0,\forall q\in Q_0\}$ is the set of initial states. $〈{\pi }_j,q_n〉\in {\delta }^{\prime }\left(〈{\pi }_i,q_m〉\right)$ if $\left({\pi }_i,{\pi }_j\right)\in {\to }_c$ and $q_n\in \delta \left(q_m,L_c\left({\pi }_i\right)\right)$. $F^{\prime }=\left\{〈\pi ,q〉\right|\forall \pi \in \Pi ,\forall q\in F\}$ is the set of accepting states. $W_p:Q^{\prime }\times Q^{\prime }$ is the weighted function, $W_p(〈{\pi }_i,q_m〉,〈{\pi }_j,q_n〉)=W_c({\pi }_i,{\pi }_j)$ and $〈{\pi }_j,q_n〉\in {\delta }^{\prime }\left(〈{\pi }_i,q_m〉\right)$.

2.3. Problem Formulation

Considering the defense problem in an RTS game, it is necessary to give a goal prediction and identify the enemy’s intention by analyzing the states of the enemy units. However, due to the non-cooperation of the enemy units, we cannot know the enemy’s attack goals and current task. The enemy plan has a temporal logic property, e.g., $\diamond base\wedge (\neg base\mathit{U}bunker)$, indicating that the base cannot be attacked until the bunker has been destroyed. The MAPR problem under temporal logic tasks can be defined as follows.

Definition 4.

The MAPR problem under temporal logic tasks: The multi-agent system $A=\{a_1,a_2,\dots ,a_n\}$ consists of n agents, while the plan library $P=\{{\phi }_1,{\phi }_2,\dots \}$ contains $\left|P\right|$ tasks that may be performed by the multi-agent system and specified in the form of Formula (1). According to the observed multi-agent trajectory, $M=\left[m_{ij}\right]$, where $m_{ij}$ indicates the action executed by $a_j$ in time i. The MAPR needs to predict the future goals and identify the current execution tasks of each agent.

In this paper, three key sub-problems need to be considered to solve the above problem.

• Problem 1: Description of multi-agent collaborative tasks. We consider the situation where multiple agents cooperate to complete the same task, so there are multiple ways of completing the same task. For the modeling and description of each task, it is necessary to consider the cooperation among multiple agents. Furthermore, the increase in the number of agents creates difficulties for the task description.
• Problem 2: Matching of multi-agent action sequences and the plan library. We consider the multi-agent system with deceptive behaviors, where the agent will interrupt the current task to perform others and then return to the execution of the unfinished task. Therefore, it is difficult to judge whether the agent abandons the current task and reasonably sets the current task abandonment threshold.
• Problem 3: Calculation of goal prediction probability for multiple agents. We extend Ramirez and Geffner’s [14] Plan Recognition as Planning (PRAP) approach to temporal logic tasks. Therefore, it is necessary to consider the temporal logic task cost calculation caused by multi-agent cooperation. Moreover, in the case of cold start, where the trajectory matrix M is empty, how to identify the task currently being performed by the agent and give the target probability is a difficult point.

3. Probabilistic Plan Recognition Algorithm

Based on Definition 4, we divide the planning recognition into two parts: task identification and goal prediction. The task recognition algorithm aims to identify two categories of tasks: task completed and task in progress. Based on the task recognition results, we calculate the goal probability while considering the temporal logic constraints, the trajectory, and the potential team setup. In particular, we tackle the problem of plan recognition in the cold start situation. Before that, we need to establish an appropriate model for the description of collaborative tasks.

3.1. Automata-Based Model

Generally, we can describe the tasks in the planning library as a matrix by artificial coding. However, this is only adapted to the scenario of simple tasks where one task consists of only a few actions. For complex temporal tasks, artificial coding is time-consuming and error-prone. Therefore, the automata-based model is introduced to describe the tasks [26,27].

The conversion relationship in automaton is composed of specific actions, which can be a single action or a combination of actions. Therefore, the automata-based model can represent the cooperative execution of tasks by multi-agents. The conversion relationship of a single action requires a single agent to be completed, and the conversion relationship of a combined action requires a corresponding number of multi-agents to be completed collaboratively. Automata is modeled for tasks, and the conversion relationship has a non-directivity relative to multi-agent systems, that is, the actions corresponding to each edge can be executed by different agents. Therefore, for complex temporal logic tasks with multiple priorities, the temporal logic language is used for task description, and the planning library P is composed of corresponding automaton models.

Since it is acceptable for machines and readable for humans, we use LTL to describe temporal logic tasks [28]. According to the preliminaries, LTL formulas can be transformed into a corresponding Nondeterministic B$\ddot{\mathrm{u}}$chi Automation (NBA). Therefore, we use the NBA to model the task ${\phi }_i$ in plan library P, called $A_{{\phi }_i}$. As a formal language, LTL is sound and complete. Similarly, NBA inherits these properties. The action sequence corresponding to any path from the initial state to the acceptable state is correct for the task, and the model contains all possible action sequences to complete the task.

3.2. Task Recognition for Temporal Logic Tasks

After modeling the collaborative tasks in the plan library, we map the plan library P to the trajectory M and obtain a boolean matrix E, which contains task execution information and multi-agent behavior information. The boolean matrix E contains $Tn$ columns, where n is the number of agents and T is the time period. For task ${\phi }_i$, the corresponding row $e_r$ in the boolean matrix can be defined as follows:

$$e_{r,c}=\left\{\begin{array}{cc}\hfill 1& c=tn+a_j\hfill \\ \hfill 0& others\hfill \end{array}\right.$$

(2)

where $e_{r,c}$ means the agent $a_j$ is executing task ${\phi }_i$ at time t.

Since the NBA is a graph model, the trajectory matrix M can be put into the graph model and compared with the transition relation $\delta$ to determine whether it is consistent with the task. The Boolean matrix generation algorithm is shown in Algorithm 1. First, extract the sub-matrix according to the starting time (steps 2–4). Then, combine the agent actions and extract the parts that are completely equal to the transition relation of $A_{{\phi }_i}$ (steps 6–11). Finally, form the Boolean matrix according to Formula (2) (step 15–21). It should be noted that $\delta (state,next\_state)$ represents the transition relation required from the $state$ to $next\_state$ in the NBA $A_{{\phi }_i}$. The time complexity of the algorithm is $O\left(\right|P\left|\right|Q\left|n{T}^2\right)$, where $\left|Q\right|$ represents the number of states of the NBA.

Algorithm 1 Boolean matrix generation algorithm
Input: The trajectory M, Plan library P, Number of agents n, Time period T
Output: Boolean matrix E
1:	$S=⌀$
2:	for each NBA $A_{{\phi }_i}\in P$ do
3:	for $t=1,2,\dots ,T$ do
4:	$\mu =\left[\begin{array}{cccc}{m}_{t,1}& m_{t,2}& \dots & m_{t,c}\\ m_{t+1,1}& m_{t+1,2}& \dots & m_{t+1,c}\\ \dots & \dots & \dots & \dots \\ m_{T,1}& m_{T,2}& \dots & m_{T,c}\end{array}\right]$
5:	$N=⌀$
6:	$state$ = the initial state $Q_0$ of the NBA $A_{{\phi }_i}$
7:	for ${\mu }_{row}\in \mu$ do
8:	for $next\_state\in \delta \left(state\right)$ do
9:	if $\delta (state,next\_state)\subseteq {\mu }_{row}$ then
10:	$N\leftarrow {\mu }_{}={\mu }_{row}\cap \delta (state,next\_state)$
11:	$state=next\_state$     break
12:	if $state\ne Q_0$ then
13:	Calculate the completion degree and the last             state in the NBA of N, and add S.
14:	Sort the elements in the set S in descending order according to the completion degree.
15:	for$N\in S$do
16:	$G=0_{1\times Tn}$
17:	Parse the agent number i, start time $t_{start}$, and last     time $t_{last}$ from N.
18:	for $j^{\prime }=t_{start},\dots t_{last}$ do
19:	$G[j^{\prime }\times n+i]=1$
20:	$E=\left[\begin{array}{c}E\\ G\end{array}\right]$
21:	$E=\left[\begin{array}{c}E\\ I\end{array}\right]$, where I is the identity matrix.
22:	return E

The state in the NBA indicates the progress of the task. The task completion degree $comp\left(q\right)$ of state q can be measured as follows:

$$comp\left(q\right)=1-{\displaystyle \frac{action(q,F)}{action(Q_0,F)}},$$

(3)

where $action(Q_0,F)$ represents the number of actions to be performed from the initial state $Q_0$ to the acceptable state F. The completion degree $comp\left(q\right)$ increases as the remaining actions $action(q,F)$ decrease .

Due to different task progress states, the focus of the task recognition algorithm is different. For completed tasks, the completion degree is 1, and there is no subsequent execution cost. The focus is to find the unique explanation as much as possible. For the task in progress with possible subsequent execution, it is necessary to consider the task completion degree and the subsequent execution cost.

(1) Task completed

We use Knuth’s Algorithm X to solve the completed task recognition problem. Algorithm X finds a set of rows of E, such that for every column of E, there is exactly one row that contains 1. The details are shown in Algorithm 2. The result $res$ represents the position of the selected rows in the initial Boolean matrix E, indicating that the corresponding task is completed. For instance, if $res=i$, it means that task ${\phi }_i$ is completed.

Algorithm 2 Task recognition based on Algorithm X

Input:  Boolean matrix E Output:  The task recognition result $res$   1: Preprocess the Boolean matrix E. Delete the row of E with a completion degree less than 1.   2: $i=1$   3: Select the i-th row of E and mark it in red.   4: Mark in blue the column with a value of 1 in the i-th row.   5: Mark in purple the row with a value of 1 in the column marked blue.   6: Delete the red, blue, and purple in E, and get a new Boolean matrix $E^{\prime }$.   7: if$E^{\prime }=⌀$then   8:     if  $E(i,j)=1,\forall i\in row\left(E\right),j\in col\left(E\right)$ then         go to step 11.   9:     else $i=i+1$, back to step 3. 10: else$E=E^{\prime }$, back to step 2. 11: Record the position of the finally selected red rows in the initial Boolean matrix E, called $res$. 12: return$res$

(2) Task in progress

Once the task completion degree has been obtained by the Boolean matrix generation algorithm, the cost of the subsequent task execution can be calculated with the use of the following procedure.

• Step 1: According to Definition 3, for every task ${\phi }_i$ in the plan library, construct a PBA, called $A_{{\phi }_i}^{\prime }$.
• Step 2: In every $A_{{\phi }_i}^{\prime }$, we use Dijkstra’s algorithm to calculate the minimum cost of each state $q^{\prime }\in Q^{\prime }$ to accept state $f^{\prime }\in F^{\prime }$, called $costPBA(q^{\prime },f^{\prime })$. Step 1 and 2 can be calculated off-line without the requirement for trajectory.
• Step 3: According to the Boolean matrix E, the set of tasks that the agent i may perform at time t can be defined as ${\phi }_{possible}(i,t)=\left\{〈{\phi }_r,q_r〉\right|ifcomp<1$ and $E(r,nt+i)=1,$$\forall r\in row\left(E\right)\}$, where $q_r$ indicates the current state in the NBA $A_{{\phi }_r}$.

In the case of multi-agent cooperative task execution, the number of agents performing task ${\phi }_i$ at time t is called $num({\phi }_r,t)$,

$$num({\phi }_r,t)=\left|\left\{i\right|\forall i\in n,if\phi (i,t-1)={\phi }_r,\}\right|.$$

(4)

Details of the subsequent task cost calculation is shown in Algorithm 3.

Algorithm 3 The subsequent task cost calculation

Input:  Possible task set $〈{\phi }_r,q_r〉$, Plan library P, $costPBA$ Output:  Subsequent task cost $cost(i,{\phi }_r)$ 1: for$q_{next}\in \delta \left(q_r\right)$do 2:     if $|\delta (q_r,q_{next})|<=num({\phi }_r,t)$ then 3:         $costAlpha\left(q_{next}\right)=min\left\{costPBA(q^{\prime },f^{\prime })\right|q^{\prime }=$         $〈q_{next},\pi 〉,\forall f^{\prime }\in F^{\prime },\pi \in \Pi \}$ 4:         The action to be executed is $\delta (q_r,q_{next})$.         Use the auction algorithm to assign $\delta (q_r,q_{next})$.         The cost of the result is $costBeta(i,q_{next})$. 5: $cost(i,{\phi }_r)=min\{costBeta(i,q_{next})+$                   $costAlpha\left(q_{next}\right)|\forall q_{next}\in \delta \left(q_r\right)\}$ 6: return$cost(i,{\phi }_r)$

The most probable task is determined by considering the task completion degree and the subsequent execution cost of the task in progress. The probability of the task increases with a higher task completion degree and a lower subsequent execution cost. The task $\phi (i,t)$ to be executed by agent i at time t is formulated as follows:

$$\phi (i,t)=argmax\{{\displaystyle \frac{comp\left({\phi }_r\right)}{\sqrt{cost(i,{\phi }_r)}}}|\forall {\phi }_r\in {\phi }_{possible}(i,t)\}.$$

(5)

3.3. Goal Prediction with Multi-Agent Cooperation

At the current time t, according to the task recognition algorithm, the task performed by the agent i is ${\phi }_i$, the current state of the corresponding NBA is $q_{{\phi }_i}$, and the task completion degree is $comp\left({\phi }_i\right)$. If $comp\left({\phi }_i\right)<1$, the next state of the agent i is divided into two cases: the agent i continues to execute the task ${\phi }_i$, recorded as the event $Con$, or interrupts the task ${\phi }_i$ to perform other tasks, recorded as event $New$.

On the trajectory defined as M, the goal prediction conditional probability $P\left(goa{l}_j\right|M)$ is calculated as follows:

$$\begin{array}{cc}\hfill P\left(goa{l}_j\right|M)=& P\left(goa{l}_j\right|Con)P\left(Con\right|M)+\hfill \\ & P\left(goa{l}_j\right|New)P\left(New\right|M),\hfill \end{array}$$

(6)

where $\forall goa{l}_j\in \Pi$, $P\left(goa{l}_j\right|Con)=P\left(goa{l}_j\right|{\phi }_i)$, $P\left(New\right|M)=1-P(Con\left|M\right)$, and $P\left(goa{l}_j\right|{\phi }_i)$ represents the probability of selecting $goa{l}_j$ under the task ${\phi }_i$. In the case of a multi-agent collaboration scenario:

$$P\left(goa{l}_j\right|{\phi }_i)=\sum _{k}P\left(goa{l}_j\right|se{t}_k)P\left(se{t}_k\right|{\phi }_i)$$

(7)

where ${\sum }_{k}se{t}_k=\left\{\delta (q_{{\phi }_i},q_{next})\right|\forall q_{next}\in \delta \left(q_{{\phi }_i}\right)\}$, and $P\left(se{t}_k\right|{\phi }_i)$ represents the possibility of selecting the action set $se{t}_k$ in the case of task ${\phi }_i$:

$$P\left(se{t}_k\right|{\phi }_i)={\displaystyle \frac{1/cost\left(se{t}_k\right)}{{\sum }_{k}1/cost\left(se{t}_k\right)}},$$

(8)

where $cost\left(se{t}_k\right)$ is the cost of the multi-agent cooperative executing the action set $se{t}_k$ with agent i and can be defined according to steps 1–4 of Algorithm 3 as:

$$cost\left(se{t}_k\right)=costAlpha\left(q_{next}\right)+costBeta(i,q_{next}).$$

(9)

$P\left(goa{l}_j\right|se{t}_k)$ indicates the probability of selecting $goa{l}_j$ in the action set $se{t}_k$ of agent i.

$$\begin{array}{cc}\hfill P\left(goa{l}_j\right|se{t}_k)& ={\displaystyle \frac{1/cost(i,goa{l}_j)}{{\displaystyle \sum _{goa{l}_l\in se{t}_k}}cost(i,goa{l}_l)}},\hfill \\ \hfill cost(i,goa{l}_j)& =(1+\lambda Cos(i,goa{l}_j))dist(i,goa{l}_j),\hfill \end{array}$$

(10)

where $Cos(i,goa{l}_j)$ is the cosine of the angle between agent i and $goa{l}_j$, while $dist(i,goa{l}_j)$ is the distance between agent i and $goa{l}_j$, and $\lambda$ is a constant.

$P\left(Con\right|M)$ measures the possibility that agent i interrupts task ${\phi }_i$. The goal prediction algorithm is an online real-time prediction, while the task recognition results of the current moment t are the same as the previous moment $t^{\prime }$ most of the time. Consequently, we can infer $P\left(Con\right|M)$ by comparing $cost(i,{\phi }_i;t)$ and $cost(i,{\phi }_i;t^{\prime })$:

$$\begin{array}{ccc}\hfill & P\left(Con\right|M;t)=\hfill & \\ & P\left(Con\right|M;t^{\prime })(1-\alpha {\displaystyle \frac{cost(i,{\phi }_i;t)-cost(i,{\phi }_i;t^{\prime })}{cost(i,{\phi }_i;t^{\prime })}})\hfill \end{array}$$

(11)

where $\alpha$ is a tuning parameter for the effect of cost on the possibility to interrupt task ${\phi }_i$, and the initial value of $P\left(Con\right|M)$ needs to be set manually. It can be seen that if the cost of executing task ${\phi }_i$ increases at the current time t as compared to the previous time $t^{\prime }$, the possibility of continuing the execution of task ${\phi }_i$ decreases.

$P\left(goa{l}_j\right|New)$ represents the target prediction probability when agents interrupt task ${\phi }_i$ to perform other tasks:

$$P\left(goa{l}_j\right|New)=\sum _{{\phi }_k\in P\backslash {\phi }_i}P\left(goa{l}_j\right|{\phi }_k)P\left({\phi }_k\right|New),$$

(12)

where $P\left(goa{l}_j\right|{\phi }_k)$ is shown in Formula (7), $P\left({\phi }_k\right|New)$ represents the possibility of choosing another task and is defined as follows:

$$P\left({\phi }_k\right|New)={\displaystyle \frac{1/cost(i,{\phi }_k)}{{\displaystyle \sum _{{\phi }_k\in P\backslash {\phi }_i}}1/cost(i,{\phi }_k)}},$$

(13)

where the task execution cost $cost(i,{\phi }_k)$ is calculated according to Algorithm 3.

In summary, the framework of the multi-agent plan recognition algorithm under a temporal logic task is shown in Figure 1.

3.4. Cold Start

In the situation of cold start, where the trajectory $M=⌀$ and the task recognition result cannot be obtained through the Boolean matrix E, the goal probability calculation formula is as follows:

$$P\left(goa{l}_j\right|M)=P\left(goa{l}_j\right|New)P\left(New\right|M)$$

(14)

Based on the Bayesian formula [29,30], it can be inferred that the probability $P\left({\phi }_i\right|M)$ of the agent i is the following:

$$\begin{array}{ccc}\hfill P\left({\phi }_i\right|M)& =\sum _{j}P\left(\phi \right|goa{l}_j)P\left(goa{l}_j\right|M)\hfill & \\ \hfill & =\sum _j{\displaystyle \frac{P\left(goa{l}_j\right|{\phi }_i)P\left({\phi }_i\right)}{P\left(goa{l}_j\right)}}P\left(goa{l}_j\right|M)\hfill \end{array}$$

(15)

where $P\left({\phi }_i\right)=1/\left|P\right|$, $P\left(goa{l}_j\right)=1/|\Pi |$, and $P\left(goa{l}_j\right|{\phi }_i)$ is calculated by Formula (7).

4. Simulations and Analysis

In order to verify the effectiveness of the proposed algorithm in task recognition and target prediction, we designed a comparative simulation for multi-agent plan recognition. The environment in the simulation is a two-dimensional continuous system. Furthermore, the agent is considered as a particle with a fixed velocity. In order to accurately describe the moving trajectory of a multi-agent system and consider the increase in computation caused by the number of agents, we abstract the workspace as an FTS, which we defined earlier.

4.1. Scene Introduction

The scene is defined as a defensive game in which the planes need to cooperate with each other to destroy all buildings, while the tanks need to defend the buildings. As shown in Figure 2, red planes are used as the attackers and are controlled by a human operator with a keyboard. The tanks, representing the defensive side, are controlled by an artificial intelligence module composed of the proposed plan recognition algorithm. The yellow circles, blue rectangles, and red diamonds represent buildings of different security levels. For the defensive tanks, their objectives are to identify the attacking plane’s task, to predict their future goal in advance, to reach the corresponding building in time, and to resist the plane.

According to the rules of the game, the building can be attacked if the previous-level buildings are destroyed. Therefore, the planes have the following attack strategies, as described by LTL formulas:

$$\begin{array}{ccc}\hfill & {\phi }_1=\diamond u\wedge (\neg u\mathit{U}(j\wedge k)\wedge (\neg (j\vee k)\mathit{U}a)\hfill & \\ \hfill & {\phi }_2=\diamond v\wedge (\neg v\mathit{U}(a\wedge d\wedge p)\hfill & \\ \hfill & {\phi }_3=\diamond z\wedge (\neg z\mathit{U}(f\wedge p\wedge g)\hfill & \\ \hfill & {\phi }_4=\diamond y\wedge (\neg y\mathit{U}(m\wedge n)\wedge (\neg (m\vee n)\mathit{U}g)\hfill \end{array}$$

(16)

These formulas represent the standard order for the attacking planes to destroy the targeted buildings. Taking ${\phi }_1$ as an example, this means attacking a first, then attacking $J,K$, and finally, destroying u. Other formulas have similar meanings.

4.2. Simulation Verification and Analysis

The simulation is configured to include three attacking planes controlled by humans to verify the accuracy of the plan recognition algorithm. The trajectories of the multi-agent system are shown in Figure 3, which are represented by the red, blue, and green lines. The goal prediction probability curves of the algorithm designed in this paper are shown in Figure 4a, Figure 5a and Figure 6a, while the task recognition probability curves are shown in Figure 7. The parameters of the algorithm are set as $P(ConM;t=0)=0.7$ and $\alpha =0.5$. As a comparison, the curves of the algorithm proposed by Ramirez and Geffner are shown in Figure 4b, Figure 5b and Figure 6b.

The actual goals of the three agents are: $[a,j,k,u,d]$, $[f,g,p,v]$, $[n,m,y]$. The corresponding arrival times are: $[52,170,648,684,812]$, $[63,327,785,911]$, $[464,528,570]$. The actual tasks of the three agents are: $[{\phi }_1,{\phi }_1,{\phi }_1,{\phi }_1,{\phi }_2]$, $[{\phi }_3,{\phi }_3,{\phi }_2,{\phi }_2]$, $[{\phi }_4,{\phi }_4,{\phi }_4]$. It can be seen that agent 1 and agent 2 cooperatively execute task ${\phi }_2$, and agent 2 and agent 3 cooperatively execute task ${\phi }_4$. The simulation was run 4000 times, and the goal prediction accuracy of the proposed algorithm and the R&G algorithm are shown in

Table 1. Goal prediction accuracy (%).

Agent	${\mathit{a}}_1$	${\mathit{a}}_2$	${\mathit{a}}_3$
R&G Algorithm	52.79	38.31	82.94
Proposed Algorithm	87.95	91.58	100

. The task recognition accuracy of the proposed algorithm for the three agents are: 99.88%, 85.01%, and 100%.

In terms of goal prediction, it can be seen that the proposed algorithm has a higher accuracy. Because the constraint of temporal logic is considered in the proposed algorithm, the candidate set of actions can be filtered well; for example, time $t_1$ as shown in Figure 3. Although agent 1 is closer to u, according to Formula (8), the action candidate set is $\{k,p,d,g\}$. Moreover, we can get the prediction goal k based on Formula (6). The same situation follows for agent 2 with $t_2=210$ and agent 3 with $t_3=468$.

In terms of task recognition, first, the proposed algorithm can determine whether the agent has abandoned the current task. For example, agent 2 shows a trend that moves away from the current task ${\phi }_3$ in $t_4=790$ and performs a new task ${\phi }_2$. Second, the proposed algorithm can predict new tasks after the current task is completed. For example, for agent 1 at $t_4=790$, after completing the task ${\phi }_1$, according to Formula (15), the future task is ${\phi }_2$. Third, the proposed algorithm can identify the organization structure of a multi-agent system. For example, agent 1 and agent 2 are cooperating in task ${\phi }_2$ at $t_4=790$.

In order to verify the effectiveness and real-time performance of the algorithm, we designed 10 comparative experiments using different numbers of agents. The test result is shown in

Table 2. Comparative simulation results (%).

Amount	GA (%)		MAPR
	MAPR	R&G	RT (ms)	TA (%)
1	90.25	60.34	0.5	98.62
2	87.63	53.25	0.9	93.82
3	86.37	51.82	1.6	94.24
4	84.26	49.76	2.2	90.72

, where the MAPR indicates the proposed algorithm, RT is the run time, GA represents the goal prediction accuracy, and TA is the task recognition accuracy. The experiment results show that the proposed algorithm has a higher goal prediction accuracy than traditional algorithms for the MAPR problem under temporal logic tasks and completes task recognition with a higher quality. The algorithm run time is linearly related to the number of agents and meet the real-time requirement.

5. Conclusions

We have proposed a multi-agent probabilistic plan recognition algorithm under temporal logic tasks for goal prediction and task identification in a confrontational multi-agent system, in which the opponents could behave confusingly in order to prevent being identified. The task in the plan library has been modeled as Nondeterministic B$\ddot{\mathrm{u}}$chi Automation, and the trajectory of the multi-agent has been transformed into a Boolean matrix. Then, a task recognition algorithm has been proposed to parse the Boolean matrix. In particular, the cold start situation has been solved based on the Bayes formula. The algorithm has been verified in extensive comparative simulations, showing that the proposed algorithm can predict the next goal and identify the task of the multi-agent system effectively and accurately. This paper is based on simulation, but the proposed algorithm has more real-world application scenarios. The algorithm provides ideas not only for adversarial game scenarios, but also for other multi-agent scenarios involving complex temporal logic tasks such as goal recognition in smart home environments[31,32]. In the future, we will apply the algorithm to plan recognition in realistic real-time strategy games.