Evolutionary Dynamics of Division of Labor Games for Underwater Searching Tasks

Abstract

Division of labor in self-organized groups is a problem of both theoretical significance and application value. Many application problems in the real world require efficient task allocation. We propose a model combining bio-inspiration and evolutionary game theory. This research model theoretically analyzes the problem of target search in unknown areas for multi-robot systems. If the robot’s operating area is underwater, the problem becomes more complicated due to its information sharing restrictions. Additionally, it drives strategy updates and calculates the fixed probability of relevant strategies, using evolutionary game theory and the commonly used Fermi function. Our study estimates the fixed probability under arbitrary selection intensity and the fixed probability and time under weak selection for the two-player game model. In the multi-player game, we get these results for weak selection, which is conducive to the coexistence of the two strategies. Moreover, the conducted simulations confirm our analysis. These results help to understand and design effective mechanisms in which self-organizing collective dynamics appears in the form of maximizing the benefits of multi-agent systems in the case of the asymmetric game.

1. Introduction

It is a long-standing problem to study cooperative behavior in a limited population involving many disciplines. The reason is simple. In many scenarios, multiple robots are needed to perform tasks. Cooperative machines are widely used in real society and engineering applications. We need to use cooperative robots to form an efficient group in real society and engineering applications. In many cases, the optimal individual strategy in the group conflicts with the optimal decision of the group [1,2,3,4]. It poses a major challenge for an evolutionary model, as many models predict that this cooperation discontinues in the case of free rider exploitation. Evolutionary game theory is widely used in such problems [5,6,7,8].

A variety of specific game models describe the conflict of interest among individuals and groups [9,10]. A very widely used example is the prisoner’s dilemma [11]. Another example is the public goods game, a common model for describing the intragroup game [12,13]. The public goods game is a concise description of the cooperative conflict between group and individual interests [14,15]. The cross-study of evolutionary game theory and cooperation has also obtained many factors to promote cooperation [16,17].

Usually, real game scenarios have a limited number of players. For analyzing this case, researchers mostly employ the finite group dynamic evolution analysis methods based on the stochastic process [18]. Examples of the common analysis methods include the Moran [19], Wright Fisher [19], local update [20], and pairwise comparison processes. In these studies, it is essential to calculate and analyze fixed probability and time, respectively referring to the probability and time that the strategy succeeds and dominates the whole population. These studies employ selection intensity to measure the impact of game returns on fitness, which directly affects the difference of fitness. Weak selection is a common assumption, implying that the payoff difference based on different strategies plays a relatively minor role in the evolution process.

Labor division is very representative in the specific applications of the evolutionary game [21,22,23]. In real societies, it is common to see the scenarios of labor division in which groups of individuals repeatedly and non-randomly perform specific tasks [24,25]. In these scenarios, groups show the need for labor division, that is, there is unequal work distribution among or within specific tasks [26,27,28,29]. The cooperative search of multiple robots is a typical problem [30,31,32,33]. Searching targets within an unknown environment is a real problem with broad application backgrounds. In this problem, multiple robot individuals search for targets with unknown positions in an unknown area [34,35]. In this actual scenario, a problem is whether the individuals cooperate. Cooperation is an altruistic strategy, which means sharing information with other individuals. In contrast, selfishness means not sharing information with others.

Sharing information implies that individuals have to spend a certain cost to store and transmit information [36,37,38]. The cost infers that the computing and communication units consume energy, affecting the robot’s running time and speed. This impact is relatively negligible for land robots, but profound for UAVs, especially the underwater ones [39,40].

When designing task allocation algorithms, we find that designing low-cost swarm robots may be a better solution in some specific scenarios, but what kind of swarm can better perform complex tasks is a difficult problem. We wish to propose a framework for exploring whether individuals in a group cooperate when performing tasks. Of course, we know that there is no framework that can adapt to all scenarios. The motivation of this study is to develop a strategy evolution model of the labor division game based on an underwater multi-robot search problem, and investigate the evolutionary dynamics in a finite population. This study assumes two types of strategies, representing two categories of robots, A and B. Each player can choose one of them. The implementation of each task brings corresponding benefits and costs. Tasks A and B correspond to selfish and cooperative individual strategies, respectively. This research studies two- and multi-player games. Results can theoretically explain the impact of different selection intensity on the fixed probability of strategies, and derive the fixed probability and time under weak selection. Simulation is an approach to check the effectiveness of the theoretical analysis.

The main goal of our work is to establish the relationship between theoretical analysis and real applications. When we use multiple robots to perform a task, how to select or design a suitable robot swarm is a difficult problem. At the same time, task allocation for the robot swarm is also difficult. It is well known that self-organizing task allocation is the basic attribute of an intelligent group. Collaboration among individuals can improve the efficiency of the swarm to perform tasks. However, this will make the design of the swarm more complex, and may also increase the cost of the robots. Here, we propose a model from the perspective of information sharing by introducing game theory. Information sharing undoubtedly promotes cooperation among individuals and may facilitate the execution of tasks. At the same time, information sharing may also affect the efficiency with which individuals perform tasks. We hope to find an optimal solution between information sharing and personal selfishness in a specific scenario. We expect our results can be applied to the actual unknown area search task. This is one of the main differences from the previous works.

The organizational structure of this paper is as follows. Section 2 describes the main model and theoretical results of the two-player games. Section 3 represents the fixed probability and time under weak selection. Section 4 explains the results of a multi-player model involving multiple players in a game. Section 5 offers the results of computer simulations. Finally, the conclusion summarizes the research.

2. Two-Player Game with Dual-Strategy Task

We consider two types of robot players, strategy A and strategy B. Each player chooses a task as the strategy to gain its benefits and bear its costs. In this case, A robots only search by themselves without sharing information, while B robots share information with other individuals, regardless of each other’s strategies. Because of this difference, their corresponding benefits, respectively, $b_A$ and $b_B$, are also different. Similarly, their costs are $c_A$ and $c_B$, respectively. Because of the altruistic behavior of B robots, that is, they share information with other individuals, their opponents obtain additional synergy benefits. As an opponent of B, A gets $\alpha$ and B gets $\beta$. The following payoff matrix defines the above scenarios:

$$\begin{array}{c}\begin{array}{ccc}& & \begin{array}{ccccccc}& & A& & & & \begin{array}{c}B\end{array}\end{array}\end{array}\hfill \\ \begin{array}{c}A\\ B\end{array}\left(\begin{array}{cc}{b}_A-c_A& b_A-c_A+\alpha \\ b_B-c_B& b_B-c_B+\beta \end{array}\right).\hfill \end{array}$$

Let $b_A-c_A=x$, $b_B-c_B=y$, and $\sigma =x-y$ representing the net income difference between tasks A and B. A simplified payoff matrix is as follows

$$\begin{array}{c}\begin{array}{cc}& \begin{array}{ccccc}& A& & & \begin{array}{c}B\end{array}\end{array}\end{array}\hfill \\ \begin{array}{c}A\\ B\end{array}\left(\begin{array}{ccc}x& & x+\alpha \\ y& & y+\beta \end{array}\right).\hfill \end{array}$$

Presumably, N is the size of the investigated population, where jA-players and $N-j$B-players coexist. Equation (1) shows ${\pi }_A$, the expected payoffs of the players who perform task A,

$${\pi }_A=\frac{j-1}{N-1}x+\frac{N-j}{N-1}(x+\alpha )=x+\frac{N-j}{N-1}\alpha$$

(1)

Equation (2) represents the B-players’ expected payoffs,

$${\pi }_B=\frac{j}{N-1}y+\frac{N-j-1}{N-1}(y+\beta )=y+\frac{N-j-1}{N-1}\beta$$

(2)

Equation (3) describes the payoff difference between tasks A and B in the group,

$${\pi }_A-{\pi }_B=\sigma +\frac{(N-j)\alpha -(N-j-1)\beta }{N-1}.$$

(3)

Equation (4) is the often-used Fermi function for the strategy update,

$$p=\frac{1}{1+exp[-\omega ({\pi }_A-{\pi }_B)]},$$

(4)

where P is the probability that an A-player replaces a B-player in the evolution; ${\pi }_A$ and ${\pi }_B$ represent the focal A- and B-player’s payoffs, respectively; and $\omega$ indicates the selection strength. This research also focuses on the results under weak selection (Section 3 has more details). Firstly, we focus on the generated results ignoring the selection strength.

Following the analysis of the Moran process, Equation (5) presents the probability to increase the number of A-players from j to $j+1$: $T+$ and decrease from j to $j-1$: $T_j^{-}$,

$$T_j^{\pm }=\frac{j}{N}\frac{N-j}{N}\frac{1}{1+exp[\mp \omega ({\pi }_A-{\pi }_B)]}.$$

(5)

The probability that all players finally choose to perform strategy A for any initial configuration relies on the ratio ${\gamma }_j=T_j^{-}/T_j^{+}$, according to Equation (6),

$${\gamma }_j=\frac{T_j^{-}}{T_j^{+}}=exp[-\omega ({\pi }_A-{\pi }_B)].$$

(6)

Equation (7) represents the fixation probability

$${\varphi }_k=\frac{{\displaystyle \sum _{i=0}^{k-1}}{\displaystyle \prod _{j=1}^i}{\gamma }_j}{{\displaystyle \sum _{i=0}^{N-1}}{\displaystyle \prod _{j=1}^i}{\gamma }_j}.$$

(7)

Substituting Equations (3) and (6) into Equation (7) gives Equation (8),

$${\phi }_k=\frac{{\sum }_{i=0}^{k-1}exp[-i\sigma \omega +\frac{\omega i}{N-1}(\frac{(i+1)(\alpha -\beta )}{2}+(N-1)\beta -N\alpha )]}{{\sum }_{i=0}^{N-1}exp[-i\sigma \omega +\frac{\omega i}{N-1}(\frac{(i+1)(\alpha -\beta )}{2}+(N-1)\beta -N\alpha )]}.$$

(8)

Calculate result of the fixation probability by the integral approximation formula [27]:

$${\phi }_k=\frac{erf\left({\epsilon }_k\right)-erf\left({\epsilon }_0\right)}{erf\left({\epsilon }_N\right)-erf\left({\epsilon }_0\right)},$$

(9)

where $erf\left(x\right)=\frac{2}{\sqrt{\pi }}{\int }_0^x{e}^{-y^2}dy$, ${\epsilon }_k=\sqrt{\frac{\omega }{2(\beta -\alpha )(N-1)}}[(N-1)\sigma +(N-k)\alpha +(k-N+1)\beta ]$.

2.1. Scenario 1: Game with a Dominating Strategy

Suppose $\sigma >0$ and $\sigma +\alpha >\beta$, the payoff matrix in the framework of labor division situations is a prisoner’s dilemma game. In this case, an A-player always gets a higher income than a B-player, so strategy A has an advantage over strategy B. Figure 1 presents such a scenario and shows the evolution of strategy in dependence on the $\sigma$ (e.g., 2.01, 3, 4 and 5 respectively) and the values of ω (e.g., 0.01, 0.05, 0.1 and 1), where $\beta =1$ and $\alpha =1$.

If $\sigma <0$ and $\sigma +\alpha <\beta$, the payoff matrix in the framework of labor division situations is a prisoner’s dilemma game with a dominating strategy B. Figure 2 shows the evolution of strategy in dependence on the $\sigma$ (e.g., 2.01, 3, 4 and 5 respectively) and the values of ω (e.g., 0.01, 0.05, 0.1 and 1), where $\beta =8$ and $\alpha =6$.

2.2. Scenario 2: Coordination Game

If $\sigma >0$ and $\sigma +\alpha <\beta$, the payoff matrix in the framework of labor division situations is a coordination game. In this case, players should always use the same strategy. Figure 3 shows the evolution of the strategy in such a scenario.

2.3. Scenario 3: Game with Coexisting A and B

Suppose $\sigma <0$ and $\sigma +\alpha >\beta$, the payoff matrix in the framework of labor division situations is a coordination game. In this case, players should always use different strategies. A typical example is a hawk-dove or snowdrift game. Figure 4 illustrates the evolution of the strategy in this scenario.

3. Two-Player Game under Weak Selection

3.1. Fixation Probability

According to the Taylor series expansion, Equation (10) simplifies the transition probability in Equation (5) to obtain the updated transition probabilities,

$$T_j^{\pm }\approx \frac{j}{N}\frac{N-j}{N}[\frac{1}{2}\pm \frac{1}{4}\omega ({\pi }_A-{\pi }_B)].$$

(10)

The ratio of these transition probabilities plays a vital role in measuring the state that the system evolves into, given by Equation (11),

$${\gamma }_j=\frac{T_j^{-}}{T_j^{+}}\approx 1-\omega ({\pi }_A-{\pi }_B).$$

(11)

In the Fermi process, $T_j^{+}\left({}_{\omega =0}\right)=T_j^{-}\left({}_{\omega =0}\right)=\frac{1}{2}\frac{j}{N}\frac{N-j}{N}$ is the neutral transition probabilities, giving ${\gamma }_j=1$. Based on Equation (7), the function probability under neutral selection is as Equation (12),

$${\varphi }_k\left({}_{\omega =0}\right)=\frac{k}{N}.$$

(12)

3.2. Average Time

Under weak selection, the average time with only one A-player in the population initially is $t_1$. Equation (13) shows the general form of $t_1$

$$t_1\approx 2N{H}_{{}_{N-1}}+\omega v(N-1-H_{{}_{N-1}}),$$

(13)

where $H_{{}_{N-1}}={\sum }_{l=1}^{N-1}\frac{1}{l}$, ${\pi }_A-{\pi }_B=uj+v$. In this research, ${\pi }_A-{\pi }_B=\sigma +\frac{(N-j)\alpha -(N-j-1)\beta }{N-1}$. Therefore, Equation (14) represents the estimated average time,

$$t_1\approx 2N{H}_{N-1}+\omega (N-1-H_{N-1})(\sigma +\frac{N-1}{N}\alpha -\beta ).$$

(14)

3.3. Conditional Fixation Time

Under weak selection, the conditional fixation time with only one A-player in the population initially is $t_1^A$. Equation (14) shows the general form of $t_1^A$,

$$t_1^A\approx 2N(N-1)-\omega uN(N-1)\frac{N^2+N-6}{18}.$$

(15)

Equation (16) represents the conditional function time,

$$t_1^A\approx 2N(N-1)-\omega N(\beta -\alpha )\frac{N^2+N-6}{18}.$$

(16)

4. Multi-Player Game

Assuming that the group size is N, and n people participate in each game. If there are m participants among other $n-1$ individuals who complete task A, then $a_m$ and $b_m$ represent the corresponding income when the central individual is task A and B, respectively. For more details, please see

Table 1. The income matrix of n-person participation in the dual-strategy task allocation game model.

The Number of the Remaining A		A		B
n − 1		$a_{n-1}$		$b_{n-1}$
⋮	⋮		⋮
m		$a_m$		$b_m$
⋮	⋮		⋮
0		$a_0$		$b_0$

.

According to the income matrix, the expression of $a_m$ and $b_m$ are as below.

$$\begin{array}{cc}\hfill a_m& =b_{A}m+(b_A+\alpha )(n-1-m)-c_A\\ & =b_A(n-1)+\alpha (n-1-m)-c_A,\end{array}$$

(17)

$$\begin{array}{cc}\hfill b_m& =b_{B}m+(b_B+\beta )(n-1-m)-c_B\\ & =b_B(n-1)+\beta (n-1-m)-c_B.\end{array}$$

(18)

let $p=b_A(n-1)-c_A$, $q=b_B(n-1)-c_B$ and $\tau =p-q$, then $a_m$ can be simplified to:

$$a_m=p+\alpha (n-1-m).$$

(19)

$b_m$ can be simplified to:

$$b_m=q+\beta (n-1-m).$$

(20)

Assume that in a thoroughly mixed limited group of size N, the number of participants in strategy A is j, and the number of participants in strategy B is $N-j$. The probability of an individual with task A interacting with the remaining m individuals with task A follows a hypergeometric distribution. Therefore, Equations (21) and (22) show the average income of the individual of tasks A and B, respectively,

$${\pi }_A={\sum }_{m=0}^{n-1}\frac{C_{j-1}^m{C}_{N-j}^{n-1-m}}{C_{N-1}^{n-1}}{a}_m.$$

(21)

Equation (22) shows the average income of the individual of task B,

$${\pi }_B={\sum }_{m=0}^{n-1}\frac{C_j^m{C}_{N-j-1}^{n-1-m}}{C_{N-1}^{n-1}}{b}_m.$$

(22)

Equation (23) represents the function probability ${\varphi }_k$,

$$\begin{array}{cc}\hfill {\phi }_k& =\frac{{\sum }_{i=0}^{k-1}{\prod }_{j=1}^i{\gamma }_j}{{\sum }_{i=0}^{N-1}{\prod }_{j=1}^i{\gamma }_j}\\ & =\frac{{\sum }_{i=0}^{k-1}{\prod }_{j=1}^{i}exp[-\omega ({\sum }_{m=0}^{n-1}\frac{C_{j-1}^m{C}_{N-j}^{n-1-m}}{C_{N-1}^{n-1}}{a}_m-{\sum }_{m=0}^{n-1}\frac{C_j^m{C}_{N-j-1}^{n-1-m}}{C_{N-1}^{n-1}}{b}_m)]}{{\sum }_{i=0}^{N-1}{\prod }_{j=1}^{i}exp[-\omega ({\sum }_{m=0}^{n-1}\frac{C_{j-1}^m{C}_{N-j}^{n-1-m}}{C_{N-1}^{n-1}}{a}_m-{\sum }_{m=0}^{n-1}\frac{C_j^m{C}_{N-j-1}^{n-1-m}}{C_{N-1}^{n-1}}{b}_m)]}.\end{array}$$

(23)

Figure 5 and Figure 6 display the results in the case of $n=10$. Based on the results, the task allocation in the multi-player game model is similar to that in the two-player game, and also reflects various similar evolution dynamics. This also shows that the two-player model can be extended to a multi-player model.

5. Simulations

To verify rationality of the proposed model, we perform several multi-robot search experiments in unknown environments. In our experiments, an unknown area is a square region. In one experiment, we use 10 robots to search for targets in an unknown area. Each area has 10 targets at unknown locations. In each experiment, a swarm of 10 robots with selfish or altruistic strategies performs the search task together. It should be noted that in our experiments, all the targets are the same, and when a robot reaches a target’s position, it means that this robot has found the target. When all the targets are found, the experiment ends. In order to be closer to the real application scenarios, we created different searching maps and $30\%$ of each map are random obstacle areas. We assume that the area to be searched is connected in each experiment. In a search, individuals with strategy A are selfish. They only search by themselves and do not share search information with other individuals, that is, they do not tell other individuals which areas they have searched. Notably, sharing information reduces replicated searches. In addition, the speed of robot movement slows down because sharing information requires additional computing, storage, and communication units.

Figure 7 represents the results when the speed ratio is different. Regarding Figure 7, some of these curves show an upward trend, some show a downward trend, and some have peaks. Such results imply that the assumptions are correct in our previous theoretical analysis. That is, individuals have an optimal value in choosing a selfish strategy and altruistic strategy.

When r is relatively large, our results show that selfish individuals have great advantages in moving speed, and the curves shows a monotonous increasing trend. This shows that the speed plays a more decisive role than being able to obtain the benefits of shared information from other individuals. The best strategy in such a scenario is not to share information. When r is relatively small, we see a trend of increasing first and then decreasing. This shows that sharing information can bring benefits and reduce the number of search steps. Overall, our conclusion is that if not sharing information can bring more advantages in speed, not sharing information is an dominant strategy. On the contrary, sharing information is a good strategy if it does not reduce the moving speed too much.

The simulation results show that for the same task, different individual combinations will bring different results. In such a scenario, individuals need to cooperate, but cooperation requires costs. Such costs reduce the ability of the individual, and thus the efficiency of the group, to perform the task. The core issue is to strike a balance between individual competence and group coordination. Of course, such a balance will be highly uncertain. When generalizing such a model, we need to take this uncertainty into account.

6. Conclusions

Effective labor division is widely used in many fields, such as engineering and multi-robot system. Theoretically, the realization of effective labor division is closely related to individual behavior and scheduling group-level tasks. The critical point here is how the rationality of labor division contributes to the collective interests of a multi-agent system. In a distributed group, self-organizing task allocation results from self-organization without central command or global information exchange.

The search of underwater robot swarms for unknown environments is a typical scenario of task allocation. Both designing and combining different robots to perform such tasks may be reasonable solutions. Which is better? Our research tries to answer this question. To respond to this question, we use evolutionary game theory to model and study the evolution time of cooperation in labor division games by theoretically calculating fixed probability and fixed probability. We develop two- and multi-player models according to different scenarios requiring labor division. In this specific scenario, whether individuals share information is a decisive factor. Underwater information sharing requires additional sensors as information transmission and computing devices. Due to the difficulty of underwater information transmission, the cost is high. Therefore, here we take sharing information as a strategy of cooperation between individuals. Cooperation means paying a certain cost, but getting more task information. Selfish individuals only pay attention to their tasks. Moreover, we conduct simulations. The results confirm the reliability of our model, and provide new clues for the self-organization behavior in the task allocation problem faced by groups.

This study is geared towards a specific problem, but we believe it can be generalized. The results suggest that unconditional cooperation is not always the best strategy. Moreover, this study provides an idea that individuals can adopt different strategies to improve the efficiency of the group. When considering using clusters to perform tasks, we can refer to this study. We can have some individuals cooperate while others adopt selfish strategies. In the process of performing the task, the strategies can be updated dynamically. This may yield the best results. We find that such modeling and analysis can be carried out in the study of search problems in unknown underwater environment, but we think that when faced with similar problems, this may be a good idea, such as multi-UAV search and multi-robot search. In the future, we will conduct more experiments and apply this framework to more practical applications.