Arbitrage Equilibrium, Invariance, and the Emergence of Spontaneous Order in the Dynamics of Bird-like Agents

Abstract

The physics of active biological matter, such as bacterial colonies and bird flocks, exhibiting interesting self-organizing dynamical behavior has gained considerable importance in recent years. Current theoretical advances use techniques from hydrodynamics, kinetic theory, and non-equilibrium statistical physics. However, for biological agents, these approaches do not seem to recognize explicitly their critical feature: namely, the role of survival-driven purpose and the attendant pursuit of maximum utility. Here, we propose a game-theoretic framework, statistical teleodynamics, that demonstrates that the bird-like agents self-organize dynamically into flocks to approach a stable arbitrage equilibriumof equal effective utilities. This is essentially the invisible handmechanism of Adam Smith’s in an ecological context. What we demonstrate is for ideal systems, similar to the ideal gas or Ising model in thermodynamics. The next steps would involve examining and learning how real swarms behave compared to their ideal versions. Our theory is not limited to just birds flocking but can be adapted for the self-organizing dynamics of other active matter systems.

1. Self-Organization in Active Matter: Background

Active matter describes systems composed of large numbers of self-actualizing agents that consume and dissipate energy resulting in interesting dynamical behavior [1,2,3,4,5,6,7,8,9]. Biological examples of such systems include self-organizing bio-polymers, bacteria, schools of fish, flocks of birds, and so on. This study is focused on birds flocking.

Flocking has been studied extensively from dynamical systems and statistical mechanics perspectives [10,11,12,13,14]. Such analyses have contributed substantially to our evolving understanding of interesting emergent properties such as phase segregation, flock stability, etc. However, these approaches typically do not seem to model explicitly the critical feature of active biological agents, namely, the role of purpose and its attendant pursuit of maximum utility. Being biological agents, birds are innately purposeful, driven by the goal to survive and thrive in challenging environments as Darwin explained. We believe any comprehensive theory of active biological matter needs to overtly account for this defining characteristic of these agents.

We address this need by using a game-theoretic framework, which we call statistical teleodynamics [15,16,17,18,19,20]. The name comes from the Greek word telos, which means goal. Just as the dynamical behavior of gas molecules is driven by thermal agitation (hence, thermodynamics), the dynamics of purposeful agents is driven by the pursuit of their goals and, hence, teleodynamics. Statistical teleodynamics may be considered as the natural generalization of statistical thermodynamics for purpose-driven agents in active matter. It is a synthesis of the central concepts and techniques of potential games theory with those of statistical mechanics toward a unified theory of emergent equilibrium phenomena in active and passive matter [20].

As noted, there is considerable literature on the Reynolds and Vicsek models of birds flocking. Our purpose here is not to reproduce the results of these models, although our approach, too, leads to similar flocking behavior. Our goal is to propose an alternative modeling framework that explicitly accounts for the goal-driven behavior of biologically active agents. Furthermore, our framework answers important questions that are not addressed by the Reynolds model: (i) Is there an equilibrium outcome of this dynamics? (ii) If yes, what is the nature of the equilibrium? (iii) What is the equilibrium configuration? and (iv) Is the equilibrium stable? In addition, our formulation also has the benefit of readily generalizing to other active matter systems in biology, ecology, sociology, and economics, as we describe in the discussion section.

We wish to emphasize that the spirit of our modeling is similar to that of the ideal gas or the Ising model in statistical thermodynamics. Just as real molecules are not point-like objects or devoid of intermolecular interactions, as assumed in the ideal gas model in statistical mechanics, we make similar simplifying assumptions in our bird model. These can be relaxed to make them more realistic in subsequent refinements as van der Waals did, for example, in thermodynamics. The ideal versions serve as useful starting and reference points to develop more comprehensive models of self-organization in active matter systems.

2. Statistical Teleodynamics of Flocking: A Game-Theoretic Formulation

In modeling the dynamics of flocking, there are two dominant approaches: (i) the bottom–up agent-based modeling perspective, as seen in the Reynolds’ Boids model [10] and the Vicsek model [11], and (ii) the top–down statistical mechanics formulation that starts with the specification of the Hamiltonian of the flock and then imposes the maximum entropy distribution on it [13,14].

The Reynolds and Vicsek models specify agent-level dynamical behavior (such as the three rules of cohesion, separation, and alignment for boids). However, they do not predict a final equilibrium state, as there is no notion of equilibrium. The final state is determined only computationally via agent-based simulations.

On the other hand, in the statistical mechanics formulation, it is not clear why maximum entropy, which is obviously relevant for passive matter systems, would be applicable for survival-driven birds. The typical statistical mechanics approach uses the superficial similarity between spins in magnetic systems (e.g., the Ising model) and the orientation of the birds to apply maximum entropy methods. The deeper question of why is this conceptually relevant for birds is not addressed. Most importantly, all these approaches do not seem to recognize explicitly that active agents such as birds act instinctively to improve their survival prospects.

We address these challenges using our statistical teleodynamics framework [15,16,17,18,19,20]. In this theory, the fundamental quantity is an agent’s effective utility, which is a measure of the net benefits pursued by the agent. Every agent behaves strategically to increase its effective utility by switching states and exploiting arbitrage opportunities. In our theory of flocking, we propose that birds are arbitrageurs that maneuver to increase their effective utilities dynamically in flight. The effective utility of a bird depends on its position, speed, and alignment with the rest of the members in its neighborhood.

Hence, we believe that the proper formulation of flocking (in general, any herding behavior) ought to start with a model of effective utility that an agent uses to make such decisions dynamically in flight. Seen from this perspective, we suggest that birds do not fly randomly (as statistical mechanics-based formulations implicitly assume) but maneuver strategically to improve their utilities. We exploit this insight to model the dynamical behavior of birds in flight by using the concepts and techniques from potential games.

In potential games, there exists a single scalar-valued global function called a potential ($\varphi \left(\mathbf{x}\right)$) that has the necessary information about the payoffs or the utilities of the agents. The gradient of the potential is the utility, $h_{ik}$, of the $i$th agent in the $k$th state [15,21,22,23,24]. Therefore, we have

$$h_{ik}\left(\mathbf{x}\right)\equiv \partial \varphi \left(\mathbf{x}\right)/\partial x_k$$

(1)

where $x_k=N_k/N$ and $\mathbf{x}$ is the population vector, where $N_k$ is the number of agents in state k, and N is the total number of agents. A potential game reaches equilibrium, called Nash equilibrium (NE), when the potential $\varphi \left(\mathbf{x}\right)$ is maximized. Furthermore, this Nash equilibrium is unique if $\varphi \left(\mathbf{x}\right)$ is strictly concave [22]. At Nash equilibrium, all agents enjoy the same effective utility, i.e., $h_{ik}=h^{*}$. It is an arbitrage equilibrium [19] where the agents do not have any incentive to switch states anymore, as all states provide the same effective utility $h^{*}$. Thus, the maximization of $\varphi$ and $h_{ik}=h^{*}$ are exactly equivalent criteria (if $\varphi \left(\mathbf{x}\right)$ is strictly concave), and both specify the same outcome, namely, an arbitrage equilibrium.

We are keenly aware, of course, of the simplifications we have made to make the analysis analytically tractable. We realize that our bird-like agents are not real birds, and our models and simulations are not real biological systems. They are stylized ideal systems formulated in the spirit of similar ideal systems in statistical mechanics, as noted. Despite such an ideal approximation, our results nevertheless suggest intriguing possibilities for real biological entities that need to be explored further.

2.1. Agent’s Utility: Position Dependence

Our goal is to develop a simple model of the effective utility ($h_{ik}$) of our bird-like agent i (which we call Garud to differentiate it from the agent Boid of the Reynolds model) in the state k. We want the model to be an appropriate coarse-grained description of the system that can make useful predictions not restricted by system-specific nuances. We have tried, deliberately, to keep the model as simple as possible without losing key insights and relevance to empirical phenomena. One can add more complexity, such as cooperation among agents or environmental conditions, as and when desired later on. Again, what we are aiming for is the equivalent of the ideal gas model or the Ising model for flocking.

Flocking dynamics is modeled by describing the time evolution of the position ${\mathbf{r}}_i$ of the $i$th agent and its velocity ${\mathbf{v}}_i$. Therefore, the state of the $i$th agent is specified by its position ${\mathbf{r}}_i$ and velocity ${\mathbf{v}}_i$. Since the index i itself now denotes the state, we do not need k as a separate index and therefore drop it. The state of the flock at any given time is specified by specifying ${\mathbf{r}}_i$ and ${\mathbf{v}}_i$ for all agents. When the agents move at a constant speed $v_0$, the state of the system is then determined by the set of agents’ positions and velocity directions or orientations ${\{{\mathbf{r}}_i,{\mathbf{s}}_i\}}_{i=1}^N$, where N is the total number of agents.

An agent i is said to have affected an agent j if j is in the neighborhood of i, ${\mathcal{N}}^i$. The neighborhood ${\mathcal{N}}^i$ of i is defined by a matrix whose elements are $n_{ij}$, where

$$\begin{array}{c}\hfill n_{ij}=\left\{\begin{array}{cc}1\hfill & \mathrm{if}j\mathrm{is}\mathrm{a}\mathrm{neighbor}\mathrm{of}i\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.\end{array}$$

(2)

The span of the neighborhood is specified in terms of the absolute distance between i and j, and a size parameter $r_0$ [11,25], such that

$$\begin{array}{c}\hfill n_{ij}=\left\{\begin{array}{cc}1\hfill & |{\mathbf{r}}_i\left(t\right)-{\mathbf{r}}_j\left(t\right)|<r_0\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.\end{array}$$

(3)

As seen from Equation (3), we consider agent i to be its own neighbor. One can also define a neighborhood in terms of a fixed topology of nearest neighbors [12,13,14], but we do not use this specification in this study. It follows that the number of neighbors of an agent i is given by $n_i={\sum }_j{n}_{ij}$.

The effective utility $h_i$ is a model of the cost–benefit trade-offs that the agents use to make decisions dynamically as they maneuver to increase their utilities. There are four terms that contribute to the effective utility of an agent in state i: (i) utility of cohesion, (ii) disutility of congestion, (iii) utility of alignment, and (iv) disutility of competition. We believe that this structure is fairly general, capturing the essence of a wide variety of herding phenomena. The first three are similar to the three rules of agent behavior in the Reynolds model: (i) rule of cohesion, (ii) rule of separation, and (iii) rule of alignment. Our fourth term is unique and is not present explicitly in the Reynolds or the Vicsek models. As we explain below, this term corresponds to an entropic restlessness of the agents.

We consider the $i$th agent’s position in the frame of reference of the center of mass of its neighborhood. The utility of cohesion for the $i$th agent is proportional to its number of neighbors, $n_i$. However, this benefit comes at the cost of congestion, the disutility of congestion. The trade-off between the two terms, the cost–benefit trade-off, results in an inverted-U profile, which, following Venkatasubramanian [17], can be modeled as,

$$h_r^{\left(i\right)}=\alpha n_i-\beta n_i^2$$

(4)

where $h_r^{\left(i\right)}$ is the position component of the utility for the $i$th agent; $\alpha ,\beta >0$. For many things in life, the cost–benefit trade-off is in the form of an inverted-U curve. Consider, for example, taking some medicine to cure an ailment. If one is suffering from a severe headache, taking one tablet might make you feel a little better, but taking two might help more. This does not mean that taking ten would help a lot more! It could actually make one sicker, triggering a whole set of more serious problems. As the dosage increases beyond a critical point, the benefit begins to go down, sometimes dramatically. This is so because the cost of the treatment (in the form of negative side effects) begins to exceed the benefit. Equation (4) captures and models this essential and near-universal trait in most things in life.

Note that the positional dependence is accounted for in the computation of $n_i$. Given a configuration of all agents $\left\{{\mathbf{r}}_i\right\}$, the neighborhood of the $i$th agent is defined by the parameter $r_0$, where if the $j$th agent is within this radius, then it is considered a neighbor.

This, in turn, identifies a direction of increased utility, which is given by,

$$\frac{\partial h_r^{\left(i\right)}}{\partial {\mathbf{r}}_i}=\alpha \frac{\partial n_i}{\partial {\mathbf{r}}_i}-2\beta n_i\frac{\partial n_i}{\partial {\mathbf{r}}_i}$$

(5)

${\displaystyle \frac{\partial n_i}{\partial {\mathbf{r}}_i}}$ is dependent on the agents in the perimeter of the neighborhood of the reference agent i.

2.2. Agent’s Utility: Velocity Dependence

The utility of an agent is also dependent on the velocity of its neighbors in that the agent attempts to match the orientation with its neighboring agents. This utility of alignment ($h_v^{\left(i\right)}$) can be written as ($\gamma >0$),

$$h_v^{\left(i\right)}=\gamma \sum _j{n}_{ij}\frac{{\mathbf{v}}_i}{|{\mathbf{v}}_i|}·\frac{{\mathbf{v}}_j}{|{\mathbf{v}}_j|}$$

(6)

The utility of the $i$th agent, then, depends on the orientation of the other agents in its neighborhood, i.e., ${\mathbf{s}}_i·{\mathbf{s}}_j$, where j is a neighbor of agent i. This gives the alignment utility for the $i$th agent as,

$$h_v^{\left(i\right)}=\gamma \sum _j{n}_{ij}{\mathbf{s}}_i·{\mathbf{s}}_j$$

(7)

where $n_{ij}$ shows if the $j$th agent is a neighbor of agent i, i.e.,

$$\begin{array}{c}\hfill n_{ij}=\left\{\begin{array}{cc}1\hfill & |{\mathbf{r}}_i\left(t\right)-{\mathbf{r}}_j\left(t\right)|<r_0\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.\end{array}$$

(8)

If each agent is perfectly aligned with its neighbors, this utility component is maximal, whereas if they are oriented in the opposite direction, this is minimal. Therefore, the agents prefer to be aligned. This gives the $i$th agent an arbitrage opportunity to adjust its velocity vector toward this direction to increase its utility. This opportunity for increasing its utility generates a self-propelled force on the $i$th agent. If the $i$th agent is not aligned with its neighbors, this direction of increased utility is given by,

$$\frac{\partial h_v^{\left(i\right)}}{\partial {\mathbf{s}}_i}=\gamma \sum _j{n}_{ij}{\mathbf{s}}_j$$

(9)

2.3. Agent’s Effective Utility

There is the fourth utility component remaining to be considered. As noted, this is not stated explicitly in the three rules of the boids. However, it is implied because it is assumed that the boids have to be constantly moving. So, as an agent incessantly jockeys and moves for better positions and orientations, its ability to do so is hampered by the competition from other agents in its neighborhood that are also trying to do the same. As Venkatasubramanian explains [17], this disutility of competition can be modeled as $-\delta ln{n}_i$. This term, when integrated to obtain the potential $\varphi \left(\mathbf{x}\right)$, leads to entropy in statistical mechanics. Thus, maximizing the potential $\varphi \left(\mathbf{x}\right)$ is equivalent to maximizing entropy. This is why we call this term entropic restlessness. In order to increase the effective utility $h_i$, the agents try to minimize $ln{n}_i$ (because of the negative sign in front of it). Thus, the agents try to move to other states that have lower $n_i$. Since they are constantly doing this, this leads to restless behavior and constant movement. For more details, the reader is referred to Venkatasubramanian [17,20].

Now, by combining all these components, we arrive at the effective utility for the $i$th agent given by

$$\begin{array}{cc}\hfill h_i& =\alpha n_i-\beta n_i^2+\gamma \sum _j{n}_{ij}{\mathbf{s}}_i·{\mathbf{s}}_j-\delta ln{n}_i\\ \hfill h_i& =\alpha n_i-\beta n_i^2+\gamma n_i{l}_i-\delta ln{n}_i\end{array}$$

(10)

where $l_i$ = $\frac{1}{n_i}{\sum }_j{n}_{ij}{\mathbf{s}}_i·{\mathbf{s}}_j$ is the average alignment of agent i. Without any loss of generality, $\delta$ can be assumed to be 1 and will be assumed as such for the rest of this paper. When $\alpha ,\beta ,\gamma =0$, the agents do not have any preferences and hence fly around randomly. This is what is captured by the remaining $-ln{n}_i$ term, i.e., entropic restlessness. These four components are summarized in

Table 1. Components of the effective utility $h_i$.

Utility of Cohesion	$\alpha n_i$	Benefit of having $n_i$ neighbors
Disutility of Congestion	$-\beta n_i^2$	Cost of crowding by $n_i$ neighbors
Utility of Alignment	$\gamma {\sum }_j{n}_{ij}{\mathbf{s}}_i·{\mathbf{s}}_j$	Benefit of being aligned with the neighbors
Disutility of Competition	$-ln{n}_i$	Cost of competition from the $n_i$ neighbors
		Entropic restlessness

for the convenience of the reader.

Statistical teleodynamics, via potential game theory, proves that the self-organizing dynamics of the agents would eventually result in arbitrage equilibrium, where $\varphi \left(\mathbf{x}\right)$ is maximized, and the effective utilities of all the agents are the same, i.e., $h_i=h^{*}$. This proof is a well-known result, and our reader is referred to the literature for more details [15,16,18,20,22]. Thus, this answers the first question we raised in Section 1 in the affirmative. The answer to the second question is that it is an arbitrage equilibrium where the effective utilities of all the agents are equal.

In the next section, we discuss our agent-based simulation results that confirm this prediction. We also answer the remaining two questions. Although our framework is somewhat similar to the reward-driven agent-based model of Durve et al. [26] or of Katare et al. [27], it is entirely different in the conceptual and mathematical formulation. The other approaches do not use potential game theory and arbitrage equilibrium in their formulations, which are important distinctions.

3. Results

For illustrative purposes, we show that our formulation leads to flocking behavior that is similar to that of the Reynolds model. Again, we wish to remind the reader that our objective is not to mimic Reynolds’s model. Our objective is to demonstrate an entirely different modeling framework of self-organization that is founded on goal-driven arbitrage behavior and utility maximization by biological agents.

The Reynolds model for the $i\mathrm{th}$ agent is given by the equation (see Supplementary Information, Equation (11)),

$$\begin{array}{c}\hfill {\mathbf{v}}_i(t+\Delta t)={\mathbf{v}}_i\left(t\right)+\left(\frac{a}{n_i}\sum _j{n}_{ij}({\mathbf{r}}_j-{\mathbf{r}}_i)+b\sum _j{n}_{ij}({\mathbf{r}}_i-{\mathbf{r}}_j)+\frac{c}{n_i}\sum _j{n}_{ij}({\mathbf{v}}_j-{\mathbf{v}}_i)+\mathit{\eta }\left(t\right)\right)\Delta t\end{array}$$

(11)

where $a,b,$ and c are parameters corresponding to the rules of cohesion, separation, and alignment, respectively. Parameter $\mathit{\eta }\left(t\right)$ is the uncorrelated noise in the agent’s velocity. The time-scale, $\Delta t$, in Equation (11), can be subsumed in $a,b,c$. For the simulation details, the reader is referred to Methodology in Section 5 below. If the agents are flying randomly, without pursuing utility, then this base case corresponds to $\alpha ,\beta ,\gamma ,=0;\delta =1$. This result is discussed in Section S4.

For the other cases, the effective utility function in Equation (10) is plotted in Figure 1 in terms of the number of neighbors of agent i ($n_i$) for different alignments ($l_i$) for a given set of $\alpha ,\beta ,\gamma$. We see that there are two values of ${\widehat{n}}_i$ (${\widehat{n}}_{-}$ and ${\widehat{n}}_{+}$) where the gradient of utility, for a given value of alignment, is zero. These correspond to the optimal flock size ${\widehat{n}}_i$ for maximum effective utility ($h_i$). These values are determined by (see Supplementary Information, Section S1).

$${\widehat{n}}_i={\displaystyle \frac{(\alpha +\gamma l_i)\mp \sqrt{{(\alpha +\gamma l_i)}^2-8\beta \delta }}{4\beta }}$$

(12)

In Figure 1, at the lower value (${\widehat{n}}_{-}$), any deviation in ${\widehat{n}}_{-}$ increases the utility of the agent and hence leads to an unstable point. However, for the higher point (${\widehat{n}}_{+}$), we see that any deviation reduces the agent’s utility. Therefore, this leads to a stable point, as any deviation would bring an agent back to the higher utility state. Therefore, this is the point an agent will try to reach to maximize its utility. For example, for the red curve, this would correspond to the point where ${\widehat{n}}_{+}=73.6$. However, despite this point’s stability, an agent will not be able to stay there indefinitely, as the other agents in its neighborhood are competing for its position by constantly changing their positions and orientations in their flights. Therefore, the $i$th agent would be fluctuating around this point. This behavior is similar to what is seen in other systems around the spinodal and binodal points [20]. While the individual agents would prefer to be at their maximum utility point (the spinodal point), the system as a whole is driven by the competition among the agents to the binodal point where the potential $\varphi \left(\mathbf{x}\right)$ is maximized. Thus, the agents wander around in phase space between these two points at equilibrium, as shown in the following.

In Figure 2 and Figure 3, we show the simulation results of both the Reynolds model and our utility-driven agent model. The results are shown for different parameter values of ($a,b,c$) and ($\alpha ,\beta ,\gamma$). From the simulations, we obtain a set of position and velocity values $\{{\mathbf{r}}_i,{\mathbf{v}}_i\}$ of each agent i at every time-step. Once this is obtained, we extract the features $n_i$ and $l_i$ for all the agents. This, in turn, is used to compute the average number of neighbors $\overline{n}$ and average alignment $\overline{l}$ for the entire population for all time points (Figure 2).

Figure 2 shows the snapshots at different time points of the evolution in 3D space and the $n_i-l_i$ phase space. While the exact dynamics, the exact configuration of the population, and the time scale of evolution of these two models cannot be the same (and that is not the aim, either), we observe that the qualitative patterns of collective behavior are very similar in both cases. In particular, we see that all agents gravitate toward a certain region in the $n_i-l_i$ phase space for both models. They both start at the lower far right point (where the average alignment is near zero as the agents are all randomly oriented initially and closely packed) and evolve toward the upper center-right region in black. We notice a qualitative match of both trajectories.

Furthermore, we can also see a quantitative similarity between the two models for specific parameters (Figure 3). Figure 3a shows the phase space for the Reynolds model, and Figure 3b shows the same for the utility-driven model. In both models, we notice similar features of evolution toward the arbitrage equilibrium states, starting from the lower right point at time $t=0$ to ending in the colored regions, where the average number of neighbors and the average alignment fall in similar corresponding regions. The plots on the right show the average alignment and average number of neighbors of the agents in the last 100 time-steps.

Although both models exhibit similar collective behaviors, it is not apparent, however, from the three rules of the Reynolds model that its dynamics would result in an equilibrium state in the $n_i-l_i$ phase space. This is an important difference between the two approaches. Since the potential game formulation predicts an arbitrage equilibrium outcome (thus answering questions (i) and (ii)), it is clear right from the beginning where in the phase space the system is going to end up (and hence answer the question (iii)). We can therefore make a quantitative prediction about the optimal flock size ${\widehat{n}}_{+}$ and the corresponding effective utility $\widehat{h}$ values at equilibrium. From Equation (12), we can calculate the optimal flock size ${\widehat{n}}_{+}$ for the three configurations shown in Figure 3b. We compare these predictions with the observed values from the simulations (

Table 2. Optimal flock sizes at equilibrium.

Configuration	Predicted Flock Size	Observed Flock Size
$\alpha =0.1,\beta =0.005,\gamma =0.25$	31.9	25.3 ± 8.0
$\alpha =0.3,\beta =0.005,\gamma =0.25$	53.1	50.1 ± 16.0
$\alpha =0.5,\beta =0.005,\gamma =0.02$	50.0	44.5 ± 13.4

). As we can see, the predictions are consistent with the simulation results. As noted above, given the competitive dynamics of the garuds, the agents are not able to stay with the optimal flock size but keep fluctuating around it as the standard deviation metrics indicate.

This ability to predict the final outcome of the collective behavior of the population, given the individual agent-level properties captured in the utility function $h_i$, is an important defining feature and strength of the statistical teleodynamics framework. An additional characteristic is the ability to prove the stability of the final outcome, as we discuss in Section 3.1, and answer our fourth question [15,17].

We also ran the simulations for different time-step sizes of $\Delta t=0.01,0.1,0.5$ in Equation (11) to understand the dynamics of the evolution better. Note in Figure 4 that at the start ($t=0$), the utilities of all the agents are spread out, with many having negative utility values and the average utility being ($\overline{h}$) low. However, as the dynamics evolves, every agent tries to increase its utility by maneuvering to a better neighborhood and better orientation, the distribution becomes narrower, the average utility keeps increasing, and it reaches a near-maximum value ($\overline{h}=22.17\pm 2.90$ in Figure 4a) and fluctuates around it. Note that this is around the maximum theoretical value of about 23.8 (given by Equation (10)), where the histogram peaks. This suggests that nearly all the agents have similar effective utilities asymptotically, approaching the maximum. This, of course, is the arbitrage equilibrium outcome predicted by the theory (see also Supplementary information, Figure S3). The agents do not converge exactly on $h^{*}$ but fluctuate around it because of the stochastic dynamics. This is also seen in

Table 3. Utility of different percentiles of the agents at the 1000th time-step corresponding to Figure 4.

Population	Time-Step Size, $\Delta \mathit{t}$	Average Utility
Top 1%	0.01	23.79
	0.1	23.80
	0.5	23.76
Top 1–10%	0.01	23.77
	0.1	23.78
	0.5	23.68
Top 10–50%	0.01	23.60
	0.1	23.51
	0.5	23.35
Top 50–75%	0.01	22.83
	0.1	22.64
	0.5	22.58
Bottom $50\%$	0.01	18.59
	0.1	15.59
	0.5	17.91

where nearly the top 10 % of the agents at a particular time-step are very close to the maximum utility value. In fact, the top $50\%$ of the agents have an average utility of greater than 23.

At the risk of belaboring this point, we reiterate, in order to be clear, that as noted in Figure 1 and Figure 4, the self-organized dynamics strives toward maximum utility. From Equation (10), for a given set of $\alpha$, $\beta$, $\gamma$, and $\delta$ values, we can determine the arbitrage equilibrium state defined by $\widehat{h}$ and ${\widehat{n}}_{+}$. For $\alpha =0.5$, $\beta =0.005$, $\gamma =0.25$, and $\delta =1$, we have $\widehat{h}=23.8$ and ${\widehat{n}}_{+}=73.6$. This is predicted by our model analytically. We see this confirmed in the simulation as shown in Figure 4 and Table 1. Due to the inherent stochastic dynamical nature of the agents’ movements, the agents cannot stay at the optimum even after finding it. They keep moving around about it. We see this from the table: we see that about 50% of the agents enjoy a utility of 23.6, which is very close to the theoretical maximum of 23.8 (for a step size of 0.01).

This arbitrage equilibrium state is unique only if the potential $\varphi \left(\mathbf{x}\right)$ is strictly concave [22]. For our boids-like agents, this is not the case in general, as the concavity would depend on $\alpha$, $\beta$, and $\gamma$ having some particular values. So, for the typical case where $\varphi \left(\mathbf{x}\right)$ is not concave, there could be multiple equilibrium configurations of the agents. Thus, instead of an equilibrium point in the $n_i-l_i$ phase space, one has an equilibrium region in general. In other words, invoking a terminology from chaos and nonlinear dynamics, there is a basin of attraction in the phase space where the agents finally settle in and fly around. We see this in Figure 2 and Figure 3—the basins are the colored regions.

3.1. Stability of the Arbitrage Equilibrium

We can answer our fourth question and determine the stability of this equilibrium by performing a Lyapunov stability analysis [15,17]. A Lyapunov function V is a continuously differentiable function that takes positive values everywhere except at the equilibrium point (i.e., V is positive definite), and it decreases (or is nonincreasing) along every trajectory traversed by the dynamical system ($\dot{V}$ is negative definite or negative semidefinite). A dynamical system is locally stable at equilibrium if $\dot{V}$ is negative semidefinite and is asymptotically stable if $\dot{V}$ is negative definite.

Following Venkatasubramanian [17], we identify our Lyapunov function $V\left(\mathit{n}\right)$

$$\begin{array}{c}\hfill V\left(\mathit{n}\right)={\varphi }^{*}\left({\mathit{n}}^{*}\right)-\varphi \left(\mathit{n}\right)\end{array}$$

(13)

where ${\varphi }^{*}$ is the potential at the Nash equilibrium (recall that ${\varphi }^{*}$ is at its maximum at NE), $\varphi \left(\mathit{n}\right)$ is the potential at any other state, and $\mathit{n}$ is the vector of the neighbors ($n_i$) of all agents. Note that $V\left(\mathit{n}\right)$ has the desirable properties we seek: (i) $V({\mathit{n}}^{*}$) = 0 at NE and $V\left(\mathit{n}\right)$> 0 elsewhere, i.e., $V\left(\mathit{n}\right)$ is positive definite; (ii) since $\varphi \left(\mathit{n}\right)$ increases as it approaches the maximum, $V\left(\mathit{n}\right)$ decreases with time, and hence, it is easy to see that $\dot{V}$ is negative definite. Therefore, the arbitrage equilibrium is not only stable but also asymptotically stable.

Our simulation results confirm this theoretical prediction (see Figure 5). After the agents population reached equilibrium, we disturbed the equilibrium state by randomly changing the positions and/or velocities of the agents. The simulation is then continued from the new disturbed far-from-equilibrium state. We conducted experiments with three kinds of disturbances:

• Disturbance 1: Velocity disturbance, where each agent’s velocity is changed to a random orientation and magnitude.
• Disturbance 2: Position disturbance, where each agent’s position is randomly changed.
• Disturbance 3: Position and velocity disturbance, where both position and velocity vectors are changed.

As seen in Figure 5, after the 100th time-step, when the population had reached equilibrium, we introduced these disturbances. The 101st time-step shows in red color the new far-from equilibrium states, where the average utility (h) has dropped considerably. In all cases, the population recovers quickly, typically in another 100 time-steps or so, to reach the original equilibrium region (shown in green). The figure shows the disturbance (red) and recovery (green) in both the 3D space and the phase space.

In Figure 5a, as the velocities are randomized at the 101st time-step, the alignment goes down to 0, but it recovers to the original equilibrium quickly. In Figure 5b, as the new configuration corresponds to a similar value of an average number of neighbors as before, the disturbance is not that much. Note that the drop in average utility is small. In Figure 5c, we see that this disturbance is huge, pushing the configuration close to the original random state. Still, the population is able to recover to the arbitrage equilibrium quickly.

This shows that the arbitrage equilibrium region is not only stable but asymptotically stable. That is, the agents’ flocking configuration is resilient and self-healing. Given the speed of the recovery, it could possibly be exponentially stable, but we have not proved this analytically here. It is interesting to observe that this result is similar to that of the income-game dynamics [15,17].

4. Conclusions

For three centuries, we have known that there are constants of motion, such as energy and momentum, for passive matter. Nevertheless, it comes as a surprise to discover that the dynamics of active matter populations could also have an invariant, namely, the effective utility. However, the role of invariance here is different from its role in classical mechanics. The constants of motion, such as energy and momentum, are conserved, whereas the effective utility is not. The role of this invariance is more like that of set-point tracking and disturbance rejection in feedback control systems [28,29]. The system, i.e., the agents’ population, adjusts itself dynamically and continually, in a feedback control-like manner, to maintain its overall maximum potential $\varphi \left(x\right)$.

It is important to emphasize, however, that this control action is decentralized as opposed to the typical centralized control system in many engineering applications. The agents individually self-organize, adapt, and dynamically course correct to offset the negative impact on their effective utilities by other agents or other external sources of disturbance. The population as a whole stochastically evolves toward the stable basin of attraction in the phase space in a self-organized and distributed-control fashion.

The same mathematical framework has been demonstrated for other dynamical systems, as summarized in

Table 4. Utility function in different domains.

Domain	System	Utility Function (${\mathit{h}}_{\mathit{i}}$)
Physics	Thermodynamic game	$-\beta ln{E}_i-ln{n}_i$
Biology	Bacterial chemotaxis	$\alpha c_i-ln{n}_i$
Ecology	Ant crater formation	$b-{\displaystyle \frac{\omega r_i^a}{a}}-ln{n}_i$
Sociology	Segregation dynamics	$\eta n_i-\xi n_i^2+ln(H-n_i)-ln{n}_i$
Economics	Income game	$\alpha ln{S}_i-\beta {\left(ln{S}_i\right)}^2-ln{n}_i$
Ecology	Garuds game	$\alpha n_i-\beta n_i^2+\gamma n_i{l}_i-ln{n}_i$

, in biology [20], ecology [20], sociology [20], and economics [15,17,19] to predict emergent phenomena via self-organization. As Venkatasubramanian et al. [20] showed, the emergence of the exponential energy (i.e., Boltzmann) distribution for gas molecules can be modeled by the effective utility

$$h_i=-\beta ln{E}_i-ln{n}_i.$$

(14)

Similarly, they showed [20] that for biological systems, the cost–benefit trade-off in effective utility $h_i$ for bacterial chemotaxis can be modeled by

$$h_i=\alpha c_i-ln{n}_i$$

(15)

where the first term is the benefit derived from a resource ($c_i$, $\alpha >0$) and the second is the cost of competition as modeled in Equation (10).

In a similar vein, the emergence of ant craters can be modeled by

$$h_i=b-\frac{\omega r_i^a}{a}-ln{n}_i.$$

(16)

where the first term (b) is the benefit of having a nest for an ant, the second term is the cost of work performed when carrying the sand grains out to build the nest, and the last term is again the cost of competition as before.

The same study showed how the Schelling game-like segregation dynamics in sociology can be modeled by

$$\begin{array}{c}\hfill h_i=\eta n_i-\xi n_i^2+ln(H-n_i)-ln{n}_i\end{array}$$

(17)

where the first term is the benefit of the community of neighbors, the second is the congestion cost of such neighbors, the third is the benefit of relocation options, and the last is again the cost of the competition.

In economics, the emergence of an income distribution can be modeled by

$$h_i=\alpha ln{S}_i-\beta {\left(ln{S}_i\right)}^2-ln{n}_i.$$

(18)

where the first term is the benefit of income, the second is the cost of work expended to earn this income, and the last is again the cost of the competition.

By comparing Equations (14) through (18) with equation Equation (10) for garuds ($\delta =1$),

$$h_i=\alpha n_i-\beta n_i^2+\gamma n_i{l}_i-ln{n}_i$$

We observe a certain universality in the structure of the effective utility functions in different domains. They are all based on cost–benefit trade-offs, but the actual nature of the benefits and costs depends on the details of the specific domain, as one would expect. Thus, we see that the same conceptual and mathematical framework is able to predict and explain the emergence of spontaneous order via self-organization to reach arbitrage equilibrium in dynamical systems in physics, biology, ecology, sociology, and economics.

These results suggest that the pursuit of maximum utility or survival fitness could be a universal self-organizing mechanism. In the case of birds, the incessant search for improving survival fitness occurs in the three-dimensional physical space, with the birds trying to move to a better location. In biology, in general, the search for improving one’s fitness occurs in the design space of genetic features. Here, the mutation and crossover operations facilitate the movements in the feature space, such that an agent improves itself genetically via Darwinian evolution to increase its utility, i.e., the survival fitness. In economics, on the other hand, agents search in the products and/or services space so that they can offer better products/services to improve their economic survival fitness in a competitive marketplace. Thus, this mechanism is essentially the same as Adam Smith’s invisible hand. In all these different domains, every agent is pursuing its own self-interest to increase its own $h_i$, but a stable collective order emerges, nevertheless, spontaneously via self-organization.

Thus, our theory shows an important result that these so-called out-of-equilibrium systems are actually in equilibrium, an arbitrage equilibrium. Just as systems can be in mechanical equilibrium when forces or pressures are equal or in thermal equilibrium when temperatures are equal, or in phase equilibrium when the chemical potentials are equal, we have active matter systems in arbitrage equilibrium when the utilities are equal.

As noted, we do realize that our bird-like agents are not real birds, and we are not implying that real birds make decisions along the lines of our model necessarily. Nevertheless, our results suggest interesting possibilities for real biological entities that need to be studied further. What we have here is for ideal systems, of course, similar to the ideal gas or the Ising model in thermodynamics. Just as real gases and liquids do not behave exactly like their ideal versions in statistical thermodynamics, we do not expect real biological systems (or economic or ecological systems) to behave like their ideal counterparts in statistical teleodynamics. Nevertheless, the ideal versions serve as useful starting and reference points as we develop more comprehensive models of active matter systems. The next steps would involve examining and learning how real-world biological systems behave compared to their ideal versions. This would, of course, necessitate several modifications to the ideal models.

5. Methods

We created a simulation of 1000 agents in a periodic box of dimensions $20\times 20\times 20$, where each agent’s neighborhood is a sphere with radius $r_0=3$. Each agent starts at a random location and orientation inside a $10\times 10\times 10$ block. The speed of each agent is limited between $0.5$ and 1. The update algorithm works similar to Reynolds’ agent’s update, except the force is driven by the numerical estimates of the direction of increased utility (additively based on position and velocity). An additional noise is also added to the velocity update strategy similar to Reynolds’ model to capture the erroneous strategies of velocity update for each agent. This is given by a noise parameter (0.01, unless specified) times the magnitude of the velocity. The noise indicates that an agent does not make perfect choices in updating its velocity. It is to be noted that there are different ways of realizing the dynamics of the system, which may result in distinctive dynamic behaviors. The goal, however, is to show that they eventually reach the same game-theoretic equilibrium.