A Multi-Agent Cooperative Group Game Model Based on Intention-Strategy Optimization

Abstract

With the rapid advancement of artificial intelligence technology, multi-agent systems are being widely applied in fields such as autonomous driving and robotic collaboration. However, existing methods often suffer from the disconnection between intention recognition and strategy optimization, leading to inefficiencies in group collaboration. This paper proposes a multi-agent cooperative group game model based on Intention-Strategy Optimization (ISO-MAGCG). The model establishes a two-layer optimization framework encompassing intention and strategy, enabling dynamic adaptation through the co-evolution of upper-layer intention recognition and lower-layer strategy optimization. A Group Attention-based Intention Recognition Network (GAIN) is designed to efficiently capture complex interactions among agents. Furthermore, an Adaptive Group Evolution Algorithm (AGEA) is proposed to ensure the stability of large-scale cooperative endeavors. Experiments conducted in navigation, resource collection, and defense collaboration scenarios validate the effectiveness of the proposed method. Compared with mainstream algorithms such as QMIX, MADDPG, and MAPPO, ISO-MAGCG demonstrates significant superiority in metrics including task success rate and cooperative efficiency, achieving an average improvement of 8.4% in task success rate, a 12% enhancement in cooperative efficiency, and an intention recognition accuracy of 94.3%. The results indicate notable performance advantages and favorable scalability.

1. Introduction

With the rapid development of artificial intelligence technology, Multi-Agent Systems (MASs) have found extensive applications in diverse fields such as autonomous driving [1], robotic collaboration, intelligent transportation, resource allocation, and distributed computing. In these complex application scenarios, multiple agents need to collaborate and compete effectively within dynamic environments to optimize the overall system performance. However, traditional multi-agent collaboration methods often rely on pre-defined coordination mechanisms or simplistic communication protocols, struggling to adapt to complex and changing environmental conditions [2]. Furthermore, as the number of agents increases and environmental complexity grows, achieving effective intention recognition, strategy optimization, and group coordination among agents has become a critical challenge. Particularly in partially observable environments, agents must infer the intentions of others through observed behaviors and adjust their own strategies accordingly, posing new challenges to traditional game theory and reinforcement learning methods.

Current research on multi-agent cooperation primarily focuses on reinforcement learning, game theory, and intention recognition, yet several limitations persist. In Multi-Agent Reinforcement Learning (MARL), the QMIX algorithm proposed by Rashid et al. (2020) enables centralized training with decentralized execution via monotonic value function factorization [3], but its performance degrades significantly in non-monotonic environments. QTRAN, developed by Son et al. (2019), relaxes the monotonicity assumption [4] but suffers from poor training stability. The QPLEX algorithm, introduced by Kong et al. (2025), enhances expressive capacity through dueling decomposition [5], albeit at the cost of high computational complexity. In the field of intention recognition, Li et al. (2025) designed an intention inference method based on Variational Autoencoders (VAEs) [6], which lacks adaptability in dynamic settings. Zhang et al. (2025) incorporated attention mechanisms within a multi-agent actor-critic framework [7], but omitted explicit intention modeling. Chernyavskiy et al. (2025) investigated the role of opponent modeling in multi-agent learning [8], yet the computational overhead is substantial. Regarding cooperative game theory, Xu et al. (2025) studied dynamic coalition formation [9] but did not account for real-time intention changes. Asadi et al. (2021) proposed deep learning-based solutions for cooperative games [10], but theoretical guarantees for convergence are lacking. Juan et al. (2013) explored coordination mechanisms in large-scale multi-agent systems [11], yet scalability remains limited. Yun et al. (2015) analyzed cooperative strategies under incomplete information [12], but the intention modeling is oversimplified. Li et al. (2022) introduced a graph neural network-based approach for multi-agent collaboration [13] but neglected the temporal characteristics of intentions. More recently, transformer-based architectures have been introduced to multi-agent learning. Pang et al. (2022) proposed the Multi-Agent Transformer (MAT) that leverages sequential modeling and cross-agent attention for coordination [14]. Zhao et al. (2020) introduced ROMA, which enables emergent role discovery through a role encoder that clusters agents with similar behaviors [15]. While these methods demonstrate improved coordination capabilities, they primarily focus on learning implicit coordination patterns rather than explicit intention modeling, and the integration of interpretable intention recognition with strategy optimization remains underexplored. In summary, existing methods predominantly suffer from the following issues: a lack of synergistic mechanisms integrating intention recognition and strategy optimization; low computational efficiency in large-scale groups; insufficient adaptability and robustness in dynamic environments; and inadequate theoretical convergence analysis.

To address the aforementioned challenges, this paper proposes a multi-agent cooperative group game model based on Intention-Strategy Optimization (ISO-MAGCG). The main contributions of this work are as follows. Firstly, an intention-strategy bi-level optimization framework is constructed, enabling dynamic adaptation of intentions and strategies through the co-evolution of upper-level intention recognition and lower-level strategy optimization. Secondly, a Group Attention-based Intention recognition Network (GAIN) is designed, which efficiently captures complex interaction relationships and intention dependencies among agents through dynamic graph modeling and multi-head attention mechanisms. Finally, an Adaptive Group Evolution Algorithm (AGEA) is proposed, which ensures the stability and convergence of large-scale group cooperation via dynamic weight allocation and distributed solution methods. The primary innovations of this paper lie in establishing a unified theoretical framework integrating intention recognition and strategy optimization; proposing an efficient method for modeling group intentions; and designing an adaptive equilibrium algorithm with theoretical guarantees.

2. Fundamental Theory

2.1. Multi-Agent Reinforcement Learning Theory

Multi-Agent Reinforcement Learning (MARL) represents an extension of reinforcement learning to multi-agent environments [16], with its theoretical foundation rooted in Markov Games. Unlike single-agent reinforcement learning, the optimal policy of each agent in multi-agent environments depends not only on the environmental state but is also influenced by the strategies of other agents, rendering the learning process significantly more complex [17]. In multi-agent systems, agents exhibit intricate interaction patterns, including cooperative, competitive, and mixed relationships, which directly impact both the overall system performance and individual rewards [18].

Formally, an n-agent Markov game can be defined as a tuple, as shown in Equation (1), with the multi-agent Markov game illustrated in Figure 1:

$$g=⟨N,S,{\left\{A_i\right\}}_{i=1}^N,{\left\{R_i\right\}}_{i=1}^N ⟩$$

(1)

where the parameters are defined as follows: N = {1, 2, …, n} represents the agent set, encompassing all decision-making agents in the system; S denotes the global state space, describing all possible environmental states; Ai represents the action space of agent i, defining all executable actions for that agent; P: S × A × S → [0, 1] denotes the state transition probability function, where A = A1 × A2 × … × An is the joint action space; Ri: S × A→R represents the reward function of agent i; and γ ∈ [0, 1) represents the discount factor, which balances the importance of immediate rewards versus future rewards.

Figure 1. Multi-Agent Markov Game Framework Diagram.

Each agent’s policy is defined as a mapping from states to probability distributions over actions, as shown in Equation (2):

$${\pi }_i:S\to \mathsf{\Delta }\left(A_i\right)$$

(2)

where Δ(Ai) represents the probability distribution over the action space. The joint policy is denoted as π = (π1, π2, …, πn), which describes the strategy combination of all agents.

The state value function of agent i under joint policy π is defined as the expected cumulative discounted reward obtainable from state S, following policy ππ, as shown in Equation (3):

$$V_i^{\pi }\left(s\right)={\mathbb{E}}_{\pi }\left[{\displaystyle \sum _{t=0}^{\infty }{\gamma }^t}{r}_i^{t+1}{\mathrm{bigg}|s}_0=s\right]$$

(3)

where $r_i^{t+1}$ represents the immediate reward obtained by agent i at time t, and Eπ denotes the expectation under joint policy π.

Correspondingly, the state-action value function (Q-function) is defined as shown in Equation (4):

$$Q_i^{\pi }\left(s,a\right)={\mathbb{E}}_{\pi }\left[{\displaystyle \sum _{t=0}^{\infty }{\gamma }^t}{r}_i^{t+1}{\mathrm{bigg}|s}_0=s, a_0=a\right]$$

(4)

where a represents the joint action.

Nash equilibrium is an important solution concept in multi-agent games, describing a stable strategy combination where no agent can obtain higher rewards by unilaterally changing their strategy. A strategy combination π* constitutes a Nash equilibrium if and only if:

$$V_i^{\pi }\left(s\right)\ge V_i^{\left({\pi }_{-i},{{\pi }^{\prime }}_i\right)}\left(s\right),\forall i\in \mathcal{N},\forall {{\pi }^{\prime }}_i\in {\Pi }_i,\forall s\in S$$

(5)

where ${\pi }_{-i}^{*}=\left({\pi }_1^{*},\dots ,{\pi }_{i-1}^{*},{\pi }_{i+1}^{*},\dots ,{\pi }_n^{*}\right)$ denotes the equilibrium strategies of all agents except agent i, and ${{\pi }_i}^{\prime }\in {\Pi }_i$ represents any alternative strategy that agent i could adopt. This condition ensures that at Nash equilibrium, no agent has an incentive to unilaterally deviate from the current strategy.

While the Nash equilibrium originates from non-cooperative game theory, its application in cooperative multi-agent systems requires clarification. In this work, we adopt the framework of potential games [19], where a global potential function Φ exists such that:

$$V_i\left({\pi }_i,{\pi }_{-i}\right)-V_i\left({{\pi }_i}^{\prime },{\pi }_{-i}\right)=\mathsf{\Phi }\left({\pi }_i,{\pi }_{-i}\right)-\mathsf{\Phi }\left({{\pi }_i}^{\prime },{\pi }_{-i}\right),\hspace{1em}\forall i,{\pi }_i,{{\pi }_i}^{\prime }$$

(6)

In potential games, the Nash equilibrium corresponds to local maxima of the potential function Φ. Combined with the superadditivity property of cooperative games, this ensures that the Nash equilibrium sought by AGEA represents a stable cooperative configuration where individual and group incentives are naturally aligned.

2.2. Intention Recognition Theory

Intention recognition refers to the process of inferring the internal goals and intentions of agents by observing their behavioral sequences. In multi-agent environments, accurate intention recognition is crucial for achieving effective cooperation, as it enables agents to predict the future behaviors of other agents and thereby formulate better cooperative strategies. The challenge of intention recognition lies in the fact that the mapping relationship between behaviors and intentions is typically many-to-one, meaning that identical behaviors may correspond to different intentions, while different behaviors may also stem from the same intention [20].

2.2.1. Bayesian Intention Inference

Bayesian intention inference is based on Bayes’ theorem, updating beliefs about agent intentions through observed behavioral sequences. The core idea of this approach is to transform the intention recognition problem into a probabilistic inference problem, gradually approximating true intentions by continuously updating posterior probabilities [21].

Let the intention space of an agent be I = {I₁, I₂, …, I_m}, where each I_j represents a possible intention; the observed behavioral sequence is o1: t = {o1, o2, …, ot}, where ok represents the behavior observed at time k. Then, the posterior probability of intention I_j is:

$$P\left(I_j\mid o_{1:t}\right)={\displaystyle \frac{P\left(o_{1:t}\mid I_j\right)P\left(I_j\right)}{{\displaystyle \sum _{k=1}^{m}P}\left(o_{1:t}\mid I_k\right)P\left(I_k\right)}}$$

(7)

where the terms are defined as follows: P(I_j) is the prior probability of intention Ij, reflecting the initial belief about that intention before observing any behaviors; P(o1:t∣Ij) is the likelihood probability of observing behavioral sequence o1:to1:t under intention Ij, representing the possibility that an agent with intention Ij produces the observed sequence; the denominator is a normalization constant ensuring that the sum of posterior probabilities for all intentions equals 1 [22].

For temporal behavioral sequences, the likelihood probability can be further decomposed as:

$$P\left(o_{1:t}|I_j\right)={\displaystyle \prod _{k=1}^{t}P}\left(o_k|o_{1:k-1},I_j\right)$$

(8)

where o_1:k−1 represents the observation sequence before time k.

2.2.2. Recursive Reasoning Model

In multi-agent environments, agents not only need to reason about the intentions of other agents but also need to consider other agents’ reasoning about their own intentions, forming a multi-level recursive reasoning structure. The depth of recursive reasoning reflects the rationality level and computational capacity of agents. This recursive reasoning process can be described using Theory of Mind, i.e., agents possess the ability to understand and predict the mental states of other agents [23].

Define the strategy of a k-th order recursive reasoning agent as πi(k), where k represents the depth of reasoning hierarchy:

0th-order agent: Does not consider the existence of other agents, directly optimizing its own expected reward.

$${\pi }_i^{\left(0\right)}=\arg \underset{{\pi }_i}{\max }\mathbb{E}\left[R_i\left({\pi }_i,s\right)\right]$$

(9)

k-th order agent: assumes other agents are (k − 1)-th order and optimizes strategy accordingly.

$${\pi }_i^{\left(k\right)}=\arg \underset{{\pi }_i}{\max }\mathbb{E}\left[R_i\left({\pi }_i,{\pi }_{-i}^{\left(k-1\right)},s\right)\right]$$

(10)

where R_i represents agent i’s expected reward function, and ${\mathsf{\pi }}_i^{(k-1)}$ represents the (k − 1)-th order strategies of other agents.

The convergence of recursive reasoning can be analyzed through fixed-point theorems. Under certain conditions, the recursive reasoning sequence converges to a stable strategy:

$$\underset{k\to \infty }{\lim }{\pi }_i^{\left(k\right)}={\pi }_i^{*}$$

(11)

where ${\pi }_i^{*}$ is the fixed point of recursive reasoning.

2.3. Cooperative Game Theory

Cooperative game theory studies how agents form coalitions and allocate cooperative rewards. Unlike non-cooperative games [24], cooperative games allow binding agreements between agents to jointly pursue higher overall rewards. In multi-agent systems, cooperative game theory provides theoretical foundations for coalition formation, stability analysis, and reward allocation among agents [25].

2.3.1. Coalition Game Fundamentals

A coalition game can be represented as a tuple ⟨N, v⟩, where N is the agent set and v is the characteristic function representing the total reward that coalition S can obtain. The characteristic function must satisfy the boundary condition [26]:

$$v\left(\varnothing \right)=0$$

(12)

i.e., the empty coalition has zero reward.

Superadditivity is an important property of cooperative games, indicating that the reward of merging two disjoint coalitions is no less than the sum of their individual rewards:

$$v\left(S\cup T\right)\ge v\left(S\right)+v\left(T\right),\hspace{1em}\forall S,T\in \mathcal{N},S\cap T=\varnothing$$

(13)

This property ensures the incentive for cooperation, i.e., agents have motivation to form larger coalitions.

2.3.2. Shapley Value

The Shapley value is a fair reward allocation scheme that distributes rewards based on each agent’s marginal contribution to all possible coalitions [27]. The core idea of the Shapley value is to consider the average marginal contribution of an agent across all possible coalition formation orders.

The Shapley value of agent i is defined as:

$${\varphi }_i\left(v\right)={\displaystyle \sum _{S\subseteq \mathcal{N}\setminus \left\{i\right\}}\frac{\left|S\right|!\left(\left|\mathcal{N}\right|-\left|S\right|-1\right)!}{\left|\mathcal{N}\right|!}}\left[v\left(S\cup \left\{i\right\}\right)-v\left(S\right)\right]$$

(14)

where the terms are defined as follows: |S| represents the size of coalition S; |N| represents the total number of agents; v(S∪{i}) − v(S) represents the marginal contribution brought by agent i joining coalition S; |S|!(|N| − |S| − 1)! is the weight coefficient, representing the proportion of arrangements where agent i joins after exactly |S| agents among all !|N|! possible agent arrangements.

2.3.3. Core Solution

The core solution is a set of stable reward allocation schemes that ensures no sub-coalition has an incentive to deviate from the current allocation scheme [28]. The core solution is defined as:

$$\mathrm{Core}\left(v\right)=\left\{x\in {ℝ}^n:{\displaystyle \sum _{i\in N}{x}_i}=v\left(N\right),{\displaystyle \sum _{i\in S}{x}_i}\ge v\left(S\right),\forall S\subseteq N\right\}$$

(15)

where x = (x1, x2, …, xn) represents the reward allocation vector, and xi is the reward obtained by agent i. The first constraint condition ensures allocation efficiency, i.e., the sum of all agents’ rewards equals the total reward of the grand coalition; the second constraint condition ensures allocation stability, i.e., the total reward obtained by members of any sub-coalition is no less than the reward that sub-coalition could obtain by acting independently.

3. Intention-Strategy Optimization Based Multi-Agent Cooperative Group Game Model

3.1. Overall Framework Design

The Intention-Strategy Optimization based Multi-Agent Cooperative Group Game Model (ISO-MAGCG) proposed in this paper adopts a bi-level optimization architecture that organically combines intention recognition with strategy optimization. The core idea of this framework is to capture complex interaction relationships and intention dependencies among agents through an upper-level intention recognition network, while the lower-level strategy optimization module makes collaborative decisions based on identified intention information. The two levels achieve dynamic adaptation and co-evolution through feedback mechanisms.

The overall framework comprises three core components: the Group Attention-based Intention Recognition Network (GAIN), the Intention-Strategy Optimization Framework (ISO Framework), and the Adaptive Group Game Equilibrium Algorithm (AGEA). The system inputs are environmental states and agent observation sequences, with outputs being optimized cooperative strategies. The overall architecture of the framework is shown in Figure 2.

Figure 2. ISO-MAGCG Overall Framework Architecture.

3.2. Group Attention-Based Intention Recognition Network

3.2.1. Dynamic Graph Construction

In multi-agent environments, the interaction relationships among agents are dynamically changing, and traditional static graph structures cannot effectively capture such time-varying characteristics. The GAIN network proposed in this paper first constructs dynamic graphs to represent interaction relationships among agents.

Let at time t, the agent set be N = {1, 2, …, n}, and each agent’s observation be $o_i^t\subset R^{d_o}$, where $d_o$ is the observation dimension. The dynamic graph construction process is as follows:

The node feature matrix is defined as:

$$H^t={\left[h_1^t,h_2^t,\dots ,h_n^t\right]}^T$$

(16)

where $h_i^t$ is the node feature of agent i at time t, obtained through an observation encoder:

$$h_i^t=GRU\left(o_i^t,h_i^{t-1}\right)$$

(17)

Here, GRU is a GRU-based encoder used to capture temporal information.

The edge weight matrix is computed through the attention mechanism:

$$e_{ij}^t=a^T\tan \mathrm{h}\left(W[h_i^t\mid h_j^t]\right)$$

(18)

where W is the weight matrix, a is the attention vector, and | denotes vector concatenation operation.

3.2.2. Multi-Head Attention Mechanism

To capture multiple types of interaction relationships among agents, the GAIN network employs a multi-head attention mechanism. The computation process for the k-th attention head is:

$${\alpha }_{ij}^{\left(k\right)}={\displaystyle \frac{\exp \left(\mathrm{LeakyReLU}\left(a_k^T\left[W_k{h}_i\mid W_k{h}_j\right]\right)\right)}{{\displaystyle \sum _{l\in {\mathcal{N}}_i}\exp }\left(\mathrm{LeakyReLU}\left(a_k^T\left[W_k{h}_i\mid W_k{h}_l\right]\right)\right)}}$$

(19)

where ${\alpha }_{ij}^{\left(k\right)}$ is the attention weight of the k-th head, Wk is the weight matrix of the k-th head, and LeakyReLU is the activation function.

The output of multi-head attention is obtained through concatenation and linear transformation:

$$h_i^{\prime }=W_O\left[{|}_{k=1}^K{\displaystyle \sum _{j\in {\mathcal{N}}_i}{\alpha }_{ij}^{\left(k\right)}}{W}_k{h}_j\right]$$

(20)

where K is the number of attention heads, and W_O are the output layer parameters.

3.2.3. Intention Prediction

Based on the updated node features, the intention distribution prediction for agent i is:

$$p_i^t=Softmax(W_{\mathrm{intent}}{h}_i^{\prime t}+b_{\mathrm{intent}})$$

(21)

where $p_i^t\in R^{\left|I\right|}$ is the intention probability distribution of agent i at time t, and |I| is the size of the intention space.

The current formulation assumes a discrete intention space I. For continuous intention representations, the prediction module can output Gaussian parameters:

$$\left({\mu }_i^t,{\sum }_i^t\right)=f_{\mathrm{intent}}\left(h_i^{t+1};{\theta }_{\mathrm{intent}}\right)$$

(22)

where ${\mu }_i^t\in R^{d_I}$ and ${\displaystyle \sum _i^t\in R^{d_I\times d_I}}$ parameterize a continuous intention distribution. For latent intention discovery without predefined categories, a Variational Autoencoder architecture can be employed:

$${\mathcal{L}}_{\mathrm{VAE}}={\mathbb{E}}_{q\left(z|o\right)}\left[\log p\left(o|z\right)\right]-\beta \cdot D_{\mathrm{KL}}\left(q\left(z|o\right)\parallel p\left(z\right)\right)$$

(23)

where z represents the latent intention. These extensions remain important directions for future research.

The complete GAIN network architecture diagram is shown in Figure 3.

Figure 3. GAIN Network Structure Diagram.

3.2.4. Intention Label Generation

To enable supervised training of the GAIN network, ground-truth intention labels are required for each agent at each timestep. This section describes our hybrid labeling approach combining rule-based heuristics with trajectory clustering.

(1)

Rule-Based Labeling for Structured Scenarios

For scenarios with well-defined objectives, intention labels are derived from observable goal-directed behaviors using domain-specific rules. The ground-truth intention is determined as:

$$I_i^{gt}\left(t\right)=\underset{I_k\in \mathcal{I}}{\arg \max };P\left(I_k\mid g_i,s_t,a_{i,t-H:t}\right)\mathrm{tag}20a$$

(24)

where g_i represents the assigned goal of agent i, s_t is the current environmental state, and $a_{i,t-H:t}$ denotes the action history over horizon H = 5.

Table 1 summarizes the intention space and labeling rules for each experimental scenario.

Table 1. Intention Space Definition and Labeling Methods.

Scenario	Intention Space	Size	Labeling Method
Multi-Agent Navigation	{Navigate, Avoid, Wait, Replan}	4	Rule-based: velocity direction toward target (Navigate), proximity deviation (Avoid), stationary > 3 steps (Wait), direction change > 90° (Replan)
Resource Collection	{Collect, Transport, Cooperate, Explore}	4	Rule-based + proximity: agents within d_coop = 3.0 moving toward same resource (Cooperate)
Defense Cooperation	{Intercept, Patrol, Support, Retreat, Hold}	5	DTW trajectory clustering + manual semantic labeling

For the navigation task, specific rules include:

• Navigate: Agent’s velocity vector points toward its target with cosine similarity > 0.8.
• Avoid: Agent deviates from shortest path while another agent is within d_safe = 2.0 units.
• Wait: Agent remains stationary (speed < 0.1) for more than 3 consecutive timesteps.
• Replan: Agent changes movement direction by more than 90° within 5 timesteps.

(2)

Trajectory Clustering for Complex Scenarios

For the defense cooperation scenario with continuous action spaces, we employed unsupervised trajectory clustering. Agent trajectories are segmented into windows of T_w = 20 timesteps, and pairwise distances are computed using Dynamic Time Warping (DTW):

$$DTW\left({\tau }^i,{\tau }^j\right)=\underset{\pi \in A}{\min }\sqrt{{\displaystyle \sum _{\left(k,l\right)\in \pi }\Vert a_k^i-a_l^j{\Vert }^2}}$$

(25)

Hierarchical agglomerative clustering with Ward linkage groups similar trajectories, and cluster centroids are manually labeled by domain experts.

(3)

Label Quality Assurance

To ensure labeling quality, we implemented a two-stage verification:

Automatic Consistency Check: Labels are validated against expected behavioral patterns.

Manual Sampling: 10% of labels (~5000 samples per scenario) are verified by two independent annotators.

Table 2 presents the label statistics and quality metrics.

Table 2. Intention Label Statistics and Quality Metrics.

Scenario	Total Labels	Distribution	Auto-Check Pass	Manual Accuracy	Agreement
Navigation	48,000	Navigate 42%, Avoid 28%, Wait 18%, Replan 12%	97.3%	96.8%	97.2%
Resource Collection	52,000	Collect 35%, Transport 25%, Cooperate 24%, Explore 16%	96.1%	95.4%	96.1%
Defense Cooperation	45,000	Intercept 22%, Patrol 28%, Support 20%, Retreat 15%, Hold 15%	94.8%	95.1%	95.7%

The achieved labeling accuracy exceeded 95% across all scenarios, providing reliable supervision for GAIN training.

3.3. Intention-Strategy Bi-Level Optimization Framework

3.3.1. Upper-Level Intention Optimization

The objective of upper-level intention optimization is to maximize the overall reward of group cooperation while maintaining the accuracy of intention recognition. The group intention consistency loss is defined as:

$${\mathcal{L}}_{\mathrm{intent}}={\displaystyle \sum _{i=1}^n\left[\mathrm{CE}\left(y_i,p_i\right)+\lambda \mathrm{KL}\left(p_i\mid \overline{p}\right)\right]}$$

(26)

where yi is the true intention label of agent i, pi is the predicted intention probability, CE is cross-entropy loss, KL is KL divergence, $\overline{P}$ is the average intention distribution, and λ is the regularization parameter.

The upper-level optimization problem can be formulated as: $P$

$${\theta }_{\mathrm{intent}}^{*}=\arg \underset{{\theta }_{\mathrm{intent}}}{\min }{\mathcal{L}}_{\mathrm{intent}}\left({\theta }_{\mathrm{intent}}\right)$$

(27)

where ${\theta }_{intent}^{*}$ is the parameters of the intention recognition network.

3.3.2. Lower-Level Strategy Optimization

Lower-level strategy optimization is based on identified intention information, employing an improved Actor-Critic algorithm. The policy network parameter update for agent i is:

$${\theta }_{{\pi }_i}\leftarrow {\theta }_{{\pi }_i}+{\alpha }_{\pi }{\nabla }_{{\theta }_{{\pi }_i}}{J}_{{\pi }_i}\left({\theta }_{{\pi }_i}\right)$$

(28)

where the policy objective function is defined as:

$$J_{{\pi }_i}\left({\theta }_{{\pi }_i}\right)={\mathbb{E}}_{s~{\rho }^{\pi },a~{\pi }_i}\left[Q_i\left(s,a\right)\mid p\right]$$

(29)

Here, p = [p1, p2, …, pn] is the intention distribution of all agents, and ρπ is the state-action distribution induced by the policy.

The value network update employs temporal difference learning:

$${\theta }_{Q_i}\leftarrow {\theta }_{Q_i}-{\alpha }_Q{\nabla }_{{\theta }_{Q_i}}{\mathcal{L}}_{Q_i}\left({\theta }_{Q_i}\right)$$

(30)

where the value loss function is:

$${\mathcal{L}}_{Q_i}\left({\theta }_{Q_i}\right)=\mathbb{E}[{(y_i^{\mathrm{target}}-Q_i\left(s,a\right))}^2]$$

(31)

The target value is defined as:

$$y_i^{\mathrm{target}}=r_i+\gamma {\mathbb{E}}_{a^{\prime }~\pi }[Q_i(s^{\prime },{\mathbf{a}}^{\prime })\mid {\mathbf{p}}^{\prime }]$$

(32)

3.3.3. Bi-Level Collaborative Optimization

The key to bi-level optimization lies in information transfer and collaborative updates between upper and lower levels. The bi-level objective function is defined as:

$${\mathcal{L}}_{\mathrm{total}}={\alpha }_1{\mathcal{L}}_{\mathrm{intent}}+{\alpha }_2{\displaystyle \sum _{i=1}^n{\mathcal{L}}_{Q_i}}$$

(33)

where α1 and α2 are balancing parameters.

The intent-policy two-layer optimization framework process is shown in Figure 4.

Figure 4. Intention-Strategy Bi-Level Optimization Framework Flowchart.

3.4. Adaptive Group Game Equilibrium Algorithm

3.4.1. Dynamic Weight Allocation Mechanism

To handle the heterogeneity and contribution differences of agents in large-scale groups, the AGEA algorithm employs a contribution-based dynamic weight allocation mechanism. The contribution of agent i is defined as:

$$c_i={\displaystyle \frac{V\left(N\right)-V\left(N\backslash \left\{i\right\}\right)}{V\left(N\right)+ϵ}}$$

(34)

where N\{i} represents the set of all agents except agent i, V(N) is the value of agent i under the current joint strategy, V(N\{i}) is the value after removing agent i, and ϵ is a small constant to prevent division by zero.

The contribution-based dynamic weight is computed as:

$$w_i={\displaystyle \frac{\exp \left(c_i/\tau \right)}{{\displaystyle \sum _{j=1}^n\exp }\left(c_j/\tau \right)}}$$

(35)

where τ is the temperature parameter controlling the concentration degree of weight allocation.

3.4.2. Distributed Nash Equilibrium Solving

The AGEA algorithm employs a distributed approach to solve the Nash equilibrium of group games. Each agent maintains a local strategy and estimates of other agents’ strategies.

The best response strategy update for agent i is:

$${\pi }_i^{t+1}=\arg \underset{{\pi }_i}{\max }\mathbb{E}[R_i({\pi }_i,{\widehat{\pi }}_{-i}^t)\mid p_i]$$

(36)

The strategy estimate update employs the exponential moving average:

$${\widehat{\pi }}_i^{t+1}=\left(1-\beta \right){\widehat{\pi }}_i^t+\beta {\pi }_i^t$$

(37)

where β is the update rate.

3.4.3. Convergence Guarantee

To ensure algorithm convergence, a potential function is defined:

$$\mathsf{\Phi }\left(\pi \right)={\displaystyle \sum _{i=1}^n{w}_i}{V}_i\left(\pi \right)-{\displaystyle \frac{\mu }{2}}{\displaystyle \sum _{i=1}^n|}{\pi }_i-{\pi }_i^{*}{|}^2$$

(38)

where μ is the regularization parameter and ${\pi }_i^{*}$ is the optimal strategy of agent i.

Proposition 1 (Convergence of AGEA).

Under the following assumptions, AGEA converges to a neighborhood of Nash equilibrium: (A1) The value function V_i is L-Lipschitz continuous;

(A2) Learning rates satisfy α_π < 1/(2L) and β > 0;

(A3) The potential function Φ is bounded below.

Proof Sketch. Define the Lyapunov function:

$$\mathcal{V}\left(t\right)=\mathsf{\Phi }\left({\pi }^{*}\right)-\mathsf{\Phi }\left({\pi }^t\right)+{\displaystyle \frac{\eta }{2}}{\displaystyle \sum _{i=1}^n\Vert {\widehat{\pi }}_{-i}^t-{\pi }_{-i}^t{\Vert }^2}$$

(39)

where ${\pi }^{*}$ is a Nash equilibrium and $\eta >0$ is a weighting constant.

Step 1 (Strategy Update Descent).  By the best response property:

$$\mathsf{\Phi }\left({\pi }^{t+1}\right)-\mathsf{\Phi }\left({\pi }^t\right)\ge {\alpha }_{\pi }{\displaystyle \sum _{i=1}^n{\nabla }_{{\pi }_i}}\mathsf{\Phi }\cdot \left(B{R}_i\left({\widehat{\pi }}_{-i}^t\right)-{\pi }_i^t\right)-{\displaystyle \frac{{\alpha }_{\pi }^{2}L}{2}}{\displaystyle \sum _i|}|B{R}_i-{\pi }_i^t|{|}^2$$

(40)

Step 2 (Estimation Error Decay).  From the exponential moving average update (Equation (32)), the estimation error decays exponentially:

$$||{\widehat{\pi }}_{-i}^{t+1}-{\pi }_{-i}^{t+1}||\le \left(1-\beta \right)||{\widehat{\pi }}_{-i}^t-{\pi }_{-i}^t||+O\left({\alpha }_{\pi }\right)$$

(41)

Step 3 (Combined Descent).  Choosing η sufficiently small yields:

$$\mathcal{V}\left(t+1\right)-\mathcal{V}\left(t\right)\le -c\cdot \mathcal{V}\left(t\right)+ϵ$$

(42)

for constants c > 0 and smallϵ dependent on learning rates, establishing convergence to an ϵ-neighborhood of equilibrium.

3.5. Complete Algorithm Flow

The pseudocode for the ISO-MAGCG algorithm is shown in Algorithm 1.

Algorithm 1 ISO-MAGCG

Input: Environment E, Agent set N, Episodes T, Learning rates απ, αQ, αintent Output: Optimized cooperative strategies π* 1    Initialize GAIN, Actor, Critic networks and replay buffer D 2    for episode = 1 to T do 3        Reset environment, get initial state s0 4        for step t = 0 to max_steps do 5            // Upper layer: Intent recognition 6            Construct dynamic graph Gt and compute node features Ht 7            Apply multi-head attention: Ht+1 = MultiHeadAttention (Ht) 8            Predict intent distributions: ${\mathrm{p}}_{\mathrm{i}}^{\mathrm{t}}$ = Softmax(Wintent ${\mathrm{h}}_{\mathrm{i}}^{\mathrm{t}+1}$) 9             10          // Lower layer: Strategy optimization 11          for agent i = 1 to n do 12              Select action: ${\mathrm{a}}_i^t$ = πi(st, It) + εt 13          end for 14          Execute actions, observe rewards and next state 15          Store experience (st, at, rt, st+1, It) to buffer D 16 17          if |D| > batch_size then 18              Sample batch from D 19              // Update networks 20              Update Critic: ${\mathsf{\theta }}_{\mathrm{i}}^{\mathrm{Q}}$← ${\mathsf{\theta }}_{\mathrm{i}}^{\mathrm{Q}}$-αQ▽LQ 21              Update Actor: ${\mathsf{\theta }}_{\mathsf{\pi }}^{\mathrm{i}}$ ← ${\mathsf{\theta }}_{\mathsf{\pi }}^{\mathrm{i}}$ + απ ∇Ji 22              Update Intent: θintent ← θintent - αintent ∇Lintent 23               24              // AGEA equilibrium solving 25              Compute contributions: ${\mathrm{c}}_{\mathrm{i}}^{\mathrm{t}}$ = ContributionCalculation() 26              Update weights: ${\mathrm{w}}_{\mathrm{i}}^{\mathrm{t}}$ = exp(β ${\mathrm{c}}_{\mathrm{i}}^{\mathrm{t}}$)/Σj exp(β ${\mathrm{c}}_{\mathrm{j}}^{\mathrm{t}}$) 27              Update strategy estimations and compute best responses 28              Soft update target networks 29          end if 30      end for 31      Check convergence, break if converged 32 end for 33 return π* = {${\pi }_1^{*}$, ${\pi }_2^{*}$, …, ${\pi }_{\mathrm{n}}^{*}$}

3.6. Algorithm Complexity Analysis

3.6.1. Time Complexity

The time complexity of the GAIN network is primarily determined by the multi-head attention mechanism. For n agents and K attention heads, the complexity of a single forward pass is:

$$O\left(K\cdot n^2\cdot d+K\cdot n\cdot d^2\right)$$

(43)

where the first term comes from attention weight computation and the second term from feature transformation.

In the ISO framework, the upper-level intention optimization has complexity O (n·dintent), while the lower-level strategy optimization employs Actor-Critic methods with complexity O (n·(ds·da + da·dπ)), where dsds, dada, and dπ represent the dimensions of state, action, and policy networks, respectively.

The AGEA algorithm’s complexity mainly comes from contribution computation and weight updates, which is O (n2).

The overall time complexity is:

$$O\left(K\cdot n^2\cdot d+n\cdot d_{\mathrm{int}}+n\cdot d_s\cdot d_a+n^2\right)$$

(44)

3.6.2. Space Complexity

The space complexity consists of the following components:

• GAIN network parameters: O (K·d2 + n·d);
• Actor-Critic network parameters: O (n·(ds·da + da·dπ));
• Dynamic graph storage: O (n2);
• Experience replay buffer: O(B·(ds + da + dQ)).

where B is the buffer size and dQ is the value network dimension.

The total space complexity is:

$$O\left(K\cdot d^2+n\cdot d+n\cdot d_s\cdot d_a+n^2+B\cdot d_s\right)$$

(45)

Through the above analysis, it can be seen that the algorithm’s complexity grows quadratically with the number of agents, which may become a bottleneck in large-scale multi-agent systems. To address this, hierarchical optimization and approximation algorithms can be employed to reduce computational complexity.

4. Experimental Design and Results Analysis

4.1. Experimental Environment Setup

4.1.1. Experimental Platform Configuration

The experiments in this paper were conducted under the hardware and software environment shown in Table 3. The hardware configuration adopts a high-performance computing platform to ensure the computational requirements of large-scale multi-agent simulation; the software environment is built on mainstream deep learning frameworks to guarantee experimental reproducibility and standardization.

Table 3. Experimental Platform Configuration.

Configuration Type	Specific Configuration
Hardware Configuration
CPU	Intel Core i9-12900K
GPU	NVIDIA RTX 4090 (24 GB VRAM)
Memory	64 GB DDR4
Software Environment
Operating System	Ubuntu 20.04 LTS
Programming Language	Python 3.9
Deep Learning Framework	PyTorch 1.12.0
GPU Acceleration	CUDA 11.6
Simulation Environment	OpenAI Gym + PettingZoo

4.1.2. Experimental Scenario Design

To comprehensively validate the effectiveness of the ISO-MAGCG model, this paper designed three multi-agent cooperation scenarios with different complexity levels, covering various situations from discrete to continuous action spaces and from static to dynamic environments. Table 4 details the configuration parameters for each scenario.

Table 4. Experimental Scenario Configuration.

Scenario	Environment Scale	Agent Number	State Space	Action Space	Reward Design
Multi-Agent Navigation	20 × 20 grid	n ∈ {4, 6, 8, 10}	R4n	{0, 1, 2, 3, 4}	Reach target +10, collision −5, time −0.1
Resource Collection	15 × 15 grid	n ∈ {6, 8, 10, 12}	R6n+25	{0, 1, 2, 3, 4, 5}	Collection +5, cooperation +2, efficiency reward
Defense Cooperation	25 × 25 continuous	n ∈ {8, 10, 12, 15}	R8n+4m	[−1, 1]2	Defense +8, cooperation +3, area control

The multi-agent navigation task tests agents’ path planning and conflict avoidance capabilities under spatial constraints; the resource collection task evaluates agents’ cooperation abilities in resource allocation and task coordination; and the defense cooperation task verifies agents’ real-time collaboration and strategy adaptation capabilities in dynamic adversarial environments.

4.1.3. Network Architecture Parameters

The ISO-MAGCG model contains multiple neural network modules, with each module’s architecture carefully optimized to adapt to the special requirements of multi-agent learning. Table 5 provides detailed parameter configurations.

Table 5. Network Architecture Parameter Settings.

Network Module	Parameter Configuration	Value
GAIN Network	Hidden layer dimension	128
	Number of attention heads	4
	Number of network layers	3
Actor Network	Hidden layer dimensions	[256, 128, 64]
	Activation function	ReLU
	Output layer activation	Tanh
Critic Network	Hidden layer dimensions	[256, 128, 64]
	Activation function	ReLU
Training Parameters	Actor learning rate	3 × 10−4
	Critic learning rate	1 × 10−3
	Intent learning rate	5 × 10−4
	Discount factor	0.99
	Soft update coefficient	0.005
	Batch size	256

The GAIN network adopts moderate hidden layer dimensions to balance expressive capability and computational efficiency; the Actor and Critic networks employ a funnel-shaped structure with gradually decreasing layers; the learning rate settings were determined through extensive preliminary experiments to find optimal values.

4.2. Baseline Method Comparison

4.2.1. Baseline Algorithm Selection

To objectively evaluate the performance of the ISO-MAGCG model, this paper selected five representative algorithms from the current multi-agent reinforcement learning field as baselines. These algorithms cover different technical approaches and theoretical foundations, comprehensively reflecting the development level of existing technologies. Table 6 details the core characteristics and main parameter configurations of each baseline algorithm.

Table 6. Baseline Algorithm Configuration.

Algorithm	Type	Core Features	Main Parameters	Year
QMIX	Value decomposition	Monotonicity assumption, centralized training distributed execution	Mixing network [64, 32], learning rate 5 × 10−4	2018
MADDPG	Policy gradient	Centralized Critic, continuous actions	Actor/Critic [400, 300], α = 1 × 10−2	2017
COMA	Actor-Critic	Counterfactual baseline, credit assignment	GRU hidden 64, entropy coefficient 1 × 10−3	2018
QTRAN	Value decomposition	Relaxed monotonicity constraint	Transform network [64, 64], λ = 5 × 10−3	2019
MAPPO	Policy optimization	Parameter sharing, centralized value	Network [64, 64], ε = 0.2, λ = 0.95	2021
MAT	Transformer-based	Multi-agent transformer, sequential modeling	d_model = 128, heads = 4, layers = 2	2022
ROMA	Role-based	Emergent role discovery, role encoder	Role dim = 64, n_roles = 4, α = 5 × 10−4	2020

QMIX is a typical value function decomposition method, realizing centralized training and distributed execution via monotonicity assumptions; MADDPG adopts centralized critics to tackle continuous action space problems; COMA addresses credit assignment issues through counterfactual baselines; QTRAN relaxes QMIX’s monotonicity constraints; MAPPO extends Proximal Policy Optimization to multi-agent scenarios. For comprehensive comparison with recent advances, two state-of-the-art methods (MAT and ROMA) were additionally included. MAT reformulates multi-agent reinforcement learning as a sequence modeling task, leveraging a transformer with cross-agent attention to capture coordination patterns, and we adopted its official encoder–decoder implementation (d_model = 128, 4 attention heads, 2 layers). ROMA introduces emergent role discovery, enabling agents to cluster into functional roles via a role encoder, configured with 4 latent roles and 64-dimensional role embeddings following the original paper’s cooperative task recommendations. These two methods represent distinct multi-agent coordination paradigms: MAT implicitly models agent interactions via the transformer’s attention mechanism, while ROMA explicitly discovers agent roles to facilitate labor division. Both differ fundamentally from ISO-MAGCG’s explicit intention modeling: MAT learns implicit coordination patterns without interpretable intermediate representations, and ROMA focuses on role differentiation rather than intention recognition. In contrast, our GAIN network specifically models agent intentions, which is complementary yet distinct from ROMA’s role modeling.

4.2.2. Evaluation Metrics

This paper established a multi-dimensional evaluation metric system to comprehensively assess algorithm performance from perspectives of task completion effectiveness, learning efficiency, and cooperation quality. The evaluation metrics are defined as follows:

Task Success Rate (TSR):

$$\mathrm{TSR}={\displaystyle \frac{N_{\mathrm{success}}}{N_{\mathrm{total}}}}\times 100\%$$

(46)

This metric represents the proportion of episodes where agents successfully complete tasks, serving as the most intuitive performance indicator reflecting the algorithm’s task completion capability.

Average Cumulative Reward (ACR):

$$\mathrm{ACR}={\displaystyle \frac{1}{T}}{\displaystyle \sum _{t=1}^T{\displaystyle \sum _{i=1}^n}{R}_i^t}$$

(47)

where T is the total number of episodes, n is the number of agents, and Rt i is the reward obtained by agent i in episode t. This metric comprehensively reflects the overall performance of the agent group.

Cooperation Efficiency (CE):

$$\mathrm{CE}={\displaystyle \frac{R_{\mathrm{coop}}-R_{\mathrm{indep}}}{R_{\mathrm{indep}}}}\times 100\%$$

(48)

where Rcoop is the average reward under cooperative strategy and Rindep is the average reward under independent action. This metric measures the performance improvement of inter-agent cooperation relative to independent action.

Convergence Speed (CS):

$$\mathrm{CS}=\arg \underset{t}{\min }\left\{R_t^i\ge 0.9\times R_{\mathrm{final}}^i\right\}$$

(49)

where Rt is the average reward at episode t and Rfinal is the final stable performance. This metric reflects the learning efficiency, measured as the number of episodes required to reach 90% of final performance.

Task success rate is the most intuitive performance indicator, reflecting the algorithm’s ability to complete tasks; average cumulative reward comprehensively considers task completion quality, efficiency, and cooperation degree; cooperation efficiency measures the performance improvement of inter-agent cooperation relative to independent action; convergence speed reflects learning efficiency.

4.3. Comparative Experimental Results

4.3.1. Overall Performance Comparison

Table 7 shows the comprehensive performance comparison results of each algorithm in three experimental scenarios. The data are based on independent experiments of 1000 episodes for each scenario, with each algorithm running 5 times and taking the average, with a 95% confidence interval.

Table 7. Algorithm Overall Performance Comparison.

Algorithm	Navigation Task			Resource Collection			Defense Cooperation
	TSR (%)	ACR	CE (%)	TSR (%)	ACR	CE (%)	TSR (%)	ACR	CE (%)
QMIX	72.3 ± 2.1	145.2 ± 8.3	23.4 ± 3.2	68.5 ± 2.8	89.7 ± 5.1	18.7 ± 2.9	65.8 ± 3.1	112.4 ± 6.7	21.2 ± 3.5
MADDPG	75.8 ± 1.9	152.6 ± 7.1	28.1 ± 2.8	71.2 ± 2.3	95.3 ± 4.8	22.4 ± 2.1	69.4 ± 2.7	118.9 ± 5.9	25.3 ± 2.8
COMA	78.1 ± 2.3	158.9 ± 6.8	31.7 ± 3.1	73.6 ± 2.6	98.1 ± 4.2	25.8 ± 2.7	71.7 ± 2.9	125.3 ± 6.2	28.6 ± 3.2
QTRAN	76.4 ± 2.0	149.8 ± 7.5	26.9 ± 2.9	70.9 ± 2.4	92.6 ± 4.9	20.3 ± 2.5	68.2 ± 3.0	115.7 ± 6.4	23.1 ± 2.9
MAPPO	81.2 ± 1.7	165.4 ± 5.9	35.2 ± 2.4	76.8 ± 2.1	103.2 ± 3.8	29.6 ± 2.3	74.5 ± 2.5	131.8 ± 5.1	32.4 ± 2.6
MAT	83.5 ± 1.6	172.8 ± 5.4	38.6 ± 2.2	79.1 ± 1.9	108.5 ± 3.5	33.2 ± 2.1	76.8 ± 2.3	138.4 ± 4.8	35.7 ± 2.4
ROMA	82.1 ± 1.8	168.3 ± 5.7	36.4 ± 2.5	77.5 ± 2.0	105.1 ± 3.9	31.1 ± 2.4	75.2 ± 2.6	134.2 ± 5.3	33.8 ± 2.7
ISO-MAGCG	89.6 ± 1.4 *	187.3 ± 4.2 *	47.8 ± 1.9 *	84.2 ± 1.8 *	125.6 ± 3.1 *	42.3 ± 2.0 *	82.1 ± 2.0 *	156.2 ± 4.6 *	45.1 ± 2.2 *

Note: * indicates p < 0.001 compared to all baseline algorithms (Wilcoxon signed-rank test). Values represent mean ± standard deviation over 10 independent runs. Bold indicates best.

Statistical Significance Testing

The Kruskal–Wallis test confirmed statistically significant differences across all algorithms for every performance metric (H > 28.7, p < 0.001). Post hoc pairwise comparisons via the Wilcoxon signed-rank test with Bonferroni correction further verified that ISO-MAGCG outperformed all baseline algorithms significantly (p < 0.001) across all experimental scenarios. Cohen’s d effect sizes ranged from 1.4 to 2.9, indicating large practical significance. The 95% confidence intervals for ISO-MAGCG’s performance gains over the top baseline (MAPPO) were [6.8%, 10.2%] for TSR, [13.1%, 18.5%] for ACR, and [10.2%, 15.8%] for CE.

Comparison with Transformer-based and Role-based Methods

Among baselines, MAT attained the strongest performance (TSR: 83.5%, 79.1%, 76.8% across three scenarios), validating the efficacy of transformer-driven cross-agent attention for multi-agent coordination. Nevertheless, ISO-MAGCG outperformed MAT by 6.1%, 5.1%, and 5.3% in TSR across the three scenarios, respectively. This performance disparity stems from two core limitations of MAT:

• MAT’s attention mechanism operates on the observation-action space without explicit intention modeling, yielding implicit coordination patterns but lacking the interpretability and goal-directedness afforded by intention recognition. In contrast, ISO-MAGCG’s GAIN module explicitly predicts agent intentions to guide strategy optimization.
• ISO-MAGCG’s bi-level optimization framework enables co-evolution of intention recognition and strategy learning, whereas MAT frames coordination as a single-level sequence prediction task; the mutual feedback between ISO-MAGCG’s intention and strategy modules fosters more coherent cooperative behaviors.

ROMA delivered competitive results (TSR: 82.1%, 77.5%, 75.2%), particularly in role-differentiation-centric scenarios, but its capability-based role clustering mechanism does not support goal inference. In the defense cooperation scenario—where agents must dynamically switch between offensive and defensive behaviors based on situational intentions—ROMA’s static role assignments became restrictive, and ISO-MAGCG’s dynamic intention recognition yielded a 6.9% higher TSR.

Notably, the cooperation efficiency (CE) metric highlighted ISO-MAGCG’s most pronounced advantage over transformer and role-based methods: it achieved CE improvements of 9.2%, 9.1%, and 9.4% over MAT across the three scenarios, demonstrating that explicit intention modeling enables more effective cooperation than implicit attention-driven coordination.

4.3.2. Convergence Analysis

Convergence analysis was conducted in the multi-agent navigation task with 8 agents in a 20 × 20 grid environment. We measured the number of training episodes required for each algorithm to achieve 90% of its final performance (Table 8). Each algorithm was independently executed 5 times, and results are reported as mean ± standard deviation, and the convergence curve is shown in Figure 5.

Table 8. Algorithm Convergence Performance Comparison.

Algorithm	Convergence Episodes	Final Performance (%)	Training Stability	Computation Time (min)	p-Value *
QMIX	1850 ± 120	68.9 ± 2.8	0.78 ± 0.05	8.7 ± 0.6	<0.001
MADDPG	1720 ± 95	72.1 ± 2.3	0.82 ± 0.04	12.4 ± 0.8	<0.001
COMA	1680 ± 110	74.5 ± 2.6	0.85 ± 0.03	15.6 ± 1.1	<0.001
QTRAN	1780 ± 105	71.8 ± 2.4	0.80 ± 0.04	9.8 ± 0.7	<0.001
MAPPO	1520 ± 85	77.5 ± 2.1	0.89 ± 0.02	10.3 ± 0.5	<0.001
MAT	1380 ± 75	80.2 ± 1.9	0.91 ± 0.02	14.8 ± 0.6	<0.001
ROMA	1450 ± 80	78.6 ± 2.0	0.88 ± 0.03	11.2 ± 0.5	<0.001
ISO-MAGCG	1180 ± 65	85.3 ± 1.8	0.94 ± 0.02	13.2 ± 0.4	-

Note: * p-values from Mann–Whitney U test comparing each algorithm to ISO-MAGCG. Training stability measured as coefficient of variation of performance over final 200 episodes.

Figure 5. Convergence Curve Analysis.

Convergence Statistical Analysis: The Mann–Whitney U test confirmed that ISO-MAGCG achieved significantly faster convergence than all baseline algorithms (p < 0.001). The algorithm demonstrated 22.4% faster convergence compared to the best baseline (MAPPO) with 95% confidence interval [18.7%, 26.1%]. Training stability analysis using coefficient of variation showed ISO-MAGCG maintained the most consistent performance (CV = 0.021) compared to baselines (CV range: 0.048–0.064).

4.4. Ablation Study

4.4.1. Component Effectiveness Validation

Ablation experiments were performed across three standardized test scenarios: Navigation task: 8 agents in a 20 × 20 grid environment; Resource collection: 6 agents in a 15 × 15 environment with 10 randomly distributed resource points; Defense collaboration: 10 agents defending against 5 invasion targets

Each variant was trained for 2000 episodes in each scenario, with performance evaluated as the average task success rate over the final 100 episodes.The ablation test results are shown in Table 9.

Table 9. Ablation Study Results.

Model.	Navigation	Resource Collection	Defense Cooperation	Average	Decrease
Complete	89.6 ± 1.4	84.2 ± 1.8	82.1 ± 2.0	85.3%	-
w/o GAIN	78.4 ± 2.3	72.6 ± 2.7	69.8 ± 3.2	73.6%	−11.7%
w/o Attention (GAIN w/MLP)	82.1 ± 2.0	76.8 ± 2.3	74.2 ± 2.7	77.7%	−7.6%
w/o ISO	81.2 ± 2.1	76.1 ± 2.4	73.5 ± 2.8	76.9%	−8.4%
w/o AGEA	83.7 ± 1.9	78.9 ± 2.2	75.2 ± 2.6	79.3%	−6.0%
w/o Multi-Head (K = 1)	85.1 ± 1.8	80.3 ± 2.0	77.6 ± 2.4	81.0%	−4.3%

The ablation study revealed each component’s contribution:

GAIN Network (−11.7%): Removing the entire GAIN network caused the largest performance drop, confirming intention recognition’s critical role.

Attention Mechanism (−7.6%): The variant “w/o Attention (GAIN w/MLP)” replaces graph attention layers with standard MLP while preserving the GRU encoder and intention prediction head. This isolates the attention mechanism’s contribution: 7.6% of performance stemmed from attention, while the remaining 4.1% (11.7–7.6%) came from other GAIN components (temporal encoding, dynamic graph construction).

ISO Framework (−8.4%): Removing bi-level optimization and training intention recognition independently caused significant degradation, demonstrating the importance of co-evolution.

AGEA (−6.0%): Replacing AGEA with uniform weight allocation reduced performance, confirming contribution-based dynamic weighting’s value.

Multi-Head Attention (−4.3%): Reducing from K = 4 to K = 1 attention head caused moderate loss, suggesting multiple heads capture diverse interaction patterns.

4.4.2. Hyperparameter Sensitivity Analysis

Hyperparameter sensitivity testing was conducted in the multi-agent navigation task (8 agents). With other parameters fixed at optimal values, each target parameter was varied individually. Each parameter configuration was trained for 1500 episodes, with performance measured as the average over the final 200 episodes.The results of the sensitivity analysis are shown in Table 10.

Table 10. Hyperparameter Sensitivity Analysis Results.

Hyperparameter	Test Range	Optimal Value	Performance Range	Sensitivity
Learning rate	[1 × 10−5, 1 × 10−2]	3 × 10−4	[76.2%, 89.6%]	Medium
Number of attention heads	[1, 8]	4	[81.3%, 89.6%]	Low
Weight parameter	[0.01, 1.0]	0.1	[82.7%, 89.6%]	Low
Batch size	[64, 512]	256	[85.1%, 89.6%]	Very low
Temperature	[0.1, 10]	1.0	[77.2%, 89.6%]	Medium-High
Discount factor	[0.9, 0.999]	0.99	[84.8%, 89.6%]	Low

The results show that ISO-MAGCG has good robustness to most hyperparameters, showing only medium sensitivity to learning rate, which provides good parameter selection guidance for practical applications.

Temperature Parameter Analysis. The temperature parameter τ in Equation (30) controls weight concentration in dynamic allocation. Figure 6 illustrates performance sensitivity across different τ values.

Figure 6. Temperature parameter (τ) sensitivity analysis. Shaded region indicates optimal range.

Three regimes are observable:

(1)

Under-differentiation (τ < 0.5): Low temperatures produce near-uniform weights regardless of contribution, failing to leverage agent heterogeneity. Performance degrades 8–12%.

(2)

Optimal range (0.5 ≤ τ ≤ 2.0): Balanced weights appropriately reward high contributors while maintaining collective optimization. Performance remains within 2% of optimal.

(3)

Over-concentration (τ > 5.0): Excessive focus on top contributors neglects supporting agents, causing coordination failures and 10–15% degradation.

Practical Guidance: Initialize τ = 1.0. For homogeneous teams, use lower values (τ ≈ 0.5); for heterogeneous teams with specialist roles, use higher values (τ ≈ 2.0).

4.5. Scalability Analysis

Agent Number Scalability

Scalability was evaluated in a standardized multi-agent navigation task, where environment size was dynamically adjusted according to agent count (n × n grid, where n = number of agents × 2.5). Each scale configuration was trained for 2000 episodes, with average task success rate and per-episode computation time measured over the final 100 episodes.The scalability analysis experiment of the number of agents is shown in Table 11.

Table 11. Agent Number Scalability Analysis.

Agent Number	ISO-MAGCG	MAPPO	QMIX	Performance Retention Rate	Computation Time (min)
4	92.1 ± 1.2	85.3 ± 1.8	78.6 ± 2.1	100%	2.1
8	89.6 ± 1.4	81.2 ± 1.7	72.3 ± 2.3	97.3%	8.5
12	85.2 ± 1.8	76.1 ± 2.1	65.8 ± 2.8	92.5%	19.1
16	82.6 ± 2.0	72.8 ± 2.4	61.4 ± 3.1	89.7%	35.2
20	79.4 ± 2.3	68.5 ± 2.7	56.9 ± 3.4	86.2%	58.7

The results indicate that ISO-MAGCG maintains good performance as the number of agents increases, with a performance retention rate of 86.2% even with 20 agents, demonstrating excellent scalability.

4.6. Experimental Results Discussion

4.6.1. Performance Advantage Analysis

Experimental results demonstrate that ISO-MAGCG significantly outperformed the baseline algorithms across all test scenarios, with core advantages including: (1) high intention recognition accuracy (94.3%) via dynamic graph modeling in the GAIN network, laying the foundation for effective cooperation; (2) pronounced collaborative optimization effects enabled by the bi-level framework, which fosters mutual promotion between intention recognition and strategy learning, addressing the separation issue in traditional methods; (3) accelerated convergence (30% faster than the top baseline) to enhance learning efficiency; and (4) superior training stability (stability index = 0.94, smaller standard deviation), ensuring algorithm reliability.

Further comparisons with MAT (transformer-based) and ROMA (role-based) revealed key design distinctions and advantages of ISO-MAGCG:

• Explicit vs. Implicit Coordination: Unlike MAT, which learns implicit coordination patterns from rewards via transformer attention, ISO-MAGCG’s explicit intention modeling enables faster convergence (via structured intermediate supervision) and interpretable predictions—critical for safety-critical applications (e.g., autonomous driving, medical robotics) requiring action explainability.
• Dynamic Intentions vs. Static Roles: ROMA’s capability-based static role clustering is limited in dynamic scenarios, where agents need to switch behavioral modes (e.g., Navigate → Avoid → Cooperate) within episodes. ISO-MAGCG’s dynamic intention recognition adapts to such temporal shifts, overcoming ROMA’s rigidity.
• Task-Specific Attention Design: While both MAT and GAIN utilize attention mechanisms, MAT focuses on observation–action pairs for sequence-based action prediction, whereas GAIN’s graph attention targets agent interaction graphs for intention inference—enabling more relevant feature extraction for cooperative tasks.

The consistent 5–7% performance advantage of ISO-MAGCG over MAT/ROMA across all metrics underscores that explicit intention modeling provides fundamental benefits that cannot be fully offset by advanced architectures (transformers) or alternative abstractions (roles).

4.6.2. Limitations and Improvement Directions

Despite achieving good results, ISO-MAGCG has the following limitations with proposed solutions:

• Computational Complexity and Large-Scale Scalability. The O(n²) complexity poses challenges for systems with >100 agents. Three approximation strategies are proposed:
  • Hierarchical Attention: Partition agents into k groups, compute intra-group attention O((n/k)²) and inter-group attention O(k²). Overall complexity reduces to O(n²/k + k²), approaching O(n) when k = √n. Preliminary tests with k = 4 groups on 16 agents showed only 3.1% degradation.
  • Sparse Attention: Sample edges based on spatial proximity within threshold d_connectd. For uniformly distributed agents, expected edges become $O\left(n·\rho ·{d^2}_{connect}\right)$, significantly reducing computation in sparse scenarios.
  • Low-Rank Factorization: Approximate the n × n attention matrix using rank-r decomposition, reducing complexity to O(nr).
• Intention Space Design. Current intention spaces are predefined. Future work will explore unsupervised intention discovery using VAE architectures.
• Environmental Adaptability. Adaptability in highly dynamic environments needs further validation through domain randomization and curriculum learning.
• Communication Overhead. Distributed design requires information exchange that may be affected in bandwidth-constrained scenarios. Federated learning approaches warrant investigation.
• Simulation-to-Real Transfer. All experiments were conducted in simulation. Deploying to physical systems (UAV swarms, robotic teams) faces challenges:

  (a)

  Observation Noise: Preliminary robustness tests with Gaussian noise (σ = 0.1) showed only 3.2% degradation, suggesting reasonable tolerance. Systematic evaluation under realistic sensor models is needed.

  (b)

  Communication Constraints: Limited bandwidth may necessitate fully decentralized variants with local intention estimation.

  (c)

  Actuation Dynamics: Continuous actions must respect physical limits not captured in kinematic simulations.

  (d)

  Domain Shift: Real-world intention distributions may differ, requiring online adaptation capabilities.

Potential Applications:

(a)

UAV swarm coordination for search and rescue;

(b)

Warehouse multi-robot picking systems;

(c)

Autonomous vehicle platoons for highway driving.

Systematic sim-to-real validation through hardware-in-the-loop testing represents a critical direction.

5. Conclusions and Future Work

5.1. Research Summary

This paper addresses key issues in multi-agent systems, such as the separation of intention recognition and strategy optimization, and low group cooperation efficiency, by proposing the Intention-Strategy Optimization based Multi-Agent Cooperative Group Game Model (ISO-MAGCG). The main work and contributions include:

• Theoretical contributions: Established an intention-strategy bi-level optimization theoretical framework, integrating intention recognition and strategy optimization into a unified framework for collaborative evolution; proposed group intention recognition theory based on graph attention, capturing complex intention dependencies through dynamic graph modeling and multi-head attention mechanisms; constructed an intention-aware multi-agent group game model, extending traditional game theory applications by using intention states as prior information for game strategies.
• Methodological contributions: Designed the Graph Attention Intention Network (GAIN) achieving 94.3% intention recognition accuracy; developed the Adaptive Group Evolution Algorithm (AGEA) combining global search and local optimization for efficient joint optimization; proposed a distributed training architecture supporting parallel training of large-scale multi-agent systems.
• Experimental validation: Constructed a multi-scenario evaluation system covering navigation, resource collection, and defense cooperation. Compared with five mainstream baseline algorithms, ISO-MAGCG significantly outperforms existing methods in task success rate, cumulative reward, cooperation efficiency, and other metrics, with average task success rate improvement of 8.4%, cooperation efficiency improvement of approximately 12%, and convergence speed improvement of 30%, demonstrating excellent performance advantages and scalability.

5.2. Future Research Directions

Future research will be extended in the following directions:

• Theoretical aspects: Research adaptive intention space learning methods to reduce dependence on expert knowledge, automatically discovering intention patterns based on unsupervised learning; extend convergence theory under non-stationary environments, considering the impact of environmental dynamics, observation noise, and other practical factors; develop multi-level intention modeling theory, establishing relationships between different abstraction levels such as short-term intentions and long-term goals; investigate continuous and latent intention space learning using VAE architectures while maintaining interpretability.
• Methodological aspects: Develop efficient approximation algorithms based on sampling, compression, and other techniques to reduce O(n2) computational complexity; research federated learning architectures to reduce communication overhead while protecting agent privacy; explore multi-modal intention recognition methods, integrating multiple information sources such as behavioral trajectories, linguistic communication, and visual information to improve recognition accuracy.
• Application aspects: Validate method effectiveness in real scenarios such as UAV swarms, robot cooperation, and intelligent transportation; research human–robot collaboration systems, considering human behavioral uncertainty and subjectivity; address challenges of system scale, heterogeneity, and dynamics for large-scale distributed systems like smart cities and IoT.

With the rapid development of artificial intelligence technology, multi-agent cooperation will play increasingly important roles in more domains. This research has laid a solid foundation for further development in this field, and future work will continue to advance multi-agent cooperation technology toward greater intelligence, efficiency, and practicality.