DEDMAC: Disentangling Environment and Decision Messages for Multi-Agent Communication

Abstract

In cooperative multi-agent reinforcement learning (MARL), communication can address the challenges of partial observability and environmental non-stationarity by conveying environmental features and decision intents, respectively. However, existing methods either focus on only one type of information—failing to tackle both challenges simultaneously—or conflate these signals, causing agents to confuse environmental context with decision intents. This paper introduces Disentangling Environment and Decision messages for Multi-Agent Communication (DEDMAC), a framework that explicitly separates these two information types into two distinct message streams and processes them independently. Specifically, environment messages are integrated into long-term memory to resolve partial observability, while decision messages provide instantaneous intent signals to mitigate non-stationarity and facilitate coordination. To prevent semantic confusion between the two message streams, we employ mutual information constraints to ensure semantic disentanglement. Furthermore, we design a mechanism that leverages global information to correct intent biases in decision messages resulting from limited local perspectives during generation. Evaluations across complex multi-agent benchmarks demonstrate that DEDMAC significantly outperforms state-of-the-art communication-based methods. These findings indicate that the explicit separation and specialized processing of environment and decision semantics are critical for achieving optimal performance in dynamic, collaborative multi-agent systems.

1. Introduction

Multi-agent reinforcement learning (MARL) has achieved significant advances in recent years and has found widespread applications in various real-world scenarios, such as the coordination of robot swarms [1], autonomous cars [2], traffic signal control [3], dynamic algorithm configuration [4], and Game AI [5]. Compared with single-agent systems, multi-agent systems face several new challenges, including non-stationarity of the environment and partial observability. Partial observability refers to the fact that each agent only has access to local observations, lacking a broader perspective of the global state of the environment [6]. Moreover, non-stationarity of the environment means that the environment perceived by each agent is influenced by other agents, so the changes in other agents’ policies lead to environment changes [7].

While the standard Centralized Training with Decentralized Execution (CTDE) paradigm effectively addresses environmental non-stationarity [8,9], its decentralized nature often limits performance in tasks requiring intricate coordination. Although non-communicative alternatives—such as agent modeling or behavior inference—can alleviate these constraints [10,11], they are frequently hampered by prohibitive training costs and high sample complexity [12]. Consequently, communication mechanisms have emerged as a more concise and efficient paradigm, enabling direct information exchange to achieve precise and robust multi-agent coordination [13,14].

In a standard multi-agent reinforcement learning (MARL) paradigm incorporating communication, agents typically synthesize messages from their internal hidden states or directly from raw local observations for transmission to peers. For example, MASIA [15] directly uses raw local observations as communication messages. MAIC [16] focuses on extracting action suggestions from the agent’s hidden state for communication. TarMAC [17] extracts information from the agent’s hidden state as messages but does not distinguish the content of the information. We categorize the exchanged information into two distinct classes: environment information and decision information. Environment information primarily consists of local state representations, which allow receivers to aggregate disparate observations into a comprehensive global view, thereby addressing the challenges of partial observability. In contrast, decision information conveys action tendencies or strategic intents, fostering inter-agent awareness and mitigating the detrimental effects of environmental non-stationarity.

Environment and decision information are the keys to effective coordination. Environment information helps resolve partial observability by providing a global perspective beyond local limits. Meanwhile, decision information mitigates environmental non-stationarity by aligning agent intents to reduce coordination uncertainty. Both are essential for high-performance multi-agent teamwork. However, contemporary communication mechanisms often exhibit a significant functional disconnect between environmental cognition and intentional alignment, which we contend severely hampers overall collaborative efficiency. Specifically, an exclusive focus on environmental perception without a corresponding intent-alignment mechanism leaves agents unable to predict peer strategies, rendering even rich global information insufficient for synchronized action. Conversely, protocols focused solely on decision-level alignment while ignoring environmental synchronization risk generating “misleading noise,” where proposed cooperation deviates from the objective physical reality. These two information types are complementary: environment information prevents blind spots, while decision intent drives policy alignment. To this end, we propose a MARL communication framework that simultaneously leverages environment information and decision information.

However, the intrinsic divergence in temporal characteristics and functional objectives between environment and decision information presents a challenge: the conflict between long-term scene reconstruction and instantaneous strategic alignment. We contend that environment information consists of objective, persistent features that must be integrated by agents over time to mitigate deep partial observability. In contrast, decision information, like action intents, is highly transient and uncertain, intended solely for immediate coordination. If agents erroneously treat ephemeral intent signals as stable environment features, significant noise is introduced into the agent’s long-term memory and exacerbates non-stationarity. To address this, we implement a differentiated processing architecture tailored to the distinct roles of environment information and decision information. Specifically, aggregated environment information is injected into a Gated Recurrent Unit (GRU) [18] to update the agent’s hidden state, ensuring a persistent and evolving global representation. Meanwhile, instantaneous decision information is transformed into a direct bias term for the current local Q-value, facilitating real-time policy refinement without contaminating the agent’s internal memory with transient noise. This dual-pathway mechanism allows the system to maintain robust background cognition while achieving precise strategic alignment.

However, another significant challenge is the temporal bias and consistency of asynchronous decision intents. In conventional communication, agents generate “pre-decisions” based on restricted local observations $o_i$ before receiving peer feedback, creating a temporal gap where the initial signaled intent frequently diverges from the final executed action. This inconsistency renders communication signals unreliable and fails to effectively mitigate environmental non-stationarity. We overcome this by introducing an explicit intent bias correction mechanism grounded in a “perception-first, intent-recalibration” logic. Specifically, the agent first utilizes aggregated global environment messages to achieve information completion, enabling it to identify and rectify biases in the original intent that stemmed from insufficient local data. This mechanism ensures high-fidelity, consistent coordination within a single communication round.

Furthermore, a fundamental challenge in multi-agent communication lies in the high coupling of internal representations, where the recurrent hidden state acts as a conflated repository for both environmental features and decision tendencies. Conventional methods typically broadcast a message vector derived from this hidden state, resulting in semantic ambiguity and information overload at the receiver, as they fail to distinguish between long-term physical modeling and transient strategic intent in this message vector. To resolve this, we implement an independent encoding and explicit disentangling strategy to ensure semantic purity at the source. While both environment messages $m_e$ and decision messages $m_d$ are extracted from the shared hidden state, we introduce specialized objective functions to guide their functional specialization: a global state reconstruction loss [15] for environment messages $m_e$ to ensure environmental fidelity and an action-selection mutual information loss [16] for decision messages $m_d$ to encapsulate strategic tendencies. To eliminate “semantic pollution” and cross-redundancy between environment messages and decision messages, we further incorporate a mutual information upper bound constraint via CLUB [19] to minimize $I(m_e;m_d)$. This explicit decoupling mechanism forcibly severs the correlation between the two message streams in the feature space, allowing the receiver to perform precise feature extraction and maximize the utility of differentiated information for robust global coordination.

In summary, we propose Disentangling Environment and Decision messages for Multi-Agent Communication (DEDMAC). Our main contributions are as follows:

• We propose DEDMAC, a communication framework that explicitly separates environment and decision information. It applies distinct processing pathways tailored to the specific characteristics of each, effectively addressing both partial observability and environmental non-stationarity in one unified framework.
• We design a differentiated processing mechanism that integrates environment messages into recurrent states for long-term modeling while utilizing decision messages as instantaneous biases for real-time policy alignment.
• We introduce a perception-first recalibration logic that utilizes aggregated global information to correct the temporal bias of asynchronous intents, ensuring consistency between signaled messages and final actions.
• We implement an independent encoding and explicit disentangling strategy to ensure semantic purity in communication. By employing specialized losses, DEDMAC disentangles the message streams. This mechanism prevents semantic ambiguity and enables precise feature extraction for robust multi-agent coordination.

2. Preliminary

2.1. Problem Formulation

We consider a fully cooperative multi-agent task with partial observations under communication, which can be modeled as a decentralized Partially Observable Markov Decision Process under Communication (Dec-POMDP-Com) [20] and formulated as a tuple $<\mathcal{N},\mathcal{S},\mathcal{A},P,\Omega ,O,R,\gamma ,\mathcal{M}>$, where $\mathcal{N}=1,2,...,n$ denotes the set of agents, $\mathcal{S}$ is the set of global environmental states, offering a comprehensive environment overview. $\mathcal{A}$ is the set of actions, $P:\mathcal{S}\times \mathcal{A}\times \mathcal{S}\to [0,1]$ is the transition function denoting the state transition probability, $\Omega$ is the set of observations, O refers to the observation function, $R:\mathcal{S}\times \mathcal{A}\to \mathbb{R}$ represents the reward function which gives rise to the instantaneous reward, $\gamma \in [0,1)$ stands for the discount factor that gives smaller weights to future rewards, and $\mathcal{M}$ indicates the set of messages. At each timestep t, because of partial observability, each agent $i\in \mathcal{N}$ owns only the local observation $o_i^t=O(s_t,i)$, $s_t\in \mathcal{S}$, and then sends a message $m_{i,j}^t\in \mathcal{M}$ to every agent $j\in \mathcal{N},i\ne j$; after all messages from other agents are received, each agent chooses an action $a_i^t\in \mathcal{A}$ with individual policy ${\pi }_i\left(a_i\right|{\tau }_i,m_i)$, where ${\tau }_i$ is the history $(o_i^0,a_i^0,...,o_i^{t-1},a_i^{t-1},o_i^t)$ of agent i, and $m_i$ contains all messages received by the agent i. The joint action of all agents ${\mathbf{a}}_t=<a_1^t,...,a_n^t>$ leads to next state $s_{t+1}\sim P\left(s_{t+1}\right|s_t,{\mathbf{a}}_t)$ and the global reward $r_t=R(s_t,{\mathbf{a}}_t)$. The formal objective is to find a joint policy $\pi (\tau ,\mathbf{a})$ to maximize the expectation of the accumulated reward: ${\mathbb{E}}_{\pi }\left[{\Sigma }_{t=0}^{\infty }{\gamma }^t{r}_t\right]$, where $\tau =<{\tau }_1,...{\tau }_n>$, and the joint policy $\pi ={\pi }_1\times \cdots \times {\pi }_n$.

2.2. Deep Q-Learning

We use value-based MARL algorithms to optimize the policy, where Deep Q-learning [21] employs a deep neural network parameterized by $\theta$, denoted as $Q(\tau ,\mathbf{a};\theta )$, to represent the action-value function $Q(s,\mathbf{a})$. In the training phase, Deep Q-learning uses a replay memory $\mathcal{D}$ to store the transition tuple $<\tau ,\mathbf{a},r,{\tau }^{\prime }>$. We use $Q(\tau ,\mathbf{a};\theta )$ to approximate $Q(s,\mathbf{a};\theta )$ to relieve the partial observability problem. Thus, the parameters $\theta$ are learned by sampling transitions from replay memory $\mathcal{D}$ and minimizing the expected TD error:

$$\mathcal{L}\left(\theta \right)={\mathbb{E}}_{(\tau ,\mathbf{a},r,{\tau }^{\prime })\in \mathcal{D}}\left[{\left(r+\gamma V({\tau }^{\prime };{\theta }^{-})-Q(\tau ,\mathbf{a};\theta )\right)}^2\right],$$

where $V({\tau }^{\prime };{\theta }^{-})=ma{x}_{{\mathbf{a}}^{\prime }}Q({\tau }^{\prime },{\mathbf{a}}^{\prime };{\theta }^{-})$ is the expected future return of the TD target and ${\theta }^{-}$ are parameters of the target network periodically updated with $\theta$.

3. Method

The overall network architecture of the Disentangling Environment and Decision messages for Multi-Agent Communication (DEDMAC) framework is illustrated in Figure 1. Black arrows in the figure represent the data flow during forward propagation, while red arrows indicate the computation of key loss functions during the training phase.

DEDMAC adheres to the Centralized Training with Decentralized Execution (CTDE) paradigm. During the execution phase, as shown in Figure 1b, each agent i receives its local observation $o_i$ at each timestep t, subsequently generating and broadcasting environment message $m_{i,e}^t$ and decision message $m_{i,d}^t$ via internal modules. Upon receiving messages $m_{\ast ,e}^t$ and $m_{\ast ,d}^t$ from other agents, the agent network outputs the local value function $Q_i({\tau }_i,a_i)$ for action selection. In the training phase, a mixing network utilizes the global state $s_t$ to aggregate individual local $Q_i({\tau }_i,a_i)$ values into a joint value $Q_{tot}$. Since DEDMAC operates at the local $Q_i$ generation level, it can be seamlessly integrated with mainstream value-based algorithms, such as VDN [22], QMIX [9], and QPLEX [23].

The internal architecture of a DEDMAC agent is driven by two core engines: the Message Generator and the Decision Generator. The Message Generator is responsible for the decoupled encoding of environment and decision messages within the latent space, while the Decision Generator performs differentiated integration and policy output based on the received information. This section will sequentially detail the overall agent architecture, message generation logic, decision-processing mechanism, and the corresponding loss functions of DEDMAC.

3.1. Overall Architecture of the DEDMAC Agent

At each timestep t, the DEDMAC agent executes the complete process from “perception” to “communication” and finally to “decision-making” through the coordination of its internal modules. As illustrated in Figure 1b, the overall logical architecture can be divided into the following three phases.

1. Local Feature Encoding Phase: The agent first integrates the current local raw observation $o_i$ with the historical memory $H_i^{t-1}$ from the previous timestep $t-1$ via an Input Encoder. To extract nonlinear local features and maintain temporal consistency, the Input Encoder is composed of a Multi-Layer Perceptron (MLP) and a Gated Recurrent Unit (GRU) in cascade, outputting a temporary hidden state $h_i^t$. This process can be formalized as (1)

$$h_i^t=\mathrm{GRU}(\mathrm{MLP}\left(o_i\right),H_i^{t-1}),$$

(1)

where latent vector $h_i^t$ serves as the fundamental input for all subsequent modules, encapsulating the agent’s entire local cognition prior to obtaining external communication information.

2. Message Generation and Broadcasting Phase: The temporary hidden state $h_i^t$ is then fed into the Message Generator. Within this module, the environment message $m_{i,e}^t$ and decision message $m_{i,d}^t$ are generated based on the temporary hidden state $h_i^t$ and broadcast to other agents.

3. Differentiated Integration and Decision Generation Phase: Upon receiving the environment messages $m_{\ast ,e}^t$ and decision messages $m_{\ast ,d}^t$ sent by other agents, the agent computes the final local value function $Q_i$ and updates its memory based on the local information $h_i^t$ and the received messages, resulting in the updated memory $H_i^t$ for the subsequent timestep $t+1$. Finally, the generated $Q_i$ is used for action selection while simultaneously contributing to the joint training of the centralized Mixing Network.

3.2. Message Generator

The primary objective of the Message Generator is to facilitate the semantic mapping from raw latent vectors to environmental perception and decision intent. As illustrated in Figure 1c, this module employs a hierarchical encoding architecture to transform the temporary hidden state $h_i^t$ into two distinct message streams: the environment message $m_{i,e}^t$ and the decision message $m_{i,d}^t$.

The message generation process is initiated by a foundational Message Encoder. Utilizing a Multi-Layer Perceptron (MLP) structure, this encoder performs preliminary dimensionality reduction and feature reconstruction on the input temporary hidden state $h_i^t$. This stage aims to extract essential communication features that serve as the base representation for subsequent semantic extraction. The computation is defined as (2)

$$h_{i,m}^t=\mathrm{MLP}\left(h_i^t\right).$$

(2)

Subsequently, the feature $h_{i,m}^t$ is directed into two parallel specialized branches. The Environment Encoder focuses on filtering stable descriptions of the physical world from historical memory, while the Decision Encoder is designed to capture the agent’s current strategic bias and latent action intent.

To enhance representation robustness against overfitting and provide support for the disentangling phase, DEDMAC does not utilize the encoder outputs directly as messages. Instead, it incorporates a parameterized sampling mechanism. The Environment and Decision Encoders, respectively, output the parameters—mean $\mu$ and standard deviation $\sigma$—of their corresponding semantic distributions, as shown in (3):

$$[{\mu }_{i,e}^t,{\sigma }_{i,e}^t]=\mathrm{MLP}\left(h_{i,m}^t\right),[{\mu }_{i,d}^t,{\sigma }_{i,d}^t]=\mathrm{MLP}\left(h_{i,m}^t\right).$$

(3)

The system samples the final communication vectors from these distributions. Specifically, the environment message $m_{i,e}^t$ provides a condensed representation of the agent’s current and historical environment modeling, with the sampling process defined as $m_{i,e}^t\sim \mathcal{N}({\mu }_{i,e}^t,{\left({\sigma }_{i,e}^t\right)}^2)$. Similarly, the decision message $m_{i,d}^t$ constitutes a preliminary collaborative broadcast based on a local perspective before receiving peer feedback, formalized as $m_{i,d}^t\sim \mathcal{N}({\mu }_{i,d}^t,{\left({\sigma }_{i,d}^t\right)}^2)$.

This hierarchical encoding and distribution sampling design, built upon MLPs, ensures that the generated heterogeneous messages possess distinct and independent semantic orientations. Furthermore, this stochastic approach provides inherent resistance to interference, guaranteeing the robustness of the communication system in complex, dynamic environments.

3.3. Decision Generator

The Decision Generator is the core module of the DEDMAC agent for achieving differentiated information integration and policy output. As illustrated in Figure 1a, at each timestep t, this module receives the temporary hidden state $h_i^t$ from the Input Encoder, as well as the environment messages $m_{\ast ,e}^t$ and decision messages $m_{\ast ,d}^t$ transmitted from other agents via the communication network. By implementing asymmetric processing paths for these two types of heterogeneous messages with distinct temporal characteristics, the Decision Generator ultimately outputs the current local value function $Q_i$ for action selection and the updated memory state $H_i^t$ as the temporal context for the next timestep $t+1$. The design philosophy of this module is to fundamentally resolve the semantic confusion caused by undifferentiated processing. By orthogonalizing the long-term memory updates of environmental perception and the instantaneous value interference of decision intent, the system ensures both a stable global scene cognition and an agile response to peer collaborative signals.

3.3.1. Environment Message Integration and Long-Term Memory Update

To complement the agent’s local perspective and construct temporally consistent environment cognition, the Decision Generator first processes the environment messages $m_{\ast ,e}^t$, which carry physical environment modeling features. As shown in Figure 1a, the process is as follows:

• Feature Aggregation and State Reconstruction: The system utilizes a self-attention mechanism [24] to capture the spatial dependencies among disparate environment messages. Subsequently, through flattening and Multi-Layer Perceptron (MLP) processing, the global state representation ${\widehat{s}}_t$ is reconstructed. This process is formalized in (4):

  $$\begin{array}{cc}\hfill z_1,\cdots ,z_n& =\mathrm{Attention}(m_{\ast ,e}^t,m_{\ast ,e}^t,m_{\ast ,e}^t),\hfill \\ \hfill {\widehat{s}}_t& =\mathrm{MLP}\left([z_1,\cdots ,z_n]\right).\hfill \end{array}$$

  (4)
• Memory State Update: The reconstructed global state representation ${\widehat{s}}_t$ and the temporary hidden state $h_i^t$ (containing initial local cognition) are fed into a Gated Recurrent Unit (GRU). Through its internal gating mechanism, the GRU consolidates the shared environment information into the agent’s final hidden state $H_i^t$, as defined in (5):

  $$H_i^t=\mathrm{GRU}({\widehat{s}}_t,h_i^t).$$

  (5)
  This path achieves a “long-term update” of cognition, ensuring that the environmental context provided by peers can guide the agent’s subsequent decisions across multiple timesteps.

3.3.2. Decision Message Integration and Instantaneous Intent Bias

Regarding the decision messages $m_{\ast ,d}^t$, since agents only possess a local perspective when generating intent, their broadcast messages often contain significant decision biases resulting from insufficient information. To achieve precise policy alignment, DEDMAC draws inspiration from the Differential Transformer [25] to design an intent filtering and bias correction module. This module leverages a differential attention mechanism, which cancels out redundant information bias by subtracting the results of two attention heads, thereby enabling the agent to focus on the most critical components of peer intent. The specific calculation process is defined in (6) and (7):

$$\begin{array}{cc}\hfill \left[Q_1;Q_2\right]& = H_i^t{W}^Q,\hfill \\ \hfill \left[K_1;K_2\right]& = m_{\ast ,d}^t{W}^K,\hfill \\ \hfill V& = m_{\ast ,d}^t{W}^V,\hfill \end{array}$$

(6)

$$\begin{array}{c}\hfill z_{i,d}=\left(\mathrm{softmax}\left({\displaystyle \frac{Q_1{K}_1^T}{\sqrt{d}}}\right)-\lambda \mathrm{softmax}\left({\displaystyle \frac{Q_2{K}_2^T}{\sqrt{d}}}\right)\right)V,\end{array}$$

(7)

where $W^Q,W^K,W^V$ are learnable parameter matrices. The scaling factor $\lambda$ determines the correction strength and is dynamically calculated via specific learnable parameters, as shown in (8):

$$\lambda =exp({\lambda }_{q1}·{\lambda }_{k1})-exp({\lambda }_{q2}·{\lambda }_{k2})+{\lambda }_{init},$$

(8)

where ${\lambda }_{init}=0.2$ is a parameter used to initialize $\lambda$, and ${\lambda }_{q1},{\lambda }_{q2},{\lambda }_{k1},{\lambda }_{k2}$ are learnable vectors initialized from a Gaussian distribution with a mean of 0 and a standard deviation of 0.1.

In the collaborative context of DEDMAC, the significance of this differential design lies in performing “Signal-to-Noise Ratio (SNR) enhancement.” The first attention head extracts the raw collaborative intent components, while the second head captures redundant decision biases in the latent space. The subtraction operation explicitly removes decision biases arising from local information deficiency, ensuring that the final generated decision bias is based on “calibrated intent” after information completion.

In the final value evaluation stage, DEDMAC fuses long-term environment cognition with instantaneous collaborative intent through a unified formulation. Logically, this formula consists of a baseline evaluation based on temporal memory and a policy bias based on decision messages, both parameterized by MLPs, as formalized in (9):

$$Q_i({\tau }_i,a_i)=\mathrm{MLP}\left(H_i^t\right)+\mathrm{MLP}\left(z_{i,d}\right),$$

(9)

Specifically, the baseline policy term $\mathrm{MLP}\left(H_i^t\right)$ represents the agent’s fundamental strategy, outputting base action values based on the final hidden state $H_i^t$ integrated with global environmental perception. It signifies the agent’s standard decision evaluation based on the completed global environment modeling, ensuring the agent executes task logic according to a stable physical background. The message policy term $\mathrm{MLP}\left(z_{i,d}\right)$ represents the bias exerted by decision messages on the baseline policy. This term takes the aggregated decision intent $z_{i,d}$, corrected by differential attention, as input to generate a dynamic Q-value bias. This term achieves an agile response to peer collaborative signals, guiding the agent toward more precise policy alignment through instantaneous correction of the baseline value.

3.4. Loss Functions of DEDMAC

To guide the DEDMAC framework toward synergistic optimization across three dimensions—semantic disentanglement of communication, perceptual completion, and intent consistency—we design a joint training objective. This objective function is a weighted sum of four loss terms with distinct theoretical interpretations, ensuring that the agents maintain communication semantic purity while enhancing collaborative performance.

The total loss function of DEDMAC, ${\mathcal{L}}_{total}$, is composed of the temporal difference (TD) loss, the state reconstruction loss, the intent consistency loss, and the message disentangling loss. These components are balanced via weighting coefficients, as formalized in (10):

$$\begin{array}{cc}\hfill {\mathcal{L}}_{total}(\theta ,\xi ,\psi )& ={\mathcal{L}}_{TD}\left(\theta \right)\hfill \\ \hfill & + {\lambda }_s{\mathcal{L}}_s\left(\theta \right)\hfill \\ \hfill & + {\lambda }_a{\mathcal{L}}_a(\theta ,\xi )\hfill \\ \hfill & + {\lambda }_{msg}{\mathcal{L}}_{msg}\left(\theta \right)+{\mathcal{L}}_{\psi }\left(\psi \right),\hfill \end{array}$$

(10)

where the weight of the primary task loss ${\mathcal{L}}_{TD}$ is set to 1. ${\lambda }_s,{\lambda }_a,{\lambda }_{msg}$ represent the hyperparameter weights for the respective auxiliary tasks. Since ${\mathcal{L}}_{\psi }\left(\psi \right)$ only involves the update of the auxiliary network, its weight is also set to 1.

3.4.1. Temporal Difference Loss

As the foundational optimization objective in multi-agent reinforcement learning, the temporal difference loss ${\mathcal{L}}_{TD}$ drives the system to complete preset tasks by maximizing cumulative rewards. During the training phase, the algorithm utilizes a mixing network to aggregate the local value functions $Q_i$ output by all agents into a joint value $Q_{tot}$, updating the network parameters by minimizing the TD-error, as shown in (11):

$$\begin{array}{c}\hfill {\mathcal{L}}_{TD}\left(\theta \right)={\mathbb{E}}_{\mathcal{D}}\left[\left(r+\gamma \underset{a^{\prime }}{max}{Q}_{tot}({\tau }^{\prime },{\mathbf{a}}^{\prime };{\theta }^{-})\right.\right.\\ \hfill \left.{\left.-Q_{tot}(\tau ,\mathbf{a};\theta )\right)}^2\right],\end{array}$$

(11)

where $\theta$ denotes the current network parameters, ${\theta }^{-}$ denotes the parameters of a target network that is periodically synchronized with $\theta$, and $\mathcal{D}$ denotes the replay buffer storing the history of agent-environment interactions. Guided by ${\mathcal{L}}_{TD}$, the global reward signal is backpropagated to each agent to steer policy optimization.

3.4.2. Global State Reconstruction Loss

To guide the environment message $m_e$ to iteratively evolve toward the desired environmental descriptive semantics during training, we introduce the state reconstruction loss ${\mathcal{L}}_s$. This term calculates the Mean Squared Error (MSE) between the global state ${\widehat{s}}_t$ reconstructed from environment messages and the ground-truth global state $s_t$. It enforces the environment messages to possess the capability to reconstruct the global physical scene, ensuring that agents share high-fidelity perceptual representations. The loss is defined in (12):

$${\mathcal{L}}_s\left(\theta \right)={\mathbb{E}}_{\mathcal{D}}\left[\parallel s_t-{\widehat{s}}_t{\parallel }_2^2\right].$$

(12)

By optimizing this objective, the environment pathway focuses on capturing persistent and stable physical background information, providing accurate global state support for the hidden state updates in the Decision Generator.

3.4.3. Intent Consistency Loss

The intent consistency loss ${\mathcal{L}}_a$ aims to narrow the policy gap between an agent’s broadcast intent and its final executed action, ensuring the accuracy of intent signaling. In collaborative multi-agent tasks, if the decision message $m_{i,d}^t$ broadcast by an agent lacks correlation with its final action $a_i^t$, the message becomes disruptive noise.

To enforce this “consistency between words and actions,” the framework explicitly guides intent generation by maximizing the mutual information between the decision message and the actual action, $I(m_{i,d}^t;a_i^t|h_i^t)$. As direct optimization of mutual information is difficult, we employ a variational lower bound estimation [26,27]. We introduce a variational distribution $q_{\xi }\left(m_{i,d}^t\right|a_i^t,h_i^t)$ to approximate the true posterior of the decision message, resulting in the loss function for minimization in (13):

$${\mathcal{L}}_a(\theta ,\xi )={\mathbb{E}}_{\mathcal{D}}\left[D_{KL}(p(m_{i,d}^t|h_i^t)\parallel q_{\xi }(m_{i,d}^t|a_i^t,h_i^t))\right],$$

(13)

where $p\left(m_{i,d}^t\right|h_i^t)$ is the decision message generation distribution defined by the message generator parameters $\theta$, and $q_{\xi }$ is the variational distribution defined by parameters $\xi$.

By minimizing the KL divergence, the model forces the message generator to produce $m_{i,d}^t$ that closely resembles the posterior distribution after the final action $a_i^t$ is known, even when the agent only possesses local cognition $h_i^t$. Consequently, the decision message serves as a reliable “preview” of the agent’s future actions.

3.4.4. Message Semantic Disentangling Loss

As the core mechanism for resolving semantic confusion in DEDMAC, the semantic disentangling loss ${\mathcal{L}}_{msg}$ is motivated by explicitly severing the correlation between environment background and decision intent in the latent space. This prevents semantic leakage between the two message streams that originate from a shared temporary hidden state $h_i^t$. To achieve this “hard disentangling,” we introduce the CLUB (Contrastive Log-ratio Upper Bound) estimator [19] to minimize the mutual information $I(m_{i,e}^t;m_{i,d}^t)$.

Given that the message generator employs a parameterized sampling mechanism based on mean $\mu$ and standard deviation $\sigma$, we utilize the efficient upper bound estimation provided by CLUB. As shown in (14), this term forces the feature distributions of the two messages toward orthogonality by contrasting the log-likelihood of positive sample pairs against the average log-likelihood of negative sample pairs:

$${\mathcal{L}}_{msg}\left(\theta \right)={\mathbb{E}}_{m_{i,d}^t\in \mathcal{D}}[log{q}_{\psi }(m_{i,d}^t|m_{i,e}^t)-{\mathbb{E}}_{m_{j,d}^t\in \mathcal{D}}[log{q}_{\psi }(m_{i,d}^t|m_{j,e}^t)]]$$

(14)

where $q_{\psi }$ is a variational estimator used to approximate the conditional probability distribution. Note that ${\mathcal{L}}_{msg}\left(\theta \right)$ is utilized only to update the agent network parameters $\theta$, not the parameters $\psi$ of the variational estimator.

To ensure the CLUB bound remains tight, the variational estimator $q_{\psi }$ must be trained independently to accurately predict the distribution of decision messages given environment messages. Its optimization objective is to maximize the log-likelihood of the observed data, which is equivalent to minimizing the loss function described in (15):

$${\mathcal{L}}_{\psi }\left(\psi \right)={\mathbb{E}}_{\mathcal{D}}\left[-log{q}_{\psi }(m_{i,d}^t|m_{i,e}^t)\right].$$

(15)

This “alternating optimization” design ensures the variational estimator dynamically captures any residual correlation between the two message streams, providing a strict mutual information upper bound for ${\mathcal{L}}_{msg}\left(\theta \right)$. This results in thorough semantic disentanglement, ensuring that environment messages are stripped of unstable decision noise while decision messages are purged of redundant environmental background.

4. Experiments

In this section, we conduct a series of rigorous experiments to evaluate DEDMAC’s performance and architectural efficacy in complex multi-agent tasks. Our evaluation focuses on five key objectives: (1) Benchmark Performance, comparing DEDMAC against state-of-the-art (SOTA) MARL and communication-based baselines in partially observable scenarios to demonstrate robustness; (2) Semantic Interpretability, utilizing visualization and ablation studies to verify the successful disentanglement and supervision of heterogeneous message streams; (3) Denoising Efficacy, investigating the differential attention module’s capacity to filter erroneous intent signals compared with standard multi-head mechanisms; (4) Structural Validity, validating that the asymmetric processing of environment and decision information—integrating $m_e$ into long-term memory while using $m_d$ as an instantaneous bias—is the critical path for efficient policy alignment; and (5) Communication Robustness, assessing the resilience of DEDMAC under non-ideal communication conditions, specifically examining its ability to maintain stable coordination via continuous environmental integration even when subjected to significant packet loss.

4.1. Experimental Settings

4.1.1. Baselines

To evaluate the performance of DEDMAC comprehensively and effectively, we select a representative suite of methods ranging from non-communicative models to state-of-the-art (SOTA) communication mechanisms as baselines:

• QMIX [9]: A state-of-the-art non-communicative MARL algorithm that utilizes a monotonic value decomposition mechanism. It employs a mixing network conditioned on the global state to integrate individual Q-functions into a joint Q-function during training. This approach implements the Centralized Training with Decentralized Execution (CTDE) paradigm and effectively addresses the credit assignment problem while maintaining scalability.
• MASIA [15]: This method adopts a self-supervised learning pathway to extract abstract representations of the global state by aggregating observations from all agents and performing future predictions. Each agent subsequently extracts decision-relevant information from this global representation.
• TarMAC [17]: This algorithm introduces an attention mechanism to process communication messages. It allows agents to selectively extract information most critical to their own decision-making from the incoming message stream via a “signature-query” mechanism.
• MAIC [16]: This method facilitates explicit collaboration by generating incentive messages that directly influence the Q-functions of other agents. It utilizes target teammate models to customize message content and enhances communication efficiency through a sparse communication mechanism.
• T2MAC [28]: A recent communication framework where agents generate “evidence messages” and learn to selectively participate in communication. It improves information integration capabilities in complex environments through an evidence-driven fusion mechanism.

4.1.2. Experimental Benchmarks

We select four representative collaborative multi-agent reinforcement learning (MARL) environments as experimental benchmarks. These environments present substantial communication requirements and are widely recognized for evaluating the efficacy of communication mechanisms [15,28,29]. All experiments are implemented using the PyMARL2 [30] framework. Our experimental benchmarks are detailed as follows:

• StarCraft Multi-Agent Challenge (SMAC) [31]: This is an authoritative collaborative MARL benchmark based on the game StarCraft II. We select two maps characterized by highly asymmetric information: 1o2r_vs_4r and 1o10b_vs_1r [29]. In these maps, the agent team consists of several combat units and an Overseer. The Overseer possesses superior reconnaissance vision but lacks combat capabilities, whereas combat units have restricted sight ranges. This setup creates an intrinsic asymmetric information structure, requiring the Overseer to extract and accurately communicate critical environment information to guide the combat units’ decision-making. We utilize StarCraft II version 4.6.2.6923.
• Hallway [29]: This environment is specifically designed to evaluate temporal coordination under extreme information asymmetry. In this task, multiple agents are randomly placed on Markov chains of varying lengths. Each agent’s perception is strictly localized, knowing only its own position within its hallway. Since agents cannot observe the states or progress of their teammates, effective communication is the sole means to complete collaborative tasks. The task requires all agents belonging to the same group to reach the goal state g simultaneously at the same timestep. Due to differing path lengths and random initial positions, agents must utilize communication to align their progress in real-time. In extended multi-group settings (referencing MASIA [15]), different subgroups are required to reach g at different time points; touching the goal simultaneously results in collision penalties. For instance, Hallway: 3x5-4x6x10 describes a complex task involving a two-agent group (paths of length 3 and 5) and a three-agent group (paths of length 4, 6, and 10). Here, agents must not only achieve internal synchrony but also negotiate the “passing order” across groups, providing an ideal scenario to validate DEDMAC’s semantic decoupling.
• Level-Based Foraging (LBF) [32]: LBF is a grid-world task that demands high levels of collaboration and local perception. Each agent and food item is randomly assigned a level. The success of a “load” action depends on the sum of the participating agents’ levels: a collection succeeds only if the sum of the levels of all agents adjacent to a food item who simultaneously perform the “load” action is greater than or equal to the food item’s level. This rule compels agents to flexibly seek partners based on level disparities. We test two difficulty configurations: for example, LBF: 11x11-6p-4f-s1 denotes an $11\times 11$ map with six agents (p) and four food items (f), where each agent’s sight range (s) is limited to one. In this configuration, task allocation—determining which agents should converge on which food item—rigorously tests the efficiency of communication algorithms in sharing both environment and intentional information.
• Traffic Junction (TJ) [33]: This benchmark simulates urban traffic flow where agents must navigate pre-defined intersecting routes while avoiding collisions. We evaluate DEDMAC on the Medium ($14\times 14$ grid with 10 agents) and Hard ($18\times 18$ grid with 20 agents) configurations. The action space is discrete, consisting of two operations: GAS (moving one step forward) and BRAKE (remaining in the current cell). A critical constraint is the extreme partial observability (vision = 0), meaning agents can only perceive their own identity and current cell. To encourage both efficiency and safety, the reward function issues a constant penalty for each timestep ($r_t=-0.01$) and a severe penalty for each collision ($r_c=-10$). Furthermore, the environment implements a curriculum learning mechanism that gradually increases the agent arrival rate (add_rate) over training epochs, significantly raising traffic density and the frequency of potential conflicts. This setup rigorously tests the architectural efficacy of DEDMAC in filtering high-value decision intent and maintaining a stable environment state representation under increasingly congested and safety-critical conditions.

4.1.3. Implementation Details

The DEDMAC agent employs a deep recurrent neural network architecture, with the specifications of each module detailed as follows:

• Input Encoder: Utilizes a single-layer linear transformation to project raw observations into a 64-dimensional latent space, followed by a ReLU activation function and a 64-unit GRU.
• Message Generator: Composed of a three-layer MLP ($64\to 64\to 32\to 2\times (d_{env}+d_{dec})$), where the environment message dimension $d_{env}$ is set to 6 and the decision message dimension $d_{dec}$ is set to 4. This network generates the mean and variance for both environment and decision messages in parallel.
• Self-Attention Aggregation Module: Employs a single self-attention head with an embedding dimension of 16. A linear layer then maps the flattened output of the self-attention module to a vector with a dimension eight times the number of agents, followed by a 64-unit GRU.
• Differential Attention Module: Consists of two cross-attention heads to perform the subtraction operation, with an embedding dimension of 16.
• Policy Network: Implemented as an MLP that maps inputs to a 64-dimensional hidden representation, followed by a ReLU activation function before being projected into the action space.
• Auxiliary Evaluation Networks: The variational intent estimators used exclusively during training are designed as MLPs with a single hidden layer of 64 units and ReLU activation.

To ensure a fair comparison, all baseline methods with GRU or hidden layer dimensions below 64 were standardized to 64 dimensions. Furthermore, baseline methods incorporating attention mechanisms were configured to use two attention heads.

Regarding communication costs, the communication dimensions for all baseline methods strictly adhere to the default configurations specified in their respective original papers or official codebases. The detailed configurations are as follows:

• QMIX [9]: As a non-communicative baseline, its communication dimension is 0.
• MASIA [15]: Following its self-supervised representation learning design, the message dimension is set to match the agent’s raw observation dimension to facilitate full feature transmission.
• TarMAC [17]: Referring to the original settings, this method employs a 16-dimensional signature/query pair coupled with a 32-dimensional transmission message, resulting in a fixed total communication overhead of 48 dimensions per timestep.
• MAIC [16] and T2MAC [28]: Since their message designs are coupled with the action space, the communication dimensions are set to the number of discrete actions in the corresponding environment.
• DEDMAC (Ours): We restrict the message space to a minimal fixed dimension of 10 in total. Specifically, six dimensions are allocated for environment semantic encoding, while four dimensions are reserved for decision intent representation. We set $d_{dec}=4$ to cover the entropy of the maximum action space (${log}_2\left(10\right)\approx 3.32$), while $d_{env}=6$ is chosen based on empirical results showing diminishing returns with higher dimensions.

The communication dimensions for each algorithm in specific experimental scenarios (measured as the number of floats generated by a single agent per timestep) are summarized in

Table 1. Communication costs in different scenarios.

Scenario	QMIX	MASIA	TarMAC	MAIC	T2MAC	DEDMAC (Ours)
SMAC
1o10b_vs_1r	0	84	48	7	7	10 (6 + 4)
1o2r_vs_4r	0	49	48	10	10	10 (6 + 4)
Hallway
4x6x10	0	1	48	3	3	10 (6 + 4)
3x5-4x6x10	0	2	48	3	3	10 (6 + 4)
LBF
11x11-6p-4f-s1	0	30	48	6	6	10 (6 + 4)
20x20-10p-6f-s1	0	48	48	6	6	10 (6 + 4)
TJ
Medium	0	61	48	2	2	10 (6 + 4)
Hard	0	149	48	2	2	10 (6 + 4)

.

Additionally, concerning computational costs, a detailed analysis of the complexity—comprising trainable parameters and FLOPs relative to the strongest baseline (MASIA)—is presented in Appendix A. This section features an additional evaluation using an isoparametric version of the baseline to further demonstrate that DEDMAC’s performance gain originates from its structural innovation rather than merely a byproduct of increased model capacity.

Under these settings, DEDMAC maintains a constant and low-bandwidth communication load across all scenarios. Particularly in environments with high-dimensional observations (e.g., SMAC), its communication dimension is significantly lower than those of MASIA and TarMAC; in environments with low-dimensional action spaces (e.g., Hallway), the load remains comparable in magnitude to methods like MAIC. This configuration is intended to verify whether critical information extracted through the semantic decoupling mechanism is sufficient to support complex multi-agent collaboration under constrained bandwidth conditions. All algorithms are executed within the same underlying framework, ensuring a controlled evaluation process and reliable results.

All experiments are conducted using the Adam optimizer. To balance the multiple auxiliary tasks, the loss function weights were meticulously tuned; the specific hyperparameter configurations are summarized in

Table 2. DEDMAC Hyperparameter Settings.

Category	Parameter	Symbol	Value
General	Batch size	-	32 episodes
	Buffer size	-	5000 episodes
	Target update interval	-	200 episodes
Optimizer	Learning rate	$\eta$	$5\times {10}^{-4}$
	Grad norm clip	-	10.0
	Discount factor	$\gamma$	0.99
Loss Weights	State reconstruction	${\lambda }_s$	0.5
	Intent consistency	${\lambda }_a$	0.1
	Semantic disentanglement	${\lambda }_{msg}$	0.01

, and a detailed sensitivity analysis evaluating the robustness of these configurations is provided in Appendix B.

To ensure empirical fairness and rigor, DEDMAC and all baseline methods—including both non-communicative and communicative variants—consistently utilize QMIX [9] as the underlying value decomposition framework. Consequently, the credit assignment logic remains identical across all evaluated methods, ensuring that any performance variations stem exclusively from the architectural designs of the communication mechanisms or representation learning modules.

The specific QMIX configuration is as follows:

• Mixing Network: We employ the standard QMIX monotonic constraint network with a mixing embedding dimension of 32.
• Hypernetworks: The hypernetworks responsible for generating the weights of the mixing network utilize a two-layer MLP architecture (including one hidden layer), with the internal hypernet embedding dimension standardized to 64.

4.2. Comparison with SOTA Methods

To evaluate the effectiveness of the proposed DEDMAC framework, we conducted experiments across several benchmark environments. All performance metrics reported herein were obtained by suspending training every M timesteps and evaluating the agents over N independent test episodes. During the evaluation phase, agents adopt a greedy action selection strategy; furthermore, to ensure deterministic assessments, all probability distributions are replaced by their respective means. The specific values for the evaluation parameters $(M,N)$ are $(5\times {10}^4,100)$ for SMAC, LBF and TJ; $(1\times {10}^4,100)$ for Hallway. To ensure statistical reliability, all training curves and data are reported based on five random seeds, represented as the median performance and the $95\%$ confidence interval. These evaluation protocols are strictly aligned with the experimental setup in MASIA [15], ensuring a fair and rigorous comparison.

The experimental results, illustrated in Figure 2, compare DEDMAC with various state-of-the-art baselines, with the x-axis denoting training timesteps and the y-axis representing the test win rate. Across all evaluated benchmarks, DEDMAC performs at least on par with the strongest baselines while demonstrating significant superiority in terms of peak win rate, convergence speed, and training stability in several key scenarios.

In the StarCraft II (SMAC) scenarios (Figure 2a,e), particularly in the complex 1o10b_vs_1r scenario, DEDMAC exhibits the most rapid performance growth, stabilizing at a high win rate exceeding $0.85$ during the mid-to-late training stages. This result significantly surpasses contemporary communication-based algorithms such as MASIA and MAIC, highlighting the robustness of our approach in complex environments.

The advantages of our framework are even more pronounced in the Hallway scenarios (Figure 2b,f), which demand extreme levels of temporal coordination. While the non-communicative baseline QMIX fails to develop any effective collaborative behaviors, DEDMAC consistently reaches a win rate of $1.0$ in both scenarios. Notably, in the most challenging 3x5-4x6x10 scenario (Figure 2f), DEDMAC achieves full convergence approximately $0.25\times {10}^6$ timesteps earlier than MASIA, the fastest performing baseline. This result validates that the dual-pathway communication mechanism, coupled with the explicit intent bias correction, enables precise alignment of collaborative intentions.

In the LBF tasks (Figure 2c,g), baselines such as TarMAC, T2MAC, and MAIC struggle significantly due to sparse rewards, occasionally underperforming even the non-communicative QMIX. In contrast, DEDMAC exhibits performance comparable to the strongest baseline, MASIA, even in the larger-scale 20x20-10p-6f-s1 scenario (Figure 2g) where rewards are exceptionally sparse. These results demonstrate DEDMAC’s capacity to extract meaningful coordination signals from sparse-reward environments.

Lastly, in the Traffic Junction (TJ) scenarios (Figure 2d,h), we restricted the agents’ sight range to 0, creating an environment highly dependent on communication. Under these conditions, QMIX exhibits the lowest performance, confirming the necessity of shared information. Despite this extreme partial observability and the large agent populations (10 in Medium, 20 in Hard), DEDMAC maintains exceptional training stability and converges rapidly to a performance level that matches or exceeds all baselines. Its significant margin over TarMAC, MAIC, and T2MAC underscores the robust nature of the DEDMAC algorithm in managing large-scale, communication-critical coordination tasks.

Overall, the experimental results across diverse benchmarks validate that by disentangling environment and decision semantics, DEDMAC consistently facilitates more effective and efficient multi-agent coordination, particularly in scenarios characterized by high information asymmetry or sparse feedback.

4.3. Ablation Studies and Structural Analysis

To systematically evaluate the specific contributions of the core components of DEDMAC, we conduct a series of targeted ablation experiments in the LBF: 11x11-6p-4f-s1 environment. The quantitative impact of each component on the final performance is summarized in

Table 3. Ablation results in the LBF: 11x11-6p-4f-s1 scenario. We report the final win rate (%) and the performance gap ($\Delta$) compared to the complete DEDMAC.

Variant	Final Win Rate (%)	Performance Drop ($\Delta$)
DEDMAC (Full)	100.0	–
Optimization Objectives (Losses)
w/o l_a (Intent Consistency)	39.9	−60.1
w/o l_s (Global Reconstruction)	62.6	−37.4
w/o l_msg (Semantic Disentangling)	91.9	−8.1
Structural Components
w/o Env-Msg (Environment Messages)	77.4	−22.6
w/o Dec-Msg (Decision Messages)	93.8	−6.2
w/o Env-Proc (Long-term Memory Update)	96.2	−3.8
w/o Dec-Proc (Policy Bias)	82.4	−17.6
w/o Diff-Attn (Differential Attention)	71.7	−28.3

, while the dynamic learning processes are visualized in Figure 3a,b.

We first focus on the optimization objectives by investigating the necessity of the auxiliary loss functions: the global reconstruction loss ${\mathcal{L}}_s$, the intent consistency loss ${\mathcal{L}}_a$, and the semantic disentangling loss ${\mathcal{L}}_{msg}$. As shown in Table 3, the individual removal of these objectives (DEDMAC w/o l_a, DEDMAC w/o l_s, and DEDMAC w/o l_msg) leads to a measurable decline in the final win rate. More importantly, while the table highlights the final performance gap, the learning curves in Figure 3a reveal that these auxiliary tasks significantly accelerate network convergence. By providing high-quality gradient guidance early in the training process, these losses allow DEDMAC to reach peak performance with substantially fewer environment steps compared with the ablation variants, which often exhibit sluggish progress or severe oscillations.

Complementary to these optimization goals, we examine the structural validity of the dual-pathway architecture of DEDMAC using five variants: DEDMAC w/o Env-Msg and DEDMAC w/o Dec-Msg (pathway removal); DEDMAC w/o Env-Proc and DEDMAC w/o Dec-Proc (logic homogenization); and DEDMAC w/o Diff-Attn (standard attention). According to Table 3, omitting the specialized long-term memory update or policy bias logic results in significant performance degradation. Beyond the final win rates, Figure 3b further illustrates the structural efficiency of our design. The specialized pathways and differential attention module effectively suppress redundant information and “semantic interference” from the outset, enabling the agents to align their policies much faster than the variants using unified or standard attention mechanisms.

Collectively, these findings demonstrate that environmental perception, intent consistency, and semantic disentangling are indispensable for efficient multi-agent coordination. The synergy between targeted optimization objectives and a specialized dual-pathway structure ensures that shared information is not only semantically pure but also structurally utilized to its fullest extent, leading to a framework that is both highly performant at convergence and exceptionally efficient during training.

4.4. Visualization Analysis of Messages

To provide further intuitive verification of whether the DEDMAC algorithm successfully achieves the semantic disentanglement of environment and decision information during the message generation process, we employ the t-SNE [34] algorithm to reduce the dimensionality of message features and visualize their distribution patterns. This analysis focuses on messages sent by agents in the Hallway: 3x5-4x6x10 scenario.

Figure 4a illustrates the visualization results for environment messages. In this plot, feature points are colored according to the agent’s physical position (Positions 1, 2, 3, and 4). It is observed that message features corresponding to the same or adjacent physical locations exhibit significant clustering characteristics in the latent space, with clear boundaries between clusters representing different positions. This strongly demonstrates that the environment message encoder accurately extracts features related to spatial status from local observations. Such environment-based semantic representations provide a solid foundation for the Decision Generator to integrate global state information through the self-attention mechanism.

In contrast, Figure 4b shows the distribution of decision messages, colored based on the final decision executed by the agent at that timestep (moving toward vs. away from the goal). The results indicate that decision messages cluster on the left and right sides of the plot according to the intended action type. This provides compelling evidence that the features embedded in these messages effectively predict the agent’s policy tendencies. These proactive message semantics enable agents to inform their teammates of likely future decisions, achieving deep alignment at the intentional level.

4.5. Study on the Effectiveness of the Intent Filtering and Bias Correction Module

Focusing on the processing of decision messages, this study utilizes the Hallway: 3x5-4x6x10 scenario as a benchmark to investigate the synergistic filtering mechanism between the dual-pathway differential attention heads. We introduce the metric “information pass rate”—defined as the ratio of the final attention score to the score from the positive attention head—to qualitatively characterize the attenuation or enhancement of the information flow during the filtering process by the negative attention head. In this framework, only information with critical collaborative value achieves a high pass rate.

We analyzed a specific scenario in Hallway: 3x5-4x6x10 where one group of agents is about to enter the final goal while the second group waits at the entrance. The resulting information pass rates are illustrated in Figure 5.

The numerical distribution of the pass rates reveals significant asymmetric information filtering characteristics. As shown in Figure 5a, when the 3x5 team enters first while the 4x6x10 team waits, precise identification of critical signals is evident: agents in the first group (+3, +5) exhibit significantly higher pass rates as senders. Because these agents are in a critical phase of changing the environmental state (entering the goal), their decision messages carry core collaborative intent. Consequently, the positive attention head assigns high weights, resulting in retention rates exceeding 0.4, and even surpassing 0.83 when transmitted to the second group.

Conversely, the effective suppression of redundant information is validated when the second group (−4, −6, −10) acts as senders. In this state, these agents are either waiting or attempting to enter prematurely. If an agent intends to wait, its behavioral intent is of low value; if it intends to enter, that intent constitutes “erroneous information” that must be filtered out. In both cases, the pass rates drop to near zero or even become negative. This indicates that the negative attention head successfully identifies and filters these low-value or disruptive signals, preventing redundant noise from interfering with the final joint decision.

As a comparison, when the roles are reversed (the 4x6x10 team enters while the 3x5 team waits), the pass rates exhibit corresponding adaptive patterns (Figure 5b). This situation-based dynamic pass rate allocation demonstrates the interpretability of the DEDMAC differential attention mechanism: agents at critical task nodes are granted a stronger “voice” in the communication channel, while the valueless or erroneous intents of other agents are effectively suppressed.

4.6. Communication Robustness Analysis

To evaluate the robustness of our proposed algorithm, we conducted experiments under non-ideal communication conditions where messages are subject to random dropout. Specifically, we simulated packet loss rates ranging from 10% to 30%, where lost messages were replaced by all-zero vectors. The experiments were performed in the 1o2r_vs_4r scenario. For each loss rate, we retrained the models for 2 million timesteps to assess their adaptability and report the final win rates in

Table 4. Performance robustness under communication packet loss (1o2r_vs_4r). We report the win rate (%) and the performance drop (indicated in brackets) compared to the baseline under ideal communication.

Algorithm	Packet Loss Rate
	0% (Baseline)	10%	20%	30%
TarMAC	82.0	65.7 (−16.3)	54.0 (−28.0)	42.8 (−39.2)
T2MAC	10.6	16.2 (+5.6)	8.1 (−2.5)	7.6 (−3.0)
MAIC	55.6	54.1 (−1.5)	52 (−3.6)	46.7 (−8.9)
MASIA	78.2	71.8 (−6.4)	69.4 (−8.8)	60.1 (−18.1)
DEDMAC (Ours)	86.2	81.9 (−4.3)	78.7 (−7.5)	71.5 (−14.7)

. The results demonstrate that DEDMAC consistently achieves the highest average win rate across all packet loss intensities.

We attribute the resilience to DEDMAC’s disentangled semantic processing and its dual-pathway architecture. Unlike methods like TarMAC, which treat exchanged information as a black box and suffer from catastrophic performance decay (dropping by 39.2% at a 30% loss rate) due to their reliance on conflated and sensitive hidden representations, DEDMAC maintains a much steadier performance. Specifically, by leveraging a reliable global context derived from previously shared environment information, agents can sustain a coherent and clear understanding of the overall system state throughout the episode. This stable environmental grounding provides a crucial anchor for rectifying transient deviations in subsequent decision-making when intent packets are missing, effectively preventing the coordination divergence observed in other baselines. Notably, while MAIC shows some degree of stability, its absolute win rate remains significantly lower than that of DEDMAC, indicating that DEDMAC’s disentangled approach not only resists noise but also maximizes the utility of successfully transmitted bits for more precise coordination. These results validate that the structural disentanglement of message streams serves as a natural defense against communication unreliability in complex multi-agent environments.

5. Discussion

The experimental results and visualization analysis provide compelling evidence that DEDMAC effectively addresses the “semantic confusion” problem—a common bottleneck in existing multi-agent communication frameworks. Our working hypothesis was that “what an agent sees” (environment) and “what an agent plans to do” (intent) possess inherently different statistical and functional properties; thus, they should not be conflated into a single latent vector.

The performance gains observed in SMAC and the rigorous coordination demands of the Hallway environment substantiate this hypothesis. The asymmetric processing mechanism of DEDMAC enables agents to maintain a stable, long-term “world model” through environment messages while simultaneously leveraging decision messages for agile, real-time strategic adjustments. Furthermore, the t-SNE visualizations (Figure 4) serve as a definitive evidence, qualitatively demonstrating that our auxiliary losses effectively facilitate the policy learning process.

In a broader context, DEDMAC offers a blueprint for interpretable MARL. By calculating the “information pass rate” via differential attention, we can effectively identify which agents are leading the group at any given timestep. This has profound implications for real-world robotics and autonomous swarms, where bandwidth is limited and the ability to verify why an agent is communicating specific information is critical for ensuring safety and trust.

Limitations and Future Work: While DEDMAC uses fixed weights ($\lambda$) for auxiliary losses, future research could explore dynamic weighting mechanisms (e.g., using GradNorm or uncertainty-based weighting) to further stabilize the early stages of training.

6. Conclusions

In this paper, we propose DEDMAC, a novel multi-agent communication framework designed to achieve semantic disentanglement and intent consistency. By introducing a dual-pathway architecture and a suite of auxiliary representation learning objectives, we successfully separated environmental background information from strategic decision-making intent.

Our findings demonstrate the following:

• Disentanglement Is Essential: Explicitly severing the mutual information between $m_e$ and $m_d$ prevents “semantic leakage,” leading to more robust coordination in partially observable environments.
• Differentiated Processing Is Key: The effectiveness of DEDMAC stems from aligning processing pathways with the distinct natures of information. Stable, spatial environment messages are integrated into long-term memory to maintain a consistent world model, while transient, intent-driven decision messages are used for real-time strategic correction. This asymmetric design ensures that heterogeneous information is processed according to its specific functional role.
• Superior Efficiency: DEDMAC outperforms SOTA baselines in win rate and convergence speed across SMAC, Hallway, and LBF benchmarks, all while maintaining a minimal communication overhead of only 10 dimensions.