A Multi-Agent Emergency Material Allocation Approach Based on a Markov Decision Process Under Demand Uncertainty for Sustainable Disaster Response

Abstract

Effective emergency relief allocation in dynamic post-disaster environments depends critically on accurate and timely demand information. From a sustainability perspective, improving allocation accuracy is essential for using scarce rescue resources efficiently and supporting resilient disaster response. However, existing demand forecasting approaches frequently exhibit systematic bias, leading to resource misallocation and diminished rescue outcomes. Although deploying on-site assessment teams can partially mitigate this limitation, a unified framework that systematically embeds field assessment feedback into operational allocation processes remains lacking. To bridge this gap, this study proposes a multi-agent joint assessment-allocation model that facilitates coordinated operations between demand assessment and resource distribution activities. The sequential decision-making process is formulated as a Markov Decision Process (MDP), and deep reinforcement learning is employed to coordinate the actions of assessment and allocation teams, enabling allocation policies to be continuously refined through real-time field feedback. By improving the match between actual demand and material supply, the proposed model aims to support more resource-efficient disaster response under demand uncertainty. An empirical case study based on the 2025 Dingri County earthquake in Tibet is conducted to validate the proposed framework. Results demonstrate that integrating assessment feedback substantially improves resource allocation performance: in multi-site rescue scenarios, the framework increases the number of rescued individuals, reduces mission completion time, and enhances overall demand satisfaction. Further sensitivity analysis reveals that a moderate increase in team size strengthens cross-site coordination, whereas excessive team deployment yields diminishing returns and may generate operational redundancy. These findings suggest that sustainable emergency management depends not only on the availability of relief resources, but also on the efficient coordination of real-time information acquisition and material allocation. The proposed framework offers a generalizable approach for integrating real-time information acquisition with dynamic relief allocation. It improves the efficient utilization of scarce rescue resources, reduces avoidable operational redundancy, and strengthens the resilience of emergency response systems, thereby contributing to sustainable disaster risk reduction.

1. Introduction

The escalating frequency and severity of natural disasters, compounded by increasing societal vulnerability and exposure, have emerged as one of the most pressing challenges in contemporary disaster risk management. Events such as earthquakes and large-scale floods are typically characterized by rapid evolution, cascading effects, and complex spatial dynamics, propagating impacts across multiple affected sites and posing severe threats to human safety, social stability, and economic continuity. In this context, sustainable disaster response is closely linked to disaster risk reduction and response-system resilience, as emphasized by the United Nations Sustainable Development Goals and the Sendai Framework for Disaster Risk Reduction. Against this backdrop, the timely delivery and efficient allocation of emergency materials, including food, water, medicine, tents, blankets, as well as the deployment of rescue personnel, constitute a critical component of post-disaster response. The quality of allocation decisions directly determines rescue effectiveness and the degree to which the needs of affected populations are met. Evidence from recent large-scale relief operations, including the 2023 Turkiye earthquake and the Luding earthquake in Sichuan, indicates that resource allocation commonly confronts two fundamental challenges [1,2,3]: (i) allocation efficiency is constrained by limited resource stocks and transportation capacity, resulting in critical supply mismatches and delayed assistance; and (ii) demand assessment and resource distribution are typically managed by different organizations with limited coordination mechanisms, which can substantially degrade system performance and amplify disaster losses [4,5]. There is therefore a compelling need for a collaborative multi-actor framework that establishes a structured, closed-loop assessment-allocation feedback mechanism to enhance response performance.

Post-disaster emergency supply allocation has accordingly attracted substantial research attention [6]. Early contributions primarily relied on static optimization models assuming deterministic demand, employing quantitative methods such as integer linear programming, multi-objective programming, and bi-level programming [7,8] to address problems including emergency supply distribution [9], vehicle routing [10], and facility location-allocation [11]. Despite their methodological rigor, static models are poorly suited to practice, as the inherently dynamic and uncertain nature of disaster events causes demand to fluctuate in ways that static solutions cannot accommodate [12]. Dynamic resource allocation models, by contrast, continuously reassess demand across multiple decision epochs and adapt allocation plans accordingly, yielding more responsive and practically effective relief dispatch strategies [13,14,15,16,17,18].

Demand uncertainty constitutes a core challenge in emergency supply allocation [19]. Stochastic programming approaches have been widely adopted to address this challenge by constructing explicit scenario sets to represent uncertain conditions. Illustratively, Wang et al. [9] proposed a two-stage stochastic programming model to jointly optimize facility location and material allocation over predefined scenarios, while Mohamadi and Yaghoubi [20] developed a bi-objective stochastic model for casualty transportation and medical supply distribution under uncertain demand and response timing. A fundamental limitation of stochastic programming approaches, however, is their dependence on predefined scenario sets [6], which fail to capture dynamic feedback effects of decisions on system state transitions and often fail to reflect the rapidly evolving realities of actual disaster response. This limitation is particularly consequential because allocation decisions made at any point in time directly constrain demand fulfillment capacity in subsequent periods. This sequential interdependence highlights the need for decision frameworks that possess foresight, dynamic updating capabilities, and the capacity to account for the long-run consequences of early-stage decisions [21,22,23,24].

The effectiveness of resource allocation depends not only on optimization methodology but equally on the availability of accurate and timely demand information [25]. In the early phase following a disaster, the true magnitude of need is rarely directly observable, and estimates based on pre-disaster population exposure models and vulnerability indices frequently deviate substantially from actual ground conditions [20]. Field assessment is widely recognized as the primary means of obtaining accurate demand data. However, assessment resources, including rescue teams and unmanned aerial vehicles, are themselves limited, and the problem of deploying them optimally to maximize the informational value of assessments under strict time constraints is a critical operational challenge. More fundamentally, information gathered through assessment must be effectively translated into timely allocation decisions [26]: when assessment and allocation agents operate without adequate coordination, information delays and distortions can rapidly erode the value of field intelligence and undermine response quality [27]. However, research that directly integrates field information into decision optimization models remains limited.

Motivated by observations from both practice and academic research, this paper aims to address the following questions:

• How can the assessment process increase the number of people rescued by improving the information required for allocation decisions?
• How can the assessment agent and the allocation agent be effectively coordinated within a highly uncertain, time-sensitive operational environment?

Multi-agent systems (MAS) and agent-based modeling (ABM) offer powerful tools for addressing such coordination problems. These approaches have been applied to simulate inter-departmental cooperation in emergency management [28,29], evaluate collaborative response strategies [30], and optimize multi-vehicle logistics scheduling [31,32]. A related strand of research combines agent-based simulation with heuristic optimization methods to support emergency decision-making. Beklaryan et al. [33] modeled adaptive rescuer behavior under emergency conditions using agent-based simulation with modified fuzzy clustering, while Hawe et al. [34] employed agent-based simulation to plan resource allocation for a hypothetical two-site major incident. Building on this foundation, Akopov et al. [35] integrated a parallel bi-objective genetic algorithm with agent-based rescuer simulation to optimize evacuation parameters, and Guerrero Granados et al. [36] further combined an Improved Genetic Algorithm with Multi-objective Particle Swarm Optimization within a coordinated multi-agent framework for technological disaster management. However, existing studies largely focus on macro-organizational behavior or isolated operational subtasks. A unified framework that coherently integrates on-site demand assessment and operational resource allocation as a coupled system remains absent from the literature, and the mechanisms by which human and artificial agents can collaborate to complete a closed-loop assessment-allocation process under high uncertainty are not well understood. The Markov Decision Process (MDP) framework offers a principled foundation for modeling such problems, owing to its strengths in sequential decision-making, explicit state transition modeling, and long-term reward optimization [37,38]. The MDP framework has been successfully applied in emergency management contexts, including ambulance dispatch [39,40] and medical resource allocation [41,42]. Although several researchers have extended MDP-based approaches to post-disaster resource allocation [24,43], the coordinated integration of assessment and allocation operations within a unified MDP framework has not been adequately investigated.

To address these research gaps, this study proposes an integrated multi-agent framework for disaster response in which the post-disaster decision-making process is formulated as an MDP. A central artificial Decision-Making Agent (DMA) learns an adaptive response policy from environmental interactions and coordinates the operations of a human-staffed Demand Assessment Team (AT) and Resource Allocation Team (ST), both composed of human agents operating in the field, thereby enabling a principled and adaptive assessment-allocation feedback loop. To validate the proposed framework, a case study is conducted based on the 2025 Dingri County earthquake in Tibet, comparing rescue performance with and without integrated demand assessment. Complementary sensitivity analyses on team size provide actionable managerial guidance for the optimal configuration of multi-agent disaster response systems.

This study makes three primary contributions to the literature. First, it introduces a novel multi-agent collaborative MDP model for emergency material allocation, offering a methodological advance in this domain. While decision-making under uncertainty is inherently challenging in single-agent settings, the systematic integration of multiple specialized agents within an MDP framework remains relatively underexplored, particularly when explicit coordination between demand assessment and material allocation roles is required. This work provides practical guidance for future applications of MDP-based models in complex disaster response environments. Second, it proposes a unified operational framework that coordinates the DMA, AT, and ST within a single allocation process, establishing a structured approach to human and artificial intelligence agent collaboration under conditions of spatiotemporal uncertainty in emergency operations. Third, from a sustainability perspective, this study shows that real-time demand assessment and adaptive material allocation can promote the efficient use of scarce emergency resources, reduce avoidable operational delays, and enhance the resilience of disaster response systems. In this regard, the proposed framework extends emergency supply allocation optimization toward sustainable disaster risk reduction.

The remainder of this paper is structured as follows. Section 2 presents the proposed multi-agent decision model and its mathematical formulation. Section 3 describes the empirical case study and discusses the experimental results. Section 4 presents the conclusions, managerial implications, and directions for future research.

2. Problem Definition and Model Description

2.1. Problem Description

The model proposed in this study focuses on optimizing the coordination between demand assessment and material distribution to maximize the number of lives saved in post-disaster response operations. All sets, indices, parameters, and variables are defined in

Table 1. Sets, indices, parameters, and variables included in the model.

Notation	Definition
Sets, Indices, and Parameters
I	Set of demand zones, indexed by $i,j$
K	Set of medical supply points, indexed by $\mathrm{k}$
S	All feasible states
$X_{S_t}$	Set of actions at decision epoch $\mathrm{t}$ in state ${\mathrm{S}}_{\mathrm{t}}$
$L^t$	Zone list defining the material allocation sequence for disaster zones
$z$	Route link, $\left(i,j\right)\mathsf{ϵ}\mathrm{z}$, $\mathrm{z}=\left\{1,2,3\dots \right\}$
$i,j$	Index of disaster zones
${\theta }_{i,j}$	Travel time from zone $i$ to zone $j$
${\epsilon }_i$	Service time at zone $i$
$T_{i,j}$	Travel time from zone $i$ to zone $j$
${FT}_i$	Completion time at zone $i$
$h_i$	Performance factor
$N_i$	Nominal demand in zone $i$ (vulnerable population in zone $i$)
$N_{ir}$	Actual demand in zone $i$ (actual injured population in zone $i$)
$\gamma$	Discount factor
State variables
$S_t$	Consists of two components: material status and disaster zone request status, at decision epoch $t$
$M_{tk}$	The status of supply point k, indicating which disaster zones can be serviced at decision epoch $t$
$R^{tk}$	The total available materials from supply point k at decision epoch $t$
$D_{ti}$	Characteristics of the disaster zone $i$ request at decision epoch $t$
$O^{ti}$	Location of a disaster zone $i$ at decision epoch $t$
$e^{ti}$	Demand level of the corresponding zone at decision epoch $t$
$\mathcal{J}\left({\mathsf{\mu }}_{ti},{\mathsf{\sigma }}_{ti}\right)$	Probability distribution; explanatory variables ${\mathsf{\mu }}_{ti},{\mathsf{\sigma }}_{ti}$
$W^{ti}$	Whether the disaster zone $i$ is being served at time $t$, If $W^{ti}=0$, it means the zone is not being served at decision epoch $t$
Decision variables
$x_t$	Indices of actions, $x_t\mathsf{ϵ}{X}_{S_t}$ at decision epoch $t$
$a_{tij}^z$	Whether disaster zone $i$ chooses route z for the material need assessment of the next disaster zone $j$ ($a_{tij}^z=1$) or not ($a_{tij}^z=0$), at decision epoch $t$
$b_{ti}^k$	Quantity of materials allocated from supply point k to disaster zone $i$, at decision epoch $t$
Function
$V{(S}_t)$	Value function of states $S_t$ at time $t$
$C\left(S_t,x_t\right)$	The reward function
$o_t^{i,x_t}$	The rescue effect of taking feasible action $x_t$
$\delta \left({\theta }_{i,j}\right)$	Survival function describing the effectiveness of relief assessment
$Q\left(s_t,x_t\right)$	Approximation function (Q-value), represented by the number of people rescued
$X^{\pi }\left(S_t\right)$	Decision function
$V^{*}\left(\phi \left(s_t,x_t\right)\right)$	The optimal value of the next state $\phi \left({\mathrm{s}}_{\mathrm{t}},{\mathrm{x}}_{\mathrm{t}}\right)$

.

Two foundational demand concepts are adopted: nominal demand ($N_i$) and actual demand ($N_{ir}$). Nominal demand represents estimated supply requirements derived from historical exposure data, serving as the initial planning benchmark for relief operations. Actual demand, by contrast, is determined by field personnel using real-time observations and predictive models upon arrival at the affected zone [44]. The central methodological challenge is to establish a mechanism that enables the efficient transition from nominal to actual demand through structured field assessment, thereby improving allocation accuracy. To address this challenge, this study proposes a multi-agent framework for demand assessment and material allocation comprising three specialized agents, each fulfilling an indispensable and complementary role. The multi-agent structure is essential for managing the complexity of real-time decision-making in disaster environments, where task specialization, inter-agent communication, and adaptive coordination are critical determinants of system performance.

The DMA serves as the central artificial learning agent, responsible for scenario simulation, demand forecast refinement, and allocation guidance. In the initial phase of response, the DMA generates demand forecasts using simulated data drawn from a Gaussian distribution with zero mean and unit variance. As the response progresses, the DMA continuously assimilates real-time field intelligence provided by the AT and the ST to revise its demand estimates, enabling dynamic and context-adaptive adjustments to the overall relief strategy.

The Demand Assessment Team (AT) consists of human agents deployed in the affected areas to collect and evaluate on-site demand data. In addition to gathering field information, the AT conducts preliminary route assessments and provides regular updates to the DMA. When unanticipated events arise, such as sudden demand surges or newly identified hazards, the AT promptly evaluates the situation and communicates this information to the DMA. This feedback mechanism allows the DMA to refine its decision models in real time, ensuring that the relief strategy remains adaptive to evolving ground conditions. The DMA, in turn, guides the AT’s operational priorities by directing field personnel toward zones or issues requiring the most urgent attention.

Due to the time-sensitive nature of emergency response, the efficient routing of field agents is critical. The time elapsed between an agent’s dispatch and the completion of service at each zone directly affects the overall efficiency of material allocation. For an agent serving zone $i,$ the start time $\mathrm{S}{F}_i$, service time ${\epsilon }_i,$ and completion time $\mathrm{F}{T}_i$ are calculated using Equations (1), (2) and (3), respectively:

$${\epsilon }_i=h_i·N_i$$

(1)

$$\mathrm{F}{T}_i=\mathrm{S}{F}_i+{\epsilon }_i$$

(2)

$$\mathrm{S}{F}_j=\mathrm{F}{T}_i+T_{i,j}$$

(3)

where the service time ${\epsilon }_i$ is proportional to the material demand $N_i$. The performance factor $h_i$ is treated as a zone-specific stochastic parameter to reflect the variability in operational capacity across different field conditions. Equation (2) defines the completion time at zone $i$ as the sum of start time and service time. Equation (3) specifies that the start time at the subsequent zone $j$ equals the completion time at the current zone $i$ plus the inter-zone travel time $T_{i,j}$. Equation (4) represents the survival function $\mathsf{\delta }\left(\mathsf{\theta }\right)$, shown in Figure 1. This function indicates that the survival rate decreases exponentially as time ${\theta }_{i,j}$ increases; consequently, the visit priority assigned to a region also decreases as time progresses. The reverse S-curve further emphasizes the critical importance of the initial few time units in saving lives, as detailed in Edrisi [45].

$$\mathsf{\delta }\left(\mathsf{\theta }\right)=\mathsf{\alpha }{\mathrm{e}}^{\mathrm{b}{\mathsf{\theta }}^{\mathrm{c}}}+\mathrm{d}\mathsf{\alpha },\mathrm{c},\mathrm{d}>0,\mathrm{b}<0,\mathsf{\alpha }+\mathrm{d}<1,\mathrm{d}\ll \mathsf{\alpha }$$

(4)

The Resource Allocation Team (ST) consists of human agents responsible for distributing material resources in accordance with DMA’s directives. In addition to executing prescribed allocation plans, ST agents also continuously monitor ground conditions for unforeseen disruptions, including road damage and infrastructure failures. Upon identifying such obstacles, the ST immediately notifies the DMA, which can then revise routing instructions or reformulate the distribution plan. This real-time disruption detection and reporting capability represents a critical field intelligence function that the DMA alone cannot replicate. Figure 2 demonstrates how the three agents collaborate within the proposed model.

2.2. Markov Decision Process Formulation

The multi-agent coordination problem described above is formally modeled as a Markov Decision Process (MDP). The MDP formulation presented in this section is based on standard frameworks in the operations research and reinforcement learning literature [24]. The original contributions of this study lie in the problem-specific state space and decision variable design, the multi-agent coordination structure, and the survival-probability-based reward function, which are tailored to the post-disaster demand assessment and material allocation context.

2.2.1. State Variable

The system state ${\mathrm{S}}_{\mathrm{t}}$ = (${\mathrm{M}}_{\mathrm{t}}$,${\mathrm{D}}_{\mathrm{t}}$) $\in \mathrm{S}$ at decision epoch $t$ consists of two components: material status and zone request status. The material status component is defined as ${\mathrm{M}}_{\mathrm{t}}={\left({\mathrm{M}}_{\mathrm{t}\mathrm{k}}\right)}_{\mathrm{k}\mathsf{ϵ}\mathrm{K}}=\left({\mathrm{L}}^{\mathrm{t}},{\mathrm{R}}^{\mathrm{t}\mathrm{k}}\right)$, where each element ${\mathrm{M}}_{\mathrm{t}\mathrm{k}}$ characterizes supply point k, indicating which disaster zones can be served and the total materials available ${\mathrm{R}}^{\mathrm{t}\mathrm{k}}$. The ordered list ${\mathrm{L}}^{\mathrm{t}}$ specifies the sequence in which zones are to receive material allocation. The zone request status component is denoted ${\mathrm{D}}_{\mathrm{t}}={\left({\mathrm{D}}_{\mathrm{t}\mathrm{i}}\right)}_{\mathrm{i}\mathsf{ϵ}\mathrm{I}}$, where ${\mathrm{D}}_{\mathrm{t}\mathrm{i}}=\left({\mathrm{O}}^{\mathrm{t}\mathrm{i}},{\mathrm{e}}^{\mathrm{t}\mathrm{i}},{\mathrm{W}}^{\mathrm{t}\mathrm{i}}\right)$ encodes the geographic location, demand level, and service status of each zone. The demand level ${\mathrm{e}}^{\mathrm{t}\mathrm{i}}$ is treated as a random variable following a specified probability distribution $\mathcal{J}\left({\mathsf{\mu }}_{\mathrm{t}\mathrm{i}},{\mathsf{\sigma }}_{\mathrm{t}\mathrm{i}}\right)$ (e.g., normal or log-normal) that captures the inherent uncertainty in post-disaster needs [46,47]. The binary indicator ${\mathrm{W}}^{\mathrm{t}\mathrm{i}}=0$ denotes that zone $i$ is currently unserved. The complete state space S contains all feasible realizations of ${\mathrm{S}}_{\mathrm{t}}$.

2.2.2. Decision Variable

For each state $S_t\in S,$ the feasible action set $X_{S_t}$ defines all permissible allocation and routing decisions:

$$X_{S_t}=\left\{{\left(A_{ti}\right)}_{i\mathsf{ϵ}I},{\left(B_{ti}\right)}_{i\mathsf{ϵ}I}\right\}$$

(5)

The routing component $A_{ti}$= $({a_{tij}^z)}_{\left(i,j\right)\in z}\in \left\{0,1\right\}$ encodes whether the AT selects route z when moving from zone $i$ to zone $j$, while the allocation component $B_{ti}={\left(b_{ti}^k\right)}_{i\mathsf{ϵ}{L}^t}$ specifies the quantity of materials allocated from supply point k to disaster zone $i$, following the allocation sequence defined by ${\mathrm{L}}^{\mathrm{t}}$. The composite action is:

$${\mathrm{x}}_{\mathrm{t}}=\left(\right({a_{tij}^z)}_{\left(\mathrm{i},\mathrm{j}\right)\in \mathrm{z}},{\left({\mathrm{b}}_{\mathrm{t}\mathrm{i}}^{\mathrm{k}}\right)}_{\mathrm{i}\mathsf{ϵ}{\mathrm{L}}^{\mathrm{t}}})$$

(6)

All decisions must satisfy the following operational constraints:

$${\sum }_{i\mathsf{ϵ}I}{b}_{ti}^k\le R^{tk},\forall i\in I,i\in L^t\forall k\in K$$

(7)

$$b_{ti}^k\le e^{ti},\forall i\in I,\forall k\in K$$

(8)

$$a_{tij}^z\mathsf{ϵ}\left\{0,1\right\}\forall i,j\in I,\mathrm{z}=\left\{1,2,3\dots \right\}$$

(9)

$${\sum }_{j\mathsf{ϵ}I}{a}_{tij}^z\le 1\forall i\in I,\left(i,j\right)\in z$$

(10)

Equation (7) ensures that the total quantity of materials dispatched across all zones does not exceed the available supply at each depot. Equation (8) restricts material allocation to each zone to its assessed actual demand, preventing over-allocation. Equations (9) and (10) jointly enforce that each zone selects at most one route for movement to the next target zone, ensuring routing feasibility.

2.3. Markov Decision Process-Based Recursive Model

To maximize the total number of individuals rescued, the model integrates the objectives of all three agents into a unified global objective, defined as the product of the survival probability function and the material allocation quantity, rather than optimizing each agent’s objective independently. This formulation captures the joint effect of timely resource delivery and adequate supply on rescue outcomes.

The value function $\mathrm{V}{(\mathrm{S}}_{\mathrm{t}})$ denotes the maximum expected total discounted reward achievable from state ${\mathrm{S}}_{\mathrm{t}}$ under the optimal policy. A myopic baseline, which maximizes only the immediate reward $\mathrm{C}\left({\mathrm{S}}_{\mathrm{t}},{\mathrm{x}}_{\mathrm{t}}\right)$ at each decision epoch, provides a lower-performance benchmark; however, it disregards the long-run consequences of current decisions on future system states [39].

$$V_0\left(S_t\right)={max}_{x_t}C\left(S_t,x_t\right)$$

(11)

The myopic approach, however, is fundamentally limited because it disregards the impact of current decisions on future system states. Given the dynamic and resource-constrained nature of disaster response, routing and allocation decisions made at any epoch directly shape the system’s capacity to address demand in subsequent periods. The full dynamic formulation based on the Bellman optimality principle is [38,39]:

$$V(S_t)=\underset{x_t}{\max }\left(\mathrm{C}\left({(S}_t,x_{\mathrm{t}}\right)+\gamma {\sum }_{S_{t+1}\mathsf{ϵ}S}P\left(S_{t+1}|{(S}_t,x_{\mathrm{t}}\right)V{(S}_{t+1})\right)$$

(12)

where P is the one-step transition matrix encoding the stochastic evolution of demand and network conditions, and $\mathsf{\gamma }\in$[0,1] is the discount factor governing the relative weight assigned to delayed rewards. The contribution function $\mathrm{C}\left({\mathrm{S}}_{\mathrm{t}},{\mathrm{x}}_{\mathrm{t}}\right)$ aggregates rescue effects across all zones:

$$C\left(S_t,x_t\right)={\sum }_{i\in I}{o}_t^{i,x_t}$$

(13)

where $o_t^{i,x_t}=b_{\mathrm{ti}}^k{\delta \left({\theta }_{i,j}\right)}_{\begin{array}{c}i,j\in I\\ .\end{array}}$, quantifies the rescue effect at zone $i$ by combining the material allocation quantity with the time-dependent survival probability [45]. Let ${\mathrm{X}}^{\mathsf{\pi }}\left({\mathrm{S}}_{\mathrm{t}}\right)$ be a decision function (i.e., policy) that depicts allocation decisions for each state ${\mathrm{S}}_{\mathrm{t}}\in \mathrm{S}$. The optimization objective is to identify the policy ${\pi }^{*}$ that maximizes the expected total discounted contribution over a finite planning horizon T:

$$\underset{\mathsf{\pi }\mathsf{ϵ}\prod }{\max }{\mathrm{E}}^{\mathsf{\pi }}{\sum }_{\mathrm{t}=0}^T{\gamma }^{t}C\left(S_t,X^{\pi }\left(S_t\right)\right)$$

(14)

The state transition function $\phi$ describes the deterministic mapping from the current state-action pair $\left({\mathrm{S}}_{\mathrm{t}},{\mathrm{x}}_{\mathrm{t}}\right)$ to the subsequent system state, such that ${\mathrm{S}}_{\mathrm{t}+1}=\phi \left({\mathrm{S}}_{\mathrm{t}},{\mathrm{x}}_{\mathrm{t}}\right)$. The optimal policy ${\mathsf{\pi }}^{\mathrm{*}}$ satisfies the Bellman optimality equations [24,40] using Equations (15) and (16).

$$V\left(S_t\right)={max}_{x_t\mathsf{ϵ}{X}_{S_t}}\left(C\left(S_t,x_t\right)+\gamma E\left[V\left(S_{t+1}\right)\left|\right(S_t,x_t\right]\right)$$

(15)

$${\pi }^{*}\left(S_t\right)={argmax}_{x_t}\left[E\left(C\left(S_t,x_t\right)\right)+\gamma E\left[V^{*}\left(\phi \left(S_t,x_t\right)\right)\right]\right]$$

(16)

Obtaining ${\mathsf{\pi }}^{\mathrm{*}}$ from ${\mathrm{V}}^{\mathrm{*}}$ requires evaluating ${\mathrm{V}}^{\mathrm{*}}\left(\phi \left({\mathrm{S}}_{\mathrm{t}},{\mathrm{x}}_{\mathrm{t}}\right)\right)$ across all state-action pairs $\left({\mathrm{S}}_{\mathrm{t}},{\mathrm{x}}_{\mathrm{t}}\right)$. This is computationally intractable in practice due to the lack of complete information on state-action pairs. Therefore, we employ the Q-function as a tractable approximation for ${\mathrm{V}}^{\mathrm{*}}$, enabling the DMA to learn a near-optimal policy through iterative interaction with the simulated environment:

$$Q\left(S_t,x_t\right)=E\left[C\left(S_t,x_t\right)+\gamma \underset{x_{t+1}}{\max }Q\left(S_{t+1},x_{t+1}\right)\right]$$

(17)

Based on the above definitions of $o_t^{i,x_t},C\left(S_t,x_t\right)$ represents the total number of individuals rescued across all zones at epoch $t$, while the Q-value $Q\left(S_t,x_t\right)$ represents the expected discounted cumulative number of rescued individuals over the planning horizon, starting from a given state $S_t$ and action $x_t$ under the optimal policy.

The multi-agent decision coordination process is provided in Appendix A.

3. Solution Approach

Upon the onset of a disaster, the system transitions to the online response phase, during which the DMA receives the current environment state from the AT or ST and selects the most appropriate feasible action. The decision problem formulated in Equation (14) is an optimal control problem that aims to maximize the expected discounted cumulative contribution over the disaster-response horizon. In principle, such an optimal control problem can be solved using dynamic programming if the complete transition probability model and reward structure are explicitly known. However, in the proposed post-disaster resource allocation problem, the state-action space is large and high-dimensional, zone demands evolve stochastically, route accessibility changes dynamically, and transition probabilities are difficult to obtain in closed form [48]. Therefore, exact dynamic programming is computationally intractable.

To address the challenges posed by large, high-dimensional state-action spaces and to support stable and efficient learning, we adopt a Dueling Double Deep Q-Network with Prioritized Experience Replay (D3QN-PER) as the learning algorithm [48,49,50]. Unlike exact dynamic programming, deep reinforcement learning learns a parameterized policy directly from environmental interactions without requiring an explicit model of state transitions, making it well-suited to the sequential, uncertain nature of the problem [48]. It should be noted that this study does not employ external metaheuristic optimization methods, such as genetic algorithms, particle swarm optimization, differential evolution, or exact dynamic programming, to generate precomputed optimal or suboptimal decision labels for actions. Instead, training samples are generated as state-transition experiences during simulation, and the policy is improved through temporal-difference learning.

The standard D3QN-PER architecture is drawn from the deep reinforcement learning literature; its technical components are briefly described in Section 3.3 for completeness. The distinguishing features of this study lie in three problem-specific elements embedded within this framework: (i) the multi-agent field intelligence integration mechanism that updates the state representation in real time; (ii) the survival-probability-based reward function that couples material allocation with time-sensitive rescue outcomes; and (iii) the action masking mechanism that enforces operational feasibility constraints at every decision epoch.

3.1. Agent Collaboration in the Proposed Model

This subsection describes the problem-specific coordination mechanism, which represents a distinguishing methodological feature of the proposed framework. At each decision epoch $t$, the DMA maintains the environment state $S_t=\left(M_t,D_t\right)\in S$, where $M_t={\left\{\left(L^t,R^{tk}\right)\right\}}_{k\mathsf{ϵ}\mathrm{K}}$ denotes resource status and $D_t={\left\{\left(O^{ti},e^{ti},W^{ti}\right)\right\}}_{i\mathsf{ϵ}I}$ captures the request status of each zone $i$. Zone demand is modeled as a stochastic process $e^{ti}\sim \mathcal{J}\left({\mathsf{\mu }}_{ti},{\mathsf{\sigma }}_{ti}\right)$, reflecting the spatiotemporal uncertainty inherent in post-disaster conditions. During online operation, the AT furnishes the DMA with a field intelligence set ${\mathcal{l}}_t^{AT}$ = $\{\left({\tilde{e}}_{\mathit{ti}},{\tilde{\mathsf{\mu }}}_{\mathit{ti}},{\tilde{\mathsf{\sigma }}}_{\mathit{ti}}\right),{\tilde{\mathsf{\varrho }}}_{\mathit{ij}}\}$, where ${\tilde{e}}_{\mathit{ti}}$ is the demand information for zone $\mathrm{i},$ and ${\tilde{\mathsf{\varrho }}}_{\mathit{ij}}\in \left\{0,1\right\}$ indicates the operational feasibility of the route between zone $\mathrm{i}$ and $j$. The DMA integrates this field intelligence to construct the updated state ${\tilde{S}}_t=\mathcal{H}\left(S_t,{\mathcal{l}}_t^{DAT}\right)$, and subsequently makes decisions based on ${\tilde{S}}_t,$ thereby reducing allocation decision bias. The DMA selects action $x_t$ = $\left(A_{t,}{B}_t\right)$ so as to maximize the expected discounted total contribution in Equation (14). After the DMA outputs an action ($x_t\in X_{{\tilde{S}}_t}$) subject to Equations (5) and (6), the ST executes the prescribed allocation plan, realizing contribution $r_{t+1}=C({\tilde{S}}_t,x_t)$, updated resource levels $R^{k(t+1)}$, revised service-state indicators $W^{i(t+1)}$, and the subsequent environment state ${\mathrm{S}}_{\mathrm{t}+1}$ in which zone demands are resampled as $e^{(t+1)i}\sim \mathcal{J}\left({\mathsf{\mu }}_{(t+1)i},{\mathsf{\sigma }}_{(t+1)i}\right)$. The DMA then forms ${\tilde{S}}_{t+1}$ in a similar manner and stores the experience transition tuple (${\tilde{S}}_t,x_t{,r}_{t+1},{\tilde{S}}_{t+1}$) in the prioritized replay buffer for subsequent network training.

3.2. Feasibility Enforcement

Action masking is a problem-specific mechanism introduced to address the large set of infeasible actions arising from route disruptions and zone service constraints in disaster environments [51]. Unlike standard settings in previous studies, where all actions are typically assumed feasible, the highly constrained nature of disaster logistics requires explicit enforcement of feasibility at every decision epoch to prevent operationally invalid decisions. The feasible decision set ${\mathrm{X}}_{{\tilde{\mathrm{S}}}_{\mathrm{t}}}$ is constrained by Equations (7)–(10). Additional operational feasibility conditions are enforced during online dispatch: routes for which ${\tilde{\mathsf{\varrho }}}_{\mathit{ij}}$ = 0 are treated as physically infeasible and excluded from consideration; zones already under service (${\mathrm{W}}^{\mathrm{t}\mathrm{i}}$ = 1) are excluded from the candidate set to prevent duplicated resource deployment. To ensure that the DMA never outputs invalid actions, action masking is implemented by restricting decisions to the feasible action subset:

$$\mathcal{f}\left({\tilde{S}}_t\right)=\left\{x\in X_{{\tilde{S}}_t}|x\mathrm{s}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{s}\mathrm{f}\mathrm{i}\mathrm{e}\mathrm{s}\mathrm{E}\mathrm{q}\mathrm{u}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{s}(7)–(9)\mathrm{a}\mathrm{n}\mathrm{d}\mathrm{a}\mathrm{l}\mathrm{l}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{t}\mathrm{e},\mathrm{s}\mathrm{e}\mathrm{r}\mathrm{v}\mathrm{i}\mathrm{c}\mathrm{e}\mathrm{f}\mathrm{e}\mathrm{a}\mathrm{s}\mathrm{i}\mathrm{b}\mathrm{i}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{d}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{s}\right\}$$

(18)

The DMA is constrained to select actions exclusively from $\mathcal{f}\left({\tilde{\mathrm{S}}}_{\mathrm{t}}\right)$, guaranteeing operational validity at every decision epoch.

3.3. D3QN-PER: Neural Architecture

• Bellman optimality and Q-network approximation

The optimal action-value function satisfies the Bellman optimality equation:

$${\mathrm{Q}}^{\mathrm{*}}\left({\tilde{\mathrm{S}}}_{\mathrm{t}},{\mathrm{x}}_{\mathrm{t}}\right)=E\left[C\left({\tilde{S}}_t,x_t\right)+\gamma \underset{x_{t+1}}{\max }{\mathrm{Q}}^{\mathrm{*}}\left({\tilde{S}}_{t+1},{\mathrm{x}}_{\mathrm{t}+1}\right)\right]$$

(19)

The optimal action-value function ${\mathrm{Q}}^{\mathrm{*}}\left({\tilde{\mathrm{S}}}_{\mathrm{t}},{\mathrm{x}}_{\mathrm{t}}\right)$ is approximated by a parameterized neural network $Q\left({\tilde{S}}_t,x_t;\vartheta \right),$ where $\mathsf{\vartheta }$ denotes the set of trainable network weights and $Q\left({\tilde{S}}_t,x_t;\vartheta \right)\mathrm{r}\mathrm{e}\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{s}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{s}$ the value estimate produced by the current Q-network for state ${\tilde{\mathrm{S}}}_{\mathrm{t}}$ and action ${\mathrm{x}}_{\mathrm{t}}$, following the Deep Q-Network (DQN) paradigm of Mnih [50].

2.

Dueling architecture and Double DQN target

The dueling network architecture decomposes the Q-value into a state value term $\mathrm{V}\left({\tilde{\mathrm{S}}}_{\mathrm{t}},\vartheta \right)$ and an action-specific advantage term $\mathcal{g}\left({\tilde{\mathrm{S}}}_{\mathrm{t}},{\mathrm{x}}_{\mathrm{t}}\right)$, as formalized in Equation (20):

$$Q\left({\tilde{S}}_t,x_t;\vartheta \right)=V\left({\tilde{S}}_t;\vartheta \right)+\left(\mathcal{g}\left({\tilde{S}}_t,x_t;\vartheta \right)-{\displaystyle \frac{1}{\left|\mathcal{f}\left({\tilde{S}}_t\right)\right|}}{\sum }_{x^{\prime }}\mathcal{g}\left({\tilde{S}}_t,x^{\prime };\vartheta \right)\right)$$

(20)

Standard Q-learning suffers from systematic overestimation due to the maximization operator. Double DQN alleviates this bias by decoupling action selection from action evaluation when forming the TD target. Specifically, the next action is selected using the online network parameters $\mathsf{\vartheta }$, while its value is evaluated using the target network parameters ${\mathsf{\vartheta }}^{-}$. The resulting TD target is:

$$y_t=r_{t+1}+\gamma Q\left({\tilde{S}}_{t+1},arg\underset{x^{\prime }\in \mathcal{f}\left({\tilde{S}}_t\right)}{\max }Q\left({\tilde{S}}_{t+1},x^{\prime };\mathsf{\vartheta }\right);{\mathsf{\vartheta }}^{-}\right),$$

(21)

where $y_t$ is the TD target used to update the Q-value estimate for (${\tilde{\mathrm{S}}}_{\mathrm{t}},{\mathrm{x}}_{\mathrm{t}}$), and the target network parameters ${\mathsf{\vartheta }}^{-}$ are periodically synchronized with the online network to provide stable value estimates throughout training.

3.

Prioritized Experience Replay (PER)

PER is a prioritized sampling mechanism that improves the efficiency and stability of Q-network training by replaying transitions with larger temporal-difference errors more frequently.

In standard experience replay, transitions sampled uniformly from the experience replay buffer (ERB) may contain limited learning signal, thereby reducing sample efficiency. Prioritized Experience Replay (PER) addresses this issue by assigning each stored transition a sampling probability proportional to the magnitude of its TD error [52], ensuring that transitions with greater informational content receive disproportionately higher training attention [48]. For the $\mathcal{K}$-th transition stored in the ERB, we define the TD error as:

$${\wp }_{\mathcal{K}}=y_{\mathcal{K}}-Q\left({\tilde{S}}_{\mathcal{K}},x_{\mathcal{K}};\vartheta \right)$$

(22)

where ${\wp }_{\mathcal{K}}$ denotes TD errors for transition $\mathcal{K}$. The TD error quantifies the discrepancy between the Q-value estimates produced by the online (prediction) and target networks, and directly determines the priority assigned to each stored transition. The priority score for transition $\mathcal{K}$ is defined as

$${\psi }_{\mathcal{K}}=\left|{\wp }_{\mathcal{K}}\right|+Ⅎ,$$

(23)

where $\mathrm{Ⅎ}>0$ is a small constant ensuring that every transition retains a non-zero sampling probability. Based on these priority scores, the probability of sampling the $\mathcal{K}$-th transition is:

$$\Lambda \left(\mathcal{K}\right)={\displaystyle \frac{{\psi }_{\mathcal{K}}^{\mathcal{A}}}{{\sum }_{\mathcal{K}}{\psi }_{\mathcal{K}}^{\mathcal{A}}}}$$

(24)

where $\mathcal{A}\in \left[0,1\right]$ governs the degree of prioritization ($\mathcal{A}$ = 0 recovers uniform sampling). Since non-uniform sampling introduces bias relative to the uniform replay distribution, PER applies an importance-sampling (IS) correction weight to each sampled transition [53]:

$${\Phi }_{\mathcal{K}}={\left({\displaystyle \frac{1}{\mathrm{N}\times \Psi \left(\mathcal{K}\right)}}\right)}^{\beta }$$

(25)

where $\mathsf{\beta }\in \left[0,1\right]$ controls the strength of the IS correction and is typically annealed toward 1 during training, and N denotes the current number of transitions stored in the ERB. The PER-weighted loss function for Q-network training is given by

$$\mathcal{L}\left(\vartheta \right)=E\left[{\Phi }_{\mathcal{K}}\left(y_{\mathcal{K}}-{Q\left({\tilde{S}}_{\mathcal{K}},x_{\mathcal{K}};\vartheta \right)}^2\right)\right]$$

(26)

where $\vartheta$ is updated via stochastic gradient descent on this weighted loss. The target network is periodically synchronized with the online Q-network by copying ${\mathsf{\vartheta }}^{-}\leftarrow \vartheta$. The complete D3QN-PER training procedure is presented in Algorithm 1.

Algorithm 1: D3QN-PER Training Procedure

Input: discount factor $\mathsf{\gamma }$, replay buffer capacity N, mini-batch size B,     learning rate, target update interval, feasibility masking     rules (Equations (7)–(10), served-zone exclusion rule, route feasibility conditions), PER parameters ($\mathcal{A}$, $\beta$). Output: learned DMA policy $\pi \left(\tilde{S}\right)=$ arg $\underset{x\in A\left(\tilde{S}\right)}{\max }Q\left(\tilde{S},x;\mathsf{\vartheta }\right)$. 1.  Initialize online dueling Q-network $Q\left(\tilde{S},x;\mathsf{\vartheta }\right)$ and target network $Q\left(\tilde{S},x;{\mathsf{\vartheta }}^{-}\right)\leftarrow$   $Q\left(\tilde{S},x;\mathsf{\vartheta }\right)$. 2.  Initialize prioritized replay buffer $\mathrm{\varkappa }$ with capacity N. 3.  For each training episode (disaster scenario) do 4.   Initialize state $S_0=\left(M_0,D_0\right)$, initialize state ${\tilde{S}}_0$, where $e^{0i}$ $\sim \mathcal{J}\left({\mathsf{\mu }}_{0i},{\mathsf{\sigma }}_{0i}\right)$. 5.   for t = 0, 1, …, T − 1 do 6.    AT provides field intelligence ${\mathcal{l}}_t^{DAT}$ = $\{\left({\tilde{e}}_{\mathit{ti}},{\tilde{\mathsf{\mu }}}_{\mathit{ti}},{\tilde{\mathsf{\sigma }}}_{\mathit{ti}}\right),{\tilde{\mathsf{\varrho }}}_{\mathit{ij}}\}$;     The DMA updates the state ${\tilde{S}}_t=\mathcal{H}\left(S_t,{\mathcal{l}}_t^{DAT}\right)$. 7.   Construct the feasible set $\mathcal{f}\left({\tilde{S}}_t\right)$ via constraints (7)–(10) and $W^{ti}$. 8.   Select action $x_t$: with probability $ϵ$, sample $x_t$ uniformly from $\mathcal{f}\left({\tilde{S}}_t\right)$; 9.   Otherwise, $x_t=$ arg $\underset{x\in A\left(\tilde{S}\right)}{\max }Q\left(\tilde{S},x;\mathsf{\vartheta }\right)$. 10.   The ST executes the prescribed allocation; the AT executes routing if applicable; compute reward $r_{t+1}=C\left({\tilde{S}}_t,x_t\right)$ and next state $S_{t+1}$; 11.   Construct the next augmented state ${\tilde{S}}_{t+1}=\mathcal{H}\left(S_{t+1},{\mathcal{l}}_{t+1}^{DAT}\right)$ by updating $R^{k(t+1)}$, $W^{i(t+1)}$ and sampling new demands ${\mathrm{e}}^{(\mathrm{t}+1)\mathrm{i}}\sim \mathrm{J}\left({\mathsf{\mu }}_{(\mathrm{t}+1)\mathrm{i}},{\mathsf{\sigma }}_{(\mathrm{t}+1)\mathrm{i}}\right)$. 12.   Store transition (${\tilde{S}}_t,x_t$,$r_{t+1}$,${\tilde{S}}_{t+1}$) in $\mathrm{\varkappa }$ with maximum priority $\psi$. 13.   Sample mini-batch of K transitions from $\mathrm{\varkappa }$ using $\Lambda \left(\mathcal{K}\right)$. 14.    For each sampled transition K do 15.    Compute TD target $y_{\mathcal{K}}$ using Equation (21). 16.    Compute TD error and update priority:     ${\wp }_{\mathcal{K}}=y_{\mathcal{K}}-Q\left({\tilde{S}}_{\mathcal{K}},x_{\mathcal{K}};\vartheta \right)$; ${\psi }_{\mathcal{K}}$=|${\wp }_{\mathcal{K}}$|+$Ⅎ$. 17.    Compute IS weight ${\Phi }_{\mathcal{K}}$ using Equation (25). 18.    Update ϑ by minimizing the loss function     $\mathcal{L}\left(\vartheta \right)=E\left[{\Phi }_{\mathcal{K}}\left(y_{\mathcal{K}}-{Q\left({\tilde{S}}_{\mathcal{K}},x_{\mathcal{K}};\vartheta \right)}^2\right)\right]$. 19.    At every target update interval $\tau$, synchronize the target network ${\mathsf{\vartheta }}^{-}\leftarrow$ $\vartheta$. 20.  end for 21. end for 22. Return the learned policy $\pi \left(\tilde{S}\right)=$ arg $\underset{x\in A\left(\tilde{S}\right)}{\max }Q\left(\tilde{S},x;\mathsf{\vartheta }\right)$.

4. Case Study

This study employs the M6.8 earthquake that struck Dingri County, Shigatse City, Tibet Autonomous Region on 7 January 2025, as an empirical test case to assess the applicability and practical relevance of the proposed framework. According to the China Earthquake Networks Center (CENC), the epicenter was located at 87.45° E, 28.50° N, at a focal depth of approximately 10 km. As of 14 January 2025, a total of 3614 aftershocks had been recorded, with the largest reaching magnitude 5.0. According to official statistics released by the Ministry of Emergency Management of the People’s Republic of China and the Tibet Autonomous Region Emergency Management Department, the earthquake caused direct economic losses of RMB 34.884 billion, affected approximately 300,500 people, injured 860 individuals, and caused the collapse of 2865 houses and damage to an additional 38,806 structures. In response, a national Level II emergency protocol was activated; the Tibet Autonomous Region elevated its response to Level II; and a field command headquarters was established to coordinate rescue operations. Although the response involved a broad coalition of actors, including government agencies, the People’s Liberation Army, non-governmental organizations, and community volunteers, relief supply allocation remained largely fragmented across organizations. Such fragmentation can reduce inter-agency coordination efficiency and overall response effectiveness [49,54]. Therefore, this case provides a highly relevant empirical context for investigating how structured demand assessment can improve allocation decisions under conditions of high spatiotemporal uncertainty, heterogeneous demand profiles, and complex logistics networks.

4.1. Study Area and Data Settings

The study area comprises twelve townships with seismic intensities of VII (Modified Mercalli Scale) or above, namely Quluo Township (QLT), Changsuo Township (CT), Cuoguo Township (TT), Nixia Township (NT), Jiacuo Township (GT), Qudang Township (QDT), Xiegeer Town (XGT), Zhaxizong Township (ZT), Mangpu Township (MT), Lazi Town (LT), Quxia Town (QT), and Xiyin Township (XT), as shown in Figure 3. The failure probabilities of transportation links in the affected zones are estimated primarily using route-specific parameters, including link length and the presence of critical infrastructure elements, such as bridges, following Nadi et al. [44]. Complete zone-level information is provided in Appendix A and Appendix B. Inter-zone travel times were obtained from Baidu Maps. Regarding model parameterization, service performance factors were set to 0.3 for assessment teams and 0.6 for relief supply teams [44]. The survival function is given by $\delta \left(\theta \right)=0.8e^{-0.0001{\theta }^2},$ where $\theta$ denotes the rescue time in hours [45]. The discount factor is set to γ = 0.98 to account for discounted future rewards in sequential decision-making. Following established modeling conventions [47,55,56], the injured population at each zone is assumed to follow a normal distribution with the nominal demand as its mean and a standard deviation of 3. The distribution function is used to generate real-time estimates of the injured population, which is also used to estimate rewards. Material demand at zone $i$ is calculated as ${DS}_i=N_{ir}\times d\times \mathfrak{I}\times \left(1+\aleph \right)$, where $\mathrm{d}$ is per-capita demand, $\mathfrak{I}$ is the seasonal factor, and $\aleph$ is the loss factor. Following Lin et al. [57] and considering the 2025 average low temperature of −11 °C in Shigatse, Tibet, and its high-altitude conditions, the food demand parameters are set as $\mathrm{d}$ = 1 kg, $\mathfrak{I}$ = 1.2, $\aleph$ = 0.05. All algorithm hyperparameters are configured in accordance with Zeng et al. [49] and Mnih et al. [50], as detailed in Appendix C. The model is solved using the framework described in Section 3. All algorithms were implemented using Python libraries.

4.2. Results and Analysis

The single-agent-per-category configuration used in this subsection is intentionally simplified to isolate and quantify the contribution of the AT agent in a controlled and tractable setting. Section 4.2.2 then extends the analysis to a twelve-zone, multi-agent configuration to examine system-level scalability and coordination effects under more realistic operational conditions.

4.2.1. Performance Analysis and Contribution of the AT Agent

To isolate and quantify the contribution of the AT agent, the initial analysis focuses on six severely affected zones, namely QLT, CT, QDT, XGT, MT, and LT, which are selected as the primary response targets. These zones are located within or adjacent to the highest-intensity region (seismic intensity ≥ VIII on the Modified Mercalli Scale) and collectively account for the majority of the affected population and collapsed structures in the study area. The transportation network connecting these zones comprises 14 road links (Figure 4). For analytical clarity, a single representative agent is assumed for each agent category. The D3QN-PER algorithm is applied to train the DMA, yielding the optimal assessment route illustrated in Figure 5. The resulting agent properties and operational timelines for the AT and ST are presented in

Table 2. Properties of AT and ST agents.

Zone	${\mathsf{\epsilon }}_{\mathbf{i}}^{\mathbf{A}\mathbf{T}}$ (h)	${\mathsf{\epsilon }}_{\mathbf{i}}^{\mathbf{S}\mathbf{T}}$ (h)	Route ($\mathit{i}$→$\mathit{j}$)	${\mathit{T}}_{\mathit{i},\mathit{j}}$ (h)	${\mathbf{F}\mathbf{T}}_{\mathbf{i}}^{\mathbf{A}\mathbf{T}}$ (h)	${\mathbf{F}\mathbf{T}}_{\mathbf{i}}^{\mathbf{S}\mathbf{T}}$ (h)
QT	0.9775	2.6067	QLT → CT	0.5500	0.9775	2.6067
CT	1.1250	3.0000	CT → XGT	0.9667	2.6525	6.1567
XGT	1.0875	2.9000	XGT → MT	1.9333	4.7067	10.0234
MT	3.00	8.0000	MT → LT	0.6667	9.6400	19.9567
LT	1.0875	2.9000	LT → QDT	4.0500	11.3942	23.5234
QDT	2.7624	7.3664	QDT → QDT	0.00	18.2066	34.9398

. To examine the system’s responsiveness to field disruptions, a link failure event is introduced at t = 4.7067 h. After completing service at zone XGT, the AT agent reports that Link 10 has become impassable (Link 10 is selected as the failure location because it represents a critical connectivity segment in the six-zone network, lying on the primary route connecting XGT to adjacent zones; its disruption forces a non-trivial re-routing decision that meaningfully tests the DMA’s adaptive replanning capability). At that moment, the ST agent has just commenced service at XGT. The DMA is required to identify an alternative optimal routing plan based on the updated network state. Under this configuration, two scenarios are evaluated: (Scenario 1) full coordination, in which AT, ST, and DMAs are all active; and (Scenario 2) assessment-absent, in which the AT agent is removed and only the ST and DMA operate.

Q-value convergence. Both scenarios were trained using D3QN-PER, and the resulting Q-value convergence curves (measured in number of people rescued) are plotted over training episodes in Figure 6. After approximately 1200 training episodes, the Q-values of both scenarios stabilized within the range of 1800 to 2500. Scenario 1 (full coordination) converged to approximately 2300, whereas Scenario 2 (assessment-absent) stabilized at approximately 2010. During the first 300 episodes, Scenario 1 exhibited a markedly faster learning rate, reflecting the informational advantage conferred by the AT agent’s real-time field data. Beyond 300 episodes, Q-value fluctuations gradually diminished in both scenarios, indicating convergence toward stable allocation strategies. A detailed zone-level comparison of optimal actions, demand satisfaction rates, and Q-values is provided in

Table 3. Zone performance comparison between Scenario 1 (full coordination) and Scenario 2 (assessment-absent) using D3QN-PER.

Zone	Scenario 1 (with AT)			Scenario 2 (Without AT)
	AT Route	ST Satisfaction (%)	Q-Value	AT Route	ST Satisfaction (%)	Q-Value
QLT	7	70	343	—	63	235
CT	4	80	297	—	80	247
XGT	9	60	339	—	36	311
MT	1	40	754	—	40	298
LT	11	74	337	—	58	703
QDT	5	64	229	—	73	267

. Table 3 further shows that the number of rescued individuals in Scenario 1 is higher than that in Scenario 2.

Robustness evaluation. To assess the robustness of the DMA decision-making in a stochastic environment, 100 independent simulation runs were conducted for each scenario. Performance was evaluated using four metrics: failure rate (failure occurs when an agent encounters a destroyed link), mission completion time, number of rescued individuals, and the mean absolute percentage error (MAPE) of Q-values. The integration of AT information reduced the probability of the ST agent encountering an impassable link from 9.50% (Scenario 2) to 1.56% (Scenario 1) (Appendix D, Figure A2), confirming that proactive route assessment substantially reduces operational disruptions. Real-time demand information from the AT also improved Q-value accuracy, with MAPE decreasing from 9.74% to 0.96% (Appendix D, Figure A3), indicating substantially improved state representation and consequent action quality. These gains collectively reduced link failures, shortened the average mission completion time from 42.50 h to 35.60 h (Appendix D, Figure A4), and yielded an 11.34% increase in the rescued population in Scenario 1 relative to Scenario 2 (Appendix D, Figure A5), which is consistent with the zone-level findings in Table 3.

4.2.2. Performance Analysis of Multi-Agent Coordination

This section extends the analysis to all 12 affected zones to evaluate the scalability and generalizability of the proposed framework under more realistic multi-zone operational conditions. Specifically, the impact of multi-agent coordination is evaluated across these 12 zones. Following Yang et al. [43], four AT agents and four ST agents are deployed. Two scenarios are compared: DR1 (full coordination, all agents active) and DR2 (assessment-absent, all AT agents removed). Comparative results are summarized in Figure 6 and

Table 4. Comparative performance of four methods under full-coordination (DR1) and assessment-absent (DR2) scenarios.

Scenario	Method	No. of People Rescued	Completion Time (h)
DR1	D3QN-PER	6173.98	64.69
	DQN	5500.01	66.03
	Q-learning	5290.14	67.00
	Myopic policy	4874.20	64.09
DR2	D3QN-PER	5524.09	67.49
	DQN	5300.14	68.67
	Q-learning	5199.14	70.45
	Myopic policy	4549.25	72.60

.

DR1 consistently outperforms DR2 in both rescued population and mission completion time across all tested methods, confirming that AT deployment and the associated real-time information sharing generate meaningful and consistent system-level performance gains. The proposed D3QN-PER is further benchmarked against three established baselines: DQN [49], Q-learning [58], and a myopic policy [39,41]. Under DR1, D3QN-PER achieves the highest rescued population of 6173.98, followed by DQN (5500.01), Q-learning (5290.14), and the myopic policy (4874.20). D3QN-PER also attains its maximum rescued population in the shortest time among the learning-based methods (64.69 h). Notably, while the myopic policy reaches its maximum marginally faster (64.09 h), it produces the lowest rescued population of all methods. This finding reflects the inherent limitation of short-horizon greedy decision-making in complex, multi-zone disaster response: optimizing immediate reward without accounting for future consequences ultimately sacrifices substantial rescue capacity.

Overall, DR1 outperforms DR2 in both rescued population and completion time, confirming the benefit of deploying multiple AT agents and supporting the mechanism identified in Table 3 (Section 4.2.1).

4.2.3. Solution Stability Analysis

To assess solution stability and robustness, 20 independent training runs were performed for each method (D3QN-PER, DQN, Q-learning, and myopic policy) under scenarios DR1 and DR2 (Section 4.2.2). Stability was quantified by the coefficient of variation ($\mathrm{CV}=\mathsf{\sigma }/\mathsf{\mu }$), where σ and μ are the standard deviation and mean of rescued individuals across runs; a lower CV denotes greater consistency. Results are shown in Figure 7 and

Table 5. Solution statistics of rescued population comparison across different methods under DR1 and DR2 scenarios.

Scenario	Method	Mean	Std Dev	CV (%)	Min	Max	IQR
DR1	D3QN-PER	6233	143	2.3	5924	6483	172
	DQN	5557	283	5.1	5070	5981	476
	Q-learning	5284	256	4.8	4619	5632	339
	Myopic policy	4973	447	9.0	4132	5701	627
DR2	D3QN-PER	5592	165	3.0	5235	5881	199
	DQN	5361	304	5.7	4839	5816	511
	Q-learning	5192	277	5.3	4471	5570	368
	Myopic policy	4658	489	10.5	3737	5454	686

. D3QN-PER achieved the highest mean number of rescued individuals and the lowest CV values in both scenarios (DR1: 6233, CV = 2.3%; DR2: 5592, CV = 3.0%). Strikingly, its worst-case result under DR1 (5924) still surpassed the mean of DQN (5557), confirming consistent superiority even under adverse runs. The myopic policy showed the poorest stability (DR1: 9.0%; DR2: 10.5%), reflecting its inherent sensitivity to stochastic environmental fluctuations, while DQN and Q-learning exhibited moderate CV values of 4.8–5.7%. The stability of D3QN-PER stems from two structural mechanisms: periodic target network synchronization [50], which prevents oscillation of training targets, and importance sampling correction [53], which eliminates gradient bias induced by PER’s non-uniform sampling.

4.2.4. Sensitivity Analysis

This section systematically examines how variations in the numbers of AT and ST agents affect overall rescue performance. The total affected population is fixed at 6499, with a baseline of 3249 rescued individuals established under the condition of zero deployed ST agents. The numbers of AT and ST agents are varied independently while all other parameters are held constant. As illustrated in Figure 8, increasing the total number of deployed agents generally improves the rescued population. However, the marginal benefit of additional AT agents exhibits clear diminishing returns: once the AT count exceeds five, the rescued population reaches a plateau and further AT deployment yields negligible incremental improvement. This finding has a direct practical implication: relief organizations should prioritize balanced AT and ST configurations over unilateral increases in assessment capacity, particularly when operating under tight resource and time constraints. From a sustainable development perspective, this finding is particularly important. Deploying too many assessment teams may consume limited human, logistical, and organizational resources, without necessarily producing a corresponding improvement in rescue effectiveness.

Table 6. Rescue routing paths under different AT sizes (ST = 4).

AT	Routing Paths (Disaster Zones Sequences)
AT = 3	AT 1: ZT-LT-MT-XT; AT 2: XGT-CT-TT-QDT; AT 3: QLT-GT-QT-NT
AT = 4	AT 1: QLT-TT-NT; AT 2: XGT-CT-QDT; AT 3: GT-LT-QT; AT 4: ZT-MT-XT
AT = 5	AT 1: QLT-GT-QT; AT 2: TT-NT; AT 3: XGT-LT-XT; AT 4: ZT-MT; AT 5: CT-QDT

Note: Bold and underlined zone names indicate zones for which service is initiated simultaneously; the remaining zones are served sequentially.

presents the AT routing sequences under the reference configuration of ST = 4 as the number of AT agents varies from 3 to 5. Zones for which service is initiated simultaneously are indicated in the original analysis using bold and underlined formatting, whereas the remaining zones are served sequentially. Importantly, the service priority order of the three key affected zones, QLT, XGT, and ZT, remains unchanged across all AT configurations. This stability provides strong evidence that the proposed framework maintains consistent prioritization of key zones even as team compositions vary, demonstrating structural robustness to moderate staffing fluctuations.

Figure 9 presents the zone-level demand satisfaction rates for ST = 4 under different AT levels. The aggregate satisfaction rate is maximized at AT = 4, and the improvement from AT = 3 to AT = 4 substantially exceeds the marginal gain from AT = 4 to AT = 5, further confirming the existence of diminishing returns beyond an optimal AT threshold. Together, the results in Table 6 and Figure 9 show that, although AT staffing influences routing schedules and zone-level satisfaction rates, the fundamental allocation structure and the prioritization of critical zones remain stable. This provides evidence of the operational robustness of the proposed framework under realistic staffing variations.

5. Conclusions

Timely and accurate demand assessment is a fundamental prerequisite for effective and sustainable post-disaster resource allocation. In practice, however, demand information is frequently incomplete, dynamically evolving, and subject to significant noise, posing severe challenges to reliable and timely decision-making. From a sustainability perspective, such information uncertainty may lead not only to inefficient logistics operations, but also to avoidable resource waste and delayed assistance to vulnerable populations. To address this problem, this study proposes a multi-agent decision framework that integrates on-site demand assessment with dynamic resource allocation, thereby overcoming the efficiency bottleneck associated with information delays in conventional stage-wise response approaches. The framework formalizes the post-disaster response process as a Markov Decision Process and enables dynamic coordination among a Demand Assessment Team (AT), a Resource Allocation Team (ST), and a central artificial Decision-Making Agent (DMA), supporting adaptive, field-feedback-driven resource allocation. Three principal findings are derived from an empirical evaluation based on the 2025 Dingri County earthquake in Tibet.

First, the integration of real-time demand assessment substantially improves the quality of individual allocation decisions and overall system-level rescue performance. Incorporating on-site field intelligence from the AT reduces decision bias in the DMA’s state representations, improves the quality of ST allocation decisions, and shortens the time required to rescue a given proportion of the affected population. In multi-zone settings, this mechanism results in larger rescued populations, shorter mission completion times, and higher demand satisfaction rates, collectively highlighting the strategic importance of structured, field-feedback-driven information flow in emergency response systems.

Second, expanding team size generally improves rescue outcomes, but AT capacity exhibits pronounced diminishing marginal returns. The total rescued population increases as the numbers of both AT and ST agents grow; however, the marginal gains from adding AT agents plateau once a threshold is exceeded. In operational terms, this finding implies that relief organizations should seek balanced AT and ST configurations rather than indefinitely expanding assessment capacity, particularly under binding time and resource constraints.

Third, increasing AT capacity alone yields only marginal improvements in system-wide demand satisfaction, while substantial disparities in inter-zone satisfaction rates persist. This finding underscores the necessity of incorporating equity objectives explicitly into emergency response planning: frameworks that optimize solely for aggregate performance metrics may systematically under-serve the most vulnerable zones. Response planners should therefore adopt multi-objective approaches that balance system-level efficiency with equitable resource distribution across all affected areas.

From an algorithmic perspective, the proposed D3QN-PER outperforms all baseline methods in both solution quality and stability. In the DR1 scenario, D3QN-PER yields the highest rescue count of 6173.98, exceeding DQN (5500.01), Q-learning (5290.14), and the myopic policy (4874.20) by 12.3%, 16.7%, and 26.7%, respectively. This performance advantage also persists in the DR2 scenario, indicating that the superiority of D3QN-PER holds across varying scenario configurations. Stability analysis further corroborates these findings: D3QN-PER achieves the lowest coefficient of variation among all compared methods (CV = 2.3% under DR1), confirming that the stochastic nature of its prioritized experience replay mechanism does not compromise solution reliability. Therefore, these results further demonstrate the consistent advantages of D3QN-PER in both solution quality and robustness.

Overall, the findings show that the proposed framework has important implications for sustainability in disaster response. By increasing the number of rescued individuals and improving demand satisfaction across affected zones, the framework supports the social dimension of sustainability. It also improves resource-use efficiency by reducing allocation bias, the selection of failed routes, and redundant team deployment. In addition, the continuous feedback between field assessment and allocation decisions enhances the adaptability and resilience of the emergency response system.

The framework has also been designed with practical deployment in mind, and its resource requirements merit consideration. While the precise figures will depend on local administrative arrangements and the maturity of existing emergency management systems, the 2025 Dingri response—which covered twelve affected townships under a Level II national emergency protocol—suggests that approximately twenty trained personnel would be sufficient at the provincial level. Such a team would comprise a core of operations research analysts overseeing decision-support processes, four to six system and data engineers maintaining the geospatial and real-time information flow, and field coordinators supporting rescue teams in the affected zones. Compared with the RMB 34.884 billion in direct economic losses recorded during the Dingri earthquake, the operational cost of deploying the framework is almost negligible, reinforcing its role as a complementary tool rather than a substitute for existing disaster response systems.

This study has several limitations that point to promising directions for future research. First, the scope of the decision problem could be broadened to encompass a wider range of interdependent emergency response tasks, including search-and-rescue, medical triage, and mass evacuation, thereby extending the current single-phase allocation model into a comprehensive multi-task emergency management framework. Second, the practical applicability of the model could be enhanced by incorporating additional real-world features such as zone-level urgency prioritization, explicit equity constraints, and more realistic operational allocation rules. Third, algorithmic scalability and computational efficiency remain important open challenges. Future research could develop more efficient deep reinforcement learning architectures specifically tailored to disaster response decision problems, and explore the integration of large-scale heterogeneous data sources, including satellite imagery, population mobility data, and telecommunications signals, to improve the robustness and cross-context generalizability of the multi-agent framework. Fourth, while D3QN-PER has been benchmarked against DQN, Q-learning, and the myopic policy—confirming its advantages in solution quality (DR1: 6173.98 rescued individuals) and stability (CV = 2.3%)—direct comparisons with hybrid variants that integrate D3QN with external optimization techniques (i.e., D3QN-PSO, D3QN-DE, and D3QN-GA) were not conducted in this study. Although combinations of DDQN with PSO, DE, and GA have been explored in related domains, the systematic integration of these optimization techniques with the full D3QN architecture remains largely unexamined in the literature. Therefore, this comparison represents a meaningful direction for future research. Fifth, the case study integrates real-world data, including inter-site travel times and local meteorological conditions, with scenario-specific parameter assumptions such as service performance factors and demand distribution settings. The results are, therefore, best interpreted as scenario-based analyses, which is consistent with established practice in disaster response modeling [59]. However, the sensitivity of these parameters has not been formally quantified [60]. Systematic sensitivity analysis represents a natural extension of this work, particularly for examining how input uncertainty propagates through allocation decisions and ultimately affects rescue outcomes.