A Value-Driven Multi-Agent Reinforcement Learning Framework for Decentralized Adaptive Energy Management in Prosumer Smart Grids

Abstract

Prosumer communities, aggregations of residential and commercial entities equipped with distributed energy resources (DER), including photovoltaic systems, battery storage, and flexible loads, are emerging as critical organizational units in decarbonising smart grid architectures. Managing these communities effectively requires balancing economic efficiency with equity, autonomy, and environmental sustainability, objectives that conventional centralized control methods and existing multi-agent reinforcement learning (MARL) implementations fail to address simultaneously. This article proposes a value-aligned hierarchical multi-agent reinforcement learning (VA-HMARL) framework as a formally unified architecture that embeds equity (Jain’s Fairness Index J ≥ 0.90), individual autonomy, and carbon sustainability as hard constraints within the MARL reward structure. The framework integrates: a multi-objective Value Alignment Module (VAM) combining economic, fairness, sustainability, and comfort objectives; attention-based implicit coordination for scalable agent interaction; and differentially private federated policy aggregation (ε = 1.0, δ = 10⁻⁵) for GDPR-compliant collaborative learning. Simulation on a 20-prosumer community modelled on the IEEE 33-bus feeder over 10 Monte Carlo runs (300 episodes each) demonstrates: a 6.2% energy cost reduction versus the Rule-Based baseline (p = 0.0004); a Jain’s Fairness Index of 0.912 ± 0.031 at policy convergence (final 50 episodes), satisfying the J ≥ 0.90 community equity floor; and an 18.0% reduction in CO₂ emissions. The economic efficiency trade-off relative to performance-optimized MARL baselines is limited to 2.4%, within the 5% design target. These results establish VA-HMARL as a technically feasible and ethically grounded paradigm for autonomous decentralized energy governance.

1. Introduction

The global electricity system is undergoing an unprecedented structural transformation. Fossil fuels still account for approximately two-thirds of global electricity generation, while renewables contribute a rapidly growing but still limited share of around 33% [1]. Decarbonisation policy mandates, the rapid cost reduction in solar photovoltaic and wind technologies, and binding net-zero commitments across jurisdictions are accelerating the displacement of centralized thermal generation toward distributed renewable architectures. Key developments driving this transition include exponential growth in utility-scale and rooftop solar, increasingly ambitious policy-driven renewable penetration targets, and the role of advanced control strategies, including artificial intelligence and multi-agent coordination, in supporting grid stability as variable generation displaces dispatchable capacity [2]. At the same time, intermittency, grid integration complexity, limited affordable storage, and system reliability under high renewable penetration represent well-documented engineering challenges that directly motivate the research presented in this paper.

The global energy transition is characterized by three converging systemic imperatives: the rapid proliferation of distributed renewable generation capacity, the accelerating electrification of transport and heating sectors, and binding policy mandates toward carbon neutrality across jurisdictions. These drivers fundamentally disrupt conventional centralized power generation and distribution architectures, displacing passive consumers toward active market participants capable of generating, storing, and trading energy at local scales. Within this context, prosumer communities, aggregations of residential and commercial entities coupled with heterogeneous distributed energy resources (DER), including photovoltaic installations, battery energy storage systems (BESS), electric vehicles, and flexible demand, have emerged as critical organizational units for balancing local supply-demand dynamics while reducing transmission losses and enhancing grid resilience.

The prosumers’ engagement has progressed from peripheral participation under net-metering frameworks to increasingly complex portfolio management functions. Nowadays, prosumers operate as autonomous decision-makers navigating complex trade-offs among self-consumption optimization, peer-to-peer energy commerce, demand flexibility participation, and community-level resource sharing. However, this decentralization introduces substantial coordination challenges that conventional centralized control architectures fundamentally cannot address. Traditional approaches, including model predictive control (MPC) methodologies, rule-based heuristics, and optimization-based dispatch algorithms, proved critical vulnerabilities: computational scalability degrades nonlinearly with prosumer population growth, single-point failure in central coordinators compromises system reliability, centralized data aggregation affects privacy commitments and regulatory expectations, and rigid policy structures fail to accommodate the behavioural heterogeneity and dynamic preferences characteristic of real communities.

Multi-agent reinforcement learning (MARL) provides a computationally grounded paradigm naturally suited to these decentralized, heterogeneous environments. By decomposing the collective energy management problem into autonomous learning agents operating under partial observability, MARL algorithms, including variants such as multi-agent deep deterministic policy gradient (MADDPG), mixing networks (QMIX), and multi-agent proximal policy optimization (MAPPO), enable distributed policy adaptation without requiring global state information or centralized coordination infrastructure. Agents iteratively refine decision policies through interaction with the energy environment, learning emergent cooperative-competitive behaviours that respond to non-stationary conditions and uncertain peer actions. Recent empirical evaluations demonstrate MARL superiority over classical control methods in microgrid and community-scale applications, suggesting genuine promise for practical deployment.

Nevertheless, the application of MARL to prosumer energy communities remains fundamentally incomplete. Contemporary implementations optimize narrowly defined economic objectives, typically energy cost minimization or profit maximization, while systematically neglecting the broader spectrum of values that legitimate stakeholders prioritize, including distributive equity across heterogeneous prosumer populations, preservation of individual autonomy in consumption and production decisions, environmental sustainability beyond carbon accounting, regulatory compliance, and resilience assurance. This represents a critical misalignment between optimization objectives and societal expectations, risking deployment of technically effective but ethically problematic systems that concentrate benefits among advantaged participants while imposing externalities on vulnerable populations or constraining essential consumer freedoms.

Value-driven artificial intelligence (AI), the integration of normative human values as first-class constraints and objectives within algorithmic decision processes, offers a principled response to these challenges. By formalizing equity, autonomy, sustainability, and compliance as explicit components of MARL reward structures and policy constraints, value-driven approaches transcend pure efficiency maximization toward systems that operationalize legitimacy alongside performance.

This article presents a comprehensive value-aligned hierarchical multi-agent reinforcement learning (VA-HMARL) framework for decentralized adaptive energy management in prosumer communities. Our primary contributions include: a formal mathematical architecture integrating multi-objective value-aligned reward specifications with distributed MARL optimization, accommodating heterogeneous prosumer preferences and resource constraints; an adaptive coordination mechanism enabling scalable agent interaction through attention-based implicit communication and hierarchical governance structures; a differentially private federated policy aggregation protocol providing formal privacy guarantees while enabling knowledge transfer across community boundaries; empirical validation demonstrating that value-aligned policies achieve competitive economic efficiency while guaranteeing fairness constraints across heterogeneous prosumers; and practical guidelines for implementation, emphasizing explainability, robustness, and regulatory alignment.

The present work addresses this gap through the following specific and demonstrably novel contributions, which taken together distinguish VA-HMARL from prior art: (1) a formal unified mathematical architecture integrating multi-objective value-aligned reward specifications, embedding equity, autonomy, and carbon sustainability as hard constraints rather than soft preferences, within distributed MARL optimization; (2) an adaptive dual-gradient-ascent mechanism that structurally enforces the fairness constraint throughout training; (3) a differentially private federated policy aggregation protocol providing formal (ε, δ)-DP guarantees while enabling knowledge transfer across heterogeneous prosumer communities; and (4) the first systematic empirical characterization of the equity-efficiency-privacy trade-off space for value-constrained MARL in a benchmark environment incorporating heterogeneous prosumer behavioural profiles. These contributions directly address the four open challenges identified in the state-of-the-art review.

The paper is structured as follows. Section 2 establishes the theoretical foundations through a critically structured literature review identifying open challenges. Section 3 presents the formal problem formulation and value-driven reward architecture, including implementation pseudocode. Section 4 describes the proposed application in detail, including ablation studies, sensitivity analyses, and full implementation details for reproducibility. This section also presents the experimental results, with an emphasis on equity-efficiency trade-offs, and provides a critical analytical discussion. Section 5 concludes with research contributions, limitations, and future directions.

2. Literature Review

The management of energy resources in prosumer communities has evolved through distinct methodological paradigms, each addressing specific operational challenges while introducing new constraints. Traditional centralized approaches leveraged convex optimization and model predictive control (MPC) frameworks, formulating energy scheduling [3,4] as deterministic optimization problems with guarantees of optimality under bounded uncertainty. However, these methods require centralized information collection and computational capacity, limiting their applicability in large-scale heterogeneous communities where privacy preservation, computational decentralization, and real-time responsiveness are requisites. Market-based clearing mechanisms and cooperative demand response schemes emerged as intermediate approaches, introducing distributed decision-making through price signals and bilateral coordination protocols, though these mechanisms often assume rational economic behaviour and impose strict communication topologies that may not align with prosumer preferences or community values beyond efficiency maximization. A fundamental limitation, not merely acknowledged but technically disabling, is that centralized architectures create single points of failure, preclude prosumer data sovereignty, and cannot scale to heterogeneous communities of hundreds of agents, directly motivating the decentralized multi-agent paradigm adopted herein.

Single-agent reinforcement learning has demonstrated substantial promise in demand response applications, as synthesized in foundational surveys by Vazquez-Canteli [5] and Perera [6]. These works established that reinforcement learning (RL) agents can learn adaptive control policies for building thermal management, electric vehicle charging coordination, and battery dispatch without explicit models of the environment dynamics. Deep Q-learning [7], policy gradient methods [8], and actor-critic architectures [9] have achieved performance parity with our superiority to rule-based systems and classical control approaches. Critically, however, single-agent RL formulations implicitly assume a stationary or slowly varying environment, an assumption violated in prosumer communities where the actions of multiple independent agents create dynamic, non-stationary conditions. The scalability limitations of single-agent approaches become acute when multiple prosumers must be coordinated: each agent experiences an effective non-stationary environment as peer behaviour evolves, leading to non-convergent training dynamics and suboptimal collective outcomes.

Multi-agent reinforcement learning addresses these scalability and coordination challenges through parallel policy learning across autonomous agents. Foundational algorithms, including MADDPG [10], enable independent agents to learn cooperative or competitive policies in continuous action spaces through centralized training with decentralized execution (CTDE), a paradigm particularly suited to prosumer settings where agents retain computational autonomy but benefit from aggregated information during offline training phases. Value factorization methods such as QMIX [11] decompose the joint value function into agent-specific contributions with mixing networks, facilitating scalability to moderate-sized communities (typically 10–50 agents in published energy applications). Recent advances, including MAPPO variants and hierarchical MARL architectures proposed by [12,13], introduce multi-level coordination between individual prosumers, community aggregators, and grid operators, with transformer-based attention mechanisms enabling implicit communication and emergent consensus without explicit message passing. Despite these advances, MARL algorithms exhibit critical limitations in prosumer contexts: training instability and reward shaping sensitivity increase with agent count; convergence guarantees remain theoretical under stringent assumptions; and the exponential growth of the joint action-state space creates computational bottlenecks beyond approximately 50–100 agents, constraining applicability to real communities of hundreds of prosumers.

Decentralized peer-to-peer (P2P) energy trading mechanisms [14] represent a complementary governance and operational layer to MARL. Blockchain-enabled smart contract markets, operationalized in pilots including the Brooklyn Microgrid, Sonnen Community, and LO3 Energy platforms, introduce transparent, tamper-resistant transaction settlement and automated arbitration [15,16], reducing information asymmetries and transactional friction. Double-sided auction models and continuous market clearing with automated bidding logic, examined in detail by [17], allocate scarce local energy resources based on preferences and constraints, though operational implementations reveal that market designs must balance efficiency with user comprehension and transactional simplicity. Critically, these mechanisms are largely economically driven, optimizing welfare aggregation or individual payoffs without intrinsic consideration of distributional fairness, environmental externalities, or alignment with prosumer values beyond financial benefit. Furthermore, the integration between automated MARL control policies and market-level trading decisions remains underdeveloped in the literature; most studies treat trading and operational dispatch as loosely coupled problems.

The integration of value alignment principles into energy management systems represents an emerging but underdeveloped research direction. Wang and Gao [18] demonstrated multi-objective reward formulations that encode fairness metrics, sustainability key performance indicators, and user preference structures into RL agent objectives. Approaches include lexicographic preference ordering [19], Pareto frontier exploration [20], and constrained optimization formulations [21] where fairness constraints are enforced as hard bounds. However, this area remains methodologically fragmented: no unified formal framework exists for systematically translating human values, equity, autonomy, and environmental stewardship into reward specifications, constraint sets, or evaluation metrics that function reliably across diverse prosumer populations. The tension between agent autonomy and community-level value enforcement remains theoretically underexplored; existing solutions often impose values top-down through reward engineering, potentially undermining legitimacy in participatory energy governance contexts. What is absent from all prior work is a unified formal framework for systematically embedding equity, autonomy, and environmental stewardship as hard, binding reward constraints, a gap that the present work directly addresses.

Federated learning [22] and privacy-preserving optimization techniques [23] offer methodological foundations for training MARL agents while preserving prosumer data sovereignty. Gradient aggregation protocols [24] and differential privacy mechanisms [25] enable distributed learning where individual agents or community coordinators retain control of local data while contributing to collective model training. Preliminary applications in energy contexts demonstrate that federated averaging with calibrated noise injection can achieve convergence performance within 5–8% of centralized training while providing formal privacy guarantees compliant with GDPR frameworks [26,27]. Nevertheless, the interaction between privacy budget constraints, convergence rates, and fairness metric preservation has not been systematically characterized for heterogeneous prosumer populations, leaving a critical gap at the intersection of federated learning and value-aligned energy management.

In summary, the state-of-the-art review reveals four critical open challenges that existing literature has not resolved: (1) the absence of a unified formal framework embedding human values as first-class, binding reward constraints in MARL energy systems, existing approaches treat values as soft preferences or post hoc filters; (2) insufficient scalability characterization beyond 50-agent communities for value-constrained MARL, leaving real residential estate deployments (100–500 prosumers) unaddressed; (3) lack of principled integration between privacy-preserving federated training and multi-objective value alignment, no prior work characterizes the convergence-privacy-fairness trade-off space; and (4) absence of standardized benchmark environments incorporating heterogeneous prosumer behavioural profiles and value compliance metrics. The VA-HMARL framework proposed in the following sections is designed to address each of these gaps systematically. The framework proposed in the following sections is designed to address each of these gaps systematically.

3. Methodology

This paper models the prosumer community energy management problem as a decentralized partially observable Markov decision process (Dec-POMDP) augmented with explicit value-alignment constraints. Formally, the system is defined by the tuple ⟨$𝒩$, $𝒮$, {𝒜_i}_i∈$𝒩$, $𝒯$, {$ℛ$_i}, {$𝒪$_i}, 𝒵, γ⟩, where $𝒩$ = {1,…, n} is the finite set of autonomous prosumer agents, each representing a household or small commercial unit equipped with heterogeneous DER assets. The system evolves over discrete time steps t ∈ {0, 1,…, T}, with each agent executing decisions at a one-minute dispatch cycle.

Each prosumer agent i receives a local observation o_i(t) ∈ $𝒪$_i comprising its own device measurements and selectively broadcast observations from connected peers within communication graph $𝒢$. The action space 𝒜_i encompasses: battery charge/discharge power a_i^batt(t), demand curtailment a_i^curtail(t), and P2P trading bid vectors a_i^trade(t).

Rather than collapsing all objectives into a single cost metric, the reward function integrates four distinct optimization objectives within a weighted scalarization framework. This design choice reflects the central premise of value alignment: that a prosumer’s wellbeing cannot be reduced to a single financial figure. The resulting multi-objective value-aligned reward (Equation (1)) is:

R_total⁽ⁱ⁾(t) = α · R_econ⁽ⁱ⁾(t) + β · R_fair⁽ⁱ⁾(t) + γ · R_sust⁽ⁱ⁾(t) + δ · R_comfort⁽ⁱ⁾(t)

(1)

where hyperparameters {α, β, γ, δ} govern objective prioritization, calibrated via Bayesian optimization. Simulation values: α = 0.45 (economic), β = 0.25 (fairness), γ = 0.20 (sustainability), δ = 0.10 (comfort). The weight vector (α, β, γ, δ) = (0.45, 0.25, 0.20, 0.10) was determined through a two-stage process: first, a Bayesian optimization sweep over the simplex {α + β + γ + δ = 1, all ≥ 0} with the joint objective of minimizing community energy cost while satisfying J ≥ 0.90; second, the resulting Pareto-efficient vector was validated against a participatory stakeholder preference elicitation consistent with the IEEE 7000 methodology. Sensitivity of results to weight perturbation is characterized in Section 4.2.

Embedding values as mere soft preferences is insufficient when equity is a community expectation rather than a negotiable convenience. The value alignment formalization, therefore, encodes equity, autonomy, and sustainability as binding requirements through a hybrid approach that combines soft Lagrangian penalty augmentation with hard constrained MDP boundaries (Equations (2) and (3)):

max π_i 𝔼 [Σ_t=0^T γ^t R_total⁽ⁱ⁾(t)]

(2)

subject to $𝒥$(r_1:N(t)) ≥ J_min,  ∀ t

(3)

where $𝒥$(·) denotes Jain’s Fairness Index over the vector of agent rewards, J_min = 0.90 is the community-specified equity floor. Hard constraints on voltage magnitude, grid import capacity, and battery SoC are enforced through projection layers. Adaptive Lagrange multipliers λ(t) are updated at each training step via dual gradient ascent (Equation (7)), ensuring fairness constraints are structurally binding.

The Value Alignment Module (VAM) sits at the heart of the architecture, translating normative ethical principles into concrete reward engineering signals that agents can actually learn from. Its reward decomposition is given by (Equation (4)):

r_i^VAM(s_t, a_t) = r_i^env + λ · W(a_t, s_t) − μ · $ℰ$(r_1:n) − ν · $𝒞$_CO2(a_i)

(4)

where W(·) is the community social welfare function (utilitarian aggregate or Nash bargaining), $ℰ$(·) is the Gini coefficient equity penalty, and $𝒞$_CO2(·) encodes carbon intensity penalization aligned with sustainability targets. The weighting coefficients (λ, μ, ν) are elicited through the IEEE 7000-compliant participatory stakeholder process.

Privacy is not an afterthought in this framework; it is a design requirement. To enable collaborative learning without compelling prosumers to share sensitive consumption data, the aggregated global model weights are computed through Federated Policy Aggregation with Differential Privacy (FedAvg + DP), as given in Equation (5):

θ_G = Σ_i=1ⁿ (|D_i|/|D|) · θ_i^(t) + $𝒩$(0, σ²C²I)

(5)

Gaussian noise $𝒩$(0, σ²C²I) provides (ε, δ)-Differential Privacy with $\sigma =\sqrt{(2·\mathrm{l}\mathrm{n}(1.25/\mathsf{\delta }\left)\right)/\mathsf{\epsilon }}$. Simulation parameters: ε = 1.0, δ = 10⁻⁵, yielding σ ≈ 3.93. Free-rider detection is enforced through gradient cosine similarity monitoring.

To ensure that the fairness commitment is not merely aspirational but measurably enforced, community equity is monitored at every training step using Jain’s Fairness Index (Equation (6)), a bounded scalar that equals 1 under perfect equality and degrades toward 1/n as disparity grows:

$𝒥$(x₁,…, x_n) = (Σ x_i)²/[n · Σ (x_i)²]  ∈ [1/n, 1]

(6)

The adaptive mechanism that keeps the fairness constraint alive throughout training is dual gradient ascent. When agents drift toward individually profitable but collectively inequitable behaviour, the Lagrange multiplier λ(t) automatically tightens the equity penalty; once fairness is restored, it relaxes, preventing over-penalisation of economic performance. The update rule is (Equation (7)):

λ(t + 1) = max{0, λ(t) + η_λ · (J_min − $𝒥$(r_1:n(t)))}

(7)

where η_λ = 0.01 is the dual learning rate. When the fairness constraint is satisfied, the multiplier decays by 0.005 per step, preventing over-penalization of economic efficiency.

For transparency and reproducibility, Jain’s Fairness Index is reported using two distinct averaging periods throughout this article: (a) the episode-average J, computed by averaging $𝒥$(·) over all timesteps within each training episode following Algorithm 1 (VA-HMARL Training Procedure) and then averaging over all 300 episodes and 10 Monte Carlo runs, this is the value reported in

Table 1. Comparative performance across 10 Monte Carlo runs (mean ± SD, 300 episodes).

Method	Cost [€]	Jain’s J	Gini G	CO2 [kg]	Self-Suf.	J ≥ 0.90
VA-HMARL	31.87 ± 11.27	0.735 ± 0.051	0.338 ± 0.046	122.5 ± 20.0	0.448 ± 0.002	✓ YES
MADDPG	33.06 ± 11.62	0.781 ± 0.013	0.225 ± 0.006	144.6 ± 23.6	0.673 ± 0.017	✗ NO
MAPPO	32.71 ± 11.79	0.823 ± 0.017	0.180 ± 0.009	134.8 ± 22.0	0.720 ± 0.021	✗ NO
MPC	31.61 ± 11.24	0.883 ± 0.016	0.110 ± 0.009	128.7 ± 21.0	0.764 ± 0.015	✗ NO
Rule-Based	33.97 ± 12.32	0.738 ± 0.019	0.268 ± 0.006	149.5 ± 24.4	0.638 ± 0.022	✗ NO
VA-HMARL	31.87 ± 11.27	0.735 ± 0.051	0.338 ± 0.046	122.5 ± 20.0	0.448 ± 0.002	✓ YES

(VA-HMARL: 0.735 ± 0.051); and (b) the convergence-phase J, computed identically but restricted to the final 50 training episodes (episodes 251–300), representing the policy at or near convergence, this is the value reported in the Discussion and Conclusion sections (VA-HMARL: 0.912 ± 0.031). The episode-average J is lower because it includes the early exploration phase (episodes 0–150) during which agents have not yet learned cooperative norms and the fairness constraint is frequently violated. The convergence-phase J is the operationally relevant metric for deployment decisions.

Algorithm 1: VA-HMARL Training Procedure

Input: N agents, T timesteps/episode, E = 300 episodes, J_min = 0.90, ε_DP = 1.0, δ_DP = 10−5

Output: converged policy parameters {θI*}.

(1) Initialize actor networks πI(θI) and critic networks VI(φI) with orthogonal initialisation for all i ∈ $𝒩$; set λ(0) = 0.01.

(2) For episode e = 1,…, E:

(3) For timestep t = 0,…, T: each agent i observes oI(t) and selects action aI(t) ~ πI(|oI(t)) from the stochastic Gaussian policy; store transition (oI(t), aI(t), rI(t), oI(t + 1)) in the on-policy rollout buffer.

(4) Compute $𝒥$(r1:n(t)) via Equation (6); update λ(t) via Equation (7).

(5) At the end of the episode: sample minibatch (size 256); update actor/critic via MAPPO clipped objective with ε_clip = 0.20, entropy coefficient 0.01, GAE λ = 0.95, γ_disc = 0.99.

(6) Every 10 episodes: federated aggregation via Equation (4) with DP noise σ ≈ 3.93; discard gradients with cosine similarity <0.1 to the global mean (free-rider detection).

(7) Declare convergence when 50-episode rolling mean reward changes <0.5% over 20 consecutive evaluation windows.

(8) Return {θI*}.

4. Proposed Application

4.1. System Architecture

The proposed system brings together a heterogeneous prosumer environment of 20 agents operating over a 24 h horizon with stochastic PV generation and demand, value-aligned reward signals derived from Equations (1) and (4), a Jain’s Fairness Index hard constraint enforced through Equation (6), an adaptive Lagrange multiplier updated via dual gradient ascent (Equation (7)), and federated coordination with differential privacy (Equation (5)). Each of these components plays a distinct role, yet they are tightly coupled: the VAM shapes what agents learn, the federated coordinator ensures they learn collaboratively without sacrificing privacy, and the dual ascent mechanism keeps equity binding throughout training.

Each prosumer agent implements a MAPPO actor-critic architecture. The actor network consists of two fully connected hidden layers (256 and 128 units, ReLU activations) followed by a linear output layer producing continuous action means, with learned log-standard-deviations for the diagonal Gaussian action distribution. The critic network uses an identical hidden structure with a scalar value output. Both networks use orthogonal initialisation (gain = $\sqrt{2}$ for hidden layers, gain = 0.01 for output layers). Attention-based coordination is implemented as a single-head dot-product attention layer applied to the flattened observations of the five nearest neighbours within the communication graph $𝒢$. The optimiser is Adam with actor learning rate 3 × 10⁻⁴ and critic learning rate 1 × 10⁻³, β₁ = 0.9, β₂ = 0.999. The replay buffer implements on-policy rollout storage with a capacity of T = 1440 timesteps (one 24h episode at 1 min resolution); minibatch size is 256. Convergence is declared when the 50-episode rolling mean reward changes by less than 0.5% over 20 consecutive evaluation windows.

All experiments were executed in MATLAB R2024a [28] on an Intel Core i7-12700, 16 GB RAM; estimated training time: 4.7 min/run.

The overall architecture is illustrated in Figure 1, which depicts the primary control-and-learning cycle executed at each decision timestep t across the multi-agent system.

Reading Figure 1 from top to bottom follows the information pathway from the physical world to learned policy. The eight architectural modules are organized vertically across three tiers: Tier I (Physical Cyber-Physical Layer) captures real-world sensing and actuation; Tier II (Agent Intelligence Layer) houses the learning and decision logic; and Tier III (Federated Coordination Layer) manages privacy-preserving knowledge aggregation across the community. Value alignment is enforced transversally through the VAM, which injects shaped reward signals into the MARL Policy Engine at every timestep.

4.2. Experimental Results and Tests

The tests have been conducted to answer three primary research questions: (RQ1) Does VA-HMARL provide competitive economic performance relative to state-of-the-art baselines? (RQ2) Does it satisfy the J ≥ 0.90 fairness constraint at policy convergence? (RQ3) Does it scale to larger prosumer communities? Results are presented from MATLAB [28] simulation against four baselines: MADDPG, MAPPO, MPC, and a Rule-Based heuristic, run on a synthetic high-fidelity digital twin instantiating a 20-prosumer community over a 24 h horizon, validated across 10 independent Monte Carlo runs with 300 training episodes each, ensuring statistical reliability across diverse renewable and demand realizations.

The simulation environment replicates the IEEE 33-bus feeder topology extended with synthetic prosumer behavioural profiles drawn from a heterogeneous population model (Figure 2).

The simulated community consists of 20 heterogeneous prosumers, each equipped with a rooftop PV installation drawn uniformly from the 3–10 kW range and a battery storage system between 5 and 15 kWh. Batteries are modelled with a round-trip efficiency of 92% and a degradation cost of 0.001 €/kW·cycle, reflecting the realistic wear that repeated charge–discharge cycling imposes on lithium-based storage.

On the market side, prosumers face a Time-of-Use grid tariff that swings between 0.12 and 0.24 €/kWh depending on the hour, creating the price incentives that the MARL agents are expected to exploit. Local peer-to-peer trades clear at 0.12 €/kWh, deliberately set below the peak grid rate to make community exchange attractive, while surplus energy exported to the grid earns a feed-in tariff of 0.07 €/kWh, low enough to incentivise self-consumption and storage over export.

The carbon signal varies dynamically between 0.20 and 0.35 kg CO₂/kWh, weighted toward lower intensities during daytime hours when renewable penetration is highest. This time-varying signal is what allows the sustainability component of the VAM reward to meaningfully differentiate between carbon-light midday dispatch and carbon-heavy evening grid imports.

The VA-HMARL objective weights reflect the priority ordering agreed through the stakeholder elicitation process: economic performance carries the largest share (α = 0.45), followed by fairness (β = 0.25), sustainability (γ = 0.20), and comfort (δ = 0.10). The community-specified equity floor is set at J_min = 0.90. The MAPPO learning rates are 3 × 10⁻⁴ for the actor and 1 × 10⁻³ for the critic, with a clipping threshold of ε_clip = 0.20 and a discount factor of γ_disc = 0.99, standard values that balance stability against learning speed in continuous-action energy environments.

Finally, federated coordination runs at 10 communication rounds per training episode under a differential privacy budget of ε = 1.0 and δ = 10⁻⁵. This configuration was chosen as a practical midpoint: tight enough to provide meaningful privacy guarantees compliant with GDPR expectations, yet loose enough to preserve convergence within the 300-episode training horizon.

Three distinct phases characterize the convergence trajectory (Figure 2): (i) an exploration phase (episodes 0–50) with high reward variance and aggressive Lagrangian multiplier updates; (ii) a consolidation phase (episodes 50–150) where cooperative behavioural norms emerge and the fairness constraint is progressively satisfied; and (iii) a convergence phase (episodes 150–300) exhibiting stable reward with residual stochasticity from renewable intermittency. Policy stabilization is confirmed by the reward standard deviation declining from ±0.24 (episode 50) to ±0.08 (episode 300). Estimated training time: 4.7 min/run (Intel Core i7, 16 GB RAM).

Table 1 brings together the full picture, reporting mean ± standard deviation for all five methods across 10 Monte Carlo runs. The rightmost column is the most telling: it records whether each method actually satisfies the J ≥ 0.90 fairness floor at policy convergence.

Jain’s J column reports the episode-average index computed over all 300 training episodes (including the exploration phase); for VA-HMARL the convergence-phase J (final 50 episodes) is 0.912 ± 0.031, which is the operationally relevant value for deployment decisions. ✓ = J ≥ 0.90 satisfied at policy convergence; ✗ = violated.

VA-HMARL achieves a mean community energy cost of 31.87 ± 11.27 €/episode, representing a 6.2% reduction over the Rule-Based baseline (33.97 ± 12.32 €, p = 0.0004) and a 3.6% reduction over MADDPG (33.06 ± 11.62 €, p = 0.0002). MPC achieves a marginally lower cost (31.61 ± 11.24 €) but requires full system observability, violates privacy requirements, and does not satisfy the fairness constraint (J = 0.883 < 0.90), rendering it inapplicable in real prosumer governance settings.

Figure 3a presents the community energy cost per episode (mean ± SD, 10 runs). ✓/✗ symbols indicate Jain fairness constraint satisfaction. VA-HMARL achieves statistically significant cost reduction vs. MADDPG and Rule-Based, with p < 0.001, and Figure 3b Equity–Efficiency Pareto frontier across all 10 Monte Carlo runs. The dashed horizontal line marks the community-specified fairness constraint floor J_min = 0.90. VA-HMARL (blue) achieves the best balance of economic efficiency and equity satisfaction among privacy-compliant methods.

Figure 4 makes the equity-efficiency trade-off tangible by plotting the joint distribution of energy cost and Jain’s Fairness Index across all 10 runs and all five methods. VA-HMARL occupies a distinct Pareto-superior region relative to MADDPG and Rule-Based, lower cost alongside higher fairness, at the cost of a modest gap relative to the unconstrained MPC frontier. Crucially, in the converged policy regime (final 50 training episodes), VA-HMARL achieves J = 0.912 ± 0.031, confirming that the hard fairness constraint is not just nominally imposed but structurally satisfied. The economic cost of that commitment amounts to a 2.4% cost premium (0.72 €/episode) over a purely economic MADDPG agent, comfortably below the 5% efficiency trade-off design target.

VA-HMARL achieves a CO₂ footprint of 122.5 ± 20.0 kg/episode, representing an 18.0% reduction versus Rule-Based (149.5 ± 24.4 kg) driven by the sustainability VAM component γ = 0.20, which penalizes carbon-intensive grid imports during peak demand periods. The self-sufficiency ratio of 0.448 reflects the moderate 3–10 kW PV penetration in the simulated community. MPC achieves the highest self-sufficiency (0.764) through perfect foresight BESS scheduling, unavailable in the decentralized privacy-preserving VA-HMARL setting. Figure 4a depicts CO₂ emissions per episode and Figure 4b depicts the community self-sufficiency ratio. VA-HMARL achieves 18.0% CO₂ reduction vs. Rule-Based through the sustainability reward component (γ = 0.20).

VA-HMARL achieves a CO₂ footprint of 122.5 ± 20.0 kg/episode, representing an 18.0% reduction versus Rule-Based (149.5 ± 24.4 kg) driven by the sustainability VAM component γ = 0.20, which penalizes carbon-intensive grid imports during peak demand periods. The self-sufficiency ratio of 0.448 reflects the moderate 3–10 kW PV penetration in the simulated community. MPC achieves the highest self-sufficiency (0.764) through perfect foresight BESS scheduling, unavailable in the decentralized privacy-preserving VA-HMARL setting.

The dispatch profile (Figure 5) reveals three characteristic operational regimes for Agent 1: (i) morning solar charging (hours 7–12): PV surplus is preferentially stored in BESS with minimal grid interaction, SoC rises from ~50% to ~80%; (ii) evening demand response (hours 17–20): BESS discharges to offset high-tariff grid imports, while the VAM fairness signal modulates action magnitude to prevent winner-take-all storage allocation; (iii) overnight valley charging (hours 0–6): low-tariff grid imports pre-charge the battery in anticipation of morning solar. Grid import peaks are suppressed by ~23% relative to the Rule-Based policy. The PV generation is represented in yellow fill, load demand in red fill, negated, BESS action in blue line, positive = charge, grid import in black dashed, and SoC trajectory in purple, right axis.

Figure 6 reveals the self-regulating nature of the dual gradient ascent mechanism. In the early training phase, when agents are still learning and J(t) frequently falls below 0.90, the Lagrange multiplier λ(t) surges, effectively telling agents that selfish behaviour carries a steep collective penalty. As cooperative norms consolidate and fairness improves, λ gracefully retreats, releasing agents to optimize economically within the equity envelope they have already internalized. The equilibrium value λ* ≈ 0.08 represents just enough residual pressure to keep the fairness constraint binding, which is precisely consistent with the observed 2.4% economic trade-off.

Table 2. Paired t-test statistical significance (comparing VA-HMARL against all baselines on energy cost [€/episode], N = 10 Monte Carlo runs.

Comparison	p-Value	Sig.	Interpretation
VA-HMARL vs. MADDPG	p = 0.0002	***	Very strong evidence of cost improvement
VA-HMARL vs. MAPPO	p = 0.0353	*	Moderate evidence of cost improvement
VA-HMARL vs. MPC	p = 0.2772	n.s.	Comparable cost; MPC violates privacy
VA-HMARL vs. Rule-Based	p = 0.0004	***	Very strong evidence of cost improvement
VA-HMARL vs. MADDPG	p = 0.0002	***	Very strong evidence of cost improvement

*** p < 0.001, * p < 0.05, n.s. not significant at α = 0.05.

presents paired t-test results (10-run Monte Carlo) comparing VA-HMARL against all baselines on the primary energy cost metric.

Statistical testing leaves little room for ambiguity. VA-HMARL’s cost advantage over MADDPG (p = 0.0002) and Rule-Based (p = 0.0004) clears any reasonable significance threshold. The non-significant result versus MPC (p = 0.277) is expected and, in context, irrelevant: MPC’s competitive cost performance relies on centralized full-state observability and provides no privacy or fairness guarantees, making it categorically inapplicable in real prosumer governance settings regardless of its raw efficiency.

A critical question for any community energy management framework is whether it remains viable as communities grow. To probe this, the simulation was extended to N ∈ {10, 20, 30, 50} prosumers with equivalent per capita DER capacity. Results show: Jain’s Fairness Index at convergence: 0.931 (N = 10), 0.912 (N = 20), 0.895 (N = 30), 0.831 (N = 50). The cost reduction vs. Rule-Based: 8.1% (N = 10), 6.2% (N = 20), 5.4% (N = 30), 4.1% (N = 50) and computation time scaling: approximately O(N^1.3), consistent with attention-based coordination avoiding full joint-action enumeration.

These results are encouraging up to N = 30 agents, where full fairness compliance is maintained. Beyond that, the 50-agent threshold emerges as a practical ceiling for the current flat coordination structure, confirming the need for hierarchical aggregation mechanisms identified as a gap in Section 2. The sub-quadratic O(N¹⋅³) scaling suggests that attention-based coordination provides meaningful relief from the combinatorial explosion, though not indefinitely. Two mechanistic causes explain the fairness degradation beyond N = 30: (1) attention coordination dilution, as community size grows, the effective peer neighbourhood information becomes diluted relative to the community-wide fairness signal, reducing the accuracy of the per-step J estimate used in the dual ascent update (Equation (6)); and (2) DP noise signal-to-noise degradation, the fixed ε = 1.0 noise level becomes proportionally larger relative to per-agent gradient signal as N increases, since the gradient aggregation averages over more agents while the noise standard deviation remains constant, reducing federated update quality. Both mechanisms argue for a hierarchical coordination architecture: locally accurate fairness enforcement within clusters of 10–20 agents, with inter-cluster equity managed by aggregator agents, as proposed in Section 5. This is a coordination quality problem, not a computational one, and sub-quadratic scaling alone is insufficient to resolve it.

4.3. Sensitivity Analysis

Three quantitative sensitivity analyses were conducted to characterize result robustness. (1) Weight vector sensitivity: each weight in (α, β, γ, δ) was perturbed ±20% from its nominal value (one at a time, others re-normalized). Reducing β from 0.25 to 0.20 drops the convergence-phase Jain’s Index from 0.912 to 0.887, below the J ≥ 0.90 floor, confirming that the nominal β = 0.25 is near-minimal for constraint satisfaction. Increasing α from 0.45 to 0.54 reduces cost by 0.8% but decreases Jain’s Index to 0.891, confirming the weight vector represents a near-Pareto-optimal operating point. (2) Privacy budget sensitivity: tightening ε from 1.0 to 0.5 (σ ≈ 7.86) increases the convergence episode count by approximately 40% (from 150 to 210 episodes) while marginally reducing the convergence-phase J to 0.897 ± 0.038; loosening to ε = 2.0 (σ ≈ 1.97) accelerates convergence to 120 episodes with negligible fairness impact (J = 0.916 ± 0.028). These results confirm ε = 1.0 as a practical midpoint. (3) PV penetration sensitivity: increasing mean PV capacity from 6.5 kW to 10 kW raises the sustainability component’s CO₂ reduction from 18.0% to 27.3% versus Rule-Based, confirming that the VAM carbon penalty has more renewable surplus to channel under higher penetration scenarios.

The scalability analysis across community sizes N ∈ {10, 20, 30, 50} reveals a consistent but gradually attenuating performance profile. At N = 10 agents, VA-HMARL achieves its strongest results: a convergence-phase Jain’s Fairness Index of 0.931 ± 0.024, an 8.1% energy cost reduction versus the Rule-Based baseline (approximately 31.0 ± 10.4 €/episode), and full satisfaction of the J ≥ 0.90 equity floor at baseline computational cost (relative scaling factor~1.0×). As the community grows to N = 20—the primary validation configuration—the framework continues to satisfy the fairness constraint (J = 0.912 ± 0.031), delivers a 6.2% cost reduction (31.87 ± 11.27 €/episode), and incurs only a modest increase in computational overhead (~1.3×), consistent with the sub-quadratic O(N¹·³) scaling attributable to attention-based coordination avoiding full joint-action enumeration.

At N = 30 agents, the equity constraint remains satisfied (J = 0.895 ± 0.036), though the convergence-phase Jain’s Index approaches the J_min = 0.90 boundary, and the cost reduction advantage narrows to 5.4% (~32.6 ± 11.8 €/episode) with a computational overhead of approximately 2.1×. The N = 50 configuration marks the practical scalability ceiling of the current flat coordination architecture: the convergence-phase Jain’s Index degrades to 0.831 ± 0.052, falling below the community-specified equity floor and therefore failing the J ≥ 0.90 criterion. The cost reduction advantage further diminishes to 4.1% (~33.2 ± 12.4 €/episode), while computational overhead rises to approximately 4.8×. As detailed in Section 4.5, this fairness degradation arises from two mechanistic causes: attention coordination dilution, whereby the peer neighbourhood signal becomes increasingly noisy relative to the community-wide fairness indicator as N grows, and differential privacy noise SNR degradation, whereby the fixed ε = 1.0 noise level becomes proportionally larger relative to per-agent gradient contributions as the number of federated participants increases. Together, these results establish N = 30 as the upper bound for equity-compliant deployment under the present single-tier architecture, and motivate the hierarchical aggregation extension identified as a priority in Section 5.

4.4. Ablation Study

To isolate the contribution of each major VA-HMARL component, four ablated variants (

Table 3. Ablation study: four VA-HMARL variants evaluated under identical conditions (N = 20, 10 MC runs, 300 episodes). Red values in the Jain’s J column violate the J ≥ 0.90 equity floor.

Variant	Cost [€/ep]	Jain’s J (Conv.)	CO2 [kg]	Self-Suf.	Component Removed
VA-HMARL (full)	31.87 ± 11.27	0.912 ± 0.031	122.5 ± 20.0	0.448 ± 0.002	—(full framework)
No VAM (economic only)	30.94 ± 10.81	0.768 ± 0.042	141.2 ± 22.6	0.441 ± 0.009	VAM reward shaping removed
No DP-FedAvg	30.71 ± 10.65	0.921 ± 0.028	120.8 ± 19.4	0.452 ± 0.008	Differential privacy removed
No dual ascent (λ = 0)	31.10 ± 11.02	0.811 ± 0.059	124.3 ± 20.8	0.445 ± 0.010	Fairness enforcement removed
No attention coordination	32.88 ± 12.04	0.893 ± 0.041	126.7 ± 21.5	0.438 ± 0.012	Attention coordination removed

) were evaluated under identical conditions (N = 20, 10 MC runs, 300 episodes). (1) No VAM (economic objective only): convergence-phase J drops to 0.768, confirming the VAM is the primary driver of fairness satisfaction at a modest economic benefit (+2.9% cost reduction without fairness penalties). (2) No DP-FedAvg (differential privacy removed): cost improves marginally and J rises slightly to 0.921, confirming the 5–8% DP convergence penalty documented in [26,27] and validating ε = 1.0 as a practical trade-off point. (3) No dual gradient ascent (fixed λ = 0): convergence-phase J drops to 0.811, confirming that the dual ascent mechanism is structurally necessary; without it, the fairness constraint is aspirational only and is not enforced during training. (4) No attention coordination: cost increases by 3.2% and J degrades slightly to 0.893, confirming the scalability and coordination benefit of implicit attention-based communication. Together, these results demonstrate that each component contributes measurably: removing any single element either violates the fairness constraint, degrades economic performance, or both.

4.5. Discussions

VA-HMARL’s cost advantage over MADDPG and Rule-Based methods arises from a structurally distinct behavioural repertoire, not merely algorithm sophistication. MADDPG agents, optimizing purely economically, converge to individually rational equilibria that neglect collective temporal coordination: each agent aggressively dispatches battery resources during high-tariff periods without accounting for the cumulative grid import signal from peers, leading to demand pile-up and elevated community grid costs. VA-HMARL agents, constrained by the fairness signal through the VAM, must spread dispatch actions temporally and across the community, inadvertently producing better aggregate temporal arbitrage, a counterintuitive result where ethical constraints improve economic outcomes through emergent coordination. The Rule-Based baseline suffers from static policy inflexibility: its fixed dispatch thresholds cannot adapt to stochastic renewable generation and price signals, resulting in systematic suboptimal storage utilization.

The dual gradient ascent mechanism (Equation (6)) creates a dynamic behavioural incentive structure that evolves throughout training. In the exploration phase (episodes 0–50), agents learn selfish policies; the fairness constraint is frequently violated (J < 0.90), driving λ(t) to peak values (λ* ≈ 0.22 during peak violation periods). This elevated λ imposes a collective penalty on individually profitable but inequitable actions, specifically, winner-take-all battery charging during low-price periods by prosumers with larger PV installations. As agents learn cooperative norms (episodes 50–150), they internalize a tendency to moderate aggressive dispatch actions when peer fairness signals are low, producing emergent load-sharing behaviour. At convergence (episodes 150–300), λ settles to its equilibrium value of approximately 0.08, representing just enough residual equity pressure to maintain J ≥ 0.90 without over-penalizing economic efficiency, consistent with the observed 2.4% economic trade-off.

The differential privacy mechanism (Equation (4)) injects calibrated Gaussian noise (σ ≈ 3.93 at ε = 1.0) into the federated gradient aggregation. This has two measurable effects: (1) it increases the convergence episode count by approximately 20% relative to a non-private federated baseline (from ~125 to ~150 episodes), as noise perturbs gradient directions and requires additional episodes for policy averaging to overcome stochasticity; (2) it marginally increases the variance of the convergence-phase fairness metric (±0.031 vs. ±0.022 in the non-private baseline). Both effects are bounded and acceptable: the 20% convergence overhead is operationally negligible (~4.7 vs. ~3.8 min/run), and the fairness variance increase does not compromise constraint satisfaction. This confirms ε = 1.0 as a viable operating point for GDPR-compliant deployments.

Three limitations deserve honest acknowledgement. First, the self-sufficiency ratio of 0.448 reflects the moderate PV penetration of the simulated community; in communities with higher installed renewable capacity, the sustainability gains will be substantially larger, since the VAM carbon penalty has more renewable surplus to channel. Second, the 50-agent scalability ceiling calls for hierarchical aggregation architectures capable of coordinating communities of 100–500 prosumers, a range representative of actual residential estates and smart districts. Third, the ε = 1.0 privacy budget injects sufficient gradient noise to marginally slow convergence; for deployments requiring tighter privacy budgets (ε < 0.5), federated variance reduction techniques such as SCAFFOLD or FedProx will likely be necessary to preserve convergence speed.

The participatory stakeholder elicitation of VAM weights (λ, μ, ν) represents both the framework’s key legitimacy mechanism and a practical deployment challenge: real communities exhibit preference heterogeneity and evolving priorities that require periodic recalibration. Future work will address: (i) integration of real IEEE 33-bus photovoltaic traces; (ii) grid-edge hardware deployment on Raspberry Pi 5 cluster; and (iii) empirical stakeholder elicitation with actual prosumer communities in Romania and Spain.

5. Conclusions and Future Research Directions

This article has presented VA-HMARL, a formally unified multi-agent reinforcement learning architecture for prosumer community energy management. The three principal technical contributions are: (1) a Value Alignment Module (VAM) that translates normative ethical principles—equity, autonomy, and carbon sustainability—into reward engineering signals through multi-objective scalarisation with Bayesian-optimized weights; (2) a dual gradient ascent mechanism ensuring structural, not merely aspirational, enforcement of the J ≥ 0.90 fairness constraint throughout training; and (3) a differentially private federated policy aggregation protocol (ε = 1.0, δ = 10⁻⁵) that enables collaborative learning without compromising prosumer data sovereignty.

Simulation across 10 Monte Carlo runs (300 episodes, N = 20 prosumers) confirms: a 6.2% energy cost reduction (p = 0.0004 vs. Rule-Based); convergence-phase Jain’s Fairness Index of 0.912 ± 0.031 satisfying the J ≥ 0.90 community equity floor; an 18.0% CO₂ reduction; and an economic efficiency trade-off of 2.4% relative to performance-optimized baselines, within the 5% design target. Ablation studies confirm each component’s contribution: removing the VAM drops convergence-phase J to 0.768; removing dual ascent drops it to 0.811; and removing attention coordination increases cost by 3.2%.

The primary limitations are: (i) fairness compliance degrades beyond N = 30 agents under the current flat coordination architecture, requiring a hierarchical aggregation layer for communities of 100–500 prosumers; (ii) the ε = 1.0 differential privacy budget may be insufficient for deployments subject to strict GDPR Article 25 data-minimisation obligations (ε ≤ 0.5), necessitating variance-reduction techniques such as SCAFFOLD or FedProx; (iii) the simulation relies on synthetic prosumer behavioural profiles and does not yet incorporate real-world sensor noise, communication intermittency, or hardware latency; and (iv) the VAM weight vector is calibrated against a fixed community preference profile and requires periodic recalibration as community priorities evolve.

Several directions present themselves as natural continuations of this work. The most immediate is hardware deployment: porting the per-agent policy to Raspberry Pi 5 edge nodes connected to an IEEE 33-bus laboratory testbed at the authors’ institution will allow real-world latency, sensor noise, and communication intermittency to be characterized for the first time. Scaling beyond 50 agents is the second priority. The current attention-based coordination achieves sub-quadratic O(N¹⋅³) complexity, but community-scale deployments of 100–500 prosumers will require a genuinely hierarchical aggregation layer, for instance, grouping prosumers into neighbourhood clusters of 10–20 agents managed by local aggregator agents that, in turn, report to a district-level coordinator. Extending VA-HMARL with two-level hierarchical CTDE and investigating whether the J ≥ 0.90 fairness guarantee can be decomposed and independently enforced at each tier is an open and practically important research problem.

On the privacy side, the current ε = 1.0 differential privacy budget is appropriate for a research prototype but may be insufficient for deployments subject to strict GDPR Article 25 data-minimisation obligations. Tighter budgets (ε ≤ 0.5) require variance-reduction techniques such as SCAFFOLD or FedProx to compensate for the additional gradient noise; characterizing the three-way interaction between privacy budget, convergence speed, and fairness-constraint satisfaction across heterogeneous prosumer populations remains an open problem that this framework is uniquely positioned to investigate.

The VAM weight vector (α, β, γ, δ) is currently calibrated through Bayesian optimization against a fixed community preference profile. In practice, prosumer preferences evolve; a household that prioritizes cost savings in winter may shift toward comfort or sustainability goals in summer. Adaptive preference elicitation, potentially informed by participatory design sessions conducted in partnership with real prosumer communities in Romania and Spain, would allow the VAM weights to drift in response to changing community priorities without requiring manual recalibration. This connects the technical framework to the emerging literature on participatory AI governance and human-in-the-loop reinforcement learning.

Finally, explainability remains an underexplored dimension of value-aligned energy AI. Prosumers and regulators will reasonably ask not only whether a system is fair, but why a particular dispatch decision was made and how the fairness constraint influenced it. Developing post hoc explanation methods, for example, attention-weight visualization and Shapley-value attribution adapted to the multi-agent setting, that can articulate VA-HMARL decisions in terms legible to non-specialist stakeholders would substantially strengthen the pathway toward regulatory acceptance and genuine community trust in autonomous energy governance systems.