Energy-Availability-Constrained Deep Reinforcement Learning for Blackout-Aware Task Offloading in Renewable-Powered SDN-MEC Networks

Abstract

Renewable-powered multi-access edge computing (MEC) exposes edge nodes to a failure mode absent from grid-powered systems: a node whose battery is depleted enters a service blackout. We study task offloading in a software-defined networking (SDN)-controlled MEC network with solar-powered, finite-battery servers, and enforce each node’s per-slot energy-availability constraint at the controller at two levels: a per-assignment screening test that prunes the action set of a deep reinforcement learning agent whose state carries battery state-of-charge and a harvest forecast, and a commit-time aggregate admission test that verifies the total energy committed to a node at the allocated frequencies. Policies are evaluated end-to-end, on the fraction of all generated tasks served within their deadline by a node that did not black out, over ten seeds with paired statistics. The aggregate admission test reduces the blackout rate to at most $0.4\%$ of server-slots for every policy it wraps, learned or heuristic, while increasing delivered goodput ($+19\%$ for the learned agent, $+9\%$ for a delay-greedy heuristic). An ablation shows that the learned agent’s $88\%$ blackout reduction relative to an energy-agnostic learner comes from the battery information in its state rather than from the screening test, and that fixed-penalty and Lagrangian soft-constraint baselines do not reach the same operating point. On end-to-end task success, a delay-greedy association under the same admission test delivers more than any learned policy at the reference operating point, and the learned policy overtakes it only under heavy load; we report this, together with the failure of preference conditioning to control the delay–energy trade-off, as limitations of learning in this setting rather than as properties of the admission mechanism.

1. Introduction

Next-generation wireless networks must deliver stringent latency guarantees under increasingly tight energy and sustainability budgets. Multi-access edge computing (MEC) reduces service delay by relocating computation to edge servers near users, and the growing energy footprint of dense edge deployments has made energy sustainability a first-order concern for beyond-5G and 6G systems [1], motivating on-site renewable (solar) harvesting at edge nodes.

Renewable-powered edge nodes introduce a challenge absent from grid-powered MEC: energy availability is intermittent and only partially predictable. A node’s ability to execute offloaded tasks depends not only on its compute capacity but on its instantaneous battery state-of-charge (SoC), which evolves under stochastic harvesting and accumulated load. Offloading decisions that ignore these dynamics can deplete a node and trigger a service blackout, in which the node cannot serve any users, which is an outcome more damaging to quality of service than transient congestion.

Energy-availability limits of harvesting-powered infrastructure have been characterized analytically for cellular base stations, where the fraction of time a node can remain on is a fundamental availability metric [2], and battery-aware association rules that admit a user only at stations able to sustain it have been derived in closed form [3]. Learning-based MEC offloading, by contrast, has mostly treated energy as an objective or a soft penalty; the few works that model on-site renewable supply charge depletion as a cost rather than enforcing availability structurally [4]. Our earlier work on constrained reinforcement learning for MEC offloading [5] enforced capacity and delay budgets in a grid-powered setting and contains no energy, battery, or harvesting model; the present paper differs in the problem addressed, the constraints enforced, and the metrics reported.

We study SDN-controlled offloading in a renewable-powered MEC network in which the controller enforces the per-slot energy-availability constraint of each node at two levels: a per-assignment screening test that prunes the action set of a deep reinforcement learning (RL) agent, and a commit-time aggregate admission test that verifies the total energy committed to a node at the frequencies actually allocated, and refuses the assignments that would exceed it. The learned policy is a PPO [6] actor–critic with SoC and a short-term harvest forecast in its state. Rather than reporting only metrics conditioned on admitted tasks, we evaluate every policy on the fraction of all generated tasks served within their deadline by a node that did not black out, alongside blackout and admitted-load rates, over ten seeds and ten evaluation episodes per seed with paired statistics.

The main contributions of this paper are as follows:

• We formulate blackout-aware offloading in renewable-powered SDN-MEC with battery SoC as a dynamic state, and we enforce the per-slot energy-availability constraint at the controller at two levels: per-assignment screening, and a commit-time aggregate admission test that guarantees the constraint at the committed frequencies. The aggregate test is policy-agnostic: we apply it to learned and heuristic policies alike.
• We evaluate on an end-to-end service metric (task success over all generated tasks, with tasks on a blacked-out node counted as failed) in addition to the conditional delay-violation rate, and we report a four-way ablation (SoC, forecast, screening, and their combination), a fixed-penalty soft-constraint baseline, a Lyapunov-inspired drift-plus-penalty heuristic, and sensitivity to battery capacity, user density, task size, forecast error, and the blackout severity coefficient.
• We find that commit-time aggregate admission reduces the blackout rate to at most $0.4\%$ of server-slots for every policy while increasing delivered goodput ($+19\%$ for the learned agent, $+9\%$ for the delay-greedy heuristic, seed-matched); that the SoC information in the state, rather than the per-assignment screen, is the main driver of the learned agent’s $88\%$ blackout reduction relative to an energy-agnostic learner; and that, on end-to-end service, a delay-greedy heuristic under aggregate admission delivers more tasks than any learned policy at the reference operating point, and the learned policy overtakes it only under heavy load. We explain the last finding through the proportional-sharing resource model and state it as a limitation of learning in this setting rather than as a property of the mechanism.
• We document three negative results: preference conditioning of a single policy neither controlled the delay–energy operating point nor reduced blackouts below an unconditioned policy; a dual-ascent Lagrangian (CMDP) baseline stalled once its multiplier grew; and the neutral treatment of declined users in the delay reward caused systematic under-admission, which a coverage-aware reward term corrects.

The remainder is organized as follows. Section 2 reviews related work. Section 3 presents the system model. Section 4 gives the constrained formulation and the learning algorithm. Section 5 reports the evaluation. Section 6 concludes.

2. Related Work

2.1. Energy-Aware MEC Offloading

Deep reinforcement learning (RL) has been widely applied to wireless resource allocation [7,8] and to MEC offloading, optimizing latency, energy, or their weighted sum, typically treating energy as a single objective or a soft penalty [9,10,11,12]. Constrained formulations enforce capacity or delay budgets during learning. The constrained Markov decision process (CMDP) framework [13] underlies most safe-RL methods, which enforce a cost budget in expectation either through a Lagrangian relaxation with a learned multiplier or through trust-region projections such as constrained policy optimization [14]; our own prior work enforced capacity and delay budgets through constrained double deep Q-learning [5]. Constraint formulations in safe RL differ in whether they are enforced in expectation or instantaneously; the latter, which our per-slot availability test instantiates, is the stricter class [15]. Recent learning-based schemes address hybrid grid-and-renewable supply [16], energy-efficiency maximization under wireless power transfer [17], and multi-layer offloading across integrated networks [18,19]. These approaches assume grid-powered nodes with effectively unbounded supply: they penalize consumption but never face node unavailability, and none models battery SoC as a dynamic constraint or addresses service blackouts.

Recent learning-based offloading schemes confirm this picture. Tang and Wong [20] learn offloading decisions for delay-sensitive tasks with a long short-term memory (LSTM)-augmented deep Q-network and report queueing delay and drop rate; He et al. [21] minimize average task delay under explicit per-task deadlines with multi-agent deep RL in device-to-device MEC; Du et al. [17] and Liu et al. [11] maximize energy efficiency or minimize energy provision in wirelessly powered MEC; and Luo et al. [22] co-manage computing power and battery charging in solar-powered edge servers. Deadlines, energy, and batteries thus all appear in this literature, but as terms of an objective or as constraints on the device side; the server-side availability constraint that a finite on-site battery imposes on the controller’s admission decision, and the service interruption that follows its violation, are not enforced structurally nor reported.

2.2. Service Availability of Harvesting-Powered Infrastructure

Outside the learning literature, the availability of energy-harvesting network nodes has been analyzed directly. Dhillon et al. [2] model the battery of a harvesting-powered base station as a birth–death process and show that the fraction of time a node can remain on—its availability—is a fundamental performance limit, characterizing the region of availability vectors reachable by uncoordinated on/off policies. Parzysz et al. [3] design a power-availability-aware association rule for off-grid small cells with finite batteries: each terminal restricts itself to the set of stations whose battery level, required transmit power, and estimated load from other terminals can sustain service, and closed-form power-outage and coverage probabilities are derived by stochastic geometry. These works establish availability as the primary metric and battery-aware admission as the primary lever, but they address radio access rather than computation offloading, rely on static stochastic-geometry analysis rather than per-slot control, and do not learn. Our commit-time aggregate admission test (Section 4.3) is the per-slot, computation-aware counterpart of the availability-aware association of [3], embedded in a learning loop.

Renewable-aware admission has also been studied in the computing-systems literature. Wiesner et al. [23] propose Cucumber, an admission-control policy for compute nodes with on-site renewable sources that accepts a delay-tolerant job only if probabilistic forecasts of load, consumption and production indicate that it can be completed within its deadline without grid energy; carbon-aware schedulers for serverless functions pursue the same goal at data-center scale [24]. These policies operate on jobs of minutes to hours with a grid as fallback; our commit-time test operates on tasks of tens of milliseconds, with no fallback and a battery that couples successive decisions, and it is placed after a learned policy has proposed its allocation rather than at the entrance of a queue. The idea is the same, namely to admit only what the available energy can complete within the deadline, and we cite it as the antecedent of Section 4.3.

2.3. Multi-Objective and Conditioned RL

Multi-objective RL commonly scalarizes objectives with a fixed weight and trains one policy per operating point. Conditioning a single policy on the objective weights so that it generalizes across the weight simplex has been studied in deep RL, notably through weight-conditioned multi-objective Q-networks [25]; subsequent work trains a single network covering the full preference simplex [26]. Feature-wise linear modulation (FiLM) [27] is a standard mechanism for conditioning a network on auxiliary inputs. We evaluate a FiLM-conditioned variant of our policy and report that it neither controlled the operating point along the trade-off nor matched an unconditioned policy at the delay-only corner (Section 5).

2.4. Renewable-Powered and Energy-Harvesting Edge

Energy harvesting has been studied extensively in wireless networks [28] and, more recently, in MEC, where harvested or wirelessly transferred energy must be balanced against execution demands [11,12,29]. A seminal line of work applies Lyapunov optimization to computation offloading with energy-harvesting devices [30], optimizing time-averaged objectives under stationarity assumptions; related approaches treat harvested energy as an average budget rather than a per-slot, battery-coupled feasibility condition. Closest to our setting, Xu et al. [4] learn workload offloading and server autoscaling for an edge site powered by on-site renewables with a finite battery, using a post-decision-state RL algorithm; depletion is handled as a backup-power cost in the objective, at a single site, without a structural admission test and without reporting node unavailability as a metric. More recent work models the interaction between server power management and battery charging characteristics in solar-powered edge servers [22], applies Lyapunov optimization to energy-harvesting MEC under queue-stability and energy constraints [31], and schedules uploads in solar-powered industrial IoT [32]; related scheduling problems arise in energy-harvesting federated learning [33]. None of these enforces a per-slot admission test on individual assignments, nor reports service continuity as an evaluation metric. To our knowledge, no prior learning-based offloading scheme combines SoC dynamics and harvest forecasting in the state with a structural, per-slot availability test at the controller and reports node unavailability and end-to-end task success as evaluation metrics; Table 1 summarizes the positioning. Table 2 lists the abbreviations used throughout the paper.

Table 1. Positioning of this work against representative prior work.

Work	En.	SoC	Hard	Blk.	Learn.
Dhillon et al. [2]	✓	✓	–	✓	–
Parzysz et al. [3]	✓	✓	∼	✓	–
Mao et al. [30]	✓	✓	∼	–	–
Xu et al. [4]	✓	✓	–	∼	✓
Liu et al. [11]	✓	∼	–	–	✓
Zheng et al. [12]	✓	∼	–	–	✓
Amzil et al. [5]	–	–	✓	–	✓
This work	✓	✓	✓	✓	✓

En.: explicit energy-consumption model. SoC: finite battery with state-of-charge dynamics coupling successive slots; ∼ denotes an energy-availability model without a battery state carried across successive slots (e.g., harvest-then-consume within a slot or a long-term average energy constraint). Hard: constraint enforced structurally (feasible-set restriction) rather than as a soft penalty; ∼ denotes a time-averaged, expected-cost, or static analytical formulation (Lyapunov drift-plus-penalty, CMDP, stochastic-geometry association) rather than a per-slot feasibility test. Blk.: service-continuity (node unavailability) reported as an evaluation metric; ∼ denotes depletion charged as a cost but not reported as a rate. Learn.: learning-based policy. ✓ denotes that the criterion is fully satisfied; – denotes that it is not addressed; ∼ denotes partial satisfaction, with the column-specific meaning stated above.

Table 2. List of abbreviations.

Abbreviation	Definition
CMDP	Constrained Markov Decision Process
CMOS	Complementary Metal-Oxide-Semiconductor
DVFS	Dynamic Voltage and Frequency Scaling
FiLM	Feature-wise Linear Modulation
GAE	Generalized Advantage Estimation
LSTM	Long Short-Term Memory
MEC	Multi-access Edge Computing
NREL	National Renewable Energy Laboratory
NSRDB	National Solar Radiation Database
PC-AC	Preference-Conditioned Actor–Critic
PPO	Proximal Policy Optimization
RL	Reinforcement Learning
SDN	Software-Defined Networking
SINR	Signal-to-Interference-plus-Noise Ratio
SoC	State of Charge

3. System Model

We consider an SDN-controlled MEC network of M solar-powered edge servers serving U mobile users over discrete slots of duration $\tau$ (Figure 1). Each server j integrates an access point, a compute unit, a photovoltaic panel, and a finite battery. A centralized controller observes per-slot channel, task, battery, and forecast state and issues offloading and allocation decisions, following the SDN-controlled edge architecture in which a logically centralized control plane collects telemetry from and programs the edge nodes [34]. Because the communication and delay components follow the standard MEC formulation also adopted in our prior constrained-offloading work [5], we state them compactly and devote the remainder of this section to the energy, harvesting, and battery dynamics that are specific to the renewable-powered setting studied here.

Figure 1. Renewable-powered SDN-MEC architecture. The SDN controller collects per-slot telemetry (channel state, battery state-of-charge, last measured harvest), computes the one-step harvest forecast from it, and issues offloading and resource-allocation decisions. Each edge server integrates an access point, a compute unit, a photovoltaic panel, and a finite battery; the energy path (harvest → battery → computation) and the resulting service-blackout risk (rightmost server, depleted battery) are absent from grid-powered MEC. The energy-availability constraint is enforced at the controller by per-assignment screening of the agent’s action set and by a commit-time aggregate admission test on the committed energy (Section 4.3).

3.1. Communication and Delay (Summary)

Let $x_{u,j}\left(t\right)\in \{0,1\}$ denote the association of user u to server j, with ${\sum }_j{x}_{u,j}\left(t\right)\le 1$. Following the channel and rate model of [5], the uplink rate $R_{u,j}\left(t\right)$ follows the Shannon model, i.e., a logarithmic function of the signal-to-interference-plus-noise ratio (SINR) determined by the allocated bandwidth $W_{u,j}\left(t\right)$, transmit power, and a fading-plus-path-loss gain $G_{u,j}\left(t\right)$, subject to ${\sum }_u{W}_{u,j}\left(t\right)\le W_j^{max}$. The end-to-end delay is the sum of transmission ($B_u\left(t\right)/R_{u,j}\left(t\right)$) and execution (${\kappa }_u\left(t\right)/f_{u,j}\left(t\right)$) terms, with ${\sum }_u{f}_{u,j}\left(t\right)x_{u,j}\left(t\right)\le F_j^{max}$, and an offloaded task is served within its deadline if $D_{u,j}\left(t\right)\le D^{max}$. When the bandwidth or frequency levels requested on a server exceed $W_j^{max}$ or $F_j^{max}$, the controller scales them proportionally so that the budgets hold with equality; each admitted task then receives a share $W_{u,j}\left(t\right)$ and $f_{u,j}\left(t\right)$ of the server for the whole slot (processor sharing), and its execution delay is ${\kappa }_u\left(t\right)/f_{u,j}\left(t\right)$ under that share; the resulting contention among co-located users is the multi-user coupling analyzed in [35]. Unlike [5], we do not model a queueing stage. Two conditions make this reasonable here and bound what it omits. First, each user generates at most one task per slot and every admitted task is either completed within the slot’s allocation or counted as a deadline violation, so no backlog carries over between slots; the processor-sharing model is exact for such slot-synchronous service and conservative with respect to a first-come-first-served queue, which would serve some tasks earlier and others later than the shared rate. Second, the mobility and channel state are resampled per slot and the delay deadline (150 ms) exceeds the slot (10 ms), so a task’s fate depends on the allocation it receives, not on its arrival order within the slot. What the model omits is the burstiness of multi-slot backlogs under sustained overload, which would increase delay violations for every policy equally and which we leave to a queue-aware extension. Congestion here manifests through the shared compute budget and, critically, through the energy coupling developed next.

3.2. Energy Consumption

The energy drawn from server j’s battery in slot t combines computation, reception, and idle terms. Computation follows the dynamic complementary metal-oxide-semiconductor (CMOS) model widely adopted in MEC energy analysis [30], in which processing a workload ${\kappa }_u\left(t\right)$ at frequency $f_{u,j}\left(t\right)$ dissipates ${\xi }_j{\kappa }_u\left(t\right)f_{u,j}{\left(t\right)}^2$, where ${\xi }_j$ is the effective switched-capacitance coefficient. Hence

$$E_j\left(t\right)=\sum _u{x}_{u,j}\left(t\right)\left({\xi }_j{\kappa }_u\left(t\right)f_{u,j}{\left(t\right)}^2+{\varrho }_j{B}_u\left(t\right)\right)+E_j^{\mathrm{idle}},$$

(1)

with ${\varrho }_j$ a per-bit reception cost. The frequency in (1) is the share $f_{u,j}\left(t\right)$ actually allocated to the task after proportional scaling, which models a multi-core node with per-task dynamic voltage and frequency scaling (DVFS) under an aggregate frequency budget $F_j^{max}$: a task allocated a lower clock runs slower and dissipates quadratically less, the standard assumption of DVFS-based offloading models [36,37,38]. A consequence that matters for interpreting the results is that a heavily loaded server serves each task slower but cheaper, so that concentrating load does not, by itself, increase the energy drawn per task; the opposite regime (a fixed-clock processor time-sliced among tasks, with energy per cycle independent of load) is discussed as a limitation in Section 5. The quadratic dependence on $f_{u,j}\left(t\right)$ is central: serving a task faster to meet its deadline costs energy that scales with the square of the clock, whereas the execution delay scales with its inverse. This $E\propto f^2$ versus $D\propto 1/f$ tension is the physical origin of the delay–energy trade-off examined in Section 5.

3.3. Solar Harvesting

Server j harvests $H_j\left(t\right)={\eta }_j{A}_j{c}_j\left(t\right)\overline{I}\left(t\right)\tau$ joules per slot, where ${\eta }_j$ and $A_j$ are the panel efficiency and area, $\overline{I}\left(t\right)$ is a synthetic diurnal irradiance profile with peak irradiance matched to clear-sky midday conditions, and $c_j\left(t\right)\in [0,1]$ is a per-node cloud attenuation following a bounded first-order autoregressive process, $c_j(t+1)=\varphi c_j\left(t\right)+(1-\varphi )\overline{c}+{ϵ}_j\left(t\right)$ clipped to $[0,1]$, with ${ϵ}_j\left(t\right)\sim \mathcal{N}(0,{\sigma }_c^2)$. The profile is a half-sine over a diurnal period of $T_{\mathrm{day}}$ slots with peak irradiance ${\overline{I}}_{max}=800$ W/m², the order of clear-sky midday irradiance at mid-latitudes; $\varphi =0.9$ gives the cloud process a correlation time of $1/(1-\varphi )=10$ slots, ${\sigma }_c=0.1$ sets its slot-to-slot variability, and $\overline{c}=0.7$ makes the mean effective irradiance 560 W/m². The diurnal period is compressed to the episode length ($T_{\mathrm{day}}=200$ slots of 10 ms), so that one episode spans one harvest cycle; the panel area, battery capacity, and idle power are scaled to this compressed cycle so that the ratios that govern the availability constraint (harvest per slot relative to task energy, and battery capacity relative to both) are those of an energy-limited node, while absolute wall-clock durations are not physical. Field deployments of solar-powered base stations, whose panel, battery and load dimensioning and outage statistics are documented in [39,40,41], are the physical counterpart of this operating point. The sensitivity analysis in Section 5 varies battery capacity, user density, task size, and forecast error around this operating point. The profile captures the diurnal structure that drives the availability constraint, while $c_j\left(t\right)$ provides a controllable variability knob; validation against measured irradiance data such as the National Renewable Energy Laboratory (NREL) National Solar Radiation Database (NSRDB) [42] is left to future work.

3.4. Battery Dynamics and the Energy-Availability Constraint

Let $B_j\left(t\right)\in [0,B_j^{max}]$ be the battery state-of-charge at the start of slot t. It evolves as

$$B_j(t+1)=min\left(B_j^{max},{[B_j\left(t\right)+H_j\left(t\right)-E_j\left(t\right)]}^{+}\right).$$

(2)

A service blackout occurs at server j whenever the committed slot energy exceeds what is available, $E_j\left(t\right)>B_j\left(t\right)+H_j\left(t\right)$, leaving the node unable to complete its assigned tasks; every task committed to that node in that slot is therefore counted as failed, in the reward and in every metric of Section 5, and the battery is left empty for the next slot. To counter this, we require, for every server and slot, the energy-availability constraint

$$E_j\left(t\right)\le B_j\left(t\right)+H_j\left(t\right).$$

(3)

This is the defining constraint of our formulation and has no counterpart in grid-powered offloading. Through (2), (3) couples successive slots: energy spent now shrinks the feasible action set later, so respecting it demands anticipation rather than instantaneous greediness. This motivates embedding both $B_j\left(t\right)$ and a one-step harvest forecast ${\hat{H}}_j(t+1)$ in the learning state (Section 4). The forecast is the conditional mean of the AR(1) cloud process, ${\hat{H}}_j(t+1)={\eta }_j{A}_j[\varphi c_j\left(t\right)+(1-\varphi )\overline{c}]\overline{I}(t+1)\tau$, where $c_j\left(t\right)$ is recovered at the controller from the last measured harvest as $c_j\left(t\right)=H_j\left(t\right)/({\eta }_j{A}_j\overline{I}\left(t\right)\tau )$; it therefore uses only telemetry the controller already receives (Section 4.5). Section 5 reports the effect of multiplicative forecast error on blackout and service.

4. Constrained Formulation and Algorithm

4.1. Decision Problem

The controller chooses, each slot, the associations $\left\{x_{u,j}\right\}$, bandwidth $\left\{W_{u,j}\right\}$, and CPU rates $\left\{f_{u,j}\right\}$, subject to the single-association, bandwidth, compute, and energy-availability (3) constraints. We cast this as a sequential decision process and solve it with a learned policy whose action set is restricted by a per-assignment screening test and whose committed allocation is verified by an aggregate admission test, both evaluated at the controller (Section 4.3), rather than by penalizing violations in the reward.

4.2. State, Action, Reward

The state $s_t$ concatenates channel gains, task descriptors, battery SoC $\left\{B_j\left(t\right)\right\}$, and one-step harvest forecasts $\left\{{\hat{H}}_j(t+1)\right\}$; exposing SoC and forecast enables proactive avoidance of near-future scarcity. Each user selects among a top-K candidate server set, with bandwidth and CPU rate discretized into $\left|\mathcal{W}\right|=4$ and $\left|\mathcal{F}\right|=4$ levels, so the per-user action is a (server, bandwidth level, frequency level) triple, plus a “do not offload” option. The per-slot reward scalarizes a delay objective and an energy objective through a preference vector $\omega =({\omega }_{\mathrm{d}},{\omega }_{\mathrm{e}})$ on the 2-simplex:

$$\begin{array}{cc}\hfill r_t& ={\omega }_{\mathrm{d}}{r}_t^{\mathrm{d}}+{\omega }_{\mathrm{e}}{r}_t^{\mathrm{e}},\hfill \\ \hfill r_t^{\mathrm{d}}& ={\displaystyle \frac{1}{U}}[\sum _{u:x_{u,·}\left(t\right)=1}\left(2\mathbb{1}\left\{\mathrm{task}u\mathrm{served}\mathrm{on}\mathrm{time}\right\}-1\right)-\delta \sum _{u:x_{u,·}\left(t\right)=0}1],\hfill \\ \hfill r_t^{\mathrm{e}}& ={\displaystyle \frac{1}{M}}\sum _j{\displaystyle \frac{B_j(t+1)}{B_j^{max}}}-{\displaystyle \frac{\zeta }{M}}\sum _j\mathbb{1}\{E_j\left(t\right)>B_j\left(t\right)+H_j\left(t\right)\},\hfill \end{array}$$

(4)

where “served on time” means $D_{u,j}\left(t\right)\le D^{max}$ and server j did not black out in slot t (Section 3), and $\delta \ge 0$ is a coverage coefficient.

The delay term $r_t^{\mathrm{d}}$ rewards each task served on time with $+1$, charges each admitted task that misses its deadline or is lost to a blackout with $-1$, and charges each user left unserved with $-\delta$, all normalized by U. With $\delta =0$, declining to offload is neutral; we retain $\delta =0$ as the default and show in Section 5 that it induces systematic under-admission, because declining a user is then never worse than risking a violation, and that $\delta =1$ removes that bias: it makes an unserved user exactly as costly as a failed one and turns $r_t^{\mathrm{d}}$ into an affine function of the end-to-end task success rate. The energy term $r_t^{\mathrm{e}}$ averages over the M servers the post-transition SoC $B_j(t+1)$ normalized by capacity, rewarding maintained reserves and discouraging myopic depletion, minus a blackout penalty in which the indicator flags every violation of (3), weighted by a severity coefficient $\zeta =5$; Section 5 reports a sensitivity analysis over $\zeta$.

Unless stated otherwise, the policy is trained and evaluated at the fixed preference $\omega =(1,0)$, i.e., on the delay objective alone, so that the energy component of the reward vanishes, and blackout avoidance is left to the SoC and forecast information in the state and to the structural admission of Section 4.3. A preference-conditioned variant trained with $\omega$ resampled every episode (an extreme preference with probability $0.4$, uniform on the simplex otherwise) is evaluated in Section 5; its shared layers are then exposed to $r_t^{\mathrm{e}}$ during training even when evaluated at $\omega =(1,0)$, which is one reason we do not use it as the main policy.

4.3. Energy-Availability Enforcement: Screening and Aggregate Admission

The controller enforces (3) at two levels, illustrated in Algorithm 1.

• Per-assignment screening (action mask).

For each candidate association $(u,j)$, the controller evaluates the energy that user u would draw if served alone at the lowest frequency level, ${\xi }_j{\kappa }_u\left(t\right)f_{min}^2+{\varrho }_j{B}_u\left(t\right)$, and removes server j from user u’s action set if that energy exceeds $B_j\left(t\right)+H_j\left(t\right)$. This test is a screening rule: it certifies feasibility per assignment, in isolation, at $f_{min}$, and it prunes only assignments that no allocation could serve. Masking infeasible actions before the softmax leaves the policy gradient unbiased and is standard practice in large discrete action spaces [43,44]. It does not guarantee (3): the policy may allocate a frequency above $f_{min}$, whose quadratic energy cost can exceed the battery, and several individually feasible users may accumulate on one node so that their aggregate energy is infeasible. Used alone, this level is not a hard constraint; the ablation in Section 5 shows that, on its own, it reduces the blackout rate of the learned agent only marginally.

• Commit-time aggregate admission.

After the policy has proposed a joint action and the controller has applied proportional scaling, it computes for each server the committed energy $E_j\left(t\right)$ of (1) at the allocated frequencies. If $E_j\left(t\right)>B_j\left(t\right)+H_j\left(t\right)$, it removes the admitted task with the largest energy draw from server j, rescales the remaining allocations, and repeats until (3) holds or the server is empty. The removed users are treated as not offloaded in that slot and contribute $-\delta /U$ to $r_t^{\mathrm{d}}$. By construction, after this step a blackout can occur only if the idle energy alone exceeds the available energy, that is, at a node whose battery is empty and that receives no harvest; (3) is therefore enforced exactly for every served task. The test is applied to whatever policy proposes the joint action, and we report it wrapped around the learned agent and around the greedy heuristics alike. It is the per-slot, battery-coupled analogue of renewable-aware admission control [23]. Its cost is at most ${\sum }_j{n}_j$ evaluations of (1), where $n_j$ is the number of users proposed to server j (Section 4.5).

Algorithm 1 Energy-availability enforcement at the SDN controller (one slot)

1: Input: SoC $\left\{B_j\left(t\right)\right\}$, harvest $\left\{H_j\left(t\right)\right\}$, task descriptors, candidate sets. 2: Screening: for each $(u,j)$ in u’s candidate set, mark $(u,j)$ infeasible if ${\xi }_j{\kappa }_u\left(t\right)f_{min}^2+{\varrho }_j{B}_u\left(t\right)>B_j\left(t\right)+H_j\left(t\right)$. 3: Sample the joint action from the policy over the screened action sets. 4: Apply proportional scaling of bandwidth and frequency on each oversubscribed server. 5: for each server j do 6:     while $E_j\left(t\right)>B_j\left(t\right)+H_j\left(t\right)$ and some user is assigned to j do 7:       Remove the assigned user with the largest ${\xi }_j{\kappa }_u{f}_{u,j}^2+{\varrho }_j{B}_u$; rescale; recompute $E_j\left(t\right)$. 8:     end while 9: end for 10: Commit allocations; update SoC via (2); emit reward (4).

4.4. Actor–Critic Policy and PPO

The policy is a two-layer actor–critic with a factorized per-user output: for each user, a categorical head over the K screened candidates plus “do not offload”, a head over bandwidth levels, and a head over frequency levels, i.e., a branching output whose size grows linearly rather than combinatorially with the number of action dimensions [45]. It is trained with PPO [6] using generalized advantage estimation (GAE) [46], a clipped surrogate objective, and multiple epochs per rollout; one rollout is one episode. Algorithm 2 summarizes training and Figure 2 shows the complete per-slot and training workflow.

Figure 2. Workflow of the proposed scheme. Left lane: the per-slot control loop executed at the SDN controller, with the two enforcement levels (screening before the policy, aggregate admission after it; Algorithm 1). Right lane: the quantities stored and the updates performed only during training (Algorithm 2); the reward is computed from post-transition telemetry and is not needed at inference.

• Preference-conditioned variant.

In the conditioned variant (PC-AC), the shared layers are modulated by the preference vector via feature-wise linear modulation (FiLM) [27]: a small conditioning network maps $\omega$ to per-layer scale and shift parameters $({\gamma }_l\left(\omega \right),{\beta }_l\left(\omega \right))$, and each shared hidden representation $h_l$ is modulated as

$${\tilde{h}}_l={\gamma }_l\left(\omega \right)\odot h_l+{\beta }_l\left(\omega \right),$$

(5)

where ⊙ denotes element-wise (Hadamard) multiplication, so that a single set of policy weights can, in principle, express a family of delay–energy trade-offs indexed by $\omega$. We report this variant for completeness and as a negative result (Section 5); the main policy is unconditioned.

Algorithm 2 Energy-availability-constrained PPO offloading

1: Initialize actor–critic parameters. 2: for each episode do 3:     for each slot t do 4:       Observe state (incl. $\left\{B_j\left(t\right)\right\},\left\{{\hat{H}}_j(t+1)\right\}$). 5:       Screen candidate assignments and sample the joint action (Algorithm 1, steps 2–3). 6:       Apply aggregate admission (Algorithm 1, steps 4–9); execute; observe reward (4). 7:       Update SoC via (2). 8:     end for 9:     Compute GAE advantages; PPO-clip update for $K_{\mathrm{ep}}$ epochs. 10: end for

4.5. Controller Feasibility: Observability, Reward Computation, and Overhead

A learned controller is deployable only if every quantity in its state can be observed by the entity that acts, and every quantity in its reward can be computed by the entity that trains, from information available to them [47]. Table 3 lists, for each quantity used in this paper, its source, the entity that produces it, when it becomes available, and the signaling it requires. Two points deserve emphasis. First, the state is composed entirely of quantities reported to the SDN controller at the start of the slot: channel gains (from the per-slot measurement reports the SDN control plane already collects for association), task descriptors (declared by the users with their offloading requests), and per-server battery SoC and last-slot harvest, two scalars per server per slot. The harvest forecast is not a separate measurement: it is computed at the controller from the last measured harvest through the closed-form conditional mean of Section 3. Second, the reward is computed at the controller after the slot from three post-transition quantities that edge servers report with their next telemetry: the per-task completion outcome (deadline met or not), the new SoC, and a blackout flag. The reward is therefore delayed by one slot with respect to the action, which is immaterial for training (rollouts are processed per episode) and which does not affect inference, where no reward is needed. All quantities marked “simulator” in Table 3 are ground-truth values used only to generate the telemetry; none is read directly by the agent.

Table 3. Observability of state and reward inputs at the SDN controller.

Quantity	Used in	Produced by	Available	Signaling per Slot
Channel gains $G_{u,j}\left(t\right)$	state, screening	user measurement reports	start of slot	$UK$ scalars (existing SDN telemetry)
Task size $B_u\left(t\right)$, cycles ${\kappa }_u\left(t\right)$	state, screening, admission	user request	start of slot	$2U$ scalars (with the request)
Battery SoC $B_j\left(t\right)$	state, screening, admission	server battery monitor	start of slot	M scalars
Last harvest $H_j(t-1)$	forecast	server charge controller	start of slot	M scalars
Forecast ${\hat{H}}_j(t+1)$	state	controller (closed form)	start of slot	none (derived)
Current harvest $H_j\left(t\right)$	admission	server charge controller	start of slot	included above (one-slot lag in practice; see text)
Committed energy $E_j\left(t\right)$	admission	controller from (1)	before commit	none (derived)
Deadline outcome	reward	server task monitor	end of slot	U bits (training only)
New SoC $B_j(t+1)$, blackout flag	reward	server battery monitor	end of slot	M scalars + M bits (training only)
Cloud state $c_j\left(t\right)$, irradiance $\overline{I}\left(t\right)$	–	simulator	–	not observed by the agent

In the simulation, the admission test uses the harvest of the current slot, $H_j\left(t\right)$; in a deployment, the controller would substitute the forecast ${\hat{H}}_j\left(t\right)$ or the last measured harvest, both available at the start of the slot. Because the harvest per slot is small relative to the battery (≤0.56 J against 200 J; see Section 5.1), the difference is a second-order effect, and the forecast-error sensitivity in Section 5 bounds its impact.

• Measured overhead.

Per slot, the controller performs the screening test ($UK$ constant-time checks), one forward pass of the actor–critic, and the aggregate admission loop (at most U evaluations of (1)). We timed these on two commodity platforms with a single CPU thread (Table 4). On a desktop core, the whole per-slot control cost is $0.4$ ms; on a 2017 laptop core, it is $4.9$ ms on average and $6.6$ ms at the 95th percentile, inside the $\tau =10$ ms slot but with a margin of only about a third, which is the figure a deployment on modest controller hardware should budget for. Telemetry parsing is not included; the environment step, which computes the same delay and energy quantities a telemetry pipeline would, is listed as a proxy. The state is $UM+2U+2M+2$ scalars (418 for $U=40$, $M=8$).

Table 4. Measured per-slot controller cost (single thread, PyTorch CPU; mean/p95 over 380 slots).

Component (Mean/p95, ms)	Intel i9-14900K	Intel i5-7300U
Screening mask ($UK$ checks)	0.007/0.007	0.058/0.079
State build + actor–critic forward	0.390/0.437	4.84/6.50
Environment step (delay, energy, battery; proxy for telemetry processing)	0.152/0.167	4.34/6.61
Total mask + inference	0.397/0.445	4.90/6.60

5. Performance Evaluation

5.1. Setup, Metrics, and Baselines

We evaluate with $U=40$ users and $M=8$ solar-powered servers. Each server has finite battery capacity and harvests from a diurnal profile modulated by a bounded AR(1) cloud process; the switched-capacitance coefficient, CPU rate, battery capacity, and harvest scale form a jointly calibrated operating point (Table 5). The frequency–energy coupling ($E\propto f^2$) versus the frequency–delay coupling ($D\propto 1/f$) makes these parameters interdependent. Following the energy model of [30], ${\xi }_j$, $F_j^{max}$, $B_j^{max}$, and the harvest scale are set jointly so that a task served alone at $F_j^{max}$ draws about $2.8$ J, five times the peak harvest per slot ($0.56$ J), while the battery (200 J) holds about 70 such tasks: the deadline is attainable but the node is energy-limited under low irradiance. As stated in Section 3, the diurnal cycle is compressed to the episode, so these values fix ratios, not physical durations.

Table 5. Simulation parameters (jointly calibrated operating point).

Parameter	Value
Users U/servers M	40/8
Area/user speed	$500\times 500$ m/$[1,5]$ m/s
Transmit power $P_u$/noise PSD $N_0$	200 mW/$4\times {10}^{-21}$ W/Hz
Path-loss exponent/reference gain	$3.5$/${10}^{-3}$ at 1 m
Bandwidth per server $W_j^{max}$	10 MHz
CPU rate $F_j^{max}$	$5\times {10}^{10}$ cycles/s
Task input size/intensity	$[0.5,2]$ Mbits/900 cycles/bit
Delay deadline $D^{max}$	150 ms
Slot duration $\tau$/episode	10 ms/200 slots
Candidate-set size K	3
Switched-capacitance ${\xi }_j$	${10}^{-30}$
Reception cost ${\varrho }_j$/idle energy $E_j^{\mathrm{idle}}$	${10}^{-9}$ J/bit/$0.05$ J per slot
Panel efficiency ${\eta }_j$/area $A_j$	$0.20$/$0.5$ m2
Battery capacity $B_j^{max}$/initial SoC	200 J/$50\%$
Peak irradiance ${\overline{I}}_{max}$/mean cloud $\overline{c}$	800 W/m2/$0.7$
Cloud AR(1) coeff. $\varphi$/innovation ${\sigma }_c$	$0.9$/$0.1$
Blackout severity $\zeta$/coverage $\delta$	5/0 (default), $\{0.5,1\}$ (sweep)
Training episodes	2000
Seeds $N_{\mathrm{seeds}}$/evaluation episodes per seed	10/10

We report seven metrics, all computed per evaluation episode and averaged. Three are conditioned on admitted tasks or on server-slots: the delay-violation rate (fraction of admitted tasks whose delay exceeds $D^{max}$), the blackout rate (fraction of server-slots in which (3) is violated), and the blackout-loss rate (fraction of admitted tasks lost because their server blacked out). Four are end-to-end: the task success rate (fraction of all generated tasks served within their deadline by a server that did not black out), the deadline-constrained goodput (bits of such tasks per slot), the service rate (fraction of generated tasks admitted), and the admitted load (bits admitted per slot). The task success rate is the metric on which policies should be compared: a policy that admits few tasks can attain a low delay-violation rate without delivering service. We also report the mean normalized SoC as an energy-reserve indicator.

We compare the following policies. Delay-greedy assigns every user to the feasible candidate with the highest uplink rate at maximum bandwidth and frequency; Energy-Greedy assigns every user to the feasible candidate with the highest SoC; Lyapunov-Greedy assigns each user to the candidate minimizing $D_{u,j}/D^{max}+V{Q}_j{e}_{u,j}/e_{\mathrm{ref}}$, where $Q_j=(B_j^{max}-B_j)/B_j^{max}$ is a normalized energy-deficit virtual queue and $e_{u,j}$ the task energy at maximum frequency; this is the per-slot minimization of a drift-plus-penalty bound in the spirit of [30], reported for $V\in \{0.3,1,3\}$. Energy-Agnostic is the PPO agent with the SoC and forecast blocks of the state zeroed and no screening. PPO-SoC, PPO-SoC+F, and PPO-Screen add, respectively, the SoC block, the SoC and forecast blocks, and the screening mask alone; PPO-Masked adds all three and is the proposed learned policy. Penalty-PPO has the same state as PPO-Masked but no screening and a fixed penalty $\lambda {\overline{\mathbb{1}}}_{\mathrm{blk}}\left(t\right)$ subtracted from the reward, where ${\overline{\mathbb{1}}}_{\mathrm{blk}}\left(t\right)$ is the fraction of servers in blackout, for $\lambda \in \{1,5,20\}$; it is the soft-constraint counterpart of the structural test. CMDP-PPO replaces the fixed $\lambda$ by dual ascent, $\lambda \leftarrow {[\lambda +0.05(\overline{c}-0)]}_0^{50}$ per episode. PC-AC is the FiLM-conditioned variant evaluated at $\omega =(1,0)$. The suffix -agg denotes the same policy wrapped in the commit-time aggregate admission test of Section 4.3. All learned agents share architecture, hyperparameters (Table 6), candidate construction, and a training budget of 2000 episodes; Section 5.2.5 reports the convergence state of each agent at that budget.

Table 6. Policy architecture and PPO hyperparameters (identical for all learned agents).

Hyperparameter	Value
State dimension	$UM+2U+2M+2=418$
Hidden layers	$2\times 128$, ReLU
Per-user output heads	server ($K+1$), bandwidth (4), frequency (4)
Bandwidth levels $\mathcal{W}$ (fraction of $W_j^{max}$)	$\{0.1,0.25,0.5,1.0\}$
Frequency levels $\mathcal{F}$ (fraction of $F_j^{max}$)	$\{0.25,0.5,0.75,1.0\}$
Optimizer/learning rate	Adam/$3\times {10}^{-4}$
Discount $\gamma$/GAE ${\lambda }_{\mathrm{GAE}}$	$0.99$/$0.95$
PPO clip $ϵ$/epochs per rollout $K_{\mathrm{ep}}$	$0.2$/4
Rollout	one episode (200 slots), normalized advantages
Entropy/value coefficients	$0.01$/$0.5$
Gradient-norm clip	$0.5$
FiLM conditioning network (PC-AC only)	$2\to 64\to 2(128+128)$, ReLU
PC-AC preference sampling	extreme w.p. $0.4$, else uniform
Penalty-PPO $\lambda$/CMDP-PPO dual step	$\{1,5,20\}$/$0.05$ (cap 50)
Training wall-clock per agent	≈10 min (one i9-14900K core)

Each seed fixes the network initialization, the training environments, and ten evaluation episodes of 200 slots whose seeds are shared by all policies; results are mean ± standard deviation (sample, $n-1$) over $N_{\mathrm{seeds}}=10$ seeds (five for the sensitivity sweeps, stated where applicable). Because the design is seed-matched, comparisons use paired t-tests with Cohen’s $d_z$, the 95% confidence interval of the mean difference, the number of seeds in which the difference has the reported sign, and a Wilcoxon signed-rank test as a distribution-free check; for non-significant differences we report the achieved power at $\alpha =0.05$. Simulation and training were run with PyTorch 2.14 (CPU) and NumPy 2.5; the same seeds reproduce the same numbers on the two platforms in Table 4.

5.2. Results at the Reference Operating Point

Table 7 reports all policies, Table 8 the seed-matched tests, and Figure 3, Figure 4, Figure 5 and Figure 6 visualize them.

Table 7. Performance at the reference operating point (Table 5): mean ± std over 10 seeds, 10 evaluation episodes per seed. Viol.: delay-violation rate over admitted tasks; Blk.: blackout rate over server-slots; Adm.: admitted load (Mbit/slot); Goodput: bits served within deadline on a non-blacked-out node (Mbit/slot); Success: fraction of all generated tasks so served; SoC: mean normalized battery level. Lyapunov-Greedy is shown at its best V (0.3); Penalty-PPO at $\lambda \in \{1,5,20\}$. Bold: best value in the Viol., Goodput and Success columns.

Method	Viol.	Blk.	Adm.	Goodput	Success	SoC
Delay-Greedy	$0.673\pm 0.015$	$0.127\pm 0.018$	$49.9\pm 0.0$	$10.45\pm 0.72$	$0.284\pm 0.020$	$0.289$
Delay-Greedy+agg	$0.680\pm 0.018$	$0.001\pm 0.000$	$47.8\pm 0.2$	$\mathbf{11}.\mathbf{41}\pm \mathbf{0}.\mathbf{63}$	$\mathbf{0}.\mathbf{307}\pm \mathbf{0}.\mathbf{018}$	$0.289$
Energy-Greedy	$0.885\pm 0.006$	$0.000\pm 0.000$	$50.0\pm 0.1$	$4.61\pm 0.27$	$0.115\pm 0.006$	$0.385$
Lyapunov-Greedy ($V=0.3$)	$0.801\pm 0.012$	$0.094\pm 0.018$	$49.9\pm 0.1$	$6.58\pm 0.49$	$0.176\pm 0.013$	$0.288$
Lyapunov-Greedy+agg	$0.811\pm 0.013$	$0.001\pm 0.000$	$49.0\pm 0.2$	$6.99\pm 0.47$	$0.185\pm 0.013$	$0.289$
Energy-Agnostic	$0.385\pm 0.040$	$0.239\pm 0.031$	$20.3\pm 3.0$	$7.70\pm 0.53$	$0.177\pm 0.014$	$0.173$
PPO-Screen	$0.303\pm 0.081$	$0.186\pm 0.049$	$15.5\pm 2.7$	$7.40\pm 0.85$	$0.165\pm 0.018$	$0.192$
PPO-SoC	$0.194\pm 0.038$	$0.006\pm 0.006$	$8.8\pm 1.9$	$6.32\pm 1.08$	$0.139\pm 0.025$	$0.455$
PPO-SoC+F	$0.168\pm 0.030$	$0.012\pm 0.005$	$9.6\pm 1.2$	$7.14\pm 0.52$	$0.155\pm 0.013$	$0.429$
PPO-Masked	$\mathbf{0}.\mathbf{147}\pm \mathbf{0}.\mathbf{022}$	$0.029\pm 0.016$	$10.7\pm 1.3$	$8.23\pm 0.73$	$0.175\pm 0.016$	$0.348$
PPO-Masked+agg	$0.175\pm 0.032$	$0.004\pm 0.001$	$12.7\pm 0.9$	$9.79\pm 0.48$	$0.211\pm 0.010$	$0.306$
Penalty-PPO ($\lambda =1$)	$0.437\pm 0.075$	$0.116\pm 0.091$	$22.3\pm 5.3$	$8.84\pm 0.82$	$0.210\pm 0.022$	$0.264$
Penalty-PPO ($\lambda =5$)	$0.642\pm 0.044$	$0.162\pm 0.036$	$36.5\pm 3.9$	$8.33\pm 0.39$	$0.213\pm 0.008$	$0.215$
Penalty-PPO ($\lambda =20$)	$0.637\pm 0.061$	$0.171\pm 0.042$	$35.6\pm 4.3$	$8.09\pm 0.65$	$0.207\pm 0.015$	$0.210$
CMDP-PPO	$0.554\pm 0.069$	$0.201\pm 0.052$	$30.3\pm 3.8$	$8.48\pm 0.76$	$0.209\pm 0.017$	$0.195$
PC-AC ($\omega =(1,0)$)	$0.376\pm 0.066$	$0.073\pm 0.083$	$18.1\pm 3.7$	$8.73\pm 0.64$	$0.203\pm 0.017$	$0.306$

Table 8. Seed-matched comparisons at the reference operating point ($n=10$): mean difference, 95% CI, paired t, Cohen’s $d_z$, and number of seeds in which the difference has the sign shown. †: non-significant; achieved power in brackets.

Comparison	Metric	Diff. [95% CI]	t (p)	${\mathit{d}}_{\mathit{z}}$	Seeds
PPO-Masked+agg vs. PPO-Masked	Blackout	$-0.025[-0.037,-0.013]$	$-4.85$ ($9\times {10}^{-4}$)	$-1.53$	10/10
	Success	$+0.036[+0.023,+0.049]$	$6.20$ ($2\times {10}^{-4}$)	$1.96$	10/10
	Goodput (Mbit/slot)	$+1.55[+0.95,+2.16]$	$5.84$ ($2\times {10}^{-4}$)	$1.85$	10/10
Delay-Greedy+agg vs. Delay-Greedy	Blackout	$-0.125[-0.139,-0.112]$	$-21.6$ ($5\times {10}^{-9}$)	$-6.83$	10/10
	Success	$+0.023[+0.021,+0.025]$	$29.1$ ($3\times {10}^{-10}$)	$9.19$	10/10
	Goodput (Mbit/slot)	$+0.96[+0.88,+1.03]$	$28.6$ ($4\times {10}^{-10}$)	$9.05$	10/10
PPO-Masked vs. Energy-Agnostic	Blackout	$-0.210[-0.231,-0.188]$	$-21.8$ ($4\times {10}^{-9}$)	$-6.88$	10/10
	Success	$-0.003[-0.017,+0.012]$ †	$-0.40$ ($0.70$) [0.06]	$-0.13$	7/10
	Goodput (Mbit/slot)	$+0.53[-0.07,+1.13]$ †	$1.99$ ($0.078$) [0.43]	$0.63$	7/10
PPO-SoC vs. Energy-Agnostic	Blackout	$-0.233[-0.255,-0.211]$	$-24.1$ ($2\times {10}^{-9}$)	$-7.62$	10/10
PPO-Screen vs. Energy-Agnostic	Blackout	$-0.053[-0.087,-0.019]$	$-3.55$ ($6\times {10}^{-3}$)	$-1.12$	8/10
PPO-SoC+F vs. PPO-SoC	Blackout	$+0.006[+0.003,+0.009]$	$3.91$ ($4\times {10}^{-3}$)	$1.24$	9/10
	Goodput (Mbit/slot)	$+0.82[+0.12,+1.51]$	$2.66$ ($0.026$)	$0.84$	8/10
PPO-Masked+agg vs. Delay-Greedy+agg	Success	$-0.097[-0.110,-0.083]$	$-16.0$ ($6\times {10}^{-8}$)	$-5.06$	10/10
	Goodput (Mbit/slot)	$-1.62[-2.10,-1.15]$	$-7.74$ ($3\times {10}^{-5}$)	$-2.45$	10/10
	SoC	$+0.017[+0.004,+0.030]$	$3.00$ ($0.015$)	$0.95$	8/10
PPO-Masked vs. PC-AC	Viol.	$-0.229[-0.271,-0.186]$	$-12.1$ ($7\times {10}^{-7}$)	$-3.82$	10/10
	Blackout	$-0.043[-0.107,+0.021]$ †	$-1.54$ ($0.16$) [0.28]	$-0.49$	6/10
	Success	$-0.029[-0.037,-0.020]$	$-7.34$ ($4\times {10}^{-5}$)	$-2.32$	10/10

Figure 3. Task success, blackout rate, and goodput at the reference operating point (mean ± std, 10 seeds).

Figure 4. Ablation of the learned agent’s ingredients (mean ± std, 10 seeds). Screen: per-assignment screening; SoC: battery state in the state; F: harvest forecast; agg: commit-time aggregate admission.

Figure 5. Per-seed blackout rate of the energy-agnostic and masked learners (10 seed-matched runs; paired $t=-21.8$, $p=4\times {10}^{-9}$). Each grey line connects the two blackout rates obtained under one seed, and the blue line marked “Mean” is their average across the ten seeds.

Figure 6. Success–blackout plane (mean ± std over seeds). (a) Policies at the reference operating point: marker shape identifies the policy, filled markers denote the same policy wrapped in commit-time aggregate admission, and the arrows show the effect of adding admission to delay-greedy and to PPO-Masked (blackout collapses, success increases). (b) Parameter sweeps, each drawn as a path in increasing parameter order: fixed penalty $\lambda$, Lyapunov drift weight V (with and without admission), coverage coefficient $\delta$ (with and without admission), and severity coefficient $\zeta$ for the conditioned agent; the reference point delay-greedy+agg is repeated for scale. No penalty, drift weight, or reward coefficient reaches the lower-right corner that aggregate admission attains.

5.2.1. Aggregate Admission Is Policy-Agnostic and Improves Service

Wrapping any policy in the commit-time aggregate admission test brings the blackout rate to at most $0.004$ (learned agent) or $0.001$ (heuristics), the residual being nodes whose idle energy exceeds their empty battery plus harvest. Contrary to the expectation that refusing assignments must cost service, admission increases delivered goodput and task success for every policy it wraps: by $19\%$ and $21\%$ for the learned agent (ten of ten seeds, $d_z=1.85$ and $1.96$), and by $9\%$ and $8\%$ for delay-greedy (ten of ten seeds, $d_z=9.05$ and $9.19$; the per-seed differences are almost constant). The mechanism is direct: a task that would have been lost to a blackout is dropped before it consumes the energy that would have taken the node down, and the tasks that remain are completed. The admission test removes on average $1.5$ assignments per slot from delay-greedy’s 40 and $0.7$ from the learned agent’s $\approx 10$.

5.2.2. What Drives the Learned Agent’s Blackout Reduction

Relative to the energy-agnostic learner (blackout $0.239$), the full PPO-Masked agent reaches $0.029$, an $88\%$ reduction in ten of ten seeds ($d_z=-6.88$, Figure 5). The ablation (Figure 4) attributes this reduction: adding the SoC block alone brings the rate to $0.006$, adding the forecast on top of it to $0.012$, whereas adding the per-assignment screen alone leaves it at $0.186$: a $22\%$ reduction ($d_z=-1.12$, eight of ten seeds) against $98\%$ for the SoC block ($d_z=-7.62$, ten of ten). Adding the forecast to the SoC block raises the blackout rate slightly ($+0.006$, nine of ten seeds) and the goodput by $13\%$ ($+0.82$ Mbit/slot, $d_z=0.84$): the forecast makes the agent marginally less conservative. The information in the state, not the screen, is the driver; the screen prunes only assignments no allocation could serve, which rarely binds at this operating point. Attributing the reduction to the screening test would therefore be wrong: a policy that observes battery state learns to avoid depletion, mostly by declining service (its service rate falls from $0.41$ to $0.18$–$0.21$), while the screen contributes little. The aggregate admission test is what removes the residual.

5.2.3. Soft Constraints Versus Structural Admission

Penalty-PPO has the same information as the masked agent and replaces the screen by a fixed penalty $\lambda$ on the blackout fraction. It reduces the blackout rate from $0.239$ to $0.116$–$0.171$, non-monotonically in $\lambda$ (larger penalties give more blackouts), with a seed-to-seed standard deviation of $0.091$ at $\lambda =1$, which suggests bimodal outcomes across seeds. The behavior differs from the masked agent’s: penalized agents serve 71–$73\%$ of users rather than $21\%$ because, under proportional sharing, a crowded server runs each task slower and cheaper (Section 3); the penalty therefore steers the policy toward concentrating load rather than toward declining it, and the blackout rate plateaus near $0.16$. Task success is $0.21$ in all three cases, the same as the aggregate-admission agent, which attains it at a blackout rate of $0.004$. The dual-ascent variant (CMDP-PPO) is worse: its multiplier grows linearly to $\lambda \approx 21$ over training while the blackout rate stays at $0.20$–$0.23$ in every 200-episode window, and its delay return freezes at $-30$ from episode 400 onward, whereas the same architecture without the penalty reaches $+24$. A monotonically increasing multiplier makes the reward non-stationary and, after advantage normalization, drowns the delay signal; this is the failure mode that PID-controlled multipliers were introduced to correct [48]. We report it as a negative result rather than tune it away, because the comparison that matters is with the structural test, which needs no multiplier.

5.2.4. End-to-End Service: The Heuristic Delivers More

On the metric that counts every generated task, delay-greedy+agg delivers the most ($0.307$, $11.4$ Mbit/slot); the best learned policy, PPO-Masked+agg, delivers $0.211$ and $9.8$ Mbit/slot, less in ten of ten seeds ($d_z=-5.06$ and $-2.45$). The learned agents’ low conditional violation rates ($0.15$–$0.18$ against $0.68$) are obtained by admitting a fifth of the users. Two facts explain this. First, the delay reward (4) with $\delta =0$ charges a missed deadline $-1$ and a declined user 0, so declining is never worse than risking a violation, and the agents learn to decline: this is a property of the reward we specified, not of the environment. Second, under proportional sharing, the bandwidth and frequency levels chosen for an oversubscribed server are renormalized away, so the only decisions that remain are association and admission; association by channel gain is close to optimal for delay, and the aggregate admission test already performs the energy-aware part of admission. The learned policy has little left to improve on, and PPO with a joint action over 40 users and three heads does not find it.

Setting $\delta =1$, which makes an unserved user as costly as a failed one and turns $r_t^{\mathrm{d}}$ into an affine function of the task success rate, tests the first explanation (Table 9). All learned agents then serve 66–$75\%$ of users, PPO-Masked+agg’s success rises from $0.211$ to $0.253$ ($+20\%$), and the ordering among learned agents is preserved, but none reaches delay-greedy+agg’s $0.307$; the goodput of PPO-Masked+agg during training is flat at ≈9 Mbit/slot from the first episode to the last, i.e., PPO does not improve the end-to-end metric over a random policy in this action space. We therefore state the limitation explicitly: at the reference operating point, learning does not add end-to-end service over a channel-greedy association wrapped in the same admission test. Section 5.3 shows where it does.

Table 9. Effect of the coverage coefficient $\delta$ (10 seeds). $\delta =0$ is the default reward; – denotes a condition not evaluated, since the delay-greedy heuristic does not depend on $\delta$.

	$\mathit{\delta }=0$			$\mathit{\delta }=0.5$			$\mathit{\delta }=1$
Method	Blk.	Success	Service	Blk.	Success	Service	Blk.	Success	Service
Energy-Agnostic	$0.239$	$0.177$	$0.41$	$0.215$	$0.214$	$0.62$	$0.179$	$0.230$	$0.72$
PPO-Masked	$0.029$	$0.175$	$0.21$	$0.183$	$0.219$	$0.54$	$0.131$	$0.243$	$0.75$
PPO-Masked+agg	$0.004$	$0.211$	$0.26$	$0.008$	$0.242$	$0.43$	$0.008$	$0.253$	$0.66$
CMDP-PPO	$0.201$	$0.209$	$0.61$	$0.190$	$0.227$	$0.58$	$0.174$	$0.211$	$0.72$
Delay-Greedy+agg (ref.)	$0.001$	$0.307$	$0.96$	–	–	–	–	–	–

5.2.5. Training Budget and Convergence

Figure 7 reports the delay return per training episode. The agents converge at very different speeds. PPO-Masked+agg reaches its plateau within about 300 episodes and PPO-Masked within about 1300, in every seed. The energy-agnostic learner and PC-AC are still improving slowly at episode 2000 (about $+5$ per 500 episodes for the former), so their reported performance is that of a fixed budget rather than of a converged policy. This asymmetry is itself informative: the SoC signal makes the delay-only objective easy to learn, whereas an agent that cannot observe the battery keeps paying for blackouts it cannot anticipate. A longer budget would at best narrow the energy-agnostic learner’s delay-violation gap; it would not change the ablation, in which all SoC-informed agents are compared at plateau, nor the blackout comparison, whose driver is the information in the state. (For PC-AC, only episodes drawn with ${\omega }_{\mathrm{d}}\ge 0.9$ are shown, since its return depends on the sampled preference.)

Figure 7. Learning curves (delayed return per training episode). Left: mean ± std across the 10 seeds on a common episode grid (25-sample moving average). Right: per-seed trajectories. For PC-AC, only the episodes drawn with ${\omega }_{\mathrm{d}}\ge 0.9$ (≈490 of 2000) are comparable to the delay-only agents; they are plotted at their true episode index, hence the sparser curve, and the agent was trained for the same 2000 episodes as the others.

5.3. Sensitivity Analysis

Table 10 and Figure 8 vary battery capacity, user density and task size around the reference point, with five seeds per condition (the reference column is the ten-seed value). All conditions run are reported.

Table 10. Sensitivity to battery capacity, user density and task size (task success/blackout rate; mean over 5 seeds, reference column over 10). Bold: best success in the column.

	${\mathit{B}}^{max}$ (J)			Users U			Task Size ×
Method	100	200	400	20	40	60	0.5	1	2
Delay-Greedy	0.201/0.300	0.284/0.127	0.327/0.024	0.425/0.315	0.284/0.127	0.166/0.036	0.744/0.008	0.284/0.127	0.024/0.347
Delay-Greedy+agg	0.260/0.003	0.307/0.001	0.331/0.000	0.515/0.001	0.307/0.001	0.170/0.000	0.745/0.000	0.307/0.001	0.039/0.002
Energy-Agnostic	0.108/0.527	0.177/0.239	0.247/0.024	0.291/0.187	0.177/0.239	0.127/0.246	0.574/0.000	0.177/0.239	0.023/0.585
PPO-Masked	0.125/0.199	0.175/0.029	0.253/0.002	0.330/0.022	0.175/0.029	0.115/0.055	0.571/0.000	0.175/0.029	0.024/0.328
PPO-Masked+agg	0.157/0.007	0.211/0.004	0.255/0.000	0.420/0.004	0.211/0.004	0.142/0.007	0.581/0.000	0.211/0.004	0.051/0.002

Figure 8. Task success (top) and blackout rate (bottom) versus battery capacity, user density, and task size (mean ± std; 5 seeds per off-reference condition).

Three regularities hold across the nine conditions. First, aggregate admission keeps the blackout rate at or below $0.007$ everywhere, while every other mechanism degrades as energy becomes scarcer: at $B^{max}=100$ J the energy-agnostic learner blacks out in $53\%$ of server-slots, delay-greedy in $30\%$, and the screening-only agent becomes unstable across seeds ($0.199\pm 0.189$). Second, the ordering on task success is stable in the easy regimes: with abundant energy ($B^{max}=400$ J) or light load (task size $\times 0.5$), every policy converges to a similar success rate and blackouts vanish for all, so the choice of policy is immaterial there; at the reference point and below it, delay-greedy+agg leads. Third, the learned agent under aggregate admission overtakes the heuristic exactly where the problem becomes hard: at task size $\times 2$, its success rate is $0.051$ against $0.039$ and its goodput $3.9$ against $2.3$ Mbit/slot ($+67\%$, five of five seeds, $d_z=2.53$), and at $U=60$, it delivers $9.8$ against $8.5$ Mbit/slot ($+15\%$, five of five seeds) with a lower task count. Under heavy load, being selective (serving fewer, larger tasks that fit) pays, and the learned policy is the only one that is selective. This is the regime in which the learning component is justified; at the reference operating point it is not, and we do not claim otherwise.

5.4. Forecast Error

Figure 9 evaluates the trained forecast-using agents under multiplicative Gaussian forecast error with relative standard deviation ${\sigma }_f\in \{0.1,0.3,0.5\}$, without retraining. No metric of any agent moves by more than $0.001$ up to ${\sigma }_f=0.5$. Taken with the ablation, where adding the forecast to the SoC block changed the blackout rate from $0.006$ to $0.012$, this shows that the one-step forecast contributes little beyond the SoC in this setting: the harvest per slot is at most $0.28\%$ of battery capacity, so a one-slot look-ahead carries almost no information that the current SoC does not. We keep the forecast in the state because it costs nothing (it is derived from telemetry the controller already has) and because a longer horizon or a larger harvest-to-capacity ratio would make it matter, but we do not claim a benefit from it here.

Figure 9. Blackout rate and task success under multiplicative forecast error (evaluation only, 10 seeds).

5.5. Severity Coefficient and Preference Conditioning

The severity coefficient $\zeta$ acts only when ${\omega }_{\mathrm{e}}>0$. We therefore sweep it on the preference-conditioned agent evaluated at $\omega =(0.5,0.5)$ (five seeds): the blackout rate is $0.006\pm 0.011$ at $\zeta =1$, $0.077\pm 0.100$ at $\zeta =5$, and $0.165\pm 0.078$ at $\zeta =20$, with task success $0.147$, $0.193$ and $0.197$. A larger blackout penalty thus yields more blackouts, for the same reason as the fixed-penalty baseline: it pushes the policy toward crowding servers. The magnitude of a penalty is not a reliable control knob for availability in this model, which is the practical argument for the structural test.

The preference-conditioned agent (PC-AC) at $\omega =(1,0)$ has a higher conditional violation rate than the unconditioned masked agent ($0.376$ versus $0.147$, ten of ten seeds), a blackout rate of the same order but with three times the seed-to-seed spread ($0.073\pm 0.083$), and a higher service and task-success rate ($0.203$ versus $0.175$), because its shared layers were trained on the energy reward in the episodes drawn with ${\omega }_{\mathrm{e}}>0$. It did not provide reliable control of the operating point along $\omega$: the $\zeta$ sweep above shows blackout varying by an order of magnitude at a fixed $\omega =(0.5,0.5)$ across a coefficient that should only rescale one term. This is consistent with reports that a single preference-conditioned policy often fails to recover the full front [49]; we retain PC-AC as a documented variant and not as the main policy.

5.6. Limitations

Four limitations bound the claims. (i) Under proportional sharing, crowding a server lowers its energy per task (Section 3); with a fixed-clock processor time-sliced among tasks, energy per task would be load-independent; serving every user would be far more expensive, and the admission decision, where the learned policy is selective, would carry more weight. Our results are for the DVFS regime. (ii) The diurnal cycle is compressed to the episode, so the results fix energy ratios, not durations; the $B^{max}$ sweep spans a factor of four in the harvest-to-capacity ratio. (iii) PPO with a joint action over 40 users did not improve the end-to-end metric during training; the difficulty of large discrete action spaces and the action-space shaping techniques developed for them [50] are the natural next step and may change the comparison with the heuristic. (iv) No queueing across slots is modeled (Section 3).

6. Conclusions

We studied blackout-aware task offloading in renewable-powered SDN-MEC networks, where finite batteries and intermittent solar supply expose edge nodes to service blackouts absent from grid-powered systems. The energy-availability constraint of each node is enforced at the SDN controller at two levels: a per-assignment screening test that prunes the action set of a PPO agent whose state carries battery state-of-charge and a harvest forecast, and a commit-time aggregate admission test that verifies the total energy committed to a node at the allocated frequencies and refuses what would exceed it. Over ten seeds and an end-to-end evaluation that counts every generated task, the aggregate test reduces the blackout rate to at most $0.4\%$ of server-slots for every policy it wraps and increases delivered goodput ($+19\%$ for the learned agent, $+9\%$ for a delay-greedy heuristic). The learned agent’s $88\%$ blackout reduction relative to an energy-agnostic learner is driven by the battery information in its state, not by the screening test, which the ablation shows to be weak on its own. On end-to-end task success, a delay-greedy association wrapped in the same admission test delivers more than any learned policy at the reference operating point; the learned policy overtakes it only under heavy load, where selectivity pays. We reported three negative results: preference conditioning of a single policy did not control the operating point; a dual-ascent CMDP baseline stalled; and the neutral treatment of declined users in the delay reward caused systematic under-admission, which a coverage-aware term corrects without changing the ordering of the learned agents. Future work should replace the per-task DVFS energy model with a fixed-clock alternative, in which admission becomes the dominant decision, adopt factorized action models that let the learned policy improve the end-to-end metric, and validate the harvest model against measured irradiance.