A Multi-Timescale Control Framework for Energy and SLA-Aware O-RAN Network Slicing

Abstract

The transition toward Open Radio Access Network (O-RAN) architecture has enabled unprecedented intelligence and flexibility in 5G and 6G network slicing. However, a fundamental challenge remains in managing the tension between radio unit energy efficiency and the strict Service Level Agreement (SLA) requirements of Ultra-Reliable Low-Latency Communication (URLLC) slices, particularly under highly dynamic traffic conditions. Existing O-RAN approaches suffer from a timescale conflict where Non-Real-Time (Non-RT) policy planners optimize for long-term energy but fail to react to rapid traffic surges, while Near-Real-Time (Near-RT) controllers prioritize reliability at the cost of significant energy over-provisioning. To address this, we propose H-RLS, a hierarchical multi-timescale framework that decouples control into a Non-RT Proximal Policy Optimization (PPO) agent for strategic, energy-aware policy planning and a Near-RT Recursive Least Squares (RLS)-assisted xApp. By predicting millisecond-level delay risks, the xApp acts as a mathematically constrained safety net, applying bounded tactical adjustments when critical SLA violations are detected. Extensive evaluations across dynamic traffic transitions demonstrate that H-RLS maintains zero SLA violations. By actively preventing resource over-provisioning, the framework achieves the lowest composite Energy-SLA cost across all tested regimes, significantly minimizing dynamic power consumption while preserving Enhanced Mobile Broadband (eMBB) service integrity.

1. Introduction

The evolution toward 6G wireless networks demands unprecedented flexibility to support an explosion of heterogeneous services, most notably high-throughput Enhanced Mobile Broadband (eMBB) and strict Ultra-Reliable Low-Latency Communication (URLLC) [1,2]. Concurrently, the Open Radio Access Network (O-RAN) architecture introduces programmable intelligence via the RAN Intelligent Controller (RIC) framework, creating a disaggregated platform where control functions can be explicitly optimized [3,4,5]. However, realizing intelligent network slicing in O-RAN presents a critical challenge in balancing base station energy efficiency with the strict Service Level Agreements (SLAs) of URLLC traffic [6,7,8,9]. Aggressive energy conservation often leads to unacceptable latency, whereas strict SLA protection typically requires resource over-provisioning that degrades both overall energy efficiency and eMBB performance [10,11].

This tension is highly exacerbated by dynamic, non-stationary edge traffic, such as bursty loads and sudden flash-crowd events [12,13,14]. Under these conditions, traditional resource allocation models, including deterministic mathematical programming and iterative heuristic algorithms, suffer from fundamental operational bottlenecks, as their combinatorial nature renders optimal solutions impractical in real-time environments [15]. These conventional approaches exhibit polynomial or exponential time complexity and rely heavily on pre-calibrated traffic distributions, leaving them highly vulnerable to dynamic congestion.

To overcome these computational limits, recent literature has increasingly turned to Deep Reinforcement Learning (DRL) for Radio Resource Management (RRM), typically embedding intelligence into a single operational layer [16,17,18]. However, managing highly conflicting objectives within a unified control loop fundamentally limits system scalability and convergence stability. As demonstrated by recent studies on dynamic O-RAN environments, standalone DRL agents struggle to balance exploration and exploitation under rapidly changing traffic patterns [19]. This introduces an inherent timescale conflict: a Non-Real-Time (Non-RT) controller effectively optimizes long-term energy efficiency but cannot react on time to mitigate sudden URLLC queue surges. Conversely, a Near-Real-Time (Near-RT) controller can respond rapidly to SLA risks but tends to overreact to transient noise, sustaining high-energy modes longer than necessary and wasting resources.

Recognizing these limitations, recent studies have explicitly mapped optimization problems across distinct temporal granularities, utilizing hierarchical or joint slicing architectures [20,21]. While these frameworks successfully distribute computational complexity, they largely rely on stationary traffic assumptions. During severe non-stationary transition events, purely Non-RT agents fail to mitigate immediate queue pressure, while standalone Near-RT learning agents consume excess dynamic energy. Consequently, existing architectures cannot simultaneously satisfy strategic energy optimization and instantaneous service reliability.

To address this fundamental gap, we propose H-RLS, a hierarchical multi-timescale O-RAN slicing framework. H-RLS explicitly decouples control logic across the Non-RT and Near-RT RICs, combining long-term strategic energy minimization with strictly bounded, near-real-time resource adaptation. The key contributions of this paper are summarized as follows:

• We develop a hierarchical network slicing framework aligned with O-RAN architecture, separating the Non-RT resource optimization controller from the Near-RT adaptive controller.
• We design a PPO-based rApp that generates nominal PRB allocations and energy-mode decisions, enabling long-term optimization of the energy-efficiency and SLA trade-off.
• We introduce a lightweight RLS-assisted xApp that utilizes Near-RT telemetry to identify potential URLLC performance degradation and apply bounded adjustment actions. The mechanism preserves the nominal energy-aware policy while providing rapid adaptation to traffic fluctuations.
• We evaluate the proposed framework under challenging within-episode traffic transitions, including bursty traffic, flash-crowd events, and demand-shift phases. The results demonstrate the effectiveness of hierarchical multi-timescale control in balancing energy efficiency, URLLC reliability, and eMBB performance under dynamic network conditions.

The remainder of this paper is organized as follows. Section 2 reviews the related work on DRL-based resource allocation and hierarchical control and discusses the limitations of existing approaches. Section 3 details the system model and formulates the constrained energy-SLA optimization problem. Section 4 presents the proposed H-RLS framework. Section 5 analyzes the computational complexity and signaling overhead of the proposed framework. Section 6 evaluates the performance of H-RLS and compares it with benchmark methods under both stationary and non-stationary traffic scenarios. Finally, Section 7 concludes the paper and discusses future research directions.

2. Related Work

Efficient RAN slicing at the network edge has attracted significant attention to balance resource utilization, energy efficiency, and service quality [2,22]. This section reviews the evolution of resource allocation from traditional heuristics to hierarchical intelligent control, highlighting the limitations that motivate the proposed approach.

2.1. Traditional Optimization and Heuristic Resource Allocation

Prior to the extensive adoption of AI-driven control, resource allocation in network slicing was largely formulated via predefined mathematical programming. For instance, Harutyunyan et al. [23] formulated network slicing as a Mixed-Integer Linear Programming (MILP) problem, employing deterministic heuristics to approximate optimal placement and minimize migration overhead. To bypass centralized computational bottlenecks, Xiao et al. [24] modeled inter-operator slicing as an overlapping coalition formation game to enable distributed resource sharing. Addressing dense physical deployments, Fayad and Cinkler [25] leveraged Genetic Algorithms (GA) to optimize joint user and power allocation in millimeter-wave networks, while Girycki et al. [26] proposed a heuristic delayed-scheduling mechanism to bound packet delays and maximize energy efficiency in cell-free architectures. Furthermore, Fryganiotis et al. [27] applied Model Predictive Control (MPC) for dynamic slice admission, operating over a receding time horizon based on deterministic traffic forecasts.

While these control-theoretical schemes provide provable convergence and optimal baselines, they suffer from fundamental operational bottlenecks. Iterative solutions leveraging MILP, MPC, or game theory exhibit polynomial or exponential time complexity and scale poorly as network density increases. Additionally, deterministic heuristics rely heavily on pre-calibrated traffic distributions, leaving them highly vulnerable to dynamic traffic congestion.

2.2. Single-Timescale DRL Solution

Recognizing the inherent rigidity of the aforementioned heuristics, recent work has adopted DRL to automate Radio Resource Management (RRM). The majority of these approaches condense the learning intelligence into a single, unified execution layer. For example, Dorcheh et al. [28] developed a single-agent PPO framework to dynamically allocate Physical Resource Blocks (PRBs) across heterogeneous slices, while Sever et al. [17] implemented a Deep Q-Network (DQN) xApp to adjust inter-slice resource allocation weights. Expanding on this, Sohaib et al. [18] provided a comprehensive benchmark of multiple DRL agents operating within a unified loop to balance cross-slice Quality of Service (QoS). Beyond basic PRB allocation, researchers have augmented these single-layer agents with Graph Neural Networks (GNNs) for joint functional split selection [29], explored sequential single- and multi-agent formulations for spectrum sharing [30], and designed partially observable stochastic games [31] to capture structural dependencies.

Concurrently, several studies have attempted to integrate energy-saving mechanisms directly into these standalone controllers. Eskandari et al. [32] utilized Soft Actor–Critic (SAC) to optimize transmission power, while Mouawad et al. [33] executed dynamic Radio Unit (RU) sleep-state adjustments at rapid 1 ms intervals. To manage the instability of exploring these environments, Sherif et al. [34] introduced policy filtration, and Raftopoulos et al. [8] embedded time-variant latency boundaries directly into the PPO state space.

While these single-layer formulations successfully automate baseline resource allocation, embedding disparate objectives into a unified Markov Decision Process (MDP) triggers an exponential expansion of the state-action space. Consequently, these standalone agents suffer from severe exploration inefficiencies and reward sparsity when deployed outside of stable, simulated environments. By compressing conflicting temporal requirements into one objective function, the agent is mathematically forced to compromise; as demonstrated by the aforementioned studies, this inevitably restricts the agent to a suboptimal policy that continuously over-provisions dynamic power to avoid instantaneous SLA penalties.

2.3. Multi-Timescale and Joint Slicing Architectures

To alleviate the severe computational burden and convergence instability of single control loops, recent architectures partition decision-making across multiple temporal granularities. Truong et al. [20] decoupled network slicing into long-term admission at the Near-RT RIC and short-term transmission power optimization at the Distributed Unit (DU). Qiao et al. [21] introduced a Parallel Hierarchical DRL-based Resource Allocation (PHDRA) algorithm to isolate large-timescale inter-slice planning from small-timescale intra-slice operations. This two-level structure is further validated by frameworks that separate GCN-DRL-guided near-real-time slicing from real-time MAC scheduling [35], couple coarse-granularity DDPG agents with proportional-fair schedulers [6], and apply Lyapunov-aided weighted DRL for power constraints [36]. Furthermore, to explicitly target energy consumption, multi-loop configurations have been proposed by Xu et al. [4] using Multi-Agent RL (MARL) for base station switching, Ho et al. [37] across standardized O-RAN control loops, and Salvat Lozano et al. [38] via Advanced Sleep Mode (ASM)-aware schedulers.

As synthesized in

Table 1. Feature comparison of the Proposed H-RLS Framework against existing literature.

Ref.	Hierarchical Control	Non-RT Stability	Near-RT Adaptation	Low Edge Overhead	Core Algorithm
[25]	✗	✗	✓	✗	Genetic Algorithm (GA)
[27]	✗	✗	✓	✗	Model Predictive Control (MPC)
[28]	✗	✗	✓	✗	PPO
[29]	✗	✓	✗	✗	Graph-Augmented PPO (GPPO)
[4]	✓	✓	✓	✗	Multi-Agent RL (MARL)
[21]	✓	✓	✓	✗	D3QN + TD3
[Our Work]	✓	✓	✓	✓	PPO + RLS

, although distributing intelligence across hierarchical layers successfully reduces computational load during steady-state operations, these dual-loop systems remain critically vulnerable to dynamic traffic transitions. The fundamental weakness in existing hierarchical designs lies in their lower-layer execution logic. Frameworks deploying a secondary DRL agent at the Near-RT timescale inevitably suffer from delayed convergence during unseen flash-crowd events, as the lower agent must blindly explore the new state distribution before stabilizing. Conversely, frameworks relying on static proportional-fair schedulers at the lower layer cannot preemptively scale resources before physical queues overflow.

The proposed H-RLS framework diverges from these conventional multi-loop designs by resolving the edge computational bottlenecks highlighted in Table 1. Rather than relying on a secondary learning agent that requires unconstrained exploration or a rigid heuristic, H-RLS integrates a strategic Non-RT PPO policy planner with a lightweight, stateful Recursive Least Squares (RLS) predictor at the Near-RT layer. By mathematically predicting imminent delay risks at millisecond resolution and strictly bounding the resulting tactical adjustments, H-RLS consistently maintains SLA preservation during severe traffic fluctuations without compromising the energy efficiencies established by the macro-policy.

3. System Model and Problem Formulation

This section formalizes the network environment by capturing the dynamic relation among slice-specific traffic demands, queue evolution, and load-dependent power consumption. Building upon this foundation, we formulate the hierarchical control challenge as a constrained multi-objective optimization problem that balances long-term energy conservation with strict, short-term SLA protection. The key notation used in the system model is summarized in

Table 2. A summary of the system model’s important notation and definitions.

Notation	Definition
S	Set of supported network slices, S = {e, u}
$e, u$	eMBB and URLLC slice representation
$t$	Discrete Near-RT time step
$k$	Discrete Non-RT time step
$T$	Total number of time steps in one simulation episode
$B_{total}$	Total available Physical Resource Block (PRB) budget
$b_s\left(t\right)$	Number of PRBs physically allocated to slice $s$ at timestep $t$
$m\left(t\right)$	Executed discrete energy-saving mode at time $t$
$b_s^{min}, b_s^{max}$	Minimum and maximum PRB allocation bounds for slice $s$
$A_s\left(t\right)$	Arrival traffic volume for slice $s$ at timestep $t$
${\eta }_s$	Spectral efficiency constant (bits per PRB) for slice $s$
$k_{m\left(t\right)}$	Service capacity multiplier determined by energy mode $m\left(t\right)$
$C_s\left(t\right)$	Service capacity for slice $s$
${\mu }_s\left(t\right)$	Actual volume of traffic served for slice $s$
$Q_s\left(t\right)$	Queue backlog of slice $s$ at time $t$
$Q_s^{max}$	Maximum buffer capacity for slice $s$
$Q_u^{th}\left(t\right)$	Critical queue threshold for the URLLC slice
$D_u\left(t\right)$	Queue-based delay proxy for URLLC traffic
$D_u^{max}$	URLLC delay threshold
$v_u\left(t\right)$	Binary indicator for an SLA violation at step $t$
$V_u$	Aggregated URLLC delay violation ratio over the episode
${ϵ}_u$	Maximum acceptable URLLC violation ratio over the episode
$R_e\left(t\right)$	Achieved service rate (throughput) for the eMBB slice
$R_e^{target}$	Target throughput required to fully satisfy the eMBB slice
${\rho }_e\left(t\right)$	Normalized throughput satisfaction ratio for eMBB slice
${\overline{\rho }}_e$	Average eMBB throughput satisfaction over the episode
${\rho }_e^{min}$	Minimum acceptable ratio
$m\left(t\right)$	Executed discrete energy-saving mode at time $t$
$P\left(t\right)$	Total base station power consumption at time $t$
$P_{ba}$	Static baseline power consumption of the base station
$P_{load}$	Maximum load-dependent dynamic power
${\rho }_{load}\left(t\right)$	Normalized traffic load ratio of the cell
${\alpha }_{m\left(t\right)}$	Dynamic power scaling factor determined by $m\left(t\right)$
${\overline{b}}_u\left(k\right)$	Nominal URLLC PRB budget planned by rApp PPO agent
$\overline{m}\left(k\right)$	Nominal energy mode planned by the rApp PPO agent
${\Delta b}_{max}, {\Delta \mathrm{m}}_{max}$	Maximum allowable bounds for Near-RT xApp tactical adjustments
$\pi$	Control policy governing the resource allocation and energy mode
$r\left(t\right)$	Immediate RL reward function observed at time $t$
$J_{\pi }$	Expected cumulative RL learning objective
$C_{Energy.SLA}$	Composite evaluation cost metric

.

3.1. System Model

We consider an O-RAN compliant next-generation NodeB (gNB) serving heterogeneous traffic demands, specifically focusing on an eMBB slice and URLLC slice, denoted by the set $S=\left\{e, u\right\}$. To isolate the macroscopic effects of O-RAN RIC decisions and maintain computational scalability, the system operates at the slice-level PRB budget abstraction, avoiding user-level packet scheduling complexity. The system evaluates control action over discrete time steps $t \in \{1, 2, \dots , T\}$, corresponding to the Near-RT RIC execution granularity. The total available PRB budget, $B_{total}$:

$$b_u\left(t\right)+ b_e\left(t\right)\le B_{total}$$

(1)

The incoming traffic volume $A_s\left(t\right)$ for slice $s$ is drawn from a phase-dependent distribution governed by a dynamic transition state $z\left(t\right)$, which encompasses diverse network conditions: normal, bursty, flash-crowd, and demand shifting. As for traffic that cannot be immediately served is buffered in a finite queue. The theoretical service capacity of slice $s$ is proportional to allocated PRBs and the active energy saving mode, calculated as:

$$C_s\left(t\right)= {\eta }_s{b}_s\left(t\right)k_{m\left(t\right)}$$

(2)

Here, ${\eta }_s$ is the spectral efficiency constant (bits per PRB) and $k_{m\left(t\right)}$ is the service capacity multiplier determined by the active energy mode $m\left(t\right)$. The actual served traffic ${\mu }_s\left(t\right)$is limited by the current queue backlog $Q_s\left(t\right)$ and new arrivals, yielding:

$${\mu }_s\left(t\right)=\mathit{min}\left(Q_s\left(t\right)+A_s\left(t\right), C_s\left(t\right)\right)$$

(3)

The queue backlog for the subsequent step is updated and capped by the maximum buffer size:

$$Q_s\left(t+1\right)=min\left(Q_s^{max},{\left[Q_s\left(t\right)+A_s\left(t\right)-{\mu }_s\left(t\right)\right]}^{+}\right)$$

(4)

where ${\left[x\right]}^{+}=max(x,0)$.

The system translates physical queue states into service-level metrics. URLLC performance is measured via a queue-based delay proxy, penalizing any instance where latency exceeds the strict SLA limit; eMBB performance is measured by how well the delivered throughput matches a predefined target.

Let $Q_u^{+}\left(t\right)\triangleq max\left(0, Q_u\left(t\right)+A_u\left(t\right)-{\mu }_u\left(t\right)\right)$ denotes the remaining URLLC backlog immediately after service. The URLLC delay proxy is calculated as:

$$D_u\left(t\right)={\displaystyle \frac{Q_u^{+}\left(t\right)}{\mathit{max}\left(C_u\left(t\right), ϵ\right)}}\Delta t$$

(5)

where $ϵ$ is an arbitrarily small positive constant introduced to ensure numerical stability and prevent division by zero during periods of complete resource starvation.

This queue-based proxy leverages a fluid-flow approximation model to estimate the upper bound of the clearing time for buffered packets. It is important to note the strict timescale decoupling inherent in the O-RAN architecture. While the Near-RT RIC operates at a 10 ms control interval, physical packet scheduling is executed by the MAC layer at the O-DU at sub-millisecond granularities. Therefore, satisfying a 5 ms latency SLA does not necessitate a 5 ms control loop at the RIC. Instead, the RIC maintains the SLA by provisioning a sufficient service capacity $C_u\left(t\right)$ so that the underlying O-DU scheduler has the necessary spectrum to drain the remaining queue backlog $Q_u^{+}\left(t\right)$ within the 5 ms threshold.

The delay SLA violation indicator and total episode violation ratio are:

$$v_u\left(t\right)=\left\{\begin{array}{c}1, 1000D_u\left(t\right)>D_u^{max}\\ 0, 1000D_u\left(t\right)\le D_u^{max}\end{array}\right.$$

(6)

$$V_u={\displaystyle \frac{1}{T}}\sum _{t=1}^T{v}_u\left(t\right)$$

(7)

For the throughput-oriented eMBB slice, performance is evaluated by the volume of data successfully delivered relative to a predefined SLA target. Let $R_e\left(t\right)$ denote the instantaneous achieved service rate, calculated as $R_e\left(t\right)={\eta }_e{b}_e\left(t\right) k_{m\left(t\right)}$, which represents the actual data volume processed during step $t$. Let $R_e^{target}$ represent the ideal throughput required to fully satisfy the eMBB users. The eMBB throughput satisfaction ratio, ${\rho }_e\left(t\right)$, normalizes the achieved performance into a metric between 0 and 1:

$${\rho }_e\left(t\right)=min\left({\displaystyle \frac{R_e\left(t\right)}{R_e^{target}}}, 1\right)$$

(8)

In a shared O-RAN environment, the total power consumption of the base station comprises a static, traffic-independent baseline and a load-dependent dynamic component. The proposed controller actively optimizes dynamic power expenditure by manipulating the physical resource allocation and selecting discrete energy-saving configurations. Let $m\in \left\{0, 1, 2\right\}$ denote the active energy mode selected by the Near-RT controller at time $t$, corresponding to performance, balanced, and deep energy-saving states, respectively. The instantaneous power consumption is modeled as:

$$P\left(t\right)=P_{ba}+{\alpha }_{m\left(t\right)}{P}_{load}{\rho }_{load}\left(t\right)$$

(9)

Specifically, the normalized traffic load ratio ${\rho }_{load}\left(t\right)$ is defined as the ratio of actually served traffic to total available service capacity across all slices:

$${\rho }_{load}\left(t\right)={\displaystyle \frac{{\mu }_e\left(t\right)+{\mu }_u\left(t\right)}{C_e\left(t\right)+C_u\left(t\right)+ϵ}}$$

(10)

where $ϵ$ is a small stabilization constant to prevent division by zero.

The total energy consumed over an entire transition episode is the integral of the power over time:

$$E_{total}=\sum _{t=1}^{T}P\left(t\right)\Delta t$$

(11)

3.2. Problem Formulation

To bridge the gap between long-term energy efficiency and the SLA protection, the control action is split into two hierarchical components: a Non-RT policy planner and a Near-RT bounded adjustment.

The policy planner operates at the decision index $k$ and defines the baseline URLLC PRB budget $\overline{b_u}\left(k\right)$ and the nominal energy-saving mode $\overline{m}\left(k\right)$. To react to rapid, non-stationary traffic transitions, real-time adjustments ${\Delta b}_u\left(t\right)$ and $\Delta m\left(t\right)$ are computed at timestep $t$. Therefore, the final control action executed by the base station at time step $t$ combines the nominal energy plan with the Near-RT adjustment. The executed URLLC PRB allocation is:

$$b_u\left(t\right)=clip\left(\overline{b_u}\left(k\right)+{\Delta b}_u\left(t\right), b_u^{min},b_u^{max}\right)$$

(12)

The executed energy mode applies a subtractive correction:

$$m\left(t\right)=clip\left(\overline{m}\left(k\right)-\Delta m\left(t\right),\mathrm{0,2}\right)$$

(13)

where the $clip\left(·\right)$ function enforces the absolute system boundaries.

The ultimate objective of the hierarchical controller is to minimize the total energy consumed by the base station across the non-stationary traffic episode, without violating the strict latency requirement of the URLLC slice or starving the eMBB slice. This represents a multi-objective optimization problem bounded by strict physical and service-level constraints. The ideal optimization objective is to find a policy $\mathsf{\pi }$ that minimizes the expected total energy consumption:

$${min}_{\pi }E\left[\sum _{t=1}^{T}P\left(t\right)∆t\right]$$

(14)

This minimization is subject to the following constraints:

$$V_u\le {ϵ}_u$$

(15a)

$${\overline{\rho }}_e\ge {\rho }_e^{min}$$

(15b)

$$b_u\left(t\right)+b_e\left(t\right)\le B_{total}$$

(15c)

$$b_s^{min}\le b_s\left(t\right)\le b_s^{max}, \forall s\in S$$

(15d)

$$∆b_u\left(t\right)\in \left[-{∆b}_{max},{∆b}_{max}\right]$$

(15e)

$$∆m\left(t\right)\in \left[-{∆m}_{max},{∆m}_{max}\right]$$

(15f)

where $V_u,{\overline{\rho }}_e, b_s\left(t\right)$ and the delta parameters $\left(∆b_u\left(t\right), ∆m\left(t\right) \right)$ refer to the aggregate URLLC delay violation ratio, the average eMBB throughput satisfaction, the physical PRB allocation for slice $s$, and the Near-RT tactical adjustments, respectively. Here, Equation (15a) denotes the strict limit on the URLLC delay violations, ensuring that the critical reliability requirements are met. Constraint (15b) guarantees that the average throughput satisfaction for the eMBB slice does not fall below the minimum acceptable baseline, preventing service starvation. Constraint (15c) specifies that the sum of allocated PRBs across all slices cannot exceed the total available capacity of the cell. Equation (15d) defines the absolute allocation boundaries for each slice. Finally, Equations (15e) and (15f) dictate the restrictions related to the Near-RT tactical adjustments for PRB allocation and energy mode; these bounds ensure hierarchical stability by preventing the Near-RT execution loop from completely overriding the strategic nominal plan established by the Non-RT controller.

The immediate reward $r\left(t\right)$ observed by the learning agent at time step $t$ is formulated as:

$$r_t=-{\omega }_E\tilde{P}\left(t\right)-{\omega }_{D}max\left(0,{\displaystyle \frac{D_u\left(t\right)}{D_u^{max}}}-1\right)-{\omega }_V{v}_u\left(t\right)+{\omega }_T{\rho }_e\left(t\right)-{\omega }_c\left|∆b_u\left(t\right)\right|$$

(16)

where ${\omega }_E$ represents the penalty weight for instantaneous base station power consumption, ${\omega }_D$ and ${\omega }_V$ denote the continuous and discrete penalty weights for URLLC delay degradation and strict SLA violations, respectively, ${\omega }_T$ scales the reward for eMBB throughput satisfaction, and ${\omega }_C$ acts as a penalty weight for Near-RT control volatility to prevent high-frequency resource oscillations. These hyperparameters are empirically tuned to normalize disparate physical metrics (e.g., Joules for energy, milliseconds for delay) into a unified scalar reward. To mathematically enforce that the agent prioritizes critical network reliability, the SLA penalty weights $\left({\omega }_D,{\omega }_v\right)$ are assigned orders of magnitude higher than the energy coefficient ${\omega }_E$. This strict weighting hierarchy forces the policy to aggressively penalize latency breaches over baseline power savings. The RL objective is to minimize the aggregate penalties over the episode:

$$J_{\pi }=E_{\pi }\left[\sum _{t=1}^{T}r\left(t\right)\right]$$

(17)

While the penalized learning objective is used during the training phase of the DRL controllers, comparing different baseline algorithms requires a unified, unbiased metric. To effectively quantify the best constrained SLA trade-off, we utilized a normalized composite cost function at test time.

The final evaluation metric is the Energy-SLA Cost ($C_{Energy.SLA}$), where a lower score indicates a more optimal system state:

$$C_{Energy.SLA}={\beta }_E{\tilde{E}}_{dyn}+{\beta }_v{V}_u+{\beta }_D{\tilde{D}}_u+{\beta }_B{G}_e$$

(18)

where the weighting hyperparameters (${\beta }_E, {\beta }_v, {\beta }_D,{\beta }_B)$ dictate the relative operational importance of each metric. To ensure strict evaluation of latency-sensitive reliability, the normalized delay risk utilizes the 99th percentile of observed latency, defined as ${\tilde{D}}_u=d_u^{99}/D_u^{max}$, and the eMBB service deficit is captured via the satisfaction gap, $G_e=1-{\overline{\rho }}_e.$

4. Proposed H-RLS Hierarchical Framework

The proposed H-RLS framework maps directly into the standardized O-RAN architecture by exploiting the functional separation between the Service Management Orchestration (SMO) layer and the Near-RT RIC. The core philosophy of this design is that strategic energy conservation and tactical SLA protection operate most effectively on fundamentally different timescales. To achieve this, the control logic is divided into two independent loops:

(1)

The Non-RT Strategic Control Loop: Operating within the SMO, a PPO agent acts as an rApp. It observes aggregated, long-term historical Key Performance Indicators (KPIs) to learn the broader energy-SLA trade-off. It periodically issues a nominal or baseline policy consisting of a URLLC PRB budget and a target energy-saving mode down to the Near-RT RIC via the A1 interface.

(2)

The Near-RT Tactical Execution Loop: Operating closer to the radio edge, a lightweight RLS agent is deployed within the xApp. It monitors real-time E2 telemetry, including instantaneous queue depths and packet arrivals. Instead of overriding the Non-RT policy, it computes a bounded adjustment to the nominal plan to safely accommodate sudden traffic bursts or flash crowds.

The temporal relationship between the two loops is critical to the system’s stability. If the Near-RT xApp were allowed unbounded control, it could lead to chaotic resource oscillation. Conversely, relying solely on the Non-RT loop would leave the URLLC slice vulnerable to sudden traffic spikes that occur between A1 policy updates. By utilizing bounded stateful adjustment, the Near-RT RIC is restricted to acting as a tactical safety net, as illustrated in Figure 1.

The execution flow of the proposed architecture, as depicted in Figure 2, operates in two distinct phases:

(1)

Over the course of the nominal interval ($T_N=100$ base steps), the O-DU aggregates historical network states, including traffic profiles, queue depths, and delay metrics, then forwards them to the SMO layer via the O1 interface.

(2)

The Non-RT RIC rApp processes this historical data using the PPO agent to evaluate the long-term energy-SLA trade-off and update its strategic policy.

(3)

The rApp issues an updated policy intent, $a^{NonRT}\left(k\right)$, which establishes the baseline URLLC PRB budget and the target energy mode. This nominal plan is transmitted down to the Near-RT RIC via the A1 interface and remains anchored for the duration of the subsequent interval.

(4)

Within the established nominal interval, the O-DU streams instantaneous, millisecond-level telemetry to the Near-RT RIC via the E2 interface at every control step $t$.

(5)

The Near-RT xApp feeds this E2 telemetry into the RLS predictor to predict imminent URLLC SLA risks.

(6)

Based on the predicted risk severity, the xApp computes a bounded tactical adjustment, generating $\Delta b_u\left(t\right)$ and $\Delta m\left(t\right)$ to safely modify the nominal plan without overriding the strategic energy-saving goals.

(7)

The Near-RT xApp formulates the final control action by integrating the strategic Non-RT nominal plan with the newly computed tactical adjustments.

(8)

This combined resource and energy command is subsequently transmitted from the Near-RT RIC back to the O-DU via the E2 interface.

(9)

Upon receiving the command, the O-DU executes the physical PRB slicing and applies the selected energy-saving mode to the RU, repeating this adjustment loop continuously at every control step $t$.

4.1. Non-RT RIC: PPO-Based Policy Learning

In this section, we detail the design of the policy-learning intelligence residing in the Non-RT RIC rApp. The primary objective is to learn a nominal resource and energy policy that balances long-term power reduction with an SLA protection strategy.

The control problem is formulated as a discrete-time MDP operating at the macro-timescale $k$, defined by the tuple $\left(o, A,P,R,\gamma \right)$.

4.1.1. State Space

The state observation $o_k^{NonRT}\in O$ provides the PPO agent with a macro-level view of the network’s health. Because the Non-RT RIC operates at an interval of $T_N$ base steps (specifically 100 base steps or 1 s), it does not react to instantaneous packet arrivals; instead, it processes aggregated E2 telemetry collected over the previous interval.

$$o_k^{NonRT}={\left[{\overline{D}}_u\left(k\right), D_u^{max, obs}\left(k\right),{\overline{Q}}_e\left(k\right),{\overline{Q}}_u\left(k\right), {\overline{A}}_e\left(k\right), {\overline{A}}_u\left(k\right), {\overline{\rho }}_e\left(k\right), \overline{m}\left(k-1\right), z_k\right]}^T$$

(19)

where ${\overline{D}}_u\left(k\right)$ and $D_u^{max, obs}\left(k\right)$ denote the average and maximum URLLC delay proxies observed over the previous $T_N$ interval, quantifying recent SLA compliance. To reflect macroscopic buffer congestion states ${\overline{Q}}_e\left(k\right)$ and ${\overline{Q}}_u\left(k\right)$ represent the average queue backlogs for the eMBB and URLLC slices, respectively. Furthermore, ${\overline{A}}_e\left(k\right)$ and ${\overline{A}}_u\left(k\right)$ capture the moving averages of the traffic arrival rates for both slices, serving as primary indicators of the network’s offered load. The term $\overline{m}\left(k-1\right)$ indicates the nominal energy-saving mode executed during the previous interval, which provides the necessary temporal context for the agent to evaluate energy transition costs. Finally, $z_k$ is a categorical traffic phase indicator that captures the current macroscopic condition.

4.1.2. Action Space

The continuous action space defines the strategic policy guidance sent to the Near-RT RIC xApp via the A1 interface. While the PPO actor network generates continuous values to ensure smooth gradient updates during training, the nominal energy mode $\overline{m}\left(k\right)$ is deterministically rounded to the nearest valid integer $\in \left\{0, 1, 2\right\}$ before being issued to the Near-RT RIC. The PRB budget ${\overline{b}}_u\left(k\right)$ similarly represents a continuous target that is discretized during physical execution. At each decision epoch $k$, the actor network outputs a nominal action vector, which is defined as:

$$a^{NonRT}\left(k\right)=\left[{\overline{b}}_u\left(k\right), \overline{m}\left(k\right) \right]$$

(20)

4.1.3. Reward Formulation

To mathematically capture the multi-objective energy-SLA trade-off, the reward $R_k$ for the Non-RT agent is formulated as a cumulative sum of the immediate Near-RT step penalties $r\left(t\right)$ over the decision interval $T_N$:

$$R_k=\sum _{t=k{T}_N}^{\left(k+1\right)T_N-1}r\left(t\right)$$

(21)

This cumulative formulation strictly couples the strategic policy to the Near-RT physical environment. It forces the PPO agent to evaluate the real-time consequences of its nominal plan, as under-provisioning the URLLC budget triggers severe latency penalties during traffic spikes, whereas over-provisioning accumulates continuous energy costs throughout the interval.

4.1.4. PPO Agent Training Process

We selected PPO [39] for its stable convergence properties and its ability to prevent destructively large policy updates. PPO utilizes a clipped surrogate objective function to ensure that the new policy ${\pi }_{\theta }$ does not deviate significantly from the old policy ${\pi }_{{\theta }_{old}}$.

The optimization objective maximized by the actor network is:

$${\mathcal{L}}^{CLIP}\left(\theta \right)=\mathbb{E}\left[min\left(p_k\left(\theta \right){\widehat{A}}_k, clip\left(p_k\left(\theta \right), 1-{ϵ}_{clip},1+{ϵ}_{clip}\right){\widehat{A}}_k\right)\right]$$

(22)

where $p_k\left(\theta \right)$ is the probability ratio between the new and old policies, ${\widehat{A}}_k$ is the Generalized Advantage Estimate (GAE) computed by the critic network, and ${ϵ}_{clip}$ is the clipping hyperparameter bounding the policy update.

To manage the learning process, several operational hyper-parameters must be established. The offline training spans a total of $N_{ep}$ episodes, each lasting for a total of $T$ base steps. Future reward evaluations are controlled by the discount factor $\gamma$. Furthermore, the actor and critic neural network updates are scaled by their respective learning rates, ${\alpha }_{\theta }$ and ${\alpha }_{\varphi }$. The complete offline training procedure is detailed in Algorithm 1.

Algorithm 1. Offline training procedure for the Non-RT rApp
Input: PPO agent with initial actor network ${\pi }_{\theta }$ and critic network $V_{\varphi }$
System parameters: $N_{ep}$, episode duration $T$, Non-RT interval $T_N$
Hyperparameters: discount factor $\gamma , \lambda ,$ clipping ratio ${ϵ}_{clip}$, ${\alpha }_{\theta }$ and ${\alpha }_{\varphi }$
Output: Optimized policy weight ${\pi }_{{\theta }^{\ast }}^{NonRT}$
1:	Initialization: transition replay buffer $B$
2:	for episode $e=1$ to $N_{ep}$ do
3:		Reset O-RAN environment, initialize queues, set initial $z\left(0\right)$
4:		Observe initial aggregated Non-RT state $o_0^{NonRT}$
5:		for $k=0$ to $T/T_N-1$ do
6:			Nominal action $a^{NonRT}\left(k\right)==\left[{\overline{b}}_u\left(k\right), \overline{m}\left(k\right) \right]$ ~ ${\pi }_{\theta }\left(o_k^{NonRT}\right)$
7:			Initialize interval reward accumulator $R_k=0$
8:			for $t={kT}_n$ to $\left(k+1\right)T_N-1$ do
9:				xApp calculate real-time bounded adjustment $\Delta b_u\left(t\right)$ and $\Delta m\left(t\right)$
10:				Execute clipped physical action $b_u\left(t\right)$ and $m\left(t\right)$
11:				Observe immediate step reward $r\left(t\right)$
12:				Accumulate interval reward: $R_k=R_k+r\left(t\right)$
13:			end for
14:			Compute state-value estimate $V_{\varphi }\left(o_k^{NonRT}\right)$ for all state in $B$
15:			Store transition $\left(o_k^{NonRT},a^{NonRT}\left(k\right),R_k,o_{k+1}^{NonRT},\mathit{log}{\pi }_{\theta } \left(a^{NonRT}\left(k\right)|o_k^{NonRT}\right)\right)$in $B$
16:		end for
17:		Compute state-value estimate $V_{\varphi }\left(o_k^{NonRT}\right)$ for all state in $B$
18:		Compute ${\overline{A}}_k$ using $R_k, \gamma , \lambda$ and $V_{\varphi }$
19:		Compute reward target values ${\widehat{V}}_k={\widehat{A}}_k+V_{\varphi }\left(o_k^{NonRT}\right)$
20:		for several optimization epochs do
21:			Calculate probability ratio: $p_k\left(\theta \right)={\displaystyle \frac{{\pi }_{\theta }\left(a^{NonRT}\left(k\right)|o_k^{NonRT}\right)}{{\pi }_{{\theta }_{old}}\left(a^{NonRT}\left(k\right)|o_k^{NonRT}\right)}}$
22:			Calculate surrogate objective ${\mathcal{L}}^{CLIP}\left(\theta \right)$
23:			Update Actor $\theta$ via Adam optimizer to maximize ${\mathcal{L}}^{CLIP}\left(\theta \right)$ with ${\alpha }_{\theta }$
24:		end for
25:		Clear relay buffer $B$
26:	end

4.2. Near-RT RIC: Real-Time Adaptive Control

Sudden bursty traffic or flash-crowd events can cause the URLLC queue to overflow long before the next nominal action ${\overline{b}}_u\left(k\right)$ is issued. To mitigate this, the Near-RT RIC hosts an xApp designed to act as a localized safety net. To ensure the xApp remains lightweight and capable of executing at the strict near-real-time granularity, RLS is employed rather than a computationally heavy mapping the current instantaneous state to future SLA pressure. RLS is strategically selected over alternatives like deep neural networks or Kalman filter. This precision is achieved due to its deterministic, lightweight nature and robust tracking capabilities in non-stationary environments. Crucially, because the xApp update 10 ms interval is longer than the strict URLLC 5 ms delay target, a purely reactive controller would inevitably suffer from intra-interval SLA breaches during sudden traffic spikes. The predictive nature of the RLS overcomes this limitation by forecasting imminent queue pressure $\widehat{y}\left(t\right)$ and proactively expanding the PRB capacity before the physical queue reaches a critical threshold. This mitigation ensures the downstream O-DU MAC scheduler continuously maintains adequate capacity space throughout the subsequent 10 ms execution window to safely clear packets within the 5 ms deadline. Furthermore, while deep learning models introduce high inference latency and are highly vulnerable to out-of-distribution (OOD) failures during unseen traffic bursts, the model-free RLS avoids these pitfalls. By dynamically adapting its weights using the forgetting factor ${\lambda }_{RLS}$, the framework ensures rapid convergence and tactical mitigation during sudden traffic spikes without overriding the strategic goals of the Non-RT layer.

4.2.1. RLS Feature Representation

At each step $t$, the xApp constructs a feature vector $X\left(t\right)$ from real-time $E2$ telemetry. To ensure numerical stability and consistent gradient tracking across varying traffic scales, the features are strictly normalized against their maximum theoretical bounds.

$$X\left(t\right)={\left[1, {\displaystyle \frac{D_u\left(t\right)}{D_u^{max}}},{\displaystyle \frac{Q_u\left(t\right)}{Q_u^{th}}},{\displaystyle \frac{Q_e\left(t\right)}{Q_e^{max}}},{\widehat{A}}_u\left(t\right),{\widehat{A}}_e\left(t\right),{\displaystyle \frac{A_u\left(t\right)}{A_u\left(t\right)+A_e\left(t\right)+ϵ}},{\displaystyle \frac{{\overline{b}}_u\left(k\right)}{B_{total}}},f_{flash}\left(t\right),f_{trans}\left(t\right)\right]}^T$$

(23)

where the ${\widehat{A}}_u\left(t\right)$ and ${\widehat{A}}_e\left(t\right)$ represent the high-frequency packet arrival rate for both slices, capturing short-term burstiness. To measure the relative proportion of URLLC traffic against the total cell load, a small constraint $ϵ$ is added to prevent zero-division during idle periods. To enable the RLS to anticipate and react to non-stationary traffic changes, two binary flags are introduced: $f_{flash}\left(t\right)\in \left\{0, 1\right\}$, which indicate the immediate presence of a severe flash-crowd event, and $f_{trans}\left(t\right)\in \left\{0, 1\right\}$, which alerts the system that the network is currently transitioning between categorical traffic phases and should expect elevated variance. In practical implementations, these flags are not externally supplied ground-truth labels. Instead, they are computed entirely online by the xApp using lightweight statistical thresholding on the first derivative of the incoming packet arrival rate observed at the millisecond level over the E2 interface.

4.2.2. Risk Prediction and Recursive Update

Using the feature vector, the xApp computes a linear prediction of imminent URLLC delay pressure, denoted as $\widehat{y}\left(t\right)$:

$$\widehat{y}\left(t\right)=W^T\left(t-1\right)X\left(t\right)$$

(24)

where $W(t-1)$ represents the actively maintained weight vector from the previous control step. Upon observing the actual delay pressure $y\left(t\right)$ resulting from the environment’s state transition, the xApp updates its predictive weights. The RLS gain vector $K\left(t\right)$, which dictates the magnitude of the correction based on the current input confidence, is computed as:

$$K\left(t\right)={\displaystyle \frac{P\left(t-1\right)X\left(t\right)}{{\lambda }_{RLS}+X^T\left(t\right)P\left(t-1\right)X\left(t\right)}}$$

(25)

where $P(t-1)$ is the inverse covariance matrix and ${\lambda }_{RLS}\in \{0, 1]$ is the forgetting factor. The forgetting factor is a critical hyperparameter for non-stationary traffic in which it exponentially discounts older observations, allowing the xApp to rapidly adapt its predictions when the traffic phase abruptly shifts.

The weight vector and the inverse covariance matrix are subsequently recursively updated for the next time step:

$$W\left(t\right)=W\left(t-1\right)+K\left(t\right)\left[y\left(t\right)-X^T\left(t\right)W\left(t-1\right)\right]$$

(26)

$$P\left(t\right)={\displaystyle \frac{1}{{\lambda }_{RLS}}}\left[P\left(t-1\right)-K\left(t\right)X^T\left(t\right)P\left(t-1\right)\right]$$

(27)

This continuous online tracking ensures that the prediction $\widehat{y}\left(t\right)$ remains an accurate indicator for SLA risk, regardless of whether the traffic is in a stable, bursty, or flash-crowd condition.

4.3. Bounded Resource Adjustment Mechanism

The Near-RT xApp classifies the predicted SLA risk $\widehat{y}\left(t\right)$ against predefined thresholds. Under stable conditions $\widehat{y}\left(t\right)\le {\mathsf{\Gamma }}_{risk}$, the xApp remains inactive ${(∆b}_u\left(t\right)=0, ∆m\left(t\right)=0)$, preserving the baseline energy-saving strategy established by the Non-RT layer. Conversely, the xApp executes a two-stage escalation based on the severity of the predicted SLA risk. If a moderate risk is detected ${\mathsf{\Gamma }}_{risk}<\widehat{y}\left(t\right)\le {\mathsf{\Gamma }}_{critical}$, the xApp triggers a bounded stateful resource response, calculating a proportional PRB adjustment:

$${∆b}_u\left(t\right)=min\left(⌈\eta \cdot \left(\widehat{y}\left(t\right)-{\mathsf{\Gamma }}_{risk}\right)⌉,{∆b}_{max}\right)$$

(28)

where $\eta$ is the sensitivity scaling factor mapping the delay pressure to discrete PRBs, and the $min\left(\cdot \right)$ operator enforces the maximum allowable deviation ${∆b}_{max}$ to ensure hierarchical stability. During severe non-stationary conditions, PRB reallocation alone may be insufficient. If the adjustment saturates ${∆b}_u\left(t\right)={∆b}_{max}$ and the predicted risk breaches the secondary critical threshold $\widehat{y}\left(t\right)>{\mathsf{\Gamma }}_{critical}$, the xApp initiates a tactical energy mode downgrade $∆m\left(t\right)\in \left\{1, 2\right\}$ to temporarily maximize processing capacity. Ultimately, these millisecond-level adjustments are combined with the Non-RT anchor and strictly bounded by the physical limits of the RU before execution at the O-DU. The step-by-step online tactical adjustment process is summarized in Algorithm 2.

Algorithm 2. Online Near-Real-Time Execution of xApp
Input: Active nominal policy $a^{NonRT}\left(k\right)$
System parameters: Risk thresholds ${\mathsf{\Gamma }}_{\mathrm{r}\mathrm{i}\mathrm{s}\mathrm{k}},{\mathsf{\Gamma }}_{\mathrm{c}\mathrm{r}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{c}\mathrm{a}\mathrm{l}}$, sensitivity scale $\eta$, ${∆b}_{max}$, ${∆m}_{max}$
RLS state: Current weights $W(t-1)$, inverse covariance $P(t-1)$
1:	for each base control step $t$ do
2:		Observe instantaneous E2 telemetry: $D_u\left(t\right), Q\left(t\right), A\left(t\right)$, traffic flags
3:		Construct and normalize feature vector $X\left(t\right)$
4:		Compute predicted SLA risk $\widehat{y}\left(t\right)$
5:		Observe actual environment delay pressure $y\left(t\right)$
6:		Compute RLS gain $K\left(t\right)$
7:		Update weights $W\left(t\right)$
8:		Update covariance $P\left(t\right)$
9:		if $\widehat{y}\left(t\right)\le {\mathsf{\Gamma }}_{risk}$ then
10:			${∆b}_u\left(t\right)\leftarrow 0$
11:			$∆m\left(t\right)\leftarrow 0$
12:		else if ${\mathsf{\Gamma }}_{risk}<\widehat{y}\left(t\right)\le {\mathsf{\Gamma }}_{critical}$ then
13:			${∆b}_u\left(t\right)\leftarrow min\left(⌈\eta \cdot \left(\widehat{y}\left(t\right)-{\mathsf{\Gamma }}_{risk}\right)⌉,{∆b}_{max}\right)$
14:			$∆m\left(t\right)\leftarrow 0$
15:		else
16:			${∆b}_u\left(t\right)={∆b}_{max}$
17:			$∆m\left(t\right)\leftarrow Compute$appropriate energy decrease $\le {∆m}_{max}$
18:		else if
19:		$b_u\left(t\right)=clip\left({\overline{b}}_u\left(k\right)+∆b_u\left(t\right),b_u^{min},b_u^{max}\right)$
20:		$m\left(t\right)=clip\left(\overline{m}\left(k\right)-∆m\left(t\right),\mathrm{0,2}\right)$
21:		Send $\left[b_u\left(t\right),∆m\left(t\right)\right]$ to O-DU MAC scheduler via E2 interface
22:	end

5. Computational Complexity and Signal Overhead

A fundamental motivation for the H-RLS architecture is the computational and signaling constraints of the O-RAN Near-RT RIC. Deploying deep learning models at the millisecond-level introduces inference latency and interface saturation [20,21]. By decoupling the architecture, H-RLS systematically resolves these bottlenecks.

5.1. Computational Complexity

The primary justification for deploying the RLS predictor at the Near-RT controller is its deterministic, low-complexity execution. For a feature vector $N_f$, the standard RLS algorithm updates the inverse covariance matrix $P\left(t\right)$ and weight vector $W\left(t\right)$ with a time complexity of $\mathcal{O}\left(N_f^2\right)$. As defined in Equation (23), the H-RLS feature vector abstracts the environment into a highly compact state containing exactly 8 features $\left(N_f=8\right)$. Consequently, the xApp requires minimal computational overhead per 10 ms execution cycle. In contrast, deploying a standalone Deep Neural Network (DNN) policy at the Near-RT layer, as seen in the Flat Near-RT baseline, demands $\mathcal{O}\left(\sum L_i{L}_{i+1}\right)$ operations per step, where $L_i$ denotes the number of neurons in the $i^{th}$ layer. This typically requires thousands of matrix multiplications for even a lightweight multi-layer perceptron, alongside the memory overhead of maintaining deep actor–critic graphs at the edge. By elevating the computationally heavy PPO optimization to the Non-RT SMO layer, where it executes only once per second, the framework frees up critical Near-RT CPU cycles.

5.2. Signaling Overhead

Hierarchical decoupling significantly reduces the signaling payload across the O-RAN interfaces. In a single-loop Near-RT DRL deployment, the controller must continuously pull high-dimensional state arrays and transmit absolute resource commands across the E2 interface at high frequency nominal policy updates ($a^{NonRT}\left(k\right))$, generating only one message per second. At the Near-RT controller, the xApp relies on localized E2 telemetry subscriptions for lightweight metrics. Furthermore, since the xApp only applies adjustments when the predicted risk exceeds the safety threshold $\widehat{y}\left(t\right)>{\mathsf{\Gamma }}_{risk}$, the E2 control message payload is strictly limited to sparse, event-driven delta values ($∆b_u,∆m)$. This mechanism minimizes continuous E2 interface saturation during stable traffic periods.

6. Performance Evaluation

This section details the simulation environment, defines the baseline methodologies used for comparison, and analyzes the framework’s performance across both stable and highly dynamic traffic scenarios.

6.1. Simulation Setup

In our simulation, we developed a discrete-time simulation testbed mapped directly to the standardized O-RAN logical interfaces. Specifically, the Non-RT PPO policy planner updates every 1 s via the A1/O1 interfaces, while the Near-RT RLS processes E2 telemetry at a 10 ms resolution. The simulated base station allocates a finite pool of 100 PRBs to balance a capacity-driven eMBB slice against a latency-critical URLLC slice bound by a strict 5 ms delay SLA. Algorithmic stability is enforced using hyperparameters, notably an RLS forgetting factor of ${\lambda }_{RLS}=0.92$ and PPO clipping ratio of ${ϵ}_{clip}=0.2$. The learning hyperparameters were aligned with standard PPO convergence guidelines, while the reward weights and risk thresholds were empirically tuned via grid search to strictly prioritize zero-tolerance SLA constraints over aggressive energy conservation. Furthermore, the maximum Near-RT adjustments were deliberately hardcoded to represent exactly 25% of the physical spectrum, ensuring the predictive filter does not accidentally starve the eMBB slice during the emergency reallocation. The full set of important simulation parameters and hyperparameters used for training the proposed H-RLS framework is detailed in

Table 3. Key simulation parameters and hyperparameters.

Parameters	Specifications
Cell capacity	100 Mbps
Number of Slices	2 (eMBB and URLLC)
Total PRB ($B_{total}$)	100 PRBs
Spectral efficiency (${\eta }_e,{\eta }_u$)	1 Mbps/PRB
URLLC queue threshold $\left(Q_u^{th}\right)$	25
Non-RT decision interval	100 base steps (1 s update interval)
Near-RT decision interval	1 base step (10 ms update interval)
Maximum dynamic power ($P_{load}$)	130 W
Static baseline power ($P_{ba}$)	25 W
SLA delay threshold	5 ms
Discrete Energy Modes	Performance (0), Balanced (1), Saving (2)
Discount factor ($\gamma$)	0.99
GAE $\left(\lambda \right)$	0.95
Learning rate $\left(\alpha \right)$	${3\times 10}^{-4}$
Clipping ratio $\left({ϵ}_{clip}\right)$	0.2
Episode length	3000-steps (30 s per episode)
Batch size	128
URLLC buffer size	50
eMBB buffer size	500
RLS forgetting factor (${\lambda }_{RLS}$)	0.92
Moderate risk threshold (${\mathsf{\Gamma }}_{risk}$)	0.45
Critical risk threshold (${\mathsf{\Gamma }}_{critical}$)	0.60
Max PRB adjustment (${∆b}_{max}$)	25 PRBs
Max mode adjustment (${∆m}_{max}$)	2
Adjustment gain sensitivity ($\eta$)	1.0
Penalized objective weight (${\omega }_E,{\omega }_D,{\omega }_v,{\omega }_T,{\omega }_C$)	(2.0, 8.0, 100.0, 10.0, 0.10)
Evaluation objective weight (${\beta }_E,{\beta }_v,{\beta }_D,{\beta }_B$)	(1.0, 50.0, 10.0, 2.0)
Random seeds	41, 42, 43, 44, 45

. To explicitly capture the energy-SLA trade-off, the environment dynamically scales hardware processing capacity across 3 discrete energy modes, modeling a maximum dynamic load of approximately 130W. Rather than relying on stationary assumptions, the framework’s robustness is stress-tested by forcing each 3000-step episode through four severe non-stationary traffic transition phases:

• Normal: Represents a stable network operation. The total offered load ratio is maintained between 40% and 50%. The traffic is distributed with 45 Mbps allocated to eMBB and 5 Mbps allocated to URLLC.
• Bursty: Introduces temporary traffic spikes. The overall offered load ratio rises to 80%. During these spikes, eMBB demand rises to 60 Mbps, and URLLC demand increases to 10 Mbps.
• Flash-crowd: Models severe congestion stress on the network. The offered load ratio reaches a critical level between 90% and 120%. Traffic surges significantly across both slices, with eMBB reaching 80 Mbps and URLLC spiking to 30 Mbps.
• Shifted Demand: Simulates a fundamental service demand shift resulting in high URLLC pressure. The standard traffic distribution is inverted, with URLLC becoming the dominant service class at 45 Mbps, while eMBB decreases to 35 Mbps.

To ensure rigorous methodology isolation of the Medium Access Control (MAC) layer scheduling dynamics, physical layer phenomena such as fast-fading and multi-cell interference are intentionally abstracted. This assumption eliminates stochastic physical-layer degradation as a confounding variable, strictly aligning with O-RAN architectural principles. However, we acknowledge that in practical deployments, spatial randomness and multi-cell interference materially affect reliability. Recent advances in programmable propagation environments, particularly Reconfigurable Intelligent Surfaces (RIS), offer powerful solutions to these physical-layer challenges. For instance, recent studies demonstrate that RIS can achieve statistical interference elimination in distributed networks [40] and secure transmissions against spatial randomness and eavesdropping [41]. While the current study isolates MAC-layer queue dynamics, integrating RIS-assisted interference suppression represents a highly complementary direction for extending the H-RLS framework toward channel-aware O-RAN slicing. Furthermore, although commercial 5G/6G deployments operating on wideband carriers support gigabit-level capacities, this simulation normalizes the physical environment to a constrained 100 PRB resource pool. This abstraction intentionally forces a severe mathematical bottleneck. Simulating unconstrained gigabit-level capacities introduces a combinatorial explosion in the RL state-action space that obscures MAC layer dynamics. By scaling the capacity and injecting aggressive stochastic traffic anomalies that push the load ratio beyond 100%, the environment explicitly isolates and stress-tests the predictive SLA preservation, queue dynamics, and energy state transitions without confounding physical-layer variables. Despite this simulated constraint, the proposed H-RLS architecture is inherently designed to scale effectively to the unconstrained, gigabit-level capacities expected in real 6G deployments. To avoid the curse of dimensionality and state-space explosion in practical massive wideband scenarios, the hierarchical framework operates independently of absolute physical resource counts. Specifically, the Non-RT PPO agent learns continuous, normalized resource allocations (e.g., budget percentages) rather than discrete block-by-block mappings, ensuring convergence stability regardless of total available bandwidth. Furthermore, the Near-RT xApp utilizes an RLS predictor operating on a fixed-size feature vector of normalized metrics $\left(N_f=8\right)$. As a result, its computational complexity of $\mathcal{O}\left(N_f^2\right)$ remains strictly constant as cell capacity grows, equipping the framework to maintain stable execution and scalable SLA preservation in real-world environments.

6.2. Benchmarking Methods

Our framework is benchmarked against four representative baselines under identical physical resource constraints. The first is a Reactive Rule-Based Controller, which relies on static delay thresholds to scale PRBs and energy modes, demonstrating the inherent latency vulnerabilities and legacy, non-predictive heuristics. The second baseline, a Single-Loop DRL (Flat Non-RT PPO), employs a standalone agent updating at 1 s intervals; this isolates the necessity of the Near-RT safety net, as the Non-RT predictably fails to mitigate SLA violations during the congestion stage. Conversely, the third baseline, a Tactical Near-RT Single-Loop DRL (Flat Near-RT PPO), shifts the entire intelligence to the 10 ms interval, serving as a benchmark for over-provisioning and dynamic energy waste due to its hypersensitivity to stochastic noise and lack of a policy guidance layer. Finally, an Unconstrained Dual-Loop hierarchy is evaluated, which utilizes the identical Non-RT PPO agent but replaces the mathematically bounded RLS with a rigid, reactive rule-based Near-RT resource adjustment. Specifically, this plain hierarchical baseline allocates a fixed, maximum block of tactical PRBs only after the URLLC delay breaches a static 4 ms warning threshold. By comparing the H-RLS against this rigid step-function baseline, the evaluation explicitly isolates the contribution of the RLS predictor in preventing delayed resource release and unnecessary power consumption after transient traffic fluctuations diminish.

6.3. Evaluation Results and Discussion

6.3.1. Training Convergence

We first evaluate algorithmic stability by tracking the moving average of episodic rewards, illustrated in Figure 3. Both single-loop architectures, the Flat Non-RT and Flat Near-RT agents, exhibit highly chaotic learning trajectories, with rewards frequently plummeting into severe negative values due to massive URLLC delay violations triggered by unsafe exploratory actions. This volatility empirically proves that a single-timescale DRL agent cannot safely balance long-term efficiency and instantaneous reliability simultaneously.

In contrast, both dual-loop architectures (the Plain Hierarchy and the proposed H-RLS) successfully stabilize the learning process by structurally decoupling the timescales. However, by utilizing the RLS predictive wrapper rather than a rigid rule-based adjustment, the H-RLS framework converges to a significantly higher reward ceiling, confirming its superior optimization capacity and tighter variance control.

6.3.2. Slice-Level Performance Evaluation Under Heterogeneous Traffic Conditions

To evaluate the core trade-off between reliability and power consumption, the maximum observed URLLC delay and the corresponding dynamic energy consumption are presented side-by-side in Figure 4.

As illustrated in Figure 4a, the baseline performance diverges significantly under dynamic stress. The reactive Heuristic baseline catastrophically fails the SLA across all stress scenarios, reaching a peak absolute delay of 6.04 ms during the Flash Crowd phase. Its rule-based logic cannot account for rapid exponential queue growth until after a violation has already materialized. Similarly, the Flat Non-RT baseline breaches the SLA during the Bursty (5.19 ms) and Flash Crowd (5.01 ms) scenarios; its coarse 1 s update interval renders it fundamentally blind to intra-interval traffic spikes. In contrast, the proposed H-RLS framework strictly bounds the maximum delay below the 5.0 ms threshold in every scenario, safely peaking at 4.44 ms during the Flash Crowd phase. While the Flat Near-RT baseline achieves slightly lower peak delays than H-RLS, it secures this latency through highly inefficient resource over-provisioning. This hypersensitivity translates directly into unacceptable dynamic power costs, as shown in Figure 4b. Because the Flat Near-RT agent overreacts to stochastic packet arrivals, it prematurely exits energy-saving modes. Conversely, the H-RLS framework demonstrates the highest energy efficiency across all AI-driven baselines. During the Shifted Demand scenario, H-RLS consumes only 48.30 J of dynamic energy—a reduction of over 51% compared to the Plain Hierarchy (100.31 J) and the Flat Near-RT baseline (108.96 J). The precision of the Near-RT RLS predictor effectively minimizes unnecessary resource adjustments when the traffic environment stabilizes, allowing the system to securely maintain lower-power nominal states for longer durations.

Furthermore, it is critical that URLLC prioritization and energy conservation do not trigger eMBB service starvation. Figure 5 presents the secondary evaluation metrics, pairing the eMBB throughput satisfaction with the final composite score. As seen in Figure 5a, the Flat Near-RT approach suffers a severe degradation in eMBB throughput satisfaction during the Flash Crowd scenario, plummeting to roughly 0.70, well below the 0.9 minimum SLA requirement. In contrast, the proposed H-RLS framework seamlessly balances the shared PRB pool to preserve a perfect throughput satisfaction ratio of 1.0, protecting eMBB service integrity even under extreme network congestion. Finally, this precision yields the lowest overall composite Energy-SLA Objective Score across all fixed traffic phases, as demonstrated in Figure 5b, confirming that H-RLS achieves the optimal operational balance before entering the continuous transition test.

6.3.3. Transient Response Under Dynamic Traffic Transitions

The architectural superiority of H-RLS is most evident when analyzing the continuous time-series transient response as shown in Figure 6 and Figure 7. Figure 6a traces the dynamic power consumption over time, while Figure 6b simultaneously tracks the URLLC delay. During the initial Normal and Bursty phases, the Non-RT policy planner successfully anchors the framework to a highly efficient nominal energy state, consuming significantly less dynamic power than the fast-acting baselines in Figure 6a.

The critical validation occurs during the non-stationary transition into the Flash Crowd phase. As extreme traffic volume floods the network, the heuristic baselines struggle to adapt. Figure 6b shows the Rule-Based Heuristic dramatically breaching the 5.0 ms SLA limit. While the Flat Non-RT agent manages to avoid an SLA breach in this specific rolling window, its fundamental timescale flaw is visually evident in Figure 6b through its rigid, staircase-like trajectory. Because its policy is locked to a 1 s update interval, it is entirely blind to sudden millisecond-level queue volatility, leaving it incapable of making tactical, fine-grained adjustments. In contrast, the H-RLS framework detects this imminent risk via the Near-RT RLS predictor. As shown in Figure 6a, the xApp immediately executes a bounded tactical override, temporarily increasing dynamic power consumption to absorb the traffic surge. This instantaneous energy investment successfully buffers the network, capping the URLLC delay smoothly below the SLA threshold in Figure 6b without the chaotic rigidity of the single-timescale baselines. Crucially, because this adjustment is stateful and mathematically bounded, H-RLS instantaneously releases the excess resources the moment the non-stationary surge subsides. This precision yields the lowest overall composite Energy-SLA Objective Score (0.326 ± 0.027 on average) across the entire transition episode. To ensure these results are statistically robust and not reliant on favorable initialization,

Table 4. Performance variance under the transition scenario (Average across 5 random seeds).

Method	Energy-SLA Cost	Dynamic Energy	URLLC Max Delay	eMBB Satisfaction Ratio
Proposed H-RLS	0.326 ± 0.027	70.16 ± 3.19	4.08 ± 0.33	1.00 ± 0.00
Plain Hierarchical	0.510 ± 0.086	110.97 ± 0.59	4.31 ± 0.37	1.00 ± 0.00
Flat Non-RT PPO	0.562 ± 0.076	113.12 ± 2.62	3.90 ± 0.71	0.97 ± 0.04
Flat Near-RT PPO	1.129 ± 0.969	132.88 ± 27.05	4.31 ± 1.21	0.97 ± 0.04
Heuristic	2.430 ± 0.000	83.24 ± 0.00	5.88 ± 0.00	1.00 ± 0.00

reports the mean and standard deviation of the primary performance metrics evaluated across five independent random seeds (41, 42, 43, 44, 45) during the continuous transition episode.

To ensure these results are statistically robust and not reliant on favorable initialization, Table 4 reports the mean and standard deviation of the primary performance metrics evaluated across five independent random seeds (41, 42, 43, 44 and 45) during the continuous transition episode.

As detailed in Table 4, the proposed H-RLS framework maintains exceptionally tight variance bounds, definitively proving that the architecture achieves highly stable, optimal performance in unpredictable O-RAN environments regardless of stochastic initialization (Figure 7). Furthermore, to ensure real-world deployability, an ablation evaluation was conducted where the explicit categorical phase flags $z_k, f_{flash}\left(t\right), f_{trans}\left(t\right)$ were completely removed from both the PPO and RLS state spaces. The transient response of this ablated model is compared against the proposed framework in Figure 8. Figure 8a, without the preemptive knowledge provided by explicit phase labels, the ablated model relies entirely on raw network telemetry, leading to slight variations in dynamic power scaling. Specifically, during the Shifted Demand phase, the controller conservatively utilizes more dynamic power to establish a safe operating margin. Figure 8b illustrates the corresponding effect on URLLC latency. Because the RLS must derive its risk predictions purely from the dynamic gradients of physical queue depths and raw arrival rates rather than explicit flags, the ablated model reacts marginally later to the sudden packet surges during the Flash Crowd and Shifted Demand phases. However, the conservative energy scaling ensures that the model still successfully tracks the anomalies, capping the maximum URLLC delay at 4.19 ms and safely remaining below the 5.0 ms SLA target. This empirically demonstrates that the H-RLS framework autonomously tracks and mitigates unseen traffic anomalies based on observable network physics, without relying on predefined phase labels.

7. Conclusions

This research addressed the critical challenge of balancing the energy-delay tension in O-RAN network slicing under dynamic traffic conditions. By proposing the H-RLS framework, this work successfully decoupled network control into a non-real-time policy planner and a near-real-time predictive correction mechanism. The evaluations confirmed that this hierarchical multi-timescale architecture minimizes dynamic energy consumption while strictly avoiding URLLC SLA violations across the evaluated non-stationary traffic transitions. Ultimately, this demonstrates that hierarchical intelligence is a fundamental requirement for achieving both operational sustainability and strict service reliability in emerging 6G environments. Future work will extend this framework to multi-cell coordination and massive heterogeneous slice management, including mMTC. Additionally, the integration of Digital Twin technology as a “Shadow RAN” will be explored to safely pre-train RL policies without risking physical network SLAs. Finally, incorporating Federated Learning for privacy-preserving, decentralized model training, alongside Generative AI for intent-to-reward translation within the SMO layer, represents a highly promising frontier for fully autonomous O-RAN architectures.