Reducing Port Container Congestion with Reinforcement Learning: The Serial Mediation Role of Operational Learning Stability and Logistics Efficiency

Abstract

Container congestion remains a persistent operational challenge in seaports because berth, yard, and gate processes are tightly coupled, demand is volatile, and control actions often operate under delayed feedback. Reinforcement learning (RL) is increasingly proposed for adaptive terminal decision support, yet the literature still says little about the mechanism through which RL may reduce congestion in practice. This study therefore develops a simulation-based mechanism framework in which RL improves congestion outcomes primarily by increasing Operational Learning Stability (OLStab), defined here as the consistency and governability of learning-enabled operational decisions under variability and disruption. A queueing-based, gate-focused terminal simulator is used as the data-generating process, with gate congestion treated as a reduced-form proxy for broader terminal congestion pressure. The statistical layer is interpreted cautiously as an internal mechanism consistency check within synthetic data rather than as empirical causal identification. Results show that RL is strongly associated with higher OLStab and that OLStab is the dominant pathway linking RL to lower congestion pressure in the simulated environment. Logistics Efficiency (LE) is directionally consistent with congestion reduction in bivariate analysis but adds limited incremental mediation once OLStab is jointly modeled. The theorized moderation by Decision Latency Sensitivity (DLS) is not robustly recovered within the examined latency range. Overall, the study contributes a more bounded explanation of how RL may reduce congestion in a designed gate-focused terminal control environment and highlights learning stability as a practical screening criterion for future digital twin and pilot deployment studies.

1. Introduction

Global seaborne transport depends on container terminals that must coordinate berth, yard, and gate activities under fluctuating demand, resource limits, and operational disruptions. When these tightly coupled processes lose coordination, waiting accumulates across the system in the form of vessel delay, yard pressure, and truck queues. Reinforcement learning (RL) is increasingly discussed as a promising approach for adaptive terminal control because it can update decisions from feedback rather than rely only on fixed rules [1,2]. Yet a persistent gap remains between showing that an RL policy performs well in a simulation and interpreting the operational mechanism through which that performance may matter for congestion reduction [3]. This paper addresses that gap by focusing on the stability of learned operational behavior rather than on prediction accuracy alone. We define Operational Learning Stability (OLStab) as a simulation-grounded operational metric that captures whether an RL-enabled controller produces consistent, robust, governable decisions under changing demand and disruption [4,5]. To keep the environment tractable, the simulator models the gate interface directly and treats gate congestion as a reduced-form proxy for broader terminal congestion pressure rather than as a full micro-dynamic replica of berth and yard processes [6]. Using this queueing-based environment, we generate scenario episode observations and analyze them with regression and structural path models as internal mechanism consistency checks. This design allows us to ask three focused questions: whether RL improves OLStab, whether OLStab is associated with lower congestion pressure, and whether downstream efficiency and latency effects add explanatory value beyond the primary stability pathway. By narrowing the contribution in this way, the paper shifts the discussion from whether RL is generally useful to how learning-enabled terminal control may reduce congestion under clearly stated simulation assumptions.

2. Literature Review and Hypothesis Logic

This section reviews the literature needed to explain congestion reduction through learning-enabled operational control. It first clarifies container congestion as a system-level waiting outcome in port terminals, then distinguishes prediction-oriented machine learning from decision control, positions RL as an adaptive control approach, and finally develops the mechanism logic around OLStab, LE, and DLS. Throughout, the emphasis is on a bounded mechanism argument that is later assessed in a simulated environment.

2.1. Container Terminal Congestion as an Operational Outcome (DV)

Container terminal congestion is generally treated as a system-level condition in which demand, capacity, and process control interactions generate persistent waiting and queue buildup across coupled subsystems (berth, yard, and gate) [7,8]. Operationally, congestion manifests when terminal service processes slow under strain and delays propagate across the logistics chain, disrupting service performance and increasing waiting at multiple interfaces [9,10]. Because congestion is not a single bottleneck but an interdependent system state, its causes are usually framed in four interacting driver groups: (1) demand-side variability (peaky arrivals and batching), (2) capacity/resource limitations (berths, cranes, yard space, gates, and labor), (3) process/control choices (dispatching, stacking/relocation policies, and appointment and release rules), and (4) external disruptions (e.g., operational shocks that reduce continuity and coordination) [11]. Importantly, congestion measurement follows this operational framing by using time/queue and throughput-related indicators that capture accumulated waiting and process friction. In terminal decision support research, operational performance is often monitored through time-based indicators and compliance behaviors that reveal flow instability, such as short planning horizons and weak adherence to designated time windows—patterns that can amplify queues and spillovers when coordination fails [12]. In sum, congestion is best understood as the accumulation of waiting induced by resource constraints and control frictions; consequently, any explanatory model of congestion reduction must connect decision control and process stability to measurable reductions in waiting and queue accumulation [13].

2.2. ML for Congestion Prediction: Capabilities and Limitations for Mechanism Interpretation

A major stream uses ML to predict congestion states and related KPIs using operational data, typically combining transactional terminal logs (e.g., gate events, yard moves), time-window compliance signals, and broader system indicators such as utilization proxies [14]. In applied terminal analytics, business intelligence (BI) and decision support tools operationalize data collection and performance dashboards to improve visibility and enable faster managerial responses, often emphasizing monitoring of operational behavior (e.g., scheduling lead times and adherence patterns) that correlate with congestion formation [12]. In parallel, smart port technology research expands the data layer by enabling real-time sensing and recognition—e.g., vision-based identification and monitoring pipelines designed for port environments—thereby increasing the granularity and timeliness of observable operational state [15,16]. Across this prediction-oriented stream, model families range from classical regression/trees to deep learning architectures, and performance is typically evaluated via forecasting accuracy and generalization metrics [17,18,19].

Beyond gate control and yard operations, recent maritime analytics also examine demurrage and scheduling inefficiencies as drivers of congestion and delay. For example, Kweon et al. [20] analyze demurrage patterns using logical analysis of data to identify scheduling-related risk factors in port operations. This stream complements our work by motivating how learning-based operational control can be evaluated alongside broader congestion drivers such as vessel scheduling and delay propagation across the port ecosystem.

However, prediction and mechanism are not equivalent. Accurate congestion prediction can improve situational awareness and trigger earlier interventions, but it does not specify which operational actions should be taken, nor does it explain why a chosen intervention should reduce congestion across interdependent subsystems [4,21]. Said differently, prediction can tell ports what congestion is likely to look like, while a mechanism model must clarify how decisions reshape operational stability and flow efficiency to reduce accumulated waiting [22]. This limitation motivates a shift from forecasting to decision control and from descriptive performance gains to mechanism interpretation.

2.3. RL in Port/Terminal Operations: Decision Control, Reward Alignment, and Why Outcomes Vary

RL is not a new methodology; it is a well-established framework for learning policies in sequential decision problems. Its relevance to terminal operations comes from the fact that ports face dynamic environments where decision quality depends on evolving state (arrivals, workload, resource availability) and where actions have delayed and system-wide consequences. In principle, RL can be used to control operational decisions such as dynamic dispatching priorities, resource allocation, appointment/release control, and sequencing—provided that the decision problem is well-defined and the reward function faithfully reflects congestion-relevant objectives (e.g., reduced waiting, reduced queue buildup, improved throughput consistency) under operational constraints. Yet the literature also shows why RL sometimes works and sometimes fails outside carefully controlled settings. Surveys of RL in complex systems repeatedly highlight that many RL solutions remain “proof-of-concept” and struggle when moving from simulation into real operational environments due to realism gaps, data and interaction constraints, and the difficulty of maintaining safe, reliable performance under non-stationarity [23]. Similarly, broad reviews emphasize persistent “simulation-to-field” barriers for deep RL applications, noting that deployment requires robustness to uncertainty, constraints, and imperfect feedback loops [24]. These observations are not simply implementation footnotes: they imply that RL’s effectiveness in ports is conditional on whether learning produces stable operational behavior over time and whether the decision loop remains reliable under practical latency and execution frictions. This motivates the need for an explicit mechanism construct—OLS—and an explicit boundary condition—DLS—to explain heterogeneous RL outcomes rather than assuming uniform performance gains.

2.4. Operational Learning Stability (OLStab) as a Mechanism for Reliable RL Impact

To explain why RL can generate sustained congestion reductions in some operating regimes but not others, we introduce Operational Learning Stability (OLStab) as a simulation-grounded operational metric for RL-enabled terminal control. OLStab refers to the extent to which a learned policy produces consistent, low-volatility, governable operational behavior over time under variability, disruption, and implementation frictions. It is conceptually distinct from congestion outcomes and from Logistics Efficiency because it focuses on stability of the control process itself. In this manuscript, OLStab is presented as a measurement construct designed for this simulation framework rather than as a universally validated latent theory.

Conceptual Boundaries of OLStab

OLStab should be distinguished from several neighboring ideas. Policy stability refers to the consistency of the learned policy or action profile. Robustness refers to maintaining acceptable performance under perturbation. Operational resilience concerns the broader terminal system’s ability to absorb, adapt to, and recover from shocks. Logistics Efficiency (LE) refers to downstream flow performance and process smoothness. By contrast, OLStab is the episode-level operational stability of an RL controller, observed through controller traces and performance variability inside the simulation. In this study, OLStab is not proposed as a universal latent construct. Rather, it is a simulation-grounded operational metric designed for this study that operationalizes whether an RL-enabled controller behaves consistently and governably under variability and disruption. It is therefore narrower than resilience and different from efficiency because it refers to stability of the control process, not terminal-wide outcome quality.

2.5. Logistics Efficiency (LE) as a Downstream Mediator and Hypothesis Development (H3, H4)

Logistics Efficiency (LE) reflects the effectiveness of terminal logistics flow—how smoothly and quickly operations execute, how well coordination holds, and how effectively resources convert arrivals into completed service without avoidable waiting. LE is not the same as congestion: congestion KPIs capture accumulated waiting (queues, delays, overflow), whereas LE captures the process capability that prevents such accumulation. In terminal decision support research, efficiency is often represented through time-based process performance and coordination indicators (e.g., turnaround/cycle behavior and compliance patterns), which are central to how quickly the system “drains” demand and prevents waiting accumulation [12]. Resilience-oriented port network research similarly emphasizes that smoother operational continuity is critical for avoiding bottlenecks that propagate disruptions and waiting across interdependent components [6,7].

This stream supports the theorized directional ordering that higher LE should align with lower congestion, because a more efficient flow reduces the buildup of waiting across the gate-yard-berth interfaces. Accordingly, the study specifies the following outcome hypothesis: H3: Higher Logistics Efficiency (LE) is expected to be associated with lower container congestion in the simulated port operations setting.

Building on OLS and LE together, a serial mechanism remains operationally plausible in terminals because efficiency gains typically require stable decision execution: unstable decision cycles can prevent processes from settling into efficient rhythms, whereas stable learning enables persistent operational routines that convert into improved flow. This logic also aligns with the broader view that real-world RL success depends on sustained, robust performance rather than isolated short-run gains [4,5]. Therefore, the study specifies a theorized serial pathway in which RL reduces congestion through the ordered sequence RL → OLS → LE → congestion reduction [25,26]:

H4: The proposed mechanism framework anticipates an ordered pathway in which RL improves OLStab, OLStab enhances LE, and higher LE aligns with lower container congestion.

2.6. Decision Latency Sensitivity (DLS) as a Boundary Condition Explaining RL Success and Failure

A key reason RL can fail to deliver stable improvements in real operations is the presence of latency in the decision loop—delays between observing state, selecting actions, executing actions, and receiving feedback. To capture this, Decision Latency Sensitivity (DLS) is introduced as the extent to which such delays weaken learning stability and operational effectiveness. The importance of latency is reinforced by smart port research, emphasizing real-time monitoring and state recognition capabilities; these systems increase the speed and quality of observation, but the value of observation depends on how quickly decisions can be integrated into control actions [15]. When latency is high, decisions can be made on stale states, feedback can arrive too late to properly credit actions, and the learned policy can become unstable—reducing OLS and limiting downstream efficiency gains. This framing is consistent with broader RL deployment literature: simulation-to-real and simulation-to-field challenges often arise because real environments introduce delayed, imperfect, and constrained feedback loops that destabilize learning and reduce reliability [4,5].

Decision latency was implemented as an action execution delay of τ steps (action selected at time, t, applied at t + τ), and robustness was assessed by comparing performance and OLStab across τ ∈ {0,2,4} under identical regimes and training protocol. In the proposed model, DLS is therefore positioned as a boundary condition that moderates the RL → OLS relationship and, by extension, the indirect serial pathway from RL through OLS and LE to congestion reduction.

2.7. Synthesis and Transition to the Conceptual Framework and Hypotheses

Across these streams, three synthesis conclusions follow. First, container congestion should be treated as accumulated waiting across coupled subsystems rather than as a single local bottleneck. Second, prediction and sensing improve visibility but do not by themselves explain how control actions reduce congestion. Third, RL is potentially valuable for terminal control when it yields stable operational behavior under realistic constraints. The conceptual framework therefore centers on OLStab as the primary explanatory mechanism, while LE and DLS are examined as downstream and boundary condition extensions within a simulation-based research design.

3. Conceptual Framework and Hypotheses

Figure 1 summarizes the proposed mechanism framework. The model treats RL as an upstream decision capability, OLStab as the primary internal operating mechanism, LE as a downstream process performance pathway, and container congestion as the system-level outcome. DLS is included as a boundary condition test on the RL-to-OLStab pathway. Importantly, the framework is evaluated in a simulated terminal environment, so the path estimates are interpreted as evidence of internal consistency with the theorized control architecture rather than as empirical causal identification from real terminal logs.

Figure 1. Conceptual framework showing Reinforcement Learning (RL), Operational Learning Stability (OLStab), Logistics Efficiency (LE), and Decision Latency Sensitivity (DLS).

4. Methodology and Research Design

This study adopts a simulation-based design that combines a queueing terminal environment with an empirical-style statistical analysis layer. The simulator generates scenario episode observations under normal and disruption regimes, and those observation-level outputs are then analyzed with regression and SEM to evaluate whether the simulated data are consistent with the proposed mechanism structure.

To reduce methodological overextension, the reported evidence is limited to three components: (i) simulation-based RL evaluation against clear baselines, (ii) reward and latency robustness checks, and (iii) regression/SEM analyses used as mechanism consistency checks on episode-aggregated synthetic observations. Additional exploratory analyses that do not materially strengthen the central argument are omitted from the revised manuscript.

The study is situated in a seaport container terminal context, but the simulator directly models the landside gate interface where queue buildup and waiting are immediately observable. In this design, gate congestion is treated as a reduced-form proxy for broader terminal congestion pressure because truck delays, queue accumulation, and gate processing capacity interact with upstream and downstream coordination. The environment should therefore be read as a gate-focused congestion control setting embedded in a broader terminal context, not as a full digital twin of berth and yard micro-dynamics. This still captures operationally relevant pressure points because lane availability, demand variability, and disruption can amplify waiting across the landside interface and create spillover risk for surrounding terminal processes. Figure 2 presents the experimental setup used for machine learning model comparison and validation.

Figure 2. Experimental setup for machine learning model comparison and validation.

4.1. Reinforcement Learning Controller and Queueing-Based Terminal Environment

Gate-side congestion control is modeled as a Markov Decision Process (MDP) in a queueing-based terminal environment, where a reinforcement learning (RL) agent adaptively controls gate processing capacity to reduce queue buildup and waiting time. The environment evolves in discrete time steps of length, Δt (e.g., 5–15 min). At each time step, trucks arrive at the terminal gate according to a stochastic arrival process with time-varying intensity (e.g., piecewise stationary Poisson arrivals calibrated to normal and disruption regimes). Gate service is represented by a multi-server queue with parallel servers (open lanes) and stochastic service times (e.g., lognormal or empirical distributions). Disruptions are modeled as exogenous shocks that reduce effective service capacity and/or increase service time variability (e.g., temporary lane closures, reduced staffing, and documentation delays). The system state updates according to standard queue dynamics: the queue length increases with arrivals and decreases with completed services, subject to capacity constraints and disruption conditions. The state vector summarizes the operational conditions relevant to gate control and includes (i) current queue length, (ii) estimated short-horizon arrival rate (e.g., moving average of recent arrivals), (iii) recent average waiting time (e.g., last hour), (iv) current number of open lanes, and (v) a disruption indicator capturing abnormal operational conditions. The agent selects an action from a discrete action set that adjusts lane capacity corresponding to decreasing, maintaining, or increasing the number of open lanes. Lane counts are bounded by operational limits, and actions that would violate these bounds are clipped to the nearest feasible lane count. This formulation reflects realistic gate control where staffing and lane allocation are adjusted incrementally rather than continuously.

The reward function is designed to minimize congestion while explicitly penalizing unstable or overly frequent control changes, enabling a direct operationalization of decision stability. Specifically, at each time step, the reward is defined as a negative cost:

$${\mathrm{r}}_{\mathrm{t}}=-\left(\mathrm{\alpha }\cdot {\overline{\mathrm{W}}}_{\mathrm{t}}+\mathrm{\beta }\cdot {\mathrm{Q}}_{\mathrm{t}}+\mathrm{\delta }\cdot \left|{\mathrm{L}}_{\mathrm{t}}-{\mathrm{L}}_{\mathrm{t}-1}\right|\right),$$

where ${\overline{W}}_t$ and $Q_t$ capture congestion outcomes, and $\left|L_t-L_{t-1}\right|$ penalizes lane reconfiguration to encourage stable decision policies. The weights $\alpha , \beta ,$ and $\delta$ are selected via sensitivity analysis to balance throughput performance and operational stability.

Robustness Protocol (Single-Factor Perturbations). Robustness experiments vary one design factor at a time while holding constant the simulator configuration (arrival/service process family, lane bounds, and disruption logic), RL algorithm and hyperparameters (Double DQN architecture, discount factor, exploration schedule, and replay/batch settings), training budget (episodes × steps per episode), and evaluation protocol (fixed evaluation horizon and identical reporting metrics). For each configuration, the results are aggregated over independent random seeds and summarized as mean (and dispersion where reported). The goal is not “causal validation” but to demonstrate that the substantive conclusions—RL reduces congestion primarily through OLStab and that latency/reward shaping do not trivially hard-code the findings—remain stable under controlled perturbations of reward shaping and decision latency regimes. Robustness to reward shaping was evaluated by sweeping the stability penalty weight, w_s ∈ {0,0.05,0.10,0.15,0.30}, while holding all other environment and training settings constant; the results are reported for Moderate and Heavy regimes using consistent congestion and stability metrics.

In this setting, the environment transition, $P\left(s_{t+1}\mid s_t,a_t\right)$ is induced by the stochastic arrival and service processes and the lane capacity chosen by the agent, which affects the number of services completed during the next step.

The RL agent is trained using Double Deep Q-Networks (Double DQNs), a value-based method suitable for discrete control actions and stochastic dynamics. The agent approximates the action value function, $Q\left(s,a\right)$, with a neural network and selects actions using an $ϵ$-greedy exploration strategy, where $ϵ$ is annealed over training to shift from exploration to exploitation. Double DQN mitigates overestimation bias by decoupling action selection and evaluation through a target network updated periodically. Training proceeds over multiple episodes, each representing a full operational horizon (e.g., one day), with randomized initial conditions and scenario sampling across normal and disruption regimes to improve generalization. The replay buffer stores transition tuples $\left(s_t,a_t,r_t,s_{t+1}\right)$ and mini-batches are sampled uniformly for stochastic gradient updates. Model selection is based on stabilized returns and performance on held-out scenarios.

To quantify Operational Learning Stability (OLStab) as a measurable property of the learned policy (rather than an assumed latent construct), stability metrics are computed directly from the trained agent’s action traces. OLS is defined using the lane adjustment behavior as (i) the action change rate (percentage of time steps with $L_t\ne L_{t-1}$) and/or (ii) the average magnitude of adjustments, $\mathbb{E}\left[\left|L_t-L_{t-1}\right|\right]$, over an episode. Higher OLS corresponds to fewer and smaller policy changes while maintaining low waiting time and queue length. This operational definition links the RL controller to stability-aware decision-making and enables empirical testing of whether improved policy stability mediates congestion reduction under both normal and disruption conditions. The proposed Double DQN gate controller is evaluated against standard baselines, including (i) a static policy that keeps a fixed lane count and (ii) a threshold heuristic that increases lanes when $Q_t$ exceeds a preset threshold and decreases lanes when $Q_t$ falls below a lower bound. Performance is assessed using congestion and service metrics such as mean and 95th percentile waiting time, mean queue length, throughput, and stability metrics (OLS). Additionally, ablation experiments are conducted by removing the stability penalty term ($\delta =0$) to isolate the contribution of stability-aware reward shaping and to test the robustness of the congestion improvements across arrival intensities and disruption severities.

This table compares the full reward with a stability penalty (w_s = 0.15) against an ablation that removes this term (w_s = 0) under Moderate and Heavy regimes. In both regimes, removing the stability penalty yields lower mean and P95 queues and therefore a lower Congestion Index. OLStab is also higher under ablation, indicating fewer lane changes and greater operational stability. This suggests that the observed performance gains are not mechanically driven by the stability term.

Robustness Summary (What Changes, What Stays). Across the reward weight sweep (Table 1) and the reward ablation comparison (w_s = 0 versus w_s = 0.15), congestion outcomes change gradually with reward shaping rather than showing a single cherry-picked improvement. The ablation results further indicate that congestion improvements are not mechanically embedded in the stability term itself, supporting the interpretation that RL performance is linked to learned control behavior captured by OLStab.

Table 1. Real sensitivity outputs: reward—weight sweep (τ = 0; Moderate and Heavy). Mean queue (episode average), P95 queue (tail), Congestion Index (mean queue + 0.25 × P95 queue), and OLStab proxy 1/ (1 + lane changes). Averaged across seeds.

Regime	$\mathit{\tau }$ (Steps)	${\mathit{w}}_{\mathit{s}}$	Mean Queue	P95 Queue	Congestion Index	OLStab
Moderate	0	0.00	312.972	562.325	453.553	0.041
Moderate	0	0.05	239.785	342.575	325.428	0.071
Moderate	0	0.10	231.201	370.250	323.764	0.021
Moderate	0	0.15	227.514	422.600	333.164	0.081
Moderate	0	0.30	104.493	138.225	139.049	0.125
Heavy	0	0.00	396.826	670.875	564.545	0.083
Heavy	0	0.05	369.889	659.700	534.814	0.058
Heavy	0	0.10	380.076	671.700	548.001	0.057
Heavy	0	0.15	418.410	757.875	607.878	0.039
Heavy	0	0.30	387.889	661.875	553.358	0.067

All results are averaged over multiple random seeds and scenario replications to ensure reproducibility and to quantify variability under stochastic arrivals and service processes.

Table 2 presents the reward ablation results for the full model (w_s = 0.15) and the ablation setting (w_s = 0) under Moderate and Heavy regimes.

Table 2. Reward ablation results (w_s = 0.15 vs w_s = 0).

Regime	Reward Setting	Mean Queue	P95 Queue	Congestion Index	OLStab
Moderate	full (w_s = 0.15)	125.198	168.833	167.406	0.043
Moderate	ablation (w_s = 0)	103.896	133.750	137.333	0.071
Heavy	full (w_s = 0.15)	541.677	974.583	785.323	0.015
Heavy	ablation (w_s = 0)	524.479	917.833	753.938	0.091

4.2. Data-Generating Process (DGP) and Unit of Analysis

The Consistency of the Unit of Analysis Across RL and SEM. The SEM does not operate on within-episode time steps; it models between-episode variation: Reinforcement Learning (RL)_i → OLS_i → LE_i → Congestion_i (with DLS_i as a boundary condition). Accordingly, the mediation estimates are interpreted as between-episode associations that are consistent with the theorized mechanism ordering, not as within-day causal dynamics. This alignment avoids a mismatch where an RL process is sequential, but the SEM is estimated on temporally correlated slices.

Observation-Level Construct Mapping. RL is recorded as a scenario-level RL capability score (scaled to [0,1]) representing the strength of the RL-driven decision signal in that scenario episode. Operational Learning Stability (OLStab) is computed as an episode-level stability index derived from the RL action trace and performance trace (e.g., lower action-switching frequency and smaller action magnitudes, together with lower variability of congestion-related cost/reward across the episode), then normalized to [0,1]. Logistics Efficiency (LE) is computed as an episode-level process performance index, summarizing how effectively the terminal converts arrivals into a completed service (e.g., higher throughput, lower average service/turnaround time, and smoother flow), then normalized to [0,1]. Decision Latency Sensitivity (DLS) is recorded as an episode-level latency/friction index (scaled to [0,1]) representing the scenario’s latency condition or the measured sensitivity of learning/operations to imposed delay τ_i. Container congestion is computed as an episode-level congestion outcome that aggregates waiting/queue accumulation over the full episode (e.g., average/percentile waiting time and queue length), yielding the continuous congestion score used as the SEM dependent variable.

Step-By-Step DGP (one row). For observation i: (1) draw a random seed s_i; (2) sample an operating regime r_i ∈ {normal, disruption} and scenario parameters θ_i (e.g., base truck arrival intensity λ_i, service-capacity/service-rate parameters μ_i, lane bounds, and disruption severity/shock timing if r_i = disruption); (3) sample a decision latency condition τ_i that governs the delay between state observation, action execution, and reward/feedback availability; (4) run the discrete time queueing-based terminal environment for T steps under the RL controller (trained policy) and log episode trajectories of state, action, reward/cost, queue length, waiting times, and throughput; and (5) compute observation-level indices by aggregating the logged trajectories into single scalars and applying normalization where appropriate.

Sampling Frame and Independence. We generate n = 500 observations as independent scenario realizations. Observation i is generated with its own random seed, s_i, and its own sampled scenario parameters, θ_i. No observation is created by chopping a long run into multiple rows; instead, each row corresponds to a full episode simulated from t = 1,…,T decision steps. Dependence is therefore confined within an episode (the simulator’s internal time evolution), while the exported dataset used for regression/SEM is cross-sectional: the 500 rows are independent across i because they arise from non-overlapping runs with distinct seeds and re-sampled parameters.

4.3. Analysis Structure

The empirical evaluation proceeds in three steps: first, the study benchmarks RL control performance in the simulator; second, it checks robustness through reward and latency perturbations; third, it estimates regression and structural path models on episode-level synthetic observations to examine whether the resulting data are consistent with the theorized mechanism.

The simulator is therefore the data-generating process for the analytical dataset. Each row corresponds to one complete scenario episode, and the SEM layer models variation across those scenario episodes rather than within-episode time steps.

4.4. Data Sources/Simulation Parameters

The empirical tests were conducted using a synthetic, gate-focused congestion dataset containing 500 observations and five continuous variables: RL, OLS, LE, DLS, and congestion. The empirical dataset is generated from a queueing-based terminal simulator that explicitly models truck arrivals, gate service capacity, and disruption shocks, producing scenario-level observations used for subsequent statistical testing. Table 3 summarizes the simulator and RL parameterization used in this study.

Table 3. Simulator + RL parameterization.

Parameter	Symbol	Value/Range	Notes
Arrival process	$\lambda \left(t\right)$	Piecewise-stationary Poisson; Light/Moderate/Heavy $\lambda \in \{10,18,26\}$ trucks per $\Delta \mathrm{t}$	Regime conditioning; scenario families vary demand level
Service model	$S/\mu$	Per-lane effective capacity $\mu \approx 3$ trucks/lane/$\Delta t$	Baseline service; lane-based capacity
Time step	$\Delta t$	10 min	Discrete simulator step
Episode length	$T$	144 steps (≈24 h)	One episode = one operational day
Lane bounds	$C_{\mathrm{m}\mathrm{i}\mathrm{n}},C_{\mathrm{m}\mathrm{a}\mathrm{x}}$	1 to 6 lanes	Actions clipped to bounds
Disruption timing	$t_{\mathrm{shock}}$	Uniform $\left[36,96\right]$ steps	Shock start during the day
Disruption duration	—	3–9 steps	Scenario-dependent
Disruption severity	$\kappa$	Capacity loss 20–40% and/or service inflation +10–30%	“Normal vs disruption” regime families
Latency mechanism	$\tau$	Action execution delay $\tau \in \{0,2,4\}$ steps	Action chosen at $t$ applied at $t+\tau$
Discount factor	$\gamma$	0.95	RL return discounting
Exploration schedule	$ϵ\left(t\right)$	$ϵ:$ 0.30 → 0.05 (annealed)	Evaluation uses greedy policy $ϵ=0$
Reward weights (queue)	$w_c$	1.0	Queue/congestion penalty weight
Reward weights (stability)	$w_s$	Sweep (Table 1): 0.00–0.30	Sensitivity/ablation against stability penalty
Seeds	$s_i$	500 unique seeds	One seed per scenario episode observation
Scenario families	—	18 families	$3$ traffic × $2$ regimes × $3$ latencies

Simulator parameters are chosen to span Light, Moderate, and Heavy traffic regimes, as well as normal and disruption conditions, rather than to optimize a single tuned case. Arrivals, service capacity, disruption timing, and action execution delay are all varied within stated ranges so that the generated observations reflect multiple plausible operating conditions inside the queueing environment.

Episode-level synthetic observations were analyzed using a small set of descriptive, regression, and structural path models to evaluate whether the generated data were consistent with the theorized mechanism. The analysis was intentionally limited to baseline RL performance, robustness checks, and mechanism-oriented regression/path analysis in order to avoid methodological overextension.

Interpretation Scope (Simulation-Based Inference). All regression, mediation-style, and structural path analyses are performed on synthetic scenario episode observations generated by the simulator. Accordingly, the coefficients, indirect effects, and fit summaries are interpreted as evidence that the simulated data are consistent with the proposed mechanism structure, not as empirical causal estimates for real terminal operations.

Basic data screening confirmed no missing values or duplicated rows and indicated that the modeled variables behaved within plausible ranges. Because the core variables are simulation-derived single-index measures rather than multi-item survey scales, reliability is treated here as computational reproducibility and stability across repeated runs rather than as survey-style internal consistency.

The Figure 3 presents Q–Q plots for reinforcement learning capability (RL), Operational Learning Stability (OLStab), Logistics Efficiency (LE), Decision Latency Sensitivity (DLS), and container congestion. Across variables, the empirical quantiles largely align with theoretical normal quantiles, with only mild deviations at the distribution tails. These patterns support using linear models as descriptive summaries of the synthetic observations.

Figure 3. Quantile–quantile (Q–Q) plots for key study variables. The red lines represent the theoretical normal reference lines.

All analyses were implemented in standard R 4.5.1+Python 3.11.9 workflows for transparency and reproducibility.

5. Results

The results are reported with an emphasis on the revised manuscript’s narrower claim: whether the simulated evidence supports OLStab as the main pathway connecting RL-based control to lower congestion pressure in a gate-focused terminal environment, and whether LE and DLS add robust explanatory value beyond that primary pathway. Descriptive statistics for the key study variables are presented in Table 4.

Table 4. Descriptive statistics for key study variables (N = 500).

Variable	Mean	SD	Min	Median (50%)	Max
RL	0.4986	0.2987	0.0051	0.5132	0.9930
OLS	0.3939	0.1729	0.0145	0.4051	0.7723
LE	0.2611	0.0927	0.0280	0.2629	0.4857
DLS	0.4965	0.2870	0.0032	0.5089	0.9983
Congestion	58.4648	12.0619	27.3718	57.7788	87.9704

The pairwise relationships among Reinforcement Learning (RL), Operational Learning Stability (OLStab), Logistics Efficiency (LE), Decision Latency Sensitivity (DLS), and congestion are shown in Figure 4.

Figure 4. Pairwise relationships among RL, OLS, LE, DLS, and congestion, showing positive associations among learning variables and negative associations with congestion.

The pairplot provides a descriptive overview of the generated relationships. RL, OLStab, and LE are positively associated with one another, while congestion is negatively associated with RL and OLStab. DLS varies across the simulated operating space but shows comparatively weak associations with the other variables, which already suggests that the boundary condition effect may be limited in this dataset.

Nested regression results show that RL alone explains a substantial share of congestion variation and that model fit improves when OLStab and DLS are added. The strongest overall specification is the full model, including RL, OLStab, LE, and DLS, but the key interpretive point is not simply predictive performance. Rather, the incremental improvement after adding OLStab supports the view that stability-related information is central to explaining simulated congestion outcomes.

Table 5 reports train test performance for the nested congestion models. The pattern suggests that RL and OLStab jointly capture the main signal in the synthetic data, while LE adds relatively little incremental explanatory power once the primary stability pathway is included.

Table 5. Train test performance comparison for nested congestion prediction models.

Model	Train RMSE	R2	Adj. R2	Test RMSE	Test R2
Model 1: RL only	7.083	0.654	0.654
Model 2: RL + OLS	6.766	0.685	0.683
Model 3: RL + OLS + LE	6.747	0.686	0.685
Model 4: RL + OLS + LE + DLS	4.899	0.835	0.833	4.531	0.864
Model 5: OLS + LE + DLS	5.780	0.770	0.768
Model 6: Full model	4.899	0.835	0.833

When operational factors are compared across specifications, the explanatory contribution of OLStab is more stable than that of LE. This is consistent with the paper’s revised interpretation that learning stability is the principal mechanism in the simulated environment, whereas efficiency is better treated as a related downstream outcome whose unique role becomes less clear once shared variance with RL and OLStab is taken into account.

Mediation results should therefore be read cautiously. The total association between RL and congestion is negative, and the indirect pathway through OLStab is also negative, which is consistent with a primary stability-based mechanism. Because the estimates are derived from synthetic scenario episode observations generated by a designed simulator, the path coefficients and indirect effects are interpreted as evidence that the simulated data are consistent with the proposed mechanism structure, not as empirical causal estimates for real terminal operations.

H1 Pattern. The simulated data are consistent with H1: RL is positively associated with OLStab in both the correlation structure and the regression results, indicating that stronger RL-driven decision capability aligns with more stable learning-enabled operational behavior in the simulated environment.

H2 Pattern. The expected directional pattern for H2 appears in the simulated environment: OLStab is positively associated with LE, which is consistent with the idea that a more stable control policy can sustain smoother operational flow and reduce process friction.

H3 Pattern. The simulated data are directionally consistent with H3 at the bivariate level: LE is negatively associated with congestion in the descriptive structure of the data, matching the expectation that smoother terminal flow aligns with lower accumulated waiting.

H4 Pattern. The full serial pathway hypothesized in H4 is not robustly recovered in the joint specification. Once predictors are modeled together, LE no longer behaves as a stable final stage mediator and its unique coefficient changes sign. This indicates shared variance and suppression with RL and OLStab, so the revised interpretation emphasizes OLStab as the dominant mechanism and treats the full serial chain as a non-robust extension within the simulated data.

To assess the boundary condition argument, OLS was regressed on RL, DLS, and the RL × DLS interaction. This specification tests whether decision latency weakens the ability of RL to generate stable learning behavior.

The interaction term is not statistically significant, so the theorized moderation by DLS is not robustly recovered within the examined latency range. In the revised manuscript, this null result is treated as informative evidence about the limits of the current simulation rather than as support for a strong boundary condition claim. The moderation regression results for Operational Learning Stability (OLStab), including Reinforcement Learning (RL), Decision Latency Sensitivity (DLS), and their interaction (RL × DLS), are presented in Table 6.

Table 6. Moderation regression for OLStab (RL, DLS, and RL × DLS).

Term	Estimate (β)	Std. Error	t	p-Value	95% CI
RL	0.5159	0.026	20.106	<0.001	[0.465, 0.566]
DLS	0.0132	0.026	0.506	0.613	[−0.038, 0.065]
RL × DLS	−0.0254	0.044	−0.581	0.562	[−0.111, 0.060]

6. Operational Learning Stability (OLStab)

Operational Learning Stability (OLStab) is operationalized here as an episode-level stability index derived from observable controller behavior and performance traces rather than as a latent survey construct. The intent is practical: to capture whether a learned policy remains consistent, low-volatility, and governable under variability and disruption. In that sense, OLStab is best viewed as a simulation-grounded operational metric designed for this study rather than as a universal theory term.

This operationalization allows OLStab to be monitored from routine runtime signals such as action change frequency, adjustment magnitude, and variability in congestion-related cost. It also makes the measure reproducible within the simulation framework. At the same time, its broader construct validity remains a question for future research using real terminal logs and alternative stability formulations.

7. Discussion

The revised results indicate that RL-enabled decision capability can reduce simulated congestion pressure in a gate-focused terminal environment, primarily by improving Operational Learning Stability rather than by relying on prediction accuracy alone. This matters because congestion in terminals is a system-level waiting problem across coupled processes, while the present simulator captures the gate interface as the most direct control point and uses it as a reduced-form proxy for broader terminal pressure.

Downstream relationships are directionally consistent with the broader argument, but they are weaker than the primary stability mechanism. OLStab is positively associated with LE, and LE is negatively associated with congestion in bivariate structure. However, the serial pathway through LE is not robust once RL and OLStab are jointly modeled. This suggests that in tightly coupled terminal systems, efficiency improvements may partly reflect upstream stability rather than function as a clearly separable final stage mediator.

The null moderation result also sharpens the contribution. DLS was theorized as a deployment friction that could weaken the RL-to-OLStab relationship through delayed observationֲ–action–feedback loops, yet that pattern does not emerge in the examined latency range. The present evidence therefore does not justify a strong moderation claim, although it does motivate future work on threshold latency regimes, delayed rewards, and partial observability.

Taken together, the findings support a narrower but stronger conclusion: within this simulation framework, RL appears to reduce congestion mainly when it produces stable operational behavior. The paper’s contribution is therefore best understood as theory generalization from a designed control environment, not as population generalization from real-world terminal data.

This reframing also clarifies the practical boundary of the study. The results provide a useful mechanism lens for digital twin or simulator-based evaluation of terminal control policies, but external validity should be established through replication on real operational logs, richer process KPIs, and multi-terminal settings.

8. Implications

For real terminal environments, the framework is most relevant as a staged evaluation tool rather than as a ready-to-deploy optimization system. The most plausible near-term users are terminal operators, automation managers, and digital twin teams who are testing gate operations control policies before live deployment. In practice, the framework is best applied first in simulation or a digital twin, where candidate RL controllers can be screened under both normal and disruption regimes before any live pilot is attempted.

In that screening process, operators should monitor not only average queue reduction but also stability-oriented indicators such as action switch frequency, policy churn, waiting time variability, truck turn time, service consistency, and any spillover signals linked to yard pressure or berth delay. These measures clarify whether the controller is merely aggressive in reducing queues or is behaving in a stable, governable way that operations staff can trust.

OLStab therefore informs a concrete go/no-go decision: whether an RL controller is stable enough to move from simulated testing to a bounded live pilot. Controllers that improve queue outcomes but remain high-churn, fragile, or overly sensitive to delay should be held back. Only controllers that remain stable under variability and disruption should progress to limited pilots with clear escalation rules.

That deployment pathway should remain staged: simulation → digital twin → limited live pilot → broader rollout. At each step, safeguards such as operator override, bounded exploration, fallback rule-based control, and explicit latency thresholds should be in place. The main practical contribution of this study is therefore not a claim of immediate deployment readiness, but a disciplined criterion for deciding whether an RL controller is operationally governable enough to justify real-world testing.

Seen this way, the paper’s practical value lies in offering a governance-oriented screening logic for RL-enabled terminal control rather than a universally validated deployment recipe.

9. Limitations and Future Research

This study has several limitations that motivate a focused future research agenda. First, the evidence is generated from a simulation-based dataset derived from a gate-focused control environment. That supports internal consistency and controlled mechanism testing, but it does not reproduce all institutional, behavioral, and infrastructural frictions of real ports or the full berth–yard–gate micro-dynamics. Future work should therefore validate the framework with operational logs and KPI archives such as yard occupancy, truck turn time, berth productivity, and rehandle rates.

Second, the design is effectively single-environment in its parameterization, which may under-represent structural differences across terminal types, automation maturity, and regulatory settings. Multi-terminal and multi-regime studies are needed to test whether the RL-to-OLStab mechanism remains stable across contexts.

Third, the theorized boundary condition involving DLS is not robustly recovered in the current interaction model. Future studies should examine whether latency effects emerge only beyond threshold values, under delayed-reward settings, or when latency is measured directly from operational systems rather than imposed as a stylized simulation condition.

Fourth, construct overlap among RL, OLStab, and LE remains an important measurement issue. Future work should refine LE into more orthogonal process dimensions and compare alternative formulations of OLStab using observable stability indicators such as policy churn, variance of dispatch decisions, and convergence diagnostics.

Finally, deployment feasibility depends on human–AI interaction, fail-safe design, and low-latency system integration. Future research should therefore examine governance, operator override behavior, explanation interfaces, and compute latency budgets alongside algorithmic performance.

10. Conclusions

This study offers a simulation-based mechanism interpretation of how reinforcement learning may reduce container congestion pressure. The central result is not that a full serial mediation chain is established but that Operational Learning Stability emerges as the most credible mechanism linking RL-enabled control to lower congestion within the designed gate-focused queueing environment. LE is directionally relevant but not a robust independent final stage mediator once stability is jointly modeled, and DLS does not robustly moderate the RL-to-OLStab pathway in the examined range. The main contribution is therefore a more disciplined mechanism account: RL appears operationally useful when it generates stable, repeatable, governable control behavior under variability and disruption. That conclusion is bounded to a synthetic setting, but it provides a clearer foundation for future digital twin studies and for empirical validation on real smart port operations.