Container Slot Allocation with Empty Container Repositioning: A Multi-Objective Optimization Approach

Abstract

Trade imbalances and equipment shortages are making it increasingly important to coordinate container slot allocation with empty container repositioning on liner services. This paper develops an integrated bi-objective mixed-integer model for voyage-level slot planning on a fixed cyclic route. The model jointly decides booking acceptance, inter-voyage shipment, and empty repositioning with port-level empty-inventory dynamics and leg-based vessel capacity constraints. We optimize two conflicting objectives: maximizing operational profit and minimizing empty container TEU-miles. To solve the model at practical scales, we propose a hybrid evolutionary framework, NSGA-II-RL, which uses a lightweight Q-learning controller to adapt operator and repair choices during NSGA-II evolution. Computational experiments on representative service route instances show that NSGA-II-RL produces diverse Pareto-efficient solutions and improves hypervolume relative to fixed-operator and random-control variants, revealing clear trade-offs between profitability and repositioning intensity.

1. Introduction

Container shipping has established itself as the fundamental element of global trade, carrying more than 80% of world cargo by volume [1]. Despite recent raises in freight rates, liner shipping companies nevertheless face substantial operating cost constraints and uncertainties that limit their total profitability. First, it is expected that carrier operating expenses would increase in order to comply with new environmental laws, such as stricter fuel emission standards and carbon intensity requirements [2]. Second, operational volatility has lately increased due to geopolitical interruptions. In late 2023 and early 2024, container shipping rates sharply and unpredictably increased due to rerouting brought on by the Red Sea crisis and restrictions in the Suez and Panama Canals. The authors of Sun et al. [3] estimated that Red Sea rerouting alone raised overall route economic costs by at least 51% using a fleet size optimization model and an AIS-based bottom–up routing model. Third, enduring structural overcapacity continues to pose a significant long-term challenge. As noted by UNCTAD [1], between 2010 and 2023, global fleet capacity increased by 78.5%, whereas demand grew by merely 34%, with some of this discrepancy obscured by rises in distance-adjusted demand. These factors suggest that capacity efficiency remains critical for carriers aiming to remain competitive in increasingly current operating environments.

Against this background, the efficient allocation of limited vessel capacity has become a central issue for liner shipping companies. In the short term, shipping routes, service schedules, and vessel capacities are predominantly fixed. As a result, profitability largely relies on the allocation of available slot capacity among containers with heterogeneous demand characteristics and freight rates [4]. This decision problem, widely known as the container slot allocation problem (CSAP), lies at the core of liner shipping operations and has attracted sustained attention from both practitioners and researchers. Under the domain of revenue management, CSAP focuses on dynamically accepting or rejecting booking requests in order to maximize revenue under capacity constraints [5]. Nonetheless, the majority of CSAP models focus exclusively on capacity allocation for laden containers. This simplification overlooks the fact that empty containers must also be relocated utilizing the same limited vessel slots, thereby directly competing with revenue-generating cargo for capacity [6,7].

Empty container repositioning (ECR) constitutes one of the most enduring structural challenges within liner shipping and is broadly acknowledged as a core planning issue in container shipping [8,9]. Persistent trade imbalances between export-dominant and import-dominant regions create systematic mismatches between the places where empty containers become available and the places where export demand arises across ports. As a result, liner shipping companies have to reposition large volumes of empty containers from surplus ports to deficit ports at their own expense, making the economic and operational significance of ECR difficult to ignore [10,11]. Industry estimates suggest that empty container repositioning costs carriers between USD 15 and 20 billion annually, accounting for approximately 5% to 8% of total operating costs [12].

From a capacity occupation perspective, ECR also constitutes a considerable share of global container shipping activity. Drewry estimates that, historically, approximately 20% of all ocean container movements have involved the repositioning of empty containers since 1998 [13]. Furthermore, since ECR often involves long-haul movements along major trade lanes, it consumes a disproportionately large amount of vessel slot capacity. Recent industry analyses further indicate that, when measured in terms of empty container TEU-miles (empty TEU-miles), empty containers may account for up to 40% of global container shipping activity [14]. Consequently, ECR and CSAP are inherently coupled through shared vessel capacity. Slots assigned to empty containers reduce the space available for revenue-generating laden cargo in the short term, while insufficient repositioning may lead to empty container shortages at export-oriented ports and lost bookings in subsequent periods. This interdependence highlights the necessity of considering ECR jointly with CSAP, rather than treating ECR as a separate downstream operational task.

In practice, carriers care about both profitability and capacity efficiency when empty repositioning competes with laden shipments for limited vessel slots [15]. When empty repositioning is involved, this trade-off becomes more evident. Allocating more slots to empty containers can secure equipment availability for future demand, but it directly reduces the slots available for laden containers and may lower current-period laden slot share and overall profit [16,17].

Motivated by the coupling between CSAP and ECR, this study formulates an integrated multi-objective optimization model for voyage-level slot planning on a given liner shipping service. The model determines how limited vessel slot capacity is allocated to laden containers and empty containers across voyages. Two objectives are considered to reflect key managerial concerns. The first objective maximizes total profit calculated by total freight revenue minus operational cost. The second objective minimizes empty TEU-miles, which discourages excessive empty movements and thus reflects utilization-related efficiency.

Solving this model requires both effective exploration of trade-offs and robust feasibility handling. We therefore develop an enhanced NSGA-II solution approach with reinforcement-learning-guided operator control. A Q-learning agent observes the population state during the evolutionary process and adaptively selects genetic operators, while receiving rewards based on hypervolume improvement and feasibility-related signals. This design improves search efficiency and solution stability under tight capacity and inventory constraints, and it produces a diverse set of Pareto-efficient solutions that make the trade-off between profit and empty TEU-miles explicit. Numerical experiments based on representative liner service scenarios are conducted to evaluate solution quality and computational performance, and to derive managerial insights on when capacity should be reserved for empty container repositioning versus allocated to revenue-generating laden cargo.

The main contributions of this work are summarized as follows. First, we develop an integrated voyage-level formulation that jointly determines CSAP and ECR decisions under shared vessel capacity. Second, we formulate a profit–empty TEU-miles bi-objective optimization model to explicitly capture the resulting trade-off. Third, we design an efficient solution approach that can generate diverse Pareto-efficient solutions for realistic instance scales.

The remainder of this paper is organized as follows. Section 2 reviews related studies on CSAP, ECR and multi-objective optimization methods. Section 3 presents the integrated multi-objective formulation and key modeling assumptions. Section 4 describes the proposed NSGA-II based solution approach with reinforcement-learning-guided operator control. Section 5 reports experimental settings and computational results. Section 6 discusses managerial insights of the results and outlines directions for future work. Section 7 concludes the paper.

2. Literature Review

2.1. CSAP in Liner Shipping

In the short term, liner carriers typically operate pre-announced services with fixed shipping routes, service schedules, and vessel capacities. Under such a setting, operational revenue management largely depends on how vessel slots are allocated to heterogeneous bookings, which has a direct impact on carriers’ profitability and service performance [4,18]. Typical decisions of CSAP include accepting or rejecting booking requests and allocating slots across OD flows and customer groups, subject to operational constraints such as vessel capacity, reefer-plug limits, weight and slot-type restrictions, and the fixed schedules of liner services. Due to the large capacity scale of modern containerships and the leg-coupled capacity interactions induced by network operations, CSAP models often lead to large-scale mixed-integer programs that are computationally challenging [4].

A central modeling ingredient in CSAP is customer and product segmentation, because different demand segments contribute differently to volume, revenue, and service obligations. In particular, liner demand is often partitioned into contract and spot segments [4]. Contract customers typically commit volumes under longer-term agreements with relatively stable rates and implicit service expectations. This can stabilize demand for the carrier, but it also constrains operational flexibility when capacity becomes tight. Spot demand is more price-sensitive and uncertain, which often requires different acceptance and allocation decisions. This segmentation motivates deterministic slot allocation models that compute segment-specific allocations or acceptance thresholds as a planning backbone [5,16,19,20,21]. For instance, Wang et al. [20] studied seasonal-scale revenue management and formulated a deterministic optimization model that selects and routes multiple container types under volume and weight capacities while linking operating costs to operational decisions such as sailing speed and service deployment requirements. This study shows that even under deterministic demand, CSAP is tightly coupled with multiple capacity dimensions and cost drivers. In a later study, Wang et al. [21] investigated overbooking and delivery-delay-allowed strategies for slot allocation to reduce capacity waste and improve profitability when actual demand realization or shipment execution deviates from plan, thereby moving from pure static planning toward implementable operational control rules. More recently, CSAP research has increasingly incorporated stochastic demand and execution uncertainty, developing uncertainty-aware allocation models and control policies that hedge against demand volatility while maintaining feasibility under network capacity constraints [22,23,24,25,26,27,28].

In addition, CSAP decisions are rarely assessed solely by short-term profitability. Capacity efficiency is often treated as a key operational indicator in liner services, because insufficient utilization weakens scale economies and raises unit costs, yet maintaining some slack capacity may be necessary for operations, implying an inherent profit–utilization trade-off [15]. In this study, we characterize this trade-off using a profit–empty TEU-miles bi-objective setting, where empty TEU-miles measure the distance-weighted transport work associated with empty movements. This practical emphasis is also reflected in the slot allocation literature. For example, Lu et al. [16] formulate an integer-programming planning model that maximizes potential profits and report that it also provides a higher slot utilization rate. More explicitly, Wong et al. [17] develop a three-echelon slot allocation framework for yield and capacity utilization management, highlighting that utilization and revenue considerations are often treated jointly in operational planning.

2.2. Integrated CSAP and ECR Models

Beyond allocating scarce slots to laden container bookings, liner carriers must ensure that empty containers are available at the right ports and times to support export demand, which gives rise to the ECR problem [29]. Existing ECR studies can be broadly distinguished by the underlying decision network, including vessel-based repositioning on seaborne shipping networks, inland or inter-modal repositioning across depots and hinterland nodes, and ECR embedded in broader fleet or network planning problems [8,9,29,30]. Among these scopes, the seaborne liner shipping setting is particularly relevant to slot allocation, because empty containers are transported on the same maritime legs as laden cargo and therefore consume the same vessel slots. Consequently, ECR is inherently capacity-coupled with slot allocation in seaborne shipping networks, motivating integrated CSAP and ECR models that jointly determine laden allocations and empty flows under shared leg-capacity constraints.

A stream of studies couples empty and laden decisions within a single voyage or planning cycle, while treating empty containers as exogenous inputs to the allocation model. The authors of Ting and Tzeng [5] developed an integer-programming model that simultaneously allocates slots to laden and empty containers between port pairs, and enforces empty container repositioning through dedicated port-level replenishment requirements specified exogenous. Extending this idea from a single voyage to a seasonal planning horizon, Lu et al. [16] optimized slot allocation for a cyclic liner service and explicitly recognized that slot spaces must be assigned across heterogeneous container categories under practical vessel limits, while still relying on forecast- or policy-driven empty needs rather than endogenizing container circulation. Moving beyond allocation on a fixed service, Meng and Wang [31] integrated ECR into liner shipping service network design, showing how empty container flows can be embedded in broader network-structure decisions under maritime capacity constraints. More recently, Wong et al. [17] proposed a three-echelon slot allocation framework for yield and utilization management across planning levels, where empty container movements are part of the planning background but the emphasis is on coordinated slot allocation mechanisms. Overall, these studies demonstrate how empty-related considerations can be embedded into CSAP under shared capacity, yet most describe empty needs through exogenous targets or inputs rather than explicitly modeling inter-temporal container circulation and availability within the allocation model.

In contrast, another stream introduced port-side empty container inventories and balance over time, thereby linking container allocation decisions to the feasibility of empty container supply. The authors of Zurheide and Fischer [19] developed a network-based revenue-management formulation in which the service network is indexed with explicit time and ship-cycle relations and slot capacity is allocated to both laden and empty containers accordingly. More recently, Wang et al. [7] modeled port-side empty container state transitions and incorporated auxiliary expressions for remaining onboard containers across voyages. However, the correspondence between the voyage index and physical vessel cycles in a multi-vessel, multi-voyage service is left implicit, making the cross-voyage port-and-onboard container flows unclear. Moreover, to the best of our knowledge, existing integrated CSAP and ECR models are still predominantly formulated with a single economic objective, and explicit Pareto-based multi-objective formulations that jointly optimize profit and ECR effort remain scarce.

Table 1. Representative studies on slot allocation, empty container repositioning, and integrated planning.

Paper	CSAP Decisions	ECR Decisions	Obj.	Method
Ting 2004 [5]	Accept/reject bookings; allocate laden slots across OD pairs	Allocate empty slots/flows with exogenous replenishment requirements	Profit	Mathematical programming
Lu 2010 [16]	Seasonal slot allocation planning across OD flows	Allocate vessel slots to empty containers	Profit	Mathematical programming
Meng 2011 [31]	Service network design decisions	Reposition empty container flows on the designed network	Cost	Mixed-integer programming
Zurheide 2012 [19]	Network-based slot allocation on a time-expanded service network	Port–time empty balancing with repositioning, leasing, and storage decisions	Profit	Rolling re-optimization
Wang 2015 [20]	Assignment multi-type laden containers to container routes; fleet size and sailing speed	—	Profit	Mixed-integer nonlinear programming model; global optimization
Fu 2016 [23]	Slot allocation with minimum quantity commitment under uncertain demand	—	Robust profit/ commitment satisfaction	Robust optimization
Wang 2016 [24]	Stochastic resource allocation for containerized cargo networks with uncertain capacities	—	Expected profit/ risk-aware feasibility	Stochastic programming; sample average approximation
Wang 2017 [25]	Tactical slot-configuration and deployment control	—	Minimum delay of dry/reefer containers	Two-stage simulation-based optimization
Wang 2019 [21]	Slot allocation with overbooking and delivery-delay-allowed control	—	Profit	Lagrangian relaxation; surrogate subgradient method
Wang 2021 [26]	Two-stage stochastic slot allocation differentiating contract vs. spot segments	—	Expected profit	Two-stage stochastic programming; sample average approximation
Wong 2022 [17]	Three-echelon slot allocation planning with cargo shifting and slot exchange	Allocate vessel slots to empty containers	Yield and utilization	Branch-and-bound search; genetic algorithm; deep neural network
Zhao 2022 [27]	Planning-level slot allocation under uncertain demand	—	Robust profit	Distributionally robust optimization
Liang 2023 [28]	Service-oriented slot allocation policy with explicit fulfillment rate under stochastic demand	—	Profit; service level	Stochastic policy
Wang 2024 [7]	Collaborative slot allocation for laden containers	Empty repositioning decisions	Profit	Branch-and-cut algorithm

summarizes representative studies across three closely related streams: CSAP, ECR, and integrated planning where CSAP and ECR are jointly considered. As the table shows, most CSAP studies are profit-driven and focus on booking acceptance and slot allocation decisions, whereas ECR is often handled separately through balancing, leasing, and inventory decisions, or is treated as an exogenous requirement. While integrated models exist, explicitly capturing the interaction between laden and empty decisions under shared vessel capacity remains limited in the literature. Moreover, only a limited number of studies go beyond purely profit-oriented objectives to explicitly quantify the operational burden of empty movements, and even fewer model and optimize the profit–empty TEU-miles trade-off in an integrated setting where laden and empty containers compete for the same vessel capacity. These observations motivate our profit–empty TEU-miles bi-objective formulation and the solution framework presented next.

2.3. Multi-Objective Optimization Methods in Maritime Logistics

In maritime logistics and liner operations, decision quality is usually assessed against multiple criteria rather than profit alone, such as capacity efficiency, equipment-balance stability, and operational robustness. These criteria can be intrinsically conflicting. For example, aggressively prioritizing short-term profit by allocating scarce slots to laden shipments may crowd out empty repositioning moves, which can exacerbate equipment imbalance and increase shortage-related operational costs later on.

Multi-objective optimization formalizes such trade-offs by seeking a set of Pareto-efficient solutions, where improving one objective necessarily worsens at least one other objective. Compared with scalarization approaches such as weighted-sum or $ϵ$-constraint methods, which can be sensitive to parameter choices and often require repeated solves to map a Pareto frontier, evolutionary multi-objective optimization offers a practical alternative for complex discrete planning models by generating a diverse approximation of the non-dominated set in a single run [32,33].

Among multi-objective evolutionary algorithms, the NSGA-II remains one of the most widely used baselines due to its fast non-dominated sorting, elitist selection, and crowding-distance mechanism for diversity preservation [34,35]. These features make NSGA-II particularly suitable for constrained, leg-coupled, integer decision problems in maritime logistics, where problem-specific encoding and feasibility-handling operators can be embedded to exploit structure and maintain solution feasibility during evolution.

Evidence from the maritime and port-operations literature further supports the suitability of NSGA-II for networked operational problems with competing objectives and complex feasibility structures. Representative port-side applications include multi-objective berth allocation and integrated berth–quay–crane planning at container terminals, where modified or enhanced NSGA-II variants embed constraint-handling and local search to cope with tightly coupled spatial–temporal constraints [36,37,38]. On the seaborne side, Song et al. [39] formulate a stochastic multi-objective tactical planning problem for liner services under uncertain port times and solve it using a simulation-based NSGA-II, while Ma et al. [40] apply NSGA-II to route and speed optimization under weather conditions and emission control area regulations to generate explicit cost–time trade-offs. These studies suggest that NSGA-II provides a natural foundation for our setting, where we aim to reveal the trade-off between profit and empty repositioning effort, quantified by empty TEU-miles, in an integrated CSAP and ECR model.

3. The Integrated Slot Allocation and Empty Repositioning Problem

3.1. Problem Setting

We study a short-term slot planning problem on a given liner service network. Route design, service schedules, and vessel deployment are taken as exogenous, and the focus is on voyage-level slot allocation for laden container shipments and empty container repositioning. Under this setting, the carrier uses limited vessel slot capacity both to transport laden containers requested by shippers and to reposition empty containers to mitigate imbalances across ports. Laden and empty containers therefore compete for the same slots, implying that allocation and repositioning decisions should be determined jointly rather than in isolation.

Laden container booking requests are heterogeneous. Following common practice, we distinguish contract shipments from spot-market shipments [28]. Contract shippers book capacity under long-term agreements and typically receive lower rates. In return, the carrier is expected to provide a guaranteed level of fulfillment over the planning horizon, which may be specified through minimum volume commitments or agreed reliability targets. Importantly, freight is generally charged on actually shipped volume, while failing to meet contractual commitments may lead to penalties or loss of future business. Spot shippers, in contrast, request capacity on a short-term basis with higher market-driven rates and without explicit fulfillment commitments. This heterogeneity implies that the carrier must balance contractual fulfillment and revenue opportunities, while reserving sufficient capacity for ECR to ensure equipment availability at export-oriented ports in subsequent voyages.

3.1.1. Service Network and Planning Horizon

We consider a liner company operating a cyclic service with a fixed port rotation. Let $\mathcal{L}=\{1,2,\dots ,L\}$ denote the ordered set of port-call indices along the route, where a physical port may appear multiple times and thus have multiple indices. Each consecutive pair of port calls defines a directed shipping leg. We denote the set of legs by $\mathcal{A}$, indexed by a, where legs are in one-to-one correspondence with origin port-call indices. Specifically, $a\left(i\right)=(i,i+1)$ for $i\in \{1,\dots ,L-1\}$, and the wrap-around leg is $a\left(L\right)=(L,1)$. The planning horizon contains a sequence of voyages $\mathcal{V}=\{1,2,\dots ,V\}$, and voyage v is operated by a predetermined vessel with slot capacity $Ca{p}^v$ (in TEU). A container loaded at port-call index i and discharged at index j occupies slots on every leg along the forward path from i to j on the cyclic route. Once discharged, those slots become available again for downstream shipments, so OD flows compete for capacity on overlapping legs. To represent transportation requests on this cyclic service, we define the set of origin–destination pairs as $\mathcal{OD}$, indexed by $(i,j)$, allowing both $i<j$ and $j<i$ to capture wrap-around demand. Laden booking requests are classified into a finite set of service classes $\mathcal{S}$ that reflect heterogeneous rates and service-level requirements. In particular, we distinguish two classes, namely contract shipments and spot-market shipments. Contract shipments are associated with explicit service-fulfillment requirements specified in long-term agreements, whereas spot-market shipments are accepted on a short-term basis without such commitments and typically yield higher market-driven rates.

3.1.2. Integrated Decisions and Container Sourcing

The integrated planning problem jointly determines (i) voyage-level acceptance and execution of laden container bookings and (ii) empty container repositioning over the planning horizon. For each voyage $v\in \mathcal{V}$, OD pair $(i,j)\in \mathcal{OD}$, and service class $s\in \mathcal{S}$, the carrier decides the accepted bookings $z_{ij}^{sv}$ and the corresponding executed shipments. We distinguish between shipments executed with carrier-owned equipment and those executed with externally leased equipment. Let $x_{ij}^{sv}$ denote the shipped laden volume (in TEU) of class s on OD pair $(i,j)$ in voyage v using owned containers. When local availability of owned empties is insufficient, the carrier may lease containers from external lessors. Let $r_{ij}^{sv}$ denote the shipped laden volume executed using leased containers. The unit leasing cost ${\rho }_{ij}^v$ is modeled as a daily rental rate multiplied by an expected on-hire duration, approximated by the OD transportation duration $t_{ij}$.

Due to spatial mismatches between where empty containers become available and where laden demand arises, the carrier may reposition empty containers between ports. Let $x_{ij}^{Ev}$ denote the repositioning flow (in TEU) of empty containers shipped from i to j in voyage v. At each port-call index, owned empty containers from three sources can be used to support outbound laden shipments or be sent as repositioned empties, including on-hand inventory, inbound repositioned empties, and empties generated by discharging owned laden containers. Accordingly, the remaining owned empty inventory after serving port-call index i in voyage v is denoted by $I_i^v$, which becomes the available stock for subsequent decisions. We assume leased containers are on-hired at the origin and off-hired upon arrival, so shipments executed with owned containers $x_{ij}^{sv}$ participate in container circulation and affect the evolution of owned empty inventories, whereas leased shipments $r_{ij}^{sv}$ do not.

Finally, accepted bookings may not be executed on the intended voyage due to slot or empty shortages. Such quantities are deferred to subsequent voyages. We use $h_{ij}^{sv}$ to track stranded bookings, which incur delay-type costs when carried over to later voyages, while any remaining stranded quantity at the end of the planning horizon is penalized as unserved demand.

3.1.3. Performance Measures

The above decisions are evaluated from both financial and operational perspectives. On the one hand, the carrier aims to maximize profit by allocating scarce slots to revenue-generating laden container shipments while accounting for transportation, leasing, inventory holding, and delay or shortage penalties. On the other hand, ECR consumes vessel capacity without generating freight revenue and can be substantial on cyclic services [17]. We therefore measure the operational burden of repositioning using a distance-weighted metric in empty TEU-miles, which captures the transport work performed for empty containers over all legs in the planning horizon. Accordingly, we formulate a bi-objective optimization model that maximizes total profit while minimizing empty TEU-miles, making the trade-off between profitability and ECR intensity explicit.

3.2. Model Assumptions

To keep the formulation tractable while preserving the key operational features of the integrated CSAP and ECR problem, we make the following assumptions:

• Fixed service network and sailing schedule. The shipping routes, service schedules, and vessel capacities over the planning horizon are given. The model optimizes how to use the announced capacity through laden acceptance, empty repositioning, and leasing decisions, rather than adjusting service frequency, speed, or port calls.
• Contract fulfillment requirement over the planning horizon. Contract demand is subject to an explicit fulfillment requirement over the planning horizon. We enforce a reliability-based service target by requiring that the cumulative shipped volume for each contract OD pair covers the cumulative contract demand with at least a prescribed fulfillment rate.
• Deferred fulfillment of unserved shipments with delay cost. When demand cannot be loaded on the intended sailing due to slot or empty shortages, it is stranded and may be served by subsequent sailings within the horizon. Each period of delay incurs a demurrage or holding penalty to reflect service deterioration.
• External leasing as an empty container sourcing option. If internal repositioning and local stocks are insufficient, the carrier may lease empties at origin ports. Leasing is priced on a per-day basis, and the expected on-hire duration is OD-dependent, reflecting heterogeneous transit and turnaround cycles. Leased containers are treated as exogenous supply and are not assumed to permanently expand the carrier-owned container storage.

3.3. Mathematical Formulation

In this subsection, we formulate a bi-objective mixed-integer model for the integrated CSAP and ECR problem over multiple voyages. The formulation jointly determines booking acceptance, shipment execution, and empty procurement/repositioning decisions under vessel capacity and inventory feasibility requirements.

3.3.1. Notations

The notations used throughout the formulation are summarized in

Table 2. List of notations.

Sets
$\mathcal{L}$	A set of port-call indices along the cyclic liner route, $\mathcal{L}=\{1,2,\dots ,L\}$, indexed by i (and j and k when needed);
$\mathcal{A}$	A set of directed shipping legs on the service network, indexed by a;
	On the cyclic route, legs are in one-to-one correspondence with origin port-call indices $i\in \mathcal{L}$, i.e., $a=a\left(i\right)=(i,i+1)$ and the wrap-around leg is $(L,1)$;
$\mathcal{V}$	A set of voyages in the planning horizon, $\mathcal{V}=\{1,2,\dots ,V\}$, indexed by v;
$\mathcal{OD}$	A set of origin–destination pairs on the liner service, indexed by $(i,j)$;
	OD pairs may satisfy either $i<j$ or $j<i$ to represent wrap-around demand on the cyclic route;
$\mathcal{S}$	A set of laden container service classes, indexed by s. Specifically, we set $\mathcal{S}=\{\mathrm{C}1,\mathrm{C}2\}$, corresponding to contract and spot demand, respectively;
Parameters
$Ca{p}^v$	Total slot capacity (in TEU) of the vessel operating voyage v;
$o\left(a\right)$	Origin port-call index of shipping leg $a\in \mathcal{A}$; on the cyclic route, $o\left(a\right(i\left)\right)=i$;
${\delta }_{ij}^a$	OD–leg incidence: ${\delta }_{ij}^a=1$ if leg a lies on the path from i to j on the cyclic route, and 0 otherwise;
$t_{ij}$	Transportation duration (in days) for OD pair $(i,j)\in \mathcal{OD};$
$d_a$	Sailing distance (in miles) of shipping leg $a\in \mathcal{A}$;
$d_{ij}$	Sailing distance (in miles) of OD pair $(i,j)$, with $d_{ij}={\sum }_{a\in \mathcal{A}}{\delta }_{ij}^a{d}_a$;
$q_{ij}^{sv}$	Demand (in TEU) of laden shipments for OD pair $(i,j)\in \mathcal{OD}$ with service class $s\in \mathcal{S}$ in voyage $v\in \mathcal{V}$;
${\beta }_{ij}$	Minimum service level (fill-rate) requirement for contract demand on OD pair $(i,j)$ over the planning horizon;
$f_{ij}^{sv}$	Freight rate (revenue) per TEU for OD pair $(i,j)\in \mathcal{OD}$ with service class $s\in \mathcal{S}$ in voyage $v\in \mathcal{V}$;
$c_{ij}^{Lv}$	Unit transportation cost per TEU for laden containers on OD pair $(i,j)$ in voyage v, assumed identical across demand classes $s\in \mathcal{S}$;
$c_{ij}^{Ev}$	Unit transportation cost per TEU for empty containers on OD pair $(i,j)$ in voyage v;
${\overline{\rho }}^v$	Daily rental rate per TEU-day for externally leasing empty containers in voyage v;
${\rho }_{ij}^v$	Unit leasing cost per TEU for OD pair $(i,j)$ in voyage v, where ${\rho }_{ij}^v={\overline{\rho }}^v{t}_{ij}$;
${\zeta }_i$	Unit inventory holding cost per TEU of empty containers remaining at port-call index i at the end of each voyage;
${\kappa }_{ij}^{sv}$	Delay cost per TEU for accepted laden demand of class s on OD pair $(i,j)$ that is postponed from voyage v to subsequent voyages;
${\pi }_{ij}^s$	Terminal penalty cost per TEU for accepted laden demand of class s on OD pair $(i,j)$ that remains unshipped by the end of the planning horizon;
$I_i^0$	Initial inventory (in TEU) of empty containers at port-call index i at the beginning of the planning horizon;
M	Number of vessels deployed on the liner service;
Variables
$z_{ij}^{sv}$	Accepted booking quantity (in TEU) for laden demand of class $s\in \mathcal{S}$ on OD pair $(i,j)$ in voyage v;
$x_{ij}^{sv}$	Shipped quantity (in TEU) of laden containers of class $s\in \mathcal{S}$ on OD pair $(i,j)$ in voyage v using carrier-owned containers;
$x_{ij}^{Ev}$	Repositioned empty container flow (in TEU) shipped from i to j in voyage v;
$r_{ij}^{sv}$	Shipped laden quantity (in TEU) of class $s\in \mathcal{S}$ on OD pair $(i,j)$ in voyage v executed using leased containers;
Auxiliary variables
$I_i^v$	Remaining empty container inventory (in TEU) at port-call index i immediately after departure in voyage v;
$y_a^v$	Total onboard volume (in TEU) on shipping leg $a\in \mathcal{A}$ during voyage v;
$h_{ij}^{sv}$	Stranded laden bookings (in TEU) of class $s\in \mathcal{S}$ on OD pair $(i,j)$ after voyage v, which are deferred to subsequent voyages and incur a demurrage-type postponement penalty;

.

We introduced a set of auxiliary variables here, denoted as $y_a^v$, representing for the total onboard volume (in TEU) on shipping leg $a\in \mathcal{A}$ during voyage $v\in \mathcal{V}$. On a cyclic route, an OD flow occupies slots on every leg along its transportation path. In our setting, laden shipments may be carried either by owned containers ($x_{jk}^{sv}$) or by leased containers ($r_{jk}^{sv}$). Although leased containers are off-hired upon arrival and thus do not contribute to the carrier’s owned empty-inventory evolution, they occupy vessel slots during transportation and therefore must be counted toward onboard slot occupancy. In addition, empty repositioning flows $x_{jk}^{Ev}$ also consume slots along the legs they traverse.

Voyages are indexed in chronological order by their departures from the origin port. We assume a regular cyclic deployment with M vessels on the service. Accordingly, a given vessel operates the voyage set $\{v+kM\mid k=0,1,2,\dots \}$ within the planning horizon. This deployment implies that, for wrap-around OD pairs, the onboard volume observed on an early leg of voyage v may include containers that were loaded in the previous voyage operated by the same vessel, namely voyage $v-M$. To capture this effect, we use $o\left(a\right)$ to denote the origin (tail) port-call index of leg a. For an OD pair $(j,k)$ traversing leg a, if the loading index j has already been visited within the current voyage before reaching leg a (i.e., $j\le o\left(a\right)$), the onboard contribution comes from voyage v; otherwise (i.e., $j>o\left(a\right)$), the onboard contribution comes from voyage $v-M$.

For the first M voyages, we assume no pre-existing onboard containers from prior operations at the beginning of the planning horizon. Hence,

$$y_a^v=\sum _{\begin{array}{c}(j,k)\in \mathcal{OD}:\\ j\le o\left(a\right)\end{array}}{\delta }_{jk}^a\left(\sum _{s\in S}\left(x_{jk}^{sv}+r_{jk}^{sv}\right)+x_{jk}^{Ev}\right),\forall a\in \mathcal{A},\forall v\in \{1,\dots ,M\}.$$

(1)

For subsequent voyages, the onboard volume on leg a in voyage v consists of two parts: (i) containers loaded in voyage v whose loading indices satisfy $j\le o\left(a\right)$, and (ii) wrap-around containers loaded in voyage $v-M$ whose loading indices satisfy $j>o\left(a\right)$:

$$\begin{array}{c}\hfill y_a^v=\sum _{\begin{array}{c}(j,k)\in \mathcal{OD}:\\ j\le o\left(a\right)\end{array}}{\delta }_{jk}^a\left(\sum _{s\in \mathcal{S}}\left(x_{jk}^{sv}+r_{jk}^{sv}\right)+x_{jk}^{Ev}\right)+\sum _{\begin{array}{c}(j,k)\in \mathcal{OD}:\\ j>o\left(a\right)\end{array}}{\delta }_{jk}^a\left(\sum _{s\in \mathcal{S}}\left(x_{jk}^{s(v-M)}+r_{jk}^{s(v-M)}\right)+x_{jk}^{E(v-M)}\right),\\ \hfill \forall a\in \mathcal{A},\forall v\in \{M+1,\dots ,V\}.\end{array}$$

(2)

To explain the cross-voyage accounting in (1) and (2), Figure 1 illustrates three representative cases on a cyclic route and clarifies why splitting OD pairs by $j\le o\left(a\right)$ and $j>o\left(a\right)$ is sufficient to determine whether the onboard contribution on leg a in voyage v comes from variables of voyage v or from the previous cycle $v-M$. Panels (a) and (c) correspond to part (i) in (2): leg a is traversed after the vessel visits the loading port j within voyage v, so the containers occupying leg a are those loaded in voyage v. This situation is captured by $j\le o\left(a\right)$ and is therefore accounted for by the first summation term. Only in the wrap-around case in panel (b) ($o\left(a\right)<k<j$) is leg a traversed before visiting j in voyage v; in this case, the containers occupying leg a in voyage v must have been loaded in voyage $v-M$ by the same vessel. This situation is captured by $j>o\left(a\right)$ and is therefore accounted for by the second summation term in (2). Accordingly, (1) and (2) provide a compact onboard-accounting equality under regular cyclic deployment, compared with the more general formulation in Wang et al. [7].

3.3.2. Objective Functions

The proposed problem is inherently multi-objective. For clarity and focus, we consider two objectives in this study. On the one hand, the carrier aims to maximize profit by allocating limited vessel slots to revenue-generating laden shipments, while accounting for transportation costs of both laden and empty moves, external leasing or procurement costs when owned empty containers are insufficient, inventory holding costs for residual empties at ports, and penalty costs associated with deferred and ultimately unshipped accepted bookings. On the other hand, since empty repositioning consumes slot capacity and transportation effort without directly generating freight revenue, the carrier also seeks to limit unnecessary empty movements. We therefore minimize the total empty TEU-miles over the planning horizon. These two objectives are generally conflicting, leading to a set of Pareto-optimal solutions that reveal the trade-off between profit and empty repositioning effort [32]. The two objective functions are defined as follows.

• Objective 1: Profit maximization.

The profit objective equals the freight revenue generated by shipped laden containers minus the container transportation costs, empty container repositioning costs, external leasing costs, inventory holding costs, and penalty costs arising from postponed and ultimately unshipped accepted bookings:

$$\begin{array}{cc}\hfill max{Z}_1=& \sum _{v\in \mathcal{V}}\sum _{(i,j)\in \mathcal{OD}}\sum _{s\in \mathcal{S}}{f}_{ij}^{sv}(x_{ij}^{sv}+r_{ij}^{sv})-\sum _{v\in \mathcal{V}}\sum _{(i,j)\in \mathcal{OD}}\sum _{s\in \mathcal{S}}{c}_{ij}^{Lv}(x_{ij}^{sv}+r_{ij}^{sv})-\sum _{v\in \mathcal{V}}\sum _{(i,j)\in \mathcal{OD}}{c}_{ij}^{Ev}{x}_{ij}^{Ev}\hfill \\ \hfill & -\sum _{v\in \mathcal{V}}\sum _{(i,j)\in \mathcal{OD}}\sum _{s\in \mathcal{S}}{\rho }_{ij}^v{r}_{ij}^{sv}-\sum _{v\in \mathcal{V}}\sum _{i\in \mathcal{L}}{\zeta }_i{I}_i^v-\sum _{v\in \mathcal{V}\setminus \left\{V\right\}}\sum _{(i,j)\in \mathcal{OD}}\sum _{s\in \mathcal{S}}{\kappa }_{ij}^{sv}{h}_{ij}^{sv}\hfill \\ \hfill & -\sum _{(i,j)\in \mathcal{OD}}\sum _{s\in \mathcal{S}}{\pi }_{ij}^s{h}_{ij}^{sV}.\hfill \end{array}$$

(3)

For $v\in \mathcal{V}\setminus \left\{V\right\}$, stranded bookings $h_{ij}^{sv}$ are carried over to subsequent voyages and incur a postponement cost ${\kappa }_{ij}^{sv}$, capturing demurrage-type charges or service-recovery expenses due to delayed fulfillment. At the end of the planning horizon, any remaining stranded bookings $h_{ij}^{sV}$ cannot be postponed further and therefore incur a terminal penalty ${\pi }_{ij}^s$, representing contractual compensation, loss of goodwill, or the cost of outsourcing transportation to alternative carriers.

• Objective 2: Minimization of empty TEU-miles.

To limit unnecessary empty repositioning effort, we minimize the total transport work induced by repositioned empty containers over the planning horizon. Accordingly, the second objective is defined as

$$\begin{array}{c}\hfill min{Z}_2=\sum _{v\in \mathcal{V}}\sum _{a\in \mathcal{A}}\sum _{(j,k)\in \mathcal{OD}}{\delta }_{jk}^a{d}_a{x}_{jk}^{Ev}.\end{array}$$

(4)

3.3.3. Model Constraints

The above objectives are optimized subject to operational constraints on (i) booking acceptance and contract fulfillment, (ii) inter-voyage shipment execution and postponement handling, (iii) vessel-slot capacity on each shipping leg, and (iv) owned empty-inventory dynamics across voyages. Capacity is enforced on every leg along the cyclic route, reflecting that each OD flow occupies slots on all intermediate legs until discharge. External leasing provides an additional equipment-sourcing option when local owned empty availability is insufficient. The complete bi-objective model is given as follows.

$$\begin{array}{cc}\hfill max& Z_1\hfill \\ \hfill min& Z_2\hfill \end{array}$$

$$\begin{array}{cccc}\hfill s.t.& z_{ij}^{sv}\le q_{ij}^{sv}\hfill & \hfill & \forall (i,j)\in \mathcal{OD},\forall s\in \mathcal{S},\forall v\in \mathcal{V}.\hfill \end{array}$$

(5)

$$\begin{array}{cccc}\hfill & \sum _{v\in \mathcal{V}}\left(x_{ij}^{sv}+r_{ij}^{sv}\right)\ge {\beta }_{ij}\sum _{v\in \mathcal{V}}{q}_{ij}^{sv},\hfill & \hfill & \forall (i,j)\in \mathcal{OD},s=C2\hfill \end{array}$$

(6)

$$\begin{array}{cccc}\hfill & h_{ij}^{s0}=0,\hfill & \hfill & \forall (i,j)\in \mathcal{OD},\forall s\in \mathcal{S}\hfill \end{array}$$

(7)

(1) and (2),

$$\begin{array}{cccc}\hfill & h_{ij}^{s(v-1)}+z_{ij}^{sv}=x_{ij}^{sv}+r_{ij}^{sv}+h_{ij}^{sv},\hfill & \hfill & \forall (i,j)\in \mathcal{OD},\forall s\in \mathcal{S},\forall v\in \mathcal{V}\hfill \\ \hfill & \left(\right)--\left(\right),\hfill \end{array}$$

(8)

$$\begin{array}{cccc}\hfill & y_a^v\le Ca{p}^v,\hfill & \hfill & \forall a\in \mathcal{A},\forall v\in \mathcal{V}\hfill \\ \hfill & I_i^v=I_i^{v-1}+\sum _{j:(j,i)\in \mathcal{OD}}\sum _{s\in \mathcal{S}}{x}_{ji}^{sv}+\sum _{j:(j,i)\in \mathcal{OD}}{x}_{ji}^{Ev},\hfill \end{array}$$

(9)

$$\begin{array}{cccc}\hfill & -\sum _{k:(i,k)\in \mathcal{OD}}\sum _{s\in \mathcal{S}}{x}_{ik}^{sv}-\sum _{k:(i,k)\in \mathcal{OD}}{x}_{ik}^{Ev},\hfill & \hfill & \forall i\in \mathcal{L},\forall v\in \mathcal{V}\hfill \end{array}$$

(10)

$$\begin{array}{cccc}\hfill & z_{ij}^{sv},x_{ij}^{sv},x_{ij}^{Ev},r_{ij}^{sv},h_{ij}^{sv}\in \mathbb{Z},\hfill & \hfill & \forall (i,j)\in \mathcal{OD},\forall s\in \mathcal{S},\forall v\in \mathcal{V}\hfill \end{array}$$

(11)

$$\begin{array}{cccc}\hfill & I_i^v\ge 0,\hfill & \hfill & \forall i\in \mathcal{L},\forall v\in \mathcal{V}\hfill \end{array}$$

(12)

$$\begin{array}{cccc}\hfill & y_a^v\ge 0,\hfill & \hfill & \forall a\in \mathcal{A},\forall v\in \mathcal{V}\hfill \end{array}$$

(13)

Constraint (5) limits booking acceptance. For each voyage v, OD pair $(i,j)$, and service class s, the accepted bookings $z_{ij}^{sv}$ cannot exceed the corresponding demand $q_{ij}^{sv}$.

Constraint (6) enforces the minimum contract-fulfillment requirement over the planning horizon. It requires that the cumulative executed volume of contract shipments on each OD pair, including both owned-container and leased-container executions, reaches at least a prescribed fraction ${\beta }_{ij}$ of the planned contract demand.

Constraints (7) and (8) govern shipment execution and the carryover of unshipped bookings across voyages. We initialize stranded bookings by (7). In each voyage v, the total amount to be served consists of newly accepted bookings $z_{ij}^{sv}$ and the previously stranded amount $h_{ij}^{s(v-1)}$. Constraint (8) balances these quantities by splitting them into executed shipments using owned containers $x_{ij}^{sv}$, executed shipments using leased containers $r_{ij}^{sv}$, and the remaining stranded amount $h_{ij}^{sv}$ carried forward. These stranded quantities are penalized in the profit objective through postponement costs for $v<V$ and terminal penalties at $v=V$.

Constraints (1) and (2) together with (9) enforce vessel capacity on each shipping leg. The onboard volume $y_a^v$ aggregates all laden and empty flows traversing leg a via the OD–leg incidence parameter ${\delta }_{ij}^a$, including owned-container shipments $x_{ij}^{sv}$, leased-container shipments $r_{ij}^{sv}$, and repositioned empties $x_{ij}^{Ev}$. Constraint (9) then limits $y_a^v$ by the slot capacity $Ca{p}^v$ on every leg.

Constraint (10) describes the evolution of owned empty inventories across voyages. The state variable $I_i^v$ is updated by adding owned empties generated by discharging owned-container shipments and inbound repositioned empties from the previous voyage, and subtracting owned empties consumed by loading owned-container shipments and by outbound repositioning departures in voyage v. Leased containers are assumed to be on-hired at the origin and off-hired at the destination. Therefore, the leased-shipment decision $r_{ij}^{sv}$ occupies vessel slots during transportation but does not enter the owned empty-inventory balance.

Finally, constraints (11)–(13) specify variable domains.

4. Solution Algorithm

This section presents a hybrid solution framework that integrates a multi-objective evolutionary algorithm with reinforcement learning to solve the integrated slot allocation and empty container repositioning problem. The proposed approach is designed to handle the high-dimensional decision space, nonlinear interactions between laden and empty flows, and tight vessel slot-capacity constraints via repair and penalty mechanisms.

4.1. Overall Framework

We develop a hybrid learning-augmented heuristic that integrates NSGA-II with a lightweight Q-learning controller, named NSGA-II-RL, to solve the integrated slot allocation and empty container repositioning problem, balancing profit and repositioning effort. The method combines evolutionary search for Pareto approximation with online learning for adaptive operator control, and the overall procedure is summarized in Algorithm 1.   

Algorithm 1: NSGA-II-RL

The problem involves a high-dimensional decision space and strong nonlinear coupling between laden container acceptance, empty container repositioning, and voyage-leg capacity usage. The resulting trade-offs between economic performance and repositioning effort are difficult to reflect with a pre-specified scalarization and can exhibit multiple locally efficient patterns. NSGA-II is therefore adopted as the main search engine because it can efficiently approximate a diverse set of non-dominated solutions in multi-objective settings while maintaining frontier diversity through elitist selection and crowding-based preservation.

Although NSGA-II provides a robust baseline, its performance is sensitive to operator configurations, especially under a discrete decode–repair–evaluate routine. Early generations often benefit from stronger exploration, whereas later generations require more conservative variation for refinement. In addition, feasibility is challenging due to capacity coupling across voyage legs. If repair is too aggressive, diversity may collapse; if the repair is too weak, infeasible offspring may dominate and slow down progress. We therefore introduce an RL controller to adapt mutation intensity and repair behavior online based on population-level feedback, rather than relying on fixed parameters.

The hybrid method can be viewed as a two-level procedure. At the population level, NSGA-II iteratively generates offspring and updates the population through selection and elitist replacement. At the individual level, every offspring must pass through a fixed processing routine before it can be compared and selected. This routine links the real-coded search space with discrete operational decisions and feasibility handling, and it forms the main interface where evolutionary operators and problem constraints interact.

Each candidate solution is represented as a real-coded chromosome. For evaluation, the chromosome is decoded into integer voyage-level decisions for laden acceptance and empty repositioning. The decoded solution may then be repaired to mitigate capacity overloads on sailing legs; remaining overloads are handled through a penalty term during evaluation. After repair, the phenotype is written back to the chromosome as a Lamarckian update, so that subsequent genetic operators act on an already feasible or near-feasible representation. Finally, the repaired solution is evaluated by sequential inventory propagation along the port-call sequence, where owned empty containers are allocated to repositioning first, and remaining inventory is used to serve laden shipments with leasing covering any unmet laden demand.

The RL controller operates at the generation level and only controls operator and repair settings, while the decode–repair–evaluate routine described above remains unchanged. At the beginning of each generation, the agent observes a compact state that summarizes search progress, population diversity, and feasibility status of the current population. It then selects an action that determines the operator configuration for the current generation, including mutation intensity and repair behavior used during offspring creation and feasibility handling. After NSGA-II completes offspring evaluation and elitist environmental selection, the agent receives a reward based on hypervolume improvement together with an additional penalty when the population feasibility rate falls below a target level, and updates its action value table for subsequent generations.

4.2. NSGA-II-Based Evolutionary Search

We adopt NSGA-II as the main evolutionary engine to approximate the trade-off frontier between the two objectives. Each solution is encoded as a real-coded chromosome that concatenates the voyage-level laden allocation variables and the empty repositioning variables. During evaluation, chromosomes are decoded into integer decisions by rounding, and the laden part is further capped by demand to avoid systematic overflow caused by rounding. This design allows standard genetic operators to work in a continuous search space, while the decoded phenotype remains consistent with discrete operational decisions. Let $n_{\mathrm{var}}$ denote the chromosome length. The procedure Evaluate$(·)$ in Algorithm 1 implements this fixed decode–repair–evaluate routine and returns objective values together with the capacity-violation metric.

As formulated in Section 3, let $Z_1$ denote the total profit and $Z_2$ denote the empty TEU-miles. Since NSGA-II is implemented in a minimization form, we use the following objective vector for evolutionary search:

$$\begin{array}{cc}\hfill & f_1=-Z_1+{\lambda }_{\mathrm{cap}}\mathrm{CapViol},\hfill \end{array}$$

(14)

$$\begin{array}{cc}\hfill & f_2=Z_2,\hfill \end{array}$$

(15)

where $\mathrm{CapViol}$ is the aggregated vessel slot capacity violation over all sailing legs, and ${\lambda }_{\mathrm{cap}}$ is the penalty coefficient.

The initial population is generated by a mixed strategy that balances exploration and informed seeding. Most individuals are sampled uniformly within variable bounds. A subset is generated by a greedy heuristic biased toward moving empties to OD pairs associated with higher leasing costs. Another subset sets all empty-repositioning genes to zero to provide a profit-oriented baseline. The population is then shuffled to avoid positional bias before evolution starts.

At each generation, parent solutions are selected using binary tournament selection based on Pareto rank and crowding distance. Selected parents undergo simulated binary crossover with crossover probability $p_{\mathrm{cx}}$ and distribution index ${\eta }_c$, followed by polynomial mutation with mutation probability $p_{\mathrm{mut}}$ and distribution index ${\eta }_m$. When not specified, the default mutation probability is set to $p_{\mathrm{mut}}=1/n_{\mathrm{var}}$. All operators are applied in the real-coded chromosome space, and offspring genes are clipped to their bounds to preserve feasibility of the encoding. The procedure Variation$(·)$ consists of binary tournament selection, SBX crossover, and polynomial mutation under the parameter set ${\mathrm{\Theta }}_g$.

4.2.1. Offspring Evaluation and Feasibility Handling

Each offspring chromosome is processed by a fixed decode–repair–evaluate routine before it can participate in non-dominated sorting. First, genes are decoded and rounded to obtain integer voyage-level decisions, and the laden decisions are capped by the corresponding demands to eliminate rounding-induced overflow. The resulting decisions are then evaluated through sequential inventory propagation along the port-call sequence, where owned empty containers are allocated to repositioning first, and remaining inventory is used to serve laden shipments. Any unmet laden demand is covered by leasing, so infeasibility due to insufficient inventories is avoided by construction.

Vessel slot-capacity feasibility on each leg is handled by a dedicated repair operator that scales down flows on overloaded legs under different prioritization modes, including laden-first, empty-first, and balanced. The repair is applied before evaluation, possibly under a prescribed probability in the NSGA-II-RL variant. If overloads remain after repair, they are quantified by $\mathrm{CapViol}$ and penalized in $f_1$ via the coefficient ${\lambda }_{\mathrm{cap}}$. After repair, the repaired phenotype is written back into the chromosome as a Lamarckian update, so that subsequent crossover and mutation operate on an already feasible or near-feasible genotype.

4.2.2. Environmental Selection

NSGA-II adopts elitist environmental selection to form the next generation. Parent and offspring populations are merged and ranked by fast non-dominated sorting into Pareto fronts. Fronts are inserted into the next generation in ascending order of rank until the population limit is reached. If the last admissible front cannot be fully included, individuals in that front are selected in descending order of crowding distance to preserve diversity along the frontier. The algorithm outputs the first non-dominated front of the final population in Algorithm 1.

In the NSGA-II-RL variant, the evolutionary loop described above remains unchanged. The Q-learning controller only adjusts operator and repair settings across generations, while the decode–repair–evaluate routine stays the same. The state design, action-to-parameter mapping, and reward definition are detailed in the next subsection.

4.3. Reinforcement Learning Assisted Operator Control

NSGA-II provides a strong baseline for multi-objective search, yet its performance is sensitive to operator configurations at different stages of evolution. Fixed variation settings may lead to insufficient exploration in early generations or slow refinement in later generations, especially under the discrete decode–repair–evaluate routine. We therefore introduce a lightweight Q-learning controller to adapt mutation intensity and repair behavior online based on population-level feedback, while keeping the decoding, repair, and evaluation pipeline unchanged.

4.3.1. MDP Formulation and Controlled Parameters

The interaction between NSGA-II and the RL controller is formulated as a generation-level Markov decision process. At generation g, the environment is characterized by the current population $P_g$ and summary indicators of its diversity and feasibility. The agent selects an action $a_g$, which is mapped to a generation-specific operator configuration

$$\begin{array}{c}\hfill {\mathrm{\Theta }}_g=\left(p_{\mathrm{mut}}^{\left(g\right)},{\eta }_c^{\left(g\right)},{\eta }_m^{\left(g\right)},p_{\mathrm{rep}}^{\left(g\right)},{\chi }_{\mathrm{rep}}^{\left(g\right)}\right).\end{array}$$

(16)

Here $p_{\mathrm{mut}}^{\left(g\right)}$ is the mutation probability; ${\eta }_c^{\left(g\right)}$ and ${\eta }_m^{\left(g\right)}$ are the distribution indices of SBX crossover and polynomial mutation; $p_{\mathrm{rep}}^{\left(g\right)}$ is the probability of applying the capacity repair; and ${\chi }_{\mathrm{rep}}^{\left(g\right)}$ specifies the repair mode. The agent does not modify decision variables directly. Instead, it steers the search dynamics by adjusting variation and feasibility-handling behaviors across generations. The baseline parameter set ${\mathrm{\Theta }}^0$ in Algorithm 1 provides default values for these controlled parameters when actions do not overwrite them.

4.3.2. State Representation

We use a compact discrete state to summarize search progress, population diversity, and feasibility. Progress is categorized into three stages based on the ratio $g/G$. Diversity is measured by the average finite crowding distance of the current population, and feasibility is measured by the fraction of solutions whose total constraint violation is below a tolerance. In this study, the violation metric mainly reflects voyage-leg capacity overloads, since inventory-related shortfalls are resolved by truncation of empty repositioning and leasing for laden demand during evaluation. All thresholds and tolerances used for state discretization are user-specified constants. The resulting state space contains $3\times 2\times 2=12$ states and is indexed by

$$\begin{array}{c}\hfill s_g=4p_g+2d_g+b_g,\end{array}$$

(17)

where $p_g\in \{0,1,2\}$ is the progress stage, $d_g\in \{0,1\}$ indicates whether diversity exceeds a threshold, and $b_g\in \{0,1\}$ indicates whether the feasibility rate exceeds a target level.

4.3.3. Action Design and Parameter Mapping

The agent selects one of three actions representing exploration, balance, and exploitation. Each action corresponds to a predefined mapping from $a_g$ to ${\mathrm{\Theta }}_g$. This mapping implements MapAction$(·)$ in Algorithm 1. The exploration action increases variation intensity and applies repair with moderate probability to avoid population collapse. The balanced action uses baseline variation settings and higher repair probability to stabilize feasibility. The exploitation action reduces variation intensity for refinement and applies repair more aggressively with the laden-first mode.

4.3.4. Reward and Q-Learning Update

After offspring evaluation and elitist environmental selection, the agent receives a scalar reward based on population-level improvement. This reward computation corresponds to Reward$(·)$ in Algorithm 1. We employ the hypervolume improvement of the first non-dominated front as the primary signal and add an extra penalty when the population feasibility rate falls below a target level. Hypervolume is computed using a fixed reference point and consistent normalization across generations. The Q-table is updated online using the standard Q-learning rule with an $ϵ$-greedy behavior policy, where $ϵ$ is the exploration rate, $\alpha$ is the learning rate, and $\gamma$ is the discount factor. The learned Q-table can be stored and reused to warm-start subsequent runs.

5. Numerical Experiments

5.1. Data Description

We use the standardized benchmark dataset LINERLIB proposed by Brouer et al. [41] for the Liner Shipping Network Design Problem. The dataset provides real-world instances with port sets, inter-port distances, fleet information, and an origin–destination demand matrix. Since our study focuses on the downstream operational stage given a fixed service network, we do not use the original design instances directly. Instead, we randomly select five services the published solution of Brouer et al. [41]. Each service is operated as a weekly string. Let $T_r$ denote the round-trip cycle time of route r measured in weeks. The vessel capacity on each route is taken from the deployed vessel class in the corresponding solution and is reported in TEU.

Table 3. Selected routes and service attributes.

Route ID	Port Calls	Cycle Time (Weeks)	Vessels	Capacity (TEU)
1	Tanjung Pelepas–Port Klang–Port Said–Salalah–Tanjung Pelepas–Singapore–Hong Kong–Busan–Qingdao–Los Angeles–Shanghai–Kaohsiung	13	13	9600
2	Vancouver–Brisbane–Auckland–Brisbane–Port Klang–Tanjung Pelepas–Kaohsiung	8	8	9600
3	Algeciras–Tangier–Port Said–Jeddah–Gioia Tauro–Jeddah–Newark–Charleston–Felixstowe–Bremerhaven	10	10	9600
4	Busan–Vancouver–Los Angeles–Yokohama–Tanjung Pelepas–Colombo–Tanjung Pelepas–Hong Kong–Shenzhen	9	9	9600
5	Newark–Los Angeles–Balboa–Manzanillo–Santos–Apapa–Algeciras–Gioia Tauro–Algeciras–Tangier	14	14	9600

summarizes the selected routes and their key attributes.

The original LINERLIB solution provides a representative weekly OD demand matrix and the inter-port distances. To construct a multi-week planning horizon, we generate weekly demand scenarios following Wang et al. [7]. Specifically, for each OD pair, the realized weekly demand is sampled around its nominal value with a coefficient of variation $\mathrm{CV}=\sigma /\mu =0.1$. For each OD pair and each week, the realized demand is split into two parts, with $50\%$ treated as contract demand and the remaining $50\%$ treated as spot demand.

The revenue and cost parameters are calibrated based on Wang et al. [7] and adjusted to match the scale of the selected services. The resulting parameter values are summarized in

Table 4. Parameter settings used in numerical experiments.

Parameter	Value/Range
$f_{ij}^{sv}/d_{ij}(s=\mathrm{C}1)$	$0.70$
$f_{ij}^{sv}/d_{ij}(s=\mathrm{C}2)$	$0.50$
${\beta }_{ij}$	$[0.70,0.95]$
$c_{ij}^{Lv}/f_{ij}^{sv}$	$[0.40,0.50]$
$c_{ij}^{Ev}/c_{ij}^{Lv}$	$[0.40,0.50]$
${\overline{\rho }}^v$	$[50,60]$
${\kappa }_{ij}^{sv}/f_{ij}^{sv}$	$[0.10,0.15]$
${\pi }_{ij}^s/f_{ij}^{sv}$	$[2.0,5.0]$
$I_i^0(i=1)$	$[5000,6000]$
$I_i^0(i\ne 1)$	$[200,500]$
${\zeta }_i$	$[50,60]$

. Note that we do not assume exogenous empty container demand. Empty repositioning is modeled as a derived operational decision driven by imbalanced laden flows and the availability of empty inventories at ports.

5.2. Experimental Setup and Evaluation Protocol

We evaluate the proposed NSGA-II-RL against two reference methods. The first is a baseline NSGA-II where the Q-learning controller is disabled and the operator configuration remains fixed throughout the run (Algorithm A1). The second is a random-control variant that samples an action uniformly at random at each generation and applies the corresponding operator and repair settings, while disabling Q-learning updates. All methods share the same solution encoding, decode–repair–evaluate routine (Evaluate), feasibility repair mechanism, objective evaluation procedure, population size N, and stopping criterion (generations G) to ensure a fair comparison.

Solution quality is measured by the hypervolume (HV) indicator computed from the non-dominated set returned by each run. We report the mean and standard deviation of the final-generation HV over independent runs, together with the best HV observed. To characterize feasibility and search stability, we track the feasibility rate, defined as the fraction of solutions whose total constraint violation is below a tolerance ${ϵ}_{\mathrm{cv}}$, and report its trajectory across generations. Runtime is recorded under the same implementation environment for all methods. Hypervolume is computed on normalized objectives using fixed bounds ${\mathbf{f}}^{min}$ and ${\mathbf{f}}^{max}$ and a fixed reference point $\mathbf{r}$. The same normalization bounds and reference point are used for all methods and all runs on the same instance. Detailed hyperparameter values, including normalization bounds and the reference point, are provided in Appendix B.

The Q-learning controller operates at the generation level. It observes a compact discrete state derived from progress stage, diversity level, and feasibility level. Progress is categorized into three stages based on the ratio $g/G$. Diversity is measured by the average crowding distance of the current population and compared with a threshold ${\tau }_{CD}$. Feasibility is measured by the feasibility rate ${\varphi }^g$ and compared with a target level ${\tau }_{\varphi }$, where feasibility is judged using tolerance ${ϵ}_{\mathrm{cv}}$ on total constraint violation. The resulting state space contains $3\times 2\times 2=12$ discrete states. The action set contains three actions corresponding to exploration, balance, and exploitation. Each action maps to a predefined operator and repair configuration Θ_g = Mapaction (a_g, Θ⁰) , which determines the settings used by Variation$(·)$ and Evaluate$(·)$. The reward is based on hypervolume improvement with an additional penalty when the population feasibility rate falls below the target. The controller adopts an $ϵ$-greedy behavior policy and updates action values using learning rate $\alpha$ and discount factor $\gamma$. The Q-table is initialized as zeros and can be warm-started by loading a previously saved table.

5.3. Computational Results

We report results on the five routes presented in Table 3. For each route, we generate three instance sizes by scaling the planning horizon length as $V_{r,k}=k·n_r$ with $k\in \{2,5,10\}$, where $n_r$ is the cycle length in weeks under weekly departures. For each route–size pair, each method is independently executed for 10 runs with different random seeds.

5.3.1. Overall Comparison on Solution Quality

Table 5. Overall comparison on solution quality, runtime, and feasibility.

Route	Size	Voyages	Method	HV Mean	HV Std	Best HV	Runtime Mean(s)	Feasibility Final
1	S	26	NSGA-II-RL	0.8375	0.0021	0.8401	85.72	1
			Baseline	0.8363	0.0015	0.8389	86.88	1
			Random	0.8340	0.0015	0.8363	87.75	0.970
1	M	65	NSGA-II-RL	0.8285	0.0008	0.8298	181.10	1
			Baseline	0.8270	0.0009	0.8285	185.25	1
			Random	0.8248	0.0016	0.8279	183.26	0.9218
1	L	130	NSGA-II-RL	0.8236	0.0017	0.8262	331.62	1
			Baseline	0.8204	0.0007	0.8217	335.63	1
			Random	0.8188	0.0006	0.8195	333.37	0.944
2	S	16	NSGA-II-RL	0.8518	0.0012	0.8529	34.39	1
			Baseline	0.8502	0.0017	0.8525	35.58	1
			Random	0.8483	0.0016	0.8498	34.89	0.978
2	M	40	NSGA-II-RL	0.8431	0.0006	0.8445	67.82	1
			Baseline	0.8420	0.0010	0.8434	70.68	1
			Random	0.8400	0.0007	0.8413	69.95	0.9691
2	L	80	NSGA-II-RL	0.8374	0.0006	0.8386	140.49	1
			Baseline	0.8369	0.0006	0.8378	144.76	1
			Random	0.8357	0.0007	0.8369	143.19	0.966
3	S	20	NSGA-II-RL	0.8466	0.0010	0.8478	38.30	1
			Baseline	0.8460	0.0008	0.8474	37.99	1
			Random	0.8446	0.0007	0.8457	38.01	0.964
3	M	50	NSGA-II-RL	0.8391	0.0011	0.8414	95.49	1
			Baseline	0.8376	0.0006	0.8385	97.28	1
			Random	0.8352	0.0016	0.8375	97.11	0.938
3	L	100	NSGA-II-RL	0.8344	0.0011	0.8361	176.20	1
			Baseline	0.8322	0.0009	0.8337	177.33	1
			Random	0.8309	0.0008	0.8323	178.32	0.962
4	S	18	NSGA-II-RL	0.8423	0.0019	0.8453	51.34	1
			Baseline	0.8422	0.0023	0.8459	52.78	1
			Random	0.8391	0.0018	0.8421	54.09	0.968
4	M	45	NSGA-II-RL	0.8322	0.0012	0.8352	106.19	1
			Baseline	0.8300	0.0012	0.8318	108.66	1
			Random	0.8274	0.0017	0.8300	104.13	0.966
4	L	90	NSGA-II-RL	0.8248	0.0009	0.8266	212.48	1
			Baseline	0.8235	0.0007	0.8248	217.32	1
			Random	0.8218	0.0011	0.8229	217.01	0.948
5	S	28	NSGA-II-RL	0.8450	0.0005	0.8457	52.36	1
			Baseline	0.8448	0.0009	0.8465	53.18	1
			Random	0.8437	0.0008	0.8448	53.40	0.938
5	M	70	NSGA-II-RL	0.8341	0.0009	0.8355	113.52	1
			Baseline	0.8336	0.0010	0.8347	114.87	1
			Random	0.8316	0.0010	0.8335	114.77	0.922
5	L	140	NSGA-II-RL	0.8257	0.0007	0.8272	159.51	1
			Baseline	0.8247	0.0007	0.8260	156.20	1
			Random	0.8245	0.0012	0.8263	157.76	0.938

Notes: (1) S, M, and L denote $k\in \{2,5,10\}$, respectively. (2) Within each (Route, Size) block, bold numbers indicate the best-performing method for HV mean and best HV (higher is better) and for runtime mean (lower is better); bold “1” indicates full final feasibility.

reports the overall performance across the 5 selected routes and three horizon scales. Several consistent patterns can be observed. All statistics are computed over 10 independent runs per route–size case.

First, NSGA-II-RL achieves the highest mean hypervolume in all route–size cases. Across the 15 route–size pairs, the average improvement in HV mean over the Baseline is $0.00125$. The gain increases with the planning-horizon scale, with average improvements of $0.00074$ for the short cases, $0.00136$ for the medium cases, and $0.00164$ for the long cases. Although the absolute differences are numerically small, the improvement is consistent and becomes more pronounced when the horizon is sufficiently long, where the controller can accumulate feedback and stabilize its action selection.

Second, feasibility behavior clearly separates the three methods. Both NSGA-II-RL and the Baseline achieve a final feasibility rate of $1.0$ on all cases, meaning every run terminates with fully feasible solutions under the adopted tolerance. In contrast, the random-control variant shows systematically lower final feasibility rates, with an average of about $0.953$ across cases. This is consistent with the role of operator control in our framework, where mismatched operators can slow down feasibility recovery and leave part of the population infeasible at termination.

Finally, runtime is comparable across methods. Runtime increases with the number of voyages as expected, while the differences between NSGA-II-RL and the Baseline are small. In most cases, NSGA-II-RL incurs no noticeable computational overhead despite the additional Q-learning update.

5.3.2. Convergence Behavior and Stability

To reveal how the performance differences arise, we examine the generation-wise evolution of hypervolume (HV) and feasibility rate. Figure 2 provides a consolidated view across Routes 1–5 and three horizon scales $k\in \{2,5,10\}$, where rows correspond to routes and columns correspond to horizon scales. Each panel is a dual-axis plot, with HV on the left axis and feasibility rate on the right axis. Curves show the mean trajectory across runs and the shaded band indicates one standard deviation.

Overall, NSGA-II-RL attains higher HV than the two reference methods across the tested settings. The random-control variant exhibits lower HV and larger fluctuations, and its feasibility rate is less stable over generations, which is consistent with the final feasibility reported in Table 5.

5.3.3. Pareto Front Visualization and Trade-Off Interpretation

To visualize the trade-off between profit maximization and empty repositioning effort, we plot the Pareto front approximations across Routes 1–5 under three horizon scales $k\in \{2,5,10\}$. For each method, we merge the final non-dominated sets from 10 independent runs and extract the non-dominated subset of the union for visualization. This merged front reduces sensitivity to any single seed and better reflects the attainable trade-off set of each method.

Figure 3 shows that NSGA-II-RL yields a stronger trade-off set overall across routes and horizon scales. The RL-controlled front is shifted toward the upper-left region, indicating that it can achieve higher profit with the same empty TEU-miles, or equivalently reduce empty repositioning effort without sacrificing profit. The advantage is often most visible in the mid-range of the front where operational decisions face stronger tension between allocating slots to laden demand and repositioning empties. In contrast, the baseline and random-control variants exhibit very similar coverage in many panels, with only modest differences over certain ranges.

6. Discussion

6.1. Managerial Interpretation of the Bi-Objective Trade-Off

Our model highlights a fundamental profit–empty repositioning effort trade-off in integrated slot allocation and empty repositioning, where laden and empty containers compete for the same vessel capacity. In this study, we operationalize empty repositioning effort by empty TEU-miles, which reflects the transport work devoted to empty movements over the planning horizon.

Profit-oriented solutions allocate more capacity to revenue-generating laden flows and tend to reduce or postpone empty repositioning on some legs. Such solutions are attractive when freight rates are high or when capacity is tight, but they may exacerbate equipment imbalance at certain ports in later voyages and increase the risk of shortage-related costs or external sourcing needs, depending on the operational context. In contrast, solutions with lower empty TEU-miles emphasize a more conservative repositioning plan in terms of empty repositioning effort, which can be preferable when empty repositioning is expensive or operationally constrained.

From a planning perspective, the Pareto front provides a set of implementable policies rather than a single solution. Decision makers can select a point that matches current market conditions and operating priorities, and the framework also supports sensitivity analysis to examine how the profit–empty repositioning effort trade-off shifts under changes in demand mix, capacity tightness, and relevant cost parameters (e.g., shortage/holding costs or external sourcing penalties).

6.2. Horizon Length and Scalability

An important deployment question is how model size and runtime scale when the planning horizon is extended. For a fixed service network and fixed operational rules, the formulation scales linearly with the voyage count $\left|V\right|$ in the sense that most decision variables and constraints are voyage-indexed. Adding one voyage introduces an additional block of voyage-indexed variables and constraints with the same structural pattern; therefore, increasing $\left|V\right|$ by a factor k leads to a proportional increase in the number of voyage-indexed variables and constraints, i.e., the model size grows on the order of $\mathcal{O}\left(\right|V\left|\right)$ with respect to $\left|V\right|$.

This structural scaling is consistent with the computational results summarized in Table 5. When the horizon is extended so that the voyage count increases by a fixed multiplier, the observed runtime under the same solver configuration exhibits a broadly proportional increase, therefore the empirical pattern suggests that extending the horizon yields a predictable increase in computational effort. Practically, these observations provide guidance for selecting horizon length.

6.3. Limitations and Future Research

Several limitations define the current scope and motivate future extensions.

First, the current formulation focuses on a bi-objective trade-off between profit and empty TEU-miles, while service-related requirements are already reflected through the fulfillment-related constraints in our model, practical planning may still involve additional criteria that are not explicitly modeled as objectives, such as emissions and energy costs, schedule robustness to operational disruptions, and richer service-quality measures beyond the current fulfillment targets. The proposed framework is not restricted to two objectives and can be extended to multi-objective settings by incorporating additional performance metrics within the same Pareto-based solution paradigm. Future research can investigate tri-objective or many-objective variants and study how preference articulation and interactive decision support can be integrated into the planning process.

Second, our computational study relies on benchmark-derived services and scenario-based demand generation with calibrated parameters. Further validation using practical operational data would strengthen the empirical credibility of the findings. Future research can conduct practical case studies using operational records, such as booking data and equipment availability information, to evaluate how the recommended policies perform in real settings.

Third, the RL-based operator controller is implemented using a compact tabular Q-learning design for stability and interpretability. We do not provide a systematic comparison with alternative adaptive schemes, such as continuous-state parameter adaptation or deep-RL controllers. Benchmarking these alternatives in terms of solution quality, feasibility stability, and computational overhead, especially on larger instances, remains a meaningful direction for future work.

7. Conclusions

This paper studied an integrated short-term planning problem on a fixed cyclic liner service, where laden shipment execution and empty container repositioning compete for the same vessel slots across multiple voyages. We formulated a bi-objective mixed-integer model that maximizes profit while minimizing empty TEU-miles, and it explicitly captures key operational elements including stranded bookings, port-level empty-inventory evolution, and the use of externally leased containers as emergency supply that occupies slots but does not enter the owned-inventory balance. The resulting Pareto set reveals an operational trade-off between profitability and repositioning effort, providing decision support for carriers facing simultaneous revenue pressure and equipment imbalance.

To solve the bi-objective problem at practical scales, we developed an NSGA-II-RL approach that equips the NSGA-II framework with Q-learning based operator control to adapt variation and feasibility-handling behavior across generations. Computational experiments on five services derived from LINERLIB show that NSGA-II-RL consistently improves solution quality compared with both a fixed-operator baseline and a random-control variant. In particular, it achieves the best mean hypervolume across all tested route and horizon settings, and its advantage becomes more pronounced as the planning horizon length increases, suggesting that learning-based control is especially beneficial when richer feedback can be accumulated over a longer search process. Moreover, the feasibility trajectory stabilizes more clearly under NSGA-II-RL, indicating that the approach can navigate tight capacity and inventory constraints without sacrificing runtime. A discussion of limitations and potential extensions is provided in Section 6.3. Overall, the proposed model and solution framework offer a practical tool for jointly balancing profit and empty TEU-miles in integrated slot allocation and empty repositioning decisions.