Graph Neural Networks and Deep Reinforcement Learning for Warehouse Order Picking and Representation Learning

Abstract

Order picking is one of the most resource-intensive warehouse operations; therefore, improving routing efficiency remains an important challenge. Deep reinforcement learning (DRL) has shown promise in complex optimization problems. However, its application to warehouse order picking is still limited, and graph-based representation learning using graph neural networks (GNNs) in this context remains largely unexplored. This paper proposes a GNN-based DRL method that models warehouse layouts as graphs to optimize order-picking paths while simultaneously learning graph-based structural embeddings of storage locations. The approach is evaluated against exact optimal solutions for smaller instances and against classical heuristic baselines, including the Lin–Kernighan algorithm, in simulated warehouse environments of different scales. The results show that the proposed GNN–DRL approach produces routing solutions with low optimality gaps across different order sizes and remains effective across different warehouse sizes when fine-tuned. In addition, a preliminary small-scale multi-picker experiment illustrates that the proposed framework could be extended toward more complex warehouse optimization settings. Moreover, the learned node embeddings capture meaningful structural properties of warehouse layouts and adapt to different operational contexts, highlighting the potential of integrating GNNs and DRL as a flexible foundation for advanced warehouse optimization.

1. Introduction

Order picking is one of the most resource-intensive warehouse operations, accounting for a substantial share of operational costs in traditional and semi-automated facilities [1]. In its basic form, the order picking routing problem can be modeled as a Traveling Salesman Problem (TSP), which is NP-hard. In practice, however, warehouse systems introduce additional complexities such as order batching, multiple pickers, congestion effects, and integrated assignment and scheduling decisions. A wide range of heuristics and optimization approaches have been proposed for warehouse routing and related problems, often relying on mathematical programming and decomposition techniques to manage congestion, workload balancing, and fleet coordination for robotic warehouses [2,3,4]. In addition, the literature on dynamic vehicle routing problems (DVRPs) addresses routing decisions that must adapt to newly arriving requests or changing system conditions [5] which is relevant in a warehouse setting. Some of the recent work in the research area of DVRP explores anticipatory decision-making using approximate dynamic programming and hybrid offline–online approaches to account for future uncertainty and system dynamics [6].

The TSP is typically formulated as a graph-based problem. Graphs can be represented in two common ways: 1. where nodes correspond to coordinates in a geometric space, and 2. where nodes are defined by pairwise distances in a completely weighted graph. Warehouse layouts often impose grid-like constraints (orthogonal aisles and cross-aisles) [1], where distances follow Manhattan metrics rather than direct Euclidean distances. To model these settings, warehouse graphs are usually constructed with intersections and storage locations as nodes, with edges representing feasible traversal paths. These graphs are reduced to or constructed only from storage locations on order path, with distances computed via shortest paths through the layout [7].

Recent advances in Deep Reinforcement Learning (DRL) have shown promising results for combinatorial optimization tasks such as the TSP by training agents to take actions based on environment states and rewards from reward functions [8]. Although a few studies have explored DRL in warehouse optimization, it remains limited and underexplored [9,10,11]. Use of DRL in warehouse environments could enable agents capable of making dynamic operational decisions, such as route selection, order batching, and item allocation.

A promising research direction also lies in graph representation learning, which aims to learn vector embeddings of graphs, nodes, or edges [12]. Since warehouses can be represented as graphs, Graph Neural Networks (GNNs) [13] can offer a powerful way to extract embeddings for storage locations and other entities. These embeddings could capture structural information about warehouse layouts and provide richer inputs for decision-making tasks compared to hand crafted features. For example, item allocation policies might be improved by leveraging learned embeddings that encode layout-specific characteristics. The representation learning capabilities of the proposed approach enable the model to capture structural relationships within the warehouse graph, which can help the policy better reflect operational constraints. In addition, the learned node embeddings provide a compact representation of node roles within the graph, offering insight into the model’s decision process and improving the interpretability of the learning-based approach.

Importantly, the purpose of learning embeddings in this work is not limited to solving a single small TSP instance. Instead, embeddings allow the model to learn structural regularities of warehouse layouts and support policy learning rather than solving each routing instance independently from scratch. This method also provides a foundation for extending the approach toward more complex operational settings, including multiple pickers.

Several studies demonstrate that exploiting structural properties of routing instances can significantly improve performance. For example, clustering-based decomposition methods partition nodes into groups and construct local routes before integrating them into a global solution, thereby improving heuristic efficiency for TSP variants [14]. In warehouse environments, routing problems are often modeled as hierarchical extensions of the TSP that capture additional operational structure [15]. These approaches highlight the importance of identifying and leveraging latent structural properties of the underlying graph.

Graph neural networks address this challenge by learning continuous vector embeddings that encode structural position, connectivity patterns, and relational context within the graph. In contrast to predefined clustering strategies, embeddings provide a structural abstraction that allows the model to capture latent groupings and dependencies without explicit manual decomposition. Such an approach would be essential for developing scalable policies capable of generalizing to realistic warehouse settings involving multiple orders, alternative storage locations, and dynamic operational constraints without decomposition.

In this work, we propose an integrated GNN-based deep reinforcement learning framework for warehouse optimization that jointly addresses order picking path optimization and warehouse layout representation learning. Specifically, we aim to demonstrate the feasibility of training GNN models to produce near optimal order picking routes and learn embeddings of warehouse storage locations. Currently we only focus on optimizing order picking path, but we envision extending this approach to other problems by including orders and items as graph nodes and defining new reward functions, thereby laying the groundwork for a foundational GNN–DRL model that can be fine-tuned for diverse warehouse optimization tasks.

The present study deliberately focuses on a simplified single-picker routing scenario to evaluate the proposed GNN–DRL method in a controlled and analytically transparent setting. This formulation enables direct comparison with well-established TSP solution methods and allows us to isolate and assess the capabilities of the model without additional confounding operational complexities.

The objective of this work is not to replace exact TSP solvers in static environments, where optimal or near-optimal solutions can be obtained efficiently, but rather to establish a generalizable methodology for dynamic graph-structured warehouse optimization. By demonstrating that GNNs can learn meaningful structural embeddings while producing competitive routing policies, we lay the groundwork for extending the approach to more realistic and dynamic settings.

In particular, the proposed method is designed to accommodate future extensions involving multiple pickers, order batching, dynamically arriving orders, congestion effects, and jointly optimized warehouse decisions. The current formulation therefore represents a foundational step toward a holistic approach to optimization in complex warehouse systems. To illustrate the potential of the framework beyond the single-picker setting, we additionally conducted a small-scale experiment involving multiple pickers.

DRL applications with GNNs integration in the domain of internal logistics remain scarce [16,17,18], and to the best of our knowledge, no existing study explicitly focuses on representation learning, leaving significant unexplored potential for advancing real-time, data-driven warehouse decision making.

Accordingly, this paper addresses the following research question: can graph neural networks learn meaningful embeddings of warehouse layouts that capture their structural properties while producing near-optimal order-picking paths, and can such models generalize across different warehouse configurations?

The main contributions of this paper are threefold:

(i)

We formulate the warehouse order-picking problem as a graph-based deep reinforcement learning task using graph neural networks.

(ii)

We demonstrate that the proposed GNN–DRL approach produces routing solutions with low optimality gaps in simulated warehouse environments, with increasing advantages for larger and more complex orders and can generalize across different warehouse configurations with appropriate fine-tuning.

(iii)

We show that the learned node embeddings capture meaningful structural properties of warehouse layouts.

We evaluate the proposed approach in simulated warehouse environment and discuss its applicability to a broader class of warehouse optimization problems. Overall, the presented GNN–DRL framework provides a unified methodology for order-picking optimization and warehouse representation learning, and establishes a flexible foundation for advanced, real-time, data-driven warehouse decision-making systems.

Section 2 describes the warehouse graph construction and the GNN–DRL methodology. Section 3 presents experimental results, embedding analysis and a smaller preliminary multi-picker experiment. Section 4 discusses implications, limitations, and future research directions.

2. Materials and Methods

In this study we used synthetic warehouse layouts that were modeled as graphs, transforming them into inputs suitable for Graph Neural Networks (GNNs). GNN was trained using Deep Reinforcement Learning (DRL) to address the order-picking problem, with the additional objective of learning graph representations that capture warehouse structure and support decision-making. Our methodology follows a multi-step pipeline consisting of the following steps:

• Graph Construction from Relational Data: Warehouse layout is modeled as graph derived from relational database representations of storage locations and aisles of synthetic warehouse layout. We define nodes corresponding to intersections and storage locations, and edges corresponding to feasible traversal paths. We employ both PyTorch Geometric (version 2.7.0) [19,20,21] and NetworkX (version 3.6.1) [22] python (version 3.13.11) libraries to construct graphs. We use PyTorch graph for GNNs training and NetworkX for calculation of shortest distances for reward functions and for construction of distance matrices. PyTorch Geometric graphs are processed to conform to the input requirements of GNN training.
• Graph Verification: To verify the correctness of the constructed graphs, we manually checked the graph representation against the warehouse layout by comparing the distances between connected nodes in the graph with the corresponding distances in the tiny sized warehouse layout. This check was performed for all edges near intersections, where graph-construction errors are most likely to occur, and for a random sample of edges inside the aisles. This ensured that aisle, cross-aisle, and entrance connections were correctly represented, we also computed and visualized small set of optimal picking paths using python-tsp [23] implementation of Dynamic programing [24] to check for possible visible errors in graph. This ensures that the graph representation accurately reflects warehouse layout constraints and distances.
• Training GNN with DRL: We integrate GNNs with DRL to learn both embeddings of storage locations that capture layout structure and policy for generating optimized picking routes. The DRL agent produces actions based on warehouse graphs and receives rewards based on path efficiency.
• Evaluation of order picking paths produced by GNN: We compare the performance of the GNN–DRL model against classical heuristics in terms of average travel distance. Algorithms used for comparison are: Christofides algorithm [25], Local Search [26] and basic (ejection-chain) implementation of Lin–Kernighan heuristic [27]. We used python-tsp [23] implementation of those algorithms except for Christofides for which we used our own implementation.
• Embedding Visualization and Evaluation: The learned embeddings from the GNN are visualized and evaluated to assess whether they capture meaningful structural properties of warehouse layouts.

All experiments are conducted on synthetic warehouse layouts of different sizes generated to simulate a realistic grid-like layout. GNN training is run on a consumer-grade GPU (RTX 5060ti) inside a personal computer with 8 core CPU with 16 threads and 64 GB of RAM, demonstrating the feasibility of our approach without requiring large-scale computational resources.

The warehouse layout is formalized as a weighted graph $G=(V,E)$, where the vertex (nodes) set $V$ represents storage locations, aisles, intersections, and entrances, and edge weights; $E$ corresponds to physical travel distances (Figure 1). Distances are derived from typical Warehouse Management System (WMS) data or, when unavailable, estimated from the geometric layout [28]. Edge weights (distances) between two adjacent storage locations are defined as the physical widths of the storage location closer to the entrance. In our experiments, all storage locations have the same width of 1 m. For example, in Figure 1, the distance between storage location 0 and 3 is defined by the width of location 0 and is therefore 1 m. Non-ground locations are connected exclusively to the corresponding ground-level location with distance of 1 m. This allows locations at the same ground position to be treated consistently in the model. Horizontal storage locations are connected to neighboring ground-level storage nodes, with additional edges introduced between nodes on opposite sides of an aisle and weighted by the aisle width. Edges connecting aisle nodes to storage nodes are assigned distances of full aisle length. Intersection nodes are positioned along aisles and connected to the intersecting aisles, the nearest storage nodes, and to successive intersections within the same aisle, thereby forming continuous traversal paths. Both intersection and aisle nodes are removed in the final graph representation for GNN training, after which storage locations previously connected through intersections are linked directly. Further modifications required to prepare the graph as input for the GNN, including the selection of anchor nodes and the picker node, are described later in the text.

The smaller demo warehouse layout used for training contains 600 storage locations, arranged along 3 vertical aisles and 10 horizontal aisles with storage locations being assigned only to horizontal aisles, with two opposing storage walls per aisle of size x × y (x = 10, y = 3). In addition, the layout instance includes 15 intersections, modeled as graph nodes that connect vertical and horizontal aisles. For additional fine tuning and evaluation, we used larger warehouse with 1200 storage locations that has 10 additional horizontal aisles (two columns of 10 horizontal aisles versus two columns of 5 aisles for smaller warehouse).

Figure 2 illustrates the database schema employed to construct the experimental warehouse graph. The schema formalizes the warehouse layout through relational entities representing warehouses, aisles, intersections, storage locations, entrance nodes, and edges. These entities and their relationships are used to represent the corresponding graph structure, where nodes represent physical locations and edges encode navigable connections. For graph construction, distances between intersections located on the same aisle are approximated as 3 m. Note that the illustrated schema does not represent a real-world warehouse management system (WMS) database. Instead, it serves as a representation of the data required to generate the warehouse graph and to improve the reproducibility of the study.

Figure 3 depicts the graph-based representation of the experimental warehouse layout constructed using the Python NetworkX library (version 3.6.1). Nodes represent physical warehouse entities, including entrances, aisles, intersections, and storage locations, and are distinguished by color. Aisles 0, 6 and 12 are vertical aisles with no assigned locations. Edges denote feasible traversal paths between nodes. A similar graph structure as depicted in Figure 1 and Figure 3 serves as the input to the GNN, supporting the learning of node embeddings and the extraction of warehouse structural representations for downstream decision-making in order-picking tasks.

Figure 4 illustrates an example of an optimal order-picking route generated from a graph-based representation of the warehouse in a top-down view. The warehouse entrance (denoted by E) represents the starting and ending location of the picking tour. Storage locations with visible IDs and marked with green are ones on the picking path. The numbers displayed above the blue points denote the order in which locations are visited in the optimal picking sequence. Shortest-path distances between all pairs of storage locations are computed using Dijkstra’s algorithm [29], while the overall optimal picking sequence is determined using dynamic programming. Distances between locations are defined as described in Figure 1. Each square in Figure 4 represents an area of $1 \mathrm{m}\times 1 \mathrm{m}$. For non-ground-level locations ($\mathrm{i}\mathrm{d} \mathrm{m}\mathrm{o}\mathrm{d} 3\ne 0$), an additional distance of 1 m must be considered, as illustrated in Figure 1—although this is not visible in Figure 4. Similarly, a side change within the same aisle adds 1 m.

The diagram in Figure 5 illustrates the process of constructing the initial graph structure from warehouse layout data. It begins with retrieving data from a database and then creating nodes representing storage locations, aisles, intersections, and the entrance. Edges are added between nodes to reflect connectivity, with distance attributes based on physical measurements (such as storage width or height, and aisle length or width).

Next, all pairs shortest-path distances are computed using Dijkstra’s algorithm. For GNN processing, intersection and aisle nodes are subsequently removed, and storage nodes that were previously connected through these nodes are directly connected as shown in Figure 1. This graph simplification was observed to have a positive impact on the quality of storage-node embedding representations, while no change, in the average shortest-path distance in early training stages was observed, when compared to the original graph containing intersection and aisle nodes. We attribute this to the aggregation behavior of GNN layers, whereby storage nodes adjacent to any intersections become more similar to one another.

Finally, a set of anchor nodes (e.g., the first, middle, and last node ID) is selected, and additional anchor-type edges are introduced to store shortest-path distances from each node to these anchor points as shown in Figure 1. While it is not critical which specific nodes are selected as anchors, it is important that anchor nodes are not located next to each other and that they are well distributed across the warehouse layout. In our implementation, anchor nodes are selected as the first, middle, and last nodes according to the node ID ordering. These nodes provide a good spatial distribution across the warehouse, and the distance between the first and last node corresponds to the maximum possible distance between two storage locations in the graph. All distances are then normalized using minimum and maximum distances from anchor edges, resulting in a graph representation that is subsequently used for training and simulation.

DRL is used to train the GNN. Unlike supervised learning, where a model is trained to reproduce predefined outputs or labels, DRL trains the model to maximize a reward through interaction with the environment. Training is organized into episodes. During each episode, the model selects actions by following some learned policy, and the resulting changes in the environment state are observed. At the end of the episode, rewards are calculated and used to update the model so that actions leading to higher rewards become more likely. This approach is well suited to problems for which the optimal solution is unknown or computationally expensive to obtain [8].

In the proposed method, the GNN is trained to construct picking routes for a given order. At each decision step, the GNN selects the next action, which corresponds to the next storage node to be visited. A complete route is generated by repeatedly selecting actions from the learned policy and updating the environment state, including the picker position and the remaining locations to be visited, until all required locations have been visited. The diagram (Figure 6) illustrates the workflow of a single episode during simulation and training of the GNN-based order-picking agent. Each episode begins with the generation of random orders, after which all subsequent steps are executed in parallel for the entire batch of orders. For each order, a copy of the initial warehouse graph is constructed, and additional features are assigned to storage nodes to indicate their role. Storage nodes along the current path are marked with a binary feature (0 or 1) indicating whether they are part of the active path. An additional picker node is introduced to represent the current location of the picker. This node is connected to the entrance node or storage node that the picker most recently visited (for example see Figure 1). GNN produces two outputs:

• A score for each storage node, representing the likelihood that the node should be chosen as the next step in the path.
• A graph-level score, obtained via mean pooling over all storage node representations in the final GNN layer followed by feed forward layer (linear), giving value estimation (approximation of total distance left on order picking path).

During action selection, non-candidate storage nodes (those not on the path) are masked out, and a SoftMax is applied over candidates scores to generate probabilities. During training, the next action is sampled from this distribution, while during testing, the action with the maximum probability (argmax) is chosen. During training we remove last step from each order from our training dataset as number of candidates is only 1.

The resulting states, actions, value estimates, and rewards are stored, and the process is repeated until all required storage locations for each order have been visited. After completion of the picking tour, rewards are computed based on the total travel distance, including the return to the entrance, and normalized using the minimum and maximum values derived from anchor-based distances. Finally, the GNN is trained for several epochs using the collected episode data, after which the system proceeds to the next episode and repeats the cycle.

Finally, the trained model is evaluated on randomly generated test orders. The testing workflow is the same as the training workflow, except that the GNN training step is skipped because the model has already been trained. The trained model is therefore applied directly to previously unseen orders without need for retraining.

The GNN is trained using DRL training method named Proximal Policy Optimization (PPO) [30], which is an improvement over policy gradient methods [31]. The algorithm aims to maximize the average of action probabilities of taken actions based on the advantage estimator. Actions with higher advantage are reinforced, increasing their probability in future decisions, whereas actions with lower or negative advantage reduce their likelihood of being chosen in future decisions. This guides the policy toward selecting more effective actions and gradually improving overall performance. The policy is optimized by maximizing the policy gradient objective using gradient ascent, which is equivalent to minimizing the negative objective via gradient descent. Formally, the policy gradient loss is defined as:

$$L^{PG}\left(\theta \right)={\widehat{\mathbb{E}}}_t\left[\log {\pi }_{\theta }\left(a_t\right|s_t){\widehat{A}}_t\right.]$$

(1)

where ${\pi }_{\theta }$ denotes the policy parameterized by θ, representing the probability distribution over possible actions. ${\pi }_{\theta }\left(a_t|s_t\right)$ is the probability of taking action $a_t$ given the state $s_t$ at time step $t$ and ${\widehat{A}}_t$ is the estimated advantage function, which measures how much better (or worse) an action is compared to the expected value under the current policy. ${\widehat{\mathbb{E}}}_t$ represents empirical expectation over time steps, approximated in practice by averaging over a batch.

In algorithms like PPO, we are not just increasing the raw probability of an action in isolation; we are increasing the probability relative to the previous policy. Algorithms control updates relative to the previous policy rather than boosting absolute probabilities. To quantify this change, PPO defines a probability ratio between the new and old policies as

$$R_t\left(\theta \right)={\displaystyle \frac{{\pi }_{\theta }\left(a_t\right|s_t)}{{\pi }_{{\theta }_{old}}\left(a_t\right|s_t)}}$$

(2)

which measures how much more (or less) likely the current policy ${\pi }_{\theta }$ is to select action $a_t$ in state $s_t$ compared to the previous policy ${\pi }_{{\theta }_{old}}$. The unclipped objective is then expressed as

$$L\left(\theta \right)={\widehat{\mathbb{E}}}_t\left[R_t\left(\theta \right){\widehat{A}}_t\right]$$

(3)

In PPO, the clipping mechanism is used to prevent the policy from changing too much in a single update. While the policy ratio measures how much more (or less) likely the new policy is to select an action compared to the old policy, large deviations can destabilize learning. Clipping restricts the ratio to stay within a small range (e.g., $1\pm ϵ$). If the ratio moves outside this range, the objective uses the clipped value instead. This ensures that actions with high advantage are still encouraged, but the update remains controlled, preventing overly aggressive policy changes that could harm performance. The clipped objective is defined as:

$$L^{CLIP}\left(\theta \right)={\widehat{\mathbb{E}}}_t[{\min (R}_t\left(\theta \right){\widehat{A}}_t, clip\left(R_t\left(\theta \right),1-ϵ,1+ϵ\right){\widehat{A}}_t\left)\right]$$

(4)

where $ϵ$ is the clipping parameter (typically set to 0.1–0.2).

Full PPO loss function is sum of $L^{CLIP}$ and value loss $L^{Value}$ that is mean squared error between predicted and target value estimation multiplied by value coefficient and subtracted by entropy of probability distribution $\mathcal{H}$ multiplied by entropy coefficient). The full loss is defined as:

$$L=L^{CLIP}\left(\theta \right)-c_{value}{L}^{Value}\left(\theta \right)+c_{entropy}\mathcal{H}\left({\pi }_{\theta }\right)$$

(5)

where $c_{value}$ and $c_{entropy}$ are weighting coefficients for the value loss $L^{Value}$ (mean squared error between predicted and target value estimation) and entropy $\mathcal{H}$. Increasing the entropy of the policy helps promote exploration during training. A higher-entropy policy corresponds to a more diverse or less deterministic action distribution.

To improve training stability, methods such as Generalized Advantage Estimation (GAE) [32] are used. Without GAE, when the full sequence of rewards for an episode is available, the value target for the critic can be computed as the discounted sum of all future rewards. In our setting, the reward corresponds to the negative distance to the next location, and at the final timestep of an episode (when the agent reaches the last location in the path), we add the distance back to the entrance. All distances are normalized using the minimum and maximum values derived from anchor distances. Formally, target value estimate ${\widehat{V}}_t$ is defined as:

$${\widehat{\mathrm{V}}}_t=\sum _{l=0}^{\infty }\gamma r_{t+1}$$

(6)

where $r_{t+1}$ is the reward at timestep $t$ and $\gamma$ is the discount factor. The corresponding advantage can then be expressed as:

$$A_t={\widehat{V}}_t-V\left(s_t\right)$$

(7)

where $V\left(s_t\right)$ denotes the value function prediction for state $\left(s_t\right)$ and ${\widehat{V}}_t$ is target value estimate.

GAE improves this by using a smoothed combination of temporal-difference errors denoted as ${\delta }_t$. across multiple time steps. The temporal-difference at time step t is defined as:

$${\delta }_t=r_t+\gamma V\left(s_{t+1}\right)-V\left(s_t\right)$$

(8)

where $r_t$ is the immediate reward, $V\left(s_t\right)$ and $V\left(s_{t+1}\right)$ are value function estimates for the current and next state, respectively, and $\gamma$ is the discount factor. The parameter λ is subsequently used in GAE to control the exponential weighting of ${\delta }_t$ over time. Advantage estimation ${\widehat{A}}_t$ is then calculated as sum of ${\delta }_t$ and advantage of next step. ${\widehat{A}}_t$ satisfies the following recurrence relation:

$${\widehat{A}}_t={\delta }_t+\gamma \lambda {\widehat{A}}_{t+1}$$

(9)

where $\lambda$ is the GAE smoothing parameter. The value target ${\widehat{V}}_t$ is obtained by adding the advantage estimate ${\widehat{A}}_t$ to the baseline value function estimate $V\left(s_t\right)$.

$${\widehat{V}}_t={\widehat{A}}_t+V(s_t)$$

(10)

We applied Generalized Advantage Estimation (GAE) using a smoothing parameter $\lambda =0.9$ and a discount factor $\gamma =0.99$.

GNN training hyperparameters are set as follows: the entropy coefficient is 0.001, the value function coefficient is 0.5, and the PPO clipping parameter ($\mathrm{clip}\text{\_}\mathrm{eps}$) is 0.1. Optimization is performed using the AdamW [33] optimizer with a learning rate of $1\times {10}^{-4}$ and a batch size of 64.

At each episode, 42 random orders of size 25 are generated. For each order, the picker’s path is simulated, rewards are computed, and the model is trained for 8 epochs using the collected trajectories.

The model architecture begins with an embedding layer which increases dimensionality to 32. This is followed by 12 TransformerConv [34] layers with 4 attention heads and an output dimension of 32. After the TransformerConv layers, a linear layer is applied to nodes of the target type (storage nodes), followed by an additional linear layer that produces node-level scores. A SoftMax function is then applied to obtain action probabilities.

To estimate the global value, mean pooling is applied to the node embeddings after the first linear layer, followed by a linear layer that outputs the graph-level value estimate.

3. Results

Figure 7 shows the progression of the average distance per order during training of the GNN–DRL model and its comparison with classical heuristics. The x-axis represents the number of training episodes, ranging from 0 to 18,500, while the y-axis denotes the average distance traveled per order, measured in meters. Performance is evaluated every 100 episodes on a new set of 1000 randomly generated orders of size 25. The GNN–DRL model is compared against three baseline methods: Local Search, Christofides, and Lin–Kernighan indicated in the legend. To make figures more readable the y-axis is limited to the value of 185 m (values for episodes 100, 200, 300, 400 with y values cut from figure are approximately 259 m, 211 m, 189 m, 189 m).

During early training, the GNN–DRL model exhibits higher variance and longer average distances, followed by a gradual improvement as training progresses. After approximately 3000 episodes, the GNN–DRL model consistently outperforms the Christofides heuristic; after around 8000 episodes, it surpasses Local Search; and after approximately 12,000 episodes, it achieves sustained improvements over the Lin–Kernighan heuristic. After that, the GNN–DRL model maintains a consistently lower average distance per order than Local Search, Christofides, and Lin–Kernighan, demonstrating its advantage in the converged regime.

Temporary spikes in the GNN–DRL performance curve are visible throughout training and are attributed to continued exploration, which may allow the model to escape local minima. Each training episode requires on average 1.34 min, resulting in a total training time of approximately 17 days for 18,500 episodes of continuous training. Note that this is a one time cost and inference is done in around 0.057 s for every step (Simulation part in Figure 6) so for an order size of 15 it takes around 0.86 s and for an order size of 25 it takes around 1.44 s. For an order size of 25, the Lin–Kernighan heuristic is substantially faster for a single order, requiring less than 0.005 s per order. The computation time of the GNN–DRL approach can be substantially reduced by processing multiple orders in batches. When 1000 orders are evaluated together using the GNN model with a maximum batch size of 256, the average computation time decreases from 1.44 s to approximately 0.086 s per order. Although the Lin–Kernighan heuristic is faster, the GNN–DRL runtime is still practical for real-time use.

In the final phase of training, the model was trained on orders with sizes ranging from 5 to 25 storage locations, using 42 randomly generated orders per order size for each episode for a total of 50 training episodes. Model performance was first evaluated against optimal solutions for order sizes between 5 and 20. For each order size, 1000 random orders were generated, and the optimal route for each order was computed using dynamic programming. The optimality gap was calculated as the percentage deviation from the optimal solution:

$${Gap}_i(\%)={\displaystyle \frac{L_i^{\mathrm{method}}-L_i^{\mathrm{optimal}}}{L_i^{\mathrm{optimal}}}}100$$

(11)

where $L_i^{\mathrm{method}}$ denotes the route length produced by the evaluated method for order $i$, and $L_i^{\mathrm{optimal}}$ denotes the corresponding optimal route length. The reported gap represents the average gap across all 1000 test orders for a given order size.

Table 1. Optimality-gap statistics for GNN–DRL and Lin–Kernighan relative to exact solutions across different order sizes.

Order Size	Avg. Optimal Length	GNN–DRL				Lin–Kernighan
		Avg. Lengths	Gap (%)	% Optimal	P95 Gap (%)	Avg. Lengths	Gap (%)	% Optimal	P95 Gap (%)
5	72.02	72.56	0.77	83.6	5.07	72.46	0.61	88.8	5.49
6	78.23	78.82	0.76	81.6	5.06	78.89	0.86	84.6	6.67
7	83.94	84.69	0.89	76.9	5.44	84.77	1.00	79.9	6.98
8	89.76	90.73	1.06	72.3	6.38	90.78	1.16	74.4	6.67
9	94.97	96.22	1.31	65.1	6.38	96.20	1.32	70.2	7.06
10	99.86	101.21	1.34	63.0	6.36	101.46	1.63	64.9	7.93
11	104.62	106.01	1.32	60.1	6.00	106.30	1.61	60.0	7.15
12	109.06	110.70	1.48	54.9	6.13	111.21	1.98	53.6	9.40
13	114.43	116.38	1.70	49.3	6.67	116.57	1.88	51.6	8.04
14	118.60	120.61	1.69	47.9	6.45	121.04	2.08	44.9	7.90
15	122.46	124.48	1.65	46.9	6.25	125.20	2.26	38.7	7.94
16	127.10	129.30	1.74	40.6	6.06	130.13	2.42	36.6	8.66
17	130.75	133.10	1.80	40.5	6.80	133.91	2.43	36.2	8.40
18	134.19	136.61	1.82	35.9	6.72	137.92	2.79	28.4	8.96
19	137.75	140.24	1.81	36.0	6.20	141.72	2.90	26.5	9.09
20	142.06	144.49	1.72	33.8	5.57	146.27	2.98	22.0	8.80

reports the average route lengths and instance-level optimality gaps for the proposed GNN–DRL method and the Lin–Kernighan heuristic. The mean GNN–DRL optimality gap remains below 2% for all tested order sizes, increasing from 0.77% at order size 5 to 1.72% at order size 20. From order size 13 onward, the GNN–DRL gap remains within a narrow range of 1.65–1.82%, indicating that the increase in the gap becomes less pronounced for larger orders. The percentage of optimal solutions decreases as the order size increases for both methods, which is expected as the routing problem becomes more complex. However, GNN–DRL maintains a comparable or higher percentage of optimal solutions than Lin–Kernighan for larger order sizes. At order size 20, GNN–DRL produces optimal solutions for 33.8% of the tested orders, compared with 22.0% for Lin–Kernighan. The 95th percentile gap further shows that GNN–DRL remains relatively stable across order sizes. For GNN–DRL, the P95 gap stays mostly between approximately 5% and 7%, reaching 5.57% at order size 20. In contrast, the Lin–Kernighan P95 gap increases more clearly and reaches 8.80% at order size 20. Overall, these results indicate that the proposed GNN–DRL method remains close to the exact optimum across the tested order sizes and shows more stable high-percentile behavior for larger instances.

Those results should be interpreted as evidence that the proposed GNN–DRL policy can approximate optimal routing decisions, rather than as a claim that it replaces exact methods or highly optimized OR solvers for static TSP-like instances. Exact methods and strong heuristics remain well suited for the current static single-picker setting. However, the low optimality gaps indicate that the proposed graph-based learning approach may provide a useful foundation for more complex joint optimization settings where the problem cannot be reduced to a standard static TSP formulation.

A separate comparison was then conducted for larger order sizes, where exact dynamic programming was not used as the primary reference due to its increasing computational cost. In this experiment, order sizes from 5 to 35 items were evaluated, with 1000 randomly generated orders for each order size. The routes produced by the proposed GNN–DRL approach were compared with those generated by the Lin–Kernighan heuristic. This larger-scale comparison is reported separately from the optimality-gap analysis and is intended to evaluate the scalability of the learned policy relative to a strong heuristic baseline.

Figure 8 jointly illustrates the effect size and statistical significance of the performance difference between the GNN–DRL model and the Lin–Kernighan heuristic as a function of order size. Statistical significance is assessed using a Wilcoxon signed-rank test [35], applied separately for each order size to the paired route lengths obtained from 1000 randomly generated orders. The x-axis represents the order size, while the left y-axis shows the relative improvement of GNN–DRL over Lin–Kernighan, expressed as the percentage reduction in average route length. Negative values indicate superior performance of the GNN–DRL model. The right y-axis reports the statistical significance of this difference using the Wilcoxon signed-rank test, expressed as $-{\mathrm{l}\mathrm{o}\mathrm{g}}_{10}\left(p\text{-}\mathrm{value}\right)$, with the dotted horizontal line indicating the conventional significance threshold of $p=0.05$.

Statistically significant improvements (p-value < 0.05) emerge from order size 15 onward, with significance increasing rapidly for larger orders and reaching extremely small p-values that are close to zero (below 10⁻³⁰) for most order sizes above 20. The standard deviation values for both methods remain comparable across all order sizes, indicating that the observed performance gains are not driven by increased variance or outliers.

The corresponding

Table 2. Comparison of GNN–DRL and Lin–Kernighan performance across order sizes.

Order Size	Avg. Lengths GNN–DRL	Avg. Lengths Lin–Kernighan	% Change	Stat Wilcoxon	p-Value Wilcoxon	Standard Deviation GNN	Standard Deviation LK
5	72.29	72.29	0.00	13,377	8.04 × 10−1	9.36	9.46
6	79.08	79.24	−0.20	18,518	4.52 × 10−2	9.18	9.20
7	84.83	84.97	−0.16	27,512	3.18 × 10−1	9.06	9.03
8	90.95	91.10	−0.16	38,477.5	2.29 × 10−1	9.03	8.93
9	96.07	96.26	−0.20	65,057	5.26 × 10−1	9.14	9.18
10	101.20	101.49	−0.29	61,272	9.70 × 10−3	9.58	9.36
11	106.45	106.71	−0.24	73,991.5	9.33 × 10−2	9.83	9.67
12	111.17	111.53	−0.32	95,931.5	4.52 × 10−2	9.66	9.61
13	114.87	115.33	−0.39	95,396.5	1.91 × 10−3	9.81	9.68
14	119.95	120.26	−0.26	115,454.5	5.58 × 10−2	9.76	9.37
15	124.46	125.33	−0.69	99,094	2.92 × 10−9	9.80	9.83
16	128.91	129.78	−0.67	119,242.5	1.05 × 10−7	10.17	10.09
17	132.94	133.56	−0.47	121,430.5	9.14 × 10−6	9.86	10.03
18	136.52	137.79	−0.92	104,934	2.41 × 10−16	10.17	10.42
19	140.19	141.57	−0.98	116,505	5.90 × 10−14	9.74	9.99
20	143.87	145.66	−1.23	106,291.5	2.35 × 10−24	10.11	10.24
21	147.63	149.54	−1.27	108,883	8.96 × 10−25	9.63	9.82
22	150.93	153.30	−1.55	100,548.5	4.72 × 10−36	9.78	10.07
23	154.21	156.32	−1.35	110,766.5	5.27 × 10−26	9.55	10.07
24	157.76	160.15	−1.49	98,666	2.12 × 10−36	9.37	9.92
25	160.77	163.26	−1.53	105,693	3.29 × 10−34	9.54	9.99
26	163.91	166.67	−1.66	105,217.5	8.33 × 10−39	9.63	10.09
27	166.88	169.96	−1.81	102,923.5	2.88 × 10−41	10.10	10.22
28	169.32	172.50	−1.84	96,287.5	1.24 × 10−45	9.96	10.08
29	173.10	176.72	−2.05	86,931.5	1.84 × 10−51	9.60	10.21
30	176.08	179.50	−1.90	99,242	4.27 × 10−46	9.75	10.17
31	178.68	182.66	−2.18	80,420.5	1.87 × 10−62	9.10	9.96
32	181.63	185.62	−2.15	87,307	1.66 × 10−56	9.50	10.37
33	183.67	187.74	−2.17	90,216	2.99 × 10−56	9.65	9.92
34	186.87	190.97	−2.14	88,614.5	7.82 × 10−56	9.77	10.41
35	189.10	193.70	−2.38	86,958	1.11 × 10−60	9.87	10.76

provides a quantitative comparison between GNN–DRL and Lin–Kernighan across all evaluated order sizes, reporting average route lengths, percentage changes, standard deviations, and Wilcoxon signed-rank test statistics. For small order sizes (5–10 locations), average route lengths are nearly identical, and differences are not consistently statistically significant. As order size increases, the GNN–DRL model increasingly outperforms Lin–Kernighan, as reflected by a growing negative percentage change in route length.

p-values in the tables are reported in scientific notation (e.g., $1.11\times {10}^{-60}$), representing very small numbers close to zero and indicating strong statistical significance. Overall, the results shown in Figure 8 and the accompanying table (Table 2) demonstrate that the performance advantage of the GNN–DRL approach against Lin–Kernighan becomes more pronounced as order size and combinatorial complexity increase as distinguishable by increasingly bigger percentage difference and p-values close to zero which shows bigger significance in difference between approaches, confirming its effectiveness for larger and more challenging order-picking problems.

When the model trained on the smaller warehouse was directly applied to the larger warehouse containing approximately 1200 storage locations, a decrease in performance was observed. To address this, the model was fine-tuned on the larger warehouse by training for an additional 2800 episodes using orders of size 25, with 4 training epochs per episode. Due to GPU memory constraints, the batch size was reduced from 64 (used for the smaller warehouse) to 48, and the learning rate was decreased from $1\times {10}^{-4}$ to $1\times {10}^{-5}$. After this fine-tuning phase, an additional 50 episodes of training were performed using variable order sizes ranging from 5 to 25 locations, following the same procedure as for the smaller warehouse.

Figure 9 illustrates the performance of the GNN–DRL model during fine tuning and baseline heuristics when applied to a larger warehouse instance. The x-axis shows the number of training episodes, while the y-axis reports the average distance per order in meters. Results are shown for the GNN–DRL model and three comparison methods: Local Search, Christofides, and Lin–Kernighan.

As shown in Figure 9, the GNN–DRL model consistently achieves lower average distances per order compared to the Lin–Kernighan heuristic throughout training, despite the increased problem scale. However, the relative performance improvement is smaller than that observed for the smaller warehouse, indicating a slight reduction in percentage gain as warehouse size increases. These results demonstrate that while the GNN–DRL approach generalizes to larger warehouse instance, additional fine-tuning is necessary and performance gains diminish moderately with the increase in total number of storage locations in warehouse.

Figure 10 illustrates the effect size and statistical significance of the performance difference like Figure 8 but for a bigger warehouse. The solid curve with circular markers represents the percentage improvement of GNN–DRL relative to Lin–Kernighan. For small order sizes, improvements are close to zero and fluctuate around the baseline, indicating negligible performance differences. Order sizes for which the Wilcoxon test does not indicate statistical significance ($p\ge 0.05$) are sizes 7, 8, 9, and 11 and are highlighted with large red cross markers. These cases correspond to the entries in

Table 3. Average route lengths and wilcoxon signed-rank test results for GNN–DRL and Lin–Kernighan across order sizes.

Order Size	Avg. Lengths GNN–DRL	Avg. Lengths Lin–Kernighan	% Change	Stat Wilcoxon	p-Value Wilcoxon	Standard Deviation GNN	Standard Deviation LK
5	98.51	98.20	0.31	16,723.0	1.77 × 10−4	12.01	11.99
6	107.38	107.14	0.22	24,573.0	1.11 × 10−3	11.57	11.66
7	115.48	115.47	0.01	40,093.0	5.47 × 10−1	11.57	11.59
8	123.34	123.42	−0.07	65,723.5	8.32 × 10−1	11.29	11.46
9	130.20	130.50	−0.23	66,902.0	1.03 × 10−1	12.38	12.43
10	137.74	138.11	−0.26	80,217.5	1.57 × 10−2	11.84	11.69
11	143.94	144.04	−0.07	92,446.0	8.11 × 10−1	11.92	11.98
12	150.09	150.52	−0.29	96,304.0	2.50 × 10−2	12.11	11.99
13	156.91	157.50	−0.37	98,883.0	3.89 × 10−3	11.84	11.97
14	163.46	164.50	−0.63	98,354.0	5.49 × 10−8	12.13	12.46
15	169.52	170.84	−0.77	95,319.5	4.16 × 10−11	12.32	12.52
16	175.80	176.69	−0.50	121,590.5	1.32 × 10−5	12.65	12.86
17	181.67	182.85	−0.64	120,991.0	4.97 × 10−8	13.21	13.31
18	187.31	189.00	−0.90	119,723.0	5.11 × 10−13	12.92	13.32
19	192.95	194.15	−0.62	138,789.0	1.39 × 10−6	12.89	13.32
20	197.29	198.77	−0.74	132,526.5	1.83 × 10−10	13.30	13.93
21	202.95	204.82	−0.91	130,349.0	8.72 × 10−15	12.60	13.10
22	207.94	209.92	−0.94	129,352.0	1.01 × 10−15	13.41	13.68
23	213.94	215.81	−0.87	136,634.0	1.57 × 10−14	13.62	14.29
24	218.28	220.81	−1.14	124,959.5	1.45 × 10−22	13.53	14.24
25	223.75	226.37	−1.16	124,855.0	8.71 × 10−25	13.61	14.04
26	227.86	230.19	−1.01	134,447.5	8.93 × 10−19	13.44	14.14
27	233.15	235.99	−1.20	135,423.0	1.18 × 10−20	13.47	14.42
28	237.77	240.30	−1.05	131,533.0	6.76 × 10−19	14.03	14.66
29	240.56	243.56	−1.23	128,496.5	7.21 × 10−28	13.70	13.97
30	246.04	249.35	−1.32	122,143.5	1.16 × 10−27	13.41	14.26
31	250.68	254.04	−1.32	117,603.0	5.30 × 10−32	13.90	14.45
32	255.20	258.18	−1.16	137,896.5	1.21 × 10−24	13.65	14.31
33	259.02	262.85	−1.45	108,961.0	6.04 × 10−38	14.12	14.89
34	261.88	266.23	−1.63	102,239.5	4.12 × 10−46	13.86	14.75
35	267.86	271.85	−1.47	120,201.5	2.70 × 10−32	14.26	15.47

where p-values exceed the 0.05 threshold, confirming that observed differences for small order sizes are not statistically significant.

As order size increases, the magnitude of improvement steadily grows, reaching reductions of approximately 1.0–1.6% for larger orders. In parallel, the $-{\mathrm{l}\mathrm{o}\mathrm{g}}_{10}\left(p\text{-}\mathrm{value}\right)$ curve rises sharply, indicating rapidly increasing statistical significance. From order size 11 onward, all improvements are statistically significant, with p-values decreasing by several orders of magnitude. For orders above 24 locations, the p-values reported in the table mostly fall below ${10}^{-20}$, demonstrating that the observed performance gains are both substantial and highly robust.

Overall, Figure 10 and the accompanying Table 3 consistently show that while performance differences between GNN–DRL and Lin–Kernighan are marginal and statistically insignificant for small orders, the GNN–DRL model achieves increasingly larger and statistically significant improvements as order size and therefore combinatorial complexity increases. This confirms that the advantage of the GNN–DRL approach becomes more pronounced in larger and more challenging order-picking scenarios.

To better understand how the GNN captures the warehouse structure, we visualize the learned embeddings of storage nodes. The embeddings are taken from the output of the final TransformerConv layer under the conditions where all storage locations are marked as being on the order path and the picker location is set to the entrance. For visualization purposes, the embedding dimensionality is reduced to two using Uniform Manifold Approximation and Projection (UMAP) [36].

Figure 11 presents the resulting visualization alongside the corresponding warehouse layout grid. On the left, storage locations are colored according to their physical aisles, while on the right, colors represent k-means clusters [37] derived from the learned embeddings. The number of clusters is set equal to the number of aisles containing storage locations. Each point corresponds to a storage location: the point shape (square vs. circle) indicates the side of the aisle, border thickness encodes the x-coordinate (thicker borders correspond to larger x values), and opacity represents the y-coordinate, with more opaque points indicating higher vertical positions.

Spatial mapping enables a direct comparison between the learned embedding space and the physical warehouse layout. In the left part of Figure 11, colors denote individual aisles. While storage locations from the same aisle tend to form broadly coherent clusters, the embedding structure indicates that physical proximity, both between storage locations and relative to the warehouse entrance, also plays an important role and in some cases appears more influential than aisle membership. Storage locations that are spatially close to one another, as well as those with similar distances to the entrance, appear closer in the embedding space even when they belong to different aisles. The strong alignment between the relative positions of storage locations in the embedding space and their positions in the physical warehouse layout demonstrates that the learned embeddings successfully preserve meaningful spatial organization.

Figure 12 shows the corresponding warehouse layout grid, where storage locations are colored to match the k-means clusters visible in the UMAP visualization. Color is based on cluster of ground (y = 0) location. This spatial mapping allows a direct comparison between the learned embedding structure and the physical warehouse layout. The strong correspondence between UMAP, physical aisles and distance to entrance confirms that the learned embeddings preserve meaningful spatial organization.

Anchor nodes (with identifiers 0, 300, and 599) are visually distinguishable in the embedding space, indicating that anchor-based distance features influence the learned representations. While these anchors improve structural awareness during learning, it remains an open question whether their explicit inclusion could be reduced or removed without degrading model performance.

Figure 13 illustrates how the storage-node embeddings change when the picker’s location is modified. In this experiment, the picker node is placed at storage location with ID 495. Projection shows a noticeable reorganization of clusters, which visually correlates strongly with graph distance to the selected picker location. Notably, all locations within the same aisle as picker (Aisle 10) form a compact cluster in the embedding space.

Storage locations closer to the picker in the warehouse graph appear more similar in the embedding space which can also be identified in k-means clusters projected on warehouse layout (Figure 14). This behavior suggests that the learned embeddings dynamically encode task-relevant context, such as the current picker position. In practical applications, this property could be exploited to identify similar storage locations for items that frequently co-occur in orders, as well as to identify dissimilar locations for items that should be spatially separated (e.g., due to congestion or incompatibility constraints).

Presented results support the main motivation outlined in the Introduction: that graph-based representation learning achieved with deep reinforcement learning can effectively capture warehouse layout structure. The visualizations demonstrate that the GNN learns meaningful embeddings.

Importantly, these embeddings are not only descriptive but also operationally useful. The previously demonstrated improvements over classical heuristics indicate that the learned representations can directly support decision-making in order picking. The sensitivity of embeddings to the picker’s location further highlights their adaptability to dynamic operational contexts.

Overall, these results validate proposed approach and illustrate the potential of GNN-based warehouse representation learning done with deep reinforcement learning for real-time, adaptive warehouse optimization, as anticipated in the Introduction.

Preliminary Multipicker Experiment

Real warehouse optimization problems rarely correspond to a simple TSP, as they often involve multiple pickers operating simultaneously and additional constraints such as, for example, congestion. To explore how our approach could be extended to a multi-picker setting, we conducted a smaller-scale experiment in which the model produced predictions for two pickers. In this experiment, both pickers worked on the same order. The model therefore had to assign order locations to the two pickers at each step and generate a picking route for each picker, with the objective of minimizing the total distance traveled by both pickers. This experiment is intended as a preliminary illustration of how the proposed graph-based learning framework may be extended toward multi-picker settings, rather than as a full evaluation of practical multi-picker warehouse performance.

Compared to the single-picker formulation, the graph representation is extended by adding a second picker node. After the final TransformerConv layer, we compute element-wise multiplications between storage node embeddings and picker embeddings to produce storage node embeddings for each picker. These embeddings are then passed through linear layers that produce the policy outputs. The value estimate is computed directly from a global pooling operation over storage node embeddings.

At each simulation step, the model outputs two actions, one for each picker. If only a single storage location remains to be visited (when the number of locations in the order is odd), the final location is assigned to the picker whose current route has the lower cost. To account for route overlap we introduce an additional penalty: if the predicted paths of the two pickers intersect during a step, the step cost is multiplied by 1.2.

As a baseline, we combine agglomerative clustering [38] with the Lin–Kernighan TSP heuristic. Storage locations are first partitioned into two clusters using agglomerative clustering. The clusters are then rebalanced to obtain approximately equal sizes based on the average distance to cluster members. Finally, a Lin–Kernighan solver computes a TSP route within each cluster.

The model used in this experiment consists of eight TransformerConv layers with one attention head and an embedding dimension of 128. The policy head includes a linear layer of size 1024. Training was performed with a learning rate of $1\times {10}^{-4}$ for the first 700 episodes. Each episode contained 42 orders with sizes between 6 and 10 locations and took around 52 s to finish. Additional hyperparameters included a PPO clipping parameter of 0.2, entropy coefficient of 0.0001, value coefficient of 0.5, batch size of 64, and four training epochs per episode.

After 700 episodes, we reduced the learning rate to $1\times {10}^{-5}$ and the clipping parameter to 0.1, while increasing order sizes to between 10 and 15 locations. Each episode took about 90 s to finish. We observed that the model converged more quickly when first trained on smaller instances and then gradually exposed to more complex orders. However, the optimal training curriculum remains an open question. For this experiment we trained model for total of 4000 episodes taking around 93 h.

The results in Figure 15 show that the GNN–DRL approach converges to solutions that start to outperform clustering + Lin–Kernighan baseline after approximately 1000 training episodes for order size of 10 locations.

Figure 16 and accompaning

Table 4. Average route lengths and Wilcoxon signed-rank test results comparing GNN–DRL and Clustering + Lin–Kernighan across order sizes in the multi-picker setting.

Order Size	Avg. Lengths GNN–DRL	Avg. Lengths Clustering + L–K	% Change	Stat Wilcoxon	p-Value Wilcoxon	Standard Deviation GNN	Standard Deviation LK
6	91.57	106.02	−13.62	56,663.0	2.00 × 10−95	15.83	13.06
7	97.63	112.26	−13.03	58,777.5	1.03 × 10−93	15.77	12.63
8	106.96	118.16	−9.48	98,181.0	5.47 × 10−59	16.56	12.27
9	112.03	123.95	−9.62	93,177.5	3.82 × 10−62	17.12	12.00
10	122.40	128.07	−4.43	162,993.5	1.13 × 10−17	17.38	11.57
11	128.77	132.91	−3.11	189,083.5	5.52 × 10−9	18.91	11.62
12	137.99	137.94	0.04	237,257.0	7.28 × 10−1	20.28	12.03
13	144.61	143.03	1.11	224,598.0	5.99 × 10−2	20.27	11.51
14	155.73	146.24	6.49	140,432.5	1.77 × 10−29	20.92	11.79
15	163.23	150.55	8.43	106,824.0	2.71 × 10−51	20.94	12.57

illustrates the effect size and statistical significance of the performance difference for multi-picker setting at Episode 4000. The results show that the method is capable of quickly outperform Clustering + Lin–Kernighan baseline for smaller order sizes (order size ≤ 11). While short training time was not yet sufficient for the model to learn to effectively solve the TSP for larger order sizes. At 4000 episodes the model is still improving, indicating that it has not yet reached its converged state.

The corresponding Table 4 provides a quantitative comparison between the GNN–DRL approach and the Clustering + Lin–Kernighan baseline across all evaluated order sizes. The table reports average route lengths, percentage differences, standard deviations, and the results of the Wilcoxon signed-rank test. For smaller order sizes (≤11), the average route lengths produced by GNN–DRL are significantly shorter than those obtained using the Clustering + Lin–Kernighan heuristic. The higher standard deviation observed for the GNN–DRL approach compared to the Clustering + Lin–Kernighan baseline indicates greater variability in the generated routes. This variability is expected, as the model is still in relatively early stages of training and has not yet consistently converged to near-optimal TSP solutions.

These preliminary results suggest that the proposed framework can be extended toward multi-picker and possibly other warehouse optimization scenarios. However, further work is required to incorporate more realistic operational constraints, including detailed congestion effects and dynamic order arrivals, as well as to provide a more comprehensive large-scale evaluation.

4. Discussion and Conclusions

This study demonstrates that the combined GNN–DRL approach consistently produces routing solutions with low optimality gaps while simultaneously learning meaningful graph-based representations of warehouse layouts across different warehouse sizes and order configurations. While exact optimization methods and advanced heuristics remain highly effective for static TSP formulations, the proposed framework provides a foundation for extending graph-based representation learning and adaptive policy learning toward more dynamic and operationally complex warehouse optimization settings. Additionally, preliminary experiments suggest that the proposed method can be extended to a multi-picker setting.

The embedding analysis together with low optimality gaps in order picking path lengths provides clear evidence that GNNs can capture meaningful structural information. UMAP visualizations reveal that storage locations belonging to the same aisle are visually distinguishable in embedding projections, while neighboring aisles are positioned close to each other in the embedding space. This indicates that warehouse structure is reflected in the learned representations. These observations confirm that the GNNs can properly encode higher-level structural properties of the warehouse layout. Unlike other DRL routing approaches, the learned representations can be inspected and related to physical warehouse structure, revealing clear alignment with aisle organization, spatial proximity, and picker location. This transparency supports interpretability by enabling validation of whether the model encodes meaningful and operationally relevant information rather than spurious patterns. The consistency between embedding structure, warehouse layout, and observed routing performance also serves as an additional sanity check for the correctness of the modeling assumptions and experimental setup, strengthening confidence in the validity of the proposed framework.

Further insight is provided by experiments in which the picker’s location is changed. The resulting reorganization of the embedding space correlates strongly with graph distance to the picker, demonstrating that the learned representations are context-sensitive and dynamically adapt to task-relevant information. This property suggests that the embeddings could support additional warehouse decision-making tasks beyond routing, such as identifying similar or dissimilar storage locations based on operational criteria.

Regarding generalization, the results show that models trained on smaller warehouse can be transferred to larger warehouse through additional fine-tuning. Although direct application without adaptation results in reduced performance, a limited fine-tuning phase is sufficient to restore and maintain a consistent advantage over classical heuristics. While relative improvements are slightly smaller in the larger warehouse, the GNN–DRL approach remains robust across scales, suggesting that the learned representations capture reusable, structural patterns that generalize across warehouse sizes, rather than overfitting to a single configuration.

Further research direction could be a more detailed exploration of multi-picker setting and extending warehouse graph to include more graph entities like orders and items. This extension would allow joint optimization of tasks such as order batching and item allocation or reallocation, while extending representation learning to additional entities including orders and items.

Despite the encouraging results, several limitations of this study should be acknowledged. First, all experiments were conducted on synthetic warehouse layout, which, while designed to reflect realistic grid-based structures, may not fully capture the heterogeneity and operational constraints of real-world warehouses. Second, the proposed GNN–DRL approach requires a non-trivial training time, which may limit its applicability in settings where rapid deployment or frequent retraining is required, although this cost is incurred offline and inference remains efficient. However, our additional experiments indicate that training efficiency may be improved through staged training procedures, where the model is first trained on simpler problem instances (e.g., smaller order sizes) and subsequently fine-tuned on more complex instances. Such curriculum-style training showed faster convergence in our experiments, although identifying optimal training procedures remains an open direction for future research. Third, the current formulation relies on anchor-based distance features to support representation learning; while effective, the extent to which these engineered features are necessary warrants further investigation. Finally, the experimental setup considers a single-picker, single-tour scenario, and does not yet address multi-picker interactions, congestion effects, or dynamic order arrivals, which are important aspects of practical warehouse operations and represent natural directions for future work.

Overall, the findings provide affirmative answers to the research question posed in this study. First, the results demonstrate that GNNs can learn embeddings that capture meaningful structural properties of warehouse layout while providing low optimality gaps in order-picking performance. Second, the learned models exhibit the ability to generalize across different warehouse configurations, with fine-tuning providing an effective mechanism for adapting to different warehouse sizes. Taken together, these findings position GNN–DRL not merely as a competitive routing heuristic, but as a general representation-learning framework for adaptive, graph-structured warehouse decision making.