Representing Search State to a Large Language Model: A Structured Telemetry Interface for Runtime Control of Adaptive Large Neighbourhood Search

Highlights

Please indicate how your work links to systems science via your contributions to systems practice, theory, and/or methodology.

• Introduces a seven-block telemetry methodology that represents metaheuristic search as an observable and controllable dynamic system.
• Develops a closed-loop architecture separating stochastic LLM supervision from deterministic optimization, validation, and feasibility management.

What are the main findings and/or the implications of the main findings?

• Under a fixed 1000-iteration budget, LLM-ALNS reduced the mean gap from 2.93% for adaptive ALNS to 0.18% across nine real-road CVRPTW instances.
• Telemetry-conditioned actions improved short-budget search control while preserving feasibility, demonstrating the potential of bounded LLM supervision despite its computational overhead.

Abstract

Adaptive large neighbourhood search is steered by high-level control decisions, yet its standard adaptation reacts to a narrow reward signal that conveys little about the state of the search. Recent work has strengthened this control with machine-learning and reinforcement-learning policies, but these generally require task-specific training. This paper proposes a framework in which a large language model acts as a bounded, training-free runtime supervisor of the search. The central contribution is a structured telemetry interface that compresses the evolving search into a compact, reproducible state representation over which the model can reason. The supervisor adjusts only search-control parameters and never modifies the solution, so construction, feasibility, and objective evaluation remain inside the metaheuristic. The framework is evaluated on a multi-depot capacitated vehicle routing problem with time windows on real city street networks, against conventional variants and two dedicated solvers. Under an equal iteration budget, the supervised algorithm reduces the mean gap to the best per-instance result from 2.93% to 0.18%, and its difference from the strongest dedicated solver is not statistically significant, although it runs at a higher computational cost. These results indicate that a language model can control metaheuristic search effectively when supported by a structured interface between the algorithm and the model.

1. Introduction

An experienced practitioner watching a metaheuristic run can often tell, long before the iteration budget is spent, that something has gone wrong: operators have stopped contributing, the acceptance rate has collapsed, the search keeps revisiting the same regions, or the incumbent has not improved for hundreds of iterations. Adaptive large neighbourhood search (ALNS) makes such control decisions automatically, choosing which operators to emphasise, how aggressively to perturb, how much worsening to accept, and when to diversify [1,2]. Its standard mechanism, however, reacts to a single narrow signal, recent operator rewards [1,3], and cannot diagnose the richer pathologies a human observer recognises. A meta-analysis finds this adaptive layer yields only a modest average gain over non-adaptive selection [4], pointing to the controller’s limited view rather than to adaptation itself [5,6].

A natural response is to widen what the controller sees and to learn the policy. Reinforcement-learning controllers do this for ALNS, mapping a search-state representation to operator, parameter, and acceptance decisions [7,8], including routing with time windows [9]; each, however, needs task-specific training, engineered rewards, and re-validation when the instance distribution shifts [10]. Whether a general-purpose reasoner can make these decisions without training is therefore an open question. A large language model (LLM) is a natural candidate, since it requires no task-specific training and reasons over heterogeneous, mixed-type state indicators.

Hyper-heuristics offer a template for such supervision: a high-level strategy observes the state of the search and selects, configures, or sequences low-level heuristics without ever manipulating the solution itself [11], and this separation is what makes supervision safe. Most LLM-for-optimisation work departs from the template and uses the LLM to generate or configure solutions directly, from programmes [12] to the automated design of heuristics and metaheuristics [13,14,15,16]. Under hard constraints, this invites infeasibility and weak reproducibility, and on network-structured problems LLMs frequently violate constraints and need error correction [17]. This paper adopts the opposite division of labour: the LLM occupies the high-level role of the hyper-heuristic controller and never sees or edits the solution, while ALNS builds routes, enforces feasibility, and evaluates cost.

The central hypothesis is that effective LLM supervision depends less on the model than on how the search state is represented to it. We therefore propose a structured telemetry interface that compresses thousands of raw per-iteration events into a compact, mathematically defined, reproducible state representation, turning an opaque event stream into segment-level indicators that make stagnation, operator starvation, acceptance collapse, and infeasibility pressure directly readable in a bounded format that a general-purpose reasoner can parse reliably at every decision point.

The interface is evaluated on a multi-depot capacitated vehicle routing problem with time windows (CVRPTW) built on the real street networks of three cities, whose asymmetric, non-Euclidean travel costs give a demanding, reproducible stress test rather than a bid for best-known results. The scheme is compared against adaptive ALNS, a fixed-weight variant, and rule-based and random supervisors that read the same telemetry within the same bounds.

The research question is therefore: can a structured telemetry interface let a bounded, training-free LLM supervisor make runtime control decisions that improve ALNS under constrained dynamics? The aim is not to beat dedicated solvers on runtime, since an LLM supervisor is inherently slower, but to test whether a compact, training-free interface lets a general-purpose reasoner steer search. The paper makes the following contributions.

• A structured seven-block telemetry interface, the central contribution, that converts raw search logs into segment-level, mathematically defined indicators and thereby defines how the state of a running ALNS is represented to a general-purpose reasoner.
• A bounded LLM-supervised ALNS (LLM-ALNS) architecture in which the LLM reads only this telemetry and adjusts only search-control parameters, with a deterministic validator that checks, clips, or rejects every action before it reaches the search.
• An experimental design that isolates the effect of the decision policy, comparing LLM-ALNS against adaptive and fixed-weight ALNS and against rule-based and random supervisors that read the same telemetry within the same bounds.

The remainder of the paper is organised as follows. Section 2 reviews adaptive control in ALNS, learning-based search control, and LLMs in optimisation. Section 3 defines the optimisation problem and the supervisory-control problem. Section 4 presents the telemetry interface, the bounded intervention space, and the validation of supervisory actions. Section 5 reports the experimental design and results, and Section 6 concludes.

2. Literature Review

The CVRPTW is a widely studied constrained routing problem [18,19,20] for which exact methods guarantee optimality but scale poorly [21], classical heuristics are fast but limited in quality [22], and metaheuristics trade the optimality guarantee for high-quality solutions within practical budgets [23]. Among them, ALNS is one of the most successful routing frameworks: its destroy-and-repair design rebuilds large portions of a solution while preserving useful structure, and an adaptive rule favours operators that have recently performed well [1,2]. ALNS remains in active use for time-window variants with competitive results [24,25], so CVRPTW provides an established benchmark on which to study adaptive search control.

ALNS adaptation is driven by a narrow signal: operator scores record only recent improvement, acceptance, or rejection, and both the acceptance criterion and the operator set markedly affect performance [5,6], while the adaptive layer yields, on average, only a modest gain over non-adaptive selection [4]. Existing adaptive approaches therefore give limited visibility into the causes of stagnation or degradation, which in CVRPTW may arise from infeasible repairs, structural cycling, excessive destruction, or tight time-window structure [26,27].

Controlling an algorithm during its execution is a long-standing theme. Online parameter control adjusts an algorithm during its run [28], while offline tools such as irace tune parameters beforehand [29]. Hyper-heuristics formalise the underlying separation: a high-level strategy selecting or controlling low-level heuristics without modifying the solution directly [11], and adaptive operator selection, including bandit formulations, treats operator choice as a state-dependent decision [30,31,32]. Common to all is a representation of the search state, expressed as operator-reward histories, landscape descriptors, or hand-designed features; the design of that representation, more than the control rule alone, is the concern of this paper.

Reinforcement learning (RL) has made this representation richer, and the control policy has been learned. Surveys of machine-learning-assisted metaheuristics and of RL for combinatorial optimisation document learned policies for operator choice, parameter adaptation, and broader search behaviour [33,34,35], and neural large neighbourhood search embeds learning within the LNS procedure [36]. For ALNS specifically, deep-RL, graph-RL, and proximal-policy-optimisation controllers select operators, set parameters, and adjust acceptance and termination from learned search-state or graph encodings [7,8,9], including an electric-routing variant that draws features from the instance and the evolving solution [10], while offline RL has learned dispatching policies from historical data [37]. Learned online control of ALNS is therefore effective, at the cost of task-specific offline training, reward engineering, and re-validation of transfer when the instance distribution changes [38,39].

LLMs have entered optimisation along two lines. In the first, the LLM generates artefacts, from programmes [12] to entire metaheuristics [13,14,15,16]; used as a direct solver of network-structured problems, it frequently violates constraints and needs error correction [17]. In the second, it acts as a reasoning agent that observes a state and decides rather than producing a solution, as in the literature on LLM-based autonomous agents and controllers [40], and reasons over structured, non-textual input in tasks such as data wrangling [41]. This work belongs to the second line and differs from using an LLM as an offline tuner or configurator [42,43,44]: here, it is an online controller reasoning over a structured representation of search dynamics.

These lines locate the contribution. Existing approaches occupy two extremes: reward-based adaptation with limited state awareness, and learning-based control that achieves richer awareness at the cost of offline training. We investigate a third option: a pre-trained LLM as a training-free runtime supervisor over a structured telemetry representation of search dynamics. Because telemetry is a shared interface, the identical state can drive an LLM, a rule-based, or a random policy, which makes the LLM’s contribution, as distinct from the interface, directly testable. Table 1 summarises these control paradigms.

Table 1. Control paradigms for ALNS-style search: The search-state signal each uses, where feasibility is enforced, and the associated reproducibility risk.

Control Paradigm	Search-State Signal	Feasibility Owner	Reproducibility Risk
Standard adaptive ALNS [1,2]	Scalar operator rewards	ALNS	Low
Bandit/adaptive operator selection [30,31]	Operator reward history with exploration	ALNS	Low
Rule-based hyper-heuristic [11]	Hand-specified state features and fixed rules	ALNS	Low
DRL-controlled ALNS [7,8,10]	Learned features of instance, solution, search	ALNS	Medium (policy transfer)
LLM as optimiser or heuristic generator [12,13,17]	Raw problem or solution, or generated code	LLM (at risk)	High (feasibility, nondeterminism)
This work: bounded telemetry-driven LLM supervisor	Structured multi-signal telemetry; solution withheld	ALNS	Controlled (fixed model, decoding, prompt, seeds; subject to served-model drift)

3. Problem Description

This section defines the optimisation problem (Section 3.1), casts its runtime control as a supervisory-control problem (Section 3.2), and states the hypotheses (Section 3.3). The former is the benchmark; the latter is the object of study.

3.1. The CVRPTW Optimisation Problem

We consider a multi-depot capacitated vehicle routing problem with time windows on a directed graph $G=(N,A)$. The node set $N=D\cup C$ comprises depots $D$ and customers $C$, and each arc $(i,j)\in A$ carries a travel distance $c_{ij}$ and a travel time $t_{ij}$. Costs are asymmetric, with $c_{ij}\ne c_{ji}$ in general. A homogeneous fleet $K$ of vehicles, each of capacity $Q$, serves the customers. Every vehicle is assigned to a depot, and each route starts and ends at that depot. Each customer $i\in C$ has a demand $q_i$, a service duration ${\sigma }_i$, and a time window $[a_i,b_i]$ within which service must begin, with waiting permitted before $a_i$. The number of depots is part of the instance data, and the supervision method is independent of it.

Let $x_{ijk}\in \left\{\mathrm{0,1}\right\}$ equal one if vehicle $k\in K$ traverses arc $(i,j)$, and let $T_{ik}\ge 0$ be the service-start time of vehicle $k$ at node $i$. The problem is

$$\min {\displaystyle \sum _{k\in K}}{\displaystyle \sum _{(i,j)\in A}}{c}_{ij} x_{ijk}$$

(1)

subject to

$${\displaystyle \sum _{k\in K}}{\displaystyle \sum _{j:(i,j)\in A}}{x}_{ijk}=1,\hspace{1em}\hspace{1em}\forall i\in C$$

(2)

$${\displaystyle \sum _{j:(i,j)\in A}}{x}_{ijk}-{\displaystyle \sum _{j:(j,i)\in A}}{x}_{jik}=0,\hspace{1em}\hspace{1em}\forall i\in N, \forall k\in K$$

(3)

$${\displaystyle \sum _{i\in D}}{\displaystyle \sum _{j:(i,j)\in A}}{x}_{ijk}\le 1,\hspace{1em}\hspace{1em}\forall k\in K$$

(4)

$${\displaystyle \sum _{i\in C}}{q}_i{\displaystyle \sum _{j:(i,j)\in A}}{x}_{ijk}\le Q,\hspace{1em}\hspace{1em}\forall k\in K$$

(5)

$$x_{ijk}=1 \Rightarrow T_{jk}\ge T_{ik}+{\sigma }_i+t_{ij},\hspace{1em}\hspace{1em}\forall (i,j)\in A, \forall k\in K$$

(6)

$$a_i\le T_{ik}\le b_i,\hspace{1em}\hspace{1em}\forall i\in C, \forall k\in K$$

(7)

Objective (1) minimises the total travel distance over all vehicles. Constraint (2) assigns every customer to exactly one vehicle. Constraint (3) enforces flow conservation, so each vehicle enters and leaves every node it visits. Constraint (4) restricts each vehicle to at most one route rooted at a single depot. Constraint (5) enforces the vehicle capacity limit. Constraint (6) propagates service-start times along traversed arcs, accounting for service duration and travel time, and is linearised with a big-M constant. Constraint (7) requires service at each customer to begin within its time window. Capacity and time windows are hard constraints, so every reported solution is feasible. The fleet size is fixed and identical across all compared methods, so comparisons are not confounded by fleet size.

3.2. The Supervisory-Control Problem

The object of study is the runtime control of the metaheuristic, cast as a discrete-time problem layered above ALNS. The search advances in segments of $L$ iterations; at each segment boundary $b$ a supervisor observes the search and may adjust its control parameters, while construction, repair, feasibility, and evaluation remain internal to ALNS. The problem is defined by four objects.

• Observation space  $\mathcal{O}$ . At boundary $b$ the supervisor receives an observation $o_b\in \mathcal{O}$, a structured telemetry vector that summarises the search state (Section 4.2). It excludes the solution: routes, coordinates, and the distance matrix are never exposed.
• Action space $\mathcal{A}$. An action $a_b\in \mathcal{A}$ is a bounded parameter-level adjustment, comprising a regime label and bounded changes to the destroy degree, operator emphasis, acceptance temperature and cooling, and diversification timing.
• Validator $V:\mathcal{A}\to \mathcal{A}$. A deterministic operator that projects any proposed action onto the admissible set, clipping out-of-range values and rejecting malformed ones, so that no supervisor output can drive ALNS into an invalid state.
• Supervisory policy $\pi :\mathcal{O}\to \mathcal{A}$. The controller maps an observation to an action, interchangeable across an LLM, a rule-based, and a random policy over identical observations and actions.

Writing $s_b$ for the incumbent solution and ${\theta }_b$ for the control-parameter vector, the closed loop is, for $b=\mathrm{1,2},\dots$,

$$s_{b+1}=\mathrm{A}\mathrm{L}\mathrm{N}\mathrm{S}(s_b,{\theta }_b),\hspace{1em}\hspace{1em}{\theta }_{b+1}=\mathrm{a}\mathrm{p}\mathrm{p}\mathrm{l}\mathrm{y}({\theta }_b, V(\pi (o_b))).$$

(8)

The supervisor therefore acts only in parameter space, while construction, evaluation, and feasibility remain internal to ALNS. This hyper-heuristic separation makes the supervisory policy, not any particular language model, the unit of study.

3.3. Research Hypotheses

Let $f(\cdot )$ denote final solution quality, measured as total travel distance under a fixed iteration budget. The paper tests two hypotheses.

• H1 (control quality). Under an equal budget, an LLM policy ${\pi }_{\mathrm{L}\mathrm{L}\mathrm{M}}$ that observes the telemetry $o_b$ and adjusts only ${\theta }_b$ attains better $f$ than reward-based adaptive ALNS and than rule-based and random policies operating over the same $\mathcal{O}$ and $\mathcal{A}$.
• H2 (source of improvement). Any observed improvement is associated with state-aware control rather than solution manipulation. Only the second part is testable by construction: the policy never edits the solution, and a rule-based policy with identical observations isolates the decision rule from the interface. This establishes the necessary condition that the supervisor does not alter the solution, not the sufficient condition that the telemetry causes the improvement; alternative mechanisms, such as pre-training priors or beneficial stochasticity from bounded perturbations, are not excluded by the design and are noted among the limitations in Section 6.

Improvement is assessed primarily by solution quality (gap to the best per-instance result) and secondarily by convergence speed and recovery from stagnation, operationalised in Section 5.

4. Methodology

This section presents the framework components and their interaction: the baseline ALNS (Section 4.1), the telemetry aggregator (Section 4.2), the LLM supervisor (Section 4.3), and the safety validator (Section 4.4). The components run as a closed loop over segments of L iterations, so control is exercised at fixed checkpoints rather than continuously. Running a full segment before each query keeps the search efficient, with one query per segment, gives the telemetry enough events to be statistically meaningful, and bounds inference cost. The final segment runs without a query, as there is no later segment to control. Figure 1 shows the architecture and the data circulating around the loop.

Figure 1. The supervised control loop connecting the ALNS solver, telemetry aggregator, LLM supervisor, and safety validator.

4.1. Baseline ALNS

The baseline is a standard ALNS on the open-source alns library [45] with an added segment-level control loop. Components follow established ALNS practice except for the telemetry aggregator (Section 4.2), the LLM supervisor (Section 4.3), and the safety validator (Section 4.4). All variants in Section 5.1 share the operator pool, acceptance criterion, stopping condition, objective, and seeds, so only the control policy differs.

Initial solution. A feasible start is built by greedy insertion, placing each customer at its cheapest feasible position across the available depot routes.

Destroy operators. The destroy pool comprises random, worst, and related (Shaw) removal, which removes customers similar in location and time [1], plus route, string, and time-window-cluster removal targeting temporally tight customers [46].

Repair operators. The repair pool comprises greedy insertion and regret-2 insertion, with a time-window-aware regret-3 insertion; the regret heuristics follow Ropke and Pisinger [1,46].

Local search. No separate local-search phase is used; improvement comes entirely from the destroy-and-repair cycle [1].

Adaptive operator weights. A roulette-wheel scheme updates operator weights from segment rewards [1]: the largest reward for a new global best, smaller for improving and accepted non-improving moves, none for rejected, with reaction-factor smoothing and credit assignment following Fialho et al. [3]. Because this layer is only modestly effective on its own [4], it is the limitation that the telemetry of Section 4.2 addresses.

Acceptance criterion. Candidates are accepted by simulated annealing (SA) [47], with an initial temperature of 5000 and geometric cooling of about 0.985, following the acceptance-criterion comparison of Santini et al. [5].

Stopping condition. Each run uses a fixed iteration budget.

Objective. The objective minimises total travel distance; capacity and time-window violations are penalised during the search to keep the neighbourhood connected, while reported solutions are feasible.

Random seeds. All stochastic choices draw from a single seeded generator; each configuration runs from several seeds, with the seed recorded per run.

4.2. Telemetry Aggregator

The telemetry aggregator is the central contribution of this framework: it defines how the state of the running search is represented to the supervisor. At each segment boundary, that is, every $L$ ALNS iteration, it maps the raw per-iteration log to a fixed-length, mathematically defined observation $o_b\in \mathcal{O}$ organised into seven blocks. The blocks answer, in order, four questions any runtime controller must resolve: whether the search is still progressing, why it is or is not, whether an apparent stall is real, and how much budget and constraint headroom remain. The solution itself, the routes, coordinates, and distance matrix, is never part of $o_b$. Table 2 names the blocks and relates each to the state descriptors used in adaptive operator selection and reinforcement-learning control; Appendix A gives the full definition of every indicator.

Table 2. The seven telemetry blocks, the search phenomenon each captures, and its analogue in adaptive and reinforcement-learning search control.

Block	Telemetry Block	Search Phenomenon Captured	Analogue in Adaptive/RL Control
1	Search dynamics	improvement velocity, stagnation, acceptance behaviour	return/reward signal and episode progress
2	Operator effectiveness	per-operator usage and productivity, selection entropy	action-value estimates and exploration spread
3	Destroy/repair calibration	yield and repair feasibility by destroy size	action-effect and transition information
4	Search movement	structural change, revisits, neutrality	state-novelty and exploration signals
5	Constraint stress	capacity and time-window slack, infeasible-repair rate	constraint and cost-shaping signals
6	Run context	run phase, budget consumed, fleet utilisation	time-to-horizon features
7	Memory and governance	realised effect of past actions, validator activity	credit assignment and off-policy correction

Figure 2 illustrates this transformation: a log of thousands of per-iteration events, impractical to transmit at every checkpoint and ambiguous for an LLM to parse, becomes a stable, low-dimensional, reproducible view of the search state. The blocks are described below, with names matching the implemented packet.

Figure 2. The telemetry aggregator converts the raw per-iteration log into a compact seven-block packet. Each block summarises a distinct diagnostic aspect of the search before the packet is provided to the LLM supervisor.

Search dynamics (Block 1).

This block reports whether the search is still improving and how acceptance is set: current and best objective, improvement velocity over a short and a longer window, iterations since the last best, and the acceptance rate. The two velocities are the core signal, both near zero meaning a stall, and a near-zero short with a still-positive longer reading a recent slowdown, while the acceptance rate shows whether the simulated-annealing temperature is too strict or too loose. Block 1 is the primary trigger: it flags that attention is needed but not what, and the other blocks supply the diagnosis.

Operator effectiveness (Block 2).

Standard ALNS shifts weight toward recently successful operators, but the weights do not reveal when the pool is unhealthy. This block reports, per operator, how often it is chosen and how often it yields an improvement or accepted move, the evenness of selection across the pool, and a counter flagging operators ignored for several segments. When Block 1 shows stagnation, Block 2 distinguishes over-commitment to a few operators from a starving productive operator; the remedy, rebalancing operator emphasis, differs from changing how aggressively the search perturbs the solution.

Destroy/repair calibration (Block 3).

The destroy degree, the number of customers removed and reinserted per step, balances local refinement against diversification: small removals are safe and local, large removals explore more but are harder to repair feasibly. This block reports how well each operator performs at different removal sizes, in improvement and in feasible-and-accepted rate. Its role is protective: if large removals are already failing to repair, increasing the degree in response to stagnation would make the search worse, and Block 3 lets the supervisor see this before acting.

Search movement (Block 4).

A flat objective curve is ambiguous: the search may be crossing a wide region of similar-cost solutions, which is healthy, or circling the same few structures, which is not. This block separates them by measuring how much route structure changes and how often the search revisits solutions already seen; great structural change with little cost change indicates productive plateau movement, frequent revisits with little structural change indicate cycling. This refines Block 1: the same near-zero progress reading calls for patience in the first case and aggressive diversification, or a restart, in the second.

Constraint stress (Block 5).

In CVRPTW, a solution can look good on distance yet sit on the edge of infeasibility. This block measures that pressure: how close routes are to capacity, how much slack remains before time windows are violated, and whether recent repairs fail mainly because constraints cannot be met. Since improving the objective and improving feasibility are different goals, if repairs keep returning infeasible, the supervisor should ease pressure on tightly constrained solutions rather than explore harder. Block 5 keeps the decision honest about why the search is failing.

Run context (Block 6).

The same signals should not always trigger the same action: aggressive diversification is sensible early, when budget can recover, but risky late, when good structure should be protected. This block makes the supervisor budget-aware by reporting the phase, early, middle, or late, of the iteration budget, with basic fleet context; it conditions every other reading, so a late stagnation signal is treated more cautiously than the same signal early on.

Memory and governance (Block 7).

The final block closes the loop. After each intervention, it records whether the next segment improved, stayed flat, or worsened, the LLM’s stated confidence, and whether the validator scaled or rejected the action, with a counter for consecutive no-effect interventions. This gives the supervisor a short memory of its own track record and gives the validator evidence to discount low-confidence or repeatedly ineffective recommendations. Block 7 is the governance layer, holding the LLM accountable to recent outcomes.

Together, the seven blocks give a compact, consistent picture: whether progress has stalled, why, whether the stall is real, how much feasibility and budget headroom remain, and whether the last action helped. Every entry is an aggregated number or categorical flag.

4.3. LLM Supervisor

The supervisor is an LLM consulted once per segment; it instantiates the supervisory policy $\pi$ of Section 3.2, mapping the telemetry observation $o_b$ to a bounded action $a_b$, and nothing else in the framework depends on the policy being an LLM rather than another controller. Its inputs, outputs, and settings are fixed for all runs so that the layer is reproducible, since the output of an LLM is sensitive to the LLM, the decoding settings, and the prompt [48]. The structure of the supervisor is shown in Figure 3.

Figure 3. The LLM supervisor: the seven-block telemetry packet is merged into the fixed system prompt and sent to the LLM, which analyses it and returns a JSON action that the safety validator then checks.

Prompt structure. A fixed system prompt carries the supervisor role (it never edits the solution), every telemetry field’s definition, the permitted actions with their bounds, and the output schema; the seven-block telemetry is serialised as one compact JSON object and appended as the user message. State is passed as structured JSON, so the LLM reads named fields and its reply parses deterministically.

Prompt contents and interpretation rules. Beyond role and schema, the prompt fixes the action bounds, for example, operator bias in [−0.15, +0.15], a destroy degree clamped to [1, x · n] with a destroy-fraction cap (x = 0.2, chosen by preliminary trial), and a bounded acceptance-temperature multiplier. It gives eight interpretation rules covering improvement velocity, the acceptance threshold, the neutrality-ratio band, cycling versus revisiting, phase acceptance floors, the governance counters, q-stagnation diagnostics, and accepted-drift detection. Each maps a telemetry pattern to one of the same intensify, diversify, or rotate, and escape regimes as the rule-based comparator, so the two share a control vocabulary and differ only in how flexibly they apply it. The interpretation rules provide domain-specific guidance rather than a complete deterministic action policy; the LLM remains responsible for integrating the telemetry blocks and selecting a context-dependent joint control action. The full prompt is in Appendix B.

Action schema and output. The supervisor returns strict JSON: a regime label and bounded changes to the destroy degree, operator emphasis, acceptance temperature and cooling, and the timing of diversification or restart, plus a self-reported confidence in [0, 1]. No free text, routes, or customer sequences are permitted.

Model and decoding. The supervisor is OpenAI’s open-weight gpt-oss-120b model, served through Ollama Cloud and used consistently in every run. The model was invoked using the Ollama identifier gpt-oss:120b, corresponding to the fixed artefact digest a951a23b46a1 and an artefact size of 65 GB. It uses a mixture-of-experts architecture with 117 B total parameters, 5.1 B active parameters per token, and native MXFP4 quantization. Decoding was fixed at a temperature of 0.1 with nucleus sampling, a 40,000-token generation limit, and a 60 s timeout. The fixed artefact digest identifies the exact model weights used across the experiments. Although stochastic decoding may prevent bit-for-bit reproduction of individual responses, the released prompt, telemetry schema, inference settings, validator rules, random seeds, and logged supervisory decisions support reproduction and auditing of the experimental procedure.

Frequency and retry. The supervisor is consulted once per segment of L iterations, and the final segment is not sent. If the output is invalid JSON or violates the schema, it is queried once more with the same packet; if it still fails, the validator applies the fallback no-change action.

4.4. Safety Validator

The validator is a deterministic layer between the supervisor and ALNS that guarantees bounded interventions and keeps feasibility owned by ALNS. It is necessary because an LLM’s output is non-deterministic and, without guards, can produce constraint-violating actions [17,48]. A deterministic check ensures no model output can place the solver in an invalid state, preserving the hyper-heuristic separation of high-level control from low-level modification [11]. Figure 4 shows the pipeline.

Figure 4. The safety validator: Every action is parsed, clipped to normalised bounds, and confidence-gated before it can affect ALNS; the solution is never modified.

Every action passes deterministic checks. It must parse and match the JSON schema, or the no-change fallback is used. Only permitted fields are kept. Each value is clipped to its normalised range: the destroy size q is clipped to [1, x · n] at x = 0.2, so q/|C| never exceeds 0.20, and the acceptance-temperature multiplier is clipped to [0.75, 1.35]. Per-segment change is limited: operator weights are normalised to sum to one, each per-operator bias is clipped to ±0.25, and the total absolute change is capped at 0.30 for destroy and 0.25 for repair. Any field altering the solution is rejected. Finally, the action is confidence-gated: it is applied in full above 0.60, at half strength between 0.40 and 0.59, and ignored below 0.40, with weights reverting toward uniform on any failure. The self-reported confidence is used solely as an uncalibrated gating signal to attenuate or suppress actions; it is not interpreted as a calibrated probability of improvement or as a predictor of decision quality. The gate is deliberately one-sided: low confidence only attenuates or suppresses an action and never amplifies it beyond the bounds, so miscalibration can make the supervisor more cautious but never larger or unsafe.

5. Experimental Results

5.1. Experimental Design

Benchmark instances are built on the real road networks of three cities, Genoa, New York, and Valencia. Road graphs are obtained with OSMnxOSMNX 2.1.0 [49], and arc travel times and distances are computed as shortest paths over the street network with NetworkX using posted speed limits. The resulting costs are asymmetric and non-Euclidean, which reflects operational conditions more faithfully than Euclidean or randomly generated benchmarks and weakens the geometric regularities that destroy and repair operators often exploit, making the test bed more demanding [50]. For each city, we generate instances with 50, 100, and 150 customers, giving nine instances. Their parameters are summarised in Table 3. Although nine instances is a modest count, the design prioritises controlled comparison over benchmark breadth: the three cities span distinct street-network topologies, the three sizes (50, 100, 150 customers) cover small to medium CVRPTW, and every solver runs on the identical instances and seeds, so each comparison is paired and instance difficulty is held constant. The evaluation is therefore aimed at detecting consistent within-instance differences between control policies, which is the object of study, rather than at establishing absolute performance across a broad suite. Wider external validity across more networks and sizes is left to future work. The multi-depot configuration is deliberate: assigning routes across several depots increases spatial and constraint interaction and yields a more demanding control problem than a single-depot setting.

Table 3. CVRPTW instance parameters.

Parameter	Value or Setting
Cities	Genoa, New York, Valencia
Customers per instance	50, 100, 150
Depots	3; each route starts and ends at its depot
Fleet	Homogeneous; vehicle capacity $Q=200$
Customer demand	Integer, U(1, 10)
Service time	U(2, 10) minutes
Working day	08:00 to 16:00 (minutes from midnight, [480, 960])
Time-window classes	Endpoints uniform within the class range: early U(480, 600), medium U(480, 720), free U(480, 960)
Travel time and distance	Shortest paths on the road network; speed from street limits
Objective	Minimise total travel distance

Every method is run on every instance from four random seeds (42, 123, 456, and 789), and statistics are aggregated across seeds. Each run uses a budget of 1000 iterations, and the LLM supervisor is consulted every $L=50$ iterations, giving 19 decision checkpoints per run. The compared methods are as follows:

• Standalone ALNS with fixed equal operator weights and no adaptation.
• ALNS with standard adaptive operator weighting [1].
• Rule-based supervised ALNS: A deterministic policy over the same telemetry and bounded action space. It intensifies while the incumbent improves, diversifies after one flat segment, and escapes after several consecutive flat segments (large destroy degree near the cap, reheating, restart from the best), nudging the acceptance temperature when the acceptance rate is too low or too high. Both policies act over the same intensify/diversify/escape vocabulary and the identical telemetry and action bounds, the rule-based one applying fixed thresholds and the LLM the same regimes with context-dependent conditioning, so the comparison isolates how the decision is made rather than which control actions are available.
• Random supervised ALNS, sampling actions uniformly within the same bounds.
• LLM-supervised ALNS, the proposed method.
• External reference solvers PyVRP 0.013.4 [51] and Google OR-Tools 9.15 [52], reported to situate the ALNS family rather than as the primary comparison.

In total, solvers run on nine instances with four seeds, giving 252 runs, of which the 36 LLM-ALNS runs yield 684 logged supervisory decisions (19 checkpoints per run). All reported solutions are feasible. Because distances differ across instance sizes, quality is summarised by the gap to the lowest mean distance on each instance, where the mean is taken over the four seeded runs, the mean per-instance rank among the seven solvers (1 = best to 7, then averaged), and the number of per-instance wins. The evaluation follows the hypotheses of Section 3.3: solution quality tests H1 (Section 5.3 and Section 5.4) and supervisor behaviour tests H2 (Section 5.5 and Section 5.6).

5.2. Metrics

The final objective value alone is not sufficient for a rigorous evaluation. Search behaviour and intervention quality are therefore reported as first-class results. Table 4 lists the reported metrics, their definitions, and the reason each is included.

Table 4. Evaluation metrics.

Metric	Definition	Purpose
Best objective and distance	Final incumbent cost and total travel distance of each run	Primary solution-quality measure
Gap to per-instance best (%)	Relative distance above the lowest mean distance, over four seeds, achieved by any solver on that instance	Within-study ranking; optimal solutions exist for these instances but are not known
Feasibility rate	Share of runs whose reported solution satisfies all hard constraints	Confirms supervision never compromises feasibility
Convergence and time-to-best	Iteration at which the best solution is found and the longest non-improving stretch	Anytime behaviour under a fixed budget
Computational overhead	Runtime per run, number of LLM calls, and valid-versus-invalid output rate	Cost of supervision
Intervention profile	Distribution of intervention types and their measured effect across search states	Links decisions to search states; evidence for H2

These metrics follow established practice for routing metaheuristics. Solution quality and the gap to a reference are the standard basis for VRP comparison [25,53]. Convergence speed and time-to-best capture anytime behaviour, which is shaped by the controls the supervisor adjusts [5]. The four seeded replications per instance guard against single-run conclusions, as emphasised by meta-analyses of stochastic search [4], with seed ranges shown in Figure 5 and Figure 6. The supervisor-specific metrics are reported because the final objective alone cannot show whether supervision, rather than chance, drives any change. The rule-based and random baselines are held to the same standard.

Figure 5. Objective-gap distribution across the 36 runs of each solver (nine instances × four seeds; diamond = mean; lower is better). Gaps are measured to the lowest per-instance mean distance over the four seeds. For the seed-deterministic reference OR-Tools, the spread reflects differences across the nine instances, not across seeds.

Figure 6. Convergence on the nine instances (rows = cities, columns = sizes): mean best objective over four seeds (line) with the seed range (shaded); ALNS family only.

5.3. Overall Solution Quality

Table 5 reports mean performance; lower gap, rank, and runtime are better, and wins count the instances, nine at most, on which a solver attains the best mean distance over the four seeds. The external solvers behave as expected: PyVRP is very strong and far faster (0.15% gap, rank 1.89), and OR-Tools reaches a 2.14% gap. Both are treated as competitors and are included in the omnibus and pairwise tests of this subsection. Within the ALNS family, the fixed-weight (2.86%) and adaptive (2.93%) variants trail the rule-based (1.39%) and random (1.57%) supervisors. LLM-ALNS attains the lowest ALNS-family mean gap (0.18%) and the best mean per-instance rank (1.56), with five wins, ranking first on five of the nine instances, second on three, and third on one (Figure 5 and Figure 7). Because the reference is the best result within the tested pool, and the strong external solver PyVRP anchors that pool, these gaps quantify relative standing among the compared solvers rather than proximity to a proven optimum. The random supervisor uses the identical seven-block packet and action bounds but does not condition its choices on the packet’s content, so it functions as a telemetry-blind control over the same interface. The margin between it and the two telemetry-reading policies, LLM-ALNS and the rule-based supervisor, therefore isolates the value of conditioning decisions on the telemetry content. The fact that random already improves on the adaptive and fixed-weight variants indicates that part of the gain comes from the bounded action space itself. As an external anchor, PyVRP is a peer-reviewed state-of-the-art CVRPTW solver, so the 0.03 percentage-point difference between LLM-ALNS (0.18%) and PyVRP (0.15%) places the supervised search close to a strong dedicated baseline.

Table 5. Mean solver performance across the nine instances and four seeds. Bold marks the best value in each numeric column (lowest gap, lowest rank, most wins, shortest runtime).

Solver	Type	Mean Gap (%)	Mean Rank	Wins	Runtime (s)
PyVRP	External reference	0.15	1.89	4	0.8
OR-Tools	External reference	2.14	4.11	0	42
ALNS (fixed-weight)	Ordinary ALNS	2.86	5.89	0	244
ALNS (adaptive)	Ordinary ALNS	2.93	6.33	0	238
ALNS (rule-based)	Ordinary ALNS	1.39	3.89	0	528
ALNS (random)	Ordinary ALNS	1.57	4.33	0	551
LLM-ALNS	LLM-supervised	0.18	1.56	5	979

Figure 7. Solver rank on each of the nine instances (1 = best).

These differences are statistically supported. Statistical tests are conducted on per-instance means rather than on the 36 individual runs, because seeds are replicates within an instance rather than independent experimental units. Testing the per-instance means avoids pseudo-replication and keeps every comparison paired at the level of the instance. A Friedman test over the nine instances rejects the null of equal performance across the seven solvers (${\chi }^2=37.8$, $p<0.001$). Pairwise Wilcoxon signed-rank tests, with Holm–Bonferroni correction for the six simultaneous comparisons, show that LLM-ALNS 7.0.0 significantly improves on every ALNS variant, namely adaptive, fixed-weight, rule-based, and random ($p=0.004$ in each case, the minimum attainable for nine paired instances, since LLM-ALNS wins on all nine, so all four survive the correction). Against OR-Tools, the uncorrected value ($p=0.027$) does not survive the correction, so this difference is not statistically significant after adjustment. The difference from PyVRP is likewise not significant ($p=1.0$). LLM-ALNS therefore significantly outperforms the four conventional ALNS variants, whereas its differences from OR-Tools and PyVRP are not statistically significant after correction. The non-significant PyVRP result should not be read as evidence of equivalence, since a test over nine instances, even with four seeded replications per instance, has limited power to establish it, and a formal equivalence claim would require a two-one-sided-test analysis with a pre-specified margin.

5.4. Convergence and Runtime

All solvers make most of their progress early. LLM-ALNS reaches its best solution at about 66% of the budget, and its longest stretch without improvement is 446 iterations, below adaptive (540), fixed-weight (516), and random (543), though the rule-based variant is shorter still at 411 (Figure 6). Supervision costs runtime (Figure 8): LLM-ALNS averages about 979 s per run against 238 s for adaptive ALNS and 0.8 s for PyVRP, with about 17 completed model calls per run. The remaining checkpoints fall back to the deterministic no-change action, for example, after a decoding timeout, and every checkpoint is logged as a decision regardless, which is why 19 logged decisions per run coexist with about 17 completed calls. Most of the overhead is telemetry instrumentation rather than the LLM calls, which account for roughly 13% of the wall-clock time.

Figure 8. Mean runtime per run (log scale) versus average gap to the per-instance best; lower-left is better.

Figure 8. Mean runtime per run (log scale) versus average gap to the per-instance best; lower-left is better.

5.5. Supervisor Behaviour

This subsection characterises what the supervisor does across the 684 logged decisions.

Action vocabulary. The bounded action space is used in a diversification-led manner: diversify, escape, and explore together account for 450 of the 684 decisions (65.8%) and intensification for 234 (34.2%), so the supervisor defaults to neither pole (Figure 9).

Figure 9. Bounded action vocabulary selected by the supervisor.

Telemetry coverage. The blocks contribute unevenly by design: Persistent blocks fire on almost every decision, operator effectiveness (B2) at 95.9% and constraint stress (B5) at 83.6%, with search dynamics (B1) at 68.6% and search movement (B4) at 56.1%, while event-driven blocks fire less often, memory and governance (B7) at 32.9%, destroy and repair calibration (B3) at 27.3%, and run context (B6) at 16.4% (Figure 10).

Figure 10. Decisions in which each telemetry block surfaced an actionable signal.

Situation-conditioned regimes. The chosen regime tracks the diagnosed search situation (Table 6): Healthy states retain substantial intensification (diversification-family share 52.1%), plateau and stagnation shift the supervisor toward escape and diversification (70.7% and 70.4%), cycling is handled mainly by diversification (83.9%), and frozen simulated-annealing states are treated as full diversification territory (100%).

Table 6. Action regimes conditioned on the diagnosed search situation. Bold marks the most frequently selected regime for each situation.

Situation	Intensify	Explore	Escape	Diversify	Diversification-Family (%)
Healthy	112	29	9	84	52.1
Plateau	66	9	122	28	70.7
Stagnation	47	0	47	65	70.4
Cycling	9	0	9	38	83.9
Frozen SA	0	0	0	10	100.0

Operator-bias policy. A supervisory action also reshapes the operator-selection distribution rather than only emitting a label. The most frequent biases boost structure-aware operators, regret-$k$ insertion (525 decisions) and Shaw removal (478), while cutting greedier or less productive choices, greedy insertion (384) and worst removal (468).

5.6. Decision Quality

This subsection tests H2: that the seven-block telemetry, not the mere presence of an LLM, shapes the supervisor’s decisions across the 684 checkpoints. Each block reports a state, the model maps that state to a bounded ALNS control action, and the deterministic engine keeps feasibility, construction, and objective evaluation. We therefore ask whether the supervisor changes its action when a signal becomes active, whether it then selects the matching lever, and what the immediate effect is. All evidence is reported as percentages and percentage-point (pp) shifts, and a shift is read as a decision-conditioning effect, not as causal proof. In the absence of a controlled interface ablation, these per-block conditioning effects are the primary evidence that each block contributes distinct, actionable information to the supervisor; a degraded-interface ablation that isolates the causal contribution of individual blocks remains future work.

Table 7 reports, for each block, how often the mapped control action is selected without and with that block’s signal. The difference is the decision-conditioning effect. Every block moves the action distribution in the expected direction, and the shift is largest for the most decisive levers: when destroy/repair telemetry (B3) reports that large removals have stopped improving, q-reduction rises from 35% to 93% (+58 pp). When search movement (B4) detects cycling, diversification or operator rotation rises from 42% to 80% (+38 pp). When search dynamics (B1) reports a stagnant or frozen search, relax, reheat, or escape rises from 39% to 71% (+32 pp). The remaining blocks shift from already high baselines because their responses are partly active by default (B6 +16 pp, B2 +10 pp, B7 +10 pp, B5 +7 pp). B5 and B6 are feasibility-protection and phase-control blocks whose value is contextual control rather than a single mechanical switch. Because each mapped action is the one the prompt’s interpretation rules prescribe for the corresponding signal, Table 7 is best read as a fidelity, or prompt-compliance, audit of the interface: it confirms that the supervisor reliably operationalises the encoded rules over the telemetry, not that it independently infers these mappings.

Table 7. Telemetry-conditioned decision shift: the rate at which the supervisor selects each block’s mapped control action without and with that block’s actionable signal (in percentage points). Computed over the 684 logged LLM-ALNS decisions (36 runs). Bold marks the decision-conditioning shifts of at least 30 percentage points.

Block	Actionable Signal	Mapped Control Action	Without Signal (%)	With Signal (%)	Shift (pp)
B1 Search dynamics	stagnation or frozen search	relax/reheat/escape	39	71	+32
B2 Operator effectiveness	operator imbalance	weight recalibration	84	94	+10
B3 Destroy/repair calibration	high q without yield	reduce q	35	93	+58
B4 Search movement	cycling detected	diversify/rotate	42	80	+38
B5 Constraint stress	feasibility pressure	repair/cap-q protection	92	99	+7
B6 Run context	phase and budget	phase control	54	70	+16
B7 Memory and governance	recent neutral plateau	break-out/reset/reheat	86	96	+10

Beyond changing the action distribution, signal-to-control fidelity asks whether, when a signal is active, the supervisor usually selects the corresponding lever. The high-leverage signals show near-complete fidelity: cycling leads to diversification or rotation, a neutral plateau to escape or reset, operator imbalance to operator movement, and constraint stress to repair-safe protection. This is the practical meaning of a useful telemetry interface: it does not merely describe the search; it directs the supervisor to the correct bounded control lever.

The decisions are also state-appropriate (Table 8). A decision counts as matching when the selected action is the response the interpretation rules prescribe for the diagnosed state: a diversification-family action for plateau, stagnation, and cycling; intensification for healthy states; and reheating or continued refinement for frozen states. Late-phase intensification without increasing the destroy size also counts as prescribed consolidation, which is why plateau and cycling reach 100% even though Table 6 records intensification decisions under both states. Under this criterion, the match rate is 100% for plateau and cycling and 70.5% in healthy states. The share of decisions improving within the next segment is high in healthy and frozen states (72.2% and 70.0%) and lower under plateau and cycling (28.4% and 32.1%), as expected, since diversification and escape position the search for later gains rather than immediate ones.

Table 8. State-appropriate supervisory decisions: the share of decisions matching the diagnosed state, the share improving within the next segment, and the preferred control logic for each state. Shares are computed over the per-state decision counts of Table 6 (Healthy 234, Frozen SA 10, Stagnation 159, Plateau 225, Cycling 56; 684 in total). Bold marks the highest matching share. The improvement column is left unmarked, since a lower immediate improvement share under plateau and cycling reflects the deferred payoff of diversification rather than weaker decisions.

Telemetry-Reported State	Decisions Matching the State (%)	Improved Within Next Segment (%)	Preferred Control Logic
Healthy	70.5	72.2	productive intensification
Frozen SA	60.0	70.0	reheat or relax a cold trajectory
Stagnation	54.1	17.0	reduce local exploitation
Plateau	100.0	28.4	escape or reset a neutral plateau
Cycling	100.0	32.1	diversify out of repeated basins

The temperature lever is the exception. When a reheat is specifically recommended, the supervisor follows it with an actual reheat in only about 30% of cases, preferring relax or escape, so the acceptance channel is the least reliably controlled. This is a genuine controller weakness and the clearest target for the next revision, either as a stronger prompt constraint or as a deterministic reheat rule inside the validator under frozen simulated-annealing conditions.

Immediate objective improvement is not the only valid outcome, since diversification and escape have deferred rather than immediate payoff: across the logged decisions, 17.8% improved the incumbent immediately, 69.9% left it unchanged, and 12.3% worsened it, a distribution consistent with a positioning role rather than greedy improvement. These rates are reported for the LLM supervisor alone; establishing supervisory effectiveness would also require the corresponding post-decision improvement rates for the random and rule-based supervisors and for unsupervised ALNS, which we identify as part of the extended analysis in Section 6.

Every decision is also safe under the validator: no audited action is hard-invalid, all parse and respect the bounded ranges of Section 4.4, and every reported solution is feasible. The cost is canonicalisation: 72.7% of actions require some soft normalisation, most often clipping the requested destroy fraction to its cap, a reproducibility consideration rather than a solver failure. The high rate, dominated by clipping the requested destroy fraction to its cap, reflects the raw model frequently proposing actions at or beyond a bound rather than structurally invalid ones; the validator’s clipping keeps them admissible, and the rate is reported for transparency about how often the bounds bind.

Taken together, the audit supports H2 in the form the evidence allows: the seven-block telemetry gives the supervisor a compact but actionable view of the search, and when high-leverage signals become active, the supervisor changes its bounded control actions by clear percentage-point margins and with high signal-to-control fidelity. The improvement is associated with state-aware decisions over a representation the supervisor reasons about, not with manipulation of a solution it never accesses. The central contribution is therefore not a universal final-distance win over every ALNS variant, but state-aware, telemetry-conditioned control that the reward-based and fixed-weight baselines cannot produce. One caveat qualifies this attribution: the rule-based comparator applies the shared intensify/diversify/escape regimes with fixed thresholds, whereas the LLM applies the same regimes with the finer, context-dependent conditioning of its eight interpretation rules, so the measured margin is an upper bound on the contribution of that conditioning relative to an equally well-specified fixed-threshold controller. Read across Table 7 and Table 8, the block-by-block decision shifts and the state-appropriate response rates give observational, not experimental, evidence that the individual telemetry blocks carry distinct and usable signal; they show that the supervisor’s actions track each block’s content, though they do not, on their own, isolate the causal contribution of any single block.

6. Conclusions

This study investigated whether a structured telemetry interface can enable a large language model to supervise ALNS without accessing or modifying the underlying solution. The proposed interface summarises the evolving search through seven telemetry blocks, while a deterministic validator restricts the LLM to bounded parameter-level actions and preserves feasibility within the ALNS solver.

The framework was evaluated on nine multi-depot CVRPTW instances derived from real road networks, using four random seeds and a fixed budget of 1000 iterations. LLM-ALNS achieved the lowest mean gap among the ALNS variants, reducing it from 2.93% for adaptive ALNS to 0.18%, and significantly outperformed the fixed-weight, adaptive, rule-based, and random ALNS variants. Its difference from PyVRP was not statistically significant, although PyVRP remained substantially faster. All reported solutions were feasible, and all supervisory actions remained within the validator’s admissible bounds. The decision analysis further showed that the supervisor changed its control actions consistently with the telemetry signals, supporting the usefulness of the interface for state-aware runtime control.

These findings are limited to the evaluated short-budget setting and should not be interpreted as evidence of general superiority over dedicated CVRPTW solvers or the broader ALNS literature. The study also used one LLM, a fixed telemetry configuration, and no individual-block ablation. By construction, the supervisory role is model-agnostic: any model that returns schema-conformant, bounded actions from the same telemetry could occupy it, but this cross-model generalisability has not been empirically tested here. Future work should therefore examine longer iteration budgets, alternative models, threshold sensitivity, and degraded or single-block telemetry configurations. Within the evaluated setting, the results demonstrate that structured telemetry can provide a practical and reproducible interface for bounded LLM supervision of metaheuristic search.