A Learning-Enhanced Metaheuristic Algorithm for Multi-Zone Orienteering Problem with Time Windows

Abstract

Inspired by real-world logistics scenarios, in this paper, we introduce a new variant of the Orienteering Problem known as the Multi-zone Orienteering Problem with Time Windows (MzOPTW). In the MzOPTW, customers are situated in distinct zones, each with multiple entrances and exits. Each customer has specific time window requirements; access to them will generate certain profits. This problem is to simultaneously determine which zones and customers to visit, select the zonal entrances and exits, and generate the routes for visiting each zone and its customers, all while maximizing total profits within a limited time frame. To tackle the MzOPTW, this paper develops an integer programming model. There are significant computational challenges in the strong interdependencies among zone selection, customer selection within zones, entrance and exit selection for each zone, the sequence of visits to zones and customers, and arrival and stay times. To address these challenges, this paper proposes a learning-enhanced metaheuristic algorithm called the Hybrid Ant Colony Optimization (HACO) algorithm, which incorporates Pointer Network learning. The HACO algorithm combines the global search capabilities of a population-based algorithm with the parallel decision-making abilities of the Pointer Network learning model. Additionally, a method to optimize zonal stay time limits is proposed to further enhance the solution. Experimental results demonstrate that the HACO algorithm outperforms comparative algorithms, achieving better solutions in 73% of the instances within the same time frame. Furthermore, the proposed optimization method for zonal stay time limits results in improvements in 78% of instances.

1. Introduction

The Orienteering Problem (OP) originated from orienteering competitions, which are typically held in densely forested areas. Participants start from a designated origin node and, using a compass and map, visit as many nodes as possible before reaching a specified destination. Each node has a certain reward and can only be visited once. The objective is to maximize the total reward within a limited time frame [1]. Over the past few decades, the OP has attracted significant attention. Researchers such as Vansteenwegen et al. [2], Gunawan et al. [3], and Kant and Mishra [4] have conducted in-depth studies on the problem and its various variants, solution algorithms, and practical applications. Moreover, the OP can also be modeled as the Maximum Collection Problem [5], the Traveling Salesman Problem With Profits [6], and the Tourist Trip Design Problem [7], among others.

In many practical scenarios of the OP, the visited nodes exhibit clustering characteristics, such as groups of tourist attractions with similar attributes in the Tourist Trip Design Problem or multiple customers within the same community in logistics delivery. Based on the clustering characteristics of the visited nodes, Angelelli et al. [8] introduced the Clustered Orienteering Problem (COP), where each customer belongs to one or more clusters, and the reward for a cluster is obtained only by serving all customers within that cluster. The objective is to maximize the total reward within the given time frame. Similarly, Archetti et al. [9] proposed the Set Orienteering Problem (SOP), where customers are assigned to clusters, each of which contains only one cluster, and the reward for a cluster is obtained only when at least one customer from the cluster is served. The optimization objective is similar to that of the COP. These problems consider the instance with a single vehicle, while He et al. [10] and Nguyen et al. [11] studied the COP and SOP with multiple vehicles, respectively. In the study of the Traveling Salesman Problem, some scholars have also considered the distribution of clusters. For instance, Laporte et al. [12] and Khan et al. [13] studied the Generalized Traveling Salesman Problem (GTSP), where the objective is to find the minimum-cost Hamiltonian circuit to visit clusters of customers. Similarly, Bernardino and Paias [14], and Chaves et al. [15] investigated the Family Traveling Salesman Problem (FTSP), aiming to visit a predefined number of customers in each cluster at the minimum cost.

However, the research on OP with node clustering characteristics has the following limitations when applied to certain practical scenarios: (1) Lack of consideration of the entry and exit nodes of zones. In real-world scenarios, such as last-mile logistics delivery to community customers, communities (or residential areas) typically have multiple entry and exit nodes. Delivery personnel must select an appropriate entry node to access the zone and serve the customers within the community. For larger zones, different entry-node decisions can affect the travel time to customers, thereby influencing the sequence of customer visits, zone access, and total profits. (2) Inflexibility in the number of customers visited. For example, in the COP, it is assumed that all customers within a cluster must be visited, while in the SOP, only one customer from each cluster is required to be visited. This lack of flexibility in optimizing the number of customers visited within a zone affects the total profits. The characteristics of MzOPTW and related problems are compared in

Table 1. Characteristics of MzOPTW and its related problems.

Problem	Multiple Zones	Time Windows	Entry and Exit Nodes of Zones	Customer Flexibility	Algorithm
COP [8]	√			All customers per zone	Heuristic
SOP [9]	√			Only one customer per zone	Matheuristic
GTSP [13]	√			Only one customer per zone	Heuristic
FTSP [14,15]	√			Predefined customers per zone	Branch-and-cut
MzOPTW	√	√	√	Any customers per zone	Learning-enhanced metaheuristic algorithm

√ Multiple Zones: The problem involves multiple zones; Time Windows: Customers have specific time windows; Entry and Exit Nodes of Zones: Each zone has multiple entry and exit nodes.

.

To better align with practical application scenarios, this study introduces a new class of OP—the Multi-zone Orienteering Problem with Time Windows (MzOPTW). Compared to existing research, this problem incorporates the decision-making process for the entry and exit nodes of zones, while relaxing the constraints that all customers within a zone must be visited or only one customer needs to be visited.

As the Orienteering Problem (OP) is a well-known NP-Hard problem [1], the proposed MzOPTW is also NP-Hard and even more complex, as it requires simultaneous optimization of intra-zone and inter-zone routing. These two layers interact dynamically—changes in zone entry and exit nodes, arrival times, and stay times can significantly affect the overall route, making exact algorithms computationally intractable. Heuristic approaches offer scalability but face trade-offs: local search algorithms often suffer from limited exploration, while global metaheuristics such as Ant Colony Optimization (ACO) may require long computational times. Recent advances in deep learning, particularly Pointer Networks (Ptr-Nets) [16], enable efficient and parallelizable solutions to combinatorial optimization problems [17]. To leverage the strengths of both paradigms, this study proposes a Hybrid Ant Colony Optimization (HACO) algorithm that integrates Pointer Networks to enhance solution efficiency and quality. Empirical results demonstrate that HACO algorithm achieves notable improvements in stability, scalability, and overall performance.

This paper makes the following four key contributions:

• Problem Formulation: We introduce the Multi-zone Orienteering Problem with Time Windows (MzOPTW), a novel variant of OP that explicitly incorporates zone-level entry and exit node decisions, aligning with real-world logistics and tourism scenarios.
• Model Flexibility: The MzOPTW supports flexible customer selection within each zone, bridging the gap between the restrictive assumptions of COP (visit all) and SOP (visit one), and enabling more realistic service strategies.
• Hybrid Algorithm Design: We propose a Hybrid Ant Colony Optimization (HACO) algorithm that integrates Pointer Networks, effectively combining global metaheuristic search with learning-based inference to improve search quality and computational efficiency.
• Empirical Validation: Extensive numerical experiments verify the effectiveness of HACO algorithm, showing superior performance over baseline methods in terms of solution quality, robustness, and computational time.

2. Problem Description and Formulation

2.1. Problem Description

The Multi-zone Orienteering Problem with Time Windows (MzOPTW) can be described as follows: A service worker needs to visit customers across multiple zones, each designed with multiple entry and exit nodes. The service worker must pass through these entry and exit nodes to reach the customers. Each customer has a specific time window, and if the service worker arrives before the time window starts, they must wait; if they arrive after the time window ends, the customer cannot be visited. Each customer can only be visited once, and the service worker cannot revisit the same zone. Additionally, considering that external road networks are usually well-developed outside the zones (communities), the distances between zones are calculated using the shortest route algorithm, while the Euclidean distance is used between customers within each zone.

The problem requires simultaneous decisions regarding which zones and customers to visit, which entry and exit nodes to select for each zone, the visiting sequence of zones, and the customer visiting sequence within each zone, with the goal of maximizing the total profit for the day. The scenario of this problem is illustrated in Figure 1.

2.2. Notation

The parameters and variables for the MzOPTW are summarized in

Table 2. Parameters and variables.

Set	Description
$S$	The set of zones, $S=\left\{1,2,\dots ,s\right\}$
$N_0,N_{n+1}$	Set of start and end nodes, $N_0=\left\{0\right\},N_{n+1}=\left\{n+1\right\}$
$N$	The set of entry and exit nodes for all zones, virtual entry/exit nodes, and the set of customers, $N=\left\{1,2,\dots ,n\right\}$
$N_s$	The set of entry and exit nodes, virtual entry/exit nodes, and customers of zone $s$
$N_s^1,N_s^3$	The set of entry and exit nodes of zone $s$, and the set of virtual entry/exit nodes of zone $s$
$N_s^2$	The set of customers in zone $s$
$N^1,N^3$	The set of entry and exit nodes for all zones, and the set of virtual entry/exit nodes for all zones
$N^2$	The set of all customers
$N^4$	The set of entry and exit nodes, virtual entry/exit nodes, and start and end nodes for all zones
$L_{ij}^F$	The shortest route distance between node $i$ and node $j$
$L_{ij}^E$	The Euclidean distance between node $i$ and node $j$
$v_F$	The driving speed outside the zone
$v_E$	The driving speed within the zone
$d_i$	The service time of customer $i$
$p_i$	The profit of customer $i$
$T_{max}$	Total time limit
$[o_i,c_i]$	The service time window of customer $i$
$T_i$	Auxiliary variable: the sum of the service time of customer $i$ and the waiting time of the subsequent customer
$a_i$	Auxiliary variable: the start time of the service for customer $i$
$y_i$	Auxiliary variable: 1 if the service worker visits node $i$, otherwise 0.
$x_{ij}$	Decision variable: 1 if the service worker traverses arc (i, j), otherwise 0.

below.

2.3. Problem Formulation

Based on the above assumptions and definitions, the mathematical model for the MzOPTW is formulated as follows:

$$max {\displaystyle \sum }_{i\in N^2}{p}_i\cdot y_i$$

(1)

$${\displaystyle \sum }_{j\in N^1\cup N^3}{x}_{0,j}={\displaystyle \sum }_{i\in N^1\cup N^3}{x}_{i,n+1}=1$$

(2)

$$y_0=y_{n+1}=1$$

(3)

$$\begin{array}{c}{\displaystyle \sum _{i\in N_0\cup N^1\cup N^3\backslash (N_s^1\cup N_s^3)}}{\displaystyle \sum _{j\in N_s^1\cup N_s^3}}{x}_{ij}=\\ {\displaystyle \sum _{j\in N_s^1\cup N_s^3}}{\displaystyle {\displaystyle \sum }_{i\in N_{n+1}\cup N^1\cup N^3\backslash (N_s^1\cup N_s^3)}}{x}_{ji}\le 1,\forall s\in S\end{array}$$

(4)

$${\displaystyle \sum }_{i\in N\cup N_0}{x}_{ij}=y_j={\displaystyle \sum }_{i\in N\cup N_{n+1}}{x}_{ji},\forall j\in N$$

(5)

$$\begin{array}{c}{a}_i+L_{ij}^F/v_F-a_j\le M\left(1-x_{ij}\right)\\ \forall i\in N_0\cup N^1\cup N^3,\forall j\in N_{n+1}\cup N^1\cup N^3\end{array}$$

(6)

$$\begin{array}{c}{a}_i+T_i+L_{ij}^E/v_E-a_j\le M\left(1-x_{ij}\right)\\ \forall i,j\in N_s,\forall s\in S\end{array}$$

(7)

$$o_i{y}_i\le a_i\le c_i{y}_i,\forall i\in N\cup N_0\cup N_{n+1}$$

(8)

$${\displaystyle \sum }_{i,j\in N^4}{x}_{ij}{L}_{ij}^F/v_F+{\displaystyle \sum }_{s\in S}{\displaystyle \sum }_{i,j\in N_s}{x}_{ij}{L}_{ij}^E/v_E+{\displaystyle \sum }_{i\in N^2}{y}_i{T}_i\le T_{max}$$

(9)

$${\displaystyle \sum }_{s\in S}{\displaystyle \sum }_{i\in N_s^2}{\displaystyle \sum }_{j\in (N_0\cup N_{n+1}\cup N)\backslash N_s}{x}_{ij}=0$$

(10)

$${\displaystyle \sum }_{s\in S}{\displaystyle \sum }_{i\in N_s^1\cup N_s^3}{\displaystyle \sum }_{j\in N^2\backslash N_s^2}{x}_{ij}=0$$

(11)

$${\displaystyle \sum }_{s\in S}{\displaystyle \sum }_{i\in N_s^1\cup N_s^3}{x}_{ii}=0$$

(12)

$${\displaystyle \sum }_{s\in S}{\displaystyle \sum }_{i\in N_s^1}{\displaystyle \sum }_{j\in N_s^3}{x}_{ij}+x_{ji}=0,j is the virtual point corresponding to i$$

(13)

$$T_i\ge d_i,\forall i\in N_0\cup N_{n+1}\cup N$$

(14)

$$a_i\ge 0,\forall i\in N_0\cup N_{n+1}\cup N$$

(15)

$$y_i\in \left\{0,1\right\},\forall i\in N_0\cup N_{n+1}\cup N$$

(16)

$$x_{ij}\in \left\{0,1\right\},\forall i,j\in N_0\cup N_{n+1}\cup N$$

(17)

The objective function (1) represents the maximization of the total profit. Constraints (2) and (3) ensure that the start and end nodes of the service worker’s route are the same. Constraint (4) ensures that each zone is visited at most once and guarantees connectivity between zones. Constraint (5) ensures connectivity between zones and customers, as the service worker may enter and exit the same entry/exit node of a zone. To facilitate modeling, a virtual node (virtual entry/exit node) is added to each zone’s entry/exit node. Constraint (6) captures the time dependency between two consecutively visited zones, ensuring that the arrival time is non-decreasing. Constraint (7) defines the temporal relationships among the arrival times at zone entry and exit nodes, as well as between customers within the same zone. To simplify the model, we combine each customer’s service time and the possible waiting time before the next zone or customer into a single variable $T_i$, rather than modeling waiting time separately. This reduces model complexity while preserving temporal feasibility. Constraints (6) and (7), respectively, describe the temporal relationships between zones and within zones, corresponding to the blue solid and dashed lines in Figure 2. In the formulation, the constant $M$ is used to relax time-related constraints. In practice, different values are assigned depending on the context: for Constraint (6), $M=T_{max}+{\max }_{i\in N_0\cup N^1\cup N^3,\forall j\in N_{n+1}\cup N^1\cup N^3}\{L_{ij}^F/v_F\}$ represents an upper bound on inter-zone arrival time, while for Constraint (7), $M=T_{max}+{\max }_{i,j\in N_s}\{L_{ij}^F/v_F\}$ reflects intra-zone timing bounds. These values are determined based on the maximum travel and service durations, ensuring feasibility while minimizing unnecessary relaxation. Constraint (8) ensures that the service at each customer begins within the specified time window. This constraint ensures that the arrival time is bounded by the customer’s time window only if the customer is visited. If a customer is not visited, the binary variable forces the arrival time to be zero, thus avoiding unconstrained variables and maintaining model strength. Constraint (9) imposes a global time limit on the service worker’s entire route. Constraint (10) ensures that after visiting the customers in a zone, the service worker must pass through the zone’s entry/exit node to access other zones. Constraint (11) ensures that the service worker must pass through the zone’s entry/exit node before visiting the customers within that zone. Constraints (12) and (13) prevent sub-cycles between the entry/exit nodes of zones. These structural constraints significantly restrict the actual number of feasible arcs during optimization, reducing redundant or unrealistic transitions without requiring explicit arc pruning. This implicit sparsification improves computational efficiency and ensures that the routing structure aligns with realistic operational logic. Constraints (14) to (17) define the ranges for intermediate and decision variables.

3. Solution Technique

The computational complexity of the MzOPTW problem is high due to the need to optimize both intra- and inter-zone routes, as well as due to the strong coupling between the selection of entry/exit nodes, visit sequence, arrival times, and stay times within each zone. Therefore, we designed a Hybrid Ant Colony Optimization (HACO) algorithm integrated with Pointer Networks, leveraging the global search advantages of population-based optimization algorithms and the parallel processing capabilities of machine learning networks to improve both the solution quality and efficiency. The algorithm includes the following modules: an inter-zone route optimization module; an intra-zone route optimization module; a maximum zone stay time optimization module; and an elite ant re-optimization module. The Algorithm 1 flow is shown as follows.

Algorithm 1. HACO algorithm flow.

1 Initialize the number of ants $n_k$ and pheromones, $rout{e}^{\ast }=\varnothing$

2 While $t<t_{\max }$ do

3  Use roulette wheel selection for ${\theta }_d$

4  Construct ant routes in parallel

5  Optimize the elite ant routes

6  if   $f(rout{e}_k)>f(rout{e}^{\ast })$ then

7    $rout{e}^{\ast }=rout{e}_k,{\omega }_d={\omega }_d+\epsilon$

8  Update the pheromones

9 Return $rout{e}^{\ast }$

3.1. Inter-Zone Route Optimization

The inter-zone route optimization problem involves deciding which zones to visit, the order in which to visit the zones, and the selection of entry/exit nodes for each zone.

3.1.1. Zone Selection

After the ant $k$ visits the customers within a zone $s$, it exits zone $s$ from one of the entry/exit nodes $i\left(i\in N_s^1\cup N_s^3\right)$ and selects the next zone to visit. The set of accessible entry/exit nodes for the next zone can be described as

$$candidat{e}_k^1(i)=\left\{j\right|j\in N_{\gamma }^1\cup N_{\gamma }^3,\gamma \in unvisitZone,t_{ki}+L_{ij}^F/v_F+L_{j,n+1}^F/v_F\le T_{max}\}$$

where $\gamma$ represents the zones that the current ant $k$ has not yet visited, and $t_{ki}$ represents the time at which the ant arrives at node $i$. Furthermore, when the ant enters the zone through the entry/exit node $i$, and chooses to exit through node $j$ of the same zone, the set of accessible entry/exit nodes $candidat{e}_k^2\left(i\right)$ can be described as follows:

$$candidat{e}_k^2(i)=\left\{j\right|j\in N_s^1\cup N_s^3,t_{ki}+{\theta }_d+L_{j,n+1}^F/v_F\le T_{max},L_{ij}^E/v_E\le {\theta }_d\}$$

where ${\theta }_d$ represents the maximum stay time of the ant in zone $s$.

When the ant selects the next zone to visit, in order to balance exploration and exploitation, the ant located at node $i$ generates a random number $q$ that follows a uniform distribution $U\left(0,1\right)$. If $q \le q_0$, then the ant will greedily select the next node $j$ as follows:

$$j=\left\{\begin{array}{l}\underset{ j\in candidat{e}_k^1\left(i\right)}{argmax}\left\{{\tau }_{ij}^{\alpha }{\left({\eta }_{ij}^1\right)}^{\beta }\right\},Select the next zone entry\hfill \\ \underset{ j\in candidat{e}_k^2\left(i\right)}{argmax}\left\{{\tau }_{ij}^{\alpha }{\left({\eta }_{ij}^2\right)}^{\beta }\right\},Select the exit of the current zone\hfill \end{array}\right.$$

(18)

where $q_0$ is a given parameter, ${\tau }_{ij}$ represents the pheromone value between node $i$ and node $j$, and ${\eta }_{ij}^1$ and ${\eta }_{ij}^2$ are the heuristic information (${\eta }_{ij}^1=1/L_{ij}^F$,${\eta }_{ij}^2=1/L_{j,n+1}^F$). $\alpha$ and $\beta$ are the pheromone influence factor and the heuristic information influence factor, respectively, indicating the relative importance of pheromone and heuristic information.

If $q > q_0$, the next node is selected using the roulette wheel method; this will expand the ant’s search space.

$${\varphi }_{ij}^k=\left\{\begin{array}{l}{\displaystyle \frac{{\tau }_{ij}^{\alpha }{\left({\eta }_{ij}^1\right)}^{\beta }}{{{\displaystyle \sum }}_{j\in candidat{e}_k^1\left(i\right)}{\tau }_{ij}^{\alpha }{\left({\eta }_{ij}^1\right)}^{\beta }}},Select the next zone entry\hfill \\ {\displaystyle \frac{{\tau }_{ij}^{\alpha }{\left({\eta }_{ij}^2\right)}^{\beta }}{{{\displaystyle \sum }}_{j\in candidat{e}_k^2\left(i\right)}{\tau }_{ij}^{\alpha }{\left({\eta }_{ij}^2\right)}^{\beta }}},Select the exit of the current zone\hfill \end{array}\right.$$

(19)

3.1.2. Pheromone Update

After each iteration, local pheromone updates are performed on the route:

$${\tau }_{ij}:=\left(1-\rho \right)\cdot {\tau }_{ij}+\rho \cdot {\tau }_0,\left(i,j\in N^4\right)$$

(20)

where ${\tau }_{ij}\left(i,j\in N^4\right)$ represents the pheromone value on the route between nodes $i$ and node $j$, $\rho$ represents the pheromone evaporation factor, and ${\tau }_0$ represents the initial pheromone value on each route. Local updates allow the pheromone values on the route to decay toward ${\tau }_0$, thereby increasing the diversity of the search and enhancing the ants’ global search capability. After that, the global pheromone is updated based on the route taken by the ant with the highest profit, as follows:

$${\tau }_{ij}:=\left\{\begin{array}{l}\left(1-\rho \right)\cdot {\tau }_{ij}+\rho \cdot {\tau }_1,\left(i,j\right)\in rout{e}^{\ast }\hfill \\ \left(1-\rho \right)\cdot {\tau }_{ij},other\hfill \end{array}\right.$$

(21)

where ${\tau }_1$ represents the pheromone value released by the ant with the highest profit, and $rout{e}^{\ast }$ represents the route taken by that ant. Global updates not only help prevent pheromone overaccumulation, which can overwhelm the heuristic information, but also contribute to improving the convergence speed of the algorithm.

3.2. Intra-Zone Route Optimization

Intra-zone route optimization involves deciding which customer nodes within a zone to visit and the order of visits. To efficiently process the access routes of multiple ants within a zone in parallel and improve the solution efficiency, this study designs a Pointer Network learning model to solve the intra-zone route problem. The Pointer Network, first proposed by Vinyals et al. [16], is a learning model that extracts features from discrete, variable-length input sequences and outputs pointers that probabilistically point to elements of the input sequence.

3.2.1. Markov Decision Process Construction

First, the route-solving process within zone $s$ is modeled as a Markov Decision Process (MDP) with discrete stages and deterministic state transitions:

(1)

The next node to visit is chosen at each decision stage $g=0,1,2,\dots ,G < \left|N_s^2\right|$, where the value of $G$ is uncertain. The process ends when the node reached is the exit of the zone.

(2)

The state set ${\mathcal{S}}_g=\left({\tau }_g,r_g,V_g\right)$, ${\tau }_g$ is the time at stage $g$, $r_g$ is the node reached at ${\tau }_g$, and when $g=0$, ${\tau }_0=t_{s^{in}}$,$r_0=s^{in}$. $V_g$ is the next set of nodes that can be visited at ${\tau }_g$, where node $v_i$($v_i\in V_g$) must satisfy the following conditions:

• ${\tau }_g+L_{r_g,v_i}^E/v_E+{\varpi }_{v_i}\le c_{v_i},{\varpi }_{v_i}=\max \left(0,o_{v_i}-\left({\tau }_g+L_{r_g,v_i}^E/v_E\right)\right)$
• ${\tau }_g+L_{r_g,v_i}^E/v_E+{\varpi }_{v_i}+d_{v_i}+L_{v_i,s^{out}}^E/v_E\le {\tau }_0+{\pi }^s$
• $v_i\cap \left({\cup }_{j=0}^g{r}_j\right)=\varnothing$

Here, condition (i) is the time window constraint for visiting node $v_i$, where ${\varpi }_{v_i}$ represents the waiting time at node $v_i$; condition (ii) is that the stay time in zone $s$ cannot exceed the maximum zone stay time ${\pi }^s$ (as described in Section 3.3); and condition (iii) ensures that each node can only be visited once.

(3)

The action set $x\left({\mathcal{S}}_g\right)=a_g$, where $a_g$ represents the action of choosing the next node to visit at ${\tau }_g$ ($a_g\in V_g$).

(4)

After visiting node $a_g$, the state transitions to ${\mathcal{S}}_{g+1}=\left({\tau }_{g+1},r_{g+1},V_{g+1}\right)$, where ${\tau }_{g+1}={\tau }_g+L_{r_g,a_g}^E/v_E+{\varpi }_{a_g}+d_{a_g}$, $r_{g+1}=a_g$, and, based on the conditions satisfied, $V_g$ is updated to $V_{g+1}$.

(5)

$R\left({\mathcal{S}}_g,a_g\right)=p_{a_g}$, the reward for selecting action $a_g$ at ${\tau }_g$.

3.2.2. Pointer Network

When solving the intra-zone route using the Pointer Network learning model, the goal is to maximize the total reward. The structure of the Pointer Network is shown in Figure 3.

The input features of the Pointer Network learning model for solving the intra-zone route include both static features ${\mathit{f}}^s$ and dynamic features ${\mathit{f}}^{d,g}$. The static features consist of the coordinates of the nodes, service duration, time windows, rewards, and the stay time in the zone. The dynamic features are related to the current time ${\tau }_g$ at stage $g$, and include the difference between the current time ${\tau }_g$ and the time window of each node, the time ${\tau }_0$ at stage $g=0$, and the stay time limit for reaching the zone’s endpoint.

In addition, this study introduces the foresight dynamic features for each node [18] (such as the difference between the arrival time at the node and its corresponding time window). At the beginning of each stage $g$, the static and dynamic features of the nodes within the zone are encoded into a vector ${\mathit{h}}^{e,g}$ using a Transformer [19]. This vector ${\mathit{h}}^{e,g-1}$, along with the Transformer-encoded vector from the previous stage, the cell state ${\mathit{c}}^{g-1}$ of the Long Short-Term Memory (LSTM) network, and the previous output vector ${\mathit{h}}^{d,g-1}$, is then fed into the LSTM to guide the decision process. The output of the LSTM at this stage, ${\mathit{h}}^{d,g}$, and the encoded vector ${\mathit{h}}^{e,g}$ from the Transformer are passed into the Attention module to generate the pointer vector ${\mathit{u}}_{a_g}^g$:

$${\mathit{u}}_{a_g}^g=Ctanh\left({\mathit{W}}_{1}tanh\left({\mathit{W}}_2{\mathit{h}}^{e,g}+{\mathit{W}}_3{\mathit{h}}^{d,g}\right)\right),a_g\in V_g$$

(22)

$$p\left(a_g|{\mathcal{S}}_g\right)=softmax\left({\mathit{u}}_{a_g}^g\right),a_g\in V_g$$

(23)

where ${\mathit{W}}_1$, ${\mathit{W}}_2$, and ${\mathit{W}}_3$ are the learnable parameters of the model, and $C$ is a hyperparameter [20]. Finally, the pointer vector ${\mathit{u}}_{a_g}^g$ is normalized into an output distribution over the nodes, and a search strategy is applied to select the next node to visit. Since the static features of the nodes at the beginning of each stage remain unchanged, but the dynamic features evolve over time, the Transformer will re-encode the dynamic feature vector at each stage to update the node features based on the current state. During the training process of the Pointer Network, the REINFORCE algorithm [21] is used to estimate the gradient. To improve the solution efficiency and ensure the quality of the solution, this study employs two types of search strategies, greedy search and beam search, which are used at different stages of the algorithm.

3.3. Maximum Zone Stay Time

The stay time of an ant in each zone affects the number of customers selected within the zone and the sequence in which customers are visited. This indirectly influences the selection and visit order of the zones, which in turn impacts the total profit. If visiting certain zones yields high profits, setting a longer stay time can help in visiting more high-profit customers. Therefore, this study proposes setting a maximum stay time for each zone to flexibly adjust the selection of customers with different profit values, thereby increasing the total profit.

(1)

Enumeration Strategy

This strategy is used at the beginning of each iteration in the HACO algorithm. It traverses a set of initial values for the maximum zone stay times and estimates the benefit value for each zone. This provides a basis for redistributing the maximum zone stay times during the subsequent elite ant re-optimization phase (see Section 3.4). A set of initial maximum zone stay times, denoted as $\Theta =\left\{{\theta }_1,{\theta }_2,\dots ,{\theta }_D\right\}$, is given based on factors such as the number of zones and the total time limit. The stay time for each zone in the set is sequentially assigned as the stay time limit for each ant during its visits to the zones. Afterward, both inter-zone and intra-zone route problems are solved. To obtain sufficient zonal benefit information, a random selection strategy is applied when selecting the next zone to visit. The Pointer Network model, based on a greedy search strategy, is used to process all ants’ intra-zone routes in batches. After obtaining the zone and customer visit route solutions, the benefit value for each zone is estimated as ${\delta }_s=scor{e}_s/su{m}_s$,$\forall s\in S$, where $scor{e}_s$ and $su{m}_s$ represent the total profit and total visit count, respectively. The larger ${\delta }_s$ is, the greater the potential benefit of zone s.

(2)

Roulette Wheel Strategy

This strategy is used to determine the maximum stay time for each zone for all ants during iterations of the HACO algorithm. For each element in the set $\Theta =\left\{{\theta }_1,{\theta }_2,\dots ,{\theta }_D\right\}$, an initial weight coefficient ${\omega }_d=(d=1,2,\dots ,D)$ is set. The zone’s stay time limit is then selected from $\mathsf{\Theta }$ using the roulette wheel method, and this stay time limit ${\theta }_d$ is assigned to each ant for its visit to the corresponding zone during the current iteration. At the end of each iteration, if the best solution found in the current iteration is better than the historical best solution, the weight coefficient ${\omega }_d$ is increased by $\mathsf{\epsilon }$. Otherwise, the weight remains unchanged.

3.4. Elite Ant Re-Optimization

In each iteration, after all ants generate their route solutions, only a set number of elite ants $n_{elite}$ are selected for re-optimization. This strategy consists of two stages: Stage 1: reassign the maximum zone stay time limits for the elite ants and optimize the intra-zone routes; Stage 2: for the elite ant with the highest profit, use the zone time fine-tuning algorithm to further optimize the intra-zone route.

Stage 1: If a zone has a high benefit value, allocating a longer stay time will help in visiting more high-profit customers, thus increasing the total profit. In this stage, the stay time limits for the zones traversed by the elite ants are reallocated based on the benefit value of each zone.

First, the total allocable time for the elite ant is calculated, denoted as ${\pi }_k$, which is the sum of the actual stay time in the zones traversed by the ant $k$ and the time difference when arriving at the destination ahead of schedule. This is calculated as follows:

$${\pi }_k={\displaystyle \sum }_{s\in S_k}\left(t_{k{s}^{out}}-t_{k{s}^{in}}\right)+T_{max}-t_{k,n+1}$$

(24)

where $S_k$ is the set of zones traversed by the elite ant $k$, $t_{k{s}^{in}}$ and $t_{k{s}^{out}}$ are the times at which the ant $k$ arrives and leaves zone $s$, and $t_{k,n+1}$ is the time at which the ant reaches the destination. After obtaining ${\pi }_k$, the maximum stay time for the zone $s$ traversed by the ant is reallocated as follows:

$${\pi }_k^s={\displaystyle \frac{{\delta }_s}{{{\displaystyle \sum }}_{r\in S_k}{\delta }_r}}\cdot {\pi }_k,s\in S_k$$

(25)

Under the constraint of ${\pi }_k^s$, the Pointer Network model based on the beam search strategy is used to generate the route of ant $k$. Then, the route is further optimized using three local operators: Insert [22], Swap, and Replace [23].

Stage 2: To improve the solution quality, a zone stay time fine-tuning algorithm is designed in this study. This further optimizes the maximum zone stay time for the elite ant with the highest profit in order to enhance the quality of the route solution. The framework of this Algorithm 2 is outlined as follows (see Appendix A for details):

Algorithm 2. Flow of the zone stay time fine-tuning algorithm

Optimize both the intra-zone and inter-zone routes

Input: $rout{e}_k,S_k,{\Delta }_t,\kappa$

Output: $rout{e}_k^{\ast }$

1 $rout{e}_k^{\ast }\leftarrow rout{e}_k,\phi \leftarrow 0,s_1\leftarrow S_k\left(0\right),s_2\leftarrow S_k\left(1\right)$

2 While $s_1\ne S_k\left(len\left(S_k\right)-1\right)$ do

3  $Imp\leftarrow False,toLeft\leftarrow True,Iters\leftarrow 1,$ calculate ${\mathsf{\Pi }}_k^{s_1}{ \mathrm{and} \mathsf{\Pi }}_k^{s_2}$

4  While $True$ do

5   If $toLeft$ = $True$ then

6    ${\pi }_k^{s_1}\leftarrow {\mathsf{\Pi }}_k^{s_1}+{\Delta }_t,{\pi }_k^{s_2}\leftarrow {\mathsf{\Pi }}_k^{s_2}-{\Delta }_t,$

7    $checkContinue\left({\pi }_k^{s_2},Iters\right)$

8   Else

9    ${\pi }_k^{s_1}\leftarrow {\mathsf{\Pi }}_k^{s_1}-{\Delta }_t,{\pi }_k^{s_2}\leftarrow {\mathsf{\Pi }}_k^{s_2}+{\Delta }_t,checkContinue\left({\pi }_k^{s_1}\right)$

10   End If

11   Use the Pointer Network model based on the beam search strategy and local search algorithms to optimize the zone $s_1$ and $s_2$.

12   If the current solution is improved then

13    $rout{e}_k^{\ast }\left(s_1\right)\leftarrow solutio{n}_{s_1},rout{e}_k^{\ast }\left(s_2\right)\leftarrow solutio{n}_{s_2}$

14    $Imp\leftarrow True$

15   Else

16    $checkContinue\left({\pi }_k^{s_2},toLeft\right)$

17   End If

18   $Iters\leftarrow Iters+1$

19  End While

20  If $Imp$ = True then

21    $\phi \leftarrow 0,s_1\leftarrow S_k\left(0\right),s_2\leftarrow S_k\left(1\right)$

22  Else

23   $\phi \leftarrow \phi +1,s_1\leftarrow S_k\left(\phi \right),s_2\leftarrow S_k\left(\phi +1\right)$

24  End If

25 End While

The algorithm optimizes the stay time of adjacent zones sequentially, combining the Pointer Network model and local search algorithms, continuously adjusting the intra-zone routes to find a better overall route solution.

4. Numerical Experiments

In this study, the proposed algorithm is implemented using Python 3.9. To verify its effectiveness, a series of numerical experiments are conducted, including tests to evaluate the overall performance of the algorithm as well as the effectiveness of the related strategies.

4.1. Instance Description

Due to the lack of standard test instances for this problem, this study references the VRPTW example generation method [24] and the road network generation approach from another study [25], and randomly generates test instances of three different sizes, each with two, four, and six zones. Each size includes 20 test instances. The example generation method and parameter settings are as follows:

First, a road network is generated on a 6 × 6 two-dimensional plane, consisting of 36 square grids with a side length of 10. The start and end nodes are located at the center $\left(30,30\right)$. A certain number of grids are randomly selected, and a specified number of points $k_1$ are randomly generated within each grid in the two-dimensional plane ${\left[1,9\right]}^2$. The convex hull of these points is calculated, and the points in the convex hull are connected in sequence to form the zone’s boundary. Then, three random points are selected from the convex hull and connected to the nearest main road, and $k_2$ points are chosen as the entry/exit nodes ($k_2~U\left[3,5\right]$) of the zone. Points not included in the convex hull are considered customers within the zone. Figure 4 illustrates an example with four zones, where the service worker’s start and end points are at the center. The black lines represent the main road routes, the green dashed lines represent the zone boundaries, the red stars indicate the zone’s entry/exit nodes, and the blue dots represent the customers within the zone. The values for $v_E$ and $v_F$ are set to 1.6 and 4, respectively. The working time is set to $\left[0,540\right]$, and the customers’ profits and service durations are uniformly distributed as random integers within the specified ranges $\left[50,200\right]$ and $\left[15,30\right]$, respectively. The customers’ service time windows are defined by the time window center and width, which are uniformly distributed as random integers within the specified ranges $U\left[o_0+t_{0i},c_0-t_{i0}-d_i\right]$ and, respectively.

4.2. Parameters and Pointer Network Training Process

In the HACO algorithm, the number of ants $n_k$ is set to 100, and the number of elite ants $n_{elite}$ is set to 5. The set of maximum zone stay time limits is denoted as $\mathsf{\Theta }=\left\{50,75,100,150,200,400\right\}$, with an initial value $q_0$ of 0.6. If the historical best solution has not improved after three iterations, the value of $q_0$ is reduced to 0.2 to increase the search space for the ants. The parameters are set as follows: ${\tau }_0$ = 1, ${\tau }_1$ = 1.2, $\alpha$ = 2.2, $\beta$ = 1.2, $\rho$ = 0.15, $\kappa$ = 30, ${\Delta }_t$ = 20, $n_b$ = 128, C = 10, $\mathsf{\epsilon }$ = 0.2, where the initial values of $q_0$, ${\tau }_1$, $\alpha$, $\beta$, and $\rho$ are determined through IRACE [26], which performs iterative racing procedures to identify high-performing parameter combinations based on experimental performance. The remaining parameters are set based on instance characteristics and preliminary tuning experiments.

In the training process of the Pointer Network, a global model is trained for each of the three different sizes of test instances. For each zone’s customers, their geographical location remains fixed, but their time windows, service durations, and profits are generated randomly. Therefore, variable feature data is sampled during the training process. In each training iteration for each size, a zone and its entry/exit positions are randomly selected, and $batch$ samples are generated. The time range for entering and exiting the zone is between 0 and 540. Each training for a given size is conducted for 300,000 iterations, with a batch size of 32. The Transformer encoder in the Pointer Network consists of two neural modules, each with an 8-head multi-head attention layer. The vector dimensions for both the encoder and decoder are set to 128, and the feed-forward layer dimension is set to 256. Each model is trained offline within a few hours on a standard GPU, and the resulting models are reused across all corresponding test instances. This eliminates the need for retraining and ensures that the training cost is incurred only once per problem size.

4.3. Performance of the HACO

In this study, the Gurobi solver is used to obtain exact solutions for the three different sizes of test instances, with a time limit of 7200 s. If the solution time exceeds the time limit, the current best integer solution is output. Additionally, since there is no similar comprehensive comparison algorithm, and considering that the Variable Neighborhood Search (VNS) algorithm proposed in another study [23] can solve problems such as TDOPTW by fixing the service time, this study replaces the Pointer Network in HACO algorithm with VNS to solve the lower-level route problem, which serves as the comparison algorithm ACO-VNS. Moreover, as iterated local search (ILS) [22] has been shown to effectively balance solution quality and computation time, it is chosen as the comparison algorithm for the Pointer Network. Thus, this study replaces the Pointer Network in HACO algorithm with ILS to solve the lower-level route problem, resulting in the comparison algorithm ACO-ILS. Considering that the MzOPTW requires multiple solutions for the OPTW, which leads to long computation times, VNS only uses the operators from the Initialization phase in the zone benefit estimation and general ant route generation phases. ILS uses only the Insert operator. In the elite ant re-optimization phase, both VNS and ILS use their full algorithm frameworks to solve the OPTW.

The running time limit for each algorithm on the three different sizes of test instances is set to 30 s, 45 s, and 60 s, respectively, and each algorithm is run 10 times. These time limits are kept consistent across all algorithms to ensure a fair comparison. The overall experimental results are shown in Figure 5 (see Appendix B for details), where the vertical axis represents the average objective function value (optimal value) of each algorithm on the different-size test instances.

Due to the need to solve multiple OPTWs for different zones and optimize the stay time in each zone, the computational complexity of MzOPTW is relatively high. Within the 7200 s time limit, Gurobi was unable to find the optimal solution for all test instances. However, in the 20 test instances of each size, HACO algorithm outperformed Gurobi in 18, 19, and 17 instances, with the maximum improvements in solution quality reaching 20.17%, 22.12%, and 20.82%, and the average improvements being 5.46%, 7.95%, and 8.8%, respectively. This shows that HACO algorithm outperforms Gurobi in overall performance, and its performance becomes more stable as the size of the problem increases.

Compared to ACO-ILS, HACO algorithm performed similarly in small-scale test instances and outperformed ACO-ILS in 15 test instances, with the maximum improvement being 2.93% and the average improvement 0.73%. In the other two scales, HACO algorithm outperformed ACO-ILS in 15 and 16 test instances, with maximum improvements of 4.91% and 4.15% and average improvements of 1.34% and 1.31%, respectively. When compared to ACO-VNS, HACO algorithm outperformed ACO-VNS in 11, 16, and 14 test instances for the three scales, with the maximum improvements reaching 2.93%, 5.33%, and 3.49%, and the average improvements being 0.52%, 1.01%, and 0.93%, respectively. It should be noted that while there are isolated cases where HACO algorithm performs slightly worse than ACO-ILS and ACO-VNS, the algorithm consistently shows superior performance in terms of average objective values across multiple runs, demonstrating its overall robustness and effectiveness.

Furthermore, in the 60 test instances across all three sizes, HACO algorithm achieved the optimal solution in 44 instances, accounting for 73% of the total test instances. On average, the HACO algorithm outperformed ACO-ILS and ACO-VNS in 88% and 97% of the test instances, respectively. Notably, these results were obtained within strict computational time limits (30 s for small, 45 s for medium, and 60 s for large instances), demonstrating that the integration of the Pointer Network learning model enhances not only solution quality but also computational efficiency. Compared to the baseline heuristics, the HACO algorithm consistently reached better solutions within comparable time limits, highlighting its practical potential in time-constrained decision environments.

4.4. Performance of the Elite Ant Re-Optimization Strategy

To verify the effectiveness of the elite ant re-optimization strategy, this study removes the elite ant re-optimization from the HACO algorithm (denoted as HACO¹) and removes only the second stage (zone time fine-tuning algorithm) from the HACO algorithm (denoted as HACO²). These two variants are then compared with the original HACO algorithm. The running time limit for each test instance remains unchanged for all three sizes, with each test instance being run 10 times, and the optimal value is selected. The experimental results are shown in

Table 3. Results of HACO vs. HACO¹ and HACO².

Number of Zones		Obj.			Gap
		HACO	HACO1	HACO2	HACO1	HACO2
|S| = 2	Max	3454	2765	3454	32.14	4.81
	Min	2518	1783	2397	11.41	0.00
	Avg	3030.3	2268.1	2994.9	25.19	1.18
|S| = 4	Max	3865	3161	3707	24.55	7.80
	Min	2943	2338	2905	11.16	0.00
	Avg	3494.6	2826.6	3412	19.13	2.37
|S| = 6	Max	3878	3399	3798	20.47	5.96
	Min	3098	2846	2990	6.85	0.00
	Avg	3584.1	3134.1	3498.2	12.46	2.39

, where Avg, Max, and Min represent the average, maximum, and minimum values, respectively, for the 20 test instances of each size.

In Table 3, HACO outperforms HACO¹ in every test instance. For the three different sizes, the maximum improvements are 32.14%, 24.55%, and 20.47%; the minimum improvements are 14.41%, 11.16%, and 6.85%; and the average improvements are 25.19%, 19.13%, and 12.46%, respectively. This demonstrates that the elite ant re-optimization strategy significantly enhances the solution quality.

Furthermore, HACO, which incorporates the zone time fine-tuning algorithm, outperforms HACO² in the majority of the test instances. For the three scales, the maximum improvements are 4.81%, 7.8%, and 5.96%, with average improvements of 1.18%, 2.37%, and 2.39%, respectively. The proportion of test instances with improvements is 60%, 85%, and 90% (see Appendix C). This indicates that the zone time fine-tuning algorithm further optimizes the solution.

5. Conclusions

This study proposes a new variant of the OP—the MzOPTW. In this problem, customers are located in different zones, and service personnel can only access customers within a zone by entering specific positions in that zone. The optimization objective is to develop the optimal service route within a limited time, serving the selected customers within their designated time windows, in order to maximize the total profit. The contributions are as follows: (1) First, a mathematical model for the problem is constructed, and a population-based optimization algorithm, HACO, which integrates the Pointer Network learning model, is designed. (2) Second, an algorithm is developed to effectively optimize the zone stay times. (3) Third, compared to the exact solver Gurobi and heuristic algorithms such as ACO-ILS and ACO-VNS, the proposed HACO algorithm framework performs better and more stably. Additionally, the elite ant re-optimization strategy significantly improves solution quality.

The proposed model and algorithm can be applied to scenarios such as the Tourist Trip Design Problem, where zones represent scenic areas and customers correspond to points of interest. Future work may explore the Team MzOPTW with multiple service personnel and develop more efficient algorithms. In addition, future work could extend the proposed approach to dynamic and online settings, where routing instances evolve over time and require real-time decision-making. Incorporating principles from online combinatorial optimization, such as instance evaluation, prioritization, and adaptive resource allocation, may enhance the responsiveness of the framework in practical applications [27].