A Review of the Vehicle Routing Problem and the Current Routing Services in Smart Cities ^†

Abstract

In this survey, the issues of urban routing are analyzed, and critical considerations for smart and cost-effective delivery services are highlighted. Smart cities require intelligent services and solutions to address their routing issues. This article gives a brief description of current services that either apply classical methods or services that employ machine learning approaches. Furthermore, a comparison of the most promising research options in regard to VRP is provided. Finally, an initial design of a holistic scheme that would optimally combine several tools and approaches to serve the needs of different users with regard to the VRP is presented.

1. Introduction

One of the most prominent trends of recent years is urbanization, as more and more people opt to live in large cities. The process of urbanization is associated with both economic and social development. However, as the urban population continues to increase, so do the problems that people in large cities face. Consequently, both academia and industry are seeking to find optimal solutions to urbanization’s problems.

Given this situation, an effort is made towards the aim of the improvement of infrastructure, people’s quality of life and sustainability, through the development of smart cities. The growth of modern smart cities can alleviate various problems faced by large cities’ citizens. One of the most-concerned smart cities challenges is intelligent transportation and significantly solving the vehicle routing problem.

Both modern societies and smart cities require smart applications and services to remedy the problem of the last-mile procedure of goods shipped. That need has been generated from various social, economic, and climate requirements for improving their citizens’ quality of life and reducing wasted fuels, CO${}_1$ and CO${}_2$ emissions, overall transportation costs, and traffic congestion.

This paper focuses on the urban vehicle routing problem (VRP) and examines both classical VRP and its variants and a multi-objective VRP. In addition, several existing solutions for finding the optimal path through the use of classical routing algorithms are presented. However, as Machine Learning (ML) technology continues to develop rapidly, it is increasingly being incorporated as a solution to various existing problems, one of which is VRP. Thus, it was deemed necessary to review ML-based solutions that find the optimal route.

The contributions of the present survey are as follows:

• Presentation of the VRP problem and its variants
• Analysis of the state of the art of routing algorithms
• Research upon ML based research efforts that contribute to solving VRP problems

The remainder of this paper is organized as follows: In Section 2, the categorization of single objective and multiobjective VRPs is presented; Section 3 provides a brief review of existing routing services; While Section 4 identifies the key points of existing routing services; In Section 5 a review of routing services incorporating ML techniques is presented; While Section 6 identifies the key points of existing ML-based routing services; Regarding the VRP, Section 7 provides an initial design of a holistic scheme to serve the needs of various users; Finally, Section 8 concludes with the findings of this review.

2. The Vehicle Routing Problem

The VRP is a complex optimization problem, in which there exists a set of clients at various locations, each one with a shipment need, and a fleet of vehicles, departing from the central depot that shall optimally satisfy the needs of the clients [1]. The aim of a typical VRP is to find out the optimal route to minimize the total costs. Furthermore, various factors affecting route planning, such as vehicle capacity, fuel consumption, traffic congestion, etc., have to be considered to accomplish the minimization of the total route costs.

Over the years, various studies have been proposed by researchers for VRPs that focus on a single objective regarding route planning. The most known and most studied VRP is the Capacitated Vehicle Routing Problem (CVRP), which focuses on cases where the vehicle’s capacity is constrained. On the CVRP route finding, only one vehicle visits every customer a single time; the total client’s requirements should not go beyond the vehicle’s capacity, and the total cost has to be minimized given all the aforementioned parameters [2].

The client’s need for the delivery process to be accomplished in a restricted time frame has led to the definition of the Vehicle Routing Problem with Time Windows (VRPTW), in which the clients should be served by the vehicles, in a certain time window of a day [3].

The vehicle’s fuel emissions are a significant concern in many research fields, including VRP. Thus, the Fuel Consumption Vehicle Routing Problem (FCVRP) focuses on the minimization of the overall vehicle consumption and emission in the planning of a route [4]. The green VRP variant is focused on including both the minimum distance and the CO${}_2$ emission in route planning [5].

Another factor that can decrease the overall costs of logistic companies, and therefore the costs of the delivery process is the design of the supply chain network in a better way. This process consists of a better establishment of the depots’ locations and a better firmness of serving clients from the depots leading to the Location Routing Problem (LRP) [6]. Furthermore, a similar problem with the LRP is the Inventory Routing Problem (IRP), which emphasizes the minimization of distribution costs without affecting the client’s stock [7]. Lastly, the combination of LRP and IRP led to the implementation of Combined Location Routing and Inventory Problem (CLRIP) which minimize the overall costs by assigning the depots’ locations, arranging vehicles route planning, and defining the inventory policy based on the client’s needs [8].

However, in the reality, optimal route planning is not comprised of only one objective/factor but of combinations of those. Thus, the research community focused on the Multiobjective Routing Problems, which consider partial objectives/factors to generate the optimal route. An expansion of CVRP is Multiobjective Capacitated Vehicle Routing Problem (MOCVRP), in which two or three objectives are usually considered, such as distance, transportation time, or the number of the fleet of vehicles, to find the optimal route [9]. The Green Capacitated Vehicle Routing Problem (GCVRP) is classified in the same subcategory. The GCVRP focuses on green transportation by including the reduction of hazardous particles such as greenhouse gases, CO${}_2$, etc., and fuel consumption in a route [10,11]. The Multiobjective Vehicle Routing Problem with Time Windows (MOVRPTW) is a dilatation of VRPTW which takes into account the various aspects of the multiobjective problem of time window delivery [12]. Furthermore, the multiobjective extension of LRP is the Multiobjective Location Routing Problem (MOLRP) that considers at least transportation costs, lateness, and the number of vehicles combined with maximization of clients’ service quality [13,14]. Moreover, it should be noted that more VRPs have been proposed in the scientific literature. However, this review studies the most known and common VRPs, the categorization of which is depicted in Figure 1.

The VRP and its more complex variations are constantly being studied by the research community as more variants are emerging in recent years. Additional information can be read in the following works: [15,16,17,18].

3. Exploration of Existing Routing Services

Several routing services have been presented over the years regards in order to address the VRP in smart cities. The present section reviews in chronological order various smart routing services in urban environments that have already been proposed and implemented.

The authors in [19] considered the employment of GAs to the basic VRP, in which customers are served from a single point. Vehicles are subject to a weight limit and, in some cases, to a distance limit, while only one vehicle is allowed to supply each customer.

Berger and Barkaoui propose a hybrid GA solving the VRPTW [20]. The proposed algorithm involves the simultaneous evolution of two solutions’ populations that face separate objectives with a relaxed time constraint. The first population evolves the solutions, to minimize the total travel distance, while the second population aims to minimize the transgression of the time constraint.

The authors in [21] proposed a meta-heuristic ant colony procedure for solving the DVRP. Initially, a working day is divided into time slots. Then, a sequence of static routing problems is generated for the vehicles. Finally, the meta-heuristic ant colony algorithm is used to solve the problem. A series of tests showed that the proposed method outperformed others for both artificial and real problems.

The proposed work in [22] includes a dynamic route evaluation model so that vehicles respond to changing traffic information, a modified Dijkstra algorithm to find out the shortest routes in real-time, and finally, an improved evolutionary algorithm to search for the best routes in a dynamic network. The proposed approach has been evaluated through simulations and is effective in finding the best vehicle routes in real-time when customer nodes and network information are dynamically changing.

A novel hybrid GA for vehicle routing is proposed in [23]. The proposed solution finds the optimal path for the VRP, while simultaneously considering the vehicles’ heterogeneity, dual routes, and multiple depots.

In [24] a hybrid approach that combines a GA with the Dijkstra algorithm to solve a dynamic multi-objective problem is proposed. The proposed algorithm finds the solution simultaneously for three objective functions: route length, travel time, and ease of driving. In order to apply a GA to a traffic system, the authors use Dijkstra’s algorithm to calculate the initial population of high-quality paths. On the initial population, the proposed approach applies a GA algorithm to generate the subsequent routes’ generations.

Ho et al. [25] developed two hybrid genetic algorithms for routing and scheduling deliveries in supply chain cases, dealing with the multi-depot VRP (MDVRP). The first hybrid genetic algorithm randomly generates the initial solutions, while the second hybrid genetic algorithm applies heuristics to generate the initial solutions.

A method to reduce industrial costs and the unfavorable impacts of fleet management in a Hub and Spoke transmission network is given out in the paper [26]. The distribution of goods in large cities may be best addressed by the time-dependent problem of pick-up and delivery and the time window models. The aforementioned parameters led to the development of a dynamic version of an algorithm that analyzes how daytime traffic congestion affected road traffic. An adaptable heuristic solution is suggested, which has been verified using actual data from third-party logistics service providers in the transportation industry. By significantly decreasing air pollution, noise pollution, and traffic jams, the heuristic algorithm demonstrates its capability to diminish route time and trip distance, while significantly increasing the level of service beyond the community’s necessity.

Moreover, a new approach of the ant colony algorithm to solve the VRP is presented in [27]. The main feature of this method is the hybridization of the solution construction mechanism of the ant colony algorithm, and the combination with a scatter search.

The authors in [28] observed that most car drivers use routing paths based on the shortest distance between the starting and ending points. However, the shortest distance route may differ from the shortest time route. For that reason, they used the ant colony algorithm and combined it with traffic data, correctly predicting both the travel times and the fastest routes, through avoiding future traffic jams.

A modified A* algorithm was proposed to calculate and generate an automatic optimal path [29]. The proposed modified algorithm is able to avoid repeatedly searching in invalid areas, which makes it efficient and accurate in finding the feasible path in unknown environments.

In [30] a Tabu search algorithm was implemented to solve the VRPTW, in which the constraints of the vehicle’s capacity and time window for each customer must be taken into account. The proposed algorithm was applied to real-world conditions for identifying the optimal delivery routes for a transportation company. The experiments resulted in the minimization of the transportation cost while the time constraint requirements were satisfied.

An improved ant colony algorithm for solving the PVRPTW is proposed in [31]. In this problem, the scheduling period extends over several days and each customer must be served within a specific time window. Firstly, the information for different days is stored in a multidimensional array and then two cross-over functions improve the algorithm’s performance.

In [32] the proposed ant colony algorithm is used for the time windows-dependent routing problem (TDVRPTW). In this problem, a fleet of vehicles has to deliver goods to a set of customers, where the customers’ time interval constraints should be respected and the travel time between two points depends on the departure time.

Another modified Tabu search algorithm for the VRP is proposed in [33]. After the comparison of the proposed algorithm to other algorithms, the adoption of the Tabu search shows that the proposed solution is more stable, fast, and with high calculation efficiency, both for small and for large-scale problems.

Billhardt et al. [34] introduced a dynamic cyber fleet management system. The proposed technology seeks to increase the productivity, safety, and autonomy of drivers and vehicles. The first layer of the system’s architecture is made up of vehicles, the second layer of fleet coordination modules, and the third layer of components for monitoring, task management, global fleet control, etc.

An efficient and flexible framework for dynamic waste management and collection is presented in the paper [35]. The presented framework’s design consists of the physical infrastructure, including smartphones and smart bins, a middleware layer, with dynamic routing models and OpenIoT, and the physical infrastructure, among municipality stakeholders and smart waste trucks. Furthermore, the authors examined the dedicated trucks model, detour model, minimum distance model, and reassignment model. While the CPU’s elapsed time, collected load, distance, routing time, response time, and fuel quantity characteristics were used to determine the performance of these four aforementioned models.

In [36], a team chose to use the Bellman–Ford algorithm to solve the problem of the energy efficiency of electric vehicles. First, a model that represents electric vehicles is implemented, and then a Bellman–Ford search is applied to the model to find the most energy-efficient path.

In the SOUL project, information and communication technology are applied to the local food supply chain, such as the e-grocery supply chain [37]. The project’s architecture is made up of a central unit, a traffic handler service, a data broker layer, various hosted services, and third-party services, enabling technologies, sensors and other external data sources, and mobile devices. The aim of the proposed work is to enhance the efficiency and effectiveness of e-grocery activity in urban areas by incorporating a decision support system and a mobile application.

A fleet management service solution for enhancing mobility and safety in urban areas is presented in [38], and it offers both fleet control and fleet monitoring. Information about a vehicle’s mobility, including fuel usage, geolocation, speed, and $C{O}_2$ vehicle’s emissions level, is provided via the fleet monitoring service. While in order to minimize the distance between the vehicles and reduce traffic shockwaves, the service of fleet control delivers various information about the moving objects, such as the absolute position, velocity, and acceleration.

The modified Bellman–Ford algorithm proposed in [39], deals with the multi-route single-vehicle capacity problem (mt-CCSVRP). In this problem, a single vehicle can make consecutive trips to serve a group of locations by minimizing the sum of arrival times.

PreGo system [40] was implemented to offer real-time tailored routing service in urban environments. PreGo’s solution suggests routes that may be customized to the preferences of a particular user are non-objective before the beginning of the route and are flexible to changes in the roads’ conditions. Additionally, the PreGo system was subjected to a detailed experimental evaluation utilizing both real and generated data to validate its performance.

For the VRP with discrete deliveries and pickups, in which customer demands are discrete in terms of lots (or orders), the team in [41] used the Tabu search algorithm. The proposed algorithm avoids unnecessary travel expenses. The experimental results showed that the proposed algorithm is a more efficient solution than the existing literature algorithms.

The authors in paper [42] try to find out the optimum solution for the shortest path in multi-point delivery problems, that can be used by drivers and autonomous vehicles. First, a graph representing the roads is created, and then Dijkstra’s algorithm is used to find the shortest path from the starting point to all the points that should be served. Then, starting from the nearest point which is now considered the new starting point, the proposed solution tries to find the shortest path to the remaining points.

An IoT system to guide emergency vehicles to the least congested route, avoiding traffic jams in a smart city is proposed in [43]. Sensors are the main real-time data source of the system’s architecture, while a data fusion technique based on fuzzy logic is used.

The authors in [44], considered two sub-models with the objective of minimizing the total transportation costs and maximizing the recycled revenue from municipal solid waste management. The first sub-model uses the VRP to route the waste fleet management to the separation facilities. The second sub-model was implemented to allocate resources from separation facilities to total recovery units or landfills.

An IoT-based food supply chain network that effectively locates and monitors food quality within a supply chain, and finds the source of a contaminated food product is presented in [45]. Moreover, a dynamic vehicle routing using the bee colony algorithm to minimize both the travel and execution time during the transportation is proposed.

4. Identifying Key Points of the Existing Routing Services

The inspection of the previously mentioned routing services led to a set of features that may be used to assess the offered solutions and to directly compare those.

Table 1. Routing Services Features Comparison Table.

Project	Time Window	Green Routing	Vehicle Capacity	Fleet Management	Transportation Costs	Traffic Handler	Travel Time/Distance
Baker & Ayechew [19]	∘	∘	•	∘	∘	∘	•
Berger & Barkaoui [20]	•	∘	∘	∘	•	∘	•
Montemanni et al. [21]	∘	∘	•	•	•	∘	∘
Wang et al. [22]	•	∘	∘	∘	∘	•	•
Jeon et al. [23]	∘	∘	•	•	∘	∘	•
Kanoh & Kenta [24]	∘	∘	∘	∘	∘	•	•
Ho et al. [25]	∘	∘	•	•	∘	∘	•
Falsini et al. [26]	•	•	•	•	∘	•	•
Zhang & Tang [27]	∘	∘	•	∘	•	∘	•
Tatomir et al. [28]	∘	∘	∘	∘	∘	•	•
Yao et al. [29]	∘	∘	∘	∘	•	∘	•
Cheeneebash & Nadal [30]	•	∘	•	•	•	∘	•
Yu & Zhong [31]	•	∘	•	∘	∘	∘	•
Balsiciro et al. [32]	•	∘	•	•	∘	•	∘
Jia et al. [33]	∘	∘	•	∘	•	∘	•
Billhardt et al. [34]	∘	∘	∘	•	•	∘	•
Anagnostopoulos et al. [35]	∘	•	∘	•	•	•	•
Abousleiman & Rawashdeh [36]	∘	•	∘	∘	∘	∘	•
Tadei et al.[37]	•	•	∘	•	•	•	•
Natale et al. [38]	∘	•	∘	•	•	•	∘
Rivera et al. [39]	∘	∘	•	∘	∘	∘	•
Hendawi et al. [40]	∘	∘	∘	∘	∘	•	•
Qiu et al. [41]	∘	∘	•	•	•	∘	∘
Adnan & Abdulmuhsin [42]	∘	∘	∘	∘	•	∘	•
Rout et al. [43]	∘	•	∘	∘	∘	•	•
Akbarpour et al. [44]	∘	∘	•	•	•	∘	•
Nagarajan et al. [45]	∘	∘	∘	∘	•	∘	•

presents a coherent summary of whether each solution takes into account each one of the identified main features. The black dots in Table 1 indicate whether a feature applies to the service. These features are the following:

• Time window: The requirement for delivery to happen under specific timing restrictions
• Fleet management: The management of a number of vehicles and the optimization of the utilization of those.
• Transportation cost: The combined economic cost of a route that consists of fuel cost, vehicle maintenance cost, and human labour cost.
• Traffic handler: Taking into account traffic conditions for optimizing the route planning.
• Travel time/distance: Minimising the route time and distance.
• Green routing: Minimising the exhaust emissions.
• Vehicle capacity: Take into consideration the capacity of a vehicle or fleet of vehicles for optimizing route planning.

Moreover, Table 1 shows the integration of the features in studied research routing approaches. The basic approach that is common throughout almost all cases is travel time/distance optimization. The main objective of VRP is identifying the fastest/shortest routes. It has to be noted though that there are cases where the proposed systems ignore this parameter and focus on other objectives, such as transportation costs [19,20,22,23,24,25,26,27,28,29,30,31,33,34,35,36,37,39,40,42,43,44,45].

The second most evident feature is transportation costs. In a simplistic approach, this metric can be assessed as directly connected to the travel distance. The actual transportation cost depends heavily on travel distance but is also related to other parameters such as the altitude variance along a route, the fuel type of the vehicle or the maintenance plan for the fleet. According to the problem set-up, it may be required to take into account one of the two aforementioned features or both. A route planning solution has to combine both and allow the user to set his preferences [20,21,27,29,30,33,34,35,37,38,41,42,44,45].

An interesting case is the integration of the fleet management feature. More than half of the proposed solutions do not take into account fleet management and are restricted to solving the problem of route planning either with a single vehicle or under a simplistic approach that the iteration of optimal route planning with a single vehicle for a pool of orders and a number of vehicles can efficiently solve the problem. In practice, route planning with more than one vehicle provides flexibility and can facilitate better locality of batched orders or better management of time off the road for each vehicle. As in a real-world set-up, it is not common to have a single vehicle; the discussed approaches that take into account more than one vehicle and do proper fleet management are favorable and more efficient [21,23,25,26,30,32,34,35,41,44].

Another important feature that seems not to receive significant attention is the time window feature. When the routes refer to products that impose specific timing restrictions, either for preserving their quality or because the delivery time is critical for the application. Only seven of the discussed services take the timing window into account and this for sure presents a limitation in the current state of the art. Along with that, an interesting feature that can facilitate more efficient route planning, especially in urban environments, is taking into account traffic information and that is present in approximately half of the proposed solutions. Another popular feature for route planning is taking into account the environmental footprint of the routes. The number of the solutions that actually do that is limited [20,22,26,30,31,32,37].

Furthermore, considering the table below, it appears that several studies [19,21,23,25,26,27,30,31,32,33,39,41,44] incorporate the feature of vehicle capacity into their provided VRP services. Vehicle capacity is a feature that affects the planning of the optimal route quite a lot, as the lack of vehicle capacity can cause many modifications to the route, execution time, delivery time, route costs, etc., of the optimal path. However, although several of the studies [26,30,31,32,41] include both time window and vehicle capacity features, the majority of them find the optimal route by including only one of the two features, and not both of them concurrently.

5. A Review of Routing Services Incorporating ML Techniques

In addition to the traditional routing services proposed in the scientific literature over the past years, new services incorporating new technologies are being constantly presented. ML is the most popular technology used in addition to conventional algorithms, to improve the performance of the aforementioned algorithms. In the following paragraphs, an overview of routing solutions that integrate ML is provided.

The objective of work [46] is to solve the CVRP using ML-based techniques. The authors proposed the “Learn to Improve” (L2I), a learning-based solution for CVRP that excels Operation Research (OR) algorithms in terms of solving speed. Specifically, the authors demonstrated a learning-based algorithm for the CVRP, proposing a framework capable of splitting heuristic operators into two different groups to accelerate the operation and concentrate Reinforcement Learning (RL) on those identified as improvement operators. Lastly, they presented an ensemble technique in which RL rules are taught simultaneously, yielding improved outcomes at the same computational cost.

The authors of work [47] constructed a database of optimum solutions for sampled events and characteristics. The aforementioned database is utilized for training ML models capable of predicting features of optimum solutions for unobserved cases and creating branching scores responsible for assessment, a process that resembles the behavior of an expert. This generates a new form of the VRPTW, namely Sampled Vehicle Routing Problem with Time Windows (SVRPTW). Experiments indicated that the technique described above surpasses conventional algorithms in terms of both the number of nodes handled during the query and the runtime required.

The authors of study [48] suggested Deep Policy Dynamic Programming (DPDP), which intends to combine the benefits of established neural heuristics and DP techniques. DPDP prioritizes and restricts the DP state space based on a policy established from a deep neural network trained to estimate edges based on sample solutions. Three different variations of routing problems were utilized and tested. First, the original traveling salesman problem (TSP), then the simplest routing problem regarding vehicles, the VRP, and finally, a variation of the classical TSP, the TSP with time windows (TSPTW). The experiments demonstrated that the performance of the algorithms was greatly enhanced; thus, the algorithms were capable of outperforming several other known solutions.

Study [49] aims to address the Electric Vehicle Routing Problem with Chance Constraints (EVRP-CC) and partial recharging. The authors presented a strategy that consists of two independent phases. The first determines the pathways between all nodes to be visited, while the second determines the optimal sequence of the trip to reduce energy use and arrange to charge the vehicle as required. In the second phase, the algorithm identifies the least energy-intensive routes that begin and end at the depot, stopping at all customers and charging stations as needed. Customers, charging stations, and depots are considered to be positioned at junctions as nodes in the road network. All of the aforementioned employ an author-developed probabilistic Bayesian ML model. In conclusion, the trials conducted in the city of Gothenburg, along with several realistic simulations, reveal a high degree of accuracy for the energy forecast, as well as energy improvements and increased route dependability. More specifically, according to the results of the experiments, no total consumption fell beyond the prediction’s 95% confidence zone. Furthermore, it was feasible to save up to 19.5% of energy on the ten itineraries without charging, with an average of 10.6%.

The authors of study [50], attempt to address the multi-objective VRP with stochastic demand (MO-VRPSD), one of the more complex variants of the VRP. There are two significant obstacles associated with the aforementioned issue; first, the unpredictability of client needs; and second, the possibility of conflicting purposes. To address the aforementioned challenges, the authors provide a decision tree-based ML model to solve the MO-VRPSD, which helps in producing suitable populations based on the information gained from previous search processes and significantly decreases the number of iterations. In addition, they provided a double input about the MO-VRPSD’s primary difficulties. First, they presented a novel way to encode and decode the route and chromosome so that it can successfully manage route failure, and then they utilized a robust multi-objective optimization algorithm to cope with the competing objectives in the MO-VRPSD. Their strategy is assessed using modified Solomon VRP benchmark cases. The experimental findings demonstrate that the suggested evolution model is capable of locating solutions with a better Pareto front and outperforming previous evolutionary methods.

Work [51] provides a solution to the Energy Minimizing Vehicle Routing Challenge (EMVRP), a problem that focuses on locating routes of vehicles that consume the lowest amount of energy when servicing a collection of cities or clients. The authors offer an implementation of a genetic algorithm that is augmented by ML approaches to tweak its parameters in a subsequent phase. The strategy under discussion is the application of k-means clustering, which demonstrated that the identification of distinct data clusters has a substantial influence on enhancing the effectiveness of the utilized genetic algorithm.

In the paper [52], the authors suggested a revolutionary extensive neighborhood search (LNS) architecture for routing-related problems that uses learnt heuristics to generate new outcomes. The learning process is based on a deep neural network with an attention mechanism and was created specifically for integration into an LNS search environment. The described system, NLNS, is an extension to LNS that discovers and deploys repair operators for VRP instances using a guided heuristic search. The authors examined the technique with respect to CVRP and the Split Delivery VRP (SDVRP). Considering CVRP instances featuring approximately 297 customers, the proposed method outperforms an LNS that employs simply handmade heuristics plus a well-known heuristic algorithm from the literature by a large margin. In addition, for the CVRP and the SDVRP, the authors demonstrated that the aforementioned technique outperforms current ML approaches and methods.

The authors of study [53] provide an end-to-end framework for employing RL to solve the VRP. In this method, a single policy model is trained to propose optimal solutions for a diverse variety of issue situations of comparable size by detecting reward signals and according to feasibility restrictions. The authors evaluated a parameterized stochastic policy, and by using a policy gradient method to maximize its parameters, the trained model generates the solution as a series of successive actions in real-time without requiring re-training for each new issue occurrence. Regarding the CVRP, the suggested method beats traditional heuristics and Google’s OR-tools on moderate instances with equivalent computation time in terms of solution quality. In addition, the authors explored the effect of separated shipments on the reliability of the solution and demonstrated how the approach under consideration can deal with delivery problems. Lastly, the aforementioned paradigm may be used for various variations of the VRP.

Authors in [54] presented a framework based upon a value-function-based DRL scheme that uses a combinatorial action space, where the action selection issue is explicitly described as a mixed-integer optimization problem. As an illustration, they applied this paradigm to the CVRP. In each instance, an action is characterized as the creation of a single route, and a basic policy iteration method is used to develop a deterministic policy. According to the authors, the suggested methodology is competitive with existing RL approaches and accomplishes similar performance when compared with state-of-the-art OR methods on medium-sized standard library instances.

In order to solve one of the most common problems encountered in the field of transportation and supply chain delivery, the CVRP, a research team used a recursive approach of the k-means clustering algorithm along with the Dijkstra shortest path algorithm [55]. The proposed solution divides into parts the CVRP to find the optimal route. Firstly, it takes into account the capacity of the fleet of vehicles to optimize the total route’s capacity; then the k-means algorithm, considering the travel time, distance travel, and vehicles’ capacity, is implemented; in the next step, the optimal capacity of vehicles is ensured; while in the last step, the Dijkstra’s algorithm finds the shortest route for the fleet of vehicles.

6. Identifying Key Points of the ML-Based Routing Services

While employing ML algorithms in the context of solving the different variations of the VRP, is an expected approach the existing state of the art is not at a level that can significantly change the developed systems and services. The main conclusions that can be made upon the analysis of existing literature (depicted in

Table 2. ML-based Routing Services Features Comparison Table.

Project	Time Window	Green Routing	Vehicle Capacity	Multi Objective	ML Techniques
Lu et al. [46]	∘	∘	•	∘	Reinforcement Learning
Furian et al. [47]	•	∘	∘	∘	ML algorithms
Kool et al. [48]	•	∘	∘	∘	ML algorithms
Basso et al. [49]	∘	•	∘	∘	Bayesian ML model
Niu et al. [50]	∘	∘	∘	•	Decision Tree
Cooray & Thashika [51]	∘	•	∘	∘	k-means clustering
Hottung & Kevin [52]	∘	∘	•	∘	ML algorithms
Nazari et al. [53]	∘	∘	•	∘	Reinforcement Learning
Delarue et al. [54]	∘	∘	•	∘	Reinforcement Learning
Moussa [55]	∘	∘	•	∘	k-means clustering

) are as follows:

• Limited number of research papers: Scientific community tends to employ ML techniques to solve (or solve more efficiently) any existing problem, and this results in a huge volume of papers (of mixed quality level) in multiple domains that offer ML-based solutions. Our findings regarding the ML solutions for VRP problems are limited in number, and this indicates that either the problems are efficiently solved with other approaches or that the employment of ML algorithms for such problems is not sufficiently effective (e.g., because of the form of the VRP).
• Fragmentation with regards to parameters of the problem to be solved: The analyzed research efforts tend to take into account different criteria when applying the ML algorithms to the VRP problem. Vehicle capacity parameter is present in approximately half of the approaches, while the rest tend to prioritize differently from each other upon the objectives of the route planning.
• Insufficient justification on the selection of ML algorithms: On average the efforts analyzed tend to insufficiently explain what ML algorithms are used and the rationale behind that. In some works this is straightforward (e.g., stating that a specific algorithm is used to solve a well-defined problem), but in the rest, the ML approach is not well explained and the contribution of each paper is not clearly defined.
• Comparison with traditional approaches: While using ML for VRPs sounds promising, it has to be justified by concrete results and through the comparison of those to the results of existing alternatives. In the majority of the works presented in the previous section, such a comparison is missing. This may be due to either authors neglecting this task but also due to the fact that such a comparison requires a lot of effort, in order to equally test the different approaches, under identical test parameters (e.g., same routes, same traffic load, etc.)

In general, we can conclude that while ML adoption in the VRP domain seems to be an interesting approach, the current state of the art is not at a satisfactory level. A combination of reasons that relate to both the practical difficulties of the domain and the complexity of the problem has made difficult the offering of an effective and robust ML service that could replace the traditional solutions to the variations of the VRP.

7. An Efficient Solution Regarding the VRP

On the basis of the aforementioned discoveries, we initiated the design of a framework capable of delivering an efficient solution for the VRP and its variants, including the CVRP and the GVRP. The proposed system considers all the stages of the logistics workflow, from the initial order placement through the delivery of the goods to the client. Initially, the set of delivery points is processed, and batches of close geographical locations of the delivery addresses are created dynamically by utilizing ML models. Subsequently, an ensemble scheme of various genetic algorithms is used to identify the optimal route for each one of those batches. Regarding the optimal route, the vehicles start from the same starting point, make exactly one stop at each client, and finally, return to the starting point. The aforementioned procedure considers the overall $C{O}_x$ emissions, and the total distance travelled.

To evaluate the system in each and every stage of its development, real data were collected from orders which took place in the city of Lamia, Greece. In the following paragraphs, important experimental findings obtained during the early stages of the development of the suggested solution are reported. In the experiments conducted, 200 real-world routes (originating from 9 different starting points) followed by logistics vendors were used.

7.1. Experimental Results Regarding the Initial Stage of System’s Development

Regarding the initial stage of development, the first observations made were related to the percentage of routes that were successfully handled by the system, as well as the percentage of routes deemed more efficient in terms of distance travelled when compared to actual data.

Initial simulations showed that the proposed ensemble scheme was able to handle successfully (identify a route that complies with the VRP restrictions) the 74% of the routes, as displayed in Figure 2.

Out of the routes that were successfully handled, 86% were deemed to be a better route recommendation in terms of total distance travelled, whereas only 14% of the displayed routes were unable to locate a more efficient solution. Figure 3 shows the percentage of the routes which achieved a more efficient route.

The results indicate that the ensemble scheme must be fine-tuned or enhanced by additionally employing ML algorithms in order to appropriately present a route, even if it is not the ideal one based on the data entered into the system. The final stage is to evaluate the data regarding total journey time and distance to determine when the system presents the optimal route and what occurs when it fails to do so.

7.2. Evaluation of the Routes Displayed by the System

For each starting point, a number of routes are included in the real-world data. Figure 4 depicts the minimum (orange) and maximum (blue) distance deviation values for the whole set of routes from each starting point for the proposed system. In comparison to the real conditions of Starting Point 5, a variance of 8517 meters emerges in this set of routes.

Similarly, in regard to the total time, Figure 5 depicts, with respect to the total time travelled, the minimum (orange) and maximum (blue) time deviations in the set of all routes from each beginning point, based on the proposed system, versus the actual conditions of a route. On a number of instances, the system requires a longer period of time to deliver orders than actual data. This is not a major concern, as our primary focus is on the GVRP variant, for which the total distance and total $COx$ emissions must be decreased. Nevertheless, as stated previously, the system is still in its early stages of development, so the scheme could be improved to achieve even faster time travel.

The results regarding the maximum and minimum time and distance deviations indicate that the system is capable of finding almost every time a better route in regards to the total distance travelled, and when it fails to do so, it displays a route with a similar distance to the routes which correspond to the real-world data. Contrariwise, the system displays routes in which the time travel required for the order demands a higher amount of time for the delivery. This could be explained due to the mechanism which is in charge of displaying an estimation of the required time calculated with many factors such as traffic and weather conditions. Additionally, the system with respect to road safety regulations always calculates the time respecting the speed limit and traffic lights.

8. Discussion

In the present survey, we have provided a concise introduction to the VRP and its variants and thoroughly discussed the proposed route planning algorithms. An analysis of the most prevalent routing difficulties and the corresponding parameters that have to be taken into account has also been conducted. The routing algorithms proposed have been compared on the basis of the scope under which they provide solutions to the VRP and a systematic overview of the current state of the art has been attempted. Additionally, some of the most well-explored multiobjective problems were discussed.

Extending this analysis we opted for identifying the research efforts in the literature that focus on applying ML methods to solve variants of the VRP or parts of it. Existing intelligent routing systems for urban areas and routing systems that incorporate ML methods in order to enhance the performance of the classical routing algorithms have been assessed. The results were not of the expected robustness and quality.

On the basis of these discoveries, we presented a holistic scheme that would optimally combine different tools and approaches to provide a modular configurable solution that would serve the needs of different users with regard to the VRP.