A* Based Routing and Scheduling Modules for Multiple AGVs in an Industrial Scenario

A multi-AGV based logistic system is typically associated with two fundamental problems, critical for its overall performance: the AGV’s route planning for collision and deadlock avoidance; and the task scheduling to determine which vehicle should transport which load. Several heuristic functions can be used according to the application. This paper proposes a time-based algorithm to dynamically control a fleet of Autonomous Guided Vehicles (AGVs) in an automatic warehouse scenario. Our approach includes a routing algorithm based on the A* heuristic search (TEA*—Time Enhanced A*) to generate free-collisions paths and a scheduling module to improve the results of the routing algorithm. These modules work cooperatively to provide an efficient task execution time considering as basis the routing algorithm information. Simulation experiments are presented using a typical industrial layout for 10 and 20 AGVs. Moreover, a comparison with an alternative approach from the state-of-the-art is also presented.


Introduction
In recent years, and due to the mandatory need to continuously adapt the production flow, industrial companies are increasingly adopting fully automated internal logistic systems, namely based on AGVs, instead of manual or inflexible mechanical solutions (e.g., forklifts, conveyors, and others).
Considering the Industry 4.0 initiative, both AGVs, as well as mobile manipulators, are seen as strategic tools in the Factories of the Future. In a very competitive industrial environment, these can contribute to increase productivity and reduce the costs associated with the internal logistic system, ensuring an efficient material flow. Likewise, their introduction also allows human operators to be reallocated to more complex and ergonomic tasks, with increasing value to the final product. These set of characteristic makes AGV's appealing for a wide range of industrial applications, such as goods transportation, end-ofline automation chain, warehouse and distribution. However, and despite their versatility, there is the need to deploy advanced multi-robot coordination algorithms in order to ensure the AGV's continuous operation, guaranteeing the minimum tasks execution time and the smoothness of the vehicle's movements.
The AGV's fleet coordination problem has received wide attention from both the research and industrial fields. Typically, two main systems comprise any multi-AGV application: free-collision routing system [1,2] and scheduling system, that encompass both task scheduling and dispatching [3]. The vehicle routing system is responsible for computing trajectories that minimize the total distance traveled by AGVs considering different constraints such as each vehicle's carrying capacity and the plant layout where vehicles can circulate, while ensuring free-collision routes. Some approaches are based on time windows as proposed by the authors in [4][5][6][7][8]. Here, the AGV route is constructed considering that one point can only be visited one time for only one vehicle at a given time interval. The feasibility of each route is evaluated, checking windows overlapping. In its turn, the scheduling system is associated with the task scheduling and their attribution to individual AGVs. This scheduling and dispatching should take into account some decision criteria, namely the task deadlines and traffic status, among others.
Bearing these ideas in mind, this paper proposes an integrated approach for both AGV route planning and task scheduling and dispatching. More in detail, the proposed routing algorithm called Time Enhanced A* (TEA*) is an extension of our previous work addressed in [9,10], that is, in this paper, further integrated with a scheduling module in order to minimize the tasks execution time. The main feature of TEA* is the addition of a temporal component to the known A* algorithm that generates routes efficiently, considering that each robot knows other robots' positions during the time. Furthermore, TEA* is an on-line approach allowing its integration in dynamic environments.
The major contributions of our paper are to propose a Multi-Robot Coordination System which includes a time-based algorithm to generate free-collision routes based on the A* algorithm and a scheduling module that use the routing algorithm information to minimize a cost function, dependent on the following parameters:

1.
Average execution time of all tasks; 2.
Execution time for the last vehicle; This paper is organized as follows. In Section 2 the state-of-the-art of path planning algorithms and multi-robot systems are presented. Section 3 describes the proposed multirobot routing algorithm. Section 4 describes the industrial case scenario, followed by the comparison between the proposed approach with an alternative state-of-the-art [11]. Sections 5 and 6 present the results achieved using Tabu-Search Method. Finally, some conclusions and the contribution of this paper are presented in Section 7.

Related Work
In the last decade, the multi-robot coordination problem has been a target of many scientific studies. To solve it, several authors have proposed the use of meta-heuristic approaches to address both the problem of AGV routing and task scheduling. The authors in [12][13][14][15][16], propose using a Genetic Algorithm to find the optimal or sub-optimal solution, which satisfies the routing system goals, including the minimization of the tasks completion time, minimal distance, among others. Likewise, ref. [17] proposes an integrated solution that comprises a routing module based on genetic algorithms and a scheduling system based on bid auctions. Despite the potential of the proposed methods, normally these approaches treat each robot as an individual agent (without physical constraints) [17,18], simplifying the problem at hand. Furthermore, the results do not include industrial case scenarios. Alternatively, particle swarm strategies can also be applied to multi-AGV systems as in [19,20]. Ref. [21] proposes a Particle Swarm Optimization (PSO) based algorithm, called Fractional Order Robotic Darwinian Particle Swarm Optimization (FORDPSO), integrated with a fuzzy system to optimize the driving of multi-robots in unknown environments. However, these methodologies are not yet sufficiently tested in real industrial environments.
Considering solely the use of a bid auction strategy, the authors in [3,22] propose a solution where each robot constructs 'bids' for each task and a central module receives the bids and assigns tasks considering the fitness function's maximization. Similarly, ref. [23] proposed different combinatorial bidding strategies, comparing its performance with single-item auctions.
Recognize the use of analytical methods, the authors in [24] define a robot mission through Linear Temporal Logic formulas (LTL). An LTL approach considers that the truth of a declaration can be changed during the time. This work focus on minimizing the cost function, which is the maximum time between candidate solutions of an optimizing proposition. In [25] the authors use a model based on Integer Linear Programming (ILP) to find paths that minimize the time until the last robot reaches its goal or minimizes the total traveled distance.
Generally, two different approaches define the architecture of any multi-robot system: a Centralized [26] or Distributed [27] methodology. The authors in [28,29] use a centralized architecture where one of the robots is the leader, and the others are the followers. Here, the major challenges are related to ensuring communication robustness and the algorithm flexibility to change the leadership. Other works use distributed architecture like in [27,30], where each robot calculates its path independently using, for example, a D* Algorithm [31] and then, the path is broadcast for all robots, making that every robot knows all path information.
Regarding only task scheduling system, it typically aims for minimizing an objective function which includes system characteristics, such as the number of vehicles, the order in which the vehicles execute their missions, etc. In [32], Kelen et al. compares two scheduling methods that determine the ideal number of vehicles for a given industrial scenario: the Shortest Job First and Tabu Search. Additionally, a routing method based on an enhanced Dijkstra algorithm, was used to manage the AGV's path. The number of stoppages and the time that each vehicle waits for the mission assignment was not measured.
In [20], Yu Zhang et al. proposes simple heuristics at the high-level layer, referred to as the 'coalition' level, that creates an abstraction layer relatively to specific details in robots' specifications. Simple heuristics can be 'MinProcTime', that gives priority to the missions with shorter processing times, and another one can be the 'MinStepSum', similar to the 'MinProcTime', but determines the best solution incrementally when the ordering of the assignment are not pre-determined. However, in the simulation experiments presented were not considered an industrial scenario with real-world scheduling problems.
In the past half-decade, new approaches based on Artificial Intelligence (AI) are emerging. AI addressed in [33,34], is the science that seeks to study and understand the phenomenon of intelligence and, at the same time, a branch of engineering, as it seeks to build instruments to support human intelligence. In practice, an AI system besides storing and manipulate data can also acquire, represent, and manipulate knowledge. This manipulation concerns the ability to deduce or infer new knowledge from existing knowledge and use representation and manipulation methods to solve complex problems. This area of engineering is vast and has been the subject of huge investment from both business and research institutes. More specifically in Robotics, there is already a recent line of work on Multi-Agent Path-Finding (MAPF) [35][36][37][38]. On AGVs, the MAPF problem is to find the best paths, for a fixed number of agents, from their current locations to the final task position, where all agents have a free-collision path, as it is described in [39][40][41]. Another use of AI in Robotics, is the combination of non-AI algorithms (time-window based or greedy) for the AGV coordination with AI for the prediction of future tasks [42].
Despite the many scientific studies carried out, task scheduling solutions often resort to small heuristics (First-in-First-Out, Shortest-Distance), often decoupled from the trajectory planning system. In turn, trajectories are predefined in offline mode and designed to prevent, as far as possible, the occurrence of deadlocks. This type of solution often leads to the logistics system being oversized concerning the operation's real needs.

Proposed Routing Algorithm
To overcome the challenges related with the multi-AGV coordination problem, presented in the previous sections, in this section a new methodology for AGV's route planning is proposed. This novel methodology is called TEA* Algorithm [9,10], in which the paths are recalculated continuously, making it an online method. According to the vehicles' movements and the environment changes, TEA* updates the paths of each AGV in order to avoid collisions and to guarantee the continuous operation of the logistic system. In fact, as in the majority of the industrial logistic systems, it is expectable that changes on initial task information and/or unpredictable events, such as delays in the transportation system as a result of obstacles presence [21], can occur. In these scenarios, the adoption of online path planning algorithms, capable of dealing with such events, becomes mandatory.
As referred before, TEA* is based on the traditional A*, where a third dimension was added, the time. The input map has three dimensions: vertex's coordinates (x and y) and a representation of the time, as shown in Figure 1. The time is represented with temporal layers given by k ∈ [0, k Max ] (k max denotes the maximum number of layers). Each temporal graph is a set of free and occupied/obstacles vertexes. As far as computational complexity is concerned, many factors can influence the system (RAM, CPU, others), however, through [43] where the test conditions are the same for all algorithms, it is possible to validate that A* is the most suitable algorithm for robot fleet management in different environments. Therefore, taking into account the case study carried out in [43], and the results obtained in [9], it can be concluded that TEA* is a practical, versatile and quite optimized algorithm in terms of path planning algorithms.

The TEA* Method
In multi-AGV systems, time is a crucial component for a better prediction of the vehicles' positions. Besides the constantly recalculated paths, TEA* determines the route for each AGV during the temporal layers. This grants the identification of upcoming collisions, allowing them to be avoided with considerable anticipation. The path information for each AGV is converted to a busy vertex on the following robot's map, allowing collision avoidance.
Consider a graph G with a set of vertexes V = {0, 1, ..., NU M_VERTEXES} and edges E = {0, 1, ...NU M_EDGES} (links between the vertexes), with a representation of the time [ 0, 1, ..., k max ] (as can be seen in Figure 1). Each AGV can only starts and stops in vertexes and each vertex can only be occupied by only one vehicle at the time.
During the path search for a single AGV the neighboring vertexes are evaluated using a similar approach as A* algorithm [44]. Moreover, each edge in each temporal layer, has a cost function value, denoted as f (j, k), given by the sum of two terms (see Equation (1)).
Considering the path between j 0 and j f , the first term αg(j, k), represents the distance between the current vertex j to the initial vertex j 0 , in the k temporal layer. The second term, denoted as βh(j, k), is a heuristic value that calculates the distance to the final vertex j f . The terms α and β assign different weights to the distance and the heuristic function.
Each vertex, in each temporal layer, has different values of g(j, k) and h(j, k), according with the Equation (2). Here, g(j, k) is given by the sum of the distance between the current vertex j and the initial vertex j 0 , being the edge distance between j and its adjacent vertex j + 1 denoted as dis(j, j + 1, k). g(j, k) = dis(j, j 0 , k) + dis(j, j + 1, k) h(j, k) = dis(j, j f , k) (2) The main differences between TEA* and the known A* algorithm are mainly concerned with the addition of the time component and can be defined as follows:

Smoothing Trajectories
For TEA* to be applied, the shop floor layout requires to be modeled as a set of vertexes and edges (links between the vertexes). Each AGV travels on these graph paths, from one node to another, through a pre-defined set of edges. Each edge is represented as a cubic Bézier curve (as proposed in article [45]), given by Equation (3).
Here, λ denotes an integer value between 0 and 1 according with the AGV's position in the curve, (x 0 , y 0 ) is the initial point of the curve, and a x , b x , c x , defines the spline's curvature. Figure 2 represents a portion of the map which contains Bézier curves and straight lines.

Industrial Layout and Comparison Scenario Description
In this section, the TEA*, based on Robot Operating System (ROS), is compared with a state of the art alternative, namely the coordination algorithm presented in [46]. This algorithm relies on coordination diagrams for planning the coordinated motion of a fleet of AGVs. One of the contributions of [46] is the definition of a heuristic function that estimates the number of times a vehicle starts and stops during its path execution.
For the comparison of the two algorithms, a set of experiments were conducted using the same layout, the same number of vehicles and the same missions' list used by the authors of [11]. Each mission is defined by four tasks (Sn-Starting positions; Pn-Pick-up Positions; Dn-Drop-off Stations; Rn-Rest Positions). The layout dimensions are 80 × 110 m and 10 AGVs were used to generate the results of [11]. Figure 3 represents the input map of TEA* Algorithm. The graph was built using a graph editor, which was created with the Robot Operating System Visualization (RVIZ) platform and the Interactive Markers tool.

Routing Algorithms Comparison
Considering the industrial scenario previously presented, Table 1 illustrates the mission execution time for each vehicle with the TEA* Algorithm, using 10 AGVs and 30 tasks. For each AGV, the time of advancement (T adv ) and the stopping time (T stop ) are reported. In Figure 4 is illustrated the final solution found for each vehicle. Two black circles represent possible collision points in different robot paths, but the third dimension of TEA* allows AGVs to share the same trajectory by passing at different times at the common points, i.e., avoiding possible collisions.
Comparing the results achieved with the TEA* (Table 1), with the results presented by the authors in [11] (Table 2), it is possible to conclude that TEA* is advantageous mainly considering the T adv of the last vehicles (AGVs 6,7,8,9,10). Conversely, the T adv of the AGVs 3, 4 and 5 surpasses the value of the same robots in Table 2. However, in these cases the T average and the T max are lower in TEA*. Therefore, this algorithm presents itself as a better solution for multi-robot path planning. The average time (T average ) to complete the missions list with TEA* is 116.7 s and in the case of [11] this time is approximately 121.78 s. The last vehicle in [11] takes 144.2 s to finish its tasks, while in TEA*, the last vehicle performs its tasks in 135.8 s.
To highlight the capability of TEA* Algorithm to optimize the routes even with considerable workload in the system, Table 3 presents the same industrial scenario but now with 20 AGVs and 60 tasks. Note that the difference in the average time between 10 and 20 AGVs are approximately 11 s. The fact that a vehicle considers the current and future positions of each vehicle as obstacles during the discrete-time, gives it the possibility to wait for some instants for the traveling of previous AGV, instead of calculating a longer deviation. For the sake of completeness, it is important to refer that the TEA* Algorithm has been designed to be integrated into dynamic industrial environments, thus allowing direct scaling concerning the number of robots to be used. The structure of the algorithm itself is already prepared for different exchanges of industrial scenarios.

Task Scheduling Algorithm
As referred earlier, the problem of AGV coordination is not only closely related with the route planning of the AGVs, but also with the task scheduling. The performance of the routing algorithm can be improved. The AGV execution order affects the calculation of routes since the path positions over time for a given vehicle are obstacles for the following AGVs. If the execution order changes, the paths and respective task execution times for each vehicle are different.

Tabu Search Method
The Tabu Search Method is a 'meta-heuristic' adaptive method of local search in continuous exploration within a search space, moving from one solution to another, the Tabu Moves, diversifying the solutions found in this process of the search for an improved solution [47]. The best permissible movement is the one with the highest evaluation in the vicinity of the current solution regarding target function value and taboo restrictions. Thus, the 'meta-heuristic' Tabu Search is an iterative search algorithm characterized by dynamic memory and consisting of two parts: initialization and search.
Starting from an initial randomly generated solution or using a heuristic, the Tabu Search will evaluate a set of different mutations (neighborhood exploration) of the current solution in each iteration. The best mutation will be accepted, and the changes made saved in a Tabu List adopted to store the most used changes, which are classified as prohibited in later iterations. This strategy is necessary to avoid a return to solutions already checked previously.
Therefore, in this method, in each iteration, the evaluation function consists of validating a certain quantity of new solutions, where the best solution, based on the objective function, is accepted, even if its cost is higher than the cost of the current solution. Thus, the algorithm chooses the new solution that produces an improvement or the least deterioration in the cost function (an attempt to evade minimal locations). The Tabu Search algorithm runs until a stop criteria is reached. Figure 5 presents the main blocks of the Tabu Search method. In the AGV scheduling problem presented, the final objective is the allocation of the n sub-tasks by the j available AGVs, in order to minimize the total time for the overall task completion. The main goal is to determine/distribute the best sequence of attendance of the sub-tasks by the AGVs to minimize the total time of execution.
For the problem presented, the initial solution (sequence of sub-tasks assigned to each AGV) is generated using the closest neighboring heuristic, as presented in Algorithm 2. In other words, each AGV (and taking into account the last sub-task performed, which influences its position on the map) is assigned to the next sub-task with lower cost (shorter travel time/distance).
This process is executed cyclically until all sub-tasks have been assigned to one, and only one, AGV. AllUnassignedSubTasks ← delete < NewSubTaskAGV(j) >; The Algorithm 3, describes the implementation of the Tabu Search used in the proposed approach.

TEA* Algorithm with Tabu Search Method-Results
The Tabu Search Method was implemented to find better vehicle configuration and to schedule the order in which vehicles execute their tasks. A configuration comprises the order in which vehicles should be processed by the TEA* Algorithm.

return sBest
The AGV execution order affects the calculation of routes since the path positions over time for a given vehicle are obstacles for the following AGVs. If the execution order changes, each vehicle's paths and respective task execution times will be different.
The optimization goal is the minimization of the three following parameters: 1.
Time of the last vehicle, denoted as T max in seconds; 2.
Average Time of the missions execution, denoted as T average in seconds; 3.
Number of Stoppages, denoted as n stop ; That were aggregated in the following cost function (Equation (4)): Here, γ, ψ and τ are components that are weighting parameters. In the simulation experiments the following values were used, respectively 0.1, 0.7, 0.2. These were manually defined considering an iterative and experimental way, with the main goal of minimizing the cost function c.
The configuration that leads to the lowest cost function is chosen as the better solution. In our approach is not required to achieve the optimal solution, a near-optimal configuration that leads to an efficient TEA* execution is enough.
To obtain the initial configuration of the Tabu Search Method, a heuristic function was defined. It consists of the path computation for each vehicle without consider the other vehicles as obstacles (optimal solution) and ordering it by decreasing the order of execution task times. The objective is to process firstly the longer paths minimizing the number of stoppages. The candidate solutions are generated, changing two by two the vehicle execution order from the current solution. Table 4 presents the results for the TEA* Algorithm using the AGV configuration solution found by the Tabu Search method. The average time for completing all tasks is lower, but the more significant improvement was the waiting time. In Table 1  Considering 20 AGVs, besides the stoppage time to be higher, the task completion time is lower than TEA* results without scheduling module. In Table 3 the total time to complete all tasks was 179.3 s and in Table 5 this time was 159.1 s.

Conclusions
This article proposes a multi-AGV system that comprises a routing algorithm based on the search method A*. This algorithm is suitable for multi-robot applications, avoiding collisions and deadlocks and guaranteeing any industrial scenario required safety levels. To optimize the results achieved scheduling method (Tabu Search) minimizes a fitness function defined by several parameters calculated by TEA*. The two modules work cooperatively, sharing the TEA* information.
Our work's major contributions are: (i) Presentation of a promising approach for multi-AGV applications in warehouse environment, improving the flexibility and efficiency of the complete system; (ii) Validation of an on-line Multi-Robot Coordination Algorithm comparing it with a state-of-the-art alternative.
As future work, it will be interesting to validate the TEA* Algorithm in a real environment, with a real robotic system, by comparing it to the simulation results.  Data Availability Statement: This study didn't report any new data.