Grid-Based Mobile Robot Path Planning Using Aging-Based Ant Colony Optimization Algorithm in Static and Dynamic Environments.

Planning an optimal path for a mobile robot is a complicated problem as it allows the mobile robots to navigate autonomously by following the safest and shortest path between starting and goal points. The present work deals with the design of intelligent path planning algorithms for a mobile robot in static and dynamic environments based on swarm intelligence optimization. A modification based on the age of the ant is introduced to standard ant colony optimization, called modified aging ant colony optimization (AACO). The AACO was implemented in association with grid-based modeling for the static and dynamic environments to solve the path planning problem. The simulations are run in the MATLAB environment to test the validity of the proposed algorithms. Simulations showed that the proposed path planning algorithms result in superior performance by finding the shortest and the most free-collision path under various static and dynamic scenarios. Furthermore, the superiority of the proposed algorithms was proved through comparisons with other traditional path planning algorithms with different static environments.


Introduction
Robot navigation is the process of guiding a mobile robot toward the destination to perform complex tasks, such as cleaning. There are two approaches for navigation: reactive navigation and map-based navigation. In the first approach, the mobile robot has no map or any idea of where it is. The mobile robot uses random motion and acquires the information about the environment only from the contact sensor, i.e., the machine has the ability of sensing and action. On the other hand, map-based navigation is the process of creating a path for the mobile robot to move from one place to another that satisfies some criteria, such as the shortest distance and/or the lowest cost. The machine is able to sense, plan, and act, which is called path planning [1]. Several studies have been conducted to cover the problem of route planning. A grid map and improved a visible graph based on global path planning using A* algorithm was pointed out in Reference [2] and the improved A* algorithm, i.e., by considering the influence of parent node on the heuristic function of the A* algorithm, was adopted in Reference [3] for autonomous parade robot in the indoor environment. In Reference [4], the memory-efficient A* (MEA*) algorithm generated a shorter path with less time and less memory requirement when in a grid set of simulation results are presented to demonstrate the effectiveness of the proposed methodology, as compared with other previous works. Section 6 presents the conclusion of the obtained results.

Problem Statement and Preliminaries
Suppose the mobile robot (MR) moves from the start position (SP) to the goal position (GP) in an environment with static and dynamic obstacles to obtain certain performance criteria. The objective of a path planner is to find the optimal/near-optimal path for the mobile robot without any collision with obstacles existing in the environment.

Grid-Based Environment Modelling
The first step of mobile robot path planning is to establish an environment model for the 2-Dimensional (2-D) workspace of the mobile robot. Grids are used to represent the mobile robot workspace as equal square cells. Each cell is either traversable, i.e., logic 0, or obstructed by an obstacle, i.e., logic 1, as shown in Figure 1. Each cell is identified by a unique number, called the "address". The address of the cells is defined by two methods. The first method is the 2D grid coordinate (r, c), where the origin of the grid coordinate is the cell in the top left of the grid, with the first location having the address (1,1). The second method is the serial number (SN) method, where the addressing of the cells begins from the left top cell and continues from left to right and top to bottom. The serial number for each cell can also be converted to its equivalent (x, y) coordinate, as shown in Figure 2.
The mapping from the (r, c) grid to the serial number id given in Equations (1)- (3) and from the (x, y) coordinates to the serial number given in Equations (4)- (6) where . is the least integer function, c t is the number of columns of the matrix, R is the remainder, x max is the end of the abscissa, and y max is the end of the y-coordinate.
where ⌈. ⌉ is the least integer function, i.e., the cell function, is the number of columns of the matrix, R is the remainder, is the end of the abscissa, and is the end of the y-coordinate.

Performance Criteria
The shortest distance is the main objective for a mobile robot to move from its start position to its goal position, provided that it is a safe path, i.e., the mobile robot moves without colliding with obstacles. This criterion is given by: where n is the number of steps that the mobile robot needs to navigate toward its goal, g is the fitness of the solution, and ℇ is a small number, e.g., ℇ = 0.001, used to prevent a division by 0. In every step, the mobile robot can move from its current location to another one of its surrounding free cells, as shown in Figure 3.

Performance Criteria
The shortest distance is the main objective for a mobile robot to move from its start position to the goal position, provided that it is a safe path, i.e., the mobile robot moves without colliding with obstacles. This criterion is given by: where n is the number of steps that the mobile robot needs to navigate toward its goal, g is the fitness of the solution, and ε is a small number, e.g., ε = 0.001, used to prevent a division by 0. In every step, the mobile robot can move from its current location to another one of its surrounding free cells, as shown in Figure 3.
where n is the number of steps that the mobile robot needs to navigate toward its goal, g is the fitness of the solution, and ℇ is a small number, e.g., ℇ = 0.001, used to prevent a division by 0. In every step, the mobile robot can move from its current location to another one of its surrounding free cells, as shown in Figure 3. The optimization problem can be defined as: The optimization problem can be defined as: "Find the shortest distance between the start point (SP) and the goal point (GP), such that the above criteria g (Equation (8)) is maximized".

Obstacles Movement
In this case, the obstacles change their location continuously at each time step, and, in dynamic environments, the position of the obstacle (x obs , y obs ) is updated according to, y obs = y obs + v obs × sin θ obs (10) where v obs is the velocity of obstacles and θ obs is the slope of the linear motion.

Ant Colony Optimization
Swarm intelligence is based on nature-inspired behavior and is successfully applied to optimize problems in a variety of applications. The ant colony optimization (ACO) algorithm is a stochastic-based optimization technique that replicates the behavior of real ants when searching for food, invented by Dorigo [26]. It discovers the shortest route from an ant nest to food places through the interchange of information collaboration [27]. The ants move along the same path by following one another. This is because every ant leaves a chemical substance called pheromone while moving on the path. The other ants sense the intensity of the pheromone and follow the path with a higher concentration of pheromone. This is their tactic to find an optimized path. Initially, the ants wander randomly to find their way to the destination. On their back tour, the ants sense the pheromone intensity and choose the path with a higher concentration of pheromone. The pheromone evaporates with time and hence the concentration of pheromone would be higher along the shortest path as the time taken to cover the shortest path would be minimal as compared to other paths. Hence, almost every ant would be attracted by the higher intensity of the pheromone along the shortest path and selects the optimized path. The philosophy of ant behavior can be summarized as follows. For the kth ant at position i to move to the next node j, the following probability formula is used [28,29]: where α and β are the degrees of importance of pheromones and heuristic function, respectively, τ ij is the pheromone concentration on the path between i and j, and η ij is the heuristic function, i.e., which is equivalent to the reciprocal of the distance between the i and j positions.

Path Retracing and Pheromone Updating
After ants complete their tour, the pheromone trial values are updated according to [30]: where ρ is the pheromone decay parameter range ∈ (0, 1) to mimic the evaporation of the pheromone and ∆τ ij is the amount of pheromones added by all the ants.
where ∆τ k ij is the amount of pheromones added by the kth ant, Q is the pheromone update constant, N is the total number of ants in the nest, and L k is the length of the path traveled by the kth ant.

Aging-Based Ant Colony Optimization (ABACO)
In standard ACO, the amount of pheromone deposits by the ants is assumed to be constant. Pheromone control can be used to reduce the influence of non-optimal solutions and encourages the exploration of new paths that are near-optimal or optimal. Bad experiments can also be reduced by controlling the amount of pheromones for each ant according to its age. Ant aging is based on the rationale that old ants are less successful in locating the optimal paths, since they take a longer time to reach their destination. Both aging and evaporation encourage the discovery of new paths that are previously non-optimal. The amount of pheromone released by each ant is given by Equation (13), and Equation (14) indicates that ∆τ ij (k) varies according to the age of the ant. The inclusion of the age of ants is suggested by allocating random values of Q to different ants. This is reflected in the updated pheromones in Equation (14), being different for each ant. Hopefully, the assignment process for the random values of Q is done in such a way that the ant with the shortest path, i.e., the younger ant, gets a higher value of Q, which in turn has a higher value of pheromone. In contrast, the ant with the longest path, i.e., the old ant, gets a lower pheromone update, which means it is assigned a lower value of Q. Figure 4 shows the flow chart of the aging-based ant colony optimization (ABACO) algorithm. updated pheromones in Equation (14), being different for each ant. Hopefully, the assignment process for the random values of Q is done in such a way that the ant with the shortest path, i.e., the younger ant, gets a higher value of Q, which in turn has a higher value of pheromone. In contrast, the ant with the longest path, i.e., the old ant, gets a lower pheromone update, which means it is assigned a lower value of Q. Figure 4 shows the flow chart of the aging ant colony optimization (AACO) algorithm. The optimization problem of finding the shortest distance will be solved by AACO, which will be explained next in this paper. The optimization problem of finding the shortest distance will be solved by ABACO, which will be explained next in this paper.

Algorithmic Iteration Concepts and Feasible Solutions
In the ABACO algorithm, the solution from the start point to the goal point through various layers in between represents one complete iteration. Each ant can select one cell in each layer in accordance with the state transition rule given by Equation (11). Figure 5 depicts three iterations with two ants; each ant constructs its solution, i.e., a series of nodes selected by each ant from the start point (SP) to the goal point (GP). The best path is obtained after a certain number of iterations with a higher amount of pheromone, i.e., the shaded cells, i.e., the green cells, represent the best solution.

Algorithmic Iteration Concepts and Feasible Solutions
In the AACO algorithm, the solution from the start point to the goal point through various layers in between represents one complete iteration. Each ant can select one cell in each layer in accordance with the state transition rule given by Equation (11). Figure 5 depicts three iterations with two ants; each ant constructs its solution, i.e., a series of nodes selected by each ant from the start point (SP) to the goal point (GP). The best path is obtained after a certain number of iterations with a higher amount of pheromone, i.e., the shaded cells, i.e., the green cells, represent the best solution.

Proposed Path Planning Algorithm
Since the mobile robot is a physical body, the obstacles are expanded by the radius of the mobile robot , i.e., to take into account the actual size of the mobile robot, and then the mobile robot is

Proposed Path Planning Algorithm
Since the mobile robot is a physical body, the obstacles are expanded by the radius of the mobile robot r MR , i.e., to take into account the actual size of the mobile robot, and then the mobile robot is considered as a point. Figure 6 illustrates the expansion of the obstacle size. The flowchart for a mobile robot path planning in grid static and dynamic environments using ABACO is shown in Figures 7  and 8, respectively.

Proposed Path Planning Algorithm
Since the mobile robot is a physical body, the obstacles are expanded by the radius of the mobile robot , i.e., to take into account the actual size of the mobile robot, and then the mobile robot is considered as a point. Figure 6 illustrates the expansion of the obstacle size. The flowchart for a mobile robot path planning in grid static and dynamic environments using AACO is shown in Figures 7 and 8, respectively.

Effect of Design Parameters on AACO
This section presents the influence of design parameters: number of iteration, ants, and evaporation factor ( ) on a global search. The results were applied to 18 × 18 (m 2 ) and 28 × 28 (m 2 ), as given below:

Number of Iterations
From Figure 9a, it is evident that the algorithm found an optimal path with an increase in the total number of iterations. As shown in Figure 9a, at iteration (70), the path lengths were (26.3848 m) and (28.5563 m) for 18 × 18 (m 2 ) and 28 × 28 (m 2 ) map sizes, respectively. Additionally, the whole time required is increased as shown in Figure 9b.

Effect of Design Parameters on ABACO
This section presents the influence of design parameters: number of iteration, ants, and evaporation factor (ρ) on a global search. The results were applied to 18 × 18 (m 2 ) and 28 × 28 (m 2 ), as given below:

Number of Iterations
From Figure 9a, it is evident that the algorithm found an optimal path with an increase in the total number of iterations. As shown in Figure 9a, at iteration (70), the path lengths were (26.3848 m) and (28.5563 m) for 18 × 18 (m 2 ) and 28 × 28 (m 2 ) map sizes, respectively. Additionally, the whole time required is increased as shown in Figure 9b.

Number of Ants
By changing the total number of ants in the colony from (10) to (100), as shown in Figure 10a, the best path lengths obtained for 18 × 18 (m 2 ) map and 28 × 28 (m 2 ) were (26.3848 m) and (28.5563 m), respectively, with 40 ants in the colony, as shown in Figure 10a,b. In addition, the total time required increased, as shown in Figure 10b.

Number of Ants
By changing the total number of ants in the colony from (10) to (100), as shown in Figure 10a, the best path lengths obtained for 18 × 18 (m 2 ) map and 28 × 28 (m 2 ) were (26.3848 m) and (28.5563 m), respectively, with 40 ants in the colony, as shown in Figure 10a,b. In addition, the total time required increased, as shown in Figure 10b.

Evaporation Factor (ρ)
The best value of evaporation factor ( ) was between (0.3) and (0.7) for all dimensions of the searching space, as shown in Figure 11a, and had no effect on the computation time, as depicted in Figure 11b.

Evaporation Factor (ρ)
The best value of evaporation factor (ρ) was between (0.3) and (0.7) for all dimensions of the searching space, as shown in Figure 11a, and had no effect on the computation time, as depicted in Figure 11b.

Static Environment with Grid-Based Modeling
The AACO algorithm applied for path planning plans the entire path from the start point up to the endpoint in one complete iteration.

Static Environment with Grid-Based Modeling
The ABACO algorithm applied for path planning plans the entire path from the start point up to the endpoint in one complete iteration.

Grid-Based Static Environment 1
In this experiment, the size of the search space was (18 × 18 m 2 ) per grid cell, and there were eight obstacles with different sizes and locations, as shown in Table 1. After executing the ABACO algorithm 10 times, the shortest path length was (26.3848 m) at iteration (54) with different execution times, ranging from (0.2255 min) to (0.3057 min). The best path obtained (see Figure 12a) and the convergence curve for this environment (See Figure 12b) are shown in Figure 12a,b respectively. In this experiment, the size of the search space was (18 × 18 m 2 ) per grid cell, and there were eight obstacles with different sizes and locations, as shown in Table 1. After executing the AACO algorithm 10 times, the shortest path length was (26.3848 m) at iteration (54) with different execution times, ranging from (0.2255 min) to (0.3057 min). The best path obtained (see Figure 12a) and the convergence curve for this environment (See Figure 12b) are shown in Figure 12a,b respectively.

Grid-Based Static Environment 2
In this experiment, the size of the search space was (28 × 28 m 2 ) per grid cell, and there were 14 obstacles with different sizes and locations, as shown in Table 2. After running AACO 10 times, the shortest path length was found to be (28.5563 m) at iteration (48), with different execution times ranging from (0.8231 min) to (0.875 min). The best path obtained

Grid-Based Static Environment 2
In this experiment, the size of the search space was (28 × 28 m 2 ) per grid cell, and there were 14 obstacles with different sizes and locations, as shown in Table 2.  (11:12, 18:26) After running ABACO 10 times, the shortest path length was found to be (28.5563 m) at iteration (48), with different execution times ranging from (0.8231 min) to (0.875 min). The best path obtained (see Figure 13a) and the convergence curve for this environment (see Figure 13b) are shown in Figure 13a,b, respectively.
Sensors 2020, 20, x FOR PEER REVIEW 15 of 27 (see Figure 13a) and the convergence curve for this environment (see Figure 13b) are shown in Figure  13a,b, respectively.
(a) (b) Figure 13. The best path achieved within the AACO Grid-based static environment 2.

Grid-Based Dynamic Environment 1
This environment consisted of three moving obstacles, defined in Table 3: The starting point (SP) at the location (5, 3) grid coordinate, the goal point (GP) at the (15,15) grid coordinate, and the r MR at (0.5).  Table 4 shows the obtained solutions and their corresponding computation times using the ABACO algorithm, where the optimize fitness function is defined as in Equation (8). From Table 4, it can be concluded that run number (1) gave the best results with the shortest path length of (19.7279 m) and computation time (1.3876 min) (see bold values). Figure 14 shows the planned path of the mobile robot while avoiding three dynamic obstacles with different velocity, and each obstacle moved according to Equations (9) and (10), then the mapping in the new position from (x, y) coordinated to the corresponding (r, c) grid coordinate.  Figure 14. The best path achieved within the AACO grid-based dynamic environment 1.

Grid-Based Dynamic Environment 2
This environment included three moving obstacles with initial locations and orientation, as defined in Table 5: The start point (SP) at the location (4, 15) grid coordinate, the goal point )GP) at the (17, 5) grid coordinate, and the is (0.5).  Figure 14. The best path achieved within the ABACO grid-based dynamic environment 1, (a) the mobile robot starts from its starting position, (b) the mobile robot is moving toward its goal position, (c) the mobile robot is at its midway and is avoiding the obstacles, (d) the mobile robot is approaching its goal position.

Grid-Based Dynamic Environment 2
This environment included five moving obstacles with initial locations and orientation, as defined in Table 5: The start point (SP) at the location (4, 15) grid coordinate, the goal point (GP) at the (17,5) grid coordinate, and the r MR is (0.5).  Table 6 shows the achieved solutions and computation times for 10 executions using the ABACO algorithm. The best results are found at run number (3) with the shortest path length of (23.7279 m) and computation time (7.4359 min) (see bold values). Figure 15 shows the planned path of the mobile robot while avoiding five dynamic obstacles with different velocity and direction. Each obstacle moved according to Equations (9) and (10), then the mapping of the new position from (x, y) coordinated to the corresponding (r, c) grid coordinate.

Comparison with Other Previous Works
In order to show the efficiency of the proposed algorithms, these algorithms were compared with other algorithms in different researches. For a fair comparison, the same assumptions, including the map dimension, number of obstacles, and consideration of the mobile robot as a single point, were used.
Performance Evaluation for AACO in the Static Grid environment. In Reference [31], three algorithms were implemented to solve path planning in a static grid for two environments. These algorithms were the pattern search (PS) algorithm, the genetic algorithm (GA), and particle swarm optimization (PSO). Figure 16 represents the first environment with five static obstacles, while the second environment consists of four static obstacles, as shown in Figure 17. The proposed algorithm (AACO) applied to both environments. The comparison results between all these algorithms are listed in Table  7. Figure 15. The best path achieved using the ABACO -based path planning algorithm in grid-based dynamic environment 2, (a) the mobile robot starts from its starting position, (b) the mobile robot is moving toward its goal position, (c) the mobile robot is at its midway and is avoiding the obstacles, (d) the mobile robot is approaching its goal position.

Comparison with Other Previous Works
In order to show the efficiency of the proposed algorithms, these algorithms were compared with other algorithms in different researches. For a fair comparison, the same assumptions, including the map dimension, number of obstacles, and consideration of the mobile robot as a single point, were used. Now, we discuss the Performance Evaluation of the ABACO based path planning algorithm in a Static Grid environment. In [31], three algorithms were implemented to solve path planning in a static grid for two environments. These algorithms were the pattern search (PS) algorithm, the genetic algorithm (GA), and particle swarm optimization (PSO). Figure 16 represents the first environment with five static obstacles, while the second environment consists of four static obstacles, as shown in Figure 17. The proposed ABACO algorithm was applied to both environments and the comparison results between all these algorithms are listed in Table 7.   Table 7 shows that the shortest path for the first and second environments was obtained using the AACO algorithm, which were (13.8995) and (14.4853), respectively (see bold values). The improvement ratio (IR) of the proposed algorithm compared with other algorithms is calculated by the following relation: where is the proposed algorithm and is the other algorithms. The IR for the AACO, compared with algorithms listed in Table 7, is presented in Table 8, in terms of the shortest distance.
In Reference [16], the case study was performed using Search Results Web results   Table 7 shows that the shortest path for the first and second environments was obtained using the ABACO algorithm, which were (13.8995) and (14.4853), respectively (see bold values). The improvement ratio (IR) of the proposed algorithm compared with other algorithms is calculated by the following relation: where PA is the proposed algorithm and OA is the other algorithms. The IR for the ABACO, compared with algorithms listed in Table 7, is presented in Table 8, in terms of the shortest distance. In [16], the case study was performed using artificial bee colony (ABC) optimization algorithms. The environment consisted of a 20 × 20 m 2 grid with different obstacles. The start position was from the middle-top of the grid to the middle-bottom of the grid as a goal point. The ABC algorithm was applied to obtain the shortest distance. The ABACO was applied to the same environment and obtained the results, as shown in Figure 18. The best path obtained using ABC, as shown in Figure 18a, was (25.9706), while the path obtained using ABACO algorithm, as shown in Figure 18b, was 23.1421 m for the same environment. The improvement ratio (IR) for the ABACO over the ABC was (10.89%). Artificial bee colony (ABC) optimization algorithms. The environment consisted of a 20 × 20 m 2 grid with different obstacles. The start position was from the middle-top of the grid to the middle-bottom of the grid as a goal point. The ABC algorithm was applied to obtain the shortest distance. The AACO was applied to the same environment and obtained the results, as shown in Figure 18. The best path obtained using ABC, as shown in Figure 18a, was (25.9706), while the path obtained using AACO, as shown in Figure 18b, was 23.1421 m for the same environment. The improvement ratio (IR) for the AACO over the ABC was (10.89%).

Conclusions
The path planner based on ABACO and grid-based modelling is proposed in this paper to find the optimal/near-optimal path of the mobile robot in static and dynamic environments. The age of the ant was taken into consideration to produce a new kind of ACO optimization. which is exploited in the design of mobile robot path planning. Grid-based modelling was found to be less flexible (hard of implementation) in dynamic environments as compared to free-space-based modelling. However, it easily modelled the mobile robot and the obstacles in the 2D-environment. When the proposed ABACO algorithm was applied to mobile robot path planning, it was found that the proposed ABACO algorithm outperformed the standard ACO algorithm in terms of the shortest path. Based on the comparison results, the ABACO algorithm is superior to GA, PSO, and ABC. Finally, to overcome the unused space inside the cell using grid-based modelling, a future work will be conducted by increasing the resolution of the grid, which will supposedly help in reducing the waste of space of the individual cells and result in a smoother path from the start to end points.