To design the proposed algorithm, basic rules should be defined to determine the order in which the drone visits the grid. Therefore, the basic rules used in the proposed algorithm are explained with some examples, and then the proposed algorithm is explained in detail.
4.1. Basic Rules
In general, the UAV performs the CPP through spiral and boustrophedon maneuvers [
33]. The spiral maneuver is a method of flying inward by starting the circular flight from the outskirts of the ROI and gradually reducing the radius. The flight direction is set to either clockwise or counterclockwise. It takes a short return time compared to the boustrophedon method if the ROI area is small or there is enough flight time to scan the entire ROI at once. The boustrophedon maneuver is a flight method in which the drone gradually advances while sweeping left to right or up to down. Unlike the spiral maneuver, it has the advantage of gradually flying from a familiar area to an unknown area. Both methods can be applied regardless of the shape of the ROI; the flight patterns for both methods are depicted in
Figure 1.
However, these two methods have two problems. Even if the entire ROI can be scanned in one flight, as shown in
Figure 1, it is inefficient because the distance to return to the starting point is long. The other problem occurs when the ROI is huge or the maximum flight time of the drone is short. In this case, multiple flights are required to scan the entire ROI. In other words, multiple paths should be generated to cover the ROI. To generate multiple paths, methods of generating paths by applying the flight direction shown in
Figure 1 or by dividing the ROI evenly can be considered.
Figure 2 shows the expected results when multiple paths are generated by the methods mentioned above, based on the spiral and boustrophedon maneuvers. Note that the color of the visited grids by each path is displayed differently in
Figure 2. Also, the color of the flight directions for each path is depicted differently to distinguish between paths.
Figure 2a shows the expected CPP result considering the flight direction of spiral motion. The spiral motion derives a path starting from the outskirts of the ROI and gradually reducing the radius towards the center of the ROI. So, the drone scans along the red line on the outer edge of the ROI, starting from a corner grid close to the drone’s initial position. However, the drone can only scan a part of the ROI, so it has to return during the scan. After replacing the drone’s battery, the drone will scan along the green line on the outer edge of the remaining ROI, starting from where the red path ends since the drone should scan the ROI except for the red area. Following this procedure, the drone will fly along the lines in the following order: red, green, blue, purple, pink, and yellow. Note that the farther the distance from the drone’s initial position to the start/end grid of the scan, the smaller the number of grids that the drone can scan. So, the number of flights may increase unnecessarily to scan small unvisited areas if the drone flies using this method.
Figure 2b shows the expected CPP results by dividing the ROI evenly and using spiral motion. For CPP for multiple robots, a technique that simply divides the ROI area equally into the number of robots is widely used [
34]. To apply this method, it assumes that the drone has enough maximum flight time to scan each divided area and return.
Figure 2b shows the result of dividing the ROI into four equal-sized parts and generating paths through the spiral motion. As can be expected from the figure, the distance from the initial position of the drone to the start/end point of each path is different, so the time required to fly each path is different. This arrangement results in further increasing the time required to scan the entire ROI.
Figure 2c shows the expected CPP result considering the flight direction of boustrophedon motion. The boustrophedon motion creates a path that sweeps forward, starting from the ROI edge closest to the drone’s initial position. So, the drone scans along the red line on the bottom of the ROI, starting from a corner grid close to the drone’s initial position. However, the drone can only scan a part of the ROI, so it has to return during the scan. After replacing the drone’s battery, the drone will scan along the green line on the bottom of the remaining ROI starting from where the red path ends since the drone should scan the ROI except the red area. Following this procedure, the drone will fly along the lines in the following order: red, green, blue, purple, and pink. The farther the distance from the drone’s initial position to the start/end grid of the scan, the smaller the number of grids that the drone can scan, as shown in
Figure 2a. So, the number of flights may also increase unnecessarily to scan small, unvisited areas.
Figure 2d shows the expected CPP results by dividing the ROI evenly and using boustrophedon motion. It also assumes that the drone has enough maximum flight time to scan each divided area and return.
Figure 2d shows the result of dividing the ROI into four equal-sized parts and generating paths through the boustrophedon motion. As in
Figure 2b, the distance from the initial position of the drone to the start/end point of each path is different, so the time required to fly each path is different. This arrangement results in further increasing the time required to scan the entire ROI.
As can be seen above, existing flight direction determination methods cannot be used to generate efficient multiple coverage paths. Therefore, a novel flight direction determination method to generate efficient multiple coverage paths is proposed. The proposed flight direction determination method is based on the basic rules for generating efficient coverage paths, which are described below. Note that DIP means the drone’s initial position in the rules.
Rule 1. The closer the start and end grids of the path are to the DIP, the more efficient the path is since it increases the number of grids that can be scanned within the maximum flight time.
Rule 2. The path is efficient to start the path from a grid close to the DIP, move away from the DIP, and then return to the vicinity of the DIP. If one path is only generated near the DIP, the efficiency of the path is good but the other paths are less efficient because they start and end far from the DIP.
Rule 3. The path should be generated without any space. If the path is generated in a way that encloses some small spaces, it is necessary to fly once more to fill that empty space. It may increase the makespan to cover the whole ROI.
Rule 4. The fewer paths are to cover the entire ROI, the more efficient the CPP. As the number of paths increases, the number of times the drone lands at the DIP increases. This means the time the drone does not perform a scan, which includes the time to fly from the DIP to the start point of the path, the time to return from the end of the path to the DIP, and the time to replace the drone’s battery from the DIP, increases.
Rule 5. If the time required to cover the ROI is the same, reducing the maximum time required to fly along a path is effective when considering multiple drone operations. For example, if the time required to fly two paths is 10 s and 5 s, respectively, the time required to operate one drone is 15 s plus battery replacement time while the time required to operate two drones is 10 s only. If the time required to scan the same area is 7.5 s for both paths, the time required to operate one drone is the same, but the time required to operate two drones is 7.5 s.
As seen in Rules 1, 2, and 4, the initial position of the drone should be considered for deriving an efficient flight direction.
Figure 3 shows the proposed flight direction of the drone considering the drone’s initial position.
According to Rule 1, each path should start near the initial position of the drone. The fifth and sixth grids in the bottom row of the ROI can be selected as the initial position. If the fifth grid is selected as the first grid of the path, the flight direction is made, as shown in
Figure 3a. The drone should start near the initial position of the drone and move away from it, as mentioned in Rule 2, so it will fly along the bottom side of the ROI and then go up along the left side. This part is similar to the spiral motion in
Figure 1a. When the drone reaches the upper left, it will fly to the right to scan the upper edge, which is the farthest from the drone’s initial position. The drone will go down one grid and then fly to the left since there should be no empty grid while scanning, according to Rule 3. To avoid creating empty grids, the drone will fly in the same way as the boustrophedon maneuver in
Figure 1b, afterward.
Figure 3a can be derived by setting the priority of directions to left > up > right > down. When flying to the left from the first grid and reaching the left side of the ROI, it is impossible to fly to the left so the drone will fly upward, which is the next priority. If the drone reaches the upper side of the ROI, it cannot fly to the left or upward so it flies to the right, which is the next priority. Since the drone cannot fly to a grid that has already been visited, it can reach the right side of the ROI. Then, the drone cannot fly to the left, up, or right, so it will move downward, which is the last priority. When the drone goes down one grid, there are unvisited grids on the left, so it will fly to the left. If the sixth grid in the bottom of the ROI is selected as the first grid of the path, the flight direction is made, as shown in
Figure 3b, following the same logic as explained above. In that case, the priority of directions is set to right > up > left > down. If the drone can scan the whole ROI through one flight, flying in this way satisfies rules 1 and 2 because the final position of the path will be near the drone’s initial position. Note that even if the initial position of the drone is not at the bottom of the ROI, the direction priority can be set by referring to
Figure 3.
4.2. Main Algorithm
Once the drone’s flight direction has been determined, multiple paths for covering the ROI should be created based on it. The proposed CPP algorithm consists of five steps: ROI adjustment, initial GRID selection, GRID path generation, rebalancing multiple path, and multiple coverage paths generation. The meaning of the word ”GRID” will be explained in
Section 4.2.1.
4.2.1. Adjust Region of Interest
When the ROI is set by the user, the ROI should be divided into several small grids to generate coverage paths based on the ADO. If there is no overlap between the captured areas, it is impossible to confirm whether the photos were taken at the desired position and the photos cannot be aligned. Therefore, the grid is generally set to be smaller than the ADO, considering the overlap rate between the photos, as depicted in
Figure 4 [
35].
The ADO is determined as the largest rectangular area within the area determined by the FOV of the camera mounted on the drone and the height of the camera, as shown in
Figure 4a. The ratio of the rectangular ADO is determined by the ratio of the image sensor in the camera.
Figure 4b shows the areas when viewed directly downward. The yellow circle and blue rectangle mean the FOV region of the camera and the ADO, respectively. If the ADO is reduced by the overlap ratio, a rectangular area created by four black dashed lines is derived. If the rectangular area is set to a grid area, the distance to the next grid will vary depending on the flight direction. Therefore, the square area created based on the short side of the rectangular area, indicated by the red square in
Figure 4b, is set as the grid area.
Once the grid size is determined, the ROI can be divided into grids. If the path is generated by simply dividing the ROI into grids, it is not guaranteed that the last grid of the path will be near the initial position of the drone even if the proposed flight direction is applied. Therefore, a GRID is formed by combining four grids
, and the area of the ROI is adjusted so that the ROI is exactly divided into the GRID to guarantee the return path. The pseudo-code for the ROI adjustment step is described in Algorithm 1.
Algorithm 1. ROI Adjustment step |
1: | input |
2: | |
3: | , |
4: | |
5: | |
6: | , |
7: | |
8: | if |
9: |
if
|
10: |
|
11: |
, |
12: |
else |
13: |
|
14: |
, |
15: |
end if |
16: | end if |
17: | if |
18: |
if
|
19: |
|
20: |
, |
21: |
else |
22: |
|
23: |
, |
24: |
end if |
25: | end if |
26: | |
In line 1, the inputs are received from the user to generate paths for covering the ROI, where and are the lower left and upper right coordinates of the ROI, respectively; is the height of drone operation; is the camera FOV angle; is the ratio of image sensor’s size of the camera; and is the overlap rate. Then, the size of the ADO is calculated through lines 2–3, where is the diagonal length of the ADO, and and are the width and height of the ADO, respectively. In line 4, the size of the grid is calculated by reducing the short side of the ADO by the overlap ratio, where and are the width and height of the grid, respectively. The size of the GRID is calculated by doubling the width and height of the grid in line 5, where and are the width and height of the GRID, respectively. In line 6, the initial size of the ROI is calculated, where and are the width and height of the ROI, respectively. If and are not multiples of and , respectively, the size of the ROI is adjusted to make and multiples of and , respectively, through lines 8–25. For each of the horizontal and vertical axes, the sizes of the adjustment values for increasing and decreasing the ROI are compared, and the adjustment is performed in a smaller direction. Then, in Equations (1) and (6) can be calculated as described in line 26.
4.2.2. Initial GRID Selection
Once the size of the ROI is adjusted, the number of paths required to scan the entire ROI is roughly estimated. The maximum number of GRIDs that can be scanned within the drone’s maximum flight time is calculated assuming that the drone starts from the GRID closest to its initial position and ends at that GRID, and then returns to its initial position. The number of paths required to scan the entire area is roughly calculated by dividing the total number of GRIDs by the maximum number of GRIDs derived previously. Note that the number of GRIDs that a drone can scan in one flight and the number of paths required to scan the entire ROI are adjusted in the subsequent algorithms.
After determining the approximate number of paths, a determined number of paths that cover the entire ROI should be created. According to Rule 1, the closer the starting grid of the path is to the initial position of the drone, the better. If a method that first selects the starting GRIDs of all paths close to the drone’s initial position and then completes the paths sequentially is used, it may hinder the progress of each path during the step for generating multiple coverage paths. Therefore, the proposed algorithm uses a method of generating the next path after completely generating one path. To generate each path, an initial GRID should be selected first, and then the GRID path should be generated according to the maneuver methods specified in
Section 4.1. The method of selecting the initial GRID of the path is presented in pseudo-code in Algorithm 2.
Algorithm 2. Initial GRID selection |
1: | , n = # of horizontal GRIDs in ROI |
2: | for i = 1: n |
3: |
|
4: |
|
5: | end for |
6: | Sort |
7: | (1: # of reserved GRID path), |
In line 1, the index list and the distance vector , which have as many elements as the number of horizontal GRIDs of the ROI, are initialized, where is the number of horizontal GRIDs in the ROI. In lines 2~5, the unassigned GRID that is close to the initial position of the drone in each column is found, and the distance from the initial position of the drone to that GRID is calculated, where and are the initial position of the drone and the position of the GRID, respectively. The calculated distance vector is sorted in ascending order in line 6, and the column with the largest value in the top of the generated vector is found, where is the number of GRID paths that are not generated in line 7. The smallest unassigned row in that column is selected as the first GRID of the path, where and are the horizontal and vertical index when the ROI is divided into GRIDs, respectively. Note that Algorithm 2 is used when the initial position of the drone is at the bottom of the ROI and can be applied by modifying line 3 if the initial position is in a different direction of the ROI.
4.2.3. Make GRID Path
After the starting GRID of a path is determined, the remaining path of the GRID should be generated taking into account the maximum flight time of the drone. The GRID path can be expended in four directions (up, down, left, right), and furcating is possible from the previously visited GRID if the path is blocked, as shown in
Figure 5. The reason is to create a GRID path-based grid path that starts from the initial GRID, flies the path in four directions, and then returns to the initial GRID.
In
Figure 5, it is assumed that the path of the red line passing through all the red GRIDs is generated first. Since the red path does not cover the entire ROI, the green line path is additionally generated based on the flight direction described in
Section 4.1. However, the green path is blocked by the previously generated path. So, the black path is generated by branching from the previously visited GRID. GRID paths that cover the entire ROI are generated through this procedure. The procedure to extend the GRID path according to the flight direction, as defined in
Section 4.1, is described in Algorithm 3.
Algorithm 3. Make GRID path |
1: | |
2: | ) |
3: |
|
4: |
priority = [left, up, right, down] |
5: |
else |
6: |
priority = [right, up, left, down] |
7: |
end if |
8: |
|
9: |
AddOneGRIDtoEnd(priority) |
10: |
if not added |
11: |
AddOneGRIDtoMiddle (priority) |
12: |
end if |
13: |
end if |
14: |
if not added |
15: |
AddOneGRIDtoFirst (priority_inverse) |
16: |
end if |
17: |
if added |
18: |
Calculate Makespan |
19: |
else |
20: |
break |
21: |
end if |
22: | end while |
A GRID path is created by sequentially adding GRIDs one by one until no more GRIDs can be added to the current path. In line 1, the incremental time when a GRID is added is calculated, where
is the length of one side of the grid,
is the horizontal velocity of the drone, and
is the hovering time at the center of the grid. In lines 3~7, as explained in
Section 4.1, the priority is set to left > up > right > down if the initial GRID position is to the left of the drone’s initial position, or it is set to right > up > left > down otherwise, where
means the
coordinate values of the initial GRID. If the makespan does not exceed the drone’s maximum flight time when one GRID is added to the path, adding one GRID at the end of the path is tried first in line 9, based on the priority in lines 4 or 6. If adding one GRID fails, adding one GRID by furcating the previously visited GRIDs is tried based on the priority. This procedure is also based on the determined priority. If this also fails, adding one GRID before the starting GRID of the path is tried in line 15. Note that the priority used here is the inverse of the priority determined in lines 4 or 6. Here, the inverse means swapping left and right, up and down, respectively. For example, the inverse priority will be right > down > left > up if the priority is left > up > right > down. It should be added with inverse priority before the first GRID of the path so that the previously created path is added to the changed starting GRID with the determined priority. If a GRID is added to the path, the makespan is calculated in line 18. Note that the makespan simply increases by
if a GRID is added by line 9 or 11, but if a GRID is added in line 18, the starting GRID changes, so the time to go back and forth from the drone’s initial position to the starting GRID changes, requiring recalculation of the makespan. If a GRID cannot be added to the path, the path extension is stopped and the loop is exited in line 20.
4.2.4. Rebalancing
If the number of paths roughly determined in
Section 4.2.2 is not enough to cover the entire ROI, the number of paths is increased by one, and Algorithms 2 and 3 are performed again. This process is to minimize the number of paths to cover the entire ROI, as mentioned in Rule 4. When multiple paths covering the entire ROI are generated through this procedure, the generated paths are rebalanced to find the optimal solution according to Rule 5. The rebalancing process is to repeatedly remove one GRID from the path with the largest makespan and give it to the path with the smallest makespan, as described in Algorithm 4.
Algorithm 4. Rebalancing |
1: | |
2: | while true |
3: |
|
4: |
|
5: |
|
6: |
|
7: |
|
8: |
|
9: |
|
10: |
for i = 1: |
11: |
if |
12: |
|
13: |
|
14: |
|
15: |
break |
16: |
else if |
17: |
break |
18: |
end if |
19: |
end for |
20: |
if makespan did not changed |
21: |
break |
22: |
end if |
23: | end while |
In line 1, the incremental time when a GRID is added to a GRID path is calculated, as in line 1 of Algorithm 3. In line 3, the makespans are copied to for further processing. In lines 4~6, is sorted in descending order to find path , which has the largest makespan, and path , which has the smallest makespan. The makespans that are changed when a GRID is removed from and given to is calculated in lines 7~8. The changed makespans are also sorted in descending order to compare with in line 9. In lines 10~19, is compared with for each element to check whether is more efficient than the previous makespan, where means the number of GRID paths. If it is better than the previous one, the number of GRIDs in and are adjusted and the makespan is renewed, where and are the number of GRIDs in and , respectively. If not, the previous solution is maintained. In line 21, the while loop is exited if there is no change in the makespan. After that, Algorithm 3 is run so that each path is assigned the number of GRIDs derived from Algorithm 4.
4.2.5. Make Multiple Coverage Paths
Once multiple GRID paths covering the entire ROI are created, a grid path should be generated based on each GRID path. The grid path is created in a way that surrounds the GRID path. In other words, the grid path starts from the starting GRID, reaches the last GRID, and then returns to the starting GRID. Since the starting GRID is selected close to the initial position of the drone, the grid path satisfies Rule 2 described in
Section 4.1. The method of generating a grid path based on the GRID path is described in Algorithm 5.
Algorithm 5. Make Multiple Coverage Paths |
1: | for i = 1: |
2: |
for j = 1: |
3: |
if |
4: |
|
5: |
|
6: |
|
7: |
|
8: |
|
9: |
else if and are neighbors each other |
10: |
|
11: |
insert near the path associated with among |
12: |
else |
13: |
find where and are neighbors each other, |
14: |
|
15: |
insert near the path associated with among |
16: |
end if |
17: |
end for |
18: | end for |
In lines 3~16, a grid path is expended sequentially along the GRID path. If the selected GRID is the first GRID in the GRID path,
, the positions of four grids, are calculated based on the GRID
in line 4. The distance from each grid to the initial position of the drone is calculated in line 5, where
is the distance vector from each grid to the initial position of the drone. The two shortest grids are set as the start and end points of the path,
and
, in lines 6~7. Note that the fourth grid becomes the endpoint if a path is generated for the first GRID since one GRID consists of four grids. The grid path is generated in line 8 by determining the order of the remaining two grids so that they can reach the end point by moving only up, down, left, and right sequentially from the start point, as shown in
Figure 6.
Here, the square with a black border represents the first GRID of the GRID path. The four squares divided by gray dashed lines represent grids. The red circle with ”S” and the gray circle with ”E” represent the start and end points of the grid path, respectively.
Figure 6 shows the results of grid path generation for all cases where the start and end points of the grid path can be selected from the first GRID, represented by red arrows. From the second GRID in the GRID path onwards, it is checked whether the previous GRID and the current GRID are neighbors. Here, neighbor means that the distance between the centers of the two GRIDs matches the length of a side of the GRID. If they adjoin each other, the current GRID is divided into four grids, and the path that visits all the generated grids is inserted near the path associated with the previous GRID among the existing grid path in lines 10~11, where
,
, and
mean the previous GRID, the current GRID, and the existing grid path, respectively. If they do not adjoin each other, a GRID
that is adjacent to the current GRID is found by going back from the previous GRID in the GRID path until it reaches the neighboring GRID of the current GRID in line 13. The current GRID is divided into four grids in line 14, and they are inserted near the path associated with
among
in line 15.
Figure 7 shows an example of grid path generation when the previous GRID and the current GRID are neighbors and when they are not.
In
Figure 7, the blue line and the red dotted line mean before and after generating the grid path for the current GRID, respectively. In
Figure 7a, it is assumed that the first and the second GRIDs in the GRID path are the GRIDs on the right and left, respectively. The second GRID is divided into four grids in line 10 of Algorithm 5, as shown as the left GRID in
Figure 7a. Then, the red-dotted grid path is generated in line 11. Note that the red dotted path starts from the starting point, visits the entire grid generated by line 10, and reaches the endpoint by only moving up, down, left, and right. In
Figure 7b, it is assumed that the top three GRIDs are occupied by another GRID path. Then, the GRID path will be made in the order of the numbers written in the white box. Note that the fifth GRID is added by furcating from the first GRID. In line 13 of Algorithm 5, the adjacent GRID to the fifth GRID is found starting from the fourth GRID and going backward. The first GRID will be found to be adjacent to the fifth GRID, and four grids generated by dividing the current GRID in line 14 are added near the path associated with the first GRID in the existing grid path in line 15. As a result, the existing blue path is updated to a red dotted path. The whole pseudo-code of the proposed multi-CPP algorithm is shown in Algorithm 6.
Algorithm 6. Proposed Multi-CPP Algorithm |
1: | run Algorithm 1 to adjust the ROI |
2: | decide the number of GRID path roughly to cover the ROI |
3: | while true |
4: |
for i = 1: |
5: |
run Algorithm 2 to select the initial GRID for GRID path |
6: |
run Algorithm 3 to generate the GRID path |
7: |
end for |
8: |
if there are unvisited GRIDs |
9: |
|
10: |
else |
11: |
run Algorithm 4 for rebalancing |
12: |
initialize all GRIDs except the first GRID of each path |
13: |
for i = 1: |
14: |
run Algorithm 3 to generate the GRID path which has the number of
GRIDs determined in Algorithm 4 |
15: |
end for |
16: |
break |
17: |
end if |
18: | end while |
19: | run Algorithm 5 to generate grid paths |
When the user inputs for CPP are received, Algorithm 1 is run to adjust the ROI in line 1. Once the ROI adjustment is complete, the number of GRID paths is roughly determined in line 2 based on the GRID size determined in Algorithm 1 and the initial position of the drone. The GRID paths are generated in lines 5~6, as many as , and it is checked whether the generated GRID paths cover the entire ROI. If not, is increased by 1, and lines 5~6 are performed until the generated GRID paths cover the entire ROI. If the entire ROI is covered by the paths, Algorithm 4 is run for rebalancing the paths. In line 12, each GRID path is initialized, leaving only the initial GRID. Then, a GRID path with a path length equal to the number of GRIDs determined in Algorithm 4 is generated sequentially in lines 13~15. Once the GRID paths covering the entire ROI are generated, Algorithm 5 is run to generate multiple grid paths, which are the results of the proposed algorithm.