A Hierarchical Approach to Intelligent Mission Planning for Heterogeneous Fleets of Autonomous Underwater Vehicles

Abstract

The rapid development of marine robotic technology in recent decades has resulted in significant improvements in the self-sufficiency of autonomous underwater vehicles (AUVs). However, simple scenario-based approaches are no longer sufficient when it comes to ensuring the efficient interaction of multiple autonomous vehicles in complex dynamic missions. The necessity to respond cooperatively to constant changes under severe operating constraints, such as energy or communication limitations, results in the challenge of developing intelligent adaptive approaches for planning and organizing group activities. The current study presents a novel hierarchical approach to the group control system designed for large heterogeneous fleets of AUVs. The high-level core of the approach is rendezvous-based mission planning and is aimed to effectively decompose the mission, ensure regular communication, and schedule AUVs recharging activities. The high-level planning problem is formulated as an original acyclic variation of the inverse shift scheduling problem, which is NP-hard. Since regular schedule adjustments are supposed to be made by the robots themselves right in the course of the mission, a meta-heuristic hybrid evolutionary algorithm is developed to construct feasible sub-optimal solutions in a short time. The high efficiency of the proposed approach is shown through a series of computational experiments.

1. Introduction

The substantial advancement of robotic technologies in the past few decades, has led to significant reliability and autonomy improvements in the field of unmanned marine vehicles both surface and underwater. This technological leap has encouraged new trends in the development of autonomous mobile systems, the biggest of which is to combine multiple vehicles operating as a team into a single fleet improving the overall performance in terms of cost and safety [1]. On the other hand, the team-based application poses a number of new challenges, as the fleet performance in a dynamic environment relies more on the effectiveness of cooperation schemes than on the robustness of individual team members.

This paper aims to explore some of these challenges and address them in the proposed hierarchical approach of organizing a group control system for heterogeneous fleets of AUVs. Underwater vehicles are chosen in the paper specifically because they have the most severe communications constraints, although the approach could easily be extended for surface unmanned vehicles either.

In general, large-scale underwater operations require an AUVs fleet to visit a set of locations to perform some research activities under current operational requirements [2]. This problem of task and resource allocation can be mathematically formulated in terms of vehicle routing problems. As with any realistic real-world model, this formulation will feature a complex set of interrelated spatial, temporal and technical constraints and limitations, which makes it near impossible to be solved explicitly in a reasonable time [3].

This leads to the standard decomposition approach, where some restrictions are taken out of the problem and are satisfied by a standalone technique at the expense of overall efficiency and resource loss. To make this decomposition, the literature survey was carried out to explore which exact constraints can be satisfied in this way.

In the field of autonomous mobile vehicle routing, communication constraints are by far the most popular in the literature as efficient inter-vehicle communication is the foundation of any successful cooperation. Most commonly, such constraints are the requirement to ensure regular data sharing within a group under a limited communication range. In this context, two types of approaches can be distinguished: rigid and flexible schemes.

Rigid schemes restrict group movement to connectivity structures, ensuring that any permissible positioning of vehicles a priori allows conducting communication [4,5]. Unfortunately, this approach is not robust in the fully dynamic mission environment [6], although recently there have been improvements in this direction as well allowing to deal with communication delays and dropouts [7,8]. Rigid schemes also tend to be too restrictive, thus, preventing vehicles from accomplishing their tasks [9]. Flexible schemes, in turn, allow vehicles to work freely most of the time, ensuring only periodic network connectivity, usually only within specified meeting areas [9,10,11].

One more approach is exclusive to the underwater fleets. It involves a network of underwater acoustic communication systems (UACS) installed on stationary landers and/or movable surface vessels to ensure constant data transmission between the operating AUVs and the command centre [12]. It provides centralized control, but requires using extra resources and may not be robust to network breakdowns.

Another major type of constraint, which is often considered independently from the general task allocation problem, is the fuel (energy) constraint, which is also a significant limitation of long-term planning, especially for heterogeneous fleets [13]. Under this constraint, vehicles have to interrupt their activity to periodically replenish their battery. Problem-solving schemes for such models can also be divided into two classes: embedded approaches and rule-based logical approaches.

Embedded approaches treat the energy constraint as yet another spatio-temporal routing constraint, making it a so-called green variation of the multi-vehicle routing problem. Among the plenty of techniques presented in the literature to solve green transportation problems: exact methods seem to be too slow; fast heuristic methods scale too poorly [14]; the advanced metaheuristic approaches require too much problem-specific fine-tuning [15], and even being tuned still rarely deal with functionally heterogeneous fleets [16].

Rule-based logical approaches, as their name suggests, propose the development of a set of control rules from which each vehicle makes its own independent decision on recharging timing [17]. This approach is particularly effective when all vehicles are identical and replaceable, so it is mostly applied to robotic fulfilment systems and AGVs [18]. The set of rules usually regulates two types of behaviour: when to recharge and when to leave the recharging station. In the former case, a decision can be made based on the remaining energy [19] or the current robot workload [20]. For the latter type of decision, a negotiation approach [21] is usually applied with the emerging demands [22] as an alternative. A big advantage of this approach is its simplicity, reliability, and speed. At the same time, it appears to be too greedy, looking at the needs of the individual worker irrespective of the overall group state. Thus, it fails in many cases, such as charging facilities shortage or multi-skilled workers, due to an inability to predict group state.

Despite the huge variety of different approaches dealing with the two mentioned types of constraints, no relevant work has been found where both of these constraints are applied simultaneously to a heterogeneous group of mixed-capabilities robots, for which the accurate recharging order is especially vital. An approach capable of meeting such a challenge should combine the key features from both areas to arrange all events related to energy and communication requirements in a flexible and intelligent way. In this regard, time management becomes a fundamental requirement for complex mission success [23].

The paper’s contribution is in providing a versatile hierarchical approach to efficiently organize and control the implementation of long-term dynamic operations by heterogeneous AUVs fleets. The main idea behind the approach is to offload the mid-level planner, which is generally responsible for the computationally intensive problems of task allocation and route generation, by transferring some of its harshest constraints to the high-level mission planning problem. The designed high-level planner is based on the team rendezvous concept and provides the high-performance capability of the fleet throughout the mission while maintaining the communication and recharging requirements.

In this paper, a mathematical model for the high-level mission planning problem is presented and formulated in terms of shift scheduling problems with hard and soft constraints interchanged. The hybrid evolutionary algorithm to efficiently solve this problem is also developed and presented. This work extends our previous studies [2,24] on high-level scheduling by introducing a more accurate mathematical model, a completely redesigned evolutionary algorithm, which allowed us to significantly improve our previous results, and a few relevant simulation studies.

The remainder of the paper is organized as follows. The structure of the proposed hierarchical approach is presented in the next section. Section 3 provides a new mathematical model for the high-level planning problem. An in-detail description of the developed hybrid evolutionary algorithm is given in Section 4. Section 5 is devoted to the computational experiments and their discussion. The last section concludes the paper.

2. Hierarchical Control System

A generic mission for a fleet of AUVs, in general, can be represented as a vehicle routing problem with specific requirements and constraints of different nature [25]. In the autonomous mode, the construction of even approximative solutions becomes a difficult challenge when the dimensionality increases (in terms of both fleet size and time). And even solving the problem distributively by all vehicles is still an extremely resource-intensive process, especially when it comes to rich multi-attribute problem statements [3,26].

In this work, a hierarchical group control approach is proposed to significantly relieve the route planner workload. Here, an additional upper level is designed to coordinate the mission implementation ensuring that some concrete constraints are met a priori.

For this purpose, two severe types of constraints have been chosen, which are not mission-specific but seem to be relevant for most long-term fully autonomous operations. The first type is fuel or energy constraints, where vehicles are periodically forced to interrupt task execution due to the limited battery in order to replenish their energy supply at the specialized recharging facilities. This type of constraint poses two challenges at once: to determine an effective order for timely recharging of vehicles and to provide an efficient routing for a fleet with a dynamically changing composition. The second type is communication constraints, where group movement must be arranged to allow for regular control sessions with data interchanging under a limited communication range.

The nature of two of these constraints leads to the core idea of the approach: to coordinate teamwork based on group rendezvous scheduled to maintain regular communication and timely vehicle rotation (Figure 1).

A single rendezvous involves all active AUVs gathering at the specified area to hold a communication session. All preplanned changes in the fleet composition, such as individual AUVs leaving the group for recharging or returning back, should also be assigned to these group rendezvous. Thus, each rendezvous includes one or more events of two different types: gathering vehicles for communication needs and/or gathering vehicles to update the group composition (rotation activities).

Figure 1 shows an example of mission decomposition for a fleet of three colour-coded AUVs. Initially, all three vehicles participate in the mission implementation. When it is time for AUV #2 to leave for recharging, all three vehicles gather in the rendezvous location to update the current group state, as well as to form a strategy for the two AUVs remaining for the next shift. The duration of shift #2 is determined by the next expected event of the group composition change (AUV #1 leaving for recharging). Thus, all tasks on the subsequent shift #3 will be carried out by the remaining AUV #3, after which it will collect the already recharged vehicle #2. Since one of the shifts between line-up changes turns out to be too long, the group splits it into two shifts #5 and #6 to meet the communication frequency requirement. This splitting rendezvous does not include any composition changes and involves AUVs gathering for a communication session only.

With an efficiently constructed rendezvous schedule, this scheme can provide the group strategy with several advantageous features at once:

• Regular communication sessions of all vehicles with required periodicity to keep up with the up-to-date information including environmental and fleet state changes;
• Efficiently organized fleet rotation with centralized sending AUVs for recharging and collecting them after;
• Switching from long-term to a shorter planning horizon of a single shift (the period between two consecutive rendezvous);
• Static fleet composition on each shift in the absence of unforeseen malfunctions.

This scheme results in the following procedure of mission implementation. Initially, vehicles jointly produce an efficient rendezvous schedule and then make plans of individual activities for the upcoming shift. Following that, vehicles spread out across the operational area and perform their planned tasks, and then return to the rendezvous location at the start of the next communication session. After completing the scheduled composition changes (sending/receiving AUVs), the current team generates a plan for the next shift. In case of any significant condition changes, the fleet should first readjust the rendezvous schedule to address these changes properly.

The structure of the proposed approach is represented in Figure 2. The high-level planner is responsible for decomposing a mission into a sequence of shifts considering communication and energy requirements (Figure 1). A mission-specific mid-level planner performs efficient task allocation and path-planning for each shift.The construction of group routes at the middle level should not only address spatio-temporal factors of current mission objectives but also must ensure the validity and reliability of individual paths. The latter includes energy-saving and collision avoidance, along with the necessity for all vehicles to return to the rendezvous location by the end of the shift. Thus, the routing problem itself here is related in many ways to the team orienteering problem with non-identical agents, which can be treated by a variety of approaches [27]. Finally, low-level controllers provide a proper and safe implementation of the planned actions on the individual level.

Summarizing the above, the task of the high-level planner is to maintain the autonomous fleet in its best working order throughout the mission under dynamic conditions, communication requirements, and energy constraints by constructing efficient rendezvous schedules. To construct a group schedule, the mission planner should allocate multiple rendezvous in time and assign to them vehicles’ recharging activities and communication sessions in a feasible and efficient way. That allows one to elude from any detailed attributes of the problem-specific routing statements in order to provide a general time-management approach for large-scale heterogeneous fleets of AUVs with high autonomy.

Apart from the suggested high-level planner, which is the main paper contribution as it allows to implement of long-term autonomous fleet operations, the proposed scheme follows the common approach to simultaneously release available robots to execute tasks, having done accurate route planning beforehand, and then assemble them back for the next iteration of work [28,29].

3. Mathematical Formulation

Let T denote the overall mission length. For those cases where T is disproportionately large or even unknown (e.g., the fleet works until some specific event occurs), T can be taken as the duration of the medium-term planning horizon.

The active fleet of AUVs consists of n vehicles, which may differ both parametrically by cruising speed $v_i$, battery capacity ${\overline{b}}_i$, and energy level $b_i\left(0\right)\le {\overline{b}}_i$ and functionally by the set of onboard equipment installed $\left\{u_i^{\left(j\right)}\right\}\in \left\{0,1\right\}$, $j=1,\dots ,l$, $i=1,\dots ,n$. Here l is the number of available equipment types (sonars, sampling systems, cameras, etc.).

Running out of battery power during the mission forces AUVs to temporarily leave the group for replenishment. In this paper, the replenishment facility will be referred to as a charging station, although it could also be some mothership (to instantly replace a discharged robot with an already charged one) or even the ocean surface in the case of solar-powered AUVs. Also, a single-station model is considered here with the station located at a distance $d\ge 0$ from the rendezvous location. Let the station has several docking slots $w\ge 1$ for simultaneous recharging of multiple vehicles at $c\ge 1$ rate (the discharging rate is equal to 1). In other words, it is assumed that AUVs recharge c times faster than they discharge.

A j-th period when i-th AUV leaves an operating group to replenish its energy supply is referred to as the charging (or recharging) shift $Q_{ij}=(t_{ij}^s,t_{ij}^f)$ of this vehicle. Within each charging shift, AUV travels to the charging station at its cruising speed, recharges via an available docking slot, and then travels back to the rendezvous location. Following the same logic, a time interval between two consecutive recharging shifts of any vehicle i will be referred to as its working shift $W_{ij}=(t_{ij}^f,t_{ij+1}^s)$.

Thus, the group recharging schedule S is an aggregation of all recharging shifts of all vehicles:

$$S=\{Q_{11},Q_{12},\dots ,Q_{1q_1},Q_{21},Q_{22},\dots ,Q_{2q_2},\dots ,Q_{n1},Q_{n2},\dots ,Q_{n{q}_n}\},$$

(1)

where $q_i$ is a total number of recharging shifts of i-th AUV.

Each time any vehicle i leaves the group at some moment $t_{ij}^s$ or reenters the group after recharging at $t_{ik}^f$, it triggers the holding of a group rendezvous. These two event types also mark the beginning of new operational shifts where the group operates in a fixed lineup composition. The communication periodicity requirement imposes a length limit of P on the duration of each operational shift. To avoid unnecessarily frequent recharges to satisfy this constraint, additional rendezvous are allowed only for the sake of communication, with no change in fleet composition (Figure 1).

The group recharging schedule is feasible if it ensures well-timed recharging of all vehicles considering docking slots limitation:

$$\begin{array}{c}{b}_i\left(t\right)>0,t\in [0,T],i=1,\dots ,n,\\ \forall t\in T\left|\{Q\in S:t\in Q\}\right|\le w,\end{array}$$

(2)

$$\begin{array}{c}{b}_i\left(t\right)=b_i(t-1)+a,0\le b_i\left(t\right)\le {\overline{b}}_i,\\ a=\left\{\begin{array}{cc}c\hfill & t\in [t_{ij}^s+d/{\nu }_i,t_{ij}^f-d/{\nu }_i]\hfill \\ -1\hfill & \mathrm{otherwise}\hfill \end{array}\right.,j=1,\dots ,q_i.\end{array}$$

(3)

The following conditions determine the battery charge $b_i\left(t\right)$ evolution of each AUV over time, depending on all its preceding charging and working shifts (Figure 3). An abstract linear model of energy consumption (3) is applied here, while more advanced and realistic models are in our plans [30].

Before moving on to the schedule efficiency criterion, a transition of the solution representation into a discrete-time form is suggested, as it has shown better performance during the development of the problem-solving technique. In this way, group schedule (1) becomes a $(n+1)$-by-z binary matrix $H=\left\{h_{ij}\right\}$ (Figure 4). Matrix length z is a number of equal time intervals $T_i$ of $\Delta T$ length (sampling step) within the planning horizon $[0,T]$: $T=〈T_1,\dots ,T_z〉$, $z=T/\Delta T$. To reduce inaccuracy error, the preferred choice of $\Delta T$ is a divisor of P.

All rows $i=1,\dots ,n$ but one of H encodes the discrete-time schedule of i-th AUV, where elements $h_{ij}$, $i=1,\dots ,n$ denote the i-th vehicle’s shift type at time interval $T_j$. So, the sequences of $h_{ij}=0$ (blank cells in Figure 4) are assigned to AUV’s working shifts and the sequences of $h_{ij}=1$ (coloured cells in Figure 4) to AUV’s recharging shifts. The last row $n+1$ encodes group rendezvous initiated by the communication needs with no lineup changes.

The efficiency of a feasible recharging schedule H is determined through the efficiency of the working AUVs sub-group (squad) on each operational shift. Disregarding the specific problem being handled at the mid-level, the efficiency of the squad can be evaluated according to its quantitative and qualitative composition. Provided that the highest efficiency is achieved when the current squad includes all n vehicles (no AUV is on recharge), the problem is to minimize the performance losses caused by both AUVs recharging and communication needs, since holding group rendezvous also distracts all robots from completing their mission tasks.

Thus, an efficient group schedule must meet the following list of requirements:

• The main requirement is to maintain all AUVs timely recharged (2).
• The working squad should bear all available types of equipment on each shift.
• A minimum number of AUVs should be unavailable on each shift.
• A minimum number of rendezvous should be arranged on top of those required by the communication constraints.

The first requirement here is a feasibility criterion and three others (2–4) are components of the efficiency loss factor. The following weighted sum of two specific loss functions $f_E\left(H\right)$ and $f_R\left(H\right)$ are used to rate the overall efficiency loss:

$$f\left(H\right)=\Delta T·f_E\left(H\right)+f_R\left(H\right),$$

(4)

where $\Delta T$ is a sampling step.

The function $f_E\left(H\right)$ in (4) totals up the expected squad efficiency losses on each $T_i,i=1,\dots ,z$ caused by the scheduled recharging activities:

$$f_E\left(H\right)=\sum _{i=1}^z(f_i^{\left(1\right)}\left(H\right)·f_i^{\left(2\right)}\left(H\right)·f_i^{\left(3\right)}\left(H\right)).$$

(5)

To estimate fleet efficiency loss on a single time interval $T_i$ three factors are suggested to be considered. The first factor is the total number of missing (recharging) vehicles:

$$f_i^{\left(1\right)}\left(H\right)=\sum _{j=1}^n{h}_{ji},$$

(6)

where $h_{ji}$ is a shift type of j-th vehicle at time interval $T_i$.

The second factor is the speed proportion of recharging vehicles since it is unwelcome to miss the fastest vehicles in the fleet at once:

$$f_i^{\left(2\right)}\left(H\right)=\frac{{\sum }_{j=1}^n{h}_{ji}{v}_j}{{\sum }_{j=1}^n{v}_j},$$

(7)

where $v_j$ is the cruising speed of j-th AUV.

The last factor is the proportion of missing equipment of each type in the form of a standard deviation:

$$f_i^{\left(3\right)}\left(H\right)=\sqrt{\frac{{\sum }_{k=1}^l{\left(E_{ik}\left(H\right)\right)}^2}{l}},E_{ik}\left(H\right)=\frac{{\sum }_{j=1}^n{h}_{ji}{u}_j^{\left(k\right)}}{{\sum }_{j=1}^n{u}_j^{\left(k\right)}},$$

(8)

where $u_j^{\left(k\right)}\in \left\{0,1\right\}$, $k=1,\dots ,l$, $j=1,\dots ,n$ indicates the presence of k-th equipment type in robot j, and l is the total number of equipment types on-site.

On each interval $T_j$, the product under the summation sign in (5) takes the value on the interval $[0, n]$ depending on the working squad lineup. To be specific, 1-value matches the worst scenario, where all n AUVs are simultaneously recharging while 0-value stands for the best situation when all vehicles are currently working.

The second objective function $f_R\left(H\right)$ in (4) is to evaluate the total time distraction caused by holding group rendezvous:

$$f_R\left(H\right)=(f_r\left(H\right)-T/P)·Y,$$

(9)

where $f_r\left(H\right)$ is the expected number of rendezvous according to the group schedule H (10), T is a mission duration, P is a required periodicity of rendezvous, and Y is an estimated duration of an average rendezvous.

The expected number of rendezvous $f_r\left(H\right)$ is calculated as the number of columns in H, where at least one vehicle $j=1,\dots ,n$ changes its shift type $h_{ij}$ or the element of the last row is equal to 1 (communication-only rendezvous).

$$f_r\left(H\right)=\sum _{i=2}^z(\underset{j=1,\dots ,n}{max}\left(\right|h_{ji}-h_{ji-1}|,h_{n+1i}\left)\right).$$

(10)

Estimating the rendezvous duration Y is quite a challenging problem since it involves evaluating the time for vehicles to gather at the rendezvous location, conduct a communication session, and get back to executing mission tasks. This process is mission-specific and it relies on several different attributes including operational area size, tasks’ positions, AUVs speed, and, most important, the rendezvous location size. Since this size depends mainly on the AUVs’ ability to communicate at a distance, the better the communication equipment, the shorter and less distracting each group’s rendezvous will be.

The proposed problem in its essence appears to be an original variation of the well-known NP-complete combinatorial optimization problem class variously referred to as shift-, workforce-, employee-, personnel-, or staff-scheduling [31]. The problems of this class are, in general, to select a set of employees from the available pool of candidates and arrange shift schedules for each of them to fully cover the workforce demands in a least-cost way. Each employee’s schedule requirements, including days off, shift lengths, etc., must be complied with. In these problems, the demand coverage requirements are usually treated as hard constraints, and the workload distribution and sequence constraints are considered as soft ones, which can be violated [32].

The formulated high-level scheduling problem contains features of the inverse shift scheduling problem, where the workload distribution and sequence constraints, in turn, act like hard constraints, and the desirable group composition presence (coverage) acts as a soft one. Among other features of the proposed model are multi-skilled employees and flexible scheduling requirements due to the dynamic nature of the problem, which both lead to the non-cyclical type of solutions [33].

4. Evolutionary Algorithm

Like any other combinatorial optimization problem, the shift scheduling problem allows for two kinds of solutions: exact and approximate ones. The choice for the required solution type, and therefore for the method of its obtaining, is determined primarily by the specific characteristics of the problem itself and its subject area. Exact integer programming methods are used when the quality requirements outweigh the time and computational costs, while the resulting solution is expected to be utilized over a long period of time [34]. On the other hand, feasible approximate solutions become sufficient enough whenever the solution construction speed is a determining factor, whether it is high dynamics conditions or undesirable idling requirements. In this case, different heuristic and metaheuristic approaches come to the fore. Among these, evolutionary algorithms (EAs) are of the biggest interest [35].

Despite the great variety of existing scheduling approaches and their modifications, EAs remain the most frequently used method for many years due to their high performance in solving complex high-dimensional computational problems [36]. EAs combine the general versatility with extensive possibilities for hybridization with problem-specific optimization procedures. Their structure provides the ability to separately manage the exploration and the exploitation of the search space, allowing for feasible space boundary search where high-quality solutions are expected to be found [37].

Among other advantageous features is the natural parallelism of EAs, allowing both parallel calculations on several available cores and distributed problem-solution within the team of several communicating agents. Meanwhile, the pool of solution candidates acts as a kind of memory structure that facilitates faster solution adjustment in case of unexpected condition changes [38]. In a certain way, EAs can be considered as an efficient construction tool for successful population-based methods to address the essential features of the specific problem. At the same time, they remain a fast, flexible, and well scalable method.

Based on the above, a problem-oriented modification of evolutionary algorithms enriched with several specialized improvement heuristics and operators is suggested to solve the proposed problem (Figure 5). A variety of specific changes were implemented on almost each algorithm step to respond to different arising search challenges. This allowed to significantly enhance the overall performance of the approach.

A truncated n-by-z matrix $\overline{H}$ serves here as a chromosome. The last row of H (communication-only purposed rendezvous) is omitted here as it is not able to provide any positive structural effect during the solution generation. Communication-only rendezvous can be easily added to $\overline{H}$ afterwards by splitting over-long shifts of duration greater than P (shifts #3 and #5 in Figure 6).

4.1. Fitness Function

First of all, aiming to explore a wider diversity of structurally different feasible solutions, the fitness function was refined to evaluate not only feasible ones but also those which do not provide on-time recharging for all AUVs. For this purpose, an additional penalty function $V\left(\overline{H}\right)$ id added to estimate the total amount of energy shortage for each vehicle in the fleet:

$$V\left(\overline{H}\right)=\sum _{i=1}^n\sum _{j=1}^{ou{t}_i}{p}_{ij},$$

(11)

where $ou{t}_i$ is the number of times when i-th vehicle is expected to run out of energy due to the schedule $\overline{H}$.

To calculate the value of (11), the energy level of each vehicle throughout the mission should be estimated based on its schedule in H with consideration of initial energy $b_i\left(0\right)$, battery capacity ${\overline{b}}_i$, travel time $d/v_i$, station charging speed c and docks number w. The idea behind this penalty function is to pay a penalty each time an AUV is expected to run out of power and then “transfer” this vehicle to the nearest scheduled station as if it had managed to reach it with the minimum energy allowed (Figure 7). The size of each penalty p is directly related to the energy amount that AUV was short. This function is the equivalent of the “time-warp” relaxation commonly used for vehicle routing problems with time windows [39].

With the addition of an extra weighting factor n (AUVs fleet size) intended to increase the significance of the feasibility criterion $V\left(\overline{H}\right)$ (11) over the efficiency criterion $f\left(\overline{H}\right)$ (4), the resulting fitness function is as follows:

$$F\left(\overline{H}\right)=n·V\left(\overline{H}\right)+f\left(\overline{H}\right)\to min.$$

(12)

4.2. Constructive Heuristics

Most intelligent meta-heuristic algorithms use custom-designed constructive heuristics to generate an initial population of reasonable quality [40]. The general quality of any population is defined through its three properties: the general diversity, good coverage of the search space, and the presence of good feasible solutions. To achieve these requirements, it is essential to understand which exact qualities shape the particular features of each individual solution.

Three various constructive heuristics were developed for the first step of EA. Using all of those heuristics simultaneously allows the algorithm to perform well under different combinations of problem conditions and constraints.

(a)

The first heuristic is a problem-specific interpretation of the classic random one, traditionally serving to “flood” a population with diverse genetic material. The idea of this heuristic here is to generate an unbroken sequence of shifts, with exactly one random AUV leaving the group each time. The duration of each shift never exceeds the maximum time needed to fully recharge the vehicle, taking into account its travel time. The resulting schedules are infeasible but always deliver low values of $f_E\left(\overline{H}\right)$. An example of the generated schedule for four AUVs is shown in Figure 8a.

(b)

The second heuristic is a sequential heuristic that reliably generates energy-feasible solutions by constructing the entire schedule for each robot one by one. The feasibility is achieved by guaranteeing two properties: no working shift lasts longer than AUVs battery capacity; the duration of each charging shift always allows for a full recharge. The charging docks limitation is ignored here as for some combinations of conditions building even a single entirely-feasible solution is a challenging problem itself. An example of the generated schedule for four AUVs is presented in Figure 8b.

(c)

The last heuristic is aimed to form variable schedules with a small number of shifts (solutions with low-value $f_R\left(\overline{H}\right)$). To do this, it first figures out which shift length provides the minimal rendezvous number while being enough to regularly recharge all vehicles, taking into account the number of docks of the charging station w. Then, the heuristic distributes the recharging periods of each AUV over this set of shifts according to their energy demands. An example of the generated schedule for a fleet of four vehicles is pictured in Figure 8c.

Figure 8. Three schedules for the same fleet of four colour-coded AUVs generated by different constructive heuristics: (a) Diverse random heuristic to achieve low $f_E\left(\overline{H}\right)$. (b) Energy feasible heuristic to achieve low $V\left(\overline{H}\right)$. (c) Rendezvous minimizing heuristic to achieve low $f_R\left(\overline{H}\right)$.

Figure 8. Three schedules for the same fleet of four colour-coded AUVs generated by different constructive heuristics: (a) Diverse random heuristic to achieve low $f_E\left(\overline{H}\right)$. (b) Energy feasible heuristic to achieve low $V\left(\overline{H}\right)$. (c) Rendezvous minimizing heuristic to achieve low $f_R\left(\overline{H}\right)$.

4.3. Improvement Heuristics

A standard three-step scheme of selection, crossover, and mutation is used to reproduce new solutions. For the selection, a 2-way tournament is used to give a better chance for different-quality solutions (especially unfeasible) to undergo procreation.

Then two competing crossovers are applied to generate offspring solutions. The first crossover is a classical one-point crossover that is a simple and efficient genetic operator for a variety of scheduling problems [41]. The second crossover is a uniform modification that combines solutions by picking the entire route of each vehicle from a random parent.

The multi-mode mutation is applied consisting of six different operators:

• Replace a random sequence of no more than $P/\Delta T$ elements of a single vehicle in $\overline{H}$ with 0-values;
• Replace a random sequence of no more than $P/\Delta T$ elements of a single vehicle in $\overline{H}$ with 1-values;
• Select a random shift in $\overline{H}$ and then replace the elements of a single working vehicle within this shift with 1-values;
• Select a random shift in $\overline{H}$ and then replace the elements of a single recharging vehicle within this shift with 0-values;
• Insert a new column into a random place of $\overline{H}$, copy the values of the column on the left, and then remove the last column of $\overline{H}$;
• Remove a random column, then insert a new column at the end of $\overline{H}$ copying the values of the column on the left.

Local search techniques are the most common form of hybridization to enhance search performance [42]. A variable neighbourhood search heuristic [43] is used here to descend to local optima each time an evolutionary process begins to stall. To do this, four neighbouring search operators have been designed to eliminate those trouble spots which are hard to fix by crossover and mutation operators:

• Select a random recharging shift of a random vehicle and prolong it to the nearest rendezvous;
• Select a random recharging shift of a random vehicle and shorten it to the nearest rendezvous if possible;
• Select a random recharging shift of a random vehicle and move it to a random adjacent shift;
• Swap two random adjacent recharging shifts of two different vehicles.

The elitism approach and the island model (several parallel trading populations) are applied to form the next generation of newly acquired solutions. The clone elimination procedure is also used to constantly diversify the population and refill vacated spaces with newly generated solutions.

5. Results and Discussion

The proposed hierarchical approach along with the high-level mission-planning algorithm have been software-implemented in C++ in our simulation framework “AUV Multiobjective Mission Planner” (Figure 9) to run a series of simulation studies. Two types of computational experiments were conducted in the developed framework.

The first and the main type of experiment concerned exclusively the high-level scheduling. Here, the quality and speed of the developed evolutionary algorithm are evaluated on different-sized instances along with the comparison with a few available data. The second type of experiment explores the synchronous work of two different levels of the group control system during long-term missions. For this purpose, the earlier developed genetic algorithm for solving the vehicle routing problem for monitoring and patrolling tasks was implemented at the middle level [44]. In this trial scenario (second panel of Figure 9), the proposed high-level approach performs dynamic mission scheduling while the genetic algorithm constructs AUVs routes for each shift, taking into account the necessity of on-time returning to the rendezvous location. The approach showed its viability and high capability, but, since its quite a lot going on here, the authors still search for an intelligible way to systematize this type of experiment and represent its results visually and statistically in a clear, accurate, and understandable manner.

For the high-level scheduling problem, two sets of examples were developed: the first set “A” is manually constructed to know the global optimum a priori, and the other set “B” consists of randomly generated complex problems with a high level of heterogeneity. The name of each instance $\Gamma \gamma :z-n|w-c|l$ encodes the set type $\Gamma \in \{A,B\}$, the instance index $\gamma$ within the set, solution size z, fleet size n, recharging docks number w, recharging speed c, and the number of different equipment types l (heterogeneity degree).

For the construction of the set “A”, a standard cyclic scheduling approach is used. Following this approach, the shortest optimal recharging sub-cycle for all vehicles is constructed and then reproduced to fill the whole schedule. The requirements for such a sub-cycle are as follows: the final battery level of each robot at the end of the cycle must exceed the starting battery level; the ideal balance between working and charging shifts (full battery utilization); the charging shifts of AUVs must overlap as little as possible but should share common rendezvous. Several combinations of problem conditions were found by authors allowing the optimum sub-cycle to be easily constructed manually [45].

The A4:100-4|1-3|1 instance is a good illustrative example of the “A”-type cases. It is a mission of four identical vehicles with one charging dock $w=1$ and the station charging speed by one less than the fleet size $c=3$. The obvious optimal solution is cycle-based and presented in Figure 10a. Despite the seeming simplicity of this solution and the multiple global optima (the order of recharges may vary), it is quite difficult to be found by the approaches based on the exploration of the search space. The reason is a poor neighbourhood structure of such solutions, where good schedules are not located in the neighbourhood of other good ones. In other words, such cycle solutions are too rigid [33]. Thus, any minor change of the optimal solution (even the slightest reduction or extension of one single shift) leads to a striking feasibility loss. It means that the likelihood of producing such a solution by mutation or local search operators from other feasible solutions is extremely low.

Indeed, being unable to simultaneously “allocate” all recharging periods of AUVs into equal periodic shifts of maximally allowed duration, the evolutionary algorithm seeks the feasible solution with the lowest efficiency loss (5). As soon as optimal $f_E\left(H\right)$ is achieved (i.e., 3 of 4 AUVs are always available), EA then changes its focus back to minimizing the number of rendezvous (shifts). A typical structure of resulting schedules for such instances is shown in Figure 10b. As each shift here is shorter than in Figure 10a, this solution is more flexible allowing recharging order irregularities and permutations.

More complex instances of the identical structure were designed similarly for the set “A” with several AUVs simultaneous recharging ($w>1$) and even for some combinations of functionally differing vehicles ($l>1$). Anyway, it is still an open major problem for battery-differing vehicles and heterogeneous workforce in general, especially for big fleets [32,46]. The obtained results for some of the “A” test instances are presented in

Table 1. The comparison of group schedules for the test set “A”.

Instance	EA	Rotation	Optimal
A1:100-1|1-1|1	(0; 4300; 0)	(0; 4300; 0)	(0; 4300; 0)
A2:100-2|1-1|1	(0; 2500; 6)	(0; 2500; 0)	(0; 2500; 6)
A4:100-4|1-3|1	(0; 625; 29)	(0; 2875; 0)	(0; 625; 19)
A7:100-4|2-3|1	(0; 1256; 20)	(0; 1734; 0)	(0; 625; 19)
A10:100-4|2-1|2	(0; 4506; 8)	(0; 11,202; 0)	(0; 5000; 6)
A12:150-5|2-2|4	(0; 3176; 22)	(0; 12,638; 0)	(0; 1966; 19)
A15:150-6|3-3|3	(0; 10,042; 25)	(0; 14,020; 0)	(0; 3300; 44)

. The performance value here is a list $(V\left(H\right);f_E\left(H\right);f_r\left(H\right))$ of three objective function values: feasibility penalty (11), efficiency loss (5), and the total rendezvous number (10). It should be mentioned that in the following results the values of $V\left(H\right)$ are all equal to zero since both comparing approaches are capable of ensuring the energy feasibility of the final solution.

Alongside the optimal solutions, the proposed approach is compared with the decentralized group rotation scheme [47]. This scheme is designed for robotic mobile fulfilment systems and features threshold-based timing decisions for recharging considering both the current energy level of the vehicle and the distance up to the recharging station. For the sake of the team’s survivability, each vehicle during recharging is also able to cede the recharging dock to another vehicle in a greater need. The main advantage of this approach is the lack of the necessity for planning recharge schedules and, thus, for group rendezvous.

As a little summary of the comparison on the set “A”, two issues have to be addressed. The first issue concerns the comparison with optimal solutions. As discussed above, cycle-based optimal solutions in general have a rigid structure that does not allow for any feasible fluctuations. This causes the EA to tend of finding solutions with small efficiency losses at the expense of increasing the number of rendezvous (e.g., A4, A10). In the case of relaxation of the hardest recharging constraints (A7, A12, A15), the algorithm first seeks to maximize the utilization of available docking stations, as this allows to reach feasibility quickly, and only then tries to “lower the density” of a schedule by making more space. This leads to solutions with fewer shifts at the expense of additional AUV idle time while recharging.

The second issue involves comparing the developed approach with threshold-based rotation. The latter shows poor results for problems with limited docks and slow charge, as these conditions require careful planning to avoid vehicle congestion in a charging queue. At the same time, with the high “bandwidth” of the charging station, the rotation approach shows still inferior but rather competitive results.

For the set “B”, 30 highly diverse instances of various mostly large problem sizes and heterogeneity degrees are developed. The degree of heterogeneity in this context refers to the extent to which all the robots in the fleet differ from one another in their parameters, including starting and maximum battery charge, speed, and equipment set. The purpose of this set is not just to compare the proposed EA with the threshold-based approach, but also to figure out the limitations of the EA’s performance capabilities.

For example, the best-found schedule for the instance B7:200-10|3-4|4 is presented in Figure 11 along with the corresponding energy chart. This instance includes ten colour-coded robots (represented by the rectangles on the left) that carry different sets of 4 types of equipment (coloured squares inside) and vary in cruising speed and energy capacity (the upper bound is outlined horizontally on the chart). For such a diverse fleet, the search for sub-cycles becomes over-complicated, and the acyclic types of schedules come to the fore.

The approach results for the set “B” are presented in

Table 2. The comparison of group schedules for the test set “B”.

Instance	EA	Computing Time (s)	Rotation
B1:200-4|2-2|3	(0; 3614; 15)	14.1	(0; 11,098; 0)
B4:200-6|2-3|4	(0; 3497; 15)	47.5	(0; 13,570; 0)
B7:200-10|3-4|4	(0; 2816; 22)	81.8	(0; 14,506; 0)
B10:200-12|3-4|6	(0; 5873; 30)	138.0	(0; 22,881; 0)
B11:200-12|4-4|6	(0; 6192; 23)	110.4	(0; 18,273; 0)
B16:200-20|5-5|6	(0; 11,010; 26)	323.7	(0; 30,170; 0)
B20:200-30|7-6|8	(0; 24,211; 38)	437.7	(0; 59,904; 0)
B24:300-20|5-5|6	(0; 19,207; 45)	482.1	(0; 44,399; 0)
B30:300-30|8-8|10	(0; 34,636; 74)	1016.3	(0; 102,639; 0)

. An extra column here refers to the average time it takes for EA to achieve the reference solution quality. A note on computation time: all calculations are running on the single core of a 2.667 GHz processor of an Intel Core 2 Duo E6750 Conroe.

The approach showed good scalability and demonstrated the ability to maintain the general number of shifts as the fleet size increased and as the constraints became more complex. It is capable of making reasonably effective schedules for a diverse fleet of a couple of dozen AUVs in a matter of minutes of background computing. In the case of long-term planning with huge heterogeneous fleets, due to the computational overload, it is reasonable to limit the planning horizon to some mid-term window, that moves as the mission progresses. The threshold-based approach [47], in turn, has confirmed its success only for the specific class of the group maintaining problems with no strict recharging time limits. It does, however, allow for group management without any scheduling at all.

Also, these results were compared with the performance of our previous revision of the suggested approach [24]. The rejection of a compressed solution representation in favour of more explicit and nuanced genetic operators and local search operators, along with a more flexible solution evaluation, resulted in over twice the performance improvement in terms of both the efficiency of the final solutions and the speed of obtaining them.

6. Conclusions

This paper proposes a hierarchical approach to managing a heterogeneous fleet of AUVs in autonomous long-term underwater operations. The main idea of the approach is to dynamically decompose the mission into a sequence of shifts, arranged in a way to provide an efficient rotation for vehicles to recharge their batteries on time. A group meeting at the end of each shift at the specified rendezvous location ensures regular communication between AUVs to keep the collective knowledge about mission conditions up-to-date and, if necessary, to adjust the current group strategy.

This approach not only serves to reduce the complexity of the initial group routing problem by managing planning horizons but also eliminates the most difficult communication and fuel constraints from the mid-level routing problem while assuring the consistency of the operating squad within each single shift. Thus, the proposed upper-level planner is to construct a shift schedule that: ensures that each vehicle is on-time recharged; provides the required regularity of communication sessions; minimizes the average loss of team performance capabilities due to the AUVs’ recharging trips on each shift. It combines concepts of existing smart recharging schemes to efficiently treat big multi-skilled fleets of vehicles and can be classified as a new variation of the inverse shift scheduling problem.

To deal with the high-level scheduling, a problem-oriented modification of a hybrid evolutionary algorithm is designed featuring several constructive and improvement heuristics along with specialized neighbourhood search operators. In a series of computational experiments and comparisons, the developed algorithm proved to be a reliable, fast, and efficient method for generating feasible solutions, even for fairly complex high-dimensional problems. The approach also has been validated in conjunction with a mid-level routing algorithm, albeit on a single scenario so far.

Among the future developments of this line of research, it is worth mentioning the merging of the two developed population-based algorithms into a single synchronous group control system to have it tested on a diverse set of multi-vehicle scenarios.