Formation Control of Multiple Autonomous Underwater Vehicles under Communication Delay, Packet Discreteness and Dropout

: Effective communication between multiple autonomous underwater vehicles (AUVs) is necessary for formation control. As the most reliable underwater communication method, acoustic communication still has many constraints compared with radio communication, which affects the effectiveness of formation control. Therefore, this paper proposes a formation control scheme for multiple AUVs under communication delay, packet discreteness and dropout. Firstly, the communication delay is estimated based on the kernel density estimation method. To solve the problem of packet discreteness and dropout, the curve ﬁtting method is used to predict the states of the AUV. Secondly, a follower controller is designed based on the leader–follower approach using input–output feedback linearization, which is proven to be stable with Lyapunov stability theory. Then, some simulation results are presented to demonstrate the stability and accuracy of the formation control in different communication environments. Finally, the ﬁeld tests on the lake show that the scheme introduced in this paper is valid and practical.


Introduction
Autonomous underwater vehicles (AUVs) are widely used in military and commercial fields due to their small size, low cost, high degree of autonomy and flexible deployment. They are irreplaceable tools in applications such as accident rescue, seabed topographic mapping, object detection, observation of ocean phenomena and marine resource development [1][2][3][4][5][6][7][8]. With the increasing complexity of underwater tasks, it is necessary to improve work efficiency through the cooperation of multiple AUVs, which can even complete tasks that cannot be performed by a single AUV [9]. At the same time, formation control has become a fundamental issue in the cooperative control of multiple AUVs. In recent years, many scholars have carried out research on the formation control of multiple agents and the common methods are as follows: the virtual structure method [10,11]; the behaviorbased strategy [12]; the leader-follower approach [13][14][15][16][17][18][19][20]; the potential field function approach [21]; the path-following method [22]; and the consensus theory method [23,24]. Due to the complicated underwater environment, most methods cannot be directly applied to the formation control of multiple AUVs. However, the leader-follower approach is the most widely used in engineering because of its simple structure and easy expansion [25]. In this method, the leader moves according to a given trajectory, and the followers track the leader to maintain the desired formation structure. Therefore, formation control can be Packet discreteness and dropout in acoustic communication are mainly caused by path loss, noise and multipath propagation. It has rarely been considered in previous studies. In [41], Chen proposes a distributed event-triggered communication mechanism to overcome the problems of packet discreteness and dropout. This method is mainly ap-plied to linear models and is not easily implemented in engineering. Most researchers assume that AUVs can communicate underwater continuously with acoustic communication [42]. However, it is obviously unreasonable in practical applications. On the one hand, the high-powered acoustic modem cannot work continuously for a long time. On the other hand, packet dropout is inevitable and can also be considered a special case of packet discreteness. Discrete packets will reduce the frequency of information exchange between multiple AUVs, which decreases the accuracy and stability of formation control. As a result, packet discreteness and dropout are also important issues that must be considered for formation control.
Based on the above discussion, this paper is concerned with the formation control of multiple AUVs under communication delay, packet discreteness and dropout. The main contributions are as follows: (1) The communication delay is estimated based on the kernel density estimation method.
Kernel density estimation is a non-parametric estimation method that does not need prior knowledge and an accurate mathematical model of communication delay. Instead, we can obtain an accurate distribution of delay with this method according to the characteristics and properties of delay values based on underwater experiments. Compared with other methods, this method has more extensive applications. (2) The packet discreteness and dropout problems are solved by information prediction based on the curve fitting method. This method can be used to predict the key states of the leader AUV to generate a continuous and precise trajectory for the follower AUVs. (3) We derive a kinematic model for the error of the formation control system. The follower controller is designed using the input-output feedback linearization method and the stability of this method is proved by Lyapunov stability theory. (4) Both the simulation in MATLAB and the field tests on the lake are carried out to verify the feasibility of the scheme presented in this paper.
This paper is organized as follows. Section 2 includes the kinematic model of the AUV and the mathematical description of formation control. Section 3 proposes methods to solve the communication delay, packet discreteness and dropout problems. Section 4 presents the formation control scheme of multiple AUVs. Sections 5 and 6 show the simulation and test results. Finally, Section 7 gives the conclusions and future perspectives.

AUV Model
In actual engineering applications, AUVs run in a three-dimensional underwater environment. However, most underwater missions, such as ocean floor surveys, only require multiple AUVs to move at a constant depth. Therefore, without loss of generality, only the movement in the horizontal plane is considered. Roll, pitch and heave are ignored in this paper. To focus on the communication constraints, the ocean currents and obstacles in the underwater environment are ignored. Thus, the kinematics equation with 3-DOF of the AUV is presented as follows: where N represents the number of AUVs; η i = [x i , y i , Ψ i ] T , i = 1, 2, . . . , N represents the pose of AUV i in the global coordinate frame, consisting of the position (x i , y i ) and yaw T is the standard velocity vector of AUV i in the body coordinate frame; u i and v i are respectively the velocity in surge and sway; r i is the yaw velocity. The transformation matrix J(Ψ i ) can be described as

Description of Formation Control
In the leader-follower formation control method, the leader moves according to a given trajectory and the followers maintain a desired distance L d ∈ R (N−1)×1 and angle Φ d ∈ R (N−1)×1 from the leader. The relationship between the leader AUV 1 and the follower AUV i is shown in Figure 1 T are the actual distance and the desired distance between the leader and the follower in the body coordinate frame of the leader, respectively.
In the leader-follower formation control method, the leader moves according to a given trajectory and the followers maintain a desired distance from the leader. The relationship between the leader AUV1 and the follower AUVi is shown in Figure 1, where 1 are the leader's position, the follower's position, and the follower's desired position, respectively; d denotes the distance between the center of mass and the front of the AUV; ( ) represents the desired distance and angle relative to the leader.

T [ , ]
x y l l = l and T [ , ] d d x y l l = d l are the actual distance and the desired distance between the leader and the follower in the body coordinate frame of the leader, respectively.
T T [ , ] cos( ),sin( ) According to Equation (1), the actual distance can be calculated as follows: is the yaw of the leader AUV1. The tracking errors of the follower AUVi are defined as: where x e and y e are the errors in the x and y directions, respectively. Based on the above analysis, the formation control problem can be transformed into a problem of the follower tracking a series of desired target points. Formation control can be realized by designing a follower controller to eliminate tracking errors. According to Equation (1), the actual distance can be calculated as follows: where is the yaw of the leader AUV 1 . The tracking errors of the follower AUV i are defined as: where e x and e y are the errors in the x and y directions, respectively. Based on the above analysis, the formation control problem can be transformed into a problem of the follower tracking a series of desired target points. Formation control can be realized by designing a follower controller to eliminate tracking errors.

Solutions of Communication Delay, Packet Discreteness and Dropout
To achieve formation control, each follower AUV needs constant and accurate information from the leader AUV. However, when the AUV moves underwater, the conditions of acoustic communication become complex, which causes communication delay, packet discreteness and dropout. The information flow of the communication from the leader to the follower is shown in Figure 2. The horizontal axis above represents the time when a series of packets are sent by the leader AUV, where the number in the square boxes represents the serial number of packets sent by the leader AUV (i.e., 1, 2, 3, 4 in the square boxes), T i is the time when the packet i is sent by the leader, T represents a fixed interval between the sent packets. The horizontal axis below represents the time when a series of packets are received by the follower AUV, where the numbers in the squares represent the serial number of packets received by the follower AUV (i.e., 1, 3, 4 in the squares), t i denotes the time when the packet i is received by the followers, {τ i } is a set of communication delay (for example, packet 1 is sent and received at T 1 and t 1 , respectively, then τ 1 can be calculated as τ 1 = t 1 − T 1 ) and t i is a set of the discrete time between the received packets. It can be seen from Figure 2 that the sent packets may be received after a period of time or not. For example, packet 1 sent at time T 1 is received by the follower at time t 1 , but packet 2 sent at time T 2 is not received. In this case, packet 1 is received by the follower with a communication delay, and packet 2 sent by the leader is lost.
resents the serial number of packets sent by the leader AUV (i.e., 1, 2, 3, 4 in the square boxes), i T is the time when the packet i is sent by the leader, T represents a fixed interval between the sent packets. The horizontal axis below represents the time when a series of packets are received by the follower AUV, where the numbers in the squares represent the serial number of packets received by the follower AUV (i.e., 1, 3, 4 in the squares), i t denotes the time when the packet i is received by the followers, { } i τ is a set of communication delay (for example, packet 1 is sent and received at 1 T and 1 t , respectively, then 1 τ can be calculated as 1 1 1 t is a set of the discrete time between the received packets. It can be seen from Figure 2 that the sent packets may be received after a period of time or not. For example, packet 1 sent at time 1 T is received by the follower at time 1 t , but packet 2 sent at time 2 T is not received. In this case, packet 1 is received by the follower with a communication delay, and packet 2 sent by the leader is lost. In short, the follower cannot obtain constant and accurate states of the leader because of communication delay, packet discreteness and dropout. In Section 3.1, the kernel density method is used to build the communication delay distribution model and generate the delay subject to the distribution model. Furthermore, in Section 3.2, we use the curve fitting method to predict the states of the leader to solve the problems of packet discreteness and dropout.

Estimation of Communication Delay
Communication delay is composed of the propagation delay and parsing delay of the packets. In practice, the propagation delay is related to the speed and distance of propagation. The propagation speed of the sound is affected by environmental factors such as temperature, depth, salinity, etc. Therefore, we adopt a propagation model approved by the United Nations Educational, Scientific and Cultural Organization (UNESCO) to calculate the speed of sound underwater [43]. Then, the propagation delay can be approximately expressed as: In short, the follower cannot obtain constant and accurate states of the leader because of communication delay, packet discreteness and dropout. In Section 3.1, the kernel density method is used to build the communication delay distribution model and generate the delay subject to the distribution model. Furthermore, in Section 3.2, we use the curve fitting method to predict the states of the leader to solve the problems of packet discreteness and dropout.

Estimation of Communication Delay
Communication delay is composed of the propagation delay and parsing delay of the packets. In practice, the propagation delay is related to the speed and distance of propagation. The propagation speed of the sound is affected by environmental factors such as temperature, depth, salinity, etc. Therefore, we adopt a propagation model approved by the United Nations Educational, Scientific and Cultural Organization (UNESCO) to calculate the speed of sound underwater [43]. Then, the propagation delay can be approximately expressed as: where O 1 O i is the Euclidean distance between the leader and the follower and v sound is the speed of the sound. Based on the experimental data, the parsing delay is obtained using the kernel density method. Let t 0 1 , t 0 2 , . . . , t 0 n be independent and identically distributed samples of the parsing delay. Let f (x) be the probability density function of the samples. Then, the estimated probability density function [44] can be written as: where h > 0 is the bandwidth, a neighborhood of the variable x; n is the total number of samples; K(x) is a kernel function and the following three equations are satisfied.
According to Equation (7), given the total number of samples n,f (x) is associated with the bandwidth h and kernel function K(x). If h is too small, the number of samples used to estimate f (x) is also small, resulting in a decrease in the bias and an increase in the variance of the estimation. Conversely, if h is too large, we can obtain an inaccurate estimation with a higher bias and lower variance. Therefore, how to obtain the best h is extremely critical.
Define the integral mean square error function as: To minimize MISE(h), the optimal bandwidth is calculated as: where f (x) can be approximately replaced byf (x). The Gaussian function has convenient mathematical properties and satisfies the requirements of the kernel function, so it is chosen as the kernel function, that is, Then, the best bandwidth can be calculated according to Equation (10).
By combining Equations (7) and (11), we can obtain the estimated probability density function: After that, we introduce the method of generating the parsing delay that conforms to the distribution. Let the random variable Y obey a uniform distribution between 0 and 1, i.e., Y ∼ U(0, 1). Define the random variable of the parsing delay as: where Ψ(·) is a monotonically increasing mapping function from Y to X. Then, where F(·) and G(·) are the distribution functions of X and Y, respectively. Hence, we can obtain G(y) = y and The mapping function Ψ(·) can be determined according to Equation (15). By combining Equations (13) and (15), the parsing delay can be calculated as: where τ 2 represents the parsing delay and Random(0, 1) denotes the uniformly distributed random number generated by the computer in the interval (0, 1). Finally, we can obtain the propagation delay and parsing delay of acoustic communication from Equations (6) and (16), respectively: where τ is the estimation of the communication delay.

Prediction of Leader States
When multiple AUVs move in formation underwater, the followers can only receive discrete states of the leader due to packet discreteness and dropout. To improve the accuracy and stability of formation control, the followers need continuous states of the leader. Therefore, the curve fitting method is applied to predict the states of the leader [45]. The reliable and discrete packet of leader states received by the followers is defined as where K represents the number of received packets. Let k be the minimum number of packets required f the curve fitting method. If K is not less than k, a third-degree polynomial (simulation results show that a higher-degree polynomial does not mean a more precise prediction) is used to predict the x-coordinate of the leader.
To obtain polynomial parameters, linear equations are expressed in matrix form: The pseudoinverse method is used to solve Equation (19). We can then obtain: where of the leader is calculated as: where P(t) = t 3 t 2 t 1 .
The y-coordinate of the leader can also be predicted in the same way, i.e., y 1 (m) = A y m 3 + B y m 2 + C y m + D y . Referring to Equations (19)-(21), the y-coordinate of the leader can be obtained as: where N = A y B y C y D y T . Equations (21) and (22) are the predictions of the leader states.

Formation Control Scheme
In this paper, it is assumed that the leader can move along the predefined trajectory. Then, the formation control scheme can be understood as the path tracking algorithm for the followers, composed of the generation of continuous trajectories of the leader and the design of the controller for the followers. The former provides an accurate reference trajectory for the followers under communication delay, packet discreteness and dropout. The latter designs a motion controller for the followers to achieve tracking control of the desired path. Figure 3 shows the system control block diagram and the dashed box is a low-level closed-loop controller of velocity, which was designed in previous work and is not the focus of this paper.

Formation Control Scheme
In this paper, it is assumed that the leader can move along the predefined trajectory. Then, the formation control scheme can be understood as the path tracking algorithm for the followers, composed of the generation of continuous trajectories of the leader and the design of the controller for the followers. The former provides an accurate reference trajectory for the followers under communication delay, packet discreteness and dropout. The latter designs a motion controller for the followers to achieve tracking control of the desired path. Figure 3 shows the system control block diagram and the dashed box is a low-level closed-loop controller of velocity, which was designed in previous work and is not the focus of this paper.

Generate Continuous Trajectories of the Leader
In this section, combined with Sections 3.1 and 3.2, the algorithm pseudocode of a continuous trajectory of the leader is shown in Algorithm 1. Define the information of the leader received by the followers as sents the sequence number of the packets.
where t denotes the time 4: then we obtain the communication delay using Equation (17) and the states of the leader are corrected 5:

Generate Continuous Trajectories of the Leader
In this section, combined with Sections 3.1 and 3.2, the algorithm pseudocode of a continuous trajectory of the leader is shown in Algorithm 1. Define the information of the leader received by the followers as ( represents the sequence number of the packets. then we obtain the communication delay using Equation (17) and the states of the leader are corrected 5: 6: else t = t i 7: then continuous trajectories of the leader are predicted 8: then we obtain the communication delay using Equation (17) and the states of the leader are corrected 12: 13: else t = t i 14: then M and N are updated using Equation (20) and the continuous trajectories of the leader are predicted according to Equations (21) and (22) 15: end

Design the Follower Controller
In the formation control of multiple AUVs, the desired distance L d and angle Φ d between the leader and the followers can be constant to maintain a fixed formation or timevariant to form formations of different structures. In this section, the follower controller is designed to track the leader with an expected distance and angle to achieve formation control. Firstly, the system error kinematic model is derived based on the formation model in Section 2.2. Then, a stable follower controller is designed with the input-output feedback linearization method. Finally, the stability of the controller is proved using Lyapunov stability theory.
Taking the derivative of Equations (3)-(5), we can obtain: where r 1 = · Ψ 1 and r i = · Ψ i are the angular velocities of the leader and follower, respectively and u 1 = · x 1 cos(Ψ 1 ) − · y 1 sin(Ψ 1 ) represents the linear velocity of a surge by the leader. Then, Equation (31) can be simplified as: where Ψ e = Ψ 1 − Ψ i denotes the error of yaw. In addition, we consider an underactuated AUV and ignore the linear velocity in sway. Therefore, we can obtain: where u i represents the linear velocity of a surge by the follower. Then, Equation (32) can be written as: Combining Equations (29), (30) and (34) yields: Similarly, we can obtain: Therefore, the system error kinematic model can be given by: where U i = [u i , r i ] T is the control input of the system, A = 0 −r 1 r 1 0 , For Equation (37), we use the input-output feedback linearization method and obtain: where K = k 1 0 0 k 2 and k 1 , k 2 > 0. The formation control law of the system is obtained using Equation (38). Lyapunov stability theory is used to prove that the system is stable. Define the Lyapunov function as: Obviously, the function V is positive definite. The derivative of V is: Considering Equation (38), Equation (40) can be written as: where · V is negative definite. Therefore, the control system is asymptotically stable according to Lyapunov stability theory.

Simulation Results
To verify the effectiveness and robustness of the proposed scheme, four different methods of formation control were simulated in MATLAB under the same experimental scenario and different communication environments.

• Experimental methods
Method-1. Ignoring the problems of communication delay, packet discreteness and drop, we only adopt the controller designed in Section 4.2. In this method, it is assumed that the states of the leader at time t i remain constant for the followers until the next time the packet is successfully received by the followers.
Method-2. Considering the actual communication conditions, the problems of the communication delay, packet discreteness and dropout are solved in Section 3, the continuous trajectories of the leader are generated in Section 4.1 and the follower controller is designed in Section 4.2. In this method, k = 5 is used to predict the state of the leader.
Method-3. We use the formation control method combining the consensus theory and leader-follower method under communication delay in [36]. The communication delay is assumed to have a fixed upper limit.
Method-4. The formation control method under event-triggered mechanism is adopted as in [41], which can overcome the problems of packet discreteness and dropout.

•
Experimental scenario The multiple AUV system consists of one leader AUV and three followers AUV i , I = 1,2,3. The leader moves along a predefined trajectory and the followers keep a relative distance and angle from the leader to form a square formation. The motion parameters and formation structures of the four AUVs are shown in Table 1.

• Communication environments
Underwater acoustic communication environments are simulated in the simulation and the evaluation indexes of the communication quality include communication delay, the interval of sent packets and packet dropout rate. Table 2 shows the different communication environments in the simulation.

Simulation Results and Discussion
Simulation results in Environment-1 are shown in Figures 4-6 and Figure 7a. Figure 4a,b provide the motion trajectories of AUVs with Method-1 and Method-2, respectively, which are the direct performance of the formation control scheme. It can be seen from Figure 4a that with Method-1, the follower can follow the leader to some extent but cannot accurately maintain the desired square formation. Method-2 is used in Figure 4b, in which multiple AUVs move in a square formation and the effectiveness and robustness are significantly better than those of Method-1.

Simulation Results and Discussion
Simulation results in Environment-1 are shown in Figures 4-6 and 7a. Figure 4a,b provide the motion trajectories of AUVs with Method-1 and Method-2, respectively, which are the direct performance of the formation control scheme. It can be seen from Figure 4a that with Method-1, the follower can follow the leader to some extent but cannot accurately maintain the desired square formation. Method-2 is used in Figure 4b, in which multiple AUVs move in a square formation and the effectiveness and robustness are significantly better than those of Method-1.            Figure 5 shows the effect of solving the problems of communication delay, packet discreteness and dropout. Figure 5a illustrates the results of the communication delay estimation, where Delay r , Delay p and Delay e denote the real communication delay between the leader AUV and the follower AUV 1 , the estimation of the delay and the error of the estimated delay, respectively. As we can see from the figure, the error of the estimated delay is small and the effectiveness of the kernel density method is significant. The prediction error of the states of the leader is shown in Figure 5b. The figure shows that the follower can accurately predict the states of the leader. Figure 6 shows the formation errors of Method-1 and Method-2. In Figure 6a, with respect to the desired distance of the formation L d , there is a significant error in Method-1, while the error gradually converges in Method-2. The same result is shown in Figure 6b regarding the desired angle of the formation Φ d . For further analysis, Figure 7a shows the tracking errors between the follower AUV 1 and the desired target point in Method-1 and Method-2, i.e., e x and e y . In Method-1, due to the communication delay, packet discreteness and dropout, the follower cannot obtain a continuous and accurate motion trajectory of the leader, so the tracking errors are not converged. In Method-2, the communication delay is estimated, and the states of the leader are predicted so that the follower obtains a continuous and accurate motion trajectory of the leader. Although Method-2 adopts the same follower controller as Method-1, the tracking errors converge asymptotically.  Figure 7a, the packet dropout rate is higher in Figure 7d, resulting in a larger maximum tracking error, average tracking error and a greater fluctuation of the tracking errors. However, the tracking errors always converge to zero in Method-2, indicating that this method can be applied for formation control under different communication environments. Tables 3 and 4 show the main experimental results using Method-1 and Method-2. The errors of distance and angle between the leader and the followers are included in the tables. Furthermore, Figure 4 also shows the errors of the estimated communication delay and the prediction of the leader trajectory in Method-2, which are small enough. Comparing the two tables, we can see that the formation errors are smaller in Method-2, which proves the effectiveness and robustness of the formation control scheme for multiple AUVs under communication delay, packet discreteness and dropout. Finally, we compare the results of the method in this paper with previous methods as shown in Table 5. It shows the results of average formation errors in all four methods under different communication environments. We can see from the table that the errors in Method-2 are much lower than those in other methods. Compared with Method-2, the communication delay in Method-3 has an upper bound. Therefore, the accuracy of this method cannot be guaranteed with time-varying unbounded communication delay. In addition, packet discreteness and dropout also affect the control effect of Method-3 to some extent. As for Method-4, it solves the problems of packet discreteness and dropout with the event-triggered mechanism. However, the communication delay has a great influence on this method. The effect of formation control in this method is inferior to that in Method-3.
In conclusion, compared with previous methods, the proposed method in this paper can significantly improve the accuracy of formation control under communication delay, packet discreteness and dropout.

Field Tests
Three self-developed "TS-100" AUVs were used on a lake to verify the proposed scheme. In addition, the formation control effect of Method-1 and Method-2 proposed in this paper was compared during the field tests.

Vehicle Characteristics
The "TS-100" AUV, as shown in Figure 8, has a torpedo-like design, underactuated propulsion and a maximum speed of 5 kn. It is equipped with an acoustic communication system, Doppler velocity log (DVL), inertial navigation system (INS) and conductivity temperature and depth (CTD) sensor. The acoustic communication system is composed of an acoustic modem and transducer, which can send and receive packets underwater. During the test, the leader AUV moved according to a preset trajectory and communicated periodically with the followers in the form of a broadcast. The communication interval was set as a constant 5 s. The followers only received the packets from the leader and attempted to form an expected formation using Method-1 and Method-2. The formation control scheme was written in C++ and ran on the AUV's control unit in real-time.

Vehicle Characteristics
The "TS-100" AUV, as shown in Figure 8, has a torpedo-like design, underactuated propulsion and a maximum speed of 5 kn. It is equipped with an acoustic communication system, Doppler velocity log (DVL), inertial navigation system (INS) and conductivity temperature and depth (CTD) sensor. The acoustic communication system is composed of an acoustic modem and transducer, which can send and receive packets underwater. During the test, the leader AUV moved according to a preset trajectory and communicated periodically with the followers in the form of a broadcast. The communication interval was set as a constant 5 s. The followers only received the packets from the leader and attempted to form an expected formation using Method-1 and Method-2. The formation control scheme was written in C++ and ran on the AUV's control unit in real-time.

Preparation and Scenario
Before the test, the communication delay was measured to obtain the data on the parsing delay. The specific method is to place two AUVs adjacent to each other underwater, as shown in Figure 9. One AUV periodically sends packets and records the time when the packets are sent in an offline log file, and the other AUV receives packets and records the time when the packets are received in another offline log file. Then, we can calculate the communication delay according to the two log files. Since the two AUVs are very close to each other, the propagation delay can be ignored and the communication delay ob-

Preparation and Scenario
Before the test, the communication delay was measured to obtain the data on the parsing delay. The specific method is to place two AUVs adjacent to each other underwater, as shown in Figure 9. One AUV periodically sends packets and records the time when the packets are sent in an offline log file, and the other AUV receives packets and records the time when the packets are received in another offline log file. Then, we can calculate the communication delay according to the two log files. Since the two AUVs are very close to each other, the propagation delay can be ignored and the communication delay obtained can be considered the parsing delay. According to Section 3.1, the estimated probability density function of the parsing delay is calculated as shown in Figure 10.

Preparation and Scenario
Before the test, the communication delay was measured to obtain the data o parsing delay. The specific method is to place two AUVs adjacent to each other und ter, as shown in Figure 9. One AUV periodically sends packets and records the time the packets are sent in an offline log file, and the other AUV receives packets and re the time when the packets are received in another offline log file. Then, we can cal the communication delay according to the two log files. Since the two AUVs are very to each other, the propagation delay can be ignored and the communication dela tained can be considered the parsing delay. According to Section 3.1, the estimated ability density function of the parsing delay is calculated as shown in Figure 10.  During the test, three AUVs were deployed, including one leader and two follow Multiple AUVs moved approximately 1.2 km in a southwest direction at a fixed d and the formation structures were equilateral triangles and straight lines in the firs and second half of the course, respectively. The test scenario is shown in Figure 11 AUV's voyage data, including the time when the packets were sent and received, recorded in an offline log file. The log files of all the AUVs were used later to calculat communication delay and the errors in formation control. During the test, three AUVs were deployed, including one leader and two followers. Multiple AUVs moved approximately 1.2 km in a southwest direction at a fixed depth and the formation structures were equilateral triangles and straight lines in the first half and second half of the course, respectively. The test scenario is shown in Figure 11. The AUV's voyage data, including the time when the packets were sent and received, was recorded in an offline log file. The log files of all the AUVs were used later to calculate the communication delay and the errors in formation control.
The test was divided into two parts. In the first part, the communication environment was regarded as ideal, and we adopted Method-1 for formation control. In the second part, we considered the actual communication constraints and adopted Method-2 for formation control. Several tests were conducted using the two methods and the acoustic communication environment was basically the same in each test. The results of multiple tests using the same method do not differ significantly. Therefore, we arbitrarily took two test results for comparison and analysis, which are shown in Figures 12-16 and Tables 6 and 7.
Multiple AUVs moved approximately 1.2 km in a southwest direction at a fixed depth and the formation structures were equilateral triangles and straight lines in the first half and second half of the course, respectively. The test scenario is shown in Figure 11. The AUV's voyage data, including the time when the packets were sent and received, was recorded in an offline log file. The log files of all the AUVs were used later to calculate the communication delay and the errors in formation control. The test was divided into two parts. In the first part, the communication environment was regarded as ideal, and we adopted Method-1 for formation control. In the second part, we considered the actual communication constraints and adopted Method-2 for formation control. Several tests were conducted using the two methods and the acoustic communication environment was basically the same in each test. The results of multiple tests using the same method do not differ significantly. Therefore, we arbitrarily took two test results for comparison and analysis, which are shown in Figures 12-16 and Tables 6 and 7.           Figure 12 shows the motion trajectories of the multiple AUVs. We can see that the multiple AUVs maintain an equilateral triangle formation in the first half of the course and change into a straight-line formation in the second half. The overall and local enlarged figures describe that the followers can follow the leader in both methods. However, the accuracy of formation control using Method-2 is significantly better than that using Method-1. Figure 13 shows the results of the communication delay estimation and the prediction of the leader states using Method-2. Figure 13a illustrates the results of the communication delay estimation, where Delay r , Delay p and Delay e denote the real communication delay between the leader AUV and the follower AUV 1 , the estimation of the delay and the error of the estimated delay, respectively. From the figure, we can find that the method of delay estimation works well and the error of the estimation is small. It can also be seen that there is packet dropout in the acoustic communication, such as five consecutive packet dropouts between t = 200 s and t = 230 s (that is, the follower receives no packet during that time). Figure 13b shows the error of the prediction of leader states. Compared with the simulation, the interval of sent packets and packet dropout rate in the field tests are larger, so the error of prediction is also slightly larger. When the follower cannot receive the packets from the leader for a long time, a cumulative error of prediction may occur. Nevertheless, the error varies within an acceptable range and will be eliminated with the update of the leader states. Figure 14 shows the tracking errors of follower AUV 1 and follower AUV 2 . Even though the initial error is larger in Method-2, the steady-state error is smaller. However, the errors suddenly increase when the formation structures change (t = 579 s and t = 560 s). This is due to the rapid change in the desired target points of the follower. The rise in time is required to form a new formation. Therefore, after approximately 50 s, multiple AUVs reform the formation and the errors tend to converge again. Since the scheme in this paper does not consider the influence of external environmental disturbances such as water currents on formation control, the tracking errors fluctuate slightly in actual applications. Figure 15 shows the actual distance between the leader and the followers. Combined with the parameter of formation structures in Figure 11, we can see that the actual distance is closer to the desired in Method-2. Figure 16 shows the actual angles between the leader and the followers. It can be seen that the followers cannot maintain the desired angles well with the leader in both methods. In Method-1, it is mainly caused by the problem of communication constraints. In Method-2, it may be affected by underwater environmental disturbances.

Test Results and Discussion
The detailed results of the field tests are shown in Tables 6 and 7. From the tables, we can see that the packet dropout rate of acoustic communication in each test is almost the same; that is, the communication environment can be considered the same. The estimation of communication delay and prediction of leader states in Method-2 are so precise that the average errors of distance and angle between the leader and the followers (5.411 m and 0.208 rad) are smaller than those in Method-1 (12.15 m and 0.41 rad). A comparison of the two tables shows the effectiveness and robustness of the kernel density estimation and curve fitting methods, as well as the designed follower controller.
It is worth noting that the acoustic communication system and navigation equipment can affect the effectiveness of formation control. The method in this paper works well on the "TS-100" AUVs. However, it is not only applicable to a specific communication system. We believe that our method can improve the accuracy and robustness of formation control even for different communication systems. As for navigation equipment, we use an advanced integrated navigation method to improve the accuracy of the individual location. So, the navigation errors in formation control are small enough to be ignored, even though they can actually decrease the accuracy of formation control a little.

Conclusions
In this paper, the formation control of multiple AUVs under communication delay, packet discreteness and dropout has been investigated. The kernel density estimation and curve fitting methods have been used to solve the problem of communication constraints. Then a continuous and precise trajectory of the leader was generated for the followers. On this basis, a follower controller designed based on input-output feedback linearization can achieve accurate tracking of the leader. In this way, the formation control of multiple AUVs is realized. Simulation and field test results show that acoustic communication constraints have a great impact on formation control. However, the scheme presented can improve the accuracy and stability of formation control under an unsatisfactory communication environment. Therefore, the scheme proposed can be widely used in actual applications. For example, with this scheme, multiple AUVs can perform efficient and full coverage detection of underwater ecological environments or dangerous underwater targets. In addition, multiple AUVs equipped with different sensors can expel and even capture dangerous underwater targets through precise formation control. Furthermore, multiple AUVs can be used as mobile nodes in underwater communication networks. The movement of AUVs in a precise formation construction may improve the efficiency and coverage of communication networks. In conclusion, the formation control scheme presented in this paper can be applied to various underwater tasks, which will make some contribution to the field of ocean engineering.
Although the method in this paper has some practicability, it also ignores some potential problems. For example, model uncertainty and external disturbances such as ocean currents and obstacles are not considered, which to some extent affects the accuracy and stability of formation control and the safety of the AUVs when they move in a complex underwater environment. These problems have been studied by many researchers in recent years [46][47][48][49][50]. Moreover, 3-D coordinated formation control, which can be used for more underwater missions, is not studied in this paper. In the future, the problem of formation control subject to them will be solved based on the research in this paper.