Adaptive Navigation Algorithm with Deep Learning for Autonomous Underwater Vehicle

Precise navigation is essential for autonomous underwater vehicles (AUVs). The measurement deviation of the navigation sensors, especially the microelectromechanical systems (MEMS) sensors, is a crucial factor that affects the localization accuracy. Deep learning is a novel method to solve this problem. However, the calculation cycle and robustness of the deep learning method may be insufficient in practical application. This paper proposes an adaptive navigation algorithm with deep learning to address these questions and realize accurate navigation. Firstly, this algorithm uses deep learning to generate low-frequency position information to correct the error accumulation of the navigation system. Secondly, the χ2 rule is selected to judge if the Doppler velocity log (DVL) measurement fails, which could avoid interference from DVL outliers. Thirdly, the adaptive filter, based on the variational Bayesian (VB) method, is employed to estimate the navigation information simultaneous with the measurement covariance, improving navigation accuracy even more. The experimental results, based on AUV field data, show that the proposed algorithm could realize robust navigation performance and significantly improve position accuracy.


Introduction
The autonomous underwater vehicle (AUV) is an essential equipment for ocean exploration and has been widely applied in recent years. By collecting information from various relevant sensors, the navigation system could estimate the position, velocity, and other navigation information of the AUV [1,2]. Therefore, AUV navigation technology is an essential prerequisite for ocean exploration. Because the global positioning system (GPS) is invalid when the AUV operates underwater, it is necessary to develop underwater autonomous navigation technology. The main navigation methods at present include inertial navigation [3,4], acoustic navigation [5,6], simultaneous localization and mapping (SLAM) [7,8], geophysical map-based navigation [9], and integrated navigation [10]. Among the above methods, integrated navigation is the most widely used navigation method for AUVs.
Considering the cost and convenience, the integrated navigation system for the smallscale AUV, which works in shallow seas and lakes, mainly relays on the Doppler velocity log (DVL) and the attitude and heading reference system (AHRS). Based on the kinematics equation, the dead-reckoning method can be used to calculate the location of AUVs [11,12]. However, the measurement error, which could be considered Gaussian white noise, would cause the position error accumulation. The navigation system needs to use the filter to reduce the interference of measurement error. The regularly used filters include extended the Kalman filter (EKF) [13,14], and the unscented Kalman filter (UKF) [15,16]. EKF is the most commonly used nonlinear filtering method for AUV navigation. By intercepting the first-order Taylor series, it could approximate the nonlinear transformation process, Figure 1 is the structure of the Sailfish AUV. The Sailfish AUV includes the propulsion system, communication system, environmental perception system, and navigation system. The propulsion system includes the propeller, rudder, and their related control units, which provide the power of AUV motion. The communication system aims to provide stable information interaction between the AUV and the offshore control unit. The environmental perception system uses sensors to obtain environmental information that can be used for mission performance and obstacle avoidance. Among them, the navigation system could provide navigation information for other systems, so it is the fundamental function of AUVs. To realize the navigation state estimation, the AUV is equipped with various sensors and equipment. The navigation system is integrated in the middle cable of the whole AUV. The related sensors and equipment include a GPS, DVL, AHRS, and a depth meter (DM). Each sensor and equipment will be introduced in the following parts. The specification of part sensors and equipment could refer to our previous work [37].  Figure 1 is the structure of the Sailfish AUV. The Sailfish AUV includes the propulsion system, communication system, environmental perception system, and navigation system. The propulsion system includes the propeller, rudder, and their related control units, which provide the power of AUV motion. The communication system aims to provide stable information interaction between the AUV and the offshore control unit. The environmental perception system uses sensors to obtain environmental information that can be used for mission performance and obstacle avoidance. Among them, the navigation system could provide navigation information for other systems, so it is the fundamental function of AUVs. To realize the navigation state estimation, the AUV is equipped with various sensors and equipment. The navigation system is integrated in the middle cable of the whole AUV. The related sensors and equipment include a GPS, DVL, AHRS, and a depth meter (DM). Each sensor and equipment will be introduced in the following parts. The specification of part sensors and equipment could refer to our previous work [37]. As a piece of passive positioning equipment, the GPS could obtain an accurate position when the AUV floats on the surface. The GPS calculates its localization by receiving the signal of the satellite, and the position information has no error accumulation. However, because of the interference from the wind and waves, the GPS navigation information may have outliers. A filter is needed before the GPS information can used in the navigation system. Table 1 presents the specifications of the GPS. The DVL is aimed at measuring the linear velocity of the AUV. When the DVL works, a short sound pulse with a fixed frequency will be transmitted and reflected by the bottom of the seafloor. The transducer of the DVL calculates the linear velocity based on the frequency change of the echo. The sound echo would be interference with environment and the AUV motion state, which caused the DVL measurement to fail in practical application. When the DVL operates normally, the measurement error is considered to be the white noise. Table 2 presents the specifications of the DVL.  As a piece of passive positioning equipment, the GPS could obtain an accurate position when the AUV floats on the surface. The GPS calculates its localization by receiving the signal of the satellite, and the position information has no error accumulation. However, because of the interference from the wind and waves, the GPS navigation information may have outliers. A filter is needed before the GPS information can used in the navigation system. Table 1 presents the specifications of the GPS. The DVL is aimed at measuring the linear velocity of the AUV. When the DVL works, a short sound pulse with a fixed frequency will be transmitted and reflected by the bottom of the seafloor. The transducer of the DVL calculates the linear velocity based on the frequency change of the echo. The sound echo would be interference with environment and the AUV motion state, which caused the DVL measurement to fail in practical application. When the DVL operates normally, the measurement error is considered to be the white noise. Table 2 presents the specifications of the DVL. The AHRS, which includes a three-axial gyroscope, accelerometer, and magnetic compass, could directly measure the linear acceleration, angular velocity, and magnetic intensity. The above data would be a fusion with a Kalman filter to get a stable heading and attitude. However, due to the magnetic sensitivity of the external ferromagnetic material and the electromagnetic wave of AUV, the information of AHRS, especially the yaw data, may have a deviation from the true value, which could not be treated as white noise. Table 3 presents the specifications of AHRS. The DM is an essential sensor that can provide AUV depth during vehicle operation in the water. The core component of the DM is a pressure sensor that can obtain the intensity of pressure. The micro process unit of the DM uses the pressure, fluid density, and constant of gravitation to calculate the depth of the AUV. The data from DMs are relatively accurate and can be employed directly in AUV navigation. Therefore, the main challenge in our AUV navigation is to achieve a position estimation in the horizontal coordinate. Table 4 presents the specifications of the DM.

Conventional EKF Navigation Algorithm
The conventional navigation algorithm of the AUV platform is EKF. EKF can be divided into two parts: time update and measurement update. In the time update stage, the system employs the kinematics equation to predict the system state, and the Jacobin matrix of the system update function is used to forecast the covariance of system noise. Equations (1) and (2) represent the system propagate process: where X is the system state, f is the system update function based on the kinematics equation, P is the system error covariance, Q is the covariance of the system noise, and F is the system noise transition matrix.
In the measurement update stage, the system state and covariance would be corrected by the measurement value. The Kalman gain, which is determined by the covariance of system noise and measurement noise, represents the degree of correction. The measurement update process is shown in the following equations: where K is the Kalman gain, Z is the measurement vector, h is the measurement function, H is the Jacobin matrix of measurement function, and R is the measurement noise covariance matrix.
In the AUV navigation modeling process, the kinematics equation of AUV navigation is represented as Equation (6): where x and y are the positions along the north and east axis, Ψ is the heading of AUV, u and v are the longitudinal and transverse velocities in horizontal coordinates, a u and a v are the related acceleration, ω is the angular velocity of heading, and t is the time interval.
Since the error propagation is a complex coupled process, to simplify the calculations, the system noise is assumed as the white Gaussian noise. The system noise is shown as Equation (7). n = [ n x n y n ψ n u n v n au n av n w ] T (7) As the system update function is nonlinear, the system noise transition function, which is shown in Equation (8), is the Jacobin matrix of the system update function.
The measurement state includes the velocity, acceleration, heading, and angular velocity and is shown in Equation (9): As the measurement state has a linear relationship to the system state, the measurement transition matrix is easy to express: According to the above navigation system modeling, the navigation information of the AUV can be calculated in real-time by using the EKF algorithm. However, the data of the AHRS is vulnerable to the external ferromagnetic material and the electromagnetic waves of the AUV. In the actual AUV application, the noise of the heading is colored noise and difficult to estimate, which is the main reason that affects the positioning accuracy of the navigation system. The velocity measurement error of the DVL is another factor that affects navigation accuracy.

Adaptive Navigation Algorithm with Deep Learning
This paper proposed an adaptive navigation algorithm with deep learning to address sensor noise in the navigation system. Figure 2 is the flowchart of the developed method. The algorithm is based on the EKF method, and regular observation contains the data of the DVL and AHRS. When the deep learning calculation cycle is finished, the position calculated by deep learning would be added to the observation Z D . To avoid interference from DVL outliers, the observation Z A would remove the data of DVL when it fails. The VB method is employed in the EKF to adjust the covariance of noise. waves of the AUV. In the actual AUV application, the noise of the heading is colored noise and difficult to estimate, which is the main reason that affects the positioning accuracy of the navigation system. The velocity measurement error of the DVL is another factor that affects navigation accuracy.

Adaptive Navigation Algorithm with Deep Learning
This paper proposed an adaptive navigation algorithm with deep learning to address sensor noise in the navigation system. Figure 2 is the flowchart of the developed method. The algorithm is based on the EKF method, and regular observation contains the data of the DVL and AHRS. When the deep learning calculation cycle is finished, the position calculated by deep learning would be added to the observation Z D . To avoid interference from DVL outliers, the observation Z A would remove the data of DVL when it fails. The VB method is employed in the EKF to adjust the covariance of noise.

Deep Learning Navigation Method
The yaw data of AHRS includes colored noise, which is hard to estimate by the conventional method. Therefore, we employed a hybrid recurrent neural networks (RNNs) framework to realize AUV position estimation. Since the training process uses the GPS movement as the label, and the raw data of the sensors as the input, the trained framework could include the interference from yaw error [32]. Figure 3 presents the structure of hybrid RNNs. This framework uses unidirectional and bidirectional long short-term memory (LSTM) to handle different sensor data. The calculation process of LSTM is as follows [38]. Figure 4 is the structure of LSTM.

Deep Learning Navigation Method
The yaw data of AHRS includes colored noise, which is hard to estimate by the conventional method. Therefore, we employed a hybrid recurrent neural networks (RNNs) framework to realize AUV position estimation. Since the training process uses the GPS movement as the label, and the raw data of the sensors as the input, the trained framework could include the interference from yaw error [32]. Figure 3 presents the structure of hybrid RNNs. This framework uses unidirectional and bidirectional long short-term memory (LSTM) to handle different sensor data. The calculation process of LSTM is as follows [38]. Figure 4 is the structure of LSTM.
where Xs is the sequence input of data, the W and b are the weight and bias, the fg is the forgetting gate, ig is the input gate, C is the cell state, Og is the output gate, and hl is the hidden layer. The activation function σ uses the sigmoid. The • represents the point-wise product operation.   [ ] ( )     The RNN structure is used to preprocess the raw data of the DVL and AHRS. Then the output of different RNNs and the time interval are used as the input to transform the displacement by a fully connected layer [39]. The GPS trajectory is smoothed by an adaptive fault-tolerance filter and separated into segments to generate labels for training. In this network, root mean square error (RMSE) is selected to calculate the loss between the label and prediction. The RMSE calculation process is shown in Equation (16).
where m is the number of train datasets, χ is the label, and Xd is the predicted value of deep learning.
To improve the training efficiency of the network, the activation function in the fully connected layers employs the rectified linear unit (ReLu) to overcome the vanishing gradient problem. According to our experiments, the learning rate should be reduced during the training process, so the adaptive gradient (AdaGrad) is selected to adjust the learning rate, and AdaGard could effectively decrease training cycles. The trained network could achieve AUV position estimation.
Deep learning is an end-to-end navigation method and does not need to handle various complex matrix operations. Therefore, it is easy to implement. Since the neural network method could approximate the optimal solution, it could obtain relatively precise localization when the raw data has a large deviation. However, the performance is relatively weak when the raw data is accurate. The calculation cycle of the deep learning method needs data from a duration time, which causes the deep learning method to only obtain navigation information with a low frequency. In this work, the position information of deep learning is used as the element in the observation to correct the navigation information.
Here the measurement vector Zd includes the deep learning output and the data from the DVL and AHRS. The RMSE of deep learning describes the deviation degree from truth value, and the square of RMSE is mean square error (MSE), which is the same as the meaning of measurement variance. Hence, we use the MSE as an approximate measurement variance of the neural network position estimation. The measurement noise covariance matrix Rd concludes the MSE of deep learning and the measurement noise. Because the variance obtained by the MSE is still not accurate, it needs to be corrected by the VB method.

DVL Fault Diagnosis
When AUVs cruise in the water, the motion state is interfered with by the waves and surge, which may have an adverse impact on the DVL measurement. The moving objects near the DVL, or a rapid change in the terrain, is another factor for DVL accuracy.
To restrain the above impact, the χ 2 rule is selected to judge if the DVL measurement fails [35,36]. The innovation of DVL measurement is satisfied with white noise when the data of DVL is available.
In our navigation system, the covariance matrix N DVL is expressed as follows: The fault detection criterion is defined as Equation (19).
If C DVL is above the threshold, we consider the DVL measurement to have failed, and the DVL measurement is removed from the observation matrix Za. The measurement noise covariance matrix Ra only contains the measurement noise of AHRS. The χ2 rule is suitable for mutant fault detection, which could effectively avoid the impact of DVL measurement failure in navigation.

Variational Bayesian Method
The accuracy of the covariance estimation is significant to the performance of the Kalman filter. Since the system covariance is the inherent characteristics, and approximate estimation could maintain the stability of the filter, the measurement covariance is the main factor that affects the accuracy of the state estimation. In our navigation algorithm, the measurement includes position, yaw angle, and velocity. Among them, the covariance of position is obtained by deep learning, which is an approximate value, and the covariance of the DVL would be affected by the environment. The VB method is introduced to the state estimation to simultaneously estimate the measurement covariance to address this question.
The VB method aims to obtain the conditional probability density of joint distribution p(X k , R k |Z k ). To simplify the calculation process, the joint distribution is approximate to the product of two independent probability densities, as shown in Equation (20).
According to the VB theory, the approximate probability density can be obtained by minimizing the Kullback-Leibler divergence from the actual probability density. Since the measurement follows the law of normal distribution, the covariance of covariance is assumed to satisfy the inverse Wishart distribution [37]. The VB method can be concluded as follows.
In the time update process, the VB method is similar to the standard Kalman filter. In addition, the parameter initialization is as Equation (21).
where α and β are the elements in the probabilistic distribution of measurement covariance, m is the degree of the observation matrix, and ρ is the factor that approximates the posterior of the measurement covariance. The measurement update is an iterative process. The calculation is expressed in Equation (22). (22) where i represents the number of iterations. After the iteration is finished, the last system state and measurement covariance are the estimations of the VB method.
In the proposed navigation algorithm, the covariance of measurement is uncertain, which could seriously impact navigation accuracy. The VB method could simultaneously estimate the measurement noise covariance and system state, which could reduce the interference from imprecise covariance. For simplicity, the notation of the AUV model and the developed algorithm is regrouped for clarity in Table 5.

Experiments and Analysis
In this section, a series of experiments based on the AUV field data is carried out. To evaluate the performance of different navigation methods, a truth value, such as GPS position, is necessary for the experiment platform. As the AUV could not obtain a GPS position while immersed underwater, the field data were collected when the AUV was cruising on the surface. Moreover, the GPS-smoothed trajectories could be generated as the labels to train the deep learning network. Figure 5 is the Sailfish AUV during the Tuandao Bay experiments. The scene is the Sailfish cruising on the surface. For simplicity, the notation of the AUV model and the developed algorithm is regrouped for clarity in Table 5.

Experiments and Analysis
In this section, a series of experiments based on the AUV field data is carried out. To evaluate the performance of different navigation methods, a truth value, such as GPS position, is necessary for the experiment platform. As the AUV could not obtain a GPS position while immersed underwater, the field data were collected when the AUV was cruising on the surface. Moreover, the GPS-smoothed trajectories could be generated as the labels to train the deep learning network. Figure 5 is the Sailfish AUV during the Tuandao Bay experiments. The scene is the Sailfish cruising on the surface. The field data of our experiments were acquired from different places, such as the Menlou Reservoir, Jiaozhou Bay, and Tuandao Bay. Table 6 depicts the details of the experimental data. The field data of four groups covers straight lines, turns, and cycles, which represent almost all of the motion modes of AUVs. Additionally, the experimental The field data of our experiments were acquired from different places, such as the Menlou Reservoir, Jiaozhou Bay, and Tuandao Bay. Table 6 depicts the details of the experimental data. The field data of four groups covers straight lines, turns, and cycles, which represent almost all of the motion modes of AUVs. Additionally, the experimental environment in different places is varied. The winds and waves in bay and coastal waters are more intense than those in the reservoir. In our experiments, the conventional EKF, pure deep learning, and the proposed algorithm were employed to generate the AUV trajectory. Figure 6 shows the paths produced by various algorithms. Since the GPS data frequency in our platform is 1 Hz, the GPS trajectory is divided into movements per second. Therefore, the navigation data frequency of deep learning is 1 Hz, while the conventional method and the proposed method are 10 Hz.
Sensors 2021, 21, x FOR PEER REVIEW 11 of 15 environment in different places is varied. The winds and waves in bay and coastal waters are more intense than those in the reservoir. In our experiments, the conventional EKF, pure deep learning, and the proposed algorithm were employed to generate the AUV trajectory. Figure 6 shows the paths produced by various algorithms. Since the GPS data frequency in our platform is 1Hz, the GPS trajectory is divided into movements per second. Therefore, the navigation data frequency of deep learning is 1 Hz, while the conventional method and the proposed method are 10 Hz. In Figure 6, black lines represent the smoothed GPS trajectories that are considered as the ground truth, red lines represent the conventional EKF trajectories, blue lines represent pure deep learning trajectories, and violet lines represent the proposed method. According to our previous research, the performance of the deep learning method would be better than the conventional EKF algorithm in most cases. However, the EKF trajectory would be closer to the ground truth when the accuracy of navigation sensors is high, which caused the pure deep learning in Test1 to be worse than the EKF. The proposed method could adaptively fuse the data from sensors and the deep learning method by the VB method. Therefore, the navigation accuracy in Test1 can be improved more than other methods. In Test2, DVL raw data has a jump that causes the EKF to deviate from the expected trajectory. The deep learning method is robust to the outliers and could effectively avoid interference. In the proposed method, the DVL fault diagnosis method detects the measurement fails and removes the velocity data from the observation, so the proposed method could maintain the robustness towards the measurement outliers. Test3 and Test4 show that the proposed method could effectively improve navigation accuracy more than the conventional EKF method. Figure 7 shows the position errors between the ground truth and different algorithms. The position errors are the distance between the GPS and the estimation results of different methods. In Figure 7a, because of the slight sensor deviation and the data fusion strategy, the performance of the proposed method is better than other methods. Figure 7b shows the error of Test2. Since the DVL measurement has outliers, the deep learning method could significantly improve the navigation accuracy. The proposed method, with a fault diagnosis function leading to the algorithm, has positive fault tolerance ability to the DVL measurement fails. Figure 7c,d show that the proposed method could improve the position accuracy compared to the conventional EKF method, and the accuracy is close to the deep learning method. Table 7 summarizes the RMSE of all the above algorithms during the four experiments. The RMSE results evidence that, although the proposed algorithm is insufficient compared to deep learning in most cases, it could achieve norm frequency and robust navigation, and improve accuracy to a larger extent than the conventional method. A number of experimental tests verify that the RMSE could improve by at least 14.4%.  In Figure 6, black lines represent the smoothed GPS trajectories that are considered as the ground truth, red lines represent the conventional EKF trajectories, blue lines represent pure deep learning trajectories, and violet lines represent the proposed method. According to our previous research, the performance of the deep learning method would be better than the conventional EKF algorithm in most cases. However, the EKF trajectory would be closer to the ground truth when the accuracy of navigation sensors is high, which caused the pure deep learning in Test1 to be worse than the EKF. The proposed method could adaptively fuse the data from sensors and the deep learning method by the VB method. Therefore, the navigation accuracy in Test1 can be improved more than other methods. In Test2, DVL raw data has a jump that causes the EKF to deviate from the expected trajectory. The deep learning method is robust to the outliers and could effectively avoid interference. In the proposed method, the DVL fault diagnosis method detects the measurement fails and removes the velocity data from the observation, so the proposed method could maintain the robustness towards the measurement outliers. Test3 and Test4 show that the proposed method could effectively improve navigation accuracy more than the conventional EKF method. Figure 7 shows the position errors between the ground truth and different algorithms. The position errors are the distance between the GPS and the estimation results of different methods. In Figure 7a, because of the slight sensor deviation and the data fusion strategy, the performance of the proposed method is better than other methods. Figure 7b shows the error of Test2. Since the DVL measurement has outliers, the deep learning method could significantly improve the navigation accuracy. The proposed method, with a fault diagnosis function leading to the algorithm, has positive fault tolerance ability to the DVL measurement fails. Figures 7c,d show that the proposed method could improve the position accuracy compared to the conventional EKF method, and the accuracy is close to the deep learning method. Table 7 summarizes the RMSE of all the above algorithms during the four experiments. The RMSE results evidence that, although the proposed algorithm is insufficient compared to deep learning in most cases, it could achieve norm frequency and robust navigation, and improve accuracy to a larger extent than the conventional method. A number of experimental tests verify that the RMSE could improve by at least 14.4%.

Conclusions
In this work, we developed an adaptive navigation algorithm based on deep learning. Firstly, this algorithm uses deep learning to generate low-frequency position information to correct the navigation error. Secondly, the χ2 rule is selected to judge if the DVL measurement fails, which could avoid the interference from DVL outliers. Thirdly, the adaptive filter based on the VB method is employed to estimate navigation information simultaneous to the measurement covariance, improving navigation accuracy even more.
Different from the pure deep learning navigation method, this work could achieve robustness and high accuracy navigation with a normal frequency, which could be satisfied by the requirements of various missions. The experimental results based on AUV field data verified that even the performance of the proposed algorithm is slightly worse than pure deep learning. However, it has good robustness and could effectively improve navigation accuracy compared to the conventional navigation algorithms. In the future, we will carry on more complex integrated navigation system design, such as the integration of different acoustic equipment, and investigate the performance of the proposed algorithm.

Conclusions
In this work, we developed an adaptive navigation algorithm based on deep learning. Firstly, this algorithm uses deep learning to generate low-frequency position information to correct the navigation error. Secondly, the χ2 rule is selected to judge if the DVL measurement fails, which could avoid the interference from DVL outliers. Thirdly, the adaptive filter based on the VB method is employed to estimate navigation information simultaneous to the measurement covariance, improving navigation accuracy even more.
Different from the pure deep learning navigation method, this work could achieve robustness and high accuracy navigation with a normal frequency, which could be satisfied by the requirements of various missions. The experimental results based on AUV field data verified that even the performance of the proposed algorithm is slightly worse than pure deep learning. However, it has good robustness and could effectively improve navigation accuracy compared to the conventional navigation algorithms. In the future, we will carry on more complex integrated navigation system design, such as the integration of different acoustic equipment, and investigate the performance of the proposed algorithm.