An Attitude Prediction Method for Autonomous Recovery Operation of Unmanned Surface Vehicle

Abstract

The development of launch and recovery technology is key for the application to the unmanned surface vehicle (USV). Also, a launch and recovery system (L&RS) based on a pneumatic ejection mechanism has been developed in our previous study. To improve the launch accuracy and reduce the influence of the sea waves, we propose a stacking model of one-dimensional convolutional neural network and long short-term memory neural network predicting the attitude of the USV. The data from experiments by “Jinghai VII” USV developed by Shanghai University, China, under levels 1–4 sea conditions are used to train and test the network. The results show that the stabilized platform with the proposed prediction method can keep the launching angle of the launching mechanism constant by regulating the pitching joint and rotation joint under the random influence from the wave. Finally, the efficiency and effectiveness of the L&RS are demonstrated by the successful application in actual environments.

1. Introduction

Many countries are developing intelligent unmanned maritime equipment for marine exploitation and protection of maritime rights and interests owing to the rich biological and mineral resources. Unmanned surface vehicle (USV) is a kind of unmanned surface platform with autonomous navigation and obstacle avoidance ability, and it can independently complete tasks such as marine environment information perception, inshore island mapping, and disaster rescue, which is suitable for dangerous and human-unsuited missions instead of manned surface boats vessels [1,2,3,4,5].

Limited by endurance, the USV is usually carried by the mother ship to the mission area and then placed on the surface for autonomous operation. After the mission is completed, it would be recovered to the mother ship’s deck. Therefore, the launch and recovery technology is a key technology for the application of USV [6]. The launch and recovery system (L&RS) for the manned surface boat can be divided into two types. The first one is the stern ramp type, which is mainly composed of a slide and a winch. Kern et al. have designed a device with an inclined chute and traction mechanism that can recover both autonomous underwater vehicle and remote-operated vehicle [7]. Hayashi et al. developed a set of devices composed of an obstacle avoidance system, slope L&RS, and a matching remote control for reducing the number of operators in the recovery process [8]. However, the requirement of slideway is limited to the application range, and thus, the davit system is more widely used compared with the slide system. The davit is generally installed on both sides of the mother ship’s deck. During the operation, the USV is out of the ship’s side and is lowered to or hoisted from the sea surface [9]. The RHP L&RS developed by Global davit gmbH can be used for the launch and recovery of boats with mass from 1000 to 3500 kg [10]. Marine Equipment Pellegrini, a company from Italy, has developed a marine L&RS capable of operating at level 6 sea conditions and has strong adaptability and large load capacity. The existing L&RS, however, requires human intervention to operate the boat, which is a difficulty when this is used for USV.

One of the challenges of the launch and recovery technology is the connection of USV to the mother ship’s recovery system, dealing with the uncertainty and randomness under the impact of the waves, especially during high seas conditions. Thus, predicting the movement trend of USVs is important during launch and recovery operations. Consequently, several methods have been used, such as statistical forecast, Kalman filter, and time series [11]. Wiener et al. proposed an optimal linear prediction method based on the minimum mean square error [12]. The method can obtain better prediction results just within 5–6 s; however, the prediction error significantly increased with the prolonged prediction time [13]. Furthermore, Dodin and Sidar obtained the ship’s motion state equation based on the force analysis and then deduced the multi-step prediction [14]. On the basis of the autoregressive model, Peng et al. proposed a real-time modeling and prediction method for predicting attitude motion of large ships under random wave action. The method is found suitable for application under non-stationary motion conditions, and the prediction time takes only 7–10 s [15]. Khan et al. combined the autoregressive model and the moving average model with an artificial neural network for predicting ship motion to achieve better prediction accuracy [16]. On the other hand, methods of attitude prediction were focused on large tonnage ships. The amplitude and frequency of the attitude are smaller than that of common USVs because of the inertia. In improving the maneuverability, the mass of the USV should be generally small. However, because of the influence of the wind and waves, large and high-frequency changes in attitudes have occurred.

For previously developed L&RS based on the pneumatic ejection mechanism, this study presents the attitude prediction for USV to improve the operation success rate. The rest of the paper is organized as follows. Section 2 introduces the concept and mechanism of the L&RS. In Section 3, the USV attitude prediction algorithm for stacking one-dimensional convolutional neural network (1D CNN) and long short-term memory (LSTM) neural network is proposed for improving the aiming accuracy of the L&RS under the influence of the waves. The experiments in Section 4 verify the validity of the algorithm, and efficiency of the automatic L&RS. The conclusions and plans for future studies are presented in Section 5.

2. Launch and Recovery System

2.1. Mechanism

A L&RS for USV based on pneumatic projectile has been developed in our previous study [17]. As shown in Figure 1, it is composed of a launching mechanism, a 2-degree-of-freedom-stabilized platform mechanism and a docking mechanism. During the launching process, the USV was lowered to the sea surface by the davit, and then the locking mechanism separated the USV from the conical butt joint.

The recovery process after the completion of a mission is as follows (Figure 2).

• First, by automatic regulation of the pitching and rotation joints of the stabilized platform mechanism, the launching mechanism is aimed to the mother ship’s deck (Figure 2a).
• After the launching switch was acted, the air projectile was separated from the catapult mechanism and was driven by the high-pressure gas. It drives the guide rope to drop on the mother ship’s deck (Figure 2b).
• Then, the mother ship’s crew passes the guide rope through the hole of the conical butt joint, and the conical butt joint slides along with the guide rope into the docking mechanism (Figure 2c).
• Finally, the docking mechanism locks the conical butt joint, and the USV is lifted from the sea surface by the davit (Figure 2d).

2.2. Launching Angle

The accurate launching of the air projectile to the mother ship is important for the recovery operation of USV. When the launching angle is too large, it could result in high elevation and short-range, causing the air projectile to fall into the sea. Hence, when the launching angle is too small, it could result in low elevation and longer range, causing the air projectile to hit the mother ship’s sidewall. Therefore, during the recovery process, it is required to regulate the launching angle of the stabilized platform according to the measured distance and direction information between the USV and the mother ship.

A world coordinate system $O_0{X}_0{Y}_0{Z}_0$ is established, as shown in Figure 3a. The $O_0{Y}_0$ axis points to the bow; the $O_0{Z}_0$ axis is perpendicular to the sea level, opposite to the direction of gravity, and direction of the $O_0{X}_0$ is determined by the right-hand rule. The coordinates of the port of the canister launcher $O_p$ and the target landing point of the air projectile $O_h$ are denoted by ($x_p$,$y_p$,$z_p$) and ($x_h$,$y_h$,$z_h$), respectively. On the basis of the aerodynamics, the desired angle of the azimuth angle $\phi$ and the elevation angle $\eta$ in the world coordinate system (Figure 3b) can be calculated as follows [18]:

$$\phi =arctan\frac{y_h-y_p}{x_h-x_p}=arctan\frac{\mathrm{sgn}(x_h-x_p)·s_1}{s_0+s_j}$$

(1)

$$\eta =\arcsin \left[\frac{cW\left(\frac{c^2-1}{e}{e}^{\frac{b^2{l}_h}{m^{2}g}}\right)}{c^2-1-W\left(\frac{c^2-1}{e}{e}^{\frac{b^2{l}_h}{m^{2}g}}\right)}\right]$$

(2)

$$c=\frac{b{v}_0}{mg}$$

(3)

where $s_0$ is the distance between the USV and mother ship’s side, $s_1$ is the distance between the USV and the landing point along the direction of the $O_0{Y}_0$ axis, $h_0$ is the altitude difference between the mother ship and the USV, $s_j$ is the distance from the mother ship’s side to the landing point, $l_h$ and $v_0$ are the elevation and initial velocity of the air projectile, respectively, m is the mass of air projectile, and b is the damping coefficient.“sgn” is a symbolic function, which returns an integer variable indicating the sign of the argument:

$$sgn\left(x\right)=\left\{\begin{array}{cc}1,\hfill & x>0\hfill \\ 0,\hfill & x=0\hfill \\ -1,\hfill & x<0\hfill \end{array}\right.$$

(4)

The Lambert W function is a multivalued complex function:

$$W\left(x\right)e^{W\left(x\right)}=x$$

(5)

where x is a complex number. The detailed relationship between x and the Lambert $W\left(x\right)$ function can be found in [19].

By coordinate transformation, the corresponding angles of the pitching and rotation joints of the stabilized platform can be derived. Thus, the launching angle can be kept constant by regulating the stabilized platform in real time to compensate for the attitude change of the USV due to the influence of sea waves.

3. USV Attitude Prediction

Having the features of being small in size and lightweight, the USV has superior maneuverability over the common manned surface vehicle. However, the changes in attitude due to the influence of waves will affect the landing point accuracy of air projectile in a recovery operation. By reducing the tracking error of the catapult mechanism due to time delays, we predict the attitude of the USV based on the previous state information. In this study, the LSTM neural network model is used to predict the attitude of the USV at sea in real time. To improve the performance of the prediction model, 1D CNN is superimposed on the LSTM. It can reduce the fluctuation range of prediction results and prediction errors, resulting in a successful recovery operation.

3.1. LSTM Neural Network

LSTM is an advanced recurrent neural network (RNN) structure that can learn and predict time series data [20,21]. For an ordinary RNN network, the output at the time t is as follows:

$$Y_t=\delta (W_o{X}_t+U_o{S}_t+b_o)$$

(6)

where $X_t$ is the input at the current moment; $S_t$ is the state of the network at time t, which is derived from the output of the network at the previous moment (i.e., $S_t=Y_{t-1}$); $W_o$ is the weight matrix of the input; $U_o$ is the weight matrix for the states; $b_o$ is the bias; and $\delta$ is the activation function of the network. After $Y_t$ is inputted to the softmax layer, the final prediction result can be obtained as follows:

$$prediction=softmax(V_s{Y}_t+b_s)$$

(7)

where $V_s$ and $b_s$ are the weights of the softmax layer. Because state $S_t$ is a recursive variable, the derivative term increases with the time step when calculating the gradient, causing the gradient to disappear. To solve this problem, we improved the LSTM neural network on the basis of the original RNN network.

Figure 4 shows the internal structure of the LSTM, which adds a memory cell state $C_t$ on the original basis. During the training process, the signal is not only controlled by the input and output but also passes the forgetting control unit. The forget gate $F_t$ enables the network to delete some memory cell state information according to the previous training feedback without changing the weight, and it also selects certain neurons to update the weight. Also, an input gate $I_t$ and an output gate $K_t$ are added to make the model nonlinear. The input gate determines the amount of current input information used to calculate the carrying value. The output gate determines the amount of output from the carrying value to the final state. The state information of the three gates is calculated as follows:

$$\left\{\begin{array}{c}{I}_t={\delta }_i(W_i{X}_t+U_i{S}_t+b_i)\hfill \\ F_t={\delta }_f(W_f{X}_t+U_f{S}_t+b_f)\hfill \\ K_t={\delta }_k(W_k{X}_t+U_k{S}_t+b_k)\hfill \end{array}\right.$$

(8)

The memory cell state information carried by the network at the next moment is as follows:

$${\tilde{C}}_t=tanh(W_c{X}_t+U_c{S}_t+b_c)$$

(9)

$$C_t=F_t\ast C_{t-1}+I_t\ast {\tilde{C}}_t$$

(10)

where ${\tilde{C}}_t$ is the candidate value used to calculate the memory cell state information. The output of the network is as follows:

$$Y_t=K_t\ast tanh\left(C_t\right)$$

(11)

where $Y_t$ is the input to the softmax layer, obtaining the final prediction result. The carrying information can still be retained even after several time steps. The final output can derive long-term dependencies from the carrying information, thus solving the problem of gradient disappearance. At the same time, the input gate and the output gate can also adjust the influence of the output of different timing on the model, so it can effectively solve the problem of the gradient explosion of RNN.

3.2. One-Dimensional Convolutional Neural Network

Although LSTM has a suitable performance for processing time series, it is difficult to apply to a huge number of input data, which significantly reduced the calculation efficiency. 1D CNN is an effective method in dealing with sequence objects. It can extract high-level features from local input data through convolution operations, which can efficiently use data and reduce the input dimension. As a result, computational cost can be significantly reduced [22].

As shown in Figure 5, a local one-dimensional sequence segment is extracted from the original sequence in a 1D CNN. It is dotted with the weights in the convolution kernel to generate a shorter one-dimensional sequence. The sequence is trained as the input to the LSTM layer. As the input sequence length is shortened, the input dimension of the LSTM layer and the required training parameters are reduced. Therefore, the computing load can be effectively reduced, and the training time of the network can be shortened. Also, the 1D CNN extracts more advanced and abstract features from the original sequence in advance, so that the data use is high and the performance of the network can be effectively improved.

3.3. USV Attitude Data

Owing to the strong correlation between the attitude motion sequences of USV, the attitude in the next moment can be predicted by the LSTM neural network based on the attitude sequence value within the past time. In this paper, the training data are measured by a six-axis gyroscope mounted on the “Jinghai VII” USV developed by Shanghai University. The attitude data include posture angle and angular velocity in the heading, pitch, and roll directions of the USV. The measurement frequency of the sensing system is 10 Hz, which is far faster than that of the large tonnage ships. The experimental data with a time step of 0.1 s can be converted into dimensionless data through standardized formulas:

$$x*\left(t\right)=\frac{x\left(t\right)-x_{min}\left(t\right)}{x_{max}\left(t\right)-x_{min}\left(t\right)}$$

(12)

where t is time; $x\left(t\right)$ is input data at time t; $x_{max}\left(t\right)$ and $x_{min}\left(t\right)$ are maximum and minimum values of the input data at time t, respectively; and $x*\left(t\right)$ is the normalized value of the input data in which the range is from 0 to 1.

3.4. Determination of Hyper-Parameters

In this study, we applied a neural network model with two hidden layers and ten hidden units. The input layer inputs the attitude angle and angular velocity information of the USV in one direction. The output layer uses the tanh function as the activation function to output the predicted result. The uniform distribution randomly generates the network weights. The range is (−limit, limit):

$$limit=\sqrt{\frac{6}{n_j+n_{j+1}}}$$

(13)

where $n_j$ is the number of units in the layer j network. The bias is initialized at 0. The square sum of the difference between the predicted value and the actual value of the output is selected as the loss function. By minimizing the loss function, all weights and bias parameters in the network can be obtained:

$$loss=\frac{{\displaystyle \sum _{j=1}^n}{\left(y_i-{y_i}^p\right)}^2}{n}$$

(14)

where $y_i$ is the actual value, ${y_i}^p$ is the predicted value, and n is the total number of data.

The loss function can also be used to evaluate the accuracy of the neural network training model. The trained heading, pitch, and roll neural network models of the USV can be respectively derived from the posture angle and angular velocity in three directions.

3.5. Training Process

As shown in Figure 6, first, the weights and biases of the stacked LSTM network are initialized. After entering the normalized attitude data, the forecasted value of heading, roll, and pitch angles can be obtained. Equation (14) is used to calculate the loss between the desired value and the real value. Finally, the gradient descent method is used to constantly adjust the weight and bias until the loss value converges or the number of iterations reaches the peak.

It is noted that a large size of database takes a long training time. Thus, a reasonable batch size needs to be set to reduce the number of iterations required for the training model. To obtain the suitable batch size, it was set to 1 at the beginning, and then the value increased until the improvement in training accuracy was no longer apparent in this study.

4. Experiments and Discussion

4.1. Experimental Overview

To verify the validity of the proposed prediction algorithm, we used the experimental data measured from the “Jinghai VII” USV under levels 1–4 sea conditions for training the testing.

Table 1. Specifications of “Jinghai VII” USV.

Parameters	Values	Unit
Length	8.2	m
Width	2.45	m
Height	1.84	m
Mass	3000	kg
Depth of immersion	0.5	m

lists the specifications of the USV. According to [23,24], the sea level conditions were generally defined by the ranges of significant wave heights as listed in

Table 2. Definition of sea level conditions [23,24].

Sea Condition Level	Sea States	Significant Wave Height (m)
0	Calm (glassy)	0
1	Calm (ripples)	0–0.1
2	Smooth (wavelets)	0.1–0.5
3	Slight	0.5–1.25
4	Moderate	1.25–2.5
5	Rough	2.5–4.0
6	Very rough	4.0–6.0
7	High	6.0–9.0
8	Very high	9.0–14.0
9	Phenomenal (Extreme)	Over 14.0

. The data set contains the posture angles and angular velocities of the USV in the heading, pitch, and roll directions. Each set has approximately 3000 sets of data, in which 60%, 20%, and 20% were taken as training, verification, and test sets, respectively. The batch size is 128 in both training process and validation process.

To prevent model overfitting, we took samples 100 times per turn for a total of 50 rounds. The neural network model is trained by inputting the angle and angular velocity information simultaneously. To demonstrate the performance of the proposed method, we compared the results with that predicted by the original LSTM, Nonlinear Autoregressive Exogenous Model (NARX) network, and Time Delay Neural Network (TDNN) under 1–4 sea levels.

4.2. Results and Discussion

To evaluate the performance of the 1D CNN-LSTM neural network, we compared the effect of that LSTM neural network model, in which the parameter settings and training test data are the same for the two models. The results of the heading angle of USV are taken as an example. As shown in Figure 7a, although the sea condition is at level 1, the heading angle fluctuates greatly because of the slight weight of the USV. Both neural network models can predict the trend of the USV attitude, but the predicted results by LSTM neural network have a rougher degree of agreement with the actual curve. It is deduced that as the data set is small, the original LSTM neural network is difficult to learn the changing characteristics of the heading angle, reducing the accuracy of the prediction model. It can be seen from the enlarged area of Figure 7a that when the heading angle suddenly changed, the prediction error of the LSTM neural network increased. However, the predicted results by 1D CNN-LSTM neural network can achieve higher accuracy. It is attributed that after the addition of CNN, the model can effectively extract the features of the USV attitude data and reduce the redundant information input to the LSTM neural network. CNN enables LSTM neural network to mine the deeper features of heading angle variation so that the predicted results are closer to the actual test data.

Table 3. Comparison of the results by LSTM, 1D CNN-LSTM, NARX and TDNN.

Sea Condition	Network Model	Training Speed (ms/step)	Training Loss	Validation Loss	Test Loss
Level 1	LSTM	22	0.1013	0.1232	0.0511
	1D CNN-LSTM	12	0.0289	0.0128	0.0212
	NARX	16	0.0301	0.0120	0.0222
	TDNN	13	0.0247	0.0416	0.0283
Level 2	LSTM	20	0.0402	0.0264	0.0240
	1D CNN-LSTM	14	0.0284	0.0201	0.0132
	NARX	17	0.0308	0.0244	0.0232
	TDNN	14	0.0222	0.0188	0.0216
Level 3	LSTM	20	0.0309	0.0107	0.0310
	1D CNN-LSTM	15	0.0279	0.0211	0.0218
	NARX	18	0.0305	0.0186	0.0261
	TDNN	14	0.0212	0.0221	0.0308
Level 4	LSTM	21	0.0765	0.0461	0.0381
	1D CNN-LSTM	15	0.0415	0.0125	0.0133
	NARX	19	0.0377	0.0239	0.0141
	TDNN	16	0.0286	0.0232	0.0179

shows the comparison of the training speed and various losses of the model. It can be confirmed that the training speed of the LSTM neural network is 22 ms/step, whereas the training speed of the 1D CNN-LSTM network model is increased by 55% to 10 ms/step. The LSTM test loss was 0.0511, whereas the test loss of the 1D CNN-LSTM network model was reduced by 59% to 0.0212. The 1D CNN-LSTM neural network has higher training efficiency and higher prediction accuracy.

The proposed prediction algorithm is required to guarantee suitable prediction accuracy for various sea conditions and environments. To demonstrate the performance of the proposed network model under unknown complex sea conditions and higher noise disturbances, we showed the results of the heading angle under level 2–4 sea conditions in Figure 7b–d, respectively. As shown in Table 3, the test error of the LSTM neural network increased with the increase of sea level. It means that the complexity of high sea conditions has a great influence on the accuracy of the LSTM neural network prediction result. However, the predicted results by the 1D CNN-LSTM network model after training agreed with the test data well. The proposed network model test error is approximately 0.015–0.025.

As shown in Figure 8, all the neural network models can predict the trend of the USV attitude, in which the 1D CNN-LSTM can achieve the higher prediction accuracy than the other 3 networks. Table 3 demonstrated the comparison of the training speed and various losses of the four networks at different sea level. Compared with NARX, the training speed of 1D CNN-LSTM increased by 25% (level 1), 18% (level 2), 17% (level 3) and 21% (level 4), and test loss decreased by 4% (level 1), 43% (level 2), 16% (level 3) and 6% (level 4). Although the training speed of 1D CNN-LSTM has slight increase with TDNN, the test loss was 25% (level 1), 39% (level 2), 29% (level 3) and 26% (level 4) lower than that of TDNN. In addition, it can be found that the training speed of 1D CNN-LSTM in levels 1–4 sea conditions is increased by 45%, 30%, 25% and 29% respectively, compared with the test loss of LSTM. The test loss of 1D CNN-LSTM in levels 1–4 sea conditions is reduced by 59%, 45%, 30%, and 65%, respectively, compared with the test loss of LSTM. It is noticed that the degree of the improvement in the prediction accuracy has no relationship with the sea level conditions.

It is noticed that compared with the common online prediction algorithms, 1D CNN-LSTM requires no calculation of the model parameters in real time. It ensures the high speed, real-time and reliability of the usv’s attitude data prediction. The fluctuation range of model prediction error under sea conditions is relatively stable regardless of the time, and the prediction error keep stable in the prolonged prediction time. Therefore, the 1D CNN-LSTM network is superior to the other three networks in terms of prediction accuracy, adaptability, and efficiency. It can achieve suitable prediction accuracy under various sea conditions, which is important for the recovery operation of the USV.

4.3. Field Application

The autonomous launch and recovery operation of “Jinghai VII” USV was performed in the East China Sea to demonstrate the effect of the proposed attitude prediction method in the practical application. After the mission was completed, the USV returned to the vicinity of the mother ship in preparation for recovery. The mother ship was stopped, and the position was continuously changing due to the influence of sea waves. By the guidance of the navigation system, the USV finally stopped approximately 3 m away from the port side of the mother ship with the same heading.

After the propeller was stalled, the USV switched to the aiming phase. Owing to the influence of the wave, the attitude of the USV continuously changes. From Equations (1) and (2), the desired launching angle of the air projectile can be calculated from a distance between the stabilized platform and the target landing point in the world coordinate system measured by the laser rangefinder and GPS positioning system mounted on the USV.

Figure 9 demonstrates the effect of the proposed prediction algorithm on the stability control of the stabilized platform. It can be seen that the tracking error of the azimuth and elevation angles can be improved by using the proposed prediction algorithm. Without the attitude prediction, the stabilized platform regulated the azimuth and elevation angles based on the tracking error by the PID algorithm. As shown in Figure 9a,c, although the desired values and actual values have similar trends, it has the obvious time delay. It is attributed that the attitude of the USV changes instantly, and the launching angle of the air projectile changes synchronously with the change of the USV attitude. As the stabilized platform moves to the desired angle according to the calculated value at the previous moment, the USV attitude has changed. However, using the proposed prediction algorithm, the joint angles can follow the desired values with a slight tracking error. On the basis of the predicted results, the stabilized platform can respond in advance. It compensated the influence of the wave and increased the accuracy of the air projectile launch.

By applying the proposed prediction method, we can obtain an accurate projectile on the mother ship’s deck. As shown in Figure 10, following the operation process in Section 2.1, the mother ship’s crew operated the hoisting boom to drop the conical butt joint to slide into the conical dock entrance along with the guide rope. Then, the locking mechanism locked the conical butt joint, and the USV was lifted and placed on the mother ship’s deck.

During the operation, reducing the sway of the USV was time-consuming. Moreover, in reducing the sway of the USV during the lifting process, the operation required eight operators, a crane operator, a commander, and other people that could pull the USV in the head and stern directions through ropes. All people were working on the mother ship’s deck, and no operator is required onboard the USV. It significantly improved the safety of the USV recovery operation.

5. Conclusions and Future Work

To improve the launching accuracy of the developed pneumatic ejection mechanism-based L&RS of USV, we proposed a stacking model of 1D CNN and LSTM network in this study to predict the attitude of the USV. On the basis of the predicted results, the pitching joint and rotation joint of the stabilized platform can be regulated in real time to compensate for the disturbance of the waves, keeping the launching angle stable. The data from the experiments by “Jinghai VII” USV were used to train and test the proposed network. The results demonstrated that the algorithm has suitable prediction accuracy and calculation efficiency. From the filed application, it can be confirmed that the L&RS with the trained network can successfully recover USV. It requires no operator working on the USV, and it significantly improved safety and adaptability. In the future, the proposed method will be applied to model predictive control techniques, automatic speech recognition, etc.