1. Introduction
Unmanned surface vehicles (USVs) [
1] are small unmanned marine vehicles that travel on water in a remote or autonomous manner. USVs are characterized by small size, fast movement, and high flexibility. In recent years, they have been adopted to conduct missions such as marine rescue, environmental monitoring [
2], island reef mapping [
3], and resource exploration [
4]. The safety of USVs is critical when conducting missions, and the roll motion is directly related to the safety and operating performance [
5]. In order to ensure that USVs conduct their missions safely, it is necessary to predict the USV roll motion, so that the operator or automatic control system has sufficient time to avoid serious accidents.
The USV roll motion at sea is a time-varying, nonlinear, and uncertain complex dynamic system. It is affected by control systems such as steering and propulsion systems, as well as external disturbances produced by wind, waves, and sea current [
6,
7]. Moreover, the USV roll motion and the movements in other degrees of freedom (e.g., heave and pitch motions) are coupled with each other. Therefore, it is difficult to establish precise mathematical models to represent the USV roll motion at sea.
Over the past decades, many methods have been developed to model and predict ship motion. Francescutto et al. [
8] applied an available mathematical model with concentrated parameters to the roll-sloshing problem. Daalen et al. [
9] formulated a differential equation model (a Mathieu-type equation) for modeling the roll dynamics of a ship sailing in large-amplitude head waves. Silva et al. [
10] described a time-domain non-linear strip theory model of ship motions in six degrees of freedom based on a potential flow strip theory using Frank’s close fit method. These methods require mechanical analysis and mathematical modeling of the hull, so there are some difficulties in practical application. Later, scholars proposed some simple and practical methods. According to the theoretical differences, these methods can be classified into three types: Kalman filtering, time series methods, and neural-network-based methods [
11]. Kalman filtering is a recursive linear minimum variance filter for online forecasting [
12]. Triantafyllou et al. [
13] proposed a Kalman filtering model for ship motion prediction and applied it to a DD-963 destroyer. In implementing the Kalman filtering, accurate state-space equations and noise statistics are necessary. However, in practical engineering applications, these are difficult to obtain. Therefore, although Kalman filtering is simple in calculation, its use is difficult in practical applications.
Time series methods provide a feasible solution, which only requires the history and current ship motion status data for prediction. These methods are suitable for practical engineering applications, as they do not require a comprehensive understanding of the ship’s dynamic system. Classic time series prediction methods which are widely used include autoregressive (AR) and moving average (MA) models, as well as many extended models based on them [
14,
15,
16]. Yumori et al. [
17] proposed a time domain model based on autoregressive moving average (ARMA) to predict real-time ship motion. This model best fitted an input wave sensor time history to the ship response time history and was applied to aircraft landing on the ship. However, as ship motion is non-stationary, it conflicts with the stationary assumption of classic time series methods. Therefore, many improved methods based on classic time series methods have been proposed. Zhou et al. [
18] conducted a nonlinear autoregressive (NAR) model using an orthogonalization technique, and the experimental results indicated that the NAR model gave better prediction accuracy than the AR model. Jun et al. [
19] combined empirical mode decomposition (EMD) and discrete wavelet transform decomposition (DWT) to improve the AR model for ship motion prediction. Compared with the conventional AR model, this model is more capable of handling nonlinear and non-stationary signals. Suhermi et al. [
20] adapted a hybrid methodology to combine autoregressive integrated moving average (ARIMA) and deep neural network (DNN) models for predicting the roll motion. The hybrid model had a good ability to capture the linear and nonlinear patterns. Although these improved methods have shown their effectiveness, they are still limited in nonlinear ship motion prediction, as explicit relationships between input and output variables for the data sets have to be hypothesized.
Neural networks are a simulation of the biological nervous system [
21]. In contrast to time series methods, neural networks are more capable of modeling nonlinear systems without a priori knowledge of the relationships between input and output variables. In recent decades, neural networks have been successfully applied in various fields, such as natural language processing [
22], computer vision [
23], and autonomous driving [
24]. There are also some USV applications. Wang et al. [
25] presented a path-following controller based on a radial basis function neural network (RBFNN) for formation control in single unmanned surface vehicles. In theory, a neural network can approximate any nonlinear system with arbitrary accuracy [
26]. Accordingly, they have been used to model the nonlinear ship motion dynastic system for prediction. Yang et al. [
27] obtained a ship motion prediction model based on a BP neural network by training the ship motion MATLAB simulation data. Their experimental results showed that the trends of the true and predicted values were consistent. Huang et al. [
28] proposed a coarse and fine-tuning fixed-grid wavelet network model to predict ship roll motion, which is trained by simulated ship roll motion data in regular waves. These neural network models are trained by MATLAB simulation data instead of actual measured sensor data, and therefore the applicability of these models in the real world remains to be verified.
Yin et al. [
29] constructed an ensemble prediction scheme by combining the discrete wavelet transform (DWT) method with a variable-structure radial basis function (RBF) network for real-time ship roll motion prediction. Yin et al. [
30] constructed a variable-structure RBF network sequentially by an adaptive sliding data window (SDW) learning scheme to predict ship roll motion online. The two references adopted a variable-structure RBF neural network as the prediction model. In their model, an innovative algorithm was proposed to adjust the data sliding window online. Both the structure and parameters of RBF neural network were tuned according to the data of the current window in order to achieve real-time prediction. However, in the process of training the neural network, they used real-time data, making the training set relatively small. In this way, the neural network model could be overfitting and its generalization ability was poor. Moreover, they only used ship roll motion time series data to train the neural network model, and ignored the coupling characteristics between the ship movements in six degrees of freedom [
31,
32], as well as the influence of the control system on the ship’s movements.
With the rise of deep learning (DL) [
33], data-driven prediction methods are increasingly applied in various fields [
34,
35]. In maritime applications, Joohyun Woo [
36] proposed a deep-learning-based dynamic model identification method. The long short-term memory (LSTM) [
37] based dynamic model extracted patterns of USV dynamics from free-running test data, and outperformed conventional simplified maneuvering models. However, the data-driven method requires a large amount of data for training. Fortunately, there are normally many sensors installed on the ship at various positions. During the sea trials, these sensors measure the ship’s motion status data and control system status data in real time. These time series sensor data are stored. Therefore, this approach provides a new strategy for ship motion modeling and prediction. We can model the ship motion by mining the hidden information of sensor data.
In this paper, we propose a coupled convolutional neural network (CNN) [
38] and long short-term memory (LSTM) [
37] model for USV roll motion prediction. The data measured by sensors installed on the USV are adopted as the data set of the proposed model. These sensor data contain six-degree-of-freedom motion status data and control status data, and constitute a multidimensional space of the USV movement. The USV roll motion of the next moment is influenced not only by the other five degrees of freedom and control states, but also by the movement of the previous period. CNNs have been proven to be powerful for processing spatial data, and have been widely used in computer vision [
38,
39,
40]. LSTM is a type of recurrent neural network (RNN) [
41] designed for the time-series problem. Therefore, in our proposed model, a CNN is used to extract spatial features and local time-series features of the USV sensor data. The output of the CNN layer are higher-dimensional feature maps, which are the input of the LSTM layer. The wind, waves, and sea current are natural phenomena that usually change continuously over a period of time. They act on the USV, causing a series of changes in the USV’s motion status. So, the current USV roll motion is affected by the motion status in the past. Therefore, the LSTM layer is exploited to model the long-term movement process of the USV and predict the roll motion in the next moment. After that, the fully connected layer is utilized to decode the LSTM output and calculate the final prediction results. The proposed model is able to extract features in both spatial and temporal dimensions to obtain better prediction results for USV roll motion. To prove the effectiveness of the proposed model, it was applied to “JingHai-VI” and “JingHai-III” USVs of Shanghai University.
The paper is organized as follows:
Section 2 presents the proposed coupled CNN and LSTM model for USV roll motion prediction in detail.
Section 3 describes the source of the data set and the data preprocessing process. The experimental results and discussion of two case studies are shown in
Section 4. Finally, the paper is concluded in
Section 5.