End-to-End Underwater Acoustic Communication Based on Autoencoder with Dense Convolution

Abstract

To address the problems of the high complexity and poor bit error rate (BER) performance of traditional communication systems in underwater acoustic environments, this paper incorporates the theory of deep learning into a conventional communication system and proposes data-driven underwater acoustic filter bank multicarrier (FBMC) communications based on convolutional autoencoder networks. The proposed system is globally optimized by two one-dimensional convolutional (Conv1D) modules at the transmitter and receiver, it realizes signal reconstruction through end-to-end training, it effectively avoids the inherent imaginary interference of the system, and it improves the reliability of the communication system. Furthermore, dense-block modules are constructed between Conv1D layers and are connected across layers to achieve feature reuse in the network. Simulation results show that the BER performance of the proposed method outperforms that of the conventional FBMC system with channel equalization algorithms such as least squares (LS) estimation and virtual time reversal mirrors (VTRM) under the measured channel conditions at a certain moment in the Qingjiang River.

1. Introduction

Currently, underwater acoustic (UWA) communication plays an important role in the transmission of underwater information in military and civilian applications. However, unlike wireless channels, the complexity and variability of the marine environment leads to UWA channels having physical characteristics such as narrow bandwidths, large time delays, strong multipaths, large undulations and susceptibility to interference from a variety of factors, which cause the degradation of the received signal quality [1,2,3]. Therefore, the exploration of high-speed and robust underwater acoustic communication technologies has always been a focus of research. Orthogonal frequency division multiplexing (OFDM) is a commonly used underwater acoustic multicarrier modulation method with high spectrum utilization and can effectively resist frequency selective fading. The basic idea is to change a high-speed serial bit stream into a low-speed parallel bit stream through serial parallel conversion and to then modulate the multiple parallel bit streams into several sub carriers for transmission. FBMC introduces offset quadrature amplitude modulation on the basis of OFDM and adopts prototype filters with excellent time-frequency (TF) focusing characteristics, which further reduces band leakage and relaxes the strict orthogonality between subcarriers [4,5]. It maintains strong resistance to inter-symbol interference and inter-carrier interference even without cyclic prefixes and thus has great potential for application in underwater acoustic communications.

In recent years, deep learning has been successfully applied in the fields of computer vision and natural language processing with its strong ability to fit nonlinear information, and the concept of introducing deep learning into wireless communication systems is gradually emerging. In the physical layer of communication systems, researchers have attempted to combine original signal processing modules, such as channel coding [6,7], modulation identification [8,9], channel estimation [10,11,12] and signal detection [13,14,15], with deep neural network models, demonstrating the effectiveness and potential of future data-driven wireless communication systems. For example, Ye et al. [16] proposed to replace the channel estimation and signal detection module at the receiver with a deep neural network to optimize the conventional receiver and improve the robustness of the system. Zhang Jing et al. [17] proposed a deep-neural-network-based underwater OFDM receiver with only one neural network for the whole signal processing, and Zhu Yunan et al. [18] proposed to use the basic CNN model for signal detection at the FBMC receiver, both of which achieved better BER performance than that of conventional receivers. However, the above approach only designs the receiver side of the communication system, so only local optimization can be achieved. If the conventional transmitter module is also involved, the system can be further data-driven and globally optimized. Autoencoder networks are typical of self-supervised learning and consist of an encoder network that aims to extract the input signal features and compress the data and a decoder network that is responsible for recovering the original data to the maximum extent possible. Sebastian Dörner et al. [19] built a complete communications system solely composed of neural networks with open-source deep learning software libraries, which only presents a 1 dB worse block error rate than that of a practical baseline system. O’Shea et al. [20] interpreted the communications system as an autoencoder, where the traditional signal recovery process is considered an end-to-end reconstruction task that strives to jointly optimize transmitter and receiver components in a deep learning way. Fayçal Ait Aoudia et al. [21] analyzed two sources of end-to-end learning performance improvement in wireless channel models. One is to reduce the dependence on pilot quantity without losing BER performance, and the other is to completely eliminate orthogonal pilots by learning the receiver together with expert information.

In this paper, we apply deep learning theory to the physical layer and propose underwater acoustic filter banks multicarrier communication based on convolutional autoencoder networks. The encoder and decoder network of the convolutional autoencoder network is distributed at both the transmitter and receiver for global optimization, and the signals are restored through compression and reconstruction. A dense connection structure is also used between each convolutional layer to achieve feature reuse in the network learning process. The simulation results show that the BER performance and robustness of the proposed scheme are better than those of conventional pilot-based channel estimation methods.

2. FBMC System Model

As shown in Figure 1, the FBMC transmitter first performs symbol mapping on the input bit stream, and the mapped complex symbols are rearranged by offset quadrature amplitude modulation (OQAM) modulation, interleaving the real parts and imaginary parts by half a symbol period. After inserting pilots, the $M$ symbols are modulated onto $M$ subcarriers through an inverse fast fourier transform (IFFT) transform and a polyphase network consisting of prototype filters.

The discrete-time signal output after FBMC modulation can be expressed as [4]

$$x\left(l\right)={\displaystyle \sum _{m=0}^{M-1}{\displaystyle \sum _{n=0}^{N-1}{a}_{m,n}{g}_{m,n}\left(l\right)}}$$

(1)

where $M$ is the number of subcarriers, $N$ is the number of FBMC symbols, $a_{m,n}$ represents the real-valued transmitted symbols at the TF point $\left(m,n\right)$ and

$$g_{m,n}\left(l\right)=g\left(l-n\frac{M}{2}\right)e^{j\frac{\left(m+n\right)\pi }{2}}{e}^{j\frac{2\pi ml}{M}}$$

(2)

is the prototype filter impulse response at this location, also known as the subcarrier basis function. $e^{j\frac{\left(m+n\right)\pi }{2}}$ is the phase rotation factor. The prototype filter is designed to maintain orthogonality in the real domain, and the inner product of the subcarrier basis functions at $\left(m,n\right)$ and $\left(p,q\right)$ satisfies

$$\Re \left\{\langle g_{m,n}|g_{p,q}\rangle \right\}=\Re \left\{{\displaystyle \sum _l{g}_{m,n}\left(l\right)g_{p,q}^{*}\left(l\right)}\right\}={\delta }_{m,p}{\delta }_{n,q}$$

(3)

where $\Re \{\cdot \}$ represents taking the real part. ${\delta }_{m,p}$ is the Kronecker delta that satisfies ${\delta }_{m,p}=1$ when $m=p$. Therefore, only when $\left(m,n\right)=\left(p,q\right)$, ${\delta }_{m,p}{\delta }_{n,q}=1$; otherwise, ${\delta }_{m,p}{\delta }_{n,q}=0$. Assuming that the channel is frequency flat and invariant in a filter duration, the received data can be expressed as

$$y_{p,q}=H_{p,q}{a}_{p,q,}+\underset{I_{p,q}}{\underbrace{j{\displaystyle \sum _{\left(m,n\ne p,q\right)}{H}_{m,n}{a}_{m,n}\langle g_{m,n}|g_{p,q}\rangle }}}+{\eta }_{p,q}$$

(4)

where $H_{p,q}$ is the channel frequency response (CFR) and $I_{p,q}$ and ${\eta }_{p,q}$ are the Gaussian white noise and inherent imaginary interference at $\left(p,q\right)$. Due to the excellent TF focus characteristics of the prototype filter, the interference at the TF point $\left(p,q\right)$ mainly comes from the first-order neighborhood defined as ${\Omega }_{p,q}^1=\{\left(p\pm 1,q\pm 1\right),\left(p\pm 1,q\right),\left(p,q\pm 1\right)\}$. Given that the CFR remains constant over adjacent TF points, Equation (4) can be simplified as [22]

$$y_{p,q}\approx H_{p,q}{c}_{p,q}+{\eta }_{p,q}$$

(5)

where

$$c_{p,q}=\left(a_{p,q}+j{b}_{p,q}\right)$$

(6)

$j{b}_{p,q}$ is the imaginary part of the interference term, which can be approximated as $j{b}_{p,q}=j{\displaystyle \sum _{\left(m,n\right)\in {\Omega }_{p,q}^1}{a}_{m,n}\langle g_{m,n}|g_{p,q}\rangle }$. If the transmitted data at TF point $\left(p,q\right)$ and its first-order neighbors are known, $c_{p,q}$ can be used as a pseudo pilot to estimate the channel frequency response through the following equation.

$${\widehat{H}}_{p,q}=\frac{y_{p,q}}{c_{p,q}}\approx H_{p,q}+\frac{{\eta }_{p,q}}{c_{p,q}}$$

(7)

3. Dense-Convolutional-Autoencoder-Based UWA FBMC Communications System

3.1. Network Structure

An autoencoder network [23] is a deep learning model proposed to reconstruct the input signal within a reasonable error range. In a perfect case, the output signal is completely consistent with the input signal. The idea coincides with the purpose of underwater communication. An autoencoder network can be structurally divided into encoder and decoder networks. The encoder network compresses the high-dimensional input information to a low-dimensional abstract representation by extracting the sample features to achieve dimension reduction and the compression of data. The decoder network maps low-dimensional abstract data into desired outputs to reproduce the input samples. The traditional autoencoder network is mostly constructed based on fully connected (FC) layers. When the input is a long sequence, the number of neuron nodes at each layer increases, and the network complexity increases accordingly. A convolutional autoencoder network uses convolutional layers and pooling layers to replace fully connected layers, and it reduces training parameters and increases network throughput by means of a partial connection mode and weight sharing mechanism of convolutional layers. The structure of the Conv1D autoencoder network is shown in Figure 2.

The coding process of a Conv1D autoencoder network can be expressed as

$$O_k=g\left(X\otimes W_k^e+b_k^e\right)$$

(8)

where $g(\cdot )$ is the activation function, $X$ is the input volume and $W_k^e$ and $b_k^e$ are the weights and biases of the convolutional kernel at the $k-\mathrm{th}$ layer of the encoder network, respectively. $\otimes$ stands for the convolutional operation, $O_k$ is the abstract features extracted by the $k-\mathrm{th}$ convolutional layer of the encoder network. The $k$ compression features generated by the coding network are eventually sent to the decoding network to reconstruct the input signal. The decoding process of a convolutional autoencoder network can be written as a reverse process

$$\widehat{X}=g\left(O_d\otimes W_k^d+b_k^d\right)$$

(9)

where $W_k^d$ and $b_k^d$ denote the weight and bias of the $k-\mathrm{th}$ convolutional kernel in the decoder network, respectively. The Sigmoid function $f\left(z\right)=1/\left(e^{-z}+1\right)$ is used as the activation function for the output layer to map the predicted values to the interval [0, 1]. When $\widehat{X}\ge 0.5$, $\widehat{X}=1$, and when $\widehat{X}<0.5$, $\widehat{X}=0$, which caters to the binary input bit stream.

In addition, the dense connection is adopted between convolutional layers, which is constructed as a Dense-Net module, to further improve feature learning efficiency. This feature reuse method can guarantee the accuracy of prediction without relying too much on the kernel size. The input to each layer is derived from the output mapping features of all layers before the current layer, and the output of the current layer is used as the input for each layer follows. The output characteristics of all previous layers are concatenated and delivered on the channel-wise dimension, as depicted in Figure 3.

Assuming the Dense-Net module has $L$ layers, the output of layer $l-\mathrm{th}$ can be expressed as [24]

$$Z^l=g\left(Z^{l-1},Z^{l-2},\dots Z^{l-L}\right)$$

(10)

where $Z^l$ denotes the output features at layer $l$ and $L$ is the depth of dense connections.

3.2. System Model

The idea of employing an autoencoder network is extended to the underwater acoustic FBMC system by placing the encoder network and the decoder network at the transmitter and the receiver, respectively, and the proposed scheme is shown in Figure 4. On the transmitter side, the encoder learns an appropriate mapping method to convert the binary bit stream to a complex symbol, which is then modulated by the integrated filter bank and sent to the underwater acoustic channel for transmission. No explicit channel estimation and equalization process is required on the receiver side, and the received complex symbols are directly input to the decoder after fast fourier transform (FFT) transformation to recover the original transmitted signal. The densely connected Conv1D layers are only employed in the decoder network because the initial Conv1D encoder networks are able to reproduce the constellations perfectly.

Specifically, the binary bit stream $b$ is randomly generated and entered into the well-trained encoder to map to a complex vector after a series of nonlinear operations. The parameter settings in the encoder and decoder with three Conv1D layers, respectively, are shown in

Table 1. Parameter settings of autoencoder network.

Layer Name		Kernel Size	Stride	Input Features Dimension
Encoder	Conv1D + BN	$m_1^e$	3	1
	Conv1D + BN	$m_2^e$	3	$m_1^e$
	Conv1D + BN	2	2	$m_2^e$
Untrainable FBMC modulation
Decoder	Conv1D + BN	$m_1^d$	2	2
	Conv1D + BN	$m_2^d$	3	$m_1^d$
	Conv1D + BN	$m_3^d$	3	$m_1^d+m_2^d$
	Flatten + FC	-	-	-

, where $m_l^e$ and $m_l^d$ denote the number of convolutional kernel channels in the $l-\mathrm{th}$ layer of the coding network and decoding network. Batch normalization (BN) layers are used to speed up the training and convergence of the network and to prevent overfitting. The kernel size determines the dimension of the output features of the current convolution layer. Because only constellation mapping is replaced, the encoder has an input feature dimension of 1 for the first Conv1D layer and an output feature dimension of 2 for the last Conv1D layer. Then, the real and imaginary parts of the mapped symbols are staggered by half a symbol period, i.e., OQAM modulation. With the effect of the phase rotation factor, the adjacent real and imaginary parts at the TF points remain orthogonal in the real domain. After IFFT and filtering by a prototype filter bank with different offsets named the polyphase network (PPN), the data symbols are modulated onto $M$ corresponding subcarriers, forming $N$ FBMC symbols $x_{FBMC}$. Typically, the prototype filter uses a non-rectangular pulse with length $L_g=KT$, where $K$ is the overlap factor and $T$ is one FBMC symbol period. The prototype filter used in this paper is PHYDYAS [25] with overlapping factor $K=4$. Finally, the signals on each subcarrier are superimposed through the underwater acoustic channel.

The received signal is sent to the decoder for signal recovery after PPN and FFT. As the neural network cannot process complex data directly, the signal needs to be pre-processed. By taking the real part, the real and imaginary parts of the signal are stacked as all real tensors as from $\pm a\pm bi$ to $[\pm a,\pm b]$, which also resists imaginary interference as in a traditional FBMC receiver. Dense-Net modules are constructed by using dense connections between each convolutional layer of a decoder to maximize feature sharing for better recovery of the original data. The decoder is highly similar in structure to an encoder and can be seen as its inverse process where the input features dimension is accumulated layer by layer. In addition, the Flatten function is used to transform a higher-dimensional tensor into a one-dimensional tensor, and the fully connected layer is used as the output layer to predict all the binary bits.

We record $\widehat{b}$ and the corresponding input $b$ as a set of training data, and the output tensor is consistent with the input. The cross-entropy function is considered the loss function to measure the gap between $\widehat{b}$ and $b$, which is expressed as:

$$J\left(b,\widehat{b}\right)=-{\displaystyle \sum _{i=1}^{n_b}{b}_i\log \left({\widehat{b}}_i\right)+\left(1-b_i\right)\log \left(1-{\widehat{b}}_i\right)}$$

(11)

where $b_i$ and ${\widehat{b}}_i$ are the supervised value and output prediction of the $i-\mathrm{th}$ node in the output layer and $n_b$ is the number of bits predicted at a time. The Adam stochastic gradient descent algorithm is selected as the optimizer, which is updated in reverse to adjust parameters such as network weights and bias, and the loss function is minimized by iterative finetuning to fit the nonlinear relationship between the input and output. An early stopping mechanism with monitoring on loss is employed to shorten the redundant training epoch. The transmitted data can be recovered without explicit channel estimation and equalization through a completely data-driven approach. The training phase is completed when the value of the loss function or the number of iterations learned reaches a preset value, and the network weights and biases stop being updated at the same time. When a new received signal is input, the well-trained detector directly restores the original transmitted bits.

4. Simulation Analysis

In order to demonstrate the effectiveness of the proposed convolutional-autoencoder-network-based communication method for underwater acoustic FBMC systems, the channel impulse response measured by linear-frequency-modulated signals at a certain moment in the Qingjiang experiment in July 2017 is used for simulation. The water depth at the experimental site is about 100 m, the transmitting and receiving vessels are about 1.5 km apart and both are in free drifting conditions. The transmitting transducer and receiving hydrophone are suspended at a depth of 30 m and 10 m, respectively. The channel impulse response is shown in Figure 5. The system parameters of the underwater acoustic FBMC modulation and the hyperparameter setting for the Conv1D autoencoder network are shown in

Table 2. FBMC parameter settings.

Parameter	Setting
Sampling rate	128 kHz
Number of subcarriers	512
Prototype filter	PHYDYAS
Bandwidth Carrier frequency	6.4 kHz 12.8 kHz
Constellation mapping	4 QAM

and

Table 3. Conv1D autoencoder hyperparameter settings.

Parameter	Setting
Learning rate	${10}^{-4}$
Initialization algorithm	He initialization
Mini-batch size	512
Zero padding	Same
Early stopping (patience = 5)	${\Delta }_{\min }={10}^{-4}$

. To pursue a compromise between detection efficiency and accuracy, $n_b=256$ bits are predicted at a time. The training samples and test data are divided by 9:1.

The LS estimation algorithm and VTRM equalization algorithm are usually used in traditional signal detection based on pilots. The advantage of the LS algorithm is its simple structure and low computational effort. The VTRM algorithm first requires accurate Doppler compensation of the received signal, and the channel characteristics are estimated from the compensated received signal. The estimated channel is then time-inverted and convolved with the compensated received signal. In this paper, the BER performance of the system is analyzed in a signal-to-noise ratio (SNR) from 0 dB to 30 dB. Figure 6 compares the BER performance of the Conv1D autoencoder underwater FBMC communications when $D=5\times {10}^4$ samples are generated. $\left(m_1^e,m_2^e,m_3^e\right)=\left(8,4,2\right)$ for the encoder network, and $\left(m_1^d,m_2^d,m_3^d\right)=\left(2,4,8\right)$ for the decoder network. The proposed scheme is much better than that of the conventional method, and the performance of the system is further improved when dense connections are adopted between convolutional layers in the decoder. The advantage is even more obvious when the SNR is greater than 15 dB. This indicates that, by training with plenty of communication data, the proposed scheme is able to achieve similar constellation mapping at the transmitter and a better demodulation at the receiver, which provides a new idea for constructing end-to-end underwater communication systems.

Figure 7 compares the effects of different data sample sizes on the system’s BER performance under the same network parameter settings. The kernel size of each convolutional layer remains unchanged, and the system is evaluated when the sample size is $1\times {10}^4$, $2\times {10}^4$, $5\times {10}^4$ and $2\times {10}^5$. The simulation results show that the BER performance of the proposed scheme decreases significantly with the increase in the number of training samples and is superior to the hand-crafted schemes in all cases. It indicates that the data-driven communication system is sensitive to the number of training samples.

To explore the effect of the network structure, especially the effect of the number of convolution kernels on the system performance, we set up two comparisons: case1 with $\left(m_1^e,m_2^e,m_3^e\right)=\left(4,2,2\right)$, $\left(m_1^d,m_2^d,m_3^d\right)=\left(2,2,4\right)$ and case2 with $\left(m_1^e,m_2^e,m_3^e\right)=\left(32,8,2\right)$, $\left(m_1^d,m_2^d,m_3^d\right)=\left(2,8,32\right)$. The BER curve is shown in Figure 8. When $D = 2\times {10}^4$, the BER performance gain of case2 compared with that of case1 slightly increases with an improvement in SNR. The deeper model performs better with a handful of training data. When $D = 2\times {10}^5$, case2 and case1 show similar performances when the SNR is lower than 15 dB. This indicates that the proposed scheme is also influenced by the complexity of the model, and the gain from a deeper model is limited when the amount of data is sufficient.

Here, to explore the detection complexity of Conv1D AE and Dense Conv1D AE, we compare the running time per step of case1 and case2 in the training and testing phases in

Table 4. Complexity comparison between case1 and case2.

		Conv1D AE		Dense Conv1D AE
		Training	Testing	Training	Testing
case1	$D=2\times {10}^4$	1 s 15 ms	7 ms	1 s 22 ms	8 ms
	$D=2\times {10}^5$	6 s 15 ms	3 s 8 ms	9 s 23 ms	3 s 8 ms
case2	$D=2\times {10}^4$	2 s 38 ms	1 s 12 ms	2 s 48 ms	1 s 14 ms
	$D=2\times {10}^5$	21 s 54 ms	5 s 13 ms	24 s 3 ms	6 s 16 ms

. The network construction in this paper is implemented based on Nvidia Tesla T4 and the Keras platform. The simulation results show that the cost of the proposed method is mainly concentrated in the model training phase, and the system complexity increases with deeper network structures and increasing data sizes. In the testing phase, Dense Conv1D AE and Conv1D AE share similar run times per step for signal detection.

5. Conclusions

In this paper, we propose convolutional-autoencoder-network-based underwater acoustic FBMC communications, which uses an encoder and decoder to partially replace the functional modules of the transmitter and receiver in original communication system. In particular, dense connection is adopted between Conv1D layers in the decoder to further improve the detection capacity, which inherits the features of CNN, such as feature learning, fast training convergence, feature reuse to enhance feature extraction, etc. The end-to-end structures can perform the modulation and demodulation tasks jointly. The simulation results show that the proposed method exhibits better system performance than that of the conventional method under the same conditions. The deep-learning-based model effectively optimizes the system structure and achieves more efficient and reliable transmission, which is of great significance to the future development of underwater communication.