An Adaptive Hierarchical Sliding Mode Controller for Autonomous Underwater Vehicles

: The paper addresses a problem of efﬁciently controlling an autonomous underwater vehicle (AUV), where its typical underactuated model is considered. Due to critical uncertainties and nonlinearities in the system caused by unavoidable external disturbances such as ocean currents when it operates, it is paramount to robustly maintain motions of the vehicle over time as expected. Therefore, it is proposed to employ the hierarchical sliding mode control technique to design the closed-loop control scheme for the device. However, exactly determining parameters of the AUV control system is impractical since its nonlinearities and external disturbances can vary those parameters over time. Thus, it is proposed to exploit neural networks to develop an adaptive learning mechanism that allows the system to learn its parameters adaptively. More importantly, stability of the AUV system controlled by the proposed approach is theoretically proved to be guaranteed by the use of the Lyapunov theory. Effectiveness of the proposed control scheme was veriﬁed by the experiments implemented in a synthetic environment, where the obtained results are highly promising.


Introduction
It is well-known that about 70% of our earth surface is covered by water, and there are many underwater areas that have not been discovered by humans yet. For a long history, humans have wanted to discover every corner of the earth but have not completely been successful due to our own limited capability. More importantly, a lot of unexploited resources and treasure covered by water have not been uncovered yet, which has triggered many countries and organizations around the world to continuously invest their effort and money to develop systems or devices that can enable them to efficiently explore those. In fact, many manned underwater submersibles, where human pilots are required to be on board, have been developed though they are not widely implemented due to considerable life-threatening risks to the pilots if they are involved in long, deep and distant journeys [1]. Therefore, more recently, there has been increasing attention in developing unmanned autonomous underwater vehicles (AUVs). It is noted that in this work we consider AUVs and underwater unmanned vehicles (UUVs) are similar. In other words, these are underwater devices that can operate autonomously. For the simplicity purpose, we now call these devices as AUVs.
Since the AUV could operate autonomously, it promises potential to be efficiently deployed in many underwater exploration and exploitation including scientific research as well as military and civilian applications [2][3][4]. The potential applications of using AUVs comprise exploring oil and gas resources [5] and building offshore industries [6], monitoring ocean ecosystems [7,8], surveying and mapping sea floor [9] and inspecting pipeline conditions [3]. In terms of scientific research, AUVs can be employed to collect data for hydrological investigation [8] and monitor environmental parameters for marine geosciences [5]. More importantly, AUVs are widely utilized in defense including surveillance [10], rescue [3] or clearing sea mines installed by enemies [11]. Thus, continuous improvement of efficiency of operations of AUVs is paramount in developing the devices. It is noted that underwater environment is very uncertain and highly complicated due to limited visibility. In ocean, for instance, environment for operations of AUVs is almost unknown and unpredictable, where there are many unavoidable external disturbances such as ocean currents.
One of the fundamental problems in an AUV is how to effectively and robustly control its motions given uncertainties including its own nonlinearities and unpredictable ocean environment, where it is required to precisely present in both spatial and temporal scales [4,12]. Controlling motions of an AUV is even more challenging due to complexity of its internal electromechanical system and unavoidable but unpredictable external disturbances including ocean currents [4,13]. In other words, in those conditions, many parameters in the AUV system can not exactly be determined, which is burdensome for controller design. Moreover, the AUV itself is underactuated; that is, the number of control inputs is less than that of control outputs [14,15]. It is noted that designing controllers for underactuated systems is a fundamental but challenging task [16][17][18].
In the past years, considerable research has been conducted to design motion controllers for AUVs. For instance, the author of the work [19] proposed to exploit a proportional-integralderivative (PID) controller for controlling depth of Delphin2 AUV, where the parameters of the control law are determined by the use of the Ziegler-Nichols technique. Wei et al. [20] exploited the backstepping method to develop a control scheme based on linear statespace dynamics to control AUVs in vertical direction, where a nonlinear observer was also proposed to monitor disturbances in the system. In similar fashion of controlling vertical motions of AUVs, the work [21] employs both the nonlinear H-infinity and backstepping algorithms to develop the control law.
To maintain robustness in operations of an AUV given nonlinearities in its internal electromechanical systems and uncertainties caused unpredictable but unavoidable external disturbances, sliding mode control (SMC) has attracted much attention from practitioners, engineers and researchers [22]. For instance, the authors of the work [23] proposed a second order SMC scheme for a four degrees of freedom (DoFs) Cyclops AUV though the internal stability of the proposed system has not been proved yet [2]. Yan et al. [24] took the parameter uncertainties and the external disturbances caused by oceanic waves into consideration when they designed an integral fast terminal SMC algorithm for a three DoFs linear motion AUV. Recently, Tran et al. [25] developed an indirect control strategy to adjust vertical motions of a hybrid AUV where the pitch angle was considered. The results demonstrate simplicity of the proposed controller; however, it was not verified under conditions of the system parameter uncertainties [2].
Nevertheless, in the SMC schemes, it is assumed that all the parameters are determined and certain, which is not the case for AUVs. AUVs themselves are highly nonlinear and operate in extremely uncertain environments, e.g., deep oceans. In other words, the parameters of the AUV and control systems are highly uncertain and impractical to be exactly determined. Therefore, it is practically expected that those parameters are adaptively estimated online [26,27]. Theoretically, the unknown, uncertain and nonlinear parameters can be learned through some adaptive strategies such as neural networks and fuzzy logic systems [25]. For instance, Chen et al. [28] exploited the radial basis function neural network to estimate the unknown parameters in a system, which could then be used in a dynamic surface controller for a class of uncertain strict-feedback nonlinear systems. In the work of synchronizing multiple mobile platforms in trajectory tracking [29], the authors employed an adaptive neural network to learn the unknown functions. A recurrent neural network was also utilized in designing a predictive feedback control strategy for a class of uncertain, time-delay, dynamic and nonlinear systems [30].
By bringing robust control and adaptive learning together, in this paper, we propose to employ the hierarchical sliding mode control (HSMC) technique to develop a robust control scheme for AUVs. There are two levels in the proposed controller. In the first level, there are two sliding surfaces designed for controlling the fully actuated and underactuated outputs, respectively. To handle nonlinearities and uncertainties in the AUV systems, it is proposed to exploit neural networks to adaptively learn the system parameters over time. More importantly, by the use of the Lyapunov theory, we theoretically prove that stability of the proposed control system is guaranteed. The proposed approach was validated by the different scenarios in the synthetic simulation experiments, where the obtained results demonstrate its practicality.
The remaining of the paper is arranged as follows. A model of an AUV is introduced in Section 2. Section 3 presents the proposed controller, where its adaptive learning mechanism and stability analysis are also discussed. Evaluation of the proposed algorithm is summarized in Section 4 before conclusions are drawn in Section 5.

Motion Modelling
In order to efficiently control motion of an autonomous underwater vehicle (AUV), we consider a model with six degrees of freedom (DoFs) including surge, sway, heave, roll, pitch and yaw [31,32], which present for both the position and orientation of the marine device. Parameters of the model are summarized in Table 1.

Force and Moments Velocities Positions and Angles
Motion in x direction (surge) Table 1, it can be seen that the first three rows present positions, coordinates, linear velocities and corresponding translational motions of a marine AUV along three x, y and z directions. Moreover, the last three rows present its rotations, coordinates, angular velocities and rotational motions. It is noted that controlling an AUV with six DoFs is very complicated while in most applications roll (i.e., rotating about x axis) and pitch (i.e., rotating about y axis) rarely happen [23] since either roll or pitch is applied, it may cause upside down everything inside the body of AUVs and hurt pilots. Therefore, in this work we propose to consider a model of four DoFs including three translational motions in x, y and z directions and rotational motion about z axis, which is presented as follows [1].
where m is mass of the marine vehicle.
Let η = [x, y, z, ψ] T define a vector of its positions along x, y and z axes and rotation about z axis while we denote v = [u, s, w, r] T as a vector of the marine device's linear velocities along x, y and z axes and angular one around z axis, respectively. Hence, the nonlinear dynamic equations in (1) can be rewritten in a matrix form as follows.
where τ is a vector of the control signal inputs.
presents rotation about z axis while is the system inertia matrix. Moreover, presents the Coriolis and system radial force matrix, and is the hydrodynamic attenuation matrix. It is noticed that the matrices J(η), M(v), C(v), D(v) hold the following properties. •

Dynamics of Under-Actuated AUV Systems
It is assumed that in the proposed system one will provide control signals for driving the marine vehicle along x and y axes. That is, there are two inputs in the control system. However, as presented in the model (1) the system has four outputs including x, y, z and ψ, which leads to an AUV to be considered as an underactuated device (i.e., the number of inputs is smaller than that of outputs). To effectively deal with an underactuated control system, we propose to mathematically split its model into fully actuated subsystem and underactuated one. In other words, we define η = [η 1 , where Since M 22 is a positive definite matrix, from (7) it yieldṡ In other words, (9) can be rewritten in a simplified form as follows.
If one assumes that the parameters are chosen so thatM is a positive definite matrix, from (10) it hasv which results iṅ Substituting (11) and (12) into (7), the dynamic equations of an underactuated AUV can be represented as follows. where

Adaptive Hierarchical Sliding Mode Control Law
In order to design a control system to effectively control a multi-input multi-output (MIMO) underactuated AUV, in this section a controller based on the sliding mode control (SMC) method is first introduced. Though the SMC is robust [26], its parameters are required to be known. Nevertheless, in practice all the parameters in an AUV are nonlinear, unknown and uncertain due to external disturbance such as ocean currents. Therefore, an adaptive learning scheme to adaptively learn those parameters by using neural networks is also presented. More importantly, stability of the proposed approach is then theoretically analysed by the use of the Lyapunov theory.

Control Scheme
Without loss of generality, the dynamic model of an AUV (13) can be represented in a form of a MIMO underactuated system as follows. where Now let us define the vectors of errors between the system responses and the desired references by where η 1d is a vector of the desired translational position references of the marine device along x and y axes while η 2d is a vector of the desired translational position and rotational angle references along and about z axis, respectively. Given the model in (14), a hierarchical SMC (HSMC) can be designed by defining two first level sliding surfaces for the fully actuated output η 1 and the underactuated output η 2 , specified by where k 1 and k 2 are positive constants. The second level sliding surface for the whole control system is then added by where λ and β are positive parameters. In the control context, if a HSMC law is appropriately designed, the second level sliding surface can eventually converge to zero [33]. In other words, the HSMC scheme should be given by two laws including switch and equivalent control ones as follows.
Theoretically, the switch control law τ sw is used to navigate the system states to a particular sliding surface whilst the equivalent control law τ eq is utilized to maintain those states on the sliding surface. Now, in order to ensure stability of the underactuated AUV given the proposed control law, a Lyapunov function candidate of the second level sliding surface is considered by To analyse the stability, first derivative V with respect to time can be computed by where τ eq = τ eq1 + τ eq2 , τ sw = τ sw1 + τ sw2 and sgn(·) is a sign function. k is chosen so that the chattering phenomena can be eliminated. To guarantee stability of the underactuated AUV, the control laws are proposed to be computed by which leads to dV dt = S TṠ = −(kS T S + σS T sgn(S)) ≤ 0.
Therefore, the total control signal in the proposed control approach for a marine underactuated AUV is summarized as follows.

Adaptive Learning
As presented in Section 2, in the model of the marine AUV, it is assumed that the matrices C and D are constant. Nonetheless, in fact, accurately determining the parameters in those two matrices given the system nonlinearity, noise and external disturbances such as ocean currents is impractical. Thus, in this paper, it is proposed to exploit neural networks to approximately estimate these uncertain matrices.
To this end, let us rewrite f 2 (X) in (14) as follows. v where If an approximate nonlinear function can be defined by where W is an ideal weight matrix and h(·) is a Gaussian activation function, then f 2 (v) can be formulated by where A ∈ R n×n is a Hurwitz matrix. The definition leads tȯ If a neural network with n neurons and N inputs is designed to approximately estimate the weightŴ ∈ R N×n , (27) can approximately be presented bẏv The errors between the exact values in (27) and their approximate estimation in (28) can be calculated bẏṽ

Stability Analysis
In the controller design, it is critical to analyse stability of the proposed control algorithm, which can be used to guarantee its effectiveness in controlling an underactuated AUV. We mathematically demonstrate stability of our proposed approach as follows.
We first define a Lyapunov function as We then split the Lyapunov function into two sub-functions as follows.
Now we compute first derivative of V 1 as given bẏ In general, one has where θ ≥ 0, I n is an n × n identity matrix while P ∈ R n×n is a positive definite symmetric matrix, which leads to (33) to be simplified bẏ In other words,V In similar fashion, first derivative of V 2 can be also calculated bẏ If the weight parameters can be estimated through a neural network as followṡŴ thenV 2 can be represented bẏ By using some following properties of a matrix can be rewritten bẏ which leads tȯ where h m ,W andH are positive bounds of h(Ŷ Tv 2 ), W and H , respectively.

Combining (34) and (36), one haṡ
If we define then (37) can be represented bẏ On the other hands, if we choose Derivative of the Lyapunov function in (38) is simplified tȯ Ultimately, by choosing it yieldsV which theoretically guarantees stability of the underactuated AUV control system, following the Lyapunov criteria.

Simulation Results and Discussions
In order to illustrate effectiveness of the proposed control scheme in controlling an underactuated AUV, we conducted experiments in a synthetic environment. It is noted that in the experiments, we selected a marine glider, a typical AUV, where its parameters are summarized in Table 2. It is noted that these parameters were obtained from our designed and fabricated AUV [32].
To measurement the control performance, we run the simulations in 200 s and then compared the system's out responses with the desired references. The results are presented in the following.
There are three scenarios that were considered in our experiments.
• In the first scenario, the desired references were set to constants and the system was noiseless.
• In the second scenario, the desired references were still set to constants but the system operated under an external disturbance. • In the last scenario, the desired references along x and y axes were set to the predefined sinusoidal signals while the desired references along and about z axis were set to constants. In this experiment, the system was also assumed to operate under an external disturbance.

Parameters Values
It is noted that by utilizing the external disturbance, we had opportunity to verify stability of our proposed control algorithm when the AUV system operates under noise. We understand that the most critical external disturbance in oceans influencing on operations of the marine devices is ocean currents. Therefore, in this work, we approximately mimicked ocean currents by the sinusoidal noises as ∆ = [20 sin(0.01t), 10 cos(0.01t)] T for both x and y directions, respectively. The external disturbance is depicted in Figure 1.
It is also noted that this work only focuses on a controller for AUVs. Thus, sensor perception for obstacle detection, environment mapping and path planning is beyond scope of this work. In other words, one can assume that those sensors are already embedded on AUVs to serve those perception purposes.

First Scenario
In all the first experiment, it was assumed that origin of the coordinate was set to the current location of the glider. It was then expected the marine device navigated to 11 m and 8 m along x and y directions, respectively. The AUV was also required to sink down 5 m from the current water level while rotating about z axis at an angle of 0.6 rad once it reached the desired location. The obtained results are demonstrated in Figure 2. There is no overshoot in motions along x and z axes as illustrated in Figure 2a,c though the overshoots in translational and rotational motions along y axis and about z axis are approximate 11% and 21%, respectively as shown in Figure 2b,e. More importantly, the parameters of the system were adaptively learned over time. The learning obtained by the adaptive learning mechanism as demonstrated in Figure 2e shows that norm of the weight parameters reached to zero once the marine vehicle reached to the desired references.

Second Scenario
Similar to the first scenario, in the second experiment, the glider was expected to be driven to 8 m and 5 m along x and y axes, respectively, from its current location while being submerged down 12 m from the current water level. Though the vehicle was not set to rotate about z axis, it operated under the external disturbance as illustrated in Figure 1. Not surprisingly, as the obtained results are shown in Figure 3, the proposed control algorithm controlled the marine device very well, which allowed the vehicle to be able to reach the desired destination in less than 40 s. Interestingly, there is no overshoot in the motions along x, y and z directions. However, under influence of the external disturbance, there is overshoot in the rotational motion as illustrated in Figure 3e though the AUV was not required to rotate at all. Though the peak overshoot was suppressed quickly, the rotational motion was still fluctuated over time as a result of the external disturbance interference. Nevertheless, the vibration over time in the rotational motion is negligible; that is, stability of the control system was guaranteed.

Third Scenario
In the third experiment, a more complicated scenario was set up. From its current location that was considered as the origin of the coordinate, the glider was expected to be driven along sinusoidal trajectories [3 sin(0.01t), 2 cos(0.01t)] T in x and y directions. However, its underactuated outputs including the translational and rotational motions along and about z axis were set to constants of −10 m and 0 rad, respectively. In other words, the AUV was required to be submerged down 10 m from the current water level. In this experiment, the marine vehicle also operated under the external disturbance as shown in Figure 1.
Being controlled by our proposed control scheme, the output responses of the AUV over time in four DoFs (the motions along x, y, z axes and the rotation about z axis) are demonstrated in Figure 4. It can be clearly seen that the vehicle could track the desired references very well. Moreover, the device could submerge to the set point within first 20 s without overshoot as illustrated in Figure 4c. Regarding the rotational motion about z axis as shown in Figure 4d, there was overshoot in the response though it happened in very short period of time. This overshoot could be considered as noise as in practice the corresponding actuators could not respond in that short period of time. The noise could be caused by not only the external disturbance as can be seen in Figure 3d but more importantly also the pre-set sinusoidal references for the motions in both x and y directions. More interestingly, regarding adaptively learning the system parameters over time as can be seen in Figure 4e, norm of the weight parameters could not be suppressed down to zero as compared with those in Figures 2e and 3e but was maintained around 2 over time. The difference was resulted in by the fact that the AUV was required to track the sinusoidal trajectories in x and y directions, which was not the case in the first two experiments.

Conclusions
The paper has discussed a new approach to efficiently and adaptively control AUVs. A four DoF AUV model is considered, where the vehicle is controlled to move along x, y and z directions and rotate about z axis to reach a desired destination. The hierarchical SMC algorithm is exploited to design the control law, where its unknown and uncertain parameters are adaptively learned by the use of neural networks. The adaptive learning mechanism allows the marine device to robustly operate under unavoidable external disturbances such as ocean currents. The proposed method was verified in three different possible scenarios in a synthetic environment, where the obtained results are highly practical.
Overall, the simulation results has verified effectiveness of our proposed controller. In other words, two levels in the sliding surfaces of our control scheme guarantee robustness of AUV operations. Moreover, the adaptive mechanism in our design ensures AUVs to efficiently operate in unknown and uncertain conditions. In future works, we will implement our controller in a real-life AUV [32].

Acknowledgments:
The authors would like to thank the editor and anonymous reviewers for their useful comments for improving the quality of this paper.

Conflicts of Interest:
The authors declare no conflict of interest.