A Self-triggered Position Based Visual Servoing Model Predictive Control Scheme for Underwater Robotic Vehicles ^†

Abstract

An efficient position based visual sevroing control approach for Autonomous Underwater Vehicles (AUVs) by employing Non-linear Model Predictive Control (N-MPC) is designed and presented in this work. In the proposed scheme, a mechanism is incorporated within the vision-based controller that determines when the Visual Tracking Algorithm (VTA) should be activated and new control inputs should be calculated. More specifically, the control loop does not close periodically, i.e., between two consecutive activations (triggering instants), the control inputs calculated by the N-MPC at the previous triggering time instant are applied to the underwater robot in an open-loop mode. This results in a significantly smaller number of requested measurements from the vision tracking algorithm, as well as less frequent computations of the non-linear predictive control law. This results in a reduction in processing time as well as energy consumption and, therefore, increases the accuracy and autonomy of the Autonomous Underwater Vehicle. The latter is of paramount importance for persistent underwater inspection tasks. Moreover, the Field of View constraints (FoV), control input saturation, the kinematic limitations due to the underactuated degree of freedom in sway direction, and the effect of the model uncertainties as well as external disturbances have been considered during the control design. In addition, the stability and convergence of the closed-loop system has been guaranteed analytically. Finally, the efficiency and performance of the proposed vision-based control framework is demonstrated through a comparative real-time experimental study while using a small underwater vehicle.

1. Introduction

Vision-based control has been extensively investigated in recent decades for the operation of autonomous underwater vehicles [1,2]. Complex underwater missions, such as surveillance of underwater oil/gas pipelines [3,4,5], inspection of underwater communication cables [6,7], and search for hazardous materials (e.g., naval mines) [8,9,10], require detailed and continuous visual feedback, which can be obtained from either monocular or stereo vision systems.

In general, the visual servoing can be categorized in (i) Position-Based Visual Servoing (PBVS), where the visual features extracted along with the help of the visual tracking algorithm are used for the estimation of the three-dimensional (3D) relative position between the camera and visual target; (ii) Image-Based Visual Servoing (IBVS), where the error function is defined directly on the position of the image features in the image plane between the current and desired images [11]; and, (iii) 2-1/2 Visual Servoing, where the error function is partially formulated in both the Cartesian and the image plane. More information regarding the standard visual servoing techniques can be found in the literature [12,13,14]. Regarding visual servo control in underwater robotics, some previous work for the pipe inspection task (for example, oil platforms) were realized in [15,16]. In [17], visual servoing using the Laser Vision System (LVS) combined with an on-line identification mechanism has been investigated and verified experimentally. In [18,19], stereo vision frameworks have been investigated for underwater operation. The docking of underwater robots employing visual feedback has been addressed in [20,21]. Some applications of visual servoing for station keeping of autonomous underwater vehicles are given in [22,23,24].

The control of an underwater vehicle is generally a highly non-linear problem [25,26]. Conventional control strategies, such as input-output decoupling and local linearization [27,28], Output feedback linearization [29,30,31], and combined frameworks involving Lyapunov theory and backstepping have been investigated in the past for the design of motion controllers for autonomous underwater vehicles. However, most of the aforementioned control strategies yield low closed-loop performance and often demand very precise dynamic parameters, which, in most cases, is quite difficult to obtain [32,33]. Moreover, the effect of ocean currents is either assumed to be known or an exponential observer is adopted for its estimation, thus increasing the design complexity [34]. In addition, by employing all of the aforementioned control strategies, it is not always straightforward to incorporate operational limitations (i.e., visual and/or kinematic constraints) into the vehicle’s closed-loop system [35]. In this spirit, the efficient controlling of underwater robotic vehicles continues to pose significant challenges for control designers in view of the numerous limitations and constraints that arise from the nature of the underwater environment [36]. In particular, AUVs are characterized by constrained high-dimensional non-linear dynamics, especially in the case of underactuated systems [37], which induce significant complexity regarding model uncertainty as well as various operational constraints, such as sensing capabilities and visibility constraints [38,39]. In this context, Non-linear Model Predictive Control (N-MPC) [40], is a proper control approach that is to be used in complex underwater visual servoing missions due to its efficient ability to handle input and state constraints. In a vision based MPC setup, the filed of view limitations could be integrated as state constraints [41]. In this spirit, vision based MPC controller have been employed in medical application [42], as well as for navigation of autonomous aerial vehicles [43] and mobile robots [44]. Furthermore, a vision based terrain mapping-model predictive control approach for autonomous landing of an UAV in unknown terrain is given in [45]. In [46], a vision based approach for path following of an omni-directional mobile robot using MPC is presented.

In a typical vision-based control setup, at every sampling time, the visual feedback that is extracted from the image is used for the generation of a proper error [47]. This requires the selection and extraction of appropriate image features and matching them with the corresponding features in the desired image [48]. This process is usually referred in the literature as Visual Tracking [49,50]. Accuracy and robustness are the main concerns of a Visual Tracking Algorithm (VTA) [51]. However, it is known that the accurate and robust VTA in real-time robotic applications is a very heavy process that demands high computational cost [52]. The latter will result in large energy consumption and might cause delays in the closed-loop system. The aforementioned issues become more apparent in the case of small autonomous robotic vehicles (e.g., UAVs, AUVs), where they are usually equipped with a weak computing unit and, in most cases, suffer from limited energy resources and their recharging procedure is difficult and time and cost consuming [53]. On the other hand, in a N-MPC setup, a constrained Optimal Control Problem (OCP) must be solved at each sampling time. The latter is usually considered to be a very computationally demanding task. In addition, these systems are usually equipped with weak computing units that need to solve the VTA and the OCP of the NMPC at each time instant. This usually results in the reduction of the system accuracy, as bigger sampling times are required. Here, the problem is to design an automatic framework that relaxes the rate of control input calculations and visual tracking activation while maintaining the efficiency of the system. In other words, an automatic visual servoing framework that determines when the system requires tracking the visual data and calculating new control inputs while maintaining system performance at the desired level. This encourages the design of a self-triggered visual servoing strategy that is addressed in this work.

1.1. The Self-Triggered Control Framework

Nowadays, periodic control is the standard control framework that is used in most applications. Quite recently, though, a novel formulation of control schemes in a self-triggered manner is becoming popular. The key idea behind the self-triggered control is that the execution of the control task is not made ad-hoc at every sampling time, but, instead, it uses system feedback in order to sample as infrequently as possible while guaranteeing to preserve the stability of the system, see Figure 1. Consequently, this results in an aperiodic sampling system, while also preserving the system performance and stability. In particular, the self-triggered strategy leads to reducing the number of sampling data from the system, a feature that is important and desirable in a various number of applications with operational limitations in sensing, energy, and communication.

The self-triggered control framework along with a closely related framework, named event-triggered control, comprise the recently introduced event-based control framework. Both approaches, self and event-based control, are comprised, inter alia, by a feedback control framework that calculates the control input and a triggering mechanism that decides when the new control update should occur. However, these frameworks are different; the event-triggered control is more reactive with respect to the self-triggered control, as control inputs in this strategy are calculated when the robot state deviates more than a certain threshold from a desired value, while the self-triggered framework can be considered to be proactive, as it computes the next control input ahead of time. Notice, that, in the event-triggered framework, a constant measurement of the system state is required in order to determine the time of control update. However, self-triggered strategy only requires the latest measurement of the system’s state for determining the next triggering time instant [54]. More preliminary information regarding the event-triggering control techniques can be found in [55,56,57] and the papers quoted therein.

1.2. Contributions

In this paper (A preliminary version of this work, in the absence of a detailed analysis of the methodology, including detailed stability and convergence analysis, detailed controller design, detailed description of real time implementation and experimental results has been reported in to the IEEE European Control Conference [54] as well as in to the IEEE International Conference on Robotics and Automation [58].), by employing N-MPC, a Self-triggered Position Based Visual Servoing control strategy is designed for the motion control of autonomous underwater vehicles. The purpose of this control framework is to guide and stabilize the underwater robot towards a visual target while assuring that the target will not leave the camera’s field of view (Figure 2).

The 3D position of the vehicle with respect to the target is estimated while employing computer vision algorithms and it is described in more detail in Section 4. The choice of PBVS instead of IBVS or 2-1/2 visual servoing is mainly motivated by the inherited advantage of PBVS to control the onboard camera and, as a result, the vehicle itself directly in the 3D space. This makes the design of the N-MPC framework more easy and efficient. The fact that PBVS cannot guarantee the preservation of the visual target inside the image frame is handled by defining of the field of view limitations in the N-MPC structure. The main contribution of this work relies on the design of a vision-based control strategy that automatically determines when the controller and the vision algorithm should be activated while maintaining the closed-loop performance and stability of the system. This leads to reduced tracking of the vision algorithm, CPU effort, and energy consumption, which are of paramount importance in the case of autonomous underwater vehicles in persistent inspection tasks that demand higher system autonomy. Experimental results on event-based formulations are scarce in the literature [59,60,61,62,63,64,65,66]. In this work, the efficiency and performance of the proposed control framework is verified through a comparative real-time experimental study using a small underactuated underwater robot in a small water tank. To the best of our knowledge, this work presents the first experimental validation of an event-based visual servoing control framework for underwater robotic vehicles. In addition, the experimental results are quite satisfactory, as the vehicle reaches and stabilizes at the desired position relative to the visual target, while the number of activation of the visual algorithm is significantly decreased as compared to the conventional case employing classical N-MPC.

The remainder of the paper is organized, as follows: Section 2 presents the problem statement of this paper where the system and operation limitations are formulated in detail. In Section 3, the robust stability analysis for the proposed vision-based self-triggered N-MPC framework is accommodated. Section 4 demonstrates the performance of the proposed motion control framework through a set of experimental results. Finally, Section 5 concludes the paper.

2. Problem Formulation

In this section, initially, the mathematical modeling of the under-actuated underwater vehicle and its constraints are formulated. Subsequently, taking into account the external disturbances and uncertainties of the model, a perturbed version of the system is defined. Finally, the proposed motion control scheme is designed.

2.1. Mathematical Modeling

An autonomous underwater vehicle can be defined as a 6 Degree Of Freedom (DOF) free body with position and Euler angle vector $x={\left[\chi yz\varphi \theta \psi \right]}^{\top }$. Moreover, $v={\left[u\upsilon wpqr\right]}^{\top }$ is the vector of vehicle body velocities, where its components, according to SNAME [67], are surge, sway, heave, roll, pitch, and yaw, respectively (Figure 3). In addition, $\mathit{\tau }={\left[XYZKMN\right]}^{\top }$ is the vector of forces and moments acting on the vehicle center of mass. In this spirit, the dynamics of an underwater robotic vehicle are given in as [68]:

$$\begin{array}{c}\mathit{M}\dot{\mathit{v}}+\mathit{C}\left(\mathit{v}\right)\mathit{v}+\mathit{D}\left(\mathit{v}\right)\mathit{v}+\mathit{g}\left(\mathit{x}\right)=\mathit{\tau }\hfill \\ \dot{\mathit{x}}=\mathit{J}\left(\mathit{x}\right)\mathit{v}\hfill \end{array}$$

(1)

where: $\mathit{M}={\mathit{M}}_{\mathit{RB}}+{\mathit{M}}_{\mathit{A}}$ is the inertia matrix for rigid body and added mass, respectively, $\mathit{C}\left(\mathit{v}\right)={\mathit{C}}_{\mathit{RB}}\left(\mathit{v}\right)+{\mathit{C}}_{\mathit{A}}\left(\mathit{v}\right)$ is the Coriolis and centripetal matrix for rigid body and added mass respectively, $\mathit{D}\left(\mathit{v}\right)={\mathit{D}}_{\mathit{quad}}\left(\mathit{v}\right)+{\mathit{D}}_{\mathit{lin}}\left(\mathit{v}\right)$ is the quadratic and linear drag matrix, respectively, $\mathit{g}\left(\mathit{x}\right)$ is the hydrostatic restoring force vector, $\mathit{\tau }$, is the thruster input vector and $\mathit{J}\left(\mathit{x}\right)$ is the well-known Jacobian matrix [68]. The underwater robot considered in this paper is a 3 DOF VideoRay Pro ROV (Remotely Operated underwater Vehicle) that is equipped with three thrusters, which enable it effective in surge, heave, and yaw motion (Figure 3). This means that the considered underwater robot is under-actuated along its sway axis. Here, due to the robot design, we simply neglect the angles $\varphi$, $\theta$ and angular velocities p and q. In addition, because of the robot symmetry regarding x-z and y-z planes we can safely assume that motions in heave, roll, and pitch are decoupled [68]. Furthermore, the coupling effects are safely can be considered to be negligible since the robot is operating at relatively low speeds. Finally, based on the aforementioned considerations, in this work we consider the kinematic model of the robot, which can be given, as follows [69]:

$$\begin{array}{c}{x}_{k+1}=f(x_k,V_k)+g(x_k,v_k)\Rightarrow \\ \left[\begin{array}{c}{\chi }_{k+1}\\ y_{k+1}\\ z_{k+1}\\ {\psi }_{k+1}\end{array}\right]=\left[\begin{array}{c}{\chi }_k\\ y_k\\ z_k\\ {\psi }_k\end{array}\right]+\left[\begin{array}{ccc}cos{\psi }_k& 0& 0\\ sin{\psi }_k& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{c}{u}_k\\ w_k\\ r_k\end{array}\right]dt+\left[\begin{array}{c}-sin{\psi }_k\\ cos{\psi }_k\\ 0\\ 0\end{array}\right]v_{k}dt,\end{array}$$

(2)

where $x_k={[{\chi }_k,y_k,z_k,{\psi }_k]}^{\top }$ is the state vector at the time-step k, including the position and orientation of the robot relative to the target frame $\mathcal{G}$. Moreover, the vector of control input is $V_k={[u_k,w_k,r_k]}^{\top }$ and $dt$ denotes the sampling period. In addition, following theresults given in [70] and, by employing Input-to-State Stability (ISS), it can be shown that, by applying any bounded control input $[u_k,r_k]$ to the considered nonholonomic robotic system, the velocity about the sway direction $v_k$ can be seen as a bounded disturbance with upper bound $\left|\right|v_k\left|\right|\le \overline{v}$ that vanishes at the point $x=0$. Therefore, the aforementioned point is an equilibrium of the kinematic system of Equation (2). Note, in this work, we denote the upper bound for each variable by the notation $\left(\overline{·}\right)$. Therefore, based on the aforementioned discussion, we consider the system:

$$\begin{array}{c}{x}_{k+1}=f(x_k,V_k)\Rightarrow \\ \left[\begin{array}{c}{\chi }_{k+1}\\ y_{k+1}\\ z_{k+1}\\ {\psi }_{k+1}\end{array}\right]=\left[\begin{array}{c}{\chi }_k\\ y_k\\ z_k\\ {\psi }_k\end{array}\right]+\left[\begin{array}{ccc}cos{\psi }_k& 0& 0\\ sin{\psi }_k& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{c}{u}_k\\ w_k\\ r_k\end{array}\right]dt\end{array}$$

(3)

as the nominal kinematic system of the underwater robotic vehicle. It is worth mentioning that the function $g(x_k,v_k)\in \mathrm{\Gamma }\subset {\mathbb{R}}^4$ is considered as a bounded inner disturbance of the system that vanishes at the origin and $\mathrm{\Gamma }$ is a compact set, such that:

$$\begin{array}{c}\hfill \left|\right|g(x_k,v_k)\left|\right|\le \overline{\gamma }\mathrm{with}\overline{\gamma }\triangleq \overline{v}dt\end{array}$$

(4)

The underwater robot that is considered in this work moves under the influence of an irrotational current, which behaves as an external disturbance to the system. The current has components with respect to the $\chi$, y and z axes, denoted by ${\delta }_{\chi }$, ${\delta }_y$ and ${\delta }_z$, respectively. Moreover, we denote by ${\delta }_c$ the slowly-varying velocity of the current that is bounded by $\left|\right|{\delta }_{c_k}\left|\right|\le {\overline{\delta }}_c$ and it has direction $\beta$ in $\chi -y$ plane and $\alpha$ with respect to the z axis of the global frame, see Figure 4. In particular, we define ${\delta }_k={[{\delta }_{(\chi /k)},{\delta }_{(y/k)},{\delta }_{(z/k)},0]}^{\top }\in \Delta \subset {\mathbb{R}}^4$, with $\Delta$ being a compact set, where:

$$\begin{array}{cc}\hfill & {\delta }_{(\chi /k)}\triangleq {\delta }_{c_k}cos{\beta }_{k}sin{\alpha }_{k}dt\hfill \\ \hfill & {\delta }_{(y/k)}\triangleq {\delta }_{c_k}sin{\beta }_{k}sin{\alpha }_{k}dt\hfill \\ \hfill & {\delta }_{(z/k)}\triangleq {\delta }_{c_k}cos{\alpha }_{k}dt\hfill \end{array}$$

(5)

It is straightforward to show that $\left|\right|{\delta }_k\left|\right|\le \overline{\delta }$, with $\overline{\delta }={\overline{\delta }}_{c}dt$. When considering the aforementioned external disturbances, the perturbed model of the underwater robotic system can be given, as follows:

$$\begin{array}{c}\hfill x_{k+1}=f(x_k,V_k)+{\omega }_k\end{array}$$

(6)

with ${\omega }_k=g(x_k,v_k)+{\delta }_k\in \mathrm{\Omega }\subset {\mathbb{R}}^4$ as the result of adding the inner and external disturbances of the system. $\mathrm{\Omega }$ is a compact set, such that $\mathrm{\Omega }=\Delta \oplus \mathrm{\Gamma }$, where “⊕” denotes the Minkowski addition of sets $\Delta$ and $\mathrm{\Gamma }$. It is worth mentioning that the Minkowski addition set C of two sets $A,B\subset {\mathbb{R}}^n$ is given as $C=A\oplus B=\{a+b|a\in A,b\in B\}$. In this respect, since the sets $\Delta$ and $\mathrm{\Gamma }$ are compact, we can conclude that $\mathrm{\Omega }$ is also a bounded compact set, which is: $\left|\right|{\omega }_k\left|\right|\le \overline{\omega }$ with $\overline{\omega }\triangleq \overline{\delta }+\overline{\gamma }$.

The underwater robot is equipped by a pinhole camera with limited angles of view a and b for $\chi -y$ and $\chi -z$ plane, respectively. In this respect, the state vector of the system x with respect to the visual target is estimated by employing a proper vision algorithm, see Figure 4.

The requirements for the vision system, namely the visibility constraints, are imposed in order to ensure that the target will remains always within the image-plane during the control operation. That is: $[-y_T,y_T]\subseteq [f_{\chi y/1},f_{\chi y/2}]$ and $[-z_T,z_T]\subseteq [f_{\chi z/1},f_{\chi z/2}]$, where $2y_T$ and $2z_T$ denote the width and height of the visual target. In this context, the $[f_{\chi y/1},f_{\chi y/2}]$ and $[f_{\chi z/1},f_{\chi z/2}]$ indicate the camera’s field-of-view on $\chi -y$ and $\chi -z$ plane, respectively (Figure 4). Moreover, we consider a maximum distance $R_{max}$, where the visual target is visible and recognizable for the vision system. The aforementioned requirements are captured by the state constraint set X of the system, given by:

$$x_k\in X\subset {\mathbb{R}}^4,$$

(7)

which is formed by:   

$$\begin{array}{cc}\hfill & -y+\chi tan(\psi -\frac{\mathbf{a}}{2})-y_T\ge 0\hfill \end{array}$$

(8a)

$$\begin{array}{cc}\hfill & y-\chi tan(\psi +\frac{\mathbf{a}}{2})-y_T\ge 0\hfill \end{array}$$

(8b)

$$\begin{array}{cc}\hfill & -z-\chi tan\left(\frac{\mathbf{b}}{2}\right)-z_T\ge 0\hfill \end{array}$$

(8c)

$$\begin{array}{cc}\hfill & z-\chi tan\left(\frac{\mathbf{b}}{2}\right)-z_T\ge 0\hfill \end{array}$$

(8d)

$$\begin{array}{cc}\hfill & R_{max}^2-{\chi }^2-y^2\ge 0\hfill \end{array}$$

(8e)

In addition, the control constraint set $V_{set}$ of the system is formulated, as follows:

$$V_k\triangleq [u_k,w_k,r_k]\in V_{set}\subseteq {\mathbb{R}}^3$$

(9)

It is worth mentioning that the control input constraints are of the form $\left|u\right|\le \overline{u}$, $\left|w\right|\le \overline{w}$ and $\left|r\right|\le \overline{r}$. Thus, we obtain $\parallel V_k\parallel \le \overline{V}$ with $\overline{V}={({\overline{u}}^2+{\overline{w}}^2+{\overline{r}}^2)}^{\frac{1}{2}}$ and $\overline{V},\overline{u},\overline{w},\overline{r}\in {\mathbb{R}}_{\ge 0}$. Therefore, it can be easily shown that system Equation (3) is Lipschitz continuous:

Lemma 1.

The nominal model Equation (3), subject to constraints Equations (8b)–(8e) and (9), is locally Lipschitz in x for all $x\in X$, with a Lipschitz constant $L_f\triangleq {(max\{8,8{\left(\overline{u}dt\right)}^2\}+1)}^{\frac{1}{2}}$.

See Appendix A.1 for the proof.

2.2. Control Design and Objective

The objective here is to guide the perturbed system Equation (6) to a desired compact set that includes the desired state $x_d\triangleq {[{\chi }_d,y_d,z_d,{\psi }_d]}^{\top }\in X$, while respecting the state and control constraints described in Equations (8b)–(8e) and (9), respectively. We employ a predictive controller in order to achieve the aforementioned objective. More specifically, the N-MPC consists in solving an Optimal Control Problem (OCP) at time instant k, with respect to a control sequence $V_f\left(k\right)\triangleq [V\left(k\right|k),V(k+1|k),\cdots ,V(k+N-1|k)]$, for a prediction horizon N. The OCP of the N-MPC is formulated, as follows:

$$\begin{array}{cc}\hfill & \underset{V_f\left(k\right)}{min}{J}_N(x_k,V_f\left(k\right))=\hfill \end{array}$$

(10a)

$$\begin{array}{cc}\hfill & \underset{V_f\left(k\right)}{min}\sum _{j=0}^{N-1}F(\widehat{x}(k+j|k),V(k+j|k))+E\left(\widehat{x}(k+N|k)\right)\hfill \\ \hfill & \mathrm{subject}\mathrm{to}:\hfill \end{array}$$

(10b)

$$\begin{array}{cc}\hfill & \widehat{x}(k+j|k)\in X_j,\forall j=1,\dots ,N-1,\hfill \end{array}$$

(10c)

$$\begin{array}{cc}\hfill & V(k+j|k)\in V_{set},\forall j=0,\dots ,N-1,\hfill \end{array}$$

(10d)

$$\begin{array}{cc}\hfill & \widehat{x}(k+N|k)\in {\mathcal{E}}_f\hfill \end{array}$$

(10e)

where F, E, and ${\mathcal{E}}_f$ are the running, terminal cost, and terminal set, respectively. The solution of the aforementioned OCP Equations (10a)–(10d) at time instant k is an optimal control sequence, being indicated as $V_f^{*}(·)$. It should be pointed out that the specifics for the design parameters, such as the running and terminal costs, as well as the state sets, will be provided in more detail in the sequel. In this context, we denote the predicted state of the nominal system Equation (3) at sampling time $k+j$ by $\widehat{x}(k+j|k)$, where $j\in {\mathbb{Z}}_{\ge 0}$. The state prediction is based on the measurement of the real system at sampling time k, denoted by $x_k$, while applying a sequence of control inputs $\left[V\right(k\left|k\right),V(k+1|k),\cdots ,V(k+j-1\left|k\right)]$. Thus:

$$\widehat{x}(k+j|k)=f(\widehat{x}(k+j-1|k),V(k+j-1|k))$$

Therefore, we have that $\widehat{x}\left(k\right|k)=x_k$. It is worth mentioning that the OCP is formulated and solved for the nominal system and for a specific time horizon. That makes it impossible to address the disturbances beforehand. However, we distinguish the nominal system, denoted as $\widehat{x}(·)$, with the actual one denoted as $x(·)$. Therefore, we can obtain the following preliminary result:

Lemma 2.

The difference between the actual state $x_{k+j}$ at the time-step $k+j$ and the predicted state $\widehat{x}(k+j|k)$ at the same time-step, under the same control sequence, is upper bounded by:

$$\left|\right|x_{k+j}-\widehat{x}(k+j|k)\left|\right|\le \sum _{i=0}^{j-1}{\left(L_f\right)}^i\overline{w}$$

(11)

See Appendix A.2 for the proof.

More specifically, Lemma 2, gives the difference between the real state of the system Equation (6) with the predicted state of the nominal system Equation (3). In order to address this, we employ constraint tightening technique and use a restricted constraint set $X_j\subseteq X$ in Equation (10b) instead of the state constraint set X (More details regarding the constraint tightening technique can be found in the literature [71,72]). By employing the aforementioned constraint tightening technique, we guarantee that the evolution of the perturbed system Equation (6), when the control sequence developed in Equations (10a)–(10d) is applied to it, will necessarily satisfy the state constraint set X. In particular, we denote the restricted constraint set as $X_j=X\sim {\mathcal{B}}_j$, where ${\mathcal{B}}_j=\{x\in {\mathbb{R}}^4:\left|\right|x\left|\right|\le {\sum }_{i=0}^{j-1}{\left(L_f\right)}^i\overline{w}\}$. The set operator “∼” denotes the Pontryagin difference $A,B\subset {\mathbb{R}}^n$ that is defined as the set $C=A\sim B=\{\zeta \in {\mathbb{R}}^n:\zeta +\xi \in A,\forall \xi \in B\}$. Moreover, we define the running and terminal cost functions $F(·)$, $E(·)$, both of quadratic form, i.e., $F(\widehat{x},V)={\widehat{x}}^{\top }Q\widehat{x}+V^{\top }RV$ and $E\left(\widehat{x}\right)={\widehat{x}}^{\top }P\widehat{x}$, respectively, with P, Q and R are positive definite matrices. In particular, we define $Q=diag\{q_1,q_2,q_3,q_4\}$, $R=diag\{r_1,r_2,r_3\}$, and $P=diag\{p_1,p_2,p_3,p_4\}$. For the running cost function F, we have $F(0,0)=0$, and we can also obtain the following:

Lemma 3.

Regarding the cost function $F(x,V)$ we have:

$$\begin{array}{c}\hfill F(x,V)\ge min(q_1,q_2,q_3,q_4,r_1,r_2,r_3){\left|\right|x\left|\right|}^2\end{array}$$

(12)

See Appendix A.3 for the proof.

As we have already mentioned, the state and input constraint sets are bounded; therefore, we have:

Lemma 4.

The cost function $F(x,V)$ is Lipschitz continuous in $X\times V_{set}$, with a Lipschitz constant:

$$L_F=2{(R_{max}^2+z_{max}^2+{(\frac{\pi }{2})}^2)}^{\frac{1}{2}}{\sigma }_{max}\left(Q\right)$$

(13)

where ${\sigma }_{max}\left(Q\right)$ denotes the largest singular value of the matrix Q. Moreover, $z_{max}=R_{max}tan\left(\frac{\mathbf{b}}{2}\right)-z_T$ is the maximum feasible value along the z axis.

See Appendix A.4 for the proof.

Before proceeding with the analysis, we employ some standard stability conditions that are used in N-MPC frameworks:

Assumption 1.

For the nominal system Equation (3), there is an admissible positively invariant set $\mathcal{E}\subset X$, such that the terminal region ${\mathcal{E}}_f\subset \mathcal{E}$, where $\mathcal{E}=\{x\in X:\left|\right|x\left|\right|\le {\epsilon }_0\}$ and ${\epsilon }_0$ being a positive parameter.

Assumption 2.

We assume that in the terminal set ${\mathcal{E}}_f$, there exists a local stabilizing controller $V_k=h\left(x_k\right)\in V_{set}$ for all $x\in \mathcal{E}$, and that E satisfies $E\left(f(x_k,h\left(x_k\right))\right)-E\left(x_k\right)+F(x_k,h\left(x_k\right))\le 0$ for all $x\in \mathcal{E}$.

Assumption 3.

The terminal cost function E is Lipschitz in $\mathcal{E}$, with Lipschitz constant $L_E=2{\epsilon }_0{\sigma }_{max}\left(P\right)$ for all $x\in \mathcal{E}$.

Assumption 4.

Inside the set $\mathcal{E}$ we have $E\left(x\right)=x^{T}Px\le {\alpha }_{\epsilon }$, where ${\alpha }_{\epsilon }=max\{p_1,p_2,p_3,p_4\}{\epsilon }_0^2>0$. Assuming that $\mathcal{E}=\{x\in X_{(N-1)}:h\left(x\right)\in V_{set}\}$ and taking a positive parameter ${\alpha }_{{\epsilon }_f}$, such that ${\alpha }_{{\epsilon }_f}\in (0,{\alpha }_{\epsilon })$, we assume that the terminal set designed as ${\mathcal{E}}_f=\{x\in {\mathbb{R}}^4:E\left(x\right)\le {\alpha }_{{\epsilon }_f}\}$ is such that $\forall x\in \mathcal{E}$, $f(x,h\left(x\right))\in {\mathcal{E}}_f$.

2.3. Problem Statement

At time step k, the solution of the N-MPC Equations (10a)–(10d) provides an control sequence, denoted as $V_f^{*}(·)$, which equals to $V_f^{*}\left(k\right)\triangleq [V^{*}\left(k\right|k),\cdots ,V^{*}(k+N-1|k)]$. In a conventional N-MPC framework, only the first control vector, i.e., $V^{*}\left(k\right|k)$, is applied to the robotic system and the remaining parts of the optimal control sequence $V_f^{*}\left(k\right)$ is discarded. At the next sampling time $k+1$, again, a new state measurement is obtained from the vision algorithm and a new OCP based on this new state measurement is calculated. This is iteratively repeated until the robot has reached to the desired position. However, the proposed self-triggered strategy in this work suggests that a portion of the computed control sequence $V_f^{*}\left(k\right)$ might be applied to the underwater robot and not only the first vector. Let us suppose $k_i$ to be a triggering instant. In the proposed self-triggered control strategy, the control input that is applied to the robotic system is of the form:

$$[V^{*}\left(k_i\right|k_i),V^{*}(k_i+1|k_i),\cdots V^{*}(k_i+d_i|k_i)]$$

(14)

for all $d_i\in [0,k_{i+1}-k_i]\in {\mathbb{Z}}_{⩾0}$, where $k_{i+1}$ is the next triggering instant. Between two consecutive triggering instants, i.e., $[k_i,k_{i+1})$, the control inputs calculated by the N-MPC at the previous triggering time instant are applied to the underwater robot in an open-loop mode i.e., the vision algorithm is not activated and no image processing is performed. Obviously, the smallest and largest possible time intervals are 1 (i.e., $k_{i+1}=k_i+1$) and $N-1$, respectively. The self-triggered framework that is proposed in this work will provide sufficient conditions for the activation of the vision algorithm and triggering the computation of the N-MPC. Currently, we are ready to state the problem treated in this paper:

Problem 1.

Consider the system Equation (6) that is subject to the constraints Equations (7) and (9). The control goal is $\left(i\right)$ to design a robust position based visual servoing control framework provided by Equations (10a)–(10d), such that the system Equation (6) converges to the desired terminal set and $\left(ii\right)$ to construct a mechanism that determines when the control updates, state measurement and next VTA should occur.

3. Stability Analysis of Self-Triggering NMPC Framework

The stability analysis for the system Equations (6)–(14) is addressed in this section. It is already shown in the literature that the closed-loop system in the case of classic N-MPC is Input-to-State Stable (ISS) with respect to the disturbances [71] (More details on the notion of ISS in the discrete-time case can be found in [73].). In the subsequent analysis, we are going to use the ISS notion in order to derive the self-triggering mechanism.

The traditional approach in establishing stability in predictive control consists of two parts, named feasibility and convergence analysis. The aim in the first is to prove that the initial feasibility implies feasibility afterwards and, based on this, in the second part, it is then shown that the system state converges to a bounded set around the desired state.

3.1. Feasibility Analysis

We begin by treating the feasibility property. Before proceeding with the analysis, we provide a necessary definition:

Definition 1.

${\mathcal{X}}^{MPC}=\{x_0\in {\mathbb{R}}^n|\exists$ a control sequence $V_f\in V_{set},{\widehat{x}}_f\left(j\right)\in {\mathcal{X}}_j\forall j\in \{1,\cdots ,N\}$ and $\widehat{x}\left(N\right)\in {\mathcal{E}}_f\}$.

In other words, ${\mathcal{X}}^{MPC}$, a set that contain all of the state vectors for which a feasible control sequence exists satisfying the constraints of the optimal control problem. Assume, now, that, at $k_i\triangleq k$, an event is triggered, thus an OCP is solved and a new control sequence $V_f^{*}\left(k\right)\triangleq [V^{*}\left(k\right|k),\cdots ,V^{*}(k+N-1|k)]$ is provided. Now, consider control inputs at time instants $k+m$ with $m=1,\cdots ,N-1$, which are based on the solution at sampling time k, $V_f^{*}\left(k\right)$. These can be defined, as follows:

$$\begin{array}{c}\hfill \tilde{V}(k+j|k+m)=\left\{\begin{array}{cc}\hfill V^{*}(k+j|k)& \mathrm{for}j=m,\cdots ,N-1\hfill \\ \hfill h\left(\widehat{x}(k+j|k+m)\right)& \mathrm{for}j=N,\cdots ,N+m-1\hfill \end{array}\right\}\end{array}$$

(15)

Let $N-1$ control sequences ${\tilde{V}}_f^m\left(k\right)$ be comprised by the control inputs of Equation (15), i.e.,

$$\begin{array}{cc}\hfill & {\tilde{V}}_f^1\left(k\right)=[V^{*}(k+1|k),V^{*}(k+2|k),\cdots ,h\left(\widehat{x}(k+N|k+1)\right)]\hfill \\ \hfill & {\tilde{V}}_f^2\left(k\right)=[V^{*}(k+2|k),\cdots ,h\left(\widehat{x}(k+N|k+2)\right),h\left(\widehat{x}(k+N+1|k+2)\right)]\hfill \\ \hfill & ⋮\hfill \\ \hfill & {\tilde{V}}_f^{N-1}\left(k\right)=\left[V^{*}(k+N-1|k)\right),\cdots ,h\left(\widehat{x}(k+2N-2|k+N-1)\right)]\hfill \end{array}$$

Notice that the time-steps $k+m$ are the discrete-time instants after the time-step of the triggering instant $k_i$, i.e., $[k,k+1,k+2,\cdots ,k+N-1]\equiv [k_i,k_i+1,k_i+2,\cdots ,k_i+N-1]$. With the help of Assumption 2 and by taking the feasibility of the initial control sequence at sampling time k into account, it follows that, for $m=1,\dots N-1$, we have $\tilde{V}(k+j|k+m)\in V_{set}$. We can prove finally that $\widehat{x}(k+N+1|k+m)\in {\mathcal{E}}_f$ for all $m=1,\dots ,N-1$:

Proof. 

From Lemma 2, we can derive that:

$$\begin{array}{cc}\hfill & \left|\right|\widehat{x}(k+N|k+1)-\widehat{x}(k+N|k)\left|\right|\le L_f^{N-1}\overline{w}\hfill \\ \hfill & \left|\right|\widehat{x}(k+N|k+2)-\widehat{x}(k+N|k)\left|\right|\le L_f^{N-2}\left((1+L_f)\overline{w}\right)\hfill \\ \hfill & ⋮\hfill \\ \hfill & \left|\right|\widehat{x}(k+N|k+m)-\widehat{x}(k+N|k)\left|\right|\le L_f^{(N-m)}\sum _{i=0}^{m-1}{\left(L_f\right)}^i\overline{w}\hfill \end{array}$$

by employing the Lipschitz property of $E(·)$, we have:

$$\begin{array}{cc}\hfill & E\left(\widehat{x}(k+N|k+m)\right)-E\left(\widehat{x}(k+N|k)\right)\hfill \\ \hfill & \le L_E\left|\right|\widehat{x}(k+N|k+m)-\widehat{x}(k+N|k)\left|\right|\hfill \\ \hfill & \le L_E{L}_f^{(N-m)}·\sum _{i=0}^{m-1}{\left(L_f\right)}^i\overline{w}\hfill \end{array}$$

Having in mind that $\widehat{x}(k+N|k)\in {\mathcal{E}}_f$ and by employing Assumption 4, we obtain the following:

$$\begin{array}{c}\hfill E(\widehat{x}(k+N|k+m)\le {\alpha }_{{\epsilon }_f}+L_{E}G\left(m\right)\overline{w}\end{array}$$

with $G\left(m\right)\triangleq L_f^{(N-m)}·{\sum }_{i=0}^{m-1}{\left(L_f\right)}^i$. It should hold that $E\left(\widehat{x}(k+N|k+m)\right)\le {\alpha }_{\epsilon }$, i.e., $\widehat{x}(k+N|k+m)\in \mathcal{E}$, thus:

$$\begin{array}{cc}\hfill & {\alpha }_{{\epsilon }_f}+L_{E}G\left(m\right)\overline{w}\le {\alpha }_{\epsilon }\Rightarrow \hfill \\ \hfill & \overline{w}\le \frac{({\alpha }_{\epsilon }-{\alpha }_{{\epsilon }_f})}{L_E{L}_f^{(N-m)}·{\sum }_{i=0}^{m-1}{\left(L_f\right)}^i}\hfill \end{array}$$

(16)

Now, applying a local control law, we get $\widehat{x}(k+N+1|k+m)\in {\mathcal{E}}_f$ for all $m=1,\cdots ,N-1$. From these results, it can be concluded that ${\mathcal{X}}^{MPC}$ is a robust positive invariant set if the uncertainties are bounded by Equation (16) for all $m=1,\cdots ,N-1$. Notice that Equation (16) should still hold for $m=1$ for the problem to be meaningful, in the sense that it should be feasible at least in the time-triggered case. □

3.2. Convergence Analysis

Herein, we show that the state of the actual system convergences to a desired terminal set. In order to prove this, we show that a proper value function is decreasing. First, we define the optimal cost at the time-step k as $J_N^{*}\left(k\right)=J_N(x_k,V_f^{*}\left(k\right))$, which is evaluated under the optimal control sequence. In the same spirit, the optimal cost at a time-step $k+m$ with $m\in [1,N-1]$ is denoted as $J_N^{*}(k+m)=J^{*}(x_{k+m},V_f^{*}(k+m))$. Now, we denote by ${\tilde{J}}_N(k+m)$ the “feasible” cost, which is evaluated from the control sequence ${\tilde{V}}_f^m\left(k\right)$, which is ${\tilde{J}}_N(k+m)={\tilde{J}}_N(x_{k+m},{\tilde{V}}_f^m\left(k\right))$. In the following, we will employ this “feasible” cost in order to obtain the difference $J_N^{*}(k+m)-J_N^{*}\left(k\right)$. More specifically, the difference between the optimal cost at time k and the feasible sequence at time-step $k+j$ by employing Equation (15) is:

$$\begin{array}{c}\Delta J_m={\tilde{J}}_N(k+m)-J_N^{*}\left(k\right)\le \\ (L_E{\left(L_f\right)}^{(N-m)}+L_F\sum _{i=0}^{N-(m+1)}{\left(L_f\right)}^i)\overline{w}-\sum _{i=0}^{m-1}min(q_1,q_2,q_3,q_4,r_1,r_2,r_3)\left|\right|\widehat{x}(k+i|k)\left|\right|\end{array}$$

(17)

See Appendix B for the proof. From the optimality of the solution, we have:

$$\begin{array}{c}\hfill J_N^{*}(k+m)-J_N^{*}\left(k\right)\le {\tilde{J}}_N(k+m)-J_N^{*}\left(k\right)\end{array}$$

(18)

This result along with the triggering condition that is going to be derived in the next subsection will enable us to provide conclusions for the stability and convergence of the closed-loop system.

3.3. The Self-Triggered Mechanism

This section presents the self-triggering framework that is proposed in this work. Let us consider that, at time-step $k_i$, an event is triggered. We assume that the next triggering time $k_{i+1}$ is unknown and should be found. More specifically, triggering time $k_{i+1}\triangleq k_i+d_i$ should be such that the closed-loop maintains its predefined desired properties. Therefore, a value function $J_N^{*}(·)$ is required to be decreasing. In particular, given Equations (17) and (18), for a triggering instant $k_i$ and a number of time-step $d_i$ after $k_i$, with $d_i=1,2,\dots ,N-1$ it can be obtained the following:

$$\begin{array}{c}\hfill J_N^{*}\left(k_{i+1}\right)-J_N^{*}\left(k_i\right)\le (L_E{\left(L_f\right)}^{(N-d_i)}+L_F\sum _{i=0}^{N-(d_i+1)}{\left(L_f\right)}^i)\overline{w}-L_Q\left(d_i\right)\end{array}$$

(19)

where:

$$L_Q\left(d_i\right)=\sum _{i=0}^{d_i-1}min(q_1,q_2,q_3,q_4,r_1,r_2,r_3)\left|\right|\widehat{x}(k+i|k)\left|\right|$$

The time instant $k_{i+1}$ should be such that:   

$$\begin{array}{c}\hfill (L_E{\left(L_f\right)}^{(N-d_i)}+L_F\sum _{i=0}^{i=N-(d_i+1)}{\left(L_f\right)}^i)\overline{w}\le \sigma L_Q\left(d_i\right),\end{array}$$

(20)

where $0<\sigma <1$. Substituting Equation (20) to (19), we obtain

$$\begin{array}{c}\hfill J_N^{*}\left(k_{i+1}\right)-J_N^{*}\left(k_i\right)\le (\sigma -1)L_Q\left(d_i\right)\end{array}$$

(21)

This suggests that, by considering $0<\sigma <1$, decreasing of the value function is guaranteed. In particular, in view of Equation (21), we can conclude that the value function $J_N^{*}(·)$ has been proven to be decreasing at the triggering instants. Next, we study the convergence of the state of the system under the proposed self-triggered framework:

Convergence of System under the Proposed Self-Triggered Framework

We have proven in Equation (20) that the value function $J_N^{*}(·)$ is always decreasing with respect to the previous triggering instant. In other words, the value function cannot be guaranteed to be monotonically decreasing at every time-step, as the standard Lyapunov theory dictates. Thus, additional arguments will be provided in order to prove convergence of the state of the closed-loop system to a bounded set. In particular, the following steps are going to be followed: first, we are going to provide the steps to derive a suitable Lyapunov-function candidate and, secondly, we are going to show that this Lyapunov function is an ISS-Lyapunov function and, according to standard definitions, if a system admits an ISS-Lyapunov function, then the system is ISS with respect to the external disturbances, [73]. Thus, finding a suitable ISS-Lyapunov function immediately implies that our system is ISS with respect to disturbances and, thus, the states of the closed-loop system are converging to a bounded set.

Proposition 1.

Our proposed Lyapunov function candidate is the following:

$$W\left(k\right)\triangleq \left\{\begin{array}{cc}{J}_N^{*}\left(k\right)& \mathit{for}{d}_i=1\\ \sum _{j=1}^{d_i-1}\{J_N^{*}(k+j)·(d_i-j)\}+d_i{J}_N^{*}\left(k\right)& \mathit{for}{d}_i>1\end{array}\right.$$

(22)

Now, if $d_i=1$ at every time instant then our system boils down to the classic time-triggered MPC, where, in [71], it has been shown that the closed-loop system is ISS with respect to the disturbances. However, we are going to show that Equation (22) is also an ISS-Lyapunov function for $d_i\ge 1$. This is going to be shown for $d_1=2$ and then Equation (22) is derived by induction.

Proof. 

Now, assume that $d_i=2$. From Equation (19), it follows that:

$$\begin{array}{c}\hfill J_N^{*}(k+1)-J_N^{*}\left(k\right)\le (L_E{\left(L_f\right)}^{(N-1)}+L_F\sum _{i=0}^{N-2}{\left(L_f\right)}^i)\overline{w}-L_Q\left(1\right),\end{array}$$

as well as:

$$\begin{array}{c}\hfill J_N^{*}(k+2)-J_N^{*}\left(k\right)\le (L_E{\left(L_f\right)}^{(N-2)}+L_F\sum _{i=0}^{N-3}{\left(L_f\right)}^i)\overline{w}-L_Q\left(2\right)\end{array}$$

adding the last two inequalities, yields:

$$\begin{array}{c}\hfill J_N^{*}(k+2)+J_N^{*}(k+1)\le 2J_N^{*}\left(k\right)-(L_Q\left(1\right)+L_Q\left(2\right))+(L_E{\left(L_f\right)}^{(N-2)}+L_F\sum _{i=0}^{N-3}{\left(L_f\right)}^i)(1+L_f)\overline{w}\end{array}$$

(23)

adding and subtracting the terms ${\sum }_{j=1}^{d_i-1}{J}_N^{*}(k+j)(d_i-j)$ in Equation (23), we can obtain:   

$$\begin{array}{c}{J}_N^{*}(k+2)+2J_N^{*}(k+1)\le \\ J_N^{*}(k+1)+2J_N^{*}\left(k\right)+(L_E{\left(L_f\right)}^{(N-2)}+L_F\sum _{i=0}^{N-3}{\left(L_f\right)}^i)(1+L_f)\overline{w}-(L_Q\left(1\right)+L_Q\left(2\right))\end{array}$$

(24)

Considering as a Lyapunov function:

$$W\left(k\right)=J_N^{*}(k+1)+2J_N^{*}\left(k\right)$$

Equation (24) is re-written as:

$$\begin{array}{c}\hfill W(k+1)-W\left(k\right)\le +(L_E{\left(L_f\right)}^{(N-2)}+L_F\sum _{i=0}^{N-3}{\left(L_f\right)}^i)(1+L_f)\overline{w}-(L_Q\left(1\right)+L_Q\left(2\right))\end{array}$$

(25)

It is now evident that, by induction, we can reach to Equation (22) for an arbitrary $d_i$, following the same procedure. Moreover, from Equation (25), it is obvious that $W\left(k\right)$, as defined in Equation (22), is an ISS-Lyapunov function, thus the proposed framework is ISS stable with respect to the external disturbances and the proof is completed. □

Thus, having the aforementioned analysis in mind, the next tracking of the vision system as well as updating the control law should be occur when Equation (20) is violated. This means that, at the triggering time instance, the condition Equation (20) must be checked for each consecutive time-step, i.e., for $d_i=1,2,\cdots$. Thus, we check which time step does not meet this condition and set it to the next triggering instant $k_{i+1}$. Based on the above discussion, it can be understood that, in the proposed self-activated framework, the time step $k_{i+1}$ is found beforehand at time $k_i$. Moreover, it is worth mentioning that the term $L_Q\left(d_i\right)$ only includes predictions of the nominal system that can be easily computed by forward integration of Equation (3) for time-steps $d_i\in [1,N-1]$. Now, based on the aforementioned stability results, we state the theorem for the proposed vision-based self-triggered framework:

Theorem 1.

Consider the system of autonomous underwater vehicle described by Equation (6), which is subject to state and input constraints given in Equations (7) and (9) under the N-MPC framework and assume that the Assumptions 1–4 hold. The vision tracking and control update times that are provided by Equation (20) and the N-MPC framework given in Equations (10a)–(10d), which is applied to the autonomous underwater vehicle in an open-loop fashion during the inter-sampling periods, drive the closed-loop system into the terminal set ${\mathcal{E}}_f$ that includes the desired pose configuration with respect to the visual target.

The pseudo-code description of the proposed real-time self-triggering position that is based visual servoing is given in Algorithm 1:

Algorithm 1 Real-time algorithm of the proposed self Triggered PBVS-NMPC framework:
1: Triggering time:	▷ At triggering time $k_i$
2:   $x\left(k_i\right)\leftarrow \mathrm{VTA}$	▷ Trigger the VTA, get $s\left(k_i\right)$
3:   $V_f^{*}\left(k_i\right)\leftarrow \mathrm{OCP}\left(x\left(k_i\right)\right)$	▷ Run OCP of (10a)–(10d)
4:   $k_{i+1}=k_i+d_i\leftarrow$Solve Equation (20) for $d_i$	▷ The next triggering time
5: for$i=1\to d_i$do
6:   Apply the control inputs $V^{*}(k_i+i|k_i)$ to the underwater robot.
7: gotoTriggering time.

At time $k_i$, we assume that the Vision Tracking Algorithm (VTA) is triggered, the optimal control problem of the N-MPC Equations (10a)–(10d) is run and a control sequence for the time interval $[k_i,k_i+N-1]$ is provided. The solution of Equation (20) provides the next triggering time $k_{i+1}$, as it is already stated. During the time interval $i\in [k_i,k_{i+1})$, the control inputs $V^{*}(k_i+i|k_i)$ are applied to the underwater robot in an open-loop fashion. Next, at $k_{i+1}$ the vision system is tracked and the OCP of the N-MPC Equations (10a)–(10d) is solved again while employing the new state measurement $x\left(k_{i+1}\right)$ as the initial values in Equations (10a)–(10d). The controller follows this procedure until the robot converges and stabilizes towards the visual target.

4. Experiments

In this section, the efficacy of the proposed position based self triggered framework demonstrated through a real time comparative experimental study. A real-time stabilization scenario was considered by employing a small and under-actuated underwater vehicle.

4.1. System Components

The small underwater robot used in following experiments is a 3-DOF (VideoRay PRO, VideoRay LLC, Figure 3), which is equipped with 3 thrusters, and a USB camera. The image dimensions are 640x480 pixels. A visual target is located on an aluminum surface plane that is fixed inside the tank. The system software is conducted in the Robotics Operating System (ROS, http://www.ros.org), and the code is written in C++ and Python.

The state vector of the underwater robot regarding the visual target is estimated in real time while using the ROS package ar_pose (http://www.ros.org/wiki/ar_pose), which is an Augmented Reality Marker Pose Estimation algorithm that is based on the ARToolkit software library (http://www.hitl.washington.edu/artoolkit/). The target detection and robot localization in initial and desired pose configuration are shown in Figure 5. The constrained N-MPC that was used in this real-time experiment was designed using the NLopt Optimization library [74].

4.2. Experimental Results

The goal in the following comparative experimental studies is the stabilization of the underwater robot at the desired configuration towards the visual target. Two experiments were held for comparison. More specifically, in the first experiment, we employed a classic N-MPC (i.e., time activation at each sampling time), while, in the second experiment, the self-triggered framework proposed in this work was used. The initial as well as desired position of the underwater vehicle relative to the target frame is $[{\chi }_{in},y_{in},z_{in},{\psi }_{in}]=[-1.2,0.45,0.1,-0.401]$ and $x_d={[{\chi }_d,y_d,z_d,{\psi }_d]}^{\top }=[-0.6,0.0,0.0,0.0]$, respectively. In the initial pose, the target appears in the right side of the camera view because of the negative yaw angles of the vehicle with respect to the target frame (See Figure 5). Note that this is a difficult initial pose and, if one does not take the visual constraints into account, the experiment will fail. The sampling time and prediction horizon were selected as to be $dt=0.15$ sec and $N=6$, respectively. It is worth mentioning that the sampling time is selected based on the response frequency of the closed-loop system, while the prediction horizon is selected based on the computational capabilities of the onboard unit to solve the optimization problem. The more capable the computational unit is, the larger the prediction horizon can be considered. The maximum allowable velocities for the considered underwater robot in surge, heave, and yaw direction were selected as $\overline{u}=0.2$ m/s, $\overline{w}=0.3$ m/s and $\overline{r}=0.3$ rad/s, respectively. It is worth mentioning that the considered allowable velocities might be required for the needs of several common underwater tasks (e.g., seabed inspection, mosaicking), where the vehicle is required to move with relatively low speeds with a predefined upper bound. The design matrices Q, R, and P are defined as $Q=diag(0.5,4.5,4.5,0.1)$, $R=diag(0.17,0.1,1)$, and $P=diag(1,1,1,1)$, respectively. The maximum permissible distance on the referred water tank is $R_{max}=1.5$ m. The results of the experiment are presented in Figure 6, Figure 7, Figure 8, Figure 9 and Figure 10. In Figure 6, the evolution of the robot coordinates in x, y, and $\psi$ for both experiments are depicted. When comparing two experiments, it is obvious that the underwater robot in both experiments has reached and stabilized in the desired position towards the visual target and the operational limitations (F.O.V and control saturation) remained satisfied. It can also be seen that the system performances in the case of proposed self triggering framework are better (or at least the same) as compared to the classical approach.

In Figure 7 and Figure 8, the camera view and the coordinates of the visual target center during the experiments are presented, respectively. It is obvious that the target remains inside of the F.O.V of the camera. Figure 9 presents the triggering evaluation in the case of the proposed self triggered framework. For the value 1 in vertical axis, the vision algorithm has been activated, thus the image has been processed, the state vector has been estimated, N-MPC has been evaluated, and control inputs are calculated. For value 0, the rest of the last computed control inputs implemented on the underwater robot in an open-loop fashion, and, therefore, no optimization and no image processing is running. Moreover, in the case of classic N-MPC, vision tracking and N-MMPC are running always at all sampling times, as has been already stated. It is worth mentioning that by employing the proposed self triggered condition, the triggering of the vision tracking algorithm and the N-MPC have been reduced by 50% (124 triggering instead of 253) regarding the classic N-MPC framework. When comparing the triggering instants of Figure 9 to the image target center coordinate in Figure 8, one might notice that when the target is going to leave the image plane (at the region of six and 14–17 s of the experiment), the triggering instants are more frequent. This fact appears at region of the 40 and the 80–110 sampling times, respectively, in the Figure 9. When comparing triggering instants of Figure 9 to the state evolution of the system Figure 6, one might notice that when the robot is getting near to the desired position the triggering instants are more frequent. This is because, close to the desired position, the system becomes more demanding due to the visibility limitations as the target becomes larger in the camera view and because external disturbances move the robot from the desired position.

The computational time in the case when a new state information from vision system and a new control sequence are calculated (triggering instant) is approximately $0.1$ s, while, in the case of the open loop control (employing the proposed self triggered framework), it is being reduced to $0.0002$ s. This is because, in the case of self-triggering framework, neither the vision tracking algorithm nor the optimization process is executed between two triggering instants. Finally, Figure 10 presents the control inputs during the experiments. It is easy to see that the control constraints remained satisfied during the experiment.

4.3. Video

This work is accompanied by a video presenting the experimental procedure of Section 4: https://youtu.be/mdRM2ThaOQM.

5. Conclusions

In this paper, a self-triggered position-based visual servoing control framework for autonomous underwater vehicles was presented. The main idea of this work is to activate the vision tracking algorithm and the optimization of the N-MPC in an aperiodic way, which is only when required and not in each sampling time. By employing the proposed vision-based self-triggered control strategy, both the control inputs and the next activation time are evaluated to avoid continuous measurements from the vision system. During the inter-sampling instants, the control inputs that are calculated by N-MPC are applied to the underwater robot in an open-loop mode and, therefore, no optimization and no image processing is running between two triggering instants. This results in a reduction in processing time as well as energy consumption and, therefore, increases the accuracy and autonomy of the Autonomous Underwater Vehicle. The latter is of paramount importance for persistent underwater inspection tasks. Rigorous robustness analysis, along with sufficient conditions for triggering, is provided in this work. The effectiveness of the proposed vision-based self-triggered control framework is verified through a comparative experimental study using an underwater robot. In these experiments, by employing the proposed self triggered control strategy, we achieved a significant 50% reduction in the activation of the vision tracking algorithm and the OCP as compared to the classic N-MPC framework. Future research efforts will be devoted towards extending the proposed methodology for multiple Autonomous Underwater Vehicles, including not only static, but also moving target as well as conducting complex real-time experiments employing a team of cooperative AUVs.