Research on Nonlinear Control Method of Underactuated Gantry Crane Based on Machine Vision Positioning

Abstract

The movement of the gantry crane is controlled by an symmetry underactuated system, and has poor robustness in precise positioning. A new active control method based on the machine vision positioning is proposed in this paper, and the trajectories are planned after the detection of starting and ending points. A new type of energy storage function is given in this paper, and a coupling control law is derived to minimize the load swing in the process of precise positioning. The equilibrium point of the closed-loop system is checked though Lyapunov and LaSalle’s theorems, and the calculation results are verified through experimental investigations. The results show that the equilibrium points are asymptotically stable, and the proposed control method is of better robustness. The findings provide a new kind of control method with higher efficiency, and can help with the precise control of gantry cranes.

1. Introduction

Gantry cranes have the advantages of high payload capacity, good operational flexibility, and less energy consumption, and are widely used in loading and transportation processes at material yards and ports. The payload is led to the reserved position by the control system, and the unexpected payload oscillation is designed to be minimized. The swing freedom of the payload lacks the driver to control. Therefore, the control system of the gantry crane is symmetry underactuated, and is of poor robustness because the independent control inputs are fewer than the degrees of freedom (DOF). During the past several decades, the control system has attracted research interests from many researchers. Fang et al. [1] proposed a control scheme based on motion planning and adaptive crane underactuated system, which can still achieve good control performance under unknown system parameters. Pezeshki et al. [2] proposed a feedback linearized modelless adaptive controller and a fuzzy approximation adaptive sliding membrane controller to control the underactuated crane system. Sun et al. designed a crane underactuated control strategy based on the energy optimal solution [3] and quasi-proportional integral derivative (PID) [4], considering the hook and payload double swing problem. Lu et al. [5] designed a dual crane cooperative operation controller, and proved the stability of the system to equilibrium point by using Lyapunov techniques and LaSalle’s invariance theorem. Most of the studies focused on the passive target control, and the target positions need to be input several times in consecutive works during lifting and translation. Multiple manual inputs lead to accumulative errors, and the accuracy and efficiency are decreased. Therefore, a control system with high accuracy and robustness is needed.

The active control system takes the working conditions into consideration, and can reduce the payload oscillation in the working process. Visual systems are widely applied in active control systems, and the accumulative errors brought by the multiple position inputs can be minimized. For the studies of vision-based active control systems, Yasir et al. [6] proposed a combination of visible light and smartphone acceleration sensors. Hossen et al. [7] used a combination of computer vision sensors and visible light communication with light-emitting diodes (LED). Truc [8] designed a navigation method for ground fuzzy line tracking and quick response (QR) code recognition. The above methods have achieved good results in the application of visual positioning and detection, and the control precision is proven to be improved compared with the passive control systems. The control strategies of the vision-based systems are derived from the motions of the objectives, and are closely related with the dynamic characteristics of the system. Recent applications of the control system on the gantry crane mostly focus on the two-dimensional (2D) motions, and only the oscillation motions in the trolley plane are taken into consideration. However, the payload moves in a three-dimensional (3D) space when the trolley and the gantry move simultaneously, and the impact of gantry motion on the oscillation cannot be ignored, as shown in Figure 1.

There are two general kinematic structures for 2D gantry cranes. Firstly, the gantry is transferred in the X direction, at which time the trolley does not transfer; secondly, the trolley is transferred in the Y direction, but the gantry is fixed. The general kinematic structure of a 3D gantry crane refers to the movement of the gantry in the X direction while the trolley is transferring in the Y direction. The oscillation of the payload is affected by the coupling motions of the gantry and the trolley; therefore, the motion of the payload shows nonlinear changes. Moustafa et al. [9] used the Lagrange method to establish a dynamic model of a 3D crane. A linear feedback control system is proposed to realize the anti-swing motion control of the crane under various motion states. Fang et al. [10] designed several nonlinear feedback controllers for the underactuated gantry crane with constant rope length, which realized the “micro-swing” control of the system. Tuan et al. [11] designed a second-order synovial controller for gantry cranes with uncertain system parameters. Wu et al. [12] designed a 3D crane partial feedback linearization controller. The new swing suppression unit and the positioning error signal are constructed to ensure effective suppression of the rapid transferring and swing of the payload. Zhang [13] designed an enhanced coupled nonlinear tracking control method for underactuated 3D crane systems. However, changes in system parameters such as the mass of the payload and the length of the rope were not taken into consideration in these studies, which is not consistent with the actual situations. Furthermore, the external load is not included in the models, and the swing angle exceeded the threshold when the system was faced with external disturbance.

As discussed above, the lack of precise control of an underactuated gantry crane system is the result of the inaccurate dynamic model and improper control method. Therefore, a precise model of the 3D payload motion with the vision-based active control method is quite essential. In this paper, a precise 3D dynamic model including the coupling effect of gantry and trolley motions is proposed, and a method of machine vision recognition using QR code artificial beacons is put forward. The QR code is applied to obtain the coordinates of the start and end positions of the payload transfer, and a nonlinear control law based on energy coupling is designed for the precise control of the payload. As the main contribution of this paper, the proposed method successfully merges the advantages of machine vision positioning and energy-based control law, achieving an improved control performance. They are presented as follows:

(1)

The newly designed machine visual positioning system improves the positioning accuracy and automation efficiency of the gantry crane.

(2)

The control law has a concise structure without requiring exact knowledge of system parameters, which is convenient for practical implementation. It is robust against external disturbances, which is validated by experimental results.

The contents are as follows. The overall design of control method is given in the Section 2, and the dynamic model is described in Section 3. The machine vision positioning system is introduced in Section 4, and the trajectory planning is designed in Section 5. The control law is designed and analyzed in Section 6. In Section 7, there are simulation and experimental investigations on a gantry crane test rig to verify the performance of the designed control system, and finally the conclusions are drawn.

2. Control Method Design

This section describes the implementation steps of the underactuated gantry crane control method based on machine vision positioning.

Step 1: The 3D dynamic model of the gantry crane was built using the Euler–Lagrange equation.

Step 2: The multi-state nonlinear coupled differential equation of the gantry crane is obtained by solving the dynamic equation. Obtaining a trolley, the payload swing angle is based on a function of time when the gantry is operating at a constant acceleration.

Step 3: Image acquisition of gantry crane working surface, image preprocessing, detection of payload transfer end point, and calculation of target displacement of trolley and gantry.

Step 4: The payload swing angle obtained in step 2 is based on the time function, and the trolley and gantry frame obtained in step 3 are displaced. The trajectory planning strategy of the trolley, the trajectory planning strategy of the gantry, and the planning strategy of the payload trajectory are designed.

Step 5: A new energy function is constructed. On the basis of the multi-state nonlinear coupled differential equation of the gantry crane obtained in step 2, the non-linear trolley based on energy coupling and the gantry operating control law are designed. The stability of the closed-loop control system is proven by the Lyapunov techniques and Lasalle’s invariance theorem.

Step 6: The target displacement of the trolley and the gantry frame obtained in step 3 and the control law obtained in step 5 are input into the trolley and the gantry controller, respectively. The controller controls the drive to drive the trolley and the gantry’s motion motor mechanical system with voltage signals. The encoder feeds the trolley, the gantry displacement, and the payload swing angle information to the controller in real time with the pulse signal. Under the action of the closed-loop control system, the trolley and the gantry run smoothly, and the payload swing angle is effectively suppressed within the safe range.

Step 7: End. The trolley and the gantry are accurately moved to the target position, and the payload residual swing angle is eliminated.

The overall schematic diagram of the underactuated gantry crane control method based on machine vision positioning is shown in Figure 2.

3. Gantry Crane Dynamics Modeling

The schematic diagram of the gantry crane is shown in Figure 3. The gantry crane dynamics model is established using the Euler–Lagrange equation [14]:

$$M(q)\ddot{q}+V_m(q,\dot{q})\dot{q}+G(q)=U,$$

(1)

where $q$ represents the state vector of the gantry crane system; $M(q)$ represents the inertia matrix of the system; $V_m(q,\dot{q})$ represents the centripetal-Coriolis force matrix of the system; $G(q)$ represents the gravitational potential energy vector of the system; and $U$ represents the control vector of the system. The specific definition of the above parameters is as follows:

$$q={[\begin{array}{cccc}x(t)& y(t)& {\theta }_x(t)& {\theta }_y(t)\end{array}]}^T,$$

(2)

$$M(q)=\left[\begin{array}{cccc}{M}_1+m& 0& ml\cos {\theta }_x& 0\\ 0& M_2+m& 0& ml\cos {\theta }_y\\ ml\cos {\theta }_x& 0& m{l}^2& 0\\ 0& ml\cos {\theta }_y& 0& m{l}^2\end{array}\right],$$

(3)

$$V_m(q,\dot{q})=\left[\begin{array}{cccc}0& 0& -ml{\dot{\theta }}_x\sin {\theta }_x& 0\\ 0& 0& -ml{\dot{\theta }}_y\sin {\theta }_y& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 0\end{array}\right],$$

(4)

$$G(q)={\left[\begin{array}{cccc}0& 0& mgl\sin {\theta }_x& mgl\sin {\theta }_y\end{array}\right]}^T,$$

(5)

$$U={\left[\begin{array}{cccc}{F}_x& F_y& 0& 0\end{array}\right]}^T,$$

(6)

where $M_1$ represents the sum of the mass of the gantry and the trolley; $M_2$ indicates the mass of the trolley; $m$ represents the payload mass; $l$ indicates the length of the sling rope; $g$ represents the acceleration of gravity; ${\theta }_x$ and ${\theta }_y$ represent the payload swing angle generated by the X direction operation of the gantry and the Y direction operation of the trolley, respectively; and $F_x$ and $F_y$ stand for the resultant force applied to the gantry and trolley, respectively, which consists of the following two parts:

$$\{\begin{array}{c}{F}_x=F_{ax}-F_{rx}\\ F_y=F_{ay}-F_{ry}\end{array},$$

(7)

where $F_{ax}$ and $F_{ay}$ denote the actuating force of the gantry and the trolley, respectively; $F_{rx}$ donates the mechanical friction between the gantry and the rail; and $F_{ry}$ donates the friction between the trolley and the gantry. Inspired by the friction model presented in the work of [15] and after some experimental studies, we utilize the following model to describe the friction:

$$\{\begin{array}{c}{F}_{rx}(\dot{x})=f_{rx}\tanh (\dot{x}/{\xi }_x)-k_{rx}\left|\dot{x}\right|\dot{x}\\ F_{ry}(\dot{y})=f_{ry}\tanh (\dot{y}/{\xi }_y)-k_{ry}\left|\dot{y}\right|\dot{y}\end{array},$$

(8)

where $f_{rx},f_{ry},{\xi }_x,{\xi }_y,k_{rx}$, and $k_{ry}\in R$ are the friction-related parameters to be determined.

4. Machine Vision Positioning System

The QR code [16] is tiled on the working surface of the gantry crane at regular intervals (see Figure 4). We use the QR code as the target point to establish the positioning coordinate system. Firstly, the image of the collected gantry crane working surface is preprocessed, then the target is detected and positioned, and finally the machine vision positioning system is established.

4.1. Image Preprocessing

In order to improve the real-time performance of data analysis and processing, eliminate the influence of irrelevant information, and improve the detectability of effective information, it is necessary to preprocess the original image of the working plane of the gantry crane shot by the HD industrial camera. Preprocessing of the original image includes grayscale, binarization, and Canny edge detection.

The process of converting a color image into a grayscale image is called grayscale processing of the image. The RGB color image is a combination of three different color components: red (R), green (G), and blue (B). Each pixel in the grayscale image can be represented by a luminance value of 0 to 255. We use the weighted averaging algorithm of (9) to grayscale the color image:

$$f_{gay}(x,y)=0.299R(x,y)+0.587G(x,y)+0.114B(x,y).$$

(9)

Image binarization is used to set the gray value of the pixel on the image to 0 (black) or 255 (white). Converting 256 luminance level grayscale images to binarized images with appropriate threshold selection can still reflect the overall and local features of the image. The binarized image presents a black and white effect that highlights the contrast between the target image and the background. The image binarization formula is as follows:

$$f_{bin}(x,y)=\{\begin{array}{cc}255& f_{gay}(x,y)\ge \phi \\ 0& f_{gay}(x,y)<\phi \end{array},$$

(10)

where $\phi$ represents the threshold.

Canny edge detection is a multi-level edge detection algorithm. The Canny edge detection algorithm can be divided into the following five steps [17,18]: (1) use a Gaussian filter to smooth the image and filter out noise; (2) calculate the gradient strength and direction of each pixel in the image; (3) apply non-maximum suppression to eliminate spurious responses from edge detection; (4) apply dual threshold detection to determine true and potential edges; (5) finalize edge detection by suppressing isolated weak edges.

4.2. Target Detection and Positioning

The camera uses the principle of small hole imaging. The world coordinate system and the image pixel coordinate system can be converted by a linear camera model. The camera imaging principle is shown in Figure 5.

The camera coordinate conversion relationship is as follows [19,20]:

$$Z_c\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=\left[\begin{array}{cccc}\frac{f}{d_x}& 0& u_0& 0\\ 0& \frac{f}{d_y}& v_0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cc}R& T\\ 0^T& 1\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]=M_1{M}_2\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]=M\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right],$$

(11)

where $f$ is the focal length, $d_x$ and $d_y$ represent the unit pixel size of the image in the X direction and the Y direction, respectively; $u_0,v_0$ represent the coordinates of the principal point of the pixel coordinate system; $R$ is a rotation matrix; $T$ is a translation vector; $M_1$ is a parameter matrix within the camera; $M_2$ is the parameter matrix outside the camera; and $M$ is a camera parameter matrix.

In order to determine the location of the pixel plane where the QR code (see Figure 6) target is located, we use the OpenCV [21,22] computer vision library. The findContours function is used to extract the boundary contours of a, b, and c of the QR code in the preprocessed picture. The 4 corner points of d (d − 1, d − 2, d − 3, d − 4) can then be inferred from the 12 corner points of a (a − 1, a − 2, a − 3, and a − 4), b (b − 1, b − 2, b − 3, and b − 4), and c (c − 1, c − 2, c − 3, and c − 4). We can determine the smallest rectangle surrounding the QR code based on the four corner points (a − 1, b − 2, c − 3, and d − 4), and the image coordinates of center point $(u_n,v_n)$ are shown as follows:

$$\{\begin{array}{c}{u}_n=(u_{a-1}+u_{c-3})/2\\ v_n=(v_{a-1}+v_{c-3})/2\end{array}.$$

(12)

Image center pixel point coordinates $(u_1,v_1)$ and QR code target center point pixel coordinates $(u_n,v_n)$ based on the pixel coordinate system $o-uv$ can be obtained. Substituting the coordinate system conversion Formula (11), the coordinates of the target point of the world coordinate system can be obtained by the Formula (13). In the world coordinate system, the payload projection point $\left(X_{w1},Y_{w1}\right)$ is the transition starting point, and the QR code center point $\left(X_{wn},Y_{wn}\right)$ is the transition end point. The target displacement $x_{ub}$ and $y_{ub}$ of the trolley and gantry can be obtained by (14).

$$\left[\begin{array}{c}{X}_{wn}\\ Y_{wn}\\ Z_{wn}\end{array}\right]={\left[\begin{array}{ccc}{M}_{11}& M_{12}& M_{13}\\ M_{21}& M_{22}& M_{23}\\ M_{31}& M_{32}& M_{33}\end{array}\right]}^{-1}\times \left\{Z_c\left[\begin{array}{c}{u}_n\\ v_n\\ 1\end{array}\right]-\left[\begin{array}{c}{M}_{14}\\ M_{24}\\ M_{34}\end{array}\right]\right\}$$

(13)

$$\{\begin{array}{c}{x}_{ub}=X_{wn}-X_{w1}\\ y_{ub}=Y_{wn}-Y_{w1}\end{array}$$

(14)

5. Trajectory Planning

In order to achieve accurate and smooth control of the gantry crane, the trajectory of the gantry and trolley should meet a smooth S-shaped curve [23]. First, we introduce a three-stage acceleration trajectory to make the gantry run separately in the X direction. Its expression is as follows:

$$\ddot{x}(t)=\{\begin{array}{cc}{a}_{x\max }& 0\le t\le t_{x1}\\ -a_{x\max }& t_{x1}+t_{x2}\le t\le 2t_{x1}+t_{x2}\\ 0& else\end{array},$$

(15)

$$\dot{x}(t)=\{\begin{array}{cc}{a}_{x\max }t& 0\le t\le t_{x1}\\ a_{x\max }{t}_{x1}& t_{x1}\le t\le t_{x1}+t_{x2}\\ -a_{x\max }[t-(2t_{x1}+t_{x2})]& t_{x1}+t_{x2}\le t\le 2t_{x1}+t_{x2}\\ 0& else\end{array},$$

(16)

$$x_{ub}={\displaystyle {\int }_0^{2t_{x1}+t_{x2}}\dot{x}(t)dt=a_{x\max }{t}_{x1}(t_{x1}+t_{x2})},$$

(17)

where $\ddot{x}(t)$, $\dot{x}(t)$, and $x_{ub}$ represent the acceleration, velocity, and target displacement of the gantry in the X direction, respectively; $a_{x\max }$ represents the maximum acceleration; and $t_{x1}$ and $t_{x2}$ indicate the acceleration/deceleration time and the uniform speed time, respectively. $a_{x\max }$, $t_{x1}$, and $t_{x2}$ can be obtained from the following equations [24]:

$$a_{x\max }=\min \left\{\begin{array}{cccc}\frac{v_{xub}}{T}& \frac{x_{ub}}{T^2}& a_{xub}& \frac{g{\theta }_{xub}}{2}\end{array}\right\},$$

(18)

$$t_{x1}=T=\frac{2\pi }{w_n},$$

(19)

$$w_n=\sqrt{\frac{g}{l}},$$

(20)

$$t_{x2}=\frac{x_{ub}}{a_{x\max }T}-T,$$

(21)

where $a_{xub}$, $v_{xub}$, and ${\theta }_{xub}$ represent the preset acceleration, speed, and swing angle upper limit of the X direction running system of the gantry, respectively; $T$ represents an acceleration/deceleration cycle; and $w_n$ represents the natural frequency of the system and is related to the length $l$ of the sling.

The differential equation for the dynamic coupling relationship between the acceleration of the gantry and the swing angle of the payload can be obtained by the expansion of (1):

$$l{\ddot{\theta }}_x+\ddot{x}\cos {\theta }_x+g\sin {\theta }_x=0.$$

(22)

The actual gantry crane system meets $\sin {\theta }_x\approx {\theta }_x$, $\cos {\theta }_x\approx 1$ [25,26,27,28,29]. Therefore, Equation (22) can be expressed as a partial differential equation:

$${\ddot{\theta }}_x+w_n^2{\theta }_x=-\frac{w_n^2}{g}\ddot{x}.$$

(23)

When the gantry is running at a constant acceleration $\ddot{x}=a_x$, Equation (23) can be solved as follows, and the payload swing angle is obtained as a function of time $t$:

$${\theta }_x(t)={\theta }_x(0)\cos w_{n}t+\frac{{\dot{\theta }}_x(0)}{w_n}\sin w_{n}t-\frac{a_x}{g}(1-\cos w_{n}t),$$

(24)

where ${\theta }_x(0)$ and ${\dot{\theta }}_x(0)$ represent the initial angle and angular velocity of the gantry in the X direction, and the actual conditions satisfy ${\theta }_x(0)=0,{\dot{\theta }}_x(0)=0$.

The trajectory planning of the gantry crane trolley in the Y direction can refer to the operation law of the gantry in the X direction, and will not be described here.

In order to improve the operating efficiency, the gantry and trolley often run simultaneously in practical applications. In the compound direction, acceleration $\ddot{p}(t)$, speed $\dot{p}(t)$, displacement $p(t)$, swing angle ${\theta }_1(t)$, swing angle ${\theta }_2(t)$, plus/deceleration time $t_1$, and uniform speed time $t_2$ are defined as follows:

$$\ddot{p}(t)=\sqrt{\ddot{x}{(t)}^2+\ddot{y}{(t)}^2},$$

(25)

$$\dot{p}(t)=\sqrt{\dot{x}{(t)}^2+\dot{y}{(t)}^2},$$

(26)

$$p(t)=\sqrt{x{(t)}^2+y{(t)}^2},$$

(27)

$${\theta }_1(t)=-\sqrt{\frac{a_x^2+a_y^2}{g}}(1-\cos w_{n}t),$$

(28)

$${\theta }_2(t)=\arctan \frac{a_x}{a_y},$$

(29)

$$t_1=t_{x1}=t_{y1},$$

(30)

$$t_2=t_{x2}=t_{y2}.$$

(31)

6. Control Law Design and Stability Analysis

6.1. Control Law Design

The energy function shown below is constructed from the mechanical energy of the gantry crane:

$$E(t)=\frac{1}{2}{\dot{q}}^{T}H(q)\dot{q}+[ml+k_{ax}(M_1+m)g(1-\cos {\theta }_x)]+[ml+k_{ay}(M_2+m)g(1-\cos {\theta }_y)],$$

(32)

where

$$H(q)=\left[\begin{array}{cccc}{M}_1+m& 0& ml\cos {\theta }_x& 0\\ 0& M_2+m& 0& ml\cos {\theta }_y\\ ml\cos {\theta }_x& 0& m{l}^2+k_{ax}(M_1+m{\sin }^2{\theta }_x)l& 0\\ 0& ml\cos {\theta }_y& 0& m{l}^2+k_{ay}(M_2+m{\sin }^2{\theta }_y)l\end{array}\right].$$

(33)

We hope to enhance the dynamic coupling relationship between the displacement of the gantry $x(t)$; the displacement of the trolley $y(t)$; and the swing angle of the payload ${\theta }_x(t)$, ${\theta }_y(t)$. Therefore, a positive definite scalar function as shown below is constructed based on the energy function form:

$$V(t)=E(t)+\frac{k_{px}}{2}{\sigma }_x^2+\frac{k_{py}}{2}{\sigma }_y^2,$$

(34)

where $k_{px},k_{py}\in R^{+}$ are positive control gains; ${\sigma }_x$ is the positioning errors of the payload in the X direction; and ${\sigma }_y$ is the positioning errors of the payload in the Y direction:

$$\{\begin{array}{c}{\sigma }_x=e_x-k_{ax}\sin {\theta }_x\\ {\sigma }_y=e_y-k_{ay}\sin {\theta }_y\end{array},$$

(35)

where $e_x$ represents the errors between the actual displacement of the gantry and the target displacement in the X direction, and $e_y$ represents the errors between the actual displacement of the trolley and the target displacement in the Y direction. They are defined as follows:

$$\{\begin{array}{c}{e}_x=x(t)-x_{ub}\\ e_y=y(t)-y_{ub}\end{array}.$$

(36)

For $V(t)$ to refer to time, substituting (1), we get the following:

$$\dot{V}(t)=(F_x+k_{px}{\sigma }_x){\dot{\sigma }}_x+(F_y+k_{py}{\sigma }_y){\dot{\sigma }}_y.$$

(37)

On the basis of the form shown in $\dot{V}(t)$, design the controller is shown below:

$$\{\begin{array}{c}{F}_x=-k_{px}{\sigma }_x-k_{dx}{\dot{\sigma }}_x=-k_{px}(e_x-k_{ax}\sin {\theta }_x)-k_{dx}({\dot{e}}_x-k_{ax}{\dot{\theta }}_x\cos {\theta }_x)\\ F_y=-k_{py}{\sigma }_y-k_{dy}{\dot{\sigma }}_y=-k_{py}(e_y-k_{ay}\sin {\theta }_y)-k_{dy}({\dot{e}}_y-k_{ay}{\dot{\theta }}_y\cos {\theta }_y)\end{array},$$

(38)

where $k_{dx},k_{dy}\in R^{+}$ are positive control gains.

6.2. Closed-Loop System Stability Analysis

Theorem 1.

Under the control law (38), the gantry and trolley running displacements converge to the target displacement, and the payload swing angle is suppressed and eliminated. Satisfy the following theorem:

$$\underset{t\to \infty }{\lim }[x(t),\dot{x}(t),y(t),\dot{y}(t),{\theta }_x(t),{\dot{\theta }}_x(t),{\theta }_y(t),{\dot{\theta }}_y(t)]=[x_{ub},0,y_{ub},0,0,0,0,0].$$

(39)

Proof. 

The positive definite scalar function of (30) is chosen as the Lyapunov candidate function. We substitute the control law (38) into (37) and get the following formula:

$$\dot{V}(t)=-k_{dx}{\dot{\sigma }}_x^2-k_{dy}{\dot{\sigma }}_y^2\le 0,$$

(40)

which shows that the gantry crane closed-loop control system is Lyapunov stable at the origin. In consequence, it is easily concluded that

$$V(t)\in {\mathcal{L}}_{\infty },$$

(41)

and the closed-loop system state is bounded. Therefore,

$$e_x(t),e_y(t),{\dot{e}}_x(t),{\dot{e}}_y(t),\dot{x}(t),\dot{y}(t),{\dot{\theta }}_x(t),{\dot{\theta }}_y(t),F_x,F_y\in {\mathcal{L}}_{\infty }.$$

(42)

To accomplish the proof, let $\mathsf{\Gamma }$ be the largest invariant set in $\mathsf{\Omega }$, where

$$\mathsf{\Omega }=\left\{\left(x,y,\dot{x},\dot{y},{\theta }_x,{\theta }_y,{\dot{\theta }}_x,{\dot{\theta }}_y\right)|\dot{V}(t)=0\right\}.$$

(43)

Then, based on (40), the following is clear in $\mathsf{\Gamma }$:

$$\{\begin{array}{c}{\dot{\sigma }}_x={\dot{e}}_x-k_{ax}{\dot{\theta }}_x\cos {\theta }_x=0\\ {\dot{\sigma }}_y={\dot{e}}_y-k_{ay}{\dot{\theta }}_y\cos {\theta }_y=0\end{array}.$$

(44)

Therefore, the following conclusions can be drawn:

$$\{\begin{array}{c}{\sigma }_x=e_x-k_{ax}\sin {\theta }_x={\alpha }_x\\ {\sigma }_y=e_y-k_{ay}\sin {\theta }_y={\alpha }_y\end{array},$$

(45)

$$\{\begin{array}{c}{\ddot{\sigma }}_x={\ddot{e}}_x+k_{ax}{\dot{\theta }}_x^2\sin {\theta }_x-k_{ax}{\ddot{\theta }}_x\cos {\theta }_x=0\\ {\ddot{\sigma }}_y={\ddot{e}}_y+k_{ay}{\dot{\theta }}_y^2\sin {\theta }_y-k_{ay}{\ddot{\theta }}_y\cos {\theta }_y=0\end{array},$$

(46)

where ${\alpha }_x,{\alpha }_y\in R$ are pending constants. ${\alpha }_x={\alpha }_y=0$ in the maximum variable set $\mathsf{\Gamma }$ will then be demonstrated. Combining (1), (38), (44), and (46), we get the following:

$$\{\begin{array}{c}\ddot{x}=-\frac{k_{ax}{k}_{px}}{ml+k_{ax}(M_1+m)}{\alpha }_x\\ \ddot{y}=-\frac{k_{ay}{k}_{py}}{ml+k_{ay}(M_2+m)}{\alpha }_y\end{array}.$$

(47)

Therefore, $\dot{x}(t),\dot{y}(t)$ will vary at a constant speed as shown in Equation (46), assuming ${\alpha }_x\ne 0,{\alpha }_y\ne 0$, which implies the following:

$$\{\begin{array}{c}\underset{t\to \infty }{\lim }\dot{x}(t)=\{\begin{array}{c}+\infty ,{\alpha }_x<0\\ -\infty ,{\alpha }_x>0\end{array}\\ \underset{t\to \infty }{\lim }\dot{y}(t)=\{\begin{array}{c}+\infty ,{\alpha }_y<0\\ -\infty ,{\alpha }_y>0\end{array}\end{array}.$$

(48)

Obviously, this contradicts Equation (42), so the assumption is incorrect. Therefore,

$${\alpha }_x={\alpha }_y=0.$$

(49)

Then, from Equations (37), (43), (44), (46), and (48), we get the following:

$$\{\begin{array}{c}{\sigma }_x=e_x-k_{ax}\sin {\theta }_x=0\\ {\sigma }_y=e_y-k_{ay}\sin {\theta }_y=0\end{array},$$

(50)

$$F_x=F_y=0,$$

(51)

$$\{\begin{array}{c}{\ddot{e}}_x=\ddot{x}=0\\ {\ddot{e}}_y=\ddot{y}=0\end{array},$$

(52)

$$\{\begin{array}{c}{\dot{e}}_x=\dot{x}={\beta }_x\\ {\dot{e}}_y=\dot{y}={\beta }_y\end{array},$$

(53)

where ${\beta }_x,{\beta }_y\in R$ are constants to be determined. In the maximum invariant set $\mathsf{\Gamma }$, the same analysis as above shows the following:

$$\{\begin{array}{c}{\dot{e}}_x={\beta }_x=0\\ {\dot{e}}_y={\beta }_y=0\end{array},$$

(54)

$$\{\begin{array}{c}{e}_x={\lambda }_x\\ e_y={\lambda }_y\end{array},$$

(55)

where ${\lambda }_x,{\lambda }_y\in R$ are the constants to be determined. It will then be proven that ${\lambda }_x={\lambda }_y=0$ in the largest constant set $\mathsf{\Gamma }$.

First, assume that ${\lambda }_x\ne 0,{\lambda }_y\ne 0$, by combining (50) and (55), we can obtain the following equation set:

$$\{\begin{array}{c}\sin {\theta }_x=\frac{e_x}{k_{ax}}=\frac{{\lambda }_x}{k_{ax}}\ne 0\\ \sin {\theta }_y=\frac{e_y}{k_{ay}}=\frac{{\lambda }_y}{k_{ay}}\ne 0\end{array}.$$

(56)

The condition in (22) is imported in (52), and we get the following:

$$\{\begin{array}{c}{\ddot{\theta }}_x=-\frac{g}{l}\sin {\theta }_x\ne 0\\ {\ddot{\theta }}_y=-\frac{g}{l}\sin {\theta }_y\ne 0\end{array}.$$

(57)

Combining Equations (46), (52), (56), and (57), the following equation can be obtained:

$$\{\begin{array}{c}{\dot{\theta }}_x^2+\frac{g}{l}\cos {\theta }_x=0\\ {\dot{\theta }}_y^2+\frac{g}{l}\cos {\theta }_y=0\end{array}.$$

(58)

The condition in (44) is imported in (54), and we get the following:

$$\{\begin{array}{c}{\dot{\theta }}_x\cos {\theta }_x=0\\ {\dot{\theta }}_y\cos {\theta }_y=0\end{array}$$

(59)

Solving the system of Equations (58) and (59) gives the following:

$$\{\begin{array}{c}{\dot{\theta }}_x=0\\ {\dot{\theta }}_y=0\\ \cos {\theta }_x=0\\ \cos {\theta }_y=0\end{array}$$

(60)

It can be further inferred that in the maximum invariant set $\mathsf{\Gamma }$,

$${\ddot{\theta }}_x={\ddot{\theta }}_y=0.$$

(61)

Obviously, Equation (61) contradicts Equation (57), so it is assumed that ${\lambda }_x\ne 0,{\lambda }_y\ne 0$ does not hold. Therefore, ${\lambda }_x=0,{\lambda }_y=0$ is satisfied in the maximum invariant set $\mathsf{\Gamma }$, thereby inferring the following:

$$e_x(t)=e_y(t)=0,$$

(62)

$$\{\begin{array}{c}x(t)=x_{ub}\\ y(t)=y_{ub}\end{array}.$$

(63)

Therefore, combining (50) and (58) shows

$${\theta }_x={\theta }_y=0,$$

(64)

$${\dot{\theta }}_x={\dot{\theta }}_y=0.$$

(65)

On the basis of the above analysis, it can be known that the maximum invariant set $\mathsf{\Gamma }$ contains only the equilibrium point $\underset{t\to \infty }{\lim }[x(t),\dot{x}(t),y(t),\dot{y}(t),{\theta }_x(t),{\theta }_y(t),{\dot{\theta }}_x(t),{\dot{\theta }}_y(t)]=[x_{ub},0,y_{ub},0,0,0,0,0]$.

According to the LaSalle’s invariance theorem [30], the closed-loop system converges to equilibrium point with time under the control law (38). Therefore, the Theorem 1 is proved. □

7. Simulation Experiment and Result Analysis

7.1. Machine Vision Positioning Experimental Results and Analysis

The machine vision positioning experiment is carried out on the gantry crane test bench (See Figure 7). The HD industrial camera in our vision system is a Mercury series camera made by Daheng IMAVISION (see Figure 8), of which the frame rate is 120 fps, the pixel size is 5.6 um × 5.6 um, the focal length is 3.5 mm, and the exposure time is also adjustable. The camera is fixedly mounted under the center of the trolley. The camera plane is parallel to the ground. As the gantry is highly fixed, the camera moves with the trolley at a fixed height plane.

We analyzed the image of the gantry crane working surface. Image preprocessing not only reduces the amount of data computation, but also helps extract useful information about the target and background. Then, we use the target recognition and localization algorithm to determine the target displacement of the trolley and the gantry.

The processing of the image of the working plane of the gantry crane is shown in Figure 9. The final recognition result based on machine vision positioning is shown in Figure 10. It can be seen that $x_{ub}=0.548 \mathrm{m}$ and $y_{ub}=0.430 \mathrm{m}$.

7.2. Friction Simulation and Analysis

For this article, we employ the compensation method for the nonlinear friction. To this end, the friction model of (8) is utilized as a feedforward component to compensate for the track friction. After plenty of simulation test, the friction parameters in (8) are determined as $f_{rx}=1.2$, ${\xi }_x=0.2$, $k_{rx}=-0.4$, $f_{ry}=0.7$, ${\xi }_y=0.2$, and $k_{ry}=-0.5$.

On the basis of the simulation results shown in Figure 11, it is concluded that friction changes with the transfer of the gantry and the trolley, and eventually tends to 0 when the gantry and trolley reach the target position. Friction is biggest when the gantry and trolley start to move ($F_{rx\max }=1.32\mathrm{N}$, $F_{ry\max }=0.88\mathrm{N}$).

7.3. Trajectory Planning Simulation and Analysis

The numerical simulation method is used to analyze the effectiveness of the trajectory planning of the gantry crane transportation path from the perspective of kinematics. The simulation can be divided into two types: (1) the gantry is running in the X direction when the trolley stops, and trolley is running in the Y direction when the gantry stops; (2) the gantry and the trolley run simultaneously. The simulation uses the related parameters shown in

Table 1. Related parameters.

Name	Symbol	Numerical Value	Unit
Rope length	$l$	1	$\mathrm{m}$
Gravity acceleration	$g$	9.8	${\mathrm{m}/\mathrm{s}}^2$
Gantry preset acceleration	$a_{xub}$	0.4	$\mathrm{m}/{\mathrm{s}}^2$
Gantry preset speed	$v_{xub}$	0.8	$\mathrm{m}/\mathrm{s}$
Gantry target displacement	$x_{ub}$	0.548	$\mathrm{m}$
Gantry frame swing angle upper limit	${\theta }_{xub}$	5	deg
Trolley preset acceleration	$a_{yub}$	0.5	$\mathrm{m}/{\mathrm{s}}^2$
Trolley preset speed	$v_{yub}$	1	$\mathrm{m}/\mathrm{s}$
Trolley target displacement	$y_{ub}$	0.430	$\mathrm{m}$
Trolley swing angle upper limit	${\theta }_{yub}$	5	deg
Compound acceleration	${\ddot{p}}_{ub}$	0.5	$\mathrm{m}/{\mathrm{s}}^2$
Compound speed	${\dot{p}}_{ub}$	1	$\mathrm{m}/\mathrm{s}$
Compound displacement	$p_{ub}$	0.697	$\mathrm{m}$
Compound angle 1	${\theta }_{1ub}$	5	deg
Compound angle 2	${\theta }_{2ub}$	1	rad

, and the results are shown in Figure 12.

It can be seen from the simulation results that the trajectory planning strategy can quickly and accurately transfer the gantry and the trolley to the target position. The acceleration and speed are always within a limited range. The payload swing angle is always in the range of less than $5^{°}$. After the trolley and gantry reach the target displacement, the payload does not have any residual swing.

7.4. Control Law Experimental Results and Analysis

To validate the actual performance of the proposed approach, three groups of experiments are implemented on the gantry crane test bench (see Figure 7). Specifically, we first compare the proposed scheme with the traditional energy based method. Then, the robustness of the proposed approach is tested by modifying the system parameters. At last, the proposed approach is further verified by taking into account the external perturbation. The system physical parameters are $M_1=15.25\mathrm{kg}$ and $M_2=6.25\mathrm{kg}$, and the identifying locations are set to be $x_{ub}=0.548\mathrm{m}$ and $y_{ub}=0.430\mathrm{m}$.

To be self-contained, we provide the expressions of energy-based controllers designed in the work of [10]. The gantry kinetic energy coupling control law (GKE) and the trolley kinetic energy coupling control law (TKE) are as follows:

$$\{\begin{array}{c}{F}_{gke}=\frac{-k_{px}e-k_{dx}\dot{x}-k_{vx}[m\sin \theta (l{\dot{\theta }}_x^2+g\cos {\theta }_x)]}{k_{ex}+k_{vx}}\\ F_{tke}=\frac{-k_{py}e-k_{dy}\dot{y}-k_{vy}[m\sin \theta (l{\dot{\theta }}_y^2+g\cos {\theta }_y)]}{k_{ey}+k_{vy}}\end{array}.$$

(66)

Experiment 1.

Control performances validation with exact parameter information. In this experiment, the mass of the payload and the length of the rope are determined as

$$\{\begin{array}{c}m=1\mathrm{kg}\\ l=0.6\mathrm{m}\end{array}.$$

(67)

The control gains for the two control laws are fully tuned until the respective best performance is achieved, which are explicitly given in

Table 2. Control gain for experiment 1. GKE, gantry kinetic energy; TKE, trolley kinetic energy.

Control Law	${\mathit{k}}_{\mathit{p}\mathit{x}}$	${\mathit{k}}_{\mathit{d}\mathit{x}}$	${\mathit{k}}_{\mathit{a}\mathit{x}}$	${\mathit{k}}_{\mathit{e}\mathit{x}}$	${\mathit{k}}_{\mathit{v}\mathit{x}}$	${\mathit{k}}_{\mathit{p}\mathit{y}}$	${\mathit{k}}_{\mathit{d}\mathit{y}}$	${\mathit{k}}_{\mathit{a}\mathit{y}}$	${\mathit{k}}_{\mathit{e}\mathit{y}}$	${\mathit{k}}_{\mathit{v}\mathit{y}}$
GKE and TKE (66)	18	28	1	NA	NA	16	30	1.95	NA	NA
proposed controller (38)	36	80	NA	1	0.5	35	65	NA	1	1

.

First, the gantry is transferred separately for 30 s in the X direction. Then, the trolley is transferred separately for 30 s in the Y direction. Finally, the trolley and the gantry are transferred for 30 s at the same time. The experimental results are shown in Figure 13 and Figure 14, and the quantified results are detailed in

Table 3. Control performance comparison for Experiment 1.

Control Law	${\mathit{\theta }}_{\mathit{x}\mathbf{max}}$	${\mathit{\theta }}_{\mathit{x}\mathit{r}\mathit{e}\mathit{s}}$	${\mathit{t}}_{\mathit{x}\mathit{s}}$	${\mathit{F}}_{\mathit{x}\mathbf{max}}$	${\mathit{\theta }}_{\mathit{y}\mathbf{max}}$	${\mathit{\theta }}_{\mathit{y}\mathit{r}\mathit{e}\mathit{s}}$	${\mathit{t}}_{\mathit{y}\mathit{s}}$	${\mathit{F}}_{\mathit{y}\mathbf{max}}$	${\mathit{\theta }}_{\mathit{p}\mathbf{max}}$	${\mathit{\theta }}_{\mathit{p}\mathit{r}\mathit{e}\mathit{s}}$	${\mathit{t}}_{\mathit{p}\mathit{s}}$
GKE and TKE (66)	3.6	1.9	16.8	11.9	3.4	0.9	11.5	10.2	4.8	2.1	17.3
proposed controller (38)	2.4	0.3	6.5	10.3	1.8	0.4	7.3	4.9	2.9	0.5	7.4

, which includes the following four performance indices: (1) ${\theta }_{\max }$ that denotes the maximum swing amplitude during the transferring process; (2) payload residual swing ${\theta }_{res}$, which refers to the maximum swing amplitude after the gantry and the trolley stop; (3) settling time $t_s$, which is defined as the time when the swing angle enters the range of $\left|\theta (t)\right|\le {0.5}^{\circ }\forall t\ge t_s$; (4) maximum actuating force $F_{\max }$.

On the basis of the experiment results shown in Figure 13 and Figure 14 and Table 3, it is concluded that both control laws can transfer the gantry and trolley to the target position within 10 s. However, the transient response of the proposed controller (38) is superior over that of the other control law. Payload swing is much better suppressed and eliminated by the proposed control law. There is almost no residual swing when the granty and trolley reach the desired location.

Experiment 2.

Robustness verification experiment. Here, to test the robustness of the proposed control law (38) against parameter variations, we change the payload mass and rope length to distinguish them from corresponding nominal values. The actual parameters are decided as

$$\{\begin{array}{c}m=0.5\mathrm{kg}\\ l=0.8\mathrm{m}\end{array}.$$

(68)

The control gains for the control law (38) and (66) are kept the same as those in Experiment 1.

Figure 15 and Figure 16 show the results of Experiment 2, and the explicit quantified data are provided in

Table 4. Control performance comparison for Experiment 2.

Control Law	${\mathit{\theta }}_{\mathit{x}\mathbf{max}}$	${\mathit{\theta }}_{\mathit{x}\mathit{r}\mathit{e}\mathit{s}}$	${\mathit{t}}_{\mathit{x}\mathit{s}}$	${\mathit{F}}_{\mathit{x}\mathbf{max}}$	${\mathit{\theta }}_{\mathit{y}\mathbf{max}}$	${\mathit{\theta }}_{\mathit{y}\mathit{r}\mathit{e}\mathit{s}}$	${\mathit{t}}_{\mathit{y}\mathit{s}}$	${\mathit{F}}_{\mathit{y}\mathbf{max}}$	${\mathit{\theta }}_{\mathit{p}\mathbf{max}}$	${\mathit{\theta }}_{\mathit{p}\mathit{r}\mathit{e}\mathit{s}}$	${\mathit{t}}_{\mathit{p}\mathit{s}}$
GKE and TKE (66)	4.6	3.2	>30	12.0	4.8	2.3	27.2	9.1	>5	3.8	>30
proposed controller (38)	2.5	0.3	6.	10.3	1.7	0.3	7.5	5.3	3.0	1.1	7.6

. It is clear by comparing Figure 14 and Figure 16 that the performance of the proposed control law (38) is hardly influenced by the parameter changes in the sense that the transportation efficiency and payload suppression effect remain almost the same as those of Experiment 1. By contrast, the performance of the GKE and TKE control law (66) is largely degraded because of the fact that the model-dependent structures make them sensitive to parameter uncertainties, while in real-world applications, it is impossible to always know the exact values including rope lengths and payload masses; in this sense, the proposed method is more practical.

Experiment 3.

To further verify its robustness against external perturbation, we examine the performance of the proposed controller (38) by using a wooden stick to strike the payload. The parameters including the control gains are chosen the same as those in Experiment 1.

First, the gantry is transferred separately for 30 s in the X direction. We use a wooden stick to strike the payload in the X direction at 20 s. Then, the trolley is transferred separately for 30 s in the Y direction. We use a wooden stick to strike the payload in the Y direction at 20 s. Finally, the trolley and the gantry are transferred for 30 s at the same time. We use a wooden stick to strike the payload in the compound direction at 20 s.

The experimental results are shown in Figure 17 and Figure 18 and

Table 5. Control performance comparison for Experiment 3.

Control Law	${\mathit{\theta }}_{\mathit{x}\mathbf{max}}$	${\mathit{\theta }}_{\mathit{x}\mathit{r}\mathit{e}\mathit{s}}$	${\mathit{t}}_{\mathit{x}\mathit{s}}$	${\mathit{F}}_{\mathit{x}\mathbf{max}}$	${\mathit{\theta }}_{\mathit{y}\mathbf{max}}$	${\mathit{\theta }}_{\mathit{y}\mathit{r}\mathit{e}\mathit{s}}$	${\mathit{t}}_{\mathit{y}\mathit{s}}$	${\mathit{F}}_{\mathit{y}\mathbf{max}}$	${\mathit{\theta }}_{\mathit{p}\mathbf{max}}$	${\mathit{\theta }}_{\mathit{p}\mathit{r}\mathit{e}\mathit{s}}$	${\mathit{t}}_{\mathit{p}\mathit{s}}$
GKE and TKE (66)	3.6	1.9	17.2	12.0	3.4	0.9	11.5	9.8	4.8	2.1	17.3
proposed controller (38)	2.4	0.3	6.5	11.2	1.8	0.4	7.3	11.4	3.3	0.5	8.4

. We can conclude that both control laws can act against external perturbation, but the transient response of the proposed controller (38) is superior over that of the other control law. Payload swing is much better suppressed and eliminated by the proposed control law when the payload meets the external perturbation.

8. Conclusions

Aiming at the symmetry underactuated gantry crane control system, this paper proposes a nonlinear control method of symmetry underactuated gantry crane based on machine vision positioning. The method detects the transfer target by the visual sensor and plans the trajectory, which improves the automation efficiency of the gantry crane. The designed control law not only makes the payload transfer to the target position accurately, but also effectively suppresses the payload swing and enhances the stability of the system in response to external disturbances. The proposed control law enhances the coupling behavior between the payload and the various state quantities of the gantry crane and leads to an improved control performance. Theoretical analysis and experimental results show that the proposed control method has good accuracy, stability, and rapidity for the symmetry underactuated gantry crane control system. Future work will focus on extending the proposed method to the overall-process control of gantry cranes with payload hoisting and lowering.