Improving the Quality of Industrial Robot Control Using an Iterative Learning Method with Online Optimal Learning and Intelligent Online Learning Function Parameters

Abstract

It is inevitable that the characteristics of a robot system change inaccurately or cannot be accurately determined during movement and are affected by external disturbances. There are many adaptive control methods, such as the exact linearization method, sliding control, or neural control, to improve the quality of trajectory tracking for a robot’s motion system. However, those methods require a great deal of computation to solve the constrained nonlinear optimization problem. This article first presents some techniques for determining the online learning function parameters of an intelligent controller, including two circuits: the inner circuit is an uncertain function component estimator to compensate for the robot’s input, and the outer circuit is an iterative learning controller and does not use a mathematical model of the robot with optimal online learning function parameters. The optimal condition is based on the model in the time domain to determine the learning function parameters that change adaptively according to the sum of squared tracking errors of each loop. As for the intelligent online learning function parameters, they closely follow the general model to stabilize the robot system, based on the principle of intelligent estimation of the uncertainty component and total noise. This method is built on Taylor series analysis for the state vector and does not use a mathematical model of the system at all. It allows feedback linearization, as well as intelligent stabilization of the system. This article’s content uses a 2-DOF flat robot implemented on MatlabR2022b software to verify the theory. These findings indicate that superior tracking performance is achievable.

1. Introduction

Robots have been widely used in many industrial fields, so countless robot control methods have been researched and applied, helping robots achieve accuracy. It cannot be prevented that, during movement, the robot system’s properties may change imprecisely or cannot be accurately identified and will be influenced by outside disturbances. The quality of trajectory tracking for robot motion systems may be enhanced using a variety of adaptive control techniques, including sliding control and neural control. However, those approaches involve a great deal of computing to solve the constrained nonlinear optimization issue. This article offers some methods for figuring out the parameters of an intelligent controller’s online learning function, using two circuits: the outer circuit is an interactive learning controller, which does not use the robot’s mathematical mode, and the inner circuit is a state feedback linearized controller. The parameters of the intelligent controller’s online learning function closely follow the joint. However, these methods are all built based on the mathematical model describing the robot, and they are classified depending on the level of accuracy of that mathematical model. The Euler—Lagrange model is a form of mathematical model often used for developing and synthesizing controllers, and it is impacted by external input noise d(t) as follows:

$${\underset{\_}{u}}_{dk}+{\underset{\_}{\tau }}_d=H(\underset{\_}{q},\underset{¯}{\theta })\ddot{\underset{\_}{q}}+C(\underset{\_}{q},\dot{\underset{\_}{q}},\underset{¯}{\theta })\dot{\underset{\_}{q}}+g(\underset{\_}{q},\underset{¯}{\theta })+F(\underset{\_}{\dot{q}},\underset{¯}{\theta })$$

(1)

In which $\underset{\_}{q}$ is the vector of joint variables; $\underset{\_}{\theta }$ is the vector of uncertain parameters; $H(\underset{\_}{q},\underset{\_}{\theta })$ is the inertia matrix that is always positive definite symmetry; $C(\underset{\_}{q},\dot{\underset{\_}{q}},\underset{\_}{\theta })$ is the vector of Coriolis and centrifugal components; $F(\underset{\_}{\dot{q}},\underset{\_}{\theta })$ describes the effect of friction; $g(\underset{\_}{q},\underset{\_}{\theta })$ is the vector gravity; ${\underset{\_}{t}}_d(t)$ is the noise vector in the actuator; and you is the control signal.

With the matching variables equal to the number of input signals, such a robot system has enough actuators. The robot control task is to build a feedback controller so that the output is the joint variables $\underset{\_}{q}$ that follow the desired trajectory $\underset{\_}{R}(t)$ and the tracking quality does not depend on uncertain parameters and components and external noise ${\underset{\_}{\tau }}_{\mathrm{d}}\left(\mathrm{t}\right)$. Over time, due to many objective factors, such as equipment wear and tear from environmental impacts, the initially set design quality is no longer guaranteed. In this case, it is common to rebuild the controller. Currently, one unchangeable aspect of designing an industrial robot motion control system via the traditional approach is that the control object must always be understood clearly. This means that the control object needs to be controlled and expressed in modeling in the form of a mathematical model that precisely describes the object; in this case, this model can take the form of a transfer function or a state model represented by a system of differential equations/highest classification. At the same time, when building and designing a motion control system, it is necessary to anticipate objective factors that will not impact the system as expected, such as external interference, etc., leading to damage to the system. The control is no longer as effective as before, and one must redefine the mathematical model of the control object, re-evaluate the rules of unwanted effects, bring objectivity into the system, rebuild the controller, or, at least, redefine the parameters for the controller. There have been many control methods to solve the above problem, for example, the exact linearization method [1]. If the external noise component $\underset{\_}{q}$ passes but the indeterminate constant parameter still exists, we have the inverse control method of the model [2]. The reverse learning or Li-Slotine method applies when the parameter q is arbitrary. The song must be constant and must have ${\underset{\_}{t}}_d(t)=0$ [3]. In cases where both external noise components and uncertain constant parameters exist, there is a sliding control method [4,5]. On the other hand, chattering occurs when the system slides on the sliding surface too often, which is a drawback of the sliding control approach. To improve this vibration phenomenon, there is also a high-order sliding control method [6], but it still requires an estimated value of the maximum standard deviation of the model caused and cannot eliminate it. In addition, to reduce the chattering phenomenon, people use neural networks [7] to estimate the uncertainty components based on collecting information from robots (through $\underset{\_}{q}$ and its derivative $\dot{\underset{\_}{q}}$) by the output estimation component and then uncertainty compensation through control signals. The intelligent control method mainly applied to robots mentioned in this article is the iterative learning method [8,9]. This integrated control method with cyclic working systems requires the same state. At the same time, the parameters $\underset{\_}{q}(t)$ and ${\underset{\_}{\tau }}_d\left(t\right)$ are also required to be periodic and have the exact change period as the working cycle of industrial robots. The primary iterative learning control method is only sometimes applied to meet the desired control quality. There have been many improvements to iterative learning to improve control quality and expand the scope of practical applications. The first improvement is the improvement of combining iterative learning with traditional control methods, often called indirect iterative learning or direct transmission iterative learning [9,10,11]. These improvements include [12] when the friction component alone cannot be determined [13], when that is not possible, or [14] when the learning function parameters cannot be found, as well as when the learning function parameters need to be changed. In relation to change during work cycles, a control trend for a class of robotic systems is model matching control, including [15]. This control method requires the mathematical model (1), which will encounter the problems and disadvantages of the previous traditional control methods. The next innovation is to use model-based control but to almost completely omit the mathematical model (1) of the robot, applicable to both cases where $\underset{\_}{q}(t)$ and ${\underset{\_}{\tau }}_d\left(t\right)$ are dependent. This depends on time and does not require periodicity, so it can be considered an intelligent model-matching control method. This method is based on theoretical results on the optimal control of each segment on the time axis available in [16] and applying disturbance estimation to control self-balancing two-wheeled vehicles [17]. The main content of this article is to analyze and evaluate additional improvements to essential iterative learning to improve the quality of motion control of robot systems or microrobot systems [18,19,20] and to expand the scope of practical applications, specifically. Intelligent control based on model matching (model matching) almost wholly does not need to use the mathematical model of the robot system, as applicable to the case where $\underset{\_}{q}(t)$ and ${\underset{\_}{\tau }}_d\left(t\right)$ depends on time, and there is no need for circulation for the robot system. The content of this article includes six parts. Part 1 is the problem of researching iterative learning for robot systems. Part 2 presents the results of the 2-DOF robot dynamic equation. Part 3 presents two robot control structures using iterative learning. Part 4 presents the first structure controls for the robot using iterative learning. Part 5 and Part 6 present the second structure controls for the robot using iterative learning. The conclusion and ideas for further research are covered in Part 7.

2. Robot Planar Dynamic Model

A 2-DOF planar robot arm is shown in Figure 1, where ${\theta }_{\mathrm{i}},{\mathrm{l}}_{\mathrm{i}} \mathrm{and} {\mathrm{m}}_{\mathrm{i}}$, respectively.

The dynamic equations for this system can be found using the Lagrange–Euler formulation [1]. The dynamic model of the 2-DOF is:

$${\underset{\_}{u}}_{dk}=H(\underset{\_}{q},\underset{\_}{\theta })\ddot{\underset{\_}{q}}+C(\underset{\_}{q},\dot{\underset{\_}{q}},\underset{\_}{\theta })+g(\underset{\_}{q},\underset{\_}{\theta })$$

(2)

With the following results:

$$\begin{array}{l}\left\{\begin{array}{ll}{H}_{11}=& {\mathrm{m}}_1{\mathrm{l}}_{C1}^2+{\mathrm{I}}_1+{\mathrm{m}}_2\left(l_1^2+l_{C2}^2+2l_1{l}_{C2}\cos {\theta }_2\right)+I_2\\ H_{12}=& \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}{m}_2({\mathrm{l}}_{C2}^2+l_1{l}_{C2}\cos {\theta }_2)+I_2\\ \begin{array}{l}{H}_{21}=\\ H_{22}=\end{array}& \begin{array}{l}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}{m}_2({\mathrm{l}}_{C2}^2+l_1{l}_{C2}\cos {\theta }_2)+I_2\\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}{m}_2{\mathrm{l}}_{C2}^2+I_2\end{array}\end{array}\right.;\left\{\begin{array}{ll}{C}_{11}=& -m_2{\mathrm{l}}_1{l}_{C2}\sin {\theta }_2(2{\dot{\theta }}_1{\dot{\theta }}_2+{\dot{\theta }}_2^2)\\ C_{12}=& {\mathrm{m}}_2{l}_1{l}_{C2}\left({\dot{\theta }}_1^2+{\dot{\theta }}_1{\dot{\theta }}_2\right)\sin {\dot{\theta }}_2-m_2{\mathrm{l}}_1{l}_{C2}\sin {\theta }_2{\dot{\theta }}_1{\dot{\theta }}_2\end{array}\right.\\ \left\{\begin{array}{ll}{g}_{11}=& {\mathrm{m}}_{2}g\left(l_1\cos {\theta }_1+l_{C2}\cos ({\theta }_1+{\theta }_2)\right)\\ \hspace{0.33em}{g}_{12}=& {\mathrm{m}}_2\mathrm{g}{\mathrm{l}}_{C2}\cos ({\theta }_1+{\theta }_2)\end{array}\right.+{\mathrm{m}}_1{\mathrm{l}}_{C1}\cos {\theta }_1\end{array}$$

(3)

Problems that arise in working in real-time, factors that cannot be ignored, include the impact of interference on the system and unwanted changes to model parameters. The control problem is the signal tracking control for the industrial robot motion system. The most comprehensive solution would be the method of using iterative learning control. This method can be added to a traditional control system to compensate for system errors, or it can also be applied immediately to robots without a conventional controller. The content of this article proposes two solutions to control industrial robots by iterative learning using innovative online learning parameters presented explicitly in Section 4, Section 5 and Section 6.

3. Two Structure Diagrams for Robot Control Using the Iterative Learning Method

The content of this article proposes two solutions to control industrial robots by iterative learning using innovative online learning parameters, as shown in Figure 2 and Figure 3.

These two solutions are different because we need to add an intermediate controller in the inner loop with the first solution, but we do not with the second solution. The task of this medium inner-loop controller is to make the industrial robot system adapt to the iterative learning control principle later because the iterative learning controller often requires the control object to be stable (with possibly no need for asymptotic stabilization). We need the mathematical model (1) of the industrial robot system to design an intermediate controller as a stable system. In the second solution, the uncertainty estimation block, in addition to the pure task of estimating the function uncertainty components to perform input compensation, must also carry out the additional function of making a stable system. If we apply the “smart” estimation principle introduced in Part 5 to design the uncertainty estimation block, then, with the second solution, we do not need the industrial robot system’s mathematical model (1). However, the price for this is that the processing speed of the second solution will be much slower than the first solution. According to the simulation results obtained, it is about 140 times slower. In addition, since both of these solutions use an uncertainty estimation block for compensation control, they will be applicable to the adaptive control of industrial robot systems with additional noise in the input, i.e., applying the method described by (4) and (15). This is also a difference from the original model (1) because interference in the actuators is unavoidable. Furthermore, it is possible to assume that this input disturbance can contain matched model uncertainties. In that case, both control solutions are adaptive to the trouble and robustness of model bias.

4. Structure of the First Diagram for Robot Control, the Iterative Learning Method

4.1. Algorithm Content According to the First Control Diagram

Take the mathematical model (1) and assume that $C(\underset{\_}{q},\underset{\_}{\dot{q}}),g(\underset{\_}{q})$ is undetermined. Then we have:

$$H(\underset{\_}{q})\underset{\_}{\ddot{q}}={\underset{\_}{u}}_{dk}+{\underset{\_}{\tau }}_d$$

(4)

With the sum of all the functional uncertainty components of the model:

$${\underset{\_}{\tau }}_d=\underset{\_}{n}(\underset{\_}{q},\underset{\_}{\dot{q}},\underset{\_}{\ddot{q}},t)-C(\underset{\_}{q},\underset{\_}{\dot{q}})\underset{\_}{\dot{q}}-g(\underset{\_}{q})$$

(5)

It can be seen immediately that when used:

$${\underset{\_}{u}}_{dk}=H(\underset{\_}{q})[\underset{\_}{\vartheta }-g_1\underset{\_}{q}-g_2\underset{\_}{\dot{q}}]$$

(6)

With $g_1,g_2$ being two properly chosen matrices, the system becomes:

$$\underset{\_}{\ddot{q}}=\underset{\_}{\vartheta }-g_1\underset{\_}{q}-g_2\underset{\_}{\dot{q}}+\varsigma \iff \left(\begin{array}{c}\underset{\_}{\dot{q}}\\ \underset{\_}{\ddot{q}}\end{array}\right)=\left(\begin{array}{cc}{0}_n& I_n\\ -g_1& -g_2\end{array}\right)\left(\begin{array}{c}\underset{\_}{q}\\ \underset{\_}{\dot{q}}\end{array}\right)+\left(\begin{array}{c}{0}_n\\ I_n\end{array}\right)\left[\underset{\_}{\vartheta }+\underset{\_}{\xi }\right]$$

(7)

where $\underset{\_}{\xi }=M{(\underset{\_}{q})}^{-1}{\underset{\_}{\tau }}_d$ is an uncertain component of system (7).

$$\underset{\_}{X}=\left(\begin{array}{c}\underset{\_}{q}\\ \underset{\_}{\dot{q}}\end{array}\right),g=\left(\begin{array}{cc}{0}_n& I_n\\ -g_1& -g_2\end{array}\right),B=\left(\begin{array}{c}{0}_n\\ I_n\end{array}\right)$$

(8)

The model (7) will have the canonical form:

$$\dot{\underset{\_}{X}}=G\underset{\_}{X}+B\left[\underset{¯}{\vartheta }+\underset{\_}{\xi }\right]$$

(9)

In addition, for system (9) to be stable, that is, to have a Hurwitz matrix A, we must choose two matrices $g_1,g_2$ so that the eigenvalues of A given in (8) lie to the left of the imaginary axis. The simplest way is to choose them according to the method of assigning pole points using Matlab’s place(.) command, and these pole points should be selected as accurate (negative) numbers to eliminate fluctuations during transitions.

And then the system output is:

$$\underset{\_}{Y}=\underset{\_}{q}=\left(I_n,0_n\right)\underset{\_}{X}$$

(10)

following the given periodic $\underset{\_}{R}(t)$ pattern trajectory.

Now we use notation: ${\underset{\_}{\vartheta }}_k(n)={\underset{\_}{\vartheta }}_k(t){|}_{t=n{t}_S},{\underset{\_}{Y}}_k(n)={\underset{\_}{Y}}_k(t){|}_{t=n{t}_s}$

Given the input and output of systems (9) and (10), respectively, $n{t}_s$ at the time in the k test, with $t_s$ being the sample extraction cycle, then the P-type learning function, standard to all update time i at the k time, will be:

$${\underset{\_}{\vartheta }}_{k+1}={\underset{\_}{\vartheta }}_k+K{\underset{\_}{e}}_k$$

(11)

where:

$$\begin{array}{l}{\underset{\_}{\vartheta }}_k=col({\underset{\_}{\vartheta }}_k(0),{\underset{\_}{\vartheta }}_k(1),\dots ,{\underset{\_}{\vartheta }}_k(S-1));\\ {\underset{\_}{E}}_k=\underset{\_}{\vartheta }-{\underset{\_}{\omega }}_k;\\ {\underset{\_}{\omega }}_k=col({\underset{\_}{Y}}_k(0),{\underset{\_}{Y}}_k(1),\dots ,{\underset{\_}{Y}}_k(S-1));\\ {\underset{\_}{\upsilon }}_k=col({\underset{\_}{R}}_k(0),{\underset{\_}{R}}_k(1),\dots ,{\underset{\_}{R}}_k(S-1));\\ K=\left(\begin{array}{cc}\begin{array}{l}K\\ ⋮\end{array}& \begin{array}{l}{0}_{nxn}\\ ⋮\end{array}\\ 0_{nxn}& K\end{array}\right)\in {ℝ}^{\mathrm{nSxnS}} and K\in {ℝ}^{nxn}\end{array}$$

(12)

K is the matrix of learning functions that must be chosen so that the learning function converges.

The uncertainty component of the system, in general, can be estimated and explicitly compensated as follows: First, we must approximate model (9) at time i in the k time, then when paid by ${\underset{\_}{\widehat{\xi }}}_k(n-1)$, becomes:

$$\frac{{\underset{\_}{X}}_k(n)-{\underset{\_}{X}}_k(n-1)}{t_s}\approx g{\underset{\_}{X}}_k(n)+B\left[{\underset{\_}{\vartheta }}_k(n)-{\underset{\_}{\widehat{\xi }}}_k(n-1)+{\underset{\_}{\widehat{\xi }}}_k(n)\right]$$

Hence, $\widehat{g}{\underset{\_}{X}}_k(n)\approx {\underset{\_}{X}}_k(n-1)+\widehat{B}\left[{\underset{\_}{\vartheta }}_k(n)-{\underset{\_}{\widehat{\xi }}}_k(n-1)+{\underset{\_}{\varsigma }}_k(n)\right]\iff \widehat{B}{\underset{\_}{\widehat{\xi }}}_k(n)\approx {\underset{\_}{h}}_k(n)$

In which: $\widehat{g}=\left(I_{2n}-t_{s}g\right),\widehat{B}=t_{s}B,{\underset{\_}{h}}_k(n)=\widehat{g}{\underset{\_}{X}}_k(n)-{\underset{\_}{X}}_k(n-1)-\widehat{B}\left[{\underset{\_}{\vartheta }}_k(n)-{\underset{\_}{\widehat{\xi }}}_k(n-1)\right]$

So, from ${\widehat{B}}^T\widehat{B}=t_s^2{I}_n$, we have the estimated value ${\underset{\_}{\widehat{\xi }}}_k(n)\approx {\underset{\_}{\xi }}_k(n)$ for the next offset at time n + 1 as:

$$\begin{array}{ll}{\underset{\_}{\widehat{\xi }}}_k(n)& ={({\widehat{B}}^T\widehat{B})}^{-1}{\widehat{B}}^T{\underset{\_}{h}}_k(n)=\frac{1}{t_s^2}{\widehat{B}}^T{\underset{\_}{h}}_k(n)=\frac{1}{t_s^2}{B}^T{\underset{\_}{h}}_k(n)\\ & =\frac{1}{t_s}(0_n,I_n)\left\{\widehat{g}{\underset{\_}{X}}_k(n)-{\underset{\_}{X}}_k(n-1)-\widehat{B}\left[{\underset{\_}{\vartheta }}_k(n)-{\underset{\_}{\widehat{\xi }}}_k(n-1)\right]\right\}\\ & =\frac{1}{t_s}{B}^T\left[\widehat{g}{\underset{\_}{X}}_k(n)-{\underset{\_}{X}}_k(n-1)\right]-{\underset{\_}{\vartheta }}_k(n)+{\underset{\_}{\widehat{\xi }}}_k(n-1)\end{array}$$

(13)

The structure of the intelligent control system for industrial robots is shown in Figure 2.

Below is the Algorithm 1 describing the steps to calculate an iterative learning controller, combined with identifying and compensating uncertain components (with no need to be repetitive) for industrial robots after accurate linearization; after several tries k is continuous along with the control process and only ends when the control stops.

Algorithm 1: The first structure for robot control, the iterative learning method, is with function parameters smart online learning.
1	Assign $t=0,\widehat{\underset{\_}{\xi }}=\underset{\_}{0}.$ Choose $t_s>0$; Calculate $\widehat{g},S$. Choose K; Assign small value $x_0=0,{\underset{\_}{\widehat{\xi }}}_k=\underset{\_}{0},{\underset{\_}{\vartheta }}_k=col({\underset{\_}{\vartheta }}_k(0),\dots ,{\underset{\_}{\vartheta }}_k(S-1))=col(\underset{\_}{R}(0),\dots ,\underset{\_}{R}(S-1))$.
2	while continue the control do
3	for $n=0,1,\dots ,S-1$ do
4	Send $\underset{\_}{\vartheta }(n)-\widehat{\underset{\_}{\varsigma }}$ to into uncertain control (9) and determine $\underset{\_}{X}=\underset{\_}{X}(t+n{t}_s)$.
5	Calculate ${\underset{\_}{\widehat{\xi }}}_k\leftarrow B^T(\widehat{g}\underset{¯}{X}-{\underset{¯}{X}}_0)/t_s-{\underset{\_}{\vartheta }}_k(n)+\widehat{\underset{\_}{\varsigma }} and {\underset{¯}{X}}_0\leftarrow \underset{¯}{X}$.
6	Establish $\underset{¯}{Y}\left(n\right)=(I_n,0_n)\underset{¯}{X}$ and calculate $\underset{\_}{e}(n)=\underset{\_}{R}(n)-\underset{\_}{Y}(n)$.
7	end for
8	Set up the sum vector $\underset{\_}{\epsilon }$ từ $\underset{\_}{e}(n)$, $\mathrm{n}=0,1\dots ,\mathrm{S}-1$ theo (12)
9	Update vector $\underset{\_}{\vartheta }(n)$ from its existing value according to (11) that is, calculate the values $\underset{\_}{\vartheta }(n)$,$n=0,1,\dots ,S-1$ for the next try.
10	Set $t\leftarrow t+T$
11	end while

4.2. Applied to Robot Control

For the simulation, there are assigned:

$$g_1=\left[\begin{array}{cc}30& 0\\ 0& 12\end{array}\right],g_2=\left[\begin{array}{cc}11& 0\\ 0& 7\end{array}\right],K=\left[\begin{array}{cc}2.9& 0\\ 0& 1.8\end{array}\right],t_s=0.05s,T=20s$$

There are periodic functions of period T as follows:

$${\underset{\_}{R}}_1(t)=\left\{\begin{array}{c}\min \left\{t_{s}t/2, 1.5\right\} khi t\le \raisebox{1ex}{$T$}\!\left/ \!\raisebox{-1ex}{$2$}\right.\\ {\underset{\_}{R}}_1(T-t)khi \raisebox{1ex}{$T$}\!\left/ \!\raisebox{-1ex}{$2$}\right.<t\le T\end{array}\right. ; {\underset{\_}{R}}_2(t)=\sin (\raisebox{1ex}{$2\pi t$}\!\left/ \!\raisebox{-1ex}{$T$}\right.)$$

We obtain the simulation results in Figure 4, Figure 5, Figure 6 and Figure 7.

Comment. 

Figure 4 and Figure 5 depict the simulation results, which demonstrate that, when combined with the identification and noise compensation stages of the traditional controller (13), the iterative learning controller (11) has achieved a good signal tracking quality. The preset error with tracking error after 60 trials is small enough and acceptable:

$$-0.03\le {\mathrm{e}}_1(\mathrm{t})\le 0.065 \mathrm{and}-0.017\le {\mathrm{e}}_2(\mathrm{t})\le 0.02$$

The residual error $e_1(t)$ in the simulation results is due to the non-smooth trapezoidal signal set ${\underset{\_}{R}}_1(t)$ at $t_0=0;t_1=1.5t_s;t_2=T-1.5t_s;t_3=T$. It also affects the tracking error $e_2(t)$ due to channel interleaving in the system.

5. The Structure of the Second Iterative Learning Controller with Model-Free Determination of Optimal Learning Parameters for an Industrial Robot

Stabilizing the system without using traditional control methods makes it possible to apply iterative learning control to even unstable systems without using a mathematical model of the system.

Tool: Estimating the derivative using Taylor. Besides Taylor analysis, there are many numerical methods to approximate the derivative, such as relying on Lagrangian interpolation or comparison with sample models, etc., as in [21,22,23] series analysis.

5.1. Algorithm Content According to the First Control Diagram

It is obvious that the Euler-Lagrange model (1) of robot manipulators is equivalent to

$$\ddot{\underset{\_}{q}}=-A_1\underset{\_}{q}-A_2\dot{\underset{\_}{q}}+{\underset{\_}{u}}_{dk}+\underset{\_}{\mu }$$

(14)

where $A_1,A_2$ are two arbitrarily chosen matrices and

$$\underset{\_}{\mu }={\underset{\_}{\tau }}_d+\left[I_n-H(\underset{\_}{q},\underset{\_}{\theta })\right]\ddot{\underset{\_}{q}}-\left[C(\underset{\_}{q},\dot{\underset{\_}{q}},\underset{\_}{\theta })-A_2\right]\dot{\underset{\_}{q}}-\left[g(\underset{\_}{q},\underset{\_}{\theta })-A_1\underset{\_}{q}\right].$$

(15)

From (15), we see that the route cost calculation of model (1) is contained in this vector $\underset{\_}{\eta }$. Based on the equivalent model (14), the proposed intelligent controller for the robot is as follows:

First, the estimation errors of the new unknown function vector $\underset{\_}{\eta }$ will be adjusted, as shown below.

$${\underset{\_}{u}}_{dk}=\underset{\_}{\vartheta }-\widehat{\underset{\_}{\mu }}$$

(16)

With the estimation of bias $\underset{¯}{\delta }$ as (17) system (14) will become linear:

$$\underset{¯}{\dot{X}}=A\underset{¯}{X}+B\left[\underset{¯}{\vartheta }+\underset{¯}{\delta }\right],\underset{¯}{Y}=\underset{¯}{q}=\left(I_n,0_n\right)\underset{¯}{X}$$

(17)

with

$$\dot{\underset{\_}{X}}=\left(\begin{array}{c}\underset{¯}{q}\\ \underset{¯}{\dot{q}}\end{array}\right),A=\left(\begin{array}{cc}{0}_n& {\mathrm{I}}_n\\ -A_1& -A_2\end{array}\right),B=\left(\begin{array}{c}{0}_n\\ I_n\end{array}\right),\underset{¯}{\delta }=\underset{\_}{\mu }-\widehat{\underset{\_}{\mu }}$$

(18)

and $0_n$, $I_n$ are $nxn$ square matrices with zero.

Secondly, we control the compensation system in Equation (17) so that its output $\underset{¯}{Y}(t)$ will converge to a desired reference value $\underset{\_}{R}(t)$.

5.2. Model-Free Disturbance Compensation for Internal Loop Control by Feedback Linearization

Going back to the pay provider ${\underset{\_}{\vartheta }}_k$ in Equation (16) and designating as the present with $t=kT+\tau ;0\le \tau <T and {\underset{¯}{\vartheta }}_k={\underset{¯}{\vartheta }}_k(\tau );{\underset{¯}{X}}_k={\underset{¯}{X}}_k(\tau )$ then (17) is rewritten as:

$${\underset{¯}{\dot{X}}}_k(\tau )=A{\underset{¯}{X}}_k(\tau )+B\left[{\underset{\_}{\vartheta }}_k(\tau )-{\widehat{\underset{\_}{\mu }}}_k(\tau )+{\underset{¯}{\mu }}_k(\tau )\right]$$

(19)

The automatic model-free feedback linearization block (Figure 2) aims to estimate $\underset{¯}{\widehat{\eta }}$ based on the measured values of ${\underset{¯}{X}}_k(\tau )$ at two times $\tau =n{t}_s$ and $t-t_s=t_s(n-1)$ for each corresponding ${\underset{¯}{X}}_k(n{t}_s);{\underset{¯}{X}}_k(n-1)t_s$ value. With $0<t_s<<1$ being an arbitrarily chosen constant small value, we then expand the Taylor function of the function ${\underset{¯}{X}}_k(\tau )$ around $(n-1)t_s$, as follows:

$${\underset{¯}{X}}_k\left((n-1)t_s\right)=\frac{t_s^2}{2}{\underset{¯}{\ddot{X}}}_k\left(\varsigma \right)-t_s{\underset{¯}{\dot{X}}}_k(n{t}_s)+{\underset{¯}{X}}_k(n{t}_s)$$

(20)

where $(n-1)t_s\le \varsigma \le n{t}_s,or$

$${\underset{¯}{\dot{X}}}_k(n{t}_s)\approx \frac{{\underset{¯}{X}}_k(n{t}_s)-{\underset{¯}{X}}_k\left((n-1)t_s\right)}{t_s}.$$

(21)

Assuming the last term of expression (20) is very small and can be ignored, the expression ${\underset{¯}{\widehat{\eta }}}_k(n{t}_s)$ will be used to approximate Equation (19) by replacing the signs “$\approx$” and ${\underset{¯}{\eta }}_k(n{t}_s)$ in (10) with “=” and ${\underset{¯}{\widehat{\eta }}}_k(n{t}_s)$

$$\frac{{\underset{¯}{X}}_k(n{t}_s)-{\underset{¯}{X}}_k\left((n-1)t_s\right)}{t_s}\approx A{\underset{¯}{X}}_k(n{t}_s)+B\left[{\underset{\_}{\vartheta }}_k(n{t}_s)-{\widehat{\underset{\_}{\mu }}}_k\left((n-1)t_s\right)+{\underset{¯}{\mu }}_k(n{t}_s)\right]$$

(22)

Then, we calculate:

$$B{\underset{¯}{\widehat{\eta }}}_k(n{t}_s)=\frac{{\underset{¯}{X}}_k(n{t}_s)-{\underset{¯}{X}}_k\left((n-1)t_s\right)}{t_s}-A{\underset{¯}{X}}_k(n{t}_s)-B\left[{\underset{¯}{\vartheta }}_k(n{t}_s)-{\widehat{\underset{\_}{\mu }}}_k\left((n-1)t_s\right)\right]$$

Which yields:

$${\widehat{\underset{\_}{\mu }}}_k(n{t}_s)=B^T\left[\frac{{\underset{¯}{X}}_k(n{t}_s)-{\underset{¯}{X}}_k\left((n-1)t_s\right)}{t_s}-A{\underset{¯}{X}}_k(n{t}_s)\right]-\left[{\underset{¯}{\vartheta }}_k(n{t}_s)-{\widehat{\underset{\_}{\mu }}}_k\left((n-1)t_s\right)\right]$$

(23)

Theorem 1. 

The value ${\underset{¯}{\widehat{\eta }}}_k(n{t}_s)$ calculated in Equation (23) causes the approximation error of expression (22) to decrease.

Proof. 

Error in expression (22) $\underset{¯}{\epsilon }$ will be calculated:

$$\begin{array}{ll}\underset{\_}{\sigma }& =A{\underset{¯}{X}}_k(n{t}_s)+B\left[{\underset{\_}{\vartheta }}_k(n{t}_s)-{\widehat{\underset{\_}{\mu }}}_k\left((n-1)T_s\right)+{\underset{¯}{\eta }}_k(n{t}_s)\right]-\frac{{\underset{¯}{X}}_k(n{t}_s)-{\underset{¯}{X}}_k\left((n-1)t_s\right)}{t_s}\\ & =B{\underset{¯}{\mu }}_k(n{t}_s)+\underset{¯}{\lambda }\end{array}$$

where

$$\underset{¯}{\lambda }=A{\underset{¯}{X}}_k(n{t}_s)+B\left[{\underset{¯}{\vartheta }}_k(n{t}_s)-{\widehat{\underset{\_}{\mu }}}_k\left((n-1)t_s\right)\right]-\frac{{\underset{¯}{X}}_k(n{t}_s)-{\underset{¯}{X}}_k\left((n-1)t_s\right)}{t_s}.$$

And the estimated value will also be calculated according to expression (22) as follows:

$$\frac{{\underset{¯}{X}}_k(n{t}_s)-{\underset{¯}{X}}_k\left((n-1)t_s\right)}{t_s}=A\ast {\underset{¯}{X}}_k(n{t}_s)+B\left[{\underset{¯}{\vartheta }}_k(n{t}_s)-{\widehat{\underset{\_}{\mu }}}_k\left((n-1)t_s\right)+{\underset{\_}{\mu }}_k(n{t}_s)\right]$$

The optimization problem would then be written:

$$\begin{array}{ll}{\mu }^{*}& =\underset{{\underset{¯}{\eta }}_k}{\mathit{arg}\mathit{min}}{‖{\underset{¯}{\epsilon }}_k‖}^2=\underset{{\underset{¯}{\eta }}_k}{\mathit{arg}\mathit{min}}{‖B{\underset{\_}{\mu }}_k+\underset{¯}{\gamma }‖}^2=\underset{{\underset{¯}{\eta }}_k}{\mathit{arg}\mathit{min}}{‖B{\underset{\_}{\mu }}_k+\underset{¯}{\gamma }‖}^T‖B{\underset{\_}{\mu }}_k+\underset{¯}{\gamma }‖\\ & =\underset{{\underset{¯}{\eta }}_k}{\mathit{arg}\mathit{min}}{‖{\underset{\_}{\mu }}_k^T{\underset{\_}{\mu }}_k+\underset{¯}{2{\underset{¯}{\gamma }}^{T}B{\underset{\_}{\mu }}_k+{\underset{¯}{\gamma }}^T\underset{¯}{\gamma }}‖}^T\end{array}$$

This has a unique solution:

$${\mu }^{*}=-B^T\underset{¯}{\gamma }$$

which coincides with ${\widehat{\underset{\_}{\mu }}}_k(n{t}_s)$ given in (23).

From (23), we see that the estimation tool created to compensate for noise and uncertainty in the vector model $\underset{\_}{\mu }$ does not use the mathematical model (1). Therefore, the noise compensator (16) with $\widehat{\underset{\_}{\mu }}$ obtained from expression (23) will be model-free. □

5.3. Outer Loop Control Is by Iterative Learning Controller Design

Expression (17) describes the system (14) that becomes LTI by compensating the $\underset{¯}{\eta }$ synthetic disturbances in (15) by the compensator (16), and, for iterative learning systems, it is rewritten as follows in ILC language:

$$\left\{\begin{array}{c}{\underset{¯}{X}}_k(n+1)=\widehat{A}\ast {\underset{¯}{X}}_k(n)+\widehat{B}\left[{\underset{¯}{\vartheta }}_k(i)+{\underset{¯}{\delta }}_k(n)\right]\\ {\underset{¯}{Y}}_k(n)=\widehat{C}\ast {\underset{¯}{X}}_k(n)\end{array}\right\}$$

(24)

where $n=0,1,\dots ,N=T/t_s,{\underset{¯}{X}}_k(N)={\underset{¯}{X}}_{k+1}(0),and$

$$\begin{array}{l}\widehat{A}=\mathit{exp}(A{t}_s)\\ \widehat{B}={\displaystyle \underset{0}{\overset{T_s}{\int }}{e}^{At}Bdt}\\ \widehat{C}=(I_n,O_n)\end{array}$$

(25)

The control objective at this point is to choose the proper learning parameter K for the update rule P-Type, assuming we have identified two matrices that make the matrix A provided in Equation (6).

$${\underset{¯}{\vartheta }}_{k+1}\left(n\right)={\underset{¯}{\vartheta }}_k\left(n\right)+K{\underset{\_}{E}}_k,with {\underset{\_}{E}}_k(n)={\underset{¯}{R}}_k(n)+{\underset{¯}{Y}}_k(n)$$

(26)

For the necessary convergence to be satisfied $‖{\underset{\_}{E}}_k(n)‖\to 0$ for all n, or at least as close as possible to the origin.

The value ${\underset{¯}{\vartheta }}_k$ given by ILC (26) is partially constant, so the discrete time model in Equation (24) obtained is equivalent to the continuous time model in Equation (17). In expression (24), no information about the system model (1) is used either. Therefore (24) can be applied to all robot controllers. From (24), it is obtained with the assumption ${\underset{¯}{\delta }}_k(n)=\underset{¯}{0}$, ${\underset{¯}{Y}}_{k+1}(n)=C{\widehat{A}}^n{\underset{¯}{X}}_{k+1}(0)+{\displaystyle \sum _{j=0}^{n-1}C{\widehat{A}}^{n-n-1}\widehat{B}}{\underset{\_}{\vartheta }}_{k+1}(j),$

This results from (24) and the clear demonstration of repeatable ability ${\underset{¯}{X}}_k(0)=X_{k+1}(0),\forall k$

$$\begin{array}{ll}{\underset{\_}{E}}_{k+1}(n)& =\underset{¯}{R}(n)-{\underset{¯}{Y}}_{k+1}(n)=\underset{¯}{R}(n)-\left[C{\widehat{A}}^n{\underset{¯}{X}}_k(0)+{\displaystyle \sum _{j=0}^{n-1}C{\widehat{A}}^{n-n-1}}\widehat{B}(\underset{\_}{\vartheta }(j)+K{\underset{\_}{E}}_k(j))\right]\\ & =\underset{¯}{R}(n)-{\underset{¯}{Y}}_k(n)-{\displaystyle \sum _{j=0}^{n-1}C{\widehat{A}}^{n-j-1}\widehat{B}}K{\underset{\_}{E}}_k(j)=(I-C\widehat{B}K){\underset{\_}{E}}_k(n)-{\displaystyle \sum _{j=0}^{n-2}C{\widehat{A}}^{n-j-1}\widehat{B}}K{\underset{\_}{E}}_k(j)\end{array}$$

Hence,

$${\underset{\_}{E}}_{k+1}=\Phi {\underset{\_}{E}}_k$$

(27)

where

$${\underset{\_}{E}}_k=\left(\begin{array}{c}{\underset{\_}{E}}_k(0)\\ {\underset{\_}{E}}_k(1)\\ ⋮\\ {\underset{\_}{E}}_k(N-1)\end{array}\right) and \Phi =\left(\begin{array}{cccc}I-C\widehat{B}K& 0& \cdots & 0\\ -C\widehat{A}\widehat{B}K& I-C\widehat{B}K& \cdots & 0\\ ⋮& ⋮& ⋮& ⋮\\ C{\widehat{A}}^{N-2}\widehat{B}K& -C{\widehat{A}}^{N-2}\widehat{B}K& \cdots & I-C\widehat{B}K\end{array}\right)$$

(28)

Theorem 2. 

For the situation ${\underset{¯}{\delta }}_k(n)=0$ , all requirements $‖{\underset{¯}{E}}_k(j)‖\to 0$ for all $n=0,1,\dots ,N-1$ will only be fulfilled if and when the P-Type learning parameter K is adjusted in a way that makes the value given in (16) Schur.

Proof. 

For the autonomous system (27) the proof is comprehensive as it is clear that it is stable if and only if it is Schur. □

5.4. The Closed-Loop System’s Performance and Control Algorithm

The following Algorithm 2 is established to implement the proposed model-free controller (16) where the $\underset{¯}{\vartheta }$ and $\underset{¯}{\mu }$ values are obtained from expressions (26) and (13), respectively. In this control algorithm, the working time T of the robots is repeatedly tested by each While loop. Figure 2 shows the closed-loop system’s output tracking performance with the output scenario ${\underset{¯}{\delta }}_k(n)\ne \underset{¯}{0}$.

Theorem 3. 

The robot manipulators (1) output tracking error is driven by the proposed model-free control framework in Figure 1, which comprises the feedback linearization block via disturbance compensator (16), (23), and the ILC block (26) drives the robot manipulators (1) output tracking error $\left|{\underset{¯}{E}}_k(\tau )\right|$ to $T_s$ dependent neighborhood $O$ of origin if d is limited and continuous. If a smaller $t_s$ is selected, there will be a smaller $O$.

Proof. 

Because noise ${\underset{\_}{\tau }}_d$ is continuous and bounded, the total noise $\underset{¯}{\mu }$ and component $\underset{¯}{\delta }$ are also continuous and bounded. The upper limit of $O$ is $\Delta$ and this limit can be reduced arbitrarily by reducing $t_s$ according to Theorem 1.

• The validity of Theorem 2 was confirmed by:

$$\underset{¯}{\dot{\omega }}=A\underset{¯}{\omega }+B\underset{\_}{\vartheta } \mathrm{with} \underset{¯}{\omega }=vec(\underset{¯}{R},\underset{¯}{\dot{R}})$$

(29)

The subtraction of (29) from (17) yields:

$$\dot{\underset{\_}{\sigma }}=A\underset{\_}{\sigma }-B\underset{¯}{\delta } where \underset{\_}{\sigma }=vec(\underset{\_}{E},\dot{\underset{\_}{E}})$$

(30)

Using the Lyapaunov equation $A^{T}P+PA=-Q$ with optional positive definite matrix Q we always have positive definite P because A is Hurwitz. From the positive definite function $V(\underset{\_}{\sigma })={\underset{\_}{\sigma }}^{T}P\underset{\_}{\sigma }$ we have the result:

$$\begin{array}{ll}\dot{V}& =(A\underset{\_}{\sigma }-B\underset{¯}{\delta }{)}^T+{\underset{\_}{\sigma }}^{T}P(A\underset{\_}{\sigma }-B\underset{¯}{\delta })\\ & ={\underset{\_}{\sigma }}^T(A^{T}P+PA)\underset{\_}{\sigma }-2{\underset{\_}{\sigma }}^{T}PB\underset{¯}{\delta }=-{\underset{\_}{\sigma }}^T(-A^{T}P-PA)\underset{\_}{\sigma }-2{\underset{\_}{\sigma }}^{T}PB\underset{¯}{\delta }\\ & =\le -{\lambda }_{\mathit{max}}(-A^{T}P-PA){‖\underset{\_}{\sigma }‖}^2+2‖PB‖\Delta ‖\underset{¯}{\epsilon }‖=‖\underset{¯}{\epsilon }‖\left[-{\lambda }_{\mathit{max}}(-A^{T}P-PA)‖\underset{\_}{\sigma }‖+2‖PB‖\Delta \right]\end{array}$$

(31)

with ${\lambda }_{\mathit{max}}(-A^{T}P-PA)$ is the maximum value of the Q matrix.

Since $\dot{V}(\underset{¯}{\epsilon })<0$ as long as

$$2‖PB‖\Delta <{\lambda }_{\mathit{max}}(-A^{T}P-PA)‖\underset{¯}{\epsilon }‖ \mathit{or} \frac{2‖PB‖\Delta }{{\lambda }_{\mathit{max}}(-A^{T}P-PA)}<‖\underset{¯}{\epsilon }‖$$

(32)

The orbital error vector tends to the origin:

$$O=\left\{\underset{\_}{\sigma }\in R^{2n}\left| ‖\underset{\_}{\sigma }‖\le \frac{2‖PB‖\Delta }{{\lambda }_{\mathit{max}}(-A^{T}P-PA)}\right.\right\}$$

(33)

This control algorithm contains all the necessary calculations and has the following structure:

Algorithm 2: The structure of the second iterative learning controller with model-free determination of optimal learning parameters for an industrial robot
1	Choose two matrices $A_1$,$A_2$, given in (25) become Hurwitz. Determine $\widehat{A},\widehat{B},\widehat{C}$ given in (25) and $\Phi$ given in (28). Choose $0<t_s<<1$. Calculate $S=T/t_s$. Determine $D_1^T=\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$T_s$}\right.;D_0^T=-D_1^T$ given in (23) Choose learning $\widehat{\underset{\_}{\mu }}$ and tracking error ${\underset{\_}{E}}_0$. Allocate $\underset{¯}{\vartheta }\left(\mathrm{n}\right)=\underset{¯}{\mathrm{R}}\left(\mathrm{n}\right), \mathrm{with} \mathrm{n}=0,1,\dots ,\mathrm{S}-1 \mathrm{and} \underset{¯}{\mathrm{Z}}=\underset{¯}{0}$. Choose learning parameter K so that Ф of (28) becomes Schur.
2	while continue the control do
3	for  $n=0,1,\dots ,S-1$ do
4	Send ${\underset{\_}{u}}_{dk}=A_1\underset{\_}{q}-A_2\dot{\underset{\_}{q}}-\widehat{\underset{\_}{\mu }}$ to robot for a while of $t_s$.
	measure $\underset{¯}{X}=vec(\underset{\_}{E},\dot{\underset{\_}{E}})$, $\underset{¯}{Y}(n)= \underset{¯}{q}$.
5	calculate ${\widehat{\underset{\_}{\mu }}}_k\leftarrow B^T\left[\frac{\underset{¯}{X}-\underset{¯}{Z}}{t_s}-A\underset{¯}{X}\right]-(\underset{¯}{\vartheta }(n)-\widehat{\underset{\_}{\mu }})$.
	$\mathit{Set} \underset{¯}{Z}\leftarrow \underset{\_}{X}$
6	end for
7	assemble $\underset{\_}{\vartheta }=vec( \underset{\_}{\vartheta }(0),\dots , \underset{\_}{\vartheta }(N-1)), \underset{\_}{E}=vec(\underset{\_}{E}(0),\dots ,\underset{\_}{E}(N-1))$.
8	calculate $K=\underset{a\le K\le b}{\mathit{arg}\mathit{min}}‖\left(I-\Phi K\right)\underset{\_}{E}‖$ and $\underset{¯}{\vartheta }\leftarrow \underset{¯}{\vartheta }+K\underset{\_}{E};{\underset{\_}{E}}_0\leftarrow \underset{\_}{E};$
9	end while

Thus, each while—do loop represents one working cycle (one attempt) in this algorithm. In addition, it can be seen that the above control algorithm does not use model (1) of the robot at all, so it is also an intelligent algorithm. □

5.5. Applied to Robot Control

With

$${\mathrm{t}}_{\mathrm{s}}=0.02\mathrm{s} \mathrm{and} {\underset{\_}{\tau }}_d\left(t\right)=\left(\begin{array}{c}{\theta }_{11}\mathit{sin}{\theta }_{12}t\\ {\theta }_{13}\mathit{sin}{\theta }_{14}t\end{array}\right)$$

(34)

the parameters ${\theta }_{\mathrm{n}},\mathrm{n}=1÷14$ are random. The robot is assumed to have a period cycle: $\mathrm{T}=10\mathrm{s}$. A robot with two degrees of freedom has the following parameters (Figure 1): ${\mathrm{l}}_1=0.45,{\mathrm{l}}_2=0.65,\mathrm{g}=9.81$.

Assigned:

$$\begin{array}{l}{\underset{\_}{R}}_1(t)=\left\{\begin{array}{c}\min \left\{t_{s}t/2, 1.5\right\} khi t\le \raisebox{1ex}{$T$}\!\left/ \!\raisebox{-1ex}{$2$}\right.\\ {\underset{\_}{R}}_1(T-t)khi \raisebox{1ex}{$T$}\!\left/ \!\raisebox{-1ex}{$2$}\right.<t\le T\end{array}\right.\\ {\underset{\_}{R}}_2(t)=\sin (\raisebox{1ex}{$2\pi t$}\!\left/ \!\raisebox{-1ex}{$T$}\right.)\end{array}$$

$$A_1=\left[\begin{array}{cc}31& 0\\ 0& 12\end{array}\right];A_2=\left[\begin{array}{cc}12& 0\\ 0& 8\end{array}\right];K=\left[\begin{array}{cc}0.8& 0\\ 0& 0.3\end{array}\right];t_s=0.02s$$

Simulation results to verify the above algorithm are shown in Figure 8 and Figure 9.

Comment. 

These visual tracking results demonstrate the convergence of both joint variables to the required references. Over the course of the working day, the highest tracking error value is around 250 $\mathit{max}\left|{\underset{\_}{E}}_{250}(1)\right|\approx 0.03$ and $\mathit{max}\left|{\underset{\_}{E}}_{250}(2)\right|\approx 0.02$. Furthermore, they confirmed that the tracking error will decrease with the number of trials conducted. Therefore, these simulation findings fully validate all of the above-mentioned theoretical claims.

6. The Structure of the Second Iterative Learning Controller with Model-Free Determination of Online Learning Parameters for an Industrial Robot

The structure of two control loop circuits is still used, as shown in Figure 3.

6.1. Control the Inner Loop

Next is the inner loop. This loop uses Taylor series analysis to estimate $\underset{¯}{\widehat{\eta }}$ at time $\tau =kT+n{t}_s$, the previously measured value of ${\underset{\_}{q}}_k(n),{\dot{\underset{\_}{q}}}_k(n),$ with $j=n;n-1$. This estimation formula is given in (35).

The remarkable thing is that while the iterative learning controller works in a cyclic cycle T, the inner loop controller is not cyclic. In other words, the internal loop controller also estimates the non-periodic functional uncertainty component. It can be seen that neither control loop uses the mathematical model (1) of the robot at all, and that is their intelligent feature.

6.2. Outer Loop Control Is by Iterative Learning Controller Design

This is a loop circuit that uses cybernetics P-style iteration with a diagonal learning function parameter matrix:

$$K_k=diag\left(K_k^{(n)}\right),n=1,2,\dots ,n$$

(35)

The number of elements equals the number of joint variables $\underset{\_}{q}\in R^n$ (output) and the input variable ${\underset{\_}{u}}_{dk}$ because robot (1) is a sufficient actuator. Each $K_k^{(j)}$ element is determined according to:

$$K={\underset{\_}{a}}_k.\underset{0\le Z\le 1}{\mathit{arg}\mathit{min}}‖\underset{\_}{Z}{\underset{\_}{E}}_k\left(1-z\right){\underset{\_}{E}}_{k-1}‖$$

(36)

or

$$K={\underset{\_}{a}}_k.\underset{0\le Z\le 1}{\mathit{arg}\mathit{min}}‖{\underset{\_}{E}}_k\left(1-\underset{\_}{Z}\right)+\underset{\_}{Z}{\underset{\_}{E}}_{k-1}‖$$

(37)

This loop generates the control signal

$${\underset{\_}{u}}_{dk}=-A_1\underset{\_}{q}-A_2\dot{\underset{\_}{q}}-\underset{\_}{\widehat{\mu }}$$

(38)

where we choose two matrices $A_1$, $A_1$ in (6), which become Hurwitz, and a sufficiently small constant $0<t_s<<1$, $\underset{\_}{\vartheta }\in R^n$ is obtained from the iterative learning controller $\underset{\_}{\vartheta }(t)={\underset{\_}{\vartheta }}_k(\tau )$ piecewise constant form, i.e.,

${\underset{\_}{\vartheta }}_k(\tau )={\underset{\_}{\vartheta }}_k(n)$ when $nT<\tau <<\left(n+1\right)t_s$ with $\tau <<n{t}_s,n=0,1,\dots ,N-1$

and ${\underset{\_}{\vartheta }}_{k+1}(\tau )={\underset{\_}{\vartheta }}_k+K_k{\underset{\_}{E}}_k$

with $\underset{\_}{\vartheta }=\underset{\_}{E}=vec(\underset{\_}{E}(0),\dots ,\underset{\_}{E}(N-1)) and vec(\underset{\_}{\vartheta }(0),\dots ,\underset{\_}{\vartheta }(N-1))$.

The remaining component m in (38) is the estimated value of $\underset{\_}{\widehat{\mu }}$ given in (39), including input uncertainty ${\underset{\_}{\tau }}_d$ and model bias, i.e.,

$$\underset{\_}{\mu }={\underset{\_}{\tau }}_d+\left[I_n-H(\underset{\_}{q},\underset{\_}{\theta })\right]\ddot{\underset{\_}{q}}-C(\underset{\_}{q},\dot{\underset{\_}{q}},\underset{\_}{\theta })\dot{\underset{\_}{q}}-g(\underset{\_}{q},\underset{\_}{\theta })-F(\underset{\_}{\theta })\dot{\underset{\_}{q}}$$

(39)

at time $\tau =kT+n{t}_s$. The inner loop controller determines it.

To facilitate installation, the following Algorithm 3 summarizes the calculations for implementing the controller (16) proposed above.

Algorithm 3: The structure of the second iterative learning controller with model-free determination of online learning parameters for an industrial robot.
1	Choose two matrices $A_1$, $A_2$, in (6), which become Hurwitz and a sufficiently small constant $0<t_s<<1$. Calculate $N=T/t_s$. Determine $D_1^T=\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$T_s$}\right.;D_0^T=-D_1^T;$ Choose learning $\underset{\_}{\hat{\mu }}$ and tracking error ${\underset{\_}{E}}_0$. Assign the robot’s initial state and initial output to the outer loop controller (iterative learning controller) $\underset{¯}{\vartheta }\left(\mathrm{n}\right)=\underset{¯}{R}\left(\mathrm{n}\right), \mathrm{n}=0,1,\dots ,\mathrm{S}-1 \mathrm{and} \underset{\_}{\mathrm{Z}}=\underset{\_}{0}$
2	while keeping the controls in place
3	for $n=0,1,\dots ,S-1$ do
4	Forward ${\underset{\_}{u}}_{dk}=A_1\underset{\_}{q}-A_2\dot{\underset{\_}{q}}-\widehat{\underset{\_}{\eta }}$ to robot for a while of $t_s$.
	Measure $\underset{¯}{Y}(n)= \underset{¯}{q}$ and $\underset{¯}{X}=vec(\underset{\_}{E},\dot{\underset{\_}{E}})$.
	Assess ${\underset{\_}{E}}_k\left(n\right)=\underset{¯}{R}\left(n\right)-\underset{¯}{Y}\left(n\right)$.
5	Calculate ${\widehat{\underset{\_}{\mu }}}_k\leftarrow (\Theta ,I)\left[\underset{¯}{z}{D}_0^T+\underset{¯}{X}{D}_1^T-A\underset{¯}{X}\right]-(\underset{¯}{\vartheta }(i)-\widehat{\underset{\_}{\mu }})$.
	$\mathit{Set} \underset{¯}{z}\leftarrow \underset{¯}{X}$
6	end for
7	Assemble $\underset{¯}{\vartheta }=vec(\underset{¯}{\vartheta }(0),\dots ,\underset{¯}{\vartheta }(S-1)) and \underset{\_}{E}=vec(\underset{\_}{E}(0),\dots ,\underset{\_}{E}(S-1))$.
8	Calculate $K={\underset{\_}{a}}_k\underset{0\le Z\le 1}{.\mathit{arg}\mathit{min}}‖Z{\underset{\_}{E}}_k\left(1-Z\right){\underset{\_}{E}}_{k-1}‖$ or $K={\underset{\_}{a}}_k\underset{0\le Z\le 1}{.\mathit{arg}\mathit{min}}‖{\underset{¯}{E}}_k\left(1-Z\right)+Z{\underset{\_}{E}}_{k-1}‖$
	and $\underset{\_}{\vartheta }\leftarrow \underset{\_}{\vartheta }+K\underset{¯}{E};{\underset{¯}{E}}_0\leftarrow \underset{¯}{E};$
9	end while

6.3. Applied to Robot Control

Using the following information, assess and confirm the effectiveness of the control algorithm for a two-DOF planar robot:

Friction components:

$$\begin{array}{l}F(\underset{\_}{q},\underset{\_}{\dot{q}},\underset{\_}{\theta })\underset{\_}{\dot{q}}=\left(\begin{array}{c}{\theta }_{11}{f}_1\\ {\theta }_{21}{f}_2\end{array}\right);\\ f_1={\theta }_{11}{\dot{q}}_1+{\theta }_{12}{q}_{1}tanh{\dot{q}}_1;\\ f_2={\theta }_{21}{\dot{q}}_2+{\theta }_{22}{q}_{2}tanh{\dot{q}}_2.\end{array}$$

With input noise assumed as follows:

$${\underset{\_}{\tau }}_d(t)=\left(\begin{array}{c}{\theta }_{31}\sin (2.3\pi t)\\ {\theta }_{32}\sin (0.85\pi t)\end{array}\right)$$

Instead of using the controller design, the parameters (20) and disturbances (21) will be employed to simulate the robot dynamics as in model (1). The parameter ${\theta }_{\mathrm{i}},\mathrm{i}=1÷14$ is chosen randomly.

A robot with two degrees of freedom has the following parameters:

$${\mathrm{l}}_1=0.45;{\mathrm{l}}_2=0.65;\mathrm{g}=9.81$$

The robot is assumed to have a duty cycle $T=10s$. The time to update the control signal (and also to measure the output) is chosen as $t_s=0.02s$. The signal given to the two channels is two periodic functions of period T as follows:

$${\underset{\_}{R}}_1(t)={\underset{\_}{R}}_1(T-t)khi \raisebox{1ex}{$T$}\!\left/ \!\raisebox{-1ex}{$2$}\right.; {\underset{\_}{R}}_2(t)=\sin (\raisebox{1ex}{$2\pi t$}\!\left/ \!\raisebox{-1ex}{$T$}\right.);A_1=\left[\begin{array}{cc}31& 0\\ 0& 12\end{array}\right],A_2=\left[\begin{array}{cc}12& 0\\ 0& 8\end{array}\right]$$

We obtain the simulation results in Figure 10, Figure 11, Figure 12, Figure 13, Figure 14, Figure 15, Figure 16 and Figure 17.

Comment. 

Figure 10 and Figure 11 are the results of two output channels, respectively, after 20 and 300 trials when the controller learns iteratively using the Formulas (36) and (37). For “smart” determination, the learning function parameter is $K_k=diag\left(K_k^{(j)}\right),j=1,2,\dots ,n$, the input/output signal pair index $u_j\mapsto q_j$. This result shows that the controller has provided the required tracking quality, and after 300 trials, the output signal has tracked the password set in both channels.

The quality of tracking the set signal remains the same when Formula (40) is used in place of Formula (39) to compute the intelligent learning parameter in the iterative learning controller. The difference now is only the appropriate change. The results of the two learning parameters $K_k^1,K_k^2$ are shown in Figure 14, Figure 15, Figure 16 and Figure 17.

7. Conclusions

We propose a two-loop control structure for a robot, in which the inner loop is an uncertain function component estimator to compensate for the robot’s input, and the outer circle is an iterative learning controller (Figure 2). The Taylor series analysis method was used to create an inner loop controller. This method does not necessitate the creation of a mathematical model of the robot. For estimating, it simply takes measured signal values from previous systems. We developed two options for installing an iterative learning controller in the outer loop. The difference between these two options lies in the online determination of learning function parameters. The two proposed control circuits were installed into the control algorithm and simulated with the planar robot to evaluate the quality. The correctness of the above-mentioned solutions was demonstrated by theory and validated by simulation. We created a control algorithm based on the concepts above for controlling two items that work in batch processes in industry robots. These control algorithms are all distinguished because they do not (or only very infrequently) employ a mathematical model of the control object. The quality of these control algorithms has also been verified through a 2-DOF robot simulation. However, there are still disadvantages when building methods to determine convergence parameter sets for nonlinear learning functions with known structures. The problem of constrained control by iterative learning still needs to be solved. These will be the following research directions.