Data-Driven Model-Free Adaptive Control Based on Error Minimized Regularized Online Sequential Extreme Learning Machine

: Model-free adaptive control (MFAC) builds a virtual equivalent dynamic linearized model by using a dynamic linearization technique. The virtual equivalent dynamic linearized model contains some time-varying parameters, time-varying parameters usually include high nonlinearity implicitly, and the performance will degrade if the nonlinearity of these time-varying parameters is high. In this paper, ﬁrst, a novel learning algorithm named error minimized regularized online sequential extreme learning machine (EMREOS-ELM) is investigated. Second, EMREOS-ELM is used to estimate those time-varying parameters, a model-free adaptive control method based on EMREOS-ELM is introduced for single-input single-output unknown discrete-time nonlinear systems, and the stability of the proposed algorithm is guaranteed by theoretical analysis. Finally, the proposed algorithm is compared with ﬁve other control algorithms for an unknown discrete-time nonlinear system, and simulation results show that the proposed algorithm can improve the performance of control systems.

MFAC is a class of DDC, and it builds a virtual equivalent dynamic linearized model [24] by using a dynamic linearization technique. The virtual equivalent dynamic linearized model contains some time-varying parameters. In practice, it is often not easy to obtain the resulting time-varying parameters, but they can be estimated by using historical data of control systems. Those time-varying parameters usually include high nonlinearity implicitly, and the performance will degrade if the nonlinearity of the resulting time-varying parameters is too severe [25]. The traditional methods of getting those time-varying parameters are projection algorithm, recursive least squares, and so on. Besides every time-varying parameter in the virtual equivalent dynamic linearized model can be considered as a nonlinear function, and in [25], radial basis function neural network (RBFNN) is used to estimate those time-varying parameters.
Extreme learning machine (ELM) is developed for offline learning and training a single-hiddenlayer feed forward neural networks (SLFNs) [26]. Online sequential extreme learning machine (OS-ELM) [27] is an online learning algorithm, it adjusts the output weights online, besides the input weights and hidden biases are randomly chose. In recent years OS-ELM has gained a large amount of interests and been uesed to estimate unknown parameters of systems [28,29]. Some improved OS-ELM algorithms were introduced by some scholars, such as regularized online sequential extreme learning machine (REOS-ELM) [30], initial-training-free online extreme learning machine (ITF-OELM) [31], etc. Although, the learning speed of OS-ELM is extremely high, it could yield singular and ill-posed problens. To overcome adverse effects, which cuased by the noisy data of control systems, REOS-ELM, which is an improvement of OS-ELM, was investigated. In [30], REOS-ELM was introduced, however the stability of REOS-ELM for unknown discrete-time nonlinear systems was not analysed, and how to select the hidden node number of neural networks is unknown.
In this paper, first, a updating formula which contains dead-zone characteristics for REOS-ELM is introduced. Second, in order to obtain a more compact network structure, error minimized regularized online sequential extreme learning machine (EMREOS-ELM) is investigated, EMREOS-ELM is different from EMOS-ELM [28], and it updates the output weights using the pseudoinverse of a partitioned matrix during the growth of the networks. Finally, a novel model-free adaptive control method based on EMREOS-ELM is proposed. This paper is structured as follows, in Section 2 REOS-ELM is briefly introduced. Dynamic linearization technique and the updating formula which contains dead-zone characteristics for REOS-ELM are introduced in Section 3. The model-free adaptive control method based on EMREOS-ELM and the stability analysis of the proposed method for unknown discrete-time nonlinear systems are stated in Section 4. Section 5 shows simulation experiments. In Section 6 some conclusions are brought.

REOS-ELM
In [32], the universal approximation capability of ELM was analyzed. ELM is developed for offline learning, however training data could arrive one-by-one or chunk-by-chunk; OS-ELM is an online version of ELM, and it is sensitive to noise data; REOS-ELM is an improvement of OS-ELM. In this part REOS-ELM will be introduced.
Suppose that there is an initial training dataset with N 0 training patterns (x j , t j ) ∈ R n × R where x j = [x j1 , x j1 , · · · , x jn ] T . For a single hidden layer neural network with n input nodes and one output node, the jth output corresponding to the jth input pattern is where L denotes the number of hidden nodes, and the subscript i indicates the ith hidden node; G(·) and o j are the active function of hidden nodes and the output of the network, respectively, b i and β i are the hidden bias of the ith hidden node and the output weight connecting the ith hidden layer and the output node, respectively. OS-ELM is a novel online learning algorithm due to its highlighted features, and its highlighted features are that the input weights and hidden biases are randomly chose, and the main goal of training process is to determine the output weights. In order to obtain output weights, they minimize the error function defined by The collected data for training process often contains noise, and the only objective of empirical risk minimization as shown in (2) may lead to poor generalization and over-fitting [30]. REOS-ELM is different from OS-ELM, and REOS-ELM tries to minimize the empirical error and obtain small norm of network weight vector [30]. In [30] the followin cost function was considered, and the cost function is where λ is a regularization factor. The solution of β 0 is and where I L is an identity matrix, the size of I L is L, T 0 = t 1 , · · · , t N 0 T , and When the kth chunk of data is recerived The weight updating algorithm, which is used in REOS-ELM, takes a similar form to recursive least squares (RLS) algorithm, and and where I N k is an identity matrix, and the size of I N k is N k ; T k and H k are the target of kth arriving training data and the hidden layer outputs for the kth arriving training data, respectively.

Dynamic Linearization Technique
MFAC builds a virtual equivalent dynamic linearized model by using a dynamic linearization technique.
The following unknown discrete-time nonlinear system is considered where f (·) represents an unknown nonlinear function, and here L y and L u indicate the orders of the system input and the system output, respectively.
To make further study, the following assumptions are used. Assumption 1. The system (10) is observable and controllable.
Assumption 2. The f (·) is a smooth nonlinear function, and the partial derivative of f (·) with respect to u k is continuous.

Assumption 3.
Suppose that for all k ∈ 0, 1, · · · , T, and if u k = 0, then system (10) satisfies the generalized Lipschitz condition along the iteration axis, that is where y k+1 = y k+1 − y k , u k = u k − u k−1 , L b is a constant, and L b > 0.

Remark 1.
This assumption means that bounded change of the input can not cause unbounded change of the system output, which can be guaranteed by many industrial processes [25].

Assumption 4.
For the unknown discrete-time nonlinear system (10), satisfying Assumptions (Remarks 1-3), there is β * , when u k = 0, the system (10) can be rewritten as where sup| k | ≤ , and is a given upper bound; w i , b i are randomly generated, then they kept in constant.

The Updating Formula Based on Dead-Zone Characteristics
The discrete time nonlinear system (10) is a class of special system where the training data comes one by one, and N k = 1.
The updating formula, which contains dead-zone characteristics, for REOS-ELM is where and where the scalar σ k > 0, then P k is related to P k−1 via and Proof. It is a direct conclusion of [33].

The MFAC Method Based on EMREOS-ELM
In this section, we will introduce the MFAC method based on EMREOS-ELM, and the MFAC method based on EMREOS-ELM is performed in three main phases, and they are parameter initialization, parameter learning and the adjustment of network structure. EMREOS-ELM is an online learning algorithm and it can train a single hidden layer neural network, besides it can adjust the network structure; When the system is initialized or the network structure is adjusted, EMREOS-ELM need much training data, which is used for the training network structure. In this paper, the size of those training data is 200, than is, K m = 200 I m = 200; The maximum number of hidden nodes and the minimum number of hidden nodes are usually selected based on the complexity of unknown discrete-time nonlinear systems. Figure 1 illustrates the process of the proposed algorithm. Step

Initialization Phase
1. u 1 and u 2 are two random values, |u 1 | ≤ 1 |u 2 | ≤ 1, and set k = 3. 2. Set M = 100, L m = 20, and L 1 = L 2 = 8. L m denotes the maximum number of hidden nodes, and L 1 and L 2 denote the minimum number of hidden nodes. 3. Measure the output y 2 and y 3 of system (10). 4. Assign random parameters of hidden nodes (w i , b i ) where i = 1, · · · , L 2 , λ = 0.1. 5. Using the first sample data x 1 = [y 2 , u 1 ], initialize P 1 andβ 2 , and when where E V and E m are two values, and they are important for adjusting network structure; The appropriate E V and E m can improve the tracking effect of systems, and too small E V or too big E m make EMREOS-ELM become invalid; We can choose those values E V and E m based on performance requirements of systems. 4. When the tracking error does not meet the requirements of systems, network structure will be adjusted. The core of the proposed EMREOS-ELM is the adjustment of network structure. if E k ≥ E m ∧ L k ≤ L m ∧ I ≥ I m ∧ k ≥ K m then execute II; or I. (I) Using the kth training data x k = [y k , u k−1 ] and e * k+1 , update the output weights where and 5. k ⇐ k + 1, L k = L k−1 , I ⇐ I + 1, and go to Section 4.2.
(II) Go to Section 4.3.

The Adjustment of Network Structure
1. Set L k = L k−1 + 1, and assign random parameters of the L k th hidden node (w L k , b L k ). 2. When network structure is adjusted, it is equivalent to add a new column to H k , and Then

The pseudo inverse of the new (H
and 4. Calculateβ k+1 , andβ 5. Initialize P k and where I L k denotes L k × L k identify matrix. 6. Set u k as a random number, and |u k | ≤ 1. 7. Set k ⇐ k + 1, I = 1, and go to Section 4.2.

Stability Analysis
For EMREOS-ELM, the adjustment of network structure and online learning are independent of each other, when the structure of the network is adjusted, online learning is not executed in this period. If the structure, i.e., the hidden node number is settled to a fixed value, EMREOS-ELM will be changed into REOS-ELM. Thus, the stability analysis of the model-free adaptive control method based on EMREOS-ELM can be similarly established the stability analysis of the model-free adaptive control method based on REOS-ELM. In this section, the stability of the model-free adaptive control method based on REOS-ELM is analyzed. Theorem 2. For the system (10), if the updating formulas described by (16) and (17) are adopted, the following results can be obtained.
, and λ max (P −1 0 ) and λ min (P −1 0 ) are the maximum eigenvalue and the minimum eigenvalue of the matrix P −1 0 , respectively.

Equation (43) can be obtained from Assumption 4 and (42), and
Introducing Lyapunov candidate as V k+1 =β T k+1 P −1 kβ k+1 , taking (20) into it, hence, we can obtain Substitute (43) in the above equation then and V k is nonnegative. Thenβ The following inequalities can be obtained from the matrix inversion lemma, and this implies This establishes part (1). Summing both sides of (47) from 1 to N with V k being nonnegative, we have and we immediately get (2) holds, then We also immediately have Because the activation function G(w i , b i , x j ) is bounded, e * k+1 is bounded. Because

Analysis of Experimental Results
In this paper, EMREOS-ELM is investigated, and EMREOS-ELM is adopted for estimating the value ψ k ; EMREOS-ELM is an online learning algorithm and can train a single hidden layer neural network; besides it can adjust the network structure. In order to further show the effectiveness of the MFAC method based on EMREOS-ELM, the MFAC method based on another online learning algorithm is compared, and the online learning algorithm adjusts network structure using an incremental form. In order to describe this online learning algorithm easily, in this paper, this online learning algorithm is named as IREOS-ELM. The MFAC method based on IREOS-ELM also has three phases, parameter initialization, parameter learning and the adjustment of network structure. During the adjustment of network structure, all input weights and all hidden biases are reinitialized, and the network is retrained by using the k − 1th training data. The MFAC method based on EMREOS-ELM is different from the MFAC method based on IREOS-ELM, it only initializes the input weight of the increased hidden node and the hidden bias of the increased hidden node, and it updates the output weights using the pseudoinverse of a partitioned matrix.
In this part, a simulation experiment is did, besides the MFAC algorithm based on EMREOS-ELM is compared with the MFAC algorithm based on RLS ( In this numerical simulation experiment, the following single-input single-output unknown discrete-time nonlinear system is considered, and The discrete-time nonlinear system (55) comes from [35]. The reference signal is where T s is the sampling time, and T s = 0.01. The parameters of MFAC algorithm based on RLS are ρ k = 1, λ 1 = 0.01, and a 0 = 0.95. The activation function of EMREOS-ELM is sigmoid, λ = 0.1, and = 0.002. Figures 2-6 show the main results of this numerical simulation experiment. Figure 2 plots the tracking curves, and in Figure 3, the curve denotes the tracking error change trend. Figure 4 shows the change trends of E k and L k . Figure 4 indicates that the final value of L k is 9, when L k = 9, the tracking error satisfies E k ≤ E m , and the result of Figure 4 shows that EMEROS-ELM is effective in obtaining a compact network structure. Figure 5 indicates the change curve ofψ k ,ψ k = H kβk , andψ k is the estimated value of ψ k . In Section 4.4, the stability analysis of the MFAC algorithm based on EMREOS-ELM for unknown discrete-time nonlinear was stated, and we get the inequality (47). In order to verify the inequality (47), a new variable as (57) Figure 6 shows the change curve of of ∆v k , and it indicates that ∆v k ≤ 0.   Besides the error change curves of e (1) k , e k , e k and e In order to further show the effectiveness of algorithms, the integral square error (ISE) index of predicted output is introduced, and Twenty groups of simulation experiments were done, and the ISE of those six control algorithms are shown in Table 1. The final hidden node numbers of simulation experiments are showed in Table 2, and Table 2 shows that the MFAC method based on EMREOS-ELM is effective in adjusting network structure. The average values of ISE indicate that compared with five other control algorithms, the proposed algorithm has an improvement in the performance of control systems.  The MFAC algorithm 9 9 10 11 10 14 9 9 10 10 10 10 9 11 10 11 11 11 10 9 based on EMREOS-ELM The MFAC algorithm 10 11 10 9 12 11 11 11 10 10 12 10 9 11 10 14 11 9 9 10 based on IREOS-ELM

Conclusions
In order to analyse the stability of the MFAC algorithm based on REOS-ELM for unknown discrete-time nonlinear systems, an updating formula, which contains dead-zone characteristics, is introduced, and EMREOS-EM is investigated for the purpose of getting a more compact network structure and improving the performances of control systems. The proposed MFAC method based on EMREOS-ELM is compared with five other algorithms, and the simulation results show that the proposed algorithm has an improvement in the performance of control systems.
In this paper, the MFAC method based on EMREOS-ELM is introduced for single-input single-output unknown discrete-time nonlinear systems. In the future, we plan to do some work for multiple-input, multiple-output unknown discrete-time nonlinear systems based on the proposed algorithm, and in order to improve the performance of robot control systems, we will also apply this algorithm to robot control systems which are complex nonlinear systems.

Conflicts of Interest:
The authors declare no conflict of interest.