Next Article in Journal
Esterification of Free Fatty Acids with Glycerol within the Biodiesel Production Framework
Previous Article in Journal / Special Issue
Data Science in the Chemical Engineering Curriculum
 
 
Article

Extremum Seeking Control for Discrete-Time with Quantized and Saturated Actuators

1
Department of Chemical Engineering, Queens’ University, Kingston, ON K7L3N6, Canada
2
iRobot Corp., 8 Crosby Rd, Bedford, MA 01730, USA
*
Author to whom correspondence should be addressed.
Processes 2019, 7(11), 831; https://doi.org/10.3390/pr7110831
Received: 27 September 2019 / Revised: 31 October 2019 / Accepted: 4 November 2019 / Published: 8 November 2019
(This article belongs to the Special Issue Process Systems Engineering à la Canada)

Abstract

This paper proposes an extremum-seeking controller (ESC) design for a class of discrete-time nonlinear control systems subject to input constraints or quantized inputs. The proposed method implements a proportional-integral ESC design along with a discrete-time anti-windup mechanism. The anti-windup enforces input saturation while preserving the input dither signal. The technique incorporates a mechanism for adjusting the amplitude of the extremum seeking control dither signal. This mechanism ensures that any violation of constraints due to the dither signal is removed while maintaining the probing signal active. An amplitude update routine is also proposed. The amplitude update is coupled with a saturation bias estimation algorithm that correctly accounts for the inherent bias associated with systems operated at or near saturation conditions. The amplitude update is designed to remove the dither signal when the system approaches the optimum. It also ensures that a lower bound of the amplitude is enforced to guarantee that excitation conditions are maintained.
Keywords: real-time optimization; constrained systems; extremum-seeking control real-time optimization; constrained systems; extremum-seeking control

1. Introduction

Extremum-seeking control (ESC) has grown to become the leading approach to solve real-time optimization problems [1]. Following the seminal work of Krstic and coworkers ([2,3,4,5,6,7]), this general and practically relevant control approach is equipped with an established and well understood theoretical framework, as highlighted in the proof of Krstic and Wang [2]. The standard perturbation ESC algorithm has been generalized in various forms to handle output and input constraints. ESC in the presence of constraints has been investigated in various form in the literature. Constrained ESC was first considered in [8] where a trajectory tracking approach was used to address the constrained ESC problem for a class of nonlinear systems with parametric uncertainties. In this approach, a barrier function or interior-point method was used to enforce constraints and feasibility of the closed-loop trajectories. A similar model-free extremum-seeking approach was presented in [9]. A Lagrangian, saddle-point, ESC technique is proposed in [10], and a similar approach is proposed in [11] to handle a class of stochastic control systems. In [12], a Shahshahani gradient approach was proposed. This techniques allows one to handle ESC problems subject to linear constraints by a simple reformulation of the gradient descent dynamics. In contrast to Lagrangian-based techniques, the main advantage of the Shahshahani gradient and barrier function approaches is the ability to preserve feasibility throughout the optimization.
For ESC in the presence of input constraints, a variety of techniques have been proposed. In [13] and [14], a projection algorithm is used to solve ESC problems subject to constraints in the decision variables. In [15], a comprehensive study of anti-windup mechanisms for standard ESC is presented. The approach draws a parallel between penalty (barrier) function methods and an anti-windup mechanism. A proof of convergence of the ESC in the presence of constraints is provided. In [16], a simplistic windup algorithm for a standard ESC technique is implemented experimentally for the real-time optimization of airside economizers.
The vast majority of existing results on ESC have focussed on continuous-time systems, as is the case for the existing approaches for ESC in the presence of constraints. Although discrete-time systems can be treated in an essentially similar fashion, the application of gradient descent in a discrete-time setting requires some care. A discrete-time version of the standard ESC loop was studied in [4,6] where convergence results similar to continuous time systems are obtained. A similar algorithm was also proposed in [17] for the tuning of PID controllers in unknown dynamical systems using ESC. Discrete-time ESC subject to stochastic perturbations is studied in [18]. The use of approximate parameterizations of the unknown cost function using quadratic functions was recently proposed in [19]. An alternative ESC-like method was proposed in [20]. In this study, a trajectory-based technique is used to analyze the properties of nonlinear optimization algorithms as dynamical systems. It is shown that properties of the nonlinear-optimization algorithms are suitable to assess the convergence of certain classes of ESC applied in a sampled-data approach. This method was recently studied in the context of global sampling methods in [21] where trajectory-based properties of nonlinear optimization methods are used to establish robust convergence. The main objective with the trajectory-based techniques is to analyze the properties of optimization algorithms assuming that they can converge to the true optimum using only the measurement of the objective function and possibly the constraints.
This paper proposes an extremum-seeking controller (ESC) design for a class of discrete-time nonlinear control systems subject to input constraints. Two actuation scenarios are considered. In the first scenario, we consider the ESC in the presence of saturated inputs. The proposed method generalizes the discrete-time proportional-integral ESC proposed in [22] to incorporate a new discrete-time anti-windup mechanism for ESC. One contribution of this study is the development of a saturation bias estimation mechanism that can be used to remove the impact of dither on or near the saturation level. This mechanism ensures that violation of the constraints due to the dither signal are removed without the introduction of a gradient estimation bias. Moreover, it allows the system to remain responsive to changes in the system despite operating on or very close to the saturation level. An amplitude update routine is also proposed as a discrete-time generalization of the method proposed in [23]. The amplitude update is coupled with the saturation bias estimation algorithm to account for the inherent bias associated with systems operated at or near saturation conditions.
In the second scenario, we adapt the application of the anti-reset windup strategy and the saturation bias estimation routine to handle systems with quantized actuators. We focus on ESC design for systems with “on/off” actuators. Since the excitation signal is limited to the on or off position, the application of the saturation bias estimation is able to remove the impact of the dither to allow the ESC system to converge to the correct position. Such actuators have not been treated in the literature.
The paper is organized as follows. A description of the ESC problem along with the key assumptions are given in Section 2. The proportional-integral ESC controller are presented in Section 3. The anti-windup mechanism and amplitude adjustment mechanism are described in Section 4. The design of ESC for quantized actuators are presented in Section 5. Simulation examples are presented in Section 6 followed by brief conclusions and proposed future work are in Section 7.

2. Problem Description

We consider a class of nonlinear systems of the form:
x k + 1 = x k + f ( x k ) + g ( x k ) u k
y k = h ( x k )
where x k R n is the vector of state variables at time k, u k is the input variable at time k taking values in U R and y k R is the objective function at step k, to be minimized. It is assumed that f ( x k ) and g ( x k ) are smooth vector valued functions and that h ( x k ) is a unknown smooth function.
The objective is to stabilize the system at the equilibrium conditions, x * and u * , that achieves the minimum value of y ( = h ( x * ) ) subject to saturation of the input. The input variable, u k , is required to lie in the interval U = [ u , u + ] . At equilibrium, the state variables are given by the map x = π ( u ) that solves the following equation:
f ( π ( u ) ) + g ( π ( u ) ) u = 0 .
The corresponding equilibrium cost function is given by:
y = h ( π ( u ) ) = ( u )
The steady-state optimization problem is to find the minimizer u * of y = ( u * ) subject to u * U . The set D ( u ) represents a neighbourhood of the equilibrium x = π ( u ) .
The steady-state cost function, ( u ) , meets the following assumptions.
Assumption 1.
The nonlinear system is such that
x h ( π ( u ) ) g ( π ( u ) ) ( u u * ) 0
u U .
Assumption 2.
The cost h ( x ) is such that
  • h ( x * ) x = 0
  • 2 h ( x ) x x T > β I , x R n
where β is a strictly positive constant.
Following [22], we write the cost dynamics as:
y k + 1 y k = Ψ 0 , k ( x k , u ^ k ) + Ψ 1 , k ( x k , u k , u ^ k ) ( u k u ^ k ) .
where Ψ 0 , k ( x k , u ^ k ) = h ( α ( x k , u ^ k ) ) h ( x k ) ,
Ψ 1 , k ( x k , u k , u ^ k ) = ( h ( α ( x k ) ) g ( x k ) + 1 2 ( u k u ^ k ) g ( x k ) 2 h ( y ˜ k ) g ( x k ) .
and y ¯ k = α ( x k ) + θ g ( x k ) ( u k u ^ k ) for θ ( 0 , 1 ) .
The following assumptions are required to ensure the stability of the closed-loop system.
Assumption 3.
There exists a function u k = α F ( x k , u ^ k ) that solves the identity:
α F ( x k , u ^ k ) = Sat ( k g Ψ 1 , k ( x k , α F ( x k , u ^ k ) , u ^ k ) + u ^ k ) .
This assumption states that the feedback:
u k = Sat ( k g Ψ 1 , k ( x k , u k , u ^ k ) + u ^ k ) .
is well defined.
The following stabilizability condition for the nonlinear system subject to input saturation is also required.
Assumption 4.
There exists a positive definite function ( x ) that satisfies the following inequalities:
β 1 x π ( u ^ ) 2 V ( x ) β 2 x π ( u ^ ) 2
with positive constants β 1 and β 2 . For all x Ω β = x R n V ( x ) β D ( u ^ ) there exists a positive constant k g * such that:
V ( α ¯ ( x k ) ) V ( x k ) α e x k π ( u ^ ) 2
with positive constant α e and u ^ U and
α ¯ ( x k ) = x k + f ( x k ) k g * g ( x k ) Ψ 1 , k ( x k , α F , u ^ ) + g ( x k ) u ^ .

3. Proportional-Integral Discrete-Time ESC

In this section, we present the basic PI-ESC controller and the proposed parameter (gradient) estimation algorithm.

PI-ESC Controller

From (4), the objective function dynamics are parameterized as follows:
y k + 1 = y k + θ 0 , k + θ 1 , k T ( u k u ^ k )
where the time-varying parameters θ 0 , k and θ 1 , k are identified with θ 0 , k = Ψ 0 , k and θ 1 , k = Ψ 1 , k T .
The unknown parameters θ 0 , k and θ 1 , k must be estimated using a parameter estimation approach described in the next subsection. We let θ ^ 0 , k and θ ^ 1 , k denote the estimates of θ 0 , k and θ 1 , k , respectively. The proportional-integral extremum-seeking controller is given by:
u k = Sat k g θ ^ 1 , k + u ^ k + d k u ^ k + 1 = u ^ k k g τ I θ ^ 1 , k ,
where k g and τ I are positive constants to be assigned. The term d k is a dither signal used to provide a sufficiently excited signal in closed-loop. The dither signal is bounded such that d k D where D a known positive constant denotes the amplitude of the dither.
In this study, the estimation of the parameter varying parameters θ 0 , k and θ 1 , k is performed using the estimation routine described in [22]. The estimation routine implements a modified recursive least squares approach for the estimation of time-varying parameters. It is described briefly in Appendix A.
The stability properties of the ESC considered are summarized in Appendix B.

4. Input Constrained ESC

In this section, we present the main contribution of this study. The proposed technique incorporates three mechanisms for the solution of ESC problems in the presence of input constraints. The first mechanism consists of a standard anti-windup mechanism that exploits the proportional integral formulation of the ESC considered. The second mechanism proposes a dither bias estimation routine that eliminates the presence of biases introduced when the dither signal input pushes the input to its saturation limit. The third mechanism is a dither amplitude update that is used to remove the dither signal when the system has converged to its optimal value, or its optimal saturation limit.

4.1. Anti-Windup Mechanism

In this paper, we propose the use of an anti-windup mechanism for the proportional integral ESC controller (5). A block diagram of the mechanism is shown in Figure 1.
In Figure 1, C ( z ) represents the discrete-time transfer function of the proportional-integral controller
C ( z ) = k g + k g τ I 1 z 1 .
The mechanism places the dither addition after the anti-windup loop but before the final saturation. This mechanism guarantees that the dither signal is not removed when the system operates at the saturation limits. It also guarantees that the dithered input does not violate the input constraints.
The operator Sat ( · ) denotes the saturation function:
Sat ( u ) = u i f u u , u i f u < u u + , u + i f u u + .
The proposed dynamics of the anti-windup mechanism is given by:
z k + 1 = z k 1 τ I z k 1 k g τ I Sat ( k g θ ^ 1 , k k g z k ) u k = Sat ( Sat ( k g θ ^ 1 , k k g z k ) + d k ) .
The anti windup loop is such that, in the absence of saturation, the control law reduces to the proportional integral law and the control law becomes:
u k = Sat ( k g θ ^ 1 , k k g z ^ k + d k ) z ^ k + 1 = z ^ k + 1 τ I θ ^ 1 , k .
Please note that the Sat ( · ) remains in the control loop to ensure that the added dither signal does not cause input constraint violation.
One of the difficulties associated with such an approach is that the saturation creates a bias in the dither signal. This is problematic in cases where the optimum input lies close to or on the saturation limit. This bias in the dither signal can lead to a bias in the estimation of the parameters. As result, the value of the parameter θ ^ 1 , k does not converge to zero even when the true value θ 1 , k vanishes.
It is, therefore, imperative to provide a mechanism to introduce the dither signal that prevents the estimation bias. We consider two mechanisms in this study.

4.2. Saturation Bias Estimation

Let us consider the case in which the optimum occurs on the upper saturation level u ¯ . At the optimum, the control signal for the ESC is given by
u = Sat ( u ¯ + d k ) .
The filter (or regressor) vector yields:
w k + 1 T = w k T K w k T + 1 , Sat ( u ¯ + d k )
One of the key properties of the dither signal is that 1 N i = k k + N 1 d i = 0 . In the absence of input saturation, the average regressor is such that
lim k 1 N i = k k + N 1 w i = 1 K [ 1 , u ¯ ] T .
In the presence of a bias in the dither signal, the regressor vector does not average to the correct value. As a result, the parameter estimation of θ 1 , k is subject to a bias, and the system would converge to an erroneous optimum state and input.
In this section, we design an update mechanism that accounts for this saturation bias. This is achieve by introducing a signal δ k in the control, u k , which is such that the average input is unbiased. That is, for a fixed value of the input u k = u ¯ , the following property is achieved:
1 N i = k k + N 1 Sat ( u ¯ + d i + δ k ) = u ¯ .
We first define the variable
Υ k = ( Sat ( Sat ( k g θ ^ 1 , k k g z ^ k ) + d k + δ k ) Sat ( k g θ ^ 1 , k k g z ^ k ) ) .
The bias estimation update proposed in this study is given by:
δ k + 1 = δ k λ Υ k if u k = u or u k = u + λ δ k otherwise
Proposition 1.
The saturation bias estimate update (10) is such that:
  • For u k ( u , u + ) ,
    lim k δ k = 0
  • For u = u , or u = u + ,
    lim k i = k k + N 1 Sat ( u + d i + δ k ) = u .
Proof. 
For Statement 1, the conclusion is straightforward.
The proof of Statement 2 is as follows. To establish the property (9), we first compute the average in the case where the value of the input is at one its saturation limits. Let us consider the case where the input is at its upper limit, u + . From a set of N samples of the input, assume that there are N 1 samples at which Sat ( u + + d k + δ k ) = u + with the remaining N 2 samples for which Sat ( u + + d k + δ k ) < u + . Let μ 2 ( j ) , j = 1 , , N 2 denote the indices of the samples that are not saturated. As a result, we can decompose the averaged quantity as follows:
i = k k + N 1 Sat ( u + + d i + δ i ) = u + + j = 1 N 2 ( d μ 2 ( j ) + δ μ 2 ( j ) ) = u + + j = 1 N 2 d μ 2 ( j ) + j = 1 N 2 δ μ 2 ( j ) .
Thus if one considers the update (10). Following the above argument, we average both sides by summing over N samples. Let us consider the situation where the input is at its upper saturation limit u + and decompose the overall average into N 1 saturated values and N 2 inputs whose perturbed value is not saturated. This yields
i = k k + N 1 δ i + 1 = i = k k + N 1 δ i λ j = 1 N 1 Sat ( u + + d μ 1 ( j ) + δ μ 1 ( j ) ) Sat ( u + ) λ j = 1 N 2 Sat ( u + + d μ 2 ( j ) + δ μ 2 ( j ) ) Sat ( u + )
This gives the following recursion of sums:
i = k k + N 1 δ i + 1 = i = k k + N 1 δ i λ j = 1 N 2 d μ 2 ( j ) λ j = 1 N 2 δ μ 2 ( j ) .
For every sample from the set of points that are saturated at step k, it follows that δ μ 1 ( j ) + 1 = δ μ 1 ( j ) . As a result, we can write
j = 1 N 2 δ μ 2 ( j ) + 1 = i = 1 N 2 δ μ 2 ( j ) λ j = 1 N 2 d μ 2 ( j ) λ j = 1 N 2 δ μ 2 ( j ) .
Defining the variable
δ ¯ k = j = 1 N 2 δ μ 2 ( j ) ,
we obtain the following recursion:
δ ¯ k + 1 = δ ¯ k λ δ ¯ k λ d ¯ k .
As a result, we see that the average δ ¯ k approaches the negative value of the mean dither d ¯ k . As a result, the bias (9) is completely removed by the saturation bias update (10). This completes the proof of Statement 2. □
In cases where the optimum lies on or close to a saturation, the update (10) would lead to an effective removal of the dither signal. However, in fact, the dither is not removed. It is simply compensated for by the bias estimate δ k . If a disturbances affects the system, moving the optimum inside the saturation, then the dither signal would resume and the ESC would operate in a normal way. As a result, the dither would not be effectively removed.

4.3. Dither Amplitude Update

In this study, we consider a dither signal of the form: d k = a k sin ( ν k ) where ν k can be taken to be a zero-mean Gaussian variable or simply ν k = ω k for some frequency ω . The amplitude of the dither signal a k is obtained using an amplitude update.
Let the upper or lower limit if u k be denoted generically by u ¯ . We first define the signal:
Θ k = θ ^ 1 , k u k u ¯ , θ ^ 0 , k + θ ^ 1 , k u ¯ otherwise .
The proposed amplitude update is given by:
a k + 1 = a k + σ 1 γ 1 2 π tan 1 ( Θ k ) γ 2 λ min Σ k a k
where a 0 0 , σ 1 , γ 1 and γ 2 are tuning parameters. This mechanism confers two actions to adjust the amplitude of the dither signal.
The term γ 1 2 π tan 1 ( Θ k ) decreases the amplitude when the gradient estimate decreases or when the system has reached an equilibrium corresponding to a saturation input level. In [23], a similar amplitude update is proposed. The proposed method complements that approach in two ways. First, we adjust for the situation in which the input has stabilized on a saturation level. In this case, the estimated value of θ 1 , k cannot reach 0. As a result, the approach of [23] using only θ ^ 1 , k would yield a larger value of the amplitude. If the optimization does not lead to a saturated value of the input, the update acts as the update of [23] and reduces the amplitude to a suitable lower level.
Second, the proposed method assigns a minimum value of the amplitude a k . As the proof of stability of the PI-ESC algorithm demonstrates [22], the practical stability of the unknown optimum requires a persistent dither signal with a k > 0 for all k. In practice, setting a k = 0 would prevent the system from responding to possible changes in the changes that may arise from changing conditions. This property of the ESC system was recognized in [23], which required a fixed lower bound for the amplitude. However, the choice of this lower bound can be conservative. The second term in the update (11), γ 2 λ min Σ k , aims to increase the amplitude a k when the smallest eigenvalue of the matrix Σ decreases. This update guarantees a minimum amount of excitation in the system in order to respond to possible process changes.
The action of the amplitude update can be summarized as follows.
Proposition 2.
For σ 1 < 1 , the update (11) is such that a k is bounded and lim k a k approaches γ 2 λ min Σ in a region of an unconstrained optimum θ ^ 1 , k = 0 or on a saturation level of the input θ ^ 0 , k + θ ^ 1 , k u ¯ = 0 .
The combination of the anti-windup (Figure 1) and the amplitude update (11) provides an effective mechanism to minimize the bias of the system arising from the saturation. It also removes the need for the tuning of the amplitude. We demonstrate this in simulations in the next section.

5. ESC for Systems with Quantized Actuators

The three mechanisms proposed in the previous section can be easily adapted to a situation where the actuators of the system are limited to quantized (or on-off) input settings. In this case, we consider an actuator whose on-off action can be implemented using a hysteresis mechanism of the form:
u = Γ ( u , ϵ ) = u if u ( u + u + ) / 2 + ϵ u + if u ( u + u + ) / 2 ϵ
where ϵ > 0 is a small positive constant. The function Γ ( u ) implements the discrete actuator using a hysteresis mechanism.
In this study, we propose a quantized actuator ESC using the mechanism depicted in Figure 2.
As above, we consider the anti-windup ESC given by:
u k = Γ ( k g θ ^ 1 , k k g z ^ k + d k , ϵ ) z ^ k + 1 = z ^ k + 1 τ I θ ^ 1 , k .
Since the ESC only provides quantized control action u k = u + or u k = u , we must consider a saturation bias estimation to eliminate the bias and remove the b presented in Section 4.3. The reason for this is that the update (14) yields the required property of the saturation bias as the system reaches the limits. The proposed bias update is given by:
δ k + 1 = δ k λ Υ k if u k = u or u k = u + λ δ k otherwise
where
Υ k = Γ ( Γ ( k g θ ^ 1 , k k g z ^ k ) + d k + δ k , ϵ ) Γ ( k g θ ^ 1 , k k g z ^ k , ϵ ) .
The amplitude of the dither signal is implemented as in Section 4.3.

6. Simulation

6.1. Anti-Windup PIESC

We consider the application of the PI-ESC approach to the following linear discrete-time system:
x k + 1 = a 1 x k + u k y k = ( x k p 1 ) 2 + q 1
where a 1 = 0.8 ,
p 1 = 3 k < 200 2 200 k < 300 4 300 k < 400 2 k 400 , q 1 = 1 k < 200 2 200 k < 300 5 300 k < 400 2 k 400 .
The input variable u k in constrained to values over the interval [ 0 , 0.6 ] .
We consider the PIESC algorithm with k = 0.1 and τ I = 5 . The estimation routine parameters are set to α = 0.25 , σ = 10 5 and K k = 0.99 . The amplitude is used with γ 1 = 0.1 , γ 2 = 0.01 and σ 1 = 0.1 with an initial condition a 0 = 0 . The dither signal is d k = a k sin ( 2 k ) . The choice of these tuning parameters reflect the tuning guidelines that have been presented in [22]. For the saturation bias update is implemented with λ = 0.05 . This choice guarantees that the bias update responds quickly to changing conditions.
The simulation results are shown in Figure 3 and Figure 4. Figure 3 shows the input and output trajectories for the resulting closed-loop system. It also shows the changes in the amplitude of the dither signal. Figure 4 shows the corresponding parameter estimates.
In this simulation, the location of the optimum is changed. For k [ 0 , 200 ] , the optimum occurs at y * = 1 with u * = 0.6 . This places the minimizer directly on the input constraint. The ESC system identifies the optimum correctly. The amplitude update rule is also able to reduce the amplitude to a suitable minimum level. For k [ 200 , 300 ] , the unknown optimum occurs at y * = 2 and u * = 0.4 . Since the dither amplitude update routine (11) prevented the amplitude from values that are too small, the PIESC is able to respond quickly to the change in conditions.
For the period k [ 300 , 400 ] , the unconstrained minimum occurs at y * = 5 and u * = 0.8 . As required in this case, the PIESC system converges to the saturated value of u k = 0.6 with cost y k = 6 . Finally, for k 400 , the system cannot reach the unconstrained optimum y * = 2 and u * = 0.4 but converges correctly to the lower saturation level of u k = 0 with cost y k = 6 .
Overall, the PIESC with anti-windup mechanism performs effectively. First, the system is able to perform the optimization task in the presence of input saturation. Second, the proposed approach allows the partial removal of the dither signal when the system operates at saturation.

6.2. Quantized Actuator ESC

In this section, we consider the same dynamical system subject to the quantized actuator:
Γ ( u , ϵ ) = 0 if u 0.3 + ϵ 0.6 if u 0.3 ϵ
where ϵ = 0.01 for the purpose of simulation.
The proposed dither signal is given by the sine wave signal:
d k = a k sign ( sin ( ω k ) ) + δ k
where δ k is the bias update and sign ( · ) is the sign function.
The parameters, p 1 and q 1 , are chosen as follows:
p 1 = 3 k < 200 1 200 k < 300 4 300 k < 400 2 k 400 , q 1 = 1 k < 200 2 200 k < 300 5 300 k < 400 2 k 400 .
We consider the PIESC algorithm with k = 0.1 and τ I = 5 . The estimation routine parameters are set to α = 0.25 , σ = 10 5 and K k = 0.99 . The amplitude is used with γ 1 = 0.1 , γ 2 = 0.01 and σ 1 = 0.9 with an initial condition a 0 = 0.5 . We consider the bias update (14) with update parameter λ = 0.95 .
The simulation results are shown in Figure 5. The ESC system is able to respond quickly to the changing conditions. The correct optimal setting of the discrete actuator is correctly identified, u * = 0.6 for 0 t 200 , u * = 0 for 200 < t 300 , u * = 0.6 for 300 < t 400 and u * = 0 for t 400 . The combination of the amplitude update and the bias estimation works very effectively in this case. The systems reintroduces excitation in the system in response to the change in conditions. The resulting excitation introduces a short sequence of discrete switches that vanish once the correct optimum is identified.

7. Conclusions

This study proposed an extremum-seeking control algorithm for systems subject to input saturated or quantized actuators. The approach couples a well known anti-reset windup technique with a saturation bias estimation routine that improves the performance of the ESC near or on the saturation level by removing the impact of the dither signal. An amplitude update is also proposed to further improve the performance of the ESC system.

Author Contributions

The two authors have contributed equally to every aspects of the paper.

Funding

This research was partially funded by the Natural Sciences and Engineering Council of Canada.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A. Projection Operator

The estimation routine consists of the following output predictor dynamics
y ^ k + 1 = y ^ k + θ ^ 0 , k + θ ^ 1 , k T ( u k u ^ k ) + K k e k w k + 1 T ( θ ^ k θ ^ k + 1 )
with auxiliary regressor variable,
w k + 1 = w k + ϕ k K k w k ,
where θ ^ k = [ θ ^ 0 , k , θ ^ 1 , k T ] T is the vector of parameter estimates at time step k given by any update law, K k is a correction factor at time step k, e k = y k y ^ k is the state estimation error at time step k. We let ϕ k = [ 1 , ( u k u ^ k ) T ] T . The variable w k is initiated w 0 = 0 .
The parameter estimation update dynamics are given by:
Σ k + 1 1 = ( α Σ k + σ I ) 1 ( α Σ k + σ I ) 1 w k Q k w k T ( α Σ k + σ I ) 1 ,
where Q k = ( 1 + 1 α w k T ( α w k + σ I ) 1 w k ) 1 , and
θ ^ ¯ k + 1 = P r o j { θ ^ k + ( α Σ k + σ I ) 1 w k Q k ( e k ) , Θ k }
where, Proj, denotes the projection operator.The operator Proj represents an orthogonal projection onto the surface of the uncertainty set applied to the parameter estimate. The parameter uncertainty set is defined by the ball function B ( θ ^ , z θ ^ ) , where θ ^ and z θ ^ are the parameter estimates and uncertainty set radius, respectively.
Following [24], the projection operator is such that
  • θ ^ k + 1 Θ 0
  • θ ˜ ¯ k + 1 T Σ k + 1 θ ˜ ¯ k + 1 θ ˜ k + 1 T Σ k + 1 θ ˜ k + 1
One possible algorithm for the projection algorithm is as follows. Define the upper bound for θ ( = L 1 ) . Let R = Chol ( Σ k + 1 ) denote the Cholesky factor of Σ k + 1 . Then we perform the following:
Algorithm A1:
If θ ^ k + 1 L 1 then
  • Let δ = L 1 θ ^ k + 1 θ ^ k + 1 and z ρ = δ T Σ k + 1 δ ,
  • With ρ = R θ ^ k + 1 define ρ ¯ = ρ z ρ ρ ,
  • Set θ ^ ¯ k + 1 = R 1 ρ ¯ .
Otherwise,
  • Set θ ^ ¯ k + 1 = θ ^ k + 1 .

Appendix B. Stability Properties

Assumption A1.
There exists constants β > 0 , β T > 0 and T > 0 such that
β T I < 1 T i = k k + T 1 w i w i T β I , k > T .
This requirement is a standard persistency of excitation condition that can be found in most references on adaptive control and adaptive estimation. The reader is referred to [24] for more details.
The following theorem establishes the stability properties of the ESC system in the presence of the input saturation. The result is a direct consequence of the results of [22].
Theorem A1.
Consider the nonlinear discrete-time system (1) with cost function (2), the extremum seeking controller (5) and parameter estimation scheme (A1)–(A4). Let Assumptions 1–4 and A1 be fulfilled. Then, for any x 0 Ω β , there exists positive constants α, K, k g and τ I such that for every τ I τ I * , the states x k and input u k of the closed-loop system enter a neighbourhood of the unknown optimum ( x * , u * ) .
Proof. 
The proof follows the result presented in [22] subjected to a saturated controller that, by Assumption 4, locally asymptotically stabilizes the unknown optimum. The key element of the result is the existence of suitable tuning parameters that achieves stability of the optimum. □
The proof presented in [22] establishes that the stability of the closed-loop ESC relies on the bounded of the covariance matrix. Assuming that Σ 0 = α 0 I , it is shown that:
α T 1 α β T I + σ I Σ k α 0 I + 1 1 α T ( β + σ ) I .
In this study, the application of parameter projection algorithm and the saturation of the input guarantees that the upper bounds on (A5) and Σ k are met. The lower bound on Σ k can only be met by ensuring that the lower bound on (A5) is met. We propose a dither amplitude update that can minimize the impact of the dither signal on the closed-loop system close to the optimum. The amplitude also monitors the lower bound of Σ k to guarantee that the closed-loop maintains a sufficient amount of excitation.

References

  1. Tan, Y.; Moase, W.; Manzie, C.; Nesic, D.; Mareels, I. Extremum seeking from 1922 to 2010. In Proceedings of the 29th Chinese Control Conference (CCC), Beijing, China, 29–31 July 2010; pp. 14–26. [Google Scholar]
  2. Krstic, M.; Wang, H. Stability of Extremum Seeking Feedback for General Dynamic Systems. Automatica 2000, 36, 595–601. [Google Scholar] [CrossRef]
  3. Krstic, M. Performance Improvement and Limitation in Extremum Seeking Control. Syst. Control Lett. 2000, 39, 313–326. [Google Scholar] [CrossRef]
  4. Ariyur, K.B.; Krstic, M. Real-Time Optimization by Extremum-Seeking Control; John Wiley and Sons Inc.: Hoboken, NJ, USA, 2003. [Google Scholar]
  5. Ariyur, K.; Krstic, M. Analysis and design of multivariable extremum seeking. In Proceedings of the American Control Conference, Anchorage, AK, USA, 8–10 May 2002; pp. 2903–2908. [Google Scholar]
  6. Choi, J.Y.; Krstic, M.; Ariyur, K.; Lee, J. Extremum seeking control for discrete-time systems. IEEE Trans. Autom. Control 2002, 47, 318–323. [Google Scholar] [CrossRef]
  7. Wang, H.; Krstic, M.; Bastin, G. Optimizing Bioreactors by Extremum Seeking. Int. J. Adapt. Control Signal Process. 1999, 13, 651–669. [Google Scholar] [CrossRef]
  8. DeHaan, D.; Guay, M. Extremum-seeking control of state-constrained nonlinear systems. Automatica 2005, 41, 1567–1574. [Google Scholar] [CrossRef]
  9. Guay, M.; Moshksar, E.; Dochain, D. A constrained extremum-seeking control approach. Int. J. Robust Nonlinear Control 2015, 25, 3132–3153. [Google Scholar] [CrossRef]
  10. Durr, H.B.; Zeng, C.; Ebenbauer, C. Saddle Point Seeking for Convex Optimization Problems. In Proceedings of the IFAC Symposium on Nonlinear Control Systems, Toulouse, France, 4–6 September 2013; pp. 540–545. [Google Scholar]
  11. Coito, F.; Lemos, J.; Alves, S. Stochastic Extremum Seeking in the Presence of Constraints. In Proceedings of the IFAC World Congress, Prague, Czech Republic, 3–8 July 2005; Volume 16, pp. 266–271. [Google Scholar]
  12. Poveda, J.; Quijano, N. A Shahshahani Gradient based extremum seeking scheme. In Proceedings of the 2012 IEEE 51st Annual CDC, Maui, HI, USA, 10–13 December 2012; pp. 5104–5109. [Google Scholar] [CrossRef]
  13. Mills, G.; Krstic, M. Constrained extremum seeking in 1 dimension. In Proceedings of the 53rd IEEE Conference on Decision and Control, Los Angeles, CA, USA, 15–17 December 2014; pp. 2654–2659. [Google Scholar] [CrossRef]
  14. Mills, G.; Krstic, M. Gradient based projection method for constrained optimization. In Proceedings of the 52nd IEEE Conference on Decision and Control, Florence, Italy, 10–13 December 2013; pp. 2966–2971. [Google Scholar] [CrossRef]
  15. Tan, Y.; Li, Y.; Mareels, I.M.Y. Extremum Seeking for Constrained Inputs. IEEE Trans. Autom. Control 2013, 58, 2405–2410. [Google Scholar] [CrossRef]
  16. Mu, B.; Li, Y.; House, J.M.; Salsbury, T.I. Experimental evaluation of anti-windup extremum seeking control for airside economizers. Control Eng. Pract. 2016, 50, 37–47. [Google Scholar] [CrossRef]
  17. Killingsworth, N.J.; Krstic, M. PID tuning using extremum seeking: Online, model-free performance optimization. IEEE Control Syst. Mag. 2006, 26, 70–79. [Google Scholar]
  18. Manzie, C.; Krstic, M. Extremum seeking with stochastic perturbations. IEEE Trans. Autom. Control 2009, 54, 580–585. [Google Scholar] [CrossRef]
  19. Ryan, J.J.; Speyer, J.L. Peak-seeking control using gradient and hessian estimates. In Proceedings of the American Control Conference (ACC), Baltimore, MD, USA, 30 June–2 July 2010; pp. 611–616. [Google Scholar]
  20. Teel, A.; Popovic, D. Solving smooth and nonsmooth multivariable extremum seeking problems by the methods of nonlinear programming. In Proceedings of the 2001 American Control Conference, Arlington, VA, USA, 25–27 June 2001; Volume 3, pp. 2394–2399. [Google Scholar]
  21. Nesic, D.; Nguyen, T.; Tan, Y.; Manzie, C. A non-gradient approach to global extremum seeking: An adaptation of the Shubert algorithm. Automatica 2013, 49, 809–815. [Google Scholar] [CrossRef]
  22. Guay, M.; Burns, D.J. A proportional integral extremum-seeking control approach for discrete-time nonlinear systems. Int. J. Control 2016, 90, 1543–1554. [Google Scholar] [CrossRef]
  23. Atta, K.T.; Hostettler, R.; Birk, W.; Johansson, A. Phasor extremum seeking control with adaptive perturbation amplitude. In Proceedings of the 2016 IEEE 55th Conference on Decision and Control (CDC), Las Vegas, NV, USA, 12–14 December 2016; pp. 7069–7074. [Google Scholar] [CrossRef]
  24. Goodwin, G.; Sin, K. Adaptive Filtering Prediction and Control; Dover Publications, Incorporated: Mineola, NY, USA, 2013. [Google Scholar]
Figure 1. Anti-windup proportional-integral ESC.
Figure 1. Anti-windup proportional-integral ESC.
Processes 07 00831 g001
Figure 2. Proportional-integral ESC with quantized actuator.
Figure 2. Proportional-integral ESC with quantized actuator.
Processes 07 00831 g002
Figure 3. Performance of the anti-windup PIESC for Example 1. The top plot shows the progression of the cost function. The corresponding input variable is shown in the middle plot. The bottom plot shows the resulting amplitude of the dither signal.
Figure 3. Performance of the anti-windup PIESC for Example 1. The top plot shows the progression of the cost function. The corresponding input variable is shown in the middle plot. The bottom plot shows the resulting amplitude of the dither signal.
Processes 07 00831 g003
Figure 4. Parameter estimates of the anti-windup PIESC with the saturation bias update for Example 1.
Figure 4. Parameter estimates of the anti-windup PIESC with the saturation bias update for Example 1.
Processes 07 00831 g004
Figure 5. Performance of the discrete actuator PIESC for Example 1. The top plot shows the progression of the cost function. The corresponding input variable is shown in the middle plot. The bottom plot shows the resulting amplitude of the dither signal.
Figure 5. Performance of the discrete actuator PIESC for Example 1. The top plot shows the progression of the cost function. The corresponding input variable is shown in the middle plot. The bottom plot shows the resulting amplitude of the dither signal.
Processes 07 00831 g005
Back to TopTop