- freely available
- re-usable

*Systems*
**2014**,
*2*(4),
606-660;
doi:10.3390/systems2040606

^{1}

^{2}

## Abstract

**:**We survey some of the rich history of control over the past century with a focus on the major milestones in adaptive systems. We review classic methods and examples in adaptive linear systems for both control and observation/identification. The focus is on linear plants to facilitate understanding, but we also provide the tools necessary for many classes of nonlinear systems. We discuss practical issues encountered in making these systems stable and robust with respect to additive and multiplicative uncertainties. We discuss various perspectives on adaptive systems and their role in various fields. Finally, we present some of the ongoing research and expose problems in the field of adaptive control.

## 1. Introduction

Any system—engineering, natural, biological, or social—is considered adaptive if it can maintain its performance, or survive in spite of large changes in its environment or in its own components. In contrast, small changes or small ranges of change in system structure or parameters can be treated as system uncertainty, which can be remedied in dynamic operation either by the static process of design or the design of feedback and feed-forward control systems. By systems, we mean those in the sense of classical mechanics. The knowledge of initial conditions and governing equations determines, in principle, the evolution of the system state or degrees of freedom (a rigid body for example has twelve states–three components each of position, velocity, orientation and angular velocity). All system performance, including survival or stability, is in principle expressible as functions or functionals of system state. The maintenance of such performance functions in the presence of large changes to either the system or its environment is termed adaptation in the control systems literature. Adaptation of a system, as in biological evolution, can be of two kinds–adapting the environment to maintain performance, and adapting itself to environmental changes. In all cases, adaptive systems are inherently nonlinear, as they possess parameters that are functions of their states. Thus, adaptive systems are simply a special class of nonlinear systems that measure their own performance, operating environment, and operating condition of components, and adapt their dynamics, or those of their operating environments to ensure that measured performance is close to targeted performance or specifications.

The organization of the paper is as follows: Section 2 surveys some of the rich history of adaptive systems over the last century, followed by Section 3 with provides a tutorial on some of the more popular and common methods used in the field: Model Reference Adaptive Control, Adaptive Pole Placement, Adaptive Sliding Mode Control, and Extremum Seeking. Section 4 provides a tutorial for the early adaptive identification methods of Kudva, Luders, and Narendra. A brief introductory discussion is provided for the non-minimal realizations used by Luders, Narendra, Kreisselmeier, Marino, and Tomei. Section 5 discusses some of the weak points of control and identification methods such as nonlinear behavior, observability and controllability for nonlinear systems, stability, and robustness. This section also includes some of the solutions for handling these problems. Section 6 discusses some of the interesting perspectives related to control, observation, and adaptation. Section 7 presents some of the open problems and future work related to control and adaptation such as nonlinear regression, partial stability, non-autonomous systems, and averaging.

## 2. History of Adaptive Control and Identification

The first notable and widespread use of ‘adaptive control’ was in the aerospace industry during the 1950s in an attempt to further the design of autopilots [1]. After the successful implementation of jet engines into aircraft, flight envelopes increased by large amounts and resulted in a wide range of operating conditions for a single aircraft. Flight envelopes grew even more with developing interest in hypersonic vehicles from the community. The existing autopilots at the time left much to be desired in the performance across the flight envelope, and engineers began experimenting with methods that would eventually lead to Model Reference Adaptive Control (MRAC). One of the earliest MRAC designs, developed by Whitaker [2,3], was used for flight control. During this time however, the notion of stability in the feedback loop and in adaptation was not well understood or as mature as today. Parks was one of the first to implement Lyapunov based adaptation into MRAC [4]. An immature theory coupled with bad and/or incomplete hardware configurations led to significant doubts and concerns in the adaptive control community, especially after the crash of the X-15. This caused a major, albeit necessary, detour from the problem of adaptation to focus on stability.

The late 1950s and early 1960s saw the formulation of the state-space system representation as well as the use of Lyapunov stability for general control systems, by both Kalman and Bertram [5,6]. Aleksandr Lyapunov first published his book on stability in 1892, but the work went relatively unnoticed (at least outside of Russia) until this time. It has since been the main tool used for general system stability and adaptation law design. The first MRAC adaptation law based on Lyapunov design was published by Parks in 1966 [1]. During this time Filippov, Dubrovskii and Emelyanov were working on the adaptation of variable structure systems, more commonly known as sliding mode control [7]. Similar to Lyapunov’s method, sliding mode control had received little attention outside of Russia until researchers such as Utkin published translations as well as novel work on the subject [8]. Adaptive Pole Placement, often referred to as Self-Tuning Regulators, were also developed in the 1970s by Astrom and Egardt with many successful applications [9,10], with the added benefit of application to non-minimum phase systems. Adaptive identifiers/observers for LTI systems were another main focal point during this decade with numerous publications relating to model reference designs as well as additional stabilization problems associated with not having full state measurement [11,12,13,14,15,16]. However, Egardt [17] showed instability in adaptive control laws due to small disturbances which, along with other concerns such as instabilities due to: high gains, high frequencies, fast adaptation, and time-varying parameters, led to a focus on making adaptive control (and observation) robust in the 1980s. This led to the creation of Robust Adaptive Control law modifications such as: σ-modification [18, ϵ-modification [19], Parameter Projection [20], and Deadzone [21]. As an alternative for making systems more robust with relatively fast transients a resurgence in Sliding Mode Control and its adaptive counterpart was seen, particularly in the field of robotics [22,23,24]. The ideas of persistent excitation and sufficient richness were also formulated in response to the stability movement by Boyd, Sastry, Bai, and Shimkin [25,26,27,28].

These three decades were also a fertile time for nonlinear systems theory. Kalman published his work on controllability and observability for linear systems in the early 1960s, and it took about 10 years to extend these ideas to nonlinear systems through the use of Lie theory [29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44]. Feedback Linearization was formulated in the early to mid-1980s as a natural extension from applying Lie theory to control problems [45,46,47,48,49,50,51,52]. Significant improvements on our understanding of nonlinear systems and adaptation in the early 1990s was facilitated by the work on Backstepping and its adaptive counterpart by Kokotovic, Tsinias, Krstic, and Kanellakopoulos [53]. While Backstepping was being developed for matched and mismatched uncertainties, Yao and Tomizuka created a novel control method Adaptive Robust Control [54]. Rather than design an adaptive controller and include robustness later, Yao and Tomizuka proposed designing a robust controller first to guarantee transient performance to some error bound, and include parameter adaptation later using some of the methods developed in the 1980s. The previous work on nonlinear controller design also led to the first adaptive nonlinear observers during this time [55].

Another side of the story is related to Extremum Seeking control and Neural Networks, whose inception came earlier but development and widespread use as non-model and non-Lyapunov based adaptation methods took much longer. The first known appearance of Extremum Seeking (ES) in the literature was published by LeBlanc in 1922 [56]; well before the controls community was focused on adaptation. However, after the first few publications, work on ES slowed to a crawl with only a handful of papers being published over the next 78 years [57]. In 2000, Krstic and Wang provided the first rigorous stability proof [58], which rekindled excitement and interest in the subject. Choi, Ariyur, Lee, and Krstic then extended ES to discrete-time systems in 2002 [59]. Extremum Seeking was also extended to slope seeking by Ariyur and Krstic in 2004 [60], and Tan et al. discussed global properties of Extremum Seeking in [61,62]. This sudden resurgence of interest, has also led to the discovery of many interesting applications of Extremum Seeking such as antiskid braking [63], antilock braking systems [64], combustion instabilities [65], formation flight [66], bioreactor kinetics [67], particle accelerator beam matching [68], and PID tuning [69].

The idea of Neural Networks as a mathematical logic system was developed during the 1940s by McCulloch and Pitts [70]. The first presentation of a learning rule for synaptic modification came from Hebb in 1949 [71]. While many papers and books were published on subjects related to neural networks over the next two decades, perhaps the most important accomplishment was the introduction of the Perceptron and its convergence theorem by Rosenblatt in 1958 [72]. Widrow and Hoff then proposed the trainable Multi-Layered Perceptron in 1962 using the Least Mean Square Algorithm [73], but Minsky and Papert then showed the fundamental limitations of single Perceptrons, and also proposed the ‘credit assignment problem’ for Multi-Layer Perceptron structures [74]. After a period of diminished funding and interest, these problems were finally solved in the early 1980s. Shortly after this, Hopfield [75] showed that information could be stored in these networks which led to a revival in the field. He was also able to prove stability, but convergence only to a local minimum not necessarily to the expected/desired minimum. This period also saw the re-introduction of the back-propagation algorithm [76], which has become extremely relevant to neural networks in control. Radial Basis Functions (RBFs) were created in the late 80s by Broomhead and Lowe [77] and were shortly followed by Support Vector Machines (SVMs) in the early 90s [78]. Support Vector Machines dominated the field until the new millennium, after which previous methods came back into popularity due to significant technological improvements as well as the popularization of deep learning for fast ANN training [79].

In terms of the most recent developments (2006–Present) in adaptive control, the situation is a little complicated. The period from 2006 to 2011 saw the creation of the L_{1}-AC method [80,81,82,83,84,85] which garnered a lot of excitement and widespread implementation for several years. Some of the claimed advantages of the method included: decoupling adaptation and robustness, guaranteed fast adaptation, guaranteed transient response (without persistent excitation), and guaranteed time-delay margin. However, in 2014 two high profile papers [86,87] brought many of the method’s proofs and claimed advantages into question. The creators of the method were invited to write rebuttal papers in response to these criticisms, but ultimately declined these opportunities and opted instead to post non-peer-reviewed comments on their websites [88]. Other supporters of the method also posted non-peer-reviewed rebuttals on their website [89]. Many in the controls community are uncertain about the future of the method, especially since all of the main papers were reviewed and published in very reputable journals. In order to sort out the truths with respect to the proofs and claims of the method, more work needs to be done.

## 3. Adaptive Control Techniques

The goal of this section is to provide a survey of the more popular methods in adaptive control through analysis and examples. The following analyses and examples assume a familiarity with Lyapunov stability theory, but an in-depth treatise on the subject may be found in [90] if more background is needed.

#### 3.1. Model Reference Adaptive Control

Model Reference Adaptive Control, or MRAC, is a control system structure in which the desired performance of a minimum phase (stable zeros) system is expressed in terms of a reference model that gives a desired response to a command signal. The command signal is fed to both the model as well as the actual system, and the controller adapts the gains such that the output errors are minimized and the actual system responds like the model (desired) system. We show the block diagram for this structure in Figure 1.

To show a simple example of MRAC, consider a simple first order LTI system

Now we match the equations with x in place of ${x}_{m}$

At this point, we may choose to make our adaptive controller ‘direct’ or ‘indirect’. The direct method adapts controller parameters directly, so $(a-{a}_{m})/b$ becomes one parameter, even though ${a}_{m}$ is already known. The indirect method adapts estimates of the plant parameters, and then uses these values to update static controller relations. In other words a and b are estimated and then used in $(a-{a}_{m})/b$ and ${b}_{m}/b$ to calculate the controller parameters. Direct methods typically rely on a gradient method such as the MIT rule, or are calculated based on Lyapunov stability theory. Indirect methods include algorithms such as Recursive Least Squares. For this example, we will choose the direct method and leave an indirect example for the next section.

Defining the parameters ${p}_{1}$ and ${p}_{2}$, we replace them with their estimates in the control law

We the define the output error as the difference between the system and the reference model, that is

We substitute the control law into the error dynamics equation, and attempt to find a solution such that the error will be driven to zero, and parameter errors will go to zero as well. The error dynamics are written as

Using the relation $\widehat{p}=p-\tilde{p}$, we cancel out all of the terms with the exact parameter values that we do not know to get

Finally we get a representation that only relies on the parameter errors,

We construct a Lyapunov candidate

We can see that the first error term will be stable, and the entire system will be stable if we can force the other terms to be zero. We then simplify the expression and attempt to solve for the parameter adaptation. It is important to note here that since b is a constant and Γ is a gain matrix that we design, b can easily be ‘absorbed’ by Γ. The final representation of the Lyapunov analysis is shown as

The parameter adaptation law is, as we might think, a negative gradient descent relation that is a function of the output error of the system,

It should be clear that in the case of a system of dimension larger than one, the preceding analysis will require linear algebra as well as the need to solve the Lyapunov equation ${A}^{T}P+PA=-Q$, but the results are more or less the same. Using the parameters $a=b=0.75$, ${a}_{m}={b}_{m}=2$, ${\widehat{p}}_{1}\left(0\right)=0.8$, ${\widehat{p}}_{2}\left(0\right)=0.5$, ${\gamma}_{1}={\gamma}_{2}=5000$ we get the following simulation results. Figure 2 compares the output response of the plant with the reference model and reference signal. The figure clearly shows that the plant tracks the reference model will little to no error. Figure 3 shows the control input that forces the plant to follow the reference model. The initial oscillations come from the parameters being adapted to force the error to zero. The control parameter estimates are shown in Figure 4. We may note that the convergence of parameters for this system is quite fast, but also that their convergence to incorrect values has no effect on the output response of the system.

**Remark 1** (Simulation). Simulations were performed using MATLAB Simulink. While simulations may be performed using coded loops, Simulink provides a convenient graphical environment that allows the user to use the control block diagrams to construct their system and controller. Each function block then contains the necessary equations to solve for the closed loop response at each time step. Transfer function blocks may be used in place of function blocks in many cases. This methodology may be applied to all control block diagrams. Keep in mind that the nonlinear nature of adaptive controllers will require a small time-step in simulation (typically on the order of ${10}^{-3}$). We show an example Simulink structure in Figure 5 for clarity.

#### 3.2. Adaptive Pole-Placement

Adaptive Pole Placement Control (APPC) methods represent the largest class of adaptive control methods [91], and may be applied to minimum and non-minimum phase (NMP) systems. The idea behind APPC is to use the feedback loop to place the closed loop poles in locations that give us dynamics we desire. Figure 6 shows the basic control structure for APPC. If we also choose to design the zeros of the system, this adds an assumption of a minimum phase system, and leads to the Model Reference Adaptive Control (MRAC) method from the previous section. Consider the system and feedback control law

If we want our system to follow a specified model we construct the relation

So far we have not made any assumptions about the stability of the system, only that A and B do not have any common factors. First we factor B into two parts, the stable and the unstable $B={B}^{+}{B}^{-}$ as suggested in [1]. A cancellation must exist in $BT/{A}_{c}$ in order to achieve ${B}_{m}/{A}_{m}$ and we know that we cannot cancel ${B}^{-}$ with the controller, so ${B}^{+}$ must be a factor of ${A}_{c}$. We also know that ${A}_{m}$ must be a factor of ${A}_{c}$, so we may separate ${A}_{c}$ into three parts

Since ${B}^{+}$ is a factor of B and ${A}_{c}$ it must also be a factor of R, giving $R={R}^{\prime}{B}^{+}$. This is due to the fact that it cannot be a factor of A since A and B are coprime. The closed loop characteristic equation finally reduces to

Going back to the numerator, since we cannot cancel ${B}^{-}$ it must be a factor of ${B}_{m}$, giving ${B}_{m}={B}^{-}{B}_{m}^{\prime}$. Finally, using the closed loop relation

We did not need the assumption of a minimum phase system as with MRAC because we did not cancel ${B}^{-}$, which is a big advantage over that method. Finally we introduce some causality conditions for the polynomials ${A}_{0}$, T, R, and S [1]:

Consider a second order system of relative degree one and its desired reference model

Choosing not to cancel the zero, and using the minimal design and causality conditions $\text{deg}\left({A}_{c}\right)=3$, $\text{deg}\left(S\right)=1$, $\text{deg}\left(R\right)=1$, $\text{deg}\left({A}_{0}\right)=1$, $\text{deg}\left(T\right)=1$ we may solve for the controller parameter values

While both direct and indirect adaptation methods may be used with Self-Tuning Regulators, the indirect option is typically chosen. This is because solving for adaptation laws in the direct sense can become intractable since we may choose not to cancel zeros, which may lead to parametric nonlinearities depending on the system. Performing system identification through something like Recursive Least Squares (RLS) and having static control relationships is a much easier alternative that works provided signals are persistently exciting enough and parameters values are not ill-conditioned. We consider the popular example of a DC motor [1] whose transfer function and desired model are

On-line system identification is done with the RLS algorithm

**Remark 2** (Implementation Issue). It turns out that there can be some implementation issues specifically related to adapting transfer function blocks in certain graphical control simulation software, so the control needs to be converted to state space but cannot depend on derivative signals of inputs. Consider an example input

**Remark 3** (Stochastic and Predictive Methods). Stochastic and predictive methods, as mentioned above, have seen the most widespread implementation out of all of the adaptive control methods, especially in the oil and chemical industries. These methods consist of optimizing the current time step and predicting future ones, according to models obtained from system identification and specified cost functions. In the case of adaptive stochastic and predictive methods, the identification is done on-line, and since sampling often creates non-minimum phase properties most of these methods are naturally based on the Self-Tuning Regulator structure. Some examples of these methods are: Minimum-Variance, Moving-Average, Linear Quadratic Gaussian, and the so called ‘Shooting’ methods for nonlinear systems.

#### 3.3. Adaptive Sliding Mode Control

Adaptive Sliding Mode Control, or ASMC, is a variable structure control method that specifies a manifold or surface along with the system will operate or ‘slide’. When the performance deviates from the manifold, the controller provides an input in the direction back towards the manifold to force the system back to the desired output. We show the control structure in Figure 10. ASMC has been shown to be much more robust to noise, uncertainty, and disturbances than MRAC, but requires larger input signals.

We start by guaranteeing the sliding dynamics are always driven towards a specified sliding surface, that is

The above simplifies to

However, we can see that if s is close to zero the controller will chatter, which can damage the system. A fix for this is to introduce a ‘boundary layer’ around the sliding surface in the form of a saturation function, to smooth out the controller response and remove the chatter. The saturation function is defined as

Then we define some stable manifold we want our system to slide along and determine its dynamics, in this case:

We replace the surface dynamics with the discrete term that represents the trajectory motion towards the manifold which is

Solving for u we obtain the control law

Then replacing the parameters with their estimates, we attempt to substitute u into $\dot{s}$ and obtain an equation for $\dot{s}$ that is stable and includes $\tilde{p}$ which is shown by:

In order to solve for an adaptation law that forces the error to go to zero, we consider the Lyapunov candidate

#### 3.4. Extremum Seeking

Extremum Seeking is a strong optimization tool that is widely used in industry and more often categorized as a method of adaptive control. The reason is that Extremum Seeking is capable of dealing with unknown plants whose input to output maps possess an extremum (a minimum or a maximum), and this extremum depends on some parameter. The way Extremum Seeking works is by measuring the gradient of the output through adding (sinusoidal) perturbations to the system. This makes Extremum Seeking a gradient estimate method, with the additional advantage that the estimate happens in real-time. This has led to many industrial applications. The problem is formulated as follows. Suppose we have an unknown map $f\left(\theta \right)$. All we know about this map is that it has a minimum, but the value of this minimum and the $\theta ={\theta}^{*}$ at which it occurs are both unknown to us. We would like to find the value of ${\theta}^{*}$ that minimizes this map. Figure 14 shows the basic Extremum Seeking loop. The output of this map is fed to a washout filter. The purpose of this filter is to remove the bias of the map from the origin. The signal is then demodulated and modulated by a sinusoidal perturbation and integrated to estimate ${\theta}^{*}$ and the result is fed back to $f\left(\theta \right)$, which is also referred to as the cost function. Running this loop several times will lead to the exponential convergence of θ to ${\theta}^{*}$. For simplicity, we will not explain the details of how Extremum Seeking works. The reader can refer to [92] and references therein for more information. Instead, we provide the following simple example.

Suppose that the map is described by $f\left(\theta \right)={(\theta +5)}^{2}+2$. This means that the optimal value for the parameter is $\theta ={\theta}^{*}=-5$. Assuming $f\left(\theta \right)$ is unknown to us, we run Extremum Seeking as shown in Figure 14 with an initial guess of $\theta \left(0\right)=50$ for the parameter. Furthermore, we set the perturbation frequency and amplitude to $\omega =5$ rad/s and $a=0.2$, respectively. The integral gain is set to $k=1$, and finally, the washout filter is designed with $h=5$. Simulation results are shown in Figure 15 and Figure 16. We see that despite our poor initial guess, the algorithm manages to detect the true value of ${\theta}^{*}$ exponentially fast.

## 4. Adaptive Observer Techniques

Thus far we have only considered instances where full state measurement is available, a rather unlikely scenario for controlling real systems. Work on the adaptive observer problem has been on-going since the early seventies, but typically does not receive as much attention as the control problem. Adaptive identification methods not only solve the output feedback problem for adaptive systems, but they may also be used in the field of health monitoring. The various observers take advantage of a variety of canonical forms, each having their own advantages and disadvantages for implementation. Two realizations of an adaptive Luenberger-type observer will be discussed in detail, while the others briefly mentioned are non-minimal extensions of these two basic forms. The adaptive observer/identifier to be discussed can be visualized as an MRAS (Model Reference Adaptive System) structure shown in Figure 17. Exchanging the locations of the plant and model is the key to creating the MRAS structure identifier [12]. In the control case we were modifying the plant to behave like the reference model, where-as in this case we are modifying the observer/model to behave like the unknown plant.

#### 4.1. Adaptive Luenberger Observer Type I

The following introduces one of the first adaptive observer designs, originally published by Kudva and Narenda [14]. It is well known throughout the dynamics and control literature that any observable LTI system may be transformed into the so-called ‘Observable Canonical Form’, and this will be our natural starting point:

We may then equivalently express this system (for reasons seen later) as

Define the state observer to have a similar structure with state and parameter estimates $\widehat{x}$, $\widehat{a}$, and $\widehat{b}$

The purpose of the two additional signals w and r may not be clear at first, especially because we did not need additional signals in the control problem. In the control problem, we assumed that we had access to all states which also meant that we had access to each error term, but this is not the case here. In the observer problem, these two additional signals are used to maintain stability for the system, since everything must now be based only on the signals that we do have access to. Next we define the observer tracking and parameter errors as $e=\widehat{x}-x$, $\tilde{a}=a-\widehat{a}$, and $\tilde{b}=\widehat{b}-b$. The error dynamics become

In order to design the governing equations for our additional signals w and r, we consider the scalar representation of the system

The following equality

The Lyapunov candidate is chosen to be

We are still unsure about the first term, but we may choose the adaptation laws as

The first term is proven to be stable by choosing K and d to satisfy the Kalman-Yakubovich (Strictly-Positive-Real) Lemma

The adaptive laws then change to $\dot{\tilde{a}}=-{\Gamma}_{1}{e}_{1}v$ and $\dot{\tilde{b}}=-{\Gamma}_{2}{e}_{1}q$. Thus the system is proven to be stable in the Lyapunov sense. Now consider the system given in [14] already in observable canonical form

The gains are chosen to be

#### 4.2. Adaptive Luenberger Observer Type **II**

The next adaptive observer we will design is based on systems of the form

It was shown in [15], that any observable LTI system may be transformed into the structure above. The purpose of converting systems to this form is to combat implementation issues with the previous observer as will be seen. The transformation is defined assuming the system is already in the OCF structure. First we design the diagonal matrix

The transformation is finally defined as

**w**is chosen to make the observer dynamics stable. The error dynamics of the system then become

Choose w to be

It follows that

The standard Lyapunov candidate will then lead to adaptive laws similar to the previous section. Notice that we were able to make it to this point without the need for complicated stabilizing signals, without needing to design K and d as they are both taken care of through Λ, and most importantly without the need to check the K-Y Lemma which can be tedious. By using a slightly more complex transformation (a one time event), Luders and Narendra were able to significantly simplify the observer.

#### 4.3. Non-Minimal Adaptive Observers

The previous sections focused on minimal adaptive observer forms and detailed analysis. We now provide a brief exposure to some of the concepts of non-minimal observer form. Shortly after the progress in creating the minimal observers from the previous sections, Luders and Narendra developed a non-minimal representation [11,16]

Choosing the standard Lyapunov candidate

Two other non-minimal observers that came from this design were developed by Kreisselmeier [13], and Marino and Tomei [55]. The K-Filter [13] was originally developed for LTI systems as a reduced order ‘Series-Parallel’ observer, but was quickly extended to cases where nonlinearities may be expressed as functions of the measurements [53]. The K-Filter representation is given by

We have briefly introduced some of the non-minimal realizations of adaptive observers for LTI systems, but a much more detailed presentation and analysis of these methods may be found in [11,53].

## 5. Problems in Control and Adaptation

#### 5.1. Nonlinear Systems

Nonlinear behavior is one of the most (if not the most) difficult aspects of adaptive control. Unfortunately there cannot be a general nonlinear theory. We must settle for using specific tools and methods that apply to the sets of system structures that we do understand. There are many types of nonlinear behaviors: limit cycles, bifurcations, chaos, deadzone, saturation, backlash, hysteresis, nonlinear friction, stiction, etc. Figure 22 shows some example plots of common nonlinearities.

Nonlinear behaviors are sometimes divided into two classes: ‘hard’ and ‘soft’. Soft nonlinearities are those which may be linearly approximated, such as ${x}^{2}$ or special types of hysteresis. Typically this means that as long as we do not stray too far from our operating point, we may use linear control methods since we can linearize the system. Hard nonlinearities are those which may not be linearly approximated, such as: Coulomb friction, saturation, deadzones, backlash, and most forms of hysteresis. Hard nonlinearities may easily lead to instability and/or limit cycles, and they unfortunately appear in many real systems. Moreover, since we cannot linearize we are forced to use nonlinear control methods in addition to adaptation. Fortunately for us there are methods for handling nonlinear control design: Feedback Linearization and Backstepping.

**Figure 22.**Examples of non-lipschitz nonlinearities. (

**a**) Relay; (

**b**) Deadzone; (

**c**) Saturation; (

**d**) Quantization; (

**e**) Backlash; (

**f**) Hysteresis-Relay.

Feedback Linearization is a method in which a nonlinear coordinate transformation between the input and output is found such that the transformed system is linear along all trajectories. The first r-derivatives of the output are the coordinate transformations, and the coordinate transformation is required to be a diffeomorphism (invertible and smooth). We then design the input such that the ${r}^{\text{th}}$ output derivative is equivalent to some desired dynamics, ν, and all nonlinearities are canceled. Consider the following example of system and output dynamics [93]:

For input-output linearization we essentially take derivatives of y until the control input shows up in our equations

Our goal is to replace the $\ddot{y}$ with some desired dynamics ν, so we choose our control input to be

This transforms the system dynamics into

We can see that an easy solution is ${z}_{3}={x}_{2}+{x}_{3}$. Now we check the Jacobian to see if it is regular (no critical points) for all x with

The Jacobian is regular for all x and invertible, thus it is a global diffeomorphism. The last step one would normally take is to try and determine whether the zero dynamics are stable or not through functional analysis methods. However, we should beware of the complications that arise when we require all nonlinearities to be canceled. This means that we need to know our system exactly in order to truly have linear dynamics; any unmodeled dynamics can have disastrous effects. The other downside to this method is that the control signal may be unnecessarily large because we also cancel helpful nonlinearities (like $\dot{x}=-{x}^{3}$) in the process. It seems as though Feedback Linearization will perform well on systems containing soft nonlinearities, but maybe not on systems with hard nonlinearities unless we use neural networks.

Backstepping was created shortly after Feedback Linearization to address some of the aforementioned issues. It is often called a ‘Lyapunov-Synthesis’ method, because it recursively uses Lyapunov’s second method to design virtual inputs all the way back to the original control input. The approach removes the restrictions of having to know the system exactly and remove all nonlinearities because we use Lyapunov’s method at each step to guarantee stability. Backstepping is typically applied to systems of the triangular form:

We view each state equation in this structure as its own subsystem, where the term coupled to the next state equation is viewed as a virtual control signal. An ideal value for signal is constructed, and the difference between the ideal and actual values is constructed such that the error is exponentially stable by Lyapunov. We may the error of the system to be the state or the difference between the system and a model in the cases of regulation and model following respectively. Assuming a model following problem, we consider a Lyapunov candidate for the first subsystem

Following Lyapunov’s method, we would take the derivative which results in

The ${x}_{2}$ term that connects the first subsystem to the next, is treated as the virtual control input. We choose ${\alpha}_{1}$ as the ideal value for virtual control ${x}_{2}$ such that ${e}_{2}$ becomes ${e}_{2}={x}_{2}-{\alpha}_{1}$. We will be able to prove that the term containing ${e}_{1}$ is negative definite (guaranteeing stability), but we will be left with an ${e}_{2}$ from substituting in ${x}_{2}={e}_{2}+\alpha $. This leads us to design a virtual control for the next subsystem in the same fashion, and then combining the Lyapunov candidates to get

This continues on with ${\alpha}_{i-1}$ as the ideal value for ${x}_{i}$ and their corresponding errors, until we reach the real control input u. The final Lyapunov candidate function is

One of the main advantages to Backstepping is that we may leave helpful nonlinear terms in the equations. In Feedback Linearization, we have to cancel out all of the nonlinearities using the control input and various integrators. This makes Backstepping much more robust than Feedback Linearization, and also allows us to use nonlinear damping for control augmentation as well as extended matching for adaptation with tuning functions [53].

#### 5.2. Observability

First consider the LTI system:

For observability, we want to see if we find the initial conditions based on our outputs which is represented by

The observability matrix for this system is then

The observability matrix must be full rank for the system to be fully observable, that is

This is a corollary to saying the system dynamics are injective, or a one-to-one mapping. This states that if the function f is injective for all a and b in its domain, then $f\left(a\right)=f\left(b\right)$ implies $a=b$. More intuitively, for linear systems this means that if the rows are linearly independent, each state is observable through linear combinations of the output. We can also notice that we can get each column vector in O by taking the first $n-1$ derivatives of the output y. So in order to extend this to nonlinear systems, we can just use the Lie derivative. For nonlinear systems, the observation space ${O}_{s}$ is defined as the space of all repeated Lie derivatives of the covector $h\left(x\right)$, written as

The system is said to be observable if

We can see that this is true if we substitute $Cx$ for $h\left(x\right)$, and $Ax$ for $f\left(x\right)$. The collection of Lie derivatives will then produce the observability matrix for the LTI system. Finally, an often over-looked but very important part of observability is to check the condition number for the observability matrix of the actual system once sensors and their locations have been chosen. The condition number of the observability matrix can give an indication of how well the sensor choices and locations are for the system at hand. It may also be possible to optimize this process through the use of LMIs. This is especially important for adaptive systems considering their nonlinear behavior, which requires reliable observations to facilitate the adaptation process as well as provide good feedback for the controller.

#### 5.3. Controllability

Consider the same LTI system dynamics

The reachable subspace for the LTI system by the Cayley-Hamilton theorem is

Expanding out the matrix exponential we get the controllability matrix

As with the observability matrix we say the system is controllable if this matrix is full rank, shown as

This is a corollary to saying the system dynamics are surjective, a mapping that is onto. The function f is said to be surjective if for every y in its range there exists at least one x in its domain such that $f\left(x\right)=y$. For linear systems this implies that the columns are linearly independent. It is also important to note that in linear systems, controllability and observability are related through transposition.

We want to find the controllability or reachability for the nonlinear system, so we look at the reachability equation above. After expanding the matrix exponential, we will get a collection of matrix multiplications of A and B. For nonlinear systems, we can multiply vector fields by using the Lie bracket. Thus the controllability matrix is the collection of Lie brackets on the vector fields of the nonlinear triangular system, shown as

Like the observability problem, controllability is obtained if the matrix is full rank, and we can easily check this formulation and see that it even works for linear systems by substituting $Ax$ for $f\left(x\right)$ and B for $g\left(x\right)$. Similarly, checking the condition number of the actual controllability matrix after actuators have been chosen and placed is important. An ill-conditioned matrix indicates that the control authority could be improved, which is important for adaptive systems because of the need for high bandwidth response due to the nonlinear nature of the control law.

#### 5.4. Stability & Robustness

Thus far we have not imposed any conditions on the external signals for stability, convergence, or robustness. In doing this we will explore another difference between the direct and indirect methods, as well as the overall robustness of adaptive controllers to external disturbances and unmodeled dynamics. First consider the problem of finding a set of parameters that best fit a data set. We have the actual and estimated outputs y and $\widehat{y}$, the set of inputs ${\varphi}^{T}$, and the actual and estimated parameters θ and $\widehat{\theta}$.

The goal is to minimize the error e between the actual and estimated outputs y and $\widehat{y}$, which is done by considering the squared error below.

We minimize the squared error by differentiating with respect to the parameter estimates

In order for the estimates to be valid, the matrix $\left(\varphi {\varphi}^{T}\right)$ must be full rank (non-singular). Extending this to a recursive estimation, we want to minimize the integral of the squared error

We follow the same approach as before by differentiating with respect to the parameter estimate and equating with zero,

The parameter estimate may then be expressed using

Apart from the mathematical condition itself, signals are often determined to be persistently exciting enough based on the number of fundamental frequencies contained within the signal. From the frequency space perspective signals may be approximated by sums of sines and cosines of these frequencies, and n frequencies may identify up to $2n$ independent parameters. When the input signal is persistently exciting, both simulations and analysis indicate that adaptive control systems have some robustness with respect to non-parametric uncertainties. This makes sense because the parameter estimates will converge to their true values, thus the equivalent control principle is truly satisfied and we get a ‘perfect’ controller. However, when the signals are not persistently exciting, even small uncertainties may lead to severe problems for adaptive controllers. Consider Rohrs’ examples [95], where the system and model are assumed to be

However, the real plant is

A direct MRAC controller is constructed assuming the plant is of first order, and the initial parameter estimates are ${\widehat{p}}_{1}\left(0\right)=1.14$ and ${\widehat{p}}_{2}\left(0\right)=-0.65$. The first example we look at, is when the reference signal is a constant $r=4.3$ and the gains we have chosen are ${\gamma}_{1}={\gamma}_{2}=2$.

Figure 23 and Figure 24 show that having unmodeled dynamics in the system can have disastrous effects even for small adaptation gains. The second example considers the reference signal $r=0.3+1.85sin(16.1t)$ (gains the same), which we would naturally assume to be persistently exciting without the unmodeled dynamics.

Figure 25 and Figure 26 show that the system starts to converge to the constant input, but becomes unstable as the oscillations grow and the parameters drift. The final example considers the case in which we have noise instead of a disturbance, with the reference signal and noise being $r=2$ and $n=0.5sin(16.1t)$ respectively. We also increase the gains to ${\gamma}_{i}=8.7075$.

Figure 27 and Figure 28 show a particularly troubling result. The system initial converges in both tracking error and parameter estimates, but after some time it becomes wildly unstable and the parameters drift by drastic amounts. Astrom correctly pointed out in a commentary (later added to [95]) that Rohrs’ examples did not completely characterize the robustness problem, but were never-the-less important to present to the community. While they may not fully characterize the issues, they should successfully motivate the results of the next section.

#### 5.5. Robust Adaptive Control

Now that we’ve discussed some of the issues related to adaptation in the presence of noise and disturbances, we present some of the adaptation law modifications that were developed to handle these issues. Control law modifications are mentioned in the next section. First consider the reference model and system with uncertainty:

We construct the control law in the same fashion as MRAC

We next consider the derivative of the standard quadratic Lyapunov candidate

After plugging in the traditional Lyapunov adaptation law, we may convert this to the following inequality

Anytime the error norm is within this bound, $\dot{V}$ can become positive and stability may be lost. The parameter error growth will become unstable regardless of the disturbance size, and is known as ‘parameter drift.’ One of the first adaptation law modifications created to handle this problem was the Deadzone method [21]

After plugging this into the derivative of the Lyapunov function we can construct the inequality

As long as the tracking and parameter errors are outside of these limits the system will remain stable. This method helped to remove the requirement on a-priori disturbance bounds, but presented another problem. If we look at the adaptation law itself, when the tracking error becomes small the trajectory of parameter estimates becomes

The other main drawback for the previous methods is that they can slow down the adaptation, which is the opposite of what we want. The Projection Method [20] is robust to disturbances and does not slow down the adaptation. Projection does, however, require bounds on the parameter values, but this is often acceptable for real systems. The Discontinuous Projection Algorithm is defined as [54]

The discontinuous projection operator has similar chattering problems to adaptation with a deadzone, which led to the creation of smoothing terms similar to the saturation term in sliding mode control [22]. Stability analyses for these methods are excluded here to retain some form of brevity, but can be found in their respective references.

#### 5.6. Adaptive Robust Control

An alternative to just modifying adaptation laws, is to also experiment with various non-equivalent control structures. Yao [54] approached the robustness problem from this perspective to create the Adaptive Robust Control (ARC) method whose structure is shown in Figure 29.

Assuming that the parameters lie in a compact set and the uncertainty is bounded, we start with a similar system as the previous section (input parameter removed for convenience)

We typically separate the robust input into two parts ${u}_{{r}_{1}}$ and ${u}_{{r}_{2}}$. The first part is typically chosen as ${u}_{{r}_{1}}=-ke$. Thus we get

We want to design the robust input ${u}_{{r}_{2}}$ such that it stabilizes the system with respect to the terms ${\varphi}^{T}\tilde{\theta}$ and Δ. We assume that the components of $\varphi $ are measurable or observable, and we already know that the parameters θ and uncertainty Δ are bounded. With this information, we may construct a bounding function $h(x,t)$ that we may use for feedback control such that

An example of a function $h(x,t)$ that satisfies this inequality is

As before in the sliding mode section, we will want to replace the $\text{sgn}\left(e\right)$ term with a smooth function such as $\text{sat}(e/b)$ (other options can be found in [54]). Consider the Lyapunov equation (no adaptation yet) $V={e}^{2}/2$ along with its derivative

Our goal here is to reduce the derivative of the Lyapunov stability equation to

Using the final value theorem we get an error bound for the robust portion of control

In order to satisfy the robust tracking error bound as well as its transient, we require the robust portion of the control law to satisfy two requirements

If we choose $b=4\u03f5/h(x,t)$, then we may rewrite the inequality as

Using the Discontinuous Projection Method for on-line adaptation ($\dot{\tilde{\theta}}=\text{Proj}(\widehat{\theta},\Gamma \varphi e)$), we may decouple the robust and adaptive control designs, and finally analyze closed loop stability for the overall system. Consider the derivative of the standard quadratic Lyapunov candidate

Plugging in the error dynamics and adaptation law we get

It is clear that the first term is negative definite, the second term is negative definite from our robust control law design conditions, and the third term is also negative definite due to the properties of the Discontinuous Projection law. Using this method in conjunction with Backstepping for nonlinear systems makes it extremely powerful especially due to bounded transient performance, but at the expense of a more complicated analysis. The interested reader may refer to [54] for details and simulations.

## 6. Perspectives in Control and Adaptation

#### 6.1. Discrete Systems

Adaptive systems are inherently nonlinear, as must be evident by now, and hence their behavior cannot be captured without sufficiently large sampling frequencies. Hence, implementations of adaptation in engineering systems are based on high frequency sampling of system performance and environment, and therefore based on continuous system models and control designs. In those cases where the system is inherently discrete event, e.g., an assembly line, adaptation occurs from batch to batch as in Iterative Learning Control. The most widely used adaptive control is based on a combination of online system identification and Model Predictive Control. This method is ideal for process plants where system dynamics are slow compared to the sampling frequencies, speed of computation, and that of actuation. The actuators in these plants do not have weight or space constraints, and hence are made extremely powerful to handle a wide range of outputs and therefore of uncertainty and transient behavior. In this case a plant performance index such as time of reaction, or operating cost is minimized subject to the constraints of plant state, actuator and sensor constraints, and the discretized dynamics of the plant. Thus, this method explicitly permits incorporation of real world constraints into adaptation. The online system identification is performed using several tiny sinusoidal or square wave perturbations to actuator commands, and measuring resultant changes of performance. It is ideal for the process industry because of the great variability of the physical and chemical properties of feedstock, such as coal, iron ore, or crude oil.

#### 6.2. Machine Learning and Artificial Intelligence

In general there are two types of models: ‘grey box’ and ‘black box’. Black box modeling indicates that hardly anything is known about the system except perhaps the output signal given some input. We have no idea what is inside the black box, but we may be able to figure it out based on the input/output relation. Grey box modeling is a situation where some a-priori information about the system is known and can be used. For example, knowing the design of the system and being able to use Newton’s laws of motions to understand the dominating dynamics would be considered grey box modeling. Even though we may know a lot about the system, there is no way we can know everything about it, hence the ‘grey’ designation. Uncertainty in our models and parameters are fundamental problems that we must unfortunately deal with in the real world.

The presence of uncertainty and modeling difficulties often leads to the field of machine learning, the process of machines ‘learning’ from data. Put simply, we approximate a relationship (function) between data points, and then hopefully achieving correct predictions based on its function approximation if it is to be used in control. There are many methods associated with machine learning (e.g., Neural Networks, Bayesian Networks, etc.), but we should be very careful in how and where we implement these algorithms. There are situations where using machine learning algorithms provide a significant advantage to the user and situations where it would be disadvantageous.

An example of a system where machine learning would be advantageous is a highly articulated robot such as a humanoid robot. A system of this type has too many states to handle in practice, not to mention complexities due to nonlinearities and coupling, so a machine learning algorithm would be very advantageous here. However, since the algorithm can only improve the function approximation by analyzing data, this approach obviously requires a lot of ‘learning’ time as well as computational resources. Therefore the system (or its data) has to be available to be tested in a controlled environment which is not always possible (consider a hypersonic vehicle). The other disadvantage to certain types of machine learning algorithms is illustrated by Anscombe’s Quartet, shown in Figure 30.

Anscombe’s Quartet was a famous data set given by Francis Anscombe [96] to show the limitations of regression analysis without graphing the data. Each of the significantly different data sets has the exact same mean, variance, correlation, and linear regression line. This shows that there will always be outliers that manipulate the system, but also that we should attempt to have a universal function identifier if we cannot narrow down the distributions of data we will see. There are machine learning algorithms that are universal function identifiers (see the Cybenko Theorem), but these theorems rest on the assumption that design parameters are correctly chosen, which is far from easy. This further illustrates the importance of the ‘learning’ period and training data, as well as using as much a-priori information as possible when it is available. That being said, machine learning algorithms have been successfully applied to a wide variety of very complex problems and constitute a fertile research topic. Some open problems related to Neural Networks are presented later.

#### 6.3. Abstract Viewpoints

The fields of differential topology, differential geometry, and Lie theory are considerably more abstract than many are used to, but may offer a tremendous advantage to visualizing and thinking about more complex spaces. Topology is the study of the properties of a mathematical space under continuous deformations, and its differential counterpart is the study of these with respect to differentiable spaces and functions/vector fields. Differential geometry is quite similar to differential topology, but typically considers cases in which the spaces considered are equipped with metrics that define various local properties such as curvature. Lie theory is the mathematical theory that encompasses Lie algebras and Lie groups. Lie groups are groups that are differential manifolds, and Lie algebras define the structures of these groups. The combination of these fields allows us to classify and perform calculus on these spaces abstractly, regardless of their dimension. An example of the advantage of even understanding some of the most basic concepts is finding the curvature of a higher dimensional space. For problems in 3-dimensional space the cross-product can be used to find curvature, but the cross-product itself is only defined up to this dimension and thus cannot be used for n-dimensional problems. Differential geometry allows one to define curvature in an abstract sense for systems of any order, which will also be equivalent to the cross-product in 3-dimensional problems. We have already used some methods from these fields earlier in this paper, such as finding diffeomorphisms for Feedback Linearization and the Lie bracket for observability/controllability in nonlinear systems. For brevity we did not include the backgrounds of all of these fields, but the interested reader may refer to for more complete treatments of the subjects.

## 7. Open Problems and Future Work

Despite the extensive decades of research in adaptive control and adaptive systems, there are still many unsolved problems to be addressed. Some of these problems may not be ‘low hanging fruit’; however, their solution could lead to important applications. We discuss a few of these problems in this section. First, we discuss nonlinear regression problems, followed by transient performance issues. Finally qualify developing analysis tools that can ease the proofs of stability and boundedness for complex systems, as well as developing novel paradigms for adaptive control.

#### 7.1. Nonlinear Regression

As with systems that are nonlinear in their states, there is no general nonlinear regression approach, it consists of a toolbox of methods that depend on the problem. Parameters appear linearly for many systems, but there are some systems (especially biological) where they appear nonlinearly. This happens to be one of the largest obstacles in the implementation of neural networks. In many neural network configurations, some parameters show up nonlinearly, and are typically chosen to be constant (lack of confidence guarantees), or an MIT rule approach is used (lack of stability guarantees). Even if network input and output weights show up linearly and adaptation laws are chosen using Lyapunov, it is all for naught if the nonlinear parameters are incorrect which can require extensive trial and error tuning by the designer or large amounts of training. An example of how control and identification may be used in an MRAC-like system is given in Figure 31.

The Radial Basis Function Neural Network design is a good example of when we might encounter difficulties related to nonlinear parameters. A basic RBFNN controller uses Gaussian activation functions that are weighted to form the control input u as shown in

The weights w appear linearly, and if we had ideal values for the centers c and biases b then we could easily construct a Lyapunov function to adapt the weights. However, finding ideal values for c and b can be quite difficult, so it would be advantageous to also find Lyapunov stable update laws for them as well. Logarithmic transformations exist for regression problems of the form $y=a{e}^{bx}$, but performing these transformations simultaneously with Lyapunov analysis can become quite overwhelming and even produce further problems related to the parameters that do not appear in the exponent. If more progress is made in finding transformations for turning nonlinear regression problems into linear ones, the field of neural networks and artificial intelligence can grow significantly. Training time for many systems will be significantly reduced, proving Lyapunov stability may become more tractable, and the problem illustrated by Anscombe’s quartet may be reduced.

#### 7.2. Transient Performance

Adaptive controllers have the advantage that the error will converge to zero asymptotically, but for controlling real systems we often care more about the transient performance of the controlled system. It is generally not possible to give an a-priori transient performance bound for an adaptive controller because of the adaptation itself. When we use Lyapunov’s second method to derive an adaptation law, we are guaranteeing error convergence at the expense of parameter error convergence. Higher adaptation gains typically lead to faster convergence rates, but this is not always true given the instabilities that arise from high adaptation gains. Consider the standard scalar Lyapunov function

After the adaptation law is chosen $\dot{V}=-{a}_{m}{e}^{2}$ (assuming no disturbances or modeling uncertainty). Setting up an inequality we may say that $\dot{V}\le -{a}_{m}{e}^{2}$. Now let’s analyze the ${\mathcal{L}}_{2}$ norm

The main problem here is that we do not have a good idea what the initial parameter errors are, so it is difficult to give a-priori predictions of transient performance. There are four common ways to help improve the transient performance: increase ${a}_{m}$, increase ${\gamma}_{i}$, use correct trajectory initialization to minimize $e\left(0\right)$, and perform system identification to minimize $\tilde{a}\left(0\right)$ and $\tilde{b}\left(0\right)$. Increasing ${a}_{m}$ is obviously a good way to improve performance, but this may not be an option depending on the control authority in the system (e.g., bandwidth, power, etc.). Increasing ${\gamma}_{i}$ is an option for ideal systems but as we discovered before, high adaptation gains in real systems can lead to instability and this will depend on the specific system at hand.

The use of robust adaptive control methods helps mitigate the stability problem, but at the expense of slowed transient performance or convergence to an error bound rather than zero. The projection method is reported to maintain fast adaptation and stability, but requires the designer to have bounds on parameters. This is not an unreasonable assumption, but there may be cases where it does not apply. Carefully initializing the trajectory to set $e\left(0\right)=0$ is an option that can be applied to most systems and does not pose any additional problems. Lastly, good system identification can provide accurate parameter estimates to initialize with in order to minimize $\tilde{a}$ and $\tilde{b}$, but this also assumes that the system may easily/cheaply be identified and that the correct identification method is used. There are many systems in existence that have highly nonlinear parametric models that are difficult (or impossible) to accurately identify with existing methods, or may be too costly to identify frequently. Improving transient performance is an on-going research problem in adaptive systems.

#### 7.3. Developing Analysis Tools

The complexity of the tools needed to analyze the stability of a system grows with the complexity of the system. In this context, system has a broad meaning and is defined as a set of ordinary or partial differential equations. With this point of view, a control problem also reduces to a stability problem. For example, a control problem in which the objective is the perfect tracking of state variables, turns into a stability problem when we define tracking error and look at the governing error dynamics as our “system”. The stability of systems have been widely studied for more than a century, from the work of Lyapunov in 1900s. Hence, we will not discuss those results; however, we will mention a few areas where improvements could be made. The benefit of such research is twofold. Any new tool developed will have applications to any branch of science that deals with “dynamics”, i.e., any system that changes with time.

In order to provide more motivation, consider the following scenario. We have derived an adaptive control law for a nonlinear non-autonomous system under external disturbances. We would like to prove that the state tracking error converges to zero despite disturbances and the parameter tracking errors are at least bounded. If we fail this task, we would like to at least prove that the states remain bounded under disturbances, and that the error remains within a finite bound the whole time.

Non-autonomous systems are explicitly time-dependent and are generally described as

#### 7.3.1. Averaging Theorems

The first method is known as the averaging method, and applies to systems of the form

**Theorem 1**

([90]). Let $f(t,x,\epsilon )$ and its partial derivatives with respect to $(x,\epsilon )$ up to the second order be continuous and bounded for $(t,x,\epsilon )\in [0,\infty )\times {D}_{0}\times [0,{\epsilon}_{0}]$, for every compact set ${D}_{0}\subset D$, where $D\subset {R}^{n}$ is a domain. Suppose f is T-periodic in t for some $T>0$ and ε is a positive parameter. Let $x(t,\epsilon )$ and ${x}_{av}\left(\epsilon t\right)$ denote the solutions of (206) and (207), respectively.

If ${x}_{av}\left(\epsilon t\right)\in D\phantom{\rule{4pt}{0ex}}\forall \phantom{\rule{4pt}{0ex}}t\in [0,b/\epsilon ]$ and $x(0,\epsilon )-{x}_{av}\left(0\right)=O\left(\epsilon \right)$, then there exists ${\epsilon}^{*}>0$ such that for all $0<\epsilon <{\epsilon}^{*}$, $x(t,\epsilon )$ is defined and

$$x(t,\epsilon )-{x}_{av}\left(\epsilon t\right)=O\left(\epsilon \right)\phantom{\rule{1.em}{0ex}}on\phantom{\rule{1.em}{0ex}}[0,b/\epsilon ]$$If the origin $x=0\in D$ is an exponentially stable equilibrium point of the average system Equation (207), $\Omega \subset D$ is a compact subset of its region of attraction, ${x}_{av}\left(0\right)\in \Omega $, and $x(0,\epsilon )-{x}_{av}\left(0\right)=O\left(\epsilon \right)$, then there exists ${\epsilon}^{*}>0$ such that for all $0<\epsilon <{\epsilon}^{*}$, $x(t,\epsilon )$ is defined and

$$x(t,\epsilon )-{x}_{av}\left(\epsilon t\right)=O\left(\epsilon \right)\phantom{\rule{1.em}{0ex}}forall\phantom{\rule{1.em}{0ex}}t\in [0,\infty )$$If the origin $x=0\in D$ is an exponentially stable equilibrium point of the average system, then there exists positive constants ${\epsilon}^{*}$ and k such that, for all $0<\epsilon <{\epsilon}^{*}$, Equation (206) has a unique, exponentially stable, T-periodic solution $\overline{x}(t,\epsilon )$ with the property $\left|\right|\overline{x}(t,\epsilon )\left|\right|\le k\epsilon $.

Now suppose we want to apply this theorem to our hypothetical scenario and prove the stability of our adaptive controller. A reader familiar with adaptive control already knows that unless PE conditions are satisfied, the stability is only asymptotic. This means that parts 2 and 3 of this theorem will not be applicable, leaving us only with part 1. However, this part is also a weak result which is only valid for finite times. Therefore, averaging theorem in this form is not helpful in proving stability of adaptive control applied to non-autonomous systems.

Teel et al. [97,98] have shown that if the origin of the system is asymptotically stable with some additional conditions, then we can deduce practical asymptotic stability of the actual system. However, their theorem cannot be used in this scenario either. This is due to the fact that when we have asymptotic stability in adaptive control, the parameter errors generally do not converge to the origin. So even though the state tracking error converges to the origin, the parameters in general will converge to an unknown equilibrium manifold. This is sometimes referred to as “partial stability” and we will discuss it further. Therefore, one area of improving tools is to devise averaging theorems that not only work for asymptotically stable system, but also account for systems with partial stability where only state errors converge to the origin.

#### 7.3.2. Non-Autonomous Lyapunov Stability

In search for a proper tool to analyze our hypothetical scenario, we next move on to Lyapunov theorems for non-autonomous systems. Such theorems, directly deal with systems that are explicitly time-dependent and are much less developed than theorems regarding autonomous systems. Khalil [90], Chapter 4, provides three such theorems one of which is mentioned here.

**Theorem 2**

([90]). Let $x=0$ be an equilibrium point for Equation (205) and $D\subset {R}^{n}$ be a domain containing $x=0$. Let $V:[0,\infty )\times D\to R$ be a continuously differentiable function such that

In adaptive control, it is extremely difficult to satisfy condition (211). The derivative of the Lyapunov function is usually only semi-negative definite at best. This means that the right hand side of Equation (211) will only have some of the states and the inequality will not hold. We believe that creating new Lyapunov tools with less restricting conditions is an area that needs more attention.

#### 7.3.3. Boundedness Theorems

When we cannot study the stability of the origin using known tools, or when we do not expect convergence to the origin due to the disturbances, the least we hope for is that x will be bounded in a small region. Boundedness theorems consider such cases and several of them are addressed in Chapter 9 of [90] who categorizes perturbations into vanishing perturbations and non-vanishing perturbations. The general approach is to separate the perturbation terms from the system. Therefore, the system is described as

Vanishing perturbations refer to the case where $g(t,0)=0$. Therefore, if $x=0$ is an equilibrium of the nominal system $\dot{x}=f(t,x)$, then it also becomes an equilibrium point of the perturbed system. Non-vanishing perturbations refer to cases where we cannot determine whether $g(t,0)=0$ . Therefore, the origin may not be an equilibrium point of the perturbed system.

Such theorems, although very useful in many cases, still need further development before they can be applied to our scenario and be useful for adaptive control. Most boundedness theorems require exponential stability at the origin. We know that in adaptive control, exponential stability is only possible when PE conditions are satisfied (in which case our first approach to use averaging theorems would have worked already!). Furthermore, in the absence of PE, the origin is not a unique equilibrium for the adaptive controller: some parameter estimates may converge to an unknown equilibrium manifold and we cannot transform the equilibrium of the adaptive controller to the origin. Since the objective of the adaptive controller is not the identification of parameters, but rather the convergence of tracking error, one wonders whether it is possible to study the stability of only parts of the states (i.e., the stability of the state tracking error). This is sometimes referred to as partial stability.

#### 7.3.4. Partial Stability and Control

The first person to ever formulate partial stability was Lyapunov himself – the founder of modern stability theory. During the cold war, with the resurgence of interest in stability theories, this problem was pursued and Rumyantsev [99,100] published the first results. Much research has been done on partial stability all over the world, but mostly in Russia and the former USSR. For example, see [101,102,103,104,105,106]. Since this topic might be unfamiliar to many readers, we explain it a little further and refer the enthusiastic reader to the papers cited. In particular [103] provides a comprehensive survey of problems in partial stability.

Partial stability deals with systems for which the origin is an equilibrium point, however, only some of the states approach the origin. Such systems commonly occur in practice, and there have been a fair amount of research on their stability analysis using invariant sets and Lyapunov-like lemmas such as Barbalat’s Lemma or LaSalle’s Principle. Some of the motives for studying partial stability are [103]: systems with superfluous variables, sufficiency of partial stability for normal operations of system, estimation of system performance in “emergency” situations where regular stability is impossible, and the difficulties in rigorous proofs of global stability.

The problem of partial stability is formulated as follows. Consider the system

**Definition 1.** An equilibrium point $x=0$ of Equation (215) is

y-stable, if for any $\epsilon >0$, and ${t}_{0}\ge 0$, there exists a $\delta =\delta (\epsilon ,{t}_{0})>0$ such that

$$\left|\right|x\left({t}_{0}\right)\left|\right|<\delta \phantom{\rule{4pt}{0ex}}\Rightarrow \phantom{\rule{4pt}{0ex}}\left|\right|y\left(t\right)\left|\right|<\epsilon ,\phantom{\rule{4pt}{0ex}}\forall \phantom{\rule{4pt}{0ex}}t\ge {t}_{0}\ge 0$$uniformly y-stable, if it is y-stable, and for each ε, $\delta =\delta \left(\epsilon \right)$ is independent of ${t}_{0}$.

asymptotically y-stable, if it is y-stable, and for any ${t}_{0}$, there exists a positive constant $c=c\left({t}_{0}\right)>0$ such that every solution of Equation (215) for which $\left|\right|x\left({t}_{0}\right)\left|\right|<c$, satisfies $lim\left|\right|y\left(t\right)\left|\right|\to 0$ as $t\to \infty $.

uniformly asymptotically y-stable, if it is uniformly y-stable, and there is a positive constant c, independent of ${t}_{0}$, such that for all $\left|\right|x\left({t}_{0}\right)\left|\right|<c$, $lim\left|\right|y\left(t\right)\left|\right|\to 0$ as $t\to \infty $, uniformly in t. Meaning that for each $\eta >0$, there exists $T=T\left(\eta \right)>0$ such that

$$\left|\right|y\left(t\right)\left|\right|<\eta ,\phantom{\rule{4pt}{0ex}}\forall \phantom{\rule{4pt}{0ex}}t\ge {t}_{0}+T\left(\eta \right),\phantom{\rule{4pt}{0ex}}\forall \phantom{\rule{4pt}{0ex}}\left|\right|x\left({t}_{0}\right)\left|\right|<c$$

There’s a myriad of theorems regarding partial stability of systems. However, in most these works, the conditions on the Lyapunov function are too restrictive, rendering them ineffective for the adaptive control problem of our interest. Furthermore, the behavior of systems under perturbations is not abundantly studied when the best we can do is partial stability. However, we believe that adaptive control could in general benefit from this tool due to the nature of its stability. Further development of partial stability tools and its application to adaptive control is an interesting problem that can be addressed.

#### 7.4. Underactuated Systems

Systems that have fewer actuators than states to be controlled are referred to as underactuated. These systems are of interest from several viewpoints. First, in some applications it may not be possible to have actuators for all the desired states. Secondly, if an actuator failure happens the system descends into an underactuated mode. A successful control design for such situations can greatly enhance the safety and performance of the systems. Thirdly, a deliberate reduction in the number of actuators and reduce the manufacturing costs.

Underactuated systems have been studied for more than two decades now. Energy and passivity based control [107,108,109], energy shaping [110], and Controlled Lagrangians and Hamiltonians [111,112,113,114,115,116] are just a few methods among others proposed [117,118,119,120].

A survey on the methods and problems of underactuated systems requires a separate full length paper. However, we only look at them from the adaptive control perspective. Most of these proposed methods do not deal with uncertainties. Very few papers have been published that address the uncertainty issue in underactuated systems [121]. Addition of adaptation and adaptive control laws to methods that deal with underactuated systems is a subject that has been left mostly untouched. Research in these areas can greatly enhance the toolbox that we currently have for dealing with uncertain systems.

#### 7.5. Possible New Methods

Although very difficult, it is still possible to create novel paradigms for adaptive control. Recently, [122,123,124,125] attempted at a new paradigm of adaptive control by employing Extremum Seeking as a means of adaptation (rather than a means of optimization). Their method augments the Model Reference Adaptive Controller with an adaptation law using Extremum Seeking loops. The main difference between this approach and the mainstream adaptive methods is that the adaptation occurs in real-time and no mathematical adaptive laws need be derived. This makes the implementation simple; however, it also brings several downsides that have not been addressed. Extremum Seeking perturbs the system. Therefore, the system requires some inherent robustness to perturbations, or if this is not the case, the controller must provide such robustness. Due to the addition of deliberate perturbations to the system, one expects that PE conditions would be automatically satisfied, making the real-time identification of parameters possible. However, this does not seem to be the case. Further study needs to be done, before this new paradigm becomes an acceptable method.

## 8. Conclusions

We have reviewed the major history of adaptive control over the last century as well as several of the popular methods in adaptive control including: Model Reference Adaptive Control, Adaptive Pole Placement (Self-Tuning Regulators), Adaptive Sliding Mode Control, and Extremum Seeking. We presented the Model Reference Identification approaches through detailed analysis and examples, and briefly discussed their non-minimal realizations. It has been made clear that the application of adaptive systems can solve many interesting problems. The necessary tools for extending these methods to nonlinear systems were also discussed. Stability and robustness issues related to adaptive control methods were shown through analysis and example, followed by possible solutions using Robust Adaptive Control and Adaptive Robust Control methods. We also provided various perspectives in control, observation, and adaptive systems as well as some of the important open problems and the direction of future work. Despite the length of the open problems section, we have only covered a few problems where improvements can be made in adaptive control, showing that the field is still open contrary to common belief. There are still plenty of unsolved problems to be addressed in adaptive control and we hope to see more researchers address them in years to come.

## Acknowledgments

We would like to thank the reviewers for their comments and suggestions which helped improve the overall presentation of the paper.

## Author Contributions

William Black contributed the majority of the work with the exception of: the introduction, Extremum Seeking, discrete systems, developing analysis tools, underactuated systems, and possible new methods. Poorya Haghi contributed the sections on Extremum Seeking, developing analysis tools (including subsections), underactuated systems, and possible new methods. Kartik Ariyur contributed the introduction and discrete systems sections.

## Conflicts of Interest

The authors declare no conflict of interest, and the preceding work was not funded or influenced by any grants.

## References

- Astrom, K.; Wittenmark, B. Adaptive Control; Dover Publications, Inc.: Mineola, NY, USA, 2008. [Google Scholar]
- Whitaker, H.; Yamron, J.; Kezer, A. Design of Model Reference Adaptive Control Systems for Aircraft (Report R-164); MIT Press Instrumentation Laboratory: Cambridge, MA, USA, 1958. [Google Scholar]
- Osburn, P.; Whitaker, H.; Kezer, A. New Developments in the Design of Model Reference Adaptive Control Systems (Paper No. 61-39); Institute of the Aerospace Sciences: Easton, PA, USA, 1961. [Google Scholar]
- Parks, P. Lyapunov Redesign of Model Reference Adaptive Control Systems. IEEE Trans. Autom. Control
**1966**, 11, 362–367. [Google Scholar] [CrossRef] - Kalman, R.; Bertram, J. Control System Analysis and Design via the Second Method of Lyapunov I: Continuous-Time Systems. J. Basic Eng.
**1960**, 82, 371–393. [Google Scholar] [CrossRef] - Kalman, R.; Bertram, J. Control System Analysis and Design via the Second Method of Lyapunov II: Discrete-Time Systems. J. Basic Eng.
**1960**, 82, 394–400. [Google Scholar] [CrossRef] - Utkin, V. Sliding Modes and Their Application in Variable Structure Systems; MIR Publishers: Moscow, Russia, 1978. [Google Scholar]
- Utkin, V. Variable Structure Systems with Sliding Modes. IEEE Trans. Autom. Control
**1997**, 22, 212–222. [Google Scholar] [CrossRef] - Astrom, K.; Wittenmark, B. On Self-Tuning Regulators. Automatica
**1973**, 9, 185–199. [Google Scholar] [CrossRef] - Astrom, K.; Borisson, U.; Ljung, L.; Wittenmark, B. Theory and Applications of Self-Tuning Regulators. Automatica
**1977**, 13, 457–476. [Google Scholar] [CrossRef] - Narendra, K.; Annaswamy, A. Stable Adaptive Systems; Dover-Publications, Inc.: Mineola, NY, USA, 1989. [Google Scholar]
- Landau, Y. Adaptive Control: The Model Reference Approach; Marcel Dekker Inc.: New York, NY, USA, 1979. [Google Scholar]
- Kreisselmeier, G. Adaptive Observers with Exponential Rate of Convergence. IEEE Trans. Autom. Control
**1977**, 22, 2–8. [Google Scholar] [CrossRef] - Kudva, P.; Narendra, K. Synthesis of an Adaptive Observer Using Lyapunov’s Direct Method. Int. J. Control
**1973**, 18, 1201–1210. [Google Scholar] [CrossRef] - Luders, G.; Narendra, K. An Adaptive Observer and Identifier for Linear Systems. IEEE Trans. Autom. Control
**1973**, 18, 496–499. [Google Scholar] [CrossRef] - Luders, G.; Narendra, K. A New Canonical Form for an Adaptive Observer. IEEE Trans. Autom. Control
**1974**, 19, 117–119. [Google Scholar] [CrossRef] - Egardt, B. Stability of Adaptive Controllers; Springer-Verlag: Berlin, Germany, 1979. [Google Scholar]
- Ioannou, P.; Kokotovic, P. Adaptive Systems with Reduced Models; Springer-Verlag: Secaucus, NJ, USA, 1983. [Google Scholar]
- Narendra, K.; Annaswamy, A. A New Adaptive Law for Robust Adaptation without Persistent Excitation. IEEE Trans. Autom. Control
**1987**, 32, 134–145. [Google Scholar] [CrossRef] - Goodwin, G.; Mayne, D. A Parameter Estimation Perspective of Continuous Time Model Reference Adaptive Control. Automatica
**1987**, 23, 57–70. [Google Scholar] [CrossRef] - Peterson, B.; Narendra, K. Bounded Error Adaptive Control. IEEE Trans. Autom. Control
**1982**, 27, 1161–1168. [Google Scholar] [CrossRef] - Slotine, J.; Coetsee, J. Adaptive Sliding Controller Synthesis for Nonlinear Systems. Int. J. Control
**1986**, 43, 1631–1651. [Google Scholar] - Slotine, J.; Li, W. On the Adaptive Control of Robot Manipulators. Int. J. Robot. Res.
**1987**, 6, 49–59. [Google Scholar] [CrossRef] - Slotine, J.; Li, W. Applied Nonlinear Control; Prentice-Hall, Inc.: Upper Saddle River, NJ, USA, 1991. [Google Scholar]
- Boyd, S.; Sastry, S. On Parameter Convergence in Adaptive Control. Syst. Control Lett.
**1983**, 3, 311–319. [Google Scholar] [CrossRef] - Bai, E.; Sastry, S. Persistency of Excitation, Sufficient Richness, and Parameter Convergence in Discrete Time Adaptive Control. Syst. Control Lett.
**1985**, 6, 143–163. [Google Scholar] [CrossRef] - Boyd, S.; Sastry, S. Necessary and Sufficient Conditions for Parameter Convergence in Adaptive Control. Automatica
**1986**, 22, 629–639. [Google Scholar] [CrossRef] - Shimkin, N.; Feuer, A. Persistency of Excitation in Continuous-Time Systems. Syst. Control Lett.
**1987**, 9, 225–233. [Google Scholar] [CrossRef] - Krener, A. A Generalization of Chowś Theorem and the Bang Bang Theorem to Nonlinear Control Problems. SIAM J. Control
**1974**, 12, 44–52. [Google Scholar] [CrossRef] - Sussmann, H.; Jurdjevic, V. Controllability of Nonlinear Systems. J. Differ. Equat.
**1972**, 12, 95–116. [Google Scholar] [CrossRef] - Jurdjevic, V.; Sussmann, H. Control Systems on Lie Groups. J. Differ. Equat.
**1972**, 12, 313–329. [Google Scholar] [CrossRef] - Lobry, C. Controllability of Nonlinear Systems on Compact Manifolds. SIAM J. Control
**1974**, 12, 1–4. [Google Scholar] [CrossRef] - Sussmann, H. Existence and Uniqueness of Minimal Realizations of Nonlinear Systems. Math. Syst. Theor.
**1977**, 10, 263–284. [Google Scholar] [CrossRef] - Hermann, R.; Krener, A. Nonlinear Controllability and Observability. IEEE Trans. Autom. Control
**1977**, 22, 728–740. [Google Scholar] [CrossRef] - Haynes, G.; Hermes, H. Nonlinear Controllability via Lie Theory. SIAM J. Control
**1970**, 8, 450–460. [Google Scholar] [CrossRef] - Hwang, M.; Seinfeld, J. Observability of Nonlinear Systems. J. Optim. Theor. Appl.
**1972**, 10, 67–77. [Google Scholar] [CrossRef] - Kou, S.; Elliott, D.; Tarn, T. Observability of Nonlinear Systems. Inform. Contr.
**1973**, 22, 89–99. [Google Scholar] [CrossRef] - Inouye, Y. On the Observability of Autonomous Nonlinear Systems. J. Math. Anal. Appl.
**1977**, 60, 236–247. [Google Scholar] [CrossRef] - Griffith, E.; Kumar, K. On the Observability of Nonlinear Systems I. J. Math. Anal. Appl.
**1971**, 35, 135–147. [Google Scholar] [CrossRef] - Fitts, J. On the Observability of Nonlinear Systems with Applications to Nonlinear Regression Analysis. Inform. Sci.
**1972**, 4, 129–156. [Google Scholar] [CrossRef] - Yamamoto, Y.; Sugiura, I. Some Sufficient Conditions for the Observability of Nonlinear Systems. J. Optim. Theor. Appl.
**1974**, 13, 660–669. [Google Scholar] [CrossRef] - Brockett, R. System Theory on Group Manifolds and Coset Spaces. SIAM J. Control
**1972**, 10, 265–284. [Google Scholar] [CrossRef] - Krener, A. The High Order Maximal Principle and its Application to Singular Extremals. SIAM J. Contr. Optim.
**1977**, 15, 256–293. [Google Scholar] [CrossRef] - Brandin, V.; Kostyukovskii, Y.; Razorenov, G. Global Observability Conditions of Nonlinear Dynamical Systems. Autom. Remote Control
**1975**, 36, 1585–1591. [Google Scholar] - Krener, A. Approximate Linearization by State Feedback and Coordinate Change. Syst. Control Lett.
**1984**, 5, 181–185. [Google Scholar] [CrossRef] - Hunt, L.; Su, R.; Meyer, G. Global Transformations of Nonlinear Systems. IEEE Trans. Autom. Control
**1983**, 28, 24–31. [Google Scholar] [CrossRef] - van der Schaft, A. Linearization and Input-Output Decoupling for General Nonlinear Systems. Syst. Control Lett.
**1984**, 5, 27–33. [Google Scholar] [CrossRef] - Krener, A. Linearization by Output Injection and Nonlinear Observers. Syst. Control Lett.
**1983**, 3, 47–52. [Google Scholar] [CrossRef] - Isidori, A.; Krener, A. On Feedback Equivalence of Nonlinear Systems. Syst. Control Lett.
**1982**, 2, 118–121. [Google Scholar] [CrossRef] - Marino, R. On the Largest Feedback Linearizable Subsystem. Syst. Control Lett.
**1986**, 6, 345–351. [Google Scholar] [CrossRef] - Su, R. On the Linear Equivalents of Nonlinear Systems. Syst. Control Lett.
**1982**, 2, 48–52. [Google Scholar] [CrossRef] - Boothby, W. Some Comments on Global Linearization of Nonlinear Systems. Syst. Control Lett.
**1984**, 4, 143–147. [Google Scholar] [CrossRef] - Krstic, M.; Kanellakopoulos, I.; Kokotovic, P. Nonlinear and Adaptive Control Design; John Wiley and Sons, Inc.: New York, NY, USA, 1995. [Google Scholar]
- Yao, B. Adaptive Robust Control of Nonlinear Systems with Application to Control of Mechanical Systems. Ph.D. Thesis, UC Berkeley, Berkeley, CA, USA, 1996. [Google Scholar]
- Marino, R.; Tomei, P. Global Adaptive Observers for Nonlinear Systems via Filtered Transformations. IEEE Trans. Autom. Control
**1992**, 37, 1239–1245. [Google Scholar] [CrossRef] - Leblanc, M. Sur l’electrification des chemins de fer au moyen de courants alternatifs de frequence elevee. Revue Generale de l’Electricite
**1922**, 12, 275–277. [Google Scholar] - Tan, Y.; Moase, W.; Nesic, D.; Mareels, I. Extremum Seeking From 1922 to 2010. In Proceedings of the 29th Chinese Control Conference, Beijing, China, 29–31 July 2010; pp. 14–26.
- Krstic, M.; Wang, H. Stability of Extremum Seeking Feedback for General Nonlinear Dynamic Systems. Automatica
**2000**, 36, 595–601. [Google Scholar] [CrossRef] - Choi, J.; Krstic, M.; Ariyur, K.; Lee, J. Extremum Seeking Control for Discrete-Time Systems. IEEE Trans. Autom. Control
**2002**, 47, 318–323. [Google Scholar] [CrossRef] - Ariyur, K.; Krstic, M. Slope Seeking: A Generalization of Extremum Seeking. Int. J. Adapt. Contr. Signal Process.
**2004**, 18, 1–22. [Google Scholar] [CrossRef] - Tan, I.; Nesic, D. On Non-Local Stability Properties of Extremum Seeking Control. Automatica
**2006**, 42, 889–903. [Google Scholar] [CrossRef] - Tan, Y.; Nešić, N.; Mareels, I.M.Y.; Astolfi, A. Global Extremum Seeking in the Presence of Local Extrema. Automatica
**2009**, 45, 245–251. [Google Scholar] [CrossRef] - Tunay, I. Antiskid Control for Aircraft via Extremum Seeking. In Proceedings of the American Control Conference, Arlington, VA, USA, 25–27 June 2001; pp. 665–671.
- Zhang, C.; Ordornez, R. Numerical Optimization-Based Extremum Seeking Control with Application to ABS Design. IEEE Trans. Autom. Control
**2007**, 52, 454–467. [Google Scholar] [CrossRef] - Killingsworth, N.; Krstic, M.; Flowers, D.; Espinoza-Loza, F.; Ross, T. HCCI Engine Combustion Timing Control: Optimizing Gains and Fuel Consumption via Extremum Seeking. Contr. Syst. Mag.
**2006**, 17, 70–79. [Google Scholar] [CrossRef] - Binetti, P.; Ariyur, K.; Krstic, M.; Bernelli, F. Formation Flight Optimization Using Extremum Seeking Feedback. J. Guid. Contr. Dynam.
**2003**, 26, 132–142. [Google Scholar] [CrossRef] - Marcos, N.; Guay, M.; Dochain, D.; Zhang, T. Adaptive Extremum-Seeking Control of a Continuous Stirred Tank Bioreactor with Haldane’s Kinetics. Process Contr.
**2004**, 14, 317–328. [Google Scholar] [CrossRef] - Schuster, E.; Xu, C.; Torres, N.; Morinaga, E.; Allen, C.; Krstic, M. Beam Matching Adaptive Control via Extremum Seeking. Nucl. Instr. Meth. Phys. Res.
**2007**, 581, 799–815. [Google Scholar] [CrossRef] - Killingsworth, N.; Krstic, M. PID Tuning Using Extremum Seeking. IEEE Trans. Contr. Syst. Tech.
**2009**, 17, 1350–1361. [Google Scholar] [CrossRef] - McCulloch, W.; Pitts, W. A Logical Calculus of the Ideas Immanent in Nervous Activity. Bull. Math. Biophys.
**1943**, 5, 115–133. [Google Scholar] [CrossRef] - Hebb, D. The Organization of Behavior; Wiley: Hoboken, NJ, USA, 1949. [Google Scholar]
- Rosenblatt, F. The Perceptron: A Perceiving and Recognizing Automaton (Report: 85-460-1); Cornell Aeronatical Laboratory: Buffalo, NY, USA, 1957. [Google Scholar]
- Widrow, B.; Hoff, M. Adaptive Switching Circuits. IRE WESCON Convention Record. 1960, pp. 96–104. Available online: http://www-isl.stanford.edu/~widrow/papers/c1960adaptiveswitching.pdf (accessed on 6 August 2014). [Google Scholar]
- Minsky, M.; Papert, S. Perceptrons; MIT Press: Cambridge, MA, USA, 1969. [Google Scholar]
- Hopfield, J.J. Neural Networks and Physical Systems with Emergent Collective Computational Abilities. Proc. Nat. Acad. Sci. USA
**1982**, 79, 2554–2558. [Google Scholar] [CrossRef] - Bryson, A.; Denham, W.; Dreyfus, S. Optimal Programming Problems with Inequality Constraints I: Necessary Conditions for Extremal Solutions. AIAA J.
**1963**, 1, 2544–2550. [Google Scholar] [CrossRef] - Broomhead, D.; Lowe, D. Multivariable Functional Interpolation and Adaptive Networks. Complex Syst.
**1988**, 2, 321–355. [Google Scholar] - Cortes, C.; Vapnik, V. Support-Vector Networks. Mach. Learn.
**1995**, 20, 273–297. [Google Scholar] [CrossRef] - Hinton, G.; Osindero, S.; Teh, Y. A fast learning algorithm for deep belief nets. J. Neural Comput.
**2006**, 18, 1527–1554. [Google Scholar] [CrossRef] [PubMed] - Cao, C.; Hovakimyan, N. Design and Analysis of a Novel L1 Adaptive Control Architecture, Part I: Control Signal and Asymptotic Stability. In Proceedings of the American Control Conference, Minneapolis, MN, USA, 14–16 June 2006; pp. 3397–3402.
- Cao, C.; Hovakimyan, N. Design and Analysis of a Novel L1 Adaptive Control Architecture, Part II: Guaranteed Transient Performance. In Proceedings of the American Control Conference, Minneapolis, MN, USA, 14–16 June 2006; pp. 3403–3408.
- Hovakimyan, N.; Cao, C. Design and Analysis of a Novel L1 Adaptive Control Architecture with Guaranteed Transient Performance. IEEE Trans. Autom. Control
**2008**, 53, 586–591. [Google Scholar] - Hovakimyan, N.; Cao, C. Stability Margins of L1 Adaptive Control Architecture. IEEE Trans. Autom. Control
**2010**, 55, 480–487. [Google Scholar] - Hovakimyan, N.; Cao, C. L1 Adaptive Control Theory: Guaranteed Robustness with Fast Adaptation; SIAM: Philadelphia, PA, USA, 2010. [Google Scholar]
- Hovakimyan, N.; Cao, C.; Kharisov, E.; Xargay, E.; Gregory, I. L1 Adaptive Control for Safety-Critical Systems. IEEE Contr. Syst. Mag.
**2011**, 54–104. [Google Scholar] [CrossRef] - Ioannou, P.; Annaswamy, A.; Narendra, K.; Jafari, S.; Rudd, L.; Ortega, R. L1 Adaptive Control: Stability, Robustness, and Interpretations. IEEE Trans. Autom. Control
**2014**, 59, 3075–3080. [Google Scholar] [CrossRef] - Ortega, R.; Panteley, E. Comments on L1 adaptive control: stabilization mechanism, existing conditions for stability and performance limitations. Int. J. Control
**2014**, 87, 581–588. [Google Scholar] [CrossRef] - Hovakimyan, N. L1 Adaptive Control. Available online: http://naira.mechse.illinois.edu/paper-misperceptions-l1 (accessed on 6 August 2014).
- Souanef, T.; Fichter, W. Comments on L1 Stability Condition. Available online: http://naira.mechse.illinois.edu/paper-misperceptions-l1 (accessed on 6 August 2014).
- Khalil, H. Nonlinear Systems; Prentice-Hall, Inc.: Upper Saddle River, NJ, USA, 2002. [Google Scholar]
- Ioannou, P.; Sun, J. Robust Adaptive Control; Dover Publications, Inc.: Mineola, NY, USA, 2012. [Google Scholar]
- Ariyur, K.; Krstic, M. Real Time Optimization by Extremum Seeking Control; John Wiley and Sons, Inc.: Hoboken, NJ, USA, 2003. [Google Scholar]
- Isidori, A. Nonlinear Control Systems; Springer-Verlag: London, UK, 1995. [Google Scholar]
- Sastry, S.; Bodson, M. Adaptive Control: Stability, Convergence, and Robustness; Dover Publications, Inc.: Mineola, NY, USA, 2011. [Google Scholar]
- Rohrs, C.; Valavani, L.; Athans, M.; Stein, G. Robustness of Continuous-Time Adaptive Control Algorithms in the Presence of Unmodeled Dynamics. IEEE Trans. Autom. Control
**1985**, 30, 881–889. [Google Scholar] [CrossRef] - Anscombe, F. Graphs in Statistical Analysis. Am. Stat.
**1973**, 27, 17–21. [Google Scholar] - Teel, A.; Peuteman, J.; Aeyels, D. Global Asymptotic Stability for the Averaged Implies Semi-Global Practical Stability for the Actual. In Proceedings of the 37th IEEE Conference on Decision and Control, Tampa, FL, USA, 16–18 December 1998; pp. 1458–1463.
- Teel, A.; Peuteman, J.; Aeyels, D. Semi-global practical asymptotic stability and averaging. Syst. Control Lett.
**1999**, 37, 329–334. [Google Scholar] [CrossRef] - Rumyantsev, V. Partial stability of motion. (in Russian). Vestn. Mosk. Gos. Univ. Mat. Mekh. Fiz. Astronom. Khim.
**1957**, 4, 9–16. [Google Scholar] - Rumyantsev, V. The asymptotic stability and instability of motion with respect to part of the variables. (in Russian). Priklad. Matem. Mekh
**1971**, 35, 138–143. [Google Scholar] - Vorotnikov, V. Problems of stability with respect to part of the variables. J. Appl. Math. Mech.
**1999**, 63, 695–703. [Google Scholar] [CrossRef] - Vorotnikov, V. Theory of stability with respect to part of the variables and the problem of coordinate synchronization for dynamical systems. Dokl. Phys.
**2000**, 45, 685–689. [Google Scholar] [CrossRef] - Vorotnikov, V. Partial stability and control: The state-of-the-art and development prospects. Autom. Remote Control
**2005**, 66, 511–561. [Google Scholar] [CrossRef] - Vorotnikov, V. Optimal stabilization of motion with respect to some of the variables. Prikl. Matem. Mekhan.
**1990**, 54, 597–605. [Google Scholar] [CrossRef] - Vorotnikov, V. On the theory of partial stability. Prikl. Matem. Mekhan
**1995**, 59, 525–531. [Google Scholar] [CrossRef] - Alekseyeva, C.; Vorotnikov, V.; Feofanova, V. Problems of the partial stability and detectability of dynamical systems. J. Appl. Math. Mech.
**2007**, 71, 869–879. [Google Scholar] [CrossRef] - Ortega, R.; van der Schaft, A.; Mareels, I.; Maschke, B. Putting energy back in control. IEEE Contr. Syst. Mag.
**2001**, 21, 18–33. [Google Scholar] [CrossRef] - Ortega, R.; van der Schaft, A.; Maschke, B. Interconnection and damping assignment passivity-based control of port-controlled hamiltonian systems. Automatica
**2002**, 38, 585–596. [Google Scholar] [CrossRef] - Rodriguez, H.; Ortega, R. Stabilization of electromechanical systems via interconnection and damping assignment. Int. J. Robust Nonlinear Control
**2003**, 13, 1095–1111. [Google Scholar] [CrossRef] - Zhong, W.; Rock, H. Energy and Passivity Based Control of the Double Inverted Pendulum on a Cart. In Proceedings of IEEE International Conference on Control Applications, Mexico City, Mexico, 5–7 September 2001; pp. 896–901.
- Bloch, A.; Leonard, N.; Marsden, J. Stabilization of Mechanical Systems Using Controlled Lagrangians. In Proceedings of the 36th IEEE Conference on Decision and Control, San Diego, CA, USA, 10–12 December 1997; pp. 2356–2361.
- Bloch, A.; Leonard, N.; Marsden, J. Matching and Stabilization by the Method of Controlled Lagrangians. In Proceedings of the 37th IEEE Conference on Decision and Control, Tampa, FL, USA, 16–18 December 1998; pp. 1446–1451.
- Bloch, A.; Leonard, N.; Marsden, J. Controlled Lagrangians and the stabilization of mechanical systems I: The first matching theorem. IEEE Trans. Autom. Control
**2000**, 45, 2253–2270. [Google Scholar] [CrossRef] - Bloch, A.; Leonard, N.; Marsden, J. Potential Shaping and the Method of Controlled Lagrangians. In Proceedings of the 38th IEEE Conference on Decision and Control, Phoenix, AZ, USA, 7–10 December 1999; pp. 1653–1657.
- Haghi, P.; Ghaffari-Saadat, M. Challenges in the Stabilization of a Satellite Using Controlled Lagrangians I: Gravitation. In Proceedings of the 18th IEEE International Conference on Control Applications, St.Petersburg, Russia, 8–10 July 2009; pp. 764–769.
- Haghi, P.; Ghaffari-Saadat, M. Challenges in the Stabilization of a Satellite Using Controlled Lagrangians II: Unbalance. In Proceedings of the 18th IEEE International Conference on Control Applications, St.Petersburg, Russia, 8–10 July 2009; pp. 770–775.
- Olfati-Saber, R. Control of Underactuated Mechanical Systems with Two Degrees of Freedom and Symmetry. In Proceedings of the American Control Conference, Chicago, IL, USA, 28–30 June 2000; pp. 4092–4096.
- Olfati-Saber, R. Normal forms for underactuated mechanical systems with symmetry. IEEE Trans. Autom. Control
**2002**, 47, 305–308. [Google Scholar] [CrossRef] - Spong, M. The swing up control problem for the acrobot. IEEE Contr. Syst. Mag.
**1995**, 15, 49–55. [Google Scholar] [CrossRef] - Choukchou-Braham, A.; Cherki, B.; Djemai, M. A Backstepping Procedure for a Class of Underactuated System with Tree Structure. In Proceedings of the International Conference on Communications, Computing and Control Applications, Hammamet, Tunisia, 3–5 March 2011; pp. 1–6.
- Gu, Y. A Direct Adaptive Control Scheme for Under-Actuated Dynamic Systems. In Proceedings of the 32nd Conference on Decision and Control, San Antonio, TX, USA, 15–17 December 1993; pp. 1625–1627.
- Haghi, P.; Ariyur, K. On the Extremum Seeking of Model Reference Adaptive Control in Higher-Dimensional Systems. In Proceedings of the American Control Conference, San Francisco, CA, USA, 29 June–1 July 2011; pp. 1176–1181.
- Haghi, P.; Ariyur, K. Adaptive First Order Nonlinear Systems Using Extremum Seeking. In Proceedings of the 50th Annual Allerton Conference on Communication Control, Monticello, IL, USA, 1–5 October 2012; pp. 1510–1516.
- Haghi, P.; Ariyur, K. Adaptive Feedback Linearization of Nonlinear MIMO Systems Using ES-MRAC. In Proceedings of the American Control Conference, Washington, DC, USA, 17–19 June 2013; pp. 1828–1833.
- Haghi, P.; Ariyur, K. A Novel Approach to Parameter Adaptation Using Extremum Seeking. Int. J. Control
**2014**. submitted. [Google Scholar]

© 2014 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/).