This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/).

We survey some of the rich history of control over the past century with a focus on the major milestones in adaptive systems. We review classic methods and examples in adaptive linear systems for both control and observation/identification. The focus is on linear plants to facilitate understanding, but we also provide the tools necessary for many classes of nonlinear systems. We discuss practical issues encountered in making these systems stable and robust with respect to additive and multiplicative uncertainties. We discuss various perspectives on adaptive systems and their role in various fields. Finally, we present some of the ongoing research and expose problems in the field of adaptive control.

Any system—engineering, natural, biological, or social—is considered adaptive if it can maintain its performance, or survive in spite of large changes in its environment or in its own components. In contrast, small changes or small ranges of change in system structure or parameters can be treated as system uncertainty, which can be remedied in dynamic operation either by the static process of design or the design of feedback and feed-forward control systems. By systems, we mean those in the sense of classical mechanics. The knowledge of initial conditions and governing equations determines, in principle, the evolution of the system state or degrees of freedom (a rigid body for example has twelve states–three components each of position, velocity, orientation and angular velocity). All system performance, including survival or stability, is in principle expressible as functions or functionals of system state. The maintenance of such performance functions in the presence of large changes to either the system or its environment is termed adaptation in the control systems literature. Adaptation of a system, as in biological evolution, can be of two kinds–adapting the environment to maintain performance, and adapting itself to environmental changes. In all cases, adaptive systems are inherently nonlinear, as they possess parameters that are functions of their states. Thus, adaptive systems are simply a special class of nonlinear systems that measure their own performance, operating environment, and operating condition of components, and adapt their dynamics, or those of their operating environments to ensure that measured performance is close to targeted performance or specifications.

The organization of the paper is as follows:

The first notable and widespread use of ‘adaptive control’ was in the aerospace industry during the 1950s in an attempt to further the design of autopilots [

The late 1950s and early 1960s saw the formulation of the state-space system representation as well as the use of Lyapunov stability for general control systems, by both Kalman and Bertram [

These three decades were also a fertile time for nonlinear systems theory. Kalman published his work on controllability and observability for linear systems in the early 1960s, and it took about 10 years to extend these ideas to nonlinear systems through the use of Lie theory [

Another side of the story is related to Extremum Seeking control and Neural Networks, whose inception came earlier but development and widespread use as non-model and non-Lyapunov based adaptation methods took much longer. The first known appearance of Extremum Seeking (ES) in the literature was published by LeBlanc in 1922 [

The idea of Neural Networks as a mathematical logic system was developed during the 1940s by McCulloch and Pitts [

In terms of the most recent developments (2006–Present) in adaptive control, the situation is a little complicated. The period from 2006 to 2011 saw the creation of the L_{1}-AC method [

The goal of this section is to provide a survey of the more popular methods in adaptive control through analysis and examples. The following analyses and examples assume a familiarity with Lyapunov stability theory, but an in-depth treatise on the subject may be found in [

Model Reference Adaptive Control, or MRAC, is a control system structure in which the desired performance of a minimum phase (stable zeros) system is expressed in terms of a reference model that gives a desired response to a command signal. The command signal is fed to both the model as well as the actual system, and the controller adapts the gains such that the output errors are minimized and the actual system responds like the model (desired) system. We show the block diagram for this structure in

Model reference adaptive control structure.

To show a simple example of MRAC, consider a simple first order LTI system

Now we match the equations with

At this point, we may choose to make our adaptive controller ‘direct’ or ‘indirect’. The direct method adapts controller parameters directly, so

Defining the parameters

We the define the output error as the difference between the system and the reference model, that is

We substitute the control law into the error dynamics equation, and attempt to find a solution such that the error will be driven to zero, and parameter errors will go to zero as well. The error dynamics are written as

Using the relation

Finally we get a representation that only relies on the parameter errors,

We construct a Lyapunov candidate

We can see that the first error term will be stable, and the entire system will be stable if we can force the other terms to be zero. We then simplify the expression and attempt to solve for the parameter adaptation. It is important to note here that since

The parameter adaptation law is, as we might think, a negative gradient descent relation that is a function of the output error of the system,

It should be clear that in the case of a system of dimension larger than one, the preceding analysis will require linear algebra as well as the need to solve the Lyapunov equation

Output response for MRAC.

Control input for MRAC.

Parameter estimates for MRAC.

MRAC simulink structure.

Adaptive Pole Placement Control (APPC) methods represent the largest class of adaptive control methods [

If we want our system to follow a specified model we construct the relation

Indirect adaptive pole placement structure.

So far we have not made any assumptions about the stability of the system, only that

Since

Going back to the numerator, since we cannot cancel

We did not need the assumption of a minimum phase system as with MRAC because we did not cancel

Consider a second order system of relative degree one and its desired reference model

Choosing not to cancel the zero, and using the minimal design and causality conditions

While both direct and indirect adaptation methods may be used with Self-Tuning Regulators, the indirect option is typically chosen. This is because solving for adaptation laws in the direct sense can become intractable since we may choose not to cancel zeros, which may lead to parametric nonlinearities depending on the system. Performing system identification through something like Recursive Least Squares (RLS) and having static control relationships is a much easier alternative that works provided signals are persistently exciting enough and parameters values are not ill-conditioned. We consider the popular example of a DC motor [

On-line system identification is done with the RLS algorithm

Output response for indirect APPC.

Plant parameter estimates for indirect APPC.

Control input for indirect APPC.

Adaptive Sliding Mode Control, or ASMC, is a variable structure control method that specifies a manifold or surface along with the system will operate or ‘slide’. When the performance deviates from the manifold, the controller provides an input in the direction back towards the manifold to force the system back to the desired output. We show the control structure in

Adaptive sliding mode control structure.

We start by guaranteeing the sliding dynamics are always driven towards a specified sliding surface, that is

The above simplifies to

However, we can see that if

Then we define some stable manifold we want our system to slide along and determine its dynamics, in this case:

We replace the surface dynamics with the discrete term that represents the trajectory motion towards the manifold which is

Solving for

Then replacing the parameters with their estimates, we attempt to substitute

In order to solve for an adaptation law that forces the error to go to zero, we consider the Lyapunov candidate

Output response for ASMC.

Control input for ASMC.

Parameter estimates for ASMC.

Extremum Seeking is a strong optimization tool that is widely used in industry and more often categorized as a method of adaptive control. The reason is that Extremum Seeking is capable of dealing with unknown plants whose input to output maps possess an extremum (a minimum or a maximum), and this extremum depends on some parameter. The way Extremum Seeking works is by measuring the gradient of the output through adding (sinusoidal) perturbations to the system. This makes Extremum Seeking a gradient estimate method, with the additional advantage that the estimate happens in real-time. This has led to many industrial applications. The problem is formulated as follows. Suppose we have an unknown map

Extremum seeking structure.

Suppose that the map is described by

Output response for ES.

Parameter convergence for ES.

Thus far we have only considered instances where full state measurement is available, a rather unlikely scenario for controlling real systems. Work on the adaptive observer problem has been on-going since the early seventies, but typically does not receive as much attention as the control problem. Adaptive identification methods not only solve the output feedback problem for adaptive systems, but they may also be used in the field of health monitoring. The various observers take advantage of a variety of canonical forms, each having their own advantages and disadvantages for implementation. Two realizations of an adaptive Luenberger-type observer will be discussed in detail, while the others briefly mentioned are non-minimal extensions of these two basic forms. The adaptive observer/identifier to be discussed can be visualized as an MRAS (Model Reference Adaptive System) structure shown in

MRAS observer structure.

The following introduces one of the first adaptive observer designs, originally published by Kudva and Narenda [

We may then equivalently express this system (for reasons seen later) as

Define the state observer to have a similar structure with state and parameter estimates

The purpose of the two additional signals

In order to design the governing equations for our additional signals

The following equality

The Lyapunov candidate is chosen to be

We are still unsure about the first term, but we may choose the adaptation laws as

The first term is proven to be stable by choosing

The adaptive laws then change to

The gains are chosen to be

Observed state estimate for adaptive observer type

Unobserved state estimate for adaptive observer type

State coefficient estimates for adaptive observer type

Control coefficient estimates for adaptive observer type

The next adaptive observer we will design is based on systems of the form

It was shown in [

The transformation is finally defined as

Choose

It follows that

The standard Lyapunov candidate will then lead to adaptive laws similar to the previous section. Notice that we were able to make it to this point without the need for complicated stabilizing signals, without needing to design

The previous sections focused on minimal adaptive observer forms and detailed analysis. We now provide a brief exposure to some of the concepts of non-minimal observer form. Shortly after the progress in creating the minimal observers from the previous sections, Luders and Narendra developed a non-minimal representation [

Choosing the standard Lyapunov candidate

Two other non-minimal observers that came from this design were developed by Kreisselmeier [

We have briefly introduced some of the non-minimal realizations of adaptive observers for LTI systems, but a much more detailed presentation and analysis of these methods may be found in [

Nonlinear behavior is one of the most (if not

Nonlinear behaviors are sometimes divided into two classes: ‘hard’ and ‘soft’. Soft nonlinearities are those which may be linearly approximated, such as

Examples of non-lipschitz nonlinearities. (

Feedback Linearization is a method in which a nonlinear coordinate transformation between the input and output is found such that the transformed system is linear along all trajectories. The first r-derivatives of the output are the coordinate transformations, and the coordinate transformation is required to be a diffeomorphism (invertible and smooth). We then design the input such that the

For input-output linearization we essentially take derivatives of

Our goal is to replace the

This transforms the system dynamics into

We can see that an easy solution is

The Jacobian is regular for all

Backstepping was created shortly after Feedback Linearization to address some of the aforementioned issues. It is often called a ‘Lyapunov-Synthesis’ method, because it recursively uses Lyapunov’s second method to design virtual inputs all the way back to the original control input. The approach removes the restrictions of having to know the system exactly and remove all nonlinearities because we use Lyapunov’s method at each step to guarantee stability. Backstepping is typically applied to systems of the triangular form:

We view each state equation in this structure as its own subsystem, where the term coupled to the next state equation is viewed as a virtual control signal. An ideal value for signal is constructed, and the difference between the ideal and actual values is constructed such that the error is exponentially stable by Lyapunov. We may the error of the system to be the state or the difference between the system and a model in the cases of regulation and model following respectively. Assuming a model following problem, we consider a Lyapunov candidate for the first subsystem

Following Lyapunov’s method, we would take the derivative which results in

The

This continues on with

One of the main advantages to Backstepping is that we may leave helpful nonlinear terms in the equations. In Feedback Linearization, we have to cancel out all of the nonlinearities using the control input and various integrators. This makes Backstepping much more robust than Feedback Linearization, and also allows us to use nonlinear damping for control augmentation as well as extended matching for adaptation with tuning functions [

First consider the LTI system:

For observability, we want to see if we find the initial conditions based on our outputs which is represented by

The observability matrix for this system is then

The observability matrix must be full rank for the system to be fully observable, that is

This is a corollary to saying the system dynamics are injective, or a one-to-one mapping. This states that if the function

The system is said to be observable if

We can see that this is true if we substitute

Consider the same LTI system dynamics

The reachable subspace for the LTI system by the Cayley-Hamilton theorem is

Expanding out the matrix exponential we get the controllability matrix

As with the observability matrix we say the system is controllable if this matrix is full rank, shown as

This is a corollary to saying the system dynamics are surjective, a mapping that is onto. The function

We want to find the controllability or reachability for the nonlinear system, so we look at the reachability equation above. After expanding the matrix exponential, we will get a collection of matrix multiplications of

Like the observability problem, controllability is obtained if the matrix is full rank, and we can easily check this formulation and see that it even works for linear systems by substituting

Thus far we have not imposed any conditions on the external signals for stability, convergence, or robustness. In doing this we will explore another difference between the direct and indirect methods, as well as the overall robustness of adaptive controllers to external disturbances and unmodeled dynamics. First consider the problem of finding a set of parameters that best fit a data set. We have the actual and estimated outputs

The goal is to minimize the error

We minimize the squared error by differentiating with respect to the parameter estimates

In order for the estimates to be valid, the matrix

We follow the same approach as before by differentiating with respect to the parameter estimate and equating with zero,

The parameter estimate may then be expressed using

Apart from the mathematical condition itself, signals are often determined to be persistently exciting enough based on the number of fundamental frequencies contained within the signal. From the frequency space perspective signals may be approximated by sums of sines and cosines of these frequencies, and

However, the real plant is

A direct MRAC controller is constructed assuming the plant is of first order, and the initial parameter estimates are

Output response for Rohrs’ first example.

Parameter estimates for Rohrs’ first example.

Output response for Rohrs’ second example.

Parameter estimates for Rohrs’ second example.

Output response for Rohrs’ third example.

Parameter estimates for Rohrs’ third example.

Now that we’ve discussed some of the issues related to adaptation in the presence of noise and disturbances, we present some of the adaptation law modifications that were developed to handle these issues. Control law modifications are mentioned in the next section. First consider the reference model and system with uncertainty:

We construct the control law in the same fashion as MRAC

We next consider the derivative of the standard quadratic Lyapunov candidate

After plugging in the traditional Lyapunov adaptation law, we may convert this to the following inequality

Anytime the error norm is within this bound,

After plugging this into the derivative of the Lyapunov function we can construct the inequality

As long as the tracking and parameter errors are outside of these limits the system will remain stable. This method helped to remove the requirement on a-priori disturbance bounds, but presented another problem. If we look at the adaptation law itself, when the tracking error becomes small the trajectory of parameter estimates becomes

The other main drawback for the previous methods is that they can slow down the adaptation, which is the opposite of what we want. The Projection Method [

The discontinuous projection operator has similar chattering problems to adaptation with a deadzone, which led to the creation of smoothing terms similar to the saturation term in sliding mode control [

An alternative to just modifying adaptation laws, is to also experiment with various non-equivalent control structures. Yao [

Adaptive robust control structure.

Assuming that the parameters lie in a compact set and the uncertainty is bounded, we start with a similar system as the previous section (input parameter removed for convenience)

We typically separate the robust input into two parts

We want to design the robust input

An example of a function

As before in the sliding mode section, we will want to replace the

Our goal here is to reduce the derivative of the Lyapunov stability equation to

Using the final value theorem we get an error bound for the robust portion of control

In order to satisfy the robust tracking error bound as well as its transient, we require the robust portion of the control law to satisfy two requirements

If we choose

Using the Discontinuous Projection Method for on-line adaptation (

Plugging in the error dynamics and adaptation law we get

It is clear that the first term is negative definite, the second term is negative definite from our robust control law design conditions, and the third term is also negative definite due to the properties of the Discontinuous Projection law. Using this method in conjunction with Backstepping for nonlinear systems makes it extremely powerful especially due to bounded transient performance, but at the expense of a more complicated analysis. The interested reader may refer to [

Adaptive systems are inherently nonlinear, as must be evident by now, and hence their behavior cannot be captured without sufficiently large sampling frequencies. Hence, implementations of adaptation in engineering systems are based on high frequency sampling of system performance and environment, and therefore based on continuous system models and control designs. In those cases where the system is inherently discrete event, e.g., an assembly line, adaptation occurs from batch to batch as in Iterative Learning Control. The most widely used adaptive control is based on a combination of online system identification and Model Predictive Control. This method is ideal for process plants where system dynamics are slow compared to the sampling frequencies, speed of computation, and that of actuation. The actuators in these plants do not have weight or space constraints, and hence are made extremely powerful to handle a wide range of outputs and therefore of uncertainty and transient behavior. In this case a plant performance index such as time of reaction, or operating cost is minimized subject to the constraints of plant state, actuator and sensor constraints, and the discretized dynamics of the plant. Thus, this method explicitly permits incorporation of real world constraints into adaptation. The online system identification is performed using several tiny sinusoidal or square wave perturbations to actuator commands, and measuring resultant changes of performance. It is ideal for the process industry because of the great variability of the physical and chemical properties of feedstock, such as coal, iron ore, or crude oil.

In general there are two types of models: ‘grey box’ and ‘black box’. Black box modeling indicates that hardly anything is known about the system except perhaps the output signal given some input. We have no idea what is inside the black box, but we may be able to figure it out based on the input/output relation. Grey box modeling is a situation where some a-priori information about the system is known and can be used. For example, knowing the design of the system and being able to use Newton’s laws of motions to understand the dominating dynamics would be considered grey box modeling. Even though we may know a lot about the system, there is no way we can know everything about it, hence the ‘grey’ designation. Uncertainty in our models and parameters are fundamental problems that we must unfortunately deal with in the real world.

The presence of uncertainty and modeling difficulties often leads to the field of machine learning, the process of machines ‘learning’ from data. Put simply, we approximate a relationship (function) between data points, and then hopefully achieving correct predictions based on its function approximation if it is to be used in control. There are many methods associated with machine learning (e.g., Neural Networks, Bayesian Networks,

An example of a system where machine learning would be advantageous is a highly articulated robot such as a humanoid robot. A system of this type has too many states to handle in practice, not to mention complexities due to nonlinearities and coupling, so a machine learning algorithm would be very advantageous here. However, since the algorithm can only improve the function approximation by analyzing data, this approach obviously requires a lot of ‘learning’ time as well as computational resources. Therefore the system (or its data) has to be available to be tested in a controlled environment which is not always possible (consider a hypersonic vehicle). The other disadvantage to certain types of machine learning algorithms is illustrated by Anscombe’s Quartet, shown in

Anscombe’s quartet.

Anscombe’s Quartet was a famous data set given by Francis Anscombe [

The fields of differential topology, differential geometry, and Lie theory are considerably more abstract than many are used to, but may offer a tremendous advantage to visualizing and thinking about more complex spaces. Topology is the study of the properties of a mathematical space under continuous deformations, and its differential counterpart is the study of these with respect to differentiable spaces and functions/vector fields. Differential geometry is quite similar to differential topology, but typically considers cases in which the spaces considered are equipped with metrics that define various local properties such as curvature. Lie theory is the mathematical theory that encompasses Lie algebras and Lie groups. Lie groups are groups that are differential manifolds, and Lie algebras define the structures of these groups. The combination of these fields allows us to classify and perform calculus on these spaces abstractly, regardless of their dimension. An example of the advantage of even understanding some of the most basic concepts is finding the curvature of a higher dimensional space. For problems in 3-dimensional space the cross-product can be used to find curvature, but the cross-product itself is only defined up to this dimension and thus cannot be used for n-dimensional problems. Differential geometry allows one to define curvature in an abstract sense for systems of any order, which will also be equivalent to the cross-product in 3-dimensional problems. We have already used some methods from these fields earlier in this paper, such as finding diffeomorphisms for Feedback Linearization and the Lie bracket for observability/controllability in nonlinear systems. For brevity we did not include the backgrounds of all of these fields, but the interested reader may refer to for more complete treatments of the subjects.

Despite the extensive decades of research in adaptive control and adaptive systems, there are still many unsolved problems to be addressed. Some of these problems may not be ‘low hanging fruit’; however, their solution could lead to important applications. We discuss a few of these problems in this section. First, we discuss nonlinear regression problems, followed by transient performance issues. Finally qualify developing analysis tools that can ease the proofs of stability and boundedness for complex systems, as well as developing novel paradigms for adaptive control.

As with systems that are nonlinear in their states, there is no general nonlinear regression approach, it consists of a toolbox of methods that depend on the problem. Parameters appear linearly for many systems, but there are some systems (especially biological) where they appear nonlinearly. This happens to be one of the largest obstacles in the implementation of neural networks. In many neural network configurations, some parameters show up nonlinearly, and are typically chosen to be constant (lack of confidence guarantees), or an MIT rule approach is used (lack of stability guarantees). Even if network input and output weights show up linearly and adaptation laws are chosen using Lyapunov, it is all for naught if the nonlinear parameters are incorrect which can require extensive trial and error tuning by the designer or large amounts of training. An example of how control and identification may be used in an MRAC-like system is given in

The Radial Basis Function Neural Network design is a good example of when we might encounter difficulties related to nonlinear parameters. A basic RBFNN controller uses Gaussian activation functions that are weighted to form the control input

Neural networks for MRAC.

The weights

Adaptive controllers have the advantage that the error will converge to zero asymptotically, but for controlling real systems we often care more about the transient performance of the controlled system. It is generally not possible to give an a-priori transient performance bound for an adaptive controller because of the adaptation itself. When we use Lyapunov’s second method to derive an adaptation law, we are guaranteeing error convergence at the expense of parameter error convergence. Higher adaptation gains typically lead to faster convergence rates, but this is not always true given the instabilities that arise from high adaptation gains. Consider the standard scalar Lyapunov function

After the adaptation law is chosen

The main problem here is that we do not have a good idea what the initial parameter errors are, so it is difficult to give a-priori predictions of transient performance. There are four common ways to help improve the transient performance: increase

The use of robust adaptive control methods helps mitigate the stability problem, but at the expense of slowed transient performance or convergence to an error bound rather than zero. The projection method is reported to maintain fast adaptation and stability, but requires the designer to have bounds on parameters. This is not an unreasonable assumption, but there may be cases where it does not apply. Carefully initializing the trajectory to set

The complexity of the tools needed to analyze the stability of a system grows with the complexity of the system. In this context, system has a broad meaning and is defined as a set of ordinary or partial differential equations. With this point of view, a control problem also reduces to a stability problem. For example, a control problem in which the objective is the perfect tracking of state variables, turns into a stability problem when we define tracking error and look at the governing error dynamics as our “system”. The stability of systems have been widely studied for more than a century, from the work of Lyapunov in 1900s. Hence, we will not discuss those results; however, we will mention a few areas where improvements could be made. The benefit of such research is twofold. Any new tool developed will have applications to any branch of science that deals with “dynamics”,

In order to provide more motivation, consider the following scenario. We have derived an adaptive control law for a nonlinear non-autonomous system under external disturbances. We would like to prove that the state tracking error converges to zero despite disturbances and the parameter tracking errors are at least bounded. If we fail this task, we would like to at least prove that the states remain bounded under disturbances, and that the error remains within a finite bound the whole time.

Non-autonomous systems are explicitly time-dependent and are generally described as

The first method is known as the averaging method, and applies to systems of the form

([

Now suppose we want to apply this theorem to our hypothetical scenario and prove the stability of our adaptive controller. A reader familiar with adaptive control already knows that unless PE conditions are satisfied, the stability is only asymptotic. This means that parts 2 and 3 of this theorem will not be applicable, leaving us only with part 1. However, this part is also a weak result which is only valid for finite times. Therefore, averaging theorem in this form is not helpful in proving stability of adaptive control applied to non-autonomous systems.

Teel

In search for a proper tool to analyze our hypothetical scenario, we next move on to Lyapunov theorems for non-autonomous systems. Such theorems, directly deal with systems that are explicitly time-dependent and are much less developed than theorems regarding autonomous systems. Khalil [

([

In adaptive control, it is extremely difficult to satisfy condition (211). The derivative of the Lyapunov function is usually only semi-negative definite at best. This means that the right hand side of Equation (211) will only have some of the states and the inequality will not hold. We believe that creating new Lyapunov tools with less restricting conditions is an area that needs more attention.

When we cannot study the stability of the origin using known tools, or when we do not expect convergence to the origin due to the disturbances, the least we hope for is that

Vanishing perturbations refer to the case where

Such theorems, although very useful in many cases, still need further development before they can be applied to our scenario and be useful for adaptive control. Most boundedness theorems require exponential stability at the origin. We know that in adaptive control, exponential stability is only possible when PE conditions are satisfied (in which case our first approach to use averaging theorems would have worked already!). Furthermore, in the absence of PE, the origin is not a unique equilibrium for the adaptive controller: some parameter estimates may converge to an unknown equilibrium manifold and we cannot transform the equilibrium of the adaptive controller to the origin. Since the objective of the adaptive controller is not the identification of parameters, but rather the convergence of tracking error, one wonders whether it is possible to study the stability of only parts of the states (

The first person to ever formulate partial stability was Lyapunov himself – the founder of modern stability theory. During the cold war, with the resurgence of interest in stability theories, this problem was pursued and Rumyantsev [

Partial stability deals with systems for which the origin is an equilibrium point, however, only some of the states approach the origin. Such systems commonly occur in practice, and there have been a fair amount of research on their stability analysis using invariant sets and Lyapunov-like lemmas such as Barbalat’s Lemma or LaSalle’s Principle. Some of the motives for studying partial stability are [

The problem of partial stability is formulated as follows. Consider the system

There’s a myriad of theorems regarding partial stability of systems. However, in most these works, the conditions on the Lyapunov function are too restrictive, rendering them ineffective for the adaptive control problem of our interest. Furthermore, the behavior of systems under perturbations is not abundantly studied when the best we can do is partial stability. However, we believe that adaptive control could in general benefit from this tool due to the nature of its stability. Further development of partial stability tools and its application to adaptive control is an interesting problem that can be addressed.

Systems that have fewer actuators than states to be controlled are referred to as

Underactuated systems have been studied for more than two decades now. Energy and passivity based control [

A survey on the methods and problems of underactuated systems requires a separate full length paper. However, we only look at them from the adaptive control perspective. Most of these proposed methods do not deal with uncertainties. Very few papers have been published that address the uncertainty issue in underactuated systems [

Although very difficult, it is still possible to create novel paradigms for adaptive control. Recently, [

We have reviewed the major history of adaptive control over the last century as well as several of the popular methods in adaptive control including: Model Reference Adaptive Control, Adaptive Pole Placement (Self-Tuning Regulators), Adaptive Sliding Mode Control, and Extremum Seeking. We presented the Model Reference Identification approaches through detailed analysis and examples, and briefly discussed their non-minimal realizations. It has been made clear that the application of adaptive systems can solve many interesting problems. The necessary tools for extending these methods to nonlinear systems were also discussed. Stability and robustness issues related to adaptive control methods were shown through analysis and example, followed by possible solutions using Robust Adaptive Control and Adaptive Robust Control methods. We also provided various perspectives in control, observation, and adaptive systems as well as some of the important open problems and the direction of future work. Despite the length of the open problems section, we have only covered a few problems where improvements can be made in adaptive control, showing that the field is still open contrary to common belief. There are still plenty of unsolved problems to be addressed in adaptive control and we hope to see more researchers address them in years to come.

We would like to thank the reviewers for their comments and suggestions which helped improve the overall presentation of the paper.

William Black contributed the majority of the work with the exception of: the introduction, Extremum Seeking, discrete systems, developing analysis tools, underactuated systems, and possible new methods. Poorya Haghi contributed the sections on Extremum Seeking, developing analysis tools (including subsections), underactuated systems, and possible new methods. Kartik Ariyur contributed the introduction and discrete systems sections.

The authors declare no conflict of interest, and the preceding work was not funded or influenced by any grants.