1. Introduction
The main disadvantage of the optimal control problem [
1] is that its solution is an optimal control as a function of time, and it cannot be implemented in practice since its implementation leads to an open control system that is insensitive to model disturbances.
Consider a well-known optimal control problem
where
is a state vector,
u is a control signal.
The control values are limited
It is necessary to find a control that will move the object (
1) from the initial state
to the given terminal position
as fast as possible
The analytical solution of the stated problem was presented in [
1]. According to the maximum principle, the optimal control takes only limit values (
2) and has no more than one switch. According to (
3), initially, the control has the value
. Then, when a certain state is reached, the control switches to the value
.
where
is the moment of control switching.
Let us find the moment of switching. A particular solution of the system (
1) from the initial state (
3) has the following form
The general solution (
1) for the control
is the following
where
,
are the coordinates of the switching point.
Let us express in (
8)
as a function of
The relation for the switching point follows from the terminal conditions (
4)
Let us now find the moment of time for a particular solution (
7) that satisfies the relation (
10).
The switching time (
11) is the solution to this optimal control problem. To determine the value of the functional (
5), we calculate the coordinates of the switching point. Substituting (
11) into the particular solution (
7), we obtain
From the second equation in (
8), we obtain the optimal time of reaching the terminal state
Now, we introduce perturbations into the initial conditions (
3)
where
,
are random variables from a limited range.
During the time
, the object does not get to the switching point (
12), and after switching accordingly, does not get into the terminal state. Based on the optimal value of the functional (
13), let us limit the control time to
and determine the state of the object at the moment
, taking into account the switching of the control at the moment (
11)
Figure 1 shows trajectories of eight randomly perturbed solutions of the problem (
1)–(
5) from the range
In
Figure 1, the blue curve represents the optimal unperturbed solution.
All perturbed solutions do not reach the terminal state. It is obvious from the plots that the solution of the optimal control problem as a function of time (
6) cannot be implemented in practice since according to the model (
1) with control (
6) due to disturbances, it is impossible to assess the state of the control object.
In this regard, it is necessary to introduce additional requirements into the formulation of the optimal control problem so that the resulting controls can be directly implemented on a real plant.
2. Optimal Control Problem Statement with Additional Requirement
Consider the formulation of the optimal control problem, the solution of which can be directly implemented on the plant.
Given the mathematical model of the control object
where
,
,
.
Given the initial
and terminal conditions
where
is the time to reach the terminal conditions, not specified, but limited,
,
is the specified limit time of the control process.
A quality criterion is set. It may include conditions for fulfilling phase constraints
We need to find a control in the form
The found control must be such that the particular solution
of the system
from the initial state (
18) reaches the terminal state (
19) with the optimal value of the quality criterion (
20). Moreover, the optimal particular solution
of the system (
22) would have a neighborhood
such that if for any other particular solution
of the system (
22) from another initial state
at time
,
then
,
, this particular solution does not leave the neighborhood of the optimal
The neighborhood
shrinks near the terminal state. This means that for any particular solution from the neighborhood of the optimal one for which the conditions (
23) are satisfied
such that
where
is a given small positive value.
The existence of a neighborhood of the optimal solution in many cases can worsen the value of the functional. For example, in a problem with phase constraints, which are obstacles on the path of movement of the control object to the terminal state, the optimal trajectory often passes along the boundary of the obstacle. Such a trajectory will not have a neighborhood, so in this case, it is necessary to find another trajectory that will not be optimal according to the classical formulation of the optimal control problem but allows variations in the initial values with a small change in the value of the functional.
3. Overview of Methods for Solving the Extended Optimal Control Problem with Additional Requirement
Therefore, an additional requirement has been put forward in the formulation of the optimal control problem, which makes it possible to implement the obtained controls on real objects. Consider the existing methods for solving the problem in the presented extended formulation.
The solution to the problem of general control synthesis for a certain region of initial conditions leads to the fact that each particular solution from this region of initial conditions will be optimal. In this case, each particular solution has a neighborhood containing other optimal solutions. The neighborhood will be open, but will also shrink near the terminal state.
For the model (
1), there is a solution to the problem of general control synthesis, in which a control is found that ensures the optimal achievement of the terminal state from any initial condition.
where
Plots of eight perturbed solutions for the object model (
1) with control (
26) are shown in
Figure 2.
As can be seen from
Figure 2, all perturbed solutions have reached the terminal state. This control (
26) is practically feasible.
However, the problem of general control synthesis is a complex mathematical problem for which there is no universal numerical solution method. In this case, this problem was solved because the plant model is simple, the optimal control takes only two limit values, and for both of these values, general solutions of the differential equations of the model (
1) are obtained.
Another approach to solving the optimal control problem and fulfilling additional requirements is stabilizing motion along the trajectory based on the theory of stability of A.M. Lyapunov [
2]. As a result of constructing the stabilization system, the optimal trajectory should become asymptotically stable. The construction of such a stabilization system is not always possible; in particular, in the problem under consideration (
1), the control resources are exhausted to obtain the optimal trajectory and there are no more control resources for the stabilization system.
Another approach that also allows solving the optimal control problem in the presented extended formulation with additional requirements is the synthesized control method. It consists of two stages [
3,
4]. Initially, the problem of control synthesis is solved in order to ensure the stability of the control object relative to some point in the state space. In the second stage, the problem of optimal control is solved, while the coordinates of the stability points of the control object are used as control. It should be noted that the solution of the control synthesis problem at the first stage will significantly change the mathematical model of the control object. When solving the control synthesis problem, the functional of the optimal control problem is not used to ensure stability; therefore, various methods for solving the control synthesis problem will lead to different mathematical models of a closed control system and to different solutions to the optimal control problem. The presence of a neighborhood with attraction properties for the optimal solution requires the choice of such a position of the stability points in the state space so that particular solutions from a certain region of initial states, being attracted to these stability points, are close to each other, moving to the terminal state.
Consider the application of the synthesized optimal control to problems (
1)–(
5). For stabilization system synthesis different methods can be used from traditional regulators [
5] to modern machine learning techniques [
6,
7,
8]. As far as the considered object is rather simple (
1), it is enough to use a proportional regulator. Taking into account the limits on control, it has the following form
where
The object is stable if
,
. A stable equilibrium point exists if
. The coordinates of the equilibrium point for the given
,
depend on the values of
,
As a result, we obtain the following control object model
The control in the model is the vector
, whose values are limited by the following inequalities
To solve the problem of optimal control, we include in the quality criterion the accuracy of hitting the terminal state
where
is a terminal time,
is a weight coefficient,
.
When solving the problem, we divide the control time
into intervals
, and on each interval, we look for the values
,
, taking into account the constraints (
32). The following parameter values were used:
,
,
,
,
,
. An evolutionary hybrid algorithm [
9] was used for the solution. The optimal solution gave the value of the functional (
33)
.
Figure 3 shows particular solutions of the system (
31) with random perturbations of the initial values in the range (
16). The blue curve in the figure shows the unperturbed optimal solution. The black dots show the positions of the found
control points. As we see from the experimental results, the perturbed solutions stabilize in the vicinity of the optimal one. Compared to
Figure 1, it is obvious that the resulting model (
31) is feasible.