Necessary Optimality Conditions for a Class of Control Problems with State Constraint

: An elementary approach to a class of optimal control problems with pathwise state con ‐ straint is proposed. Based on spike variations of control, it yields simple proofs and constructive necessary conditions, including some new characterizations of optimal control. Two examples are discussed.


Introduction
Necessary optimality conditions for control problems with pathwise state constraints have been widely studied since the beginnings of optimal control theory [1], and this domain of research is still vivid nowadays [2]. Most of the existing approaches may be divided into two streams [3]. The first one, characterized by the use of classical methods of analysis with often heuristic proofs, yields results of limited generality (see the review [4], and [3]). The other is based on the abstract theory of infinite-dimensional optimization and its results encompass a wide class of problems, with rigorous but demanding proofs (see [5][6][7][8][9]). However, these results are difficult for practical verification, because of too general characterizations of the adjoint variables and multiplier functions. Generally, the existing approaches are hardly constructive, meaning they do not give sufficient indications how to improve a nonoptimal control.
We propose an elementary approach to necessary optimality conditions for problems in Mayer form with free final state and a scalar state constraint. The controls are scalar functions and nontangentiality is assumed at all entry and exit points. As is well known, the proof of the minimum principle with free final state and without pathwise state constraints can be made elementary and simple by considering the cost increment caused by a single spike variation of control. Our first purpose is to show that a similar proof technique may be effective when a pathwise state constraint is present, with the difference that additionally a coordinated pair of spikes is used. A second purpose is to extend the known results for state constraints of index one, mainly to nonregular problems in which the optimal control and the corresponding state trajectory may at the same time take values on the boundaries of their respective admissible sets. In particular, we allow for discrete sets of admissible control values. From a conceptual point of view, this work also offers a clear geometrical interpretation of the results. On the practical side, an advantage of our approach is that the obtained conditions are readily verifiable and constructive: if they are not fulfilled, a gradient optimization procedure can be indicated and initialized which guarantees an improvement of the control, up to numerical precision (as in the method of Monotone Structural Evolution [10]). Of course, the other approaches clearly prevail in a wider perspective, when problems of greater complexity are also taken into account. They then produce optimality conditions, which can be effectively used in optimal control computations (see [11][12][13][14]). Consider a control system described by a state equation with a given initial condition 0 x and a given time horizon T. The controls : [0, ] u T R  are piecewise continuous functions of time, taking values in a given set U, that is, they belong to (0, ; ) PC T U . 1 The function   : n n f R R R is of class 1 C in its both arguments. We make a general assumption that all solutions of (1) appearing in the sequel are well defined in the whole time interval [0, ] T . The state is subject to a scalar pathwise constraint, The function is minimized on the trajectories of (1). The function : n q R R  is of class 1 C . For a control (0, ; ) u PC T U  , let x be the corresponding solution of the initial value problem (1). The control u is admissible if the trajectory x satisfies the state constraint (2). The control u is optimal if it is admissible and minimizes the cost Q in the set of all admissible controls. A boundary interval of u is defined as any nonempty and right-open interval of time in which ( ( )) 0 g x t  . Any nonempty and right-open interval of time, such that ( ( )) 0 g x t  for every t in that interval, is nonboundary. If 1 2 [ , [ t t is an inclusion-maximal boundary interval of u and 1 0 t  , then 1 t is called an entry point of u.
The derivative of the function ( ( )) t g x t  along the trajectories of (1) is equal to ( ( ), ( )) g x t u t  . For admissible controls we introduce the concept of verifiability, aiming to distinguish the controls to which the spike technique of (non)optimality verification, developed below, can be effectively applied. Let u be an admissible control with the corresponding state trajectory x. We call this control verifiable if (i) it has a finite number of inclusion-maximal boundary intervals, (ii) an implication holds that if ( ( )) 0 g x t  for a certain t, then t belongs to the closure of some boundary interval of u, 2 (iii) the conditions of nontangentiality en en hold at all entry points en t and all exit points ex t of u, where u  denotes the union of all boundary intervals of u. Obviously, the corresponding state trajectory satisfies ( )

The One-Spike Control Variation and Trajectory Variation
Let u be a verifiable control, and x, the corresponding state trajectory. Denote For any [0, [ T   , any v U   , and any sufficiently small 0   , we shall define a control (0, ; ) u PC T U   . We also define x  as the solution of the initial value problem and u has no exit points in 1 [ , ] t t , where en t is the greatest entry point of u less than or equal to t, Let now  belong to  , an inclusion-maximal boundary interval of u, and   The spike variation of control is the difference u u   . Note that the control u  is admissible for every sufficiently small positive  . moreover, and at every entry point Here en en en en en en en en en en  (7) and (6) in that interval. We shall prove that the relationships (5) and (6) as en en en en . Hence en en en en en By the definition of entry points, en en and so en en en en en en en Substituting this into (10), we obtain en en en en en en en en As en en en en en (8), we arrive at the extension of (5) and (6) to [ , ] T  because of the same classical theorems on differential equations. For en 0 t   , an analogous argument leads to the same result. The proof can be easily generalized to an arbitrary finite number of entry points. 

The Adjoint Function and The One-Spike Necessary Optimality Condition
As in Section 2, let u be a verifiable control and x, the corresponding solution of (1). With every such control we associate an adjoint function absolutely continuous except at the entry points of u and satisfying the final condition At every entry point en Let  be an arbitrary point from [0, [ T , and x  , the trajectory variation determined in Lemma 1. It is easy to notice that the function equals zero at every t where  and x  are differentiable, and at the entry points en t   we have by virtue of (13) and (8): Thus, , and its increment for any v U  and  [0, ] T (note that H  is only defined for a uniquely predetermined control u). We can now express the value of cost on the control u  , defined in Section 2 A sufficient condition for the existence of spike variations which improve the cost is a straightforward consequence.

Lemma 2. Assume that
for every sufficiently small   0 .
A theorem on optimal control of the minimum principle type follows from Lemma 2.
Theorem 1. Assume that the control u is optimal. Then: Proof. Conclusion (i) is a direct consequence of Lemma 2. Conclusion (ii) for the nonboundary intervals is proved exactly as in the classical proofs of the minimum principle without pathwise state constraints. In the interior of every boundary interval, the function  is of class 1 C with the derivative identically zero. The continuity of  at entry points readily follows from (13) and (9). Let now ex t be an exit point of u. By (i) and (4), . The continuity of  at ex t is shown by limit for the first of these inequalities, and   ex t t for the second.  Corollary 1. Assume that the control u is optimal and ( ( )) 0

The Two-Spike Necessary Optimality Condition
For a verifiable control u and the corresponding state trajectory x, we shall define a two-spike control variation. Let  be an inclusion-maximal boundary interval of u. For for any other t in  . Points (i), (ii) and (iii) of the definition in Section 2 apply to all the remaining values of t. The two-spike control variation is the difference The control u may sometimes be improved even if it fulfills the necessary optimality condition (i) of Theorem 1. We shall now give conditions, sufficient for the existence of a two-spike control variation in  which is admissible and guarantees a cost improvement.
Then for every sufficiently small   0 the control u  is admissible, and ( ) Proof. It follows from the definition of u  and the inequalities (15), (16) that for every sufficiently small 0   the function From this we infer that the control u  is admissible for all sufficiently small 0   . Reasoning similarly as in Section 3 and using the adjoint function defined therein, we estimate the value of the performance index on the control u  Hence, (17) is a sufficient condition for the two-spike control variation to reduce the cost for every sufficiently small Assume also that if Under these assumptions there is an 0   , such that for every sufficiently small 0   the con- Proof. We shall show that the assumptions of Lemma 3 follow from the assumptions of Lemma 4. Denote Of course   Proof. Let first , and the inequality (20) holds by virtue of (rc). 

A Geometrical Interpretation and a Minimum Condition
Let u be a verifiable control with a boundary interval  . The corresponding state and adjoint trajectories are denoted by x and  , respectively. Define a family of sets It readily follows from this definition that 0 . Corollary 1 says that if u is optimal, then t C has no points in quadrant III of the coordinate system 1 2 y y . The following theorem is a straightforward consequence of that corollary and of Theorem 2. It proves useful to describe the consequences of control optimality in terms of the straight lines supporting the sets t C at zero. This allows easier verification of the necessary conditions of optimality, and also expressing a partial optimality criterion as a minimum condition imposed on the extended pre-Hamiltonian. We say that a straight line is a supporting line of t The equality max min ( ) ( ) t t     occurs in two practically important situations (mutually nonexclusive). One of them, in which the right-hand side of the system equation (1) is affine in control, will be discussed in Section 7. Here we consider the other situation, in which t C has a tangent at the origin. A sufficient condition for that reads

Theorem 3. Assume that the control u is optimal. Then
Under this condition, the tangent has the equation Of course, if t C has both an SLO and a tangent at the origin, they coincide.
Suppose u is optimal and max min ( ) , has a unique SLO. The SLO is vertical if The function p thus defined is nondecreasing in all that part of  where it is determined. Define the extended pre-Hamiltonian ˆ( , , , ) If the control u is optimal and the function p is determined as above in all the interval  , then the following minimum condition is straightforward by the properties of SLO This necessary optimality condition is similar to the minimum condition of indirect adjoining. We postpone a discussion of relations with the classical results to Section 9.

Example 1
In this example we apply the above necessary conditions to verify optimality of two controls, the first of which is optimal, and the second is not. We show that the nonoptimality is easily detected. The control system is described by state equations The adjoints satisfy the final conditions In consequence, the state and adjoint equations in the boundary intervals take the form To determine the behavior of the adjoint function at entry points, we calculate the matrix (9) en en Hence by (13) In accordance with (14), It is evident that the nontangentiality conditions (4) are fulfilled at all entry and exit points. The optimality of u should be verified with Theorem 1(i) in the whole interval [0, ] T , and additionally with Theorem 2 or 3 in the boundary intervals. Every set t C introduced in Section 5 consists of three points, t Example 1a. A numerically computed approximation of optimal control and optimal state trajectory is presented in Figure 1. The control has discontinuities at 1 0.49483839 s  , 2 0.99973751 s  , 3 1.4945759 s  and 4 3.1366809 s  . Figure 2 shows the corresponding adjoint trajectory. Let us verify the necessary conditions of optimality. Figure 3 shows that the condition of Theorem 1(i) is fulfilled. In the boundary interval   3 4 [ , [ s s , we additionally have to verify the conditions of Theorem 2 or 3. It can be seen in Figure 3 that    Example 1b. Consider a nonoptimal control It is plotted in Figure 5 together with the corresponding state trajectory. The adjoint function is depicted in Figure 6. As follows from Figure 7, the necessary optimality condition of Theorem 1(i) is satisfied and in consequence, there are no onespike variations described in Section 2 which guarantee an improvement of the cost. Let  Figure 8 shows that this inequality holds only in  . This proves that the control u is not optimal and there are two-spike variations in the boundary interval (defined in Section 4) which yield a cost reduction for any sufficiently small positive value of the parameter  . A closer analysis of the conditions of Lemma 4 shows that the difference 2 1    , that is, the distance between the spikes in such a variation cannot be arbitrarily small.

The Control Affine Case
Consider the system (1) Figure 15 shows the state trajectory, and Figure 16, an enlargement of the plot of 3 x  .

Connections with Some Classical Results
There are essential connections between some of our results presented in Sections 5 and 7, and certain classical results obtained by the so called indirect adjoining method, dating back to the works of R.V. Gamkrelidze, A.E. Bryson, H. Maurer, D.H. Jacobson, and many others (see [1,[3][4][5]). As we have no space to discuss all similarities and analogies that can be found in the vast literature, we shall concentrate on one representative theorem due to H. Maurer [3]. We shall use a reduced version of that theorem, specialized to the case of state constraint of order one, verifiable control, fixed initial state and free final state.
Consider the optimal control problem formulated in Section 1, with the additional assumption that U is a closed interval with nonempty interior. Define ( , , , ) Finally, note that this work's approach does not require that the optimal control in the boundary intervals takes values in the interior of U, whereas that assumption is essential in [3]. Also, in contrast to [3], we give an explicit representation of the jump of the adjoint function (Section 3, see also [11,12]