Discrete-Time Fractional Optimal Control

Abstract: A formulation and solution of the discrete-time fractional optimal control problem in terms of the Caputo fractional derivative is presented in this paper. The performance index (PI) is considered in a quadratic form. The necessary and transversality conditions are obtained using a Hamiltonian approach. Both the free and fixed final state cases have been considered. Numerical examples are taken up and their solution technique is presented. Results are produced for different values of α.


Introduction
Fractional calculus is a generalization of classical calculus.It has been reported in the literature that systems described using fractional derivatives give more realistic behavior .There exists many definitions of a fractional derivative [28][29][30][31][32][33].A commonly known fractional derivatives is the Riemann-Liouville derivative, which is not always suitable for modeling physical systems because the solution of Riemann-Liouville derivative problems require unnatural initial conditions.On the other hand the Caputo fractional derivative accepts initial conditions similar to the integer order systems.Therefore, Caputo fractional derivative is preferred for modeling physical systems [5,6,[14][15][16][17][18][19][20][21][22][23][24].
Fractional derivative plays important role in many areas of science and engineering.It also finds application in optimal control problems.Minimization of a given performance index subject to dynamic constraints on state and control variables is referred to as a constrained dynamic optimization problem [34,35].This paper presents a formulation and solution scheme of a discrete-time fractional optimal control problem defined in terms of the Caputo fractional derivative.
Only limited work has been done in the area of fractional optimal control problems particularly in the discrete-time domain.In this context authors in [36][37][38][39][40] approximate fractional derivative using an approximation technique for solving differential equations defined in terms of the Caputo fractional derivative.Analysis of the Caputo derivative with the help of the infinite state approach has been presented in [41].Several works have been published on the calculus of the variations where the fractional derivatives are taken in the sense of Caputo [42,43], Riesz-Caputo [44], and combined Caputo [45].Malinowska et al. [46] discussed advanced methods in the fractional calculus of variations.
Fractional optimal control problems of various cases have been formulated and investigated using the fractional calculus of variations.In [47], Agrawal presented a general formulation and solution scheme for a class of fractional optimal control problems in terms of the Reimann-Liouville fractional derivative.In [48], a numerical technique for the solution of a class of fractional optimal control problems in terms of both Reimann-Liouville and the Caputo fractional derivatives is presented.Authors in [49,50] presented a pseudo-state-space-based fractional optimal control problem formulation and a solution scheme.Fixed and free final-time fractional optimal control problems are considered in [51,52].In [53] Yuan et al. investigated fractional optimal control problems in terms of the left and the right Caputo fractional derivatives.Guo [54] formulated a second-order necessary optimality condition for fractional optimal control problems in the sense of Caputo.Optimal control of a fractional-order HIV-immune system in terms of the Caputo fractional derivative is discussed in [55].In [56], authors proposed a fractional-order optimal control model for malaria infection in terms of the Caputo fractional derivative.In the literature we can find several methods for numerical solutions of fractional optimal control problems, including the direct numerical technique [57], central difference numerical scheme [58], quadratic numerical scheme [59], the method based on Legendre wavelets [60], modified Jacobi polynomials [61], Legendre orthonormal polynomials [62], Bernoulli polynomials [63], Bessel collocation method [64], Bezier curve method [65], neural networks [66], the Ritz method [67], etc.All of the above works consider continuous time fractional order systems.
Dzielinski and Czyronis present a general formulation and solution scheme for discrete-time fractional optimal control problems with fixed final state [68] and free final state [69] in terms of the Reimann-Liouville fractional derivative.Dynamic programming methods for solving discrete-time fractional optimal control problems have been proposed by Dzielinski and Czyronis in [70,71].Authors in [72] proposed algorithms for discrete-time fractional optimal control problems based on dynamic programming method and discrete-time fractional calculus of variations.In [73], Al-Maaitah investigated the motion of discrete dynamical systems in terms of the Caputo fractional derivative.Malinowska et al. [74] introduced the multidimensional discrete-time fractional calculus of variations.Basic concepts and solution to state equations of the fractional discrete-time linear systems are presented in [75].
In the area of fractional optimal control, most of the existing works in the literature consider continuous time problems.Discrete-time fractional optimal control problems that exist in the literature is presented in terms of Riemann-Liouville derivative [68][69][70][71][72].It has been observed that the Caputo fractional derivative is preferred as it requires natural initial conditions.In this paper the optimal control of the discrete-time fractional order system in terms of the Caputo fractional derivative has been considered.Both the fixed and free final state cases are considered.The necessary and transversality conditions are obtained using a Hamiltonian approach.A numerical method [68,69,72] is used for the solution of the resulting equations obtained from the formulation.

Problem Formulation
Consider the fractional continuous-time linear system described by [75]: where u is the input vector, x is the state vector, A is state matrix, B is input matrix and C 0 D α t x(t) is the Caputo fractional derivative.
The relation between Reimann-Liouville and Caputo fractional derivatives is [32]: The Grunwald-Letnikov shifted approximation of the fractional order derivative [27] is given as: Here we assume T = 1.
Using Equations ( 1)-(3), we can obtain a fractional discrete-time system described by: where: and: Consider a performance index of the form: where F ≥ 0, Q ≥ 0, R > 0 and N is number of discrete-time moments.
Using Lagrange multiplier concept, we can write Equation ( 5) as: We can define a Hamiltonian function as follows: Using Hamiltonian function in (6) and simplifying, we write augmented PI as: The increment J a due to the variations of x(k), u(k) and λ(k) is: The necessary condition for optimum of a functional is that the first variation vanishes, i.e., δJ a = 0.This leads to: x(k Equations ( 10)-( 12) are the necessary conditions for a constrained minimum.The general transversality condition for fixed final time problem is: In the case of fixed final state x(N) is fixed, so the variation δx T (N) is equal to zero.The transversality condition for the fixed final state problem is: In the case of free final state x(N) is free, so the variation δx T (N) is not equal to zero.The transversality condition for the free final state problem is: At α = 1, the conditions Equations ( 10)-( 12) are equivalent to the conditions of integer order discrete-time systems.Substituting α = 1 to ( 10)-( 12), we get: x(k

Numerical Examples
In this section we present two numerical examples, one free final state problem and the other fixed final state problem.

Free Final State Problem
Consider a fractional discrete-time system Equation ( 4) with [69]: Performance index Equations ( 5) with: and the given conditions as x(0) = 0.6 0.8 and N = 5.
The optimal conditions for this problem are given by: where: x(0) = 0.6 0.8 , and λ(5 The state, co-state, and control Equations ( 19)-( 21) along with the boundary conditions Equations ( 22) form a two-point boundary value problem (TPBVP).
Replacing u(k) in the Equation ( 19) we obtain: A general method for solving the state and co-state equations in terms of the Reimann-Liouville derivative is presented in [68,69,72].Here, the same method is used for solving the present problem defined in terms of Caputo fractional derivatives.
The solution of state Equation ( 23) is: where: Equation ( 24) in matrix form is: The solution of co-state Equation ( 20) is: Using Equation (26) in Equation ( 25) we obtain the state vector x(k).Then substituting x(k) into Equation ( 26) we get the co-state vector λ(k).Once λ(k) is known, the control vector u(k) can be calculated using Equation (21).Solving the above equations we obtain the following results.
This problem is solved for different values of α.Figures 1 and 2 show the state variables and Figure 3 shows the optimal control for N = 5 and different values of α.These figures show that as α → 1 , the numerical solutions for both the state and control variables approach to the analytical solutions.At α = 1, these results recover the solutions obtained with the integer order problem.While comparing these results with the results presented in [69,72] in terms of the Riemann-Liouville fractional derivative, we observe that for α = 1 the results are the same as expected.
Mathematics 2017, 5, 25 5 of 12 A general method for solving the state and co-state equations in terms of the Reimann-Liouville derivative is presented in [68,69,72].Here, the same method is used for solving the present problem defined in terms of Caputo fractional derivatives.
The solution of state Equation ( 23) is: where: 24) in matrix form is: The solution of co-state Equation ( 20) is: Using Equation (26) in Equation ( 25) we obtain the state vector ) (k x .Then substituting ) (k x into Equation (26) we get the co-state vector ) (k


is known, the control vector ) (k u can be calculated using Equation (21).Solving the above equations we obtain the following results.
This problem is solved for different values of  .Figures 1 and 2 show the state variables and Figure 3 shows the optimal control for 5  N and different values of  .These figures show that as 1   , the numerical solutions for both the state and control variables approach to the analytical solutions.At 1   , these results recover the solutions obtained with the integer order problem.While comparing these results with the results presented in [69,72] in terms of the Riemann-Liouville fractional derivative, we observe that for 1   the results are the same as expected.

Figure 1. State 1
x for the free final state problem for different  .

Fixed Final State Problem
Consider a fractional discrete-time system (Equation ( 4)) with [68]: A performance index (Equation ( 5)) with: and the given conditions as , and 10  N .
The optimal conditions for this problem are given by: and the boundary conditions are:

Fixed Final State Problem
Consider a fractional discrete-time system (Equation ( 4)) with [68]: A performance index (Equation ( 5)) with: and the given conditions as , and 10  N .
The optimal conditions for this problem are given by: and the boundary conditions are:
The optimal conditions for this problem are given by: and the boundary conditions are: x(0) = 0.5 0.7 , and x(10) = 0.06 −0.05 (32) Using ( 31) in ( 29) we obtain Using a method presented in [68,69,72], we solve the problem.The solution of state Equation ( 33) is: The Equation ( 34) in matrix form is: The solution of co-state Equation ( 30) is: Using Equation (36) in Equation (35) we can get state variables x(1), x(2), . . ., x(10) in terms of λ (10).This is fixed final state problem, so x(10) is known.Using this value we can obtain λ (10).Once λ( 10) is known, we can obtain state variable vector x(k).Substituting x(k) and λ (10) in Equation (36) we get the co-state vector λ(k).Using λ(k) in Equation ( 31) we get the control vector u(k).Solving the above equations we obtain the following results.The problem is solved for different values of α.Figures 4-7 show the state variables, optimal control, and the minimum value of performance index for N = 10, and different values of α.From these figures, it is clear that the amplitudes of the state variables, the control variable, and the performance index increases as α is increased, and the numerical solution approaches the analytical solution as α approaches 1.At α = 1 these results also agree with the results obtained with the integer order problem.
Using ( 31) in ( 29) we obtain Using a method presented in [68,69,72], we solve the problem.solution of state Equation ( 33) is: The Equation (34) in matrix form is: The solution of co-state Equation ( 30) is: Using Equation (36) in Equation (35)

). (k u
Solving the above equations we obtain the following results.
The problem is solved for different values of  .Figures 4-7 show the state variables, optimal and the minimum value of performance index for 10  N , and different values of  .From these figures, it is clear that the amplitudes of the state variables, the control variable, and the performance index increases as  is increased, and the numerical solution approaches the analytical solution as  approaches 1.At 1   these results also agree with the results obtained with the integer order problem.

Conclusions
A discrete-time fractional optimal control problem formulation and solution scheme is presented in this paper.A general cost function is considered and the dynamical equation considered is described in terms of the Caputo fractional derivative.The Hamiltonian approach is used to obtain the necessary and transversality conditions.Both the specified and unspecified final state problems are considered, keeping the final time fixed.The state and co-state equations are solved using the numerical technique.Numerical results are presented to demonstrate the method.
From the numerical results, it is clear that as  approaches to one, the numerical solution also

Conclusions
A discrete-time fractional optimal control problem formulation and solution scheme is presented in this paper.A general cost function is considered and the dynamical equation considered is described in terms of the Caputo fractional derivative.The Hamiltonian approach is used to obtain the necessary and transversality conditions.Both the specified and unspecified final state problems are considered, keeping the final time fixed.The state and co-state equations are solved using the numerical technique.Numerical results are presented to demonstrate the method.
From the numerical results, it is clear that as  approaches to one, the numerical solution also

Conclusions
A discrete-time fractional optimal control problem formulation and solution scheme is presented in this paper.A general cost function is considered and the dynamical equation considered is described in terms of the Caputo fractional derivative.The Hamiltonian approach is used to obtain the necessary and transversality conditions.Both the specified and unspecified final state problems are considered, keeping the final time fixed.The state and co-state equations are solved using the numerical technique.Numerical results are presented to demonstrate the method.
From the numerical results, it is clear that as  approaches to one, the numerical solution also

Conclusions
A discrete-time fractional optimal control problem formulation and solution scheme is presented in this paper.A general cost function is considered and the dynamical equation considered is described in terms of the Caputo fractional derivative.The Hamiltonian approach is used to obtain the necessary and transversality conditions.Both the specified and unspecified final state problems are considered, keeping the final time fixed.The state and co-state equations are solved using the numerical technique.Numerical results are presented to demonstrate the method.From the numerical results, it is clear that as α approaches to one, the numerical solution also approaches to the analytical solution.The numerical result, presented in Figure 7, also show that the performance index reduces as α is decreased.It can be concluded that the discrete-time fractional optimal control problem consideration can lead to considerable benefits than the discrete-time integer order problems.

Figure 1 .
Figure 1.State x 1 for the free final state problem for different α.

Figure 2 . 2 x
Figure 2. State 2x for the free final state problem for different  .

Figure 2 .
Figure 2. State x 2 for the free final state problem for different α.

Figure 2 . 2 x
Figure 2. State 2x for the free final state problem for different  .

Figure 3 .
Figure 3. Optimal control U opt for the free final state problem for different α.

Figure 4 .
Figure 4. State 1x for the fixed final state problem for different  .

Figure 4 .
Figure 4. State x 1 for the fixed final state problem for different α.

Figure 5 . 2 x
Figure 5. State 2x for the fixed final state problem for different  .

Figure 6 .
Figure 6.The optimal control

Figure 7 .
Figure 7.The minimum value of the performance index min J for the fixed final state problem for

Figure 5 .
Figure 5. State x 2 for the fixed final state problem for different α.

Figure 5 . 2 x
Figure 5. State 2x for the fixed final state problem for different  .

Figure 6 .
Figure 6.The optimal control

Figure 7 .
Figure 7.The minimum value of the performance index min J for the fixed final state problem for

Figure 6 .
Figure 6.The optimal control U opt for the fixed final state problem for different α.

Figure 5 . 2 x
Figure 5. State 2x for the fixed final state problem for different  .

Figure 6 .
Figure 6.The optimal control

Figure 7 .
Figure 7.The minimum value of the performance index min J for the fixed final state problem for

Figure 7 .
Figure 7.The minimum value of the performance index J min for the fixed final state problem for different α.