Optimal Control Theory and Calculus of Variations in Mathematical Models of Chemotherapy of Malignant Tumors

: This paper is devoted to the analysis of mathematical models of chemotherapy for malignant tumors growing according to the Gompertz law or the generalized logistic law. The inﬂuence of the therapeutic agent on the tumor dynamics is determined by a therapy function depending on the time-varying concentration of the drug in the patient’s body. The case of a non-monotonic therapy function with two maxima is studied. It reﬂects the use of two different therapeutic agents. The state variables of the dynamics are the tumor volume and the amount of the therapeutic agent able to suppress malignant cells (concentration of the drug in the body). The treatment protocol (the rate of administration of the therapeutic agent) is the control in the dynamics. The optimal control problem for this models is considered. It is the problem of the construction of treatment protocols that provide the minimal tumor volume at the end of the treatment. The solution of this problem was obtained by the authors in previous works via the optimal control theory. The form of the considered therapy functions provides a speciﬁc structure for the optimal controls. The managerial insights of this structure are discussed. In this paper, the structure of the viability set is described for the model according to the generalized logistic law. It is the set of the initial states of the model for which one can ﬁnd a treatment protocol that guarantees that the tumor volume remains within the prescribed limits throughout the treatment. The description of the viability set’s structure is based on the optimal control theory and the theory of Hamilton–Jacobi equations. An inverse problem of therapy is also considered, namely the problem of reconstruction of the treatment protocol and identification of the unknown parameter of the intensity of the tumor growth. Reconstruction is carried out by processing information about the observations of the tumor volume dynamics and the measurements of the drug concentration in the body. A solution to this problem is obtained through the use of a method based on the calculus of variations. The results of the numerical simulations are presented herein.


Introduction
This paper is devoted to the study of mathematical models of chemotherapy of malignant tumors with different laws of the growth of the tumor volume (the generalized logistic law and Gompertz's law).The research focuses on solid tumors, such as breast, lung, liver, colon, pancreas and prostate cancers.As known, a malignant tumor formed in an organ is capable of uncontrolled or poorly controlled rapid growth of its cells and can penetrate into adjacent tissues, damage them and penetrate further into other organs, resulting in metastases.Spatially homogeneous mathematical models of the growth of solid avascular tumors are considered (on this theme, see, for example, [1,2]).
Usually, in such mathematical models, the total volume of the tumor is considered.The tumor's dynamics are described using ordinary or partial differential equations.In this paper, we consider tumors with dynamics described by ordinary differential equations that obey the generalized logistic law or the Gompertz law [2][3][4].
Chemotherapy is one of the most effective ways to fight cancer.The effect of a therapeutic agent on a tumor is defined by adding a therapy function to equations of the tumor's dynamics.This function depends on the time-varying concentration of the drug in the body.The case of a non-monotonic therapy function with two maxima is studied.It reflects the use of two different therapeutic agents.This case has not been previously considered.The infusion and the dissipation of the drug in the body are described in the considered model by ordinary differential equations.
Let us also note another well-known mathematical model of cancer therapy: the Lotka-Volterra predator-prey model.In this model, the therapeutic agent plays the role of the predator, and the tumor plays the role of the victim.For further information on this model, see, for example, [5][6][7].
The tumor growth dynamics are considered over a certain fixed period of time.At the final time instant (the control point), the total tumor volume is estimated.The state variables of the dynamics are the tumor volume and the amount of the therapeutic agent able the to suppress malignant cells (concentration of the drug in the body).The treatment protocol (the rate of administration of the therapeutic agent) is the control in the dynamics.
The optimal control problem is considered.The problem is to construct a treatment protocol (a control) for any initial state of the model (the tumor volume and the concentration of the drug in the body at the beginning of the treatment) such that the tumor volume is minimal at the control point.An optimal treatment strategy and optimal protocols were suggested in [8,9].Their construction is based on the results of the optimal control theory [10] and the theory of generalized solutions of partial differential equations of the first order [11,12].The optimal treatment strategy is a feedback depending on the current state of the model: the tumor volume and the concentration of the drug in the body.The optimal treatment protocol is formed on the basis of information about the current state of the model according to the optimal strategy.For any fixed initial state of the model, the optimal treatment protocol ensures the minimal tumor volume at the end of the treatment.It is has been proven that the optimal protocol has a piecewise constant structure with no more than one switch.Construction of optimal therapy strategies and optimal treatment protocols has been previously discussed, for example, in [3,[5][6][7][13][14][15][16][17][18].The innovation of the research presented in this paper is that a new type of therapy function is considered, namely a non-monotonic therapy function with two maxima.It reflects the use of two different therapeutic agents.Such a therapy function provides a specific structure of the optimal control.The managerial insights of such an optimal control are discussed in this paper.
The viability set [19] is described for the model according to the generalized logistic law.It is the set of initial states of the model for which one can find a treatment protocol that guarantees that the tumor volume remains within the prescribed limits and is compatible with life throughout the treatment.In this paper, we propose and substantiate the description of the structure of the viability set for the model according to the generalized logistic law.The justifications of the constructions are based on the theory of generalized solutions of partial differential equations of the first order [11,12].The viability set for the dynamics according to the Gompertz law was described in [20].
Research on models of chemotherapy of malignant tumors is also of interest to solve inverse problems, for example, identification of the parameter of the model characterizing the intensity of the tumor's growth and reconstruction of the treatment protocol (the control) for a given patient using the observations of the tumor volume dynamics and the current measurements of the concentration of the drug in the body.
The results of the numerical simulations of the solution of such inverse problems are presented in this paper.They were obtained using a numerical method [21] based on the theory of calculus of variations [22].
It should be noted that a process as complex as the growth and suppression of malignant cells is presented in the models in a simplified, idealized form.Still, the obtained results may be of use for the analysis of experimental data or for the study of new effective therapy methods.

Dynamics of the Model
The state variables of the dynamics are the tumor volume (m(t)) and the amount of the chemotherapy drug (h(t)) that is able to suppress malignant cells.The therapy function ( f (h)) describes the effect of the drug on the malignant cells.The control (the treatment protocol) (u(t)) is the amount of the drug injected within the time unit.
The dynamics of the malignant tumor and the concentration of the drug are described by the known model [2][3][4]: where 0 ≤ t 0 < T < ∞ are the starting and control points, γ > 0 is the effectiveness coefficient of the therapy and α > 0 is the dissipation coefficient.The feasible controls are measurable (piecewise constant) bounded functions: where Q is the maximum allowed value.
In the model (1), the function g(m) describes the law of the tumor's growth.In this paper, two different models of this law are considered.

1.
The Gompertz law: where the parameter r > 0 characterizes the speed of the tumor's growth, and θ > 0 characterizes the speed of tumor suppression.

2.
The generalized logistic law: where the parameter r > 0 again characterizes the speed of the tumor's growth, θ > 0 characterizes the threshold tumor volume and 0 < β ≤ 1 changes the steepness of the model's curve.
In the both models, the following restrictions are considered: where M is the threshold tumor volume compatible with life, and L is the maximum allowed amount of the drug in the body.

The Therapy Function
The therapy function ( f (h)) describes the effect of the drug on the malignant cells and is an important part of the considered model (1).In this paper, we consider a non-monotonic, continuously differentiable, positive therapy function such that its derivative ( f (h)) has three different roots: We assume that f (h) satisfies the following conditions: It follows from conditions A1-A4 that the roots ĥ1 and ĥ3 are the maximum points and that root ĥ2 is the minimum point of the therapy function ( f (h)).

Optimal Control Problem
The optimal control problem consists of constructing feasible (piecewise constant) feedback that minimizes the terminal therapy quality index (the value function) (σ(m(T))).It is different for the two considered models.
The system (1) can be integrated analytically for both g 1 and g 2 .The solutions are: 1.
For the Gompertz law (3) [8]: For the generalized logistic law (4) [8]: Here, is the optimal value in the reduced optimal control problem: For the Gompertz law (3), the cost value functional is For the generalized logistic law (4), The value functions (10) are constructed piecewise from several functions (ϕ i (•)): The geometric regions are For visualization, see Figure 1.The graph of the x(•) function is the Rankine-Hugoniot line as defined by the points where It is shown in [9] that the Γ line satisfies the Rankine-Hugoniot conditions with a boundary condition of x(T) = ĥ2 and has the following form: where The functions ϕ i (t, h), i = 1, 6 are constructed by the Cauchy characteristic method for the auxiliary linear Hamilton-Jacobi equations with boundary conditions of the special form [9].
It was proven in [8,20] that the optimal strategy in problems ( 1), ( 5) and ( 1), ( 6) has the following form: Note that all optimal controls (u 0 (t) = u 0 (t, h 0 (t))) satisfy the Pontryagin maximum principle.Remark 1.The process of construction of the optimal treatment strategy ( 16) is explained and justified in [8,20].The optimal strategy is a feedback, so the optimal protocols are defined by the current state of the system (m(t), h(t)).Their values are chosen following (16).
When the state of the system is in each of the regions (Π 1 , Π 2 , Π 3 , Π 4 ), the control is either minimal (0) or maximal (Q).When the state of the model transfers from one region to another, the control is switched.The specifics are that the border between regions Π 3 and Π 4 is defined by the Rankine-Hugoniot line (Γ).The conditions that determine this line are non-trivial (15).
The other feature is that the optimal control does not always have a maximal or minimal value.If the state of the system belongs to lines G 1 and G 2 , the optimal control corresponds to special regimes u 0 (t, h) = α ĥ1 and u 0 (t, h) = α ĥ3 .These regimes are stable and provide the optimal effect of the drug, which is achieved at the maximum points of the therapy function.
It follows from Pontryagin's maximum principle, which provides the basis for the optimal strategy construction, that this control provides the optimal terminal therapy quality index ( 5) and (6).In other words, it minimizes the tumor volume (m(T)) at the control point.
The Rankine-Hugoniot line (Γ) plays an important role in the construction of the optimal protocols.The selection of the therapeutic agent depends on what regions the initial state of the model belongs to:
Definition 1.The viability set for problem (1), ( 6) according to the generalized logistic law is a set of the points where M 2 is the threshold tumor volume compatible with life specified for the model according to the generalized logistic law.
The parameter M 2 is the stable equilibrium point of the dynamics (1) when the drug effectiveness (that is, the therapy function ( f (h))) is maximal [3].Thus, consider the following equation: where Let us find the roots of the right side of (17).The trivial zero root provides an unstable equilibrium.Stable equilibrium takes place when m = m, Therefore, we assume We introduce set W 2 : We will show that this set is the maximal viability set for problem (1), (6).

Construction of Set W 2
In this section, we describe the structure of set W 2 .First, we transform the expression (9) for Val 2 (t 0 , h 0 , m 0 ) into a simpler form.It follows from (12) that in expression (9), where h 0 (t) = h(t; t 0 , h 0 , u 0 (•)) is the solution of the second independent equation of the system (1) that is generated by the optimal feedback control (u 0 (t, h) ( 16)).Also note that Val 2 (T, h 0 ) = 0.Then, where h 0 (•) is the solution of system (1) that is generated by the optimal feedback control (u 0 (t, h)) (16).Note that (t 0 , h 0 , m 0 ) Thus, it follows from (20 We now successively consider the regions ( 14) of set [0, T] × [0, L] to describe the construction of set W 2 .
(1) Consider the following points: where set G 1 is defined in (14).It follows from the form of the optimal control (16) that in system (1), Then, it follows from (21) that (2) Consider the following points: where set G 2 is defined in (14).It follows from the form of the optimal control (16) that in system (1), Then, it follows from (21) that (3) Consider the following points: where set Π 1 is defined in (14).
There exist two cases: (3.a)The graph of h 0 (t) does not reach ĥ1 on [t 0 , T].
Then, it follows from the form of the optimal control (16) that the solution of ( 1) is It follows from (27) that case 3.a occurs when Therefore, it follows from (21) that (3.b)The graph of h 0 (t) reaches ĥ1 : It follows from (27) that this case occurs when It follows from the optimality principle that However, it follows from case 1 that the latter expression is true when Therefore, Remark 2. Hereafter, the value of m 0 2 (t; t 0 , h 0 , m 0 ) can be calculated according to Formula (7).
Similarly to case 3, there exist two cases: (4.a)The graph of h 0 (t) does not reach ĥ3 on [t 0 , T].
It follows from (31) that this case occurs when It follows from the optimality principle that However, it follows from case 1 that the latter expression is true when Therefore, (5) Consider the following points: where set Π 3 is defined in (14).
There exist two cases: (5.a)The graph of h 0 (t) does not reach ĥ1 on [t 0 , T].
Then, it follows from the form of the optimal control (16) that the solution of ( 1) is It follows from (35) that this case occurs when Therefore, it follows from (21) that where x(t 0 ) is the starting point of Γ curve (see Figure 1).

The Maximal Viability Set
Set W 2 , as described in the previous section, is the maximal viability set for problem ( 1), ( 6), and the following theorem is true.
Theorem 1.The following statements are true: Set W 2 (19) is weakly invariant with respect to the differential inclusion: ẇ ∈ Y(w), where
Also note the expression for M 2 (18): where The following estimates are true: Apply ( 43) and ( 45) to (44): Therefore, for any (2) It follows from the construction of W 2 that all the trajectories (m 0 2 (t; t 0 , h 0 , m 0 , u(•))) that start from the inner points ((t 0 , h 0 , m 0 )) of W 2 stay inside this set for any t ∈ [t 0 , T].
We assume that such a trajectory corresponds to a point (t * , h * , m * ) on the boundary of W 2 .Then, it follows from the optimality principle that In other words, the trajectory does not leave W 2 .Thus, set W 2 is weakly invariant with respect to the differential inclusion (42).

The Inverse Problem
For models of chemotherapy for malignant tumors, it is often of interest to solve inverse problems, for example, to identify the growth parameter (r) in the model of this tumor and to reconstruct the treatment protocol (u(•)) for the patient using the current measurements of the tumor volume (m(t)) and the amount of drug (h(t)) in the body of the patient.
Consider a model according to the Gompertz law: The known parameters are: The feasible controls are measurable (piecewise constant) functions satisfying the following restrictions: The parameters to reconstruct are: It is assumed that the process (m * (•), h * (•)) generated by some unknown feasible control (u * (•)) is observed at the following time interval:

Discrete inaccurate measurements (m δ
i , h δ i ) of the process arrive in real time with a uniform time step ( t).The measurement error is δ = (δ m , δ h ): The inverse problem is to construct approximations (r δ ) of parameter r and piecewise constant approximating functions (u δ (•)) such that: 1.
u δ (t) ∈ U, t ∈ [0, T]; The trajectories (m δ (•), h δ (•)) of the system (47) that are generated by the reconstructed parameters (r δ and u δ (•)) converge uniformly to the observed process: An approach for constructing such approximations was suggested and justified in [21] using the constructions from auxiliary problems of calculus of variations.These constructions are the solutions of Hamiltonian systems.The feature of the approach is the use of non-convex auxiliary variational problems, namely, the integrands of the functionals in the auxiliary problems are d.c. functions [23].Such an approach proves the stability of the methods based on this approach with respect to the measurement and approximation errors.This approach reduces the inverse problems for dynamical systems to integration of linear ODE systems.
In this paper, we present the results of the numerical simulation for a method based on this approach, as explained and justified in [21].
This method was used in two numerical simulations.In the simulations, the parameters to reconstruct were: The graph of the therapy function f (h) is shown in Figure 2. To obtain the inaccurate measurements (m δ i , h δ i ) of process m * (•), h * (•) generated by u * (•), this process was constructed numerically and randomly perturbed with uniform distribution for two sets of the parameters of observation (the measurement error (δ) and the time step ( t)).In the first simulation, (δ m , δ h ) = (5 * 10 5 , 3 * 10 −2 ), t = T/20 = 0.4.In the second simulation, (δ m , δ h ) = (10 5 , 0.6 * 10 −2 ), t = T/100 = 0.08.The approximations of the presumably unknown parameters (52) were constructed based on these sets of data using the method proposed in [21].
The result of the identification in the first simulation are presented as follows.The approximation (r δ ) of parameter r (52), which characterizes the speed of the tumor's growth, is The results of the reconstruction of the control u * (•) in the first simulation are shown in Figure 3.In this figure (and in Figure 4), the black graph is the control (u * (t)) (52) to be reconstructed.This control was applied to the model's system (47) to simulate the inaccurate measurements of the basic (observed) trajectory.The red graph is the piecewise constant control (u δ (t)), which approximates the reconstructed control (u * (t)).It was constructed using the variational method suggested in [21].
The process (m δ (t), h δ (t)) generated by r δ and u δ (•) in the first simulation is shown in Figure 5.In this figure (and in Figure 6), the blue graph is the "observed" process ({m * (t), h * (t)}).It is the trajectory of the model's system (47) generated by the "unknown" control (u * (•)) (52) (to be reconstructed).It is supposed that the inaccurate measurements of this trajectory are known.To simulate them, this trajectory was perturbed at discrete points with random error (with uniform distribution).These data are indicated by black dots.The red graph is the process {m δ (t), h δ (t)}.It is the trajectory of the model's system (47) generated by the control (u δ (•)) (52) (which is the constructed approximation of the "unknown" control (u * (•))).The aim of these figures is to show that the reconstructed trajectory of the system remains close to the observed trajectory (see condition 4 (51) in the inverse problem statement).The results of identification in the second simulation are reported as follows.The approximation (r δ ) of parameter r (52), which characterizes the speed of the tumor's growth, is The results of reconstruction of the control (u * (•)) in the second simulation are shown in Figure 4.In this figure, the black graph is the control (u * (t)) (52) to be reconstructed.The red graph is the piecewise constant control (u δ (t)), which approximates the reconstructed control (u * (t)).
The process (m δ (•), h δ (•)) generated by r δ and u δ (•) in the second simulation is shown in Figure 6.In this figure, the blue graph is the "observed" process ({m * (t), h * (t)}).The black dots are the inaccurate measurements of this process.The red graph is the process ({m δ (t), h δ (t)}) generated by the control (u δ (•)) (52).The aim of these figures is to show that the reconstructed trajectory of the system remains close to the observed trajectory (see condition 4 (51) in the inverse problem statement).Legend:

Discussion
In this paper, models of chemotherapy for malignant tumors were studied.In particular, the problem of constructing the optimal control with the aim of minimizing the tumor volume at a given time point was considered.Possible directions for future research may include minimization of the total cost of the drug injected into the patient's body.For such cases, the cost functional (which is a treatment quality criterion) should have the following form: Another problem to consider in the future is minimization of the total volume of the drug in the body for the entire period of treatment, i.e., minimization of the quality criterion

dt + σ(m(T)).
All these problems can be solved by following the scheme proposed in this paper.The viability sets in these problems can also be constructed.
The models of chemotherapy considered in this paper are the simplest ones.They were previously studied in [3] for the case of a therapy function that is either linear or has a single maximum point.In this paper, we studied a model with a therapy function that has two maximum points.It describes a case in which two different therapeutic agents are used.One maximum point corresponds to maximum efficiency of the first agent and the second to maximum efficiency of the second agent.
The future development of the research presented in this paper will include the study of models tumor growth, which include the dynamics of the healthy cells and the body's immune system and their interaction with the cancer cells and the drug (see, for example, [6,7,16,17]).
The results of the numerical construction of the viability sets will be presented in the future works.
It should be noted that the studied models are simplified idealizations of real processes.Therefore, the obtained results are informative and recommendatory in nature and require further experimental verification.

Conclusions
This paper presents the results of a study of a chemotherapy model of a malignant tumor growing according to the Gompertz law and the generalized logistic law.The optimal therapy strategy and optimal treatment protocols, which were constructed in previous works, were discussed with the aim of minimizing the tumor volume at the control point.We also discussed managerial insights with respect to their structure.
In the model of chemotherapy, a non-monotonic therapy function with two maxima is considered.This function defines the effect of the therapeutic agents on the tumor.Such a therapy function leads to a specific form of the optimal controls.They are piecewise constant functions with no more than one switch.The specifics are that the Rankine-Hugoniot line plays an important role in the construction of the optimal protocols.The position of the initial state of the model with respect to this line affects the construction of the optimal treatment protocols.
The structure of the maximal viability set for the considered model with the generalized logistic law is described.It is the set of the initial states of the model (the tumor volume and the concentration of the drug in the body) such that there exists a treatment protocol (a control) that guarantees that the tumor volume will not exceed the threshold amount compatible with life.
An inverse problem of reconstruction of the protocol of the ongoing treatment and identification of the parameter of the intensity of the tumor's growth based on the observations of the dynamics of the treatment process is considered.Such problems have never been considered before.A solution to this problem is suggested.The results of the numerical simulations for this problem are presented herein.