HIV vs. the Immune System: A Differential Game

: A differential game is formulated in order to model the interaction between the immune system and the HIV virus. One player is represented by the immune system of a patient subject to a therapeutic treatment and the other player is the HIV virus. The aim of our study is to determine the optimal therapy that allows to prevent viral replication inside the body, so as to reduce the damage caused to the immune system, and allow greater survival and quality of life. We propose a model that considers all the most common classes of antiretroviral drugs taking into account different immune cells dynamics. We validate the model with numerical simulations, and determine optimal structured treatment interruption (STI) schedules for medications.


Introduction
The genetic code of the HIV virus consists of a ribonucleic acid, the RNA.HIV belongs to the retroviruses family, characterized by the presence of an enzyme, DNA-polymerase RNA-independent, capable of transcribing the genetic code RNA into DNA.This ability allows the virus to integrate its genome into the one of the cells it infects, so that the integrated virus would not be defeated nor by the immune response nor by drugs.
HIV has a marked tendency to mutate: mutations are errors that individual viral particles make in replicative cycles.Each error leads to the appearance of a virus, which is more or less analogous to the original one.Mutations are mostly "disadvantageous" for the viral species, as mutated viruses tend to disappear.Nevertheless some mutations are "beneficial" and allow mutated viruses to acquire drug resistance and immune response.One of the most effective mechanisms used by HIV to evade the immune response and treatment is, in fact, its aptitude to change.
The main HIV target cells are T lymphocytes of type CD4, fundamental in the adaptive response against a variety of pathogens and oncogenes.
The overall function of the immune system is to prevent or limit infections.An immune response is generally divided into innate and adaptive immunity.Innate immunity occurs immediately, when circulating innate cells recognize a problem.Adaptive immunity occurs later, as it relies on the coordination and expansion of specific adaptive immune cells.
In particular, CD4+ lymphocytes are very important cells in the immune system, in fact they recognize the various uninvited guests organism (viruses, bacteria, protozoa, fungi, worms and cancer cells) through biochemical messages, and they activate the areas of the immune system most suitable to counter their presence.A large number of CD4+ paralyzes the immune system, exposing the body to any risk of infection and cancer [1].
Actually, HIV can be suppressed by a combination of antiretroviral therapy (ART) consisting of three or more antiretroviral (ARV) drugs.In principle, ART cannot eradicate HIV infection from the "sanctuaries" (mainly lymph-nodes and lymphatic system); it controls viral replication within a body reducing viral burden, thus in turn it allows a functional improvement of an individual's immune system that basically regains the capacity to fight off infections [2].It has been proved that people with HIV, subject to ART therapy, can have a healthy and productive life, in fact after the advent of the ART therapy the mortality curve of HIV-infected patients started to decline.
On the other hand, the lack of a definitive eradication of viral reservoirs determines two important consequences: the anti-HIV treatment should be life-long; the longer the treatment, the more likely the development of drug-resistance.On this basis, an intense research activity has been developed in last decades, usefully enlarging the anti-HIV molecules repertoire.Actually there are more than 20 approved antiretroviral drugs, divided into 6 different pharmacological classes.Anti-retroviral drugs are broadly classified through the phase of the retrovirus life-cycle that they inhibit, as follows Typical combinations involve the conjunctive use of either two NRTIs and one NNRTI, or one PI (with or without Ritonavir) in combination with two NRTIs.
HAART (Highly active antiretroviral therapy) is an abbreviation for all protocols involving combinations of drugs, which are active against different molecular targets in the life cycle of HIV.These medications are administered in the form of the high concentration cocktails.This approach was born in 1995-1996 with the introduction of the second class of antiretroviral drugs: Protease inhibitor (PI), administered in combination with drugs of the first generation, or Reverse Transcriptase Inhibitors (NRTI and NNRTI: see zidovudine or AZT azidothymidine).
Among the benefits of treatment there is a decreased risk of progression to AIDS and a decreased risk of death.The adverse effects and the complexity of treatment regimens (due to the high number of pills and to dosing frequency) may reduce patient's compliance [3].
A correct administration of antiretroviral drugs should be daily scheduled and uninterrupted; it requires care and precision, even in relation to meals and to subministration of other drugs.The omission of a few doses leads to a reduction of drug concentration in blood, therefore the residual level could become lower than the one necessary to inhibit the virus.Not only does this allow the resumption of viral replication, but it also facilitates the inexorable emergence of resistant virus.Resistant mutants tend to persist, making the drug ineffective, even if the intake of drugs is carried out according to the rhythms and the correct doses.A scarce adherence to the prescriptions and its consequent emergence of resistant mutants may cause failure of the treatment.
The therapy must prevent viral replication in the body, rather than the complete eradication of the infection, that remains chronic, so as to reduce the damage caused to the immune system and to allow greater survival and quality of life.
The study of HIV biological evolution and of its relationship with the immune system is used to determine a therapy policy that can defeat the viruses.Despite it seems counterintuitive, the aim is to determine a proper dosage of drugs, which defends against viruses only moderately and which may benefit both the host and the virus, i.e., without creating strong environmental pressures.A stable long-term coexistence can be reached, hopefully avoiding drug-resistance development.
Mathematical modeling of HIV infection has proven to be instrumental for the modern understanding basis of the AIDS pathogenesis [4].There exist several attempts to formalize the evolution of HIV and the use of drugs to limit its diffusion.Most of the related literature is linked to the study of the HIV dynamics in the body and only few of them apply the Optimal control theory considering also the possibility to control the drugs action.An exhaustive survey that collects all these works and classifies them according to the most relevant criteria con be found in [5].
In this work we present an application of the differential Game Theory to a medical-therapeutic context for the HIV treatment A first attempt has been performed by Wu and Zhang in [6].Here we present a differential game which considers the classes of antiretroviral drugs currently most used and different immune cells dynamics, with the aim of representing as much as possible the real setting of this problem.We consider two players: The HIV virus and the immune system, supported by antiretroviral therapy.We look for an optimal therapeutic treatment in terms of a Nash Equilibrium, with the aim of finding a proper dosage of drugs which defends against viruses, such that a stable long-term life expectation may be obtained.
This paper is structured as follows.In Section 2 we introduce the involved variables and parameters and we formalize the problem in terms of a differential game.In Section 3 we characterizes the Open Loop Nash Equilibrium.In Section 4 we present some numerical simulations to show two different situations that may occur and how the algorithm performs.Section 5. concludes with some comments.All analytical computations are reported in Appendix A within the Theorems' proofs.In Appendix B are listed the parameter values used in the numerical simulations.

The Model
We present a generalized model within the Game Theory approach, in order to determine the optimal antiretroviral treatment against the HIV infection.We take into accounts 4 types of cells: HIV viruses, CD4+ T Helper cells (adaptive immunity), macrophage and monocytes cells (innate immunity) and immune precursor/effector Cytotoxic T lymphocyte cells.
After the primary HIV infection has established, an acute HIV syndrome appears with a wide dissemination of viruses and with seeding of lymphoid organs.This phase can last between three and nine weeks.From the ninth week the clinical latency phase may start, here the AIDS symptoms are not macroscopically manifested, nevertheless the number of T lymphocytes begins to decrease and the viral load increases.This stage can persist many years.Finally, at a last stage, there is the occurrence of the AIDS symptoms and of other opportunistic infections.In addition, this stage can last for years and ends with the death of the host [7].As there exist so many strains of viruses, in order to take into account all of them, without making the model intractable, we distinguish the HIV viruses between "sensitive", (V s ) and "resistant" (V r ) to the therapy.
In the model we adopt the idea of Herod et al. [8] and Nowak [9], assuming that the immune response (T h ) to a viral infection creates some subpopulations of lymphocytes (T s ) that are specific only for sensitive viruses, some others (T r ) that are specific only for resistant viruses, and finally the subpopulation T g that is effective against both sensitive and resistant viruses.Furthermore, the mutation of the initial viral infection may cause the death of the entire population of lymphocytes.
Given the fact that HIV actually cannot be eradicated, our aim is to formulate a differential game between HIV and the Immune System in which the existence of an equilibrium would lead to an optimal drug therapy, which enables a stable long-term coexistence between virus and host, in other terms, a longer life expectancy.
In the following tables we present, in alphabetical order for the symbols, the variables and the parameters of the model.In particular: Table 1 lists the state functions; Table 2 lists the control functions and finally Tables 3 and 4 list the parameters.
Percentage of i cells that recognize the virus j ∈ [0, 1] ρ j Average number of virions j infecting a cell Let us collect all the state variables in the following array: The last four controls are related to the HIV player, nevertheless, in line with most notation of the cited literature, we can limit to the two controls u v s,s and u v r,r only, observing that u v s,s = 1 − u v s,r and u v r,r = 1 − u v r,s .Let's collect the immune system-therapy controls in the array: and the viruses controls in the array: In Table 3 we show the parameters with the same unit of measure (1/day), while all the other parameters are in Table 4.
HAART therapy can prolong the patient's life, however, it has many side effects, so that a long-term administration may become difficult and it is necessary to determine the minimal dose of drugs that prevents the viral replication.
The immune system has to maximize the number of uninfected macrophages, uninfected T-cells, and immuno precursors/effectors, minimizing, at the same time, the side effects of drugs.Such an object is represented by the following payoff: where Γ = {h, s, r, g} and Ω ∈ {co, f i, ib, ii, pi, rt}.
On the other hand, HIV tends to maximize the number of sensitive and resistant viruses, the number of latently and actively infected T-cells, minimizing, at the same time, the mutation costs.Its payoff fuction is: where Γ = {h, s, r, g} and Θ = {s, r}.Observe that drug toxicity costs and HIV mutation costs are assumed quadratic in the controls to represent their increasing increments of scale.
The T-cells undergo to different evolutions depending on whether they have been infected or not by the virus.In what follows we distinguish between uninfected and infected cells.In particular, we expose the dynamics of the following five classes of cells: 1. Uninfected CD4+ T-helper cells; 2. Uninfected macrophages and CD8+ CTL precursors and effectors; 3. Latently and actively infected CD4+ T cells by sensitive and resistant viruses; 4. Infected macrophages by sensitive and resistant viruses; 5. Sensitive and resistant viruses; Writing first the terms which contribute positively to their growth, and after the negative terms.We also stress the features that characterize each group of cells.
The evolution of every type of cell is subject to a natural death rate that can be represented in its dynamics by a (negative) decaying term.Namely, let x i ∈ x be a particular type of cell, such a decaying term is −µ x i x i and appears at the end of each considered dynamics.
In the evolution of the uninfected CD4+ T-Helper cells we consider features taken from different models by Kirschner et al. [10], Caetano et al. [11] and Joshi [12].The dynamics are the following One common characteristic among the above cells is constituted by the presence of the immune busting effect that causes an increase of the immune barriers, see [12].It is proportional to the number of particular uninfected CD4+ T-Helper cells and it depends on the control u ib .Note the positive term u ib (t)T i , i ∈ Γ = {h, s, r, g} that appears at the end of the first line in the dynamics of each type of cell T i .
Moreover, some cells are infected by sensitive viruses: Their number is proportional to the number of sensitive viruses (V s ) and to the dimension of the community of cells itself.Coefficient k 1,s represents the infection rate, the factors 1 − u f i (t) and 1 − u co (t) describe the actions of fusion inhibitors and coreceptor antagonists against the infection of the cells.These drugs interfere with binding, fusion and entry of HIV to the host cell.Finally (1 − Q T i ,Vs is the portion of generic CD4+ T Helper cells that does not recognize and does not obstruct sensitive viruses.Similarly, cells are infected also by resistant viruses, and their number is proportional to the number of resistant viruses (V r ) and to the dimension of the community of cells itself.What makes the difference is that fusion inhibitors and coreceptor antagonists can not counteract T i cells infection by resistant viruses (V r ), therefore the drugs action is null.The factor (1 − Q T i ,Vr is the portion of generic CD4+ T Helper cells that does not recognize and does not obstruct resistant viruses.These effects are represented by the second lines of the dynamics.
Observe that the negative terms −kV which appear in the third line of Equation ( 3) represent the number of T h cells converted to T s , T r or T g cells respectively.Each one of such terms obviously appears with a positive sign in Equations ( 4)-( 6) for uninfected T s , T r and T g cells.
The term −k 3 M a h,s (t)+M a h,r (t) gives the portion of T i cells contaminated by infected macrophages, and the sensitivity/resistant differentiation in it constitutes a novelty of our model.
Once explained the common features of the CD4+ T-Helper cells, let's explicate the terms which characterise the dynamics of the naive T lymphocytes (T h ) in Equation (3).
The term: gives the proliferation of uninfected CD4+ T h cells.It includes both an external (not plasma) contribution of cells from sources, such as the thymus and lymph nodes, and an internal (plasma) contribution from CD4+ T h cells differentiation.
The term: represents the production of T h -cells due to cloning in the presence of an antigen, taking into account the maximum number of lymphocytes, T max .
The dynamics for the uninfected macrophages, CD8 + CTL precursors and effectors are the following: According with [10], macrophages dynamics Equation ( 7) consider a constant proliferation source (s 1,M h ) and sensitive and resistant infections elements Here reverse transcriptase inhibitors (RTI) and protease inhibitors (PI) counteract the action of sensitive viruses.In particular the first ones inhibit reverse transcription, and the second ones block the viral protease enzyme necessary to produce mature virions upon budding from the host membrane.Each one of such terms obviously appears with a positive sign in Equations ( 26) and (27).
About immune precursors/effectors (I p , I e ) dynamics Equations ( 8) and ( 9) we extend the model presented by Wodarz and Nowak in [13] by differentiating infected T-cells with actively infected T h , T s , T r and T g cells ( T a i,j ).If a T h cells is infected it becomes either latently or actively infected.The latently infected cells (T l h,j with j ∈ {s, r}) can be activated and become actively infected (T a h,j with j ∈ {s, r}) their activation rate is denoted by k 2 .The actively infected cells are short living and will normally be killed upon activation with a high death rate µ T a h,j with j ∈ {s, r}.Latently and actively infected T-cells dynamics are the following: The positive terms +k 10) represent the number of healthy T h -cells infected by sensitive viruses and healthy T h -cells contaminated by actively infected macrophages (M a h,s ), respectively.Observe that in the first one V s can be counteracted by fusion and co-receptor inhibitors, in the second one drugs cannot obstruct M a h,s (t)-action.
is the number of infected cells that convert from latently to actively infected.This process is counteract by reverse transcriptase and integrase inhibitors which inhibit reverse transcription and integration of viral DNA respectively.This element appears with a positive sign in Equation ( 11) for actively T h -cells infected by sensitive viruses.
The negative contribute −i s I e (t)T a h,s (t) is a novelty of our model, it represents the number of actively infected naive T cells (T a h,s ) killed by CTL effectors (I e ), at a constant rate i s .We assume this number to be proportional to the number of immune effectors (I e ).
Similar considerations can be observed for Equations ( 12) and ( 13) with the variation that there isn't any drug that can counteract resistant viruses.Moreover, the action of CTL effectors occurs at a constant rate i r .
The dynamics for infected T s cells are the following: The dynamics for infected T r cells are the following: The dynamics for infected T g cells are the following: Ṫ a g,r (t) = k 2 T l g,r (t) − i r I e (t)T a g,r (t) − µ T a g,r T a g,r (t) As said for naive T lymphocytes (T h ) the same holds for Equations ( 14)-(17) of T s cells, for Equations ( 18)-( 21) of T r cells and for Equations ( 22)-(25) of T g cells.
The dynamics for infected macrophages are the following: We assumed that there is no latently infected macrophage population since the virus seems to replicate once inside them.We also assume that macrophages produce virus at a slow constant rate, sparing the host cell, so there is only natural death, not death by bursting like that for infected T cells [14].These equations present two infections rate, the former (k M 1,s ) related to sensitive viruses (V s ), the latter related to resistant viruses (V r ).
Finally, we assume that reverse transcriptase and protease inhibitors can counteract macrophages infected by sensitive viruses [15,16], while there are not drugs to counteract resistant viruses.
The dynamics for sensitive (V s ) and resistant (V r ) viruses are the following: The term: 28) is a source of virus that accounts for viral contributions to the plasma from both external compartments, such as the lymph system, as well as virus produced by infected cells in the plasma [17].This source is counteracted by the protease inhibitors therapy u pi (t).
The term: T a i,s (t) represents the growth of sensitive viruses at a constant rate π s .This growth is counteracted by protease inhibitors, and it is proportional to the number of actively infected cells contaminated by sensitive viruses and to the mutation rate from sensitive to other sensitive viruses (u v s,s ).Finally: represents the growth (at a constant rate π r ) of resistant viruses that mutate into sensitive viruses (at a rate 1 − u v r,r = u v r,s ).Also this growth is counteracted by protease inhibitors, and it is proportional to the number of actively infected cells, contaminated by resistant viruses.Parameters b 2 , g, π s , π r are assumed positive and constant.
The negative contributions: indicate that viruses which infect lymphocytes are not free to circulate in the blood, and so they can not infect other cells at the same time.The constant ρ s indicates the average number of sensitive virions infecting a cell: in our numerical simulations we set this number equal to 1, but the model permits also other values for this parameter, that is a cell could be infected by several viruses.The last term: represents the number of sensitive virions blocked by specific immune response (represented by T s cells) and by unspecific immune response (represented by T g cells).
For what concerns resistant viruses dynamics Equation (28), considerations are the same just seen for sensitive viruses, with the only difference that protease inhibitors can not counteract V r .
The boundary conditions include the initial states: and the final values: Each therapy control, that represents the various drugs dosage, is assumed to vary within the range [0, 1], the same holds for the HIV-controls, which represent the mutation rates of viruses, so that: and recalling the control u 2 (t) definition, we have: Summarizing, the payoffs Equations ( 1) and ( 2) together with the associated state functions Equations ( 3)-( 29) and with the constraints Equations ( 30)-(33) constitute a differential game characterized by: • 27 state variables (Table 1); • 8 controls (Table 2): 6 controls for the therapy and 2 controls for the virus; • 98 parameters (Tables 3 and 4).

Open Loop Nash Equilibrium
In line with [6], we look for an Open Loop Nash equilibrium of the HIV/therapy game, considering the Pontryagin Maximum Principle necessary conditions.The model is difficult to analyze due to the number of variables and constraints and to the fact that state and co-state equations are "coupled".For the sake of simplicity, let us relax the non negativity constraints Equation (31) on the state variables and verify them ex post, assuming that: Theorem 1. [Therapy equilibrium necessary conditions] The optimal Therapy control satisfies the following necessary conditions: where û1 (t) is the unique stationary point of the Therapy Hamiltonian Equation (A2).The co-states functions λ 1 (t) satisfy the co-state equations: with transversality conditions The Therapy Hamiltonian is not necessarily concave w.r.t.its controls, nevertheless, some specific values of the model parameters lead to concavity, as the following Theorem states.
Proof.See Appendix A.
If concavity holds, the necessary conditions for the Therapy control Equation (35) takes the form: where û1 (t) is the unique stationary point of the Therapy player.Something more specific can be said for the HIV equilibrium strategies.

Theorem 3. [HIV equilibrium necessary conditions]
The HIV mutation rates satisfy the following necessary conditions u * 2 (t) = (u v * s,s (t), u v * r,r (t)), where: The functions λ 2,26 (t) and λ 2,27 (t) are the co-states associated to the HIV virus variables V s and V r respectively.More generally the co-state equations are: with transversality conditions: Proof.See Appendix A.
Observe that, due to the transversality conditions, it straight follows that u v * s,s (t f ) = u v * r,r (t f ) = 1, so that by Equation (33) we obtain that in equilibrium condition the virus mutation rates from sensitive viruses to resistant viruses and vice versa vanish, that is Although it might appear as negative the fact that resistant viruses do not turn to sensitive any more, we must underline that this happens at the final time, when the number of viruses is sensibly decreased and a consequent long life expectancy has been obtained.In fact, our aim is to determine a therapy capable to prevent viral replication with its related side effects.
Substituting the optimal controls in the state Equations ( 3)-( 29) and in the co-state Equations ( 36) and ( 43), together with Equations (30), ( 37) and (44), we obtain a system of 81 autonomous strongly non linear first order differential equations, more precisely • 27 equations on the states; • 54 equations on the adjoint variables; • 27 boundary conditions for the states at time t 0 ; • 54 boundary conditions for the adjoint variables at time t f .

Numerical Simulations
An analytical solution of the state/co-state system can be obtained through a symbolic software, like for example, Wolfram Mathematica, nevertheless such a solution turns out to be as much long as uninformative.So that a numerical resolution turns out to be crucial in order to get some enlightening results.We formulate and apply an efficient and intuitive algorithm by combining the Forward-Backward Sweep algorithm [18][19][20][21] and a 4th order Runge-Kutta routine (see e.g., [22]).We performed several simulations and we always got a unique solution to the necessary conditions.In what follows we present two particular instances, using the most common parameter values cited in the related literature, (see Appendix B).Please observe that differently from the theoretical analysis, here the HIV mutation rates u v r,s and u v s,r are explicitly shown, simply for an interpretation point of view.

Concave Instance
We first present a situation where the therapy Hamiltonian is concave, that is if conditions Equations ( 38) and (39) hold.Such instance is characterized by very effective although highly toxic drugs which significantly reduce the virus mutation rates u v r,s and u v s,r .
In Figure 1, the first six graphs represent therapy dosages, the last two graphs show the HIV mutations, and the table shows the optimal payoffs related to immune system-therapy and to HIV, respectively.
Coreceptor antagonist (u co ) and fusion inhibitors (u f i ) are both used for interfering with binding, fusion and entry of HIV to the host cell.Their dosages are the same for all therapy period (1000 days), more precisely they are administered at maximum doses for a few days and then they are no longer employed.
Immune boosting drugs (u ib ) were introduced into this type of dynamics by Joshi in 2006 [12].They represent the drugs that increase immune defenses enhancing the number of cells of immune system.In Joshi's studies these drugs are used at a very low constant level (around 0.02) for the first 50 days.In our simulations, such a value turns out to be constantly equal to zero and this may have two explanations: first of all, it is not necessary to administrate a high toxicity drug if the lymphocyte counts remains high enough (around 800-900 cells/mm 3 ), (see Figure 2).Secondarily, increasing the number of T lymphocytes means increasing the number of target cells for HIV virus, thus risking an increase in viral replication.As we expected, Integrase (u ii ) and Reverse Transcriptase Inhibitors (u rt ) have very similar optimal dosages, because they are both used in the same phase of virions creation.To be more accurate, Reverse Transcriptase Inhibitors are used at the highest level for a few more days, in fact the interaction between the two enzymes, integrase (IN) and reverse transcriptase (RT) is functional and plays an important role in replication of the virus.Evidence has been accumulating that IN not only interacts closely with RT during reverse transcription but also remains associated with RT during the formation of the preintegrative complex.
Protease inhibitors drugs (u pi ) block the viral protease enzyme necessary to produce mature virions upon budding from the host membrane.These drugs are used at maximum level for about the first 350 days, then the dosage decreases gradually becoming zero the last 100 days.
In fact it is well known that the resistance to protease inhibitors is a stepwise process.Briefly, primary mutations often have only a small effect on resistance, so that the PI therapy persists longer.Conversely, during continuous PI therapy, additional mutations emerge in the protease (commonly referred to as secondary mutations) leading to high-level PI resistance.These secondary mutations often have only a minor effect on drug susceptibility if they appear all alone but, in the presence of primary mutations, they can lead to a drastic increase in resistance, with the consequent decrease of the PI therapy, [23].
For what concerns HIV, the resistant-to-sensitive mutation rate from (u v r,s ) is maximum for the first 350 days, then it drastically decreases, thanks to drug effects.At the same instant, the sensitive-to-resistant virus mutation rate (u v s,r ), which is initially null, moves to the maximum level, because of the virus survival tendency.Nevertheless it does not remain at the maximum level, it begins to decrease, (around day 550) thanks to the protease inhibitors effect (u pi ).At the end of the planning interval both the virus mutation rates vanish, as observed in Section 3 after Theorem 3. The Therapy and HIV payoffs are listed in the lower part of Figure 1.Observe that the high toxicity of drugs causes high costs for the immune system, and leads to low payoff values.
The white blood cell count is one of the most important tests to verify the presence of the HIV virus.It is an indicator of the health status of a patient and is one of the key factors in determining when to start HAART and prophylaxis for opportunistic infections.The tests usually measure the white blood cells present in the sample and calculate the percentage of CD4+ T lymphocytes.This value can be variable among different individuals and influenced by several factors.In general, the percentage of CD4+ T-cells is stable and it can be a parameter for evaluating the immune functions of a patient.Usually, it has a level between 500 and 1000 cells/cm 3 in a healthy patient, and if it goes below this threshold, a possible infection could appear and damage the immune system.Observe that, despite the high toxicity of drugs, the lymphocyte count is sufficiently high (around 800-900 cells/mm 3 ,) for the entire time interval (see Figure 2).
In Figures 2-4 are shown the dynamics of all the 27 state variables of the model.In particular, in Figure 2 we find the dynamics of various types of healthy immune cells and in Figure 3 we find the infected cells dynamics.Finally in Figure 4 we find the dynamics of sensitive (V s ) and resistant viruses (V r ).Note that all optimal state functions are positive, so that the non-negativity constraints are fully satisfied.Sensitive viruses population decreases right from the start, under the effects of the therapy, and it becomes almost zero around day 400.Resistant viruses have the same behaviour and they become zero around day 800.

Nonconcave Instance
Let us now consider a nonconcave therapy instance, characterized by drugs with low toxicity but not very effective.In Figure 5, we can observe a prolonged intake over time is required.The performed algorithm is stable, in fact, results do not vary with the initial values given to adjoint variables at time t 0 (as it happens for other studies of the related literature, see e.g., [19,20]).Computational time is about 15-60 min in an Intel(R) core(TM) i-7 4.700 HQ CPU, 2.40 GHz.
HIV mutation rate u v r,s is maximum for the first 20 days then it decreases thanks to the combined action of therapy controls u co , u f i and u pi .The reduction of u v r,s involves the immediate increase of the sensitive-to-resistant viruses mutation rate (u v s,r ) to the maximum level.This causes the necessity to use the controls u ii and u rt at the highest level till day 250 in order to reduce to zero mutations u v s,r .Thanks to the prolonged therapy the latter remain at that null level until the end of the considered time interval.HIV attempts a further sting in the tail, by increasing the rate of mutation u v r,s to the highest level.In the meantime controls u ii and u rt are brought to zero for a small time interval, while the controls u co , u f i and u pi continue to be used at their maximum level.Until the control u v r,s remains at a high level, the controls u ii and u rt perform a gradual increase.
We can observe how the combined action of 5 drugs leads to a decrease of the mutation rate u v r,s , around day 700.It continues to decrease for the remaining 300 days, after which both the viruses mutation rates are null.Immune boosting (u ib ) continues to have a null dosage, for the same reasons seen in the concave instance.
The payoff for therapy and HIV are listed in the lower part of Figure 5. Due to the low toxicity of the drugs, the optimal immune system-therapy payoff turns to greater w.r.t. the one obtained in the concave instance, whilst the HIV payoff turns to be lower.In Figures 6-8 we report the dynamics of all 27 variable states.Note again that all optimal state functions are positive, so that the non-negativity constraints are fully satisfied.Although the drugs are not so effective, the lymphocytes count is high (around 800-900 cells/mm 3 ).Sensitive viruses population (V s ) becomes null around day 300, while resistant viruses population (V r ) goes to zero after 900 days.

Conclusions
We have considered the game theory approach in a medical-therapeutic context for HIV treatment.We have formulated a differential game which describes the interaction between the immune system of a patient and the virus action.In order to model the efficient HAART therapy, we have considered 27 state functions and eight controls (six for the therapy and two for HIV).We look for an open Loop Nash equilibrium in order to find a stable long-term coexistence between the two players.
The optimality necessary conditions lead to a system of 81 first order strongly non-linear differential equations.Once proved the existence and uniqueness of the solution of such a system, we have performed some numerical simulations in order to get enlightening results.Just for explanatory examples, we considered two specific instances.Mathematically speaking the two instances differ in the concavity properties of the Hamiltonians w.r.t. the related controls.From the medical point of view, the concave instance is characterized by very effective although highly toxic drugs, whereas the nonconcave instance is represented by low effective and moderately toxic drugs.We observe quite different behaviours in the therapy strategies according to the type of drugs used.And this, in our opinion, is very interesting, as our aim is to find a tool which can support the clinician when combining the different drugs in the HAART therapy.Having an idea of the dynamical reactions to a given biological effect can help to make the optimal therapeutic decision.
An immediate, although not trivial, prospect could be the analysis of the problem following the Hamilton-Jacobi-Bellman approach to find a time consistent Markovian Nash equilibrium [24].
Another possible further extension of this model could be obtained in matching the Game Theory approach with the Fuzzy system issue [25][26][27][28], paying attention to the unavoidable complexity increase.
In this Appendix we report the proofs of Theorems 1 and 2. We first introduce the Hamiltonian functions for the two players in order to make the proofs easier to be read, see [29].The Therapy Hamiltonian is: The HIV Hamiltonian is The 6 × 6 Therapy Hessian matrix w.r.t.its controls is where As the first leading principal minor (−A co ) is negative, the matrix cannot be in any case positively or semi-positively defined, so that the unique stationary point can be either a a saddle point or a local maximum.In the former case, one of the two extremes of the control feasible set is the maximum point.In the latter case, we must evaluate the Hamiltonian function not only in the extremes of the feasible control set but also in the stationary point, getting the point with the maximum objective value.In both case Equation (35) holds.The function evaluation necessary to determine the optimal Therapy control can be performed by mean of the numerical quadrature formulas with three different methods of numerical integration: the Composite rectangle method, the Composite trapezoid method; and the Composite Cavalieri-Simpson method, see [22].
Proof of Theorem 2. Let Equations ( 38) and (39) hold, then the leading principal minors of the Therapy Hessian matrix are the following 2th order det H Finally the determinant of the Hessian matrix, (principal minor of 6th order), is The Hessian matrix is negative definite, viz the Therapy Hamiltonian function is concave in u 1 .
Proof of Theorem 3. The partial derivatives of the HIV Hamiltonian w.r.t. the related controls are they admit a unique HIV stationary point, namely Equation (42).The HIV Hessian matrix is it is negative definite, because B s,r , B r,s > 0, therefore the HIV Hamiltonian function is strictly concave in its controls u v s,s (t) and u v r,r (t), and so the necessary conditions for the optimal controls are given by Equation (41).

B. Parameters
The sets of parameter values we presented in the numerical Section 4 are very similar to the ones most commonly used in the related literature [5].The two instances mainly differ from each other for the drugs toxicity parameters and HIV mutation costs, to consider concave and nonconcave properties.Table B1 shows the values we assigned to drugs toxicity and mutation costs for each instance.Tables B2 and B3 show all common parameter values for both instances.Finally Table B4

3 T 3 T 3 V 0 j
Initial value of latentlyT i cells infected by viruses V j cells mm Initial value of actively T i cells infected by viruses V j cells mm max Maximum CD4+ T Helper cell population level cells mm Initial value of viruses V j cells mm 3

I e 0 0 0 h 0 h 0 s 0 r 0 g 0 s 0 r
Initial value of I e cells 10 I p Initial value of I p cells 10 M Initial value of M h cells 1 T Initial value of T h cells 900 T Initial value of T s cells 100 T Initial value of T r cells 100 T Initial value of T g cells 1 T l,0 i,s Initial value of latently T i cells infected by viruses V s 10 T a,0 i,s Initial value of actively T i cells infected by viruses V s 0 T l,0 i,r Initial value of latently T i cells infected by viruses V s 10 T a,0 i,r Initial value of actively T i cells infected by viruses V r 0 M a,0 h,s Initial value of actively M h cells infected by viruses V s 10 M a,0 h,r Initial value of actively M h cells infected by viruses V r 10 V Initial value of viruses V s 500 V Initial value of viruses V r 300

Table B2 .
Common parameters at both instances (Part 1).Death rate of cells infected by V s due to immune response 10 −2 1 day i r Death rate of cells infected by V r due to immune response 5 × 10 −4 1 day k Rate T h convert to specific immune reaction cells (T s or T r ) 10 −6 1 day k Rate T h convert to unspecific immune reaction cells (T g ) 10 −6 1

Table B3 .
Common parameters at both instances (Part 2) Growth rate of I e due to infected cells and I p 0.51 day Q T h ,Vs Percentage of T h cells that recognize the virus V s 0 Q T h ,Vr Percentage of T h cells that recognize the virus V r 0 Q Ts,VsPercentage of T s cells that recognize the virus V s 0.90Q Ts,Vr Percentage of T s cells that recognize the virus V r 0 Q Tr,VsPercentage of T r cells that recognize the virusV s 0 Q Tr,VrPercentage of T r cells that recognize the virus V r 0.90 Q Tg,Vs Percentage of T g cells that recognize the virusV s 0 Q Tg,VrPercentage of T g cells that recognize the virus V r 0 r rate of growth for CD4+ T Helper cell population 0.03 1

Table B4 .
Common boundary conditions at both instances