Price and Treatment Decisions in Epidemics: A Differential Game Approach

Abstract

We consider a pharmaceutical company that sells a drug that is useful in the treatment of an infectious disease. A public authority buys the drug to heal at least a portion of the infected population. The authority has an overall budget for all health care costs in the country and can only allocate a (small) part of the budget to the purchase of the drug. The government chooses the amount of drug to be purchased in order to minimize both the number of infectious people and the perceived cost of the operation along a given time horizon. This cost can be modeled through a linear or quadratic function of the monetary cost (as generally happens in the literature) or through a specific function (blow-up) that makes the budget constraint endogenous. The pharmaceutical company chooses the price of the drug in order to maximize its profit and knowing the budget constraints of the buyer. The resulting differential game is studied by supposing the simplest possible dynamics for the population. Two different games are proposed and their solutions are discussed: a cooperative game in which the two players bargain for the price of the drug and the quantity is purchased with the aim of maximizing the overall payoff and a competitive game in which the seller announces a price strategy to the buyer and binds to it; the buyer reacts by choosing the quantity to be purchased. In the case of linear or quadratic costs, the solution provided (for budget levels is not high enough) that the government spends the entire budget to purchase the drug. This drawback does not occur when the blow-up cost function is used.

1. Introduction

Starting from the seminal paper Ref. [1], a huge body of literature on dynamic models for infectious diseases has been produced, as evidenced by the numerous books on the subject (see, e.g., [2,3,4,5,6]). Mathematical models of epidemics usually consist of a system of differential equations governing the dynamics of the relevant state variables (susceptible, infective, recovered, etc.). According to Ref. [7], the main scope of mathematical modeling in epidemiology is “to develop models that assist in the decision-making process by helping to evaluate the consequences of choosing one of the available alternative strategies.”

A very interesting issue in the study of epidemic models is the identification of therapeutic strategies that minimize the relevant negative features of the disease at a minimum cost. A social planner wishes to minimize the infected part of the population by the administration of a therapy to the highest possible number of patients. However, therapy is usually costly, and this poses a dilemma.

The choice of the therapy’s cost function is relevant in the formulation of the optimal control problem for the treatment of infectious diseases. For example, in the administration of a drug for a certain number of patients, the most natural way to introduce the cost would be through the $(priceofthedrugsadministered)\times (numberoftreatedpatients)$ (monetary cost). However, in epidemic models, as well as in other economic applications, the cost is usually modelled by a function that represents the subjective perception of the decision-maker.

On the other hand, often, the cost function is simply given to make the problem tractable.

Let u be the fraction of a population (P) treated by some drug, and let p be the unitary cost of the therapy. Then, the (monetary) cost of the treatment is $k=puP$. Surprisingly, in many papers, that cost is simply represented by u or by $u^2$ without any reference to the price or to the number of people treated.

A specific problem is given by extremely expensive therapies in the presence of a very high number of patients to be healed. To treat all infected patients would mean sometimes spending all or a very large part of the national budget allocated to health. This is the case, for example, for the treatment of hepatitis C virus (HCV) by a costly drug, such as sofosbuvir (see e.g., Ref. [8]). Such a choice is impractical or, in other words, has a cost (in terms of utility) that is infinite. A cost function incorporating such characteristics should be adopted in this case, e.g., the blow-up cost function introduced in Ref. [9] seems appropriate in such a case.

In this paper, we consider a pharmaceutical company which is establishing the price of a drug and a social planner who is determining the number of people to treat.

In Section 2, the basic epidemic model used as a test problem is described. The decision problem, together with a discussion about the cost functions, is presented in Section 3. Section 4 deals with the cooperative bargaining game, while, in Section 5, a characterization of the Stackelberg equilibrium is provided. Section 6 concludes the paper.

2. The Epidemic Model

The simplest epidemic model proposed in the literature, i.e., the Susceptible-Infective-Susceptible (SIS) (see [10]) is used as test model. This model is appropriate for certain types of bacterial diseases, such as gonorrhea and meningitis.

Let us consider a population of N individuals, divided into two classes: the class S of those who are healthy, but susceptible to contracting the infection, and the class I of those who have contracted the infection and can transmit it to susceptible individuals through interpersonal contacts. It is $N=S+I$. In order to make the problem simple and meaningful at the same time, we neglect the demographic processes and assume that the transmission coefficient is normalized to one, so that the dynamics of S and I are governed by the system of differential equations

$$\left\{\begin{array}{ccc}{\displaystyle \frac{dS}{dt}}\hfill & =& -{\displaystyle \frac{IS}{N}},\hfill \\ {\displaystyle \frac{dI}{dt}}\hfill & =& {\displaystyle \frac{IS}{N}}.\hfill \end{array}\right.$$

(1)

Adding the two equations in (1), we have $dN/dt=0$, so that N is constant and will be normalized to one.

In the rest of this section, we are interested in the “worst” case in which the total population is initially infected. Hence, let us assume that

$$I(0)=1;S(0)=0.$$

It follows that

$$S(t)=1-I(t),\forall t\ge 0.$$

System (1) reduces to the scalar equation

$${\displaystyle \frac{dI}{dt}}=(1-I)I.$$

(2)

Equation (2) has only one stable feasible equilibrium, $I=1$.

Assume that a fraction $u(t)\in [0,1]$ of the infected population is treated, at the time t, by some drug. Then, some of the healed infectives become susceptibles again. System (1) becomes:

$$\left\{\begin{array}{ccc}{\displaystyle \frac{dS}{dt}}\hfill & =& -IS+u(t)I,\hfill \\ {\displaystyle \frac{dI}{dt}}\hfill & =& IS-u(t)I.\hfill \end{array}\right.$$

(3)

Again, in Equation (3), it is $S=1-I$, so that

$${\displaystyle \frac{dI}{dt}}=\left(1-I-u(t)\right)I;I(0)=1.$$

(4)

3. The Decision Problem

The monetary cost of the treatment borne by the social planner is $k(t)=p(t)u(t)I(t)$, where $p(t)$ is the unitary price of the therapy. The pharmaceutical firm chooses the price ($p(t)$) that maximizes its total profits, given by

$$J_1(p)={\int }_0^1(p(t)-c)u(t)I(t)dt,$$

where $c\ge 0$ is a constant representing the production cost for the treatment of the entire population. In the rest of the paper, for the sake of simplicity, c is normalized to 1.

Hence, the firm solves the following optimal control problem:

$${\displaystyle \underset{0\le p}{max}{J}_1(p)}$$

subject to the constraint (4).

Let B be the total budget available to the authorities to meet healthcare costs. The social planner wishes to minimize the number of infectives at the minimum cost and, in any case, spending only part of his budget.

As is usual in epidemic literature, the treatment cost is perceived through a cost function

$$\zeta (p(t),u(t),I(t))\ge 0,$$

which is increasing with respect to its arguments.

The social planner wishes to minimize the number of infectives at the minimum perceived cost (see also Ref. [9]).

Consequently, the social planner chooses the fraction ($u(t)$) of the infectious population to be treated that minimizes its total costs, given by

$${\int }_0^1\left(I(t)+\zeta \left(p(t),u(t),I(t)\right)\right)dt+I(1).$$

(5)

Note that, as is usual for optimal control problems with a finite horizon, there is a salvage term, $I(1)$, in (5) that takes into account the number of infected people at the end of the considered time interval.

By setting

$$J_2(u)=-{\int }_0^1\left(I(t)+\zeta \left(p(t),u(t),I(t)\right)\right)dt-I(1),$$

the social planner solves the maximization problem

$${\displaystyle \underset{0\le u\le 1}{max}{J}_2(u),}$$

subject to the state variable dynamics (4) and to the budget constraint

$$B-p(t)u(t)I(t)\ge 0.$$

Cost Functions

The following cost functions are usually used in the literature:

• ${\zeta }_1=pu$, linear state independent cost (LSI);
• ${\zeta }_2=puI$, linear state dependent cost (LSD);
• ${\zeta }_3=\frac{p^2{u}^2}{2}$, quadratic state independent cost (QSI);
• ${\zeta }_4=\frac{p^2{u}^2{I}^2}{2}$, quadratic state dependent cost (QSD).

The LSD and QSD costs represent the linear and quadratic monetary costs, respectively. They have been used, e.g., in Refs. [11,12]. Although the LSI and QSI have been frequently used in literature (see, e.g., Ref. [13] for LSI and [14,15,16,17] for QSI), they remain unconvincing because they do not depend on the number of people treated.

The presence of expensive therapies and a large number of people to treat constitute an interesting case. To heal all infected patients would mean sometimes spending all or a very large part of the national budget allocated to health. Such a choice is impractical or, in other words, has a cost (in terms of utility) that is infinite. The blow-up cost function incorporates such characteristics. It is defined by ${\zeta }_5=\varphi (k(t))$ where

$$\varphi (k)={\displaystyle \frac{Bk}{4(B-k)}},blow-up state dependent cost (BSD).$$

The BSD cost function tends to infinity as the monetary cost approaches the available budget.

In the rest of the paper, we neglect cost functions that are independent of the number of treated people and definitely consider the following three cases:

• $\varphi (k)=k$, linear;
• $\varphi (k)={\displaystyle \frac{k^2}{2}}$, quadratic;
• $\varphi (k)={\displaystyle \frac{Bk}{4(B-k)}}$, blow-up;

where $k=p(t)u(t)I(t)$.

4. Cooperative Solution

Assume that the pharmaceutical firm and the social planner negotiate an agreement about the price of the drug and the quantity to be bought by the social planner. In other words, the two players act as a single player (cartel).

According to Ref. [18], a number of issues arise. Among them are the following questions:

• What is the most reasonable way to represent the cartel’s payoff?
• How should the players’ cooperation gains be divided among all players?

One way to represent the cartel pay-off is that proposed by Ref. [19], that is, a weighted average of the individual players’ pay-offs.

The determination of weights constitutes a problem in itself and will not be addressed here. Here, we simply use the same weights for the two players so that the cartel payoff is given by

$$J=J_1+J_2.$$

It is

$$J={\int }_0^1\left(\left(p(t)u(t)I(t)-\varphi (p(t)u(t)I(t)\right)\right)dt-{\int }_0^{1}u(t)I(t)dt-{\int }_0^{1}I(t)dt-I(1).$$

The two players solve the following joint optimal control problem:

$$\underset{u,p}{max}J(u,p)$$

(6)

subject to the constraints (4),

$$0\le u\le 1;p\ge 0;$$

(7)

and

$$B-p(t)u(t)I(t)\ge 0.$$

(8)

The problem (6)–(8) is a standard optimal control problem. However, in order for a solution to be acceptable, it must satisfy the requirements of both players, i.e., to respect the budget constraint of the buyer and to guarantee to both parties a pay-off higher than what they would get in a condition of disagreement.

If no agreement is reached, then the government does not buy the drug and nobody is treated, that is, $u(t)=0$ for any $0\le t\le 1$. Then, the disagreement payoffs are $J_1^d=0$ and $J_2^d=-2$ so that $J^d=J_1^d+J_2^d=-2$.

Assume that the optimal control problem (6)–(8) has a solution, $(u^{*}(t),p^{*}(t))$, and let $J^{*}$ be the corresponding cartel payoff. Then, an agreement is reached if and only if

$$J^{*}>-2.$$

The “dividend of cooperation”is given by

$${\Delta }^{*}=J^{*}-J^d=J^{*}+2.$$

Let us denote by ${\Delta }_1^{*},{\Delta }_2^{*}$ the dividends of cooperation given to player 1 and player 2, respectively. It must be ${\Delta }_1^{*}+{\Delta }_2^{*}={\Delta }^{*}.$

$$J^{*}=J^d+{\Delta }^{*}=\left(J_1^d+{\Delta }_1^{*}\right)+\left(J_2^d+{\Delta }_2^{*}\right).$$

The firm will receive the payoff $J_1^d+{\Delta }_1^{*},$ while the social planner will receive $J_2^d+{\Delta }_2^{*}.$

A fundamental problem that comes out is how to distribute the cooperation dividend among the players. There are different approaches to this problem (see, e.g., Ref. [18] for a general discussion). We suppose there are no transfers of utility so that the firm receives the payoff

$$J_1^{*}={\int }_0^1\left((p^{*}(t)-1)u^{*}(t)I^{*}(t)\right)dt,$$

while the payoff of the social planner is

$$J_2^{*}=-{\int }_0^1\left(I^{*}(t)+\varphi \left(p^{*}(t)u^{*}(t)I^{*}(t)\right)\right)dt-I^{*}(1).$$

Hence, an agreement is signed if and only if $J_1^{*}>0$ and $J_2^{*}>-2$.

4.1. Necessary Conditions

Assuming that a solution of Equations (6)–(8) exists, candidate optimal controls are selected through suitable necessary conditions (see Ref. [20]). The Hamiltonian associated with Equations (6)–(8) is

$$H(I,u,p,\lambda )=puI-\varphi (puI)-uI-I+\lambda I(1-u-I).$$

Assume that $(u^{*}(t),p^{*}(t))$ is a solution of Equations (6)–(8), and let $I^{*}(t)$ be the corresponding optimal path of the state variable. Then,

• $(u^{*}(t),p^{*}(t))$ maximizes $H(I^{*}(t),u,p,\lambda (t))$ subject to the constraints (7) and (8).
• $${\displaystyle \frac{d\lambda }{dt}}=-{\displaystyle \frac{\partial H^{*}}{\partial I}}.$$
• $$\lambda (1)=-1.$$

Note that the derivatives of H are computed along the optimal control and the optimal state variable path.

Remark 1.(Interior maxima).

Let us look for the interior maxima of the hamiltonian (H). It should be

$${\displaystyle \frac{\partial H}{\partial p}}=0,{\displaystyle \frac{\partial H}{\partial u}}=0,$$

that is,

$$(1-{\varphi }^{\prime }(puI))uI=0;(1-{\varphi }^{\prime }(puI))pI-(1+\lambda )I=0.$$

From the first equation, we obtain $1-{\varphi }^{\prime }(puI)=0.$ Substituting in the second equation, we get $1+\lambda =0$. We conclude that H has no interior maxima.

Remark 2. (Corner Solutions).

The domain of H is

$${\mathbb{D}}_1=\left\{(u,p)|u=0;p\ge 0\right\}\cup \left\{(u,p)|0<u\le 1;0\le p\le {\displaystyle \frac{B}{uI}}\right\}$$

in the linear and quadratic case and

$${\mathbb{D}}_2=\left\{(u,p)|u=0;p\ge 0\right\}\cup \left\{(u,p)|0<u\le 1;0\le p<{\displaystyle \frac{B}{uI}}\right\}$$

in the blow-up case.

It is easy to see that H is bounded above on its domain. Nevertheless, ${\mathbb{D}}_1$ is closed but unbounded, while ${\mathbb{D}}_2$ is not closed and unbounded. It follows that H can fail to have a maximum.

Note that if $0<u\le 1$, it is ${\displaystyle \frac{\partial H}{\partial p}}>0$ iff

$$1-{\varphi }^{\prime }(puI)>0.$$

(9)

1. 

Linear case.

H is constant with respect to p. The following can be immediately verified:

(a) 

If $1+\lambda <0$, then H obtains its maximum at $u=1;$

(b) 

If $1+\lambda =0$, then H obtains its maximum at any point $0\le u\le 1;$

(c) 

If $1+\lambda >0$, then H obtains its maximum at $u=0.$

2. 

Quadratic case.

The following proposition holds.

Proposition 1.

The inequality (9) is true for all $0\le k<1$. It follows that the maximum of H, whenever it exists, belongs to the set $\left\{u=0;p\ge 0\right\}$ or to the set $\left\{0<u\le 1;puI=min(1,B)\right\}$.

Proof. 

See Appendix. □

It is $H(u=0,p)=\lambda I(1-I)-I.$

(a) 

Let $0<B<1.$ It is

$$H(0<u\le 1,puI=B)=B-{\displaystyle \frac{B^2}{2}}-(1+\lambda )uI+\lambda I(1-I)-I.$$

i. 

If $1+\lambda <0$, then H obtains its maximum at $\left(u=1;p=\frac{B}{I}\right).$

ii. 

If $1+\lambda =0$, then H obtains its maximum at any point $\left(0<u\le 1;p=\frac{B}{uI}\right).$

iii. 

If $1+\lambda >0$, then $H\left(u,\frac{B}{uI}\right)$ decreases with respect to u and

$$\underset{u\to 0^{+}}{lim}H\left(u,\frac{B}{uI}\right)=B-{\displaystyle \frac{B^2}{2}}+\lambda I(1-I)-I>H(u=0,p).$$

It follows that ${\displaystyle \underset{{\mathbb{D}}_1}{sup}H=B-{\displaystyle \frac{B^2}{2}}+\lambda I(1-I)-I}$, but H fails to have a maximum.

(b) 

Let $B\ge 1.$ It is

$$H\left(0<u\le 1, puI=1\right)=\frac{1}{2}-(1+\lambda )uI+\lambda I(1-I)-I.$$

i. 

If $1+\lambda <0$, then H obtains its maximum at $\left(u=1;p=\frac{1}{I}\right).$

ii. 

If $1+\lambda =0$, then H obtains its maximum at any point $\left(0<u\le 1;p=\frac{1}{I}\right).$

iii. 

If $1+\lambda >0$, then $H\left(u,\frac{1}{uI}\right)$ decreases with respect to u and

$$\underset{u\to 0^{+}}{lim}H\left(u,\frac{1}{uI}\right)={\displaystyle \frac{1}{2}}+\lambda I(1-I)-I>H(u=0,p).$$

It follows that ${\displaystyle \underset{{\mathbb{D}}_1}{sup}H=\frac{1}{2}+\lambda I(1-I)-I}$, but H fails to have a maximum.

3. 

Blow-up case.

The following proposition holds.

Proposition 2.

The inequality (9) is true for all $0\le k<{\displaystyle \frac{B}{2}}$. It follows that the maximum of H, whenever it exists, belongs to the set $\left\{u=0;p\ge 0\right\}$ or to the set $\left\{0<u\le 1;puI={\displaystyle \frac{B}{2}}\right\}$.

Proof. 

See Appendix. □

It is $H(u=0,p)=\lambda I(1-I)-I.$ Moreover,

$$H(0<u\le 1,puI={\displaystyle \frac{B}{2}})={\displaystyle \frac{B}{4}}-(1+\lambda )uI+\lambda I(1-I)-I.$$

(a) 

If $1+\lambda <0$, then H obtains its maximum at $\left(u=1;p=\frac{B}{2I}\right).$

(b) 

If $1+\lambda =0$, then H obtains its maximum at any point $\left(0<u\le 1;p=\frac{B}{2uI}\right).$

(c) 

If $1+\lambda >0$, then $H\left(u,\frac{B}{2uI}\right)$ decreases with respect to u and

$$\underset{u\to 0^{+}}{lim}H\left(u,\frac{B}{2uI}\right)={\displaystyle \frac{B}{4}}+\lambda I(1-I)-I>H(u=0,p).$$

It follows that ${\displaystyle \underset{{\mathbb{D}}_2}{sup}H={\displaystyle \frac{B}{4}}+\lambda I(1-I)-I}$, but H fails to have a maximum.

4.2. Individual Rationality

If an agreement is not reached between the seller and the public authority, then the drug is not bought at all, that is, $u(t)=0$ for any $0\le t\le 1$. Hence,

$$J_1=J_1^d=0;J_2=J_2^d=-2.$$

An agreement, to be acceptable, must guarantee to each of the contractors a payoff that is at least equal to what he would get in the absence of an agreement. Therefore, if $(u^{*}(t),p^{*}(t))$ is a solution of (6)–(8), then it must be

$$J_1^{*}\ge 0;J_2^{*}\ge -2;J^{*}>-2.$$

(10)

Equation (10) gives additional constraints on the control variables and essentially says that the price ($p^{*}(t)$) must be high enough to guarantee positive profits to the producer.

At the same time, the price must not be excessively high so as to make the purchase by the social planner not convenient because the costs arising from the purchase would exceed the social benefits obtained with the treatment of patients.

4.3. Sufficient Conditions

Despite the apparent simplicity of the problem and the boundedness of the functional to be maximized, we have seen that problems can be posed on the existence of a solution that satisfies the classic necessary conditions. With regard to sufficient conditions, the Hamiltonian’s structure renders the classical theorems of Mangasarian or Arrow type inapplicable (see Ref. [20], pp. 287–289). This is essentially due to the negative sign of the multiplier $\lambda$ and ultimately, to the presence of the salvage value $-I(1)$.

4.4. One Time-Switch Treatments

Necessary conditions suggest that a solution of the cooperative differential game, provided it exists, must be a corner solution. Unfortunately, no further information comes from the application of them.

In this section, following the insights given by the necessary conditions, we try to calculate the optimal treatment and price strategies within a special class of treatment policies.

We assume that the control ($u(t)$) is a bang-bang control, that is, $u(t)\in \{0,1\}.$ The simplest case is given when there is only one time switch between zero and one.

There are two possibilities: to delay the start of the treatments (i.e., $u=0$ for $0\le t<\widehat{t}$ and $u=1$ for $\widehat{t}\le t\le 1$) or to stop treatments before the final time (i.e., $u=1$ for $0\le t\le \widehat{t}$ and $u=0$ for $\widehat{t}<t\le 1$).

It can be shown that the two classes of treatment control generate the same optimal solution. Hence, we only give details for one case, i.e., delayed treatments.

Proposition 3.

Let us assume that

$$u(t)=\left\{\begin{array}{cc}0\hfill & 0\le t<\widehat{t},\hfill \\ 1\hfill & \widehat{t}\le t\le 1,\hfill \end{array}\right.$$

$$p(t)=\omega (1+t-\widehat{t}),\widehat{t}<t\le 1,$$

where $\widehat{t}\in [0,1]$. Note that the value of $p(t)$ as $0\le t<\widehat{t}$ does not affect the value of the functional to be maximized. It follows that

$$I(t)=\left\{\begin{array}{cc}1& 0\le t<\widehat{t},\hfill \\ {\displaystyle \frac{1}{1+t-\widehat{t}}}& \widehat{t}\le t\le 1,\hfill \end{array}\right.$$

and

$$I(1)={\displaystyle \frac{1}{2-\widehat{t}}}.$$

Consequently, it is

$$J_1=\omega (1-\widehat{t})-ln(2-\widehat{t}).$$

$$J_2=-\varphi \left(\omega \right)(1-\widehat{t})-\widehat{t}-ln(2-\widehat{t})-{\displaystyle \frac{1}{2-\widehat{t}}}.$$

$$J=\left(\omega -\varphi (\omega )\right)(1-\widehat{t})-\widehat{t}-2ln(2-\widehat{t})-{\displaystyle \frac{1}{2-\widehat{t}}}.$$

Proof. 

See Appendix. □

Observe that whenever $\widehat{t}\le t\le 1$, it is $p(t)u(t)I(t)=\omega$. Hence, $\omega$ is the monetary cost of the treatment at time t. In this special case, it is constant in time.

The following problem,

$${\displaystyle \underset{\omega ,\widehat{t}}{max}J,}$$

(11)

is solved subject to the constraints

$$0\le \widehat{t}\le 1;0\le \omega \le B,$$

(12)

$$\omega \ge \delta (\widehat{t}),$$

(13)

$$\varphi (\omega )\le g(\widehat{t}),$$

(14)

$$\delta (\widehat{t})={\displaystyle \frac{ln(2-\widehat{t})}{1-\widehat{t}}},g(\widehat{t})={\displaystyle \frac{3-4\widehat{t}+{\widehat{t}}^2-(2-\widehat{t})ln(2-\widehat{t})}{(1-\widehat{t})(2-\widehat{t})}}.$$

Let

$$\mathbb{D}=\left\{(\widehat{t},\omega )|0\le \widehat{t}<1;0\le \omega \le B;\delta (\widehat{t})\le \omega ;\varphi (\omega )\le g(\widehat{t})\right\}$$

be the domain of our maximization problem.

Note that $\delta$ and g are strictly increasing in $[0,1]$. Moreover, $\delta (0)=ln2\approx 0.693$, $g(0)={\displaystyle \frac{3-2ln2}{2}}\approx 0.806$, ${\displaystyle \underset{\widehat{t}\to 1}{lim}\delta (\widehat{t})=\underset{\widehat{t}\to 1}{lim}g(\widehat{t})=1.}$

The Linear Case

In this case, the functional J does not depend on $\omega$, but $\omega$ plays a role through the constraints. Precisely,

$$J=-\widehat{t}-2ln(2-\widehat{t})-{\displaystyle \frac{1}{2-\widehat{t}}}.$$

The following proposition holds:

Proposition 4.

Let $({\widehat{t}}^{*},{\omega }^{*})$ be the solution of (11)–(14).

• If $0\le B\le ln2\approx 0.693$, then ${\widehat{t}}^{*}=1$, and the value of ${\omega }^{*}$ does not matter.
• If $B>ln2$, then

  $${\widehat{t}}^{*}=0,{\omega }^{*}\in \left[ln2,min\left(B,{\displaystyle \frac{3-2ln2}{2}}\right)\right].$$

Proof. 

See Appendix. □

Remark 3.

Proposition 4 shows that an agreement arises if and only if the budget (B) is large enough. The agreement price is $p^{*}(t)={\omega }^{*}(1+t)$, where ${\omega }^{*}$ is any value belonging to the nonempty interval $\left[ln2,min\left(B,{\displaystyle \frac{3-2ln2}{2}}\right)\right]$.

The dividends of the cooperation are ${\Delta }_1^{*}={\omega }^{*}-ln2$ and ${\Delta }_2^{*}=\frac{3}{2}-ln2-{\omega }^{*}$, respectively. If we adopt the equal dividend criteria, then ${\omega }^{*}=3/4,$ provided that $3/4<B$. Otherwise, it should be ${\omega }^{*}=B$.

The Quadratic Case

In this case, the functional is

$$J=\left(\omega -{\displaystyle \frac{{\omega }^2}{2}}\right)(1-\widehat{t})-\widehat{t}-2ln(2-\widehat{t})-{\displaystyle \frac{1}{2-\widehat{t}}}.$$

The individual rationality conditions (13) and (14) must be satisfied, that is, for any $0\le \widehat{t}<1$,

$$\delta (\widehat{t})\le \omega \le \sqrt{2g(\widehat{t})}.$$

The following proposition holds’:

Proposition 5.

Let $({\widehat{t}}^{*},{\omega }^{*})$ be the solution of Equations (11)–(14).

• If $0\le B\le ln2\approx 0.693$, then ${\widehat{t}}^{*}=1$, and the value of ${\omega }^{*}$ does not matter.
• If $ln2<B$, then

  $${\widehat{t}}^{*}=0,{\omega }^{*}=min(B,1).$$

Proof. 

See Appendix. □

Remark 4.

Proposition 5 shows that an agreement arises if and only if the budget (B) is large enough. In that case, it is optimal to treat all the infected people at every time point. This result is similar to the one obtained in the linear cost case.

The agreement price is $p^{*}(t)={\omega }^{*}(1+t).$ If the budget is low ($B<1$), then ${\omega }^{*}=B$, so the social planner spends the entire healthcare budget on the purchase of the drug. If the budget is high ($B>1$), then ${\omega }^{*}=1$, and the social planner spends a fraction of the healthcare budget on the purchase of the drug.

If $B<1$, then ${\Delta }_2^{*}>{\Delta }_1^{*}$, that is, the agreement gives the government a larger dividend than the one that it gives the company. If $B\ge 1$, then ${\Delta }_2^{*}={\Delta }_1^{*}.$

The Blow-up Case

In this case, the payoff is

$$J=\left(\omega -{\displaystyle \frac{B\omega }{4(B-\omega )}}\right)(1-\widehat{t})-\widehat{t}-2ln(2-\widehat{t})-{\displaystyle \frac{1}{2-\widehat{t}}}.$$

The individual rationality conditions (13) and (14) must be satisfied, that is, for any $0\le \widehat{t}<1$,

$$\delta (\widehat{t})\le \omega \le \gamma (\widehat{t};B),$$

where $\gamma$ is defined by

$$\gamma (t;B)={\displaystyle \frac{4Bg(t)}{B+4g(t)}}.$$

The following proposition holds:

Proposition 6.

Let $({\widehat{t}}^{*},{\omega }^{*})$ be the solution of Equations (11)–(14).

• If $0\le B\le {\displaystyle \frac{2ln2(3-2ln2)}{(6-5ln2)}}\approx 0.882$, then ${\widehat{t}}^{*}=1$, and the value of ${\omega }^{*}$ does not matter.
• If ${\displaystyle \frac{2ln2(3-2ln2)}{(6-5ln2)}}<B<2ln2\approx 1.386$, then

  $${\widehat{t}}^{*}=0,{\omega }^{*}=ln2.$$
• If $2ln2<B<6-4ln2\approx 3.227$, then

  $${\widehat{t}}^{*}=0,{\omega }^{*}={\displaystyle \frac{B}{2}}.$$
• If $6-4ln2\le B$, then

  $${\widehat{t}}^{*}=0,{\omega }^{*}={\displaystyle \frac{2(3-2ln2)B}{6+B-4ln2}}.$$

Proof. 

See Appendix. □

Remark 5.

Again, an agreement arises if and only if the budget B is large enough. The agreement price is $p^{*}(t)={\omega }^{*}(1+t).$

If the budget is low, then ${\omega }^{*}=ln2$, and the agreement gives zero dividend to the firm.

For intermediate budget levels, it is ${\omega }^{*}=B/2$, and the agreement gives positive dividends to both players. It is ${\Delta }_1^{*}=B/2-ln2$, ${\Delta }_2^{*}=3/2-B/4-ln2.$ The cooperation dividend of the firm increases with respect to the social planner’s budget. On the contrary, the dividend of the social planner decreases with respect to the budget which is a little bit surprising. It is ${\Delta }_1^{*}>{\Delta }_2^{*}$ if $B>2.$

If the budget is high, then ${\omega }^{*}=\frac{2B(3-2ln2)}{6+B-4ln2}$, and the agreement gives zero dividend to the social planner.

The solution of the cooperative game can be also computed numerically. Following Ref. [21] and using a Runge–Kutta fourth order discretization scheme for the state and the adjoint equations, we obtained Figure 1, Figure 2 and Figure 3. The budget was set at $B=1$. In Figure 1, the graphs of I, $\lambda$, and u are illustrated. Moreover, the maximum admissible price ($p_{max}$) is also shown, that is, the price requiring the total consumption of the budget.

In Figure 2 and Figure 3, the graphs of I, $\lambda$, u, and p are illustrated. Furthermore, the fraction of the budget used for the treatment is shown.

5. Stackelberg Equilibria

The Nash equilibria presume that the government chooses the treatment level without any information about the price of the drug. This seems rather unfeasible. It is more convincing to assume that the firm commits itself to a pricing strategy, and the government, knowing it, decides the treatment level. Hence, the framework of a Stackelberg game seems more suitable for this problem—having the firm as the leader and the government as the follower.

Suppose that the leader (the firm) announces, at time $t=0$, a price strategy, $p(t)\ge 0,0\le t\le 1$, and commits itself to that strategy. The follower (the social planner), knowing the leader’s decision, selects the treatment rate, $u(t)\in [0,1],0\le t\le 1$, to maximize the objective functional

$$J_2=-{\int }_0^1\left(I(t)+\varphi \left(p(t)u(t)I(t)\right)\right)dt-I(1),$$

subject to the constraints

$${\displaystyle \frac{dI}{dt}}=I(1-u-I);I(0)=1;$$

(15)

$$k(t)=p(t)u(t)I(t)\le B;0\le t\le 1.$$

(16)

The follower’s Hamiltonian is

$$H_2(I,\lambda ,u)=-I-\varphi (k)+\lambda I(1-u-I),$$

where $\lambda$ is the adjoint variable. We look for interior maxima of the Hamiltonian. It is ${\displaystyle \frac{\partial H_2}{\partial u}}=0$ if and only if

$$\lambda +{\varphi }^{\prime }(k)p=0.$$

• If $\varphi (k)=k$ (linear cost function), then no interior maxima of $H_2$ occurs, and the best response of the follower is a corner solution (bang-bang control).
• If $\varphi (k)=k^2/2$ (quadratic cost function), then a local maximum of the Hamiltonian is

  $$u^{*}=-{\displaystyle \frac{\lambda }{p^{2}I}},$$

  (17)

  and it is the best response of the social planner, provided that it satisfies the budget constraint.
  Moreover, the adjoint variable $\lambda (t)$ satisfies the differential equation ${\displaystyle \frac{\partial \lambda }{\partial t}}=-{\displaystyle \frac{\partial H_2}{\partial I}}$. From inserting the value of u given by Equation (17), we have

  $${\displaystyle \frac{\partial \lambda }{\partial t}}=1-(1-2I)\lambda ,$$

  with the transversality condition $\lambda (1)=-1$.
  The leader, foreseeing the the follower’s decision, maximizes the functional

  $$J_1={\int }_0^1(p(t)-1)u^{*}(t)I(t)dt,$$

  (18)

  subject to the constraints (15) and (16). From inserting $u^{*}$ into Equation (18), we have that the leader maximizes

  $$J_1={\int }_0^1-{\displaystyle \frac{\lambda (t)(p(t)-1)}{p^2(t)dt}},$$

  subject to the constraints

  $$\left\{\begin{array}{cccc}{\displaystyle \frac{dI}{dt}}\hfill & =& {\displaystyle \frac{\lambda }{p^2}}+I(1-I);\hfill & I(0)=1;\hfill \\ {\displaystyle \frac{\partial \lambda }{\partial t}}\hfill & =& 1-(1-2I)\lambda ,;\hfill & \lambda (1)=-1.\hfill \end{array}\right.$$
  The leader’s Hamiltonian is

  $$H_1(I,\lambda ,p,\psi ,\sigma )=-\lambda {\displaystyle \frac{p-1}{p^2}}+\psi \left[I\left(1-I\right)+{\displaystyle \frac{\lambda }{p^{2}I}}\right]+\sigma \left[1-(1-2I)\lambda \right]$$

  where $\psi$ and $\sigma$ are the adjoint variables. By maximizing $H_1$ with respect to p, we obtain

  $$p^{*}=2(1+\psi ),$$

  and substituting in Equation (17) gives

  $$u^{*}=-{\displaystyle \frac{\lambda }{4I{(1+\psi )}^2}}.$$
  The adjoint variables $\psi (t)$ and $\sigma (t)$ satisfy the differential equations ${\displaystyle \frac{\partial \psi }{\partial t}}=-{\displaystyle \frac{\partial H_1}{\partial I}}$, ${\displaystyle \frac{\partial \sigma }{\partial t}}=-{\displaystyle \frac{\partial H_1}{\partial \lambda }},$ respectively, and the transversality conditions $\psi (1)=0$, $\sigma (0)=0$.
  Inserting $p^{*}=2(1+\psi )$ into the state and adjoint equations leads to the following two-point boundary value problems:

  $$\left\{\begin{array}{cccc}{\displaystyle \frac{dI}{dt}}\hfill & =& {\displaystyle \frac{\lambda }{4{(1+\psi )}^2}}+I(1-I);\hfill & I(0)=1;\hfill \\ {\displaystyle \frac{\partial \lambda }{\partial t}}\hfill & =& 1+(2I-1)\lambda ;\hfill & \lambda (1)=-1;\hfill \\ {\displaystyle \frac{\partial \psi }{\partial t}}\hfill & =& (2I-1)\psi -2\lambda \sigma ;\hfill & \psi (1)=0;\hfill \\ {\displaystyle \frac{\partial \sigma }{\partial t}}\hfill & =& {\displaystyle \frac{1}{4(1+\psi )}}-(2I-1)\sigma ;\hfill & \sigma (0)=0.\hfill \end{array}\right.$$

  (19)
  System (19) can be solved numerically to give the solution of the Stackelberg game, provided that $p^{*}\ge 0$, $0\le u^{*}\le 1$, $0\le k^{*}\le B$, where $k^{*}(t)=-{\displaystyle \frac{\lambda (t)}{2(1+\psi (t))}}.$
• If $\varphi (k)={\displaystyle \frac{Bk}{4(B-k)}}$ (blow-up cost function) then a local maximum of the Hamiltonian is

  $$u^{*}={\displaystyle \frac{B\left(2\sqrt{-\lambda }-\sqrt{p}\right)}{2Ip\sqrt{-\lambda }}}.$$

  (20)
  By substituting (20) into the leader Hamiltonian and differentiating with respect to p, we obtain that $x=\sqrt{p^{*}}$ is the solution of the cubic equation

  $$x^3+(1+\psi )x-4(1+\psi )\sqrt{-\lambda }.$$

  (21)
  As in the quadratic case, it is argued that the solution of the Stackelberg game is characterized by the following two-point boundary value problems:

  $$\left\{\begin{array}{cccc}{\displaystyle \frac{dI}{dt}}\hfill & =& -{\displaystyle \frac{B(2\sqrt{-\lambda }-x)}{2x^2\sqrt{-\lambda }}}+I(1-I);\hfill & I(0)=1;\hfill \\ {\displaystyle \frac{\partial \lambda }{\partial t}}\hfill & =& 1+(2I-1)\lambda ;\hfill & \lambda (1)=-1;\hfill \\ {\displaystyle \frac{\partial \psi }{\partial t}}\hfill & =& (2I-1)\psi -2\lambda \sigma ;\hfill & \psi (1)=0;\hfill \\ {\displaystyle \frac{\partial \sigma }{\partial t}}\hfill & =& {\displaystyle \frac{B(x^2-1-\psi )}{4x{(-\lambda )}^{3/2}}}-(2I-1)\sigma ;\hfill & \sigma (0)=0.\hfill \end{array}\right.$$

  (22)
  Equations (21) and (22) can be solved numerically to give the solution of the Stackelberg game provided that $p^{*}\ge 0$, $0\le u^{*}\le 1$. Note that $k^{*}=B\left[1-\sqrt{{\displaystyle \frac{-x^2}{4\lambda }}}\right]$, so that the budget constraint is satisfied.

6. Concluding Remarks

In this paper, we considered a pharmaceutical company that sells a drug that is useful for the treatment of an infectious disease. A public authority buys the drug to heal at least a portion of the population that has contracted the disease for which the drug is indicated.

The authority has an overall budget for all healthcare costs in the country and can only allocate one (small) part of the budget to the purchase of the drug. This constitutes a constraint for government decisions (budget constraint).

The government chooses the amount of drug to be purchased in order to minimize both the number of infectious people and the perceived cost of the operation along a given time horizon. Its control is given by the fraction (u) of treated people (treatment rate).

This cost can be modelled through a linear or quadratic function of the monetary cost (as generally happens in the literature) or through a specific function (blow-up) that makes the budget constraint endogenous.

The pharmaceutical company chooses the price (p) of the drug in order to maximize its profit and knowing the budget constraint of the buyer.

The resulting differential game is studied by supposing the simplest possible dynamics for the population. Nevertheless, the solution of the problem presents many difficulties.

Two different games are proposed: a cooperative game in which the two players bargain for the price of the drug and the quantity purchased with the aim of maximizing the overall payoff and leaving to each of the players a payoff at least equal to that of disagreement, or a competitive game in which the seller announces a price strategy to the buyer and binds to it; the buyer reacts by choosing the quantity to be purchased (both players make their choices in order to maximize their payoff).

As far as the cooperative game is concerned, from applying the Pontryagin maximum principle, we observed that the problem only possesses corner solutions. A solution of the game was then calculated within the class of u-bang-bang controls with a single time switch. We observed that the solution depends on the available budget. A minimum level of budget is required so that the agreement between the seller and buyer can be reached. In the case of linear or quadratic costs, the solution means (for insufficient budget levels) that the government spends the entire budget to purchase the drug. This drawback does not occur when the blow-up cost function is used.

In the leader-follower game, we tried to characterize the solution using the maximum principle. In the case of linear costs, the buyer adopts a bang-bang strategy, and this makes it difficult to formulate and solve the maximization problem of the leader. In the case of quadratic costs, it is possible to explicitly calculate (depending on the state and adjoint variables) both the treatment rate chosen by the buyer and the selling price and to successively characterize the problem as a two-point boundary value problem (TPBVP) constituted by four differential equations. In the case of blow-up costs, it is not possible to explicitly calculate the price, and therefore, it is necessary to couple the system of four differential equations with an algebraic equation which has the price as a solution.

The purpose of the paper was two-fold: to show how the choice of the cost function has an important influence on the type of solution and that it must be closely linked to the type of problem dealt with and not be exclusively made to make the problem tractable, and to show how, even for seemingly simple problems, the use of necessary and sufficient conditions can be non-trivial and that caution is needed when trusting numerical solutions.

The hypothesis that, in the Stackelberg game, the seller adopts an open-loop strategy that constitutes a rather severe limitation (even if in this case we can think of the need to sign a binding contract over time) goes without saying. I think that it is interesting to study feedback solutions and this, together with further extensions, is a reason for further research.