# Optimal Control of Heterogeneous Mutating Viruses

^{1}

^{2}

^{*}

^{†}

## Abstract

**:**

## 1. Introduction

## 2. Related Works

## 3. Evolutionary Model of Virus Mutations

- if virus ${V}_{1}$ meets virus ${V}_{1}$, then utility for both is equal to $\frac{{b}_{1}-{C}_{1}}{2}$. The virus incurs energy costs ${C}_{1}$ with probability $1/2$ if it cannot occupy a host organism, and achieves utility of ${b}_{1}$ with probability $1/2$ if it succeeds in occupation.
- if virus ${V}_{1}$ competes with virus ${V}_{2}$, then virus ${V}_{1}$ obtains utility of 0, and ${V}_{2}$ obtains a payoff of ${b}_{2}$.
- if virus ${V}_{2}$ meets ${V}_{2}$, then both viruses obtain a payoff of $\frac{{b}_{2}-{C}_{2}}{2}$.

${V}_{1}$ | ${V}_{2}$ | |

${V}_{1}$ | $({\displaystyle \frac{{b}_{1}-{C}_{1}}{2}},{\displaystyle \frac{{b}_{1}-{C}_{1}}{2}})$ | $(0,{b}_{2})$ |

${V}_{2}$ | $({b}_{2},0)$ | $({\displaystyle \frac{{b}_{2}-{C}_{2}}{2}},{\displaystyle \frac{{b}_{2}-{C}_{2}}{2}})$. |

## 4. Epidemic Process for an Urban Population

#### 4.1. Epidemic Dynamics

#### 4.2. Objective Function

## 5. Optimal Control of Epidemics

#### 5.1. Structure of Optimal Control

**Proposition**

**1.**

- When ${h}_{i}(\xb7)$ are concave functions, then there exist time moments ${t}_{1}\in [0,T]$, such that:$$u\left(t\right)=({u}_{1}\left(t\right),{u}_{2}\left(t\right))=\left\{\begin{array}{c}(1,1),\phantom{\rule{4pt}{0ex}}for\phantom{\rule{4pt}{0ex}}0<t<{t}_{1};\hfill \\ (0,0),\phantom{\rule{4pt}{0ex}}for\phantom{\rule{4pt}{0ex}}{t}_{1}<t<T.\hfill \end{array}\right.$$
- When ${h}_{i}(\xb7)$ are strictly convex functions, then there exist time moments ${t}_{0},{t}_{1}$, $0<{t}_{0}<{t}_{1}<T$ such that:$${u}_{i}\left(t\right)=\left\{\begin{array}{c}0,\phantom{\rule{4pt}{0ex}}\phantom{\rule{4pt}{0ex}}\phantom{\rule{4pt}{0ex}}{\varphi}_{i}\left(t\right)\le {h}_{i}^{\prime}\left(0\right),\phantom{\rule{4pt}{0ex}}i=1,2;\hfill \\ {h}^{\prime -1}\left({\varphi}_{i}\right),\phantom{\rule{4pt}{0ex}}\phantom{\rule{4pt}{0ex}}\phantom{\rule{4pt}{0ex}}{h}_{i}^{\prime}\left(0\right)<{\varphi}_{i}\left(t\right)\le {h}_{i}^{\prime}\left(1\right),\phantom{\rule{4pt}{0ex}}i=1,2;\hfill \\ 1,\phantom{\rule{4pt}{0ex}}\phantom{\rule{4pt}{0ex}}\phantom{\rule{4pt}{0ex}}{h}_{i}^{\prime}\left(1\right)<{\varphi}_{i}\left(t\right),\phantom{\rule{4pt}{0ex}}i=1,2.\hfill \end{array}\right.$$

**Lemma**

**1.**

**Proof**

**of**

**Lemma**

**1.**

#### 5.2. Proof of Proposition 1

#### 5.2.1. ${h}_{i}$ Are Concave

#### 5.2.2. ${h}_{i}(\xb7)$ Are Convex

**Lemma**

**2.**

**Property**

**1.**

**Property**

**2.**

**Step 1.**At time T, we have $({\lambda}_{{I}_{1}}\left(T\right)-{\lambda}_{S}\left(T\right))=0$, $({\lambda}_{{I}_{2}}\left(T\right)-{\lambda}_{S}\left(T\right))=0$, and $({\lambda}_{R}\left(T\right)-{\lambda}_{{I}_{1}}\left(T\right))=0$ according to (8). ${\dot{\lambda}}_{{I}_{1}}\left(T\right)-{\dot{\lambda}}_{S}\left(T\right)=-{f}_{1}^{\prime}\left({I}_{1}\left(T\right)\right)<0$ and by analogy ${\dot{\lambda}}_{{I}_{2}}\left(T\right)-{\dot{\lambda}}_{S}\left(T\right)=-{f}_{2}^{\prime}\left({I}_{2}\left(T\right)\right)<0$ and ${\dot{\lambda}}_{R}\left(T\right)-{\dot{\lambda}}_{{I}_{1}\left(T\right)}={f}_{1}^{\prime}\left({I}_{1}\left(T\right)\right)+g\left(R\left(T\right)\right)>0$, therefore expressions $({\lambda}_{{I}_{1}}\left(T\right)-{\lambda}_{S}\left(T\right))$, $({\lambda}_{{I}_{2}}\left(T\right)-{\lambda}_{S}\left(T\right))$, $({\lambda}_{R}\left(T\right)-{\lambda}_{S}{I}_{1}\left(T\right))$ are positive in open interval $(0,T)$.

**Step 2.**(Proof by contradiction).

**Case I.**In this case, we prove that $({\lambda}_{{I}_{1}}\left(t\right)-{\lambda}_{S}\left(t\right))>0.$ Suppose that $({\lambda}_{{I}_{1}}\left(t\right)-{\lambda}_{S}\left(t\right))=0$, $({\lambda}_{{I}_{2}}\left(t\right)-{\lambda}_{S}\left(t\right))=0$ and $({\lambda}_{R}\left(t\right)-{\lambda}_{{I}_{1}}\left(t\right))>0$ then

**Case II.**We have to prove that $({\lambda}_{{I}_{2}}\left(t\right)-{\lambda}_{S}\left(t\right))>0$. This is a symmetric case to Case I. Using the same reasoning, we obtain

**Case III.**In this case, we prove that $({\lambda}_{R}\left(t\right)-{\lambda}_{{I}_{1}}\left(t\right))>0$ in a similar way.

#### 5.3. Quadratic Cost Functions

## 6. Numerical Simulation

## 7. Conclusions and Discussion

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## References

- Capasso, V. Mathematical Structures of Epidemic Systems; Lecture Notes in Biomathematics; Springer: Berlin/Heidelberg, Germany, 1993; Volume 97. [Google Scholar]
- Conn, M. Handbook of Models for Human Aging; Academic Press: London, UK, 2006. [Google Scholar]
- Newman, M.E.J. Threshold effects for two pathogens spreading on a network. Phys. Rev. Lett.
**2005**, 95, 108701. [Google Scholar] [CrossRef] [PubMed] - Pastor-Satorras, R.; Castellano, C.; Van Mieghem, P.; Vespignani, A. Epidemic processes in complex networks. Rev. Mod. Phys.
**2015**, 87, 925–979. [Google Scholar] [CrossRef][Green Version] - Rota Bulò, S.; Bomze, I. Infection and immunization: A new class of evolutionary game dynamics. Games Econ. Behav.
**2011**, 71, 193–211. [Google Scholar] [CrossRef][Green Version] - Kermack, W.O.; McKendrick, A.G. A contribution to the mathematical theory of epidemics. Proc. R. Soc.
**1927**, 115, 700–721. [Google Scholar] [CrossRef] - Wickwire, K.H. A note on the optimal control of carrier-borne epidemics. J. Appl. Probab.
**1975**, 12, 565–568. [Google Scholar] [CrossRef] - Behncke, H. Optimal control of deterministic epidemics. Optim. Control Appl. Methods
**2000**, 21, 269–285. [Google Scholar] [CrossRef] - Francis, P.J. Optimal tax/subsidy combinations for the flu season. J. Econ. Dyn. Control
**2004**, 28, 2037–2054. [Google Scholar] [CrossRef] - Butler, D. Flu surveillance lacking. Nature
**2012**, 483, 520–522. [Google Scholar] [CrossRef] [PubMed][Green Version] - Taynitskiy, V.; Gubar, E.; Zhu, Q. Optimal Security Policy for Protection Against Heterogeneous Malware. In Proceedings of the International Conference on “Network Games, Control and Optimization” (NETGCOOP 2016), Avignon, France, 23–25 November 2016; pp. 199–210. [Google Scholar]
- Khouzani, M.H.R.; Sarkar, S.; Altman, E. Optimal control of epidemic evolution. In Proceedings of the 30th International Conference on Computer Communications (INFOCOM), Shanghai, China, 10–25 April 2011; pp. 1683–1691. [Google Scholar]
- Newman, L.H. What we know about Friday’s massive east coast Internet outage. Wired Magazine, 20 October 2016. [Google Scholar]
- Rohloff, K.R.; Başar, T. Deterministic and stochastic models for the detection of random constant scanning worms. ACM Trans. Model. Comput. Simul. (ACM TOMACS)
**2008**, 18, 1–24. [Google Scholar] [CrossRef] - Khouzani, M.H.R.; Sarkar, S.; Altman, E. Dispatch then stop: Optimal dissemination of security patches in mobile wireless networks. In Proceedings of the 48th IEEE Conference on Decisions and Control (CDC), Atlanta, Georgia, USA, 15–17 December 2010; pp. 2354–2359. [Google Scholar]
- Khouzani, M.H.R.; Sarkar, S.; Altman, E. Maximum damage malware attack mobile wireless networks. In Proceedings of the 29th International Conference on Computer Communications (INFOCOM), San Diego, CA, USA, 15–19 March 2010; pp. 749–757. [Google Scholar]
- Omic, J.; Orda, A.; Van Mieghem, P. Protecting against network infections: A game theoretic perspective. In Proceedings of the 28th IEEE Conference on Computer Communications (INFOCOM), Rio de Janeiro, Brazil, 19–25 April 2009. [Google Scholar]
- Zhu, Q.; Bushnell, L.; Başar, T. Game-theoretic analysis of node capture and cloning attack with multiple attackers in wireless sensor networks. In Proceedings of the 51st IEEE Conference on Decision and Control (CDC’12), Maui, HI, USA, 10–13 December 2012. [Google Scholar]
- Fu, F.; Rosenbloom, D.; Wang, L.; Nowak, M. Imitation dynamics of vaccination behaviour on social networks. Proc. R. Soc. Lond. Ser. B
**2011**, 278, 42–49. [Google Scholar] [CrossRef] [PubMed][Green Version] - Taynitskiy, V.A.; Gubar, E.A.; Zhitkova, E.M. Structure of optimal control in the model of propagation of two malicious softwares. In Proceedings of the International Conference “Stability and Control Processes” in Memory of V.I. Zubov (SCP), Saint-Petersburg, Russia, 5–9 October 2015; pp. 261–264. [Google Scholar]
- Chowell, G.; Viboud, C.; Merler, S.; Vespignani, A. Perspectives on model forecasts of the 2014–2015 Ebola epidemic in West Africa: Lessons and the way forward. BMC Med.
**2017**, 15, 42. [Google Scholar] [CrossRef] [PubMed] - Gjorgjieva, J.; Smith, K.; Chowell, G.; Sanchez, F.; Snyder, J.; Castillo-Chavez, C. The role of vaccination in the control of SARS. Math. Biosci. Eng.
**2005**, 2, 753–769. [Google Scholar] [PubMed] - Bonhoeffer, S.; May, R.M.; Shaw, G.M.; Nowak, M.A. Virus dynamics and drug therapy. Proc. Natl. Acad. Sci. USA
**1997**, 94, 6971–6976. [Google Scholar] [CrossRef] [PubMed][Green Version] - Gubar, E.; Zhu, Q. Optimal Control of Influenza Epidemic Model with Virus Mutations. In Proceedings of the 12th Biannual European Control Conference, Zurich, Switzerland, 17–19 July 2013; IEEE Control Systems Society: New York, NY, USA, 2013; pp. 3125–3130. [Google Scholar]
- Maynard, S.J. Evolution and the Theory of Games; Cambridge University Press: Cambridge, UK, 1982. [Google Scholar]
- Weibull, J. Evolutionary Game Theory; The M.I.T. Press: Cambridge, MA, USA, 1995. [Google Scholar]
- Taylor, P.D.; Jonker, L.B. Evolutionarily stable strategies and game dynamics. Math. Biosci.
**1978**, 40, 145–156. [Google Scholar] [CrossRef] - Khatri, S.; Rael, R.; Hyman, J. The Role of Network Topology on the Initial Growth Rate of Influenza Epidemic; Technical Report BU-1643-M; KeAi: Beijing, China, 2003. [Google Scholar]
- Pontryagin, L.S.; Boltyanskii, V.G.; Gamkrelidze, R.V.; Mishchenko, E.F. The Mathematical Theory of Optimal Processes; Interscience Publishers: Hoboken, NJ, USA, 1962. [Google Scholar]

**Figure 1.**Transition rule: Scheme describes the reaction to two heterogeneous viruses circulating in a population. We assumed that the epidemic process could be controlled using treatment or quarantine methods. These measures can be considered as the control parameters in the system, which are used to reduce the size of the infected population and terminate the epidemic process.

**Figure 2.**Experiment 1.

**Left**: SIR model without virus mutation and without applying control. Initial states are ${I}_{1}\left(0\right)=0.18$, ${I}_{2}\left(0\right)=0.32$, maximum values are ${I}_{1max}=0.3031$ and ${I}_{2max}=0.6742$. Epidemic peaks a reached at 11th and 10th days.

**Right**: SIR model without virus mutation with application of control. Vertical axes show the fractions of the subpopulations.

**Figure 3.**Experiment 1.

**Left**: Optimal control in SIR model without virus mutation; cost functions are convex ${h}_{i}\left({u}_{i}\right)$. Vertical axis shows the amount of applied control.

**Right**: Comparison of aggregated costs of SIR model without virus mutation (Controlled model: $J=268.35$; uncontrolled model: $J=1004.43$). Vertical axis shows the aggregated costs at time moment t in m.u.

**Figure 4.**Experiment 2.

**Left**: SIR model with virus mutation and without applying control. Maximum values are ${I}_{1max}=0.2165$, ${I}_{2max}=0.7746$. Epidemic peaks reached at the 11th and 8th days.

**Right**: SIR model with virus mutation and application of control. Functions ${h}_{i}\left({u}_{i}\right)$ were convex. Vertical axes show the fractions of the subpopulations.

**Figure 5.**Experiment 2.

**Left**: Optimal control in the SIR model with virus mutation and convex cost functions ${h}_{i}\left({u}_{i}\right)$. Vertical axis shows the amount of applied control.

**Right**: Comparison of aggregated costs of the SIR model with virus mutation (Controlled model $J=294.59$, uncontrolled model $J=1048.5$). Vertical axis shows the aggregated costs at time moment t in m.u.

**Figure 6.**Experiment 3.

**Left**: SIR model with virus mutation and application of control. Functions ${h}_{i}\left({u}_{i}\right)$ are convex. Vertical axis shows the fractions of the subpopulations.

**Right**: Optimal control in SIR model with virus mutation and convex-cost functions ${h}_{i}\left({u}_{i}\right)$. Vertical axis shows the amount of applied control at time moment t.

**Figure 7.**

**Left**: Experiment 2. Infected rates of the SIR model with virus mutations (uncontrolled case). Utility-of-occupation functions were ${b}_{1}\left({I}_{1}\right)={I}_{1}$ and ${b}_{2}\left({I}_{2}\right)={I}_{2}$. Cost functions were ${C}_{1}\left({I}_{1}\right)=5{I}_{1}$ and ${C}_{2}\left({I}_{2}\right)=2{I}_{2}$. Infectious rates were ${\delta}_{1}=0.43$ and ${\delta}_{2}=0.67$.

**Right**: Experiment 3. Infected rates of the SIR model with virus mutations (controlled case). Utility-of-occupation functions were ${b}_{1}\left({I}_{1}\right)=2{I}_{1}$ and ${b}_{2}\left({I}_{2}\right)=2{I}_{2}$. Cost functions were ${C}_{1}\left({I}_{1}\right)=5{I}_{1}$ and ${C}_{2}\left({I}_{2}\right)=3{I}_{2}$. Infectious rates were ${\delta}_{1}=0.339$ and ${\delta}_{2}=0.342$.

**Figure 8.**Experiment 4.

**Left**: SIR model with virus mutation and application of control. Functions ${h}_{i}\left({u}_{i}\right)$ are concave. Vertical axis shows the fractions of the subpopulations.

**Right**: Optimal control in SIR model with virus mutation and concave-cost functions ${h}_{i}\left({u}_{i}\right)$. Control was switched off at the seventh day for ${V}_{1}$, and at the ninth day for ${V}_{2}$. Vertical axis shows the amount of applied control at time moment t.

**Figure 9.**

**Left**: Experiment 4. Comparison of aggregated costs of the SIR model with virus mutation (Controlled model $J=289.42$, uncontrolled model $J=1035.4$). Vertical axis shows the aggregated costs at time moment t in m.u.

**Right**: Simplex of mixed strategies of the symmetric bimatrix game for modeling virus mutations. In this numerical example, the set of Nash equilibrium was found to be $\left\{\right(1,0),(0,1),(0.5,0.5\left)\right\}$.

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Gubar, E.; Taynitskiy, V.; Zhu, Q. Optimal Control of Heterogeneous Mutating Viruses. *Games* **2018**, *9*, 103.
https://doi.org/10.3390/g9040103

**AMA Style**

Gubar E, Taynitskiy V, Zhu Q. Optimal Control of Heterogeneous Mutating Viruses. *Games*. 2018; 9(4):103.
https://doi.org/10.3390/g9040103

**Chicago/Turabian Style**

Gubar, Elena, Vladislav Taynitskiy, and Quanyan Zhu. 2018. "Optimal Control of Heterogeneous Mutating Viruses" *Games* 9, no. 4: 103.
https://doi.org/10.3390/g9040103