# Game Theory of Tumor–Stroma Interactions in Multiple Myeloma: Effect of Nonlinear Benefits

^{1}

^{2}

^{3}

^{*}

*Keywords:*game theory; cancer; stroma; tumor microenvironment; nonlinear benefits

Next Article in Journal

Previous Article in Journal

Previous Article in Special Issue

Previous Article in Special Issue

Department of Electrical and Computer Engineering, University of Kashan, Kashan 8731753153, Iran

Department of Electrical and Computer Engineering, Isfahan University of Technology, Isfahan 8415683111, Iran

School of Biological Sciences, University of East Anglia, Norwich NR4 7TJ, UK

Authors to whom correspondence should be addressed.

Received: 4 April 2018 / Revised: 18 May 2018 / Accepted: 22 May 2018 / Published: 28 May 2018

(This article belongs to the Special Issue Game Theory and Cancer)

Cancer cells and stromal cells often exchange growth factors with paracrine effects that promote cell growth: a form of cooperation that can be studied by evolutionary game theory. Previous models have assumed that interactions between cells are pairwise or that the benefit of a growth factor is a linear function of its concentration. Diffusible factors, however, affect multiple cells and generally have nonlinear effects, and these differences are known to have important consequences for evolutionary dynamics. Here, we study tumor–stroma paracrine signaling using a model with multiplayer collective interactions in which growth factors have nonlinear effects. We use multiple myeloma as an example, modelling interactions between malignant plasma cells, osteoblasts, and osteoclasts. Nonlinear benefits can lead to results not observed in linear models, including internal mixed stable equilibria and cyclical dynamics. Models with linear effects, therefore, do not lead to a meaningful characterization of the dynamics of tumor–stroma interactions. To understand the dynamics and the effect of therapies it is necessary to estimate the shape of the benefit functions experimentally and parametrize models based on these functions.

Cancer cells secrete growth factors that promote proliferation by stimulating growth, resistance against apoptosis and against the immune system, or angiogenesis. Clones producing different essential growth factors can rely on each other for survival [1], and non-producer clones can free-ride on the growth factors produced by its cooperative neighbours [2]. The production of growth factors by cancer cells therefore is an example of intra-tumor cooperation and can be studied in the framework of evolutionary game theory. Concepts from game theory have been used to describe competition between cancer cells as pairwise games [3,4] with up to four strategies [5,6,7,8,9], and more recently as multiplayer public goods games to model the interactions between different clones within a tumor in well-mixed populations [10,11], in spatially structured populations that resemble monolayers in vitro [12,13,14], and to analyse the Warburg effect [15,16,17,18]; experiments with cancer cells have been used to test the theory [2].

Cancer cells also interact with non-cancerous stromal cells by exchanging diffusible factors [19,20,21,22] and these interactions with the tumor microenvironment are a fundamental part of cancer progression. While this is often referred to as “cooperation” between cancer and stroma, the secretion of growth factors by the stroma is often triggered by the cancer cells themselves, which recruit and activate stromal cells by producing diffusible cytokines; these cytokines stimulate the stroma to release growth factors that sustain tumor growth. Stromal cells, therefore, are manipulated by the cancer cells to produce growth factors that are beneficial to the tumor. A form of proper cooperation exists among the cancer cells themselves, as the cytokines they produce are a type of public good that can be exploited by non-producer cancer cells. Such a paracrine tumor–stroma signaling system, therefore, has non-trivial strategic interactions, which can also be studied in the framework of evolutionary game theory, but have received relatively less attention so far. Tumor–stroma interactions have been analysed in the context of prostate cancer [8], multiple myeloma [5,23], and MMP–TIMP (matrix metalloproteinases and tissue inhibitors of metalloproteinases) interactions [24].

One problem with early models of tumor–stroma interactions, as with early models of intra-tumor cooperation, is that they assume pairwise interactions between cells. Cells, however, compete with more than one neighbor, and multiple cells contribute to the overall amount of growth factors available to a cell. A model with collective interactions [23] has been shown to lead to different results compared to a model [5] with pairwise interactions, with implications for disease progression and treatment (in particular, this new model suggests that reducing the number of malignant plasma cells below a critical threshold can lead to their extinction and thus to restore a healthy balance between osteoclast and osteoblast).

Even this approach, however, made an unrealistic assumption: that the effect of the growth factors exchanged by the cells is a linear function of its concentration. While linear effects are more amenable to analytical study, it is known that most growth factors have a nonlinear, often sigmoid, effect on cell proliferation, and that nonlinear effects can lead to very different dynamics [25,26]. We decided, therefore, to analyse tumor–stroma interactions using a multiplayer public goods game with nonlinear benefits, sacrificing analytical tractability for the sake of realism. Our goal is to understand whether nonlinear effects lead to qualitatively different results in the dynamics and equilibria in the study of tumor–stroma interactions, in particular in multiple myeloma (a type of cancer of plasma cells [27,28]).

We use multiple myeloma as a case study, continuing a previous extension to multiplayer linear games [23] of an earlier model with pairwise interactions [5]. In multiple myeloma, interleukin-6 (IL6), which promotes tumor proliferation, is produced not only by malignant plasma cells, but also by bone marrow stromal cells and osteoblasts; the stroma, however, is induced to produce IL6 by cytokines secreted by the multiple myeloma cells themselves [29,30], including FGF-beta [31], IL-1 [32], TNF-alpha [33], TGF-1 [34], and VEGF [35]. More in general, in multiple myeloma three types of cells contribute to bone resorption and bone formation by exchanging diffusible factors: malignant plasma cells (multiple myeloma, MM), osteoclasts (OC), and osteoblasts (OB). MM cells produce cytokines that activate OC and inhibit OB, altering the normal OB–OC equilibrium of healthy cells (and inducing, among other symptoms, bone fracture typical of multiple myeloma). OC and OB also secrete growth factors that affect each other and MM cells [36,37]. Our model is consistent with assumptions used previously in a model with linear benefits [23]—which was itself an extension with collective interactions of a pairwise game [5]—with the difference that here we use nonlinear benefit functions.

The most fundamental result of previous models is described in Figure 1: OB and OC cells are at a mixed equilibrium, balancing each other until MM cells are introduced in the population: if enough MM cells are introduced, the population evolves to a stable equilibrium on the MM–OC edge, which explains the potential for bone fractures observed in multiple myeloma. This result is obtained for the special case s_{i,j}→0 (linear benefits) in our model, and is consistent with previous models of linear benefits [23]. Observe, however (Figure 2), that if benefits are nonlinear this is not necessarily the case: the OB–OC equilibrium can persist irrespective of the number of MM cells introduced in the population or disappear entirely, depending on the shape of the benefit functions (in Figure 2 the position of the inflection point). While a model with linear benefits predicts a bistable system, nonlinear benefits can have both or only one of the two stable equilibria observed in linear models. Further examples are shown in Figure 3 and Figure 4, where additional mixed equilibria, not observed with linear benefits, arise with nonlinear benefits; in other cases, instead, the dynamics are similar (Figure 5). Unfortunately, given the huge number of combinations of parameters in the nonlinear model, any useful generalization is impossible. To understand and predict the dynamics of the system we need to know the shape of the benefit functions.

Nonlinear benefits introduce a new type of equilibrium, never observed in linear models, in which all three types coexist (Figure 6). Coexistence of the three types can also be reached with cyclical oscillations that converge to a stable equilibrium (Figure 7) and even persist as a limit cycle (Figure 8): two types of dynamics that are never observed in linear models (and, more in general, in public goods games with constant group size).

Perturbations of the equilibria can reveal the effect of therapies against multiple myeloma and help understand the evolution of resistance to therapies. Figure 9 shows an example. At the OB–OC equilibrium, a mutant MM clone invades the population and leads to a MM–OC equilibrium: the typical fundamental result of current models (see Figure 1). A therapy that reduces the fraction of MM cells (such as chemotherapy that targets MM cells) immediately reduces the average fitness of the population; MM cells, however, eventually increase in frequency again and the population moves back to the previous MM–OC equilibrium. A therapy that increases the position of the inflection point (see Figure 2), for instance targeting the growth factors produced by the stroma and by the tumor (the inflection point increases when some of the growth factors are removed or made unavailable by the therapy because additional growth factors must be secreted—hence more producer cells are necessary—in order to achieve the same pre-therapy benefit level), instead, could lead to a spontaneous extinction of the MM types and a reduction of fitness; that is, a reduction in the speed of tumor growth. Understanding the effects of the two approaches quantitatively requires an estimation of the shape of the benefit functions.

In summary, in addition to the pure equilibria and the two-type mixed equilibria observed in linear models [23], two additional results arise when benefits are nonlinear benefits: a mixed stable equilibrium with three types; and cyclical dynamics of the three types. This reconciles evolutionary game theory models with observations in real tumors (where the three types of cells can clearly coexist — something that previous linear models could not account for).

Our results, however, also show that it is impossible to generalize the results and predict the dynamics of the system without knowing exactly the shape of the benefit functions; small changes can lead to very different dynamics. In order to understand the dynamics and to be able to predict the effect of therapies, the shape of these benefit functions must be estimated from experimental data.

Even though, in the absence of measures of the shape of the benefit functions, we cannot yet provide an exact description of the dynamics and suggest how to calibrate therapies, our results suggest that a therapy will be more effective if, in addition, or even in alternative, to reducing the amount of MM cells, it increases the value of the inflection point of the benefit functions, which could be achieved, for example, using monoclonal antibodies against the growth factors produced by the tumor and the stroma (still an experimental approach [38,39]). Monoclonal antibodies increase the value of the inflection point because they make some of the secreted growth factors unavailable to the tumor; hence, further growth factors must be produced—hence more producer cells are necessary—in order to achieve the same pre-therapy benefit level. Reducing the amount of MM cells, instead, does not affect the shape of the benefit function. In other words, therapies that change the parameters of the game may be more effective, and lead to a stable outcome, than therapies that change the frequency of the players.

More in general, besides multiple myeloma, our results show that models with linear benefits (not to mention models with pairwise interactions) do not provide a reliable characterization of the dynamics of tumor–stroma interactions, and that unexpected types of dynamics, such as cyclical dynamics, can arise due to nonlinearities.

Let us call x_{OC}, x_{OB}, and x_{MM} the frequencies of three types of cells, respectively OC (osteoclasts), OB (osteoblasts), and MM (malignant plasma cells), in the population (x_{OC} + x_{OB} + x_{MM} = 1). N is the number of cells within the diffusion range of the growth factors (hence, equal to group size), which we assume is the same for all types and all growth factors. Fitness is calculated by considering the payoffs obtained in the randomly formed groups weighted by the probability that such groups occur. In a well-mixed population, the probability that a group contains n_{OC}, n_{OB}, and N − n_{OC} − n_{OB} individuals (excluding the focal cell itself) of type OC, OB, and MM, respectively, is given by

$$\left(\begin{array}{c}N-1\\ {n}_{OC},{n}_{OB}\end{array}\right){x}_{OC}{}^{{n}_{OC}}\xb7{x}_{OB}{}^{{n}_{OB}}\xb7{x}_{MM}{}^{N-{n}_{OB}-{n}_{OC}-1}.$$

The payoffs for OC, OB, and MM are, respectively:
where c_{i} is the cost of producing growth factors by type i and b_{i,j} is a function that indicates the effect on type j of the growth factors produced by type i. Note that each player’s strategy has also an effect on itself. The fitness of type i is given by

V_{OC} = b_{OC,OC}(n_{OC} + 1) + b_{OB,OC}(n_{OB}) + b_{MM,OC}(N − 1 − n_{OC} − n_{OB}) − c_{OC}

V_{OB} = b_{OC,OB}(n_{OC}) + b_{OB,OB}(n_{OB} + 1) + b_{MM,OB}(N − 1 − n_{OC} − n_{OB}) − c_{OB}

V_{MM} = b_{OC,MM}(n_{OC}) + b_{OB,MM}(n_{OB}) + b_{MM,MM}(N − n_{OC} − n_{OB}) − c_{MM}

$${W}_{i}={\displaystyle \sum}_{{n}_{OC}=1}^{N-1}{\displaystyle \sum}_{{n}_{OB}=0}^{N-1-{n}_{OC}}\left(\begin{array}{c}N-1\\ {n}_{OC},{n}_{OB}\end{array}\right){x}_{OC}{}^{{n}_{OC}}\xb7{x}_{OB}{}^{{n}_{OB}}\xb7{x}_{MM}{}^{N-{n}_{OB}-{n}_{OC}-1}\xb7{V}_{i}.$$

The frequencies change based on the replicator dynamics:
where W * = x_{OC}W_{OC} + x_{OB}W_{OB} + x_{MM}W_{MM}. To introduce nonlinearities, we assume a benefit function defined by
the normalized version of the logistic function
where n_{i} is the number of cells of type i; the parameters define the effect of the growth factors produced by type i on type j: B_{i,j} defines the maximum benefit; h_{i,j} defines the position of the inflection point: h_{i,j}→1 gives increasing returns and h_{i,j}→0 diminishing returns; s_{i,j} defines the steepness of the function at the inflection point (s_{i,j}→∞ models a threshold public goods game; s_{i,j}→0 models a linear benefit (Figure 10); and the normalization in (9) in prevents the logistic function (10) from becoming constant for s_{i,j}→0).

x_{OC}′ = x_{OC}(W_{OC} − W *)

x_{OB}′ = x_{OB}(W_{OB} − W *)

x_{MM}′ = x_{MM}(W_{MM} − W *)

$${b}_{i,j}\left({n}_{i}\right)=\frac{\left[{l}_{i,j}\left({n}_{i}\right)-{l}_{i,j}\left(0\right)\right]}{\left[{l}_{i,j}\left(N\right)-{l}_{i,j}\left(0\right)\right]}$$

$${l}_{i,j}\left({n}_{i}\right)=\frac{{B}_{i,j}}{1+{e}^{{s}_{i,j}\left({h}_{i,j}-{n}_{i}/N\right)}}$$

J.S.S., M.H.M. and M.A. conceived and designed the analysis; J.S.S. and M.A. performed the analysis with contributions from M.H.M.; M.A. wrote the paper with contributions from J.S.S.

This project has received funding from the Marie Curie International Outgoing Fellowship within the 7th European Community Framework Program under grant agreement No. 627816-dunharrow.

The authors declare no conflict of interest. The founding sponsors had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, and in the decision to publish the results.

- Axelrod, R.; Axelrod, D.E.; Pienta, K.J. Evolution of cooperation among tumor cells. Proc. Natl. Acad. Sci. USA
**2006**, 103, 13474–13479. [Google Scholar] [CrossRef] [PubMed] - Archetti, M.; Ferraro, D.A.; Christofori, G. Heterogeneity for IGF-II production maintained by public goods dynamics in neuroendocrine pancreatic cancer. Proc. Natl. Acad. Sci. USA
**2015**, 112, 1833–1838. [Google Scholar] [CrossRef] [PubMed][Green Version] - Tomlinson, I.P. Game-theory models of interactions between tumour cells. Eur. J. Cancer
**1997**, 33, 1495–1500. [Google Scholar] [CrossRef] - Tomlinson, I.P.; Bodmer, W.F. Modelling consequences of interactions between tumour cells. Br. J. Cancer
**1997**, 75, 157–160. [Google Scholar] [CrossRef] [PubMed] - Dingli, D.; Chalub, F.; Santos, F.; Van Segbroeck, A.; Pacheco, J. Cancer phenotype as the outcome of an evolutionary game between normal and malignant cells. Br. J. Cancer
**2009**, 101, 1130–1136. [Google Scholar] [CrossRef] [PubMed] - Basanta, D.; Simon, M.; Hatzikirou, H.; Deutsch, A. Evolutionary game theory elucidates the role of glycolysis in glioma progression and invasion. Cell Prolif.
**2008**, 41, 980–987. [Google Scholar] [CrossRef] [PubMed] - Basanta, D.; Deutsch, A. A game theoretical perspective on the somatic evolution of cancer. In Selected Topics on Cancer Modelling: Genesis Evolution Immune Competition Therapy; Bellomo, N., Chaplain, M., De Angelis, E., Eds.; Birkhauser Boston: Cambridge, MA, USA, 2008. [Google Scholar]
- Basanta, D.; Scott, J.G.; Fishman, M.N.; Ayala, G.; Hayward, S.W.; Anderson, A.R. Investigating prostate cancer tumour-stroma interactions: Clinical and biological insights from an evolutionary game. Br. J. Cancer
**2012**, 106, 174–181. [Google Scholar] [CrossRef] [PubMed][Green Version] - Gerstung, M.; Nakhoul, H.; Beerenwinkel, N. Evolutionary games with affine fitness functions: Applications to cancer. Dyn. Games Appl.
**2011**, 1, 370–385. [Google Scholar] [CrossRef] - Archetti, M. Evolutionary game theory of growth factor production: Implications for tumor heterogeneity and resistance to therapies. Br. J. Cancer
**2013**, 109, 1056–1062. [Google Scholar] [CrossRef] [PubMed][Green Version] - Archetti, M. Evolutionarily stable anti-cancer therapies by autologous cell defection. Evol. Med. Public Heatlh
**2013**, 1, 161–172. [Google Scholar] [CrossRef] [PubMed] - Archetti, M. Dynamics of growth factor production in monolayers of cancer cells. Evol. Appl.
**2013**, 6, 1146–1159. [Google Scholar] [CrossRef] [PubMed][Green Version] - Archetti, M. Stable heterogeneity for the production of diffusible factors in cell populations. PLoS ONE
**2014**, 9, e108526. [Google Scholar] [CrossRef] [PubMed][Green Version] - Archetti, M. Cooperation among cancer cells as public goods games on Voronoi networks. J. Theor. Biol.
**2016**, 396, 191–203. [Google Scholar] [CrossRef] [PubMed][Green Version] - Basanta, D.; Hatzikirou, H.; Deutsch, A. Studying the emergence of invasiveness in tumours using game theory. Eur. Phys. J.
**2008**, 63, 393–397. [Google Scholar] [CrossRef] - Basanta, D.; Scott, J.G.; Rockne, R.; Swanson, K.R.; Anderson, A.R. The role of IDH1 mutated tumour cells in secondary glioblastomas: An evolutionary game theoretical view. Phys. Biol.
**2011**, 8, 15016. [Google Scholar] [CrossRef] [PubMed] - Archetti, M. Evolutionary dynamics of the Warburg effect: Glycolysis as a collective action problem among cancer cells. J. Theor. Biol.
**2014**, 341, 1–8. [Google Scholar] [CrossRef] [PubMed][Green Version] - Archetti, M. Heterogeneity and proliferation of invasive cancer subclones in game theory models of the Warburg effect. Cell Prolif.
**2015**, 48, 259–269. [Google Scholar] [CrossRef] [PubMed][Green Version] - Li, H.; Fan, X.; Houghton, J. Tumor microenvironment: The role of the tumor stroma in cancer. J. Cell. Biochem.
**2007**, 101, 805–815. [Google Scholar] [CrossRef] [PubMed] - Pietras, K.; Ostman, A. Hallmarks of cancer: Interactions with the tumor stroma. Exp. Cell Res.
**2010**, 316, 1324–1331. [Google Scholar] [CrossRef] [PubMed] - Cirri, P.; Chiarugi, P. Cancer associated fibroblasts: The dark side of the coin. Am. J. Cancer Res.
**2011**, 1, 482–497. [Google Scholar] [PubMed] - Hanahan, D.; Coussens, L.M. Accessories to the crime: Functions of cells recruited to the tumor microenvironment. Cancer Cell
**2012**, 21, 309–322. [Google Scholar] [CrossRef] [PubMed] - Sartakhti, J.S.; Manshaei, M.H.; Bateni, S.; Archetti, M. Evolutionary Dynamics of Tumor-Stroma Interactions in Multiple Myeloma. PLoS ONE
**2016**, 11, e0168856. [Google Scholar] [CrossRef] [PubMed] - Sartakhti, J.S.; Manshaei, M.H.; Sadeghi, M. MMP–TIMP interactions in cancer invasion: An evolutionary game- theoretical framework. J. Theor. Biol.
**2017**, 412, 17–26. [Google Scholar] [CrossRef] [PubMed] - Archetti, M.; Scheuring, I. Review: Evolution of cooperation in one-shot social dilemmas without assortment. J. Theor. Biol.
**2012**, 299, 9–20. [Google Scholar] [CrossRef] [PubMed] - Archetti, M. How to analyze models of nonlinear public goods. Games
**2018**, 9, 17. [Google Scholar] [CrossRef] - Roodman, G. Pathogenesis of myeloma bone disease. Leukemia
**2009**, 23, 435–441. [Google Scholar] [CrossRef] [PubMed] - Yaccoby, S. Advances in the understanding of myeloma bone disease and tumour growth. Br. J. Haematol.
**2010**, 149, 311–321. [Google Scholar] [CrossRef] [PubMed] - Manier, S.; Sacco, A.; Leleu, X.; Ghobrial, I.M.; Roccaro, A.M. Bone marrow microenvironment in multiple myeloma progression. J. Biomed. Biotechnol.
**2012**, 2012, 157496. [Google Scholar] [CrossRef] [PubMed] - Shalaby, M.R.; Waage, A.; Espevik, T. Cytokine regulation of interleukin-6 production by human endothelial cells. Cell. Immunol.
**1989**, 121, 372–382. [Google Scholar] [CrossRef] - Bisping, G.; Leo, R.; Wenning, D.; Dankbar, B.; Padró, T.; Kropff, M.; Scheffold, C.; Kröger, M.; Mesters, R.M.; Berdel, W.E.; et al. Paracrine interactions of basic fibroblast growth factor and interleukin-6 in multiple myeloma. Blood
**2003**, 101, 2775–2783. [Google Scholar] [CrossRef] [PubMed] - Dinarello, C.A. Interleukin-1 in the pathogenesis and treatment of inflammatory diseases. Blood
**2011**, 117, 3720–3732. [Google Scholar] [CrossRef] [PubMed] - Carter, A.; Merchav, S.; Silvian-Draxler, I.; Tatarsky, I. The role of interleukin-1 and tumor necrosis factor-alpha in human multiple myeloma. Br. J. Haematol.
**1990**, 74, 424–431. [Google Scholar] [CrossRef] [PubMed] - Urashima, M.; Ogata, A.; Chauhan, D.; Hatziyanni, M.; Vidriales, M.B.; Dedera, D.A.; Schlossman, R.L.; Anderson, K.C. Transforming growth factor-beta1: Differential effects on multiple myeloma versus normal B cells. Blood
**1996**, 87, 1928–1938. [Google Scholar] [PubMed] - Dankbar, B.; Padró, T.; Leo, R.; Feldmann, B.; Kropff, M.; Mesters, R.M.; Serve, H.; Berdel, W.E.; Kienast, J. Vascular endothelial growth factor and interleukin-6 in paracrine tumor-stromal cell interactions in multiple myeloma. Blood
**2000**, 95, 2630–2636. [Google Scholar] [PubMed] - Ehrlich, L.A.; Chung, H.Y.; Ghobrial, I.; Choi, S.J.; Morandi, F.; Colla, S.; Rizzoli, V.; Roodman, G.D.; Giuliani, N. IL-3 is a potential inhibitor of osteoblast differentiation in multiple myeloma. Blood
**2005**, 106, 1407–1414. [Google Scholar] [CrossRef] [PubMed] - Lee, J.W.; Chung, H.Y.; Ehrlich, L.A.; Jelinek, D.F.; Callander, N.S.; Roodman, G.D.; Choi, S.J. IL-3 expression by myeloma cells increases both osteoclast formation and growth of myeloma cells. Blood
**2004**, 103, 2308–2315. [Google Scholar] [CrossRef] [PubMed] - Trikha, M.; Corringham, R.; Klein, B.; Rossi, J.F. Targeted anti-interleukin-6 monoclonal antibody therapy for cancer: A review of the rationale and clinical evidence. Clin. Cancer Res.
**2003**, 9, 4653–4665. [Google Scholar] [PubMed] - Lonial, S.; Durie, B.; Palumbo, A.; San-Miguel, J. Monoclonal antibodies in the treatment of multiple myeloma: Current status and future perspectives. Leukemia
**2016**, 30, 526–535. [Google Scholar] [CrossRef] [PubMed]

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).