Space Debris Removal: a Game Theoretic Analysis

We analyse active space debris removal efforts from a strategic, game-theoretical perspective. Space debris is non-manoeuvrable, human-made objects orbiting Earth, which pose a significant threat to operational spacecraft. Active debris removal missions have been considered and investigated by different space agencies with the goal to protect valuable assets present in strategic orbital environments. An active debris removal mission is costly, but has a positive effect for all satellites in the same orbital band. This leads to a dilemma: each agency is faced with the choice between the individually costly action of debris removal, which has a positive impact on all players; or wait and hope that others jump in and do the 'dirty' work. The risk of the latter action is that, if everyone waits, the joint outcome will be catastrophic, leading to what in game theory is referred to as the 'tragedy of the commons'. We introduce and thoroughly analyse this dilemma using empirical game theory and a space debris simulator. We consider two-and three-player settings, investigate the strategic properties and equilibria of the game and find that the cost/benefit ratio of debris removal strongly affects the game dynamics.


Introduction
In this work we apply empirical game theoretic methods to study the strategic real-world problem of space debris removal.The complexity of the space debris environment typically prohibits direct game theoretic analysis.However, by defining appropriate heuristic strategies and using a suitable simulator we can estimate heuristic payoff functions to model the strategic dilemma as a game.This then enables us to apply (evolutionary) game theoretic analysis.
Since the late 1950s space agencies have launched many objects into Earth orbits with low or no incentive to remove them after their life span.Due to this, there are now many inactive objects orbiting Earth, which can pose a risk to active spacecraft.By far, the highest spatial density of such objects is in the Low Earth Orbit (LEO) environment, defined as the region of space around Earth with an altitude of 160 km to 2,000 km.
The density of objects in LEO will most likely increase due to new launches, on-orbit explosions, and object collisions.NORAD tracks and catalogues objects in orbit, currently listing around 15,000 objects of 10cm 2     and larger. 1 However, it is estimated that the true number of objects is several orders of magnitude larger, with estimates of over 100,000 pieces of untracked debris of sizes 1-10cm 2 [1].At orbital speeds of approximately 7.5 km/s such small pieces can cause considerable damage to active satellites.
In recent years there have been several incidents producing a high number of debris.Two of them have been especially severe: (i) a 2007 Chinese anti-satellite missile test producing more than 1,200 catalogued pieces of debris, and an estimated 35,000 pieces of size 1cm and larger, resulting in the most severe orbital debris cloud in history [2]; (ii) the collision of the Iridium-33 and Kosmos-2251 satellites in 2009, which was the first accidental hyper-velocity collision of two intact spacecraft [3].More than 823 debris objects were catalogued forming two debris clouds in LEO.This incident introduced a high risk of potential collisions to many active objects in LEO.For example, the International Space Station (ISS) had to perform a manoeuvre in March 2011 to avoid a piece of debris from the 2009 Iridium-Kosmos collision [4].
These two incidents show that space debris is a serious problem with potentially disastrous consequences.
As a result, space agencies are now investigating ways in which end-of-life satellites can be safely de-orbited.
However, measures that apply only to newly launched spacecraft may not be sufficient to prevent an exponential build-up of debris.Therefore, active space debris removal becomes relevant.An active debris removal mission has a positive effect (or risk reduction) for all satellites in the same orbital band.This leads to a dilemma: each agency has an incentive to delay its actions and wait for others to respond.
We model this scenario as a non-cooperative game between self-interested agents in which the agents are space agencies.Using a high-fidelity simulator we estimate payoffs to agents for different combinations of actions taken, and analyse the resulting game in terms of best-response dynamics and (Nash) equilibria.
Contrary to the urgency of the space debris dilemma there has not been much attention to this problem in scientific circles.To the best of our knowledge we are the first to consider this dilemma in the context of multi-agent strategic decision making using empirical game theoretic techniques.This paper proceeds as follows.We firstly position our study in the context of related work.Next, we present our space debris simulator which includes a collision model, a break-up model, and an orbital propagator.We then outline our game theoretic methodology.Using our simulator we analyze the potential impact of several removal strategies on the orbital environment, and present a game theoretic analysis.

Related Work
Our study can be placed in the context of two different areas of related work.Firstly, from a simulation modelling perspective various attempts have been made to accurately predict the evolution of space debris and the resulting risk of collisions for active space craft.Secondly, from a game theoretic perspective, researchers have utilized similar methods to study related problems of environmental pollution, and the shared exploitation of scarce resources [5].
One of the earliest analyses of the projected evolution of space debris was done by Donald J. Kessler in 1978 [6,7].This study led to the definition of the "Kessler Syndrome", a particular scenario where the density of objects in LEO becomes high enough to cause a cascade of collisions, each producing new debris and eventually saturating the environment, rendering future space missions virtually impossible.Follow-up studies have been published mostly by scientists from NASA, most well-known being the work of J.-C.Liou and Nicholas L.
Johnson [8][9][10], in which the authors consider active removal strategies to mitigate the space debris problem.[8] present a sensitivity analysis of object removal strategies.They propose removing 5, 10, or 20 objects per year, which can be seen as a single-agent approach.The authors compare these mitigation strategies with baselines "business as usual" or "no new launches" and show the effectiveness of object removals.

Liou and Johnson
The objects to be removed are chosen according to their mass and collision probability.We base our study on Liou and Johnson's approach, but in contrast consider a multi-agent scenario in which different space agencies independently choose their removal strategy.In our model we implement individualised object removal criteria based on potential risk to important assets of each of the agents.
The space debris removal dilemma is in many ways similar to other environmental clean-up efforts that have been studied using game theoretic tools in the past.For example, Tahvonen models carbon dioxide abatement as a differential game, taking into account both abatement costs and environmental damage [5].
More complex models have been studied as well, including for example the ability to negotiate emission contracts [11].Another related model is the Great Fish War of Levhari and Mirman [12].Although not the same as environmental clean-up, this scenario deals with shared use of a scarce common resource, which potentially leads to the same dilemma in game theoretic terms, known as the "tragedy of the commons" [13].However, each of these studies has focused solely on a (simplified) mathematical model of the underlying system.In contrast, we use a complex simulator to obtain an approximate model using empirical game theoretic methods.
Analysis of complex strategic interactions using game theoretic tools is often hindered by the large action-spaces available to the agents in such scenarios.For example, in the space debris removal dilemma, each possible piece of debris to remove is potentially an action.Additionally, it is often impossible to define payoffs to all (combinations of) actions in advance.This has led recently to the advent of empirical game theory [14,15].The main idea is to limit the strategy space of each agent by introducing high level generic profiles, or meta-strategies, that capture the main aspects of the interaction.Then, the payoff table for this reduced strategy space can be estimated empirically, either by analysing data from a real system, or by simulating a model of the system.Standard methods and techniques from (evolutionary) game theory can then be applied to the estimated payoff table, e.g. to find approximate equilibria [16].
Such empirical game theoretic analysis has proven valuable in getting insights into various complex real-world domains, such as automated trading [17], auction mechanism design [18], the game of Poker [19], collision avoidance in multi-robot systems [20], adaptive cyber-defence strategies [21], and large-scale bargaining [22].In this work we follow a similar approach but focus on the domain of space debris removal.

Space Debris Simulation Model
Our simulator is built on top of the Python scientific library PyKep [23], which provides basic tools for astrodynamics research such as satellite orbit propagators.To simulate the future development of space debris in Low Earth Orbit (LEO) we develop several sub-modules, including a collision model and a break-up model, which we describe below.
The input data to our model comes from two satellite catalogues/databases: (i) the SATCAT2 database containing descriptions of all objects on earth orbits that have ever been documented, and (ii) the TLE (two-line element set) 3 database providing up-to-date information on all active (not decayed) objects on earth orbits, including the orbital elements which uniquely identify an object's orbit, and which are used for orbit propagation.

Collision model
To evaluate probability of collision between objects we follow the Cube approach [24].The Cube approach samples uniformly in time rather than space and is thus compatible with any orbital evolution simulation as it does not impose assumptions on the orbital geometry.This is particularly important in LEO, where orbital progression is significant in the considered time frame.We use the SGP4 [25] orbital propagator to calculate the evolution of the ephemeris (i.e., position and velocity) of an orbiting object given its TLE description.Ephemerides of all objects are calculated at regular time intervals.Space is then partitioned by a regular 3D-lattice and for any pair i, j of objects that fall into the same volume, the collision probability is evaluated as follows: where s i = s j are the spatial densities of object i and j in the cube, σ = π(r i + r j ) 2 is the cross-sectional collision area, V rel is the collision (relative) velocity of the two objects, and U is the volume of the cube.
For each pair, a pseudo-random number x is generated from a uniform distribution over the interval [0, 1); if p i,j > x, a collision event is triggered.

Breakup model
We use NASA's standard breakup model [26] to generate the population of fragments resulting from a collision event.The NASA/JSC breakup model represents a widely accepted understanding of the fragmentation process of in-orbit collisions and explosions based on multiple ground-tests and radar observations of past events.The model provides distributions for size, mass and ejection velocity of the fragment population parametrised by total mass and collision velocity of the parent objects.The number of fragments larger than a characteristic length-scale follows a power-law, the area-to-mass ratio follows a multivariate normal distributions, and the ejection velocity is sampled from a log-normal distribution.For details we refer the reader to the original paper [26] as well as the description of the model in [27].For each sampled fragment, we create a new TLE entry and add it to the population of objects being propagated by SGP4.
Although the breakup model covers also explosions as well as non-catastrophic collisions, we only consider catastrophic collisions (i.e., leading to complete disintegration) in this work.

Repeating launch sequence
To simulate future launches of new satellites we assume a "business as usual" scenario based on past data.
One can assume that future launches will differ to past launches by many factors, e.g. the mission purpose, the number of launches, their success rate and technology level, and the satellite's ability to decay in given time frame, etc.However, as a first step simplification we base our model on repeating a 10 year window from 2005 to 2015.From the SATCAT catalogue we filter all space objects introduced in this time window, excluding debris.For all these objects (both decayed and not decayed) we store the TLE data (for the decayed objects we store the last TLE recorded).We then repeat this 10 year launch sequence and introduce each month all the objects that were launched exactly (a multiple of) ten years ago.We keep all the orbital elements the same, except for the inclination, which we sample randomly from the distribution of inclinations of all objects in the repeated sequence.This way, newly launched satellites will have slightly different orbits, as can be expected.
Figure 1 shows the distribution of orbital inclinations.We can see that the highest number of objects has an inclination of around 95°.
We assume an increase over time in the number of launches due to technological development and changing needs.In addition to the 10 year repeating sequence, we increase the number of launches by 0.5% each year, by randomly sampling from the 10 year sequence.Note that each launch has a small probability of failing, due to the instability of some orbits resulting from the randomly sampled orbital inclination.Thus, some objects decay very soon after being launched, which can be thought of as e.g.unsuccessful launches, break-up during first stage, etc.

Validation
In order to validate our model we simulate the evolution of the total number of debris and compute the resulting spatial density in different altitude ranges for the next 150 years, and compare our predictions to those of Liou and Johnson [8].In Figure 2

Game Methodology
Game theory models strategic interactions in the form of games.The most basic type are normal-form games, in which n players each have a set of actions to choose from.Without prior communication, each player selects an action, and the combination of actions by all player (the joint action) determines the payoff to each.
Players are assumed to be rational, i.e. they will always want to play a best response (in terms of individual payoff) to the joint action of all remaining players.A joint action in which each action is a best response to all other actions is called a Nash equilibrium (NE). 4    The space debris removal dilemma can be modelled as a game in which the players are space agencies, their actions are debris removal strategies, and the payoffs are derived from removal costs as well as collision risks.The strategic interaction results from the fact that debris removal by one agency may affect the collision risks to others as well.
Players -Our main analysis focuses on a two-player game: (1) the United States (US) represented by The National Aeronautics and Space Administration (NASA); and (2) the European Union (EU) represented by European Space Agency (ESA) and all EU member states.Additionally we consider adding China (CN) as a third player.The fourth major space agency, Russia (Roscosmos), is not included in our game, but does play a role in the simulator in terms of repeating past launch sequences.
Important Assets -For each player we store a list of important assets.Important assets are all active objects owned by that player which are not debris, and which have been launched in the last 10 years (we assume a 10 year life span of satellites).The list of important assets is continuously updated during the simulation.Figure 3 shows an example of the development of important assets for each of the agencies.One can observe that a small difference in the number of important assets at the beginning causes a big difference at the end of the projection due to the repetition of launches from the same sequence, combined with the 0.5% yearly increase.
Actions -The players' actions are defined by the number of debris objects that will be removed each year.In our game, the players can remove 0, 1, or 2 risky objects every 2 years.We assume self-interested agents, meaning that each player first removes objects which directly threaten their important assets, and then removes objects which may potentially collide in general.The reasoning for the latter is that debris resulting from any collision may pose a potential future risk to a player's important assets.Therefore, removing any risky debris object (not only those that threaten important assets) may benefit all the players to some extent.Each agency decides on their strategy at the beginning of the game, and does not change it later.Thus, we model this as a one-shot normal form game.Moreover, we assume that removal actions are always successful.
Risks and Payoffs -During simulation we keep track of the risk of collision (p ij in Equation 1) to each player's important assets.The total sum of these risks over the full time horizon is taken as the overall risk r to each player under the simulated scenario.Subsequently, we derive payoffs from the costs of losing important assets, and the costs of object removal.These payoffs are computed by multiplying the player's overall risk r by the associated cost of loosing an asset C l , and adding the cost of removing one object each year C r multiplied by the number of removed objects and the time horizon T. Specifically, Table 1 lists the payoff functions that are used given the player's strategy.Note that we assume that costs linearly increase with the risk, which intuitively makes sense from a purely monetary perspective, where each lost asset costs the same amount to replace.However, other cost function could be constructed to incorporate for example loss aversion.Finally, since the term r • C l is common to each strategy, we can assume without loss of generality that C l = 1 (in arbitrary units) and focus only on the ratio C r/C l = C r in the remainder.
Table 1.Payoff functions for the different strategies.

Strategy Payoff function
Remove 2 −(r

Simulation Results and Projections
We use our simulator to project the evolution of debris and collision risks with a time horizon of 150 years, i.e. the period 2016-2165, while repeating the launch history of 2006-2015 with a 0.5% yearly increase.We first focus on a 2-player 3-action game, with players US and EU, and the actions to remove 0, 1, or 2 objects every two years as described above.For each combination of actions we average over 160 Monte-Carlo runs to account for randomness in the collision and break-up modules.Error margins are omitted in the figures for readability, but are reported below in Table 2.

Debris evolution
Figure 4 shows the evolution of objects in LEO for different combinations of strategies taken by the US and the EU.We observe an exponential growth trend without mitigation, in line with findings previously reported by Liou and Johnson [8].One can clearly see that removing risky objects has a positive effect as it leads to a much lower total number of objects in LEO.Note that when both players remove 2 objects every two years, this means that in total 300 objects are actively removed over the course of 150 years.In contrast, this leads to a reduction in total number of objects in LEO of over 60,000, due to a strong decrease in number of collisions and resulting debris.Also note that the total number of active satellites in each scenario is less than 1,500 (see 3), a small fraction of the total number of objects.

Risk evolution
We now look at the potential risks to the agencies' important assets, as described in Section 4, that result from the debris evolution in LEO. Figure 5a shows the evolution of the expected overall risk to the US.One can observe that if the EU removes objects it helps US as well.However, objects removed by the US have greater impact on their overall risk, which is explained by the fact that each agency removes firstly the objects that

Chinese risk of collision
Chinese risk evolution threaten their important assets directly, and only then they remove objects that pose a risk in general.Therefore, we can see that the joint action {US 1, EU 0} helps the US substantially more than {US 0, EU 2}, even though in the latter case more objects are removed in total.
In Figure 5b we can see the expected overall risk of losing important assets for the EU.We observe similar trends as in the previous figure: the EU is better off when they remove objects that directly threaten their assets.
However, even when the EU removes nothing but the US does, the EU risks decrease.This means that, as expected, there is in fact a dilemma as each agency benefits from mitigation efforts of others, without having to pay a cost (free-riding).
The free-riding effect can be observed as well when looking at the risk evolution for both China and Russia.
Even though these agencies did not take part in mitigation in our scenario (essentially playing the fixed action of remove 0), they still benefit from a reduced risk to their important assets.Figure 6a shows this for the case of China, and similar results are observed in Figure 6b for Russia.One can notice an abrupt increase in the Chinese risks around the year 2080, which is eliminated when more objects are removed in total.The joint efforts of the US and the EU in fact remove the one object which causes this high risk to the Chinese important assets.

Game Theoretic Analysis
We now turn to the game theoretic analysis of the space debris removal dilemma.First, we use the results reported in Section 5 to derive a normal-form game representation of the two-player scenario.We then thoroughly analyse this game.Finally, we give an example of a three-player game.

Two player game
Using the simulation results of Section 5, we can now construct a normal-form game representation of the two-player space debris removal dilemma.First, we construct a risk matrix, showing the overall risks to the two players (US and EU) for each combination of actions.Then, we use this risk table together with the cost functions defined in Table 1 to derive payoff matrices for different removal costs C r , and analyse all possible Nash equilibria outcomes.
Table 2 shows the average cumulative risks accrued by both players, taken from the results in Figures 5a     and 5b (time horizon 150 years, 160 runs for each scenario).A cumulative risk of 0.36385 for the EU in the no removal case can be interpreted as an expected loss of 0.36385 assets in total for the EU.The lower part of Table 2 shows the 95% confidence intervals for these averages.Clearly, when no removal costs are taken into  1 we can transform the risk matrix into a payoff matrix for any given cost C r .Table 3 shows an example payoff matrix for cost C r = 0.003 (in arbitrary units, see Section 4).The player's best responses as indicated in bold.One can see that there are two pure Nash equilibria in this scenario, {US 0, EU 1} and {US 1, EU 0}.Moreover there is one mixed equilibrium at where US and EU mix between removing 1 and 0 with probability (0.488, 0.512) and (0.218, 0.782), respectively.
Different choices for C r lead to different games in terms of best-responses and Nash equilibria.We can identify two interesting regions in the range of costs C r .For very low costs, removing 0 will never be a best response for either player.Similarly, for high costs, removing 2 will never be a best response.Therefore we can focus on two sub-games defined by the action-pairs {0, 1} and {1, 2}.
We compute Nash equilibria for a range of C r for the sub-game {0, 1} and visualise the results in Figure 7.
The x-axis shows the cost of removal C r .Each value of C r corresponds to a specific set of Nash equilibria, which are indicated by the coloured lines.The y-axis indicates the (mixed) actions by both players that make up the equilibrium, given as the probability with which each player (US -top graph; and EU -bottom graph) chooses the action remove 0. This equals 1 minus the probability of remove 1 in the two-action sub-game.For example, the solid red line indicates that for C r 0.0026 there exists a pure Nash equilibrium in which both players remove 1 object (the probability of not removing is 0).Similarly, the dashed black line indicates a mixed Nash equilibrium in the range 0.0028 C r 0.0031, the location of which changes linearly with C r .The Nash equilibria for the sub-game {1, 2} are likewise visualised in Figure 8.
In both figures we see interesting transitions from the single Nash equilibrium at (0, 0), to a situation where three equilibria exist at (0, 1), (1, 0) and one mixed, and finally back to a single pure equilibrium at (1,1).
These transitions phases also include a stage in which only one of the asymmetric pure equilibria at (1, 0) or (0, 1) exists.The existence of these asymmetric equilibria is interesting, and results from the asymmetry that is inherent in the risk matrix due to agencies having different numbers of assets and in different orbits.

Strategic substitutes and existence of pure equilibrium
In games we construct in this paper are finite strategic-form games.The celebrated result of Nash [29] shows that every finite game possesses at least one Nash equilibrium in mixed strategies.While mixing makes a lot of sense in some settings, e.g., zero-sum games like poker and sports matches, in other settings pure strategy Equilibria evolution -EU removes 0 or 1 equilibria are more compelling.In this section we discuss properties, relevant for the games we construct, that guarantees the existence of pure equilibria.
In general, active debris removal has a positive effect not only for the instigator of the removal but also for other players, and this is the cause of the dilemma that we are studying.In game-theoretic terminology, this suggests that we have games with a weak strategic substitutes property.The most well-known economic game with this property is Cournot oligopoly [30,31].First we formally define the property.
Our exposition is based on [32], but, for simplicity, is specialized to the setting of finite pure strategy sets.
Denote the set of players by N = {1, 2, . . ., n}.Each player i ∈ N has a finite pure strategy set S i that is a subset of non-negative real numbers, i.e., S i ⊂ R ≥0 .In our space debris removal games, S i can be thought of as the set of choices of how much debris player i removes, so in Table 1, the three strategies remove 0, remove 1, remove 2 would correspond to S i = {0, 1, 2}.Let S denote the set of all pure strategy profiles, i.e., Denote the payoff function of player i by π i : S → R.
For the purpose of stating known results on the existence of pure equilibria, we are going to assume that the payoff of player i depends only on his choice and the aggregate (i.e., sum) of the strategy choices of the other players.Formally, for any pure strategy profile s = (s 1 , s 2 , . . ., s n ) ∈ S, we denote by s−i the additive aggregate of other players strategies, i.e., Then we write our restricted payoff function as π i (s i , s−i ).For any choice s −i ∈ ∏ j∈N\{i} S j , the set β i (s −i ) of best responses of player i is given by for all i ∈ N.For a given player i ∈ N, we denote by S−i the set of all possible values of s−i , the additive aggregate of other players' strategies, i.e., S−i = {s −i | s ∈ S}.We say that a game like this has the weak strategic substitutes property if there exists a function b i : S−i for these games with restricted payoffs functions such that: Such a game with the weak strategic substitutes property, and where payoffs depend only on one's own strategy and the sum of others' strategy, are known to always possess at least one pure strategy Nash equilibrium, which is shown via a potential-function type argument [32][33][34].
Notice that the weak strategic substitutes property can be defined as above even without the restriction that the payoffs are of the form π i (s i , s−i ) for player i.However, in that case a pure equilibrium may not always exist.The games that we construct comprise payoffs that arise from (noisy) simulations and thus do not satisfy the restricted payoff form.However, the games we construct do either have the weak strategic substitutes property, or their violation of it is not statistically significant.Thus it is an interesting future direction to see if we can fit restricted payoff functions to closely approximate the empirical payoffs that arise from our simulations.We discuss this further below, where we also discuss slightly more general aggregation functions for defining restricted payoff functions that, along with the weak substitutes property, guarantee the existence of pure equilibria.First though we note that when considering only two players, the restriction of the payoff functions is without loss of generality, and so we have the following.
Observation 1.Any two-player game that has the weak strategic substitutes property admits a pure equilibrium.
For example, in Table 3, we can see that this game has the weak strategic substitutes property since, as the EU removes more (going from 0 to 1 to 2), the best responses of the US (as indicated by the boxes in Table 3 is to weakly remove less (going from 1 to 0 to 0, respectively), and similarly for the best responses of the EU as US changes pure strategy.This game has two pure equilibria (US 0, EU 1) and (US 1, EU 0) and one mixed equilibrium as one can also see in Figure 7.
As mentioned above, for games with the weak strategic substitutes property, the existence of pure equilibria is known for a wider class of games than just those where the payoff of i depends on his strategy and the sum of the others'.This aggregation of players' strategies done by s−i can in fact be an arbitrary linear combination rather than just a sum, and further can include linear combinations of products of strategies too; for full details see [35].Thus there is actually a lot of scope to fit payoff functions that meet these criteria for existence of a pure equilibrium due to strategic substitution and also are consistent with the payoff estimates that arise from our simulations.We leave this as an interesting direction for further work.

Evolutionary dynamics
Another way to study the strategic properties of a game is by looking at the corresponding evolutionary dynamics.Evolutionary game theory5 represents a player's strategy by a population of individuals, each of a certain type which corresponds to one of the player's possible actions.The fraction of the population belonging to each type indicates the probability with which the player will play the corresponding pure action.The replicator dynamics dictate how the fraction x i of each type i in the population x changes over time due to evolutionary pressure: where f i (x) is the fitness (expected payoff) to type (action) i in the population, and f (x) is the weighted average fitness of the whole population.Under the replicator dynamics, types that do better than average will increase in abundance, whereas types that do worse will decline.
Figure 9 shows the directional field of the replicator dynamics for the sub-game {remove 0, remove 1} for different values of C r corresponding to the different sets of equilibria observed in Figure 7.The axes show the probability with which both players play the action "remove 0" (US 0 and EU 0).The arrows indicate the direction and magnitude of change.The replicator dynamics give insight into the stability of the different equilibria and their corresponding basins of attraction.We can conclude that the mixed Nash equilibria in panels (c) and (d) are unstable, as a small perturbation will cause the population to move towards one of the stable pure equilibria.Moreover we can see that the basin of attraction for the pure equilibrium {US 0, EU 1} (bottom right corner) is larger than for {US 1, EU 0} indicating that this equilibrium is more likely to arise when both players iteratively optimise their strategy.This is of particular interest when full knowledge of the game is not available and the players need to learn by interacting, e.g. when space agencies mutually adapt their policy based on an estimate of other agencies' policies.In fact, the replicator dynamics are descriptive of various multi-agent learning processes, and as such studying these dynamics provides valuable insights in the context of adaptive agents as well [37].
Figure 10 shows similar results for the sub-game {remove 1, remove 2}, corresponding to the different sets of equilibria observed in Figure 8. Again we observe the instability of mixed equilibria, and the differently sized basins of attraction showing asymmetry in the game's payoff structure.

Three player game
So far we have only considered two active players.Here, we take a first step in analysing a larger game between three players (agencies): the US, the EU, and China (CN).We focus on the two-action sub-game {remove 0, remove 1} only to facilitate analysis.Table 4 shows the cumulative risks for all three players, averaged over 180 Monte Carlo runs, as well as the corresponding confidence intervals.The risks for each player are distinguished by different font styles.We can see that the risks for China are considerably higher than for the US or the EU, even though their total number of important assets is lower (see Figure 3).This interesting result may be due to the specific orbits used by each of the player, some being more dense in terms of debris than others, which requires further investigation.
We can again convert the risk matrix into a payoff matrix using the payoff functions defined in Table 1.
In Figure 11 we visualise the Nash equilibria for varying costs of removal C r .At the left part of the figure the cost of removal is low, and therefore it is in the best interest of all three players to remove debris.However, for increasing costs it becomes a best response for the US to stop removing, and there exists a pure equilibrium (US 0, EU 1, CN 1).The reason that the US opts out first is due to their lower overall risk compared to the two other players.In contrast, the higher risks to China mean it is in their interest to keep removing, even when both US and EU have opted out.When the cost rises even further (the right side of the figure), we see that for none of the players removing is viable.
Although for most removal costs C r the strategic substitute property discussed previously holds, there is a range of costs for which the property is violated.However, the payoff differences leading to this violation are not statistically significant and may be resolved by increasing the number of Monte Carlo samples of our simulation, which is left for the future work.

Conclusion and Future Work
In this paper we have presented a new approach to study space debris removal, by introducing a multi-player non-cooperative game named the Space Debris Removal Dilemma.We implemented a realistic model of earth orbit environment, where we projected new future launches, collisions of space objects and natural decay for the next 150 years, for different debris removal strategies.Our experiments confirmed the   commonly predicted exponential growth of space debris in near earth orbits.This is an important motivation to come up with mitigation strategies such as active object removal.
In our game-theoretical analysis of this game we identified Nash equilibria for different levels of cost of removals; although the costs of active debris removal are still prohibitively high at the moment they are expected to decrease with future technological development.Additionally, we investigated the strategic substitute property that appears in this type of game scenario, and which guarantees existence of a pure equilibrium under certain conditions.Although a mixed equilibrium exists for some costs as well, it is often more desirable to focus on pure equilibria.Specifically, in our scenario, it cannot be expected that space agencies will randomize over pure strategies to decide on their space debris removal policy.Another disadvantage of a mixed equilibrium in this game is its instability (as shown in Figure 9), which is undesirable in our scenario, where the choice of action taken has a huge impact on the earth orbit environment.The results of this study help agencies to better understand the debris removal problem and its short and long term consequences, in order to prepare for mitigation strategies.For instance, we show that removing just one high risk debris object every two years can already substantially decrease the risk of collision for active satellites.Additionally, removal of indirect collision risks is beneficial as well as it reduces the number of potential future on-orbit collisions.
There are many routes for future work.Projecting the future evolution of space debris itself is a very complex problem with many unknown variables and inputs, and therefore some necessary simplifications and assumptions have been made.Despite these simplifications the simulation is computationally demanding, which makes it difficult to obtain the necessary number of Monte Carlo runs, especially for larger games.In future work we aim to use HPC clusters to obtain statistically significant results for more extensive scenarios, in which we can include more players (e.g.Russia, India) as well as more diverse debris removal strategies.For example, different types of removal missions may come with different associated cost and success rates, which could further enrich the strategic aspects of the model.
From a game-theoretic point of view, our approach has been limited to a one-shot normal-form game, which assumes that agencies fix their removal policy for the entire time horizon up front.More realistically, these strategies may be adaptive and dependent on the state of the LEO environment as well as on current and past actions by others.One possible future direction is to move to the framework of stochastic or dynamic games.Finally, the strategic substitutes property can be further investigated, for example by attempting to fit a parametrised game to the empirical results.If successful, this would greatly reduce the computational burden of running a variety of similar experiments.

Figure 2 .
Figure 2. Spatial density prediction in LEO

Figure 4 .
Figure 4. Debris evolution for next 150 years

( b )Figure 5 .
Figure 5. Evolution of overall risk to important assets for different combinations of actions of both US and EU.

Figure 6 .
Figure 6.Free-riding effect in the overall risk to important assets for non-active players China and Russia, for combinations of actions of both US and EU.

Figure 7 .Figure 8 .
Figure 7. Equilibrium strategies for the sub-game {remove 0, remove 1} for a range of removal costs C r .

Figure 9 .
Figure 9. Evolutionary dynamics of the sub-game {remove 0, remove 1} for different values of C r .Stable attractors are indicated with and unstable attractors with .The dotted line indicates the trajectory on which the mixed equilibrium moves as C r changes.

Figure 10 .
Figure 10.Evolutionary dynamics of the subgame {Remove 1, Remove 2} for different values of C r .Stable attractors are indicated with and unstable attractors with .The dotted line indicates the trajectory on which the mixed equilibrium moves as C r changes.

Figure 11 .
Figure 11.US, EU and China equilibria for different costs of removal

Table 2 .
Risk matrix for both players for each combination of strategies.The risks are the average cumulative risk of losing an asset over the course of 150 years.We show 95 % confidence intervals in the lower table.

Table 3 .
Payoff matrix for both players for each combination of strategies for C r = 0.003.
account, it is in the best interest of each player to remove as many debris as possible.However, one should assume non-zero removal costs.Using the cost functions of Table