Using Game Theory to Model the Evolution of Information: an Illustrative Game

The application of information theory to biology can be broadly split into three areas: (i) At the level of the genome; considering the storage of information using the genetic code. (ii) At the level of the individual animal; communication between animals passes information from one animal to another (usually, but not always, for mutual benefit). (iii) At the level of the population; the diversity of a population can be measured using population entropy. This paper is concerned with the second area. We consider the evolution of an individual\'s ability to obtain and process information using the ideas of evolutionary game theory. An important part of game theory is the definition of the information available to the participants. Such games tend to treat information as a static quantity whilst behaviour is strategic. We consider game theoretic modelling where use of information is strategic and can thus evolve. A simple model is developed which shows how the information acquiring ability of animals can evolve through time. The model predicts that it is likely that there is an optimal level of information for any particular contest, rather than more information being inherently better. The total information required for optimal performance corresponded to approximately the same entropy, regardless of the value of the individual pieces of information concerned.


Introduction
The classical entropy of an observation with discrete probability distribution (p i ) is de ned as H = ; P i p i log(p i ).This useful concept is of value in describing the information available in living systems in a variety of contexts, and is a bedrock of classical information theory (introduced by Shannon, 1948).For a concise discussion of the important mathematical concepts of information theory and entropy, see Khinchin (1957).
The application of information theory, to biological problems can be broadly split into three areas.The most common application has been at the molecular level, considering the storage of information in long molecules, such as genetic information stored in DNA.It is of particular interest to nd if two sequences are related, i.e. that the pattern of residues in the sequences are similar in some sense.The initial attempts in this area were concerned with global sequence alignments, where all residues of the compared sequences had to be used (e.g.Needleman and Wunsch, 1970).However since distantly related proteins may o n l y h a ve isolated regions in common, more recent w ork has considered local alignments.For an example of this type of approach, see Altschul (1991).This paper considers how to discover local sequence alignments.In particular it focuses on substitution matrices, discussing which matrix is best in which circumstances.What constitutes a signi cant alignment depends upon the model (in particular the level of mutation expected since the time of sequence divergence).
H = P ij q ij log(q ij =p i p j ) is the relative e n tropy of the target and background populations.For some other examples of the use of these ideas, see Borodovsky and Peresetsky (1994), Hannenhalli and Russell (2000) and Kawashima et. al. (1994).Another type of application is at the level of the population.The concept of population entropy as a measure of the diversity of the population, was introduced in Demetrius (1992).H is the population entropy, where xdx and p(x) is the density of new o spring with parents aged x.
One aim of this paper is to distinguish between two distinct population types Type I: the population spends most of its history with a population size uctuating around some constant v alue.Type II: evolution occurs by recurring periods of rapid exponential growth.According to the model, mutation-selection drives Type I to higher entropy, T ype II to a higher growth rate.
For a related idea using discrete age classes applied to a real case study, generating rather di erent predictions, see Smith (1998).
Thirdly at the level of the organism, individuals gain information from other individuals either by direct communication or indirectly through some encounter.
For instance Reznikova a n d R y abko (2000) performed a series of experiments using the ant Formica p olyctena.T w o basic ideas of communication underpin this work.
1) In an e cient c o m m unication system, the more frequent a message, the shorter it should be (l / ; log(p)) -this is common to all human languages.
2) In a complicated numerical system, you need to be able to add and subtract small numbers.Scout ants nd aphid colonies, return to their group, and communicate the location.The group then nds the colony without its help.Initially messages relating to food in box n had length proportional to n (which w ould be sensible if food appeared in each b o x, independently of all others, since the number of the box with the rst food item would follow a geometric distribution).As ants learnt t h a t s o m e b o xes had food more frequently than others, their communication changed.The authors concluded that the initial ant language seems binary the ants can change their numbering system to increase e ciency the ants seem to be able to add and subtract small numbers.
The actual methods used by animals to communicate with each other can be complex and varied.Holland et al. (2000) studied communication in wrens, in particular looking at how t h e y convey messages in an environment with a large amount of noise degradation.Signals contain messages derived from the signalling behaviour (e.g.timing and location) or by coded signal parameters.Songs consist of `elements' and between element pauses which together form a rhythm.They found that rhythmicity of song is not essential, whilst actual song elements are required.Alterations to rhythm, syntax, spectra still elicit a territorial response, although less that the control.None are essential, but all add to the options for discrimination.
The evolution of language has been considered in terms of information theory in a series of papers by N o wak and collaborators (Nowak and Krakauer, 1999Nowak et al. 1999Plotkin and Nowak, 2000).It is assumed that communication is of bene t to both communicating parties.In particular they model the way that particular signals evolve to gain speci c meanings.It is shown in Plotkin and Nowak (2000) that if there is a chance of mistaking signals for others then evolution leads to a given error limit, and that this limit has a natural interpretation in classical information theory.

Information and Game Theory
Important aspects of animal behaviour have been well modelled using game theory.T o nd the best strategy, t h e a vailable strategies, the payo s and the information available are required.Often the information available to animals is surprisingly limited this in turn reduces their strategic options (see below).
2.1 Some examples of animals and their information use in contests 1) Hammerstein (2000) looks at the e ect of the limitations imposed by an animal's psychology on its choice of behaviour.Two examples are considered to illustrate this.The desert ant Cataglyphis h a s a n a vigation system that enables it to travel long distances and then return to its origin.It is not possible to predict the ant's system of navigation merely by k n o wing the problem the ant has to solve.Through experiments (see Wehner, 1997) it is shown that the ant is not using any trigonometry but simpler inaccurate systems leading, on occasion, to substantial errors, which however are correctable.The ant's navigation system looks odd, but works quite well.It has optimised within a restricted strategy/information set.
2) The second example of Hammerstein (2000) is the desert spider Agelenopsis aperta (see also Hammerstein and Riechert, 1988).Territorial contests between a web owner and an intruder occur frequently.The winner takes possession of the web.What are the available spider strategies?Simplemindedly, the options available to the spider are to ght or to concede.In particular it may v ary the intensity and duration of any e ort it puts into a contest before conceding if this is insu cient.What information do the spiders possess?They use the web to gauge the weight o f the opponent.They could also in principle compile statistics, to see how large they are compared to the general population (contests are frequent).This would be useful e.g. if a spider is in an equal contest it would be advisable to ght hard if small (this is one of the few opponents you have a c hance of beating) and similarly less hard if large.The spiders do not seem to possess this ability, and contests tend to depend upon the relative sizes of the protagonists, irrespective of their position within the overall population.In general large spiders beat small ones and for relatively equal spiders, the web owner wins, so that the results of contests are as might be expected, although the durations/intensities are not optimal.
3) Bridge et al. ( 2000) examines what information is available to males during contests over females in the orb-weaving spider Metellina mengel.Males engage in pre-copulatory guarding of females, waiting on the female's web for the arrival of a y prior to courtship.During this time other males may arrive.Contests occur over who will remain.Intruders appear unable to assess female quality.The contest duration is primarily decided by the absolute size of the loser, not the relative size of the opponents.This is theoretically consistent with individuals not being able to discern their opponent's size (at least not very quickly).Fighting is clearly sub-optimal, due probably to paucity of information.
As the above illustrates, in any situation, the information available a ects the possible strategies and thus the nal outcome in any ecological situation.This has been recognised for some time and built into evolutionary models.Maynard Smith and Parker (1976) modelled several situations, such a s asymmetric contests with perfect information contests with unequal payo s and perfect information contests with incomplete information information acquired during a contest.In these and later models, one consistent feature was that the information available is xed.

Information as a strategic concept
Although an important feature of any game-theoretic situation, the information available to the players is usually considered a constant, unchanging property.However, suppose that animals could increase their information (at a cost) or reduce it to save resources.Extra information would have t o h a ve a certain minimum value to be worth having (in the same way a s y ou are willing to pay the price to acquire some books, but not others).Thus natural selection can act on the information level as well as on the playing strategies, and so the available information can evolve in the same way that strategies do.
We consider a very simpli ed situation, not necessarily realistic for any population, but with the aim of demonstrating the general evolutionary principles and the kind of results that are possible with the minimum of complication.
A contest is held between two individuals over an item of value V .The contestants have a choice of two strategies, the classical options of Hawk (H) and Dove (D) introduced by M a ynard Smith and Price (1973).Each individual has a dominant feature which is critical for the outcome of contests we shall call this `size'.For convenience size is scaled between 0 and 1, so that every animal has a size between these two extremes.
We further suppose that in any c o n test, if one animal plays the aggressive H a wk and the other the passive D o ve, then the Hawk player will win, otherwise the larger animal wins (in a Hawk versus Hawk contest, the smaller loser incurs a cost C, i.e. receives a reward ;C where C > V ).
Thus, in a contest between a larger animal Player 1 (P 1 ) and a smaller one Player 2 (P 2 ) rewards are decided by the relative size of individuals, so that the following contest types lead to the rewards given P 1 plays P 2 plays P 1 receives P 2 receives H H Entropy 2002, 4 , 40 2.3 The use of information Let the size of an individual be represented by its expansion in base h.W e suppose that individuals cannot gauge size exactly, b u t h a ve information which they use to make an estimate of the opponent's size.P 1 has n pieces of information about the size of individuals (we s h a l l s a y that this individual has information n, o r i s a n n-individual), which are the rst n terms in the expansion, so that it knows the other's size to n places in the expansion.
Thus if h = 2 then the expansion will be binary, e .g .0 :1101001 : : : .If another animal has n = 3, then it will see only 0.110, and so the animal could have size anywhere between 0.110 and 0.111.P 2 knows the rst m places (both players know w h i c h has the greater information, but not the extent of their (dis)advantage).
For simplicity w e further suppose that in a particular population of animals, the sizes of the individuals are uniformly distributed on (0,1).
What strategies should players choose and what rewards do they get?
2.4 The game where information is xed Let us suppose that m < n Thus P 1 is at an informative a d v antage, knows this, but does not know the extent of its advantage.
Denoting z] as the greatest integer less than or equal to z, if the size of P 1 is x and the size of P 2 is y, then we h a ve the following possibilities: 1) h m x] > h m y].Both players know t h a t P 1 is the larger, so that P 1 plays H and receives V , P 2 plays D and receives 0. 2) h m x] < h m y].Both players know t h a t P 2 is the larger, so that P 2 plays H and receives V , P 1 plays D and receives 0. 3) case 1) does not occur but h n x] > h n y].P 1 knows that it is larger, but P 2 is not sure (both players appear the same to it).Thus P 1 plays H, P 2 plays H with probability .4) case 2) does not occur but h n x] < h n y].P 1 knows that it is smaller, but P 2 is not sure (both players again seem equal to it).P 1 does not know that it is not in situation 2 (if it knew, it may be able to `blu ').Thus P 1 plays D, P 2 plays H with probability , since 3) and 4) are indistinguishable to it.5) h n x] = h n y].Neither player is sure which is the larger (both players appear the same).Thus P 1 plays H with probability , P 2 plays H with probability .
Assuming that the size of an animal is independent of its information level, the overall expected rewards are thus The payo for P 2 is maximised when = 0 .G i v en that this is true, the payo to P 1 is maximised when = 1 (thus when P 2 is unsure (cases 3-5) it plays D, when P 1 is unsure it plays H).
The optimal play can thus be summarised as follows When n > m P 1 play s H w h e n i t k n o ws it is larger or is unsure, and plays D when it knows it is smaller.P 2 plays D when it is smaller or unsure, and plays H when larger.
The payo s are E P 1 ] = V=2 + ( h ;n + h ;m )V=4 (3) E P 2 ] = V=2 ; (h ;n + h ;m )V=4: (4) When n = m the players know that they are evenly matched in information when they both know that one is larger, then that player plays H receiving V , the smaller playing D and receiving 0. When both are unsure, we h a ve the classical Hawk-Dove game where both play H with probability V=C.

Variable information
Suppose that each piece of information costs an amount K. Early information is important, later increasingly irrelevant.If each animal could choose which v alue of n to pick, which w ould be the `best'?Of course it is unlikely that all players have such a free choice, but mutation may cause individuals with di erent information levels to emerge whether these dominate the population or die out is then determined by natural selection.
If everyone has information n, c a n y ou do better by c hanging to any other information?(if the answer is no, then n is a stable information state).In particular changing to n;1 o r n+1 are the most signi cant alternatives, but not the only ones of interest (see later).To nd if changing to n;1 is better can be found by comparing the payo to a player with information n;1 against one with n (E n ; 1 n ]), against the payo to an n individual against another n individual (E n n]), since when almost all players are n individuals, it is how a player does against these that matter.
Doing the same thing for the change to n + 1 g i v es Let us represent the expected payo to an n individual in a population comprised of a proportion p of n individuals and 1 ; p m individuals by E n p(n) + ( 1 ; p)(m)].It is assumed that opponents are met at random so that this expression In general we shall say that n beats m if in any mixture of n and m animals, evolution leads to a complete population of n animals i.e.E n p(n) + ( 1 ; p)(m)] > E m p(n) + ( 1 ; p)(m)] for every value of p 2 0 1]: Similarly n loses to m if m beats n.
Suppose that in the current population there is a proportion of individuals p of information n, and a proportion 1 ; (8) which decreases with p, so that n individuals do relatively worse, the more of them that there are.
Thus this reduces to saying that n beats n + 1 i f E n + 1 n ] ; E n n] < 0, n + 1 b e a t s n if E n + 1 n + 1 ] ; E n n + 1 ]> 0 and a mixture of the two informations will result if neither of the above is true (solving for p when setting the above expression equal to zero gives the composition of this mixture.
It is easy to show t h a t i f m > n then E m + 1 n ] < E m n] s o t h a t i f E n n] > E n + 1 n ] then E n n] > E m n] for all m > n .
Let us consider another m s.t.m < n .
This is true whenever E n n] ; E n ; 1 n ] > 0, except when Cf1 + h n;m ; (n ; m)(h + 1 ) g < ;2(n ; m + 1 ) V (10) which is only true if m = n ; 2, h = 2 and C < 6V (since h 2 a n d C > V ).Thus if n beats n ; 1 a n d n + 1 then it is the uniquely best information level, except in the above case where it must also be compared to n ; 2.

Discussion
A simple model has been developed which s h o ws how the information acquiring ability of animals can evolve through time.Furthermore the model predicts that it is likely that there is an optimal level of information for any particular contest, rather than more information being inherently better.The value of the extra information in the game diminished as more was obtained, whereas the cost did not.This is reasonable for con ict situations in general.An animal's ghting ability can usually be assessed fairly accurately by a small number of important measures, other measures adding relatively little to this.Mixtures of information levels were possible, but for our example at least, these would not occur frequently.T h e n umber of pieces of information required depended upon the value of the information in our game the larger h the more valuable the information, in the sense that the same number of pieces of information identi ed an animal's size to within a smaller range of possibilities the larger h was.The entropy H of each piece of information also increased with h (H=log(h)).The total information required for optimal performance corresponded to approximately the same entropy, regardless of the value of the individual pieces of information concerned.
There was an unusual situation for a restricted set of parameters in the game, which meant that in some circumstances individuals with information n ; 2 could invaded a population of individuals with information n, but that individuals with information n ; 1 could not.This could potentially lead to three (or more) information levels coexisting in a stable mixture.Whether this would be important depends upon the nature of the evolution process.If mutations occur so that new individuals are at most one information level away from the current dominant t ype (so that possible changes occur as small steps, not large leaps) then if n ; 1 cannot invade n, then the situation of n ; 2 individuals competing in a population of n individuals would never occur.
The model that has been considered here is very idealised, and clearly will not be applicable for real populations, but was rather designed to illustrate the possibilities.It would be of great interest to examine real populations to see if the behaviour of this type is exhibited.In particular it would be of use to compare radically di erent populations with di erent options and rewards to see if the information levels predicted by t h i s t ype of model are realistic.Of course for genuine populations, a more sophisticated and specialised version of the model would be required.