Journal of Theoretical Biology 399 (2016) 103-116

Journal of Theoretical Biology 399 (2016) 103-116 Contents lists available at ScienceDirect Journal of Theoretical Biology journal homopago::::vw.elsevier.com/locate/yjtbi Jam. 1 or Theoretical lhoOgy Evolution of worker policing Jason W. Olejarza, Benjamin Allen b•a'c, Carl VelleraA, Raghavendra Gadagkai Martin A. Nowaka•d-gs ▪ Program for Evolutionary Dynamic. Harvard University, Cambridge. MA 02738, USA b Department of Mathematics. Emmanuel Cortege, Boston, MA 02115. USA Center for Mathematical Sciences and Applications. Harvard University, Cambridge. MA 02738, USA ▪ Department of Organismic and Evolutionary Biology. Harvard University, Cambridge, MA 0213B, USA ' Centre for Ecological Sciences and Centre for Contemporary Studies. Indian Institute of Science, Bangalore 560 012. India Indian National Science Academy, New Delhi 170 002. India 2 Department of Mathematics, Harvard University, Cambridge. AM 02138, USA CrossMatk ARTICLE INFO Ankle history. Received 2 February 2015 Received in revised form 23 January 2016 Accepted 2 March 2016 Available online II March 2016 Keywords: Sociobiology Natural selection Evolutionary dynamics Models/simulations ABSTRACT Workers in insect societies are sometimes observed to kill male eggs of other workers, a phenomenon known as worker policing. We perform a mathematical analysis of the evolutionary dynamics of policing. We investigate the selective forces behind policing for both dominant and recessive mutations for dif- ferent numbers of matings of the queen. The traditional, relatedness -based argument suggests that policing evolves if the queen mates with more than two mates, but does not evolve if the queen mates with a single male. We derive precise conditions for the invasion and stability of policing alleles. We find that the relatedness -based argument is not robust with respect to small changes in colony efficiency caused by policing. We also calculate evolutionarily singular strategics and determine when they are evolutionarily stable. We use a population genetics approach that applies to dominant or recessive mutations of any effect size. e. 2016 Elsevier Ltd. All rights reserved. 1. Introduction In populations with haplodiploid genetics. unfertilized female workers are capable of laying male eggs. Thus, in a haplodiploid colony, male eggs can in principle originate from the queen or from the workers. Worker policing is a phenomenon where female workers kill the male eggs of unmated female workers (Ratnieks, 1988: Ratnieks and Visscher.1989: Ratnieks et al„ 2006: Gadagkar, 2001: Wenseleers and Ratnieks, 2006a). Worker policing is observed in many social insects, including ants, bees, and wasps. Yet the precise conditions for the evolution of worker policing are still unclear. Worker policing (Ratnieks, 1988: Rah-licks et al.. 2006: Gadagkar, 2001: Wenseleers and Ratnieks, 2006a) and worker sterility (Wilson, 1971: Hamilton, 1972: Olejarz et al.. 2015) are two distinct phe- nomena that are widespread in the eusocial Hymenoptera. In addi- tion to worker policing. a subset of workers in a colony may forego their own reproductive potential to aid in raising their siblings. Prior relatedness -based arguments have suggested that queen monogamy is important for the evolution of a non-reproductive worker caste 'Corresponding author. E-mail address: martin_nowaloPharvard.edu (MA Nowak). http:fidx.dokorg/10.1016/mtbi.2016.03.001 0022-51930 2016 Elsevier Ltd. All rights reserved. (Hughes et al., 2008: Cornwallis et al.. 2010: Queller and Strassmann, 1998: Foster et al.. 2006: Boomsma, 2007, 2009). In contrast• it is believed that polygamy—not monogamy —is important for the evo- lution of police workers. Several papers have studied the evolution of policing. Starr (1984) explores various topics in the reproductive biology and sociobiology of eusocial Hymenoptera. He defines promiscuity as 1/(E7_,J1), where n is the number of matings of each queen. and .h is the fractional contribution to daughters by the i-th male mate. He writes, regarding workers, that "They are on average less related to nephews than brothers whenever (promiscuity is greater than twol and should prefer that the queen lay all the male eggs. Workers would therefore be expected to interfere with each others reproduction.' Thus, Starr (1984) was the first to suggest that workers should raise their nephews (sons of other workers) if the queen mates once, but should only raise their brothers (sons of the queen) if the queen mates more than twice. Starr (1984) uses a relatedness -based argument, but he does not provide any calcu- lation of evolutionary dynamics in support of his argument: he uses neither population genetics nor inclusive fitness theory. In a book on honeybee ecology. Seeley (1985) also proposed, using a relatedness -based argument, that worker policing should occur in colonies with multiply mated queens, but that worker policing should be absent if queens are singly mated. EFTA01070997 104 OkJorz et al /Journal of theatrical Biology 399 (2016) 103-116 Woyciechowski and Lomnicki (1987) perform a calculation based on population genetics and conclude that workers should raise their nephews (sons of other workers) if the queen mates once, but should only raise their brothers (sons of the queen) if the queen mates more than twice—the case of double mating is neu- tral with respect to preference. From this result, they claim that. under multiple mating of the queen, natural selection should favor non-reproductive workers. Woyciechowski and Lomnicki (1987) consider both dominant and recessive alleles affecting worker behavior, but they do not consider colony efficiency effects. Ratnieks (1988) considers the invasion of a dominant allele for policing. Using population genetics, he arrives at essentially the same conclusion as Woyciechowski and Lomnicki (1987): In the absence of efficiency effects, policing evolves with triple mating but not with single mating. But Ratnieks also considers colony efficiency effects, focusing mainly on the case where policing improves colony efficiency. Since policing occurs alongside other maintenance tasks (such as cleaning of cells, removal of patho- gens. incubation of brood), and since eating worker-laid eggs might allow workers to recycle some of the energy lost from laying eggs, Ratnieks supposes that policing improves colony efficiency. He finds that worker policing with singly mated queens may evolve if policing improves colony reproductive efficiency. He also finds that worker policing with triply mated queens may not evolve if policing reduces colony reproductive efficiency, but he considered this case to be unlikely on empirical grounds. Ratnieks does not study recessive policing alleles. He also does not calculate evolutionary stability conditions. Both papers (Woyciechowski and Lomnicki. 1987: Ratnieks, 1988) offer calculations based on population genetics without mentioning or calculating inclusive fitness. These early studies (Starr, 1984: Seeley, 1985: Woyciechowski and Lomnicki, 1987: Ratnieks. 1988) were instrumental in establishing the field of worker policing. Testing theoretical predictions on the evolution of worker policing in the field or in the lab is difficult. Due to the complex- ities inherent in insect sociality, published empirical results are not always easy to interpret. While, so far, worker policing has been found in all species with multiple mating that have been studied, it has also been found in about 20% of species with singly mated queens (Hammond and Keller, 2004: Wenseleers and Ratnieks. 2006b: Bonckaert et al.. 2008). Herein lies the difficulty: When worker policing is found in multiply mated species and found to be absent in singly mated species, this is taken as evi- dence supporting the relatedness argument, and when worker policing is found in singly mated species, it is explained away as not being evidence against the theory, but as having evolved for other reasons (such as colony efficiency). See, for example, the following quotation by Bonckaert et al. (2008): "Nevertheless, our results are important in that they show that V. germanico forms no exception to the rule that worker reproduction should be effec- tively policed in a species where queens mate multiple times (Ratnieks. 1988). Indeed, any exception to this pattern would be a much bigger challenge to the theory than the occurrence of worker policing in species with single mating, which can be readily explained (Ratnieks. 1988: Foster and Ratnieks, 2001b)' This is precisely why a careful simultaneous consideration of relatedness, male parentage, and colony efficiency is important for understanding worker policing. We do not aim to provide an exhaustive catalog of all species in which worker policing has been studied. We merely cite some specific examples to add context Policing is rampant in colonies of the honeybee (Ratnieks and Visscher, 1989), the wasp Vespula vulgaris (Foster and Ratnieks. 2001c), and the wasp Vespula ger- manica (Bonckaert et al.. 2008), which are all multiply mated. (As mentioned above, worker policing has been found in all of the studied species to date that are multiply mated.) Worker removal of worker-laid eggs is much less prevalent in colonies of the bumblebee (Velthuis et al.. 2002), the stingless bee. (Peters et al.. 1999), and the wasp, Vespula rub (Wenseleers et al.. 2005). which are predominantly singly mated. (As mentioned above, worker policing has been found only in about 20% of the studied species to date that are singly mated.) There are some studies based on observational evidence that find policing in singly mated species: examples of species with single mating and worker policing are Vespa crabro (Foster et al., 2002), Camponotusfloridanus (Endler et al.. 2004). Aphaenogasrer smythiesi (Wenseleers and Ratnieks. 2006b), and Diacamma (Wenseleers and Ratnieks, 2006b). Interspecies comparisons are somewhat problematic, because even though phylogeny can be controlled for, there are many (known and unknown) ways in which species differ in addition to mating frequency that may also affect the absence or presence of worker policing. Furthermore. many empirical studies are based on genetic analyses of male parentage. (Though studies of some species are based on actual observational evidence: see, e.g., Wenseleers and Ratnieks. 2006b.) Regarding species for which the study of policing is based on genetic analyses, policing is often inferred if males are found to originate predominantly from the queen. But such an inference. in cases where it is made. pre- supposes that workers actively try to lay male eggs in the first place. It is therefore not clear how reliably genetic investigations can measure policing. The small number of attempts at measuring the prevalence of worker policing in intraspecific experiments have also returned conflicting results. Foster and Ratnieks (2000) report that facul- tative worker policing in the saxon wasp. Dotichovespula scuronica, is more common in colonies headed by multiply mated queens. But their sample size is only nine colonies. The phenomenon was reinvestigated by Bonckaert et al. (2011) who report no evidence of facultative worker policing depending on queen mating fre- quencies. and argue that the previous result may have been flawed or that there were interpopulational variations. Many empirical studies have emphasized that factors besides intracolony relatedness —including the effects of policing on a colony's rate of production of offspring—may play a role in explaining evolution of worker policing (Foster and Ratnieks 2001a,c: Hartmann et al., 2003: Hammond and Keller, 2004: Wenseleers and Ratnieks. 2006b: Helantera and Sundstrom, 2007: Khila and Abouheif, 2008: Zanette et al.. 2012). Yet reliable pub- lished data on the effect that policing has on colony reproductive efficiency are often hard to find. (For some exceptions, see Wenseleers et al., 2013 and references therein.) In this paper. we derive precise conditions for the evolutionary invasion and evolutionary stability of police alleles. We consider any number of matings, changes in the proportion of queen- derived males. changes in colony efficiency, and both dominant and recessive mutations that affect the intensity of policing. Our paper is based on an analysis of evolutionary dynamics and population genetics of haplodiploid species (Nowak et al., 2010: Olejarz et al.. 2015). It does not use inclusive fitness theory. Spe- cifically. we adapt the mathematical approach that was developed by Olejarz et al. (2015) for the evolution of non-reproductive workers. We derive evolutionary invasion and stability conditions for police alleles. Mathematical details are given in Appendix A. In Section 2, we present the basic model and state the general result for any number of matings for dominant policing alleles. In Sections 3-5. we specifically discuss single. double, and triple mating for dominant policing alleles. We take dominance of the policing allele to be the more realistic possibility because the policing phenotype is a gained function. Nonetheless. for com- pleteness. we give the general result for recessive policing alleles in Section 6. In Section 7. we discuss how the shape of the EFTA01070998 J.W. OleJeri et al. /Journal of theoretical Biology 399 (2016) 103-116 105 efficiency function determines whether or not policing is more likely to evolve for single or multiple matings. In Section 8. we analyze our results for the case where the phenotypic mutation induced by the mutant allele is weak (or. equivalently in our formalism, the case of weak penetrance). In this setting, the quantity of interest is the intensity of policing. We locate the evolutionarily singular strategies. These are the values of intensity of policing for which mutant workers with slightly different policing behavior are, to first order in the mutant phenotype, neither advantageous nor disadvantageous. We then determine if a singular strategy is an evolutionarily stable strategy (ESS). In Section 9. we discuss the relationship between policing and inclusive fitness theory, together with the limitations of the relatedness -based argument. Section 10 concludes. 2. The model We investigate worker policing in insect colonies with haplo- diploid genetics. Each queen mates n times. We derive conditions under which a mutation that effects worker policing can spread in a population. We make the simplifying assumption, as do Woyciechowski and Lomnicki (1987) and Ratnieks (1988), that the colony's sex ratio is not affected by the intensity of worker policing. First we consider the case of a dominant mutant allele. Because the policing allele confers a gain of function on its bearer, the assumption that it is dominant is reasonable. There are two types of males. A and a. There are three types of females. AA, Ac. and aa. If the mutant allele is dominant, then Aa and aa workers kill the male eggs of other workers, while AA workers do not. (Alter- natively. AA workers police with intensity 44 while Aa and aa workers police with intensity ZA0 = Z2a = 44+w. We consider this case in Section 8.) For n matings, there are 3(n + 1) types of mated queens. We use the notation AA. m: Aa, m: and aa,m to denote the genome of the queen and the number. m. of her matings that were with mutant males, a. The parameter m can assume values 0,1...., n. A schematic of the possible mating events is shown in Fig. 1(a). There are three types of females, AA. Act, and ca. and there are n+1 possible combinations of males that each queen can mate with. (For example, a queen that mates three times (n=3) can mate with three type A males, two type A males and one type a male, one type A male and two type a males, or three type a males.) Fig. 1(b) shows the different colony types and the offspring of each type of colony when each queen is singly mated. Fig. 1 (c) shows the different colony types and the offspring of each type of colony when each queen mates n times. The invasion of the mutant allele only depends on a subset of colony types. The cal- culations of invasion conditions are presented in detail in Appendix A. 2.1. Fraction of male offspring produced by the queen Pr represents the fraction of males that are queen-derived if the fraction of police workers is z. (This quantity was already employed by Ratnieks. 1988.) The parameter z can vary between 0 and 1. For z=0. there are no police workers in the colony, and for z=1. all workers in the colony are policing. We expect that pr is an increasing function of z. Increasing the fraction of police workers increases the fraction of surviving male eggs that come from the queen (Fig. 2). a Virgin Queens nMales Fertilized Queens AA +(n—m) A +m a AA,m Aa +(n-m) A +m a Aa,m aa +(n—m) A +m a aa,m b Oman Gomm OM Ousts MO•Onellos MO Oen mina Man On VIManelms MO AA A A A•• M1/21 As A An MA AA•Pa 5A•• AM *aim An Ana ma As A•• M1 la a C IsehOINOI MOE • TOM Omen Omega Omen Oman I Clain- MINIONIfielt• la na AnmPa A ft...inns fan-r••ta•h•Ael. - Mon I 05-MM•Mann oil A • Ilp 16-nIsa•maa In-MA•ln•mla Flg. I (a) The possible mating events with haplodiploid genetics are shown. Each queen mates with n males. in denotes the number of times hat a queen mates with mutant type a males and can take values between 0 and n Thus, there are 3(n +1) types of colonies. (b) If each queen mates with only a single male, then there are six types of colonies. The female and male offspring (right hree columns) of each colony (leftmost column) are shown. For example. AA. I co onies arise from a type AA female mating with a single mutant type a male. M.1 queens produce female offspring of type Aa and male offspring of type A. 50% of the offspring of workers in M. I colonies are of type A, while the remaining 50% of the offspring of workers in M. I colonies are of type a. (c) The female and male offspring (right three columns) of each colony (leftmost column) when each queen mates it times are shown. 0 0 2E 0.75 rngi 0.50 Possible Forms for the Function p: wo 0.25 O LI-0 0 0.25 0.50 0.75 Fraction of police workers, z Fig 2. The queen's production of male eggs, p., increases with the fraction of workers that are policing, 2. This is intuitive, since having a larger worker police force means that a greater amount of worker-laid eggs can be eaten of removed. Three possibilities for a monotonically increasing function pt are shown. 22. Colony efficiency as a function of policing re represents the rate at which a colony produces offspring (virgin queens and males) if the fraction of police workers is z. (This quantity was also employed by Ratnieks. 1988.) Without loss of generality, we can set ro = 1. For a given mutation that affects the intensity of policing, and for a given biological setting, the efficiency function re may take any one of a variety of forms (Fig. 3). Colony efficiency depends on interactions among police work- ers and other colony members. It also depends on the interactions of colonies and their environment. There are some obvious nega- tive effects that policing can have on colony efficiency. By the act EFTA01070999 106 OkJarz et al. 1 puma, of theoretical Biology 399 (2016)193-116 1.3 1.2 0 1.1 z 0.9 0.8 Possible Forms for the Function r 0 0.25 0.50 0.75 Fraction of police workers, z Hg. 3. The functional dependence of colony efficiency. rz. on the fraction of workers that are policing. z, may take any one of many possibilities. of killing eggs, police workers are directly diminishing the number of potential offspring. In the process of identifying and killing nephews, police workers may also be expending energy that could otherwise be spent on important colony maintenance tasks (Cole. 1986: Naeger et al.. 2013). Policing can also be costly if there are recognitional mistakes. Le, queen-laid eggs may accidentally be removed by workers. Recognitional errors could result in modifications to the sex ratio, which is an important extension of our model but is beyond the scope of this paper. We can also identify positive effects that policing may have on colony efficiency. It has been hypothesized that the eggs which are killed by police workers may be less viable than other male eggs (Velthuis et al.. 2002: Pirk et al., 1999: Cadagkat 2004: Nonacs. 2006), although this possibility has been disputed (Beekman and Oldroyd. 2005: Helantera et al., 2006: Zanette et al., 2012). If less- viable worker-laid eggs are competing with more-viable queen-laid male eggs, then policing may contribute positively to overall colony efficiency. Moreover, policing decreases the incentive for workers to expend their energy laying eggs in the first place (Foster and Ratnieks, 2001a: Wenseleers et al.. 2004a,b: Wenseleers and Ratnieks, 2006a), which could be another positive influence on colony efficiency. (However, the decrease in incentive for workers to reproduce due to policing would only arise on a short time scale if there is a facultative response to policing, which is unlikely.) As another speculative possibility: Could it be that worker egg- laying and subsequent policing acts as a form of redistribution within the colony? That is, suppose that it is better for colony efficiency to have many average-condition workers than to have some in poor condition and some in good condition. Suppose further, as seems realistic, that good-condition workers are more likely to lay eggs (which are high in nutritional content. of course) If the average police worker is of condition below the average egg- laying worker, then worker egg-laying and policing serves to redistribute condition among the workers, improving overall col- ony efficiency. The special case, where policing has no effect on colony effi- ciency and which has informed the conventional wisdom. is ungeneric, because policing certainly has energetic consequences for the colony that cannot be expected to balance out completely. An early theoretical investigation of colony efficiency effects regarding invasion of dominant mutations that effect worker policing was performed by Ratnieks (1988). Although monotonically increasing or monotonically decreas- ing functions rz are the simplest possibilities, these cases are not exhaustive. For example, a small or moderate amount of policing may be expected to improve colony efficiency. However, the pre- cise number of police workers that are needed to effectively police the entire worker population is unclear. It is possible that a frac- tion ze 1 of police workers can effectively police the entire population, and adding additional police workers beyond a certain point could result in wasted energy, inefficient use of colony resources, additional recognitional errors. etc. These effects may correspond to colony efficiency rz reaching a maximum value for some 0 < z < 1. As another possibility, suppose that police workers, when their number is rare, directly decrease colony efficiency by the act of killing male eggs. It is possible that for some z < 1. police workers are sufficiently abundant that their presence can be detected by other workers. Assuming the possibility of some type of facultative response, the potentially reproductive workers may behaviorally adapt by reducing their propensity to lay male eggs. instead directing their energy at raising the queen's offspring. In this scenario. colony efficiency rz may reach a minimum value for some 0 ez e 1. 2.3. Main results for dominant police alleles We derive the following main results for dominant police alleles. If the queen mates with n males, then the a allele for policing can invade an A resident population provided the fol- lowing -evolutionary invasion condition" holds: Ptin +Pia Cm) (rig) >2_ CI /2) _ _ Cin) (1) ro ro ro Pito ro When considering only one mutation. ro can be set as 1 without loss of generality. Why are the four parameters. rim. rla, pun. and pia. sufficient to quantify the condition for invasion of the mutant allele, a? Since we consider invasion of a. the frequency of the mutant allele is low. Therefore, almost all colonies are of type AA, 0. which means a wild-type queen. AA, has mated with n wild- type males. A, and 0 mutant males, a. In addition, the colonies Act 0 and AA, 1 are relevant These are all colony types that include exactly one mutant allele. Colony types that include more than one mutant allele (such as Aa, 1 or AA,2) are too rare to contribute to the invasion dynamics. For an Ac. 0 colony, half of all workers are policing, and therefore the parameters r112 and p1/2 occur in Eq. (1). For an AA. 1 colony, 1/n of all workers are policing, which explains the occurrence of run and pim in Eq. (1). Next. we ask the converse question: What happens if a popu- lation in which all workers are policing is perturbed by the introduction of a rare mutant allele that prevents workers from policing? If the a allele for worker policing is fully dominant. and if colony efficiency is affected by policing, then a resident policing population is stable against invasion by non-police workers if the following 'evolutionary stability condition" holds: +n)(2 +Pi ) + Pan - oita,o(n —2) (2) 2(2+n+np i) roan-mart What is the intuition behind the occurrence of the four para- meters, ri, ran_ wan). Pt. and Pan-102M? The condition applies to a population in which all workers are initially policing. Note that. because the allele. a, for policing is fully dominant in our treat- ment. non-policing behavior arises if at least two mutant A alleles for non-policing are present in the genome of the colony, which is the combination of the queen's genome and the sperm she has stored. To study the invasion of a non-policing mutant allele. we must consider all colony types that have 0.1. or 2 mutant A alleles: these are aa.n: aa,n — 1: Aa,n: n-2: Aa,n-1: and AA, n. The colonies aa.n: mai —1: Aa,n: aa, n —2: and AA, n do not contain non-police workers: the efficiency of those colonies is r1, and the fraction of male eggs that originate from the queen in those colonies is pi. Both of these parameters occur in Eq. (2). Colonies of type At n —1 produce a fraction of 1/(2n) non-police workers. EFTA01071000 J.W. Oleforz er al. /Journal of theoretical Biology 399 (2016) M3-116 107 Single mating. n=1 10 1010 To 104 '1104 .6104 4 1. 10 ! 1.1. 104 .7 Double mating, n=2 2 Time (104) 6 Fig. a Numerical simulations of the evolutionary dynamics of worker policing confirm the condition given by Eq. (1). The policing allele is dominant. For numerically probing invasion, we use the initial condition XA40 —1-10 s and Kai — 10 a. We set ra —1 without loss of generality. Other parameters are: (a) pi,2 —0.75, pi—0.9, and rig-1A1: (b)pla —0.6, p,—0.8. — 1.005, and r,-1.01. 0 Police allele can invade and is evolutionarily stable can invade, but is not stable cannot invade. but is stable cannot invade and is not stable 1 kequency of police allele S. There are four possibilities for the dynamical behavior in the proximity of two pure equilibria a n= 1 mating 1.2 Does Not Invade Stable a 8 0.9 0.8 08 0.9 1 1.1 12 Colony efficiency. r„2P1/2 = 0.75. p1 =1 Stable Does Not Invade Unstable Invades Unstable Colony efficiency. r1 b n = 1 mating 1.2 1.1 1 0.9 0.8 08 Does Not Invade Stable pv2 = 0.99. Pi = I Invades Stable Does Not Invade Invades Unstable 0.9 1 1.1 Colony efficiency. r112 12 Mg. 6. If queens are singly mated (n-1). then a plot of ri versus rin clearly shows all four possibilities for the behavior around the two pure equilibria. For (a). we set pi,2 —0.75 and p, — I. For (b), we set pc.2 —0.99 and pi — I. which explains the occurrence of rota _ Iva," and pan_ i)!2,0 in Eq. (2). Numerical simulations of the evolutionary dynamics with a dominant police allele are shown in Fig. 4. Generally, four scenarios regarding the two pure equilibria are possible: Policing may not be able to invade and be unstable, policing may not be able to invade but be stable, policing may be able to invade but be unstable, or policing may be able to invade and be stable. The possibilities are shown in Fig. 5. In the cases where policing cannot invade but is stable, or where policing can invade but is unstable, Brouwer's fixed-point theorem guarantees the existence of at least one mixed equilibrium. In the case where policing can invade but is unstable, police and non-police workers will coexist indefinitely. We will now discuss the implications of our results for parti- cular numbers of matings. EFTA01071001 108 Okjarz er aL /journal of theoretical Biology 399 (2016)103-n6 a The police allele invades and is stable. p„.2=0.75. rue 1.0344. r1=1.0767 Frequency of policing allele, a 2' 2 LL1 2 Time (101) 3 4 The police and non-police alleles are bistable. pw=0.75, p1=1. r„2=1.0244. r1.1.0667 1 0.8 0.8 0.4 0.2 1 10 15 Time (10) b to 0.8 co :9. • 0.6 o 1 0.4 8. 0.2 d 1 0.8 0.8 8 0.4 0.2 LLThe non-police allele invades and is stable. p.„2=0.75, p1=1, r„e1.0344, r,=1.0567 82 84 86 88 Time (101) The police and non-police alleles coexist. pm l3.75. p1=1, re 1.0444. r1=1.0667 OB 8 2 4 Time (104)10 Fig. 7. Numerical simulations of the evolutionary dynamics of worker policing that show the four behaviors in fig. 6(a). The policing allele is dominant. For each of the four panels. we use the initial conditions: la) Xm,0 and443-10-1; (b)X„,2 — 1 —10"1 and X„0 (C)440 0.02 and X2A.I — 0.98 (lower cum), and XAA0 —0.01 and X,,,„, —0.99 (upper cum): (d)X.„,„ — 1-10-2 and X.,," — 10-2 (lower curve), and X„,,, —1-10 2 and Xmo —10 2 (upper curie) We set re —1 without loss of generality. 1.003 1.002 1.001 1 O O 0.999 0.998 O 1/2 Fraction of poice workers, Fig a Possible rz efficiency curves for n-1 mating which demonstrate different behaviors. For this plot. we set pi.,7 0.99 and p, — 1. Here, each curve has the functional form r, — I + art/IS. For example. we can have: (blue) policing invades but is unstable, a—0.003, p--0.O004: (green) policing invades and is stable, a—0.0026,1-0: (red) policing does not invade and is unstable, a — 0.0024. p—0: (black) policing does not invade but is stable, a—0.002. 0-0.0004. (For inter- pretation of the references to color in this figure caption, the reader is referred to die web version of this paper.) n = 1 mating 3. Single mating 1 For single mating. n=1. the invasion condition for a dominant police allele is > 2(1 -1:01)+(P +P tp) p 2(2 — rip) (Recall that r0 = 1.) (3) The stability condition for a dominant police allele is 6 —pia +3pt r1 > (4) 6+2p1 Evolution of policing is highly sensitive to changes in colony effi- ciency. For example, let us consider p in = 0.99 and pi =1. This means that if half of all workers police then 99% of all males come from the queen. If all workers police then all males come from the queen. In this case, efficiency values such as r112 = 1.001 and rt = t0031 lead to the evolution of policing. In principle, arbitrarily small increases in colony efficiency can lead to the evolution of policing for single mating. A plot of rt versus r112 for singly mated queens (Fig. 6) illus- trates the rich behavior highlighted in Fig. 5. Numerical simula- tions of the evolutionary dynamics are shown in Fig. 7. Another intriguing feature is that increases in colony efficiency due to policing do not necessarily result in a higher frequency of police workers at equilibrium Fig. 8 illustrates this phenomenon. Four possibilities for the efficiency function rz are shown. Notice that the r, curve which results in coexistence of police workers and non- police workers (blue, top) is strictly greater than the rz curve which results in all workers policing (green, second from top). How can increased efficiency due to policing possibly result in policing being less abundant at equilibrium? If a mutation for non-policing behavior is introduced into a resident policing population, then the evolu- tionary success of the non-policing mutation depends on the success of M. 0 colonies relative to aa. 1, aa.0, Aa, 1, and AA, 1 colonies. Aa. 0 colonies have an efficiency parameter r112. while the other four relevant colonies each have an efficiency parameter rt. Thus, if r112 is too large relative to rt, then the non-police allele can invade a resi- dent policing population, and there is coexistence. Also notice that the rz curve which results in bistability of police workers and non-police workers (black, bottom) is strictly less than EFTA01071002 J.W. OleJarz er al. /Journal of Theoretical Biology 399 (2016) 103-116 n = 2 matings 109 n=3 matings 1.003 1.003 POIleng InvedeS but IS Pa cog yaks INA is unstable °Datable Poems' 1.002 1.002 POrcna ades and is stable Porteng does not invade and is unstable invades and is stable - P011eng We, nOlInvaile BM unSbablePoIrceig does but PcI cog does not invade txt 4 Stable not invade is stable 1.001 ro 1.001 1 1 g. O 8 0.999 0 00.999 0.998 0.998 0 1/2 3/4 1 0 1)3 112 (213) 516 1 Fraction of police workers, z Fig. 9. Possible r, efficiency curves for n-2 matings which demonstrate different behaviors. Here, each curve has the functional form r, — az+rs.z2. For example, we can have: (blue) poking invades but is unstable. a-0.0005. p--0.0004: (green) policing invades and is stable. a-0.0001, p-0; (red) policing does not invade and is unstable, a — 0.0001, 0-0; (black) policing does not invade but is stable, a— -0.0005.p-0.0004. (For interpretation of the references to color in this figure caption, the reader is referred to the web version of this paper.) the r, curve which results in policing being dominated by non-policing (red, second from bottom). This phenomenon arises in a similar way: if ri ;2 is too small relative tor,, then the non-police allele cannot invade a resident policing population, and there is bistability. 4. Double mating For double mating. n=2. the invasion condition for a dominant police allele is given by ria >1 (5) Thus, policing can invade if there is an infinitesimal increase in colony efficiency when half of all workers police. Policing cannot invade if there is an infinitesimal decrease in colony efficiency when half of all workers police. The stability condition for policing is given by rl > F3/4 (6) Therefore, the policing allele is stable if the colony efficiency is greater for z=1 (when all workers police) than for z= 3/4 (when only three quarters of the workers police). Four possible efficiency curves r, and the corresponding behavior of the police allele are shown in Fig. 9. 5. Triple mating For triple mating n=3. the invasion condition for a dominant police allele is given by 4-20 -PO115 rig> (7) 2+(Pia+Ptp) ,113 The stability condition for policing is given by 10+Josio+Spiri > F5/610+6pi(8) As a numerical example, let us consider pv3=0.98 and p112 =0.99. If z= 1/3 of workers police, then 98% of males come from the queen. If z = 1/2 of workers police, then 99% of males come from the queen. In this case, policing cannot invade if ri/3 = 0.9990 and ria =0.9979. In principle, arbitrarily small reduc- tions in colony efficiency can prevent evolution of policing for triple mating. Fraction of police workers, z Fig. 10. Possible rt efficiency curves for n-3 matings which demonstrate different behaviors. For this plot we set pl.., — 0.986, p,, a — 0.99, p, — 0.996, and pi —1. Here, each curve has the functional form r, -1 + a2 pz2. For example. we can have: (blue) policing invades but is unstable. a— —0.0006. p- -0.0006: (green) policing invades and is stable, a--0.0012, p-0; (red) policing does not invade and is unstable, a - 0.0015, 0-0; (black) policing does not invade but is stable, a- -0.0021, /1-0.0006. Note that the value r2,., affects the population dynamics but does not appear in the conditions for invasion and stability of the police allele, hence the parentheses on the horizontal axis. (For interpretation of the references to color in this figure caption, the reader is referred to the web version of this paper.) Just as for single mating we observe the intriguing feature that increases in colony efficiency due to policing do not necessarily result in a higher frequency of police workers at equilibrium. Fig. 10 illus- trates this phenomenon. Four possibilities for the efficiency function r, are shown. Notice that the r, curve which results in coexistence of police workers and non-police workers (blue, top) is strictly greater than the r, curve which results in all workers policing (green, second from top). Also notice that the r, curve which results in bistability of police workers and non-police workers (black, bottom) is strictly less than the r, curve which results in policing being dominated by non- policing (red, second from bottom). 6. Recessive police allele We have also derived the conditions for the emergence and evo- lutionary stability of worker policing if the police allele is fully reces- sive. In this case. M and Aa workers are phenotypically identical and do not police, while aa workers do police. (Alternatively, M and Aa workers police with intensity 44= Zilch while aa workers police with intensity Zoo =Zvi+ W = ZA0 -1-w. We consider this case in Section 8.) 6.1. Emergence of worker policing The invasion condition for a recessive police allele, a, is given by Two> 2(2+n+np o) ro (2 +n)(2 +Po) +Pin2A)(n — 2) Note that Eq. (9) for invasion of a recessive police allele has the same mathematical form as Eq. (2) for evolutionary stability of a dominant police allele. Starting from Eq. (2), making the substitution z -.1-z. and reversing the inequality, we recover Eq. (9). The intuition behind this correspondence is described in Appendix A. 62. Stability of worker policing A recessive police allele, a. is evolutionarily stable if [2( \ 11 (1 pi ri >cfa_iv.+pia Vrn -run/ I. Via./ ' kria) 2 Note that Eq. (10) for evolutionary stability of a recessive police allele has the same mathematical form as Eq. (1) for invasion of a (9) (10) EFTA01071003 110 J.W. OkJazz et al. &Journal of theoretical Biology 399 (2016) 103-116 Single mating, n=1 b le To To 2 rr Double mating, n=2 - ru4=1.002 - ru4=1.001 - r=0.999 - r=0.998 16 20 0 8 12 16 20 Time (105 F1/4.11. Numerical simulations of the evolutionary dynamic of worker policing confirm the condition given by Eq. (9). The policing allele is recessive. For numerically probing invasion, we use the initial condition XAm, - 1 -10-2 and XAki - 10-e. We set I'D 1 without loss of generality. Other parameters are: (a)p0-0.6. en -0.8, and 6-1.06: (b) Po-035. R14 —0.9. ri,2 - recra, and G -1.012. Under what conditions does worker policing invade? n a 1 Mathlg n = 2 footings as 3 matings is constant (equal to 1) NEUTRAL a decreases monotonically YES Increases monotonically NO YES YES aYES YES YES reaches a maximum for scaitt0<z< I NO NO NO YES NO YES YES YES reaches a minimum for some 0< z <1 NO YES NO NO NO kitLYESA YES YES NO NO YES NO YES NO YES YES NO NO NO YES YES YES NO YES Fig. 12. Depending on the funttional form of colony efficiency. G. on the fraction of police workers. z. policing alleles may or may not invade for single, double, of triple mating. Various possibilities of r, are shown. The outcomes hold for both dominant and recessive police alleles. If G is constant, then policing does not invade for single mating, is neutral for double mating and invades for triple mating. If rz decreases monotonically, then policing does not invade of invades only for triple mating, If i, increases monotonically. then policing either invades only for double and triple mating or for single, double, and tnpk mating. If G reaches a maximum at an intermediate value 0 <z < 1, then policing does not invade of may invade for tnpk mating only. for double and triple mating or for single, double, and triple mating, If rz reaches a minimum at an intermediate value 0 <z < 1. then any pattern is possible. dominant police allele. Starting from Eq. (1). making the sub- stitution z-.1-z. and reversing the inequality, we recover Eq. , 10 '. Again, the intuition behind this correspondence is described in Appendix A. Numerical simulations of the evolutionary dynamics with a recessive police allele are shown in Fig. 11. 7. Shape of the efficiency function, rj, The shape of the efficiency function. rz. determines whether policing is more likely to evolve for single mating or multiple matings. Recall that r, is the colony efficiency (defined as the rate of generation of reproductives) if a fraction, z. of all workers per- form policing. The variable z can assume values between 0 and 1. If no workers police. z=0. then the colony efficiency is at baseline. which we set to one: therefore, we have ro =1. Policing can in principle increase or decrease colony efficiency (Fig. 12). We have the following results regarding the invasion and sta- bility of police workers. We discuss single (n=1). double (n=2). and triple (n=3) mating. All results apply to both dominant and recessive police alleles. They can be instantiated with arbitrarily small changes in colony efficiency. 7.1. Evotutionary invasion of policing (i) If r2 is strictly constant (which is ungeneric), then policing does not invade for single mating, is neutral for double mating, and does invade for triple mating. (ii) If rz is monotonically decreasing, then policing either invades not at all or only for triple mating. (iii) If rz is monotonically increasing, then policing either invades for single, double, and triple mating or only for double and triple mating. (iv) If rz reaches an intermediate maximum (which means colony efficiency is highest for an intermediate fraction of police EFTA01071004 Olefarz er al. /Journal of Theoretkaf Biology 399 (2016) 103-116 a 1.012 1.010 1.088 1.006 1.004 _g 1.002 8 0.998 0 Policing invades for n=1 but not for n=2 - • • • 1/2 3/4 Fraction of police workers.: Policing invades but is unstable (for n=1 and n=2) 1 b g. 0.999 1 0.998 6 0.997 d 0.996 Policing does not evade but is stable (for n=1 and n=2) 0 1/2 3/4 Fraction of police workers.: Policing does not invade but is stable (fix n=1) Policing invades but is unstable (for n=2) 1.002 • • 1.003 C • g .21.002 € 1.001 0 1.001 1 - • 0 1/2 3/4 Fraction of police workers.: 0 1/2 314 Fraction of police workers,: Fig. 13. Non-monotonic efficiency funcnons can lead to rich and counterintuitive behavior. We consider invasion and stability of a dominant police allele for single (n-1) and double (n-2) mating. The baseline colony efficiency without policing is ro — 1.Three other values must be specified: r,„2, 44. and r1. Moreover, we need to specify two values for how the presence of police workers affects the fraction of male offspring coming from the queen: we choose oil —0.99 and p, — I. A variety of behaviors can be realized by a very small variation in colony efficiency. (a) Policing invades for single mating but not for double mating. (b) Policing does not invade but is stable for single and double mating. (c) Policing invades but is unstable for single and double mating. (d) Policing does not invade but is stable for single mating, while policing invades but is unstable for double mating. workers), then policing can invade for n = 1,2,3 or n=2.3 or n=3 or not at all. (v) If rz reaches an intermediate minimum (which means colony efficiency is lowest for an intermediate fraction of police workers), then policing can invade with any pattern of mat- ings. For example, it is possible that policing invades only for single mating but neither for double nor triple mating. Or it invades for single and double mating but not for triple mating. 72. Evolutionary stability of policing (i) If rz is constant, then policing is unstable for single mating, is neutral for double mating, and is stable for triple mating. (ii) If rz is monotonically decreasing, then policing is unstable for single and double mating. For triple mating it can be stable or unstable. (iii) If rz is monotonically increasing, then policing either is always stable or is stable only for double and triple mating. (iv) If rz reaches an intermediate maximum, then policing can be stable for any pattern of matings. For example, policing can be stable for single mating but neither for double nor triple mating. (v) If rz reaches an intermediate minimum, then policing can be stable for n = 1.2.3 or n=2,3 or n=3 or not at all. 7.3. Examples for single and double mating Fig. 13 gives some interesting examples for how non- monotonic efficiency functions can influence the evolution of policing for single (n=1) and double (n=2) mating. In order to discuss the invasion and stability of a dominant police allele for single and double mating, we need to specify efficiency at three discrete values for the fraction of police workers present in a colony: ria. r314. and r1. Note that ro =1 is the baseline. Moreover. we need to specify the fraction of male offspring coming from the queen at two values: po and pi. For all examples in Fig. 13, we assume pla =0.99 and pi = 1. We show four cases: (a) policing invades for single mating but not for double mating: (b) for both single and double mating. policing does not invade but is stable: (c) for both single and double mating, policing invades but is unstable (leading to coexistence of policing and non-policing alleles): (d) policing does not invade but is stable for single mat- ing: policing invades but is unstable for double mating. These cases demonstrate the rich behavior of the system, which goes beyond the simple view that multiple matings are always favor- able for the evolution of policing. 8. Gradual evolution of worker policing Our main calculation applies to mutations of any effect size. In this section, we calculate the limit of incremental mutation (small mutational effect size). Our calculations in this section are remi- niscent of adaptive dynamics (Nowak and Sigmund, 1990; Hof- bauer and Sigmund, 1990; Dieckmann and Law, 1996; Metz et al., 1996; Geritz et al., 1998), which is usually formulated for asexual and haploid models. The analysis in this section applies both to the case of small phenotypic effect and to the case of weak penetrance. Mathematically, we consider the evolutionary dynamics of poli- cing if the phenotypic mutations induced by the a allele are smalL If an allele affecting intensity of policing is dominant, then it is intui- tive to think of wild-type workers as policing with intensity 4.,, EFTA01071005 112 Okjorz et al /journal of theorerkol Biology 399 (2016) 10-116 while mutant workers police with intensity Im =Zoo =ZoAl-w. If an allele affecting intensity of policing is recessive, then it is intuitive to think of wild-type workers as policing with intensity ZM =140, while mutant workers police with intensity 1,0 = ZAA = ZA0+ w. In the limit of incremental mutation, the fraction, p. of queen- derived males and the colony efficiency. r, become functions of the average intensity of policing in the colony, which is Z+wz, where z is the fraction of mutant workers in the colony. We have -. p(Z+ wz)= P(z)+PV)wz+1 PV)w22+0(w3) r,-.R(Zl-wz)= Rail+ K(Z)wz +1 R.(Z)w2z2 +O(w3) (11) We have made the substitutions p,—.P(Zl-wz) and r1—.5t(Z+wz). and (11) gives the Taylor expansions of these quantities in terms of their first and second derivatives at intensity 1 (For conciseness, we will often omit the argument Z from the functions P and R and their derivatives.) Here. I wi 41. so that workers with the phenotype corresponding to the mutant allele only have an incremental effect on colony dynamics. Thus, the expansions (11) are accurate approximations. We assume that P > 0. The sign of w can be posi- tive or negative. If w is positive, then the mutant allele's effect is to increase the intensity of policing. If w is negative, then the mutant allele's effect is to decrease the intensity of policing. Note that this formalism could also be interpreted as describing the case of weak penetrance, in which only a small fraction of all workers that have the mutant genotype express the mutant phenotype. For considering the dynamics of a dominant police allele with weak phenotypic mutation, we introduce the quantity Qom= Pun +Pi a 01 Oa) [2_ trial _ _Pim) (.11. )1 (12) 2 ro 11/4 re / ro I f ro If Corm > 0. then increased intensity of policing is selected, and if Com <0. then increased intensity of policing is not selected. This is just a different way of writing (1). We substitute (11) into (12) and collect powers of w. To first order in w we get Cdom = Wr-2)KRA-2(2+n+nP)R1 4nR +O(w2) (13) For considering the dynamics of a recessive police allele with weak phenotypic mutation, we introduce the quantity Aram 2(2-i-nl-np o) Crec — ro (2+n)(2 +Po)+Pita,o(n —2) (14) If Crec > a then increased intensity of policing is selected, and if Crec < 0, then increased intensity of policing is not selected. This is just a different way of writing (9). We substitute (11) into (14) and collect powers of w To first order in w. we get Cre, — —2)PRA-2(2+n-EnP)K1 4nR(2+n+nP) +O0412) (15) Notice that (13) and (15) are, up to a multiplicative factor, the same to first order in w Usin