Inbreeding occurs when individuals mate with their relatives. Depending on how closely related the parents are, the resulting inbred offspring will have a moderate to large proportion of alleles that are identical by descent, in other words they will exhibit a genome-wide increase in homozygosity relative to outbred individuals. This often leads to a reduction in fitness through a phenomenon that is known as inbreeding depression. Two processes can lead to inbreeding depression: dominance and overdominance. Dominance refers to the unmasking of deleterious recessive alleles that accompanies the overall increase in homozygosity; this occurs when unfavourable alleles that formerly occurred primarily in heterozygous individuals become more prevalent in a homozygous state, and therefore their deleterious effects are manifested. Overdominance, also known as heterozygote advantage, means that individuals that are heterozygous at a particular locus have higher fitness than individuals that are homozygous for either allele; the general increase in homozygosity that accompanies inbreeding means that beneficial heterozygotes become less common, once again reducing fitness.

Quantifying genome-wide heterozygosity is impractical, and is therefore typically inferred from a subset of loci such as microsatellites (e.g., [33]). Similarly, there are logistical constraints to quantifying fitness based on lifetime reproductive success, and therefore one or more surrogate measures such as clutch size, sperm count, or seed production is most commonly used (e.g., [34]). Multilocus genotype data and fitness estimates can be combined to test for heterozygosity fitness correlations (HFCs), which occur when there is a correlation between overall heterozygosity and a measure of fitness; a positive HFC suggests that low heterozygosity is reducing fitness within a population. Although a correlation between heterozygosity and fitness is widely accepted as evidence of inbreeding depression (reviewed in [35]), others have argued that HFCs that are based on only a small number of neutral markers may not reflect inbreeding depression because they are unlikely to represent genome-wide changes in homozygosity [36]. This was recently illustrated by a study of a free-ranging pedigreed population of the endangered takahe (Porphyrio hochstetteri) in which even relatively large numbers (>20) of microsatellite loci provided imprecise estimates of individual genome-wide heterozygosity [37]. The shortcomings of inbreeding estimates based on HFCs may therefore be twofold: first, it may be inappropriate to extrapolate genome-wide estimates of heterozygosity from small numbers of loci, and second, such extrapolation may be further weakened by the fact that heterozygosity is typically calculated on a subset of alleles that are neutral, and that have no functional significance in terms of adaptation and fitness (but see [38]).

An alternative approach to studying inbreeding depression is to seek specific information about its underlying molecular basis [39]. The first whole-genome study on the relationship between inbreeding and gene expression was done on Drosophila melanogaster [40]. The authors of that study compared gene expression in inbred and outbred lines of D. melanogaster, and determined that inbreeding changes transcription levels for a number of genes. The genes that showed differential expression in inbred lines were disproportionately involved in metabolism and stress responses, for example heat shock protein genes, which are involved in stress response, were upregulated more (i.e., expressed in greater amounts) in inbred flies. This suggests that inbreeding acts like an environmental stressor that confers metabolic costs, and therefore leaves less energy for reproduction; in other words, inbreeding reduces fitness because stress responses are using energy that would otherwise be allocated to reproduction. This effect was even more pronounced when flies were placed in a high temperature environment, which conferred even greater stress and had the effect of further increasing the differential expression of heat-shock protein and metabolism genes in inbred versus outbred flies [41]. This latter study supports the idea that inbred organisms will be particularly challenged in stressful environments, and is consistent with an earlier study which found that inbreeding depression is on average 6.9 times higher for mammals in the wild compared to mammals that are kept in the relatively stress-free confines of captivity [42].

Demontis et al. [43] extended the study of gene expression in inbred Drosophila by investigating 40 SNPs in coding regions of genes that were identified in the earlier studies as being differentially expressed in inbred and outbred lines. They compared fast inbred lines, which took one generation to reach a predefined level of inbreeding, with slow inbred lines, which took 19 generations to reach the same level of inbreeding. Specifically, they wished to test the hypothesis that slow inbreeding leads to lower levels of inbreeding depression compared to fast inbreeding, because the former may allow more efficient purging of deleterious alleles and/or or more efficient selection for heterozygotes. They found a significantly higher level of genetic variation in the slow inbred lines compared to the fast inbred lines, including fewer homozygotes, and concluded that higher genetic diversity in slow inbred lines is a result of more efficient selection for heterozygotes (balancing selection) compared to the fast inbred lines. This indicates that, at least in this case, overdominance is likely the primary mechanism of inbreeding depression. Studies such as these strongly suggest that the use of “omic” approaches (e.g., genomics, proteomics) to unravel some of the cellular mechanisms behind inbreeding depression will feature more prominently in the near future [44].

Populations with low levels of genetic diversity should be less able to adapt to novel selection pressures, because a limited gene pool should decrease the likelihood that adaptive alleles will be present within a population. This expectation has been upheld by a growing number of studies. For example, populations of Mercurialis annua with reduced genetic diversity following range expansion had a reduced ability to respond to natural selection on a key life history trait [45]. In another example, laboratory populations of an estuarine crustacean (Americamysis bahia) with low genetic diversity had reduced fitness compared to populations with high genetic diversity; under stressful conditions the majority of low diversity populations went extinct, whereas populations with high genetic diversity were able to survive, albeit with reduced population sizes and less frequent reproduction [46].

The genetic diversity within populations is influenced by a range of factors, the most important of which is effective population size (N_e), a measure that was introduced in the 1930s by Sewall Wright [47,48] who defined it as “the number of breeding individuals in an idealized population that would show the same amount of dispersion of allele frequencies under random genetic drift or the same amount of inbreeding as the population under consideration” [48]. In other words, the N_e of a population reflects the rate at which genetic diversity will be lost following genetic drift: only in an ideal population (sensu Wright) will the loss of genetic diversity as a result of drift occur at a rate that is commensurate with its actual population size. Understanding N_e is relevant to predictions about the viability of populations, because populations with low N_e are expected to have little evolutionary potential, and hence may be unable to respond to changing environmental conditions. However, this leaves us with a conundrum: estimates of N_e that are derived from molecular genetic data must be based on neutral markers (most commonly microsatellites) because N_e reflects the rate at which genetic drift—not selection—is altering allele frequencies from one generation to the next. As a result, N_e may tell us little about adaptive potential. This was recently illustrated by a study of the evolution of pesticide resistance in populations of the fruitfly Drosophila melanogaster, which concluded that resistance alleles have evolved quickly and repeatedly within multiple populations [49]. The authors of that study argue that such extensive evolutionary change would require a substantially larger (>100-fold) effective population size than had previously been identified. They further suggest that this discrepancy arises from the fact that estimates of N_e are usually derived from levels of standing variation which in turn is influenced by long-term population dynamics, whereas short-term effective population sizes are more relevant for rapid adaptation, and these may be much closer to N_c.

To date, most studies that have managed to quantify adaptively important genetic diversity have been based on three gene families whose diversity is maintained by balancing selection: major histocompatibility complex (MHC) loci in vertebrates [50], self-incompatibility loci in plants [51], and sex loci in Hymenoptera [52]. However, these families collectively represent only a modest proportion of all adaptive genetic variation. Furthermore, it is not entirely clear what impacts the loss of diversity at these loci may have on the survival of populations, because the interactions between selection and drift have often resulted in correlations between levels of MHC variation and variation at neutral loci [53]. An understanding of the link between neutral and adaptive diversity, and their collective influence on long-term survival, must therefore be based on a larger number of adaptive genes. There are a number of different methods that can be used to identify these genes, some of which are outlined in Box 1. Depending on the methods used, researchers may be able to identify genes that appear to be under selection (candidate genes) on the basis of allele frequency distributions. An example of this was reported in a study of threespine stickleback (Gasterosteus aculeatus), in which the authors used next generation sequencing to genotype 100 fish from each of three freshwater and two oceanic populations at 45,000 single nucleotide polymorphisms (SNPs) [54]. The population genetic signal from neutral markers indicated that a panmictic oceanic population gave rise to freshwater populations multiple independent times, while outlier loci provided evidence that balancing and divergent selection occurred in parallel genomic regions in different freshwater populations with independent origins. A number of candidate genes involved in differentiation were identified, providing the basis for further studies of adaptation at these loci.

The search for a link between adaptive genes and evolutionary potential has been complicated in recent years by the growing awareness that gene expression can play a role in the adaptive divergence of populations. Gene expression is influenced by both genetic and environmental factors, with the relevant genetic factors being changes in either regulatory genes or cis-regulatory regions (as opposed to protein-coding regions) of functional genes. Examples in the literature which show how gene expression can influence the adaptive divergence of populations are growing. In one study, at least 4% of the compared transcriptome significantly differed between two sympatric ecotypes of the marine snail Littorina saxatilis. One of the identified transcripts was cytochrome c oxidase subunit I (COI), a mitochondrial gene involved in energy metabolism. This gene was overexpressed in the lower shore ecotype which is subject to the strongest wave action, and which therefore may need a particularly effective energy supply [55]. In another study, this time on the model species Drosophila melanogaster, population differentiation of gene expression (measure as Q_ST, or quantitative trait variation; see [56]) was not correlated with G_ST (an analogue of F_ST [57]) when based on all nucleotide polymorphisms; however, a correlation between Q_ST and G_ST was found when based on a more specific comparison in which G_ST was based solely on nucleotide differences in the 5′ coding regions of genes, in other words the regions that contain regulatory sequences [58].

Overall, neutral molecular markers have some clear advantages when used to estimate the genetic diversity of populations: they are relatively easy to characterize, and they can provide unbiased estimates of random processes such as genetic drift [59,60]. However, microsatellites, which are currently the most widely used markers for inferring genetic diversity [1], may not accurately reflect the genome-wide genetic diversity of natural populations [61,62], in part because a relatively small number of microsatellite loci are usually characterized. Although neutral markers will undoubtedly continue to play an important role in at least initial estimates of heterozygosity, we will likely see in the future a greater emphasis on whole genome scans, patterns of gene expression, and the functional analyses of genes [44,63]. (See also Box 1).

Migration and drift are expected to have approximately equal effects on all neutral loci, whereas the effects of selection will vary between neutral and non-neutral loci. All neutral loci may therefore show similar levels of genetic divergence among populations (once variable mutation rates have been accounted for), whereas non-neutral loci (or loci linked to non-neutral loci) are expected to show anomalous levels of divergence. These anomalous levels may be either unusually high or unusually low, depending on the type of selection that the relevant genes have been subjected to; for example, directional selection will increase population differentiation if different alleles are selected for in different populations, whereas balancing selection may decrease population differentiation by maintaining the same suite of alleles in multiple populations. A comparison of multiple measures of population differentiation, each based on a different locus, may reveal a marker with unusual levels of differentiation; this is often referred to as an outlier, and if the marker is found within a coding region, the latter may be considered a candidate gene [81,82]. An outlier may be used to identify a genetic region that is either directly under selection, or is linked to a gene that is under selection [83,84]. Approaches for using genome scans to identify markers of potential adaptive significance are comprehensively reviewed in [85]. However, an element of caution must be introduced to this approach because differentiating between adaptive and neutral genes can be problematic in expanding populations: expansions can impact neutral allele frequencies in ways that are similar to the effects of directional selection [86]. In addition, false positives are common even with the most rigorous analytical methods [87].

The ease with which data can now be simultaneously collected for many markers means that studies of discordant genetic differentiation (i.e., the identification of outlier loci) have increased in recent years. A growing number of these studies are based on a genome scanning approach, which means that hundreds or even thousands of markers are used to sample broadly from across the genome (as opposed to a handful of microsatellite loci), and this increases the likelihood of identifying markers linked to genes that are under the influence of natural selection. A number of such studies have been based on dominant markers, specifically amplified fragment length polymorphisms (AFLPs) (e.g., [88–90]). More recently, studies have also been taking advantage of the advent of high-throughput SNP genotyping technologies which can generate data from thousands of markers [91,92] (but see [93] for a discussion of some of the challenges associated with using SNPs).

Adaptive genes can also be inferred from clinal gradients in allele frequencies, which arise when allele frequencies vary along an environmental cline in a seemingly adaptive manner. Studies that have identified such clines sometimes target specific genes that may be expected to show signatures of natural selection. One example of this was the discovery of a latitudinal gradient in Chinook salmon (Oncorhynchus tshawytscha) clock gene allele frequencies which corresponded to latitudinal variation in reproductive timing; because clock genes are known to be involved with the regulation of circadian rhythm, the authors of this study had an a priori reason to expect that such a cline may exist [94]. Another approach is to use genome scans to search for genes that may be correlated with environmental clines; as with outlier detections, these scans are increasingly based on high-throughput genotyping of hundreds or thousands of SNP markers. This formed the basis of a study of loblolly pine (Pinus taeda) sampled across its range: the frequencies of several SNPs, identified from a total of 1730 loci, corresponded with aspects of geography, temperature, growing degree-days, precipitation and aridity [95]. The authors were then able to assign putative function to a number of SNPs by using annotated orthologs from Arabidopsis. Several SNPs that were correlated with climatic variables (such as temperature and precipation) were located within abiotic stress response genes ranging from transmembrane proteins to proteins involved in sugar metabolism.

A note of caution about using clinal patterns to infer patterns of selection is that random events or processes such as founder effects, isolation by distance, or secondary contact of populations that have previously differentiated by genetic drift can create an illusion of an adaptive cline [96]. As with outliers, conclusions may be strengthened by common garden experiments or geographically independent replicates. The latter approach revealed that an insulin signalling gene, the Insulin-like Receptor (InR), had replicate latitudinal clines in allele frequencies among Drosophila melanogater populations in both Australia and North America [97]. Replicate findings also strengthened conclusions regarding parallel temperature-associated clines in SNPs which were found in Atlantic cod (Gadus morhua) populations in the eastern and western north Atlantic: in both regions, allele frequencies at temperature-associated loci were significantly correlated with the ocean temperature, whereas neutral markers showed no such correlation [98]. See also Box 2 for other approaches that can be used to infer natural selection from genetic data; these are summarized in Table 2.

As noted above, one large stumbling block in the identification of non-neutral markers has been the difficulty in accounting for complex population demography, including historical patterns of population expansion and contraction, unequal migration rates between populations, and inbreeding. A failure to account for such demographic processes can lead to either spurious signatures of selection at loci that are in fact neutral [86], or to a lack of power to detect loci under selection. Advances in the sophistication of statistical tools and models available for the analysis of molecular data are facilitating a much more intricate and comprehensive understanding of the processes that shape neutral and adaptive genetic variation. For example, currently available Bayesian approaches [114,115] and coalescence models (e.g., [116,117]) incorporate relatively realistic scenarios in which the migration rate can differ between pairs of subpopulations, and multiple historical population bottlenecks and expansions can be accounted for. Additionally, a number of software packages incorporate spatial and environmental data with genetic data to identify loci that are associated with specific environmental variables (e.g., [118]). Many of these model-based advances are reviewed in detail by [119].

Finally, when examining patterns of population differentiation at neutral versus non-neutral loci, it is important to keep in mind that gene flow will not necessarily ensure that non-adaptive genes are continually exchanged between even proximate populations. Although the differentiation of neutral markers is driven primarily by stochastic processes, whereas that of non-netural markers is driven by both selective and stochastic processes (e.g., [120]), natural selection can also influence the distribution of markers that are neither being directly selected, nor are linked to regions under selection [121]. This arises if divergent selection is sufficiently strong to promote reproductive isolation between populations. In these cases, a reproductive barrier will then create a barrier to gene flow that results in the potentially genome-wide differentiation of populations following genetic drift. This will lead to an inverse correlation between gene flow and the adaptive divergence of populations, and thus a positive association between the phenotypic divergence and neutral molecular genetic differentiation of populations following a pattern that is known as isolation by adaptation (IBA) [122,123]. In other words, populations will not only diverge at adaptive loci as a direct result of selection, but will also diverge at neutral loci as a direct result of drift, which is indirectly a result of selection via a reproductive barrier. This pattern was identified in a semi-natural experiment in which adjacent populations of sweet vernal grass (Anthoxanthum odoratum) had diverged from one another as a result of adaptation to different nutrient additions. Genetic differentiation was evident at outlier loci, and also across a wider survey of putatively neutral loci. This was interpreted as evidence that the selection pressures from varying combinations of nutrient additions in different plots was strong enough to cause reproductive isolation between populations, which in turn has led to neutral genetic differentiation as a result of genetic drift [89]. Studies such as this, which have identified candidate adaptive molecular markers, should in the future find it increasingly feasible to take the next step and characterize the phenotypic outcomes of alternative genotypes (Box 3).

There are a number of exciting avenues for future research that will allow researchers to increasingly incorporate data on adaptive genes into studies of molecular ecology. The sequencing of a greater number of genomes from non-model organisms will be among the most obvious and rapid advancement in genomics over the coming years, and this will provide opportunities for the identification of non-neutral markers in numerous and diverse species. Furthermore, cross-validation using a combination of different approaches will lead to a greater understanding of the interaction between demographic processes and selection, the interaction between selection at linked loci in the genome, and fine-scale patterns of molecular evolution. Studies of signatures of selection in the human genome have led the way in this regard (reviewed by [131]), and can serve as models for similar studies in other organisms.

There are two widespread challenges that arise in many studies that target genes that are under the influence of selection. First, studies in non-model organisms now frequently hone in on relatively broad genomic regions that are under selection, but it remains difficult to actually identify the genes (or the mutations) that are subject to selection. Increasing the density of markers in genome scans is paramount to overcoming this problem, and validating signals of selection from particular genes using multiple methods should also help. Second, once a candidate gene has been identified, it may have no known annotated function. This occurs because annotated functional genes from model organisms may not overlap with genes that are under the influence of selection in non-model organisms that are being studied in the context of ecological and evolutionary genomics. Advances in identifying the functional significance of genes subject to selection will require ongoing integration between genomics methods and functional experiments that provide mechanistic insights into molecular pathways controlled by candidate genes (reviewed by [132]). Studies of gene expression, which can be carried out using microarrays, quantitative PCR (qPCR), and comparative sequencing of the transcriptome, can also provide evidence of differential expression of candidate genes. Genome wide scans based on complete genomic data—already close to fruition in humans—will permit a much more detailed understanding of fine scale processes involved in genome evolution (reviewed by [131]). Finally, future work on adaptive genes will likely also focus on epigenetic modifications in DNA methylation and DNA-associated proteins such as histones, which can vary among individuals and populations of the same species. The heritability of some of these modifications is now widely accepted, and means that heritable variation in ecologically important phenotypic traits may be apparent even in the absence of DNA polymorphisms (see [63]). We are therefore entering a truly exciting time in molecular ecology, in which we seem poised to make numerous important discoveries about the interactions between genotypes and phenotypes in varied—and often rapidly changing—environmental conditions.