This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).

The predominant analytical approach to associate landscape patterns with gene flow processes is based on the association of cost distances with genetic distances between individuals. Mantel and partial Mantel tests have been the dominant statistical tools used to correlate cost distances and genetic distances in landscape genetics. However, the inherent high correlation among alternative resistance models results in a high risk of spurious correlations using simple Mantel tests. Several refinements, including causal modeling, have been developed to reduce the risk of affirming spurious correlations and to assist model selection. However, the evaluation of these approaches has been incomplete in several respects. To demonstrate the general reliability of the causal modeling approach with Mantel tests, it must be shown to be able to correctly identify a wide range of landscape resistance models as the correct drivers relative to alternative hypotheses. The objectives of this study were to (1) evaluate the effectiveness of the originally published causal modeling framework to support the correct model and reject alternative hypotheses of isolation by distance and isolation by barriers and to (2) evaluate the effectiveness of causal modeling involving direct competition of all hypotheses to support the correct model and reject all alternative landscape resistance models. We found that partial Mantel tests have very low Type II error rates, but elevated Type I error rates. This leads to frequent identification of support for spurious correlations between alternative resistance hypotheses and genetic distance, independent of the true resistance model. The frequency in which this occurs is directly related to the degree of correlation between true and alternative resistance models. We propose an improvement based on the relative support of the causal modeling diagnostic tests.

Landscape genetics provides a powerful approach to evaluate the effects of multiple landscape features on population connectivity [

The predominant analytical approach to associate landscape patterns with gene flow processes is based on pair-wise calculation of cost distances, using least cost paths (e.g., [

There has been controversy in the literature about the appropriateness of Mantel testing in landscape genetics. Raufaste and Rousset [

Recently, Guillot and Rousset [

Cushman

Two additional questions need to be evaluated to determine the overall reliability of the causal modeling approach using partial Mantel tests in landscape genetics. First, to demonstrate the general reliability of the Cushman and Landguth [

The objectives of this study were to (1) evaluate the effectiveness of the Cushman

Schematic describing the two different approaches to causal modeling with partial Mantel tests used in this paper. (

We chose a real landscape in northern Idaho, USA (

Map of study area, which contains 4,500 square kilometers encompassing the extreme northern part of the Idaho panhandle and adjacent areas of Washington, Montana and British Columbia.

List and description of the 35 resistance models evaluated in the present study. The models were a combination of the effects of elevation, forest cover and roads on resistance to gene flow (for details see [

Model Acronym | Model Description |
---|---|

EH | Minimum resistance at high elevations (1,500 m) |

EHFH | Minimum resistance in forest (strong) at high elevations |

EHFL | Minimum resistance in forest (weak) at high elevations |

EHRH | Minimum resistance at high elevations with high resistance of roads |

EHRL | Minimum resistance at high elevations with weak resistance of roads |

EL | Minimum resistance at low elevations (500 m) |

ELFH | Minimum resistance at in forest (strong) at low elevations |

ELFL | Minimum resistance in forest (weak) at low elevations |

ELRH | Minimum resistance at low elevations with high resistance of roads |

ELRL | Minimum resistance at low elevations with weak resistance of roads |

EM | Minimum resistance at middle elevations (1,000 m) |

EMFH | Minimum resistance in forest (strong) at middle elevations |

EMFL | Minimum resistance in forest (weak) at middle elevations |

EMRH | Minimum resistance at middle elevations with high resistance of roads |

EMRL | Minimum resistance at middle elevations with weak resistance of roads |

FH | Minimum resistance in forest (strong) |

FHEHRH | Minimum resistance in forest (strong) at high elevations with high resistance of roads |

FHEHRL | Minimum resistance in forest (strong) at high elevations with weak resistance of roads |

FHELRH | Minimum resistance in forest (strong) at low elevations with high resistance of roads |

FHELRL | Minimum resistance in forest (strong) at low elevations with weak resistance of roads |

FHEMRH | Minimum resistance in forest (strong) at middle elevations with high resistance of roads |

FHEMRL | Minimum resistance in forest (strong) at middle elevations with weak resistance of roads |

FHRH | Minimum resistance in forest (strong) with high resistance of roads |

FHRL | Minimum resistance in forest (strong) with low resistance of roads |

FL | Minimum resistance in forest (weak) |

FLEHRH | Minimum resistance in forest (weak) at high elevations with high resistance of roads |

FLEHRL | Minimum resistance in forest (weak) at high elevations with weak resistance of roads |

FLELRH | Minimum resistance in forest (weak) at low elevations with high resistance of roads |

FLELRL | Minimum resistance in forest (weak) at low elevations with weak resistance of roads |

FLEMRH | Minimum resistance in forest (weak) at middle elevations with high resistance of roads |

FLEMRL | Minimum resistance in forest (weak) at middle elevations with weak resistance of roads |

FLRH | Minimum resistance in forest (weak) with high resistance of roads |

FLRL | Minimum resistance in forest (weak) with low resistance of roads |

RH | Strong resistance of roads |

RL | Weak resistance of roads |

We used CDPOP version 0.84 [

In each of the 37 alternative landscape models, we placed 1,248 individuals in a uniform grid at a 2 km spacing within forested cover (

Example of one resistance model (minimum resistance in forest (strong) at middle elevations with high resistance of roads (FHEMRH),

CDPOP calculated a matrix of pair-wise genetic distances between all 1,248 simulated individuals based on the proportion of shared alleles (_{PS}

Following Cushman

For each of the 35 alternative landscape resistance hypotheses, we calculated four partial Mantel tests to assess the degree of association between each genetic distance matrix and landscape distance matrix, partialling out the effect of an alternative landscape distance matrix (

The four partial Mantel tests used in the causal modeling framework to assess the degree of association between each genetic distance matrix and three cost distance matrices, representing the two null models (Isolation by Distance, Isolation by Barrier), and the correct landscape resistance model. The expected outcomes are for the situation where the landscape resistance model is a true driver of the observed genetic differentiation.

Test Number | Dependent Variable | Independent Variable | Covariate | Expected Outcome |
---|---|---|---|---|

1 | Genetic Distance | Landscape Resistance Model Cost Distance | Isolation by Distance | Significant |

2 | Genetic Distance | Landscape Resistance Model Cost Distance | Isolation by Barrier | Significant |

3 | Genetic Distance | Isolation by Distance | Landscape.Resistance Model Cost Distance | Not Significant |

4 | Genetic Distance | Isolation by Barrier | Landscape.Resistance.Model Cost Distance | Not Significant |

Wasserman

We found high correlation of the cost distances among pairs of resistance hypotheses (

Matrix of Mantel correlations between cost distances between all pairs of 1,248 source points in all pairs of resistance hypotheses. The rows and columns of the matrix represent each of the 35 resistance hypotheses (

There were four diagnostic partial Mantel tests in the Cushman

Thirty-one of the 35 alternative landscape resistance models had perfect performance on Test 3. Of the four that had less than perfect performance, all performed perfectly in over 80% of model runs. In contrast, 12 of 35 alternative resistance models had less than perfect performance in Test 4. In nine of these, the expectations of Test 4 were not met in the majority of runs, and three alternative resistance models always failed to meet the expectations of Test 4. These were models EHFH, FHEHRH and FLEHRL (

There was a strong association between the correlation of cost-distances between resistance models and the frequency with which they failed to meet Test 3 or Test 4 (

In the Wasserman

Parameters for logistic regression equations predicting whether or not each of the diagnostic partial Mantel tests fails to produce the correct results as a function of the correlation between the true resistance model and the alternative resistance model. IBD | Model: simple causal modeling diagnostic test of whether there is independent (spurious) support for isolation by distance independent of the true model. IBB | Model: simple causal modeling diagnostic test of whether there is independent (spurious) support for isolation by barrier independent of the true model. True | Alternative: causal modeling test of whether there is independent support for the true model independent of the alternative model. Alternative | True: causal modeling test of whether there is independent (spurious) support for the alternative model independent of the true model. The simple causal modeling tests, Model | IBD and Model | IBB, are not shown, as they both had 100% correct performance across all alternative resistance models and model runs (

Test Number | Logistic Regression Model | Estimate | Std. Error | Z value | Pr(>|z|) | |
---|---|---|---|---|---|---|

1 | IBD | Model | Intercept | 10.207 | 6.276 | 1.626 | 0.1039 |

DD | −17.451 | 7.855 | −2.222 | 0.0263 | ||

2 | IBB | Model | Intercept | −2.8887 | 0.3562 | −8.111 | 5.03 x 10^{-16} |

DD | 22.7708 | 3.9155 | 5.816 | 6.04 x 10^{-9} |
||

3 | True | Alternative | Intercept | −36.6 | 1.187 | −30.82 | <2 × 10^{−16} |

DD | 36.472 | 1.211 | 30.11 | <2 × 10^{−16} |
||

4 | Alternative | True | Intercept | −3.7234 | 0.1289 | −28.88 | <2 × 10^{−16} |

DD | 3.916 | 0.1471 | 26.62 | <2 × 10^{−16} |

There was the perfect ability of a resistance hypothesis to be shown to be independently supported compared to alternative resistance hypotheses when the correlation between the true and alternative resistance models was less than 0.85 (

Frequency of significant independent association between a simulated landscape resistance model and genetic distance. The rows of the matrix represent each of the 37 resistance hypotheses. The first 35 rows are the alternative landscape resistance models, with the bottom two rows representing the two null models of isolation by barrier (rd) and isolation by distance (ed). The columns represent the 35 resistance hypotheses simulated as truth in CDPOP. The color of the cell corresponds to the frequency with which the partial Mantel correlation between the model associated with a given column and genetic distance, partialling out the model associated with a given row, is statistically significant (alpha = 0.05). Cells in blue have a very high frequency of correctly finding independent correlation between the simulated resistance model and genetic distance, while red cells have a high frequency of failing to find significant correlation between the true resistance model and genetic distance, partialling out the model associated with that row of the matrix.

Frequency of significant spurious association between an alternative resistance model and genetic distance, independent of the simulated landscape resistance model. The rows of the matrix represent each of the 37 landscape models. The first 35 rows are the alternative landscape resistance models, with the bottom two rows representing the two null models of isolation by barrier (rd) and isolation by distance (ed). The columns represent the 35 resistance hypotheses simulated as being true in CDPOP. The color of the cell corresponds to the frequency with which the partial Mantel correlation between the model associated with a given row and genetic distance, partialling out the model associated with a given column, is statistically significant (alpha = 0.05). Cells in blue have a very high frequency of correctly finding independent correlation between the simulated resistance model and genetic distance, while red cells have a high frequency of failing to find significant correlation between the true resistance model and genetic distance, partialling out the model associated with that row of the matrix.

Binary scatterplots of the frequency of (

Binary scatterplots of the frequency of failing (

As expected, the Cushman

In the Wasserman

Reliable inferences regarding the effects of landscape features on gene flow and population connectivity depend on analytical methods that have high power to correctly identify the driving process and reject spurious, correlated alternatives. Our results indicate that partial Mantel tests in a causal modeling framework have high power to do the former, but have a relatively weak ability to accomplish the latter. There is often a tradeoff between Type I and Type II error rates in statistical analysis. Our results show that partial Mantel tests in an individual-based, causal modeling framework have low Type II error rates (extremely high power to detect a relationship). The Cushman

The elevated Type I error rates reported here have several effects on the interpretation of the results of Mantel and partial Mantel tests in landscape genetics. First, as argued by Cushman and Landguth [

We can use this knowledge to provide guidance to interpret the outcomes of the diagnostic partial Mantel tests. First, when one finds that a particular resistance hypothesis is supported independently of alternative models using causal modeling with partial Mantel tests, this is likely to be correct, given that the elevated Type I error is a bias in the opposite direction. Second, when one finds that the resistance hypothesis is not supported independently of the alternative model, but the alternative model is supported independently of the resistance model, this suggests that the proposed resistance model is incorrect and that gene flow could be either governed by the alternative model or another resistance model not tested. Third, when the proposed resistance model is not significantly supported independently of the alternative model and the alternative model is not supported independently of the proposed model, this suggests that gene flow is not governed by either the proposed resistance model or the alternative model. In such cases, genetic structure may be influenced by a third untested resistance hypothesis. The final potential outcome is when the proposed landscape resistance model is supported independently of the alternative model and the alternative model is supported independently of the resistance model. This is the case most commonly seen in the present analysis, due to elevated Type I error rates leading to failure to correctly reject spurious correlations. In this case, it is impossible to determine, using causal modeling, if one of the two models is correct and the other spurious or if gene flow is governed by an untested third model that is correlated with the two.

Given the very high correlation among resistance models (average of over 0.84 in the present study), it is not surprising that causal modeling had less than perfect performance. Given that landscape resistance models are models of cumulative cost-over-distance, it is likely that most alternative models will be highly correlated [

Another way of improving the implementation of causal modeling with partial Mantel tests is to use the relative support, rather than formal rejection thresholds. For example, instead of relying on formal probabilistic statistical hypothesis testing, we propose evaluating the relative support for each of the diagnostic causal modeling tests. In the case of the Wasserman

We found that partial Mantel tests have very low Type II error rates, but elevated Type I error rates. This leads to frequent identification of support for spurious correlations between alternative resistance hypotheses and genetic distance, independent of the true resistance model. The frequency in which this occurs is positively related to the degree of correlation between true and alternative resistance models. We propose an improvement based on the relative support of the causal modeling diagnostic tests. We show that using the difference between the support among alternative models improves the performance of causal modeling. Specifically, it did not reduce the power of the approach to identify the correct driver, and simultaneously, it decreased the chance of Type I errors in which incorrect alternative models that are highly correlated with the true driving process are spuriously affirmed. Overall, the present study indicates that causal modeling with partial Mantel tests is a large improvement over simple Mantel testing, but that elevated Type I error rates associated with Mantel testing still need to be addressed. The calculation of relative support among a full combination of alternative hypotheses appears to be a robust way of reducing Type I error rates in Mantel testing in individual-based landscape genetics. The large improvement of performance using this method appears to at least partially alleviate the reported bias in partial Mantel tests relating to autocorrelation [

This work was supported by the USDS Forest Service Rocky Mountain Research Station.

Effects of changing the alpha level on the success rate of causal modeling across the 35 alternative resistance models simulated. Proportion Model | Null: proportion of the resistance models significantly supported independent of the two null models of isolation by distance and isolation by barriers, using the one-step form of causal modeling proposed by [

Model | | | Model | |||||
---|---|---|---|---|---|---|

Alpha | Proportion Model | Null Sig (one-step) | Proportion Model | Alt. Models Sig. (two-step) | Proportion Model | Alt. Models Sig. (all-pairs) | Proportion Null | Model Sig (one-step) | Proportion Alt. Models | Model Sig. (two-step) | Proportion Alt. Models | Model Sig. (all-pairs) |

0.05 | 1 | 0.828571 | 0.101587 | 0.428571 | 0 | 0.506349 |

0.04 | 1 | 0.828571 | 0.111905 | 0.4 | 0.028571 | 0.512698 |

0.03 | 1 | 0.828571 | 0.114286 | 0.371429 | 0.028571 | 0.52381 |

0.02 | 1 | 0.828571 | 0.119841 | 0.342857 | 0.028571 | 0.536508 |

0.01 | 1 | 0.828571 | 0.13254 | 0.342857 | 0.028571 | 0.554762 |

0.009 | 1 | 0.828571 | 0.13254 | 0.342857 | 0.028571 | 0.554762 |

0.008 | 1 | 0.828571 | 0.13254 | 0.342857 | 0.028571 | 0.554762 |

0.007 | 1 | 0.828571 | 0.13254 | 0.342857 | 0.028571 | 0.554762 |

0.006 | 1 | 0.828571 | 0.13254 | 0.342857 | 0.028571 | 0.554762 |

0.005 | 1 | 0.828571 | 0.13254 | 0.342857 | 0.028571 | 0.554762 |

Difference of support for Test 1 and Test 2 in the two-step form of causal modeling (Wasserman, 2010). Rows represent the alternative models and columns the correct, simulated resistance model. The color indicates the difference in partial Mantel