Revisiting the Contested Role of Natural Resources in Violent Conflict Risk through Machine Learning

The integrated character of the sustainable development goals in Agenda 2030, as well as research in environmental security, flag that sustainable peace requires sustainable and conflict-sensitive natural resource use. The precise relationship between the risk for violent conflict and natural resources remains contested because of the interplay with socio-economic variables. This paper aims to improve the understanding of natural resources’ role in the risk of violent conflicts by accounting for complex interactions with socio-economic conditions. Conflict data was analysed with machine learning techniques, which can account for complex patterns, such as variable interactions. More commonly used logistic regression models are compared with neural network models and random forest models. The results indicate that a country’s natural resource features are important predictors of its risk for violent conflict and that they interact with socio-economic conditions. Based on these empirical results and the existing literature, we interpret that natural resources can be root causes of violent intrastate conflict, and that signals from natural resources leading to conflict risk are reflected in and influenced by interacting socio-economic conditions. More specifically, the results show that variables such as access to water and food security are important predictors of conflict, while resource rents and oil and ore exports are relatively less important than other natural resource variables, contrasting what prior research has suggested. Given the potential of natural resource features to act as an early warning for violent conflict, we argue that natural resources should be included in conflict risk models for conflict prevention.


Introduction
Both peace and sustainable natural resource management are integral parts of the interlinked Sustainable Development Goals of Agenda 2030 [1]. Concerns about the effect of unsustainable natural resource use on violent conflict have been expressed by the field of environmental security since the 1990s. Homer-Dixon [2] demonstrated in the early 1990s with six case studies how natural resource scarcities lead to violent conflict and thereby opened the scientific discussion. He concluded that violence due to natural resource scarcity 'will usually be sub-national, persistent, and diffuse' [2] (p. 6). Sachs and Warner [3] statistically showed that natural resource abundance hinders development and termed this process the 'resource curse'. Later, Lujala [4] showed that resource-rich countries experienced higher risks for violent conflict. Collier and Hoeffler [5] and Le Billon [6] investigated the mechanisms behind the resource curse, using econometrics and case-study approaches respectively. On the one hand, finances from natural resource exploitation support rebellion. On the other hand, for complex interactions among the predictors? This paper presents and compares the performance of machine learning and common logistic regression models, more specifically the role of natural resource variables within those models. It discusses how several of the most important natural resource variables, e.g., population, food, water, forest, ores and oil, interact with socio-economic variables.

Materials and Methods
To analyse the role of natural resource variables for conflict prediction, we compared four modelling approaches on four different sets of data. That way, on the one hand, three machine learning models, i.e., two neural networks and a random forest model, were developed alongside an existing logistic regression model that predicts intrastate conflict risk on a country-year basis. On the other hand, three new sets of predicting variables concerning natural resources were developed next to the existing model's training dataset. The combination of four modelling approaches and four variable sets yields 16 different models of which the performance is compared using 10-fold cross-validation. For the four random forest models, trained on the four different datasets, we also evaluated the importance and interactions of predictor variables.

Modelling Approaches
As a starting point, we used the Global Conflict Risk Index (GCRI) [48,49]. The GCRI is developed by the EC JRC as an early warning system for policy support on conflict prevention [49]. The model has the practical purpose to display countries' risk for intrastate violent conflict, allowing for a robust comprehensive global scan based on measurable structural factors [48,49]. Using a logistic regression with country-year as the unit of analysis, the risk for violent conflict is predicted based on 24 structural variables describing the political, social, security, economic, and geographic conditions of a country. The dependent variable, conflict risk, is a Boolean which indicates whether a violent conflict occurs within the next four years or not. A violent conflict event is recorded within the database with 25 or more battle-related deaths, based on the Uppsala Conflict Data Program's (UCDP) One-sided Violence, Non-State Conflict, and Battle-Related Deaths datasets [54]. For more details on the GCRI, we refer to Halkia et al. [49].
Secondly, we applied two artificial neural networks, computing systems inspired by the biological learning mechanism in our brains. Through iteratively applying learning algorithms, a network of nodes connected by progressively improved weights fits itself to reproduce certain output data from the related input data [55]. Many theoretical and empirical discussions remain unresolved about finding the optimal network structure [56,57]. In this study, we followed rules-of-thumb by practitioners for a competent network structure, instead of heuristic procedures for the optimal structure. Two network architectures were chosen: one with one layer of hidden nodes and the second with two layers of hidden nodes. The consensus exists that one hidden layer is sufficient for the large majority of problems [58]. A deeper network with two hidden layers can represent arbitrary decision boundaries and smoothing [58] and can provide a more efficient computation compared to one large hidden layer [59]. Our number of hidden nodes, for both the one layer and two layer models, followed Masters' [60] geometric pyramid as a rule of thumb, making sure that the number of hidden nodes lies between the number of input nodes, i.e., the predictor variables, and output nodes, i.e., here only one: the Boolean indicating a conflict event.
Lastly, we applied a random decision forest, or random forest in short. This machine learning technique creates a multitude of decision trees on random subsets of the dataset and then averages out the outcome over all decision trees [61]. Thereby, they address the inaccurate predictions and the overfitting issues of a single decision tree. The set-up of this random forest includes 500 decision trees. Each tree is constructed from a different random training sample of two thirds of the observations of the whole training dataset. The amount of predictor variables randomly sampled for each split in a decision tree is the square root of the amount of predictor variables in the dataset. This thus depends on the variable set used. Lastly, the decision trees are grown to their maximum size [62].
There is not much known about the relative advantages of neural networks vs. random forests, especially for conflict data. Studies including many different research fields show the two methods perform similarly, where sometimes the neural networks perform a bit better and sometimes the random forests [63,64]. Reasons to choose one over the other approach have been practically grounded, rather than theoretically grounded. For example, neural networks are able to classify unstructured data such as texts, images, audio and video, whilst random forest are not [65]. Random forests, on the other hand, provide more opportunities to interpret the structure of the model leading to the results, as well as requiring less tuning of model structure and parameters in the algorithm to get satisfactory results [64]. Therefore, in this paper, we compare both techniques to the most common statistical model, a logistic regression. We apply two different architectures of the neural network models as their "tunable" character is said to influence prediction results [56,57,64]. We analysed the data with standard configurations for each modelling technique, meaning we did not optimize the basic logistic regression with statistical advances, nor the neural networks topology, nor the parameters of the random forest models' topologies to provide the most accurate results. The reason is that this study aimed to cover a range of tools to test our research question, i.e., to understand whether natural resources play an important role in conflict risk when allowing for complex interactions. All techniques performed sufficiently well to answer the research question, without unequally optimising one technique over the other.

Datasets
Our reference model, the GCRI, was trained to produce conflict risk predictions based on a set of 23 variables [49]. These were selected through a review of more than 200 articles on quantitative conflict research and expert workshops. Halkia et al. [49] categorized the 23 country-wide predictor variables within five thematic areas: political institutions and performance, a country's social fabric, the security situation, the economic situation, and geographical or structural aspects (Table 1). Information about the statistical distribution of each of those variables can be found in Halkia et al. [49]. Only 5 out of the 23 predictor variables concern environment and natural resources: fuel export, food security, water stress, population size, and structural constraints. The data, in country-year units, spans 1989 to 2014 worldwide. Country-year units imply that the dependent variable of conflict risk is auto-correlated through time and space. One way to reduce the effects of auto-correlation is by including the following variables: Recent internal conflict, Years since highly violent conflict, and Neighbouring conflict (security variables in Table 1). The use of the GCRI as the reference model and dataset dictates the variable names and content of the original variable set in this analysis ( Table 1).
The dataset of the reference model is compared to alternative model versions trained on three different sets of predictive variables, including extra natural resource variables or excluding all natural resource variables. Next to the original set, we compared the performance of models based on a solely socio-economic set, solely natural resource set, and a combined set. Table 1 lists all predictor variables in this analysis, including both original GCRI variables and newly added natural resource variables, and indicates which of the four analysed datasets it is part of. The choice of the newly introduced variables was based on the Environmental Performance Index (EPI), which ranks countries' performances on a comprehensive range of environmental issues [66]. Because the data provided by the EPI was incomplete, we searched for more complete country-year open-access data to cover all aspects of the EPI: health impacts, air quality, water and sanitation, water resources, agriculture, forests, fisheries, biodiversity and habitat, and energy and climate. The decision was made not to include fisheries information so that landlocked countries without marine fish stocks would not be excluded from the overall analysis. Additionally, variables about resource stocks or extraction were added, such as natural resource rents and ores and metals exports. The original water stress variable was of low quality [67,68] and therefore replaced by water access, water withdrawal, and water reserves in the new variable sets. Population density was added to replace the log transformed population size.
Lastly, the original variable structural constraints was split up into an accessibility variable and natural disaster variable.

Predictive Performance
We trained models from these four different variable sets using the four different modelling approaches described above, which resulted in 16 distinct models. We compared the models among each other using 10-fold cross-validation [84]. Based on empirical and theoretical evidence, Arlot and Celisse [84] and Bergmeier et al. [85] recommend 10-fold cross-validation over alternative model evaluation techniques, such as leave-one-out cross-validation, classical out-of-sample evaluation, or temporal partitioning of the training and test sets.
Per modelling approach and per variable set, we present boxplots of six performance metrics calculated over the 10 cross-validation iterations: area under curve (AUC), Brier score, Tjur's R 2 , overall accuracy, Kappa index of agreement, and sensitivity. Appendix A provides more information on the performance metrics and what values are desirable for accurately predicting models. We also compared the performance metrics calculated on the training data and test data to identify overfitting. The larger the difference of a performance metric between the training and test data (∆AUC = AUC train − AUC test ), the more the model is overfitted to the data it was trained on. Lastly, we assessed the internal stability of each of the 16 models by analysing their spread in performance over the 10 cross-validation iterations. If a model performs inconsistently over the 10 iterations, the model cannot be considered a stable tool for prediction or analysis. It indicates that the trained parameters and coefficients of the model are highly dependent on that specific part of the observations used to train the model [86].

Variable Importance and Interactions
Next to comparing the overall predictive performance of the 16 models, we investigated the relevance of each variable separately as predictor within the conflict risk models. We focused on the role of natural resource variables.
Commonly used variable importance tests from parametric statistics, such as standardized coefficients or odds ratios, could not be used for the logistic regression models. This is because the predictor variables do not adhere to statistical prerequisites, such as near-normally distributed data, no autoregression in time or space, and no multicollinearity. Without adhering to those assumptions, the interpretation of the effects of the predictor variables on conflict risk is not valid. In contrast to models for causal inference that need to adhere to many statistical assumptions to make statistically significant interpretations about correlations and causal relations, this study applied models for predictive analytics that aim to predict the outcome variable accurately, without the need to adhere to any statistical assumptions. Therefore, we calculated variable importance measures based on predictive analytics, focusing on the machine learning models that can take into account complex variable interactions.
Neural networks are considered black boxes, which means it is difficult to understand what is driving a certain prediction [55]. Although random forest models are often described as black boxes too, e.g., [38], many metrics exist to evaluate the importance of predictor variables within them. Some examples are the number of nodes the predictor variable is applied in within the forest, the amount of times the predictor variable is a root node, and the node purity increase [87]. Breiman [62] developed the original and most often applied metrics of variable importance, called the permutation accuracy decrease. To calculate this metric, the observations of a specific predictor variable are permuted randomly, i.e., the order of observations are mixed. The difference in prediction accuracy before and after permuting this variable indicates the predictive importance of this variable for the model. The mean squared error (MSE) is most commonly used as accuracy metric before and after permutation. A key advantage of this metrics is that "they cover the impact of each predictor variable individually as well as in multivariate interactions with other predictor variables" [44] (p. 307). Thus, Sustainability 2020, 12, 6574 7 of 28 they can "help identify relevant predictor variables even in such high dimensional settings involving complex interactions" [44] (p. 308).
Hence, we analysed the variable importance of all variables, but with special interest for natural resource variables, within the four random forest models. We used the following variable importance metrics: permutation accuracy decrease, mean minimal depth of a variable within all decision trees of the forest, and number of nodes that a variable is present in within the forest. We present these three importance metrics simultaneously for each predictor variable in four variable importance plots: one per random forest model trained on the four different datasets. Important variables have • a high permutation accuracy decrease, meaning that changing this variable in the model will have large effects on the prediction outputs; • a low mean minimal depth, meaning that this variable is present early in the decision structure of the decision trees and thus classifies the largest chunks of data into the right output; • a high number of nodes, meaning this variable is present/considered in many nodes of the decision structure of the trees and is thus an important predictor at several levels of detail.
Since natural resource and environmental variables are said to lead to conflict only through interaction with socio-economic variables [17,18], we also investigated variable interactions. The random forest model provides opportunities to study the decision structures of separate trees and the forest as a whole [87]. If two variables systematically appear more frequent in combination with each other/sequence of one another in the decision paths, it could indicate an interaction between those variables [87]. If the combination occurs high up in the tree structure, i.e., early in the decision pathway, it can be considered an important interaction [87]. The distance between two variables is the difference in the depth of one variable compared to the depth of the maximal subtree of the other variable. The maximal subtree is used because a variable can occur several times in a decision tree. Interactions between two variables can then be identified by a low average distance between the two over all trees, also called conditional mean minimal depth [87]. Therefore, we developed four variable interactions plots, again one per random forest model trained on the four different datasets. For each pair of variables within the models, we calculated three interaction metrics. First, the average minimal distance between the two is called the "conditional mean minimal depth", and indicates the strength of the interaction. Second, the average depth at which this interaction occurs is called the "unconditional mean minimal depth", and indicates the importance of the interactions for accurate predictions. The third metric is the frequency of occurrences of each pair in the 500 decision trees. The three are presented respectively on the y-axis, x-axis, and by the size of the data point. Only the 10 interactions with the smallest conditional mean minimal depth per model are presented.

Computing
All statistical computing was conducted in R, version 3.4.4 [88], using pROC, Neuralnet [89], randomForest [90], and randomForestExplainer [87] packages. The distribution of the GCRI's original code and dataset [49] is currently limited, although attempts have been made to obtain an open-access license to share the material free online. Other replication materials are available online as Supplementary Material connected to this manuscript. This includes data for the natural resource variables, and codebook for development of neural networks, random forest models, model performance comparison, and variable importance and interaction analysis. Table 1. Overview of all predictor variables, i.e., both from the original variable set and newly added variable, listed per thematic content of the variables. Each variable is described, its source given, and it is indicated which of the four datasets analysed it is part of. x x Unemployment Unemployment, total (% of total labour force), of the World Bank's Worldwide Development Indicators [75] x x

Variable
The natural resource-related variables originally included in the GCRI (5 variables) [49] x (x) (x) Fuel export % of merchandise export products [75] x x x Food security A weighted mean of 4 sub-indexes of the FAO food security index: Average dietary energy supply adequacy, Domestic food price level index, Prevalence of undernourishment, Domestic food price volatility [78] x x x Water stress Total overall water risk in the Aqueduct Country and River Basin Rankings: raw country scores for 'tdefm' [79] x Population size Total population, log transformed [76] x

Structural constraints
Extent to which structural difficulties constrain the political leadership's governance capacity, including extreme poverty, lack of educated workforce, disadvantageous geographical location, infrastructural deficiencies, natural disasters and pandemics [80] x

Natural resource base
Arable land % of arable land [75] x x Food production net food production per capita [78], additional to Food security (which more relates to access to food) x x Forest area % of forest area [75] x x Ores and metals exports % of merchandise export products [75] x x Renewable energy production Renewable electricity output (% of total electricity output) [75] x x Natural resource rents Total natural resources rents are the sum of oil rents, natural gas rents, coal rents (hard and soft), mineral rents, and forest rents (% of GDP) [75] x

Predictive Performance
When comparing the four different modelling methods the random forest models perform clearly better than the logistic regressions most commonly used in the conflict modelling field today ( Figure 1). For example, the median AUC scores over the 10 iterations are at least 0.1 higher than the logistic regression and the neural networks. Neural networks, however, do not perform better than the logistic regression based on AUC scores and most of the other performance metrics (Figure 1). They do outperform the logistic regression models according to Tjurs' R 2 and the sensitivity metric.
Contrary to our expectations, the addition of natural resource variables does not substantially improve the performance of conflict risk models ( Figure 1). All six performance metrics demonstrate this evidence consistently. Figure 1 shows that the models based on the dataset combining socio-economic and natural resource variables perform similar as models trained on the original dataset when applying logistic regression and random forest. The combined dataset model performs slightly better than the original when applying both neural networks. Models based on only natural resource variables perform much worse when applying logistic regression, slightly worse in the neural networks, but slightly better when applying random forest compared to all other variable sets ( Figure 1). Lastly, the models trained on only socio-economic variables tend to perform slightly worse than the models based on the original dataset for the logistic regression and the random forest models, but similar for the neural networks ( Figure 1).
The highest overfitting is found in the combined dataset model that applies neural network 2 (∆AUC = 0.08). The other neural networks are slightly overfitted (∆AUC ≈ 0.04). The random forest and logistic regression models practically were not overfitted (∆AUC ≤ 0.02). These patterns can be explained technically, rather than content-wise: neural networks have the tendency to overfit, especially when trained on large datasets such as the combined set. Random forest models on the other hand were developed with the specific goal of addressing overfitting issues of other machine learning techniques. Concerning model stability, the logistic regression model based on only natural resource variables is less stable than the logistic models based on the other three variable sets, consistently over all six performance metrics ( Figure 1). Otherwise, the stability of the 16 models do not demonstrate any specific pattern related to the variable sets. Generally, the random forest models are most stable, while the neural networks perform the least stable.

Variable Importance and Interactions
Natural resource variables stand high in the predictor importance hierarchy ( Figure 2). Recent internal conflict is noticeably the most important predictor in the three variable sets that contain socio-economic data, with the highest accuracy decrease and lowest mean minimal depth. Concerning natural resources, population size and food security are among the 10 most important predictors in the original dataset. In the natural resource set and combined set, population density, food security, water access, and forest area are among the 10 most important predictors. Arable land is important too with similar accuracy decrease and mean minimal depth as population density, but present in less nodes. Fuel export, ore export and resource rent occur in all models among the least important half of all predictors. Water stress is only among the eight least important of 24 predictors in the original dataset. However, when replacing the low-quality water stress variable with better water-related variables in the new natural resource set (Table 1), water access is among the top 10 predictors (Figure 2). In the three models including socio-economic data, i.e., based on the original dataset, on the socio-economic dataset, and on the combined dataset, the socio-economic variables among the top 10 predictors are: recent internal conflict, level of repression, infant mortality, GDP per capita, government effectiveness, and youth bulge.  The top 10 interactions of the combined set, with conditional minimal depths smaller than 3.6, are all closer than the original set's interactions (Figure 3). The closest interaction is Food production-GDP per capita with an importance (unconditional depth) below 4. All other of the top 10 interactions have Food production as a root variable too, combined with socio-economic variables such as infant mortality, youth bulge, or unemployment; as well as with natural resource variables such as food security, water access, accessibility, forest area, and resource rent. All the interactions within the Figure 2. Variable importance plots of the four random forest models, i.e., trained on the four variable sets. Per model, we plotted for each variable: the MSE accuracy decrease, the mean minimal depth, and the number of nodes using that specific variable for splitting, respectively on the x-axis, y-axis, and by the size of the data point. Important variables have a high accuracy decrease, a low mean minimal depth, and high number of nodes, and thus are plotted towards the upper right corner as large data points. Orange indicates socio-economic predictors; green indicates natural resource predictors. The 10 most important predictors are tinted darker, based on a joined ranking of the three importance metrics presented.
To better interpret the importance and interaction plots (Figures 2 and 3), the conditional and unconditional depths of variable interactions can be compared with the average size of decision trees in the random forest models ( Table 2). In general, the 10 closest variable interactions per model all have mean conditional depths lower than 4.1 (Figure 3). In other words, those 10 most closely interacting variables are on average closer than 4 four from each other in all the decision trees of the random forest. In comparison to the average tree size and forest structure (Table 2), this can be considered a close positioning of those two variables and thus a relevant potential interaction between them. In technical terms, those 10 pairs of most closely interacting variables per model are on average positioned closer to each other in the decision tree structures than half of the average tree size (depth) or than a quarter of the maximum tree size ( Table 2). Remember that only a random subsample of all predictor variables is considered at each split in a decision tree. This means that two closely interacting predictors will never occur in two splits right beside each other in all 500 trees of the random forest. Thus, a distance between two interacting variables, i.e., conditional depth, of 1 or smaller is impossible. Consequently, we can interpret a mean conditional depth lower than 4.1 (Figure 3) as a close interaction compared to the average size of the decision trees in the random forest models ( Table 2).
combined set have relatively low number of occurrences: on average each of the top 10 interactions occurs 84 times among the 500 trees of the forest. * 'Avg.' means averaged over the 500 decision trees within each random forest model. ** Here 'average three depth' means the average depth from root to each terminal node within one decision tree.   * 'Avg.' means averaged over the 500 decision trees within each random forest model. ** Here 'average three depth' means the average depth from root to each terminal node within one decision tree.
In the original variable set, population size is the most interacting variable (Figure 3). It interacts with recent internal conflict, regime type, lack of democracy, and income inequality, in increasing order of conditional depth (Figure 3). The other main interactions for this model are purely socio-economic with regime type and infant mortality present in four and three interactions respectively out of the top 10 interactions. The interaction of recent internal conflict with population size is clearly the closest and most important with an average distance (conditional mean minimal depth) of 3.4 between them; an unconditional mean minimal depth of 2.5; and 420 occurrences over 500 trees.
The socio-economic based model overall shows the closest and most frequent interactions for predicting conflict risk ( Figure 3). The strongest interactions occur between recent internal conflict and infant mortality, youth bulge, or corruption. The second root variable occurring in the top 10 interactions of this model is repression, interacting with GDP per capita, infant mortality, government effectiveness, and youth bulge.
The top 10 interactions of the natural resource set show a similar closeness and importance as the model based on the original dataset, i.e., average distances between interacting variables (conditional mean minimal depths) of around 4 and unconditional mean minimal depths around between 2.3 and 4 ( Figure 3). Two interactions, food production-water access and food production-food security, have the highest importance in predicting (lowest unconditional mean minimal depths) of all interactions present within the four models.
The top 10 interactions of the combined set, with conditional minimal depths smaller than 3.6, are all closer than the original set's interactions (Figure 3). The closest interaction is Food production-GDP per capita with an importance (unconditional depth) below 4. All other of the top 10 interactions have Food production as a root variable too, combined with socio-economic variables such as infant mortality, youth bulge, or unemployment; as well as with natural resource variables such as food security, water access, accessibility, forest area, and resource rent. All the interactions within the combined set have relatively low number of occurrences: on average each of the top 10 interactions occurs 84 times among the 500 trees of the forest.

Seemingly Conflicting Role of Natural Resources
Our findings convey two seemingly conflicting messages: (1) adding natural resource variables does not substantially improve conflict risk predictions, and (2) natural resources stand high in the predictor importance hierarchy. Yet, we see compatibility. We interpret that signals from natural resources leading to conflict risk are reflected in and influenced by socio-economic conditions. In other words, we infer that natural resources are one of several root causes leading to violent conflict, mediated by their socio-economic and political context. We base this interpretation on the following four considerations from this study's empirical results, in light of the existing literature discussed below in Section 4.2.
First, no strong direct relation was found between only natural resource variables and risk for violent conflict: Figure 1 shows a weak predictive performance of the logistic regression based on the natural resource set. This has been shown by various peace and conflict scholars before, e.g., [21,91]. However, their effect on conflict risk may be indirect, as supported by many researchers, e.g., [17]. Second, socio-economic conditions alone are strong conflict predictors when linked directly in logistic regression models without complex interactions (Figure 1). They can potentially incorporate effects from natural resources, such as water access for example influences child mortality [92]. Third, the models with only natural resource variables are able to predict conflict risk very well when using more complex models (Figure 1). We interpret that complex-systems-based methods can recreate the indirect effects of natural resources on socio-economic drivers that can directly predict conflict risk, based on only natural resource variables. The complex-systems-based models based on only natural resource variables seem to be able to capture the indirect, mediating signal of socio-economic drivers. Lastly, the overall high predictive performance of the machine learning approaches prevents us from detecting substantial improvements by adding or removing natural resource variables (Figure 1). Likewise, the inclusion of variables capturing autocorrelation such as recent internal conflict in the models based on the original, socio-economic, and combined datasets improve the predictive performance very much (Figure 2), leaving less room for improvement when including or excluding natural resource variables.
While it is not necessary to include natural resource variables in purely predictive models on the short term, precious conflict prevention time can be gained by considering natural resources in predicting conflicts. Including natural resources is also imperative in models that aim to investigate the root causes of violent conflicts, either for research or policy-making on conflict prevention. The following sub-section will attempt to clarify plausible causal pathways from natural resources to violent conflicts by building on our importance and interaction analysis of natural resources variables and comparing it with existing literature.

Natural Resources Are Important, Interacting Predictors
The natural resource-related variables with important roles as predictors were Population size and density, Food security, Water access, Forest area, and Arable land. The natural resource variable most closely interacting with other variables was Food production.

Population
Our results show that population size and density are important predictors of conflict risk in models that allow for complex variable interactions ( Figure 2). There is consensus among the majority of studies that population growth or a high population density alone does not directly lead to violent conflict [93][94][95]. Indirect effects of population changes on violent conflict are supported in the literature. On the one hand, population growth leads to environmental degradation and increased resource scarcity, which can reduce agricultural and economic productivity [94,95]. On the other hand, unequal population growth or migration changes the local balance between religious, ethnic, linguistic, or class groups [94,95]. Competition over scarcer resources due to population changes is said to increase the risk for internal violent conflict if the following two conditions co-occur: (1) reduced agricultural or economic productivity, and (2) social stratification [93][94][95]. In Figure 3, we identified the interaction between population density and food production as the closest one of the model based on natural resource variables, as well as the interaction between population size and income inequality as an important one for predicting violent conflict in the model based on the original dataset. Those two identified interactions support such an argument about reduced productivity and social stratification. Yet we identified closer interactions between population and style of governance, i.e., regime type and lack of democracy. We did not find an explanatory hypothesis in the literature that links population to violent conflict through intervening governance factors. Lastly, there is some evidence that large populations increase the risk for armed conflict because of a higher probability of different cultural groups within one country and more opportunities for the mobilization of opposition [95].

Food
The effects of food security and arable land on violent conflict follows three different causal pathways within scientific literature. First, risk of violent conflict has been said to increase with higher food prices in urban areas, prolonged food insecurity in rural areas, severe volatility in food prices, and rapid food shortages [19,20]. Food security, in our analysis, encompasses food price, volatility, and nourishment aspects (Table 1). Because food security is an important predictor variable in our models (Figure 2), our results agree with these earlier findings. Secondly, cropland has been shown to support violent conflict because food is a valuable and lootable resource that can feed and pay combatants [96,97]. Note that this argument only applies to already conflictual situations, with cropland making these conflicts more violent. Even though arable land surfaced as an important predictor (Figure 2), our analysis cannot distinguish between different intensities of violence because of a binary outcome variable. Thirdly, several researchers found no consistent relation between agricultural output and violent conflict [19,21,22]. They argue that the socio-economic and political context is more important than agricultural productivity to understand conflict risk. Hendrix and Brinkman [19] (p. 13) explain that socio-economic and political context "can either mitigate or amplify the effects of food insecurity on conflict". Based on our results, we only partly agree with these statements. In agreement with Buhaug et al.'s [21] results, food production is not an important predictor in our random forest models ( Figure 2). Still, of all the natural resource variables, it is the one that interacts most with other variables (Figure 3). Our findings specified that food production interacts more with economic indicators (GDP per capita and unemployment) and demographic indicators (infant mortality and youth bulge) than with political variables (Figure 3). Therefore, we support the findings of Hendrix and Brinkman [19], Buhaug et al. [21], and Jones, Mattiacci, and Braumoeller [22] that food production is important within its socio-economic context, but not as much within its political context.

Water
The newly added water variables (water access, water reserves/capita, and water withdrawal) are relatively important predictors for the two models with the new natural resource variables, which indicates that water indeed affects conflict risk. The water stress variable of the original dataset can be considered of low quality with little variation in values over the studied period [67,68], which probably explains its low predictive importance. Water seems the most studied natural resource in relation to violent conflict, including droughts, precipitation, and shared international river systems. This is probably because of an intuitive recognition that water is essential to human life [98]. We only discuss a handful of water-conflict studies in relation to our results, covering the main arguments made within the wider water security literature.
Even though many studies start from the hypothesis that water scarcity leads to violent conflict, most studies do not conclude this after their empirical analyses. In a large-scale survey by Marcantonio, Attari, and Evans [28] (p. 14) in sub-Saharan Africa, respondents did "not think water scarcity is a direct source of conflict", compared to other causes such as land grabbing, crop damage, and politics. Link, Scheffran, and Ide [26] (p. 510) concluded from their review that "despite the fundamental differences in water stress in the various parts of the world, the actual allocation of water is rarely at the heart of the conflict". Accompanying political and cultural considerations, such as electricity production, hegemonic status, water quality standards, preservation, water service distribution, adaptive capacity of societies, and upstream vs. downstream users, are important steps in the causal pathways from water to violent struggles [24,26,27]. Selby and Hoffmann [25] (p. 1010) even go as far to state that their case studies of Cyprus and Israel-Palestine "provide negligible support for the Malthusian thesis that water scarcity can cause or contribute to conflict and migration". Further, Gizelis, and Wooden [23] and Adano et al. [99], in an empirical large-n study and a Kenya case study found that water abundance, rather than water scarcity, leads to violent conflict. What is distinct about the water scarcity literature, compared to other natural resource scarcities, is the argument that water scarcity or degradation could also lead to cooperation instead of conflict. Most of the literature points to a significant role for institutions, be they traditional or official, to mediate how water resources influence violent conflict or cooperation [23,27,99,100].
Our analysis disagrees with most findings by the researchers mentioned above. Water access is among the top 10 predictors for violent conflict risk in the random forest models that include the new natural resource variables. Water reserves per capita and water withdrawal are the 13th and 14th most important predictors of 35 variables in the combined set, following the top 10 closely. Moreover, our results do not identify any close socio-economic interaction with one of the water variables to predict conflict risk, and thus indicate that water access, reserves and withdrawal are important factors independent of mediating socio-economic effects. The only important interaction found is with food production, which is a logical biophysical relation.

Forest
Although less studied than other natural resources, forest area is among the important natural resource predictors for conflict risk (Figure 2). Researchers have described three distinct links between forest area and violent conflict. First, rich forests are generally located far from mainstream commercial activity and government control [29]. Communities living in forests are often marginalised geographically, politically, and socially, as well as economically [29]. Development and implementation of property rights are often lacking, inadequate, or discriminatory [29]. Competition over the forests' rich resources between disadvantaged groups, in combination with weak institutions, promotes violent conflict in forested areas [30,31]. Secondly, scarcity of forest land has been perceived as a direct cause for relapse into violent conflict only in a post-war setting, after contested redistributions of forest land [101,102]. Thirdly, even when forests are not among the initial causes of conflicts, they tend to become involved because of the specific goods and services they provide [29]. Forests provide marketable materials to finance conflicts, such as timber, wildlife, medicinal plants, and hidden land for the cultivation of illegal drug crops. Rivalry over access to and control over these goods turns forest lands into strategic resources [29]. Moreover, forests function as venues for violent conflicts because of the shelter they provide to civilians and refugees, as well as guerrilla-style militia [32]. According to our interaction analysis, forest area only interacts with food production (Figure 3). This finding does not support nor is it in disagreement with any of the causal paths suggested in the literature.

Ores and Oil
Surprisingly, ore and metals exports, oil export, and resource rent are not as important for predicting conflict risk than the natural resource variables described above (Figure 2). This is an interesting finding with respect to the seminal studies on the resource curse. Collier and Hoeffler's [5] econometric analysis shows that natural resource extraction significantly increases conflict risk. They explain that finances from resource exports make rebellion feasible and attractive. Additionally, they argue that a strong dependence on primary commodities worsens governance and so generates strong grievances. Le Billon [6] comes to the same conclusions as Collier and Hoeffler [5] through several in-depth case studies: resources and wars are linked because resource revenues finance rebellions, resource exploitation generates tension, and resource dependence presents major governmental challenges. Our analysis does not contest these findings, but indicates that non-renewable resource variables, which are related rather to abundance theories of conflict risk [18], are less important for predicting intrastate violent conflict than renewable resource variables, such as food security, water access, forest area, and arable land, which have been more related to resource scarcity theories of conflict risk [18].

Socio-Economic vs. Natural Resource Variables
The discussion above demonstrates that natural resources variables are important to predict conflict risk. Still, they are less important than the following socio-economic variables: recent internal conflict, level of repression, infant mortality, GDP per capita, and government effectiveness (Figure 2). Youth bulge is of similar importance as most natural resource variables, while other common predictors, such as regime type, neighbouring conflict, ethnic compilation, lack of democracy, and empowerment rights are less important than the discussed natural resource variables. Additionally, the 10 closest interactions in the combined model and the four most important interactions in the original model all include at least one natural resource variable (Figure 3). For these reasons, and following the discussion paragraphs above, we interpret that scarcity, as well as abundance of natural resources, can drive violent conflict as root causes moderated by socio-economic or political contexts. Current globally unsustainable trends, such as deforestation, increased consumption, population growth, and climate change, are projected to increasingly impact the natural resource base [9]. Through natural resources, we infer that these trends are contributing and will continue to contribute to violent conflicts worldwide. To achieve sustainable peace, we deem it necessary to include natural resources in explanatory research and decision-making on conflict prevention.

Limitations
There are some limitations to the quality of data used in the analysis. Water stress and soil degradation miss many observations and do not change much per country over the studied years. This probably explains their low importance for predicting conflict risk. Both water stress and soil degradation relate biophysically to arable land, food production, and eventually food security, which are all important predictors. Therefore, we expect that, with better data, soil degradation would be a more important predictor or interact more closely with other food related variables, similar to the new water variables (Figure 2 and Table 1). Further, we can improve the causal analysis by increasing the country-year resolution of the studied data [103]. More and more conflict data are becoming globally available on sub-national statistical units, even on a grid scale of 0.5 • by 0.5 • , and at monthly time slices [104]. We dealt with the auto-correlation through time and space of the conflict data in country-year units by including the variables recent internal conflict, years since highly violent conflict, and neighbouring conflict. Another possible way to reduce the effects of autocorrelation on prediction and analysis results is by working with events-based data. This means every conflict is one observation in the dataset, even if it crosses years and political boundaries. Then the role of natural resources can be studied in terms of the onset, duration and end of violent conflicts. Difficulties with this approach would be to delineate one conflict observation, i.e., when do different violent actions belong to one and the same conflict observation and when are they part of different conflicts. This is because conflicts have spill-over effects creating new grievances, new militia, and new conflicts [105]. Hence, it remains difficult to separate between different conflict observations and to completely cancel out spatial and temporal autoregression.
Model development, for logistic regression as well as machine learning methods, is an iterative process. Variable selection for the optimal predictive model should be a stepwise process in including necessary, or excluding superfluous, variables. Furthermore, the model structure of neural networks and random forests can be optimised. It was not our aim to create the best predictive models, but to investigate the relevance of adding natural resource variables to complexity-based conflict models. Towards that aim, standard settings for model structure suffice. Superfluous variables were identified as less important, without influencing the analysis of the role of other variables towards predicting conflict risk too much. Examples of such unimportant, uninfluential variables were biodiversity conservation and renewable energy production ( Figure 2). We chose a large standard number of decision trees within the random forests to be able to detect frequently occurring interactions.
Currently, methods and tools for interpretation of machine learning and predictive models are being continuously developed, tested, and improved, e.g., [65]. We applied variable importance and interaction metrics specifically developed specifically for random forest models [87]. More and more importance and interaction metrics are being developed that can be applied to many different types of prediction models [106]. In this paper, such model-agnostic metrics could be applied to and compared between the logistic regression, neural networks and random forest. The advantages of such model-agnostic tools are that the role of variables can be interpreted and compared between different types of models. Permutation accuracy decrease, as used in the above analysis, is one of those and can be extended to the logistic regression and neural network provided many extra coding efforts [65]. The other variable importance and interactions metrics, i.e., mean minimal depth, number of nodes, and conditional mean minimal depth, are only applicable to random forest models and could be replaced by model-agnostic tools currently being developed.
Even though complex computational models, such as neural networks and random forests, are able to incorporate nonlinearities and interactions between variables, these empirically based methods have their limits. They only learn from observations recorded within a limited time span, in this case 1989 to 2014 [51]. The statistical metrics are not able to distinguish between root variables and mediating variables and to test plausible indirect causal pathways, which is why these needed to be interpreted form the data-driven results of this paper and existing literature. Data-driven approaches more easily neglect existing causal theories, as well as avoid contributing to new theory [51]. Other complex system methods, such as system dynamics modelling, agent-based modelling, network analysis, and scenario development, can complement complex empirical models to study the natural resources-violent conflict link. These methods provide greater freedom of choice between an empirical or theoretical grounding to analyse complex systems.

Conclusions
This study found that natural resource variables are important predictors of conflict, though we interpret their effects to be often captured by the intervening socio-economic variables. Adding natural resource variables does not substantially increase the performance of conflict risk models, even when allowing for nonlinearities and complex interactions through machine learning. Nevertheless, several natural resources ranked among the top 10 of important predictor variables, including population density, food security, water access, forest area, and arable land. The closest natural resource-society interaction to predict conflict risk according to our models was food production within its economic and demographic context, e.g., with GDP per capita, unemployment, infant mortality and youth bulge. The results demonstrated that resource rents, oil export, ore and metal export are less important predictors relative to the natural resource variables mentioned above than prior research has suggested.
From our analysis, we interpreted, in light of the existing literature, that natural resources can be among the root causes of violent conflict, moderated within their socio-economic or political context. We inferred that natural resource signals to conflict risk are indirect, with probably several mediating socio-economic and political steps in the causal pathway from natural resources to violent conflict. If so, precious conflict prevention time and strategies can be gained by considering natural resources. In consequence, we consider it important to include natural resources in models that can account for complex interactions if the purpose is to investigate root causes of and complex causal pathways to violent conflicts, either for research or policy-making on conflict prevention. We consider that sustainable peace requires sustainable and conflict-sensitive resource use and management. This is even more imperative because of ongoing and projected trends that affect natural resources, such as deforestation, increased consumption, biodiversity loss, and climate change. However, it is not necessary to include natural resource variables in purely short-term predictive models used as conflict early warning systems.
Many research opportunities follow from this analysis. First, we recommend that researchers harness the full, also explanatory, power of complex computational tools to understand drivers of violent conflicts. The variable analysis within predictive models can be deepened by extracting and visualising the direction of the effects of certain important variables-positive or negative-on conflict risk and also in consideration of the interaction with other variables. Furthermore, it would be possible to investigate three-step interactions instead of only two-variable interactions. Preferably, model-agnostic interpretation methods would be used. Secondly, we mentioned possible model improvements by including better data for the soil variable, the use of higher-resolution data spatially and temporally rather than country-year observations, the use of conflict-event data, stepwise variable selection, and optimisation of the neural network and random forest structures. Lastly, we are searching for statistical metrics to distinguish root causes from mediating variables and to test plausible indirect causal pathways. While investigating the former, we recommend the use of other complex system methods, as well as qualitative theoretical studies, to complement these empirical findings with more theory driven approaches on the complex link between natural resources and the risk for violent conflict.
Supplementary Materials: The following are available online at http://www.mdpi.com/2071-1050/12/16/6574/s1: data for the natural resource variables, and codebook for development of neural networks, random forest models, model performance comparison, and variable importance and interaction analysis.

Acknowledgments:
We would like to thank the Peace and Conflict team at the EC JRC who supported us in working with the Global Conflict Risk Index. We are also grateful to John Miller and Scott Page from the Santa Fe Institute who helped us develop questions and tools to analyse conflicts computationally as complex social phenomena. Lastly, we are grateful to Maartje Oostdijk and several anonymous reviewers who improved this manuscript with their constructive feedback.

Conflicts of Interest:
The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results. Table A1. Overview of metrics calculated to compare the models' performances.

Performance Metric
Calculation Motivation/Advantages Source Key for Interpretation Calculated directly from predicted probabilities *

Area Under Curve (AUC)
Area under the receiver operating characteristic (ROC) curve. The ROC curve plots the sensitivity against fall out rate (see further down in this table) over the whole range of possible classification thresholds (0 to 1) Commonly used to assess how well a model discriminates between high-risk and low-risk subjects. High sensitivity combined with low fall out rates over the whole range of thresholds leads to a larger area under the ROC curve. [86,107] Between 0.5 and 1. The higher, the better.
Brier score The average of the squared prediction error, similar to the Mean Squared Error (MSE) for linear regression. It evaluates the calibration of the model while all other measures evaluate the discrimination by the model. [107,108] It generally ranges from 0 (perfect) to 0.25 (worthless). It is the only measure in this analysis for which holds the lower the better.
Tjur's R 2 π 1 − π 2 Coefficient of discrimination, a highly recommended R 2 -substitute for logistic regression models. It calculates the difference between the averages of predicted values for 1 and 0 observations, respectively. Its ease of interpretation comes from the fact it uses terms and concepts that are directly related to models for binary observations on their own premises, without any reference to variance and variation concepts. On top, it has close direct mathematical relations with the standard formulas of R 2 .
Calculated from predictions classified into "no conflict risk" vs. "conflict risk", based on based on a probability threshold of 0.5 ** Overall accuracy TP+TN TP+TN+FP+FN Proportion correctly predicted, easy to understand. The large amount of easy to predict peace events makes this performance measure less useful.
As a percentage or ratio. The closer to one the better. and total = TP + TN + FP + FN The amount of observed accuracy corrected by the accuracy expected by chance. With unbalanced data, there is a higher chance you will randomly classify the less common group so this should be and is accounted for in kappa. [110] Can range from −1 to +1. Suggested guidelines by [110]: < or = 0: no agreement 0.01-0.20: non to slight 0.21-0.40: fair 0.41-0.60: moderate 0.61-0.80: substantial 0.81-1.00: almost perfect agreement Sensitivity (hit rate, recall, true positive rate)

TP TP+FN
The chance that a conflict event will be predicted, probability of detection, the percentage of conflict events which are classified as such. Slightly more difficult to understand than overall accuracy, but more relevant in a rare event context, such as violent conflicts.
As a percentage or ratio. The closer to 1 the better.
* y denotes the observed outcome and p the prediction for subject i in a dataset of n subjects; π 1 and π 2 denote respectively the averages of predicted values for observed 1-values (i.e., observed conflict risk) and 0-values (i.e., no observed conflict risk). ** TN = True Negative, TP = True Positive, FN = False Negative (Type II error), FP (Type I error).