Exploring the Socioeconomic Co-benefits of Global Environment Facility Projects in Uganda Using a Quasi-Experimental Geospatial Interpolation (QGI) Approach

Since 1992, the Global Environment Facility (GEF) has mobilized over $131 billion in funds to enable developing and transitioning countries to meet the objectives of international environmental conventions and agreements. While multiple studies and reports have sought to examine the environmental impact of these funds, relatively little work has examined the potential for socioeconomic co-benefits. Leveraging a novel database on the geographic location of GEF project interventions in Uganda, this paper explores the impact of GEF projects on household assets in Uganda. It employs a new methodological approach, Quasi-experimental Geospatial Interpolation (QGI), which seeks to overcome many of the core biases and limitations of previous implementations of causal matching studies leveraging geospatial information. Findings suggest that Sustainable Forest Management (SFM) GEF projects with initial implementation dates prior to 2009 in Uganda had a positive, statistically significant impact of approximately $184.81 on the change in total household assets between 2009 and 2011. Leveraging QGI, we identify that (1) this effect was statistically significant at distances between 2 and 7 km away from GEF projects, (2) the effect was positive but not statistically significant at distances less than 2 km, and (3) there was insufficient evidence to establish the impact of projects beyond a distance of approximately 7 km.


Introduction
Since the Rio Earth Summit in 1992, the Global Environment Facility (GEF) has been one of the largest actors in the environmental sector, providing approximately $24 billion in grants, and marshaling an additional $107 billion in co-financing for more than 4700 projects in 170 countries. With the explicit goal of supporting international environmental conventions and agreements, a number of these projects have been subjected to evaluation by the GEF Independent Evaluation Office (IEO) to assess their effectiveness in terms of environmental outcomes. However, relatively little research has been conducted to examine the socioeconomic co-benefits that may accrue due to environmental interventions. This is reflective of a broader shortage of impact evaluations of development projects, as the primary environmental impacts of development projects and programs are rarely assessed and quantified conclusively [1]. Recognizing this gap in the literature, and leveraging a unique spatial dataset and methodology, this paper explores the research question: What was the impact of GEF Sustainable Forest Management projects on household income in Uganda between 2009 and 2011?
To examine this question, we introduce a new method called Quasi-experimental Geospatial Interpolation (QGI). QGI is a matching-based approach to causal inference using historical, spatial data on the location of interventions of interest (i.e., a GEF project). Spatial locations at which a GEF intervention occurred are contrasted to spatial locations at which an intervention did not occur, but were otherwise as similar as possible, following increasingly common spatial propensity score matching approaches [2,3].
QGI mitigates two core shortfalls of existing spatial propensity score matching methods. First, it explicitly provides a measurement of the relationship between distance and impact effect, allowing for analyses of the distance(s) away from an intervention at which effects are no longer detectable. Second, it provides a formalized approach to identifying spatial matches that ensure key statistical principals are not violated (specifically focusing on the Stable Unit Treatment Value Assumption [SUTVA]), minimizing the chance of model selection bias through "p-score hacking" or other similar techniques.
The paper is structured as follows. First, we provide a background literature review on the use of satellite information for environmental impact assessment, and related gaps in the literature in Section 1.1. In Section 2.1 we present the data used for this analysis, and Section 2.2 introduces the Quasi-experimental Geospatial Interpolation approach. Sections 3 and 4 provide our results and conclusions, respectively.
Taken together, this body of work has served to highlight the inconsistent nature of socioeconomic co-benefits attributable to environmental interventions [1,22,25]. This is reflective of the breadth of approaches to environmental interventions-the literature has sought to examine the benefits which accrue from projects that sought to do everything from implementing geographically defined regions of enforced conservation [12,16,19] to training farmers on improved irrigation strategies [26]. The wide range of intervention strategies-set in dramatically different political and social settings-necessarily results in the need to examine the localized impacts of intervention strategies, rather than seek to establish a single overarching relationship between "environmental interventions" and socioeconomic co-benefits.
Further complicating our understanding of the co-benefits of environmental interventions are limitations in available data on the nature of interventions and related outcomes-including the varying spatial and temporal scales at which data is collected [1,22]. Recognizing the limitations of data collected by traditional survey instruments (i.e., telephone, in person) alone, recent literature has begun to leverage satellite and other sources of spatial data to study the causal effects of environmental interventions [2,3,27,28]. These articles have illustrated how satellite sources of data can enable historic reviews of intervention impacts, which is highly valuable for contemporary planning [2]. However, recent literature has also begun to highlight many of the methodological challenges inherent with the use of satellite data [28,29]. These challenges include: 1.
The spatial spillover of intervention effects from recipient regions to neighboring regions [3,30,31].

2.
The robustness of selecting different distance bands to distinguish between areas determined to be "treated" as contrasted to "untreated" [32].

3.
The spatial imprecision of measurement in where interventions are implemented [28].

4.
The accuracy of measurements derived from satellite images [29].
Recent literature has begun to engage with some of these challenges. These efforts have included providing a solution to spatial imprecision in measurements of intervention locations [28] and experimenting with variation in distance band selection [30,32]. Other research details recent advances for overcoming errors in measurements derived from satellites [29]. While these pieces have been pathbreaking in providing solutions to many of the above enumerated challenges, researchers are still left with acute uncertainty regarding (for example) how to model the distance-decay relationship of a treatment effect.
In this piece, we engage with two gaps in the literature. First, we examine the localized impacts of GEF interventions in Uganda, providing an estimation of the degree to which GEF projects have contributed to household incomes in the region; this contribution is a small but important extension of our knowledge on the contexts in which socioeconomic co-benefits might be expected from environmental interventions. Second, we introduce a novel methodology to overcome issues associated with spatial spillover and the selection of distance bands (items 1 and 2 in the above enumerated list).

Data
This paper leverages a novel dataset of Global Environment Facility Sustainable Forest Management projects, represented by thousands of spatial boundaries defining areas of intervention. The analysis relies on a subset of 33 project locations in Uganda for which exact geographic boundaries are known, which can be seen outlined in red in Figure 1. Criteria for selection included projects which had direct implementations (i.e., activities that would impact the geographic region of intervention), intersection with Uganda, and project implementation dates before our baseline period of 2009. Projects include both small-scale projects implemented solely within Uganda, and regional-scale initiatives that were also inclusive of other projects within the region. As Figure 1 illustrates, the majority of projects analyzed here are along the border of Uganda, with a smaller number located in the interior. More generally, the GEF interventions we analyze here tend to be located in more remote areas (i.e., that do not overlap with the Living Standards Measurement Survey (LSMS) survey locations); this is likely reflective of GEF strategies which allocate aid to protected areas.
In addition to information on the geographic location of GEF SFM projects, data from the World Bank Living Standards Measurement Survey (LSMS) was used to estimate the outcome measure in this study-household assets. Collected in 2009 and 2011 within Uganda, the LSMS provides geographic latitude and longitude information on where household clusters were surveyed, accurate to within 5 km. In both surveys, an identical question was asked which is used in this analysis-"What is the total estimated value of all the assets owned by your household?". In 2009, the LSMS surveyed 2960 households with an instrument that asked these questions; of these households, 2926 provided a response to this information along with geographic location information. In 2011, the LSMS surveyed 2497 households, of which 2316 households provided the same information. The locations of the 2009 surveys can be seen in Figure 1.
Geospatial covariates were retrieved and integrated into a standardized grid with a resolution of 10 km following the approach outlined in [33]; a full list of ancillary covariates is included in Table 1. This grid was used as our frame for analysis. The data collected from LSMS was aggregated to each cell to generate annual average measurements of household assets (2009 and 2011), and the distance between each cell and GEF SFM project was calculated; cells for which no household asset information existed were removed from the analysis. This resulted in the dataset used for analysis-each row being representative of a 10 square kilometer region across Uganda (as shown in Figure 1), containing measurements from the data sources in Table 1. Yearly measurements were derived where feasible, with average, max, and minimum yearly values being retrieved and saved. In temporally invariant cases such as roads, the distance to the nearest observation was calculated and assumed to be similar across the time periods surveyed (2009 and 2011).

Methods
In this section, we introduce Quasi-experimental Geospatial Interpolation (QGI). QGI is designed to mitigate two interrelated gaps in our current ability to model causal relationships with spatial data: (1) The challenge of spatial spillover, and (2) the challenge of selecting an appropriate distance band. We also briefly discuss the ways in which (3) QGI mitigates the possibility of omitted variables impacting the estimate of the treatment.

Spillover
The challenge of spatial spillover is well summarized in [30]-specifically, the need to ensure that the units defined as "control" units (i.e., those that did not receive intervention) did not in any way benefit from the treatment. In a spatial context, this is a common challenge-i.e., if a village is a beneficiary of a sustainable farming training program, individuals from neighboring villages may travel to attend, or information from the program may naturally diffuse along social networks. This is also a challenge among treated units-i.e., a treatment effect may compound if more neighboring units are treated. The method presented here does not explicitly estimate this compounding effect, but such effects would be included in the overall treatment effect estimate (see [31] for a complete conversation of the challenges of SUTVA with spatial data).)

Distance Bands
Interrelated to spillover is the challenge of selecting distance bands to distinguish between areas to be defined as "treated" or "untreated" (see, for example [30,32]). This challenge relates to the unknown nature of how interventions will diffuse over geographic space-i.e., if an animal clinic might benefit farmers within 10 or 50 km. Because this distance threshold is generally unknown, researchers are left to arbitrarily determine a threshold at which they anticipate no effect would be reasonable to expect; locations beyond this threshold are defined as "control" units. Arbitrarily choosing this distance can result in an increased likelihood of "p-score hacking", or selecting for distances at which a desired effect is identified; varying this distance can result in bias due to the fact that a single unit can be defined as a "treated" unit in one analysis, and a "control" in another (see Figure 2 for an illustrative example of this challenge).

Omitted Variables
QGI extends a long history of research into quasi-observational research which seeks to enable causal attribution, specifically building on the "propensity score" matching approach [45,46]. This approach is intuitively described as one in which "twins" are identified, where one twin received a treatment and one did not. In such "twinned" studies, estimation for the purpose of causal attribution generally seeks to fit a model of a form similar to: where y i is the outcome of interest (i.e., income), θ is the intervention (or treatment) effect on the outcome of interest for which an accurate estimate is desired, T i represents a binary indication of if unit of observation i received a given intervention, β j represent parameters we seek to fit to control for potential effects from sources other than the intervention j, and X i,j represents the value for unit of observation i for each control variable j. The most common approach to estimating θ is to construct a sample set in which all units that received an intervention (T = 1) are matched one-to-one with units that did not receive an intervention (T = 0), where matches are conducted to minimize differences along relevant dimensions ("twins"). Observations which are not a part of an optimally paired twin in the dataset are either removed (in one-to-one matching) or weighted (in many-to-one matching). This sample set is then used to estimate Equation (1). By only including twins in the analyses, the potential for omitted variable bias is reduced, as-in aggregate-it can be hypothesized that biases between twins due to unmeasured data will be similar in nature [45,46].

Quasi-Experiemental Geospatial Interpolation (QGI)
This paper introduces a new method for the estimation of causal impacts of an intervention using spatial data, Quasi-experimental Geospatial Interpolation (QGI). QGI provides a unique insight into the causal effect of interventions by allowing researchers to examine the spatial distance-decay of impacts (i.e., if a clinic has an effect only on the village it is located in, or also neighboring villages). QGI broadly mitigates two interrelated sources of bias: (1) The arbitrary selection of a distance threshold and (2) the possibility of spillover of interventions from treated to control locations.
Unlike other models used to examine causal impacts, QGI explicitly models the distance-decay function of the treatment estimate (θ in Equation (1)) through an iterative approach of changing the distance thresholds at which controls and treatments are demarcated (similar to the approach adopted by kriging-based spatial interpolation). It follows a three step process in which:

1.
Two hyperparameters are chosen: (a) The maximum distance (δ) for which a treatment effect will be constructed.
The number of distance bands κ, with the geographic distance for each band denoted by κ i .

2.
For each distance κ i , we build a regression model following: The quality of the matches (φ i ) is calculated and recorded. (d) A regression model, similar to that in Equation (1), is estimated; both θ i and the standard deviation σ i are recorded.

3.
After θ is recorded for all distance bands i, the relationship between distance and θ is estimated using a model of the users choice-i.e., spherical or polynomial, with a weighting approach in which distance bands with better match qualities (φ i ) are given more weight. This is repeated for the standard deviation of the estimates (σ).
Following this approach, the relationship between distance, standard errors and estimated treatment effect can be visualized as illustrated in Figure 3.

Step 1-Hyperparameter Selection
Two hyperparameters must be selected to implement QGI-δ, or the maximum distance for which an effect will be estimated, and κ, or the number of distance bands for which effects will be estimated (i.e., the resolution of the estimation). The range 0-δ is split into κ equally-spaced distances, and represented for each distance as κ i . For example, take the case of a researcher attempting to identify the impact of a training program in which a number of stakeholders were trained in a new irrigation strategy at Village A. If δ = 100 km, the implicit assumption would be that villages at least 100 km from Village A were not impacted by the training program in any way. If κ is set to 3 in this case, a total of three distance bands would be saved: κ 1 = 33.3, κ 2 = 66.6, κ 3 = 99.9. Of note, selecting a large δ value can help mitigate the probability that a study may be biased by spillover effects; however, this comes at the cost of increasing the probability quality matches cannot be found.

Step 2-Iterative Modeling
Step 2(a) Defining Treatment and Control for each Distance κ i For each value of κ, we define all units within distance κ i of an intervention as being "treated" (i.e., T = 1 in Equation (1)). All units that are at least distance δ away from an intervention are identified as a control (T = 0). This means that, for any given distance κ i , we have a varying set of treated locations, and a static set of control locations they will be contrasted to.
Step 2(b) Matching "Twin" Treatment and Control Locations for each Distance κ i Our goal with the dataset constructed in step 2a is to contrast locations that were close (distance <= κ i ) to an intervention to those that were at least distance δ away, but otherwise as similar as possible. To facilitate matching these "twin" locations, we estimate the likelihood that any unit was close to an intervention-referred to as a propensity score: where T j is a binary value indicating if location j is within distance κ i of an intervention site; β 0 is an intercept, β k is a fitted parameter value for covariate k, and X k,j is covariate k's value for location j.
An optimal parameterization of this model is identified through a logistic regression least squares approach, and the resultant propensity score predicted for every unit of observation j are saved as P j . For every treated unit T j = 1, the single control unit T j = 0 with the most similar propensity score is identified and saved; a final dataset is then constructed that only contains these matches (unmatched cases are discarded). For more information on the mechanics of propensity score matching for causal inference we direct readers to [47].
Step 2(c) Calculating Match Quality for each Distance κ i Because matches are constructed iteratively across each distance, it is possible that some distances may have poor matches-i.e., there are not sufficient "twins". To account for this, for each distance κ i a metric of match quality is calculated by taking the absolute value of the difference between the average propensity score (P) for treated and untreated cases: This information is used in subsequent steps of QGI.
Step 2(d) Estimating θ for each Distance κ i After matching, constructing a new dataset which only includes matched cases, and recording the overall match quality φ, a linear regression model is fit: in which the ultimate goal is to estimate θ with a high degree of accuracy, using a maximum likelihood estimator. For each distance κ i , the estimate of θ is saved as θ i , along with the standard error of the estimate σ i .

Step 3-Estimating the Spatial Relationship between Distances κ i and Estimates θ i
Once all estimates of θ and σ are constructed for each distance κ i , we seek to establish the relationship between distance κ and treatment effect θ. While many methods for establishing the relationship between distance and another variable can be selected, here we implement a weighted third order polynomial. Weighting is accomplished by (a) dropping (weight = 0) any matches that did not meet a threshold of match quality (in this case, a difference of 0.1), and (b) assigning more weight to the best relative matches using the estimation of match quality φ. Weighting is accomplished by adjusting the loss function from ordinary least squares to account for the weights provided by φ i : By repeating this process for the standard errors recorded in each model iteration (σ i ), we can establish a predicted confidence interval around the polynomial trend line. By adding the model standard errors to the standard errors of the polynomial fit itself, we get a final envelope of possible errors for any given distance. In the results for the case study, Figure 3, the blue line indicates the 3rd order polynomial trend relating θ and distance; the area bracketed by yellow lines indicates the standard errors attributable to uncertainty regarding the estimate of this trend. The remainder of the grey region indicates standard errors attributable to the estimate of θ at each iteration i.

Results
This section introduces QGI using an illustrative case study in Uganda; the subsections here are reflective of each of the steps outlined in Section 2.2.

Step 1-Hyperparameter Selection for the Uganda Case Study
For this analysis, an upper distance bound of δ ≈ 50 km was selected as a theoretical approximation of the maximum distance for which an effect could be detected (as the reader will see in later sections, the resultant estimate is-in this case-insensitive to this choice, as the maximum distance for which the data support conclusions is approximately 7 km). κ was selected according to available computation, with 30 iterations being performed on each of 16 compute cores available (for a maximum index for κ of 480 (κ 480 ) (Initial exploration suggests that-for reasonably smooth surfaces-results are robust to this hyperparameter selection).

Step 2-Iterative Modeling for the Uganda Case Study
Within Uganda, we seek to contrast households surveyed by LSMS proximate to GEF projects to those that are not proximate. The goal of this contrast is to estimate the effect of GEF projects on household assets. In this iterative model fitting procedure, we contrast "close" households to "far" households, with the distance band defining "close" (κ i ) increasing at each step until the maximum step δ is reached. A total of 480 distance thresholds tested, ranging from 0 to 50 km (see Section 3.1). The results of each of these 480 models are displayed in Figure 3, where a points place on the x-axis reflects distance threshold κ i , and the y-axis position represents the estimate θ i . The size of each dot is representative of it's relative match quality, φ i (see Section 2.2.4).
As an example, in the case of i = 41, κ i ≈ 5 km, and θ 41 = 209.41. This can be interpreted as households within 5 km of GEF projects tend to have an income $209.41 higher than those at least 50 km away in Uganda, where 50 km is established by our selection of hyperparameter δ in Section 3.1.

Step 3-Distance Decay of the Observed Treatment Effects
The trend line and concomitant standard errors shown in Figure 3 represent the QGI estimate of the effect of GEF projects on household assets at distances κ i . To fit this spatial model, first for each of our 480 iterations i we drop any cases that did not have sufficient matches to fit a model estimate, or did not have at least a certain threshold of match quality (in this case, a difference of 0.1). We then fit a 3rd-order polynomial trend, following the weighted model detailed in Section 2.2.4. As can be observed in Figure 3, all estimates suggest a positive impact on household assets; however, only for distances after approximately 2 km does statistically significant evidence exist. For these distances, an average effect of approximately $180 is detected.

Discussion and Conclusions
This paper sought to identify the relationship between the siting of GEF Sustainable Forest Management projects and changes in household assets in households proximate to those locations. By applying Quasi-experimental Geospatial Interpolation (QGI), we found robust and consistent evidence that the GEF had a positive impact on local household assets, to the order of approximately 180 USD between 2009 and 2011 (the survey years; Figure 3). While all distances estimated in this study suggested a positive impact, only distances greater than 2 km illustrated statistically significant positive impacts (see Figure 3); this is likely due to the relatively small number of cells proximate to GEF projects (making it challenge to establish a large enough N to detect significance at close distances). Insufficient evidence existed to examine impacts beyond approximately 7 km from GEF project intervention locations. We note evidence for an impact of approximately $180 is markedly consistent for distances between around 3 and 7 km from GEF projects.
A number of novel issues arose during the exploration of QGI. First, we note that previous approaches to Geospatial Impact Evaluation have relied heavily on re-using data as both control and treatment cases-i.e., as Figure 2 illustrates, by changing the distance at which you consider locations to have been effected by a treatment, you can end up classifying the same unit as "treated" in one iteration, and "control" in another. This, by it's very nature, encourages "p-score hacking", or searching through the data to find distance bands that support a particular finding. The proposed approach in this paper precludes this-by defining a maximum distance band, only units of observation that are beyond that distance can be included as potential counterfactual ("control") units. This means that, no matter what distance band is chosen, the same set of controls are eligible for selection (and these same controls will never be treated as "treated" locations).
This approach has a second benefit of illustrating the impact of an intervention as a function of geographic distance. It is not reasonable to assume that an intervention that manifests spatially has no spatial pattern-i.e., a clinic implemented in one village will naturally impact neighboring villages more than those farther away. Past research-even that conducted by the authors (see [48])-has ignored this spatial decay pattern, leading to single estimates for entire regions. QGI provides researchers with an approach to overcome this shortfall.
QGI also offers the advantage of being deterministic-i.e., there is no simulation or random chance associated with the outcome. This not only promotes rapid computation (i.e., solutions can be arrived at in minutes on normal desktop hardware for reasonably sized datasets), but also promotes our ability to replicate findings.

Future Research
The use of quasi-experimental methods within the geographic literature is relatively new-little modeling has coupled, for example, the approaches discussed in this piece with more traditional spatial lag, error, or Durbin models. There are currently not approaches to solve for distance-decay patterns on a local basis-i.e., here we present a single global relationship, rather than estimating the likely decay pattern of any individual intervention. Further, there is an opportunity to integrate recent studies on spatial imprecision in measurements with QGI, or explore alternatives to propensity score matching that incorporate spatial effects. More broadly, there is limited literature on the relative accuracy of "Geospatial Impact Evaluation" as contrasted to more traditional a-spatial quasi-observational designs for causal identification; followup fieldwork could help provide such a contrast. Given the nascent nature of this subfield of study, the "sky is the limit" in terms of questions that can be asked.
Beyond the novel methodological research to be done, the substantive findings in this piece suggest that GEF SFM environmental interventions are linked to positive changes in household assets. However, significant research remains to be done on the specific nature of these linkages, and more broadly linkages between the environment and poverty. Such efforts could be implemented through qualitative surveys following a triangulation framework (see [49]) to validate findings and understand human perceptions on the ground. We further argue that replicating this study with more contemporary data could be of high value.