Careful management of erosion is critical to maintaining aquatic habitat and long-term hydropower potential in the Shire River Basin of Malawi [1
]. As a means to reduce sediment loads to the Shire Basin system, the Government of Malawi encourages adoption of conservation agriculture (CA), a trinity of agricultural practices that, while taking various forms across the globe [2
], generally encourages farmers to avoid tilling soil, to keep soil covered with mulch or cover crops, and to introduce a (usually nitrogen-fixing) intercrop or rotation. In Malawi, CA is written into the country’s Agricultural Sector Wide Approach (ASWAp) as (i) crop residue mulching (CR); (ii) zero tillage (ZT); and (iii) legume intercropping or rotation (IC) [3
]. Several different approaches to financing incentives for CA adoption are under design and evaluation in Malawi, including the UNDP’s Green Water Credit Scheme [4
] and a private trust for promoting CA in riparian micro-catchments developed by the MCC [5
]. The role of CA in reducing sediment loading to surface waters is fairly well characterized in the literature at around 65% [6
], but the calculus of adoption for households (e.g., constraints and net benefits) is much murkier, highlighting the need for context-specific assessments. In the current study we present a method for allowing farmers themselves to provide key criteria in their decisions to adopt or not, and evaluate the importance of these criteria in a large-n
Examination of the factors shaping uptake and diffusion of CA, which is acknowledged across Southern Africa to be generally low overall [8
], is an area of active research [9
]. So far, the findings can be well described as “mixed”; in the words of one research group reviewing this literature, “There are few if any universal variables that regularly explain the adoption of conservation agriculture” [13
]. Giller (2009) notes, for example, that in many contexts, competing uses for crop residues, availability of labor, and access to physical inputs can restrict adoption of CA [14
]. Pannell (2014) notes crop residues as well, pointing additionally to short-term yield gaps and farmers’ general aversion to uncertainty [15
]. Although researchers frequently point to the competing uses for crop residues (e.g., animal fodder) as a constraint to the diffusion of CA [16
], it is also observed that—for Malawi in particular—the low density of grazers makes this less likely to be a constraint [17
]. Summarizing the challenge of understanding and encouraging CA adoption, we suggest that various factors including costs, agronomic knowledge, and production risks in the early stages of CA adoption discourage farmers from sustained adoption of CA, misaligning their actions with the longer-term needs of their communities.
Andersson and D’Souza (2014) note that many of the studies within this literature are conducted in the context of an intervention promoting CA, so their results are largely limited to revealing “general characteristics of CA (component) adopters, rather than revealing farmers’ resource allocation strategies that underpin adoption and non-adoption” [17
]. The present study also examines CA from within the context of an intervention promoting CA, and it also employs tools to compare decision-making across both our treatment and control groups in order to look more closely at the structure of CA adoption decisions and how they are shaped by the presence of incentives. That is, our study design allows us to look at whether incentives re-organize decision-making in a meaningful way, such that different factors matter, which might offer some explanation as to why there is little clear consistency across different contexts in terms of which factors shape adoption.
Specifically, we draw from the ethnographic decision tree modeling (EDTM) work pioneered by Christina Gladwin (in Malawi, largely) in the 1980s and conduct interviews in which key points in respondents’ decision processes (specifically those pertaining to farmers’ decisions to adopt each of the three constituent practices of CA) are identified and articulated, and subsequently applied to construct classification trees whose terminal nodes best classify the order of factors that affect the decision taken by participants [18
]. This technique has enjoyed continued use through recent decades with continued application to agricultural decisions, such as farm planning [19
] and weed management [20
], as well as in other contexts, such as counselling psychology [21
]. By asking farmers directly about how they make decisions, as compared to traditional models where researchers provide potentially relevant variables a priori
as is the case in household surveys, we are better able to capture and characterize the issues that matter most to farmers. Our study contributes and expands this methodology by combining ethnographic methods with machine learning, which can more efficiently determine decision trees and discover complex structures that were not specified in advance [22
]. Bootstrap aggregating (or bagging) is a popular ensemble learning method for statistical classification. Compared with other meta-algorithms for data classification, bagging has been shown to be a stable, accurate, and easy-to-implement algorithm for data classification [23
]. Bagging generates multiple versions of a predictor and then uses model averaging to arrive at an aggregated predictor. In our study, we depart from the typical small-n
approach to building trees by re-applying our identified key decision criteria in a large-n
endline survey. This allows us (i) to make use of machine learning tools (specifically the random forest and bootstrap aggregation algorithms) to identify structure in farmer decision-making; (ii) to examine the explanatory power of farmer-identified factors for the adoption of CA alongside that of other sets of variables commonly solicited in a household survey; and (iii) to compare differences in decision structure across groups who have (treatment) and have not (control) been offered incentives to adopt CA.
This study is structured to inform the question: “How do farmers decide to adopt the specific activities of CA?” Within this broader question, we also wish to understand how decisions to adopt each of the different practices may differ, and how these decisions may differ in the presence of an incentive, with the goal of informing the behavioral response side of potential soil and water conservation programs. Through our novel integration of ethnography and machine learning, we find that more than any other factor, adoption by neighbors (i.e., peer effects) matters, with possible implications for the overall cost of encouraging CA (e.g., through subsidies) as it is taken up across a landscape. Further, we note that little else within our household survey (save for more detailed articulation of neighbor and neighborhood characteristics) offers greater explanatory power than those factors identified by farmers themselves. Finally, we note that decisions made in the presence of an incentive are structurally different than those made without incentives, validating previous concerns in the literature regarding the basis most CA adoption studies, within CA promotion interventions.
2. Materials and Methods
Our primary data collection occurred via two activities within a larger project evaluating the impact of different incentive structures [24
] on the adoption of the three practices of conservation agriculture in the Shire River Basin, Malawi. This larger project includes a random sample of 63 villages selected from the pooled sampling frame of the 5 extension planning areas (EPAs) riparian to the upper Shire River Basin, including Balaka and Machinga districts and a portion of Zomba district (Figure 1
). The work presented in the current paper does not draw on distinctions across treatments, and rather focuses on the aggregate effect of having been offered an incentive or not, but the interested reader is directed to [25
] or to Appendix A
for further details on project design. The two activities whose data we draw on in the current study include a small-n ethnographic exercise (n
= 96 respondents) conducted in a selection of project villages between the end of the intervention and the endline study, as well as a survey module within the project’s endline study (n
= 1923 respondents).
The ethnographic exercise drew a clustered, quota sample of participants from within the larger project’s sample. Specifically, we visited a random selection of 8 of the 12 control villages and 24 of the 48 treatment villages, and in each village identified three participants for interviews: one who was practicing intercropping/rotation (irrespective of whether they also practiced zero tillage or mulching); one who was practicing zero tillage or mulching (irrespective of whether they practiced intercropping/rotation); and one participant who was practicing none of these three CA practices. Following the method of ethnographic decision tree modeling (EDTM) of [18
], our team of interviewers from the National Smallholder Farmers’ Association of Malawi (NASFAM) conducted face-to-face interviews with 96 participants to probe for key factors shaping the decision whether to adopt the three constituent practices of CA. Coding of the interviews yielded 26 unique criteria from across the set of interviews (summarized in our results section as Table 1
). In the traditional EDTM espoused by [18
], these decision criteria would be used to manually construct decision trees that could then be quantitatively analyzed to assess the predictive accuracy of the decision tree. In our analysis, we departed from Gladwin’s method in two distinct ways. First, rather than construct decision trees from this relatively small sample of 96 interviews, we instead leveraged the knowledge gleaned from these interviews to design a household survey module (consisting of a series of binary response items) to be included in the larger project’s endline survey (n
= 1923 respondents). Additionally, for each item we added a follow-up question that asked explicitly whether the respondent believed that factor had shaped their choice to adopt or not any of the three CA practices. Second, rather than constructing the decision trees manually based on the ethnographic interviews, we applied machine learning tools to this large-n dataset (from the endline survey) to identify structure in farmers’ decision-making using decision trees (and forests) as described below.
We used the TreeBagger package in Matlab’s Machine Learning Toolbox to train a “forest” of 100 classification trees to predict farmers’ adoption of each of the three CA practices using data from the EDTM module, separately for both (i) responses from respondents in control villages (hereafter, “control”) and (ii) responses from respondents in treatment villages (hereafter, “treatment”). Each forest is generated using 10-fold cross validation, meaning that the available data are sliced into 10 parts, with a part excluded from a set of trees trained on the remaining nine parts, and repeated 10 times. The trained trees are then validated on the excluded part, generating an estimate of “out of bag” (OOB) prediction error. TreeBagger also generates an estimate of predictor importance, by estimating the relative OOB prediction error when individual predictors are excluded from the trees. To compare the predictive capacity of the EDTM module with that of other survey modules common to household surveys, we incorporated other modules from the endline survey (treatment characteristics, household characteristics, neighborhood characteristics, farm characteristics, and risk perceptions; a list of included items as well as our complete Android ODK protocol is attached as supplementary information
) and repeated the above TreeBagger exercise, giving us estimates of predictor importance as well as average prediction error for forests of trees (and predictors within them) built from pooling the EDTM module with these additional modules.
Following this exercise, we selected “best fit” trees for each case by identifying the tree structure that minimized OOB prediction error, using the fitctree.m and cvloss.m functions in the Machine Learning Toolbox and following Matlab’s “Classification” example [26
]. There are several options for selecting a best fit, with MathWorks suggesting both (i) choosing the tree that gives an absolute minimum in prediction error (without considering tree complexity) and (ii) finding an optimum that identifies the simplest tree structure whose OOB prediction error is within one standard error of the absolute minimum; we select the latter approach for selecting trees.
3.1. Key Decision Criteria and Perceived Relevance
We identified 26 different criteria from our ethnographic interviews that farmers mentioned as shaping their decisions to adopt any of the agricultural practices within CA; these are summarized in Table 1
in the question form through which they were verified in return interviews and included in our endline survey. These criteria span a range of different factors, and we have tagged them (for organization and visualization only) based on our judgments as belonging to social, physical environment, climatic, economic, risk perception, and governance themes. In-depth descriptions of these criteria, along with supporting quotations from ethnographic interviews, are included as Appendix B
We asked respondents in our endline survey about each of the 26 decision criteria (Table 1
), and whether they felt that particular criteria influenced their decisions regarding each of the three CA practices individually. The endline survey encompassed 1923 respondents, in which 1669 (87 percent) respondents reported practicing intercropping, 961 (50 percent) reported practicing crop residue mulching, and 750 (39 percent) reported practicing zero tillage; 535 (28 percent) respondents reported practicing all three. In general, respondents reported that the choice to intercrop was based on economic motivations such as improving yields or reducing the risk of food insecurity (in the case of intercropping) or economic constraints such as the limited availability of inputs (in the case of mulching residues and practicing zero tillage). In addition to these economic factors, the decision(s) to mulch crop residues and practice zero tillage were also influenced by the environmental conditions of their agricultural land (Table 1
Specifically, farmers identified that their decision of whether to intercrop was based on their perception that they would have more crops to consume (661) or to sell (623), indicating that people intercrop because they think it will increase their total output. Similarly, whether there was a market for the crop was an identified decision criterion (540). In addition to market forces, farmers were influenced to do intercropping in order to mitigate risks to yields like crop failure (469), planting delays (173), and concerns of pests whether insect (149) or other animals (118).
Farmers most often explained their choice of whether to retain crop residues in terms of the inputs required like sufficient access to crop residues (831) and sufficient labor to apply them (226). They also explained that they were influenced by the competing benefits and drawbacks of crop residue application. Farmers may be influenced to adopt crop residues because they are concerned with weed growth (251), which can be stifled with crop residue application and of soil moisture (206). However, they also may be discouraged from adopting it since they are concerned with increased insect pests (217), which may increase when residues are transferred across fields.
Farmers indicated that their choice to adopt zero tillage was influenced (either positively or negatively) by the conditions of their plots like if they had soil erosion problems (123), waterlogging issues (111), which are associated with hard pans (164)—a condition of the soil that makes it compact and more difficult for plant roots to establish. Farmers were also influenced by their concern of weed growth (219) and if they could buy herbicide (187) to address it.
In sum, across our ethnographic surveys and in the perceptions reported by respondents in our endline survey, different factors shape decisions for the different CA practices. In particular, farmers link the decision to intercrop most commonly with crop performance and market opportunities, while they link the decisions for zero tillage and crop residue mulching more commonly with physical constraints such as access to residues, labor constraints, and weed issues. Our next analyses draw on machine learning tools to examine whether these same reported factors also help to predict adoption within our sample.
3.2. Random Forest Results
We analyzed “forests” of classification trees trained on the dataset of adoption decisions and the 26 criteria from Table 1
, and found that the variables that explain the most variation in farmers’ choices to adopt were different than what farmers self-reported (Figure 2
and Figure 3
). Additionally, the factors that influenced farmers’ decisions to adopt intercropping were different between control and treatment groups. However, the decision criteria for applying crop residues and zero tillage were similar for each activity and between treatment and control groups.
In ensemble predictions for the control samples, the choice to intercrop was driven by a number of variables, but the top decision criterion was concern about soil erosion (Figure 2
), along with the perception that there would be a market and the ability to sell the leguminous crop, or that they could be consumed by the household. The choice to do intercropping within the treatment group was also driven by market factors, as well as concerns about pests, and the ability to purchase herbicides.
The ensemble predictions for the decision to adopt crop residues were generally most influenced by whether people had been exposed to the practice. In control groups people were most influenced by whether their neighbors did it and if they had access to extension and, to a lesser extent, whether they were concerned with weed growth. The main factor for applying crop residues for the treatment group was the monetary incentive, along with whether they had exposure to it either through extension or if their neighbors did it, and also if crop residues were available to them.
Similar to crop residue application, the decision to do zero tillage, in both control and treatment groups, was driven by whether or not people had exposure to it. In the control group people adopted zero tillage when they had access to extension or if their neighbors did it. To a lesser extent, people in the control group were influenced by whether there was sufficient rain in the previous year. Those in the treatment group were most influenced by whether their neighbors did zero tillage (Figure 3
). To a much lesser extent the treatment group was also influenced by the monetary incentive and whether they had access to residues.
3.3. Expanded Ensemble Predictions
As a means of comparing how well the set of criteria outlined by farmers (in the EDTM module) performed as predictors relative to the kinds of variables commonly identified in standard agricultural household surveys, we trained additional forests using the EDTM module, plus one of the following sets of variables also included in our endline survey: treatment characteristics, household characteristics, neighborhood characteristics, farm characteristics, and risk perceptions (list of survey items included in these modules, along with full survey protocol, is included as supplementary information
). Additionally, we trained one final forest on the combined set of all modules.
Prediction error is reduced in most cases by inclusion of neighborhood characteristics; inclusion of risk variables offers some improvement to explaining crop residue mulching and zero tillage in the treatment group; inclusion of farm characteristics improves prediction of IC in both the treatment and control groups. It is notable that the module with the greatest additional impact on prediction, neighborhood characteristics, is captured coarsely by the EDTM module (“Do any of your neighbors practice …”); and it is possible that more detailed probing of this topic in the interviews may have yielded additional criteria to capture the variability explained by the neighborhood module. Additional points to draw from Figure 4
, reinforcing the message from other results in this study are that (i) the prediction of intercropping and the factors that improve prediction of intercropping are distinct from those which predict crop residue mulching and zero tillage and (ii) the factors that improve prediction in the treatment group appear to be different from those that improve prediction in the control group. Finally, it is noteworthy that in most cases, the reduction in prediction error with additional modules is modest, highlighting that the criteria outlined by farmers themselves were able to explain variability in adoption nearly as well as would the items in a larger household survey.
3.4. Decision Tree Results
Finally, we present individual trees for each of the three practices across treatment and control (Figure 5
), whose structure was selected as optimal (the simplest tree structure whose cross-validation prediction error falls within one standard error of the minimum prediction error across all trees—see Methods). Once again, the decision trees for each of the CA activities are different, which indicates that farmers use different decision criteria when deciding to do a particular CA practice. In addition, the availability of an incentive changes the way farmers structure decisions to adopt CA. In the case of intercropping, for both treatment and control groups, there is no decision “tree,” in large part because rates of intercropping are simply very high across both samples (>80%) and there are no variables that improve upon the simple prediction that a farmer will intercrop.
The choice to apply crop residues is best explained in the dataset by whether or not one’s neighbor does it, regardless of treatment. Within the control group the decision is straightforward: if your neighbors apply cover crops, you likely will too and if they do not, you likely will not. The treatment group is similar in that if your neighbors are doing it, you likely will too, but there are additional factors that explain variability. Where their neighbors are not mulching residues, farmers in treatment villages who have residues and are offered an incentive are likely to adopt as well. For those in treatment villages whose neighbors are practicing zero tillage, but who have not received extension training on zero tillage, a lack of labor or a lack of residues are the key things that keep them from also adopting. Interestingly, the presence of neighbors practicing mulching makes the availability of residues a much lower-order factor than when such role models are absent.
Like the case for intercropping, there is no decision “tree” for farmers to adopt zero tillage in the control group. Most simply do not do zero tillage (>70%); and of those that are doing it, their decision criteria are not held enough in common to structure a tree. The tree for the treatment group is more complex, and like the decision to apply crop residues, is most strongly influenced by neighbors’ practices. Those whose neighbors have not adopted, most likely will also not adopt. Those whose neighbors have adopted and who were offered an incentive most likely will also adopt. Where farmers have neighbors who have adopted and were not offered an incentive are more likely to adopt if the last year’s planting went as planned and if they feel they have available crop residues for residue mulching. Though of minor importance to the zero tillage decision, the presence of crop residues in this tree supports the idea that these two decisions are linked.
As a last note on the importance of neighbors, we note that the importance of neighbors transcends both the availability of an incentive (i.e., neighbors matter in both control and treatment groups) as well as the structure of incentive. The “neighbors CA” variable remains the most important (top node) variable in the optimal trees for adoption of crop residues and zero tillage in the data subsets for both payment structures offered in our main intervention (standard subsidy, and agglomeration payment; see Methods) (additional trees included as supplementary information
). As outlined in the Methods section, the “agglomeration payment” structure offers bonuses to adoptees whose neighbors also adopt, creating a network externality in which neighborhood adoption should matter [26
]. Our analysis in the current paper does not focus on the agglomeration payment directly but only on the broader effect of having been offered an incentive. However, the importance of the neighbor effect across all treatments as well as the control highlights that there are clear peer effects latent in the system that incentives like the agglomeration payment can be designed to harness.
We have presented results from two data collection efforts: (i) a set of ethnographic interviews in which respondents identified a set of key criteria shaping their decisions to adopt the different practices that comprise conservation agriculture (CA), and (ii) a large-scale endline survey in which farmers from both treatment and control villages were asked to report on these same criteria, as well as the role they perceived them to have in their CA decision-making. In our analysis we examined farmers’ own perceptions of which factors shaped their decision-making; which factors better explained variability in adoption of CA within the sample; and how the set of factors identified by farmers themselves compared with other common household survey items in predicting adoption.
Most notably the set of factors explaining adoption of intercropping differs sharply from the factors that explain adoption of zero tillage or crop residue mulching (between which there is much overlap), a finding that is consistent across both the farmers’ perceived factors of importance, as well as the factors identified as important by our random forest approach. This finding supports the findings of [27
], who characterized the adoption of CA not as one decision nor three but closer to two; one regards intercropping and a second regards the mulching of crop residues over (generally) untilled soil. The second finding of interest is that the set of factors emerging as important, as perceived by the farmers, is different from the set of factors identified as important within the decision tree and random forest approaches (i.e., explaining variation in adoption within the dataset).
More specifically, variation in adoption of zero tillage and crop residue mulching is explained by a small number of variables, most notably adoption by neighbors and the presence of incentives. Though adoption by neighbors emerged as perhaps the most important factor in shaping adoption, it was perceived by only a handful of respondents as having shaped their decisions (2–3%; Table 1
). At least in the case of crop residue mulching, early adopters might feel otherwise; one interviewee noted, “People used to laugh at us when we adopted ground cover, but now the same people are coming to me asking for assistance on how they can also do it”.
A unique element of our study is that we examined the structure of decision-making across farmers both inside and outside of an intervention. Our decision tree analysis demonstrated that, while neighbor effects seem to matter both inside and outside the intervention, decisions on CA practices within the treatment may be structured differently. That is, variation in adoption by members of the treatment group could be better explained (in terms of cross-validated errors) by additional factors such as the presence of incentives and access to factors such as crop residues, extension, and labor, while variation in adoption within the control group could not. One implication of the appearance of crop residues in the trees for both zero tillage and crop residue mulching is that access to sufficient crop residues may be a key entry point to shifting adoption of both CA practices.
Implications for Programmatic Soil and Water Conservation
Our analysis shows that, within our sample and timeframe, neighbors and incentives emerge as the key factors shaping adoption. An implication of this is that in the absence of neighbors as role models, incentives are likely necessary to encourage initial adopters in a region. However, the importance of neighbors in our results suggests that once some adoption has been achieved, further adoption may be more likely, rather than less. Additionally, the incentives necessary to encourage further adoption may be lower (rather than higher), as farmers need less of an encouragement to follow in the footsteps of their peers. Though the role of peer effects to encourage adoption and diffusion is nothing new [28
], it is by no means universal, nor are there generally universal factors shaping adoption in CA contexts [13
]. Identifying neighbors’ adoption as a clear driver of new adoption in the Shire Basin context has clear application in the design of new research and development programs in the region aimed at improving livelihoods, landscapes, and water systems. We mean to suggest that, based on our findings in this paper, the CA adoption problem may have a tipping point (beyond which further adoption is self-reinforcing as people observe their neighbors also adopting). This kind of system behavior supports the argument that payments need to be high upfront, but may not need to stay that way—as neighbors adopt, and as the private benefits of CA accrue to the adopter (reduced input costs, improved soil structure yields, etc.), payments can be phased out. In the sustainable land management context, demonstrated reduction in sedimentation could even provide the basis for payments for ecosystem services (PES) program [29
], with a direct beneficiary willing to take over any provision of encouragement. Though adoption of CA is generally low across Southern Africa [8
], we suggest the importance of research that identifies landscapes within the region where CA adoption is comparatively high, in order to better test the potential for tipping point behavior in the addressing the challenge of land degradation.